Merge ~nacc/git-ubuntu:refactor-main-v2 into git-ubuntu:master

Proposed by Nish Aravamudan
Status: Superseded
Proposed branch: ~nacc/git-ubuntu:refactor-main-v2
Merge into: git-ubuntu:master
Diff against target: 8300 lines (+4121/-3559)
17 files modified
bin/git-ubuntu (+1/-1)
bin/import-cron (+1/-1)
bin/snap/snap-git-ubuntu (+1/-0)
gitubuntu/__main__.py (+141/-124)
gitubuntu/build.py (+132/-129)
gitubuntu/buildsource.py (+55/-59)
gitubuntu/clone.py (+107/-105)
gitubuntu/importer.py (+1388/-1176)
gitubuntu/importlocal.py (+312/-260)
gitubuntu/importppa.py (+256/-175)
gitubuntu/lint.py (+548/-509)
gitubuntu/merge.py (+423/-355)
gitubuntu/queue.py (+299/-281)
gitubuntu/remote.py (+120/-97)
gitubuntu/source_information.py (+15/-0)
gitubuntu/submit.py (+190/-174)
gitubuntu/tag.py (+132/-113)
Reviewer Review Type Date Requested Status
Server Team CI bot continuous-integration Needs Fixing
Andreas Hasenack Pending
Robie Basak Pending
Review via email: mp+330321@code.launchpad.net

This proposal has been superseded by a proposal from 2017-09-08.

Description of the change

This is admittedly a messy patchset. I am tired of rebasing it, though, and think we can land it in roughly this state (even if only in the edge channel of the snap) and then decide what to do from there after testing.

A ton of change, but we are now pylint3 clean for errors except for pygit2 imports:

$ pylint3 -E gitubuntu --ignored-modules=pygit2
No config file found, using default configuration

All unit tests still pass and I've run a from-scratch import, a clone and a build using the resulting git repository.

To post a comment you must log in.
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

FAILED: Continuous integration, rev:bd7e040556cd86c24859dea6af7d56fbb92954ba
https://jenkins.ubuntu.com/server/job/git-ubuntu-ci/24/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Tests
    FAILED: Build

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/git-ubuntu-ci/24/rebuild

review: Needs Fixing (continuous-integration)
~nacc/git-ubuntu:refactor-main-v2 updated
48775be... by Nish Aravamudan

lint: drop unused pullfile variable

Revision history for this message
Server Team CI bot (server-team-bot) wrote :

FAILED: Continuous integration, rev:a54e9ee31944d9af5820b98bf66e12329408c788
https://jenkins.ubuntu.com/server/job/git-ubuntu-ci/25/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Tests
    FAILED: Build

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/git-ubuntu-ci/25/rebuild

review: Needs Fixing (continuous-integration)
~nacc/git-ubuntu:refactor-main-v2 updated
a3a9190... by Nish Aravamudan

clone: fix stray space

29e6277... by Nish Aravamudan

lint: linting historical merges with default arguments can fail

Typically, this is because pkg/debian/sid now is not the same as
pkg/debian/sid when the MP was proposed. This can lead to all sorts of
oddness, but should not be fatal to lint (and the proper way to lint a
historical change is presumably passing a --target-branch, or changing
that functionality to be --target-commitish).

One specific case is when the to-be-linted branch is older than the
target branch (e.g., a missed MP) and thus it only contains changelog
deletions.

7af2b70... by Nish Aravamudan

git-ubuntu: change default fetch protocol to https

LP: #1715688

6a15a64... by Nish Aravamudan

git-ubuntu: fix dictionary style

19aa857... by Nish Aravamudan

use environment python3

Rather than a hardcoded path.

Revision history for this message
Nish Aravamudan (nacc) wrote :

The overall structure of this is good, I'm just still fine-tuning.

~nacc/git-ubuntu:refactor-main-v2 updated
49b3a96... by Nish Aravamudan

snap: export QUILT_DIR in git-ubuntu wrapper

Reported by powers and rbasak, quilt is looking in a hardcoded
/usr/share/quilt for subcommands. This is in our snap, not in the
root filesystem.

3b2b418... by Nish Aravamudan

lint: drop unused pullfile variable

e269f5a... by Nish Aravamudan

importppa: fix undefined variable UbuntuSourceInformation

87abbad... by Nish Aravamudan

importppa: fix call to add_remote

57c917e... by Nish Aravamudan

importppa: save pkgname to object

This is used by the importer code

bfba350... by Nish Aravamudan

importppa: fix parameters to parse_parentfile

Unmerged commits

6a15a64... by Nish Aravamudan

git-ubuntu: fix dictionary style

48775be... by Nish Aravamudan

lint: drop unused pullfile variable

6b2fe22... by Nish Aravamudan

git ubuntu: refactor main to call module parse_args

This is the final step in the refactor -- rather than using per-module
classes, just import each module in __main__ in order to call its
parse_args and func (which in the current implementation is that
module's cli_main).

Before merging with master, I think we will squash all of these
together.

At this point, all errors from pylint3 are from pygit2 imports.

LP: #1707321

2d4263a... by Nish Aravamudan

submit: refactor class

Leave only parse_args and cli_main in the class. Make cli_main nothing
more than a wrapper to the module's main function.

LP: #1707321

ce50ef5... by Nish Aravamudan

submit: use lint.derive_target_branch

submit can be used for distributed development among team members by
proposing MPs against each other's repositories, via the --target-user
flag. Rather than having submit-specific code for deriving the target
branch, use the existing code from lint, after extending it to take a
namespace argument.

b6762e7... by Nish Aravamudan

tag: refactor class

Leave only parse_args and cli_main in the class. Make cli_main nothing
more than a wrapper to the module's main function.

LP: #1707321

2bb9d17... by Nish Aravamudan

remote: refactor class

Leave only parse_args and cli_main in the class. Make cli_main nothing
more than a wrapper to the module's main function.

LP: #1707321

8359506... by Nish Aravamudan

queue: refactor class

Leave only parse_args and cli_main in the class. Make cli_main nothing
more than a wrapper to the module's main function.

LP: #1707321

92d78b6... by Nish Aravamudan

merge: refactor class

Leave only parse_args and cli_main in the class. Make cli_main nothing
more than a wrapper to the module's main function.

LP: #1707321

f28dca4... by Nish Aravamudan

lint: refactor class

Leave only parse_args and cli_main in the class. Make cli_main nothing
more than a wrapper to the module's main function.

LP: #1707321

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1diff --git a/bin/git-ubuntu b/bin/git-ubuntu
2index 23f908e..2b15471 100755
3--- a/bin/git-ubuntu
4+++ b/bin/git-ubuntu
5@@ -1,4 +1,4 @@
6-#!/usr/bin/python3
7+#!/usr/bin/env python3
8
9 # PYTHON_ARGCOMPLETE_OK
10
11diff --git a/bin/import-cron b/bin/import-cron
12index b23e512..0044b74 100755
13--- a/bin/import-cron
14+++ b/bin/import-cron
15@@ -1,4 +1,4 @@
16-#!/usr/bin/python3
17+#!/usr/bin/env python3
18
19 from copy import copy
20 import os
21diff --git a/bin/snap/snap-git-ubuntu b/bin/snap/snap-git-ubuntu
22index 17ba583..9a73451 100755
23--- a/bin/snap/snap-git-ubuntu
24+++ b/bin/snap/snap-git-ubuntu
25@@ -6,5 +6,6 @@ export PATH="$SNAP/bin/snap:$SNAP/usr/local/sbin:$SNAP/usr/local/bin:$SNAP/usr/s
26 export LD_LIBRARY_PATH="$SNAP/lib/x86_64-linux-gnu:$SNAP/usr/lib/x86_64-linux-gnu"
27 export PERL5LIB="$SNAP/usr/share/perl5"
28 export DPKG_DATADIR="$SNAP/usr/share/dpkg"
29+export QUILT_DIR="$SNAP/usr/share/quilt"
30 eval "$(register-python-argcomplete3 git-ubuntu)"
31 exec "$SNAP/usr/bin/python3" "$SNAP/bin/git-ubuntu" "$@"
32diff --git a/gitubuntu/__main__.py b/gitubuntu/__main__.py
33index ba4a435..49373a7 100644
34--- a/gitubuntu/__main__.py
35+++ b/gitubuntu/__main__.py
36@@ -1,26 +1,15 @@
37-#!/usr/bin/python3
38+#!/usr/bin/env python3
39
40 def main():
41 try:
42 import argparse
43+ from importlib import import_module
44 import logging
45 from os import isatty
46 import shutil
47 from subprocess import CalledProcessError
48 import sys
49 import textwrap
50- from gitubuntu.importer import GitUbuntuImport
51- from gitubuntu.build import GitUbuntuBuild
52- from gitubuntu.buildsource import GitUbuntuBuildSource
53- from gitubuntu.merge import GitUbuntuMerge
54- from gitubuntu.clone import GitUbuntuClone
55- from gitubuntu.tag import GitUbuntuTag
56- from gitubuntu.queue import GitUbuntuQueue
57- from gitubuntu.importlocal import GitUbuntuImportLocal
58- from gitubuntu.importppa import GitUbuntuImportPPA
59- from gitubuntu.remote import GitUbuntuRemote
60- from gitubuntu.submit import GitUbuntuSubmit
61- from gitubuntu.lint import GitUbuntuLint
62 from gitubuntu.run import run
63
64 try:
65@@ -34,123 +23,147 @@ def main():
66
67 logging.getLogger('keyring').setLevel(logging.WARNING)
68
69- known_subcommands = {'import':GitUbuntuImport,
70- 'build':GitUbuntuBuild,
71- 'build-source':GitUbuntuBuildSource,
72- 'merge':GitUbuntuMerge,
73- 'clone':GitUbuntuClone,
74- 'tag':GitUbuntuTag,
75- 'queue':GitUbuntuQueue,
76- 'import-local':GitUbuntuImportLocal,
77- 'import-ppa':GitUbuntuImportPPA,
78- 'remote':GitUbuntuRemote,
79- 'submit':GitUbuntuSubmit,
80- 'lint':GitUbuntuLint,
81- }
82+ known_subcommands = {
83+ 'import': 'gitubuntu.importer',
84+ 'build': 'gitubuntu.build',
85+ 'build-source': 'gitubuntu.buildsource',
86+ 'merge': 'gitubuntu.merge',
87+ 'clone': 'gitubuntu.clone',
88+ 'tag': 'gitubuntu.tag',
89+ 'queue': 'gitubuntu.queue',
90+ 'import-local': 'gitubuntu.importlocal',
91+ 'import-ppa': 'gitubuntu.importppa',
92+ 'remote': 'gitubuntu.remote',
93+ 'submit': 'gitubuntu.submit',
94+ 'lint': 'gitubuntu.lint',
95+ }
96
97- known_network_subcommands = {'import', 'import-local',
98- 'import-ppa', 'clone', 'build',
99- 'build-source', 'queue',
100- 'remote', 'submit'}
101+ known_network_subcommands = {
102+ 'import',
103+ 'import-local',
104+ 'import-ppa',
105+ 'clone',
106+ 'build',
107+ 'build-source',
108+ 'queue',
109+ 'remote',
110+ 'submit',
111+ }
112
113 parser = argparse.ArgumentParser(
114 description='Ubuntu git development tool',
115 formatter_class=argparse.RawTextHelpFormatter,
116- epilog='More information is available at https://wiki.ubuntu.com/UbuntuDevelopment/Merging/GitWorkflow.'
117- )
118- subparsers = parser.add_subparsers(dest='subcommand',
119- help='',
120- metavar='%s' % '|'.join(sorted(known_subcommands))
121- )
122+ epilog='More information is available at https://wiki.ubuntu.com/UbuntuDevelopment/Merging/GitWorkflow.',
123+ )
124+ subparsers = parser.add_subparsers(
125+ dest='subcommand',
126+ help='',
127+ metavar='%s' % '|'.join(sorted(known_subcommands)),
128+ )
129 # This help ends up not being very useful to the end user
130 # subparsers.required = True
131
132 # common flags to all subcommands
133 base_subparser = argparse.ArgumentParser(add_help=False)
134- base_subparser.add_argument('-v', '--verbose', action='store_true',
135- help='Increase verbosity')
136+ base_subparser.add_argument(
137+ '-v', '--verbose',
138+ action='store_true',
139+ help='Increase verbosity'
140+ )
141 network_base_subparser = argparse.ArgumentParser(add_help=False)
142 network_base_subparser.add_argument(
143- '--retries', type=int,
144- help='Number of times to attempt to retry '
145- 'downloading from Launchpad.',
146- default=3
147- )
148+ '--retries',
149+ type=int,
150+ help='Number of times to attempt to retry '
151+ 'downloading from Launchpad.',
152+ default=3,
153+ )
154 network_base_subparser.add_argument(
155- '--retry-backoffs',
156- type=lambda s : [int(item) for item in s.split(',')],
157- help='Comma-separated list of backoffs in '
158- 'seconds to use between each retry '
159- 'attempt. Default is exponential backoff.',
160- default=argparse.SUPPRESS
161- )
162+ '--retry-backoffs',
163+ type=lambda s : [int(item) for item in s.split(',')],
164+ help='Comma-separated list of backoffs in '
165+ 'seconds to use between each retry '
166+ 'attempt. Default is exponential backoff.',
167+ default=argparse.SUPPRESS,
168+ )
169 known_protos = ['git', 'http', 'https']
170 network_base_subparser.add_argument(
171- '--proto',
172- metavar='[%s]' % '|'.join(known_protos),
173- choices=known_protos,
174- help='Specify protocol to use for fetch. Default: %(default)s',
175- default='git'
176- )
177+ '--proto',
178+ metavar='[%s]' % '|'.join(known_protos),
179+ choices=known_protos,
180+ help='Specify protocol to use for fetch. Default: %(default)s',
181+ default='https',
182+ )
183
184 width, _ = shutil.get_terminal_size()
185 subhelp_width = width - 30
186 for sub in sorted(known_subcommands):
187- s = known_subcommands[sub]()
188+ m = import_module(known_subcommands[sub])
189 if sub in known_network_subcommands:
190- help_text = s.parse_args(subparsers, [base_subparser, network_base_subparser])
191+ help_text = m.parse_args(
192+ subparsers,
193+ [base_subparser, network_base_subparser],
194+ )
195 else:
196- help_text = s.parse_args(subparsers, [base_subparser])
197+ help_text = m.parse_args(subparsers, [base_subparser])
198 subparsers.help += '\n'
199- subparsers.help += '\n'.join(textwrap.wrap(help_text,
200- subhelp_width,
201- break_on_hyphens=False
202- )
203- )
204- help_text = '\n'.join(textwrap.wrap('File specifying '
205- 'parent-child relationship '
206- 'overrides as:',
207- subhelp_width
208- )
209- )
210+ subparsers.help += '\n'.join(
211+ textwrap.wrap(
212+ help_text,
213+ subhelp_width,
214+ break_on_hyphens=False,
215+ )
216+ )
217+ help_text = '\n'.join(
218+ textwrap.wrap(
219+ "File specifying parent-child relationship overrides as:",
220+ subhelp_width,
221+ )
222+ )
223 help_text += '\n'
224 help_text += '<pkgname> <child version> <publish parent version> <changelog parent version>\n'
225- help_text += '\n'.join(textwrap.wrap('with one override per '
226- 'line.',
227- subhelp_width
228- )
229- )
230+ help_text += '\n'.join(
231+ textwrap.wrap(
232+ "with one override per line.",
233+ subhelp_width,
234+ )
235+ )
236 help_text += '\ndefault=%(default)s'
237- parser.add_argument('-P', '--parentfile', type=str,
238- help=argparse.SUPPRESS,
239- default=pkg_resources.resource_filename(
240- 'gitubuntu',
241- 'parent_overrides.txt'
242- )
243- )
244- help_text = '\n'.join(textwrap.wrap('File specifying source '
245- 'files to use instead of '
246- 'Launchpad to work around '
247- 'source file shadowing: ',
248- subhelp_width
249- )
250- )
251+ parser.add_argument(
252+ '-P', '--parentfile',
253+ type=str,
254+ help=argparse.SUPPRESS,
255+ default=pkg_resources.resource_filename(
256+ 'gitubuntu',
257+ 'parent_overrides.txt',
258+ ),
259+ )
260+ help_text = '\n'.join(
261+ textwrap.wrap(
262+ "File specifying source files to use instead of "
263+ "Launchpad to work around source file shadowing: ",
264+ subhelp_width,
265+ )
266+ )
267 help_text += '\n'
268 help_text += '<pkgname> <version> <URL of DSC file> <URLs of orig files>\n'
269- help_text += '\n'.join(textwrap.wrap('with one package per line. '
270- '\'-\' for the DSC file '
271- 'indicates to use the one '
272- 'from Launchpad.'
273- )
274- )
275+ help_text += '\n'.join(
276+ textwrap.wrap(
277+ "with one package per line. '-' for the DSC file "
278+ "indicates to use the one from Launchpad.",
279+ subhelp_width,
280+ )
281+ )
282 help_text += '\ndefault=%(default)s'
283- parser.add_argument('-L', '--pullfile', type=str,
284- help=argparse.SUPPRESS,
285- default=pkg_resources.resource_filename(
286- 'gitubuntu',
287- 'pull_overrides.txt'
288- )
289- )
290+ parser.add_argument(
291+ '-L', '--pullfile',
292+ type=str,
293+ help=argparse.SUPPRESS,
294+ default=pkg_resources.resource_filename(
295+ 'gitubuntu',
296+ 'pull_overrides.txt',
297+ ),
298+ )
299
300 argcomplete.autocomplete(parser)
301 args = parser.parse_args()
302@@ -161,22 +174,22 @@ def main():
303 # fragile, assumes developers will pass the base_subparser above
304 # should make this structural in the classes
305 if args.verbose:
306- logging.basicConfig(level=logging.DEBUG,
307- format='%(asctime)s - %(levelname)s:%(message)s',
308- datefmt='%m/%d/%Y %H:%M:%S',
309- )
310+ level=logging.DEBUG
311 else:
312- logging.basicConfig(level=logging.INFO,
313- format='%(asctime)s - %(levelname)s:%(message)s',
314- datefmt='%m/%d/%Y %H:%M:%S',
315- )
316+ level=logging.INFO
317+ logging.basicConfig(
318+ level=level,
319+ format='%(asctime)s - %(levelname)s:%(message)s',
320+ datefmt='%m/%d/%Y %H:%M:%S',
321+ )
322 if args.subcommand in known_network_subcommands:
323 try:
324 retry_backoffs = args.retry_backoffs
325 if len(retry_backoffs) != args.retries:
326- logging.error('Number of backoffs specified in '
327- '--retry-backoffs must match --retries.'
328- )
329+ logging.error(
330+ "Number of backoffs specified in "
331+ "--retry-backoffs must match --retries."
332+ )
333 sys.exit(1)
334 except AttributeError:
335 args.retry_backoffs = [2 ** i for i in range(args.retries)]
336@@ -188,26 +201,30 @@ def main():
337 )
338 except CalledProcessError:
339 if isatty(sys.stdin.fileno()):
340- user = input("gitubuntu.lpuser is not set. What is your "
341+ user = input(
342+ "gitubuntu.lpuser is not set. What is your "
343 "Launchpad username? We will set this in your "
344 "personally global git configuration for you. "
345 "To abort, press Enter.\nUsername: "
346 )
347 if len(user) == 0:
348- logging.error('git-ubuntu requires a launchpad user')
349+ logging.error("git-ubuntu requires a launchpad user")
350 sys.exit(1)
351 else:
352- logging.warning('Setting the Launchpad user for '
353- 'git-ubuntu to %s with '
354- '`git config --global gitubuntu.lpuser %s`.',
355- user, user)
356+ logging.warning(
357+ "Setting the Launchpad user for git-ubuntu to %s "
358+ "with `git config --global gitubuntu.lpuser %s`.",
359+ user,
360+ user,
361+ )
362 run(['git', 'config', '--global', 'gitubuntu.lpuser', user])
363 else:
364- logging.error('Not connected to a TTY and no '
365- 'git-ubuntu lpuser defined. Please run '
366- '`git config --global gitubuntu.lpuser '
367- '<Launchpad username>` and re-run this '
368- 'command.')
369+ logging.error(
370+ "Not connected to a TTY and no git-ubuntu lpuser "
371+ "defined. Please run `git config --global "
372+ "gitubuntu.lpuser <Launchpad username>` and re-run "
373+ "this command."
374+ )
375 sys.exit(1)
376
377 args.func(args)
378diff --git a/gitubuntu/build.py b/gitubuntu/build.py
379index 8143995..d3d292e 100644
380--- a/gitubuntu/build.py
381+++ b/gitubuntu/build.py
382@@ -38,12 +38,11 @@ from gitubuntu.run import run, runq
383 from gitubuntu.source_information import (
384 GitUbuntuSourceInformation,
385 NoPublicationHistoryException,
386+ derive_source_from_series,
387 )
388 try:
389 pkg = 'python3-debian'
390 from debian.debfile import PART_EXTS
391- pkg = 'python3-distro-info'
392- from distro_info import DebianDistroInfo, UbuntuDistroInfo
393 pkg = 'python3-pytest'
394 import pytest
395 except ImportError:
396@@ -228,8 +227,7 @@ def fetch_orig_from_cache(changelog, source, dl_cache=None):
397 return None
398 if dsc.verify() is None:
399 logging.warn(
400- "Cache dir found, but verification of orig tarball %s failed.",
401- orig,
402+ "Cache dir found, but verification of orig tarball(s) failed."
403 )
404 return None # XXX decide - see docstring
405 _symlink_paths_into_parent_dir(dsc.all_tarball_paths)
406@@ -320,8 +318,8 @@ def fetch_orig_from_launchpad(changelog, source, pullfile, retries,
407 except NoPublicationHistoryException:
408 logging.warning(
409 "No publication history found for %s in %s. ",
410- pkgname,
411- dist_name,
412+ changelog.srcpkg,
413+ source,
414 )
415 return None
416
417@@ -374,151 +372,156 @@ def fetch_orig_from_launchpad(changelog, source, pullfile, retries,
418 return None
419
420
421-class GitUbuntuBuild:
422- def parse_args(self, subparsers=None, base_subparsers=None):
423- kwargs = dict(
424- description='Build a git-ubuntu-cloned tree with dpkg-buildpackage',
425- formatter_class=argparse.ArgumentDefaultsHelpFormatter,
426- usage='%(prog)s [options] -- ...',
427- epilog='Any arguments after -- are passed to dpkg-buildpackage.'
428- )
429- if base_subparsers:
430- kwargs['parents'] = base_subparsers
431- if subparsers:
432- parser = subparsers.add_parser('build', **kwargs)
433- parser.set_defaults(func=self.main)
434- else:
435- parser = argparse.ArgumentParser(**kwargs)
436- parser.add_argument('--dl-cache', type=str,
437- help='Cache directory for downloads.',
438- default=os.path.join(os.getcwd(), '.git', CACHE_PATH),
439- )
440- parser.add_argument(
441- '--for-merge',
442- action='store_true',
443- help="Build an Ubuntu merge. Implies searching Debian for "
444- "published versions first.",
445+def check_repository():
446+ try:
447+ cp = run(['git', 'status', '--porcelain'])
448+ except CalledProcessError:
449+ logging.error('Is the current directory a git repository?')
450+ sys.exit(1)
451+
452+ if len(cp.stdout) > 0:
453+ logging.warning(
454+ "Working tree is not clean. `git status` output follows."
455 )
456- parser.add_argument(
457- '--no-pristine-tar',
458- action='store_true',
459- help="Do not use pristine-tar branches to generate orig "
460- "tarballs. This can be necessary as we stabilize the pristine-tar "
461- "data generation, particularly if multiple component tarballs are "
462- "present.",
463+ run(['git', 'status'], stdout=None)
464+ sys.exit(1)
465
466+ if not os.path.isfile('debian/changelog'):
467+ logging.error(
468+ "No debian/changelog found, is this a source "
469+ "package repository? See `git ubuntu clone`."
470 )
471- parser.add_argument('rem_args', help=argparse.SUPPRESS,
472- nargs=argparse.REMAINDER)
473- if not subparsers:
474- return parser.parse_args()
475- return 'build - %s' % kwargs['description']
476+ sys.exit(1)
477
478- def check_repository(self):
479- try:
480- cp = run(['git', 'status', '--porcelain'])
481- except CalledProcessError:
482- logging.error('Is the current directory a git repository?')
483- sys.exit(1)
484
485- if len(cp.stdout) > 0:
486- logging.warning(
487- "Working tree is not clean. `git status` output follows."
488- )
489- run(['git', 'status'], stdout=None)
490- sys.exit(1)
491+def main(search_list, changelog, rem_args):
492+ """main entry point for build
493
494- if not os.path.isfile('debian/changelog'):
495- logging.error(
496- "No debian/changelog found, is this a source "
497- "package repository? See `git ubuntu clone`."
498- )
499- sys.exit(1)
500+ @param search_list: list of OrigSearchListEntry namedtuples
501+ @param changelog: gitubuntu.git_repository.Changelog object for source package to build
502+ @param rem_args: namespace object of remaining arguments to pass onto dpkg-buildpackage
503+ """
504+ check_repository()
505+
506+ fetch_orig_and_build(search_list, changelog, rem_args)
507+
508+
509+def parse_args(subparsers=None, base_subparsers=None):
510+ kwargs = dict(
511+ description='Build a git-ubuntu-cloned tree with dpkg-buildpackage',
512+ formatter_class=argparse.ArgumentDefaultsHelpFormatter,
513+ usage='%(prog)s [options] -- ...',
514+ epilog='Any arguments after -- are passed to dpkg-buildpackage.'
515+ )
516+ if base_subparsers:
517+ kwargs['parents'] = base_subparsers
518+ if subparsers:
519+ parser = subparsers.add_parser('build', **kwargs)
520+ parser.set_defaults(func=cli_main)
521+ else:
522+ parser = argparse.ArgumentParser(**kwargs)
523+ parser.add_argument('--dl-cache', type=str,
524+ help='Cache directory for downloads.',
525+ default=os.path.join(os.getcwd(), '.git', CACHE_PATH),
526+ )
527+ parser.add_argument(
528+ '--for-merge',
529+ action='store_true',
530+ help="Build an Ubuntu merge. Implies searching Debian for "
531+ "published versions first.",
532+ )
533+ parser.add_argument(
534+ '--no-pristine-tar',
535+ action='store_true',
536+ help="Do not use pristine-tar branches to generate orig "
537+ "tarballs. This can be necessary as we stabilize the pristine-tar "
538+ "data generation, particularly if multiple component tarballs are "
539+ "present.",
540
541- def derive_orig_search_list_from_args(self, args):
542- source = 'debian' if args.for_merge else 'ubuntu'
543- orig_search_list = [
544- OrigSearchListEntry(
545- mechanism=fetch_orig_from_parent_dir,
546- source=None,
547- must_build=True,
548- ),
549- OrigSearchListEntry(
550- mechanism=fetch_orig_from_cache,
551- source=source,
552- must_build=True,
553- ),
554- ]
555- if not args.no_pristine_tar:
556- orig_search_list.append(
557- OrigSearchListEntry(
558- mechanism=functools.partial(fetch_orig_from_pristine_tar,
559- repo=self.local_repo,
560- ),
561- source=source,
562- must_build=True,
563- )
564- )
565- orig_search_list.extend([
566+ )
567+ parser.add_argument('rem_args', help=argparse.SUPPRESS,
568+ nargs=argparse.REMAINDER)
569+ if not subparsers:
570+ return parser.parse_args()
571+ return 'build - %s' % kwargs['description']
572+
573+
574+def derive_orig_search_list_from_args(args):
575+ source = 'debian' if args.for_merge else 'ubuntu'
576+ orig_search_list = [
577+ OrigSearchListEntry(
578+ mechanism=fetch_orig_from_parent_dir,
579+ source=None,
580+ must_build=True,
581+ ),
582+ OrigSearchListEntry(
583+ mechanism=fetch_orig_from_cache,
584+ source=source,
585+ must_build=True,
586+ ),
587+ ]
588+ if not args.no_pristine_tar:
589+ orig_search_list.append(
590 OrigSearchListEntry(
591- mechanism=functools.partial(fetch_orig_from_launchpad,
592- pullfile=args.pullfile,
593- retries=args.retries,
594- retry_backoffs=args.retry_backoffs,
595- dl_cache=args.dl_cache,
596+ mechanism=functools.partial(fetch_orig_from_pristine_tar,
597+ repo=GitUbuntuRepository('.'),
598 ),
599 source=source,
600 must_build=True,
601+ )
602+ )
603+ orig_search_list.extend([
604+ OrigSearchListEntry(
605+ mechanism=functools.partial(fetch_orig_from_launchpad,
606+ pullfile=args.pullfile,
607+ retries=args.retries,
608+ retry_backoffs=args.retry_backoffs,
609+ dl_cache=args.dl_cache,
610 ),
611- OrigSearchListEntry(
612- mechanism=fetch_orig_noop,
613- source=[],
614- must_build=True,
615- ),
616- ])
617+ source=source,
618+ must_build=True,
619+ ),
620+ OrigSearchListEntry(
621+ mechanism=fetch_orig_noop,
622+ source=[],
623+ must_build=True,
624+ ),
625+ ])
626
627- return orig_search_list
628+ return orig_search_list
629
630
631- def main(self, args):
632- rem_args = args.rem_args
633- if len(rem_args) > 0:
634- rem_args = rem_args[1:]
635+def cli_main(args):
636+ rem_args = args.rem_args
637+ if len(rem_args) > 0:
638+ rem_args = rem_args[1:]
639
640- self.check_repository()
641- self.local_repo = GitUbuntuRepository('.')
642+ changelog = Changelog.from_path('debian/changelog')
643
644- changelog = Changelog.from_path('debian/changelog')
645+ try:
646+ native = is_native_package(changelog)
647+ except NativenessMismatchError as e:
648+ logging.error("%s" % e)
649+ sys.exit(1)
650
651- try:
652- native = is_native_package(changelog)
653- except NativenessMismatchError as e:
654- logging.error("%s" % e)
655- sys.exit(1)
656-
657- if native:
658- # No orig tarball required
659- orig_search_list = [
660- OrigSearchListEntry(
661- mechanism=fetch_orig_noop,
662- source=[],
663- must_build=True,
664- ),
665- ]
666- else:
667- orig_search_list = self.derive_orig_search_list_from_args(args)
668+ if native:
669+ # No orig tarball required
670+ orig_search_list = [
671+ OrigSearchListEntry(
672+ mechanism=fetch_orig_noop,
673+ source=[],
674+ must_build=True,
675+ ),
676+ ]
677+ else:
678+ orig_search_list = derive_orig_search_list_from_args(args)
679
680- # See http://pad.ubuntu.com/KKB1kMR0JH for logic
681- fetch_orig_and_build(orig_search_list, changelog, rem_args)
682+ # See http://pad.ubuntu.com/KKB1kMR0JH for logic
683+ main(orig_search_list, changelog, rem_args)
684
685
686 def derive_source_from_changelog(changelog):
687- series = changelog.distribution
688- if DebianDistroInfo().valid(codename=series):
689- return 'debian'
690- if UbuntuDistroInfo().valid(codename=series):
691- return 'ubuntu'
692- raise ValueError("Unable to determine distribution from %s" % series)
693+ return derive_source_from_series(changelog.distribution)
694
695 def expand_changelog_source_aliases(orig_search_list, changelog):
696 """Replace 'changelog' sources by reading debian/changelog
697diff --git a/gitubuntu/buildsource.py b/gitubuntu/buildsource.py
698index 4e9b69b..1735129 100644
699--- a/gitubuntu/buildsource.py
700+++ b/gitubuntu/buildsource.py
701@@ -5,66 +5,62 @@ import argparse
702 import logging
703 import os
704 import sys
705-from gitubuntu.build import GitUbuntuBuild
706+from gitubuntu.build import cli_main as build_cli_main
707 from gitubuntu.cache import CACHE_PATH
708
709-class GitUbuntuBuildSource(GitUbuntuBuild):
710- def __init__(self):
711- super().__init__()
712+def parse_args(subparsers=None, base_subparsers=None):
713+ kwargs = dict(
714+ description="Build a source package and changes file",
715+ formatter_class=argparse.ArgumentDefaultsHelpFormatter
716+ )
717+ if base_subparsers:
718+ kwargs['parents'] = base_subparsers
719+ if subparsers:
720+ parser = subparsers.add_parser('build-source', **kwargs)
721+ help_text = "build-source - Build a source package"
722+ parser.set_defaults(func=cli_main)
723+ else:
724+ parser = argparse.ArgumentParser(**kwargs)
725+ parser.add_argument('--dl-cache', type=str,
726+ help=("Cache directory for downloads."),
727+ default=os.path.join(os.getcwd(), '.git', CACHE_PATH),
728+ )
729+ parser.add_argument('--sign', action='store_true',
730+ help="Sign the source package.",
731+ )
732+ parser.add_argument('--for-merge', action='store_true',
733+ help=("Build an Ubuntu merge. Implies searching Debian for "
734+ "published versions first and passing -sa to "
735+ "dpkg-buildpackage."
736+ ),
737+ )
738+ parser.add_argument(
739+ '--no-pristine-tar',
740+ action='store_true',
741+ help="Do not use pristine-tar branches to generate orig "
742+ "tarballs. This can be necessary as we stabilize the pristine-tar "
743+ "data generation, particularly if multiple component tarballs are "
744+ "present.",
745
746- def parse_args(self, subparsers=None, base_subparsers=None):
747- kwargs = dict(
748- description="Build a source package and changes file",
749- formatter_class=argparse.ArgumentDefaultsHelpFormatter
750- )
751- if base_subparsers:
752- kwargs['parents'] = base_subparsers
753- if subparsers:
754- parser = subparsers.add_parser('build-source', **kwargs)
755- help_text = "build-source - Build a source package"
756- parser.set_defaults(func=self.main)
757- else:
758- parser = argparse.ArgumentParser(**kwargs)
759- parser.add_argument('--dl-cache', type=str,
760- help=("Cache directory for downloads."),
761- default=os.path.join(os.getcwd(), '.git', CACHE_PATH),
762- )
763- parser.add_argument('--sign', action='store_true',
764- help="Sign the source package.",
765- )
766- parser.add_argument('--for-merge', action='store_true',
767- help=("Build an Ubuntu merge. Implies searching Debian for "
768- "published versions first and passing -sa to "
769- "dpkg-buildpackage."
770- ),
771- )
772- parser.add_argument(
773- '--no-pristine-tar',
774- action='store_true',
775- help="Do not use pristine-tar branches to generate orig "
776- "tarballs. This can be necessary as we stabilize the pristine-tar "
777- "data generation, particularly if multiple component tarballs are "
778- "present.",
779+ )
780+ parser.add_argument('rem_args', help=argparse.SUPPRESS,
781+ nargs=argparse.REMAINDER
782+ )
783+ if not subparsers:
784+ return parser.parse_args()
785+ return "build-source - %s" % kwargs['description']
786
787- )
788- parser.add_argument('rem_args', help=argparse.SUPPRESS,
789- nargs=argparse.REMAINDER
790- )
791- if not subparsers:
792- return parser.parse_args()
793- return "build-source - %s" % kwargs['description']
794-
795- def main(self, args):
796- if len(args.rem_args) != 0:
797- logging.warning('Passing specified flags (%s) on to '
798- 'dpkg-buildpackage. Did you mean to '
799- 'call `git ubuntu build` instead?' %
800- ' '.join(args.rem_args)
801- )
802- else:
803- args.rem_args = ['--', '-S', '-nc', '-d', '-i', '-I']
804- if args.for_merge:
805- args.rem_args += ['-sa']
806- if not args.sign:
807- args.rem_args += ['-us', '-uc']
808- super().main(args)
809+def cli_main(args):
810+ if len(args.rem_args) != 0:
811+ logging.warning('Passing specified flags (%s) on to '
812+ 'dpkg-buildpackage. Did you mean to '
813+ 'call `git ubuntu build` instead?' %
814+ ' '.join(args.rem_args)
815+ )
816+ else:
817+ args.rem_args = ['--', '-S', '-nc', '-d', '-i', '-I']
818+ if args.for_merge:
819+ args.rem_args += ['-sa']
820+ if not args.sign:
821+ args.rem_args += ['-us', '-uc']
822+ build_cli_main(args)
823diff --git a/gitubuntu/clone.py b/gitubuntu/clone.py
824index a1bc8cb..e2dcedf 100644
825--- a/gitubuntu/clone.py
826+++ b/gitubuntu/clone.py
827@@ -18,12 +18,80 @@ except ImportError:
828 logging.error('Is %s installed?', pkg)
829 sys.exit(1)
830
831-class GitUbuntuClone:
832- def parse_args(self, subparsers=None, base_subparsers=None):
833- kwargs = dict(
834- description='Clone a source package git repository to a directory',
835- formatter_class=argparse.RawTextHelpFormatter,
836- epilog='''
837+def copy_hooks(src, dst):
838+ try:
839+ os.mkdir(dst)
840+ except FileExistsError:
841+ pass
842+
843+ for hook in os.listdir(src):
844+ shutil.copy2(
845+ os.path.join(src, hook),
846+ dst,
847+ )
848+
849+def main(directory, lp_user, proto, package):
850+ if os.path.isdir(directory):
851+ logging.error('directory %s exists' % directory)
852+ sys.exit(1)
853+
854+ local_repo = GitUbuntuRepository(
855+ local_dir=directory,
856+ lp_user=lp_user,
857+ fetch_proto=proto,
858+ )
859+
860+ copy_hooks(
861+ os.path.join(os.path.dirname(__file__), os.path.pardir, 'hooks'),
862+ os.path.join(directory, '.git', 'hooks')
863+ )
864+
865+ try:
866+ local_repo.add_base_remotes(package)
867+ local_repo.fetch_base_remotes(verbose=True)
868+ except GitUbuntuRepositoryFetchError:
869+ logging.error("Unable to find an imported repository for %s. "
870+ "Please request an import by e-mailing "
871+ "usd-import-team@lists.launchpad.net.",
872+ package
873+ )
874+ shutil.rmtree(local_repo.local_dir)
875+ raise
876+
877+ try:
878+ local_repo.add_lpuser_remote(pkgname=package)
879+ local_repo.fetch_lpuser_remote(verbose=True)
880+
881+ logging.debug("added remote '%s' -> %s", local_repo.lp_user,
882+ local_repo.raw_repo.remotes[local_repo.lp_user].url
883+ )
884+ except GitUbuntuRepositoryFetchError:
885+ pass
886+
887+ try:
888+ local_repo.create_tracking_branch(
889+ 'ubuntu/devel',
890+ 'pkg/ubuntu/devel'
891+ )
892+ local_repo.checkout_commitish('ubuntu/devel')
893+ except:
894+ logging.error('Unable to checkout ubuntu/devel, does '
895+ 'pkg/ubuntu/devel branch exist?'
896+ )
897+
898+ if os.path.isfile(os.path.join(directory, '.gitignore')):
899+ logging.warning('A .gitignore file exists in the source '
900+ 'package. This will affect the behavior of git. Consider '
901+ 'backing up the gitignore while working on this package '
902+ 'to ensure all changes are tracked or passing appropriate '
903+ 'flags to git commands (e.g., git status --ignored).'
904+ )
905+
906+def parse_args(subparsers=None, base_subparsers=None):
907+ kwargs = dict(
908+ description='Clone a source package git repository to a directory',
909+ formatter_class=argparse.RawTextHelpFormatter,
910+ epilog='''
911 Example:
912 * clone to open-iscsi/
913 %(prog)s open-iscsi
914@@ -32,107 +100,41 @@ Example:
915 * use https rather than git protocol for read-only remotes:
916 %(prog)s --https open-iscsi
917 '''
918- )
919- if base_subparsers:
920- kwargs['parents'] = base_subparsers
921- if subparsers:
922- parser = subparsers.add_parser('clone', **kwargs)
923- parser.set_defaults(func=self.main)
924- else:
925- parser = argparse.ArgumentParser(**kwargs)
926- parser.add_argument('package', type=str,
927- help='Name of source package to clone'
928- )
929- parser.add_argument('directory', type=str,
930- help='Local directory to clone to. If not specified, a '
931- ' directory with the same name as PACKAGE will be '
932- 'used',
933- default=None,
934- nargs='?'
935- )
936- parser.add_argument('-l', '--lp-user', type=str, help=argparse.SUPPRESS)
937- if not subparsers:
938- return parser.parse_args()
939- return 'clone - %s' % kwargs['description']
940-
941- @staticmethod
942- def copy_hooks(src, dst):
943- try:
944- os.mkdir(dst)
945- except FileExistsError:
946- pass
947-
948- for hook in os.listdir(src):
949- shutil.copy2(
950- os.path.join(src, hook),
951- dst,
952- )
953-
954- def main(self, args):
955- try:
956- lp_user = args.lp_user
957- except AttributeError:
958- lp_user = None
959- directory = (
960- os.path.abspath(args.directory)
961- if args.directory
962- else os.path.join(os.path.abspath(os.getcwd()), args.package)
963- )
964+ )
965+ if base_subparsers:
966+ kwargs['parents'] = base_subparsers
967+ if subparsers:
968+ parser = subparsers.add_parser('clone', **kwargs)
969+ parser.set_defaults(func=cli_main)
970+ else:
971+ parser = argparse.ArgumentParser(**kwargs)
972+ parser.add_argument('package', type=str,
973+ help='Name of source package to clone'
974+ )
975+ parser.add_argument('directory', type=str,
976+ help='Local directory to clone to. If not specified, a '
977+ ' directory with the same name as PACKAGE will be '
978+ 'used',
979+ default=None,
980+ nargs='?'
981+ )
982+ parser.add_argument('-l', '--lp-user', type=str, help=argparse.SUPPRESS)
983+ if not subparsers:
984+ return parser.parse_args()
985+ return 'clone - %s' % kwargs['description']
986
987- if os.path.isdir(directory):
988- logging.error('directory %s exists' % directory)
989- sys.exit(1)
990+def cli_main(args):
991+ try:
992+ lp_user = args.lp_user
993+ except AttributeError:
994+ lp_user = None
995+ directory = (
996+ os.path.abspath(args.directory)
997+ if args.directory
998+ else os.path.join(os.path.abspath(os.getcwd()), args.package)
999+ )
1000
1001- local_repo = GitUbuntuRepository(
1002- local_dir=directory,
1003- lp_user=lp_user,
1004- fetch_proto=args.proto,
1005- )
1006-
1007- self.copy_hooks(
1008- os.path.join(os.path.dirname(__file__), os.path.pardir, 'hooks'),
1009- os.path.join(directory, '.git', 'hooks')
1010- )
1011+ main(directory, lp_user, args.proto, args.package)
1012
1013- try:
1014- local_repo.add_base_remotes(args.package)
1015- local_repo.fetch_base_remotes(verbose=True)
1016- except GitUbuntuRepositoryFetchError:
1017- logging.error("Unable to find an imported repository for %s. "
1018- "Please request an import by e-mailing "
1019- "usd-import-team@lists.launchpad.net.",
1020- args.package
1021- )
1022- shutil.rmtree(local_repo.local_dir)
1023- raise
1024-
1025- try:
1026- local_repo.add_lpuser_remote(pkgname=args.package)
1027- local_repo.fetch_lpuser_remote(verbose=True)
1028-
1029- logging.debug("added remote '%s' -> %s", local_repo.lp_user,
1030- local_repo.raw_repo.remotes[local_repo.lp_user].url
1031- )
1032- except GitUbuntuRepositoryFetchError:
1033- pass
1034-
1035- try:
1036- local_repo.create_tracking_branch(
1037- 'ubuntu/devel',
1038- 'pkg/ubuntu/devel'
1039- )
1040- local_repo.checkout_commitish('ubuntu/devel')
1041- except:
1042- logging.error('Unable to checkout ubuntu/devel, does '
1043- 'pkg/ubuntu/devel branch exist?'
1044- )
1045-
1046- if os.path.isfile(os.path.join(directory, '.gitignore')):
1047- logging.warning('A .gitignore file exists in the source '
1048- 'package. This will affect the behavior of git. Consider '
1049- 'backing up the gitignore while working on this package '
1050- 'to ensure all changes are tracked or passing appropriate '
1051- 'flags to git commands (e.g., git status --ignored).'
1052- )
1053
1054 # vi: ts=4 expandtab
1055diff --git a/gitubuntu/importer.py b/gitubuntu/importer.py
1056index 22fd08a..4a67729 100644
1057--- a/gitubuntu/importer.py
1058+++ b/gitubuntu/importer.py
1059@@ -26,6 +26,7 @@
1060
1061 import argparse
1062 import atexit
1063+import functools
1064 import getpass
1065 import logging
1066 import os
1067@@ -50,7 +51,12 @@ from gitubuntu.git_repository import (
1068 PristineTarError,
1069 )
1070 from gitubuntu.run import decode_binary, run, runq
1071-from gitubuntu.source_information import GitUbuntuSourceInformation, NoPublicationHistoryException, SourceExtractionException, launchpad_login_auth
1072+from gitubuntu.source_information import (
1073+ GitUbuntuSourceInformation,
1074+ NoPublicationHistoryException,
1075+ SourceExtractionException,
1076+ launchpad_login_auth,
1077+)
1078 from gitubuntu.version import VERSION
1079 from gitubuntu.versioning import version_compare
1080 try:
1081@@ -86,16 +92,18 @@ def dsc_to_tree_hash(repo, dsc_path):
1082 # extracted_dir.
1083
1084 extracted_dir = os.path.join(temp_dir, 'x')
1085- run(['dpkg-source',
1086- '-x',
1087- '--skip-patches',
1088- dsc_path,
1089- extracted_dir, # this is <temp_dir>/x
1090+ run(
1091+ [
1092+ 'dpkg-source',
1093+ '-x',
1094+ '--skip-patches',
1095+ dsc_path,
1096+ extracted_dir, # this is <temp_dir>/x
1097 ]
1098- )
1099+ )
1100
1101 if not os.path.exists(extracted_dir):
1102- logging.exception('No source extracted?')
1103+ logging.error("No source extracted?")
1104 raise SourceExtractionException(
1105 "Failed to find an extracted directory from "
1106 "dpkg-source -x"
1107@@ -104,1258 +112,1462 @@ def dsc_to_tree_hash(repo, dsc_path):
1108 return repo.dir_to_tree(extracted_dir)
1109
1110
1111-class GitUbuntuImport:
1112- def __init__(self):
1113- self.parent_overrides = None
1114-
1115- def parse_args(self, subparsers=None, base_subparsers=None):
1116- kwargs = dict(description='Update a launchpad git tree based upon '
1117- 'the state of the Ubuntu and Debian archives',
1118- formatter_class=argparse.ArgumentDefaultsHelpFormatter
1119- )
1120- if base_subparsers:
1121- kwargs['parents'] = base_subparsers
1122- if subparsers:
1123- parser = subparsers.add_parser('import', **kwargs)
1124- parser.set_defaults(func=self.main)
1125- else:
1126- parser = argparse.ArgumentParser(**kwargs)
1127- parser.add_argument('package', type=str,
1128- help='Which package to update in the git tree')
1129- parser.add_argument('-o', '--lp-owner', type=str,
1130- help=argparse.SUPPRESS,
1131- default='usd-import-team')
1132- parser.add_argument('-l', '--lp-user', type=str,
1133- help=argparse.SUPPRESS)
1134- parser.add_argument('--dl-cache', type=str,
1135- help=('Cache directory for downloads. Default is to '
1136- 'put downloads in <directory>/.git/%s' %
1137- CACHE_PATH
1138- ),
1139- default=argparse.SUPPRESS)
1140- parser.add_argument('--no-fetch', action='store_true',
1141- help='Do not fetch from the remote (DANGEROUS; '
1142- 'only useful for debugging); implies '
1143- '--no-push')
1144- parser.add_argument('--no-push', action='store_true',
1145- help='Do not push to the remote')
1146- parser.add_argument('--no-clean', action='store_true',
1147- help='Do not clean the temporary directory')
1148- parser.add_argument('-d', '--directory', type=str,
1149- help='Use git repository at specified location rather '
1150- 'than a temporary directory (implies --no-clean). '
1151- 'Will create the local repository if needed.',
1152- default=argparse.SUPPRESS
1153- )
1154- parser.add_argument('--fixup-devel', action='store_true',
1155- help=argparse.SUPPRESS)
1156- parser.add_argument('--active-series-only', action='store_true',
1157- help='Do an import of only active Ubuntu series '
1158- 'history, implies --no-push.')
1159- parser.add_argument('--skip-applied', action='store_true',
1160- help=argparse.SUPPRESS)
1161- parser.add_argument('--skip-orig', action='store_true',
1162- help=argparse.SUPPRESS)
1163- parser.add_argument('--reimport', action='store_true',
1164- help=argparse.SUPPRESS)
1165- if not subparsers:
1166- return parser.parse_args()
1167- return 'import - %s' % kwargs['description']
1168-
1169- def cleanup(self, no_clean, local_dir):
1170- """Recursively remove local_dir if no_clean is not True"""
1171- if not no_clean:
1172- shutil.rmtree(local_dir)
1173- else:
1174- logging.info('Leaving %s as directed' % local_dir)
1175-
1176- def get_changelog_for_commit(self, tree_hash, publish_parent_commit, changelog_parent_commit):
1177- """Extract changes to debian/changelog in this publish
1178-
1179- LP: #1633114 -- favor the changelog parent's diff
1180- """
1181- raw_clog_entry = b''
1182- if changelog_parent_commit is not None:
1183- cmd = ['diff-tree', '-p', changelog_parent_commit,
1184- tree_hash, '--', 'debian/changelog']
1185- raw_clog_entry = self.local_repo.git_run(cmd).stdout
1186- elif publish_parent_commit is not None:
1187- cmd = ['diff-tree', '-p', publish_parent_commit,
1188- tree_hash, '--', 'debian/changelog']
1189- raw_clog_entry = self.local_repo.git_run(cmd).stdout
1190-
1191- changelog_entry = b''
1192- changelog_entry_found = False
1193- i = 0
1194- for line in raw_clog_entry.split(b'\n'):
1195- # skip the header lines
1196- if i < 4:
1197- i += 1
1198- continue
1199- if not line.startswith(b'+'):
1200- # in case there are stray changelog changes, just take
1201- # the first complete section of changelog additions
1202- if changelog_entry_found:
1203- break
1204- continue
1205- line = re.sub(b'^[+]', b'', line)
1206- if not line.startswith(b' '):
1207- continue
1208- if not changelog_entry_found:
1209- changelog_entry = b'\n\nNew changelog entries:\n'
1210- changelog_entry += line + b'\n'
1211- changelog_entry_found = True
1212- return changelog_entry
1213-
1214- def _commit_import(self, spi, tree_hash, publish_parent_commit, changelog_parent_commit, upload_parent_commit, unapplied_parent_commit):
1215- """Commit a tree object into the repository with the specified
1216- parents
1217-
1218- The importer algorithm works by 'staging' the import of a
1219- publication as a tree, then using that tree to determine the
1220- appropriate parents from the publishing history and
1221- debian/changelog.
1222-
1223- Arguments:
1224- spi - A SourcePackageInformation instance for this publish
1225- tree_hash - SHA1 from git-dsc-commit --tree-only of this publish
1226- publish_parent_commit = SHA1 of publishing parent
1227- changelog_parent_commit = SHA1 of changelog parent
1228- upload_parent_commit = SHA1 of upload parent
1229- unapplied_parent_commit = SHA1 of unapplied-patches import parent
1230- """
1231- tag = None
1232-
1233- if unapplied_parent_commit:
1234- import_type = 'patches-applied'
1235- target_head_name = spi.applied_head_name(self.namespace)
1236- if self.local_repo.get_applied_tag(spi.version, self.namespace) is None:
1237- # Not imported before
1238- tag = applied_tag(spi.version, self.namespace)
1239- else:
1240- import_type = 'patches-unapplied'
1241- target_head_name = spi.head_name(self.namespace)
1242- if self.local_repo.get_import_tag(spi.version, self.namespace) is None:
1243- # Not imported before
1244- tag = import_tag(spi.version, self.namespace)
1245-
1246- # Do not show importer/ namespace to user
1247- _, _, pretty_head_name = target_head_name.partition('%s/' % self.namespace)
1248-
1249- changelog_entry = self.get_changelog_for_commit(
1250- tree_hash,
1251- publish_parent_commit,
1252- changelog_parent_commit
1253- )
1254- msg = (b'Import %s version %b to %b\n\nImported using git-ubuntu import.' %
1255- (import_type.encode(), spi.version.encode(), pretty_head_name.encode())
1256- )
1257-
1258- parents = []
1259-
1260- if publish_parent_commit is None and \
1261- changelog_parent_commit is None and \
1262- upload_parent_commit is None and \
1263- unapplied_parent_commit is None and \
1264- target_head_name in self.local_repo.local_branch_names:
1265- # No parents and not creating a new branch
1266- tag = orphan_tag(spi.version, self.namespace)
1267- else:
1268- msg += b'\n'
1269-
1270- if publish_parent_commit is not None:
1271- parents.append(publish_parent_commit)
1272- msg += b'\nPublish parent: %b' % publish_parent_commit.encode()
1273- if changelog_parent_commit is not None:
1274- parents.append(changelog_parent_commit)
1275- msg += b'\nChangelog parent: %b' % changelog_parent_commit.encode()
1276- if upload_parent_commit is not None:
1277- parents.append(upload_parent_commit)
1278- msg += b'\nUpload parent: %b' % upload_parent_commit.encode()
1279- if unapplied_parent_commit is not None:
1280- parents.append(unapplied_parent_commit)
1281- msg += b'\nUnapplied parent: %b' % unapplied_parent_commit.encode()
1282-
1283- msg += b'%b' % changelog_entry
1284-
1285- commit_hash = self.local_repo.commit_tree_hash(
1286- tree_hash,
1287- parents,
1288- msg,
1289- environment_spi=spi
1290- )
1291+def cleanup(no_clean, local_dir):
1292+ '''Remove a local directory, conditionally
1293
1294- self.local_repo.update_head_to_commit(target_head_name, commit_hash)
1295+ no_clean: if True, do not remove @local_dir and print a message instead
1296+ local_dir: directory to remove
1297+ '''
1298+ if not no_clean:
1299+ shutil.rmtree(local_dir)
1300+ else:
1301+ logging.info('Leaving %s as directed' % local_dir)
1302
1303- logging.debug('Committed %s import of %s as %s in %s',
1304- import_type, spi.version, commit_hash, pretty_head_name
1305- )
1306
1307- if tag is not None:
1308- # should be annotated to use create_tag API
1309- logging.debug('Creating tag %s pointing to %s', tag, commit_hash)
1310- self.local_repo.git_run(['tag', '-a', '-m', 'git-ubuntu import v%s' % VERSION,
1311- tag, commit_hash
1312- ]
1313- )
1314+# XXX: need a namedtuple to hold common arguments
1315+def main(pkgname, owner, no_clean, directory, user, proto, no_fetch,
1316+ no_push, dl_cache, fixup_devel, active_series_only, skip_orig,
1317+ pullfile, parentfile, retries, retry_backoffs, skip_applied,
1318+ reimport,
1319+):
1320+ if owner == 'usd-import-team':
1321+ namespace = 'importer'
1322+ else:
1323+ namespace = owner
1324
1325- def commit_unapplied_patches_import(self, spi, import_tree_hash, publish_parent_commit, changelog_parent_commit, upload_parent_commit):
1326- self._commit_import(
1327- spi,
1328- import_tree_hash,
1329- publish_parent_commit,
1330- changelog_parent_commit,
1331- upload_parent_commit,
1332- # unapplied trees do not have a distinct unapplied parent
1333- None
1334- )
1335+ logging.info('Ubuntu Server Team importer v%s' % VERSION)
1336
1337- def commit_applied_patches_import(self, spi, import_tree_hash, publish_parent_commit, changelog_parent_commit, unapplied_parent_commit):
1338- self._commit_import(
1339- spi,
1340- import_tree_hash,
1341- publish_parent_commit,
1342- changelog_parent_commit,
1343- # uploads will be unapplied trees currently, so applied trees
1344- # can not have them as direct parents
1345- None,
1346- unapplied_parent_commit
1347- )
1348+ repo = GitUbuntuRepository(directory, user, proto)
1349+ if not directory:
1350+ logging.info('Using git repository at %s', repo.local_dir)
1351+
1352+ atexit.register(cleanup, no_clean, repo.local_dir)
1353
1354- def import_dsc(self, dsc, version, dist):
1355- """Imports the dsc file to importer/{debian,ubuntu}/dsc
1356-
1357- Arguments:
1358- spi - A GitUbuntuSourcePackageInformation instance
1359- """
1360-
1361- # Add DSC file to the appropriate branch
1362- runq(['git', 'checkout', '%s/importer/%s/dsc' % (self.namespace, dist)], env=self.local_repo.env)
1363- # It is possible the same-named DSC is published multiple times
1364- # with different contents, e.g. on epoch bumps
1365- local_dir = os.path.join(self.local_repo.local_dir, version)
1366- os.makedirs(local_dir, exist_ok=True)
1367- shutil.copy(dsc.dsc_path,
1368- local_dir)
1369- self.local_repo.git_run(['add', self.local_repo.local_dir])
1370+ repo.add_base_remotes(pkgname, repo_owner=owner)
1371+ if not no_fetch and not reimport:
1372 try:
1373- # Anything to commit? (this will be 0 if, for instance, the
1374- # same dsc is being imported again)
1375- runq(['git', 'diff', '--exit-code', 'HEAD'], env=self.local_repo.env)
1376- except:
1377- self.local_repo.git_run(['commit', '-m', 'DSC file for %s' % version])
1378+ repo.fetch_base_remotes()
1379+ except GitUbuntuRepositoryFetchError:
1380+ pass
1381
1382- self.local_repo.clean_repository_state()
1383+ if not no_fetch:
1384+ repo.delete_branches_in_namespace(namespace)
1385+ repo.delete_tags_in_namespace(namespace)
1386+ repo.copy_base_references(namespace)
1387
1388- def orig_imported(self, orig_tarball_paths, dist):
1389- """Determines if a list of paths to tarballs have already been imported to @dist
1390+ if not no_fetch and not reimport:
1391+ try:
1392+ repo.fetch_remote_refspecs('pkg',
1393+ refspecs=['refs/tags/*:refs/tags/%s/*' % namespace],
1394+ )
1395+ except GitUbuntuRepositoryFetchError:
1396+ pass
1397
1398- Assumes that the pristine-tar branch exists
1399- """
1400+ if reimport:
1401+ if no_fetch:
1402+ logging.warning(
1403+ "--reimport specified with --no-fetch. Upload "
1404+ "tags would not be incorporated into the new import "
1405+ "and would be lost forever. This is not currently "
1406+ "supported and --no-fetch will be ignored for upload "
1407+ "tags."
1408+ )
1409 try:
1410- return self.local_repo.verify_pristine_tar(
1411- orig_tarball_paths,
1412- dist,
1413- self.namespace
1414+ repo.fetch_remote_refspecs('pkg',
1415+ refspecs=['refs/tags/upload/*:refs/tags/%s/upload/*' % namespace],
1416 )
1417- except PristineTarError as e:
1418- raise GitUbuntuImportOrigError from e
1419+ except GitUbuntuRepositoryFetchError:
1420+ pass
1421
1422- def import_orig(self, spi):
1423- """Imports the orig-tarball using gbp import-org --pristine-tar
1424+ repo.ensure_importer_branches_exist(namespace)
1425+
1426+ debian_sinfo = GitUbuntuSourceInformation(
1427+ 'debian',
1428+ pkgname,
1429+ os.path.abspath(pullfile),
1430+ retries,
1431+ retry_backoffs,
1432+ )
1433+
1434+ ubuntu_sinfo = GitUbuntuSourceInformation(
1435+ 'ubuntu',
1436+ pkgname,
1437+ os.path.abspath(pullfile),
1438+ retries,
1439+ retry_backoffs,
1440+ )
1441+
1442+ debian_head_versions = (
1443+ repo.get_heads_and_versions('debian', namespace)
1444+ )
1445+ for head_name in sorted(debian_head_versions):
1446+ _, _, pretty_head_name = head_name.partition('%s/' % namespace)
1447+ logging.debug('Last imported %s version is %s',
1448+ pretty_head_name,
1449+ debian_head_versions[head_name]['version']
1450+ )
1451
1452- Arguments:
1453- spi - A GitUbuntuSourcePackageInformation instance
1454- """
1455- dsc = GitUbuntuDsc(spi.dsc_pathname)
1456- upstream_version = str(spi.upstream_version)
1457- version = str(spi.version)
1458- dist = spi.distribution.name.lower()
1459+ ubuntu_head_versions = (
1460+ repo.get_heads_and_versions('ubuntu', namespace)
1461+ )
1462+ for head_name in sorted(ubuntu_head_versions):
1463+ _, _, pretty_head_name = head_name.partition('%s/' % namespace)
1464+ logging.debug('Last imported %s version is %s',
1465+ pretty_head_name,
1466+ ubuntu_head_versions[head_name]['version']
1467+ )
1468
1469- try:
1470- orig_tarball_path = dsc.orig_tarball_path
1471- except GitUbuntuDscError as e:
1472- raise GitUbuntuImportOrigError(
1473- 'Unable to import orig tarball in DSC for version %s' % version
1474- ) from e
1475-
1476- if orig_tarball_path is None:
1477- # native packages only have a .tar.*
1478- native_re = re.compile(r'.*\.tar\.[^.]+$')
1479- for entry in dsc['Files']:
1480- if native_re.match(entry['name']):
1481- return
1482- raise GitUbuntuImportOrigError('No orig tarball in '
1483- 'DSC for version %s.' % version)
1484+ if not skip_applied:
1485+ applied_debian_head_versions = (
1486+ repo.get_heads_and_versions(
1487+ 'applied/debian', namespace
1488+ )
1489+ )
1490+ for head_name in sorted(applied_debian_head_versions):
1491+ _, _, pretty_head_name = head_name.partition(
1492+ '%s/' % namespace
1493+ )
1494+ logging.debug('Last applied %s version is %s',
1495+ pretty_head_name,
1496+ applied_debian_head_versions[head_name]['version']
1497+ )
1498
1499- try:
1500- orig_component_paths = dsc.component_tarball_paths
1501- except GitUbuntuDscError as e:
1502- raise GitUbuntuImportOrigError(
1503- 'Unable to import component tarballs in DSC for version '
1504- '%s' % version
1505- ) from e
1506-
1507- orig_paths = [orig_tarball_path] + list(orig_component_paths.values())
1508- if (not orig_tarball_path or
1509- self.orig_imported(orig_paths, dist)):
1510- return
1511- if not all(map(os.path.exists, orig_paths)):
1512- raise GitUbuntuImportOrigError(
1513- 'Unable to find tarball: %s' %
1514- [p for p in orig_paths if not os.path.exists(p)]
1515+ applied_ubuntu_head_versions = (
1516+ repo.get_heads_and_versions(
1517+ 'applied/ubuntu', namespace
1518+ )
1519+ )
1520+ for head_name in sorted(applied_ubuntu_head_versions):
1521+ _, _, pretty_head_name = head_name.partition(
1522+ '%s/' % namespace
1523+ )
1524+ logging.debug('Last applied %s version is %s',
1525+ pretty_head_name,
1526+ applied_ubuntu_head_versions[head_name]['version']
1527 )
1528
1529- try:
1530- with self.local_repo.pristine_tar_branches(dist, self.namespace):
1531- oldcwd = os.getcwd()
1532- # gbp does not support running from arbitrary git trees or
1533- # working directories
1534- # https://github.com/agx/git-buildpackage/pull/16
1535- os.chdir(self.local_repo.local_dir)
1536-
1537- ext = os.path.splitext(dsc.orig_tarball_path)[1]
1538-
1539- # make this non-fatal if upstream already exist as tagged?
1540- cmd = ['gbp', 'import-orig', '--no-merge',
1541- '--upstream-branch', 'do-not-push',
1542- '--pristine-tar', '--no-interactive',
1543- '--no-symlink-orig',
1544- '--upstream-tag=%s/upstream/%s/%%(version)s%s' %
1545- (self.namespace, dist, ext)]
1546- cmd.extend(map(lambda x: '--component=%s' % x,
1547- list(orig_component_paths.keys()))
1548- )
1549- cmd.extend([orig_tarball_path,])
1550- run(cmd, env=self.local_repo.env)
1551- except CalledProcessError as e:
1552- raise GitUbuntuImportOrigError(
1553- 'Unable to import tarballs: %s' % orig_paths
1554- ) from e
1555- finally:
1556- os.chdir(oldcwd)
1557+ oldcwd = os.getcwd()
1558+ os.chdir(repo.local_dir)
1559+
1560+ if dl_cache is None:
1561+ workdir = os.path.join(repo.git_dir, CACHE_PATH)
1562+ else:
1563+ workdir = dl_cache
1564+
1565+ os.makedirs(workdir, exist_ok=True)
1566+
1567+ # now sets a global _PARENT_OVERRIDES
1568+ parse_parentfile(parentfile, pkgname)
1569+
1570+ only_debian, history_found = import_publishes(
1571+ repo=repo,
1572+ pkgname=pkgname,
1573+ namespace=namespace,
1574+ patches_applied=False,
1575+ debian_head_versions=debian_head_versions,
1576+ ubuntu_head_versions=ubuntu_head_versions,
1577+ debian_sinfo=debian_sinfo,
1578+ ubuntu_sinfo=ubuntu_sinfo,
1579+ active_series_only=active_series_only,
1580+ workdir=workdir,
1581+ fixup_devel=fixup_devel,
1582+ skip_orig=skip_orig,
1583+ )
1584+
1585+ if not history_found:
1586+ logging.error("No publication history for '%s' in debian or ubuntu. "
1587+ "Wrong source package name?", pkgname)
1588+ sys.exit(1)
1589+
1590+ if not skip_applied:
1591+ import_publishes(
1592+ repo=repo,
1593+ pkgname=pkgname,
1594+ namespace=namespace,
1595+ patches_applied=True,
1596+ debian_head_versions=applied_debian_head_versions,
1597+ ubuntu_head_versions=applied_ubuntu_head_versions,
1598+ debian_sinfo=debian_sinfo,
1599+ ubuntu_sinfo=ubuntu_sinfo,
1600+ active_series_only=active_series_only,
1601+ workdir=workdir,
1602+ fixup_devel=fixup_devel,
1603+ skip_orig=skip_orig,
1604+ )
1605
1606- self.local_repo.clean_repository_state()
1607+ os.chdir(oldcwd)
1608
1609- def import_patches_unapplied_tree(self, dsc_pathname):
1610- """Imports the patches-unapplied source package and writes the
1611- corresponding working tree for a given srcpkg
1612+ repo.garbage_collect()
1613
1614- Arguments:
1615- dsc_pathname - A string path to a DSC file
1616- """
1617+ if no_push:
1618+ logging.info('Not pushing to remote as specified')
1619+ else:
1620+ lp = launchpad_login_auth()
1621+ repo_path = '~%s/ubuntu/+source/%s/+git/%s' % (
1622+ owner,
1623+ pkgname,
1624+ pkgname,
1625+ )
1626+ lp_git_repo = lp.git_repositories.getByPath(path=repo_path)
1627+ if reimport:
1628+ if not lp_git_repo.canBeDeleted():
1629+ logging.warning(
1630+ "There are linked MPs to the pkg repository, "
1631+ "which will be deleted:"
1632+ )
1633+ for mp in lp_git_repo.landing_candidates:
1634+ logging.warning(
1635+ "Adding comment to %s indicating reimport",
1636+ mp.web_link
1637+ )
1638+ mp.createComment(
1639+ content="%s has been reimported and this MP "
1640+ "must be manually resubmitted." %
1641+ pkgname,
1642+ subject="%s has been reimported" % pkgname,
1643+ )
1644+ # repo deletion will delete the MP, but hopefully
1645+ # the above comment will be sufficient to explain
1646+ # why
1647+ lp_git_repo.lp_delete()
1648+
1649+ repo.git_run([
1650+ 'push', '--atomic', 'pkg',
1651+ 'refs/heads/%s/*:refs/heads/*' % namespace,
1652+ 'refs/tags/%s/*:refs/tags/*' % namespace,
1653+ ])
1654+ # update our reference
1655+ lp_git_repo = lp.git_repositories.getByPath(path=repo_path)
1656+ for i in range(retries):
1657+ try:
1658+ if only_debian:
1659+ lp_git_repo.default_branch = 'refs/heads/debian/sid'
1660+ else:
1661+ lp_git_repo.default_branch = 'refs/heads/ubuntu/devel'
1662+ lp_git_repo.lp_save()
1663+ break
1664+ except (NotFound, PreconditionFailed) as e:
1665+ time.sleep(retry_backoffs[i])
1666+ lp_git_repo.lp_refresh()
1667+
1668+
1669+def get_changelog_for_commit(
1670+ repo,
1671+ tree_hash,
1672+ publish_parent_commit,
1673+ changelog_parent_commit,
1674+):
1675+ """Extract changes to debian/changelog in this publish
1676+
1677+ LP: #1633114 -- favor the changelog parent's diff
1678+ """
1679+ raw_clog_entry = b''
1680+ if changelog_parent_commit is not None:
1681+ cmd = ['diff-tree', '-p', changelog_parent_commit,
1682+ tree_hash, '--', 'debian/changelog']
1683+ raw_clog_entry = repo.git_run(cmd).stdout
1684+ elif publish_parent_commit is not None:
1685+ cmd = ['diff-tree', '-p', publish_parent_commit,
1686+ tree_hash, '--', 'debian/changelog']
1687+ raw_clog_entry = repo.git_run(cmd).stdout
1688+
1689+ changelog_entry = b''
1690+ changelog_entry_found = False
1691+ i = 0
1692+ for line in raw_clog_entry.split(b'\n'):
1693+ # skip the header lines
1694+ if i < 4:
1695+ i += 1
1696+ continue
1697+ if not line.startswith(b'+'):
1698+ # in case there are stray changelog changes, just take
1699+ # the first complete section of changelog additions
1700+ if changelog_entry_found:
1701+ break
1702+ continue
1703+ line = re.sub(b'^[+]', b'', line)
1704+ if not line.startswith(b' '):
1705+ continue
1706+ if not changelog_entry_found:
1707+ changelog_entry = b'\n\nNew changelog entries:\n'
1708+ changelog_entry += line + b'\n'
1709+ changelog_entry_found = True
1710+ return changelog_entry
1711+
1712+def _commit_import(repo, spi, tree_hash, namespace,
1713+ publish_parent_commit,
1714+ changelog_parent_commit, upload_parent_commit,
1715+ unapplied_parent_commit
1716+):
1717+ """Commit a tree object into the repository with the specified
1718+ parents
1719+
1720+ The importer algorithm works by 'staging' the import of a
1721+ publication as a tree, then using that tree to determine the
1722+ appropriate parents from the publishing history and
1723+ debian/changelog.
1724+
1725+ Arguments:
1726+ spi - A SourcePackageInformation instance for this publish
1727+ tree_hash - SHA1 from git-dsc-commit --tree-only of this publish
1728+ publish_parent_commit = SHA1 of publishing parent
1729+ changelog_parent_commit = SHA1 of changelog parent
1730+ upload_parent_commit = SHA1 of upload parent
1731+ unapplied_parent_commit = SHA1 of unapplied-patches import parent
1732+ """
1733+ tag = None
1734+
1735+ if unapplied_parent_commit:
1736+ import_type = 'patches-applied'
1737+ target_head_name = spi.applied_head_name(namespace)
1738+ if repo.get_applied_tag(spi.version, namespace) is None:
1739+ # Not imported before
1740+ tag = applied_tag(spi.version, namespace)
1741+ else:
1742+ import_type = 'patches-unapplied'
1743+ target_head_name = spi.head_name(namespace)
1744+ if repo.get_import_tag(spi.version, namespace) is None:
1745+ # Not imported before
1746+ tag = import_tag(spi.version, namespace)
1747+
1748+ # Do not show importer/ namespace to user
1749+ _, _, pretty_head_name = target_head_name.partition('%s/' % namespace)
1750+
1751+ changelog_entry = get_changelog_for_commit(
1752+ repo,
1753+ tree_hash,
1754+ publish_parent_commit,
1755+ changelog_parent_commit
1756+ )
1757+ msg = (b'Import %s version %b to %b\n\nImported using git-ubuntu import.' %
1758+ (import_type.encode(), spi.version.encode(), pretty_head_name.encode())
1759+ )
1760+
1761+ parents = []
1762+
1763+ if publish_parent_commit is None and \
1764+ changelog_parent_commit is None and \
1765+ upload_parent_commit is None and \
1766+ unapplied_parent_commit is None and \
1767+ target_head_name in repo.local_branch_names:
1768+ # No parents and not creating a new branch
1769+ tag = orphan_tag(spi.version, namespace)
1770+ else:
1771+ msg += b'\n'
1772+
1773+ if publish_parent_commit is not None:
1774+ parents.append(publish_parent_commit)
1775+ msg += b'\nPublish parent: %b' % publish_parent_commit.encode()
1776+ if changelog_parent_commit is not None:
1777+ parents.append(changelog_parent_commit)
1778+ msg += b'\nChangelog parent: %b' % changelog_parent_commit.encode()
1779+ if upload_parent_commit is not None:
1780+ parents.append(upload_parent_commit)
1781+ msg += b'\nUpload parent: %b' % upload_parent_commit.encode()
1782+ if unapplied_parent_commit is not None:
1783+ parents.append(unapplied_parent_commit)
1784+ msg += b'\nUnapplied parent: %b' % unapplied_parent_commit.encode()
1785+
1786+ msg += b'%b' % changelog_entry
1787+
1788+ commit_hash = repo.commit_tree_hash(
1789+ tree_hash,
1790+ parents,
1791+ msg,
1792+ environment_spi=spi
1793+ )
1794+
1795+ repo.update_head_to_commit(target_head_name, commit_hash)
1796+
1797+ logging.debug('Committed %s import of %s as %s in %s',
1798+ import_type, spi.version, commit_hash, pretty_head_name
1799+ )
1800
1801- import_tree_hash = dsc_to_tree_hash(self.local_repo, dsc_pathname)
1802- self.local_repo.clean_repository_state()
1803+ if tag is not None:
1804+ # should be annotated to use create_tag API
1805+ logging.debug('Creating tag %s pointing to %s', tag, commit_hash)
1806+ repo.git_run(['tag', '-a', '-m', 'git-ubuntu import v%s' % VERSION,
1807+ tag, commit_hash
1808+ ]
1809+ )
1810
1811- return import_tree_hash
1812+def commit_unapplied_patches_import(
1813+ repo,
1814+ spi,
1815+ import_tree_hash,
1816+ namespace,
1817+ publish_parent_commit,
1818+ changelog_parent_commit,
1819+ upload_parent_commit,
1820+):
1821+ _commit_import(
1822+ repo,
1823+ spi,
1824+ import_tree_hash,
1825+ namespace,
1826+ publish_parent_commit,
1827+ changelog_parent_commit,
1828+ upload_parent_commit,
1829+ # unapplied trees do not have a distinct unapplied parent
1830+ None,
1831+ )
1832+
1833+def commit_applied_patches_import(
1834+ repo,
1835+ spi,
1836+ import_tree_hash,
1837+ namespace,
1838+ publish_parent_commit,
1839+ changelog_parent_commit,
1840+ unapplied_parent_commit
1841+):
1842+ _commit_import(
1843+ repo,
1844+ spi,
1845+ import_tree_hash,
1846+ namespace,
1847+ publish_parent_commit,
1848+ changelog_parent_commit,
1849+ # uploads will be unapplied trees currently, so applied trees
1850+ # can not have them as direct parents
1851+ None,
1852+ unapplied_parent_commit,
1853+ )
1854+
1855+def import_dsc(repo, dsc, namespace, version, dist):
1856+ """Imports the dsc file to importer/{debian,ubuntu}/dsc
1857+
1858+ Arguments:
1859+ spi - A GitUbuntuSourcePackageInformation instance
1860+ """
1861+
1862+ # Add DSC file to the appropriate branch
1863+ runq(['git', 'checkout', '%s/importer/%s/dsc' % (namespace, dist)], env=repo.env)
1864+ # It is possible the same-named DSC is published multiple times
1865+ # with different contents, e.g. on epoch bumps
1866+ local_dir = os.path.join(repo.local_dir, version)
1867+ os.makedirs(local_dir, exist_ok=True)
1868+ shutil.copy(dsc.dsc_path, local_dir)
1869+ repo.git_run(['add', repo.local_dir])
1870+ try:
1871+ # Anything to commit? (this will be 0 if, for instance, the
1872+ # same dsc is being imported again)
1873+ runq(['git', 'diff', '--exit-code', 'HEAD'], env=repo.env)
1874+ except:
1875+ repo.git_run(['commit', '-m', 'DSC file for %s' % version])
1876+
1877+ repo.clean_repository_state()
1878+
1879+def orig_imported(repo, orig_tarball_paths, namespace, dist):
1880+ """Determines if a list of paths to tarballs have already been imported to @dist
1881+
1882+ Assumes that the pristine-tar branch exists
1883+ """
1884+ try:
1885+ return repo.verify_pristine_tar(
1886+ orig_tarball_paths,
1887+ dist,
1888+ namespace,
1889+ )
1890+ except PristineTarError as e:
1891+ raise GitUbuntuImportOrigError from e
1892+
1893+def import_orig(repo, dsc, namespace, version, dist):
1894+ """Imports the orig-tarball using gbp import-org --pristine-tar
1895+
1896+ Arguments:
1897+ repo - A GitUbuntuRepository object
1898+ dsc - a GitUbuntuDsc object
1899+ namespace - string namespace to search for tags, etc.
1900+ version - the string package version
1901+ dist - the string distribution, either 'ubuntu' or 'debian'
1902+ """
1903+ try:
1904+ orig_tarball_path = dsc.orig_tarball_path
1905+ except GitUbuntuDscError as e:
1906+ raise GitUbuntuImportOrigError(
1907+ 'Unable to import orig tarball in DSC for version %s' % version
1908+ ) from e
1909+
1910+ if orig_tarball_path is None:
1911+ # native packages only have a .tar.*
1912+ native_re = re.compile(r'.*\.tar\.[^.]+$')
1913+ for entry in dsc['Files']:
1914+ if native_re.match(entry['name']):
1915+ return
1916+ raise GitUbuntuImportOrigError('No orig tarball in '
1917+ 'DSC for version %s.' % version
1918+ )
1919
1920- def import_patches_applied_tree(self, dsc_pathname):
1921- """Imports the patches-applied source package and writes the
1922- corresponding working tree for a given srcpkg, patch by patch
1923+ try:
1924+ orig_component_paths = dsc.component_tarball_paths
1925+ except GitUbuntuDscError as e:
1926+ raise GitUbuntuImportOrigError(
1927+ 'Unable to import component tarballs in DSC for version '
1928+ '%s' % version
1929+ ) from e
1930+
1931+ orig_paths = [orig_tarball_path] + list(orig_component_paths.values())
1932+ if (not orig_tarball_path or
1933+ orig_imported(repo, orig_paths, namespace, dist)
1934+ ):
1935+ return
1936+ if not all(map(os.path.exists, orig_paths)):
1937+ raise GitUbuntuImportOrigError('Unable to find tarball: '
1938+ '%s' % [p for p in orig_paths if not os.path.exists(p)])
1939+
1940+ try:
1941+ with repo.pristine_tar_branches(dist, namespace):
1942+ oldcwd = os.getcwd()
1943+ # gbp does not support running from arbitrary git trees or
1944+ # working directories
1945+ # https://github.com/agx/git-buildpackage/pull/16
1946+ os.chdir(repo.local_dir)
1947+
1948+ ext = os.path.splitext(dsc.orig_tarball_path)[1]
1949+
1950+ # make this non-fatal if upstream already exist as tagged?
1951+ cmd = ['gbp', 'import-orig', '--no-merge',
1952+ '--upstream-branch', 'do-not-push',
1953+ '--pristine-tar', '--no-interactive',
1954+ '--no-symlink-orig',
1955+ '--upstream-tag=%s/upstream/%s/%%(version)s%s' %
1956+ (namespace, dist, ext)]
1957+ cmd.extend(map(lambda x: '--component=%s' % x,
1958+ list(orig_component_paths.keys()))
1959+ )
1960+ cmd.extend([orig_tarball_path,])
1961+ run(cmd, env=repo.env)
1962+ except CalledProcessError as e:
1963+ raise GitUbuntuImportOrigError(
1964+ 'Unable to import tarballs: %s' % orig_paths
1965+ ) from e
1966+ finally:
1967+ os.chdir(oldcwd)
1968
1969- Arguments:
1970- dsc_pathname - A string path to a DSC file
1971- """
1972- oldcwd = os.getcwd()
1973+ repo.clean_repository_state()
1974
1975- extract_dir = tempfile.mkdtemp()
1976- os.chdir(extract_dir)
1977+def import_patches_unapplied_tree(repo, dsc_pathname):
1978+ """Imports the patches-unapplied source package and writes the
1979+ corresponding working tree for a given srcpkg
1980
1981- run(['dpkg-source', '-x',
1982- '--skip-patches',
1983- os.path.join(oldcwd, dsc_pathname)
1984- ]
1985- )
1986+ Arguments:
1987+ dsc_pathname - A string path to a DSC file
1988+ """
1989
1990- extracted_dir = None
1991- for path in os.listdir(extract_dir):
1992- if os.path.isdir(path):
1993- extracted_dir = os.path.join(extract_dir, path)
1994- break
1995- if extracted_dir is None:
1996- logging.exception('No source extracted?')
1997- raise SourceExtractionException("Failed to find an extracted "
1998- "directory from dpkg-source -x")
1999-
2000- if os.path.isdir(os.path.join(extracted_dir, '.pc')):
2001- self.local_repo.git_run(
2002- ['--work-tree', extracted_dir, 'add', '-f', '-A']
2003- )
2004- self.local_repo.git_run(
2005- ['--work-tree', extracted_dir, 'rm', '-r', '-f', '.pc']
2006- )
2007- cp = self.local_repo.git_run(
2008- ['--work-tree', extracted_dir, 'write-tree']
2009- )
2010- import_tree_hash = decode_binary(cp.stdout).strip()
2011- yield (
2012- import_tree_hash,
2013- None,
2014- 'Remove .pc directory from source package',
2015- )
2016+ import_tree_hash = dsc_to_tree_hash(repo, dsc_pathname)
2017+ repo.clean_repository_state()
2018
2019- try:
2020- try:
2021- cp = run(['dpkg-source', '--print-format', extracted_dir])
2022- fmt = decode_binary(cp.stdout).strip()
2023- if '3.0 (quilt)' not in fmt:
2024- raise StopIteration()
2025- except CalledProcessError as e:
2026- try:
2027- with open('debian/source/format', 'r') as f:
2028- for line in f:
2029- if re.match(r'3.0 (.*)', line):
2030- break
2031- else:
2032- raise StopIteration()
2033- # `man dpkg-source` indicates no d/s/format implies 1.0
2034- except OSError:
2035- raise StopIteration()
2036-
2037- while True:
2038- try:
2039- os.chdir(extracted_dir)
2040- run(['quilt', 'push'], rcs=[2])
2041- cp = run(['quilt', 'top'])
2042- patch_name = decode_binary(cp.stdout).strip()
2043- cp = run(['quilt', 'header'])
2044- header = decode_binary(cp.stdout).strip()
2045- patch_desc = None
2046- for regex in (r'Subject:\s*(.*?)$',
2047- r'Description:\s*(.*?)$'
2048- ):
2049- try:
2050- m = re.search(regex, header, re.MULTILINE)
2051- patch_desc = m.group(1)
2052- except AttributeError:
2053- pass
2054- if patch_desc is None:
2055- logging.debug('Unable to find Subject or Description in %s' % patch_name)
2056- self.local_repo.git_run(['--work-tree', extracted_dir, 'add', '-f', '-A'])
2057- self.local_repo.git_run(['--work-tree', extracted_dir, 'reset', 'HEAD', '--', '.pc'])
2058- cp = self.local_repo.git_run(['--work-tree', extracted_dir, 'write-tree'])
2059- import_tree_hash = decode_binary(cp.stdout).strip()
2060-
2061- yield (import_tree_hash, patch_name, patch_desc)
2062- except CalledProcessError as e:
2063- # quilt returns 2 when done pushing
2064- if e.returncode != 2:
2065- raise
2066- raise StopIteration()
2067- finally:
2068- os.chdir(oldcwd)
2069- shutil.rmtree(extract_dir)
2070- self.local_repo.clean_repository_state()
2071+ return import_tree_hash
2072
2073- def parse_parentfile(self, parentfile):
2074- """Extract parent overrides from a file
2075+def import_patches_applied_tree(repo, dsc_pathname):
2076+ """Imports the patches-applied source package and writes the
2077+ corresponding working tree for a given srcpkg, patch by patch
2078
2079- The parent overrides file specifies parent-child relationship(s),
2080- where the importer may not be able to determine them automatically.
2081- The format of the file is:
2082- <pkgname> <child version> <publish parent version> <changelog parent version>
2083- with one override per line.
2084+ Arguments:
2085+ dsc_pathname - A string path to a DSC file
2086+ """
2087+ oldcwd = os.getcwd()
2088
2089- <child version> is the published version to which this override
2090- applies.
2091+ extract_dir = tempfile.mkdtemp()
2092+ os.chdir(extract_dir)
2093
2094- The <publish parent version> is the prior version published in the
2095- same series/pocket, or the immediately preceding series/pocket, if
2096- this is the first publish in a series/pocket.
2097+ run(['dpkg-source', '-x',
2098+ '--skip-patches',
2099+ os.path.join(oldcwd, dsc_pathname)
2100+ ]
2101+ )
2102
2103- The <changelog parent version> is the prior version in the
2104- child publish's debian/changelog.
2105+ extracted_dir = None
2106+ for path in os.listdir(extract_dir):
2107+ if os.path.isdir(path):
2108+ extracted_dir = os.path.join(extract_dir, path)
2109+ break
2110+ if extracted_dir is None:
2111+ logging.exception('No source extracted?')
2112+ raise SourceExtractionException("Failed to find an extracted "
2113+ "directory from dpkg-source -x")
2114
2115- Keyword Arguments:
2116- parentfile -- Path to parent overrides file
2117- """
2118- if self.parent_overrides:
2119- return
2120- parent_overrides = dict()
2121+ if os.path.isdir(os.path.join(extracted_dir, '.pc')):
2122+ repo.git_run(
2123+ ['--work-tree', extracted_dir, 'add', '-f', '-A']
2124+ )
2125+ repo.git_run(
2126+ ['--work-tree', extracted_dir, 'rm', '-r', '-f', '.pc']
2127+ )
2128+ cp = repo.git_run(
2129+ ['--work-tree', extracted_dir, 'write-tree']
2130+ )
2131+ import_tree_hash = decode_binary(cp.stdout).strip()
2132+ yield (
2133+ import_tree_hash,
2134+ None,
2135+ 'Remove .pc directory from source package',
2136+ )
2137+
2138+ try:
2139 try:
2140- with open(parentfile) as f:
2141- for line in f:
2142- if line.startswith('#'):
2143- continue
2144- m = re.match(
2145- r'(?P<pkgname>\S*)\s*(?P<childversion>\S*)\s*(?P<parent1version>\S*)\s*(?P<parent2version>\S*)',
2146- line
2147- )
2148- if m is None:
2149- continue
2150- if m.group('pkgname') != self.pkgname:
2151- continue
2152- parent_overrides[m.group('childversion')] = {
2153- 'publish_parent':m.group('parent1version'),
2154- 'changelog_parent':m.group('parent2version')
2155- }
2156- except FileNotFoundError:
2157- pass
2158- self.parent_overrides = parent_overrides
2159+ cp = run(['dpkg-source', '--print-format', extracted_dir])
2160+ fmt = decode_binary(cp.stdout).strip()
2161+ if '3.0 (quilt)' not in fmt:
2162+ raise StopIteration()
2163+ except CalledProcessError as e:
2164+ try:
2165+ with open('debian/source/format', 'r') as f:
2166+ for line in f:
2167+ if re.match(r'3.0 (.*)', line):
2168+ break
2169+ else:
2170+ raise StopIteration()
2171+ # `man dpkg-source` indicates no d/s/format implies 1.0
2172+ except OSError:
2173+ raise StopIteration()
2174
2175- def override_parents(self, spi):
2176- unapplied_publish_parent_commit = None
2177- unapplied_changelog_parent_commit = None
2178- applied_publish_parent_commit = None
2179- applied_changelog_parent_commit = None
2180+ while True:
2181+ try:
2182+ os.chdir(extracted_dir)
2183+ run(['quilt', 'push'], rcs=[2])
2184+ cp = run(['quilt', 'top'])
2185+ patch_name = decode_binary(cp.stdout).strip()
2186+ cp = run(['quilt', 'header'])
2187+ header = decode_binary(cp.stdout).strip()
2188+ patch_desc = None
2189+ for regex in (r'Subject:\s*(.*?)$',
2190+ r'Description:\s*(.*?)$'
2191+ ):
2192+ try:
2193+ m = re.search(regex, header, re.MULTILINE)
2194+ patch_desc = m.group(1)
2195+ except AttributeError:
2196+ pass
2197+ if patch_desc is None:
2198+ logging.debug('Unable to find Subject or Description '
2199+ 'in %s' % patch_name
2200+ )
2201+ repo.git_run([
2202+ '--work-tree',
2203+ extracted_dir,
2204+ 'add',
2205+ '-f',
2206+ '-A',
2207+ ])
2208+ repo.git_run([
2209+ '--work-tree',
2210+ extracted_dir,
2211+ 'reset',
2212+ 'HEAD',
2213+ '--',
2214+ '.pc',
2215+ ])
2216+ cp = repo.git_run(['--work-tree', extracted_dir, 'write-tree'])
2217+ import_tree_hash = decode_binary(cp.stdout).strip()
2218+
2219+ yield (import_tree_hash, patch_name, patch_desc)
2220+ except CalledProcessError as e:
2221+ # quilt returns 2 when done pushing
2222+ if e.returncode != 2:
2223+ raise
2224+ raise StopIteration()
2225+ finally:
2226+ os.chdir(oldcwd)
2227+ shutil.rmtree(extract_dir)
2228+ repo.clean_repository_state()
2229+
2230+_PARENT_OVERRIDES = None
2231+def parse_parentfile(parentfile, pkgname):
2232+ """Extract parent overrides from a file
2233+
2234+ The parent overrides file specifies parent-child relationship(s),
2235+ where the importer may not be able to determine them automatically.
2236+ The format of the file is:
2237+ <pkgname> <child version> <publish parent version> <changelog parent version>
2238+ with one override per line.
2239+
2240+ <child version> is the published version to which this override
2241+ applies.
2242+
2243+ The <publish parent version> is the prior version published in the
2244+ same series/pocket, or the immediately preceding series/pocket, if
2245+ this is the first publish in a series/pocket.
2246+
2247+ The <changelog parent version> is the prior version in the
2248+ child publish's debian/changelog.
2249+
2250+ Keyword Arguments:
2251+ parentfile -- Path to parent overrides file
2252+ """
2253+ global _PARENT_OVERRIDES
2254+ if _PARENT_OVERRIDES:
2255+ return
2256+ _PARENT_OVERRIDES = dict()
2257+ try:
2258+ with open(parentfile) as f:
2259+ for line in f:
2260+ if line.startswith('#'):
2261+ continue
2262+ m = re.match(
2263+ r'(?P<pkgname>\S*)\s*(?P<childversion>\S*)\s*(?P<parent1version>\S*)\s*(?P<parent2version>\S*)',
2264+ line
2265+ )
2266+ if m is None:
2267+ continue
2268+ if m.group('pkgname') != pkgname:
2269+ continue
2270+ _PARENT_OVERRIDES[m.group('childversion')] = {
2271+ 'publish_parent':m.group('parent1version'),
2272+ 'changelog_parent':m.group('parent2version')
2273+ }
2274+ except FileNotFoundError:
2275+ pass
2276+
2277+def override_parents(repo, spi, namespace):
2278+ unapplied_publish_parent_commit = None
2279+ unapplied_changelog_parent_commit = None
2280+ applied_publish_parent_commit = None
2281+ applied_changelog_parent_commit = None
2282+
2283+ unapplied_publish_parent_tag = repo.get_import_tag(
2284+ _PARENT_OVERRIDES[spi.version]['publish_parent'],
2285+ namespace
2286+ )
2287+ applied_publish_parent_tag = repo.get_applied_tag(
2288+ _PARENT_OVERRIDES[spi.version]['publish_parent'],
2289+ namespace
2290+ )
2291+ if unapplied_publish_parent_tag is not None:
2292+ # sanity check that version from d/changelog of the
2293+ # tagged commit matches ours
2294+ parent_publish_version, _ = \
2295+ repo.get_changelog_versions_from_treeish(
2296+ str(unapplied_publish_parent_tag.peel().id),
2297+ )
2298+ if parent_publish_version != _PARENT_OVERRIDES[spi.version]['publish_parent']:
2299+ logging.error('Found a tag corresponding to publish parent '
2300+ 'override version %s, but d/changelog version (%s) '
2301+ 'differs. Will orphan commit.',
2302+ _PARENT_OVERRIDES[spi.version]['publish_parent'],
2303+ parent_publish_version
2304+ )
2305+ else:
2306+ unapplied_publish_parent_commit = str(unapplied_publish_parent_tag.peel().id)
2307+ if applied_publish_parent_tag is not None:
2308+ applied_publish_parent_commit = str(applied_publish_parent_tag.peel().id)
2309+ logging.debug('Overriding publish parent (tag) to %s',
2310+ repo.tag_to_pretty_name(unapplied_publish_parent_tag)
2311+ )
2312+ else:
2313+ if _PARENT_OVERRIDES[spi.version]['publish_parent'] == '-':
2314+ logging.debug('Not setting publish parent as '
2315+ 'specified in override file.'
2316+ )
2317+ else:
2318+ logging.error('Specified publish parent override '
2319+ '(%s) for version (%s) not found in tags. '
2320+ 'Unable to proceed.' % (
2321+ _PARENT_OVERRIDES[spi.version]['publish_parent'],
2322+ spi.version
2323+ )
2324+ )
2325+ sys.exit(1)
2326
2327- unapplied_publish_parent_tag = self.local_repo.get_import_tag(
2328- self.parent_overrides[spi.version]['publish_parent'],
2329- self.namespace)
2330- applied_publish_parent_tag = self.local_repo.get_applied_tag(
2331- self.parent_overrides[spi.version]['publish_parent'],
2332- self.namespace)
2333- if unapplied_publish_parent_tag is not None:
2334- # sanity check that version from d/changelog of the
2335- # tagged commit matches ours
2336- parent_publish_version, _ = \
2337- self.local_repo.get_changelog_versions_from_treeish(
2338- str(unapplied_publish_parent_tag.peel().id),
2339- )
2340- if parent_publish_version != self.parent_overrides[spi.version]['publish_parent']:
2341- logging.error('Found a tag corresponding to '
2342- 'publish parent override '
2343- 'version %s, but d/changelog '
2344- 'version (%s) differs. Will '
2345- 'orphan commit.' % (
2346- self.parent_overrides[spi.version]['publish_parent'],
2347- parent_publish_version
2348- )
2349- )
2350- else:
2351- unapplied_publish_parent_commit = str(unapplied_publish_parent_tag.peel().id)
2352- if applied_publish_parent_tag is not None:
2353- applied_publish_parent_commit = str(applied_publish_parent_tag.peel().id)
2354- logging.debug('Overriding publish parent (tag) to %s',
2355- self.local_repo.tag_to_pretty_name(unapplied_publish_parent_tag)
2356- )
2357+ unapplied_changelog_parent_tag = repo.get_import_tag(
2358+ _PARENT_OVERRIDES[spi.version]['changelog_parent'],
2359+ namespace
2360+ )
2361+ applied_changelog_parent_tag = repo.get_applied_tag(
2362+ _PARENT_OVERRIDES[spi.version]['changelog_parent'],
2363+ namespace
2364+ )
2365+ if unapplied_changelog_parent_tag is not None:
2366+ # sanity check that version from d/changelog of the
2367+ # tagged commit matches ours
2368+ parent_changelog_version, _ = repo.get_changelog_versions_from_treeish(
2369+ str(unapplied_changelog_parent_tag.peel().id),
2370+ )
2371+ if parent_changelog_version != _PARENT_OVERRIDES[spi.version]['changelog_parent']:
2372+ logging.error('Found a tag corresponding to '
2373+ 'changelog parent override '
2374+ 'version %s, but d/changelog '
2375+ 'version (%s) differs. Will '
2376+ 'orphan commit.' % (
2377+ _PARENT_OVERRIDES[spi.version]['changelog_parent'],
2378+ parent_changelog_version
2379+ )
2380+ )
2381 else:
2382- if self.parent_overrides[spi.version]['publish_parent'] == '-':
2383- logging.debug('Not setting publish parent as '
2384- 'specified in override file.'
2385- )
2386- else:
2387- logging.error('Specified publish parent override '
2388- '(%s) for version (%s) not found in tags. '
2389- 'Unable to proceed.' % (
2390- self.parent_overrides[spi.version]['publish_parent'],
2391- spi.version
2392- )
2393- )
2394- sys.exit(1)
2395-
2396- unapplied_changelog_parent_tag = self.local_repo.get_import_tag(
2397- self.parent_overrides[spi.version]['changelog_parent'],
2398- self.namespace)
2399- applied_changelog_parent_tag = self.local_repo.get_applied_tag(
2400- self.parent_overrides[spi.version]['changelog_parent'],
2401- self.namespace)
2402- if unapplied_changelog_parent_tag is not None:
2403- # sanity check that version from d/changelog of the
2404- # tagged commit matches ours
2405- parent_changelog_version, _ = \
2406- self.local_repo.get_changelog_versions_from_treeish(
2407- str(unapplied_changelog_parent_tag.peel().id),
2408- )
2409- if parent_changelog_version != self.parent_overrides[spi.version]['changelog_parent']:
2410- logging.error('Found a tag corresponding to '
2411- 'changelog parent override '
2412- 'version %s, but d/changelog '
2413- 'version (%s) differs. Will '
2414- 'orphan commit.' % (
2415- self.parent_overrides[spi.version]['changelog_parent'],
2416- parent_changelog_version
2417- )
2418- )
2419- else:
2420- unapplied_changelog_parent_commit = str(unapplied_changelog_parent_tag.peel().id)
2421- if applied_changelog_parent_tag is not None:
2422- applied_changelog_parent_commit = str(applied_changelog_parent_tag.peel().id)
2423- logging.debug('Overriding changelog parent (tag) to %s',
2424- self.local_repo.tag_to_pretty_name(unapplied_changelog_parent_tag)
2425- )
2426+ unapplied_changelog_parent_commit = str(unapplied_changelog_parent_tag.peel().id)
2427+ if applied_changelog_parent_tag is not None:
2428+ applied_changelog_parent_commit = str(applied_changelog_parent_tag.peel().id)
2429+ logging.debug('Overriding changelog parent (tag) to %s',
2430+ repo.tag_to_pretty_name(unapplied_changelog_parent_tag)
2431+ )
2432+ else:
2433+ if _PARENT_OVERRIDES[spi.version]['changelog_parent'] == '-':
2434+ logging.debug('Not setting changelog parent as specified '
2435+ 'in override file.'
2436+ )
2437 else:
2438- if self.parent_overrides[spi.version]['changelog_parent'] == '-':
2439- logging.debug('Not setting changelog parent as '
2440- 'specified in override file.'
2441- )
2442- else:
2443- logging.error('Specified changelog parent override '
2444- '(%s) for version (%s) not found in tags. '
2445- 'Unable to proceed.' % (
2446- self.parent_overrides[spi.version]['changelog_parent'],
2447- spi.version
2448- )
2449- )
2450- sys.exit(1)
2451- return (unapplied_publish_parent_commit,
2452- unapplied_changelog_parent_commit,
2453- applied_publish_parent_commit,
2454- applied_changelog_parent_commit)
2455-
2456- def _update_devel_branches(self, namespace, spi=None):
2457- if spi:
2458- # we do not maintain -devel pointers for debian
2459- if str(spi.distribution.name.lower()) != "ubuntu":
2460+ logging.error('Specified changelog parent override (%s) '
2461+ 'for version (%s) not found in tags. Unable to proceed.',
2462+ _PARENT_OVERRIDES[spi.version]['changelog_parent'],
2463+ spi.version
2464+ )
2465+ sys.exit(1)
2466+ return (
2467+ unapplied_publish_parent_commit,
2468+ unapplied_changelog_parent_commit,
2469+ applied_publish_parent_commit,
2470+ applied_changelog_parent_commit,
2471+ )
2472+
2473+def _update_devel_branches(
2474+ repo,
2475+ namespace,
2476+ pkgname,
2477+ ubuntu_sinfo,
2478+ spi=None,
2479+):
2480+ if spi:
2481+ # we do not maintain -devel pointers for debian
2482+ if str(spi.distribution.name.lower()) != "ubuntu":
2483+ return
2484+ for devel_head in (
2485+ '%s/ubuntu/%s-devel' % (namespace, spi.series.name.lower()),
2486+ '%s/ubuntu/devel' % namespace
2487+ ):
2488+ try:
2489+ devel_head_hash = repo.head_to_commit(devel_head)
2490+ devel_head_version, _ = repo.get_changelog_versions_from_treeish(devel_head_hash)
2491+ except AttributeError:
2492+ devel_head_version = None
2493+ if version_compare(spi.version, devel_head_version) <= 0:
2494 return
2495- for devel_head in (
2496- '%s/ubuntu/%s-devel' % (namespace, spi.series.name.lower()),
2497- '%s/ubuntu/devel' % namespace
2498- ):
2499- try:
2500- devel_head_hash = self.local_repo.head_to_commit(devel_head)
2501- devel_head_version, _ = self.local_repo.get_changelog_versions_from_treeish(devel_head_hash)
2502- except AttributeError:
2503- devel_head_version = None
2504- if version_compare(spi.version, devel_head_version) <= 0:
2505- return
2506- logging.debug("Updating %s to %s" % (devel_head,
2507- spi.head_name(namespace)))
2508- self.local_repo.merge_commit_to_devel_head(
2509- namespace, devel_head,
2510- self.local_repo.head_to_commit(spi.head_name(namespace)),
2511- environment_spi=spi)
2512+ logging.debug("Updating %s to %s" % (devel_head,
2513+ spi.head_name(namespace)))
2514+ repo.merge_commit_to_devel_head(
2515+ namespace,
2516+ devel_head,
2517+ repo.head_to_commit(spi.head_name(namespace)),
2518+ environment_spi=spi,
2519+ )
2520+ else:
2521+ ubuntu_head_versions = repo.get_heads_and_versions(
2522+ 'ubuntu',
2523+ namespace,
2524+ )
2525+ devel_hash = None
2526+ devel_name = None
2527+ for rel in ubuntu_sinfo.active_series_name_list:
2528+ series_devel_hash = None
2529+ series_devel_name = None
2530+ series_devel_version = None
2531+ # Find highest version publish in these pockets
2532+ for suff in ("-proposed", "-updates", "-security", ""):
2533+ name = "%s/ubuntu/%s%s" % (namespace, rel, suff)
2534+ if name in ubuntu_head_versions and version_compare(
2535+ ubuntu_head_versions[name]['version'],
2536+ series_devel_version
2537+ ) > 0:
2538+ series_devel_name = name
2539+ series_devel_version = ubuntu_head_versions[name]['version']
2540+ series_devel_hash = str(ubuntu_head_versions[name]['head'].peel().id)
2541+ if series_devel_hash:
2542+ if not devel_hash:
2543+ devel_hash = series_devel_hash
2544+ devel_name = series_devel_name
2545+ logging.debug("Updating %s/ubuntu/%s-devel branch to '%s' (%s)",
2546+ namespace, rel, series_devel_name, series_devel_hash)
2547+ repo.merge_commit_to_devel_head(namespace,
2548+ "%s/ubuntu/%s-devel" % (namespace, rel), series_devel_hash)
2549+ if devel_hash is None:
2550+ logging.warn("Package '%s' does not appear to have been published "
2551+ "in Ubuntu. No %s/ubuntu/devel branch created.",
2552+ pkgname, namespace)
2553 else:
2554- ubuntu_head_versions = self.local_repo.get_heads_and_versions(
2555- 'ubuntu', namespace
2556+ logging.debug("Setting %s/ubuntu/devel branch to '%s' (%s)",
2557+ namespace, devel_name, devel_hash)
2558+ repo.merge_commit_to_devel_head(
2559+ namespace, "%s/ubuntu/devel" % namespace, devel_hash
2560 )
2561- devel_hash = None
2562- devel_name = None
2563- for rel in self.ubuntu_sinfo.active_series_name_list:
2564- series_devel_hash = None
2565- series_devel_name = None
2566- series_devel_version = None
2567- # Find highest version publish in these pockets
2568- for suff in ("-proposed", "-updates", "-security", ""):
2569- name = "%s/ubuntu/%s%s" % (namespace, rel, suff)
2570- if name in ubuntu_head_versions and version_compare(
2571- ubuntu_head_versions[name]['version'],
2572- series_devel_version
2573- ) > 0:
2574- series_devel_name = name
2575- series_devel_version = ubuntu_head_versions[name]['version']
2576- series_devel_hash = str(ubuntu_head_versions[name]['head'].peel().id)
2577- if series_devel_hash:
2578- if not devel_hash:
2579- devel_hash = series_devel_hash
2580- devel_name = series_devel_name
2581- logging.debug("Updating %s/ubuntu/%s-devel branch to '%s' (%s)",
2582- namespace, rel, series_devel_name, series_devel_hash)
2583- self.local_repo.merge_commit_to_devel_head(namespace,
2584- "%s/ubuntu/%s-devel" % (namespace, rel), series_devel_hash)
2585- if devel_hash is None:
2586- logging.warn("Package '%s' does not appear to have been published "
2587- "in Ubuntu. No %s/ubuntu/devel branch created.",
2588- self.pkgname, namespace)
2589- else:
2590- logging.debug("Setting %s/ubuntu/devel branch to '%s' (%s)",
2591- namespace, devel_name, devel_hash)
2592- self.local_repo.merge_commit_to_devel_head(
2593- namespace, "%s/ubuntu/devel" % namespace, devel_hash
2594- )
2595-
2596- def update_devel_branches(self, spi=None):
2597- self._update_devel_branches(self.namespace, spi)
2598-
2599- def update_applied_devel_branches(self, spi=None):
2600- self._update_devel_branches('%s/applied' % self.namespace, spi)
2601-
2602- # imports a package based upon source package information
2603- def import_unapplied_spi(self, spi):
2604- """Imports a source package from Launchpad into the git
2605- repository
2606-
2607- Arguments:
2608- spi - a SourcePackageInformation instance
2609- """
2610- pretty_head_name = spi.pretty_head_name
2611
2612- logging.info('Importing patches-unapplied %s to %s' %
2613- (spi.version, pretty_head_name))
2614- unapplied_tip_head = self.local_repo.get_head_by_name(spi.head_name(self.namespace))
2615-
2616- spi.pull()
2617-
2618- self.local_repo.clean_repository_state()
2619-
2620- self.import_dsc(
2621+def update_devel_branches(
2622+ repo,
2623+ namespace,
2624+ pkgname,
2625+ ubuntu_sinfo,
2626+ spi=None,
2627+):
2628+ _update_devel_branches(
2629+ repo=repo,
2630+ namespace=namespace,
2631+ pkgname=pkgname,
2632+ ubuntu_sinfo=ubuntu_sinfo,
2633+ spi=spi,
2634+ )
2635+
2636+def update_applied_devel_branches(
2637+ repo,
2638+ namespace,
2639+ pkgname,
2640+ ubuntu_sinfo,
2641+ spi=None,
2642+):
2643+ _update_devel_branches(
2644+ repo=repo,
2645+ namespace='%s/applied' % namespace,
2646+ pkgname=pkgname,
2647+ ubuntu_sinfo=ubuntu_sinfo,
2648+ spi=spi,
2649+ )
2650+
2651+# imports a package based upon source package information
2652+def import_unapplied_spi(repo, spi, namespace, skip_orig, ubuntu_sinfo):
2653+ """Imports a source package from Launchpad into the git
2654+ repository
2655+
2656+ Arguments:
2657+ spi - a SourcePackageInformation instance
2658+ """
2659+ pretty_head_name = spi.pretty_head_name
2660+
2661+ logging.info('Importing patches-unapplied %s to %s',
2662+ spi.version, pretty_head_name
2663+ )
2664+ unapplied_tip_head = repo.get_head_by_name(spi.head_name(namespace))
2665+
2666+ spi.pull()
2667+
2668+ repo.clean_repository_state()
2669+
2670+ import_dsc(
2671+ repo,
2672+ GitUbuntuDsc(spi.dsc_pathname),
2673+ namespace,
2674+ str(spi.version),
2675+ spi.distribution.name.lower(),
2676+ )
2677+ if not skip_orig:
2678+ import_orig(
2679+ repo,
2680 GitUbuntuDsc(spi.dsc_pathname),
2681+ namespace,
2682 str(spi.version),
2683- spi.distribution.name.lower()
2684+ spi.distribution.name.lower(),
2685 )
2686- if not self.skip_orig:
2687- self.import_orig(spi)
2688
2689- unapplied_import_tree_hash = self.import_patches_unapplied_tree(spi.dsc_pathname)
2690- logging.debug('Imported patches-unapplied version %s as tree %s',
2691- spi.version, unapplied_import_tree_hash
2692- )
2693+ unapplied_import_tree_hash = import_patches_unapplied_tree(
2694+ repo,
2695+ spi.dsc_pathname
2696+ )
2697+ logging.debug(
2698+ "Imported patches-unapplied version %s as tree %s",
2699+ spi.version,
2700+ unapplied_import_tree_hash
2701+ )
2702+
2703+ import_tree_versions = repo.get_all_changelog_versions_from_treeish(
2704+ unapplied_import_tree_hash
2705+ )
2706+ changelog_version = import_tree_versions[0]
2707+
2708+ logging.debug(
2709+ "Found changelog version %s in newly imported tree" %
2710+ changelog_version
2711+ )
2712+
2713+ # sanity check on spph and d/changelog agreeing on the latest version
2714+ if spi.version not in _PARENT_OVERRIDES:
2715+ assert spi.version == changelog_version, (
2716+ 'source pkg version: {} != changelog version: {}'.format(
2717+ spi.version, changelog_version))
2718+
2719+ unapplied_tip_version = None
2720+ # check if the version to import has already been imported to
2721+ # this head
2722+ if unapplied_tip_head is not None:
2723+ if repo.treeishs_identical(
2724+ unapplied_import_tree_hash, str(unapplied_tip_head.peel().id)
2725+ ):
2726+ logging.warn('%s is identical to %s',
2727+ pretty_head_name, spi.version
2728+ )
2729+ return
2730+ unapplied_tip_version, _ = repo.get_changelog_versions_from_treeish(
2731+ str(unapplied_tip_head.peel().id),
2732+ )
2733
2734- import_tree_versions = self.local_repo.get_all_changelog_versions_from_treeish(
2735- unapplied_import_tree_hash
2736- )
2737- changelog_version = import_tree_versions[0]
2738+ logging.debug('Tip version is %s', unapplied_tip_version)
2739
2740+ unapplied_publish_parent_commit = None
2741+ unapplied_changelog_parent_commit = None
2742+ upload_parent_commit = None
2743+ unapplied_parent_commit = None
2744+
2745+ if spi.version in _PARENT_OVERRIDES:
2746 logging.debug(
2747- 'Found changelog version %s in newly imported tree' %
2748- changelog_version
2749- )
2750+ '%s is specified in the parent override file.',
2751+ spi.version
2752+ )
2753
2754- # sanity check on spph and d/changelog agreeing on the latest version
2755- if spi.version not in self.parent_overrides:
2756- assert spi.version == changelog_version, (
2757- 'source pkg version: {} != changelog version: {}'.format(
2758- spi.version, changelog_version))
2759-
2760- unapplied_tip_version = None
2761- # check if the version to import has already been imported to
2762- # this head
2763- if unapplied_tip_head is not None:
2764- if self.local_repo.treeishs_identical(
2765- unapplied_import_tree_hash, str(unapplied_tip_head.peel().id)
2766- ):
2767- logging.warn('%s is identical to %s',
2768- pretty_head_name, spi.version
2769+ (
2770+ unapplied_publish_parent_commit,
2771+ unapplied_changelog_parent_commit,
2772+ _,
2773+ _
2774+ ) = override_parents(repo, spi, namespace)
2775+ else:
2776+ # Get parent from publishing history (which is the last
2777+ # published version in the corresponding series)
2778+ try:
2779+ unapplied_publish_parent_commit = str(unapplied_tip_head.peel().id)
2780+ except AttributeError:
2781+ try:
2782+ unapplied_publish_parent_head = repo.get_head_by_name(spi.parent_head_name(namespace))
2783+ unapplied_publish_parent_commit = str(unapplied_publish_parent_head.peel().id)
2784+ except AttributeError:
2785+ pass
2786+ finally:
2787+ try:
2788+ unapplied_publish_parent_tag = repo.nearest_import_tag(unapplied_publish_parent_commit, namespace)
2789+ logging.debug('Publishing parent (tag) is %s',
2790+ repo.tag_to_pretty_name(unapplied_publish_parent_tag)
2791 )
2792- return
2793- unapplied_tip_version, _ = self.local_repo.get_changelog_versions_from_treeish(
2794- str(unapplied_tip_head.peel().id),
2795- )
2796+ except AttributeError:
2797+ pass
2798
2799- logging.debug('Tip version is %s', unapplied_tip_version)
2800+ if version_compare(str(spi.version), unapplied_tip_version) <= 0:
2801+ logging.warn('Version to import (%s) is not after %s tip (%s)',
2802+ spi.version, pretty_head_name, unapplied_tip_version
2803+ )
2804+
2805+ # Walk changelog backwards until we find an imported version
2806+ for version in import_tree_versions:
2807+ unapplied_changelog_parent_tag = repo.get_import_tag(version, namespace)
2808+ if unapplied_changelog_parent_tag is not None:
2809+ # sanity check that version from d/changelog of the
2810+ # tagged parent matches ours
2811+ parent_changelog_version, _ = \
2812+ repo.get_changelog_versions_from_treeish(
2813+ str(unapplied_changelog_parent_tag.peel(pygit2.Tree).id),
2814+ )
2815+ # if the parent was imported via a parent override
2816+ # itself, assume it is valid without checking its
2817+ # tree's debian/changelog, as it wasn't used
2818+ if (version not in _PARENT_OVERRIDES and
2819+ parent_changelog_version != version
2820+ ):
2821+ logging.error('Found a tag corresponding to '
2822+ 'parent version %s, but d/changelog '
2823+ 'version (%s) differs. Will '
2824+ 'orphan commit.' % (
2825+ version,
2826+ parent_changelog_version
2827+ )
2828+ )
2829+ else:
2830+ unapplied_changelog_parent_commit = str(unapplied_changelog_parent_tag.peel().id)
2831+ logging.debug('Changelog parent (tag) is %s',
2832+ repo.tag_to_pretty_name(unapplied_changelog_parent_tag)
2833+ )
2834+ break
2835
2836- unapplied_publish_parent_commit = None
2837+ # If the two parents are tree-identical, then favor publication
2838+ # history. This deals with copy-forwards between series and with
2839+ # syncs.
2840+ if repo.treeishs_identical(unapplied_publish_parent_commit, unapplied_changelog_parent_commit):
2841 unapplied_changelog_parent_commit = None
2842- upload_parent_commit = None
2843- unapplied_parent_commit = None
2844
2845- if spi.version in self.parent_overrides:
2846- logging.debug('%s is specified in the parent override file.' %
2847+ # check if the version to import has already been uploaded to
2848+ # this head -- we leverage the above code to perform a 'dry-run'
2849+ # of the import tree, to obtain the (up to) two parents
2850+ upload_tag = repo.get_upload_tag(spi.version, namespace)
2851+ import_tag = repo.get_import_tag(spi.version, namespace)
2852+ if upload_tag and not import_tag:
2853+ if str(upload_tag.peel().tree.id) != unapplied_import_tree_hash:
2854+ logging.warn('Found upload tag %s, '
2855+ 'but the corresponding tree '
2856+ 'does not match the published '
2857+ 'version. Will import %s as '
2858+ 'normal, ignoring the upload tag.',
2859+ repo.tag_to_pretty_name(upload_tag),
2860 spi.version
2861 )
2862-
2863- (unapplied_publish_parent_commit,
2864- unapplied_changelog_parent_commit,
2865- _, _) = self.override_parents(spi)
2866 else:
2867- # Get parent from publishing history (which is the last
2868- # published version in the corresponding series)
2869- try:
2870- unapplied_publish_parent_commit = str(unapplied_tip_head.peel().id)
2871- except AttributeError:
2872- try:
2873- unapplied_publish_parent_head = self.local_repo.get_head_by_name(spi.parent_head_name(self.namespace))
2874- unapplied_publish_parent_commit = str(unapplied_publish_parent_head.peel().id)
2875- except AttributeError:
2876- pass
2877- finally:
2878+ upload_parent_commit = str(upload_tag.peel().id)
2879+ if unapplied_publish_parent_commit is not None:
2880 try:
2881- unapplied_publish_parent_tag = self.local_repo.nearest_import_tag(unapplied_publish_parent_commit, self.namespace)
2882- logging.debug('Publishing parent (tag) is %s',
2883- self.local_repo.tag_to_pretty_name(unapplied_publish_parent_tag)
2884+ repo.git_run(['merge-base', '--is-ancestor',
2885+ unapplied_publish_parent_commit,
2886+ upload_parent_commit
2887+ ]
2888 )
2889- except AttributeError:
2890- pass
2891-
2892- if version_compare(str(spi.version), unapplied_tip_version) <= 0:
2893- logging.warn('Version to import (%s) is not after %s tip (%s)',
2894- spi.version, pretty_head_name, unapplied_tip_version
2895- )
2896-
2897- # Walk changelog backwards until we find an imported version
2898- for version in import_tree_versions:
2899- unapplied_changelog_parent_tag = self.local_repo.get_import_tag(version, self.namespace)
2900- if unapplied_changelog_parent_tag is not None:
2901- # sanity check that version from d/changelog of the
2902- # tagged parent matches ours
2903- parent_changelog_version, _ = \
2904- self.local_repo.get_changelog_versions_from_treeish(
2905- str(unapplied_changelog_parent_tag.peel(pygit2.Tree).id),
2906- )
2907- # if the parent was imported via a parent override
2908- # itself, assume it is valid without checking its
2909- # tree's debian/changelog, as it wasn't used
2910- if (version not in self.parent_overrides and
2911- parent_changelog_version != version
2912- ):
2913- logging.error('Found a tag corresponding to '
2914- 'parent version %s, but d/changelog '
2915- 'version (%s) differs. Will '
2916- 'orphan commit.' % (
2917- version,
2918- parent_changelog_version
2919- )
2920- )
2921- else:
2922- unapplied_changelog_parent_commit = str(unapplied_changelog_parent_tag.peel().id)
2923- logging.debug('Changelog parent (tag) is %s',
2924- self.local_repo.tag_to_pretty_name(unapplied_changelog_parent_tag)
2925- )
2926- break
2927-
2928- # If the two parents are tree-identical, then favor publication
2929- # history. This deals with copy-forwards between series and with
2930- # syncs.
2931- if self.local_repo.treeishs_identical(unapplied_publish_parent_commit, unapplied_changelog_parent_commit):
2932- unapplied_changelog_parent_commit = None
2933-
2934- # check if the version to import has already been uploaded to
2935- # this head -- we leverage the above code to perform a 'dry-run'
2936- # of the import tree, to obtain the (up to) two parents
2937- upload_tag = self.local_repo.get_upload_tag(spi.version, self.namespace)
2938- import_tag = self.local_repo.get_import_tag(spi.version, self.namespace)
2939- if upload_tag and not import_tag:
2940- if str(upload_tag.peel().tree.id) != unapplied_import_tree_hash:
2941- logging.warn('Found upload tag %s, '
2942- 'but the corresponding tree '
2943- 'does not match the published '
2944- 'version. Will import %s as '
2945- 'normal, ignoring the upload tag.',
2946- self.local_repo.tag_to_pretty_name(upload_tag),
2947- spi.version
2948- )
2949- else:
2950- upload_parent_commit = str(upload_tag.peel().id)
2951- if unapplied_publish_parent_commit is not None:
2952- try:
2953- self.local_repo.git_run(['merge-base', '--is-ancestor',
2954- unapplied_publish_parent_commit,
2955- upload_parent_commit
2956- ]
2957- )
2958- unapplied_publish_parent_commit = None
2959- except CalledProcessError as e:
2960- if e.returncode != 1:
2961- raise
2962-
2963- if unapplied_changelog_parent_commit is not None:
2964- try:
2965- self.local_repo.git_run(['merge-base', '--is-ancestor',
2966- unapplied_changelog_parent_commit,
2967- upload_parent_commit
2968- ]
2969- )
2970- unapplied_changelog_parent_commit = None
2971- except CalledProcessError as e:
2972- if e.returncode != 1:
2973- raise
2974-
2975- self.commit_unapplied_patches_import(
2976- spi,
2977- unapplied_import_tree_hash,
2978- unapplied_publish_parent_commit,
2979- unapplied_changelog_parent_commit,
2980- upload_parent_commit,
2981- )
2982-
2983- self.update_devel_branches(spi)
2984-
2985- def import_applied_spi(self, spi):
2986- """Imports a source package from Launchpad into the git
2987- repository
2988-
2989- Arguments:
2990- spi - a SourcePackageInformation instance
2991- """
2992- pretty_head_name = spi.pretty_head_name
2993-
2994- logging.info('Importing patches-applied %s to %s' % (spi.version, pretty_head_name))
2995- applied_tip_head = self.local_repo.get_head_by_name(spi.applied_head_name(self.namespace))
2996-
2997- spi.pull()
2998-
2999- self.local_repo.clean_repository_state()
3000-
3001- unapplied_parent_tag = self.local_repo.get_import_tag(spi.version, self.namespace)
3002- unapplied_parent_commit = str(unapplied_parent_tag.peel().id)
3003- unapplied_import_tree_hash = str(unapplied_parent_tag.peel(pygit2.Tree).id)
3004- logging.debug('Found patches-unapplied version %s as commit %s',
3005- str(spi.version), unapplied_parent_commit)
3006-
3007- import_tree_versions = self.local_repo.get_all_changelog_versions_from_treeish(
3008- unapplied_import_tree_hash
3009- )
3010- changelog_version = import_tree_versions[0]
3011-
3012- applied_tip_version = None
3013- # check if the version to import has already been imported to
3014- # this head
3015- if applied_tip_head is not None:
3016- if self.local_repo.treeishs_identical(
3017- unapplied_import_tree_hash, str(applied_tip_head.peel().id)
3018- ):
3019- logging.warn('%s is identical to %s',
3020- pretty_head_name, spi.version
3021- )
3022- return
3023- applied_tip_version, _ = self.local_repo.get_changelog_versions_from_treeish(
3024- str(applied_tip_head.peel().id),
3025- )
3026-
3027- logging.debug('Tip version is %s', applied_tip_version)
3028+ unapplied_publish_parent_commit = None
3029+ except CalledProcessError as e:
3030+ if e.returncode != 1:
3031+ raise
3032
3033- applied_publish_parent_commit = None
3034- applied_changelog_parent_commit = None
3035+ if unapplied_changelog_parent_commit is not None:
3036+ try:
3037+ repo.git_run(['merge-base', '--is-ancestor',
3038+ unapplied_changelog_parent_commit,
3039+ upload_parent_commit
3040+ ]
3041+ )
3042+ unapplied_changelog_parent_commit = None
3043+ except CalledProcessError as e:
3044+ if e.returncode != 1:
3045+ raise
3046
3047- if spi.version in self.parent_overrides:
3048- logging.debug('%s is specified in the parent override file.' %
3049- spi.version
3050+ commit_unapplied_patches_import(
3051+ repo,
3052+ spi,
3053+ unapplied_import_tree_hash,
3054+ namespace,
3055+ unapplied_publish_parent_commit,
3056+ unapplied_changelog_parent_commit,
3057+ upload_parent_commit,
3058+ )
3059+
3060+ update_devel_branches(
3061+ repo=repo,
3062+ namespace=namespace,
3063+ pkgname=spi.name,
3064+ ubuntu_sinfo=ubuntu_sinfo,
3065+ spi=spi,
3066+ )
3067+
3068+def import_applied_spi(repo, spi, namespace, ubuntu_sinfo):
3069+ """Imports a source package from Launchpad into the git
3070+ repository
3071+
3072+ Arguments:
3073+ spi - a SourcePackageInformation instance
3074+ """
3075+ pretty_head_name = spi.pretty_head_name
3076+
3077+ logging.info('Importing patches-applied %s to %s' % (spi.version, pretty_head_name))
3078+ applied_tip_head = repo.get_head_by_name(spi.applied_head_name(namespace))
3079+
3080+ spi.pull()
3081+
3082+ repo.clean_repository_state()
3083+
3084+ unapplied_parent_tag = repo.get_import_tag(spi.version, namespace)
3085+ unapplied_parent_commit = str(unapplied_parent_tag.peel().id)
3086+ unapplied_import_tree_hash = str(unapplied_parent_tag.peel(pygit2.Tree).id)
3087+ logging.debug('Found patches-unapplied version %s as commit %s',
3088+ str(spi.version), unapplied_parent_commit)
3089+
3090+ import_tree_versions = repo.get_all_changelog_versions_from_treeish(
3091+ unapplied_import_tree_hash
3092+ )
3093+ changelog_version = import_tree_versions[0]
3094+
3095+ applied_tip_version = None
3096+ # check if the version to import has already been imported to
3097+ # this head
3098+ if applied_tip_head is not None:
3099+ if repo.treeishs_identical(
3100+ unapplied_import_tree_hash, str(applied_tip_head.peel().id)
3101+ ):
3102+ logging.warn('%s is identical to %s',
3103+ pretty_head_name, spi.version
3104 )
3105+ return
3106+ applied_tip_version, _ = repo.get_changelog_versions_from_treeish(
3107+ str(applied_tip_head.peel().id),
3108+ )
3109
3110- (_, _,
3111- applied_publish_parent_commit,
3112- applied_changelog_parent_commit) = self.override_parents(spi)
3113- else:
3114- # Get parent from publishing history (which is the last
3115- # published version in the corresponding series)
3116- try:
3117- applied_publish_parent_commit = str(applied_tip_head.peel().id)
3118- except AttributeError:
3119- try:
3120- applied_publish_parent_head = self.local_repo.get_head_by_name(spi.applied_parent_head_name(self.namespace))
3121- applied_publish_parent_head_commit = str(applied_publish_parent_head.peel().id)
3122- except AttributeError:
3123- pass
3124-
3125- if version_compare(str(spi.version), applied_tip_version) <= 0:
3126- logging.warn('Version to import (%s) is not after %s tip (%s)',
3127- spi.version, pretty_head_name, applied_tip_version
3128- )
3129+ logging.debug('Tip version is %s', applied_tip_version)
3130
3131- # Walk changelog backwards until we find an imported version
3132- for version in import_tree_versions:
3133- applied_changelog_parent_tag = self.local_repo.get_applied_tag(version, self.namespace)
3134- if applied_changelog_parent_tag is not None:
3135- # sanity check that version from d/changelog of the
3136- # tagged parent matches ours
3137- parent_changelog_version, _ = \
3138- self.local_repo.get_changelog_versions_from_treeish(
3139- str(applied_changelog_parent_tag.peel(pygit2.Tree).id),
3140- )
3141- # if the parent was imported via a parent override
3142- # itself, assume it is valid without checking its
3143- # tree's debian/changelog, as it wasn't used
3144- if (version not in self.parent_overrides and
3145- parent_changelog_version != version
3146- ):
3147- logging.error('Found a tag corresponding to '
3148- 'parent version %s, but d/changelog '
3149- 'version (%s) differs. Will '
3150- 'orphan commit.' % (
3151- version,
3152- parent_changelog_version
3153- )
3154- )
3155- else:
3156- applied_changelog_parent_commit = str(applied_changelog_parent_tag.peel().id)
3157- logging.debug('Changelog parent (tag) is %s',
3158- self.local_repo.tag_to_pretty_name(applied_changelog_parent_tag)
3159- )
3160- break
3161-
3162- # If the two parents are tree-identical, then favor publication
3163- # history. This deals with copy-forwards between series and with
3164- # syncs.
3165- if self.local_repo.treeishs_identical(applied_publish_parent_commit, applied_changelog_parent_commit):
3166- applied_changelog_parent_commit = None
3167-
3168- # Assume no patches to apply
3169- applied_import_tree_hash = unapplied_import_tree_hash
3170- # get tree id from above commit
3171- for (applied_import_tree_hash, patch_name, patch_desc) in self.import_patches_applied_tree(spi.dsc_pathname):
3172- # special case for .pc removal
3173- if patch_name is None:
3174- msg = b'%b.' % (patch_desc.encode())
3175- else:
3176- if patch_desc is None:
3177- patch_desc = '%s\n\nNo DEP3 Subject or Description header found' % patch_name
3178- msg = b'%b\n\nGbp-Pq: %b.' % (patch_desc.encode(), patch_name.encode())
3179- unapplied_parent_commit = self.local_repo.commit_tree_hash(
3180- applied_import_tree_hash,
3181- [unapplied_parent_commit],
3182- msg,
3183- spi)
3184-
3185- logging.debug('Committed patch-application of %s as %s',
3186- patch_name, unapplied_parent_commit
3187- )
3188+ applied_publish_parent_commit = None
3189+ applied_changelog_parent_commit = None
3190
3191- self.commit_applied_patches_import(
3192- spi,
3193- applied_import_tree_hash,
3194+ if spi.version in _PARENT_OVERRIDES:
3195+ logging.debug('%s is specified in the parent override file.' %
3196+ spi.version
3197+ )
3198+
3199+ (
3200+ _,
3201+ _,
3202 applied_publish_parent_commit,
3203 applied_changelog_parent_commit,
3204- unapplied_parent_commit
3205- )
3206-
3207- self.update_applied_devel_branches(spi)
3208-
3209- def import_publishes(self, patches_applied,
3210- debian_head_versions, ubuntu_head_versions):
3211- history_found = False
3212- only_debian = False
3213- srcpkg_information = None
3214- if patches_applied:
3215- namespace = '%s/applied' % self.namespace
3216- import_type = 'patches-applied'
3217- import_func = self.import_applied_spi
3218- else:
3219- namespace = self.namespace
3220- import_type = 'patches-unapplied'
3221- import_func = self.import_unapplied_spi
3222- for distname, versions, dist_sinfo in (
3223- ("debian", debian_head_versions, self.debian_sinfo),
3224- ("ubuntu", ubuntu_head_versions, self.ubuntu_sinfo)):
3225- if self.active_series_only and distname == "debian":
3226- continue
3227- try:
3228- for srcpkg_information in dist_sinfo.launchpad_versions_published_after(
3229- versions,
3230- namespace,
3231- workdir=self.workdir,
3232- active_series_only=self.active_series_only
3233- ):
3234- history_found = True
3235- import_func(srcpkg_information)
3236- # Do not push upstream
3237- if self.fixup_devel and distname == "ubuntu":
3238- self.update_devel_branches(spi=None)
3239- except NoPublicationHistoryException:
3240- logging.warning("No publication history found for %s in %s. ",
3241- self.pkgname, distname)
3242- if distname == 'ubuntu':
3243- only_debian = True
3244- except Exception as e:
3245- if srcpkg_information is None:
3246- msg = 'Unable to import %s to %s' % (import_type, distname)
3247- else:
3248- msg = 'Unable to import %s %s to %s' % (import_type,
3249- str(srcpkg_information.version), distname)
3250- if not patches_applied:
3251- raise GitUbuntuImportError(msg) from e
3252- else:
3253- logging.error(msg)
3254- else:
3255- history_found = True
3256-
3257- return (only_debian, history_found)
3258-
3259- def main(self, args):
3260- self.pkgname = args.package
3261- owner = args.lp_owner
3262- no_clean = args.no_clean
3263- try:
3264- directory = args.directory
3265- no_clean = True
3266- except AttributeError:
3267- directory = None
3268- try:
3269- user = args.lp_user
3270- except AttributeError:
3271- user = None
3272- # --active-series-only/--no-fetch to a tmpdir/--skip-orig
3273- # implies --no-push
3274- no_push = (
3275- args.no_push or
3276- (args.no_fetch and not directory) or
3277- args.active_series_only or
3278- args.skip_orig
3279- )
3280+ ) = override_parents(
3281+ repo,
3282+ spi,
3283+ namespace,
3284+ )
3285+ else:
3286+ # Get parent from publishing history (which is the last
3287+ # published version in the corresponding series)
3288 try:
3289- dl_cache = args.dl_cache
3290+ applied_publish_parent_commit = str(applied_tip_head.peel().id)
3291 except AttributeError:
3292- dl_cache = None
3293- self.fixup_devel = args.fixup_devel
3294- self.active_series_only = args.active_series_only
3295- self.skip_orig = args.skip_orig
3296-
3297- if owner == 'usd-import-team':
3298- self.namespace = 'importer'
3299- else:
3300- self.namespace = owner
3301-
3302- logging.info('Ubuntu Server Team importer v%s' % VERSION)
3303-
3304- self.local_repo = GitUbuntuRepository(directory, user, args.proto)
3305- if not directory:
3306- logging.info('Using git repository at %s', self.local_repo.local_dir)
3307-
3308- atexit.register(self.cleanup, no_clean, self.local_repo.local_dir)
3309-
3310- self.local_repo.add_base_remotes(self.pkgname, repo_owner=owner)
3311- if not args.no_fetch and not args.reimport:
3312- try:
3313- self.local_repo.fetch_base_remotes()
3314- except GitUbuntuRepositoryFetchError:
3315- pass
3316-
3317- if not args.no_fetch:
3318- self.local_repo.delete_branches_in_namespace(self.namespace)
3319- self.local_repo.delete_tags_in_namespace(self.namespace)
3320- self.local_repo.copy_base_references(self.namespace)
3321-
3322- if not args.no_fetch and not args.reimport:
3323- try:
3324- self.local_repo.fetch_remote_refspecs('pkg',
3325- refspecs=['refs/tags/*:refs/tags/%s/*' % self.namespace],
3326- )
3327- except GitUbuntuRepositoryFetchError:
3328- pass
3329-
3330- if args.reimport:
3331- if args.no_fetch:
3332- logging.warning(
3333- "--reimport specified with --no-fetch. Upload "
3334- "tags would not be incorporated into the new import "
3335- "and would be lost forever. This is not currently "
3336- "supported and --no-fetch will be ignored for upload "
3337- "tags."
3338- )
3339 try:
3340- self.local_repo.fetch_remote_refspecs('pkg',
3341- refspecs=['refs/tags/upload/*:refs/tags/%s/upload/*' % self.namespace],
3342- )
3343- except GitUbuntuRepositoryFetchError:
3344+ applied_publish_parent_head = repo.get_head_by_name(spi.applied_parent_head_name(namespace))
3345+ applied_publish_parent_head_commit = str(applied_publish_parent_head.peel().id)
3346+ except AttributeError:
3347 pass
3348
3349- self.local_repo.ensure_importer_branches_exist(self.namespace)
3350+ if version_compare(str(spi.version), applied_tip_version) <= 0:
3351+ logging.warn('Version to import (%s) is not after %s tip (%s)',
3352+ spi.version, pretty_head_name, applied_tip_version
3353+ )
3354
3355- self.debian_sinfo = GitUbuntuSourceInformation('debian',
3356- self.pkgname, os.path.abspath(args.pullfile),
3357- args.retries, args.retry_backoffs)
3358+ # Walk changelog backwards until we find an imported version
3359+ for version in import_tree_versions:
3360+ applied_changelog_parent_tag = repo.get_applied_tag(version, namespace)
3361+ if applied_changelog_parent_tag is not None:
3362+ # sanity check that version from d/changelog of the
3363+ # tagged parent matches ours
3364+ parent_changelog_version, _ = \
3365+ repo.get_changelog_versions_from_treeish(
3366+ str(applied_changelog_parent_tag.peel(pygit2.Tree).id),
3367+ )
3368+ # if the parent was imported via a parent override
3369+ # itself, assume it is valid without checking its
3370+ # tree's debian/changelog, as it wasn't used
3371+ if (version not in _PARENT_OVERRIDES and
3372+ parent_changelog_version != version
3373+ ):
3374+ logging.error('Found a tag corresponding to '
3375+ 'parent version %s, but d/changelog '
3376+ 'version (%s) differs. Will '
3377+ 'orphan commit.' % (
3378+ version,
3379+ parent_changelog_version
3380+ )
3381+ )
3382+ else:
3383+ applied_changelog_parent_commit = str(applied_changelog_parent_tag.peel().id)
3384+ logging.debug('Changelog parent (tag) is %s',
3385+ repo.tag_to_pretty_name(applied_changelog_parent_tag)
3386+ )
3387+ break
3388
3389- self.ubuntu_sinfo = GitUbuntuSourceInformation('ubuntu',
3390- self.pkgname, os.path.abspath(args.pullfile),
3391- args.retries, args.retry_backoffs)
3392+ # If the two parents are tree-identical, then favor publication
3393+ # history. This deals with copy-forwards between series and with
3394+ # syncs.
3395+ if repo.treeishs_identical(applied_publish_parent_commit, applied_changelog_parent_commit):
3396+ applied_changelog_parent_commit = None
3397
3398- debian_head_versions = (
3399- self.local_repo.get_heads_and_versions('debian', self.namespace)
3400- )
3401- for head_name in sorted(debian_head_versions):
3402- _, _, pretty_head_name = head_name.partition('%s/' % self.namespace)
3403- logging.debug('Last imported %s version is %s',
3404- pretty_head_name,
3405- debian_head_versions[head_name]['version']
3406+ # Assume no patches to apply
3407+ applied_import_tree_hash = unapplied_import_tree_hash
3408+ # get tree id from above commit
3409+ for (
3410+ applied_import_tree_hash,
3411+ patch_name,
3412+ patch_desc
3413+ ) in import_patches_applied_tree(repo, spi.dsc_pathname):
3414+ # special case for .pc removal
3415+ if patch_name is None:
3416+ msg = b'%b.' % (patch_desc.encode())
3417+ else:
3418+ if patch_desc is None:
3419+ patch_desc = (
3420+ "%s\n\nNo DEP3 Subject or Description header found" %
3421+ patch_name
3422+ )
3423+ msg = b'%b\n\nGbp-Pq: %b.' % (
3424+ patch_desc.encode(),
3425+ patch_name.encode()
3426 )
3427+ unapplied_parent_commit = repo.commit_tree_hash(
3428+ applied_import_tree_hash,
3429+ [unapplied_parent_commit],
3430+ msg,
3431+ spi
3432+ )
3433
3434- ubuntu_head_versions = (
3435- self.local_repo.get_heads_and_versions('ubuntu', self.namespace)
3436+ logging.debug(
3437+ "Committed patch-application of %s as %s",
3438+ patch_name, unapplied_parent_commit
3439 )
3440- for head_name in sorted(ubuntu_head_versions):
3441- _, _, pretty_head_name = head_name.partition('%s/' % self.namespace)
3442- logging.debug('Last imported %s version is %s',
3443- pretty_head_name,
3444- ubuntu_head_versions[head_name]['version']
3445- )
3446
3447- if not args.skip_applied:
3448- applied_debian_head_versions = (
3449- self.local_repo.get_heads_and_versions(
3450- 'applied/debian', self.namespace
3451- )
3452- )
3453- for head_name in sorted(applied_debian_head_versions):
3454- _, _, pretty_head_name = head_name.partition(
3455- '%s/' % self.namespace
3456- )
3457- logging.debug('Last applied %s version is %s',
3458- pretty_head_name,
3459- applied_debian_head_versions[head_name]['version']
3460+ commit_applied_patches_import(
3461+ repo,
3462+ spi,
3463+ applied_import_tree_hash,
3464+ namespace,
3465+ applied_publish_parent_commit,
3466+ applied_changelog_parent_commit,
3467+ unapplied_parent_commit,
3468+ )
3469+
3470+ update_applied_devel_branches(
3471+ repo=repo,
3472+ pkgname=spi.name,
3473+ namespace=namespace,
3474+ ubuntu_sinfo=ubuntu_sinfo,
3475+ spi=spi,
3476+ )
3477+
3478+def import_publishes(repo, pkgname, namespace, patches_applied,
3479+ debian_head_versions, ubuntu_head_versions, debian_sinfo,
3480+ ubuntu_sinfo, active_series_only, workdir, fixup_devel,
3481+ skip_orig,
3482+):
3483+ history_found = False
3484+ only_debian = False
3485+ srcpkg_information = None
3486+ if patches_applied:
3487+ _namespace = namespace
3488+ namespace = '%s/applied' % namespace
3489+ import_type = 'patches-applied'
3490+ import_func = import_applied_spi
3491+ else:
3492+ _namespace = namespace
3493+ import_type = 'patches-unapplied'
3494+ import_func = functools.partial(
3495+ import_unapplied_spi,
3496+ skip_orig=skip_orig
3497+ )
3498+ for distname, versions, dist_sinfo in (
3499+ ("debian", debian_head_versions, debian_sinfo),
3500+ ("ubuntu", ubuntu_head_versions, ubuntu_sinfo)):
3501+ if active_series_only and distname == "debian":
3502+ continue
3503+ try:
3504+ for srcpkg_information in dist_sinfo.launchpad_versions_published_after(
3505+ versions,
3506+ namespace,
3507+ workdir=workdir,
3508+ active_series_only=active_series_only
3509+ ):
3510+ history_found = True
3511+ import_func(
3512+ repo=repo,
3513+ spi=srcpkg_information,
3514+ namespace=_namespace,
3515+ ubuntu_sinfo=ubuntu_sinfo,
3516 )
3517-
3518- applied_ubuntu_head_versions = (
3519- self.local_repo.get_heads_and_versions(
3520- 'applied/ubuntu', self.namespace
3521+ # Do not push upstream
3522+ if fixup_devel and distname == "ubuntu":
3523+ update_devel_branches(
3524+ repo=repo,
3525+ namespace=namespace,
3526+ pkgname=pkgname,
3527+ ubuntu_sinfo=ubuntu_sinfo,
3528+ spi=None,
3529 )
3530+ except NoPublicationHistoryException:
3531+ logging.warning("No publication history found for %s in %s.",
3532+ pkgname, distname
3533 )
3534- for head_name in sorted(applied_ubuntu_head_versions):
3535- _, _, pretty_head_name = head_name.partition(
3536- '%s/' % self.namespace
3537- )
3538- logging.debug('Last applied %s version is %s',
3539- pretty_head_name,
3540- applied_ubuntu_head_versions[head_name]['version']
3541- )
3542-
3543- oldcwd = os.getcwd()
3544- os.chdir(self.local_repo.local_dir)
3545-
3546- if dl_cache is None:
3547- self.workdir = os.path.join(self.local_repo.git_dir, CACHE_PATH)
3548- else:
3549- self.workdir = dl_cache
3550-
3551- os.makedirs(self.workdir, exist_ok=True)
3552-
3553- self.parse_parentfile(args.parentfile)
3554-
3555- only_debian, history_found = self.import_publishes(
3556- patches_applied=False,
3557- debian_head_versions=debian_head_versions,
3558- ubuntu_head_versions=ubuntu_head_versions)
3559-
3560- if not history_found:
3561- logging.error("No publication history for '%s' in debian or ubuntu."
3562- " Wrong source package name?", self.pkgname)
3563- sys.exit(1)
3564-
3565- if not args.skip_applied:
3566- self.import_publishes(patches_applied=True,
3567- debian_head_versions=applied_debian_head_versions,
3568- ubuntu_head_versions=applied_ubuntu_head_versions)
3569-
3570- os.chdir(oldcwd)
3571-
3572- self.local_repo.garbage_collect()
3573-
3574- if no_push:
3575- logging.info('Not pushing to remote as specified')
3576+ if distname == 'ubuntu':
3577+ only_debian = True
3578+ except Exception as e:
3579+ if srcpkg_information is None:
3580+ msg = 'Unable to import %s to %s' % (import_type, distname)
3581+ else:
3582+ msg = 'Unable to import %s %s to %s' % (import_type,
3583+ str(srcpkg_information.version), distname)
3584+ #if not patches_applied:
3585+ raise GitUbuntuImportError(msg) from e
3586+ #else:
3587+ # logging.error(msg)
3588 else:
3589- lp = launchpad_login_auth()
3590- repo_path = '~%s/ubuntu/+source/%s/+git/%s' % (
3591- owner,
3592- self.pkgname,
3593- self.pkgname,
3594- )
3595- lp_git_repo = lp.git_repositories.getByPath(path=repo_path)
3596- if args.reimport:
3597- if not lp_git_repo.canBeDeleted():
3598- logging.warning(
3599- "There are linked MPs to the pkg repository, "
3600- "which will be deleted:"
3601- )
3602- for mp in lp_git_repo.landing_candidates:
3603- logging.warning(
3604- "Adding comment to %s indicating reimport",
3605- mp.web_link
3606- )
3607- mp.createComment(
3608- content="%s has been reimported and this MP "
3609- "must be manually resubmitted." %
3610- self.pkgname,
3611- subject="%s has been reimported" % self.pkgname,
3612+ history_found = True
3613+
3614+ return (only_debian, history_found)
3615+
3616+
3617+def parse_args(subparsers=None, base_subparsers=None):
3618+ kwargs = dict(description='Update a launchpad git tree based upon '
3619+ 'the state of the Ubuntu and Debian archives',
3620+ formatter_class=argparse.ArgumentDefaultsHelpFormatter
3621+ )
3622+ if base_subparsers:
3623+ kwargs['parents'] = base_subparsers
3624+ if subparsers:
3625+ parser = subparsers.add_parser('import', **kwargs)
3626+ parser.set_defaults(func=cli_main)
3627+ else:
3628+ parser = argparse.ArgumentParser(**kwargs)
3629+ parser.add_argument('package', type=str,
3630+ help='Which package to update in the git tree')
3631+ parser.add_argument('-o', '--lp-owner', type=str,
3632+ help=argparse.SUPPRESS,
3633+ default='usd-import-team')
3634+ parser.add_argument('-l', '--lp-user', type=str,
3635+ help=argparse.SUPPRESS)
3636+ parser.add_argument('--dl-cache', type=str,
3637+ help=('Cache directory for downloads. Default is to '
3638+ 'put downloads in <directory>/.git/%s' %
3639+ CACHE_PATH
3640+ ),
3641+ default=argparse.SUPPRESS)
3642+ parser.add_argument('--no-fetch', action='store_true',
3643+ help='Do not fetch from the remote (DANGEROUS; '
3644+ 'only useful for debugging); implies '
3645+ '--no-push')
3646+ parser.add_argument('--no-push', action='store_true',
3647+ help='Do not push to the remote')
3648+ parser.add_argument('--no-clean', action='store_true',
3649+ help='Do not clean the temporary directory')
3650+ parser.add_argument('-d', '--directory', type=str,
3651+ help='Use git repository at specified location rather '
3652+ 'than a temporary directory (implies --no-clean). '
3653+ 'Will create the local repository if needed.',
3654+ default=argparse.SUPPRESS
3655 )
3656- # repo deletion will delete the MP, but hopefully
3657- # the above comment will be sufficient to explain
3658- # why
3659- lp_git_repo.lp_delete()
3660-
3661- self.local_repo.git_run([
3662- 'push', '--atomic', 'pkg',
3663- 'refs/heads/%s/*:refs/heads/*' % self.namespace,
3664- 'refs/tags/%s/*:refs/tags/*' % self.namespace,
3665- ])
3666- # update our reference
3667- lp_git_repo = lp.git_repositories.getByPath(path=repo_path)
3668- for i in range(args.retries):
3669- try:
3670- if only_debian:
3671- lp_git_repo.default_branch = 'refs/heads/debian/sid'
3672- else:
3673- lp_git_repo.default_branch = 'refs/heads/ubuntu/devel'
3674- lp_git_repo.lp_save()
3675- break
3676- except (NotFound, PreconditionFailed) as e:
3677- time.sleep(args.retry_backoffs[i])
3678- lp_git_repo.lp_refresh()
3679+ parser.add_argument('--fixup-devel', action='store_true',
3680+ help=argparse.SUPPRESS)
3681+ parser.add_argument('--active-series-only', action='store_true',
3682+ help='Do an import of only active Ubuntu series '
3683+ 'history, implies --no-push.')
3684+ parser.add_argument('--skip-applied', action='store_true',
3685+ help=argparse.SUPPRESS)
3686+ parser.add_argument('--skip-orig', action='store_true',
3687+ help=argparse.SUPPRESS)
3688+ parser.add_argument('--reimport', action='store_true',
3689+ help=argparse.SUPPRESS)
3690+ if not subparsers:
3691+ return parser.parse_args()
3692+ return 'import - %s' % kwargs['description']
3693+
3694+def cli_main(args):
3695+ no_clean = args.no_clean
3696+ try:
3697+ directory = args.directory
3698+ no_clean = True
3699+ except AttributeError:
3700+ directory = None
3701+ try:
3702+ user = args.lp_user
3703+ except AttributeError:
3704+ user = None
3705+ # --active-series-only/--no-fetch to a tmpdir/--skip-orig
3706+ # implies --no-push
3707+ no_push = (
3708+ args.no_push or
3709+ (args.no_fetch and not directory) or
3710+ args.active_series_only or
3711+ args.skip_orig
3712+ )
3713+ try:
3714+ dl_cache = args.dl_cache
3715+ except AttributeError:
3716+ dl_cache = None
3717+
3718+ main(
3719+ args.package,
3720+ args.lp_owner,
3721+ no_clean,
3722+ directory,
3723+ user,
3724+ args.proto,
3725+ args.no_fetch,
3726+ no_push,
3727+ dl_cache,
3728+ args.fixup_devel,
3729+ args.active_series_only,
3730+ args.skip_orig,
3731+ args.pullfile,
3732+ args.parentfile,
3733+ args.retries,
3734+ args.retry_backoffs,
3735+ args.skip_applied,
3736+ args.reimport,
3737+ )
3738diff --git a/gitubuntu/importlocal.py b/gitubuntu/importlocal.py
3739index bf1700e..c237551 100644
3740--- a/gitubuntu/importlocal.py
3741+++ b/gitubuntu/importlocal.py
3742@@ -2,15 +2,22 @@ import argparse
3743 import glob
3744 import logging
3745 import os
3746-import re
3747-from subprocess import CalledProcessError
3748 import sys
3749 import tempfile
3750 from gitubuntu.dsc import GitUbuntuDsc
3751 from gitubuntu.git_repository import GitUbuntuRepository, git_dep14_tag
3752-from gitubuntu.run import decode_binary, run, runq
3753+from gitubuntu.run import run, runq
3754 from gitubuntu.version import VERSION
3755-from gitubuntu.importer import GitUbuntuImport
3756+from gitubuntu.importer import (
3757+ parse_parentfile,
3758+ _PARENT_OVERRIDES,
3759+ import_orig,
3760+ import_dsc,
3761+ get_changelog_for_commit,
3762+ import_patches_unapplied_tree,
3763+ import_patches_applied_tree,
3764+)
3765+from gitubuntu.source_information import derive_source_from_series
3766 from gitubuntu.versioning import version_compare
3767 try:
3768 pkg = 'python3-pygit2'
3769@@ -20,264 +27,309 @@ except ImportError:
3770 sys.exit(1)
3771
3772
3773-class GitUbuntuImportLocal(GitUbuntuImport):
3774- def __init__(self):
3775- self.parent_overrides = None
3776-
3777- def parse_args(self, subparsers=None, base_subparsers=None):
3778- kwargs = dict(description='Import a DSC file locally',
3779- #usage='%(prog)s [options] -- ...',
3780- epilog='Unlike Launchpad imports, local imports '
3781- 'require specifying a namespace to uniquely '
3782- 'identify the source of the DSC file. This '
3783- 'means that if you want to tie history '
3784- 'together for several DSCs from the same '
3785- 'source, you must use the same namespace value.',
3786- formatter_class=argparse.ArgumentDefaultsHelpFormatter
3787- )
3788- if base_subparsers:
3789- kwargs['parents'] = base_subparsers
3790- if subparsers:
3791- parser = subparsers.add_parser('import-local', **kwargs)
3792- parser.set_defaults(func=self.main)
3793- else:
3794- parser = argparse.ArgumentParser(**kwargs)
3795- parser.add_argument('namespace',
3796- help='String namespace to use for tagging the imports')
3797- parser.add_argument('-d', '--directory', type=str,
3798- help='Use git repository at specified location rather '
3799- 'than a temporary directory. '
3800- 'Will create the local repository if needed.',
3801- default=argparse.SUPPRESS
3802- )
3803- parser.add_argument('dsc', type=str,
3804- help='Path to or URL of DSC file')
3805- parser.add_argument('--skip-applied', action='store_true',
3806- help=argparse.SUPPRESS)
3807- parser.add_argument('--skip-orig', action='store_true',
3808- help=argparse.SUPPRESS)
3809- if not subparsers:
3810- return parser.parse_args()
3811- return 'import-local - %s' % kwargs['description']
3812-
3813- def main(self, args):
3814- try:
3815- directory = args.directory
3816- no_clean = True
3817- except AttributeError:
3818- directory = None
3819- self.namespace = args.namespace
3820- self.dsc_path = args.dsc
3821- self.skip_applied = args.skip_applied
3822- self.skip_orig = args.skip_orig
3823-
3824- self.local_repo = GitUbuntuRepository(directory)
3825- if not directory:
3826- logging.info('Using git repository at %s', self.local_repo.local_dir)
3827-
3828- self.local_repo.ensure_importer_branches_exist(self.namespace)
3829- if len(self.local_repo.raw_repo.status()) != 0:
3830- logging.error('Working tree must be clean to continue.')
3831- run(['git', 'status'], stdout=None)
3832- sys.exit(1)
3833-
3834- if os.path.exists(self.dsc_path):
3835- self.dsc_path = os.path.abspath(self.dsc_path)
3836- else:
3837- dsc_dir = tempfile.mkdtemp()
3838-
3839- oldcwd = os.getcwd()
3840- os.chdir(dsc_dir)
3841-
3842- runq(['dget', '--quiet', '--allow-unauthenticated',
3843- '--download-only', self.dsc_path])
3844- dscs = glob.glob(os.path.join(dsc_dir, '*.dsc'))
3845- if len(dscs) == 0:
3846- logging.error('Unable to obtain DSC file.')
3847- sys.exit(1)
3848- if len(dscs) > 1:
3849- logging.error('Multiple DSCs found in temporary directory.')
3850- sys.exit(1)
3851- self.dsc_path = os.path.join(dsc_dir, dscs[0])
3852- os.chdir(oldcwd)
3853- logging.debug('Importing DSC at local path %s' % self.dsc_path)
3854- self.dsc = GitUbuntuDsc(self.dsc_path)
3855- if not self.dsc.verify():
3856- logging.error('Unable to verify orig tarball specified by DSC %s',
3857- self.dsc_path)
3858- sys.exit(1)
3859-
3860- self.pkgname = self.dsc['Source']
3861+def main(
3862+ directory,
3863+ namespace,
3864+ dsc_path,
3865+ skip_applied,
3866+ skip_orig,
3867+ parentfile,
3868+):
3869+ repo = GitUbuntuRepository(directory)
3870+ if not directory:
3871+ logging.info('Using git repository at %s', repo.local_dir)
3872+
3873+ repo.ensure_importer_branches_exist(namespace)
3874+ if len(repo.raw_repo.status()) != 0:
3875+ logging.error('Working tree must be clean to continue.')
3876+ run(['git', 'status'], stdout=None)
3877+ sys.exit(1)
3878+
3879+ if os.path.exists(dsc_path):
3880+ dsc_path = os.path.abspath(dsc_path)
3881+ else:
3882+ dsc_dir = tempfile.mkdtemp()
3883
3884 oldcwd = os.getcwd()
3885- os.chdir(self.local_repo.local_dir)
3886-
3887- self.local_repo.clean_repository_state()
3888+ os.chdir(dsc_dir)
3889
3890- self.parse_parentfile(args.parentfile)
3891-
3892- unapplied_import_tree_hash = self.import_patches_unapplied_tree(self.dsc_path)
3893-
3894- import_tree_versions = self.local_repo.get_all_changelog_versions_from_treeish(
3895- unapplied_import_tree_hash
3896- )
3897-
3898- changelog_version = import_tree_versions[0]
3899-
3900- logging.debug(
3901- 'Found changelog version %s in newly imported tree' %
3902- changelog_version
3903- )
3904-
3905- head = None
3906- head_version = None
3907- if not self.local_repo.raw_repo.head_is_unborn:
3908- head = self.local_repo.raw_repo.head
3909- # check if the version to import has already been imported to
3910- # this head
3911- if head is not None:
3912- head_version, _ = self.local_repo.get_changelog_versions_from_treeish(
3913- str(head.peel().id))
3914-
3915- logging.debug('Head version is %s', head_version)
3916-
3917- unapplied_changelog_parent_commit = None
3918- applied_changelog_parent_commit = None
3919-
3920- if version_compare(changelog_version, head_version) <= 0:
3921- logging.warn('Version to import (%s) is not after HEAD '
3922- 'version (%s), history may not make sense.',
3923- changelog_version, head_version
3924- )
3925-
3926- # Walk changelog backwards until we find an imported version
3927- for version in import_tree_versions:
3928- unapplied_tag_ref = '%s/import/%s' % (self.namespace, git_dep14_tag(version))
3929- applied_tag_ref = '%s/applied/%s' % (self.namespace, git_dep14_tag(version))
3930- unapplied_changelog_parent_tag = self.local_repo.get_tag_reference(unapplied_tag_ref)
3931- applied_changelog_parent_tag = self.local_repo.get_tag_reference(applied_tag_ref)
3932- if unapplied_changelog_parent_tag is not None:
3933- # sanity check that version from d/changelog of the
3934- # tagged parent matches ours
3935- parent_changelog_version, _ = \
3936- self.local_repo.get_changelog_versions_from_treeish(
3937- str(unapplied_changelog_parent_tag.peel(pygit2.Tree).id),
3938- )
3939- # if the parent was imported via a parent override
3940- # itself, assume it is valid without checking its
3941- # tree's debian/changelog, as it wasn't used
3942- if (version not in self.parent_overrides and
3943- parent_changelog_version != version
3944- ):
3945- logging.error('Found a tag corresponding to '
3946- 'parent version %s, but d/changelog '
3947- 'version (%s) differs. Will '
3948- 'orphan commit.' % (
3949- version,
3950- parent_changelog_version
3951- )
3952- )
3953- else:
3954- unapplied_changelog_parent_commit = str(unapplied_changelog_parent_tag.peel().id)
3955- if applied_changelog_parent_tag:
3956- applied_changelog_parent_commit = str(applied_changelog_parent_tag.peel().id)
3957- logging.debug('Changelog parent (tag) is %s',
3958- self.local_repo.tag_to_pretty_name(unapplied_changelog_parent_tag)
3959- )
3960- break
3961-
3962- distribution = self.local_repo.get_changelog_distribution_from_treeish(unapplied_import_tree_hash)
3963- # do lookup of series for dist on lp?
3964- dist = 'ubuntu'
3965- if distribution in ("stable", "testing", "unstable", "experimental"):
3966- dist = 'debian'
3967- self.import_dsc(self.dsc, changelog_version, dist)
3968- try:
3969- if not self.skip_orig:
3970- self.import_orig(self.dsc, changelog_version, dist)
3971- except Exception as e:
3972- logging.error('Unable to import orig tarball for %s: %s',
3973- changelog_version, e)
3974+ runq(['dget', '--quiet', '--allow-unauthenticated',
3975+ '--download-only', dsc_path])
3976+ dscs = glob.glob(os.path.join(dsc_dir, '*.dsc'))
3977+ if len(dscs) == 0:
3978+ logging.error('Unable to obtain DSC file.')
3979 sys.exit(1)
3980-
3981- msg = (b'Import unapplied version %b\n\nImported using git-ubuntu-import.' %
3982- (changelog_version.encode())
3983- )
3984-
3985- parents = []
3986-
3987- tag = None
3988- tag_ref = '%s/import/%s' % (self.namespace,
3989- git_dep14_tag(changelog_version))
3990- if self.local_repo.get_tag_reference(tag_ref) is None:
3991- # Not imported before
3992- tag = tag_ref
3993-
3994- if unapplied_changelog_parent_commit is not None:
3995- msg += b'\n'
3996- parents.append(unapplied_changelog_parent_commit)
3997- msg += b'\nChangelog parent: %b' % unapplied_changelog_parent_commit.encode()
3998-
3999- changelog_entry = self.get_changelog_for_commit(
4000- unapplied_import_tree_hash,
4001- None,
4002- unapplied_changelog_parent_commit
4003- )
4004-
4005- msg += b'%b' % changelog_entry
4006- unapplied_parent_commit = self.local_repo.commit_tree_hash(
4007- unapplied_import_tree_hash, parents, msg)
4008- if tag is None:
4009- logging.info('Committed unapplied import of %s as %s',
4010- changelog_version, unapplied_parent_commit)
4011- else:
4012- logging.debug('Creating tag %s pointing to %s', tag,
4013- unapplied_parent_commit)
4014- self.local_repo.git_run(['tag', '-a', '-m', 'git-ubuntu import v%s' % VERSION,
4015- tag, unapplied_parent_commit
4016- ]
4017- )
4018-
4019- if not args.skip_applied:
4020- # Assume no patches to apply
4021- applied_import_tree_hash = unapplied_import_tree_hash
4022- for (applied_import_tree_hash, patch_name, patch_desc) in self.import_patches_applied_tree(self.dsc_path):
4023- # special case for .pc removal
4024- if patch_name is None:
4025- msg = b'%b.' % (patch_desc.encode())
4026- else:
4027- if patch_desc is None:
4028- patch_desc = '%s\n\nNo DEP3 Subject or Description header found' % patch_name
4029- msg = b'%b\n\nGbp-Pq: %b.' % (patch_desc.encode(), patch_name.encode())
4030- unapplied_parent_commit = self.local_repo.commit_tree_hash(
4031- applied_import_tree_hash,
4032- [unapplied_parent_commit],
4033- msg)
4034-
4035- logging.debug('Committed patch-application of %s as %s',
4036- patch_name, unapplied_parent_commit
4037- )
4038-
4039- tag = None
4040- tag_ref = '%s/applied/%s' % (self.namespace,
4041- git_dep14_tag(changelog_version))
4042- if self.local_repo.get_tag_reference(tag_ref) is None:
4043- # Not imported before
4044- tag= tag_ref
4045-
4046- if tag is None:
4047- logging.info('Committed applied import of %s as %s',
4048- changelog_version, unapplied_parent_commit)
4049- else:
4050- logging.debug('Creating tag %s pointing to %s', tag,
4051- unapplied_parent_commit)
4052- self.local_repo.git_run(['tag', '-a', '-m', 'git-ubuntu import v%s' % VERSION,
4053- tag, unapplied_parent_commit
4054- ]
4055- )
4056-
4057- self.local_repo.clean_repository_state()
4058-
4059+ if len(dscs) > 1:
4060+ logging.error('Multiple DSCs found in temporary directory.')
4061+ sys.exit(1)
4062+ dsc_path = os.path.join(dsc_dir, dscs[0])
4063 os.chdir(oldcwd)
4064-
4065- self.local_repo.garbage_collect()
4066+ logging.debug('Importing DSC at local path %s', dsc_path)
4067+ dsc = GitUbuntuDsc(dsc_path)
4068+ if not dsc.verify():
4069+ logging.error(
4070+ 'Unable to verify orig tarball specified by DSC %s',
4071+ dsc_path,
4072+ )
4073+ sys.exit(1)
4074+
4075+ pkgname = dsc['Source']
4076+
4077+ oldcwd = os.getcwd()
4078+ os.chdir(repo.local_dir)
4079+
4080+ repo.clean_repository_state()
4081+
4082+ parse_parentfile(parentfile, pkgname)
4083+
4084+ unapplied_import_tree_hash = import_patches_unapplied_tree(repo, dsc_path)
4085+
4086+ import_tree_versions = repo.get_all_changelog_versions_from_treeish(
4087+ unapplied_import_tree_hash
4088+ )
4089+
4090+ changelog_version = import_tree_versions[0]
4091+
4092+ logging.debug(
4093+ 'Found changelog version %s in newly imported tree',
4094+ changelog_version
4095+ )
4096+
4097+ head = None
4098+ head_version = None
4099+ if not repo.raw_repo.head_is_unborn:
4100+ head = repo.raw_repo.head
4101+ # check if the version to import has already been imported to
4102+ # this head
4103+ if head is not None:
4104+ head_version, _ = repo.get_changelog_versions_from_treeish(
4105+ str(head.peel().id)
4106+ )
4107+
4108+ logging.debug('Head version is %s', head_version)
4109+
4110+ unapplied_changelog_parent_commit = None
4111+ applied_changelog_parent_commit = None
4112+
4113+ if version_compare(changelog_version, head_version) <= 0:
4114+ logging.warn('Version to import (%s) is not after HEAD '
4115+ 'version (%s), history may not make sense.',
4116+ changelog_version, head_version
4117+ )
4118+
4119+ # Walk changelog backwards until we find an imported version
4120+ for version in import_tree_versions:
4121+ unapplied_tag_ref = '%s/import/%s' % (namespace, git_dep14_tag(version))
4122+ applied_tag_ref = '%s/applied/%s' % (namespace, git_dep14_tag(version))
4123+ unapplied_changelog_parent_tag = repo.get_tag_reference(unapplied_tag_ref)
4124+ applied_changelog_parent_tag = repo.get_tag_reference(applied_tag_ref)
4125+ if unapplied_changelog_parent_tag is not None:
4126+ # sanity check that version from d/changelog of the
4127+ # tagged parent matches ours
4128+ parent_changelog_version, _ = \
4129+ repo.get_changelog_versions_from_treeish(
4130+ str(unapplied_changelog_parent_tag.peel(pygit2.Tree).id),
4131+ )
4132+ # if the parent was imported via a parent override
4133+ # itself, assume it is valid without checking its
4134+ # tree's debian/changelog, as it wasn't used
4135+ if (version not in _PARENT_OVERRIDES and
4136+ parent_changelog_version != version
4137+ ):
4138+ logging.error('Found a tag corresponding to '
4139+ 'parent version %s, but d/changelog '
4140+ 'version (%s) differs. Will '
4141+ 'orphan commit.',
4142+ version,
4143+ parent_changelog_version
4144+ )
4145+ else:
4146+ unapplied_changelog_parent_commit = str(
4147+ unapplied_changelog_parent_tag.peel().id
4148+ )
4149+ if applied_changelog_parent_tag:
4150+ applied_changelog_parent_commit = str(
4151+ applied_changelog_parent_tag.peel().id
4152+ )
4153+ logging.debug('Changelog parent (tag) is %s',
4154+ repo.tag_to_pretty_name(unapplied_changelog_parent_tag)
4155+ )
4156+ break
4157+
4158+ distribution = repo.get_changelog_distribution_from_treeish(
4159+ unapplied_import_tree_hash
4160+ )
4161+ dist = derive_source_from_series(distribution)
4162+ import_dsc(
4163+ repo=repo,
4164+ dsc=dsc,
4165+ namespace=namespace,
4166+ version=changelog_version,
4167+ dist=dist,
4168+ )
4169+ try:
4170+ if not skip_orig:
4171+ import_orig(
4172+ repo=repo,
4173+ dsc=dsc,
4174+ namespace=namespace,
4175+ version=changelog_version,
4176+ dist=dist,
4177+ )
4178+ except Exception as e:
4179+ logging.error('Unable to import orig tarball for %s: %s',
4180+ changelog_version, e)
4181+ raise
4182+ sys.exit(1)
4183+
4184+ msg = (
4185+ b'Import unapplied version %b\n\nImported using git-ubuntu-import.' %
4186+ (changelog_version.encode())
4187+ )
4188+
4189+ parents = []
4190+
4191+ tag = None
4192+ tag_ref = '%s/import/%s' % (
4193+ namespace,
4194+ git_dep14_tag(changelog_version)
4195+ )
4196+ if repo.get_tag_reference(tag_ref) is None:
4197+ # Not imported before
4198+ tag = tag_ref
4199+
4200+ if unapplied_changelog_parent_commit is not None:
4201+ msg += b'\n'
4202+ parents.append(unapplied_changelog_parent_commit)
4203+ msg += b'\nChangelog parent: %b' % unapplied_changelog_parent_commit.encode()
4204+
4205+ changelog_entry = get_changelog_for_commit(
4206+ repo=repo,
4207+ tree_hash=unapplied_import_tree_hash,
4208+ publish_parent_commit=None,
4209+ changelog_parent_commit=unapplied_changelog_parent_commit,
4210+ )
4211+
4212+ msg += b'%b' % changelog_entry
4213+ unapplied_parent_commit = repo.commit_tree_hash(
4214+ unapplied_import_tree_hash, parents, msg
4215+ )
4216+ if tag is None:
4217+ logging.info('Committed unapplied import of %s as %s',
4218+ changelog_version, unapplied_parent_commit)
4219+ else:
4220+ logging.debug('Creating tag %s pointing to %s', tag,
4221+ unapplied_parent_commit
4222+ )
4223+ repo.git_run(['tag', '-a', '-m', 'git-ubuntu import v%s' % VERSION,
4224+ tag, unapplied_parent_commit
4225+ ]
4226+ )
4227+
4228+ if not skip_applied:
4229+ # Assume no patches to apply
4230+ applied_import_tree_hash = unapplied_import_tree_hash
4231+ for (
4232+ applied_import_tree_hash,
4233+ patch_name,
4234+ patch_desc,
4235+ ) in import_patches_applied_tree(repo, dsc_path):
4236+ # special case for .pc removal
4237+ if patch_name is None:
4238+ msg = b'%b.' % (patch_desc.encode())
4239+ else:
4240+ if patch_desc is None:
4241+ patch_desc = '%s\n\nNo DEP3 Subject or Description header found' % patch_name
4242+ msg = b'%b\n\nGbp-Pq: %b.' % (patch_desc.encode(), patch_name.encode())
4243+ unapplied_parent_commit = repo.commit_tree_hash(
4244+ applied_import_tree_hash,
4245+ [unapplied_parent_commit],
4246+ msg)
4247+
4248+ logging.debug('Committed patch-application of %s as %s',
4249+ patch_name, unapplied_parent_commit
4250+ )
4251+
4252+ tag = None
4253+ tag_ref = '%s/applied/%s' % (namespace,
4254+ git_dep14_tag(changelog_version))
4255+ if repo.get_tag_reference(tag_ref) is None:
4256+ # Not imported before
4257+ tag = tag_ref
4258+
4259+ if tag is None:
4260+ logging.info('Committed applied import of %s as %s',
4261+ changelog_version,
4262+ unapplied_parent_commit
4263+ )
4264+ else:
4265+ logging.debug('Creating tag %s pointing to %s', tag,
4266+ unapplied_parent_commit
4267+ )
4268+ repo.git_run(
4269+ [
4270+ 'tag',
4271+ '-a',
4272+ '-m',
4273+ 'git-ubuntu import v%s' % VERSION,
4274+ tag,
4275+ unapplied_parent_commit,
4276+ ]
4277+ )
4278+
4279+ repo.clean_repository_state()
4280+
4281+ os.chdir(oldcwd)
4282+
4283+ repo.garbage_collect()
4284+
4285+def parse_args(subparsers=None, base_subparsers=None):
4286+ kwargs = dict(description='Import a DSC file locally',
4287+ #usage='%(prog)s [options] -- ...',
4288+ epilog='Unlike Launchpad imports, local imports '
4289+ 'require specifying a namespace to uniquely '
4290+ 'identify the source of the DSC file. This '
4291+ 'means that if you want to tie history '
4292+ 'together for several DSCs from the same '
4293+ 'source, you must use the same namespace value.',
4294+ formatter_class=argparse.ArgumentDefaultsHelpFormatter
4295+ )
4296+ if base_subparsers:
4297+ kwargs['parents'] = base_subparsers
4298+ if subparsers:
4299+ parser = subparsers.add_parser('import-local', **kwargs)
4300+ parser.set_defaults(func=cli_main)
4301+ else:
4302+ parser = argparse.ArgumentParser(**kwargs)
4303+ parser.add_argument('namespace',
4304+ help='String namespace to use for tagging the imports')
4305+ parser.add_argument('-d', '--directory', type=str,
4306+ help='Use git repository at specified location rather '
4307+ 'than a temporary directory. '
4308+ 'Will create the local repository if needed.',
4309+ default=argparse.SUPPRESS
4310+ )
4311+ parser.add_argument('dsc', type=str,
4312+ help='Path to or URL of DSC file')
4313+ parser.add_argument('--skip-applied', action='store_true',
4314+ help=argparse.SUPPRESS)
4315+ parser.add_argument('--skip-orig', action='store_true',
4316+ help=argparse.SUPPRESS)
4317+ if not subparsers:
4318+ return parser.parse_args()
4319+ return 'import-local - %s' % kwargs['description']
4320+
4321+def cli_main(args):
4322+ try:
4323+ directory = args.directory
4324+ except AttributeError:
4325+ directory = None
4326+
4327+ main(
4328+ directory,
4329+ args.namespace,
4330+ args.dsc,
4331+ args.skip_applied,
4332+ args.skip_orig,
4333+ args.parentfile,
4334+ )
4335diff --git a/gitubuntu/importppa.py b/gitubuntu/importppa.py
4336index a4a3ec1..b4790f1 100644
4337--- a/gitubuntu/importppa.py
4338+++ b/gitubuntu/importppa.py
4339@@ -10,8 +10,25 @@ from gitubuntu.git_repository import (
4340 GitUbuntuRepository,
4341 GitUbuntuRepositoryFetchError,
4342 )
4343-from gitubuntu.importer import GitUbuntuImport
4344-from gitubuntu.source_information import GitUbuntuSourceInformation, NoPublicationHistoryException, SourceExtractionException, launchpad_login_auth
4345+from gitubuntu.importer import (
4346+ cleanup,
4347+ parse_parentfile,
4348+ _PARENT_OVERRIDES,
4349+ import_orig,
4350+ import_dsc,
4351+ get_changelog_for_commit,
4352+ import_unapplied_spi,
4353+ import_applied_spi,
4354+ import_patches_unapplied_tree,
4355+ import_patches_applied_tree,
4356+)
4357+from gitubuntu.source_information import (
4358+ GitUbuntuSourceInformation,
4359+ NoPublicationHistoryException,
4360+ SourceExtractionException,
4361+ launchpad_login_auth,
4362+ GitUbuntuSourceInformation,
4363+)
4364 from gitubuntu.version import VERSION
4365 try:
4366 pkg = 'python3-pygit2'
4367@@ -23,181 +40,245 @@ except ImportError:
4368 sys.exit(1)
4369
4370
4371-class GitUbuntuImportPPA(GitUbuntuImport):
4372- def __init__(self):
4373- self.parent_overrides = None
4374-
4375- def parse_args(self, subparsers=None, base_subparsers=None):
4376- kwargs = dict(description='Update a launchpad git tree based upon '
4377- 'the state of a PPA archive',
4378- epilog='PPA must be specified as ppa:<name>',
4379- formatter_class=argparse.ArgumentDefaultsHelpFormatter
4380- )
4381- if base_subparsers:
4382- kwargs['parents'] = base_subparsers
4383- if subparsers:
4384- parser = subparsers.add_parser('import-ppa', **kwargs)
4385- parser.set_defaults(func=self.main)
4386- else:
4387- parser = argparse.ArgumentParser(**kwargs)
4388- parser.add_argument('ppa', type=str,
4389- help='Name of PPA to update in the git tree')
4390- parser.add_argument('package', type=str,
4391- help='Which package to update in the git tree')
4392- parser.add_argument('-o', '--lp-owner', type=str,
4393- help='Which launchpad user\'s tree to update',
4394- default='usd-import-team')
4395- parser.add_argument('-l', '--lp-user', type=str,
4396- help='Specify the Launchpad user to use',
4397- default=getpass.getuser())
4398- parser.add_argument('--dl-cache', type=str,
4399- help=('Cache directory for downloads. Default is to '
4400- 'put downloads in <directory>/.git/%s' %
4401- CACHE_PATH
4402- ),
4403- default=argparse.SUPPRESS)
4404- parser.add_argument('--no-fetch', action='store_true',
4405- help='Do not fetch from the remote (DANGEROUS: '
4406- 'only useful for debugging); implies '
4407- '--no-push')
4408- parser.add_argument('--no-push', action='store_true',
4409- help='Do not push to the remote (WARNING: '
4410- 'PPA imports are a work-in-progress and '
4411- 'pushing is currently disabled)',
4412- default=True)
4413- parser.add_argument('--no-clean', action='store_true',
4414- help='Do not clean the temporary directory')
4415- parser.add_argument('-d', '--directory', type=str,
4416- help='Use git repository at specified location rather '
4417- 'than a temporary directory (implies --no-clean). '
4418- 'Will create the local repository if needed.',
4419- default=argparse.SUPPRESS
4420- )
4421- if not subparsers:
4422- return parser.parse_args()
4423- return 'import-ppa - %s' % kwargs['description']
4424-
4425- def main(self, args):
4426- if re.match(r'ppa:\w+', args.ppa) is None:
4427- logging.error('Specified PPA (%s) is not in the format '
4428- 'ppa:<name>', args.ppa)
4429- sys.exit(1)
4430- ppa = args.ppa
4431- pkgname = args.package
4432- owner = args.lp_owner
4433- user = args.lp_user
4434- no_clean = args.no_clean
4435- try:
4436- directory = args.directory
4437- no_clean = True
4438- except AttributeError:
4439- directory = None
4440- no_push = True
4441- try:
4442- dl_cache = args.dl_cache
4443- except AttributeError:
4444- dl_cache = None
4445-
4446- # what other transformations?
4447- self.namespace = ppa.replace(':', '_')
4448-
4449- logging.info('Ubuntu Server Team importer v%s' % VERSION)
4450-
4451- self.local_repo = GitUbuntuRepository(directory, user, args.proto)
4452- if not directory:
4453- logging.info('Using git repository at %s', self.local_repo.local_dir)
4454- self.local_repo.ensure_importer_branches_exist(self.namespace)
4455-
4456- atexit.register(self.cleanup, no_clean, self.local_repo.local_dir)
4457-
4458- self.local_repo.add_remote(pkgname, owner, self.namespace, user)
4459- if not args.no_fetch:
4460+def main(
4461+ ppa,
4462+ pkgname,
4463+ directory,
4464+ owner,
4465+ user,
4466+ pullfile,
4467+ parentfile,
4468+ no_clean,
4469+ no_fetch,
4470+ no_push,
4471+ dl_cache,
4472+ proto,
4473+ retries,
4474+ retry_backoffs,
4475+ skip_orig,
4476+):
4477+ if re.match(r'ppa:\w+', ppa) is None:
4478+ logging.error(
4479+ "Specified PPA (%s) is not in the format ppa:<name>",
4480+ ppa,
4481+ )
4482+ sys.exit(1)
4483+ # what other transformations?
4484+ namespace = ppa.replace(':', '_')
4485+
4486+ logging.info('Ubuntu Server Team importer v%s' % VERSION)
4487+
4488+ repo = GitUbuntuRepository(directory, user, proto)
4489+ if not directory:
4490+ logging.info('Using git repository at %s', repo.local_dir)
4491+ repo.ensure_importer_branches_exist(namespace)
4492+
4493+ atexit.register(cleanup, no_clean, repo.local_dir)
4494+
4495+ # This seems terminally broken and I'm not sure what we should do
4496+ # here
4497+ #repo.add_remote(pkgname, owner, namespace)
4498+ #if not no_fetch:
4499+ # try:
4500+ # repo.fetch_remote(namespace)
4501+ # except GitUbuntuRepositoryFetchError:
4502+ # pass
4503+
4504+ source_information = GitUbuntuSourceInformation(
4505+ ppa,
4506+ pkgname,
4507+ os.path.abspath(pullfile),
4508+ retries,
4509+ retry_backoffs,
4510+ )
4511+
4512+ ubuntu_head_versions = repo.get_heads_and_versions(
4513+ 'ubuntu',
4514+ namespace,
4515+ )
4516+ for head_name in sorted(ubuntu_head_versions):
4517+ _, _, pretty_head_name = head_name.partition('%s/', namespace)
4518+ logging.debug('Last imported %s version is %s',
4519+ pretty_head_name,
4520+ ubuntu_head_versions[head_name]['version']
4521+ )
4522+
4523+ applied_ubuntu_head_versions = repo.get_heads_and_versions(
4524+ 'applied/ubuntu',
4525+ namespace,
4526+ )
4527+ for head_name in sorted(applied_ubuntu_head_versions):
4528+ _, _, pretty_head_name = head_name.partition('%s/', namespace)
4529+ logging.debug('Last applied %s version is %s',
4530+ pretty_head_name,
4531+ applied_ubuntu_head_versions[head_name]['version']
4532+ )
4533+
4534+ oldcwd = os.getcwd()
4535+ os.chdir(repo.local_dir)
4536+
4537+ if dl_cache is None:
4538+ workdir = os.path.join(repo.git_dir, CACHE_PATH)
4539+ else:
4540+ workdir = dl_cache
4541+
4542+ os.makedirs(workdir, exist_ok=True)
4543+
4544+ parse_parentfile(pkgname, parentfile)
4545+
4546+ history_found = False
4547+ try:
4548+ for srcpkg_information in \
4549+ source_information.launchpad_versions_published_after(
4550+ ubuntu_head_versions,
4551+ namespace,
4552+ workdir=workdir
4553+ ):
4554 try:
4555- self.local_repo.fetch_remote(self.namespace)
4556- except GitUbuntuRepositoryFetchError:
4557+ import_unapplied_spi(
4558+ repo=repo,
4559+ spi=srcpkg_information,
4560+ namespace=namespace,
4561+ ubuntu_sinfo=source_information,
4562+ skip_orig=skip_orig,
4563+ )
4564+ except DownloadError:
4565+ # it is non-fatal for a PPA to not have files for older
4566+ # publishes
4567 pass
4568+ history_found = True
4569+ except NoPublicationHistoryException:
4570+ logging.warning("No publication history found for %s in %s.",
4571+ pkgname,
4572+ ppa,
4573+ )
4574
4575- source_information = UbuntuSourceInformation(ppa, pkgname,
4576- os.path.abspath(args.pullfile),
4577- args.retries, args.retry_backoffs)
4578-
4579- ubuntu_head_versions = self.local_repo.get_heads_and_versions(
4580- 'ubuntu', self.namespace
4581+ if not history_found:
4582+ logging.error(
4583+ "No publication history for '%s' in %s. Wrong source package name?",
4584+ pkgname,
4585+ ppa,
4586 )
4587- for head_name in sorted(ubuntu_head_versions):
4588- _, _, pretty_head_name = head_name.partition('%s/', self.namespace)
4589- logging.debug('Last imported %s version is %s',
4590- pretty_head_name,
4591- ubuntu_head_versions[head_name]['version']
4592- )
4593-
4594- applied_ubuntu_head_versions = self.local_repo.get_heads_and_versions(
4595- 'applied/ubuntu', self.namespace
4596+ sys.exit(1)
4597+
4598+ try:
4599+ for srcpkg_information in \
4600+ source_information.launchpad_versions_published_after(
4601+ applied_ubuntu_head_versions,
4602+ '%s/applied' % namespace,
4603+ workdir=workdir
4604+ ):
4605+ try:
4606+ import_applied_spi(
4607+ repo=repo,
4608+ spi=srcpkg_information,
4609+ namespace=namespace,
4610+ ubuntu_sinfo=source_information,
4611+ )
4612+ except DownloadError:
4613+ # it is non-fatal for a PPA to not have files for older
4614+ # publishes
4615+ pass
4616+ except NoPublicationHistoryException:
4617+ logging.warning("No publication history found for %s in %s.",
4618+ pkgname, ppa
4619 )
4620- for head_name in sorted(applied_ubuntu_head_versions):
4621- _, _, pretty_head_name = head_name.partition('%s/', self.namespace)
4622- logging.debug('Last applied %s version is %s',
4623- pretty_head_name,
4624- applied_ubuntu_head_versions[head_name]['version']
4625- )
4626-
4627- oldcwd = os.getcwd()
4628- os.chdir(self.local_repo.local_dir)
4629-
4630- if dl_cache is None:
4631- workdir = os.path.join(self.local_repo.git_dir, CACHE_PATH)
4632- else:
4633- workdir = dl_cache
4634-
4635- os.makedirs(workdir, exist_ok=True)
4636-
4637- self.parse_parentfile(pkgname, args.parentfile)
4638-
4639- history_found = False
4640- try:
4641- for srcpkg_information in \
4642- source_information.launchpad_versions_published_after(ubuntu_head_versions,
4643- self.namespace,
4644- workdir=workdir):
4645- try:
4646- self.import_unapplied_spi(srcpkg_information)
4647- except DownloadError:
4648- # it is non-fatal for a PPA to not have files for older
4649- # publishes
4650- pass
4651- history_found = True
4652- except NoPublicationHistoryException:
4653- logging.warning("No publication history found for %s in %s. ",
4654- pkgname, distname)
4655-
4656- if not history_found:
4657- logging.error("No publication history for '%s' in %s. "
4658- "Wrong source package name?", pkgname, ppa)
4659- sys.exit(1)
4660-
4661- try:
4662- for srcpkg_information in \
4663- source_information.launchpad_versions_published_after(applied_ubuntu_head_versions,
4664- '%s/applied' % self.namespace,
4665- workdir=workdir):
4666- try:
4667- self.import_applied_spi(srcpkg_information)
4668- except DownloadError:
4669- # it is non-fatal for a PPA to not have files for older
4670- # publishes
4671- pass
4672- except NoPublicationHistoryException:
4673- logging.warning("No publication history found for %s in %s. ",
4674- pkgname, ppa)
4675- except:
4676- logging.error("Unable to import patches-applied to %s", ppa)
4677-
4678- os.chdir(oldcwd)
4679-
4680- self.local_repo.garbage_collect()
4681-
4682- if no_push:
4683- logging.info('Not pushing to remote as specified')
4684- else:
4685- # what are appropriate refspects to push and where?
4686- pass
4687+ except:
4688+ logging.error("Unable to import patches-applied to %s", ppa)
4689+
4690+ os.chdir(oldcwd)
4691+
4692+ repo.garbage_collect()
4693+
4694+ if no_push:
4695+ logging.info('Not pushing to remote as specified')
4696+ else:
4697+ # what are appropriate refspects to push and where?
4698+ pass
4699+
4700+
4701+def parse_args(subparsers=None, base_subparsers=None):
4702+ kwargs = dict(description='Update a launchpad git tree based upon '
4703+ 'the state of a PPA archive',
4704+ epilog='PPA must be specified as ppa:<name>',
4705+ formatter_class=argparse.ArgumentDefaultsHelpFormatter
4706+ )
4707+ if base_subparsers:
4708+ kwargs['parents'] = base_subparsers
4709+ if subparsers:
4710+ parser = subparsers.add_parser('import-ppa', **kwargs)
4711+ parser.set_defaults(func=cli_main)
4712+ else:
4713+ parser = argparse.ArgumentParser(**kwargs)
4714+ parser.add_argument('ppa', type=str,
4715+ help='Name of PPA to update in the git tree')
4716+ parser.add_argument('package', type=str,
4717+ help='Which package to update in the git tree')
4718+ parser.add_argument('-o', '--lp-owner', type=str,
4719+ help='Which launchpad user\'s tree to update',
4720+ default='usd-import-team')
4721+ parser.add_argument('-l', '--lp-user', type=str,
4722+ help='Specify the Launchpad user to use',
4723+ default=getpass.getuser())
4724+ parser.add_argument('--dl-cache', type=str,
4725+ help=('Cache directory for downloads. Default is to '
4726+ 'put downloads in <directory>/.git/%s' %
4727+ CACHE_PATH
4728+ ),
4729+ default=argparse.SUPPRESS)
4730+ parser.add_argument('--no-fetch', action='store_true',
4731+ help='Do not fetch from the remote (DANGEROUS: '
4732+ 'only useful for debugging); implies '
4733+ '--no-push')
4734+ parser.add_argument('--no-push', action='store_true',
4735+ help='Do not push to the remote (WARNING: '
4736+ 'PPA imports are a work-in-progress and '
4737+ 'pushing is currently disabled)',
4738+ default=True)
4739+ parser.add_argument('--no-clean', action='store_true',
4740+ help='Do not clean the temporary directory')
4741+ parser.add_argument('-d', '--directory', type=str,
4742+ help='Use git repository at specified location rather '
4743+ 'than a temporary directory (implies --no-clean). '
4744+ 'Will create the local repository if needed.',
4745+ default=argparse.SUPPRESS
4746+ )
4747+ parser.add_argument(
4748+ '--skip-orig',
4749+ action='store_true',
4750+ help=argparse.SUPPRESS,
4751+ )
4752+ if not subparsers:
4753+ return parser.parse_args()
4754+ return 'import-ppa - %s' % kwargs['description']
4755+
4756+def cli_main(args):
4757+ no_clean = args.no_clean
4758+ try:
4759+ directory = args.directory
4760+ no_clean = True
4761+ except AttributeError:
4762+ directory = None
4763+ no_push = True
4764+ try:
4765+ dl_cache = args.dl_cache
4766+ except AttributeError:
4767+ dl_cache = None
4768+
4769+ main(
4770+ args.ppa,
4771+ args.package,
4772+ directory,
4773+ args.lp_owner,
4774+ args.lp_user,
4775+ args.pullfile,
4776+ args.parentfile,
4777+ no_clean,
4778+ args.no_fetch,
4779+ no_push,
4780+ dl_cache,
4781+ args.proto,
4782+ args.retries,
4783+ args.retry_backoffs,
4784+ args.skip_orig,
4785+ )
4786diff --git a/gitubuntu/lint.py b/gitubuntu/lint.py
4787index 435e7f2..109eb13 100644
4788--- a/gitubuntu/lint.py
4789+++ b/gitubuntu/lint.py
4790@@ -23,10 +23,6 @@ except ImportError:
4791 logging.error("Is %s installed?", pkg)
4792 sys.exit(1)
4793
4794-__all__ = [
4795- 'GitUbuntuLint',
4796-]
4797-
4798 class LintNamespaceError(Exception):
4799 pass
4800
4801@@ -162,579 +158,622 @@ def test__derive_target_branch_string(same_remote_branch_names,
4802 )
4803 assert target_branch_string == expected
4804
4805-class GitUbuntuLint:
4806- def __init__(self):
4807- pass
4808
4809- def success(self, msg, *args):
4810- if self.verbose:
4811- print(msg % args)
4812+_verbose = False
4813+def success(msg, *args):
4814+ if _verbose:
4815+ print(msg % args)
4816
4817- def parse_args(self, subparsers=None, base_subparsers=None):
4818- kwargs = dict(
4819- description="Given a commitish, perform automated checks",
4820- formatter_class=argparse.RawTextHelpFormatter,
4821- epilog="An exit code of 0 indicates all checks passed",
4822- )
4823- if base_subparsers:
4824- kwargs["parents"] = base_subparsers
4825- if subparsers:
4826- parser = subparsers.add_parser("lint", **kwargs)
4827- parser.set_defaults(func=self.main)
4828- else:
4829- parser = argparse.ArgumentParser(**kwargs)
4830
4831- parser.add_argument("commitish", type=str, default="HEAD", nargs="?",
4832- help="Commitish to lint"
4833- )
4834- parser.add_argument("--lint-namespace", type=str,
4835- help="The git-ref prefix under which the branches and tags "
4836- "to lint can be found. If linting a remote-tracking "
4837- "branch, this will be default to the remote name. If "
4838- "linting a local branch, this will default to ''."
4839- )
4840- parser.add_argument("--target-branch", type=str,
4841- help="Target branch (typically a remote-tracking branch in pkg/) "
4842- "the current branch is against. This will be derived if "
4843- "possible."
4844- )
4845- parser.add_argument("-d", "--directory", type=str,
4846- help="Use git repository at specified location.",
4847- default=os.path.abspath(os.getcwd())
4848- )
4849- if not subparsers:
4850- return parser.parse_args()
4851- return "lint - %s" % kwargs["description"]
4852+def parse_args(subparsers=None, base_subparsers=None):
4853+ kwargs = dict(
4854+ description="Given a commitish, perform automated checks",
4855+ formatter_class=argparse.RawTextHelpFormatter,
4856+ epilog="An exit code of 0 indicates all checks passed",
4857+ )
4858+ if base_subparsers:
4859+ kwargs["parents"] = base_subparsers
4860+ if subparsers:
4861+ parser = subparsers.add_parser("lint", **kwargs)
4862+ parser.set_defaults(func=cli_main)
4863+ else:
4864+ parser = argparse.ArgumentParser(**kwargs)
4865
4866- def _check_commitish_exists(self, commitish):
4867- try:
4868- actual_commitish_obj = self.local_repo.get_commitish(
4869- commitish
4870- )
4871- self.success("Verified %s exists", commitish)
4872- return True
4873- except KeyError:
4874- warning("%s not found", commitish)
4875- return False
4876+ parser.add_argument("commitish", type=str, default="HEAD", nargs="?",
4877+ help="Commitish to lint"
4878+ )
4879+ parser.add_argument("--lint-namespace", type=str,
4880+ help="The git-ref prefix under which the branches and tags "
4881+ "to lint can be found. If linting a remote-tracking "
4882+ "branch, this will be default to the remote name. If "
4883+ "linting a local branch, this will default to ''."
4884+ )
4885+ parser.add_argument("--target-branch", type=str,
4886+ help="Target branch (typically a remote-tracking branch in pkg/) "
4887+ "the current branch is against. This will be derived if "
4888+ "possible."
4889+ )
4890+ parser.add_argument("-d", "--directory", type=str,
4891+ help="Use git repository at specified location.",
4892+ default=os.path.abspath(os.getcwd())
4893+ )
4894+ if not subparsers:
4895+ return parser.parse_args()
4896+ return "lint - %s" % kwargs["description"]
4897+
4898+def cli_main(args):
4899+ main(
4900+ args.directory,
4901+ args.commitish,
4902+ args.lint_namespace,
4903+ args.target_branch,
4904+ args.verbose,
4905+ )
4906
4907- def _check_commitish_exists_and_same(self, expected_commitish_pretty_name,
4908- actual_commitish, expected_commitish, check_commit_id=False
4909- ):
4910- """Check @actual_commitish exists ad is the same tree as
4911- @expected_commitish
4912-
4913- Arguments:
4914- expected_commitish_pretty_name - a pretty name to print to refer to the
4915- expected value
4916- actual_commitish - commitish to check for
4917- expected_commitish - commitish to check against
4918- check_commit_id - if True, check the commit hashes match as well
4919- """
4920- ret = True
4921- if not self._check_commitish_exists(actual_commitish):
4922- return False
4923+def _check_commitish_exists(repo, commitish):
4924+ try:
4925+ actual_commitish_obj = repo.get_commitish(commitish)
4926+ success("Verified %s exists", commitish)
4927+ return True
4928+ except KeyError:
4929+ warning("%s not found", commitish)
4930+ return False
4931
4932- try:
4933- expected_commitish_obj = self.local_repo.get_commitish(
4934- expected_commitish
4935- )
4936- except KeyError:
4937- fatal("Unable to find expected commit: %s", expected_commitish)
4938+def _check_commitish_exists_and_same(repo, expected_commitish_pretty_name,
4939+ actual_commitish, expected_commitish, check_commit_id=False
4940+):
4941+ """Check @actual_commitish exists ad is the same tree as
4942+ @expected_commitish
4943+
4944+ Arguments:
4945+ expected_commitish_pretty_name - a pretty name to print to refer to the
4946+ expected value
4947+ actual_commitish - commitish to check for
4948+ expected_commitish - commitish to check against
4949+ check_commit_id - if True, check the commit hashes match as well
4950+ """
4951+ ret = True
4952+ if not _check_commitish_exists(repo, actual_commitish):
4953+ return False
4954
4955- actual_commitish_obj = self.local_repo.get_commitish(
4956- actual_commitish
4957- )
4958- if check_commit_id:
4959- expected_commit_id = str(expected_commitish_obj.id)
4960- actual_commit_id = str(actual_commitish_obj.id)
4961- if expected_commit_id == actual_commit_id:
4962- self.success("Verified %s is the same commit as %s",
4963- expected_commitish_pretty_name, actual_commitish
4964- )
4965- else:
4966- warning("Expected %s (%s) is not the same commit as %s (%s)",
4967- expected_commitish_pretty_name, expected_commit_id,
4968- actual_commitish, actual_commit_id
4969- )
4970- ret = False
4971- expected_commit_tree_id = str(
4972- expected_commitish_obj.peel(pygit2.Tree).id
4973- )
4974- actual_commit_tree_id = str(
4975- actual_commitish_obj.peel(pygit2.Tree).id
4976- )
4977- if expected_commit_tree_id == actual_commit_tree_id:
4978- self.success("Verified %s is the same tree as %s",
4979+ try:
4980+ expected_commitish_obj = repo.get_commitish(expected_commitish)
4981+ except KeyError:
4982+ fatal("Unable to find expected commit: %s", expected_commitish)
4983+
4984+ actual_commitish_obj = repo.get_commitish(actual_commitish)
4985+ if check_commit_id:
4986+ expected_commit_id = str(expected_commitish_obj.id)
4987+ actual_commit_id = str(actual_commitish_obj.id)
4988+ if expected_commit_id == actual_commit_id:
4989+ success("Verified %s is the same commit as %s",
4990 expected_commitish_pretty_name, actual_commitish
4991 )
4992 else:
4993- error("Expected %s (%s) is not the same tree as %s (%s)",
4994- expected_commitish_pretty_name, expected_commit_tree_id,
4995- actual_commitish, actual_commit_tree_id
4996+ warning("Expected %s (%s) is not the same commit as %s (%s)",
4997+ expected_commitish_pretty_name, expected_commit_id,
4998+ actual_commitish, actual_commit_id
4999 )
5000 ret = False
The diff has been truncated for viewing.

Subscribers

People subscribed via source and target branches