Merge ~nacc/git-ubuntu:gu-review into git-ubuntu:master

Proposed by Nish Aravamudan
Status: Superseded
Proposed branch: ~nacc/git-ubuntu:gu-review
Merge into: git-ubuntu:master
Diff against target: 8362 lines (+4219/-3553)
15 files modified
gitubuntu/__main__.py (+142/-123)
gitubuntu/build.py (+132/-129)
gitubuntu/buildsource.py (+55/-59)
gitubuntu/clone.py (+109/-105)
gitubuntu/importer.py (+1388/-1176)
gitubuntu/importlocal.py (+312/-260)
gitubuntu/importppa.py (+256/-175)
gitubuntu/lint.py (+529/-506)
gitubuntu/merge.py (+423/-355)
gitubuntu/queue.py (+299/-281)
gitubuntu/remote.py (+120/-97)
gitubuntu/review.py (+117/-0)
gitubuntu/source_information.py (+15/-0)
gitubuntu/submit.py (+190/-174)
gitubuntu/tag.py (+132/-113)
Reviewer Review Type Date Requested Status
Robie Basak Needs Fixing
Review via email: mp+328203@code.launchpad.net

This proposal has been superseded by a proposal from 2017-09-06.

Description of the change

This is probably not perfect, but it's basically (right now) just a wrapper around other commands.

To post a comment you must log in.
Revision history for this message
Robie Basak (racb) :
review: Needs Fixing
~nacc/git-ubuntu:gu-review updated
00c4243... by Nish Aravamudan

build/buildsource: refactor class

Leave only parse_args and cli_main in the class. Make cli_main nothing
more than a wrapper to the module's main function.

LP: #1707321

9f2c0f3... by Nish Aravamudan

clone: refactor class

Leave only parse_args and cli_main in the class. Make cli_main nothing
more than a wrapper to the module's main function.

LP: #1707321

980b55b... by Nish Aravamudan

import: refactor class

Leave only parse_args and cli_main in the class. Make cli_main nothing
more than a wrapper to the module's main function.

LP: #1707321

ab02d0a... by Nish Aravamudan

importlocal: refactor class

Leave only parse_args and cli_main in the class. Make cli_main nothing
more than a wrapper to the module's main function.

LP: #1707321

d37db34... by Nish Aravamudan

git_repository: move derive_source_from_series from build

It will be used by multiple modules, move it to common code.

17d1c2d... by Nish Aravamudan

import: refactor import_orig

Rather than passing a spi in, pass the parts of the spi we immediately
derive. This allows the function to be re-used by importppa.

dc060d7... by Nish Aravamudan

source_information: raise ValueError if no person can be found

When we do PPA lookups, it's possible we are unable to find the supposed
PPA owner. Rather than failing in launchpadlib later, raise a ValueError
as the PPA is invalid.

3909203... by Nish Aravamudan

importppa: refactor class

Leave only parse_args and cli_main in the class. Make cli_main nothing
more than a wrapper to the module's main function.

LP: #1707321

02928c7... by Nish Aravamudan

lint: refactor class

Leave only parse_args and cli_main in the class. Make cli_main nothing
more than a wrapper to the module's main function.

LP: #1707321

7b56298... by Nish Aravamudan

merge: refactor class

Leave only parse_args and cli_main in the class. Make cli_main nothing
more than a wrapper to the module's main function.

LP: #1707321

83da23b... by Nish Aravamudan

queue: refactor class

Leave only parse_args and cli_main in the class. Make cli_main nothing
more than a wrapper to the module's main function.

LP: #1707321

1eaa251... by Nish Aravamudan

remote: refactor class

Leave only parse_args and cli_main in the class. Make cli_main nothing
more than a wrapper to the module's main function.

LP: #1707321

a0c296d... by Nish Aravamudan

submit: use lint.derive_target_branch

submit can be used for distributed development among team members by
proposing MPs against each other's repositories, via the --target-user
flag. Rather than having submit-specific code for deriving the target
branch, use the existing code from lint, after extending it to take a
namespace argument.

b1d970f... by Nish Aravamudan

tag: refactor class

Leave only parse_args and cli_main in the class. Make cli_main nothing
more than a wrapper to the module's main function.

LP: #1707321

cfcdc7a... by Nish Aravamudan

submit: drop unused pullfile member variable

2937467... by Nish Aravamudan

submit: fix implementation of --branch

We currently default to using HEAD, but allow the user to specify a
different branch to submit. However, we use HEAD in a few places
anyways, even if the user did specify --branch, which is incorrect.

cdc1cb0... by Nish Aravamudan

submit: refactor class

Leave only parse_args and cli_main in the class. Make cli_main nothing
more than a wrapper to the module's main function.

LP: #1707321

b3ae5b8... by Nish Aravamudan

submit: fix stray tab

877e70f... by Nish Aravamudan

git ubuntu: refactor main to call module parse_args

This is the final step in the refactor -- rather than using per-module
classes, just import each module in __main__ in order to call its
parse_args and func (which in the current implementation is that
module's cli_main).

Before merging with master, I think we will squash all of these
together.

At this point, all errors from pylint3 are from pygit2 imports.

LP: #1707321

f39ea92... by Nish Aravamudan

build: fix use of undefined variables

6591602... by Nish Aravamudan

lint: fix use of head_is_detached

9def1cf... by Nish Aravamudan

importppa: fix use of undefined variables

a54e9ee... by Nish Aravamudan

lint: drop unused pullfile variable

dbfe7bc... by Nish Aravamudan

clone: have main return the resulting directory path

16cab88... by Nish Aravamudan

clone: fix whitespace issue

e2e5cce... by Nish Aravamudan

lint: linting historical merges with default arguments can fail

Typically, this is because pkg/debian/sid now is not the same as
pkg/debian/sid when the MP was proposed. This can lead to all sorts of
oddness, but should not be fatal to lint.

One specific case is when the to-be-linted branch is older than the
target branch (e.g., a missed MP) and thus it only contains changelog
deletions.

Unmerged commits

e0831bb... by Nish Aravamudan

git ubuntu review: add new subcommand

Given a MP URL, this attempts to provide a local directory to use for
reviewing the MP (and allows one to auto-approve with lint results).

LP: #1702960

16cab88... by Nish Aravamudan

clone: fix whitespace issue

e2e5cce... by Nish Aravamudan

lint: linting historical merges with default arguments can fail

Typically, this is because pkg/debian/sid now is not the same as
pkg/debian/sid when the MP was proposed. This can lead to all sorts of
oddness, but should not be fatal to lint.

One specific case is when the to-be-linted branch is older than the
target branch (e.g., a missed MP) and thus it only contains changelog
deletions.

dbfe7bc... by Nish Aravamudan

clone: have main return the resulting directory path

a54e9ee... by Nish Aravamudan

lint: drop unused pullfile variable

877e70f... by Nish Aravamudan

git ubuntu: refactor main to call module parse_args

This is the final step in the refactor -- rather than using per-module
classes, just import each module in __main__ in order to call its
parse_args and func (which in the current implementation is that
module's cli_main).

Before merging with master, I think we will squash all of these
together.

At this point, all errors from pylint3 are from pygit2 imports.

LP: #1707321

cdc1cb0... by Nish Aravamudan

submit: refactor class

Leave only parse_args and cli_main in the class. Make cli_main nothing
more than a wrapper to the module's main function.

LP: #1707321

a0c296d... by Nish Aravamudan

submit: use lint.derive_target_branch

submit can be used for distributed development among team members by
proposing MPs against each other's repositories, via the --target-user
flag. Rather than having submit-specific code for deriving the target
branch, use the existing code from lint, after extending it to take a
namespace argument.

b1d970f... by Nish Aravamudan

tag: refactor class

Leave only parse_args and cli_main in the class. Make cli_main nothing
more than a wrapper to the module's main function.

LP: #1707321

1eaa251... by Nish Aravamudan

remote: refactor class

Leave only parse_args and cli_main in the class. Make cli_main nothing
more than a wrapper to the module's main function.

LP: #1707321

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1diff --git a/gitubuntu/__main__.py b/gitubuntu/__main__.py
2index ba4a435..ea2de9f 100644
3--- a/gitubuntu/__main__.py
4+++ b/gitubuntu/__main__.py
5@@ -3,24 +3,13 @@
6 def main():
7 try:
8 import argparse
9+ from importlib import import_module
10 import logging
11 from os import isatty
12 import shutil
13 from subprocess import CalledProcessError
14 import sys
15 import textwrap
16- from gitubuntu.importer import GitUbuntuImport
17- from gitubuntu.build import GitUbuntuBuild
18- from gitubuntu.buildsource import GitUbuntuBuildSource
19- from gitubuntu.merge import GitUbuntuMerge
20- from gitubuntu.clone import GitUbuntuClone
21- from gitubuntu.tag import GitUbuntuTag
22- from gitubuntu.queue import GitUbuntuQueue
23- from gitubuntu.importlocal import GitUbuntuImportLocal
24- from gitubuntu.importppa import GitUbuntuImportPPA
25- from gitubuntu.remote import GitUbuntuRemote
26- from gitubuntu.submit import GitUbuntuSubmit
27- from gitubuntu.lint import GitUbuntuLint
28 from gitubuntu.run import run
29
30 try:
31@@ -34,123 +23,149 @@ def main():
32
33 logging.getLogger('keyring').setLevel(logging.WARNING)
34
35- known_subcommands = {'import':GitUbuntuImport,
36- 'build':GitUbuntuBuild,
37- 'build-source':GitUbuntuBuildSource,
38- 'merge':GitUbuntuMerge,
39- 'clone':GitUbuntuClone,
40- 'tag':GitUbuntuTag,
41- 'queue':GitUbuntuQueue,
42- 'import-local':GitUbuntuImportLocal,
43- 'import-ppa':GitUbuntuImportPPA,
44- 'remote':GitUbuntuRemote,
45- 'submit':GitUbuntuSubmit,
46- 'lint':GitUbuntuLint,
47- }
48+ known_subcommands = {
49+ 'import':'gitubuntu.importer',
50+ 'build':'gitubuntu.build',
51+ 'build-source':'gitubuntu.buildsource',
52+ 'merge':'gitubuntu.merge',
53+ 'clone':'gitubuntu.clone',
54+ 'tag':'gitubuntu.tag',
55+ 'queue':'gitubuntu.queue',
56+ 'import-local':'gitubuntu.importlocal',
57+ 'import-ppa':'gitubuntu.importppa',
58+ 'remote':'gitubuntu.remote',
59+ 'submit':'gitubuntu.submit',
60+ 'lint':'gitubuntu.lint',
61+ 'review':'gitubuntu.review',
62+ }
63
64- known_network_subcommands = {'import', 'import-local',
65- 'import-ppa', 'clone', 'build',
66- 'build-source', 'queue',
67- 'remote', 'submit'}
68+ known_network_subcommands = {
69+ 'import',
70+ 'import-local',
71+ 'import-ppa',
72+ 'clone',
73+ 'build',
74+ 'build-source',
75+ 'queue',
76+ 'remote',
77+ 'submit',
78+ 'review',
79+ }
80
81 parser = argparse.ArgumentParser(
82 description='Ubuntu git development tool',
83 formatter_class=argparse.RawTextHelpFormatter,
84- epilog='More information is available at https://wiki.ubuntu.com/UbuntuDevelopment/Merging/GitWorkflow.'
85- )
86- subparsers = parser.add_subparsers(dest='subcommand',
87- help='',
88- metavar='%s' % '|'.join(sorted(known_subcommands))
89- )
90+ epilog='More information is available at https://wiki.ubuntu.com/UbuntuDevelopment/Merging/GitWorkflow.',
91+ )
92+ subparsers = parser.add_subparsers(
93+ dest='subcommand',
94+ help='',
95+ metavar='%s' % '|'.join(sorted(known_subcommands)),
96+ )
97 # This help ends up not being very useful to the end user
98 # subparsers.required = True
99
100 # common flags to all subcommands
101 base_subparser = argparse.ArgumentParser(add_help=False)
102- base_subparser.add_argument('-v', '--verbose', action='store_true',
103- help='Increase verbosity')
104+ base_subparser.add_argument(
105+ '-v', '--verbose',
106+ action='store_true',
107+ help='Increase verbosity'
108+ )
109 network_base_subparser = argparse.ArgumentParser(add_help=False)
110 network_base_subparser.add_argument(
111- '--retries', type=int,
112- help='Number of times to attempt to retry '
113- 'downloading from Launchpad.',
114- default=3
115- )
116+ '--retries',
117+ type=int,
118+ help='Number of times to attempt to retry '
119+ 'downloading from Launchpad.',
120+ default=3,
121+ )
122 network_base_subparser.add_argument(
123- '--retry-backoffs',
124- type=lambda s : [int(item) for item in s.split(',')],
125- help='Comma-separated list of backoffs in '
126- 'seconds to use between each retry '
127- 'attempt. Default is exponential backoff.',
128- default=argparse.SUPPRESS
129- )
130+ '--retry-backoffs',
131+ type=lambda s : [int(item) for item in s.split(',')],
132+ help='Comma-separated list of backoffs in '
133+ 'seconds to use between each retry '
134+ 'attempt. Default is exponential backoff.',
135+ default=argparse.SUPPRESS,
136+ )
137 known_protos = ['git', 'http', 'https']
138 network_base_subparser.add_argument(
139- '--proto',
140- metavar='[%s]' % '|'.join(known_protos),
141- choices=known_protos,
142- help='Specify protocol to use for fetch. Default: %(default)s',
143- default='git'
144- )
145+ '--proto',
146+ metavar='[%s]' % '|'.join(known_protos),
147+ choices=known_protos,
148+ help='Specify protocol to use for fetch. Default: %(default)s',
149+ default='git',
150+ )
151
152 width, _ = shutil.get_terminal_size()
153 subhelp_width = width - 30
154 for sub in sorted(known_subcommands):
155- s = known_subcommands[sub]()
156+ m = import_module(known_subcommands[sub])
157 if sub in known_network_subcommands:
158- help_text = s.parse_args(subparsers, [base_subparser, network_base_subparser])
159+ help_text = m.parse_args(
160+ subparsers,
161+ [base_subparser, network_base_subparser],
162+ )
163 else:
164- help_text = s.parse_args(subparsers, [base_subparser])
165+ help_text = m.parse_args(subparsers, [base_subparser])
166 subparsers.help += '\n'
167- subparsers.help += '\n'.join(textwrap.wrap(help_text,
168- subhelp_width,
169- break_on_hyphens=False
170- )
171- )
172- help_text = '\n'.join(textwrap.wrap('File specifying '
173- 'parent-child relationship '
174- 'overrides as:',
175- subhelp_width
176- )
177- )
178+ subparsers.help += '\n'.join(
179+ textwrap.wrap(
180+ help_text,
181+ subhelp_width,
182+ break_on_hyphens=False,
183+ )
184+ )
185+ help_text = '\n'.join(
186+ textwrap.wrap(
187+ "File specifying parent-child relationship overrides as:",
188+ subhelp_width,
189+ )
190+ )
191 help_text += '\n'
192 help_text += '<pkgname> <child version> <publish parent version> <changelog parent version>\n'
193- help_text += '\n'.join(textwrap.wrap('with one override per '
194- 'line.',
195- subhelp_width
196- )
197- )
198+ help_text += '\n'.join(
199+ textwrap.wrap(
200+ "with one override per line.",
201+ subhelp_width,
202+ )
203+ )
204 help_text += '\ndefault=%(default)s'
205- parser.add_argument('-P', '--parentfile', type=str,
206- help=argparse.SUPPRESS,
207- default=pkg_resources.resource_filename(
208- 'gitubuntu',
209- 'parent_overrides.txt'
210- )
211- )
212- help_text = '\n'.join(textwrap.wrap('File specifying source '
213- 'files to use instead of '
214- 'Launchpad to work around '
215- 'source file shadowing: ',
216- subhelp_width
217- )
218- )
219+ parser.add_argument(
220+ '-P', '--parentfile',
221+ type=str,
222+ help=argparse.SUPPRESS,
223+ default=pkg_resources.resource_filename(
224+ 'gitubuntu',
225+ 'parent_overrides.txt',
226+ ),
227+ )
228+ help_text = '\n'.join(
229+ textwrap.wrap(
230+ "File specifying source files to use instead of "
231+ "Launchpad to work around source file shadowing: ",
232+ subhelp_width,
233+ )
234+ )
235 help_text += '\n'
236 help_text += '<pkgname> <version> <URL of DSC file> <URLs of orig files>\n'
237- help_text += '\n'.join(textwrap.wrap('with one package per line. '
238- '\'-\' for the DSC file '
239- 'indicates to use the one '
240- 'from Launchpad.'
241- )
242- )
243+ help_text += '\n'.join(
244+ textwrap.wrap(
245+ "with one package per line. '-' for the DSC file "
246+ "indicates to use the one from Launchpad.",
247+ subhelp_width,
248+ )
249+ )
250 help_text += '\ndefault=%(default)s'
251- parser.add_argument('-L', '--pullfile', type=str,
252- help=argparse.SUPPRESS,
253- default=pkg_resources.resource_filename(
254- 'gitubuntu',
255- 'pull_overrides.txt'
256- )
257- )
258+ parser.add_argument(
259+ '-L', '--pullfile',
260+ type=str,
261+ help=argparse.SUPPRESS,
262+ default=pkg_resources.resource_filename(
263+ 'gitubuntu',
264+ 'pull_overrides.txt',
265+ ),
266+ )
267
268 argcomplete.autocomplete(parser)
269 args = parser.parse_args()
270@@ -161,22 +176,22 @@ def main():
271 # fragile, assumes developers will pass the base_subparser above
272 # should make this structural in the classes
273 if args.verbose:
274- logging.basicConfig(level=logging.DEBUG,
275- format='%(asctime)s - %(levelname)s:%(message)s',
276- datefmt='%m/%d/%Y %H:%M:%S',
277- )
278+ level=logging.DEBUG
279 else:
280- logging.basicConfig(level=logging.INFO,
281- format='%(asctime)s - %(levelname)s:%(message)s',
282- datefmt='%m/%d/%Y %H:%M:%S',
283- )
284+ level=logging.INFO
285+ logging.basicConfig(
286+ level=level,
287+ format='%(asctime)s - %(levelname)s:%(message)s',
288+ datefmt='%m/%d/%Y %H:%M:%S',
289+ )
290 if args.subcommand in known_network_subcommands:
291 try:
292 retry_backoffs = args.retry_backoffs
293 if len(retry_backoffs) != args.retries:
294- logging.error('Number of backoffs specified in '
295- '--retry-backoffs must match --retries.'
296- )
297+ logging.error(
298+ "Number of backoffs specified in "
299+ "--retry-backoffs must match --retries."
300+ )
301 sys.exit(1)
302 except AttributeError:
303 args.retry_backoffs = [2 ** i for i in range(args.retries)]
304@@ -188,26 +203,30 @@ def main():
305 )
306 except CalledProcessError:
307 if isatty(sys.stdin.fileno()):
308- user = input("gitubuntu.lpuser is not set. What is your "
309+ user = input(
310+ "gitubuntu.lpuser is not set. What is your "
311 "Launchpad username? We will set this in your "
312 "personally global git configuration for you. "
313 "To abort, press Enter.\nUsername: "
314 )
315 if len(user) == 0:
316- logging.error('git-ubuntu requires a launchpad user')
317+ logging.error("git-ubuntu requires a launchpad user")
318 sys.exit(1)
319 else:
320- logging.warning('Setting the Launchpad user for '
321- 'git-ubuntu to %s with '
322- '`git config --global gitubuntu.lpuser %s`.',
323- user, user)
324+ logging.warning(
325+ "Setting the Launchpad user for git-ubuntu to %s "
326+ "with `git config --global gitubuntu.lpuser %s`.",
327+ user,
328+ user,
329+ )
330 run(['git', 'config', '--global', 'gitubuntu.lpuser', user])
331 else:
332- logging.error('Not connected to a TTY and no '
333- 'git-ubuntu lpuser defined. Please run '
334- '`git config --global gitubuntu.lpuser '
335- '<Launchpad username>` and re-run this '
336- 'command.')
337+ logging.error(
338+ "Not connected to a TTY and no git-ubuntu lpuser "
339+ "defined. Please run `git config --global "
340+ "gitubuntu.lpuser <Launchpad username>` and re-run "
341+ "this command."
342+ )
343 sys.exit(1)
344
345 args.func(args)
346diff --git a/gitubuntu/build.py b/gitubuntu/build.py
347index 8143995..d3d292e 100644
348--- a/gitubuntu/build.py
349+++ b/gitubuntu/build.py
350@@ -38,12 +38,11 @@ from gitubuntu.run import run, runq
351 from gitubuntu.source_information import (
352 GitUbuntuSourceInformation,
353 NoPublicationHistoryException,
354+ derive_source_from_series,
355 )
356 try:
357 pkg = 'python3-debian'
358 from debian.debfile import PART_EXTS
359- pkg = 'python3-distro-info'
360- from distro_info import DebianDistroInfo, UbuntuDistroInfo
361 pkg = 'python3-pytest'
362 import pytest
363 except ImportError:
364@@ -228,8 +227,7 @@ def fetch_orig_from_cache(changelog, source, dl_cache=None):
365 return None
366 if dsc.verify() is None:
367 logging.warn(
368- "Cache dir found, but verification of orig tarball %s failed.",
369- orig,
370+ "Cache dir found, but verification of orig tarball(s) failed."
371 )
372 return None # XXX decide - see docstring
373 _symlink_paths_into_parent_dir(dsc.all_tarball_paths)
374@@ -320,8 +318,8 @@ def fetch_orig_from_launchpad(changelog, source, pullfile, retries,
375 except NoPublicationHistoryException:
376 logging.warning(
377 "No publication history found for %s in %s. ",
378- pkgname,
379- dist_name,
380+ changelog.srcpkg,
381+ source,
382 )
383 return None
384
385@@ -374,151 +372,156 @@ def fetch_orig_from_launchpad(changelog, source, pullfile, retries,
386 return None
387
388
389-class GitUbuntuBuild:
390- def parse_args(self, subparsers=None, base_subparsers=None):
391- kwargs = dict(
392- description='Build a git-ubuntu-cloned tree with dpkg-buildpackage',
393- formatter_class=argparse.ArgumentDefaultsHelpFormatter,
394- usage='%(prog)s [options] -- ...',
395- epilog='Any arguments after -- are passed to dpkg-buildpackage.'
396- )
397- if base_subparsers:
398- kwargs['parents'] = base_subparsers
399- if subparsers:
400- parser = subparsers.add_parser('build', **kwargs)
401- parser.set_defaults(func=self.main)
402- else:
403- parser = argparse.ArgumentParser(**kwargs)
404- parser.add_argument('--dl-cache', type=str,
405- help='Cache directory for downloads.',
406- default=os.path.join(os.getcwd(), '.git', CACHE_PATH),
407- )
408- parser.add_argument(
409- '--for-merge',
410- action='store_true',
411- help="Build an Ubuntu merge. Implies searching Debian for "
412- "published versions first.",
413+def check_repository():
414+ try:
415+ cp = run(['git', 'status', '--porcelain'])
416+ except CalledProcessError:
417+ logging.error('Is the current directory a git repository?')
418+ sys.exit(1)
419+
420+ if len(cp.stdout) > 0:
421+ logging.warning(
422+ "Working tree is not clean. `git status` output follows."
423 )
424- parser.add_argument(
425- '--no-pristine-tar',
426- action='store_true',
427- help="Do not use pristine-tar branches to generate orig "
428- "tarballs. This can be necessary as we stabilize the pristine-tar "
429- "data generation, particularly if multiple component tarballs are "
430- "present.",
431+ run(['git', 'status'], stdout=None)
432+ sys.exit(1)
433
434+ if not os.path.isfile('debian/changelog'):
435+ logging.error(
436+ "No debian/changelog found, is this a source "
437+ "package repository? See `git ubuntu clone`."
438 )
439- parser.add_argument('rem_args', help=argparse.SUPPRESS,
440- nargs=argparse.REMAINDER)
441- if not subparsers:
442- return parser.parse_args()
443- return 'build - %s' % kwargs['description']
444+ sys.exit(1)
445
446- def check_repository(self):
447- try:
448- cp = run(['git', 'status', '--porcelain'])
449- except CalledProcessError:
450- logging.error('Is the current directory a git repository?')
451- sys.exit(1)
452
453- if len(cp.stdout) > 0:
454- logging.warning(
455- "Working tree is not clean. `git status` output follows."
456- )
457- run(['git', 'status'], stdout=None)
458- sys.exit(1)
459+def main(search_list, changelog, rem_args):
460+ """main entry point for build
461
462- if not os.path.isfile('debian/changelog'):
463- logging.error(
464- "No debian/changelog found, is this a source "
465- "package repository? See `git ubuntu clone`."
466- )
467- sys.exit(1)
468+ @param search_list: list of OrigSearchListEntry namedtuples
469+ @param changelog: gitubuntu.git_repository.Changelog object for source package to build
470+ @param rem_args: namespace object of remaining arguments to pass onto dpkg-buildpackage
471+ """
472+ check_repository()
473+
474+ fetch_orig_and_build(search_list, changelog, rem_args)
475+
476+
477+def parse_args(subparsers=None, base_subparsers=None):
478+ kwargs = dict(
479+ description='Build a git-ubuntu-cloned tree with dpkg-buildpackage',
480+ formatter_class=argparse.ArgumentDefaultsHelpFormatter,
481+ usage='%(prog)s [options] -- ...',
482+ epilog='Any arguments after -- are passed to dpkg-buildpackage.'
483+ )
484+ if base_subparsers:
485+ kwargs['parents'] = base_subparsers
486+ if subparsers:
487+ parser = subparsers.add_parser('build', **kwargs)
488+ parser.set_defaults(func=cli_main)
489+ else:
490+ parser = argparse.ArgumentParser(**kwargs)
491+ parser.add_argument('--dl-cache', type=str,
492+ help='Cache directory for downloads.',
493+ default=os.path.join(os.getcwd(), '.git', CACHE_PATH),
494+ )
495+ parser.add_argument(
496+ '--for-merge',
497+ action='store_true',
498+ help="Build an Ubuntu merge. Implies searching Debian for "
499+ "published versions first.",
500+ )
501+ parser.add_argument(
502+ '--no-pristine-tar',
503+ action='store_true',
504+ help="Do not use pristine-tar branches to generate orig "
505+ "tarballs. This can be necessary as we stabilize the pristine-tar "
506+ "data generation, particularly if multiple component tarballs are "
507+ "present.",
508
509- def derive_orig_search_list_from_args(self, args):
510- source = 'debian' if args.for_merge else 'ubuntu'
511- orig_search_list = [
512- OrigSearchListEntry(
513- mechanism=fetch_orig_from_parent_dir,
514- source=None,
515- must_build=True,
516- ),
517- OrigSearchListEntry(
518- mechanism=fetch_orig_from_cache,
519- source=source,
520- must_build=True,
521- ),
522- ]
523- if not args.no_pristine_tar:
524- orig_search_list.append(
525- OrigSearchListEntry(
526- mechanism=functools.partial(fetch_orig_from_pristine_tar,
527- repo=self.local_repo,
528- ),
529- source=source,
530- must_build=True,
531- )
532- )
533- orig_search_list.extend([
534+ )
535+ parser.add_argument('rem_args', help=argparse.SUPPRESS,
536+ nargs=argparse.REMAINDER)
537+ if not subparsers:
538+ return parser.parse_args()
539+ return 'build - %s' % kwargs['description']
540+
541+
542+def derive_orig_search_list_from_args(args):
543+ source = 'debian' if args.for_merge else 'ubuntu'
544+ orig_search_list = [
545+ OrigSearchListEntry(
546+ mechanism=fetch_orig_from_parent_dir,
547+ source=None,
548+ must_build=True,
549+ ),
550+ OrigSearchListEntry(
551+ mechanism=fetch_orig_from_cache,
552+ source=source,
553+ must_build=True,
554+ ),
555+ ]
556+ if not args.no_pristine_tar:
557+ orig_search_list.append(
558 OrigSearchListEntry(
559- mechanism=functools.partial(fetch_orig_from_launchpad,
560- pullfile=args.pullfile,
561- retries=args.retries,
562- retry_backoffs=args.retry_backoffs,
563- dl_cache=args.dl_cache,
564+ mechanism=functools.partial(fetch_orig_from_pristine_tar,
565+ repo=GitUbuntuRepository('.'),
566 ),
567 source=source,
568 must_build=True,
569+ )
570+ )
571+ orig_search_list.extend([
572+ OrigSearchListEntry(
573+ mechanism=functools.partial(fetch_orig_from_launchpad,
574+ pullfile=args.pullfile,
575+ retries=args.retries,
576+ retry_backoffs=args.retry_backoffs,
577+ dl_cache=args.dl_cache,
578 ),
579- OrigSearchListEntry(
580- mechanism=fetch_orig_noop,
581- source=[],
582- must_build=True,
583- ),
584- ])
585+ source=source,
586+ must_build=True,
587+ ),
588+ OrigSearchListEntry(
589+ mechanism=fetch_orig_noop,
590+ source=[],
591+ must_build=True,
592+ ),
593+ ])
594
595- return orig_search_list
596+ return orig_search_list
597
598
599- def main(self, args):
600- rem_args = args.rem_args
601- if len(rem_args) > 0:
602- rem_args = rem_args[1:]
603+def cli_main(args):
604+ rem_args = args.rem_args
605+ if len(rem_args) > 0:
606+ rem_args = rem_args[1:]
607
608- self.check_repository()
609- self.local_repo = GitUbuntuRepository('.')
610+ changelog = Changelog.from_path('debian/changelog')
611
612- changelog = Changelog.from_path('debian/changelog')
613+ try:
614+ native = is_native_package(changelog)
615+ except NativenessMismatchError as e:
616+ logging.error("%s" % e)
617+ sys.exit(1)
618
619- try:
620- native = is_native_package(changelog)
621- except NativenessMismatchError as e:
622- logging.error("%s" % e)
623- sys.exit(1)
624-
625- if native:
626- # No orig tarball required
627- orig_search_list = [
628- OrigSearchListEntry(
629- mechanism=fetch_orig_noop,
630- source=[],
631- must_build=True,
632- ),
633- ]
634- else:
635- orig_search_list = self.derive_orig_search_list_from_args(args)
636+ if native:
637+ # No orig tarball required
638+ orig_search_list = [
639+ OrigSearchListEntry(
640+ mechanism=fetch_orig_noop,
641+ source=[],
642+ must_build=True,
643+ ),
644+ ]
645+ else:
646+ orig_search_list = derive_orig_search_list_from_args(args)
647
648- # See http://pad.ubuntu.com/KKB1kMR0JH for logic
649- fetch_orig_and_build(orig_search_list, changelog, rem_args)
650+ # See http://pad.ubuntu.com/KKB1kMR0JH for logic
651+ main(orig_search_list, changelog, rem_args)
652
653
654 def derive_source_from_changelog(changelog):
655- series = changelog.distribution
656- if DebianDistroInfo().valid(codename=series):
657- return 'debian'
658- if UbuntuDistroInfo().valid(codename=series):
659- return 'ubuntu'
660- raise ValueError("Unable to determine distribution from %s" % series)
661+ return derive_source_from_series(changelog.distribution)
662
663 def expand_changelog_source_aliases(orig_search_list, changelog):
664 """Replace 'changelog' sources by reading debian/changelog
665diff --git a/gitubuntu/buildsource.py b/gitubuntu/buildsource.py
666index 4e9b69b..1735129 100644
667--- a/gitubuntu/buildsource.py
668+++ b/gitubuntu/buildsource.py
669@@ -5,66 +5,62 @@ import argparse
670 import logging
671 import os
672 import sys
673-from gitubuntu.build import GitUbuntuBuild
674+from gitubuntu.build import cli_main as build_cli_main
675 from gitubuntu.cache import CACHE_PATH
676
677-class GitUbuntuBuildSource(GitUbuntuBuild):
678- def __init__(self):
679- super().__init__()
680+def parse_args(subparsers=None, base_subparsers=None):
681+ kwargs = dict(
682+ description="Build a source package and changes file",
683+ formatter_class=argparse.ArgumentDefaultsHelpFormatter
684+ )
685+ if base_subparsers:
686+ kwargs['parents'] = base_subparsers
687+ if subparsers:
688+ parser = subparsers.add_parser('build-source', **kwargs)
689+ help_text = "build-source - Build a source package"
690+ parser.set_defaults(func=cli_main)
691+ else:
692+ parser = argparse.ArgumentParser(**kwargs)
693+ parser.add_argument('--dl-cache', type=str,
694+ help=("Cache directory for downloads."),
695+ default=os.path.join(os.getcwd(), '.git', CACHE_PATH),
696+ )
697+ parser.add_argument('--sign', action='store_true',
698+ help="Sign the source package.",
699+ )
700+ parser.add_argument('--for-merge', action='store_true',
701+ help=("Build an Ubuntu merge. Implies searching Debian for "
702+ "published versions first and passing -sa to "
703+ "dpkg-buildpackage."
704+ ),
705+ )
706+ parser.add_argument(
707+ '--no-pristine-tar',
708+ action='store_true',
709+ help="Do not use pristine-tar branches to generate orig "
710+ "tarballs. This can be necessary as we stabilize the pristine-tar "
711+ "data generation, particularly if multiple component tarballs are "
712+ "present.",
713
714- def parse_args(self, subparsers=None, base_subparsers=None):
715- kwargs = dict(
716- description="Build a source package and changes file",
717- formatter_class=argparse.ArgumentDefaultsHelpFormatter
718- )
719- if base_subparsers:
720- kwargs['parents'] = base_subparsers
721- if subparsers:
722- parser = subparsers.add_parser('build-source', **kwargs)
723- help_text = "build-source - Build a source package"
724- parser.set_defaults(func=self.main)
725- else:
726- parser = argparse.ArgumentParser(**kwargs)
727- parser.add_argument('--dl-cache', type=str,
728- help=("Cache directory for downloads."),
729- default=os.path.join(os.getcwd(), '.git', CACHE_PATH),
730- )
731- parser.add_argument('--sign', action='store_true',
732- help="Sign the source package.",
733- )
734- parser.add_argument('--for-merge', action='store_true',
735- help=("Build an Ubuntu merge. Implies searching Debian for "
736- "published versions first and passing -sa to "
737- "dpkg-buildpackage."
738- ),
739- )
740- parser.add_argument(
741- '--no-pristine-tar',
742- action='store_true',
743- help="Do not use pristine-tar branches to generate orig "
744- "tarballs. This can be necessary as we stabilize the pristine-tar "
745- "data generation, particularly if multiple component tarballs are "
746- "present.",
747+ )
748+ parser.add_argument('rem_args', help=argparse.SUPPRESS,
749+ nargs=argparse.REMAINDER
750+ )
751+ if not subparsers:
752+ return parser.parse_args()
753+ return "build-source - %s" % kwargs['description']
754
755- )
756- parser.add_argument('rem_args', help=argparse.SUPPRESS,
757- nargs=argparse.REMAINDER
758- )
759- if not subparsers:
760- return parser.parse_args()
761- return "build-source - %s" % kwargs['description']
762-
763- def main(self, args):
764- if len(args.rem_args) != 0:
765- logging.warning('Passing specified flags (%s) on to '
766- 'dpkg-buildpackage. Did you mean to '
767- 'call `git ubuntu build` instead?' %
768- ' '.join(args.rem_args)
769- )
770- else:
771- args.rem_args = ['--', '-S', '-nc', '-d', '-i', '-I']
772- if args.for_merge:
773- args.rem_args += ['-sa']
774- if not args.sign:
775- args.rem_args += ['-us', '-uc']
776- super().main(args)
777+def cli_main(args):
778+ if len(args.rem_args) != 0:
779+ logging.warning('Passing specified flags (%s) on to '
780+ 'dpkg-buildpackage. Did you mean to '
781+ 'call `git ubuntu build` instead?' %
782+ ' '.join(args.rem_args)
783+ )
784+ else:
785+ args.rem_args = ['--', '-S', '-nc', '-d', '-i', '-I']
786+ if args.for_merge:
787+ args.rem_args += ['-sa']
788+ if not args.sign:
789+ args.rem_args += ['-us', '-uc']
790+ build_cli_main(args)
791diff --git a/gitubuntu/clone.py b/gitubuntu/clone.py
792index a1bc8cb..6fdc8e9 100644
793--- a/gitubuntu/clone.py
794+++ b/gitubuntu/clone.py
795@@ -18,12 +18,82 @@ except ImportError:
796 logging.error('Is %s installed?', pkg)
797 sys.exit(1)
798
799-class GitUbuntuClone:
800- def parse_args(self, subparsers=None, base_subparsers=None):
801- kwargs = dict(
802- description='Clone a source package git repository to a directory',
803- formatter_class=argparse.RawTextHelpFormatter,
804- epilog='''
805+def copy_hooks(src, dst):
806+ try:
807+ os.mkdir(dst)
808+ except FileExistsError:
809+ pass
810+
811+ for hook in os.listdir(src):
812+ shutil.copy2(
813+ os.path.join(src, hook),
814+ dst,
815+ )
816+
817+def main(directory, lp_user, proto, package):
818+ if os.path.isdir(directory):
819+ logging.error('directory %s exists' % directory)
820+ sys.exit(1)
821+
822+ local_repo = GitUbuntuRepository(
823+ local_dir=directory,
824+ lp_user=lp_user,
825+ fetch_proto=proto,
826+ )
827+
828+ copy_hooks(
829+ os.path.join(os.path.dirname(__file__), os.path.pardir, 'hooks'),
830+ os.path.join(directory, '.git', 'hooks')
831+ )
832+
833+ try:
834+ local_repo.add_base_remotes(package)
835+ local_repo.fetch_base_remotes(verbose=True)
836+ except GitUbuntuRepositoryFetchError:
837+ logging.error("Unable to find an imported repository for %s. "
838+ "Please request an import by e-mailing "
839+ "usd-import-team@lists.launchpad.net.",
840+ package
841+ )
842+ shutil.rmtree(local_repo.local_dir)
843+ raise
844+
845+ try:
846+ local_repo.add_lpuser_remote(pkgname=package)
847+ local_repo.fetch_lpuser_remote(verbose=True)
848+
849+ logging.debug("added remote '%s' -> %s", local_repo.lp_user,
850+ local_repo.raw_repo.remotes[local_repo.lp_user].url
851+ )
852+ except GitUbuntuRepositoryFetchError:
853+ pass
854+
855+ try:
856+ local_repo.create_tracking_branch(
857+ 'ubuntu/devel',
858+ 'pkg/ubuntu/devel'
859+ )
860+ local_repo.checkout_commitish('ubuntu/devel')
861+ except:
862+ logging.error('Unable to checkout ubuntu/devel, does '
863+ 'pkg/ubuntu/devel branch exist?'
864+ )
865+
866+ if os.path.isfile(os.path.join(directory, '.gitignore')):
867+ logging.warning('A .gitignore file exists in the source '
868+ 'package. This will affect the behavior of git. Consider '
869+ 'backing up the gitignore while working on this package '
870+ 'to ensure all changes are tracked or passing appropriate '
871+ 'flags to git commands (e.g., git status --ignored).'
872+ )
873+
874+ return directory
875+
876+def parse_args(subparsers=None, base_subparsers=None):
877+ kwargs = dict(
878+ description='Clone a source package git repository to a directory',
879+ formatter_class=argparse.RawTextHelpFormatter,
880+ epilog='''
881 Example:
882 * clone to open-iscsi/
883 %(prog)s open-iscsi
884@@ -32,107 +102,41 @@ Example:
885 * use https rather than git protocol for read-only remotes:
886 %(prog)s --https open-iscsi
887 '''
888- )
889- if base_subparsers:
890- kwargs['parents'] = base_subparsers
891- if subparsers:
892- parser = subparsers.add_parser('clone', **kwargs)
893- parser.set_defaults(func=self.main)
894- else:
895- parser = argparse.ArgumentParser(**kwargs)
896- parser.add_argument('package', type=str,
897- help='Name of source package to clone'
898- )
899- parser.add_argument('directory', type=str,
900- help='Local directory to clone to. If not specified, a '
901- ' directory with the same name as PACKAGE will be '
902- 'used',
903- default=None,
904- nargs='?'
905- )
906- parser.add_argument('-l', '--lp-user', type=str, help=argparse.SUPPRESS)
907- if not subparsers:
908- return parser.parse_args()
909- return 'clone - %s' % kwargs['description']
910-
911- @staticmethod
912- def copy_hooks(src, dst):
913- try:
914- os.mkdir(dst)
915- except FileExistsError:
916- pass
917-
918- for hook in os.listdir(src):
919- shutil.copy2(
920- os.path.join(src, hook),
921- dst,
922- )
923-
924- def main(self, args):
925- try:
926- lp_user = args.lp_user
927- except AttributeError:
928- lp_user = None
929- directory = (
930- os.path.abspath(args.directory)
931- if args.directory
932- else os.path.join(os.path.abspath(os.getcwd()), args.package)
933- )
934+ )
935+ if base_subparsers:
936+ kwargs['parents'] = base_subparsers
937+ if subparsers:
938+ parser = subparsers.add_parser('clone', **kwargs)
939+ parser.set_defaults(func=cli_main)
940+ else:
941+ parser = argparse.ArgumentParser(**kwargs)
942+ parser.add_argument('package', type=str,
943+ help='Name of source package to clone'
944+ )
945+ parser.add_argument('directory', type=str,
946+ help='Local directory to clone to. If not specified, a '
947+ 'directory with the same name as PACKAGE will be '
948+ 'used',
949+ default=None,
950+ nargs='?'
951+ )
952+ parser.add_argument('-l', '--lp-user', type=str, help=argparse.SUPPRESS)
953+ if not subparsers:
954+ return parser.parse_args()
955+ return 'clone - %s' % kwargs['description']
956
957- if os.path.isdir(directory):
958- logging.error('directory %s exists' % directory)
959- sys.exit(1)
960+def cli_main(args):
961+ try:
962+ lp_user = args.lp_user
963+ except AttributeError:
964+ lp_user = None
965+ directory = (
966+ os.path.abspath(args.directory)
967+ if args.directory
968+ else os.path.join(os.path.abspath(os.getcwd()), args.package)
969+ )
970
971- local_repo = GitUbuntuRepository(
972- local_dir=directory,
973- lp_user=lp_user,
974- fetch_proto=args.proto,
975- )
976-
977- self.copy_hooks(
978- os.path.join(os.path.dirname(__file__), os.path.pardir, 'hooks'),
979- os.path.join(directory, '.git', 'hooks')
980- )
981+ main(directory, lp_user, args.proto, args.package)
982
983- try:
984- local_repo.add_base_remotes(args.package)
985- local_repo.fetch_base_remotes(verbose=True)
986- except GitUbuntuRepositoryFetchError:
987- logging.error("Unable to find an imported repository for %s. "
988- "Please request an import by e-mailing "
989- "usd-import-team@lists.launchpad.net.",
990- args.package
991- )
992- shutil.rmtree(local_repo.local_dir)
993- raise
994-
995- try:
996- local_repo.add_lpuser_remote(pkgname=args.package)
997- local_repo.fetch_lpuser_remote(verbose=True)
998-
999- logging.debug("added remote '%s' -> %s", local_repo.lp_user,
1000- local_repo.raw_repo.remotes[local_repo.lp_user].url
1001- )
1002- except GitUbuntuRepositoryFetchError:
1003- pass
1004-
1005- try:
1006- local_repo.create_tracking_branch(
1007- 'ubuntu/devel',
1008- 'pkg/ubuntu/devel'
1009- )
1010- local_repo.checkout_commitish('ubuntu/devel')
1011- except:
1012- logging.error('Unable to checkout ubuntu/devel, does '
1013- 'pkg/ubuntu/devel branch exist?'
1014- )
1015-
1016- if os.path.isfile(os.path.join(directory, '.gitignore')):
1017- logging.warning('A .gitignore file exists in the source '
1018- 'package. This will affect the behavior of git. Consider '
1019- 'backing up the gitignore while working on this package '
1020- 'to ensure all changes are tracked or passing appropriate '
1021- 'flags to git commands (e.g., git status --ignored).'
1022- )
1023
1024 # vi: ts=4 expandtab
1025diff --git a/gitubuntu/importer.py b/gitubuntu/importer.py
1026index 22fd08a..c3903cb 100644
1027--- a/gitubuntu/importer.py
1028+++ b/gitubuntu/importer.py
1029@@ -26,6 +26,7 @@
1030
1031 import argparse
1032 import atexit
1033+import functools
1034 import getpass
1035 import logging
1036 import os
1037@@ -50,7 +51,12 @@ from gitubuntu.git_repository import (
1038 PristineTarError,
1039 )
1040 from gitubuntu.run import decode_binary, run, runq
1041-from gitubuntu.source_information import GitUbuntuSourceInformation, NoPublicationHistoryException, SourceExtractionException, launchpad_login_auth
1042+from gitubuntu.source_information import (
1043+ GitUbuntuSourceInformation,
1044+ NoPublicationHistoryException,
1045+ SourceExtractionException,
1046+ launchpad_login_auth,
1047+)
1048 from gitubuntu.version import VERSION
1049 from gitubuntu.versioning import version_compare
1050 try:
1051@@ -86,16 +92,18 @@ def dsc_to_tree_hash(repo, dsc_path):
1052 # extracted_dir.
1053
1054 extracted_dir = os.path.join(temp_dir, 'x')
1055- run(['dpkg-source',
1056- '-x',
1057- '--skip-patches',
1058- dsc_path,
1059- extracted_dir, # this is <temp_dir>/x
1060+ run(
1061+ [
1062+ 'dpkg-source',
1063+ '-x',
1064+ '--skip-patches',
1065+ dsc_path,
1066+ extracted_dir, # this is <temp_dir>/x
1067 ]
1068- )
1069+ )
1070
1071 if not os.path.exists(extracted_dir):
1072- logging.exception('No source extracted?')
1073+ logging.error("No source extracted?")
1074 raise SourceExtractionException(
1075 "Failed to find an extracted directory from "
1076 "dpkg-source -x"
1077@@ -104,1258 +112,1462 @@ def dsc_to_tree_hash(repo, dsc_path):
1078 return repo.dir_to_tree(extracted_dir)
1079
1080
1081-class GitUbuntuImport:
1082- def __init__(self):
1083- self.parent_overrides = None
1084-
1085- def parse_args(self, subparsers=None, base_subparsers=None):
1086- kwargs = dict(description='Update a launchpad git tree based upon '
1087- 'the state of the Ubuntu and Debian archives',
1088- formatter_class=argparse.ArgumentDefaultsHelpFormatter
1089- )
1090- if base_subparsers:
1091- kwargs['parents'] = base_subparsers
1092- if subparsers:
1093- parser = subparsers.add_parser('import', **kwargs)
1094- parser.set_defaults(func=self.main)
1095- else:
1096- parser = argparse.ArgumentParser(**kwargs)
1097- parser.add_argument('package', type=str,
1098- help='Which package to update in the git tree')
1099- parser.add_argument('-o', '--lp-owner', type=str,
1100- help=argparse.SUPPRESS,
1101- default='usd-import-team')
1102- parser.add_argument('-l', '--lp-user', type=str,
1103- help=argparse.SUPPRESS)
1104- parser.add_argument('--dl-cache', type=str,
1105- help=('Cache directory for downloads. Default is to '
1106- 'put downloads in <directory>/.git/%s' %
1107- CACHE_PATH
1108- ),
1109- default=argparse.SUPPRESS)
1110- parser.add_argument('--no-fetch', action='store_true',
1111- help='Do not fetch from the remote (DANGEROUS; '
1112- 'only useful for debugging); implies '
1113- '--no-push')
1114- parser.add_argument('--no-push', action='store_true',
1115- help='Do not push to the remote')
1116- parser.add_argument('--no-clean', action='store_true',
1117- help='Do not clean the temporary directory')
1118- parser.add_argument('-d', '--directory', type=str,
1119- help='Use git repository at specified location rather '
1120- 'than a temporary directory (implies --no-clean). '
1121- 'Will create the local repository if needed.',
1122- default=argparse.SUPPRESS
1123- )
1124- parser.add_argument('--fixup-devel', action='store_true',
1125- help=argparse.SUPPRESS)
1126- parser.add_argument('--active-series-only', action='store_true',
1127- help='Do an import of only active Ubuntu series '
1128- 'history, implies --no-push.')
1129- parser.add_argument('--skip-applied', action='store_true',
1130- help=argparse.SUPPRESS)
1131- parser.add_argument('--skip-orig', action='store_true',
1132- help=argparse.SUPPRESS)
1133- parser.add_argument('--reimport', action='store_true',
1134- help=argparse.SUPPRESS)
1135- if not subparsers:
1136- return parser.parse_args()
1137- return 'import - %s' % kwargs['description']
1138-
1139- def cleanup(self, no_clean, local_dir):
1140- """Recursively remove local_dir if no_clean is not True"""
1141- if not no_clean:
1142- shutil.rmtree(local_dir)
1143- else:
1144- logging.info('Leaving %s as directed' % local_dir)
1145-
1146- def get_changelog_for_commit(self, tree_hash, publish_parent_commit, changelog_parent_commit):
1147- """Extract changes to debian/changelog in this publish
1148-
1149- LP: #1633114 -- favor the changelog parent's diff
1150- """
1151- raw_clog_entry = b''
1152- if changelog_parent_commit is not None:
1153- cmd = ['diff-tree', '-p', changelog_parent_commit,
1154- tree_hash, '--', 'debian/changelog']
1155- raw_clog_entry = self.local_repo.git_run(cmd).stdout
1156- elif publish_parent_commit is not None:
1157- cmd = ['diff-tree', '-p', publish_parent_commit,
1158- tree_hash, '--', 'debian/changelog']
1159- raw_clog_entry = self.local_repo.git_run(cmd).stdout
1160-
1161- changelog_entry = b''
1162- changelog_entry_found = False
1163- i = 0
1164- for line in raw_clog_entry.split(b'\n'):
1165- # skip the header lines
1166- if i < 4:
1167- i += 1
1168- continue
1169- if not line.startswith(b'+'):
1170- # in case there are stray changelog changes, just take
1171- # the first complete section of changelog additions
1172- if changelog_entry_found:
1173- break
1174- continue
1175- line = re.sub(b'^[+]', b'', line)
1176- if not line.startswith(b' '):
1177- continue
1178- if not changelog_entry_found:
1179- changelog_entry = b'\n\nNew changelog entries:\n'
1180- changelog_entry += line + b'\n'
1181- changelog_entry_found = True
1182- return changelog_entry
1183-
1184- def _commit_import(self, spi, tree_hash, publish_parent_commit, changelog_parent_commit, upload_parent_commit, unapplied_parent_commit):
1185- """Commit a tree object into the repository with the specified
1186- parents
1187-
1188- The importer algorithm works by 'staging' the import of a
1189- publication as a tree, then using that tree to determine the
1190- appropriate parents from the publishing history and
1191- debian/changelog.
1192-
1193- Arguments:
1194- spi - A SourcePackageInformation instance for this publish
1195- tree_hash - SHA1 from git-dsc-commit --tree-only of this publish
1196- publish_parent_commit = SHA1 of publishing parent
1197- changelog_parent_commit = SHA1 of changelog parent
1198- upload_parent_commit = SHA1 of upload parent
1199- unapplied_parent_commit = SHA1 of unapplied-patches import parent
1200- """
1201- tag = None
1202-
1203- if unapplied_parent_commit:
1204- import_type = 'patches-applied'
1205- target_head_name = spi.applied_head_name(self.namespace)
1206- if self.local_repo.get_applied_tag(spi.version, self.namespace) is None:
1207- # Not imported before
1208- tag = applied_tag(spi.version, self.namespace)
1209- else:
1210- import_type = 'patches-unapplied'
1211- target_head_name = spi.head_name(self.namespace)
1212- if self.local_repo.get_import_tag(spi.version, self.namespace) is None:
1213- # Not imported before
1214- tag = import_tag(spi.version, self.namespace)
1215-
1216- # Do not show importer/ namespace to user
1217- _, _, pretty_head_name = target_head_name.partition('%s/' % self.namespace)
1218-
1219- changelog_entry = self.get_changelog_for_commit(
1220- tree_hash,
1221- publish_parent_commit,
1222- changelog_parent_commit
1223- )
1224- msg = (b'Import %s version %b to %b\n\nImported using git-ubuntu import.' %
1225- (import_type.encode(), spi.version.encode(), pretty_head_name.encode())
1226- )
1227-
1228- parents = []
1229-
1230- if publish_parent_commit is None and \
1231- changelog_parent_commit is None and \
1232- upload_parent_commit is None and \
1233- unapplied_parent_commit is None and \
1234- target_head_name in self.local_repo.local_branch_names:
1235- # No parents and not creating a new branch
1236- tag = orphan_tag(spi.version, self.namespace)
1237- else:
1238- msg += b'\n'
1239-
1240- if publish_parent_commit is not None:
1241- parents.append(publish_parent_commit)
1242- msg += b'\nPublish parent: %b' % publish_parent_commit.encode()
1243- if changelog_parent_commit is not None:
1244- parents.append(changelog_parent_commit)
1245- msg += b'\nChangelog parent: %b' % changelog_parent_commit.encode()
1246- if upload_parent_commit is not None:
1247- parents.append(upload_parent_commit)
1248- msg += b'\nUpload parent: %b' % upload_parent_commit.encode()
1249- if unapplied_parent_commit is not None:
1250- parents.append(unapplied_parent_commit)
1251- msg += b'\nUnapplied parent: %b' % unapplied_parent_commit.encode()
1252-
1253- msg += b'%b' % changelog_entry
1254-
1255- commit_hash = self.local_repo.commit_tree_hash(
1256- tree_hash,
1257- parents,
1258- msg,
1259- environment_spi=spi
1260- )
1261+def cleanup(no_clean, local_dir):
1262+ '''Remove a local directory, conditionally
1263
1264- self.local_repo.update_head_to_commit(target_head_name, commit_hash)
1265+ no_clean: if True, do not remove @local_dir and print a message instead
1266+ local_dir: directory to remove
1267+ '''
1268+ if not no_clean:
1269+ shutil.rmtree(local_dir)
1270+ else:
1271+ logging.info('Leaving %s as directed' % local_dir)
1272
1273- logging.debug('Committed %s import of %s as %s in %s',
1274- import_type, spi.version, commit_hash, pretty_head_name
1275- )
1276
1277- if tag is not None:
1278- # should be annotated to use create_tag API
1279- logging.debug('Creating tag %s pointing to %s', tag, commit_hash)
1280- self.local_repo.git_run(['tag', '-a', '-m', 'git-ubuntu import v%s' % VERSION,
1281- tag, commit_hash
1282- ]
1283- )
1284+# XXX: need a namedtuple to hold common arguments
1285+def main(pkgname, owner, no_clean, directory, user, proto, no_fetch,
1286+ no_push, dl_cache, fixup_devel, active_series_only, skip_orig,
1287+ pullfile, parentfile, retries, retry_backoffs, skip_applied,
1288+ reimport,
1289+):
1290+ if owner == 'usd-import-team':
1291+ namespace = 'importer'
1292+ else:
1293+ namespace = owner
1294
1295- def commit_unapplied_patches_import(self, spi, import_tree_hash, publish_parent_commit, changelog_parent_commit, upload_parent_commit):
1296- self._commit_import(
1297- spi,
1298- import_tree_hash,
1299- publish_parent_commit,
1300- changelog_parent_commit,
1301- upload_parent_commit,
1302- # unapplied trees do not have a distinct unapplied parent
1303- None
1304- )
1305+ logging.info('Ubuntu Server Team importer v%s' % VERSION)
1306
1307- def commit_applied_patches_import(self, spi, import_tree_hash, publish_parent_commit, changelog_parent_commit, unapplied_parent_commit):
1308- self._commit_import(
1309- spi,
1310- import_tree_hash,
1311- publish_parent_commit,
1312- changelog_parent_commit,
1313- # uploads will be unapplied trees currently, so applied trees
1314- # can not have them as direct parents
1315- None,
1316- unapplied_parent_commit
1317- )
1318+ repo = GitUbuntuRepository(directory, user, proto)
1319+ if not directory:
1320+ logging.info('Using git repository at %s', repo.local_dir)
1321+
1322+ atexit.register(cleanup, no_clean, repo.local_dir)
1323
1324- def import_dsc(self, dsc, version, dist):
1325- """Imports the dsc file to importer/{debian,ubuntu}/dsc
1326-
1327- Arguments:
1328- spi - A GitUbuntuSourcePackageInformation instance
1329- """
1330-
1331- # Add DSC file to the appropriate branch
1332- runq(['git', 'checkout', '%s/importer/%s/dsc' % (self.namespace, dist)], env=self.local_repo.env)
1333- # It is possible the same-named DSC is published multiple times
1334- # with different contents, e.g. on epoch bumps
1335- local_dir = os.path.join(self.local_repo.local_dir, version)
1336- os.makedirs(local_dir, exist_ok=True)
1337- shutil.copy(dsc.dsc_path,
1338- local_dir)
1339- self.local_repo.git_run(['add', self.local_repo.local_dir])
1340+ repo.add_base_remotes(pkgname, repo_owner=owner)
1341+ if not no_fetch and not reimport:
1342 try:
1343- # Anything to commit? (this will be 0 if, for instance, the
1344- # same dsc is being imported again)
1345- runq(['git', 'diff', '--exit-code', 'HEAD'], env=self.local_repo.env)
1346- except:
1347- self.local_repo.git_run(['commit', '-m', 'DSC file for %s' % version])
1348+ repo.fetch_base_remotes()
1349+ except GitUbuntuRepositoryFetchError:
1350+ pass
1351
1352- self.local_repo.clean_repository_state()
1353+ if not no_fetch:
1354+ repo.delete_branches_in_namespace(namespace)
1355+ repo.delete_tags_in_namespace(namespace)
1356+ repo.copy_base_references(namespace)
1357
1358- def orig_imported(self, orig_tarball_paths, dist):
1359- """Determines if a list of paths to tarballs have already been imported to @dist
1360+ if not no_fetch and not reimport:
1361+ try:
1362+ repo.fetch_remote_refspecs('pkg',
1363+ refspecs=['refs/tags/*:refs/tags/%s/*' % namespace],
1364+ )
1365+ except GitUbuntuRepositoryFetchError:
1366+ pass
1367
1368- Assumes that the pristine-tar branch exists
1369- """
1370+ if reimport:
1371+ if no_fetch:
1372+ logging.warning(
1373+ "--reimport specified with --no-fetch. Upload "
1374+ "tags would not be incorporated into the new import "
1375+ "and would be lost forever. This is not currently "
1376+ "supported and --no-fetch will be ignored for upload "
1377+ "tags."
1378+ )
1379 try:
1380- return self.local_repo.verify_pristine_tar(
1381- orig_tarball_paths,
1382- dist,
1383- self.namespace
1384+ repo.fetch_remote_refspecs('pkg',
1385+ refspecs=['refs/tags/upload/*:refs/tags/%s/upload/*' % namespace],
1386 )
1387- except PristineTarError as e:
1388- raise GitUbuntuImportOrigError from e
1389+ except GitUbuntuRepositoryFetchError:
1390+ pass
1391
1392- def import_orig(self, spi):
1393- """Imports the orig-tarball using gbp import-org --pristine-tar
1394+ repo.ensure_importer_branches_exist(namespace)
1395+
1396+ debian_sinfo = GitUbuntuSourceInformation(
1397+ 'debian',
1398+ pkgname,
1399+ os.path.abspath(pullfile),
1400+ retries,
1401+ retry_backoffs,
1402+ )
1403+
1404+ ubuntu_sinfo = GitUbuntuSourceInformation(
1405+ 'ubuntu',
1406+ pkgname,
1407+ os.path.abspath(pullfile),
1408+ retries,
1409+ retry_backoffs,
1410+ )
1411+
1412+ debian_head_versions = (
1413+ repo.get_heads_and_versions('debian', namespace)
1414+ )
1415+ for head_name in sorted(debian_head_versions):
1416+ _, _, pretty_head_name = head_name.partition('%s/' % namespace)
1417+ logging.debug('Last imported %s version is %s',
1418+ pretty_head_name,
1419+ debian_head_versions[head_name]['version']
1420+ )
1421
1422- Arguments:
1423- spi - A GitUbuntuSourcePackageInformation instance
1424- """
1425- dsc = GitUbuntuDsc(spi.dsc_pathname)
1426- upstream_version = str(spi.upstream_version)
1427- version = str(spi.version)
1428- dist = spi.distribution.name.lower()
1429+ ubuntu_head_versions = (
1430+ repo.get_heads_and_versions('ubuntu', namespace)
1431+ )
1432+ for head_name in sorted(ubuntu_head_versions):
1433+ _, _, pretty_head_name = head_name.partition('%s/' % namespace)
1434+ logging.debug('Last imported %s version is %s',
1435+ pretty_head_name,
1436+ ubuntu_head_versions[head_name]['version']
1437+ )
1438
1439- try:
1440- orig_tarball_path = dsc.orig_tarball_path
1441- except GitUbuntuDscError as e:
1442- raise GitUbuntuImportOrigError(
1443- 'Unable to import orig tarball in DSC for version %s' % version
1444- ) from e
1445-
1446- if orig_tarball_path is None:
1447- # native packages only have a .tar.*
1448- native_re = re.compile(r'.*\.tar\.[^.]+$')
1449- for entry in dsc['Files']:
1450- if native_re.match(entry['name']):
1451- return
1452- raise GitUbuntuImportOrigError('No orig tarball in '
1453- 'DSC for version %s.' % version)
1454+ if not skip_applied:
1455+ applied_debian_head_versions = (
1456+ repo.get_heads_and_versions(
1457+ 'applied/debian', namespace
1458+ )
1459+ )
1460+ for head_name in sorted(applied_debian_head_versions):
1461+ _, _, pretty_head_name = head_name.partition(
1462+ '%s/' % namespace
1463+ )
1464+ logging.debug('Last applied %s version is %s',
1465+ pretty_head_name,
1466+ applied_debian_head_versions[head_name]['version']
1467+ )
1468
1469- try:
1470- orig_component_paths = dsc.component_tarball_paths
1471- except GitUbuntuDscError as e:
1472- raise GitUbuntuImportOrigError(
1473- 'Unable to import component tarballs in DSC for version '
1474- '%s' % version
1475- ) from e
1476-
1477- orig_paths = [orig_tarball_path] + list(orig_component_paths.values())
1478- if (not orig_tarball_path or
1479- self.orig_imported(orig_paths, dist)):
1480- return
1481- if not all(map(os.path.exists, orig_paths)):
1482- raise GitUbuntuImportOrigError(
1483- 'Unable to find tarball: %s' %
1484- [p for p in orig_paths if not os.path.exists(p)]
1485+ applied_ubuntu_head_versions = (
1486+ repo.get_heads_and_versions(
1487+ 'applied/ubuntu', namespace
1488+ )
1489+ )
1490+ for head_name in sorted(applied_ubuntu_head_versions):
1491+ _, _, pretty_head_name = head_name.partition(
1492+ '%s/' % namespace
1493+ )
1494+ logging.debug('Last applied %s version is %s',
1495+ pretty_head_name,
1496+ applied_ubuntu_head_versions[head_name]['version']
1497 )
1498
1499- try:
1500- with self.local_repo.pristine_tar_branches(dist, self.namespace):
1501- oldcwd = os.getcwd()
1502- # gbp does not support running from arbitrary git trees or
1503- # working directories
1504- # https://github.com/agx/git-buildpackage/pull/16
1505- os.chdir(self.local_repo.local_dir)
1506-
1507- ext = os.path.splitext(dsc.orig_tarball_path)[1]
1508-
1509- # make this non-fatal if upstream already exist as tagged?
1510- cmd = ['gbp', 'import-orig', '--no-merge',
1511- '--upstream-branch', 'do-not-push',
1512- '--pristine-tar', '--no-interactive',
1513- '--no-symlink-orig',
1514- '--upstream-tag=%s/upstream/%s/%%(version)s%s' %
1515- (self.namespace, dist, ext)]
1516- cmd.extend(map(lambda x: '--component=%s' % x,
1517- list(orig_component_paths.keys()))
1518- )
1519- cmd.extend([orig_tarball_path,])
1520- run(cmd, env=self.local_repo.env)
1521- except CalledProcessError as e:
1522- raise GitUbuntuImportOrigError(
1523- 'Unable to import tarballs: %s' % orig_paths
1524- ) from e
1525- finally:
1526- os.chdir(oldcwd)
1527+ oldcwd = os.getcwd()
1528+ os.chdir(repo.local_dir)
1529+
1530+ if dl_cache is None:
1531+ workdir = os.path.join(repo.git_dir, CACHE_PATH)
1532+ else:
1533+ workdir = dl_cache
1534+
1535+ os.makedirs(workdir, exist_ok=True)
1536+
1537+ # now sets a global _PARENT_OVERRIDES
1538+ parse_parentfile(parentfile, pkgname)
1539+
1540+ only_debian, history_found = import_publishes(
1541+ repo=repo,
1542+ pkgname=pkgname,
1543+ namespace=namespace,
1544+ patches_applied=False,
1545+ debian_head_versions=debian_head_versions,
1546+ ubuntu_head_versions=ubuntu_head_versions,
1547+ debian_sinfo=debian_sinfo,
1548+ ubuntu_sinfo=ubuntu_sinfo,
1549+ active_series_only=active_series_only,
1550+ workdir=workdir,
1551+ fixup_devel=fixup_devel,
1552+ skip_orig=skip_orig,
1553+ )
1554+
1555+ if not history_found:
1556+ logging.error("No publication history for '%s' in debian or ubuntu. "
1557+ "Wrong source package name?", pkgname)
1558+ sys.exit(1)
1559+
1560+ if not skip_applied:
1561+ import_publishes(
1562+ repo=repo,
1563+ pkgname=pkgname,
1564+ namespace=namespace,
1565+ patches_applied=True,
1566+ debian_head_versions=applied_debian_head_versions,
1567+ ubuntu_head_versions=applied_ubuntu_head_versions,
1568+ debian_sinfo=debian_sinfo,
1569+ ubuntu_sinfo=ubuntu_sinfo,
1570+ active_series_only=active_series_only,
1571+ workdir=workdir,
1572+ fixup_devel=fixup_devel,
1573+ skip_orig=skip_orig,
1574+ )
1575
1576- self.local_repo.clean_repository_state()
1577+ os.chdir(oldcwd)
1578
1579- def import_patches_unapplied_tree(self, dsc_pathname):
1580- """Imports the patches-unapplied source package and writes the
1581- corresponding working tree for a given srcpkg
1582+ repo.garbage_collect()
1583
1584- Arguments:
1585- dsc_pathname - A string path to a DSC file
1586- """
1587+ if no_push:
1588+ logging.info('Not pushing to remote as specified')
1589+ else:
1590+ lp = launchpad_login_auth()
1591+ repo_path = '~%s/ubuntu/+source/%s/+git/%s' % (
1592+ owner,
1593+ pkgname,
1594+ pkgname,
1595+ )
1596+ lp_git_repo = lp.git_repositories.getByPath(path=repo_path)
1597+ if reimport:
1598+ if not lp_git_repo.canBeDeleted():
1599+ logging.warning(
1600+ "There are linked MPs to the pkg repository, "
1601+ "which will be deleted:"
1602+ )
1603+ for mp in lp_git_repo.landing_candidates:
1604+ logging.warning(
1605+ "Adding comment to %s indicating reimport",
1606+ mp.web_link
1607+ )
1608+ mp.createComment(
1609+ content="%s has been reimported and this MP "
1610+ "must be manually resubmitted." %
1611+ pkgname,
1612+ subject="%s has been reimported" % pkgname,
1613+ )
1614+ # repo deletion will delete the MP, but hopefully
1615+ # the above comment will be sufficient to explain
1616+ # why
1617+ lp_git_repo.lp_delete()
1618+
1619+ repo.git_run([
1620+ 'push', '--atomic', 'pkg',
1621+ 'refs/heads/%s/*:refs/heads/*' % namespace,
1622+ 'refs/tags/%s/*:refs/tags/*' % namespace,
1623+ ])
1624+ # update our reference
1625+ lp_git_repo = lp.git_repositories.getByPath(path=repo_path)
1626+ for i in range(retries):
1627+ try:
1628+ if only_debian:
1629+ lp_git_repo.default_branch = 'refs/heads/debian/sid'
1630+ else:
1631+ lp_git_repo.default_branch = 'refs/heads/ubuntu/devel'
1632+ lp_git_repo.lp_save()
1633+ break
1634+ except (NotFound, PreconditionFailed) as e:
1635+ time.sleep(retry_backoffs[i])
1636+ lp_git_repo.lp_refresh()
1637+
1638+
1639+def get_changelog_for_commit(
1640+ repo,
1641+ tree_hash,
1642+ publish_parent_commit,
1643+ changelog_parent_commit,
1644+):
1645+ """Extract changes to debian/changelog in this publish
1646+
1647+ LP: #1633114 -- favor the changelog parent's diff
1648+ """
1649+ raw_clog_entry = b''
1650+ if changelog_parent_commit is not None:
1651+ cmd = ['diff-tree', '-p', changelog_parent_commit,
1652+ tree_hash, '--', 'debian/changelog']
1653+ raw_clog_entry = repo.git_run(cmd).stdout
1654+ elif publish_parent_commit is not None:
1655+ cmd = ['diff-tree', '-p', publish_parent_commit,
1656+ tree_hash, '--', 'debian/changelog']
1657+ raw_clog_entry = repo.git_run(cmd).stdout
1658+
1659+ changelog_entry = b''
1660+ changelog_entry_found = False
1661+ i = 0
1662+ for line in raw_clog_entry.split(b'\n'):
1663+ # skip the header lines
1664+ if i < 4:
1665+ i += 1
1666+ continue
1667+ if not line.startswith(b'+'):
1668+ # in case there are stray changelog changes, just take
1669+ # the first complete section of changelog additions
1670+ if changelog_entry_found:
1671+ break
1672+ continue
1673+ line = re.sub(b'^[+]', b'', line)
1674+ if not line.startswith(b' '):
1675+ continue
1676+ if not changelog_entry_found:
1677+ changelog_entry = b'\n\nNew changelog entries:\n'
1678+ changelog_entry += line + b'\n'
1679+ changelog_entry_found = True
1680+ return changelog_entry
1681+
1682+def _commit_import(repo, spi, tree_hash, namespace,
1683+ publish_parent_commit,
1684+ changelog_parent_commit, upload_parent_commit,
1685+ unapplied_parent_commit
1686+):
1687+ """Commit a tree object into the repository with the specified
1688+ parents
1689+
1690+ The importer algorithm works by 'staging' the import of a
1691+ publication as a tree, then using that tree to determine the
1692+ appropriate parents from the publishing history and
1693+ debian/changelog.
1694+
1695+ Arguments:
1696+ spi - A SourcePackageInformation instance for this publish
1697+ tree_hash - SHA1 from git-dsc-commit --tree-only of this publish
1698+ publish_parent_commit = SHA1 of publishing parent
1699+ changelog_parent_commit = SHA1 of changelog parent
1700+ upload_parent_commit = SHA1 of upload parent
1701+ unapplied_parent_commit = SHA1 of unapplied-patches import parent
1702+ """
1703+ tag = None
1704+
1705+ if unapplied_parent_commit:
1706+ import_type = 'patches-applied'
1707+ target_head_name = spi.applied_head_name(namespace)
1708+ if repo.get_applied_tag(spi.version, namespace) is None:
1709+ # Not imported before
1710+ tag = applied_tag(spi.version, namespace)
1711+ else:
1712+ import_type = 'patches-unapplied'
1713+ target_head_name = spi.head_name(namespace)
1714+ if repo.get_import_tag(spi.version, namespace) is None:
1715+ # Not imported before
1716+ tag = import_tag(spi.version, namespace)
1717+
1718+ # Do not show importer/ namespace to user
1719+ _, _, pretty_head_name = target_head_name.partition('%s/' % namespace)
1720+
1721+ changelog_entry = get_changelog_for_commit(
1722+ repo,
1723+ tree_hash,
1724+ publish_parent_commit,
1725+ changelog_parent_commit
1726+ )
1727+ msg = (b'Import %s version %b to %b\n\nImported using git-ubuntu import.' %
1728+ (import_type.encode(), spi.version.encode(), pretty_head_name.encode())
1729+ )
1730+
1731+ parents = []
1732+
1733+ if publish_parent_commit is None and \
1734+ changelog_parent_commit is None and \
1735+ upload_parent_commit is None and \
1736+ unapplied_parent_commit is None and \
1737+ target_head_name in repo.local_branch_names:
1738+ # No parents and not creating a new branch
1739+ tag = orphan_tag(spi.version, namespace)
1740+ else:
1741+ msg += b'\n'
1742+
1743+ if publish_parent_commit is not None:
1744+ parents.append(publish_parent_commit)
1745+ msg += b'\nPublish parent: %b' % publish_parent_commit.encode()
1746+ if changelog_parent_commit is not None:
1747+ parents.append(changelog_parent_commit)
1748+ msg += b'\nChangelog parent: %b' % changelog_parent_commit.encode()
1749+ if upload_parent_commit is not None:
1750+ parents.append(upload_parent_commit)
1751+ msg += b'\nUpload parent: %b' % upload_parent_commit.encode()
1752+ if unapplied_parent_commit is not None:
1753+ parents.append(unapplied_parent_commit)
1754+ msg += b'\nUnapplied parent: %b' % unapplied_parent_commit.encode()
1755+
1756+ msg += b'%b' % changelog_entry
1757+
1758+ commit_hash = repo.commit_tree_hash(
1759+ tree_hash,
1760+ parents,
1761+ msg,
1762+ environment_spi=spi
1763+ )
1764+
1765+ repo.update_head_to_commit(target_head_name, commit_hash)
1766+
1767+ logging.debug('Committed %s import of %s as %s in %s',
1768+ import_type, spi.version, commit_hash, pretty_head_name
1769+ )
1770
1771- import_tree_hash = dsc_to_tree_hash(self.local_repo, dsc_pathname)
1772- self.local_repo.clean_repository_state()
1773+ if tag is not None:
1774+ # should be annotated to use create_tag API
1775+ logging.debug('Creating tag %s pointing to %s', tag, commit_hash)
1776+ repo.git_run(['tag', '-a', '-m', 'git-ubuntu import v%s' % VERSION,
1777+ tag, commit_hash
1778+ ]
1779+ )
1780
1781- return import_tree_hash
1782+def commit_unapplied_patches_import(
1783+ repo,
1784+ spi,
1785+ import_tree_hash,
1786+ namespace,
1787+ publish_parent_commit,
1788+ changelog_parent_commit,
1789+ upload_parent_commit,
1790+):
1791+ _commit_import(
1792+ repo,
1793+ spi,
1794+ import_tree_hash,
1795+ namespace,
1796+ publish_parent_commit,
1797+ changelog_parent_commit,
1798+ upload_parent_commit,
1799+ # unapplied trees do not have a distinct unapplied parent
1800+ None,
1801+ )
1802+
1803+def commit_applied_patches_import(
1804+ repo,
1805+ spi,
1806+ import_tree_hash,
1807+ namespace,
1808+ publish_parent_commit,
1809+ changelog_parent_commit,
1810+ unapplied_parent_commit
1811+):
1812+ _commit_import(
1813+ repo,
1814+ spi,
1815+ import_tree_hash,
1816+ namespace,
1817+ publish_parent_commit,
1818+ changelog_parent_commit,
1819+ # uploads will be unapplied trees currently, so applied trees
1820+ # can not have them as direct parents
1821+ None,
1822+ unapplied_parent_commit,
1823+ )
1824+
1825+def import_dsc(repo, dsc, namespace, version, dist):
1826+ """Imports the dsc file to importer/{debian,ubuntu}/dsc
1827+
1828+ Arguments:
1829+ spi - A GitUbuntuSourcePackageInformation instance
1830+ """
1831+
1832+ # Add DSC file to the appropriate branch
1833+ runq(['git', 'checkout', '%s/importer/%s/dsc' % (namespace, dist)], env=repo.env)
1834+ # It is possible the same-named DSC is published multiple times
1835+ # with different contents, e.g. on epoch bumps
1836+ local_dir = os.path.join(repo.local_dir, version)
1837+ os.makedirs(local_dir, exist_ok=True)
1838+ shutil.copy(dsc.dsc_path, local_dir)
1839+ repo.git_run(['add', repo.local_dir])
1840+ try:
1841+ # Anything to commit? (this will be 0 if, for instance, the
1842+ # same dsc is being imported again)
1843+ runq(['git', 'diff', '--exit-code', 'HEAD'], env=repo.env)
1844+ except:
1845+ repo.git_run(['commit', '-m', 'DSC file for %s' % version])
1846+
1847+ repo.clean_repository_state()
1848+
1849+def orig_imported(repo, orig_tarball_paths, namespace, dist):
1850+ """Determines if a list of paths to tarballs have already been imported to @dist
1851+
1852+ Assumes that the pristine-tar branch exists
1853+ """
1854+ try:
1855+ return repo.verify_pristine_tar(
1856+ orig_tarball_paths,
1857+ dist,
1858+ namespace,
1859+ )
1860+ except PristineTarError as e:
1861+ raise GitUbuntuImportOrigError from e
1862+
1863+def import_orig(repo, dsc, namespace, version, dist):
1864+ """Imports the orig-tarball using gbp import-org --pristine-tar
1865+
1866+ Arguments:
1867+ repo - a GitUbuntuRepository object
1868+ dsc - a GitUbuntuDsc object
1869+ namespace - the string namespace in which to search for tags, etc.
1870+ version - the string package version
1871+ dist - the string distribution, either ubuntu or debian, to import to
1872+ """
1873+ try:
1874+ orig_tarball_path = dsc.orig_tarball_path
1875+ except GitUbuntuDscError as e:
1876+ raise GitUbuntuImportOrigError(
1877+ 'Unable to import orig tarball in DSC for version %s' % version
1878+ ) from e
1879+
1880+ if orig_tarball_path is None:
1881+ # native packages only have a .tar.*
1882+ native_re = re.compile(r'.*\.tar\.[^.]+$')
1883+ for entry in dsc['Files']:
1884+ if native_re.match(entry['name']):
1885+ return
1886+ raise GitUbuntuImportOrigError('No orig tarball in '
1887+ 'DSC for version %s.' % version
1888+ )
1889
1890- def import_patches_applied_tree(self, dsc_pathname):
1891- """Imports the patches-applied source package and writes the
1892- corresponding working tree for a given srcpkg, patch by patch
1893+ try:
1894+ orig_component_paths = dsc.component_tarball_paths
1895+ except GitUbuntuDscError as e:
1896+ raise GitUbuntuImportOrigError(
1897+ 'Unable to import component tarballs in DSC for version '
1898+ '%s' % version
1899+ ) from e
1900+
1901+ orig_paths = [orig_tarball_path] + list(orig_component_paths.values())
1902+ if (not orig_tarball_path or
1903+ orig_imported(repo, orig_paths, namespace, dist)
1904+ ):
1905+ return
1906+ if not all(map(os.path.exists, orig_paths)):
1907+ raise GitUbuntuImportOrigError('Unable to find tarball: '
1908+ '%s' % [p for p in orig_paths if not os.path.exists(p)])
1909+
1910+ try:
1911+ with repo.pristine_tar_branches(dist, namespace):
1912+ oldcwd = os.getcwd()
1913+ # gbp does not support running from arbitrary git trees or
1914+ # working directories
1915+ # https://github.com/agx/git-buildpackage/pull/16
1916+ os.chdir(repo.local_dir)
1917+
1918+ ext = os.path.splitext(dsc.orig_tarball_path)[1]
1919+
1920+ # make this non-fatal if upstream already exist as tagged?
1921+ cmd = ['gbp', 'import-orig', '--no-merge',
1922+ '--upstream-branch', 'do-not-push',
1923+ '--pristine-tar', '--no-interactive',
1924+ '--no-symlink-orig',
1925+ '--upstream-tag=%s/upstream/%s/%%(version)s%s' %
1926+ (namespace, dist, ext)]
1927+ cmd.extend(map(lambda x: '--component=%s' % x,
1928+ list(orig_component_paths.keys()))
1929+ )
1930+ cmd.extend([orig_tarball_path,])
1931+ run(cmd, env=repo.env)
1932+ except CalledProcessError as e:
1933+ raise GitUbuntuImportOrigError(
1934+ 'Unable to import tarballs: %s' % orig_paths
1935+ ) from e
1936+ finally:
1937+ os.chdir(oldcwd)
1938
1939- Arguments:
1940- dsc_pathname - A string path to a DSC file
1941- """
1942- oldcwd = os.getcwd()
1943+ repo.clean_repository_state()
1944
1945- extract_dir = tempfile.mkdtemp()
1946- os.chdir(extract_dir)
1947+def import_patches_unapplied_tree(repo, dsc_pathname):
1948+ """Imports the patches-unapplied source package and writes the
1949+ corresponding working tree for a given srcpkg
1950
1951- run(['dpkg-source', '-x',
1952- '--skip-patches',
1953- os.path.join(oldcwd, dsc_pathname)
1954- ]
1955- )
1956+ Arguments:
1957+ dsc_pathname - A string path to a DSC file
1958+ """
1959
1960- extracted_dir = None
1961- for path in os.listdir(extract_dir):
1962- if os.path.isdir(path):
1963- extracted_dir = os.path.join(extract_dir, path)
1964- break
1965- if extracted_dir is None:
1966- logging.exception('No source extracted?')
1967- raise SourceExtractionException("Failed to find an extracted "
1968- "directory from dpkg-source -x")
1969-
1970- if os.path.isdir(os.path.join(extracted_dir, '.pc')):
1971- self.local_repo.git_run(
1972- ['--work-tree', extracted_dir, 'add', '-f', '-A']
1973- )
1974- self.local_repo.git_run(
1975- ['--work-tree', extracted_dir, 'rm', '-r', '-f', '.pc']
1976- )
1977- cp = self.local_repo.git_run(
1978- ['--work-tree', extracted_dir, 'write-tree']
1979- )
1980- import_tree_hash = decode_binary(cp.stdout).strip()
1981- yield (
1982- import_tree_hash,
1983- None,
1984- 'Remove .pc directory from source package',
1985- )
1986+ import_tree_hash = dsc_to_tree_hash(repo, dsc_pathname)
1987+ repo.clean_repository_state()
1988
1989- try:
1990- try:
1991- cp = run(['dpkg-source', '--print-format', extracted_dir])
1992- fmt = decode_binary(cp.stdout).strip()
1993- if '3.0 (quilt)' not in fmt:
1994- raise StopIteration()
1995- except CalledProcessError as e:
1996- try:
1997- with open('debian/source/format', 'r') as f:
1998- for line in f:
1999- if re.match(r'3.0 (.*)', line):
2000- break
2001- else:
2002- raise StopIteration()
2003- # `man dpkg-source` indicates no d/s/format implies 1.0
2004- except OSError:
2005- raise StopIteration()
2006-
2007- while True:
2008- try:
2009- os.chdir(extracted_dir)
2010- run(['quilt', 'push'], rcs=[2])
2011- cp = run(['quilt', 'top'])
2012- patch_name = decode_binary(cp.stdout).strip()
2013- cp = run(['quilt', 'header'])
2014- header = decode_binary(cp.stdout).strip()
2015- patch_desc = None
2016- for regex in (r'Subject:\s*(.*?)$',
2017- r'Description:\s*(.*?)$'
2018- ):
2019- try:
2020- m = re.search(regex, header, re.MULTILINE)
2021- patch_desc = m.group(1)
2022- except AttributeError:
2023- pass
2024- if patch_desc is None:
2025- logging.debug('Unable to find Subject or Description in %s' % patch_name)
2026- self.local_repo.git_run(['--work-tree', extracted_dir, 'add', '-f', '-A'])
2027- self.local_repo.git_run(['--work-tree', extracted_dir, 'reset', 'HEAD', '--', '.pc'])
2028- cp = self.local_repo.git_run(['--work-tree', extracted_dir, 'write-tree'])
2029- import_tree_hash = decode_binary(cp.stdout).strip()
2030-
2031- yield (import_tree_hash, patch_name, patch_desc)
2032- except CalledProcessError as e:
2033- # quilt returns 2 when done pushing
2034- if e.returncode != 2:
2035- raise
2036- raise StopIteration()
2037- finally:
2038- os.chdir(oldcwd)
2039- shutil.rmtree(extract_dir)
2040- self.local_repo.clean_repository_state()
2041+ return import_tree_hash
2042
2043- def parse_parentfile(self, parentfile):
2044- """Extract parent overrides from a file
2045+def import_patches_applied_tree(repo, dsc_pathname):
2046+ """Imports the patches-applied source package and writes the
2047+ corresponding working tree for a given srcpkg, patch by patch
2048
2049- The parent overrides file specifies parent-child relationship(s),
2050- where the importer may not be able to determine them automatically.
2051- The format of the file is:
2052- <pkgname> <child version> <publish parent version> <changelog parent version>
2053- with one override per line.
2054+ Arguments:
2055+ dsc_pathname - A string path to a DSC file
2056+ """
2057+ oldcwd = os.getcwd()
2058
2059- <child version> is the published version to which this override
2060- applies.
2061+ extract_dir = tempfile.mkdtemp()
2062+ os.chdir(extract_dir)
2063
2064- The <publish parent version> is the prior version published in the
2065- same series/pocket, or the immediately preceding series/pocket, if
2066- this is the first publish in a series/pocket.
2067+ run(['dpkg-source', '-x',
2068+ '--skip-patches',
2069+ os.path.join(oldcwd, dsc_pathname)
2070+ ]
2071+ )
2072
2073- The <changelog parent version> is the prior version in the
2074- child publish's debian/changelog.
2075+ extracted_dir = None
2076+ for path in os.listdir(extract_dir):
2077+ if os.path.isdir(path):
2078+ extracted_dir = os.path.join(extract_dir, path)
2079+ break
2080+ if extracted_dir is None:
2081+ logging.exception('No source extracted?')
2082+ raise SourceExtractionException("Failed to find an extracted "
2083+ "directory from dpkg-source -x")
2084
2085- Keyword Arguments:
2086- parentfile -- Path to parent overrides file
2087- """
2088- if self.parent_overrides:
2089- return
2090- parent_overrides = dict()
2091+ if os.path.isdir(os.path.join(extracted_dir, '.pc')):
2092+ repo.git_run(
2093+ ['--work-tree', extracted_dir, 'add', '-f', '-A']
2094+ )
2095+ repo.git_run(
2096+ ['--work-tree', extracted_dir, 'rm', '-r', '-f', '.pc']
2097+ )
2098+ cp = repo.git_run(
2099+ ['--work-tree', extracted_dir, 'write-tree']
2100+ )
2101+ import_tree_hash = decode_binary(cp.stdout).strip()
2102+ yield (
2103+ import_tree_hash,
2104+ None,
2105+ 'Remove .pc directory from source package',
2106+ )
2107+
2108+ try:
2109 try:
2110- with open(parentfile) as f:
2111- for line in f:
2112- if line.startswith('#'):
2113- continue
2114- m = re.match(
2115- r'(?P<pkgname>\S*)\s*(?P<childversion>\S*)\s*(?P<parent1version>\S*)\s*(?P<parent2version>\S*)',
2116- line
2117- )
2118- if m is None:
2119- continue
2120- if m.group('pkgname') != self.pkgname:
2121- continue
2122- parent_overrides[m.group('childversion')] = {
2123- 'publish_parent':m.group('parent1version'),
2124- 'changelog_parent':m.group('parent2version')
2125- }
2126- except FileNotFoundError:
2127- pass
2128- self.parent_overrides = parent_overrides
2129+ cp = run(['dpkg-source', '--print-format', extracted_dir])
2130+ fmt = decode_binary(cp.stdout).strip()
2131+ if '3.0 (quilt)' not in fmt:
2132+ raise StopIteration()
2133+ except CalledProcessError as e:
2134+ try:
2135+ with open('debian/source/format', 'r') as f:
2136+ for line in f:
2137+ if re.match(r'3.0 (.*)', line):
2138+ break
2139+ else:
2140+ raise StopIteration()
2141+ # `man dpkg-source` indicates no d/s/format implies 1.0
2142+ except OSError:
2143+ raise StopIteration()
2144
2145- def override_parents(self, spi):
2146- unapplied_publish_parent_commit = None
2147- unapplied_changelog_parent_commit = None
2148- applied_publish_parent_commit = None
2149- applied_changelog_parent_commit = None
2150+ while True:
2151+ try:
2152+ os.chdir(extracted_dir)
2153+ run(['quilt', 'push'], rcs=[2])
2154+ cp = run(['quilt', 'top'])
2155+ patch_name = decode_binary(cp.stdout).strip()
2156+ cp = run(['quilt', 'header'])
2157+ header = decode_binary(cp.stdout).strip()
2158+ patch_desc = None
2159+ for regex in (r'Subject:\s*(.*?)$',
2160+ r'Description:\s*(.*?)$'
2161+ ):
2162+ try:
2163+ m = re.search(regex, header, re.MULTILINE)
2164+ patch_desc = m.group(1)
2165+ except AttributeError:
2166+ pass
2167+ if patch_desc is None:
2168+ logging.debug('Unable to find Subject or Description '
2169+ 'in %s' % patch_name
2170+ )
2171+ repo.git_run([
2172+ '--work-tree',
2173+ extracted_dir,
2174+ 'add',
2175+ '-f',
2176+ '-A',
2177+ ])
2178+ repo.git_run([
2179+ '--work-tree',
2180+ extracted_dir,
2181+ 'reset',
2182+ 'HEAD',
2183+ '--',
2184+ '.pc',
2185+ ])
2186+ cp = repo.git_run(['--work-tree', extracted_dir, 'write-tree'])
2187+ import_tree_hash = decode_binary(cp.stdout).strip()
2188+
2189+ yield (import_tree_hash, patch_name, patch_desc)
2190+ except CalledProcessError as e:
2191+ # quilt returns 2 when done pushing
2192+ if e.returncode != 2:
2193+ raise
2194+ raise StopIteration()
2195+ finally:
2196+ os.chdir(oldcwd)
2197+ shutil.rmtree(extract_dir)
2198+ repo.clean_repository_state()
2199+
2200+_PARENT_OVERRIDES = None
2201+def parse_parentfile(parentfile, pkgname):
2202+ """Extract parent overrides from a file
2203+
2204+ The parent overrides file specifies parent-child relationship(s),
2205+ where the importer may not be able to determine them automatically.
2206+ The format of the file is:
2207+ <pkgname> <child version> <publish parent version> <changelog parent version>
2208+ with one override per line.
2209+
2210+ <child version> is the published version to which this override
2211+ applies.
2212+
2213+ The <publish parent version> is the prior version published in the
2214+ same series/pocket, or the immediately preceding series/pocket, if
2215+ this is the first publish in a series/pocket.
2216+
2217+ The <changelog parent version> is the prior version in the
2218+ child publish's debian/changelog.
2219+
2220+ Keyword Arguments:
2221+ parentfile -- Path to parent overrides file
2222+ """
2223+ global _PARENT_OVERRIDES
2224+ if _PARENT_OVERRIDES:
2225+ return
2226+ _PARENT_OVERRIDES = dict()
2227+ try:
2228+ with open(parentfile) as f:
2229+ for line in f:
2230+ if line.startswith('#'):
2231+ continue
2232+ m = re.match(
2233+ r'(?P<pkgname>\S*)\s*(?P<childversion>\S*)\s*(?P<parent1version>\S*)\s*(?P<parent2version>\S*)',
2234+ line
2235+ )
2236+ if m is None:
2237+ continue
2238+ if m.group('pkgname') != pkgname:
2239+ continue
2240+ _PARENT_OVERRIDES[m.group('childversion')] = {
2241+ 'publish_parent':m.group('parent1version'),
2242+ 'changelog_parent':m.group('parent2version')
2243+ }
2244+ except FileNotFoundError:
2245+ pass
2246+
2247+def override_parents(repo, spi, namespace):
2248+ unapplied_publish_parent_commit = None
2249+ unapplied_changelog_parent_commit = None
2250+ applied_publish_parent_commit = None
2251+ applied_changelog_parent_commit = None
2252+
2253+ unapplied_publish_parent_tag = repo.get_import_tag(
2254+ _PARENT_OVERRIDES[spi.version]['publish_parent'],
2255+ namespace
2256+ )
2257+ applied_publish_parent_tag = repo.get_applied_tag(
2258+ _PARENT_OVERRIDES[spi.version]['publish_parent'],
2259+ namespace
2260+ )
2261+ if unapplied_publish_parent_tag is not None:
2262+ # sanity check that version from d/changelog of the
2263+ # tagged commit matches ours
2264+ parent_publish_version, _ = \
2265+ repo.get_changelog_versions_from_treeish(
2266+ str(unapplied_publish_parent_tag.peel().id),
2267+ )
2268+ if parent_publish_version != _PARENT_OVERRIDES[spi.version]['publish_parent']:
2269+ logging.error('Found a tag corresponding to publish parent '
2270+ 'override version %s, but d/changelog version (%s) '
2271+ 'differs. Will orphan commit.',
2272+ _PARENT_OVERRIDES[spi.version]['publish_parent'],
2273+ parent_publish_version
2274+ )
2275+ else:
2276+ unapplied_publish_parent_commit = str(unapplied_publish_parent_tag.peel().id)
2277+ if applied_publish_parent_tag is not None:
2278+ applied_publish_parent_commit = str(applied_publish_parent_tag.peel().id)
2279+ logging.debug('Overriding publish parent (tag) to %s',
2280+ repo.tag_to_pretty_name(unapplied_publish_parent_tag)
2281+ )
2282+ else:
2283+ if _PARENT_OVERRIDES[spi.version]['publish_parent'] == '-':
2284+ logging.debug('Not setting publish parent as '
2285+ 'specified in override file.'
2286+ )
2287+ else:
2288+ logging.error('Specified publish parent override '
2289+ '(%s) for version (%s) not found in tags. '
2290+ 'Unable to proceed.' % (
2291+ _PARENT_OVERRIDES[spi.version]['publish_parent'],
2292+ spi.version
2293+ )
2294+ )
2295+ sys.exit(1)
2296
2297- unapplied_publish_parent_tag = self.local_repo.get_import_tag(
2298- self.parent_overrides[spi.version]['publish_parent'],
2299- self.namespace)
2300- applied_publish_parent_tag = self.local_repo.get_applied_tag(
2301- self.parent_overrides[spi.version]['publish_parent'],
2302- self.namespace)
2303- if unapplied_publish_parent_tag is not None:
2304- # sanity check that version from d/changelog of the
2305- # tagged commit matches ours
2306- parent_publish_version, _ = \
2307- self.local_repo.get_changelog_versions_from_treeish(
2308- str(unapplied_publish_parent_tag.peel().id),
2309- )
2310- if parent_publish_version != self.parent_overrides[spi.version]['publish_parent']:
2311- logging.error('Found a tag corresponding to '
2312- 'publish parent override '
2313- 'version %s, but d/changelog '
2314- 'version (%s) differs. Will '
2315- 'orphan commit.' % (
2316- self.parent_overrides[spi.version]['publish_parent'],
2317- parent_publish_version
2318- )
2319- )
2320- else:
2321- unapplied_publish_parent_commit = str(unapplied_publish_parent_tag.peel().id)
2322- if applied_publish_parent_tag is not None:
2323- applied_publish_parent_commit = str(applied_publish_parent_tag.peel().id)
2324- logging.debug('Overriding publish parent (tag) to %s',
2325- self.local_repo.tag_to_pretty_name(unapplied_publish_parent_tag)
2326- )
2327+ unapplied_changelog_parent_tag = repo.get_import_tag(
2328+ _PARENT_OVERRIDES[spi.version]['changelog_parent'],
2329+ namespace
2330+ )
2331+ applied_changelog_parent_tag = repo.get_applied_tag(
2332+ _PARENT_OVERRIDES[spi.version]['changelog_parent'],
2333+ namespace
2334+ )
2335+ if unapplied_changelog_parent_tag is not None:
2336+ # sanity check that version from d/changelog of the
2337+ # tagged commit matches ours
2338+ parent_changelog_version, _ = repo.get_changelog_versions_from_treeish(
2339+ str(unapplied_changelog_parent_tag.peel().id),
2340+ )
2341+ if parent_changelog_version != _PARENT_OVERRIDES[spi.version]['changelog_parent']:
2342+ logging.error('Found a tag corresponding to '
2343+ 'changelog parent override '
2344+ 'version %s, but d/changelog '
2345+ 'version (%s) differs. Will '
2346+ 'orphan commit.' % (
2347+ _PARENT_OVERRIDES[spi.version]['changelog_parent'],
2348+ parent_changelog_version
2349+ )
2350+ )
2351 else:
2352- if self.parent_overrides[spi.version]['publish_parent'] == '-':
2353- logging.debug('Not setting publish parent as '
2354- 'specified in override file.'
2355- )
2356- else:
2357- logging.error('Specified publish parent override '
2358- '(%s) for version (%s) not found in tags. '
2359- 'Unable to proceed.' % (
2360- self.parent_overrides[spi.version]['publish_parent'],
2361- spi.version
2362- )
2363- )
2364- sys.exit(1)
2365-
2366- unapplied_changelog_parent_tag = self.local_repo.get_import_tag(
2367- self.parent_overrides[spi.version]['changelog_parent'],
2368- self.namespace)
2369- applied_changelog_parent_tag = self.local_repo.get_applied_tag(
2370- self.parent_overrides[spi.version]['changelog_parent'],
2371- self.namespace)
2372- if unapplied_changelog_parent_tag is not None:
2373- # sanity check that version from d/changelog of the
2374- # tagged commit matches ours
2375- parent_changelog_version, _ = \
2376- self.local_repo.get_changelog_versions_from_treeish(
2377- str(unapplied_changelog_parent_tag.peel().id),
2378- )
2379- if parent_changelog_version != self.parent_overrides[spi.version]['changelog_parent']:
2380- logging.error('Found a tag corresponding to '
2381- 'changelog parent override '
2382- 'version %s, but d/changelog '
2383- 'version (%s) differs. Will '
2384- 'orphan commit.' % (
2385- self.parent_overrides[spi.version]['changelog_parent'],
2386- parent_changelog_version
2387- )
2388- )
2389- else:
2390- unapplied_changelog_parent_commit = str(unapplied_changelog_parent_tag.peel().id)
2391- if applied_changelog_parent_tag is not None:
2392- applied_changelog_parent_commit = str(applied_changelog_parent_tag.peel().id)
2393- logging.debug('Overriding changelog parent (tag) to %s',
2394- self.local_repo.tag_to_pretty_name(unapplied_changelog_parent_tag)
2395- )
2396+ unapplied_changelog_parent_commit = str(unapplied_changelog_parent_tag.peel().id)
2397+ if applied_changelog_parent_tag is not None:
2398+ applied_changelog_parent_commit = str(applied_changelog_parent_tag.peel().id)
2399+ logging.debug('Overriding changelog parent (tag) to %s',
2400+ repo.tag_to_pretty_name(unapplied_changelog_parent_tag)
2401+ )
2402+ else:
2403+ if _PARENT_OVERRIDES[spi.version]['changelog_parent'] == '-':
2404+ logging.debug('Not setting changelog parent as specified '
2405+ 'in override file.'
2406+ )
2407 else:
2408- if self.parent_overrides[spi.version]['changelog_parent'] == '-':
2409- logging.debug('Not setting changelog parent as '
2410- 'specified in override file.'
2411- )
2412- else:
2413- logging.error('Specified changelog parent override '
2414- '(%s) for version (%s) not found in tags. '
2415- 'Unable to proceed.' % (
2416- self.parent_overrides[spi.version]['changelog_parent'],
2417- spi.version
2418- )
2419- )
2420- sys.exit(1)
2421- return (unapplied_publish_parent_commit,
2422- unapplied_changelog_parent_commit,
2423- applied_publish_parent_commit,
2424- applied_changelog_parent_commit)
2425-
2426- def _update_devel_branches(self, namespace, spi=None):
2427- if spi:
2428- # we do not maintain -devel pointers for debian
2429- if str(spi.distribution.name.lower()) != "ubuntu":
2430+ logging.error('Specified changelog parent override (%s) '
2431+ 'for version (%s) not found in tags. Unable to proceed.',
2432+ _PARENT_OVERRIDES[spi.version]['changelog_parent'],
2433+ spi.version
2434+ )
2435+ sys.exit(1)
2436+ return (
2437+ unapplied_publish_parent_commit,
2438+ unapplied_changelog_parent_commit,
2439+ applied_publish_parent_commit,
2440+ applied_changelog_parent_commit,
2441+ )
2442+
2443+def _update_devel_branches(
2444+ repo,
2445+ namespace,
2446+ pkgname,
2447+ ubuntu_sinfo,
2448+ spi=None,
2449+):
2450+ if spi:
2451+ # we do not maintain -devel pointers for debian
2452+ if str(spi.distribution.name.lower()) != "ubuntu":
2453+ return
2454+ for devel_head in (
2455+ '%s/ubuntu/%s-devel' % (namespace, spi.series.name.lower()),
2456+ '%s/ubuntu/devel' % namespace
2457+ ):
2458+ try:
2459+ devel_head_hash = repo.head_to_commit(devel_head)
2460+ devel_head_version, _ = repo.get_changelog_versions_from_treeish(devel_head_hash)
2461+ except AttributeError:
2462+ devel_head_version = None
2463+ if version_compare(spi.version, devel_head_version) <= 0:
2464 return
2465- for devel_head in (
2466- '%s/ubuntu/%s-devel' % (namespace, spi.series.name.lower()),
2467- '%s/ubuntu/devel' % namespace
2468- ):
2469- try:
2470- devel_head_hash = self.local_repo.head_to_commit(devel_head)
2471- devel_head_version, _ = self.local_repo.get_changelog_versions_from_treeish(devel_head_hash)
2472- except AttributeError:
2473- devel_head_version = None
2474- if version_compare(spi.version, devel_head_version) <= 0:
2475- return
2476- logging.debug("Updating %s to %s" % (devel_head,
2477- spi.head_name(namespace)))
2478- self.local_repo.merge_commit_to_devel_head(
2479- namespace, devel_head,
2480- self.local_repo.head_to_commit(spi.head_name(namespace)),
2481- environment_spi=spi)
2482+ logging.debug("Updating %s to %s" % (devel_head,
2483+ spi.head_name(namespace)))
2484+ repo.merge_commit_to_devel_head(
2485+ namespace,
2486+ devel_head,
2487+ repo.head_to_commit(spi.head_name(namespace)),
2488+ environment_spi=spi,
2489+ )
2490+ else:
2491+ ubuntu_head_versions = repo.get_heads_and_versions(
2492+ 'ubuntu',
2493+ namespace,
2494+ )
2495+ devel_hash = None
2496+ devel_name = None
2497+ for rel in ubuntu_sinfo.active_series_name_list:
2498+ series_devel_hash = None
2499+ series_devel_name = None
2500+ series_devel_version = None
2501+ # Find highest version publish in these pockets
2502+ for suff in ("-proposed", "-updates", "-security", ""):
2503+ name = "%s/ubuntu/%s%s" % (namespace, rel, suff)
2504+ if name in ubuntu_head_versions and version_compare(
2505+ ubuntu_head_versions[name]['version'],
2506+ series_devel_version
2507+ ) > 0:
2508+ series_devel_name = name
2509+ series_devel_version = ubuntu_head_versions[name]['version']
2510+ series_devel_hash = str(ubuntu_head_versions[name]['head'].peel().id)
2511+ if series_devel_hash:
2512+ if not devel_hash:
2513+ devel_hash = series_devel_hash
2514+ devel_name = series_devel_name
2515+ logging.debug("Updating %s/ubuntu/%s-devel branch to '%s' (%s)",
2516+ namespace, rel, series_devel_name, series_devel_hash)
2517+ repo.merge_commit_to_devel_head(namespace,
2518+ "%s/ubuntu/%s-devel" % (namespace, rel), series_devel_hash)
2519+ if devel_hash is None:
2520+ logging.warn("Package '%s' does not appear to have been published "
2521+ "in Ubuntu. No %s/ubuntu/devel branch created.",
2522+ pkgname, namespace)
2523 else:
2524- ubuntu_head_versions = self.local_repo.get_heads_and_versions(
2525- 'ubuntu', namespace
2526+ logging.debug("Setting %s/ubuntu/devel branch to '%s' (%s)",
2527+ namespace, devel_name, devel_hash)
2528+ repo.merge_commit_to_devel_head(
2529+ namespace, "%s/ubuntu/devel" % namespace, devel_hash
2530 )
2531- devel_hash = None
2532- devel_name = None
2533- for rel in self.ubuntu_sinfo.active_series_name_list:
2534- series_devel_hash = None
2535- series_devel_name = None
2536- series_devel_version = None
2537- # Find highest version publish in these pockets
2538- for suff in ("-proposed", "-updates", "-security", ""):
2539- name = "%s/ubuntu/%s%s" % (namespace, rel, suff)
2540- if name in ubuntu_head_versions and version_compare(
2541- ubuntu_head_versions[name]['version'],
2542- series_devel_version
2543- ) > 0:
2544- series_devel_name = name
2545- series_devel_version = ubuntu_head_versions[name]['version']
2546- series_devel_hash = str(ubuntu_head_versions[name]['head'].peel().id)
2547- if series_devel_hash:
2548- if not devel_hash:
2549- devel_hash = series_devel_hash
2550- devel_name = series_devel_name
2551- logging.debug("Updating %s/ubuntu/%s-devel branch to '%s' (%s)",
2552- namespace, rel, series_devel_name, series_devel_hash)
2553- self.local_repo.merge_commit_to_devel_head(namespace,
2554- "%s/ubuntu/%s-devel" % (namespace, rel), series_devel_hash)
2555- if devel_hash is None:
2556- logging.warn("Package '%s' does not appear to have been published "
2557- "in Ubuntu. No %s/ubuntu/devel branch created.",
2558- self.pkgname, namespace)
2559- else:
2560- logging.debug("Setting %s/ubuntu/devel branch to '%s' (%s)",
2561- namespace, devel_name, devel_hash)
2562- self.local_repo.merge_commit_to_devel_head(
2563- namespace, "%s/ubuntu/devel" % namespace, devel_hash
2564- )
2565-
2566- def update_devel_branches(self, spi=None):
2567- self._update_devel_branches(self.namespace, spi)
2568-
2569- def update_applied_devel_branches(self, spi=None):
2570- self._update_devel_branches('%s/applied' % self.namespace, spi)
2571-
2572- # imports a package based upon source package information
2573- def import_unapplied_spi(self, spi):
2574- """Imports a source package from Launchpad into the git
2575- repository
2576-
2577- Arguments:
2578- spi - a SourcePackageInformation instance
2579- """
2580- pretty_head_name = spi.pretty_head_name
2581
2582- logging.info('Importing patches-unapplied %s to %s' %
2583- (spi.version, pretty_head_name))
2584- unapplied_tip_head = self.local_repo.get_head_by_name(spi.head_name(self.namespace))
2585-
2586- spi.pull()
2587-
2588- self.local_repo.clean_repository_state()
2589-
2590- self.import_dsc(
2591+def update_devel_branches(
2592+ repo,
2593+ namespace,
2594+ pkgname,
2595+ ubuntu_sinfo,
2596+ spi=None,
2597+):
2598+ _update_devel_branches(
2599+ repo=repo,
2600+ namespace=namespace,
2601+ pkgname=pkgname,
2602+ ubuntu_sinfo=ubuntu_sinfo,
2603+ spi=spi,
2604+ )
2605+
2606+def update_applied_devel_branches(
2607+ repo,
2608+ namespace,
2609+ pkgname,
2610+ ubuntu_sinfo,
2611+ spi=None,
2612+):
2613+ _update_devel_branches(
2614+ repo=repo,
2615+ namespace='%s/applied' % namespace,
2616+ pkgname=pkgname,
2617+ ubuntu_sinfo=ubuntu_sinfo,
2618+ spi=spi,
2619+ )
2620+
2621+# imports a package based upon source package information
2622+def import_unapplied_spi(repo, spi, namespace, skip_orig, ubuntu_sinfo):
2623+ """Imports a source package from Launchpad into the git
2624+ repository
2625+
2626+ Arguments:
2627+ spi - a SourcePackageInformation instance
2628+ """
2629+ pretty_head_name = spi.pretty_head_name
2630+
2631+ logging.info('Importing patches-unapplied %s to %s',
2632+ spi.version, pretty_head_name
2633+ )
2634+ unapplied_tip_head = repo.get_head_by_name(spi.head_name(namespace))
2635+
2636+ spi.pull()
2637+
2638+ repo.clean_repository_state()
2639+
2640+ import_dsc(
2641+ repo,
2642+ GitUbuntuDsc(spi.dsc_pathname),
2643+ namespace,
2644+ str(spi.version),
2645+ spi.distribution.name.lower(),
2646+ )
2647+ if not skip_orig:
2648+ import_orig(
2649+ repo,
2650 GitUbuntuDsc(spi.dsc_pathname),
2651+ namespace,
2652 str(spi.version),
2653- spi.distribution.name.lower()
2654+ spi.distribution.name.lower(),
2655 )
2656- if not self.skip_orig:
2657- self.import_orig(spi)
2658
2659- unapplied_import_tree_hash = self.import_patches_unapplied_tree(spi.dsc_pathname)
2660- logging.debug('Imported patches-unapplied version %s as tree %s',
2661- spi.version, unapplied_import_tree_hash
2662- )
2663+ unapplied_import_tree_hash = import_patches_unapplied_tree(
2664+ repo,
2665+ spi.dsc_pathname
2666+ )
2667+ logging.debug(
2668+ "Imported patches-unapplied version %s as tree %s",
2669+ spi.version,
2670+ unapplied_import_tree_hash
2671+ )
2672+
2673+ import_tree_versions = repo.get_all_changelog_versions_from_treeish(
2674+ unapplied_import_tree_hash
2675+ )
2676+ changelog_version = import_tree_versions[0]
2677+
2678+ logging.debug(
2679+ "Found changelog version %s in newly imported tree" %
2680+ changelog_version
2681+ )
2682+
2683+ # sanity check on spph and d/changelog agreeing on the latest version
2684+ if spi.version not in _PARENT_OVERRIDES:
2685+ assert spi.version == changelog_version, (
2686+ 'source pkg version: {} != changelog version: {}'.format(
2687+ spi.version, changelog_version))
2688+
2689+ unapplied_tip_version = None
2690+ # check if the version to import has already been imported to
2691+ # this head
2692+ if unapplied_tip_head is not None:
2693+ if repo.treeishs_identical(
2694+ unapplied_import_tree_hash, str(unapplied_tip_head.peel().id)
2695+ ):
2696+ logging.warn('%s is identical to %s',
2697+ pretty_head_name, spi.version
2698+ )
2699+ return
2700+ unapplied_tip_version, _ = repo.get_changelog_versions_from_treeish(
2701+ str(unapplied_tip_head.peel().id),
2702+ )
2703
2704- import_tree_versions = self.local_repo.get_all_changelog_versions_from_treeish(
2705- unapplied_import_tree_hash
2706- )
2707- changelog_version = import_tree_versions[0]
2708+ logging.debug('Tip version is %s', unapplied_tip_version)
2709
2710+ unapplied_publish_parent_commit = None
2711+ unapplied_changelog_parent_commit = None
2712+ upload_parent_commit = None
2713+ unapplied_parent_commit = None
2714+
2715+ if spi.version in _PARENT_OVERRIDES:
2716 logging.debug(
2717- 'Found changelog version %s in newly imported tree' %
2718- changelog_version
2719- )
2720+ '%s is specified in the parent override file.',
2721+ spi.version
2722+ )
2723
2724- # sanity check on spph and d/changelog agreeing on the latest version
2725- if spi.version not in self.parent_overrides:
2726- assert spi.version == changelog_version, (
2727- 'source pkg version: {} != changelog version: {}'.format(
2728- spi.version, changelog_version))
2729-
2730- unapplied_tip_version = None
2731- # check if the version to import has already been imported to
2732- # this head
2733- if unapplied_tip_head is not None:
2734- if self.local_repo.treeishs_identical(
2735- unapplied_import_tree_hash, str(unapplied_tip_head.peel().id)
2736- ):
2737- logging.warn('%s is identical to %s',
2738- pretty_head_name, spi.version
2739+ (
2740+ unapplied_publish_parent_commit,
2741+ unapplied_changelog_parent_commit,
2742+ _,
2743+ _
2744+ ) = override_parents(repo, spi, namespace)
2745+ else:
2746+ # Get parent from publishing history (which is the last
2747+ # published version in the corresponding series)
2748+ try:
2749+ unapplied_publish_parent_commit = str(unapplied_tip_head.peel().id)
2750+ except AttributeError:
2751+ try:
2752+ unapplied_publish_parent_head = repo.get_head_by_name(spi.parent_head_name(namespace))
2753+ unapplied_publish_parent_commit = str(unapplied_publish_parent_head.peel().id)
2754+ except AttributeError:
2755+ pass
2756+ finally:
2757+ try:
2758+ unapplied_publish_parent_tag = repo.nearest_import_tag(unapplied_publish_parent_commit, namespace)
2759+ logging.debug('Publishing parent (tag) is %s',
2760+ repo.tag_to_pretty_name(unapplied_publish_parent_tag)
2761 )
2762- return
2763- unapplied_tip_version, _ = self.local_repo.get_changelog_versions_from_treeish(
2764- str(unapplied_tip_head.peel().id),
2765- )
2766+ except AttributeError:
2767+ pass
2768
2769- logging.debug('Tip version is %s', unapplied_tip_version)
2770+ if version_compare(str(spi.version), unapplied_tip_version) <= 0:
2771+ logging.warn('Version to import (%s) is not after %s tip (%s)',
2772+ spi.version, pretty_head_name, unapplied_tip_version
2773+ )
2774+
2775+ # Walk changelog backwards until we find an imported version
2776+ for version in import_tree_versions:
2777+ unapplied_changelog_parent_tag = repo.get_import_tag(version, namespace)
2778+ if unapplied_changelog_parent_tag is not None:
2779+ # sanity check that version from d/changelog of the
2780+ # tagged parent matches ours
2781+ parent_changelog_version, _ = \
2782+ repo.get_changelog_versions_from_treeish(
2783+ str(unapplied_changelog_parent_tag.peel(pygit2.Tree).id),
2784+ )
2785+ # if the parent was imported via a parent override
2786+ # itself, assume it is valid without checking its
2787+ # tree's debian/changelog, as it wasn't used
2788+ if (version not in _PARENT_OVERRIDES and
2789+ parent_changelog_version != version
2790+ ):
2791+ logging.error('Found a tag corresponding to '
2792+ 'parent version %s, but d/changelog '
2793+ 'version (%s) differs. Will '
2794+ 'orphan commit.' % (
2795+ version,
2796+ parent_changelog_version
2797+ )
2798+ )
2799+ else:
2800+ unapplied_changelog_parent_commit = str(unapplied_changelog_parent_tag.peel().id)
2801+ logging.debug('Changelog parent (tag) is %s',
2802+ repo.tag_to_pretty_name(unapplied_changelog_parent_tag)
2803+ )
2804+ break
2805
2806- unapplied_publish_parent_commit = None
2807+ # If the two parents are tree-identical, then favor publication
2808+ # history. This deals with copy-forwards between series and with
2809+ # syncs.
2810+ if repo.treeishs_identical(unapplied_publish_parent_commit, unapplied_changelog_parent_commit):
2811 unapplied_changelog_parent_commit = None
2812- upload_parent_commit = None
2813- unapplied_parent_commit = None
2814
2815- if spi.version in self.parent_overrides:
2816- logging.debug('%s is specified in the parent override file.' %
2817+ # check if the version to import has already been uploaded to
2818+ # this head -- we leverage the above code to perform a 'dry-run'
2819+ # of the import tree, to obtain the (up to) two parents
2820+ upload_tag = repo.get_upload_tag(spi.version, namespace)
2821+ import_tag = repo.get_import_tag(spi.version, namespace)
2822+ if upload_tag and not import_tag:
2823+ if str(upload_tag.peel().tree.id) != unapplied_import_tree_hash:
2824+ logging.warn('Found upload tag %s, '
2825+ 'but the corresponding tree '
2826+ 'does not match the published '
2827+ 'version. Will import %s as '
2828+ 'normal, ignoring the upload tag.',
2829+ repo.tag_to_pretty_name(upload_tag),
2830 spi.version
2831 )
2832-
2833- (unapplied_publish_parent_commit,
2834- unapplied_changelog_parent_commit,
2835- _, _) = self.override_parents(spi)
2836 else:
2837- # Get parent from publishing history (which is the last
2838- # published version in the corresponding series)
2839- try:
2840- unapplied_publish_parent_commit = str(unapplied_tip_head.peel().id)
2841- except AttributeError:
2842- try:
2843- unapplied_publish_parent_head = self.local_repo.get_head_by_name(spi.parent_head_name(self.namespace))
2844- unapplied_publish_parent_commit = str(unapplied_publish_parent_head.peel().id)
2845- except AttributeError:
2846- pass
2847- finally:
2848+ upload_parent_commit = str(upload_tag.peel().id)
2849+ if unapplied_publish_parent_commit is not None:
2850 try:
2851- unapplied_publish_parent_tag = self.local_repo.nearest_import_tag(unapplied_publish_parent_commit, self.namespace)
2852- logging.debug('Publishing parent (tag) is %s',
2853- self.local_repo.tag_to_pretty_name(unapplied_publish_parent_tag)
2854+ repo.git_run(['merge-base', '--is-ancestor',
2855+ unapplied_publish_parent_commit,
2856+ upload_parent_commit
2857+ ]
2858 )
2859- except AttributeError:
2860- pass
2861-
2862- if version_compare(str(spi.version), unapplied_tip_version) <= 0:
2863- logging.warn('Version to import (%s) is not after %s tip (%s)',
2864- spi.version, pretty_head_name, unapplied_tip_version
2865- )
2866-
2867- # Walk changelog backwards until we find an imported version
2868- for version in import_tree_versions:
2869- unapplied_changelog_parent_tag = self.local_repo.get_import_tag(version, self.namespace)
2870- if unapplied_changelog_parent_tag is not None:
2871- # sanity check that version from d/changelog of the
2872- # tagged parent matches ours
2873- parent_changelog_version, _ = \
2874- self.local_repo.get_changelog_versions_from_treeish(
2875- str(unapplied_changelog_parent_tag.peel(pygit2.Tree).id),
2876- )
2877- # if the parent was imported via a parent override
2878- # itself, assume it is valid without checking its
2879- # tree's debian/changelog, as it wasn't used
2880- if (version not in self.parent_overrides and
2881- parent_changelog_version != version
2882- ):
2883- logging.error('Found a tag corresponding to '
2884- 'parent version %s, but d/changelog '
2885- 'version (%s) differs. Will '
2886- 'orphan commit.' % (
2887- version,
2888- parent_changelog_version
2889- )
2890- )
2891- else:
2892- unapplied_changelog_parent_commit = str(unapplied_changelog_parent_tag.peel().id)
2893- logging.debug('Changelog parent (tag) is %s',
2894- self.local_repo.tag_to_pretty_name(unapplied_changelog_parent_tag)
2895- )
2896- break
2897-
2898- # If the two parents are tree-identical, then favor publication
2899- # history. This deals with copy-forwards between series and with
2900- # syncs.
2901- if self.local_repo.treeishs_identical(unapplied_publish_parent_commit, unapplied_changelog_parent_commit):
2902- unapplied_changelog_parent_commit = None
2903-
2904- # check if the version to import has already been uploaded to
2905- # this head -- we leverage the above code to perform a 'dry-run'
2906- # of the import tree, to obtain the (up to) two parents
2907- upload_tag = self.local_repo.get_upload_tag(spi.version, self.namespace)
2908- import_tag = self.local_repo.get_import_tag(spi.version, self.namespace)
2909- if upload_tag and not import_tag:
2910- if str(upload_tag.peel().tree.id) != unapplied_import_tree_hash:
2911- logging.warn('Found upload tag %s, '
2912- 'but the corresponding tree '
2913- 'does not match the published '
2914- 'version. Will import %s as '
2915- 'normal, ignoring the upload tag.',
2916- self.local_repo.tag_to_pretty_name(upload_tag),
2917- spi.version
2918- )
2919- else:
2920- upload_parent_commit = str(upload_tag.peel().id)
2921- if unapplied_publish_parent_commit is not None:
2922- try:
2923- self.local_repo.git_run(['merge-base', '--is-ancestor',
2924- unapplied_publish_parent_commit,
2925- upload_parent_commit
2926- ]
2927- )
2928- unapplied_publish_parent_commit = None
2929- except CalledProcessError as e:
2930- if e.returncode != 1:
2931- raise
2932-
2933- if unapplied_changelog_parent_commit is not None:
2934- try:
2935- self.local_repo.git_run(['merge-base', '--is-ancestor',
2936- unapplied_changelog_parent_commit,
2937- upload_parent_commit
2938- ]
2939- )
2940- unapplied_changelog_parent_commit = None
2941- except CalledProcessError as e:
2942- if e.returncode != 1:
2943- raise
2944-
2945- self.commit_unapplied_patches_import(
2946- spi,
2947- unapplied_import_tree_hash,
2948- unapplied_publish_parent_commit,
2949- unapplied_changelog_parent_commit,
2950- upload_parent_commit,
2951- )
2952-
2953- self.update_devel_branches(spi)
2954-
2955- def import_applied_spi(self, spi):
2956- """Imports a source package from Launchpad into the git
2957- repository
2958-
2959- Arguments:
2960- spi - a SourcePackageInformation instance
2961- """
2962- pretty_head_name = spi.pretty_head_name
2963-
2964- logging.info('Importing patches-applied %s to %s' % (spi.version, pretty_head_name))
2965- applied_tip_head = self.local_repo.get_head_by_name(spi.applied_head_name(self.namespace))
2966-
2967- spi.pull()
2968-
2969- self.local_repo.clean_repository_state()
2970-
2971- unapplied_parent_tag = self.local_repo.get_import_tag(spi.version, self.namespace)
2972- unapplied_parent_commit = str(unapplied_parent_tag.peel().id)
2973- unapplied_import_tree_hash = str(unapplied_parent_tag.peel(pygit2.Tree).id)
2974- logging.debug('Found patches-unapplied version %s as commit %s',
2975- str(spi.version), unapplied_parent_commit)
2976-
2977- import_tree_versions = self.local_repo.get_all_changelog_versions_from_treeish(
2978- unapplied_import_tree_hash
2979- )
2980- changelog_version = import_tree_versions[0]
2981-
2982- applied_tip_version = None
2983- # check if the version to import has already been imported to
2984- # this head
2985- if applied_tip_head is not None:
2986- if self.local_repo.treeishs_identical(
2987- unapplied_import_tree_hash, str(applied_tip_head.peel().id)
2988- ):
2989- logging.warn('%s is identical to %s',
2990- pretty_head_name, spi.version
2991- )
2992- return
2993- applied_tip_version, _ = self.local_repo.get_changelog_versions_from_treeish(
2994- str(applied_tip_head.peel().id),
2995- )
2996-
2997- logging.debug('Tip version is %s', applied_tip_version)
2998+ unapplied_publish_parent_commit = None
2999+ except CalledProcessError as e:
3000+ if e.returncode != 1:
3001+ raise
3002
3003- applied_publish_parent_commit = None
3004- applied_changelog_parent_commit = None
3005+ if unapplied_changelog_parent_commit is not None:
3006+ try:
3007+ repo.git_run(['merge-base', '--is-ancestor',
3008+ unapplied_changelog_parent_commit,
3009+ upload_parent_commit
3010+ ]
3011+ )
3012+ unapplied_changelog_parent_commit = None
3013+ except CalledProcessError as e:
3014+ if e.returncode != 1:
3015+ raise
3016
3017- if spi.version in self.parent_overrides:
3018- logging.debug('%s is specified in the parent override file.' %
3019- spi.version
3020+ commit_unapplied_patches_import(
3021+ repo,
3022+ spi,
3023+ unapplied_import_tree_hash,
3024+ namespace,
3025+ unapplied_publish_parent_commit,
3026+ unapplied_changelog_parent_commit,
3027+ upload_parent_commit,
3028+ )
3029+
3030+ update_devel_branches(
3031+ repo=repo,
3032+ namespace=namespace,
3033+ pkgname=spi.name,
3034+ ubuntu_sinfo=ubuntu_sinfo,
3035+ spi=spi,
3036+ )
3037+
3038+def import_applied_spi(repo, spi, namespace, ubuntu_sinfo):
3039+ """Imports a source package from Launchpad into the git
3040+ repository
3041+
3042+ Arguments:
3043+ spi - a SourcePackageInformation instance
3044+ """
3045+ pretty_head_name = spi.pretty_head_name
3046+
3047+ logging.info('Importing patches-applied %s to %s' % (spi.version, pretty_head_name))
3048+ applied_tip_head = repo.get_head_by_name(spi.applied_head_name(namespace))
3049+
3050+ spi.pull()
3051+
3052+ repo.clean_repository_state()
3053+
3054+ unapplied_parent_tag = repo.get_import_tag(spi.version, namespace)
3055+ unapplied_parent_commit = str(unapplied_parent_tag.peel().id)
3056+ unapplied_import_tree_hash = str(unapplied_parent_tag.peel(pygit2.Tree).id)
3057+ logging.debug('Found patches-unapplied version %s as commit %s',
3058+ str(spi.version), unapplied_parent_commit)
3059+
3060+ import_tree_versions = repo.get_all_changelog_versions_from_treeish(
3061+ unapplied_import_tree_hash
3062+ )
3063+ changelog_version = import_tree_versions[0]
3064+
3065+ applied_tip_version = None
3066+ # check if the version to import has already been imported to
3067+ # this head
3068+ if applied_tip_head is not None:
3069+ if repo.treeishs_identical(
3070+ unapplied_import_tree_hash, str(applied_tip_head.peel().id)
3071+ ):
3072+ logging.warn('%s is identical to %s',
3073+ pretty_head_name, spi.version
3074 )
3075+ return
3076+ applied_tip_version, _ = repo.get_changelog_versions_from_treeish(
3077+ str(applied_tip_head.peel().id),
3078+ )
3079
3080- (_, _,
3081- applied_publish_parent_commit,
3082- applied_changelog_parent_commit) = self.override_parents(spi)
3083- else:
3084- # Get parent from publishing history (which is the last
3085- # published version in the corresponding series)
3086- try:
3087- applied_publish_parent_commit = str(applied_tip_head.peel().id)
3088- except AttributeError:
3089- try:
3090- applied_publish_parent_head = self.local_repo.get_head_by_name(spi.applied_parent_head_name(self.namespace))
3091- applied_publish_parent_head_commit = str(applied_publish_parent_head.peel().id)
3092- except AttributeError:
3093- pass
3094-
3095- if version_compare(str(spi.version), applied_tip_version) <= 0:
3096- logging.warn('Version to import (%s) is not after %s tip (%s)',
3097- spi.version, pretty_head_name, applied_tip_version
3098- )
3099+ logging.debug('Tip version is %s', applied_tip_version)
3100
3101- # Walk changelog backwards until we find an imported version
3102- for version in import_tree_versions:
3103- applied_changelog_parent_tag = self.local_repo.get_applied_tag(version, self.namespace)
3104- if applied_changelog_parent_tag is not None:
3105- # sanity check that version from d/changelog of the
3106- # tagged parent matches ours
3107- parent_changelog_version, _ = \
3108- self.local_repo.get_changelog_versions_from_treeish(
3109- str(applied_changelog_parent_tag.peel(pygit2.Tree).id),
3110- )
3111- # if the parent was imported via a parent override
3112- # itself, assume it is valid without checking its
3113- # tree's debian/changelog, as it wasn't used
3114- if (version not in self.parent_overrides and
3115- parent_changelog_version != version
3116- ):
3117- logging.error('Found a tag corresponding to '
3118- 'parent version %s, but d/changelog '
3119- 'version (%s) differs. Will '
3120- 'orphan commit.' % (
3121- version,
3122- parent_changelog_version
3123- )
3124- )
3125- else:
3126- applied_changelog_parent_commit = str(applied_changelog_parent_tag.peel().id)
3127- logging.debug('Changelog parent (tag) is %s',
3128- self.local_repo.tag_to_pretty_name(applied_changelog_parent_tag)
3129- )
3130- break
3131-
3132- # If the two parents are tree-identical, then favor publication
3133- # history. This deals with copy-forwards between series and with
3134- # syncs.
3135- if self.local_repo.treeishs_identical(applied_publish_parent_commit, applied_changelog_parent_commit):
3136- applied_changelog_parent_commit = None
3137-
3138- # Assume no patches to apply
3139- applied_import_tree_hash = unapplied_import_tree_hash
3140- # get tree id from above commit
3141- for (applied_import_tree_hash, patch_name, patch_desc) in self.import_patches_applied_tree(spi.dsc_pathname):
3142- # special case for .pc removal
3143- if patch_name is None:
3144- msg = b'%b.' % (patch_desc.encode())
3145- else:
3146- if patch_desc is None:
3147- patch_desc = '%s\n\nNo DEP3 Subject or Description header found' % patch_name
3148- msg = b'%b\n\nGbp-Pq: %b.' % (patch_desc.encode(), patch_name.encode())
3149- unapplied_parent_commit = self.local_repo.commit_tree_hash(
3150- applied_import_tree_hash,
3151- [unapplied_parent_commit],
3152- msg,
3153- spi)
3154-
3155- logging.debug('Committed patch-application of %s as %s',
3156- patch_name, unapplied_parent_commit
3157- )
3158+ applied_publish_parent_commit = None
3159+ applied_changelog_parent_commit = None
3160
3161- self.commit_applied_patches_import(
3162- spi,
3163- applied_import_tree_hash,
3164+ if spi.version in _PARENT_OVERRIDES:
3165+ logging.debug('%s is specified in the parent override file.' %
3166+ spi.version
3167+ )
3168+
3169+ (
3170+ _,
3171+ _,
3172 applied_publish_parent_commit,
3173 applied_changelog_parent_commit,
3174- unapplied_parent_commit
3175- )
3176-
3177- self.update_applied_devel_branches(spi)
3178-
3179- def import_publishes(self, patches_applied,
3180- debian_head_versions, ubuntu_head_versions):
3181- history_found = False
3182- only_debian = False
3183- srcpkg_information = None
3184- if patches_applied:
3185- namespace = '%s/applied' % self.namespace
3186- import_type = 'patches-applied'
3187- import_func = self.import_applied_spi
3188- else:
3189- namespace = self.namespace
3190- import_type = 'patches-unapplied'
3191- import_func = self.import_unapplied_spi
3192- for distname, versions, dist_sinfo in (
3193- ("debian", debian_head_versions, self.debian_sinfo),
3194- ("ubuntu", ubuntu_head_versions, self.ubuntu_sinfo)):
3195- if self.active_series_only and distname == "debian":
3196- continue
3197- try:
3198- for srcpkg_information in dist_sinfo.launchpad_versions_published_after(
3199- versions,
3200- namespace,
3201- workdir=self.workdir,
3202- active_series_only=self.active_series_only
3203- ):
3204- history_found = True
3205- import_func(srcpkg_information)
3206- # Do not push upstream
3207- if self.fixup_devel and distname == "ubuntu":
3208- self.update_devel_branches(spi=None)
3209- except NoPublicationHistoryException:
3210- logging.warning("No publication history found for %s in %s. ",
3211- self.pkgname, distname)
3212- if distname == 'ubuntu':
3213- only_debian = True
3214- except Exception as e:
3215- if srcpkg_information is None:
3216- msg = 'Unable to import %s to %s' % (import_type, distname)
3217- else:
3218- msg = 'Unable to import %s %s to %s' % (import_type,
3219- str(srcpkg_information.version), distname)
3220- if not patches_applied:
3221- raise GitUbuntuImportError(msg) from e
3222- else:
3223- logging.error(msg)
3224- else:
3225- history_found = True
3226-
3227- return (only_debian, history_found)
3228-
3229- def main(self, args):
3230- self.pkgname = args.package
3231- owner = args.lp_owner
3232- no_clean = args.no_clean
3233- try:
3234- directory = args.directory
3235- no_clean = True
3236- except AttributeError:
3237- directory = None
3238- try:
3239- user = args.lp_user
3240- except AttributeError:
3241- user = None
3242- # --active-series-only/--no-fetch to a tmpdir/--skip-orig
3243- # implies --no-push
3244- no_push = (
3245- args.no_push or
3246- (args.no_fetch and not directory) or
3247- args.active_series_only or
3248- args.skip_orig
3249- )
3250+ ) = override_parents(
3251+ repo,
3252+ spi,
3253+ namespace,
3254+ )
3255+ else:
3256+ # Get parent from publishing history (which is the last
3257+ # published version in the corresponding series)
3258 try:
3259- dl_cache = args.dl_cache
3260+ applied_publish_parent_commit = str(applied_tip_head.peel().id)
3261 except AttributeError:
3262- dl_cache = None
3263- self.fixup_devel = args.fixup_devel
3264- self.active_series_only = args.active_series_only
3265- self.skip_orig = args.skip_orig
3266-
3267- if owner == 'usd-import-team':
3268- self.namespace = 'importer'
3269- else:
3270- self.namespace = owner
3271-
3272- logging.info('Ubuntu Server Team importer v%s' % VERSION)
3273-
3274- self.local_repo = GitUbuntuRepository(directory, user, args.proto)
3275- if not directory:
3276- logging.info('Using git repository at %s', self.local_repo.local_dir)
3277-
3278- atexit.register(self.cleanup, no_clean, self.local_repo.local_dir)
3279-
3280- self.local_repo.add_base_remotes(self.pkgname, repo_owner=owner)
3281- if not args.no_fetch and not args.reimport:
3282- try:
3283- self.local_repo.fetch_base_remotes()
3284- except GitUbuntuRepositoryFetchError:
3285- pass
3286-
3287- if not args.no_fetch:
3288- self.local_repo.delete_branches_in_namespace(self.namespace)
3289- self.local_repo.delete_tags_in_namespace(self.namespace)
3290- self.local_repo.copy_base_references(self.namespace)
3291-
3292- if not args.no_fetch and not args.reimport:
3293- try:
3294- self.local_repo.fetch_remote_refspecs('pkg',
3295- refspecs=['refs/tags/*:refs/tags/%s/*' % self.namespace],
3296- )
3297- except GitUbuntuRepositoryFetchError:
3298- pass
3299-
3300- if args.reimport:
3301- if args.no_fetch:
3302- logging.warning(
3303- "--reimport specified with --no-fetch. Upload "
3304- "tags would not be incorporated into the new import "
3305- "and would be lost forever. This is not currently "
3306- "supported and --no-fetch will be ignored for upload "
3307- "tags."
3308- )
3309 try:
3310- self.local_repo.fetch_remote_refspecs('pkg',
3311- refspecs=['refs/tags/upload/*:refs/tags/%s/upload/*' % self.namespace],
3312- )
3313- except GitUbuntuRepositoryFetchError:
3314+ applied_publish_parent_head = repo.get_head_by_name(spi.applied_parent_head_name(namespace))
3315+ applied_publish_parent_head_commit = str(applied_publish_parent_head.peel().id)
3316+ except AttributeError:
3317 pass
3318
3319- self.local_repo.ensure_importer_branches_exist(self.namespace)
3320+ if version_compare(str(spi.version), applied_tip_version) <= 0:
3321+ logging.warn('Version to import (%s) is not after %s tip (%s)',
3322+ spi.version, pretty_head_name, applied_tip_version
3323+ )
3324
3325- self.debian_sinfo = GitUbuntuSourceInformation('debian',
3326- self.pkgname, os.path.abspath(args.pullfile),
3327- args.retries, args.retry_backoffs)
3328+ # Walk changelog backwards until we find an imported version
3329+ for version in import_tree_versions:
3330+ applied_changelog_parent_tag = repo.get_applied_tag(version, namespace)
3331+ if applied_changelog_parent_tag is not None:
3332+ # sanity check that version from d/changelog of the
3333+ # tagged parent matches ours
3334+ parent_changelog_version, _ = \
3335+ repo.get_changelog_versions_from_treeish(
3336+ str(applied_changelog_parent_tag.peel(pygit2.Tree).id),
3337+ )
3338+ # if the parent was imported via a parent override
3339+ # itself, assume it is valid without checking its
3340+ # tree's debian/changelog, as it wasn't used
3341+ if (version not in _PARENT_OVERRIDES and
3342+ parent_changelog_version != version
3343+ ):
3344+ logging.error('Found a tag corresponding to '
3345+ 'parent version %s, but d/changelog '
3346+ 'version (%s) differs. Will '
3347+ 'orphan commit.' % (
3348+ version,
3349+ parent_changelog_version
3350+ )
3351+ )
3352+ else:
3353+ applied_changelog_parent_commit = str(applied_changelog_parent_tag.peel().id)
3354+ logging.debug('Changelog parent (tag) is %s',
3355+ repo.tag_to_pretty_name(applied_changelog_parent_tag)
3356+ )
3357+ break
3358
3359- self.ubuntu_sinfo = GitUbuntuSourceInformation('ubuntu',
3360- self.pkgname, os.path.abspath(args.pullfile),
3361- args.retries, args.retry_backoffs)
3362+ # If the two parents are tree-identical, then favor publication
3363+ # history. This deals with copy-forwards between series and with
3364+ # syncs.
3365+ if repo.treeishs_identical(applied_publish_parent_commit, applied_changelog_parent_commit):
3366+ applied_changelog_parent_commit = None
3367
3368- debian_head_versions = (
3369- self.local_repo.get_heads_and_versions('debian', self.namespace)
3370- )
3371- for head_name in sorted(debian_head_versions):
3372- _, _, pretty_head_name = head_name.partition('%s/' % self.namespace)
3373- logging.debug('Last imported %s version is %s',
3374- pretty_head_name,
3375- debian_head_versions[head_name]['version']
3376+ # Assume no patches to apply
3377+ applied_import_tree_hash = unapplied_import_tree_hash
3378+ # get tree id from above commit
3379+ for (
3380+ applied_import_tree_hash,
3381+ patch_name,
3382+ patch_desc
3383+ ) in import_patches_applied_tree(repo, spi.dsc_pathname):
3384+ # special case for .pc removal
3385+ if patch_name is None:
3386+ msg = b'%b.' % (patch_desc.encode())
3387+ else:
3388+ if patch_desc is None:
3389+ patch_desc = (
3390+ "%s\n\nNo DEP3 Subject or Description header found" %
3391+ patch_name
3392+ )
3393+ msg = b'%b\n\nGbp-Pq: %b.' % (
3394+ patch_desc.encode(),
3395+ patch_name.encode()
3396 )
3397+ unapplied_parent_commit = repo.commit_tree_hash(
3398+ applied_import_tree_hash,
3399+ [unapplied_parent_commit],
3400+ msg,
3401+ spi
3402+ )
3403
3404- ubuntu_head_versions = (
3405- self.local_repo.get_heads_and_versions('ubuntu', self.namespace)
3406+ logging.debug(
3407+ "Committed patch-application of %s as %s",
3408+ patch_name, unapplied_parent_commit
3409 )
3410- for head_name in sorted(ubuntu_head_versions):
3411- _, _, pretty_head_name = head_name.partition('%s/' % self.namespace)
3412- logging.debug('Last imported %s version is %s',
3413- pretty_head_name,
3414- ubuntu_head_versions[head_name]['version']
3415- )
3416
3417- if not args.skip_applied:
3418- applied_debian_head_versions = (
3419- self.local_repo.get_heads_and_versions(
3420- 'applied/debian', self.namespace
3421- )
3422- )
3423- for head_name in sorted(applied_debian_head_versions):
3424- _, _, pretty_head_name = head_name.partition(
3425- '%s/' % self.namespace
3426- )
3427- logging.debug('Last applied %s version is %s',
3428- pretty_head_name,
3429- applied_debian_head_versions[head_name]['version']
3430+ commit_applied_patches_import(
3431+ repo,
3432+ spi,
3433+ applied_import_tree_hash,
3434+ namespace,
3435+ applied_publish_parent_commit,
3436+ applied_changelog_parent_commit,
3437+ unapplied_parent_commit,
3438+ )
3439+
3440+ update_applied_devel_branches(
3441+ repo=repo,
3442+ pkgname=spi.name,
3443+ namespace=namespace,
3444+ ubuntu_sinfo=ubuntu_sinfo,
3445+ spi=spi,
3446+ )
3447+
3448+def import_publishes(repo, pkgname, namespace, patches_applied,
3449+ debian_head_versions, ubuntu_head_versions, debian_sinfo,
3450+ ubuntu_sinfo, active_series_only, workdir, fixup_devel,
3451+ skip_orig,
3452+):
3453+ history_found = False
3454+ only_debian = False
3455+ srcpkg_information = None
3456+ if patches_applied:
3457+ _namespace = namespace
3458+ namespace = '%s/applied' % namespace
3459+ import_type = 'patches-applied'
3460+ import_func = import_applied_spi
3461+ else:
3462+ _namespace = namespace
3463+ import_type = 'patches-unapplied'
3464+ import_func = functools.partial(
3465+ import_unapplied_spi,
3466+ skip_orig=skip_orig
3467+ )
3468+ for distname, versions, dist_sinfo in (
3469+ ("debian", debian_head_versions, debian_sinfo),
3470+ ("ubuntu", ubuntu_head_versions, ubuntu_sinfo)):
3471+ if active_series_only and distname == "debian":
3472+ continue
3473+ try:
3474+ for srcpkg_information in dist_sinfo.launchpad_versions_published_after(
3475+ versions,
3476+ namespace,
3477+ workdir=workdir,
3478+ active_series_only=active_series_only
3479+ ):
3480+ history_found = True
3481+ import_func(
3482+ repo=repo,
3483+ spi=srcpkg_information,
3484+ namespace=_namespace,
3485+ ubuntu_sinfo=ubuntu_sinfo,
3486 )
3487-
3488- applied_ubuntu_head_versions = (
3489- self.local_repo.get_heads_and_versions(
3490- 'applied/ubuntu', self.namespace
3491+ # Do not push upstream
3492+ if fixup_devel and distname == "ubuntu":
3493+ update_devel_branches(
3494+ repo=repo,
3495+ namespace=namespace,
3496+ pkgname=pkgname,
3497+ ubuntu_sinfo=ubuntu_sinfo,
3498+ spi=None,
3499 )
3500+ except NoPublicationHistoryException:
3501+ logging.warning("No publication history found for %s in %s.",
3502+ pkgname, distname
3503 )
3504- for head_name in sorted(applied_ubuntu_head_versions):
3505- _, _, pretty_head_name = head_name.partition(
3506- '%s/' % self.namespace
3507- )
3508- logging.debug('Last applied %s version is %s',
3509- pretty_head_name,
3510- applied_ubuntu_head_versions[head_name]['version']
3511- )
3512-
3513- oldcwd = os.getcwd()
3514- os.chdir(self.local_repo.local_dir)
3515-
3516- if dl_cache is None:
3517- self.workdir = os.path.join(self.local_repo.git_dir, CACHE_PATH)
3518- else:
3519- self.workdir = dl_cache
3520-
3521- os.makedirs(self.workdir, exist_ok=True)
3522-
3523- self.parse_parentfile(args.parentfile)
3524-
3525- only_debian, history_found = self.import_publishes(
3526- patches_applied=False,
3527- debian_head_versions=debian_head_versions,
3528- ubuntu_head_versions=ubuntu_head_versions)
3529-
3530- if not history_found:
3531- logging.error("No publication history for '%s' in debian or ubuntu."
3532- " Wrong source package name?", self.pkgname)
3533- sys.exit(1)
3534-
3535- if not args.skip_applied:
3536- self.import_publishes(patches_applied=True,
3537- debian_head_versions=applied_debian_head_versions,
3538- ubuntu_head_versions=applied_ubuntu_head_versions)
3539-
3540- os.chdir(oldcwd)
3541-
3542- self.local_repo.garbage_collect()
3543-
3544- if no_push:
3545- logging.info('Not pushing to remote as specified')
3546+ if distname == 'ubuntu':
3547+ only_debian = True
3548+ except Exception as e:
3549+ if srcpkg_information is None:
3550+ msg = 'Unable to import %s to %s' % (import_type, distname)
3551+ else:
3552+ msg = 'Unable to import %s %s to %s' % (import_type,
3553+ str(srcpkg_information.version), distname)
3554+ #if not patches_applied:
3555+ raise GitUbuntuImportError(msg) from e
3556+ #else:
3557+ # logging.error(msg)
3558 else:
3559- lp = launchpad_login_auth()
3560- repo_path = '~%s/ubuntu/+source/%s/+git/%s' % (
3561- owner,
3562- self.pkgname,
3563- self.pkgname,
3564- )
3565- lp_git_repo = lp.git_repositories.getByPath(path=repo_path)
3566- if args.reimport:
3567- if not lp_git_repo.canBeDeleted():
3568- logging.warning(
3569- "There are linked MPs to the pkg repository, "
3570- "which will be deleted:"
3571- )
3572- for mp in lp_git_repo.landing_candidates:
3573- logging.warning(
3574- "Adding comment to %s indicating reimport",
3575- mp.web_link
3576- )
3577- mp.createComment(
3578- content="%s has been reimported and this MP "
3579- "must be manually resubmitted." %
3580- self.pkgname,
3581- subject="%s has been reimported" % self.pkgname,
3582+ history_found = True
3583+
3584+ return (only_debian, history_found)
3585+
3586+
3587+def parse_args(subparsers=None, base_subparsers=None):
3588+ kwargs = dict(description='Update a launchpad git tree based upon '
3589+ 'the state of the Ubuntu and Debian archives',
3590+ formatter_class=argparse.ArgumentDefaultsHelpFormatter
3591+ )
3592+ if base_subparsers:
3593+ kwargs['parents'] = base_subparsers
3594+ if subparsers:
3595+ parser = subparsers.add_parser('import', **kwargs)
3596+ parser.set_defaults(func=cli_main)
3597+ else:
3598+ parser = argparse.ArgumentParser(**kwargs)
3599+ parser.add_argument('package', type=str,
3600+ help='Which package to update in the git tree')
3601+ parser.add_argument('-o', '--lp-owner', type=str,
3602+ help=argparse.SUPPRESS,
3603+ default='usd-import-team')
3604+ parser.add_argument('-l', '--lp-user', type=str,
3605+ help=argparse.SUPPRESS)
3606+ parser.add_argument('--dl-cache', type=str,
3607+ help=('Cache directory for downloads. Default is to '
3608+ 'put downloads in <directory>/.git/%s' %
3609+ CACHE_PATH
3610+ ),
3611+ default=argparse.SUPPRESS)
3612+ parser.add_argument('--no-fetch', action='store_true',
3613+ help='Do not fetch from the remote (DANGEROUS; '
3614+ 'only useful for debugging); implies '
3615+ '--no-push')
3616+ parser.add_argument('--no-push', action='store_true',
3617+ help='Do not push to the remote')
3618+ parser.add_argument('--no-clean', action='store_true',
3619+ help='Do not clean the temporary directory')
3620+ parser.add_argument('-d', '--directory', type=str,
3621+ help='Use git repository at specified location rather '
3622+ 'than a temporary directory (implies --no-clean). '
3623+ 'Will create the local repository if needed.',
3624+ default=argparse.SUPPRESS
3625 )
3626- # repo deletion will delete the MP, but hopefully
3627- # the above comment will be sufficient to explain
3628- # why
3629- lp_git_repo.lp_delete()
3630-
3631- self.local_repo.git_run([
3632- 'push', '--atomic', 'pkg',
3633- 'refs/heads/%s/*:refs/heads/*' % self.namespace,
3634- 'refs/tags/%s/*:refs/tags/*' % self.namespace,
3635- ])
3636- # update our reference
3637- lp_git_repo = lp.git_repositories.getByPath(path=repo_path)
3638- for i in range(args.retries):
3639- try:
3640- if only_debian:
3641- lp_git_repo.default_branch = 'refs/heads/debian/sid'
3642- else:
3643- lp_git_repo.default_branch = 'refs/heads/ubuntu/devel'
3644- lp_git_repo.lp_save()
3645- break
3646- except (NotFound, PreconditionFailed) as e:
3647- time.sleep(args.retry_backoffs[i])
3648- lp_git_repo.lp_refresh()
3649+ parser.add_argument('--fixup-devel', action='store_true',
3650+ help=argparse.SUPPRESS)
3651+ parser.add_argument('--active-series-only', action='store_true',
3652+ help='Do an import of only active Ubuntu series '
3653+ 'history, implies --no-push.')
3654+ parser.add_argument('--skip-applied', action='store_true',
3655+ help=argparse.SUPPRESS)
3656+ parser.add_argument('--skip-orig', action='store_true',
3657+ help=argparse.SUPPRESS)
3658+ parser.add_argument('--reimport', action='store_true',
3659+ help=argparse.SUPPRESS)
3660+ if not subparsers:
3661+ return parser.parse_args()
3662+ return 'import - %s' % kwargs['description']
3663+
3664+def cli_main(args):
3665+ no_clean = args.no_clean
3666+ try:
3667+ directory = args.directory
3668+ no_clean = True
3669+ except AttributeError:
3670+ directory = None
3671+ try:
3672+ user = args.lp_user
3673+ except AttributeError:
3674+ user = None
3675+ # --active-series-only/--no-fetch to a tmpdir/--skip-orig
3676+ # implies --no-push
3677+ no_push = (
3678+ args.no_push or
3679+ (args.no_fetch and not directory) or
3680+ args.active_series_only or
3681+ args.skip_orig
3682+ )
3683+ try:
3684+ dl_cache = args.dl_cache
3685+ except AttributeError:
3686+ dl_cache = None
3687+
3688+ main(
3689+ args.package,
3690+ args.lp_owner,
3691+ no_clean,
3692+ directory,
3693+ user,
3694+ args.proto,
3695+ args.no_fetch,
3696+ no_push,
3697+ dl_cache,
3698+ args.fixup_devel,
3699+ args.active_series_only,
3700+ args.skip_orig,
3701+ args.pullfile,
3702+ args.parentfile,
3703+ args.retries,
3704+ args.retry_backoffs,
3705+ args.skip_applied,
3706+ args.reimport,
3707+ )
3708diff --git a/gitubuntu/importlocal.py b/gitubuntu/importlocal.py
3709index bf1700e..c237551 100644
3710--- a/gitubuntu/importlocal.py
3711+++ b/gitubuntu/importlocal.py
3712@@ -2,15 +2,22 @@ import argparse
3713 import glob
3714 import logging
3715 import os
3716-import re
3717-from subprocess import CalledProcessError
3718 import sys
3719 import tempfile
3720 from gitubuntu.dsc import GitUbuntuDsc
3721 from gitubuntu.git_repository import GitUbuntuRepository, git_dep14_tag
3722-from gitubuntu.run import decode_binary, run, runq
3723+from gitubuntu.run import run, runq
3724 from gitubuntu.version import VERSION
3725-from gitubuntu.importer import GitUbuntuImport
3726+from gitubuntu.importer import (
3727+ parse_parentfile,
3728+ _PARENT_OVERRIDES,
3729+ import_orig,
3730+ import_dsc,
3731+ get_changelog_for_commit,
3732+ import_patches_unapplied_tree,
3733+ import_patches_applied_tree,
3734+)
3735+from gitubuntu.source_information import derive_source_from_series
3736 from gitubuntu.versioning import version_compare
3737 try:
3738 pkg = 'python3-pygit2'
3739@@ -20,264 +27,309 @@ except ImportError:
3740 sys.exit(1)
3741
3742
3743-class GitUbuntuImportLocal(GitUbuntuImport):
3744- def __init__(self):
3745- self.parent_overrides = None
3746-
3747- def parse_args(self, subparsers=None, base_subparsers=None):
3748- kwargs = dict(description='Import a DSC file locally',
3749- #usage='%(prog)s [options] -- ...',
3750- epilog='Unlike Launchpad imports, local imports '
3751- 'require specifying a namespace to uniquely '
3752- 'identify the source of the DSC file. This '
3753- 'means that if you want to tie history '
3754- 'together for several DSCs from the same '
3755- 'source, you must use the same namespace value.',
3756- formatter_class=argparse.ArgumentDefaultsHelpFormatter
3757- )
3758- if base_subparsers:
3759- kwargs['parents'] = base_subparsers
3760- if subparsers:
3761- parser = subparsers.add_parser('import-local', **kwargs)
3762- parser.set_defaults(func=self.main)
3763- else:
3764- parser = argparse.ArgumentParser(**kwargs)
3765- parser.add_argument('namespace',
3766- help='String namespace to use for tagging the imports')
3767- parser.add_argument('-d', '--directory', type=str,
3768- help='Use git repository at specified location rather '
3769- 'than a temporary directory. '
3770- 'Will create the local repository if needed.',
3771- default=argparse.SUPPRESS
3772- )
3773- parser.add_argument('dsc', type=str,
3774- help='Path to or URL of DSC file')
3775- parser.add_argument('--skip-applied', action='store_true',
3776- help=argparse.SUPPRESS)
3777- parser.add_argument('--skip-orig', action='store_true',
3778- help=argparse.SUPPRESS)
3779- if not subparsers:
3780- return parser.parse_args()
3781- return 'import-local - %s' % kwargs['description']
3782-
3783- def main(self, args):
3784- try:
3785- directory = args.directory
3786- no_clean = True
3787- except AttributeError:
3788- directory = None
3789- self.namespace = args.namespace
3790- self.dsc_path = args.dsc
3791- self.skip_applied = args.skip_applied
3792- self.skip_orig = args.skip_orig
3793-
3794- self.local_repo = GitUbuntuRepository(directory)
3795- if not directory:
3796- logging.info('Using git repository at %s', self.local_repo.local_dir)
3797-
3798- self.local_repo.ensure_importer_branches_exist(self.namespace)
3799- if len(self.local_repo.raw_repo.status()) != 0:
3800- logging.error('Working tree must be clean to continue.')
3801- run(['git', 'status'], stdout=None)
3802- sys.exit(1)
3803-
3804- if os.path.exists(self.dsc_path):
3805- self.dsc_path = os.path.abspath(self.dsc_path)
3806- else:
3807- dsc_dir = tempfile.mkdtemp()
3808-
3809- oldcwd = os.getcwd()
3810- os.chdir(dsc_dir)
3811-
3812- runq(['dget', '--quiet', '--allow-unauthenticated',
3813- '--download-only', self.dsc_path])
3814- dscs = glob.glob(os.path.join(dsc_dir, '*.dsc'))
3815- if len(dscs) == 0:
3816- logging.error('Unable to obtain DSC file.')
3817- sys.exit(1)
3818- if len(dscs) > 1:
3819- logging.error('Multiple DSCs found in temporary directory.')
3820- sys.exit(1)
3821- self.dsc_path = os.path.join(dsc_dir, dscs[0])
3822- os.chdir(oldcwd)
3823- logging.debug('Importing DSC at local path %s' % self.dsc_path)
3824- self.dsc = GitUbuntuDsc(self.dsc_path)
3825- if not self.dsc.verify():
3826- logging.error('Unable to verify orig tarball specified by DSC %s',
3827- self.dsc_path)
3828- sys.exit(1)
3829-
3830- self.pkgname = self.dsc['Source']
3831+def main(
3832+ directory,
3833+ namespace,
3834+ dsc_path,
3835+ skip_applied,
3836+ skip_orig,
3837+ parentfile,
3838+):
3839+ repo = GitUbuntuRepository(directory)
3840+ if not directory:
3841+ logging.info('Using git repository at %s', repo.local_dir)
3842+
3843+ repo.ensure_importer_branches_exist(namespace)
3844+ if len(repo.raw_repo.status()) != 0:
3845+ logging.error('Working tree must be clean to continue.')
3846+ run(['git', 'status'], stdout=None)
3847+ sys.exit(1)
3848+
3849+ if os.path.exists(dsc_path):
3850+ dsc_path = os.path.abspath(dsc_path)
3851+ else:
3852+ dsc_dir = tempfile.mkdtemp()
3853
3854 oldcwd = os.getcwd()
3855- os.chdir(self.local_repo.local_dir)
3856-
3857- self.local_repo.clean_repository_state()
3858+ os.chdir(dsc_dir)
3859
3860- self.parse_parentfile(args.parentfile)
3861-
3862- unapplied_import_tree_hash = self.import_patches_unapplied_tree(self.dsc_path)
3863-
3864- import_tree_versions = self.local_repo.get_all_changelog_versions_from_treeish(
3865- unapplied_import_tree_hash
3866- )
3867-
3868- changelog_version = import_tree_versions[0]
3869-
3870- logging.debug(
3871- 'Found changelog version %s in newly imported tree' %
3872- changelog_version
3873- )
3874-
3875- head = None
3876- head_version = None
3877- if not self.local_repo.raw_repo.head_is_unborn:
3878- head = self.local_repo.raw_repo.head
3879- # check if the version to import has already been imported to
3880- # this head
3881- if head is not None:
3882- head_version, _ = self.local_repo.get_changelog_versions_from_treeish(
3883- str(head.peel().id))
3884-
3885- logging.debug('Head version is %s', head_version)
3886-
3887- unapplied_changelog_parent_commit = None
3888- applied_changelog_parent_commit = None
3889-
3890- if version_compare(changelog_version, head_version) <= 0:
3891- logging.warn('Version to import (%s) is not after HEAD '
3892- 'version (%s), history may not make sense.',
3893- changelog_version, head_version
3894- )
3895-
3896- # Walk changelog backwards until we find an imported version
3897- for version in import_tree_versions:
3898- unapplied_tag_ref = '%s/import/%s' % (self.namespace, git_dep14_tag(version))
3899- applied_tag_ref = '%s/applied/%s' % (self.namespace, git_dep14_tag(version))
3900- unapplied_changelog_parent_tag = self.local_repo.get_tag_reference(unapplied_tag_ref)
3901- applied_changelog_parent_tag = self.local_repo.get_tag_reference(applied_tag_ref)
3902- if unapplied_changelog_parent_tag is not None:
3903- # sanity check that version from d/changelog of the
3904- # tagged parent matches ours
3905- parent_changelog_version, _ = \
3906- self.local_repo.get_changelog_versions_from_treeish(
3907- str(unapplied_changelog_parent_tag.peel(pygit2.Tree).id),
3908- )
3909- # if the parent was imported via a parent override
3910- # itself, assume it is valid without checking its
3911- # tree's debian/changelog, as it wasn't used
3912- if (version not in self.parent_overrides and
3913- parent_changelog_version != version
3914- ):
3915- logging.error('Found a tag corresponding to '
3916- 'parent version %s, but d/changelog '
3917- 'version (%s) differs. Will '
3918- 'orphan commit.' % (
3919- version,
3920- parent_changelog_version
3921- )
3922- )
3923- else:
3924- unapplied_changelog_parent_commit = str(unapplied_changelog_parent_tag.peel().id)
3925- if applied_changelog_parent_tag:
3926- applied_changelog_parent_commit = str(applied_changelog_parent_tag.peel().id)
3927- logging.debug('Changelog parent (tag) is %s',
3928- self.local_repo.tag_to_pretty_name(unapplied_changelog_parent_tag)
3929- )
3930- break
3931-
3932- distribution = self.local_repo.get_changelog_distribution_from_treeish(unapplied_import_tree_hash)
3933- # do lookup of series for dist on lp?
3934- dist = 'ubuntu'
3935- if distribution in ("stable", "testing", "unstable", "experimental"):
3936- dist = 'debian'
3937- self.import_dsc(self.dsc, changelog_version, dist)
3938- try:
3939- if not self.skip_orig:
3940- self.import_orig(self.dsc, changelog_version, dist)
3941- except Exception as e:
3942- logging.error('Unable to import orig tarball for %s: %s',
3943- changelog_version, e)
3944+ runq(['dget', '--quiet', '--allow-unauthenticated',
3945+ '--download-only', dsc_path])
3946+ dscs = glob.glob(os.path.join(dsc_dir, '*.dsc'))
3947+ if len(dscs) == 0:
3948+ logging.error('Unable to obtain DSC file.')
3949 sys.exit(1)
3950-
3951- msg = (b'Import unapplied version %b\n\nImported using git-ubuntu-import.' %
3952- (changelog_version.encode())
3953- )
3954-
3955- parents = []
3956-
3957- tag = None
3958- tag_ref = '%s/import/%s' % (self.namespace,
3959- git_dep14_tag(changelog_version))
3960- if self.local_repo.get_tag_reference(tag_ref) is None:
3961- # Not imported before
3962- tag = tag_ref
3963-
3964- if unapplied_changelog_parent_commit is not None:
3965- msg += b'\n'
3966- parents.append(unapplied_changelog_parent_commit)
3967- msg += b'\nChangelog parent: %b' % unapplied_changelog_parent_commit.encode()
3968-
3969- changelog_entry = self.get_changelog_for_commit(
3970- unapplied_import_tree_hash,
3971- None,
3972- unapplied_changelog_parent_commit
3973- )
3974-
3975- msg += b'%b' % changelog_entry
3976- unapplied_parent_commit = self.local_repo.commit_tree_hash(
3977- unapplied_import_tree_hash, parents, msg)
3978- if tag is None:
3979- logging.info('Committed unapplied import of %s as %s',
3980- changelog_version, unapplied_parent_commit)
3981- else:
3982- logging.debug('Creating tag %s pointing to %s', tag,
3983- unapplied_parent_commit)
3984- self.local_repo.git_run(['tag', '-a', '-m', 'git-ubuntu import v%s' % VERSION,
3985- tag, unapplied_parent_commit
3986- ]
3987- )
3988-
3989- if not args.skip_applied:
3990- # Assume no patches to apply
3991- applied_import_tree_hash = unapplied_import_tree_hash
3992- for (applied_import_tree_hash, patch_name, patch_desc) in self.import_patches_applied_tree(self.dsc_path):
3993- # special case for .pc removal
3994- if patch_name is None:
3995- msg = b'%b.' % (patch_desc.encode())
3996- else:
3997- if patch_desc is None:
3998- patch_desc = '%s\n\nNo DEP3 Subject or Description header found' % patch_name
3999- msg = b'%b\n\nGbp-Pq: %b.' % (patch_desc.encode(), patch_name.encode())
4000- unapplied_parent_commit = self.local_repo.commit_tree_hash(
4001- applied_import_tree_hash,
4002- [unapplied_parent_commit],
4003- msg)
4004-
4005- logging.debug('Committed patch-application of %s as %s',
4006- patch_name, unapplied_parent_commit
4007- )
4008-
4009- tag = None
4010- tag_ref = '%s/applied/%s' % (self.namespace,
4011- git_dep14_tag(changelog_version))
4012- if self.local_repo.get_tag_reference(tag_ref) is None:
4013- # Not imported before
4014- tag= tag_ref
4015-
4016- if tag is None:
4017- logging.info('Committed applied import of %s as %s',
4018- changelog_version, unapplied_parent_commit)
4019- else:
4020- logging.debug('Creating tag %s pointing to %s', tag,
4021- unapplied_parent_commit)
4022- self.local_repo.git_run(['tag', '-a', '-m', 'git-ubuntu import v%s' % VERSION,
4023- tag, unapplied_parent_commit
4024- ]
4025- )
4026-
4027- self.local_repo.clean_repository_state()
4028-
4029+ if len(dscs) > 1:
4030+ logging.error('Multiple DSCs found in temporary directory.')
4031+ sys.exit(1)
4032+ dsc_path = os.path.join(dsc_dir, dscs[0])
4033 os.chdir(oldcwd)
4034-
4035- self.local_repo.garbage_collect()
4036+ logging.debug('Importing DSC at local path %s', dsc_path)
4037+ dsc = GitUbuntuDsc(dsc_path)
4038+ if not dsc.verify():
4039+ logging.error(
4040+ 'Unable to verify orig tarball specified by DSC %s',
4041+ dsc_path,
4042+ )
4043+ sys.exit(1)
4044+
4045+ pkgname = dsc['Source']
4046+
4047+ oldcwd = os.getcwd()
4048+ os.chdir(repo.local_dir)
4049+
4050+ repo.clean_repository_state()
4051+
4052+ parse_parentfile(parentfile, pkgname)
4053+
4054+ unapplied_import_tree_hash = import_patches_unapplied_tree(repo, dsc_path)
4055+
4056+ import_tree_versions = repo.get_all_changelog_versions_from_treeish(
4057+ unapplied_import_tree_hash
4058+ )
4059+
4060+ changelog_version = import_tree_versions[0]
4061+
4062+ logging.debug(
4063+ 'Found changelog version %s in newly imported tree',
4064+ changelog_version
4065+ )
4066+
4067+ head = None
4068+ head_version = None
4069+ if not repo.raw_repo.head_is_unborn:
4070+ head = repo.raw_repo.head
4071+ # check if the version to import has already been imported to
4072+ # this head
4073+ if head is not None:
4074+ head_version, _ = repo.get_changelog_versions_from_treeish(
4075+ str(head.peel().id)
4076+ )
4077+
4078+ logging.debug('Head version is %s', head_version)
4079+
4080+ unapplied_changelog_parent_commit = None
4081+ applied_changelog_parent_commit = None
4082+
4083+ if version_compare(changelog_version, head_version) <= 0:
4084+ logging.warn('Version to import (%s) is not after HEAD '
4085+ 'version (%s), history may not make sense.',
4086+ changelog_version, head_version
4087+ )
4088+
4089+ # Walk changelog backwards until we find an imported version
4090+ for version in import_tree_versions:
4091+ unapplied_tag_ref = '%s/import/%s' % (namespace, git_dep14_tag(version))
4092+ applied_tag_ref = '%s/applied/%s' % (namespace, git_dep14_tag(version))
4093+ unapplied_changelog_parent_tag = repo.get_tag_reference(unapplied_tag_ref)
4094+ applied_changelog_parent_tag = repo.get_tag_reference(applied_tag_ref)
4095+ if unapplied_changelog_parent_tag is not None:
4096+ # sanity check that version from d/changelog of the
4097+ # tagged parent matches ours
4098+ parent_changelog_version, _ = \
4099+ repo.get_changelog_versions_from_treeish(
4100+ str(unapplied_changelog_parent_tag.peel(pygit2.Tree).id),
4101+ )
4102+ # if the parent was imported via a parent override
4103+ # itself, assume it is valid without checking its
4104+ # tree's debian/changelog, as it wasn't used
4105+ if (version not in _PARENT_OVERRIDES and
4106+ parent_changelog_version != version
4107+ ):
4108+ logging.error('Found a tag corresponding to '
4109+ 'parent version %s, but d/changelog '
4110+ 'version (%s) differs. Will '
4111+ 'orphan commit.',
4112+ version,
4113+ parent_changelog_version
4114+ )
4115+ else:
4116+ unapplied_changelog_parent_commit = str(
4117+ unapplied_changelog_parent_tag.peel().id
4118+ )
4119+ if applied_changelog_parent_tag:
4120+ applied_changelog_parent_commit = str(
4121+ applied_changelog_parent_tag.peel().id
4122+ )
4123+ logging.debug('Changelog parent (tag) is %s',
4124+ repo.tag_to_pretty_name(unapplied_changelog_parent_tag)
4125+ )
4126+ break
4127+
4128+ distribution = repo.get_changelog_distribution_from_treeish(
4129+ unapplied_import_tree_hash
4130+ )
4131+ dist = derive_source_from_series(distribution)
4132+ import_dsc(
4133+ repo=repo,
4134+ dsc=dsc,
4135+ namespace=namespace,
4136+ version=changelog_version,
4137+ dist=dist,
4138+ )
4139+ try:
4140+ if not skip_orig:
4141+ import_orig(
4142+ repo=repo,
4143+ dsc=dsc,
4144+ namespace=namespace,
4145+ version=changelog_version,
4146+ dist=dist,
4147+ )
4148+ except Exception as e:
4149+ logging.error('Unable to import orig tarball for %s: %s',
4150+ changelog_version, e)
4151+ raise
4152+ sys.exit(1)
4153+
4154+ msg = (
4155+ b'Import unapplied version %b\n\nImported using git-ubuntu-import.' %
4156+ (changelog_version.encode())
4157+ )
4158+
4159+ parents = []
4160+
4161+ tag = None
4162+ tag_ref = '%s/import/%s' % (
4163+ namespace,
4164+ git_dep14_tag(changelog_version)
4165+ )
4166+ if repo.get_tag_reference(tag_ref) is None:
4167+ # Not imported before
4168+ tag = tag_ref
4169+
4170+ if unapplied_changelog_parent_commit is not None:
4171+ msg += b'\n'
4172+ parents.append(unapplied_changelog_parent_commit)
4173+ msg += b'\nChangelog parent: %b' % unapplied_changelog_parent_commit.encode()
4174+
4175+ changelog_entry = get_changelog_for_commit(
4176+ repo=repo,
4177+ tree_hash=unapplied_import_tree_hash,
4178+ publish_parent_commit=None,
4179+ changelog_parent_commit=unapplied_changelog_parent_commit,
4180+ )
4181+
4182+ msg += b'%b' % changelog_entry
4183+ unapplied_parent_commit = repo.commit_tree_hash(
4184+ unapplied_import_tree_hash, parents, msg
4185+ )
4186+ if tag is None:
4187+ logging.info('Committed unapplied import of %s as %s',
4188+ changelog_version, unapplied_parent_commit)
4189+ else:
4190+ logging.debug('Creating tag %s pointing to %s', tag,
4191+ unapplied_parent_commit
4192+ )
4193+ repo.git_run(['tag', '-a', '-m', 'git-ubuntu import v%s' % VERSION,
4194+ tag, unapplied_parent_commit
4195+ ]
4196+ )
4197+
4198+ if not skip_applied:
4199+ # Assume no patches to apply
4200+ applied_import_tree_hash = unapplied_import_tree_hash
4201+ for (
4202+ applied_import_tree_hash,
4203+ patch_name,
4204+ patch_desc,
4205+ ) in import_patches_applied_tree(repo, dsc_path):
4206+ # special case for .pc removal
4207+ if patch_name is None:
4208+ msg = b'%b.' % (patch_desc.encode())
4209+ else:
4210+ if patch_desc is None:
4211+ patch_desc = '%s\n\nNo DEP3 Subject or Description header found' % patch_name
4212+ msg = b'%b\n\nGbp-Pq: %b.' % (patch_desc.encode(), patch_name.encode())
4213+ unapplied_parent_commit = repo.commit_tree_hash(
4214+ applied_import_tree_hash,
4215+ [unapplied_parent_commit],
4216+ msg)
4217+
4218+ logging.debug('Committed patch-application of %s as %s',
4219+ patch_name, unapplied_parent_commit
4220+ )
4221+
4222+ tag = None
4223+ tag_ref = '%s/applied/%s' % (namespace,
4224+ git_dep14_tag(changelog_version))
4225+ if repo.get_tag_reference(tag_ref) is None:
4226+ # Not imported before
4227+ tag = tag_ref
4228+
4229+ if tag is None:
4230+ logging.info('Committed applied import of %s as %s',
4231+ changelog_version,
4232+ unapplied_parent_commit
4233+ )
4234+ else:
4235+ logging.debug('Creating tag %s pointing to %s', tag,
4236+ unapplied_parent_commit
4237+ )
4238+ repo.git_run(
4239+ [
4240+ 'tag',
4241+ '-a',
4242+ '-m',
4243+ 'git-ubuntu import v%s' % VERSION,
4244+ tag,
4245+ unapplied_parent_commit,
4246+ ]
4247+ )
4248+
4249+ repo.clean_repository_state()
4250+
4251+ os.chdir(oldcwd)
4252+
4253+ repo.garbage_collect()
4254+
4255+def parse_args(subparsers=None, base_subparsers=None):
4256+ kwargs = dict(description='Import a DSC file locally',
4257+ #usage='%(prog)s [options] -- ...',
4258+ epilog='Unlike Launchpad imports, local imports '
4259+ 'require specifying a namespace to uniquely '
4260+ 'identify the source of the DSC file. This '
4261+ 'means that if you want to tie history '
4262+ 'together for several DSCs from the same '
4263+ 'source, you must use the same namespace value.',
4264+ formatter_class=argparse.ArgumentDefaultsHelpFormatter
4265+ )
4266+ if base_subparsers:
4267+ kwargs['parents'] = base_subparsers
4268+ if subparsers:
4269+ parser = subparsers.add_parser('import-local', **kwargs)
4270+ parser.set_defaults(func=cli_main)
4271+ else:
4272+ parser = argparse.ArgumentParser(**kwargs)
4273+ parser.add_argument('namespace',
4274+ help='String namespace to use for tagging the imports')
4275+ parser.add_argument('-d', '--directory', type=str,
4276+ help='Use git repository at specified location rather '
4277+ 'than a temporary directory. '
4278+ 'Will create the local repository if needed.',
4279+ default=argparse.SUPPRESS
4280+ )
4281+ parser.add_argument('dsc', type=str,
4282+ help='Path to or URL of DSC file')
4283+ parser.add_argument('--skip-applied', action='store_true',
4284+ help=argparse.SUPPRESS)
4285+ parser.add_argument('--skip-orig', action='store_true',
4286+ help=argparse.SUPPRESS)
4287+ if not subparsers:
4288+ return parser.parse_args()
4289+ return 'import-local - %s' % kwargs['description']
4290+
4291+def cli_main(args):
4292+ try:
4293+ directory = args.directory
4294+ except AttributeError:
4295+ directory = None
4296+
4297+ main(
4298+ directory,
4299+ args.namespace,
4300+ args.dsc,
4301+ args.skip_applied,
4302+ args.skip_orig,
4303+ args.parentfile,
4304+ )
4305diff --git a/gitubuntu/importppa.py b/gitubuntu/importppa.py
4306index a4a3ec1..b4790f1 100644
4307--- a/gitubuntu/importppa.py
4308+++ b/gitubuntu/importppa.py
4309@@ -10,8 +10,25 @@ from gitubuntu.git_repository import (
4310 GitUbuntuRepository,
4311 GitUbuntuRepositoryFetchError,
4312 )
4313-from gitubuntu.importer import GitUbuntuImport
4314-from gitubuntu.source_information import GitUbuntuSourceInformation, NoPublicationHistoryException, SourceExtractionException, launchpad_login_auth
4315+from gitubuntu.importer import (
4316+ cleanup,
4317+ parse_parentfile,
4318+ _PARENT_OVERRIDES,
4319+ import_orig,
4320+ import_dsc,
4321+ get_changelog_for_commit,
4322+ import_unapplied_spi,
4323+ import_applied_spi,
4324+ import_patches_unapplied_tree,
4325+ import_patches_applied_tree,
4326+)
4327+from gitubuntu.source_information import (
4328+ GitUbuntuSourceInformation,
4329+ NoPublicationHistoryException,
4330+ SourceExtractionException,
4331+ launchpad_login_auth,
4332+ GitUbuntuSourceInformation,
4333+)
4334 from gitubuntu.version import VERSION
4335 try:
4336 pkg = 'python3-pygit2'
4337@@ -23,181 +40,245 @@ except ImportError:
4338 sys.exit(1)
4339
4340
4341-class GitUbuntuImportPPA(GitUbuntuImport):
4342- def __init__(self):
4343- self.parent_overrides = None
4344-
4345- def parse_args(self, subparsers=None, base_subparsers=None):
4346- kwargs = dict(description='Update a launchpad git tree based upon '
4347- 'the state of a PPA archive',
4348- epilog='PPA must be specified as ppa:<name>',
4349- formatter_class=argparse.ArgumentDefaultsHelpFormatter
4350- )
4351- if base_subparsers:
4352- kwargs['parents'] = base_subparsers
4353- if subparsers:
4354- parser = subparsers.add_parser('import-ppa', **kwargs)
4355- parser.set_defaults(func=self.main)
4356- else:
4357- parser = argparse.ArgumentParser(**kwargs)
4358- parser.add_argument('ppa', type=str,
4359- help='Name of PPA to update in the git tree')
4360- parser.add_argument('package', type=str,
4361- help='Which package to update in the git tree')
4362- parser.add_argument('-o', '--lp-owner', type=str,
4363- help='Which launchpad user\'s tree to update',
4364- default='usd-import-team')
4365- parser.add_argument('-l', '--lp-user', type=str,
4366- help='Specify the Launchpad user to use',
4367- default=getpass.getuser())
4368- parser.add_argument('--dl-cache', type=str,
4369- help=('Cache directory for downloads. Default is to '
4370- 'put downloads in <directory>/.git/%s' %
4371- CACHE_PATH
4372- ),
4373- default=argparse.SUPPRESS)
4374- parser.add_argument('--no-fetch', action='store_true',
4375- help='Do not fetch from the remote (DANGEROUS: '
4376- 'only useful for debugging); implies '
4377- '--no-push')
4378- parser.add_argument('--no-push', action='store_true',
4379- help='Do not push to the remote (WARNING: '
4380- 'PPA imports are a work-in-progress and '
4381- 'pushing is currently disabled)',
4382- default=True)
4383- parser.add_argument('--no-clean', action='store_true',
4384- help='Do not clean the temporary directory')
4385- parser.add_argument('-d', '--directory', type=str,
4386- help='Use git repository at specified location rather '
4387- 'than a temporary directory (implies --no-clean). '
4388- 'Will create the local repository if needed.',
4389- default=argparse.SUPPRESS
4390- )
4391- if not subparsers:
4392- return parser.parse_args()
4393- return 'import-ppa - %s' % kwargs['description']
4394-
4395- def main(self, args):
4396- if re.match(r'ppa:\w+', args.ppa) is None:
4397- logging.error('Specified PPA (%s) is not in the format '
4398- 'ppa:<name>', args.ppa)
4399- sys.exit(1)
4400- ppa = args.ppa
4401- pkgname = args.package
4402- owner = args.lp_owner
4403- user = args.lp_user
4404- no_clean = args.no_clean
4405- try:
4406- directory = args.directory
4407- no_clean = True
4408- except AttributeError:
4409- directory = None
4410- no_push = True
4411- try:
4412- dl_cache = args.dl_cache
4413- except AttributeError:
4414- dl_cache = None
4415-
4416- # what other transformations?
4417- self.namespace = ppa.replace(':', '_')
4418-
4419- logging.info('Ubuntu Server Team importer v%s' % VERSION)
4420-
4421- self.local_repo = GitUbuntuRepository(directory, user, args.proto)
4422- if not directory:
4423- logging.info('Using git repository at %s', self.local_repo.local_dir)
4424- self.local_repo.ensure_importer_branches_exist(self.namespace)
4425-
4426- atexit.register(self.cleanup, no_clean, self.local_repo.local_dir)
4427-
4428- self.local_repo.add_remote(pkgname, owner, self.namespace, user)
4429- if not args.no_fetch:
4430+def main(
4431+ ppa,
4432+ pkgname,
4433+ directory,
4434+ owner,
4435+ user,
4436+ pullfile,
4437+ parentfile,
4438+ no_clean,
4439+ no_fetch,
4440+ no_push,
4441+ dl_cache,
4442+ proto,
4443+ retries,
4444+ retry_backoffs,
4445+ skip_orig,
4446+):
4447+ if re.match(r'ppa:\w+', ppa) is None:
4448+ logging.error(
4449+ "Specified PPA (%s) is not in the format ppa:<name>",
4450+ ppa,
4451+ )
4452+ sys.exit(1)
4453+ # what other transformations?
4454+ namespace = ppa.replace(':', '_')
4455+
4456+ logging.info('Ubuntu Server Team importer v%s' % VERSION)
4457+
4458+ repo = GitUbuntuRepository(directory, user, proto)
4459+ if not directory:
4460+ logging.info('Using git repository at %s', repo.local_dir)
4461+ repo.ensure_importer_branches_exist(namespace)
4462+
4463+ atexit.register(cleanup, no_clean, repo.local_dir)
4464+
4465+ # This seems terminally broken and I'm not sure what we should do
4466+ # here
4467+ #repo.add_remote(pkgname, owner, namespace)
4468+ #if not no_fetch:
4469+ # try:
4470+ # repo.fetch_remote(namespace)
4471+ # except GitUbuntuRepositoryFetchError:
4472+ # pass
4473+
4474+ source_information = GitUbuntuSourceInformation(
4475+ ppa,
4476+ pkgname,
4477+ os.path.abspath(pullfile),
4478+ retries,
4479+ retry_backoffs,
4480+ )
4481+
4482+ ubuntu_head_versions = repo.get_heads_and_versions(
4483+ 'ubuntu',
4484+ namespace,
4485+ )
4486+ for head_name in sorted(ubuntu_head_versions):
4487+ _, _, pretty_head_name = head_name.partition('%s/', namespace)
4488+ logging.debug('Last imported %s version is %s',
4489+ pretty_head_name,
4490+ ubuntu_head_versions[head_name]['version']
4491+ )
4492+
4493+ applied_ubuntu_head_versions = repo.get_heads_and_versions(
4494+ 'applied/ubuntu',
4495+ namespace,
4496+ )
4497+ for head_name in sorted(applied_ubuntu_head_versions):
4498+ _, _, pretty_head_name = head_name.partition('%s/', namespace)
4499+ logging.debug('Last applied %s version is %s',
4500+ pretty_head_name,
4501+ applied_ubuntu_head_versions[head_name]['version']
4502+ )
4503+
4504+ oldcwd = os.getcwd()
4505+ os.chdir(repo.local_dir)
4506+
4507+ if dl_cache is None:
4508+ workdir = os.path.join(repo.git_dir, CACHE_PATH)
4509+ else:
4510+ workdir = dl_cache
4511+
4512+ os.makedirs(workdir, exist_ok=True)
4513+
4514+ parse_parentfile(pkgname, parentfile)
4515+
4516+ history_found = False
4517+ try:
4518+ for srcpkg_information in \
4519+ source_information.launchpad_versions_published_after(
4520+ ubuntu_head_versions,
4521+ namespace,
4522+ workdir=workdir
4523+ ):
4524 try:
4525- self.local_repo.fetch_remote(self.namespace)
4526- except GitUbuntuRepositoryFetchError:
4527+ import_unapplied_spi(
4528+ repo=repo,
4529+ spi=srcpkg_information,
4530+ namespace=namespace,
4531+ ubuntu_sinfo=source_information,
4532+ skip_orig=skip_orig,
4533+ )
4534+ except DownloadError:
4535+ # it is non-fatal for a PPA to not have files for older
4536+ # publishes
4537 pass
4538+ history_found = True
4539+ except NoPublicationHistoryException:
4540+ logging.warning("No publication history found for %s in %s.",
4541+ pkgname,
4542+ ppa,
4543+ )
4544
4545- source_information = UbuntuSourceInformation(ppa, pkgname,
4546- os.path.abspath(args.pullfile),
4547- args.retries, args.retry_backoffs)
4548-
4549- ubuntu_head_versions = self.local_repo.get_heads_and_versions(
4550- 'ubuntu', self.namespace
4551+ if not history_found:
4552+ logging.error(
4553+ "No publication history for '%s' in %s. Wrong source package name?",
4554+ pkgname,
4555+ ppa,
4556 )
4557- for head_name in sorted(ubuntu_head_versions):
4558- _, _, pretty_head_name = head_name.partition('%s/', self.namespace)
4559- logging.debug('Last imported %s version is %s',
4560- pretty_head_name,
4561- ubuntu_head_versions[head_name]['version']
4562- )
4563-
4564- applied_ubuntu_head_versions = self.local_repo.get_heads_and_versions(
4565- 'applied/ubuntu', self.namespace
4566+ sys.exit(1)
4567+
4568+ try:
4569+ for srcpkg_information in \
4570+ source_information.launchpad_versions_published_after(
4571+ applied_ubuntu_head_versions,
4572+ '%s/applied' % namespace,
4573+ workdir=workdir
4574+ ):
4575+ try:
4576+ import_applied_spi(
4577+ repo=repo,
4578+ spi=srcpkg_information,
4579+ namespace=namespace,
4580+ ubuntu_sinfo=source_information,
4581+ )
4582+ except DownloadError:
4583+ # it is non-fatal for a PPA to not have files for older
4584+ # publishes
4585+ pass
4586+ except NoPublicationHistoryException:
4587+ logging.warning("No publication history found for %s in %s.",
4588+ pkgname, ppa
4589 )
4590- for head_name in sorted(applied_ubuntu_head_versions):
4591- _, _, pretty_head_name = head_name.partition('%s/', self.namespace)
4592- logging.debug('Last applied %s version is %s',
4593- pretty_head_name,
4594- applied_ubuntu_head_versions[head_name]['version']
4595- )
4596-
4597- oldcwd = os.getcwd()
4598- os.chdir(self.local_repo.local_dir)
4599-
4600- if dl_cache is None:
4601- workdir = os.path.join(self.local_repo.git_dir, CACHE_PATH)
4602- else:
4603- workdir = dl_cache
4604-
4605- os.makedirs(workdir, exist_ok=True)
4606-
4607- self.parse_parentfile(pkgname, args.parentfile)
4608-
4609- history_found = False
4610- try:
4611- for srcpkg_information in \
4612- source_information.launchpad_versions_published_after(ubuntu_head_versions,
4613- self.namespace,
4614- workdir=workdir):
4615- try:
4616- self.import_unapplied_spi(srcpkg_information)
4617- except DownloadError:
4618- # it is non-fatal for a PPA to not have files for older
4619- # publishes
4620- pass
4621- history_found = True
4622- except NoPublicationHistoryException:
4623- logging.warning("No publication history found for %s in %s. ",
4624- pkgname, distname)
4625-
4626- if not history_found:
4627- logging.error("No publication history for '%s' in %s. "
4628- "Wrong source package name?", pkgname, ppa)
4629- sys.exit(1)
4630-
4631- try:
4632- for srcpkg_information in \
4633- source_information.launchpad_versions_published_after(applied_ubuntu_head_versions,
4634- '%s/applied' % self.namespace,
4635- workdir=workdir):
4636- try:
4637- self.import_applied_spi(srcpkg_information)
4638- except DownloadError:
4639- # it is non-fatal for a PPA to not have files for older
4640- # publishes
4641- pass
4642- except NoPublicationHistoryException:
4643- logging.warning("No publication history found for %s in %s. ",
4644- pkgname, ppa)
4645- except:
4646- logging.error("Unable to import patches-applied to %s", ppa)
4647-
4648- os.chdir(oldcwd)
4649-
4650- self.local_repo.garbage_collect()
4651-
4652- if no_push:
4653- logging.info('Not pushing to remote as specified')
4654- else:
4655- # what are appropriate refspects to push and where?
4656- pass
4657+ except:
4658+ logging.error("Unable to import patches-applied to %s", ppa)
4659+
4660+ os.chdir(oldcwd)
4661+
4662+ repo.garbage_collect()
4663+
4664+ if no_push:
4665+ logging.info('Not pushing to remote as specified')
4666+ else:
4667+ # what are appropriate refspects to push and where?
4668+ pass
4669+
4670+
4671+def parse_args(subparsers=None, base_subparsers=None):
4672+ kwargs = dict(description='Update a launchpad git tree based upon '
4673+ 'the state of a PPA archive',
4674+ epilog='PPA must be specified as ppa:<name>',
4675+ formatter_class=argparse.ArgumentDefaultsHelpFormatter
4676+ )
4677+ if base_subparsers:
4678+ kwargs['parents'] = base_subparsers
4679+ if subparsers:
4680+ parser = subparsers.add_parser('import-ppa', **kwargs)
4681+ parser.set_defaults(func=cli_main)
4682+ else:
4683+ parser = argparse.ArgumentParser(**kwargs)
4684+ parser.add_argument('ppa', type=str,
4685+ help='Name of PPA to update in the git tree')
4686+ parser.add_argument('package', type=str,
4687+ help='Which package to update in the git tree')
4688+ parser.add_argument('-o', '--lp-owner', type=str,
4689+ help='Which launchpad user\'s tree to update',
4690+ default='usd-import-team')
4691+ parser.add_argument('-l', '--lp-user', type=str,
4692+ help='Specify the Launchpad user to use',
4693+ default=getpass.getuser())
4694+ parser.add_argument('--dl-cache', type=str,
4695+ help=('Cache directory for downloads. Default is to '
4696+ 'put downloads in <directory>/.git/%s' %
4697+ CACHE_PATH
4698+ ),
4699+ default=argparse.SUPPRESS)
4700+ parser.add_argument('--no-fetch', action='store_true',
4701+ help='Do not fetch from the remote (DANGEROUS: '
4702+ 'only useful for debugging); implies '
4703+ '--no-push')
4704+ parser.add_argument('--no-push', action='store_true',
4705+ help='Do not push to the remote (WARNING: '
4706+ 'PPA imports are a work-in-progress and '
4707+ 'pushing is currently disabled)',
4708+ default=True)
4709+ parser.add_argument('--no-clean', action='store_true',
4710+ help='Do not clean the temporary directory')
4711+ parser.add_argument('-d', '--directory', type=str,
4712+ help='Use git repository at specified location rather '
4713+ 'than a temporary directory (implies --no-clean). '
4714+ 'Will create the local repository if needed.',
4715+ default=argparse.SUPPRESS
4716+ )
4717+ parser.add_argument(
4718+ '--skip-orig',
4719+ action='store_true',
4720+ help=argparse.SUPPRESS,
4721+ )
4722+ if not subparsers:
4723+ return parser.parse_args()
4724+ return 'import-ppa - %s' % kwargs['description']
4725+
4726+def cli_main(args):
4727+ no_clean = args.no_clean
4728+ try:
4729+ directory = args.directory
4730+ no_clean = True
4731+ except AttributeError:
4732+ directory = None
4733+ no_push = True
4734+ try:
4735+ dl_cache = args.dl_cache
4736+ except AttributeError:
4737+ dl_cache = None
4738+
4739+ main(
4740+ args.ppa,
4741+ args.package,
4742+ directory,
4743+ args.lp_owner,
4744+ args.lp_user,
4745+ args.pullfile,
4746+ args.parentfile,
4747+ no_clean,
4748+ args.no_fetch,
4749+ no_push,
4750+ dl_cache,
4751+ args.proto,
4752+ args.retries,
4753+ args.retry_backoffs,
4754+ args.skip_orig,
4755+ )
4756diff --git a/gitubuntu/lint.py b/gitubuntu/lint.py
4757index 435e7f2..bce9388 100644
4758--- a/gitubuntu/lint.py
4759+++ b/gitubuntu/lint.py
4760@@ -23,10 +23,6 @@ except ImportError:
4761 logging.error("Is %s installed?", pkg)
4762 sys.exit(1)
4763
4764-__all__ = [
4765- 'GitUbuntuLint',
4766-]
4767-
4768 class LintNamespaceError(Exception):
4769 pass
4770
4771@@ -162,208 +158,214 @@ def test__derive_target_branch_string(same_remote_branch_names,
4772 )
4773 assert target_branch_string == expected
4774
4775-class GitUbuntuLint:
4776- def __init__(self):
4777- pass
4778
4779- def success(self, msg, *args):
4780- if self.verbose:
4781- print(msg % args)
4782+_verbose = False
4783+def success(msg, *args):
4784+ if _verbose:
4785+ print(msg % args)
4786
4787- def parse_args(self, subparsers=None, base_subparsers=None):
4788- kwargs = dict(
4789- description="Given a commitish, perform automated checks",
4790- formatter_class=argparse.RawTextHelpFormatter,
4791- epilog="An exit code of 0 indicates all checks passed",
4792- )
4793- if base_subparsers:
4794- kwargs["parents"] = base_subparsers
4795- if subparsers:
4796- parser = subparsers.add_parser("lint", **kwargs)
4797- parser.set_defaults(func=self.main)
4798- else:
4799- parser = argparse.ArgumentParser(**kwargs)
4800
4801- parser.add_argument("commitish", type=str, default="HEAD", nargs="?",
4802- help="Commitish to lint"
4803- )
4804- parser.add_argument("--lint-namespace", type=str,
4805- help="The git-ref prefix under which the branches and tags "
4806- "to lint can be found. If linting a remote-tracking "
4807- "branch, this will be default to the remote name. If "
4808- "linting a local branch, this will default to ''."
4809- )
4810- parser.add_argument("--target-branch", type=str,
4811- help="Target branch (typically a remote-tracking branch in pkg/) "
4812- "the current branch is against. This will be derived if "
4813- "possible."
4814- )
4815- parser.add_argument("-d", "--directory", type=str,
4816- help="Use git repository at specified location.",
4817- default=os.path.abspath(os.getcwd())
4818- )
4819- if not subparsers:
4820- return parser.parse_args()
4821- return "lint - %s" % kwargs["description"]
4822+def parse_args(subparsers=None, base_subparsers=None):
4823+ kwargs = dict(
4824+ description="Given a commitish, perform automated checks",
4825+ formatter_class=argparse.RawTextHelpFormatter,
4826+ epilog="An exit code of 0 indicates all checks passed",
4827+ )
4828+ if base_subparsers:
4829+ kwargs["parents"] = base_subparsers
4830+ if subparsers:
4831+ parser = subparsers.add_parser("lint", **kwargs)
4832+ parser.set_defaults(func=cli_main)
4833+ else:
4834+ parser = argparse.ArgumentParser(**kwargs)
4835
4836- def _check_commitish_exists(self, commitish):
4837- try:
4838- actual_commitish_obj = self.local_repo.get_commitish(
4839- commitish
4840- )
4841- self.success("Verified %s exists", commitish)
4842- return True
4843- except KeyError:
4844- warning("%s not found", commitish)
4845- return False
4846+ parser.add_argument("commitish", type=str, default="HEAD", nargs="?",
4847+ help="Commitish to lint"
4848+ )
4849+ parser.add_argument("--lint-namespace", type=str,
4850+ help="The git-ref prefix under which the branches and tags "
4851+ "to lint can be found. If linting a remote-tracking "
4852+ "branch, this will be default to the remote name. If "
4853+ "linting a local branch, this will default to ''."
4854+ )
4855+ parser.add_argument("--target-branch", type=str,
4856+ help="Target branch (typically a remote-tracking branch in pkg/) "
4857+ "the current branch is against. This will be derived if "
4858+ "possible."
4859+ )
4860+ parser.add_argument("-d", "--directory", type=str,
4861+ help="Use git repository at specified location.",
4862+ default=os.path.abspath(os.getcwd())
4863+ )
4864+ if not subparsers:
4865+ return parser.parse_args()
4866+ return "lint - %s" % kwargs["description"]
4867+
4868+def cli_main(args):
4869+ main(
4870+ args.directory,
4871+ args.commitish,
4872+ args.lint_namespace,
4873+ args.target_branch,
4874+ args.verbose,
4875+ )
4876
4877- def _check_commitish_exists_and_same(self, expected_commitish_pretty_name,
4878- actual_commitish, expected_commitish, check_commit_id=False
4879- ):
4880- """Check @actual_commitish exists ad is the same tree as
4881- @expected_commitish
4882-
4883- Arguments:
4884- expected_commitish_pretty_name - a pretty name to print to refer to the
4885- expected value
4886- actual_commitish - commitish to check for
4887- expected_commitish - commitish to check against
4888- check_commit_id - if True, check the commit hashes match as well
4889- """
4890- ret = True
4891- if not self._check_commitish_exists(actual_commitish):
4892- return False
4893+def _check_commitish_exists(repo, commitish):
4894+ try:
4895+ actual_commitish_obj = repo.get_commitish(commitish)
4896+ success("Verified %s exists", commitish)
4897+ return True
4898+ except KeyError:
4899+ warning("%s not found", commitish)
4900+ return False
4901
4902- try:
4903- expected_commitish_obj = self.local_repo.get_commitish(
4904- expected_commitish
4905- )
4906- except KeyError:
4907- fatal("Unable to find expected commit: %s", expected_commitish)
4908+def _check_commitish_exists_and_same(repo, expected_commitish_pretty_name,
4909+ actual_commitish, expected_commitish, check_commit_id=False
4910+):
4911+ """Check @actual_commitish exists ad is the same tree as
4912+ @expected_commitish
4913+
4914+ Arguments:
4915+ expected_commitish_pretty_name - a pretty name to print to refer to the
4916+ expected value
4917+ actual_commitish - commitish to check for
4918+ expected_commitish - commitish to check against
4919+ check_commit_id - if True, check the commit hashes match as well
4920+ """
4921+ ret = True
4922+ if not _check_commitish_exists(repo, actual_commitish):
4923+ return False
4924
4925- actual_commitish_obj = self.local_repo.get_commitish(
4926- actual_commitish
4927- )
4928- if check_commit_id:
4929- expected_commit_id = str(expected_commitish_obj.id)
4930- actual_commit_id = str(actual_commitish_obj.id)
4931- if expected_commit_id == actual_commit_id:
4932- self.success("Verified %s is the same commit as %s",
4933- expected_commitish_pretty_name, actual_commitish
4934- )
4935- else:
4936- warning("Expected %s (%s) is not the same commit as %s (%s)",
4937- expected_commitish_pretty_name, expected_commit_id,
4938- actual_commitish, actual_commit_id
4939- )
4940- ret = False
4941- expected_commit_tree_id = str(
4942- expected_commitish_obj.peel(pygit2.Tree).id
4943- )
4944- actual_commit_tree_id = str(
4945- actual_commitish_obj.peel(pygit2.Tree).id
4946- )
4947- if expected_commit_tree_id == actual_commit_tree_id:
4948- self.success("Verified %s is the same tree as %s",
4949+ try:
4950+ expected_commitish_obj = repo.get_commitish(expected_commitish)
4951+ except KeyError:
4952+ fatal("Unable to find expected commit: %s", expected_commitish)
4953+
4954+ actual_commitish_obj = repo.get_commitish(actual_commitish)
4955+ if check_commit_id:
4956+ expected_commit_id = str(expected_commitish_obj.id)
4957+ actual_commit_id = str(actual_commitish_obj.id)
4958+ if expected_commit_id == actual_commit_id:
4959+ success("Verified %s is the same commit as %s",
4960 expected_commitish_pretty_name, actual_commitish
4961 )
4962 else:
4963- error("Expected %s (%s) is not the same tree as %s (%s)",
4964- expected_commitish_pretty_name, expected_commit_tree_id,
4965- actual_commitish, actual_commit_tree_id
4966+ warning("Expected %s (%s) is not the same commit as %s (%s)",
4967+ expected_commitish_pretty_name, expected_commit_id,
4968+ actual_commitish, actual_commit_id
4969 )
4970 ret = False
4971+ expected_commit_tree_id = str(
4972+ expected_commitish_obj.peel(pygit2.Tree).id
4973+ )
4974+ actual_commit_tree_id = str(
4975+ actual_commitish_obj.peel(pygit2.Tree).id
4976+ )
4977+ if expected_commit_tree_id == actual_commit_tree_id:
4978+ success("Verified %s is the same tree as %s",
4979+ expected_commitish_pretty_name, actual_commitish
4980+ )
4981+ else:
4982+ error("Expected %s (%s) is not the same tree as %s (%s)",
4983+ expected_commitish_pretty_name, expected_commit_tree_id,
4984+ actual_commitish, actual_commit_tree_id
4985+ )
4986+ ret = False
4987
4988- return ret
4989+ return ret
4990
4991- def print_hunk(self, hunk):
4992- for line in hunk.lines:
4993- print("%s%s" % (line.origin, line.content.rstrip()))
4994+def print_hunk(hunk):
4995+ for line in hunk.lines:
4996+ print("%s%s" % (line.origin, line.content.rstrip()))
4997
4998- def print_patch(self, patch):
4999- for hunk in patch.hunks:
5000- self.print_hunk(hunk)
The diff has been truncated for viewing.

Subscribers

People subscribed via source and target branches