Merge ~nacc/git-ubuntu:lp1731554-importer-rework-v2 into git-ubuntu:master

Proposed by Nish Aravamudan
Status: Needs review
Proposed branch: ~nacc/git-ubuntu:lp1731554-importer-rework-v2
Merge into: git-ubuntu:master
Diff against target: 4178 lines (+2634/-771)
9 files modified
doc/SPECIFICATION.importer (+72/-0)
gitubuntu/dsc.py (+38/-0)
gitubuntu/git_repository.py (+23/-11)
gitubuntu/importer.py (+519/-586)
gitubuntu/importer_test.py (+1803/-43)
gitubuntu/importerutils.py (+50/-0)
gitubuntu/importlocal.py (+0/-2)
gitubuntu/importppa.py (+5/-16)
gitubuntu/source_information.py (+124/-113)
Reviewer Review Type Date Requested Status
Server Team CI bot continuous-integration Needs Fixing
git-ubuntu developers Pending
Review via email: mp+348109@code.launchpad.net

Description of the change

Make jenkins happy.

To post a comment you must log in.
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

FAILED: Continuous integration, rev:cba2c4e9c38a2b68864425894b8df2a99378e31f
https://jenkins.ubuntu.com/server/job/git-ubuntu-ci/15/
Executed test runs:
    SUCCESS: VM Setup
    SUCCESS: Build
    FAILED: Unit Tests

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/git-ubuntu-ci/15/rebuild

review: Needs Fixing (continuous-integration)

Unmerged commits

cba2c4e... by Nish Aravamudan

importer: rework algorithm to minimize looping

Basically, our current main importer loop looks like:

for applied in unapplied, applied:
    for dist in debian, ubuntu:
        for new publishes in $dist relative to $applied $dist branches
            import $applied publish
                this updates the branch pointer for the pocket + series
                if dist == 'ubuntu': this also updates the devel pointers

That is a fair amount of branch manipulation that will be discarded (the
series might have multiple publishes). Esp. on the first/reimport import
(worst case).

So instead:

for dist in debian, ubuntu:
    for applied in unapplied, applied:
        for new publishes in $dist relative to $applied $dist branches
            if publish has already been imported: continue
            import $applied publish

update all affected branch pointers,
    where affected is defined by running track of refs we *would* have
    updated in the original algorithm, and we also store the 'last'
    commit that it would have pointed to. This consolidates the branch
    and devel updates into one place as well.
    This also makes the importer algorithm match the spec on some level
    -- update the commit graph by importing only new publishes (which
    would create new import tags and new applied tags, while verifying
    any repeated publish data matches exactly) and then forcibly moving
    branch pointers to where they are "now" in Launchpad. The branch
    pointers are not part of the commit graph, so they change in a
    distinct step.

In my testing, this drops the reimport time for ipsec-tools from ~65
minutes to ~42 minutes consistently. We could probably speed it up more
starting from this base, if need be -- e.g., I'm not sure it makes sense
to to do the import action itself in the loop. Perhaps it's better to
accumulate the set of unique publishes in a set and then iterate that
set in a second loop.

Robie: this does not address your idempotency concern, yet, but I think
the solution is to make changes to importer::import_publishes so that
the order of publication events follows some specific rule. Right now,
it would be if it appears in Debian first, I think.

LP: #1731554

6dded16... by Nish Aravamudan

dsc: add compare_dsc method

FIXME: needs tests

c6e098f... by Nish Aravamudan

source_information: drop unused parent_{applied,}_head_name

These are no longer used after the importer publishing parent drop.

No functional change.

67662b8... by Nish Aravamudan

importer: fix patches-applied error

Used the wrong variable, should be squashed up.

b9ac1c3... by Nish Aravamudan

importer: fix importer algorithm to match spec

This is a large change, but the unit tests hopefully give us confidence
here.

The basic algorithm change is once we have the Git tree corresponding to
a DSC, we examine all suitable tags in the following order to determine
if the DSC has already been imported.

Patches-unapplied:
   Any existing import tag (includes reimport tags)
   Any existing upload tag
      - Algorithm change: import tag points directly to upload tag

Patches-applied:
   Any existing applied tag (includes reimport tags)

If a matching Tree tag is found, reuse the tagged Commit

If no matching Tree is found,

Patches-unapplied:
   Commit the Tree and import tag it (which might result in a reimport tag)

Patches-applied:
   Perform our patches-applied algorithm, commiting each interstitial
   tree, then commit the final Tree and applied tag it (which might result
   in a reimport tag)

In all cases, after find or creating an appropriate tag, move the
appropriate branch to point to it.

This is a significant simplification!

LP: #1761331
LP: #1754194
LP: #1754507
LP: #1754706
LP: #1734883
LP: #1761332
LP: #1763778

86a5c28... by Nish Aravamudan

importer: remove import tags that just point at upload tags

LP: #1734883

eed3b7c... by Nish Aravamudan

importer: fix detection of existing patches-applied imports

LP: #1763778

c1a66ef... by Nish Aravamudan

importer: various fixes related to tagged commits

What was before a single object (with the caller having to distinguish
between import and orphan tags) is now a list of objects always.

LP: #1755247

5404fa1... by Nish Aravamudan

importer: drop unused variable

No functional change

ffcb48d... by Nish Aravamudan

importer: purely cosmetic changes to fit with doc/STYLE.md

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1diff --git a/doc/SPECIFICATION.importer b/doc/SPECIFICATION.importer
2new file mode 100644
3index 0000000..8c7f80c
4--- /dev/null
5+++ b/doc/SPECIFICATION.importer
6@@ -0,0 +1,72 @@
7+When importing a publishing record, there are two potential logical
8+consequences:
9+
10+ 1) Some new commit is created, with an associated tag; and/or
11+ 2) Some branch is adjusted to point at some commit
12+
13+In the following, existing tags may be 'import', 'upload' or 'applied'.
14+Based upon SPECIFICATION.tags, 'import' and 'applied' tags are lists
15+tags, which might be a single import tag or a list of reimport tags for
16+any given version.
17+
18+If an existing tag with the same Git tree is found as a new publish
19+event, we should reuse the tag, and adjust the corresponding branch to
20+the same tagged commit.
21+
22+If no existing tag with same Git tree is found, a new import/applied tag
23+should be created, as appropriate, and the corresponding branch should
24+be adjusted to the the new tagged commit.
25+
26+If a new commit is created, appropriate commit parents are found in the
27+existing Git commit graph. A changelog parent is the nearest (reverse
28+chronologically) tagged version from debian/changelog of the publishing
29+record.
30+
31+For patches-unapplied imports, parents are:
32+ 1) import-tagged changelog parents.
33+
34+For patches-applied imports, parents are:
35+ 1) applied-tagged changelog parents
36+ 2) import-tagged patches-unapplied commit of the same version as
37+ this publishing record.
38+
39+Logically, therefore, it is assumed for any given version that the
40+patches-unapplied import occurs before the patches-applied import.
41+
42+Specific patches-unapplied cases, where in the following a tag exists
43+for a specific version:
44+
45+1) An existing import tag (or reimport tag) with the same Git tree
46+ - Reuse import tag
47+
48+2) An existing import tag with a different Git tree and an existing
49+ upload tag with the same Git tree
50+ - Reuse upload tag, create reimport tag
51+
52+3) An existing import tag with a different Git tree and an existing
53+ upload tag with a different Git tree
54+ - Create reimport tag
55+
56+4) An existing import tag with a different Git tree and no upload tag
57+ - Create reimport tag
58+
59+5) No import tag and an existing upload tag with the same Git tree
60+ - Reuse upload tag, create import tag
61+
62+6) No import tag and an existing upload tag with a different Git tree
63+ - Create import tag
64+
65+7) No import tags or upload tags
66+ - Create import tag
67+
68+Specific patches-applied cases, where in the following a tag exists
69+for a specific version:
70+
71+1) An existing applied tag (or reimport tag) with the same Git tree
72+ - Reuse applied tag
73+
74+2) An existing applied tag with a different Git tree
75+ - Create reimport tag
76+
77+3) No applied tags
78+ - Create applied tag
79diff --git a/gitubuntu/dsc.py b/gitubuntu/dsc.py
80index 1fc3129..91a78ea 100644
81--- a/gitubuntu/dsc.py
82+++ b/gitubuntu/dsc.py
83@@ -84,3 +84,41 @@ class GitUbuntuDsc(Dsc):
84 @property
85 def all_tarball_paths(self):
86 return [self.orig_tarball_path] + list(self.component_tarball_paths.values())
87+
88+ def _compare_dsc(self, other, field, key):
89+ if field not in self or field not in other:
90+ return False
91+ our_checksums = dict(
92+ (entry['name'], (int(entry['size']), entry[key]))
93+ for entry in self[field]
94+ )
95+ their_checksums = dict(
96+ (entry['name'], (int(entry['size']), entry[key]))
97+ for entry in other[field]
98+ )
99+ # check that every file in ours is in theirs
100+ for name, (size, checksum) in our_checksums.items():
101+ if name not in their_checksums:
102+ return False
103+ if size != their_checksums[name][0] or checksum != their_checksums[name][1]:
104+ return False
105+ # check that every file in theirs is in ours
106+ for name, (size, checksum) in their_checksums.items():
107+ if name not in our_checksums:
108+ return False
109+ if size != our_checksums[name][0] or checksum != our_checksums[name][1]:
110+ return False
111+ # all files in ours and theirs for this hash match
112+ return True
113+
114+ def compare_dsc(self, other):
115+ """Check whether some set of hashes in these two dscs are
116+ identical
117+ """
118+ return any(
119+ self._compare_dsc(other, field, key) for field, key in (
120+ ('Checksums-Sha256', 'sha256'),
121+ ('Checksums-Sha1', 'sha1'),
122+ ('Files', 'md5sum'),
123+ )
124+ )
125diff --git a/gitubuntu/git_repository.py b/gitubuntu/git_repository.py
126index 2a7d4dd..dc1e390 100644
127--- a/gitubuntu/git_repository.py
128+++ b/gitubuntu/git_repository.py
129@@ -742,9 +742,6 @@ def upload_tag(version, namespace):
130 def upstream_tag(version, namespace):
131 return '%s/upstream/%s' % (namespace, git_dep14_tag(version))
132
133-def orphan_tag(version, namespace):
134- return '%s/orphan/%s' % (namespace, git_dep14_tag(version))
135-
136 def is_dir_3_0_quilt(_dir=None):
137 _dir = _dir if _dir else '.'
138 try:
139@@ -930,6 +927,25 @@ class GitUbuntuRepository:
140 self.git_run(['checkout', '--orphan', 'master'])
141
142 @contextmanager
143+ def dsc_branch(self, dist, namespace='pkg'):
144+ """Context manager wrapping dsc branch manipulation
145+
146+ In this context, HEAD will point to the dsc branch for
147+ @namespace.
148+ """
149+ dsc_branch_name = '%s/importer/%s/dsc' % (namespace, dist)
150+ if not self.raw_repo.lookup_branch(dsc_branch_name):
151+ self.create_orphan_branch(
152+ dsc_branch_name,
153+ 'Initial %s dsc branch.' % dist,
154+ )
155+ with self.temporary_worktree(dsc_branch_name):
156+ try:
157+ yield
158+ except:
159+ raise
160+
161+ @contextmanager
162 def pristine_tar_branches(self, dist, namespace='pkg', create=True):
163 """Context manager wrapping pristine-tar branch manipulation
164
165@@ -1604,16 +1620,15 @@ class GitUbuntuRepository:
166 'Cannot get changelog source package name'
167 )
168
169- def get_heads_and_versions(self, head_prefix, namespace):
170+ def get_heads_and_versions(self, head_ref_namespace):
171 """Extract the last version in debian/changelog of all
172- '<namespace>/<head_prefix>/debian/*' and
173- '<namespace>/<head_prefix>/ubuntu/*' branches.
174+ '<namespace>/debian/*' and
175+ '<namespace>/ubuntu/*' branches.
176 """
177 versions = dict()
178 errors = False
179 for head in self.local_branches:
180- prefix = '%s/%s' % (namespace, head_prefix)
181- if not head.branch_name.startswith(prefix):
182+ if not head.branch_name.startswith(head_ref_namespace):
183 continue
184 if 'ubuntu/devel' in head.branch_name:
185 continue
186@@ -1679,9 +1694,6 @@ class GitUbuntuRepository:
187 def get_upstream_tag(self, version, namespace):
188 return self.get_tag_reference(upstream_tag(version, namespace))
189
190- def get_orphan_tag(self, version, namespace):
191- return self.get_tag_reference(orphan_tag(version, namespace))
192-
193 def create_tag(self, commit_hash, tag_name, tag_msg):
194 self.git_run(
195 [
196diff --git a/gitubuntu/importer.py b/gitubuntu/importer.py
197index d6b9db8..7b9a40d 100644
198--- a/gitubuntu/importer.py
199+++ b/gitubuntu/importer.py
200@@ -15,7 +15,6 @@
201 # Ubuntu tracking branch, OR match a newer parent version tag in debian
202 # tracking branch.
203 # Import will use the verified parent as the parent commit.
204-# If verification fails, then import anyway, but log and use an orphan commit.
205 #
206 # Results in a set of branches (git heads):
207 # - One per distribution/series/pocket
208@@ -25,8 +24,6 @@
209 # + the last imported version from debian/changelog
210
211 import argparse
212-import functools
213-import getpass
214 import logging
215 import os
216 import re
217@@ -45,17 +42,15 @@ from gitubuntu.dsc import (
218 from gitubuntu.git_repository import (
219 GitUbuntuRepository,
220 GitUbuntuRepositoryFetchError,
221- orphan_tag,
222 applied_tag,
223 import_tag,
224 reimport_tag,
225 applied_reimport_tag,
226- upstream_tag,
227 PristineTarError,
228 is_dir_3_0_quilt,
229 )
230+import gitubuntu.importerutils
231 from gitubuntu.run import (
232- decode_binary,
233 run,
234 runq,
235 run_gbp,
236@@ -204,74 +199,6 @@ def _main_with_repo(
237 except GitUbuntuRepositoryFetchError:
238 pass
239
240- debian_sinfo = GitUbuntuSourceInformation(
241- 'debian',
242- pkgname,
243- os.path.abspath(pullfile),
244- retries,
245- retry_backoffs,
246- )
247-
248- ubuntu_sinfo = GitUbuntuSourceInformation(
249- 'ubuntu',
250- pkgname,
251- os.path.abspath(pullfile),
252- retries,
253- retry_backoffs,
254- )
255-
256- debian_head_versions = (
257- repo.get_heads_and_versions('debian', namespace)
258- )
259- for head_name in sorted(debian_head_versions):
260- _, _, pretty_head_name = head_name.partition('%s/' % namespace)
261- logging.debug('Last imported %s version is %s',
262- pretty_head_name,
263- debian_head_versions[head_name]['version']
264- )
265-
266- ubuntu_head_versions = (
267- repo.get_heads_and_versions('ubuntu', namespace)
268- )
269- for head_name in sorted(ubuntu_head_versions):
270- _, _, pretty_head_name = head_name.partition('%s/' % namespace)
271- logging.debug('Last imported %s version is %s',
272- pretty_head_name,
273- ubuntu_head_versions[head_name]['version']
274- )
275-
276- if not skip_applied:
277- applied_debian_head_versions = (
278- repo.get_heads_and_versions(
279- 'applied/debian', namespace
280- )
281- )
282- for head_name in sorted(applied_debian_head_versions):
283- _, _, pretty_head_name = head_name.partition(
284- '%s/' % namespace
285- )
286- logging.debug('Last applied %s version is %s',
287- pretty_head_name,
288- applied_debian_head_versions[head_name]['version']
289- )
290-
291- applied_ubuntu_head_versions = (
292- repo.get_heads_and_versions(
293- 'applied/ubuntu', namespace
294- )
295- )
296- for head_name in sorted(applied_ubuntu_head_versions):
297- _, _, pretty_head_name = head_name.partition(
298- '%s/' % namespace
299- )
300- logging.debug('Last applied %s version is %s',
301- pretty_head_name,
302- applied_ubuntu_head_versions[head_name]['version']
303- )
304-
305- oldcwd = os.getcwd()
306- os.chdir(repo.local_dir)
307-
308 if dl_cache is None:
309 workdir = os.path.join(repo.git_dir, CACHE_PATH)
310 else:
311@@ -281,20 +208,22 @@ def _main_with_repo(
312
313 parent_overrides = parse_parentfile(parentfile, pkgname)
314
315+ oldcwd = os.getcwd()
316+ os.chdir(repo.local_dir)
317+
318 only_debian, history_found = import_publishes(
319 repo=repo,
320 pkgname=pkgname,
321 namespace=namespace,
322- patches_applied=False,
323- debian_head_versions=debian_head_versions,
324- ubuntu_head_versions=ubuntu_head_versions,
325- debian_sinfo=debian_sinfo,
326- ubuntu_sinfo=ubuntu_sinfo,
327 active_series_only=active_series_only,
328 workdir=workdir,
329 skip_orig=skip_orig,
330+ skip_applied=skip_applied,
331 allow_applied_failures=allow_applied_failures,
332 parent_overrides=parent_overrides,
333+ pullfile=pullfile,
334+ retries=retries,
335+ retry_backoffs=retry_backoffs,
336 )
337
338 if not history_found:
339@@ -302,30 +231,6 @@ def _main_with_repo(
340 "Wrong source package name?", pkgname)
341 return 1
342
343- if not skip_applied:
344- import_publishes(
345- repo=repo,
346- pkgname=pkgname,
347- namespace=namespace,
348- patches_applied=True,
349- debian_head_versions=applied_debian_head_versions,
350- ubuntu_head_versions=applied_ubuntu_head_versions,
351- debian_sinfo=debian_sinfo,
352- ubuntu_sinfo=ubuntu_sinfo,
353- active_series_only=active_series_only,
354- workdir=workdir,
355- skip_orig=skip_orig,
356- allow_applied_failures=allow_applied_failures,
357- parent_overrides=parent_overrides,
358- )
359-
360- update_devel_branches(
361- repo=repo,
362- namespace=namespace,
363- pkgname=pkgname,
364- ubuntu_sinfo=ubuntu_sinfo,
365- )
366-
367 os.chdir(oldcwd)
368
369 repo.garbage_collect()
370@@ -489,14 +394,15 @@ def main(
371 def get_changelog_for_commit(
372 repo,
373 tree_hash,
374- changelog_parent_commit,
375+ changelog_parent_commits,
376 ):
377 """Extract changes to debian/changelog in this publish
378
379 LP: #1633114 -- favor the changelog parent's diff
380 """
381 raw_clog_entry = b''
382- if changelog_parent_commit is not None:
383+ if len(changelog_parent_commits) == 1:
384+ changelog_parent_commit = changelog_parent_commits[0]
385 cmd = ['diff-tree', '-p', changelog_parent_commit,
386 tree_hash, '--', 'debian/changelog']
387 raw_clog_entry, _ = repo.git_run(cmd, decode=False)
388@@ -527,11 +433,9 @@ def get_changelog_for_commit(
389 def get_import_commit_msg(
390 repo,
391 version,
392- target_head_name,
393 namespace,
394 tree_hash,
395- changelog_parent_commit,
396- upload_parent_commit,
397+ changelog_parent_commits,
398 unapplied_parent_commit,
399 ):
400 if unapplied_parent_commit:
401@@ -540,28 +444,24 @@ def get_import_commit_msg(
402 import_type = 'patches-unapplied'
403
404 # Do not show importer/ namespace to user
405- _, _, pretty_head_name = target_head_name.partition('%s/' % namespace)
406 changelog_entry = get_changelog_for_commit(
407 repo,
408 tree_hash,
409- changelog_parent_commit
410+ changelog_parent_commits,
411 )
412
413 msg = (
414- b'Import %s version %b to %b\n\nImported using git-ubuntu import.' % (
415+ b'Import %s version %b\n\nImported using git-ubuntu import.' % (
416 import_type.encode(),
417 version.encode(),
418- pretty_head_name.encode(),
419 )
420 )
421
422- if any([changelog_parent_commit, upload_parent_commit, unapplied_parent_commit]):
423+ if changelog_parent_commits or unapplied_parent_commit:
424 msg += b'\n'
425
426- if changelog_parent_commit is not None:
427+ for changelog_parent_commit in changelog_parent_commits:
428 msg += b'\nChangelog parent: %b' % changelog_parent_commit.encode()
429- if upload_parent_commit is not None:
430- msg += b'\nUpload parent: %b' % upload_parent_commit.encode()
431 if unapplied_parent_commit is not None:
432 msg += b'\nUnapplied parent: %b' % unapplied_parent_commit.encode()
433
434@@ -572,11 +472,9 @@ def get_import_commit_msg(
435 def _commit_import(
436 repo,
437 version,
438- target_head_name,
439 tree_hash,
440 namespace,
441- changelog_parent_commit,
442- upload_parent_commit,
443+ changelog_parent_commits,
444 unapplied_parent_commit,
445 fallback_author,
446 fallback_date,
447@@ -593,11 +491,10 @@ def _commit_import(
448 which the source package contents corresponding to @spi should
449 be imported.
450 :param version str Changelog version
451- :param target_head_name str Git branch name to import to
452 :param namespace str Namespace under which relevant refs can be
453 found.
454- :param changelog_parent_commit str Git commit hash of
455- changelog parent.
456+ :param changelog_parent_commits list(str) Git commit hashes of
457+ changelog parents.
458 :param upload_parent_commit str Git commit hash of
459 upload parent. Should be None if patches-applied import.
460 :param unapplied_parent_commit str Git commit hash of unapplied
461@@ -610,18 +507,10 @@ def _commit_import(
462 :rtype str
463 :returns Hash of created commit
464 """
465- changelog_entry = get_changelog_for_commit(
466- repo,
467- tree_hash,
468- changelog_parent_commit
469- )
470-
471 parents = []
472
473- if changelog_parent_commit is not None:
474- parents.append(changelog_parent_commit)
475- if upload_parent_commit is not None:
476- parents.append(upload_parent_commit)
477+ if changelog_parent_commits:
478+ parents.extend(changelog_parent_commits)
479 if unapplied_parent_commit is not None:
480 parents.append(unapplied_parent_commit)
481
482@@ -631,11 +520,9 @@ def _commit_import(
483 get_import_commit_msg(
484 repo=repo,
485 version=version,
486- target_head_name=target_head_name,
487 namespace=namespace,
488 tree_hash=tree_hash,
489- changelog_parent_commit=changelog_parent_commit,
490- upload_parent_commit=upload_parent_commit,
491+ changelog_parent_commits=changelog_parent_commits,
492 unapplied_parent_commit=unapplied_parent_commit,
493 ),
494 fallback_author,
495@@ -647,53 +534,76 @@ def _commit_import(
496 def commit_unapplied_patches_import(
497 repo,
498 version,
499- head_name,
500 import_tree_hash,
501 namespace,
502- changelog_parent_commit,
503- upload_parent_commit,
504+ changelog_parent_commits,
505 fallback_author,
506 fallback_date,
507 ):
508 return _commit_import(
509 repo,
510 version,
511- head_name,
512 import_tree_hash,
513 namespace,
514- changelog_parent_commit,
515- upload_parent_commit,
516+ changelog_parent_commits,
517 # unapplied trees do not have a distinct unapplied parent
518 None,
519 fallback_author,
520 fallback_date,
521 )
522
523-def commit_applied_patches_import(
524+
525+def dsc_imported(
526 repo,
527- version,
528- head_name,
529- import_tree_hash,
530+ spi,
531 namespace,
532- changelog_parent_commit,
533- unapplied_parent_commit,
534- fallback_author,
535- fallback_date,
536+ distname,
537 ):
538- return _commit_import(
539- repo,
540- version,
541- head_name,
542- import_tree_hash,
543- namespace,
544- changelog_parent_commit,
545- # uploads will be unapplied trees currently, so applied trees
546- # can not have them as direct parents
547- None,
548- unapplied_parent_commit,
549- fallback_author,
550- fallback_date,
551- )
552+ # always check if the same dist has a matching dsc
553+ with repo.dsc_branch(
554+ namespace=namespace,
555+ dist=distname,
556+ ):
557+ cached_dsc_path = os.path.join(
558+ os.getcwd(),
559+ spi.version,
560+ os.path.basename(spi.dsc_pathname),
561+ )
562+
563+ if os.path.exists(cached_dsc_path):
564+ # If we have imported this version before, compare the
565+ # DSC in the DSC branch to this one
566+ imported_dsc = GitUbuntuDsc(cached_dsc_path)
567+ return GitUbuntuDsc(spi.dsc_pathname).compare_dsc(imported_dsc)
568+
569+ # if a debian dsc exists and matches, but an ubuntu dsc does not
570+ # exist, import the ubuntu dsc, but signal that a full import is
571+ # not necessary
572+ if distname == 'ubuntu':
573+ with repo.dsc_branch(
574+ namespace=namespace,
575+ dist='debian',
576+ ):
577+ imported_dsc_path = os.path.join(
578+ os.getcwd(),
579+ spi.version,
580+ os.path.basename(spi.dsc_pathname),
581+ )
582+
583+ if os.path.exists(imported_dsc_path):
584+ # If we have imported this version before, compare the
585+ # DSC in the DSC branch to this one
586+ imported_dsc = GitUbuntuDsc(imported_dsc_path)
587+ if GitUbuntuDsc(spi.dsc_pathname).compare_dsc(imported_dsc):
588+ import_dsc(
589+ repo,
590+ GitUbuntuDsc(spi.dsc_pathname),
591+ namespace,
592+ str(spi.version),
593+ spi.distribution.name.lower(),
594+ )
595+ return True
596+ return False
597
598 def import_dsc(repo, dsc, namespace, version, dist):
599 """Imports the dsc file to importer/{debian,ubuntu}/dsc
600@@ -1018,45 +928,40 @@ def get_existing_applied_tags(repo, version, namespace):
601
602 def override_parents(parent_overrides, repo, version, namespace):
603 """
604- returns: (unapplied, applied) where both elements of the tuple are commit
605- hash strings of the changelog parent to use, incorporating any defined
606- overrides.
607- rtype: tuple(str, str)
608+ returns: (unapplied, applied) where both elements of the tuple are
609+ lists of commit hash strings of the changelog parent to use,
610+ incorporating any defined overrides.
611+ rtype: tuple(list(str), list(str))
612 """
613- unapplied_changelog_parent_commit = None
614- applied_changelog_parent_commit = None
615+ unapplied_changelog_parent_commits = []
616+ applied_changelog_parent_commits = []
617
618- unapplied_changelog_parent_tag = repo.get_import_tag(
619+ unapplied_changelog_parent_tags = get_existing_import_tags(
620+ repo,
621 parent_overrides[version]['changelog_parent'],
622- namespace
623+ namespace,
624 )
625- applied_changelog_parent_tag = repo.get_applied_tag(
626+ applied_changelog_parent_tags = get_existing_applied_tags(
627+ repo,
628 parent_overrides[version]['changelog_parent'],
629- namespace
630+ namespace,
631 )
632- if unapplied_changelog_parent_tag is not None:
633- # sanity check that version from d/changelog of the
634- # tagged commit matches ours
635- parent_changelog_version, _ = repo.get_changelog_versions_from_treeish(
636- str(unapplied_changelog_parent_tag.peel().id),
637- )
638- if parent_changelog_version != parent_overrides[version]['changelog_parent']:
639- logging.error('Found a tag corresponding to '
640- 'changelog parent override '
641- 'version %s, but d/changelog '
642- 'version (%s) differs. Will '
643- 'orphan commit.' % (
644- parent_overrides[version]['changelog_parent'],
645- parent_changelog_version
646- )
647- )
648- else:
649- unapplied_changelog_parent_commit = str(unapplied_changelog_parent_tag.peel().id)
650- if applied_changelog_parent_tag is not None:
651- applied_changelog_parent_commit = str(applied_changelog_parent_tag.peel().id)
652- logging.debug('Overriding changelog parent (tag) to %s',
653- repo.tag_to_pretty_name(unapplied_changelog_parent_tag)
654- )
655+ # sanity check that version from d/changelog of the
656+ # tagged commit matches ours
657+ if any(
658+ repo.get_changelog_versions_from_treeish(
659+ str(tag.peel(pygit2.Tree).id)
660+ )[0] == parent_overrides[version]['changelog_parent']
661+ for tag in unapplied_changelog_parent_tags
662+ ):
663+ unapplied_changelog_parent_commits = [
664+ str(tag.peel(pygit2.Commit).id)
665+ for tag in unapplied_changelog_parent_tags
666+ ]
667+ applied_changelog_parent_commits = [
668+ str(tag.peel(pygit2.Commit).id)
669+ for tag in applied_changelog_parent_tags
670+ ]
671 else:
672 if parent_overrides[version]['changelog_parent'] == '-':
673 logging.debug('Not setting changelog parent as specified '
674@@ -1071,8 +976,8 @@ def override_parents(parent_overrides, repo, version, namespace):
675 )
676 )
677 return (
678- unapplied_changelog_parent_commit,
679- applied_changelog_parent_commit,
680+ unapplied_changelog_parent_commits,
681+ applied_changelog_parent_commits,
682 )
683
684 def _devel_branch_updates(
685@@ -1100,7 +1005,7 @@ def _devel_branch_updates(
686
687 devel_hash = None
688 devel_name = None
689- devel_branch_name = ref_prefix + 'ubuntu/devel'
690+ devel_branch_name = "%s/devel" % ref_prefix
691
692 for series_name in series_name_list:
693 series_devel_hash = None
694@@ -1109,7 +1014,7 @@ def _devel_branch_updates(
695 # Find highest version publish in these pockets, favoring this
696 # order of pockets
697 for suff in ['-proposed', '-updates', '-security', '']:
698- name = "%subuntu/%s%s" % (ref_prefix, series_name, suff)
699+ name = "%s/%s%s" % (ref_prefix, series_name, suff)
700 if name in head_versions:
701 head_commit, head_version = head_versions[name]
702 if version_compare(head_version, series_devel_version) > 0:
703@@ -1117,7 +1022,7 @@ def _devel_branch_updates(
704 series_devel_version = head_version
705 series_devel_hash = head_commit
706 if series_devel_hash:
707- series_devel_branch_name = '%subuntu/%s-devel' % (
708+ series_devel_branch_name = '%s/%s-devel' % (
709 ref_prefix,
710 series_name,
711 )
712@@ -1132,62 +1037,73 @@ def _devel_branch_updates(
713 devel_name = series_devel_name
714 devel_hash = series_devel_hash
715
716- yield ref_prefix + 'ubuntu/devel', devel_hash
717+ yield devel_branch_name, devel_hash
718
719
720-def update_devel_branches(
721+def update_branches(
722+ distname,
723+ dist_sinfo,
724 repo,
725 namespace,
726 pkgname,
727- ubuntu_sinfo,
728+ updated_head_refs,
729 ):
730- """update_devel_branches - move the 'meta' -devel branches to the
731- latest publication in each series
732+ """update_branches - move the series+pocket branches to the
733+ latest appropriate publication in each series
734
735 For all known series in @ubuntu_sinfo, update the
736 ubuntu/series-devel branch to the latest version in the series'
737 pocket branches.
738
739- Update the ubuntu/devel branch to the latest ubuntu/series-devel
740- branch.
741-
742- Do this for both the unapplied and applied branches.
743+ If distname is 'ubuntu', ubuntu/<series>-devel branches are also
744+ updated, and ubuntu/devel will point to the latest
745+ ubuntu/<series>-devel branch.
746
747 Arguments:
748- repo - gitubuntu.git_repository.GitUbuntuRepository object
749- pkgname - string source package name
750- namespace - string namespace under which the relevant branch refs
751- can be found
752- ubuntu_sinfo - GitUbuntuSourceInformation object for the ubuntu
753- archive
754+ distname str: distribution name (one of 'debian', 'ubuntu')
755+ dist_sinfo gitubuntu.source_information.GitUbuntuSourceInformation:
756+ Launchpad publication data for a given distribution
757+ repo gitubuntu.git_repository.GitUbuntuRepository object: repository
758+ to act on
759+ namespace str: namespace under which the relevant branch
760+ refs can be found
761+ pkgname str: source package name
762 """
763- for applied_prefix in ['', 'applied/']:
764- head_versions = {
765- ref: (str(target['head'].peel().id), target['version'])
766- for ref, target in repo.get_heads_and_versions(
767- '%subuntu' % applied_prefix,
768- namespace=namespace,
769- ).items()
770- }
771- for ref, commit in _devel_branch_updates(
772- ref_prefix=('%s/%s' % (namespace, applied_prefix)),
773- head_versions=head_versions,
774- series_name_list=ubuntu_sinfo.all_series_name_list,
775- ):
776- if commit is None:
777- logging.warn(
778- "Source package '%s' does not appear to have been published "
779- "in Ubuntu. No %s created.",
780- pkgname,
781- ref,
782- )
783- else:
784- logging.debug(
785- "Setting %s to '%s'",
786- ref,
787- commit,
788- )
789- repo.update_head_to_commit(ref, commit)
790+ for ref, commit_hash in updated_head_refs.items():
791+ repo.update_head_to_commit(
792+ ref,
793+ commit_hash,
794+ )
795+
796+ if distname == 'debian':
797+ return
798+
799+ head_versions = {
800+ ref: (str(target['head'].peel().id), target['version'])
801+ for ref, target in repo.get_heads_and_versions(
802+ namespace,
803+ ).items()
804+ }
805+
806+ for ref, commit in _devel_branch_updates(
807+ ref_prefix=namespace,
808+ head_versions=head_versions,
809+ series_name_list=dist_sinfo.all_series_name_list
810+ ):
811+ if commit is None:
812+ logging.warn(
813+ "Source package '%s' does not appear to have been published "
814+ "in Ubuntu. No %s created.",
815+ pkgname,
816+ ref,
817+ )
818+ else:
819+ logging.debug(
820+ "Setting %s to '%s'",
821+ ref,
822+ commit,
823+ )
824+ repo.update_head_to_commit(ref, commit)
825
826
827 def get_import_tag_msg():
828@@ -1294,7 +1210,7 @@ def create_import_tag(repo, commit_hash, version, namespace):
829 return repo.get_import_tag(version, namespace)
830
831
832-def get_changelog_parent_commit(
833+def get_changelog_parent_commits(
834 repo,
835 namespace,
836 parent_overrides,
837@@ -1320,44 +1236,52 @@ def get_changelog_parent_commit(
838
839 for version in import_tree_versions:
840 if patches_applied:
841- changelog_parent_tag = repo.get_applied_tag(
842+ changelog_parent_tags = get_existing_applied_tags(
843+ repo,
844 version,
845 namespace,
846 )
847 else:
848- changelog_parent_tag = repo.get_import_tag(
849- version,
850- namespace,
851- ) or repo.get_orphan_tag(
852+ changelog_parent_tags = get_existing_import_tags(
853+ repo,
854 version,
855 namespace,
856 )
857
858- if not changelog_parent_tag:
859+ if not changelog_parent_tags:
860 continue
861
862 # sanity check that version from d/changelog of the
863 # tagged parent matches ours
864- changelog_parent_version, _ = repo.get_changelog_versions_from_treeish(
865- str(changelog_parent_tag.peel(pygit2.Tree).id)
866- )
867+ changelog_parent_versions = [
868+ repo.get_changelog_versions_from_treeish(
869+ str(tag.peel(pygit2.Tree).id)
870+ )[0] for tag in changelog_parent_tags
871+ ]
872
873 # if the parent was imported via a parent override
874 # itself, assume it is valid without checking its
875 # tree's debian/changelog, as it wasn't used
876 if (version in parent_overrides or
877- changelog_parent_version == version
878- ):
879- changelog_parent_commit = str(
880- changelog_parent_tag.peel(pygit2.Commit).id
881+ any(
882+ [changelog_parent_version == version
883+ for changelog_parent_version in changelog_parent_versions]
884 )
885-
886- logging.debug("Changelog parent (tag) is %s",
887- repo.tag_to_pretty_name(changelog_parent_tag)
888+ ):
889+ changelog_parent_commits = [
890+ str(tag.peel(pygit2.Commit).id)
891+ for tag in changelog_parent_tags
892+ ]
893+
894+ logging.debug("Changelog parent(s) (tag(s)) are %s",
895+ [
896+ repo.tag_to_pretty_name(tag) for tag in
897+ changelog_parent_tags
898+ ]
899 )
900- return changelog_parent_commit
901+ return changelog_parent_commits
902
903- return None
904+ return []
905
906
907 def import_unapplied_dsc(
908@@ -1366,7 +1290,6 @@ def import_unapplied_dsc(
909 namespace,
910 dist,
911 dsc_pathname,
912- head_name,
913 skip_orig,
914 parent_overrides,
915 fallback_author,
916@@ -1382,7 +1305,6 @@ def import_unapplied_dsc(
917 found.
918 :param dist str Either 'debian' or 'ubuntu' or None
919 :param dsc_pathname str Filesystem path to a DSC file
920- :param head_name str Git branch name to import the DSC to
921 :param skip_orig bool If True, do not attempt to import orig
922 tarballs. Useful for debug import runs to make them as fast as
923 possible, or to workaround pristine-tar issues.
924@@ -1394,9 +1316,6 @@ def import_unapplied_dsc(
925
926 :rtype None
927 """
928- unapplied_tip_head = repo.get_head_by_name(head_name)
929- _, _, pretty_head_name = head_name.partition('%s/' % namespace)
930-
931 import_dsc(
932 repo,
933 GitUbuntuDsc(dsc_pathname),
934@@ -1445,26 +1364,6 @@ def import_unapplied_dsc(
935 'source pkg version: {} != changelog version: {}'.format(
936 version, changelog_version))
937
938- unapplied_tip_version = None
939- # check if the version to import has already been imported to
940- # this head
941- if unapplied_tip_head is not None:
942- if repo.treeishs_identical(
943- unapplied_import_tree_hash, str(unapplied_tip_head.peel().id)
944- ):
945- logging.warn('%s is identical to %s',
946- pretty_head_name, version
947- )
948- return
949- unapplied_tip_version, _ = repo.get_changelog_versions_from_treeish(
950- str(unapplied_tip_head.peel().id),
951- )
952-
953- logging.debug('Tip version is %s', unapplied_tip_version)
954-
955- unapplied_changelog_parent_commit = None
956- upload_parent_commit = None
957-
958 if version in parent_overrides:
959 logging.debug(
960 '%s is specified in the parent override file.',
961@@ -1473,7 +1372,7 @@ def import_unapplied_dsc(
962
963 try:
964 (
965- unapplied_changelog_parent_commit,
966+ unapplied_changelog_parent_commits,
967 _
968 ) = override_parents(
969 parent_overrides,
970@@ -1485,15 +1384,7 @@ def import_unapplied_dsc(
971 logging.error("%s" % e)
972 return
973 else:
974- if version_compare(version, unapplied_tip_version) <= 0:
975- logging.warn(
976- "Version to import (%s) is not after %s tip (%s)",
977- version,
978- pretty_head_name,
979- unapplied_tip_version,
980- )
981-
982- unapplied_changelog_parent_commit = get_changelog_parent_commit(
983+ unapplied_changelog_parent_commits = get_changelog_parent_commits(
984 repo,
985 namespace,
986 parent_overrides,
987@@ -1501,111 +1392,52 @@ def import_unapplied_dsc(
988 patches_applied=False,
989 )
990
991- # check if the version to import has already been uploaded to
992- # this head -- we leverage the above code to perform a 'dry-run'
993- # of the import tree, to obtain the changelog parent
994+ # we can only now be in this method, if the spi (dsc) has not been
995+ # imported before, so we do not need to check for an existing import
996+ # tag
997+ target_tag = None
998 existing_upload_tag = repo.get_upload_tag(version, namespace)
999- existing_import_tag = repo.get_import_tag(version, namespace)
1000- existing_orphan_tag = repo.get_orphan_tag(version, namespace)
1001- if existing_import_tag:
1002- if str(existing_import_tag.peel().tree.id) != unapplied_import_tree_hash:
1003- logging.error(
1004- "Found import tag %s, but the corresponding tree "
1005- "does not match the published version. Will orphan commit.",
1006- repo.tag_to_pretty_name(existing_import_tag),
1007- )
1008- unapplied_changelog_parent_commit = None
1009- else:
1010- repo.update_head_to_commit(
1011- head_name,
1012- str(existing_import_tag.peel().id),
1013- )
1014- return
1015- elif existing_orphan_tag:
1016- if str(existing_orphan_tag.peel().tree.id) != unapplied_import_tree_hash:
1017- raise Exception(
1018- "Found orphan tag %s, but the corresponding tree "
1019- "does not match the published version. We cannot "
1020- "currently support multiple orphan tags for the same "
1021- "published version and will now exit.",
1022- repo.tag_to_pretty_name(existing_orphan_tag),
1023- )
1024- else:
1025- repo.update_head_to_commit(
1026- head_name,
1027- str(existing_orphan_tag.peel().id),
1028- )
1029- return
1030- elif existing_upload_tag:
1031- if str(existing_upload_tag.peel().tree.id) != unapplied_import_tree_hash:
1032- logging.warn(
1033- "Found upload tag %s, but the corresponding tree "
1034- "does not match the published version. Will import %s as "
1035- "normal, ignoring the upload tag.",
1036- repo.tag_to_pretty_name(existing_upload_tag),
1037- version,
1038- )
1039- else:
1040- upload_parent_commit = str(existing_upload_tag.peel().id)
1041-
1042- if unapplied_changelog_parent_commit is not None:
1043+ if existing_upload_tag:
1044+ if str(existing_upload_tag.peel(pygit2.Tree).id) == unapplied_import_tree_hash:
1045+ for unapplied_changelog_parent_commit in unapplied_changelog_parent_commits:
1046 try:
1047 repo.git_run(
1048 [
1049 'merge-base',
1050 '--is-ancestor',
1051 unapplied_changelog_parent_commit,
1052- upload_parent_commit,
1053+ str(existing_upload_tag.peel(pygit2.Commit).id),
1054 ],
1055 verbose_on_failure=False,
1056 )
1057- unapplied_changelog_parent_commit = None
1058+ commit_hash = str(existing_upload_tag.peel(pygit2.Commit).id)
1059+ target_tag = create_import_tag(repo, commit_hash, version, namespace)
1060+ break
1061 except CalledProcessError as e:
1062 if e.returncode != 1:
1063 raise
1064+ else:
1065+ logging.warning(
1066+ "An upload tag was found for version %s, however it "
1067+ "does not have any appropriate Git ancestor (for the "
1068+ "prior changelog version). This is currently unsupported "
1069+ "and we will not integrate this upload tag.",
1070+ version,
1071+ )
1072
1073- commit_hash = commit_unapplied_patches_import(
1074- repo,
1075- version,
1076- head_name,
1077- unapplied_import_tree_hash,
1078- namespace,
1079- unapplied_changelog_parent_commit,
1080- upload_parent_commit,
1081- fallback_author,
1082- fallback_date,
1083- )
1084- logging.debug(
1085- "Committed patches-unapplied import of %s as %s in %s",
1086- version,
1087- commit_hash,
1088- pretty_head_name,
1089- )
1090-
1091- tag = None
1092- if repo.get_import_tag(version, namespace) is None:
1093- # Not imported before
1094- tag = import_tag(version, namespace)
1095- elif (
1096- unapplied_changelog_parent_commit is None and
1097- upload_parent_commit is None and
1098- head_name in repo.local_branch_names
1099- ):
1100- # No parents and not creating a new branch
1101- tag = orphan_tag(version, namespace)
1102-
1103- if tag:
1104- logging.debug("Creating tag %s pointing to %s", tag, commit_hash)
1105- repo.create_tag(
1106- commit_hash=commit_hash,
1107- tag_name=tag,
1108- tag_msg=get_import_tag_msg(),
1109+ if not target_tag:
1110+ commit_hash = commit_unapplied_patches_import(
1111+ repo=repo,
1112+ version=version,
1113+ import_tree_hash=unapplied_import_tree_hash,
1114+ namespace=namespace,
1115+ changelog_parent_commits=unapplied_changelog_parent_commits,
1116+ fallback_author=fallback_author,
1117+ fallback_date=fallback_date,
1118 )
1119+ target_tag = create_import_tag(repo, commit_hash, version, namespace)
1120
1121- repo.update_head_to_commit(
1122- head_name,
1123- commit_hash,
1124- )
1125+ return str(target_tag.peel(pygit2.Commit).id)
1126
1127 # imports a package based upon source package information
1128 def import_unapplied_spi(
1129@@ -1618,6 +1450,8 @@ def import_unapplied_spi(
1130 """Imports a publication record from Launchpad into the git
1131 repository
1132
1133+ Only called if the version has not been imported before
1134+
1135 :param repo gitubuntu.GitUbuntuRepository The Git repository into
1136 which the source package contents corresponding to @spi should
1137 be imported.
1138@@ -1632,11 +1466,11 @@ def import_unapplied_spi(
1139
1140 :rtype None
1141 """
1142- pretty_head_name = spi.pretty_head_name
1143+ logging.info('Importing patches-unapplied %s', spi.version)
1144
1145- logging.info('Importing patches-unapplied %s to %s',
1146- spi.version, pretty_head_name
1147- )
1148+ # This assert feels right, but I think will be wrong for the case of
1149+ # reimports
1150+ # assert not repo.get_import_tag(spi.version, namespace)
1151
1152 spi.pull()
1153
1154@@ -1649,13 +1483,12 @@ def import_unapplied_spi(
1155 if spi.date_published:
1156 fallback_date = str(spi.date_published)
1157
1158- import_unapplied_dsc(
1159+ return import_unapplied_dsc(
1160 repo=repo,
1161 version=str(spi.version),
1162 namespace=namespace,
1163 dist=spi.distribution.name.lower(),
1164 dsc_pathname=spi.dsc_pathname,
1165- head_name=spi.head_name(namespace),
1166 skip_orig=skip_orig,
1167 parent_overrides=parent_overrides,
1168 fallback_author=fallback_author,
1169@@ -1668,7 +1501,6 @@ def import_applied_dsc(
1170 namespace,
1171 dist,
1172 dsc_pathname,
1173- head_name,
1174 allow_applied_failures,
1175 parent_overrides,
1176 fallback_author,
1177@@ -1684,7 +1516,6 @@ def import_applied_dsc(
1178 found.
1179 :param dist str Either 'debian' or 'ubuntu' or None
1180 :param dsc_pathname str Filesystem path to a DSC file
1181- :param head_name str Git branch name to import the DSC to
1182 :param allow_applied_failures bool If True, failure to apply quilt
1183 patches is not fatal.
1184 :param parent_overrides dict See parse_parentfile.
1185@@ -1695,12 +1526,18 @@ def import_applied_dsc(
1186
1187 :rtype None
1188 """
1189- applied_tip_head = repo.get_head_by_name(head_name)
1190- _, _, pretty_head_name = head_name.partition('%s/' % namespace)
1191-
1192- unapplied_parent_tag = repo.get_import_tag(version, namespace)
1193- unapplied_parent_commit = str(unapplied_parent_tag.peel().id)
1194- unapplied_import_tree_hash = str(unapplied_parent_tag.peel(pygit2.Tree).id)
1195+ unapplied_parent_tags = get_existing_import_tags(
1196+ repo,
1197+ version,
1198+ namespace,
1199+ )
1200+ unapplied_import_tree_hash = dsc_to_tree_hash(
1201+ repo.raw_repo,
1202+ dsc_pathname,
1203+ )
1204+ for tag in unapplied_parent_tags:
1205+ if str(tag.peel(pygit2.Tree).id) == unapplied_import_tree_hash:
1206+ unapplied_parent_commit = str(tag.peel(pygit2.Commit).id)
1207
1208 # this is the result of all patches being applied
1209 applied_import_tree_hash = dsc_to_tree_hash(
1210@@ -1716,37 +1553,21 @@ def import_applied_dsc(
1211 )
1212
1213 import_tree_versions = repo.get_all_changelog_versions_from_treeish(
1214- unapplied_import_tree_hash
1215- )
1216+ unapplied_import_tree_hash
1217+ )
1218 changelog_version = import_tree_versions[0]
1219
1220- applied_tip_version = None
1221- # check if the version to import has already been imported to
1222- # this head
1223- if applied_tip_head is not None:
1224- if repo.treeishs_identical(
1225- unapplied_import_tree_hash, str(applied_tip_head.peel().id)
1226- ):
1227- logging.warn('%s is identical to %s',
1228- pretty_head_name, version
1229- )
1230- return
1231- applied_tip_version, _ = repo.get_changelog_versions_from_treeish(
1232- str(applied_tip_head.peel().id),
1233- )
1234-
1235- logging.debug('Tip version is %s', applied_tip_version)
1236-
1237- applied_changelog_parent_commit = None
1238+ applied_changelog_parent_commits = []
1239
1240 if version in parent_overrides:
1241- logging.debug('%s is specified in the parent override file.' %
1242- version
1243- )
1244+ logging.debug(
1245+ "%s is specified in the parent override file.",
1246+ version,
1247+ )
1248
1249 (
1250 _,
1251- applied_changelog_parent_commit,
1252+ applied_changelog_parent_commits,
1253 ) = override_parents(
1254 parent_overrides,
1255 repo,
1256@@ -1754,13 +1575,7 @@ def import_applied_dsc(
1257 namespace,
1258 )
1259 else:
1260- if version_compare(version, applied_tip_version) <= 0:
1261- logging.warn('Version to import (%s) is not after %s tip (%s)',
1262- version, pretty_head_name, applied_tip_version
1263- )
1264-
1265-
1266- applied_changelog_parent_commit = get_changelog_parent_commit(
1267+ applied_changelog_parent_commits = get_changelog_parent_commits(
1268 repo,
1269 namespace,
1270 parent_overrides,
1271@@ -1768,109 +1583,55 @@ def import_applied_dsc(
1272 patches_applied=True,
1273 )
1274
1275- existing_applied_tag = repo.get_applied_tag(version, namespace)
1276- if existing_applied_tag:
1277- # XXX: What should be checked here? The result of pushing all
1278- # the patches? How to do it most non-destructively? (Might be
1279- # able to use our new helper to get treeish after running a
1280- # command, in this case, `quilt push -a`)
1281- # if str(applied_tag.peel().tree.id) != applied_import_tree_hash:
1282- # logging.error(
1283- # "Found patches-applied import tag %s, but the "
1284- # "corresponding tree does not match the published "
1285- # "version. Will orphan commit.",
1286- # repo.tag_to_pretty_name(applied_tag),
1287- # )
1288- # applied_changelog_parent_commit = None
1289- #else:
1290- repo.update_head_to_commit(
1291- head_name,
1292- str(existing_applied_tag.peel().id),
1293- )
1294- return
1295-
1296- # Assume no patches to apply
1297- applied_import_tree_hash = unapplied_import_tree_hash
1298- # get tree id from above commit
1299- try:
1300- for (
1301- applied_import_tree_hash,
1302- patch_name,
1303- patch_desc
1304- ) in import_patches_applied_tree(repo, dsc_pathname):
1305- # special case for .pc removal
1306- if patch_name is None:
1307- msg = b'%b.' % (patch_desc.encode())
1308- else:
1309- if patch_desc is None:
1310- patch_desc = (
1311- "%s\n\nNo DEP3 Subject or Description header found" %
1312- patch_name
1313- )
1314- msg = b'%b\n\nGbp-Pq: %b.' % (
1315- patch_desc.encode(),
1316- patch_name.encode()
1317- )
1318-
1319- unapplied_parent_commit = repo.commit_tree_hash(
1320- applied_import_tree_hash,
1321- [unapplied_parent_commit],
1322- msg,
1323- fallback_author,
1324- fallback_date,
1325- )
1326-
1327- logging.debug(
1328- "Committed patch-application of %s as %s",
1329- patch_name, unapplied_parent_commit
1330- )
1331- except CalledProcessError as e:
1332- if allow_applied_failures:
1333- return
1334- raise
1335-
1336- commit_hash = commit_applied_patches_import(
1337+ target_tag = None
1338+ for existing_applied_tag in get_existing_applied_tags(
1339 repo,
1340 version,
1341- head_name,
1342- applied_import_tree_hash,
1343 namespace,
1344- applied_changelog_parent_commit,
1345- unapplied_parent_commit,
1346- fallback_author,
1347- fallback_date,
1348- )
1349- logging.debug(
1350- "Committed patches-applied import of %s as %s in %s",
1351- version,
1352- commit_hash,
1353- pretty_head_name,
1354- )
1355-
1356- tag = None
1357- if repo.get_applied_tag(version, namespace) is None:
1358- # Not imported before
1359- tag = applied_tag(version, namespace)
1360- elif (
1361- applied_changelog_parent_commit is None and
1362- unapplied_parent_commit is None and
1363- head_name in repo.local_branch_names
1364 ):
1365- # No parents and not creating a new branch
1366- tag = orphan_tag(version, namespace)
1367+ if str(
1368+ existing_applied_tag.peel(pygit2.Tree).id
1369+ ) == applied_import_tree_hash:
1370+ target_tag = existing_applied_tag
1371+
1372+ if not target_tag:
1373+ # Assume no patches to apply
1374+ applied_import_tree_hash = unapplied_import_tree_hash
1375+ # get tree id from above commit
1376+ try:
1377+ for (
1378+ applied_import_tree_hash,
1379+ patch_name,
1380+ patch_desc
1381+ ) in import_patches_applied_tree(repo, dsc_pathname):
1382+ # special case for .pc removal
1383+ if patch_name is None:
1384+ msg = b'%b.' % (patch_desc.encode())
1385+ else:
1386+ if patch_desc is None:
1387+ patch_desc = (
1388+ "%s\n\nNo DEP3 Subject or Description header found" %
1389+ patch_name
1390+ )
1391+ msg = b'%b\n\nGbp-Pq: %b.' % (
1392+ patch_desc.encode(),
1393+ patch_name.encode()
1394+ )
1395+ unapplied_parent_commit = repo.commit_tree_hash(
1396+ applied_import_tree_hash,
1397+ [unapplied_parent_commit] + applied_changelog_parent_commits,
1398+ msg,
1399+ fallback_author=fallback_author,
1400+ fallback_date=fallback_date,
1401+ )
1402+ except CalledProcessError as e:
1403+ if allow_applied_failures:
1404+ return
1405+ raise
1406
1407- if tag:
1408- logging.debug("Creating tag %s pointing to %s", tag, commit_hash)
1409- repo.create_tag(
1410- commit_hash=commit_hash,
1411- tag_name=tag,
1412- tag_msg=get_import_tag_msg(),
1413- )
1414+ target_tag = create_applied_tag(repo, unapplied_parent_commit, version, namespace)
1415
1416- repo.update_head_to_commit(
1417- head_name,
1418- commit_hash,
1419- )
1420+ return str(target_tag.peel(pygit2.Commit).id)
1421
1422 def import_applied_spi(
1423 repo,
1424@@ -1894,11 +1655,7 @@ def import_applied_spi(
1425
1426 :rtype None
1427 """
1428- pretty_head_name = spi.pretty_head_name
1429-
1430- logging.info('Importing patches-applied %s to %s' % (spi.version, pretty_head_name))
1431-
1432- spi.pull()
1433+ logging.info('Importing patches-applied %s', spi.version)
1434
1435 repo.clean_repository_state()
1436
1437@@ -1909,13 +1666,12 @@ def import_applied_spi(
1438 if spi.date_published:
1439 fallback_date = str(spi.date_published)
1440
1441- import_applied_dsc(
1442+ return import_applied_dsc(
1443 repo=repo,
1444 version=str(spi.version),
1445 namespace=namespace,
1446 dist=spi.distribution.name.lower(),
1447 dsc_pathname=spi.dsc_pathname,
1448- head_name=spi.applied_head_name(namespace),
1449 allow_applied_failures=allow_applied_failures,
1450 parent_overrides=parent_overrides,
1451 fallback_author=fallback_author,
1452@@ -1923,15 +1679,14 @@ def import_applied_spi(
1453 )
1454
1455
1456-def import_publishes(
1457+def import_new_publishes(
1458 repo,
1459 pkgname,
1460+ distname,
1461+ new_spis,
1462 namespace,
1463+ base_namespace,
1464 patches_applied,
1465- debian_head_versions,
1466- ubuntu_head_versions,
1467- debian_sinfo,
1468- ubuntu_sinfo,
1469 active_series_only,
1470 workdir,
1471 skip_orig,
1472@@ -1940,62 +1695,240 @@ def import_publishes(
1473 ):
1474 history_found = False
1475 only_debian = False
1476- srcpkg_information = None
1477- if patches_applied:
1478- _namespace = namespace
1479- namespace = '%s/applied' % namespace
1480- import_type = 'patches-applied'
1481- import_func = functools.partial(
1482- import_applied_spi,
1483- allow_applied_failures=allow_applied_failures,
1484- )
1485+ spi = None
1486+
1487+ updated_head_refs = dict()
1488+ try:
1489+ for spi in new_spis:
1490+ history_found = True
1491+ spi.pull()
1492+
1493+ if patches_applied:
1494+ existing_tags = get_existing_applied_tags(
1495+ repo,
1496+ spi.version,
1497+ namespace,
1498+ )
1499+ head_ref_name = gitubuntu.importerutils.applied_head_ref_name(
1500+ spi,
1501+ namespace,
1502+ )
1503+ else:
1504+ existing_tags = get_existing_import_tags(
1505+ repo,
1506+ spi.version,
1507+ namespace,
1508+ )
1509+ head_ref_name = gitubuntu.importerutils.head_ref_name(
1510+ spi,
1511+ namespace,
1512+ )
1513+
1514+ commit_hash = None
1515+ # if an identical DSC has already been imported,
1516+ # then we only need to update our branch tracker, based
1517+ # upon the import/reimport tag that matches
1518+ # dsc_to_to_tree_hash
1519+ if dsc_imported(repo, spi, namespace, distname):
1520+ existing_dsc_tree_hash = dsc_to_tree_hash(
1521+ repo.raw_repo,
1522+ spi.dsc_pathname,
1523+ patches_applied=patches_applied,
1524+ )
1525+ for existing_tag in existing_tags:
1526+ if str(
1527+ existing_tag.peel(pygit2.Tree).id
1528+ ) == existing_dsc_tree_hash:
1529+ commit_hash = str(
1530+ existing_tag.peel(pygit2.Commit).id
1531+ )
1532+ break
1533+
1534+ if not commit_hash:
1535+ try:
1536+ if patches_applied:
1537+ commit_hash = import_applied_spi(
1538+ repo=repo,
1539+ spi=spi,
1540+ namespace=namespace,
1541+ allow_applied_failures=allow_applied_failures,
1542+ parent_overrides=parent_overrides,
1543+ )
1544+ else:
1545+ commit_hash = import_unapplied_spi(
1546+ repo=repo,
1547+ spi=spi,
1548+ namespace=namespace,
1549+ skip_orig=skip_orig,
1550+ parent_overrides=parent_overrides,
1551+ )
1552+ except CalledProcessError:
1553+ if patches_applied and allow_applied_failures:
1554+ continue
1555+ raise
1556+
1557+ updated_head_refs[head_ref_name] = commit_hash
1558+ except Exception as e:
1559+ if patches_applied:
1560+ if not spi:
1561+ msg = "Unable to import patches-applied to %s" % distname
1562+ else:
1563+ msg = "Unable to import patches-applied %s to %s" % (
1564+ str(spi.version),
1565+ distname,
1566+ )
1567+ logging.error(msg)
1568+ logging.error(traceback.format_exc())
1569+ else:
1570+ if not spi:
1571+ msg = "Unable to import patches-unapplied to %s" % distname
1572+ else:
1573+ msg = "Unable to import patches-unapplied %s to %s" % (
1574+ str(spi.version),
1575+ distname,
1576+ )
1577+ raise GitUbuntuImportError(msg) from e
1578 else:
1579- _namespace = namespace
1580- import_type = 'patches-unapplied'
1581- import_func = functools.partial(
1582- import_unapplied_spi,
1583- skip_orig=skip_orig,
1584+ history_found = True
1585+
1586+ return history_found, updated_head_refs
1587+
1588+
1589+def import_publishes(
1590+ repo,
1591+ pkgname,
1592+ namespace,
1593+ active_series_only,
1594+ workdir,
1595+ skip_orig,
1596+ skip_applied,
1597+ allow_applied_failures,
1598+ parent_overrides,
1599+ pullfile,
1600+ retries,
1601+ retry_backoffs,
1602+):
1603+ only_debian = False
1604+ history_found = False
1605+
1606+ debian_sinfo = GitUbuntuSourceInformation(
1607+ 'debian',
1608+ pkgname,
1609+ os.path.abspath(pullfile),
1610+ retries,
1611+ retry_backoffs,
1612+ )
1613+
1614+ ubuntu_sinfo = GitUbuntuSourceInformation(
1615+ 'ubuntu',
1616+ pkgname,
1617+ os.path.abspath(pullfile),
1618+ retries,
1619+ retry_backoffs,
1620+ )
1621+
1622+ for distname, dist_sinfo in (
1623+ ('debian', debian_sinfo),
1624+ ('ubuntu', ubuntu_sinfo),
1625+ ):
1626+ unapplied_head_namespace = '%s/%s' % (
1627+ namespace,
1628+ distname,
1629 )
1630- for distname, versions, dist_sinfo in (
1631- ("debian", debian_head_versions, debian_sinfo),
1632- ("ubuntu", ubuntu_head_versions, ubuntu_sinfo)):
1633- if active_series_only and distname == "debian":
1634+ unapplied_versions = repo.get_heads_and_versions(
1635+ head_ref_namespace=unapplied_head_namespace,
1636+ )
1637+ gitubuntu.importerutils.debug_log_head_versions(unapplied_versions)
1638+
1639+ try:
1640+ history_found, updated_head_refs = import_new_publishes(
1641+ repo=repo,
1642+ pkgname=pkgname,
1643+ distname=distname,
1644+ new_spis=dist_sinfo.launchpad_versions_published_after(
1645+ unapplied_versions,
1646+ unapplied_head_namespace,
1647+ workdir,
1648+ active_series_only,
1649+ ),
1650+ namespace=namespace,
1651+ base_namespace=unapplied_head_namespace,
1652+ patches_applied=False,
1653+ active_series_only=active_series_only,
1654+ workdir=workdir,
1655+ skip_orig=skip_orig,
1656+ allow_applied_failures=None,
1657+ parent_overrides=parent_overrides,
1658+ )
1659+ update_branches(
1660+ distname,
1661+ dist_sinfo,
1662+ repo,
1663+ unapplied_head_namespace,
1664+ pkgname,
1665+ updated_head_refs,
1666+ )
1667+ except NoPublicationHistoryException:
1668+ logging.warning(
1669+ "No publication history found for %s in %s.",
1670+ pkgname,
1671+ distname,
1672+ )
1673+ if distname == 'ubuntu':
1674+ only_debian = True
1675 continue
1676+
1677+ if skip_applied:
1678+ continue
1679+
1680+ applied_head_namespace = '%s/applied/%s' % (
1681+ namespace,
1682+ distname,
1683+ )
1684+ applied_versions = repo.get_heads_and_versions(
1685+ head_namespace=applied_head_namespace,
1686+ )
1687+ gitubuntu.importerutils.debug_log_head_versions(applied_versions)
1688+
1689 try:
1690- for srcpkg_information in dist_sinfo.launchpad_versions_published_after(
1691- versions,
1692- namespace,
1693+ _, updated_head_refs = import_new_publishes(
1694+ repo=repo,
1695+ pkgname=pkgname,
1696+ distname=distname,
1697+ new_spis=dist_sinfo.launchpad_versions_published_after(
1698+ applied_versions,
1699+ applied_head_namespace,
1700+ workdir,
1701+ active_series_only,
1702+ ),
1703+ namespace=namespace,
1704+ base_namespace=applied_head_namespace,
1705+ patches_applied=True,
1706+ active_series_only=active_series_only,
1707 workdir=workdir,
1708- active_series_only=active_series_only
1709- ):
1710- history_found = True
1711- import_func(
1712- repo=repo,
1713- spi=srcpkg_information,
1714- namespace=_namespace,
1715- parent_overrides=parent_overrides,
1716- )
1717+ skip_orig=None,
1718+ allow_applied_failures=allow_applied_failures,
1719+ parent_overrides=parent_overrides,
1720+ )
1721+ update_branches(
1722+ distname,
1723+ dist_sinfo,
1724+ repo,
1725+ applied_head_namespace,
1726+ pkgname,
1727+ updated_head_refs,
1728+ )
1729 except NoPublicationHistoryException:
1730- logging.warning("No publication history found for %s in %s.",
1731- pkgname, distname
1732+ logging.warning(
1733+ "No publication history found for %s in %s.",
1734+ pkgname,
1735+ distname,
1736 )
1737 if distname == 'ubuntu':
1738 only_debian = True
1739- except Exception as e:
1740- if srcpkg_information is None:
1741- msg = 'Unable to import %s to %s' % (import_type, distname)
1742- else:
1743- msg = 'Unable to import %s %s to %s' % (import_type,
1744- str(srcpkg_information.version), distname)
1745- if not patches_applied:
1746- raise GitUbuntuImportError(msg) from e
1747- else:
1748- logging.error(msg)
1749- logging.error(traceback.format_exc())
1750- else:
1751- history_found = True
1752+ continue
1753
1754- return (only_debian, history_found)
1755+ return only_debian, history_found
1756
1757
1758 def parse_args(subparsers=None, base_subparsers=None):
1759diff --git a/gitubuntu/importer_test.py b/gitubuntu/importer_test.py
1760index f9537c4..8039911 100644
1761--- a/gitubuntu/importer_test.py
1762+++ b/gitubuntu/importer_test.py
1763@@ -57,9 +57,9 @@ from gitubuntu.test_fixtures import repo
1764 ])
1765 def test_devel_branch_updates(head_versions, expected):
1766 result = target._devel_branch_updates(
1767- 'prefix/',
1768- head_versions,
1769- ['yakkety', 'xenial', 'trusty'],
1770+ ref_prefix='prefix/ubuntu',
1771+ head_versions=head_versions,
1772+ series_name_list=['yakkety', 'xenial', 'trusty'],
1773 )
1774 assert set(result) == set(expected.items())
1775
1776@@ -70,50 +70,38 @@ def test_get_import_tag_msg():
1777
1778
1779 @pytest.mark.parametrize(
1780- 'head_name_value, changelog_parent_commit, upload_parent_commit, unapplied_parent_commit, expected',
1781+ 'changelog_parent_commits, unapplied_parent_commit, expected',
1782 [
1783 (
1784- 'importer/ubuntu/trusty',
1785- None,
1786- None,
1787+ [],
1788 None,
1789- b'Import patches-unapplied version 1-1 to ubuntu/trusty\n\nImported using git-ubuntu import.',
1790+ b'Import patches-unapplied version 1-1\n\nImported using git-ubuntu import.',
1791 ),
1792 (
1793- 'importer/ubuntu/trusty',
1794- '123456',
1795+ ['123456'],
1796 None,
1797- None,
1798- b'Import patches-unapplied version 1-1 to ubuntu/trusty\n\nImported using git-ubuntu import.\n\nChangelog parent: 123456',
1799+ b'Import patches-unapplied version 1-1\n\nImported using git-ubuntu import.\n\nChangelog parent: 123456',
1800 ),
1801 (
1802- 'importer/ubuntu/trusty',
1803- None,
1804+ [],
1805 '123456',
1806- None,
1807- b'Import patches-unapplied version 1-1 to ubuntu/trusty\n\nImported using git-ubuntu import.\n\nUpload parent: 123456',
1808+ b'Import patches-applied version 1-1\n\nImported using git-ubuntu import.\n\nUnapplied parent: 123456',
1809 ),
1810 (
1811- 'importer/ubuntu/trusty',
1812- None,
1813- None,
1814- '123456',
1815- b'Import patches-applied version 1-1 to ubuntu/trusty\n\nImported using git-ubuntu import.\n\nUnapplied parent: 123456',
1816+ ['123456'],
1817+ '789123',
1818+ b'Import patches-applied version 1-1\n\nImported using git-ubuntu import.\n\nChangelog parent: 123456\nUnapplied parent: 789123'
1819 ),
1820 (
1821- 'importer/ubuntu/trusty',
1822- '123456',
1823- '789123',
1824+ ['123456', '789123'],
1825 '456789',
1826- b'Import patches-applied version 1-1 to ubuntu/trusty\n\nImported using git-ubuntu import.\n\nChangelog parent: 123456\nUpload parent: 789123\nUnapplied parent: 456789'
1827+ b'Import patches-applied version 1-1\n\nImported using git-ubuntu import.\n\nChangelog parent: 123456\nChangelog parent: 789123\nUnapplied parent: 456789'
1828 ),
1829 ],
1830 )
1831 def test_get_import_commit_msg(
1832 repo,
1833- head_name_value,
1834- changelog_parent_commit,
1835- upload_parent_commit,
1836+ changelog_parent_commits,
1837 unapplied_parent_commit,
1838 expected,
1839 ):
1840@@ -134,11 +122,9 @@ def test_get_import_commit_msg(
1841 assert target.get_import_commit_msg(
1842 repo,
1843 publish_spec.version,
1844- head_name_value,
1845 'importer',
1846 publish_tree_hash,
1847- changelog_parent_commit,
1848- upload_parent_commit,
1849+ changelog_parent_commits,
1850 unapplied_parent_commit,
1851 ) == expected
1852
1853@@ -489,7 +475,7 @@ def test_create_applied_tag(repo, input_data, expected_changes, test_refs):
1854 )
1855
1856
1857-@pytest.mark.parametrize('input_data, parent_overrides, changelog_versions, patches_applied, expected_ref',
1858+@pytest.mark.parametrize('input_data, parent_overrides, changelog_versions, patches_applied, expected_refs',
1859 [
1860 (
1861 repo_builder.Repo(
1862@@ -500,7 +486,33 @@ def test_create_applied_tag(repo, input_data, expected_changes, test_refs):
1863 {},
1864 ['1-2', '1-1',],
1865 False,
1866- None,
1867+ [],
1868+ ),
1869+ (
1870+ repo_builder.Repo(
1871+ commit_list=[
1872+ repo_builder.Commit(
1873+ tree=repo_builder.SourceTree(
1874+ source_builder.Source(
1875+ source_builder.SourceSpec(
1876+ version='1-1',
1877+ native=False,
1878+ )
1879+ )
1880+ ),
1881+ name='import',
1882+ ),
1883+ ],
1884+ branches={
1885+ },
1886+ tags={
1887+ 'importer/import/1-1': repo_builder.Placeholder('import'),
1888+ },
1889+ ),
1890+ {},
1891+ ['1-2', '1-1'],
1892+ False,
1893+ ['refs/tags/importer/import/1-1'],
1894 ),
1895 (
1896 repo_builder.Repo(
1897@@ -516,17 +528,36 @@ def test_create_applied_tag(repo, input_data, expected_changes, test_refs):
1898 ),
1899 name='import',
1900 ),
1901+ repo_builder.Commit(
1902+ tree=repo_builder.SourceTree(
1903+ source_builder.Source(
1904+ source_builder.SourceSpec(
1905+ version='1-1',
1906+ native=False,
1907+ file_contents={
1908+ 'debian/random': 'reimport tag',
1909+ },
1910+ )
1911+ )
1912+ ),
1913+ name='reimport',
1914+ ),
1915 ],
1916 branches={
1917 },
1918 tags={
1919 'importer/import/1-1': repo_builder.Placeholder('import'),
1920+ 'importer/reimport/import/1-1/0': repo_builder.Placeholder('import'),
1921+ 'importer/reimport/import/1-1/1': repo_builder.Placeholder('reimport'),
1922 },
1923 ),
1924 {},
1925 ['1-2', '1-1'],
1926 False,
1927- 'refs/tags/importer/import/1-1',
1928+ [
1929+ 'refs/tags/importer/reimport/import/1-1/0',
1930+ 'refs/tags/importer/reimport/import/1-1/1',
1931+ ],
1932 ),
1933 (
1934 repo_builder.Repo(
1935@@ -552,7 +583,7 @@ def test_create_applied_tag(repo, input_data, expected_changes, test_refs):
1936 {},
1937 ['1-3', '1-2', '1-1'],
1938 False,
1939- 'refs/tags/importer/import/1-1',
1940+ ['refs/tags/importer/import/1-1'],
1941 ),
1942 (
1943 repo_builder.Repo(
1944@@ -563,7 +594,33 @@ def test_create_applied_tag(repo, input_data, expected_changes, test_refs):
1945 {},
1946 ['1-2', '1-1',],
1947 True,
1948- None,
1949+ [],
1950+ ),
1951+ (
1952+ repo_builder.Repo(
1953+ commit_list=[
1954+ repo_builder.Commit(
1955+ tree=repo_builder.SourceTree(
1956+ source_builder.Source(
1957+ source_builder.SourceSpec(
1958+ version='1-1',
1959+ native=False,
1960+ )
1961+ )
1962+ ),
1963+ name='applied',
1964+ ),
1965+ ],
1966+ branches={
1967+ },
1968+ tags={
1969+ 'importer/applied/1-1': repo_builder.Placeholder('applied'),
1970+ },
1971+ ),
1972+ {},
1973+ ['1-2', '1-1'],
1974+ True,
1975+ ['refs/tags/importer/applied/1-1'],
1976 ),
1977 (
1978 repo_builder.Repo(
1979@@ -579,17 +636,36 @@ def test_create_applied_tag(repo, input_data, expected_changes, test_refs):
1980 ),
1981 name='applied',
1982 ),
1983+ repo_builder.Commit(
1984+ tree=repo_builder.SourceTree(
1985+ source_builder.Source(
1986+ source_builder.SourceSpec(
1987+ version='1-1',
1988+ native=False,
1989+ file_contents={
1990+ 'debian/random': 'reimport tag',
1991+ },
1992+ )
1993+ )
1994+ ),
1995+ name='reimport',
1996+ ),
1997 ],
1998 branches={
1999 },
2000 tags={
2001 'importer/applied/1-1': repo_builder.Placeholder('applied'),
2002+ 'importer/reimport/applied/1-1/0': repo_builder.Placeholder('applied'),
2003+ 'importer/reimport/applied/1-1/1': repo_builder.Placeholder('reimport'),
2004 },
2005 ),
2006 {},
2007 ['1-2', '1-1'],
2008 True,
2009- 'refs/tags/importer/applied/1-1',
2010+ [
2011+ 'refs/tags/importer/reimport/applied/1-1/0',
2012+ 'refs/tags/importer/reimport/applied/1-1/1',
2013+ ],
2014 ),
2015 (
2016 repo_builder.Repo(
2017@@ -615,28 +691,30 @@ def test_create_applied_tag(repo, input_data, expected_changes, test_refs):
2018 {},
2019 ['1-3', '1-2', '1-1'],
2020 True,
2021- 'refs/tags/importer/applied/1-1',
2022+ ['refs/tags/importer/applied/1-1'],
2023 ),
2024 ],
2025 )
2026-def test_get_changelog_parent_commit(
2027+def test_get_changelog_parent_commits(
2028 repo,
2029 input_data,
2030 parent_overrides,
2031 changelog_versions,
2032 patches_applied,
2033- expected_ref,
2034+ expected_refs,
2035 ):
2036 input_data.write(repo.raw_repo)
2037- assert target.get_changelog_parent_commit(
2038+ assert target.get_changelog_parent_commits(
2039 repo,
2040 'importer',
2041 parent_overrides,
2042 changelog_versions,
2043 patches_applied,
2044 ) == (
2045- str(repo.raw_repo.lookup_reference(expected_ref).peel(pygit2.Commit).id)
2046- if expected_ref else expected_ref
2047+ [
2048+ str(repo.raw_repo.lookup_reference(expected_ref).peel(pygit2.Commit).id)
2049+ for expected_ref in expected_refs
2050+ ] if expected_refs else expected_refs
2051 )
2052
2053 @patch('gitubuntu.git_repository.GitUbuntuRepository.add_base_remotes')
2054@@ -697,3 +775,1685 @@ def test_importer_close_repository_on_exception(main_with_repo_mock):
2055 with pytest.raises(MockError):
2056 target.main(pkgname='dummy', owner='dummy', repo=repo)
2057 repo.close.assert_called()
2058+
2059+
2060+@pytest.mark.parametrize(
2061+ 'name, input_data, expected_changes, compare_refs, exist_refs, xfail', [
2062+ (
2063+ 'Case 1',
2064+ repo_builder.Repo(
2065+ commit_list=[
2066+ repo_builder.Commit(
2067+ tree=repo_builder.SourceTree(
2068+ source_builder.Source(
2069+ source_builder.SourceSpec(
2070+ version='1-1',
2071+ native=False,
2072+ )
2073+ )
2074+ ),
2075+ name='import'
2076+ ),
2077+ ],
2078+ branches={
2079+ 'importer/ubuntu/trusty-proposed': repo_builder.Placeholder('import'),
2080+ },
2081+ tags={
2082+ 'importer/import/1-1': repo_builder.Placeholder('import'),
2083+ },
2084+ ),
2085+ {
2086+ 'commit_list_to_add': [],
2087+ 'tags_to_add': {},
2088+ 'branches_to_update': {
2089+ 'importer/ubuntu/trusty': repo_builder.Placeholder('import'),
2090+ },
2091+ },
2092+ [
2093+ #'refs/heads/importer/ubuntu/trusty-proposed',
2094+ #'refs/heads/importer/ubuntu/trusty',
2095+ 'refs/tags/importer/import/1-1',
2096+ ],
2097+ [
2098+ 'refs/heads/do-not-push',
2099+ 'refs/tags/importer/upstream/ubuntu/1.gz',
2100+ 'refs/heads/importer/importer/ubuntu/dsc',
2101+ 'refs/heads/importer/importer/ubuntu/pristine-tar',
2102+ ],
2103+ False,
2104+ ),
2105+ (
2106+ 'Case 1 - reimport tag',
2107+ repo_builder.Repo(
2108+ commit_list=[
2109+ repo_builder.Commit(
2110+ tree=repo_builder.SourceTree(
2111+ source_builder.Source(
2112+ source_builder.SourceSpec(
2113+ version='1-1',
2114+ native=False,
2115+ file_contents={
2116+ 'debian/random': 'import tag contents',
2117+ },
2118+ )
2119+ )
2120+ ),
2121+ name='import',
2122+ ),
2123+ repo_builder.Commit(
2124+ tree=repo_builder.SourceTree(
2125+ source_builder.Source(
2126+ source_builder.SourceSpec(
2127+ version='1-1',
2128+ native=False,
2129+ )
2130+ )
2131+ ),
2132+ name='reimport',
2133+ ),
2134+ ],
2135+ branches={
2136+ 'importer/ubuntu/trusty-proposed': repo_builder.Placeholder('import'),
2137+ },
2138+ tags={
2139+ 'importer/import/1-1': repo_builder.Placeholder('import'),
2140+ 'importer/reimport/import/1-1/0': repo_builder.Placeholder('import'),
2141+ 'importer/reimport/import/1-1/1': repo_builder.Placeholder('reimport'),
2142+ },
2143+ ),
2144+ {
2145+ 'commit_list_to_add': [],
2146+ 'tags_to_add': {},
2147+ 'branches_to_update': {
2148+ 'importer/ubuntu/trusty': repo_builder.Placeholder('reimport'),
2149+ },
2150+ },
2151+ [
2152+ #'refs/heads/importer/ubuntu/trusty-proposed',
2153+ #'refs/heads/importer/ubuntu/trusty',
2154+ 'refs/tags/importer/import/1-1',
2155+ 'refs/tags/importer/reimport/import/1-1/0',
2156+ 'refs/tags/importer/reimport/import/1-1/1',
2157+ ],
2158+ [
2159+ 'refs/heads/do-not-push',
2160+ 'refs/tags/importer/upstream/ubuntu/1.gz',
2161+ 'refs/heads/importer/importer/ubuntu/dsc',
2162+ 'refs/heads/importer/importer/ubuntu/pristine-tar',
2163+ ],
2164+ False,
2165+ ),
2166+ (
2167+ 'Case 2',
2168+ repo_builder.Repo(
2169+ commit_list=[
2170+ repo_builder.Commit(
2171+ tree=repo_builder.SourceTree(
2172+ source_builder.Source(
2173+ source_builder.SourceSpec(
2174+ version='1-1',
2175+ native=False,
2176+ file_contents={
2177+ 'debian/random': 'The import tag contents',
2178+ },
2179+ )
2180+ )
2181+ ),
2182+ name='import'
2183+ ),
2184+ repo_builder.Commit(
2185+ tree=repo_builder.SourceTree(
2186+ source_builder.Source(
2187+ source_builder.SourceSpec(
2188+ version='1-1',
2189+ native=False,
2190+ )
2191+ )
2192+ ),
2193+ name='upload'
2194+ ),
2195+ ],
2196+ branches={
2197+ 'importer/ubuntu/trusty-proposed': repo_builder.Placeholder('import'),
2198+ },
2199+ tags={
2200+ 'importer/import/1-1': repo_builder.Placeholder('import'),
2201+ 'importer/upload/1-1': repo_builder.Placeholder('upload'),
2202+ },
2203+ ),
2204+ {
2205+ 'commit_list_to_add': [
2206+ ],
2207+ 'tags_to_add': {
2208+ 'importer/reimport/import/1-1/0': repo_builder.Placeholder('import'),
2209+ 'importer/reimport/import/1-1/1': repo_builder.Placeholder('upload'),
2210+ },
2211+ 'branches_to_update': {
2212+ 'importer/ubuntu/trusty': repo_builder.Placeholder('upload'),
2213+ },
2214+ },
2215+ [
2216+ #'refs/heads/importer/ubuntu/trusty-proposed',
2217+ #'refs/heads/importer/ubuntu/trusty',
2218+ 'refs/tags/importer/import/1-1',
2219+ 'refs/tags/importer/upload/1-1',
2220+ 'refs/tags/importer/reimport/import/1-1/0',
2221+ 'refs/tags/importer/reimport/import/1-1/1',
2222+ ],
2223+ [
2224+ 'refs/heads/do-not-push',
2225+ 'refs/tags/importer/upstream/ubuntu/1.gz',
2226+ 'refs/heads/importer/importer/ubuntu/dsc',
2227+ 'refs/heads/importer/importer/ubuntu/pristine-tar',
2228+ ],
2229+ False,
2230+ ),
2231+ (
2232+ 'Case 3',
2233+ repo_builder.Repo(
2234+ commit_list=[
2235+ repo_builder.Commit(
2236+ tree=repo_builder.SourceTree(
2237+ source_builder.Source(
2238+ source_builder.SourceSpec(
2239+ version='1-1',
2240+ native=False,
2241+ file_contents={
2242+ 'debian/random': 'The import tag contents',
2243+ },
2244+ )
2245+ )
2246+ ),
2247+ name='import'
2248+ ),
2249+ repo_builder.Commit(
2250+ tree=repo_builder.SourceTree(
2251+ source_builder.Source(
2252+ source_builder.SourceSpec(
2253+ version='1-1',
2254+ native=False,
2255+ file_contents={
2256+ 'debian/random': 'The upload tag contents',
2257+ },
2258+ )
2259+ )
2260+ ),
2261+ name='upload'
2262+ ),
2263+ ],
2264+ branches={
2265+ 'importer/ubuntu/trusty-proposed': repo_builder.Placeholder('import'),
2266+ },
2267+ tags={
2268+ 'importer/import/1-1': repo_builder.Placeholder('import'),
2269+ 'importer/upload/1-1': repo_builder.Placeholder('upload'),
2270+ },
2271+ ),
2272+ {
2273+ 'commit_list_to_add': [
2274+ repo_builder.Commit(
2275+ tree=repo_builder.SourceTree(
2276+ source_builder.Source(
2277+ source_builder.SourceSpec(
2278+ version='1-1',
2279+ native=False,
2280+ )
2281+ )
2282+ ),
2283+ name='reimport'
2284+ ),
2285+ ],
2286+ 'tags_to_add': {
2287+ 'importer/reimport/import/1-1/0': repo_builder.Placeholder('import'),
2288+ 'importer/reimport/import/1-1/1': repo_builder.Placeholder('reimport'),
2289+ },
2290+ 'branches_to_update': {
2291+ 'importer/ubuntu/trusty': repo_builder.Placeholder('reimport'),
2292+ },
2293+ },
2294+ [
2295+ #'refs/heads/importer/ubuntu/trusty-proposed',
2296+ #'refs/heads/importer/ubuntu/trusty',
2297+ 'refs/tags/importer/import/1-1',
2298+ 'refs/tags/importer/upload/1-1',
2299+ 'refs/tags/importer/reimport/import/1-1/0',
2300+ 'refs/tags/importer/reimport/import/1-1/1',
2301+ ],
2302+ [
2303+ 'refs/heads/do-not-push',
2304+ 'refs/tags/importer/upstream/ubuntu/1.gz',
2305+ 'refs/heads/importer/importer/ubuntu/dsc',
2306+ 'refs/heads/importer/importer/ubuntu/pristine-tar',
2307+ ],
2308+ False,
2309+ ),
2310+ (
2311+ 'Case 4',
2312+ repo_builder.Repo(
2313+ commit_list=[
2314+ repo_builder.Commit(
2315+ tree=repo_builder.SourceTree(
2316+ source_builder.Source(
2317+ source_builder.SourceSpec(
2318+ version='1-1',
2319+ native=False,
2320+ file_contents={
2321+ 'debian/random': 'The import tag contents',
2322+ },
2323+ )
2324+ )
2325+ ),
2326+ name='import'
2327+ ),
2328+ ],
2329+ branches={
2330+ 'importer/ubuntu/trusty-proposed': repo_builder.Placeholder('import'),
2331+ },
2332+ tags={
2333+ 'importer/import/1-1': repo_builder.Placeholder('import'),
2334+ },
2335+ ),
2336+ {
2337+ 'commit_list_to_add': [
2338+ repo_builder.Commit(
2339+ tree=repo_builder.SourceTree(
2340+ source_builder.Source(
2341+ source_builder.SourceSpec(
2342+ version='1-1',
2343+ native=False,
2344+ )
2345+ )
2346+ ),
2347+ name='reimport'
2348+ ),
2349+ ],
2350+ 'tags_to_add': {
2351+ 'importer/reimport/import/1-1/0': repo_builder.Placeholder('import'),
2352+ 'importer/reimport/import/1-1/1': repo_builder.Placeholder('reimport'),
2353+ },
2354+ 'branches_to_update': {
2355+ 'importer/ubuntu/trusty': repo_builder.Placeholder('reimport'),
2356+ },
2357+ },
2358+ [
2359+ #'refs/heads/importer/ubuntu/trusty-proposed',
2360+ #'refs/heads/importer/ubuntu/trusty',
2361+ 'refs/tags/importer/import/1-1',
2362+ 'refs/tags/importer/reimport/import/1-1/0',
2363+ 'refs/tags/importer/reimport/import/1-1/1',
2364+ ],
2365+ [
2366+ 'refs/heads/do-not-push',
2367+ 'refs/tags/importer/upstream/ubuntu/1.gz',
2368+ 'refs/heads/importer/importer/ubuntu/dsc',
2369+ 'refs/heads/importer/importer/ubuntu/pristine-tar',
2370+ ],
2371+ False,
2372+ ),
2373+ (
2374+ 'Case 5',
2375+ repo_builder.Repo(
2376+ commit_list=[
2377+ repo_builder.Commit(
2378+ tree=repo_builder.SourceTree(
2379+ source_builder.Source(
2380+ source_builder.SourceSpec(
2381+ version='1-1',
2382+ native=False,
2383+ )
2384+ )
2385+ ),
2386+ name='upload'
2387+ ),
2388+ ],
2389+ branches={
2390+ },
2391+ tags={
2392+ 'importer/upload/1-1': repo_builder.Placeholder('upload'),
2393+ },
2394+ ),
2395+ {
2396+ 'commit_list_to_add': [
2397+ ],
2398+ 'tags_to_add': {
2399+ 'importer/import/1-1': repo_builder.Placeholder('upload'),
2400+ },
2401+ 'branches_to_update': {
2402+ 'importer/ubuntu/trusty': repo_builder.Placeholder('upload'),
2403+ },
2404+ },
2405+ [
2406+ #'refs/heads/importer/ubuntu/trusty',
2407+ 'refs/tags/importer/import/1-1',
2408+ 'refs/tags/importer/upload/1-1',
2409+ ],
2410+ [
2411+ 'refs/heads/do-not-push',
2412+ 'refs/tags/importer/upstream/ubuntu/1.gz',
2413+ 'refs/heads/importer/importer/ubuntu/dsc',
2414+ 'refs/heads/importer/importer/ubuntu/pristine-tar',
2415+ ],
2416+ False,
2417+ ),
2418+ (
2419+ 'Case 6',
2420+ repo_builder.Repo(
2421+ commit_list=[
2422+ repo_builder.Commit(
2423+ tree=repo_builder.SourceTree(
2424+ source_builder.Source(
2425+ source_builder.SourceSpec(
2426+ version='1-1',
2427+ native=False,
2428+ file_contents={
2429+ 'debian/random': 'The upload tag contents',
2430+ },
2431+ )
2432+ )
2433+ ),
2434+ name='upload'
2435+ ),
2436+ ],
2437+ branches={
2438+ },
2439+ tags={
2440+ 'importer/upload/1-1': repo_builder.Placeholder('upload'),
2441+ },
2442+ ),
2443+ {
2444+ 'commit_list_to_add': [
2445+ repo_builder.Commit(
2446+ tree=repo_builder.SourceTree(
2447+ source_builder.Source(
2448+ source_builder.SourceSpec(
2449+ version='1-1',
2450+ native=False,
2451+ )
2452+ )
2453+ ),
2454+ name='publish'
2455+ ),
2456+ ],
2457+ 'tags_to_add': {
2458+ 'importer/import/1-1': repo_builder.Placeholder('publish'),
2459+ },
2460+ 'branches_to_update': {
2461+ 'importer/ubuntu/trusty': repo_builder.Placeholder('publish'),
2462+ },
2463+ },
2464+ [
2465+ #'refs/heads/importer/ubuntu/trusty',
2466+ 'refs/tags/importer/import/1-1',
2467+ 'refs/tags/importer/upload/1-1',
2468+ ],
2469+ [
2470+ 'refs/heads/do-not-push',
2471+ 'refs/tags/importer/upstream/ubuntu/1.gz',
2472+ 'refs/heads/importer/importer/ubuntu/dsc',
2473+ 'refs/heads/importer/importer/ubuntu/pristine-tar',
2474+ ],
2475+ False,
2476+ ),
2477+ (
2478+ 'Case 7',
2479+ repo_builder.Repo(
2480+ commit_list=[
2481+ ],
2482+ branches={
2483+ },
2484+ tags={
2485+ },
2486+ ),
2487+ {
2488+ 'commit_list_to_add': [
2489+ repo_builder.Commit(
2490+ tree=repo_builder.SourceTree(
2491+ source_builder.Source(
2492+ source_builder.SourceSpec(
2493+ version='1-1',
2494+ native=False,
2495+ )
2496+ )
2497+ ),
2498+ name='publish'
2499+ ),
2500+ ],
2501+ 'tags_to_add': {
2502+ 'importer/import/1-1': repo_builder.Placeholder('publish'),
2503+ },
2504+ 'branches_to_update': {
2505+ 'importer/ubuntu/trusty': repo_builder.Placeholder('publish'),
2506+ },
2507+ },
2508+ [
2509+ #'refs/heads/importer/ubuntu/trusty',
2510+ 'refs/tags/importer/import/1-1'
2511+ ],
2512+ [
2513+ 'refs/heads/do-not-push',
2514+ 'refs/tags/importer/upstream/ubuntu/1.gz',
2515+ 'refs/heads/importer/importer/ubuntu/dsc',
2516+ 'refs/heads/importer/importer/ubuntu/pristine-tar',
2517+ ],
2518+ False,
2519+ ),
2520+ ]
2521+)
2522+@patch('gitubuntu.importer.get_import_tag_msg')
2523+@patch('gitubuntu.importer.get_import_commit_msg')
2524+def test_import_unapplied_spi_tags(
2525+ get_import_commit_msg_mock,
2526+ get_import_tag_msg_mock,
2527+ repo,
2528+ name,
2529+ input_data,
2530+ expected_changes,
2531+ compare_refs,
2532+ exist_refs,
2533+ xfail,
2534+):
2535+ # Match the repo_builder objects
2536+ get_import_tag_msg_mock.return_value = 'Test tag'
2537+ get_import_commit_msg_mock.return_value = b'Test commit'
2538+
2539+ publish_spec = source_builder.SourceSpec(
2540+ version='1-1',
2541+ native=False,
2542+ )
2543+ publish_source = source_builder.Source(publish_spec)
2544+
2545+ # copy before write, so that placeholders are still present
2546+ expected_result = input_data.copy(**expected_changes)
2547+
2548+ # Ensure reproducible commit hashes
2549+ with patch.object(
2550+ repo,
2551+ 'get_commit_environment',
2552+ return_value={
2553+ 'GIT_AUTHOR_NAME':'Test Builder',
2554+ 'GIT_AUTHOR_EMAIL':'test@example.com',
2555+ 'GIT_AUTHOR_DATE':'1970-01-01T00:00:00Z',
2556+ 'GIT_COMMITTER_NAME':'Test Builder',
2557+ 'GIT_COMMITTER_EMAIL':'test@example.com',
2558+ 'GIT_COMMITTER_DATE':'1970-01-01T00:00:00Z',
2559+ },
2560+ ):
2561+ input_data.write(repo.raw_repo)
2562+
2563+ with publish_source as dsc_path:
2564+ spi = Mock()
2565+ spi.dsc_pathname = dsc_path
2566+ spi.distribution.name = 'Ubuntu'
2567+ spi.version = publish_spec.version
2568+ # avoid automatic parenting for head_name to make the assertion later
2569+ # easier
2570+ head_name = Mock(name='head_name')
2571+ head_name.return_value = 'importer/ubuntu/trusty'
2572+ spi.head_name = head_name
2573+ spi.date_published = '1970-01-01T00:00:00Z'
2574+ target.import_unapplied_spi(
2575+ repo=repo,
2576+ spi=spi,
2577+ namespace='importer',
2578+ skip_orig=False,
2579+ parent_overrides={},
2580+ )
2581+
2582+ # test that some refs are the same
2583+ assert repo_comparator.equals(
2584+ repoA=repo.raw_repo,
2585+ repoB=expected_result,
2586+ test_refs=compare_refs,
2587+ ) or xfail
2588+
2589+ # test that some refs exist
2590+ for ref in exist_refs:
2591+ assert repo.raw_repo.lookup_reference(ref) or xfail
2592+
2593+ # test that no other refs exist
2594+ #assert not (
2595+ # set(repo.raw_repo.listall_references()) ^
2596+ # (set(compare_refs) | set(exist_refs))
2597+ #) or xfail
2598+
2599+ if xfail:
2600+ pytest.xfail(xfail)
2601+
2602+
2603+@patch('gitubuntu.importer.get_import_tag_msg')
2604+@patch('gitubuntu.importer.get_import_commit_msg')
2605+def test_import_unapplied_spi_quilt_patches(
2606+ get_import_commit_msg_mock,
2607+ get_import_tag_msg_mock,
2608+ repo,
2609+):
2610+ # Match the repo_builder objects
2611+ get_import_tag_msg_mock.return_value = 'Test tag'
2612+ get_import_commit_msg_mock.return_value = b'Test commit'
2613+
2614+ input_data = repo_builder.Repo(
2615+ commit_list=[
2616+ ],
2617+ branches={
2618+ },
2619+ tags={
2620+ },
2621+ )
2622+ expected_changes = {
2623+ 'commit_list_to_add': [
2624+ repo_builder.Commit(
2625+ tree=repo_builder.SourceTree(
2626+ source_builder.Source(
2627+ source_builder.SourceSpec(
2628+ version='1-1',
2629+ native=False,
2630+ has_patches=True,
2631+ )
2632+ )
2633+ ),
2634+ name='publish'
2635+ ),
2636+ ],
2637+ 'tags_to_add': {
2638+ 'importer/import/1-1': repo_builder.Placeholder('publish'),
2639+ },
2640+ 'branches_to_update': {
2641+ 'importer/ubuntu/trusty': repo_builder.Placeholder('publish'),
2642+ },
2643+ }
2644+ test_refs = [
2645+ #'refs/heads/importer/ubuntu/trusty',
2646+ 'refs/tags/importer/import/1-1'
2647+ ]
2648+
2649+ publish_spec = source_builder.SourceSpec(
2650+ version='1-1',
2651+ native=False,
2652+ has_patches=True,
2653+ )
2654+ publish_source = source_builder.Source(publish_spec)
2655+
2656+ # copy before write, so that placeholders are still present
2657+ expected_result = input_data.copy(**expected_changes)
2658+
2659+ with patch.object(
2660+ repo,
2661+ 'get_commit_environment',
2662+ return_value={
2663+ 'GIT_AUTHOR_NAME':'Test Builder',
2664+ 'GIT_AUTHOR_EMAIL':'test@example.com',
2665+ 'GIT_AUTHOR_DATE':'1970-01-01T00:00:00Z',
2666+ 'GIT_COMMITTER_NAME':'Test Builder',
2667+ 'GIT_COMMITTER_EMAIL':'test@example.com',
2668+ 'GIT_COMMITTER_DATE':'1970-01-01T00:00:00Z',
2669+ },
2670+ ):
2671+ input_data.write(repo.raw_repo)
2672+
2673+ with publish_source as dsc_path:
2674+ spi = Mock()
2675+ spi.dsc_pathname = dsc_path
2676+ spi.distribution.name = 'Ubuntu'
2677+ spi.version = publish_spec.version
2678+ # avoid automatic parenting for head_name to make the assertion later
2679+ # easier
2680+ head_name = Mock(name='head_name')
2681+ head_name.return_value = 'importer/ubuntu/trusty'
2682+ spi.head_name = head_name
2683+ spi.date_published = '1970-01-01T00:00:00Z'
2684+ # import_unapplied_spi currently assumes it is called from the
2685+ # repository directory (pristine-tar and other commands rely on
2686+ # this)
2687+ target.import_unapplied_spi(
2688+ repo=repo,
2689+ spi=spi,
2690+ namespace='importer',
2691+ skip_orig=False,
2692+ parent_overrides={},
2693+ )
2694+
2695+ assert repo_comparator.equals(
2696+ repoA=repo.raw_repo,
2697+ repoB=expected_result,
2698+ test_refs=test_refs,
2699+ )
2700+
2701+
2702+@pytest.mark.parametrize(
2703+ 'input_data, expected_changes, changelog_versions, test_refs, xfail', [
2704+ (
2705+ repo_builder.Repo(
2706+ commit_list=[
2707+ repo_builder.Commit(
2708+ tree=repo_builder.SourceTree(
2709+ source_builder.Source(
2710+ source_builder.SourceSpec(
2711+ version='1-1',
2712+ native=False,
2713+ )
2714+ )
2715+ ),
2716+ name='import'
2717+ ),
2718+ ],
2719+ branches={
2720+ },
2721+ tags={
2722+ 'importer/import/1-1': repo_builder.Placeholder('import'),
2723+ },
2724+ ),
2725+ {
2726+ 'commit_list_to_add': [
2727+ repo_builder.Commit(
2728+ tree=repo_builder.SourceTree(
2729+ source_builder.Source(
2730+ source_builder.SourceSpec(
2731+ changelog_versions=['1-2', '1-1'],
2732+ native=False,
2733+ )
2734+ )
2735+ ),
2736+ parents=[repo_builder.Placeholder('import')],
2737+ name='publish'
2738+ ),
2739+ ],
2740+ 'tags_to_add': {
2741+ 'importer/import/1-2': repo_builder.Placeholder('publish'),
2742+ },
2743+ 'branches_to_update': {
2744+ },
2745+ },
2746+ ['1-2', '1-1'],
2747+ [
2748+ 'refs/tags/importer/import/1-1',
2749+ 'refs/tags/importer/import/1-2',
2750+ ],
2751+ False,
2752+ ),
2753+ (
2754+ repo_builder.Repo(
2755+ commit_list=[
2756+ repo_builder.Commit(
2757+ tree=repo_builder.SourceTree(
2758+ source_builder.Source(
2759+ source_builder.SourceSpec(
2760+ version='1-1',
2761+ native=False,
2762+ )
2763+ )
2764+ ),
2765+ name='import'
2766+ ),
2767+ ],
2768+ branches={
2769+ },
2770+ tags={
2771+ 'importer/import/1-1': repo_builder.Placeholder('import'),
2772+ },
2773+ ),
2774+ {
2775+ 'commit_list_to_add': [
2776+ repo_builder.Commit(
2777+ tree=repo_builder.SourceTree(
2778+ source_builder.Source(
2779+ source_builder.SourceSpec(
2780+ changelog_versions=['1-3', '1-2', '1-1'],
2781+ native=False,
2782+ )
2783+ )
2784+ ),
2785+ parents=[repo_builder.Placeholder('import')],
2786+ name='publish'
2787+ ),
2788+ ],
2789+ 'tags_to_add': {
2790+ 'importer/import/1-3': repo_builder.Placeholder('publish'),
2791+ },
2792+ 'branches_to_update': {
2793+ },
2794+ },
2795+ ['1-3', '1-2', '1-1'],
2796+ [
2797+ 'refs/tags/importer/import/1-1',
2798+ 'refs/tags/importer/import/1-3',
2799+ ],
2800+ False,
2801+ ),
2802+ (
2803+ repo_builder.Repo(
2804+ commit_list=[
2805+ repo_builder.Commit(
2806+ tree=repo_builder.SourceTree(
2807+ source_builder.Source(
2808+ source_builder.SourceSpec(
2809+ version='1-1',
2810+ native=False,
2811+ )
2812+ )
2813+ ),
2814+ name='import'
2815+ ),
2816+ repo_builder.Commit(
2817+ tree=repo_builder.SourceTree(
2818+ source_builder.Source(
2819+ source_builder.SourceSpec(
2820+ version='1-1',
2821+ native=False,
2822+ file_contents={
2823+ 'debian/random': 'Reimport tag contents',
2824+ },
2825+ )
2826+ )
2827+ ),
2828+ name='reimport'
2829+ ),
2830+ ],
2831+ branches={
2832+ },
2833+ tags={
2834+ 'importer/import/1-1': repo_builder.Placeholder('import'),
2835+ 'importer/reimport/import/1-1/0': repo_builder.Placeholder('import'),
2836+ 'importer/reimport/import/1-1/1': repo_builder.Placeholder('reimport'),
2837+ },
2838+ ),
2839+ {
2840+ 'commit_list_to_add': [
2841+ repo_builder.Commit(
2842+ tree=repo_builder.SourceTree(
2843+ source_builder.Source(
2844+ source_builder.SourceSpec(
2845+ changelog_versions=['1-2', '1-1'],
2846+ native=False,
2847+ )
2848+ )
2849+ ),
2850+ parents=[
2851+ repo_builder.Placeholder('import'),
2852+ repo_builder.Placeholder('reimport'),
2853+ ],
2854+ name='publish'
2855+ ),
2856+ ],
2857+ 'tags_to_add': {
2858+ 'importer/import/1-2': repo_builder.Placeholder('publish'),
2859+ },
2860+ 'branches_to_update': {
2861+ },
2862+ },
2863+ ['1-2', '1-1'],
2864+ [
2865+ 'refs/tags/importer/import/1-1',
2866+ 'refs/tags/importer/reimport/import/1-1/0',
2867+ 'refs/tags/importer/reimport/import/1-1/1',
2868+ 'refs/tags/importer/import/1-2',
2869+ ],
2870+ False,
2871+ ),
2872+ ]
2873+)
2874+@patch('gitubuntu.importer.get_import_tag_msg')
2875+@patch('gitubuntu.importer.get_import_commit_msg')
2876+def test_import_unapplied_spi_parenting(
2877+ get_import_commit_msg_mock,
2878+ get_import_tag_msg_mock,
2879+ repo,
2880+ input_data,
2881+ expected_changes,
2882+ changelog_versions,
2883+ test_refs,
2884+ xfail,
2885+):
2886+ # Match the repo_builder objects
2887+ get_import_tag_msg_mock.return_value = 'Test tag'
2888+ get_import_commit_msg_mock.return_value = b'Test commit'
2889+
2890+ publish_spec = source_builder.SourceSpec(
2891+ changelog_versions=changelog_versions,
2892+ native=False,
2893+ )
2894+ publish_source = source_builder.Source(publish_spec)
2895+
2896+ # copy before write, so that placeholders are still present
2897+ expected_result = input_data.copy(**expected_changes)
2898+
2899+ with patch.object(
2900+ repo,
2901+ 'get_commit_environment',
2902+ return_value={
2903+ 'GIT_AUTHOR_NAME':'Test Builder',
2904+ 'GIT_AUTHOR_EMAIL':'test@example.com',
2905+ 'GIT_AUTHOR_DATE':'1970-01-01T00:00:00Z',
2906+ 'GIT_COMMITTER_NAME':'Test Builder',
2907+ 'GIT_COMMITTER_EMAIL':'test@example.com',
2908+ 'GIT_COMMITTER_DATE':'1970-01-01T00:00:00Z',
2909+ },
2910+ ):
2911+ input_data.write(repo.raw_repo)
2912+
2913+ with publish_source as dsc_path:
2914+ spi = Mock()
2915+ spi.dsc_pathname = dsc_path
2916+ spi.distribution.name = 'Ubuntu'
2917+ spi.version = publish_spec.version
2918+ # avoid automatic parenting for head_name to make the assertion later
2919+ # easier
2920+ head_name = Mock(name='head_name')
2921+ head_name.return_value = 'importer/ubuntu/trusty'
2922+ spi.head_name = head_name
2923+ spi.date_published = '1970-01-01T00:00:00Z'
2924+ # import_unapplied_spi currently assumes it is called from the
2925+ # repository directory (pristine-tar and other commands rely on
2926+ # this)
2927+ target.import_unapplied_spi(
2928+ repo=repo,
2929+ spi=spi,
2930+ namespace='importer',
2931+ skip_orig=False,
2932+ parent_overrides={},
2933+ )
2934+
2935+ # we would like to check the commit hashes, but we cannot
2936+ # currently, as the commit messages are specific to the importer
2937+ # and not codified
2938+ assert repo_comparator.equals(
2939+ repoA=repo.raw_repo,
2940+ repoB=expected_result,
2941+ test_refs=test_refs,
2942+ ) or xfail
2943+
2944+ if xfail:
2945+ pytest.xfail(xfail)
2946+
2947+
2948+@patch('gitubuntu.importer.get_import_tag_msg')
2949+@patch('gitubuntu.importer.get_import_commit_msg')
2950+def test_import_unapplied_spi_parent_override(
2951+ get_import_commit_msg_mock,
2952+ get_import_tag_msg_mock,
2953+ repo,
2954+):
2955+ # Match the repo_builder objects
2956+ get_import_tag_msg_mock.return_value = 'Test tag'
2957+ get_import_commit_msg_mock.return_value = b'Test commit'
2958+
2959+ input_data = repo_builder.Repo(
2960+ commit_list=[
2961+ repo_builder.Commit(
2962+ tree=repo_builder.SourceTree(
2963+ source_builder.Source(
2964+ source_builder.SourceSpec(
2965+ version='1-1',
2966+ native=False,
2967+ )
2968+ )
2969+ ),
2970+ name='import1-1'
2971+ ),
2972+ repo_builder.Commit(
2973+ tree=repo_builder.SourceTree(
2974+ source_builder.Source(
2975+ source_builder.SourceSpec(
2976+ version='1-2',
2977+ native=False,
2978+ )
2979+ )
2980+ ),
2981+ name='import1-2'
2982+ ),
2983+ ],
2984+ branches={
2985+ },
2986+ tags={
2987+ 'importer/import/1-1': repo_builder.Placeholder('import1-1'),
2988+ 'importer/import/1-2': repo_builder.Placeholder('import1-2'),
2989+ },
2990+ )
2991+
2992+ expected_changes = {
2993+ 'commit_list_to_add': [
2994+ repo_builder.Commit(
2995+ tree=repo_builder.SourceTree(
2996+ source_builder.Source(
2997+ source_builder.SourceSpec(
2998+ changelog_versions=['2-1', '1-1'],
2999+ native=False,
3000+ )
3001+ )
3002+ ),
3003+ parents=[repo_builder.Placeholder('import1-2')],
3004+ name='publish'
3005+ ),
3006+ ],
3007+ 'tags_to_add': {
3008+ 'importer/import/2-1': repo_builder.Placeholder('publish'),
3009+ },
3010+ 'branches_to_update': {
3011+ },
3012+ }
3013+
3014+ test_refs = [
3015+ 'refs/tags/importer/import/1-1',
3016+ 'refs/tags/importer/import/1-2',
3017+ 'refs/tags/importer/import/2-1',
3018+ ]
3019+
3020+ publish_spec = source_builder.SourceSpec(
3021+ changelog_versions=['2-1', '1-1'],
3022+ native=False,
3023+ )
3024+ publish_source = source_builder.Source(publish_spec)
3025+
3026+ # copy before write, so that placeholders are still present
3027+ expected_result = input_data.copy(**expected_changes)
3028+
3029+ with patch.object(
3030+ repo,
3031+ 'get_commit_environment',
3032+ return_value={
3033+ 'GIT_AUTHOR_NAME':'Test Builder',
3034+ 'GIT_AUTHOR_EMAIL':'test@example.com',
3035+ 'GIT_AUTHOR_DATE':'1970-01-01T00:00:00Z',
3036+ 'GIT_COMMITTER_NAME':'Test Builder',
3037+ 'GIT_COMMITTER_EMAIL':'test@example.com',
3038+ 'GIT_COMMITTER_DATE':'1970-01-01T00:00:00Z',
3039+ },
3040+ ):
3041+ input_data.write(repo.raw_repo)
3042+
3043+ with publish_source as dsc_path:
3044+ spi = Mock()
3045+ spi.dsc_pathname = dsc_path
3046+ spi.distribution.name = 'Ubuntu'
3047+ spi.version = publish_spec.version
3048+ # avoid automatic parenting for head_name to make the assertion later
3049+ # easier
3050+ head_name = Mock(name='head_name')
3051+ head_name.return_value = 'importer/ubuntu/trusty'
3052+ spi.head_name = head_name
3053+ spi.date_published = '1970-01-01T00:00:00Z'
3054+ # import_unapplied_spi currently assumes it is called from the
3055+ # repository directory (pristine-tar and other commands rely on
3056+ # this)
3057+ target.import_unapplied_spi(
3058+ repo=repo,
3059+ spi=spi,
3060+ namespace='importer',
3061+ skip_orig=False,
3062+ parent_overrides={'2-1': {'changelog_parent': '1-2'}},
3063+ )
3064+
3065+ assert repo_comparator.equals(
3066+ repoA=repo.raw_repo,
3067+ repoB=expected_result,
3068+ test_refs=test_refs,
3069+ )
3070+
3071+
3072+def test_import_unapplied_spi_parent_override_failure(repo):
3073+ input_data = repo_builder.Repo(
3074+ commit_list=[
3075+ repo_builder.Commit(
3076+ tree=repo_builder.SourceTree(
3077+ source_builder.Source(
3078+ source_builder.SourceSpec(
3079+ version='1-1',
3080+ native=False,
3081+ )
3082+ )
3083+ ),
3084+ name='import1-1'
3085+ ),
3086+ ],
3087+ branches={
3088+ },
3089+ tags={
3090+ 'importer/import/1-1': repo_builder.Placeholder('import1-1'),
3091+ },
3092+ )
3093+
3094+ input_data.write(repo.raw_repo)
3095+
3096+ with pytest.raises(target.ParentOverrideError):
3097+ target.override_parents(
3098+ parent_overrides={'2-1': {'changelog_parent': '1-2'}},
3099+ repo=repo,
3100+ version='2-1',
3101+ namespace='importer',
3102+ )
3103+
3104+
3105+@pytest.mark.parametrize(
3106+ 'name, input_data, expected_changes, test_refs, xfail', [
3107+ (
3108+ 'Case 1',
3109+ repo_builder.Repo(
3110+ commit_list=[
3111+ repo_builder.Commit(
3112+ tree=repo_builder.SourceTree(
3113+ source_builder.Source(
3114+ source_builder.SourceSpec(
3115+ version='1-1',
3116+ native=False,
3117+ has_patches=True,
3118+ )
3119+ ),
3120+ patches_applied=False,
3121+ ),
3122+ name='unapplied'
3123+ ),
3124+ repo_builder.Commit(
3125+ tree=repo_builder.SourceTree(
3126+ source_builder.Source(
3127+ source_builder.SourceSpec(
3128+ version='1-1',
3129+ native=False,
3130+ has_patches=True,
3131+ )
3132+ ),
3133+ patches_applied=True,
3134+ ),
3135+ name='applied'
3136+ ),
3137+ ],
3138+ branches={
3139+ 'importer/ubuntu/trusty': repo_builder.Placeholder('unapplied'),
3140+ 'importer/applied/ubuntu/trusty': repo_builder.Placeholder('applied'),
3141+ },
3142+ tags={
3143+ 'importer/import/1-1': repo_builder.Placeholder('unapplied'),
3144+ 'importer/applied/1-1': repo_builder.Placeholder('applied'),
3145+ },
3146+ ),
3147+ {
3148+ 'commit_list_to_add': [],
3149+ 'tags_to_add': {},
3150+ 'branches_to_update': {},
3151+ },
3152+ [
3153+ #'refs/heads/importer/ubuntu/trusty',
3154+ #'refs/heads/importer/applied/ubuntu/trusty',
3155+ 'refs/tags/importer/import/1-1',
3156+ 'refs/tags/importer/applied/1-1',
3157+ ],
3158+ False,
3159+ ),
3160+ (
3161+ 'Case 2',
3162+ repo_builder.Repo(
3163+ commit_list=[
3164+ repo_builder.Commit(
3165+ tree=repo_builder.SourceTree(
3166+ source_builder.Source(
3167+ source_builder.SourceSpec(
3168+ version='1-1',
3169+ native=False,
3170+ has_patches=True,
3171+ file_contents={
3172+ 'debian/random': 'import tag contents'
3173+ },
3174+ )
3175+ ),
3176+ patches_applied=False,
3177+ ),
3178+ name='unapplied'
3179+ ),
3180+ repo_builder.Commit(
3181+ tree=repo_builder.SourceTree(
3182+ source_builder.Source(
3183+ source_builder.SourceSpec(
3184+ version='1-1',
3185+ native=False,
3186+ has_patches=True,
3187+ )
3188+ ),
3189+ patches_applied=False,
3190+ ),
3191+ name='unapplied_reimport'
3192+ ),
3193+ repo_builder.Commit(
3194+ tree=repo_builder.SourceTree(
3195+ source_builder.Source(
3196+ source_builder.SourceSpec(
3197+ version='1-1',
3198+ native=False,
3199+ has_patches=True,
3200+ file_contents={
3201+ 'debian/random': 'import tag contents'
3202+ },
3203+ )
3204+ ),
3205+ patches_applied=True,
3206+ ),
3207+ name='applied'
3208+ ),
3209+ ],
3210+ branches={
3211+ 'importer/ubuntu/trusty': repo_builder.Placeholder('unapplied'),
3212+ },
3213+ tags={
3214+ 'importer/import/1-1': repo_builder.Placeholder('unapplied'),
3215+ 'importer/reimport/import/1-1/0': repo_builder.Placeholder('unapplied'),
3216+ 'importer/reimport/import/1-1/1': repo_builder.Placeholder('unapplied_reimport'),
3217+ 'importer/applied/1-1': repo_builder.Placeholder('applied'),
3218+ },
3219+ ),
3220+ {
3221+ 'commit_list_to_add': [
3222+ repo_builder.Commit(
3223+ tree=repo_builder.SourceTree(
3224+ source_builder.Source(
3225+ source_builder.SourceSpec(
3226+ version='1-1',
3227+ native=False,
3228+ has_patches=True,
3229+ )
3230+ ),
3231+ patches_applied=True,
3232+ ),
3233+ parents=[repo_builder.Placeholder('unapplied_reimport')],
3234+ name='applied_reimport'
3235+ ),
3236+ ],
3237+ 'tags_to_add': {
3238+ 'importer/reimport/applied/1-1/0': repo_builder.Placeholder('applied'),
3239+ 'importer/reimport/applied/1-1/1': repo_builder.Placeholder('applied_reimport'),
3240+ },
3241+ 'branches_to_update': {
3242+ 'importer/applied/ubuntu/trusty': repo_builder.Placeholder('applied_reimport'),
3243+ },
3244+ },
3245+ [
3246+ #'refs/heads/importer/ubuntu/trusty',
3247+ #'refs/heads/importer/applied/ubuntu/trusty',
3248+ 'refs/tags/importer/import/1-1',
3249+ 'refs/tags/importer/reimport/import/1-1/0',
3250+ 'refs/tags/importer/reimport/import/1-1/1',
3251+ 'refs/tags/importer/applied/1-1',
3252+ 'refs/tags/importer/reimport/applied/1-1/0',
3253+ 'refs/tags/importer/reimport/applied/1-1/1',
3254+ ],
3255+ False,
3256+ ),
3257+ (
3258+ 'Case 3',
3259+ repo_builder.Repo(
3260+ commit_list=[
3261+ repo_builder.Commit(
3262+ tree=repo_builder.SourceTree(
3263+ source_builder.Source(
3264+ source_builder.SourceSpec(
3265+ version='1-1',
3266+ native=False,
3267+ has_patches=True,
3268+ )
3269+ ),
3270+ patches_applied=False,
3271+ ),
3272+ name='unapplied'
3273+ ),
3274+ ],
3275+ branches={
3276+ 'importer/ubuntu/trusty': repo_builder.Placeholder('unapplied'),
3277+ },
3278+ tags={
3279+ 'importer/import/1-1': repo_builder.Placeholder('unapplied'),
3280+ },
3281+ ),
3282+ {
3283+ 'commit_list_to_add': [
3284+ repo_builder.Commit(
3285+ tree=repo_builder.SourceTree(
3286+ source_builder.Source(
3287+ source_builder.SourceSpec(
3288+ version='1-1',
3289+ native=False,
3290+ has_patches=True,
3291+ )
3292+ ),
3293+ patches_applied=True,
3294+ ),
3295+ parents=[repo_builder.Placeholder('unapplied')],
3296+ name='applied'
3297+ ),
3298+ ],
3299+ 'tags_to_add': {
3300+ 'importer/applied/1-1': repo_builder.Placeholder('applied')
3301+ },
3302+ 'branches_to_update': {
3303+ 'importer/applied/ubuntu/trusty': repo_builder.Placeholder('applied'),
3304+ },
3305+ },
3306+ [
3307+ #'refs/heads/importer/ubuntu/trusty',
3308+ #'refs/heads/importer/applied/ubuntu/trusty',
3309+ 'refs/tags/importer/import/1-1',
3310+ 'refs/tags/importer/applied/1-1'
3311+ ],
3312+ False,
3313+ ),
3314+ ]
3315+)
3316+@patch('gitubuntu.importer.get_import_tag_msg')
3317+@patch('gitubuntu.importer.get_import_commit_msg')
3318+def test_import_applied_spi_tags(
3319+ get_import_commit_msg_mock,
3320+ get_import_tag_msg_mock,
3321+ repo,
3322+ name,
3323+ input_data,
3324+ expected_changes,
3325+ test_refs,
3326+ xfail,
3327+):
3328+ # Match the repo_builder objects
3329+ get_import_tag_msg_mock.return_value = 'Test tag'
3330+ get_import_commit_msg_mock.return_value = b'Test commit'
3331+
3332+ publish_spec = source_builder.SourceSpec(
3333+ version='1-1',
3334+ native=False,
3335+ has_patches=True,
3336+ )
3337+ publish_source = source_builder.Source(publish_spec)
3338+
3339+ # copy before write, so that placeholders are still present
3340+ expected_result = input_data.copy(**expected_changes)
3341+
3342+ # Ensure reproducible commit hashes
3343+ with patch.object(
3344+ repo,
3345+ 'get_commit_environment',
3346+ return_value={
3347+ 'GIT_AUTHOR_NAME':'Test Builder',
3348+ 'GIT_AUTHOR_EMAIL':'test@example.com',
3349+ 'GIT_AUTHOR_DATE':'1970-01-01T00:00:00Z',
3350+ 'GIT_COMMITTER_NAME':'Test Builder',
3351+ 'GIT_COMMITTER_EMAIL':'test@example.com',
3352+ 'GIT_COMMITTER_DATE':'1970-01-01T00:00:00Z',
3353+ },
3354+ ):
3355+ input_data.write(repo.raw_repo)
3356+
3357+ with publish_source as dsc_path:
3358+ spi = Mock()
3359+ spi.dsc_pathname = dsc_path
3360+ spi.distribution.name = 'Ubuntu'
3361+ spi.version = publish_spec.version
3362+ # avoid automatic parenting for head_name to make the assertion later
3363+ # easier
3364+ applied_head_name = Mock(name='applied_head_name')
3365+ applied_head_name.return_value = 'importer/applied/ubuntu/trusty'
3366+ spi.applied_head_name = applied_head_name
3367+ spi.date_published = '1970-01-01T00:00:00Z'
3368+ target.import_applied_spi(
3369+ repo=repo,
3370+ spi=spi,
3371+ namespace='importer',
3372+ allow_applied_failures=False,
3373+ parent_overrides={},
3374+ )
3375+
3376+ assert repo_comparator.equals(
3377+ repoA=repo.raw_repo,
3378+ repoB=expected_result,
3379+ test_refs=test_refs,
3380+ # this should be Commit, but we do not have a method yet
3381+ # to obtain the stage-wise `quilt push` commits for a
3382+ # patches-applied import
3383+ comparison_type=repo_comparator.RepoComparisonType.Tree,
3384+ ) or xfail
3385+
3386+ if xfail:
3387+ pytest.xfail(xfail)
3388+
3389+
3390+# test:
3391+# if only one import tag exists, then it is the parent
3392+# if two import tags exist, then only the are all parents
3393+# corresponding applied tags are also a parent?
3394+@pytest.mark.parametrize(
3395+ 'input_data, expected_ancestor_commits, expected_parent_commits, xfail', [
3396+ (
3397+ repo_builder.Repo(
3398+ commit_list=[
3399+ repo_builder.Commit(
3400+ tree=repo_builder.SourceTree(
3401+ source_builder.Source(
3402+ source_builder.SourceSpec(
3403+ version='1-1',
3404+ native=False,
3405+ has_patches=True,
3406+ )
3407+ ),
3408+ patches_applied=False,
3409+ ),
3410+ name='unapplied1'
3411+ ),
3412+ repo_builder.Commit(
3413+ tree=repo_builder.SourceTree(
3414+ source_builder.Source(
3415+ source_builder.SourceSpec(
3416+ changelog_versions=['1-2', '1-1'],
3417+ native=False,
3418+ has_patches=True,
3419+ )
3420+ ),
3421+ patches_applied=False,
3422+ ),
3423+ parents=[repo_builder.Placeholder('unapplied1')],
3424+ name='unapplied2'
3425+ ),
3426+ repo_builder.Commit(
3427+ tree=repo_builder.SourceTree(
3428+ source_builder.Source(
3429+ source_builder.SourceSpec(
3430+ version='1-1',
3431+ native=False,
3432+ has_patches=True,
3433+ )
3434+ ),
3435+ patches_applied=True,
3436+ ),
3437+ name='applied1'
3438+ ),
3439+ ],
3440+ branches={},
3441+ tags={
3442+ 'importer/import/1-1': repo_builder.Placeholder('unapplied1'),
3443+ 'importer/import/1-2': repo_builder.Placeholder('unapplied2'),
3444+ 'importer/applied/1-1': repo_builder.Placeholder('applied1'),
3445+ },
3446+ ),
3447+ [
3448+ 'refs/tags/importer/import/1-2',
3449+ ],
3450+ [
3451+ 'refs/tags/importer/applied/1-1',
3452+ ],
3453+ False,
3454+ ),
3455+ (
3456+ repo_builder.Repo(
3457+ commit_list=[
3458+ repo_builder.Commit(
3459+ tree=repo_builder.SourceTree(
3460+ source_builder.Source(
3461+ source_builder.SourceSpec(
3462+ version='1-1',
3463+ native=False,
3464+ has_patches=True,
3465+ )
3466+ ),
3467+ patches_applied=False,
3468+ ),
3469+ name='unapplied1'
3470+ ),
3471+ repo_builder.Commit(
3472+ tree=repo_builder.SourceTree(
3473+ source_builder.Source(
3474+ source_builder.SourceSpec(
3475+ changelog_versions=['1-2', '1-1'],
3476+ native=False,
3477+ has_patches=True,
3478+ )
3479+ ),
3480+ patches_applied=False,
3481+ ),
3482+ parents=[repo_builder.Placeholder('unapplied1')],
3483+ name='unapplied2'
3484+ ),
3485+ repo_builder.Commit(
3486+ tree=repo_builder.SourceTree(
3487+ source_builder.Source(
3488+ source_builder.SourceSpec(
3489+ changelog_versions=['1-2', '1-1'],
3490+ native=False,
3491+ has_patches=True,
3492+ file_contents={
3493+ 'debian/random': 'reimport tag',
3494+ },
3495+ )
3496+ ),
3497+ patches_applied=False,
3498+ ),
3499+ parents=[repo_builder.Placeholder('unapplied1')],
3500+ name='unapplied2reimport'
3501+ ),
3502+ repo_builder.Commit(
3503+ tree=repo_builder.SourceTree(
3504+ source_builder.Source(
3505+ source_builder.SourceSpec(
3506+ version='1-1',
3507+ native=False,
3508+ has_patches=True,
3509+ )
3510+ ),
3511+ patches_applied=True,
3512+ ),
3513+ name='applied1'
3514+ ),
3515+ ],
3516+ branches={},
3517+ tags={
3518+ 'importer/import/1-1': repo_builder.Placeholder('unapplied1'),
3519+ 'importer/import/1-2': repo_builder.Placeholder('unapplied2'),
3520+ 'importer/reimport/import/1-2/0': repo_builder.Placeholder('unapplied2'),
3521+ 'importer/reimport/import/1-2/1': repo_builder.Placeholder('unapplied2reimport'),
3522+ 'importer/applied/1-1': repo_builder.Placeholder('applied1'),
3523+ },
3524+ ),
3525+ [
3526+ 'refs/tags/importer/import/1-2',
3527+ ],
3528+ [
3529+ 'refs/tags/importer/applied/1-1',
3530+ ],
3531+ False,
3532+ ),
3533+ (
3534+ repo_builder.Repo(
3535+ commit_list=[
3536+ repo_builder.Commit(
3537+ tree=repo_builder.SourceTree(
3538+ source_builder.Source(
3539+ source_builder.SourceSpec(
3540+ version='1-1',
3541+ native=False,
3542+ has_patches=True,
3543+ )
3544+ ),
3545+ patches_applied=False,
3546+ ),
3547+ name='unapplied1'
3548+ ),
3549+ repo_builder.Commit(
3550+ tree=repo_builder.SourceTree(
3551+ source_builder.Source(
3552+ source_builder.SourceSpec(
3553+ version='1-1',
3554+ native=False,
3555+ has_patches=True,
3556+ file_contents={
3557+ 'debian/random': 'reimport contents',
3558+ },
3559+ )
3560+ ),
3561+ patches_applied=False,
3562+ ),
3563+ name='unapplied1_reimport'
3564+ ),
3565+ repo_builder.Commit(
3566+ tree=repo_builder.SourceTree(
3567+ source_builder.Source(
3568+ source_builder.SourceSpec(
3569+ changelog_versions=['1-2', '1-1'],
3570+ native=False,
3571+ has_patches=True,
3572+ )
3573+ ),
3574+ patches_applied=False,
3575+ ),
3576+ parents=[repo_builder.Placeholder('unapplied1')],
3577+ name='unapplied2'
3578+ ),
3579+ repo_builder.Commit(
3580+ tree=repo_builder.SourceTree(
3581+ source_builder.Source(
3582+ source_builder.SourceSpec(
3583+ version='1-1',
3584+ native=False,
3585+ has_patches=True,
3586+ )
3587+ ),
3588+ patches_applied=True,
3589+ ),
3590+ name='applied1'
3591+ ),
3592+ repo_builder.Commit(
3593+ tree=repo_builder.SourceTree(
3594+ source_builder.Source(
3595+ source_builder.SourceSpec(
3596+ version='1-1',
3597+ native=False,
3598+ has_patches=True,
3599+ file_contents={
3600+ 'debian/random': 'reimport contents',
3601+ },
3602+ )
3603+ ),
3604+ patches_applied=True,
3605+ ),
3606+ name='applied1_reimport'
3607+ ),
3608+ ],
3609+ branches={},
3610+ tags={
3611+ 'importer/import/1-1': repo_builder.Placeholder('unapplied1'),
3612+ 'importer/reimport/import/1-1/0': repo_builder.Placeholder('unapplied1'),
3613+ 'importer/reimport/import/1-1/1': repo_builder.Placeholder('unapplied1_reimport'),
3614+ 'importer/import/1-2': repo_builder.Placeholder('unapplied2'),
3615+ 'importer/applied/1-1': repo_builder.Placeholder('applied1'),
3616+ 'importer/reimport/applied/1-1/0': repo_builder.Placeholder('applied1'),
3617+ 'importer/reimport/applied/1-1/1': repo_builder.Placeholder('applied1_reimport'),
3618+ },
3619+ ),
3620+ [
3621+ 'refs/tags/importer/import/1-2',
3622+ ],
3623+ [
3624+ 'refs/tags/importer/reimport/applied/1-1/0',
3625+ 'refs/tags/importer/reimport/applied/1-1/1',
3626+ ],
3627+ False,
3628+ ),
3629+ ],
3630+)
3631+@patch('gitubuntu.importer.get_import_tag_msg')
3632+@patch('gitubuntu.importer.get_import_commit_msg')
3633+def test_import_applied_spi_parenting(
3634+ get_import_commit_msg_mock,
3635+ get_import_tag_msg_mock,
3636+ repo,
3637+ input_data,
3638+ expected_ancestor_commits,
3639+ expected_parent_commits,
3640+ xfail,
3641+):
3642+ """Test that import_applied_spi correctly parents the new commit
3643+
3644+ This test is ugly because we do not yet have a programmatic way
3645+ to get the interstitial commits of the patch applications.
3646+ """
3647+ # Match the repo_builder objects
3648+ get_import_tag_msg_mock.return_value = 'Test tag'
3649+ get_import_commit_msg_mock.return_value = b'Test commit'
3650+
3651+ publish_spec = source_builder.SourceSpec(
3652+ changelog_versions=['1-2', '1-1'],
3653+ native=False,
3654+ has_patches=True,
3655+ )
3656+ publish_source = source_builder.Source(publish_spec)
3657+
3658+ # Ensure reproducible commit hashes
3659+ with patch.object(
3660+ repo,
3661+ 'get_commit_environment',
3662+ return_value={
3663+ 'GIT_AUTHOR_NAME':'Test Builder',
3664+ 'GIT_AUTHOR_EMAIL':'test@example.com',
3665+ 'GIT_AUTHOR_DATE':'1970-01-01T00:00:00Z',
3666+ 'GIT_COMMITTER_NAME':'Test Builder',
3667+ 'GIT_COMMITTER_EMAIL':'test@example.com',
3668+ 'GIT_COMMITTER_DATE':'1970-01-01T00:00:00Z',
3669+ },
3670+ ):
3671+ input_data.write(repo.raw_repo)
3672+
3673+ with publish_source as dsc_path:
3674+ spi = Mock()
3675+ spi.dsc_pathname = dsc_path
3676+ spi.distribution.name = 'Ubuntu'
3677+ spi.version = publish_spec.version
3678+ # avoid automatic parenting for head_name to make the assertion later
3679+ # easier
3680+ applied_head_name = Mock(name='applied_head_name')
3681+ applied_head_name.return_value = 'importer/applied/ubuntu/trusty'
3682+ spi.applied_head_name = applied_head_name
3683+ spi.date_published = '1970-01-01T00:00:00Z'
3684+ target.import_applied_spi(
3685+ repo=repo,
3686+ spi=spi,
3687+ namespace='importer',
3688+ allow_applied_failures=False,
3689+ parent_overrides={},
3690+ )
3691+
3692+ applied_tag = repo.raw_repo.lookup_reference('refs/tags/importer/applied/1-2')
3693+ applied_commit = applied_tag.peel(pygit2.Commit)
3694+ applied_commit_parents = applied_commit.parents
3695+
3696+ # convert refs to commits
3697+ expected_ancestor_commit_hashes = [
3698+ str(
3699+ repo.raw_repo.lookup_reference(
3700+ expected_ancestor
3701+ ).peel(pygit2.Commit).id
3702+ )
3703+ for expected_ancestor in expected_ancestor_commits
3704+ ]
3705+ expected_parent_commit_hashes = [
3706+ str(
3707+ repo.raw_repo.lookup_reference(
3708+ expected_parent
3709+ ).peel(pygit2.Commit).id
3710+ )
3711+ for expected_parent in expected_parent_commits
3712+ ]
3713+
3714+ missing_ancestor_commit_hashes = []
3715+ for ancestor_commit_hash in expected_ancestor_commit_hashes:
3716+ merge_base = repo.raw_repo.merge_base(
3717+ repo.raw_repo.get(ancestor_commit_hash).id,
3718+ applied_commit.id,
3719+ )
3720+ if str(
3721+ repo.raw_repo.get(merge_base).peel(pygit2.Commit).id
3722+ ) != ancestor_commit_hash:
3723+ missing_ancestor_commit_hashes.append(ancestor_commit_hash)
3724+
3725+ missing_parent_commit_hashes = []
3726+ for parent_commit_hash in expected_parent_commit_hashes:
3727+ for parent in applied_commit_parents:
3728+ if str(parent.id) == parent_commit_hash:
3729+ break
3730+ else:
3731+ missing_parent_commit_hashes.append(parent_commit_hash)
3732+
3733+ assert (
3734+ not missing_ancestor_commit_hashes and
3735+ not missing_parent_commit_hashes
3736+ ) or xfail
3737+
3738+ if xfail:
3739+ pytest.xfail(xfail)
3740diff --git a/gitubuntu/importerutils.py b/gitubuntu/importerutils.py
3741new file mode 100644
3742index 0000000..8df2ea5
3743--- /dev/null
3744+++ b/gitubuntu/importerutils.py
3745@@ -0,0 +1,50 @@
3746+import logging
3747+
3748+def debug_log_head_versions(head_versions):
3749+ """Emit debug logging about the version in Git branches
3750+
3751+ :param head_versions dict a dictionary of dictionaries, keyed by Git
3752+ branch names, where each elemetn is a dictionary with elements
3753+ 'version' (a string package version) and 'head' (a pygit2.Branch
3754+ object)
3755+ :rtype None
3756+ """
3757+ for head_name in sorted(head_versions):
3758+ logging.debug('Last imported %s version is %s',
3759+ head_name,
3760+ head_versions[head_name]['version']
3761+ )
3762+
3763+
3764+def head_ref_name(spi, ref_namespace):
3765+ """Obtain the patches-unapplied Git ref name a spi would import to
3766+
3767+ :param gitubuntu.source_information.GitUbuntuSourcePackageInformation spi
3768+ The source package to examine
3769+ :param ref_namespace str The namespace the resulting Git ref should be in
3770+
3771+ :rtype str
3772+ """
3773+ if spi.pocket.lower() == 'release':
3774+ pocket_suffix = ''
3775+ else:
3776+ pocket_suffix = '-' + spi.pocket.lower()
3777+
3778+ return '%s/%s/%s%s' % (
3779+ ref_namespace,
3780+ spi.distribution.name.lower(),
3781+ spi.series.name.lower(),
3782+ pocket_suffix,
3783+ )
3784+
3785+
3786+def applied_head_ref_name(spi, ref_namespace):
3787+ """Obtain the patches-unapplied Git ref name a spi would import to
3788+
3789+ :param gitubuntu.source_information.GitUbuntuSourcePackageInformation spi
3790+ The source package to examine
3791+ :param ref_namespace str The namespace the resulting Git ref should be in
3792+
3793+ :rtype str
3794+ """
3795+ return head_ref_name(spi, '%s/applied' % ref_namespace)
3796diff --git a/gitubuntu/importlocal.py b/gitubuntu/importlocal.py
3797index 778a673..386919c 100644
3798--- a/gitubuntu/importlocal.py
3799+++ b/gitubuntu/importlocal.py
3800@@ -98,7 +98,6 @@ def main(
3801 namespace=namespace,
3802 dist=None,
3803 dsc_pathname=dsc_path,
3804- head_name=None,
3805 skip_orig=skip_orig,
3806 parent_overrides=parent_overrides,
3807 fallback_author=None,
3808@@ -111,7 +110,6 @@ def main(
3809 namespace=namespace,
3810 dist=None,
3811 dsc_pathname=dsc_path,
3812- head_name=None,
3813 allow_applied_failures=False,
3814 parent_overrides=parent_overrides,
3815 fallback_author=None,
3816diff --git a/gitubuntu/importppa.py b/gitubuntu/importppa.py
3817index 024143f..668c01f 100644
3818--- a/gitubuntu/importppa.py
3819+++ b/gitubuntu/importppa.py
3820@@ -14,6 +14,7 @@ from gitubuntu.importer import (
3821 import_unapplied_spi,
3822 import_applied_spi,
3823 )
3824+from gitubuntu.importerutils import debug_log_head_versions
3825 from gitubuntu.source_information import (
3826 GitUbuntuSourceInformation,
3827 NoPublicationHistoryException,
3828@@ -67,26 +68,14 @@ def main(
3829 )
3830
3831 ubuntu_head_versions = repo.get_heads_and_versions(
3832- 'ubuntu',
3833- namespace,
3834+ '%s/ubuntu' % namespace
3835 )
3836- for head_name in sorted(ubuntu_head_versions):
3837- _, _, pretty_head_name = head_name.partition('%s/', namespace)
3838- logging.debug('Last imported %s version is %s',
3839- pretty_head_name,
3840- ubuntu_head_versions[head_name]['version']
3841- )
3842+ debug_log_head_versions(ubuntu_head_versions)
3843
3844 applied_ubuntu_head_versions = repo.get_heads_and_versions(
3845- 'applied/ubuntu',
3846- namespace,
3847+ '%s/applied/ubuntu' % namespace
3848 )
3849- for head_name in sorted(applied_ubuntu_head_versions):
3850- _, _, pretty_head_name = head_name.partition('%s/', namespace)
3851- logging.debug('Last applied %s version is %s',
3852- pretty_head_name,
3853- applied_ubuntu_head_versions[head_name]['version']
3854- )
3855+ debug_log_head_versions(applied_ubuntu_head_versions)
3856
3857 oldcwd = os.getcwd()
3858 os.chdir(repo.local_dir)
3859diff --git a/gitubuntu/source_information.py b/gitubuntu/source_information.py
3860index 9e8f172..f287954 100644
3861--- a/gitubuntu/source_information.py
3862+++ b/gitubuntu/source_information.py
3863@@ -9,6 +9,8 @@ _LP_LOGIN_AUTH = None
3864 _lp_service = 'production'
3865 _lp_api_version = 'devel'
3866
3867+import gitubuntu.importerutils
3868+
3869 try:
3870 pkg = 'python3-distro-info'
3871 from distro_info import DebianDistroInfo, UbuntuDistroInfo
3872@@ -134,8 +136,18 @@ class GitUbuntuPPASourcePackage(UbuntuSourcePackage):
3873
3874
3875 class GitUbuntuSourcePackageInformation:
3876- def __init__(self, spphr, dist_name, retries=0, retry_backoffs=[],
3877- workdir=None, dsc=None, files=list()):
3878+ def __init__(
3879+ self,
3880+ spphr,
3881+ dist_name,
3882+ retries=0,
3883+ retry_backoffs=[],
3884+ workdir=None,
3885+ dsc=None,
3886+ files=None,
3887+ ):
3888+ if not files:
3889+ files = list()
3890 self._spphr = spphr
3891 self._dist_name = dist_name
3892 self._pkgname = self.spphr.source_package_name
3893@@ -154,13 +166,15 @@ class GitUbuntuSourcePackageInformation:
3894
3895 # do this here, in case files is passed
3896 if workdir and not os.path.isdir(workdir):
3897- os.makedirs(workdir, exist_ok=True)
3898-
3899- self._archive_srcpkg = func(package=self._pkgname,
3900- version=self._version,
3901- workdir=workdir,
3902- quiet=True,
3903- dscfile=dsc)
3904+ os.makedirs(workdir, exist_ok=True)
3905+
3906+ self._archive_srcpkg = func(
3907+ package=self._pkgname,
3908+ version=self._version,
3909+ workdir=workdir,
3910+ quiet=True,
3911+ dscfile=dsc,
3912+ )
3913 for f in files:
3914 self._archive_srcpkg._download_file(f, f.split('/')[-1])
3915
3916@@ -205,62 +219,6 @@ class GitUbuntuSourcePackageInformation:
3917 return self._spphr.pocket
3918
3919 @property
3920- def pretty_head_name(self):
3921- if self._spphr.pocket.lower() == 'release':
3922- head_name = '%s/%s' % (
3923- self.distribution.name.lower(),
3924- self.series.name.lower(),
3925- )
3926- else:
3927- head_name = '%s/%s-%s' % (
3928- self.distribution.name.lower(),
3929- self.series.name.lower(),
3930- self.pocket.lower()
3931- )
3932- return head_name
3933-
3934- def head_name(self, prefix):
3935- if self._spphr.pocket.lower() == 'release':
3936- head_name = '%s/%s/%s' % (
3937- prefix,
3938- self.distribution.name.lower(),
3939- self.series.name.lower(),
3940- )
3941- else:
3942- head_name = '%s/%s/%s-%s' % (
3943- prefix,
3944- self.distribution.name.lower(),
3945- self.series.name.lower(),
3946- self.pocket.lower()
3947- )
3948- return head_name
3949-
3950- def applied_head_name(self, prefix):
3951- return self.head_name('%s/applied' % prefix)
3952-
3953- def parent_head_name(self, prefix):
3954- if self.parent_series == None:
3955- return None
3956- if self.pocket.lower() == 'release':
3957- # release pockets descend from prior series
3958- head_name = '%s/%s/%s' % (
3959- prefix,
3960- self.distribution.name.lower(),
3961- self.parent_series.name.lower()
3962- )
3963- else:
3964- # non-release pockets descend from release
3965- head_name = '%s/%s/%s' % (
3966- prefix,
3967- self.distribution.name.lower(),
3968- self.series.name.lower()
3969- )
3970- return head_name
3971-
3972- def parent_applied_head_name(self, prefix):
3973- return self.parent_head_name('%s/applied' % prefix)
3974-
3975- @property
3976 def dsc(self):
3977 return self._archive_srcpkg.dsc
3978
3979@@ -298,11 +256,16 @@ class GitUbuntuSourceInformation(object):
3980 _stable_series_list = None
3981 _current_series = None
3982
3983- def __init__(self, dist_name, pkgname=None,
3984- pull_overrides_filename='/dev/null',
3985- retries=0,
3986- retry_backoffs=[]
3987- ):
3988+ def __init__(
3989+ self,
3990+ dist_name,
3991+ pkgname=None,
3992+ pull_overrides_filename='/dev/null',
3993+ retries=0,
3994+ retry_backoffs=None,
3995+ ):
3996+ if not retry_backoffs:
3997+ retry_backoffs = []
3998 self.launchpad = launchpad_login()
3999 self.dist_name = dist_name
4000 if self.dist_name.startswith('ppa:'):
4001@@ -327,18 +290,6 @@ class GitUbuntuSourceInformation(object):
4002 self.retries = retries
4003 self.retry_backoffs = retry_backoffs
4004
4005- @staticmethod
4006- def _head_version_is_equal(head_versions, namespace, spi):
4007- try:
4008- if (head_versions[spi.head_name(namespace)]['version'] == spi.version and
4009- (spi.date_published is None or
4010- int(spi.date_published.timestamp()) == head_versions[spi.head_name(namespace)]['head'].peel().commit_time)
4011- ):
4012- return True
4013- return False
4014- except KeyError:
4015- return False
4016-
4017 @property
4018 def current_series(self):
4019 if self.dist_name.startswith('ppa:'):
4020@@ -405,19 +356,25 @@ class GitUbuntuSourceInformation(object):
4021 def all_series_name_list(self):
4022 return [r.name for r in self.all_series]
4023
4024- def get_corrected_spi(self, srcpkg, workdir=None):
4025- try:
4026- pull_override = self.pull_overrides[srcpkg.source_package_version]
4027- dsc = pull_override['dsc']
4028- files = pull_override['files']
4029- except KeyError:
4030- dsc = None
4031- files = list()
4032- return GitUbuntuSourcePackageInformation(srcpkg, self.dist_name,
4033- self.retries, self.retry_backoffs, workdir=workdir,
4034- dsc=dsc, files=files)
4035-
4036- def launchpad_version_is_published(self, version, workdir=None):
4037+ def get_corrected_spi(self, srcpkg, workdir):
4038+ try:
4039+ pull_override = self.pull_overrides[srcpkg.source_package_version]
4040+ dsc = pull_override['dsc']
4041+ files = pull_override['files']
4042+ except KeyError:
4043+ dsc = None
4044+ files = list()
4045+ return GitUbuntuSourcePackageInformation(
4046+ srcpkg,
4047+ self.dist_name,
4048+ self.retries,
4049+ self.retry_backoffs,
4050+ workdir=workdir,
4051+ dsc=dsc,
4052+ files=files,
4053+ )
4054+
4055+ def launchpad_version_is_published(self, version):
4056 spph = self.archive.getPublishedSources(
4057 exact_match=True,
4058 source_name=self.pkgname,
4059@@ -426,13 +383,16 @@ class GitUbuntuSourceInformation(object):
4060 )
4061 return len(spph) != 0
4062
4063- def launchpad_versions_published(self, workdir=None,
4064- sorted_by_version=False, series=None
4065+ def launchpad_versions_published(
4066+ self,
4067+ workdir=None,
4068+ sorted_by_version=False,
4069+ series=None,
4070 ):
4071 args = {
4072- 'exact_match':True,
4073- 'source_name':self.pkgname,
4074- }
4075+ 'exact_match':True,
4076+ 'source_name':self.pkgname,
4077+ }
4078 if not sorted_by_version:
4079 args['order_by_date'] = True
4080 if series:
4081@@ -440,18 +400,50 @@ class GitUbuntuSourceInformation(object):
4082
4083 spph = self.archive.getPublishedSources(**args)
4084 if len(spph) == 0:
4085- raise NoPublicationHistoryException("Is %s published in %s?" %
4086- (self.pkgname, self.dist_name))
4087+ raise NoPublicationHistoryException(
4088+ "Is %s published in %s?" % (
4089+ self.pkgname,
4090+ self.dist_name,
4091+ )
4092+ )
4093
4094 for srcpkg in spph:
4095 yield self.get_corrected_spi(srcpkg, workdir)
4096
4097- def launchpad_versions_published_after(self, head_versions, namespace, workdir=None, active_series_only=False):
4098+ def launchpad_versions_published_after(
4099+ self,
4100+ head_versions,
4101+ head_ref_namespace,
4102+ workdir=None,
4103+ active_series_only=False,
4104+ ):
4105+ """Provide Launchpad publishing information after a certain
4106+ publishing event
4107+
4108+ :param head_versions dict Dictionary of dictionaries, keyed by
4109+ Git branch names, where each element is a dictionary with
4110+ elements 'version' and 'head'
4111+ :param head_ref_namespace str Git ref namespace corresponding to
4112+ the branches in @head_versions
4113+ :param workdir str Filesystem path to store the downloaded
4114+ source package(s) in
4115+ :param active_series_only bool Only publishing records from
4116+ active series should be considered
4117+
4118+ @head_versions[<branch>]['head'] is a pygit2.Branch object
4119+ @head_versions[<branch>]['version'] is the string version of the
4120+ top debian/changelog in the corresponding Git branch.
4121+
4122+ :rtype iterable
4123+ :returns iterable over GitUbuntuSourcePackageInformation
4124+ objects, one per Launchpad publishing record after
4125+ @head_version
4126+ """
4127 args = {
4128- 'exact_match':True,
4129- 'source_name':self.pkgname,
4130- 'order_by_date':True,
4131- }
4132+ 'exact_match':True,
4133+ 'source_name':self.pkgname,
4134+ 'order_by_date':True,
4135+ }
4136
4137 # we have the date of the commit too, so we can double-check
4138 # that it matches
4139@@ -472,14 +464,33 @@ class GitUbuntuSourceInformation(object):
4140 # Sanity check that the passed in srcpkg name has a publication
4141 # history
4142 if len(spph) == 0:
4143- raise NoPublicationHistoryException("Is %s published in %s?" %
4144- (self.pkgname, self.dist_name))
4145- if len(head_versions) > 0:
4146+ raise NoPublicationHistoryException(
4147+ "Is %s published in %s?" % (
4148+ self.pkgname,
4149+ self.dist_name,
4150+ )
4151+ )
4152+ if head_versions:
4153 _spph = list()
4154 for spphr in spph:
4155- spi = GitUbuntuSourcePackageInformation(spphr, self.dist_name,
4156- workdir=workdir)
4157- if self._head_version_is_equal(head_versions, namespace, spi):
4158+ spi = GitUbuntuSourcePackageInformation(
4159+ spphr,
4160+ self.dist_name,
4161+ workdir=workdir,
4162+ )
4163+ head_version = head_versions[
4164+ gitubuntu.importerutils.head_ref_name(
4165+ spi,
4166+ head_ref_namespace,
4167+ )
4168+ ]
4169+ if (
4170+ head_version['version'] == str(spi.version) and
4171+ spi.date_published and
4172+ int(spi.date_published.timestamp()) == head_version[
4173+ 'head'
4174+ ].peel().commit_time
4175+ ):
4176 break
4177 _spph.append(spphr)
4178 spph = _spph

Subscribers

People subscribed via source and target branches