Merge ~cjwatson/launchpad:built-using-domination into launchpad:master

Proposed by Colin Watson
Status: Needs review
Proposed branch: ~cjwatson/launchpad:built-using-domination
Merge into: launchpad:master
Prerequisite: ~cjwatson/launchpad:built-using-guard-deletion
Diff against target: 576 lines (+347/-42)
3 files modified
lib/lp/archivepublisher/domination.py (+156/-41)
lib/lp/archivepublisher/tests/test_dominator.py (+183/-1)
lib/lp/soyuz/interfaces/binarysourcereference.py (+8/-0)
Reviewer Review Type Date Requested Status
Launchpad code reviewers Pending
Review via email: mp+381240@code.launchpad.net

Commit message

Handle Built-Using references in the dominator

Description of the change

Keep source publications with Built-Using references from active binary publications. This may extend to reinstating the source publication (via a copy) if it had already been superseded or deleted.

It's possible for this to cause confusing effects if a manual deletion races with a build that produces binaries with a Built-Using reference to the deleted source. I've guarded against this as best I can, and hope the remaining cases will be rare, but err on the side of honouring the reference.

To post a comment you must log in.
Revision history for this message
Colin Watson (cjwatson) wrote (last edit ):

I'm reasonably sure that it would be possible to extend this to cover the self-reference case (any active binary publications should keep their parent source publication active too) with minimal trouble, but I've already spent quite a long time on this so haven't attempted to actually do that extension here.

c0c4aa2... by Colin Watson

Expand deletion guard to other pockets

It now checks all pockets that could legitimately depend on the one from
which the publication is being deleted.

06ccc3e... by Colin Watson

Simplify tests using createFromSourcePackageReleases

88f502f... by Colin Watson

Fix Built-Using domination twice in a row

If a source has Built-Using references to it but has already been
superseded by the dominator, then a subsequent run of the dominator will
still consider that source in case reinstatement is needed, and would
fail an assertion when trying to supersede a source that's already
superseded.

The simplest fix for this seems to be to have `planPackageDomination`
not add the publication to `supersede` or `delete` if it's already
inactive.

965bddb... by Colin Watson

Tighten up tests slightly

7bfb655... by Colin Watson

Fix calculation of live source versions

The dominator previously incorrectly reinstated source publications if
they were the latest one being considered for domination, even if that
was an inactive publication with only inactive Built-Using references.

Unmerged commits

7bfb655... by Colin Watson

Fix calculation of live source versions

The dominator previously incorrectly reinstated source publications if
they were the latest one being considered for domination, even if that
was an inactive publication with only inactive Built-Using references.

965bddb... by Colin Watson

Tighten up tests slightly

88f502f... by Colin Watson

Fix Built-Using domination twice in a row

If a source has Built-Using references to it but has already been
superseded by the dominator, then a subsequent run of the dominator will
still consider that source in case reinstatement is needed, and would
fail an assertion when trying to supersede a source that's already
superseded.

The simplest fix for this seems to be to have `planPackageDomination`
not add the publication to `supersede` or `delete` if it's already
inactive.

f4479f2... by Colin Watson

Handle Built-Using references in the dominator

Keep source publications with Built-Using references from active binary
publications. This may extend to reinstating the source publication
(via a copy) if it had already been superseded or deleted.

It's possible for this to cause confusing effects if a manual deletion
races with a build that produces binaries with a Built-Using reference
to the deleted source. I've guarded against this as best I can, and
hope the remaining cases will be rare, but err on the side of honouring
the reference.

LP: #1868558

06ccc3e... by Colin Watson

Simplify tests using createFromSourcePackageReleases

c0c4aa2... by Colin Watson

Expand deletion guard to other pockets

It now checks all pockets that could legitimately depend on the one from
which the publication is being deleted.

d7fbcfd... by Colin Watson

Guard removal of sources referenced by Built-Using

Prevent SourcePackagePublishingHistory.requestDeletion from deleting
source publications that have Built-Using references from active binary
publications in the same archive and suite.

This isn't necessarily complete: in particular, it can miss references
from other pockets, and in any case it might race with a build still in
progress. The intent of this is not to ensure integrity, but to avoid
some easily-detectable mistakes that could cause confusion.

LP: #1868558

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
diff --git a/lib/lp/archivepublisher/domination.py b/lib/lp/archivepublisher/domination.py
index a02fd78..6721c64 100644
--- a/lib/lp/archivepublisher/domination.py
+++ b/lib/lp/archivepublisher/domination.py
@@ -1,4 +1,4 @@
1# Copyright 2009-2019 Canonical Ltd. This software is licensed under the1# Copyright 2009-2020 Canonical Ltd. This software is licensed under the
2# GNU Affero General Public License version 3 (see the file LICENSE).2# GNU Affero General Public License version 3 (see the file LICENSE).
33
4"""Archive Domination class.4"""Archive Domination class.
@@ -68,10 +68,12 @@ from storm.expr import (
68 And,68 And,
69 Count,69 Count,
70 Desc,70 Desc,
71 Or,
71 Select,72 Select,
72 )73 )
73from zope.component import getUtility74from zope.component import getUtility
7475
76from lp.app.interfaces.launchpad import ILaunchpadCelebrities
75from lp.registry.model.sourcepackagename import SourcePackageName77from lp.registry.model.sourcepackagename import SourcePackageName
76from lp.services.database.bulk import load_related78from lp.services.database.bulk import load_related
77from lp.services.database.constants import UTC_NOW79from lp.services.database.constants import UTC_NOW
@@ -82,17 +84,24 @@ from lp.services.database.sqlbase import (
82 sqlvalues,84 sqlvalues,
83 )85 )
84from lp.services.orderingcheck import OrderingCheck86from lp.services.orderingcheck import OrderingCheck
87from lp.soyuz.adapters.archivedependencies import pocket_dependencies
85from lp.soyuz.enums import (88from lp.soyuz.enums import (
86 BinaryPackageFormat,89 BinaryPackageFormat,
90 BinarySourceReferenceType,
87 PackagePublishingStatus,91 PackagePublishingStatus,
88 )92 )
93from lp.soyuz.interfaces.binarysourcereference import (
94 IBinarySourceReferenceSet,
95 )
89from lp.soyuz.interfaces.publishing import (96from lp.soyuz.interfaces.publishing import (
97 active_publishing_status,
90 inactive_publishing_status,98 inactive_publishing_status,
91 IPublishingSet,99 IPublishingSet,
92 )100 )
93from lp.soyuz.model.binarypackagebuild import BinaryPackageBuild101from lp.soyuz.model.binarypackagebuild import BinaryPackageBuild
94from lp.soyuz.model.binarypackagename import BinaryPackageName102from lp.soyuz.model.binarypackagename import BinaryPackageName
95from lp.soyuz.model.binarypackagerelease import BinaryPackageRelease103from lp.soyuz.model.binarypackagerelease import BinaryPackageRelease
104from lp.soyuz.model.binarysourcereference import BinarySourceReference
96from lp.soyuz.model.publishing import (105from lp.soyuz.model.publishing import (
97 BinaryPackagePublishingHistory,106 BinaryPackagePublishingHistory,
98 SourcePackagePublishingHistory,107 SourcePackagePublishingHistory,
@@ -207,19 +216,50 @@ class GeneralizedPublication:
207 return sorted(publications, cmp=self.compare, reverse=True)216 return sorted(publications, cmp=self.compare, reverse=True)
208217
209218
210def find_live_source_versions(sorted_pubs):219def get_source_versions(source_publications):
211 """Find versions out of Published publications that should stay live.220 """List versions for sequence of `SourcePackagePublishingHistory`.
221
222 :param source_publications: An iterable of
223 `SourcePackagePublishingHistory`.
224 :return: A list of the publications' respective versions.
225 """
226 return [pub.sourcepackagerelease.version for pub in source_publications]
212227
213 This particular notion of liveness applies to source domination: the228
214 latest version stays live, and that's it.229def find_live_source_versions(sorted_pubs, built_using=None):
230 """Find versions of source publications that should stay live.
231
232 This particular notion of liveness applies to source domination:
233 normally, the latest Published version stays live, and that's it. The
234 exception is if live binary publications have Built-Using fields
235 referring to some source versions, in which case those versions stay
236 live too.
215237
216 :param sorted_pubs: An iterable of `SourcePackagePublishingHistory`238 :param sorted_pubs: An iterable of `SourcePackagePublishingHistory`
217 sorted by descending package version.239 sorted by descending package version.
240 :param built_using: An optional collection of `BinarySourceReference`s
241 corresponding to published binary publications with Built-Using
242 references; the SPRs that these refer to should stay live.
218 :return: A list of live versions.243 :return: A list of live versions.
219 """244 """
245 if built_using is None:
246 built_using = set()
247 built_using_spr_ids = set(
248 bsr.source_package_release_id for bsr in built_using)
249
220 # Given the required sort order, the latest version is at the head250 # Given the required sort order, the latest version is at the head
221 # of the list.251 # of the list.
222 return [sorted_pubs[0].sourcepackagerelease.version]252 sorted_pubs = list(sorted_pubs)
253 live_pubs = []
254 latest = None
255 for pub in sorted_pubs:
256 if latest is None and pub.status == PackagePublishingStatus.PUBLISHED:
257 live_pubs.append(pub)
258 latest = pub
259 elif (pub != latest and
260 pub.sourcepackagereleaseID in built_using_spr_ids):
261 live_pubs.append(pub)
262 return get_source_versions(live_pubs)
223263
224264
225def get_binary_versions(binary_publications):265def get_binary_versions(binary_publications):
@@ -372,11 +412,11 @@ class Dominator:
372 generalization):412 generalization):
373 """Plan domination of publications for a single package.413 """Plan domination of publications for a single package.
374414
375 The latest publication for any version in `live_versions` stays415 The latest active publication for any version in `live_versions`
376 active. Any older publications (including older publications for416 stays active. Any older active publications (including older
377 live versions with multiple publications) are marked as superseded by417 publications for live versions with multiple publications) are
378 the respective oldest live releases that are newer than the superseded418 marked as superseded by the respective oldest live releases that are
379 ones.419 newer than the superseded ones.
380420
381 Any versions that are newer than anything in `live_versions` are421 Any versions that are newer than anything in `live_versions` are
382 marked as deleted. This should not be possible in Soyuz-native422 marked as deleted. This should not be possible in Soyuz-native
@@ -384,11 +424,16 @@ class Dominator:
384 previous latest version of a package has disappeared from the Sources424 previous latest version of a package has disappeared from the Sources
385 list we import.425 list we import.
386426
427 Inactive publications that are listed in `live_versions` are
428 reinstated.
429
387 :param sorted_pubs: A list of publications for the same package,430 :param sorted_pubs: A list of publications for the same package,
388 in the same archive, series, and pocket, all with status431 in the same archive, series, and pocket. These will normally
389 `PackagePublishingStatus.PUBLISHED`. They must be sorted from432 all have status `PackagePublishingStatus.PUBLISHED`, but source
390 most current to least current, as would be the result of433 publications referenced by Built-Using may reach here with
391 `generalization.sortPublications`.434 different statuses and be considered for reinstatement. They
435 must be sorted from most current to least current, as would be
436 the result of `generalization.sortPublications`.
392 :param live_versions: Iterable of versions that are still considered437 :param live_versions: Iterable of versions that are still considered
393 "live" for this package. For any of these, the latest publication438 "live" for this package. For any of these, the latest publication
394 among `publications` will remain Published. Publications for439 among `publications` will remain Published. Publications for
@@ -434,28 +479,38 @@ class Dominator:
434 # This publication is for a live version, but has been479 # This publication is for a live version, but has been
435 # superseded by a newer publication of the same version.480 # superseded by a newer publication of the same version.
436 # Supersede it.481 # Supersede it.
437 supersede.append((pub, current_dominant))482 if pub.status in active_publishing_status:
438 self.logger.debug2(483 supersede.append((pub, current_dominant))
439 "Superseding older publication for version %s.", version)484 self.logger.debug2(
485 "Superseding older publication for version %s.",
486 version)
440 elif version in live_versions:487 elif version in live_versions:
441 # This publication stays active; if any publications488 if pub.status in active_publishing_status:
442 # that follow right after this are to be superseded,489 # This publication stays active; if any publications
443 # this is the release that they are superseded by.490 # that follow right after this are to be superseded,
444 current_dominant = pub491 # this is the release that they are superseded by.
445 dominant_version = version492 current_dominant = pub
446 keep.add(pub)493 dominant_version = version
447 self.logger.debug2("Keeping version %s.", version)494 keep.add(pub)
495 self.logger.debug2("Keeping version %s.", version)
496 else:
497 # This publication is currently inactive, but is
498 # referenced by an active publication. Reinstate it.
499 keep.add(pub)
500 self.logger.debug2("Reinstating version %s.", version)
448 elif current_dominant is None:501 elif current_dominant is None:
449 # This publication is no longer live, but there is no502 # This publication is no longer live, but there is no
450 # newer version to supersede it either. Therefore it503 # newer version to supersede it either. Therefore it
451 # must be deleted.504 # must be deleted.
452 delete.append(pub)505 if pub.status in active_publishing_status:
453 self.logger.debug2("Deleting version %s.", version)506 delete.append(pub)
507 self.logger.debug2("Deleting version %s.", version)
454 else:508 else:
455 # This publication is superseded. This is what we're509 # This publication is superseded. This is what we're
456 # here to do.510 # here to do.
457 supersede.append((pub, current_dominant))511 if pub.status in active_publishing_status:
458 self.logger.debug2("Superseding version %s.", version)512 supersede.append((pub, current_dominant))
513 self.logger.debug2("Superseding version %s.", version)
459514
460 return supersede, keep, delete515 return supersede, keep, delete
461516
@@ -707,19 +762,22 @@ class Dominator:
707762
708 execute_plan()763 execute_plan()
709764
710 def _composeActiveSourcePubsCondition(self, distroseries, pocket):765 def _composeRelevantSourcePubsCondition(self, distroseries, pocket,
766 active=True):
711 """Compose ORM condition for restricting relevant source pubs."""767 """Compose ORM condition for restricting relevant source pubs."""
712 SPPH = SourcePackagePublishingHistory768 SPPH = SourcePackagePublishingHistory
713769
714 return And(770 clauses = [
715 SPPH.status == PackagePublishingStatus.PUBLISHED,
716 SPPH.distroseries == distroseries,771 SPPH.distroseries == distroseries,
717 SPPH.archive == self.archive,772 SPPH.archive == self.archive,
718 SPPH.pocket == pocket,773 SPPH.pocket == pocket,
719 )774 ]
775 if active:
776 clauses.append(SPPH.status == PackagePublishingStatus.PUBLISHED)
777 return And(*clauses)
720778
721 def findSourcesForDomination(self, distroseries, pocket):779 def findSourcesForDomination(self, distroseries, pocket):
722 """Find binary publications that need dominating.780 """Find source publications that need dominating.
723781
724 This is only for traditional domination, where the latest published782 This is only for traditional domination, where the latest published
725 publication is always kept published. See `find_live_source_versions`783 publication is always kept published. See `find_live_source_versions`
@@ -728,17 +786,32 @@ class Dominator:
728 To optimize for that logic, `findSourcesForDomination` will ignore786 To optimize for that logic, `findSourcesForDomination` will ignore
729 publications that have no other publications competing for the same787 publications that have no other publications competing for the same
730 binary package. There'd be nothing to do for those cases.788 binary package. There'd be nothing to do for those cases.
789
790 This also includes source publications whose source package releases
791 have a Built-Using reference pointing to them, whether active or
792 not. These may need to be superseded or kept/reinstated depending
793 on whether binary publications referring to them are active.
731 """794 """
732 SPPH = SourcePackagePublishingHistory795 SPPH = SourcePackagePublishingHistory
733 SPR = SourcePackageRelease796 SPR = SourcePackageRelease
797 BSR = BinarySourceReference
734798
735 spph_location_clauses = self._composeActiveSourcePubsCondition(799 spph_location_clauses = self._composeRelevantSourcePubsCondition(
736 distroseries, pocket)800 distroseries, pocket)
737 candidate_source_names = Select(801 candidate_source_names = Select(
738 SPPH.sourcepackagenameID,802 SPPH.sourcepackagenameID,
739 And(join_spph_spr(), spph_location_clauses),803 And(join_spph_spr(), spph_location_clauses),
740 group_by=SPPH.sourcepackagenameID,804 group_by=SPPH.sourcepackagenameID,
741 having=(Count() > 1))805 having=(Count() > 1))
806 built_using_spph_location_clauses = (
807 self._composeRelevantSourcePubsCondition(
808 distroseries, pocket, active=False))
809 built_using_sprs = Select(
810 SPPH.sourcepackagereleaseID,
811 And(
812 SPPH.sourcepackagereleaseID == BSR.source_package_release_id,
813 BSR.reference_type == BinarySourceReferenceType.BUILT_USING,
814 built_using_spph_location_clauses))
742815
743 # We'll also access the SourcePackageReleases associated with816 # We'll also access the SourcePackageReleases associated with
744 # the publications we find. Since they're in the join anyway,817 # the publications we find. Since they're in the join anyway,
@@ -749,8 +822,13 @@ class Dominator:
749 query = IStore(SPPH).find(822 query = IStore(SPPH).find(
750 (SPPH, SPR),823 (SPPH, SPR),
751 join_spph_spr(),824 join_spph_spr(),
752 SPPH.sourcepackagenameID.is_in(candidate_source_names),825 Or(
753 spph_location_clauses)826 And(
827 SPPH.sourcepackagenameID.is_in(candidate_source_names),
828 spph_location_clauses),
829 And(
830 SPPH.sourcepackagereleaseID.is_in(built_using_sprs),
831 built_using_spph_location_clauses)))
754 spphs = DecoratedResultSet(query, itemgetter(0))832 spphs = DecoratedResultSet(query, itemgetter(0))
755 load_related(SourcePackageName, spphs, ['sourcepackagenameID'])833 load_related(SourcePackageName, spphs, ['sourcepackagenameID'])
756 return spphs834 return spphs
@@ -771,20 +849,57 @@ class Dominator:
771 sources = self.findSourcesForDomination(distroseries, pocket)849 sources = self.findSourcesForDomination(distroseries, pocket)
772 sorted_packages = self._sortPackages(sources, generalization)850 sorted_packages = self._sortPackages(sources, generalization)
773 supersede = []851 supersede = []
852 keep = set()
774 delete = []853 delete = []
775854
855 # Of the SPRs associated with publications being considered for
856 # domination, find those that have a Built-Using reference pointing
857 # to them from a live binary publication.
858 bsr_set = getUtility(IBinarySourceReferenceSet)
859 reverse_pockets = {
860 source_pocket
861 for source_pocket, expanded_pockets in pocket_dependencies.items()
862 if pocket in expanded_pockets}
863 built_using = bsr_set.findPublished(
864 self.archive, distroseries, reverse_pockets,
865 BinarySourceReferenceType.BUILT_USING,
866 source_package_releases=[
867 pub.sourcepackagerelease for pub in sources])
868
776 self.logger.debug("Dominating sources...")869 self.logger.debug("Dominating sources...")
777 for name, pubs in sorted_packages.iteritems():870 for name, pubs in sorted_packages.iteritems():
778 self.logger.debug("Dominating %s" % name)871 self.logger.debug("Dominating %s" % name)
779 assert len(pubs) > 0, "Dominating zero sources!"872 assert len(pubs) > 0, "Dominating zero sources!"
780 live_versions = find_live_source_versions(pubs)873 live_versions = find_live_source_versions(pubs, built_using)
781 cur_supersede, _, cur_delete = self.planPackageDomination(874 cur_supersede, cur_keep, cur_delete = self.planPackageDomination(
782 pubs, live_versions, generalization)875 pubs, live_versions, generalization)
783 supersede.extend(cur_supersede)876 supersede.extend(cur_supersede)
877 keep.update(cur_keep)
784 delete.extend(cur_delete)878 delete.extend(cur_delete)
785879
786 for pub, dominant in supersede:880 for pub, dominant in supersede:
787 pub.supersede(dominant, logger=self.logger)881 pub.supersede(dominant, logger=self.logger)
882 for pub in keep:
883 if pub.status not in active_publishing_status:
884 # The dominator thinks that we should keep this package, but
885 # it isn't currently published. This can happen when a
886 # source package has initially been superseded, but then a
887 # binary package that references it in Built-Using is
888 # published, requiring the source package to be reinstated.
889 # To cope with this, we'll copy the source back into place,
890 # thereby creating a new PENDING publication record for it,
891 # which will cause the publisher to put it back on disk.
892 #
893 # In general we'd prefer that the dominator not undo manual
894 # deletions, but Built-Using often expresses a compliance
895 # requirement, so we intentionally don't check that here.
896 # Instead, SPPH.requestDeletion checks for live
897 # BinarySourceReferences, and the uploader rejects uploads
898 # of builds with Built-Using references to source packages
899 # that have been manually deleted.
900 pub.copyTo(
901 distroseries, pocket, self.archive,
902 creator=getUtility(ILaunchpadCelebrities).janitor)
788 for pub in delete:903 for pub in delete:
789 pub.requestDeletion(None)904 pub.requestDeletion(None)
790905
@@ -804,7 +919,7 @@ class Dominator:
804 looking_for,919 looking_for,
805 join_spph_spr(),920 join_spph_spr(),
806 join_spph_spn(),921 join_spph_spn(),
807 self._composeActiveSourcePubsCondition(distroseries, pocket))922 self._composeRelevantSourcePubsCondition(distroseries, pocket))
808 return result.group_by(SourcePackageName.name)923 return result.group_by(SourcePackageName.name)
809924
810 def findPublishedSPPHs(self, distroseries, pocket, package_name):925 def findPublishedSPPHs(self, distroseries, pocket, package_name):
@@ -817,7 +932,7 @@ class Dominator:
817 join_spph_spr(),932 join_spph_spr(),
818 join_spph_spn(),933 join_spph_spn(),
819 SourcePackageName.name == package_name,934 SourcePackageName.name == package_name,
820 self._composeActiveSourcePubsCondition(distroseries, pocket))935 self._composeRelevantSourcePubsCondition(distroseries, pocket))
821 # Sort by descending version (SPR.version has type debversion in936 # Sort by descending version (SPR.version has type debversion in
822 # the database, so this should be a real proper comparison) so937 # the database, so this should be a real proper comparison) so
823 # that _sortPackage will have slightly less work to do later.938 # that _sortPackage will have slightly less work to do later.
diff --git a/lib/lp/archivepublisher/tests/test_dominator.py b/lib/lp/archivepublisher/tests/test_dominator.py
index b88da56..7db5336 100755
--- a/lib/lp/archivepublisher/tests/test_dominator.py
+++ b/lib/lp/archivepublisher/tests/test_dominator.py
@@ -1,4 +1,4 @@
1# Copyright 2009-2019 Canonical Ltd. This software is licensed under the1# Copyright 2009-2020 Canonical Ltd. This software is licensed under the
2# GNU Affero General Public License version 3 (see the file LICENSE).2# GNU Affero General Public License version 3 (see the file LICENSE).
33
4"""Tests for domination.py."""4"""Tests for domination.py."""
@@ -14,6 +14,8 @@ import apt_pkg
14from testtools.matchers import (14from testtools.matchers import (
15 GreaterThan,15 GreaterThan,
16 LessThan,16 LessThan,
17 MatchesSetwise,
18 MatchesStructure,
17 )19 )
18import transaction20import transaction
19from zope.component import getUtility21from zope.component import getUtility
@@ -397,6 +399,186 @@ class TestDominator(TestNativePublishingBase):
397 for pub in overrides_2:399 for pub in overrides_2:
398 self.assertEqual(PackagePublishingStatus.PUBLISHED, pub.status)400 self.assertEqual(PackagePublishingStatus.PUBLISHED, pub.status)
399401
402 def test_dominateSources_keeps_built_using_refs(self):
403 # If a source publication has a Built-Using reference from an active
404 # binary publication in the same archive and series and in a pocket
405 # that could legitimately refer to it, it is retained.
406 foo_10_src = self.getPubSource(
407 sourcename="foo", version="1.0", architecturehintlist="i386",
408 status=PackagePublishingStatus.PUBLISHED)
409 [foo_10_i386_bin] = self.getPubBinaries(
410 binaryname="foo-bin", status=PackagePublishingStatus.PUBLISHED,
411 architecturespecific=True, version="1.0", pub_source=foo_10_src)
412 foo_11_src = self.getPubSource(
413 sourcename="foo", version="1.1", architecturehintlist="i386",
414 status=PackagePublishingStatus.PUBLISHED)
415 [foo_11_i386_bin] = self.getPubBinaries(
416 binaryname="foo-bin", status=PackagePublishingStatus.PUBLISHED,
417 architecturespecific=True, version="1.1", pub_source=foo_11_src)
418 bar_10_src = self.getPubSource(
419 sourcename="bar", version="1.0", architecturehintlist="i386",
420 status=PackagePublishingStatus.PUBLISHED)
421 [bar_10_i386_bin] = self.getPubBinaries(
422 binaryname="bar-bin", status=PackagePublishingStatus.PUBLISHED,
423 architecturespecific=True, version="1.0", pub_source=bar_10_src,
424 built_using="foo (= 1.0)")
425
426 dominator = Dominator(self.logger, self.ubuntutest.main_archive)
427 dominator.judgeAndDominate(foo_10_src.distroseries, foo_10_src.pocket)
428
429 self.checkPublications(
430 [foo_10_src,
431 foo_11_src, foo_11_i386_bin,
432 bar_10_src, bar_10_i386_bin],
433 PackagePublishingStatus.PUBLISHED)
434 self.checkPublication(
435 foo_10_i386_bin, PackagePublishingStatus.SUPERSEDED)
436
437 # No copies were performed.
438 pending_srcs = foo_10_src.archive.getPublishedSources(
439 status=PackagePublishingStatus.PENDING)
440 pending_bins = foo_10_src.archive.getAllPublishedBinaries(
441 status=PackagePublishingStatus.PENDING)
442 self.assertEqual(0, pending_srcs.count())
443 self.assertEqual(0, pending_bins.count())
444
445 bar_11_src = self.getPubSource(
446 sourcename="bar", version="1.1", architecturehintlist="i386",
447 status=PackagePublishingStatus.PUBLISHED)
448 [bar_11_i386_bin] = self.getPubBinaries(
449 binaryname="bar-bin", status=PackagePublishingStatus.PUBLISHED,
450 architecturespecific=True, version="1.1", pub_source=bar_11_src,
451 built_using="foo (= 1.1)")
452
453 # Dominate twice to make sure the result is stable.
454 for _ in range(2):
455 dominator.judgeAndDominate(
456 foo_10_src.distroseries, foo_10_src.pocket)
457
458 self.checkPublications(
459 [foo_11_src, foo_11_i386_bin,
460 bar_11_src, bar_11_i386_bin],
461 PackagePublishingStatus.PUBLISHED)
462 self.checkPublications(
463 [foo_10_src, foo_10_i386_bin,
464 bar_10_src, bar_10_i386_bin],
465 PackagePublishingStatus.SUPERSEDED)
466
467 # No copies were performed.
468 pending_srcs = foo_10_src.archive.getPublishedSources(
469 status=PackagePublishingStatus.PENDING)
470 pending_bins = foo_10_src.archive.getAllPublishedBinaries(
471 status=PackagePublishingStatus.PENDING)
472 self.assertEqual(0, pending_srcs.count())
473 self.assertEqual(0, pending_bins.count())
474
475 def test_dominateSources_reinstates_superseded_built_using_refs(self):
476 # If a superseded source publication has a Built-Using reference
477 # from an active binary publication in the same archive and series
478 # and in a pocket that could legitimately refer to it, it is
479 # reinstated.
480 foo_10_src = self.getPubSource(
481 sourcename="foo", version="1.0", architecturehintlist="i386",
482 status=PackagePublishingStatus.PUBLISHED)
483 [foo_10_i386_bin] = self.getPubBinaries(
484 binaryname="foo-bin", status=PackagePublishingStatus.PUBLISHED,
485 architecturespecific=True, version="1.0", pub_source=foo_10_src)
486 foo_11_src = self.getPubSource(
487 sourcename="foo", version="1.1", architecturehintlist="i386",
488 status=PackagePublishingStatus.PUBLISHED)
489 [foo_11_i386_bin] = self.getPubBinaries(
490 binaryname="foo-bin", status=PackagePublishingStatus.PUBLISHED,
491 architecturespecific=True, version="1.1", pub_source=foo_11_src)
492 bar_10_src = self.getPubSource(
493 sourcename="bar", version="1.0", architecturehintlist="i386",
494 status=PackagePublishingStatus.PUBLISHED,
495 pocket=PackagePublishingPocket.PROPOSED)
496
497 dominator = Dominator(self.logger, self.ubuntutest.main_archive)
498 dominator.judgeAndDominate(foo_10_src.distroseries, foo_10_src.pocket)
499
500 self.checkPublications(
501 [foo_11_src, foo_11_i386_bin,
502 bar_10_src],
503 PackagePublishingStatus.PUBLISHED)
504 self.checkPublications(
505 [foo_10_src, foo_10_i386_bin], PackagePublishingStatus.SUPERSEDED)
506
507 [bar_10_i386_bin] = self.getPubBinaries(
508 binaryname="bar-bin", status=PackagePublishingStatus.PUBLISHED,
509 pocket=PackagePublishingPocket.PROPOSED, architecturespecific=True,
510 version="1.0", pub_source=bar_10_src, built_using="foo (= 1.0)")
511
512 # Dominate twice to make sure the result is stable.
513 for _ in range(2):
514 dominator.judgeAndDominate(
515 foo_10_src.distroseries, foo_10_src.pocket)
516
517 self.checkPublications(
518 [foo_11_src, foo_11_i386_bin,
519 bar_10_src, bar_10_i386_bin],
520 PackagePublishingStatus.PUBLISHED)
521 self.checkPublications(
522 [foo_10_src, foo_10_i386_bin],
523 PackagePublishingStatus.SUPERSEDED)
524
525 # The foo 1.0 source was reinstated.
526 pending_srcs = foo_10_src.archive.getPublishedSources(
527 status=PackagePublishingStatus.PENDING)
528 pending_bins = foo_10_src.archive.getAllPublishedBinaries(
529 status=PackagePublishingStatus.PENDING)
530 self.assertThat(pending_srcs, MatchesSetwise(
531 MatchesStructure(
532 sourcepackagerelease=MatchesStructure.byEquality(
533 name="foo", version="1.0"))))
534 self.assertEqual(0, pending_bins.count())
535
536 def test_dominateSources_skips_inactive_built_using_refs(self):
537 # While inactive source publications are considered for
538 # reinstatement if they are referenced by Built-Using from any
539 # binary publication, they are only actually reinstated if they are
540 # referenced from an *active* binary publication; merely being the
541 # most recent publication of the source package in question is not
542 # enough.
543 foo_10_src = self.getPubSource(
544 sourcename="foo", version="1.0", architecturehintlist="i386",
545 status=PackagePublishingStatus.SUPERSEDED)
546 foo_11_src = self.getPubSource(
547 sourcename="foo", version="1.1", architecturehintlist="i386",
548 status=PackagePublishingStatus.DELETED)
549 bar_10_src = self.getPubSource(
550 sourcename="bar", version="1.0", architecturehintlist="i386",
551 status=PackagePublishingStatus.SUPERSEDED)
552 [bar_10_i386_bin] = self.getPubBinaries(
553 binaryname="bar-bin", status=PackagePublishingStatus.SUPERSEDED,
554 architecturespecific=True, version="1.0", pub_source=bar_10_src,
555 built_using="foo (= 1.0)")
556 bar_11_src = self.getPubSource(
557 sourcename="bar", version="1.1", architecturehintlist="i386",
558 status=PackagePublishingStatus.SUPERSEDED)
559 [bar_11_i386_bin] = self.getPubBinaries(
560 binaryname="bar-bin", status=PackagePublishingStatus.SUPERSEDED,
561 architecturespecific=True, version="1.1", pub_source=bar_11_src,
562 built_using="foo (= 1.1)")
563
564 dominator = Dominator(self.logger, self.ubuntutest.main_archive)
565 dominator.judgeAndDominate(foo_10_src.distroseries, foo_10_src.pocket)
566
567 self.checkPublications(
568 [foo_10_src,
569 bar_10_src, bar_10_i386_bin,
570 bar_11_src, bar_11_i386_bin],
571 PackagePublishingStatus.SUPERSEDED)
572 self.checkPublications([foo_11_src], PackagePublishingStatus.DELETED)
573
574 # No copies were performed.
575 pending_srcs = foo_10_src.archive.getPublishedSources(
576 status=PackagePublishingStatus.PENDING)
577 pending_bins = foo_10_src.archive.getAllPublishedBinaries(
578 status=PackagePublishingStatus.PENDING)
579 self.assertEqual(0, pending_srcs.count())
580 self.assertEqual(0, pending_bins.count())
581
400582
401class TestDomination(TestNativePublishingBase):583class TestDomination(TestNativePublishingBase):
402 """Test overall domination procedure."""584 """Test overall domination procedure."""
diff --git a/lib/lp/soyuz/interfaces/binarysourcereference.py b/lib/lp/soyuz/interfaces/binarysourcereference.py
index e625cf9..d110256 100644
--- a/lib/lp/soyuz/interfaces/binarysourcereference.py
+++ b/lib/lp/soyuz/interfaces/binarysourcereference.py
@@ -47,6 +47,14 @@ class IBinarySourceReference(Interface):
47 vocabulary=BinarySourceReferenceType,47 vocabulary=BinarySourceReferenceType,
48 required=True, readonly=True)48 required=True, readonly=True)
4949
50 # Export IDs for use by the dominator.
51 binary_package_release_id = Int(
52 title=_("The referencing binary package release ID."),
53 required=True, readonly=True)
54 source_package_release_id = Int(
55 title=_("The referencing source package release ID."),
56 required=True, readonly=True)
57
5058
51class IBinarySourceReferenceSet(Interface):59class IBinarySourceReferenceSet(Interface):
52 """A set of references from binary packages to source packages."""60 """A set of references from binary packages to source packages."""

Subscribers

People subscribed via source and target branches

to status/vote changes: