Merge ~cjwatson/launchpad:built-using-domination into launchpad:master
- Git
- lp:~cjwatson/launchpad
- built-using-domination
- Merge into master
Status: | Needs review | ||||
---|---|---|---|---|---|
Proposed branch: | ~cjwatson/launchpad:built-using-domination | ||||
Merge into: | launchpad:master | ||||
Prerequisite: | ~cjwatson/launchpad:built-using-guard-deletion | ||||
Diff against target: |
576 lines (+347/-42) 3 files modified
lib/lp/archivepublisher/domination.py (+156/-41) lib/lp/archivepublisher/tests/test_dominator.py (+183/-1) lib/lp/soyuz/interfaces/binarysourcereference.py (+8/-0) |
||||
Related bugs: |
|
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Launchpad code reviewers | Pending | ||
Review via email: mp+381240@code.launchpad.net |
Commit message
Handle Built-Using references in the dominator
Description of the change
Keep source publications with Built-Using references from active binary publications. This may extend to reinstating the source publication (via a copy) if it had already been superseded or deleted.
It's possible for this to cause confusing effects if a manual deletion races with a build that produces binaries with a Built-Using reference to the deleted source. I've guarded against this as best I can, and hope the remaining cases will be rare, but err on the side of honouring the reference.
- c0c4aa2... by Colin Watson
-
Expand deletion guard to other pockets
It now checks all pockets that could legitimately depend on the one from
which the publication is being deleted. - 06ccc3e... by Colin Watson
-
Simplify tests using createFromSourc
ePackageRelease s - 88f502f... by Colin Watson
-
Fix Built-Using domination twice in a row
If a source has Built-Using references to it but has already been
superseded by the dominator, then a subsequent run of the dominator will
still consider that source in case reinstatement is needed, and would
fail an assertion when trying to supersede a source that's already
superseded.The simplest fix for this seems to be to have `planPackageDom
ination`
not add the publication to `supersede` or `delete` if it's already
inactive. - 965bddb... by Colin Watson
-
Tighten up tests slightly
- 7bfb655... by Colin Watson
-
Fix calculation of live source versions
The dominator previously incorrectly reinstated source publications if
they were the latest one being considered for domination, even if that
was an inactive publication with only inactive Built-Using references.
Unmerged commits
- 7bfb655... by Colin Watson
-
Fix calculation of live source versions
The dominator previously incorrectly reinstated source publications if
they were the latest one being considered for domination, even if that
was an inactive publication with only inactive Built-Using references. - 965bddb... by Colin Watson
-
Tighten up tests slightly
- 88f502f... by Colin Watson
-
Fix Built-Using domination twice in a row
If a source has Built-Using references to it but has already been
superseded by the dominator, then a subsequent run of the dominator will
still consider that source in case reinstatement is needed, and would
fail an assertion when trying to supersede a source that's already
superseded.The simplest fix for this seems to be to have `planPackageDom
ination`
not add the publication to `supersede` or `delete` if it's already
inactive. - f4479f2... by Colin Watson
-
Handle Built-Using references in the dominator
Keep source publications with Built-Using references from active binary
publications. This may extend to reinstating the source publication
(via a copy) if it had already been superseded or deleted.It's possible for this to cause confusing effects if a manual deletion
races with a build that produces binaries with a Built-Using reference
to the deleted source. I've guarded against this as best I can, and
hope the remaining cases will be rare, but err on the side of honouring
the reference.LP: #1868558
- 06ccc3e... by Colin Watson
-
Simplify tests using createFromSourc
ePackageRelease s - c0c4aa2... by Colin Watson
-
Expand deletion guard to other pockets
It now checks all pockets that could legitimately depend on the one from
which the publication is being deleted. - d7fbcfd... by Colin Watson
-
Guard removal of sources referenced by Built-Using
Prevent SourcePackagePu
blishingHistory .requestDeletio n from deleting
source publications that have Built-Using references from active binary
publications in the same archive and suite.This isn't necessarily complete: in particular, it can miss references
from other pockets, and in any case it might race with a build still in
progress. The intent of this is not to ensure integrity, but to avoid
some easily-detectable mistakes that could cause confusion.LP: #1868558
Preview Diff
1 | diff --git a/lib/lp/archivepublisher/domination.py b/lib/lp/archivepublisher/domination.py |
2 | index a02fd78..6721c64 100644 |
3 | --- a/lib/lp/archivepublisher/domination.py |
4 | +++ b/lib/lp/archivepublisher/domination.py |
5 | @@ -1,4 +1,4 @@ |
6 | -# Copyright 2009-2019 Canonical Ltd. This software is licensed under the |
7 | +# Copyright 2009-2020 Canonical Ltd. This software is licensed under the |
8 | # GNU Affero General Public License version 3 (see the file LICENSE). |
9 | |
10 | """Archive Domination class. |
11 | @@ -68,10 +68,12 @@ from storm.expr import ( |
12 | And, |
13 | Count, |
14 | Desc, |
15 | + Or, |
16 | Select, |
17 | ) |
18 | from zope.component import getUtility |
19 | |
20 | +from lp.app.interfaces.launchpad import ILaunchpadCelebrities |
21 | from lp.registry.model.sourcepackagename import SourcePackageName |
22 | from lp.services.database.bulk import load_related |
23 | from lp.services.database.constants import UTC_NOW |
24 | @@ -82,17 +84,24 @@ from lp.services.database.sqlbase import ( |
25 | sqlvalues, |
26 | ) |
27 | from lp.services.orderingcheck import OrderingCheck |
28 | +from lp.soyuz.adapters.archivedependencies import pocket_dependencies |
29 | from lp.soyuz.enums import ( |
30 | BinaryPackageFormat, |
31 | + BinarySourceReferenceType, |
32 | PackagePublishingStatus, |
33 | ) |
34 | +from lp.soyuz.interfaces.binarysourcereference import ( |
35 | + IBinarySourceReferenceSet, |
36 | + ) |
37 | from lp.soyuz.interfaces.publishing import ( |
38 | + active_publishing_status, |
39 | inactive_publishing_status, |
40 | IPublishingSet, |
41 | ) |
42 | from lp.soyuz.model.binarypackagebuild import BinaryPackageBuild |
43 | from lp.soyuz.model.binarypackagename import BinaryPackageName |
44 | from lp.soyuz.model.binarypackagerelease import BinaryPackageRelease |
45 | +from lp.soyuz.model.binarysourcereference import BinarySourceReference |
46 | from lp.soyuz.model.publishing import ( |
47 | BinaryPackagePublishingHistory, |
48 | SourcePackagePublishingHistory, |
49 | @@ -207,19 +216,50 @@ class GeneralizedPublication: |
50 | return sorted(publications, cmp=self.compare, reverse=True) |
51 | |
52 | |
53 | -def find_live_source_versions(sorted_pubs): |
54 | - """Find versions out of Published publications that should stay live. |
55 | +def get_source_versions(source_publications): |
56 | + """List versions for sequence of `SourcePackagePublishingHistory`. |
57 | + |
58 | + :param source_publications: An iterable of |
59 | + `SourcePackagePublishingHistory`. |
60 | + :return: A list of the publications' respective versions. |
61 | + """ |
62 | + return [pub.sourcepackagerelease.version for pub in source_publications] |
63 | |
64 | - This particular notion of liveness applies to source domination: the |
65 | - latest version stays live, and that's it. |
66 | + |
67 | +def find_live_source_versions(sorted_pubs, built_using=None): |
68 | + """Find versions of source publications that should stay live. |
69 | + |
70 | + This particular notion of liveness applies to source domination: |
71 | + normally, the latest Published version stays live, and that's it. The |
72 | + exception is if live binary publications have Built-Using fields |
73 | + referring to some source versions, in which case those versions stay |
74 | + live too. |
75 | |
76 | :param sorted_pubs: An iterable of `SourcePackagePublishingHistory` |
77 | sorted by descending package version. |
78 | + :param built_using: An optional collection of `BinarySourceReference`s |
79 | + corresponding to published binary publications with Built-Using |
80 | + references; the SPRs that these refer to should stay live. |
81 | :return: A list of live versions. |
82 | """ |
83 | + if built_using is None: |
84 | + built_using = set() |
85 | + built_using_spr_ids = set( |
86 | + bsr.source_package_release_id for bsr in built_using) |
87 | + |
88 | # Given the required sort order, the latest version is at the head |
89 | # of the list. |
90 | - return [sorted_pubs[0].sourcepackagerelease.version] |
91 | + sorted_pubs = list(sorted_pubs) |
92 | + live_pubs = [] |
93 | + latest = None |
94 | + for pub in sorted_pubs: |
95 | + if latest is None and pub.status == PackagePublishingStatus.PUBLISHED: |
96 | + live_pubs.append(pub) |
97 | + latest = pub |
98 | + elif (pub != latest and |
99 | + pub.sourcepackagereleaseID in built_using_spr_ids): |
100 | + live_pubs.append(pub) |
101 | + return get_source_versions(live_pubs) |
102 | |
103 | |
104 | def get_binary_versions(binary_publications): |
105 | @@ -372,11 +412,11 @@ class Dominator: |
106 | generalization): |
107 | """Plan domination of publications for a single package. |
108 | |
109 | - The latest publication for any version in `live_versions` stays |
110 | - active. Any older publications (including older publications for |
111 | - live versions with multiple publications) are marked as superseded by |
112 | - the respective oldest live releases that are newer than the superseded |
113 | - ones. |
114 | + The latest active publication for any version in `live_versions` |
115 | + stays active. Any older active publications (including older |
116 | + publications for live versions with multiple publications) are |
117 | + marked as superseded by the respective oldest live releases that are |
118 | + newer than the superseded ones. |
119 | |
120 | Any versions that are newer than anything in `live_versions` are |
121 | marked as deleted. This should not be possible in Soyuz-native |
122 | @@ -384,11 +424,16 @@ class Dominator: |
123 | previous latest version of a package has disappeared from the Sources |
124 | list we import. |
125 | |
126 | + Inactive publications that are listed in `live_versions` are |
127 | + reinstated. |
128 | + |
129 | :param sorted_pubs: A list of publications for the same package, |
130 | - in the same archive, series, and pocket, all with status |
131 | - `PackagePublishingStatus.PUBLISHED`. They must be sorted from |
132 | - most current to least current, as would be the result of |
133 | - `generalization.sortPublications`. |
134 | + in the same archive, series, and pocket. These will normally |
135 | + all have status `PackagePublishingStatus.PUBLISHED`, but source |
136 | + publications referenced by Built-Using may reach here with |
137 | + different statuses and be considered for reinstatement. They |
138 | + must be sorted from most current to least current, as would be |
139 | + the result of `generalization.sortPublications`. |
140 | :param live_versions: Iterable of versions that are still considered |
141 | "live" for this package. For any of these, the latest publication |
142 | among `publications` will remain Published. Publications for |
143 | @@ -434,28 +479,38 @@ class Dominator: |
144 | # This publication is for a live version, but has been |
145 | # superseded by a newer publication of the same version. |
146 | # Supersede it. |
147 | - supersede.append((pub, current_dominant)) |
148 | - self.logger.debug2( |
149 | - "Superseding older publication for version %s.", version) |
150 | + if pub.status in active_publishing_status: |
151 | + supersede.append((pub, current_dominant)) |
152 | + self.logger.debug2( |
153 | + "Superseding older publication for version %s.", |
154 | + version) |
155 | elif version in live_versions: |
156 | - # This publication stays active; if any publications |
157 | - # that follow right after this are to be superseded, |
158 | - # this is the release that they are superseded by. |
159 | - current_dominant = pub |
160 | - dominant_version = version |
161 | - keep.add(pub) |
162 | - self.logger.debug2("Keeping version %s.", version) |
163 | + if pub.status in active_publishing_status: |
164 | + # This publication stays active; if any publications |
165 | + # that follow right after this are to be superseded, |
166 | + # this is the release that they are superseded by. |
167 | + current_dominant = pub |
168 | + dominant_version = version |
169 | + keep.add(pub) |
170 | + self.logger.debug2("Keeping version %s.", version) |
171 | + else: |
172 | + # This publication is currently inactive, but is |
173 | + # referenced by an active publication. Reinstate it. |
174 | + keep.add(pub) |
175 | + self.logger.debug2("Reinstating version %s.", version) |
176 | elif current_dominant is None: |
177 | # This publication is no longer live, but there is no |
178 | # newer version to supersede it either. Therefore it |
179 | # must be deleted. |
180 | - delete.append(pub) |
181 | - self.logger.debug2("Deleting version %s.", version) |
182 | + if pub.status in active_publishing_status: |
183 | + delete.append(pub) |
184 | + self.logger.debug2("Deleting version %s.", version) |
185 | else: |
186 | # This publication is superseded. This is what we're |
187 | # here to do. |
188 | - supersede.append((pub, current_dominant)) |
189 | - self.logger.debug2("Superseding version %s.", version) |
190 | + if pub.status in active_publishing_status: |
191 | + supersede.append((pub, current_dominant)) |
192 | + self.logger.debug2("Superseding version %s.", version) |
193 | |
194 | return supersede, keep, delete |
195 | |
196 | @@ -707,19 +762,22 @@ class Dominator: |
197 | |
198 | execute_plan() |
199 | |
200 | - def _composeActiveSourcePubsCondition(self, distroseries, pocket): |
201 | + def _composeRelevantSourcePubsCondition(self, distroseries, pocket, |
202 | + active=True): |
203 | """Compose ORM condition for restricting relevant source pubs.""" |
204 | SPPH = SourcePackagePublishingHistory |
205 | |
206 | - return And( |
207 | - SPPH.status == PackagePublishingStatus.PUBLISHED, |
208 | + clauses = [ |
209 | SPPH.distroseries == distroseries, |
210 | SPPH.archive == self.archive, |
211 | SPPH.pocket == pocket, |
212 | - ) |
213 | + ] |
214 | + if active: |
215 | + clauses.append(SPPH.status == PackagePublishingStatus.PUBLISHED) |
216 | + return And(*clauses) |
217 | |
218 | def findSourcesForDomination(self, distroseries, pocket): |
219 | - """Find binary publications that need dominating. |
220 | + """Find source publications that need dominating. |
221 | |
222 | This is only for traditional domination, where the latest published |
223 | publication is always kept published. See `find_live_source_versions` |
224 | @@ -728,17 +786,32 @@ class Dominator: |
225 | To optimize for that logic, `findSourcesForDomination` will ignore |
226 | publications that have no other publications competing for the same |
227 | binary package. There'd be nothing to do for those cases. |
228 | + |
229 | + This also includes source publications whose source package releases |
230 | + have a Built-Using reference pointing to them, whether active or |
231 | + not. These may need to be superseded or kept/reinstated depending |
232 | + on whether binary publications referring to them are active. |
233 | """ |
234 | SPPH = SourcePackagePublishingHistory |
235 | SPR = SourcePackageRelease |
236 | + BSR = BinarySourceReference |
237 | |
238 | - spph_location_clauses = self._composeActiveSourcePubsCondition( |
239 | + spph_location_clauses = self._composeRelevantSourcePubsCondition( |
240 | distroseries, pocket) |
241 | candidate_source_names = Select( |
242 | SPPH.sourcepackagenameID, |
243 | And(join_spph_spr(), spph_location_clauses), |
244 | group_by=SPPH.sourcepackagenameID, |
245 | having=(Count() > 1)) |
246 | + built_using_spph_location_clauses = ( |
247 | + self._composeRelevantSourcePubsCondition( |
248 | + distroseries, pocket, active=False)) |
249 | + built_using_sprs = Select( |
250 | + SPPH.sourcepackagereleaseID, |
251 | + And( |
252 | + SPPH.sourcepackagereleaseID == BSR.source_package_release_id, |
253 | + BSR.reference_type == BinarySourceReferenceType.BUILT_USING, |
254 | + built_using_spph_location_clauses)) |
255 | |
256 | # We'll also access the SourcePackageReleases associated with |
257 | # the publications we find. Since they're in the join anyway, |
258 | @@ -749,8 +822,13 @@ class Dominator: |
259 | query = IStore(SPPH).find( |
260 | (SPPH, SPR), |
261 | join_spph_spr(), |
262 | - SPPH.sourcepackagenameID.is_in(candidate_source_names), |
263 | - spph_location_clauses) |
264 | + Or( |
265 | + And( |
266 | + SPPH.sourcepackagenameID.is_in(candidate_source_names), |
267 | + spph_location_clauses), |
268 | + And( |
269 | + SPPH.sourcepackagereleaseID.is_in(built_using_sprs), |
270 | + built_using_spph_location_clauses))) |
271 | spphs = DecoratedResultSet(query, itemgetter(0)) |
272 | load_related(SourcePackageName, spphs, ['sourcepackagenameID']) |
273 | return spphs |
274 | @@ -771,20 +849,57 @@ class Dominator: |
275 | sources = self.findSourcesForDomination(distroseries, pocket) |
276 | sorted_packages = self._sortPackages(sources, generalization) |
277 | supersede = [] |
278 | + keep = set() |
279 | delete = [] |
280 | |
281 | + # Of the SPRs associated with publications being considered for |
282 | + # domination, find those that have a Built-Using reference pointing |
283 | + # to them from a live binary publication. |
284 | + bsr_set = getUtility(IBinarySourceReferenceSet) |
285 | + reverse_pockets = { |
286 | + source_pocket |
287 | + for source_pocket, expanded_pockets in pocket_dependencies.items() |
288 | + if pocket in expanded_pockets} |
289 | + built_using = bsr_set.findPublished( |
290 | + self.archive, distroseries, reverse_pockets, |
291 | + BinarySourceReferenceType.BUILT_USING, |
292 | + source_package_releases=[ |
293 | + pub.sourcepackagerelease for pub in sources]) |
294 | + |
295 | self.logger.debug("Dominating sources...") |
296 | for name, pubs in sorted_packages.iteritems(): |
297 | self.logger.debug("Dominating %s" % name) |
298 | assert len(pubs) > 0, "Dominating zero sources!" |
299 | - live_versions = find_live_source_versions(pubs) |
300 | - cur_supersede, _, cur_delete = self.planPackageDomination( |
301 | + live_versions = find_live_source_versions(pubs, built_using) |
302 | + cur_supersede, cur_keep, cur_delete = self.planPackageDomination( |
303 | pubs, live_versions, generalization) |
304 | supersede.extend(cur_supersede) |
305 | + keep.update(cur_keep) |
306 | delete.extend(cur_delete) |
307 | |
308 | for pub, dominant in supersede: |
309 | pub.supersede(dominant, logger=self.logger) |
310 | + for pub in keep: |
311 | + if pub.status not in active_publishing_status: |
312 | + # The dominator thinks that we should keep this package, but |
313 | + # it isn't currently published. This can happen when a |
314 | + # source package has initially been superseded, but then a |
315 | + # binary package that references it in Built-Using is |
316 | + # published, requiring the source package to be reinstated. |
317 | + # To cope with this, we'll copy the source back into place, |
318 | + # thereby creating a new PENDING publication record for it, |
319 | + # which will cause the publisher to put it back on disk. |
320 | + # |
321 | + # In general we'd prefer that the dominator not undo manual |
322 | + # deletions, but Built-Using often expresses a compliance |
323 | + # requirement, so we intentionally don't check that here. |
324 | + # Instead, SPPH.requestDeletion checks for live |
325 | + # BinarySourceReferences, and the uploader rejects uploads |
326 | + # of builds with Built-Using references to source packages |
327 | + # that have been manually deleted. |
328 | + pub.copyTo( |
329 | + distroseries, pocket, self.archive, |
330 | + creator=getUtility(ILaunchpadCelebrities).janitor) |
331 | for pub in delete: |
332 | pub.requestDeletion(None) |
333 | |
334 | @@ -804,7 +919,7 @@ class Dominator: |
335 | looking_for, |
336 | join_spph_spr(), |
337 | join_spph_spn(), |
338 | - self._composeActiveSourcePubsCondition(distroseries, pocket)) |
339 | + self._composeRelevantSourcePubsCondition(distroseries, pocket)) |
340 | return result.group_by(SourcePackageName.name) |
341 | |
342 | def findPublishedSPPHs(self, distroseries, pocket, package_name): |
343 | @@ -817,7 +932,7 @@ class Dominator: |
344 | join_spph_spr(), |
345 | join_spph_spn(), |
346 | SourcePackageName.name == package_name, |
347 | - self._composeActiveSourcePubsCondition(distroseries, pocket)) |
348 | + self._composeRelevantSourcePubsCondition(distroseries, pocket)) |
349 | # Sort by descending version (SPR.version has type debversion in |
350 | # the database, so this should be a real proper comparison) so |
351 | # that _sortPackage will have slightly less work to do later. |
352 | diff --git a/lib/lp/archivepublisher/tests/test_dominator.py b/lib/lp/archivepublisher/tests/test_dominator.py |
353 | index b88da56..7db5336 100755 |
354 | --- a/lib/lp/archivepublisher/tests/test_dominator.py |
355 | +++ b/lib/lp/archivepublisher/tests/test_dominator.py |
356 | @@ -1,4 +1,4 @@ |
357 | -# Copyright 2009-2019 Canonical Ltd. This software is licensed under the |
358 | +# Copyright 2009-2020 Canonical Ltd. This software is licensed under the |
359 | # GNU Affero General Public License version 3 (see the file LICENSE). |
360 | |
361 | """Tests for domination.py.""" |
362 | @@ -14,6 +14,8 @@ import apt_pkg |
363 | from testtools.matchers import ( |
364 | GreaterThan, |
365 | LessThan, |
366 | + MatchesSetwise, |
367 | + MatchesStructure, |
368 | ) |
369 | import transaction |
370 | from zope.component import getUtility |
371 | @@ -397,6 +399,186 @@ class TestDominator(TestNativePublishingBase): |
372 | for pub in overrides_2: |
373 | self.assertEqual(PackagePublishingStatus.PUBLISHED, pub.status) |
374 | |
375 | + def test_dominateSources_keeps_built_using_refs(self): |
376 | + # If a source publication has a Built-Using reference from an active |
377 | + # binary publication in the same archive and series and in a pocket |
378 | + # that could legitimately refer to it, it is retained. |
379 | + foo_10_src = self.getPubSource( |
380 | + sourcename="foo", version="1.0", architecturehintlist="i386", |
381 | + status=PackagePublishingStatus.PUBLISHED) |
382 | + [foo_10_i386_bin] = self.getPubBinaries( |
383 | + binaryname="foo-bin", status=PackagePublishingStatus.PUBLISHED, |
384 | + architecturespecific=True, version="1.0", pub_source=foo_10_src) |
385 | + foo_11_src = self.getPubSource( |
386 | + sourcename="foo", version="1.1", architecturehintlist="i386", |
387 | + status=PackagePublishingStatus.PUBLISHED) |
388 | + [foo_11_i386_bin] = self.getPubBinaries( |
389 | + binaryname="foo-bin", status=PackagePublishingStatus.PUBLISHED, |
390 | + architecturespecific=True, version="1.1", pub_source=foo_11_src) |
391 | + bar_10_src = self.getPubSource( |
392 | + sourcename="bar", version="1.0", architecturehintlist="i386", |
393 | + status=PackagePublishingStatus.PUBLISHED) |
394 | + [bar_10_i386_bin] = self.getPubBinaries( |
395 | + binaryname="bar-bin", status=PackagePublishingStatus.PUBLISHED, |
396 | + architecturespecific=True, version="1.0", pub_source=bar_10_src, |
397 | + built_using="foo (= 1.0)") |
398 | + |
399 | + dominator = Dominator(self.logger, self.ubuntutest.main_archive) |
400 | + dominator.judgeAndDominate(foo_10_src.distroseries, foo_10_src.pocket) |
401 | + |
402 | + self.checkPublications( |
403 | + [foo_10_src, |
404 | + foo_11_src, foo_11_i386_bin, |
405 | + bar_10_src, bar_10_i386_bin], |
406 | + PackagePublishingStatus.PUBLISHED) |
407 | + self.checkPublication( |
408 | + foo_10_i386_bin, PackagePublishingStatus.SUPERSEDED) |
409 | + |
410 | + # No copies were performed. |
411 | + pending_srcs = foo_10_src.archive.getPublishedSources( |
412 | + status=PackagePublishingStatus.PENDING) |
413 | + pending_bins = foo_10_src.archive.getAllPublishedBinaries( |
414 | + status=PackagePublishingStatus.PENDING) |
415 | + self.assertEqual(0, pending_srcs.count()) |
416 | + self.assertEqual(0, pending_bins.count()) |
417 | + |
418 | + bar_11_src = self.getPubSource( |
419 | + sourcename="bar", version="1.1", architecturehintlist="i386", |
420 | + status=PackagePublishingStatus.PUBLISHED) |
421 | + [bar_11_i386_bin] = self.getPubBinaries( |
422 | + binaryname="bar-bin", status=PackagePublishingStatus.PUBLISHED, |
423 | + architecturespecific=True, version="1.1", pub_source=bar_11_src, |
424 | + built_using="foo (= 1.1)") |
425 | + |
426 | + # Dominate twice to make sure the result is stable. |
427 | + for _ in range(2): |
428 | + dominator.judgeAndDominate( |
429 | + foo_10_src.distroseries, foo_10_src.pocket) |
430 | + |
431 | + self.checkPublications( |
432 | + [foo_11_src, foo_11_i386_bin, |
433 | + bar_11_src, bar_11_i386_bin], |
434 | + PackagePublishingStatus.PUBLISHED) |
435 | + self.checkPublications( |
436 | + [foo_10_src, foo_10_i386_bin, |
437 | + bar_10_src, bar_10_i386_bin], |
438 | + PackagePublishingStatus.SUPERSEDED) |
439 | + |
440 | + # No copies were performed. |
441 | + pending_srcs = foo_10_src.archive.getPublishedSources( |
442 | + status=PackagePublishingStatus.PENDING) |
443 | + pending_bins = foo_10_src.archive.getAllPublishedBinaries( |
444 | + status=PackagePublishingStatus.PENDING) |
445 | + self.assertEqual(0, pending_srcs.count()) |
446 | + self.assertEqual(0, pending_bins.count()) |
447 | + |
448 | + def test_dominateSources_reinstates_superseded_built_using_refs(self): |
449 | + # If a superseded source publication has a Built-Using reference |
450 | + # from an active binary publication in the same archive and series |
451 | + # and in a pocket that could legitimately refer to it, it is |
452 | + # reinstated. |
453 | + foo_10_src = self.getPubSource( |
454 | + sourcename="foo", version="1.0", architecturehintlist="i386", |
455 | + status=PackagePublishingStatus.PUBLISHED) |
456 | + [foo_10_i386_bin] = self.getPubBinaries( |
457 | + binaryname="foo-bin", status=PackagePublishingStatus.PUBLISHED, |
458 | + architecturespecific=True, version="1.0", pub_source=foo_10_src) |
459 | + foo_11_src = self.getPubSource( |
460 | + sourcename="foo", version="1.1", architecturehintlist="i386", |
461 | + status=PackagePublishingStatus.PUBLISHED) |
462 | + [foo_11_i386_bin] = self.getPubBinaries( |
463 | + binaryname="foo-bin", status=PackagePublishingStatus.PUBLISHED, |
464 | + architecturespecific=True, version="1.1", pub_source=foo_11_src) |
465 | + bar_10_src = self.getPubSource( |
466 | + sourcename="bar", version="1.0", architecturehintlist="i386", |
467 | + status=PackagePublishingStatus.PUBLISHED, |
468 | + pocket=PackagePublishingPocket.PROPOSED) |
469 | + |
470 | + dominator = Dominator(self.logger, self.ubuntutest.main_archive) |
471 | + dominator.judgeAndDominate(foo_10_src.distroseries, foo_10_src.pocket) |
472 | + |
473 | + self.checkPublications( |
474 | + [foo_11_src, foo_11_i386_bin, |
475 | + bar_10_src], |
476 | + PackagePublishingStatus.PUBLISHED) |
477 | + self.checkPublications( |
478 | + [foo_10_src, foo_10_i386_bin], PackagePublishingStatus.SUPERSEDED) |
479 | + |
480 | + [bar_10_i386_bin] = self.getPubBinaries( |
481 | + binaryname="bar-bin", status=PackagePublishingStatus.PUBLISHED, |
482 | + pocket=PackagePublishingPocket.PROPOSED, architecturespecific=True, |
483 | + version="1.0", pub_source=bar_10_src, built_using="foo (= 1.0)") |
484 | + |
485 | + # Dominate twice to make sure the result is stable. |
486 | + for _ in range(2): |
487 | + dominator.judgeAndDominate( |
488 | + foo_10_src.distroseries, foo_10_src.pocket) |
489 | + |
490 | + self.checkPublications( |
491 | + [foo_11_src, foo_11_i386_bin, |
492 | + bar_10_src, bar_10_i386_bin], |
493 | + PackagePublishingStatus.PUBLISHED) |
494 | + self.checkPublications( |
495 | + [foo_10_src, foo_10_i386_bin], |
496 | + PackagePublishingStatus.SUPERSEDED) |
497 | + |
498 | + # The foo 1.0 source was reinstated. |
499 | + pending_srcs = foo_10_src.archive.getPublishedSources( |
500 | + status=PackagePublishingStatus.PENDING) |
501 | + pending_bins = foo_10_src.archive.getAllPublishedBinaries( |
502 | + status=PackagePublishingStatus.PENDING) |
503 | + self.assertThat(pending_srcs, MatchesSetwise( |
504 | + MatchesStructure( |
505 | + sourcepackagerelease=MatchesStructure.byEquality( |
506 | + name="foo", version="1.0")))) |
507 | + self.assertEqual(0, pending_bins.count()) |
508 | + |
509 | + def test_dominateSources_skips_inactive_built_using_refs(self): |
510 | + # While inactive source publications are considered for |
511 | + # reinstatement if they are referenced by Built-Using from any |
512 | + # binary publication, they are only actually reinstated if they are |
513 | + # referenced from an *active* binary publication; merely being the |
514 | + # most recent publication of the source package in question is not |
515 | + # enough. |
516 | + foo_10_src = self.getPubSource( |
517 | + sourcename="foo", version="1.0", architecturehintlist="i386", |
518 | + status=PackagePublishingStatus.SUPERSEDED) |
519 | + foo_11_src = self.getPubSource( |
520 | + sourcename="foo", version="1.1", architecturehintlist="i386", |
521 | + status=PackagePublishingStatus.DELETED) |
522 | + bar_10_src = self.getPubSource( |
523 | + sourcename="bar", version="1.0", architecturehintlist="i386", |
524 | + status=PackagePublishingStatus.SUPERSEDED) |
525 | + [bar_10_i386_bin] = self.getPubBinaries( |
526 | + binaryname="bar-bin", status=PackagePublishingStatus.SUPERSEDED, |
527 | + architecturespecific=True, version="1.0", pub_source=bar_10_src, |
528 | + built_using="foo (= 1.0)") |
529 | + bar_11_src = self.getPubSource( |
530 | + sourcename="bar", version="1.1", architecturehintlist="i386", |
531 | + status=PackagePublishingStatus.SUPERSEDED) |
532 | + [bar_11_i386_bin] = self.getPubBinaries( |
533 | + binaryname="bar-bin", status=PackagePublishingStatus.SUPERSEDED, |
534 | + architecturespecific=True, version="1.1", pub_source=bar_11_src, |
535 | + built_using="foo (= 1.1)") |
536 | + |
537 | + dominator = Dominator(self.logger, self.ubuntutest.main_archive) |
538 | + dominator.judgeAndDominate(foo_10_src.distroseries, foo_10_src.pocket) |
539 | + |
540 | + self.checkPublications( |
541 | + [foo_10_src, |
542 | + bar_10_src, bar_10_i386_bin, |
543 | + bar_11_src, bar_11_i386_bin], |
544 | + PackagePublishingStatus.SUPERSEDED) |
545 | + self.checkPublications([foo_11_src], PackagePublishingStatus.DELETED) |
546 | + |
547 | + # No copies were performed. |
548 | + pending_srcs = foo_10_src.archive.getPublishedSources( |
549 | + status=PackagePublishingStatus.PENDING) |
550 | + pending_bins = foo_10_src.archive.getAllPublishedBinaries( |
551 | + status=PackagePublishingStatus.PENDING) |
552 | + self.assertEqual(0, pending_srcs.count()) |
553 | + self.assertEqual(0, pending_bins.count()) |
554 | + |
555 | |
556 | class TestDomination(TestNativePublishingBase): |
557 | """Test overall domination procedure.""" |
558 | diff --git a/lib/lp/soyuz/interfaces/binarysourcereference.py b/lib/lp/soyuz/interfaces/binarysourcereference.py |
559 | index e625cf9..d110256 100644 |
560 | --- a/lib/lp/soyuz/interfaces/binarysourcereference.py |
561 | +++ b/lib/lp/soyuz/interfaces/binarysourcereference.py |
562 | @@ -47,6 +47,14 @@ class IBinarySourceReference(Interface): |
563 | vocabulary=BinarySourceReferenceType, |
564 | required=True, readonly=True) |
565 | |
566 | + # Export IDs for use by the dominator. |
567 | + binary_package_release_id = Int( |
568 | + title=_("The referencing binary package release ID."), |
569 | + required=True, readonly=True) |
570 | + source_package_release_id = Int( |
571 | + title=_("The referencing source package release ID."), |
572 | + required=True, readonly=True) |
573 | + |
574 | |
575 | class IBinarySourceReferenceSet(Interface): |
576 | """A set of references from binary packages to source packages.""" |
I'm reasonably sure that it would be possible to extend this to cover the self-reference case (any active binary publications should keep their parent source publication active too) with minimal trouble, but I've already spent quite a long time on this so haven't attempted to actually do that extension here.