Merge lp:~jtv/launchpad/bug-845326 into lp:launchpad

Proposed by Jeroen T. Vermeulen
Status: Superseded
Proposed branch: lp:~jtv/launchpad/bug-845326
Merge into: lp:launchpad
Diff against target: 1644 lines (+1123/-161)
9 files modified
lib/lp/archivepublisher/domination.py (+244/-60)
lib/lp/archivepublisher/tests/test_dominator.py (+550/-4)
lib/lp/soyuz/doc/gina.txt (+24/-24)
lib/lp/soyuz/interfaces/publishing.py (+0/-3)
lib/lp/soyuz/model/publishing.py (+6/-10)
lib/lp/soyuz/scripts/gina/dominate.py (+82/-0)
lib/lp/soyuz/scripts/gina/handlers.py (+37/-51)
lib/lp/soyuz/scripts/tests/test_gina.py (+171/-3)
scripts/gina.py (+9/-6)
To merge this branch: bzr merge lp:~jtv/launchpad/bug-845326
Reviewer Review Type Date Requested Status
Launchpad code reviewers Pending
Review via email: mp+74740@code.launchpad.net

This proposal has been superseded by a proposal from 2011-09-09.

Commit message

Make newer [BS]PPHs supersede older ones for same version.

Description of the change

= Summary =

I'm re-doing the domination process so that one Dominator can serve the needs of traditional, latest-publication-record-wins domination; versions-we-see-in-Sources-lists-live domination as needed in Gina; and the as-yet unspecified multiple-versions-can-live-side-by-side domination we are supposed to get in the future.

My code for this was not a full replacement for traditional domination. A single package release can have multiple active publications in the same archive, distroseries, and pocket (e.g. when the package is being re-published into a different component) and traditional domination would mark the newer publication as superseding the older one. The challenge for this branch was to fix that, in a way that was still general enough to serve Gina's needs. All that Gina really knows is what version numbers should survive.

== Proposed fix ==

Extend the new-style domination algorithm: when domination finds multiple publication records for the same package (and archive etc.), for a version that should stay published, then have the newest one supersede the older ones.

(My previous generalized code would keep all publications for the version published. The classic dominator code would only keep the newest publication of the very latest version published and supersede all older ones.)

== Pre-implementation notes ==

William seems to think the solution is sane. As far as we're aware right now, this is the last step before we can land and deploy transitional Gina domination.

== Implementation details ==

The fix itself is fairly straightforward. It's in the first file in the diff. It involves one redundant variable and an extra case in an if/elif cadence, but overall I think it works out pretty cleanly.

== Tests ==

Besides a unit test for the new behaviour, for good measure I also added a massive test for complex combined data. This is not meant to replace proper unit tests; it's too complex for that. But it may reveal any breakage that the unit tests might miss.

{{{
./bin/test -vvc lp.archivepublisher.tests.test_dominator
}}}

== Demo and Q/A ==

A bunch of branches are going to land together. We'll have to make sure that domination still works; that Gina still works; and that Gina now does proper domination.

= Launchpad lint =

There's some pre-existing lint in the dependent branches that either can't be fixed, or would increase the risk of conflicts too much. I did not create any lint of my own, however, and left less than I found.

Checking for conflicts and issues in changed files.

Linting changed files:
  lib/lp/archivepublisher/domination.py
  lib/lp/soyuz/scripts/gina/dominate.py
  lib/lp/soyuz/model/publishing.py
  lib/lp/soyuz/doc/gina.txt
  lib/lp/soyuz/interfaces/publishing.py
  scripts/gina.py
  lib/lp/archivepublisher/tests/test_dominator.py
  lib/lp/soyuz/scripts/tests/test_gina.py
  lib/lp/soyuz/scripts/gina/handlers.py

./lib/lp/soyuz/doc/gina.txt
     113: narrative exceeds 78 characters.
     162: want exceeds 78 characters.
     179: want exceeds 78 characters.
     189: narrative uses a moin header.
     221: want exceeds 78 characters.
     234: want exceeds 78 characters.
     240: want exceeds 78 characters.
     295: source exceeds 78 characters.
     324: narrative uses a moin header.
     342: narrative exceeds 78 characters.
     354: narrative uses a moin header.
     360: narrative exceeds 78 characters.
     361: narrative exceeds 78 characters.
     459: narrative uses a moin header.
     461: narrative exceeds 78 characters.
     462: narrative exceeds 78 characters.
     477: narrative uses a moin header.
     563: narrative exceeds 78 characters.
     600: narrative uses a moin header.
     657: narrative uses a moin header.
     746: narrative uses a moin header.
     767: narrative uses a moin header.
     780: narrative uses a moin header.
./lib/lp/soyuz/interfaces/publishing.py
     381: E261 at least two spaces before inline comment
     478: E261 at least two spaces before inline comment
     511: E261 at least two spaces before inline comment
     681: E261 at least two spaces before inline comment
     767: E261 at least two spaces before inline comment
./scripts/gina.py
      26: '_pythonpath' imported but unused

To post a comment you must log in.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'lib/lp/archivepublisher/domination.py'
2--- lib/lp/archivepublisher/domination.py 2011-08-30 06:37:55 +0000
3+++ lib/lp/archivepublisher/domination.py 2011-09-09 09:33:30 +0000
4@@ -53,8 +53,6 @@
5 __all__ = ['Dominator']
6
7 from datetime import timedelta
8-import functools
9-import operator
10
11 import apt_pkg
12 from storm.expr import (
13@@ -68,7 +66,7 @@
14 flush_database_updates,
15 sqlvalues,
16 )
17-from canonical.launchpad.interfaces.lpstorm import IMasterStore
18+from canonical.launchpad.interfaces.lpstorm import IStore
19 from lp.registry.model.sourcepackagename import SourcePackageName
20 from lp.soyuz.enums import (
21 BinaryPackageFormat,
22@@ -87,17 +85,93 @@
23 apt_pkg.InitSystem()
24
25
26-def _compare_packages_by_version_and_date(get_release, p1, p2):
27- """Compare publications p1 and p2 by their version; using Debian rules.
28-
29- If the publications are for the same package, compare by datecreated
30- instead. This lets newer records win.
31- """
32- if get_release(p1).id == get_release(p2).id:
33- return cmp(p1.datecreated, p2.datecreated)
34-
35- return apt_pkg.VersionCompare(get_release(p1).version,
36- get_release(p2).version)
37+def join_spr_spn():
38+ """Join condition: SourcePackageRelease/SourcePackageName."""
39+ return (
40+ SourcePackageName.id == SourcePackageRelease.sourcepackagenameID)
41+
42+
43+def join_spph_spr():
44+ """Join condition: SourcePackageRelease/SourcePackagePublishingHistory.
45+ """
46+ # Avoid circular imports.
47+ from lp.soyuz.model.publishing import SourcePackagePublishingHistory
48+
49+ return (
50+ SourcePackageRelease.id ==
51+ SourcePackagePublishingHistory.sourcepackagereleaseID)
52+
53+
54+class SourcePublicationTraits:
55+ """Basic generalized attributes for `SourcePackagePublishingHistory`.
56+
57+ Used by `GeneralizedPublication` to hide the differences from
58+ `BinaryPackagePublishingHistory`.
59+ """
60+ @staticmethod
61+ def getPackageName(spph):
62+ """Return the name of this publication's source package."""
63+ return spph.sourcepackagerelease.sourcepackagename.name
64+
65+ @staticmethod
66+ def getPackageRelease(spph):
67+ """Return this publication's `SourcePackageRelease`."""
68+ return spph.sourcepackagerelease
69+
70+
71+class BinaryPublicationTraits:
72+ """Basic generalized attributes for `BinaryPackagePublishingHistory`.
73+
74+ Used by `GeneralizedPublication` to hide the differences from
75+ `SourcePackagePublishingHistory`.
76+ """
77+ @staticmethod
78+ def getPackageName(bpph):
79+ """Return the name of this publication's binary package."""
80+ return bpph.binarypackagerelease.binarypackagename.name
81+
82+ @staticmethod
83+ def getPackageRelease(bpph):
84+ """Return this publication's `BinaryPackageRelease`."""
85+ return bpph.binarypackagerelease
86+
87+
88+class GeneralizedPublication:
89+ """Generalize handling of publication records.
90+
91+ This allows us to write code that can be dealing with either
92+ `SourcePackagePublishingHistory`s or `BinaryPackagePublishingHistory`s
93+ without caring which. Differences are abstracted away in a traits
94+ class.
95+ """
96+ def __init__(self, is_source=True):
97+ if is_source:
98+ self.traits = SourcePublicationTraits
99+ else:
100+ self.traits = BinaryPublicationTraits
101+
102+ def getPackageName(self, pub):
103+ """Get the package's name."""
104+ return self.traits.getPackageName(pub)
105+
106+ def getPackageVersion(self, pub):
107+ """Obtain the version string for a publicaiton record."""
108+ return self.traits.getPackageRelease(pub).version
109+
110+ def compare(self, pub1, pub2):
111+ """Compare publications by version.
112+
113+ If both publications are for the same version, their creation dates
114+ break the tie.
115+ """
116+ version_comparison = apt_pkg.VersionCompare(
117+ self.getPackageVersion(pub1), self.getPackageVersion(pub2))
118+
119+ if version_comparison == 0:
120+ # Use dates as tie breaker.
121+ return cmp(pub1.datecreated, pub2.datecreated)
122+ else:
123+ return version_comparison
124
125
126 class Dominator:
127@@ -116,50 +190,106 @@
128 self.logger = logger
129 self.archive = archive
130
131- def _dominatePublications(self, pubs):
132+ def dominatePackage(self, publications, live_versions, generalization):
133+ """Dominate publications for a single package.
134+
135+ The latest publication for any version in `live_versions` stays
136+ active. Any older publications (including older publications for
137+ live versions with multiple publications) are marked as superseded by
138+ the respective oldest live releases that are newer than the superseded
139+ ones.
140+
141+ Any versions that are newer than anything in `live_versions` are
142+ marked as deleted. This should not be possible in Soyuz-native
143+ archives, but it can happen during archive imports when the
144+ previous latest version of a package has disappeared from the Sources
145+ list we import.
146+
147+ :param publications: Iterable of publications for the same package,
148+ in the same archive, series, and pocket, all with status
149+ `PackagePublishingStatus.PUBLISHED`.
150+ :param live_versions: Iterable of version strings that are still
151+ considered live for this package. The given publications will
152+ remain active insofar as they represent any of these versions;
153+ older publications will be marked as superseded and newer ones
154+ as deleted.
155+ :param generalization: A `GeneralizedPublication` helper representing
156+ the kind of publications these are--source or binary.
157+ """
158+ # Go through publications from latest version to oldest. This
159+ # makes it easy to figure out which release superseded which:
160+ # the dominant is always the oldest live release that is newer
161+ # than the one being superseded.
162+ publications = sorted(
163+ publications, cmp=generalization.compare, reverse=True)
164+
165+ current_dominant = None
166+ dominant_version = None
167+
168+ for pub in publications:
169+ version = generalization.getPackageVersion(pub)
170+ if dominant_version is not None and version == dominant_version:
171+ # This publication is for a live version, but has been
172+ # superseded by a newer publication of the same version.
173+ # Supersede it.
174+ pub.supersede(current_dominant, logger=self.logger)
175+ elif version in live_versions:
176+ # This publication stays active; if any publications
177+ # that follow right after this are to be superseded,
178+ # this is the release that they are superseded by.
179+ current_dominant = pub
180+ dominant_version = version
181+ elif current_dominant is None:
182+ # This publication is no longer live, but there is no
183+ # newer version to supersede it either. Therefore it
184+ # must be deleted.
185+ pub.requestDeletion(None)
186+ else:
187+ # This publication is superseded. This is what we're
188+ # here to do.
189+ pub.supersede(current_dominant, logger=self.logger)
190+
191+ def _dominatePublications(self, pubs, generalization):
192 """Perform dominations for the given publications.
193
194+ Keep the latest published version for each package active,
195+ superseding older versions.
196+
197 :param pubs: A dict mapping names to a list of publications. Every
198 publication must be PUBLISHED or PENDING, and the first in each
199 list will be treated as dominant (so should be the latest).
200+ :param generalization: A `GeneralizedPublication` helper representing
201+ the kind of publications these are--source or binary.
202 """
203 self.logger.debug("Dominating packages...")
204-
205- for name in pubs.keys():
206- assert pubs[name], (
207- "Empty list of publications for %s" % name)
208- for pubrec in pubs[name][1:]:
209- pubrec.supersede(pubs[name][0], logger=self.logger)
210-
211- def _sortPackages(self, pkglist, is_source=True):
212+ for name, publications in pubs.iteritems():
213+ assert publications, "Empty list of publications for %s." % name
214+ # Since this always picks the latest version as the live
215+ # one, this dominatePackage call will never result in a
216+ # deletion.
217+ latest_version = generalization.getPackageVersion(publications[0])
218+ self.dominatePackage(
219+ publications, [latest_version], generalization)
220+
221+ def _sortPackages(self, pkglist, generalization):
222 """Map out packages by name, and sort by descending version.
223
224 :param pkglist: An iterable of `SourcePackagePublishingHistory` or
225 `BinaryPackagePublishingHistory`.
226- :param is_source: Whether this call involves source package
227- publications. If so, work with `SourcePackagePublishingHistory`.
228- If not, work with `BinaryPackagepublishingHistory`.
229- :return: A dict mapping each package name (as UTF-8 encoded string)
230- to a list of publications from `pkglist`, newest first.
231+ :param generalization: A `GeneralizedPublication` helper representing
232+ the kind of publications these are--source or binary.
233+ :return: A dict mapping each package name to a list of publications
234+ from `pkglist`, newest first.
235 """
236 self.logger.debug("Sorting packages...")
237
238- if is_source:
239- get_release = operator.attrgetter("sourcepackagerelease")
240- get_name = operator.attrgetter("sourcepackagename")
241- else:
242- get_release = operator.attrgetter("binarypackagerelease")
243- get_name = operator.attrgetter("binarypackagename")
244-
245 outpkgs = {}
246 for inpkg in pkglist:
247- key = get_name(get_release(inpkg)).name.encode('utf-8')
248+ key = generalization.getPackageName(inpkg)
249 outpkgs.setdefault(key, []).append(inpkg)
250
251- sort_order = functools.partial(
252- _compare_packages_by_version_and_date, get_release)
253 for package_pubs in outpkgs.itervalues():
254- package_pubs.sort(cmp=sort_order, reverse=True)
255+ package_pubs.sort(cmp=generalization.compare, reverse=True)
256
257 return outpkgs
258
259@@ -287,6 +417,8 @@
260 # Avoid circular imports.
261 from lp.soyuz.model.publishing import BinaryPackagePublishingHistory
262
263+ generalization = GeneralizedPublication(is_source=False)
264+
265 for distroarchseries in distroseries.architectures:
266 self.logger.debug(
267 "Performing domination across %s/%s (%s)",
268@@ -312,7 +444,7 @@
269 ),
270 group_by=BinaryPackageName.id,
271 having=Count(BinaryPackagePublishingHistory.id) > 1)
272- binaries = IMasterStore(BinaryPackagePublishingHistory).find(
273+ binaries = IStore(BinaryPackagePublishingHistory).find(
274 BinaryPackagePublishingHistory,
275 BinaryPackageRelease.id ==
276 BinaryPackagePublishingHistory.binarypackagereleaseID,
277@@ -322,7 +454,21 @@
278 BinaryPackageFormat.DDEB,
279 bpph_location_clauses)
280 self.logger.debug("Dominating binaries...")
281- self._dominatePublications(self._sortPackages(binaries, False))
282+ self._dominatePublications(
283+ self._sortPackages(binaries, generalization), generalization)
284+
285+ def _composeActiveSourcePubsCondition(self, distroseries, pocket):
286+ """Compose ORM condition for restricting relevant source pubs."""
287+ # Avoid circular imports.
288+ from lp.soyuz.model.publishing import SourcePackagePublishingHistory
289+
290+ return And(
291+ SourcePackagePublishingHistory.status ==
292+ PackagePublishingStatus.PUBLISHED,
293+ SourcePackagePublishingHistory.distroseries == distroseries,
294+ SourcePackagePublishingHistory.archive == self.archive,
295+ SourcePackagePublishingHistory.pocket == pocket,
296+ )
297
298 def dominateSources(self, distroseries, pocket):
299 """Perform domination on source package publications.
300@@ -332,38 +478,76 @@
301 """
302 # Avoid circular imports.
303 from lp.soyuz.model.publishing import SourcePackagePublishingHistory
304+
305+ generalization = GeneralizedPublication(is_source=True)
306+
307 self.logger.debug(
308 "Performing domination across %s/%s (Source)",
309 distroseries.name, pocket.title)
310- spph_location_clauses = And(
311- SourcePackagePublishingHistory.status ==
312- PackagePublishingStatus.PUBLISHED,
313- SourcePackagePublishingHistory.distroseries == distroseries,
314- SourcePackagePublishingHistory.archive == self.archive,
315- SourcePackagePublishingHistory.pocket == pocket,
316- )
317+
318+ spph_location_clauses = self._composeActiveSourcePubsCondition(
319+ distroseries, pocket)
320+ having_multiple_active_publications = (
321+ Count(SourcePackagePublishingHistory.id) > 1)
322 candidate_source_names = Select(
323 SourcePackageName.id,
324- And(
325- SourcePackageRelease.sourcepackagenameID ==
326- SourcePackageName.id,
327- SourcePackagePublishingHistory.sourcepackagereleaseID ==
328- SourcePackageRelease.id,
329- spph_location_clauses,
330- ),
331+ And(join_spph_spr(), join_spr_spn(), spph_location_clauses),
332 group_by=SourcePackageName.id,
333- having=Count(SourcePackagePublishingHistory.id) > 1)
334- sources = IMasterStore(SourcePackagePublishingHistory).find(
335+ having=having_multiple_active_publications)
336+ sources = IStore(SourcePackagePublishingHistory).find(
337 SourcePackagePublishingHistory,
338- SourcePackageRelease.id ==
339- SourcePackagePublishingHistory.sourcepackagereleaseID,
340+ join_spph_spr(),
341 SourcePackageRelease.sourcepackagenameID.is_in(
342 candidate_source_names),
343 spph_location_clauses)
344+
345 self.logger.debug("Dominating sources...")
346- self._dominatePublications(self._sortPackages(sources))
347+ self._dominatePublications(
348+ self._sortPackages(sources, generalization), generalization)
349 flush_database_updates()
350
351+ def findPublishedSourcePackageNames(self, distroseries, pocket):
352+ """Find names of currently published source packages."""
353+ result = IStore(SourcePackageName).find(
354+ SourcePackageName.name,
355+ join_spph_spr(),
356+ join_spr_spn(),
357+ self._composeActiveSourcePubsCondition(distroseries, pocket))
358+ return result.config(distinct=True)
359+
360+ def findPublishedSPPHs(self, distroseries, pocket, package_name):
361+ """Find currently published source publications for given package."""
362+ # Avoid circular imports.
363+ from lp.soyuz.model.publishing import SourcePackagePublishingHistory
364+
365+ return IStore(SourcePackagePublishingHistory).find(
366+ SourcePackagePublishingHistory,
367+ join_spph_spr(),
368+ join_spr_spn(),
369+ SourcePackageName.name == package_name,
370+ self._composeActiveSourcePubsCondition(distroseries, pocket))
371+
372+ def dominateRemovedSourceVersions(self, distroseries, pocket,
373+ package_name, live_versions):
374+ """Dominate source publications based on a set of "live" versions.
375+
376+ Active publications for the "live" versions will remain active. All
377+ other active publications for the same package (and the same archive,
378+ distroseries, and pocket) are marked superseded.
379+
380+ Unlike traditional domination, this allows multiple versions of a
381+ package to stay active in the same distroseries, archive, and pocket.
382+
383+ :param distroseries: `DistroSeries` to dominate.
384+ :param pocket: `PackagePublishingPocket` to dominate.
385+ :param package_name: Source package name, as text.
386+ :param live_versions: Iterable of all version strings that are to
387+ remain active.
388+ """
389+ generalization = GeneralizedPublication(is_source=True)
390+ pubs = self.findPublishedSPPHs(distroseries, pocket, package_name)
391+ self.dominatePackage(pubs, live_versions, generalization)
392+
393 def judge(self, distroseries, pocket):
394 """Judge superseded sources and binaries."""
395 # Avoid circular imports.
396
397=== modified file 'lib/lp/archivepublisher/tests/test_dominator.py'
398--- lib/lp/archivepublisher/tests/test_dominator.py 2011-02-04 05:11:00 +0000
399+++ lib/lp/archivepublisher/tests/test_dominator.py 2011-09-09 09:33:30 +0000
400@@ -1,18 +1,30 @@
401-# Copyright 2009-2010 Canonical Ltd. This software is licensed under the
402+# Copyright 2009-2011 Canonical Ltd. This software is licensed under the
403 # GNU Affero General Public License version 3 (see the file LICENSE).
404
405 """Tests for domination.py."""
406
407 __metaclass__ = type
408
409+import apt_pkg
410 import datetime
411+from operator import attrgetter
412+from zope.security.proxy import removeSecurityProxy
413
414 from canonical.database.sqlbase import flush_database_updates
415-from lp.archivepublisher.domination import Dominator, STAY_OF_EXECUTION
416+from canonical.testing.layers import ZopelessDatabaseLayer
417+from lp.archivepublisher.domination import (
418+ Dominator,
419+ GeneralizedPublication,
420+ STAY_OF_EXECUTION,
421+ )
422 from lp.archivepublisher.publishing import Publisher
423+from lp.registry.interfaces.pocket import PackagePublishingPocket
424 from lp.registry.interfaces.series import SeriesStatus
425+from lp.services.log.logger import DevNullLogger
426 from lp.soyuz.enums import PackagePublishingStatus
427+from lp.soyuz.interfaces.publishing import ISourcePackagePublishingHistory
428 from lp.soyuz.tests.test_publishing import TestNativePublishingBase
429+from lp.testing import TestCaseWithFactory
430
431
432 class TestDominator(TestNativePublishingBase):
433@@ -50,6 +62,8 @@
434 foo_10_source, foo_10_binaries[0])
435
436 def dominateAndCheck(self, dominant, dominated, supersededby):
437+ generalization = GeneralizedPublication(
438+ is_source=ISourcePackagePublishingHistory.providedBy(dominant))
439 dominator = Dominator(self.logger, self.ubuntutest.main_archive)
440
441 # The _dominate* test methods require a dictionary where the
442@@ -58,7 +72,7 @@
443 # and dominated, the subsequents.
444 pubs = {'foo': [dominant, dominated]}
445
446- dominator._dominatePublications(pubs)
447+ dominator._dominatePublications(pubs, generalization)
448 flush_database_updates()
449
450 # The dominant version remains correctly published.
451@@ -145,7 +159,9 @@
452 # This isn't a really good exception. It should probably be
453 # something more indicative of bad input.
454 self.assertRaises(
455- AssertionError, dominator._dominatePublications, pubs)
456+ AssertionError,
457+ dominator._dominatePublications,
458+ pubs, GeneralizedPublication(True))
459
460
461 class TestDomination(TestNativePublishingBase):
462@@ -200,3 +216,533 @@
463 TestDomination.setUp(self)
464 self.ubuntutest['breezy-autotest'].status = (
465 SeriesStatus.OBSOLETE)
466+
467+
468+def make_spphs_for_versions(factory, versions):
469+ """Create publication records for each of `versions`.
470+
471+ They records are created in the same order in which they are specified.
472+ Make the order irregular to prove that version ordering is not a
473+ coincidence of object creation order etc.
474+
475+ Versions may also be identical; each publication record will still have
476+ its own package release.
477+ """
478+ spn = factory.makeSourcePackageName()
479+ distroseries = factory.makeDistroSeries()
480+ pocket = factory.getAnyPocket()
481+ sprs = [
482+ factory.makeSourcePackageRelease(
483+ sourcepackagename=spn, version=version)
484+ for version in versions]
485+ return [
486+ factory.makeSourcePackagePublishingHistory(
487+ distroseries=distroseries, pocket=pocket,
488+ sourcepackagerelease=spr,
489+ status=PackagePublishingStatus.PUBLISHED)
490+ for spr in sprs]
491+
492+
493+def list_source_versions(spphs):
494+ """Extract the versions from `spphs` as a list, in the same order."""
495+ return [spph.sourcepackagerelease.version for spph in spphs]
496+
497+
498+def alter_creation_dates(spphs, ages):
499+ """Set `datecreated` on each of `spphs` according to `ages`.
500+
501+ :param spphs: Iterable of `SourcePackagePublishingHistory`. Their
502+ respective creation dates will be offset by the respective ages found
503+ in `ages` (with the two being matched up in the same order).
504+ :param ages: Iterable of ages. Must provide the same number of items as
505+ `spphs`. Ages are `timedelta` objects that will be subtracted from
506+ the creation dates on the respective records in `spph`.
507+ """
508+ for spph, age in zip(spphs, ages):
509+ spph.datecreated -= age
510+
511+
512+class TestGeneralizedPublication(TestCaseWithFactory):
513+ """Test publication generalization helpers."""
514+
515+ layer = ZopelessDatabaseLayer
516+
517+ def test_getPackageVersion_gets_source_version(self):
518+ spph = self.factory.makeSourcePackagePublishingHistory()
519+ self.assertEqual(
520+ spph.sourcepackagerelease.version,
521+ GeneralizedPublication(is_source=True).getPackageVersion(spph))
522+
523+ def test_getPackageVersion_gets_binary_version(self):
524+ bpph = self.factory.makeBinaryPackagePublishingHistory()
525+ self.assertEqual(
526+ bpph.binarypackagerelease.version,
527+ GeneralizedPublication(is_source=False).getPackageVersion(bpph))
528+
529+ def test_compare_sorts_versions(self):
530+ versions = [
531+ '1.1v2',
532+ '1.1v1',
533+ '1.1v3',
534+ ]
535+ spphs = make_spphs_for_versions(self.factory, versions)
536+ sorted_spphs = sorted(spphs, cmp=GeneralizedPublication().compare)
537+ self.assertEqual(
538+ sorted(versions), list_source_versions(sorted_spphs))
539+
540+ def test_compare_orders_versions_by_debian_rules(self):
541+ versions = [
542+ '1.1.0',
543+ '1.10',
544+ '1.1',
545+ '1.1ubuntu0',
546+ ]
547+ spphs = make_spphs_for_versions(self.factory, versions)
548+
549+ debian_sorted_versions = sorted(versions, cmp=apt_pkg.VersionCompare)
550+
551+ # Assumption: in this case, Debian version ordering is not the
552+ # same as alphabetical version ordering.
553+ self.assertNotEqual(sorted(versions), debian_sorted_versions)
554+
555+ # The compare method produces the Debian ordering.
556+ sorted_spphs = sorted(spphs, cmp=GeneralizedPublication().compare)
557+ self.assertEqual(
558+ sorted(versions, cmp=apt_pkg.VersionCompare),
559+ list_source_versions(sorted_spphs))
560+
561+ def test_compare_breaks_tie_with_creation_date(self):
562+ # When two publications are tied for comparison because they are
563+ # for the same package release, they are ordered by creation
564+ # date.
565+ distroseries = self.factory.makeDistroSeries()
566+ pocket = self.factory.getAnyPocket()
567+ spr = self.factory.makeSourcePackageRelease()
568+ ages = [
569+ datetime.timedelta(2),
570+ datetime.timedelta(1),
571+ datetime.timedelta(3),
572+ ]
573+ spphs = [
574+ self.factory.makeSourcePackagePublishingHistory(
575+ sourcepackagerelease=spr, distroseries=distroseries,
576+ pocket=pocket)
577+ for counter in xrange(len(ages))]
578+ alter_creation_dates(spphs, ages)
579+
580+ self.assertEqual(
581+ [spphs[2], spphs[0], spphs[1]],
582+ sorted(spphs, cmp=GeneralizedPublication().compare))
583+
584+ def test_compare_breaks_tie_for_releases_with_same_version(self):
585+ # When two publications are tied for comparison because they
586+ # belong to releases with the same version string, they are
587+ # ordered by creation date.
588+ version = "1.%d" % self.factory.getUniqueInteger()
589+ ages = [
590+ datetime.timedelta(2),
591+ datetime.timedelta(1),
592+ datetime.timedelta(3),
593+ ]
594+ distroseries = self.factory.makeDistroSeries()
595+ pocket = self.factory.getAnyPocket()
596+ spphs = [
597+ self.factory.makeSourcePackagePublishingHistory(
598+ distroseries=distroseries, pocket=pocket,
599+ sourcepackagerelease=self.factory.makeSourcePackageRelease(
600+ version=version))
601+ for counter in xrange(len(ages))]
602+ alter_creation_dates(spphs, ages)
603+
604+ self.assertEqual(
605+ [spphs[2], spphs[0], spphs[1]],
606+ sorted(spphs, cmp=GeneralizedPublication().compare))
607+
608+
609+def jumble(ordered_list):
610+ """Jumble the elements of `ordered_list` into a weird order.
611+
612+ Ordering is very important in domination. We jumble some of our lists to
613+ insure against "lucky coincidences" that might give our tests the right
614+ answers for the wrong reasons.
615+ """
616+ even = [
617+ item for offset, item in enumerate(ordered_list) if offset % 2 == 0]
618+ odd = [
619+ item for offset, item in enumerate(ordered_list) if offset % 2 != 0]
620+ return list(reversed(odd)) + even
621+
622+
623+class TestDominatorMethods(TestCaseWithFactory):
624+
625+ layer = ZopelessDatabaseLayer
626+
627+ def makeDominator(self, publications):
628+ """Create a `Dominator` suitable for `publications`."""
629+ if len(publications) == 0:
630+ archive = self.factory.makeArchive()
631+ else:
632+ archive = publications[0].archive
633+ return Dominator(DevNullLogger(), archive)
634+
635+ def test_dominatePackage_survives_empty_publications_list(self):
636+ # Nothing explodes when dominatePackage is called with an empty
637+ # packages list.
638+ self.makeDominator([]).dominatePackage(
639+ [], [], GeneralizedPublication(True))
640+ # The test is that we get here without error.
641+ pass
642+
643+ def test_dominatePackage_leaves_live_version_untouched(self):
644+ # dominatePackage does not supersede live versions.
645+ [pub] = make_spphs_for_versions(self.factory, ['3.1'])
646+ self.makeDominator([pub]).dominatePackage(
647+ [pub], ['3.1'], GeneralizedPublication(True))
648+ self.assertEqual(PackagePublishingStatus.PUBLISHED, pub.status)
649+
650+ def test_dominatePackage_deletes_dead_version_without_successor(self):
651+ # dominatePackage marks non-live package versions without
652+ # superseding versions as deleted.
653+ [pub] = make_spphs_for_versions(self.factory, ['1.1'])
654+ self.makeDominator([pub]).dominatePackage(
655+ [pub], [], GeneralizedPublication(True))
656+ self.assertEqual(PackagePublishingStatus.DELETED, pub.status)
657+
658+ def test_dominatePackage_supersedes_older_pub_with_newer_live_pub(self):
659+ # When marking a package as superseded, dominatePackage
660+ # designates a newer live version as the superseding version.
661+ pubs = make_spphs_for_versions(self.factory, ['1.0', '1.1'])
662+ self.makeDominator(pubs).dominatePackage(
663+ pubs, ['1.1'], GeneralizedPublication(True))
664+ self.assertEqual(PackagePublishingStatus.SUPERSEDED, pubs[0].status)
665+ self.assertEqual(pubs[1].sourcepackagerelease, pubs[0].supersededby)
666+ self.assertEqual(PackagePublishingStatus.PUBLISHED, pubs[1].status)
667+
668+ def test_dominatePackage_only_supersedes_with_live_pub(self):
669+ # When marking a package as superseded, dominatePackage will
670+ # only pick a live version as the superseding one.
671+ pubs = make_spphs_for_versions(
672+ self.factory, ['1.0', '2.0', '3.0', '4.0'])
673+ self.makeDominator(pubs).dominatePackage(
674+ pubs, ['3.0'], GeneralizedPublication(True))
675+ self.assertEqual([
676+ pubs[2].sourcepackagerelease,
677+ pubs[2].sourcepackagerelease,
678+ None,
679+ None,
680+ ],
681+ [pub.supersededby for pub in pubs])
682+
683+ def test_dominatePackage_supersedes_with_oldest_newer_live_pub(self):
684+ # When marking a package as superseded, dominatePackage picks
685+ # the oldest of the newer, live versions as the superseding one.
686+ pubs = make_spphs_for_versions(self.factory, ['2.7', '2.8', '2.9'])
687+ self.makeDominator(pubs).dominatePackage(
688+ pubs, ['2.8', '2.9'], GeneralizedPublication(True))
689+ self.assertEqual(pubs[1].sourcepackagerelease, pubs[0].supersededby)
690+
691+ def test_dominatePackage_only_supersedes_with_newer_live_pub(self):
692+ # When marking a package as superseded, dominatePackage only
693+ # considers a newer version as the superseding one.
694+ pubs = make_spphs_for_versions(self.factory, ['0.1', '0.2'])
695+ self.makeDominator(pubs).dominatePackage(
696+ pubs, ['0.1'], GeneralizedPublication(True))
697+ self.assertEqual(None, pubs[1].supersededby)
698+ self.assertEqual(PackagePublishingStatus.DELETED, pubs[1].status)
699+
700+ def test_dominatePackage_supersedes_replaced_pub_for_live_version(self):
701+ # Even if a publication record is for a live version, a newer
702+ # one for the same version supersedes it.
703+ spr = self.factory.makeSourcePackageRelease()
704+ series = self.factory.makeDistroSeries()
705+ pocket = PackagePublishingPocket.RELEASE
706+ pubs = [
707+ self.factory.makeSourcePackagePublishingHistory(
708+ archive=series.main_archive, distroseries=series,
709+ pocket=pocket, status=PackagePublishingStatus.PUBLISHED,
710+ sourcepackagerelease=spr)
711+ for counter in xrange(3)]
712+ alter_creation_dates(pubs, [
713+ datetime.timedelta(3),
714+ datetime.timedelta(2),
715+ datetime.timedelta(1),
716+ ])
717+
718+ self.makeDominator(pubs).dominatePackage(
719+ pubs, [spr.version], GeneralizedPublication(True))
720+ self.assertEqual([
721+ PackagePublishingStatus.SUPERSEDED,
722+ PackagePublishingStatus.SUPERSEDED,
723+ PackagePublishingStatus.PUBLISHED,
724+ ],
725+ [pub.status for pub in pubs])
726+ self.assertEqual(
727+ [spr, spr, None], [pub.supersededby for pub in pubs])
728+
729+ def test_dominatePackage_advanced_scenario(self):
730+ # Put dominatePackage through its paces with complex combined
731+ # data.
732+ # This test should be redundant in theory (which in theory
733+ # equates practice but in practice does not). If this fails,
734+ # don't just patch up the code or this test. Create unit tests
735+ # that specifically cover the difference, then change the code
736+ # and/or adapt this test to return to harmony.
737+ series = self.factory.makeDistroSeries()
738+ package = self.factory.makeSourcePackageName()
739+ pocket = PackagePublishingPocket.RELEASE
740+
741+ versions = ["1.%d" % number for number in xrange(4)]
742+
743+ # We have one package releases for each version.
744+ relevant_releases = dict(
745+ (version, self.factory.makeSourcePackageRelease(
746+ sourcepackagename=package, version=version))
747+ for version in jumble(versions))
748+
749+ # Each of those releases is subsequently published in
750+ # different components.
751+ components = jumble(
752+ [self.factory.makeComponent() for version in versions])
753+
754+ # Map versions to lists of publications for that version, from
755+ # oldest to newest. Each re-publishing into a different
756+ # component is meant to supersede publication into the previous
757+ # component.
758+ pubs_by_version = dict(
759+ (version, [
760+ self.factory.makeSourcePackagePublishingHistory(
761+ archive=series.main_archive, distroseries=series,
762+ pocket=pocket, status=PackagePublishingStatus.PUBLISHED,
763+ sourcepackagerelease=relevant_releases[version],
764+ component=component)
765+ for component in components])
766+ for version in jumble(versions))
767+
768+ ages = jumble(
769+ [datetime.timedelta(age) for age in xrange(len(versions))])
770+
771+ # Actually the "oldest to newest" order on the publications only
772+ # applies to their creation dates. Their creation orders are
773+ # irrelevant.
774+ for pubs_list in pubs_by_version.itervalues():
775+ alter_creation_dates(pubs_list, ages)
776+ pubs_list.sort(key=attrgetter('datecreated'))
777+
778+ live_versions = ["1.1", "1.2"]
779+ last_version_alive = sorted(live_versions)[-1]
780+
781+ all_pubs = sum(pubs_by_version.itervalues(), [])
782+ Dominator(DevNullLogger(), series.main_archive).dominatePackage(
783+ all_pubs, live_versions, GeneralizedPublication(True))
784+
785+ for version in reversed(versions):
786+ pubs = pubs_by_version[version]
787+
788+ if version in live_versions:
789+ # Beware: loop-carried variable. Used locally as well,
790+ # but tells later iterations what the highest-versioned
791+ # release so far was. This is used in tracking
792+ # supersededby links.
793+ superseding_release = pubs[-1].sourcepackagerelease
794+
795+ if version in live_versions:
796+ # The live versions' latest publications are Published,
797+ # their older ones Superseded.
798+ expected_status = (
799+ [PackagePublishingStatus.SUPERSEDED] * (len(pubs) - 1) +
800+ [PackagePublishingStatus.PUBLISHED])
801+ expected_supersededby = (
802+ [superseding_release] * (len(pubs) - 1) + [None])
803+ elif version < last_version_alive:
804+ # The superseded versions older than the last live
805+ # version have all been superseded.
806+ expected_status = (
807+ [PackagePublishingStatus.SUPERSEDED] * len(pubs))
808+ expected_supersededby = [superseding_release] * len(pubs)
809+ else:
810+ # Versions that are newer than any live release have
811+ # been deleted.
812+ expected_status = (
813+ [PackagePublishingStatus.DELETED] * len(pubs))
814+ expected_supersededby = [None] * len(pubs)
815+
816+ self.assertEqual(expected_status, [pub.status for pub in pubs])
817+ self.assertEqual(
818+ expected_supersededby, [pub.supersededby for pub in pubs])
819+
820+ def test_dominateRemovedSourceVersions_dominates_publications(self):
821+ # dominateRemovedSourceVersions finds the publications for a
822+ # package and calls dominatePackage on them.
823+ pubs = make_spphs_for_versions(self.factory, ['0.1', '0.2', '0.3'])
824+ package_name = pubs[0].sourcepackagerelease.sourcepackagename.name
825+
826+ self.makeDominator(pubs).dominateRemovedSourceVersions(
827+ pubs[0].distroseries, pubs[0].pocket, package_name, ['0.2'])
828+ self.assertEqual([
829+ PackagePublishingStatus.SUPERSEDED,
830+ PackagePublishingStatus.PUBLISHED,
831+ PackagePublishingStatus.DELETED,
832+ ],
833+ [pub.status for pub in pubs])
834+ self.assertEqual(
835+ [pubs[1].sourcepackagerelease, None, None],
836+ [pub.supersededby for pub in pubs])
837+
838+ def test_dominateRemovedSourceVersions_ignores_other_pockets(self):
839+ # dominateRemovedSourceVersions ignores publications in other
840+ # pockets than the one specified.
841+ pubs = make_spphs_for_versions(self.factory, ['2.3', '2.4'])
842+ package_name = pubs[0].sourcepackagerelease.sourcepackagename.name
843+ removeSecurityProxy(pubs[0]).pocket = PackagePublishingPocket.UPDATES
844+ removeSecurityProxy(pubs[1]).pocket = PackagePublishingPocket.PROPOSED
845+ self.makeDominator(pubs).dominateRemovedSourceVersions(
846+ pubs[0].distroseries, pubs[0].pocket, package_name, ['2.3'])
847+ self.assertEqual(PackagePublishingStatus.PUBLISHED, pubs[1].status)
848+
849+ def test_dominateRemovedSourceVersions_ignores_other_packages(self):
850+ pubs = make_spphs_for_versions(self.factory, ['1.0', '1.1'])
851+ other_package_name = self.factory.makeSourcePackageName().name
852+ self.makeDominator(pubs).dominateRemovedSourceVersions(
853+ pubs[0].distroseries, pubs[0].pocket, other_package_name, ['1.1'])
854+ self.assertEqual(PackagePublishingStatus.PUBLISHED, pubs[0].status)
855+
856+ def test_findPublishedSourcePackageNames_finds_package(self):
857+ spph = self.factory.makeSourcePackagePublishingHistory(
858+ status=PackagePublishingStatus.PUBLISHED)
859+ dominator = self.makeDominator([spph])
860+ self.assertContentEqual(
861+ [spph.sourcepackagerelease.sourcepackagename.name],
862+ dominator.findPublishedSourcePackageNames(
863+ spph.distroseries, spph.pocket))
864+
865+ def test_findPublishedSourcePackageNames_ignores_other_states(self):
866+ series = self.factory.makeDistroSeries()
867+ pocket = PackagePublishingPocket.RELEASE
868+ spphs = dict(
869+ (status, self.factory.makeSourcePackagePublishingHistory(
870+ distroseries=series, archive=series.main_archive,
871+ pocket=pocket, status=status))
872+ for status in PackagePublishingStatus.items)
873+ published_spph = spphs[PackagePublishingStatus.PUBLISHED]
874+ dominator = self.makeDominator(spphs.values())
875+ self.assertContentEqual(
876+ [published_spph.sourcepackagerelease.sourcepackagename.name],
877+ dominator.findPublishedSourcePackageNames(series, pocket))
878+
879+ def test_findPublishedSourcePackageNames_ignores_other_archives(self):
880+ spph = self.factory.makeSourcePackagePublishingHistory(
881+ status=PackagePublishingStatus.PUBLISHED)
882+ dominator = self.makeDominator([spph])
883+ dominator.archive = self.factory.makeArchive()
884+ self.assertContentEqual(
885+ [],
886+ dominator.findPublishedSourcePackageNames(
887+ spph.distroseries, spph.pocket))
888+
889+ def test_findPublishedSourcePackageNames_ignores_other_series(self):
890+ spph = self.factory.makeSourcePackagePublishingHistory(
891+ status=PackagePublishingStatus.PUBLISHED)
892+ distro = spph.distroseries.distribution
893+ other_series = self.factory.makeDistroSeries(distribution=distro)
894+ dominator = self.makeDominator([spph])
895+ self.assertContentEqual(
896+ [],
897+ dominator.findPublishedSourcePackageNames(
898+ other_series, spph.pocket))
899+
900+ def test_findPublishedSourcePackageNames_ignores_other_pockets(self):
901+ spph = self.factory.makeSourcePackagePublishingHistory(
902+ status=PackagePublishingStatus.PUBLISHED,
903+ pocket=PackagePublishingPocket.RELEASE)
904+ dominator = self.makeDominator([spph])
905+ self.assertContentEqual(
906+ [],
907+ dominator.findPublishedSourcePackageNames(
908+ spph.distroseries, PackagePublishingPocket.SECURITY))
909+
910+ def test_findPublishedSourcePackageNames_does_not_return_duplicates(self):
911+ series = self.factory.makeDistroSeries()
912+ pocket = PackagePublishingPocket.RELEASE
913+ package = self.factory.makeSourcePackageName()
914+ spphs = [
915+ self.factory.makeSourcePackagePublishingHistory(
916+ distroseries=series, archive=series.main_archive,
917+ pocket=pocket, status=PackagePublishingStatus.PUBLISHED,
918+ sourcepackagerelease=self.factory.makeSourcePackageRelease(
919+ sourcepackagename=package))
920+ for counter in xrange(2)]
921+ dominator = self.makeDominator(spphs)
922+ self.assertEqual(
923+ [package.name],
924+ list(dominator.findPublishedSourcePackageNames(series, pocket)))
925+
926+ def test_findPublishedSPPHs_finds_published_SPPH(self):
927+ spph = self.factory.makeSourcePackagePublishingHistory(
928+ status=PackagePublishingStatus.PUBLISHED)
929+ package_name = spph.sourcepackagerelease.sourcepackagename.name
930+ dominator = self.makeDominator([spph])
931+ self.assertContentEqual(
932+ [spph],
933+ dominator.findPublishedSPPHs(
934+ spph.distroseries, spph.pocket, package_name))
935+
936+ def test_findPublishedSPPHs_ignores_other_states(self):
937+ series = self.factory.makeDistroSeries()
938+ package = self.factory.makeSourcePackageName()
939+ pocket = PackagePublishingPocket.RELEASE
940+ spphs = dict(
941+ (status, self.factory.makeSourcePackagePublishingHistory(
942+ distroseries=series, archive=series.main_archive,
943+ pocket=pocket, status=status,
944+ sourcepackagerelease=self.factory.makeSourcePackageRelease(
945+ sourcepackagename=package)))
946+ for status in PackagePublishingStatus.items)
947+ dominator = self.makeDominator(spphs.values())
948+ self.assertContentEqual(
949+ [spphs[PackagePublishingStatus.PUBLISHED]],
950+ dominator.findPublishedSPPHs(series, pocket, package.name))
951+
952+ def test_findPublishedSPPHs_ignores_other_archives(self):
953+ spph = self.factory.makeSourcePackagePublishingHistory(
954+ status=PackagePublishingStatus.PUBLISHED)
955+ package = spph.sourcepackagerelease.sourcepackagename
956+ dominator = self.makeDominator([spph])
957+ dominator.archive = self.factory.makeArchive()
958+ self.assertContentEqual(
959+ [],
960+ dominator.findPublishedSPPHs(
961+ spph.distroseries, spph.pocket, package.name))
962+
963+ def test_findPublishedSPPHs_ignores_other_series(self):
964+ spph = self.factory.makeSourcePackagePublishingHistory(
965+ status=PackagePublishingStatus.PUBLISHED)
966+ distro = spph.distroseries.distribution
967+ package = spph.sourcepackagerelease.sourcepackagename
968+ other_series = self.factory.makeDistroSeries(distribution=distro)
969+ dominator = self.makeDominator([spph])
970+ self.assertContentEqual(
971+ [],
972+ dominator.findPublishedSPPHs(
973+ other_series, spph.pocket, package.name))
974+
975+ def test_findPublishedSPPHs_ignores_other_pockets(self):
976+ spph = self.factory.makeSourcePackagePublishingHistory(
977+ status=PackagePublishingStatus.PUBLISHED,
978+ pocket=PackagePublishingPocket.RELEASE)
979+ package = spph.sourcepackagerelease.sourcepackagename
980+ dominator = self.makeDominator([spph])
981+ self.assertContentEqual(
982+ [],
983+ dominator.findPublishedSPPHs(
984+ spph.distroseries, PackagePublishingPocket.SECURITY,
985+ package.name))
986+
987+ def test_findPublishedSPPHs_ignores_other_packages(self):
988+ spph = self.factory.makeSourcePackagePublishingHistory(
989+ status=PackagePublishingStatus.PUBLISHED)
990+ other_package = self.factory.makeSourcePackageName()
991+ dominator = self.makeDominator([spph])
992+ self.assertContentEqual(
993+ [],
994+ dominator.findPublishedSPPHs(
995+ spph.distroseries, spph.pocket, other_package.name))
996
997=== modified file 'lib/lp/soyuz/doc/gina.txt'
998--- lib/lp/soyuz/doc/gina.txt 2011-07-29 11:35:28 +0000
999+++ lib/lp/soyuz/doc/gina.txt 2011-09-09 09:33:30 +0000
1000@@ -8,6 +8,7 @@
1001 Get the current counts of stuff in the database:
1002
1003 >>> from canonical.launchpad.database.emailaddress import EmailAddress
1004+ >>> from lp.soyuz.interfaces.publishing import active_publishing_status
1005 >>> from lp.soyuz.model.publishing import (
1006 ... BinaryPackagePublishingHistory,
1007 ... SourcePackagePublishingHistory)
1008@@ -564,35 +565,34 @@
1009 that's what overrides actually do.
1010
1011 >>> from canonical.database.sqlbase import sqlvalues
1012- >>> from lp.soyuz.enums import PackagePublishingStatus
1013- >>> x11_pub = SSPPH.select("""sourcepackagerelease = %s
1014- ... AND distroseries = %s
1015- ... AND status in (%s, %s)""" %
1016- ... sqlvalues(x11p, breezy,
1017- ... PackagePublishingStatus.PUBLISHED,
1018- ... PackagePublishingStatus.PENDING),
1019- ... orderBy=["-datecreated"])[0]
1020+ >>> x11_pub = SSPPH.select("""
1021+ ... sourcepackagerelease = %s AND
1022+ ... distroseries = %s AND
1023+ ... status in %s
1024+ ... """ % sqlvalues(
1025+ ... x11p, breezy, active_publishing_status),
1026+ ... orderBy=["-datecreated"])[0]
1027 >>> print x11_pub.section.name
1028 net
1029- >>> ed_pub = SBPPH.select("""binarypackagerelease = %s
1030- ... AND distroarchseries = %s
1031- ... AND status in (%s, %s)""" %
1032- ... sqlvalues(ed, breezy_i386,
1033- ... PackagePublishingStatus.PUBLISHED,
1034- ... PackagePublishingStatus.PENDING),
1035- ... orderBy=["-datecreated"])[0]
1036+ >>> ed_pub = SBPPH.select("""
1037+ ... binarypackagerelease = %s AND
1038+ ... distroarchseries = %s AND
1039+ ... status in %s
1040+ ... """ % sqlvalues(
1041+ ... ed, breezy_i386, active_publishing_status),
1042+ ... orderBy=["-datecreated"])[0]
1043 >>> print ed_pub.priority
1044 Extra
1045 >>> n = SourcePackageName.selectOneBy(name="archive-copier")
1046 >>> ac = SourcePackageRelease.selectOneBy(sourcepackagenameID=n.id,
1047 ... version="0.3.6")
1048- >>> ac_pub = SSPPH.select("""sourcepackagerelease = %s
1049- ... AND distroseries = %s
1050- ... AND status in (%s, %s)""" %
1051- ... sqlvalues(ac, breezy,
1052- ... PackagePublishingStatus.PUBLISHED,
1053- ... PackagePublishingStatus.PENDING),
1054- ... orderBy=["-datecreated"])[0]
1055+ >>> ac_pub = SSPPH.select("""
1056+ ... sourcepackagerelease = %s AND
1057+ ... distroseries = %s AND
1058+ ... status in %s
1059+ ... """ % sqlvalues(
1060+ ... ac, breezy, active_publishing_status),
1061+ ... orderBy=["-datecreated"])[0]
1062 >>> print ac_pub.component.name
1063 universe
1064
1065@@ -720,7 +720,7 @@
1066
1067 >>> transaction.commit()
1068
1069-There is now a number of source publications in PENDING status for the
1070+There is now a number of source publications in PUBLISHED status for the
1071 targetted distroseries, 'lenny'.
1072
1073 >>> lenny_sources = SSPPH.select("distroseries = %s" % sqlvalues(lenny))
1074@@ -728,7 +728,7 @@
1075 12
1076
1077 >>> print set([pub.status.name for pub in lenny_sources])
1078- set(['PENDING'])
1079+ set(['PUBLISHED'])
1080
1081 As mentioned before, lenny/i386 is empty, no binaries were imported.
1082 Also, the number of binaries published in the whole debian distribution
1083
1084=== modified file 'lib/lp/soyuz/interfaces/publishing.py'
1085--- lib/lp/soyuz/interfaces/publishing.py 2011-09-02 04:51:25 +0000
1086+++ lib/lp/soyuz/interfaces/publishing.py 2011-09-09 09:33:30 +0000
1087@@ -195,9 +195,6 @@
1088 the field name and value is the value string.
1089 """
1090
1091- def supersede():
1092- """Supersede this publication."""
1093-
1094 def requestObsolescence():
1095 """Make this publication obsolete.
1096
1097
1098=== modified file 'lib/lp/soyuz/model/publishing.py'
1099--- lib/lp/soyuz/model/publishing.py 2011-08-31 04:40:44 +0000
1100+++ lib/lp/soyuz/model/publishing.py 2011-09-09 09:33:30 +0000
1101@@ -114,10 +114,6 @@
1102 from lp.soyuz.scripts.changeoverride import ArchiveOverriderError
1103
1104
1105-PENDING = PackagePublishingStatus.PENDING
1106-PUBLISHED = PackagePublishingStatus.PUBLISHED
1107-
1108-
1109 # XXX cprov 2006-08-18: move it away, perhaps archivepublisher/pool.py
1110
1111 def makePoolPath(source_name, component_name):
1112@@ -327,8 +323,8 @@
1113 fields = self.buildIndexStanzaFields()
1114 return fields.makeOutput()
1115
1116- def supersede(self):
1117- """See `IPublishing`."""
1118+ def setSuperseded(self):
1119+ """Set to SUPERSEDED status."""
1120 self.status = PackagePublishingStatus.SUPERSEDED
1121 self.datesuperseded = UTC_NOW
1122
1123@@ -742,7 +738,7 @@
1124 "Should not dominate unpublished source %s" %
1125 self.sourcepackagerelease.title)
1126
1127- super(SourcePackagePublishingHistory, self).supersede()
1128+ self.setSuperseded()
1129
1130 if dominant is not None:
1131 if logger is not None:
1132@@ -1081,7 +1077,7 @@
1133 return IMasterStore(BinaryPackagePublishingHistory).find(
1134 BinaryPackagePublishingHistory,
1135 BinaryPackagePublishingHistory.status.is_in(
1136- [PUBLISHED, PENDING]),
1137+ active_publishing_status),
1138 BinaryPackagePublishingHistory.distroarchseriesID.is_in(
1139 available_architectures),
1140 binarypackagerelease=self.binarypackagerelease,
1141@@ -1101,7 +1097,7 @@
1142 return IMasterStore(BinaryPackagePublishingHistory).find(
1143 BinaryPackagePublishingHistory,
1144 BinaryPackagePublishingHistory.status.is_in(
1145- [PUBLISHED, PENDING]),
1146+ active_publishing_status),
1147 BinaryPackagePublishingHistory.distroarchseries ==
1148 self.distroarchseries,
1149 binarypackagerelease=self.binarypackagerelease.debug_package,
1150@@ -1126,7 +1122,7 @@
1151 self.distroarchseries.architecturetag))
1152 return
1153
1154- super(BinaryPackagePublishingHistory, self).supersede()
1155+ self.setSuperseded()
1156
1157 if dominant is not None:
1158 # DDEBs cannot themselves be dominant; they are always dominated
1159
1160=== added file 'lib/lp/soyuz/scripts/gina/dominate.py'
1161--- lib/lp/soyuz/scripts/gina/dominate.py 1970-01-01 00:00:00 +0000
1162+++ lib/lp/soyuz/scripts/gina/dominate.py 2011-09-09 09:33:30 +0000
1163@@ -0,0 +1,82 @@
1164+# Copyright 2011 Canonical Ltd. This software is licensed under the
1165+# GNU Affero General Public License version 3 (see the file LICENSE).
1166+
1167+"""Retirement of packages that are removed upstream."""
1168+
1169+__metaclass__ = type
1170+__all__ = [
1171+ 'dominate_imported_source_packages',
1172+ ]
1173+
1174+from zope.component import getUtility
1175+
1176+# XXX JeroenVermeulen 2011-09-08, bug=844550: The GeneralizedPublication
1177+# import violates import policy and elicits a warning from the test
1178+# suite. The warning helps remind us to retire this code as soon as
1179+# possible.
1180+from lp.archivepublisher.domination import (
1181+ Dominator,
1182+ GeneralizedPublication,
1183+ )
1184+from lp.registry.interfaces.distribution import IDistributionSet
1185+
1186+
1187+def dominate_imported_source_packages(logger, distro_name, series_name,
1188+ pocket, packages_map):
1189+ """Perform domination."""
1190+ series = getUtility(IDistributionSet)[distro_name].getSeries(series_name)
1191+ dominator = Dominator(logger, series.main_archive)
1192+
1193+ # XXX JeroenVermeulen 2011-09-08, bug=844550: This is a transitional
1194+ # hack. Gina used to create SPPHs in Pending state. We cleaned up
1195+ # the bulk of them, and changed the code to create Published ones, but
1196+ # some new ones will have been created since.
1197+ # Update those to match what the new Gina does.
1198+ from canonical.launchpad.interfaces.lpstorm import IStore
1199+ from lp.soyuz.enums import PackagePublishingStatus
1200+ from lp.soyuz.model.publishing import SourcePackagePublishingHistory
1201+ SPPH = SourcePackagePublishingHistory
1202+ store = IStore(SPPH)
1203+ spphs = store.find(
1204+ SPPH,
1205+ SPPH.archive == series.main_archive,
1206+ SPPH.distroseries == series,
1207+ SPPH.pocket == pocket,
1208+ SPPH.status == PackagePublishingStatus.PENDING)
1209+ spphs.set(status=PackagePublishingStatus.PUBLISHED)
1210+
1211+ # Dominate packages found in the Sources list we're importing.
1212+ package_names = dominator.findPublishedSourcePackageNames(series, pocket)
1213+ for package_name in package_names:
1214+ entries = packages_map.src_map.get(package_name)
1215+
1216+ if entries is None:
1217+ # XXX JeroenVermeulen 2011-09-08, bug=844550: This is a
1218+ # transitional hack. The database is full of "Published"
1219+ # Debian SPPHs whose packages have actually been deleted.
1220+ # In the future such publications should simply be marked
1221+ # Deleted, but for the legacy baggage we currently carry
1222+ # around we'll just do traditional domination first: pick
1223+ # the latest Published version, and mark the rest of the
1224+ # SPPHs as superseded by that version. The latest version
1225+ # will then, finally, be marked appropriately Deleted once
1226+ # we remove this transitional hack.
1227+ # To remove the transitional hack, just let live_versions
1228+ # default to the empty list instead of doing this:
1229+ pubs = dominator.findPublishedSPPHs(series, pocket, package_name)
1230+ generalization = GeneralizedPublication(is_source=True)
1231+ pubs_dict = dominator._sortPackages(pubs, generalization)
1232+ sorted_pubs = pubs_dict[package_name]
1233+ if len(sorted_pubs) <= 1:
1234+ # If there's only one published SPPH, the transitional
1235+ # code will just leave it Published. Don't bother; the
1236+ # migration will be costly enough as it is.
1237+ continue
1238+ live_versions = [sorted_pubs[0].sourcepackagerelease.version]
1239+ else:
1240+ live_versions = [
1241+ entry['Version']
1242+ for entry in entries if 'Version' in entry]
1243+
1244+ dominator.dominateRemovedSourceVersions(
1245+ series, pocket, package_name, live_versions)
1246
1247=== modified file 'lib/lp/soyuz/scripts/gina/handlers.py'
1248--- lib/lp/soyuz/scripts/gina/handlers.py 2011-05-20 07:43:58 +0000
1249+++ lib/lp/soyuz/scripts/gina/handlers.py 2011-09-09 09:33:30 +0000
1250@@ -35,12 +35,12 @@
1251 from canonical.launchpad.interfaces.librarian import ILibraryFileAliasSet
1252 from canonical.launchpad.scripts import log
1253 from lp.archivepublisher.diskpool import poolify
1254+from lp.archiveuploader.changesfile import ChangesFile
1255 from lp.archiveuploader.tagfiles import parse_tagfile
1256 from lp.archiveuploader.utils import (
1257 determine_binary_file_type,
1258 determine_source_file_type,
1259 )
1260-from lp.archiveuploader.changesfile import ChangesFile
1261 from lp.buildmaster.enums import BuildStatus
1262 from lp.registry.interfaces.person import (
1263 IPersonSet,
1264@@ -54,7 +54,10 @@
1265 )
1266 from lp.soyuz.interfaces.binarypackagebuild import IBinaryPackageBuildSet
1267 from lp.soyuz.interfaces.binarypackagename import IBinaryPackageNameSet
1268-from lp.soyuz.interfaces.publishing import IPublishingSet
1269+from lp.soyuz.interfaces.publishing import (
1270+ active_publishing_status,
1271+ IPublishingSet,
1272+ )
1273 from lp.soyuz.model.component import Component
1274 from lp.soyuz.model.files import (
1275 BinaryPackageFile,
1276@@ -723,8 +726,6 @@
1277 source_publishinghistory.status.title)
1278 return
1279
1280- # Create the Publishing entry, with status PENDING so that we
1281- # can republish this later into a Soyuz archive.
1282 entry = getUtility(IPublishingSet).newSourcePublication(
1283 distroseries=self.distroseries,
1284 sourcepackagerelease=sourcepackagerelease,
1285@@ -732,6 +733,7 @@
1286 section=section,
1287 pocket=self.pocket,
1288 archive=archive)
1289+ entry.setPublished()
1290 log.info('Source package %s (%s) published' % (
1291 entry.sourcepackagerelease.sourcepackagename.name,
1292 entry.sourcepackagerelease.version))
1293@@ -742,16 +744,14 @@
1294 from lp.soyuz.model.publishing import (
1295 SourcePackagePublishingHistory)
1296
1297- ret = SourcePackagePublishingHistory.select(
1298- """sourcepackagerelease = %s
1299- AND distroseries = %s
1300- AND archive = %s
1301- AND status in (%s, %s)""" %
1302- sqlvalues(sourcepackagerelease, self.distroseries,
1303- self.distroseries.main_archive,
1304- PackagePublishingStatus.PUBLISHED,
1305- PackagePublishingStatus.PENDING),
1306- orderBy=["-datecreated"])
1307+ ret = SourcePackagePublishingHistory.select("""
1308+ sourcepackagerelease = %s AND
1309+ distroseries = %s AND
1310+ archive = %s AND
1311+ status in %s""" % sqlvalues(
1312+ sourcepackagerelease, self.distroseries,
1313+ self.distroseries.main_archive, active_publishing_status),
1314+ orderBy=["-datecreated"])
1315 ret = list(ret)
1316 if ret:
1317 return ret[0]
1318@@ -917,14 +917,6 @@
1319 "for package %s (%s)" %
1320 (build.id, binary.package, binary.version))
1321 else:
1322-
1323- # XXX Debonzi 2005-05-16: Check it later
1324- # if bin.gpg_signing_key_owner:
1325- # key = self.getGPGKey(bin.gpg_signing_key,
1326- # *bin.gpg_signing_key_owner)
1327- # else:
1328- key = None
1329-
1330 processor = distroarchinfo['processor']
1331 build = getUtility(IBinaryPackageBuildSet).new(
1332 processor=processor.id,
1333@@ -948,8 +940,7 @@
1334 def publish(self, binarypackage, bpdata):
1335 """Create the publishing entry on db if does not exist."""
1336 # Avoid circular imports.
1337- from lp.soyuz.model.publishing import (
1338- BinaryPackagePublishingHistory)
1339+ from lp.soyuz.model.publishing import BinaryPackagePublishingHistory
1340
1341 # These need to be pulled from the binary package data, not the
1342 # binary package release: the data represents data from /this
1343@@ -983,22 +974,20 @@
1344 binpkg_publishinghistory.status.title)
1345 return
1346
1347-
1348- # Create the Publishing entry with status PENDING.
1349 BinaryPackagePublishingHistory(
1350- binarypackagerelease = binarypackage.id,
1351- component = component.id,
1352- section = section.id,
1353- priority = priority,
1354- distroarchseries = self.distroarchseries.id,
1355- status = PackagePublishingStatus.PENDING,
1356- datecreated = UTC_NOW,
1357- datepublished = UTC_NOW,
1358- pocket = self.pocket,
1359- datesuperseded = None,
1360- supersededby = None,
1361- datemadepending = None,
1362- dateremoved = None,
1363+ binarypackagerelease=binarypackage.id,
1364+ component=component.id,
1365+ section=section.id,
1366+ priority=priority,
1367+ distroarchseries=self.distroarchseries.id,
1368+ status=PackagePublishingStatus.PUBLISHED,
1369+ datecreated=UTC_NOW,
1370+ datepublished=UTC_NOW,
1371+ pocket=self.pocket,
1372+ datesuperseded=None,
1373+ supersededby=None,
1374+ datemadepending=None,
1375+ dateremoved=None,
1376 archive=archive)
1377
1378 log.info('BinaryPackage %s-%s published into %s.' % (
1379@@ -1008,19 +997,16 @@
1380 def _checkPublishing(self, binarypackage):
1381 """Query for the publishing entry"""
1382 # Avoid circular imports.
1383- from lp.soyuz.model.publishing import (
1384- BinaryPackagePublishingHistory)
1385+ from lp.soyuz.model.publishing import BinaryPackagePublishingHistory
1386
1387- ret = BinaryPackagePublishingHistory.select(
1388- """binarypackagerelease = %s
1389- AND distroarchseries = %s
1390- AND archive = %s
1391- AND status in (%s, %s)""" %
1392- sqlvalues(binarypackage, self.distroarchseries,
1393- self.distroarchseries.main_archive,
1394- PackagePublishingStatus.PUBLISHED,
1395- PackagePublishingStatus.PENDING),
1396- orderBy=["-datecreated"])
1397+ ret = BinaryPackagePublishingHistory.select("""
1398+ binarypackagerelease = %s AND
1399+ distroarchseries = %s AND
1400+ archive = %s AND
1401+ status in %s""" % sqlvalues(
1402+ binarypackage, self.distroarchseries,
1403+ self.distroarchseries.main_archive, active_publishing_status),
1404+ orderBy=["-datecreated"])
1405 ret = list(ret)
1406 if ret:
1407 return ret[0]
1408
1409=== modified file 'lib/lp/soyuz/scripts/tests/test_gina.py'
1410--- lib/lp/soyuz/scripts/tests/test_gina.py 2010-08-20 20:31:18 +0000
1411+++ lib/lp/soyuz/scripts/tests/test_gina.py 2011-09-09 09:33:30 +0000
1412@@ -1,13 +1,181 @@
1413-# Copyright 2009-2010 Canonical Ltd. This software is licensed under the
1414+# Copyright 2009-2011 Canonical Ltd. This software is licensed under the
1415 # GNU Affero General Public License version 3 (see the file LICENSE).
1416
1417 from doctest import DocTestSuite
1418-import unittest
1419+from unittest import TestLoader
1420
1421+from canonical.testing.layers import ZopelessDatabaseLayer
1422+from lp.registry.interfaces.pocket import PackagePublishingPocket
1423+from lp.services.log.logger import DevNullLogger
1424+from lp.soyuz.enums import PackagePublishingStatus
1425+from lp.soyuz.scripts.gina.dominate import dominate_imported_source_packages
1426 import lp.soyuz.scripts.gina.handlers
1427+from lp.soyuz.scripts.gina.handlers import (
1428+ BinaryPackagePublisher,
1429+ SourcePackagePublisher,
1430+ )
1431+from lp.soyuz.scripts.gina.packages import (
1432+ BinaryPackageData,
1433+ SourcePackageData,
1434+ )
1435+from lp.testing import TestCaseWithFactory
1436+
1437+
1438+class FakePackagesMap:
1439+ def __init__(self, src_map):
1440+ self.src_map = src_map
1441+
1442+
1443+class TestGina(TestCaseWithFactory):
1444+
1445+ layer = ZopelessDatabaseLayer
1446+
1447+ def test_dominate_imported_source_packages_dominates_imports(self):
1448+ # dominate_imported_source_packages dominates the source
1449+ # packages that Gina imports.
1450+ logger = DevNullLogger()
1451+ pub = self.factory.makeSourcePackagePublishingHistory(
1452+ status=PackagePublishingStatus.PUBLISHED)
1453+ series = pub.distroseries
1454+ spr = pub.sourcepackagerelease
1455+ package = spr.sourcepackagename
1456+ dominate_imported_source_packages(
1457+ logger, series.distribution.name, series.name, pub.pocket,
1458+ FakePackagesMap({package.name: []}))
1459+ self.assertEqual(PackagePublishingStatus.DELETED, pub.status)
1460+
1461+ def test_dominate_imported_source_packages_dominates_deletions(self):
1462+ # dominate_imported_source_packages dominates the source
1463+ # packages that have been deleted from the Sources lists that
1464+ # Gina imports.
1465+ series = self.factory.makeDistroSeries()
1466+ pocket = PackagePublishingPocket.RELEASE
1467+ package = self.factory.makeSourcePackageName()
1468+ pubs = [
1469+ self.factory.makeSourcePackagePublishingHistory(
1470+ archive=series.main_archive, distroseries=series,
1471+ pocket=pocket, status=PackagePublishingStatus.PUBLISHED,
1472+ sourcepackagerelease=self.factory.makeSourcePackageRelease(
1473+ sourcepackagename=package, version=version))
1474+ for version in ['1.0', '1.1', '1.1a']]
1475+ logger = DevNullLogger()
1476+ dominate_imported_source_packages(
1477+ logger, series.distribution.name, series.name, pocket,
1478+ FakePackagesMap({}))
1479+ # XXX JeroenVermeulen 2011-09-08, bug=844550: This is
1480+ # "transitional" domination which supersedes older versions of
1481+ # deleted packages with the last known version. Permanent
1482+ # domination will then mark the last known version as deleted.
1483+ # For permanent domination, the expected outcome is that all
1484+ # these publications will be Deleted (but any pre-existing
1485+ # Superseded publications for older versions will remain
1486+ # Superseded).
1487+ self.assertEqual([
1488+ PackagePublishingStatus.SUPERSEDED,
1489+ PackagePublishingStatus.SUPERSEDED,
1490+ PackagePublishingStatus.PUBLISHED,
1491+ ],
1492+ [pub.status for pub in pubs])
1493+
1494+ def test_dominate_imported_source_packages_cleans_up_pending_spphs(self):
1495+ # XXX JeroenVermeulen 2011-09-08, bug=844550: For transition to
1496+ # Gina domination, dominate_imported_source_packages turns any
1497+ # remaining Pending SPPHS into Published ones.
1498+ series = self.factory.makeDistroSeries()
1499+ spph = self.factory.makeSourcePackagePublishingHistory(
1500+ distroseries=series, archive=series.main_archive,
1501+ status=PackagePublishingStatus.PENDING)
1502+ spr = spph.sourcepackagerelease
1503+ package_name = spr.sourcepackagename.name
1504+ logger = DevNullLogger()
1505+ dominate_imported_source_packages(
1506+ logger, series.distribution.name, series.name, spph.pocket,
1507+ FakePackagesMap({package_name: [{"Version": spr.version}]}))
1508+ self.assertEqual(PackagePublishingStatus.PUBLISHED, spph.status)
1509+
1510+ def test_dominate_imported_source_packages_cleans_up_first(self):
1511+ # XXX JeroenVermeulen 2011-09-08, bug=844550: For transition to
1512+ # Gina domination, dominate_imported_source_packages turns any
1513+ # remaining Pending SPPHS into Published ones. It does this
1514+ # *before* dominating, so no domination happens while some of
1515+ # the SPPHs are still mistakenly Pending (which would result in
1516+ # mistaken deletions).
1517+ series = self.factory.makeDistroSeries()
1518+ package = self.factory.makeSourcePackageName()
1519+ pocket = PackagePublishingPocket.RELEASE
1520+ versions = ['1.0', '1.1']
1521+ statuses_before = [
1522+ PackagePublishingStatus.PUBLISHED,
1523+ PackagePublishingStatus.PENDING,
1524+ ]
1525+ statuses_after = [
1526+ PackagePublishingStatus.SUPERSEDED,
1527+ PackagePublishingStatus.PUBLISHED,
1528+ ]
1529+ live_version = versions[-1]
1530+ sprs = [
1531+ self.factory.makeSourcePackageRelease(
1532+ sourcepackagename=package, version=version)
1533+ for version in versions]
1534+ spphs = [
1535+ self.factory.makeSourcePackagePublishingHistory(
1536+ archive=series.main_archive, distroseries=series,
1537+ sourcepackagerelease=spr, pocket=pocket, status=status)
1538+ for spr, status in zip(sprs, statuses_before)]
1539+
1540+ logger = DevNullLogger()
1541+ dominate_imported_source_packages(
1542+ logger, series.distribution.name, series.name, pocket,
1543+ FakePackagesMap({package.name: [{"Version": live_version}]}))
1544+
1545+ self.assertEqual(statuses_after, [spph.status for spph in spphs])
1546+
1547+
1548+class TestSourcePackagePublisher(TestCaseWithFactory):
1549+
1550+ layer = ZopelessDatabaseLayer
1551+
1552+ def test_publish_creates_published_publication(self):
1553+ maintainer = self.factory.makePerson()
1554+ series = self.factory.makeDistroSeries()
1555+ section = self.factory.makeSection()
1556+ pocket = PackagePublishingPocket.RELEASE
1557+ spr = self.factory.makeSourcePackageRelease()
1558+
1559+ publisher = SourcePackagePublisher(series, pocket, None)
1560+ publisher.publish(spr, SourcePackageData(
1561+ component='main', section=section.name, version='1.0',
1562+ maintainer=maintainer.preferredemail, architecture='all',
1563+ files='foo.py', binaries='foo.py'))
1564+
1565+ [spph] = series.main_archive.getPublishedSources()
1566+ self.assertEqual(PackagePublishingStatus.PUBLISHED, spph.status)
1567+
1568+
1569+class TestBinaryPackagePublisher(TestCaseWithFactory):
1570+
1571+ layer = ZopelessDatabaseLayer
1572+
1573+ def test_publish_creates_published_publication(self):
1574+ maintainer = self.factory.makePerson()
1575+ series = self.factory.makeDistroArchSeries()
1576+ section = self.factory.makeSection()
1577+ pocket = PackagePublishingPocket.RELEASE
1578+ bpr = self.factory.makeBinaryPackageRelease()
1579+
1580+ publisher = BinaryPackagePublisher(series, pocket, None)
1581+ publisher.publish(bpr, BinaryPackageData(
1582+ component='main', section=section.name, version='1.0',
1583+ maintainer=maintainer.preferredemail, architecture='all',
1584+ files='foo.py', binaries='foo.py', size=128, installed_size=1024,
1585+ md5sum='e83b5dd68079d727a494a469d40dc8db', description='test',
1586+ summary='Test!'))
1587+
1588+ [bpph] = series.main_archive.getAllPublishedBinaries()
1589+ self.assertEqual(PackagePublishingStatus.PUBLISHED, bpph.status)
1590
1591
1592 def test_suite():
1593- suite = unittest.TestSuite()
1594+ suite = TestLoader().loadTestsFromName(__name__)
1595 suite.addTest(DocTestSuite(lp.soyuz.scripts.gina.handlers))
1596 return suite
1597
1598=== modified file 'scripts/gina.py'
1599--- scripts/gina.py 2011-08-23 08:35:13 +0000
1600+++ scripts/gina.py 2011-09-09 09:33:30 +0000
1601@@ -38,6 +38,7 @@
1602 MangledArchiveError,
1603 PackagesMap,
1604 )
1605+from lp.soyuz.scripts.gina.dominate import dominate_imported_source_packages
1606 from lp.soyuz.scripts.gina.handlers import (
1607 DataSetupError,
1608 ImporterHandler,
1609@@ -152,6 +153,10 @@
1610 packages_map, kdb, package_root, keyrings, importer_handler)
1611 importer_handler.commit()
1612
1613+ # XXX JeroenVermeulen 2011-09-07 bug=843728: Dominate binaries as well.
1614+ dominate_imported_source_packages(
1615+ log, distro, distroseries, pocket, packages_map)
1616+
1617 if source_only:
1618 log.info('Source only mode... done')
1619 return
1620@@ -209,9 +214,8 @@
1621 npacks = len(packages_map.src_map)
1622 log.info('%i Source Packages to be imported', npacks)
1623
1624- for list_source in sorted(
1625- packages_map.src_map.values(), key=lambda x: x[0].get("Package")):
1626- for source in list_source:
1627+ for package in sorted(packages_map.src_map.iterkeys()):
1628+ for source in packages_map.src_map[package]:
1629 count += 1
1630 attempt_source_package_import(
1631 source, kdb, package_root, keyrings, importer_handler)
1632@@ -244,10 +248,9 @@
1633 log.info(
1634 '%i Binary Packages to be imported for %s', npacks, archtag)
1635 # Go over binarypackages importing them for this architecture
1636- for binary in sorted(packages_map.bin_map[archtag].values(),
1637- key=lambda x: x.get("Package")):
1638+ for package_name in sorted(packages_map.bin_map[archtag].iterkeys()):
1639+ binary = packages_map.bin_map[archtag][package_name]
1640 count += 1
1641- package_name = binary.get("Package", "unknown")
1642 try:
1643 try:
1644 do_one_binarypackage(binary, archtag, kdb, package_root,