Merge lp:~cjwatson/launchpad/stop-publishing-obsolete-series into lp:launchpad

[Colin Watson]

[r=rvb][bug=905281] Skip publishing OBSOLETE and FUTURE series in PRIMARY and PARTNER archives.

== Summary ==

About 15 seconds of the primary archive publisher's runtime is spent considering obsolete distroseries. It's not a huge amount, but it would be nice to shave this off.

== Proposed fix ==

Exclude OBSOLETE and FUTURE series when publishing PRIMARY and PARTNER archives. The archive purpose restriction is because apparently OEM still has PPAs for obsolete series, as noted by William Grant pre-implementation.

== Tests ==

bin/test -vvct archivepublisher.tests.test_publisher.TestPublisher

== Demo and Q/A ==

Run the publisher on mawson. Given the current state of the dogfood database, the publisher log under "Step A: Publishing packages" should show it considering rusty, precise, oneiric, natty, maverick, lucid, newdog, hardy, testagain, and dozing, but not q-series, karmic, jaunty, intrepid, gutsy, feisty, edgy, dapper, breezy, hoary, or warty.

== lint ==

None.

[Raphaël Badin (rvb)]

This looks good to land. Maybe you could merge testPublishingSkipsObsoletePrimarySeries and testPublishingSkipsFuturePrimarySeries into one single method to avoid code duplication.

review: Approve

=== modified file 'lib/lp/archivepublisher/publishing.py'
--- lib/lp/archivepublisher/publishing.py	2011-12-30 06:14:56 +0000
+++ lib/lp/archivepublisher/publishing.py	2012-01-03 01:07:25 +0000
@@ -39,6 +39,7 @@
     RepositoryIndexFile,
     )
 from lp.registry.interfaces.pocket import PackagePublishingPocket
+from lp.registry.interfaces.series import SeriesStatus
 from lp.services.database.sqlbase import sqlvalues
 from lp.services.librarian.client import LibrarianClient
 from lp.services.utils import file_exists
@@ -236,7 +237,28 @@
         """
         self.log.debug("* Step A: Publishing packages")
 
-        for distroseries in self.distro.series:
+        if self.archive.purpose in (
+            ArchivePurpose.PRIMARY,
+            ArchivePurpose.PARTNER,
+            ):
+            # For PRIMARY and PARTNER archives, skip OBSOLETE and FUTURE
+            # series.  We will never want to publish anything in them, so it
+            # isn't worth thinking about whether they have pending
+            # publications.
+            consider_series = [
+                series
+                for series in self.distro.series
+                if series.status not in (
+                    SeriesStatus.OBSOLETE,
+                    SeriesStatus.FUTURE,
+                    )]
+        else:
+            # Other archives may have reasons to continue building at least
+            # for OBSOLETE series.  For example, a PPA may be continuing to
+            # provide custom builds for users who haven't upgraded yet.
+            consider_series = self.distro.series
+
+        for distroseries in consider_series:
             for pocket in self.archive.getPockets():
                 allowed = (
                     not self.allowed_suites or

=== modified file 'lib/lp/archivepublisher/tests/test_publisher.py'
--- lib/lp/archivepublisher/tests/test_publisher.py	2011-12-30 06:14:56 +0000
+++ lib/lp/archivepublisher/tests/test_publisher.py	2012-01-03 01:07:25 +0000
@@ -56,6 +56,7 @@
 from lp.soyuz.interfaces.archive import IArchiveSet
 from lp.soyuz.tests.test_publishing import TestNativePublishingBase
 from lp.testing import TestCaseWithFactory
+from lp.testing.fakemethod import FakeMethod
 from lp.testing.gpgkeys import gpgkeysdir
 from lp.testing.keyserver import KeyServerTac
 from lp.testing.layers import ZopelessDatabaseLayer
@@ -431,6 +432,45 @@
         # remove locally created dir
         shutil.rmtree(test_pool_dir)
 
+    def testPublishingSkipsObsoleteFuturePrimarySeries(self):
+        """Publisher skips OBSOLETE/FUTURE series in PRIMARY archives."""
+        publisher = Publisher(
+            self.logger, self.config, self.disk_pool,
+            self.ubuntutest.main_archive)
+        # Remove security proxy so that the publisher can call our fake
+        # method.
+        publisher.distro = removeSecurityProxy(publisher.distro)
+
+        for status in (SeriesStatus.OBSOLETE, SeriesStatus.FUTURE):
+            naked_breezy_autotest = publisher.distro['breezy-autotest']
+            naked_breezy_autotest.status = status
+            naked_breezy_autotest.publish = FakeMethod(result=set())
+
+            publisher.A_publish(False)
+
+            self.assertEqual(0, naked_breezy_autotest.publish.call_count)
+
+    def testPublishingConsidersObsoleteFuturePPASeries(self):
+        """Publisher does not skip OBSOLETE/FUTURE series in PPA archives."""
+        ubuntu_team = getUtility(IPersonSet).getByName('ubuntu-team')
+        test_archive = getUtility(IArchiveSet).new(
+            distribution=self.ubuntutest, owner=ubuntu_team,
+            purpose=ArchivePurpose.PPA)
+        publisher = Publisher(
+            self.logger, self.config, self.disk_pool, test_archive)
+        # Remove security proxy so that the publisher can call our fake
+        # method.
+        publisher.distro = removeSecurityProxy(publisher.distro)
+
+        for status in (SeriesStatus.OBSOLETE, SeriesStatus.FUTURE):
+            naked_breezy_autotest = publisher.distro['breezy-autotest']
+            naked_breezy_autotest.status = status
+            naked_breezy_autotest.publish = FakeMethod(result=set())
+
+            publisher.A_publish(False)
+
+            self.assertEqual(1, naked_breezy_autotest.publish.call_count)
+
     def testPublisherBuilderFunctions(self):
         """Publisher can be initialized via provided helper function.