Merge lp:~sinzui/launchpad/obsolete-series-0 into lp:launchpad

[Curtis Hovey]

This is my branch to prevent timeout caused by the product-release-finder.

    lp:~sinzui/launchpad/obsolete-series-0
    Diff size:
    Launchpad bug:5
          https://bugs.launchpad.net/bugs/590806
          https://bugs.launchpad.net/bugs/490945
    Test command: ./bin/test -vv \
          -t productseries-views -t GetFiltersTestCase -t xx-project-registry
    Pre-implementation: no one.
    Target release: 10.06

prevent timeout caused by the product-release-finder
----------------------------------------------------

A review of the PRF revealed that it was downloading files and creating
released for series that belong to obsolete junk. These series are obsolete so
the PRF must ignore them. The releases were deleted when the series was moved
to obsolete junk. There are timeouts on obsolete-junks pages because there
PRF recreated information that users wanted gone.

ADDENDUM Bug #490945 [obsolete series takes precedence over stable]
    If I move milestone 1.5, for instance, to obsolete instead of stable, the
    main download page ( https://launchpad.net/kabikaboo/ ) changes the
    primary download to the newest obsolete download

Rules
-----

    * Ensure the file release glob is set to Null during series delete
    * Ensure that the series status is set to Obsolete
    * Ensure that the PRF ignores Obsolete series
    * Export releasefileglob to API so that problem series can be fixed
      before 10.06 is released.

ADDENDUM Bug #490945 [obsolete series takes precedence over stable]
    * latest_release_with_download_files() should use
      sorted_active_series_list() to filter out obsolete series.

QA
--

    * Verify that a deletes series is marked obsolete and that the
      filereleaseglob is set to None
    * Verify that the release_file_glob is visible:
        from lpscripts import lp_factory
        from launchpadlib.launchpad import (
            Launchpad, EDGE_SERVICE_ROOT, STAGING_SERVICE_ROOT)
        lp = lp_factory('edge', app_name='testing')
        project = lp.projects['gedit']
        series = project.getSeries(name='master')
        print series.name
        print series.release_file_glob

ADDENDUM Bug #490945 [obsolete series takes precedence over stable]
    * Visit a project with a release shown of the front page.
    * Mark the series obsolete
    * Verify the release changes on the front page.

Lint
----

Linting changed files:
  lib/lp/registry/browser/__init__.py
  lib/lp/registry/browser/product.py
  lib/lp/registry/browser/tests/product-files-views.txt
  lib/lp/registry/browser/tests/productseries-views.txt
  lib/lp/registry/interfaces/productseries.py
  lib/lp/registry/scripts/productreleasefinder/finder.py
  lib/lp/registry/stories/webservice/xx-project-registry.txt
  lib/lp/registry/tests/test_prf_finder.py

Test
----
    * lib/lp/registry/browser/tests/product-files-views.txt
      * Verify that obsolete releases are not shown on the project's index
        page.
    * lib/lp/registry/browser/tests/productseries-views.txt
      * Verify that the "deleted" series was marked obsolete and the
        relasefileglob was set to None.
    * lib/lp/registry/stories/webservice/xx-project-registry.txt
      * Verify that release_file_glob is exported.
    * lib/lp/registry/tests/test_prf_finder.py
      * Verify that obsolete series are not included in the finder filters.

Implementation
--------------

    * lib/lp/registry/browser/__init__.py
      * Marked the series as obsolete and clear the releasefileglob when
        the user deletes (moves) the series.
    * lib/lp/registry/browser/product.py
      * Use sorted_active_series_list in latest_release_with_download_files().
    * lib/lp/registry/interfaces/productseries.py
      * Export release_file_glob so that I can see and fix the damage.
    * lib/lp/registry/scripts/productreleasefinder/finder.py
      * Skip obsolete series.

[Brad Crittenden (bac)]

Hi Curtis, thanks for this branch.

On IRC we discussed the following:

* Test update to lib/lp/registry/browser/tests/product-files-views.txt to properly show the latest release with downloads. (http://pastebin.ubuntu.com/448271/)

Also,

* I hate 'releasefileglob' but it is too ingrained to change.

* What would you think about adding a launchpadlib test to exercise the newly exported release_file_glob? On the one hand it would be a dumb test showing just that one field out of a zillion. On the other, we've got to start somewhere.

review: Approve (code)

=== modified file 'lib/lp/registry/browser/__init__.py'
--- lib/lp/registry/browser/__init__.py	2010-05-14 14:55:26 +0000
+++ lib/lp/registry/browser/__init__.py	2010-06-12 04:32:28 +0000
@@ -22,6 +22,7 @@
 
 from lp.bugs.interfaces.bugtask import BugTaskSearchParams, IBugTaskSet
 from lp.registry.interfaces.productseries import IProductSeries
+from lp.registry.interfaces.series import SeriesStatus
 from canonical.launchpad.interfaces.launchpad import ILaunchpadCelebrities
 from canonical.launchpad.webapp.launchpadform import (
     action, LaunchpadEditFormView)
@@ -207,6 +208,8 @@
         date_time = series.datecreated.strftime('%Y%m%d-%H%M%S')
         series.name = '%s-%s-%s' % (
             series.product.name, series.name, date_time)
+        series.status = SeriesStatus.OBSOLETE
+        series.releasefileglob = None
         series.product = getUtility(ILaunchpadCelebrities).obsolete_junk
 
     def _deleteMilestone(self, milestone):

=== modified file 'lib/lp/registry/browser/product.py'
--- lib/lp/registry/browser/product.py	2010-06-04 15:07:42 +0000
+++ lib/lp/registry/browser/product.py	2010-06-12 04:32:28 +0000
@@ -859,7 +859,7 @@
     @cachedproperty
     def latest_release_with_download_files(self):
         """Return the latest release with download files."""
-        for series in self.sorted_series_list:
+        for series in self.sorted_active_series_list:
             for release in series.releases:
                 if len(list(release.files)) > 0:
                     return release

=== modified file 'lib/lp/registry/browser/tests/product-files-views.txt'
--- lib/lp/registry/browser/tests/product-files-views.txt	2010-01-15 20:41:29 +0000
+++ lib/lp/registry/browser/tests/product-files-views.txt	2010-06-12 04:32:28 +0000
@@ -68,3 +68,23 @@
     Add download file to the s3 series for release: 3.3, 3.2, 3.1
     Add download file to the s2 series for release: 2.3, 2.2, 2.1
     Add download file to the s1 series for release: 1.3, 1.2, 1.1
+
+
+Product index
+-------------
+
+The product index view shows the latest release for the project.
+
+    >>> view = create_initialized_view(product, name='+index')
+    >>> print view.latest_release_with_download_files.version
+    4.3
+
+Obsolete series are ignored.
+
+    >>> from lp.registry.interfaces.series import SeriesStatus
+
+    >>> obsolete_series = product.getSeries('s4')
+    >>> obsolete_series.status = SeriesStatus.OBSOLETE
+    >>> view = create_initialized_view(product, name='+index')
+    >>> print view.latest_release_with_download_files.version
+    3.3

=== modified file 'lib/lp/registry/browser/tests/productseries-views.txt'
--- lib/lp/registry/browser/tests/productseries-views.txt	2010-05-06 18:13:55 +0000
+++ lib/lp/registry/browser/tests/productseries-views.txt	2010-06-12 04:32:28 +0000
@@ -259,6 +259,7 @@
     >>> product = factory.makeProduct(name="field", displayname='Field')
     >>> productseries = factory.makeProductSeries(
     ...     product=product, name='rabbit')
+    >>> productseries.filereleaseglob = 'http://eg.dom/rabbit/*'
 
     # Hack the creation date for testing purposes.
     >>> test_date = datetime(2009, 05, 01, 19, 34, 24, tzinfo=UTC)
@@ -467,6 +468,13 @@
     >>> print productseries.name
     field-rabbit-20090501-193424
 
+The series status is set to obsolete and the filereleaseglob was set to None.
+
+    >>> print productseries.status
+    Obsolete
+    >>> print productseries.filereleaseglob
+    None
+
 A series cannot be deleted if it is has translation templates.
 
     >>> translated_series = factory.makeProductSeries(product=product)

=== modified file 'lib/lp/registry/interfaces/productseries.py'
--- lib/lp/registry/interfaces/productseries.py	2010-05-17 09:35:18 +0000
+++ lib/lp/registry/interfaces/productseries.py	2010-06-12 04:32:28 +0000
@@ -260,12 +260,15 @@
         """Return the POTemplate with this name for the series."""
 
     # where are the tarballs released from this branch placed?
-    releasefileglob = TextLine(title=_("Release URL pattern"),
+    releasefileglob = exported(
+        TextLine(title=_("Release URL pattern"),
         required=False, constraint=validate_release_glob,
         description=_('A URL pattern that matches releases that are part '
                       'of this series.  Launchpad automatically scans this '
                       'site to import new releases.  Example: '
-                      'http://ftp.gnu.org/gnu/emacs/emacs-21.*.tar.gz'))
+                      'http://ftp.gnu.org/gnu/emacs/emacs-21.*.tar.gz')),
+        exported_as='release_finder_url_pattern')
+
     releaseverstyle = Attribute("The version numbering style for this "
         "series of releases.")
 

=== modified file 'lib/lp/registry/scripts/productreleasefinder/finder.py'
--- lib/lp/registry/scripts/productreleasefinder/finder.py	2009-12-09 11:47:58 +0000
+++ lib/lp/registry/scripts/productreleasefinder/finder.py	2010-06-12 04:32:28 +0000
@@ -22,6 +22,7 @@
 from canonical.launchpad.validators.name import invalid_name_pattern
 from canonical.launchpad.validators.version import sane_version
 
+from lp.registry.interfaces.series import SeriesStatus
 from lp.registry.interfaces.product import IProductSet
 from lp.registry.interfaces.productrelease import UpstreamFileType
 from lp.registry.scripts.productreleasefinder.hose import Hose
@@ -105,7 +106,8 @@
             filters = []
 
             for series in product.series:
-                if not series.releasefileglob:
+                if (series.status == SeriesStatus.OBSOLETE
+                    or not series.releasefileglob):
                     continue
 
                 filters.append(FilterPattern(series.name,

=== modified file 'lib/lp/registry/stories/team/xx-team-membership.txt'
--- lib/lp/registry/stories/team/xx-team-membership.txt	2010-06-08 12:54:44 +0000
+++ lib/lp/registry/stories/team/xx-team-membership.txt	2010-06-12 04:32:28 +0000
@@ -249,3 +249,11 @@
     >>> user_browser.getLink('Change mailing list subscriptions')
     <Link ... url='http://.../~no-priv/+editemails'>
 
+Teams also have a participation page, but it does not include a mailing
+list column.
+
+    >>> admin_browser.open('http://launchpad.dev/~admins/+participation')
+    >>> print extract_text(
+    ...     find_tag_by_id(admin_browser.contents, 'participation'))
+    Team                  Joined      Role    Via
+    Mailing List Experts  2007-10-04  Owner   &mdash;

=== modified file 'lib/lp/registry/stories/webservice/xx-project-registry.txt'
--- lib/lp/registry/stories/webservice/xx-project-registry.txt	2010-06-09 08:26:26 +0000
+++ lib/lp/registry/stories/webservice/xx-project-registry.txt	2010-06-12 04:32:28 +0000
@@ -876,6 +876,7 @@
     official_bug_tags: []
     owner_link: u'http://.../~babadoo-owner'
     project_link: u'http://.../babadoo'
+    release_finder_url_pattern: None
     releases_collection_link: u'http://.../babadoo/foobadoo/releases'
     resource_type_link: u'...'
     self_link: u'http://.../babadoo/foobadoo'

=== modified file 'lib/lp/registry/templates/person-participation.pt'
--- lib/lp/registry/templates/person-participation.pt	2010-06-07 23:12:44 +0000
+++ lib/lp/registry/templates/person-participation.pt	2010-06-12 04:32:28 +0000
@@ -28,7 +28,8 @@
           <th>Joined</th>
           <th>Role</th>
           <th>Via</th>
-          <th>Mailing List</th>
+          <th
+            tal:condition="not: context/teamowner">Mailing List</th>
         </tr>
       </thead>
       <tbody>
@@ -57,7 +58,8 @@
               &mdash;
             </tal:direct>
           </td>
-          <td>
+          <td
+            tal:condition="not: context/teamowner">
             <tal:subscribed condition="participation/team/mailing_list"
               replace="participation/subscribed">
               yes
@@ -70,17 +72,19 @@
       </tbody>
     </table>
 
-    <ul id="participation-actions" class="horizontal"
-      tal:condition="context/required:launchpad.Edit">
-      <li>
-        <a class="sprite add" href="/people/+newteam">Register a team</a>
-      </li>
-      <li>
-        <a class="sprite edit"
-          tal:attributes="href context/menu:overview/editemailaddresses/fmt:url"
-          >Change mailing list subscriptions</a>
-      </li>
-    </ul>
+    <tal:user condition="not: context/teamowner">
+      <ul id="participation-actions" class="horizontal"
+        tal:condition="context/required:launchpad.Edit">
+        <li>
+          <a class="sprite add" href="/people/+newteam">Register a team</a>
+        </li>
+        <li>
+          <a class="sprite edit"
+            tal:attributes="href context/menu:overview/editemailaddresses/fmt:url"
+            >Change mailing list subscriptions</a>
+        </li>
+      </ul>
+    </tal:user>
   </div>
 </body>
 </html>

=== modified file 'lib/lp/registry/tests/test_prf_finder.py'
--- lib/lp/registry/tests/test_prf_finder.py	2009-11-02 19:43:37 +0000
+++ lib/lp/registry/tests/test_prf_finder.py	2010-06-12 04:32:28 +0000
@@ -15,6 +15,7 @@
 from canonical.config import config
 from canonical.testing import LaunchpadZopelessLayer, reset_logging
 
+from lp.registry.interfaces.series import SeriesStatus
 from lp.registry.interfaces.product import IProductSet
 from lp.registry.interfaces.productrelease import (
     IProductReleaseFile, UpstreamFileType)
@@ -22,7 +23,7 @@
     FilterPattern)
 from lp.registry.scripts.productreleasefinder.finder import (
     extract_version, ProductReleaseFinder)
-
+from lp.testing import TestCaseWithFactory
 
 class FindReleasesTestCase(unittest.TestCase):
 
@@ -51,7 +52,7 @@
                          ('product2', ['filter3', 'filter4']))
 
 
-class GetFiltersTestCase(unittest.TestCase):
+class GetFiltersTestCase(TestCaseWithFactory):
 
     layer = LaunchpadZopelessLayer
 
@@ -87,6 +88,27 @@
             'http://ftp.gnome.org/pub/GNOME/sources/evolution/2.7/'
             'evolution-2.7.1.tar.gz'))
 
+    def test_getFilters_ignore_obsolete(self):
+        # Verify that obsolete series are ignnored.
+        ztm = self.layer.txn
+        ztm.begin()
+        product = self.factory.makeProduct(name="bunny")
+        active_series = product.getSeries('trunk')
+        active_series.releasefileglob  = 'http://eg.dom/bunny/trunk/*'
+        obsolete_series = self.factory.makeProductSeries(
+            product=product, name='rabbit')
+        obsolete_series.releasefileglob  = 'http://eg.dom/bunny/rabbit/*'
+        obsolete_series.status = SeriesStatus.OBSOLETE
+        ztm.commit()
+        logging.basicConfig(level=logging.CRITICAL)
+        prf = ProductReleaseFinder(ztm, logging.getLogger())
+        product_filters = prf.getFilters()
+        self.assertEqual(1, len(product_filters))
+        found_product, filters = product_filters[0]
+        self.assertEqual('bunny', found_product)
+        self.assertEqual(1, len(filters))
+        self.assertEqual(filters[0].key, 'trunk')
+
 
 class HandleProductTestCase(unittest.TestCase):