Merge lp:~bac/launchpad/bug-652280-pg-trans into lp:launchpad

[Brad Crittenden]

= Summary =

The +translations page for project groups did not have the
"noindex,nofollow" meta data. It was added as was a new test that tests
all translations pages to ensure the meta data is included properly.

== Proposed fix ==

As above.

== Pre-implementation notes ==

Talks with Curtis and detailed work with Henning.

== Implementation details ==

It's pretty straightforward, as above. However, for distroseries
getting a view via 'create_initialized_view' and trying to render it
does not work, due, I think, to the way the pages are registered in the
ZCML with respect to layers and facets. The template uses the
navigation menus which are not adapted properly.

In the end I punted and used getUserBrowser in order to get the rendered
content of the pages which is a fine approach but leaves the mystery of
why getting the view directly and rendering it does not work. Is there
a hidden bug that I've discovered but didn't crack?

== Tests ==

bin/test -vvm lp.translations -t test_noindex

== Demo and Q/A ==

Go to +translations and inspect the HTML for the anti-robots meta
instructions.

= Launchpad lint =

Checking for conflicts and issues in changed files.

Linting changed files:
  lib/lp/translations/templates/distroseries-translations.pt
  lib/lp/translations/browser/project.py
  lib/lp/translations/browser/tests/test_noindex.py
  lib/lp/testing/views.py
  lib/lp/translations/templates/project-translations.pt

[Jeroen T. Vermeulen (jtv)]

Hi Brad,

Nice to see a branch with a solid cover letter again.

You found some interesting bugs in Storm's SQLObject compatibility layer while working on this: is_empty returns the inverse of what it should, and then __nonzero__ compounds that sin by inverting it again. I hope you'll file a bug about this.

I have no objections to the code overall. There are a few things in the test that I'd like to see fixed, but nothing to stop an approval:

 * You use lower_case_with_underscores for their helper methods, whereas our coding standard mandates sayItWithCapitals for methods (but methods only). The requirement may be going away though; I haven't kept track.

 * You pointed out that there's a redundant get_rendered_contents in the products test. To be removed.

 * You also pointed out that there's already a has_translatables on your context object, so no need to duplicate it in the view.

 * Why do you bother making "target" and "naked_translatable" properties instead of attributes that you initialize in setUp? I do appreciate the docstring but a comment would be fine as far as I'm concerned. You could scratch self.product, self.projectgroup, self.distro, and self.distroseries to make up for it; they would become unnecessary.

 * I like the orthogonal test setup, with a mixin for the test scenarios and leaf classes for the different alternate setups. But I find the inheritance graph a little confusing. I think it'd be more manageable if the actual test cases were all leaf classes, without test cases inheriting from other classes that will also run as separate test cases. I'd also put the layer specifications in those leaf classes, so that there's a clear separation between the classes that "go into" a test case and the individual test cases.

 * You have your own verification methods (verify_robots_are_blocked, verify_robots_are_not_blocked) that invoke other assertion methods. I normally avoid this myself because it hides the "where did it go wrong" information deeper in the traceback, but looking at this I quite like the way the naming documents the intent of the assertion.

 * Isn't testing for exactly "noindex,nofollow" more brittle than needed? Would "nofollow,noindex" be equally valid? If so, it'd be more robust to assertContentEqual on robots['content'].split(','). I'll admit it's far-fetched, but brittle tests have given us enough headaches to make us worry about these things.

 * Out of interest, why do you pass rootsite to create_initialized_view for distroseries but not for productseries?

Jeroen

review: Approve

[Jeroen T. Vermeulen (jtv)]

review: Approve (code)

[Brad Crittenden (bac)]

On Oct 12, 2010, at 18:34 , Jeroen T. Vermeulen wrote:

> Review: Approve
> Hi Brad,

Thanks for the review Jeroen.

>
> Nice to see a branch with a solid cover letter again.

'bzr lp-send'

It's so easy I'm not sure why everyone doesn't do it.

>
> You found some interesting bugs in Storm's SQLObject compatibility layer while working on this: is_empty returns the inverse of what it should, and then __nonzero__ compounds that sin by inverting it again. I hope you'll file a bug about this.

Filed and fixed by you in record time! Thanks for the follow through.

>
> * You use lower_case_with_underscores for their helper methods, whereas our coding standard mandates sayItWithCapitals for methods (but methods only). The requirement may be going away though; I haven't kept track.

No, I was just not paying attention. Fixed.

>
> * You pointed out that there's a redundant get_rendered_contents in the products test. To be removed.

Gone.

>
> * You also pointed out that there's already a has_translatables on your context object, so no need to duplicate it in the view.

Gone along with the view tests. Added missing tests to doc/projectgroup.txt. (Renamed from project.txt.)

>
> * Why do you bother making "target" and "naked_translatable" properties instead of attributes that you initialize in setUp? I do appreciate the docstring but a comment would be fine as far as I'm concerned. You could scratch self.product, self.projectgroup, self.distro, and self.distroseries to make up for it; they would become unnecessary.

I did that before I converted the base class to a mixin to force the base to throw a NotImplementedError for those as I thought it was more explicit than a comment stating the contract. Overkill? Probably, so I have removed it.

>
> * I like the orthogonal test setup, with a mixin for the test scenarios and leaf classes for the different alternate setups. But I find the inheritance graph a little confusing. I think it'd be more manageable if the actual test cases were all leaf classes, without test cases inheriting from other classes that will also run as separate test cases. I'd also put the layer specifications in those leaf classes, so that there's a clear separation between the classes that "go into" a test case and the individual test cases.

OK, changed to all extend BrowserTestCase and the mixin directly. I disagree with setting the layer on each one, though, as it is the same. That just seems redundant and silly.

>
> * You have your own verification methods (verify_robots_are_blocked, verify_robots_are_not_blocked) that invoke other assertion methods. I normally avoid this myself because it hides the "where did it go wrong" information deeper in the traceback, but looking at this I quite like the way the naming documents the intent of the assertion.
>
> * Isn't testing for exactly "noindex,nofollow" more brittle than needed? Would "nofollow,noindex" be equally valid? If so, it'd be more robust to assertContentEqual on robots['content'].split(','). I'll admit it's far-fetched, but brittle tests have given us enough headaches to make us worry about these things.

Yes, that probably is too brittle. I'll ...

[Jeroen T. Vermeulen (jtv)]

This looks just fine to me. Nice to see how much boilerplate you were able to strip off the tests. The layers thing was just a suggestion; I'm fine with your choice.

[Brad Crittenden (bac)]

Hi Curtis,

This branch includes a lot of good stuff...but it only addresses one part of the original bug. There was disagreement between you and Danilo about what UI changes needed to be done. What are your current thoughts?

[Curtis Hovey (sinzui)]

The Translations team is also changing these pages. There is not need to change something that the team will remove or alter in a few weeks anyway.

[Robert Collins (lifeless)]

I'm very happy with one test:

169 + def test_service_status_controls_robots(self):
170 + self.verifyRobotsAreBlocked(ServiceUsage.UNKNOWN)
173 + self.verifyRobotsAreBlocked(ServiceUsage.EXTERNAL)
176 + self.verifyRobotsAreBlocked(ServiceUsage.NOT_APPLICABLE)
179 + self.verifyRobotsNotBlocked(ServiceUsage.LAUNCHPAD)

or however you wish to spell it; using cachedproperty should be ok if you want to, but be sure to do browser.open() after you change the service usage. How many seconds difference does the cachedproperty make (perhaps we need a bug on the performance of canonical_url?)

review: Approve

=== renamed file 'lib/lp/registry/doc/project.txt' => 'lib/lp/registry/doc/projectgroup.txt'
--- lib/lp/registry/doc/project.txt	2010-10-09 16:36:22 +0000
+++ lib/lp/registry/doc/projectgroup.txt	2010-10-18 02:57:48 +0000
@@ -331,10 +331,21 @@
     >>> import transaction
     >>> transaction.abort()
 
+A project group with no translatable products is shown by
+'has_translatables' being false.
+
+    >>> product = factory.makeProduct()
+    >>> project_group = factory.makeProject()
+    >>> product.project = project_group
+    >>> project_group.has_translatable()
+    False
+
 GNOME Project is a good example that has translations.
 It has one translatable product.
 
     >>> gnome = getUtility(IProjectGroupSet)['gnome']
+    >>> gnome.has_translatable()
+    True
     >>> translatables = gnome.translatables()
     >>> translatables.count()
     1

=== modified file 'lib/lp/registry/model/projectgroup.py'
--- lib/lp/registry/model/projectgroup.py	2010-10-07 13:26:53 +0000
+++ lib/lp/registry/model/projectgroup.py	2010-10-18 02:57:48 +0000
@@ -206,9 +206,11 @@
 
     def translatables(self):
         """See `IProjectGroup`."""
-        # XXX j.c.sackett 2010-08-30 bug=627631 Once data migration has
+        # XXX j.c.sackett 2010-08-30 bug=627631: Once data migration has
         # happened for the usage enums, this sql needs to be updated to
-        # check for the translations_usage, not official_rosetta.
+        # check for the translations_usage, not official_rosetta.  At that
+        # time it should also be converted to a Storm query and the issue with
+        # has_translatables resolved.
         return Product.select('''
             Product.project = %s AND
             Product.official_rosetta = TRUE AND
@@ -220,6 +222,10 @@
 
     def has_translatable(self):
         """See `IProjectGroup`."""
+        # XXX: BradCrittenden 2010-10-12 bug=659078: The test should be
+        # converted to use is_empty but the implementation in storm's
+        # sqlobject wrapper is broken.
+        # return not self.translatables().is_empty()
         return self.translatables().count() != 0
 
     def _getBaseQueryAndClauseTablesForQueryingSprints(self):

=== modified file 'lib/lp/testing/views.py'
--- lib/lp/testing/views.py	2010-09-19 03:09:49 +0000
+++ lib/lp/testing/views.py	2010-10-18 02:57:48 +0000
@@ -85,7 +85,8 @@
 def create_initialized_view(context, name, form=None, layer=None,
                             server_url=None, method=None, principal=None,
                             query_string=None, cookie=None, request=None,
-                            path_info='/', rootsite=None):
+                            path_info='/', rootsite=None,
+                            current_request=False):
     """Return a view that has already been initialized."""
     if method is None:
         if form is None:
@@ -94,7 +95,8 @@
             method = 'POST'
     view = create_view(
         context, name, form, layer, server_url, method, principal,
-        query_string, cookie, request, path_info, rootsite=rootsite)
+        query_string, cookie, request, path_info, rootsite=rootsite,
+        current_request=current_request)
     view.initialize()
     return view
 

=== modified file 'lib/lp/translations/browser/project.py'
--- lib/lp/translations/browser/project.py	2010-08-20 20:31:18 +0000
+++ lib/lp/translations/browser/project.py	2010-10-18 02:57:48 +0000
@@ -21,6 +21,7 @@
 from canonical.launchpad.webapp.menu import NavigationMenu
 from lp.registry.browser.project import ProjectEditView
 from lp.registry.interfaces.projectgroup import IProjectGroup
+from lp.services.propertycache import cachedproperty
 from lp.translations.browser.translations import TranslationsMixin
 
 

=== added file 'lib/lp/translations/browser/tests/test_noindex.py'
--- lib/lp/translations/browser/tests/test_noindex.py	1970-01-01 00:00:00 +0000
+++ lib/lp/translations/browser/tests/test_noindex.py	2010-10-18 02:57:48 +0000
@@ -0,0 +1,152 @@
+# Copyright 2010 Canonical Ltd.  This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+__metaclass__ = type
+
+
+from BeautifulSoup import BeautifulSoup
+from zope.security.proxy import removeSecurityProxy
+
+from canonical.launchpad.webapp import canonical_url
+from canonical.testing.layers import DatabaseFunctionalLayer
+from lp.app.enums import ServiceUsage
+from lp.testing import (
+    BrowserTestCase,
+    login_person,
+    )
+from lp.services.propertycache import cachedproperty
+
+
+class TestRobotsMixin:
+    """Test the inclusion of the meta "noindex,nofollow" directives.
+
+    Subclasses using this mixin must set the following attributes:
+    target - the pillar under test
+    naked_translatable - the translatable object for the test, with the
+    security proxy removed.  It may be the target or a subordinate object.
+    (For example when testing a ProjectGroup the translatable is one of the
+    products in the project group.)
+    """
+
+    layer = DatabaseFunctionalLayer
+
+    def setUsage(self, usage):
+        self.naked_translatable.translations_usage = usage
+
+    @cachedproperty
+    def url(self):
+        return canonical_url(self.target, rootsite='translations')
+
+    @cachedproperty
+    def user_browser(self):
+        return self.getUserBrowser(user=self.user)
+
+    def getRenderedContents(self):
+        """Return an initialized view's rendered contents."""
+        # Using create_initialized_view for distroseries causes an error when
+        # rendering the view due to the way the view is registered and menus
+        # are adapted.  Getting the contents via a browser does work.
+        self.user_browser.open(self.url)
+        return self.user_browser.contents
+
+    def getRobotsDirective(self):
+        contents = self.getRenderedContents()
+        soup = BeautifulSoup(contents)
+        return soup.find('meta', attrs={'name': 'robots'})
+
+    def verifyRobotsAreBlocked(self, usage):
+        self.setUsage(usage)
+        robots = self.getRobotsDirective()
+        self.assertIsNot(None, robots,
+                         "Robot blocking meta information not present.")
+        self.assertEqual('noindex,nofollow', robots['content'])
+        expected = ('noindex', 'nofollow')
+        actual = robots['content'].split(',')
+        self.assertContentEqual(expected, actual)
+
+    def verifyRobotsNotBlocked(self, usage):
+        self.setUsage(usage)
+        robots = self.getRobotsDirective()
+        self.assertIs(
+            None, robots,
+            "Robot blocking metadata present when it should not be.")
+
+    def test_robots(self):
+        # Robots are blocked for usage that is not Launchpad.
+        self.verifyRobotsAreBlocked(ServiceUsage.UNKNOWN)
+        self.verifyRobotsAreBlocked(ServiceUsage.EXTERNAL)
+        self.verifyRobotsAreBlocked(ServiceUsage.NOT_APPLICABLE)
+        # Robots are not blocked for Launchpad usage.
+        self.verifyRobotsNotBlocked(ServiceUsage.LAUNCHPAD)
+
+
+class TestRobotsProduct(BrowserTestCase, TestRobotsMixin):
+    """Test noindex,nofollow for products."""
+
+    def setUp(self):
+        super(TestRobotsProduct, self).setUp()
+        self.target = self.factory.makeProduct()
+        self.factory.makePOTemplate(
+            productseries=self.target.development_focus)
+        self.naked_translatable = removeSecurityProxy(self.target)
+
+
+class TestRobotsProjectGroup(BrowserTestCase, TestRobotsMixin):
+    """Test noindex,nofollow for project groups."""
+
+    def setUp(self):
+        super(TestRobotsProjectGroup, self).setUp()
+        self.target = self.factory.makeProject()
+        self.product = self.factory.makeProduct()
+        self.factory.makePOTemplate(
+            productseries=self.product.development_focus)
+        self.naked_translatable = removeSecurityProxy(self.product)
+        self.naked_translatable.project = self.target
+
+
+class TestRobotsProductSeries(BrowserTestCase, TestRobotsMixin):
+    """Test noindex,nofollow for product series."""
+
+    def setUp(self):
+        super(TestRobotsProductSeries, self).setUp()
+        self.product = self.factory.makeProduct()
+        self.target = self.product.development_focus
+        self.factory.makePOTemplate(
+            productseries=self.target)
+        self.naked_translatable = removeSecurityProxy(self.product)
+
+
+class TestRobotsDistroSeries(BrowserTestCase, TestRobotsMixin):
+    """Test noindex,nofollow for distro series."""
+
+    def setUp(self):
+        super(TestRobotsDistroSeries, self).setUp()
+        login_person(self.user)
+        self.distro = self.factory.makeDistribution(
+            name="whobuntu", owner=self.user)
+        self.target = self.factory.makeDistroSeries(
+            name="zephyr", distribution=self.distro)
+        self.target.hide_all_translations = False
+        new_sourcepackagename = self.factory.makeSourcePackageName()
+        self.factory.makePOTemplate(
+            distroseries=self.target,
+            sourcepackagename=new_sourcepackagename)
+        self.naked_translatable = removeSecurityProxy(self.distro)
+
+
+class TestRobotsDistro(BrowserTestCase, TestRobotsMixin):
+    """Test noindex,nofollow for distro."""
+
+    def setUp(self):
+        super(TestRobotsDistro, self).setUp()
+        login_person(self.user)
+        self.target = self.factory.makeDistribution(
+            name="whobuntu", owner=self.user)
+        self.distroseries = self.factory.makeDistroSeries(
+            name="zephyr", distribution=self.target)
+        self.distroseries.hide_all_translations = False
+        new_sourcepackagename = self.factory.makeSourcePackageName()
+        self.factory.makePOTemplate(
+            distroseries=self.distroseries,
+            sourcepackagename=new_sourcepackagename)
+        self.naked_translatable = removeSecurityProxy(self.target)

=== modified file 'lib/lp/translations/templates/distroseries-translations.pt'
--- lib/lp/translations/templates/distroseries-translations.pt	2010-09-24 15:30:10 +0000
+++ lib/lp/translations/templates/distroseries-translations.pt	2010-10-18 02:57:48 +0000
@@ -10,10 +10,8 @@
       use-macro="context/@@+translations-macros/translations-js" />
     <metal:languages-table-js
       use-macro="context/@@+translations-macros/languages-table-js" />
-    <tal:robots
-      condition="not:context/translations_usage/enumvalue:LAUNCHPAD">
-      <meta name="robots" content="noindex,nofollow" />
-    </tal:robots>
+    <meta tal:condition="not:context/translations_usage/enumvalue:LAUNCHPAD"
+          name="robots" content="noindex,nofollow" />
   </div>
   </head>
   <body>
@@ -156,4 +154,3 @@
     </div>
   </body>
 </html>
-

=== modified file 'lib/lp/translations/templates/project-translations.pt'
--- lib/lp/translations/templates/project-translations.pt	2010-04-19 08:11:52 +0000
+++ lib/lp/translations/templates/project-translations.pt	2010-10-18 02:57:48 +0000
@@ -4,16 +4,26 @@
   xmlns:metal="http://xml.zope.org/namespaces/metal"
   metal:use-macro="view/macro:page/main_only"
 >
+
+<head>
+  <tal:head_epilogue metal:fill-slot="head_epilogue">
+    <meta tal:condition="not:context/has_translatable"
+          name="robots" content="noindex,nofollow" />
+  </tal:head_epilogue>
+</head>
+
 <body>
   <div metal:fill-slot="main">
 
     <div class="top-portlet">
-      <p tal:condition="context/translatables">
+      <p tal:condition="context/has_translatable">
         Select a project you want to translate into your own language.
       </p>
-      <p tal:condition="not:context/translatables">
+      <p tal:condition="not:context/has_translatable">
+        <strong>
         There are no translatable projects in
         <span tal:content="context/displayname">Project Group</span>.
+        </strong>
       </p>
       <p tal:condition="not:view/required:launchpad.Edit">
         If a project for <span tal:content="context/displayname">Project Group
@@ -34,7 +44,7 @@
       <div class="yui-u first">
         <div class="portlet" id="translatable-projects">
           <h3>Translatable projects</h3>
-          <dl tal:condition="context/translatables">
+          <dl tal:condition="context/has_translatable">
             <tal:project repeat="product context/translatables">
               <dt>
                 <a tal:replace="structure product/fmt:link/+translations">
@@ -80,8 +90,6 @@
         <a class="add sprite" href="+newproduct">Add another project</a>
       </p>
     </div>
-
-
   </div><!--main-->
 </body>
 </html>