Merge lp:~sinzui/launchpad/registry-tales-2 into lp:launchpad

[Curtis Hovey]

This is my branch to adjust the cache rules for registry pages.

    lp:~sinzui/launchpad/registry-tales-2
    Diff size:
    Launchpad bug:
          https://bugs.launchpad.net/bugs/599614
          https://bugs.launchpad.net/bugs/599818
    Test command: ./bin/test -vv \
          -t milestone-views -t packaging-views \
          -t xx-sourcepackage-packaging -t xx-show-distroseries-packaging
    Pre-implementation: no one
    Target release: 10.06

Adjust the cache rules for registry pages
-----------------------------------------

Bug #599614 [bad caching in milestone page]
    "Assigned to you" <- the data is user-specific, pubic, not private

Bug #599818 [distroseries +packaging and +needs-packaging could cache]
    Recent schema changes make data retrieval for +packaging and +needs-
    packaging much faster, but packages are added/removed from +packaging and
    +needs-packaging about 10 times a day. The queries rarely return *new*
    information. A 30 minute public cache will help the pages display quicker.

Rules
-----

Bug #599614 [bad caching in milestone page]
    * Use "private" cache so that the "Assigned to you" information is really
      about you.
    * Add a test for this since it is now understood that the information
      is personalised.

Bug #599818 [distroseries +packaging and +needs-packaging could cache]
    * Add tal:content="cache:public, 30 minute" to +packaging and
      +needs-packaging.

QA
--

Bug #599614 [bad caching in milestone page]
    * As anonymous, visit an old milestone that you know you were assigned
      bug.
    * Verify it states that no bugs or blueprints are assigned to you.
    * Login.
    * Verify the page states the number of bugs or blueprints assigned to you.

Bug #599818 [distroseries +packaging and +needs-packaging could cache]
    * Visit https://edge.launchpad.net/ubuntu/maverick/+needs-packaging
    * Load the page 5 times and verify it does not timeout once it renders
      (staging will probably not timeout because if the schema changes)
    * Visit https://edge.launchpad.net/ubuntu/maverick/+packaging
    * Load the page 5 times and verify it does not timeout once it renders

Lint
----

Linting changed files:
  lib/lp/registry/browser/tests/milestone-views.txt
  lib/lp/registry/templates/distroseries-needs-packaging.pt
  lib/lp/registry/templates/distroseries-packaging.pt
  lib/lp/registry/templates/milestone-index.pt

Test
----

    * lib/lp/registry/browser/tests/milestone-views.txt
      * Revised the test to work with only the content under test.
      * Added a test for the cache rules of the activity portlet.

Implementation
--------------

    * lib/lp/registry/templates/distroseries-needs-packaging.pt
      * Added a 30 minute public cache.
    * lib/lp/registry/templates/distroseries-packaging.pt
      * Added a 30 minute public cache.
    * lib/lp/registry/templates/milestone-index.pt
      * Changed the cache rules from public to private.

[Leonard Richardson (leonardr)]

review: Approve

[Martin Pool (mbp)]

If I had fixed this I would have kept the overall project counts cached as public, and just made the assigned bugs count private. I don't have a good feel for whether the cost of recalculating them actually makes that worthwhile.

[Curtis Hovey (sinzui)]

The calculations appear to be very expensive because of the possibilities of private bugs. The loop that creates the info, while heavily cached and optimised to make minimal passes through the list often appears in timeouts. Bug privacy truly sucks. I prefer to cache the whole thing because I would like to go one month without complaints of timeouts.

=== modified file 'lib/lp/registry/browser/tests/milestone-views.txt'
--- lib/lp/registry/browser/tests/milestone-views.txt	2010-06-24 17:59:02 +0000
+++ lib/lp/registry/browser/tests/milestone-views.txt	2010-06-29 15:10:49 +0000
@@ -248,7 +248,7 @@
 the product release data for a milestone.
 
     >>> from canonical.launchpad.testing.pages import (
-    ...     extract_text, find_tag_by_id)
+    ...     extract_text, find_main_content, find_tag_by_id)
 
     >>> view = create_view(
     ...     milestone, '+productrelease-data', principal=person)
@@ -307,6 +307,7 @@
 Active milestones cache bugtasks for 10 minutes.
 
     >>> from canonical.testing.layers import MemcachedLayer
+
     >>> MemcachedLayer.purge()
     >>> cached_milestone = factory.makeMilestone(
     ...     'cache-test', productseries=series)
@@ -315,11 +316,21 @@
     >>> view = create_view(cached_milestone, '+index', principal=person)
     >>> view.expire_cache_minutes
     10
+
     >>> 'Cache hit' in view.render()
     False
-    >>> print create_view(
-    ...     cached_milestone, '+index', principal=person).render()
-    <...<!-- Cache hit: memcache expression
+
+    >>> content = find_main_content(create_view(
+    ...     cached_milestone, '+index', principal=person).render())
+    >>> print find_tag_by_id(content, 'milestone-activities')
+    <div id="milestone-activities" ...<!--
+      Cache hit: memcache expression
+      (private, view/expire_cache_minutes minute) [600 seconds] ...
+    <dt>Assigned to you:</dt> ... No blueprints or bugs assigned to you. ...
+
+    >>> print find_tag_by_id(content, 'milestone-specs-bugs')
+    <div id="milestone-specs-bugs" ...
+    <!-- Cache hit: memcache expression
       (private, view/expire_cache_minutes minute) [600 seconds] -->
     <table class="listing sortable" id="milestone_bugtasks">...
 

=== modified file 'lib/lp/registry/templates/distroseries-needs-packaging.pt'
--- lib/lp/registry/templates/distroseries-needs-packaging.pt	2010-01-30 18:15:36 +0000
+++ lib/lp/registry/templates/distroseries-needs-packaging.pt	2010-06-29 15:10:49 +0000
@@ -47,7 +47,8 @@
           </thead>
 
           <tbody>
-            <tr tal:repeat="summary summaries/batch">
+            <tr tal:repeat="summary summaries/batch"
+              tal:content="cache:public, 30 minute">
               <td>
                 <a class="sprite package-source"
                   tal:attributes="href summary/package/fmt:url"

=== modified file 'lib/lp/registry/templates/distroseries-packaging.pt'
--- lib/lp/registry/templates/distroseries-packaging.pt	2010-02-26 22:59:30 +0000
+++ lib/lp/registry/templates/distroseries-packaging.pt	2010-06-29 15:10:49 +0000
@@ -31,7 +31,8 @@
         </ul>
       </div>
 
-      <div tal:condition="packagings/batch">
+      <div tal:condition="packagings/batch"
+        tal:content="cache:public, 30 minute">
         <h2>All upstream links</h2>
 
         <div class="results"

=== modified file 'lib/lp/registry/templates/milestone-index.pt'
--- lib/lp/registry/templates/milestone-index.pt	2010-06-24 18:04:24 +0000
+++ lib/lp/registry/templates/milestone-index.pt	2010-06-29 15:10:49 +0000
@@ -128,8 +128,8 @@
         </div>
 
         <div class="yui-u">
-          <div class="portlet"
-            tal:content="cache:public, view/expire_cache_minutes minute">
+          <div id="milestone-activities" class="portlet"
+            tal:content="cache:private, view/expire_cache_minutes minute">
             <h2>Activities</h2>
 
             <dl>
@@ -197,7 +197,7 @@
       tal:condition="view/release"
       tal:content="structure view/milestone/@@+productrelease-data" />
 
-    <div class="portlet full-page-width">
+    <div id="milestone-specs-bugs" class="portlet full-page-width">
         <h2>
           <span id="specification-count" style="color: #3594bb;"
             tal:content="structure view/specification_count_text">2</span>