Merge lp:~wallyworld/launchpad/built-packages-listing into lp:launchpad

[Ian Booth]

[r=stevenk,thumper][ui=none][bug=671262] provide a listing of recently built packages from recipes

= Summary =

Bug 671262 - provide a listing of recently built packages from recipes

= Implementation =

URL for new view is http://code.launchpad.net/+daily-builds
A "Recent recipe builds..." link has been added to the footer of the code index page.
Displays recipe build records which have been built in the last 30 days.

New classes:

  IRecipeBuildRecordSet
  RecipeBuildRecordSet
    Loads recipe build records - the data model for the view.
    The records for the view are constructed as a named tuple.
    By default, records built in the last 30 days are loaded.

  RecipeBuildRecordResultSet
    Extends DecoratedResultSet to override the count() method so that it works properly with the group by query.

  CompletedDailyBuildsView
    The view class - displays source package name, recipe, recipe owner, archive, most recent build datetime.
    Supports display of build records where archive is PPA (in which case a link is rendered) or Primary (renders text).

  Page Template
    lp/code/templates/daily-builds-listing.pt

= Screenshot =

Listing page:

http://people.canonical.com/~ianb/dailyrecipebuildlisting.png

Code home page:

http://people.canonical.com/~ianb/CodeHomePage-Recipes.png

= Tests =

New tests:
lp.code.browser.tests.test_recipebuildlisting.py
  The test creates records with primary and ppa archives and also outside the default 30 day window.
xx-bazaar-home.txt
  Added tests to check content on code home page.

bin/test -vvt test_recipebuildslisting
bin/test -vvt xx-baxaar-home.txt

= Launchpad lint =

Checking for conflicts and issues in changed files.

Linting changed files:
  logs/
  lib/lp/code/configure.zcml
  lib/lp/code/browser/configure.zcml
  lib/lp/code/browser/recipebuildslisting.py
  lib/lp/code/browser/tests/test_recipebuildslisting.py
  lib/lp/code/interfaces/recipebuild.py
  lib/lp/code/model/recipebuild.py
  lib/lp/code/templates/bazaar-index.pt
  lib/lp/code/templates/daily-builds-listing.pt
  lib/lp/testing/__init__.py
  lib/lp/code/stories/branches/xx-bazaar-home.txt
       1: narrative uses a moin header.
      40: narrative uses a moin header.
      48: narrative uses a moin header.
      83: narrative uses a moin header.
     109: narrative uses a moin header.
     138: narrative uses a moin header.
  lib/lp/code/model/recipebuild.py
     150: E302 expected 2 blank lines, found 1
  lib/lp/testing/__init__.py
     130: 'anonymous_logged_in' imported but unused
     130: 'with_anonymous_login' imported but unused
     149: 'launchpadlib_for' imported but unused
     149: 'launchpadlib_credentials_for' imported but unused
     130: 'person_logged_in' imported but unused
     149: 'oauth_access_token_for' imported but unused
     130: 'login_celebrity' imported but unused
     130: 'with_celebrity_logged_in' imported but unused
     148: 'test_tales' imported but unused
     130: 'celebrity_logged_in' imported but unused
     130: 'run_with_login' imported but unused
     130: 'is_logged_in' imported but unused
     130: 'with_person_logged_in' imported but unused
     130: 'login_team' imported but unused
     130: 'login_person' imported but unused
     130: 'login_as' imported but unused
     430: E301 expected 1 blank line, found 0
     859: E301 expected 1 blank line, found 0
     885: E302 expected 2 blank lines, found 1
     961: E302 expected 2 blank lines, found 1
     110: Line exceeds 78 characters.

[Steve Kowalik (stevenk)]

Hi,

This mostly looks good. I have a few comments:

* Why are you including imports in the meat of the test, rather than with the rest of them at the top of the file?
* Would _makeRecipeBuildRecords() better exist in the factory so other tests could make use of it?
* Would it make sense to fix __eq__ on SourcePackageRecipe, rather than working around it?
* Use of IStoreSelector, which is deprecated. Could you look at switching to I{Master,}Store?
* Please update your XXX per the XXX policy.

review: Needs Fixing (code*)

[Ian Booth (wallyworld)]

Thanks Steve for the review.

>
> * Why are you including imports in the meat of the test, rather than with the rest of them at the top of the file?

"Personal preference". I wanted to try and avoid the top imports list
getting too large and cluttered with imports for things that were just
used in one localised place. I'll move them.

> * Would _makeRecipeBuildRecords() better exist in the factory so other tests could make use of it?

Possibly, if and when those other tests get written. ATM, that method is
only used in the one file and I was thinking it best to avoid cluttering
the factory with a method that may never be used outside the test in
which is was originally written.

> * Would it make sense to fix __eq__ on SourcePackageRecipe, rather than working around it?

Drilling down, the __eq__ fails inside storm when attributes of
SourcePackageReipe are compared as part of the __eq__ operation. So it's
not SPR itself that needs fixing. It's a bit messy. I can't recall the
exact error ATM; it could have been because the storm reference
attributes were detached from the db and there was no connection
available to load the required data when __eq__ was invoked.

> * Use of IStoreSelector, which is deprecated. Could you look at switching to I{Master,}Store?

ok.

> * Please update your XXX per the XXX policy.

ok.

[Robert Collins (lifeless)]

A couple of meta thoughts.

On Wed, Nov 24, 2010 at 2:31 AM, Ian Booth <email address hidden> wrote:
> Thanks Steve for the review.
>
>>
>> * Why are you including imports in the meat of the test, rather than with the rest of them at the top of the file?
>
> "Personal preference". I wanted to try and avoid the top imports list
> getting too large and cluttered with imports for things that were just
> used in one localised place. I'll move them.

While I feel the same pressure the thing is that this quickly becomes
larger-than-one-list - because of redundancy in other tests, and
fixing it becomes a global-scan of the file to find such imports. My
personal conclusion has been that its a pessimisation to do such local
imports with a couple of caveats:
 - in some dynamic-loading scenarios I'll import globally with an
ImportError catch and import in the test to show the actual error if
things don't import, as a test failure
 - circular imports [which should rarely if ever affect tests].

>> * Would _makeRecipeBuildRecords() better exist in the factory so other tests could make use of it?
>
> Possibly, if and when those other tests get written. ATM, that method is
> only used in the one file and I was thinking it best to avoid cluttering
> the factory with a method that may never be used outside the test in
> which is was originally written.

Similar to the import issue, this is a lurking maintenance problem:
how will someone else working on this know to look in your test script
for a test helper. If the helper is truely specfic, then they don't
need to. If you were to write recipe tests elsewhere, would you want
this helper? If so, I think its worth being kind to your future self
and making it discoverable (using whatever idioms we are using at the
time to do that: the current one being 'extend the factory Luke').

-Rob

[Tim Penhey (thumper)]

Additional review comments:

There are a lot of classes and methods that don't have docstrings. Even a
short docstring is often more useful than nothing.

PEP-8 says a blank line between class definition line and methods.

sorted(builds, key=attrgetter('id')) is a way to avoid the zope proxy issue
and sort on something that is unique and has a __lt__ implementation.

expected_text = "No recently completed daily builds found."

is more readable IMO than: (also PEP-8)

expected_text = """
    No recently completed daily builds found.
    """

In the bazaar doctest, instead of using browser.goBack(), just open the page
directly. This way I don't have to know where the browser was before.

You want to avoid string concatenation in the TAL:
  tal:attributes="href
  string:${dailybuild/recipebuild/distribution/fmt:url}/+source/${dailybuild/sourcepackagename/name

This should be change to be a method somewhere either in the view or the items
in the result set. That way it would just be:

  tal:attributes="href xxx/distro_source_package/fmt:url"

review: Needs Fixing

[Ian Booth (wallyworld)]

Thanks for the review.

>
> There are a lot of classes and methods that don't have docstrings. Even a
> short docstring is often more useful than nothing.
>

Done.

> PEP-8 says a blank line between class definition line and methods.
>
> sorted(builds, key=attrgetter('id')) is a way to avoid the zope proxy issue
> and sort on something that is unique and has a __lt__ implementation.
>

RecipeBuildRecord, being a named tuple of entities, doesn't have an id
attribute so this suggestion is not relevant here.

> expected_text = "No recently completed daily builds found."
>
> is more readable IMO than: (also PEP-8)
>
> expected_text = """
> No recently completed daily builds found.
> """

Indeed.

>
> In the bazaar doctest, instead of using browser.goBack(), just open the page
> directly. This way I don't have to know where the browser was before.
>

The original test was written using browser.goBack() so I was extending
what was there already. I've implemented the suggestion and as a drive
by fixed the other places where goBack() was used.

> You want to avoid string concatenation in the TAL:
> tal:attributes="href
> string:${dailybuild/recipebuild/distribution/fmt:url}/+source/${dailybuild/sourcepackagename/name
>
> This should be change to be a method somewhere either in the view or the items
> in the result set. That way it would just be:
>
> tal:attributes="href xxx/distro_source_package/fmt:url"
>
>

My original implementation was lifted exactly from:
 question-portlet-details.pt
I've reworked it as per the suggestion by adding a method to the
RecipeBuildRecord class

[Steve Kowalik (stevenk)]

Hi Ian,

Thanks for all the clean-ups! My only comment is that your multi-line function definitions don't use the preferred form as stated in:

https://dev.launchpad.net/PythonStyleGuide#Multiline%20function%20definitions

I'll leave it up to you if you want to change them, though.

review: Approve (code*)

[Tim Penhey (thumper)]

A few notes...

This is just styalistic, but personally I tend to use cached properties rather
than overriding initialize.

In lib/lp/code/browser/tests/test_recipebuildslisting.py you import BaseLayer
just to use the appserver_root_url. The layers use inheritance and the method
is a class method, so you can call that through the DatabaseFunctionalLayer.
So you can do:
  root_url = self.layer.appserver_root_url(facet='code')
rather than:
  root_url = BaseLayer.appserver_root_url(facet='code')
and avoid an extra import.

Also, can you please change the code.launchpad.dev page to look like:
http://people.canonical.com/~tim/package-listing-link-on-homepage.png

I think it reads much better. The first two parts are optional, the changing of the wording on the main page isn't. Since it is a pure appearance thing, I'll approve now.

review: Approve

=== modified file 'lib/lp/code/browser/configure.zcml'
--- lib/lp/code/browser/configure.zcml	2010-11-18 12:05:34 +0000
+++ lib/lp/code/browser/configure.zcml	2010-12-13 10:14:40 +0000
@@ -138,16 +138,23 @@
         for="canonical.launchpad.interfaces.launchpad.IBazaarApplication"
         layer="lp.code.publisher.CodeLayer"
         name="+index"/>
-    <browser:pages
+    <browser:page
         for="canonical.launchpad.interfaces.launchpad.IBazaarApplication"
         layer="lp.code.publisher.CodeLayer"
         permission="zope.Public"
         class="lp.code.browser.bazaar.BazaarApplicationView"
-        facet="branches">
-        <browser:page
-            name="+index"
-            template="../templates/bazaar-index.pt"/>
-    </browser:pages>
+        facet="branches"
+        name="+index"
+        template="../templates/bazaar-index.pt" />
+    <browser:page
+        for="canonical.launchpad.webapp.interfaces.ILaunchpadRoot"
+        layer="lp.code.publisher.CodeLayer"
+        permission="zope.Public"
+        class="lp.code.browser.recipebuildslisting.CompletedDailyBuildsView"
+        facet="branches"
+        name="+daily-builds"
+        template="../templates/daily-builds-listing.pt" />
+
     <browser:page
         for="lp.code.interfaces.branchmergeproposal.IBranchMergeProposal"
         layer="lp.code.publisher.CodeLayer"

=== added file 'lib/lp/code/browser/recipebuildslisting.py'
--- lib/lp/code/browser/recipebuildslisting.py	1970-01-01 00:00:00 +0000
+++ lib/lp/code/browser/recipebuildslisting.py	2010-12-13 10:14:40 +0000
@@ -0,0 +1,42 @@
+# Copyright 2010 Canonical Ltd.  This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+"""View for daily builds listings."""
+
+__metaclass__ = type
+
+__all__ = [
+    'CompletedDailyBuildsView',
+    ]
+
+from zope.component import getUtility
+
+from canonical.launchpad.webapp import LaunchpadView
+from canonical.launchpad.webapp.batching import BatchNavigator
+from lp.code.interfaces.recipebuild import IRecipeBuildRecordSet
+
+
+class RecipeBuildBatchNavigator(BatchNavigator):
+    """A Batch Navigator turn activate table sorting for single page views."""
+
+    @property
+    def table_class(self):
+        if self.has_multiple_pages:
+            return "listing"
+        else:
+            return "listing sortable"
+
+
+class CompletedDailyBuildsView(LaunchpadView):
+    """The view to show completed builds for source package recipes."""
+
+    @property
+    def page_title(self):
+        return 'Most Recently Completed Daily Recipe Builds'
+
+    def initialize(self):
+        LaunchpadView.initialize(self)
+        recipe_build_set = getUtility(IRecipeBuildRecordSet)
+        self.dailybuilds = recipe_build_set.findCompletedDailyBuilds()
+        self.batchnav = RecipeBuildBatchNavigator(
+            self.dailybuilds, self.request)

=== added file 'lib/lp/code/browser/tests/test_recipebuildslisting.py'
--- lib/lp/code/browser/tests/test_recipebuildslisting.py	1970-01-01 00:00:00 +0000
+++ lib/lp/code/browser/tests/test_recipebuildslisting.py	2010-12-13 10:14:40 +0000
@@ -0,0 +1,89 @@
+# Copyright 2010 Canonical Ltd.  This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+"""Tests for recipe build listings."""
+
+__metaclass__ = type
+
+
+from zope.component import getUtility
+from zope.security.proxy import removeSecurityProxy
+
+from canonical.launchpad.webapp.interfaces import ILaunchpadRoot
+from canonical.testing.layers import DatabaseFunctionalLayer
+from lp.testing import (
+    ANONYMOUS,
+    BrowserTestCase,
+    login,
+    TestCaseWithFactory,
+    )
+from lp.testing.views import create_initialized_view
+
+
+class TestRecipeBuildView(TestCaseWithFactory):
+    """Tests for `CompletedDailyBuildsView`."""
+
+    layer = DatabaseFunctionalLayer
+
+    def test_recipebuildrecords(self):
+        # Check that the view is created from the url and loads the expected
+        # records.
+        records = self.factory.makeRecipeBuildRecords(10, 5)
+        login(ANONYMOUS)
+        root = getUtility(ILaunchpadRoot)
+        view = create_initialized_view(root, "+daily-builds", rootsite='code')
+        # It's easier to do it this way than sorted() since __lt__ doesn't
+        # work properly on zope proxies.
+        self.assertEqual(10, view.dailybuilds.count())
+        self.assertEqual(set(records), set(view.dailybuilds))
+
+
+class TestRecipeBuildListing(BrowserTestCase):
+    """Browser tests for the Recipe Build Listing page."""
+    layer = DatabaseFunctionalLayer
+
+    def _test_recipebuild_listing(self, no_login=False):
+        # Test the content of the listing when there is data.
+        [record] = self.factory.makeRecipeBuildRecords(1, 5)
+        naked_recipebuild = removeSecurityProxy(record.recipebuild)
+        naked_distribution = removeSecurityProxy(
+            naked_recipebuild.distribution)
+        root = getUtility(ILaunchpadRoot)
+        text = self.getMainText(
+            root, '+daily-builds', rootsite='code', no_login=no_login)
+        expected_text = '\n'.join(str(item) for item in (
+                """Most Recently Completed Daily Recipe Builds
+                Source Package
+                Recipe
+                Recipe Owner
+                Archive
+                Most Recent Build Time""",
+                naked_distribution.displayname,
+                record.sourcepackagename.name,
+                naked_recipebuild.recipe.name,
+                record.recipeowner.displayname,
+                record.archive.displayname,
+                record.most_recent_build_time.strftime('%Y-%m-%d %H:%M:%S')))
+        self.assertTextMatchesExpressionIgnoreWhitespace(expected_text, text)
+
+    def test_recipebuild_listing_no_records(self):
+        # Test the expected text when there is no data.
+        root = getUtility(ILaunchpadRoot)
+        text = self.getMainText(root, '+daily-builds', rootsite='code')
+        expected_text = "No recently completed daily builds found."
+        self.assertTextMatchesExpressionIgnoreWhitespace(expected_text, text)
+
+    def test_recipebuild_listing_anonymous(self):
+        # Ensure we can see the listing when we are not logged in.
+        self._test_recipebuild_listing(no_login=True)
+
+    def test_recipebuild_listing_with_user(self):
+        # Ensure we can see the listing when we are logged in.
+        self._test_recipebuild_listing()
+
+    def test_recipebuild_url(self):
+        # Check the browser URL is as expected.
+        root_url = self.layer.appserver_root_url(facet='code')
+        user_browser = self.getUserBrowser("%s/+daily-builds" % root_url)
+        self.assertEqual(
+            user_browser.url, "%s/+daily-builds" % root_url)

=== modified file 'lib/lp/code/configure.zcml'
--- lib/lp/code/configure.zcml	2010-11-18 12:05:34 +0000
+++ lib/lp/code/configure.zcml	2010-12-13 10:14:40 +0000
@@ -1002,4 +1002,23 @@
         factory="lp.code.model.sourcepackagerecipebuild.get_recipe_build_for_build_farm_job"
         name="RECIPEBRANCHBUILD"
         permission="zope.Public"/>
+
+  <!-- RecipeBuildRecordSet and related classes-->
+
+  <securedutility
+      class="lp.code.model.recipebuild.RecipeBuildRecordSet"
+      provides="lp.code.interfaces.recipebuild.IRecipeBuildRecordSet">
+    <allow interface="lp.code.interfaces.recipebuild.IRecipeBuildRecordSet"/>
+  </securedutility>
+
+  <class class="lp.code.model.recipebuild.RecipeBuildRecord">
+    <allow attributes="sourcepackagename recipeowner recipebuild archive
+                        most_recent_build_time distro_source_package"/>
+  </class>
+
+  <class class="lp.code.model.recipebuild.RecipeBuildRecordResultSet">
+    <allow interface="storm.zope.interfaces.IResultSet" />
+    <allow attributes="__getslice__" />
+  </class>
+
 </configure>

=== added file 'lib/lp/code/interfaces/recipebuild.py'
--- lib/lp/code/interfaces/recipebuild.py	1970-01-01 00:00:00 +0000
+++ lib/lp/code/interfaces/recipebuild.py	2010-12-13 10:14:40 +0000
@@ -0,0 +1,27 @@
+# Copyright 2010 Canonical Ltd.  This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+# pylint: disable-msg=E0211,E0213
+
+"""Recipe build interfaces."""
+
+__metaclass__ = type
+
+__all__ = [
+    'IRecipeBuildRecordSet',
+    ]
+
+from zope.interface import (
+    Interface,
+    )
+
+
+class IRecipeBuildRecordSet(Interface):
+    """Interface representing a set of recipe build records."""
+
+    def findCompletedDailyBuilds(epoch_days=30):
+        """Find the recently completed daily builds.
+
+        :param epoch_days: only find builds completed in last epoch_days
+        days. If None, return all completed builds.
+        """

=== added file 'lib/lp/code/model/recipebuild.py'
--- lib/lp/code/model/recipebuild.py	1970-01-01 00:00:00 +0000
+++ lib/lp/code/model/recipebuild.py	2010-12-13 10:14:40 +0000
@@ -0,0 +1,178 @@
+# Copyright 2010 Canonical Ltd.  This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+# pylint: disable-msg=E0611,W0212
+
+__metaclass__ = type
+
+__all__ = [
+    'RecipeBuildRecord',
+    'RecipeBuildRecordSet',
+    ]
+
+from collections import namedtuple
+from datetime import datetime, timedelta
+
+import pytz
+from storm.expr import (
+    compile,
+    EXPR,
+    Expr,
+    Join,
+    Max,
+    Select)
+from storm import Undef
+
+from zope.interface import implements
+
+from canonical.launchpad.components.decoratedresultset import (
+    DecoratedResultSet,
+    )
+from canonical.launchpad.interfaces.lpstorm import ISlaveStore
+
+from lp.buildmaster.enums import BuildStatus
+from lp.buildmaster.model.buildfarmjob import BuildFarmJob
+from lp.buildmaster.model.packagebuild import PackageBuild
+from lp.code.interfaces.recipebuild import IRecipeBuildRecordSet
+from lp.code.model.sourcepackagerecipebuild import SourcePackageRecipeBuild
+from lp.code.model.sourcepackagerecipe import SourcePackageRecipe
+from lp.registry.model.person import Person
+from lp.registry.model.sourcepackagename import SourcePackageName
+from lp.soyuz.model.archive import Archive
+from lp.soyuz.model.binarypackagebuild import BinaryPackageBuild
+from lp.soyuz.model.sourcepackagerelease import SourcePackageRelease
+
+
+class RecipeBuildRecord(namedtuple(
+    'RecipeBuildRecord',
+    """sourcepackagename, recipeowner, archive, recipebuild,
+        most_recent_build_time""")):
+    # We need to implement our own equality check since __eq__ is broken on
+    # SourcePackageRecipe. It's broken there because __eq__ is broken,
+    # or not supported, on storm's ReferenceSet implementation.
+    def __eq__(self, other):
+        return (self.sourcepackagename == other.sourcepackagename
+            and self.recipeowner == other.recipeowner
+            and self.recipebuild.recipe.name == other.recipebuild.recipe.name
+            and self.archive == other.archive
+            and self.most_recent_build_time == other.most_recent_build_time)
+
+    def __hash__(self):
+        return (
+            hash(self.sourcepackagename.name) ^
+            hash(self.recipeowner.name) ^
+            hash(self.recipebuild.recipe.name) ^
+            hash(self.archive.name) ^
+            hash(self.most_recent_build_time))
+
+    @property
+    def distro_source_package(self):
+        return self.recipebuild.distribution.getSourcePackage(
+            self.sourcepackagename)
+
+
+class RecipeBuildRecordSet:
+    """See `IRecipeBuildRecordSet`."""
+
+    implements(IRecipeBuildRecordSet)
+
+    def findCompletedDailyBuilds(self, epoch_days=30):
+        """See `IRecipeBuildRecordSet`."""
+
+        store = ISlaveStore(SourcePackageRecipe)
+        tables = [
+            SourcePackageRecipe,
+            Join(Person,
+                 Person.id == SourcePackageRecipe.owner_id),
+            Join(SourcePackageRecipeBuild,
+                 SourcePackageRecipeBuild.recipe_id ==
+                 SourcePackageRecipe.id),
+            Join(SourcePackageRelease,
+                 SourcePackageRecipeBuild.id ==
+                 SourcePackageRelease.source_package_recipe_build_id),
+            Join(SourcePackageName,
+                 SourcePackageRelease.sourcepackagename ==
+                 SourcePackageName.id),
+            Join(BinaryPackageBuild,
+                 BinaryPackageBuild.source_package_release_id ==
+                    SourcePackageRelease.id),
+            Join(PackageBuild,
+                 PackageBuild.id ==
+                 BinaryPackageBuild.package_build_id),
+            Join(Archive,
+                 Archive.id ==
+                 PackageBuild.archive_id),
+            Join(BuildFarmJob,
+                 BuildFarmJob.id ==
+                 PackageBuild.build_farm_job_id),
+        ]
+
+        where = [BuildFarmJob.status == BuildStatus.FULLYBUILT,
+                    SourcePackageRecipe.build_daily]
+        if epoch_days is not None:
+            epoch = datetime.now(pytz.UTC) - timedelta(days=epoch_days)
+            where.append(BuildFarmJob.date_finished >= epoch)
+
+        result_set = store.using(*tables).find(
+                (SourcePackageName,
+                    Person,
+                    SourcePackageRecipeBuild,
+                    Archive,
+                    Max(BuildFarmJob.date_finished),
+                    ),
+                *where
+            ).group_by(
+                SourcePackageName,
+                Person,
+                SourcePackageRecipeBuild,
+                Archive,
+            ).order_by(
+                SourcePackageName.name,
+                Person.name,
+                Archive.name,
+                )
+
+        def _makeRecipeBuildRecord(values):
+            (sourcepackagename, recipeowner, recipebuild, archive,
+                date_finished) = values
+            return RecipeBuildRecord(
+                sourcepackagename, recipeowner,
+                archive, recipebuild,
+                date_finished)
+
+        return RecipeBuildRecordResultSet(
+            result_set, _makeRecipeBuildRecord)
+
+
+# XXX: wallyworld 2010-11-26 bug=675377: storm's Count() implementation is
+# broken for distinct with > 1 column
+class CountDistinct(Expr):
+
+    __slots__ = ("columns")
+
+    def __init__(self, columns):
+        self.columns = columns
+
+
[e-mail](CountDistinct)
+def compile_countdistinct(compile, countselect, state):
+    state.push("context", EXPR)
+    col = compile(countselect.columns)
+    state.pop()
+    return "count(distinct(%s))" % col
+
+
+class RecipeBuildRecordResultSet(DecoratedResultSet):
+    """A ResultSet which can count() queries with group by."""
+
+    def count(self, expr=Undef, distinct=True):
+        """This count() knows how to handle result sets with group by."""
+
+        # We don't support distinct=False for this result set
+        select = Select(
+            columns=CountDistinct(self.result_set._group_by),
+            tables = self.result_set._tables,
+            where = self.result_set._where,
+            )
+        result = self.result_set._store.execute(select)
+        return result.get_one()[0]

=== modified file 'lib/lp/code/stories/branches/xx-bazaar-home.txt'
--- lib/lp/code/stories/branches/xx-bazaar-home.txt	2010-09-06 21:19:30 +0000
+++ lib/lp/code/stories/branches/xx-bazaar-home.txt	2010-12-13 10:14:40 +0000
@@ -77,14 +77,13 @@
     >>> links.renderContents()
     '...1...&rarr;...6...of...28...results...'
 
-    >>> browser.goBack()
-
 
 == Recently changed ==
 
 Recently changed branches are ordered with the branches with the most
 recent commits first.
 
+    >>> browser.open('http://code.launchpad.dev/')
     >>> changed = find_tag_by_id(browser.contents, 'recently-changed')
     >>> print changed.fetch('a')[-1]['href']
     /+recently-changed-branches
@@ -103,7 +102,6 @@
     Last Modified
     Last Commit
 
-    >>> browser.goBack()
 
 
 == Recently imported ==
@@ -111,6 +109,7 @@
 Recently imported branches are ordered by recent commits only in
 imported branches.
 
+    >>> browser.open('http://code.launchpad.dev/')
     >>> imported = find_tag_by_id(browser.contents, 'recent-imports')
     >>> print imported.fetch('a')[-1]['href']
     /+recently-imported-branches
@@ -132,3 +131,25 @@
     Project
     Last Modified
     Last Commit
+
+
+== Recently built recipes ==
+
+Source package recipes have their own section on the home page.
+
+    >>> browser.open('http://code.launchpad.dev/')
+    >>> changed = find_tag_by_id(browser.contents, 'build-recipes')
+    >>> print changed.fetch('a')[0]['href']
+    /+daily-builds
+    >>> print changed.fetch('a')[-1]['href']
+    https://help.launchpad.net/Packaging/SourceBuilds
+
+    >>> browser.getLink(url='daily-builds').click()
+    >>> print browser.title
+    Most Recently Completed Daily Recipe Builds
+
+    >>> text = extract_text(browser.contents)
+    >>> print text
+    Most Recently Completed Daily Recipe Builds ...
+    No recently completed daily builds found. ...
+

=== modified file 'lib/lp/code/templates/bazaar-index.pt'
--- lib/lp/code/templates/bazaar-index.pt	2010-09-06 21:19:30 +0000
+++ lib/lp/code/templates/bazaar-index.pt	2010-12-13 10:14:40 +0000
@@ -52,6 +52,12 @@
         so you can use Bazaar with those too.
         (<a href="https://help.launchpad.net/Code">Read our guide</a>)
       </p>
+      <p id="build-recipes" class="application-summary" style="padding-top: 0.25em;">
+        Launchpad can build Ubuntu packages directly from branches using recipes.
+        We currently build over <a href="/+daily-builds">100 different packages</a>
+        into PPAs automatically using this method.
+        (<a href="https://help.launchpad.net/Packaging/SourceBuilds">Learn more about recipes</a>)
+      </p>
 
       <div id="project-cloud-preview" tal:content="cache:public,6 hours">
         <h2>Most active projects in the last month</h2>

=== added file 'lib/lp/code/templates/daily-builds-listing.pt'
--- lib/lp/code/templates/daily-builds-listing.pt	1970-01-01 00:00:00 +0000
+++ lib/lp/code/templates/daily-builds-listing.pt	2010-12-13 10:14:40 +0000
@@ -0,0 +1,93 @@
+<html
+    xmlns="http://www.w3.org/1999/xhtml"
+    xmlns:tal="http://xml.zope.org/namespaces/tal"
+    xmlns:metal="http://xml.zope.org/namespaces/metal"
+    xmlns:i18n="http://xml.zope.org/namespaces/i18n"
+    metal:use-macro="view/macro:page/main_only"
+    i18n:domain="launchpad">
+
+  <body>
+
+    <metal:heading fill-slot="heading">
+      <h1 tal:content="view/page_title" />
+    </metal:heading>
+
+    <div metal:fill-slot="main"
+         tal:define="dailybuilds view/batchnav/currentBatch">
+
+      <tal:block tal:condition="not:dailybuilds">
+        <p id="no-builds">No recently completed daily builds found.</p>
+      </tal:block>
+
+      <tal:block tal:condition="dailybuilds">
+
+        <tal:needs-batch condition="view/batchnav/has_multiple_pages">
+          <div class="lesser" tal:content="structure view/batchnav/@@+navigation-links-upper"/>
+        </tal:needs-batch>
+
+        <table tal:attributes="class view/batchnav/table_class" id="daily-build-listing">
+          <thead>
+            <tr>
+              <th>
+                Source Package
+              </th>
+              <th>
+                Recipe
+              </th>
+              <th>
+                Recipe Owner
+              </th>
+              <th>
+                Archive
+              </th>
+              <th>
+                Most Recent Build Time
+              </th>
+            </tr>
+          </thead>
+          <tbody>
+            <tr tal:repeat="dailybuild dailybuilds">
+              <td>
+                <a tal:replace="structure dailybuild/recipebuild/distribution/fmt:link">distribution</a>
+                <a tal:attributes="href dailybuild/distro_source_package/fmt:url"
+                  tal:content="dailybuild/sourcepackagename/name">source package name</a>
+              </td>
+
+              <td>
+                <a href="recipe" tal:attributes="href dailybuild/recipebuild/recipe/fmt:url">
+                <span tal:replace="dailybuild/recipebuild/recipe/name">
+                  recipe
+                </span>
+                </a>
+              </td>
+
+              <td>
+                <tal:recipeowner replace="structure dailybuild/recipeowner/fmt:link">
+                  recipe owner
+                </tal:recipeowner>
+              </td>
+
+              <td>
+                <tal:archivelink tal:condition="dailybuild/archive/is_ppa"
+                              replace="structure dailybuild/archive/fmt:link">
+                  archive link
+                </tal:archivelink>
+                <tal:archive tal:condition="not:dailybuild/archive/is_ppa"
+                              replace="structure dailybuild/archive/displayname">
+                  archive name
+                </tal:archive>
+              </td>
+
+              <td tal:content="dailybuild/most_recent_build_time/fmt:datetime">
+                a date
+              </td>
+
+            </tr>
+          </tbody>
+        </table>
+        <div class="lesser" tal:content="structure view/batchnav/@@+navigation-links-lower" />
+      </tal:block>
+    </div>
+
+  </body>
+</html>

=== modified file 'lib/lp/testing/__init__.py'
--- lib/lp/testing/__init__.py	2010-12-03 16:33:03 +0000
+++ lib/lp/testing/__init__.py	2010-12-13 10:14:40 +0000
@@ -702,9 +702,10 @@
         super(BrowserTestCase, self).setUp()
         self.user = self.factory.makePerson(password='test')
 
-    def getViewBrowser(self, context, view_name=None, no_login=False):
+    def getViewBrowser(self, context, view_name=None, no_login=False,
+                       rootsite=None):
         login(ANONYMOUS)
-        url = canonical_url(context, view_name=view_name)
+        url = canonical_url(context, view_name=view_name ,rootsite=rootsite)
         logout()
         if no_login:
             from canonical.launchpad.testing.pages import setupBrowser
@@ -714,11 +715,13 @@
         else:
             return self.getUserBrowser(url, self.user)
 
-    def getMainText(self, context, view_name=None):
+    def getMainText(
+            self, context, view_name=None, rootsite=None, no_login=False):
         """Return the main text of a context's page."""
         from canonical.launchpad.testing.pages import (
             extract_text, find_main_content)
-        browser = self.getViewBrowser(context, view_name)
+        browser = self.getViewBrowser(
+            context, view_name, rootsite=rootsite, no_login=no_login)
         return extract_text(find_main_content(browser.contents))
 
 

=== modified file 'lib/lp/testing/factory.py'
--- lib/lp/testing/factory.py	2010-12-09 11:03:53 +0000
+++ lib/lp/testing/factory.py	2010-12-13 10:14:40 +0000
@@ -7,6 +7,7 @@
 
 This module should not contain tests (but it should be tested).
 """
+from lp.code.model.recipebuild import RecipeBuildRecord
 
 __metaclass__ = type
 __all__ = [
@@ -36,10 +37,12 @@
     isSequenceType,
     )
 import os
+from pytz import UTC
 from random import randint
 from StringIO import StringIO
 from textwrap import dedent
 from threading import local
+import transaction
 from types import InstanceType
 import warnings
 
@@ -1283,6 +1286,7 @@
             source_branch = self.makeBranch()
         else:
             source_branch = merge_proposal.source_branch
+
         def make_revision(parent=None):
             sequence = source_branch.revision_history.count() + 1
             if parent is None:
@@ -2284,6 +2288,80 @@
         store.add(bq)
         return bq
 
+    def makeRecipeBuildRecords(self, num_records=1,
+                               num_records_outside_epoch=0, epoch_days=30):
+        """Create some recipe build records.
+
+        A RecipeBuildRecord is a named tuple. Some records will be created
+        with archive of type ArchivePurpose.PRIMARY, others with type
+        ArchivePurpose.PPA.
+        :param num_records: the number of records within the specified time
+         window.
+        :param num_records_outside_epoch: the number of records outside the
+         specified time window.
+        :param epoch_days: the time window to use when creating records.
+        """
+
+        distroseries = self.makeDistroRelease()
+        sourcepackagename = self.makeSourcePackageName()
+        sourcepackage = self.makeSourcePackage(
+            sourcepackagename=sourcepackagename,
+            distroseries=distroseries)
+        recipeowner = self.makePerson()
+        recipe = self.makeSourcePackageRecipe(
+            build_daily=True,
+            owner=recipeowner,
+            name="Recipe_"+sourcepackagename.name,
+            distroseries=distroseries)
+
+        result = []
+        for x in range(num_records+num_records_outside_epoch):
+            # Ensure we have both ppa and primary archives
+            if x%2 == 0:
+                purpose = ArchivePurpose.PPA
+            else:
+                purpose = ArchivePurpose.PRIMARY
+            archive = self.makeArchive(purpose=purpose)
+            sprb = self.makeSourcePackageRecipeBuild(
+                requester=recipeowner,
+                recipe=recipe,
+                archive=archive,
+                sourcepackage=sourcepackage,
+                distroseries=distroseries)
+            spr = self.makeSourcePackageRelease(
+                source_package_recipe_build=sprb,
+                archive=archive,
+                sourcepackagename=sourcepackagename,
+                distroseries=distroseries)
+            binary_build = self.makeBinaryPackageBuild(
+                    source_package_release=spr)
+            naked_build = removeSecurityProxy(binary_build)
+            naked_build.queueBuild()
+            naked_build.status = BuildStatus.FULLYBUILT
+
+            from random import randrange
+            offset = randrange(0, epoch_days)
+            now = datetime.now(UTC)
+            if x >= num_records:
+                offset = epoch_days + 1 + offset
+            naked_build.date_finished = (
+                now - timedelta(days=offset))
+            naked_build.date_started = (
+                naked_build.date_finished - timedelta(minutes=5))
+            rbr = RecipeBuildRecord(
+                removeSecurityProxy(sourcepackagename),
+                removeSecurityProxy(recipeowner),
+                removeSecurityProxy(archive),
+                removeSecurityProxy(sprb),
+                naked_build.date_finished.replace(tzinfo=None))
+
+            if x < num_records:
+                result.append(rbr)
+        # We need to explicitly commit because if don't, the records don't
+        # appear in the slave datastore.
+        transaction.commit()
+        return result
+
     def makeDscFile(self, tempdir_path=None):
         """Make a DscFile.