Merge lp:~edwin-grubbs/launchpad/bug-535430-needspackaging-timeout-part2 into lp:launchpad/db-devel

[Edwin Grubbs]

Summary
-------

This branch is a follow-up to
    lp:~edwin-grubbs/launchpad/bug-535430-needspackaging-timeout-part2

It adds the attributes to the DistributionSourcePackageInDatabase and
DistributionSourcePackage classes to access the columns added to the
DistributionSourcePackage table in the last branch. Merging the
DistributionSourcePackageInDatabase and DistributionSourcePackage
classes is deferred, since it is not certain yet whether errors would
occur if a DSP object required a SPPH row to be created first. I added
code to report a silent oops if there is an unexpected creation of a
DSP.

This branch also popuplates total_bug_heat and bug_count columns in the
DistributionSourcePackage table when a bugtask is added. The
po_message_count column will be populated in a later branch.

Implementation details
----------------------

Allow db users 'lucille', 'uploader', and 'queued' to create a DSP entry
when a SPPH record is created. Allow db user 'processmail' to read the
section table, since that is necessary to check the foreign key on the
DSP table.

    database/schema/security.cfg

Added tests.
    lib/lp/bugs/doc/bug-heat.txt
    lib/lp/bugs/tests/test_bugheat.py

Automatically insert into DistributionSourcePackage table when a
SourcePackagePublishingHistory record is added. (See ensure() method.)
Renamed recalculateMaxBugHeat() to recalculateBugHeatCache() and
overrode it in the model's DistributionSourcePackage class to set
total_bug_heat and bug_count in addition to max_bug_heat, which it was
already setting.
    lib/lp/bugs/interfaces/bugtarget.py
    lib/lp/registry/interfaces/distributionsourcepackage.py
    lib/lp/bugs/model/bug.py
    lib/lp/bugs/model/bugtarget.py
    lib/lp/bugs/model/bugtask.py
    lib/lp/registry/model/distributionsourcepackage.py
    lib/lp/soyuz/model/publishing.py

Tests
-----

./bin/test -vv -t 'bug-heat.txt|test_bugheat'

[Brad Crittenden (bac)]

Hi Edwin,

This change looks great. I marked a couple of niggly bits but that's
all.

--bac

> === added file 'database/schema/patch-2207-56-0.sql'
> --- database/schema/patch-2207-56-0.sql 1970-01-01 00:00:00 +0000
> +++ database/schema/patch-2207-56-0.sql 2010-06-04 18:48:29 +0000
> @@ -0,0 +1,24 @@
> +-- Copyright 2009 Canonical Ltd. This software is licensed under the

2010

(The contents of the patch are left for a db review.)

> === modified file 'lib/lp/bugs/tests/test_bugheat.py'
> --- lib/lp/bugs/tests/test_bugheat.py 2010-06-01 09:21:32 +0000
> +++ lib/lp/bugs/tests/test_bugheat.py 2010-06-04 18:48:29 +0000
> @@ -7,8 +7,11 @@

Edwin this test is really readable and easy to follow. Good job.

> import unittest
>
> +from storm.store import Store
> +
> from canonical.testing import LaunchpadZopelessLayer
>
> +from lp.testing import TestCaseWithFactory
> from lp.testing.factory import LaunchpadObjectFactory
>
>
> @@ -58,6 +61,85 @@
> def setUp(self):
> self.target = self.factory.makeDistributionSourcePackage()
>
> +class DistributionSourcePackageNullBugHeatCacheTest(
> + TestCaseWithFactory):
> + """Ensure distro source package cache values start at None."""
> +
> + layer = LaunchpadZopelessLayer
> +
> + def setUp(self):
> + TestCaseWithFactory.setUp(self)
> + self.target = self.factory.makeDistributionSourcePackage()
> +
> + def test_null_max_bug_heat(self):
> + self.assertEqual(None, self.target.max_bug_heat)
> +
> + def test_null_total_bug_heat(self):
> + self.assertEqual(None, self.target.total_bug_heat)
> +
> + def test_null_bug_count(self):
> + self.assertEqual(None, self.target.bug_count)
> +
> +
> +class DistributionSourcePackageZeroRecalculateBugHeatCacheTest(
> + TestCaseWithFactory):
> + """Ensure distro source package cache values become zero properly."""
> +
> + layer = LaunchpadZopelessLayer
> +
> + def setUp(self):
> + TestCaseWithFactory.setUp(self)
> + self.target = self.factory.makeDistributionSourcePackage()
> + self.target.recalculateBugHeatCache()
> +
> + def test_zero_max_bug_heat(self):
> + self.assertEqual(0, self.target.max_bug_heat)
> +
> + def test_zero_total_bug_heat(self):
> + self.assertEqual(0, self.target.total_bug_heat)
> +
> + def test_zero_bug_count(self):
> + self.assertEqual(0, self.target.bug_count)
> +
> +
> +class DistributionSourcePackageMultipleBugsRecalculateBugHeatCacheTest(
> + TestCaseWithFactory):
> + """Ensure distro source package cache values are set properly."""
> +
> + layer = LaunchpadZopelessLayer
> +
> + def setUp(self):
> + TestCaseWithFactory.setUp(self)
> + self.target = self.factory.makeDistributionSourcePackage()
> + self.bugtask1 = self.factory.makeBugTask(target=self.target)
> + self.bugtask2 = self.factory.makeBugTask(target=self.target)
> + # Bug heat gets calculated by complicated rules in a db
> + # stored procedure. We will override them here to avoid
> + # testing inconsitencies if those values are calculated
> + # differently in the future.
> + #...

review: Approve (code)

[Edwin Grubbs (edwin-grubbs)]

I assume the changes to database/schema/security.cfg the needs to be reviewed by either Bjorn or Stub. I had to add permissions for db users that had been inserting into the SourcePackagePublishingHistory, since that will cause the DistributionSourcePackage table to be inserted into or updated. The permissions have been changed for the db users "lucille", "uploader", "queued", and "processmail".

I forgot to create this merge proposal with the previous branch as the dependency, so you will see the same schema changes that have already been reviewed.

[Björn Tillenius (bjornt)]

Changes to security.cfg don't need a DB review, just make sure they are tested.

review: Abstain (db)

=== modified file 'database/schema/comments.sql'
--- database/schema/comments.sql	2010-06-03 22:21:24 +0000
+++ database/schema/comments.sql	2010-06-10 01:43:25 +0000
@@ -472,7 +472,11 @@
 
 COMMENT ON TABLE DistributionSourcePackage IS 'Representing a sourcepackage in a distribution across all distribution series.';
 COMMENT ON COLUMN DistributionSourcePackage.bug_reporting_guidelines IS 'Guidelines to the end user for reporting bugs on a particular a source package in a distribution.';
-COMMENT ON COLUMN DistributionSourcePackage.max_bug_heat IS 'The highest heat value across bugs for this source package.';
+COMMENT ON COLUMN DistributionSourcePackage.max_bug_heat IS 'The highest heat value across bugs for this source package. NULL means it has not yet been calculated.';
+COMMENT ON COLUMN DistributionSourcePackage.total_bug_heat IS 'Sum of bug heat matching the package distribution and sourcepackagename. NULL means it has not yet been calculated.';
+COMMENT ON COLUMN DistributionSourcePackage.bug_count IS 'Number of bugs matching the package distribution and sourcepackagename. NULL means it has not yet been calculated.';
+COMMENT ON COLUMN DistributionSourcePackage.po_message_count IS 'Number of translations matching the package distribution and sourcepackagename. NULL means it has not yet been calculated.';
+COMMENT ON COLUMN DistributionSourcePackage.section IS 'Cached section matching the latest SourcePackagePublishingHistory record by distribution and sourcepackagename whose archive purpose is PRIMARY and whose distroseries releasestatus is CURRENT.';
 COMMENT ON COLUMN DistributionSourcePackage.bug_reported_acknowledgement IS 'A message of acknowledgement to display to a bug reporter after they\'ve reported a new bug.';
 
 -- DistributionSourcePackageCache

=== added file 'database/schema/patch-2207-56-0.sql'
--- database/schema/patch-2207-56-0.sql	1970-01-01 00:00:00 +0000
+++ database/schema/patch-2207-56-0.sql	2010-06-10 01:43:25 +0000
@@ -0,0 +1,24 @@
+-- Copyright 2010 Canonical Ltd.  This software is licensed under the
+-- GNU Affero General Public License version 3 (see the file LICENSE).
+
+SET client_min_messages=ERROR;
+
+INSERT INTO LaunchpadDatabaseRevision VALUES (2207, 56, 0);
+
+/*
+Existing Schema:
+
+CREATE TABLE distributionsourcepackage (
+    id integer NOT NULL,
+    distribution integer NOT NULL,
+    sourcepackagename integer NOT NULL,
+    bug_reporting_guidelines text,
+    max_bug_heat integer
+);
+*/
+
+ALTER TABLE DistributionSourcePackage ADD COLUMN total_bug_heat INTEGER;
+ALTER TABLE DistributionSourcePackage ADD COLUMN bug_count INTEGER;
+ALTER TABLE DistributionSourcePackage ADD COLUMN po_message_count INTEGER;
+ALTER TABLE DistributionSourcePackage
+    ADD COLUMN section INTEGER NOT NULL REFERENCES section(id);

=== modified file 'database/schema/security.cfg'
--- database/schema/security.cfg	2010-06-07 10:24:03 +0000
+++ database/schema/security.cfg	2010-06-10 01:43:25 +0000
@@ -746,6 +746,7 @@
 public.binarypackagepublishinghistory           = SELECT, INSERT, UPDATE, DELETE
 public.sourcepackagepublishinghistory           = SELECT, INSERT, UPDATE, DELETE
 public.sourcepackagepublishinghistory           = SELECT
+public.distributionsourcepackage                = SELECT, INSERT, UPDATE
 
 # Closing bugs for publication copies.
 public.bug                              = SELECT, UPDATE
@@ -1051,6 +1052,7 @@
 public.distroseries                     = SELECT, UPDATE
 public.distroarchseries                 = SELECT
 public.sourcepackagepublishinghistory   = SELECT
+public.distributionsourcepackage        = SELECT, INSERT, UPDATE
 public.sourcepackagefilepublishing      = SELECT
 public.binarypackagefilepublishing      = SELECT
 public.binarypackagepublishinghistory   = SELECT
@@ -1175,8 +1177,9 @@
 public.sourcepackagefilepublishing      = SELECT
 public.binarypackagefilepublishing      = SELECT
 public.sourcepackagepublishinghistory   = SELECT, INSERT, UPDATE
+public.distributionsourcepackage        = SELECT, INSERT, UPDATE
 public.binarypackagepublishinghistory   = SELECT, INSERT, UPDATE
-public.sourcepackagerecipebuildjob		= SELECT, INSERT, UPDATE
+public.sourcepackagerecipebuildjob      = SELECT, INSERT, UPDATE
 public.component                        = SELECT
 public.section                          = SELECT
 public.componentselection               = SELECT
@@ -1464,6 +1467,7 @@
 public.sourcepackagerelease             = SELECT
 public.sourcepackagepublishinghistory   = SELECT
 public.structuralsubscription           = SELECT
+public.section                          = SELECT
 
 # Karma
 public.karma                            = SELECT, INSERT

=== modified file 'lib/lp/bugs/doc/bug-heat.txt'
--- lib/lp/bugs/doc/bug-heat.txt	2010-05-27 16:40:54 +0000
+++ lib/lp/bugs/doc/bug-heat.txt	2010-06-10 01:43:25 +0000
@@ -308,6 +308,20 @@
     >>> print dsp.max_bug_heat
     123
 
+A DistributionSourcePackage also has a cached value for bug_count and
+total_bug_heat.
+
+    >>> print dsp.bug_count
+    1
+    >>> print dsp.total_bug_heat
+    123
+    >>> dsp_task2 = factory.makeBugTask(target=dsp)
+    >>> dsp_task2.bug.setHeat(7)
+    >>> print dsp.bug_count
+    2
+    >>> print dsp.total_bug_heat
+    130
+
 Transitioning from one target to another, calculates the value for the new
 target.
 

=== modified file 'lib/lp/bugs/interfaces/bugtarget.py'
--- lib/lp/bugs/interfaces/bugtarget.py	2010-06-09 08:26:26 +0000
+++ lib/lp/bugs/interfaces/bugtarget.py	2010-06-10 01:43:25 +0000
@@ -272,8 +272,13 @@
     def setMaxBugHeat(heat):
         """Set the max_bug_heat for this context."""
 
-    def recalculateMaxBugHeat():
-        """Recalculate and set the max_bug_heat for this context."""
+    def recalculateBugHeatCache():
+        """Recalculate and set the various bug heat values for this context.
+
+        Several different objects cache max_bug_heat.
+        When DistributionSourcePackage is the target, the total_bug_heat
+        and bug_count are also cached.
+        """
 
 
 class BugDistroSeriesTargetDetails:

=== modified file 'lib/lp/bugs/model/bug.py'
--- lib/lp/bugs/model/bug.py	2010-06-04 09:12:26 +0000
+++ lib/lp/bugs/model/bug.py	2010-06-10 01:43:25 +0000
@@ -887,7 +887,7 @@
 
         # When a new task is added the bug's heat becomes relevant to the
         # target's max_bug_heat.
-        target.recalculateMaxBugHeat()
+        target.recalculateBugHeatCache()
 
         return new_task
 
@@ -1569,7 +1569,7 @@
         self.heat = heat
         self.heat_last_updated = timestamp
         for task in self.bugtasks:
-            task.target.recalculateMaxBugHeat()
+            task.target.recalculateBugHeatCache()
 
     def updateHeat(self):
         """See `IBug`."""

=== modified file 'lib/lp/bugs/model/bugtarget.py'
--- lib/lp/bugs/model/bugtarget.py	2010-05-21 04:38:42 +0000
+++ lib/lp/bugs/model/bugtarget.py	2010-06-10 01:43:25 +0000
@@ -235,12 +235,15 @@
         else:
             raise NotImplementedError
 
-    def recalculateMaxBugHeat(self):
-        """See `IHasBugHeat`."""
+    def recalculateBugHeatCache(self):
+        """See `IHasBugHeat`.
+
+        DistributionSourcePackage overrides this method.
+        """
         if IProductSeries.providedBy(self):
-            return self.product.recalculateMaxBugHeat()
+            return self.product.recalculateBugHeatCache()
         if IDistroSeries.providedBy(self):
-            return self.distribution.recalculateMaxBugHeat()
+            return self.distribution.recalculateBugHeatCache()
         if ISourcePackage.providedBy(self):
             # Should only happen for nominations, so we can safely skip
             # recalculating max_heat.
@@ -276,13 +279,6 @@
                       WHERE Bugtask.bug = Bug.id AND
                       Bugtask.product = Product.id AND
                       Product.project =  %s""" % sqlvalues(self)]
-        elif IDistributionSourcePackage.providedBy(self):
-            sql = ["""SELECT MAX(heat)
-                      FROM Bug, Bugtask
-                      WHERE Bugtask.bug = Bug.id AND
-                      Bugtask.distribution = %s AND
-                      Bugtask.sourcepackagename = %s""" % sqlvalues(
-                 self.distribution, self.sourcepackagename)]
         else:
             raise NotImplementedError
 
@@ -299,7 +295,7 @@
         # If the product is part of a project group we calculate the maximum
         # heat for the project group too.
         if IProduct.providedBy(self) and self.project is not None:
-            self.project.recalculateMaxBugHeat()
+            self.project.recalculateBugHeatCache()
 
 
 

=== modified file 'lib/lp/bugs/model/bugtask.py'
--- lib/lp/bugs/model/bugtask.py	2010-06-07 18:11:08 +0000
+++ lib/lp/bugs/model/bugtask.py	2010-06-10 01:43:25 +0000
@@ -1045,8 +1045,8 @@
         # After the target has changed, we need to recalculate the maximum bug
         # heat for the new and old targets.
         if self.target != target_before_change:
-            target_before_change.recalculateMaxBugHeat()
-            self.target.recalculateMaxBugHeat()
+            target_before_change.recalculateBugHeatCache()
+            self.target.recalculateBugHeatCache()
 
     def updateTargetNameCache(self, newtarget=None):
         """See `IBugTask`."""

=== modified file 'lib/lp/bugs/tests/test_bugheat.py'
--- lib/lp/bugs/tests/test_bugheat.py	2010-06-01 09:21:32 +0000
+++ lib/lp/bugs/tests/test_bugheat.py	2010-06-10 01:43:25 +0000
@@ -7,8 +7,11 @@
 
 import unittest
 
+from storm.store import Store
+
 from canonical.testing import LaunchpadZopelessLayer
 
+from lp.testing import TestCaseWithFactory
 from lp.testing.factory import LaunchpadObjectFactory
 
 
@@ -58,6 +61,85 @@
     def setUp(self):
         self.target = self.factory.makeDistributionSourcePackage()
 
+class DistributionSourcePackageNullBugHeatCacheTest(
+    TestCaseWithFactory):
+    """Ensure distro source package cache values start at None."""
+
+    layer = LaunchpadZopelessLayer
+
+    def setUp(self):
+        TestCaseWithFactory.setUp(self)
+        self.target = self.factory.makeDistributionSourcePackage()
+
+    def test_null_max_bug_heat(self):
+        self.assertEqual(None, self.target.max_bug_heat)
+
+    def test_null_total_bug_heat(self):
+        self.assertEqual(None, self.target.total_bug_heat)
+
+    def test_null_bug_count(self):
+        self.assertEqual(None, self.target.bug_count)
+
+
+class DistributionSourcePackageZeroRecalculateBugHeatCacheTest(
+    TestCaseWithFactory):
+    """Ensure distro source package cache values become zero properly."""
+
+    layer = LaunchpadZopelessLayer
+
+    def setUp(self):
+        TestCaseWithFactory.setUp(self)
+        self.target = self.factory.makeDistributionSourcePackage()
+        self.target.recalculateBugHeatCache()
+
+    def test_zero_max_bug_heat(self):
+        self.assertEqual(0, self.target.max_bug_heat)
+
+    def test_zero_total_bug_heat(self):
+        self.assertEqual(0, self.target.total_bug_heat)
+
+    def test_zero_bug_count(self):
+        self.assertEqual(0, self.target.bug_count)
+
+
+class DistributionSourcePackageMultipleBugsRecalculateBugHeatCacheTest(
+    TestCaseWithFactory):
+    """Ensure distro source package cache values are set properly."""
+
+    layer = LaunchpadZopelessLayer
+
+    def setUp(self):
+        TestCaseWithFactory.setUp(self)
+        self.target = self.factory.makeDistributionSourcePackage()
+        self.bugtask1 = self.factory.makeBugTask(target=self.target)
+        self.bugtask2 = self.factory.makeBugTask(target=self.target)
+        # Bug heat gets calculated by complicated rules in a db
+        # stored procedure. We will override them here to avoid
+        # testing inconsitencies if those values are calculated
+        # differently in the future.
+        # target.recalculateBugHeatCache() should be called
+        # automatically by bug.setHeat().
+        bug1 = self.bugtask1.bug
+        bug2 = self.bugtask2.bug
+        bug1.setHeat(7)
+        bug2.setHeat(19)
+        Store.of(bug1).flush()
+        self.max_heat = max(bug1.heat, bug2.heat)
+        self.total_heat = sum([bug1.heat, bug2.heat])
+
+    def test_max_bug_heat(self):
+        self.assertEqual(self.max_heat, self.target.max_bug_heat)
+
+    def test_total_bug_heat(self):
+        self.assertEqual(self.total_heat, self.target.total_bug_heat)
+        self.failUnless(
+            self.target.total_bug_heat > self.target.max_bug_heat,
+            "Total bug heat should be more than the max bug heat, "
+            "since we know that multiple bugs have nonzero heat.")
+
+    def test_bug_count(self):
+        self.assertEqual(2, self.target.bug_count)
+
 
 class SourcePackageMaxHeatByTargetTest(
     MaxHeatByTargetBase, unittest.TestCase):

=== modified file 'lib/lp/registry/interfaces/distributionsourcepackage.py'
--- lib/lp/registry/interfaces/distributionsourcepackage.py	2010-02-12 13:39:56 +0000
+++ lib/lp/registry/interfaces/distributionsourcepackage.py	2010-06-10 01:43:25 +0000
@@ -78,9 +78,25 @@
         "no such package -- this occurs when there is no current series for "
         "the distribution.")
 
+    total_bug_heat = Attribute(
+        "Sum of the bug heat for all the bugs matching the distribution "
+        "and sourcepackagename of the IDistributionSourcePackage.")
+
+    max_bug_heat = Attribute(
+        "Maximum bug heat for a single bug matching the distribution "
+        "and sourcepackagename of the IDistributionSourcePackage.")
+
+    bug_count = Attribute(
+        "Number of bugs matching the distribution and sourcepackagename "
+        "of the IDistributionSourcePackage.")
+
+    po_message_count = Attribute(
+        "Number of translations matching the distribution and "
+        "sourcepackagename of the IDistributionSourcePackage.")
+
     def getReleasesAndPublishingHistory():
         """Return a list of all releases of this source package in this
-        distribution and their correspodning publishing history.
+        distribution and their corresponding publishing history.
 
         Items in the list are tuples comprised of a
         DistributionSourcePackage and a list of

=== modified file 'lib/lp/registry/model/distributionsourcepackage.py'
--- lib/lp/registry/model/distributionsourcepackage.py	2010-06-08 13:07:44 +0000
+++ lib/lp/registry/model/distributionsourcepackage.py	2010-06-10 01:43:25 +0000
@@ -15,13 +15,17 @@
 import operator
 
 from sqlobject.sqlbuilder import SQLConstant
-from storm.expr import And, Desc, In, Join, Lower
+from storm.expr import And, Count, Desc, In, Join, Lower, Max, Sum
 from storm.store import EmptyResultSet
 from storm.locals import Int, Reference, Store, Storm, Unicode
+from zope.component import getUtility
+from zope.error.interfaces import IErrorReportingUtility
 from zope.interface import implements
 
+
 from canonical.database.sqlbase import sqlvalues
 from canonical.launchpad.database.emailaddress import EmailAddress
+from lp.registry.interfaces.series import SeriesStatus
 from lp.registry.model.distroseries import DistroSeries
 from lp.registry.model.packaging import Packaging
 from lp.registry.model.structuralsubscription import (
@@ -30,7 +34,7 @@
 from canonical.lazr.utils import smartquote
 from lp.answers.interfaces.questiontarget import IQuestionTarget
 from lp.bugs.interfaces.bugtarget import IHasBugHeat
-from lp.bugs.model.bug import BugSet, get_bug_tags_open_count
+from lp.bugs.model.bug import Bug, BugSet, get_bug_tags_open_count
 from lp.bugs.model.bugtarget import BugTargetBase, HasBugHeatMixin
 from lp.bugs.model.bugtask import BugTask
 from lp.code.model.hasbranches import HasBranchesMixin, HasMergeProposalsMixin
@@ -43,6 +47,7 @@
     SourcePackage, SourcePackageQuestionTargetMixin)
 from lp.soyuz.interfaces.archive import ArchivePurpose
 from lp.soyuz.interfaces.publishing import PackagePublishingStatus
+from lp.soyuz.interfaces.section import ISectionSet
 from lp.soyuz.model.archive import Archive
 from lp.soyuz.model.distributionsourcepackagerelease import (
     DistributionSourcePackageRelease)
@@ -54,6 +59,40 @@
     CustomLanguageCode, HasCustomLanguageCodesMixin)
 
 
+class DistributionSourcePackageProperty:
+    def __init__(self, attrname):
+        self.attrname = attrname
+
+    def __get__(self, obj, class_):
+        return getattr(obj._self_in_database, self.attrname, None)
+
+    def __set__(self, obj, value):
+        if obj._self_in_database is None:
+            # Log an oops without raising an error.
+            exception = AssertionError(
+                "DistributionSourcePackage record should have been created "
+                "earlier in the database for distro=%s, sourcepackagename=%s"
+                % (obj.distribution.name, obj.sourcepackagename.name))
+            getUtility(IErrorReportingUtility).raising(
+                (exception.__class__, exception, None))
+            spph = Store.of(obj.distribution).find(
+                SourcePackagePublishingHistory,
+                SourcePackagePublishingHistory.distroseriesID ==
+                    DistroSeries.id,
+                DistroSeries.distributionID == obj.distribution.id,
+                SourcePackagePublishingHistory.sourcepackagereleaseID ==
+                    SourcePackageRelease.id,
+                SourcePackageRelease.sourcepackagenameID ==
+                    obj.sourcepackagename.id
+                ).order_by(Desc(SourcePackagePublishingHistory.id)).first()
+            if spph is None:
+                section = getUtility(ISectionSet)['misc']
+            else:
+                section = spph.section
+            obj._new(obj.distribution, obj.sourcepackagename, section)
+        setattr(obj._self_in_database, self.attrname, value)
+
+
 class DistributionSourcePackage(BugTargetBase,
                                 SourcePackageQuestionTargetMixin,
                                 StructuralSubscriptionTargetMixin,
@@ -72,6 +111,16 @@
         IDistributionSourcePackage, IHasBugHeat, IHasCustomLanguageCodes,
         IQuestionTarget)
 
+    bug_reporting_guidelines = DistributionSourcePackageProperty(
+        'bug_reporting_guidelines')
+    bug_reported_acknowledgement = DistributionSourcePackageProperty(
+        'bug_reported_acknowledgement')
+    max_bug_heat = DistributionSourcePackageProperty('max_bug_heat')
+    total_bug_heat = DistributionSourcePackageProperty('total_bug_heat')
+    bug_count = DistributionSourcePackageProperty('bug_count')
+    po_message_count = DistributionSourcePackageProperty('po_message_count')
+    section = DistributionSourcePackageProperty('section')
+
     def __init__(self, distribution, sourcepackagename):
         self.distribution = distribution
         self.sourcepackagename = sourcepackagename
@@ -118,75 +167,34 @@
         # measure while DistributionSourcePackage is not yet hooked
         # into the database but we need access to some of the fields
         # in the database.
-        return Store.of(self.distribution).find(
-            DistributionSourcePackageInDatabase,
-            DistributionSourcePackageInDatabase.sourcepackagename == (
-                self.sourcepackagename),
-            DistributionSourcePackageInDatabase.distribution == (
-                self.distribution)
-            ).one()
-
-    def _get_bug_reporting_guidelines(self):
-        """See `IBugTarget`."""
-        dsp_in_db = self._self_in_database
-        if dsp_in_db is not None:
-            return dsp_in_db.bug_reporting_guidelines
-        return None
-
-    def _set_bug_reporting_guidelines(self, value):
-        """See `IBugTarget`."""
-        dsp_in_db = self._self_in_database
-        if dsp_in_db is None:
-            dsp_in_db = DistributionSourcePackageInDatabase()
-            dsp_in_db.sourcepackagename = self.sourcepackagename
-            dsp_in_db.distribution = self.distribution
-            Store.of(self.distribution).add(dsp_in_db)
-        dsp_in_db.bug_reporting_guidelines = value
-
-    bug_reporting_guidelines = property(
-        _get_bug_reporting_guidelines,
-        _set_bug_reporting_guidelines)
-
-    def _get_bug_reported_acknowledgement(self):
-        """See `IBugTarget`."""
-        dsp_in_db = self._self_in_database
-        if dsp_in_db is not None:
-            return dsp_in_db.bug_reported_acknowledgement
-        return None
-
-    def _set_bug_reported_acknowledgement(self, value):
-        """See `IBugTarget`."""
-        dsp_in_db = self._self_in_database
-        if dsp_in_db is None:
-            dsp_in_db = DistributionSourcePackageInDatabase()
-            dsp_in_db.sourcepackagename = self.sourcepackagename
-            dsp_in_db.distribution = self.distribution
-            Store.of(self.distribution).add(dsp_in_db)
-        dsp_in_db.bug_reported_acknowledgement = value
-
-    bug_reported_acknowledgement = property(
-        _get_bug_reported_acknowledgement,
-        _set_bug_reported_acknowledgement)
-
-    def _get_max_bug_heat(self):
-        """See `IHasBugs`."""
-        dsp_in_db = self._self_in_database
-        if dsp_in_db is None:
-            return None
-        else:
-            return dsp_in_db.max_bug_heat
-
-    def _set_max_bug_heat(self, value):
-        """See `IHasBugs`."""
-        dsp_in_db = self._self_in_database
-        if dsp_in_db is None:
-            dsp_in_db = DistributionSourcePackageInDatabase()
-            dsp_in_db.sourcepackagename = self.sourcepackagename
-            dsp_in_db.distribution = self.distribution
-            Store.of(self.distribution).add(dsp_in_db)
-        dsp_in_db.max_bug_heat = value
-
-    max_bug_heat = property(_get_max_bug_heat, _set_max_bug_heat)
+        return self._get(self.distribution, self.sourcepackagename)
+
+    def recalculateBugHeatCache(self):
+        """See `IHasBugHeat`."""
+        self.max_bug_heat = IStore(Bug).find(
+            Max(Bug.heat),
+            BugTask.bug == Bug.id,
+            BugTask.distributionID == self.distribution.id,
+            BugTask.sourcepackagenameID == self.sourcepackagename.id).one()
+        if self.max_bug_heat is None:
+            # SELECT MAX(Bug.heat) returns NULL if zero rows match.
+            self.max_bug_heat = 0
+
+        self.total_bug_heat = IStore(Bug).find(
+            Sum(Bug.heat),
+            BugTask.bug == Bug.id,
+            BugTask.distributionID == self.distribution.id,
+            BugTask.sourcepackagenameID == self.sourcepackagename.id).one()
+        if self.total_bug_heat is None:
+            self.total_bug_heat = 0
+
+        self.bug_count = IStore(Bug).find(
+            Count(Bug.heat),
+            BugTask.bug == Bug.id,
+            BugTask.distributionID == self.distribution.id,
+            BugTask.sourcepackagenameID == self.sourcepackagename.id).one()
+        if self.bug_count is None:
+            self.bug_count = 0
 
     @property
     def latest_overall_publication(self):
@@ -504,6 +512,47 @@
             In(Lower(EmailAddress.email), email_addresses))
         return result_set
 
+    @classmethod
+    def _get(cls, distribution, sourcepackagename):
+        return Store.of(distribution).find(
+            DistributionSourcePackageInDatabase,
+            DistributionSourcePackageInDatabase.sourcepackagename ==
+                sourcepackagename,
+            DistributionSourcePackageInDatabase.distribution ==
+                distribution
+            ).one()
+
+    @classmethod
+    def _new(cls, distribution, sourcepackagename, section):
+        dsp = DistributionSourcePackageInDatabase()
+        dsp.distribution = distribution
+        dsp.sourcepackagename = sourcepackagename
+        dsp.section = section
+        Store.of(distribution).add(dsp)
+        return dsp
+
+    @classmethod
+    def ensure(cls, spph):
+        """Create DistributionSourcePackage record, if necessary.
+
+        Only create a record for primary archives (i.e. not for PPAs).
+        If it already exists, update the section attribute.
+        """
+        # Import here to avoid import loop.
+        sourcepackagename = spph.sourcepackagerelease.sourcepackagename
+        distribution = spph.distroseries.distribution
+        section = spph.section
+
+        if spph.archive.purpose == ArchivePurpose.PRIMARY:
+            dsp = cls._get(distribution, sourcepackagename)
+            if dsp is None:
+                dsp = cls._new(distribution, sourcepackagename, section)
+            elif spph.distroseries.status == SeriesStatus.CURRENT:
+                # If the distroseries.status is not CURRENT, it should
+                # not override the section from a previously created
+                # SourcePackagePublishingHistory.
+                dsp.section = section
+
 
 class DistributionSourcePackageInDatabase(Storm):
     """Temporary class to allow access to the database."""
@@ -529,4 +578,9 @@
     bug_reported_acknowledgement = Unicode()
 
     max_bug_heat = Int()
+    total_bug_heat = Int()
+    bug_count = Int()
+    po_message_count = Int()
 
+    section_id = Int(name='section')
+    section = Reference(section_id, 'Section.id')

=== modified file 'lib/lp/soyuz/model/publishing.py'
--- lib/lp/soyuz/model/publishing.py	2010-05-21 12:25:51 +0000
+++ lib/lp/soyuz/model/publishing.py	2010-06-10 01:43:25 +0000
@@ -49,12 +49,12 @@
 from lp.soyuz.model.binarypackagename import BinaryPackageName
 from lp.soyuz.model.binarypackagerelease import (BinaryPackageRelease,
     BinaryPackageReleaseDownloadCount)
+from lp.soyuz.interfaces.archive import ArchivePurpose
 from lp.soyuz.model.files import (
     BinaryPackageFile, SourcePackageReleaseFile)
 from canonical.launchpad.database.librarian import (
     LibraryFileAlias, LibraryFileContent)
 from lp.soyuz.model.packagediff import PackageDiff
-from lp.soyuz.interfaces.archive import ArchivePurpose
 from lp.soyuz.interfaces.archivearch import IArchiveArchSet
 from lp.soyuz.interfaces.binarypackagebuild import (
     BuildSetStatus, IBinaryPackageBuildSet)
@@ -1162,7 +1162,10 @@
             section=section,
             status=PackagePublishingStatus.PENDING,
             datecreated=UTC_NOW)
-
+        # Import here to prevent import loop.
+        from lp.registry.model.distributionsourcepackage import (
+            DistributionSourcePackage)
+        DistributionSourcePackage.ensure(pub)
         return pub
 
     def getBuildsForSourceIds(