Merge lp:~lifeless/launchpad/bug-722956 into lp:launchpad

Proposed by Robert Collins
Status: Merged
Approved by: Robert Collins
Approved revision: no longer in the source branch.
Merged at revision: 12489
Proposed branch: lp:~lifeless/launchpad/bug-722956
Merge into: lp:launchpad
Diff against target: 606 lines (+159/-55)
20 files modified
lib/lp/app/doc/tales.txt (+2/-0)
lib/lp/app/widgets/suggestion.py (+3/-1)
lib/lp/code/browser/branchlisting.py (+3/-3)
lib/lp/code/feed/branch.py (+1/-1)
lib/lp/code/interfaces/branch.py (+11/-3)
lib/lp/code/interfaces/branchcollection.py (+4/-1)
lib/lp/code/interfaces/branchnamespace.py (+5/-2)
lib/lp/code/interfaces/hasbranches.py (+3/-1)
lib/lp/code/interfaces/seriessourcepackagebranch.py (+7/-0)
lib/lp/code/model/branch.py (+29/-13)
lib/lp/code/model/branchcollection.py (+49/-9)
lib/lp/code/model/branchnamespace.py (+1/-1)
lib/lp/code/model/hasbranches.py (+2/-2)
lib/lp/code/model/seriessourcepackagebranch.py (+6/-1)
lib/lp/code/model/tests/test_branchset.py (+26/-12)
lib/lp/code/model/tests/test_branchtarget.py (+1/-1)
lib/lp/code/tests/test_helpers.py (+2/-1)
lib/lp/code/vocabularies/branch.py (+1/-1)
lib/lp/codehosting/branchdistro.py (+2/-1)
lib/lp/codehosting/scanner/mergedetection.py (+1/-1)
To merge this branch: bzr merge lp:~lifeless/launchpad/bug-722956
Reviewer Review Type Date Requested Status
Robert Collins (community) Approve
Stuart Bishop (community) Needs Information
Review via email: mp+51481@code.launchpad.net

Commit message

[r=lifeless][ui=none] [r=lifeless][bug=722956] Eager load productseries and suite series for the api /branches collection.

Description of the change

The API for /branches is nearly guaranteed to have branches from many different contexts, and we were doing late evaluation on many attributes trying to render the collection. This fixes 2 of those cases and lays the ground work to fix more when the page becomes an issue again.

To post a comment you must log in.
Revision history for this message
Stuart Bishop (stub) wrote :

The new eager_load parameter is exposed to the webservice. What happens when web service requests use this parameter? This seems a quick hack rather than an API improvement.

I suspect we need a new method (that is not exposed to the webservice) rather than adding an optional parameter to the existing method. Or cast the result set to an eager result set (results = IEagerResultSet(foo.getBranches())). Or just a helper (results = foo.getBranches(); eager_load_branches(results)).

Its annoying that the seemingly arbitrary and undocumented ordering in associatedSuiteSourcePackages needs to be duplicated in the eager load method. We should comment this code dependency for when someone wants to change it.

review: Needs Information
Revision history for this message
Robert Collins (lifeless) wrote :

On Mon, Feb 28, 2011 at 9:27 PM, Stuart Bishop
<email address hidden> wrote:
> Review: Needs Information
> The new eager_load parameter is exposed to the webservice. What happens when web service requests use this parameter? This seems a quick hack rather than an API improvement.

Its not exposed to the webservice. The getBranches parameters in the
web service is controlled via the @operation_parameters decorator on
IHasBranches. The variant that I have enabled eager loading for is
only exposed as a collection in the web service: setting a default
parameter lets the method be used in code, and used as a collection in
the webservice, with the default being what the webservice needs.

> I suspect we need a new method (that is not exposed to the webservice) rather than adding an optional parameter to the existing method. Or cast the result set to an eager result set (results = IEagerResultSet(foo.getBranches())). Or just a helper (results = foo.getBranches(); eager_load_branches(results)).

I don't think any of those are needed, because my change isn't exposed
as part of the web service contract; its invisible.

> Its annoying that the seemingly arbitrary and undocumented ordering in associatedSuiteSourcePackages needs to be duplicated in the eager load method. We should comment this code dependency for when someone wants to change it.

I will comment on both sides of the eager load to point this out.

Thanks for the review,
Rob

Revision history for this message
Robert Collins (lifeless) wrote :

I've addressed Stuarts comments; approving to make ec2land happy.

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'lib/lp/app/doc/tales.txt'
--- lib/lp/app/doc/tales.txt 2011-02-03 17:59:50 +0000
+++ lib/lp/app/doc/tales.txt 2011-03-01 04:23:03 +0000
@@ -498,6 +498,8 @@
498498
499 >>> login_person(fooix.owner, LaunchpadTestRequest())499 >>> login_person(fooix.owner, LaunchpadTestRequest())
500 >>> fooix.development_focus.branch = branch500 >>> fooix.development_focus.branch = branch
501 >>> from lp.services.propertycache import clear_property_cache
502 >>> clear_property_cache(branch)
501 >>> print test_tales("branch/fmt:bzr-link", branch=branch)503 >>> print test_tales("branch/fmt:bzr-link", branch=branch)
502 <a href=".../~eric/fooix/bar" class="sprite branch">lp://dev/fooix</a>504 <a href=".../~eric/fooix/bar" class="sprite branch">lp://dev/fooix</a>
503505
504506
=== modified file 'lib/lp/app/widgets/suggestion.py'
--- lib/lp/app/widgets/suggestion.py 2011-02-08 15:43:30 +0000
+++ lib/lp/app/widgets/suggestion.py 2011-03-01 04:23:03 +0000
@@ -231,7 +231,9 @@
231 collection = branch.target.collection.targetedBy(logged_in_user,231 collection = branch.target.collection.targetedBy(logged_in_user,
232 since)232 since)
233 collection = collection.visibleByUser(logged_in_user)233 collection = collection.visibleByUser(logged_in_user)
234 branches = collection.getBranches().config(distinct=True)234 # May actually need some eager loading, but the API isn't fine grained
235 # yet.
236 branches = collection.getBranches(eager_load=False).config(distinct=True)
235 target_branches = list(branches.config(limit=5))237 target_branches = list(branches.config(limit=5))
236 # If there is a development focus branch, make sure it is always238 # If there is a development focus branch, make sure it is always
237 # shown, and as the first item.239 # shown, and as the first item.
238240
=== modified file 'lib/lp/code/browser/branchlisting.py'
--- lib/lp/code/browser/branchlisting.py 2011-02-11 15:09:53 +0000
+++ lib/lp/code/browser/branchlisting.py 2011-03-01 04:23:03 +0000
@@ -650,7 +650,7 @@
650 if lifecycle_status is not None:650 if lifecycle_status is not None:
651 collection = collection.withLifecycleStatus(*lifecycle_status)651 collection = collection.withLifecycleStatus(*lifecycle_status)
652 collection = collection.visibleByUser(self.user)652 collection = collection.visibleByUser(self.user)
653 return collection.getBranches().order_by(653 return collection.getBranches(eager_load=False).order_by(
654 self._listingSortToOrderBy(self.sort_by))654 self._listingSortToOrderBy(self.sort_by))
655655
656 @property656 @property
@@ -809,7 +809,7 @@
809 if lifecycle_status is not None:809 if lifecycle_status is not None:
810 collection = collection.withLifecycleStatus(*lifecycle_status)810 collection = collection.withLifecycleStatus(*lifecycle_status)
811 collection = collection.visibleByUser(self.user)811 collection = collection.visibleByUser(self.user)
812 return collection.getBranches().order_by(812 return collection.getBranches(eager_load=False).order_by(
813 self._branch_order)813 self._branch_order)
814814
815815
@@ -1501,7 +1501,7 @@
1501 # We're only interested in active branches.1501 # We're only interested in active branches.
1502 collection = self.getBranchCollection().withLifecycleStatus(1502 collection = self.getBranchCollection().withLifecycleStatus(
1503 *DEFAULT_BRANCH_STATUS_IN_LISTING)1503 *DEFAULT_BRANCH_STATUS_IN_LISTING)
1504 for branch in collection.getBranches():1504 for branch in collection.getBranches(eager_load=False):
1505 branches.setdefault(branch.distroseries, []).append(branch)1505 branches.setdefault(branch.distroseries, []).append(branch)
1506 return branches1506 return branches
15071507
15081508
=== modified file 'lib/lp/code/feed/branch.py'
--- lib/lp/code/feed/branch.py 2010-08-24 10:45:57 +0000
+++ lib/lp/code/feed/branch.py 2011-03-01 04:23:03 +0000
@@ -163,7 +163,7 @@
163 from lp.code.model.branch import Branch163 from lp.code.model.branch import Branch
164 collection = self._getCollection().visibleByUser(164 collection = self._getCollection().visibleByUser(
165 None).withLifecycleStatus(*DEFAULT_BRANCH_STATUS_IN_LISTING)165 None).withLifecycleStatus(*DEFAULT_BRANCH_STATUS_IN_LISTING)
166 branches = collection.getBranches()166 branches = collection.getBranches(eager_load=False)
167 branches.order_by(167 branches.order_by(
168 Desc(Branch.date_last_modified),168 Desc(Branch.date_last_modified),
169 Asc(Branch.target_suffix),169 Asc(Branch.target_suffix),
170170
=== modified file 'lib/lp/code/interfaces/branch.py'
--- lib/lp/code/interfaces/branch.py 2011-02-28 14:08:54 +0000
+++ lib/lp/code/interfaces/branch.py 2011-03-01 04:23:03 +0000
@@ -708,7 +708,10 @@
708 """708 """
709709
710 def associatedSuiteSourcePackages():710 def associatedSuiteSourcePackages():
711 """Return the suite source packages that this branch is linked to."""711 """Return the suite source packages that this branch is linked to.
712
713 :return: A list of suite source packages ordered by pocket.
714 """
712715
713 def branchLinks():716 def branchLinks():
714 """Return a sorted list of ICanHasLinkedBranch objects.717 """Return a sorted list of ICanHasLinkedBranch objects.
@@ -1286,8 +1289,13 @@
1286 """1289 """
12871290
1288 @collection_default_content()1291 @collection_default_content()
1289 def getBranches(limit=50):1292 def getBranches(limit=50, eager_load=True):
1290 """Return a collection of branches."""1293 """Return a collection of branches.
1294
1295 :param eager_load: If True (the default because this is used in the
1296 web service and it needs the related objects to create links) eager
1297 load related objects (products, code imports etc).
1298 """
12911299
12921300
1293class IBranchListingQueryOptimiser(Interface):1301class IBranchListingQueryOptimiser(Interface):
12941302
=== modified file 'lib/lp/code/interfaces/branchcollection.py'
--- lib/lp/code/interfaces/branchcollection.py 2011-01-20 00:06:17 +0000
+++ lib/lp/code/interfaces/branchcollection.py 2011-03-01 04:23:03 +0000
@@ -57,13 +57,16 @@
57 of individuals and teams that own branches in this collection.57 of individuals and teams that own branches in this collection.
58 """58 """
5959
60 def getBranches():60 def getBranches(eager_load=False):
61 """Return a result set of all branches in this collection.61 """Return a result set of all branches in this collection.
6262
63 The returned result set will also join across the specified tables as63 The returned result set will also join across the specified tables as
64 defined by the arguments to this function. These extra tables are64 defined by the arguments to this function. These extra tables are
65 joined specificly to allow the caller to sort on values not in the65 joined specificly to allow the caller to sort on values not in the
66 Branch table itself.66 Branch table itself.
67
68 :param eager_load: If True trigger eager loading of all the related
69 objects in the collection.
67 """70 """
6871
69 def getMergeProposals(statuses=None, for_branches=None,72 def getMergeProposals(statuses=None, for_branches=None,
7073
=== modified file 'lib/lp/code/interfaces/branchnamespace.py'
--- lib/lp/code/interfaces/branchnamespace.py 2010-08-20 20:31:18 +0000
+++ lib/lp/code/interfaces/branchnamespace.py 2011-03-01 04:23:03 +0000
@@ -52,8 +52,11 @@
52 a given prefix, use createBranchWithPrefix.52 a given prefix, use createBranchWithPrefix.
53 """53 """
5454
55 def getBranches():55 def getBranches(eager_load=False):
56 """Return the branches in this namespace."""56 """Return the branches in this namespace.
57
58 :param eager_load: If True eager load related data for the branches.
59 """
5760
58 def getBranchName(name):61 def getBranchName(name):
59 """Get the potential unique name for a branch called 'name'.62 """Get the potential unique name for a branch called 'name'.
6063
=== modified file 'lib/lp/code/interfaces/hasbranches.py'
--- lib/lp/code/interfaces/hasbranches.py 2010-10-27 07:15:02 +0000
+++ lib/lp/code/interfaces/hasbranches.py 2011-03-01 04:23:03 +0000
@@ -60,13 +60,15 @@
60 @operation_returns_collection_of(Interface) # Really IBranch.60 @operation_returns_collection_of(Interface) # Really IBranch.
61 @export_read_operation()61 @export_read_operation()
62 def getBranches(status=None, visible_by_user=None,62 def getBranches(status=None, visible_by_user=None,
63 modified_since=None):63 modified_since=None, eager_load=False):
64 """Returns all branches with the given lifecycle status.64 """Returns all branches with the given lifecycle status.
6565
66 :param status: A list of statuses to filter with.66 :param status: A list of statuses to filter with.
67 :param visible_by_user: Normally the user who is asking.67 :param visible_by_user: Normally the user who is asking.
68 :param modified_since: If set, filters the branches being returned68 :param modified_since: If set, filters the branches being returned
69 to those that have been modified since the specified date/time.69 to those that have been modified since the specified date/time.
70 :param eager_load: If True load related objects for the whole
71 collection.
70 :returns: A list of `IBranch`.72 :returns: A list of `IBranch`.
71 """73 """
7274
7375
=== modified file 'lib/lp/code/interfaces/seriessourcepackagebranch.py'
--- lib/lp/code/interfaces/seriessourcepackagebranch.py 2010-08-20 20:31:18 +0000
+++ lib/lp/code/interfaces/seriessourcepackagebranch.py 2011-03-01 04:23:03 +0000
@@ -65,6 +65,13 @@
65 :return: An `IResultSet` of `ISeriesSourcePackageBranch` objects.65 :return: An `IResultSet` of `ISeriesSourcePackageBranch` objects.
66 """66 """
6767
68 def findForBranches(branches):
69 """Get the links to source packages from a branch.
70
71 :param branches: A an iterable of `IBranch`.
72 :return: An `IResultSet` of `ISeriesSourcePackageBranch` objects.
73 """
74
68 def findForSourcePackage(sourcepackage):75 def findForSourcePackage(sourcepackage):
69 """Get the links to branches from a source package.76 """Get the links to branches from a source package.
7077
7178
=== modified file 'lib/lp/code/model/branch.py'
--- lib/lp/code/model/branch.py 2011-02-27 19:45:44 +0000
+++ lib/lp/code/model/branch.py 2011-03-01 04:23:03 +0000
@@ -142,6 +142,7 @@
142from lp.services.job.interfaces.job import JobStatus142from lp.services.job.interfaces.job import JobStatus
143from lp.services.job.model.job import Job143from lp.services.job.model.job import Job
144from lp.services.mail.notificationrecipientset import NotificationRecipientSet144from lp.services.mail.notificationrecipientset import NotificationRecipientSet
145from lp.services.propertycache import cachedproperty
145146
146147
147class Branch(SQLBase, BzrIdentityMixin):148class Branch(SQLBase, BzrIdentityMixin):
@@ -724,13 +725,19 @@
724 DeleteCodeImport(self.code_import)()725 DeleteCodeImport(self.code_import)()
725 Store.of(self).flush()726 Store.of(self).flush()
726727
728 @cachedproperty
729 def _associatedProductSeries(self):
730 """Helper for eager loading associatedProductSeries."""
731 # This is eager loaded by BranchCollection.getBranches.
732 # Imported here to avoid circular import.
733 from lp.registry.model.productseries import ProductSeries
734 return Store.of(self).find(
735 ProductSeries,
736 ProductSeries.branch == self)
737
727 def associatedProductSeries(self):738 def associatedProductSeries(self):
728 """See `IBranch`."""739 """See `IBranch`."""
729 # Imported here to avoid circular import.740 return self._associatedProductSeries
730 from lp.registry.model.productseries import ProductSeries
731 return Store.of(self).find(
732 ProductSeries,
733 ProductSeries.branch == self)
734741
735 def getProductSeriesPushingTranslations(self):742 def getProductSeriesPushingTranslations(self):
736 """See `IBranch`."""743 """See `IBranch`."""
@@ -740,14 +747,21 @@
740 ProductSeries,747 ProductSeries,
741 ProductSeries.translations_branch == self)748 ProductSeries.translations_branch == self)
742749
743 def associatedSuiteSourcePackages(self):750 @cachedproperty
744 """See `IBranch`."""751 def _associatedSuiteSourcePackages(self):
752 """Helper for associatedSuiteSourcePackages."""
753 # This is eager loaded by BranchCollection.getBranches.
745 series_set = getUtility(IFindOfficialBranchLinks)754 series_set = getUtility(IFindOfficialBranchLinks)
746 # Order by the pocket to get the release one first.755 # Order by the pocket to get the release one first. If changing this be
756 # sure to also change BranchCollection.getBranches.
747 links = series_set.findForBranch(self).order_by(757 links = series_set.findForBranch(self).order_by(
748 SeriesSourcePackageBranch.pocket)758 SeriesSourcePackageBranch.pocket)
749 return [link.suite_sourcepackage for link in links]759 return [link.suite_sourcepackage for link in links]
750760
761 def associatedSuiteSourcePackages(self):
762 """See `IBranch`."""
763 return self._associatedSuiteSourcePackages
764
751 # subscriptions765 # subscriptions
752 def subscribe(self, person, notification_level, max_diff_lines,766 def subscribe(self, person, notification_level, max_diff_lines,
753 code_review_level, subscribed_by):767 code_review_level, subscribed_by):
@@ -1311,7 +1325,8 @@
1311 branches = all_branches.visibleByUser(1325 branches = all_branches.visibleByUser(
1312 visible_by_user).withLifecycleStatus(*lifecycle_statuses)1326 visible_by_user).withLifecycleStatus(*lifecycle_statuses)
1313 branches = branches.withBranchType(1327 branches = branches.withBranchType(
1314 BranchType.HOSTED, BranchType.MIRRORED).scanned().getBranches()1328 BranchType.HOSTED, BranchType.MIRRORED).scanned().getBranches(
1329 eager_load=False)
1315 branches.order_by(1330 branches.order_by(
1316 Desc(Branch.date_last_modified), Desc(Branch.id))1331 Desc(Branch.date_last_modified), Desc(Branch.id))
1317 if branch_count is not None:1332 if branch_count is not None:
@@ -1327,7 +1342,7 @@
1327 branches = all_branches.visibleByUser(1342 branches = all_branches.visibleByUser(
1328 visible_by_user).withLifecycleStatus(*lifecycle_statuses)1343 visible_by_user).withLifecycleStatus(*lifecycle_statuses)
1329 branches = branches.withBranchType(1344 branches = branches.withBranchType(
1330 BranchType.IMPORTED).scanned().getBranches()1345 BranchType.IMPORTED).scanned().getBranches(eager_load=False)
1331 branches.order_by(1346 branches.order_by(
1332 Desc(Branch.date_last_modified), Desc(Branch.id))1347 Desc(Branch.date_last_modified), Desc(Branch.id))
1333 if branch_count is not None:1348 if branch_count is not None:
@@ -1341,7 +1356,8 @@
1341 """See `IBranchSet`."""1356 """See `IBranchSet`."""
1342 all_branches = getUtility(IAllBranches)1357 all_branches = getUtility(IAllBranches)
1343 branches = all_branches.withLifecycleStatus(1358 branches = all_branches.withLifecycleStatus(
1344 *lifecycle_statuses).visibleByUser(visible_by_user).getBranches()1359 *lifecycle_statuses).visibleByUser(visible_by_user).getBranches(
1360 eager_load=False)
1345 branches.order_by(1361 branches.order_by(
1346 Desc(Branch.date_created), Desc(Branch.id))1362 Desc(Branch.date_created), Desc(Branch.id))
1347 if branch_count is not None:1363 if branch_count is not None:
@@ -1360,10 +1376,10 @@
1360 """See `IBranchSet`."""1376 """See `IBranchSet`."""
1361 return getUtility(IBranchLookup).getByUrls(urls)1377 return getUtility(IBranchLookup).getByUrls(urls)
13621378
1363 def getBranches(self, limit=50):1379 def getBranches(self, limit=50, eager_load=True):
1364 """See `IBranchSet`."""1380 """See `IBranchSet`."""
1365 anon_branches = getUtility(IAllBranches).visibleByUser(None)1381 anon_branches = getUtility(IAllBranches).visibleByUser(None)
1366 branches = anon_branches.scanned().getBranches()1382 branches = anon_branches.scanned().getBranches(eager_load=eager_load)
1367 branches.order_by(1383 branches.order_by(
1368 Desc(Branch.date_last_modified), Desc(Branch.id))1384 Desc(Branch.date_last_modified), Desc(Branch.id))
1369 branches.config(limit=limit)1385 branches.config(limit=limit)
13701386
=== modified file 'lib/lp/code/model/branchcollection.py'
--- lib/lp/code/model/branchcollection.py 2011-01-27 14:02:38 +0000
+++ lib/lp/code/model/branchcollection.py 2011-03-01 04:23:03 +0000
@@ -23,20 +23,26 @@
23from zope.component import getUtility23from zope.component import getUtility
24from zope.interface import implements24from zope.interface import implements
2525
26from canonical.launchpad.components.decoratedresultset import (
27 DecoratedResultSet,
28 )
26from canonical.launchpad.webapp.interfaces import (29from canonical.launchpad.webapp.interfaces import (
27 DEFAULT_FLAVOR,30 DEFAULT_FLAVOR,
28 IStoreSelector,31 IStoreSelector,
29 MAIN_STORE,32 MAIN_STORE,
30 )33 )
31from canonical.launchpad.webapp.vocabulary import CountableIterator34from canonical.launchpad.webapp.vocabulary import CountableIterator
35from canonical.lazr.utils import safe_hasattr
36from lp.bugs.model.bug import Bug
37from lp.bugs.model.bugbranch import BugBranch
32from lp.code.interfaces.branch import user_has_special_branch_access38from lp.code.interfaces.branch import user_has_special_branch_access
33from lp.code.interfaces.branchcollection import (39from lp.code.interfaces.branchcollection import (
34 IBranchCollection,40 IBranchCollection,
35 InvalidFilter,41 InvalidFilter,
36 )42 )
3743from lp.code.interfaces.seriessourcepackagebranch import (
38from lp.bugs.model.bug import Bug44 IFindOfficialBranchLinks,
39from lp.bugs.model.bugbranch import BugBranch45 )
40from lp.code.enums import BranchMergeProposalStatus46from lp.code.enums import BranchMergeProposalStatus
41from lp.code.interfaces.branchlookup import IBranchLookup47from lp.code.interfaces.branchlookup import IBranchLookup
42from lp.code.interfaces.codehosting import LAUNCHPAD_SERVICES48from lp.code.interfaces.codehosting import LAUNCHPAD_SERVICES
@@ -55,6 +61,7 @@
55from lp.registry.model.product import Product61from lp.registry.model.product import Product
56from lp.registry.model.sourcepackagename import SourcePackageName62from lp.registry.model.sourcepackagename import SourcePackageName
57from lp.registry.model.teammembership import TeamParticipation63from lp.registry.model.teammembership import TeamParticipation
64from lp.services.propertycache import get_property_cache
5865
5966
60class GenericBranchCollection:67class GenericBranchCollection:
@@ -89,7 +96,7 @@
8996
90 def count(self):97 def count(self):
91 """See `IBranchCollection`."""98 """See `IBranchCollection`."""
92 return self.getBranches().count()99 return self.getBranches(eager_load=False).count()
93100
94 def ownerCounts(self):101 def ownerCounts(self):
95 """See `IBranchCollection`."""102 """See `IBranchCollection`."""
@@ -136,7 +143,7 @@
136143
137 def _getBranchIdQuery(self):144 def _getBranchIdQuery(self):
138 """Return a Storm 'Select' for the branch IDs in this collection."""145 """Return a Storm 'Select' for the branch IDs in this collection."""
139 select = self.getBranches()._get_select()146 select = self.getBranches(eager_load=False)._get_select()
140 select.columns = (Branch.id,)147 select.columns = (Branch.id,)
141 return select148 return select
142149
@@ -144,11 +151,43 @@
144 """Return the where expressions for this collection."""151 """Return the where expressions for this collection."""
145 return self._branch_filter_expressions152 return self._branch_filter_expressions
146153
147 def getBranches(self):154 def getBranches(self, eager_load=False):
148 """See `IBranchCollection`."""155 """See `IBranchCollection`."""
149 tables = [Branch] + self._tables.values()156 tables = [Branch] + self._tables.values()
150 expressions = self._getBranchExpressions()157 expressions = self._getBranchExpressions()
151 return self.store.using(*tables).find(Branch, *expressions)158 resultset = self.store.using(*tables).find(Branch, *expressions)
159 if not eager_load:
160 return resultset
161 def do_eager_load(rows):
162 branch_ids = set(branch.id for branch in rows)
163 if not branch_ids:
164 return
165 branches = dict((branch.id, branch) for branch in rows)
166 caches = dict((branch.id, get_property_cache(branch))
167 for branch in rows)
168 for cache in caches.values():
169 if not safe_hasattr(cache, '_associatedProductSeries'):
170 cache._associatedProductSeries = []
171 if not safe_hasattr(cache, '_associatedSuiteSourcePackages'):
172 cache._associatedSuiteSourcePackages = []
173 # associatedProductSeries
174 # Imported here to avoid circular import.
175 from lp.registry.model.productseries import ProductSeries
176 for productseries in self.store.find(
177 ProductSeries,
178 ProductSeries.branchID.is_in(branch_ids)):
179 cache = caches[productseries.branchID]
180 cache._associatedProductSeries.append(productseries)
181 # associatedSuiteSourcePackages
182 series_set = getUtility(IFindOfficialBranchLinks)
183 # Order by the pocket to get the release one first. If changing
184 # this be sure to also change BranchCollection.getBranches.
185 links = series_set.findForBranches(rows).order_by(
186 SeriesSourcePackageBranch.pocket)
187 for link in links:
188 cache = caches[link.branchID]
189 cache._associatedSuiteSourcePackages.append(link)
190 return DecoratedResultSet(resultset, pre_iter_hook=do_eager_load)
152191
153 def getMergeProposals(self, statuses=None, for_branches=None,192 def getMergeProposals(self, statuses=None, for_branches=None,
154 target_branch=None, merged_revnos=None):193 target_branch=None, merged_revnos=None):
@@ -364,7 +403,7 @@
364 # of the unique name and sort based on relevance.403 # of the unique name and sort based on relevance.
365 branch = self._getExactMatch(search_term)404 branch = self._getExactMatch(search_term)
366 if branch is not None:405 if branch is not None:
367 if branch in self.getBranches():406 if branch in self.getBranches(eager_load=False):
368 return CountableIterator(1, [branch])407 return CountableIterator(1, [branch])
369 else:408 else:
370 return CountableIterator(0, [])409 return CountableIterator(0, [])
@@ -397,7 +436,8 @@
397436
398 # Get the results.437 # Get the results.
399 collection = self._filterBy([Branch.id.is_in(Union(*queries))])438 collection = self._filterBy([Branch.id.is_in(Union(*queries))])
400 results = collection.getBranches().order_by(Branch.name, Branch.id)439 results = collection.getBranches(eager_load=False).order_by(
440 Branch.name, Branch.id)
401 return CountableIterator(results.count(), results)441 return CountableIterator(results.count(), results)
402442
403 def scanned(self):443 def scanned(self):
404444
=== modified file 'lib/lp/code/model/branchnamespace.py'
--- lib/lp/code/model/branchnamespace.py 2010-10-18 02:37:53 +0000
+++ lib/lp/code/model/branchnamespace.py 2011-03-01 04:23:03 +0000
@@ -218,7 +218,7 @@
218 name = "%s-%s" % (prefix, count)218 name = "%s-%s" % (prefix, count)
219 return name219 return name
220220
221 def getBranches(self):221 def getBranches(self, eager_load=False):
222 """See `IBranchNamespace`."""222 """See `IBranchNamespace`."""
223 store = getUtility(IStoreSelector).get(MAIN_STORE, DEFAULT_FLAVOR)223 store = getUtility(IStoreSelector).get(MAIN_STORE, DEFAULT_FLAVOR)
224 return store.find(Branch, self._getBranchesClause())224 return store.find(Branch, self._getBranchesClause())
225225
=== modified file 'lib/lp/code/model/hasbranches.py'
--- lib/lp/code/model/hasbranches.py 2010-10-27 07:15:02 +0000
+++ lib/lp/code/model/hasbranches.py 2011-03-01 04:23:03 +0000
@@ -26,7 +26,7 @@
26 """A mixin implementation for `IHasBranches`."""26 """A mixin implementation for `IHasBranches`."""
2727
28 def getBranches(self, status=None, visible_by_user=None,28 def getBranches(self, status=None, visible_by_user=None,
29 modified_since=None):29 modified_since=None, eager_load=False):
30 """See `IHasBranches`."""30 """See `IHasBranches`."""
31 if status is None:31 if status is None:
32 status = DEFAULT_BRANCH_STATUS_IN_LISTING32 status = DEFAULT_BRANCH_STATUS_IN_LISTING
@@ -35,7 +35,7 @@
35 collection = collection.withLifecycleStatus(*status)35 collection = collection.withLifecycleStatus(*status)
36 if modified_since is not None:36 if modified_since is not None:
37 collection = collection.modifiedSince(modified_since)37 collection = collection.modifiedSince(modified_since)
38 return collection.getBranches()38 return collection.getBranches(eager_load=eager_load)
3939
4040
41class HasMergeProposalsMixin:41class HasMergeProposalsMixin:
4242
=== modified file 'lib/lp/code/model/seriessourcepackagebranch.py'
--- lib/lp/code/model/seriessourcepackagebranch.py 2010-08-20 20:31:18 +0000
+++ lib/lp/code/model/seriessourcepackagebranch.py 2011-03-01 04:23:03 +0000
@@ -101,10 +101,15 @@
101101
102 def findForBranch(self, branch):102 def findForBranch(self, branch):
103 """See `IFindOfficialBranchLinks`."""103 """See `IFindOfficialBranchLinks`."""
104 return self.findForBranches([branch])
105
106 def findForBranches(self, branches):
107 """See `IFindOfficialBranchLinks`."""
108 branch_ids = set(branch.id for branch in branches)
104 store = getUtility(IStoreSelector).get(MAIN_STORE, DEFAULT_FLAVOR)109 store = getUtility(IStoreSelector).get(MAIN_STORE, DEFAULT_FLAVOR)
105 return store.find(110 return store.find(
106 SeriesSourcePackageBranch,111 SeriesSourcePackageBranch,
107 SeriesSourcePackageBranch.branch == branch.id)112 SeriesSourcePackageBranch.branchID.is_in(branch_ids))
108113
109 def findForSourcePackage(self, sourcepackage):114 def findForSourcePackage(self, sourcepackage):
110 """See `IFindOfficialBranchLinks`."""115 """See `IFindOfficialBranchLinks`."""
111116
=== modified file 'lib/lp/code/model/tests/test_branchset.py'
--- lib/lp/code/model/tests/test_branchset.py 2010-10-04 19:50:45 +0000
+++ lib/lp/code/model/tests/test_branchset.py 2011-03-01 04:23:03 +0000
@@ -5,32 +5,34 @@
55
6__metaclass__ = type6__metaclass__ = type
77
8from unittest import TestLoader8from testtools.matchers import LessThan
99
10from canonical.launchpad.testing.pages import LaunchpadWebServiceCaller
10from canonical.testing.layers import DatabaseFunctionalLayer11from canonical.testing.layers import DatabaseFunctionalLayer
11from lp.code.interfaces.branch import IBranchSet12from lp.code.interfaces.branch import IBranchSet
12from lp.code.model.branch import BranchSet13from lp.code.model.branch import BranchSet
13from lp.testing import TestCaseWithFactory14from lp.testing import (
15 logout,
16 TestCaseWithFactory,
17 )
18from lp.testing._webservice import QueryCollector
19from lp.testing.matchers import HasQueryCount
1420
1521
16class TestBranchSet(TestCaseWithFactory):22class TestBranchSet(TestCaseWithFactory):
1723
18 layer = DatabaseFunctionalLayer24 layer = DatabaseFunctionalLayer
1925
20 def setUp(self):
21 TestCaseWithFactory.setUp(self)
22 self.branch_set = BranchSet()
23
24 def test_provides_IBranchSet(self):26 def test_provides_IBranchSet(self):
25 # BranchSet instances provide IBranchSet.27 # BranchSet instances provide IBranchSet.
26 self.assertProvides(self.branch_set, IBranchSet)28 self.assertProvides(BranchSet(), IBranchSet)
2729
28 def test_getByUrls(self):30 def test_getByUrls(self):
29 # getByUrls returns a list of branches matching the list of URLs that31 # getByUrls returns a list of branches matching the list of URLs that
30 # it's given.32 # it's given.
31 a = self.factory.makeAnyBranch()33 a = self.factory.makeAnyBranch()
32 b = self.factory.makeAnyBranch()34 b = self.factory.makeAnyBranch()
33 branches = self.branch_set.getByUrls(35 branches = BranchSet().getByUrls(
34 [a.bzr_identity, b.bzr_identity])36 [a.bzr_identity, b.bzr_identity])
35 self.assertEqual({a.bzr_identity: a, b.bzr_identity: b}, branches)37 self.assertEqual({a.bzr_identity: a, b.bzr_identity: b}, branches)
3638
@@ -38,9 +40,21 @@
38 # If a branch cannot be found for a URL, then None appears in the list40 # If a branch cannot be found for a URL, then None appears in the list
39 # in place of the branch.41 # in place of the branch.
40 url = 'http://example.com/doesntexist'42 url = 'http://example.com/doesntexist'
41 branches = self.branch_set.getByUrls([url])43 branches = BranchSet().getByUrls([url])
42 self.assertEqual({url: None}, branches)44 self.assertEqual({url: None}, branches)
4345
4446 def test_api_branches_query_count(self):
45def test_suite():47 webservice = LaunchpadWebServiceCaller()
46 return TestLoader().loadTestsFromName(__name__)48 collector = QueryCollector()
49 collector.register()
50 self.addCleanup(collector.unregister)
51 # Get 'all' of the 50 branches this collection is limited to - rather
52 # than the default in-test-suite pagination size of 5.
53 url = "/branches?ws.size=50"
54 logout()
55 response = webservice.get(url,
56 headers={'User-Agent': 'AnonNeedsThis'})
57 self.assertEqual(response.status, 200,
58 "Got %d for url %r with response %r" % (
59 response.status, url, response.body))
60 self.assertThat(collector, HasQueryCount(LessThan(13)))
4761
=== modified file 'lib/lp/code/model/tests/test_branchtarget.py'
--- lib/lp/code/model/tests/test_branchtarget.py 2011-01-31 02:12:30 +0000
+++ lib/lp/code/model/tests/test_branchtarget.py 2011-03-01 04:23:03 +0000
@@ -53,7 +53,7 @@
53 # branches related to the branch target.53 # branches related to the branch target.
54 self.assertEqual(self.target.collection.getBranches().count(), 0)54 self.assertEqual(self.target.collection.getBranches().count(), 0)
55 branch = self.makeBranchForTarget()55 branch = self.makeBranchForTarget()
56 branches = self.target.collection.getBranches()56 branches = self.target.collection.getBranches(eager_load=False)
57 self.assertEqual([branch], list(branches))57 self.assertEqual([branch], list(branches))
5858
59 def test_retargetBranch_packageBranch(self):59 def test_retargetBranch_packageBranch(self):
6060
=== modified file 'lib/lp/code/tests/test_helpers.py'
--- lib/lp/code/tests/test_helpers.py 2010-10-04 19:50:45 +0000
+++ lib/lp/code/tests/test_helpers.py 2011-03-01 04:23:03 +0000
@@ -35,6 +35,7 @@
35 self.assertIsNot(None, fooix)35 self.assertIsNot(None, fooix)
36 # There should be one branch with one commit.36 # There should be one branch with one commit.
37 [branch] = list(37 [branch] = list(
38 getUtility(IAllBranches).inProduct(fooix).getBranches())38 getUtility(IAllBranches).inProduct(fooix).getBranches(
39 eager_load=False))
39 self.assertEqual(1, branch.revision_count)40 self.assertEqual(1, branch.revision_count)
40 self.assertEqual(commit_time, branch.getTipRevision().revision_date)41 self.assertEqual(commit_time, branch.getTipRevision().revision_date)
4142
=== modified file 'lib/lp/code/vocabularies/branch.py'
--- lib/lp/code/vocabularies/branch.py 2010-12-01 18:58:44 +0000
+++ lib/lp/code/vocabularies/branch.py 2011-03-01 04:23:03 +0000
@@ -69,7 +69,7 @@
69 logged_in_user = getUtility(ILaunchBag).user69 logged_in_user = getUtility(ILaunchBag).user
70 collection = self._getCollection().visibleByUser(logged_in_user)70 collection = self._getCollection().visibleByUser(logged_in_user)
71 if query is None:71 if query is None:
72 branches = collection.getBranches()72 branches = collection.getBranches(eager_load=False)
73 else:73 else:
74 branches = collection.search(query)74 branches = collection.search(query)
75 return CountableIterator(branches.count(), branches, self.toTerm)75 return CountableIterator(branches.count(), branches, self.toTerm)
7676
=== modified file 'lib/lp/codehosting/branchdistro.py'
--- lib/lp/codehosting/branchdistro.py 2010-11-05 14:17:11 +0000
+++ lib/lp/codehosting/branchdistro.py 2011-03-01 04:23:03 +0000
@@ -146,7 +146,8 @@
146 """146 """
147 branches = getUtility(IAllBranches)147 branches = getUtility(IAllBranches)
148 distroseries_branches = branches.inDistroSeries(self.old_distroseries)148 distroseries_branches = branches.inDistroSeries(self.old_distroseries)
149 return distroseries_branches.officialBranches().getBranches()149 return distroseries_branches.officialBranches().getBranches(
150 eager_load=False)
150151
151 def checkConsistentOfficialPackageBranch(self, db_branch):152 def checkConsistentOfficialPackageBranch(self, db_branch):
152 """Check that `db_branch` is a consistent official package branch.153 """Check that `db_branch` is a consistent official package branch.
153154
=== modified file 'lib/lp/codehosting/scanner/mergedetection.py'
--- lib/lp/codehosting/scanner/mergedetection.py 2010-10-08 20:25:27 +0000
+++ lib/lp/codehosting/scanner/mergedetection.py 2011-03-01 04:23:03 +0000
@@ -97,7 +97,7 @@
97 BranchLifecycleStatus.DEVELOPMENT,97 BranchLifecycleStatus.DEVELOPMENT,
98 BranchLifecycleStatus.EXPERIMENTAL,98 BranchLifecycleStatus.EXPERIMENTAL,
99 BranchLifecycleStatus.MATURE,99 BranchLifecycleStatus.MATURE,
100 BranchLifecycleStatus.ABANDONED).getBranches()100 BranchLifecycleStatus.ABANDONED).getBranches(eager_load=False)
101 for branch in branches:101 for branch in branches:
102 last_scanned = branch.last_scanned_id102 last_scanned = branch.last_scanned_id
103 # If the branch doesn't have any revisions, not any point setting103 # If the branch doesn't have any revisions, not any point setting