Launchpad itself

Product:+packages timesout with no packages and one series

Bug #663857 reported by Curtis Hovey
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Launchpad itself
Fix Released
Critical
Edwin Grubbs
Launchpad itself 11.02

Bug Description

OOPS-1754F409 https://code.edge.launchpad.net/obsolete-junk/+packages timesout.
(newer oops)
OOPS-1773EA141

1. This page should not be accessible to the code vhost.
2. This project has no packages (and delete series ensures that)
3. There is only one active series.

This page is timing out as it iterates over hundred of obsolete series that do not have packages,

Branch: launchpad-rev-11858
Revno: 11858
SQL time: 12866 ms
Non-sql time: 2765 ms
Total time: 15631 ms
Statement Count: 1319

@ 1300 queries, we need 10ms performance to avoid death by sql. Changing the dimension from packages to 'sets to query' should fix this quite handily.

See original description

Related branches

lp:~edwin-grubbs/launchpad/bug-663857-product-packages-timeout
Curtis Hovey (community): Approve (code ui)
Diff: 308 lines (+86/-59)
5 files modified
lib/lp/registry/browser/product.py (+4/-19)
lib/lp/registry/browser/tests/packaging-views.txt (+19/-20)
lib/lp/registry/interfaces/product.py (+3/-0)
lib/lp/registry/model/product.py (+27/-7)
lib/lp/registry/templates/product-packages.pt (+33/-13)
lp:~edwin-grubbs/launchpad/bug-663857-product-packages-timeout-part2
Tim Penhey (community): Approve (code)
Curtis Hovey (community): Approve (code)
Steve Kowalik (community): Approve (code*)
Diff: 106 lines (+32/-16)
4 files modified
lib/lp/registry/browser/product.py (+9/-13)
lib/lp/registry/interfaces/product.py (+2/-0)
lib/lp/registry/model/product.py (+16/-0)
lib/lp/registry/templates/product-packages.pt (+5/-3)
Revision history for this message
Robert Collins (lifeless) wrote :

Hmm, that oops is from +milestones. Odd.

Perhaps 1) should be a different bug?

description: updated
Edwin Grubbs (edwin-grubbs)
Changed in launchpad:
assignee: nobody → Edwin Grubbs (edwin-grubbs)
milestone: none → 11.02
status: Triaged → In Progress
Robert Collins (lifeless)
Changed in launchpad:
importance: High → Critical
Revision history for this message
Edwin Grubbs (edwin-grubbs) wrote :

I think obsolete series with no packages can be removed from this page. A user can still link an obsolete series to a package on the series +index page.

I will paginate the results and see about reducing the number of queries. I also think it would make sense if the series were sorted descending.

Revision history for this message
Robert Collins (lifeless) wrote : Re: [Bug 663857] [NEW] Product:+packages timesout with no packages and one series

We should show series with no packages, because they may just need linking.

Revision history for this message
Edwin Grubbs (edwin-grubbs) wrote :

For an obsolete series, what benefit is there to linking its packages? How often would that even happen?

Revision history for this message
Robert Collins (lifeless) wrote : Re: [Bug 663857] Re: Product:+packages timesout with no packages and one series

Ah, obsolete series are less interesting indeed. Can they be linked in
some other fashion?

Also, I'm curious how many queries the page is doing: whats the
scaling factor per series?

Revision history for this message
Curtis Hovey (sinzui) wrote :

There is no benefit to link to an obsolete series, but series were linked to packages ans then became obsolete.

Revision history for this message
Launchpad QA Bot (lpqabot) wrote : Bug fixed by a commit

Fixed in stable r12236 <http://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable/revision/12236>.

tags: added: qa-needstesting
Changed in launchpad:
status: In Progress → Fix Committed
Revision history for this message
Robert Collins (lifeless) wrote :

So, the page does 600 queries and still times out. I think we should focus just on the efficiency rather than reducing content: 600 queries is totally excessive for this. Marking qa-ok to let us deploy - its no worse than it was AFAICT.

tags: added: qa-ok
removed: qa-needstesting
Revision history for this message
Edwin Grubbs (edwin-grubbs) wrote :

I just landed a secondary patch last night. The problem with the first fix is that there was a second place in the view where it loops through all the series to find the packages from the project. Once this gets through buildbot, the number of queries should be much more reasonable. Unfortunately, another branch that ran through buildbot at the same time had a test error.

Revision history for this message
Launchpad QA Bot (lpqabot) wrote :

Fixed in stable r12252 <http://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable/revision/12252>.

tags: added: qa-needstesting
removed: qa-ok
Revision history for this message
Robert Collins (lifeless) wrote :

https://code.qastaging.launchpad.net/obsolete-junk/+packages now passes - 0.27s. I think we could consider showing (perhaps paginated) all series again. (Because I get the impression that we're still bring back substantial data?) Anyhow, this timeout has been stomped on! great stuff Edwin.

tags: added: qa-ok
removed: qa-needstesting
Revision history for this message
Robert Collins (lifeless) wrote :

We need to deploy the later fix to truely fix this.

William Grant (wgrant)
Changed in launchpad:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.