Merge lp:~julian-edwards/launchpad/slow-retry-depwait-bug-590757 into lp:launchpad
Status: | Merged |
---|---|
Approved by: | Graham Binns |
Approved revision: | no longer in the source branch. |
Merged at revision: | 10965 |
Proposed branch: | lp:~julian-edwards/launchpad/slow-retry-depwait-bug-590757 |
Merge into: | lp:launchpad |
Diff against target: |
232 lines (+45/-24) 6 files modified
lib/lp/soyuz/doc/archive.txt (+2/-2) lib/lp/soyuz/doc/binarypackagebuild.txt (+3/-0) lib/lp/soyuz/interfaces/archive.py (+2/-2) lib/lp/soyuz/model/archive.py (+22/-15) lib/lp/soyuz/model/binarypackagebuild.py (+14/-5) lib/lp/soyuz/tests/test_binarypackagebuild.py (+2/-0) |
To merge this branch: | bzr merge lp:~julian-edwards/launchpad/slow-retry-depwait-bug-590757 |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Francis J. Lacoste (community) | release-critical | Approve | |
Graham Binns (community) | code | Approve | |
Review via email:
|
Description of the change
= Summary =
Fix slow retry-depwait script
== Proposed fix ==
buildd-
over 2 hours. This is due to some very inefficient queries which seem to have
been exposed as such by the recent build schema re-modelling.
== Pre-implementation notes ==
I've chatted to stub and there's a bunch of improvements made to the code and
to the queries:
* Materialise query results early to avoid re-issuing long ones
* Use the slave database where possible which will be much quicker than the
master, and also frees the master to make the whole of LP faster
* The query in IArchive.
It was using a view called PublishedPackage which was causing many unnecessary
joins on unrelated tables. The new query only joins what it has to, and it
also uses the slave store.
Unfortunately the use of a slave store means the tests needed a liberal
sprinkling of commit() which will slow them down. :(
There's also a bunch of lint fixed.
== Tests ==
bin/test -cvvt binarypackagebu
== Demo and Q/A ==
I've QA'ed this already on dogfood. Without the change it takes 9 hours to
run the script. With the change, it takes 50 minutes. I expect a similar
relative improvement on production.
= Launchpad lint =
Checking for conflicts. and issues in doctests and templates.
Running jslint, xmllint, pyflakes, and pylint.
Using normal rules.
Linting changed files:
lib/lp/
lib/lp/
lib/lp/
lib/lp/
lib/lp/
lib/lp/
== Pylint notices ==
*Ridiculously wrong output deleted*