Merge lp:~cjwatson/launchpad/publish-distro-many-ppas into lp:launchpad
Status: | Merged |
---|---|
Merged at revision: | 17966 |
Proposed branch: | lp:~cjwatson/launchpad/publish-distro-many-ppas |
Merge into: | lp:launchpad |
Diff against target: |
23 lines (+3/-2) 1 file modified
lib/lp/archivepublisher/scripts/publishdistro.py (+3/-2) |
To merge this branch: | bzr merge lp:~cjwatson/launchpad/publish-distro-many-ppas |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
William Grant | code | Approve | |
Review via email: mp+289925@code.launchpad.net |
Commit message
Make publish-distro use an iterator to walk over PPAs, rather than materialising them up-front.
Description of the change
Make publish-distro use an iterator to walk over PPAs, rather than materialising them up-front.
I'd noticed that commits were very slow on each individual archive when operating on all PPAs. This turns out to be because of accidentally quadratic behaviour: getTargetArchives materialised all archives into memory up-front, which meant that they were all alive as far as the Storm store was concerned, which meant that the pre-commit invalidate call had to walk over them all.
Turning this into an iterator makes things behave much better: commit still takes a variable amount of time depending on how many archives publish-distro inspected and skipped before it had to do any real work, but the total time spent is linear in len(PublishDist