Merge lp:~jelmer/launchpad/506256-remove-popen into lp:launchpad
|Approved by:||Graham Binns on 2010-09-07|
|Merged at revision:||11566|
|Diff against target:||
1176 lines (+222/-405)
17 files modified
|To merge this branch:||bzr merge lp:~jelmer/launchpad/506256-remove-popen|
Builddmaster now moves build binaries away for later processing rather than invoking process-uploader on them directly.
Description of the Change
buildd-manager currently invokes the uploadprocessor on binaries that it has fetched from the buildd slaves.
As this process is synchronous is blocks the buildd manager from doing other things at the same time - such as scheduling new builds - time during which the buildd slaves are idling.
This branch changes the buildd manager to move build results out of the way into a queue that can independently be processed by process-upload (extensions for process-upload to support this were landed earlier).
There is still a single test testNoFiles() in the archiveuploader that is itself buggy. I'm looking into this at the moment, but wanted to submit the branch for review earlier because of the upcoming PQM closure. The fix for this test shouldn't involve any changes outside of the test itself.
This branch has been running on dogfood for the past week or so in its current form and has been working well. We've tested with multiple buildds and thrown several hundred builds at it.
A cron job needs to be set up to run the following command regularly:
- 11080. By Jelmer Vernooij on 2010-09-06
Fix some more tests.
- 11081. By Jelmer Vernooij on 2010-09-06
Work with jobs rather than builds.
- 11082. By Jelmer Vernooij on 2010-09-06
- 11083. By Jelmer Vernooij on 2010-09-06
Cope with new behaviour in buildd-
- 11084. By Jelmer Vernooij on 2010-09-06
- 11085. By Jelmer Vernooij on 2010-09-06
Revert some unused changes.
- 11086. By Jelmer Vernooij on 2010-09-07
Display uploading builds in counts.