Buildmaster should not call "uploader" script for processing incoming binaries
Bug #128259 reported by
Celso Providelo
| Affects | Status | Importance | Assigned to | Milestone | |
|---|---|---|---|---|---|
| Launchpad itself |
Fix Released
|
High
|
Jelmer Vernooij | ||
Bug Description
Currently slave-scanner call a 'uploader' script, defined in configuration file, in order to process the incoming binaries.
We should modify it to build a NascentUpload object directly, it will improve the performance (it won't require another initZopeless setup), increase the level of information, including error handling, about the upload itself and also help us to send more complete build-failure-
OTOH, dealing with NU objects in the same transaction will require migrating the 'uploader' DB permissions to the 'buildmaster' scope, which should be considered as a side-effect.
Related branches
lp:~jelmer/launchpad/128259-async-1
- Abel Deuring (community): Approve (code)
-
Diff: 559 lines (+230/-44)4 files modifiedlib/lp/archiveuploader/tests/test_uploadprocessor.py (+112/-7)
lib/lp/archiveuploader/uploadpolicy.py (+2/-1)
lib/lp/archiveuploader/uploadprocessor.py (+108/-33)
lib/lp/soyuz/scripts/soyuz_process_upload.py (+8/-3)
lp:~jelmer/launchpad/506256-remove-popen-2
- Michael Nelson (community): Approve (code)
-
Diff: 1585 lines (+374/-273)27 files modifieddatabase/schema/security.cfg (+2/-0)
lib/lp/archiveuploader/dscfile.py (+0/-29)
lib/lp/archiveuploader/nascentupload.py (+31/-16)
lib/lp/archiveuploader/nascentuploadfile.py (+59/-35)
lib/lp/archiveuploader/tests/__init__.py (+5/-7)
lib/lp/archiveuploader/tests/nascentupload.txt (+4/-5)
lib/lp/archiveuploader/tests/test_buildduploads.py (+7/-8)
lib/lp/archiveuploader/tests/test_nascentuploadfile.py (+61/-0)
lib/lp/archiveuploader/tests/test_ppauploadprocessor.py (+11/-12)
lib/lp/archiveuploader/tests/test_recipeuploads.py (+8/-12)
lib/lp/archiveuploader/tests/test_uploadprocessor.py (+99/-30)
lib/lp/archiveuploader/tests/uploadpolicy.txt (+1/-8)
lib/lp/archiveuploader/uploadpolicy.py (+12/-14)
lib/lp/archiveuploader/uploadprocessor.py (+16/-19)
lib/lp/buildmaster/interfaces/packagebuild.py (+8/-4)
lib/lp/buildmaster/model/packagebuild.py (+11/-2)
lib/lp/buildmaster/tests/test_packagebuild.py (+1/-1)
lib/lp/code/configure.zcml (+1/-5)
lib/lp/code/model/sourcepackagerecipebuild.py (+4/-29)
lib/lp/code/model/tests/test_sourcepackagerecipebuild.py (+6/-0)
lib/lp/soyuz/doc/build-failedtoupload-workflow.txt (+2/-3)
lib/lp/soyuz/doc/buildd-slavescanner.txt (+0/-3)
lib/lp/soyuz/doc/distroseriesqueue-translations.txt (+3/-5)
lib/lp/soyuz/doc/soyuz-set-of-uploads.txt (+3/-20)
lib/lp/soyuz/model/binarypackagebuild.py (+4/-0)
lib/lp/soyuz/scripts/soyuz_process_upload.py (+6/-6)
lib/lp/soyuz/tests/test_binarypackagebuild.py (+9/-0)
| Changed in soyuz: | |
| importance: | Undecided → Medium |
| status: | New → Confirmed |
| Changed in soyuz: | |
| milestone: | 1.1.12 → 1.2.1 |
| Changed in soyuz: | |
| milestone: | 1.2.1 → 1.2.2 |
| Changed in soyuz: | |
| milestone: | 1.2.2 → 1.2.3 |
| Changed in soyuz: | |
| milestone: | 1.2.3 → none |
Revision history for this message
| Julian Edwards (julian-edwards) wrote | #1 |
| Changed in soyuz: | |
| importance: | Medium → High |
| tags: | added: buildd-manager |
| Changed in soyuz: | |
| assignee: | nobody → Jelmer Vernooij (jelmer) |
Revision history for this message
| Launchpad QA Bot (lpqabot) wrote Bug fixed by a commit | #2 |
| tags: | added: qa-needstesting |
| Changed in soyuz: | |
| milestone: | none → 10.09 |
| Changed in soyuz: | |
| status: | Triaged → Fix Committed |
| Changed in soyuz: | |
| status: | Fix Committed → Triaged |
| tags: | removed: qa-needstesting |
| Changed in soyuz: | |
| milestone: | 10.09 → 10.10 |
Revision history for this message
| Launchpad QA Bot (lpqabot) wrote | #3 |
| tags: | added: qa-needstesting |
| Changed in soyuz: | |
| status: | Triaged → Fix Committed |
| tags: |
added: qa-ok removed: qa-needstesting |
| Changed in soyuz: | |
| status: | Fix Committed → Fix Released |
To post a comment you must log in.
We're not doing this like the description now, instead we'll throw the uploads into a queue which will be processed later, outside of the build manager.