Launchpad itself

Merge lp:~wgrant/launchpad/link-uploaded-ddebs into lp:launchpad/db-devel

  1. link-uploaded-ddebs
  2. Merge into db-devel
Proposed by William Grant on 2010-07-12
Status: Merged
Approved by: Jelmer Vernooij on 2010-07-22
Approved revision: no longer in the source branch.
Merged at revision: 9567
Proposed branch: lp:~wgrant/launchpad/link-uploaded-ddebs
Merge into: lp:launchpad/db-devel
Diff against target: 674 lines (+269/-81)
16 files modified
database/schema/comments.sql (+1/-0)
database/schema/patch-2207-73-0.sql (+10/-0)
lib/lp/archiveuploader/nascentupload.py (+46/-3)
lib/lp/archiveuploader/nascentuploadfile.py (+11/-19)
lib/lp/archiveuploader/tests/data/suite/debug_1.0-1_broken/debug_1.0-1.dsc (+13/-0)
lib/lp/archiveuploader/tests/data/suite/debug_1.0-1_broken/debug_1.0-1_i386.changes (+31/-0)
lib/lp/archiveuploader/tests/data/suite/debug_1.0-1_broken/debug_1.0-1_source.changes (+26/-0)
lib/lp/archiveuploader/tests/nascentupload-ddebs.txt (+80/-41)
lib/lp/soyuz/doc/archive-files.txt (+8/-0)
lib/lp/soyuz/doc/binarypackagebuild.txt (+3/-2)
lib/lp/soyuz/doc/distroarchseriesbinarypackage.txt (+4/-2)
lib/lp/soyuz/interfaces/binarypackagebuild.py (+9/-1)
lib/lp/soyuz/interfaces/binarypackagerelease.py (+5/-1)
lib/lp/soyuz/model/binarypackagebuild.py (+17/-10)
lib/lp/soyuz/model/binarypackagerelease.py (+2/-0)
lib/lp/soyuz/tests/test_publishing.py (+3/-2)
To merge this branch: bzr merge lp:~wgrant/launchpad/link-uploaded-ddebs
Reviewer Review Type Date Requested Status
Jelmer Vernooij (community) code Approve on 2010-07-22
Julian Edwards (community) direction 2010-07-12 Approve on 2010-07-21
Stuart Bishop db 2010-07-21 Approve on 2010-07-21
Robert Collins (community) db 2010-07-21 Approve on 2010-07-21
Review via email: mp+29669@code.launchpad.net

Commit Message

DDEBs are now linked to and respect the overrides of their corresponding DEBs.

Description of the Change

This branch alters the upload processor to link DDEBs and their DEBs, a piece critical to the pleasant DDEB experience which will be provided by this series of branches.

DDEBs contain debug symbols for their corresponding DEBs, so they need to be very tightly tied and always have matching changes performed. The only way to determine the link between two uploaded files is by comparing names within a build, so it is most efficient and simplest to store this link explicitly as the upload is processed. BinaryPackageRelease.debug_package, a new column, will provide the link from a DEB to its matching DDEB.

In addition to being linked to the corresponding DEB, DDEBs will now also inherit the DEB's overrides. This fixes bug #430025

To post a comment you must log in.
Robert Collins (lifeless) wrote :

Ok from a schema pov, I haven't done a code review.

review: Approve (db)
Julian Edwards (julian-edwards) wrote :

<bigjools> wgrant: unit tests please, not tests in doctests
<bigjools> wgrant: also consider factoring out your new code so you can write unit tests for it
<bigjools> Running the upload processor in tests is something I want to reduce, not increase ;)
<bigjools> wgrant: finally, my alarm bells go off very strongly when I see removeSecurityProxy being used in code
<wgrant> bigjools: Fair point.
<wgrant> In my defense, it's a reasonably large test to rewrite, and I revived this branch from October last year. I'll probably replace nascentupload-ddebs.txt in a separate branch.
<bigjools> ok
<wgrant> And yes, I strongly considered factoring that out.
<bigjools> you should XXX it and file a bug plz
<wgrant> But it means a significant restructure, so... maybe for another branch.
<bigjools> that's fine
<bigjools> thanks for fixing ddebs!

review: Needs Fixing (code)
William Grant (wgrant) wrote :

The removeSecurityProxy call is gone. DDEBs are created in the DB first, and BaseBinaryUploadFile.storeInDatabase itself now finds the matching DDEB in the build and adds it to the creation arguments.

Stuart Bishop (stub) wrote :

We can make it easier to clean out unwanted binarypackages:

ALTER TABLE BinaryPackageRelease
ADD COLUMN debug_package integer REFERENCES BinaryPackageRelease ON DELETE SET NULL;

Otherwise fine. patch-2207-73-0.sql

review: Approve (db)
Julian Edwards (julian-edwards) wrote :

I'm happy with the direction this is going now, although I don't have the spare time to do a thorough review right now. Provided someone else blesses this then that's ok with me too.

<bigjools> wgrant: don't forget to file a bug about fixing the tests though
<bigjools> the basic doc tests are fine, we just don't want unit tests in them
<wgrant> Hmm.
<bigjools> and also I would prefer unit tests instead of uploading whole new packages where possible
<wgrant> Oh, certainly.
<bigjools> if you add unit tests to doctests, various people like jml will hunt you down :)
<wgrant> But archiveuploader isn't awesome for that at the moment.
<bigjools> you need to factor out your new code
<bigjools> then it becomes testable :)
<wgrant> I need to move around some other stuff before I can do that.
<bigjools> ok
<bigjools> like I said, feel free to file a bug about it

review: Approve (direction)
Jelmer Vernooij (jelmer) wrote :
Download full text (3.4 KiB)

> === added file 'database/schema/patch-2207-73-0.sql'
> --- database/schema/patch-2207-73-0.sql 1970-01-01 00:00:00 +0000
> +++ database/schema/patch-2207-73-0.sql 2010-07-21 12:53:20 +0000
> @@ -0,0 +1,10 @@
> +-- Copyright 2009 Canonical Ltd. This software is licensed under the
^^^ 2010 ? :-)

> @@ -561,8 +590,17 @@
> uploaded file targeted to an architecture not present in the
> distroseries in context. So callsites needs to be aware.
> """
> + # If we are dealing with a DDEB, use the DEB's overrides.
> + # If there's no deb_file set, don't worry about it. Rejection is
> + # already guaranteed.
> + if (isinstance(uploaded_file, DdebBinaryUploadFile)
> + and uploaded_file.deb_file):
> + ancestry_name = uploaded_file.deb_file.package
> + else:
> + ancestry_name = uploaded_file.package
> +
> binary_name = getUtility(
> - IBinaryPackageNameSet).queryByName(uploaded_file.package)
> + IBinaryPackageNameSet).queryByName(ancestry_name)
Ugh, "deb_file" is a bit confusing here as I keep reading it as
"debug_file". Not really sure what to do about that though.

> === modified file 'lib/lp/soyuz/interfaces/binarypackagebuild.py'
> --- lib/lp/soyuz/interfaces/binarypackagebuild.py 2010-06-14 09:31:26 +0000
> +++ lib/lp/soyuz/interfaces/binarypackagebuild.py 2010-07-21 12:53:20 +0000
> @@ -152,6 +153,13 @@
> :return the corresponding `ILibraryFileAlias` if the file was found.
> """
>
> + def getBinaryPackageFileByName(filename):
> + """Return the corresponding `IBinaryPackageFile` in this context.
> +
> + :param filename: the filename to look up.
> + :return: the correpsonding `IBinaryPackageFile` if it was found.
> + """
> +
corresponding ?

> class IBinaryPackageBuildEdit(Interface):
> """A Build interface for items requiring launchpad.Edit."""
>
> === modified file 'lib/lp/soyuz/interfaces/binarypackagerelease.py'
> --- lib/lp/soyuz/interfaces/binarypackagerelease.py 2010-03-17 10:07:19 +0000
> +++ lib/lp/soyuz/interfaces/binarypackagerelease.py 2010-07-21 12:53:20 +0000
> @@ -18,7 +18,7 @@
> from lazr.enum import DBEnumeratedType, DBItem
> from lazr.restful.declarations import exported, export_as_webservice_entry
> from lazr.restful.fields import Reference, ReferenceChoice
> -from zope.schema import Bool, Date, Int, Text, TextLine, Datetime
> +from zope.schema import Bool, Date, Datetime, Int, Object, Text, TextLine
> from zope.interface import Interface, Attribute
>
> from canonical.launchpad import _
Yay for drive-by fixes.

> === modified file 'lib/lp/soyuz/model/binarypackagebuild.py'
> --- lib/lp/soyuz/model/binarypackagebuild.py 2010-07-18 00:56:16 +0000
> +++ lib/lp/soyuz/model/binarypackagebuild.py 2010-07-21 12:53:20 +0000
> @@ -450,7 +450,7 @@
> binpackageformat, component,section, priority, shlibdeps,
> depends, recommends, suggests, conflicts, replaces, provides,
> pre_depends, enhances, breaks, essential, installedsize,
> - architecturespecific):
> + architecturespecific, debug_package):
> """See...

Read more...

Jelmer Vernooij (jelmer) :
review: Approve (code)

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'database/schema/comments.sql'
2--- database/schema/comments.sql 2010-07-15 15:29:28 +0000
3+++ database/schema/comments.sql 2010-07-22 11:40:00 +0000
4@@ -1483,6 +1483,7 @@
5 COMMENT ON COLUMN BinaryPackageRelease.pre_depends IS 'The list of packages this binary requires to be installed beforehand in apt/dpkg format, as it is in control file "Pre-Depends:" field.';
6 COMMENT ON COLUMN BinaryPackageRelease.enhances IS 'The list of packages pointed as "enhanced" after the installation of this package, as it is in control file "Enhances:" field.';
7 COMMENT ON COLUMN BinaryPackageRelease.breaks IS 'The list of packages which will be broken by the installtion of this package, as it is in the control file "Breaks:" field.';
8+COMMENT ON COLUMN BinaryPackageRelease.debug_package IS 'The corresponding binary package release containing debug symbols for this binary, if any.';
9
10
11 -- BinaryPackageFile
12
13=== added file 'database/schema/patch-2207-73-0.sql'
14--- database/schema/patch-2207-73-0.sql 1970-01-01 00:00:00 +0000
15+++ database/schema/patch-2207-73-0.sql 2010-07-22 11:40:00 +0000
16@@ -0,0 +1,10 @@
17+-- Copyright 2010 Canonical Ltd. This software is licensed under the
18+-- GNU Affero General Public License version 3 (see the file LICENSE).
19+
20+ALTER TABLE BinaryPackageRelease
21+ ADD COLUMN debug_package integer REFERENCES BinaryPackageRelease;
22+
23+CREATE UNIQUE INDEX binarypackagerelease__debug_package__key
24+ ON BinaryPackageRelease(debug_package);
25+
26+INSERT INTO LaunchpadDatabaseRevision VALUES (2207, 73, 0);
27
28=== modified file 'lib/lp/archiveuploader/nascentupload.py'
29--- lib/lp/archiveuploader/nascentupload.py 2010-07-15 09:42:28 +0000
30+++ lib/lp/archiveuploader/nascentupload.py 2010-07-22 11:40:00 +0000
31@@ -27,7 +27,7 @@
32 from lp.archiveuploader.dscfile import DSCFile
33 from lp.archiveuploader.nascentuploadfile import (
34 UploadError, UploadWarning, CustomUploadFile, SourceUploadFile,
35- BaseBinaryUploadFile)
36+ BaseBinaryUploadFile, DdebBinaryUploadFile, DebBinaryUploadFile)
37 from lp.archiveuploader.utils import determine_source_file_type
38 from lp.registry.interfaces.pocket import PackagePublishingPocket
39 from lp.registry.interfaces.sourcepackage import SourcePackageFileType
40@@ -158,6 +158,35 @@
41 for uploaded_file in self.changes.files:
42 self.run_and_collect_errors(uploaded_file.verify)
43
44+ unmatched_ddebs = {}
45+ for uploaded_file in self.changes.files:
46+ if isinstance(uploaded_file, DdebBinaryUploadFile):
47+ ddeb_key = (uploaded_file.package, uploaded_file.version,
48+ uploaded_file.architecture)
49+ if ddeb_key in unmatched_ddebs:
50+ self.reject("Duplicated debug packages: %s %s (%s)" %
51+ ddeb_key)
52+ else:
53+ unmatched_ddebs[ddeb_key] = uploaded_file
54+
55+ for uploaded_file in self.changes.files:
56+ # We need exactly a DEB, not a DDEB.
57+ if (isinstance(uploaded_file, DebBinaryUploadFile) and
58+ not isinstance(uploaded_file, DdebBinaryUploadFile)):
59+ try:
60+ matching_ddeb = unmatched_ddebs.pop(
61+ (uploaded_file.package + '-dbgsym',
62+ uploaded_file.version,
63+ uploaded_file.architecture))
64+ except KeyError:
65+ continue
66+ uploaded_file.ddeb_file = matching_ddeb
67+ matching_ddeb.deb_file = uploaded_file
68+
69+ if len(unmatched_ddebs) > 0:
70+ self.reject("Orphaned debug packages: %s" % ', '.join('%s %s (%s)' % d
71+ for d in unmatched_ddebs))
72+
73 if (len(self.changes.files) == 1 and
74 isinstance(self.changes.files[0], CustomUploadFile)):
75 self.logger.debug("Single Custom Upload detected.")
76@@ -561,8 +590,17 @@
77 uploaded file targeted to an architecture not present in the
78 distroseries in context. So callsites needs to be aware.
79 """
80+ # If we are dealing with a DDEB, use the DEB's overrides.
81+ # If there's no deb_file set, don't worry about it. Rejection is
82+ # already guaranteed.
83+ if (isinstance(uploaded_file, DdebBinaryUploadFile)
84+ and uploaded_file.deb_file):
85+ ancestry_name = uploaded_file.deb_file.package
86+ else:
87+ ancestry_name = uploaded_file.package
88+
89 binary_name = getUtility(
90- IBinaryPackageNameSet).queryByName(uploaded_file.package)
91+ IBinaryPackageNameSet).queryByName(ancestry_name)
92
93 if binary_name is None:
94 return None
95@@ -939,7 +977,12 @@
96 # Container for the build that will be processed.
97 processed_builds = []
98
99- for binary_package_file in self.changes.binary_package_files:
100+ # Create the BPFs referencing DDEBs last. This lets
101+ # storeInDatabase find and link DDEBs when creating DEBs.
102+ bpfs_to_create = sorted(
103+ self.changes.binary_package_files,
104+ key=lambda file: file.ddeb_file is not None)
105+ for binary_package_file in bpfs_to_create:
106 if self.sourceful:
107 # The reason we need to do this verification
108 # so late in the game is that in the
109
110=== modified file 'lib/lp/archiveuploader/nascentuploadfile.py'
111--- lib/lp/archiveuploader/nascentuploadfile.py 2010-06-25 16:26:20 +0000
112+++ lib/lp/archiveuploader/nascentuploadfile.py 2010-07-22 11:40:00 +0000
113@@ -372,6 +372,7 @@
114 """Base methods for binary upload modeling."""
115
116 format = None
117+ ddeb_file = None
118
119 # Capitalised because we extract these directly from the control file.
120 mandatory_fields = set(["Package", "Architecture", "Version"])
121@@ -864,6 +865,12 @@
122 binary_name = getUtility(
123 IBinaryPackageNameSet).getOrCreateByName(self.package)
124
125+ if self.ddeb_file:
126+ debug_package = build.getBinaryPackageFileByName(
127+ self.ddeb_file.filename).binarypackagerelease
128+ else:
129+ debug_package = None
130+
131 binary = build.createBinaryPackageRelease(
132 binarypackagename=binary_name,
133 version=self.control_version,
134@@ -885,7 +892,8 @@
135 breaks=encoded.get('Breaks', ''),
136 essential=is_essential,
137 installedsize=installedsize,
138- architecturespecific=architecturespecific)
139+ architecturespecific=architecturespecific,
140+ debug_package=debug_package)
141
142 library_file = self.librarian.create(self.filename,
143 self.size, open(self.filepath, "rb"), self.content_type,
144@@ -932,22 +940,6 @@
145
146
147 class DdebBinaryUploadFile(DebBinaryUploadFile):
148- """Represents an uploaded binary package file in ddeb format.
149-
150- DDEBs are never considered 'NEW', they don't require review since
151- they are automatically generated.
152- """
153+ """Represents an uploaded binary package file in ddeb format."""
154 format = BinaryPackageFormat.DDEB
155-
156- # Override the 'new' flag in a way any values set are ignored and
157- # it always return False.
158- def _get_new(self):
159- """DDEBs are never considered NEW."""
160- return False
161-
162- def _set_new(self, value):
163- """DDEBs cannot be made NEW."""
164- pass
165-
166- new = property(
167- _get_new, _set_new, doc="DDEBs are never flagged as NEW.")
168+ deb_file = None
169
170=== added directory 'lib/lp/archiveuploader/tests/data/suite/debug_1.0-1_broken'
171=== added file 'lib/lp/archiveuploader/tests/data/suite/debug_1.0-1_broken/debug-bin-dbgsym_1.0-1_i386.ddeb'
172Binary files lib/lp/archiveuploader/tests/data/suite/debug_1.0-1_broken/debug-bin-dbgsym_1.0-1_i386.ddeb 1970-01-01 00:00:00 +0000 and lib/lp/archiveuploader/tests/data/suite/debug_1.0-1_broken/debug-bin-dbgsym_1.0-1_i386.ddeb 2010-07-22 11:40:00 +0000 differ
173=== added file 'lib/lp/archiveuploader/tests/data/suite/debug_1.0-1_broken/debug-bin_1.0-1_i386.deb'
174Binary files lib/lp/archiveuploader/tests/data/suite/debug_1.0-1_broken/debug-bin_1.0-1_i386.deb 1970-01-01 00:00:00 +0000 and lib/lp/archiveuploader/tests/data/suite/debug_1.0-1_broken/debug-bin_1.0-1_i386.deb 2010-07-22 11:40:00 +0000 differ
175=== added file 'lib/lp/archiveuploader/tests/data/suite/debug_1.0-1_broken/debug_1.0-1.dsc'
176--- lib/lp/archiveuploader/tests/data/suite/debug_1.0-1_broken/debug_1.0-1.dsc 1970-01-01 00:00:00 +0000
177+++ lib/lp/archiveuploader/tests/data/suite/debug_1.0-1_broken/debug_1.0-1.dsc 2010-07-22 11:40:00 +0000
178@@ -0,0 +1,13 @@
179+Format: 1.0
180+Source: debug
181+Binary: debug-bin, debug-bin-dbgsym
182+Architecture: any
183+Version: 1.0-1
184+Maintainer: Launchpad team <launchpad@lists.canonical.com>
185+Standards-Version: 3.6.2
186+Checksums-Sha1:
187+ 6851f625a32f6c83a1af4c972b5bead14fbee06c 740 debug_1.0-1.tar.gz
188+Checksums-Sha256:
189+ 1a1ed198644169f16eeee93c843ab8b7d9eef25cd91b83f55f497dbe744d3c95 740 debug_1.0-1.tar.gz
190+Files:
191+ 8225b515f30dd8c0252847f17f4b7069 740 debug_1.0-1.tar.gz
192
193=== added file 'lib/lp/archiveuploader/tests/data/suite/debug_1.0-1_broken/debug_1.0-1.tar.gz'
194Binary files lib/lp/archiveuploader/tests/data/suite/debug_1.0-1_broken/debug_1.0-1.tar.gz 1970-01-01 00:00:00 +0000 and lib/lp/archiveuploader/tests/data/suite/debug_1.0-1_broken/debug_1.0-1.tar.gz 2010-07-22 11:40:00 +0000 differ
195=== added file 'lib/lp/archiveuploader/tests/data/suite/debug_1.0-1_broken/debug_1.0-1_i386.changes'
196--- lib/lp/archiveuploader/tests/data/suite/debug_1.0-1_broken/debug_1.0-1_i386.changes 1970-01-01 00:00:00 +0000
197+++ lib/lp/archiveuploader/tests/data/suite/debug_1.0-1_broken/debug_1.0-1_i386.changes 2010-07-22 11:40:00 +0000
198@@ -0,0 +1,31 @@
199+Format: 1.8
200+Date: Thu, 15 Apr 2009 15:14:09 +0100
201+Source: debug
202+Binary: debug-bin
203+Architecture: i386
204+Version: 1.0-1
205+Distribution: hoary
206+Urgency: low
207+Maintainer: Launchpad team <launchpad@lists.canonical.com>
208+Changed-By: Celso Providelo <cprov@canonical.com>
209+Description:
210+ debug-bin - Stuff for testing
211+Changes:
212+ debug (1.0-1) hoary; urgency=low
213+ .
214+ * Testing debug binaries
215+Checksums-Sha1:
216+ 91556113ad38eb35d2fe03d27ae646e0ed487a3d 654 debug-bin_1.0-1_i386.deb
217+ d120be63a4535e31ec08657cc4d3018cc8b62d54 668 debug-bin-dbgsym_1.0-1_i386.ddeb
218+ d120be63a4535e31ec08657cc4d3018cc8b62d54 668 debug-bin-dbgsym_1.0-1_i386.ddeb
219+ 473bafe5b391c52f9a527cc550bb1fd6d5256031 692 not-debug-bin-dbgsym_1.0-1_i386.ddeb
220+Checksums-Sha256:
221+ 5ca0ce3d3bfde3cc698d9ad4b027663abecb958bea641d29ac3bf8333fd3ebad 654 debug-bin_1.0-1_i386.deb
222+ 9c0013badb1bb78fa1ccbf1508398cbf51904f7242d8931f4994c1aa18519b63 668 debug-bin-dbgsym_1.0-1_i386.ddeb
223+ 9c0013badb1bb78fa1ccbf1508398cbf51904f7242d8931f4994c1aa18519b63 668 debug-bin-dbgsym_1.0-1_i386.ddeb
224+ c474f06a1481111b4af58c583df0a2c4504a18c0d2d960db3a0416aafd1bfd0e 692 not-debug-bin-dbgsym_1.0-1_i386.ddeb
225+Files:
226+ 48b4eed5980c754cc74a06ffc4372f64 654 devel optional debug-bin_1.0-1_i386.deb
227+ ac5c44025ffc026485b22722e0f13804 668 devel optional debug-bin-dbgsym_1.0-1_i386.ddeb
228+ ac5c44025ffc026485b22722e0f13804 668 devel optional debug-bin-dbgsym_1.0-1_i386.ddeb
229+ e6cb038fdfed5f1150814f719a618698 692 devel optional not-debug-bin-dbgsym_1.0-1_i386.ddeb
230
231=== added file 'lib/lp/archiveuploader/tests/data/suite/debug_1.0-1_broken/debug_1.0-1_source.changes'
232--- lib/lp/archiveuploader/tests/data/suite/debug_1.0-1_broken/debug_1.0-1_source.changes 1970-01-01 00:00:00 +0000
233+++ lib/lp/archiveuploader/tests/data/suite/debug_1.0-1_broken/debug_1.0-1_source.changes 2010-07-22 11:40:00 +0000
234@@ -0,0 +1,26 @@
235+Format: 1.8
236+Date: Thu, 15 Apr 2009 15:14:09 +0100
237+Source: debug
238+Binary: debug-bin debug-bin-dbgsym
239+Architecture: source
240+Version: 1.0-1
241+Distribution: hoary
242+Urgency: low
243+Maintainer: Launchpad team <launchpad@lists.canonical.com>
244+Changed-By: Celso Providelo <cprov@canonical.com>
245+Description:
246+ debug-bin - Stuff for testing
247+ debug-bin-dbgsym - Stuff for testing DDEBs
248+Changes:
249+ debug (1.0-1) hoary; urgency=low
250+ .
251+ * Testing debug binaries
252+Checksums-Sha1:
253+ f30c2462e897e07534a5c9960a8f653d5a51b066 434 debug_1.0-1.dsc
254+ 6851f625a32f6c83a1af4c972b5bead14fbee06c 740 debug_1.0-1.tar.gz
255+Checksums-Sha256:
256+ ab5e1eea2551828761d8734224d1a024be401ac05e83010494a0d9c69b8982c3 434 debug_1.0-1.dsc
257+ 1a1ed198644169f16eeee93c843ab8b7d9eef25cd91b83f55f497dbe744d3c95 740 debug_1.0-1.tar.gz
258+Files:
259+ 81d19cae151472dfd2f797a69a2c1f54 434 devel optional debug_1.0-1.dsc
260+ 8225b515f30dd8c0252847f17f4b7069 740 devel optional debug_1.0-1.tar.gz
261
262=== added file 'lib/lp/archiveuploader/tests/data/suite/debug_1.0-1_broken/not-debug-bin-dbgsym_1.0-1_i386.ddeb'
263Binary files lib/lp/archiveuploader/tests/data/suite/debug_1.0-1_broken/not-debug-bin-dbgsym_1.0-1_i386.ddeb 1970-01-01 00:00:00 +0000 and lib/lp/archiveuploader/tests/data/suite/debug_1.0-1_broken/not-debug-bin-dbgsym_1.0-1_i386.ddeb 2010-07-22 11:40:00 +0000 differ
264=== modified file 'lib/lp/archiveuploader/tests/nascentupload-ddebs.txt'
265--- lib/lp/archiveuploader/tests/nascentupload-ddebs.txt 2009-10-11 05:25:10 +0000
266+++ lib/lp/archiveuploader/tests/nascentupload-ddebs.txt 2010-07-22 11:40:00 +0000
267@@ -1,4 +1,5 @@
268-== Uploading DDEBs ==
269+Uploading DDEBs
270+===============
271
272 DDEBs ('.ddeb' extension) are essentially a Debian binary packages,
273 they only use a different extension as a convenience to identify their
274@@ -9,10 +10,8 @@
275 for obtaining extra information about crashes in the corresponding
276 feature.
277
278-First we will create a source and a corresponding binary publication
279-in the database, 'debug' and 'debug-bin' version '0.9', which will be
280-an ancestor for the subsequent upload containing a 'debug-bin-dbgsym'
281-DDEB. This is done using `SoyuzTestPublisher`.
282+ >>> from canonical.config import config
283+ >>> from canonical.testing import LaunchpadZopelessLayer
284
285 >>> from lp.soyuz.tests.test_publishing import (
286 ... SoyuzTestPublisher)
287@@ -26,31 +25,22 @@
288
289 >>> unused = test_publisher.setUpDefaultDistroSeries(hoary)
290
291- >>> source_ancestry = test_publisher.getPubSource(
292- ... sourcename='debug', version='0.99')
293-
294-We need a slightly different database setup for creating binaries.
295-
296- >>> from canonical.config import config
297- >>> from canonical.testing import LaunchpadZopelessLayer
298-
299- >>> LaunchpadZopelessLayer.txn.commit()
300- >>> LaunchpadZopelessLayer.switchDbUser('launchpad')
301-
302- >>> binary_ancestry = test_publisher.getPubBinaries(
303- ... binaryname='debug-bin', pub_source=source_ancestry)
304-
305- >>> LaunchpadZopelessLayer.txn.commit()
306- >>> LaunchpadZopelessLayer.switchDbUser('uploader')
307-
308-With that done, we now upload a new source, 'debug - 1.0-1', which is
309-immediately accepted and stands pending publication.
310+We first upload a new source, 'debug - 1.0-1'. Because this is NEW,
311+it is in the queue awaiting overrides and acceptance.
312
313 >>> src = getUploadForSource(
314 ... 'suite/debug_1.0-1/debug_1.0-1_source.changes')
315 >>> src.process()
316 >>> result = src.do_accept()
317 >>> print src.queue_root.status.name
318+ NEW
319+ >>> LaunchpadZopelessLayer.txn.commit()
320+
321+We don't really care where the source ends up, so we just accept the
322+default overrides. It is now pending publication.
323+
324+ >>> src.queue_root.acceptFromQueue("announce@example.com")
325+ >>> print src.queue_root.status.name
326 DONE
327
328 >>> [src_pub] = src.queue_root.archive.getPublishedSources(
329@@ -66,17 +56,46 @@
330 >>> bin = getUploadForBinary(
331 ... 'suite/debug_1.0-1/debug_1.0-1_i386.changes')
332
333-DDEBs undergo exactly the same checks as ordinary DEBs. These are
334-never considered NEW, so the binary upload has a 'never-seen-before'
335-DDEB even though it isn't parked in queue for review.
336+Because the DEB is new, the binary upload is held in NEW.
337
338 >>> bin.process()
339 >>> result = bin.do_accept()
340 >>> print bin.queue_root.status.name
341+ NEW
342+
343+This upload has one build with two binaries: a DEB and its corresponding
344+DDEB.
345+
346+ >>> build = bin.queue_root.builds[0].build
347+ >>> build.binarypackages.count()
348+ 2
349+ >>> print build.binarypackages[0].binpackageformat.name
350+ DEB
351+ >>> print build.binarypackages[1].binpackageformat.name
352+ DDEB
353+ >>> build.binarypackages[0].debug_package == build.binarypackages[1]
354+ True
355+ >>> build.binarypackages[1].debug_package is None
356+ True
357+
358+We override the binary to main/devel, and accept it into the archive.
359+
360+ >>> from lp.soyuz.interfaces.component import IComponentSet
361+ >>> from lp.soyuz.interfaces.section import ISectionSet
362+ >>> main = getUtility(IComponentSet)['main']
363+ >>> universe = getUtility(IComponentSet)['universe']
364+ >>> devel = getUtility(ISectionSet)['devel']
365+
366+ >>> LaunchpadZopelessLayer.txn.commit()
367+ >>> LaunchpadZopelessLayer.switchDbUser('launchpad')
368+
369+ >>> bin.queue_root.overrideBinaries(main, devel, None, [main, universe])
370+ True
371+ >>> bin.queue_root.acceptFromQueue('announce@example.com')
372+
373+ >>> print bin.queue_root.status.name
374 ACCEPTED
375
376-Again, binary publications need an extra database setup.
377-
378 >>> LaunchpadZopelessLayer.txn.commit()
379 >>> LaunchpadZopelessLayer.switchDbUser(config.uploadqueue.dbuser)
380
381@@ -88,11 +107,12 @@
382 Now, both, binary and debug-symbol packages are pending publication.
383
384 >>> for bin_pub in bin_pubs:
385- ... print '%s %s %s' % (
386+ ... print '%s %s %s %s %s' % (
387 ... bin_pub.displayname, bin_pub.status.name,
388+ ... bin_pub.component.name, bin_pub.section.name,
389 ... bin_pub.archive.purpose.name)
390- debug-bin 1.0-1 in hoary i386 PENDING PRIMARY
391- debug-bin-dbgsym 1.0-1 in hoary i386 PENDING DEBUG
392+ debug-bin 1.0-1 in hoary i386 PENDING main devel PRIMARY
393+ debug-bin-dbgsym 1.0-1 in hoary i386 PENDING main devel DEBUG
394
395 DEBs and DDEBs are uploaded to separate archives, because the size
396 impact of uploading them to a single archive on mirrors would be
397@@ -118,7 +138,8 @@
398 debug-bin-dbgsym_1.0-1_i386.ddeb DDEB
399
400
401-== DDEBs in other archives ==
402+DDEBs in other archives
403+-----------------------
404
405 DDEBs can be produced in other contexts than the primary archive. For
406 instance, a COPY (test-rebuild) archive. In this case, they will be
407@@ -134,14 +155,14 @@
408 ... owner=ubuntu.owner, purpose=ArchivePurpose.COPY,
409 ... name='test-rebuild')
410
411- >>> copied_source = source_ancestry.copyTo(
412- ... hoary, source_ancestry.pocket, copy)
413-
414 >>> LaunchpadZopelessLayer.txn.commit()
415 >>> LaunchpadZopelessLayer.switchDbUser('launchpad')
416
417+ >>> initial_source = test_publisher.getPubSource(
418+ ... sourcename='debug', version='0.9', archive=copy)
419+
420 >>> binary_ancestry2 = test_publisher.getPubBinaries(
421- ... binaryname='debug-bin', pub_source=copied_source, archive=copy)
422+ ... binaryname='debug-bin', pub_source=initial_source, archive=copy)
423
424 >>> src_pub = test_publisher.getPubSource(
425 ... sourcename='debug', version='1.0-1', archive=copy)
426@@ -172,9 +193,27 @@
427 >>> LaunchpadZopelessLayer.switchDbUser('uploader')
428
429 >>> for bin_pub in bin_pubs:
430- ... print '%s %s %s' % (
431+ ... print '%s %s %s %s %s' % (
432 ... bin_pub.displayname, bin_pub.status.name,
433+ ... bin_pub.component.name, bin_pub.section.name,
434 ... bin_pub.archive.purpose.name)
435- debug-bin 1.0-1 in hoary i386 PENDING COPY
436- debug-bin-dbgsym 1.0-1 in hoary i386 PENDING COPY
437-
438+ debug-bin 1.0-1 in hoary i386 PENDING main devel COPY
439+ debug-bin-dbgsym 1.0-1 in hoary i386 PENDING main devel COPY
440+
441+
442+Mismatched DDEBs
443+----------------
444+
445+Each uploaded DDEB must be associated with a normal DEB. Any duplicated
446+DDEBs or DDEBs without matching DEBs will cause the upload to be
447+rejected.
448+
449+ >>> bin = getUploadForBinary(
450+ ... 'suite/debug_1.0-1_broken/debug_1.0-1_i386.changes')
451+ >>> bin.process()
452+ >>> bin.is_rejected
453+ True
454+ >>> print bin.rejection_message
455+ Duplicated debug packages: debug-bin-dbgsym 1.0-1 (i386)
456+ Orphaned debug packages: not-debug-bin-dbgsym 1.0-1 (i386)
457+
458
459=== modified file 'lib/lp/soyuz/doc/archive-files.txt'
460--- lib/lp/soyuz/doc/archive-files.txt 2010-05-11 07:09:18 +0000
461+++ lib/lp/soyuz/doc/archive-files.txt 2010-07-22 11:40:00 +0000
462@@ -209,3 +209,11 @@
463
464 >>> deb == build.getFileByName('test_1.0_all.deb')
465 True
466+
467+We can also retrieve the corresponding BinaryPackageFile:
468+
469+ >>> bpf = build.getBinaryPackageFileByName('test_1.0_all.deb')
470+ >>> bpf
471+ <BinaryPackageFile ...>
472+ >>> bpf.libraryfile == deb
473+ True
474
475=== modified file 'lib/lp/soyuz/doc/binarypackagebuild.txt'
476--- lib/lp/soyuz/doc/binarypackagebuild.txt 2010-07-17 12:56:41 +0000
477+++ lib/lp/soyuz/doc/binarypackagebuild.txt 2010-07-22 11:40:00 +0000
478@@ -323,6 +323,7 @@
479 >>> pre_depends = None
480 >>> enhances = None
481 >>> breaks = None
482+ >>> debug_package = None
483
484 Invoke createBinaryPackageRelease with all required arguments.
485
486@@ -334,7 +335,7 @@
487 ... binpackageformat, component, section, priority, shlibdeps,
488 ... depends, recommends, suggests, conflicts, replaces, provides,
489 ... pre_depends, enhances, breaks, essential, installedsize,
490- ... architecturespecific)
491+ ... architecturespecific, debug_package)
492
493 >>> from canonical.launchpad.interfaces import IBinaryPackageRelease
494 >>> from canonical.launchpad.webapp.testing import verifyObject
495@@ -367,7 +368,7 @@
496 ... binpackageformat, component, section, priority, shlibdeps,
497 ... depends, recommends, suggests, conflicts, replaces, provides,
498 ... pre_depends, enhances, breaks, essential, installedsize,
499- ... architecturespecific)
500+ ... architecturespecific, debug_package)
501
502
503 Check if the property is still working:
504
505=== modified file 'lib/lp/soyuz/doc/distroarchseriesbinarypackage.txt'
506--- lib/lp/soyuz/doc/distroarchseriesbinarypackage.txt 2010-04-12 08:29:02 +0000
507+++ lib/lp/soyuz/doc/distroarchseriesbinarypackage.txt 2010-07-22 11:40:00 +0000
508@@ -66,7 +66,8 @@
509 ... breaks=None,
510 ... essential=False,
511 ... installedsize=0,
512- ... architecturespecific=False)
513+ ... architecturespecific=False,
514+ ... debug_package=None)
515
516 >>> pe = BinaryPackagePublishingHistory(
517 ... binarypackagerelease=bpr.id,
518@@ -107,7 +108,8 @@
519 ... breaks=None,
520 ... essential=False,
521 ... installedsize=0,
522- ... architecturespecific=False)
523+ ... architecturespecific=False,
524+ ... debug_package=None)
525
526 >>> pe = BinaryPackagePublishingHistory(
527 ... binarypackagerelease=bpr.id,
528
529=== modified file 'lib/lp/soyuz/interfaces/binarypackagebuild.py'
530--- lib/lp/soyuz/interfaces/binarypackagebuild.py 2010-06-14 09:31:26 +0000
531+++ lib/lp/soyuz/interfaces/binarypackagebuild.py 2010-07-22 11:40:00 +0000
532@@ -127,7 +127,8 @@
533 binarypackagename, version, summary, description, binpackageformat,
534 component, section, priority, shlibdeps, depends, recommends,
535 suggests, conflicts, replaces, provides, pre_depends, enhances,
536- breaks, essential, installedsize, architecturespecific):
537+ breaks, essential, installedsize, architecturespecific,
538+ debug_package):
539 """Create and return a `BinaryPackageRelease`.
540
541 The binarypackagerelease will be attached to this specific build.
542@@ -152,6 +153,13 @@
543 :return the corresponding `ILibraryFileAlias` if the file was found.
544 """
545
546+ def getBinaryPackageFileByName(filename):
547+ """Return the corresponding `IBinaryPackageFile` in this context.
548+
549+ :param filename: the filename to look up.
550+ :return: the corresponding `IBinaryPackageFile` if it was found.
551+ """
552+
553
554 class IBinaryPackageBuildEdit(Interface):
555 """A Build interface for items requiring launchpad.Edit."""
556
557=== modified file 'lib/lp/soyuz/interfaces/binarypackagerelease.py'
558--- lib/lp/soyuz/interfaces/binarypackagerelease.py 2010-03-17 10:07:19 +0000
559+++ lib/lp/soyuz/interfaces/binarypackagerelease.py 2010-07-22 11:40:00 +0000
560@@ -18,7 +18,7 @@
561 from lazr.enum import DBEnumeratedType, DBItem
562 from lazr.restful.declarations import exported, export_as_webservice_entry
563 from lazr.restful.fields import Reference, ReferenceChoice
564-from zope.schema import Bool, Date, Int, Text, TextLine, Datetime
565+from zope.schema import Bool, Date, Datetime, Int, Object, Text, TextLine
566 from zope.interface import Interface, Attribute
567
568 from canonical.launchpad import _
569@@ -52,6 +52,10 @@
570 installedsize = Int(required=False)
571 architecturespecific = Bool(required=True)
572 datecreated = Datetime(required=True, readonly=True)
573+ debug_package = Object(
574+ title=_("Debug package"), schema=Interface, required=False,
575+ description=_("The corresponding package containing debug symbols "
576+ "for this binary."))
577
578 files = Attribute("Related list of IBinaryPackageFile entries")
579
580
581=== modified file 'lib/lp/soyuz/model/binarypackagebuild.py'
582--- lib/lp/soyuz/model/binarypackagebuild.py 2010-07-20 19:12:33 +0000
583+++ lib/lp/soyuz/model/binarypackagebuild.py 2010-07-22 11:40:00 +0000
584@@ -451,7 +451,7 @@
585 binpackageformat, component,section, priority, shlibdeps,
586 depends, recommends, suggests, conflicts, replaces, provides,
587 pre_depends, enhances, breaks, essential, installedsize,
588- architecturespecific):
589+ architecturespecific, debug_package):
590 """See IBuild."""
591 return BinaryPackageRelease(
592 build=self, binarypackagename=binarypackagename, version=version,
593@@ -462,7 +462,8 @@
594 suggests=suggests, conflicts=conflicts, replaces=replaces,
595 provides=provides, pre_depends=pre_depends, enhances=enhances,
596 breaks=breaks, essential=essential, installedsize=installedsize,
597- architecturespecific=architecturespecific)
598+ architecturespecific=architecturespecific,
599+ debug_package=debug_package)
600
601 def estimateDuration(self):
602 """See `IBuildBase`."""
603@@ -678,14 +679,11 @@
604
605 def _getDebByFileName(self, filename):
606 """Helper function to get a .deb LFA in the context of this build."""
607- store = Store.of(self)
608- return store.find(
609- LibraryFileAlias,
610- BinaryPackageRelease.build == self.id,
611- BinaryPackageFile.binarypackagerelease == BinaryPackageRelease.id,
612- LibraryFileAlias.id == BinaryPackageFile.libraryfileID,
613- LibraryFileAlias.filename == filename
614- ).one()
615+ bpf = self.getBinaryPackageFileByName(filename)
616+ if bpf is not None:
617+ return bpf.libraryfile
618+ else:
619+ return None
620
621 def getFileByName(self, filename):
622 """See `IBuild`."""
623@@ -705,6 +703,15 @@
624
625 raise NotFoundError(filename)
626
627+ def getBinaryPackageFileByName(self, filename):
628+ """See `IBuild`."""
629+ return Store.of(self).find(
630+ BinaryPackageFile,
631+ BinaryPackageRelease.build == self.id,
632+ BinaryPackageFile.binarypackagerelease == BinaryPackageRelease.id,
633+ LibraryFileAlias.id == BinaryPackageFile.libraryfileID,
634+ LibraryFileAlias.filename == filename).one()
635+
636 def getSpecificJob(self):
637 """See `IBuildFarmJob`."""
638 # If we are asked to adapt an object that is already a binary
639
640=== modified file 'lib/lp/soyuz/model/binarypackagerelease.py'
641--- lib/lp/soyuz/model/binarypackagerelease.py 2010-05-11 09:35:33 +0000
642+++ lib/lp/soyuz/model/binarypackagerelease.py 2010-07-22 11:40:00 +0000
643@@ -62,6 +62,8 @@
644 architecturespecific = BoolCol(dbName='architecturespecific',
645 notNull=True)
646 datecreated = UtcDateTimeCol(notNull=True, default=UTC_NOW)
647+ debug_package = ForeignKey(dbName='debug_package',
648+ foreignKey='BinaryPackageRelease')
649
650 files = SQLMultipleJoin('BinaryPackageFile',
651 joinColumn='binarypackagerelease', orderBy="libraryfile")
652
653=== modified file 'lib/lp/soyuz/tests/test_publishing.py'
654--- lib/lp/soyuz/tests/test_publishing.py 2010-07-21 14:14:54 +0000
655+++ lib/lp/soyuz/tests/test_publishing.py 2010-07-22 11:40:00 +0000
656@@ -321,7 +321,7 @@
657 summary="summary", description="description", shlibdep=None,
658 depends=None, recommends=None, suggests=None, conflicts=None,
659 replaces=None, provides=None, pre_depends=None, enhances=None,
660- breaks=None, format=BinaryPackageFormat.DEB):
661+ breaks=None, format=BinaryPackageFormat.DEB, debug_package=None):
662 """Return the corresponding `BinaryPackageRelease`."""
663 sourcepackagerelease = build.source_package_release
664 distroarchseries = build.distro_arch_series
665@@ -352,7 +352,8 @@
666 installedsize=100,
667 architecturespecific=architecturespecific,
668 binpackageformat=format,
669- priority=PackagePublishingPriority.STANDARD)
670+ priority=PackagePublishingPriority.STANDARD,
671+ debug_package=debug_package)
672
673 # Create the corresponding binary file.
674 if architecturespecific:

Subscribers

People subscribed via source and target branches

to status/vote changes: