Merge lp:~salgado/launchpad/vostok-upload-policy into lp:launchpad

Proposed by Guilherme Salgado
Status: Merged
Approved by: Michael Nelson
Approved revision: no longer in the source branch.
Merged at revision: 11450
Proposed branch: lp:~salgado/launchpad/vostok-upload-policy
Merge into: lp:launchpad
Prerequisite: lp:~salgado/launchpad/remove-security-upload-policy
Diff against target: 719 lines (+340/-84)
13 files modified
lib/lp/archiveuploader/nascentupload.py (+6/-31)
lib/lp/archiveuploader/tests/__init__.py (+4/-0)
lib/lp/archiveuploader/tests/nascentupload-announcements.txt (+2/-1)
lib/lp/archiveuploader/tests/nascentupload.txt (+2/-1)
lib/lp/archiveuploader/tests/nascentuploadfile.txt (+3/-2)
lib/lp/archiveuploader/tests/test_buildduploads.py (+130/-4)
lib/lp/archiveuploader/tests/test_nascentupload_documentation.py (+2/-1)
lib/lp/archiveuploader/tests/test_ppauploadprocessor.py (+3/-6)
lib/lp/archiveuploader/tests/test_uploadpolicy.py (+127/-0)
lib/lp/archiveuploader/uploadpolicy.py (+52/-14)
lib/lp/code/model/sourcepackagerecipebuild.py (+2/-5)
lib/lp/soyuz/browser/tests/distroseriesqueue-views.txt (+2/-1)
lib/lp/soyuz/doc/soyuz-set-of-uploads.txt (+5/-18)
To merge this branch: bzr merge lp:~salgado/launchpad/vostok-upload-policy
Reviewer Review Type Date Requested Status
Jeroen T. Vermeulen (community) Approve
Review via email: mp+33584@code.launchpad.net

Description of the change

This branch moves the code related to validating the upload type from nascentupload to the upload policy, as a separate method. This makes it easy to unit test that code as well as adding more flexibility for subclasses that might want to validate differently.

It also replaces the three can_upload_* variables with a single enum variable.

To post a comment you must log in.
Revision history for this message
Jeroen T. Vermeulen (jtv) wrote :

Beautiful. A few notes, of course:

Lines 199 and 319 of the diff have multiple imports that don't conform to the new import formatting rules:

from foobar import (
    Bar,
    Foo,
    )

In line 379 of the diff, the "else" and the assertion throw the symmetry of the code slightly out of whack. Just an idea, but maybe you can move it out into a separate "else" clause?

    elif upload.binaryful:
        # [...]
    else:
        raise AssertionError("...")

The list of hand-wrapped text lines at line 382 of the diff puzzles me. Why not a single dedent("""triple-quoted multi-line string""")?

Finally, I think I see either an indentation problem or something involving tabs around line 390. Did you run "make lint" over the current form of this branch?

But that's all small fry. The branch is definitely an improvement.

review: Approve
Revision history for this message
Guilherme Salgado (salgado) wrote :

On Wed, 2010-08-25 at 05:27 +0000, Jeroen T. Vermeulen wrote:
> Review: Approve
> Beautiful. A few notes, of course:
>
> Lines 199 and 319 of the diff have multiple imports that don't conform
> to the new import formatting rules:
>
> from foobar import (
> Bar,
> Foo,
> )

Oops, fixed.

>
> In line 379 of the diff, the "else" and the assertion throw the
> symmetry of the code slightly out of whack. Just an idea, but maybe
> you can move it out into a separate "else" clause?
>
> elif upload.binaryful:
> # [...]
> else:
> raise AssertionError("...")

That wfm.

>
> The list of hand-wrapped text lines at line 382 of the diff puzzles
> me. Why not a single dedent("""triple-quoted multi-line string""")?

That's code I just moved around, but I've changed it as you suggest.

>
> Finally, I think I see either an indentation problem or something

Broken indentation indeed; didn't notice when moving the code around.

> involving tabs around line 390. Did you run "make lint" over the
> current form of this branch?

I never did because 'bzr send' would do that for me, but now that I'm
using bzr lp-submit...

>
> But that's all small fry. The branch is definitely an improvement.

Thanks for the review!

If I can make a suggestion as well, it'd be much better if you could
include chunks of the diff in your review[1], for context, instead of
using the line numbers of the diff. That way we don't have to go back
and forth between your comments and the diff (in my case between my mail
client and the merge-proposal's page), but more importantly because the
diff is a moving target, so there's a big chance that the line numbers
have changed when I get to reply to your review. :)

[1] I do that by replying to the m-p email on my mail client, so the
diff is quoted and I just insert my comments at the appropriate places.

Revision history for this message
Michael Nelson (michael.nelson) wrote :

r=me for -r11430..11432.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'lib/lp/archiveuploader/nascentupload.py'
2--- lib/lp/archiveuploader/nascentupload.py 2010-08-24 08:43:51 +0000
3+++ lib/lp/archiveuploader/nascentupload.py 2010-08-25 17:14:48 +0000
4@@ -146,10 +146,11 @@
5 UploadError will be raised and sent up to the caller. If this happens
6 the caller should call the reject method and process a rejection.
7 """
8+ policy = self.policy
9 self.logger.debug("Beginning processing.")
10
11 try:
12- self.policy.setDistroSeriesAndPocket(self.changes.suite_name)
13+ policy.setDistroSeriesAndPocket(self.changes.suite_name)
14 except NotFoundError:
15 self.reject(
16 "Unable to find distroseries: %s" % self.changes.suite_name)
17@@ -185,36 +186,12 @@
18 isinstance(self.changes.files[0], CustomUploadFile)):
19 self.logger.debug("Single Custom Upload detected.")
20 else:
21- if self.sourceful and not self.policy.can_upload_source:
22- self.reject("Upload is sourceful, but policy refuses "
23- "sourceful uploads.")
24-
25- elif self.binaryful and not self.policy.can_upload_binaries:
26- messages = [
27- "Upload rejected because it contains binary packages.",
28- "Ensure you are using `debuild -S`, or an equivalent",
29- "command, to generate only the source package before",
30- "re-uploading.",
31- ]
32- if self.is_ppa:
33- messages.append(
34- "See https://help.launchpad.net/Packaging/PPA for more "
35- "information.")
36- self.reject(" ".join(messages))
37-
38- elif (self.sourceful and self.binaryful and
39- not self.policy.can_upload_mixed):
40- self.reject("Upload is source/binary but policy refuses "
41- "mixed uploads.")
42-
43- elif self.sourceful and not self.changes.dsc:
44+ policy.validateUploadType(self)
45+
46+ if self.sourceful and not self.changes.dsc:
47 self.reject(
48 "Unable to find the DSC file in the source upload.")
49
50- else:
51- # Upload content are consistent with the current policy.
52- pass
53-
54 # Apply the overrides from the database. This needs to be done
55 # before doing component verifications because the component
56 # actually comes from overrides for packages that are not NEW.
57@@ -227,7 +204,7 @@
58 self.verify_acl()
59
60 # Perform policy checks.
61- self.policy.checkUpload(self)
62+ policy.checkUpload(self)
63
64 # That's all folks.
65 self.logger.debug("Finished checking upload.")
66@@ -996,8 +973,6 @@
67 # so late in the game is that in the
68 # mixed-upload case we only have a
69 # sourcepackagerelease to verify here!
70- assert self.policy.can_upload_mixed, (
71- "Current policy does not allow mixed uploads.")
72 assert sourcepackagerelease, (
73 "No sourcepackagerelease was found.")
74 binary_package_file.verifySourcePackageRelease(
75
76=== modified file 'lib/lp/archiveuploader/tests/__init__.py'
77--- lib/lp/archiveuploader/tests/__init__.py 2010-08-20 20:31:18 +0000
78+++ lib/lp/archiveuploader/tests/__init__.py 2010-08-25 17:14:48 +0000
79@@ -117,6 +117,10 @@
80 # We require the changes to be signed but not the dsc
81 self.unsigned_dsc_ok = True
82
83+ def validateUploadType(self, upload):
84+ """We accept uploads of any type."""
85+ pass
86+
87 def policySpecificChecks(self, upload):
88 """Nothing, let it go."""
89 pass
90
91=== modified file 'lib/lp/archiveuploader/tests/nascentupload-announcements.txt'
92--- lib/lp/archiveuploader/tests/nascentupload-announcements.txt 2010-08-25 17:14:46 +0000
93+++ lib/lp/archiveuploader/tests/nascentupload-announcements.txt 2010-08-25 17:14:48 +0000
94@@ -297,9 +297,10 @@
95 NEW binary upload to RELEASE pocket via 'sync' policy (we need to
96 override sync policy to allow binary uploads):
97
98+ >>> from lp.archiveuploader.uploadpolicy import ArchiveUploadType
99 >>> modified_sync_policy = getPolicy(
100 ... name='sync', distro='ubuntu', distroseries='hoary')
101- >>> modified_sync_policy.can_upload_binaries = True
102+ >>> modified_sync_policy.accepted_type = ArchiveUploadType.BINARY_ONLY
103
104 >>> bar_src = NascentUpload.from_changesfile_path(
105 ... datadir('suite/bar_1.0-1_binary/bar_1.0-1_i386.changes'),
106
107=== modified file 'lib/lp/archiveuploader/tests/nascentupload.txt'
108--- lib/lp/archiveuploader/tests/nascentupload.txt 2010-08-20 12:11:30 +0000
109+++ lib/lp/archiveuploader/tests/nascentupload.txt 2010-08-25 17:14:48 +0000
110@@ -499,9 +499,10 @@
111 need unsigned changes and binaries uploads (same as security, but also
112 accepts non-security uploads)
113
114+ >>> from lp.archiveuploader.uploadpolicy import ArchiveUploadType
115 >>> modified_sync_policy = getPolicy(
116 ... name='sync', distro='ubuntu', distroseries='hoary')
117- >>> modified_sync_policy.can_upload_binaries = True
118+ >>> modified_sync_policy.accepted_type = ArchiveUploadType.BINARY_ONLY
119
120 >>> ed_bin = NascentUpload.from_changesfile_path(
121 ... datadir('split-upload-test/ed_0.2-20_i386.changes'),
122
123=== modified file 'lib/lp/archiveuploader/tests/nascentuploadfile.txt'
124--- lib/lp/archiveuploader/tests/nascentuploadfile.txt 2010-08-20 12:11:30 +0000
125+++ lib/lp/archiveuploader/tests/nascentuploadfile.txt 2010-08-25 17:14:48 +0000
126@@ -531,9 +531,10 @@
127 The timestamp of the files in the .deb are tested against the policy for being
128 too new:
129
130+ >>> from lp.archiveuploader.uploadpolicy import ArchiveUploadType
131 >>> old_only_policy = getPolicy(
132 ... name='insecure', distro='ubuntu', distroseries='hoary')
133- >>> old_only_policy.can_upload_binaries = True
134+ >>> old_only_policy.accepted_type = ArchiveUploadType.BINARY_ONLY
135 >>> old_only_policy.future_time_grace = -5 * 365 * 24 * 60 * 60
136
137 >>> ed_binary_deb = DebBinaryUploadFile(ed_deb_path,
138@@ -549,7 +550,7 @@
139
140 >>> new_only_policy = getPolicy(
141 ... name='insecure', distro='ubuntu', distroseries='hoary')
142- >>> new_only_policy.can_upload_binaries = True
143+ >>> new_only_policy.accepted_type = ArchiveUploadType.BINARY_ONLY
144 >>> new_only_policy.earliest_year = 2010
145 >>> ed_binary_deb = DebBinaryUploadFile(ed_deb_path,
146 ... 'e31eeb0b6b3b87e1ea79378df864ffff',
147
148=== modified file 'lib/lp/archiveuploader/tests/test_buildduploads.py'
149--- lib/lp/archiveuploader/tests/test_buildduploads.py 2010-08-20 20:31:18 +0000
150+++ lib/lp/archiveuploader/tests/test_buildduploads.py 2010-08-25 17:14:48 +0000
151@@ -5,13 +5,139 @@
152
153 __metaclass__ = type
154
155+import os
156+
157+from zope.component import getUtility
158+
159 from canonical.database.constants import UTC_NOW
160 from canonical.launchpad.ftests import import_public_test_keys
161-from canonical.launchpad.interfaces import PackagePublishingStatus
162-from lp.archiveuploader.tests.test_securityuploads import (
163- TestStagedBinaryUploadBase,
164- )
165+from canonical.launchpad.interfaces import (
166+ PackagePublishingStatus,
167+ PackageUploadStatus,
168+ )
169+
170+from lp.archiveuploader.tests.test_uploadprocessor import (
171+ TestUploadProcessorBase,
172+ )
173+from lp.registry.interfaces.distribution import IDistributionSet
174 from lp.registry.interfaces.pocket import PackagePublishingPocket
175+from lp.soyuz.model.binarypackagebuild import BinaryPackageBuild
176+
177+
178+class TestStagedBinaryUploadBase(TestUploadProcessorBase):
179+ name = 'baz'
180+ version = '1.0-1'
181+ distribution_name = None
182+ distroseries_name = None
183+ pocket = None
184+ policy = 'buildd'
185+ no_mails = True
186+
187+ @property
188+ def distribution(self):
189+ return getUtility(IDistributionSet)[self.distribution_name]
190+
191+ @property
192+ def distroseries(self):
193+ return self.distribution[self.distroseries_name]
194+
195+ @property
196+ def package_name(self):
197+ return "%s_%s" % (self.name, self.version)
198+
199+ @property
200+ def source_dir(self):
201+ return self.package_name
202+
203+ @property
204+ def source_changesfile(self):
205+ return "%s_source.changes" % self.package_name
206+
207+ @property
208+ def binary_dir(self):
209+ return "%s_binary" % self.package_name
210+
211+ def getBinaryChangesfileFor(self, archtag):
212+ return "%s_%s.changes" % (self.package_name, archtag)
213+
214+ def setUp(self):
215+ """Setup environment for staged binaries upload via security policy.
216+
217+ 1. Setup queue directory and other basic attributes
218+ 2. Override policy options to get security policy and not send emails
219+ 3. Setup a common UploadProcessor with the overridden options
220+ 4. Store number of build present before issuing any upload
221+ 5. Upload the source package via security policy
222+ 6. Clean log messages.
223+ 7. Commit transaction, so the upload source can be seen.
224+ """
225+ super(TestStagedBinaryUploadBase, self).setUp()
226+ self.options.context = self.policy
227+ self.options.nomails = self.no_mails
228+ # Set up the uploadprocessor with appropriate options and logger
229+ self.uploadprocessor = self.getUploadProcessor(self.layer.txn)
230+ self.builds_before_upload = BinaryPackageBuild.select().count()
231+ self.source_queue = None
232+ self._uploadSource()
233+ self.log.lines = []
234+ self.layer.txn.commit()
235+
236+ def assertBuildsCreated(self, amount):
237+ """Assert that a given 'amount' of build records was created."""
238+ builds_count = BinaryPackageBuild.select().count()
239+ self.assertEqual(
240+ self.builds_before_upload + amount, builds_count)
241+
242+ def _prepareUpload(self, upload_dir):
243+ """Place a copy of the upload directory into incoming queue."""
244+ os.system("cp -a %s %s" %
245+ (os.path.join(self.test_files_dir, upload_dir),
246+ os.path.join(self.queue_folder, "incoming")))
247+
248+ def _uploadSource(self):
249+ """Upload and Accept (if necessary) the base source."""
250+ self._prepareUpload(self.source_dir)
251+ self.uploadprocessor.processChangesFile(
252+ os.path.join(self.queue_folder, "incoming", self.source_dir),
253+ self.source_changesfile)
254+ queue_item = self.uploadprocessor.last_processed_upload.queue_root
255+ self.assertTrue(
256+ queue_item is not None,
257+ "Source Upload Failed\nGot: %s" % "\n".join(self.log.lines))
258+ acceptable_statuses = [
259+ PackageUploadStatus.NEW,
260+ PackageUploadStatus.UNAPPROVED,
261+ ]
262+ if queue_item.status in acceptable_statuses:
263+ queue_item.setAccepted()
264+ # Store source queue item for future use.
265+ self.source_queue = queue_item
266+
267+ def _uploadBinary(self, archtag):
268+ """Upload the base binary.
269+
270+ Ensure it got processed and has a respective queue record.
271+ Return the IBuild attached to upload.
272+ """
273+ self._prepareUpload(self.binary_dir)
274+ self.uploadprocessor.processChangesFile(
275+ os.path.join(self.queue_folder, "incoming", self.binary_dir),
276+ self.getBinaryChangesfileFor(archtag))
277+ queue_item = self.uploadprocessor.last_processed_upload.queue_root
278+ self.assertTrue(
279+ queue_item is not None,
280+ "Binary Upload Failed\nGot: %s" % "\n".join(self.log.lines))
281+ self.assertEqual(queue_item.builds.count(), 1)
282+ return queue_item.builds[0].build
283+
284+ def _createBuild(self, archtag):
285+ """Create a build record attached to the base source."""
286+ spr = self.source_queue.sources[0].sourcepackagerelease
287+ build = spr.createBuild(
288+ distro_arch_series=self.distroseries[archtag],
289+ pocket=self.pocket, archive=self.distroseries.main_archive)
290+ self.layer.txn.commit()
291+ return build
292
293
294 class TestBuilddUploads(TestStagedBinaryUploadBase):
295
296=== modified file 'lib/lp/archiveuploader/tests/test_nascentupload_documentation.py'
297--- lib/lp/archiveuploader/tests/test_nascentupload_documentation.py 2010-08-23 13:39:29 +0000
298+++ lib/lp/archiveuploader/tests/test_nascentupload_documentation.py 2010-08-25 17:14:48 +0000
299@@ -30,6 +30,7 @@
300 getPolicy,
301 mock_logger_quiet,
302 )
303+from lp.archiveuploader.uploadpolicy import ArchiveUploadType
304 from lp.registry.interfaces.distribution import IDistributionSet
305 from lp.soyuz.interfaces.component import IComponentSet
306
307@@ -52,7 +53,7 @@
308 def getUploadForBinary(upload_path):
309 """Return a NascentUpload object for binaries."""
310 policy = getPolicy(name='sync', distro='ubuntu', distroseries='hoary')
311- policy.can_upload_binaries = True
312+ policy.accepted_type = ArchiveUploadType.BINARY_ONLY
313 return NascentUpload.from_changesfile_path(
314 datadir(upload_path), policy, mock_logger_quiet)
315
316
317=== modified file 'lib/lp/archiveuploader/tests/test_ppauploadprocessor.py'
318--- lib/lp/archiveuploader/tests/test_ppauploadprocessor.py 2010-08-20 20:31:18 +0000
319+++ lib/lp/archiveuploader/tests/test_ppauploadprocessor.py 2010-08-25 17:14:48 +0000
320@@ -681,12 +681,9 @@
321 "bar_1.0-1-mixed", "~name16/ubuntu")
322 self.processUpload(self.uploadprocessor, upload_dir)
323
324- self.assertEqual(
325- self.uploadprocessor.last_processed_upload.rejection_message,
326- 'Upload rejected because it contains binary packages. Ensure '
327- 'you are using `debuild -S`, or an equivalent command, to '
328- 'generate only the source package before re-uploading. See '
329- 'https://help.launchpad.net/Packaging/PPA for more information.')
330+ self.assertIn(
331+ 'Source/binary (i.e. mixed) uploads are not allowed.',
332+ self.uploadprocessor.last_processed_upload.rejection_message)
333
334 def testPGPSignatureNotPreserved(self):
335 """PGP signatures should be removed from PPA .changes files.
336
337=== added file 'lib/lp/archiveuploader/tests/test_uploadpolicy.py'
338--- lib/lp/archiveuploader/tests/test_uploadpolicy.py 1970-01-01 00:00:00 +0000
339+++ lib/lp/archiveuploader/tests/test_uploadpolicy.py 2010-08-25 17:14:48 +0000
340@@ -0,0 +1,127 @@
341+#!/usr/bin/python
342+#
343+# Copyright 2010 Canonical Ltd. This software is licensed under the
344+# GNU Affero General Public License version 3 (see the file LICENSE).
345+
346+from canonical.testing.layers import DatabaseFunctionalLayer
347+
348+from lp.app.errors import NotFoundError
349+from lp.archiveuploader.uploadpolicy import (
350+ AbstractUploadPolicy,
351+ ArchiveUploadType,
352+ )
353+from lp.testing import TestCase, TestCaseWithFactory
354+
355+
356+class TestUploadPolicy_validateUploadType(TestCase):
357+ """Test what kind (sourceful/binaryful/mixed) of uploads are accepted."""
358+
359+ def test_sourceful_accepted(self):
360+ policy = make_policy(accepted_type=ArchiveUploadType.SOURCE_ONLY)
361+ upload = make_fake_upload(sourceful=True)
362+
363+ policy.validateUploadType(upload)
364+
365+ self.assertEquals([], upload.rejections)
366+
367+ def test_binaryful_accepted(self):
368+ policy = make_policy(accepted_type=ArchiveUploadType.BINARY_ONLY)
369+ upload = make_fake_upload(binaryful=True)
370+
371+ policy.validateUploadType(upload)
372+
373+ self.assertEquals([], upload.rejections)
374+
375+ def test_mixed_accepted(self):
376+ policy = make_policy(accepted_type=ArchiveUploadType.MIXED_ONLY)
377+ upload = make_fake_upload(sourceful=True, binaryful=True)
378+
379+ policy.validateUploadType(upload)
380+
381+ self.assertEquals([], upload.rejections)
382+
383+ def test_sourceful_not_accepted(self):
384+ policy = make_policy(accepted_type=ArchiveUploadType.BINARY_ONLY)
385+ upload = make_fake_upload(sourceful=True)
386+
387+ policy.validateUploadType(upload)
388+
389+ self.assertIn(
390+ 'Sourceful uploads are not accepted by this policy.',
391+ upload.rejections)
392+
393+ def test_binaryful_not_accepted(self):
394+ policy = make_policy(accepted_type=ArchiveUploadType.SOURCE_ONLY)
395+ upload = make_fake_upload(binaryful=True)
396+
397+ policy.validateUploadType(upload)
398+
399+ self.assertTrue(len(upload.rejections) > 0)
400+ self.assertIn(
401+ 'Upload rejected because it contains binary packages.',
402+ upload.rejections[0])
403+
404+ def test_mixed_not_accepted(self):
405+ policy = make_policy(accepted_type=ArchiveUploadType.SOURCE_ONLY)
406+ upload = make_fake_upload(sourceful=True, binaryful=True)
407+
408+ policy.validateUploadType(upload)
409+
410+ self.assertIn(
411+ 'Source/binary (i.e. mixed) uploads are not allowed.',
412+ upload.rejections)
413+
414+ def test_sourceful_when_only_mixed_accepted(self):
415+ policy = make_policy(accepted_type=ArchiveUploadType.MIXED_ONLY)
416+ upload = make_fake_upload(sourceful=True, binaryful=False)
417+
418+ policy.validateUploadType(upload)
419+
420+ self.assertIn(
421+ 'Sourceful uploads are not accepted by this policy.',
422+ upload.rejections)
423+
424+ def test_binaryful_when_only_mixed_accepted(self):
425+ policy = make_policy(accepted_type=ArchiveUploadType.MIXED_ONLY)
426+ upload = make_fake_upload(sourceful=False, binaryful=True)
427+
428+ policy.validateUploadType(upload)
429+
430+ self.assertTrue(len(upload.rejections) > 0)
431+ self.assertIn(
432+ 'Upload rejected because it contains binary packages.',
433+ upload.rejections[0])
434+
435+
436+class TestUploadPolicy(TestCaseWithFactory):
437+
438+ layer = DatabaseFunctionalLayer
439+
440+ def test_setDistroSeriesAndPocket_distro_not_found(self):
441+ policy = AbstractUploadPolicy()
442+ policy.distro = self.factory.makeDistribution()
443+ self.assertRaises(
444+ NotFoundError, policy.setDistroSeriesAndPocket,
445+ 'nonexistent_security')
446+
447+
448+class FakeNascentUpload:
449+
450+ def __init__(self, sourceful, binaryful):
451+ self.sourceful = sourceful
452+ self.binaryful = binaryful
453+ self.is_ppa = False
454+ self.rejections = []
455+
456+ def reject(self, msg):
457+ self.rejections.append(msg)
458+
459+
460+def make_fake_upload(sourceful=False, binaryful=False):
461+ return FakeNascentUpload(sourceful, binaryful)
462+
463+
464+def make_policy(accepted_type):
465+ policy = AbstractUploadPolicy()
466+ policy.accepted_type = accepted_type
467+ return policy
468
469=== modified file 'lib/lp/archiveuploader/uploadpolicy.py'
470--- lib/lp/archiveuploader/uploadpolicy.py 2010-08-25 17:14:46 +0000
471+++ lib/lp/archiveuploader/uploadpolicy.py 2010-08-25 17:14:48 +0000
472@@ -7,6 +7,7 @@
473
474 __all__ = [
475 "AbstractUploadPolicy",
476+ "ArchiveUploadType",
477 "BuildDaemonUploadPolicy",
478 "findPolicyByName",
479 "IArchiveUploadPolicy",
480@@ -14,6 +15,8 @@
481 "UploadPolicyError",
482 ]
483
484+from textwrap import dedent
485+
486 from zope.component import (
487 getGlobalSiteManager,
488 getUtility,
489@@ -28,6 +31,9 @@
490 from lp.registry.interfaces.pocket import PackagePublishingPocket
491 from lp.registry.interfaces.series import SeriesStatus
492
493+from lazr.enum import EnumeratedType, Item
494+
495+
496 # Defined here so that uploadpolicy.py doesn't depend on lp.code.
497 SOURCE_PACKAGE_RECIPE_UPLOAD_POLICY_NAME = 'recipe'
498 # Number of seconds in an hour (used later)
499@@ -52,6 +58,13 @@
500 """
501
502
503+class ArchiveUploadType(EnumeratedType):
504+
505+ SOURCE_ONLY = Item("Source only")
506+ BINARY_ONLY = Item("Binary only")
507+ MIXED_ONLY = Item("Mixed only")
508+
509+
510 class AbstractUploadPolicy:
511 """Encapsulate the policy of an upload to a launchpad archive.
512
513@@ -63,8 +76,9 @@
514 """
515 implements(IArchiveUploadPolicy)
516
517+ name = 'abstract'
518 options = None
519- name = 'abstract'
520+ accepted_type = None # Must be defined in subclasses.
521
522 def __init__(self):
523 """Prepare a policy..."""
524@@ -75,14 +89,45 @@
525 self.unsigned_changes_ok = False
526 self.unsigned_dsc_ok = False
527 self.create_people = True
528- self.can_upload_source = True
529- self.can_upload_binaries = True
530- self.can_upload_mixed = True
531 # future_time_grace is in seconds. 28800 is 8 hours
532 self.future_time_grace = 8 * HOURS
533 # The earliest year we accept in a deb's file's mtime
534 self.earliest_year = 1984
535
536+ def validateUploadType(self, upload):
537+ """Check that the type of the given upload is accepted by this policy.
538+
539+ When the type (e.g. sourceful, binaryful or mixed) is not accepted,
540+ the upload is rejected.
541+ """
542+ if upload.sourceful and upload.binaryful:
543+ if self.accepted_type != ArchiveUploadType.MIXED_ONLY:
544+ upload.reject(
545+ "Source/binary (i.e. mixed) uploads are not allowed.")
546+
547+ elif upload.sourceful:
548+ if self.accepted_type != ArchiveUploadType.SOURCE_ONLY:
549+ upload.reject(
550+ "Sourceful uploads are not accepted by this policy.")
551+
552+ elif upload.binaryful:
553+ if self.accepted_type != ArchiveUploadType.BINARY_ONLY:
554+ message = dedent("""
555+ Upload rejected because it contains binary packages.
556+ Ensure you are using `debuild -S`, or an equivalent
557+ command, to generate only the source package before
558+ re-uploading.""")
559+
560+ if upload.is_ppa:
561+ message += dedent("""
562+ See https://help.launchpad.net/Packaging/PPA for
563+ more information.""")
564+ upload.reject(message)
565+
566+ else:
567+ raise AssertionError(
568+ "Upload is not sourceful, binaryful or mixed.")
569+
570 def getUploader(self, changes):
571 """Get the person who is doing the uploading."""
572 return changes.signer
573@@ -171,11 +216,7 @@
574 """The insecure upload policy is used by the poppy interface."""
575
576 name = 'insecure'
577-
578- def __init__(self):
579- AbstractUploadPolicy.__init__(self)
580- self.can_upload_binaries = False
581- self.can_upload_mixed = False
582+ accepted_type = ArchiveUploadType.SOURCE_ONLY
583
584 def rejectPPAUploads(self, upload):
585 """Insecure policy allows PPA upload."""
586@@ -283,14 +324,13 @@
587 """The build daemon upload policy is invoked by the slave scanner."""
588
589 name = 'buildd'
590+ accepted_type = ArchiveUploadType.BINARY_ONLY
591
592 def __init__(self):
593 super(BuildDaemonUploadPolicy, self).__init__()
594 # We permit unsigned uploads because we trust our build daemons
595 self.unsigned_changes_ok = True
596 self.unsigned_dsc_ok = True
597- self.can_upload_source = False
598- self.can_upload_mixed = False
599
600 def setOptions(self, options):
601 AbstractUploadPolicy.setOptions(self, options)
602@@ -314,15 +354,13 @@
603 """This policy is invoked when processing sync uploads."""
604
605 name = 'sync'
606+ accepted_type = ArchiveUploadType.SOURCE_ONLY
607
608 def __init__(self):
609 AbstractUploadPolicy.__init__(self)
610 # We don't require changes or dsc to be signed for syncs
611 self.unsigned_changes_ok = True
612 self.unsigned_dsc_ok = True
613- # We don't want binaries in a sync
614- self.can_upload_mixed = False
615- self.can_upload_binaries = False
616
617 def policySpecificChecks(self, upload):
618 """Perform sync specific checks."""
619
620=== modified file 'lib/lp/code/model/sourcepackagerecipebuild.py'
621--- lib/lp/code/model/sourcepackagerecipebuild.py 2010-08-24 09:51:26 +0000
622+++ lib/lp/code/model/sourcepackagerecipebuild.py 2010-08-25 17:14:48 +0000
623@@ -40,6 +40,7 @@
624 from canonical.launchpad.webapp import errorlog
625 from lp.app.errors import NotFoundError
626 from lp.archiveuploader.uploadpolicy import (
627+ ArchiveUploadType,
628 BuildDaemonUploadPolicy,
629 IArchiveUploadPolicy,
630 SOURCE_PACKAGE_RECIPE_UPLOAD_POLICY_NAME,
631@@ -78,11 +79,7 @@
632 """Policy for uploading the results of a source package recipe build."""
633
634 name = SOURCE_PACKAGE_RECIPE_UPLOAD_POLICY_NAME
635-
636- def __init__(self):
637- super(SourcePackageRecipeUploadPolicy, self).__init__()
638- self.can_upload_source = True
639- self.can_upload_binaries = False
640+ accepted_type = ArchiveUploadType.SOURCE_ONLY
641
642 def getUploader(self, changes):
643 """Return the person doing the upload."""
644
645=== modified file 'lib/lp/soyuz/browser/tests/distroseriesqueue-views.txt'
646--- lib/lp/soyuz/browser/tests/distroseriesqueue-views.txt 2010-08-20 12:11:30 +0000
647+++ lib/lp/soyuz/browser/tests/distroseriesqueue-views.txt 2010-08-25 17:14:48 +0000
648@@ -253,6 +253,7 @@
649 ... sourcename="foo", distroseries=hoary, version="1.0-2",
650 ... status=PackagePublishingStatus.PUBLISHED)
651
652+ >>> from lp.archiveuploader.uploadpolicy import ArchiveUploadType
653 >>> from canonical.launchpad.ftests import import_public_test_keys
654 >>> from canonical.launchpad.interfaces import IComponentSet
655 >>> from lp.soyuz.model.component import ComponentSelection
656@@ -264,7 +265,7 @@
657 >>> trash = ComponentSelection(distroseries=hoary, component=universe)
658 >>> sync_policy = getPolicy(name='sync', distro='ubuntu',
659 ... distroseries='hoary')
660- >>> sync_policy.can_upload_binaries = True
661+ >>> sync_policy.accepted_type = ArchiveUploadType.BINARY_ONLY
662 >>> foo_upload = NascentUpload.from_changesfile_path(
663 ... datadir('suite/foo_1.0-2_multi_binary/foo_1.0-2_i386.changes'),
664 ... sync_policy, mock_logger_quiet)
665
666=== modified file 'lib/lp/soyuz/doc/soyuz-set-of-uploads.txt'
667--- lib/lp/soyuz/doc/soyuz-set-of-uploads.txt 2010-08-06 14:29:36 +0000
668+++ lib/lp/soyuz/doc/soyuz-set-of-uploads.txt 2010-08-25 17:14:48 +0000
669@@ -319,22 +319,6 @@
670 Subject: bar_1.0-3_source.changes rejected
671 ...
672
673-Check the rejection (instead of failure) of an upload to a series or
674-pocket which does not exist. We use the security upload policy for easier
675-testing, as unsigned uploads are allowed.
676-
677- >>> simulate_upload(
678- ... 'unstable_1.0-1', upload_policy="security", loglevel=logging.ERROR)
679- Rejected uploads: ['unstable_1.0-1']
680-
681- >>> read_email()
682- To: Celso Providelo <celso.providelo@canonical.com>
683- Subject: unstable_1.0-1_source.changes rejected
684- ...
685- Rejected:
686- Unable to find distroseries: unstable
687- ...
688-
689 Force weird behavior with rfc2047 sentences containing '.' on
690 bar_1.0-4, which caused bug # 41102.
691
692@@ -605,6 +589,9 @@
693
694 == Staged Security Uploads ==
695
696+XXX: Salgado, 2010-08-25, bug=624078: This entire section should be removed
697+but if you do so publish-distro.py (called further down) will hang.
698+
699 Perform a staged (source-first) security upload, simulating a run of
700 the buildd-queuebuilder inbetween, to verify fix for bug 53437. To be
701 allowed a security upload, we need to use a released distroseries,
702@@ -617,7 +604,7 @@
703 >>> ss = SectionSelection(distroseries=warty, section=devel)
704
705 >>> simulate_upload(
706- ... 'baz_1.0-1', is_new=True, upload_policy="security",
707+ ... 'baz_1.0-1', is_new=True, upload_policy="anything",
708 ... series="warty-security", distro="ubuntu")
709
710 Check there's a SourcePackageRelease with no build.
711@@ -650,7 +637,7 @@
712 Upload the i386 binary:
713
714 >>> simulate_upload(
715- ... 'baz_1.0-1_single_binary', is_new=True, upload_policy="security",
716+ ... 'baz_1.0-1_single_binary', is_new=True, upload_policy="anything",
717 ... distro="ubuntu", series="warty-security")
718
719 Should still just have one build, and it should now be FULLYBUILT.