Merge lp:~cjwatson/launchpad/remove-queue-tool into lp:launchpad

Proposed by Colin Watson on 2012-07-11
Status: Merged
Approved by: William Grant on 2012-09-27
Approved revision: no longer in the source branch.
Merged at revision: 16052
Proposed branch: lp:~cjwatson/launchpad/remove-queue-tool
Merge into: lp:launchpad
Diff against target: 2562 lines (+253/-2208)
7 files modified
lib/lp/soyuz/doc/ftpmaster-tools.txt (+0/-71)
lib/lp/soyuz/model/sourcepackagerelease.py (+2/-2)
lib/lp/soyuz/scripts/queue.py (+0/-737)
lib/lp/soyuz/scripts/tests/test_processaccepted.py (+52/-3)
lib/lp/soyuz/scripts/tests/test_queue.py (+0/-1270)
lib/lp/soyuz/tests/test_packageupload.py (+199/-0)
scripts/ftpmaster-tools/queue (+0/-125)
To merge this branch: bzr merge lp:~cjwatson/launchpad/remove-queue-tool
Reviewer Review Type Date Requested Status
William Grant code 2012-07-12 Approve on 2012-09-27
j.c.sackett (community) 2012-07-11 Approve on 2012-07-12
Review via email: mp+114464@code.launchpad.net

Commit Message

Remove the queue script, now entirely superseded by the PackageUpload API.

Description of the Change

== Summary ==

Complete the migration from the queue script to an API client.

== Proposed fix ==

Once the current contents of devel pass buildbot and QA and are deployed, the queue API client will be feature-complete. We can and should therefore remove the old queue script to reclaim the lines of code spent on the API work (twice over, in fact).

== Implementation details ==

Many of the tests were already exercised elsewhere (notably, overrides are I think already reasonably well exercised via TestPackageUploadWebservice), and some of them were specific to the script and are thus obsolete, but some of them were neither. I moved the tests of DistroSeries:+queue closing private bugs to lp.soyuz.scripts.tests.test_processaccepted (there was already a comment indicating that that needed to happen); and a number of the accept/reject tests were essentially testing the model, so I turned them into proper model tests in lp.soyuz.tests.test_packageupload.

== Tests ==

bin/test -vvct test_packageupload -t test_processaccepted

== Lint ==

Just an existing false positive due to pocketlint not understanding property setters:

./lib/lp/soyuz/model/sourcepackagerelease.py
     196: redefinition of function 'copyright' from line 187

To post a comment you must log in.
j.c.sackett (jcsackett) wrote :

This looks alright to me, but I have requested that William Grant take a look at it as he's far more aware of the various bits and pieces involved in this, and he's worked with you on a lot of this work.

review: Approve
Colin Watson (cjwatson) wrote :

I've asked ubuntu-devel@ about arrangements to stress-test the new API in production:

  https://lists.ubuntu.com/archives/ubuntu-devel/2012-July/035528.html

Colin Watson (cjwatson) wrote :

Putting this on hold for now, since we're going to have to fix bug 745799 first.

  https://lists.ubuntu.com/archives/ubuntu-devel/2012-July/035552.html

Colin Watson (cjwatson) wrote :

This should be unblocked now. The fix for bug 745799 is behaving fine in production.

William Grant (wgrant) :
review: Approve (code)

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== removed file 'lib/lp/soyuz/doc/ftpmaster-tools.txt'
2--- lib/lp/soyuz/doc/ftpmaster-tools.txt 2011-12-29 05:29:36 +0000
3+++ lib/lp/soyuz/doc/ftpmaster-tools.txt 1970-01-01 00:00:00 +0000
4@@ -1,71 +0,0 @@
5-= FTPMASTER Tools =
6-
7-This test commits to the test database in subprocesses and so needs to
8-force the DatabaseLayer to fully tear down and restore the database
9-after this test.
10-
11- >>> from lp.testing.layers import DatabaseLayer
12- >>> DatabaseLayer.force_dirty_database()
13-
14-Queue Tool is a script designed to handle queue content.
15-This test will check its output, since the script itself would
16-open a new connection, let's invoke it in dry-run mode.
17-
18- >>> import subprocess
19- >>> import os
20- >>> import sys
21- >>> from lp.services.config import config
22-
23-
24- >>> script = os.path.join(config.root, "scripts", "ftpmaster-tools",
25- ... "queue")
26-
27-INFO
28-
29- >>> process = subprocess.Popen([sys.executable, script,
30- ... "-s", "breezy-autotest", "info"],
31- ... stdout=subprocess.PIPE)
32- >>> stdout, stderr = process.communicate()
33- >>> process.returncode
34- 0
35- >>> print stdout
36- Initializing connection to queue new
37- Running: "info"
38- Listing ubuntu/breezy-autotest (NEW) 6/6
39- ---------|----|----------------------|----------------------|---------------
40- 7 | -- | netapplet-1.0.0.tar. | - | ...
41- | * netapplet-1.0.0.tar.gz Format: DDTP_TARBALL
42- 6 | -- | netapplet-1.0.0.tar. | - | ...
43- | * netapplet-1.0.0.tar.gz Format: DIST_UPGRADER
44- 4 | S- | alsa-utils | 1.0.9a-4ubuntu1 | ...
45- | * alsa-utils/1.0.9a-4ubuntu1 Component: main Section: base
46- 3 | S- | netapplet | 0.99.6-1 | ...
47- | * netapplet/0.99.6-1 Component: main Section: web
48- 2 | -B | pmount (i386) | 0.1-1 | ...
49- | N pmount/0.1-1/i386 Component: main Section: base Priority: IMPORTANT
50- 1 | -B | mozilla-firefox (i38 | 0.9 | ...
51- | N mozilla-firefox/0.9/i386 Component: main Section: base Priority: EXTRA
52- ---------|----|----------------------|----------------------|---------------
53- 6/6
54- total
55-
56-
57-Check the custom uploads presentation:
58-
59- >>> process = subprocess.Popen([sys.executable, script, "-Q", "unapproved",
60- ... "-s", "breezy-autotest-updates", "info"],
61- ... stdout=subprocess.PIPE)
62- >>> stdout, stderr = process.communicate()
63- >>> process.returncode
64- 0
65- >>> print stdout
66- Initializing connection to queue unapproved
67- Running: "info"
68- Listing ubuntu/breezy-autotest-updates (UNAPPROVED) 1/1
69- ---------|----|----------------------|----------------------|---------------
70- 5 | -- | netapplet-1.0.0.tar. | - | ...
71- | * netapplet-1.0.0.tar.gz Format: ROSETTA_TRANSLATIONS
72- ---------|----|----------------------|----------------------|---------------
73- 1/1 total
74- <BLANKLINE>
75-
76
77=== modified file 'lib/lp/soyuz/model/sourcepackagerelease.py'
78--- lib/lp/soyuz/model/sourcepackagerelease.py 2012-08-01 11:02:13 +0000
79+++ lib/lp/soyuz/model/sourcepackagerelease.py 2012-09-28 10:09:21 +0000
80@@ -66,6 +66,7 @@
81 from lp.soyuz.interfaces.archive import MAIN_ARCHIVE_PURPOSES
82 from lp.soyuz.interfaces.binarypackagebuild import IBinaryPackageBuildSet
83 from lp.soyuz.interfaces.packagediff import PackageDiffAlreadyRequested
84+from lp.soyuz.interfaces.queue import QueueInconsistentStateError
85 from lp.soyuz.interfaces.sourcepackagerelease import ISourcePackageRelease
86 from lp.soyuz.model.binarypackagebuild import BinaryPackageBuild
87 from lp.soyuz.model.files import SourcePackageReleaseFile
88@@ -74,7 +75,6 @@
89 PackageUpload,
90 PackageUploadSource,
91 )
92-from lp.soyuz.scripts.queue import QueueActionError
93 from lp.translations.interfaces.translationimportqueue import (
94 ITranslationImportQueue,
95 )
96@@ -474,7 +474,7 @@
97 if new_archive is not None:
98 self.upload_archive = new_archive
99 else:
100- raise QueueActionError(
101+ raise QueueInconsistentStateError(
102 "New component '%s' requires a non-existent archive.")
103 if section is not None:
104 self.section = section
105
106=== removed file 'lib/lp/soyuz/scripts/queue.py'
107--- lib/lp/soyuz/scripts/queue.py 2012-07-03 16:04:54 +0000
108+++ lib/lp/soyuz/scripts/queue.py 1970-01-01 00:00:00 +0000
109@@ -1,737 +0,0 @@
110-# Copyright 2009-2012 Canonical Ltd. This software is licensed under the
111-# GNU Affero General Public License version 3 (see the file LICENSE).
112-
113-# pylint: disable-msg=W0231
114-"""Ftpmaster queue tool libraries."""
115-
116-# XXX StuartBishop 2007-01-31:
117-# This should be renamed to ftpmasterqueue.py or just ftpmaster.py
118-# as Launchpad contains lots of queues.
119-
120-__metaclass__ = type
121-__all__ = [
122- 'CommandRunner',
123- 'CommandRunnerError',
124- 'QueueActionError',
125- 'name_queue_map',
126- ]
127-
128-from datetime import datetime
129-import errno
130-import hashlib
131-
132-import pytz
133-from zope.component import getUtility
134-
135-from lp.app.browser.tales import DurationFormatterAPI
136-from lp.app.errors import NotFoundError
137-from lp.services.config import config
138-from lp.services.librarian.utils import filechunks
139-from lp.services.propertycache import cachedproperty
140-from lp.soyuz.enums import PackageUploadStatus
141-from lp.soyuz.interfaces.component import IComponentSet
142-from lp.soyuz.interfaces.queue import (
143- IPackageUploadSet,
144- QueueInconsistentStateError,
145- )
146-from lp.soyuz.interfaces.section import ISectionSet
147-
148-
149-name_queue_map = {
150- "new": PackageUploadStatus.NEW,
151- "unapproved": PackageUploadStatus.UNAPPROVED,
152- "accepted": PackageUploadStatus.ACCEPTED,
153- "done": PackageUploadStatus.DONE,
154- "rejected": PackageUploadStatus.REJECTED,
155- }
156-
157-#XXX cprov 2006-09-19: We need to use template engine instead of harcoded
158-# format variables.
159-HEAD = "-" * 9 + "|----|" + "-" * 22 + "|" + "-" * 22 + "|" + "-" * 15
160-FOOT_MARGIN = " " * (9 + 6 + 1 + 22 + 1 + 22 + 2)
161-RULE = "-" * (12 + 9 + 6 + 1 + 22 + 1 + 22 + 2)
162-
163-FILTERMSG = """
164- Omit the filter for all records.
165- Filter string consists of a queue ID or a pair <name>[/<version>]:
166-
167- 28
168- apt
169- apt/1
170-
171- Use '-e' command line argument for exact matches:
172-
173- -e apt
174- -e apt/1.0-1
175-"""
176-
177-
178-class QueueActionError(Exception):
179- """Identify Errors occurred within QueueAction class and its children."""
180-
181-
182-class QueueAction:
183- """Queue Action base class.
184-
185- Implements a bunch of common/useful method designed to provide easy
186- PackageUpload handling.
187- """
188-
189- def __init__(self, distribution_name, suite_name, queue, terms,
190- component_name, section_name, priority_name,
191- display, no_mail=True, exact_match=False, log=None):
192- """Initializes passed variables. """
193- self.terms = terms
194- # Some actions have addtional commands at the start of the terms
195- # so allow them to state that here by specifiying the start index.
196- self.terms_start_index = 0
197- self.component_name = component_name
198- self.section_name = section_name
199- self.priority_name = priority_name
200- self.exact_match = exact_match
201- self.queue = queue
202- self.no_mail = no_mail
203- self.distribution_name = distribution_name
204- self.suite_name = suite_name
205- self.default_sender = "%s <%s>" % (
206- config.uploader.default_sender_name,
207- config.uploader.default_sender_address)
208- self.default_recipient = "%s <%s>" % (
209- config.uploader.default_recipient_name,
210- config.uploader.default_recipient_address)
211- self.display = display
212- self.log = log
213-
214- @cachedproperty
215- def size(self):
216- """Return the size of the queue in question."""
217- return getUtility(IPackageUploadSet).count(
218- status=self.queue, distroseries=self.distroseries,
219- pocket=self.pocket)
220-
221- def setDefaultContext(self):
222- """Set default distribuiton, distroseries."""
223- # if not found defaults to 'ubuntu'
224-
225- # Avoid circular imports.
226- from lp.registry.interfaces.distribution import IDistributionSet
227- from lp.registry.interfaces.pocket import PackagePublishingPocket
228-
229- distroset = getUtility(IDistributionSet)
230- try:
231- self.distribution = distroset[self.distribution_name]
232- except NotFoundError:
233- self.distribution = distroset['ubuntu']
234-
235- if self.suite_name:
236- # defaults to distro.currentseries if passed distroseries is
237- # misapplied or not found.
238- try:
239- self.distroseries, self.pocket = (
240- self.distribution.getDistroSeriesAndPocket(
241- self.suite_name))
242- except NotFoundError:
243- raise QueueActionError('Context not found: "%s/%s"'
244- % (self.distribution.name,
245- self.suite_name))
246- else:
247- self.distroseries = self.distribution.currentseries
248- self.pocket = PackagePublishingPocket.RELEASE
249-
250- def initialize(self):
251- """Builds a list of affected records based on the filter argument."""
252- self.setDefaultContext()
253-
254- self.package_names = []
255- self.items = []
256- self.items_size = 0
257-
258- # Will be set to true if the command line specified package IDs.
259- # This is required because package_names is expanded into IDs so we
260- # need another way of knowing whether the user typed them.
261- self.explicit_ids_specified = False
262-
263- terms = self.terms[self.terms_start_index:]
264- if len(terms) == 0:
265- # If no argument is passed, present all available results in
266- # the selected queue.
267- terms.append('')
268-
269- for term in terms:
270- # refuse old-style '*' argument since we do not support
271- # wildcards yet.
272- if term == '*':
273- self.displayUsage(FILTERMSG)
274-
275- if term.isdigit():
276- # retrieve PackageUpload item by id
277- try:
278- item = getUtility(IPackageUploadSet).get(int(term))
279- except NotFoundError as info:
280- raise QueueActionError('Queue Item not found: %s' % info)
281-
282- if item.status != self.queue:
283- raise QueueActionError(
284- 'Item %s is in queue %s' % (
285- item.id, item.status.name))
286-
287- if (item.distroseries != self.distroseries or
288- item.pocket != self.pocket):
289- raise QueueActionError(
290- 'Item %s is in %s/%s-%s not in %s/%s-%s'
291- % (item.id, item.distroseries.distribution.name,
292- item.distroseries.name, item.pocket.name,
293- self.distroseries.distribution.name,
294- self.distroseries.name, self.pocket.name))
295-
296- if item not in self.items:
297- self.items.append(item)
298- self.explicit_ids_specified = True
299- else:
300- # retrieve PackageUpload item by name/version key
301- version = None
302- if '/' in term:
303- term, version = term.strip().split('/')
304-
305- # Expand SQLObject results.
306- queue_items = self.distroseries.getPackageUploads(
307- status=self.queue, name=term, version=version,
308- exact_match=self.exact_match, pocket=self.pocket)
309- for item in queue_items:
310- if item not in self.items:
311- self.items.append(item)
312- self.package_names.append(term)
313-
314- self.items_size = len(self.items)
315-
316- def run(self):
317- """Place holder for command action."""
318- raise NotImplementedError('No action implemented.')
319-
320- def displayTitle(self, action):
321- """Common title/summary presentation method."""
322- self.display("%s %s/%s (%s) %s/%s" % (
323- action, self.distribution.name, self.suite_name,
324- self.queue.name, self.items_size, self.size))
325-
326- def displayHead(self):
327- """Table head presentation method."""
328- self.display(HEAD)
329-
330- def displayBottom(self):
331- """Displays the table bottom and a small statistic information."""
332- self.display(
333- FOOT_MARGIN + "%d/%d total" % (self.items_size, self.size))
334-
335- def displayRule(self):
336- """Displays a rule line. """
337- self.display(RULE)
338-
339- def displayUsage(self, extended_info=None):
340- """Display the class docstring as usage message.
341-
342- Raise QueueActionError with optional extended_info argument
343- """
344- self.display(self.__doc__)
345- raise QueueActionError(extended_info)
346-
347- def _makeTag(self, queue_item):
348- """Compose an upload type tag for `queue_item`.
349-
350- A source upload without binaries is tagged as "S-".
351- A binary upload without source is tagged as "-B."
352- An upload with both source and binaries is tagged as "SB".
353- An upload with a package copy job is tagged as "X-".
354- """
355- # XXX cprov 2006-07-31: source_tag and build_tag ('S' & 'B')
356- # are necessary simply to keep the format legaxy.
357- # We may discuss a more reasonable output format later
358- # and avoid extra boring code. The IDRQ.displayname should
359- # do should be enough.
360- if queue_item.package_copy_job is not None:
361- return "X-"
362-
363- source_tag = {
364- True: 'S',
365- False: '-',
366- }
367- binary_tag = {
368- True: 'B',
369- False: '-',
370- }
371- return (
372- source_tag[queue_item.contains_source] +
373- binary_tag[queue_item.contains_build])
374-
375- def displayItem(self, queue_item):
376- """Display one line summary of the queue item provided."""
377- tag = self._makeTag(queue_item)
378- displayname = queue_item.displayname
379- version = queue_item.displayversion
380- age = DurationFormatterAPI(
381- datetime.now(pytz.timezone('UTC')) -
382- queue_item.date_created).approximateduration()
383-
384- if queue_item.contains_build:
385- displayname = "%s (%s)" % (queue_item.displayname,
386- queue_item.displayarchs)
387-
388- self.display("%8d | %s | %s | %s | %s" %
389- (queue_item.id, tag, displayname.ljust(20)[:20],
390- version.ljust(20)[:20], age))
391-
392- def displayInfo(self, queue_item, only=None):
393- """Displays additional information about the provided queue item.
394-
395- Optionally pass a binarypackagename via 'only' argument to display
396- only exact matches within the selected build queue items.
397- """
398- if queue_item.package_copy_job or queue_item.sources:
399- self.display(
400- "\t | * %s/%s Component: %s Section: %s" % (
401- queue_item.package_name,
402- queue_item.package_version,
403- queue_item.component_name,
404- queue_item.section_name,
405- ))
406-
407- for queue_build in queue_item.builds:
408- for bpr in queue_build.build.binarypackages:
409- if only and only != bpr.name:
410- continue
411- if bpr.is_new:
412- status_flag = "N"
413- else:
414- status_flag = "*"
415- self.display(
416- "\t | %s %s/%s/%s Component: %s Section: %s Priority: %s"
417- % (status_flag, bpr.name, bpr.version,
418- bpr.build.distro_arch_series.architecturetag,
419- bpr.component.name, bpr.section.name,
420- bpr.priority.name))
421-
422- for queue_custom in queue_item.customfiles:
423- self.display("\t | * %s Format: %s"
424- % (queue_custom.libraryfilealias.filename,
425- queue_custom.customformat.name))
426-
427-
428-class QueueActionHelp(QueueAction):
429- """Present provided actions summary"""
430-
431- def __init__(self, **kargs):
432- self.kargs = kargs
433- self.kargs['no_mail'] = True
434- self.actions = kargs['terms']
435- self.display = kargs['display']
436-
437- def initialize(self):
438- """Mock initialization """
439- pass
440-
441- def run(self):
442- """Present the actions description summary"""
443- # present summary for specific or all actions
444- if not self.actions:
445- actions_help = queue_actions.items()
446- not_available_actions = []
447- else:
448- actions_help = [
449- (action, provider)
450- for action, provider in queue_actions.items()
451- if action in self.actions]
452- not_available_actions = [
453- action for action in self.actions
454- if action not in queue_actions.keys()]
455- # present not available requested action if any.
456- if not_available_actions:
457- self.display(
458- "Not available action(s): %s" %
459- ", ".join(not_available_actions))
460-
461- # extract summary from docstring of specified available actions
462- for action, wrapper in actions_help:
463- if action is 'help':
464- continue
465- wobj = wrapper(**self.kargs)
466- summary = wobj.__doc__.splitlines()[0]
467- self.display('\t%s : %s ' % (action, summary))
468-
469-
470-class QueueActionReport(QueueAction):
471- """Present a report about the size of available queues"""
472-
473- def initialize(self):
474- """Mock initialization """
475- self.setDefaultContext()
476-
477- def run(self):
478- """Display the queues size."""
479- self.display("Report for %s/%s" % (self.distribution.name,
480- self.distroseries.name))
481-
482- for queue in name_queue_map.values():
483- size = getUtility(IPackageUploadSet).count(
484- status=queue, distroseries=self.distroseries,
485- pocket=self.pocket)
486- self.display("\t%s -> %s entries" % (queue.name, size))
487-
488-
489-class QueueActionInfo(QueueAction):
490- """Present the Queue item including its contents.
491-
492- Presents the contents of the selected upload(s).
493-
494- queue info <filter>
495- """
496-
497- def run(self):
498- """Present the filtered queue ordered by date."""
499- self.displayTitle('Listing')
500- self.displayHead()
501- for queue_item in self.items:
502- self.displayItem(queue_item)
503- self.displayInfo(queue_item)
504- self.displayHead()
505- self.displayBottom()
506-
507-
508-class QueueActionFetch(QueueAction):
509- """Fetch the contents of a queue item.
510-
511- Download the contents of the selected upload(s).
512-
513- queue fetch <filter>
514- """
515-
516- def run(self):
517- self.displayTitle('Fetching')
518- self.displayRule()
519- for queue_item in self.items:
520- file_list = []
521- if queue_item.changesfile is not None:
522- file_list.append(queue_item.changesfile)
523-
524- for source in queue_item.sources:
525- for spr_file in source.sourcepackagerelease.files:
526- file_list.append(spr_file.libraryfile)
527-
528- for build in queue_item.builds:
529- for bpr in build.build.binarypackages:
530- for bpr_file in bpr.files:
531- file_list.append(bpr_file.libraryfile)
532-
533- for custom in queue_item.customfiles:
534- file_list.append(custom.libraryfilealias)
535-
536- for libfile in file_list:
537- self.display("Constructing %s" % libfile.filename)
538- # do not overwrite files on disk (bug # 62976)
539- try:
540- existing_file = open(libfile.filename, "r")
541- except IOError as e:
542- if not e.errno == errno.ENOENT:
543- raise
544- # File does not already exist, so read file from librarian
545- # and write to disk.
546- libfile.open()
547- out_file = open(libfile.filename, "w")
548- for chunk in filechunks(libfile):
549- out_file.write(chunk)
550- out_file.close()
551- libfile.close()
552- else:
553- # Check sha against existing file (bug #67014)
554- existing_sha = hashlib.sha1()
555- for chunk in filechunks(existing_file):
556- existing_sha.update(chunk)
557- existing_file.close()
558-
559- # bail out if the sha1 differs
560- if libfile.content.sha1 != existing_sha.hexdigest():
561- raise CommandRunnerError("%s already present on disk "
562- "and differs from new file"
563- % libfile.filename)
564- else:
565- self.display("%s already on disk and checksum "
566- "matches, skipping.")
567-
568- self.displayRule()
569- self.displayBottom()
570-
571-
572-class QueueActionReject(QueueAction):
573- """Reject the contents of a queue item.
574-
575- Move the selected upload(s) to the REJECTED queue.
576-
577- queue reject <filter>
578- """
579-
580- def run(self):
581- """Perform Reject action."""
582- self.displayTitle('Rejecting')
583- self.displayRule()
584- for queue_item in self.items:
585- self.display('Rejecting %s' % queue_item.displayname)
586- try:
587- queue_item.rejectFromQueue(
588- logger=self.log, dry_run=self.no_mail)
589- except QueueInconsistentStateError as info:
590- self.display('** %s could not be rejected due %s'
591- % (queue_item.displayname, info))
592-
593- self.displayRule()
594- self.displayBottom()
595-
596-
597-class QueueActionAccept(QueueAction):
598- """Accept the contents of a queue item.
599-
600- Move the selected upload(s) to the ACCEPTED queue.
601-
602- queue accept <filter>
603- """
604-
605- def run(self):
606- """Perform Accept action."""
607- self.displayTitle('Accepting')
608- self.displayRule()
609- for queue_item in self.items:
610- self.display('Accepting %s' % queue_item.displayname)
611- try:
612- queue_item.acceptFromQueue(
613- logger=self.log, dry_run=self.no_mail)
614- except QueueInconsistentStateError as info:
615- self.display('** %s could not be accepted due to %s'
616- % (queue_item.displayname, info))
617- continue
618-
619- self.displayRule()
620- self.displayBottom()
621-
622-
623-class QueueActionOverride(QueueAction):
624- """Override information in a queue item content.
625-
626- queue override [-c|--component] [-x|--section] [-p|--priority]
627- <override_stanza> <filter>
628-
629- Where override_stanza is one of:
630- source
631- binary
632-
633- In each case, when you want to set an override supply the relevant option.
634-
635- So, to set a binary to have section 'editors' but leave the
636- component and priority alone, do:
637-
638- queue override -x editors binary <filter>
639-
640- Binaries can only be overridden by passing a name filter, so it will
641- only override the binary package which matches the filter.
642-
643- Or, to set a source's section to editors, do:
644-
645- queue override -x editors source <filter>
646- """
647- supported_override_stanzas = ['source', 'binary']
648-
649- def __init__(self, distribution_name, suite_name, queue, terms,
650- component_name, section_name, priority_name,
651- display, no_mail=True, exact_match=False, log=None):
652- """Constructor for QueueActionOverride."""
653-
654- # This exists so that self.terms_start_index can be set as this action
655- # class has a command at the start of the terms.
656- # Our first term is "binary" or "source" to specify the type of
657- # over-ride.
658- QueueAction.__init__(self, distribution_name, suite_name, queue,
659- terms, component_name, section_name,
660- priority_name, display, no_mail=True,
661- exact_match=False, log=log)
662- self.terms_start_index = 1
663- self.overrides_performed = 0
664-
665- def run(self):
666- """Perform Override action."""
667- self.displayTitle('Overriding')
668- self.displayRule()
669-
670- # "terms" is the list of arguments starting at the override stanza
671- # ("source" or "binary").
672- try:
673- override_stanza = self.terms[0]
674- except IndexError:
675- self.displayUsage('Missing override_stanza.')
676- return
677-
678- if override_stanza not in self.supported_override_stanzas:
679- self.displayUsage('Not supported override_stanza: %s'
680- % override_stanza)
681- return
682-
683- return getattr(self, '_override_' + override_stanza)()
684-
685- def _override_source(self):
686- """Overrides sourcepackagereleases selected.
687-
688- It doesn't check Component/Section Selection, this is a task
689- for queue state-machine.
690- """
691- component = None
692- section = None
693- try:
694- if self.component_name:
695- component = getUtility(IComponentSet)[self.component_name]
696- if self.section_name:
697- section = getUtility(ISectionSet)[self.section_name]
698- except NotFoundError as info:
699- raise QueueActionError('Not Found: %s' % info)
700-
701- for queue_item in self.items:
702- # We delegate to the queue_item itself to override any/all
703- # of its sources.
704- if queue_item.contains_source or queue_item.package_copy_job:
705- if queue_item.sourcepackagerelease:
706- old_component = queue_item.sourcepackagerelease.component
707- else:
708- old_component = getUtility(IComponentSet)[
709- queue_item.package_copy_job.component_name]
710- queue_item.overrideSource(
711- component, section, [
712- component, old_component])
713- self.overrides_performed += 1
714- self.displayInfo(queue_item)
715-
716- def _override_binary(self):
717- """Overrides binarypackagereleases selected"""
718- from lp.soyuz.interfaces.publishing import name_priority_map
719- if self.explicit_ids_specified:
720- self.displayUsage('Cannot Override BinaryPackage retrieved by ID')
721-
722- component = None
723- section = None
724- priority = None
725- try:
726- if self.component_name:
727- component = getUtility(IComponentSet)[self.component_name]
728- if self.section_name:
729- section = getUtility(ISectionSet)[self.section_name]
730- if self.priority_name:
731- priority = name_priority_map[self.priority_name]
732- except (NotFoundError, KeyError) as info:
733- raise QueueActionError('Not Found: %s' % info)
734-
735- overridden = []
736- for queue_item in self.items:
737- for build in queue_item.builds:
738- # Different than PackageUploadSources
739- # PackageUploadBuild points to a Build, that can,
740- # and usually does, point to multiple BinaryPackageReleases.
741- # So we need to carefully select the requested package to be
742- # overridden
743- for binary in build.build.binarypackages:
744- if binary.name in self.package_names:
745- overridden.append(binary.name)
746- self.display("Overriding %s_%s (%s/%s/%s)"
747- % (binary.name, binary.version,
748- binary.component.name,
749- binary.section.name,
750- binary.priority.name))
751- binary.override(component=component, section=section,
752- priority=priority)
753- self.overrides_performed += 1
754- self.displayInfo(queue_item, only=binary.name)
755- # See if the new component requires a new archive on the
756- # build:
757- if component:
758- distroarchseries = build.build.distro_arch_series
759- distribution = distroarchseries.distroseries.distribution
760- new_archive = distribution.getArchiveByComponent(
761- self.component_name)
762- if (new_archive != build.build.archive):
763- raise QueueActionError(
764- "Overriding component to '%s' failed because it "
765- "would require a new archive."
766- % self.component_name)
767-
768- not_overridden = set(self.package_names) - set(overridden)
769- if len(not_overridden) > 0:
770- self.displayUsage('No matches for %s' % ",".join(not_overridden))
771-
772-
773-queue_actions = {
774- 'help': QueueActionHelp,
775- 'info': QueueActionInfo,
776- 'fetch': QueueActionFetch,
777- 'accept': QueueActionAccept,
778- 'reject': QueueActionReject,
779- 'override': QueueActionOverride,
780- 'report': QueueActionReport,
781- }
782-
783-
784-def default_display(text):
785- """Unified presentation method."""
786- print text
787-
788-
789-class CommandRunnerError(Exception):
790- """Command Runner Failure"""
791-
792-
793-class CommandRunner:
794- """A wrapper for queue_action classes."""
795-
796- def __init__(self, queue, distribution_name, suite_name,
797- no_mail, component_name, section_name, priority_name,
798- display=default_display, log=None):
799- self.queue = queue
800- self.distribution_name = distribution_name
801- self.suite_name = suite_name
802- self.no_mail = no_mail
803- self.component_name = component_name
804- self.section_name = section_name
805- self.priority_name = priority_name
806- self.display = display
807- self.log = log
808-
809- def execute(self, terms, exact_match=False):
810- """Execute a single queue action."""
811- self.display('Running: "%s"' % " ".join(terms))
812-
813- # check syntax, abort process if anything gets wrong
814- try:
815- action = terms[0]
816- arguments = [unicode(term) for term in terms[1:]]
817- except IndexError:
818- raise CommandRunnerError('Invalid sentence, use help.')
819-
820- # check action availability,
821- try:
822- queue_action_class = queue_actions[action]
823- except KeyError:
824- raise CommandRunnerError('Unknown Action: %s' % action)
825-
826- # perform the required action on queue.
827- try:
828- # be sure to send every args via kargs
829- queue_action = queue_action_class(
830- distribution_name=self.distribution_name,
831- suite_name=self.suite_name,
832- queue=self.queue,
833- no_mail=self.no_mail,
834- display=self.display,
835- terms=arguments,
836- component_name=self.component_name,
837- section_name=self.section_name,
838- priority_name=self.priority_name,
839- exact_match=exact_match,
840- log=self.log)
841- queue_action.initialize()
842- queue_action.run()
843- except QueueActionError as info:
844- raise CommandRunnerError(info)
845-
846- return queue_action
847
848=== modified file 'lib/lp/soyuz/scripts/tests/test_processaccepted.py'
849--- lib/lp/soyuz/scripts/tests/test_processaccepted.py 2012-09-19 23:38:41 +0000
850+++ lib/lp/soyuz/scripts/tests/test_processaccepted.py 2012-09-28 10:09:21 +0000
851@@ -3,18 +3,32 @@
852
853 __metaclass__ = type
854
855+from StringIO import StringIO
856 from textwrap import dedent
857
858+from zope.component import getUtility
859+from zope.security.interfaces import ForbiddenAttribute
860 from zope.security.proxy import removeSecurityProxy
861
862+from lp.app.enums import InformationType
863 from lp.bugs.interfaces.bugtask import BugTaskStatus
864 from lp.registry.interfaces.pocket import PackagePublishingPocket
865+from lp.soyuz.interfaces.processacceptedbugsjob import (
866+ IProcessAcceptedBugsJobSource,
867+ )
868 from lp.soyuz.scripts.processaccepted import (
869 close_bugs_for_sourcepackagerelease,
870 close_bugs_for_sourcepublication,
871 )
872-from lp.testing import TestCaseWithFactory
873-from lp.testing.layers import LaunchpadZopelessLayer
874+from lp.testing import (
875+ celebrity_logged_in,
876+ person_logged_in,
877+ TestCaseWithFactory,
878+ )
879+from lp.testing.layers import (
880+ DatabaseFunctionalLayer,
881+ LaunchpadZopelessLayer,
882+ )
883
884
885 class TestClosingBugs(TestCaseWithFactory):
886@@ -24,7 +38,6 @@
887 start a unification in a single file and those other tests need
888 migrating here.
889 See also:
890- * lp/soyuz/scripts/tests/test_queue.py
891 * lib/lp/soyuz/doc/closing-bugs-from-changelogs.txt
892 * lib/lp/archiveuploader/tests/nascentupload-closing-bugs.txt
893 """
894@@ -127,3 +140,39 @@
895
896 for bug, bugtask in bugs:
897 self.assertEqual(BugTaskStatus.FIXRELEASED, bugtask.status)
898+
899+
900+class TestClosingPrivateBugs(TestCaseWithFactory):
901+ # The distroseries +queue page can close private bugs when accepting
902+ # packages.
903+
904+ layer = DatabaseFunctionalLayer
905+
906+ def test_close_bugs_for_sourcepackagerelease_with_private_bug(self):
907+ """close_bugs_for_sourcepackagerelease works with private bugs."""
908+ changes_file_template = "Format: 1.7\nLaunchpad-bugs-fixed: %s\n"
909+ # changelog_entry is required for an assertion inside the function
910+ # we're testing.
911+ spr = self.factory.makeSourcePackageRelease(changelog_entry="blah")
912+ archive_admin = self.factory.makePerson()
913+ series = spr.upload_distroseries
914+ dsp = series.distribution.getSourcePackage(spr.sourcepackagename)
915+ bug = self.factory.makeBug(
916+ target=dsp, information_type=InformationType.USERDATA)
917+ changes = StringIO(changes_file_template % bug.id)
918+
919+ with person_logged_in(archive_admin):
920+ # The archive admin user can't normally see this bug.
921+ self.assertRaises(ForbiddenAttribute, bug, 'status')
922+ # But the bug closure should work.
923+ close_bugs_for_sourcepackagerelease(series, spr, changes)
924+
925+ # Rather than closing the bugs immediately, this creates a
926+ # ProcessAcceptedBugsJob.
927+ with celebrity_logged_in("admin"):
928+ self.assertEqual(BugTaskStatus.NEW, bug.default_bugtask.status)
929+ job_source = getUtility(IProcessAcceptedBugsJobSource)
930+ [job] = list(job_source.iterReady())
931+ self.assertEqual(series, job.distroseries)
932+ self.assertEqual(spr, job.sourcepackagerelease)
933+ self.assertEqual([bug.id], job.bug_ids)
934
935=== removed file 'lib/lp/soyuz/scripts/tests/test_queue.py'
936--- lib/lp/soyuz/scripts/tests/test_queue.py 2012-09-24 20:37:00 +0000
937+++ lib/lp/soyuz/scripts/tests/test_queue.py 1970-01-01 00:00:00 +0000
938@@ -1,1270 +0,0 @@
939-# Copyright 2009-2012 Canonical Ltd. This software is licensed under the
940-# GNU Affero General Public License version 3 (see the file LICENSE).
941-
942-"""queue tool base class tests."""
943-
944-__metaclass__ = type
945-
946-import hashlib
947-import os
948-import shutil
949-from StringIO import StringIO
950-import tempfile
951-from unittest import TestCase
952-
953-from testtools.matchers import StartsWith
954-from zope.component import getUtility
955-from zope.security.interfaces import ForbiddenAttribute
956-from zope.security.proxy import removeSecurityProxy
957-
958-from lp.app.enums import InformationType
959-from lp.archiveuploader.nascentupload import NascentUpload
960-from lp.archiveuploader.tests import (
961- datadir,
962- getPolicy,
963- insertFakeChangesFileForAllPackageUploads,
964- )
965-from lp.bugs.interfaces.bug import IBugSet
966-from lp.bugs.interfaces.bugtask import (
967- BugTaskStatus,
968- IBugTaskSet,
969- )
970-from lp.registry.interfaces.distribution import IDistributionSet
971-from lp.registry.interfaces.person import IPersonSet
972-from lp.registry.interfaces.pocket import PackagePublishingPocket
973-from lp.registry.interfaces.series import SeriesStatus
974-from lp.services.config import config
975-from lp.services.database.lpstorm import IStore
976-from lp.services.librarian.interfaces import ILibraryFileAliasSet
977-from lp.services.librarian.model import LibraryFileAlias
978-from lp.services.librarian.utils import filechunks
979-from lp.services.librarianserver.testing.server import fillLibrarianFile
980-from lp.services.log.logger import DevNullLogger
981-from lp.services.mail import stub
982-from lp.soyuz.enums import (
983- ArchivePurpose,
984- PackagePublishingStatus,
985- PackageUploadStatus,
986- )
987-from lp.soyuz.interfaces.archive import IArchiveSet
988-from lp.soyuz.interfaces.processacceptedbugsjob import (
989- IProcessAcceptedBugsJobSource,
990- )
991-from lp.soyuz.interfaces.queue import IPackageUploadSet
992-from lp.soyuz.model.queue import PackageUploadBuild
993-from lp.soyuz.scripts.processaccepted import (
994- close_bugs_for_sourcepackagerelease,
995- )
996-from lp.soyuz.scripts.queue import (
997- CommandRunner,
998- CommandRunnerError,
999- name_queue_map,
1000- QueueAction,
1001- QueueActionOverride,
1002- )
1003-from lp.testing import (
1004- celebrity_logged_in,
1005- person_logged_in,
1006- TestCaseWithFactory,
1007- )
1008-from lp.testing.dbuser import (
1009- dbuser,
1010- lp_dbuser,
1011- switch_dbuser,
1012- )
1013-from lp.testing.fakemethod import FakeMethod
1014-from lp.testing.layers import (
1015- DatabaseFunctionalLayer,
1016- LaunchpadZopelessLayer,
1017- LibrarianLayer,
1018- )
1019-
1020-
1021-class TestQueueBase:
1022- """Base methods for queue tool test classes."""
1023-
1024- def setUp(self):
1025- # Switch database user and set isolation level to READ COMMIITTED
1026- # to avoid SERIALIZATION exceptions with the Librarian.
1027- switch_dbuser(self.dbuser)
1028-
1029- def _test_display(self, text):
1030- """Store output from queue tool for inspection."""
1031- self.test_output.append(text)
1032-
1033- def execute_command(self, argument, queue_name='new', no_mail=True,
1034- distribution_name='ubuntu', component_name=None,
1035- section_name=None, priority_name=None,
1036- suite_name='breezy-autotest', quiet=True):
1037- """Helper method to execute a queue command.
1038-
1039- Initialize output buffer and execute a command according
1040- given argument.
1041-
1042- Return the used QueueAction instance.
1043- """
1044- self.test_output = []
1045- queue = name_queue_map[queue_name]
1046- runner = CommandRunner(
1047- queue, distribution_name, suite_name, no_mail,
1048- component_name, section_name, priority_name,
1049- display=self._test_display)
1050-
1051- return runner.execute(argument.split())
1052-
1053- def assertEmail(self, expected_to_addrs):
1054- """Pop an email from the stub queue and check its recipients."""
1055- from_addr, to_addrs, raw_msg = stub.test_emails.pop()
1056- self.assertEqual(to_addrs, expected_to_addrs)
1057-
1058-
1059-class TestQueueTool(TestQueueBase, TestCase):
1060- layer = LaunchpadZopelessLayer
1061- dbuser = config.uploadqueue.dbuser
1062-
1063- def setUp(self):
1064- """Create contents in disk for librarian sampledata."""
1065- # Packageupload.notify() needs real changes file data to send
1066- # email, so this nice simple "ed" changes file will do. It's
1067- # the /wrong/ changes file for the package in the upload queue,
1068- # but that doesn't matter as only email addresses are parsed out
1069- # of it.
1070- insertFakeChangesFileForAllPackageUploads()
1071- fake_chroot = LibraryFileAlias.get(1)
1072-
1073- switch_dbuser("testadmin")
1074-
1075- ubuntu = getUtility(IDistributionSet)['ubuntu']
1076- breezy_autotest = ubuntu.getSeries('breezy-autotest')
1077- breezy_autotest['i386'].addOrUpdateChroot(fake_chroot)
1078-
1079- switch_dbuser('launchpad')
1080-
1081- TestQueueBase.setUp(self)
1082-
1083- def tearDown(self):
1084- """Remove test contents from disk."""
1085- LibrarianLayer.librarian_fixture.clear()
1086-
1087- def uploadPackage(self,
1088- changesfile="suite/bar_1.0-1/bar_1.0-1_source.changes"):
1089- """Helper function to upload a package."""
1090- with dbuser("uploader"):
1091- sync_policy = getPolicy(
1092- name='sync', distro='ubuntu', distroseries='breezy-autotest')
1093- bar_src = NascentUpload.from_changesfile_path(
1094- datadir(changesfile),
1095- sync_policy, DevNullLogger())
1096- bar_src.process()
1097- bar_src.do_accept()
1098- return bar_src
1099-
1100- def testBrokenAction(self):
1101- """Check if an unknown action raises CommandRunnerError."""
1102- self.assertRaises(
1103- CommandRunnerError, self.execute_command, 'foo')
1104-
1105- def testHelpAction(self):
1106- """Check if help is working properly.
1107-
1108- Without arguments 'help' should return the docstring summary of
1109- all available actions.
1110-
1111- Optionally we can pass arguments corresponding to the specific
1112- actions we want to see the help, not available actions will be
1113- reported.
1114- """
1115- self.execute_command('help')
1116- self.assertEqual(
1117- ['Running: "help"',
1118- '\tinfo : Present the Queue item including its contents. ',
1119- '\taccept : Accept the contents of a queue item. ',
1120- '\treport : Present a report about the size of available '
1121- 'queues ',
1122- '\treject : Reject the contents of a queue item. ',
1123- '\toverride : Override information in a queue item content. ',
1124- '\tfetch : Fetch the contents of a queue item. '],
1125- self.test_output)
1126-
1127- self.execute_command('help fetch')
1128- self.assertEqual(
1129- ['Running: "help fetch"',
1130- '\tfetch : Fetch the contents of a queue item. '],
1131- self.test_output)
1132-
1133- self.execute_command('help foo')
1134- self.assertEqual(
1135- ['Running: "help foo"',
1136- 'Not available action(s): foo'],
1137- self.test_output)
1138-
1139- def testInfoAction(self):
1140- """Check INFO queue action without arguments present all items."""
1141- queue_action = self.execute_command('info')
1142- # check if the considered queue size matches the existent number
1143- # of records in sampledata
1144- bat = getUtility(IDistributionSet)['ubuntu']['breezy-autotest']
1145- queue_size = getUtility(IPackageUploadSet).count(
1146- status=PackageUploadStatus.NEW, distroseries=bat,
1147- pocket=PackagePublishingPocket.RELEASE)
1148- self.assertEqual(queue_size, queue_action.size)
1149- # check if none of them was filtered, since not filter term
1150- # was passed.
1151- self.assertEqual(queue_size, queue_action.items_size)
1152-
1153- def testInfoActionDoesNotSupportWildCards(self):
1154- """Check if an wildcard-like filter raises CommandRunnerError."""
1155- self.assertRaises(
1156- CommandRunnerError, self.execute_command, 'info *')
1157-
1158- def testInfoActionByID(self):
1159- """Check INFO queue action filtering by ID.
1160-
1161- It should work as expected in case of existent ID in specified the
1162- location.
1163- Otherwise it raises CommandRunnerError if:
1164- * ID not found
1165- * specified ID doesn't match given suite name
1166- * specified ID doesn't match the queue name
1167- """
1168- queue_action = self.execute_command('info 1')
1169- # Check if only one item was retrieved.
1170- self.assertEqual(1, queue_action.items_size)
1171-
1172- displaynames = [item.displayname for item in queue_action.items]
1173- self.assertEqual(['mozilla-firefox'], displaynames)
1174-
1175- # Check passing multiple IDs.
1176- queue_action = self.execute_command('info 1 3 4')
1177- self.assertEqual(3, queue_action.items_size)
1178- [mozilla, netapplet, alsa] = queue_action.items
1179- self.assertEqual('mozilla-firefox', mozilla.displayname)
1180- self.assertEqual('netapplet', netapplet.displayname)
1181- self.assertEqual('alsa-utils', alsa.displayname)
1182-
1183- # Check not found ID.
1184- self.assertRaises(
1185- CommandRunnerError, self.execute_command, 'info 100')
1186-
1187- # Check looking in the wrong suite.
1188- self.assertRaises(
1189- CommandRunnerError, self.execute_command, 'info 1',
1190- suite_name='breezy-autotest-backports')
1191-
1192- # Check looking in the wrong queue.
1193- self.assertRaises(
1194- CommandRunnerError, self.execute_command, 'info 1',
1195- queue_name='done')
1196-
1197- def testInfoActionByName(self):
1198- """Check INFO queue action filtering by name"""
1199- queue_action = self.execute_command('info pmount')
1200- # check if only one item was retrieved as expected in the current
1201- # sampledata
1202- self.assertEqual(1, queue_action.items_size)
1203-
1204- displaynames = [item.displayname for item in queue_action.items]
1205- self.assertEqual(['pmount'], displaynames)
1206-
1207- # Check looking for multiple names.
1208- queue_action = self.execute_command('info pmount alsa-utils')
1209- self.assertEqual(2, queue_action.items_size)
1210- [pmount, alsa] = queue_action.items
1211- self.assertEqual('pmount', pmount.displayname)
1212- self.assertEqual('alsa-utils', alsa.displayname)
1213-
1214- def testAcceptingSourceGeneratesEmail(self):
1215- """Check if accepting a source package generates an email."""
1216- # We need to upload a new source package to do this because the
1217- # sample data is horribly broken with published sources also in
1218- # the NEW queue. Doing it this way guarantees a nice set of data.
1219- self.uploadPackage()
1220-
1221- # Swallow email generated at the upload stage.
1222- stub.test_emails.pop()
1223-
1224- # Add a chroot to breezy-autotest/i386, so the system can create
1225- # builds for it.
1226- with lp_dbuser():
1227- a_file = getUtility(ILibraryFileAliasSet)[1]
1228- breezy_autotest = getUtility(
1229- IDistributionSet)['ubuntu']['breezy-autotest']
1230- breezy_autotest['i386'].addOrUpdateChroot(a_file)
1231-
1232- queue_action = self.execute_command(
1233- 'accept bar', no_mail=False)
1234- self.assertEqual(1, queue_action.items_size)
1235- self.assertEqual(2, len(stub.test_emails))
1236- # Emails sent are the announcement and the uploader's notification:
1237- self.assertEmail(['autotest_changes@ubuntu.com'])
1238- self.assertEmail(
1239- ['Daniel Silverstone <daniel.silverstone@canonical.com>'])
1240-
1241- def testAcceptingSourceCreateBuilds(self):
1242- """Check if accepting a source package creates build records."""
1243- self.uploadPackage()
1244-
1245- # Swallow email generated at the upload stage.
1246- stub.test_emails.pop()
1247-
1248- # Add a chroot to breezy-autotest/i386, so the system can create
1249- # builds for it.
1250- with lp_dbuser():
1251- a_file = getUtility(ILibraryFileAliasSet)[1]
1252- breezy_autotest = getUtility(
1253- IDistributionSet)['ubuntu']['breezy-autotest']
1254- breezy_autotest['i386'].addOrUpdateChroot(a_file)
1255-
1256- queue_action = self.execute_command(
1257- 'accept bar', no_mail=False)
1258- self.assertEqual(1, queue_action.items_size)
1259- self.assertEqual(2, len(stub.test_emails))
1260-
1261- [queue_item] = queue_action.items
1262- [queue_source] = queue_item.sources
1263- sourcepackagerelease = queue_source.sourcepackagerelease
1264- [build] = sourcepackagerelease.builds
1265- self.assertEqual(
1266- 'i386 build of bar 1.0-1 in ubuntu breezy-autotest RELEASE',
1267- build.title)
1268- self.assertEqual(build.buildqueue_record.lastscore, 1755)
1269-
1270- def testAcceptingBinaryDoesntGenerateEmail(self):
1271- """Check if accepting a binary package does not generate email."""
1272- queue_action = self.execute_command(
1273- 'accept mozilla-firefox', no_mail=False)
1274- self.assertEqual(1, queue_action.items_size)
1275- self.assertEqual(0, len(stub.test_emails))
1276-
1277- def testAcceptingSourceClosesBug(self):
1278- """Check that accepting a source will close bugs appropriately."""
1279- # To speed up the publication process, single source uploads
1280- # are automatically published when they are accepted to avoid
1281- # another publisher cycle's worth of delay. When the source is
1282- # published, any bugs mentioned in the upload must be closed.
1283-
1284- # First we must upload the first version of 'bar' in Ubuntu Hoary.
1285- bar_src = self.uploadPackage()
1286- bar_src.queue_root.setAccepted()
1287- bar_src.queue_root.realiseUpload()
1288-
1289- # Now make a new bugtask for the "bar" package.
1290- with lp_dbuser():
1291- the_bug_id = 6
1292- bugtask_owner = getUtility(IPersonSet).getByName('kinnison')
1293- ubuntu = getUtility(IDistributionSet)['ubuntu']
1294- ubuntu_bar = ubuntu.getSourcePackage('bar')
1295- the_bug = getUtility(IBugSet).get(the_bug_id)
1296- bugtask = getUtility(IBugTaskSet).createTask(
1297- the_bug, bugtask_owner, ubuntu_bar)
1298-
1299- # The bugtask starts life as NEW.
1300- the_bug = getUtility(IBugSet).get(the_bug_id)
1301- bugtask = the_bug.getBugTask(ubuntu_bar)
1302- bug_status = bugtask.status.name
1303- self.assertEqual(
1304- bug_status, 'NEW',
1305- 'Bug status is %s, expected NEW' % bug_status)
1306-
1307- # Now, make an upload for the next version of "bar".
1308- bar2_src = self.uploadPackage(
1309- changesfile="suite/bar_1.0-2/bar_1.0-2_source.changes")
1310-
1311- # Now accept the new bar upload with the queue tool.
1312- self.execute_command('accept bar', no_mail=False)
1313-
1314- # The upload wants to close bug 6:
1315- bugs_fixed_header = bar2_src.changes._dict['Launchpad-bugs-fixed']
1316- self.assertEqual(
1317- bugs_fixed_header, str(the_bug_id),
1318- 'Expected bug %s in Launchpad-bugs-fixed, got %s'
1319- % (the_bug_id, bugs_fixed_header))
1320-
1321- # The upload should be in the DONE state:
1322- item_status = bar2_src.queue_root.status.name
1323- self.assertEqual(
1324- item_status, 'DONE',
1325- 'Upload status is %s, expected DONE' % item_status)
1326-
1327- # The bug should now be marked as fix released for the "bar"
1328- # bugtask:
1329- the_bug = getUtility(IBugSet).get(the_bug_id)
1330- bugtask = the_bug.getBugTask(ubuntu_bar)
1331- bug_status = bugtask.status.name
1332- self.assertEqual(
1333- bug_status, 'FIXRELEASED',
1334- 'Bug status is %s, expected FIXRELEASED')
1335-
1336- # Clean up.
1337- upload_data = datadir('suite/bar_1.0-2')
1338- os.remove(os.path.join(upload_data, 'bar_1.0.orig.tar.gz'))
1339-
1340- def testAcceptActionWithMultipleIDs(self):
1341- """Check if accepting multiple items at once works.
1342-
1343- We can specify multiple items to accept, even mixing IDs and names.
1344- e.g. queue accept alsa-utils 1 3
1345- """
1346- breezy_autotest = getUtility(
1347- IDistributionSet)['ubuntu']['breezy-autotest']
1348- queue_action = self.execute_command('accept 1 pmount 3')
1349-
1350- self.assertEqual(3, queue_action.items_size)
1351-
1352- self.assertQueueLength(1, breezy_autotest,
1353- PackageUploadStatus.ACCEPTED, u'mozilla-firefox')
1354- self.assertQueueLength(1, breezy_autotest,
1355- PackageUploadStatus.ACCEPTED, u'pmount')
1356- # Single-source upload went straight to DONE queue.
1357- self.assertQueueLength(1, breezy_autotest,
1358- PackageUploadStatus.DONE, u'netapplet')
1359-
1360- def testRemovedPublishRecordDoesNotAffectQueueNewness(self):
1361- """Check if REMOVED published record does not affect file NEWness.
1362-
1363- We only mark a file as *known* if there is a PUBLISHED record with
1364- the same name, other states like SUPERSEDED or REMOVED doesn't count.
1365-
1366- This is the case of 'pmount_0.1-1' in ubuntu/breezy-autotest/i386,
1367- there is a REMOVED publishing record for it as you can see in the
1368- first part of the test.
1369-
1370- Following we can see the correct presentation of the new flag ('N').
1371- Bug #59291
1372- """
1373- # inspect publishing history in sampledata for the suspicious binary
1374- # ensure is has a single entry and it is merked as REMOVED.
1375- ubuntu = getUtility(IDistributionSet)['ubuntu']
1376- bat_i386 = ubuntu['breezy-autotest']['i386']
1377- moz_publishing = bat_i386.getBinaryPackage('pmount').releases
1378-
1379- self.assertEqual(1, len(moz_publishing))
1380- self.assertEqual(PackagePublishingStatus.DELETED,
1381- moz_publishing[0].status)
1382-
1383- # invoke queue tool filtering by name
1384- queue_action = self.execute_command('info pmount')
1385-
1386- # ensure we retrived a single item
1387- self.assertEqual(1, queue_action.items_size)
1388-
1389- # and it is what we expect
1390- self.assertEqual('pmount', queue_action.items[0].displayname)
1391- self.assertEqual(moz_publishing[0].binarypackagerelease.build,
1392- queue_action.items[0].builds[0].build)
1393- # inspect output, note the presence of 'N' flag
1394- self.assertTrue(
1395- '| N pmount/0.1-1/i386' in '\n'.join(self.test_output))
1396-
1397- def testQueueSupportForSuiteNames(self):
1398- """Queue tool supports suite names properly.
1399-
1400- Two UNAPPROVED items are present for pocket RELEASE and only
1401- one for pocket UPDATES in breezy-autotest.
1402- Bug #59280
1403- """
1404- queue_action = self.execute_command(
1405- 'info', queue_name='unapproved',
1406- suite_name='breezy-autotest')
1407-
1408- self.assertEqual(2, queue_action.items_size)
1409- self.assertEqual(PackagePublishingPocket.RELEASE, queue_action.pocket)
1410-
1411- queue_action = self.execute_command(
1412- 'info', queue_name='unapproved',
1413- suite_name='breezy-autotest-updates')
1414-
1415- self.assertEqual(1, queue_action.items_size)
1416- self.assertEqual(PackagePublishingPocket.UPDATES, queue_action.pocket)
1417-
1418- def testQueueDoesNotAnnounceBackports(self):
1419- """Check if BACKPORTS acceptance are not announced publicly.
1420-
1421- Queue tool normally announce acceptance in the specified changeslist
1422- for the distroseries in question, however BACKPORTS announce doesn't
1423- fit very well in that list, they cause unwanted noise.
1424-
1425- Further details in bug #59443
1426- """
1427- with lp_dbuser():
1428- # Make breezy-autotest CURRENT in order to accept upload
1429- # to BACKPORTS.
1430- breezy_autotest = getUtility(
1431- IDistributionSet)['ubuntu']['breezy-autotest']
1432- breezy_autotest.status = SeriesStatus.CURRENT
1433-
1434- # Store the targeted queue item for future inspection.
1435- # Ensure it is what we expect.
1436- target_queue = breezy_autotest.getPackageUploads(
1437- status=PackageUploadStatus.UNAPPROVED,
1438- pocket=PackagePublishingPocket.BACKPORTS)[0]
1439- self.assertEqual(10, target_queue.id)
1440-
1441- # Ensure breezy-autotest is set.
1442- self.assertEqual(
1443- u'autotest_changes@ubuntu.com', breezy_autotest.changeslist)
1444-
1445- # Accept the sampledata item.
1446- queue_action = self.execute_command(
1447- 'accept', queue_name='unapproved',
1448- suite_name='breezy-autotest-backports', no_mail=False)
1449-
1450- # Only one item considered.
1451- self.assertEqual(1, queue_action.items_size)
1452-
1453- # Previously stored reference should have new state now
1454- self.assertEqual('ACCEPTED', target_queue.status.name)
1455-
1456- # Only one email is sent to the changed-by email on the changes
1457- # file. No announcement email is sent.
1458- self.assertEqual(len(stub.test_emails), 1)
1459- self.assertEmail(
1460- ['Daniel Silverstone <daniel.silverstone@canonical.com>'])
1461-
1462- def testQueueDoesNotSendAnyEmailsForTranslations(self):
1463- """Check if no emails are sent when accepting translations.
1464-
1465- Queue tool should not send any emails to source uploads targeted to
1466- 'translation' section.
1467- They are the 'language-pack-*' and 'language-support-*' sources.
1468-
1469- Further details in bug #57708
1470- """
1471- with lp_dbuser():
1472- # Make breezy-autotest CURRENT in order to accept upload
1473- # to PROPOSED.
1474- breezy_autotest = getUtility(
1475- IDistributionSet)['ubuntu']['breezy-autotest']
1476- breezy_autotest.status = SeriesStatus.CURRENT
1477-
1478- # Store the targeted queue item for future inspection.
1479- # Ensure it is what we expect.
1480- target_queue = breezy_autotest.getPackageUploads(
1481- status=PackageUploadStatus.UNAPPROVED,
1482- pocket=PackagePublishingPocket.PROPOSED)[0]
1483- self.assertEqual(12, target_queue.id)
1484- source = target_queue.sources[0].sourcepackagerelease
1485- self.assertEqual('translations', source.section.name)
1486-
1487- # Accept the sampledata item.
1488- queue_action = self.execute_command(
1489- 'accept', queue_name='unapproved',
1490- suite_name='breezy-autotest-proposed', no_mail=False)
1491-
1492- # Only one item considered.
1493- self.assertEqual(1, queue_action.items_size)
1494-
1495- # Previously stored reference should have new state now.
1496- self.assertEqual('DONE', target_queue.status.name)
1497-
1498- # No email was sent.
1499- self.assertEqual(0, len(stub.test_emails))
1500-
1501- def assertQueueLength(self, expected_length, distro_series, status, name):
1502- queue_items = distro_series.getPackageUploads(
1503- status=status, name=name)
1504- self.assertEqual(expected_length, queue_items.count())
1505-
1506- def assertErrorAcceptingDuplicate(self):
1507- self.assertTrue(
1508- '** cnews could not be accepted due to '
1509- 'The source cnews - 1.0 is already accepted in ubuntu/'
1510- 'breezy-autotest and you cannot upload the same version '
1511- 'within the same distribution. You have to modify the source '
1512- 'version and re-upload.' in self.test_output)
1513-
1514- def testAcceptanceWorkflowForDuplications(self):
1515- """Check how queue tool behaves dealing with duplicated entries.
1516-
1517- Sampledata provides a duplication of cnews_1.0 in breezy-autotest
1518- UNAPPROVED queue.
1519-
1520- Step 1: executing 'accept cnews in unapproved queue' with duplicate
1521- cnews items in the UNAPPROVED queue, results in the oldest being
1522- accepted and the newer one remaining UNAPPROVED (and displaying
1523- an error about it to the user).
1524-
1525- Step 2: executing 'accept cnews in unapproved queue' with duplicate
1526- cnews items in the UNAPPROVED and ACCEPTED queues has no effect on
1527- the queues, and again displays an error to the user.
1528-
1529- Step 3: executing 'accept cnews in unapproved queue' with duplicate
1530- cnews items in the UNAPPROVED and DONE queues behaves the same as 2.
1531-
1532- Step 4: the remaining duplicated cnews item in UNAPPROVED queue can
1533- only be rejected.
1534- """
1535- with lp_dbuser():
1536- # Add a chroot to breezy-autotest/i386, so the system can create
1537- # builds for it.
1538- a_file = getUtility(ILibraryFileAliasSet)[1]
1539- breezy_autotest = getUtility(
1540- IDistributionSet)['ubuntu']['breezy-autotest']
1541- breezy_autotest['i386'].addOrUpdateChroot(a_file)
1542-
1543- # Certify we have a 'cnews' upload duplication in UNAPPROVED.
1544- self.assertQueueLength(
1545- 2, breezy_autotest, PackageUploadStatus.UNAPPROVED, u"cnews")
1546-
1547- # Step 1: try to accept both.
1548- self.execute_command(
1549- 'accept cnews', queue_name='unapproved',
1550- suite_name='breezy-autotest')
1551-
1552- # The first item, being a single source upload, is automatically
1553- # published when it's accepted.
1554- self.assertQueueLength(
1555- 1, breezy_autotest, PackageUploadStatus.DONE, u"cnews")
1556-
1557- # The last can't be accepted and remains in UNAPPROVED.
1558- self.assertErrorAcceptingDuplicate()
1559- self.assertQueueLength(
1560- 1, breezy_autotest, PackageUploadStatus.UNAPPROVED, u"cnews")
1561-
1562- # Step 2: try to accept the remaining item in UNAPPROVED.
1563- self.execute_command(
1564- 'accept cnews', queue_name='unapproved',
1565- suite_name='breezy-autotest')
1566- self.assertErrorAcceptingDuplicate()
1567- self.assertQueueLength(
1568- 1, breezy_autotest, PackageUploadStatus.UNAPPROVED, u"cnews")
1569-
1570- # Step 3: try to accept the remaining item in UNAPPROVED with the
1571- # duplication already in DONE.
1572- self.execute_command(
1573- 'accept cnews', queue_name='unapproved',
1574- suite_name='breezy-autotest')
1575- # It failed and te item remains in UNAPPROVED.
1576- self.assertErrorAcceptingDuplicate()
1577- self.assertQueueLength(
1578- 1, breezy_autotest, PackageUploadStatus.UNAPPROVED, u"cnews")
1579-
1580- # Step 4: The only possible destiny for the remaining item it REJECT.
1581- self.execute_command(
1582- 'reject cnews', queue_name='unapproved',
1583- suite_name='breezy-autotest')
1584- self.assertQueueLength(
1585- 0, breezy_autotest, PackageUploadStatus.UNAPPROVED, u"cnews")
1586- self.assertQueueLength(
1587- 1, breezy_autotest, PackageUploadStatus.REJECTED, u"cnews")
1588-
1589- def testRejectSourceSendsEmail(self):
1590- """Check that rejecting a source upload sends email."""
1591- queue_action = self.execute_command(
1592- 'reject alsa-utils', no_mail=False)
1593- self.assertEqual(1, queue_action.items_size)
1594- self.assertEqual(1, len(stub.test_emails))
1595- self.assertEmail(
1596- ['Daniel Silverstone <daniel.silverstone@canonical.com>'])
1597-
1598- def testRejectBinarySendsEmail(self):
1599- """Check that rejecting a binary upload sends email."""
1600- queue_action = self.execute_command('reject 2', no_mail=False)
1601- self.assertEqual(1, queue_action.items_size)
1602- self.assertEqual(1, len(stub.test_emails))
1603- self.assertEmail(
1604- ['Daniel Silverstone <daniel.silverstone@canonical.com>'])
1605-
1606- def testRejectLangpackSendsNoEmail(self):
1607- """Check that rejecting a language pack sends no email."""
1608- queue_action = self.execute_command(
1609- 'reject language-pack-de', queue_name='unapproved',
1610- suite_name='breezy-autotest-proposed')
1611- self.assertEqual(1, queue_action.items_size)
1612- self.assertEqual(0, len(stub.test_emails))
1613-
1614- def testRejectWithMultipleIDs(self):
1615- """Check if rejecting multiple items at once works.
1616-
1617- We can specify multiple items to reject, even mixing IDs and names.
1618- e.g. queue reject alsa-utils 1 3
1619- """
1620- breezy_autotest = getUtility(
1621- IDistributionSet)['ubuntu']['breezy-autotest']
1622-
1623- # Run the command.
1624- queue_action = self.execute_command('reject 1 pmount 3')
1625-
1626- # Test what it did. Since all the queue items came out of the
1627- # NEW queue originally, the items processed should now be REJECTED.
1628- self.assertEqual(3, queue_action.items_size)
1629- self.assertQueueLength(1, breezy_autotest,
1630- PackageUploadStatus.REJECTED, u'mozilla-firefox')
1631- self.assertQueueLength(1, breezy_autotest,
1632- PackageUploadStatus.REJECTED, u'pmount')
1633- self.assertQueueLength(1, breezy_autotest,
1634- PackageUploadStatus.REJECTED, u'netapplet')
1635-
1636- def testOverrideSource(self):
1637- """Check if overriding sources works.
1638-
1639- We can specify multiple items to override, even mixing IDs and names.
1640- e.g. queue override source -c restricted alsa-utils 1 3
1641- """
1642- # Set up.
1643- breezy_autotest = getUtility(
1644- IDistributionSet)['ubuntu']['breezy-autotest']
1645-
1646- # Basic operation overriding a single source 'alsa-utils' that
1647- # is currently main/base in the sample data.
1648- queue_action = self.execute_command('override source 4',
1649- component_name='restricted', section_name='web')
1650- self.assertEqual(1, queue_action.items_size)
1651- queue_item = breezy_autotest.getPackageUploads(
1652- status=PackageUploadStatus.NEW, name=u"alsa-utils")[0]
1653- [source] = queue_item.sources
1654- self.assertEqual('restricted',
1655- source.sourcepackagerelease.component.name)
1656- self.assertEqual('web',
1657- source.sourcepackagerelease.section.name)
1658-
1659- # Override multiple sources at once and mix ID with name.
1660- queue_action = self.execute_command('override source 4 netapplet',
1661- component_name='universe', section_name='editors')
1662- # 'netapplet' appears 3 times, alsa-utils once.
1663- self.assertEqual(4, queue_action.items_size)
1664- self.assertEqual(2, queue_action.overrides_performed)
1665- # Check results.
1666- queue_items = list(breezy_autotest.getPackageUploads(
1667- status=PackageUploadStatus.NEW, name=u'alsa-utils'))
1668- queue_items.extend(list(breezy_autotest.getPackageUploads(
1669- status=PackageUploadStatus.NEW, name=u'netapplet')))
1670- for queue_item in queue_items:
1671- if queue_item.sources:
1672- [source] = queue_item.sources
1673- self.assertEqual('universe',
1674- source.sourcepackagerelease.component.name)
1675- self.assertEqual('editors',
1676- source.sourcepackagerelease.section.name)
1677-
1678- def testOverrideSourceWithArchiveChange(self):
1679- """Check if the archive changes as necessary on a source override.
1680-
1681- When overriding the component, the archive may change, so we check
1682- that here.
1683- """
1684- # Set up.
1685- ubuntu = getUtility(IDistributionSet)['ubuntu']
1686- breezy_autotest = ubuntu['breezy-autotest']
1687-
1688- # Test that it changes to partner when required.
1689- queue_action = self.execute_command('override source alsa-utils',
1690- component_name='partner')
1691- self.assertEqual(1, queue_action.items_size)
1692- [queue_item] = breezy_autotest.getPackageUploads(
1693- status=PackageUploadStatus.NEW, name=u"alsa-utils")
1694- [source] = queue_item.sources
1695- self.assertEqual(source.sourcepackagerelease.upload_archive.purpose,
1696- ArchivePurpose.PARTNER)
1697-
1698- # Test that it changes back to primary when required.
1699- queue_action = self.execute_command('override source alsa-utils',
1700- component_name='main')
1701- self.assertEqual(1, queue_action.items_size)
1702- [queue_item] = breezy_autotest.getPackageUploads(
1703- status=PackageUploadStatus.NEW, name=u"alsa-utils")
1704- [source] = queue_item.sources
1705- self.assertEqual(source.sourcepackagerelease.upload_archive.purpose,
1706- ArchivePurpose.PRIMARY)
1707-
1708- def testOverrideSourceWithNonexistentArchiveChange(self):
1709- """Check that overriding to a non-existent archive fails properly.
1710-
1711- When overriding the component, the archive may change to a
1712- non-existent one so ensure if fails.
1713- """
1714- with lp_dbuser():
1715- ubuntu = getUtility(IDistributionSet)['ubuntu']
1716- proxied_archive = getUtility(IArchiveSet).getByDistroPurpose(
1717- ubuntu, ArchivePurpose.PARTNER)
1718- comm_archive = removeSecurityProxy(proxied_archive)
1719- comm_archive.purpose = ArchivePurpose.PPA
1720-
1721- self.assertRaises(CommandRunnerError,
1722- self.execute_command,
1723- 'override source alsa-utils',
1724- component_name='partner')
1725-
1726- def testOverrideBinary(self):
1727- """Check if overriding binaries works.
1728-
1729- We can specify multiple items to override, even mixing IDs and names.
1730- e.g. queue override binary -c restricted alsa-utils 1 3
1731- """
1732- # Set up.
1733- breezy_autotest = getUtility(
1734- IDistributionSet)['ubuntu']['breezy-autotest']
1735-
1736- # Override a binary, 'pmount', from its sample data of
1737- # main/base/IMPORTANT to restricted/web/extra.
1738- queue_action = self.execute_command('override binary pmount',
1739- component_name='restricted', section_name='web',
1740- priority_name='extra')
1741- self.assertEqual(1, queue_action.items_size)
1742- [queue_item] = breezy_autotest.getPackageUploads(
1743- status=PackageUploadStatus.NEW, name=u"pmount")
1744- [packagebuild] = queue_item.builds
1745- for package in packagebuild.build.binarypackages:
1746- self.assertEqual('restricted', package.component.name)
1747- self.assertEqual('web', package.section.name)
1748- self.assertEqual('EXTRA', package.priority.name)
1749-
1750- # Override multiple binaries at once.
1751- queue_action = self.execute_command(
1752- 'override binary pmount mozilla-firefox',
1753- component_name='universe', section_name='editors',
1754- priority_name='optional')
1755- # Check results.
1756- self.assertEqual(2, queue_action.items_size)
1757- self.assertEqual(2, queue_action.overrides_performed)
1758- queue_items = list(breezy_autotest.getPackageUploads(
1759- status=PackageUploadStatus.NEW, name=u'pmount'))
1760- queue_items.extend(list(breezy_autotest.getPackageUploads(
1761- status=PackageUploadStatus.NEW, name=u'mozilla-firefox')))
1762- for queue_item in queue_items:
1763- [packagebuild] = queue_item.builds
1764- for package in packagebuild.build.binarypackages:
1765- self.assertEqual('universe', package.component.name)
1766- self.assertEqual('editors', package.section.name)
1767- self.assertEqual('OPTIONAL', package.priority.name)
1768-
1769- # Check that overriding by ID is warned to the user.
1770- self.assertRaises(
1771- CommandRunnerError, self.execute_command, 'override binary 1',
1772- component_name='multiverse')
1773-
1774- def testOverridingMulipleBinariesFromSameBuild(self):
1775- """Check that multiple binary override works for the same build.
1776-
1777- Overriding binary packages generated from the same build should
1778- override each package individually.
1779- """
1780- # Start off by setting up a packageuploadbuild that points to
1781- # a build with two binaries.
1782- switch_dbuser("launchpad")
1783-
1784- breezy_autotest = getUtility(
1785- IDistributionSet)['ubuntu']['breezy-autotest']
1786- [mozilla_queue_item] = breezy_autotest.getPackageUploads(
1787- status=PackageUploadStatus.NEW, name=u'mozilla-firefox')
1788-
1789- # The build with ID '2' is for mozilla-firefox, which produces
1790- # binaries for 'mozilla-firefox' and 'mozilla-firefox-data'.
1791- PackageUploadBuild(packageupload=mozilla_queue_item, build=2)
1792-
1793- # Switching db users starts a new transaction. We must re-fetch
1794- # breezy-autotest.
1795- switch_dbuser("queued")
1796- breezy_autotest = getUtility(
1797- IDistributionSet)['ubuntu']['breezy-autotest']
1798-
1799- queue_action = self.execute_command(
1800- 'override binary mozilla-firefox-data mozilla-firefox',
1801- component_name='restricted', section_name='editors',
1802- priority_name='optional')
1803-
1804- # There are three binaries to override on this PackageUpload:
1805- # - mozilla-firefox in breezy-autotest
1806- # - mozilla-firefox and mozilla-firefox-data in warty
1807- # Each should be overridden exactly once.
1808- self.assertEqual(1, queue_action.items_size)
1809- self.assertEqual(3, queue_action.overrides_performed)
1810-
1811- queue_items = list(breezy_autotest.getPackageUploads(
1812- status=PackageUploadStatus.NEW, name=u'mozilla-firefox-data'))
1813- queue_items.extend(list(breezy_autotest.getPackageUploads(
1814- status=PackageUploadStatus.NEW, name=u'mozilla-firefox')))
1815- for queue_item in queue_items:
1816- for packagebuild in queue_item.builds:
1817- for package in packagebuild.build.binarypackages:
1818- self.assertEqual(
1819- 'restricted', package.component.name,
1820- "The component '%s' is not the expected 'restricted'"
1821- "for package %s" % (
1822- package.component.name, package.name))
1823- self.assertEqual(
1824- 'editors', package.section.name,
1825- "The section '%s' is not the expected 'editors'"
1826- "for package %s" % (
1827- package.section.name, package.name))
1828- self.assertEqual(
1829- 'OPTIONAL', package.priority.name,
1830- "The priority '%s' is not the expected 'OPTIONAL'"
1831- "for package %s" % (
1832- package.section.name, package.name))
1833-
1834- def testOverrideBinaryWithArchiveChange(self):
1835- """Check if archive changes are disallowed for binary overrides.
1836-
1837- When overriding the component, the archive may change, so we check
1838- that here and make sure it's disallowed.
1839- """
1840- # Test that it changes to partner when required.
1841- self.assertRaises(
1842- CommandRunnerError, self.execute_command,
1843- 'override binary pmount', component_name='partner')
1844-
1845-
1846-class TestQueueActionLite(TestCaseWithFactory):
1847- """A lightweight unit test case for `QueueAction`.
1848-
1849- Meant for detailed tests that would be too expensive for full end-to-end
1850- tests.
1851- """
1852-
1853- layer = LaunchpadZopelessLayer
1854-
1855- def makeQueueAction(self, package_upload, distroseries=None,
1856- component=None, section=None,
1857- action_type=QueueAction):
1858- """Create a `QueueAction` for use with a `PackageUpload`.
1859-
1860- The action's `display` method is set to a `FakeMethod`.
1861- """
1862- if distroseries is None:
1863- distroseries = self.factory.makeDistroSeries(
1864- status=SeriesStatus.CURRENT,
1865- name="distroseriestestingpcjs")
1866- distro = distroseries.distribution
1867- if package_upload is None:
1868- package_upload = self.factory.makePackageUpload(
1869- distroseries=distroseries, archive=distro.main_archive)
1870- if component is None:
1871- component = self.factory.makeComponent()
1872- if section is None:
1873- section = self.factory.makeSection()
1874- queue = PackageUploadStatus.NEW
1875- priority_name = "STANDARD"
1876- display = FakeMethod()
1877- terms = ['*']
1878- return action_type(
1879- distro.name, distroseries.name, queue, terms, component.name,
1880- section.name, priority_name, display)
1881-
1882- def makeQueueActionOverride(self, package_upload, component, section,
1883- distroseries=None):
1884- return self.makeQueueAction(
1885- package_upload, distroseries, component, section,
1886- action_type=QueueActionOverride)
1887-
1888- def parseUploadSummaryLine(self, output_line):
1889- """Parse an output line from `QueueAction.displayItem`.
1890-
1891- :param output_line: A line of output text from `displayItem`.
1892- :return: A tuple of displayed items: (id, tag, name, version, age).
1893- """
1894- return tuple(item.strip() for item in output_line.split('|'))
1895-
1896- def test_display_actions_have_privileges_for_PackageCopyJob(self):
1897- # The methods that display uploads have privileges to work with
1898- # a PackageUpload that has a copy job.
1899- # Bundling tests for multiple operations into one test because
1900- # the database user change requires a costly commit.
1901- upload = self.factory.makeCopyJobPackageUpload()
1902- action = self.makeQueueAction(upload)
1903- switch_dbuser(config.uploadqueue.dbuser)
1904-
1905- action.displayItem(upload)
1906- self.assertNotEqual(0, action.display.call_count)
1907- action.display.calls = []
1908- action.displayInfo(upload)
1909- self.assertNotEqual(0, action.display.call_count)
1910-
1911- def test_accept_actions_have_privileges_for_PackageCopyJob(self):
1912- # The script also has privileges to approve uploads that have
1913- # copy jobs.
1914- distroseries = self.factory.makeDistroSeries(
1915- status=SeriesStatus.CURRENT)
1916- upload = self.factory.makeCopyJobPackageUpload(distroseries)
1917- switch_dbuser(config.uploadqueue.dbuser)
1918- upload.acceptFromQueue(DevNullLogger(), dry_run=True)
1919- # Flush changes to make sure we're not caching any updates that
1920- # the database won't allow. If this passes, we've got the
1921- # privileges.
1922- IStore(upload).flush()
1923-
1924- def test_displayItem_displays_PackageUpload_with_source(self):
1925- # displayItem can display a source package upload.
1926- upload = self.factory.makeSourcePackageUpload()
1927- action = self.makeQueueAction(upload)
1928-
1929- action.displayItem(upload)
1930-
1931- ((output, ), kwargs) = action.display.calls[0]
1932- (upload_id, tag, name, version, age) = self.parseUploadSummaryLine(
1933- output)
1934- self.assertEqual(str(upload.id), upload_id)
1935- self.assertEqual("S-", tag)
1936- self.assertThat(upload.displayname, StartsWith(name))
1937- self.assertThat(upload.package_version, StartsWith(version))
1938-
1939- def test_displayItem_displays_PackageUpload_with_PackageCopyJob(self):
1940- # displayItem can display a copy-job package upload.
1941- upload = self.factory.makeCopyJobPackageUpload()
1942- action = self.makeQueueAction(upload)
1943-
1944- action.displayItem(upload)
1945-
1946- ((output, ), kwargs) = action.display.calls[0]
1947- (upload_id, tag, name, version, age) = self.parseUploadSummaryLine(
1948- output)
1949- self.assertEqual(str(upload.id), upload_id)
1950- self.assertEqual("X-", tag)
1951- self.assertThat(upload.displayname, StartsWith(name))
1952- self.assertThat(upload.package_version, StartsWith(version))
1953-
1954- def test_override_works_with_PackageCopyJob(self):
1955- # "Sync" PackageUploads can be overridden just like sources,
1956- # test that here.
1957- new_component = self.factory.makeComponent()
1958- new_section = self.factory.makeSection()
1959- pocket = PackagePublishingPocket.RELEASE
1960- upload = self.factory.makeCopyJobPackageUpload(target_pocket=pocket)
1961- action = self.makeQueueActionOverride(
1962- upload, new_component, new_section,
1963- distroseries=upload.distroseries)
1964- # Patch this out because it uses data we don't have in the test;
1965- # it's unnecessary anyway.
1966- self.patch(action, "displayTitle", FakeMethod)
1967- action.terms = ["source", str(upload.id)]
1968- switch_dbuser(config.uploadqueue.dbuser)
1969- action.initialize()
1970- action.run()
1971-
1972- # Overriding a sync means putting the overrides in the job itself.
1973- self.assertEqual(
1974- new_component.name, upload.package_copy_job.component_name)
1975- self.assertEqual(
1976- new_section.name, upload.package_copy_job.section_name)
1977-
1978- def test_makeTag_returns_S_for_source_upload(self):
1979- upload = self.factory.makeSourcePackageUpload()
1980- self.assertEqual('S-', self.makeQueueAction(upload)._makeTag(upload))
1981-
1982- def test_makeTag_returns_B_for_binary_upload(self):
1983- upload = self.factory.makeBuildPackageUpload()
1984- self.assertEqual('-B', self.makeQueueAction(upload)._makeTag(upload))
1985-
1986- def test_makeTag_returns_SB_for_mixed_upload(self):
1987- upload = self.factory.makeSourcePackageUpload()
1988- upload.addBuild(self.factory.makeBinaryPackageBuild())
1989- self.assertEqual('SB', self.makeQueueAction(upload)._makeTag(upload))
1990-
1991- def test_makeTag_returns_X_for_copy_job_upload(self):
1992- upload = self.factory.makeCopyJobPackageUpload()
1993- self.assertEqual('X-', self.makeQueueAction(upload)._makeTag(upload))
1994-
1995- def test_makeTag_returns_dashes_for_custom_upload(self):
1996- upload = self.factory.makeCustomPackageUpload()
1997- self.assertEqual('--', self.makeQueueAction(upload)._makeTag(upload))
1998-
1999- def test_displayInfo_displays_PackageUpload_with_source(self):
2000- # displayInfo can display a source package upload.
2001- upload = self.factory.makeSourcePackageUpload()
2002- action = self.makeQueueAction(upload)
2003- action.displayInfo(upload)
2004- self.assertNotEqual(0, action.display.call_count)
2005-
2006- def test_displayInfo_displays_PackageUpload_with_PackageCopyJob(self):
2007- # displayInfo can display a copy-job package upload.
2008- upload = self.factory.makeCopyJobPackageUpload()
2009- action = self.makeQueueAction(upload)
2010- action.displayInfo(upload)
2011- self.assertNotEqual(0, action.display.call_count)
2012-
2013-
2014-class TestQueuePageClosingBugs(TestCaseWithFactory):
2015- # The distroseries +queue page can close bug when accepting
2016- # packages. Unit tests for that belong here.
2017-
2018- layer = DatabaseFunctionalLayer
2019-
2020- def test_close_bugs_for_sourcepackagerelease_with_private_bug(self):
2021- # lp.soyuz.scripts.processaccepted.close_bugs_for_sourcepackagerelease
2022- # should work with private bugs where the person using the queue
2023- # page doesn't have access to it.
2024- changes_file_template = "Format: 1.7\nLaunchpad-bugs-fixed: %s\n"
2025- # changelog_entry is required for an assertion inside the function
2026- # we're testing.
2027- spr = self.factory.makeSourcePackageRelease(changelog_entry="blah")
2028- archive_admin = self.factory.makePerson()
2029- series = spr.upload_distroseries
2030- dsp = series.distribution.getSourcePackage(spr.sourcepackagename)
2031- bug = self.factory.makeBug(
2032- target=dsp, information_type=InformationType.USERDATA)
2033- changes = StringIO(changes_file_template % bug.id)
2034-
2035- with person_logged_in(archive_admin):
2036- # The archive admin user can't normally see this bug.
2037- self.assertRaises(ForbiddenAttribute, bug, 'status')
2038- # But the bug closure should work.
2039- close_bugs_for_sourcepackagerelease(series, spr, changes)
2040-
2041- # Rather than closing the bugs immediately, this creates a
2042- # ProcessAcceptedBugsJob.
2043- with celebrity_logged_in("admin"):
2044- self.assertEqual(BugTaskStatus.NEW, bug.default_bugtask.status)
2045- job_source = getUtility(IProcessAcceptedBugsJobSource)
2046- [job] = list(job_source.iterReady())
2047- self.assertEqual(series, job.distroseries)
2048- self.assertEqual(spr, job.sourcepackagerelease)
2049- self.assertEqual([bug.id], job.bug_ids)
2050-
2051-
2052-class TestQueueToolInJail(TestQueueBase, TestCase):
2053- layer = LaunchpadZopelessLayer
2054- dbuser = config.uploadqueue.dbuser
2055-
2056- def setUp(self):
2057- """Create contents in disk for librarian sampledata.
2058-
2059- Setup and chdir into a temp directory, a jail, where we can
2060- control the file creation properly
2061- """
2062- fillLibrarianFile(1, content='One')
2063- fillLibrarianFile(52, content='Fifty-Two')
2064- self._home = os.path.abspath('')
2065- self._jail = tempfile.mkdtemp()
2066- os.chdir(self._jail)
2067- TestQueueBase.setUp(self)
2068-
2069- def tearDown(self):
2070- """Remove test contents from disk.
2071-
2072- chdir back to the previous path (home) and remove the temp
2073- directory used as jail.
2074- """
2075- os.chdir(self._home)
2076- LibrarianLayer.librarian_fixture.clear()
2077- shutil.rmtree(self._jail)
2078-
2079- def _listfiles(self):
2080- """Return a list of files present in jail."""
2081- return os.listdir(self._jail)
2082-
2083- def _getsha1(self, filename):
2084- """Return a sha1 hex digest of a file"""
2085- file_sha = hashlib.sha1()
2086- opened_file = open(filename, "r")
2087- for chunk in filechunks(opened_file):
2088- file_sha.update(chunk)
2089- opened_file.close()
2090- return file_sha.hexdigest()
2091-
2092- def testFetchActionByIDDoNotOverwriteFilesystem(self):
2093- """Check if queue fetch action doesn't overwrite files.
2094-
2095- Since we allow existence of duplications in NEW and UNAPPROVED
2096- queues, we are able to fetch files from queue items and they'd
2097- get overwritten causing obscure problems.
2098-
2099- Instead of overwrite a file in the working directory queue will
2100- fail, raising a CommandRunnerError.
2101-
2102- bug 67014: Don't complain if files are the same
2103- """
2104- self.execute_command('fetch 1')
2105- self.assertEqual(
2106- ['mozilla-firefox_0.9_i386.changes'], self._listfiles())
2107-
2108- # checksum the existing file
2109- existing_sha1 = self._getsha1(self._listfiles()[0])
2110-
2111- # fetch will NOT raise and not overwrite the file in disk
2112- self.execute_command('fetch 1')
2113-
2114- # checksum file again
2115- new_sha1 = self._getsha1(self._listfiles()[0])
2116-
2117- # Check that the file has not changed (we don't care if it was
2118- # re-written, just that it's not changed)
2119- self.assertEqual(existing_sha1, new_sha1)
2120-
2121- def testFetchActionRaisesErrorIfDifferentFileAlreadyFetched(self):
2122- """Check that fetching a file that has already been fetched
2123- raises an error if they are not the same file. (bug 67014)
2124- """
2125- CLOBBERED = "you're clobbered"
2126-
2127- self.execute_command('fetch 1')
2128- self.assertEqual(
2129- ['mozilla-firefox_0.9_i386.changes'], self._listfiles())
2130-
2131- # clobber the existing file, fetch it again and expect an exception
2132- f = open(self._listfiles()[0], "w")
2133- f.write(CLOBBERED)
2134- f.close()
2135-
2136- self.assertRaises(
2137- CommandRunnerError, self.execute_command, 'fetch 1')
2138-
2139- # make sure the file has not changed
2140- f = open(self._listfiles()[0], "r")
2141- line = f.read()
2142- f.close()
2143-
2144- self.assertEqual(CLOBBERED, line)
2145-
2146- def testFetchActionByNameDoNotOverwriteFilesystem(self):
2147- """Same as testFetchActionByIDDoNotOverwriteFilesystem
2148-
2149- The sampledata provides duplicated 'cnews' entries, filesystem
2150- conflict will happen inside the same batch,
2151-
2152- Queue will fetch the oldest and raise.
2153- """
2154- self.assertRaises(
2155- CommandRunnerError, self.execute_command, 'fetch cnews',
2156- queue_name='unapproved', suite_name='breezy-autotest')
2157-
2158- self.assertEqual(['netapplet-1.0.0.tar.gz'], self._listfiles())
2159-
2160- def testQueueFetch(self):
2161- """Check that a basic fetch operation works."""
2162- FAKE_CHANGESFILE_CONTENT = "Fake Changesfile"
2163- FAKE_DEB_CONTENT = "Fake DEB"
2164- fillLibrarianFile(1, FAKE_CHANGESFILE_CONTENT)
2165- fillLibrarianFile(90, FAKE_DEB_CONTENT)
2166- self.execute_command('fetch pmount')
2167-
2168- # Check the files' names.
2169- files = sorted(self._listfiles())
2170- self.assertEqual(
2171- ['netapplet-1.0.0.tar.gz', 'pmount_1.0-1_all.deb'],
2172- files)
2173-
2174- # Check the files' contents.
2175- changes_file = open('netapplet-1.0.0.tar.gz')
2176- self.assertEqual(changes_file.read(), FAKE_CHANGESFILE_CONTENT)
2177- changes_file.close()
2178- debfile = open('pmount_1.0-1_all.deb')
2179- self.assertEqual(debfile.read(), FAKE_DEB_CONTENT)
2180- debfile.close()
2181-
2182- def testFetchMultipleItems(self):
2183- """Check if fetching multiple items at once works.
2184-
2185- We can specify multiple items to fetch, even mixing IDs and names.
2186- e.g. queue fetch alsa-utils 1 3
2187- """
2188- self.execute_command('fetch 3 mozilla-firefox')
2189- files = self._listfiles()
2190- files.sort()
2191- self.assertEqual(
2192- ['mozilla-firefox_0.9_i386.changes', 'netapplet-1.0.0.tar.gz'],
2193- files)
2194-
2195- def testFetchWithoutChanges(self):
2196- """Check that fetch works without a changes file (eg. from gina)."""
2197- pus = getUtility(IDistributionSet).getByName('ubuntu').getSeries(
2198- 'breezy-autotest').getPackageUploads(name=u'pmount')
2199- for pu in pus:
2200- removeSecurityProxy(pu).changesfile = None
2201-
2202- FAKE_DEB_CONTENT = "Fake DEB"
2203- fillLibrarianFile(90, FAKE_DEB_CONTENT)
2204- self.execute_command('fetch pmount')
2205-
2206- # Check the files' names.
2207- files = sorted(self._listfiles())
2208- self.assertEqual(['pmount_1.0-1_all.deb'], files)
2209
2210=== modified file 'lib/lp/soyuz/tests/test_packageupload.py'
2211--- lib/lp/soyuz/tests/test_packageupload.py 2012-07-24 06:39:54 +0000
2212+++ lib/lp/soyuz/tests/test_packageupload.py 2012-09-28 10:09:21 +0000
2213@@ -12,6 +12,7 @@
2214 urlopen,
2215 )
2216
2217+from debian.deb822 import Changes
2218 from lazr.restfulclient.errors import (
2219 BadRequest,
2220 Unauthorized,
2221@@ -37,6 +38,7 @@
2222 from lp.services.librarian.browser import ProxiedLibraryFileAlias
2223 from lp.services.log.logger import BufferLogger
2224 from lp.services.mail import stub
2225+from lp.services.mail.sendmail import format_address_for_person
2226 from lp.soyuz.adapters.overrides import SourceOverride
2227 from lp.soyuz.enums import (
2228 ArchivePurpose,
2229@@ -412,6 +414,203 @@
2230 self.assertEqual(current_component, spr.component)
2231 self.assertEqual(new_section, spr.section)
2232
2233+ def makeSourcePackageUpload(self, pocket=None, sourcepackagename=None,
2234+ section_name=None, changes_dict=None):
2235+ """Make a useful source package upload for queue tests."""
2236+ distroseries = self.test_publisher.distroseries
2237+ distroseries.changeslist = "autotest_changes@ubuntu.com"
2238+ uploader = self.factory.makePerson()
2239+ key = self.factory.makeGPGKey(owner=uploader)
2240+ changes = Changes({"Changed-By": uploader.preferredemail.email})
2241+ if changes_dict is not None:
2242+ changes.update(changes_dict)
2243+ upload = self.factory.makePackageUpload(
2244+ archive=distroseries.main_archive, distroseries=distroseries,
2245+ pocket=pocket, changes_file_content=changes.dump().encode("UTF-8"),
2246+ signing_key=key)
2247+ spr = self.factory.makeSourcePackageRelease(
2248+ sourcepackagename=sourcepackagename, distroseries=distroseries,
2249+ component="main", section_name=section_name,
2250+ changelog_entry="dummy")
2251+ upload.addSource(spr)
2252+ spr.addFile(self.factory.makeLibraryFileAlias(
2253+ filename="%s_%s.dsc" % (spr.name, spr.version)))
2254+ transaction.commit()
2255+ return upload, uploader
2256+
2257+ def makeBuildPackageUpload(self):
2258+ """Make a useful build package upload for queue tests."""
2259+ distroseries = self.test_publisher.distroseries
2260+ distroseries.changeslist = "autotest_changes@ubuntu.com"
2261+ uploader = self.factory.makePerson()
2262+ key = self.factory.makeGPGKey(owner=uploader)
2263+ changes = Changes({"Changed-By": uploader.preferredemail.email})
2264+ upload = self.factory.makePackageUpload(
2265+ archive=distroseries.main_archive, distroseries=distroseries,
2266+ changes_file_content=changes.dump().encode("UTF-8"),
2267+ signing_key=key)
2268+ build = self.factory.makeBinaryPackageBuild(
2269+ distroarchseries=self.test_publisher.breezy_autotest_i386)
2270+ upload.addBuild(build)
2271+ bpr = self.factory.makeBinaryPackageRelease(build=build)
2272+ bpr.addFile(self.factory.makeLibraryFileAlias(
2273+ filename="%s_%s_i386.deb" % (bpr.name, bpr.version)))
2274+ transaction.commit()
2275+ return upload, uploader
2276+
2277+ def assertEmail(self, expected_to_addrs):
2278+ """Pop an email from the stub queue and check its recipients."""
2279+ _, to_addrs, _ = stub.test_emails.pop()
2280+ self.assertEqual(expected_to_addrs, to_addrs)
2281+
2282+ def test_acceptFromQueue_source_sends_email(self):
2283+ # Accepting a source package sends emails to the announcement list
2284+ # and the uploader.
2285+ self.test_publisher.prepareBreezyAutotest()
2286+ upload, uploader = self.makeSourcePackageUpload()
2287+ upload.acceptFromQueue()
2288+ self.assertEqual(2, len(stub.test_emails))
2289+ # Emails sent are the announcement and the uploader's notification:
2290+ self.assertEmail(["autotest_changes@ubuntu.com"])
2291+ self.assertEmail([format_address_for_person(uploader)])
2292+
2293+ def test_acceptFromQueue_source_backports_sends_no_announcement(self):
2294+ # Accepting a source package into BACKPORTS does not send an
2295+ # announcement email to the distroseries changeslist (see bug
2296+ # #59443). It still sends an acknowledgement to the uploader.
2297+ self.test_publisher.prepareBreezyAutotest()
2298+ self.test_publisher.distroseries.status = SeriesStatus.CURRENT
2299+ upload, uploader = self.makeSourcePackageUpload(
2300+ pocket=PackagePublishingPocket.BACKPORTS)
2301+ upload.acceptFromQueue()
2302+ self.assertEqual(1, len(stub.test_emails))
2303+ # Only one email is sent, to the person in the changed-by field. No
2304+ # announcement email is sent.
2305+ self.assertEmail([format_address_for_person(uploader)])
2306+
2307+ def test_acceptFromQueue_source_translations_sends_no_email(self):
2308+ # Accepting source packages in the "translations" section (i.e.
2309+ # language packs) does not send any email. See bug #57708.
2310+ self.test_publisher.prepareBreezyAutotest()
2311+ self.test_publisher.distroseries.status = SeriesStatus.CURRENT
2312+ upload, _ = self.makeSourcePackageUpload(
2313+ pocket=PackagePublishingPocket.PROPOSED,
2314+ section_name="translations")
2315+ upload.acceptFromQueue()
2316+ self.assertEqual("DONE", upload.status.name)
2317+ self.assertEqual(0, len(stub.test_emails))
2318+
2319+ def test_acceptFromQueue_source_creates_builds(self):
2320+ # Accepting a source package creates build records.
2321+ self.test_publisher.prepareBreezyAutotest()
2322+ upload, _ = self.makeSourcePackageUpload()
2323+ upload.acceptFromQueue()
2324+ spr = upload.sourcepackagerelease
2325+ [build] = spr.builds
2326+ self.assertEqual(
2327+ "i386 build of %s %s in ubuntutest breezy-autotest RELEASE" % (
2328+ spr.name, spr.version),
2329+ build.title)
2330+
2331+ def test_acceptFromQueue_source_closes_bug(self):
2332+ # Accepting a source package closes bugs appropriately.
2333+ self.test_publisher.prepareBreezyAutotest()
2334+
2335+ # Upload the first version of a package.
2336+ upload_one, _ = self.makeSourcePackageUpload()
2337+ upload_one.setAccepted()
2338+ upload_one.realiseUpload()
2339+ spr = upload_one.sourcepackagerelease
2340+
2341+ # Make a new bug task for this package. It starts life as NEW.
2342+ dsp = self.test_publisher.ubuntutest.getSourcePackage(spr.name)
2343+ task = self.factory.makeBugTask(target=dsp, publish=False)
2344+ self.assertEqual("NEW", task.status.name)
2345+
2346+ # Upload the next version of the same package, closing this bug.
2347+ changes = Changes({"Launchpad-Bugs-Fixed": str(task.bug.id)})
2348+ upload_two, _ = self.makeSourcePackageUpload(
2349+ sourcepackagename=spr.sourcepackagename, changes_dict=changes)
2350+
2351+ # Accept the new upload. It should reach the DONE state, and should
2352+ # close the bug.
2353+ upload_two.acceptFromQueue()
2354+ self.assertEqual("DONE", upload_two.status.name)
2355+ self.assertEqual("FIXRELEASED", task.status.name)
2356+
2357+ def test_acceptFromQueue_binary_sends_no_email(self):
2358+ # Accepting a binary package does not send email.
2359+ self.test_publisher.prepareBreezyAutotest()
2360+ upload, _ = self.makeBuildPackageUpload()
2361+ upload.acceptFromQueue()
2362+ self.assertEqual(0, len(stub.test_emails))
2363+
2364+ def test_acceptFromQueue_handles_duplicates(self):
2365+ # Duplicate queue entries are handled sensibly.
2366+ self.test_publisher.prepareBreezyAutotest()
2367+ distroseries = self.test_publisher.distroseries
2368+ upload_one = self.factory.makePackageUpload(
2369+ archive=distroseries.main_archive, distroseries=distroseries)
2370+ upload_one.addSource(self.factory.makeSourcePackageRelease(
2371+ sourcepackagename="cnews", distroseries=distroseries,
2372+ component="main", version="1.0"))
2373+ upload_two = self.factory.makePackageUpload(
2374+ archive=distroseries.main_archive, distroseries=distroseries)
2375+ upload_two.addSource(self.factory.makeSourcePackageRelease(
2376+ sourcepackagename="cnews", distroseries=distroseries,
2377+ component="main", version="1.0"))
2378+ transaction.commit()
2379+ upload_one.setUnapproved()
2380+ upload_one.syncUpdate()
2381+ upload_two.setUnapproved()
2382+ upload_two.syncUpdate()
2383+
2384+ # There are now duplicate uploads in UNAPPROVED.
2385+ unapproved = distroseries.getPackageUploads(
2386+ status=PackageUploadStatus.UNAPPROVED, name=u"cnews")
2387+ self.assertEqual(2, unapproved.count())
2388+
2389+ # Accepting one of them works. (Since it's a single source upload,
2390+ # it goes straight to DONE.)
2391+ upload_one.acceptFromQueue()
2392+ self.assertEqual("DONE", upload_one.status.name)
2393+ transaction.commit()
2394+
2395+ # Trying to accept the second fails.
2396+ self.assertRaises(
2397+ QueueInconsistentStateError, upload_two.acceptFromQueue)
2398+ self.assertEqual("UNAPPROVED", upload_two.status.name)
2399+
2400+ # Rejecting the second upload works.
2401+ upload_two.rejectFromQueue()
2402+ self.assertEqual("REJECTED", upload_two.status.name)
2403+
2404+ def test_rejectFromQueue_source_sends_email(self):
2405+ # Rejecting a source package sends an email to the uploader.
2406+ self.test_publisher.prepareBreezyAutotest()
2407+ upload, uploader = self.makeSourcePackageUpload()
2408+ upload.rejectFromQueue()
2409+ self.assertEqual(1, len(stub.test_emails))
2410+ self.assertEmail([format_address_for_person(uploader)])
2411+
2412+ def test_rejectFromQueue_binary_sends_email(self):
2413+ # Rejecting a binary package sends an email to the uploader.
2414+ self.test_publisher.prepareBreezyAutotest()
2415+ upload, uploader = self.makeBuildPackageUpload()
2416+ upload.rejectFromQueue()
2417+ self.assertEqual(1, len(stub.test_emails))
2418+ self.assertEmail([format_address_for_person(uploader)])
2419+
2420+ def test_rejectFromQueue_source_translations_sends_no_email(self):
2421+ # Rejecting a language pack sends no email.
2422+ self.test_publisher.prepareBreezyAutotest()
2423+ self.test_publisher.distroseries.status = SeriesStatus.CURRENT
2424+ upload, _ = self.makeSourcePackageUpload(
2425+ pocket=PackagePublishingPocket.PROPOSED,
2426+ section_name="translations")
2427+ upload.rejectFromQueue()
2428+ self.assertEqual(0, len(stub.test_emails))
2429+
2430
2431 class TestPackageUploadPrivacy(TestCaseWithFactory):
2432 """Test PackageUpload security."""
2433
2434=== removed file 'scripts/ftpmaster-tools/queue'
2435--- scripts/ftpmaster-tools/queue 2012-06-29 08:40:05 +0000
2436+++ scripts/ftpmaster-tools/queue 1970-01-01 00:00:00 +0000
2437@@ -1,125 +0,0 @@
2438-#!/usr/bin/python -S
2439-#
2440-# Copyright 2009 Canonical Ltd. This software is licensed under the
2441-# GNU Affero General Public License version 3 (see the file LICENSE).
2442-
2443-"""Queue management script
2444-
2445-Tool for handling and visualisation of upload queue records.
2446-"""
2447-
2448-import _pythonpath
2449-
2450-import transaction
2451-
2452-from lp.services.config import config
2453-from lp.services.scripts.base import (
2454- LaunchpadScript,
2455- LaunchpadScriptFailure,
2456- )
2457-from lp.soyuz.scripts.queue import (
2458- CommandRunner,
2459- CommandRunnerError,
2460- name_queue_map,
2461- )
2462-
2463-
2464-class QueueScript(LaunchpadScript):
2465-
2466- usage = 'Usage: %prog [options] <command>'
2467-
2468- def add_my_options(self):
2469- self.parser.add_option(
2470- "-Q", "--queue",
2471- dest="queue_name", metavar="QUEUE", default="new",
2472- help="Which queue to consider")
2473-
2474- self.parser.add_option(
2475- "-d", "--distribution",
2476- dest="distribution_name", metavar="DISTRO", default=None,
2477- help="Which distro to look in")
2478-
2479- self.parser.add_option(
2480- "-s", "--suite",
2481- dest="suite_name", metavar="DISTRORELEASE", default=None,
2482- help=("Which distrorelease to look in, defaults "
2483- "to distribution 'currentseries'."))
2484-
2485- self.parser.add_option(
2486- "-N", "--dry-run", action="store_true",
2487- dest="dryrun", metavar="DRY_RUN", default=False,
2488- help="Whether to treat this as a dry-run or not.")
2489-
2490- self.parser.add_option(
2491- "-M", "--no-mail", action="store_true",
2492- dest="nomail", metavar="NO_MAIL", default=False,
2493- help="Whether to send announce email or not.")
2494-
2495- self.parser.add_option(
2496- "-e", "--exact-match", action="store_true",
2497- dest="exact_match", metavar="EXACTMATCH", default=False,
2498- help="Whether treat filter as a exact match or not.")
2499-
2500- self.parser.add_option(
2501- "-i", "--ignore-errors", action="store_true",
2502- dest="ignore_errors", metavar="IGNOREERRORS", default=False,
2503- help="Ignore errors when performing a list of commands.")
2504-
2505- self.parser.add_option(
2506- "-f", "--file", metavar="FILE", default=None,
2507- help="file containing a sequence of command lines.")
2508-
2509- self.parser.add_option(
2510- "-c", "--component", dest="component_name",
2511- metavar="COMPONENT", default=None,
2512- help="When overriding, move package to COMPONENT")
2513-
2514- self.parser.add_option(
2515- "-x", "--section", dest="section_name",
2516- metavar="SECTION", default=None,
2517- help="When overriding, move package to SECTION")
2518-
2519- self.parser.add_option(
2520- "-p", "--priority", dest="priority_name",
2521- metavar="PRIORITY", default=None,
2522- help="When overriding, move package to PRIORITY")
2523-
2524- def main(self):
2525- if self.options.queue_name not in name_queue_map:
2526- self.parser.error(
2527- 'Unable to map queue name "%s"' % self.options.queue_name)
2528-
2529- no_mail = self.options.dryrun or self.options.nomail
2530- queue = name_queue_map[self.options.queue_name]
2531-
2532- if self.options.file:
2533- args_list = [self.args.strip().split() for args in
2534- open(self.options.file).readlines()]
2535- else:
2536- args_list = [self.args]
2537-
2538- cmd_runner = CommandRunner(
2539- queue, self.options.distribution_name, self.options.suite_name,
2540- no_mail, self.options.component_name, self.options.section_name,
2541- self.options.priority_name, log=self.logger)
2542-
2543- print "Initializing connection to queue %s" % self.options.queue_name
2544-
2545- for single_args in args_list:
2546- try:
2547- cmd_runner.execute(single_args, self.options.exact_match)
2548- except CommandRunnerError as info:
2549- print (info)
2550- if self.options.ignore_errors:
2551- continue
2552- transaction.abort()
2553- raise LaunchpadScriptFailure(
2554- 'Error encountered -- aborting current transaction')
2555- else:
2556- if not self.options.dryrun:
2557- transaction.commit()
2558- else:
2559- print "DRY RUN requested, not committing."
2560-
2561-if __name__ == '__main__':
2562- QueueScript('queue', config.uploadqueue.dbuser).run()