Merge lp:~cjwatson/launchpad/remove-queue-tool into lp:launchpad
- remove-queue-tool
- Merge into devel
| Status: | Merged | ||||
|---|---|---|---|---|---|
| Approved by: | William Grant on 2012-09-27 | ||||
| Approved revision: | no longer in the source branch. | ||||
| Merged at revision: | 16052 | ||||
| Proposed branch: | lp:~cjwatson/launchpad/remove-queue-tool | ||||
| Merge into: | lp:launchpad | ||||
| Diff against target: |
2562 lines (+253/-2208) 7 files modified
lib/lp/soyuz/doc/ftpmaster-tools.txt (+0/-71) lib/lp/soyuz/model/sourcepackagerelease.py (+2/-2) lib/lp/soyuz/scripts/queue.py (+0/-737) lib/lp/soyuz/scripts/tests/test_processaccepted.py (+52/-3) lib/lp/soyuz/scripts/tests/test_queue.py (+0/-1270) lib/lp/soyuz/tests/test_packageupload.py (+199/-0) scripts/ftpmaster-tools/queue (+0/-125) |
||||
| To merge this branch: | bzr merge lp:~cjwatson/launchpad/remove-queue-tool | ||||
| Related bugs: |
|
| Reviewer | Review Type | Date Requested | Status |
|---|---|---|---|
| William Grant | code | 2012-07-12 | Approve on 2012-09-27 |
| j.c.sackett (community) | 2012-07-11 | Approve on 2012-07-12 | |
|
Review via email:
|
|||
Commit Message
Remove the queue script, now entirely superseded by the PackageUpload API.
Description of the Change
== Summary ==
Complete the migration from the queue script to an API client.
== Proposed fix ==
Once the current contents of devel pass buildbot and QA and are deployed, the queue API client will be feature-complete. We can and should therefore remove the old queue script to reclaim the lines of code spent on the API work (twice over, in fact).
== Implementation details ==
Many of the tests were already exercised elsewhere (notably, overrides are I think already reasonably well exercised via TestPackageUplo
== Tests ==
bin/test -vvct test_packageupload -t test_processacc
== Lint ==
Just an existing false positive due to pocketlint not understanding property setters:
./lib/lp/
196: redefinition of function 'copyright' from line 187
| Colin Watson (cjwatson) wrote : | # |
I've asked ubuntu-devel@ about arrangements to stress-test the new API in production:
https:/
| Colin Watson (cjwatson) wrote : | # |
Putting this on hold for now, since we're going to have to fix bug 745799 first.
https:/
| Colin Watson (cjwatson) wrote : | # |
This should be unblocked now. The fix for bug 745799 is behaving fine in production.
Preview Diff
| 1 | === removed file 'lib/lp/soyuz/doc/ftpmaster-tools.txt' |
| 2 | --- lib/lp/soyuz/doc/ftpmaster-tools.txt 2011-12-29 05:29:36 +0000 |
| 3 | +++ lib/lp/soyuz/doc/ftpmaster-tools.txt 1970-01-01 00:00:00 +0000 |
| 4 | @@ -1,71 +0,0 @@ |
| 5 | -= FTPMASTER Tools = |
| 6 | - |
| 7 | -This test commits to the test database in subprocesses and so needs to |
| 8 | -force the DatabaseLayer to fully tear down and restore the database |
| 9 | -after this test. |
| 10 | - |
| 11 | - >>> from lp.testing.layers import DatabaseLayer |
| 12 | - >>> DatabaseLayer.force_dirty_database() |
| 13 | - |
| 14 | -Queue Tool is a script designed to handle queue content. |
| 15 | -This test will check its output, since the script itself would |
| 16 | -open a new connection, let's invoke it in dry-run mode. |
| 17 | - |
| 18 | - >>> import subprocess |
| 19 | - >>> import os |
| 20 | - >>> import sys |
| 21 | - >>> from lp.services.config import config |
| 22 | - |
| 23 | - |
| 24 | - >>> script = os.path.join(config.root, "scripts", "ftpmaster-tools", |
| 25 | - ... "queue") |
| 26 | - |
| 27 | -INFO |
| 28 | - |
| 29 | - >>> process = subprocess.Popen([sys.executable, script, |
| 30 | - ... "-s", "breezy-autotest", "info"], |
| 31 | - ... stdout=subprocess.PIPE) |
| 32 | - >>> stdout, stderr = process.communicate() |
| 33 | - >>> process.returncode |
| 34 | - 0 |
| 35 | - >>> print stdout |
| 36 | - Initializing connection to queue new |
| 37 | - Running: "info" |
| 38 | - Listing ubuntu/breezy-autotest (NEW) 6/6 |
| 39 | - ---------|----|----------------------|----------------------|--------------- |
| 40 | - 7 | -- | netapplet-1.0.0.tar. | - | ... |
| 41 | - | * netapplet-1.0.0.tar.gz Format: DDTP_TARBALL |
| 42 | - 6 | -- | netapplet-1.0.0.tar. | - | ... |
| 43 | - | * netapplet-1.0.0.tar.gz Format: DIST_UPGRADER |
| 44 | - 4 | S- | alsa-utils | 1.0.9a-4ubuntu1 | ... |
| 45 | - | * alsa-utils/1.0.9a-4ubuntu1 Component: main Section: base |
| 46 | - 3 | S- | netapplet | 0.99.6-1 | ... |
| 47 | - | * netapplet/0.99.6-1 Component: main Section: web |
| 48 | - 2 | -B | pmount (i386) | 0.1-1 | ... |
| 49 | - | N pmount/0.1-1/i386 Component: main Section: base Priority: IMPORTANT |
| 50 | - 1 | -B | mozilla-firefox (i38 | 0.9 | ... |
| 51 | - | N mozilla-firefox/0.9/i386 Component: main Section: base Priority: EXTRA |
| 52 | - ---------|----|----------------------|----------------------|--------------- |
| 53 | - 6/6 |
| 54 | - total |
| 55 | - |
| 56 | - |
| 57 | -Check the custom uploads presentation: |
| 58 | - |
| 59 | - >>> process = subprocess.Popen([sys.executable, script, "-Q", "unapproved", |
| 60 | - ... "-s", "breezy-autotest-updates", "info"], |
| 61 | - ... stdout=subprocess.PIPE) |
| 62 | - >>> stdout, stderr = process.communicate() |
| 63 | - >>> process.returncode |
| 64 | - 0 |
| 65 | - >>> print stdout |
| 66 | - Initializing connection to queue unapproved |
| 67 | - Running: "info" |
| 68 | - Listing ubuntu/breezy-autotest-updates (UNAPPROVED) 1/1 |
| 69 | - ---------|----|----------------------|----------------------|--------------- |
| 70 | - 5 | -- | netapplet-1.0.0.tar. | - | ... |
| 71 | - | * netapplet-1.0.0.tar.gz Format: ROSETTA_TRANSLATIONS |
| 72 | - ---------|----|----------------------|----------------------|--------------- |
| 73 | - 1/1 total |
| 74 | - <BLANKLINE> |
| 75 | - |
| 76 | |
| 77 | === modified file 'lib/lp/soyuz/model/sourcepackagerelease.py' |
| 78 | --- lib/lp/soyuz/model/sourcepackagerelease.py 2012-08-01 11:02:13 +0000 |
| 79 | +++ lib/lp/soyuz/model/sourcepackagerelease.py 2012-09-28 10:09:21 +0000 |
| 80 | @@ -66,6 +66,7 @@ |
| 81 | from lp.soyuz.interfaces.archive import MAIN_ARCHIVE_PURPOSES |
| 82 | from lp.soyuz.interfaces.binarypackagebuild import IBinaryPackageBuildSet |
| 83 | from lp.soyuz.interfaces.packagediff import PackageDiffAlreadyRequested |
| 84 | +from lp.soyuz.interfaces.queue import QueueInconsistentStateError |
| 85 | from lp.soyuz.interfaces.sourcepackagerelease import ISourcePackageRelease |
| 86 | from lp.soyuz.model.binarypackagebuild import BinaryPackageBuild |
| 87 | from lp.soyuz.model.files import SourcePackageReleaseFile |
| 88 | @@ -74,7 +75,6 @@ |
| 89 | PackageUpload, |
| 90 | PackageUploadSource, |
| 91 | ) |
| 92 | -from lp.soyuz.scripts.queue import QueueActionError |
| 93 | from lp.translations.interfaces.translationimportqueue import ( |
| 94 | ITranslationImportQueue, |
| 95 | ) |
| 96 | @@ -474,7 +474,7 @@ |
| 97 | if new_archive is not None: |
| 98 | self.upload_archive = new_archive |
| 99 | else: |
| 100 | - raise QueueActionError( |
| 101 | + raise QueueInconsistentStateError( |
| 102 | "New component '%s' requires a non-existent archive.") |
| 103 | if section is not None: |
| 104 | self.section = section |
| 105 | |
| 106 | === removed file 'lib/lp/soyuz/scripts/queue.py' |
| 107 | --- lib/lp/soyuz/scripts/queue.py 2012-07-03 16:04:54 +0000 |
| 108 | +++ lib/lp/soyuz/scripts/queue.py 1970-01-01 00:00:00 +0000 |
| 109 | @@ -1,737 +0,0 @@ |
| 110 | -# Copyright 2009-2012 Canonical Ltd. This software is licensed under the |
| 111 | -# GNU Affero General Public License version 3 (see the file LICENSE). |
| 112 | - |
| 113 | -# pylint: disable-msg=W0231 |
| 114 | -"""Ftpmaster queue tool libraries.""" |
| 115 | - |
| 116 | -# XXX StuartBishop 2007-01-31: |
| 117 | -# This should be renamed to ftpmasterqueue.py or just ftpmaster.py |
| 118 | -# as Launchpad contains lots of queues. |
| 119 | - |
| 120 | -__metaclass__ = type |
| 121 | -__all__ = [ |
| 122 | - 'CommandRunner', |
| 123 | - 'CommandRunnerError', |
| 124 | - 'QueueActionError', |
| 125 | - 'name_queue_map', |
| 126 | - ] |
| 127 | - |
| 128 | -from datetime import datetime |
| 129 | -import errno |
| 130 | -import hashlib |
| 131 | - |
| 132 | -import pytz |
| 133 | -from zope.component import getUtility |
| 134 | - |
| 135 | -from lp.app.browser.tales import DurationFormatterAPI |
| 136 | -from lp.app.errors import NotFoundError |
| 137 | -from lp.services.config import config |
| 138 | -from lp.services.librarian.utils import filechunks |
| 139 | -from lp.services.propertycache import cachedproperty |
| 140 | -from lp.soyuz.enums import PackageUploadStatus |
| 141 | -from lp.soyuz.interfaces.component import IComponentSet |
| 142 | -from lp.soyuz.interfaces.queue import ( |
| 143 | - IPackageUploadSet, |
| 144 | - QueueInconsistentStateError, |
| 145 | - ) |
| 146 | -from lp.soyuz.interfaces.section import ISectionSet |
| 147 | - |
| 148 | - |
| 149 | -name_queue_map = { |
| 150 | - "new": PackageUploadStatus.NEW, |
| 151 | - "unapproved": PackageUploadStatus.UNAPPROVED, |
| 152 | - "accepted": PackageUploadStatus.ACCEPTED, |
| 153 | - "done": PackageUploadStatus.DONE, |
| 154 | - "rejected": PackageUploadStatus.REJECTED, |
| 155 | - } |
| 156 | - |
| 157 | -#XXX cprov 2006-09-19: We need to use template engine instead of harcoded |
| 158 | -# format variables. |
| 159 | -HEAD = "-" * 9 + "|----|" + "-" * 22 + "|" + "-" * 22 + "|" + "-" * 15 |
| 160 | -FOOT_MARGIN = " " * (9 + 6 + 1 + 22 + 1 + 22 + 2) |
| 161 | -RULE = "-" * (12 + 9 + 6 + 1 + 22 + 1 + 22 + 2) |
| 162 | - |
| 163 | -FILTERMSG = """ |
| 164 | - Omit the filter for all records. |
| 165 | - Filter string consists of a queue ID or a pair <name>[/<version>]: |
| 166 | - |
| 167 | - 28 |
| 168 | - apt |
| 169 | - apt/1 |
| 170 | - |
| 171 | - Use '-e' command line argument for exact matches: |
| 172 | - |
| 173 | - -e apt |
| 174 | - -e apt/1.0-1 |
| 175 | -""" |
| 176 | - |
| 177 | - |
| 178 | -class QueueActionError(Exception): |
| 179 | - """Identify Errors occurred within QueueAction class and its children.""" |
| 180 | - |
| 181 | - |
| 182 | -class QueueAction: |
| 183 | - """Queue Action base class. |
| 184 | - |
| 185 | - Implements a bunch of common/useful method designed to provide easy |
| 186 | - PackageUpload handling. |
| 187 | - """ |
| 188 | - |
| 189 | - def __init__(self, distribution_name, suite_name, queue, terms, |
| 190 | - component_name, section_name, priority_name, |
| 191 | - display, no_mail=True, exact_match=False, log=None): |
| 192 | - """Initializes passed variables. """ |
| 193 | - self.terms = terms |
| 194 | - # Some actions have addtional commands at the start of the terms |
| 195 | - # so allow them to state that here by specifiying the start index. |
| 196 | - self.terms_start_index = 0 |
| 197 | - self.component_name = component_name |
| 198 | - self.section_name = section_name |
| 199 | - self.priority_name = priority_name |
| 200 | - self.exact_match = exact_match |
| 201 | - self.queue = queue |
| 202 | - self.no_mail = no_mail |
| 203 | - self.distribution_name = distribution_name |
| 204 | - self.suite_name = suite_name |
| 205 | - self.default_sender = "%s <%s>" % ( |
| 206 | - config.uploader.default_sender_name, |
| 207 | - config.uploader.default_sender_address) |
| 208 | - self.default_recipient = "%s <%s>" % ( |
| 209 | - config.uploader.default_recipient_name, |
| 210 | - config.uploader.default_recipient_address) |
| 211 | - self.display = display |
| 212 | - self.log = log |
| 213 | - |
| 214 | - @cachedproperty |
| 215 | - def size(self): |
| 216 | - """Return the size of the queue in question.""" |
| 217 | - return getUtility(IPackageUploadSet).count( |
| 218 | - status=self.queue, distroseries=self.distroseries, |
| 219 | - pocket=self.pocket) |
| 220 | - |
| 221 | - def setDefaultContext(self): |
| 222 | - """Set default distribuiton, distroseries.""" |
| 223 | - # if not found defaults to 'ubuntu' |
| 224 | - |
| 225 | - # Avoid circular imports. |
| 226 | - from lp.registry.interfaces.distribution import IDistributionSet |
| 227 | - from lp.registry.interfaces.pocket import PackagePublishingPocket |
| 228 | - |
| 229 | - distroset = getUtility(IDistributionSet) |
| 230 | - try: |
| 231 | - self.distribution = distroset[self.distribution_name] |
| 232 | - except NotFoundError: |
| 233 | - self.distribution = distroset['ubuntu'] |
| 234 | - |
| 235 | - if self.suite_name: |
| 236 | - # defaults to distro.currentseries if passed distroseries is |
| 237 | - # misapplied or not found. |
| 238 | - try: |
| 239 | - self.distroseries, self.pocket = ( |
| 240 | - self.distribution.getDistroSeriesAndPocket( |
| 241 | - self.suite_name)) |
| 242 | - except NotFoundError: |
| 243 | - raise QueueActionError('Context not found: "%s/%s"' |
| 244 | - % (self.distribution.name, |
| 245 | - self.suite_name)) |
| 246 | - else: |
| 247 | - self.distroseries = self.distribution.currentseries |
| 248 | - self.pocket = PackagePublishingPocket.RELEASE |
| 249 | - |
| 250 | - def initialize(self): |
| 251 | - """Builds a list of affected records based on the filter argument.""" |
| 252 | - self.setDefaultContext() |
| 253 | - |
| 254 | - self.package_names = [] |
| 255 | - self.items = [] |
| 256 | - self.items_size = 0 |
| 257 | - |
| 258 | - # Will be set to true if the command line specified package IDs. |
| 259 | - # This is required because package_names is expanded into IDs so we |
| 260 | - # need another way of knowing whether the user typed them. |
| 261 | - self.explicit_ids_specified = False |
| 262 | - |
| 263 | - terms = self.terms[self.terms_start_index:] |
| 264 | - if len(terms) == 0: |
| 265 | - # If no argument is passed, present all available results in |
| 266 | - # the selected queue. |
| 267 | - terms.append('') |
| 268 | - |
| 269 | - for term in terms: |
| 270 | - # refuse old-style '*' argument since we do not support |
| 271 | - # wildcards yet. |
| 272 | - if term == '*': |
| 273 | - self.displayUsage(FILTERMSG) |
| 274 | - |
| 275 | - if term.isdigit(): |
| 276 | - # retrieve PackageUpload item by id |
| 277 | - try: |
| 278 | - item = getUtility(IPackageUploadSet).get(int(term)) |
| 279 | - except NotFoundError as info: |
| 280 | - raise QueueActionError('Queue Item not found: %s' % info) |
| 281 | - |
| 282 | - if item.status != self.queue: |
| 283 | - raise QueueActionError( |
| 284 | - 'Item %s is in queue %s' % ( |
| 285 | - item.id, item.status.name)) |
| 286 | - |
| 287 | - if (item.distroseries != self.distroseries or |
| 288 | - item.pocket != self.pocket): |
| 289 | - raise QueueActionError( |
| 290 | - 'Item %s is in %s/%s-%s not in %s/%s-%s' |
| 291 | - % (item.id, item.distroseries.distribution.name, |
| 292 | - item.distroseries.name, item.pocket.name, |
| 293 | - self.distroseries.distribution.name, |
| 294 | - self.distroseries.name, self.pocket.name)) |
| 295 | - |
| 296 | - if item not in self.items: |
| 297 | - self.items.append(item) |
| 298 | - self.explicit_ids_specified = True |
| 299 | - else: |
| 300 | - # retrieve PackageUpload item by name/version key |
| 301 | - version = None |
| 302 | - if '/' in term: |
| 303 | - term, version = term.strip().split('/') |
| 304 | - |
| 305 | - # Expand SQLObject results. |
| 306 | - queue_items = self.distroseries.getPackageUploads( |
| 307 | - status=self.queue, name=term, version=version, |
| 308 | - exact_match=self.exact_match, pocket=self.pocket) |
| 309 | - for item in queue_items: |
| 310 | - if item not in self.items: |
| 311 | - self.items.append(item) |
| 312 | - self.package_names.append(term) |
| 313 | - |
| 314 | - self.items_size = len(self.items) |
| 315 | - |
| 316 | - def run(self): |
| 317 | - """Place holder for command action.""" |
| 318 | - raise NotImplementedError('No action implemented.') |
| 319 | - |
| 320 | - def displayTitle(self, action): |
| 321 | - """Common title/summary presentation method.""" |
| 322 | - self.display("%s %s/%s (%s) %s/%s" % ( |
| 323 | - action, self.distribution.name, self.suite_name, |
| 324 | - self.queue.name, self.items_size, self.size)) |
| 325 | - |
| 326 | - def displayHead(self): |
| 327 | - """Table head presentation method.""" |
| 328 | - self.display(HEAD) |
| 329 | - |
| 330 | - def displayBottom(self): |
| 331 | - """Displays the table bottom and a small statistic information.""" |
| 332 | - self.display( |
| 333 | - FOOT_MARGIN + "%d/%d total" % (self.items_size, self.size)) |
| 334 | - |
| 335 | - def displayRule(self): |
| 336 | - """Displays a rule line. """ |
| 337 | - self.display(RULE) |
| 338 | - |
| 339 | - def displayUsage(self, extended_info=None): |
| 340 | - """Display the class docstring as usage message. |
| 341 | - |
| 342 | - Raise QueueActionError with optional extended_info argument |
| 343 | - """ |
| 344 | - self.display(self.__doc__) |
| 345 | - raise QueueActionError(extended_info) |
| 346 | - |
| 347 | - def _makeTag(self, queue_item): |
| 348 | - """Compose an upload type tag for `queue_item`. |
| 349 | - |
| 350 | - A source upload without binaries is tagged as "S-". |
| 351 | - A binary upload without source is tagged as "-B." |
| 352 | - An upload with both source and binaries is tagged as "SB". |
| 353 | - An upload with a package copy job is tagged as "X-". |
| 354 | - """ |
| 355 | - # XXX cprov 2006-07-31: source_tag and build_tag ('S' & 'B') |
| 356 | - # are necessary simply to keep the format legaxy. |
| 357 | - # We may discuss a more reasonable output format later |
| 358 | - # and avoid extra boring code. The IDRQ.displayname should |
| 359 | - # do should be enough. |
| 360 | - if queue_item.package_copy_job is not None: |
| 361 | - return "X-" |
| 362 | - |
| 363 | - source_tag = { |
| 364 | - True: 'S', |
| 365 | - False: '-', |
| 366 | - } |
| 367 | - binary_tag = { |
| 368 | - True: 'B', |
| 369 | - False: '-', |
| 370 | - } |
| 371 | - return ( |
| 372 | - source_tag[queue_item.contains_source] + |
| 373 | - binary_tag[queue_item.contains_build]) |
| 374 | - |
| 375 | - def displayItem(self, queue_item): |
| 376 | - """Display one line summary of the queue item provided.""" |
| 377 | - tag = self._makeTag(queue_item) |
| 378 | - displayname = queue_item.displayname |
| 379 | - version = queue_item.displayversion |
| 380 | - age = DurationFormatterAPI( |
| 381 | - datetime.now(pytz.timezone('UTC')) - |
| 382 | - queue_item.date_created).approximateduration() |
| 383 | - |
| 384 | - if queue_item.contains_build: |
| 385 | - displayname = "%s (%s)" % (queue_item.displayname, |
| 386 | - queue_item.displayarchs) |
| 387 | - |
| 388 | - self.display("%8d | %s | %s | %s | %s" % |
| 389 | - (queue_item.id, tag, displayname.ljust(20)[:20], |
| 390 | - version.ljust(20)[:20], age)) |
| 391 | - |
| 392 | - def displayInfo(self, queue_item, only=None): |
| 393 | - """Displays additional information about the provided queue item. |
| 394 | - |
| 395 | - Optionally pass a binarypackagename via 'only' argument to display |
| 396 | - only exact matches within the selected build queue items. |
| 397 | - """ |
| 398 | - if queue_item.package_copy_job or queue_item.sources: |
| 399 | - self.display( |
| 400 | - "\t | * %s/%s Component: %s Section: %s" % ( |
| 401 | - queue_item.package_name, |
| 402 | - queue_item.package_version, |
| 403 | - queue_item.component_name, |
| 404 | - queue_item.section_name, |
| 405 | - )) |
| 406 | - |
| 407 | - for queue_build in queue_item.builds: |
| 408 | - for bpr in queue_build.build.binarypackages: |
| 409 | - if only and only != bpr.name: |
| 410 | - continue |
| 411 | - if bpr.is_new: |
| 412 | - status_flag = "N" |
| 413 | - else: |
| 414 | - status_flag = "*" |
| 415 | - self.display( |
| 416 | - "\t | %s %s/%s/%s Component: %s Section: %s Priority: %s" |
| 417 | - % (status_flag, bpr.name, bpr.version, |
| 418 | - bpr.build.distro_arch_series.architecturetag, |
| 419 | - bpr.component.name, bpr.section.name, |
| 420 | - bpr.priority.name)) |
| 421 | - |
| 422 | - for queue_custom in queue_item.customfiles: |
| 423 | - self.display("\t | * %s Format: %s" |
| 424 | - % (queue_custom.libraryfilealias.filename, |
| 425 | - queue_custom.customformat.name)) |
| 426 | - |
| 427 | - |
| 428 | -class QueueActionHelp(QueueAction): |
| 429 | - """Present provided actions summary""" |
| 430 | - |
| 431 | - def __init__(self, **kargs): |
| 432 | - self.kargs = kargs |
| 433 | - self.kargs['no_mail'] = True |
| 434 | - self.actions = kargs['terms'] |
| 435 | - self.display = kargs['display'] |
| 436 | - |
| 437 | - def initialize(self): |
| 438 | - """Mock initialization """ |
| 439 | - pass |
| 440 | - |
| 441 | - def run(self): |
| 442 | - """Present the actions description summary""" |
| 443 | - # present summary for specific or all actions |
| 444 | - if not self.actions: |
| 445 | - actions_help = queue_actions.items() |
| 446 | - not_available_actions = [] |
| 447 | - else: |
| 448 | - actions_help = [ |
| 449 | - (action, provider) |
| 450 | - for action, provider in queue_actions.items() |
| 451 | - if action in self.actions] |
| 452 | - not_available_actions = [ |
| 453 | - action for action in self.actions |
| 454 | - if action not in queue_actions.keys()] |
| 455 | - # present not available requested action if any. |
| 456 | - if not_available_actions: |
| 457 | - self.display( |
| 458 | - "Not available action(s): %s" % |
| 459 | - ", ".join(not_available_actions)) |
| 460 | - |
| 461 | - # extract summary from docstring of specified available actions |
| 462 | - for action, wrapper in actions_help: |
| 463 | - if action is 'help': |
| 464 | - continue |
| 465 | - wobj = wrapper(**self.kargs) |
| 466 | - summary = wobj.__doc__.splitlines()[0] |
| 467 | - self.display('\t%s : %s ' % (action, summary)) |
| 468 | - |
| 469 | - |
| 470 | -class QueueActionReport(QueueAction): |
| 471 | - """Present a report about the size of available queues""" |
| 472 | - |
| 473 | - def initialize(self): |
| 474 | - """Mock initialization """ |
| 475 | - self.setDefaultContext() |
| 476 | - |
| 477 | - def run(self): |
| 478 | - """Display the queues size.""" |
| 479 | - self.display("Report for %s/%s" % (self.distribution.name, |
| 480 | - self.distroseries.name)) |
| 481 | - |
| 482 | - for queue in name_queue_map.values(): |
| 483 | - size = getUtility(IPackageUploadSet).count( |
| 484 | - status=queue, distroseries=self.distroseries, |
| 485 | - pocket=self.pocket) |
| 486 | - self.display("\t%s -> %s entries" % (queue.name, size)) |
| 487 | - |
| 488 | - |
| 489 | -class QueueActionInfo(QueueAction): |
| 490 | - """Present the Queue item including its contents. |
| 491 | - |
| 492 | - Presents the contents of the selected upload(s). |
| 493 | - |
| 494 | - queue info <filter> |
| 495 | - """ |
| 496 | - |
| 497 | - def run(self): |
| 498 | - """Present the filtered queue ordered by date.""" |
| 499 | - self.displayTitle('Listing') |
| 500 | - self.displayHead() |
| 501 | - for queue_item in self.items: |
| 502 | - self.displayItem(queue_item) |
| 503 | - self.displayInfo(queue_item) |
| 504 | - self.displayHead() |
| 505 | - self.displayBottom() |
| 506 | - |
| 507 | - |
| 508 | -class QueueActionFetch(QueueAction): |
| 509 | - """Fetch the contents of a queue item. |
| 510 | - |
| 511 | - Download the contents of the selected upload(s). |
| 512 | - |
| 513 | - queue fetch <filter> |
| 514 | - """ |
| 515 | - |
| 516 | - def run(self): |
| 517 | - self.displayTitle('Fetching') |
| 518 | - self.displayRule() |
| 519 | - for queue_item in self.items: |
| 520 | - file_list = [] |
| 521 | - if queue_item.changesfile is not None: |
| 522 | - file_list.append(queue_item.changesfile) |
| 523 | - |
| 524 | - for source in queue_item.sources: |
| 525 | - for spr_file in source.sourcepackagerelease.files: |
| 526 | - file_list.append(spr_file.libraryfile) |
| 527 | - |
| 528 | - for build in queue_item.builds: |
| 529 | - for bpr in build.build.binarypackages: |
| 530 | - for bpr_file in bpr.files: |
| 531 | - file_list.append(bpr_file.libraryfile) |
| 532 | - |
| 533 | - for custom in queue_item.customfiles: |
| 534 | - file_list.append(custom.libraryfilealias) |
| 535 | - |
| 536 | - for libfile in file_list: |
| 537 | - self.display("Constructing %s" % libfile.filename) |
| 538 | - # do not overwrite files on disk (bug # 62976) |
| 539 | - try: |
| 540 | - existing_file = open(libfile.filename, "r") |
| 541 | - except IOError as e: |
| 542 | - if not e.errno == errno.ENOENT: |
| 543 | - raise |
| 544 | - # File does not already exist, so read file from librarian |
| 545 | - # and write to disk. |
| 546 | - libfile.open() |
| 547 | - out_file = open(libfile.filename, "w") |
| 548 | - for chunk in filechunks(libfile): |
| 549 | - out_file.write(chunk) |
| 550 | - out_file.close() |
| 551 | - libfile.close() |
| 552 | - else: |
| 553 | - # Check sha against existing file (bug #67014) |
| 554 | - existing_sha = hashlib.sha1() |
| 555 | - for chunk in filechunks(existing_file): |
| 556 | - existing_sha.update(chunk) |
| 557 | - existing_file.close() |
| 558 | - |
| 559 | - # bail out if the sha1 differs |
| 560 | - if libfile.content.sha1 != existing_sha.hexdigest(): |
| 561 | - raise CommandRunnerError("%s already present on disk " |
| 562 | - "and differs from new file" |
| 563 | - % libfile.filename) |
| 564 | - else: |
| 565 | - self.display("%s already on disk and checksum " |
| 566 | - "matches, skipping.") |
| 567 | - |
| 568 | - self.displayRule() |
| 569 | - self.displayBottom() |
| 570 | - |
| 571 | - |
| 572 | -class QueueActionReject(QueueAction): |
| 573 | - """Reject the contents of a queue item. |
| 574 | - |
| 575 | - Move the selected upload(s) to the REJECTED queue. |
| 576 | - |
| 577 | - queue reject <filter> |
| 578 | - """ |
| 579 | - |
| 580 | - def run(self): |
| 581 | - """Perform Reject action.""" |
| 582 | - self.displayTitle('Rejecting') |
| 583 | - self.displayRule() |
| 584 | - for queue_item in self.items: |
| 585 | - self.display('Rejecting %s' % queue_item.displayname) |
| 586 | - try: |
| 587 | - queue_item.rejectFromQueue( |
| 588 | - logger=self.log, dry_run=self.no_mail) |
| 589 | - except QueueInconsistentStateError as info: |
| 590 | - self.display('** %s could not be rejected due %s' |
| 591 | - % (queue_item.displayname, info)) |
| 592 | - |
| 593 | - self.displayRule() |
| 594 | - self.displayBottom() |
| 595 | - |
| 596 | - |
| 597 | -class QueueActionAccept(QueueAction): |
| 598 | - """Accept the contents of a queue item. |
| 599 | - |
| 600 | - Move the selected upload(s) to the ACCEPTED queue. |
| 601 | - |
| 602 | - queue accept <filter> |
| 603 | - """ |
| 604 | - |
| 605 | - def run(self): |
| 606 | - """Perform Accept action.""" |
| 607 | - self.displayTitle('Accepting') |
| 608 | - self.displayRule() |
| 609 | - for queue_item in self.items: |
| 610 | - self.display('Accepting %s' % queue_item.displayname) |
| 611 | - try: |
| 612 | - queue_item.acceptFromQueue( |
| 613 | - logger=self.log, dry_run=self.no_mail) |
| 614 | - except QueueInconsistentStateError as info: |
| 615 | - self.display('** %s could not be accepted due to %s' |
| 616 | - % (queue_item.displayname, info)) |
| 617 | - continue |
| 618 | - |
| 619 | - self.displayRule() |
| 620 | - self.displayBottom() |
| 621 | - |
| 622 | - |
| 623 | -class QueueActionOverride(QueueAction): |
| 624 | - """Override information in a queue item content. |
| 625 | - |
| 626 | - queue override [-c|--component] [-x|--section] [-p|--priority] |
| 627 | - <override_stanza> <filter> |
| 628 | - |
| 629 | - Where override_stanza is one of: |
| 630 | - source |
| 631 | - binary |
| 632 | - |
| 633 | - In each case, when you want to set an override supply the relevant option. |
| 634 | - |
| 635 | - So, to set a binary to have section 'editors' but leave the |
| 636 | - component and priority alone, do: |
| 637 | - |
| 638 | - queue override -x editors binary <filter> |
| 639 | - |
| 640 | - Binaries can only be overridden by passing a name filter, so it will |
| 641 | - only override the binary package which matches the filter. |
| 642 | - |
| 643 | - Or, to set a source's section to editors, do: |
| 644 | - |
| 645 | - queue override -x editors source <filter> |
| 646 | - """ |
| 647 | - supported_override_stanzas = ['source', 'binary'] |
| 648 | - |
| 649 | - def __init__(self, distribution_name, suite_name, queue, terms, |
| 650 | - component_name, section_name, priority_name, |
| 651 | - display, no_mail=True, exact_match=False, log=None): |
| 652 | - """Constructor for QueueActionOverride.""" |
| 653 | - |
| 654 | - # This exists so that self.terms_start_index can be set as this action |
| 655 | - # class has a command at the start of the terms. |
| 656 | - # Our first term is "binary" or "source" to specify the type of |
| 657 | - # over-ride. |
| 658 | - QueueAction.__init__(self, distribution_name, suite_name, queue, |
| 659 | - terms, component_name, section_name, |
| 660 | - priority_name, display, no_mail=True, |
| 661 | - exact_match=False, log=log) |
| 662 | - self.terms_start_index = 1 |
| 663 | - self.overrides_performed = 0 |
| 664 | - |
| 665 | - def run(self): |
| 666 | - """Perform Override action.""" |
| 667 | - self.displayTitle('Overriding') |
| 668 | - self.displayRule() |
| 669 | - |
| 670 | - # "terms" is the list of arguments starting at the override stanza |
| 671 | - # ("source" or "binary"). |
| 672 | - try: |
| 673 | - override_stanza = self.terms[0] |
| 674 | - except IndexError: |
| 675 | - self.displayUsage('Missing override_stanza.') |
| 676 | - return |
| 677 | - |
| 678 | - if override_stanza not in self.supported_override_stanzas: |
| 679 | - self.displayUsage('Not supported override_stanza: %s' |
| 680 | - % override_stanza) |
| 681 | - return |
| 682 | - |
| 683 | - return getattr(self, '_override_' + override_stanza)() |
| 684 | - |
| 685 | - def _override_source(self): |
| 686 | - """Overrides sourcepackagereleases selected. |
| 687 | - |
| 688 | - It doesn't check Component/Section Selection, this is a task |
| 689 | - for queue state-machine. |
| 690 | - """ |
| 691 | - component = None |
| 692 | - section = None |
| 693 | - try: |
| 694 | - if self.component_name: |
| 695 | - component = getUtility(IComponentSet)[self.component_name] |
| 696 | - if self.section_name: |
| 697 | - section = getUtility(ISectionSet)[self.section_name] |
| 698 | - except NotFoundError as info: |
| 699 | - raise QueueActionError('Not Found: %s' % info) |
| 700 | - |
| 701 | - for queue_item in self.items: |
| 702 | - # We delegate to the queue_item itself to override any/all |
| 703 | - # of its sources. |
| 704 | - if queue_item.contains_source or queue_item.package_copy_job: |
| 705 | - if queue_item.sourcepackagerelease: |
| 706 | - old_component = queue_item.sourcepackagerelease.component |
| 707 | - else: |
| 708 | - old_component = getUtility(IComponentSet)[ |
| 709 | - queue_item.package_copy_job.component_name] |
| 710 | - queue_item.overrideSource( |
| 711 | - component, section, [ |
| 712 | - component, old_component]) |
| 713 | - self.overrides_performed += 1 |
| 714 | - self.displayInfo(queue_item) |
| 715 | - |
| 716 | - def _override_binary(self): |
| 717 | - """Overrides binarypackagereleases selected""" |
| 718 | - from lp.soyuz.interfaces.publishing import name_priority_map |
| 719 | - if self.explicit_ids_specified: |
| 720 | - self.displayUsage('Cannot Override BinaryPackage retrieved by ID') |
| 721 | - |
| 722 | - component = None |
| 723 | - section = None |
| 724 | - priority = None |
| 725 | - try: |
| 726 | - if self.component_name: |
| 727 | - component = getUtility(IComponentSet)[self.component_name] |
| 728 | - if self.section_name: |
| 729 | - section = getUtility(ISectionSet)[self.section_name] |
| 730 | - if self.priority_name: |
| 731 | - priority = name_priority_map[self.priority_name] |
| 732 | - except (NotFoundError, KeyError) as info: |
| 733 | - raise QueueActionError('Not Found: %s' % info) |
| 734 | - |
| 735 | - overridden = [] |
| 736 | - for queue_item in self.items: |
| 737 | - for build in queue_item.builds: |
| 738 | - # Different than PackageUploadSources |
| 739 | - # PackageUploadBuild points to a Build, that can, |
| 740 | - # and usually does, point to multiple BinaryPackageReleases. |
| 741 | - # So we need to carefully select the requested package to be |
| 742 | - # overridden |
| 743 | - for binary in build.build.binarypackages: |
| 744 | - if binary.name in self.package_names: |
| 745 | - overridden.append(binary.name) |
| 746 | - self.display("Overriding %s_%s (%s/%s/%s)" |
| 747 | - % (binary.name, binary.version, |
| 748 | - binary.component.name, |
| 749 | - binary.section.name, |
| 750 | - binary.priority.name)) |
| 751 | - binary.override(component=component, section=section, |
| 752 | - priority=priority) |
| 753 | - self.overrides_performed += 1 |
| 754 | - self.displayInfo(queue_item, only=binary.name) |
| 755 | - # See if the new component requires a new archive on the |
| 756 | - # build: |
| 757 | - if component: |
| 758 | - distroarchseries = build.build.distro_arch_series |
| 759 | - distribution = distroarchseries.distroseries.distribution |
| 760 | - new_archive = distribution.getArchiveByComponent( |
| 761 | - self.component_name) |
| 762 | - if (new_archive != build.build.archive): |
| 763 | - raise QueueActionError( |
| 764 | - "Overriding component to '%s' failed because it " |
| 765 | - "would require a new archive." |
| 766 | - % self.component_name) |
| 767 | - |
| 768 | - not_overridden = set(self.package_names) - set(overridden) |
| 769 | - if len(not_overridden) > 0: |
| 770 | - self.displayUsage('No matches for %s' % ",".join(not_overridden)) |
| 771 | - |
| 772 | - |
| 773 | -queue_actions = { |
| 774 | - 'help': QueueActionHelp, |
| 775 | - 'info': QueueActionInfo, |
| 776 | - 'fetch': QueueActionFetch, |
| 777 | - 'accept': QueueActionAccept, |
| 778 | - 'reject': QueueActionReject, |
| 779 | - 'override': QueueActionOverride, |
| 780 | - 'report': QueueActionReport, |
| 781 | - } |
| 782 | - |
| 783 | - |
| 784 | -def default_display(text): |
| 785 | - """Unified presentation method.""" |
| 786 | - print text |
| 787 | - |
| 788 | - |
| 789 | -class CommandRunnerError(Exception): |
| 790 | - """Command Runner Failure""" |
| 791 | - |
| 792 | - |
| 793 | -class CommandRunner: |
| 794 | - """A wrapper for queue_action classes.""" |
| 795 | - |
| 796 | - def __init__(self, queue, distribution_name, suite_name, |
| 797 | - no_mail, component_name, section_name, priority_name, |
| 798 | - display=default_display, log=None): |
| 799 | - self.queue = queue |
| 800 | - self.distribution_name = distribution_name |
| 801 | - self.suite_name = suite_name |
| 802 | - self.no_mail = no_mail |
| 803 | - self.component_name = component_name |
| 804 | - self.section_name = section_name |
| 805 | - self.priority_name = priority_name |
| 806 | - self.display = display |
| 807 | - self.log = log |
| 808 | - |
| 809 | - def execute(self, terms, exact_match=False): |
| 810 | - """Execute a single queue action.""" |
| 811 | - self.display('Running: "%s"' % " ".join(terms)) |
| 812 | - |
| 813 | - # check syntax, abort process if anything gets wrong |
| 814 | - try: |
| 815 | - action = terms[0] |
| 816 | - arguments = [unicode(term) for term in terms[1:]] |
| 817 | - except IndexError: |
| 818 | - raise CommandRunnerError('Invalid sentence, use help.') |
| 819 | - |
| 820 | - # check action availability, |
| 821 | - try: |
| 822 | - queue_action_class = queue_actions[action] |
| 823 | - except KeyError: |
| 824 | - raise CommandRunnerError('Unknown Action: %s' % action) |
| 825 | - |
| 826 | - # perform the required action on queue. |
| 827 | - try: |
| 828 | - # be sure to send every args via kargs |
| 829 | - queue_action = queue_action_class( |
| 830 | - distribution_name=self.distribution_name, |
| 831 | - suite_name=self.suite_name, |
| 832 | - queue=self.queue, |
| 833 | - no_mail=self.no_mail, |
| 834 | - display=self.display, |
| 835 | - terms=arguments, |
| 836 | - component_name=self.component_name, |
| 837 | - section_name=self.section_name, |
| 838 | - priority_name=self.priority_name, |
| 839 | - exact_match=exact_match, |
| 840 | - log=self.log) |
| 841 | - queue_action.initialize() |
| 842 | - queue_action.run() |
| 843 | - except QueueActionError as info: |
| 844 | - raise CommandRunnerError(info) |
| 845 | - |
| 846 | - return queue_action |
| 847 | |
| 848 | === modified file 'lib/lp/soyuz/scripts/tests/test_processaccepted.py' |
| 849 | --- lib/lp/soyuz/scripts/tests/test_processaccepted.py 2012-09-19 23:38:41 +0000 |
| 850 | +++ lib/lp/soyuz/scripts/tests/test_processaccepted.py 2012-09-28 10:09:21 +0000 |
| 851 | @@ -3,18 +3,32 @@ |
| 852 | |
| 853 | __metaclass__ = type |
| 854 | |
| 855 | +from StringIO import StringIO |
| 856 | from textwrap import dedent |
| 857 | |
| 858 | +from zope.component import getUtility |
| 859 | +from zope.security.interfaces import ForbiddenAttribute |
| 860 | from zope.security.proxy import removeSecurityProxy |
| 861 | |
| 862 | +from lp.app.enums import InformationType |
| 863 | from lp.bugs.interfaces.bugtask import BugTaskStatus |
| 864 | from lp.registry.interfaces.pocket import PackagePublishingPocket |
| 865 | +from lp.soyuz.interfaces.processacceptedbugsjob import ( |
| 866 | + IProcessAcceptedBugsJobSource, |
| 867 | + ) |
| 868 | from lp.soyuz.scripts.processaccepted import ( |
| 869 | close_bugs_for_sourcepackagerelease, |
| 870 | close_bugs_for_sourcepublication, |
| 871 | ) |
| 872 | -from lp.testing import TestCaseWithFactory |
| 873 | -from lp.testing.layers import LaunchpadZopelessLayer |
| 874 | +from lp.testing import ( |
| 875 | + celebrity_logged_in, |
| 876 | + person_logged_in, |
| 877 | + TestCaseWithFactory, |
| 878 | + ) |
| 879 | +from lp.testing.layers import ( |
| 880 | + DatabaseFunctionalLayer, |
| 881 | + LaunchpadZopelessLayer, |
| 882 | + ) |
| 883 | |
| 884 | |
| 885 | class TestClosingBugs(TestCaseWithFactory): |
| 886 | @@ -24,7 +38,6 @@ |
| 887 | start a unification in a single file and those other tests need |
| 888 | migrating here. |
| 889 | See also: |
| 890 | - * lp/soyuz/scripts/tests/test_queue.py |
| 891 | * lib/lp/soyuz/doc/closing-bugs-from-changelogs.txt |
| 892 | * lib/lp/archiveuploader/tests/nascentupload-closing-bugs.txt |
| 893 | """ |
| 894 | @@ -127,3 +140,39 @@ |
| 895 | |
| 896 | for bug, bugtask in bugs: |
| 897 | self.assertEqual(BugTaskStatus.FIXRELEASED, bugtask.status) |
| 898 | + |
| 899 | + |
| 900 | +class TestClosingPrivateBugs(TestCaseWithFactory): |
| 901 | + # The distroseries +queue page can close private bugs when accepting |
| 902 | + # packages. |
| 903 | + |
| 904 | + layer = DatabaseFunctionalLayer |
| 905 | + |
| 906 | + def test_close_bugs_for_sourcepackagerelease_with_private_bug(self): |
| 907 | + """close_bugs_for_sourcepackagerelease works with private bugs.""" |
| 908 | + changes_file_template = "Format: 1.7\nLaunchpad-bugs-fixed: %s\n" |
| 909 | + # changelog_entry is required for an assertion inside the function |
| 910 | + # we're testing. |
| 911 | + spr = self.factory.makeSourcePackageRelease(changelog_entry="blah") |
| 912 | + archive_admin = self.factory.makePerson() |
| 913 | + series = spr.upload_distroseries |
| 914 | + dsp = series.distribution.getSourcePackage(spr.sourcepackagename) |
| 915 | + bug = self.factory.makeBug( |
| 916 | + target=dsp, information_type=InformationType.USERDATA) |
| 917 | + changes = StringIO(changes_file_template % bug.id) |
| 918 | + |
| 919 | + with person_logged_in(archive_admin): |
| 920 | + # The archive admin user can't normally see this bug. |
| 921 | + self.assertRaises(ForbiddenAttribute, bug, 'status') |
| 922 | + # But the bug closure should work. |
| 923 | + close_bugs_for_sourcepackagerelease(series, spr, changes) |
| 924 | + |
| 925 | + # Rather than closing the bugs immediately, this creates a |
| 926 | + # ProcessAcceptedBugsJob. |
| 927 | + with celebrity_logged_in("admin"): |
| 928 | + self.assertEqual(BugTaskStatus.NEW, bug.default_bugtask.status) |
| 929 | + job_source = getUtility(IProcessAcceptedBugsJobSource) |
| 930 | + [job] = list(job_source.iterReady()) |
| 931 | + self.assertEqual(series, job.distroseries) |
| 932 | + self.assertEqual(spr, job.sourcepackagerelease) |
| 933 | + self.assertEqual([bug.id], job.bug_ids) |
| 934 | |
| 935 | === removed file 'lib/lp/soyuz/scripts/tests/test_queue.py' |
| 936 | --- lib/lp/soyuz/scripts/tests/test_queue.py 2012-09-24 20:37:00 +0000 |
| 937 | +++ lib/lp/soyuz/scripts/tests/test_queue.py 1970-01-01 00:00:00 +0000 |
| 938 | @@ -1,1270 +0,0 @@ |
| 939 | -# Copyright 2009-2012 Canonical Ltd. This software is licensed under the |
| 940 | -# GNU Affero General Public License version 3 (see the file LICENSE). |
| 941 | - |
| 942 | -"""queue tool base class tests.""" |
| 943 | - |
| 944 | -__metaclass__ = type |
| 945 | - |
| 946 | -import hashlib |
| 947 | -import os |
| 948 | -import shutil |
| 949 | -from StringIO import StringIO |
| 950 | -import tempfile |
| 951 | -from unittest import TestCase |
| 952 | - |
| 953 | -from testtools.matchers import StartsWith |
| 954 | -from zope.component import getUtility |
| 955 | -from zope.security.interfaces import ForbiddenAttribute |
| 956 | -from zope.security.proxy import removeSecurityProxy |
| 957 | - |
| 958 | -from lp.app.enums import InformationType |
| 959 | -from lp.archiveuploader.nascentupload import NascentUpload |
| 960 | -from lp.archiveuploader.tests import ( |
| 961 | - datadir, |
| 962 | - getPolicy, |
| 963 | - insertFakeChangesFileForAllPackageUploads, |
| 964 | - ) |
| 965 | -from lp.bugs.interfaces.bug import IBugSet |
| 966 | -from lp.bugs.interfaces.bugtask import ( |
| 967 | - BugTaskStatus, |
| 968 | - IBugTaskSet, |
| 969 | - ) |
| 970 | -from lp.registry.interfaces.distribution import IDistributionSet |
| 971 | -from lp.registry.interfaces.person import IPersonSet |
| 972 | -from lp.registry.interfaces.pocket import PackagePublishingPocket |
| 973 | -from lp.registry.interfaces.series import SeriesStatus |
| 974 | -from lp.services.config import config |
| 975 | -from lp.services.database.lpstorm import IStore |
| 976 | -from lp.services.librarian.interfaces import ILibraryFileAliasSet |
| 977 | -from lp.services.librarian.model import LibraryFileAlias |
| 978 | -from lp.services.librarian.utils import filechunks |
| 979 | -from lp.services.librarianserver.testing.server import fillLibrarianFile |
| 980 | -from lp.services.log.logger import DevNullLogger |
| 981 | -from lp.services.mail import stub |
| 982 | -from lp.soyuz.enums import ( |
| 983 | - ArchivePurpose, |
| 984 | - PackagePublishingStatus, |
| 985 | - PackageUploadStatus, |
| 986 | - ) |
| 987 | -from lp.soyuz.interfaces.archive import IArchiveSet |
| 988 | -from lp.soyuz.interfaces.processacceptedbugsjob import ( |
| 989 | - IProcessAcceptedBugsJobSource, |
| 990 | - ) |
| 991 | -from lp.soyuz.interfaces.queue import IPackageUploadSet |
| 992 | -from lp.soyuz.model.queue import PackageUploadBuild |
| 993 | -from lp.soyuz.scripts.processaccepted import ( |
| 994 | - close_bugs_for_sourcepackagerelease, |
| 995 | - ) |
| 996 | -from lp.soyuz.scripts.queue import ( |
| 997 | - CommandRunner, |
| 998 | - CommandRunnerError, |
| 999 | - name_queue_map, |
| 1000 | - QueueAction, |
| 1001 | - QueueActionOverride, |
| 1002 | - ) |
| 1003 | -from lp.testing import ( |
| 1004 | - celebrity_logged_in, |
| 1005 | - person_logged_in, |
| 1006 | - TestCaseWithFactory, |
| 1007 | - ) |
| 1008 | -from lp.testing.dbuser import ( |
| 1009 | - dbuser, |
| 1010 | - lp_dbuser, |
| 1011 | - switch_dbuser, |
| 1012 | - ) |
| 1013 | -from lp.testing.fakemethod import FakeMethod |
| 1014 | -from lp.testing.layers import ( |
| 1015 | - DatabaseFunctionalLayer, |
| 1016 | - LaunchpadZopelessLayer, |
| 1017 | - LibrarianLayer, |
| 1018 | - ) |
| 1019 | - |
| 1020 | - |
| 1021 | -class TestQueueBase: |
| 1022 | - """Base methods for queue tool test classes.""" |
| 1023 | - |
| 1024 | - def setUp(self): |
| 1025 | - # Switch database user and set isolation level to READ COMMIITTED |
| 1026 | - # to avoid SERIALIZATION exceptions with the Librarian. |
| 1027 | - switch_dbuser(self.dbuser) |
| 1028 | - |
| 1029 | - def _test_display(self, text): |
| 1030 | - """Store output from queue tool for inspection.""" |
| 1031 | - self.test_output.append(text) |
| 1032 | - |
| 1033 | - def execute_command(self, argument, queue_name='new', no_mail=True, |
| 1034 | - distribution_name='ubuntu', component_name=None, |
| 1035 | - section_name=None, priority_name=None, |
| 1036 | - suite_name='breezy-autotest', quiet=True): |
| 1037 | - """Helper method to execute a queue command. |
| 1038 | - |
| 1039 | - Initialize output buffer and execute a command according |
| 1040 | - given argument. |
| 1041 | - |
| 1042 | - Return the used QueueAction instance. |
| 1043 | - """ |
| 1044 | - self.test_output = [] |
| 1045 | - queue = name_queue_map[queue_name] |
| 1046 | - runner = CommandRunner( |
| 1047 | - queue, distribution_name, suite_name, no_mail, |
| 1048 | - component_name, section_name, priority_name, |
| 1049 | - display=self._test_display) |
| 1050 | - |
| 1051 | - return runner.execute(argument.split()) |
| 1052 | - |
| 1053 | - def assertEmail(self, expected_to_addrs): |
| 1054 | - """Pop an email from the stub queue and check its recipients.""" |
| 1055 | - from_addr, to_addrs, raw_msg = stub.test_emails.pop() |
| 1056 | - self.assertEqual(to_addrs, expected_to_addrs) |
| 1057 | - |
| 1058 | - |
| 1059 | -class TestQueueTool(TestQueueBase, TestCase): |
| 1060 | - layer = LaunchpadZopelessLayer |
| 1061 | - dbuser = config.uploadqueue.dbuser |
| 1062 | - |
| 1063 | - def setUp(self): |
| 1064 | - """Create contents in disk for librarian sampledata.""" |
| 1065 | - # Packageupload.notify() needs real changes file data to send |
| 1066 | - # email, so this nice simple "ed" changes file will do. It's |
| 1067 | - # the /wrong/ changes file for the package in the upload queue, |
| 1068 | - # but that doesn't matter as only email addresses are parsed out |
| 1069 | - # of it. |
| 1070 | - insertFakeChangesFileForAllPackageUploads() |
| 1071 | - fake_chroot = LibraryFileAlias.get(1) |
| 1072 | - |
| 1073 | - switch_dbuser("testadmin") |
| 1074 | - |
| 1075 | - ubuntu = getUtility(IDistributionSet)['ubuntu'] |
| 1076 | - breezy_autotest = ubuntu.getSeries('breezy-autotest') |
| 1077 | - breezy_autotest['i386'].addOrUpdateChroot(fake_chroot) |
| 1078 | - |
| 1079 | - switch_dbuser('launchpad') |
| 1080 | - |
| 1081 | - TestQueueBase.setUp(self) |
| 1082 | - |
| 1083 | - def tearDown(self): |
| 1084 | - """Remove test contents from disk.""" |
| 1085 | - LibrarianLayer.librarian_fixture.clear() |
| 1086 | - |
| 1087 | - def uploadPackage(self, |
| 1088 | - changesfile="suite/bar_1.0-1/bar_1.0-1_source.changes"): |
| 1089 | - """Helper function to upload a package.""" |
| 1090 | - with dbuser("uploader"): |
| 1091 | - sync_policy = getPolicy( |
| 1092 | - name='sync', distro='ubuntu', distroseries='breezy-autotest') |
| 1093 | - bar_src = NascentUpload.from_changesfile_path( |
| 1094 | - datadir(changesfile), |
| 1095 | - sync_policy, DevNullLogger()) |
| 1096 | - bar_src.process() |
| 1097 | - bar_src.do_accept() |
| 1098 | - return bar_src |
| 1099 | - |
| 1100 | - def testBrokenAction(self): |
| 1101 | - """Check if an unknown action raises CommandRunnerError.""" |
| 1102 | - self.assertRaises( |
| 1103 | - CommandRunnerError, self.execute_command, 'foo') |
| 1104 | - |
| 1105 | - def testHelpAction(self): |
| 1106 | - """Check if help is working properly. |
| 1107 | - |
| 1108 | - Without arguments 'help' should return the docstring summary of |
| 1109 | - all available actions. |
| 1110 | - |
| 1111 | - Optionally we can pass arguments corresponding to the specific |
| 1112 | - actions we want to see the help, not available actions will be |
| 1113 | - reported. |
| 1114 | - """ |
| 1115 | - self.execute_command('help') |
| 1116 | - self.assertEqual( |
| 1117 | - ['Running: "help"', |
| 1118 | - '\tinfo : Present the Queue item including its contents. ', |
| 1119 | - '\taccept : Accept the contents of a queue item. ', |
| 1120 | - '\treport : Present a report about the size of available ' |
| 1121 | - 'queues ', |
| 1122 | - '\treject : Reject the contents of a queue item. ', |
| 1123 | - '\toverride : Override information in a queue item content. ', |
| 1124 | - '\tfetch : Fetch the contents of a queue item. '], |
| 1125 | - self.test_output) |
| 1126 | - |
| 1127 | - self.execute_command('help fetch') |
| 1128 | - self.assertEqual( |
| 1129 | - ['Running: "help fetch"', |
| 1130 | - '\tfetch : Fetch the contents of a queue item. '], |
| 1131 | - self.test_output) |
| 1132 | - |
| 1133 | - self.execute_command('help foo') |
| 1134 | - self.assertEqual( |
| 1135 | - ['Running: "help foo"', |
| 1136 | - 'Not available action(s): foo'], |
| 1137 | - self.test_output) |
| 1138 | - |
| 1139 | - def testInfoAction(self): |
| 1140 | - """Check INFO queue action without arguments present all items.""" |
| 1141 | - queue_action = self.execute_command('info') |
| 1142 | - # check if the considered queue size matches the existent number |
| 1143 | - # of records in sampledata |
| 1144 | - bat = getUtility(IDistributionSet)['ubuntu']['breezy-autotest'] |
| 1145 | - queue_size = getUtility(IPackageUploadSet).count( |
| 1146 | - status=PackageUploadStatus.NEW, distroseries=bat, |
| 1147 | - pocket=PackagePublishingPocket.RELEASE) |
| 1148 | - self.assertEqual(queue_size, queue_action.size) |
| 1149 | - # check if none of them was filtered, since not filter term |
| 1150 | - # was passed. |
| 1151 | - self.assertEqual(queue_size, queue_action.items_size) |
| 1152 | - |
| 1153 | - def testInfoActionDoesNotSupportWildCards(self): |
| 1154 | - """Check if an wildcard-like filter raises CommandRunnerError.""" |
| 1155 | - self.assertRaises( |
| 1156 | - CommandRunnerError, self.execute_command, 'info *') |
| 1157 | - |
| 1158 | - def testInfoActionByID(self): |
| 1159 | - """Check INFO queue action filtering by ID. |
| 1160 | - |
| 1161 | - It should work as expected in case of existent ID in specified the |
| 1162 | - location. |
| 1163 | - Otherwise it raises CommandRunnerError if: |
| 1164 | - * ID not found |
| 1165 | - * specified ID doesn't match given suite name |
| 1166 | - * specified ID doesn't match the queue name |
| 1167 | - """ |
| 1168 | - queue_action = self.execute_command('info 1') |
| 1169 | - # Check if only one item was retrieved. |
| 1170 | - self.assertEqual(1, queue_action.items_size) |
| 1171 | - |
| 1172 | - displaynames = [item.displayname for item in queue_action.items] |
| 1173 | - self.assertEqual(['mozilla-firefox'], displaynames) |
| 1174 | - |
| 1175 | - # Check passing multiple IDs. |
| 1176 | - queue_action = self.execute_command('info 1 3 4') |
| 1177 | - self.assertEqual(3, queue_action.items_size) |
| 1178 | - [mozilla, netapplet, alsa] = queue_action.items |
| 1179 | - self.assertEqual('mozilla-firefox', mozilla.displayname) |
| 1180 | - self.assertEqual('netapplet', netapplet.displayname) |
| 1181 | - self.assertEqual('alsa-utils', alsa.displayname) |
| 1182 | - |
| 1183 | - # Check not found ID. |
| 1184 | - self.assertRaises( |
| 1185 | - CommandRunnerError, self.execute_command, 'info 100') |
| 1186 | - |
| 1187 | - # Check looking in the wrong suite. |
| 1188 | - self.assertRaises( |
| 1189 | - CommandRunnerError, self.execute_command, 'info 1', |
| 1190 | - suite_name='breezy-autotest-backports') |
| 1191 | - |
| 1192 | - # Check looking in the wrong queue. |
| 1193 | - self.assertRaises( |
| 1194 | - CommandRunnerError, self.execute_command, 'info 1', |
| 1195 | - queue_name='done') |
| 1196 | - |
| 1197 | - def testInfoActionByName(self): |
| 1198 | - """Check INFO queue action filtering by name""" |
| 1199 | - queue_action = self.execute_command('info pmount') |
| 1200 | - # check if only one item was retrieved as expected in the current |
| 1201 | - # sampledata |
| 1202 | - self.assertEqual(1, queue_action.items_size) |
| 1203 | - |
| 1204 | - displaynames = [item.displayname for item in queue_action.items] |
| 1205 | - self.assertEqual(['pmount'], displaynames) |
| 1206 | - |
| 1207 | - # Check looking for multiple names. |
| 1208 | - queue_action = self.execute_command('info pmount alsa-utils') |
| 1209 | - self.assertEqual(2, queue_action.items_size) |
| 1210 | - [pmount, alsa] = queue_action.items |
| 1211 | - self.assertEqual('pmount', pmount.displayname) |
| 1212 | - self.assertEqual('alsa-utils', alsa.displayname) |
| 1213 | - |
| 1214 | - def testAcceptingSourceGeneratesEmail(self): |
| 1215 | - """Check if accepting a source package generates an email.""" |
| 1216 | - # We need to upload a new source package to do this because the |
| 1217 | - # sample data is horribly broken with published sources also in |
| 1218 | - # the NEW queue. Doing it this way guarantees a nice set of data. |
| 1219 | - self.uploadPackage() |
| 1220 | - |
| 1221 | - # Swallow email generated at the upload stage. |
| 1222 | - stub.test_emails.pop() |
| 1223 | - |
| 1224 | - # Add a chroot to breezy-autotest/i386, so the system can create |
| 1225 | - # builds for it. |
| 1226 | - with lp_dbuser(): |
| 1227 | - a_file = getUtility(ILibraryFileAliasSet)[1] |
| 1228 | - breezy_autotest = getUtility( |
| 1229 | - IDistributionSet)['ubuntu']['breezy-autotest'] |
| 1230 | - breezy_autotest['i386'].addOrUpdateChroot(a_file) |
| 1231 | - |
| 1232 | - queue_action = self.execute_command( |
| 1233 | - 'accept bar', no_mail=False) |
| 1234 | - self.assertEqual(1, queue_action.items_size) |
| 1235 | - self.assertEqual(2, len(stub.test_emails)) |
| 1236 | - # Emails sent are the announcement and the uploader's notification: |
| 1237 | - self.assertEmail(['autotest_changes@ubuntu.com']) |
| 1238 | - self.assertEmail( |
| 1239 | - ['Daniel Silverstone <daniel.silverstone@canonical.com>']) |
| 1240 | - |
| 1241 | - def testAcceptingSourceCreateBuilds(self): |
| 1242 | - """Check if accepting a source package creates build records.""" |
| 1243 | - self.uploadPackage() |
| 1244 | - |
| 1245 | - # Swallow email generated at the upload stage. |
| 1246 | - stub.test_emails.pop() |
| 1247 | - |
| 1248 | - # Add a chroot to breezy-autotest/i386, so the system can create |
| 1249 | - # builds for it. |
| 1250 | - with lp_dbuser(): |
| 1251 | - a_file = getUtility(ILibraryFileAliasSet)[1] |
| 1252 | - breezy_autotest = getUtility( |
| 1253 | - IDistributionSet)['ubuntu']['breezy-autotest'] |
| 1254 | - breezy_autotest['i386'].addOrUpdateChroot(a_file) |
| 1255 | - |
| 1256 | - queue_action = self.execute_command( |
| 1257 | - 'accept bar', no_mail=False) |
| 1258 | - self.assertEqual(1, queue_action.items_size) |
| 1259 | - self.assertEqual(2, len(stub.test_emails)) |
| 1260 | - |
| 1261 | - [queue_item] = queue_action.items |
| 1262 | - [queue_source] = queue_item.sources |
| 1263 | - sourcepackagerelease = queue_source.sourcepackagerelease |
| 1264 | - [build] = sourcepackagerelease.builds |
| 1265 | - self.assertEqual( |
| 1266 | - 'i386 build of bar 1.0-1 in ubuntu breezy-autotest RELEASE', |
| 1267 | - build.title) |
| 1268 | - self.assertEqual(build.buildqueue_record.lastscore, 1755) |
| 1269 | - |
| 1270 | - def testAcceptingBinaryDoesntGenerateEmail(self): |
| 1271 | - """Check if accepting a binary package does not generate email.""" |
| 1272 | - queue_action = self.execute_command( |
| 1273 | - 'accept mozilla-firefox', no_mail=False) |
| 1274 | - self.assertEqual(1, queue_action.items_size) |
| 1275 | - self.assertEqual(0, len(stub.test_emails)) |
| 1276 | - |
| 1277 | - def testAcceptingSourceClosesBug(self): |
| 1278 | - """Check that accepting a source will close bugs appropriately.""" |
| 1279 | - # To speed up the publication process, single source uploads |
| 1280 | - # are automatically published when they are accepted to avoid |
| 1281 | - # another publisher cycle's worth of delay. When the source is |
| 1282 | - # published, any bugs mentioned in the upload must be closed. |
| 1283 | - |
| 1284 | - # First we must upload the first version of 'bar' in Ubuntu Hoary. |
| 1285 | - bar_src = self.uploadPackage() |
| 1286 | - bar_src.queue_root.setAccepted() |
| 1287 | - bar_src.queue_root.realiseUpload() |
| 1288 | - |
| 1289 | - # Now make a new bugtask for the "bar" package. |
| 1290 | - with lp_dbuser(): |
| 1291 | - the_bug_id = 6 |
| 1292 | - bugtask_owner = getUtility(IPersonSet).getByName('kinnison') |
| 1293 | - ubuntu = getUtility(IDistributionSet)['ubuntu'] |
| 1294 | - ubuntu_bar = ubuntu.getSourcePackage('bar') |
| 1295 | - the_bug = getUtility(IBugSet).get(the_bug_id) |
| 1296 | - bugtask = getUtility(IBugTaskSet).createTask( |
| 1297 | - the_bug, bugtask_owner, ubuntu_bar) |
| 1298 | - |
| 1299 | - # The bugtask starts life as NEW. |
| 1300 | - the_bug = getUtility(IBugSet).get(the_bug_id) |
| 1301 | - bugtask = the_bug.getBugTask(ubuntu_bar) |
| 1302 | - bug_status = bugtask.status.name |
| 1303 | - self.assertEqual( |
| 1304 | - bug_status, 'NEW', |
| 1305 | - 'Bug status is %s, expected NEW' % bug_status) |
| 1306 | - |
| 1307 | - # Now, make an upload for the next version of "bar". |
| 1308 | - bar2_src = self.uploadPackage( |
| 1309 | - changesfile="suite/bar_1.0-2/bar_1.0-2_source.changes") |
| 1310 | - |
| 1311 | - # Now accept the new bar upload with the queue tool. |
| 1312 | - self.execute_command('accept bar', no_mail=False) |
| 1313 | - |
| 1314 | - # The upload wants to close bug 6: |
| 1315 | - bugs_fixed_header = bar2_src.changes._dict['Launchpad-bugs-fixed'] |
| 1316 | - self.assertEqual( |
| 1317 | - bugs_fixed_header, str(the_bug_id), |
| 1318 | - 'Expected bug %s in Launchpad-bugs-fixed, got %s' |
| 1319 | - % (the_bug_id, bugs_fixed_header)) |
| 1320 | - |
| 1321 | - # The upload should be in the DONE state: |
| 1322 | - item_status = bar2_src.queue_root.status.name |
| 1323 | - self.assertEqual( |
| 1324 | - item_status, 'DONE', |
| 1325 | - 'Upload status is %s, expected DONE' % item_status) |
| 1326 | - |
| 1327 | - # The bug should now be marked as fix released for the "bar" |
| 1328 | - # bugtask: |
| 1329 | - the_bug = getUtility(IBugSet).get(the_bug_id) |
| 1330 | - bugtask = the_bug.getBugTask(ubuntu_bar) |
| 1331 | - bug_status = bugtask.status.name |
| 1332 | - self.assertEqual( |
| 1333 | - bug_status, 'FIXRELEASED', |
| 1334 | - 'Bug status is %s, expected FIXRELEASED') |
| 1335 | - |
| 1336 | - # Clean up. |
| 1337 | - upload_data = datadir('suite/bar_1.0-2') |
| 1338 | - os.remove(os.path.join(upload_data, 'bar_1.0.orig.tar.gz')) |
| 1339 | - |
| 1340 | - def testAcceptActionWithMultipleIDs(self): |
| 1341 | - """Check if accepting multiple items at once works. |
| 1342 | - |
| 1343 | - We can specify multiple items to accept, even mixing IDs and names. |
| 1344 | - e.g. queue accept alsa-utils 1 3 |
| 1345 | - """ |
| 1346 | - breezy_autotest = getUtility( |
| 1347 | - IDistributionSet)['ubuntu']['breezy-autotest'] |
| 1348 | - queue_action = self.execute_command('accept 1 pmount 3') |
| 1349 | - |
| 1350 | - self.assertEqual(3, queue_action.items_size) |
| 1351 | - |
| 1352 | - self.assertQueueLength(1, breezy_autotest, |
| 1353 | - PackageUploadStatus.ACCEPTED, u'mozilla-firefox') |
| 1354 | - self.assertQueueLength(1, breezy_autotest, |
| 1355 | - PackageUploadStatus.ACCEPTED, u'pmount') |
| 1356 | - # Single-source upload went straight to DONE queue. |
| 1357 | - self.assertQueueLength(1, breezy_autotest, |
| 1358 | - PackageUploadStatus.DONE, u'netapplet') |
| 1359 | - |
| 1360 | - def testRemovedPublishRecordDoesNotAffectQueueNewness(self): |
| 1361 | - """Check if REMOVED published record does not affect file NEWness. |
| 1362 | - |
| 1363 | - We only mark a file as *known* if there is a PUBLISHED record with |
| 1364 | - the same name, other states like SUPERSEDED or REMOVED doesn't count. |
| 1365 | - |
| 1366 | - This is the case of 'pmount_0.1-1' in ubuntu/breezy-autotest/i386, |
| 1367 | - there is a REMOVED publishing record for it as you can see in the |
| 1368 | - first part of the test. |
| 1369 | - |
| 1370 | - Following we can see the correct presentation of the new flag ('N'). |
| 1371 | - Bug #59291 |
| 1372 | - """ |
| 1373 | - # inspect publishing history in sampledata for the suspicious binary |
| 1374 | - # ensure is has a single entry and it is merked as REMOVED. |
| 1375 | - ubuntu = getUtility(IDistributionSet)['ubuntu'] |
| 1376 | - bat_i386 = ubuntu['breezy-autotest']['i386'] |
| 1377 | - moz_publishing = bat_i386.getBinaryPackage('pmount').releases |
| 1378 | - |
| 1379 | - self.assertEqual(1, len(moz_publishing)) |
| 1380 | - self.assertEqual(PackagePublishingStatus.DELETED, |
| 1381 | - moz_publishing[0].status) |
| 1382 | - |
| 1383 | - # invoke queue tool filtering by name |
| 1384 | - queue_action = self.execute_command('info pmount') |
| 1385 | - |
| 1386 | - # ensure we retrived a single item |
| 1387 | - self.assertEqual(1, queue_action.items_size) |
| 1388 | - |
| 1389 | - # and it is what we expect |
| 1390 | - self.assertEqual('pmount', queue_action.items[0].displayname) |
| 1391 | - self.assertEqual(moz_publishing[0].binarypackagerelease.build, |
| 1392 | - queue_action.items[0].builds[0].build) |
| 1393 | - # inspect output, note the presence of 'N' flag |
| 1394 | - self.assertTrue( |
| 1395 | - '| N pmount/0.1-1/i386' in '\n'.join(self.test_output)) |
| 1396 | - |
| 1397 | - def testQueueSupportForSuiteNames(self): |
| 1398 | - """Queue tool supports suite names properly. |
| 1399 | - |
| 1400 | - Two UNAPPROVED items are present for pocket RELEASE and only |
| 1401 | - one for pocket UPDATES in breezy-autotest. |
| 1402 | - Bug #59280 |
| 1403 | - """ |
| 1404 | - queue_action = self.execute_command( |
| 1405 | - 'info', queue_name='unapproved', |
| 1406 | - suite_name='breezy-autotest') |
| 1407 | - |
| 1408 | - self.assertEqual(2, queue_action.items_size) |
| 1409 | - self.assertEqual(PackagePublishingPocket.RELEASE, queue_action.pocket) |
| 1410 | - |
| 1411 | - queue_action = self.execute_command( |
| 1412 | - 'info', queue_name='unapproved', |
| 1413 | - suite_name='breezy-autotest-updates') |
| 1414 | - |
| 1415 | - self.assertEqual(1, queue_action.items_size) |
| 1416 | - self.assertEqual(PackagePublishingPocket.UPDATES, queue_action.pocket) |
| 1417 | - |
| 1418 | - def testQueueDoesNotAnnounceBackports(self): |
| 1419 | - """Check if BACKPORTS acceptance are not announced publicly. |
| 1420 | - |
| 1421 | - Queue tool normally announce acceptance in the specified changeslist |
| 1422 | - for the distroseries in question, however BACKPORTS announce doesn't |
| 1423 | - fit very well in that list, they cause unwanted noise. |
| 1424 | - |
| 1425 | - Further details in bug #59443 |
| 1426 | - """ |
| 1427 | - with lp_dbuser(): |
| 1428 | - # Make breezy-autotest CURRENT in order to accept upload |
| 1429 | - # to BACKPORTS. |
| 1430 | - breezy_autotest = getUtility( |
| 1431 | - IDistributionSet)['ubuntu']['breezy-autotest'] |
| 1432 | - breezy_autotest.status = SeriesStatus.CURRENT |
| 1433 | - |
| 1434 | - # Store the targeted queue item for future inspection. |
| 1435 | - # Ensure it is what we expect. |
| 1436 | - target_queue = breezy_autotest.getPackageUploads( |
| 1437 | - status=PackageUploadStatus.UNAPPROVED, |
| 1438 | - pocket=PackagePublishingPocket.BACKPORTS)[0] |
| 1439 | - self.assertEqual(10, target_queue.id) |
| 1440 | - |
| 1441 | - # Ensure breezy-autotest is set. |
| 1442 | - self.assertEqual( |
| 1443 | - u'autotest_changes@ubuntu.com', breezy_autotest.changeslist) |
| 1444 | - |
| 1445 | - # Accept the sampledata item. |
| 1446 | - queue_action = self.execute_command( |
| 1447 | - 'accept', queue_name='unapproved', |
| 1448 | - suite_name='breezy-autotest-backports', no_mail=False) |
| 1449 | - |
| 1450 | - # Only one item considered. |
| 1451 | - self.assertEqual(1, queue_action.items_size) |
| 1452 | - |
| 1453 | - # Previously stored reference should have new state now |
| 1454 | - self.assertEqual('ACCEPTED', target_queue.status.name) |
| 1455 | - |
| 1456 | - # Only one email is sent to the changed-by email on the changes |
| 1457 | - # file. No announcement email is sent. |
| 1458 | - self.assertEqual(len(stub.test_emails), 1) |
| 1459 | - self.assertEmail( |
| 1460 | - ['Daniel Silverstone <daniel.silverstone@canonical.com>']) |
| 1461 | - |
| 1462 | - def testQueueDoesNotSendAnyEmailsForTranslations(self): |
| 1463 | - """Check if no emails are sent when accepting translations. |
| 1464 | - |
| 1465 | - Queue tool should not send any emails to source uploads targeted to |
| 1466 | - 'translation' section. |
| 1467 | - They are the 'language-pack-*' and 'language-support-*' sources. |
| 1468 | - |
| 1469 | - Further details in bug #57708 |
| 1470 | - """ |
| 1471 | - with lp_dbuser(): |
| 1472 | - # Make breezy-autotest CURRENT in order to accept upload |
| 1473 | - # to PROPOSED. |
| 1474 | - breezy_autotest = getUtility( |
| 1475 | - IDistributionSet)['ubuntu']['breezy-autotest'] |
| 1476 | - breezy_autotest.status = SeriesStatus.CURRENT |
| 1477 | - |
| 1478 | - # Store the targeted queue item for future inspection. |
| 1479 | - # Ensure it is what we expect. |
| 1480 | - target_queue = breezy_autotest.getPackageUploads( |
| 1481 | - status=PackageUploadStatus.UNAPPROVED, |
| 1482 | - pocket=PackagePublishingPocket.PROPOSED)[0] |
| 1483 | - self.assertEqual(12, target_queue.id) |
| 1484 | - source = target_queue.sources[0].sourcepackagerelease |
| 1485 | - self.assertEqual('translations', source.section.name) |
| 1486 | - |
| 1487 | - # Accept the sampledata item. |
| 1488 | - queue_action = self.execute_command( |
| 1489 | - 'accept', queue_name='unapproved', |
| 1490 | - suite_name='breezy-autotest-proposed', no_mail=False) |
| 1491 | - |
| 1492 | - # Only one item considered. |
| 1493 | - self.assertEqual(1, queue_action.items_size) |
| 1494 | - |
| 1495 | - # Previously stored reference should have new state now. |
| 1496 | - self.assertEqual('DONE', target_queue.status.name) |
| 1497 | - |
| 1498 | - # No email was sent. |
| 1499 | - self.assertEqual(0, len(stub.test_emails)) |
| 1500 | - |
| 1501 | - def assertQueueLength(self, expected_length, distro_series, status, name): |
| 1502 | - queue_items = distro_series.getPackageUploads( |
| 1503 | - status=status, name=name) |
| 1504 | - self.assertEqual(expected_length, queue_items.count()) |
| 1505 | - |
| 1506 | - def assertErrorAcceptingDuplicate(self): |
| 1507 | - self.assertTrue( |
| 1508 | - '** cnews could not be accepted due to ' |
| 1509 | - 'The source cnews - 1.0 is already accepted in ubuntu/' |
| 1510 | - 'breezy-autotest and you cannot upload the same version ' |
| 1511 | - 'within the same distribution. You have to modify the source ' |
| 1512 | - 'version and re-upload.' in self.test_output) |
| 1513 | - |
| 1514 | - def testAcceptanceWorkflowForDuplications(self): |
| 1515 | - """Check how queue tool behaves dealing with duplicated entries. |
| 1516 | - |
| 1517 | - Sampledata provides a duplication of cnews_1.0 in breezy-autotest |
| 1518 | - UNAPPROVED queue. |
| 1519 | - |
| 1520 | - Step 1: executing 'accept cnews in unapproved queue' with duplicate |
| 1521 | - cnews items in the UNAPPROVED queue, results in the oldest being |
| 1522 | - accepted and the newer one remaining UNAPPROVED (and displaying |
| 1523 | - an error about it to the user). |
| 1524 | - |
| 1525 | - Step 2: executing 'accept cnews in unapproved queue' with duplicate |
| 1526 | - cnews items in the UNAPPROVED and ACCEPTED queues has no effect on |
| 1527 | - the queues, and again displays an error to the user. |
| 1528 | - |
| 1529 | - Step 3: executing 'accept cnews in unapproved queue' with duplicate |
| 1530 | - cnews items in the UNAPPROVED and DONE queues behaves the same as 2. |
| 1531 | - |
| 1532 | - Step 4: the remaining duplicated cnews item in UNAPPROVED queue can |
| 1533 | - only be rejected. |
| 1534 | - """ |
| 1535 | - with lp_dbuser(): |
| 1536 | - # Add a chroot to breezy-autotest/i386, so the system can create |
| 1537 | - # builds for it. |
| 1538 | - a_file = getUtility(ILibraryFileAliasSet)[1] |
| 1539 | - breezy_autotest = getUtility( |
| 1540 | - IDistributionSet)['ubuntu']['breezy-autotest'] |
| 1541 | - breezy_autotest['i386'].addOrUpdateChroot(a_file) |
| 1542 | - |
| 1543 | - # Certify we have a 'cnews' upload duplication in UNAPPROVED. |
| 1544 | - self.assertQueueLength( |
| 1545 | - 2, breezy_autotest, PackageUploadStatus.UNAPPROVED, u"cnews") |
| 1546 | - |
| 1547 | - # Step 1: try to accept both. |
| 1548 | - self.execute_command( |
| 1549 | - 'accept cnews', queue_name='unapproved', |
| 1550 | - suite_name='breezy-autotest') |
| 1551 | - |
| 1552 | - # The first item, being a single source upload, is automatically |
| 1553 | - # published when it's accepted. |
| 1554 | - self.assertQueueLength( |
| 1555 | - 1, breezy_autotest, PackageUploadStatus.DONE, u"cnews") |
| 1556 | - |
| 1557 | - # The last can't be accepted and remains in UNAPPROVED. |
| 1558 | - self.assertErrorAcceptingDuplicate() |
| 1559 | - self.assertQueueLength( |
| 1560 | - 1, breezy_autotest, PackageUploadStatus.UNAPPROVED, u"cnews") |
| 1561 | - |
| 1562 | - # Step 2: try to accept the remaining item in UNAPPROVED. |
| 1563 | - self.execute_command( |
| 1564 | - 'accept cnews', queue_name='unapproved', |
| 1565 | - suite_name='breezy-autotest') |
| 1566 | - self.assertErrorAcceptingDuplicate() |
| 1567 | - self.assertQueueLength( |
| 1568 | - 1, breezy_autotest, PackageUploadStatus.UNAPPROVED, u"cnews") |
| 1569 | - |
| 1570 | - # Step 3: try to accept the remaining item in UNAPPROVED with the |
| 1571 | - # duplication already in DONE. |
| 1572 | - self.execute_command( |
| 1573 | - 'accept cnews', queue_name='unapproved', |
| 1574 | - suite_name='breezy-autotest') |
| 1575 | - # It failed and te item remains in UNAPPROVED. |
| 1576 | - self.assertErrorAcceptingDuplicate() |
| 1577 | - self.assertQueueLength( |
| 1578 | - 1, breezy_autotest, PackageUploadStatus.UNAPPROVED, u"cnews") |
| 1579 | - |
| 1580 | - # Step 4: The only possible destiny for the remaining item it REJECT. |
| 1581 | - self.execute_command( |
| 1582 | - 'reject cnews', queue_name='unapproved', |
| 1583 | - suite_name='breezy-autotest') |
| 1584 | - self.assertQueueLength( |
| 1585 | - 0, breezy_autotest, PackageUploadStatus.UNAPPROVED, u"cnews") |
| 1586 | - self.assertQueueLength( |
| 1587 | - 1, breezy_autotest, PackageUploadStatus.REJECTED, u"cnews") |
| 1588 | - |
| 1589 | - def testRejectSourceSendsEmail(self): |
| 1590 | - """Check that rejecting a source upload sends email.""" |
| 1591 | - queue_action = self.execute_command( |
| 1592 | - 'reject alsa-utils', no_mail=False) |
| 1593 | - self.assertEqual(1, queue_action.items_size) |
| 1594 | - self.assertEqual(1, len(stub.test_emails)) |
| 1595 | - self.assertEmail( |
| 1596 | - ['Daniel Silverstone <daniel.silverstone@canonical.com>']) |
| 1597 | - |
| 1598 | - def testRejectBinarySendsEmail(self): |
| 1599 | - """Check that rejecting a binary upload sends email.""" |
| 1600 | - queue_action = self.execute_command('reject 2', no_mail=False) |
| 1601 | - self.assertEqual(1, queue_action.items_size) |
| 1602 | - self.assertEqual(1, len(stub.test_emails)) |
| 1603 | - self.assertEmail( |
| 1604 | - ['Daniel Silverstone <daniel.silverstone@canonical.com>']) |
| 1605 | - |
| 1606 | - def testRejectLangpackSendsNoEmail(self): |
| 1607 | - """Check that rejecting a language pack sends no email.""" |
| 1608 | - queue_action = self.execute_command( |
| 1609 | - 'reject language-pack-de', queue_name='unapproved', |
| 1610 | - suite_name='breezy-autotest-proposed') |
| 1611 | - self.assertEqual(1, queue_action.items_size) |
| 1612 | - self.assertEqual(0, len(stub.test_emails)) |
| 1613 | - |
| 1614 | - def testRejectWithMultipleIDs(self): |
| 1615 | - """Check if rejecting multiple items at once works. |
| 1616 | - |
| 1617 | - We can specify multiple items to reject, even mixing IDs and names. |
| 1618 | - e.g. queue reject alsa-utils 1 3 |
| 1619 | - """ |
| 1620 | - breezy_autotest = getUtility( |
| 1621 | - IDistributionSet)['ubuntu']['breezy-autotest'] |
| 1622 | - |
| 1623 | - # Run the command. |
| 1624 | - queue_action = self.execute_command('reject 1 pmount 3') |
| 1625 | - |
| 1626 | - # Test what it did. Since all the queue items came out of the |
| 1627 | - # NEW queue originally, the items processed should now be REJECTED. |
| 1628 | - self.assertEqual(3, queue_action.items_size) |
| 1629 | - self.assertQueueLength(1, breezy_autotest, |
| 1630 | - PackageUploadStatus.REJECTED, u'mozilla-firefox') |
| 1631 | - self.assertQueueLength(1, breezy_autotest, |
| 1632 | - PackageUploadStatus.REJECTED, u'pmount') |
| 1633 | - self.assertQueueLength(1, breezy_autotest, |
| 1634 | - PackageUploadStatus.REJECTED, u'netapplet') |
| 1635 | - |
| 1636 | - def testOverrideSource(self): |
| 1637 | - """Check if overriding sources works. |
| 1638 | - |
| 1639 | - We can specify multiple items to override, even mixing IDs and names. |
| 1640 | - e.g. queue override source -c restricted alsa-utils 1 3 |
| 1641 | - """ |
| 1642 | - # Set up. |
| 1643 | - breezy_autotest = getUtility( |
| 1644 | - IDistributionSet)['ubuntu']['breezy-autotest'] |
| 1645 | - |
| 1646 | - # Basic operation overriding a single source 'alsa-utils' that |
| 1647 | - # is currently main/base in the sample data. |
| 1648 | - queue_action = self.execute_command('override source 4', |
| 1649 | - component_name='restricted', section_name='web') |
| 1650 | - self.assertEqual(1, queue_action.items_size) |
| 1651 | - queue_item = breezy_autotest.getPackageUploads( |
| 1652 | - status=PackageUploadStatus.NEW, name=u"alsa-utils")[0] |
| 1653 | - [source] = queue_item.sources |
| 1654 | - self.assertEqual('restricted', |
| 1655 | - source.sourcepackagerelease.component.name) |
| 1656 | - self.assertEqual('web', |
| 1657 | - source.sourcepackagerelease.section.name) |
| 1658 | - |
| 1659 | - # Override multiple sources at once and mix ID with name. |
| 1660 | - queue_action = self.execute_command('override source 4 netapplet', |
| 1661 | - component_name='universe', section_name='editors') |
| 1662 | - # 'netapplet' appears 3 times, alsa-utils once. |
| 1663 | - self.assertEqual(4, queue_action.items_size) |
| 1664 | - self.assertEqual(2, queue_action.overrides_performed) |
| 1665 | - # Check results. |
| 1666 | - queue_items = list(breezy_autotest.getPackageUploads( |
| 1667 | - status=PackageUploadStatus.NEW, name=u'alsa-utils')) |
| 1668 | - queue_items.extend(list(breezy_autotest.getPackageUploads( |
| 1669 | - status=PackageUploadStatus.NEW, name=u'netapplet'))) |
| 1670 | - for queue_item in queue_items: |
| 1671 | - if queue_item.sources: |
| 1672 | - [source] = queue_item.sources |
| 1673 | - self.assertEqual('universe', |
| 1674 | - source.sourcepackagerelease.component.name) |
| 1675 | - self.assertEqual('editors', |
| 1676 | - source.sourcepackagerelease.section.name) |
| 1677 | - |
| 1678 | - def testOverrideSourceWithArchiveChange(self): |
| 1679 | - """Check if the archive changes as necessary on a source override. |
| 1680 | - |
| 1681 | - When overriding the component, the archive may change, so we check |
| 1682 | - that here. |
| 1683 | - """ |
| 1684 | - # Set up. |
| 1685 | - ubuntu = getUtility(IDistributionSet)['ubuntu'] |
| 1686 | - breezy_autotest = ubuntu['breezy-autotest'] |
| 1687 | - |
| 1688 | - # Test that it changes to partner when required. |
| 1689 | - queue_action = self.execute_command('override source alsa-utils', |
| 1690 | - component_name='partner') |
| 1691 | - self.assertEqual(1, queue_action.items_size) |
| 1692 | - [queue_item] = breezy_autotest.getPackageUploads( |
| 1693 | - status=PackageUploadStatus.NEW, name=u"alsa-utils") |
| 1694 | - [source] = queue_item.sources |
| 1695 | - self.assertEqual(source.sourcepackagerelease.upload_archive.purpose, |
| 1696 | - ArchivePurpose.PARTNER) |
| 1697 | - |
| 1698 | - # Test that it changes back to primary when required. |
| 1699 | - queue_action = self.execute_command('override source alsa-utils', |
| 1700 | - component_name='main') |
| 1701 | - self.assertEqual(1, queue_action.items_size) |
| 1702 | - [queue_item] = breezy_autotest.getPackageUploads( |
| 1703 | - status=PackageUploadStatus.NEW, name=u"alsa-utils") |
| 1704 | - [source] = queue_item.sources |
| 1705 | - self.assertEqual(source.sourcepackagerelease.upload_archive.purpose, |
| 1706 | - ArchivePurpose.PRIMARY) |
| 1707 | - |
| 1708 | - def testOverrideSourceWithNonexistentArchiveChange(self): |
| 1709 | - """Check that overriding to a non-existent archive fails properly. |
| 1710 | - |
| 1711 | - When overriding the component, the archive may change to a |
| 1712 | - non-existent one so ensure if fails. |
| 1713 | - """ |
| 1714 | - with lp_dbuser(): |
| 1715 | - ubuntu = getUtility(IDistributionSet)['ubuntu'] |
| 1716 | - proxied_archive = getUtility(IArchiveSet).getByDistroPurpose( |
| 1717 | - ubuntu, ArchivePurpose.PARTNER) |
| 1718 | - comm_archive = removeSecurityProxy(proxied_archive) |
| 1719 | - comm_archive.purpose = ArchivePurpose.PPA |
| 1720 | - |
| 1721 | - self.assertRaises(CommandRunnerError, |
| 1722 | - self.execute_command, |
| 1723 | - 'override source alsa-utils', |
| 1724 | - component_name='partner') |
| 1725 | - |
| 1726 | - def testOverrideBinary(self): |
| 1727 | - """Check if overriding binaries works. |
| 1728 | - |
| 1729 | - We can specify multiple items to override, even mixing IDs and names. |
| 1730 | - e.g. queue override binary -c restricted alsa-utils 1 3 |
| 1731 | - """ |
| 1732 | - # Set up. |
| 1733 | - breezy_autotest = getUtility( |
| 1734 | - IDistributionSet)['ubuntu']['breezy-autotest'] |
| 1735 | - |
| 1736 | - # Override a binary, 'pmount', from its sample data of |
| 1737 | - # main/base/IMPORTANT to restricted/web/extra. |
| 1738 | - queue_action = self.execute_command('override binary pmount', |
| 1739 | - component_name='restricted', section_name='web', |
| 1740 | - priority_name='extra') |
| 1741 | - self.assertEqual(1, queue_action.items_size) |
| 1742 | - [queue_item] = breezy_autotest.getPackageUploads( |
| 1743 | - status=PackageUploadStatus.NEW, name=u"pmount") |
| 1744 | - [packagebuild] = queue_item.builds |
| 1745 | - for package in packagebuild.build.binarypackages: |
| 1746 | - self.assertEqual('restricted', package.component.name) |
| 1747 | - self.assertEqual('web', package.section.name) |
| 1748 | - self.assertEqual('EXTRA', package.priority.name) |
| 1749 | - |
| 1750 | - # Override multiple binaries at once. |
| 1751 | - queue_action = self.execute_command( |
| 1752 | - 'override binary pmount mozilla-firefox', |
| 1753 | - component_name='universe', section_name='editors', |
| 1754 | - priority_name='optional') |
| 1755 | - # Check results. |
| 1756 | - self.assertEqual(2, queue_action.items_size) |
| 1757 | - self.assertEqual(2, queue_action.overrides_performed) |
| 1758 | - queue_items = list(breezy_autotest.getPackageUploads( |
| 1759 | - status=PackageUploadStatus.NEW, name=u'pmount')) |
| 1760 | - queue_items.extend(list(breezy_autotest.getPackageUploads( |
| 1761 | - status=PackageUploadStatus.NEW, name=u'mozilla-firefox'))) |
| 1762 | - for queue_item in queue_items: |
| 1763 | - [packagebuild] = queue_item.builds |
| 1764 | - for package in packagebuild.build.binarypackages: |
| 1765 | - self.assertEqual('universe', package.component.name) |
| 1766 | - self.assertEqual('editors', package.section.name) |
| 1767 | - self.assertEqual('OPTIONAL', package.priority.name) |
| 1768 | - |
| 1769 | - # Check that overriding by ID is warned to the user. |
| 1770 | - self.assertRaises( |
| 1771 | - CommandRunnerError, self.execute_command, 'override binary 1', |
| 1772 | - component_name='multiverse') |
| 1773 | - |
| 1774 | - def testOverridingMulipleBinariesFromSameBuild(self): |
| 1775 | - """Check that multiple binary override works for the same build. |
| 1776 | - |
| 1777 | - Overriding binary packages generated from the same build should |
| 1778 | - override each package individually. |
| 1779 | - """ |
| 1780 | - # Start off by setting up a packageuploadbuild that points to |
| 1781 | - # a build with two binaries. |
| 1782 | - switch_dbuser("launchpad") |
| 1783 | - |
| 1784 | - breezy_autotest = getUtility( |
| 1785 | - IDistributionSet)['ubuntu']['breezy-autotest'] |
| 1786 | - [mozilla_queue_item] = breezy_autotest.getPackageUploads( |
| 1787 | - status=PackageUploadStatus.NEW, name=u'mozilla-firefox') |
| 1788 | - |
| 1789 | - # The build with ID '2' is for mozilla-firefox, which produces |
| 1790 | - # binaries for 'mozilla-firefox' and 'mozilla-firefox-data'. |
| 1791 | - PackageUploadBuild(packageupload=mozilla_queue_item, build=2) |
| 1792 | - |
| 1793 | - # Switching db users starts a new transaction. We must re-fetch |
| 1794 | - # breezy-autotest. |
| 1795 | - switch_dbuser("queued") |
| 1796 | - breezy_autotest = getUtility( |
| 1797 | - IDistributionSet)['ubuntu']['breezy-autotest'] |
| 1798 | - |
| 1799 | - queue_action = self.execute_command( |
| 1800 | - 'override binary mozilla-firefox-data mozilla-firefox', |
| 1801 | - component_name='restricted', section_name='editors', |
| 1802 | - priority_name='optional') |
| 1803 | - |
| 1804 | - # There are three binaries to override on this PackageUpload: |
| 1805 | - # - mozilla-firefox in breezy-autotest |
| 1806 | - # - mozilla-firefox and mozilla-firefox-data in warty |
| 1807 | - # Each should be overridden exactly once. |
| 1808 | - self.assertEqual(1, queue_action.items_size) |
| 1809 | - self.assertEqual(3, queue_action.overrides_performed) |
| 1810 | - |
| 1811 | - queue_items = list(breezy_autotest.getPackageUploads( |
| 1812 | - status=PackageUploadStatus.NEW, name=u'mozilla-firefox-data')) |
| 1813 | - queue_items.extend(list(breezy_autotest.getPackageUploads( |
| 1814 | - status=PackageUploadStatus.NEW, name=u'mozilla-firefox'))) |
| 1815 | - for queue_item in queue_items: |
| 1816 | - for packagebuild in queue_item.builds: |
| 1817 | - for package in packagebuild.build.binarypackages: |
| 1818 | - self.assertEqual( |
| 1819 | - 'restricted', package.component.name, |
| 1820 | - "The component '%s' is not the expected 'restricted'" |
| 1821 | - "for package %s" % ( |
| 1822 | - package.component.name, package.name)) |
| 1823 | - self.assertEqual( |
| 1824 | - 'editors', package.section.name, |
| 1825 | - "The section '%s' is not the expected 'editors'" |
| 1826 | - "for package %s" % ( |
| 1827 | - package.section.name, package.name)) |
| 1828 | - self.assertEqual( |
| 1829 | - 'OPTIONAL', package.priority.name, |
| 1830 | - "The priority '%s' is not the expected 'OPTIONAL'" |
| 1831 | - "for package %s" % ( |
| 1832 | - package.section.name, package.name)) |
| 1833 | - |
| 1834 | - def testOverrideBinaryWithArchiveChange(self): |
| 1835 | - """Check if archive changes are disallowed for binary overrides. |
| 1836 | - |
| 1837 | - When overriding the component, the archive may change, so we check |
| 1838 | - that here and make sure it's disallowed. |
| 1839 | - """ |
| 1840 | - # Test that it changes to partner when required. |
| 1841 | - self.assertRaises( |
| 1842 | - CommandRunnerError, self.execute_command, |
| 1843 | - 'override binary pmount', component_name='partner') |
| 1844 | - |
| 1845 | - |
| 1846 | -class TestQueueActionLite(TestCaseWithFactory): |
| 1847 | - """A lightweight unit test case for `QueueAction`. |
| 1848 | - |
| 1849 | - Meant for detailed tests that would be too expensive for full end-to-end |
| 1850 | - tests. |
| 1851 | - """ |
| 1852 | - |
| 1853 | - layer = LaunchpadZopelessLayer |
| 1854 | - |
| 1855 | - def makeQueueAction(self, package_upload, distroseries=None, |
| 1856 | - component=None, section=None, |
| 1857 | - action_type=QueueAction): |
| 1858 | - """Create a `QueueAction` for use with a `PackageUpload`. |
| 1859 | - |
| 1860 | - The action's `display` method is set to a `FakeMethod`. |
| 1861 | - """ |
| 1862 | - if distroseries is None: |
| 1863 | - distroseries = self.factory.makeDistroSeries( |
| 1864 | - status=SeriesStatus.CURRENT, |
| 1865 | - name="distroseriestestingpcjs") |
| 1866 | - distro = distroseries.distribution |
| 1867 | - if package_upload is None: |
| 1868 | - package_upload = self.factory.makePackageUpload( |
| 1869 | - distroseries=distroseries, archive=distro.main_archive) |
| 1870 | - if component is None: |
| 1871 | - component = self.factory.makeComponent() |
| 1872 | - if section is None: |
| 1873 | - section = self.factory.makeSection() |
| 1874 | - queue = PackageUploadStatus.NEW |
| 1875 | - priority_name = "STANDARD" |
| 1876 | - display = FakeMethod() |
| 1877 | - terms = ['*'] |
| 1878 | - return action_type( |
| 1879 | - distro.name, distroseries.name, queue, terms, component.name, |
| 1880 | - section.name, priority_name, display) |
| 1881 | - |
| 1882 | - def makeQueueActionOverride(self, package_upload, component, section, |
| 1883 | - distroseries=None): |
| 1884 | - return self.makeQueueAction( |
| 1885 | - package_upload, distroseries, component, section, |
| 1886 | - action_type=QueueActionOverride) |
| 1887 | - |
| 1888 | - def parseUploadSummaryLine(self, output_line): |
| 1889 | - """Parse an output line from `QueueAction.displayItem`. |
| 1890 | - |
| 1891 | - :param output_line: A line of output text from `displayItem`. |
| 1892 | - :return: A tuple of displayed items: (id, tag, name, version, age). |
| 1893 | - """ |
| 1894 | - return tuple(item.strip() for item in output_line.split('|')) |
| 1895 | - |
| 1896 | - def test_display_actions_have_privileges_for_PackageCopyJob(self): |
| 1897 | - # The methods that display uploads have privileges to work with |
| 1898 | - # a PackageUpload that has a copy job. |
| 1899 | - # Bundling tests for multiple operations into one test because |
| 1900 | - # the database user change requires a costly commit. |
| 1901 | - upload = self.factory.makeCopyJobPackageUpload() |
| 1902 | - action = self.makeQueueAction(upload) |
| 1903 | - switch_dbuser(config.uploadqueue.dbuser) |
| 1904 | - |
| 1905 | - action.displayItem(upload) |
| 1906 | - self.assertNotEqual(0, action.display.call_count) |
| 1907 | - action.display.calls = [] |
| 1908 | - action.displayInfo(upload) |
| 1909 | - self.assertNotEqual(0, action.display.call_count) |
| 1910 | - |
| 1911 | - def test_accept_actions_have_privileges_for_PackageCopyJob(self): |
| 1912 | - # The script also has privileges to approve uploads that have |
| 1913 | - # copy jobs. |
| 1914 | - distroseries = self.factory.makeDistroSeries( |
| 1915 | - status=SeriesStatus.CURRENT) |
| 1916 | - upload = self.factory.makeCopyJobPackageUpload(distroseries) |
| 1917 | - switch_dbuser(config.uploadqueue.dbuser) |
| 1918 | - upload.acceptFromQueue(DevNullLogger(), dry_run=True) |
| 1919 | - # Flush changes to make sure we're not caching any updates that |
| 1920 | - # the database won't allow. If this passes, we've got the |
| 1921 | - # privileges. |
| 1922 | - IStore(upload).flush() |
| 1923 | - |
| 1924 | - def test_displayItem_displays_PackageUpload_with_source(self): |
| 1925 | - # displayItem can display a source package upload. |
| 1926 | - upload = self.factory.makeSourcePackageUpload() |
| 1927 | - action = self.makeQueueAction(upload) |
| 1928 | - |
| 1929 | - action.displayItem(upload) |
| 1930 | - |
| 1931 | - ((output, ), kwargs) = action.display.calls[0] |
| 1932 | - (upload_id, tag, name, version, age) = self.parseUploadSummaryLine( |
| 1933 | - output) |
| 1934 | - self.assertEqual(str(upload.id), upload_id) |
| 1935 | - self.assertEqual("S-", tag) |
| 1936 | - self.assertThat(upload.displayname, StartsWith(name)) |
| 1937 | - self.assertThat(upload.package_version, StartsWith(version)) |
| 1938 | - |
| 1939 | - def test_displayItem_displays_PackageUpload_with_PackageCopyJob(self): |
| 1940 | - # displayItem can display a copy-job package upload. |
| 1941 | - upload = self.factory.makeCopyJobPackageUpload() |
| 1942 | - action = self.makeQueueAction(upload) |
| 1943 | - |
| 1944 | - action.displayItem(upload) |
| 1945 | - |
| 1946 | - ((output, ), kwargs) = action.display.calls[0] |
| 1947 | - (upload_id, tag, name, version, age) = self.parseUploadSummaryLine( |
| 1948 | - output) |
| 1949 | - self.assertEqual(str(upload.id), upload_id) |
| 1950 | - self.assertEqual("X-", tag) |
| 1951 | - self.assertThat(upload.displayname, StartsWith(name)) |
| 1952 | - self.assertThat(upload.package_version, StartsWith(version)) |
| 1953 | - |
| 1954 | - def test_override_works_with_PackageCopyJob(self): |
| 1955 | - # "Sync" PackageUploads can be overridden just like sources, |
| 1956 | - # test that here. |
| 1957 | - new_component = self.factory.makeComponent() |
| 1958 | - new_section = self.factory.makeSection() |
| 1959 | - pocket = PackagePublishingPocket.RELEASE |
| 1960 | - upload = self.factory.makeCopyJobPackageUpload(target_pocket=pocket) |
| 1961 | - action = self.makeQueueActionOverride( |
| 1962 | - upload, new_component, new_section, |
| 1963 | - distroseries=upload.distroseries) |
| 1964 | - # Patch this out because it uses data we don't have in the test; |
| 1965 | - # it's unnecessary anyway. |
| 1966 | - self.patch(action, "displayTitle", FakeMethod) |
| 1967 | - action.terms = ["source", str(upload.id)] |
| 1968 | - switch_dbuser(config.uploadqueue.dbuser) |
| 1969 | - action.initialize() |
| 1970 | - action.run() |
| 1971 | - |
| 1972 | - # Overriding a sync means putting the overrides in the job itself. |
| 1973 | - self.assertEqual( |
| 1974 | - new_component.name, upload.package_copy_job.component_name) |
| 1975 | - self.assertEqual( |
| 1976 | - new_section.name, upload.package_copy_job.section_name) |
| 1977 | - |
| 1978 | - def test_makeTag_returns_S_for_source_upload(self): |
| 1979 | - upload = self.factory.makeSourcePackageUpload() |
| 1980 | - self.assertEqual('S-', self.makeQueueAction(upload)._makeTag(upload)) |
| 1981 | - |
| 1982 | - def test_makeTag_returns_B_for_binary_upload(self): |
| 1983 | - upload = self.factory.makeBuildPackageUpload() |
| 1984 | - self.assertEqual('-B', self.makeQueueAction(upload)._makeTag(upload)) |
| 1985 | - |
| 1986 | - def test_makeTag_returns_SB_for_mixed_upload(self): |
| 1987 | - upload = self.factory.makeSourcePackageUpload() |
| 1988 | - upload.addBuild(self.factory.makeBinaryPackageBuild()) |
| 1989 | - self.assertEqual('SB', self.makeQueueAction(upload)._makeTag(upload)) |
| 1990 | - |
| 1991 | - def test_makeTag_returns_X_for_copy_job_upload(self): |
| 1992 | - upload = self.factory.makeCopyJobPackageUpload() |
| 1993 | - self.assertEqual('X-', self.makeQueueAction(upload)._makeTag(upload)) |
| 1994 | - |
| 1995 | - def test_makeTag_returns_dashes_for_custom_upload(self): |
| 1996 | - upload = self.factory.makeCustomPackageUpload() |
| 1997 | - self.assertEqual('--', self.makeQueueAction(upload)._makeTag(upload)) |
| 1998 | - |
| 1999 | - def test_displayInfo_displays_PackageUpload_with_source(self): |
| 2000 | - # displayInfo can display a source package upload. |
| 2001 | - upload = self.factory.makeSourcePackageUpload() |
| 2002 | - action = self.makeQueueAction(upload) |
| 2003 | - action.displayInfo(upload) |
| 2004 | - self.assertNotEqual(0, action.display.call_count) |
| 2005 | - |
| 2006 | - def test_displayInfo_displays_PackageUpload_with_PackageCopyJob(self): |
| 2007 | - # displayInfo can display a copy-job package upload. |
| 2008 | - upload = self.factory.makeCopyJobPackageUpload() |
| 2009 | - action = self.makeQueueAction(upload) |
| 2010 | - action.displayInfo(upload) |
| 2011 | - self.assertNotEqual(0, action.display.call_count) |
| 2012 | - |
| 2013 | - |
| 2014 | -class TestQueuePageClosingBugs(TestCaseWithFactory): |
| 2015 | - # The distroseries +queue page can close bug when accepting |
| 2016 | - # packages. Unit tests for that belong here. |
| 2017 | - |
| 2018 | - layer = DatabaseFunctionalLayer |
| 2019 | - |
| 2020 | - def test_close_bugs_for_sourcepackagerelease_with_private_bug(self): |
| 2021 | - # lp.soyuz.scripts.processaccepted.close_bugs_for_sourcepackagerelease |
| 2022 | - # should work with private bugs where the person using the queue |
| 2023 | - # page doesn't have access to it. |
| 2024 | - changes_file_template = "Format: 1.7\nLaunchpad-bugs-fixed: %s\n" |
| 2025 | - # changelog_entry is required for an assertion inside the function |
| 2026 | - # we're testing. |
| 2027 | - spr = self.factory.makeSourcePackageRelease(changelog_entry="blah") |
| 2028 | - archive_admin = self.factory.makePerson() |
| 2029 | - series = spr.upload_distroseries |
| 2030 | - dsp = series.distribution.getSourcePackage(spr.sourcepackagename) |
| 2031 | - bug = self.factory.makeBug( |
| 2032 | - target=dsp, information_type=InformationType.USERDATA) |
| 2033 | - changes = StringIO(changes_file_template % bug.id) |
| 2034 | - |
| 2035 | - with person_logged_in(archive_admin): |
| 2036 | - # The archive admin user can't normally see this bug. |
| 2037 | - self.assertRaises(ForbiddenAttribute, bug, 'status') |
| 2038 | - # But the bug closure should work. |
| 2039 | - close_bugs_for_sourcepackagerelease(series, spr, changes) |
| 2040 | - |
| 2041 | - # Rather than closing the bugs immediately, this creates a |
| 2042 | - # ProcessAcceptedBugsJob. |
| 2043 | - with celebrity_logged_in("admin"): |
| 2044 | - self.assertEqual(BugTaskStatus.NEW, bug.default_bugtask.status) |
| 2045 | - job_source = getUtility(IProcessAcceptedBugsJobSource) |
| 2046 | - [job] = list(job_source.iterReady()) |
| 2047 | - self.assertEqual(series, job.distroseries) |
| 2048 | - self.assertEqual(spr, job.sourcepackagerelease) |
| 2049 | - self.assertEqual([bug.id], job.bug_ids) |
| 2050 | - |
| 2051 | - |
| 2052 | -class TestQueueToolInJail(TestQueueBase, TestCase): |
| 2053 | - layer = LaunchpadZopelessLayer |
| 2054 | - dbuser = config.uploadqueue.dbuser |
| 2055 | - |
| 2056 | - def setUp(self): |
| 2057 | - """Create contents in disk for librarian sampledata. |
| 2058 | - |
| 2059 | - Setup and chdir into a temp directory, a jail, where we can |
| 2060 | - control the file creation properly |
| 2061 | - """ |
| 2062 | - fillLibrarianFile(1, content='One') |
| 2063 | - fillLibrarianFile(52, content='Fifty-Two') |
| 2064 | - self._home = os.path.abspath('') |
| 2065 | - self._jail = tempfile.mkdtemp() |
| 2066 | - os.chdir(self._jail) |
| 2067 | - TestQueueBase.setUp(self) |
| 2068 | - |
| 2069 | - def tearDown(self): |
| 2070 | - """Remove test contents from disk. |
| 2071 | - |
| 2072 | - chdir back to the previous path (home) and remove the temp |
| 2073 | - directory used as jail. |
| 2074 | - """ |
| 2075 | - os.chdir(self._home) |
| 2076 | - LibrarianLayer.librarian_fixture.clear() |
| 2077 | - shutil.rmtree(self._jail) |
| 2078 | - |
| 2079 | - def _listfiles(self): |
| 2080 | - """Return a list of files present in jail.""" |
| 2081 | - return os.listdir(self._jail) |
| 2082 | - |
| 2083 | - def _getsha1(self, filename): |
| 2084 | - """Return a sha1 hex digest of a file""" |
| 2085 | - file_sha = hashlib.sha1() |
| 2086 | - opened_file = open(filename, "r") |
| 2087 | - for chunk in filechunks(opened_file): |
| 2088 | - file_sha.update(chunk) |
| 2089 | - opened_file.close() |
| 2090 | - return file_sha.hexdigest() |
| 2091 | - |
| 2092 | - def testFetchActionByIDDoNotOverwriteFilesystem(self): |
| 2093 | - """Check if queue fetch action doesn't overwrite files. |
| 2094 | - |
| 2095 | - Since we allow existence of duplications in NEW and UNAPPROVED |
| 2096 | - queues, we are able to fetch files from queue items and they'd |
| 2097 | - get overwritten causing obscure problems. |
| 2098 | - |
| 2099 | - Instead of overwrite a file in the working directory queue will |
| 2100 | - fail, raising a CommandRunnerError. |
| 2101 | - |
| 2102 | - bug 67014: Don't complain if files are the same |
| 2103 | - """ |
| 2104 | - self.execute_command('fetch 1') |
| 2105 | - self.assertEqual( |
| 2106 | - ['mozilla-firefox_0.9_i386.changes'], self._listfiles()) |
| 2107 | - |
| 2108 | - # checksum the existing file |
| 2109 | - existing_sha1 = self._getsha1(self._listfiles()[0]) |
| 2110 | - |
| 2111 | - # fetch will NOT raise and not overwrite the file in disk |
| 2112 | - self.execute_command('fetch 1') |
| 2113 | - |
| 2114 | - # checksum file again |
| 2115 | - new_sha1 = self._getsha1(self._listfiles()[0]) |
| 2116 | - |
| 2117 | - # Check that the file has not changed (we don't care if it was |
| 2118 | - # re-written, just that it's not changed) |
| 2119 | - self.assertEqual(existing_sha1, new_sha1) |
| 2120 | - |
| 2121 | - def testFetchActionRaisesErrorIfDifferentFileAlreadyFetched(self): |
| 2122 | - """Check that fetching a file that has already been fetched |
| 2123 | - raises an error if they are not the same file. (bug 67014) |
| 2124 | - """ |
| 2125 | - CLOBBERED = "you're clobbered" |
| 2126 | - |
| 2127 | - self.execute_command('fetch 1') |
| 2128 | - self.assertEqual( |
| 2129 | - ['mozilla-firefox_0.9_i386.changes'], self._listfiles()) |
| 2130 | - |
| 2131 | - # clobber the existing file, fetch it again and expect an exception |
| 2132 | - f = open(self._listfiles()[0], "w") |
| 2133 | - f.write(CLOBBERED) |
| 2134 | - f.close() |
| 2135 | - |
| 2136 | - self.assertRaises( |
| 2137 | - CommandRunnerError, self.execute_command, 'fetch 1') |
| 2138 | - |
| 2139 | - # make sure the file has not changed |
| 2140 | - f = open(self._listfiles()[0], "r") |
| 2141 | - line = f.read() |
| 2142 | - f.close() |
| 2143 | - |
| 2144 | - self.assertEqual(CLOBBERED, line) |
| 2145 | - |
| 2146 | - def testFetchActionByNameDoNotOverwriteFilesystem(self): |
| 2147 | - """Same as testFetchActionByIDDoNotOverwriteFilesystem |
| 2148 | - |
| 2149 | - The sampledata provides duplicated 'cnews' entries, filesystem |
| 2150 | - conflict will happen inside the same batch, |
| 2151 | - |
| 2152 | - Queue will fetch the oldest and raise. |
| 2153 | - """ |
| 2154 | - self.assertRaises( |
| 2155 | - CommandRunnerError, self.execute_command, 'fetch cnews', |
| 2156 | - queue_name='unapproved', suite_name='breezy-autotest') |
| 2157 | - |
| 2158 | - self.assertEqual(['netapplet-1.0.0.tar.gz'], self._listfiles()) |
| 2159 | - |
| 2160 | - def testQueueFetch(self): |
| 2161 | - """Check that a basic fetch operation works.""" |
| 2162 | - FAKE_CHANGESFILE_CONTENT = "Fake Changesfile" |
| 2163 | - FAKE_DEB_CONTENT = "Fake DEB" |
| 2164 | - fillLibrarianFile(1, FAKE_CHANGESFILE_CONTENT) |
| 2165 | - fillLibrarianFile(90, FAKE_DEB_CONTENT) |
| 2166 | - self.execute_command('fetch pmount') |
| 2167 | - |
| 2168 | - # Check the files' names. |
| 2169 | - files = sorted(self._listfiles()) |
| 2170 | - self.assertEqual( |
| 2171 | - ['netapplet-1.0.0.tar.gz', 'pmount_1.0-1_all.deb'], |
| 2172 | - files) |
| 2173 | - |
| 2174 | - # Check the files' contents. |
| 2175 | - changes_file = open('netapplet-1.0.0.tar.gz') |
| 2176 | - self.assertEqual(changes_file.read(), FAKE_CHANGESFILE_CONTENT) |
| 2177 | - changes_file.close() |
| 2178 | - debfile = open('pmount_1.0-1_all.deb') |
| 2179 | - self.assertEqual(debfile.read(), FAKE_DEB_CONTENT) |
| 2180 | - debfile.close() |
| 2181 | - |
| 2182 | - def testFetchMultipleItems(self): |
| 2183 | - """Check if fetching multiple items at once works. |
| 2184 | - |
| 2185 | - We can specify multiple items to fetch, even mixing IDs and names. |
| 2186 | - e.g. queue fetch alsa-utils 1 3 |
| 2187 | - """ |
| 2188 | - self.execute_command('fetch 3 mozilla-firefox') |
| 2189 | - files = self._listfiles() |
| 2190 | - files.sort() |
| 2191 | - self.assertEqual( |
| 2192 | - ['mozilla-firefox_0.9_i386.changes', 'netapplet-1.0.0.tar.gz'], |
| 2193 | - files) |
| 2194 | - |
| 2195 | - def testFetchWithoutChanges(self): |
| 2196 | - """Check that fetch works without a changes file (eg. from gina).""" |
| 2197 | - pus = getUtility(IDistributionSet).getByName('ubuntu').getSeries( |
| 2198 | - 'breezy-autotest').getPackageUploads(name=u'pmount') |
| 2199 | - for pu in pus: |
| 2200 | - removeSecurityProxy(pu).changesfile = None |
| 2201 | - |
| 2202 | - FAKE_DEB_CONTENT = "Fake DEB" |
| 2203 | - fillLibrarianFile(90, FAKE_DEB_CONTENT) |
| 2204 | - self.execute_command('fetch pmount') |
| 2205 | - |
| 2206 | - # Check the files' names. |
| 2207 | - files = sorted(self._listfiles()) |
| 2208 | - self.assertEqual(['pmount_1.0-1_all.deb'], files) |
| 2209 | |
| 2210 | === modified file 'lib/lp/soyuz/tests/test_packageupload.py' |
| 2211 | --- lib/lp/soyuz/tests/test_packageupload.py 2012-07-24 06:39:54 +0000 |
| 2212 | +++ lib/lp/soyuz/tests/test_packageupload.py 2012-09-28 10:09:21 +0000 |
| 2213 | @@ -12,6 +12,7 @@ |
| 2214 | urlopen, |
| 2215 | ) |
| 2216 | |
| 2217 | +from debian.deb822 import Changes |
| 2218 | from lazr.restfulclient.errors import ( |
| 2219 | BadRequest, |
| 2220 | Unauthorized, |
| 2221 | @@ -37,6 +38,7 @@ |
| 2222 | from lp.services.librarian.browser import ProxiedLibraryFileAlias |
| 2223 | from lp.services.log.logger import BufferLogger |
| 2224 | from lp.services.mail import stub |
| 2225 | +from lp.services.mail.sendmail import format_address_for_person |
| 2226 | from lp.soyuz.adapters.overrides import SourceOverride |
| 2227 | from lp.soyuz.enums import ( |
| 2228 | ArchivePurpose, |
| 2229 | @@ -412,6 +414,203 @@ |
| 2230 | self.assertEqual(current_component, spr.component) |
| 2231 | self.assertEqual(new_section, spr.section) |
| 2232 | |
| 2233 | + def makeSourcePackageUpload(self, pocket=None, sourcepackagename=None, |
| 2234 | + section_name=None, changes_dict=None): |
| 2235 | + """Make a useful source package upload for queue tests.""" |
| 2236 | + distroseries = self.test_publisher.distroseries |
| 2237 | + distroseries.changeslist = "autotest_changes@ubuntu.com" |
| 2238 | + uploader = self.factory.makePerson() |
| 2239 | + key = self.factory.makeGPGKey(owner=uploader) |
| 2240 | + changes = Changes({"Changed-By": uploader.preferredemail.email}) |
| 2241 | + if changes_dict is not None: |
| 2242 | + changes.update(changes_dict) |
| 2243 | + upload = self.factory.makePackageUpload( |
| 2244 | + archive=distroseries.main_archive, distroseries=distroseries, |
| 2245 | + pocket=pocket, changes_file_content=changes.dump().encode("UTF-8"), |
| 2246 | + signing_key=key) |
| 2247 | + spr = self.factory.makeSourcePackageRelease( |
| 2248 | + sourcepackagename=sourcepackagename, distroseries=distroseries, |
| 2249 | + component="main", section_name=section_name, |
| 2250 | + changelog_entry="dummy") |
| 2251 | + upload.addSource(spr) |
| 2252 | + spr.addFile(self.factory.makeLibraryFileAlias( |
| 2253 | + filename="%s_%s.dsc" % (spr.name, spr.version))) |
| 2254 | + transaction.commit() |
| 2255 | + return upload, uploader |
| 2256 | + |
| 2257 | + def makeBuildPackageUpload(self): |
| 2258 | + """Make a useful build package upload for queue tests.""" |
| 2259 | + distroseries = self.test_publisher.distroseries |
| 2260 | + distroseries.changeslist = "autotest_changes@ubuntu.com" |
| 2261 | + uploader = self.factory.makePerson() |
| 2262 | + key = self.factory.makeGPGKey(owner=uploader) |
| 2263 | + changes = Changes({"Changed-By": uploader.preferredemail.email}) |
| 2264 | + upload = self.factory.makePackageUpload( |
| 2265 | + archive=distroseries.main_archive, distroseries=distroseries, |
| 2266 | + changes_file_content=changes.dump().encode("UTF-8"), |
| 2267 | + signing_key=key) |
| 2268 | + build = self.factory.makeBinaryPackageBuild( |
| 2269 | + distroarchseries=self.test_publisher.breezy_autotest_i386) |
| 2270 | + upload.addBuild(build) |
| 2271 | + bpr = self.factory.makeBinaryPackageRelease(build=build) |
| 2272 | + bpr.addFile(self.factory.makeLibraryFileAlias( |
| 2273 | + filename="%s_%s_i386.deb" % (bpr.name, bpr.version))) |
| 2274 | + transaction.commit() |
| 2275 | + return upload, uploader |
| 2276 | + |
| 2277 | + def assertEmail(self, expected_to_addrs): |
| 2278 | + """Pop an email from the stub queue and check its recipients.""" |
| 2279 | + _, to_addrs, _ = stub.test_emails.pop() |
| 2280 | + self.assertEqual(expected_to_addrs, to_addrs) |
| 2281 | + |
| 2282 | + def test_acceptFromQueue_source_sends_email(self): |
| 2283 | + # Accepting a source package sends emails to the announcement list |
| 2284 | + # and the uploader. |
| 2285 | + self.test_publisher.prepareBreezyAutotest() |
| 2286 | + upload, uploader = self.makeSourcePackageUpload() |
| 2287 | + upload.acceptFromQueue() |
| 2288 | + self.assertEqual(2, len(stub.test_emails)) |
| 2289 | + # Emails sent are the announcement and the uploader's notification: |
| 2290 | + self.assertEmail(["autotest_changes@ubuntu.com"]) |
| 2291 | + self.assertEmail([format_address_for_person(uploader)]) |
| 2292 | + |
| 2293 | + def test_acceptFromQueue_source_backports_sends_no_announcement(self): |
| 2294 | + # Accepting a source package into BACKPORTS does not send an |
| 2295 | + # announcement email to the distroseries changeslist (see bug |
| 2296 | + # #59443). It still sends an acknowledgement to the uploader. |
| 2297 | + self.test_publisher.prepareBreezyAutotest() |
| 2298 | + self.test_publisher.distroseries.status = SeriesStatus.CURRENT |
| 2299 | + upload, uploader = self.makeSourcePackageUpload( |
| 2300 | + pocket=PackagePublishingPocket.BACKPORTS) |
| 2301 | + upload.acceptFromQueue() |
| 2302 | + self.assertEqual(1, len(stub.test_emails)) |
| 2303 | + # Only one email is sent, to the person in the changed-by field. No |
| 2304 | + # announcement email is sent. |
| 2305 | + self.assertEmail([format_address_for_person(uploader)]) |
| 2306 | + |
| 2307 | + def test_acceptFromQueue_source_translations_sends_no_email(self): |
| 2308 | + # Accepting source packages in the "translations" section (i.e. |
| 2309 | + # language packs) does not send any email. See bug #57708. |
| 2310 | + self.test_publisher.prepareBreezyAutotest() |
| 2311 | + self.test_publisher.distroseries.status = SeriesStatus.CURRENT |
| 2312 | + upload, _ = self.makeSourcePackageUpload( |
| 2313 | + pocket=PackagePublishingPocket.PROPOSED, |
| 2314 | + section_name="translations") |
| 2315 | + upload.acceptFromQueue() |
| 2316 | + self.assertEqual("DONE", upload.status.name) |
| 2317 | + self.assertEqual(0, len(stub.test_emails)) |
| 2318 | + |
| 2319 | + def test_acceptFromQueue_source_creates_builds(self): |
| 2320 | + # Accepting a source package creates build records. |
| 2321 | + self.test_publisher.prepareBreezyAutotest() |
| 2322 | + upload, _ = self.makeSourcePackageUpload() |
| 2323 | + upload.acceptFromQueue() |
| 2324 | + spr = upload.sourcepackagerelease |
| 2325 | + [build] = spr.builds |
| 2326 | + self.assertEqual( |
| 2327 | + "i386 build of %s %s in ubuntutest breezy-autotest RELEASE" % ( |
| 2328 | + spr.name, spr.version), |
| 2329 | + build.title) |
| 2330 | + |
| 2331 | + def test_acceptFromQueue_source_closes_bug(self): |
| 2332 | + # Accepting a source package closes bugs appropriately. |
| 2333 | + self.test_publisher.prepareBreezyAutotest() |
| 2334 | + |
| 2335 | + # Upload the first version of a package. |
| 2336 | + upload_one, _ = self.makeSourcePackageUpload() |
| 2337 | + upload_one.setAccepted() |
| 2338 | + upload_one.realiseUpload() |
| 2339 | + spr = upload_one.sourcepackagerelease |
| 2340 | + |
| 2341 | + # Make a new bug task for this package. It starts life as NEW. |
| 2342 | + dsp = self.test_publisher.ubuntutest.getSourcePackage(spr.name) |
| 2343 | + task = self.factory.makeBugTask(target=dsp, publish=False) |
| 2344 | + self.assertEqual("NEW", task.status.name) |
| 2345 | + |
| 2346 | + # Upload the next version of the same package, closing this bug. |
| 2347 | + changes = Changes({"Launchpad-Bugs-Fixed": str(task.bug.id)}) |
| 2348 | + upload_two, _ = self.makeSourcePackageUpload( |
| 2349 | + sourcepackagename=spr.sourcepackagename, changes_dict=changes) |
| 2350 | + |
| 2351 | + # Accept the new upload. It should reach the DONE state, and should |
| 2352 | + # close the bug. |
| 2353 | + upload_two.acceptFromQueue() |
| 2354 | + self.assertEqual("DONE", upload_two.status.name) |
| 2355 | + self.assertEqual("FIXRELEASED", task.status.name) |
| 2356 | + |
| 2357 | + def test_acceptFromQueue_binary_sends_no_email(self): |
| 2358 | + # Accepting a binary package does not send email. |
| 2359 | + self.test_publisher.prepareBreezyAutotest() |
| 2360 | + upload, _ = self.makeBuildPackageUpload() |
| 2361 | + upload.acceptFromQueue() |
| 2362 | + self.assertEqual(0, len(stub.test_emails)) |
| 2363 | + |
| 2364 | + def test_acceptFromQueue_handles_duplicates(self): |
| 2365 | + # Duplicate queue entries are handled sensibly. |
| 2366 | + self.test_publisher.prepareBreezyAutotest() |
| 2367 | + distroseries = self.test_publisher.distroseries |
| 2368 | + upload_one = self.factory.makePackageUpload( |
| 2369 | + archive=distroseries.main_archive, distroseries=distroseries) |
| 2370 | + upload_one.addSource(self.factory.makeSourcePackageRelease( |
| 2371 | + sourcepackagename="cnews", distroseries=distroseries, |
| 2372 | + component="main", version="1.0")) |
| 2373 | + upload_two = self.factory.makePackageUpload( |
| 2374 | + archive=distroseries.main_archive, distroseries=distroseries) |
| 2375 | + upload_two.addSource(self.factory.makeSourcePackageRelease( |
| 2376 | + sourcepackagename="cnews", distroseries=distroseries, |
| 2377 | + component="main", version="1.0")) |
| 2378 | + transaction.commit() |
| 2379 | + upload_one.setUnapproved() |
| 2380 | + upload_one.syncUpdate() |
| 2381 | + upload_two.setUnapproved() |
| 2382 | + upload_two.syncUpdate() |
| 2383 | + |
| 2384 | + # There are now duplicate uploads in UNAPPROVED. |
| 2385 | + unapproved = distroseries.getPackageUploads( |
| 2386 | + status=PackageUploadStatus.UNAPPROVED, name=u"cnews") |
| 2387 | + self.assertEqual(2, unapproved.count()) |
| 2388 | + |
| 2389 | + # Accepting one of them works. (Since it's a single source upload, |
| 2390 | + # it goes straight to DONE.) |
| 2391 | + upload_one.acceptFromQueue() |
| 2392 | + self.assertEqual("DONE", upload_one.status.name) |
| 2393 | + transaction.commit() |
| 2394 | + |
| 2395 | + # Trying to accept the second fails. |
| 2396 | + self.assertRaises( |
| 2397 | + QueueInconsistentStateError, upload_two.acceptFromQueue) |
| 2398 | + self.assertEqual("UNAPPROVED", upload_two.status.name) |
| 2399 | + |
| 2400 | + # Rejecting the second upload works. |
| 2401 | + upload_two.rejectFromQueue() |
| 2402 | + self.assertEqual("REJECTED", upload_two.status.name) |
| 2403 | + |
| 2404 | + def test_rejectFromQueue_source_sends_email(self): |
| 2405 | + # Rejecting a source package sends an email to the uploader. |
| 2406 | + self.test_publisher.prepareBreezyAutotest() |
| 2407 | + upload, uploader = self.makeSourcePackageUpload() |
| 2408 | + upload.rejectFromQueue() |
| 2409 | + self.assertEqual(1, len(stub.test_emails)) |
| 2410 | + self.assertEmail([format_address_for_person(uploader)]) |
| 2411 | + |
| 2412 | + def test_rejectFromQueue_binary_sends_email(self): |
| 2413 | + # Rejecting a binary package sends an email to the uploader. |
| 2414 | + self.test_publisher.prepareBreezyAutotest() |
| 2415 | + upload, uploader = self.makeBuildPackageUpload() |
| 2416 | + upload.rejectFromQueue() |
| 2417 | + self.assertEqual(1, len(stub.test_emails)) |
| 2418 | + self.assertEmail([format_address_for_person(uploader)]) |
| 2419 | + |
| 2420 | + def test_rejectFromQueue_source_translations_sends_no_email(self): |
| 2421 | + # Rejecting a language pack sends no email. |
| 2422 | + self.test_publisher.prepareBreezyAutotest() |
| 2423 | + self.test_publisher.distroseries.status = SeriesStatus.CURRENT |
| 2424 | + upload, _ = self.makeSourcePackageUpload( |
| 2425 | + pocket=PackagePublishingPocket.PROPOSED, |
| 2426 | + section_name="translations") |
| 2427 | + upload.rejectFromQueue() |
| 2428 | + self.assertEqual(0, len(stub.test_emails)) |
| 2429 | + |
| 2430 | |
| 2431 | class TestPackageUploadPrivacy(TestCaseWithFactory): |
| 2432 | """Test PackageUpload security.""" |
| 2433 | |
| 2434 | === removed file 'scripts/ftpmaster-tools/queue' |
| 2435 | --- scripts/ftpmaster-tools/queue 2012-06-29 08:40:05 +0000 |
| 2436 | +++ scripts/ftpmaster-tools/queue 1970-01-01 00:00:00 +0000 |
| 2437 | @@ -1,125 +0,0 @@ |
| 2438 | -#!/usr/bin/python -S |
| 2439 | -# |
| 2440 | -# Copyright 2009 Canonical Ltd. This software is licensed under the |
| 2441 | -# GNU Affero General Public License version 3 (see the file LICENSE). |
| 2442 | - |
| 2443 | -"""Queue management script |
| 2444 | - |
| 2445 | -Tool for handling and visualisation of upload queue records. |
| 2446 | -""" |
| 2447 | - |
| 2448 | -import _pythonpath |
| 2449 | - |
| 2450 | -import transaction |
| 2451 | - |
| 2452 | -from lp.services.config import config |
| 2453 | -from lp.services.scripts.base import ( |
| 2454 | - LaunchpadScript, |
| 2455 | - LaunchpadScriptFailure, |
| 2456 | - ) |
| 2457 | -from lp.soyuz.scripts.queue import ( |
| 2458 | - CommandRunner, |
| 2459 | - CommandRunnerError, |
| 2460 | - name_queue_map, |
| 2461 | - ) |
| 2462 | - |
| 2463 | - |
| 2464 | -class QueueScript(LaunchpadScript): |
| 2465 | - |
| 2466 | - usage = 'Usage: %prog [options] <command>' |
| 2467 | - |
| 2468 | - def add_my_options(self): |
| 2469 | - self.parser.add_option( |
| 2470 | - "-Q", "--queue", |
| 2471 | - dest="queue_name", metavar="QUEUE", default="new", |
| 2472 | - help="Which queue to consider") |
| 2473 | - |
| 2474 | - self.parser.add_option( |
| 2475 | - "-d", "--distribution", |
| 2476 | - dest="distribution_name", metavar="DISTRO", default=None, |
| 2477 | - help="Which distro to look in") |
| 2478 | - |
| 2479 | - self.parser.add_option( |
| 2480 | - "-s", "--suite", |
| 2481 | - dest="suite_name", metavar="DISTRORELEASE", default=None, |
| 2482 | - help=("Which distrorelease to look in, defaults " |
| 2483 | - "to distribution 'currentseries'.")) |
| 2484 | - |
| 2485 | - self.parser.add_option( |
| 2486 | - "-N", "--dry-run", action="store_true", |
| 2487 | - dest="dryrun", metavar="DRY_RUN", default=False, |
| 2488 | - help="Whether to treat this as a dry-run or not.") |
| 2489 | - |
| 2490 | - self.parser.add_option( |
| 2491 | - "-M", "--no-mail", action="store_true", |
| 2492 | - dest="nomail", metavar="NO_MAIL", default=False, |
| 2493 | - help="Whether to send announce email or not.") |
| 2494 | - |
| 2495 | - self.parser.add_option( |
| 2496 | - "-e", "--exact-match", action="store_true", |
| 2497 | - dest="exact_match", metavar="EXACTMATCH", default=False, |
| 2498 | - help="Whether treat filter as a exact match or not.") |
| 2499 | - |
| 2500 | - self.parser.add_option( |
| 2501 | - "-i", "--ignore-errors", action="store_true", |
| 2502 | - dest="ignore_errors", metavar="IGNOREERRORS", default=False, |
| 2503 | - help="Ignore errors when performing a list of commands.") |
| 2504 | - |
| 2505 | - self.parser.add_option( |
| 2506 | - "-f", "--file", metavar="FILE", default=None, |
| 2507 | - help="file containing a sequence of command lines.") |
| 2508 | - |
| 2509 | - self.parser.add_option( |
| 2510 | - "-c", "--component", dest="component_name", |
| 2511 | - metavar="COMPONENT", default=None, |
| 2512 | - help="When overriding, move package to COMPONENT") |
| 2513 | - |
| 2514 | - self.parser.add_option( |
| 2515 | - "-x", "--section", dest="section_name", |
| 2516 | - metavar="SECTION", default=None, |
| 2517 | - help="When overriding, move package to SECTION") |
| 2518 | - |
| 2519 | - self.parser.add_option( |
| 2520 | - "-p", "--priority", dest="priority_name", |
| 2521 | - metavar="PRIORITY", default=None, |
| 2522 | - help="When overriding, move package to PRIORITY") |
| 2523 | - |
| 2524 | - def main(self): |
| 2525 | - if self.options.queue_name not in name_queue_map: |
| 2526 | - self.parser.error( |
| 2527 | - 'Unable to map queue name "%s"' % self.options.queue_name) |
| 2528 | - |
| 2529 | - no_mail = self.options.dryrun or self.options.nomail |
| 2530 | - queue = name_queue_map[self.options.queue_name] |
| 2531 | - |
| 2532 | - if self.options.file: |
| 2533 | - args_list = [self.args.strip().split() for args in |
| 2534 | - open(self.options.file).readlines()] |
| 2535 | - else: |
| 2536 | - args_list = [self.args] |
| 2537 | - |
| 2538 | - cmd_runner = CommandRunner( |
| 2539 | - queue, self.options.distribution_name, self.options.suite_name, |
| 2540 | - no_mail, self.options.component_name, self.options.section_name, |
| 2541 | - self.options.priority_name, log=self.logger) |
| 2542 | - |
| 2543 | - print "Initializing connection to queue %s" % self.options.queue_name |
| 2544 | - |
| 2545 | - for single_args in args_list: |
| 2546 | - try: |
| 2547 | - cmd_runner.execute(single_args, self.options.exact_match) |
| 2548 | - except CommandRunnerError as info: |
| 2549 | - print (info) |
| 2550 | - if self.options.ignore_errors: |
| 2551 | - continue |
| 2552 | - transaction.abort() |
| 2553 | - raise LaunchpadScriptFailure( |
| 2554 | - 'Error encountered -- aborting current transaction') |
| 2555 | - else: |
| 2556 | - if not self.options.dryrun: |
| 2557 | - transaction.commit() |
| 2558 | - else: |
| 2559 | - print "DRY RUN requested, not committing." |
| 2560 | - |
| 2561 | -if __name__ == '__main__': |
| 2562 | - QueueScript('queue', config.uploadqueue.dbuser).run() |

This looks alright to me, but I have requested that William Grant take a look at it as he's far more aware of the various bits and pieces involved in this, and he's worked with you on a lot of this work.