Merge lp:~allenap/launchpad/do-copy-async-bug-766247 into lp:launchpad
- do-copy-async-bug-766247
- Merge into devel
Status: | Merged |
---|---|
Approved by: | Gavin Panella |
Approved revision: | no longer in the source branch. |
Merged at revision: | 12917 |
Proposed branch: | lp:~allenap/launchpad/do-copy-async-bug-766247 |
Merge into: | lp:launchpad |
Diff against target: |
644 lines (+250/-161) 7 files modified
configs/development/launchpad-lazr.conf (+2/-2) cronscripts/sync-packages.py (+0/-25) lib/canonical/config/schema-lazr.conf (+9/-2) lib/lp/soyuz/configure.zcml (+6/-6) lib/lp/soyuz/interfaces/distributionjob.py (+64/-33) lib/lp/soyuz/model/packagecopyjob.py (+66/-40) lib/lp/soyuz/tests/test_packagecopyjob.py (+103/-53) |
To merge this branch: | bzr merge lp:~allenap/launchpad/do-copy-async-bug-766247 |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Brad Crittenden (community) | code | Approve | |
Review via email: mp+58714@code.launchpad.net |
Commit message
[r=bac][bug=766247] Rename SyncPackageJob to PackageCopyJob, and use the lower-level do_copy() API which can perform multiple package copies at once. Previously SyncPackageJob used the IArchive.
Description of the change
This branch takes the unused SyncPackageJob, renames it to
PackageCopyJob, and changes it to use do_copy() instead of going via
Archive.
multiple packages at a time.
It also removes the existing cronscript and instead changes it to run
as part of the job source group system, in the MAIN group. This is
already scheduled to run once every five minutes on loganberry, so no
LOSA intervention is required to make this work. If we want this to
run more frequently we should create a new group, FREQUENT perhaps,
and add it to the loganberry crontab.
Gavin Panella (allenap) wrote : | # |
> * If the XXX is something we really need to do why not file a bug so we don't
> forget?
Thank you for the review, and thank you for the reminder. Until this branch lands there isn't a bug, so I wasn't quite sure when to file it. I tend to wait until the reviewer has at least said that my branch isn't the work of a crack-addled monkey, which I think you've now done... bug 770297 has been filed :)
Preview Diff
1 | === modified file 'configs/development/launchpad-lazr.conf' |
2 | --- configs/development/launchpad-lazr.conf 2011-04-21 01:13:19 +0000 |
3 | +++ configs/development/launchpad-lazr.conf 2011-04-25 14:43:28 +0000 |
4 | @@ -137,8 +137,8 @@ |
5 | oops_prefix: IDSJ |
6 | error_dir: /var/tmp/soyuz.test |
7 | |
8 | -[sync_packages] |
9 | -oops_prefix: SPJ |
10 | +[IPackageCopyJobSource] |
11 | +oops_prefix: PCJ |
12 | error_dir: /var/tmp/soyuz.test |
13 | |
14 | [launchpad] |
15 | |
16 | === removed file 'cronscripts/sync-packages.py' |
17 | --- cronscripts/sync-packages.py 2010-11-12 10:52:15 +0000 |
18 | +++ cronscripts/sync-packages.py 1970-01-01 00:00:00 +0000 |
19 | @@ -1,25 +0,0 @@ |
20 | -#!/usr/bin/python -S |
21 | -# |
22 | -# Copyright 2010 Canonical Ltd. This software is licensed under the |
23 | -# GNU Affero General Public License version 3 (see the file LICENSE). |
24 | - |
25 | -"""Synchronize packages.""" |
26 | - |
27 | -__metaclass__ = type |
28 | - |
29 | -import _pythonpath |
30 | - |
31 | -from lp.services.job.runner import JobCronScript |
32 | -from lp.soyuz.interfaces.distributionjob import ISyncPackageJobSource |
33 | - |
34 | - |
35 | -class RunSyncPackageJob(JobCronScript): |
36 | - """Run SyncPackageJob jobs.""" |
37 | - |
38 | - config_name = 'sync_packages' |
39 | - source_interface = ISyncPackageJobSource |
40 | - |
41 | - |
42 | -if __name__ == '__main__': |
43 | - script = RunSyncPackageJob() |
44 | - script.lock_and_run() |
45 | |
46 | === modified file 'lib/canonical/config/schema-lazr.conf' |
47 | --- lib/canonical/config/schema-lazr.conf 2011-04-21 01:30:30 +0000 |
48 | +++ lib/canonical/config/schema-lazr.conf 2011-04-25 14:43:28 +0000 |
49 | @@ -977,8 +977,12 @@ |
50 | # See [error_reports]. |
51 | copy_to_zlog: false |
52 | |
53 | -[sync_packages] |
54 | +[IPackageCopyJobSource] |
55 | +module: lp.soyuz.interfaces.distributionjob |
56 | +# XXX: GavinPanella 2011-04-20 bug=770297: The sync_packages database |
57 | +# user should be renamed to copy_packages. |
58 | dbuser: sync_packages |
59 | +crontab_group: MAIN |
60 | |
61 | # See [error_reports]. |
62 | error_dir: none |
63 | @@ -2110,7 +2114,10 @@ |
64 | # Each job source class also needs its own config section to specify the |
65 | # dbuser, the crontab_group, and the module that the job source class |
66 | # can be loaded from. |
67 | -job_sources: IMembershipNotificationJobSource, IPersonMergeJobSource |
68 | +job_sources: |
69 | + IMembershipNotificationJobSource, |
70 | + IPackageCopyJobSource, |
71 | + IPersonMergeJobSource, |
72 | |
73 | [IMembershipNotificationJobSource] |
74 | # This section is used by cronscripts/process-job-source.py. |
75 | |
76 | === modified file 'lib/lp/soyuz/configure.zcml' |
77 | --- lib/lp/soyuz/configure.zcml 2011-04-11 14:34:55 +0000 |
78 | +++ lib/lp/soyuz/configure.zcml 2011-04-25 14:43:28 +0000 |
79 | @@ -908,14 +908,14 @@ |
80 | <allow interface="lp.soyuz.interfaces.distributionjob.IDistributionJob" /> |
81 | </class> |
82 | |
83 | - <!-- SyncPackageJobSource --> |
84 | + <!-- PackageCopyJobSource --> |
85 | <securedutility |
86 | - component="lp.soyuz.model.syncpackagejob.SyncPackageJob" |
87 | - provides="lp.soyuz.interfaces.distributionjob.ISyncPackageJobSource"> |
88 | - <allow interface="lp.soyuz.interfaces.distributionjob.ISyncPackageJobSource"/> |
89 | + component="lp.soyuz.model.packagecopyjob.PackageCopyJob" |
90 | + provides="lp.soyuz.interfaces.distributionjob.IPackageCopyJobSource"> |
91 | + <allow interface="lp.soyuz.interfaces.distributionjob.IPackageCopyJobSource"/> |
92 | </securedutility> |
93 | - <class class="lp.soyuz.model.syncpackagejob.SyncPackageJob"> |
94 | - <allow interface="lp.soyuz.interfaces.distributionjob.ISyncPackageJob" /> |
95 | + <class class="lp.soyuz.model.packagecopyjob.PackageCopyJob"> |
96 | + <allow interface="lp.soyuz.interfaces.distributionjob.IPackageCopyJob" /> |
97 | <allow interface="lp.soyuz.interfaces.distributionjob.IDistributionJob" /> |
98 | </class> |
99 | |
100 | |
101 | === modified file 'lib/lp/soyuz/interfaces/distributionjob.py' |
102 | --- lib/lp/soyuz/interfaces/distributionjob.py 2011-04-15 15:08:20 +0000 |
103 | +++ lib/lp/soyuz/interfaces/distributionjob.py 2011-04-25 14:43:28 +0000 |
104 | @@ -10,14 +10,15 @@ |
105 | "IDistroSeriesDifferenceJobSource", |
106 | "IInitialiseDistroSeriesJob", |
107 | "IInitialiseDistroSeriesJobSource", |
108 | - "ISyncPackageJob", |
109 | - "ISyncPackageJobSource", |
110 | + "IPackageCopyJob", |
111 | + "IPackageCopyJobSource", |
112 | ] |
113 | |
114 | from lazr.enum import ( |
115 | DBEnumeratedType, |
116 | DBItem, |
117 | ) |
118 | +from lazr.restful.fields import Reference |
119 | from zope.interface import ( |
120 | Attribute, |
121 | Interface, |
122 | @@ -25,8 +26,10 @@ |
123 | from zope.schema import ( |
124 | Bool, |
125 | Int, |
126 | + List, |
127 | Object, |
128 | TextLine, |
129 | + Tuple, |
130 | ) |
131 | |
132 | from canonical.launchpad import _ |
133 | @@ -37,6 +40,7 @@ |
134 | IJobSource, |
135 | IRunnableJob, |
136 | ) |
137 | +from lp.soyuz.interfaces.archive import IArchive |
138 | |
139 | |
140 | class IDistributionJob(Interface): |
141 | @@ -72,8 +76,8 @@ |
142 | populating the archive from the parent distroseries. |
143 | """) |
144 | |
145 | - SYNC_PACKAGE = DBItem(2, """ |
146 | - Synchronize a single package from another distribution. |
147 | + COPY_PACKAGE = DBItem(2, """ |
148 | + Copy a single package from another distribution. |
149 | |
150 | This job copies a single package, optionally including binaries. |
151 | """) |
152 | @@ -97,12 +101,26 @@ |
153 | """ |
154 | |
155 | |
156 | -class ISyncPackageJobSource(IJobSource): |
157 | - """An interface for acquiring IISyncPackageJobs.""" |
158 | - |
159 | - def create(source_archive, target_archive, distroseries, pocket, |
160 | - source_package_name, version, include_binaries): |
161 | - """Create a new sync package job.""" |
162 | +class IPackageCopyJobSource(IJobSource): |
163 | + """An interface for acquiring IIPackageCopyJobs.""" |
164 | + |
165 | + def create(cls, source_archive, source_packages, |
166 | + target_archive, target_distroseries, target_pocket, |
167 | + include_binaries=False): |
168 | + """Create a new sync package job. |
169 | + |
170 | + :param source_archive: The `IArchive` in which `source_packages` are |
171 | + found. |
172 | + :param source_packages: This is an iterable of `(source_package_name, |
173 | + version)` tuples, where both `source_package_name` and `version` |
174 | + are strings. |
175 | + :param target_archive: The `IArchive` to which to copy the packages. |
176 | + :param target_distroseries: The `IDistroSeries` to which to copy the |
177 | + packages. |
178 | + :param target_pocket: The pocket into which to copy the packages. Must |
179 | + be a member of `PackagePublishingPocket`. |
180 | + :param include_binaries: See `do_copy`. |
181 | + """ |
182 | |
183 | def getActiveJobs(archive): |
184 | """Retrieve all active sync jobs for an archive.""" |
185 | @@ -112,33 +130,46 @@ |
186 | """A Job that performs actions on a distribution.""" |
187 | |
188 | |
189 | -class ISyncPackageJob(IRunnableJob): |
190 | +class IPackageCopyJob(IRunnableJob): |
191 | """A Job that synchronizes packages.""" |
192 | |
193 | - pocket = Int( |
194 | - title=_('Target package publishing pocket'), required=True, |
195 | - readonly=True, |
196 | - ) |
197 | - |
198 | - source_archive = Int( |
199 | - title=_('Source Archive ID'), required=True, readonly=True, |
200 | - ) |
201 | - |
202 | - target_archive = Int( |
203 | - title=_('Target Archive ID'), required=True, readonly=True, |
204 | - ) |
205 | - |
206 | - source_package_name = TextLine( |
207 | - title=_("Source Package Name"), |
208 | - required=True, readonly=True) |
209 | - |
210 | - source_package_version = TextLine( |
211 | - title=_("Source Package Version"), |
212 | - required=True, readonly=True) |
213 | + source_packages = List( |
214 | + title=_("Source Packages"), |
215 | + value_type=Tuple(min_length=3, max_length=3), |
216 | + required=True, readonly=True, |
217 | + ) |
218 | + |
219 | + source_archive_id = Int( |
220 | + title=_('Source Archive ID'), required=True, readonly=True, |
221 | + ) |
222 | + |
223 | + source_archive = Reference( |
224 | + schema=IArchive, title=_('Source Archive'), |
225 | + required=True, readonly=True, |
226 | + ) |
227 | + |
228 | + target_archive_id = Int( |
229 | + title=_('Target Archive ID'), required=True, readonly=True, |
230 | + ) |
231 | + |
232 | + target_archive = Reference( |
233 | + schema=IArchive, title=_('Target Archive'), |
234 | + required=True, readonly=True, |
235 | + ) |
236 | + |
237 | + target_distroseries = Reference( |
238 | + schema=IDistroSeries, title=_('Target DistroSeries.'), |
239 | + required=True, readonly=True) |
240 | + |
241 | + target_pocket = Int( |
242 | + title=_('Target package publishing pocket'), required=True, |
243 | + readonly=True, |
244 | + ) |
245 | |
246 | include_binaries = Bool( |
247 | - title=_("Copy binaries"), |
248 | - required=False, readonly=True) |
249 | + title=_("Copy binaries"), |
250 | + required=False, readonly=True, |
251 | + ) |
252 | |
253 | |
254 | class IDistroSeriesDifferenceJob(IRunnableJob): |
255 | |
256 | === renamed file 'lib/lp/soyuz/model/syncpackagejob.py' => 'lib/lp/soyuz/model/packagecopyjob.py' |
257 | --- lib/lp/soyuz/model/syncpackagejob.py 2010-11-15 16:25:05 +0000 |
258 | +++ lib/lp/soyuz/model/packagecopyjob.py 2011-04-25 14:43:28 +0000 |
259 | @@ -4,7 +4,7 @@ |
260 | __metaclass__ = type |
261 | |
262 | __all__ = [ |
263 | - "SyncPackageJob", |
264 | + "PackageCopyJob", |
265 | ] |
266 | |
267 | from zope.component import getUtility |
268 | @@ -18,50 +18,53 @@ |
269 | IStore, |
270 | ) |
271 | from lp.registry.interfaces.pocket import PackagePublishingPocket |
272 | -from lp.soyuz.interfaces.archive import IArchiveSet |
273 | +from lp.soyuz.interfaces.archive import ( |
274 | + CannotCopy, |
275 | + IArchiveSet, |
276 | + ) |
277 | from lp.soyuz.interfaces.distributionjob import ( |
278 | DistributionJobType, |
279 | - ISyncPackageJob, |
280 | - ISyncPackageJobSource, |
281 | + IPackageCopyJob, |
282 | + IPackageCopyJobSource, |
283 | ) |
284 | from lp.soyuz.model.distributionjob import ( |
285 | DistributionJob, |
286 | DistributionJobDerived, |
287 | ) |
288 | - |
289 | - |
290 | -class SyncPackageJob(DistributionJobDerived): |
291 | +from lp.soyuz.scripts.packagecopier import do_copy |
292 | + |
293 | + |
294 | +class PackageCopyJob(DistributionJobDerived): |
295 | """Job that copies a package between archives.""" |
296 | |
297 | - implements(ISyncPackageJob) |
298 | + implements(IPackageCopyJob) |
299 | |
300 | - class_job_type = DistributionJobType.SYNC_PACKAGE |
301 | - classProvides(ISyncPackageJobSource) |
302 | + class_job_type = DistributionJobType.COPY_PACKAGE |
303 | + classProvides(IPackageCopyJobSource) |
304 | |
305 | @classmethod |
306 | - def create(cls, source_archive, target_archive, distroseries, |
307 | - pocket, source_package_name, source_package_version, |
308 | - include_binaries): |
309 | - """See `ISyncPackageJobSource`.""" |
310 | + def create(cls, source_packages, source_archive, |
311 | + target_archive, target_distroseries, target_pocket, |
312 | + include_binaries=False): |
313 | + """See `IPackageCopyJobSource`.""" |
314 | metadata = { |
315 | + 'source_packages': source_packages, |
316 | 'source_archive_id': source_archive.id, |
317 | 'target_archive_id': target_archive.id, |
318 | - 'pocket': pocket.value, |
319 | - 'source_package_name': source_package_name, |
320 | - 'source_package_version': source_package_version, |
321 | + 'target_pocket': target_pocket.value, |
322 | 'include_binaries': include_binaries, |
323 | } |
324 | job = DistributionJob( |
325 | - distroseries.distribution, distroseries, cls.class_job_type, |
326 | - metadata) |
327 | + target_distroseries.distribution, target_distroseries, |
328 | + cls.class_job_type, metadata) |
329 | IMasterStore(DistributionJob).add(job) |
330 | return cls(job) |
331 | |
332 | @classmethod |
333 | def getActiveJobs(cls, archive): |
334 | - """See `ISyncPackageJobSource`.""" |
335 | + """See `IPackageCopyJobSource`.""" |
336 | # TODO: JRV 20101104. This iterates manually over all active |
337 | - # SyncPackageJobs. This should usually be a short enough list, |
338 | + # PackageCopyJobs. This should usually be a short enough list, |
339 | # but if it really becomes an issue target_archive should |
340 | # be moved into a separate database field. |
341 | jobs = IStore(DistributionJob).find( |
342 | @@ -69,36 +72,59 @@ |
343 | DistributionJob.job_type == cls.class_job_type, |
344 | DistributionJob.distribution == archive.distribution) |
345 | jobs = [cls(job) for job in jobs] |
346 | - return (job for job in jobs if job.target_archive == archive) |
347 | + return (job for job in jobs if job.target_archive_id == archive.id) |
348 | + |
349 | + @property |
350 | + def source_packages(self): |
351 | + getPublishedSources = self.source_archive.getPublishedSources |
352 | + for name, version in self.metadata['source_packages']: |
353 | + yield name, version, getPublishedSources( |
354 | + name=name, version=version, exact_match=True).first() |
355 | + |
356 | + @property |
357 | + def source_archive_id(self): |
358 | + return self.metadata['source_archive_id'] |
359 | |
360 | @property |
361 | def source_archive(self): |
362 | - return getUtility(IArchiveSet).get(self.metadata['source_archive_id']) |
363 | + return getUtility(IArchiveSet).get(self.source_archive_id) |
364 | + |
365 | + @property |
366 | + def target_archive_id(self): |
367 | + return self.metadata['target_archive_id'] |
368 | |
369 | @property |
370 | def target_archive(self): |
371 | - return getUtility(IArchiveSet).get(self.metadata['target_archive_id']) |
372 | - |
373 | - @property |
374 | - def pocket(self): |
375 | - return PackagePublishingPocket.items[self.metadata['pocket']] |
376 | + return getUtility(IArchiveSet).get(self.target_archive_id) |
377 | + |
378 | + @property |
379 | + def target_distroseries(self): |
380 | + return self.distroseries |
381 | + |
382 | + @property |
383 | + def target_pocket(self): |
384 | + return PackagePublishingPocket.items[self.metadata['target_pocket']] |
385 | |
386 | @property |
387 | def include_binaries(self): |
388 | return self.metadata['include_binaries'] |
389 | |
390 | - @property |
391 | - def source_package_name(self): |
392 | - return self.metadata['source_package_name'] |
393 | - |
394 | - @property |
395 | - def source_package_version(self): |
396 | - return self.metadata['source_package_version'] |
397 | - |
398 | def run(self): |
399 | """See `IRunnableJob`.""" |
400 | - self.target_archive.syncSource( |
401 | - self.source_package_name, self.source_package_version, |
402 | - self.source_archive, to_pocket=str(self.pocket), |
403 | - to_series=self.distroseries.name, |
404 | + if self.target_archive.is_ppa: |
405 | + if self.target_pocket != PackagePublishingPocket.RELEASE: |
406 | + raise CannotCopy( |
407 | + "Destination pocket must be 'release' for a PPA.") |
408 | + |
409 | + source_packages = set() |
410 | + for name, version, source_package in self.source_packages: |
411 | + if source_package is None: |
412 | + raise CannotCopy( |
413 | + "Package %r %r not found." % (name, version)) |
414 | + else: |
415 | + source_packages.add(source_package) |
416 | + |
417 | + do_copy( |
418 | + sources=source_packages, archive=self.target_archive, |
419 | + series=self.target_distroseries, pocket=self.target_pocket, |
420 | include_binaries=self.include_binaries) |
421 | |
422 | === renamed file 'lib/lp/soyuz/tests/test_syncpackagejob.py' => 'lib/lp/soyuz/tests/test_packagecopyjob.py' |
423 | --- lib/lp/soyuz/tests/test_syncpackagejob.py 2011-03-03 00:43:44 +0000 |
424 | +++ lib/lp/soyuz/tests/test_packagecopyjob.py 2011-04-25 14:43:28 +0000 |
425 | @@ -3,48 +3,54 @@ |
426 | |
427 | """Tests for sync package jobs.""" |
428 | |
429 | -import os |
430 | -import subprocess |
431 | -import sys |
432 | - |
433 | +from testtools.content import text_content |
434 | import transaction |
435 | from zope.component import getUtility |
436 | from zope.security.proxy import removeSecurityProxy |
437 | |
438 | -from canonical.config import config |
439 | from canonical.testing import LaunchpadZopelessLayer |
440 | -from lp.registry.errors import NoSuchSourcePackageName |
441 | from lp.registry.interfaces.pocket import PackagePublishingPocket |
442 | +from lp.soyuz.interfaces.archive import CannotCopy |
443 | from lp.soyuz.interfaces.distributionjob import ( |
444 | - ISyncPackageJob, |
445 | - ISyncPackageJobSource, |
446 | + IPackageCopyJob, |
447 | + IPackageCopyJobSource, |
448 | ) |
449 | from lp.soyuz.interfaces.publishing import PackagePublishingStatus |
450 | from lp.soyuz.tests.test_publishing import SoyuzTestPublisher |
451 | -from lp.testing import TestCaseWithFactory |
452 | - |
453 | - |
454 | -class SyncPackageJobTests(TestCaseWithFactory): |
455 | - """Test case for SyncPackageJob.""" |
456 | +from lp.testing import ( |
457 | + run_script, |
458 | + TestCaseWithFactory, |
459 | + ) |
460 | + |
461 | + |
462 | +class PackageCopyJobTests(TestCaseWithFactory): |
463 | + """Test case for PackageCopyJob.""" |
464 | |
465 | layer = LaunchpadZopelessLayer |
466 | |
467 | def test_create(self): |
468 | - # A SyncPackageJob can be created and stores its arguments. |
469 | + # A PackageCopyJob can be created and stores its arguments. |
470 | distroseries = self.factory.makeDistroSeries() |
471 | archive1 = self.factory.makeArchive(distroseries.distribution) |
472 | archive2 = self.factory.makeArchive(distroseries.distribution) |
473 | - source = getUtility(ISyncPackageJobSource) |
474 | - job = source.create(archive1, archive2, distroseries, |
475 | - PackagePublishingPocket.RELEASE, |
476 | - "foo", "1.0-1", include_binaries=False) |
477 | - self.assertProvides(job, ISyncPackageJob) |
478 | + source = getUtility(IPackageCopyJobSource) |
479 | + job = source.create( |
480 | + source_packages=[("foo", "1.0-1"), ("bar", "2.4")], |
481 | + source_archive=archive1, target_archive=archive2, |
482 | + target_distroseries=distroseries, |
483 | + target_pocket=PackagePublishingPocket.RELEASE, |
484 | + include_binaries=False) |
485 | + self.assertProvides(job, IPackageCopyJob) |
486 | self.assertEquals(distroseries, job.distroseries) |
487 | + self.assertEquals(archive1.id, job.source_archive_id) |
488 | self.assertEquals(archive1, job.source_archive) |
489 | + self.assertEquals(archive2.id, job.target_archive_id) |
490 | self.assertEquals(archive2, job.target_archive) |
491 | - self.assertEquals(PackagePublishingPocket.RELEASE, job.pocket) |
492 | - self.assertEquals("foo", job.source_package_name) |
493 | - self.assertEquals("1.0-1", job.source_package_version) |
494 | + self.assertEquals(distroseries, job.target_distroseries) |
495 | + self.assertEquals(PackagePublishingPocket.RELEASE, job.target_pocket) |
496 | + self.assertContentEqual( |
497 | + job.source_packages, |
498 | + [("foo", "1.0-1", None), ("bar", "2.4", None)]) |
499 | self.assertEquals(False, job.include_binaries) |
500 | |
501 | def test_getActiveJobs(self): |
502 | @@ -52,32 +58,39 @@ |
503 | distroseries = self.factory.makeDistroSeries() |
504 | archive1 = self.factory.makeArchive(distroseries.distribution) |
505 | archive2 = self.factory.makeArchive(distroseries.distribution) |
506 | - source = getUtility(ISyncPackageJobSource) |
507 | - job = source.create(archive1, archive2, distroseries, |
508 | - PackagePublishingPocket.RELEASE, |
509 | - "foo", "1.0-1", include_binaries=False) |
510 | + source = getUtility(IPackageCopyJobSource) |
511 | + job = source.create( |
512 | + source_packages=[("foo", "1.0-1")], source_archive=archive1, |
513 | + target_archive=archive2, target_distroseries=distroseries, |
514 | + target_pocket=PackagePublishingPocket.RELEASE, |
515 | + include_binaries=False) |
516 | self.assertContentEqual([job], source.getActiveJobs(archive2)) |
517 | |
518 | - def test_cronscript(self): |
519 | - # The cron script runs without problems. |
520 | - script = os.path.join( |
521 | - config.root, 'cronscripts', 'sync-packages.py') |
522 | - args = [sys.executable, script, '-v'] |
523 | - process = subprocess.Popen( |
524 | - args, stdout=subprocess.PIPE, stderr=subprocess.PIPE) |
525 | - stdout, stderr = process.communicate() |
526 | - self.assertEqual(process.returncode, 0) |
527 | - |
528 | def test_run_unknown_package(self): |
529 | # A job properly records failure. |
530 | distroseries = self.factory.makeDistroSeries() |
531 | archive1 = self.factory.makeArchive(distroseries.distribution) |
532 | archive2 = self.factory.makeArchive(distroseries.distribution) |
533 | - source = getUtility(ISyncPackageJobSource) |
534 | - job = source.create(archive1, archive2, distroseries, |
535 | - PackagePublishingPocket.RELEASE, |
536 | - "foo", "1.0-1", include_binaries=False) |
537 | - self.assertRaises(NoSuchSourcePackageName, job.run) |
538 | + source = getUtility(IPackageCopyJobSource) |
539 | + job = source.create( |
540 | + source_packages=[("foo", "1.0-1")], source_archive=archive1, |
541 | + target_archive=archive2, target_distroseries=distroseries, |
542 | + target_pocket=PackagePublishingPocket.RELEASE, |
543 | + include_binaries=False) |
544 | + self.assertRaises(CannotCopy, job.run) |
545 | + |
546 | + def test_target_ppa_non_release_pocket(self): |
547 | + # When copying to a PPA archive the target must be the release pocket. |
548 | + distroseries = self.factory.makeDistroSeries() |
549 | + archive1 = self.factory.makeArchive(distroseries.distribution) |
550 | + archive2 = self.factory.makeArchive(distroseries.distribution) |
551 | + source = getUtility(IPackageCopyJobSource) |
552 | + job = source.create( |
553 | + source_packages=[], source_archive=archive1, |
554 | + target_archive=archive2, target_distroseries=distroseries, |
555 | + target_pocket=PackagePublishingPocket.UPDATES, |
556 | + include_binaries=False) |
557 | + self.assertRaises(CannotCopy, job.run) |
558 | |
559 | def test_run(self): |
560 | # A proper test run synchronizes packages. |
561 | @@ -88,19 +101,25 @@ |
562 | archive1 = self.factory.makeArchive(distroseries.distribution) |
563 | archive2 = self.factory.makeArchive(distroseries.distribution) |
564 | |
565 | - publisher.getPubBinaries( |
566 | - distroseries=distroseries, binaryname="libc", |
567 | - version="2.8-1", |
568 | - status=PackagePublishingStatus.PUBLISHED, |
569 | + source_package = publisher.getPubSource( |
570 | + distroseries=distroseries, sourcename="libc", |
571 | + version="2.8-1", status=PackagePublishingStatus.PUBLISHED, |
572 | archive=archive1) |
573 | |
574 | - source = getUtility(ISyncPackageJobSource) |
575 | - job = source.create(archive1, archive2, distroseries, |
576 | - PackagePublishingPocket.RELEASE, |
577 | - "libc", "2.8-1", include_binaries=False) |
578 | + source = getUtility(IPackageCopyJobSource) |
579 | + job = source.create( |
580 | + source_packages=[("libc", "2.8-1")], source_archive=archive1, |
581 | + target_archive=archive2, target_distroseries=distroseries, |
582 | + target_pocket=PackagePublishingPocket.RELEASE, |
583 | + include_binaries=False) |
584 | + self.assertContentEqual( |
585 | + job.source_packages, [("libc", "2.8-1", source_package)]) |
586 | + |
587 | # Make sure everything hits the database, switching db users |
588 | # aborts. |
589 | transaction.commit() |
590 | + # XXX: GavinPanella 2011-04-20 bug=770297: The sync_packages database |
591 | + # user should be renamed to copy_packages. |
592 | self.layer.switchDbUser('sync_packages') |
593 | job.run() |
594 | |
595 | @@ -113,10 +132,12 @@ |
596 | distroseries = self.factory.makeDistroSeries() |
597 | archive1 = self.factory.makeArchive(distroseries.distribution) |
598 | archive2 = self.factory.makeArchive(distroseries.distribution) |
599 | - source = getUtility(ISyncPackageJobSource) |
600 | - job = source.create(archive1, archive2, distroseries, |
601 | - PackagePublishingPocket.RELEASE, |
602 | - "foo", "1.0-1", include_binaries=False) |
603 | + source = getUtility(IPackageCopyJobSource) |
604 | + job = source.create( |
605 | + source_packages=[("foo", "1.0-1")], source_archive=archive1, |
606 | + target_archive=archive2, target_distroseries=distroseries, |
607 | + target_pocket=PackagePublishingPocket.RELEASE, |
608 | + include_binaries=False) |
609 | oops_vars = job.getOopsVars() |
610 | naked_job = removeSecurityProxy(job) |
611 | self.assertIn( |
612 | @@ -124,3 +145,32 @@ |
613 | self.assertIn(('distroseries_id', distroseries.id), oops_vars) |
614 | self.assertIn( |
615 | ('distribution_job_id', naked_job.context.id), oops_vars) |
616 | + |
617 | + def test_smoke(self): |
618 | + publisher = SoyuzTestPublisher() |
619 | + publisher.prepareBreezyAutotest() |
620 | + distroseries = publisher.breezy_autotest |
621 | + archive1 = self.factory.makeArchive(distroseries.distribution) |
622 | + archive2 = self.factory.makeArchive(distroseries.distribution) |
623 | + publisher.getPubSource( |
624 | + distroseries=distroseries, sourcename="libc", |
625 | + version="2.8-1", status=PackagePublishingStatus.PUBLISHED, |
626 | + archive=archive1) |
627 | + getUtility(IPackageCopyJobSource).create( |
628 | + source_packages=[("libc", "2.8-1")], source_archive=archive1, |
629 | + target_archive=archive2, target_distroseries=distroseries, |
630 | + target_pocket=PackagePublishingPocket.RELEASE, |
631 | + include_binaries=False) |
632 | + transaction.commit() |
633 | + |
634 | + out, err, exit_code = run_script( |
635 | + "LP_DEBUG_SQL=1 cronscripts/process-job-source.py -vv %s" % ( |
636 | + IPackageCopyJobSource.getName())) |
637 | + |
638 | + self.addDetail("stdout", text_content(out)) |
639 | + self.addDetail("stderr", text_content(err)) |
640 | + |
641 | + self.assertEqual(0, exit_code) |
642 | + copied_source_package = archive2.getPublishedSources( |
643 | + name="libc", version="2.8-1", exact_match=True).first() |
644 | + self.assertIsNot(copied_source_package, None) |
Hi Gavin,
This change makes sense and the tests look good.
Two little things:
* type "copyingto"
* If the XXX is something we really need to do why not file a bug so we don't forget?