Merge lp:~jtv/launchpad/redo-uploads into lp:launchpad
- redo-uploads
- Merge into devel
Status: | Merged | ||||
---|---|---|---|---|---|
Approved by: | Jeroen T. Vermeulen | ||||
Approved revision: | no longer in the source branch. | ||||
Merged at revision: | not available | ||||
Proposed branch: | lp:~jtv/launchpad/redo-uploads | ||||
Merge into: | lp:launchpad | ||||
Diff against target: |
449 lines 6 files modified
lib/lp/registry/interfaces/sourcepackage.py (+7/-0) lib/lp/registry/model/sourcepackage.py (+33/-0) lib/lp/soyuz/doc/distroseriesqueue-translations.txt (+43/-0) lib/lp/translations/scripts/reupload_translations.py (+110/-0) lib/lp/translations/scripts/tests/test_reupload_translations.py (+168/-0) scripts/rosetta/reupload-translations.py (+19/-0) |
||||
To merge this branch: | bzr merge lp:~jtv/launchpad/redo-uploads | ||||
Related bugs: |
|
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Jeroen T. Vermeulen (community) | Abstain | ||
Edwin Grubbs (community) | code | Approve | |
Review via email: mp+12659@code.launchpad.net |
Commit message
Description of the change
Jeroen T. Vermeulen (jtv) wrote : | # |
Edwin Grubbs (edwin-grubbs) wrote : | # |
Hi Jeroen,
This looks good. I just have a few formatting comments.
merge-conditional
-Edwin
>=== added file 'lib/lp/
>--- lib/lp/
>+++ lib/lp/
>@@ -0,0 +1,105 @@
>+__metaclass__ = type
Missing copyright notice.
>+
>+__all__ = [
>+ 'ReuploadPackag
>+ ]
Indentation should be spaces.
>+
>+from zope.component import getUtility
>+
>+from lp.services.
>+
>+from canonical.
>+from lp.registry.
>+from lp.translations
>+ ITranslationImp
>+
>+
>+class ReuploadPackage
>+ """Re-upload latest translations for given distribution packages."""
>+ description = "Re-upload latest translations uploads for package(s)."
>+
>+ def add_my_
>+ """See `LaunchpadScrip
>+ self.parser.
>+ help="Distribution to upload for.", default='ubuntu')
>+ self.parser.
>+ help="Distribution release series to upload for.")
>+ self.parser.
>+ dest='packages', default=[],
>+ help="Name(s) of source package(s) to re-upload.")
>+ self.parser.
>+ action=
>+ help="Pretend to upload, but make no actual changes.")
>+
Trailing white space.
>+ def main(self):
>+ """See `LaunchpadScrip
>+ self.uploadless
>+ self._setDistro
>+
>+ if len(self.
>+ raise LaunchpadScript
>+
>+ if self.options.
>+ self.logger.
>+
>+ for package_name in self.options.
>+ self._processPa
>+ self._commit()
>+
>+ self.logger.
>+
>+ def _commit(self):
>+ """Commit transaction (or abort if dry run)."""
>+ if self.txn:
>+ if self.options.
>+ self.txn.abort()
>+ else:
>+ self.txn.commit()
>+
>+ def _setDistroDetai
>+ """Figure out the `Distribution`
>+ # Avoid circular imports.
>+ from lp.registry.
>+
>+ distroset = getUtility(
>+ self.distro = distroset.
>+
>+ if not self.options.
>+ raise LaunchpadScript
>+ "Specify a distribution release series.")
>+
>+ self.distroseries = self.distro.
Jeroen T. Vermeulen (jtv) wrote : | # |
> Hi Jeroen,
>
> This looks good. I just have a few formatting comments.
Thanks. Whitespace problems all fixed.
"I have no idea how that tab got into my code, officer."
Preview Diff
1 | === modified file 'lib/lp/registry/interfaces/sourcepackage.py' | |||
2 | --- lib/lp/registry/interfaces/sourcepackage.py 2009-09-16 04:31:39 +0000 | |||
3 | +++ lib/lp/registry/interfaces/sourcepackage.py 2009-09-30 16:53:17 +0000 | |||
4 | @@ -225,6 +225,13 @@ | |||
5 | 225 | title=u'The component in which the package was last published.', | 225 | title=u'The component in which the package was last published.', |
6 | 226 | schema=IComponent, readonly=True, required=False) | 226 | schema=IComponent, readonly=True, required=False) |
7 | 227 | 227 | ||
8 | 228 | def getLatestTranslationsUploads(): | ||
9 | 229 | """Find latest Translations tarballs as produced by Soyuz. | ||
10 | 230 | |||
11 | 231 | :return: A list of `ILibraryFileAlias`es, usually of size zero | ||
12 | 232 | or one. If not, they are sorted from oldest to newest. | ||
13 | 233 | """ | ||
14 | 234 | |||
15 | 228 | 235 | ||
16 | 229 | class ISourcePackageFactory(Interface): | 236 | class ISourcePackageFactory(Interface): |
17 | 230 | """A creator of source packages.""" | 237 | """A creator of source packages.""" |
18 | 231 | 238 | ||
19 | === modified file 'lib/lp/registry/model/sourcepackage.py' | |||
20 | --- lib/lp/registry/model/sourcepackage.py 2009-09-25 17:00:20 +0000 | |||
21 | +++ lib/lp/registry/model/sourcepackage.py 2009-09-30 16:53:17 +0000 | |||
22 | @@ -54,6 +54,7 @@ | |||
23 | 54 | from lp.translations.interfaces.potemplate import IHasTranslationTemplates | 54 | from lp.translations.interfaces.potemplate import IHasTranslationTemplates |
24 | 55 | from lp.registry.interfaces.pocket import PackagePublishingPocket | 55 | from lp.registry.interfaces.pocket import PackagePublishingPocket |
25 | 56 | from lp.soyuz.interfaces.publishing import PackagePublishingStatus | 56 | from lp.soyuz.interfaces.publishing import PackagePublishingStatus |
26 | 57 | from lp.soyuz.interfaces.queue import PackageUploadCustomFormat | ||
27 | 57 | from lp.answers.interfaces.questioncollection import ( | 58 | from lp.answers.interfaces.questioncollection import ( |
28 | 58 | QUESTION_STATUS_DEFAULT_SEARCH) | 59 | QUESTION_STATUS_DEFAULT_SEARCH) |
29 | 59 | from lp.answers.interfaces.questiontarget import IQuestionTarget | 60 | from lp.answers.interfaces.questiontarget import IQuestionTarget |
30 | @@ -655,3 +656,35 @@ | |||
31 | 655 | self.distribution.name, | 656 | self.distribution.name, |
32 | 656 | self.distroseries.getSuite(pocket), | 657 | self.distroseries.getSuite(pocket), |
33 | 657 | self.name) | 658 | self.name) |
34 | 659 | |||
35 | 660 | def getLatestTranslationsUploads(self): | ||
36 | 661 | """See `ISourcePackage`.""" | ||
37 | 662 | our_format = PackageUploadCustomFormat.ROSETTA_TRANSLATIONS | ||
38 | 663 | |||
39 | 664 | packagename = self.sourcepackagename.name | ||
40 | 665 | displayname = self.displayname | ||
41 | 666 | distro = self.distroseries.distribution | ||
42 | 667 | |||
43 | 668 | histories = distro.main_archive.getPublishedSources( | ||
44 | 669 | name=packagename, distroseries=self.distroseries, | ||
45 | 670 | status=PackagePublishingStatus.PUBLISHED, exact_match=True) | ||
46 | 671 | histories = list(histories) | ||
47 | 672 | |||
48 | 673 | builds = [] | ||
49 | 674 | for history in histories: | ||
50 | 675 | builds += list(history.getBuilds()) | ||
51 | 676 | |||
52 | 677 | uploads = [ | ||
53 | 678 | build.package_upload | ||
54 | 679 | for build in builds | ||
55 | 680 | if build.package_upload | ||
56 | 681 | ] | ||
57 | 682 | custom_files = [] | ||
58 | 683 | for upload in uploads: | ||
59 | 684 | custom_files += [ | ||
60 | 685 | custom for custom in upload.customfiles | ||
61 | 686 | if custom.customformat == our_format | ||
62 | 687 | ] | ||
63 | 688 | |||
64 | 689 | custom_files.sort(key=attrgetter('id')) | ||
65 | 690 | return [custom.libraryfilealias for custom in custom_files] | ||
66 | 658 | 691 | ||
67 | === modified file 'lib/lp/soyuz/doc/distroseriesqueue-translations.txt' | |||
68 | --- lib/lp/soyuz/doc/distroseriesqueue-translations.txt 2009-09-04 08:35:20 +0000 | |||
69 | +++ lib/lp/soyuz/doc/distroseriesqueue-translations.txt 2009-09-30 16:53:17 +0000 | |||
70 | @@ -97,6 +97,14 @@ | |||
71 | 97 | >>> pmount_upload.is_rejected | 97 | >>> pmount_upload.is_rejected |
72 | 98 | False | 98 | False |
73 | 99 | 99 | ||
74 | 100 | At this point, no translations uploads have been registered for this | ||
75 | 101 | package. | ||
76 | 102 | |||
77 | 103 | >>> from lp.registry.model.sourcepackage import SourcePackage | ||
78 | 104 | >>> dapper_pmount = SourcePackage(pmount_sourcepackagename, dapper) | ||
79 | 105 | >>> print len(dapper_pmount.getLatestTranslationsUploads()) | ||
80 | 106 | 0 | ||
81 | 107 | |||
82 | 100 | >>> success = pmount_upload.do_accept() | 108 | >>> success = pmount_upload.do_accept() |
83 | 101 | DEBUG: Creating queue entry | 109 | DEBUG: Creating queue entry |
84 | 102 | DEBUG: Build ... found | 110 | DEBUG: Build ... found |
85 | @@ -115,6 +123,17 @@ | |||
86 | 115 | #NEW: pmount_0.9.7-2ubuntu2_amd64.deb | 123 | #NEW: pmount_0.9.7-2ubuntu2_amd64.deb |
87 | 116 | #OK: pmount_0.9.7-2ubuntu2_amd64_translations.tar.gz | 124 | #OK: pmount_0.9.7-2ubuntu2_amd64_translations.tar.gz |
88 | 117 | 125 | ||
89 | 126 | The upload now shows up as the latest translations upload for the | ||
90 | 127 | package. | ||
91 | 128 | |||
92 | 129 | >>> latest_translations_uploads = list( | ||
93 | 130 | ... dapper_pmount.getLatestTranslationsUploads()) | ||
94 | 131 | >>> print len(latest_translations_uploads) | ||
95 | 132 | 1 | ||
96 | 133 | |||
97 | 134 | We'll get back to that uploaded file later. | ||
98 | 135 | |||
99 | 136 | >>> latest_translations_upload = latest_translations_uploads[0] | ||
100 | 118 | 137 | ||
101 | 119 | # Check the import queue content, it should be empty. | 138 | # Check the import queue content, it should be empty. |
102 | 120 | >>> from lp.translations.interfaces.translationimportqueue import ( | 139 | >>> from lp.translations.interfaces.translationimportqueue import ( |
103 | @@ -259,6 +278,7 @@ | |||
104 | 259 | # component. | 278 | # component. |
105 | 260 | >>> transaction.abort() | 279 | >>> transaction.abort() |
106 | 261 | 280 | ||
107 | 281 | |||
108 | 262 | == Translations from PPA build == | 282 | == Translations from PPA build == |
109 | 263 | 283 | ||
110 | 264 | For now we simply ignore translations for archives other than the | 284 | For now we simply ignore translations for archives other than the |
111 | @@ -376,3 +396,26 @@ | |||
112 | 376 | >>> translations_upload.packageupload = carlos_package_upload | 396 | >>> translations_upload.packageupload = carlos_package_upload |
113 | 377 | >>> translations_upload.publish_ROSETTA_TRANSLATIONS() | 397 | >>> translations_upload.publish_ROSETTA_TRANSLATIONS() |
114 | 378 | Imported by: carlos | 398 | Imported by: carlos |
115 | 399 | |||
116 | 400 | |||
117 | 401 | === Translations tarball === | ||
118 | 402 | |||
119 | 403 | The LibraryFileAlias returned by getLatestTranslationsUploads on the | ||
120 | 404 | source package points to a tarball with translations files for the | ||
121 | 405 | package. | ||
122 | 406 | |||
123 | 407 | >>> import tarfile | ||
124 | 408 | >>> from StringIO import StringIO | ||
125 | 409 | >>> tarball = StringIO(latest_translations_upload.read()) | ||
126 | 410 | >>> archive = tarfile.open('', 'r|gz', tarball) | ||
127 | 411 | >>> translation_files = sorted([ | ||
128 | 412 | ... entry.name for entry in archive.getmembers() | ||
129 | 413 | ... if entry.name.endswith('.po') or entry.name.endswith('.pot') | ||
130 | 414 | ... ]) | ||
131 | 415 | >>> for filename in translation_files: | ||
132 | 416 | ... print filename | ||
133 | 417 | source/po/ca.po | ||
134 | 418 | source/po/cs.po | ||
135 | 419 | source/po/de.po | ||
136 | 420 | ... | ||
137 | 421 | source/po/pmount.pot | ||
138 | 379 | 422 | ||
139 | === added file 'lib/lp/translations/scripts/reupload_translations.py' | |||
140 | --- lib/lp/translations/scripts/reupload_translations.py 1970-01-01 00:00:00 +0000 | |||
141 | +++ lib/lp/translations/scripts/reupload_translations.py 2009-09-30 16:53:17 +0000 | |||
142 | @@ -0,0 +1,110 @@ | |||
143 | 1 | #! /usr/bin/python2.4 | ||
144 | 2 | # | ||
145 | 3 | # Copyright 2009 Canonical Ltd. This software is licensed under the | ||
146 | 4 | # GNU Affero General Public License version 3 (see the file LICENSE). | ||
147 | 5 | |||
148 | 6 | __metaclass__ = type | ||
149 | 7 | |||
150 | 8 | __all__ = [ | ||
151 | 9 | 'ReuploadPackageTranslations', | ||
152 | 10 | ] | ||
153 | 11 | |||
154 | 12 | from zope.component import getUtility | ||
155 | 13 | |||
156 | 14 | from lp.services.scripts.base import LaunchpadScript, LaunchpadScriptFailure | ||
157 | 15 | |||
158 | 16 | from canonical.launchpad.interfaces.launchpad import ILaunchpadCelebrities | ||
159 | 17 | from lp.registry.interfaces.sourcepackagename import ISourcePackageNameSet | ||
160 | 18 | from lp.translations.interfaces.translationimportqueue import ( | ||
161 | 19 | ITranslationImportQueue) | ||
162 | 20 | |||
163 | 21 | |||
164 | 22 | class ReuploadPackageTranslations(LaunchpadScript): | ||
165 | 23 | """Re-upload latest translations for given distribution packages.""" | ||
166 | 24 | description = "Re-upload latest translations uploads for package(s)." | ||
167 | 25 | |||
168 | 26 | def add_my_options(self): | ||
169 | 27 | """See `LaunchpadScript`.""" | ||
170 | 28 | self.parser.add_option('-d', '--distribution', dest='distro', | ||
171 | 29 | help="Distribution to upload for.", default='ubuntu') | ||
172 | 30 | self.parser.add_option('-s', '--series', dest='distroseries', | ||
173 | 31 | help="Distribution release series to upload for.") | ||
174 | 32 | self.parser.add_option('-p', '--package', action='append', | ||
175 | 33 | dest='packages', default=[], | ||
176 | 34 | help="Name(s) of source package(s) to re-upload.") | ||
177 | 35 | self.parser.add_option('-l', '--dry-run', dest='dryrun', | ||
178 | 36 | action='store_true', default=False, | ||
179 | 37 | help="Pretend to upload, but make no actual changes.") | ||
180 | 38 | |||
181 | 39 | def main(self): | ||
182 | 40 | """See `LaunchpadScript`.""" | ||
183 | 41 | self.uploadless_packages = [] | ||
184 | 42 | self._setDistroDetails() | ||
185 | 43 | |||
186 | 44 | if len(self.options.packages) == 0: | ||
187 | 45 | raise LaunchpadScriptFailure("No packages specified.") | ||
188 | 46 | |||
189 | 47 | if self.options.dryrun: | ||
190 | 48 | self.logger.info("Dry run. Not really uploading anything.") | ||
191 | 49 | |||
192 | 50 | for package_name in self.options.packages: | ||
193 | 51 | self._processPackage(self._findPackage(package_name)) | ||
194 | 52 | self._commit() | ||
195 | 53 | |||
196 | 54 | self.logger.info("Done.") | ||
197 | 55 | |||
198 | 56 | def _commit(self): | ||
199 | 57 | """Commit transaction (or abort if dry run).""" | ||
200 | 58 | if self.txn: | ||
201 | 59 | if self.options.dryrun: | ||
202 | 60 | self.txn.abort() | ||
203 | 61 | else: | ||
204 | 62 | self.txn.commit() | ||
205 | 63 | |||
206 | 64 | def _setDistroDetails(self): | ||
207 | 65 | """Figure out the `Distribution`/`DistroSeries` to act upon.""" | ||
208 | 66 | # Avoid circular imports. | ||
209 | 67 | from lp.registry.interfaces.distribution import IDistributionSet | ||
210 | 68 | |||
211 | 69 | distroset = getUtility(IDistributionSet) | ||
212 | 70 | self.distro = distroset.getByName(self.options.distro) | ||
213 | 71 | |||
214 | 72 | if not self.options.distroseries: | ||
215 | 73 | raise LaunchpadScriptFailure( | ||
216 | 74 | "Specify a distribution release series.") | ||
217 | 75 | |||
218 | 76 | self.distroseries = self.distro.getSeries(self.options.distroseries) | ||
219 | 77 | |||
220 | 78 | def _findPackage(self, name): | ||
221 | 79 | """Find `SourcePackage` of given name.""" | ||
222 | 80 | # Avoid circular imports. | ||
223 | 81 | from lp.registry.interfaces.sourcepackage import ISourcePackageFactory | ||
224 | 82 | |||
225 | 83 | factory = getUtility(ISourcePackageFactory) | ||
226 | 84 | nameset = getUtility(ISourcePackageNameSet) | ||
227 | 85 | |||
228 | 86 | sourcepackagename = nameset.queryByName(name) | ||
229 | 87 | |||
230 | 88 | return factory.new(sourcepackagename, self.distroseries) | ||
231 | 89 | |||
232 | 90 | def _processPackage(self, package): | ||
233 | 91 | """Get translations for `package` re-uploaded.""" | ||
234 | 92 | self.logger.info("Processing %s" % package.displayname) | ||
235 | 93 | tarball_aliases = package.getLatestTranslationsUploads() | ||
236 | 94 | queue = getUtility(ITranslationImportQueue) | ||
237 | 95 | rosetta_team = getUtility(ILaunchpadCelebrities).rosetta_experts | ||
238 | 96 | |||
239 | 97 | have_uploads = False | ||
240 | 98 | for alias in tarball_aliases: | ||
241 | 99 | have_uploads = True | ||
242 | 100 | self.logger.debug("Uploading file '%s' for %s." % ( | ||
243 | 101 | alias.filename, package.displayname)) | ||
244 | 102 | queue.addOrUpdateEntriesFromTarball( | ||
245 | 103 | alias.read(), True, rosetta_team, | ||
246 | 104 | sourcepackagename=package.sourcepackagename, | ||
247 | 105 | distroseries=self.distroseries) | ||
248 | 106 | |||
249 | 107 | if not have_uploads: | ||
250 | 108 | self.logger.warn( | ||
251 | 109 | "Found no translations upload for %s." % package.displayname) | ||
252 | 110 | self.uploadless_packages.append(package) | ||
253 | 0 | 111 | ||
254 | === added file 'lib/lp/translations/scripts/tests/test_reupload_translations.py' | |||
255 | --- lib/lp/translations/scripts/tests/test_reupload_translations.py 1970-01-01 00:00:00 +0000 | |||
256 | +++ lib/lp/translations/scripts/tests/test_reupload_translations.py 2009-09-30 16:53:17 +0000 | |||
257 | @@ -0,0 +1,168 @@ | |||
258 | 1 | #! /usr/bin/python2.4 | ||
259 | 2 | # | ||
260 | 3 | # Copyright 2009 Canonical Ltd. This software is licensed under the | ||
261 | 4 | # GNU Affero General Public License version 3 (see the file LICENSE). | ||
262 | 5 | |||
263 | 6 | """Test `reupload_translations` and `ReuploadPackageTranslations`.""" | ||
264 | 7 | |||
265 | 8 | __metaclass__ = type | ||
266 | 9 | |||
267 | 10 | from unittest import TestLoader | ||
268 | 11 | |||
269 | 12 | import re | ||
270 | 13 | from StringIO import StringIO | ||
271 | 14 | import tarfile | ||
272 | 15 | import transaction | ||
273 | 16 | |||
274 | 17 | from zope.security.proxy import removeSecurityProxy | ||
275 | 18 | |||
276 | 19 | from canonical.testing import LaunchpadZopelessLayer | ||
277 | 20 | from lp.testing import TestCaseWithFactory | ||
278 | 21 | from canonical.launchpad.scripts.tests import run_script | ||
279 | 22 | |||
280 | 23 | from canonical.launchpad.database.librarian import LibraryFileAliasSet | ||
281 | 24 | from lp.registry.model.sourcepackage import SourcePackage | ||
282 | 25 | from lp.translations.model.translationimportqueue import ( | ||
283 | 26 | TranslationImportQueue) | ||
284 | 27 | |||
285 | 28 | from lp.translations.scripts.reupload_translations import ( | ||
286 | 29 | ReuploadPackageTranslations) | ||
287 | 30 | |||
288 | 31 | |||
289 | 32 | class UploadInjector: | ||
290 | 33 | def __init__(self, script, tar_alias): | ||
291 | 34 | self.tar_alias = tar_alias | ||
292 | 35 | self.script = script | ||
293 | 36 | self.original_findPackage = script._findPackage | ||
294 | 37 | |||
295 | 38 | def __call__(self, name): | ||
296 | 39 | package = self.original_findPackage(name) | ||
297 | 40 | removeSecurityProxy(package).getLatestTranslationsUploads = ( | ||
298 | 41 | self._fakeTranslationsUpload) | ||
299 | 42 | return package | ||
300 | 43 | |||
301 | 44 | def _fakeTranslationsUpload(self): | ||
302 | 45 | return [self.tar_alias] | ||
303 | 46 | |||
304 | 47 | |||
305 | 48 | def upload_tarball(translation_files): | ||
306 | 49 | """Create a tarball and upload it to the Librarian. | ||
307 | 50 | |||
308 | 51 | :param translation_files: A dict mapping filenames to file contents. | ||
309 | 52 | :return: A `LibraryFileAlias`. | ||
310 | 53 | """ | ||
311 | 54 | buf = StringIO() | ||
312 | 55 | tarball = tarfile.open('', 'w:gz', buf) | ||
313 | 56 | for name, contents in translation_files.iteritems(): | ||
314 | 57 | pseudofile = StringIO(contents) | ||
315 | 58 | tarinfo = tarfile.TarInfo() | ||
316 | 59 | tarinfo.name = name | ||
317 | 60 | tarinfo.size = len(contents) | ||
318 | 61 | tarinfo.type = tarfile.REGTYPE | ||
319 | 62 | tarball.addfile(tarinfo, pseudofile) | ||
320 | 63 | |||
321 | 64 | tarball.close() | ||
322 | 65 | buf.flush() | ||
323 | 66 | tarsize = buf.tell() | ||
324 | 67 | buf.seek(0) | ||
325 | 68 | |||
326 | 69 | return LibraryFileAliasSet().create( | ||
327 | 70 | 'uploads.tar.gz', tarsize, buf, 'application/x-gtar') | ||
328 | 71 | |||
329 | 72 | |||
330 | 73 | def summarize_translations_queue(sourcepackage): | ||
331 | 74 | """Describe queue entries for `sourcepackage` as a name/contents dict.""" | ||
332 | 75 | entries = TranslationImportQueue().getAllEntries(sourcepackage) | ||
333 | 76 | return dict((entry.path, entry.content.read()) for entry in entries) | ||
334 | 77 | |||
335 | 78 | |||
336 | 79 | class TestReuploadPackageTranslations(TestCaseWithFactory): | ||
337 | 80 | """Test `ReuploadPackageTranslations`.""" | ||
338 | 81 | layer = LaunchpadZopelessLayer | ||
339 | 82 | |||
340 | 83 | def setUp(self): | ||
341 | 84 | super(TestReuploadPackageTranslations, self).setUp() | ||
342 | 85 | sourcepackagename = self.factory.makeSourcePackageName() | ||
343 | 86 | distroseries = self.factory.makeDistroRelease() | ||
344 | 87 | self.sourcepackage = SourcePackage(sourcepackagename, distroseries) | ||
345 | 88 | self.script = ReuploadPackageTranslations('reupload', test_args=[ | ||
346 | 89 | '-d', distroseries.distribution.name, | ||
347 | 90 | '-s', distroseries.name, | ||
348 | 91 | '-p', sourcepackagename.name, | ||
349 | 92 | '-qqq']) | ||
350 | 93 | |||
351 | 94 | def test_findPackage(self): | ||
352 | 95 | # _findPackage finds a SourcePackage by name. | ||
353 | 96 | self.script._setDistroDetails() | ||
354 | 97 | found_package = self.script._findPackage( | ||
355 | 98 | self.sourcepackage.sourcepackagename.name) | ||
356 | 99 | self.assertEqual(self.sourcepackage, found_package) | ||
357 | 100 | |||
358 | 101 | def test_processPackage_nothing(self): | ||
359 | 102 | # A package need not have a translations upload. The script | ||
360 | 103 | # notices this but does nothing about it. | ||
361 | 104 | self.script.main() | ||
362 | 105 | self.assertEqual( | ||
363 | 106 | [self.sourcepackage], self.script.uploadless_packages) | ||
364 | 107 | |||
365 | 108 | def test_processPackage(self): | ||
366 | 109 | # _processPackage will fetch the package's latest translations | ||
367 | 110 | # upload from the Librarian and re-import it. | ||
368 | 111 | translation_files = { | ||
369 | 112 | 'po/messages.pot': '# pot', | ||
370 | 113 | 'po/nl.po': '# nl', | ||
371 | 114 | } | ||
372 | 115 | tar_alias = upload_tarball(translation_files) | ||
373 | 116 | |||
374 | 117 | # Force Librarian update | ||
375 | 118 | transaction.commit() | ||
376 | 119 | |||
377 | 120 | self.script._findPackage = UploadInjector(self.script, tar_alias) | ||
378 | 121 | self.script.main() | ||
379 | 122 | self.assertEqual([], self.script.uploadless_packages) | ||
380 | 123 | |||
381 | 124 | # Force Librarian update | ||
382 | 125 | transaction.commit() | ||
383 | 126 | |||
384 | 127 | queue_summary = summarize_translations_queue(self.sourcepackage) | ||
385 | 128 | self.assertEqual(translation_files, queue_summary) | ||
386 | 129 | |||
387 | 130 | |||
388 | 131 | class TestReuploadScript(TestCaseWithFactory): | ||
389 | 132 | """Test reupload-translations script.""" | ||
390 | 133 | layer = LaunchpadZopelessLayer | ||
391 | 134 | |||
392 | 135 | def setUp(self): | ||
393 | 136 | super(TestReuploadScript, self).setUp() | ||
394 | 137 | self.distroseries = self.factory.makeDistroRelease() | ||
395 | 138 | self.sourcepackagename1 = self.factory.makeSourcePackageName() | ||
396 | 139 | self.sourcepackagename2 = self.factory.makeSourcePackageName() | ||
397 | 140 | transaction.commit() | ||
398 | 141 | |||
399 | 142 | def test_reupload_translations(self): | ||
400 | 143 | """Test a run of the script.""" | ||
401 | 144 | retcode, stdout, stderr = run_script( | ||
402 | 145 | 'scripts/rosetta/reupload-translations.py', [ | ||
403 | 146 | '-d', self.distroseries.distribution.name, | ||
404 | 147 | '-s', self.distroseries.name, | ||
405 | 148 | '-p', self.sourcepackagename1.name, | ||
406 | 149 | '-p', self.sourcepackagename2.name, | ||
407 | 150 | '-vvv', | ||
408 | 151 | '--dry-run', | ||
409 | 152 | ]) | ||
410 | 153 | |||
411 | 154 | self.assertEqual(0, retcode) | ||
412 | 155 | self.assertEqual('', stdout) | ||
413 | 156 | |||
414 | 157 | expected_output = ( | ||
415 | 158 | "INFO\s*Dry run. Not really uploading anything.\n" | ||
416 | 159 | "INFO\s*Processing [^\s]+ in .*\n" | ||
417 | 160 | "WARNING\s*Found no translations upload for .*\n" | ||
418 | 161 | "INFO\s*Processing [^\s]+ in .*\n" | ||
419 | 162 | "WARNING\s*Found no translations upload for .*\n" | ||
420 | 163 | "INFO\s*Done.\n") | ||
421 | 164 | self.assertTrue(re.match(expected_output, stderr)) | ||
422 | 165 | |||
423 | 166 | |||
424 | 167 | def test_suite(): | ||
425 | 168 | return TestLoader().loadTestsFromName(__name__) | ||
426 | 0 | 169 | ||
427 | === added file 'scripts/rosetta/reupload-translations.py' | |||
428 | --- scripts/rosetta/reupload-translations.py 1970-01-01 00:00:00 +0000 | |||
429 | +++ scripts/rosetta/reupload-translations.py 2009-09-30 16:53:17 +0000 | |||
430 | @@ -0,0 +1,19 @@ | |||
431 | 1 | #! /usr/bin/env python2.4 | ||
432 | 2 | # | ||
433 | 3 | # Copyright 2009 Canonical Ltd. This software is licensed under the | ||
434 | 4 | # GNU Affero General Public License version 3 (see the file LICENSE). | ||
435 | 5 | # pylint: disable-msg=W0403 | ||
436 | 6 | |||
437 | 7 | """Re-upload translations from given packages.""" | ||
438 | 8 | |||
439 | 9 | __metaclass__ = type | ||
440 | 10 | |||
441 | 11 | import _pythonpath | ||
442 | 12 | |||
443 | 13 | from lp.translations.scripts.reupload_translations import ( | ||
444 | 14 | ReuploadPackageTranslations) | ||
445 | 15 | |||
446 | 16 | |||
447 | 17 | if __name__ == '__main__': | ||
448 | 18 | script = ReuploadPackageTranslations('reupload-translations') | ||
449 | 19 | script.run() |
= Bug 439346 =
A stupid missing parameter in 3.0 caused the translations auto-approver
to approve uploads for the right packages but the wrong Ubuntu releases.
Due to some other bugs that we've fixed already, we don't know exactly
which translations will or won't be affected. So we ended up with a
fairly long list of packages that _may_ have had the wrong files
imported to them.
This branch provides a script that finds the latest Soyuz-generated
translations upload for a given package (a tarball containing, roughly
speaking, the upstream templates and translations) and re-uploads it.
Finding the latest translations upload for a package turns out to be...
in Julian's words, "easy." I'd probably pick another word myself.
After conferring with Celso (who helped me get it actually working) I
isolated that piece of work in SourcePackage.
You may wonder why the script keeps track of uploadless_ packages. That
is there just to facilitate testing.
There is no single end-to-end test. It's just too hard to set up and
maintain a realistic testing situation for this. Test coverage consists
of several stretches that touch more than overlap:
* A full script run, but without any uploads being found. ations in the
This exercises everything except retrieving and uploading a tarball
from the Librarian, and the tail end of getLatestTransl
success case.
* A before-and-after demonstration of getLatestTransl ationsUploads in
the doctest. This does exercise the tail end in the success case.
It also shows that a gzipped tarball containing the right files has
gone into the Librarian.
* A unit test for _findPackage, to cover up for the fact that other
unit tests patch that method for mocking purposes.
* A successful run of the LaunchpadScript object, but with nslationsUpload s mocked up to return a gzipped tarball
getLatestTra
that was stuffed straight into the Librarian by the test. This also
tests the retrieval and uploading of that tarball.
The tar-file handling in the tests is terrible. I can't really help it;
blame the tarfile module's horrible API. Makes it hard to create
tarballs in-memory using StringIOs.
== Tests == ue-translations .txt translations
{{{
./bin/test -vv -t distroseriesque
./bin/test -vv -t reupload_
}}}
== Demo and Q/A ==
Pick a source package in "main" for some Ubuntu release, one with
translations but not too much in its import queue—there has to be at
least one translation that doesn't have a Needs Review or Approved entry
in the queue. You can create a new translation by translating a string
into a dead language such as Sumerian (sux).
Run the script, passing it the name of the Ubuntu release (e.g. karmic)
and the name of the source package (e.g. tomboy). Try the --dry-run
option first; nothing will actually happen to the package's import
queue. Try the same run again without the --dry-run option, and now a
full set of upstream translations will appear on the queue in Needs
Review state.
Or if you wait too long, Approved or even Imported. The approval and
import scripts do run on staging.
No lint.
Jeroen