Merge lp:~cjwatson/launchpad/split-ftpmaster into lp:launchpad

Proposed by Colin Watson on 2012-01-19
Status: Merged
Approved by: j.c.sackett on 2012-01-19
Approved revision: no longer in the source branch.
Merged at revision: 14703
Proposed branch: lp:~cjwatson/launchpad/split-ftpmaster
Merge into: lp:launchpad
Diff against target: 2170 lines (+950/-914)
21 files modified
lib/lp/archivepublisher/scripts/generate_contents_files.py (+2/-2)
lib/lp/archivepublisher/scripts/publish_ftpmaster.py (+2/-2)
lib/lp/scripts/utilities/importfascist.py (+2/-2)
lib/lp/soyuz/doc/manage-chroot.txt (+6/-4)
lib/lp/soyuz/scripts/chrootmanager.py (+230/-0)
lib/lp/soyuz/scripts/ftpmaster.py (+0/-868)
lib/lp/soyuz/scripts/obsolete_distroseries.py (+101/-0)
lib/lp/soyuz/scripts/packageremover.py (+106/-0)
lib/lp/soyuz/scripts/pubsourcechecker.py (+214/-0)
lib/lp/soyuz/scripts/querydistro.py (+260/-0)
lib/lp/soyuz/scripts/tests/test_chrootmanager.py (+2/-2)
lib/lp/soyuz/scripts/tests/test_lpquerydistro.py (+2/-2)
lib/lp/soyuz/scripts/tests/test_obsoletedistroseries.py (+3/-5)
lib/lp/soyuz/scripts/tests/test_overrides_checker.py (+2/-2)
lib/lp/soyuz/scripts/tests/test_populatearchive.py (+3/-5)
lib/lp/soyuz/scripts/tests/test_removepackage.py (+5/-7)
scripts/ftpmaster-tools/archive-override-check.py (+2/-2)
scripts/ftpmaster-tools/lp-query-distro.py (+2/-2)
scripts/ftpmaster-tools/lp-remove-package.py (+2/-3)
scripts/ftpmaster-tools/manage-chroot.py (+2/-3)
scripts/ftpmaster-tools/obsolete-distroseries.py (+2/-3)
To merge this branch: bzr merge lp:~cjwatson/launchpad/split-ftpmaster
Reviewer Review Type Date Requested Status
j.c.sackett (community) 2012-01-19 Approve on 2012-01-19
Richard Harding (community) code* 2012-01-19 Approve on 2012-01-19
Review via email: mp+89193@code.launchpad.net

Commit Message

[r=jcsackett,rharding][bug=75621] Splits ftpmaster into several smaller files. Previously ftpmaster was sort of a grab bag of partially related objects.

Description of the Change

== Summary ==

Celso noted in 2006 that the package that's now lp.soyuz.scripts.ftpmaster ought to be split up, as it consists of several independent pieces that share no context. It hasn't got a whole lot better since then. I split it up into five new packages:

  lp.soyuz.scripts.chrootmanager
  lp.soyuz.scripts.obsolete_distroseries
  lp.soyuz.scripts.packageremover
  lp.soyuz.scripts.pubsourcechecker
  lp.soyuz.scripts.querydistro

Picking the package names was a bit awkward, as lp.soyuz.scripts has a mix of words run together and words separated by underscores. obsolete_distroseries was by analogy with initialize_distroseries, but for the rest it seemed to feel more natural to run words together.

The splitting was fairly mechanical and aided by pyflakes.vim to cut the imports down afterwards.

== lint ==

Some remaining long lines in lib/lp/soyuz/doc/manage-chroot.txt which I'm not sure if it's worth cleaning up.

To post a comment you must log in.
Richard Harding (rharding) wrote :

Looks good to me. I agree that obsolete_distroseries reads as if it's three words which gets long for running together.

Per our discussion in IRC, my only concern was that the ftpmaster was removed warned_database_imports but none of the modules took its place as potentially something that shouldn't be imported.

review: Approve (code*)
j.c.sackett (jcsackett) wrote :

Colin-

This is quie a lot to have moved around! I see no problems beyond what Rick mentioned, which I see you have already addressed.

This looks good.

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'lib/lp/archivepublisher/scripts/generate_contents_files.py'
2--- lib/lp/archivepublisher/scripts/generate_contents_files.py 2012-01-01 02:58:52 +0000
3+++ lib/lp/archivepublisher/scripts/generate_contents_files.py 2012-01-19 17:27:32 +0000
4@@ -1,4 +1,4 @@
5-# Copyright 2011 Canonical Ltd. This software is licensed under the
6+# Copyright 2011-2012 Canonical Ltd. This software is licensed under the
7 # GNU Affero General Public License version 3 (see the file LICENSE).
8
9 """Archive Contents files generator."""
10@@ -31,7 +31,7 @@
11 DatabaseBlockedPolicy,
12 SlaveOnlyDatabasePolicy,
13 )
14-from lp.soyuz.scripts.ftpmaster import LpQueryDistro
15+from lp.soyuz.scripts.querydistro import LpQueryDistro
16
17
18 COMPONENTS = [
19
20=== modified file 'lib/lp/archivepublisher/scripts/publish_ftpmaster.py'
21--- lib/lp/archivepublisher/scripts/publish_ftpmaster.py 2012-01-01 02:58:52 +0000
22+++ lib/lp/archivepublisher/scripts/publish_ftpmaster.py 2012-01-19 17:27:32 +0000
23@@ -1,4 +1,4 @@
24-# Copyright 2011 Canonical Ltd. This software is licensed under the
25+# Copyright 2011-2012 Canonical Ltd. This software is licensed under the
26 # GNU Affero General Public License version 3 (see the file LICENSE).
27
28 """Master distro publishing script."""
29@@ -28,7 +28,7 @@
30 from lp.services.utils import file_exists
31 from lp.soyuz.enums import ArchivePurpose
32 from lp.soyuz.scripts.custom_uploads_copier import CustomUploadsCopier
33-from lp.soyuz.scripts.ftpmaster import LpQueryDistro
34+from lp.soyuz.scripts.querydistro import LpQueryDistro
35 from lp.soyuz.scripts.processaccepted import ProcessAccepted
36 from lp.soyuz.scripts.publishdistro import PublishDistro
37
38
39=== modified file 'lib/lp/scripts/utilities/importfascist.py'
40--- lib/lp/scripts/utilities/importfascist.py 2011-12-30 01:48:17 +0000
41+++ lib/lp/scripts/utilities/importfascist.py 2012-01-19 17:27:32 +0000
42@@ -1,4 +1,4 @@
43-# Copyright 2009 Canonical Ltd. This software is licensed under the
44+# Copyright 2009-2012 Canonical Ltd. This software is licensed under the
45 # GNU Affero General Public License version 3 (see the file LICENSE).
46
47 import __builtin__
48@@ -47,7 +47,7 @@
49
50
51 warned_database_imports = text_lines_to_set("""
52- lp.soyuz.scripts.ftpmaster
53+ lp.soyuz.scripts.obsolete_distroseries
54 lp.soyuz.scripts.gina.handlers
55 lp.registry.browser.distroseries
56 lp.translations.scripts.po_import
57
58=== modified file 'lib/lp/soyuz/doc/manage-chroot.txt'
59--- lib/lp/soyuz/doc/manage-chroot.txt 2012-01-06 11:08:30 +0000
60+++ lib/lp/soyuz/doc/manage-chroot.txt 2012-01-19 17:27:32 +0000
61@@ -1,4 +1,6 @@
62-= Manage-Chroot Tool =
63+==================
64+Manage-Chroot Tool
65+==================
66
67 This tool is used to add or update chroots for suites (distroseries)
68 and DistroArchSeries.
69@@ -35,7 +37,7 @@
70 parameters.
71
72 >>> from lp.services.log.logger import FakeLogger
73- >>> from lp.soyuz.scripts.ftpmaster import ManageChrootScript
74+ >>> from lp.soyuz.scripts.chrootmanager import ManageChrootScript
75 >>> def getScriptObject(command, distribution='ubuntu', suite='hoary',
76 ... arch='i386', filepath=filepath):
77 ... test_args = ['-s', suite,
78@@ -92,7 +94,7 @@
79 ERROR Allowed actions: ['add', 'update', 'remove', 'get']
80 Unknown action: bogus
81
82-Specifiying a bad architecture results in an error.
83+Specifying a bad architecture results in an error.
84
85 >>> manage_chroot = getScriptObject("add", arch="bogus")
86 >>> try:
87@@ -176,7 +178,7 @@
88 DEBUG Initializing ChrootManager for 'The Hoary Hedgehog Release for i386 (x86)'
89 Chroot was deleted.
90
91- >>> os.remove(filepath)
92+ >>> os.remove(filepath)
93
94 When the librarian is not running, attempting to upload a chroot file
95 results in an appropriate error.
96
97=== added file 'lib/lp/soyuz/scripts/chrootmanager.py'
98--- lib/lp/soyuz/scripts/chrootmanager.py 1970-01-01 00:00:00 +0000
99+++ lib/lp/soyuz/scripts/chrootmanager.py 2012-01-19 17:27:32 +0000
100@@ -0,0 +1,230 @@
101+# Copyright 2009-2012 Canonical Ltd. This software is licensed under the
102+# GNU Affero General Public License version 3 (see the file LICENSE).
103+
104+"""Chroot management utilities."""
105+
106+__metaclass__ = type
107+
108+__all__ = [
109+ 'ChrootManager',
110+ 'ChrootManagerError',
111+ 'ManageChrootScript',
112+ ]
113+
114+import os
115+
116+from zope.component import getUtility
117+
118+from lp.app.errors import NotFoundError
119+from lp.services.helpers import filenameToContentType
120+from lp.services.librarian.interfaces import ILibraryFileAliasSet
121+from lp.services.librarian.interfaces.client import (
122+ ILibrarianClient,
123+ UploadFailed,
124+ )
125+from lp.services.librarian.utils import copy_and_close
126+from lp.soyuz.scripts.ftpmasterbase import (
127+ SoyuzScript,
128+ SoyuzScriptError,
129+ )
130+
131+
132+class ChrootManagerError(Exception):
133+ """Any error generated during the ChrootManager procedures."""
134+
135+
136+class ChrootManager:
137+ """Chroot actions wrapper.
138+
139+ The 'distroarchseries' argument is mandatory and 'filepath' is
140+ optional.
141+
142+ 'filepath' is required by some allowed actions as source or destination,
143+
144+ ChrootManagerError will be raised if anything wrong occurred in this
145+ class, things like missing parameter or infrastructure pieces not in
146+ place.
147+ """
148+
149+ allowed_actions = ['add', 'update', 'remove', 'get']
150+
151+ def __init__(self, distroarchseries, filepath=None):
152+ self.distroarchseries = distroarchseries
153+ self.filepath = filepath
154+ self._messages = []
155+
156+ def _upload(self):
157+ """Upload the self.filepath contents to Librarian.
158+
159+ Return the respective ILibraryFileAlias instance.
160+ Raises ChrootManagerError if it could not be found.
161+ """
162+ try:
163+ fd = open(self.filepath)
164+ except IOError:
165+ raise ChrootManagerError('Could not open: %s' % self.filepath)
166+
167+ flen = os.stat(self.filepath).st_size
168+ filename = os.path.basename(self.filepath)
169+ ftype = filenameToContentType(filename)
170+
171+ try:
172+ alias_id = getUtility(ILibrarianClient).addFile(
173+ filename, flen, fd, contentType=ftype)
174+ except UploadFailed, info:
175+ raise ChrootManagerError("Librarian upload failed: %s" % info)
176+
177+ lfa = getUtility(ILibraryFileAliasSet)[alias_id]
178+
179+ self._messages.append(
180+ "LibraryFileAlias: %d, %s bytes, %s"
181+ % (lfa.id, lfa.content.filesize, lfa.content.md5))
182+
183+ return lfa
184+
185+ def _getPocketChroot(self):
186+ """Retrive PocketChroot record.
187+
188+ Return the respective IPocketChroot instance.
189+ Raises ChrootManagerError if it could not be found.
190+ """
191+ pocket_chroot = self.distroarchseries.getPocketChroot()
192+ if pocket_chroot is None:
193+ raise ChrootManagerError(
194+ 'Could not find chroot for %s'
195+ % (self.distroarchseries.title))
196+
197+ self._messages.append(
198+ "PocketChroot for '%s' (%d) retrieved."
199+ % (pocket_chroot.distroarchseries.title, pocket_chroot.id))
200+
201+ return pocket_chroot
202+
203+ def _update(self):
204+ """Base method for add and update action."""
205+ if self.filepath is None:
206+ raise ChrootManagerError('Missing local chroot file path.')
207+ alias = self._upload()
208+ return self.distroarchseries.addOrUpdateChroot(alias)
209+
210+ def add(self):
211+ """Create a new PocketChroot record.
212+
213+ Raises ChrootManagerError if self.filepath isn't set.
214+ Update of pre-existing PocketChroot record will be automatically
215+ handled.
216+ It's a bind to the self.update method.
217+ """
218+ pocket_chroot = self._update()
219+ self._messages.append(
220+ "PocketChroot for '%s' (%d) added."
221+ % (pocket_chroot.distroarchseries.title, pocket_chroot.id))
222+
223+ def update(self):
224+ """Update a PocketChroot record.
225+
226+ Raises ChrootManagerError if filepath isn't set
227+ Creation of non-existing PocketChroot records will be automatically
228+ handled.
229+ """
230+ pocket_chroot = self._update()
231+ self._messages.append(
232+ "PocketChroot for '%s' (%d) updated."
233+ % (pocket_chroot.distroarchseries.title, pocket_chroot.id))
234+
235+ def remove(self):
236+ """Overwrite existing PocketChroot file to none.
237+
238+ Raises ChrootManagerError if the chroot record isn't found.
239+ """
240+ pocket_chroot = self._getPocketChroot()
241+ self.distroarchseries.addOrUpdateChroot(None)
242+ self._messages.append(
243+ "PocketChroot for '%s' (%d) removed."
244+ % (pocket_chroot.distroarchseries.title, pocket_chroot.id))
245+
246+ def get(self):
247+ """Download chroot file from Librarian and store."""
248+ pocket_chroot = self._getPocketChroot()
249+
250+ if self.filepath is None:
251+ abs_filepath = os.path.abspath(pocket_chroot.chroot.filename)
252+ if os.path.exists(abs_filepath):
253+ raise ChrootManagerError(
254+ 'cannot overwrite %s' % abs_filepath)
255+ self._messages.append(
256+ "Writing to '%s'." % abs_filepath)
257+ local_file = open(pocket_chroot.chroot.filename, "w")
258+ else:
259+ abs_filepath = os.path.abspath(self.filepath)
260+ if os.path.exists(abs_filepath):
261+ raise ChrootManagerError(
262+ 'cannot overwrite %s' % abs_filepath)
263+ self._messages.append(
264+ "Writing to '%s'." % abs_filepath)
265+ local_file = open(abs_filepath, "w")
266+
267+ if pocket_chroot.chroot is None:
268+ raise ChrootManagerError('Chroot was deleted.')
269+
270+ pocket_chroot.chroot.open()
271+ copy_and_close(pocket_chroot.chroot, local_file)
272+
273+
274+class ManageChrootScript(SoyuzScript):
275+ """`SoyuzScript` that manages chroot files."""
276+
277+ usage = "%prog -d <distribution> -s <suite> -a <architecture> -f file"
278+ description = "Manage the chroot files used by the builders."
279+ success_message = "Success."
280+
281+ def add_my_options(self):
282+ """Add script options."""
283+ SoyuzScript.add_distro_options(self)
284+ SoyuzScript.add_transaction_options(self)
285+ self.parser.add_option(
286+ '-a', '--architecture', dest='architecture', default=None,
287+ help='Architecture tag')
288+ self.parser.add_option(
289+ '-f', '--filepath', dest='filepath', default=None,
290+ help='Chroot file path')
291+
292+ def mainTask(self):
293+ """Set up a ChrootManager object and invoke it."""
294+ if len(self.args) != 1:
295+ raise SoyuzScriptError(
296+ "manage-chroot.py <add|update|remove|get>")
297+
298+ [action] = self.args
299+
300+ series = self.location.distroseries
301+
302+ try:
303+ distroarchseries = series[self.options.architecture]
304+ except NotFoundError, info:
305+ raise SoyuzScriptError("Architecture not found: %s" % info)
306+
307+ # We don't want to have to force the user to confirm transactions
308+ # for manage-chroot.py, so disable that feature of SoyuzScript.
309+ self.options.confirm_all = True
310+
311+ self.logger.debug(
312+ "Initializing ChrootManager for '%s'" % (distroarchseries.title))
313+ chroot_manager = ChrootManager(
314+ distroarchseries, filepath=self.options.filepath)
315+
316+ if action in chroot_manager.allowed_actions:
317+ chroot_action = getattr(chroot_manager, action)
318+ else:
319+ self.logger.error(
320+ "Allowed actions: %s" % chroot_manager.allowed_actions)
321+ raise SoyuzScriptError("Unknown action: %s" % action)
322+
323+ try:
324+ chroot_action()
325+ except ChrootManagerError, info:
326+ raise SoyuzScriptError(info)
327+ else:
328+ # Collect extra debug messages from chroot_manager.
329+ for debug_message in chroot_manager._messages:
330+ self.logger.debug(debug_message)
331
332=== removed file 'lib/lp/soyuz/scripts/ftpmaster.py'
333--- lib/lp/soyuz/scripts/ftpmaster.py 2012-01-11 11:50:01 +0000
334+++ lib/lp/soyuz/scripts/ftpmaster.py 1970-01-01 00:00:00 +0000
335@@ -1,868 +0,0 @@
336-# Copyright 2009-2012 Canonical Ltd. This software is licensed under the
337-# GNU Affero General Public License version 3 (see the file LICENSE).
338-
339-"""FTPMaster utilities."""
340-
341-__metaclass__ = type
342-
343-__all__ = [
344- 'ChrootManager',
345- 'ChrootManagerError',
346- 'LpQueryDistro',
347- 'ManageChrootScript',
348- 'ObsoleteDistroseries',
349- 'PackageRemover',
350- 'PubSourceChecker',
351- ]
352-
353-from itertools import chain
354-import os
355-
356-from zope.component import getUtility
357-
358-from lp.app.errors import NotFoundError
359-from lp.registry.interfaces.person import IPersonSet
360-from lp.registry.interfaces.pocket import pocketsuffix
361-from lp.registry.interfaces.series import SeriesStatus
362-from lp.services.browser_helpers import get_plural_text
363-from lp.services.database.constants import UTC_NOW
364-from lp.services.helpers import filenameToContentType
365-from lp.services.librarian.interfaces import ILibraryFileAliasSet
366-from lp.services.librarian.interfaces.client import (
367- ILibrarianClient,
368- UploadFailed,
369- )
370-from lp.services.librarian.utils import copy_and_close
371-from lp.services.scripts.base import (
372- LaunchpadScript,
373- LaunchpadScriptFailure,
374- )
375-from lp.soyuz.adapters.packagelocation import (
376- build_package_location,
377- PackageLocationError,
378- )
379-from lp.soyuz.enums import PackagePublishingStatus
380-from lp.soyuz.scripts.ftpmasterbase import (
381- SoyuzScript,
382- SoyuzScriptError,
383- )
384-
385-
386-class PubBinaryContent:
387- """Binary publication container.
388-
389- Currently used for auxiliary storage in PubSourceChecker.
390- """
391-
392- def __init__(self, name, version, arch, component, section, priority):
393- self.name = name
394- self.version = version
395- self.arch = arch
396- self.component = component
397- self.section = section
398- self.priority = priority
399- self.messages = []
400-
401- def warn(self, message):
402- """Append a warning in the message list."""
403- self.messages.append('W: %s' % message)
404-
405- def error(self, message):
406- """Append a error in the message list."""
407- self.messages.append('E: %s' % message)
408-
409- def renderReport(self):
410- """Render a report with the appended messages (self.messages).
411-
412- Return None if no message was found, otherwise return
413- a properly formatted string, including
414-
415- <TAB>BinaryName_Version Arch Component/Section/Priority
416- <TAB><TAB>MESSAGE
417- """
418- if not len(self.messages):
419- return
420-
421- report = [('\t%s_%s %s %s/%s/%s'
422- % (self.name, self.version, self.arch,
423- self.component, self.section, self.priority))]
424-
425- for message in self.messages:
426- report.append('\t\t%s' % message)
427-
428- return "\n".join(report)
429-
430-
431-class PubBinaryDetails:
432- """Store the component, section and priority of binary packages and, for
433- each binary package the most frequent component, section and priority.
434-
435- These are stored in the following attributes:
436-
437- - components: A dictionary mapping binary package names to other
438- dictionaries mapping component names to binary packages published
439- in this component.
440- - sections: The same as components, but for sections.
441- - priorities: The same as components, but for priorities.
442- - correct_components: a dictionary mapping binary package name
443- to the most frequent (considered the correct) component name.
444- - correct_sections: same as correct_components, but for sections
445- - correct_priorities: same as correct_components, but for priorities
446- """
447-
448- def __init__(self):
449- self.components = {}
450- self.sections = {}
451- self.priorities = {}
452- self.correct_components = {}
453- self.correct_sections = {}
454- self.correct_priorities = {}
455-
456- def addBinaryDetails(self, bin):
457- """Include a binary publication and update internal registers."""
458- name_components = self.components.setdefault(bin.name, {})
459- bin_component = name_components.setdefault(bin.component, [])
460- bin_component.append(bin)
461-
462- name_sections = self.sections.setdefault(bin.name, {})
463- bin_section = name_sections.setdefault(bin.section, [])
464- bin_section.append(bin)
465-
466- name_priorities = self.priorities.setdefault(bin.name, {})
467- bin_priority = name_priorities.setdefault(bin.priority, [])
468- bin_priority.append(bin)
469-
470- def _getMostFrequentValue(self, data):
471- """Return a dict of name and the most frequent value.
472-
473- Used for self.{components, sections, priorities}
474- """
475- results = {}
476-
477- for name, items in data.iteritems():
478- highest = 0
479- for item, occurrences in items.iteritems():
480- if len(occurrences) > highest:
481- highest = len(occurrences)
482- results[name] = item
483-
484- return results
485-
486- def setCorrectValues(self):
487- """Find out the correct values for the same binary name
488-
489- Consider correct the most frequent.
490- """
491- self.correct_components = self._getMostFrequentValue(self.components)
492- self.correct_sections = self._getMostFrequentValue(self.sections)
493- self.correct_priorities = self._getMostFrequentValue(self.priorities)
494-
495-
496-class PubSourceChecker:
497- """Map and probe a Source/Binaries publication couple.
498-
499- Receive the source publication data and its binaries and perform
500- a group of heuristic consistency checks.
501- """
502-
503- def __init__(self, name, version, component, section, urgency):
504- self.name = name
505- self.version = version
506- self.component = component
507- self.section = section
508- self.urgency = urgency
509- self.binaries = []
510- self.binaries_details = PubBinaryDetails()
511-
512- def addBinary(self, name, version, architecture, component, section,
513- priority):
514- """Append the binary data to the current publication list."""
515- bin = PubBinaryContent(
516- name, version, architecture, component, section, priority)
517-
518- self.binaries.append(bin)
519-
520- self.binaries_details.addBinaryDetails(bin)
521-
522- def check(self):
523- """Setup check environment and perform the required checks."""
524- self.binaries_details.setCorrectValues()
525-
526- for bin in self.binaries:
527- self._checkComponent(bin)
528- self._checkSection(bin)
529- self._checkPriority(bin)
530-
531- def _checkComponent(self, bin):
532- """Check if the binary component matches the correct component.
533-
534- 'correct' is the most frequent component in this binary package
535- group
536- """
537- correct_component = self.binaries_details.correct_components[bin.name]
538- if bin.component != correct_component:
539- bin.warn('Component mismatch: %s != %s'
540- % (bin.component, correct_component))
541-
542- def _checkSection(self, bin):
543- """Check if the binary section matches the correct section.
544-
545- 'correct' is the most frequent section in this binary package
546- group
547- """
548- correct_section = self.binaries_details.correct_sections[bin.name]
549- if bin.section != correct_section:
550- bin.warn('Section mismatch: %s != %s'
551- % (bin.section, correct_section))
552-
553- def _checkPriority(self, bin):
554- """Check if the binary priority matches the correct priority.
555-
556- 'correct' is the most frequent priority in this binary package
557- group
558- """
559- correct_priority = self.binaries_details.correct_priorities[bin.name]
560- if bin.priority != correct_priority:
561- bin.warn('Priority mismatch: %s != %s'
562- % (bin.priority, correct_priority))
563-
564- def renderReport(self):
565- """Render a formatted report for the publication group.
566-
567- Return None if no issue was annotated or an formatted string
568- including:
569-
570- SourceName_Version Component/Section/Urgency | # bin
571- <BINREPORTS>
572- """
573- report = []
574-
575- for bin in self.binaries:
576- bin_report = bin.renderReport()
577- if bin_report:
578- report.append(bin_report)
579-
580- if not len(report):
581- return
582-
583- result = [('%s_%s %s/%s/%s | %s bin'
584- % (self.name, self.version, self.component,
585- self.section, self.urgency, len(self.binaries)))]
586-
587- result.extend(report)
588-
589- return "\n".join(result)
590-
591-
592-class ChrootManagerError(Exception):
593- """Any error generated during the ChrootManager procedures."""
594-
595-
596-class ChrootManager:
597- """Chroot actions wrapper.
598-
599- The 'distroarchseries' argument is mandatory and 'filepath' is
600- optional.
601-
602- 'filepath' is required by some allowed actions as source or destination,
603-
604- ChrootManagerError will be raised if anything wrong occurred in this
605- class, things like missing parameter or infrastructure pieces not in
606- place.
607- """
608-
609- allowed_actions = ['add', 'update', 'remove', 'get']
610-
611- def __init__(self, distroarchseries, filepath=None):
612- self.distroarchseries = distroarchseries
613- self.filepath = filepath
614- self._messages = []
615-
616- def _upload(self):
617- """Upload the self.filepath contents to Librarian.
618-
619- Return the respective ILibraryFileAlias instance.
620- Raises ChrootManagerError if it could not be found.
621- """
622- try:
623- fd = open(self.filepath)
624- except IOError:
625- raise ChrootManagerError('Could not open: %s' % self.filepath)
626-
627- flen = os.stat(self.filepath).st_size
628- filename = os.path.basename(self.filepath)
629- ftype = filenameToContentType(filename)
630-
631- try:
632- alias_id = getUtility(ILibrarianClient).addFile(
633- filename, flen, fd, contentType=ftype)
634- except UploadFailed, info:
635- raise ChrootManagerError("Librarian upload failed: %s" % info)
636-
637- lfa = getUtility(ILibraryFileAliasSet)[alias_id]
638-
639- self._messages.append(
640- "LibraryFileAlias: %d, %s bytes, %s"
641- % (lfa.id, lfa.content.filesize, lfa.content.md5))
642-
643- return lfa
644-
645- def _getPocketChroot(self):
646- """Retrive PocketChroot record.
647-
648- Return the respective IPocketChroot instance.
649- Raises ChrootManagerError if it could not be found.
650- """
651- pocket_chroot = self.distroarchseries.getPocketChroot()
652- if pocket_chroot is None:
653- raise ChrootManagerError(
654- 'Could not find chroot for %s'
655- % (self.distroarchseries.title))
656-
657- self._messages.append(
658- "PocketChroot for '%s' (%d) retrieved."
659- % (pocket_chroot.distroarchseries.title, pocket_chroot.id))
660-
661- return pocket_chroot
662-
663- def _update(self):
664- """Base method for add and update action."""
665- if self.filepath is None:
666- raise ChrootManagerError('Missing local chroot file path.')
667- alias = self._upload()
668- return self.distroarchseries.addOrUpdateChroot(alias)
669-
670- def add(self):
671- """Create a new PocketChroot record.
672-
673- Raises ChrootManagerError if self.filepath isn't set.
674- Update of pre-existing PocketChroot record will be automatically
675- handled.
676- It's a bind to the self.update method.
677- """
678- pocket_chroot = self._update()
679- self._messages.append(
680- "PocketChroot for '%s' (%d) added."
681- % (pocket_chroot.distroarchseries.title, pocket_chroot.id))
682-
683- def update(self):
684- """Update a PocketChroot record.
685-
686- Raises ChrootManagerError if filepath isn't set
687- Creation of non-existing PocketChroot records will be automatically
688- handled.
689- """
690- pocket_chroot = self._update()
691- self._messages.append(
692- "PocketChroot for '%s' (%d) updated."
693- % (pocket_chroot.distroarchseries.title, pocket_chroot.id))
694-
695- def remove(self):
696- """Overwrite existing PocketChroot file to none.
697-
698- Raises ChrootManagerError if the chroot record isn't found.
699- """
700- pocket_chroot = self._getPocketChroot()
701- self.distroarchseries.addOrUpdateChroot(None)
702- self._messages.append(
703- "PocketChroot for '%s' (%d) removed."
704- % (pocket_chroot.distroarchseries.title, pocket_chroot.id))
705-
706- def get(self):
707- """Download chroot file from Librarian and store."""
708- pocket_chroot = self._getPocketChroot()
709-
710- if self.filepath is None:
711- abs_filepath = os.path.abspath(pocket_chroot.chroot.filename)
712- if os.path.exists(abs_filepath):
713- raise ChrootManagerError(
714- 'cannot overwrite %s' % abs_filepath)
715- self._messages.append(
716- "Writing to '%s'." % abs_filepath)
717- local_file = open(pocket_chroot.chroot.filename, "w")
718- else:
719- abs_filepath = os.path.abspath(self.filepath)
720- if os.path.exists(abs_filepath):
721- raise ChrootManagerError(
722- 'cannot overwrite %s' % abs_filepath)
723- self._messages.append(
724- "Writing to '%s'." % abs_filepath)
725- local_file = open(abs_filepath, "w")
726-
727- if pocket_chroot.chroot is None:
728- raise ChrootManagerError('Chroot was deleted.')
729-
730- pocket_chroot.chroot.open()
731- copy_and_close(pocket_chroot.chroot, local_file)
732-
733-
734-class LpQueryDistro(LaunchpadScript):
735- """Main class for scripts/ftpmaster-tools/lp-query-distro.py."""
736-
737- def __init__(self, *args, **kwargs):
738- """Initialize dynamic 'usage' message and LaunchpadScript parent.
739-
740- Also initialize the list 'allowed_arguments'.
741- """
742- self.allowed_actions = [
743- 'current', 'development', 'supported', 'pending_suites', 'archs',
744- 'official_archs', 'nominated_arch_indep', 'pocket_suffixes']
745- self.usage = '%%prog <%s>' % ' | '.join(self.allowed_actions)
746- LaunchpadScript.__init__(self, *args, **kwargs)
747-
748- def add_my_options(self):
749- """Add 'distribution' and 'suite' context options."""
750- self.parser.add_option(
751- '-d', '--distribution', dest='distribution_name',
752- default='ubuntu', help='Context distribution name.')
753- self.parser.add_option(
754- '-s', '--suite', dest='suite', default=None,
755- help='Context suite name.')
756-
757- def main(self):
758- """Main procedure, basically a runAction wrapper.
759-
760- Execute the given and allowed action using the default presenter
761- (see self.runAction for further information).
762- """
763- self.runAction()
764-
765- def _buildLocation(self):
766- """Build a PackageLocation object
767-
768- The location will correspond to the given 'distribution' and 'suite',
769- Any PackageLocationError occurring at this point will be masked into
770- LaunchpadScriptFailure.
771- """
772- try:
773- self.location = build_package_location(
774- distribution_name=self.options.distribution_name,
775- suite=self.options.suite)
776- except PackageLocationError, err:
777- raise LaunchpadScriptFailure(err)
778-
779- def defaultPresenter(self, result):
780- """Default result presenter.
781-
782- Directly prints result in the standard output (print).
783- """
784- print result
785-
786- def runAction(self, presenter=None):
787- """Run a given initialized action (self.action_name).
788-
789- It accepts an optional 'presenter' which will be used to
790- store/present the action result.
791-
792- Ensure at least one argument was passed, known as 'action'.
793- Verify if the given 'action' is listed as an 'allowed_action'.
794- Raise LaunchpadScriptFailure if those requirements were not
795- accomplished.
796-
797- It builds context 'location' object (see self._buildLocation).
798-
799- It may raise LaunchpadScriptFailure is the 'action' is not properly
800- supported by the current code (missing corresponding property).
801- """
802- if presenter is None:
803- presenter = self.defaultPresenter
804-
805- if len(self.args) != 1:
806- raise LaunchpadScriptFailure('<action> is required')
807-
808- [self.action_name] = self.args
809-
810- if self.action_name not in self.allowed_actions:
811- raise LaunchpadScriptFailure(
812- 'Action "%s" is not supported' % self.action_name)
813-
814- self._buildLocation()
815-
816- try:
817- action_result = getattr(self, self.action_name)
818- except AttributeError:
819- raise AssertionError(
820- "No handler found for action '%s'" % self.action_name)
821-
822- presenter(action_result)
823-
824- def checkNoSuiteDefined(self):
825- """Raises LaunchpadScriptError if a suite location was passed.
826-
827- It is re-used in action properties to avoid conflicting contexts,
828- i.e, passing an arbitrary 'suite' and asking for the CURRENT suite
829- in the context distribution.
830- """
831- if self.options.suite is not None:
832- raise LaunchpadScriptFailure(
833- "Action does not accept defined suite.")
834-
835- @property
836- def current(self):
837- """Return the name of the CURRENT distroseries.
838-
839- It is restricted for the context distribution.
840-
841- It may raise LaunchpadScriptFailure if a suite was passed on the
842- command-line or if not CURRENT distroseries was found.
843- """
844- self.checkNoSuiteDefined()
845- series = self.location.distribution.getSeriesByStatus(
846- SeriesStatus.CURRENT)
847- if not series:
848- raise LaunchpadScriptFailure("No CURRENT series.")
849-
850- return series[0].name
851-
852- @property
853- def development(self):
854- """Return the name of the DEVELOPMENT distroseries.
855-
856- It is restricted for the context distribution.
857-
858- It may raise `LaunchpadScriptFailure` if a suite was passed on the
859- command-line.
860-
861- Return the first FROZEN distroseries found if there is no
862- DEVELOPMENT one available.
863-
864- Raises `NotFoundError` if neither a CURRENT nor a FROZEN
865- candidate could be found.
866- """
867- self.checkNoSuiteDefined()
868- series = None
869- wanted_status = (SeriesStatus.DEVELOPMENT,
870- SeriesStatus.FROZEN)
871- for status in wanted_status:
872- series = self.location.distribution.getSeriesByStatus(status)
873- if series.count() > 0:
874- break
875- else:
876- raise LaunchpadScriptFailure(
877- 'There is no DEVELOPMENT distroseries for %s' %
878- self.location.distribution.name)
879- return series[0].name
880-
881- @property
882- def supported(self):
883- """Return the names of the distroseries currently supported.
884-
885- 'supported' means not EXPERIMENTAL or OBSOLETE.
886-
887- It is restricted for the context distribution.
888-
889- It may raise `LaunchpadScriptFailure` if a suite was passed on the
890- command-line or if there is not supported distroseries for the
891- distribution given.
892-
893- Return a space-separated list of distroseries names.
894- """
895- self.checkNoSuiteDefined()
896- supported_series = []
897- unsupported_status = (SeriesStatus.EXPERIMENTAL,
898- SeriesStatus.OBSOLETE)
899- for distroseries in self.location.distribution:
900- if distroseries.status not in unsupported_status:
901- supported_series.append(distroseries.name)
902-
903- if not supported_series:
904- raise LaunchpadScriptFailure(
905- 'There is no supported distroseries for %s' %
906- self.location.distribution.name)
907-
908- return " ".join(supported_series)
909-
910- @property
911- def pending_suites(self):
912- """Return the suite names containing PENDING publication.
913-
914- It check for sources and/or binary publications.
915- """
916- self.checkNoSuiteDefined()
917- pending_suites = set()
918- pending_sources = self.location.archive.getPublishedSources(
919- status=PackagePublishingStatus.PENDING)
920- for pub in pending_sources:
921- pending_suites.add((pub.distroseries, pub.pocket))
922-
923- pending_binaries = self.location.archive.getAllPublishedBinaries(
924- status=PackagePublishingStatus.PENDING)
925- for pub in pending_binaries:
926- pending_suites.add(
927- (pub.distroarchseries.distroseries, pub.pocket))
928-
929- return " ".join([distroseries.name + pocketsuffix[pocket]
930- for distroseries, pocket in pending_suites])
931-
932- @property
933- def archs(self):
934- """Return a space-separated list of architecture tags.
935-
936- It is restricted for the context distribution and suite.
937- """
938- architectures = self.location.distroseries.architectures
939- return " ".join(arch.architecturetag for arch in architectures)
940-
941- @property
942- def official_archs(self):
943- """Return a space-separated list of official architecture tags.
944-
945- It is restricted to the context distribution and suite.
946- """
947- architectures = self.location.distroseries.architectures
948- return " ".join(arch.architecturetag
949- for arch in architectures
950- if arch.official)
951-
952- @property
953- def nominated_arch_indep(self):
954- """Return the nominated arch indep architecture tag.
955-
956- It is restricted to the context distribution and suite.
957- """
958- series = self.location.distroseries
959- return series.nominatedarchindep.architecturetag
960-
961- @property
962- def pocket_suffixes(self):
963- """Return a space-separated list of non-empty pocket suffixes.
964-
965- The RELEASE pocket (whose suffix is the empty string) is omitted.
966-
967- The returned space-separated string is ordered alphabetically.
968- """
969- sorted_non_empty_suffixes = sorted(
970- suffix for suffix in pocketsuffix.values() if suffix != '')
971- return " ".join(sorted_non_empty_suffixes)
972-
973-
974-class PackageRemover(SoyuzScript):
975- """SoyuzScript implementation for published package removal.."""
976-
977- usage = '%prog -s warty mozilla-firefox'
978- description = 'REMOVE a published package.'
979- success_message = (
980- "The archive will be updated in the next publishing cycle.")
981-
982- def add_my_options(self):
983- """Adding local options."""
984- # XXX cprov 20071025: we need a hook for loading SoyuzScript default
985- # options automatically. This is ugly.
986- SoyuzScript.add_my_options(self)
987-
988- # Mode options.
989- self.parser.add_option("-b", "--binary", dest="binaryonly",
990- default=False, action="store_true",
991- help="Remove binaries only.")
992- self.parser.add_option("-S", "--source-only", dest="sourceonly",
993- default=False, action="store_true",
994- help="Remove source only.")
995-
996- # Removal information options.
997- self.parser.add_option("-u", "--user", dest="user",
998- help="Launchpad user name.")
999- self.parser.add_option("-m", "--removal_comment",
1000- dest="removal_comment",
1001- help="Removal comment")
1002-
1003- def mainTask(self):
1004- """Execute the package removal task.
1005-
1006- Build location and target objects.
1007-
1008- Can raise SoyuzScriptError.
1009- """
1010- if len(self.args) == 0:
1011- raise SoyuzScriptError(
1012- "At least one non-option argument must be given, "
1013- "a package name to be removed.")
1014-
1015- if self.options.user is None:
1016- raise SoyuzScriptError("Launchpad username must be given.")
1017-
1018- if self.options.removal_comment is None:
1019- raise SoyuzScriptError("Removal comment must be given.")
1020-
1021- removed_by = getUtility(IPersonSet).getByName(self.options.user)
1022- if removed_by is None:
1023- raise SoyuzScriptError(
1024- "Invalid launchpad username: %s" % self.options.user)
1025-
1026- removables = []
1027- for packagename in self.args:
1028- if self.options.binaryonly:
1029- removables.extend(
1030- self.findLatestPublishedBinaries(packagename))
1031- elif self.options.sourceonly:
1032- removables.append(self.findLatestPublishedSource(packagename))
1033- else:
1034- source_pub = self.findLatestPublishedSource(packagename)
1035- removables.append(source_pub)
1036- removables.extend(source_pub.getPublishedBinaries())
1037-
1038- self.logger.info("Removing candidates:")
1039- for removable in removables:
1040- self.logger.info('\t%s', removable.displayname)
1041-
1042- self.logger.info("Removed-by: %s", removed_by.displayname)
1043- self.logger.info("Comment: %s", self.options.removal_comment)
1044-
1045- removals = []
1046- for removable in removables:
1047- removable.requestDeletion(
1048- removed_by=removed_by,
1049- removal_comment=self.options.removal_comment)
1050- removals.append(removable)
1051-
1052- if len(removals) == 0:
1053- self.logger.info("No package removed (bug ?!?).")
1054- else:
1055- self.logger.info(
1056- "%d %s successfully removed.", len(removals),
1057- get_plural_text(len(removals), "package", "packages"))
1058-
1059- # Information returned mainly for the benefit of the test harness.
1060- return removals
1061-
1062-
1063-class ObsoleteDistroseries(SoyuzScript):
1064- """`SoyuzScript` that obsoletes a distroseries."""
1065-
1066- usage = "%prog -d <distribution> -s <suite>"
1067- description = ("Make obsolete (schedule for removal) packages in an "
1068- "obsolete distroseries.")
1069-
1070- def add_my_options(self):
1071- """Add -d, -s, dry-run and confirmation options."""
1072- SoyuzScript.add_distro_options(self)
1073- SoyuzScript.add_transaction_options(self)
1074-
1075- def mainTask(self):
1076- """Execute package obsolescence procedure.
1077-
1078- Modules using this class outside of its normal usage in the
1079- main script can call this method to start the copy.
1080-
1081- In this case the caller can override test_args on __init__
1082- to set the command line arguments.
1083-
1084- :raise SoyuzScriptError: If the distroseries is not provided or
1085- it is already obsolete.
1086- """
1087- assert self.location, (
1088- "location is not available, call SoyuzScript.setupLocation() "
1089- "before calling mainTask().")
1090-
1091- # Shortcut variable name to reduce long lines.
1092- distroseries = self.location.distroseries
1093-
1094- self._checkParameters(distroseries)
1095-
1096- self.logger.info("Obsoleting all packages for distroseries %s in "
1097- "the %s distribution." % (
1098- distroseries.name,
1099- distroseries.distribution.name))
1100-
1101- # First, mark all Published sources as Obsolete.
1102- sources = distroseries.getAllPublishedSources()
1103- binaries = distroseries.getAllPublishedBinaries()
1104- self.logger.info(
1105- "Obsoleting published packages (%d sources, %d binaries)."
1106- % (sources.count(), binaries.count()))
1107- for package in chain(sources, binaries):
1108- self.logger.debug("Obsoleting %s" % package.displayname)
1109- package.requestObsolescence()
1110-
1111- # Next, ensure that everything is scheduled for deletion. The
1112- # dominator will normally leave some superseded publications
1113- # uncondemned, for example sources that built NBSed binaries.
1114- sources = distroseries.getAllUncondemnedSources()
1115- binaries = distroseries.getAllUncondemnedBinaries()
1116- self.logger.info(
1117- "Scheduling deletion of other packages (%d sources, %d binaries)."
1118- % (sources.count(), binaries.count()))
1119- for package in chain(sources, binaries):
1120- self.logger.debug(
1121- "Scheduling deletion of %s" % package.displayname)
1122- package.scheduleddeletiondate = UTC_NOW
1123-
1124- # The packages from both phases will be caught by death row
1125- # processing the next time it runs. We skip the domination
1126- # phase in the publisher because it won't consider stable
1127- # distroseries.
1128-
1129- def _checkParameters(self, distroseries):
1130- """Sanity check the supplied script parameters."""
1131- # Did the user provide a suite name? (distribution defaults
1132- # to 'ubuntu' which is fine.)
1133- if distroseries == distroseries.distribution.currentseries:
1134- # SoyuzScript defaults to the latest series. Since this
1135- # will never get obsoleted it's safe to assume that the
1136- # user let this option default, so complain and exit.
1137- raise SoyuzScriptError(
1138- "Please specify a valid distroseries name with -s/--suite "
1139- "and which is not the most recent distroseries.")
1140-
1141- # Is the distroseries in an obsolete state? Bail out now if not.
1142- if distroseries.status != SeriesStatus.OBSOLETE:
1143- raise SoyuzScriptError(
1144- "%s is not at status OBSOLETE." % distroseries.name)
1145-
1146-
1147-class ManageChrootScript(SoyuzScript):
1148- """`SoyuzScript` that manages chroot files."""
1149-
1150- usage = "%prog -d <distribution> -s <suite> -a <architecture> -f file"
1151- description = "Manage the chroot files used by the builders."
1152- success_message = "Success."
1153-
1154- def add_my_options(self):
1155- """Add script options."""
1156- SoyuzScript.add_distro_options(self)
1157- SoyuzScript.add_transaction_options(self)
1158- self.parser.add_option(
1159- '-a', '--architecture', dest='architecture', default=None,
1160- help='Architecture tag')
1161- self.parser.add_option(
1162- '-f', '--filepath', dest='filepath', default=None,
1163- help='Chroot file path')
1164-
1165- def mainTask(self):
1166- """Set up a ChrootManager object and invoke it."""
1167- if len(self.args) != 1:
1168- raise SoyuzScriptError(
1169- "manage-chroot.py <add|update|remove|get>")
1170-
1171- [action] = self.args
1172-
1173- series = self.location.distroseries
1174-
1175- try:
1176- distroarchseries = series[self.options.architecture]
1177- except NotFoundError, info:
1178- raise SoyuzScriptError("Architecture not found: %s" % info)
1179-
1180- # We don't want to have to force the user to confirm transactions
1181- # for manage-chroot.py, so disable that feature of SoyuzScript.
1182- self.options.confirm_all = True
1183-
1184- self.logger.debug(
1185- "Initializing ChrootManager for '%s'" % (distroarchseries.title))
1186- chroot_manager = ChrootManager(
1187- distroarchseries, filepath=self.options.filepath)
1188-
1189- if action in chroot_manager.allowed_actions:
1190- chroot_action = getattr(chroot_manager, action)
1191- else:
1192- self.logger.error(
1193- "Allowed actions: %s" % chroot_manager.allowed_actions)
1194- raise SoyuzScriptError("Unknown action: %s" % action)
1195-
1196- try:
1197- chroot_action()
1198- except ChrootManagerError, info:
1199- raise SoyuzScriptError(info)
1200- else:
1201- # Collect extra debug messages from chroot_manager.
1202- for debug_message in chroot_manager._messages:
1203- self.logger.debug(debug_message)
1204
1205=== added file 'lib/lp/soyuz/scripts/obsolete_distroseries.py'
1206--- lib/lp/soyuz/scripts/obsolete_distroseries.py 1970-01-01 00:00:00 +0000
1207+++ lib/lp/soyuz/scripts/obsolete_distroseries.py 2012-01-19 17:27:32 +0000
1208@@ -0,0 +1,101 @@
1209+# Copyright 2009-2012 Canonical Ltd. This software is licensed under the
1210+# GNU Affero General Public License version 3 (see the file LICENSE).
1211+
1212+"""Make a distroseries obsolete."""
1213+
1214+__metaclass__ = type
1215+
1216+__all__ = ['ObsoleteDistroseries']
1217+
1218+from itertools import chain
1219+
1220+from lp.registry.interfaces.series import SeriesStatus
1221+from lp.services.database.constants import UTC_NOW
1222+from lp.soyuz.scripts.ftpmasterbase import (
1223+ SoyuzScript,
1224+ SoyuzScriptError,
1225+ )
1226+
1227+
1228+class ObsoleteDistroseries(SoyuzScript):
1229+ """`SoyuzScript` that obsoletes a distroseries."""
1230+
1231+ usage = "%prog -d <distribution> -s <suite>"
1232+ description = ("Make obsolete (schedule for removal) packages in an "
1233+ "obsolete distroseries.")
1234+
1235+ def add_my_options(self):
1236+ """Add -d, -s, dry-run and confirmation options."""
1237+ SoyuzScript.add_distro_options(self)
1238+ SoyuzScript.add_transaction_options(self)
1239+
1240+ def mainTask(self):
1241+ """Execute package obsolescence procedure.
1242+
1243+ Modules using this class outside of its normal usage in the
1244+ main script can call this method to start the copy.
1245+
1246+ In this case the caller can override test_args on __init__
1247+ to set the command line arguments.
1248+
1249+ :raise SoyuzScriptError: If the distroseries is not provided or
1250+ it is already obsolete.
1251+ """
1252+ assert self.location, (
1253+ "location is not available, call SoyuzScript.setupLocation() "
1254+ "before calling mainTask().")
1255+
1256+ # Shortcut variable name to reduce long lines.
1257+ distroseries = self.location.distroseries
1258+
1259+ self._checkParameters(distroseries)
1260+
1261+ self.logger.info("Obsoleting all packages for distroseries %s in "
1262+ "the %s distribution." % (
1263+ distroseries.name,
1264+ distroseries.distribution.name))
1265+
1266+ # First, mark all Published sources as Obsolete.
1267+ sources = distroseries.getAllPublishedSources()
1268+ binaries = distroseries.getAllPublishedBinaries()
1269+ self.logger.info(
1270+ "Obsoleting published packages (%d sources, %d binaries)."
1271+ % (sources.count(), binaries.count()))
1272+ for package in chain(sources, binaries):
1273+ self.logger.debug("Obsoleting %s" % package.displayname)
1274+ package.requestObsolescence()
1275+
1276+ # Next, ensure that everything is scheduled for deletion. The
1277+ # dominator will normally leave some superseded publications
1278+ # uncondemned, for example sources that built NBSed binaries.
1279+ sources = distroseries.getAllUncondemnedSources()
1280+ binaries = distroseries.getAllUncondemnedBinaries()
1281+ self.logger.info(
1282+ "Scheduling deletion of other packages (%d sources, %d binaries)."
1283+ % (sources.count(), binaries.count()))
1284+ for package in chain(sources, binaries):
1285+ self.logger.debug(
1286+ "Scheduling deletion of %s" % package.displayname)
1287+ package.scheduleddeletiondate = UTC_NOW
1288+
1289+ # The packages from both phases will be caught by death row
1290+ # processing the next time it runs. We skip the domination
1291+ # phase in the publisher because it won't consider stable
1292+ # distroseries.
1293+
1294+ def _checkParameters(self, distroseries):
1295+ """Sanity check the supplied script parameters."""
1296+ # Did the user provide a suite name? (distribution defaults
1297+ # to 'ubuntu' which is fine.)
1298+ if distroseries == distroseries.distribution.currentseries:
1299+ # SoyuzScript defaults to the latest series. Since this
1300+ # will never get obsoleted it's safe to assume that the
1301+ # user let this option default, so complain and exit.
1302+ raise SoyuzScriptError(
1303+ "Please specify a valid distroseries name with -s/--suite "
1304+ "and which is not the most recent distroseries.")
1305+
1306+ # Is the distroseries in an obsolete state? Bail out now if not.
1307+ if distroseries.status != SeriesStatus.OBSOLETE:
1308+ raise SoyuzScriptError(
1309+ "%s is not at status OBSOLETE." % distroseries.name)
1310
1311=== added file 'lib/lp/soyuz/scripts/packageremover.py'
1312--- lib/lp/soyuz/scripts/packageremover.py 1970-01-01 00:00:00 +0000
1313+++ lib/lp/soyuz/scripts/packageremover.py 2012-01-19 17:27:32 +0000
1314@@ -0,0 +1,106 @@
1315+# Copyright 2009-2012 Canonical Ltd. This software is licensed under the
1316+# GNU Affero General Public License version 3 (see the file LICENSE).
1317+
1318+"""FTPMaster utilities."""
1319+
1320+__metaclass__ = type
1321+
1322+__all__ = ['PackageRemover']
1323+
1324+from zope.component import getUtility
1325+
1326+from lp.registry.interfaces.person import IPersonSet
1327+from lp.services.browser_helpers import get_plural_text
1328+from lp.soyuz.scripts.ftpmasterbase import (
1329+ SoyuzScript,
1330+ SoyuzScriptError,
1331+ )
1332+
1333+
1334+class PackageRemover(SoyuzScript):
1335+ """SoyuzScript implementation for published package removal.."""
1336+
1337+ usage = '%prog -s warty mozilla-firefox'
1338+ description = 'REMOVE a published package.'
1339+ success_message = (
1340+ "The archive will be updated in the next publishing cycle.")
1341+
1342+ def add_my_options(self):
1343+ """Adding local options."""
1344+ # XXX cprov 20071025: we need a hook for loading SoyuzScript default
1345+ # options automatically. This is ugly.
1346+ SoyuzScript.add_my_options(self)
1347+
1348+ # Mode options.
1349+ self.parser.add_option("-b", "--binary", dest="binaryonly",
1350+ default=False, action="store_true",
1351+ help="Remove binaries only.")
1352+ self.parser.add_option("-S", "--source-only", dest="sourceonly",
1353+ default=False, action="store_true",
1354+ help="Remove source only.")
1355+
1356+ # Removal information options.
1357+ self.parser.add_option("-u", "--user", dest="user",
1358+ help="Launchpad user name.")
1359+ self.parser.add_option("-m", "--removal_comment",
1360+ dest="removal_comment",
1361+ help="Removal comment")
1362+
1363+ def mainTask(self):
1364+ """Execute the package removal task.
1365+
1366+ Build location and target objects.
1367+
1368+ Can raise SoyuzScriptError.
1369+ """
1370+ if len(self.args) == 0:
1371+ raise SoyuzScriptError(
1372+ "At least one non-option argument must be given, "
1373+ "a package name to be removed.")
1374+
1375+ if self.options.user is None:
1376+ raise SoyuzScriptError("Launchpad username must be given.")
1377+
1378+ if self.options.removal_comment is None:
1379+ raise SoyuzScriptError("Removal comment must be given.")
1380+
1381+ removed_by = getUtility(IPersonSet).getByName(self.options.user)
1382+ if removed_by is None:
1383+ raise SoyuzScriptError(
1384+ "Invalid launchpad username: %s" % self.options.user)
1385+
1386+ removables = []
1387+ for packagename in self.args:
1388+ if self.options.binaryonly:
1389+ removables.extend(
1390+ self.findLatestPublishedBinaries(packagename))
1391+ elif self.options.sourceonly:
1392+ removables.append(self.findLatestPublishedSource(packagename))
1393+ else:
1394+ source_pub = self.findLatestPublishedSource(packagename)
1395+ removables.append(source_pub)
1396+ removables.extend(source_pub.getPublishedBinaries())
1397+
1398+ self.logger.info("Removing candidates:")
1399+ for removable in removables:
1400+ self.logger.info('\t%s', removable.displayname)
1401+
1402+ self.logger.info("Removed-by: %s", removed_by.displayname)
1403+ self.logger.info("Comment: %s", self.options.removal_comment)
1404+
1405+ removals = []
1406+ for removable in removables:
1407+ removable.requestDeletion(
1408+ removed_by=removed_by,
1409+ removal_comment=self.options.removal_comment)
1410+ removals.append(removable)
1411+
1412+ if len(removals) == 0:
1413+ self.logger.info("No package removed (bug ?!?).")
1414+ else:
1415+ self.logger.info(
1416+ "%d %s successfully removed.", len(removals),
1417+ get_plural_text(len(removals), "package", "packages"))
1418+
1419+ # Information returned mainly for the benefit of the test harness.
1420+ return removals
1421
1422=== added file 'lib/lp/soyuz/scripts/pubsourcechecker.py'
1423--- lib/lp/soyuz/scripts/pubsourcechecker.py 1970-01-01 00:00:00 +0000
1424+++ lib/lp/soyuz/scripts/pubsourcechecker.py 2012-01-19 17:27:32 +0000
1425@@ -0,0 +1,214 @@
1426+# Copyright 2009-2012 Canonical Ltd. This software is licensed under the
1427+# GNU Affero General Public License version 3 (see the file LICENSE).
1428+
1429+"""FTPMaster utilities."""
1430+
1431+__metaclass__ = type
1432+
1433+__all__ = ['PubSourceChecker']
1434+
1435+
1436+class PubBinaryContent:
1437+ """Binary publication container.
1438+
1439+ Currently used for auxiliary storage in PubSourceChecker.
1440+ """
1441+
1442+ def __init__(self, name, version, arch, component, section, priority):
1443+ self.name = name
1444+ self.version = version
1445+ self.arch = arch
1446+ self.component = component
1447+ self.section = section
1448+ self.priority = priority
1449+ self.messages = []
1450+
1451+ def warn(self, message):
1452+ """Append a warning in the message list."""
1453+ self.messages.append('W: %s' % message)
1454+
1455+ def error(self, message):
1456+ """Append a error in the message list."""
1457+ self.messages.append('E: %s' % message)
1458+
1459+ def renderReport(self):
1460+ """Render a report with the appended messages (self.messages).
1461+
1462+ Return None if no message was found, otherwise return
1463+ a properly formatted string, including
1464+
1465+ <TAB>BinaryName_Version Arch Component/Section/Priority
1466+ <TAB><TAB>MESSAGE
1467+ """
1468+ if not len(self.messages):
1469+ return
1470+
1471+ report = [('\t%s_%s %s %s/%s/%s'
1472+ % (self.name, self.version, self.arch,
1473+ self.component, self.section, self.priority))]
1474+
1475+ for message in self.messages:
1476+ report.append('\t\t%s' % message)
1477+
1478+ return "\n".join(report)
1479+
1480+
1481+class PubBinaryDetails:
1482+ """Store the component, section and priority of binary packages and, for
1483+ each binary package the most frequent component, section and priority.
1484+
1485+ These are stored in the following attributes:
1486+
1487+ - components: A dictionary mapping binary package names to other
1488+ dictionaries mapping component names to binary packages published
1489+ in this component.
1490+ - sections: The same as components, but for sections.
1491+ - priorities: The same as components, but for priorities.
1492+ - correct_components: a dictionary mapping binary package name
1493+ to the most frequent (considered the correct) component name.
1494+ - correct_sections: same as correct_components, but for sections
1495+ - correct_priorities: same as correct_components, but for priorities
1496+ """
1497+
1498+ def __init__(self):
1499+ self.components = {}
1500+ self.sections = {}
1501+ self.priorities = {}
1502+ self.correct_components = {}
1503+ self.correct_sections = {}
1504+ self.correct_priorities = {}
1505+
1506+ def addBinaryDetails(self, bin):
1507+ """Include a binary publication and update internal registers."""
1508+ name_components = self.components.setdefault(bin.name, {})
1509+ bin_component = name_components.setdefault(bin.component, [])
1510+ bin_component.append(bin)
1511+
1512+ name_sections = self.sections.setdefault(bin.name, {})
1513+ bin_section = name_sections.setdefault(bin.section, [])
1514+ bin_section.append(bin)
1515+
1516+ name_priorities = self.priorities.setdefault(bin.name, {})
1517+ bin_priority = name_priorities.setdefault(bin.priority, [])
1518+ bin_priority.append(bin)
1519+
1520+ def _getMostFrequentValue(self, data):
1521+ """Return a dict of name and the most frequent value.
1522+
1523+ Used for self.{components, sections, priorities}
1524+ """
1525+ results = {}
1526+
1527+ for name, items in data.iteritems():
1528+ highest = 0
1529+ for item, occurrences in items.iteritems():
1530+ if len(occurrences) > highest:
1531+ highest = len(occurrences)
1532+ results[name] = item
1533+
1534+ return results
1535+
1536+ def setCorrectValues(self):
1537+ """Find out the correct values for the same binary name
1538+
1539+ Consider correct the most frequent.
1540+ """
1541+ self.correct_components = self._getMostFrequentValue(self.components)
1542+ self.correct_sections = self._getMostFrequentValue(self.sections)
1543+ self.correct_priorities = self._getMostFrequentValue(self.priorities)
1544+
1545+
1546+class PubSourceChecker:
1547+ """Map and probe a Source/Binaries publication couple.
1548+
1549+ Receive the source publication data and its binaries and perform
1550+ a group of heuristic consistency checks.
1551+ """
1552+
1553+ def __init__(self, name, version, component, section, urgency):
1554+ self.name = name
1555+ self.version = version
1556+ self.component = component
1557+ self.section = section
1558+ self.urgency = urgency
1559+ self.binaries = []
1560+ self.binaries_details = PubBinaryDetails()
1561+
1562+ def addBinary(self, name, version, architecture, component, section,
1563+ priority):
1564+ """Append the binary data to the current publication list."""
1565+ bin = PubBinaryContent(
1566+ name, version, architecture, component, section, priority)
1567+
1568+ self.binaries.append(bin)
1569+
1570+ self.binaries_details.addBinaryDetails(bin)
1571+
1572+ def check(self):
1573+ """Setup check environment and perform the required checks."""
1574+ self.binaries_details.setCorrectValues()
1575+
1576+ for bin in self.binaries:
1577+ self._checkComponent(bin)
1578+ self._checkSection(bin)
1579+ self._checkPriority(bin)
1580+
1581+ def _checkComponent(self, bin):
1582+ """Check if the binary component matches the correct component.
1583+
1584+ 'correct' is the most frequent component in this binary package
1585+ group
1586+ """
1587+ correct_component = self.binaries_details.correct_components[bin.name]
1588+ if bin.component != correct_component:
1589+ bin.warn('Component mismatch: %s != %s'
1590+ % (bin.component, correct_component))
1591+
1592+ def _checkSection(self, bin):
1593+ """Check if the binary section matches the correct section.
1594+
1595+ 'correct' is the most frequent section in this binary package
1596+ group
1597+ """
1598+ correct_section = self.binaries_details.correct_sections[bin.name]
1599+ if bin.section != correct_section:
1600+ bin.warn('Section mismatch: %s != %s'
1601+ % (bin.section, correct_section))
1602+
1603+ def _checkPriority(self, bin):
1604+ """Check if the binary priority matches the correct priority.
1605+
1606+ 'correct' is the most frequent priority in this binary package
1607+ group
1608+ """
1609+ correct_priority = self.binaries_details.correct_priorities[bin.name]
1610+ if bin.priority != correct_priority:
1611+ bin.warn('Priority mismatch: %s != %s'
1612+ % (bin.priority, correct_priority))
1613+
1614+ def renderReport(self):
1615+ """Render a formatted report for the publication group.
1616+
1617+ Return None if no issue was annotated or an formatted string
1618+ including:
1619+
1620+ SourceName_Version Component/Section/Urgency | # bin
1621+ <BINREPORTS>
1622+ """
1623+ report = []
1624+
1625+ for bin in self.binaries:
1626+ bin_report = bin.renderReport()
1627+ if bin_report:
1628+ report.append(bin_report)
1629+
1630+ if not len(report):
1631+ return
1632+
1633+ result = [('%s_%s %s/%s/%s | %s bin'
1634+ % (self.name, self.version, self.component,
1635+ self.section, self.urgency, len(self.binaries)))]
1636+
1637+ result.extend(report)
1638+
1639+ return "\n".join(result)
1640
1641=== added file 'lib/lp/soyuz/scripts/querydistro.py'
1642--- lib/lp/soyuz/scripts/querydistro.py 1970-01-01 00:00:00 +0000
1643+++ lib/lp/soyuz/scripts/querydistro.py 2012-01-19 17:27:32 +0000
1644@@ -0,0 +1,260 @@
1645+# Copyright 2009-2012 Canonical Ltd. This software is licensed under the
1646+# GNU Affero General Public License version 3 (see the file LICENSE).
1647+
1648+"""Distribution querying utility."""
1649+
1650+__metaclass__ = type
1651+
1652+__all__ = ['LpQueryDistro']
1653+
1654+from lp.registry.interfaces.pocket import pocketsuffix
1655+from lp.registry.interfaces.series import SeriesStatus
1656+from lp.services.scripts.base import (
1657+ LaunchpadScript,
1658+ LaunchpadScriptFailure,
1659+ )
1660+from lp.soyuz.adapters.packagelocation import (
1661+ build_package_location,
1662+ PackageLocationError,
1663+ )
1664+from lp.soyuz.enums import PackagePublishingStatus
1665+
1666+
1667+class LpQueryDistro(LaunchpadScript):
1668+ """Main class for scripts/ftpmaster-tools/lp-query-distro.py."""
1669+
1670+ def __init__(self, *args, **kwargs):
1671+ """Initialize dynamic 'usage' message and LaunchpadScript parent.
1672+
1673+ Also initialize the list 'allowed_arguments'.
1674+ """
1675+ self.allowed_actions = [
1676+ 'current', 'development', 'supported', 'pending_suites', 'archs',
1677+ 'official_archs', 'nominated_arch_indep', 'pocket_suffixes']
1678+ self.usage = '%%prog <%s>' % ' | '.join(self.allowed_actions)
1679+ LaunchpadScript.__init__(self, *args, **kwargs)
1680+
1681+ def add_my_options(self):
1682+ """Add 'distribution' and 'suite' context options."""
1683+ self.parser.add_option(
1684+ '-d', '--distribution', dest='distribution_name',
1685+ default='ubuntu', help='Context distribution name.')
1686+ self.parser.add_option(
1687+ '-s', '--suite', dest='suite', default=None,
1688+ help='Context suite name.')
1689+
1690+ def main(self):
1691+ """Main procedure, basically a runAction wrapper.
1692+
1693+ Execute the given and allowed action using the default presenter
1694+ (see self.runAction for further information).
1695+ """
1696+ self.runAction()
1697+
1698+ def _buildLocation(self):
1699+ """Build a PackageLocation object
1700+
1701+ The location will correspond to the given 'distribution' and 'suite',
1702+ Any PackageLocationError occurring at this point will be masked into
1703+ LaunchpadScriptFailure.
1704+ """
1705+ try:
1706+ self.location = build_package_location(
1707+ distribution_name=self.options.distribution_name,
1708+ suite=self.options.suite)
1709+ except PackageLocationError, err:
1710+ raise LaunchpadScriptFailure(err)
1711+
1712+ def defaultPresenter(self, result):
1713+ """Default result presenter.
1714+
1715+ Directly prints result in the standard output (print).
1716+ """
1717+ print result
1718+
1719+ def runAction(self, presenter=None):
1720+ """Run a given initialized action (self.action_name).
1721+
1722+ It accepts an optional 'presenter' which will be used to
1723+ store/present the action result.
1724+
1725+ Ensure at least one argument was passed, known as 'action'.
1726+ Verify if the given 'action' is listed as an 'allowed_action'.
1727+ Raise LaunchpadScriptFailure if those requirements were not
1728+ accomplished.
1729+
1730+ It builds context 'location' object (see self._buildLocation).
1731+
1732+ It may raise LaunchpadScriptFailure is the 'action' is not properly
1733+ supported by the current code (missing corresponding property).
1734+ """
1735+ if presenter is None:
1736+ presenter = self.defaultPresenter
1737+
1738+ if len(self.args) != 1:
1739+ raise LaunchpadScriptFailure('<action> is required')
1740+
1741+ [self.action_name] = self.args
1742+
1743+ if self.action_name not in self.allowed_actions:
1744+ raise LaunchpadScriptFailure(
1745+ 'Action "%s" is not supported' % self.action_name)
1746+
1747+ self._buildLocation()
1748+
1749+ try:
1750+ action_result = getattr(self, self.action_name)
1751+ except AttributeError:
1752+ raise AssertionError(
1753+ "No handler found for action '%s'" % self.action_name)
1754+
1755+ presenter(action_result)
1756+
1757+ def checkNoSuiteDefined(self):
1758+ """Raises LaunchpadScriptError if a suite location was passed.
1759+
1760+ It is re-used in action properties to avoid conflicting contexts,
1761+ i.e, passing an arbitrary 'suite' and asking for the CURRENT suite
1762+ in the context distribution.
1763+ """
1764+ if self.options.suite is not None:
1765+ raise LaunchpadScriptFailure(
1766+ "Action does not accept defined suite.")
1767+
1768+ @property
1769+ def current(self):
1770+ """Return the name of the CURRENT distroseries.
1771+
1772+ It is restricted for the context distribution.
1773+
1774+ It may raise LaunchpadScriptFailure if a suite was passed on the
1775+ command-line or if not CURRENT distroseries was found.
1776+ """
1777+ self.checkNoSuiteDefined()
1778+ series = self.location.distribution.getSeriesByStatus(
1779+ SeriesStatus.CURRENT)
1780+ if not series:
1781+ raise LaunchpadScriptFailure("No CURRENT series.")
1782+
1783+ return series[0].name
1784+
1785+ @property
1786+ def development(self):
1787+ """Return the name of the DEVELOPMENT distroseries.
1788+
1789+ It is restricted for the context distribution.
1790+
1791+ It may raise `LaunchpadScriptFailure` if a suite was passed on the
1792+ command-line.
1793+
1794+ Return the first FROZEN distroseries found if there is no
1795+ DEVELOPMENT one available.
1796+
1797+ Raises `NotFoundError` if neither a CURRENT nor a FROZEN
1798+ candidate could be found.
1799+ """
1800+ self.checkNoSuiteDefined()
1801+ series = None
1802+ wanted_status = (SeriesStatus.DEVELOPMENT,
1803+ SeriesStatus.FROZEN)
1804+ for status in wanted_status:
1805+ series = self.location.distribution.getSeriesByStatus(status)
1806+ if series.count() > 0:
1807+ break
1808+ else:
1809+ raise LaunchpadScriptFailure(
1810+ 'There is no DEVELOPMENT distroseries for %s' %
1811+ self.location.distribution.name)
1812+ return series[0].name
1813+
1814+ @property
1815+ def supported(self):
1816+ """Return the names of the distroseries currently supported.
1817+
1818+ 'supported' means not EXPERIMENTAL or OBSOLETE.
1819+
1820+ It is restricted for the context distribution.
1821+
1822+ It may raise `LaunchpadScriptFailure` if a suite was passed on the
1823+ command-line or if there is not supported distroseries for the
1824+ distribution given.
1825+
1826+ Return a space-separated list of distroseries names.
1827+ """
1828+ self.checkNoSuiteDefined()
1829+ supported_series = []
1830+ unsupported_status = (SeriesStatus.EXPERIMENTAL,
1831+ SeriesStatus.OBSOLETE)
1832+ for distroseries in self.location.distribution:
1833+ if distroseries.status not in unsupported_status:
1834+ supported_series.append(distroseries.name)
1835+
1836+ if not supported_series:
1837+ raise LaunchpadScriptFailure(
1838+ 'There is no supported distroseries for %s' %
1839+ self.location.distribution.name)
1840+
1841+ return " ".join(supported_series)
1842+
1843+ @property
1844+ def pending_suites(self):
1845+ """Return the suite names containing PENDING publication.
1846+
1847+ It check for sources and/or binary publications.
1848+ """
1849+ self.checkNoSuiteDefined()
1850+ pending_suites = set()
1851+ pending_sources = self.location.archive.getPublishedSources(
1852+ status=PackagePublishingStatus.PENDING)
1853+ for pub in pending_sources:
1854+ pending_suites.add((pub.distroseries, pub.pocket))
1855+
1856+ pending_binaries = self.location.archive.getAllPublishedBinaries(
1857+ status=PackagePublishingStatus.PENDING)
1858+ for pub in pending_binaries:
1859+ pending_suites.add(
1860+ (pub.distroarchseries.distroseries, pub.pocket))
1861+
1862+ return " ".join([distroseries.name + pocketsuffix[pocket]
1863+ for distroseries, pocket in pending_suites])
1864+
1865+ @property
1866+ def archs(self):
1867+ """Return a space-separated list of architecture tags.
1868+
1869+ It is restricted for the context distribution and suite.
1870+ """
1871+ architectures = self.location.distroseries.architectures
1872+ return " ".join(arch.architecturetag for arch in architectures)
1873+
1874+ @property
1875+ def official_archs(self):
1876+ """Return a space-separated list of official architecture tags.
1877+
1878+ It is restricted to the context distribution and suite.
1879+ """
1880+ architectures = self.location.distroseries.architectures
1881+ return " ".join(arch.architecturetag
1882+ for arch in architectures
1883+ if arch.official)
1884+
1885+ @property
1886+ def nominated_arch_indep(self):
1887+ """Return the nominated arch indep architecture tag.
1888+
1889+ It is restricted to the context distribution and suite.
1890+ """
1891+ series = self.location.distroseries
1892+ return series.nominatedarchindep.architecturetag
1893+
1894+ @property
1895+ def pocket_suffixes(self):
1896+ """Return a space-separated list of non-empty pocket suffixes.
1897+
1898+ The RELEASE pocket (whose suffix is the empty string) is omitted.
1899+
1900+ The returned space-separated string is ordered alphabetically.
1901+ """
1902+ sorted_non_empty_suffixes = sorted(
1903+ suffix for suffix in pocketsuffix.values() if suffix != '')
1904+ return " ".join(sorted_non_empty_suffixes)
1905
1906=== modified file 'lib/lp/soyuz/scripts/tests/test_chrootmanager.py'
1907--- lib/lp/soyuz/scripts/tests/test_chrootmanager.py 2011-12-30 06:14:56 +0000
1908+++ lib/lp/soyuz/scripts/tests/test_chrootmanager.py 2012-01-19 17:27:32 +0000
1909@@ -1,4 +1,4 @@
1910-# Copyright 2009-2010 Canonical Ltd. This software is licensed under the
1911+# Copyright 2009-2012 Canonical Ltd. This software is licensed under the
1912 # GNU Affero General Public License version 3 (see the file LICENSE).
1913
1914 """ChrootManager facilities tests."""
1915@@ -15,7 +15,7 @@
1916 from lp.registry.interfaces.distribution import IDistributionSet
1917 from lp.services.config import config
1918 from lp.services.database.sqlbase import commit
1919-from lp.soyuz.scripts.ftpmaster import (
1920+from lp.soyuz.scripts.chrootmanager import (
1921 ChrootManager,
1922 ChrootManagerError,
1923 )
1924
1925=== modified file 'lib/lp/soyuz/scripts/tests/test_lpquerydistro.py'
1926--- lib/lp/soyuz/scripts/tests/test_lpquerydistro.py 2011-12-30 06:14:56 +0000
1927+++ lib/lp/soyuz/scripts/tests/test_lpquerydistro.py 2012-01-19 17:27:32 +0000
1928@@ -1,4 +1,4 @@
1929-# Copyright 2009-2010 Canonical Ltd. This software is licensed under the
1930+# Copyright 2009-2012 Canonical Ltd. This software is licensed under the
1931 # GNU Affero General Public License version 3 (see the file LICENSE).
1932
1933 __metaclass__ = type
1934@@ -16,7 +16,7 @@
1935 from lp.services.config import config
1936 from lp.services.database.sqlbase import flush_database_updates
1937 from lp.services.scripts.base import LaunchpadScriptFailure
1938-from lp.soyuz.scripts.ftpmaster import LpQueryDistro
1939+from lp.soyuz.scripts.querydistro import LpQueryDistro
1940 from lp.testing.layers import (
1941 LaunchpadLayer,
1942 LaunchpadZopelessLayer,
1943
1944=== modified file 'lib/lp/soyuz/scripts/tests/test_obsoletedistroseries.py'
1945--- lib/lp/soyuz/scripts/tests/test_obsoletedistroseries.py 2011-12-30 06:14:56 +0000
1946+++ lib/lp/soyuz/scripts/tests/test_obsoletedistroseries.py 2012-01-19 17:27:32 +0000
1947@@ -1,4 +1,4 @@
1948-# Copyright 2009-2010 Canonical Ltd. This software is licensed under the
1949+# Copyright 2009-2012 Canonical Ltd. This software is licensed under the
1950 # GNU Affero General Public License version 3 (see the file LICENSE).
1951
1952 __metaclass__ = type
1953@@ -19,10 +19,8 @@
1954 BinaryPackagePublishingHistory,
1955 SourcePackagePublishingHistory,
1956 )
1957-from lp.soyuz.scripts.ftpmaster import (
1958- ObsoleteDistroseries,
1959- SoyuzScriptError,
1960- )
1961+from lp.soyuz.scripts.ftpmasterbase import SoyuzScriptError
1962+from lp.soyuz.scripts.obsolete_distroseries import ObsoleteDistroseries
1963 from lp.testing import (
1964 TestCase,
1965 TestCaseWithFactory,
1966
1967=== modified file 'lib/lp/soyuz/scripts/tests/test_overrides_checker.py'
1968--- lib/lp/soyuz/scripts/tests/test_overrides_checker.py 2010-08-20 20:31:18 +0000
1969+++ lib/lp/soyuz/scripts/tests/test_overrides_checker.py 2012-01-19 17:27:32 +0000
1970@@ -1,4 +1,4 @@
1971-# Copyright 2009-2010 Canonical Ltd. This software is licensed under the
1972+# Copyright 2009-2012 Canonical Ltd. This software is licensed under the
1973 # GNU Affero General Public License version 3 (see the file LICENSE).
1974
1975 """archive-override-check tool base class tests."""
1976@@ -7,7 +7,7 @@
1977
1978 from unittest import TestCase
1979
1980-from lp.soyuz.scripts.ftpmaster import (
1981+from lp.soyuz.scripts.pubsourcechecker import (
1982 PubBinaryContent,
1983 PubBinaryDetails,
1984 PubSourceChecker,
1985
1986=== modified file 'lib/lp/soyuz/scripts/tests/test_populatearchive.py'
1987--- lib/lp/soyuz/scripts/tests/test_populatearchive.py 2012-01-01 02:58:52 +0000
1988+++ lib/lp/soyuz/scripts/tests/test_populatearchive.py 2012-01-19 17:27:32 +0000
1989@@ -1,4 +1,4 @@
1990-# Copyright 2009-2010 Canonical Ltd. This software is licensed under the
1991+# Copyright 2009-2012 Canonical Ltd. This software is licensed under the
1992 # GNU Affero General Public License version 3 (see the file LICENSE).
1993
1994 __metaclass__ = type
1995@@ -18,16 +18,14 @@
1996 from lp.services.config import config
1997 from lp.services.job.interfaces.job import JobStatus
1998 from lp.services.log.logger import BufferLogger
1999+from lp.soyuz.adapters.packagelocation import PackageLocationError
2000 from lp.soyuz.enums import (
2001 ArchivePurpose,
2002 PackagePublishingStatus,
2003 )
2004 from lp.soyuz.interfaces.archive import IArchiveSet
2005 from lp.soyuz.interfaces.binarypackagebuild import IBinaryPackageBuildSet
2006-from lp.soyuz.scripts.ftpmaster import (
2007- PackageLocationError,
2008- SoyuzScriptError,
2009- )
2010+from lp.soyuz.scripts.ftpmasterbase import SoyuzScriptError
2011 from lp.soyuz.scripts.populate_archive import ArchivePopulator
2012 from lp.soyuz.tests.test_publishing import SoyuzTestPublisher
2013 from lp.testing import TestCaseWithFactory
2014
2015=== modified file 'lib/lp/soyuz/scripts/tests/test_removepackage.py'
2016--- lib/lp/soyuz/scripts/tests/test_removepackage.py 2012-01-01 02:58:52 +0000
2017+++ lib/lp/soyuz/scripts/tests/test_removepackage.py 2012-01-19 17:27:32 +0000
2018@@ -1,4 +1,4 @@
2019-# Copyright 2009 Canonical Ltd. This software is licensed under the
2020+# Copyright 2009-2012 Canonical Ltd. This software is licensed under the
2021 # GNU Affero General Public License version 3 (see the file LICENSE).
2022
2023 """Functional Tests for PackageRemover script class.
2024@@ -26,10 +26,8 @@
2025 BinaryPackagePublishingHistory,
2026 SourcePackagePublishingHistory,
2027 )
2028-from lp.soyuz.scripts.ftpmaster import (
2029- PackageRemover,
2030- SoyuzScriptError,
2031- )
2032+from lp.soyuz.scripts.ftpmasterbase import SoyuzScriptError
2033+from lp.soyuz.scripts.packageremover import PackageRemover
2034 from lp.soyuz.tests.test_publishing import SoyuzTestPublisher
2035 from lp.testing.layers import LaunchpadZopelessLayer
2036
2037@@ -393,7 +391,7 @@
2038 the same result than not passing any component filter, because
2039 all test publications are in main component.
2040 """
2041- source = self.test_publisher.getPubSource(sourcename='foo')
2042+ self.test_publisher.getPubSource(sourcename='foo')
2043
2044 self.layer.commit()
2045
2046@@ -416,7 +414,7 @@
2047 `SoyuzScriptError` because the selected publications are in main
2048 component.
2049 """
2050- source = self.test_publisher.getPubSource(sourcename='foo')
2051+ self.test_publisher.getPubSource(sourcename='foo')
2052
2053 remover = self.getRemover(component='multiverse')
2054 self.assertRaises(SoyuzScriptError, remover.mainTask)
2055
2056=== modified file 'scripts/ftpmaster-tools/archive-override-check.py'
2057--- scripts/ftpmaster-tools/archive-override-check.py 2012-01-01 03:13:08 +0000
2058+++ scripts/ftpmaster-tools/archive-override-check.py 2012-01-19 17:27:32 +0000
2059@@ -1,6 +1,6 @@
2060 #!/usr/bin/python -S
2061 #
2062-# Copyright 2009 Canonical Ltd. This software is licensed under the
2063+# Copyright 2009-2012 Canonical Ltd. This software is licensed under the
2064 # GNU Affero General Public License version 3 (see the file LICENSE).
2065
2066 """Archive Override Check
2067@@ -20,7 +20,7 @@
2068 from lp.services.config import config
2069 from lp.services.scripts.base import LaunchpadScript
2070 from lp.soyuz.enums import PackagePublishingStatus
2071-from lp.soyuz.scripts.ftpmaster import PubSourceChecker
2072+from lp.soyuz.scripts.pubsourcechecker import PubSourceChecker
2073
2074
2075 class ArchiveOverrideCheckScript(LaunchpadScript):
2076
2077=== modified file 'scripts/ftpmaster-tools/lp-query-distro.py'
2078--- scripts/ftpmaster-tools/lp-query-distro.py 2012-01-06 11:08:30 +0000
2079+++ scripts/ftpmaster-tools/lp-query-distro.py 2012-01-19 17:27:32 +0000
2080@@ -1,6 +1,6 @@
2081 #!/usr/bin/python -S
2082 #
2083-# Copyright 2009 Canonical Ltd. This software is licensed under the
2084+# Copyright 2009-2012 Canonical Ltd. This software is licensed under the
2085 # GNU Affero General Public License version 3 (see the file LICENSE).
2086
2087 # pylint: disable-msg=W0403
2088@@ -27,7 +27,7 @@
2089
2090 import _pythonpath
2091
2092-from lp.soyuz.scripts.ftpmaster import LpQueryDistro
2093+from lp.soyuz.scripts.querydistro import LpQueryDistro
2094
2095
2096 if __name__ == '__main__':
2097
2098=== modified file 'scripts/ftpmaster-tools/lp-remove-package.py'
2099--- scripts/ftpmaster-tools/lp-remove-package.py 2011-12-29 05:29:36 +0000
2100+++ scripts/ftpmaster-tools/lp-remove-package.py 2012-01-19 17:27:32 +0000
2101@@ -1,6 +1,6 @@
2102 #!/usr/bin/python -S
2103 #
2104-# Copyright 2009 Canonical Ltd. This software is licensed under the
2105+# Copyright 2009-2012 Canonical Ltd. This software is licensed under the
2106 # GNU Affero General Public License version 3 (see the file LICENSE).
2107
2108 # pylint: disable-msg=W0403
2109@@ -11,11 +11,10 @@
2110 import _pythonpath
2111
2112 from lp.services.config import config
2113-from lp.soyuz.scripts.ftpmaster import PackageRemover
2114+from lp.soyuz.scripts.packageremover import PackageRemover
2115
2116
2117 if __name__ == '__main__':
2118 script = PackageRemover(
2119 'lp-remove-package', dbuser=config.archivepublisher.dbuser)
2120 script.lock_and_run()
2121-
2122
2123=== modified file 'scripts/ftpmaster-tools/manage-chroot.py'
2124--- scripts/ftpmaster-tools/manage-chroot.py 2012-01-06 11:08:30 +0000
2125+++ scripts/ftpmaster-tools/manage-chroot.py 2012-01-19 17:27:32 +0000
2126@@ -1,6 +1,6 @@
2127 #!/usr/bin/python -S
2128 #
2129-# Copyright 2009 Canonical Ltd. This software is licensed under the
2130+# Copyright 2009-2012 Canonical Ltd. This software is licensed under the
2131 # GNU Affero General Public License version 3 (see the file LICENSE).
2132
2133 # Stop lint warning about relative import:
2134@@ -10,10 +10,9 @@
2135
2136 import _pythonpath
2137
2138-from lp.soyuz.scripts.ftpmaster import ManageChrootScript
2139+from lp.soyuz.scripts.chrootmanager import ManageChrootScript
2140
2141
2142 if __name__ == '__main__':
2143 script = ManageChrootScript('manage-chroot', dbuser="fiera")
2144 script.lock_and_run()
2145-
2146
2147=== modified file 'scripts/ftpmaster-tools/obsolete-distroseries.py'
2148--- scripts/ftpmaster-tools/obsolete-distroseries.py 2011-12-29 05:29:36 +0000
2149+++ scripts/ftpmaster-tools/obsolete-distroseries.py 2012-01-19 17:27:32 +0000
2150@@ -1,6 +1,6 @@
2151 #!/usr/bin/python -S
2152 #
2153-# Copyright 2009 Canonical Ltd. This software is licensed under the
2154+# Copyright 2009-2012 Canonical Ltd. This software is licensed under the
2155 # GNU Affero General Public License version 3 (see the file LICENSE).
2156
2157 # Stop lint warning about relative import:
2158@@ -15,11 +15,10 @@
2159 import _pythonpath
2160
2161 from lp.services.config import config
2162-from lp.soyuz.scripts.ftpmaster import ObsoleteDistroseries
2163+from lp.soyuz.scripts.obsolete_distroseries import ObsoleteDistroseries
2164
2165
2166 if __name__ == '__main__':
2167 script = ObsoleteDistroseries(
2168 'obsolete-distroseries', dbuser=config.archivepublisher.dbuser)
2169 script.lock_and_run()
2170-