Merge lp:~mbp/launchpad/800295-delete-buildd into lp:launchpad

Proposed by Martin Pool
Status: Merged
Approved by: Martin Pool
Approved revision: no longer in the source branch.
Merged at revision: 14311
Proposed branch: lp:~mbp/launchpad/800295-delete-buildd
Merge into: lp:launchpad
Diff against target: 9037 lines (+19/-8598)
63 files modified
.bzrignore (+0/-8)
buildout.cfg (+4/-2)
lib/canonical/buildd/Makefile (+0/-19)
lib/canonical/buildd/README (+0/-47)
lib/canonical/buildd/__init__.py (+0/-5)
lib/canonical/buildd/binarypackage.py (+0/-133)
lib/canonical/buildd/buildd-config.py (+0/-54)
lib/canonical/buildd/buildd-slave-example.conf (+0/-25)
lib/canonical/buildd/buildd-slave.tac (+0/-55)
lib/canonical/buildd/buildrecipe (+0/-220)
lib/canonical/buildd/check-implicit-pointer-functions (+0/-118)
lib/canonical/buildd/debian.py (+0/-239)
lib/canonical/buildd/debian/changelog (+0/-698)
lib/canonical/buildd/debian/compat (+0/-1)
lib/canonical/buildd/debian/control (+0/-17)
lib/canonical/buildd/debian/launchpad-buildd.cron.daily (+0/-10)
lib/canonical/buildd/debian/launchpad-buildd.examples (+0/-1)
lib/canonical/buildd/debian/launchpad-buildd.init (+0/-98)
lib/canonical/buildd/debian/postinst (+0/-101)
lib/canonical/buildd/debian/rules (+0/-96)
lib/canonical/buildd/debian/source/format (+0/-1)
lib/canonical/buildd/debian/upgrade-config (+0/-122)
lib/canonical/buildd/generate-translation-templates (+0/-66)
lib/canonical/buildd/mount-chroot (+0/-25)
lib/canonical/buildd/override-sources-list (+0/-31)
lib/canonical/buildd/pottery/generate_translation_templates.py (+0/-114)
lib/canonical/buildd/pottery/intltool.py (+0/-342)
lib/canonical/buildd/remove-build (+0/-25)
lib/canonical/buildd/sbuild (+0/-3658)
lib/canonical/buildd/sbuild-package (+0/-92)
lib/canonical/buildd/sbuild.conf (+0/-155)
lib/canonical/buildd/sbuildrc (+0/-32)
lib/canonical/buildd/scan-for-processes (+0/-43)
lib/canonical/buildd/slave.py (+0/-686)
lib/canonical/buildd/sourcepackagerecipe.py (+0/-144)
lib/canonical/buildd/template-buildd-slave.conf (+0/-32)
lib/canonical/buildd/test_buildd_generatetranslationtemplates (+0/-33)
lib/canonical/buildd/test_buildd_recipe (+0/-48)
lib/canonical/buildd/tests/__init__.py (+0/-4)
lib/canonical/buildd/tests/buildd-slave-test.conf (+0/-27)
lib/canonical/buildd/tests/buildlog (+0/-23)
lib/canonical/buildd/tests/buildlog.long (+0/-82)
lib/canonical/buildd/tests/harness.py (+0/-139)
lib/canonical/buildd/tests/test_1.diff (+0/-17)
lib/canonical/buildd/tests/test_2.diff (+0/-32)
lib/canonical/buildd/tests/test_buildd_slave.py (+0/-198)
lib/canonical/buildd/tests/test_check_implicit_pointer_functions.py (+0/-43)
lib/canonical/buildd/tests/test_harness.py (+0/-10)
lib/canonical/buildd/tests/test_translationtemplatesbuildmanager.py (+0/-173)
lib/canonical/buildd/translationtemplates.py (+0/-99)
lib/canonical/buildd/umount-chroot (+0/-40)
lib/canonical/buildd/unpack-chroot (+0/-52)
lib/canonical/buildd/update-debian-chroot (+0/-45)
lib/canonical/launchpad/daemons/readyservice.py (+0/-6)
lib/lp/buildmaster/model/builder.py (+3/-3)
lib/lp/buildmaster/tests/mock_slaves.py (+1/-1)
lib/lp/buildmaster/tests/test_builder.py (+2/-1)
lib/lp/buildmaster/tests/test_manager.py (+2/-1)
lib/lp/testing/factory.py (+0/-1)
lib/lp/translations/tests/test_generate_translation_templates.py (+5/-2)
lib/lp/translations/tests/test_pottery_detect_intltool.py (+1/-1)
scripts/rosetta/pottery-generate-intltool.py (+1/-1)
utilities/snakefood/lp-sfood-packages (+0/-1)
To merge this branch: bzr merge lp:~mbp/launchpad/800295-delete-buildd
Reviewer Review Type Date Requested Status
Launchpad code reviewers Pending
Review via email: mp+81815@code.launchpad.net

Commit message

[r=mbp][bug=800295] delete lib/canonical/buildd and use lp:launchpad-buildd instead

Description of the change

Further to bug 800295, this

 * deletes the lib/canonical/buildd directory
 * changes tests that need the buildd packages to load them from the 'lpbuildd' package instead
 * inserts onto the pythonpath the /usr/share/launchpad-buildd directory into which the launchpad-buildd deb package installs the lpbuildd python package

This cannot be landed until the lpbuildd package is added as a deb dependency of Launchpad, which requires some more work to separate 'acts as a buildd' from 'provides the lpbuildd python package', and preferably also a deployment of the new launchpad-buildd package to prove it works ok.

Eventually we might want to remove the test-time dependencies altogether in favour of testing against a fake <https://dev.launchpad.net/ArchitectureGuide/ServicesRequirements#Test_fake> and having separate integration qa. But I think this is a safer step forward to make the split while keeping the same tests in place.

To post a comment you must log in.
Revision history for this message
Martin Pool (mbp) wrote :

after discussion with elmo and others, i'm going to split the binaries to a separate package and make it disabled by default

Revision history for this message
Martin Pool (mbp) wrote :

lp:launchpad-buildd now splits out a separate python-lpbuildd package, and the main launchpad-buildd package will not run unless it is specifically enabled. now i'm going to proceed with adding it as a developer dependency.

Revision history for this message
Martin Pool (mbp) wrote :

it turned out that it is not easily possible for lamont to change the setting, so in http://bazaar.launchpad.net/~launchpad-dev/launchpad-buildd/trunk/revision/17 i've made it just autostart based on the hostname, which should be tolerably safe for other people without disrupting the buildds

i've added python-lpbuildd to launchpad-developer-dependencies in https://code.launchpad.net/~launchpad-committers/meta-lp-deps/trunk

i apparently need to manually update the ec2 image before it can be relied upon

i've created a recipe building from lp:launchpad-buildd into the launchpad ppa

Revision history for this message
Martin Pool (mbp) wrote :

i ran this through ec2 with https://code.launchpad.net/~mbp/launchpad/ec2-update/+merge/81946 to let it be tested, and it did not find the right buildd package. i'm not sure why; probably i've done something wrong in buildout.

Revision history for this message
Martin Pool (mbp) wrote :

With an updated ec2test image and with ec2test set to update packages before running, this did work.

based on the general approach having been thoroughly discussed and the actual patch being big and mechanical, i'm going to treat this as approved.

Revision history for this message
Jelmer Vernooij (jelmer) wrote :

W00t!

Revision history for this message
Martin Pool (mbp) wrote :

this was reverted because it failed on buildbot

 https://code.launchpad.net/~mbp/launchpad/800295-protocol/+merge/82629 removes one more bit of coupling

python-lpbuildd is installed on buildbot

i'm going to re-send this.

Revision history for this message
Martin Pool (mbp) wrote :

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file '.bzrignore'
2--- .bzrignore 2011-08-08 19:22:19 +0000
3+++ .bzrignore 2011-11-11 07:09:27 +0000
4@@ -63,14 +63,6 @@
5 twistd.pid
6 lib/canonical/launchpad/apidoc
7 *.prof
8-lib/canonical/launchpad-buildd_*.dsc
9-lib/canonical/launchpad-buildd_*.tar.gz
10-lib/canonical/launchpad-buildd_*_all.deb
11-lib/canonical/launchpad-buildd_*.changes
12-lib/canonical/launchpad-buildd_*_source.build
13-lib/canonical/launchpad-buildd_*_source.changes
14-lib/canonical/buildd/debian/*
15-lib/canonical/buildd/launchpad-files/*
16 .project
17 .pydevproject
18 librarian.log
19
20=== modified file 'buildout.cfg'
21--- buildout.cfg 2011-10-04 14:41:10 +0000
22+++ buildout.cfg 2011-11-11 07:09:27 +0000
23@@ -1,4 +1,4 @@
24-# Copyright 2009 Canonical Ltd. This software is licensed under the
25+# Copyright 2009-2011 Canonical Ltd. This software is licensed under the
26 # GNU Affero General Public License version 3 (see the file LICENSE).
27
28 [buildout]
29@@ -57,7 +57,9 @@
30 zc.zservertracelog
31 # XXX gary 2009-5-12 bug 375751:
32 # Make mailman built and installed in a more normal way.
33-extra-paths = ${buildout:directory}/lib/mailman
34+extra-paths =
35+ ${buildout:directory}/lib/mailman
36+ /usr/lib/launchpad-buildd
37 include-site-packages = true
38 allowed-eggs-from-site-packages =
39 interpreter = py
40
41=== removed directory 'lib/canonical/buildd'
42=== removed file 'lib/canonical/buildd/Makefile'
43--- lib/canonical/buildd/Makefile 2009-06-30 21:06:27 +0000
44+++ lib/canonical/buildd/Makefile 1970-01-01 00:00:00 +0000
45@@ -1,19 +0,0 @@
46-# Copyright 2009 Canonical Ltd. This software is licensed under the
47-# GNU Affero General Public License version 3 (see the file LICENSE).
48-
49-all: deb
50-
51-src: clean
52- dpkg-buildpackage -rfakeroot -uc -us -S
53-
54-deb: clean
55- dpkg-buildpackage -rfakeroot -uc -us
56-
57-clean:
58- fakeroot debian/rules clean
59- rm -f ../launchpad-buildd*tar.gz
60- rm -f ../launchpad-buildd*dsc
61- rm -f ../launchpad-buildd*deb
62- rm -f ../launchpad-buildd*changes
63-
64-.PHONY: all clean deb
65
66=== removed file 'lib/canonical/buildd/README'
67--- lib/canonical/buildd/README 2010-02-24 18:09:59 +0000
68+++ lib/canonical/buildd/README 1970-01-01 00:00:00 +0000
69@@ -1,47 +0,0 @@
70-In order to build the package you need dpkg-dev and fakeroot
71-
72-To build the package do:
73-
74-cd ..../lib/canonical/buildd
75-debian/rules package
76-dpkg-buildpackage -rfakeroot -b
77-
78-It will "fail" because the package built in the "wrong" place. Don't
79-worry about that.
80-
81-To clean up, do:
82-
83-fakeroot debian/rules clean
84-rm launchpad-buildd*deb
85-rm ../launchpad-buildd*changes
86-
87------------------------------------------------------------------------
88-
89-How to use the chroot tool:
90-
91-buildd-slave-chroot-tool --intervene <chroot tarball>
92-
93-...will unpack the chroot tarball, mount up the proc etc filesystems
94-in it and chroot into it.
95-
96-When you leave the chroot, it will ask you if you want to repack the
97-tarball. If you answer yes then it will repack it, if you answer no,
98-then it will merely clean up and exit.
99-
100-If you wish to build a new chroot then first copy the example.chroot
101-file from /usr/share/doc/launchpad-buildd/ and edit it to your
102-requirements.
103-
104-Next run buildd-slave-chroot-tool --generate config.filename
105-
106-You must have sudo capability and ideally you would run this as the
107-buildd user on a machine with launchpad-buildd installed.
108-
109-Once you have completed the chroot building, you will be left with a
110-file named chroot-<distro>-<distrorelease>-<arch>.tar.bz2 which should
111-be uploaded to the librarian from a machine with librarian access and
112-then marked as the chroot for the given pocket.
113-
114-You should retain the configuration file so that we know how the
115-chroot was built in case we need to rebuild it at a later date.
116-
117
118=== removed file 'lib/canonical/buildd/__init__.py'
119--- lib/canonical/buildd/__init__.py 2010-12-21 17:52:32 +0000
120+++ lib/canonical/buildd/__init__.py 1970-01-01 00:00:00 +0000
121@@ -1,5 +0,0 @@
122-# Copyright 2009 Canonical Ltd. This software is licensed under the
123-# GNU Affero General Public License version 3 (see the file LICENSE).
124-
125-from canonical.buildd.slave import XMLRPCBuildDSlave
126-from canonical.buildd.debian import DebianBuildManager
127
128=== removed file 'lib/canonical/buildd/binarypackage.py'
129--- lib/canonical/buildd/binarypackage.py 2010-11-23 13:21:58 +0000
130+++ lib/canonical/buildd/binarypackage.py 1970-01-01 00:00:00 +0000
131@@ -1,133 +0,0 @@
132-# Copyright 2009, 2010 Canonical Ltd. This software is licensed under the
133-# GNU Affero General Public License version 3 (see the file LICENSE).
134-
135-
136-import re
137-
138-from canonical.buildd.debian import DebianBuildManager, DebianBuildState
139-
140-
141-class SBuildExitCodes:
142- """SBUILD process result codes."""
143- OK = 0
144- DEPFAIL = 1
145- GIVENBACK = 2
146- PACKAGEFAIL = 3
147- BUILDERFAIL = 4
148-
149-
150-class BuildLogRegexes:
151- """Build log regexes for performing actions based on regexes, and extracting dependencies for auto dep-waits"""
152- GIVENBACK = [
153- ("^E: There are problems and -y was used without --force-yes"),
154- ]
155- DEPFAIL = [
156- ("(?P<pk>[\-+.\w]+)\(inst [^ ]+ ! >> wanted (?P<v>[\-.+\w:~]+)\)","\g<pk> (>> \g<v>)"),
157- ("(?P<pk>[\-+.\w]+)\(inst [^ ]+ ! >?= wanted (?P<v>[\-.+\w:~]+)\)","\g<pk> (>= \g<v>)"),
158- ("(?s)^E: Couldn't find package (?P<pk>[\-+.\w]+)(?!.*^E: Couldn't find package)","\g<pk>"),
159- ("(?s)^E: Package '?(?P<pk>[\-+.\w]+)'? has no installation candidate(?!.*^E: Package)","\g<pk>"),
160- ("(?s)^E: Unable to locate package (?P<pk>[\-+.\w]+)(?!.*^E: Unable to locate package)", "\g<pk>"),
161- ]
162-
163-
164-class BinaryPackageBuildState(DebianBuildState):
165- SBUILD = "SBUILD"
166-
167-
168-class BinaryPackageBuildManager(DebianBuildManager):
169- """Handle buildd building for a debian style binary package build"""
170-
171- initial_build_state = BinaryPackageBuildState.SBUILD
172-
173- def __init__(self, slave, buildid):
174- DebianBuildManager.__init__(self, slave, buildid)
175- self._sbuildpath = slave._config.get("binarypackagemanager", "sbuildpath")
176- self._sbuildargs = slave._config.get("binarypackagemanager",
177- "sbuildargs").split(" ")
178-
179- def initiate(self, files, chroot, extra_args):
180- """Initiate a build with a given set of files and chroot."""
181-
182- self._dscfile = None
183- for f in files:
184- if f.endswith(".dsc"):
185- self._dscfile = f
186- if self._dscfile is None:
187- raise ValueError, files
188-
189- self.archive_purpose = extra_args.get('archive_purpose')
190- self.suite = extra_args.get('suite')
191- self.component = extra_args['ogrecomponent']
192- self.arch_indep = extra_args.get('arch_indep', False)
193- self.build_debug_symbols = extra_args.get('build_debug_symbols', False)
194-
195- super(BinaryPackageBuildManager, self).initiate(
196- files, chroot, extra_args)
197-
198- def doRunBuild(self):
199- """Run the sbuild process to build the package."""
200- args = ["sbuild-package", self._buildid, self.arch_tag]
201- if self.suite:
202- args.extend([self.suite])
203- args.extend(self._sbuildargs)
204- args.extend(["--dist=" + self.suite])
205- else:
206- args.extend(['autobuild'])
207- args.extend(self._sbuildargs)
208- args.extend(["--dist=autobuild"])
209- if self.arch_indep:
210- args.extend(["-A"])
211- if self.archive_purpose:
212- args.extend(["--purpose=" + self.archive_purpose])
213- if self.build_debug_symbols:
214- args.extend(["--build-debug-symbols"])
215- args.extend(["--architecture=" + self.arch_tag])
216- args.extend(["--comp=" + self.component])
217- args.extend([self._dscfile])
218- self.runSubProcess( self._sbuildpath, args )
219-
220- def iterate_SBUILD(self, success):
221- """Finished the sbuild run."""
222- tmpLog = self.getTmpLogContents()
223- if success != SBuildExitCodes.OK:
224- if (success == SBuildExitCodes.DEPFAIL or
225- success == SBuildExitCodes.PACKAGEFAIL):
226- for rx in BuildLogRegexes.GIVENBACK:
227- mo = re.search(rx, tmpLog, re.M)
228- if mo:
229- success = SBuildExitCodes.GIVENBACK
230-
231- if success == SBuildExitCodes.DEPFAIL:
232- for rx, dep in BuildLogRegexes.DEPFAIL:
233- mo = re.search(rx, tmpLog, re.M)
234- if mo:
235- if not self.alreadyfailed:
236- print("Returning build status: DEPFAIL")
237- print("Dependencies: " + mo.expand(dep))
238- self._slave.depFail(mo.expand(dep))
239- success = SBuildExitCodes.DEPFAIL
240- break
241- else:
242- success = SBuildExitCodes.PACKAGEFAIL
243-
244- if success == SBuildExitCodes.GIVENBACK:
245- if not self.alreadyfailed:
246- print("Returning build status: GIVENBACK")
247- self._slave.giveBack()
248- elif success == SBuildExitCodes.PACKAGEFAIL:
249- if not self.alreadyfailed:
250- print("Returning build status: PACKAGEFAIL")
251- self._slave.buildFail()
252- elif success >= SBuildExitCodes.BUILDERFAIL:
253- # anything else is assumed to be a buildd failure
254- if not self.alreadyfailed:
255- print("Returning build status: BUILDERFAIL")
256- self._slave.builderFail()
257- self.alreadyfailed = True
258- self._state = DebianBuildState.REAP
259- self.doReapProcesses()
260- else:
261- print("Returning build status: OK")
262- self.gatherResults()
263- self._state = DebianBuildState.REAP
264- self.doReapProcesses()
265
266=== removed file 'lib/canonical/buildd/buildd-config.py'
267--- lib/canonical/buildd/buildd-config.py 2009-12-15 18:42:11 +0000
268+++ lib/canonical/buildd/buildd-config.py 1970-01-01 00:00:00 +0000
269@@ -1,54 +0,0 @@
270-#!/usr/bin/python
271-#
272-# Copyright 2009 Canonical Ltd. This software is licensed under the
273-# GNU Affero General Public License version 3 (see the file LICENSE).
274-
275-import os
276-
277-archtag = os.popen("dpkg --print-architecture").read().strip()
278-
279-from optparse import OptionParser
280-
281-parser = OptionParser()
282-parser.add_option("-n", "--name", dest="NAME",
283- help="the name for this buildd",
284- metavar="NAME",
285- default="default")
286-
287-parser.add_option("-H", "--host", dest="BINDHOST",
288- help="the IP/host this buildd binds to",
289- metavar="HOSTNAME",
290- default="localhost")
291-
292-parser.add_option("-p", "--port", dest="BINDPORT",
293- help="the port this buildd binds to",
294- metavar="PORT",
295- default="8221")
296-
297-parser.add_option("-a", "--arch", dest="ARCHTAG",
298- help="the arch tag this buildd claims",
299- metavar="ARCHTAG",
300- default=archtag)
301-
302-parser.add_option("-t", "--template", dest="TEMPLATE",
303- help="the template file to use",
304- metavar="FILE",
305- default="/usr/share/launchpad-buildd/template-buildd-slave.conf")
306-
307-(options, args) = parser.parse_args()
308-
309-template = open(options.TEMPLATE, "r").read()
310-
311-replacements = {
312- "@NAME@": options.NAME,
313- "@BINDHOST@": options.BINDHOST,
314- "@ARCHTAG@": options.ARCHTAG,
315- "@BINDPORT@": options.BINDPORT,
316- }
317-
318-for replacement_key in replacements:
319- template = template.replace(replacement_key,
320- replacements[replacement_key])
321-
322-print template
323-
324
325=== removed file 'lib/canonical/buildd/buildd-slave-example.conf'
326--- lib/canonical/buildd/buildd-slave-example.conf 2010-07-22 16:03:08 +0000
327+++ lib/canonical/buildd/buildd-slave-example.conf 1970-01-01 00:00:00 +0000
328@@ -1,25 +0,0 @@
329-# Copyright 2009 Canonical Ltd. This software is licensed under the
330-# GNU Affero General Public License version 3 (see the file LICENSE).
331-
332-# Example buildd slave configuration
333-# This should be refactored into a nice central configuration format when
334-# such a beast becomes available
335-
336-[slave]
337-architecturetag = i386
338-filecache = /home/buildd/filecache
339-bindhost = localhost
340-bindport = 8221
341-
342-[allmanagers]
343-unpackpath = /home/buildd/slavebin/unpack-chroot
344-cleanpath = /home/buildd/slavebin/remove-build
345-mountpath = /home/buildd/slavebin/mount-chroot
346-umountpath = /home/buildd/slavebin/umount-chroot
347-
348-[debianmanager]
349-sbuildpath = /home/buildd/slavebin/sbuild-package
350-sbuildargs = -dautobuild --nolog --batch -A
351-updatepath = /home/buildd/slavebin/update-debian-chroot
352-processscanpath = /home/buildd/slavebin/scan-for-processes
353-ogrepath = /home/buildd/slavebin/apply-ogre-model
354
355=== removed file 'lib/canonical/buildd/buildd-slave.tac'
356--- lib/canonical/buildd/buildd-slave.tac 2011-11-04 00:46:03 +0000
357+++ lib/canonical/buildd/buildd-slave.tac 1970-01-01 00:00:00 +0000
358@@ -1,55 +0,0 @@
359-# Copyright 2009-2011 Canonical Ltd. This software is licensed under the
360-# GNU Affero General Public License version 3 (see the file LICENSE).
361-
362-# CAUTION: The only modules in the Launchpad tree that this is permitted to
363-# depend on are canonical.buildd, since buildds are deployed by copying that
364-# directory only. (See also bug=800295.)
365-
366-# Buildd Slave implementation
367-# XXX: dsilvers: 2005/01/21: Currently everything logged in the slave gets
368-# passed through to the twistd log too. this could get dangerous/big
369-
370-from twisted.application import service, strports
371-from canonical.buildd import XMLRPCBuildDSlave
372-from canonical.buildd.binarypackage import BinaryPackageBuildManager
373-from canonical.buildd.sourcepackagerecipe import (
374- SourcePackageRecipeBuildManager)
375-from canonical.buildd.translationtemplates import (
376- TranslationTemplatesBuildManager)
377-
378-from twisted.web import server, resource, static
379-from ConfigParser import SafeConfigParser
380-
381-import os
382-
383-conffile = os.environ.get('BUILDD_SLAVE_CONFIG', 'buildd-slave-example.conf')
384-
385-conf = SafeConfigParser()
386-conf.read(conffile)
387-slave = XMLRPCBuildDSlave(conf)
388-
389-# 'debian' is the old name. It remains here for compatibility.
390-slave.registerBuilder(BinaryPackageBuildManager, "debian")
391-slave.registerBuilder(BinaryPackageBuildManager, "binarypackage")
392-slave.registerBuilder(SourcePackageRecipeBuildManager, "sourcepackagerecipe")
393-slave.registerBuilder(
394- TranslationTemplatesBuildManager, 'translation-templates')
395-
396-application = service.Application('BuildDSlave')
397-builddslaveService = service.IServiceCollection(application)
398-
399-root = resource.Resource()
400-root.putChild('rpc', slave)
401-root.putChild('filecache', static.File(conf.get('slave', 'filecache')))
402-slavesite = server.Site(root)
403-
404-strports.service(slave.slave._config.get("slave","bindport"),
405- slavesite).setServiceParent(builddslaveService)
406-
407-# You can interact with a running slave like this:
408-# (assuming the slave is on localhost:8221)
409-#
410-# python
411-# import xmlrpclib
412-# s = xmlrpclib.ServerProxy("http://localhost:8221/rpc")
413-# s.echo("Hello World")
414
415=== removed file 'lib/canonical/buildd/buildrecipe'
416--- lib/canonical/buildd/buildrecipe 2011-11-07 05:41:39 +0000
417+++ lib/canonical/buildd/buildrecipe 1970-01-01 00:00:00 +0000
418@@ -1,220 +0,0 @@
419-#! /usr/bin/env python -u
420-# Copyright 2010, 2011 Canonical Ltd. This software is licensed under the
421-# GNU Affero General Public License version 3 (see the file LICENSE).
422-
423-"""A script that builds a package from a recipe and a chroot."""
424-
425-__metaclass__ = type
426-
427-
428-import os
429-import os.path
430-import pwd
431-from resource import RLIMIT_AS, setrlimit
432-import socket
433-from subprocess import (
434- Popen,
435- call,
436- )
437-import sys
438-
439-
440-RETCODE_SUCCESS = 0
441-RETCODE_FAILURE_INSTALL = 200
442-RETCODE_FAILURE_BUILD_TREE = 201
443-RETCODE_FAILURE_INSTALL_BUILD_DEPS = 202
444-RETCODE_FAILURE_BUILD_SOURCE_PACKAGE = 203
445-
446-
447-class NotVirtualized(Exception):
448- """Exception raised when not running in a virtualized environment."""
449-
450- def __init__(self):
451- Exception.__init__(self, 'Not running under Xen.')
452-
453-
454-def call_report_rusage(args):
455- """Run a subprocess.
456-
457- Report that it was run, and the resources used, and complain if it fails.
458-
459- :return: The process wait status.
460- """
461- print 'RUN %r' % args
462- proc = Popen(args)
463- pid, status, rusage = os.wait4(proc.pid, 0)
464- print(rusage)
465- return status
466-
467-
468-class RecipeBuilder:
469- """Builds a package from a recipe."""
470-
471- def __init__(self, build_id, author_name, author_email,
472- suite, distroseries_name, component, archive_purpose):
473- """Constructor.
474-
475- :param build_id: The id of the build (a str).
476- :param author_name: The name of the author (a str).
477- :param author_email: The email address of the author (a str).
478- :param suite: The suite the package should be built for (a str).
479- """
480- self.build_id = build_id
481- self.author_name = author_name.decode('utf-8')
482- self.author_email = author_email
483- self.archive_purpose = archive_purpose
484- self.component = component
485- self.distroseries_name = distroseries_name
486- self.suite = suite
487- self.base_branch = None
488- self.chroot_path = get_build_path(build_id, 'chroot-autobuild')
489- self.work_dir_relative = os.environ['HOME'] + '/work'
490- self.work_dir = os.path.join(self.chroot_path,
491- self.work_dir_relative[1:])
492- self.tree_path = os.path.join(self.work_dir, 'tree')
493- self.username = pwd.getpwuid(os.getuid())[0]
494-
495- def install(self):
496- """Install all the requirements for building recipes.
497-
498- :return: A retcode from apt.
499- """
500- # XXX: AaronBentley 2010-07-07 bug=602463: pbuilder uses aptitude but
501- # does not depend on it.
502- return self.chroot([
503- 'apt-get', 'install', '-y', 'pbuilder', 'aptitude'])
504-
505- def buildTree(self):
506- """Build the recipe into a source tree.
507-
508- As a side-effect, sets self.source_dir_relative.
509- :return: a retcode from `bzr dailydeb`.
510- """
511- try:
512- ensure_virtualized()
513- except NotVirtualized, e:
514- sys.stderr.write('Aborting on failed virtualization check:\n')
515- sys.stderr.write(str(e))
516- return 1
517- assert not os.path.exists(self.tree_path)
518- recipe_path = os.path.join(self.work_dir, 'recipe')
519- manifest_path = os.path.join(self.tree_path, 'manifest')
520- recipe_file = open(recipe_path, 'rb')
521- try:
522- recipe = recipe_file.read()
523- finally:
524- recipe_file.close()
525- # As of bzr 2.2, a defined identity is needed. In this case, we're
526- # using buildd@<hostname>.
527- hostname = socket.gethostname()
528- bzr_email = 'buildd@%s' % hostname
529-
530- print 'Bazaar versions:'
531- check_call(['bzr', 'version'])
532- check_call(['bzr', 'plugins'])
533-
534- print 'Building recipe:'
535- print recipe
536- sys.stdout.flush()
537- env = {
538- 'DEBEMAIL': self.author_email,
539- 'DEBFULLNAME': self.author_name.encode('utf-8'),
540- 'BZR_EMAIL': bzr_email}
541- retcode = call_report_rusage([
542- 'bzr', 'dailydeb', '--safe', '--no-build', recipe_path,
543- self.tree_path, '--manifest', manifest_path,
544- '--allow-fallback-to-native', '--append-version',
545- '~%s1' % self.distroseries_name], env=env)
546- if retcode != 0:
547- return retcode
548- (source,) = [name for name in os.listdir(self.tree_path)
549- if name != 'manifest']
550- self.source_dir_relative = os.path.join(
551- self.work_dir_relative, 'tree', source)
552- return retcode
553-
554- def getPackageName(self):
555- source_dir = os.path.join(
556- self.chroot_path, self.source_dir_relative.lstrip('/'))
557- changelog = os.path.join(source_dir, 'debian/changelog')
558- return open(changelog, 'r').readline().split(' ')[0]
559-
560- def installBuildDeps(self):
561- """Install the build-depends of the source tree."""
562- package = self.getPackageName()
563- currently_building_path = os.path.join(
564- self.chroot_path, 'CurrentlyBuilding')
565- currently_building_contents = (
566- 'Package: %s\n'
567- 'Suite: %s\n'
568- 'Component: %s\n'
569- 'Purpose: %s\n'
570- 'Build-Debug-Symbols: no\n' %
571- (package, self.suite, self.component, self.archive_purpose))
572- currently_building = open(currently_building_path, 'w')
573- currently_building.write(currently_building_contents)
574- currently_building.close()
575- return self.chroot(['sh', '-c', 'cd %s &&'
576- '/usr/lib/pbuilder/pbuilder-satisfydepends'
577- % self.source_dir_relative])
578-
579- def chroot(self, args, echo=False):
580- """Run a command in the chroot.
581-
582- :param args: the command and arguments to run.
583- :return: the status code.
584- """
585- if echo:
586- print "Running in chroot: %s" % ' '.join(
587- "'%s'" % arg for arg in args)
588- sys.stdout.flush()
589- return call([
590- '/usr/bin/sudo', '/usr/sbin/chroot', self.chroot_path] + args)
591-
592- def buildSourcePackage(self):
593- """Build the source package.
594-
595- :return: a retcode from dpkg-buildpackage.
596- """
597- retcode = self.chroot([
598- 'su', '-c', 'cd %s && /usr/bin/dpkg-buildpackage -i -I -us -uc -S'
599- % self.source_dir_relative, self.username])
600- for filename in os.listdir(self.tree_path):
601- path = os.path.join(self.tree_path, filename)
602- if os.path.isfile(path):
603- os.rename(path, get_build_path(self.build_id, filename))
604- return retcode
605-
606-
607-def get_build_path(build_id, *extra):
608- """Generate a path within the build directory.
609-
610- :param build_id: the build id to use.
611- :param extra: the extra path segments within the build directory.
612- :return: the generated path.
613- """
614- return os.path.join(
615- os.environ["HOME"], "build-" + build_id, *extra)
616-
617-
618-def ensure_virtualized():
619- """Raise an exception if not running in a virtualized environment.
620-
621- Raises if not running under Xen.
622- """
623- if not os.path.isdir('/proc/xen') or os.path.exists('/proc/xen/xsd_kva'):
624- raise NotVirtualized()
625-
626-
627-if __name__ == '__main__':
628- setrlimit(RLIMIT_AS, (1000000000, -1))
629- builder = RecipeBuilder(*sys.argv[1:])
630- if builder.buildTree() != 0:
631- sys.exit(RETCODE_FAILURE_BUILD_TREE)
632- if builder.install() != 0:
633- sys.exit(RETCODE_FAILURE_INSTALL)
634- if builder.installBuildDeps() != 0:
635- sys.exit(RETCODE_FAILURE_INSTALL_BUILD_DEPS)
636- if builder.buildSourcePackage() != 0:
637- sys.exit(RETCODE_FAILURE_BUILD_SOURCE_PACKAGE)
638- sys.exit(RETCODE_SUCCESS)
639
640=== removed file 'lib/canonical/buildd/check-implicit-pointer-functions'
641--- lib/canonical/buildd/check-implicit-pointer-functions 2011-02-15 12:24:25 +0000
642+++ lib/canonical/buildd/check-implicit-pointer-functions 1970-01-01 00:00:00 +0000
643@@ -1,118 +0,0 @@
644-#!/usr/bin/env python
645-
646-#
647-# Copyright (c) 2004 Hewlett-Packard Development Company, L.P.
648-# David Mosberger <davidm@hpl.hp.com>
649-# Copyright 2010 Canonical Ltd.
650-#
651-# Permission is hereby granted, free of charge, to any person
652-# obtaining a copy of this software and associated documentation
653-# files (the "Software"), to deal in the Software without
654-# restriction, including without limitation the rights to use,
655-# copy, modify, merge, publish, distribute, sublicense, and/or sell
656-# copies of the Software, and to permit persons to whom the
657-# Software is furnished to do so, subject to the following
658-# conditions:
659-#
660-# The above copyright notice and this permission notice shall be
661-# included in all copies or substantial portions of the Software.
662-#
663-# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
664-# EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
665-# OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
666-# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
667-# HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
668-# WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
669-# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
670-# OTHER DEALINGS IN THE SOFTWARE.
671-
672-# Scan standard input for GCC warning messages that are likely to
673-# source of real 64-bit problems. In particular, see whether there
674-# are any implicitly declared functions whose return values are later
675-# interpreted as pointers. Those are almost guaranteed to cause
676-# crashes.
677-#
678-import re
679-import sys
680-
681-implicit_pattern = re.compile(
682- "([^:]*):(\d+):(\d+:)? warning: implicit declaration "
683- "of function [`']([^']*)'")
684-pointer_pattern = re.compile(
685- "([^:]*):(\d+):(\d+:)? warning: "
686- + "("
687- + "(assignment"
688- + "|initialization"
689- + "|return"
690- + "|passing arg \d+ of `[^']*'"
691- + "|passing arg \d+ of pointer to function"
692- + ") makes pointer from integer without a cast"
693- + "|"
694- + "cast to pointer from integer of different size)")
695-
696-def main():
697- last_implicit_filename = ""
698- last_implicit_linenum = -1
699- last_implicit_func = ""
700-
701- errlist = ""
702-
703- in_line = False
704- warn_only = False
705-
706- for arg in sys.argv[1:]:
707- if arg == '--inline':
708- in_line = True
709- elif arg == '--warnonly':
710- warn_only = True
711-
712- rv = 0
713- while True:
714- line = sys.stdin.readline()
715- if in_line:
716- sys.stdout.write(line)
717- sys.stdout.flush()
718- if line == '':
719- break
720- m = implicit_pattern.match(line)
721- if m:
722- last_implicit_filename = m.group(1)
723- last_implicit_linenum = int(m.group(2))
724- last_implicit_func = m.group(4)
725- else:
726- m = pointer_pattern.match(line)
727- if m:
728- pointer_filename = m.group(1)
729- pointer_linenum = int(m.group(2))
730- if (last_implicit_filename == pointer_filename
731- and last_implicit_linenum == pointer_linenum):
732- err = "Function `%s' implicitly converted to pointer at " \
733- "%s:%d" % (last_implicit_func, last_implicit_filename,
734- last_implicit_linenum)
735- errlist += err+"\n"
736- print err
737- if not warn_only:
738- rv = 3
739-
740- if len(errlist):
741- if in_line:
742- print errlist
743- print """
744-
745-Our automated build log filter detected the problem(s) above that will
746-likely cause your package to segfault on architectures where the size of
747-a pointer is greater than the size of an integer, such as ia64 and amd64.
748-
749-This is often due to a missing function prototype definition.
750-
751-Since use of implicitly converted pointers is always fatal to the application
752-on ia64, they are errors. Please correct them for your next upload.
753-
754-More information can be found at:
755-http://wiki.debian.org/ImplicitPointerConversions
756-
757- """
758- sys.exit(rv)
759-
760-if __name__ == '__main__':
761- main()
762
763=== removed symlink 'lib/canonical/buildd/check_implicit_pointer_functions.py'
764=== target was u'check-implicit-pointer-functions'
765=== removed directory 'lib/canonical/buildd/debian'
766=== removed file 'lib/canonical/buildd/debian.py'
767--- lib/canonical/buildd/debian.py 2011-06-09 10:50:25 +0000
768+++ lib/canonical/buildd/debian.py 1970-01-01 00:00:00 +0000
769@@ -1,239 +0,0 @@
770-# Copyright 2009, 2010 Canonical Ltd. This software is licensed under the
771-# GNU Affero General Public License version 3 (see the file LICENSE).
772-
773-# Authors: Daniel Silverstone <daniel.silverstone@canonical.com>
774-# and Adam Conrad <adam.conrad@canonical.com>
775-
776-# Buildd Slave sbuild manager implementation
777-
778-__metaclass__ = type
779-
780-import os
781-
782-from canonical.buildd.slave import (
783- BuildManager,
784- )
785-
786-
787-class DebianBuildState:
788- """States for the DebianBuildManager."""
789- INIT = "INIT"
790- UNPACK = "UNPACK"
791- MOUNT = "MOUNT"
792- SOURCES = "SOURCES"
793- UPDATE = "UPDATE"
794- REAP = "REAP"
795- UMOUNT = "UMOUNT"
796- CLEANUP = "CLEANUP"
797-
798-
799-class DebianBuildManager(BuildManager):
800- """Base behaviour for Debian chrooted builds."""
801-
802- def __init__(self, slave, buildid):
803- BuildManager.__init__(self, slave, buildid)
804- self._updatepath = slave._config.get("debianmanager", "updatepath")
805- self._scanpath = slave._config.get("debianmanager", "processscanpath")
806- self._sourcespath = slave._config.get("debianmanager", "sourcespath")
807- self._cachepath = slave._config.get("slave","filecache")
808- self._state = DebianBuildState.INIT
809- slave.emptyLog()
810- self.alreadyfailed = False
811-
812- @property
813- def initial_build_state(self):
814- raise NotImplementedError()
815-
816- def initiate(self, files, chroot, extra_args):
817- """Initiate a build with a given set of files and chroot."""
818-
819- self.arch_tag = extra_args.get('arch_tag', self._slave.getArch())
820- self.sources_list = extra_args.get('archives')
821-
822- BuildManager.initiate(self, files, chroot, extra_args)
823-
824- def doSourcesList(self):
825- """Override apt/sources.list.
826-
827- Mainly used for PPA builds.
828- """
829- args = ["override-sources-list", self._buildid]
830- args.extend(self.sources_list)
831- self.runSubProcess(self._sourcespath, args)
832-
833- def doUpdateChroot(self):
834- """Perform the chroot upgrade."""
835- self.runSubProcess(
836- self._updatepath,
837- ["update-debian-chroot", self._buildid, self.arch_tag])
838-
839- def doRunBuild(self):
840- """Run the main build process.
841-
842- Subclasses must override this.
843- """
844- raise NotImplementedError()
845-
846- def doReapProcesses(self):
847- """Reap any processes left lying around in the chroot."""
848- self.runSubProcess( self._scanpath, [self._scanpath, self._buildid] )
849-
850- @staticmethod
851- def _parseChangesFile(linesIter):
852- """A generator that iterates over files listed in a changes file.
853-
854- :param linesIter: an iterable of lines in a changes file.
855- """
856- seenfiles = False
857- for line in linesIter:
858- if line.endswith("\n"):
859- line = line[:-1]
860- if not seenfiles and line.startswith("Files:"):
861- seenfiles = True
862- elif seenfiles:
863- if not line.startswith(' '):
864- break
865- filename = line.split(' ')[-1]
866- yield filename
867-
868- def getChangesFilename(self):
869- changes = self._dscfile[:-4] + "_" + self.arch_tag + ".changes"
870- return get_build_path(self._buildid, changes)
871-
872- def gatherResults(self):
873- """Gather the results of the build and add them to the file cache.
874-
875- The primary file we care about is the .changes file. We key from there.
876- """
877- path = self.getChangesFilename()
878- name = os.path.basename(path)
879- chfile = open(path, "r")
880- self._slave.waitingfiles[name] = self._slave.storeFile(chfile.read())
881- chfile.seek(0)
882- seenfiles = False
883-
884- for fn in self._parseChangesFile(chfile):
885- self._slave.addWaitingFile(get_build_path(self._buildid, fn))
886-
887- chfile.close()
888-
889- def iterate(self, success):
890- # When a Twisted ProcessControl class is killed by SIGTERM,
891- # which we call 'build process aborted', 'None' is returned as
892- # exit_code.
893- print ("Iterating with success flag %s against stage %s"
894- % (success, self._state))
895- func = getattr(self, "iterate_" + self._state, None)
896- if func is None:
897- raise ValueError, "Unknown internal state " + self._state
898- func(success)
899-
900- def iterate_INIT(self, success):
901- """Just finished initializing the build."""
902- if success != 0:
903- if not self.alreadyfailed:
904- # The init failed, can't fathom why that would be...
905- self._slave.builderFail()
906- self.alreadyfailed = True
907- self._state = DebianBuildState.CLEANUP
908- self.doCleanup()
909- else:
910- self._state = DebianBuildState.UNPACK
911- self.doUnpack()
912-
913- def iterate_UNPACK(self, success):
914- """Just finished unpacking the tarball."""
915- if success != 0:
916- if not self.alreadyfailed:
917- # The unpack failed for some reason...
918- self._slave.chrootFail()
919- self.alreadyfailed = True
920- self._state = DebianBuildState.CLEANUP
921- self.doCleanup()
922- else:
923- self._state = DebianBuildState.MOUNT
924- self.doMounting()
925-
926- def iterate_MOUNT(self, success):
927- """Just finished doing the mounts."""
928- if success != 0:
929- if not self.alreadyfailed:
930- self._slave.chrootFail()
931- self.alreadyfailed = True
932- self._state = DebianBuildState.UMOUNT
933- self.doUnmounting()
934- else:
935- if self.sources_list is not None:
936- self._state = DebianBuildState.SOURCES
937- self.doSourcesList()
938- else:
939- self._state = DebianBuildState.UPDATE
940- self.doUpdateChroot()
941-
942- def getTmpLogContents(self):
943- try:
944- tmpLogHandle = open(os.path.join(self._cachepath, "buildlog"))
945- return tmpLogHandle.read()
946- finally:
947- tmpLogHandle.close()
948-
949- def iterate_SOURCES(self, success):
950- """Just finished overwriting sources.list."""
951- if success != 0:
952- if not self.alreadyfailed:
953- self._slave.chrootFail()
954- self.alreadyfailed = True
955- self._state = DebianBuildState.REAP
956- self.doReapProcesses()
957- else:
958- self._state = DebianBuildState.UPDATE
959- self.doUpdateChroot()
960-
961- def iterate_UPDATE(self, success):
962- """Just finished updating the chroot."""
963- if success != 0:
964- if not self.alreadyfailed:
965- self._slave.chrootFail()
966- self.alreadyfailed = True
967- self._state = DebianBuildState.REAP
968- self.doReapProcesses()
969- else:
970- self._state = self.initial_build_state
971- self.doRunBuild()
972-
973- def iterate_REAP(self, success):
974- """Finished reaping processes; ignore error returns."""
975- self._state = DebianBuildState.UMOUNT
976- self.doUnmounting()
977-
978- def iterate_UMOUNT(self, success):
979- """Just finished doing the unmounting."""
980- if success != 0:
981- if not self.alreadyfailed:
982- self._slave.builderFail()
983- self.alreadyfailed = True
984- self._state = DebianBuildState.CLEANUP
985- self.doCleanup()
986-
987- def iterate_CLEANUP(self, success):
988- """Just finished the cleanup."""
989- if success != 0:
990- if not self.alreadyfailed:
991- self._slave.builderFail()
992- self.alreadyfailed = True
993- else:
994- # Successful clean
995- if not self.alreadyfailed:
996- self._slave.buildOK()
997- self._slave.buildComplete()
998-
999-
1000-def get_build_path(build_id, *extra):
1001- """Generate a path within the build directory.
1002-
1003- :param build_id: the build id to use.
1004- :param extra: the extra path segments within the build directory.
1005- :return: the generated path.
1006- """
1007- return os.path.join(
1008- os.environ["HOME"], "build-" + build_id, *extra)
1009
1010=== removed file 'lib/canonical/buildd/debian/changelog'
1011--- lib/canonical/buildd/debian/changelog 2011-11-03 23:47:48 +0000
1012+++ lib/canonical/buildd/debian/changelog 1970-01-01 00:00:00 +0000
1013@@ -1,698 +0,0 @@
1014-launchpad-buildd (82) hardy-cat; urgency=low
1015-
1016- * Cut out readyservice from the buildds. LP: #800295
1017- * buildrecipe shows the bzr and bzr-builder versions. LP: #884092
1018- * buildrecipe shows bzr rusage. LP: 884997
1019-
1020- -- Martin Pool <mbp@canonical.com> Thu, 03 Nov 2011 17:11:25 +1100
1021-
1022-launchpad-buildd (81) hardy-cat; urgency=low
1023-
1024- * generate-translation-templates: switch to Python 2.7.
1025-
1026- -- Danilo Å egan <danilo@canonical.com> Mon, 17 Oct 2011 14:46:13 +0200
1027-
1028-launchpad-buildd (80) hardy-cat; urgency=low
1029-
1030- * binfmt-support demonstrated umount ordering issues for us. LP: #851934
1031-
1032- -- LaMont Jones <lamont@canonical.com> Mon, 19 Sep 2011 04:56:58 -0600
1033-
1034-launchpad-buildd (79) hardy-cat; urgency=low
1035-
1036- * Fix sudoers.d/buildd permissions
1037-
1038- -- LaMont Jones <lamont@canonical.com> Fri, 19 Aug 2011 07:31:54 -0600
1039-
1040-launchpad-buildd (78) hardy-cat; urgency=low
1041-
1042- * Correctly update sudoers files when needed. LP: #742881
1043-
1044- -- LaMont Jones <lamont@canonical.com> Wed, 06 Apr 2011 22:20:17 -0600
1045-
1046-launchpad-buildd (77) hardy-cat; urgency=low
1047-
1048- * Add back in ultimate-backstop umask() correction.
1049-
1050- -- LaMont Jones <lamont@canonical.com> Wed, 06 Apr 2011 13:34:05 -0600
1051-
1052-launchpad-buildd (76) hardy-cat; urgency=low
1053-
1054- [ various ]
1055- * ProjectGroup.products sort order and remove Author: comments.
1056- * Fix some tests to not print stuff
1057- * Make buildd pointer check regexes work on natty
1058- * merge before rollout + text conflict patch by wgrant
1059-
1060- -- LaMont Jones <lamont@canonical.com> Tue, 15 Mar 2011 16:59:36 -0600
1061-
1062-launchpad-buildd (74) hardy-cat; urgency=low
1063-
1064- [ Aaron Bentley]
1065- * Memory-limit recipe builds. LP#676657
1066-
1067- [ LaMont Jones]
1068- * mount a tmpfs on /dev/shm in build chroots. LP#671441
1069-
1070- [Michael Bienia]
1071- * Update regexes used for DEPWAIT. LP#615286
1072-
1073- -- LaMont Jones <lamont@canonical.com> Tue, 23 Nov 2010 06:17:57 -0700
1074-
1075-launchpad-buildd (73) hardy-cat; urgency=low
1076-
1077- * Revert to revision 70
1078-
1079- -- LaMont Jones <lamont@canonical.com> Thu, 28 Oct 2010 12:53:45 -0600
1080-
1081-launchpad-buildd (72) hardy-cat; urgency=low
1082-
1083- * break out readyservice.py from tachandler.py. LP#663828
1084-
1085- -- LaMont Jones <lamont@canonical.com> Wed, 20 Oct 2010 13:03:23 -0600
1086-
1087-launchpad-buildd (71) hardy-cat; urgency=low
1088-
1089- * Detect ppa hosts for build recipes. LP#662664
1090- * Better recipe builds. LP#599100, 627119, 479705
1091-
1092- -- LaMont Jones <lamont@canonical.com> Tue, 19 Oct 2010 13:48:33 -0600
1093-
1094-launchpad-buildd (70) hardy-cat; urgency=low
1095-
1096- [ LaMont Jones ]
1097- * Restore the rest of version 68.
1098-
1099- [ James Westby ]
1100- * buildrecipe: Specify BZR_EMAIL via sudo so that the called command
1101- sees the environment variable.
1102- * buildrecipe: call sudo -i -u instead of sudo -iu so that it works with
1103- older versions of sudo.
1104- * buildrecipe: flush stdout before calling another command so that
1105- the build log has the output correctly interleaved.
1106-
1107- [ William Grant ]
1108- * correct arch_tag arguments.
1109-
1110- -- LaMont Jones <lamont@canonical.com> Fri, 20 Aug 2010 13:27:55 -0600
1111-
1112-launchpad-buildd (69) hardy-cat; urgency=low
1113-
1114- * REVERT all of version 68 except for BZR_EMAIL LP#617072
1115- (Not reflected in bzr.)
1116-
1117- -- LaMont Jones <lamont@canonical.com> Tue, 17 Aug 2010 10:40:03 -0600
1118-
1119-launchpad-buildd (68) hardy-cat; urgency=low
1120-
1121- [ William Grant ]
1122- * Take an 'arch_tag' argument, so the master can override the slave
1123- architecture.
1124-
1125- [ Jelmer Vernooij ]
1126-
1127- * Explicitly use source format 1.0.
1128- * Add LSB information to init script.
1129- * Use debhelper >= 5 (available in dapper, not yet deprecated in
1130- maverick).
1131- * Fix spelling in description.
1132- * Install example buildd configuration.
1133-
1134- [ Paul Hummer ]
1135- * Provide BZR_EMAIL for bzr 2.2 in the buildds LP#617072
1136-
1137- -- LaMont Jones <lamont@canonical.com> Mon, 16 Aug 2010 13:25:09 -0600
1138-
1139-launchpad-buildd (67) hardy-cat; urgency=low
1140-
1141- * Force aptitude installation for recipe builds on maverick
1142-
1143- -- LaMont Jones <lamont@canonical.com> Fri, 23 Jul 2010 14:22:23 -0600
1144-
1145-launchpad-buildd (66) hardy-cat; urgency=low
1146-
1147- * handle [linux-any] build-dependencies. LP#604981
1148-
1149- -- LaMont Jones <lamont@canonical.com> Mon, 19 Jul 2010 12:13:31 -0600
1150-
1151-launchpad-buildd (65) hardy-cat; urgency=low
1152-
1153- * Drop preinst check, since human time does not scale across a large
1154- rollout. soyuz just needs to deal with upgrades mid-build better.
1155-
1156- -- LaMont Jones <lamont@canonical.com> Thu, 08 Jul 2010 05:04:02 -0600
1157-
1158-launchpad-buildd (64) hardy-cat; urgency=low
1159-
1160- * Pottery now strips quotes from variables.
1161-
1162- -- Jeroen Vermeulen <jtv@canonical.com> Wed, 30 Jun 2010 12:50:59 +0200
1163-
1164-launchpad-buildd (63) hardy-cat; urgency=low
1165-
1166- * Drop apply-ogre-model, since override-sources-list replaced it three years
1167- ago. Also clean up extra_args parsing a bit.
1168-
1169- -- William Grant <wgrant@ubuntu.com> Sat, 12 Jun 2010 11:33:11 +1000
1170-
1171-launchpad-buildd (62) hardy-cat; urgency=low
1172-
1173- * Make the buildds cope with not having a sourcepackagename LP#587109
1174-
1175- -- LaMont Jones <lamont@canonical.com> Tue, 08 Jun 2010 13:02:31 -0600
1176-
1177-launchpad-buildd (61) hardy-cat; urgency=high
1178-
1179- [ William Grant ]
1180- * Fixed translation templates slave to return files properly. LP#549422
1181-
1182- [ Danilo Segan ]
1183- * Added more output to generate-translation-templates. LP#580345
1184-
1185- [ Henning Eggers ]
1186- * Improved output of build xmplrpc call, not returning None now. LP#581746
1187- * Added apache2 dependency. LP#557634
1188- * Added preinst script to prevent installation when a build is running.
1189- LP#557347
1190-
1191- [ LaMont Jones ]
1192- * preinst needs to detect a stale buildlog as well.
1193-
1194- -- LaMont Jones <lamont@canonical.com> Fri, 21 May 2010 05:52:53 -0600
1195-
1196-launchpad-buildd (60) lucid-cat; urgency=low
1197-
1198- * Depends: lsb-release, which is ubuntu-minimal, but not essential.
1199-
1200- -- LaMont Jones <lamont@ubuntu.com> Thu, 01 Apr 2010 08:54:48 -0600
1201-
1202-launchpad-buildd (59) lucid-cat; urgency=low
1203-
1204- [ Henning Eggers ]
1205- * Added translation template generation code (pottery).
1206-
1207- [ LaMont Jones ]
1208- * set umask for twisted where supported
1209-
1210- -- LaMont Jones <lamont@canonical.com> Wed, 31 Mar 2010 10:38:15 -0600
1211-
1212-launchpad-buildd (58~1) karmic; urgency=low
1213-
1214- * Misc fixes to match APIs.
1215-
1216- -- Aaron Bentley <aaron@aaronbentley.com> Fri, 15 Jan 2010 10:03:07 +1300
1217-
1218-launchpad-buildd (58~0) karmic; urgency=low
1219-
1220- * Include buildrecipe.py.
1221-
1222- -- Aaron Bentley <aaron@aaronbentley.com> Wed, 13 Jan 2010 17:06:59 +1300
1223-
1224-launchpad-buildd (57) hardy-cat; urgency=low
1225-
1226- * Split the sbuild wrapper from DebianBuildManager into a new
1227- BinaryPackageBuildManager, and point the 'debian' builder at that
1228- instead.
1229-
1230- -- William Grant <wgrant@ubuntu.com> Tue, 12 Jan 2010 09:22:50 +1300
1231-
1232-launchpad-buildd (56) hardy-cat; urgency=low
1233-
1234- * only error out on implicit-function-pointers check on lucid or later,
1235- non-32-bit architectures. Warnings elsewhere. LP#504078
1236- * drop use of ccache and /var/cache/apt/archives, since we don't use one,
1237- and the other is just plain silly.
1238-
1239- -- LaMont Jones <lamont@canonical.com> Mon, 11 Jan 2010 13:12:49 -0700
1240-
1241-launchpad-buildd (54) hardy-cat; urgency=low
1242-
1243- [ William Grant ]
1244- * debian.py: Tell sbuild to build debug symbols if the
1245- build_debug_symbols argument is True.
1246- * sbuild: Set "Build-Debug-Symbols: yes" in CurrentlyBuilding if
1247- we have been told to build debug symbols.
1248-
1249- [ LaMont Jones ]
1250- * do not ignore SIGHUP in builds - it breaks test suites. LP#453460
1251- * create filecache-default/ccache directories in init.d as well as postinst
1252- * sbuild: run dpkg-source inside the chroot. LP#476036
1253- * sbuild: change the regexp for dpkg-source extraction to handle both karmic and pre-karmic dpkg. LP#476036
1254- * use --print-architecture instead of --print-installation-architecture
1255- * mount-chroot: copy hosts et al into chroot. LP#447919
1256- * provide and call check-implicit-function-pointers.
1257-
1258- -- LaMont Jones <lamont@canonical.com> Mon, 14 Dec 2009 12:00:10 -0700
1259-
1260-launchpad-buildd (52) dapper-cat; urgency=low
1261-
1262- * Depends: apt-transport-https
1263-
1264- -- LaMont Jones <lamont@canonical.com> Fri, 09 Oct 2009 11:00:50 -0600
1265-
1266-launchpad-buildd (50) dapper-cat; urgency=low
1267-
1268- * sbuild: Change all invocations of apt and dpkg to occur inside
1269- the build chroot, rather than happening outside the chroot with
1270- a bunch of flags to operate on data files in the chroot. This
1271- should clear up issues we see with mismatched host toolchains.
1272- * sbuild: Revert the above in the case of "apt-get source" which
1273- doesn't require any fancy features in the chroot and, frankly,
1274- is much easier to manage if it's executed externally.
1275- * scan-for-processes: Bring in a change from production to make
1276- sure that we follow symlinks in our search for process roots.
1277- * sbuild-package: Output NR_PROCESSORS in the build logs, for
1278- sightly easier debugging of possible parallel build bugs.
1279- * update-debian-chroot: Stop using chapt-get, and instead chroot
1280- into the build chroot and call the native apt-get there.
1281- * update-debian-chroot: Cargo-cult the linux32 magic from the
1282- sbuild wrapper to set our personality on chroot upgrades.
1283- * mount-chroot: Mount sys in the chroot too. While it shouldn't
1284- be, strictly-speaking, required for anything, it's nice to have.
1285- * chapt-get, slave_chroot_tool.py: Delete both as obsolete cruft.
1286-
1287- -- Adam Conrad <adconrad@ubuntu.com> Fri, 24 Jul 2009 07:21:30 -0600
1288-
1289-launchpad-buildd (49) dapper-cat; urgency=low
1290-
1291- * sbuild.conf: bump default automake from automake1.8 to automake1.9
1292-
1293- -- Adam Conrad <adconrad@ubuntu.com> Fri, 12 Sep 2008 08:54:24 -0600
1294-
1295-launchpad-buildd (48) dapper-cat; urgency=low
1296-
1297- * sbuild-package: If we're an amd64 host system, but being used
1298- to build i386 or lpia, use linux32 to pretend to be i686.
1299-
1300- -- Adam Conrad <adconrad@ubuntu.com> Fri, 12 Sep 2008 08:12:34 -0600
1301-
1302-launchpad-buildd (47) dapper-cat; urgency=low
1303-
1304- * slave.py: If the logfile doesn't currently exist on disk when
1305- getLogTail() goes looking for it (which is a possible race with
1306- the new sanitisation code), just return an empty string.
1307-
1308- -- Adam Conrad <adconrad@ubuntu.com> Mon, 02 Jun 2008 13:09:55 -0600
1309-
1310-launchpad-buildd (46) dapper-cat; urgency=low
1311-
1312- * slave.py: Accept a separate username and password to the
1313- ensurePresent() call which, if present, are used to install
1314- an auth handler to cope with basic http auth with the http
1315- server when fetching files.
1316- * slave.py: Ensure that build logs are sanitized so that any
1317- user:password@ parts in URLs are removed.
1318-
1319- -- Julian Edwards <julian.edwards@canonical.com> Tue, 29 Apr 2008 14:25:00 +0100
1320-
1321-launchpad-buildd (45) dapper-cat; urgency=low
1322-
1323- * slave.py: Stop setting BuilderStatus.WAITING in each failure
1324- method, as this gives us a race where the builddmaster might
1325- dispatch another build to us before we're done cleaning up.
1326- * slave.py: Don't set BuildStatus.OK in buildComplete(), this is
1327- now a generic "the build has ended, succesfully or not" method.
1328- * slave.py: Define a new buildOK() method that sets BuildStatus.OK.
1329- * debian.py: When done cleaning, if the build isn't already marked
1330- as failed, call buildOK, then call buildComplete unconditionally.
1331- * The above changes should resolve https://launchpad.net/bugs/179466
1332-
1333- -- Adam Conrad <adconrad@ubuntu.com> Tue, 08 Apr 2008 14:12:07 -0600
1334-
1335-launchpad-buildd (44) dapper-cat; urgency=low
1336-
1337- * slave.py: Redefine "private" _unpackChroot() as "public" doUnpack(),
1338- so we can use it from the build iteration control process.
1339- * slave.py: Make the initiate method set a _chroottarfile private
1340- variable for use by doUnpack(), rather than calling _unpackChroot().
1341- * slave.py: Trigger the forked buildd process with an echo statement.
1342- * debian.py: Add the INIT state to the DebianBuildState class.
1343- * debian.py: Start the build process at INIT state instead of UNPACK.
1344- * debian.py: Add iterate_INIT(), which just checks success of the
1345- initial variable sanitisation checks, then hands off to doUnpack().
1346- * debian.py: Adjust the failure return calls of the UNPACK and MOUNT
1347- methods to chrootFail() instead of builderFail(), for correctness.
1348- * The above changes should resolve https://launchpad.net/bugs/211974
1349-
1350- -- Adam Conrad <adconrad@ubuntu.com> Mon, 07 Apr 2008 13:53:20 -0600
1351-
1352-launchpad-buildd (43) dapper-cat; urgency=low
1353-
1354- * unpack-chroot: Move the ntpdate calls below the bunzip/exec bit,
1355- so we don't run ntpdate twice when unzipping tarballs, which
1356- happens on every single build on Xen hosts (like the PPA hosts).
1357- * debian/control: We use adduser in postinst, depending on it helps.
1358- * debian/control: Set myself as the Maintainer, since I'm in here.
1359- * debian/control: Change our section from "misc" to "admin".
1360- * sbuild{,-package}: Pass DEB_BUILD_OPTIONS="parallel=N" to dpkg.
1361-
1362- -- Adam Conrad <adconrad@ubuntu.com> Thu, 24 Jan 2008 15:39:20 -0700
1363-
1364-launchpad-buildd (42) dapper-cat; urgency=low
1365-
1366- * sbuild: using "eq" to evaluate strings instead of "==" is ever
1367- so slightly less retarded (fixed the launchpad bug #184565)
1368-
1369- -- Adam Conrad <adconrad@ubuntu.com> Tue, 22 Jan 2008 16:21:54 -0700
1370-
1371-launchpad-buildd (41) dapper-cat; urgency=low
1372-
1373- * sbuild: If we've already marked a package as "installed" with a
1374- valid version, don't overwrite that version with PROVIDED.
1375-
1376- -- Adam Conrad <adconrad@ubuntu.com> Thu, 17 Jan 2008 10:39:26 -0700
1377-
1378-launchpad-buildd (40) dapper-cat; urgency=low
1379-
1380- * sbuild: Don't allow versioned build-deps to be satisfied by provided
1381- packages, but force them to go through the "upgrade/downgrade" tests.
1382- * sbuild: Do --info and --contents on _all.deb packages as well, if
1383- we're building arch:all packages.
1384- * sbuild: Don't process ENV_OVERRIDE anymore, we only had an override
1385- for one thing anyway (LC_ALL), and this code caused bug #87077.
1386- * sbuild-package: Call sbuild with LC_ALL=C explicitely, to compensate.
1387- * Makefile: clean up the makefile a bit to DTRT (as I expect it).
1388-
1389- -- Adam Conrad <adconrad@ubuntu.com> Tue, 15 Jan 2008 16:51:08 -0700
1390-
1391-launchpad-buildd (39) unstable; urgency=low
1392-
1393- * If we're fed an archive_purpose argument from the builddmaster,
1394- we pass --purpose=$archive_purpose to sbuild, and if we get suite
1395- from the builddmaster, we pass --dist=$suite to sbuild.
1396- * Mangle sbuild to write out Suite: and Purpose: stanzas to our
1397- CurrentlyBuilding file, according to command-line input.
1398- * Now that we're no longer always feeding -dautobuild to sbuild,
1399- fix up sbuild to always look for the chroot at chroot-autobuild
1400- instead of the Debian Way of using chroot-$suite.
1401- * If the config file contains an ntphost stanza, use that with
1402- ntpdate to sync the system's clock before we unpack the chroot.
1403- * Mangle update-config to add an ntphost stanza to the default
1404- config, and to 's/-dautobuild //' from the sbuild arguments.
1405-
1406- -- Adam Conrad <adconrad@ubuntu.com> Thu, 20 Dec 2007 01:51:49 -0700
1407-
1408-launchpad-buildd (38) unstable; urgency=high
1409-
1410- * unpack-chroot: set $PATH rather than hardcoding paths to binaries
1411- since bzip2 moved from /usr/bin to /bin in edgy and didn't bother with
1412- compatability symlinks.
1413-
1414- -- James Troup <james.troup@canonical.com> Wed, 21 Nov 2007 17:08:36 +0000
1415-
1416-launchpad-buildd (37) dapper; urgency=high
1417-
1418- * update-debian-chroot: Adam's LPIA support (i.e. overriding
1419- architecture for chapt-get).
1420- * debian/launchpad-buildd.cron.daily: fix run-on-line.
1421- * debian/postinst: only create ~buildd/.sbuildrc if it doesn't exist.
1422- This avoids the problem of upgrades of the launchpad-buildd package
1423- resetting the architecture to i386 on lpia builders.
1424-
1425- -- James Troup <james.troup@canonical.com> Wed, 14 Nov 2007 18:34:46 +0000
1426-
1427-launchpad-buildd (36) dapper; urgency=low
1428-
1429- * changing override-sources to replace current sources.list with
1430- the content sent by buildmaster instead of prepend. It will allow
1431- us to cope more easily with SoyuzArchive implementation (PARTNER,
1432- EMBARGOED, PPA)
1433-
1434- -- Celso Providelo <cprov@canonical.com> Thu, 7 Aug 2007 14:10:26 -0300
1435-
1436-launchpad-buildd (35) unstable; urgency=low
1437-
1438- * including previous code changes (32 & 33).
1439-
1440- -- Celso Providelo <cprov@canonical.com> Thu, 23 May 2007 17:40:26 -0300
1441-
1442-launchpad-buildd (34) unstable; urgency=low
1443-
1444- * add suport for overriding the chroot /etc/apt/sources.list with the
1445- content of builddmaster build arguments 'archives'.
1446-
1447- -- Celso Providelo <cprov@canonical.com> Thu, 17 May 2007 15:12:26 -0300
1448-
1449-launchpad-buildd (33) unstable; urgency=low
1450-
1451- * Mangle sbuild further to allow us to publish Martin's debug debs (ddeb)
1452- to public_html/ddebs/ until such a time as soyuz can do this natively.
1453- * Fix the auto-dep-wait regexes to allow for versions with ~ in them.
1454- * Make cron.daily clean out translations and ddebs more than 1 week old.
1455-
1456- -- Adam Conrad <adconrad@ubuntu.com> Sat, 30 Sep 2006 17:25:25 +1000
1457-
1458-launchpad-buildd (32) unstable; urgency=low
1459-
1460- * We need to create /var/run/launchpad-buildd in our init script in the
1461- case (such as in current dapper) where /var/run is on a tmpfs.
1462- * Our init script shouldn't exit non-zero on "stop" if already stopped.
1463- * Remove exc_info argument from our call to self.log in slave.py, which
1464- clearly doesn't support that argument, so stop producing tracebacks.
1465- * Reset self.builddependencies in our clean routine, so the variable
1466- doesn't get leaked to the next build, causing me SERIOUS confusion.
1467- * Tidy up translation handling a bit more to deal with old chroots (where
1468- pkgstriptranslations won't dpkg-distaddfile for us), and to chmod the
1469- translation dirs after the build, so apache can actually get at them.
1470- * Add --no_save to our command line to avoid useless -shutdown.tap files.
1471- * Make sure umount-chroot doesn't fail, even if there's nothing to umount.
1472- * Append to the cron.daily cleaning to also occasionally clean up the apt
1473- cache and /home/buildd/filecache-default, so we don't run out of disk.
1474-
1475- -- Adam Conrad <adconrad@ubuntu.com> Fri, 17 Mar 2006 19:39:05 +1100
1476-
1477-launchpad-buildd (31) unstable; urgency=low
1478-
1479- * Cherry-pick patch from Ryan's sbuild that outputs dpkg --purge output
1480- line-by-line, instead of as one big blob, to make output on the web
1481- UI a little bit more friendly for people following along at home.
1482- * Install a cron.daily script (eww) to purge old build logs for now until
1483- I have the time to learn how twisted's native log rotation works.
1484-
1485- -- Adam Conrad <adconrad@ubuntu.com> Wed, 15 Mar 2006 17:23:26 +1100
1486-
1487-launchpad-buildd (30) unstable; urgency=low
1488-
1489- * Move our translation publishing mojo so it happens BEFORE we move
1490- all the files from debian/files out of the chroot, instead of after.
1491-
1492- -- Adam Conrad <adconrad@ubuntu.com> Wed, 8 Mar 2006 18:50:49 +1100
1493-
1494-launchpad-buildd (29) unstable; urgency=low
1495-
1496- * Use dpkg --print-installation-architecture in our postinst instead
1497- of --print-architecture to avoid spewing suprious error messages.
1498- * Remove the check for log_dir, since we call sbuild with --nolog,
1499- and stop creating $HOME/logs in the user setup part of postinst.
1500-
1501- -- Adam Conrad <adconrad@ubuntu.com> Tue, 7 Mar 2006 19:13:56 +1100
1502-
1503-launchpad-buildd (28) unstable; urgency=low
1504-
1505- * Modify the protocol method ensurepresent to return additional
1506- information about the target files lookup procedure. It helps to
1507- debug intermittent Librarian errors.
1508-
1509- -- Celso Providelo <celso.providelo@canonical.com> Mon, 06 Mar 2006 16:42:00 -0300
1510-
1511-launchpad-buildd (27) unstable; urgency=low
1512-
1513- * Update the slave chroot tool to use getent so it works on the production
1514- buildds
1515-
1516- -- Daniel Silverstone <daniel.silverstone@canonical.com> Mon, 20 Feb 2006 12:57:45 +0000
1517-
1518-launchpad-buildd (26) unstable; urgency=low
1519-
1520- * Update buildd-slave code to allow for GIVENBACK status returns,
1521- matching the states under which sbuild used to do --auto-give-back.
1522- * Port over sanae's build log regex parsing to allow us to do:
1523- - Automatic dep-wait handling, based on sbuild's logs of apt-get.
1524- - Automatic give-backs for a few corner cases (like kernel bugs).
1525- * Make sbuild stop dying if we have no sendmail installed, since we
1526- don't really want it sending mail in the launchpad world anyway.
1527- * Call sbuild and apt with "LANG=C", so we don't have to worry about
1528- locales matching between the base system and the autobuild chroots.
1529- * Clear up confusion in build states with 's/BUILDFAIL/PACKAGEFAIL/'
1530-
1531- -- Adam Conrad <adconrad@ubuntu.com> Mon, 27 Feb 2006 14:00:08 +1100
1532-
1533-launchpad-buildd (25) unstable; urgency=low
1534-
1535- * Update sbuild.conf to current yumminess.
1536-
1537- -- Daniel Silverstone <daniel.silverstone@canonical.com> Fri, 3 Feb 2006 19:22:01 +0000
1538-
1539-launchpad-buildd (24) unstable; urgency=low
1540-
1541- * Add /var/cache/apt/archives to the buildd chroots when mounting
1542-
1543- -- Daniel Silverstone <daniel.silverstone@canonical.com> Fri, 3 Feb 2006 00:30:07 +0000
1544-
1545-launchpad-buildd (23) unstable; urgency=low
1546-
1547- * And make apply-ogre-model use $SUDO, yay
1548-
1549- -- Daniel Silverstone <daniel.silverstone@canonical.com> Fri, 27 Jan 2006 13:59:10 +0000
1550-
1551-launchpad-buildd (22) unstable; urgency=low
1552-
1553- * Fix typo in apply-ogre-model (missing space)
1554-
1555- -- Daniel Silverstone <daniel.silverstone@canonical.com> Fri, 27 Jan 2006 13:55:12 +0000
1556-
1557-launchpad-buildd (21) unstable; urgency=low
1558-
1559- * Fix the .extend call for the --comp argument to pass it as one argument
1560- instead of as - - c o m p = m a i n (which kinda doesn't work)
1561-
1562- -- Daniel Silverstone <daniel.silverstone@canonical.com> Fri, 27 Jan 2006 13:45:34 +0000
1563-
1564-launchpad-buildd (20) unstable; urgency=low
1565-
1566- * Update sbuild to the latest sbuild from adam.
1567- * Make sure we pass --archive=ubuntu
1568- * Make sure we pass --comp=<the component we're building for>
1569-
1570- -- Daniel Silverstone <daniel.silverstone@canonical.com> Thu, 26 Jan 2006 17:20:49 +0000
1571-
1572-launchpad-buildd (19) unstable; urgency=low
1573-
1574- * Add ogre support to the slave chroot tool
1575- * Make sure the chroot tool ensures localhost in /etc/hosts in the chroot
1576-
1577- -- Daniel Silverstone <daniel.silverstone@canonical.com> Wed, 25 Jan 2006 12:29:04 +0000
1578-
1579-launchpad-buildd (18) unstable; urgency=low
1580-
1581- * Remove sbuildrc.tmp dangleberry in postinst
1582- * Add linux32 to set of depends so that hppa, sparc and powerpc can build
1583- * Make hppa, sparc, powerpc use linux32 to invoke the sbuild binary
1584- * Add --resolve-deps to debootstrap invocation
1585- * Make chroot tool use /bin/su - rather than /bin/sh for chrooting. shiny
1586- (apparently)
1587- * Add a bunch of deps infinity spotted.
1588- * Make sure we chown the chroot tarball to the calling user after packing
1589- it up.
1590-
1591- -- Daniel Silverstone <daniel.silverstone@canonical.com> Wed, 9 Nov 2005 17:37:37 -0500
1592-
1593-launchpad-buildd (17) unstable; urgency=low
1594-
1595- * Changed default UID/GID to match the ldap buildd UID/GID
1596-
1597- -- Daniel Silverstone <daniel.silverstone@canonical.com> Wed, 9 Nov 2005 17:13:22 -0500
1598-
1599-launchpad-buildd (16) unstable; urgency=low
1600-
1601- * Change the XMLRPC method 'ensure' to be 'ensurepresent'
1602-
1603- -- Daniel Silverstone <daniel.silverstone@canonical.com> Wed, 5 Oct 2005 15:50:58 +0100
1604-
1605-launchpad-buildd (15) unstable; urgency=low
1606-
1607- * Fix it so getting a logtail when less than 2k is available will work.
1608- * Actually install apply-ogre-model
1609- * Also spot arch_indep properly
1610-
1611- -- Daniel Silverstone <daniel.silverstone@canonical.com> Mon, 3 Oct 2005 14:34:55 +0100
1612-
1613-launchpad-buildd (14) unstable; urgency=low
1614-
1615- * Slight bug in slave.py meant missing .emptyLog() attribute. Fixed.
1616-
1617- -- Daniel Silverstone <daniel.silverstone@canonical.com> Mon, 3 Oct 2005 14:21:16 +0100
1618-
1619-launchpad-buildd (13) unstable; urgency=low
1620-
1621- * Fix a syntax error in the postinst
1622- * Oh, and actually include the buildd config upgrader
1623-
1624- -- Daniel Silverstone <daniel.silverstone@canonical.com> Mon, 3 Oct 2005 12:17:50 +0100
1625-
1626-launchpad-buildd (12) unstable; urgency=low
1627-
1628- * Implement V1.0new protocol.
1629- * Add in OGRE support
1630- * Add in archindep support
1631- * If upgrading from < v12, will remove -A from sbuildargs and add in
1632- a default ogrepath to any buildd configs found in /etc/launchpad-buildd
1633- * Prevent launchpad-buildd init from starting ~ files
1634-
1635- -- Daniel Silverstone <daniel.silverstone@canonical.com> Sun, 2 Oct 2005 23:20:08 +0100
1636-
1637-launchpad-buildd (11) unstable; urgency=low
1638-
1639- * Quieten down the slave scripts and make them prettier for the logs.
1640- * make unpack-chroot uncompress the chroot and keep it uncompressed if
1641- possible. This fixes bug#2699
1642- * Make the slave run the process reaper run even if the build failed.
1643-
1644- -- Daniel Silverstone <daniel.silverstone@canonical.com> Fri, 30 Sep 2005 00:24:45 +0100
1645-
1646-launchpad-buildd (10) unstable; urgency=low
1647-
1648- * Make sure /etc/source-dependencies is present in the postinst.
1649- (just need to be touched)
1650-
1651- -- Daniel Silverstone <daniel.silverstone@canonical.com> Wed, 28 Sep 2005 22:02:26 +0100
1652-
1653-launchpad-buildd (9) unstable; urgency=low
1654-
1655- * Implement /filecache/XXX urls in the slave to permit larger file transfer
1656-
1657- -- Daniel Silverstone <daniel.silverstone@canonical.com> Tue, 27 Sep 2005 13:16:52 +0100
1658-
1659-launchpad-buildd (8) unstable; urgency=low
1660-
1661- * spiv's crappy spawnFDs implementation needs an int not a file handle
1662- and can't cope with converting one to the other :-(
1663-
1664- -- Daniel Silverstone <daniel.silverstone@canonical.com> Tue, 27 Sep 2005 02:18:05 +0100
1665-
1666-launchpad-buildd (7) unstable; urgency=low
1667-
1668- * Made sure the slave puts /dev/null on the subprocess stdin.
1669-
1670- -- Daniel Silverstone <daniel.silverstone@canonical.com> Tue, 27 Sep 2005 01:52:50 +0100
1671-
1672-launchpad-buildd (6) unstable; urgency=low
1673-
1674- * Removed slavechroot.py from installed set.
1675-
1676- -- Daniel Silverstone <daniel.silverstone@canonical.com> Thu, 15 Sep 2005 11:39:25 +0100
1677-
1678-launchpad-buildd (5) unstable; urgency=low
1679-
1680- * Add slave tool and example chroot configuration
1681- * Added debootstrap and dpkg-dev to the dependencies
1682-
1683- -- Daniel Silverstone <daniel.silverstone@canonical.com> Fri, 9 Sep 2005 16:38:22 +0100
1684-
1685-launchpad-buildd (4) unstable; urgency=low
1686-
1687- * Add sbuild.conf which was previously missing
1688- * Fix up abort protocol and various other bits in the slave
1689-
1690- -- Daniel Silverstone <daniel.silverstone@canonical.com> Fri, 9 Sep 2005 14:24:31 +0100
1691-
1692-launchpad-buildd (3) unstable; urgency=low
1693-
1694- * Modified postinst to make sure ccache and log dirs are created
1695- even if the user already exists.
1696-
1697- -- Daniel Silverstone <daniel.silverstone@canonical.com> Wed, 7 Sep 2005 15:50:36 +0100
1698-
1699-launchpad-buildd (2) unstable; urgency=low
1700-
1701- * Fixes to postinst to make sure ccache and log dirs are created if missing.
1702- * Added README to explain how to build the package.
1703-
1704- -- Daniel Silverstone <daniel.silverstone@canonical.com> Thu, 1 Sep 2005 10:46:08 +0100
1705-
1706-launchpad-buildd (1) unstable; urgency=low
1707-
1708- * Initial version
1709-
1710- -- Daniel Silverstone <daniel.silverstone@canonical.com> Mon, 13 Jun 2005 11:08:38 +0100
1711-
1712
1713=== removed file 'lib/canonical/buildd/debian/compat'
1714--- lib/canonical/buildd/debian/compat 2010-07-22 16:15:58 +0000
1715+++ lib/canonical/buildd/debian/compat 1970-01-01 00:00:00 +0000
1716@@ -1,1 +0,0 @@
1717-5
1718
1719=== removed file 'lib/canonical/buildd/debian/control'
1720--- lib/canonical/buildd/debian/control 2010-09-02 19:02:27 +0000
1721+++ lib/canonical/buildd/debian/control 1970-01-01 00:00:00 +0000
1722@@ -1,17 +0,0 @@
1723-Source: launchpad-buildd
1724-Section: admin
1725-Priority: extra
1726-Maintainer: Adam Conrad <adconrad@ubuntu.com>
1727-Standards-Version: 3.5.9
1728-Build-Depends-Indep: debhelper (>= 5)
1729-
1730-Package: launchpad-buildd
1731-Section: misc
1732-Architecture: all
1733-Depends: python-twisted-core, python-twisted-web, debootstrap, dpkg-dev, linux32, file, bzip2, sudo, ntpdate, adduser, apt-transport-https, lsb-release, apache2, bzr-builder (>=0.5), ${misc:Depends}
1734-Description: Launchpad buildd slave
1735- This is the launchpad buildd slave package. It contains everything needed to
1736- get a launchpad buildd going apart from the database manipulation required to
1737- tell launchpad about the slave instance. If you are creating more than one
1738- slave instance on the same computer, be sure to give them independent configs
1739- and independent filecaches etc.
1740
1741=== removed file 'lib/canonical/buildd/debian/launchpad-buildd.cron.daily'
1742--- lib/canonical/buildd/debian/launchpad-buildd.cron.daily 2009-06-30 21:27:34 +0000
1743+++ lib/canonical/buildd/debian/launchpad-buildd.cron.daily 1970-01-01 00:00:00 +0000
1744@@ -1,10 +0,0 @@
1745-#!/bin/sh
1746-#
1747-# Copyright 2009 Canonical Ltd. This software is licensed under the
1748-# GNU Affero General Public License version 3 (see the file LICENSE).
1749-
1750-find /var/log/launchpad-buildd/ -mindepth 1 -mtime +2 -print0 | xargs -r -0 rm
1751-find /home/buildd/public_html/ddebs/ -maxdepth 1 -mindepth 1 -mtime +7 -print0 | xargs -r -0 rm -r
1752-find /home/buildd/public_html/translations/ -maxdepth 1 -mindepth 1 -mtime +7 -print0 | xargs -r -0 rm -r
1753-find /home/buildd/filecache-default/ -mindepth 1 -mtime +2 -print0 | xargs -r -0 rm
1754-[ `date +%w` = "1" ] && apt-get clean || true
1755
1756=== removed file 'lib/canonical/buildd/debian/launchpad-buildd.examples'
1757--- lib/canonical/buildd/debian/launchpad-buildd.examples 2010-07-22 17:40:33 +0000
1758+++ lib/canonical/buildd/debian/launchpad-buildd.examples 1970-01-01 00:00:00 +0000
1759@@ -1,1 +0,0 @@
1760-buildd-slave-example.conf
1761
1762=== removed file 'lib/canonical/buildd/debian/launchpad-buildd.init'
1763--- lib/canonical/buildd/debian/launchpad-buildd.init 2010-12-21 17:52:32 +0000
1764+++ lib/canonical/buildd/debian/launchpad-buildd.init 1970-01-01 00:00:00 +0000
1765@@ -1,98 +0,0 @@
1766-#!/bin/sh
1767-#
1768-# Copyright 2009 Canonical Ltd. This software is licensed under the
1769-# GNU Affero General Public License version 3 (see the file LICENSE).
1770-#
1771-# launchpad-buildd
1772-# This file is used to start and stop launchpad buildds
1773-
1774-### BEGIN INIT INFO
1775-# Provides: launchpad_buildd
1776-# Required-Start: $local_fs $network $syslog $time
1777-# Required-Stop: $local_fs $network $syslog $time
1778-# Default-Start: 2 3 4 5
1779-# Default-Stop: 0 1 6
1780-# X-Interactive: false
1781-# Short-Description: Start/stop launchpad buildds
1782-### END INIT INFO
1783-
1784-set -e
1785-
1786-PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
1787-DESC="launchpad build slaves"
1788-
1789-TACFILE="/usr/share/launchpad-buildd/buildd-slave.tac"
1790-
1791-PIDROOT="/var/run/launchpad-buildd"
1792-LOGROOT="/var/log/launchpad-buildd"
1793-CONFROOT="/etc/launchpad-buildd"
1794-
1795-# Gracefully exit if the package has been removed.
1796-test -e $TACFILE || exit 0
1797-
1798-#
1799-# Function that starts a buildd slave
1800-#
1801-d_start() {
1802- CONF=$1
1803- PIDFILE="$PIDROOT"/"$CONF".pid
1804- LOGFILE="$LOGROOT"/"$CONF".log
1805- # prior to karmic, twisted didn't support --umask, and defaulted it well.
1806- # we need it to be 022, not 077.
1807- case $(lsb_release -sc) in
1808- [a-j]*) UMASK="";;
1809- [k-z]*) UMASK="--umask 022";;
1810- esac
1811- su - buildd -c "BUILDD_SLAVE_CONFIG=$CONFROOT/$CONF PYTHONPATH=/usr/share/launchpad-buildd twistd --no_save --pidfile $PIDFILE --python $TACFILE --logfile $LOGFILE $UMASK"
1812-}
1813-
1814-#
1815-# Function that stops a buildd slave
1816-#
1817-d_stop() {
1818- CONF=$1
1819- PIDFILE="$PIDROOT"/"$CONF".pid
1820- test -r $PIDFILE && kill -TERM $(cat $PIDFILE) || true
1821-}
1822-
1823-CONFS=$(cd $CONFROOT; ls|grep -v "^-"|grep -v "~$")
1824-
1825-case "$1" in
1826- start)
1827- echo -n "Starting $DESC:"
1828- install -m 755 -o buildd -g buildd -d $PIDROOT
1829-
1830- # Create any missing directories and chown them appropriately
1831- install -d -o buildd -g buildd /home/buildd/filecache-default
1832-
1833- for conf in $CONFS; do
1834- echo -n " $conf"
1835- d_start $conf
1836- done
1837- echo "."
1838- ;;
1839- stop)
1840- echo -n "Stopping $DESC:"
1841- for conf in $CONFS; do
1842- echo -n " $conf"
1843- d_stop $conf
1844- done
1845- echo "."
1846- ;;
1847- restart|force-reload)
1848- #
1849- # If the "reload" option is implemented, move the "force-reload"
1850- # option to the "reload" entry above. If not, "force-reload" is
1851- # just the same as "restart".
1852- #
1853- $0 stop
1854- sleep 1
1855- $0 start
1856- ;;
1857- *)
1858- echo "Usage: $SCRIPTNAME {start|stop|restart|force-reload}" >&2
1859- exit 1
1860- ;;
1861-esac
1862-
1863-exit 0
1864
1865=== removed file 'lib/canonical/buildd/debian/postinst'
1866--- lib/canonical/buildd/debian/postinst 2011-08-19 13:31:48 +0000
1867+++ lib/canonical/buildd/debian/postinst 1970-01-01 00:00:00 +0000
1868@@ -1,101 +0,0 @@
1869-#!/bin/sh
1870-#
1871-# Copyright 2009 Canonical Ltd. This software is licensed under the
1872-# GNU Affero General Public License version 3 (see the file LICENSE).
1873-
1874-# Post install script
1875-
1876-set -e
1877-set -u
1878-
1879-USER=${BUILDD_USER:-buildd}
1880-BUILDDGID=${BUILDD_GID:-2501}
1881-BUILDDUID=${BUILDD_UID:-2001}
1882-
1883-make_buildd()
1884-{
1885- buildd-genconfig --name=default --host=0.0.0.0 --port=8221 > \
1886- /etc/launchpad-buildd/default
1887- echo Default buildd created.
1888-}
1889-
1890-case "$1" in
1891- configure)
1892- getent group buildd >/dev/null 2>&1 ||
1893- addgroup --gid $BUILDDGID buildd
1894-
1895- getent passwd buildd >/dev/null 2>&1 ||
1896- adduser --ingroup buildd --disabled-login --gecos 'Buildd user' \
1897- --uid $BUILDDUID ${USER}
1898-
1899- SUDO_VERSION=$(sudo -V | sed -n '/^Sudo version/s/.* //p')
1900- if dpkg --compare-versions $SUDO_VERSION lt 1.7 ||
1901- ! grep -q '^#includedir /etc/sudoers.d' /etc/sudoers; then
1902- grep -q ^${USER} /etc/sudoers ||
1903- echo "${USER} ALL=(ALL) NOPASSWD: ALL" >>/etc/sudoers
1904- else
1905- if ! sudo -l -U ${USER} | grep -q '(ALL) NOPASSWD: ALL' ||
1906- ! sudo -l -U ${USER} | grep -q umask_override; then
1907- cat << EOF >> /etc/sudoers.d/buildd
1908-Defaults:buildd umask_override
1909-Defaults>buildd umask_override
1910-
1911-${USER} ALL=(ALL) NOPASSWD: ALL
1912-%buildd ALL=(buildd) NOPASSWD: ALL, (root) NOPASSWD: /bin/su - buildd
1913-%lpbuildd ALL=(ALL) NOPASSWD: ALL
1914-EOF
1915- fi
1916- chmod 440 /etc/sudoers.d/buildd
1917- chown root:root /etc/sudoers.d/buildd
1918- fi
1919-
1920- install -d -o${USER} -m0755 /home/${USER}
1921-
1922- # Create ~buildd/.sbuildrc if needed
1923- if [ ! -f /home/${USER}/.sbuildrc ]; then
1924- cp /usr/share/launchpad-buildd/sbuildrc \
1925- /home/${USER}/sbuildrc.tmp
1926- cd /home/${USER}
1927- sed -e's/@ARCHTAG@/'$(dpkg --print-architecture)'/g' \
1928- -e's/@FQDN@/'$(hostname --fqdn)'/g' \
1929- <sbuildrc.tmp > .sbuildrc
1930- chown $USER:buildd .sbuildrc
1931- chmod 0644 .sbuildrc
1932- rm sbuildrc.tmp
1933- fi
1934-
1935- # Prepare a default buildd...
1936- test -e /etc/launchpad-buildd/default || make_buildd
1937-
1938- # Create any missing directories and chown them appropriately
1939- test -d /home/${USER}/filecache-default || mkdir /home/${USER}/filecache-default
1940- chown $USER:buildd /home/${USER}/filecache-default
1941-
1942- chown $USER:buildd /var/log/launchpad-buildd /var/run/launchpad-buildd
1943-
1944- # Check for the presence of the /etc/source-dependencies file
1945- # which sbuild will rant about the absence of...
1946- test -e /etc/source-dependencies || touch /etc/source-dependencies
1947-
1948- # Now check if we're upgrading a previous version...
1949- if [ "x$2" != "x" ]; then
1950- for CONFIG in $(ls /etc/launchpad-buildd/* \
1951- | grep -v "^-" | grep -v "~$"); do
1952- /usr/share/launchpad-buildd/upgrade-config $2 $CONFIG
1953- done
1954- fi
1955-
1956- ;;
1957- abort-upgrade|abort-remove|abort-deconfigure)
1958-
1959- ;;
1960-
1961- *)
1962- echo "postinst called with unknown argument \`$1'" >&2
1963- exit 1
1964- ;;
1965-esac
1966-
1967-#DEBHELPER#
1968-
1969-exit 0
1970
1971=== removed file 'lib/canonical/buildd/debian/rules'
1972--- lib/canonical/buildd/debian/rules 2011-11-03 05:58:20 +0000
1973+++ lib/canonical/buildd/debian/rules 1970-01-01 00:00:00 +0000
1974@@ -1,96 +0,0 @@
1975-#!/usr/bin/make -f
1976-#
1977-# Copyright 2009, 2010 Canonical Ltd. This software is licensed under the
1978-# GNU Affero General Public License version 3 (see the file LICENSE).
1979-
1980-export DH_OPTIONS
1981-
1982-# This is an incomplete debian rules file for making the launchpad-buildd deb
1983-# Only ever invoke this as debian/rules package, which will build the source
1984-# package in the parent directory, after copying in the files that live above
1985-# this directory, so that the source package is valid.
1986-# after that, build the source package found in the parent directory.
1987-
1988-target = debian/launchpad-buildd
1989-topdir = ../../..
1990-
1991-buildd = $(topdir)/lib/canonical/buildd
1992-
1993-targetshare = $(target)/usr/share/launchpad-buildd
1994-pytarget = $(targetshare)/canonical/buildd
1995-
1996-pyfiles = debian.py slave.py binarypackage.py utils.py __init__.py \
1997- sourcepackagerecipe.py translationtemplates.py
1998-slavebins = unpack-chroot mount-chroot update-debian-chroot sbuild-package \
1999- scan-for-processes umount-chroot remove-build override-sources-list \
2000- buildrecipe generate-translation-templates
2001-
2002-BUILDDUID=65500
2003-BUILDDGID=65500
2004-
2005-install: DH_OPTIONS=-plaunchpad-buildd
2006-install:
2007- dh_testdir
2008- dh_clean
2009- dh_testroot
2010- dh_installdirs usr/bin etc usr/share/launchpad-buildd/slavebin \
2011- usr/share/launchpad-buildd/canonical/buildd \
2012- var/run/launchpad-buildd var/log/launchpad-buildd \
2013- etc/launchpad-buildd \
2014- usr/share/launchpad-buildd/canonical/launchpad/daemons \
2015- usr/share/doc/launchpad-buildd
2016- dh_installexamples
2017-
2018- # Do installs here
2019- touch $(pytarget)/../launchpad/__init__.py
2020- touch $(pytarget)/../launchpad/daemons/__init__.py
2021- install -m644 buildd-slave.tac $(targetshare)/buildd-slave.tac
2022- cp -r pottery $(pytarget)
2023- for pyfile in $(pyfiles); do \
2024- install -m644 ./$$pyfile $(pytarget)/$$pyfile; \
2025- done
2026- for slavebin in $(slavebins); do \
2027- install -m755 ./$$slavebin $(targetshare)/slavebin/$$slavebin; \
2028- done
2029- install -m755 ./sbuild $(target)/usr/bin/sbuild
2030- touch $(targetshare)/canonical/__init__.py
2031- install -m644 template-buildd-slave.conf $(targetshare)/template-buildd-slave.conf
2032- install -m755 buildd-config.py $(target)/usr/bin/buildd-genconfig
2033- install -m644 sbuildrc $(targetshare)/sbuildrc
2034- install -m644 sbuild.conf $(target)/etc/sbuild.conf
2035- install -m755 debian/upgrade-config $(targetshare)/upgrade-config
2036- install -m755 check-implicit-pointer-functions $(target)/usr/bin/check-implicit-pointer-functions
2037- # Okay, that's installed all the slave-related files
2038-
2039-
2040-binary-arch:
2041- @echo No arch-specific binaries to make
2042-
2043-binary-indep: DH_OPTIONS=-plaunchpad-buildd
2044-binary-indep: install
2045- dh_installdocs
2046- dh_installchangelogs
2047- dh_installinit
2048- dh_installcron
2049- dh_strip
2050- dh_compress
2051- dh_fixperms
2052- dh_installdeb
2053- dh_gencontrol
2054- dh_md5sums
2055- dh_builddeb
2056-
2057-binary: binary-indep
2058-
2059-.PHONY: binary binary-indep binary-arch install clean build
2060-
2061-clean:
2062- dh_clean
2063-
2064-prepare:
2065-
2066-package: prepare
2067- debuild -uc -us -S
2068-
2069-build:
2070- @echo Mmm builders
2071
2072=== removed directory 'lib/canonical/buildd/debian/source'
2073=== removed file 'lib/canonical/buildd/debian/source/format'
2074--- lib/canonical/buildd/debian/source/format 2010-07-22 17:50:07 +0000
2075+++ lib/canonical/buildd/debian/source/format 1970-01-01 00:00:00 +0000
2076@@ -1,1 +0,0 @@
2077-1.0
2078
2079=== removed file 'lib/canonical/buildd/debian/upgrade-config'
2080--- lib/canonical/buildd/debian/upgrade-config 2010-06-12 03:48:34 +0000
2081+++ lib/canonical/buildd/debian/upgrade-config 1970-01-01 00:00:00 +0000
2082@@ -1,122 +0,0 @@
2083-#!/usr/bin/python
2084-#
2085-# Copyright 2009, 2010 Canonical Ltd. This software is licensed under the
2086-# GNU Affero General Public License version 3 (see the file LICENSE).
2087-
2088-"""Upgrade a launchpad-buildd configuration file."""
2089-
2090-import sys
2091-import os
2092-import subprocess
2093-
2094-(old_version, conf_file) = sys.argv[1:]
2095-
2096-def upgrade_to_12():
2097- print "Upgrading %s to version 12" % conf_file
2098- subprocess.call(["mv", conf_file, conf_file+"-prev12~"])
2099- in_file = open(conf_file+"-prev12~", "r")
2100- out_file = open(conf_file, "w")
2101- for line in in_file:
2102- if line.startswith("[debianmanager]"):
2103- line += "ogrepath = /usr/share/launchpad-buildd/slavebin/apply-ogre-model\n"
2104- if line.startswith("sbuildargs"):
2105- line = line.replace("-A", "")
2106- out_file.write(line)
2107- in_file.close()
2108- out_file.close()
2109-
2110-def upgrade_to_34():
2111- print "Upgrading %s to version 34" % conf_file
2112- subprocess.call(["mv", conf_file, conf_file+"-prev34~"])
2113- in_file = open(conf_file+"-prev34~", "r")
2114- out_file = open(conf_file, "w")
2115- for line in in_file:
2116- if line.startswith("[debianmanager]"):
2117- line += "sourcespath = /usr/share/launchpad-buildd/slavebin/override-sources-list\n"
2118- out_file.write(line)
2119- in_file.close()
2120- out_file.close()
2121-
2122-def upgrade_to_39():
2123- print "Upgrading %s to version 39" % conf_file
2124- subprocess.call(["mv", conf_file, conf_file+"-prev39~"])
2125- in_file = open(conf_file+"-prev39~", "r")
2126- out_file = open(conf_file, "w")
2127- for line in in_file:
2128- if line.startswith("sbuildargs"):
2129- line = line.replace("-dautobuild ","")
2130- if line.startswith("[slave]"):
2131- line += "ntphost = ntp.buildd\n"
2132- out_file.write(line)
2133- in_file.close()
2134- out_file.close()
2135-
2136-def upgrade_to_57():
2137- print "Upgrading %s to version 57" % conf_file
2138- subprocess.call(["mv", conf_file, conf_file+"-prev57~"])
2139- in_file = open(conf_file+"-prev57~", "r")
2140- out_file = open(conf_file, "w")
2141- # We want to move all the sbuild lines to a new
2142- # 'binarypackagemanager' section at the end.
2143- binarypackage_lines = []
2144- for line in in_file:
2145- if line.startswith("sbuild"):
2146- binarypackage_lines.append(line)
2147- else:
2148- out_file.write(line)
2149- out_file.write('[binarypackagemanager]\n')
2150- for line in binarypackage_lines:
2151- out_file.write(line)
2152- in_file.close()
2153- out_file.close()
2154-
2155-def upgrade_to_58():
2156- print "Upgrading %s to version 58" % conf_file
2157- subprocess.call(["mv", conf_file, conf_file+"-prev58~"])
2158- in_file = open(conf_file+"-prev58~", "r")
2159- out_file = open(conf_file, "w")
2160- out_file.write(in_file.read())
2161- out_file.write(
2162- '\n[sourcepackagerecipemanager]\n'
2163- 'buildrecipepath = /usr/share/launchpad-buildd'
2164- '/slavebin/buildrecipe\n')
2165-
2166-def upgrade_to_59():
2167- print "Upgrading %s to version 59" % conf_file
2168- subprocess.call(["mv", conf_file, conf_file+"-prev59~"])
2169- in_file = open(conf_file+"-prev59~", "r")
2170- out_file = open(conf_file, "w")
2171- out_file.write(in_file.read())
2172- out_file.write(
2173- '\n[translationtemplatesmanager]\n'
2174- 'generatepath = /usr/share/launchpad-buildd/slavebin/generate-translation-templates\n'
2175- 'resultarchive = translation-templates.tar.gz\n')
2176-
2177-def upgrade_to_63():
2178- print "Upgrading %s to version 63" % conf_file
2179- subprocess.call(["mv", conf_file, conf_file+"-prev63~"])
2180- in_file = open(conf_file+"-prev63~", "r")
2181- out_file = open(conf_file, "w")
2182- for line in in_file:
2183- if not line.startswith('ogrepath'):
2184- out_file.write(line)
2185-
2186-
2187-if __name__ == "__main__":
2188- if old_version.find("~") > 0:
2189- old_version = old_version[:old_version.find("~")]
2190- if int(old_version) < 12:
2191- upgrade_to_12()
2192- if int(old_version) < 34:
2193- upgrade_to_34()
2194- if int(old_version) < 39:
2195- upgrade_to_39()
2196- if int(old_version) < 57:
2197- upgrade_to_57()
2198- if int(old_version) < 58:
2199- upgrade_to_58()
2200- if int(old_version) < 59:
2201- upgrade_to_59()
2202- if int(old_version) < 63:
2203- upgrade_to_63()
2204-
2205
2206=== removed file 'lib/canonical/buildd/generate-translation-templates'
2207--- lib/canonical/buildd/generate-translation-templates 2011-10-17 14:55:49 +0000
2208+++ lib/canonical/buildd/generate-translation-templates 1970-01-01 00:00:00 +0000
2209@@ -1,66 +0,0 @@
2210-#!/bin/sh
2211-#
2212-# Copyright 2010 Canonical Ltd. This software is licensed under the
2213-# GNU Affero General Public License version 3 (see the file LICENSE).
2214-
2215-# Buildd Slave tool to generate translation templates. Boiler plate code
2216-# copied from sbuild-package.
2217-
2218-# Expects build id as arg 1.
2219-# Expects branch url as arg 2.
2220-# Expects output tarball name as arg 3.
2221-
2222-# Must run as user with password-less sudo ability.
2223-
2224-exec 2>&1
2225-
2226-export LANG=C LC_ALL=C
2227-
2228-CHMOD=/bin/chmod
2229-CHROOT=/usr/sbin/chroot
2230-CP=/bin/cp
2231-INSTALL=/usr/bin/install
2232-MKDIR=/bin/mkdir
2233-SU=/bin/su
2234-SUDO=/usr/bin/sudo
2235-TOUCH=/usr/bin/touch
2236-
2237-BUILDID=$1
2238-BRANCH_URL=$2
2239-RESULT_NAME=$3
2240-
2241-BUILDD_HOME=/usr/share/launchpad-buildd
2242-SLAVEBIN=$BUILDD_HOME/slavebin
2243-BUILD_CHROOT="$HOME/build-$BUILDID/chroot-autobuild"
2244-USER=$(whoami)
2245-
2246-# Debug output.
2247-echo "Running as $USER for build $BUILDID on $BRANCH_URL."
2248-echo "Results expected in $RESULT_NAME."
2249-
2250-BUILDD_PACKAGE=canonical/buildd
2251-POTTERY=$BUILDD_PACKAGE/pottery
2252-# The script should be smarter about detecting the python version.
2253-PYMODULES=/usr/lib/pymodules/python2.7
2254-echo -n "Default Python in the chroot is: "
2255-$BUILD_CHROOT/usr/bin/python --version
2256-
2257-GENERATE_SCRIPT=$PYMODULES/$POTTERY/generate_translation_templates.py
2258-
2259-debug_exec() {
2260- echo "Executing '$1'..."
2261- $1 || echo "Got error $? from '$1'."
2262-}
2263-
2264-# Copy pottery files to chroot.
2265-debug_exec "$SUDO $MKDIR -vp $BUILD_CHROOT$PYMODULES/$BUILDD_PACKAGE"
2266-debug_exec "$SUDO $TOUCH $BUILD_CHROOT$PYMODULES/canonical/__init__.py"
2267-debug_exec "$SUDO $TOUCH $BUILD_CHROOT$PYMODULES/canonical/buildd/__init__.py"
2268-debug_exec "$SUDO $CP -vr $BUILDD_HOME/$POTTERY $BUILD_CHROOT$PYMODULES/$BUILDD_PACKAGE"
2269-debug_exec "$SUDO $CHMOD -v -R go+rX $BUILD_CHROOT$PYMODULES/canonical"
2270-debug_exec "$SUDO $CHMOD -v 755 $BUILD_CHROOT$GENERATE_SCRIPT"
2271-
2272-# Enter chroot, switch back to unprivileged user, execute the generate script.
2273-$SUDO $CHROOT $BUILD_CHROOT \
2274- $SU - $USER \
2275- -c "PYTHONPATH=$PYMODULES $GENERATE_SCRIPT $BRANCH_URL $RESULT_NAME"
2276
2277=== removed directory 'lib/canonical/buildd/launchpad-files'
2278=== removed file 'lib/canonical/buildd/mount-chroot'
2279--- lib/canonical/buildd/mount-chroot 2010-12-21 17:52:32 +0000
2280+++ lib/canonical/buildd/mount-chroot 1970-01-01 00:00:00 +0000
2281@@ -1,25 +0,0 @@
2282-#!/bin/sh
2283-#
2284-# Copyright 2009 Canonical Ltd. This software is licensed under the
2285-# GNU Affero General Public License version 3 (see the file LICENSE).
2286-
2287-# Buildd Slave tool to mount a chroot
2288-
2289-# Expects build id as arg 1, makes build-id to contain the build
2290-
2291-# Needs SUDO to be set to a sudo instance for passwordless access
2292-
2293-SUDO=/usr/bin/sudo
2294-BUILDID="$1"
2295-
2296-set -e
2297-
2298-exec 2>&1
2299-
2300-echo "Mounting chroot for build $BUILDID"
2301-
2302-$SUDO mount -t proc none "$HOME/build-$BUILDID/chroot-autobuild/proc"
2303-$SUDO mount -t devpts none "$HOME/build-$BUILDID/chroot-autobuild/dev/pts"
2304-$SUDO mount -t sysfs none "$HOME/build-$BUILDID/chroot-autobuild/sys"
2305-$SUDO mount -t tmpfs none "$HOME/build-$BUILDID/chroot-autobuild/dev/shm"
2306-$SUDO cp /etc/hosts /etc/hostname /etc/resolv.conf $HOME/build-$BUILDID/chroot-autobuild/etc/
2307
2308=== removed file 'lib/canonical/buildd/override-sources-list'
2309--- lib/canonical/buildd/override-sources-list 2010-12-21 17:52:32 +0000
2310+++ lib/canonical/buildd/override-sources-list 1970-01-01 00:00:00 +0000
2311@@ -1,31 +0,0 @@
2312-#!/bin/sh
2313-#
2314-# Copyright 2009 Canonical Ltd. This software is licensed under the
2315-# GNU Affero General Public License version 3 (see the file LICENSE).
2316-
2317-# Buildd Slave tool to override sources.list in the chroot with a list of
2318-# archives
2319-
2320-# Expects build id as arg 1
2321-# Expects sources.list lines as subsequent args
2322-
2323-# Needs SUDO to be set to a sudo instance for passwordless access
2324-
2325-set -e
2326-exec 2>&1
2327-
2328-SUDO=/usr/bin/sudo
2329-
2330-BUILDID="$1"
2331-shift
2332-
2333-cd $HOME
2334-cd "build-$BUILDID/chroot-autobuild/etc/apt"
2335-
2336-echo "Overriding sources.list in build-$BUILDID"
2337-
2338-$SUDO rm -f sources.list.new
2339-(for archive; do
2340- echo "$archive"
2341-done) | $SUDO tee sources.list.new >/dev/null
2342-$SUDO mv sources.list.new sources.list
2343
2344=== removed directory 'lib/canonical/buildd/pottery'
2345=== removed file 'lib/canonical/buildd/pottery/__init__.py'
2346=== removed file 'lib/canonical/buildd/pottery/generate_translation_templates.py'
2347--- lib/canonical/buildd/pottery/generate_translation_templates.py 2010-12-24 11:42:15 +0000
2348+++ lib/canonical/buildd/pottery/generate_translation_templates.py 1970-01-01 00:00:00 +0000
2349@@ -1,114 +0,0 @@
2350-#! /usr/bin/python
2351-# Copyright 2010 Canonical Ltd. This software is licensed under the
2352-# GNU Affero General Public License version 3 (see the file LICENSE).
2353-
2354-__metaclass__ = type
2355-
2356-import os.path
2357-import sys
2358-import tarfile
2359-
2360-import logging
2361-
2362-from bzrlib.branch import Branch
2363-from bzrlib.export import export
2364-
2365-from canonical.buildd.pottery import intltool
2366-
2367-
2368-class GenerateTranslationTemplates:
2369- """Script to generate translation templates from a branch."""
2370-
2371- def __init__(self, branch_spec, result_name, work_dir, log_file=None):
2372- """Prepare to generate templates for a branch.
2373-
2374- :param branch_spec: Either a branch URL or the path of a local
2375- branch. URLs are recognized by the occurrence of ':'. In
2376- the case of a URL, this will make up a path for the branch
2377- and check out the branch to there.
2378- :param result_name: The name of the result tarball. Should end in
2379- .tar.gz.
2380- :param work_dir: The directory to work in. Must exist.
2381- :param log_file: File-like object to log to. If None, defaults to
2382- stderr.
2383- """
2384- self.work_dir = work_dir
2385- self.branch_spec = branch_spec
2386- self.result_name = result_name
2387- self.logger = self._setupLogger(log_file)
2388-
2389- def _setupLogger(self, log_file):
2390- """Sets up and returns a logger."""
2391- if log_file is None:
2392- log_file = sys.stderr
2393- logger = logging.getLogger("generate-templates")
2394- logger.setLevel(logging.DEBUG)
2395- ch = logging.StreamHandler(log_file)
2396- ch.setLevel(logging.DEBUG)
2397- logger.addHandler(ch)
2398- return logger
2399-
2400- def _getBranch(self):
2401- """Set `self.branch_dir`, and check out branch if needed."""
2402- if ':' in self.branch_spec:
2403- # This is a branch URL. Check out the branch.
2404- self.branch_dir = os.path.join(self.work_dir, 'source-tree')
2405- self.logger.info("Getting remote branch %s..." % self.branch_spec)
2406- self._checkout(self.branch_spec)
2407- else:
2408- # This is a local filesystem path. Use the branch in-place.
2409- self.logger.info("Using local branch %s..." % self.branch_spec)
2410- self.branch_dir = self.branch_spec
2411-
2412- def _checkout(self, branch_url):
2413- """Check out a source branch to generate from.
2414-
2415- The branch is checked out to the location specified by
2416- `self.branch_dir`.
2417- """
2418- self.logger.info("Opening branch %s..." % branch_url)
2419- branch = Branch.open(branch_url)
2420- self.logger.info("Getting branch revision tree...")
2421- rev_tree = branch.basis_tree()
2422- self.logger.info("Exporting branch to %s..." % self.branch_dir)
2423- export(rev_tree, self.branch_dir)
2424- self.logger.info("Exporting branch done.")
2425-
2426- def _makeTarball(self, files):
2427- """Put the given files into a tarball in the working directory."""
2428- tarname = os.path.join(self.work_dir, self.result_name)
2429- self.logger.info("Making tarball with templates in %s..." % tarname)
2430- tarball = tarfile.open(tarname, 'w|gz')
2431- files = [name for name in files if not name.endswith('/')]
2432- for path in files:
2433- full_path = os.path.join(self.branch_dir, path)
2434- self.logger.info("Adding template %s..." % full_path)
2435- tarball.add(full_path, path)
2436- tarball.close()
2437- self.logger.info("Tarball generated.")
2438-
2439- def generate(self):
2440- """Do It. Generate templates."""
2441- self.logger.info("Generating templates for %s." % self.branch_spec)
2442- self._getBranch()
2443- pots = intltool.generate_pots(self.branch_dir)
2444- self.logger.info("Generated %d templates." % len(pots))
2445- if len(pots) > 0:
2446- self._makeTarball(pots)
2447- return 0
2448-
2449-
2450-if __name__ == '__main__':
2451- if len(sys.argv) < 3:
2452- print "Usage: %s branch resultname [workdir]" % sys.argv[0]
2453- print " 'branch' is a branch URL or directory."
2454- print " 'resultname' is the name of the result tarball."
2455- print " 'workdir' is a directory, defaults to HOME."
2456- sys.exit(1)
2457- if len(sys.argv) == 4:
2458- workdir = sys.argv[3]
2459- else:
2460- workdir = os.environ['HOME']
2461- script = GenerateTranslationTemplates(
2462- sys.argv[1], sys.argv[2], workdir)
2463- sys.exit(script.generate())
2464
2465=== removed file 'lib/canonical/buildd/pottery/intltool.py'
2466--- lib/canonical/buildd/pottery/intltool.py 2011-04-13 09:53:01 +0000
2467+++ lib/canonical/buildd/pottery/intltool.py 1970-01-01 00:00:00 +0000
2468@@ -1,342 +0,0 @@
2469-# Copyright 2009-2010 Canonical Ltd. This software is licensed under the
2470-# GNU Affero General Public License version 3 (see the file LICENSE).
2471-
2472-"""Functions to build PO templates on the build slave."""
2473-
2474-__metaclass__ = type
2475-__all__ = [
2476- 'check_potfiles_in',
2477- 'generate_pot',
2478- 'generate_pots',
2479- 'get_translation_domain',
2480- 'find_intltool_dirs',
2481- 'find_potfiles_in',
2482- ]
2483-
2484-from contextlib import contextmanager
2485-import errno
2486-import os.path
2487-import re
2488-from subprocess import call
2489-
2490-
2491-def find_potfiles_in():
2492- """Search the current directory and its subdirectories for POTFILES.in.
2493-
2494- :returns: A list of names of directories that contain a file POTFILES.in.
2495- """
2496- result_dirs = []
2497- for dirpath, dirnames, dirfiles in os.walk("."):
2498- if "POTFILES.in" in dirfiles:
2499- result_dirs.append(dirpath)
2500- return result_dirs
2501-
2502-
2503-def check_potfiles_in(path):
2504- """Check if the files listed in the POTFILES.in file exist.
2505-
2506- Running 'intltool-update -m' will perform this check and also take a
2507- possible POTFILES.skip into account. It stores details about 'missing'
2508- (files that should be in POTFILES.in) and 'notexist'ing files (files
2509- that are listed in POTFILES.in but don't exist) in files which are
2510- named accordingly. These files are removed before the run.
2511-
2512- We don't care about files missing from POTFILES.in but want to know if
2513- all listed files exist. The presence of the 'notexist' file tells us
2514- that.
2515-
2516- :param path: The directory where POTFILES.in resides.
2517- :returns: False if the directory does not exist, if an error occurred
2518- when executing intltool-update or if files are missing from
2519- POTFILES.in. True if all went fine and all files in POTFILES.in
2520- actually exist.
2521- """
2522- current_path = os.getcwd()
2523-
2524- try:
2525- os.chdir(path)
2526- except OSError, e:
2527- # Abort nicely if the directory does not exist.
2528- if e.errno == errno.ENOENT:
2529- return False
2530- raise
2531- try:
2532- # Remove stale files from a previous run of intltool-update -m.
2533- for unlink_name in ['missing', 'notexist']:
2534- try:
2535- os.unlink(unlink_name)
2536- except OSError, e:
2537- # It's ok if the files are missing.
2538- if e.errno != errno.ENOENT:
2539- raise
2540- devnull = open("/dev/null", "w")
2541- returncode = call(
2542- ["/usr/bin/intltool-update", "-m"],
2543- stdout=devnull, stderr=devnull)
2544- devnull.close()
2545- finally:
2546- os.chdir(current_path)
2547-
2548- if returncode != 0:
2549- # An error occurred when executing intltool-update.
2550- return False
2551-
2552- notexist = os.path.join(path, "notexist")
2553- return not os.access(notexist, os.R_OK)
2554-
2555-
2556-def find_intltool_dirs():
2557- """Search for directories with intltool structure.
2558-
2559- The current directory and its subdiretories are searched. An 'intltool
2560- structure' is a directory that contains a POFILES.in file and where all
2561- files listed in that POTFILES.in do actually exist. The latter
2562- condition makes sure that the file is not stale.
2563-
2564- :returns: A list of directory names.
2565- """
2566- return sorted(filter(check_potfiles_in, find_potfiles_in()))
2567-
2568-
2569-def _get_AC_PACKAGE_NAME(config_file):
2570- """Get the value of AC_PACKAGE_NAME from function parameters.
2571-
2572- The value of AC_PACKAGE_NAME is either the first or the fourth
2573- parameter of the AC_INIT call if it is called with at least two
2574- parameters.
2575- """
2576- params = config_file.getFunctionParams("AC_INIT")
2577- if params is None or len(params) < 2:
2578- return None
2579- if len(params) < 4:
2580- return params[0]
2581- else:
2582- return params[3]
2583-
2584-
2585-def _try_substitution(config_files, varname, substitution):
2586- """Try to find a substitution in the config files.
2587-
2588- :returns: The completed substitution or None if none was found.
2589- """
2590- subst_value = None
2591- if varname == substitution.name:
2592- # Do not look for the same name in the current file.
2593- config_files = config_files[:-1]
2594- for config_file in reversed(config_files):
2595- subst_value = config_file.getVariable(substitution.name)
2596- if subst_value is not None:
2597- # Substitution found.
2598- break
2599- else:
2600- # No substitution found.
2601- return None
2602- return substitution.replace(subst_value)
2603-
2604-
2605-def get_translation_domain(dirname):
2606- """Get the translation domain for this PO directory.
2607-
2608- Imitates some of the behavior of intltool-update to find out which
2609- translation domain the build environment provides. The domain is usually
2610- defined in the GETTEXT_PACKAGE variable in one of the build files. Another
2611- variant is DOMAIN in the Makevars file. This function goes through the
2612- ordered list of these possible locations, top to bottom, and tries to
2613- find a valid value. Since the same variable name may be defined in
2614- multiple files (usually configure.ac and Makefile.in.in), it needs to
2615- keep trying with the next file, until it finds the most specific
2616- definition.
2617-
2618- If the found value contains a substitution, either autoconf style (@...@)
2619- or make style ($(...)), the search is continued in the same file and back
2620- up the list of files, now searching for the substitution. Multiple
2621- substitutions or multi-level substitutions are not supported.
2622- """
2623- locations = [
2624- ('../configure.ac', 'GETTEXT_PACKAGE', True),
2625- ('../configure.in', 'GETTEXT_PACKAGE', True),
2626- ('Makefile.in.in', 'GETTEXT_PACKAGE', False),
2627- ('Makevars', 'DOMAIN', False),
2628- ]
2629- value = None
2630- substitution = None
2631- config_files = []
2632- for filename, varname, keep_trying in locations:
2633- path = os.path.join(dirname, filename)
2634- if not os.access(path, os.R_OK):
2635- # Skip non-existent files.
2636- continue
2637- config_files.append(ConfigFile(path))
2638- new_value = config_files[-1].getVariable(varname)
2639- if new_value is not None:
2640- value = new_value
2641- if value == "AC_PACKAGE_NAME":
2642- value = _get_AC_PACKAGE_NAME(config_files[-1])
2643- else:
2644- # Check if the value needs a substitution.
2645- substitution = Substitution.get(value)
2646- if substitution is not None:
2647- # Try to substitute with value.
2648- value = _try_substitution(
2649- config_files, varname, substitution)
2650- if value is None:
2651- # No substitution found; the setup is broken.
2652- break
2653- if value is not None and not keep_trying:
2654- # A value has been found.
2655- break
2656- return value
2657-
2658-
2659-@contextmanager
2660-def chdir(directory):
2661- cwd = os.getcwd()
2662- os.chdir(directory)
2663- yield
2664- os.chdir(cwd)
2665-
2666-
2667-def generate_pot(podir, domain):
2668- """Generate one PO template using intltool.
2669-
2670- Although 'intltool-update -p' can try to find out the translation domain
2671- we trust our own code more on this one and simply specify the domain.
2672- Also, the man page for 'intltool-update' states that the '-g' option
2673- "has an additional effect: the name of current working directory is no
2674- more limited to 'po' or 'po-*'." We don't want that limit either.
2675-
2676- :param podir: The PO directory in which to build template.
2677- :param domain: The translation domain to use as the name of the template.
2678- If it is None or empty, 'messages.pot' will be used.
2679- :return: True if generation succeeded.
2680- """
2681- if domain is None or domain.strip() == "":
2682- domain = "messages"
2683- with chdir(podir):
2684- with open("/dev/null", "w") as devnull:
2685- returncode = call(
2686- ["/usr/bin/intltool-update", "-p", "-g", domain],
2687- stdout=devnull, stderr=devnull)
2688- return returncode == 0
2689-
2690-
2691-def generate_pots(package_dir='.'):
2692- """Top-level function to generate all PO templates in a package."""
2693- potpaths = []
2694- with chdir(package_dir):
2695- for podir in find_intltool_dirs():
2696- domain = get_translation_domain(podir)
2697- if generate_pot(podir, domain):
2698- potpaths.append(os.path.join(podir, domain + ".pot"))
2699- return potpaths
2700-
2701-
2702-class ConfigFile(object):
2703- """Represent a config file and return variables defined in it."""
2704-
2705- def __init__(self, file_or_name):
2706- if isinstance(file_or_name, basestring):
2707- conf_file = file(file_or_name)
2708- else:
2709- conf_file = file_or_name
2710- self.content = conf_file.read()
2711-
2712- def _stripQuotes(self, identifier):
2713- """Strip surrounding quotes from `identifier`, if present.
2714-
2715- :param identifier: a string, possibly surrounded by matching
2716- 'single,' "double," or [bracket] quotes.
2717- :return: `identifier` but with the outer pair of matching quotes
2718- removed, if they were there.
2719- """
2720- if len(identifier) < 2:
2721- return identifier
2722-
2723- quote_pairs = [
2724- ('"', '"'),
2725- ("'", "'"),
2726- ("[", "]"),
2727- ]
2728- for (left, right) in quote_pairs:
2729- if identifier.startswith(left) and identifier.endswith(right):
2730- return identifier[1:-1]
2731-
2732- return identifier
2733-
2734- def getVariable(self, name):
2735- """Search the file for a variable definition with this name."""
2736- pattern = re.compile(
2737- "^%s[ \t]*=[ \t]*([^\s]*)" % re.escape(name), re.M)
2738- result = pattern.search(self.content)
2739- if result is None:
2740- return None
2741- return self._stripQuotes(result.group(1))
2742-
2743- def getFunctionParams(self, name):
2744- """Search file for a function call with this name, return parameters.
2745- """
2746- pattern = re.compile("^%s\(([^)]*)\)" % re.escape(name), re.M)
2747- result = pattern.search(self.content)
2748- if result is None:
2749- return None
2750- else:
2751- return [
2752- self._stripQuotes(param.strip())
2753- for param in result.group(1).split(',')
2754- ]
2755-
2756-
2757-class Substitution(object):
2758- """Find and replace substitutions.
2759-
2760- Variable texts may contain other variables which should be substituted
2761- for their value. These are either marked by surrounding @ signs (autoconf
2762- style) or preceded by a $ sign with optional () (make style).
2763-
2764- This class identifies a single such substitution in a variable text and
2765- extract the name of the variable who's value is to be inserted. It also
2766- facilitates the actual replacement so that caller does not have to worry
2767- about the substitution style that is being used.
2768- """
2769-
2770- autoconf_pattern = re.compile("@([^@]+)@")
2771- makefile_pattern = re.compile("\$\(?([^\s\)]+)\)?")
2772-
2773- @staticmethod
2774- def get(variabletext):
2775- """Factory method.
2776-
2777- Creates a Substitution instance and checks if it found a substitution.
2778-
2779- :param variabletext: A variable value with possible substitution.
2780- :returns: A Substitution object or None if no substitution was found.
2781- """
2782- subst = Substitution(variabletext)
2783- if subst.name is not None:
2784- return subst
2785- return None
2786-
2787- def _searchForPatterns(self):
2788- """Search for all the available patterns in variable text."""
2789- result = self.autoconf_pattern.search(self.text)
2790- if result is None:
2791- result = self.makefile_pattern.search(self.text)
2792- return result
2793-
2794- def __init__(self, variabletext):
2795- """Extract substitution name from variable text."""
2796- self.text = variabletext
2797- self.replaced = False
2798- result = self._searchForPatterns()
2799- if result is None:
2800- self._replacement = None
2801- self.name = None
2802- else:
2803- self._replacement = result.group(0)
2804- self.name = result.group(1)
2805-
2806- def replace(self, value):
2807- """Return a copy of the variable text with the substitution resolved.
2808- """
2809- self.replaced = True
2810- return self.text.replace(self._replacement, value)
2811
2812=== removed file 'lib/canonical/buildd/remove-build'
2813--- lib/canonical/buildd/remove-build 2010-12-21 17:52:32 +0000
2814+++ lib/canonical/buildd/remove-build 1970-01-01 00:00:00 +0000
2815@@ -1,25 +0,0 @@
2816-#!/bin/sh
2817-#
2818-# Copyright 2009 Canonical Ltd. This software is licensed under the
2819-# GNU Affero General Public License version 3 (see the file LICENSE).
2820-
2821-# Buildd Slave tool to remove an unmounted chroot
2822-
2823-# Expects build id as arg 1, makes build-id to contain the build
2824-
2825-# Needs RM to be set to a gnu rm instance
2826-# Needs SUDO to be set to a sudo instance for passwordless access
2827-
2828-RM=/bin/rm
2829-SUDO=/usr/bin/sudo
2830-BUILDID="$1"
2831-
2832-set -e
2833-
2834-exec 2>&1
2835-
2836-echo "Removing build $BUILDID"
2837-
2838-cd $HOME
2839-
2840-$SUDO $RM -rf "build-$BUILDID"
2841
2842=== removed file 'lib/canonical/buildd/sbuild'
2843--- lib/canonical/buildd/sbuild 2011-11-06 04:21:34 +0000
2844+++ lib/canonical/buildd/sbuild 1970-01-01 00:00:00 +0000
2845@@ -1,3658 +0,0 @@
2846-#! /usr/bin/perl
2847-#
2848-# sbuild: build packages, obeying source dependencies
2849-# Copyright (C) 1998-2000 Roman Hodek <Roman.Hodek@informatik.uni-erlangen.de>
2850-#
2851-# This program is free software; you can redistribute it and/or
2852-# modify it under the terms of the GNU General Public License as
2853-# published by the Free Software Foundation; either version 2 of the
2854-# License, or (at your option) any later version.
2855-#
2856-# This program is distributed in the hope that it will be useful, but
2857-# WITHOUT ANY WARRANTY; without even the implied warranty of
2858-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
2859-# General Public License for more details.
2860-#
2861-# You should have received a copy of the GNU General Public License
2862-# along with this program; if not, write to the Free Software
2863-# Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
2864-#
2865-# $Id: sbuild,v 1.170 2002/10/10 18:50:44 rnhodek Exp $
2866-#
2867-# $Log: sbuild,v $
2868-# Revision 1.170 2002/10/10 18:50:44 rnhodek
2869-# Security/accepted autobuilding patch by Ryan.
2870-#
2871-# Revision 1.169 2002/03/11 09:49:55 rnhodek
2872-# chroot_apt_options: don't just set Dir::Etc::sourcelist, but whole
2873-# Dir::Etc, to avoid new preferences.
2874-#
2875-# Revision 1.168 2002/02/07 09:12:25 rnhodek
2876-# Fixed sudo call for running scripts.
2877-#
2878-# Revision 1.167 2002/01/22 08:53:10 rnhodek
2879-# Use capitalized "Noninteractive" das DEBIAN_FRONTENT.
2880-#
2881-# Revision 1.166 2001/11/08 09:32:10 rnhodek
2882-# fetch_source_file: in case of apt, need to pass up a $dscfile with
2883-# epoch stripped.
2884-#
2885-# Revision 1.165 2001/11/07 18:04:31 rnhodek
2886-# Argl.. I should learn perl first :)
2887-#
2888-# Revision 1.164 2001/11/07 09:25:59 rnhodek
2889-# fetch_source_files: after apt-get, fix $dscfile not to contain epoch.
2890-#
2891-# Revision 1.163 2001/11/06 15:11:24 rnhodek
2892-# fetch_source_files: don't touch symlinks generated by apt-get.
2893-#
2894-# Revision 1.162 2001/11/06 12:29:22 rnhodek
2895-# New config var $use_FetchFile (default 0).
2896-# Integrated patches for using apt-get to download sources; modified so
2897-# that old code (using FetchFile.pm) still present.
2898-# run_apt: Fix parsing error of apt-output (\s matched ^M printed by
2899-# pre-configuring).
2900-#
2901-# Revision 1.161 2001/10/29 12:20:41 rnhodek
2902-# Added $fakeroot var.
2903-#
2904-# Revision 1.160 2001/10/29 08:27:53 rnhodek
2905-# Fix typo in message.
2906-#
2907-# Revision 1.159 2001/06/18 09:57:55 rnhodek
2908-# Use --database option when wanna_build_dbbase set in config.
2909-#
2910-# Revision 1.158 2001/06/01 09:54:36 rnhodek
2911-# Fix two typos.
2912-#
2913-# Revision 1.157 2001/05/31 08:08:08 rnhodek
2914-# Fixed thinko in conflicts checking (tnx James)
2915-#
2916-# Revision 1.156 2001/05/30 08:20:05 rnhodek
2917-# Use /var/lib/apt as Dir::State if exists (newer apt versions use that)
2918-# (tnx to Ryan Murray).
2919-#
2920-# Revision 1.155 2001/05/30 08:17:01 rnhodek
2921-# Print also architecture at start of log.
2922-#
2923-# Revision 1.154 2001/05/29 07:33:54 rnhodek
2924-# Also handle versioned Build-Conflicts.
2925-# (Tnx to David Schleef <ds@schleef.org>)
2926-#
2927-# Revision 1.153 2001/03/13 08:45:17 rnhodek
2928-# Run chmod -R on source tree only after unpacking.
2929-#
2930-# Revision 1.152 2001/02/19 09:23:24 rnhodek
2931-# Fix typo.
2932-#
2933-# Revision 1.151 2001/02/19 08:43:22 rnhodek
2934-# Fix wrong arch logic.
2935-#
2936-# Revision 1.150 2001/01/22 12:56:16 rnhodek
2937-# Fix handling of arch-restricted build-deps (tnx to Ryan Murray).
2938-#
2939-# Revision 1.149 2001/01/15 11:17:36 rnhodek
2940-# Fixed typo.
2941-#
2942-# Revision 1.148 2001/01/15 10:59:18 rnhodek
2943-# Shortcut when looking for non-dsc files: first try to fetch from same
2944-# dir as .dsc and FF_search only if that fails.
2945-#
2946-# Revision 1.147 2001/01/10 16:16:48 rnhodek
2947-# Move all built files from chroot dir, not only *.deb.
2948-#
2949-# Revision 1.146 2000/10/23 10:39:24 rnhodek
2950-# Before doing chmod on /etc/ld.so.conf, test if it exists at all.
2951-#
2952-# Revision 1.145 2000/10/19 09:08:35 rnhodek
2953-# A couple of little fixes from Ben.
2954-# Checks and assures that /etc/ld.so.conf is readable.
2955-# Support for local overrides with $conf::srcdep_over.
2956-#
2957-# Revision 1.144 2000/06/27 12:34:00 rnhodek
2958-# Implemented new 'prepre' and 'unpack' sections for special targets;
2959-# the first is a script run before package installation, and the latter
2960-# can list source packages that should be unpacked in the build dir.
2961-#
2962-# Revision 1.143 2000/06/20 14:39:59 rnhodek
2963-# Call apt-get with some options for relocating various files instead of
2964-# chrooting it; this avoids that the archive must be available in chroot
2965-# environment; same is true for dpkg and apt-cache calls.
2966-# If chrooted, call dpkg with the chroot dir as cwd to avoid "cannot get
2967-# current directory" errors; same for apt option DPkg::Run-Directory.
2968-#
2969-# Revision 1.142 2000/06/19 14:09:00 rnhodek
2970-# Fix syntax error.
2971-#
2972-# Revision 1.141 2000/06/19 14:05:38 rnhodek
2973-# Call buildd-addpkg with --chroot=DIR options for each dist that is >=
2974-# the one building for (to run apt-get update in the chroot
2975-# environments).
2976-#
2977-# Revision 1.140 2000/06/19 09:10:24 rnhodek
2978-# Obey new config var @ignore_watches_no_build_deps, i.e. don't flag
2979-# watches that are listed there if the package doesn't have build deps.
2980-# In check_watches(), strip $chroot_dir instead of $chroot_build_dir
2981-# from pathname.
2982-#
2983-# Revision 1.139 2000/06/13 10:54:43 rnhodek
2984-# Also execute special dependency scripts in chroot environment.
2985-#
2986-# Revision 1.138 2000/06/09 12:47:52 rnhodek
2987-# File .dsc filename for rbuilder (with URL).
2988-#
2989-# Revision 1.137 2000/06/09 09:15:21 rnhodek
2990-# Always install built package (if already) when building chroot; i.e.
2991-# the $system_level test is not necessary when chrooted.
2992-#
2993-# Revision 1.136 2000/06/09 08:20:52 rnhodek
2994-# Fixed su usage in sub build.
2995-#
2996-# Revision 1.135 2000/06/08 14:02:11 rnhodek
2997-# After changing to chroot dir, change back to be the normal user again
2998-# and start dpkg-buildpackage with -rsudo again; some packages require
2999-# that the build target is executed as non-root.
3000-#
3001-# Revision 1.134 2000/06/08 13:01:54 rnhodek
3002-# apt-cache calls need sudo, too, when using chroot.
3003-#
3004-# Revision 1.133 2000/06/08 09:13:31 rnhodek
3005-# Implemented chroot builds; there are a few new global variables
3006-# $main::chroot_*; major changes are in build, where the source tree is
3007-# unpacked somewhere else, dpkg-buildpackage called under chroot and
3008-# built packages are moved back again; also all apt-get and dpkg calls
3009-# are chroot-ed and /var/lib/dpkg/status is accessed from the chroot
3010-# environment; also watches are checked under the new root dir.
3011-#
3012-# Revision 1.132 2000/06/06 14:37:05 rnhodek
3013-# New option --source (-s): Also build source package, i.e. don't pass
3014-# -b or -B to dpkg-buildpackage.
3015-#
3016-# Revision 1.131 2000/05/30 15:41:34 rnhodek
3017-# Call buildd-addpkg with --dist option.
3018-# Install freshly built packages only if $conf::system_level >= $dist.
3019-#
3020-# Revision 1.130 2000/05/16 12:34:20 rnhodek
3021-# Insert a chmod -R go+rX on the build tree to make files readable; it
3022-# happens sometimes that files in a .orig.tar.gz have restrictive
3023-# permissions and this can be inconvenient.
3024-#
3025-# Revision 1.129 2000/03/01 14:43:34 rnhodek
3026-# Also match error message "dpkg: status database area is locked" from
3027-# apt and retry call later.
3028-#
3029-# Revision 1.128 2000/02/16 15:21:33 rnhodek
3030-# Fix a print message in merge_pkg_build_deps.
3031-#
3032-# Revision 1.127 2000/02/16 15:20:38 rnhodek
3033-# Print version number of sbuild in package log.
3034-#
3035-# Revision 1.126 2000/02/16 15:15:15 rnhodek
3036-# Fix regexp for finding !needs-no-XXX packages.
3037-# Move !needs-no-XXX from central deps to $main::additional_deps so that
3038-# they can be found by prepare_watches later.
3039-#
3040-# Revision 1.125 2000/02/15 14:40:35 rnhodek
3041-# Remove forgotten debugging code.
3042-#
3043-# Revision 1.124 2000/02/15 11:12:43 rnhodek
3044-# Expand virtual packages in package build dependencies for comparing
3045-# with central deps.
3046-#
3047-# Revision 1.123 2000/02/11 11:17:07 rnhodek
3048-# Do not activate watches for packages XXX if a negative dependency
3049-# needs-no-XXX exists (used to be just a comment, now really processed
3050-# by sbuild.)
3051-# Also do not activate watches for dependencies of pkg build deps.
3052-#
3053-# Revision 1.122 2000/02/09 15:57:25 rnhodek
3054-# In merge_pkg_build_deps, do not show warnings about missing
3055-# !this-package-does-not-exist or !needs-no-xxx dependencies.
3056-#
3057-# Revision 1.121 2000/02/04 14:04:18 rnhodek
3058-# Use --no-down-propagation.
3059-#
3060-# Revision 1.120 2000/02/01 12:05:56 rnhodek
3061-# In binNMU mode, a '_' was missing in the job name.
3062-#
3063-# Revision 1.119 2000/01/28 14:54:43 rnhodek
3064-# Accept abbrevs for distribution options (-ds, -df, -du) here, too.
3065-# New option --make-binNMU=entry.
3066-# New binNMU hack to modify debian/changelog; it will add a new entry
3067-# for the NMU version.
3068-# New helper function binNMU_version to generate a new version number.
3069-#
3070-# Revision 1.118 2000/01/13 14:32:30 rnhodek
3071-# For compiling on slink systems, pass the --force-confold option to
3072-# dpkg only for versions < 1.4.1.18 (that don't understand it yet).
3073-#
3074-# Revision 1.117 1999/12/17 13:49:50 rnhodek
3075-# Improved output about missing central deps: build-essential (act.
3076-# policy) and dependencies of pkg build deps are filtered out and
3077-# printed separately.
3078-# New functions cmp_dep_lists, read_build_essential,
3079-# expand_dependencies, and get_dependencies for the above.
3080-#
3081-# Revision 1.116 1999/12/17 11:04:43 rnhodek
3082-# When pkg build-deps were read from debian/sbuild-build-deps, a wrong
3083-# package name was used.
3084-#
3085-# Revision 1.115 1999/12/09 09:54:42 rnhodek
3086-# Again fixed a fatal typo...
3087-#
3088-# Revision 1.114 1999/12/08 12:33:16 rnhodek
3089-# merge_pkg_build_deps: Fix printing of overrides.
3090-#
3091-# Revision 1.113 1999/12/08 12:25:34 rnhodek
3092-# Special dependencies are implicitly overrides, i.e. are added to the
3093-# package-provided build deps.
3094-#
3095-# Revision 1.112 1999/12/08 11:31:38 rnhodek
3096-# get_dpkg_status: don't reset $res{$pkg}->{Installed} to 0 if $pkg is
3097-# provided.
3098-#
3099-# Revision 1.111 1999/12/08 10:37:33 rnhodek
3100-# Change parsing of .dsc file so that multi-line build dependencies are
3101-# allowed.
3102-# Make warning about missing central deps a bit bigger.
3103-#
3104-# Revision 1.110 1999/12/06 15:00:33 rnhodek
3105-# Fix comparison with old deps (must copy them, not only the reference).
3106-#
3107-# Revision 1.109 1999/12/06 08:35:53 rnhodek
3108-# Fixed typo.
3109-#
3110-# Revision 1.108 1999/12/03 09:58:16 rnhodek
3111-# If a pkg has its own build deps, compare them with the central ones
3112-# and report missing ones.
3113-#
3114-# Revision 1.107 1999/11/30 13:54:38 rnhodek
3115-# Print a message if build deps from the .dsc are used (to avoid confusion).
3116-# If a pkg has build deps, store them in debian/.sbuild-build-deps to
3117-# have them available when rebuilding later (no .dsc anymore); also
3118-# check for this file and read deps from it if building without a .dsc
3119-# in unpacked source.
3120-#
3121-# Revision 1.106 1999/11/15 12:30:15 rnhodek
3122-# merge_pkg_build_deps: added missing if $main::debug.
3123-#
3124-# Revision 1.105 1999/11/03 14:56:32 rnhodek
3125-# When running apt, set env var DEBIAN_FRONTEND to noninteractive to
3126-# stop debconf from asking questions or complaining that /dev/tty can't
3127-# be opened.
3128-#
3129-# Revision 1.104 1999/11/02 16:43:51 rnhodek
3130-# check_inst_packages: also upgrade dependencies of src-deps (if they're
3131-# already installed); some -dev packages fail to correctly require an
3132-# identical versioned shlib pkg, so in some cases only the -dev pkg was
3133-# installed.
3134-#
3135-# Revision 1.103 1999/11/02 15:45:43 rnhodek
3136-# build: Use epoch-stripped version number for the .changes file.
3137-# check_inst_packages: forgot a if $main::debug.
3138-#
3139-# Revision 1.102 1999/10/29 13:07:49 rnhodek
3140-# New option --stats-dir=DIR; if used, a "1" is appended to
3141-# DIR/give-back each time a package is given back.
3142-#
3143-# Revision 1.101 1999/10/29 12:32:24 rnhodek
3144-# If using an already unpacked source tree, check (with
3145-# dpkg-parsechangelog) if it's really the requested version.
3146-# Make apt-get run dpkg with --force-confold, as the </dev/null trick
3147-# doesn't work anymore with dpkg >= 1.4.1.18.
3148-#
3149-# Revision 1.100 1999/10/25 12:12:21 rnhodek
3150-# check_inst_packages: Add packages to @deps_inst only if they're not
3151-# already to be installed.
3152-#
3153-# Revision 1.99 1999/10/22 09:01:36 rnhodek
3154-# Minor changes to output of check_inst_packages.
3155-#
3156-# Revision 1.98 1999/10/21 14:21:57 rnhodek
3157-# Oops... call check_inst_packages only if build was successful.
3158-#
3159-# Revision 1.97 1999/10/21 11:46:50 rnhodek
3160-# Deleted RCS logs for < 1.50.
3161-# New option --store-built-packages.
3162-# Fix package name parsing: \w also matches '_' which is unwanted;
3163-# replace by a-zA-Z.
3164-# Read reverse sourcedeps of $main::store_built_packages.
3165-# New sub check_inst_packages.
3166-#
3167-# Revision 1.96 1999/09/27 11:18:10 rnhodek
3168-# Added a missing PLOG.
3169-#
3170-# Revision 1.95 1999/09/15 09:10:25 rnhodek
3171-# Additionally print a warning if a special dep has a version relation.
3172-#
3173-# Revision 1.94 1999/09/15 09:08:12 rnhodek
3174-# Changed parsing of dependencies a bit so that special deps can have
3175-# arch restrictions, too.
3176-#
3177-# Revision 1.93 1999/08/30 09:44:35 rnhodek
3178-# get_dpkg_status: don't exit too early if a pkg isn't in the arg list,
3179-# as it might be provided only.
3180-#
3181-# Revision 1.92 1999/08/27 13:32:04 rnhodek
3182-# --auto-give-back has a new optional argument, the user and hostname
3183-# where to call wanna-build (like $conf::sshcmd); this is needed that
3184-# sbuild can do give-backs when there's no local wanna-build.
3185-#
3186-# Revision 1.91 1999/08/23 12:53:02 rnhodek
3187-# Support for alternatives.
3188-# Support for [ARCH1 !ARCH2] arch restriction on dependencies.
3189-# Parses only src-deps which are needed for packages to be built.
3190-# Reads Build-{Depends,Conflicts}{,-Indep}: fields from .dsc if present;
3191-# those override the central src-deps, except those marked as override
3192-# (& prefix).
3193-# Implemented abbrevs as kind of macros in src-deps.
3194-# New option --add-depends (-a).
3195-# New option --arch-all (-A).
3196-#
3197-# Revision 1.90 1999/08/11 15:28:11 rnhodek
3198-# Insert missing wait call in run_script to get correct return value.
3199-#
3200-# Revision 1.89 1999/08/10 14:01:49 rnhodek
3201-# Virtual packages as dependencies didn't work really yet -- the
3202-# consistency check didn't see them (dpkg --status doesn't know them)
3203-# and thus aborted the build; solution: get_dpkg_status now directly
3204-# reads the status file (which should be a bit faster, too) and extracts
3205-# Provides: fields of all installed packages and considers those virtual
3206-# packages installed, too.
3207-# Print "Source-dependencies not satisfied" message to package log, not
3208-# to sbuild log.
3209-# Same in run_apt for virtual package handling.
3210-# Fix stdout/stderr redirecting when running scripts.
3211-#
3212-# Revision 1.88 1999/07/13 07:23:55 rnhodek
3213-# Use GDBM for time/space databases, as perl-5.004 seems not to contain
3214-# DB_File anymore.
3215-#
3216-# Revision 1.87 1999/06/21 12:52:00 rnhodek
3217-# Seems apt has a new error message if a cached Packages file isn't
3218-# up-to-date anymore -- recognize this msg, too, and reun apt-get update.
3219-#
3220-# Revision 1.86 1999/06/09 15:05:38 rnhodek
3221-# Fix loop in apply_patches.
3222-# Don't fail due to failed patch if a global patch.
3223-# Global patches are no syntax error when parsing src-deps...
3224-#
3225-# Revision 1.85 1999/06/04 09:47:02 rnhodek
3226-# Add support for global patches, which will be tried on any package;
3227-# their names in source-dependencies start with "**".
3228-#
3229-# Revision 1.84 1999/06/04 08:17:17 rnhodek
3230-# When calling wanna-build --give-back, don't forget the --dist argument!
3231-# Added support for virtual packages as source dependencies: apt-get tells us
3232-# which alternatives are possible, and one of these is selected either by
3233-# %conf::alternatives or by random.
3234-#
3235-# Revision 1.83 1999/06/02 09:07:47 rnhodek
3236-# With --batch, write each finished job to SBUILD-FINISHED; buildd can pick up
3237-# this file if sbuild crashes and needs not rebuild already done stuff. The file
3238-# is removed on normal exit and if sbuild dumps to REDO during a shutdown.
3239-#
3240-# Revision 1.82 1999/06/02 08:47:39 rnhodek
3241-# Remove as many die's as possible -- the bad exit status can cause
3242-# buildd to retry all packages of an sbuild run; better let this one
3243-# package fail.
3244-# Make sure that after build() we're in the correct directory: some
3245-# chdir()s were missing; also don't chdir("..") because this can be
3246-# wrong if we followed a symlink, use $main::cwd instead.
3247-# If the package directory already exists as a symlink, abort the build.
3248-#
3249-# Revision 1.81 1999/05/31 12:59:41 rnhodek
3250-# Run du after build under sudo, to avoid error messages about
3251-# unreadable dirs.
3252-#
3253-# Revision 1.80 1999/05/27 13:28:04 rnhodek
3254-# Oops, missed an epoch fix (when constructing the .changes file name).
3255-#
3256-# Revision 1.79 1999/05/26 11:34:11 rnhodek
3257-# Ignore epochs for fetching files.
3258-#
3259-# Revision 1.78 1999/05/26 09:48:23 rnhodek
3260-# If dpkg-source fails, remove .tmp-nest dir.
3261-#
3262-# Revision 1.77 1999/05/05 07:56:51 rnhodek
3263-# Need to empty %main::this_watches before filling it for a new package;
3264-# otherwise we have some spurious reports :-)
3265-#
3266-# Revision 1.76 1999/05/04 14:51:40 rnhodek
3267-# Some more minor stuff for avg-build-space: Reset global
3268-# $main::this_space to 0 before each build to avoid using the figure of
3269-# the previous package in case of errors; don't write a 0 value into the
3270-# database.
3271-#
3272-# Revision 1.75 1999/05/04 14:43:01 rnhodek
3273-# Fix parsing of a single dependency: package name never should contain
3274-# a '('.
3275-#
3276-# Revision 1.74 1999/05/04 14:29:51 rnhodek
3277-# Determine how much space is required for a build (final build dir +
3278-# generated .debs) after dpkg-buildpackage is finished; display figure
3279-# in package log and also store it in $conf::avg_space_db (analogous to
3280-# avg_time_db).
3281-#
3282-# Revision 1.73 1999/05/03 12:53:25 rnhodek
3283-# After unpacking src dir, run "chmod -R g-s ." on it; some
3284-# .orig.tar.gz's are packed with the setgid bit, which causes the debian
3285-# dir and all subdirs to be created setgid, too, and later dpkg-deb
3286-# --build complains about this.
3287-#
3288-# Revision 1.72 1999/04/22 14:16:25 rnhodek
3289-# Don't kill tee process if verbose but --nolog set -- $pkg_tee_pid
3290-# undefined then!
3291-#
3292-# Revision 1.71 1999/04/21 14:54:10 rnhodek
3293-# Implemented watches if certain binaries have been used during a build
3294-# without a source dependency.
3295-#
3296-# Revision 1.70 1999/03/12 10:29:32 rnhodek
3297-# New option --force-depends (-f) to override src-deps of a package.
3298-#
3299-
3300-BEGIN {
3301- ($main::HOME = $ENV{'HOME'})
3302- or die "HOME not defined in environment!\n";
3303- push( @INC, "$main::HOME/lib" );
3304-}
3305-
3306-chomp( $main::HOSTNAME = `hostname` );
3307-
3308-package conf;
3309-$HOME = $main::HOME;
3310-# defaults:
3311-@dist_parts = qw(main contrib non-free);
3312-$source_dependencies = "/etc/source-dependencies";
3313-$mailprog = "/usr/sbin/sendmail";
3314-$dpkg = "/usr/bin/dpkg";
3315-$sudo = "/usr/bin/sudo";
3316-$su = "/bin/su";
3317-$fakeroot = "/usr/bin/fakeroot";
3318-$apt_get = "/usr/bin/apt-get";
3319-$apt_cache = "/usr/bin/apt-cache";
3320-$dpkg_source = "/usr/bin/dpkg-source";
3321-$build_env_cmnd = "";
3322-$pgp_options = "-us -uc";
3323-$log_dir = "$main::HOME/logs";
3324-$mailto = "";
3325-$purge_build_directory = "successful";
3326-@toolchain_regex = ( 'binutils$', 'gcc-[\d.]+$', 'g\+\+-[\d.]+$', 'libstdc\+\+', 'libc[\d.]+-dev$', 'linux-kernel-headers$', 'dpkg-dev$', 'make$' );
3327-$stalled_pkg_timeout = 90; # minutes
3328-$srcdep_lock_wait = 1; # minutes
3329-%individual_stalled_pkg_timeout = ();
3330-$path = "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/X11R6/bin:/usr/games";
3331-# read conf files
3332-require "/etc/sbuild.conf" if -r "/etc/sbuild.conf";
3333-require "$HOME/.sbuildrc" if -r "$HOME/.sbuildrc";
3334-# some checks
3335-#die "mailprog binary $conf::mailprog does not exist or isn't executable\n"
3336-# if !-x $conf::mailprog;
3337-die "sudo binary $conf::sudo does not exist or isn't executable\n"
3338- if !-x $conf::sudo;
3339-die "apt-get binary $conf::apt_get does not exist or isn't executable\n"
3340- if !-x $conf::apt_get;
3341-die "apt-cache binary $conf::apt_cache does not exist or isn't executable\n"
3342- if !-x $conf::apt_cache;
3343-die "dpkg-source binary $conf::dpkg_source does not exist or isn't executable\n"
3344- if !-x $conf::dpkg_source;
3345-#die "$conf::log_dir is not a directory\n" if ! -d $conf::log_dir;
3346-die "$conf::srcdep_lock_dir is not a directory\n" if ! -d $conf::srcdep_lock_dir;
3347-die "conf::mailto not set\n" if !$conf::mailto;
3348-package main;
3349-
3350-use strict;
3351-use GDBM_File;
3352-use POSIX;
3353-use FileHandle;
3354-use Cwd;
3355-
3356-# avoid intermixing of stdout and stderr
3357-$| = 1;
3358-
3359-# We should not ignore HUP for our children - it breaks test suites.
3360-# in case the terminal disappears, the build should continue
3361-#$SIG{'HUP'} = 'IGNORE';
3362-
3363-$main::distribution = "";
3364-$main::distribution = $conf::default_distribution if $conf::default_distribution;
3365-
3366-chomp( $main::arch = `$conf::dpkg --print-architecture` );
3367-$main::username = (getpwuid($<))[0] || $ENV{'LOGNAME'} || $ENV{'USER'};
3368-$main::debug = 0;
3369-$main::verbose = 0;
3370-$main::batchmode = 0;
3371-$main::auto_giveback = 0;
3372-$main::nomail = 0;
3373-$main::build_arch_all = 0;
3374-$main::build_source = 0;
3375-$main::jobs_file = cwd() . "/build-progress";
3376-$main::max_lock_trys = 120;
3377-$main::lock_interval = 5;
3378-$main::cwd = cwd();
3379-$main::ilock_file = "$conf::srcdep_lock_dir/install";
3380-$main::srcdep_lock_cnt = 0;
3381-$main::chroot_dir = "";
3382-$main::chroot_build_dir = "";
3383-$main::chroot_apt_options = "";
3384-@main::toolchain_pkgs = ();
3385-$main::component="";
3386-$main::nr_processors = $ENV{'NR_PROCESSORS'} if $ENV{'NR_PROCESSORS'};
3387-
3388-umask(022);
3389-
3390-$main::new_dpkg = 0;
3391-check_dpkg_version();
3392-
3393-while( @ARGV && $ARGV[0] =~ /^-/ ) {
3394- $_ = shift @ARGV;
3395- if (/^-v$/ || /^--verbose$/) {
3396- $main::verbose++;
3397- }
3398- elsif (/^-D$/ || /^--debug$/) {
3399- $main::debug++;
3400- }
3401- elsif (/^-b$/ || /^--batch$/) {
3402- $main::batchmode = 1;
3403- }
3404- elsif (/^-n$/ || /^--nolog$/) {
3405- $main::nolog = 1;
3406- }
3407- elsif (/^-A$/ || /^--arch-all$/) {
3408- $main::build_arch_all++;
3409- }
3410- elsif (/^-s$/ || /^--source$/) {
3411- $main::build_source++;
3412- $conf::purge_build_directory = "never";
3413- }
3414- elsif (/^--architecture=(.)/) {
3415- $conf::arch=$1.$';
3416- }
3417- elsif (/^--archive=(.)/) {
3418- $main::archive=$1.$';
3419- }
3420- elsif (/^--comp=(.)/) {
3421- $main::component=$1.$';
3422- }
3423- elsif (/^--purpose=(.)/) {
3424- $main::purpose=$1.$';
3425- }
3426- elsif (/^--build-debug-symbols$/) {
3427- $main::build_debug_symbols = 1;
3428- }
3429- elsif (/^-d/ || /^--dist/) {
3430- if (/^-d(.)/ || /^--dist=(.)/) {
3431- $main::distribution = $1.$';
3432- }
3433- elsif (!@ARGV) {
3434- die "$_ option missing argument\n";
3435- }
3436- else {
3437- $main::distribution = shift @ARGV;
3438- }
3439- $main::distribution = "stable" if $main::distribution eq "s";
3440- $main::distribution = "testing" if $main::distribution eq "t";
3441- $main::distribution = "unstable" if $main::distribution eq "u";
3442- }
3443- elsif (/^-p/ || /^--purge/) {
3444- if (/^-p(.)/ || /^--purge=(.)/) {
3445- $conf::purge_build_directory = $1.$';
3446- }
3447- elsif (!@ARGV) {
3448- die "$_ option missing argument\n";
3449- }
3450- else {
3451- $conf::purge_build_directory = shift @ARGV;
3452- }
3453- die "Bad purge mode\n"
3454- if !isin($conf::purge_build_directory, qw(always successful never));
3455- }
3456- elsif (/^-m/ || /^--maintainer/) {
3457- if (/^-m(.)/ || /^--maintainer=(.)/) {
3458- $conf::maintainer_name = $1.$';
3459- }
3460- elsif (!@ARGV) {
3461- die "$_ option missing argument\n";
3462- }
3463- else {
3464- $conf::maintainer_name = shift @ARGV;
3465- }
3466- }
3467- elsif (/^-f/ || /^--force-depends/) {
3468- if (/^-f(.)/ || /^--force-depends=(.)/) {
3469- push( @main::manual_srcdeps, "f".$1.$' );
3470- }
3471- elsif (!@ARGV) {
3472- die "$_ option missing argument\n";
3473- }
3474- else {
3475- push( @main::manual_srcdeps, "f".(shift @ARGV) );
3476- }
3477- }
3478- elsif (/^-a/ || /^--add-depends/) {
3479- if (/^-a(.)/ || /^--add-depends=(.)/) {
3480- push( @main::manual_srcdeps, "a".$1.$' );
3481- }
3482- elsif (!@ARGV) {
3483- die "$_ option missing argument\n";
3484- }
3485- else {
3486- push( @main::manual_srcdeps, "a".(shift @ARGV) );
3487- }
3488- }
3489- elsif (/^--auto-give-back(=(.*))?$/) {
3490- $main::auto_giveback = 1;
3491- if ($2) {
3492- my @parts = split( '@', $2 );
3493- $main::auto_giveback_socket = "$main::HOME/build" if @parts > 3;
3494- $main::auto_giveback_socket .= $parts[$#parts-3] if @parts > 3;
3495- $main::auto_giveback_wb_user = $parts[$#parts-2] if @parts > 2;
3496- $main::auto_giveback_user = $parts[$#parts-1] if @parts > 1;
3497- $main::auto_giveback_host = $parts[$#parts];
3498- }
3499- }
3500- elsif (/^--database=(.+)$/) {
3501- $main::database = $1;
3502- }
3503- elsif (/^--stats-dir=(.+)$/) {
3504- $main::stats_dir = $1;
3505- }
3506- elsif (/^--make-binNMU=(.+)$/) {
3507- $main::binNMU = $1;
3508- $main::binNMUver ||= 1;
3509- }
3510- elsif (/^--binNMU=(\d+)$/) {
3511- $main::binNMUver = $1;
3512- }
3513- elsif (/^--use-snapshot$/) {
3514- $main::useSNAP = 1;
3515- $main::ld_library_path = "/usr/lib/gcc-snapshot/lib";
3516- $conf::path = "/usr/lib/gcc-snapshot/bin:$conf::path";
3517- }
3518- else {
3519- die "Unknown option: $_\n";
3520- }
3521-}
3522-
3523-die "Need distribution\n" if $main::distribution eq "";
3524-
3525-$conf::mailto = $conf::mailto{$main::distribution}
3526- if $conf::mailto{$main::distribution};
3527-
3528-$main::arch = $conf::arch if $conf::arch;
3529-
3530-# variables for scripts:
3531-open_log();
3532-$SIG{'INT'} = \&shutdown;
3533-$SIG{'TERM'} = \&shutdown;
3534-$SIG{'ALRM'} = \&shutdown;
3535-$SIG{'PIPE'} = \&shutdown;
3536-read_deps( map { m,(?:.*/)?([^_/]+)[^/]*, } @ARGV );
3537-if (-d "chroot-autobuild") {
3538- $main::chroot_dir = "chroot-autobuild";
3539- $main::chroot_build_dir = "$main::chroot_dir/build/$main::username/";
3540- $conf::srcdep_lock_dir = "$main::chroot_dir/var/debbuild/srcdep-lock";
3541- $main::ilock_file = "$conf::srcdep_lock_dir/install";
3542- my $absroot = "$main::cwd/$main::chroot_dir";
3543- $main::chroot_apt_options =
3544- "-o APT::Architecture=$main::arch ".
3545- "-o Dir::State=$absroot/var/".
3546- (-d "$absroot/var/lib/apt" ? "lib":"state")."/apt ".
3547- "-o Dir::State::status=$absroot/var/lib/dpkg/status ".
3548- "-o Dir::Cache=$absroot/var/cache/apt ".
3549- "-o Dir::Etc=$absroot/etc/apt ".
3550- "-o DPkg::Options::=--root=$absroot ".
3551- "-o DPkg::Options::=--force-architecture ".
3552- "-o DPkg::Run-Directory=$absroot";
3553- $main::chroot_apt_op = '$CHROOT_OPTIONS';
3554-}
3555-write_jobs_file();
3556-
3557-my( $pkgv, $pkg );
3558-foreach $pkgv (@ARGV) {
3559- my $urlbase;
3560-
3561-
3562- ($urlbase, $pkgv) = ($1, $3) if $pkgv =~ m,^(\w+://(\S+/)?)([^/]+)$,;
3563- $pkgv =~ s/\.dsc$//;
3564- next if !open_pkg_log( $pkgv );
3565- (my $pkg = $pkgv) =~ s/_.*$//;
3566- $main::pkg_start_time = time;
3567- $main::this_space = 0;
3568- $main::pkg_status = "failed"; # assume for now
3569- $main::current_job = $main::binNMU_name || $pkgv;
3570- $main::additional_deps = [];
3571- write_jobs_file( "currently building" );
3572- if (should_skip( $pkgv )) {
3573- $main::pkg_status = "skipped";
3574- goto cleanup_close;
3575- }
3576- my $dscfile = $pkgv.".dsc";
3577- $main::pkg_fail_stage = "fetch-src";
3578- my @files_to_rm = fetch_source_files( \$dscfile );
3579- if (@files_to_rm && $files_to_rm[0] eq "ERROR") {
3580- shift @files_to_rm;
3581- goto cleanup_symlinks;
3582- }
3583-
3584- $main::pkg_fail_stage = "install-deps";
3585- if (!install_deps( $pkg )) {
3586- print PLOG "Source-dependencies not satisfied; skipping $pkg\n";
3587- goto cleanup_packages;
3588- }
3589-
3590- my $dscbase = basename( $dscfile );
3591- $main::pkg_status = "successful" if build( $dscbase, $pkgv );
3592- chdir( $main::cwd );
3593- write_jobs_file( $main::pkg_status );
3594- append_to_FINISHED( $main::current_job );
3595-
3596- cleanup_packages:
3597- undo_specials();
3598- uninstall_deps();
3599- remove_srcdep_lock_file();
3600- cleanup_symlinks:
3601- remove_files( @files_to_rm );
3602- cleanup_close:
3603- analyze_fail_stage( $pkgv );
3604- if( $main::pkg_status eq 'failed' ) {
3605- $main::pkg_status = 'failed ' . $main::pkg_fail_stage;
3606- }
3607- write_jobs_file( $main::pkg_status );
3608- close_pkg_log( $pkgv );
3609- $main::current_job = "";
3610- if ( $main::batchmode and (-f "$main::HOME/EXIT-DAEMON-PLEASE") ) {
3611- main::shutdown("NONE (flag file exit)");
3612- }
3613-}
3614-write_jobs_file();
3615-
3616-close_log();
3617-#unlink( $main::jobs_file ) if $main::batchmode;
3618-unlink( "SBUILD-FINISHED" ) if $main::batchmode;
3619-exit 0;
3620-
3621-sub fetch_source_files {
3622- my $dscfile_ref = shift;
3623- my $dscfile = $$dscfile_ref;
3624- my ($dir, $dscbase, $files, @other_files, $dscarchs, @made);
3625- my ($build_depends, $build_depends_indep, $build_conflicts,
3626- $build_conflicts_indep);
3627- local( *F );
3628-
3629- $dscfile =~ m,^(.*)/([^/]+)$,;
3630- ($dir, $dscbase) = ($1, $2);
3631- my $urlbase;
3632- $urlbase = $1 if $dscfile =~ m,^(\w+://(\S+/)?)([^/]+)$,;
3633- (my $pkgv = $dscfile) =~ s,^(.*/)?([^/]+)\.dsc$,$2,;
3634- my ($pkg, $version) = split /_/, $pkgv;
3635- @main::have_dsc_build_deps = ();
3636-
3637- if (-d $dscfile) {
3638- if (-f "$dscfile/debian/.sbuild-build-deps") {
3639- open( F, "<$dscfile/debian/.sbuild-build-deps" );
3640- my $pkg;
3641- while( <F> ) {
3642- /^Package:\s*(.*)\s*$/i and $pkg = $1;
3643- /^Build-Depends:\s*(.*)\s*$/i and $build_depends = $1;
3644- /^Build-Depends-Indep:\s*(.*)\s*$/i and $build_depends_indep = $1;
3645- /^Build-Conflicts:\s*(.*)\s*$/i and $build_conflicts = $1;
3646- /^Build-Conflicts-Indep:\s*(.*)\s*$/i and $build_conflicts_indep = $1;
3647- }
3648- close( F );
3649- if ($build_depends || $build_depends_indep || $build_conflicts ||
3650- $build_conflicts_indep) {
3651- merge_pkg_build_deps( $pkg, $build_depends,
3652- $build_depends_indep, $build_conflicts,
3653- $build_conflicts_indep );
3654- }
3655- }
3656- return;
3657- }
3658-
3659- if ($dir ne ".") {
3660- {
3661- if (-f "${pkgv}.dsc") {
3662- print PLOG "${pkgv}.dsc exists in cwd\n";
3663- }
3664- else {
3665- my %entries;
3666- my $retried = 0;
3667-
3668- retry:
3669- print PLOG "Checking available source versions...\n";
3670- if (!open( PIPE, "$conf::sudo /usr/sbin/chroot ".
3671- "$main::chroot_dir $conf::apt_cache ".
3672- "-q showsrc $pkg 2>&1 </dev/null |" )) {
3673- print PLOG "Can't open pipe to apt-cache: $!\n";
3674- return ("ERROR");
3675- }
3676- { local($/) = "";
3677- while( <PIPE> ) {
3678- my $ver = $1 if /^Version:\s+(\S+)\s*$/mi;
3679- my $tfile = $1 if /^Files:\s*\n((\s+.*\s*\n)+)/mi;
3680- @{$entries{$ver}} = map { (split( /\s+/, $_ ))[3] }
3681- split( "\n", $tfile );
3682- }
3683- }
3684- close( PIPE );
3685- if ($?) {
3686- print PLOG "$conf::apt_cache failed\n";
3687- return ("ERROR");
3688- }
3689-
3690- if (!defined($entries{$version})) {
3691- if (!$retried) {
3692- # try to update apt's cache if nothing found
3693- system "$conf::sudo /usr/sbin/chroot ".
3694- "$main::chroot_dir $conf::apt_get ".
3695- "update >/dev/null";
3696- $retried = 1;
3697- goto retry;
3698- }
3699- print PLOG "Can't find source for $pkgv\n";
3700- print PLOG "(only different version(s) ",
3701- join( ", ", sort keys %entries), " found)\n"
3702- if %entries;
3703- return( "ERROR" );
3704- }
3705-
3706- print PLOG "Fetching source files...\n";
3707- @made = @{$entries{$version}};
3708- if (!open( PIPE, "$conf::apt_get $main::chroot_apt_options ".
3709- "--only-source -q -d source $pkg=$version 2>&1 </dev/null |" )) {
3710- print PLOG "Can't open pipe to $conf::apt_get: $!\n";
3711- return ("ERROR", @made);
3712- }
3713- while( <PIPE> ) {
3714- print PLOG $_;
3715- }
3716- close( PIPE );
3717- if ($?) {
3718- print PLOG "$conf::apt_get for sources failed\n";
3719- return( "ERROR", @made );
3720- }
3721- # touch the downloaded files, otherwise buildd-watcher
3722- # will complain that they're old :)
3723- $$dscfile_ref = $dscfile = (grep { /\.dsc$/ } @made)[0];
3724- }
3725- }
3726- }
3727-
3728- if (!open( F, "<$dscfile" )) {
3729- print PLOG "Can't open $dscfile: $!\n";
3730- return( "ERROR", @made );
3731- }
3732- my $dsctext;
3733- { local($/); $dsctext = <F>; }
3734- close( F );
3735-
3736- $dsctext =~ /^Build-Depends:\s*((.|\n\s+)*)\s*$/mi
3737- and $build_depends = $1;
3738- $dsctext =~ /^Build-Depends-Indep:\s*((.|\n\s+)*)\s*$/mi
3739- and $build_depends_indep = $1;
3740- $dsctext =~ /^Build-Conflicts:\s*((.|\n\s+)*)\s*$/mi
3741- and $build_conflicts = $1;
3742- $dsctext =~ /^Build-Conflicts-Indep:\s*((.|\n\s+)*)\s*$/mi
3743- and $build_conflicts_indep = $1;
3744- $build_depends =~ s/\n\s+/ /g if defined $build_depends;
3745- $build_depends_indep =~ s/\n\s+/ /g if defined $build_depends_indep;
3746- $build_conflicts =~ s/\n\s+/ /g if defined $build_conflicts;
3747- $build_conflicts_indep =~ s/\n\s+/ /g if defined $build_conflicts_indep;
3748-
3749- $dsctext =~ /^Architecture:\s*(.*)$/mi and $dscarchs = $1;
3750-
3751- $dsctext =~ /^Files:\s*\n((\s+.*\s*\n)+)/mi and $files = $1;
3752- @other_files = map { (split( /\s+/, $_ ))[3] } split( "\n", $files );
3753-
3754- if (!$dscarchs) {
3755- print PLOG "$dscbase has no Architecture: field -- skipping arch check!\n";
3756- }
3757- else {
3758- my $valid_arch;
3759- for my $a (split(/\s+/, $dscarchs)) {
3760- if (system('dpkg-architecture', '-a' . $main::arch, '-i' . $a) eq 0) {
3761- $valid_arch = 1;
3762- last;
3763- }
3764- }
3765- if ($dscarchs ne "any" && !($valid_arch) &&
3766- !($dscarchs eq "all" && $main::build_arch_all) ) {
3767- my $msg = "$dscbase: $main::arch not in arch list or does not match any arch ";
3768- $msg .= "wildcards: $dscarchs -- skipping\n";
3769- print PLOG $msg;
3770- $main::pkg_fail_stage = "arch-check";
3771- return( "ERROR", @made );
3772- }
3773- }
3774- print "Arch check ok ($main::arch included in $dscarchs)\n"
3775- if $main::debug;
3776-
3777- if ($build_depends || $build_depends_indep || $build_conflicts ||
3778- $build_conflicts_indep) {
3779- @main::have_dsc_build_deps = ($build_depends, $build_depends_indep,
3780- $build_conflicts,$build_conflicts_indep);
3781- merge_pkg_build_deps( $pkg, $build_depends, $build_depends_indep,
3782- $build_conflicts, $build_conflicts_indep );
3783- }
3784-
3785- return @made;
3786-}
3787-
3788-sub build {
3789- my $dsc = shift;
3790- my $pkgv = shift;
3791- my( $dir, $rv, $changes );
3792- my $do_apply_patches = 1;
3793- local( *PIPE, *F, *F2 );
3794-
3795- fixup_pkgv( \$pkgv );
3796- print PLOG "-"x78, "\n";
3797- # count build time from now, ignoring the installation of source deps
3798- $main::pkg_start_time = time;
3799- $main::this_space = 0;
3800- $pkgv =~ /^([a-zA-Z\d.+-]+)_([a-zA-Z\d:.+~-]+)/;
3801- my ($pkg, $version) = ($1,$2);
3802- (my $sversion = $version) =~ s/^\d+://;
3803- my $tmpunpackdir = $dsc;
3804- $tmpunpackdir =~ s/-.*$/.orig.tmp-nest/;
3805- $tmpunpackdir =~ s/_/-/;
3806- $tmpunpackdir = "$main::chroot_build_dir$tmpunpackdir";
3807-
3808- if (-d "$main::chroot_build_dir$dsc" && -l "$main::chroot_build_dir$dsc") {
3809- # if the package dir already exists but is a symlink, complain
3810- print PLOG "Cannot unpack source: a symlink to a directory with the\n",
3811- "same name already exists.\n";
3812- return 0;
3813- }
3814- if (! -d "$main::chroot_build_dir$dsc") {
3815- $main::pkg_fail_stage = "unpack";
3816- # dpkg-source refuses to remove the remanants of an
3817- # aborted dpkg-source extraction, so we will if necessary.
3818- if (-d $tmpunpackdir) {
3819- system ("rm -fr $tmpunpackdir");
3820- }
3821- $main::sub_pid = open( PIPE, "-|" );
3822- if (!defined $main::sub_pid) {
3823- print PLOG "Can't spawn dpkg-source: $!\n";
3824- return 0;
3825- }
3826- if ($main::sub_pid == 0) {
3827- setpgrp( 0, $$ );
3828- if ($main::chroot_build_dir && !chdir( $main::chroot_build_dir )) {
3829- print PLOG "Couldn't cd to $main::chroot_build_dir: $!\n";
3830- system ("rm -fr $tmpunpackdir") if -d $tmpunpackdir;
3831- exit 1;
3832- }
3833-
3834- my @files;
3835- push( @files, $dsc );
3836- if (!open( F, "<$main::cwd/$dsc" )) {
3837- print PLOG "Can't open $main::cwd/$dsc: $!\n";
3838- return 0;
3839- }
3840- my $dsctext;
3841- { local($/); $dsctext = <F>; }
3842- close( F );
3843- my $files;
3844- $dsctext =~ /^Files:\s*\n((\s+.*\s*\n)+)/mi and $files = $1;
3845- push(@files, map { (split( /\s+/, $_ ))[3] } split( "\n", $files ));
3846-
3847- my $file;
3848- foreach $file (@files) {
3849- system ("cp", "$main::cwd/$file", "$file");
3850- }
3851- exec "$conf::sudo", "/usr/sbin/chroot", "$main::cwd/$main::chroot_dir",
3852- "$conf::su", $main::username, "-s", "/bin/sh", "-c",
3853- "cd /build/$main::username && $conf::dpkg_source -sn -x $dsc 2>&1";
3854- unlink @files;
3855- }
3856- $main::sub_task = "dpkg-source";
3857-
3858- while( <PIPE> ) {
3859- print PLOG $_;
3860- $dir = $1 if /^dpkg-source: (?:info: )?extracting \S+ in (\S+)/;
3861- $main::pkg_fail_stage = "unpack-check"
3862- if /^dpkg-source: error: file.*instead of expected/;
3863- }
3864- close( PIPE );
3865- undef $main::sub_pid;
3866- if ($?) {
3867- print PLOG "FAILED [dpkg-source died]\n";
3868-
3869- system ("rm -fr $tmpunpackdir") if -d $tmpunpackdir;
3870- return 0;
3871- }
3872- if (!$dir) {
3873- print PLOG "Couldn't find directory of $dsc in dpkg-source output\n";
3874- system ("rm -fr $tmpunpackdir") if -d $tmpunpackdir;
3875- return 0;
3876- }
3877- $dir = "$main::chroot_build_dir$dir";
3878-
3879- if (system( "chmod -R g-s,go+rX $dir" ) != 0) {
3880- print PLOG "chmod -R g-s,go+rX $dir failed.\n";
3881- return 0;
3882- }
3883-
3884- if (@main::have_dsc_build_deps && !defined $main::build_source) {
3885- my ($d, $di, $c, $ci) = @main::have_dsc_build_deps;
3886- open( F, ">$dir/debian/.sbuild-build-deps" );
3887- print F "Package: $pkg\n";
3888- print F "Build-Depends: $d\n" if $d;
3889- print F "Build-Depends-Indep: $di\n" if $di;
3890- print F "Build-Conflicts: $c\n" if $c;
3891- print F "Build-Conflicts-Indep: $ci\n" if $ci;
3892- close( F );
3893- }
3894- }
3895- else {
3896- $dir = "$main::chroot_build_dir$dsc";
3897- $do_apply_patches = 0;
3898-
3899- $main::pkg_fail_stage = "check-unpacked-version";
3900- # check if the unpacked tree is really the version we need
3901- $main::sub_pid = open( PIPE, "-|" );
3902- if (!defined $main::sub_pid) {
3903- print PLOG "Can't spawn dpkg-parsechangelog: $!\n";
3904- return 0;
3905- }
3906- if ($main::sub_pid == 0) {
3907- setpgrp( 0, $$ );
3908- chdir( $dir );
3909- exec "dpkg-parsechangelog 2>&1";
3910- }
3911- $main::sub_task = "dpkg-parsechangelog";
3912-
3913- my $clog = "";
3914- while( <PIPE> ) {
3915- $clog .= $_;
3916- }
3917- close( PIPE );
3918- undef $main::sub_pid;
3919- if ($?) {
3920- print PLOG "FAILED [dpkg-parsechangelog died]\n";
3921- return 0;
3922- }
3923- if ($clog !~ /^Version:\s*(.+)\s*$/mi) {
3924- print PLOG "dpkg-parsechangelog didn't print Version:\n";
3925- return 0;
3926- }
3927- my $tree_version = $1;
3928- my $cmp_version = ($main::binNMU && -f "$dir/debian/.sbuild-binNMU-done") ?
3929- binNMU_version($version) : $version;
3930- if ($tree_version ne $cmp_version) {
3931- print PLOG "The unpacked source tree $dir is version ".
3932- "$tree_version, not wanted $cmp_version!\n";
3933- return 0;
3934- }
3935- }
3936-
3937- if (!chdir( $dir )) {
3938- print PLOG "Couldn't cd to $dir: $!\n";
3939- system ("rm -fr $tmpunpackdir") if -d $tmpunpackdir;
3940- return 0;
3941- }
3942-
3943- $main::pkg_fail_stage = "check-space";
3944- my $current_usage = `/usr/bin/du -s .`;
3945- $current_usage =~ /^(\d+)/;
3946- $current_usage = $1;
3947- if ($current_usage) {
3948- my $free = df( "." );
3949- if ($free < 2*$current_usage) {
3950- print PLOG "Disk space is propably not enough for building.\n".
3951- "(Source needs $current_usage KB, free are $free KB.)\n";
3952- print PLOG "Purging $dir\n";
3953- chdir( $main::cwd );
3954- system "$conf::sudo rm -rf $dir";
3955- return 0;
3956- }
3957- }
3958-
3959- $main::pkg_fail_stage = "hack-binNMU";
3960- if ($main::binNMU && ! -f "debian/.sbuild-binNMU-done") {
3961- if (open( F, "<debian/changelog" )) {
3962- my($firstline, $text);
3963- $firstline = <F> while $firstline =~ /^$/;
3964- { local($/); undef $/; $text = <F>; }
3965- close( F );
3966- $firstline =~ /^(\S+)\s+\((\S+)\)\s+([^;]+)\s*;\s*urgency=(\S+)\s*$/;
3967- my ($name, $version, $dists, $urgent) = ($1, $2, $3, $4);
3968- my $NMUversion = binNMU_version($version);
3969- chomp( my $date = `822-date` );
3970- if (!open( F, ">debian/changelog" )) {
3971- print PLOG "Can't open debian/changelog for binNMU hack: $!\n";
3972- chdir( $main::cwd );
3973- return 0;
3974- }
3975- $dists = $main::distribution;
3976- print F "$name ($NMUversion) $dists; urgency=low\n\n";
3977- print F " * Binary-only non-maintainer upload for $main::arch; ",
3978- "no source changes.\n";
3979- print F " * ", join( " ", split( "\n", $main::binNMU )), "\n\n";
3980- print F " -- $conf::maintainer_name $date\n\n";
3981-
3982- print F $firstline, $text;
3983- close( F );
3984- system "touch debian/.sbuild-binNMU-done";
3985- print PLOG "*** Created changelog entry for bin-NMU version $NMUversion\n";
3986- }
3987- else {
3988- print PLOG "Can't open debian/changelog -- no binNMU hack!\n";
3989- }
3990- }
3991-
3992- if ($do_apply_patches) {
3993- if (!apply_patches( $pkg )) {
3994- chdir( $main::cwd );
3995- return 0;
3996- }
3997- }
3998-
3999- if (-f "debian/files") {
4000- local( *FILES );
4001- my @lines;
4002- open( FILES, "<debian/files" );
4003- chomp( @lines = <FILES> );
4004- close( FILES );
4005- @lines = map { my $ind = 68-length($_);
4006- $ind = 0 if $ind < 0;
4007- "| $_".(" " x $ind)." |\n"; } @lines;
4008-
4009- print PLOG <<"EOF";
4010-
4011-+----------------------------------------------------------------------+
4012-| sbuild Warning: |
4013-| --------------- |
4014-| After unpacking, there exists a file debian/files with the contents: |
4015-| |
4016-EOF
4017- print PLOG @lines;
4018- print PLOG <<"EOF";
4019-| |
4020-| This should be reported as a bug. |
4021-| The file has been removed to avoid dpkg-genchanges errors. |
4022-+----------------------------------------------------------------------+
4023-
4024-EOF
4025- unlink "debian/files";
4026- }
4027-
4028- open CURRENT, ">$main::cwd/$main::chroot_dir/CurrentlyBuilding" or die "$main::cwd/$main::chroot_dir/CurrentlyBuilding open failed";
4029- # Package: must be first
4030- print CURRENT "Package: $pkg\nComponent: $main::component\n";
4031- print CURRENT "Suite: $main::distribution\n" if $main::distribution;
4032- print CURRENT "Purpose: $main::purpose\n" if $main::purpose;
4033- print CURRENT "Build-Debug-Symbols: yes\n" if $main::build_debug_symbols;
4034- close CURRENT;
4035-
4036- $main::build_start_time = time;
4037- $main::pkg_fail_stage = "build";
4038- $main::sub_pid = open( PIPE, "-|" );
4039- if (!defined $main::sub_pid) {
4040- print PLOG "Can't spawn dpkg-buildpackage: $!\n";
4041- chdir( $main::cwd );
4042- return 0;
4043- }
4044- if ($main::sub_pid == 0) {
4045- setpgrp( 0, $$ );
4046- my $binopt = $main::build_source ? "" :
4047- $main::build_arch_all ? "-b" : "-B";
4048- my $env_cmnd = $conf::build_env_cmnd;
4049- $env_cmnd = $conf::build_env_cmnd{$pkg} if $conf::build_env_cmnd{$pkg};
4050- if ($main::chroot_dir) {
4051- my $bdir = $dir;
4052- $bdir =~ s/^\Q$main::chroot_dir\E//;
4053- if (-f "$main::chroot_dir/etc/ld.so.conf" &&
4054- ! -r "$main::chroot_dir/etc/ld.so.conf") {
4055- system "$conf::sudo chmod a+r $main::chroot_dir/etc/ld.so.conf";
4056- print PLOG "ld.so.conf was not readable! Fixed.\n";
4057- }
4058- exec "$conf::sudo", "/usr/sbin/chroot", "$main::cwd/$main::chroot_dir",
4059- "$conf::su", $main::username, "-s", "/bin/sh", "-c",
4060- "cd $bdir && PATH=$conf::path ".
4061- (defined($main::nr_processors) ?
4062- "DEB_BUILD_OPTIONS=\"parallel=".$main::nr_processors."\" " : "").
4063- (defined($main::ld_library_path) ?
4064- "LD_LIBRARY_PATH=".$main::ld_library_path." " : "").
4065- "exec $env_cmnd dpkg-buildpackage $conf::pgp_options ".
4066- "$binopt -m'$conf::maintainer_name' -r$conf::fakeroot 2>&1";
4067- }
4068- else {
4069- if (-f "/etc/ld.so.conf" && ! -r "/etc/ld.so.conf") {
4070- system "$conf::sudo chmod a+r /etc/ld.so.conf";
4071- print PLOG "ld.so.conf was not readable! Fixed.\n";
4072- }
4073- exec "$env_cmnd dpkg-buildpackage $conf::pgp_options $binopt ".
4074- "-m'$conf::maintainer_name' -r$conf::fakeroot 2>&1";
4075- }
4076- }
4077- $main::sub_task = "dpkg-buildpackage";
4078-
4079- # We must send the signal as root, because some subprocesses of
4080- # dpkg-buildpackage could run as root. So we have to use a shell
4081- # command to send the signal... but /bin/kill can't send to
4082- # process groups :-( So start another Perl :-)
4083- my $timeout = $conf::individual_stalled_pkg_timeout{$pkg} ||
4084- $conf::stalled_pkg_timeout;
4085- $timeout *= 60;
4086- my $timed_out = 0;
4087- my(@timeout_times, @timeout_sigs, $last_time);
4088- $SIG{'ALRM'} = sub {
4089- my $signal = ($timed_out > 0) ? 9 : 15;
4090- system "$conf::sudo perl -e 'kill( -$signal, $main::sub_pid )'";
4091- $timeout_times[$timed_out] = time - $last_time;
4092- $timeout_sigs[$timed_out] = $signal;
4093- $timed_out++;
4094- $timeout = 5*60; # only wait 5 minutes until next signal
4095- };
4096-
4097- alarm( $timeout );
4098- while( <PIPE> ) {
4099- alarm( $timeout );
4100- $last_time = time;
4101- print PLOG $_;
4102- }
4103- close( PIPE );
4104- undef $main::sub_pid;
4105- alarm( 0 );
4106- $rv = $?;
4107-
4108- my $i;
4109- for( $i = 0; $i < $timed_out; ++$i ) {
4110- print PLOG "Build killed with signal ", $timeout_sigs[$i],
4111- " after ", int($timeout_times[$i]/60),
4112- " minutes of inactivity\n";
4113- }
4114- $main::pkg_end_time = time;
4115- my $date = `date +%Y%m%d-%H%M`;
4116- print PLOG "*"x78, "\n";
4117- print PLOG "Build finished at $date";
4118- chdir( $main::cwd );
4119-
4120- my @space_files = ("$dir");
4121- if (!$main::nolog and defined $conf::exit_hook and open TMP, '-|', "$conf::exit_hook <$main::pkg_logfile") {
4122- local $/ = undef;
4123- my $log = <TMP>;
4124- close TMP;
4125- $rv |= $?;
4126- print PLOG $log;
4127- }
4128- if ($rv) {
4129- print PLOG "FAILED [dpkg-buildpackage died]\n";
4130- }
4131- else {
4132- my $trans_oldfmt="$main::chroot_build_dir${pkg}_${version}_translations.tar.gz";
4133- my $trans_newfmt="$main::chroot_build_dir${pkg}_${version}_${main::arch}_translations.tar.gz";
4134- my $translations="";
4135- if (-r $trans_newfmt) {
4136- $translations = $trans_newfmt;
4137- } elsif (-r $trans_oldfmt) {
4138- $translations = $trans_oldfmt;
4139- }
4140- if ($translations) {
4141- print PLOG "Publishing $translations for rosetta.\n";
4142- my $date = strftime '%Y%m%d',gmtime;
4143- my $target = "$main::HOME/public_html/translations/$date/";
4144- system "mkdir -p $target";
4145- if (system("cp",$translations,$target) != 0) {
4146- print PLOG "ERROR: Could not move $translations to $target\n";
4147- } else {
4148- open TRANS, ">>$target/translations.txt";
4149- print TRANS "File: " . basename(${translations}) . "\n".
4150- "Distribution: ${main::archive}\n".
4151- "Release: ${main::distribution}\n".
4152- "Component: ${main::component}\n".
4153- "Source: ${pkg}\n".
4154- "Version: ${version}\n\n";
4155- close TRANS;
4156- system("chmod -R go+rX $main::HOME/public_html/translations");
4157- }
4158- }
4159-
4160- my $ddebtar = "";
4161- my $ddebstring = "";
4162- if (-r glob("$main::chroot_build_dir/*.ddeb")) {
4163- my @ddeblist = glob("$main::chroot_build_dir/*.ddeb");
4164- $ddebtar="${pkg}_${version}_${main::arch}_ddebs.tar";
4165- while (@ddeblist) {
4166- $ddebstring .= basename(@ddeblist[0]) . " ";
4167- shift @ddeblist;
4168- }
4169- }
4170- if ($ddebstring) {
4171- print PLOG "Publishing debug debs.\n";
4172- my $date = strftime '%Y%m%d',gmtime;
4173- my $target = "$main::HOME/public_html/ddebs/$date/";
4174- system "mkdir -p $target";
4175- if (system("tar -C $main::chroot_build_dir -chf $target/$ddebtar $ddebstring") != 0) {
4176- print PLOG "ERROR: Could not create $ddebtar in $target\n";
4177- } else {
4178- open TRANS, ">>$target/ddebs.txt";
4179- print TRANS "File: " . basename(${ddebtar}) . "\n".
4180- "Distribution: ${main::archive}\n".
4181- "Release: ${main::distribution}\n".
4182- "Component: ${main::component}\n".
4183- "Source: ${pkg}\n".
4184- "Version: ${version}\n\n";
4185- close TRANS;
4186- system("chmod -R go+rX $main::HOME/public_html/ddebs");
4187- }
4188- }
4189-
4190- if (-r "$dir/debian/files") {
4191- my @debs;
4192- my @files;
4193- open( F, "<$dir/debian/files" );
4194- while( <F> ) {
4195- my $f = (split( /\s+/, $_ ))[0];
4196- push( @files, "$main::chroot_build_dir$f" );
4197- if ($main::build_arch_all) {
4198- next if ($f !~ /$main::arch\.[\w\d.-]*$/ && $f !~ /all\.[\w\d.-]*$/);
4199- } else {
4200- next if ($f !~ /$main::arch\.[\w\d.-]*$/);
4201- }
4202- push( @debs, "$main::chroot_build_dir$f" );
4203- push( @space_files, $f );
4204- }
4205- close( F );
4206- my @debs2 = @debs;
4207- foreach (@debs) {
4208- print PLOG "\n$_:\n";
4209- if (!open( PIPE, "dpkg --info $_ 2>&1 |" )) {
4210- print PLOG "Can't spawn dpkg: $! -- can't dump infos\n";
4211- }
4212- else {
4213- print PLOG $_ while( <PIPE> );
4214- close( PIPE );
4215- }
4216- }
4217- foreach (@debs2) {
4218- print PLOG "\n$_:\n";
4219- if (!open( PIPE, "dpkg --contents $_ 2>&1 |" )) {
4220- print PLOG "Can't spawn dpkg: $! -- can't dump infos\n";
4221- }
4222- else {
4223- print PLOG $_ while( <PIPE> );
4224- close( PIPE );
4225- }
4226- }
4227- if ($main::chroot_build_dir) {
4228- foreach (@files) {
4229- system "mv", $_, "."
4230- and print PLOG "ERROR: Could not move $_ to .\n";
4231- }
4232- }
4233- }
4234-
4235- if (-r $translations) {
4236- system("rm",$translations);
4237- }
4238-
4239- $changes = "${pkg}_".
4240- ($main::binNMU ? binNMU_version($sversion) : $sversion).
4241- "_$main::arch.changes";
4242- if (-r "$main::chroot_build_dir$changes") {
4243- my(@do_dists, @saved_dists);
4244- print PLOG "\n$changes:\n";
4245- open( F, "<$main::chroot_build_dir$changes" );
4246- if (open( F2, ">$changes.new" )) {
4247- while( <F> ) {
4248- if (/^Distribution:\s*(.*)\s*$/) {
4249- print PLOG "Distribution: $main::distribution\n";
4250- print F2 "Distribution: $main::distribution\n";
4251- }
4252- else {
4253- print F2 $_;
4254- while (length $_ > 989)
4255- {
4256- my $index = rindex($_,' ',989);
4257- print PLOG substr ($_,0,$index) . "\n";
4258- $_ = ' ' . substr ($_,$index+1);
4259- }
4260- print PLOG $_;
4261- }
4262- }
4263- close( F2 );
4264- rename( "$changes.new", "$changes" )
4265- or print PLOG "$changes.new could not be renamed ".
4266- "to $changes: $!\n";
4267- unlink( "$main::chroot_build_dir$changes" )
4268- if $main::chroot_build_dir;
4269- }
4270- else {
4271- print PLOG "Cannot create $changes.new: $!\n";
4272- print PLOG "Distribution field may be wrong!!!\n";
4273- if ($main::chroot_build_dir) {
4274- system "mv", "$main::chroot_build_dir$changes", "."
4275- and print PLOG "ERROR: Could not move $_ to .\n";
4276- }
4277- }
4278- close( F );
4279- print PLOG "\n";
4280- }
4281- else {
4282- print PLOG "Can't find $changes -- can't dump infos\n";
4283- }
4284-
4285- print PLOG "*"x78, "\n";
4286- print PLOG "Built successfully\n";
4287- }
4288-
4289- check_watches();
4290- check_space( @space_files );
4291-
4292- if ($conf::purge_build_directory eq "always" ||
4293- ($conf::purge_build_directory eq "successful" && $rv == 0)) {
4294- print PLOG "Purging $dir\n";
4295- system "$conf::sudo rm -rf $dir";
4296- }
4297-
4298- print PLOG "-"x78, "\n";
4299- return $rv == 0 ? 1 : 0;
4300-}
4301-
4302-sub apply_patches {
4303- my $pkg = shift;
4304- my $name;
4305-
4306- $main::pkg_fail_stage = "apply-patch";
4307- foreach $name ((map { $_->{'Package'} } @{$main::deps{$pkg}}),
4308- @main::global_patches) {
4309- if ($name =~ /^\*/ && exists $main::specials{$name}->{'patch'}) {
4310- if (exists $main::specials{$name}->{'patchcond'}) {
4311- print "Testing condition for $name patch:\n"
4312- if $main::debug;
4313- if (run_script("+e",$main::specials{$name}->{'patchcond'})!=0){
4314- print PLOG "Condition for $name patch not true -- ",
4315- "not applying\n" if $name !~ /^\*\*/;
4316- next;
4317- }
4318- print PLOG "Condition for $name patch ok\n";
4319- }
4320- print PLOG "Applying $name patch\n";
4321- $main::sub_pid = open( PIPE, "|-" );
4322- if (!defined $main::sub_pid) {
4323- print PLOG "Can't spawn patch: $! -- can't patch\n";
4324- return 0;
4325- }
4326- if ($main::sub_pid == 0) {
4327- setpgrp( 0, $$ );
4328- open( STDOUT, ">&PLOG" );
4329- open( STDERR, ">&PLOG" );
4330- exec "patch --batch --quiet -p1 -E -N --no-backup-if-mismatch";
4331- }
4332- $main::sub_task = "patch";
4333-
4334- print PIPE $main::specials{$name}->{'patch'};
4335- close( PIPE );
4336- undef $main::sub_pid;
4337- if ($name !~ /^\*\*/ && $?) {
4338- print PLOG "FAILED [patch died]\n";
4339- return 0;
4340- }
4341- }
4342- }
4343- return 1;
4344-}
4345-
4346-sub analyze_fail_stage {
4347- my $pkgv = shift;
4348-
4349- return if $main::pkg_status ne "failed";
4350- return if !$main::auto_giveback;
4351- if (isin( $main::pkg_fail_stage,
4352- qw(find-dsc fetch-src unpack-check check-space install-deps-env))) {
4353- $main::pkg_status = "given-back";
4354- print PLOG "Giving back package $pkgv after failure in ".
4355- "$main::pkg_fail_stage stage.\n";
4356- chdir( $main::cwd );
4357- my $cmd = "";
4358- $cmd = "ssh -l$main::auto_giveback_user $main::auto_giveback_host "
4359- if $main::auto_giveback_host;
4360- $cmd .= "-S $main::auto_giveback_socket "
4361- if ($main::auto_giveback_socket and -S "$main::auto_giveback_socket");
4362- $cmd .= "wanna-build --give-back --no-down-propagation ".
4363- "--dist=$main::distribution";
4364- $cmd .= " --database=$main::database" if $main::database;
4365- $cmd .= " --user=$main::auto_giveback_wb_user "
4366- if $main::auto_giveback_wb_user;
4367- $cmd .= " $pkgv";
4368- system $cmd;
4369- if ($?) {
4370- print PLOG "wanna-build failed with status $?\n";
4371- }
4372- else {
4373- add_givenback( $pkgv, time );
4374- if ($main::stats_dir) {
4375- local( *F );
4376- lock_file( "$main::stats_dir" );
4377- open( F, ">>$main::stats_dir/give-back" );
4378- print F "1\n";
4379- close( F );
4380- unlock_file( "$main::stats_dir" );
4381- }
4382- }
4383- }
4384-}
4385-
4386-sub remove_files {
4387-
4388- foreach (@_) {
4389- unlink $_;
4390- print "Removed $_\n" if $main::debug;
4391- }
4392-}
4393-
4394-
4395-sub install_deps {
4396- my $pkg = shift;
4397- my( @positive, @negative, @special, @instd, @rmvd );
4398-
4399- if (!exists $main::deps{$pkg}) {
4400- prepare_watches( [] );
4401- return 1;
4402- }
4403-
4404- my $dep = $main::deps{$pkg};
4405- if ($main::debug) {
4406- print "Source dependencies of $pkg: ", format_deps(@$dep), "\n";
4407- }
4408-
4409- repeat:
4410- lock_file( "$main::ilock_file", 1 );
4411-
4412- print "Filtering dependencies\n" if $main::debug;
4413- if (!filter_dependencies( $dep, \@positive, \@negative, \@special )) {
4414- print PLOG "Package installation not possible\n";
4415- unlock_file( "$main::ilock_file" );
4416- return 0;
4417- }
4418-
4419- print PLOG "Checking for source dependency conflicts...\n";
4420- if (!run_apt( "-s", \@instd, \@rmvd, @positive )) {
4421- print PLOG "Test what should be installed failed.\n";
4422- unlock_file( "$main::ilock_file" );
4423- return 0;
4424- }
4425- # add negative deps as to be removed for checking srcdep conflicts
4426- push( @rmvd, @negative );
4427- my @confl;
4428- if (@confl = check_srcdep_conflicts( \@instd, \@rmvd, \@special )) {
4429- print PLOG "Waiting for job(s) @confl to finish\n";
4430-
4431- unlock_file( "$main::ilock_file" );
4432- wait_for_srcdep_conflicts( @confl );
4433- goto repeat;
4434- }
4435-
4436- write_srcdep_lock_file( $dep, \@special );
4437-
4438- foreach my $sp (@special) {
4439- next if $sp !~ /^\*/ || !exists $main::specials{$sp}->{'prepre'};
4440- print PLOG "Running prepre script for $sp\n";
4441- if (run_script( "-e", $main::specials{$sp}->{'prepre'} ) != 0) {
4442- print PLOG "prepre script of special dependency $sp failed\n";
4443- unlock_file( "$main::ilock_file" );
4444- return 0;
4445- }
4446- }
4447-
4448- print "Installing positive dependencies: @positive\n" if $main::debug;
4449- if (!run_apt( "-y", \@instd, \@rmvd, @positive )) {
4450- print PLOG "Package installation failed\n";
4451- # try to reinstall removed packages
4452- print PLOG "Trying to reinstall removed packages:\n";
4453- print "Reinstalling removed packages: @rmvd\n" if $main::debug;
4454- my (@instd2, @rmvd2);
4455- print PLOG "Failed to reinstall removed packages!\n"
4456- if !run_apt( "-y", \@instd2, \@rmvd2, @rmvd );
4457- print "Installed were: @instd2\n" if $main::debug;
4458- print "Removed were: @rmvd2\n" if $main::debug;
4459- # remove additional packages
4460- print PLOG "Trying to uninstall newly installed packages:\n";
4461- uninstall_debs( $main::chroot_dir ? "purge" : "remove", @instd );
4462- unlock_file( "$main::ilock_file" );
4463- return 0;
4464- }
4465- set_installed( @instd );
4466- set_removed( @rmvd );
4467-
4468- print "Removing negative dependencies: @negative\n" if $main::debug;
4469- if (!uninstall_debs( $main::chroot_dir ? "purge" : "remove", @negative )) {
4470- print PLOG "Removal of packages failed\n";
4471- unlock_file( "$main::ilock_file" );
4472- return 0;
4473- }
4474- set_removed( @negative );
4475-
4476- my $fail = check_dependencies( $dep );
4477- if ($fail) {
4478- print PLOG "After installing, the following source dependencies are ".
4479- "still unsatisfied:\n$fail\n";
4480- unlock_file( "$main::ilock_file" );
4481- return 0;
4482- }
4483-
4484- foreach my $sp (@special) {
4485- next if $sp !~ /^\*/ ||
4486- (!exists $main::specials{$sp}->{'pre'} &&
4487- !exists $main::specials{$sp}->{'post'} &&
4488- !exists $main::specials{$sp}->{'unpack'});
4489- if (exists $main::specials{$sp}->{'unpack'}) {
4490- my $s = $main::specials{$sp}->{'unpack'};
4491- $s =~ s/^\s+//mg;
4492- $s =~ s/\s+$//mg;
4493- my @s = split( /\s+/, $s );
4494- my @rem;
4495- print PLOG "Unpacking special sources $sp: @s\n";
4496- if (!(@rem = unpack_special_source( @s ))) {
4497- print PLOG "unpacking of special dependency sources for $sp failed\n";
4498- unlock_file( "$main::ilock_file" );
4499- return 0;
4500- }
4501- $main::changes->{'unpacked'}->{$sp} = \@rem;
4502- }
4503- if (exists $main::specials{$sp}->{'pre'}) {
4504- print PLOG "Running pre script for $sp\n";
4505- $main::changes->{'specials'}->{$sp} = 1;
4506- if (run_script( "-e", $main::specials{$sp}->{'pre'} ) != 0) {
4507- print PLOG "pre script of special dependency $sp failed\n";
4508- unlock_file( "$main::ilock_file" );
4509- return 0;
4510- }
4511- }
4512- }
4513-
4514- local (*F);
4515- if (open( F, "| $conf::sudo /usr/sbin/chroot $main::chroot_dir $conf::dpkg --set-selections")) {
4516- foreach my $tpkg (@instd) {
4517- print F $tpkg . " purge\n";
4518- }
4519- close( F );
4520- if ($?) {
4521- print PLOG "$conf::dpkg --set-selections failed";
4522- }
4523- }
4524-
4525- unlock_file( "$main::ilock_file" );
4526-
4527- prepare_watches( $dep, @instd );
4528- return 1;
4529-}
4530-
4531-sub unpack_special_source {
4532- my @s = @_;
4533- my (@files, @dirs);
4534- local (*PIPE);
4535-
4536- foreach my $s (@s) {
4537- my $dsc;
4538-
4539- {
4540- if (!open( PIPE, "$conf::apt_get $main::chroot_apt_options ".
4541- "--only-source -q -d source $s 2>&1 </dev/null |" )) {
4542- print PLOG "Can't open pipe to apt-get: $!\n";
4543- goto failed;
4544- }
4545- while( <PIPE> ) {
4546- $dsc = "$1_$2.dsc" if /(\S+) (?:[^:]+:)?(\S+) \(dsc\)/;
4547- print PLOG $_;
4548- }
4549- close( PIPE );
4550- if ($?) {
4551- print PLOG "Apt-get of special unpack sources failed\n";
4552- goto failed;
4553- }
4554- push( @files, $dsc );
4555- if (!open( F, "<$dsc" )) {
4556- print PLOG "Can't open $dsc: $!\n";
4557- goto failed;
4558- }
4559- my $dsctext;
4560- { local($/); $dsctext = <F>; }
4561- close( F );
4562- my $files;
4563- $dsctext =~ /^Files:\s*\n((\s+.*\s*\n)+)/mi and $files = $1;
4564- push(@files, map { (split( /\s+/, $_ ))[3] } split( "\n", $files ));
4565- }
4566-
4567- my $pid = open( PIPE, "-|" );
4568- if (!defined $pid) {
4569- print PLOG "Can't spawn dpkg-source: $! -- special unpack failed\n";
4570- goto failed;
4571- }
4572- if ($pid == 0) {
4573- setpgrp( 0, $$ );
4574- if ($main::chroot_build_dir && !chdir( $main::chroot_build_dir )) {
4575- print PLOG "Couldn't cd to $main::chroot_build_dir: $! -- special unpack failed\n";
4576- exit 1;
4577- }
4578- exec "$conf::dpkg_source -sn -x $main::cwd/$dsc 2>&1";
4579- }
4580- my $dir;
4581- while( <PIPE> ) {
4582- print PLOG $_;
4583- $dir = $1 if /^dpkg-source: (?:info: )?extracting \S+ in (\S+)/;
4584- }
4585- close( PIPE );
4586- if ($?) {
4587- print PLOG "dpkg-source failure -- special unpack failed\n";
4588- goto failed;
4589- }
4590- push( @dirs, "$main::chroot_build_dir$dir" );
4591- unlink( @files );
4592- }
4593-
4594- return @dirs;
4595-
4596- failed:
4597- unlink( @files );
4598- system( "rm", "-rf", @dirs );
4599- return ();
4600-}
4601-
4602-sub wait_for_srcdep_conflicts {
4603- my @confl = @_;
4604-
4605- for(;;) {
4606- sleep( $conf::srcdep_lock_wait*60 );
4607- my $allgone = 1;
4608- for (@confl) {
4609- /^(\d+)-(\d+)$/;
4610- my $pid = $1;
4611- if (-f "$conf::srcdep_lock_dir/$_") {
4612- if (kill( 0, $pid ) == 0 && $! == ESRCH) {
4613- print PLOG "Ignoring stale src-dep lock $_\n";
4614- unlink( "$conf::srcdep_lock_dir/$_" ) or
4615- print PLOG "Cannot remove $conf::srcdep_lock_dir/$_: $!\n";
4616- }
4617- else {
4618- $allgone = 0;
4619- last;
4620- }
4621- }
4622- }
4623- last if $allgone;
4624- }
4625-}
4626-
4627-sub uninstall_deps {
4628- my( @pkgs, @instd, @rmvd );
4629-
4630- lock_file( "$main::ilock_file", 1 );
4631-
4632- @pkgs = keys %{$main::changes->{'removed'}};
4633- print "Reinstalling removed packages: @pkgs\n" if $main::debug;
4634- print PLOG "Failed to reinstall removed packages!\n"
4635- if !run_apt( "-y", \@instd, \@rmvd, @pkgs );
4636- print "Installed were: @instd\n" if $main::debug;
4637- print "Removed were: @rmvd\n" if $main::debug;
4638- unset_removed( @instd );
4639- unset_installed( @rmvd );
4640-
4641- @pkgs = keys %{$main::changes->{'installed'}};
4642- print "Removing installed packages: @pkgs\n" if $main::debug;
4643- print PLOG "Failed to remove installed packages!\n"
4644- if !uninstall_debs( "purge", @pkgs );
4645- unset_installed( @pkgs );
4646-
4647- unlock_file( "$main::ilock_file" );
4648-}
4649-
4650-sub uninstall_debs {
4651- my $mode = shift;
4652- local (*PIPE);
4653-
4654- return 1 if !@_;
4655- print "Uninstalling packages: @_\n" if $main::debug;
4656- print PLOG " $conf::sudo dpkg --$mode @_\n";
4657- repeat:
4658- my $output;
4659- if (!open( PIPE, "$conf::sudo /usr/sbin/chroot $main::chroot_dir $conf::dpkg --$mode @_ 2>&1 </dev/null |")) {
4660- print PLOG "Can't open pipe to dpkg: $!\n";
4661- return 0;
4662- }
4663- while ( <PIPE> ) {
4664- $output .= $_;
4665- print PLOG $_;
4666- }
4667- close( PIPE );
4668-
4669- if ($output =~ /status database area is locked/mi) {
4670- print PLOG "Another dpkg is running -- retrying later\n";
4671- $output = "";
4672- sleep( 2*60 );
4673- goto repeat;
4674- }
4675- print PLOG "dpkg run to remove packages (@_) failed!\n" if $?;
4676- return $? == 0;
4677-}
4678-
4679-sub undo_specials {
4680- my $sp;
4681-
4682- print "Running post scripts of special dependencies:\n" if $main::debug;
4683- foreach $sp (keys %{$main::changes->{'specials'}}) {
4684- print PLOG "Running post script for $sp\n";
4685- if (run_script( "-e", $main::specials{$sp}->{'post'} ) != 0) {
4686- print PLOG "post script of special dependency $sp failed\n";
4687- }
4688- delete $main::changes->{'specials'}->{$sp};
4689- }
4690- foreach $sp (keys %{$main::changes->{'unpacked'}}) {
4691- my @dirs = @{$main::changes->{'unpacked'}->{$sp}};
4692- print PLOG "Removing special unpacked sources for $sp: @dirs\n";
4693- system "rm", "-rf", @dirs;
4694- delete $main::changes->{'unpacked'}->{$sp};
4695- }
4696-}
4697-
4698-
4699-sub run_apt {
4700- my $mode = shift;
4701- my $inst_ret = shift;
4702- my $rem_ret = shift;
4703- my @to_install = @_;
4704- my( $msgs, $status, $pkgs, $rpkgs );
4705- local (*PIPE);
4706- local (%ENV) = %ENV; # make local environment
4707- # hardwire frontend for debconf to non-interactive
4708- $ENV{'DEBIAN_FRONTEND'} = "noninteractive";
4709-
4710- @$inst_ret = ();
4711- @$rem_ret = ();
4712- return 1 if !@to_install;
4713- repeat:
4714- print PLOG " $conf::sudo $conf::apt_get --purge $main::chroot_apt_op -q $mode install @to_install\n"
4715- if $mode ne "-s";
4716- $msgs = "";
4717- # redirection of stdin from /dev/null so that conffile question are
4718- # treated as if RETURN was pressed.
4719- # dpkg since 1.4.1.18 issues an error on the conffile question if it reads
4720- # EOF -- hardwire the new --force-confold option to avoid the questions.
4721- if (!open( PIPE, "$conf::sudo /usr/sbin/chroot ".
4722- "$main::chroot_dir $conf::apt_get --purge ".
4723- ($main::new_dpkg ? "-o DPkg::Options::=--force-confold " : "").
4724- "-q $mode install @to_install 2>&1 </dev/null |" )) {
4725- print PLOG "Can't open pipe to apt-get: $!\n";
4726- return 0;
4727- }
4728- while( <PIPE> ) {
4729- $msgs .= $_;
4730- print PLOG $_ if $mode ne "-s" || $main::debug;
4731- }
4732- close( PIPE );
4733- $status = $?;
4734-
4735- if ($status != 0 && $msgs =~ /^E: Packages file \S+ (has changed|is out of sync)/mi) {
4736- print PLOG "$conf::sudo $conf::apt_get $main::chroot_apt_op -q update\n";
4737- if (!open( PIPE, "$conf::sudo /usr/sbin/chroot $main::chroot_dir $conf::apt_get -q update 2>&1 |" )) {
4738- print PLOG "Can't open pipe to apt-get: $!\n";
4739- return 0;
4740- }
4741- $msgs = "";
4742- while( <PIPE> ) {
4743- $msgs .= $_;
4744- print PLOG $_;
4745- }
4746- close( PIPE );
4747- print PLOG "apt-get update failed\n" if $?;
4748- $msgs = "";
4749- goto repeat;
4750- }
4751-
4752- if ($status != 0 && $msgs =~ /^Package (\S+) is a virtual package provided by:\n((^\s.*\n)*)/mi) {
4753- my $to_replace = $1;
4754- my @providers;
4755- foreach (split( "\n", $2 )) {
4756- s/^\s*//;
4757- push( @providers, (split( /\s+/, $_ ))[0] );
4758- }
4759- print PLOG "$to_replace is a virtual package provided by: @providers\n";
4760- my $selected;
4761- if (@providers == 1) {
4762- $selected = $providers[0];
4763- print PLOG "Using $selected (only possibility)\n";
4764- }
4765- elsif (exists $conf::alternatives{$to_replace}) {
4766- $selected = $conf::alternatives{$to_replace};
4767- print PLOG "Using $selected (selected in sbuildrc)\n";
4768- }
4769- else {
4770- $selected = $providers[0];
4771- print PLOG "Using $selected (no default, using first one)\n";
4772- }
4773-
4774- @to_install = grep { $_ ne $to_replace } @to_install;
4775- push( @to_install, $selected );
4776-
4777- goto repeat;
4778- }
4779-
4780- if ($status != 0 && ($msgs =~ /^E: Could( not get lock|n.t lock)/mi ||
4781- $msgs =~ /^dpkg: status database area is locked/mi)) {
4782- print PLOG "Another apt-get or dpkg is running -- retrying later\n";
4783- sleep( 2*60 );
4784- goto repeat;
4785- }
4786-
4787- # check for errors that are probably caused by something broken in
4788- # the build environment, and give back the packages.
4789- if ($status != 0 && $mode ne "-s" &&
4790- (($msgs =~ /^E: dpkg was interrupted, you must manually run 'dpkg --configure -a' to correct the problem./mi) ||
4791- ($msgs =~ /^dpkg: parse error, in file `\/.+\/var\/lib\/dpkg\/(?:available|status)' near line/mi) ||
4792- ($msgs =~ /^E: Unmet dependencies. Try 'apt-get -f install' with no packages \(or specify a solution\)\./mi))) {
4793- print PLOG "Build environment unusable, giving back\n";
4794- $main::pkg_fail_stage = "install-deps-env";
4795- }
4796-
4797- if ($status != 0 && $mode ne "-s" &&
4798- (($msgs =~ /^E: Unable to fetch some archives, maybe run apt-get update or try with/mi))) {
4799- print PLOG "Unable to fetch build-depends\n";
4800- $main::pkg_fail_stage = "install-deps-env";
4801- }
4802-
4803- $pkgs = $rpkgs = "";
4804- if ($msgs =~ /NEW packages will be installed:\n((^[ ].*\n)*)/mi) {
4805- ($pkgs = $1) =~ s/^[ ]*((.|\n)*)\s*$/$1/m;
4806- $pkgs =~ s/\*//g;
4807- }
4808- if ($msgs =~ /packages will be REMOVED:\n((^[ ].*\n)*)/mi) {
4809- ($rpkgs = $1) =~ s/^[ ]*((.|\n)*)\s*$/$1/m;
4810- $rpkgs =~ s/\*//g;
4811- }
4812- @$inst_ret = split( /\s+/, $pkgs );
4813- @$rem_ret = split( /\s+/, $rpkgs );
4814-
4815- print PLOG "apt-get failed.\n" if $status && $mode ne "-s";
4816- return $mode eq "-s" || $status == 0;
4817-}
4818-
4819-sub filter_dependencies {
4820- my $dependencies = shift;
4821- my $pos_list = shift;
4822- my $neg_list = shift;
4823- my $special_list = shift;
4824- my($dep, $d, $name, %names);
4825-
4826- print PLOG "Checking for already installed source dependencies...\n";
4827-
4828- @$pos_list = @$neg_list = @$special_list = ();
4829- foreach $d (@$dependencies) {
4830- my $name = $d->{'Package'};
4831- $names{$name} = 1 if $name !~ /^\*/;
4832- foreach (@{$d->{'Alternatives'}}) {
4833- my $name = $_->{'Package'};
4834- $names{$name} = 1 if $name !~ /^\*/;
4835- }
4836- }
4837- my $status = get_dpkg_status( keys %names );
4838-
4839- foreach $dep (@$dependencies) {
4840- $name = $dep->{'Package'};
4841- next if !$name;
4842- if ($name =~ /^\*/) {
4843- my $doit = 1;
4844- if (exists $main::specials{$name}->{'condition'}) {
4845- print "Testing condition for special dependency $name:\n"
4846- if $main::debug;
4847- if (run_script("+e",$main::specials{$name}->{'condition'})!=0){
4848- print "Condition false -> not running scripts\n"
4849- if $main::debug;
4850- $doit = 0;
4851- }
4852- }
4853- push( @$special_list, $name ) if $doit;
4854- next;
4855- }
4856- my $stat = $status->{$name};
4857- if ($dep->{'Neg'}) {
4858- if ($stat->{'Installed'}) {
4859- my ($rel, $vers) = ($dep->{'Rel'}, $dep->{'Version'});
4860- my $ivers = $stat->{'Version'};
4861- if (!$rel || version_cmp( $ivers, $rel, $vers )){
4862- print "$name: neg dep, installed, not versioned or ",
4863- "version relation satisfied --> remove\n" if $main::debug;
4864- print PLOG "$name: installed (negative dependency)";
4865- print PLOG " (bad version $ivers $rel $vers)"
4866- if $rel;
4867- print PLOG "\n";
4868- push( @$neg_list, $name );
4869- }
4870- else {
4871- print PLOG "$name: installed (negative dependency)",
4872- "(but version ok $ivers $rel $vers)\n";
4873- }
4874- }
4875- else {
4876- print "$name: neg dep, not installed\n" if $main::debug;
4877- print PLOG "$name: already deinstalled\n";
4878- }
4879- next;
4880- }
4881-
4882- my $is_satisfied = 0;
4883- my $installable = "";
4884- my $upgradeable = "";
4885- my $downgradeable = "";
4886- foreach $d ($dep, @{$dep->{'Alternatives'}}) {
4887- my ($name, $rel, $vers) =
4888- ($d->{'Package'}, $d->{'Rel'}, $d->{'Version'});
4889- my $stat = $status->{$name};
4890- if (!$stat->{'Installed'}) {
4891- print "$name: pos dep, not installed\n" if $main::debug;
4892- print PLOG "$name: missing\n";
4893- my $exists = package_exists($name);
4894- print PLOG "$name: does not exist\n" if not $exists;
4895- $installable = $name if !$installable and $exists;
4896- next;
4897- }
4898- my $ivers = $stat->{'Version'};
4899- if (!$rel || version_cmp( $ivers, $rel, $vers )) {
4900- print "$name: pos dep, installed, no versioned dep or ",
4901- "version ok\n" if $main::debug;
4902- print PLOG "$name: already installed ($ivers";
4903- print PLOG " $rel $vers is satisfied"
4904- if $rel;
4905- print PLOG ")\n";
4906- $is_satisfied = 1;
4907- last;
4908- }
4909- print "$name: vers dep, installed $ivers ! $rel $vers\n"
4910- if $main::debug;
4911- print PLOG "$name: non-matching version installed ",
4912- "($ivers ! $rel $vers)\n";
4913- if ($rel =~ /^</ ||
4914- ($rel eq '=' && version_cmp($ivers, '>>', $vers))) {
4915- print "$name: would be a downgrade!\n" if $main::debug;
4916- print PLOG "$name: would have to downgrade!\n";
4917- $downgradeable = $name if !$downgradeable;
4918- }
4919- else {
4920- $upgradeable = $name if !$upgradeable;
4921- }
4922- }
4923- if (!$is_satisfied) {
4924- if ($upgradeable) {
4925- print "using $upgradeable for upgrade\n" if $main::debug;
4926- push( @$pos_list, $upgradeable );
4927- }
4928- elsif ($installable) {
4929- print "using $installable for install\n" if $main::debug;
4930- push( @$pos_list, $installable );
4931- }
4932- elsif ($downgradeable) {
4933- print PLOG "To satisfy this dependency the package(s) would ",
4934- "have\n",
4935- "to be downgraded; this is not implemented.\n";
4936- return 0;
4937- }
4938- else {
4939- # None of the build-deps exist. Return the
4940- # first one so that we get a useful dep-wait.
4941- $installable = $dep->{'Package'};
4942- print "using $installable for install (does not exist)\n" if $main::debug;
4943- push( @$pos_list, $installable );
4944- }
4945- }
4946- }
4947-
4948- return 1;
4949-}
4950-
4951-sub check_dependencies {
4952- my $dependencies = shift;
4953- my $fail = "";
4954- my($dep, $d, $name, %names);
4955-
4956- print PLOG "Checking correctness of source dependencies...\n";
4957-
4958- foreach $d (@$dependencies) {
4959- my $name = $d->{'Package'};
4960- $names{$name} = 1 if $name !~ /^\*/;
4961- foreach (@{$d->{'Alternatives'}}) {
4962- my $name = $_->{'Package'};
4963- $names{$name} = 1 if $name !~ /^\*/;
4964- }
4965- }
4966- foreach $name (@main::toolchain_pkgs) {
4967- $names{$name} = 1;
4968- }
4969- my $status = get_dpkg_status( keys %names );
4970-
4971- foreach $dep (@$dependencies) {
4972- $name = $dep->{'Package'};
4973- next if $name =~ /^\*/;
4974- my $stat = $status->{$name};
4975- if ($dep->{'Neg'}) {
4976- if ($stat->{'Installed'}) {
4977- if (!$dep->{'Rel'}) {
4978- $fail .= "$name(still installed) ";
4979- }
4980- elsif (version_cmp($stat->{'Version'}, $dep->{'Rel'},
4981- $dep->{'Version'})) {
4982- $fail .= "$name(inst $stat->{'Version'} $dep->{'Rel'} ".
4983- "conflicted $dep->{'Version'})\n";
4984- }
4985- }
4986- }
4987- else {
4988- my $is_satisfied = 0;
4989- my $f = "";
4990- foreach $d ($dep, @{$dep->{'Alternatives'}}) {
4991- my $name = $d->{'Package'};
4992- my $stat = $status->{$name};
4993- if (!$stat->{'Installed'}) {
4994- $f =~ s/ $/\|/ if $f;
4995- $f .= "$name(missing) ";
4996- }
4997- elsif ($d->{'Rel'} &&
4998- !version_cmp( $stat->{'Version'}, $d->{'Rel'},
4999- $d->{'Version'} )) {
5000- $f =~ s/ $/\|/ if $f;
The diff has been truncated for viewing.