Merge lp:~camptocamp/anybox.buildbot.openerp/trunk-fix_1283084-afe into lp:anybox.buildbot.openerp

Proposed by Alexandre Fayolle - camptocamp
Status: Needs review
Proposed branch: lp:~camptocamp/anybox.buildbot.openerp/trunk-fix_1283084-afe
Merge into: lp:anybox.buildbot.openerp
Diff against target: 302 lines (+73/-163)
1 file modified
buildouts/bootstrap.py (+73/-163)
To merge this branch: bzr merge lp:~camptocamp/anybox.buildbot.openerp/trunk-fix_1283084-afe
Reviewer Review Type Date Requested Status
Leonardo Pistone (community) Needs Information
Anybox Pending
Review via email: mp+207667@code.launchpad.net

Description of the change

update buildouts/bootstrap.py

To post a comment you must log in.
Revision history for this message
Leonardo Pistone (lepistone) wrote :

Thanks Alexandre.

For the record, where exactly did you get the new file from?

review: Needs Information
Revision history for this message
Georges Racinet (gracinet) wrote :

Thanks for the contribution.

Did you run it with exactly the MANIFEST.cfg and the standalone buildouts from that same directory ?
As far as I remember, those actually need the old v1-style bootstrap.

For the record, I'd be glad to clean all this up, and maybe (after a clean release ?) go as far as to make the most up-to-date options for bootstrap (v 2.2.1+virtualenv) the default (makes sense now, I have to tell everyone about them all the time)

Revision history for this message
Alexandre Fayolle - camptocamp (alexandre-fayolle-c2c) wrote :

> Thanks for the contribution.
>
> Did you run it with exactly the MANIFEST.cfg and the standalone buildouts from
> that same directory ?
> As far as I remember, those actually need the old v1-style bootstrap.
>
> For the record, I'd be glad to clean all this up, and maybe (after a clean
> release ?) go as far as to make the most up-to-date options for bootstrap (v
> 2.2.1+virtualenv) the default (makes sense now, I have to tell everyone about
> them all the time)

Honnestly, I don't remember what I exactly did. I've been having issues with the buildbot passing options to the the slaves that were not supported by the bootstrap.py it had installed, and updating bootstrap.py in the master did fix the issue.

Unmerged revisions

299. By Alexandre Fayolle - camptocamp on 2014-02-21

[FIX] buildouts/bootstrap.py : updated bootstrap.py to the latest version

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'buildouts/bootstrap.py'
2--- buildouts/bootstrap.py 2012-04-21 19:57:13 +0000
3+++ buildouts/bootstrap.py 2014-02-21 15:02:50 +0000
4@@ -18,74 +18,14 @@
5 use the -c option to specify an alternate configuration file.
6 """
7
8-import os, shutil, sys, tempfile, textwrap, urllib, urllib2, subprocess
9+import os
10+import shutil
11+import sys
12+import tempfile
13+
14 from optparse import OptionParser
15
16-if sys.platform == 'win32':
17- def quote(c):
18- if ' ' in c:
19- return '"%s"' % c # work around spawn lamosity on windows
20- else:
21- return c
22-else:
23- quote = str
24-
25-# See zc.buildout.easy_install._has_broken_dash_S for motivation and comments.
26-stdout, stderr = subprocess.Popen(
27- [sys.executable, '-Sc',
28- 'try:\n'
29- ' import ConfigParser\n'
30- 'except ImportError:\n'
31- ' print 1\n'
32- 'else:\n'
33- ' print 0\n'],
34- stdout=subprocess.PIPE, stderr=subprocess.PIPE).communicate()
35-has_broken_dash_S = bool(int(stdout.strip()))
36-
37-# In order to be more robust in the face of system Pythons, we want to
38-# run without site-packages loaded. This is somewhat tricky, in
39-# particular because Python 2.6's distutils imports site, so starting
40-# with the -S flag is not sufficient. However, we'll start with that:
41-if not has_broken_dash_S and 'site' in sys.modules:
42- # We will restart with python -S.
43- args = sys.argv[:]
44- args[0:0] = [sys.executable, '-S']
45- args = map(quote, args)
46- os.execv(sys.executable, args)
47-# Now we are running with -S. We'll get the clean sys.path, import site
48-# because distutils will do it later, and then reset the path and clean
49-# out any namespace packages from site-packages that might have been
50-# loaded by .pth files.
51-clean_path = sys.path[:]
52-import site
53-sys.path[:] = clean_path
54-for k, v in sys.modules.items():
55- if k in ('setuptools', 'pkg_resources') or (
56- hasattr(v, '__path__') and
57- len(v.__path__)==1 and
58- not os.path.exists(os.path.join(v.__path__[0],'__init__.py'))):
59- # This is a namespace package. Remove it.
60- sys.modules.pop(k)
61-
62-is_jython = sys.platform.startswith('java')
63-
64-setuptools_source = 'http://peak.telecommunity.com/dist/ez_setup.py'
65-distribute_source = 'http://python-distribute.org/distribute_setup.py'
66-
67-# parsing arguments
68-def normalize_to_url(option, opt_str, value, parser):
69- if value:
70- if '://' not in value: # It doesn't smell like a URL.
71- value = 'file://%s' % (
72- urllib.pathname2url(
73- os.path.abspath(os.path.expanduser(value))),)
74- if opt_str == '--download-base' and not value.endswith('/'):
75- # Download base needs a trailing slash to make the world happy.
76- value += '/'
77- else:
78- value = None
79- name = opt_str[2:].replace('-', '_')
80- setattr(parser.values, name, value)
81+tmpeggs = tempfile.mkdtemp()
82
83 usage = '''\
84 [DESIRED PYTHON FOR BUILDOUT] bootstrap.py [options]
85@@ -95,31 +35,13 @@
86 Simply run this script in a directory containing a buildout.cfg, using the
87 Python that you want bin/buildout to use.
88
89-Note that by using --setup-source and --download-base to point to
90-local resources, you can keep this script from going over the network.
91+Note that by using --find-links to point to local resources, you can keep
92+this script from going over the network.
93 '''
94
95 parser = OptionParser(usage=usage)
96-parser.add_option("-v", "--version", dest="version",
97- help="use a specific zc.buildout version")
98-parser.add_option("-d", "--distribute",
99- action="store_true", dest="use_distribute", default=True,
100- help="Use Distribute rather than Setuptools.")
101-parser.add_option("--setup-source", action="callback", dest="setup_source",
102- callback=normalize_to_url, nargs=1, type="string",
103- help=("Specify a URL or file location for the setup file. "
104- "If you use Setuptools, this will default to " +
105- setuptools_source + "; if you use Distribute, this "
106- "will default to " + distribute_source +"."))
107-parser.add_option("--download-base", action="callback", dest="download_base",
108- callback=normalize_to_url, nargs=1, type="string",
109- help=("Specify a URL or directory for downloading "
110- "zc.buildout and either Setuptools or Distribute. "
111- "Defaults to PyPI."))
112-parser.add_option("--eggs",
113- help=("Specify a directory for storing eggs. Defaults to "
114- "a temporary directory that is deleted when the "
115- "bootstrap script completes."))
116+parser.add_option("-v", "--version", help="use a specific zc.buildout version")
117+
118 parser.add_option("-t", "--accept-buildout-test-releases",
119 dest='accept_buildout_test_releases',
120 action="store_true", default=False,
121@@ -129,49 +51,38 @@
122 "extensions for you. If you use this flag, "
123 "bootstrap and buildout will get the newest releases "
124 "even if they are alphas or betas."))
125-parser.add_option("-c", None, action="store", dest="config_file",
126- help=("Specify the path to the buildout configuration "
127- "file to be used."))
128+parser.add_option("-c", "--config-file",
129+ help=("Specify the path to the buildout configuration "
130+ "file to be used."))
131+parser.add_option("-f", "--find-links",
132+ help=("Specify a URL to search for buildout releases"))
133+
134
135 options, args = parser.parse_args()
136
137-# if -c was provided, we push it back into args for buildout's main function
138-if options.config_file is not None:
139- args += ['-c', options.config_file]
140-
141-if options.eggs:
142- eggs_dir = os.path.abspath(os.path.expanduser(options.eggs))
143-else:
144- eggs_dir = tempfile.mkdtemp()
145-
146-if options.setup_source is None:
147- if options.use_distribute:
148- options.setup_source = distribute_source
149- else:
150- options.setup_source = setuptools_source
151-
152-if options.accept_buildout_test_releases:
153- args.append('buildout:accept-buildout-test-releases=true')
154-args.append('bootstrap')
155-
156+######################################################################
157+# load/install setuptools
158+
159+to_reload = False
160 try:
161 import pkg_resources
162- import setuptools # A flag. Sometimes pkg_resources is installed alone.
163- if not hasattr(pkg_resources, '_distribute'):
164- raise ImportError
165+ import setuptools
166 except ImportError:
167- ez_code = urllib2.urlopen(
168- options.setup_source).read().replace('\r\n', '\n')
169 ez = {}
170- exec ez_code in ez
171- setup_args = dict(to_dir=eggs_dir, download_delay=0)
172- if options.download_base:
173- setup_args['download_base'] = options.download_base
174- if options.use_distribute:
175- setup_args['no_fake'] = True
176+
177+ try:
178+ from urllib.request import urlopen
179+ except ImportError:
180+ from urllib2 import urlopen
181+
182+ # XXX use a more permanent ez_setup.py URL when available.
183+ exec(urlopen('https://bitbucket.org/pypa/setuptools/raw/0.7.2/ez_setup.py'
184+ ).read(), ez)
185+ setup_args = dict(to_dir=tmpeggs, download_delay=0)
186 ez['use_setuptools'](**setup_args)
187- if 'pkg_resources' in sys.modules:
188- reload(sys.modules['pkg_resources'])
189+
190+ if to_reload:
191+ reload(pkg_resources)
192 import pkg_resources
193 # This does not (always?) update the default working set. We will
194 # do it.
195@@ -179,31 +90,26 @@
196 if path not in pkg_resources.working_set.entries:
197 pkg_resources.working_set.add_entry(path)
198
199-cmd = [quote(sys.executable),
200- '-c',
201- quote('from setuptools.command.easy_install import main; main()'),
202- '-mqNxd',
203- quote(eggs_dir)]
204-
205-if not has_broken_dash_S:
206- cmd.insert(1, '-S')
207-
208-find_links = options.download_base
209-if not find_links:
210- find_links = os.environ.get('bootstrap-testing-find-links')
211+######################################################################
212+# Install buildout
213+
214+ws = pkg_resources.working_set
215+
216+cmd = [sys.executable, '-c',
217+ 'from setuptools.command.easy_install import main; main()',
218+ '-mZqNxd', tmpeggs]
219+
220+find_links = os.environ.get(
221+ 'bootstrap-testing-find-links',
222+ options.find_links or
223+ ('http://downloads.buildout.org/'
224+ if options.accept_buildout_test_releases else None)
225+ )
226 if find_links:
227- cmd.extend(['-f', quote(find_links)])
228+ cmd.extend(['-f', find_links])
229
230-if options.use_distribute:
231- setup_requirement = 'distribute'
232-else:
233- setup_requirement = 'setuptools'
234-ws = pkg_resources.working_set
235-setup_requirement_path = ws.find(
236- pkg_resources.Requirement.parse(setup_requirement)).location
237-env = dict(
238- os.environ,
239- PYTHONPATH=setup_requirement_path)
240+setuptools_path = ws.find(
241+ pkg_resources.Requirement.parse('setuptools')).location
242
243 requirement = 'zc.buildout'
244 version = options.version
245@@ -211,13 +117,14 @@
246 # Figure out the most recent final version of zc.buildout.
247 import setuptools.package_index
248 _final_parts = '*final-', '*final'
249+
250 def _final_version(parsed_version):
251 for part in parsed_version:
252 if (part[:1] == '*') and (part not in _final_parts):
253 return False
254 return True
255 index = setuptools.package_index.PackageIndex(
256- search_path=[setup_requirement_path])
257+ search_path=[setuptools_path])
258 if find_links:
259 index.add_find_links((find_links,))
260 req = pkg_resources.Requirement.parse(requirement)
261@@ -239,22 +146,25 @@
262 requirement = '=='.join((requirement, version))
263 cmd.append(requirement)
264
265-if is_jython:
266- import subprocess
267- exitcode = subprocess.Popen(cmd, env=env).wait()
268-else: # Windows prefers this, apparently; otherwise we would prefer subprocess
269- exitcode = os.spawnle(*([os.P_WAIT, sys.executable] + cmd + [env]))
270-if exitcode != 0:
271- sys.stdout.flush()
272- sys.stderr.flush()
273- print ("An error occurred when trying to install zc.buildout. "
274- "Look above this message for any errors that "
275- "were output by easy_install.")
276- sys.exit(exitcode)
277-
278-ws.add_entry(eggs_dir)
279+import subprocess
280+if subprocess.call(cmd, env=dict(os.environ, PYTHONPATH=setuptools_path)) != 0:
281+ raise Exception(
282+ "Failed to execute command:\n%s",
283+ repr(cmd)[1:-1])
284+
285+######################################################################
286+# Import and run buildout
287+
288+ws.add_entry(tmpeggs)
289 ws.require(requirement)
290 import zc.buildout.buildout
291+
292+if not [a for a in args if '=' not in a]:
293+ args.append('bootstrap')
294+
295+# if -c was provided, we push it back into args for buildout' main function
296+if options.config_file is not None:
297+ args[0:0] = ['-c', options.config_file]
298+
299 zc.buildout.buildout.main(args)
300-if not options.eggs: # clean up temporary egg directory
301- shutil.rmtree(eggs_dir)
302+shutil.rmtree(tmpeggs)

Subscribers

People subscribed via source and target branches

to all changes: