Merge lp:~sidnei/lazr-js/newer-buildout into lp:lazr-js

Proposed by Sidnei da Silva
Status: Merged
Approved by: Gary Poster
Approved revision: 173
Merged at revision: not available
Proposed branch: lp:~sidnei/lazr-js/newer-buildout
Merge into: lp:lazr-js
Diff against target: 1085 lines (+610/-384)
6 files modified
Makefile (+3/-2)
bootstrap.py (+126/-103)
distribute_setup.py (+477/-0)
ez_setup.py (+0/-275)
setup.py (+2/-2)
versions.cfg (+2/-2)
To merge this branch: bzr merge lp:~sidnei/lazr-js/newer-buildout
Reviewer Review Type Date Requested Status
Gary Poster (community) Approve
Graham Binns (community) Abstain
Review via email: mp+21417@code.launchpad.net

Description of the change

- Update zc.buildout to 1.4.3
- Replace setuptools by distribute, since the former doesn't work for
  me anymore on Lucid.

To post a comment you must log in.
lp:~sidnei/lazr-js/newer-buildout updated
172. By Sidnei da Silva

- Revert unwanted change

Revision history for this message
Graham Binns (gmb) wrote :

I'm not happy reviewing this branch as I don't have the necessary knowledge. I'm asking Gary to either review this or find someone else who has the necessary contextual knowledge to do so.

review: Abstain
Revision history for this message
Gary Poster (gary) wrote :
Download full text (15.1 KiB)

Hi Sidnei. Looks good, but there's a practical problem in that it doesn't work for me on Karmic. Interesting that setuptools doesn't work for you on Lucid. More below.

The Makefile fails for me on Karmic like this:

bzr checkout lp:lazr-source-dependencies download-cache
mkdir eggs
if test ! -h src-py/lazrjs ; then \
 ln -s ../src-js/lazrjs src-py/lazrjs;\
 fi
python -S bootstrap.py \
  --distribute_setup-source=distribute_setup.py \
  --download-base=download-cache/dist --eggs=eggs
Downloading file:///home/gary/launchpad/lp-sourcedeps/newer-buildout/download-cache/dist/distribute-0.6.10.tar.gz
Extracting in /tmp/tmpTmz0tV
Now working in /tmp/tmpTmz0tV/distribute-0.6.10
Building a Distribute egg in /home/gary/launchpad/lp-sourcedeps/newer-buildout/eggs
/home/gary/launchpad/lp-sourcedeps/newer-buildout/eggs/distribute-0.6.10-py2.6.egg
Traceback (most recent call last):
  File "bootstrap.py", line 147, in <module>
    PYTHONPATH=ws.find(pkg_resources.Requirement.parse('distribute')).location)
AttributeError: 'NoneType' object has no attribute 'location'
make: *** [bin/buildout] Error 1

This is a general problem with buildout and a system Python. I see we are using a hacked version of my old bootstrap.py. I've fixed this particular problem in my buildout branch for Jim. However, when I copy over my new bootstrap.py, I get a new error.

gary@garybuntu:~/launchpad/lp-sourcedeps/newer-buildout$ make
if test ! -h src-py/lazrjs ; then \
 ln -s ../src-js/lazrjs src-py/lazrjs;\
 fi
python -S bootstrap.py \
  --distribute \
                --setup-source=distribute_setup.py \
  --download-base=download-cache/dist --eggs=eggs
Downloading file:///home/gary/launchpad/lp-sourcedeps/newer-buildout/download-cache/dist/distribute-0.6.10.tar.gz
Extracting in /tmp/tmpBtweFh
Now working in /tmp/tmpBtweFh/distribute-0.6.10
Building a Distribute egg in /home/gary/launchpad/lp-sourcedeps/newer-buildout/eggs
/home/gary/launchpad/lp-sourcedeps/newer-buildout/eggs/distribute-0.6.10-py2.6.egg
/home/gary/launchpad/lp-sourcedeps/newer-buildout/bootstrap.py:129: UserWarning: Module pkg_resources was already imported from /usr/lib/python2.6/dist-packages/pkg_resources.pyc, but /home/gary/launchpad/lp-sourcedeps/newer-buildout/eggs/distribute-0.6.10-py2.6.egg is being added to sys.path
  pkg_resources.working_set.add_entry(path)
/home/gary/launchpad/lp-sourcedeps/newer-buildout/bootstrap.py:129: UserWarning: Module site was already imported from /usr/lib/python2.6/site.pyc, but /home/gary/launchpad/lp-sourcedeps/newer-buildout/eggs/distribute-0.6.10-py2.6.egg is being added to sys.path
  pkg_resources.working_set.add_entry(path)
Traceback (most recent call last):
  File "bootstrap.py", line 179, in <module>
    ws.require(requirement)
  File "/usr/lib/python2.6/dist-packages/pkg_resources.py", line 626, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/usr/lib/python2.6/dist-packages/pkg_resources.py", line 524, in resolve
    raise DistributionNotFound(req) # XXX put more info here
pkg_resources.DistributionNotFound: setuptools
make: *** [bin/buildout] Error 1

That's...

review: Needs Fixing
lp:~sidnei/lazr-js/newer-buildout updated
173. By Sidnei da Silva

- Merge from gary's branch

Revision history for this message
Gary Poster (gary) wrote :

You reported my changes continued to work for you on lucid.

Heh, ok, I don't like my sys.modules thing but I don't know anything better, so sure, approved. :-)

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'Makefile'
--- Makefile 2009-12-04 17:55:51 +0000
+++ Makefile 2010-03-16 15:59:22 +0000
@@ -44,8 +44,9 @@
44 ln -s ../src-js/lazrjs src-py/lazrjs;\44 ln -s ../src-js/lazrjs src-py/lazrjs;\
45 fi45 fi
4646
47 $(SHHH) $(PYTHON) -S bootstrap.py\47 $(SHHH) $(PYTHON) -S bootstrap.py \
48 --ez_setup-source=ez_setup.py \48 --distribute \
49 --setup-source=distribute_setup.py \
49 --download-base=download-cache/dist --eggs=eggs50 --download-base=download-cache/dist --eggs=eggs
5051
51$(PY): bin/buildout buildout.cfg setup.py52$(PY): bin/buildout buildout.cfg setup.py
5253
=== modified file 'bootstrap.py'
--- bootstrap.py 2009-10-21 21:43:07 +0000
+++ bootstrap.py 2010-03-16 15:59:22 +0000
@@ -17,106 +17,121 @@
17The script accepts buildout command-line options, so you can17The script accepts buildout command-line options, so you can
18use the -c option to specify an alternate configuration file.18use the -c option to specify an alternate configuration file.
1919
20$Id: bootstrap.py 101930 2009-07-15 18:34:35Z gary $20$Id$
21"""21"""
2222
23import os, re, shutil, sys, tempfile, textwrap, urllib, urllib223import os, shutil, sys, tempfile, textwrap, urllib, urllib2
2424from optparse import OptionParser
25# We have to manually parse our options rather than using one of the stdlib25
26# tools because we want to pass the ones we don't recognize along to26clean_path = sys.path[:]
27# zc.buildout.buildout.main.27import site
2828sys.path[:] = clean_path
29configuration = {29for k, v in sys.modules.items():
30 '--ez_setup-source': 'http://peak.telecommunity.com/dist/ez_setup.py',30 if (hasattr(v, '__path__') and
31 '--version': '',31 len(v.__path__)==1 and
32 '--download-base': None,32 not os.path.exists(os.path.join(v.__path__[0],'__init__.py'))):
33 '--eggs': None}33 # This is a namespace package. Remove it.
3434 sys.modules.pop(k)
35helpstring = __doc__ + textwrap.dedent('''35
36 This script recognizes the following options itself. The first option it36is_jython = sys.platform.startswith('java')
37 encounters that is not one of these will cause the script to stop parsing37
38 options and pass the rest on to buildout. Therefore, if you want to use38setuptools_source = 'http://peak.telecommunity.com/dist/ez_setup.py'
39 any of the following options *and* buildout command-line options like39distribute_source = 'http://python-distribute.org/distribute_setup.py'
40 -c, first use the following options, and then use the buildout options.40
4141# parsing arguments
42 Options: 42def normalize_to_url(option, opt_str, value, parser):
43 --version=ZC_BUILDOUT_VERSION43 if value:
44 Specify a version number of the zc.buildout to use44 if '://' not in value: # It doesn't smell like a URL.
45 --ez_setup-source=URL_OR_FILE45 value = 'file://%s' % (
46 Specify a URL or file location for the ez_setup file.46 urllib.pathname2url(
47 Defaults to47 os.path.abspath(os.path.expanduser(value))),)
48 %(--ez_setup-source)s48 if opt_str == '--download-base' and not value.endswith('/'):
49 --download-base=URL_OR_DIRECTORY49 # Download base needs a trailing slash to make the world happy.
50 Specify a URL or directory for downloading setuptools and50 value += '/'
51 zc.buildout. Defaults to PyPI.
52 --eggs=DIRECTORY
53 Specify a directory for storing eggs. Defaults to a temporary
54 directory that is deleted when the bootstrap script completes.
55
56 By using --ez_setup-source and --download-base to point to local resources,
57 you can keep this script from going over the network.
58 ''' % configuration)
59match_equals = re.compile(r'(%s)=(.*)' % ('|'.join(configuration),)).match
60args = sys.argv[1:]
61if args == ['--help']:
62 print helpstring
63 sys.exit(0)
64
65# If we end up using a temporary directory for storing our eggs, this will
66# hold the path of that directory. On the other hand, if an explicit directory
67# is specified in the argv, this will remain None.
68tmpeggs = None
69
70while args:
71 val = args[0]
72 if val in configuration:
73 del args[0]
74 if not args or args[0].startswith('-'):
75 print "ERROR: %s requires an argument."
76 print helpstring
77 sys.exit(1)
78 configuration[val] = args[0]
79 else:51 else:
80 match = match_equals(val)52 value = None
81 if match and match.group(1) in configuration:53 name = opt_str[2:].replace('-', '_')
82 configuration[match.group(1)] = match.group(2)54 setattr(parser.values, name, value)
83 else:55
84 break56usage = '''\
85 del args[0]57[DESIRED PYTHON FOR BUILDOUT] bootstrap.py [options]
8658
87for name in ('--ez_setup-source', '--download-base'):59Bootstraps a buildout-based project.
88 val = configuration[name]60
89 if val is not None and '://' not in val: # We're being lazy.61Simply run this script in a directory containing a buildout.cfg, using the
90 configuration[name] = 'file://%s' % (62Python that you want bin/buildout to use.
91 urllib.pathname2url(os.path.abspath(os.path.expanduser(val))),)63
9264Note that by using --setup-source and --download-base to point to
93if (configuration['--download-base'] and65local resources, you can keep this script from going over the network.
94 not configuration['--download-base'].endswith('/')):66'''
95 # Download base needs a trailing slash to make the world happy.67
96 configuration['--download-base'] += '/'68parser = OptionParser(usage=usage)
9769parser.add_option("-v", "--version", dest="version",
98if not configuration['--eggs']:70 help="use a specific zc.buildout version")
99 configuration['--eggs'] = tmpeggs = tempfile.mkdtemp()71parser.add_option("-d", "--distribute",
72 action="store_true", dest="use_distribute", default=False,
73 help="Use Distribute rather than Setuptools.")
74parser.add_option("--setup-source", action="callback", dest="setup_source",
75 callback=normalize_to_url, nargs=1, type="string",
76 help=("Specify a URL or file location for the setup file. "
77 "If you use Setuptools, this will default to " +
78 setuptools_source + "; if you use Distribute, this "
79 "will default to " + distribute_source +"."))
80parser.add_option("--download-base", action="callback", dest="download_base",
81 callback=normalize_to_url, nargs=1, type="string",
82 help=("Specify a URL or directory for downloading "
83 "zc.buildout and either Setuptools or Distribute. "
84 "Defaults to PyPI."))
85parser.add_option("--eggs",
86 help=("Specify a directory for storing eggs. Defaults to "
87 "a temporary directory that is deleted when the "
88 "bootstrap script completes."))
89parser.add_option("-c", None, action="store", dest="config_file",
90 help=("Specify the path to the buildout configuration "
91 "file to be used."))
92
93options, args = parser.parse_args()
94
95# if -c was provided, we push it back into args for buildout' main function
96if options.config_file is not None:
97 args += ['-c', options.config_file]
98
99if options.eggs:
100 eggs_dir = os.path.abspath(os.path.expanduser(options.eggs))
100else:101else:
101 configuration['--eggs'] = os.path.abspath(102 eggs_dir = tempfile.mkdtemp()
102 os.path.expanduser(configuration['--eggs']))103
103104if options.setup_source is None:
104# The requirement is what we will pass to setuptools to specify zc.buildout.105 if options.use_distribute:
105requirement = 'zc.buildout'106 options.setup_source = distribute_source
106if configuration['--version']:107 else:
107 requirement += '==' + configuration['--version']108 options.setup_source = setuptools_source
109
110args = args + ['bootstrap']
111
108112
109try:113try:
114 to_reload = False
115 import pkg_resources
116 to_reload = True
117 if not hasattr(pkg_resources, '_distribute'):
118 raise ImportError
110 import setuptools # A flag. Sometimes pkg_resources is installed alone.119 import setuptools # A flag. Sometimes pkg_resources is installed alone.
111 import pkg_resources
112except ImportError:120except ImportError:
121 ez_code = urllib2.urlopen(
122 options.setup_source).read().replace('\r\n', '\n')
113 ez = {}123 ez = {}
114 exec urllib2.urlopen(configuration['--ez_setup-source']).read() in ez124 exec ez_code in ez
115 setuptools_args = dict(to_dir=configuration['--eggs'], download_delay=0)125 setup_args = dict(to_dir=eggs_dir, download_delay=0)
116 if configuration['--download-base']:126 if options.download_base:
117 setuptools_args['download_base'] = configuration['--download-base']127 setup_args['download_base'] = options.download_base
118 ez['use_setuptools'](**setuptools_args)128 if options.use_distribute:
119 import pkg_resources129 setup_args['no_fake'] = True
130 ez['use_setuptools'](**setup_args)
131 if to_reload:
132 reload(pkg_resources)
133 else:
134 import pkg_resources
120 # This does not (always?) update the default working set. We will135 # This does not (always?) update the default working set. We will
121 # do it.136 # do it.
122 for path in sys.path:137 for path in sys.path:
@@ -132,39 +147,47 @@
132else:147else:
133 def quote (c):148 def quote (c):
134 return c149 return c
150
135cmd = [quote(sys.executable),151cmd = [quote(sys.executable),
136 '-c',152 '-c',
137 quote('from setuptools.command.easy_install import main; main()'),153 quote('from setuptools.command.easy_install import main; main()'),
138 '-mqNxd',154 '-mqNxd',
139 quote(configuration['--eggs'])]155 quote(eggs_dir)]
140156
141if configuration['--download-base']:157if options.download_base:
142 cmd.extend(['-f', quote(configuration['--download-base'])])158 cmd.extend(['-f', quote(options.download_base)])
143159
160requirement = 'zc.buildout'
161if options.version:
162 requirement = '=='.join((requirement, options.version))
144cmd.append(requirement)163cmd.append(requirement)
145164
165if options.use_distribute:
166 setup_requirement = 'distribute'
167else:
168 setup_requirement = 'setuptools'
146ws = pkg_resources.working_set169ws = pkg_resources.working_set
147env = dict(170env = dict(
148 os.environ,171 os.environ,
149 PYTHONPATH=ws.find(pkg_resources.Requirement.parse('setuptools')).location)172 PYTHONPATH=ws.find(
173 pkg_resources.Requirement.parse(setup_requirement)).location)
150174
151is_jython = sys.platform.startswith('java')
152if is_jython:175if is_jython:
153 import subprocess176 import subprocess
154 exitcode = subprocess.Popen(cmd, env=env).wait()177 exitcode = subprocess.Popen(cmd, env=env).wait()
155else: # Windows needs this, apparently; otherwise we would prefer subprocess178else: # Windows prefers this, apparently; otherwise we would prefer subprocess
156 exitcode = os.spawnle(*([os.P_WAIT, sys.executable] + cmd + [env]))179 exitcode = os.spawnle(*([os.P_WAIT, sys.executable] + cmd + [env]))
157if exitcode != 0:180if exitcode != 0:
158 sys.stdout.flush()181 sys.stdout.flush()
182 sys.stderr.flush()
159 print ("An error occured when trying to install zc.buildout. "183 print ("An error occured when trying to install zc.buildout. "
160 "Look above this message for any errors that "184 "Look above this message for any errors that "
161 "were output by easy_install.")185 "were output by easy_install.")
162 sys.exit(exitcode)186 sys.exit(exitcode)
163187
164ws.add_entry(configuration['--eggs'])188ws.add_entry(eggs_dir)
165ws.require(requirement)189ws.require(requirement)
166import zc.buildout.buildout190import zc.buildout.buildout
167args.append('bootstrap')
168zc.buildout.buildout.main(args)191zc.buildout.buildout.main(args)
169if tmpeggs is not None:192if not options.eggs: # clean up temporary egg directory
170 shutil.rmtree(tmpeggs)193 shutil.rmtree(eggs_dir)
171194
=== added file 'distribute_setup.py'
--- distribute_setup.py 1970-01-01 00:00:00 +0000
+++ distribute_setup.py 2010-03-16 15:59:22 +0000
@@ -0,0 +1,477 @@
1#!python
2"""Bootstrap distribute installation
3
4If you want to use setuptools in your package's setup.py, just include this
5file in the same directory with it, and add this to the top of your setup.py::
6
7 from distribute_setup import use_setuptools
8 use_setuptools()
9
10If you want to require a specific version of setuptools, set a download
11mirror, or use an alternate download directory, you can do so by supplying
12the appropriate options to ``use_setuptools()``.
13
14This file can also be run as a script to install or upgrade setuptools.
15"""
16import os
17import sys
18import time
19import fnmatch
20import tempfile
21import tarfile
22from distutils import log
23
24try:
25 from site import USER_SITE
26except ImportError:
27 USER_SITE = None
28
29try:
30 import subprocess
31
32 def _python_cmd(*args):
33 args = (sys.executable,) + args
34 return subprocess.call(args) == 0
35
36except ImportError:
37 # will be used for python 2.3
38 def _python_cmd(*args):
39 args = (sys.executable,) + args
40 # quoting arguments if windows
41 if sys.platform == 'win32':
42 def quote(arg):
43 if ' ' in arg:
44 return '"%s"' % arg
45 return arg
46 args = [quote(arg) for arg in args]
47 return os.spawnl(os.P_WAIT, sys.executable, *args) == 0
48
49DEFAULT_VERSION = "0.6.10"
50DEFAULT_URL = "http://pypi.python.org/packages/source/d/distribute/"
51SETUPTOOLS_FAKED_VERSION = "0.6c11"
52
53SETUPTOOLS_PKG_INFO = """\
54Metadata-Version: 1.0
55Name: setuptools
56Version: %s
57Summary: xxxx
58Home-page: xxx
59Author: xxx
60Author-email: xxx
61License: xxx
62Description: xxx
63""" % SETUPTOOLS_FAKED_VERSION
64
65
66def _install(tarball):
67 # extracting the tarball
68 tmpdir = tempfile.mkdtemp()
69 log.warn('Extracting in %s', tmpdir)
70 old_wd = os.getcwd()
71 try:
72 os.chdir(tmpdir)
73 tar = tarfile.open(tarball)
74 _extractall(tar)
75 tar.close()
76
77 # going in the directory
78 subdir = os.path.join(tmpdir, os.listdir(tmpdir)[0])
79 os.chdir(subdir)
80 log.warn('Now working in %s', subdir)
81
82 # installing
83 log.warn('Installing Distribute')
84 if not _python_cmd('setup.py', 'install'):
85 log.warn('Something went wrong during the installation.')
86 log.warn('See the error message above.')
87 finally:
88 os.chdir(old_wd)
89
90
91def _build_egg(egg, tarball, to_dir):
92 # extracting the tarball
93 tmpdir = tempfile.mkdtemp()
94 log.warn('Extracting in %s', tmpdir)
95 old_wd = os.getcwd()
96 try:
97 os.chdir(tmpdir)
98 tar = tarfile.open(tarball)
99 _extractall(tar)
100 tar.close()
101
102 # going in the directory
103 subdir = os.path.join(tmpdir, os.listdir(tmpdir)[0])
104 os.chdir(subdir)
105 log.warn('Now working in %s', subdir)
106
107 # building an egg
108 log.warn('Building a Distribute egg in %s', to_dir)
109 _python_cmd('setup.py', '-q', 'bdist_egg', '--dist-dir', to_dir)
110
111 finally:
112 os.chdir(old_wd)
113 # returning the result
114 log.warn(egg)
115 if not os.path.exists(egg):
116 raise IOError('Could not build the egg.')
117
118
119def _do_download(version, download_base, to_dir, download_delay):
120 egg = os.path.join(to_dir, 'distribute-%s-py%d.%d.egg'
121 % (version, sys.version_info[0], sys.version_info[1]))
122 if not os.path.exists(egg):
123 tarball = download_setuptools(version, download_base,
124 to_dir, download_delay)
125 _build_egg(egg, tarball, to_dir)
126 sys.path.insert(0, egg)
127 import setuptools
128 setuptools.bootstrap_install_from = egg
129
130
131def use_setuptools(version=DEFAULT_VERSION, download_base=DEFAULT_URL,
132 to_dir=os.curdir, download_delay=15, no_fake=True):
133 # making sure we use the absolute path
134 to_dir = os.path.abspath(to_dir)
135 was_imported = 'pkg_resources' in sys.modules or \
136 'setuptools' in sys.modules
137 try:
138 try:
139 import pkg_resources
140 if not hasattr(pkg_resources, '_distribute'):
141 if not no_fake:
142 _fake_setuptools()
143 raise ImportError
144 except ImportError:
145 return _do_download(version, download_base, to_dir, download_delay)
146 try:
147 pkg_resources.require("distribute>="+version)
148 return
149 except pkg_resources.VersionConflict:
150 e = sys.exc_info()[1]
151 if was_imported:
152 sys.stderr.write(
153 "The required version of distribute (>=%s) is not available,\n"
154 "and can't be installed while this script is running. Please\n"
155 "install a more recent version first, using\n"
156 "'easy_install -U distribute'."
157 "\n\n(Currently using %r)\n" % (version, e.args[0]))
158 sys.exit(2)
159 else:
160 del pkg_resources, sys.modules['pkg_resources'] # reload ok
161 return _do_download(version, download_base, to_dir,
162 download_delay)
163 except pkg_resources.DistributionNotFound:
164 return _do_download(version, download_base, to_dir,
165 download_delay)
166 finally:
167 if not no_fake:
168 _create_fake_setuptools_pkg_info(to_dir)
169
170def download_setuptools(version=DEFAULT_VERSION, download_base=DEFAULT_URL,
171 to_dir=os.curdir, delay=15):
172 """Download distribute from a specified location and return its filename
173
174 `version` should be a valid distribute version number that is available
175 as an egg for download under the `download_base` URL (which should end
176 with a '/'). `to_dir` is the directory where the egg will be downloaded.
177 `delay` is the number of seconds to pause before an actual download
178 attempt.
179 """
180 # making sure we use the absolute path
181 to_dir = os.path.abspath(to_dir)
182 try:
183 from urllib.request import urlopen
184 except ImportError:
185 from urllib2 import urlopen
186 tgz_name = "distribute-%s.tar.gz" % version
187 url = download_base + tgz_name
188 saveto = os.path.join(to_dir, tgz_name)
189 src = dst = None
190 if not os.path.exists(saveto): # Avoid repeated downloads
191 try:
192 log.warn("Downloading %s", url)
193 src = urlopen(url)
194 # Read/write all in one block, so we don't create a corrupt file
195 # if the download is interrupted.
196 data = src.read()
197 dst = open(saveto, "wb")
198 dst.write(data)
199 finally:
200 if src:
201 src.close()
202 if dst:
203 dst.close()
204 return os.path.realpath(saveto)
205
206
207def _patch_file(path, content):
208 """Will backup the file then patch it"""
209 existing_content = open(path).read()
210 if existing_content == content:
211 # already patched
212 log.warn('Already patched.')
213 return False
214 log.warn('Patching...')
215 _rename_path(path)
216 f = open(path, 'w')
217 try:
218 f.write(content)
219 finally:
220 f.close()
221 return True
222
223
224def _same_content(path, content):
225 return open(path).read() == content
226
227def _no_sandbox(function):
228 def __no_sandbox(*args, **kw):
229 try:
230 from setuptools.sandbox import DirectorySandbox
231 def violation(*args):
232 pass
233 DirectorySandbox._old = DirectorySandbox._violation
234 DirectorySandbox._violation = violation
235 patched = True
236 except ImportError:
237 patched = False
238
239 try:
240 return function(*args, **kw)
241 finally:
242 if patched:
243 DirectorySandbox._violation = DirectorySandbox._old
244 del DirectorySandbox._old
245
246 return __no_sandbox
247
248@_no_sandbox
249def _rename_path(path):
250 new_name = path + '.OLD.%s' % time.time()
251 log.warn('Renaming %s into %s', path, new_name)
252 os.rename(path, new_name)
253 return new_name
254
255def _remove_flat_installation(placeholder):
256 if not os.path.isdir(placeholder):
257 log.warn('Unkown installation at %s', placeholder)
258 return False
259 found = False
260 for file in os.listdir(placeholder):
261 if fnmatch.fnmatch(file, 'setuptools*.egg-info'):
262 found = True
263 break
264 if not found:
265 log.warn('Could not locate setuptools*.egg-info')
266 return
267
268 log.warn('Removing elements out of the way...')
269 pkg_info = os.path.join(placeholder, file)
270 if os.path.isdir(pkg_info):
271 patched = _patch_egg_dir(pkg_info)
272 else:
273 patched = _patch_file(pkg_info, SETUPTOOLS_PKG_INFO)
274
275 if not patched:
276 log.warn('%s already patched.', pkg_info)
277 return False
278 # now let's move the files out of the way
279 for element in ('setuptools', 'pkg_resources.py', 'site.py'):
280 element = os.path.join(placeholder, element)
281 if os.path.exists(element):
282 _rename_path(element)
283 else:
284 log.warn('Could not find the %s element of the '
285 'Setuptools distribution', element)
286 return True
287
288
289def _after_install(dist):
290 log.warn('After install bootstrap.')
291 placeholder = dist.get_command_obj('install').install_purelib
292 _create_fake_setuptools_pkg_info(placeholder)
293
294@_no_sandbox
295def _create_fake_setuptools_pkg_info(placeholder):
296 if not placeholder or not os.path.exists(placeholder):
297 log.warn('Could not find the install location')
298 return
299 pyver = '%s.%s' % (sys.version_info[0], sys.version_info[1])
300 setuptools_file = 'setuptools-%s-py%s.egg-info' % \
301 (SETUPTOOLS_FAKED_VERSION, pyver)
302 pkg_info = os.path.join(placeholder, setuptools_file)
303 if os.path.exists(pkg_info):
304 log.warn('%s already exists', pkg_info)
305 return
306
307 log.warn('Creating %s', pkg_info)
308 f = open(pkg_info, 'w')
309 try:
310 f.write(SETUPTOOLS_PKG_INFO)
311 finally:
312 f.close()
313
314 pth_file = os.path.join(placeholder, 'setuptools.pth')
315 log.warn('Creating %s', pth_file)
316 f = open(pth_file, 'w')
317 try:
318 f.write(os.path.join(os.curdir, setuptools_file))
319 finally:
320 f.close()
321
322def _patch_egg_dir(path):
323 # let's check if it's already patched
324 pkg_info = os.path.join(path, 'EGG-INFO', 'PKG-INFO')
325 if os.path.exists(pkg_info):
326 if _same_content(pkg_info, SETUPTOOLS_PKG_INFO):
327 log.warn('%s already patched.', pkg_info)
328 return False
329 _rename_path(path)
330 os.mkdir(path)
331 os.mkdir(os.path.join(path, 'EGG-INFO'))
332 pkg_info = os.path.join(path, 'EGG-INFO', 'PKG-INFO')
333 f = open(pkg_info, 'w')
334 try:
335 f.write(SETUPTOOLS_PKG_INFO)
336 finally:
337 f.close()
338 return True
339
340
341def _before_install():
342 log.warn('Before install bootstrap.')
343 _fake_setuptools()
344
345
346def _under_prefix(location):
347 if 'install' not in sys.argv:
348 return True
349 args = sys.argv[sys.argv.index('install')+1:]
350 for index, arg in enumerate(args):
351 for option in ('--root', '--prefix'):
352 if arg.startswith('%s=' % option):
353 top_dir = arg.split('root=')[-1]
354 return location.startswith(top_dir)
355 elif arg == option:
356 if len(args) > index:
357 top_dir = args[index+1]
358 return location.startswith(top_dir)
359 elif option == '--user' and USER_SITE is not None:
360 return location.startswith(USER_SITE)
361 return True
362
363
364def _fake_setuptools():
365 log.warn('Scanning installed packages')
366 try:
367 import pkg_resources
368 except ImportError:
369 # we're cool
370 log.warn('Setuptools or Distribute does not seem to be installed.')
371 return
372 ws = pkg_resources.working_set
373 try:
374 setuptools_dist = ws.find(pkg_resources.Requirement.parse('setuptools',
375 replacement=False))
376 except TypeError:
377 # old distribute API
378 setuptools_dist = ws.find(pkg_resources.Requirement.parse('setuptools'))
379
380 if setuptools_dist is None:
381 log.warn('No setuptools distribution found')
382 return
383 # detecting if it was already faked
384 setuptools_location = setuptools_dist.location
385 log.warn('Setuptools installation detected at %s', setuptools_location)
386
387 # if --root or --preix was provided, and if
388 # setuptools is not located in them, we don't patch it
389 if not _under_prefix(setuptools_location):
390 log.warn('Not patching, --root or --prefix is installing Distribute'
391 ' in another location')
392 return
393
394 # let's see if its an egg
395 if not setuptools_location.endswith('.egg'):
396 log.warn('Non-egg installation')
397 res = _remove_flat_installation(setuptools_location)
398 if not res:
399 return
400 else:
401 log.warn('Egg installation')
402 pkg_info = os.path.join(setuptools_location, 'EGG-INFO', 'PKG-INFO')
403 if (os.path.exists(pkg_info) and
404 _same_content(pkg_info, SETUPTOOLS_PKG_INFO)):
405 log.warn('Already patched.')
406 return
407 log.warn('Patching...')
408 # let's create a fake egg replacing setuptools one
409 res = _patch_egg_dir(setuptools_location)
410 if not res:
411 return
412 log.warn('Patched done.')
413 _relaunch()
414
415
416def _relaunch():
417 log.warn('Relaunching...')
418 # we have to relaunch the process
419 args = [sys.executable] + sys.argv
420 sys.exit(subprocess.call(args))
421
422
423def _extractall(self, path=".", members=None):
424 """Extract all members from the archive to the current working
425 directory and set owner, modification time and permissions on
426 directories afterwards. `path' specifies a different directory
427 to extract to. `members' is optional and must be a subset of the
428 list returned by getmembers().
429 """
430 import copy
431 import operator
432 from tarfile import ExtractError
433 directories = []
434
435 if members is None:
436 members = self
437
438 for tarinfo in members:
439 if tarinfo.isdir():
440 # Extract directories with a safe mode.
441 directories.append(tarinfo)
442 tarinfo = copy.copy(tarinfo)
443 tarinfo.mode = 448 # decimal for oct 0700
444 self.extract(tarinfo, path)
445
446 # Reverse sort directories.
447 if sys.version_info < (2, 4):
448 def sorter(dir1, dir2):
449 return cmp(dir1.name, dir2.name)
450 directories.sort(sorter)
451 directories.reverse()
452 else:
453 directories.sort(key=operator.attrgetter('name'), reverse=True)
454
455 # Set correct owner, mtime and filemode on directories.
456 for tarinfo in directories:
457 dirpath = os.path.join(path, tarinfo.name)
458 try:
459 self.chown(tarinfo, dirpath)
460 self.utime(tarinfo, dirpath)
461 self.chmod(tarinfo, dirpath)
462 except ExtractError:
463 e = sys.exc_info()[1]
464 if self.errorlevel > 1:
465 raise
466 else:
467 self._dbg(1, "tarfile: %s" % e)
468
469
470def main(argv, version=DEFAULT_VERSION):
471 """Install or upgrade setuptools and EasyInstall"""
472 tarball = download_setuptools()
473 _install(tarball)
474
475
476if __name__ == '__main__':
477 main(sys.argv[1:])
0478
=== removed file 'ez_setup.py'
--- ez_setup.py 2009-10-21 21:43:07 +0000
+++ ez_setup.py 1970-01-01 00:00:00 +0000
@@ -1,275 +0,0 @@
1#!python
2"""Bootstrap setuptools installation
3
4If you want to use setuptools in your package's setup.py, just include this
5file in the same directory with it, and add this to the top of your setup.py::
6
7 from ez_setup import use_setuptools
8 use_setuptools()
9
10If you want to require a specific version of setuptools, set a download
11mirror, or use an alternate download directory, you can do so by supplying
12the appropriate options to ``use_setuptools()``.
13
14This file can also be run as a script to install or upgrade setuptools.
15"""
16import sys
17DEFAULT_VERSION = "0.6c9"
18DEFAULT_URL = "http://pypi.python.org/packages/%s/s/setuptools/" % sys.version[:3]
19
20md5_data = {
21 'setuptools-0.6b1-py2.3.egg': '8822caf901250d848b996b7f25c6e6ca',
22 'setuptools-0.6b1-py2.4.egg': 'b79a8a403e4502fbb85ee3f1941735cb',
23 'setuptools-0.6b2-py2.3.egg': '5657759d8a6d8fc44070a9d07272d99b',
24 'setuptools-0.6b2-py2.4.egg': '4996a8d169d2be661fa32a6e52e4f82a',
25 'setuptools-0.6b3-py2.3.egg': 'bb31c0fc7399a63579975cad9f5a0618',
26 'setuptools-0.6b3-py2.4.egg': '38a8c6b3d6ecd22247f179f7da669fac',
27 'setuptools-0.6b4-py2.3.egg': '62045a24ed4e1ebc77fe039aa4e6f7e5',
28 'setuptools-0.6b4-py2.4.egg': '4cb2a185d228dacffb2d17f103b3b1c4',
29 'setuptools-0.6c1-py2.3.egg': 'b3f2b5539d65cb7f74ad79127f1a908c',
30 'setuptools-0.6c1-py2.4.egg': 'b45adeda0667d2d2ffe14009364f2a4b',
31 'setuptools-0.6c2-py2.3.egg': 'f0064bf6aa2b7d0f3ba0b43f20817c27',
32 'setuptools-0.6c2-py2.4.egg': '616192eec35f47e8ea16cd6a122b7277',
33 'setuptools-0.6c3-py2.3.egg': 'f181fa125dfe85a259c9cd6f1d7b78fa',
34 'setuptools-0.6c3-py2.4.egg': 'e0ed74682c998bfb73bf803a50e7b71e',
35 'setuptools-0.6c3-py2.5.egg': 'abef16fdd61955514841c7c6bd98965e',
36 'setuptools-0.6c4-py2.3.egg': 'b0b9131acab32022bfac7f44c5d7971f',
37 'setuptools-0.6c4-py2.4.egg': '2a1f9656d4fbf3c97bf946c0a124e6e2',
38 'setuptools-0.6c4-py2.5.egg': '8f5a052e32cdb9c72bcf4b5526f28afc',
39 'setuptools-0.6c5-py2.3.egg': 'ee9fd80965da04f2f3e6b3576e9d8167',
40 'setuptools-0.6c5-py2.4.egg': 'afe2adf1c01701ee841761f5bcd8aa64',
41 'setuptools-0.6c5-py2.5.egg': 'a8d3f61494ccaa8714dfed37bccd3d5d',
42 'setuptools-0.6c6-py2.3.egg': '35686b78116a668847237b69d549ec20',
43 'setuptools-0.6c6-py2.4.egg': '3c56af57be3225019260a644430065ab',
44 'setuptools-0.6c6-py2.5.egg': 'b2f8a7520709a5b34f80946de5f02f53',
45 'setuptools-0.6c7-py2.3.egg': '209fdf9adc3a615e5115b725658e13e2',
46 'setuptools-0.6c7-py2.4.egg': '5a8f954807d46a0fb67cf1f26c55a82e',
47 'setuptools-0.6c7-py2.5.egg': '45d2ad28f9750e7434111fde831e8372',
48 'setuptools-0.6c8-py2.3.egg': '50759d29b349db8cfd807ba8303f1902',
49 'setuptools-0.6c8-py2.4.egg': 'cba38d74f7d483c06e9daa6070cce6de',
50 'setuptools-0.6c8-py2.5.egg': '1721747ee329dc150590a58b3e1ac95b',
51 'setuptools-0.6c9-py2.3.egg': 'a83c4020414807b496e4cfbe08507c03',
52 'setuptools-0.6c9-py2.4.egg': '260a2be2e5388d66bdaee06abec6342a',
53 'setuptools-0.6c9-py2.5.egg': 'fe67c3e5a17b12c0e7c541b7ea43a8e6',
54 'setuptools-0.6c9-py2.6.egg': 'ca37b1ff16fa2ede6e19383e7b59245a',
55}
56
57import sys, os
58try: from hashlib import md5
59except ImportError: from md5 import md5
60
61def _validate_md5(egg_name, data):
62 if egg_name in md5_data:
63 digest = md5(data).hexdigest()
64 if digest != md5_data[egg_name]:
65 print >>sys.stderr, (
66 "md5 validation of %s failed! (Possible download problem?)"
67 % egg_name
68 )
69 sys.exit(2)
70 return data
71
72def use_setuptools(
73 version=DEFAULT_VERSION, download_base=DEFAULT_URL, to_dir=os.curdir,
74 download_delay=15
75):
76 """Automatically find/download setuptools and make it available on sys.path
77
78 `version` should be a valid setuptools version number that is available
79 as an egg for download under the `download_base` URL (which should end with
80 a '/'). `to_dir` is the directory where setuptools will be downloaded, if
81 it is not already available. If `download_delay` is specified, it should
82 be the number of seconds that will be paused before initiating a download,
83 should one be required. If an older version of setuptools is installed,
84 this routine will print a message to ``sys.stderr`` and raise SystemExit in
85 an attempt to abort the calling script.
86 """
87 was_imported = 'pkg_resources' in sys.modules or 'setuptools' in sys.modules
88 def do_download():
89 egg = download_setuptools(version, download_base, to_dir, download_delay)
90 sys.path.insert(0, egg)
91 import setuptools; setuptools.bootstrap_install_from = egg
92 try:
93 import pkg_resources
94 except ImportError:
95 return do_download()
96 try:
97 pkg_resources.require("setuptools>="+version); return
98 except pkg_resources.VersionConflict, e:
99 if was_imported:
100 print >>sys.stderr, (
101 "The required version of setuptools (>=%s) is not available, and\n"
102 "can't be installed while this script is running. Please install\n"
103 " a more recent version first, using 'easy_install -U setuptools'."
104 "\n\n(Currently using %r)"
105 ) % (version, e.args[0])
106 sys.exit(2)
107 else:
108 del pkg_resources, sys.modules['pkg_resources'] # reload ok
109 return do_download()
110 except pkg_resources.DistributionNotFound:
111 return do_download()
112
113def download_setuptools(
114 version=DEFAULT_VERSION, download_base=DEFAULT_URL, to_dir=os.curdir,
115 delay = 15
116):
117 """Download setuptools from a specified location and return its filename
118
119 `version` should be a valid setuptools version number that is available
120 as an egg for download under the `download_base` URL (which should end
121 with a '/'). `to_dir` is the directory where the egg will be downloaded.
122 `delay` is the number of seconds to pause before an actual download attempt.
123 """
124 import urllib2, shutil
125 egg_name = "setuptools-%s-py%s.egg" % (version,sys.version[:3])
126 url = download_base + egg_name
127 saveto = os.path.join(to_dir, egg_name)
128 src = dst = None
129 if not os.path.exists(saveto): # Avoid repeated downloads
130 try:
131 from distutils import log
132 if delay:
133 log.warn("""
134---------------------------------------------------------------------------
135This script requires setuptools version %s to run (even to display
136help). I will attempt to download it for you (from
137%s), but
138you may need to enable firewall access for this script first.
139I will start the download in %d seconds.
140
141(Note: if this machine does not have network access, please obtain the file
142
143 %s
144
145and place it in this directory before rerunning this script.)
146---------------------------------------------------------------------------""",
147 version, download_base, delay, url
148 ); from time import sleep; sleep(delay)
149 log.warn("Downloading %s", url)
150 src = urllib2.urlopen(url)
151 # Read/write all in one block, so we don't create a corrupt file
152 # if the download is interrupted.
153 data = _validate_md5(egg_name, src.read())
154 dst = open(saveto,"wb"); dst.write(data)
155 finally:
156 if src: src.close()
157 if dst: dst.close()
158 return os.path.realpath(saveto)
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195def main(argv, version=DEFAULT_VERSION):
196 """Install or upgrade setuptools and EasyInstall"""
197 try:
198 import setuptools
199 except ImportError:
200 egg = None
201 try:
202 egg = download_setuptools(version, delay=0)
203 sys.path.insert(0,egg)
204 from setuptools.command.easy_install import main
205 return main(list(argv)+[egg]) # we're done here
206 finally:
207 if egg and os.path.exists(egg):
208 os.unlink(egg)
209 else:
210 if setuptools.__version__ == '0.0.1':
211 print >>sys.stderr, (
212 "You have an obsolete version of setuptools installed. Please\n"
213 "remove it from your system entirely before rerunning this script."
214 )
215 sys.exit(2)
216
217 req = "setuptools>="+version
218 import pkg_resources
219 try:
220 pkg_resources.require(req)
221 except pkg_resources.VersionConflict:
222 try:
223 from setuptools.command.easy_install import main
224 except ImportError:
225 from easy_install import main
226 main(list(argv)+[download_setuptools(delay=0)])
227 sys.exit(0) # try to force an exit
228 else:
229 if argv:
230 from setuptools.command.easy_install import main
231 main(argv)
232 else:
233 print "Setuptools version",version,"or greater has been installed."
234 print '(Run "ez_setup.py -U setuptools" to reinstall or upgrade.)'
235
236def update_md5(filenames):
237 """Update our built-in md5 registry"""
238
239 import re
240
241 for name in filenames:
242 base = os.path.basename(name)
243 f = open(name,'rb')
244 md5_data[base] = md5(f.read()).hexdigest()
245 f.close()
246
247 data = [" %r: %r,\n" % it for it in md5_data.items()]
248 data.sort()
249 repl = "".join(data)
250
251 import inspect
252 srcfile = inspect.getsourcefile(sys.modules[__name__])
253 f = open(srcfile, 'rb'); src = f.read(); f.close()
254
255 match = re.search("\nmd5_data = {\n([^}]+)}", src)
256 if not match:
257 print >>sys.stderr, "Internal error!"
258 sys.exit(2)
259
260 src = src[:match.start(1)] + repl + src[match.end(1):]
261 f = open(srcfile,'w')
262 f.write(src)
263 f.close()
264
265
266if __name__=='__main__':
267 if len(sys.argv)>2 and sys.argv[1]=='--md5update':
268 update_md5(sys.argv[2:])
269 else:
270 main(sys.argv[1:])
271
272
273
274
275
2760
=== modified file 'setup.py'
--- setup.py 2009-12-16 13:52:09 +0000
+++ setup.py 2010-03-16 15:59:22 +0000
@@ -1,7 +1,7 @@
1# Copyright 2009 Canonical Ltd. All rights reserved.1# Copyright 2009 Canonical Ltd. All rights reserved.
22
3import ez_setup3import distribute_setup
4ez_setup.use_setuptools()4distribute_setup.use_setuptools()
55
6import sys6import sys
7from setuptools import setup, find_packages7from setuptools import setup, find_packages
88
=== modified file 'versions.cfg'
--- versions.cfg 2009-11-13 20:43:37 +0000
+++ versions.cfg 2010-03-16 15:59:22 +0000
@@ -8,10 +8,10 @@
8cssutils = 0.9.68cssutils = 0.9.6
9lazr.testing = 0.1.19lazr.testing = 0.1.1
10mocker = 0.10.110mocker = 0.10.1
11setuptools = 0.6c911distribute = 0.6.10
12simplejson = 2.0.912simplejson = 2.0.9
13z3c.recipe.tag = 0.2.013z3c.recipe.tag = 0.2.0
14zc.buildout = 1.4.114zc.buildout = 1.4.3
15zc.recipe.egg = 1.2.215zc.recipe.egg = 1.2.2
16zc.recipe.testrunner = 1.2.016zc.recipe.testrunner = 1.2.0
17zope.exceptions = 3.5.217zope.exceptions = 3.5.2

Subscribers

People subscribed via source and target branches