Buildout

Merge lp:~gary/zc.buildout/python-support-8-support-subprocess into lp:zc.buildout

  1. python-support-8-support-subprocess
  2. Merge into trunk
Proposed by Gary Poster
Status: Needs review
Proposed branch: lp:~gary/zc.buildout/python-support-8-support-subprocess
Merge into: lp:zc.buildout
Prerequisite: lp:~gary/zc.buildout/python-support-7
Diff against target: 644 lines (+324/-65) (has conflicts)
8 files modified
bootstrap/bootstrap.py (+33/-10)
buildout.cfg (+3/-3)
dev.py (+43/-3)
src/zc/buildout/buildout.py (+40/-19)
src/zc/buildout/easy_install.py (+26/-8)
src/zc/buildout/easy_install.txt (+11/-1)
src/zc/buildout/tests.py (+149/-5)
src/zc/buildout/update.txt (+19/-16)
Text conflict in CHANGES.txt
Text conflict in src/zc/buildout/buildout.py
Text conflict in src/zc/buildout/buildout.txt
Text conflict in src/zc/buildout/easy_install.py
Text conflict in src/zc/buildout/tests.py
To merge this branch: bzr merge lp:~gary/zc.buildout/python-support-8-support-subprocess
Reviewer Review Type Date Requested Status
Francis J. Lacoste (community) Approve
Review via email: mp+21732@code.launchpad.net

Description of the change

This branch does two things.

1) It makes it simple to start Python processes from scripts started with the new recipe: PYTHONPATH is set so that everything works by default.

2) It makes bootstrap.py (and dev.py) yet more robust in the face of system Pythons.

#1 affects #2.

To post a comment you must log in.
Revision history for this message
Gary Poster (gary) wrote :

The conflicts in this branch are resolved in https://code.edge.launchpad.net/~gary/zc.buildout/python-support (see https://code.edge.launchpad.net/~gary/zc.buildout/python-support/+merge/21733 ).

Reply
Revision history for this message
Francis J. Lacoste (flacoste) wrote :
Download full text (4.1 KiB)

> === modified file 'bootstrap/bootstrap.py'
> +# In order to be more robust in the face of system Pythons, we want to run
> +# with site-packages loaded. This is somewhat tricky, in particular because

With or without? It seems that you want to run with it, but remove any all
namespace from it? Anyway the comment could be clearer given the
non-obviousness of the code.

> +# Python 2.6's distutils imports site, so starting with the -S flag is not
> +# sufficient.
> +if 'site' in sys.modules:
> + # We will restart with python -S.
> + args = sys.argv[:]
> + args[0:0] = [sys.executable, '-S']
> + args = map(quote, args)
> + os.execv(sys.executable, args)
> +clean_path = sys.path[:]
> +import site
> +sys.path[:] = clean_path
> +for k, v in sys.modules.items():
> + if (hasattr(v, '__path__') and
> + len(v.__path__)==1 and
> + not os.path.exists(os.path.join(v.__path__[0],'__init__.py'))):
> + # This is a namespace package. Remove it.
> + sys.modules.pop(k)
> +

> === modified file 'dev.py'
> +# In order to be more robust in the face of system Pythons, we want to run
> +# with site-packages loaded. This is somewhat tricky, in particular because
> +# Python 2.6's distutils imports site, so starting with the -S flag is not
> +# sufficient.

Since this is copy and paste from the other location, my other comment applies
here also.

> +if 'site' in sys.modules:
> + # We will restart with python -S.
> + args = sys.argv[:]
> + args[0:0] = [sys.executable, '-S']
> + args = map(quote, args)
> + os.execv(sys.executable, args)
> +clean_path = sys.path[:]
> +import site
> +sys.path[:] = clean_path
> +for k, v in sys.modules.items():
> + if (hasattr(v, '__path__') and
> + len(v.__path__)==1 and
> + not os.path.exists(os.path.join(v.__path__[0],'__init__.py'))):
> + # This is a namespace package. Remove it.
> + sys.modules.pop(k)
> +
> is_jython = sys.platform.startswith('java')

> === modified file 'src/zc/buildout/buildout.py'

> - args.insert(0, zc.buildout.easy_install._safe_arg (sys.executable))
> -
> + args.insert(0, zc.buildout.easy_install._safe_arg(sys.executable))
> + env = os.environ.copy()
> + env['PYTHONPATH'] = partsdir
> if is_jython:
> - sys.exit(subprocess.Popen([sys.executable] + list(args)).wait())
> + sys.exit(
> + subprocess.Popen(
> + [sys.executable] + list(args), env=env).wait())
> else:
> - sys.exit(os.spawnv(os.P_WAIT, sys.executable, args))
> + sys.exit(os.spawnve(os.P_WAIT, sys.executable, args, env))
>

The intent here is to run the script with only partsdir in the path?

> === modified file 'src/zc/buildout/easy_install.py'
> --- src/zc/buildout/easy_install.py 2010-03-19 16:04:20 +0000
> +++ src/zc/buildout/easy_install.py 2010-03-19 16:04:20 +0000
> @@ -99,6 +99,7 @@
> "print repr([os.path.normpath(p) for p in sys.path if p])"])
> # Windows needs some (as yet to be determined) part of the real env.
> env = os.environ.copy()
> + env.pop('PYTHONPATH', None)

Care to explain w...

Read more...

review: Needs Information
Reply
lp:~gary/zc.buildout/python-support-8-support-subprocess updated
564. By Gary Poster

add explanatory comments; extend test to show that scripts honor explicit PYTHONPATH

Revision history for this message
Gary Poster (gary) wrote :

Good call on all comments. In the new revision, I answered your questions in the new code comments, and added a test as you described.

(Yes, we want to run without site-packages loaded.)

Reply
Revision history for this message
Francis J. Lacoste (flacoste) wrote :

All good!

review: Approve
Reply

Unmerged revisions

564. By Gary Poster

add explanatory comments; extend test to show that scripts honor explicit PYTHONPATH

563. By Gary Poster

fix intermittent test failure in update.txt

562. By Gary Poster

fixes for bootstrap and a system Python; changes based on learning what would be necessary to be able to develop buildout with a system Python (zc.recipe.testing would also need to use sitepackage_safe_scripts)

561. By Gary Poster

set up PYTHONPATH for scripts too, so subprocesses are good to go by default.

560. By Gary Poster

merge from gary-6 <- gary-5

559. By Gary Poster

with these changes, I can build zc.buildout and run its tests successfully with my system Python. To make it fully robust, zc.recipe.test probably would need to use sitepackage_safe_scripts, but this works for now.

558. By Gary Poster

add test for recent fix for buildout

557. By Gary Poster

make the buildout script itself safe for a Python with site packages.

556. By Gary Poster

merge from gary-5 <- gary-4

555. By Gary Poster

merge from gary-5 <- gary-4

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'bootstrap/bootstrap.py'
2--- bootstrap/bootstrap.py 2010-03-19 19:13:23 +0000
3+++ bootstrap/bootstrap.py 2010-03-19 19:13:23 +0000
4@@ -23,6 +23,39 @@
5 import os, shutil, sys, tempfile, textwrap, urllib, urllib2
6 from optparse import OptionParser
7
8+if sys.platform == 'win32':
9+ def quote(c):
10+ if ' ' in c:
11+ return '"%s"' % c # work around spawn lamosity on windows
12+ else:
13+ return c
14+else:
15+ quote = str
16+
17+# In order to be more robust in the face of system Pythons, we want to
18+# run without site-packages loaded. This is somewhat tricky, in
19+# particular because Python 2.6's distutils imports site, so starting
20+# with the -S flag is not sufficient. However, we'll start with that:
21+if 'site' in sys.modules:
22+ # We will restart with python -S.
23+ args = sys.argv[:]
24+ args[0:0] = [sys.executable, '-S']
25+ args = map(quote, args)
26+ os.execv(sys.executable, args)
27+# Now we are running with -S. We'll get the clean sys.path, import site
28+# because distutils will do it later, and then reset the path and clean
29+# out any namespace packages from site-packages that might have been
30+# loaded by .pth files.
31+clean_path = sys.path[:]
32+import site
33+sys.path[:] = clean_path
34+for k, v in sys.modules.items():
35+ if (hasattr(v, '__path__') and
36+ len(v.__path__)==1 and
37+ not os.path.exists(os.path.join(v.__path__[0],'__init__.py'))):
38+ # This is a namespace package. Remove it.
39+ sys.modules.pop(k)
40+
41 is_jython = sys.platform.startswith('java')
42
43 setuptools_source = 'http://peak.telecommunity.com/dist/ez_setup.py'
44@@ -128,16 +161,6 @@
45 if path not in pkg_resources.working_set.entries:
46 pkg_resources.working_set.add_entry(path)
47
48-if sys.platform == 'win32':
49- def quote(c):
50- if ' ' in c:
51- return '"%s"' % c # work around spawn lamosity on windows
52- else:
53- return c
54-else:
55- def quote (c):
56- return c
57-
58 cmd = [quote(sys.executable),
59 '-c',
60 quote('from setuptools.command.easy_install import main; main()'),
61
62=== modified file 'buildout.cfg'
63--- buildout.cfg 2010-03-19 19:13:23 +0000
64+++ buildout.cfg 2010-03-19 19:13:23 +0000
65@@ -3,14 +3,14 @@
66 parts = test oltest py
67
68 [py]
69-recipe = zc.recipe.egg
70+recipe = z3c.recipe.scripts
71 eggs = zc.buildout
72 zope.testing
73 interpreter = py
74
75 [test]
76 recipe = zc.recipe.testrunner
77-eggs =
78+eggs =
79 zc.buildout
80 zc.recipe.egg
81 z3c.recipe.scripts
82@@ -18,7 +18,7 @@
83 # Tests that can be run wo a network
84 [oltest]
85 recipe = zc.recipe.testrunner
86-eggs =
87+eggs =
88 zc.buildout
89 zc.recipe.egg
90 z3c.recipe.scripts
91
92=== modified file 'dev.py'
93--- dev.py 2010-03-19 19:13:23 +0000
94+++ dev.py 2010-03-19 19:13:23 +0000
95@@ -21,6 +21,39 @@
96
97 import os, shutil, sys, subprocess, urllib2
98
99+if sys.platform == 'win32':
100+ def quote(c):
101+ if ' ' in c:
102+ return '"%s"' % c # work around spawn lamosity on windows
103+ else:
104+ return c
105+else:
106+ quote = str
107+
108+# In order to be more robust in the face of system Pythons, we want to
109+# run without site-packages loaded. This is somewhat tricky, in
110+# particular because Python 2.6's distutils imports site, so starting
111+# with the -S flag is not sufficient. However, we'll start with that:
112+if 'site' in sys.modules:
113+ # We will restart with python -S.
114+ args = sys.argv[:]
115+ args[0:0] = [sys.executable, '-S']
116+ args = map(quote, args)
117+ os.execv(sys.executable, args)
118+# Now we are running with -S. We'll get the clean sys.path, import site
119+# because distutils will do it later, and then reset the path and clean
120+# out any namespace packages from site-packages that might have been
121+# loaded by .pth files.
122+clean_path = sys.path[:]
123+import site
124+sys.path[:] = clean_path
125+for k, v in sys.modules.items():
126+ if (hasattr(v, '__path__') and
127+ len(v.__path__)==1 and
128+ not os.path.exists(os.path.join(v.__path__[0],'__init__.py'))):
129+ # This is a namespace package. Remove it.
130+ sys.modules.pop(k)
131+
132 is_jython = sys.platform.startswith('java')
133
134 for d in 'eggs', 'develop-eggs', 'bin':
135@@ -49,14 +82,20 @@
136 env['PYTHONPATH'] = os.path.dirname(pkg_resources.__file__)
137 subprocess.Popen(
138 [sys.executable] +
139- ['setup.py', '-q', 'develop', '-m', '-x', '-d', 'develop-eggs'],
140+ ['-S', 'setup.py', '-q', 'develop', '-m', '-x', '-d', 'develop-eggs'],
141 env=env).wait()
142
143 pkg_resources.working_set.add_entry('src')
144
145 import zc.buildout.easy_install
146-zc.buildout.easy_install.scripts(
147- ['zc.buildout'], pkg_resources.working_set , sys.executable, 'bin')
148+if not os.path.exists('parts'):
149+ os.mkdir('parts')
150+partsdir = os.path.join('parts', 'buildout')
151+if not os.path.exists(partsdir):
152+ os.mkdir(partsdir)
153+zc.buildout.easy_install.sitepackage_safe_scripts(
154+ 'bin', pkg_resources.working_set, sys.executable, partsdir,
155+ reqs=['zc.buildout'])
156
157 bin_buildout = os.path.join('bin', 'buildout')
158
159@@ -64,4 +103,5 @@
160 # Jython needs the script to be called twice via sys.executable
161 assert subprocess.Popen([sys.executable] + [bin_buildout]).wait() == 0
162
163+
164 sys.exit(subprocess.Popen(bin_buildout).wait())
165
166=== modified file 'src/zc/buildout/buildout.py'
167--- src/zc/buildout/buildout.py 2010-03-19 19:13:23 +0000
168+++ src/zc/buildout/buildout.py 2010-03-19 19:13:23 +0000
169@@ -375,7 +375,9 @@
170 if options.get('offline') == 'true':
171 ws = zc.buildout.easy_install.working_set(
172 distributions, options['executable'],
173- [options['develop-eggs-directory'], options['eggs-directory']]
174+ [options['develop-eggs-directory'],
175+ options['eggs-directory']],
176+ include_site_packages=False,
177 )
178 else:
179 ws = zc.buildout.easy_install.install(
180@@ -385,7 +387,8 @@
181 executable=options['executable'],
182 path=[options['develop-eggs-directory']],
183 newest=self.newest,
184- allow_hosts=self._allow_hosts
185+ allow_hosts=self._allow_hosts,
186+ include_site_packages=False,
187 )
188
189 # Now copy buildout and setuptools eggs, and record destination eggs:
190@@ -851,16 +854,19 @@
191 if not self.newest:
192 return
193
194+ options = self['buildout']
195+
196 ws = zc.buildout.easy_install.install(
197 [
198- (spec + ' ' + self['buildout'].get(spec+'-version', '')).strip()
199+ (spec + ' ' + options.get(spec+'-version', '')).strip()
200 for spec in ('zc.buildout', 'setuptools')
201 ],
202- self['buildout']['eggs-directory'],
203- links = self['buildout'].get('find-links', '').split(),
204- index = self['buildout'].get('index'),
205- path = [self['buildout']['develop-eggs-directory']],
206- allow_hosts = self._allow_hosts
207+ options['eggs-directory'],
208+ links = options.get('find-links', '').split(),
209+ index = options.get('index'),
210+ path = [options['develop-eggs-directory']],
211+ allow_hosts = self._allow_hosts,
212+ include_site_packages=False
213 )
214
215 upgraded = []
216@@ -876,7 +882,7 @@
217 __doing__ = 'Upgrading.'
218
219 should_run = realpath(
220- os.path.join(os.path.abspath(self['buildout']['bin-directory']),
221+ os.path.join(os.path.abspath(options['bin-directory']),
222 'buildout')
223 )
224 if sys.platform == 'win32':
225@@ -908,21 +914,34 @@
226
227 # the new dist is different, so we've upgraded.
228 # Update the scripts and return True
229- zc.buildout.easy_install.scripts(
230- ['zc.buildout'], ws, sys.executable,
231- self['buildout']['bin-directory'],
232- )
233+ partsdir = os.path.join(options['parts-directory'], 'buildout')
234+ if os.path.exists(partsdir):
235+ # This is primarily for unit tests, in which .py files change too
236+ # fast for Python to know to regenerate the .pyc/.pyo files.
237+ shutil.rmtree(partsdir)
238+ os.mkdir(partsdir)
239+ zc.buildout.easy_install.sitepackage_safe_scripts(
240+ options['bin-directory'], ws, sys.executable, partsdir,
241+ reqs=['zc.buildout'])
242
243 # Restart
244 args = map(zc.buildout.easy_install._safe_arg, sys.argv)
245 if not __debug__:
246 args.insert(0, '-O')
247- args.insert(0, zc.buildout.easy_install._safe_arg (sys.executable))
248-
249+ args.insert(0, zc.buildout.easy_install._safe_arg(sys.executable))
250+ # We want to make sure that our new site.py is used for rerunning
251+ # buildout, so we put the partsdir in PYTHONPATH for our restart.
252+ # This overrides any set PYTHONPATH, but since we generally are
253+ # trying to run with a completely "clean" python (only the standard
254+ # library) then that should be fine.
255+ env = os.environ.copy()
256+ env['PYTHONPATH'] = partsdir
257 if is_jython:
258- sys.exit(subprocess.Popen([sys.executable] + list(args)).wait())
259+ sys.exit(
260+ subprocess.Popen(
261+ [sys.executable] + list(args), env=env).wait())
262 else:
263- sys.exit(os.spawnv(os.P_WAIT, sys.executable, args))
264+ sys.exit(os.spawnve(os.P_WAIT, sys.executable, args, env))
265
266 def _load_extensions(self):
267 __doing__ = 'Loading extensions.'
268@@ -943,7 +962,8 @@
269 working_set=pkg_resources.working_set,
270 links = self['buildout'].get('find-links', '').split(),
271 index = self['buildout'].get('index'),
272- newest=self.newest, allow_hosts=self._allow_hosts)
273+ newest=self.newest, allow_hosts=self._allow_hosts,
274+ include_site_packages=False)
275
276 # Clear cache because extensions might now let us read pages we
277 # couldn't read before.
278@@ -1060,7 +1080,8 @@
279 path=path,
280 working_set=pkg_resources.working_set,
281 newest=buildout.newest,
282- allow_hosts=buildout._allow_hosts
283+ allow_hosts=buildout._allow_hosts,
284+ include_site_packages=False,
285 )
286
287 __doing__ = 'Loading %s recipe entry %s:%s.', group, spec, entry
288
289=== modified file 'src/zc/buildout/easy_install.py'
290--- src/zc/buildout/easy_install.py 2010-03-19 19:13:23 +0000
291+++ src/zc/buildout/easy_install.py 2010-03-19 19:13:23 +0000
292@@ -99,6 +99,10 @@
293 "print repr([os.path.normpath(p) for p in sys.path if p])"])
294 # Windows needs some (as yet to be determined) part of the real env.
295 env = os.environ.copy()
296+ # We need to make sure that PYTHONPATH, which will often be set
297+ # to include a custom buildout-generated site.py, is not set, or
298+ # else we will not get an accurate sys.path for the executable.
299+ env.pop('PYTHONPATH', None)
300 env.update(kwargs)
301 _proc = subprocess.Popen(
302 cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, env=env)
303@@ -337,9 +341,6 @@
304 self._site_packages))
305 if self._include_site_packages:
306 path.extend(self._site_packages)
307- # else we could try to still include the buildout_and_setuptools_path
308- # if the elements are not in site_packages, but we're not bothering
309- # with this optimization for now, in the name of code simplicity.
310 if dest is not None and dest not in path:
311 path.insert(0, dest)
312 self._path = path
313@@ -1209,9 +1210,9 @@
314 generated.append(_generate_site(
315 site_py_dest, working_set, executable, extra_paths,
316 include_site_packages, relative_paths))
317- script_initialization = (
318- '\nimport site # imports custom buildout-generated site.py\n%s' % (
319- script_initialization,))
320+ script_initialization = _script_initialization_template % dict(
321+ site_py_dest=site_py_dest,
322+ script_initialization=script_initialization)
323 if not script_initialization.endswith('\n'):
324 script_initialization += '\n'
325 generated.extend(_generate_scripts(
326@@ -1222,6 +1223,15 @@
327 interpreter, dest, executable, site_py_dest, relative_paths))
328 return generated
329
330+_script_initialization_template = '''
331+import site # imports custom buildout-generated site.py
332+import os
333+path = %(site_py_dest)r
334+if os.environ.get('PYTHONPATH'):
335+ path = os.pathsep.join([path, os.environ['PYTHONPATH']])
336+os.environ['PYTHONPATH'] = path
337+%(script_initialization)s'''
338+
339 # Utilities for the script generation functions.
340
341 # These are shared by both ``scripts`` and ``sitepackage_safe_scripts``
342@@ -1504,8 +1514,14 @@
343 "fp, path, desc = imp.find_module(%r); "
344 "fp.close; "
345 "print path" % (name,)]
346+ env = os.environ.copy()
347+ # We need to make sure that PYTHONPATH, which will often be set to
348+ # include a custom buildout-generated site.py, is not set, or else
349+ # we will not get an accurate value for the "real" site.py and
350+ # sitecustomize.py.
351+ env.pop('PYTHONPATH', None)
352 _proc = subprocess.Popen(
353- cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
354+ cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, env=env)
355 stdout, stderr = _proc.communicate();
356 if _proc.returncode:
357 logger.info(
358@@ -1601,7 +1617,9 @@
359 site.close()
360 real_site.close()
361 if not successful_rewrite:
362- raise RuntimeError('Buildout did not successfully rewrite site.py')
363+ raise RuntimeError(
364+ 'Buildout did not successfully rewrite %s to %s' %
365+ (real_site_path, site_path))
366 return site_path
367
368 namespace_include_site_packages_setup = '''
369
370=== modified file 'src/zc/buildout/easy_install.txt'
371--- src/zc/buildout/easy_install.txt 2010-03-19 19:13:23 +0000
372+++ src/zc/buildout/easy_install.txt 2010-03-19 19:13:23 +0000
373@@ -1499,6 +1499,11 @@
374 <BLANKLINE>
375 <BLANKLINE>
376 import site # imports custom buildout-generated site.py
377+ import os
378+ path = '/interpreter/parts/interpreter'
379+ if os.environ.get('PYTHONPATH'):
380+ path = os.pathsep.join([path, os.environ['PYTHONPATH']])
381+ os.environ['PYTHONPATH'] = path
382 <BLANKLINE>
383 import eggrecipedemo
384 <BLANKLINE>
385@@ -1528,7 +1533,7 @@
386 >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts(
387 ... interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir,
388 ... reqs=['demo'], script_arguments='1, 2',
389- ... script_initialization='import os\nos.chdir("foo")')
390+ ... script_initialization='import os\nos.chdir("foo")')
391
392 >>> cat(demo_path) # doctest: +NORMALIZE_WHITESPACE
393 #!/usr/local/bin/python2.4 -S
394@@ -1539,6 +1544,11 @@
395 <BLANKLINE>
396 import site # imports custom buildout-generated site.py
397 import os
398+ path = '/interpreter/parts/interpreter'
399+ if os.environ.get('PYTHONPATH'):
400+ path = os.pathsep.join([path, os.environ['PYTHONPATH']])
401+ os.environ['PYTHONPATH'] = path
402+ import os
403 os.chdir("foo")
404 <BLANKLINE>
405 import eggrecipedemo
406
407=== modified file 'src/zc/buildout/tests.py'
408--- src/zc/buildout/tests.py 2010-03-19 19:13:23 +0000
409+++ src/zc/buildout/tests.py 2010-03-19 19:13:23 +0000
410@@ -2254,8 +2254,95 @@
411
412 """
413
414+def subprocesses_have_same_environment_by_default():
415+ """
416+The scripts generated by sitepackage_safe_scripts set the PYTHONPATH so that,
417+if the environment is maintained (the default behavior), subprocesses get
418+the same Python packages.
419+
420+First, we set up a script and an interpreter.
421+
422+ >>> interpreter_dir = tmpdir('interpreter')
423+ >>> interpreter_parts_dir = os.path.join(
424+ ... interpreter_dir, 'parts', 'interpreter')
425+ >>> interpreter_bin_dir = os.path.join(interpreter_dir, 'bin')
426+ >>> mkdir(interpreter_bin_dir)
427+ >>> mkdir(interpreter_dir, 'eggs')
428+ >>> mkdir(interpreter_dir, 'parts')
429+ >>> mkdir(interpreter_parts_dir)
430+ >>> ws = zc.buildout.easy_install.install(
431+ ... ['demo'], join(interpreter_dir, 'eggs'), links=[link_server],
432+ ... index=link_server+'index/')
433+ >>> test = (
434+ ... "import subprocess, sys; subprocess.call("
435+ ... "[sys.executable, '-c', "
436+ ... "'import eggrecipedemo; print eggrecipedemo.x'])")
437+ >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts(
438+ ... interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir,
439+ ... reqs=['demo'], interpreter='py',
440+ ... script_initialization=test + '; sys.exit(0)')
441+
442+This works for the script.
443+
444+ >>> print system(join(interpreter_bin_dir, 'demo'))
445+ 3
446+ <BLANKLINE>
447+
448+This also works for the generated interpreter.
449+
450+ >>> print call_py(join(interpreter_bin_dir, 'py'), test)
451+ 3
452+ <BLANKLINE>
453+
454+If you have a PYTHONPATH in your environment, it will be honored, after
455+the buildout-generated path.
456+
457+ >>> original_pythonpath = os.environ.get('PYTHONPATH')
458+ >>> os.environ['PYTHONPATH'] = 'foo'
459+ >>> test = (
460+ ... "import subprocess, sys; subprocess.call("
461+ ... "[sys.executable, '-c', "
462+ ... "'import sys, pprint; pprint.pprint(sys.path)'])")
463+ >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts(
464+ ... interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir,
465+ ... reqs=['demo'], interpreter='py',
466+ ... script_initialization=test + '; sys.exit(0)')
467+
468+This works for the script. As you can see, /sample_buildout/foo is included
469+right after the "parts" directory that contains site.py and sitecustomize.py.
470+You can also see, actually more easily than in the other example, that we
471+have the desired eggs available.
472+
473+ >>> print system(join(interpreter_bin_dir, 'demo')), # doctest: +ELLIPSIS
474+ ['',
475+ '/interpreter/parts/interpreter',
476+ '/sample-buildout/foo',
477+ ...
478+ '/interpreter/eggs/demo-0.3-pyN.N.egg',
479+ '/interpreter/eggs/demoneeded-1.1-pyN.N.egg']
480+
481+This also works for the generated interpreter, with identical results.
482+
483+ >>> print call_py(join(interpreter_bin_dir, 'py'), test),
484+ ... # doctest: +ELLIPSIS
485+ ['',
486+ '/interpreter/parts/interpreter',
487+ '/sample-buildout/foo',
488+ ...
489+ '/interpreter/eggs/demo-0.3-pyN.N.egg',
490+ '/interpreter/eggs/demoneeded-1.1-pyN.N.egg']
491+
492+ >>> # Cleanup
493+ >>> if original_pythonpath:
494+ ... os.environ['PYTHONPATH'] = original_pythonpath
495+ ... else:
496+ ... del os.environ['PYTHONPATH']
497+ ...
498+
499+ """
500+
501 def bootstrap_makes_buildout_that_works_with_system_python():
502- """
503+ r"""
504 In order to work smoothly with a system Python, bootstrapping creates
505 the buildout script with
506 zc.buildout.easy_install.sitepackage_safe_scripts. If it did not, a
507@@ -2274,15 +2361,11 @@
508 >>> write(sample_buildout, 'recipes', 'dummy.py',
509 ... '''
510 ... import logging, os, zc.buildout
511- ...
512 ... class Dummy:
513- ...
514 ... def __init__(self, buildout, name, options):
515 ... pass
516- ...
517 ... def install(self):
518 ... return ()
519- ...
520 ... def update(self):
521 ... pass
522 ... ''')
523@@ -2340,6 +2423,67 @@
524 Installing dummy.
525 <BLANKLINE>
526
527+Here's the same story with a namespace package, which has some additional
528+complications behind the scenes. First, a recipe, in the "tellmy" namespace.
529+
530+ >>> mkdir(sample_buildout, 'ns')
531+ >>> mkdir(sample_buildout, 'ns', 'tellmy')
532+ >>> write(sample_buildout, 'ns', 'tellmy', '__init__.py',
533+ ... "__import__('pkg_resources').declare_namespace(__name__)\n")
534+ >>> mkdir(sample_buildout, 'ns', 'tellmy', 'recipes')
535+ >>> write(sample_buildout, 'ns', 'tellmy', 'recipes', '__init__.py', ' ')
536+ >>> write(sample_buildout, 'ns', 'tellmy', 'recipes', 'dummy.py',
537+ ... '''
538+ ... import logging, os, zc.buildout
539+ ... class Dummy:
540+ ... def __init__(self, buildout, name, options):
541+ ... pass
542+ ... def install(self):
543+ ... return ()
544+ ... def update(self):
545+ ... pass
546+ ... ''')
547+ >>> write(sample_buildout, 'ns', 'setup.py',
548+ ... '''
549+ ... from setuptools import setup
550+ ... setup(
551+ ... name="tellmy.recipes",
552+ ... packages=['tellmy', 'tellmy.recipes'],
553+ ... install_requires=['setuptools'],
554+ ... namespace_packages=['tellmy'],
555+ ... entry_points = {'zc.buildout':
556+ ... ['dummy = tellmy.recipes.dummy:Dummy']},
557+ ... )
558+ ... ''')
559+
560+Now, a buildout that uses it.
561+
562+ >>> create_sample_namespace_eggs(sample_eggs, site_packages_path)
563+ >>> rmdir('develop-eggs')
564+ >>> from zc.buildout.testing import make_buildout
565+ >>> make_buildout(executable=py_path)
566+ >>> write(sample_buildout, 'buildout.cfg',
567+ ... '''
568+ ... [buildout]
569+ ... develop = ns
570+ ... recipes
571+ ... parts = dummy
572+ ... find-links = %(link_server)s
573+ ... executable = %(py_path)s
574+ ...
575+ ... [dummy]
576+ ... recipe = tellmy.recipes:dummy
577+ ... ''' % globals())
578+
579+Now we actually run the buildout.
580+
581+ >>> print system(buildout)
582+ Develop: '/sample-buildout/ns'
583+ Develop: '/sample-buildout/recipes'
584+ Uninstalling dummy.
585+ Installing dummy.
586+ <BLANKLINE>
587+
588 """
589
590 if sys.version_info > (2, 4):
591
592=== modified file 'src/zc/buildout/update.txt'
593--- src/zc/buildout/update.txt 2010-03-19 19:13:23 +0000
594+++ src/zc/buildout/update.txt 2010-03-19 19:13:23 +0000
595@@ -78,22 +78,26 @@
596 zc.buildout 99.99
597 setuptools 99.99
598
599-Our buildout script has been updated to use the new eggs:
600+Our buildout script's site.py has been updated to use the new eggs:
601
602- >>> cat(sample_buildout, 'bin', 'buildout')
603- ... # doctest: +NORMALIZE_WHITESPACE
604- #!/usr/local/bin/python2.4
605- <BLANKLINE>
606- import sys
607- sys.path[0:0] = [
608- '/sample-buildout/eggs/zc.buildout-99.99-py2.4.egg',
609- '/sample-buildout/eggs/setuptools-99.99-py2.4.egg',
610- ]
611- <BLANKLINE>
612- import zc.buildout.buildout
613- <BLANKLINE>
614- if __name__ == '__main__':
615- zc.buildout.buildout.main()
616+ >>> cat(sample_buildout, 'parts', 'buildout', 'site.py')
617+ ... # doctest: +NORMALIZE_WHITESPACE +ELLIPSIS
618+ "...
619+ def addsitepackages(known_paths):
620+ """Add site packages, as determined by zc.buildout.
621+ <BLANKLINE>
622+ See original_addsitepackages, below, for the original version."""
623+ buildout_paths = [
624+ '/sample-buildout/eggs/zc.buildout-99.99-pyN.N.egg',
625+ '/sample-buildout/eggs/setuptools-99.99-pyN.N.egg'
626+ ]
627+ for path in buildout_paths:
628+ sitedir, sitedircase = makepath(path)
629+ if not sitedircase in known_paths and os.path.exists(sitedir):
630+ sys.path.append(sitedir)
631+ known_paths.add(sitedircase)
632+ return known_paths
633+ ...
634
635 Now, let's recreate the sample buildout. If we specify constraints on
636 the versions of zc.buildout and setuptools to use, running the
637@@ -120,7 +124,6 @@
638 zc.buildout version 1.0.0,
639 setuptools version 0.6;
640 restarting.
641- Generated script '/sample-buildout/bin/buildout'.
642 Develop: '/sample-buildout/showversions'
643 Updating show-versions.
644 zc.buildout 1.0.0

Subscribers

People subscribed via source and target branches

to all changes: