Merge lp:~gary/zc.buildout/python-support-4 into lp:zc.buildout

Proposed by Gary Poster on 2010-02-17
Status: Needs review
Proposed branch: lp:~gary/zc.buildout/python-support-4
Merge into: lp:zc.buildout
Prerequisite: lp:~gary/zc.buildout/python-support-3-options
Diff against target: 3435 lines (+2506/-195)
26 files modified
CHANGES.txt (+32/-0)
README.txt (+9/-1)
buildout.cfg (+3/-1)
setup.py (+1/-1)
src/zc/buildout/bootstrap.txt (+3/-3)
src/zc/buildout/easy_install.py (+523/-107)
src/zc/buildout/easy_install.txt (+587/-23)
src/zc/buildout/testing.py (+86/-22)
src/zc/buildout/tests.py (+285/-0)
src/zc/buildout/testselectingpython.py (+28/-1)
src/zc/buildout/update.txt (+1/-0)
z3c.recipe.scripts_/CHANGES.txt (+7/-0)
z3c.recipe.scripts_/README.txt (+10/-0)
z3c.recipe.scripts_/setup.py (+76/-0)
z3c.recipe.scripts_/src/z3c/__init__.py (+1/-0)
z3c.recipe.scripts_/src/z3c/recipe/__init__.py (+1/-0)
z3c.recipe.scripts_/src/z3c/recipe/scripts/README.txt (+402/-0)
z3c.recipe.scripts_/src/z3c/recipe/scripts/__init__.py (+1/-0)
z3c.recipe.scripts_/src/z3c/recipe/scripts/scripts.py (+101/-0)
z3c.recipe.scripts_/src/z3c/recipe/scripts/tests.py (+293/-0)
zc.recipe.egg_/setup.py (+2/-2)
zc.recipe.egg_/src/zc/recipe/egg/README.txt (+4/-1)
zc.recipe.egg_/src/zc/recipe/egg/api.txt (+1/-0)
zc.recipe.egg_/src/zc/recipe/egg/custom.txt (+5/-0)
zc.recipe.egg_/src/zc/recipe/egg/egg.py (+42/-28)
zc.recipe.egg_/src/zc/recipe/egg/selecting-python.txt (+2/-5)
To merge this branch: bzr merge lp:~gary/zc.buildout/python-support-4
Reviewer Review Type Date Requested Status
Francis J. Lacoste (community) 2010-02-17 Approve on 2010-02-18
Review via email: mp+19547@code.launchpad.net
To post a comment you must log in.
Gary Poster (gary) wrote :
Download full text (3.5 KiB)

This is a completely reworked approach to using zc.buildout with a system Python (or any Python with a populated site packages directory), deprecating my previous attempt.

The basic approach revolves around writing our own site.py and sitecustomize.py that are responsible for setting up paths and any other environment changes desired. Then if the path to these rewritten files are included (via PYTHONPATH, or with -S followed by sys.path manipulation and ``import site``) you have the environment set up, using the standard Python mechanism.

Similarly to my past work, the point that got complicated was namespace packages. The solution shown here is straightforward, as you can see in the easy_install.txt discussion.

However, if you try to use a system Python, you expose its packages using ``add-site-packages=true``, and you use eggs with namespace packages, it will incur a startup cost for generated scripts that is quadratically related to the number of eggs added by zc.buildout, because of code in pkg_resources. (We also have that same behavior now in Launchpad.) While I did a number of experiments to try and bypass the inefficiencies of the pkg_resources code, I discarded them, because Launchpad is using pkg_resources and expects it to be fully set up. Other code may be the same. The right solution is to fix pkg_resources, in Setuptools or Distribute, at which point the approach I have in this branch should still work, and be much faster.

I did punt a bit in the Windows version of the new interpreter script: I could not get the standard Windows path hacks to work for sys.exec*. I used subprocess instead for Windows, and commented this section heavily.

Obviously, this is a huge branch. I did not see a good way of subdividing it beyond what I already did (~gary/zc.buildout/python-support-1-cleanup, ~gary/zc.buildout/python-support-2-bootstrap, and ~gary/zc.buildout/python-support-3-options). I did consider the following subdivision:

- First, you could look at the changes to easy_install.py and easy_install.txt. This is the heart of the changes. It refactors the existing script generation into bits I could reuse more easily, and then makes a new version script generation function (``generated_scripts``) that works as I describe above.

- Second, you could look at zc.recipe.egg and z3c.recipe.scripts. zc.recipe.egg is refactored so I could reuse bits more easily. z3c.recipe.scripts is a brand new recipe that uses generate_scripts, as described above.

- Finally, you could look at the new tests and test changes in zc.buildout. They use zc.recipe.egg and z3c.recipe.scripts to test that system installs can work as desired--essentially showing that all of the earlier work does in fact produce a system that does what we want.

I didn't feel that separating them made a lot of sense, because you could not test whether generate_scripts actually did what we wanted until we had z3c.recipe.scripts also available. The changes are all interrelated.

Something to note is that zc.recipe.egg does not have any changed behavior. Also, it is not safe to use with a system Python. If you want that behavior for scripts and interpreters, use z3c.rec...

Read more...

Francis J. Lacoste (flacoste) wrote :
Download full text (13.2 KiB)

Hi Gary,

Wow, this was a big one. I have a bunch of comments, mostly about
clarifications and typos.

But good to merge otherwise as far as Launchpad is concerned.

Cheers

> === modified file 'CHANGES.txt'
> --- CHANGES.txt 2010-02-17 22:17:12 +0000
> +++ CHANGES.txt 2010-02-17 22:17:12 +0000
> @@ -6,6 +6,25 @@
>
> New Features:
>
> +- Buildout can be safely used with a system Python, as long as you use the
> + new z3c.recipe.scripts recipe to generate scripts and interpreters, rather
> + than zc.recipe.egg (which is still a fully supported, and simpler, way of
> + generating scripts and interpreters if you are using a "clean" Python).

Really, I tohugh zc.recipe.egg would also do the right thing. Can you explain
the differences to me?

> +
> + A hopefully slight limitation: in no cases are distributions in your
> + site-packages used to satisfy buildout dependencies. The
> + site-packages can be used in addition to the dependencies specified in
> + your buildout, and buildout dependencies can override code in your
> + site-packages, but even if your Python's site-packages has the same
> + exact version as specified in your buildout configuration, buildout
> + will still use its own copy.

I assume dependencies related to zc.buildout itself like distribute,
and setuptools. Not general dependencies managed by buildout, right?

>
> +- Installing a namespace package using a Python that already has a package
> + in the same namespace (e.g., in the Python's site-packages) failed in
> + some cases.
> +

I assume you mean this is fixed. (IOW, that is in a fixed bugs section?)

> +- Another variation of this error showed itself when at least two
> + dependencies were in a shared location like site-packages, and the
> + first one met the "versions" setting. The first dependency would be
> + added, but subsequent dependencies from the same location (e.g.,
> + site-packages) would use the version of the package found in the
> + shared location, ignoring the version setting.
> +

I assume the same thing here.

> === modified file 'README.txt'
> --- README.txt 2010-02-17 22:17:12 +0000
> +++ README.txt 2010-02-17 22:17:12 +0000
> @@ -37,6 +37,11 @@
> dependencies. It installs their console-script entry points with
> the needed eggs included in their paths.
>
> +`z3c.recipe.scripts <http://pypi.python.org/pypi/z3c.recipe.scripts>`_
> + This scripts recipe builds interpreter scripts and entry point scripts
> + based on eggs. These scripts have more features and flexibility than the
> + ones offered by zc.recipe.egg.
> +

Again, might be worthwhile to go into more details of when one should be used
instead of the other.

=== modified file 'src/zc/buildout/easy_install.py'
--- src/zc/buildout/easy_install.py 2010-02-17 22:17:12 +0000
+++ src/zc/buildout/easy_install.py 2010-02-17 22:17:12 +0000
@@ -137,9 +137,50 @@
 else:
     _safe_arg = str

-_easy_install_cmd = _safe_arg(
- 'from setuptools.command.easy_install import main; main()'
- )
+# The following string is used to run easy_install in
+# Installer._call_easy_install. It is started with python -S (that is,
+# don't import site at start). That flag, a...

review: Approve
550. By Gary Poster on 2010-02-19

merge from gary-3

551. By Gary Poster on 2010-02-22

switch ``generate_scripts`` to ``sitepackage_safe_scripts`` per review

552. By Gary Poster on 2010-02-22

add missing reset_interpreter calls

553. By Gary Poster on 2010-02-22

change the way that we identify namespace packages and better comment the code, per review.

554. By Gary Poster on 2010-02-22

doc fixes per review.

555. By Gary Poster on 2010-02-23

try again to describe the change.

556. By Gary Poster on 2010-02-23

fix docstring

Gary Poster (gary) wrote :
Download full text (20.7 KiB)

Thank you, Francis!

Francis and I talked about the review. On the basis of his comments and the discussion, I did the revisions listed above as well as two further branches: https://code.edge.launchpad.net/~gary/zc.buildout/python-support-5-initial-egg-control and https://code.edge.launchpad.net/~gary/zc.buildout/python-support-6-egg-control .

I'll respond to his comments point by point now.

> === modified file 'CHANGES.txt'
> --- CHANGES.txt 2010-02-17 22:17:12 +0000
> +++ CHANGES.txt 2010-02-17 22:17:12 +0000
> @@ -6,6 +6,25 @@
>
> New Features:
>
> +- Buildout can be safely used with a system Python, as long as you use the
> + new z3c.recipe.scripts recipe to generate scripts and interpreters, rather
> + than zc.recipe.egg (which is still a fully supported, and simpler, way of
> + generating scripts and interpreters if you are using a "clean" Python).
>
> Really, I tohugh zc.recipe.egg would also do the right thing. Can you explain
> the differences to me?

We discussed this. He meant that he wanted the CHANGES document to more clearly describe the differences. I tried this:

- Buildout can be safely used with a system Python (or any Python with code
  in site-packages), as long as you use the new z3c.recipe.scripts
  recipe to generate scripts and interpreters, rather than zc.recipe.egg.

  zc.recipe.egg is still a fully supported, and simpler, way of
  generating scripts and interpreters if you are using a "clean" Python,
  without code installed in site-packages. It keeps its previous behavior in
  order to provide backwards compatibility.

>
> +
> + A hopefully slight limitation: in no cases are distributions in your
> + site-packages used to satisfy buildout dependencies. The
> + site-packages can be used in addition to the dependencies specified in
> + your buildout, and buildout dependencies can override code in your
> + site-packages, but even if your Python's site-packages has the same
> + exact version as specified in your buildout configuration, buildout
> + will still use its own copy.
>
> I assume dependencies related to zc.buildout itself like distribute,
> and setuptools. Not general dependencies managed by buildout, right?

My comment was wrong--and the actual situation turned out to be a big problem, leading to the two subsequent branches I mentioned above.

The actual situation was that dependencies would be obtained from site-packages--even if you said you didn't want to use site-packages. This is completely broken for the use case of wanting to use a system Python without site-packages.

I fixed this in the later branches.

The later branches mean that one can choose to get general dependencies from site-packages, or not.

> +- Installing a namespace package using a Python that already has a package
> + in the same namespace (e.g., in the Python's site-packages) failed in
> + some cases.
> +
>
> I assume you mean this is fixed. (IOW, that is in a fixed bugs section?)

Right.

> +- Another variation of this error showed itself when at least two
> + dependencies were in a shared location like site-packages, and the
> + first one met the "versions" setting. The first dependency would be
> + ad...

Unmerged revisions

556. By Gary Poster on 2010-02-23

fix docstring

555. By Gary Poster on 2010-02-23

try again to describe the change.

554. By Gary Poster on 2010-02-22

doc fixes per review.

553. By Gary Poster on 2010-02-22

change the way that we identify namespace packages and better comment the code, per review.

552. By Gary Poster on 2010-02-22

add missing reset_interpreter calls

551. By Gary Poster on 2010-02-22

switch ``generate_scripts`` to ``sitepackage_safe_scripts`` per review

550. By Gary Poster on 2010-02-19

merge from gary-3

549. By Gary Poster on 2010-02-17

propagate merge from gary-3 <- gary-2 <- gary-1 <- trunk

548. By gary on 2010-02-13

fix some tests on other Python versions

547. By gary on 2010-02-12

revert attempt to skip some of the pkg_resources dance: it caused me trouble.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'CHANGES.txt'
2--- CHANGES.txt 2010-02-23 20:41:17 +0000
3+++ CHANGES.txt 2010-02-23 20:41:17 +0000
4@@ -6,6 +6,27 @@
5
6 New Features:
7
8+- Buildout can be safely used with a system Python (or any Python with code
9+ in site-packages), as long as you use the new z3c.recipe.scripts
10+ recipe to generate scripts and interpreters, rather than zc.recipe.egg.
11+
12+ zc.recipe.egg is still a fully supported, and simpler, way of
13+ generating scripts and interpreters if you are using a "clean" Python,
14+ without code installed in site-packages. It keeps its previous behavior in
15+ order to provide backwards compatibility.
16+
17+ (Note that this branch is incomplete in its implementation of this feature:
18+ if eggs are in installed in site-packages but you do not want to use
19+ site-packages, the eggs will drag in site-packages even if you try to
20+ exclude it. This is addressed in subsequent branches in the series of
21+ which this one is a part.)
22+
23+- Added new function, ``zc.buildout.easy_install.sitepackage_safe_scripts``,
24+ to generate scripts and interpreter. It produces a full-featured
25+ interpreter (all command-line options supported) and the ability to
26+ safely let scripts include site packages, such as with a system
27+ Python. The ``z3c.recipe.scripts`` recipe uses this new function.
28+
29 - Improve bootstrap.
30
31 * New options let you specify where to find ez_setup.py and where to find
32@@ -23,6 +44,17 @@
33 This means, among other things, that ``bin/buildout -vv`` and
34 ``bin/buildout annotate`` correctly list more of the options.
35
36+- Installing a namespace package using a Python that already has a package
37+ in the same namespace (e.g., in the Python's site-packages) failed in
38+ some cases.
39+
40+- Another variation of this error showed itself when at least two
41+ dependencies were in a shared location like site-packages, and the
42+ first one met the "versions" setting. The first dependency would be
43+ added, but subsequent dependencies from the same location (e.g.,
44+ site-packages) would use the version of the package found in the
45+ shared location, ignoring the version setting.
46+
47 1.4.3 (2009-12-10)
48 ==================
49
50
51=== modified file 'README.txt'
52--- README.txt 2010-02-23 20:41:17 +0000
53+++ README.txt 2010-02-23 20:41:17 +0000
54@@ -35,7 +35,15 @@
55 `zc.recipe.egg <http://pypi.python.org/pypi/zc.recipe.egg>`_
56 The egg recipe installes one or more eggs, with their
57 dependencies. It installs their console-script entry points with
58- the needed eggs included in their paths.
59+ the needed eggs included in their paths. It is suitable for use with
60+ a "clean" Python: one without packages installed in site-packages.
61+
62+`z3c.recipe.scripts <http://pypi.python.org/pypi/z3c.recipe.scripts>`_
63+ Like zc.recipe.egg, this recipe builds interpreter scripts and entry
64+ point scripts based on eggs. It can be used with a Python that has
65+ packages installed in site-packages, such as a system Python. The
66+ interpreter also has more features than the one offered by
67+ zc.recipe.egg.
68
69 `zc.recipe.testrunner <http://pypi.python.org/pypi/zc.recipe.testrunner>`_
70 The testrunner egg creates a test runner script for one or
71
72=== modified file 'buildout.cfg'
73--- buildout.cfg 2009-10-23 08:25:24 +0000
74+++ buildout.cfg 2010-02-23 20:41:17 +0000
75@@ -1,5 +1,5 @@
76 [buildout]
77-develop = zc.recipe.egg_ .
78+develop = zc.recipe.egg_ z3c.recipe.scripts_ .
79 parts = test oltest py
80
81 [py]
82@@ -13,6 +13,7 @@
83 eggs =
84 zc.buildout
85 zc.recipe.egg
86+ z3c.recipe.scripts
87
88 # Tests that can be run wo a network
89 [oltest]
90@@ -20,6 +21,7 @@
91 eggs =
92 zc.buildout
93 zc.recipe.egg
94+ z3c.recipe.scripts
95 defaults =
96 [
97 '-t',
98
99=== modified file 'setup.py'
100--- setup.py 2009-12-10 16:19:55 +0000
101+++ setup.py 2010-02-23 20:41:17 +0000
102@@ -12,7 +12,7 @@
103 #
104 ##############################################################################
105 name = "zc.buildout"
106-version = "1.4.4dev"
107+version = "1.5.0dev"
108
109 import os
110 from setuptools import setup
111
112=== modified file 'src/zc/buildout/bootstrap.txt'
113--- src/zc/buildout/bootstrap.txt 2010-02-23 20:41:17 +0000
114+++ src/zc/buildout/bootstrap.txt 2010-02-23 20:41:17 +0000
115@@ -232,8 +232,8 @@
116 >>> print system(
117 ... zc.buildout.easy_install._safe_arg(sys.executable)+' '+
118 ... 'bootstrap.py --help'),
119- ... # doctest: +ELLIPSIS
120- usage: [DESIRED PYTHON FOR BUILDOUT] bootstrap.py [options]
121+ ... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE
122+ Usage: [DESIRED PYTHON FOR BUILDOUT] bootstrap.py [options]
123 <BLANKLINE>
124 Bootstraps a buildout-based project.
125 <BLANKLINE>
126@@ -244,7 +244,7 @@
127 local resources, you can keep this script from going over the network.
128 <BLANKLINE>
129 <BLANKLINE>
130- options:
131+ Options:
132 -h, --help show this help message and exit
133 -v VERSION, --version=VERSION
134 use a specific zc.buildout version
135
136=== modified file 'src/zc/buildout/easy_install.py'
137--- src/zc/buildout/easy_install.py 2010-02-23 20:41:17 +0000
138+++ src/zc/buildout/easy_install.py 2010-02-23 20:41:17 +0000
139@@ -60,12 +60,13 @@
140 pkg_resources.Requirement.parse('setuptools')
141 ).location
142
143-# Include buildout and setuptools eggs in paths
144-buildout_and_setuptools_path = [
145- setuptools_loc,
146- pkg_resources.working_set.find(
147- pkg_resources.Requirement.parse('zc.buildout')).location,
148- ]
149+# Include buildout and setuptools eggs in paths. We prevent dupes just to
150+# keep from duplicating any log messages about them.
151+buildout_loc = pkg_resources.working_set.find(
152+ pkg_resources.Requirement.parse('zc.buildout')).location
153+buildout_and_setuptools_path = [setuptools_loc]
154+if os.path.normpath(setuptools_loc) != os.path.normpath(buildout_loc):
155+ buildout_and_setuptools_path.append(buildout_loc)
156
157
158 class IncompatibleVersionError(zc.buildout.UserError):
159@@ -137,9 +138,73 @@
160 else:
161 _safe_arg = str
162
163-_easy_install_cmd = _safe_arg(
164- 'from setuptools.command.easy_install import main; main()'
165- )
166+# The following string is used to run easy_install in
167+# Installer._call_easy_install. It is started with python -S (that is,
168+# don't import site at start). That flag, and all of the code in this
169+# snippet above the last two lines, exist to work around a relatively rare
170+# problem. If
171+#
172+# - your buildout configuration is trying to install a package that is within
173+# a namespace package, and
174+#
175+# - you use a Python that has a different version of this package
176+# installed in in its site-packages using
177+# --single-version-externally-managed (that is, using the mechanism
178+# sometimes used by system packagers:
179+# http://peak.telecommunity.com/DevCenter/setuptools#install-command ), and
180+#
181+# - the new package tries to do sys.path tricks in the setup.py to get a
182+# __version__,
183+#
184+# then the older package will be loaded first, making the setup version
185+# the wrong number. While very arguably packages simply shouldn't do
186+# the sys.path tricks, some do, and we don't want buildout to fall over
187+# when they do.
188+#
189+# The namespace packages installed in site-packages with
190+# --single-version-externally-managed use a mechanism that cause them to
191+# be processed when site.py is imported (see
192+# http://mail.python.org/pipermail/distutils-sig/2009-May/011730.html
193+# for another description of the problem). Simply starting Python with
194+# -S addresses the problem in Python 2.4 and 2.5, but Python 2.6's
195+# distutils imports a value from the site module, so we unfortunately
196+# have to do more drastic surgery in the _easy_install_cmd code below.
197+#
198+# Here's an example of the .pth files created by setuptools when using that
199+# flag:
200+#
201+# import sys,new,os;
202+# p = os.path.join(sys._getframe(1).f_locals['sitedir'], *('<NAMESPACE>',));
203+# ie = os.path.exists(os.path.join(p,'__init__.py'));
204+# m = not ie and sys.modules.setdefault('<NAMESPACE>',new.module('<NAMESPACE>'));
205+# mp = (m or []) and m.__dict__.setdefault('__path__',[]);
206+# (p not in mp) and mp.append(p)
207+#
208+# The code, below, then, runs under -S, indicating that site.py should
209+# not be loaded initially. It gets the initial sys.path under these
210+# circumstances, and then imports site (because Python 2.6's distutils
211+# will want it, as mentioned above). It then reinstates the old sys.path
212+# value. Then it removes namespace packages (created by the setuptools
213+# code above) from sys.modules. It identifies namespace packages by
214+# iterating over every loaded module. It first looks if there is a
215+# __path__, so it is a package; and then it sees if that __path__ does
216+# not have an __init__.py. (Note that PEP 382,
217+# http://www.python.org/dev/peps/pep-0382, makes it possible to have a
218+# namespace package that has an __init__.py, but also should make it
219+# unnecessary for site.py to preprocess these packages, so it should be
220+# fine, as far as can be guessed as of this writing.) Finally, it
221+# imports easy_install and runs it.
222+
223+_easy_install_cmd = _safe_arg('''\
224+import sys,os;\
225+p = sys.path[:];\
226+import site;\
227+sys.path[:] = p;\
228+[sys.modules.pop(k) for k, v in sys.modules.items()\
229+ if hasattr(v, '__path__') and len(v.__path__)==1 and\
230+ not os.path.exists(os.path.join(v.__path__[0],'__init__.py'))];\
231+from setuptools.command.easy_install import main;\
232+main()''')
233
234
235 class Installer:
236@@ -301,7 +366,7 @@
237 try:
238 path = setuptools_loc
239
240- args = ('-c', _easy_install_cmd, '-mUNxd', _safe_arg(tmp))
241+ args = ('-Sc', _easy_install_cmd, '-mUNxd', _safe_arg(tmp))
242 if self._always_unzip:
243 args += ('-Z', )
244 level = logger.getEffectiveLevel()
245@@ -904,6 +969,9 @@
246 def working_set(specs, executable, path):
247 return install(specs, None, executable=executable, path=path)
248
249+############################################################################
250+# Script generation functions
251+
252 def scripts(reqs, working_set, executable, dest,
253 scripts=None,
254 extra_paths=(),
255@@ -912,20 +980,86 @@
256 initialization='',
257 relative_paths=False,
258 ):
259-
260+ """Generate scripts and/or an interpreter.
261+
262+ See sitepackage_safe_scripts for a version that can be used with a Python
263+ that has code installed in site-packages. It has more options and a
264+ different approach.
265+ """
266+ path = _get_path(working_set, extra_paths)
267+ if initialization:
268+ initialization = '\n'+initialization+'\n'
269+ generated = _generate_scripts(
270+ reqs, working_set, dest, path, scripts, relative_paths,
271+ initialization, executable, arguments)
272+ if interpreter:
273+ sname = os.path.join(dest, interpreter)
274+ spath, rpsetup = _relative_path_and_setup(sname, path, relative_paths)
275+ generated.extend(
276+ _pyscript(spath, sname, executable, rpsetup))
277+ return generated
278+
279+def sitepackage_safe_scripts(
280+ dest, working_set, executable, site_py_dest,
281+ reqs=(), scripts=None, interpreter=None, extra_paths=(),
282+ initialization='', add_site_packages=False, exec_sitecustomize=False,
283+ relative_paths=False, script_arguments='', script_initialization=''):
284+ """Generate scripts and/or an interpreter from a system Python.
285+
286+ This accomplishes the same job as the ``scripts`` function, above,
287+ but it does so in an alternative way that allows safely including
288+ Python site packages, if desired, and choosing to execute the Python's
289+ sitecustomize.
290+ """
291+ generated = []
292+ generated.append(_generate_sitecustomize(
293+ site_py_dest, executable, initialization, exec_sitecustomize))
294+ generated.append(_generate_site(
295+ site_py_dest, working_set, executable, extra_paths,
296+ add_site_packages, relative_paths))
297+ script_initialization = (
298+ '\nimport site # imports custom buildout-generated site.py\n%s' % (
299+ script_initialization,))
300+ if not script_initialization.endswith('\n'):
301+ script_initialization += '\n'
302+ generated.extend(_generate_scripts(
303+ reqs, working_set, dest, [site_py_dest], scripts, relative_paths,
304+ script_initialization, executable, script_arguments, block_site=True))
305+ if interpreter:
306+ generated.extend(_generate_interpreter(
307+ interpreter, dest, executable, site_py_dest, relative_paths))
308+ return generated
309+
310+# Utilities for the script generation functions.
311+
312+# These are shared by both ``scripts`` and ``sitepackage_safe_scripts``
313+
314+def _get_path(working_set, extra_paths=()):
315+ """Given working set and extra paths, return a normalized path list."""
316 path = [dist.location for dist in working_set]
317 path.extend(extra_paths)
318- path = map(realpath, path)
319-
320- generated = []
321-
322+ return map(realpath, path)
323+
324+def _generate_scripts(reqs, working_set, dest, path, scripts, relative_paths,
325+ initialization, executable, arguments,
326+ block_site=False):
327+ """Generate scripts for the given requirements.
328+
329+ - reqs is an iterable of string requirements or entry points.
330+ - The requirements must be findable in the given working_set.
331+ - The dest is the directory in which the scripts should be created.
332+ - The path is a list of paths that should be added to sys.path.
333+ - The scripts is an optional dictionary. If included, the keys should be
334+ the names of the scripts that should be created, as identified in their
335+ entry points; and the values should be the name the script should
336+ actually be created with.
337+ - relative_paths, if given, should be the path that is the root of the
338+ buildout (the common path that should be the root of what is relative).
339+ """
340 if isinstance(reqs, str):
341 raise TypeError('Expected iterable of requirements or entry points,'
342 ' got string.')
343-
344- if initialization:
345- initialization = '\n'+initialization+'\n'
346-
347+ generated = []
348 entry_points = []
349 for req in reqs:
350 if isinstance(req, str):
351@@ -939,7 +1073,6 @@
352 )
353 else:
354 entry_points.append(req)
355-
356 for name, module_name, attrs in entry_points:
357 if scripts is not None:
358 sname = scripts.get(name)
359@@ -947,40 +1080,51 @@
360 continue
361 else:
362 sname = name
363-
364 sname = os.path.join(dest, sname)
365 spath, rpsetup = _relative_path_and_setup(sname, path, relative_paths)
366-
367 generated.extend(
368- _script(module_name, attrs, spath, sname, executable, arguments,
369- initialization, rpsetup)
370- )
371-
372- if interpreter:
373- sname = os.path.join(dest, interpreter)
374- spath, rpsetup = _relative_path_and_setup(sname, path, relative_paths)
375- generated.extend(_pyscript(spath, sname, executable, rpsetup))
376-
377+ _script(sname, executable, rpsetup, spath, initialization,
378+ module_name, attrs, arguments, block_site=block_site))
379 return generated
380
381-def _relative_path_and_setup(sname, path, relative_paths):
382+def _relative_path_and_setup(sname, path,
383+ relative_paths=False, indent_level=1,
384+ omit_os_import=False):
385+ """Return a string of code of paths and of setup if appropriate.
386+
387+ - sname is the full path to the script name to be created.
388+ - path is the list of paths to be added to sys.path.
389+ - relative_paths, if given, should be the path that is the root of the
390+ buildout (the common path that should be the root of what is relative).
391+ - indent_level is the number of four-space indents that the path should
392+ insert before each element of the path.
393+ """
394 if relative_paths:
395 relative_paths = os.path.normcase(relative_paths)
396 sname = os.path.normcase(os.path.abspath(sname))
397- spath = ',\n '.join(
398+ spath = _format_paths(
399 [_relativitize(os.path.normcase(path_item), sname, relative_paths)
400- for path_item in path]
401- )
402+ for path_item in path], indent_level=indent_level)
403 rpsetup = relative_paths_setup
404+ if not omit_os_import:
405+ rpsetup = '\n\nimport os\n' + rpsetup
406 for i in range(_relative_depth(relative_paths, sname)):
407- rpsetup += "base = os.path.dirname(base)\n"
408+ rpsetup += "\nbase = os.path.dirname(base)"
409 else:
410- spath = repr(path)[1:-1].replace(', ', ',\n ')
411+ spath = _format_paths((repr(p) for p in path),
412+ indent_level=indent_level)
413 rpsetup = ''
414 return spath, rpsetup
415
416-
417 def _relative_depth(common, path):
418+ """Return number of dirs separating ``path`` from ancestor, ``common``.
419+
420+ For instance, if path is /foo/bar/baz/bing, and common is /foo, this will
421+ return 2--in UNIX, the number of ".." to get from bing's directory
422+ to foo.
423+
424+ This is a helper for _relative_path_and_setup.
425+ """
426 n = 0
427 while 1:
428 dirname = os.path.dirname(path)
429@@ -993,6 +1137,11 @@
430 return n
431
432 def _relative_path(common, path):
433+ """Return the relative path from ``common`` to ``path``.
434+
435+ This is a helper for _relativitize, which is a helper to
436+ _relative_path_and_setup.
437+ """
438 r = []
439 while 1:
440 dirname, basename = os.path.split(path)
441@@ -1006,6 +1155,11 @@
442 return os.path.join(*r)
443
444 def _relativitize(path, script, relative_paths):
445+ """Return a code string for the given path.
446+
447+ Path is relative to the base path ``relative_paths``if the common prefix
448+ between ``path`` and ``script`` starts with ``relative_paths``.
449+ """
450 if path == script:
451 raise AssertionError("path == script")
452 common = os.path.dirname(os.path.commonprefix([path, script]))
453@@ -1016,66 +1170,82 @@
454 else:
455 return repr(path)
456
457-
458 relative_paths_setup = """
459-import os
460-
461 join = os.path.join
462-base = os.path.dirname(os.path.abspath(os.path.realpath(__file__)))
463-"""
464-
465-def _script(module_name, attrs, path, dest, executable, arguments,
466- initialization, rsetup):
467+base = os.path.dirname(os.path.abspath(os.path.realpath(__file__)))"""
468+
469+def _write_script(full_name, contents, logged_type):
470+ """Write contents of script in full_name, logging the action.
471+
472+ The only tricky bit in this function is that it supports Windows by
473+ creating exe files using a pkg_resources helper.
474+ """
475 generated = []
476- script = dest
477- if is_win32:
478- dest += '-script.py'
479-
480- contents = script_template % dict(
481- python = _safe_arg(executable),
482- path = path,
483- module_name = module_name,
484- attrs = attrs,
485- arguments = arguments,
486- initialization = initialization,
487- relative_paths_setup = rsetup,
488- )
489- changed = not (os.path.exists(dest) and open(dest).read() == contents)
490-
491- if is_win32:
492- # generate exe file and give the script a magic name:
493- exe = script+'.exe'
494+ script_name = full_name
495+ if is_win32:
496+ script_name += '-script.py'
497+ # Generate exe file and give the script a magic name.
498+ exe = full_name + '.exe'
499 new_data = pkg_resources.resource_string('setuptools', 'cli.exe')
500 if not os.path.exists(exe) or (open(exe, 'rb').read() != new_data):
501 # Only write it if it's different.
502 open(exe, 'wb').write(new_data)
503 generated.append(exe)
504-
505+ changed = not (os.path.exists(script_name) and
506+ open(script_name).read() == contents)
507 if changed:
508- open(dest, 'w').write(contents)
509- logger.info("Generated script %r.", script)
510-
511+ open(script_name, 'w').write(contents)
512 try:
513- os.chmod(dest, 0755)
514+ os.chmod(script_name, 0755)
515 except (AttributeError, os.error):
516 pass
517-
518- generated.append(dest)
519+ logger.info("Generated %s %r.", logged_type, full_name)
520+ generated.append(script_name)
521 return generated
522
523+def _format_paths(paths, indent_level=1):
524+ """Format paths for inclusion in a script."""
525+ separator = ',\n' + indent_level * ' '
526+ return separator.join(paths)
527+
528+def _script(dest, executable, relative_paths_setup, path, initialization,
529+ module_name, attrs, arguments, block_site=False):
530+ if block_site:
531+ dash_S = ' -S'
532+ else:
533+ dash_S = ''
534+ contents = script_template % dict(
535+ python=_safe_arg(executable),
536+ dash_S=dash_S,
537+ path=path,
538+ module_name=module_name,
539+ attrs=attrs,
540+ arguments=arguments,
541+ initialization=initialization,
542+ relative_paths_setup=relative_paths_setup,
543+ )
544+ return _write_script(dest, contents, 'script')
545+
546 if is_jython and jython_os_name == 'linux':
547- script_header = '#!/usr/bin/env %(python)s'
548+ script_header = '#!/usr/bin/env %(python)s%(dash_S)s'
549 else:
550- script_header = '#!%(python)s'
551+ script_header = '#!%(python)s%(dash_S)s'
552
553+sys_path_template = '''\
554+import sys
555+sys.path[0:0] = [
556+ %s,
557+ ]
558+'''
559
560 script_template = script_header + '''\
561-
562 %(relative_paths_setup)s
563+
564 import sys
565 sys.path[0:0] = [
566- %(path)s,
567- ]
568+ %(path)s,
569+ ]
570+
571 %(initialization)s
572 import %(module_name)s
573
574@@ -1083,47 +1253,25 @@
575 %(module_name)s.%(attrs)s(%(arguments)s)
576 '''
577
578+# These are used only by the older ``scripts`` function.
579
580 def _pyscript(path, dest, executable, rsetup):
581- generated = []
582- script = dest
583- if is_win32:
584- dest += '-script.py'
585-
586 contents = py_script_template % dict(
587- python = _safe_arg(executable),
588- path = path,
589- relative_paths_setup = rsetup,
590+ python=_safe_arg(executable),
591+ dash_S='',
592+ path=path,
593+ relative_paths_setup=rsetup,
594 )
595- changed = not (os.path.exists(dest) and open(dest).read() == contents)
596-
597- if is_win32:
598- # generate exe file and give the script a magic name:
599- exe = script + '.exe'
600- open(exe, 'wb').write(
601- pkg_resources.resource_string('setuptools', 'cli.exe')
602- )
603- generated.append(exe)
604-
605- if changed:
606- open(dest, 'w').write(contents)
607- try:
608- os.chmod(dest,0755)
609- except (AttributeError, os.error):
610- pass
611- logger.info("Generated interpreter %r.", script)
612-
613- generated.append(dest)
614- return generated
615+ return _write_script(dest, contents, 'interpreter')
616
617 py_script_template = script_header + '''\
618-
619 %(relative_paths_setup)s
620+
621 import sys
622
623 sys.path[0:0] = [
624- %(path)s,
625- ]
626+ %(path)s,
627+ ]
628
629 _interactive = True
630 if len(sys.argv) > 1:
631@@ -1151,6 +1299,274 @@
632 __import__("code").interact(banner="", local=globals())
633 '''
634
635+# These are used only by the newer ``sitepackage_safe_scripts`` function.
636+
637+def _get_system_paths(executable):
638+ """Return lists of standard lib and site paths for executable.
639+ """
640+ # We want to get a list of the site packages, which is not easy.
641+ # The canonical way to do this is to use
642+ # distutils.sysconfig.get_python_lib(), but that only returns a
643+ # single path, which does not reflect reality for many system
644+ # Pythons, which have multiple additions. Instead, we start Python
645+ # with -S, which does not import site.py and set up the extra paths
646+ # like site-packages or (Ubuntu/Debian) dist-packages and
647+ # python-support. We then compare that sys.path with the normal one
648+ # (minus user packages if this is Python 2.6, because we don't
649+ # support those (yet?). The set of the normal one minus the set of
650+ # the ones in ``python -S`` is the set of packages that are
651+ # effectively site-packages.
652+ #
653+ # The given executable might not be the current executable, so it is
654+ # appropriate to do another subprocess to figure out what the
655+ # additional site-package paths are. Moreover, even if this
656+ # executable *is* the current executable, this code might be run in
657+ # the context of code that has manipulated the sys.path--for
658+ # instance, to add local zc.buildout or setuptools eggs.
659+ def get_sys_path(*args, **kwargs):
660+ cmd = [executable]
661+ cmd.extend(args)
662+ cmd.extend([
663+ "-c", "import sys, os;"
664+ "print repr([os.path.normpath(p) for p in sys.path if p])"])
665+ # Windows needs some (as yet to be determined) part of the real env.
666+ env = os.environ.copy()
667+ env.update(kwargs)
668+ _proc = subprocess.Popen(
669+ cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, env=env)
670+ stdout, stderr = _proc.communicate();
671+ if _proc.returncode:
672+ raise RuntimeError(
673+ 'error trying to get system packages:\n%s' % (stderr,))
674+ res = eval(stdout.strip())
675+ try:
676+ res.remove('.')
677+ except ValueError:
678+ pass
679+ return res
680+ stdlib = get_sys_path('-S') # stdlib only
681+ no_user_paths = get_sys_path(PYTHONNOUSERSITE='x')
682+ site_paths = [p for p in no_user_paths if p not in stdlib]
683+ return (stdlib, site_paths)
684+
685+def _get_module_file(executable, name):
686+ """Return a module's file path.
687+
688+ - executable is a path to the desired Python executable.
689+ - name is the name of the (pure, not C) Python module.
690+ """
691+ cmd = [executable, "-c",
692+ "import imp; "
693+ "fp, path, desc = imp.find_module(%r); "
694+ "fp.close; "
695+ "print path" % (name,)]
696+ _proc = subprocess.Popen(
697+ cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
698+ stdout, stderr = _proc.communicate();
699+ if _proc.returncode:
700+ logger.info(
701+ 'Could not find file for module %s:\n%s', name, stderr)
702+ return None
703+ # else: ...
704+ res = stdout.strip()
705+ if res.endswith('.pyc') or res.endswith('.pyo'):
706+ raise RuntimeError('Cannot find uncompiled version of %s' % (name,))
707+ if not os.path.exists(res):
708+ raise RuntimeError(
709+ 'File does not exist for module %s:\n%s' % (name, res))
710+ return res
711+
712+def _generate_sitecustomize(dest, executable, initialization='',
713+ exec_sitecustomize=False):
714+ """Write a sitecustomize file with optional custom initialization.
715+
716+ The created script will execute the underlying Python's
717+ sitecustomize if exec_sitecustomize is True.
718+ """
719+ sitecustomize_path = os.path.join(dest, 'sitecustomize.py')
720+ sitecustomize = open(sitecustomize_path, 'w')
721+ if initialization:
722+ sitecustomize.write(initialization + '\n')
723+ if exec_sitecustomize:
724+ real_sitecustomize_path = _get_module_file(
725+ executable, 'sitecustomize')
726+ if real_sitecustomize_path:
727+ real_sitecustomize = open(real_sitecustomize_path, 'r')
728+ sitecustomize.write(
729+ '\n# The following is from\n# %s\n' %
730+ (real_sitecustomize_path,))
731+ sitecustomize.write(real_sitecustomize.read())
732+ real_sitecustomize.close()
733+ sitecustomize.close()
734+ return sitecustomize_path
735+
736+def _generate_site(dest, working_set, executable, extra_paths=(),
737+ add_site_packages=False, relative_paths=False):
738+ """Write a site.py file with eggs from working_set.
739+
740+ extra_paths will be added to the path. If add_site_packages is True,
741+ paths from the underlying Python will be added.
742+ """
743+ path = _get_path(working_set, extra_paths)
744+ site_path = os.path.join(dest, 'site.py')
745+ egg_path_string, preamble = _relative_path_and_setup(
746+ site_path, path, relative_paths, indent_level=2, omit_os_import=True)
747+ if preamble:
748+ preamble = '\n'.join(
749+ [(line and ' %s' % (line,) or line)
750+ for line in preamble.split('\n')])
751+ original_path_setup = ''
752+ if add_site_packages:
753+ stdlib, site_paths = _get_system_paths(executable)
754+ original_path_setup = original_path_snippet % (
755+ _format_paths((repr(p) for p in site_paths), 2),)
756+ distribution = working_set.find(
757+ pkg_resources.Requirement.parse('setuptools'))
758+ if distribution is not None:
759+ # We need to worry about namespace packages.
760+ if relative_paths:
761+ location = _relativitize(
762+ distribution.location,
763+ os.path.normcase(os.path.abspath(site_path)),
764+ relative_paths)
765+ else:
766+ location = repr(distribution.location)
767+ preamble += namespace_add_site_packages_setup % (location,)
768+ original_path_setup = (
769+ addsitedir_namespace_originalpackages_snippet +
770+ original_path_setup)
771+ addsitepackages_marker = 'def addsitepackages('
772+ enableusersite_marker = 'ENABLE_USER_SITE = '
773+ successful_rewrite = False
774+ real_site_path = _get_module_file(executable, 'site')
775+ real_site = open(real_site_path, 'r')
776+ site = open(site_path, 'w')
777+ try:
778+ for line in real_site.readlines():
779+ if line.startswith(enableusersite_marker):
780+ site.write(enableusersite_marker)
781+ site.write('False # buildout does not support user sites.\n')
782+ elif line.startswith(addsitepackages_marker):
783+ site.write(addsitepackages_script % (
784+ preamble, egg_path_string, original_path_setup))
785+ site.write(line[len(addsitepackages_marker):])
786+ successful_rewrite = True
787+ else:
788+ site.write(line)
789+ finally:
790+ site.close()
791+ real_site.close()
792+ if not successful_rewrite:
793+ raise RuntimeError('Buildout did not successfully rewrite site.py')
794+ return site_path
795+
796+namespace_add_site_packages_setup = '''
797+ setuptools_path = %s
798+ sys.path.append(setuptools_path)
799+ known_paths.add(os.path.normcase(setuptools_path))
800+ import pkg_resources'''
801+
802+addsitedir_namespace_originalpackages_snippet = '''
803+ pkg_resources.working_set.add_entry(sitedir)'''
804+
805+original_path_snippet = '''
806+ original_paths = [
807+ %s
808+ ]
809+ for path in original_paths:
810+ addsitedir(path, known_paths)'''
811+
812+addsitepackages_script = '''\
813+def addsitepackages(known_paths):
814+ """Add site packages, as determined by zc.buildout.
815+
816+ See original_addsitepackages, below, for the original version."""%s
817+ buildout_paths = [
818+ %s
819+ ]
820+ for path in buildout_paths:
821+ sitedir, sitedircase = makepath(path)
822+ if not sitedircase in known_paths and os.path.exists(sitedir):
823+ sys.path.append(sitedir)
824+ known_paths.add(sitedircase)%s
825+ return known_paths
826+
827+def original_addsitepackages('''
828+
829+def _generate_interpreter(name, dest, executable, site_py_dest,
830+ relative_paths=False):
831+ """Write an interpreter script, using the site.py approach."""
832+ full_name = os.path.join(dest, name)
833+ site_py_dest_string, rpsetup = _relative_path_and_setup(
834+ full_name, [site_py_dest], relative_paths, omit_os_import=True)
835+ if rpsetup:
836+ rpsetup += "\n"
837+ if sys.platform == 'win32':
838+ windows_import = '\nimport subprocess'
839+ # os.exec* is a mess on Windows, particularly if the path
840+ # to the executable has spaces and the Python is using MSVCRT.
841+ # The standard fix is to surround the executable's path with quotes,
842+ # but that has been unreliable in testing.
843+ #
844+ # Here's a demonstration of the problem. Given a Python
845+ # compiled with a MSVCRT-based compiler, such as the free Visual
846+ # C++ 2008 Express Edition, and an executable path with spaces
847+ # in it such as the below, we see the following.
848+ #
849+ # >>> import os
850+ # >>> p0 = 'C:\\Documents and Settings\\Administrator\\My Documents\\Downloads\\Python-2.6.4\\PCbuild\\python.exe'
851+ # >>> os.path.exists(p0)
852+ # True
853+ # >>> os.execv(p0, [])
854+ # Traceback (most recent call last):
855+ # File "<stdin>", line 1, in <module>
856+ # OSError: [Errno 22] Invalid argument
857+ #
858+ # That seems like a standard problem. The standard solution is
859+ # to quote the path (see, for instance
860+ # http://bugs.python.org/issue436259). However, this solution,
861+ # and other variations, fail:
862+ #
863+ # >>> p1 = '"C:\\Documents and Settings\\Administrator\\My Documents\\Downloads\\Python-2.6.4\\PCbuild\\python.exe"'
864+ # >>> os.execv(p1, [])
865+ # Traceback (most recent call last):
866+ # File "<stdin>", line 1, in <module>
867+ # OSError: [Errno 22] Invalid argument
868+ #
869+ # We simply use subprocess instead, since it handles everything
870+ # nicely, and the transparency of exec* (that is, not running,
871+ # perhaps unexpectedly, in a subprocess) is arguably not a
872+ # necessity, at least for many use cases.
873+ execute = 'subprocess.call(argv, env=environ)'
874+ else:
875+ windows_import = ''
876+ execute = 'os.execve(sys.executable, argv, environ)'
877+ contents = interpreter_template % dict(
878+ python=_safe_arg(executable),
879+ dash_S=' -S',
880+ site_dest=site_py_dest_string,
881+ relative_paths_setup=rpsetup,
882+ windows_import=windows_import,
883+ execute=execute,
884+ )
885+ return _write_script(full_name, contents, 'interpreter')
886+
887+interpreter_template = script_header + '''
888+import os
889+import sys%(windows_import)s
890+%(relative_paths_setup)s
891+argv = [sys.executable] + sys.argv[1:]
892+environ = os.environ.copy()
893+path = %(site_dest)s
894+if environ.get('PYTHONPATH'):
895+ path = os.pathsep.join([path, environ['PYTHONPATH']])
896+environ['PYTHONPATH'] = path
897+%(execute)s
898+'''
899+
900+# End of script generation code.
901+############################################################################
902+
903 runsetup_template = """
904 import sys
905 sys.path.insert(0, %(setupdir)r)
906
907=== modified file 'src/zc/buildout/easy_install.txt'
908--- src/zc/buildout/easy_install.txt 2009-11-11 21:21:11 +0000
909+++ src/zc/buildout/easy_install.txt 2010-02-23 20:41:17 +0000
910@@ -521,25 +521,38 @@
911 Script generation
912 -----------------
913
914-The easy_install module provides support for creating scripts from
915-eggs. It provides a function similar to setuptools except that it
916-provides facilities for baking a script's path into the script. This
917-has two advantages:
918+The easy_install module provides support for creating scripts from eggs.
919+It provides two competing functions. One, ``scripts``, is a
920+well-established approach to generating reliable scripts with a "clean"
921+Python--e.g., one that does not have any packages in its site-packages.
922+The other, ``sitepackage_safe_scripts``, is newer, a bit trickier, and is
923+designed to work with a Python that has code in its site-packages, such
924+as a system Python.
925+
926+Both are similar to setuptools except that they provides facilities for
927+baking a script's path into the script. This has two advantages:
928
929 - The eggs to be used by a script are not chosen at run time, making
930 startup faster and, more importantly, deterministic.
931
932-- The script doesn't have to import pkg_resources because the logic
933- that pkg_resources would execute at run time is executed at
934- script-creation time.
935-
936-The scripts method can be used to generate scripts. Let's create a
937-destination directory for it to place them in:
938-
939- >>> import tempfile
940+- The script doesn't have to import pkg_resources because the logic that
941+ pkg_resources would execute at run time is executed at script-creation
942+ time. (There is an exception in ``sitepackage_safe_scripts`` if you
943+ want to have your Python's site packages available, as discussed
944+ below, but even in that case pkg_resources is only partially
945+ activated, which can be a significant time savings.)
946+
947+
948+The ``scripts`` function
949+~~~~~~~~~~~~~~~~~~~~~~~~
950+
951+The ``scripts`` function is the first way to generate scripts that we'll
952+examine. It is the earlier approach that the package offered. Let's
953+create a destination directory for it to place them in:
954+
955 >>> bin = tmpdir('bin')
956
957-Now, we'll use the scripts method to generate scripts in this directory
958+Now, we'll use the scripts function to generate scripts in this directory
959 from the demo egg:
960
961 >>> import sys
962@@ -736,8 +749,8 @@
963 >>> print system(os.path.join(bin, 'run')),
964 3 1
965
966-Including extra paths in scripts
967---------------------------------
968+The ``scripts`` function: Including extra paths in scripts
969+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
970
971 We can pass a keyword argument, extra paths, to cause additional paths
972 to be included in the a generated script:
973@@ -762,8 +775,8 @@
974 if __name__ == '__main__':
975 eggrecipedemo.main()
976
977-Providing script arguments
978---------------------------
979+The ``scripts`` function: Providing script arguments
980+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
981
982 An "argument" keyword argument can be used to pass arguments to an
983 entry point. The value passed is a source string to be placed between the
984@@ -786,8 +799,8 @@
985 if __name__ == '__main__':
986 eggrecipedemo.main(1, 2)
987
988-Passing initialization code
989----------------------------
990+The ``scripts`` function: Passing initialization code
991+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
992
993 You can also pass script initialization code:
994
995@@ -812,8 +825,8 @@
996 if __name__ == '__main__':
997 eggrecipedemo.main(1, 2)
998
999-Relative paths
1000---------------
1001+The ``scripts`` function: Relative paths
1002+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1003
1004 Sometimes, you want to be able to move a buildout directory around and
1005 have scripts still work without having to rebuild them. We can
1006@@ -836,7 +849,7 @@
1007 ... interpreter='py',
1008 ... relative_paths=bo)
1009
1010- >>> cat(bo, 'bin', 'run')
1011+ >>> cat(bo, 'bin', 'run') # doctest: +NORMALIZE_WHITESPACE
1012 #!/usr/local/bin/python2.4
1013 <BLANKLINE>
1014 import os
1015@@ -868,7 +881,7 @@
1016
1017 We specified an interpreter and its paths are adjusted too:
1018
1019- >>> cat(bo, 'bin', 'py')
1020+ >>> cat(bo, 'bin', 'py') # doctest: +NORMALIZE_WHITESPACE
1021 #!/usr/local/bin/python2.4
1022 <BLANKLINE>
1023 import os
1024@@ -911,6 +924,557 @@
1025 del _interactive
1026 __import__("code").interact(banner="", local=globals())
1027
1028+The ``sitepackage_safe_scripts`` function
1029+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1030+
1031+The newer function for creating scripts is ``sitepackage_safe_scripts``.
1032+ It has the same basic functionality as the ``scripts`` function: it can
1033+create scripts to run arbitrary entry points, and to run a Python
1034+interpreter. The following are the differences from a user's
1035+perspective.
1036+
1037+- It can be used safely with a Python that has packages installed itself,
1038+ such as a system-installed Python.
1039+
1040+- In contrast to the interpreter generated by the ``scripts`` method, which
1041+ supports only a small subset of the usual Python executable's options,
1042+ the interpreter generated by ``sitepackage_safe_scripts`` supports all
1043+ of them. This makes it possible to use as full Python replacement for
1044+ scripts that need the distributions specified in your buildout.
1045+
1046+- Both the interpreter and the entry point scripts allow you to include the
1047+ site packages, and/or the sitecustomize, of the Python executable, if
1048+ desired.
1049+
1050+It works by creating site.py and sitecustomize.py files that set up the
1051+desired paths and initialization. These must be placed within an otherwise
1052+empty directory. Typically this is in a recipe's parts directory.
1053+
1054+Here's the simplest example, building an interpreter script.
1055+
1056+ >>> interpreter_dir = tmpdir('interpreter')
1057+ >>> interpreter_parts_dir = os.path.join(
1058+ ... interpreter_dir, 'parts', 'interpreter')
1059+ >>> interpreter_bin_dir = os.path.join(interpreter_dir, 'bin')
1060+ >>> mkdir(interpreter_bin_dir)
1061+ >>> mkdir(interpreter_dir, 'eggs')
1062+ >>> mkdir(interpreter_dir, 'parts')
1063+ >>> mkdir(interpreter_parts_dir)
1064+
1065+ >>> ws = zc.buildout.easy_install.install(
1066+ ... ['demo'], join(interpreter_dir, 'eggs'), links=[link_server],
1067+ ... index=link_server+'index/')
1068+ >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts(
1069+ ... interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir,
1070+ ... interpreter='py')
1071+
1072+Depending on whether the machine being used is running Windows or not, this
1073+produces either three or four files. In both cases, we have site.py and
1074+sitecustomize.py generated in the parts/interpreter directory. For Windows,
1075+we have py.exe and py-script.py; for other operating systems, we have py.
1076+
1077+ >>> sitecustomize_path = os.path.join(
1078+ ... interpreter_parts_dir, 'sitecustomize.py')
1079+ >>> site_path = os.path.join(interpreter_parts_dir, 'site.py')
1080+ >>> interpreter_path = os.path.join(interpreter_bin_dir, 'py')
1081+ >>> if sys.platform == 'win32':
1082+ ... py_path = os.path.join(interpreter_bin_dir, 'py-script.py')
1083+ ... expected = [sitecustomize_path,
1084+ ... site_path,
1085+ ... os.path.join(interpreter_bin_dir, 'py.exe'),
1086+ ... py_path]
1087+ ... else:
1088+ ... py_path = interpreter_path
1089+ ... expected = [sitecustomize_path, site_path, py_path]
1090+ ...
1091+ >>> assert generated == expected, repr((generated, expected))
1092+
1093+We didn't ask for any initialization, and we didn't ask to use the underlying
1094+sitecustomization, so sitecustomize.py is empty.
1095+
1096+ >>> cat(sitecustomize_path)
1097+
1098+The interpreter script is simple. It puts the directory with the
1099+site.py and sitecustomize.py on the PYTHONPATH and (re)starts Python.
1100+
1101+ >>> cat(py_path)
1102+ #!/usr/bin/python -S
1103+ import os
1104+ import sys
1105+ <BLANKLINE>
1106+ argv = [sys.executable] + sys.argv[1:]
1107+ environ = os.environ.copy()
1108+ path = '/interpreter/parts/interpreter'
1109+ if environ.get('PYTHONPATH'):
1110+ path = os.pathsep.join([path, environ['PYTHONPATH']])
1111+ environ['PYTHONPATH'] = path
1112+ os.execve(sys.executable, argv, environ)
1113+
1114+The site.py file is a modified version of the underlying Python's site.py.
1115+The most important modification is that it has a different version of the
1116+addsitepackages function. It sets up the Python path, similarly to the
1117+behavior of the function it replaces. The following shows the part that
1118+buildout inserts, in the simplest case.
1119+
1120+ >>> sys.stdout.write('#\n'); cat(site_path)
1121+ ... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE
1122+ #...
1123+ def addsitepackages(known_paths):
1124+ """Add site packages, as determined by zc.buildout.
1125+ <BLANKLINE>
1126+ See original_addsitepackages, below, for the original version."""
1127+ buildout_paths = [
1128+ '/interpreter/eggs/demo-0.3-pyN.N.egg',
1129+ '/interpreter/eggs/demoneeded-1.1-pyN.N.egg'
1130+ ]
1131+ for path in buildout_paths:
1132+ sitedir, sitedircase = makepath(path)
1133+ if not sitedircase in known_paths and os.path.exists(sitedir):
1134+ sys.path.append(sitedir)
1135+ known_paths.add(sitedircase)
1136+ return known_paths
1137+ <BLANKLINE>
1138+ def original_addsitepackages(known_paths):...
1139+
1140+Here are some examples of the interpreter in use.
1141+
1142+ >>> print call_py(interpreter_path, "print 16+26")
1143+ 42
1144+ <BLANKLINE>
1145+ >>> res = call_py(interpreter_path, "import sys; print sys.path")
1146+ >>> print res # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE
1147+ ['',
1148+ '/interpreter/parts/interpreter',
1149+ ...,
1150+ '/interpreter/eggs/demo-0.3-pyN.N.egg',
1151+ '/interpreter/eggs/demoneeded-1.1-pyN.N.egg']
1152+ <BLANKLINE>
1153+ >>> clean_paths = eval(res.strip()) # This is used later for comparison.
1154+
1155+If you provide initialization, it goes in sitecustomize.py.
1156+
1157+ >>> def reset_interpreter():
1158+ ... # This is necessary because, in our tests, the timestamps of the
1159+ ... # .pyc files are not outdated when we want them to be.
1160+ ... rmdir(interpreter_bin_dir)
1161+ ... mkdir(interpreter_bin_dir)
1162+ ... rmdir(interpreter_parts_dir)
1163+ ... mkdir(interpreter_parts_dir)
1164+ ...
1165+ >>> reset_interpreter()
1166+
1167+ >>> initialization_string = """\
1168+ ... import os
1169+ ... os.environ['FOO'] = 'bar baz bing shazam'"""
1170+ >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts(
1171+ ... interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir,
1172+ ... interpreter='py', initialization=initialization_string)
1173+ >>> cat(sitecustomize_path)
1174+ import os
1175+ os.environ['FOO'] = 'bar baz bing shazam'
1176+ >>> print call_py(interpreter_path, "import os; print os.environ['FOO']")
1177+ bar baz bing shazam
1178+ <BLANKLINE>
1179+
1180+If you use relative paths, this affects the interpreter and site.py. (This is
1181+again the UNIX version; the Windows version uses subprocess instead of
1182+os.execve.)
1183+
1184+ >>> reset_interpreter()
1185+ >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts(
1186+ ... interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir,
1187+ ... interpreter='py', relative_paths=interpreter_dir)
1188+ >>> cat(py_path)
1189+ #!/usr/bin/python -S
1190+ import os
1191+ import sys
1192+ <BLANKLINE>
1193+ join = os.path.join
1194+ base = os.path.dirname(os.path.abspath(os.path.realpath(__file__)))
1195+ base = os.path.dirname(base)
1196+ <BLANKLINE>
1197+ argv = [sys.executable] + sys.argv[1:]
1198+ environ = os.environ.copy()
1199+ path = join(base, 'parts/interpreter')
1200+ if environ.get('PYTHONPATH'):
1201+ path = os.pathsep.join([path, environ['PYTHONPATH']])
1202+ environ['PYTHONPATH'] = path
1203+ os.execve(sys.executable, argv, environ)
1204+
1205+For site.py, we again show only the pertinent parts. Notice that the egg
1206+paths join a base to a path, as with the use of this argument in the
1207+``scripts`` function.
1208+
1209+ >>> sys.stdout.write('#\n'); cat(site_path) # doctest: +ELLIPSIS
1210+ #...
1211+ def addsitepackages(known_paths):
1212+ """Add site packages, as determined by zc.buildout.
1213+ <BLANKLINE>
1214+ See original_addsitepackages, below, for the original version."""
1215+ join = os.path.join
1216+ base = os.path.dirname(os.path.abspath(os.path.realpath(__file__)))
1217+ base = os.path.dirname(base)
1218+ base = os.path.dirname(base)
1219+ buildout_paths = [
1220+ join(base, 'eggs/demo-0.3-pyN.N.egg'),
1221+ join(base, 'eggs/demoneeded-1.1-pyN.N.egg')
1222+ ]...
1223+
1224+The paths resolve in practice as you would expect.
1225+
1226+ >>> print call_py(interpreter_path,
1227+ ... "import sys, pprint; pprint.pprint(sys.path)")
1228+ ... # doctest: +ELLIPSIS
1229+ ['',
1230+ '/interpreter/parts/interpreter',
1231+ ...,
1232+ '/interpreter/eggs/demo-0.3-pyN.N.egg',
1233+ '/interpreter/eggs/demoneeded-1.1-pyN.N.egg']
1234+ <BLANKLINE>
1235+
1236+The ``extra_paths`` argument affects the path in site.py. Notice that
1237+/interpreter/other is added after the eggs.
1238+
1239+ >>> reset_interpreter()
1240+ >>> mkdir(interpreter_dir, 'other')
1241+ >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts(
1242+ ... interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir,
1243+ ... interpreter='py', extra_paths=[join(interpreter_dir, 'other')])
1244+ >>> sys.stdout.write('#\n'); cat(site_path) # doctest: +ELLIPSIS
1245+ #...
1246+ def addsitepackages(known_paths):
1247+ """Add site packages, as determined by zc.buildout.
1248+ <BLANKLINE>
1249+ See original_addsitepackages, below, for the original version."""
1250+ buildout_paths = [
1251+ '/interpreter/eggs/demo-0.3-pyN.N.egg',
1252+ '/interpreter/eggs/demoneeded-1.1-pyN.N.egg',
1253+ '/interpreter/other'
1254+ ]...
1255+
1256+ >>> print call_py(interpreter_path,
1257+ ... "import sys, pprint; pprint.pprint(sys.path)")
1258+ ... # doctest: +ELLIPSIS
1259+ ['',
1260+ '/interpreter/parts/interpreter',
1261+ ...,
1262+ '/interpreter/eggs/demo-0.3-pyN.N.egg',
1263+ '/interpreter/eggs/demoneeded-1.1-pyN.N.egg',
1264+ '/interpreter/other']
1265+ <BLANKLINE>
1266+
1267+The ``sitepackage_safe_scripts`` function: using site-packages
1268+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1269+
1270+The ``sitepackage_safe_scripts`` function supports including site
1271+packages. This has some advantages and some serious dangers.
1272+
1273+A typical reason to include site-packages is that it is easier to
1274+install one or more dependencies in your Python than it is with
1275+buildout. Some packages, such as lxml or Python PostgreSQL integration,
1276+have dependencies that can be much easier to build and/or install using
1277+other mechanisms, such as your operating system's package manager. By
1278+installing some core packages into your Python's site-packages, this can
1279+significantly simplify some application installations.
1280+
1281+However, doing this has a significant danger. One of the primary goals
1282+of buildout is to provide repeatability. Some packages (one of the
1283+better known Python openid packages, for instance) change their behavior
1284+depending on what packages are available. If Python curl bindings are
1285+available, these may be preferred by the library. If a certain XML
1286+package is installed, it may be preferred by the library. These hidden
1287+choices may cause small or large behavior differences. The fact that
1288+they can be rarely encountered can actually make it worse: you forget
1289+that this might be a problem, and debugging the differences can be
1290+difficult. If you allow site-packages to be included in your buildout,
1291+and the Python you use is not managed precisely by your application (for
1292+instance, it is a system Python), you open yourself up to these
1293+possibilities. Don't be unaware of the dangers.
1294+
1295+That explained, let's see how it works. If you don't use namespace packages,
1296+this is very straightforward.
1297+
1298+ >>> reset_interpreter()
1299+ >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts(
1300+ ... interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir,
1301+ ... interpreter='py', add_site_packages=True)
1302+ >>> sys.stdout.write('#\n'); cat(site_path)
1303+ ... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE
1304+ #...
1305+ def addsitepackages(known_paths):
1306+ """Add site packages, as determined by zc.buildout.
1307+ <BLANKLINE>
1308+ See original_addsitepackages, below, for the original version."""
1309+ buildout_paths = [
1310+ '/interpreter/eggs/demo-0.3-pyN.N.egg',
1311+ '/interpreter/eggs/demoneeded-1.1-pyN.N.egg'
1312+ ]
1313+ for path in buildout_paths:
1314+ sitedir, sitedircase = makepath(path)
1315+ if not sitedircase in known_paths and os.path.exists(sitedir):
1316+ sys.path.append(sitedir)
1317+ known_paths.add(sitedircase)
1318+ original_paths = [
1319+ ...
1320+ ]
1321+ for path in original_paths:
1322+ addsitedir(path, known_paths)
1323+ return known_paths
1324+ <BLANKLINE>
1325+ def original_addsitepackages(known_paths):...
1326+
1327+It simply adds the original paths using addsitedir after the code to add the
1328+buildout paths.
1329+
1330+Here's an example of the new script in use. Other documents and tests in
1331+this package give the feature a more thorough workout, but this should
1332+give you an idea of the feature.
1333+
1334+ >>> res = call_py(interpreter_path, "import sys; print sys.path")
1335+ >>> print res # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE
1336+ ['',
1337+ '/interpreter/parts/interpreter',
1338+ ...,
1339+ '/interpreter/eggs/demo-0.3-py2.4.egg',
1340+ '/interpreter/eggs/demoneeded-1.1-py2.4.egg',
1341+ ...]
1342+ <BLANKLINE>
1343+
1344+The clean_paths gathered earlier is a subset of this full list of paths.
1345+
1346+ >>> full_paths = eval(res.strip())
1347+ >>> len(clean_paths) < len(full_paths)
1348+ True
1349+ >>> set(os.path.normpath(p) for p in clean_paths).issubset(
1350+ ... os.path.normpath(p) for p in full_paths)
1351+ True
1352+
1353+Unfortunately, because of how setuptools namespace packages are implemented
1354+differently for operating system packages (debs or rpms) as opposed to
1355+standard setuptools installation, there's a slightly trickier dance if you
1356+use them. To show this we'll needs some extra eggs that use namespaces.
1357+We'll use the ``tellmy.fortune`` package, which we'll need to make an initial
1358+call to another text fixture to create.
1359+
1360+ >>> from zc.buildout.tests import create_sample_namespace_eggs
1361+ >>> namespace_eggs = tmpdir('namespace_eggs')
1362+ >>> create_sample_namespace_eggs(namespace_eggs)
1363+
1364+ >>> reset_interpreter()
1365+ >>> ws = zc.buildout.easy_install.install(
1366+ ... ['demo', 'tellmy.fortune'], join(interpreter_dir, 'eggs'),
1367+ ... links=[link_server, namespace_eggs], index=link_server+'index/')
1368+ >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts(
1369+ ... interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir,
1370+ ... interpreter='py', add_site_packages=True)
1371+ >>> sys.stdout.write('#\n'); cat(site_path)
1372+ ... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE
1373+ #...
1374+ def addsitepackages(known_paths):
1375+ """Add site packages, as determined by zc.buildout.
1376+ <BLANKLINE>
1377+ See original_addsitepackages, below, for the original version."""
1378+ setuptools_path = '...setuptools...'
1379+ sys.path.append(setuptools_path)
1380+ known_paths.add(os.path.normcase(setuptools_path))
1381+ import pkg_resources
1382+ buildout_paths = [
1383+ '/interpreter/eggs/demo-0.3-pyN.N.egg',
1384+ '/interpreter/eggs/tellmy.fortune-1.0-pyN.N.egg',
1385+ '...setuptools...',
1386+ '/interpreter/eggs/demoneeded-1.1-pyN.N.egg'
1387+ ]
1388+ for path in buildout_paths:
1389+ sitedir, sitedircase = makepath(path)
1390+ if not sitedircase in known_paths and os.path.exists(sitedir):
1391+ sys.path.append(sitedir)
1392+ known_paths.add(sitedircase)
1393+ pkg_resources.working_set.add_entry(sitedir)
1394+ original_paths = [
1395+ ...
1396+ ]
1397+ for path in original_paths:
1398+ addsitedir(path, known_paths)
1399+ return known_paths
1400+ <BLANKLINE>
1401+ def original_addsitepackages(known_paths):...
1402+
1403+ >>> print call_py(interpreter_path, "import sys; print sys.path")
1404+ ... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE
1405+ ['',
1406+ '/interpreter/parts/interpreter',
1407+ ...,
1408+ '...setuptools...',
1409+ '/interpreter/eggs/demo-0.3-pyN.N.egg',
1410+ '/interpreter/eggs/tellmy.fortune-1.0-pyN.N.egg',
1411+ '/interpreter/eggs/demoneeded-1.1-pyN.N.egg',
1412+ ...]
1413+
1414+As you can see, the script now first imports pkg_resources. Then we
1415+need to process egg files specially to look for namespace packages there
1416+*before* we process process lines in .pth files that use the "import"
1417+feature--lines that might be part of the setuptools namespace package
1418+implementation for system packages, as mentioned above, and that must
1419+come after processing egg namespaces.
1420+
1421+The most complex that this function gets is if you use namespace packages,
1422+include site-packages, and use relative paths. For completeness, we'll look
1423+at that result.
1424+
1425+ >>> reset_interpreter()
1426+ >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts(
1427+ ... interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir,
1428+ ... interpreter='py', add_site_packages=True,
1429+ ... relative_paths=interpreter_dir)
1430+ >>> sys.stdout.write('#\n'); cat(site_path)
1431+ ... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE
1432+ #...
1433+ def addsitepackages(known_paths):
1434+ """Add site packages, as determined by zc.buildout.
1435+ <BLANKLINE>
1436+ See original_addsitepackages, below, for the original version."""
1437+ join = os.path.join
1438+ base = os.path.dirname(os.path.abspath(os.path.realpath(__file__)))
1439+ base = os.path.dirname(base)
1440+ base = os.path.dirname(base)
1441+ setuptools_path = '...setuptools...'
1442+ sys.path.append(setuptools_path)
1443+ known_paths.add(os.path.normcase(setuptools_path))
1444+ import pkg_resources
1445+ buildout_paths = [
1446+ join(base, 'eggs/demo-0.3-pyN.N.egg'),
1447+ join(base, 'eggs/tellmy.fortune-1.0-pyN.N.egg'),
1448+ '...setuptools...',
1449+ join(base, 'eggs/demoneeded-1.1-pyN.N.egg')
1450+ ]
1451+ for path in buildout_paths:
1452+ sitedir, sitedircase = makepath(path)
1453+ if not sitedircase in known_paths and os.path.exists(sitedir):
1454+ sys.path.append(sitedir)
1455+ known_paths.add(sitedircase)
1456+ pkg_resources.working_set.add_entry(sitedir)
1457+ original_paths = [
1458+ ...
1459+ ]
1460+ for path in original_paths:
1461+ addsitedir(path, known_paths)
1462+ return known_paths
1463+ <BLANKLINE>
1464+ def original_addsitepackages(known_paths):...
1465+
1466+ >>> print call_py(interpreter_path, "import sys; print sys.path")
1467+ ... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE
1468+ ['',
1469+ '/interpreter/parts/interpreter',
1470+ ...,
1471+ '...setuptools...',
1472+ '/interpreter/eggs/demo-0.3-pyN.N.egg',
1473+ '/interpreter/eggs/tellmy.fortune-1.0-pyN.N.egg',
1474+ '/interpreter/eggs/demoneeded-1.1-pyN.N.egg',
1475+ ...]
1476+
1477+The ``exec_sitecustomize`` argument does the same thing for the
1478+sitecustomize module--it allows you to include the code from the
1479+sitecustomize module in the underlying Python if you set the argument to
1480+True. The z3c.recipe.scripts package sets up the full environment necessary
1481+to demonstrate this piece.
1482+
1483+The ``sitepackage_safe_scripts`` function: writing scripts for entry points
1484+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1485+
1486+All of the examples so far for this function have been creating
1487+interpreters. The function can also write scripts for entry
1488+points. They are almost identical to the scripts that we saw for the
1489+``scripts`` function except that they ``import site`` after setting the
1490+sys.path to include our custom site.py and sitecustomize.py files. These
1491+files then initialize the Python environment as we have already seen. Let's
1492+see a simple example.
1493+
1494+ >>> reset_interpreter()
1495+ >>> ws = zc.buildout.easy_install.install(
1496+ ... ['demo'], join(interpreter_dir, 'eggs'), links=[link_server],
1497+ ... index=link_server+'index/')
1498+ >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts(
1499+ ... interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir,
1500+ ... reqs=['demo'])
1501+
1502+As before, in Windows, 2 files are generated for each script. A script
1503+file, ending in '-script.py', and an exe file that allows the script
1504+to be invoked directly without having to specify the Python
1505+interpreter and without having to provide a '.py' suffix. This is in addition
1506+to the site.py and sitecustomize.py files that are generated as with our
1507+interpreter examples above.
1508+
1509+ >>> if sys.platform == 'win32':
1510+ ... demo_path = os.path.join(interpreter_bin_dir, 'demo-script.py')
1511+ ... expected = [sitecustomize_path,
1512+ ... site_path,
1513+ ... os.path.join(interpreter_bin_dir, 'demo.exe'),
1514+ ... demo_path]
1515+ ... else:
1516+ ... demo_path = os.path.join(interpreter_bin_dir, 'demo')
1517+ ... expected = [sitecustomize_path, site_path, demo_path]
1518+ ...
1519+ >>> assert generated == expected, repr((generated, expected))
1520+
1521+The demo script runs the entry point defined in the demo egg:
1522+
1523+ >>> cat(demo_path) # doctest: +NORMALIZE_WHITESPACE
1524+ #!/usr/local/bin/python2.4 -S
1525+ <BLANKLINE>
1526+ import sys
1527+ sys.path[0:0] = [
1528+ '/interpreter/parts/interpreter',
1529+ ]
1530+ <BLANKLINE>
1531+ <BLANKLINE>
1532+ import site # imports custom buildout-generated site.py
1533+ <BLANKLINE>
1534+ import eggrecipedemo
1535+ <BLANKLINE>
1536+ if __name__ == '__main__':
1537+ eggrecipedemo.main()
1538+
1539+ >>> demo_call = join(interpreter_bin_dir, 'demo')
1540+ >>> if sys.platform == 'win32':
1541+ ... demo_call = '"%s"' % demo_call
1542+ >>> print system(demo_call)
1543+ 3 1
1544+ <BLANKLINE>
1545+
1546+There are a few differences from the ``scripts`` function. First, the
1547+``reqs`` argument (an iterable of string requirements or entry point
1548+tuples) is a keyword argument here. We see that in the example above.
1549+Second, the ``arguments`` argument is now named ``script_arguments`` to
1550+try and clarify that it does not affect interpreters. While the
1551+``initialization`` argument continues to affect both the interpreters
1552+and the entry point scripts, if you have initialization that is only
1553+pertinent to the entry point scripts, you can use the
1554+``script_initialization`` argument.
1555+
1556+Let's see ``script_arguments`` and ``script_initialization`` in action.
1557+
1558+ >>> reset_interpreter()
1559+ >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts(
1560+ ... interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir,
1561+ ... reqs=['demo'], script_arguments='1, 2',
1562+ ... script_initialization='import os\nos.chdir("foo")')
1563+
1564+ >>> cat(demo_path) # doctest: +NORMALIZE_WHITESPACE
1565+ #!/usr/local/bin/python2.4 -S
1566+ import sys
1567+ sys.path[0:0] = [
1568+ '/interpreter/parts/interpreter',
1569+ ]
1570+ <BLANKLINE>
1571+ import site # imports custom buildout-generated site.py
1572+ import os
1573+ os.chdir("foo")
1574+ <BLANKLINE>
1575+ import eggrecipedemo
1576+ <BLANKLINE>
1577+ if __name__ == '__main__':
1578+ eggrecipedemo.main(1, 2)
1579
1580 Handling custom build options for extensions provided in source distributions
1581 -----------------------------------------------------------------------------
1582
1583=== modified file 'src/zc/buildout/testing.py'
1584--- src/zc/buildout/testing.py 2010-02-23 20:41:17 +0000
1585+++ src/zc/buildout/testing.py 2010-02-23 20:41:17 +0000
1586@@ -28,6 +28,7 @@
1587 import subprocess
1588 import sys
1589 import tempfile
1590+import textwrap
1591 import threading
1592 import time
1593 import urllib2
1594@@ -105,6 +106,16 @@
1595 e.close()
1596 return result
1597
1598+def call_py(interpreter, cmd, flags=None):
1599+ if sys.platform == 'win32':
1600+ args = ['"%s"' % arg for arg in (interpreter, flags, cmd) if arg]
1601+ args.insert(-1, '"-c"')
1602+ return system('"%s"' % ' '.join(args))
1603+ else:
1604+ cmd = repr(cmd)
1605+ return system(
1606+ ' '.join(arg for arg in (interpreter, flags, '-c', cmd) if arg))
1607+
1608 def get(url):
1609 return urllib2.urlopen(url).read()
1610
1611@@ -116,7 +127,11 @@
1612 args = [zc.buildout.easy_install._safe_arg(arg)
1613 for arg in args]
1614 args.insert(0, '-q')
1615- args.append(dict(os.environ, PYTHONPATH=setuptools_location))
1616+ env = dict(os.environ)
1617+ if executable == sys.executable:
1618+ env['PYTHONPATH'] = setuptools_location
1619+ # else pass an executable that has setuptools! See testselectingpython.py.
1620+ args.append(env)
1621
1622 here = os.getcwd()
1623 try:
1624@@ -135,6 +150,11 @@
1625 def bdist_egg(setup, executable, dest):
1626 _runsetup(setup, executable, 'bdist_egg', '-d', dest)
1627
1628+def sys_install(setup, dest):
1629+ _runsetup(setup, sys.executable, 'install', '--install-purelib', dest,
1630+ '--record', os.path.join(dest, '__added_files__'),
1631+ '--single-version-externally-managed')
1632+
1633 def find_python(version):
1634 e = os.environ.get('PYTHON%s' % version)
1635 if e is not None:
1636@@ -202,6 +222,24 @@
1637 time.sleep(0.01)
1638 raise ValueError('Timed out waiting for: '+label)
1639
1640+def make_buildout():
1641+ # Create a basic buildout.cfg to avoid a warning from buildout:
1642+ open('buildout.cfg', 'w').write(
1643+ "[buildout]\nparts =\n"
1644+ )
1645+ # Use the buildout bootstrap command to create a buildout
1646+ zc.buildout.buildout.Buildout(
1647+ 'buildout.cfg',
1648+ [('buildout', 'log-level', 'WARNING'),
1649+ # trick bootstrap into putting the buildout develop egg
1650+ # in the eggs dir.
1651+ ('buildout', 'develop-eggs-directory', 'eggs'),
1652+ ]
1653+ ).bootstrap([])
1654+ # Create the develop-eggs dir, which didn't get created the usual
1655+ # way due to the trick above:
1656+ os.mkdir('develop-eggs')
1657+
1658 def buildoutSetUp(test):
1659
1660 test.globs['__tear_downs'] = __tear_downs = []
1661@@ -255,27 +293,7 @@
1662 sample = tmpdir('sample-buildout')
1663
1664 os.chdir(sample)
1665-
1666- # Create a basic buildout.cfg to avoid a warning from buildout:
1667- open('buildout.cfg', 'w').write(
1668- "[buildout]\nparts =\n"
1669- )
1670-
1671- # Use the buildout bootstrap command to create a buildout
1672- zc.buildout.buildout.Buildout(
1673- 'buildout.cfg',
1674- [('buildout', 'log-level', 'WARNING'),
1675- # trick bootstrap into putting the buildout develop egg
1676- # in the eggs dir.
1677- ('buildout', 'develop-eggs-directory', 'eggs'),
1678- ]
1679- ).bootstrap([])
1680-
1681-
1682-
1683- # Create the develop-eggs dir, which didn't get created the usual
1684- # way due to the trick above:
1685- os.mkdir('develop-eggs')
1686+ make_buildout()
1687
1688 def start_server(path):
1689 port, thread = _start_server(path, name=path)
1690@@ -283,6 +301,50 @@
1691 register_teardown(lambda: stop_server(url, thread))
1692 return url
1693
1694+ def make_py(initialization=''):
1695+ """Returns paths to new executable and to its site-packages.
1696+ """
1697+ buildout = tmpdir('executable_buildout')
1698+ site_packages_dir = os.path.join(buildout, 'site-packages')
1699+ mkdir(site_packages_dir)
1700+ old_wd = os.getcwd()
1701+ os.chdir(buildout)
1702+ make_buildout()
1703+ # Normally we don't process .pth files in extra-paths. We want to
1704+ # in this case so that we can test with setuptools system installs
1705+ # (--single-version-externally-managed), which use .pth files.
1706+ initialization = (
1707+ ('import sys\n'
1708+ 'import site\n'
1709+ 'known_paths = set(sys.path)\n'
1710+ 'site_packages_dir = %r\n'
1711+ 'site.addsitedir(site_packages_dir, known_paths)\n'
1712+ ) % (site_packages_dir,)) + initialization
1713+ initialization = '\n'.join(
1714+ ' ' + line for line in initialization.split('\n'))
1715+ install_develop(
1716+ 'zc.recipe.egg', os.path.join(buildout, 'develop-eggs'))
1717+ install_develop(
1718+ 'z3c.recipe.scripts', os.path.join(buildout, 'develop-eggs'))
1719+ write('buildout.cfg', textwrap.dedent('''\
1720+ [buildout]
1721+ parts = py
1722+
1723+ [py]
1724+ recipe = z3c.recipe.scripts
1725+ interpreter = py
1726+ initialization =
1727+ %(initialization)s
1728+ extra-paths = %(site-packages)s
1729+ eggs = setuptools
1730+ ''') % {
1731+ 'initialization': initialization,
1732+ 'site-packages': site_packages_dir})
1733+ system(os.path.join(buildout, 'bin', 'buildout'))
1734+ os.chdir(old_wd)
1735+ return (
1736+ os.path.join(buildout, 'bin', 'py'), site_packages_dir)
1737+
1738 test.globs.update(dict(
1739 sample_buildout = sample,
1740 ls = ls,
1741@@ -293,6 +355,7 @@
1742 tmpdir = tmpdir,
1743 write = write,
1744 system = system,
1745+ call_py = call_py,
1746 get = get,
1747 cd = (lambda *path: os.chdir(os.path.join(*path))),
1748 join = os.path.join,
1749@@ -301,6 +364,7 @@
1750 start_server = start_server,
1751 buildout = os.path.join(sample, 'bin', 'buildout'),
1752 wait_until = wait_until,
1753+ make_py = make_py
1754 ))
1755
1756 zc.buildout.easy_install.prefer_final(prefer_final)
1757
1758=== modified file 'src/zc/buildout/tests.py'
1759--- src/zc/buildout/tests.py 2010-02-23 20:41:17 +0000
1760+++ src/zc/buildout/tests.py 2010-02-23 20:41:17 +0000
1761@@ -53,6 +53,7 @@
1762
1763 >>> ls('develop-eggs')
1764 - foo.egg-link
1765+ - z3c.recipe.scripts.egg-link
1766 - zc.recipe.egg.egg-link
1767
1768 """
1769@@ -84,6 +85,7 @@
1770
1771 >>> ls('develop-eggs')
1772 - foo.egg-link
1773+ - z3c.recipe.scripts.egg-link
1774 - zc.recipe.egg.egg-link
1775
1776 >>> print system(join('bin', 'buildout')+' -vvv'), # doctest: +ELLIPSIS
1777@@ -668,6 +670,7 @@
1778
1779 >>> ls('develop-eggs')
1780 - foox.egg-link
1781+ - z3c.recipe.scripts.egg-link
1782 - zc.recipe.egg.egg-link
1783
1784 Create another:
1785@@ -692,6 +695,7 @@
1786 >>> ls('develop-eggs')
1787 - foox.egg-link
1788 - fooy.egg-link
1789+ - z3c.recipe.scripts.egg-link
1790 - zc.recipe.egg.egg-link
1791
1792 Remove one:
1793@@ -709,6 +713,7 @@
1794
1795 >>> ls('develop-eggs')
1796 - fooy.egg-link
1797+ - z3c.recipe.scripts.egg-link
1798 - zc.recipe.egg.egg-link
1799
1800 Remove the other:
1801@@ -723,6 +728,7 @@
1802 All gone
1803
1804 >>> ls('develop-eggs')
1805+ - z3c.recipe.scripts.egg-link
1806 - zc.recipe.egg.egg-link
1807 '''
1808
1809@@ -797,6 +803,7 @@
1810 ... + join(sample_buildout, 'eggs'))
1811
1812 >>> ls('develop-eggs')
1813+ - z3c.recipe.scripts.egg-link
1814 - zc.recipe.egg.egg-link
1815
1816 >>> ls('eggs') # doctest: +ELLIPSIS
1817@@ -1769,6 +1776,235 @@
1818 1 2
1819 """
1820
1821+def versions_section_ignored_for_dependency_in_favor_of_site_packages():
1822+ r"""
1823+This is a test for a bugfix.
1824+
1825+The error showed itself when at least two dependencies were in a shared
1826+location like site-packages, and the first one met the "versions" setting. The
1827+first dependency would be added, but subsequent dependencies from the same
1828+location (e.g., site-packages) would use the version of the package found in
1829+the shared location, ignoring the version setting.
1830+
1831+We begin with a Python that has demoneeded version 1.1 installed and a
1832+demo version 0.3, all in a site-packages-like shared directory. We need
1833+to create this. ``eggrecipedemo.main()`` shows the number after the dot
1834+(that is, ``X`` in ``1.X``), for the demo package and the demoneeded
1835+package, so this demonstrates that our Python does in fact have demo
1836+version 0.3 and demoneeded version 1.1.
1837+
1838+ >>> py_path = make_py_with_system_install(make_py, sample_eggs)
1839+ >>> print call_py(
1840+ ... py_path,
1841+ ... "import tellmy.version; print tellmy.version.__version__"),
1842+ 1.1
1843+
1844+Now here's a setup that would expose the bug, using the
1845+zc.buildout.easy_install API.
1846+
1847+ >>> example_dest = tmpdir('example_dest')
1848+ >>> workingset = zc.buildout.easy_install.install(
1849+ ... ['tellmy.version'], example_dest, links=[sample_eggs],
1850+ ... executable=py_path,
1851+ ... index=None,
1852+ ... versions={'tellmy.version': '1.0'})
1853+ >>> for dist in workingset:
1854+ ... res = str(dist)
1855+ ... if res.startswith('tellmy.version'):
1856+ ... print res
1857+ ... break
1858+ tellmy.version 1.0
1859+
1860+Before the bugfix, the desired tellmy.version distribution would have
1861+been blocked the one in site-packages.
1862+"""
1863+
1864+def handle_namespace_package_in_both_site_packages_and_buildout_eggs():
1865+ r"""
1866+If you have the same namespace package in both site-packages and in
1867+buildout, we need to be very careful that faux-Python-executables and
1868+scripts generated by easy_install.sitepackage_safe_scripts correctly
1869+combine the two. We show this with the local recipe that uses the
1870+function, z3c.recipe.scripts.
1871+
1872+To demonstrate this, we will create three packages: tellmy.version 1.0,
1873+tellmy.version 1.1, and tellmy.fortune 1.0. tellmy.version 1.1 is installed.
1874+
1875+ >>> py_path = make_py_with_system_install(make_py, sample_eggs)
1876+ >>> print call_py(
1877+ ... py_path,
1878+ ... "import tellmy.version; print tellmy.version.__version__")
1879+ 1.1
1880+ <BLANKLINE>
1881+
1882+Now we will create a buildout that creates a script and a faux-Python script.
1883+We want to see that both can successfully import the specified versions of
1884+tellmy.version and tellmy.fortune.
1885+
1886+ >>> write('buildout.cfg',
1887+ ... '''
1888+ ... [buildout]
1889+ ... parts = eggs
1890+ ... find-links = %(link_server)s
1891+ ...
1892+ ... [primed_python]
1893+ ... executable = %(py_path)s
1894+ ...
1895+ ... [eggs]
1896+ ... recipe = z3c.recipe.scripts
1897+ ... python = primed_python
1898+ ... interpreter = py
1899+ ... add-site-packages = true
1900+ ... eggs = tellmy.version == 1.0
1901+ ... tellmy.fortune == 1.0
1902+ ... demo
1903+ ... script-initialization =
1904+ ... import tellmy.version
1905+ ... print tellmy.version.__version__
1906+ ... import tellmy.fortune
1907+ ... print tellmy.fortune.__version__
1908+ ... ''' % globals())
1909+
1910+ >>> print system(buildout)
1911+ Installing eggs.
1912+ Getting distribution for 'tellmy.version==1.0'.
1913+ Got tellmy.version 1.0.
1914+ Getting distribution for 'tellmy.fortune==1.0'.
1915+ Got tellmy.fortune 1.0.
1916+ Getting distribution for 'demo'.
1917+ Got demo 0.4c1.
1918+ Getting distribution for 'demoneeded'.
1919+ Got demoneeded 1.2c1.
1920+ Generated script '/sample-buildout/bin/demo'.
1921+ Generated interpreter '/sample-buildout/bin/py'.
1922+ <BLANKLINE>
1923+
1924+Finally, we are ready for the actual test. Prior to the bug fix that
1925+this tests, the results of both calls below was the following::
1926+
1927+ 1.1
1928+ Traceback (most recent call last):
1929+ ...
1930+ ImportError: No module named fortune
1931+ <BLANKLINE>
1932+
1933+In other words, we got the site-packages version of tellmy.version, and
1934+we could not import tellmy.fortune at all. The following are the correct
1935+results for the interpreter and for the script.
1936+
1937+ >>> print call_py(
1938+ ... join('bin', 'py'),
1939+ ... "import tellmy.version; " +
1940+ ... "print tellmy.version.__version__; " +
1941+ ... "import tellmy.fortune; " +
1942+ ... "print tellmy.fortune.__version__") # doctest: +ELLIPSIS
1943+ 1.0
1944+ 1.0...
1945+
1946+ >>> print system(join('bin', 'demo'))
1947+ 1.0
1948+ 1.0
1949+ 4 2
1950+ <BLANKLINE>
1951+ """
1952+
1953+def handle_sys_path_version_hack():
1954+ r"""
1955+This is a test for a bugfix.
1956+
1957+If you use a Python that has a different version of one of your
1958+dependencies, and the new package tries to do sys.path tricks in the
1959+setup.py to get a __version__, and it uses namespace packages, the older
1960+package will be loaded first, making the setup version the wrong number.
1961+While very arguably packages simply shouldn't do this, some do, and we
1962+don't want buildout to fall over when they do.
1963+
1964+To demonstrate this, we will need to create a distribution that has one of
1965+these unpleasant tricks, and a Python that has an older version installed.
1966+
1967+ >>> py_path, site_packages_path = make_py()
1968+ >>> for version in ('1.0', '1.1'):
1969+ ... tmp = tempfile.mkdtemp()
1970+ ... try:
1971+ ... write(tmp, 'README.txt', '')
1972+ ... mkdir(tmp, 'src')
1973+ ... mkdir(tmp, 'src', 'tellmy')
1974+ ... write(tmp, 'src', 'tellmy', '__init__.py',
1975+ ... "__import__("
1976+ ... "'pkg_resources').declare_namespace(__name__)\n")
1977+ ... mkdir(tmp, 'src', 'tellmy', 'version')
1978+ ... write(tmp, 'src', 'tellmy', 'version',
1979+ ... '__init__.py', '__version__=%r\n' % version)
1980+ ... write(
1981+ ... tmp, 'setup.py',
1982+ ... "from setuptools import setup\n"
1983+ ... "import sys\n"
1984+ ... "sys.path.insert(0, 'src')\n"
1985+ ... "from tellmy.version import __version__\n"
1986+ ... "setup(\n"
1987+ ... " name='tellmy.version',\n"
1988+ ... " package_dir = {'': 'src'},\n"
1989+ ... " packages = ['tellmy', 'tellmy.version'],\n"
1990+ ... " install_requires = ['setuptools'],\n"
1991+ ... " namespace_packages=['tellmy'],\n"
1992+ ... " zip_safe=True, version=__version__,\n"
1993+ ... " author='bob', url='bob', author_email='bob')\n"
1994+ ... )
1995+ ... zc.buildout.testing.sdist(tmp, sample_eggs)
1996+ ... if version == '1.0':
1997+ ... # We install the 1.0 version in site packages the way a
1998+ ... # system packaging system (debs, rpms) would do it.
1999+ ... zc.buildout.testing.sys_install(tmp, site_packages_path)
2000+ ... finally:
2001+ ... shutil.rmtree(tmp)
2002+ >>> print call_py(
2003+ ... py_path,
2004+ ... "import tellmy.version; print tellmy.version.__version__")
2005+ 1.0
2006+ <BLANKLINE>
2007+ >>> write('buildout.cfg',
2008+ ... '''
2009+ ... [buildout]
2010+ ... parts = eggs
2011+ ... find-links = %(sample_eggs)s
2012+ ...
2013+ ... [primed_python]
2014+ ... executable = %(py_path)s
2015+ ...
2016+ ... [eggs]
2017+ ... recipe = zc.recipe.egg:eggs
2018+ ... python = primed_python
2019+ ... eggs = tellmy.version == 1.1
2020+ ... ''' % globals())
2021+
2022+Before the bugfix, running this buildout would generate this error:
2023+
2024+ Installing eggs.
2025+ Getting distribution for 'tellmy.version==1.1'.
2026+ Installing tellmy.version 1.1
2027+ Caused installation of a distribution:
2028+ tellmy.version 1.0
2029+ with a different version.
2030+ Got None.
2031+ While:
2032+ Installing eggs.
2033+ Error: There is a version conflict.
2034+ We already have: tellmy.version 1.0
2035+ <BLANKLINE>
2036+
2037+You can see the copiously commented fix for this in easy_install.py (see
2038+zc.buildout.easy_install.Installer._call_easy_install and particularly
2039+the comment leading up to zc.buildout.easy_install._easy_install_cmd).
2040+Now the install works correctly, as seen here.
2041+
2042+ >>> print system(buildout)
2043+ Installing eggs.
2044+ Getting distribution for 'tellmy.version==1.1'.
2045+ Got tellmy.version 1.1.
2046+ <BLANKLINE>
2047+
2048+ """
2049+
2050 if sys.version_info > (2, 4):
2051 def test_exit_codes():
2052 """
2053@@ -2367,6 +2603,7 @@
2054
2055 >>> ls('develop-eggs')
2056 - foo.egg-link
2057+ - z3c.recipe.scripts.egg-link
2058 - zc.recipe.egg.egg-link
2059
2060 """
2061@@ -2654,6 +2891,47 @@
2062
2063 ######################################################################
2064
2065+def make_py_with_system_install(make_py, sample_eggs):
2066+ py_path, site_packages_path = make_py()
2067+ create_sample_namespace_eggs(sample_eggs, site_packages_path)
2068+ return py_path
2069+
2070+def create_sample_namespace_eggs(dest, site_packages_path=None):
2071+ from zc.buildout.testing import write, mkdir
2072+ for pkg, version in (('version', '1.0'), ('version', '1.1'),
2073+ ('fortune', '1.0')):
2074+ tmp = tempfile.mkdtemp()
2075+ try:
2076+ write(tmp, 'README.txt', '')
2077+ mkdir(tmp, 'src')
2078+ mkdir(tmp, 'src', 'tellmy')
2079+ write(tmp, 'src', 'tellmy', '__init__.py',
2080+ "__import__("
2081+ "'pkg_resources').declare_namespace(__name__)\n")
2082+ mkdir(tmp, 'src', 'tellmy', pkg)
2083+ write(tmp, 'src', 'tellmy', pkg,
2084+ '__init__.py', '__version__=%r\n' % version)
2085+ write(
2086+ tmp, 'setup.py',
2087+ "from setuptools import setup\n"
2088+ "setup(\n"
2089+ " name='tellmy.%(pkg)s',\n"
2090+ " package_dir = {'': 'src'},\n"
2091+ " packages = ['tellmy', 'tellmy.%(pkg)s'],\n"
2092+ " install_requires = ['setuptools'],\n"
2093+ " namespace_packages=['tellmy'],\n"
2094+ " zip_safe=True, version=%(version)r,\n"
2095+ " author='bob', url='bob', author_email='bob')\n"
2096+ % locals()
2097+ )
2098+ zc.buildout.testing.sdist(tmp, dest)
2099+ if (site_packages_path and pkg == 'version' and version == '1.1'):
2100+ # We install the 1.1 version in site packages the way a
2101+ # system packaging system (debs, rpms) would do it.
2102+ zc.buildout.testing.sys_install(tmp, site_packages_path)
2103+ finally:
2104+ shutil.rmtree(tmp)
2105+
2106 def create_sample_eggs(test, executable=sys.executable):
2107 write = test.globs['write']
2108 dest = test.globs['sample_eggs']
2109@@ -2776,6 +3054,7 @@
2110 test.globs['sample_eggs'])
2111 test.globs['update_extdemo'] = lambda : add_source_dist(test, 1.5)
2112 zc.buildout.testing.install_develop('zc.recipe.egg', test)
2113+ zc.buildout.testing.install_develop('z3c.recipe.scripts', test)
2114
2115 egg_parse = re.compile('([0-9a-zA-Z_.]+)-([0-9a-zA-Z_.]+)-py(\d[.]\d).egg$'
2116 ).match
2117@@ -2934,6 +3213,10 @@
2118 (re.compile('[-d] setuptools-\S+[.]egg'), 'setuptools.egg'),
2119 (re.compile(r'\\[\\]?'), '/'),
2120 (re.compile(r'\#!\S+\bpython\S*'), '#!/usr/bin/python'),
2121+ # Normalize generate_script's Windows interpreter to UNIX:
2122+ (re.compile(r'\nimport subprocess\n'), '\n'),
2123+ (re.compile('subprocess\\.call\\(argv, env=environ\\)'),
2124+ 'os.execve(sys.executable, argv, environ)'),
2125 ]+(sys.version_info < (2, 5) and [
2126 (re.compile('.*No module named runpy.*', re.S), ''),
2127 (re.compile('.*usage: pdb.py scriptfile .*', re.S), ''),
2128@@ -3043,6 +3326,8 @@
2129 zc.buildout.testing.normalize_script,
2130 normalize_bang,
2131 (re.compile('Downloading.*setuptools.*egg\n'), ''),
2132+ (re.compile('options:'), 'Options:'),
2133+ (re.compile('usage:'), 'Usage:'),
2134 ]),
2135 ))
2136
2137
2138=== modified file 'src/zc/buildout/testselectingpython.py'
2139--- src/zc/buildout/testselectingpython.py 2009-11-27 13:20:19 +0000
2140+++ src/zc/buildout/testselectingpython.py 2010-02-23 20:41:17 +0000
2141@@ -11,7 +11,7 @@
2142 # FOR A PARTICULAR PURPOSE.
2143 #
2144 ##############################################################################
2145-import os, re, sys, unittest
2146+import os, re, subprocess, sys, textwrap, unittest
2147 from zope.testing import doctest, renormalizing
2148 import zc.buildout.tests
2149 import zc.buildout.testing
2150@@ -42,6 +42,33 @@
2151
2152 def multi_python(test):
2153 other_executable = zc.buildout.testing.find_python(other_version)
2154+ command = textwrap.dedent('''\
2155+ try:
2156+ import setuptools
2157+ except ImportError:
2158+ import sys
2159+ sys.exit(1)
2160+ ''')
2161+ if subprocess.call([other_executable, '-c', command],
2162+ env=os.environ):
2163+ # the other executable does not have setuptools. Get setuptools.
2164+ # We will do this using the same tools we are testing, for better or
2165+ # worse. Alternatively, we could try using bootstrap.
2166+ executable_dir = test.globs['tmpdir']('executable_dir')
2167+ executable_parts = os.path.join(executable_dir, 'parts')
2168+ test.globs['mkdir'](executable_parts)
2169+ ws = zc.buildout.easy_install.install(
2170+ ['setuptools'], executable_dir,
2171+ index='http://www.python.org/pypi/',
2172+ always_unzip=True, executable=other_executable)
2173+ zc.buildout.easy_install.sitepackage_safe_scripts(
2174+ executable_dir, ws, other_executable, executable_parts,
2175+ reqs=['setuptools'], interpreter='py')
2176+ original_executable = other_executable
2177+ other_executable = os.path.join(executable_dir, 'py')
2178+ assert not subprocess.call(
2179+ [other_executable, '-c', command], env=os.environ), (
2180+ 'test set up failed')
2181 sample_eggs = test.globs['tmpdir']('sample_eggs')
2182 os.mkdir(os.path.join(sample_eggs, 'index'))
2183 test.globs['sample_eggs'] = sample_eggs
2184
2185=== modified file 'src/zc/buildout/update.txt'
2186--- src/zc/buildout/update.txt 2009-11-06 22:33:23 +0000
2187+++ src/zc/buildout/update.txt 2010-02-23 20:41:17 +0000
2188@@ -81,6 +81,7 @@
2189 Our buildout script has been updated to use the new eggs:
2190
2191 >>> cat(sample_buildout, 'bin', 'buildout')
2192+ ... # doctest: +NORMALIZE_WHITESPACE
2193 #!/usr/local/bin/python2.4
2194 <BLANKLINE>
2195 import sys
2196
2197=== added directory 'z3c.recipe.scripts_'
2198=== added file 'z3c.recipe.scripts_/CHANGES.txt'
2199--- z3c.recipe.scripts_/CHANGES.txt 1970-01-01 00:00:00 +0000
2200+++ z3c.recipe.scripts_/CHANGES.txt 2010-02-23 20:41:17 +0000
2201@@ -0,0 +1,7 @@
2202+Change History
2203+**************
2204+
2205+1.0.0
2206+=====
2207+
2208+Initial public version.
2209
2210=== added file 'z3c.recipe.scripts_/README.txt'
2211--- z3c.recipe.scripts_/README.txt 1970-01-01 00:00:00 +0000
2212+++ z3c.recipe.scripts_/README.txt 2010-02-23 20:41:17 +0000
2213@@ -0,0 +1,10 @@
2214+********************************
2215+Buildout Script Recipe
2216+********************************
2217+
2218+.. contents::
2219+
2220+The script recipe installs eggs into a buildout eggs directory, exactly
2221+like zc.recipe.egg, and then generates scripts in a buildout bin
2222+directory with egg paths baked into them.
2223+
2224
2225=== added file 'z3c.recipe.scripts_/setup.py'
2226--- z3c.recipe.scripts_/setup.py 1970-01-01 00:00:00 +0000
2227+++ z3c.recipe.scripts_/setup.py 2010-02-23 20:41:17 +0000
2228@@ -0,0 +1,76 @@
2229+##############################################################################
2230+#
2231+# Copyright (c) 2007 Zope Corporation and Contributors.
2232+# All Rights Reserved.
2233+#
2234+# This software is subject to the provisions of the Zope Public License,
2235+# Version 2.1 (ZPL). A copy of the ZPL should accompany this distribution.
2236+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
2237+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
2238+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
2239+# FOR A PARTICULAR PURPOSE.
2240+#
2241+##############################################################################
2242+"""Setup for z3c.recipe.scripts package
2243+
2244+$Id: setup.py 106736 2009-12-18 02:33:08Z gary $
2245+"""
2246+
2247+version = '1.0.0dev'
2248+
2249+import os
2250+from setuptools import setup, find_packages
2251+
2252+def read(*rnames):
2253+ return open(os.path.join(os.path.dirname(__file__), *rnames)).read()
2254+
2255+name = "z3c.recipe.scripts"
2256+setup(
2257+ name = name,
2258+ version = version,
2259+ author = "Gary Poster",
2260+ author_email = "gary.poster@canonical.com",
2261+ description = "Recipe for installing Python scripts",
2262+ long_description = (
2263+ read('README.txt')
2264+ + '\n' +
2265+ read('CHANGES.txt')
2266+ + '\n' +
2267+ 'Detailed Documentation\n'
2268+ '**********************\n'
2269+ + '\n' +
2270+ read('src', 'z3c', 'recipe', 'scripts', 'README.txt')
2271+ + '\n' +
2272+ 'Download\n'
2273+ '*********\n'
2274+ ),
2275+ keywords = "development build",
2276+ classifiers = [
2277+ 'Development Status :: 5 - Production/Stable',
2278+ 'Framework :: Buildout',
2279+ 'Intended Audience :: Developers',
2280+ 'License :: OSI Approved :: Zope Public License',
2281+ 'Topic :: Software Development :: Build Tools',
2282+ 'Topic :: Software Development :: Libraries :: Python Modules',
2283+ ],
2284+ url='http://cheeseshop.python.org/pypi/z3c.recipe.scripts',
2285+ license = "ZPL 2.1",
2286+
2287+ packages = find_packages('src'),
2288+ package_dir = {'':'src'},
2289+ namespace_packages = ['z3c', 'z3c.recipe'],
2290+ install_requires = [
2291+ 'zc.buildout >=1.5.0dev',
2292+ 'zc.recipe.egg >=1.2.3dev',
2293+ 'setuptools'],
2294+ tests_require = ['zope.testing'],
2295+ test_suite = name+'.tests.test_suite',
2296+ entry_points = {'zc.buildout': ['default = %s:Scripts' % name,
2297+ 'script = %s:Scripts' % name,
2298+ 'scripts = %s:Scripts' % name,
2299+ 'interpreter = %s:Interpreter' % name,
2300+ ]
2301+ },
2302+ include_package_data = True,
2303+ zip_safe=False,
2304+ )
2305
2306=== added directory 'z3c.recipe.scripts_/src'
2307=== added directory 'z3c.recipe.scripts_/src/z3c'
2308=== added file 'z3c.recipe.scripts_/src/z3c/__init__.py'
2309--- z3c.recipe.scripts_/src/z3c/__init__.py 1970-01-01 00:00:00 +0000
2310+++ z3c.recipe.scripts_/src/z3c/__init__.py 2010-02-23 20:41:17 +0000
2311@@ -0,0 +1,1 @@
2312+__import__('pkg_resources').declare_namespace(__name__)
2313
2314=== added directory 'z3c.recipe.scripts_/src/z3c/recipe'
2315=== added file 'z3c.recipe.scripts_/src/z3c/recipe/__init__.py'
2316--- z3c.recipe.scripts_/src/z3c/recipe/__init__.py 1970-01-01 00:00:00 +0000
2317+++ z3c.recipe.scripts_/src/z3c/recipe/__init__.py 2010-02-23 20:41:17 +0000
2318@@ -0,0 +1,1 @@
2319+__import__('pkg_resources').declare_namespace(__name__)
2320
2321=== added directory 'z3c.recipe.scripts_/src/z3c/recipe/scripts'
2322=== added file 'z3c.recipe.scripts_/src/z3c/recipe/scripts/README.txt'
2323--- z3c.recipe.scripts_/src/z3c/recipe/scripts/README.txt 1970-01-01 00:00:00 +0000
2324+++ z3c.recipe.scripts_/src/z3c/recipe/scripts/README.txt 2010-02-23 20:41:17 +0000
2325@@ -0,0 +1,402 @@
2326+Script and interpreter generation
2327+=================================
2328+
2329+This recipe is very similar to zc.recipe.egg, and if you are familiar with its
2330+options, you will be able to use this one easily.
2331+
2332+The script and interpreter generation in this recipe are improved from
2333+those provided by zc.recipe.egg in two basic ways.
2334+
2335+- The interpreter generated by the script supports all interpreter
2336+ options, as opposed to the subset provided by zc.recipe.egg.
2337+
2338+- Both scripts and interpreters from this recipe can optionally choose
2339+ to include site-packages, and even sitecustomize.
2340+
2341+The recipe takes several options. First, here's the list of the options
2342+that overlap from the standard zc.recipe.eggs scripts recipe. After
2343+this, we'll list the new options and describe them.
2344+
2345+* eggs
2346+* find-links
2347+* index
2348+* python
2349+* extra-paths
2350+* entry-points
2351+* scripts
2352+* dependent-scripts
2353+* interpreter
2354+* arguments
2355+* initialization
2356+* relative-paths
2357+
2358+In addition to these, the recipe offers these new options. They are
2359+introduced here, and described more in depth below.
2360+
2361+add-site-packages
2362+ You can choose to have the site-packages of the underlying Python
2363+ available to your script or interpreter, in addition to the packages
2364+ from your eggs. See the section on this option for motivations and
2365+ warnings.
2366+
2367+extends
2368+ You can extend another section using this value. It is intended to be
2369+ used by extending a section that uses this package's scripts recipe.
2370+ In this manner, you can avoid repeating yourself.
2371+
2372+exec-sitecustomize
2373+ Normally the Python's real sitecustomize module is not processed.
2374+ If you want it to be processed, set this value to 'true'. This will
2375+ be honored irrespective of the setting for add-site-packages.
2376+
2377+script-initialization
2378+ The standard initialization code affects both an interpreter and scripts.
2379+ The code in script-initialization is used only for the generated scripts.
2380+
2381+Finally, the "interpreter" entry point ignores ``script-initialization``,
2382+``scripts``, and ``arguments``, and provides yet another additional option.
2383+
2384+name
2385+ While, by default, the interpreter recipe takes the name of the
2386+ section to be the desired interpreter name, you can specify the
2387+ interpreter name here instead.
2388+
2389+Script generation
2390+-----------------
2391+
2392+Generating a basic script looks virtually identical to using zc.recipe.egg.
2393+
2394+(Note that the find-links and index values are typically not needed; they
2395+are included to help make this document run as a test successfully.)
2396+
2397+ >>> write(sample_buildout, 'buildout.cfg',
2398+ ... """
2399+ ... [buildout]
2400+ ... parts = demo
2401+ ...
2402+ ... [demo]
2403+ ... recipe = z3c.recipe.scripts
2404+ ... eggs = demo<0.3
2405+ ... find-links = %(server)s
2406+ ... index = %(server)s/index
2407+ ... """ % dict(server=link_server))
2408+
2409+ >>> print system(buildout),
2410+ Installing demo.
2411+ Getting distribution for 'demo<0.3'.
2412+ Got demo 0.2.
2413+ Getting distribution for 'demoneeded'.
2414+ Got demoneeded 1.2c1.
2415+ Generated script '/sample-buildout/bin/demo'.
2416+
2417+ >>> print system(join(sample_buildout, 'bin', 'demo')),
2418+ 2 2
2419+
2420+Interpreter generation
2421+----------------------
2422+
2423+As with zc.recipe.egg, you can generate an interpreter with the default
2424+script recipe shown above by supplying the "interpreter" option.
2425+This example will create both an entry point script and an interpreter.
2426+
2427+ >>> write(sample_buildout, 'buildout.cfg',
2428+ ... """
2429+ ... [buildout]
2430+ ... parts = demo
2431+ ...
2432+ ... [demo]
2433+ ... recipe = z3c.recipe.scripts
2434+ ... eggs = demo<0.3
2435+ ... find-links = %(server)s
2436+ ... index = %(server)s/index
2437+ ... interpreter = py
2438+ ... """ % dict(server=link_server))
2439+
2440+ >>> print system(buildout),
2441+ Uninstalling demo.
2442+ Installing demo.
2443+ Generated script '/sample-buildout/bin/demo'.
2444+ Generated interpreter '/sample-buildout/bin/py'.
2445+
2446+You can also generate an interpreter alone with the ``interpreter`` recipe.
2447+
2448+ >>> write(sample_buildout, 'buildout.cfg',
2449+ ... """
2450+ ... [buildout]
2451+ ... parts = py
2452+ ...
2453+ ... [py]
2454+ ... recipe = z3c.recipe.scripts:interpreter
2455+ ... eggs = demo<0.3
2456+ ... find-links = %(server)s
2457+ ... index = %(server)s/index
2458+ ... """ % dict(server=link_server))
2459+
2460+ >>> print system(buildout),
2461+ Uninstalling demo.
2462+ Installing py.
2463+ Generated interpreter '/sample-buildout/bin/py'.
2464+
2465+In both cases, the bin/py script works by restarting Python after
2466+specifying a special path in PYTHONPATH. This example shows the UNIX version;
2467+the Windows version actually uses subprocess instead.
2468+
2469+ >>> cat(sample_buildout, 'bin', 'py') # doctest: +NORMALIZE_WHITESPACE
2470+ #!/usr/bin/python2.4 -S
2471+ <BLANKLINE>
2472+ import os
2473+ import sys
2474+ <BLANKLINE>
2475+ argv = [sys.executable] + sys.argv[1:]
2476+ environ = os.environ.copy()
2477+ path = '/sample-buildout/parts/py'
2478+ if environ.get('PYTHONPATH'):
2479+ path = os.pathsep.join([path, environ['PYTHONPATH']])
2480+ environ['PYTHONPATH'] = path
2481+ os.execve(sys.executable, argv, environ)
2482+
2483+The path is a directory that contains two files: our own site.py and
2484+sitecustomize.py. The site.py is modified from the underlying Python's
2485+site.py, and is responsible for setting up our paths. The
2486+sitecustomize.py is responsible for running the initialization code
2487+provided.
2488+
2489+ >>> ls(sample_buildout, 'parts', 'py')
2490+ - site.py
2491+ - sitecustomize.py
2492+
2493+Here's an example of using the generated interpreter.
2494+
2495+ >>> print system(join(sample_buildout, 'bin', 'py') +
2496+ ... ' -c "import sys, pprint; pprint.pprint(sys.path[-2:])"')
2497+ ['/sample-buildout/eggs/demo-0.2-pyN.N.egg',
2498+ '/sample-buildout/eggs/demoneeded-1.2c1-pyN.N.egg']
2499+ <BLANKLINE>
2500+
2501+Including site-packages and sitecustomize
2502+-----------------------------------------
2503+
2504+As introduced above, this recipe supports including site packages. This has
2505+some advantages and some serious dangers.
2506+
2507+A typical reason to include site-packages is that it is easier to
2508+install one or more dependencies in your Python than it is with
2509+buildout. Some packages, such as lxml or Python PostgreSQL integration,
2510+have dependencies that can be much easier to build and/or install using
2511+other mechanisms, such as your operating system's package manager. By
2512+installing some core packages into your Python's site-packages, this can
2513+significantly simplify some application installations.
2514+
2515+However, doing this has a significant danger. One of the primary goals
2516+of buildout is to provide repeatability. Some packages (one of the
2517+better known Python openid packages, for instance) change their behavior
2518+depending on what packages are available. If Python curl bindings are
2519+available, these may be preferred by the library. If a certain XML
2520+package is installed, it may be preferred by the library. These hidden
2521+choices may cause small or large behavior differences. The fact that
2522+they can be rarely encountered can actually make it worse: you forget
2523+that this might be a problem, and debugging the differences can be
2524+difficult. If you allow site-packages to be included in your buildout,
2525+and the Python you use is not managed precisely by your application (for
2526+instance, it is a system Python), you open yourself up to these
2527+possibilities. Don't be unaware of the dangers.
2528+
2529+To show off these features, we need to use buildout with a Python
2530+executable with some extra paths to show ``add-site-packages``; and one
2531+guaranteed to have a sitecustomize module to show
2532+``exec-sitecustomize``. We'll make one using a test fixture called
2533+``make_py``. The os.environ change below will go into the sitecustomize,
2534+and the site_packages_path will be in the Python's path.
2535+
2536+ >>> py_path, site_packages_path = make_py(initialization='''\
2537+ ... import os
2538+ ... os.environ['zc.buildout'] = 'foo bar baz shazam'
2539+ ... ''')
2540+ >>> print site_packages_path
2541+ /executable_buildout/site-packages
2542+
2543+Now let's take a look at add-site-packages.
2544+
2545+ >>> write(sample_buildout, 'buildout.cfg',
2546+ ... """
2547+ ... [buildout]
2548+ ... parts = py
2549+ ... executable = %(py_path)s
2550+ ...
2551+ ... [py]
2552+ ... recipe = z3c.recipe.scripts:interpreter
2553+ ... add-site-packages = true
2554+ ... eggs = demo<0.3
2555+ ... find-links = %(server)s
2556+ ... index = %(server)s/index
2557+ ... """ % dict(server=link_server, py_path=py_path))
2558+
2559+ >>> print system(buildout),
2560+ Uninstalling py.
2561+ Installing py.
2562+ Generated interpreter '/sample-buildout/bin/py'.
2563+
2564+ >>> print system(join(sample_buildout, 'bin', 'py') +
2565+ ... ''' -c "import sys, pprint; pprint.pprint(sys.path)"''')
2566+ ... # doctest: +ELLIPSIS
2567+ ['',
2568+ '/sample-buildout/parts/py',
2569+ ...,
2570+ '/sample-buildout/eggs/demo-0.2-pyN.N.egg',
2571+ '/sample-buildout/eggs/demoneeded-1.2c1-pyN.N.egg',
2572+ '/executable_buildout/eggs/setuptools-X-pyN.N.egg',
2573+ '/executable_buildout/site-packages']
2574+ <BLANKLINE>
2575+
2576+Next we will use the exec-sitecustomize option. It simply copies
2577+Python's underlying sitecustomize module, if it exists, to the local
2578+version. The os.environ change shown above in the make_py call will go
2579+into the sitecustomize.
2580+
2581+ >>> write(sample_buildout, 'buildout.cfg',
2582+ ... """
2583+ ... [buildout]
2584+ ... parts = py
2585+ ... executable = %(py_path)s
2586+ ...
2587+ ... [py]
2588+ ... recipe = z3c.recipe.scripts:interpreter
2589+ ... exec-sitecustomize = true
2590+ ... eggs = demo<0.3
2591+ ... find-links = %(server)s
2592+ ... index = %(server)s/index
2593+ ... """ % dict(server=link_server, py_path=py_path))
2594+
2595+ >>> print system(buildout),
2596+ Uninstalling py.
2597+ Installing py.
2598+ Generated interpreter '/sample-buildout/bin/py'.
2599+
2600+ >>> cat(sample_buildout, 'parts', 'py', 'sitecustomize.py')
2601+ ... # doctest: +NORMALIZE_WHITESPACE +ELLIPSIS
2602+ <BLANKLINE>
2603+ # The following is from
2604+ # /executable_buildout/parts/py/sitecustomize.py
2605+ ...
2606+ import os
2607+ os.environ['zc.buildout'] = 'foo bar baz shazam'
2608+
2609+ >>> print system(join(sample_buildout, 'bin', 'py') +
2610+ ... ''' -c "import os; print os.environ['zc.buildout']"''')
2611+ foo bar baz shazam
2612+ <BLANKLINE>
2613+
2614+Options
2615+-------
2616+
2617+We'll focus now on the options that are different than zc.recipe.egg.
2618+
2619+Let's look at the ``extends`` option first.
2620+
2621+ >>> write(sample_buildout, 'buildout.cfg',
2622+ ... """
2623+ ... [buildout]
2624+ ... parts = demo python
2625+ ...
2626+ ... [demo]
2627+ ... recipe = z3c.recipe.scripts
2628+ ... eggs = demo<0.3
2629+ ... find-links = %(server)s
2630+ ... index = %(server)s/index
2631+ ...
2632+ ... [python]
2633+ ... recipe = z3c.recipe.scripts:interpreter
2634+ ... extends = demo
2635+ ... initialization =
2636+ ... import os
2637+ ... os.environ['zc.buildout'] = 'foo bar baz shazam'
2638+ ... """ % dict(server=link_server))
2639+
2640+That makes it easier to specify some initialization for the interpreter
2641+that is different than a script, while duplicating other configuration.
2642+
2643+Now let's put it in action.
2644+
2645+ >>> print system(buildout),
2646+ Uninstalling py.
2647+ Installing demo.
2648+ Generated script '/sample-buildout/bin/demo'.
2649+ Installing python.
2650+ Generated interpreter '/sample-buildout/bin/python'.
2651+
2652+ >>> print system(join(sample_buildout, 'bin', 'python') +
2653+ ... ' -c "import sys, pprint; pprint.pprint(sys.path[-2:])"')
2654+ ['/sample-buildout/eggs/demo-0.2-pyN.N.egg',
2655+ '/sample-buildout/eggs/demoneeded-1.2c1-pyN.N.egg']
2656+ <BLANKLINE>
2657+ >>> print system(join(sample_buildout, 'bin', 'python') +
2658+ ... ''' -c "import os; print os.environ['zc.buildout']"'''),
2659+ foo bar baz shazam
2660+
2661+Note that the parts/py directory has been cleaned up, and parts/python has
2662+been created.
2663+
2664+ >>> ls(sample_buildout, 'parts')
2665+ d demo
2666+ d python
2667+
2668+If you want to have initialization that only affects scripts, not the
2669+interpreter, you can use script-initialization. Here's a demonstration.
2670+
2671+ >>> write(sample_buildout, 'buildout.cfg',
2672+ ... """
2673+ ... [buildout]
2674+ ... parts = demo
2675+ ...
2676+ ... [demo]
2677+ ... recipe = z3c.recipe.scripts
2678+ ... eggs = demo<0.3
2679+ ... find-links = %(server)s
2680+ ... index = %(server)s/index
2681+ ... interpreter = py
2682+ ... script-initialization =
2683+ ... print "Hi from the script"
2684+ ... """ % dict(server=link_server))
2685+
2686+ >>> print system(buildout),
2687+ Uninstalling python.
2688+ Uninstalling demo.
2689+ Installing demo.
2690+ Generated script '/sample-buildout/bin/demo'.
2691+ Generated interpreter '/sample-buildout/bin/py'.
2692+
2693+ >>> print system(join(sample_buildout, 'bin', 'py') +
2694+ ... ''' -c "print 'Hi from the interpreter'"'''),
2695+ Hi from the interpreter
2696+
2697+ >>> print system(join(sample_buildout, 'bin', 'demo')),
2698+ Hi from the script
2699+ 2 2
2700+
2701+The last new option is ``name``. This simply changes the name of the
2702+interpreter, so that you are not forced to use the name of the section.
2703+
2704+ >>> write(sample_buildout, 'buildout.cfg',
2705+ ... """
2706+ ... [buildout]
2707+ ... parts = interpreter
2708+ ...
2709+ ... [interpreter]
2710+ ... name = python2
2711+ ... recipe = z3c.recipe.scripts:interpreter
2712+ ... eggs = demo<0.3
2713+ ... find-links = %(server)s
2714+ ... index = %(server)s/index
2715+ ... """ % dict(server=link_server))
2716+
2717+ >>> print system(buildout),
2718+ Uninstalling demo.
2719+ Installing interpreter.
2720+ Generated interpreter '/sample-buildout/bin/python2'.
2721+
2722+ >>> print system(join(sample_buildout, 'bin', 'python2') +
2723+ ... ' -c "print 42"')
2724+ 42
2725+ <BLANKLINE>
2726+
2727+The other options all identical to zc.recipe.egg.
2728
2729=== added file 'z3c.recipe.scripts_/src/z3c/recipe/scripts/__init__.py'
2730--- z3c.recipe.scripts_/src/z3c/recipe/scripts/__init__.py 1970-01-01 00:00:00 +0000
2731+++ z3c.recipe.scripts_/src/z3c/recipe/scripts/__init__.py 2010-02-23 20:41:17 +0000
2732@@ -0,0 +1,1 @@
2733+from z3c.recipe.scripts.scripts import Scripts, Interpreter
2734
2735=== added file 'z3c.recipe.scripts_/src/z3c/recipe/scripts/scripts.py'
2736--- z3c.recipe.scripts_/src/z3c/recipe/scripts/scripts.py 1970-01-01 00:00:00 +0000
2737+++ z3c.recipe.scripts_/src/z3c/recipe/scripts/scripts.py 2010-02-23 20:41:17 +0000
2738@@ -0,0 +1,101 @@
2739+##############################################################################
2740+#
2741+# Copyright (c) 2009-2010 Zope Corporation and Contributors.
2742+# All Rights Reserved.
2743+#
2744+# This software is subject to the provisions of the Zope Public License,
2745+# Version 2.1 (ZPL). A copy of the ZPL should accompany this distribution.
2746+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
2747+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
2748+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
2749+# FOR A PARTICULAR PURPOSE.
2750+#
2751+##############################################################################
2752+"""Install scripts from eggs.
2753+"""
2754+import os
2755+import zc.buildout
2756+import zc.buildout.easy_install
2757+from zc.recipe.egg.egg import ScriptBase
2758+
2759+
2760+class Base(ScriptBase):
2761+
2762+ def __init__(self, buildout, name, options):
2763+ if 'extends' in options:
2764+ options.update(buildout[options['extends']])
2765+ super(Base, self).__init__(buildout, name, options)
2766+ self.default_eggs = '' # Disables feature from zc.recipe.egg.
2767+ b_options = buildout['buildout']
2768+ options['parts-directory'] = os.path.join(
2769+ b_options['parts-directory'], self.name)
2770+
2771+ value = options.setdefault(
2772+ 'add-site-packages',
2773+ b_options.get('add-site-packages', 'false'))
2774+ if value not in ('true', 'false'):
2775+ raise zc.buildout.UserError(
2776+ "Invalid value for add-site-packages option: %s" %
2777+ (value,))
2778+ self.add_site_packages = (value == 'true')
2779+
2780+ value = options.setdefault(
2781+ 'exec-sitecustomize',
2782+ b_options.get('exec-sitecustomize', 'false'))
2783+ if value not in ('true', 'false'):
2784+ raise zc.buildout.UserError(
2785+ "Invalid value for exec-sitecustomize option: %s" %
2786+ (value,))
2787+ self.exec_sitecustomize = (value == 'true')
2788+
2789+
2790+class Interpreter(Base):
2791+
2792+ def __init__(self, buildout, name, options):
2793+ super(Interpreter, self).__init__(buildout, name, options)
2794+
2795+ options.setdefault('name', name)
2796+
2797+ def install(self):
2798+ reqs, ws = self.working_set()
2799+ options = self.options
2800+ generated = []
2801+ if not os.path.exists(options['parts-directory']):
2802+ os.mkdir(options['parts-directory'])
2803+ generated.append(options['parts-directory'])
2804+ generated.extend(zc.buildout.easy_install.sitepackage_safe_scripts(
2805+ options['bin-directory'], ws, options['executable'],
2806+ options['parts-directory'],
2807+ interpreter=options['name'],
2808+ extra_paths=self.extra_paths,
2809+ initialization=options.get('initialization', ''),
2810+ add_site_packages=self.add_site_packages,
2811+ exec_sitecustomize=self.exec_sitecustomize,
2812+ relative_paths=self._relative_paths,
2813+ ))
2814+ return generated
2815+
2816+ update = install
2817+
2818+
2819+class Scripts(Base):
2820+
2821+ def _install(self, reqs, ws, scripts):
2822+ options = self.options
2823+ generated = []
2824+ if not os.path.exists(options['parts-directory']):
2825+ os.mkdir(options['parts-directory'])
2826+ generated.append(options['parts-directory'])
2827+ generated.extend(zc.buildout.easy_install.sitepackage_safe_scripts(
2828+ options['bin-directory'], ws, options['executable'],
2829+ options['parts-directory'], reqs=reqs, scripts=scripts,
2830+ interpreter=options.get('interpreter'),
2831+ extra_paths=self.extra_paths,
2832+ initialization=options.get('initialization', ''),
2833+ add_site_packages=self.add_site_packages,
2834+ exec_sitecustomize=self.exec_sitecustomize,
2835+ relative_paths=self._relative_paths,
2836+ script_arguments=options.get('arguments', ''),
2837+ script_initialization=options.get('script-initialization', '')
2838+ ))
2839+ return generated
2840
2841=== added file 'z3c.recipe.scripts_/src/z3c/recipe/scripts/tests.py'
2842--- z3c.recipe.scripts_/src/z3c/recipe/scripts/tests.py 1970-01-01 00:00:00 +0000
2843+++ z3c.recipe.scripts_/src/z3c/recipe/scripts/tests.py 2010-02-23 20:41:17 +0000
2844@@ -0,0 +1,293 @@
2845+##############################################################################
2846+#
2847+# Copyright (c) 2006 Zope Corporation and Contributors.
2848+# All Rights Reserved.
2849+#
2850+# This software is subject to the provisions of the Zope Public License,
2851+# Version 2.1 (ZPL). A copy of the ZPL should accompany this distribution.
2852+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
2853+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
2854+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
2855+# FOR A PARTICULAR PURPOSE.
2856+#
2857+##############################################################################
2858+
2859+import os, re, shutil, sys
2860+import zc.buildout.tests
2861+import zc.buildout.testselectingpython
2862+import zc.buildout.testing
2863+
2864+import unittest
2865+from zope.testing import doctest, renormalizing
2866+
2867+# We do not explicitly test the recipe support for the ``eggs``,
2868+# ``find-links``, and ``index`` options because they are used for most or
2869+# all of the examples. The README tests ``extends``,
2870+# ``include-site-customization`` and ``name``. That leaves ``python``,
2871+# ``extra-paths``, ``initialization``, ``relative-paths``, and
2872+# ``add-site-packages``.
2873+
2874+def supports_python_option():
2875+ """
2876+This simply shows that the ``python`` option can specify another section to
2877+find the ``executable``. (The ``python`` option defaults to looking in the
2878+``buildout`` section.) We do this by creating a custom Python that will have
2879+some initialization that we can look for.
2880+
2881+ >>> py_path, site_packages_path = make_py(initialization='''
2882+ ... import os
2883+ ... os.environ['zc.buildout'] = 'foo bar baz shazam'
2884+ ... ''')
2885+
2886+ >>> write(sample_buildout, 'buildout.cfg',
2887+ ... '''
2888+ ... [buildout]
2889+ ... parts = py
2890+ ...
2891+ ... [custom_python]
2892+ ... executable = %(py_path)s
2893+ ...
2894+ ... [py]
2895+ ... recipe = z3c.recipe.scripts:interpreter
2896+ ... exec-sitecustomize = true
2897+ ... eggs = demo<0.3
2898+ ... find-links = %(server)s
2899+ ... index = %(server)s/index
2900+ ... python = custom_python
2901+ ... ''' % dict(server=link_server, py_path=py_path))
2902+
2903+ >>> print system(buildout),
2904+ Installing py.
2905+ Getting distribution for 'demo<0.3'.
2906+ Got demo 0.2.
2907+ Getting distribution for 'demoneeded'.
2908+ Got demoneeded 1.2c1.
2909+ Generated interpreter '/sample-buildout/bin/py'.
2910+
2911+ >>> print system(join(sample_buildout, 'bin', 'py') +
2912+ ... ''' -c "import os; print os.environ['zc.buildout']"'''),
2913+ foo bar baz shazam
2914+"""
2915+
2916+def interpreter_recipe_supports_extra_paths_option():
2917+ """
2918+This shows that specifying extra-paths will affect sys.path.
2919+
2920+This recipe will not add paths that do not exist, so we create them.
2921+
2922+ >>> mkdir(sample_buildout, 'foo')
2923+ >>> mkdir(sample_buildout, 'foo', 'bar')
2924+ >>> mkdir(sample_buildout, 'spam')
2925+
2926+ >>> write(sample_buildout, 'buildout.cfg',
2927+ ... '''
2928+ ... [buildout]
2929+ ... parts = py
2930+ ...
2931+ ... [py]
2932+ ... recipe = z3c.recipe.scripts:interpreter
2933+ ... find-links = %(server)s
2934+ ... index = %(server)s/index
2935+ ... extra-paths =
2936+ ... ${buildout:directory}/foo/bar
2937+ ... ${buildout:directory}/spam
2938+ ... ''' % dict(server=link_server))
2939+
2940+ >>> print system(buildout),
2941+ Installing py.
2942+ Generated interpreter '/sample-buildout/bin/py'.
2943+ >>> print system(join(sample_buildout, 'bin', 'py') +
2944+ ... ''' -c "import sys;print 'path' + ' '.join(sys.path)"''')
2945+ ... # doctest:+ELLIPSIS
2946+ path.../foo/bar /sample-buildout/spam...
2947+
2948+"""
2949+
2950+def interpreter_recipe_supports_initialization_option():
2951+ """
2952+This simply shows that the ``initialization`` option can specify code to
2953+run on initialization.
2954+
2955+ >>> write(sample_buildout, 'buildout.cfg',
2956+ ... '''
2957+ ... [buildout]
2958+ ... parts = py
2959+ ...
2960+ ... [py]
2961+ ... recipe = z3c.recipe.scripts:interpreter
2962+ ... initialization =
2963+ ... import os
2964+ ... os.environ['zc.buildout'] = 'foo bar baz shazam'
2965+ ... eggs = demo<0.3
2966+ ... find-links = %(server)s
2967+ ... index = %(server)s/index
2968+ ... ''' % dict(server=link_server))
2969+
2970+ >>> print system(buildout),
2971+ Installing py.
2972+ Getting distribution for 'demo<0.3'.
2973+ Got demo 0.2.
2974+ Getting distribution for 'demoneeded'.
2975+ Got demoneeded 1.2c1.
2976+ Generated interpreter '/sample-buildout/bin/py'.
2977+
2978+ >>> cat(sample_buildout, 'parts', 'py', 'sitecustomize.py')
2979+ ... # doctest: +NORMALIZE_WHITESPACE
2980+ <BLANKLINE>
2981+ import os
2982+ os.environ['zc.buildout'] = 'foo bar baz shazam'
2983+ >>> print system(join(sample_buildout, 'bin', 'py') +
2984+ ... ''' -c "import os; print os.environ['zc.buildout']"'''),
2985+ foo bar baz shazam
2986+
2987+This also works with the exec-sitecustomize option, processing local
2988+initialization, and then the Python's initialization. We show this with a
2989+custom Python.
2990+
2991+ >>> py_path, site_packages_path = make_py(initialization='''
2992+ ... import os
2993+ ... os.environ['zc.buildout'] = 'foo bar baz shazam'
2994+ ... ''')
2995+
2996+ >>> write(sample_buildout, 'buildout.cfg',
2997+ ... '''
2998+ ... [buildout]
2999+ ... parts = py
3000+ ...
3001+ ... [custom_python]
3002+ ... executable = %(py_path)s
3003+ ...
3004+ ... [py]
3005+ ... recipe = z3c.recipe.scripts:interpreter
3006+ ... initialization =
3007+ ... import os
3008+ ... os.environ['zc.recipe.egg'] = 'baLOOba'
3009+ ... exec-sitecustomize = true
3010+ ... eggs = demo<0.3
3011+ ... find-links = %(server)s
3012+ ... index = %(server)s/index
3013+ ... python = custom_python
3014+ ... ''' % dict(server=link_server, py_path=py_path))
3015+
3016+ >>> print system(buildout),
3017+ Uninstalling py.
3018+ Installing py.
3019+ Generated interpreter '/sample-buildout/bin/py'.
3020+
3021+ >>> cat(sample_buildout, 'parts', 'py', 'sitecustomize.py')
3022+ ... # doctest: +NORMALIZE_WHITESPACE +ELLIPSIS
3023+ <BLANKLINE>
3024+ import os
3025+ os.environ['zc.recipe.egg'] = 'baLOOba'
3026+ <BLANKLINE>
3027+ # The following is from
3028+ # /executable_buildout/parts/py/sitecustomize.py
3029+ ...
3030+ import os
3031+ os.environ['zc.buildout'] = 'foo bar baz shazam'
3032+
3033+ >>> print system(join(sample_buildout, 'bin', 'py') + ' -c ' +
3034+ ... '''"import os; print os.environ['zc.recipe.egg']"'''),
3035+ baLOOba
3036+ >>> print system(join(sample_buildout, 'bin', 'py') +
3037+ ... ''' -c "import os; print os.environ['zc.buildout']"'''),
3038+ foo bar baz shazam
3039+
3040+"""
3041+
3042+def interpreter_recipe_supports_relative_paths_option():
3043+ """
3044+This shows that the relative-paths option affects the code for inserting
3045+paths into sys.path.
3046+
3047+ >>> write(sample_buildout, 'buildout.cfg',
3048+ ... '''
3049+ ... [buildout]
3050+ ... parts = py
3051+ ...
3052+ ... [py]
3053+ ... recipe = z3c.recipe.scripts:interpreter
3054+ ... find-links = %(server)s
3055+ ... index = %(server)s/index
3056+ ... relative-paths = true
3057+ ... extra-paths =
3058+ ... /foo/bar
3059+ ... ${buildout:directory}/spam
3060+ ... ''' % dict(server=link_server))
3061+
3062+ >>> print system(buildout),
3063+ Installing py.
3064+ Generated interpreter '/sample-buildout/bin/py'.
3065+
3066+Let's look at the site.py that was generated:
3067+
3068+ >>> import sys
3069+ >>> sys.stdout.write('#'); cat(sample_buildout, 'parts', 'py', 'site.py')
3070+ ... # doctest: +NORMALIZE_WHITESPACE +ELLIPSIS
3071+ #...
3072+ def addsitepackages(known_paths):
3073+ "..."
3074+ join = os.path.join
3075+ base = os.path.dirname(os.path.abspath(os.path.realpath(__file__)))
3076+ base = os.path.dirname(base)
3077+ base = os.path.dirname(base)
3078+ buildout_paths = [
3079+ '/foo/bar',
3080+ join(base, 'spam')
3081+ ]...
3082+
3083+
3084+"""
3085+
3086+def setUp(test):
3087+ zc.buildout.tests.easy_install_SetUp(test)
3088+ zc.buildout.testing.install_develop('zc.recipe.egg', test)
3089+ zc.buildout.testing.install_develop('z3c.recipe.scripts', test)
3090+
3091+def setUpSelecting(test):
3092+ zc.buildout.testselectingpython.setup(test)
3093+ zc.buildout.testing.install_develop('zc.recipe.egg', test)
3094+ zc.buildout.testing.install_develop('z3c.recipe.scripts', test)
3095+
3096+def test_suite():
3097+ suite = unittest.TestSuite((
3098+ doctest.DocFileSuite(
3099+ 'README.txt',
3100+ setUp=setUp, tearDown=zc.buildout.testing.buildoutTearDown,
3101+ checker=renormalizing.RENormalizing([
3102+ zc.buildout.testing.normalize_path,
3103+ zc.buildout.testing.normalize_endings,
3104+ zc.buildout.testing.normalize_script,
3105+ zc.buildout.testing.normalize_egg_py,
3106+ zc.buildout.tests.normalize_bang,
3107+ (re.compile(r'zc.buildout(-\S+)?[.]egg(-link)?'),
3108+ 'zc.buildout.egg'),
3109+ (re.compile('[-d] setuptools-[^-]+-'), 'setuptools-X-'),
3110+ (re.compile(r'setuptools-[\w.]+-py'), 'setuptools-X-py'),
3111+ (re.compile(r'eggs\\\\demo'), 'eggs/demo'),
3112+ (re.compile(r'[a-zA-Z]:\\\\foo\\\\bar'), '/foo/bar'),
3113+ (re.compile(r'\#!\S+\bpython\S*'), '#!/usr/bin/python'),
3114+ # Normalize generate_script's Windows interpreter to UNIX:
3115+ (re.compile(r'\nimport subprocess\n'), '\n'),
3116+ (re.compile('subprocess\\.call\\(argv, env=environ\\)'),
3117+ 'os.execve(sys.executable, argv, environ)'),
3118+ ])
3119+ ),
3120+ doctest.DocTestSuite(
3121+ setUp=setUp,
3122+ tearDown=zc.buildout.testing.buildoutTearDown,
3123+ checker=renormalizing.RENormalizing([
3124+ zc.buildout.testing.normalize_path,
3125+ zc.buildout.testing.normalize_endings,
3126+ zc.buildout.testing.normalize_egg_py,
3127+ (re.compile(r'[a-zA-Z]:\\\\foo\\\\bar'), '/foo/bar'),
3128+ ]),
3129+ ),
3130+
3131+ ))
3132+
3133+ return suite
3134+
3135+if __name__ == '__main__':
3136+ unittest.main(defaultTest='test_suite')
3137+
3138
3139=== modified file 'zc.recipe.egg_/setup.py'
3140--- zc.recipe.egg_/setup.py 2009-08-06 13:49:47 +0000
3141+++ zc.recipe.egg_/setup.py 2010-02-23 20:41:17 +0000
3142@@ -16,7 +16,7 @@
3143 $Id$
3144 """
3145
3146-version = '0'
3147+version = '1.2.3dev'
3148
3149 import os
3150 from setuptools import setup, find_packages
3151@@ -66,7 +66,7 @@
3152 package_dir = {'':'src'},
3153 namespace_packages = ['zc', 'zc.recipe'],
3154 install_requires = [
3155- 'zc.buildout >=1.2.0',
3156+ 'zc.buildout >=1.5.0dev',
3157 'setuptools'],
3158 tests_require = ['zope.testing'],
3159 test_suite = name+'.tests.test_suite',
3160
3161=== modified file 'zc.recipe.egg_/src/zc/recipe/egg/README.txt'
3162--- zc.recipe.egg_/src/zc/recipe/egg/README.txt 2010-02-23 20:41:17 +0000
3163+++ zc.recipe.egg_/src/zc/recipe/egg/README.txt 2010-02-23 20:41:17 +0000
3164@@ -154,6 +154,8 @@
3165 interpreter
3166 The name of a script to generate that allows access to a Python
3167 interpreter that has the path set based on the eggs installed.
3168+ (See the ``z3c.recipe.scripts`` recipe for a more full-featured
3169+ interpreter.)
3170
3171 extra-paths
3172 Extra paths to include in a generated script.
3173@@ -577,7 +579,7 @@
3174 - demo
3175 - other
3176
3177- >>> cat(sample_buildout, 'bin', 'other')
3178+ >>> cat(sample_buildout, 'bin', 'other') # doctest: +NORMALIZE_WHITESPACE
3179 #!/usr/local/bin/python2.4
3180 <BLANKLINE>
3181 import sys
3182@@ -640,3 +642,4 @@
3183 Uninstalling bigdemo.
3184 Installing demo.
3185 Generated script '/sample-buildout/bin/foo'.
3186+
3187
3188=== modified file 'zc.recipe.egg_/src/zc/recipe/egg/api.txt'
3189--- zc.recipe.egg_/src/zc/recipe/egg/api.txt 2010-02-23 20:41:17 +0000
3190+++ zc.recipe.egg_/src/zc/recipe/egg/api.txt 2010-02-23 20:41:17 +0000
3191@@ -117,6 +117,7 @@
3192 extras = other
3193 find-links = http://localhost:27071/
3194 index = http://localhost:27071/index
3195+ python = buildout
3196 recipe = sample
3197
3198 If we use the extra-paths option:
3199
3200=== modified file 'zc.recipe.egg_/src/zc/recipe/egg/custom.txt'
3201--- zc.recipe.egg_/src/zc/recipe/egg/custom.txt 2010-02-23 20:41:17 +0000
3202+++ zc.recipe.egg_/src/zc/recipe/egg/custom.txt 2010-02-23 20:41:17 +0000
3203@@ -150,6 +150,7 @@
3204
3205 >>> ls(sample_buildout, 'develop-eggs')
3206 d extdemo-1.4-py2.4-unix-i686.egg
3207+ - z3c.recipe.scripts.egg-link
3208 - zc.recipe.egg.egg-link
3209
3210 Note that no scripts or dependencies are installed. To install
3211@@ -231,6 +232,7 @@
3212 >>> ls(sample_buildout, 'develop-eggs')
3213 - demo.egg-link
3214 d extdemo-1.4-py2.4-unix-i686.egg
3215+ - z3c.recipe.scripts.egg-link
3216 - zc.recipe.egg.egg-link
3217
3218 But if we run the buildout in the default on-line and newest modes, we
3219@@ -248,6 +250,7 @@
3220 - demo.egg-link
3221 d extdemo-1.4-py2.4-linux-i686.egg
3222 d extdemo-1.5-py2.4-linux-i686.egg
3223+ - z3c.recipe.scripts.egg-link
3224 - zc.recipe.egg.egg-link
3225
3226 Controlling the version used
3227@@ -287,6 +290,7 @@
3228 >>> ls(sample_buildout, 'develop-eggs')
3229 - demo.egg-link
3230 d extdemo-1.4-py2.4-linux-i686.egg
3231+ - z3c.recipe.scripts.egg-link
3232 - zc.recipe.egg.egg-link
3233
3234
3235@@ -553,6 +557,7 @@
3236 >>> ls('develop-eggs')
3237 - demo.egg-link
3238 - extdemo.egg-link
3239+ - z3c.recipe.scripts.egg-link
3240 - zc.recipe.egg.egg-link
3241
3242 and the extdemo now has a built extension:
3243
3244=== modified file 'zc.recipe.egg_/src/zc/recipe/egg/egg.py'
3245--- zc.recipe.egg_/src/zc/recipe/egg/egg.py 2010-02-23 20:41:17 +0000
3246+++ zc.recipe.egg_/src/zc/recipe/egg/egg.py 2010-02-23 20:41:17 +0000
3247@@ -19,11 +19,12 @@
3248 import logging, os, re, zipfile
3249 import zc.buildout.easy_install
3250
3251+
3252 class Eggs(object):
3253
3254 def __init__(self, buildout, name, options):
3255 self.buildout = buildout
3256- self.name = name
3257+ self.name = self.default_eggs = name
3258 self.options = options
3259 b_options = buildout['buildout']
3260 links = options.get('find-links', b_options['find-links'])
3261@@ -52,7 +53,7 @@
3262 # verify that this is None, 'true' or 'false'
3263 get_bool(options, 'unzip')
3264
3265- python = options.get('python', b_options['python'])
3266+ python = options.setdefault('python', b_options['python'])
3267 options['executable'] = buildout[python]['executable']
3268
3269 def working_set(self, extra=()):
3270@@ -65,15 +66,16 @@
3271
3272 distributions = [
3273 r.strip()
3274- for r in options.get('eggs', self.name).split('\n')
3275+ for r in options.get('eggs', self.default_eggs).split('\n')
3276 if r.strip()]
3277 orig_distributions = distributions[:]
3278 distributions.extend(extra)
3279
3280- if self.buildout['buildout'].get('offline') == 'true':
3281+ if b_options.get('offline') == 'true':
3282 ws = zc.buildout.easy_install.working_set(
3283 distributions, options['executable'],
3284- [options['develop-eggs-directory'], options['eggs-directory']]
3285+ [options['develop-eggs-directory'],
3286+ options['eggs-directory']],
3287 )
3288 else:
3289 kw = {}
3290@@ -85,7 +87,7 @@
3291 index=self.index,
3292 executable=options['executable'],
3293 path=[options['develop-eggs-directory']],
3294- newest=self.buildout['buildout'].get('newest') == 'true',
3295+ newest=b_options.get('newest') == 'true',
3296 allow_hosts=self.allow_hosts,
3297 **kw)
3298
3299@@ -97,16 +99,19 @@
3300
3301 update = install
3302
3303-class Scripts(Eggs):
3304+
3305+class ScriptBase(Eggs):
3306
3307 def __init__(self, buildout, name, options):
3308- super(Scripts, self).__init__(buildout, name, options)
3309-
3310- options['bin-directory'] = buildout['buildout']['bin-directory']
3311+ super(ScriptBase, self).__init__(buildout, name, options)
3312+
3313+ b_options = buildout['buildout']
3314+
3315+ options['bin-directory'] = b_options['bin-directory']
3316 options['_b'] = options['bin-directory'] # backward compat.
3317
3318 self.extra_paths = [
3319- os.path.join(buildout['buildout']['directory'], p.strip())
3320+ os.path.join(b_options['directory'], p.strip())
3321 for p in options.get('extra-paths', '').split('\n')
3322 if p.strip()
3323 ]
3324@@ -115,11 +120,9 @@
3325
3326
3327 relative_paths = options.get(
3328- 'relative-paths',
3329- buildout['buildout'].get('relative-paths', 'false')
3330- )
3331+ 'relative-paths', b_options.get('relative-paths', 'false'))
3332 if relative_paths == 'true':
3333- options['buildout-directory'] = buildout['buildout']['directory']
3334+ options['buildout-directory'] = b_options['directory']
3335 self._relative_paths = options['buildout-directory']
3336 else:
3337 self._relative_paths = ''
3338@@ -128,12 +131,13 @@
3339 parse_entry_point = re.compile(
3340 '([^=]+)=(\w+(?:[.]\w+)*):(\w+(?:[.]\w+)*)$'
3341 ).match
3342+
3343 def install(self):
3344 reqs, ws = self.working_set()
3345 options = self.options
3346
3347 scripts = options.get('scripts')
3348- if scripts or scripts is None:
3349+ if scripts or scripts is None or options.get('interpreter'):
3350 if scripts is not None:
3351 scripts = scripts.split()
3352 scripts = dict([
3353@@ -157,22 +161,32 @@
3354 name = dist.project_name
3355 if name != 'setuptools' and name not in reqs:
3356 reqs.append(name)
3357-
3358- return zc.buildout.easy_install.scripts(
3359- reqs, ws, options['executable'],
3360- options['bin-directory'],
3361- scripts=scripts,
3362- extra_paths=self.extra_paths,
3363- interpreter=options.get('interpreter'),
3364- initialization=options.get('initialization', ''),
3365- arguments=options.get('arguments', ''),
3366- relative_paths=self._relative_paths,
3367- )
3368-
3369+ return self._install(reqs, ws, scripts)
3370 return ()
3371
3372 update = install
3373
3374+ def _install(self, reqs, ws, scripts):
3375+ # Subclasses implement this.
3376+ raise NotImplementedError()
3377+
3378+
3379+class Scripts(ScriptBase):
3380+
3381+ def _install(self, reqs, ws, scripts):
3382+ options = self.options
3383+ return zc.buildout.easy_install.scripts(
3384+ reqs, ws, options['executable'],
3385+ options['bin-directory'],
3386+ scripts=scripts,
3387+ extra_paths=self.extra_paths,
3388+ interpreter=options.get('interpreter'),
3389+ initialization=options.get('initialization', ''),
3390+ arguments=options.get('arguments', ''),
3391+ relative_paths=self._relative_paths
3392+ )
3393+
3394+
3395 def get_bool(options, name, default=False):
3396 value = options.get(name)
3397 if not value:
3398
3399=== modified file 'zc.recipe.egg_/src/zc/recipe/egg/selecting-python.txt'
3400--- zc.recipe.egg_/src/zc/recipe/egg/selecting-python.txt 2010-02-23 20:41:17 +0000
3401+++ zc.recipe.egg_/src/zc/recipe/egg/selecting-python.txt 2010-02-23 20:41:17 +0000
3402@@ -35,7 +35,7 @@
3403 ... index = http://www.python.org/pypi/
3404 ...
3405 ... [python2.4]
3406- ... executable = %(python23)s
3407+ ... executable = %(python24)s
3408 ...
3409 ... [demo]
3410 ... recipe = zc.recipe.egg
3411@@ -43,7 +43,7 @@
3412 ... find-links = %(server)s
3413 ... python = python2.4
3414 ... interpreter = py-demo
3415- ... """ % dict(server=link_server, python23=other_executable))
3416+ ... """ % dict(server=link_server, python24=other_executable))
3417
3418 Now, if we run the buildout:
3419
3420@@ -55,8 +55,6 @@
3421 Getting distribution for 'demo<0.3'.
3422 Got demo 0.2.
3423 Getting distribution for 'demoneeded'.
3424- Getting distribution for 'setuptools'.
3425- Got setuptools 0.6.
3426 Got demoneeded 1.2c1.
3427 Generated script '/sample-buildout/bin/demo'.
3428 Generated interpreter '/sample-buildout/bin/py-demo'.
3429@@ -66,7 +64,6 @@
3430 >>> ls(sample_buildout, 'eggs')
3431 - demo-0.2-py2.4.egg
3432 - demoneeded-1.2c1-py2.4.egg
3433- d setuptools-0.6-py2.4.egg
3434 d setuptools-0.6-py2.5.egg
3435 - zc.buildout-1.0-py2.5.egg
3436

Subscribers

People subscribed via source and target branches

to all changes: