Merge lp:~gary/zc.buildout/python-support-4 into lp:zc.buildout

[Gary Poster]

[Gary Poster (gary)]

This is a completely reworked approach to using zc.buildout with a system Python (or any Python with a populated site packages directory), deprecating my previous attempt.

The basic approach revolves around writing our own site.py and sitecustomize.py that are responsible for setting up paths and any other environment changes desired. Then if the path to these rewritten files are included (via PYTHONPATH, or with -S followed by sys.path manipulation and ``import site``) you have the environment set up, using the standard Python mechanism.

Similarly to my past work, the point that got complicated was namespace packages. The solution shown here is straightforward, as you can see in the easy_install.txt discussion.

However, if you try to use a system Python, you expose its packages using ``add-site-packages=true``, and you use eggs with namespace packages, it will incur a startup cost for generated scripts that is quadratically related to the number of eggs added by zc.buildout, because of code in pkg_resources. (We also have that same behavior now in Launchpad.) While I did a number of experiments to try and bypass the inefficiencies of the pkg_resources code, I discarded them, because Launchpad is using pkg_resources and expects it to be fully set up. Other code may be the same. The right solution is to fix pkg_resources, in Setuptools or Distribute, at which point the approach I have in this branch should still work, and be much faster.

I did punt a bit in the Windows version of the new interpreter script: I could not get the standard Windows path hacks to work for sys.exec*. I used subprocess instead for Windows, and commented this section heavily.

Obviously, this is a huge branch. I did not see a good way of subdividing it beyond what I already did (~gary/zc.buildout/python-support-1-cleanup, ~gary/zc.buildout/python-support-2-bootstrap, and ~gary/zc.buildout/python-support-3-options). I did consider the following subdivision:

- First, you could look at the changes to easy_install.py and easy_install.txt. This is the heart of the changes. It refactors the existing script generation into bits I could reuse more easily, and then makes a new version script generation function (``generated_scripts``) that works as I describe above.

- Second, you could look at zc.recipe.egg and z3c.recipe.scripts. zc.recipe.egg is refactored so I could reuse bits more easily. z3c.recipe.scripts is a brand new recipe that uses generate_scripts, as described above.

- Finally, you could look at the new tests and test changes in zc.buildout. They use zc.recipe.egg and z3c.recipe.scripts to test that system installs can work as desired--essentially showing that all of the earlier work does in fact produce a system that does what we want.

I didn't feel that separating them made a lot of sense, because you could not test whether generate_scripts actually did what we wanted until we had z3c.recipe.scripts also available. The changes are all interrelated.

Something to note is that zc.recipe.egg does not have any changed behavior. Also, it is not safe to use with a system Python. If you want that behavior for scripts and interpreters, use z3c.rec...

[Francis J. Lacoste (flacoste)]

Hi Gary,

Wow, this was a big one. I have a bunch of comments, mostly about
clarifications and typos.

But good to merge otherwise as far as Launchpad is concerned.

Cheers

> === modified file 'CHANGES.txt'
> --- CHANGES.txt 2010-02-17 22:17:12 +0000
> +++ CHANGES.txt 2010-02-17 22:17:12 +0000
> @@ -6,6 +6,25 @@
>
> New Features:
>
> +- Buildout can be safely used with a system Python, as long as you use the
> + new z3c.recipe.scripts recipe to generate scripts and interpreters, rather
> + than zc.recipe.egg (which is still a fully supported, and simpler, way of
> + generating scripts and interpreters if you are using a "clean" Python).

Really, I tohugh zc.recipe.egg would also do the right thing. Can you explain
the differences to me?

> +
> + A hopefully slight limitation: in no cases are distributions in your
> + site-packages used to satisfy buildout dependencies. The
> + site-packages can be used in addition to the dependencies specified in
> + your buildout, and buildout dependencies can override code in your
> + site-packages, but even if your Python's site-packages has the same
> + exact version as specified in your buildout configuration, buildout
> + will still use its own copy.

I assume dependencies related to zc.buildout itself like distribute,
and setuptools. Not general dependencies managed by buildout, right?

>
> +- Installing a namespace package using a Python that already has a package
> + in the same namespace (e.g., in the Python's site-packages) failed in
> + some cases.
> +

I assume you mean this is fixed. (IOW, that is in a fixed bugs section?)

> +- Another variation of this error showed itself when at least two
> + dependencies were in a shared location like site-packages, and the
> + first one met the "versions" setting. The first dependency would be
> + added, but subsequent dependencies from the same location (e.g.,
> + site-packages) would use the version of the package found in the
> + shared location, ignoring the version setting.
> +

I assume the same thing here.

> === modified file 'README.txt'
> --- README.txt 2010-02-17 22:17:12 +0000
> +++ README.txt 2010-02-17 22:17:12 +0000
> @@ -37,6 +37,11 @@
> dependencies. It installs their console-script entry points with
> the needed eggs included in their paths.
>
> +`z3c.recipe.scripts <http://pypi.python.org/pypi/z3c.recipe.scripts>`_
> + This scripts recipe builds interpreter scripts and entry point scripts
> + based on eggs. These scripts have more features and flexibility than the
> + ones offered by zc.recipe.egg.
> +

Again, might be worthwhile to go into more details of when one should be used
instead of the other.

=== modified file 'src/zc/buildout/easy_install.py'
--- src/zc/buildout/easy_install.py 2010-02-17 22:17:12 +0000
+++ src/zc/buildout/easy_install.py 2010-02-17 22:17:12 +0000
@@ -137,9 +137,50 @@
 else:
     _safe_arg = str

-_easy_install_cmd = _safe_arg(
- 'from setuptools.command.easy_install import main; main()'
- )
+# The following string is used to run easy_install in
+# Installer._call_easy_install. It is started with python -S (that is,
+# don't import site at start). That flag, a...

review: Approve

[lp:~gary/zc.buildout/python-support-4]

550. By Gary Poster on 2010-02-19

merge from gary-3

551. By Gary Poster on 2010-02-22

switch ``generate_scripts`` to ``sitepackage_safe_scripts`` per review

552. By Gary Poster on 2010-02-22

add missing reset_interpreter calls

553. By Gary Poster on 2010-02-22

change the way that we identify namespace packages and better comment the code, per review.

554. By Gary Poster on 2010-02-22

doc fixes per review.

555. By Gary Poster on 2010-02-23

try again to describe the change.

556. By Gary Poster on 2010-02-23

fix docstring

[Gary Poster (gary)]

Thank you, Francis!

Francis and I talked about the review. On the basis of his comments and the discussion, I did the revisions listed above as well as two further branches: https://code.edge.launchpad.net/~gary/zc.buildout/python-support-5-initial-egg-control and https://code.edge.launchpad.net/~gary/zc.buildout/python-support-6-egg-control .

I'll respond to his comments point by point now.

> === modified file 'CHANGES.txt'
> --- CHANGES.txt 2010-02-17 22:17:12 +0000
> +++ CHANGES.txt 2010-02-17 22:17:12 +0000
> @@ -6,6 +6,25 @@
>
> New Features:
>
> +- Buildout can be safely used with a system Python, as long as you use the
> + new z3c.recipe.scripts recipe to generate scripts and interpreters, rather
> + than zc.recipe.egg (which is still a fully supported, and simpler, way of
> + generating scripts and interpreters if you are using a "clean" Python).
>
> Really, I tohugh zc.recipe.egg would also do the right thing. Can you explain
> the differences to me?

We discussed this. He meant that he wanted the CHANGES document to more clearly describe the differences. I tried this:

- Buildout can be safely used with a system Python (or any Python with code
  in site-packages), as long as you use the new z3c.recipe.scripts
  recipe to generate scripts and interpreters, rather than zc.recipe.egg.

  zc.recipe.egg is still a fully supported, and simpler, way of
  generating scripts and interpreters if you are using a "clean" Python,
  without code installed in site-packages. It keeps its previous behavior in
  order to provide backwards compatibility.

>
> +
> + A hopefully slight limitation: in no cases are distributions in your
> + site-packages used to satisfy buildout dependencies. The
> + site-packages can be used in addition to the dependencies specified in
> + your buildout, and buildout dependencies can override code in your
> + site-packages, but even if your Python's site-packages has the same
> + exact version as specified in your buildout configuration, buildout
> + will still use its own copy.
>
> I assume dependencies related to zc.buildout itself like distribute,
> and setuptools. Not general dependencies managed by buildout, right?

My comment was wrong--and the actual situation turned out to be a big problem, leading to the two subsequent branches I mentioned above.

The actual situation was that dependencies would be obtained from site-packages--even if you said you didn't want to use site-packages. This is completely broken for the use case of wanting to use a system Python without site-packages.

I fixed this in the later branches.

The later branches mean that one can choose to get general dependencies from site-packages, or not.

> +- Installing a namespace package using a Python that already has a package
> + in the same namespace (e.g., in the Python's site-packages) failed in
> + some cases.
> +
>
> I assume you mean this is fixed. (IOW, that is in a fixed bugs section?)

Right.

> +- Another variation of this error showed itself when at least two
> + dependencies were in a shared location like site-packages, and the
> + first one met the "versions" setting. The first dependency would be
> + ad...

Unmerged revisions

556. By Gary Poster on 2010-02-23

fix docstring

555. By Gary Poster on 2010-02-23

try again to describe the change.

554. By Gary Poster on 2010-02-22

doc fixes per review.

553. By Gary Poster on 2010-02-22

change the way that we identify namespace packages and better comment the code, per review.

552. By Gary Poster on 2010-02-22

add missing reset_interpreter calls

551. By Gary Poster on 2010-02-22

switch ``generate_scripts`` to ``sitepackage_safe_scripts`` per review

550. By Gary Poster on 2010-02-19

merge from gary-3

549. By Gary Poster on 2010-02-17

propagate merge from gary-3 <- gary-2 <- gary-1 <- trunk

548. By gary on 2010-02-13

fix some tests on other Python versions

547. By gary on 2010-02-12

revert attempt to skip some of the pkg_resources dance: it caused me trouble.

=== modified file 'CHANGES.txt'
--- CHANGES.txt	2010-02-23 20:41:17 +0000
+++ CHANGES.txt	2010-02-23 20:41:17 +0000
@@ -6,6 +6,27 @@
 
 New Features:
 
+- Buildout can be safely used with a system Python (or any Python with code
+  in site-packages), as long as you use the new z3c.recipe.scripts
+  recipe to generate scripts and interpreters, rather than zc.recipe.egg.
+
+  zc.recipe.egg is still a fully supported, and simpler, way of
+  generating scripts and interpreters if you are using a "clean" Python,
+  without code installed in site-packages. It keeps its previous behavior in
+  order to provide backwards compatibility.
+
+  (Note that this branch is incomplete in its implementation of this feature:
+  if eggs are in installed in site-packages but you do not want to use
+  site-packages, the eggs will drag in site-packages even if you try to
+  exclude it.  This is addressed in subsequent branches in the series of
+  which this one is a part.)
+
+- Added new function, ``zc.buildout.easy_install.sitepackage_safe_scripts``,
+  to generate scripts and interpreter.  It produces a full-featured
+  interpreter (all command-line options supported) and the ability to
+  safely let scripts include site packages, such as with a system
+  Python.  The ``z3c.recipe.scripts`` recipe uses this new function.
+
 - Improve bootstrap.
 
   * New options let you specify where to find ez_setup.py and where to find
@@ -23,6 +44,17 @@
   This means, among other things, that ``bin/buildout -vv`` and
   ``bin/buildout annotate`` correctly list more of the options.
 
+- Installing a namespace package using a Python that already has a package
+  in the same namespace (e.g., in the Python's site-packages) failed in
+  some cases.
+
+- Another variation of this error showed itself when at least two
+  dependencies were in a shared location like site-packages, and the
+  first one met the "versions" setting.  The first dependency would be
+  added, but subsequent dependencies from the same location (e.g.,
+  site-packages) would use the version of the package found in the
+  shared location, ignoring the version setting.
+
 1.4.3 (2009-12-10)
 ==================
 

=== modified file 'README.txt'
--- README.txt	2010-02-23 20:41:17 +0000
+++ README.txt	2010-02-23 20:41:17 +0000
@@ -35,7 +35,15 @@
 `zc.recipe.egg <http://pypi.python.org/pypi/zc.recipe.egg>`_
    The egg recipe installes one or more eggs, with their
    dependencies.  It installs their console-script entry points with
-   the needed eggs included in their paths.
+   the needed eggs included in their paths.  It is suitable for use with
+   a "clean" Python: one without packages installed in site-packages.
+
+`z3c.recipe.scripts <http://pypi.python.org/pypi/z3c.recipe.scripts>`_
+  Like zc.recipe.egg, this recipe builds interpreter scripts and entry
+  point scripts based on eggs.  It can be used with a Python that has
+  packages installed in site-packages, such as a system Python.  The
+  interpreter also has more features than the one offered by
+  zc.recipe.egg.
 
 `zc.recipe.testrunner <http://pypi.python.org/pypi/zc.recipe.testrunner>`_
    The testrunner egg creates a test runner script for one or

=== modified file 'buildout.cfg'
--- buildout.cfg	2009-10-23 08:25:24 +0000
+++ buildout.cfg	2010-02-23 20:41:17 +0000
@@ -1,5 +1,5 @@
 [buildout]
-develop = zc.recipe.egg_ .
+develop = zc.recipe.egg_ z3c.recipe.scripts_ .
 parts = test oltest py
 
 [py]
@@ -13,6 +13,7 @@
 eggs = 
   zc.buildout
   zc.recipe.egg
+  z3c.recipe.scripts
 
 # Tests that can be run wo a network
 [oltest]
@@ -20,6 +21,7 @@
 eggs = 
   zc.buildout
   zc.recipe.egg
+  z3c.recipe.scripts
 defaults =
   [
   '-t',

=== modified file 'setup.py'
--- setup.py	2009-12-10 16:19:55 +0000
+++ setup.py	2010-02-23 20:41:17 +0000
@@ -12,7 +12,7 @@
 #
 ##############################################################################
 name = "zc.buildout"
-version = "1.4.4dev"
+version = "1.5.0dev"
 
 import os
 from setuptools import setup

=== modified file 'src/zc/buildout/bootstrap.txt'
--- src/zc/buildout/bootstrap.txt	2010-02-23 20:41:17 +0000
+++ src/zc/buildout/bootstrap.txt	2010-02-23 20:41:17 +0000
@@ -232,8 +232,8 @@
     >>> print system(
     ...     zc.buildout.easy_install._safe_arg(sys.executable)+' '+
     ...     'bootstrap.py --help'),
-    ... # doctest: +ELLIPSIS
-    usage: [DESIRED PYTHON FOR BUILDOUT] bootstrap.py [options]
+    ... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE
+    Usage: [DESIRED PYTHON FOR BUILDOUT] bootstrap.py [options]
     <BLANKLINE>
     Bootstraps a buildout-based project.
     <BLANKLINE>
@@ -244,7 +244,7 @@
     local resources, you can keep this script from going over the network.
     <BLANKLINE>
     <BLANKLINE>
-    options:
+    Options:
       -h, --help            show this help message and exit
       -v VERSION, --version=VERSION
                             use a specific zc.buildout version

=== modified file 'src/zc/buildout/easy_install.py'
--- src/zc/buildout/easy_install.py	2010-02-23 20:41:17 +0000
+++ src/zc/buildout/easy_install.py	2010-02-23 20:41:17 +0000
@@ -60,12 +60,13 @@
     pkg_resources.Requirement.parse('setuptools')
     ).location
 
-# Include buildout and setuptools eggs in paths
-buildout_and_setuptools_path = [
-    setuptools_loc,
-    pkg_resources.working_set.find(
-        pkg_resources.Requirement.parse('zc.buildout')).location,
-    ]
+# Include buildout and setuptools eggs in paths.  We prevent dupes just to
+# keep from duplicating any log messages about them.
+buildout_loc = pkg_resources.working_set.find(
+    pkg_resources.Requirement.parse('zc.buildout')).location
+buildout_and_setuptools_path = [setuptools_loc]
+if os.path.normpath(setuptools_loc) != os.path.normpath(buildout_loc):
+    buildout_and_setuptools_path.append(buildout_loc)
 
 
 class IncompatibleVersionError(zc.buildout.UserError):
@@ -137,9 +138,73 @@
 else:
     _safe_arg = str
 
-_easy_install_cmd = _safe_arg(
-    'from setuptools.command.easy_install import main; main()'
-    )
+# The following string is used to run easy_install in
+# Installer._call_easy_install.  It is started with python -S (that is,
+# don't import site at start).  That flag, and all of the code in this
+# snippet above the last two lines, exist to work around a relatively rare
+# problem.  If
+#
+# - your buildout configuration is trying to install a package that is within
+#   a namespace package, and
+#
+# - you use a Python that has a different version of this package
+#   installed in in its site-packages using
+#   --single-version-externally-managed (that is, using the mechanism
+#   sometimes used by system packagers:
+#   http://peak.telecommunity.com/DevCenter/setuptools#install-command ), and
+#
+# - the new package tries to do sys.path tricks in the setup.py to get a
+#   __version__,
+#
+# then the older package will be loaded first, making the setup version
+# the wrong number. While very arguably packages simply shouldn't do
+# the sys.path tricks, some do, and we don't want buildout to fall over
+# when they do.
+#
+# The namespace packages installed in site-packages with
+# --single-version-externally-managed use a mechanism that cause them to
+# be processed when site.py is imported  (see
+# http://mail.python.org/pipermail/distutils-sig/2009-May/011730.html
+# for another description of the problem).  Simply starting Python with
+# -S addresses the problem in Python 2.4 and 2.5, but Python 2.6's
+# distutils imports a value from the site module, so we unfortunately
+# have to do more drastic surgery in the _easy_install_cmd code below.
+#
+# Here's an example of the .pth files created by setuptools when using that
+# flag:
+#
+# import sys,new,os;
+# p = os.path.join(sys._getframe(1).f_locals['sitedir'], *('<NAMESPACE>',));
+# ie = os.path.exists(os.path.join(p,'__init__.py'));
+# m = not ie and sys.modules.setdefault('<NAMESPACE>',new.module('<NAMESPACE>'));
+# mp = (m or []) and m.__dict__.setdefault('__path__',[]);
+# (p not in mp) and mp.append(p)
+#
+# The code, below, then, runs under -S, indicating that site.py should
+# not be loaded initially.  It gets the initial sys.path under these
+# circumstances, and then imports site (because Python 2.6's distutils
+# will want it, as mentioned above). It then reinstates the old sys.path
+# value. Then it removes namespace packages (created by the setuptools
+# code above) from sys.modules.  It identifies namespace packages by
+# iterating over every loaded module.  It first looks if there is a
+# __path__, so it is a package; and then it sees if that __path__ does
+# not have an __init__.py.  (Note that PEP 382,
+# http://www.python.org/dev/peps/pep-0382, makes it possible to have a
+# namespace package that has an __init__.py, but also should make it
+# unnecessary for site.py to preprocess these packages, so it should be
+# fine, as far as can be guessed as of this writing.)  Finally, it
+# imports easy_install and runs it.
+
+_easy_install_cmd = _safe_arg('''\
+import sys,os;\
+p = sys.path[:];\
+import site;\
+sys.path[:] = p;\
+[sys.modules.pop(k) for k, v in sys.modules.items()\
+ if hasattr(v, '__path__') and len(v.__path__)==1 and\
+ not os.path.exists(os.path.join(v.__path__[0],'__init__.py'))];\
+from setuptools.command.easy_install import main;\
+main()''')
 
 
 class Installer:
@@ -301,7 +366,7 @@
         try:
             path = setuptools_loc
 
-            args = ('-c', _easy_install_cmd, '-mUNxd', _safe_arg(tmp))
+            args = ('-Sc', _easy_install_cmd, '-mUNxd', _safe_arg(tmp))
             if self._always_unzip:
                 args += ('-Z', )
             level = logger.getEffectiveLevel()
@@ -904,6 +969,9 @@
 def working_set(specs, executable, path):
     return install(specs, None, executable=executable, path=path)
 
+############################################################################
+# Script generation functions
+
 def scripts(reqs, working_set, executable, dest,
             scripts=None,
             extra_paths=(),
@@ -912,20 +980,86 @@
             initialization='',
             relative_paths=False,
             ):
-
+    """Generate scripts and/or an interpreter.
+
+    See sitepackage_safe_scripts for a version that can be used with a Python
+    that has code installed in site-packages. It has more options and a
+    different approach.
+    """
+    path = _get_path(working_set, extra_paths)
+    if initialization:
+        initialization = '\n'+initialization+'\n'
+    generated = _generate_scripts(
+        reqs, working_set, dest, path, scripts, relative_paths,
+        initialization, executable, arguments)
+    if interpreter:
+        sname = os.path.join(dest, interpreter)
+        spath, rpsetup = _relative_path_and_setup(sname, path, relative_paths)
+        generated.extend(
+            _pyscript(spath, sname, executable, rpsetup))
+    return generated
+
+def sitepackage_safe_scripts(
+    dest, working_set, executable, site_py_dest,
+    reqs=(), scripts=None, interpreter=None, extra_paths=(),
+    initialization='', add_site_packages=False, exec_sitecustomize=False,
+    relative_paths=False, script_arguments='', script_initialization=''):
+    """Generate scripts and/or an interpreter from a system Python.
+
+    This accomplishes the same job as the ``scripts`` function, above,
+    but it does so in an alternative way that allows safely including
+    Python site packages, if desired, and  choosing to execute the Python's
+    sitecustomize.
+    """
+    generated = []
+    generated.append(_generate_sitecustomize(
+        site_py_dest, executable, initialization, exec_sitecustomize))
+    generated.append(_generate_site(
+        site_py_dest, working_set, executable, extra_paths,
+        add_site_packages, relative_paths))
+    script_initialization = (
+        '\nimport site # imports custom buildout-generated site.py\n%s' % (
+            script_initialization,))
+    if not script_initialization.endswith('\n'):
+        script_initialization += '\n'
+    generated.extend(_generate_scripts(
+        reqs, working_set, dest, [site_py_dest], scripts, relative_paths,
+        script_initialization, executable, script_arguments, block_site=True))
+    if interpreter:
+        generated.extend(_generate_interpreter(
+            interpreter, dest, executable, site_py_dest, relative_paths))
+    return generated
+
+# Utilities for the script generation functions.
+
+# These are shared by both ``scripts`` and ``sitepackage_safe_scripts``
+
+def _get_path(working_set, extra_paths=()):
+    """Given working set and extra paths, return a normalized path list."""
     path = [dist.location for dist in working_set]
     path.extend(extra_paths)
-    path = map(realpath, path)
-
-    generated = []
-
+    return map(realpath, path)
+
+def _generate_scripts(reqs, working_set, dest, path, scripts, relative_paths,
+                      initialization, executable, arguments,
+                      block_site=False):
+    """Generate scripts for the given requirements.
+
+    - reqs is an iterable of string requirements or entry points.
+    - The requirements must be findable in the given working_set.
+    - The dest is the directory in which the scripts should be created.
+    - The path is a list of paths that should be added to sys.path.
+    - The scripts is an optional dictionary.  If included, the keys should be
+      the names of the scripts that should be created, as identified in their
+      entry points; and the values should be the name the script should
+      actually be created with.
+    - relative_paths, if given, should be the path that is the root of the
+      buildout (the common path that should be the root of what is relative).
+    """
     if isinstance(reqs, str):
         raise TypeError('Expected iterable of requirements or entry points,'
                         ' got string.')
-
-    if initialization:
-        initialization = '\n'+initialization+'\n'
-
+    generated = []
     entry_points = []
     for req in reqs:
         if isinstance(req, str):
@@ -939,7 +1073,6 @@
                     )
         else:
             entry_points.append(req)
-
     for name, module_name, attrs in entry_points:
         if scripts is not None:
             sname = scripts.get(name)
@@ -947,40 +1080,51 @@
                 continue
         else:
             sname = name
-
         sname = os.path.join(dest, sname)
         spath, rpsetup = _relative_path_and_setup(sname, path, relative_paths)
-
         generated.extend(
-            _script(module_name, attrs, spath, sname, executable, arguments,
-                    initialization, rpsetup)
-            )
-
-    if interpreter:
-        sname = os.path.join(dest, interpreter)
-        spath, rpsetup = _relative_path_and_setup(sname, path, relative_paths)
-        generated.extend(_pyscript(spath, sname, executable, rpsetup))
-
+            _script(sname, executable, rpsetup, spath, initialization,
+                    module_name, attrs, arguments, block_site=block_site))
     return generated
 
-def _relative_path_and_setup(sname, path, relative_paths):
+def _relative_path_and_setup(sname, path,
+                             relative_paths=False, indent_level=1,
+                             omit_os_import=False):
+    """Return a string of code of paths and of setup if appropriate.
+
+    - sname is the full path to the script name to be created.
+    - path is the list of paths to be added to sys.path.
+    - relative_paths, if given, should be the path that is the root of the
+      buildout (the common path that should be the root of what is relative).
+    - indent_level is the number of four-space indents that the path should
+      insert before each element of the path.
+    """
     if relative_paths:
         relative_paths = os.path.normcase(relative_paths)
         sname = os.path.normcase(os.path.abspath(sname))
-        spath = ',\n  '.join(
+        spath = _format_paths(
             [_relativitize(os.path.normcase(path_item), sname, relative_paths)
-             for path_item in path]
-            )
+             for path_item in path], indent_level=indent_level)
         rpsetup = relative_paths_setup
+        if not omit_os_import:
+            rpsetup = '\n\nimport os\n' + rpsetup
         for i in range(_relative_depth(relative_paths, sname)):
-            rpsetup += "base = os.path.dirname(base)\n"
+            rpsetup += "\nbase = os.path.dirname(base)"
     else:
-        spath = repr(path)[1:-1].replace(', ', ',\n  ')
+        spath = _format_paths((repr(p) for p in path),
+                              indent_level=indent_level)
         rpsetup = ''
     return spath, rpsetup
 
-
 def _relative_depth(common, path):
+    """Return number of dirs separating ``path`` from ancestor, ``common``.
+
+    For instance, if path is /foo/bar/baz/bing, and common is /foo, this will
+    return 2--in UNIX, the number of ".." to get from bing's directory
+    to foo.
+
+    This is a helper for _relative_path_and_setup.
+    """
     n = 0
     while 1:
         dirname = os.path.dirname(path)
@@ -993,6 +1137,11 @@
     return n
 
 def _relative_path(common, path):
+    """Return the relative path from ``common`` to ``path``.
+
+    This is a helper for _relativitize, which is a helper to
+    _relative_path_and_setup.
+    """
     r = []
     while 1:
         dirname, basename = os.path.split(path)
@@ -1006,6 +1155,11 @@
     return os.path.join(*r)
 
 def _relativitize(path, script, relative_paths):
+    """Return a code string for the given path.
+
+    Path is relative to the base path ``relative_paths``if the common prefix
+    between ``path`` and ``script`` starts with ``relative_paths``.
+    """
     if path == script:
         raise AssertionError("path == script")
     common = os.path.dirname(os.path.commonprefix([path, script]))
@@ -1016,66 +1170,82 @@
     else:
         return repr(path)
 
-
 relative_paths_setup = """
-import os
-
 join = os.path.join
-base = os.path.dirname(os.path.abspath(os.path.realpath(__file__)))
-"""
-
-def _script(module_name, attrs, path, dest, executable, arguments,
-            initialization, rsetup):
+base = os.path.dirname(os.path.abspath(os.path.realpath(__file__)))"""
+
+def _write_script(full_name, contents, logged_type):
+    """Write contents of script in full_name, logging the action.
+
+    The only tricky bit in this function is that it supports Windows by
+    creating exe files using a pkg_resources helper.
+    """
     generated = []
-    script = dest
-    if is_win32:
-        dest += '-script.py'
-
-    contents = script_template % dict(
-        python = _safe_arg(executable),
-        path = path,
-        module_name = module_name,
-        attrs = attrs,
-        arguments = arguments,
-        initialization = initialization,
-        relative_paths_setup = rsetup,
-        )
-    changed = not (os.path.exists(dest) and open(dest).read() == contents)
-
-    if is_win32:
-        # generate exe file and give the script a magic name:
-        exe = script+'.exe'
+    script_name = full_name
+    if is_win32:
+        script_name += '-script.py'
+        # Generate exe file and give the script a magic name.
+        exe = full_name + '.exe'
         new_data = pkg_resources.resource_string('setuptools', 'cli.exe')
         if not os.path.exists(exe) or (open(exe, 'rb').read() != new_data):
             # Only write it if it's different.
             open(exe, 'wb').write(new_data)
         generated.append(exe)
-
+    changed = not (os.path.exists(script_name) and
+                   open(script_name).read() == contents)
     if changed:
-        open(dest, 'w').write(contents)
-        logger.info("Generated script %r.", script)
-
+        open(script_name, 'w').write(contents)
         try:
-            os.chmod(dest, 0755)
+            os.chmod(script_name, 0755)
         except (AttributeError, os.error):
             pass
-
-    generated.append(dest)
+        logger.info("Generated %s %r.", logged_type, full_name)
+    generated.append(script_name)
     return generated
 
+def _format_paths(paths, indent_level=1):
+    """Format paths for inclusion in a script."""
+    separator = ',\n' + indent_level * '    '
+    return separator.join(paths)
+
+def _script(dest, executable, relative_paths_setup, path, initialization,
+            module_name, attrs, arguments, block_site=False):
+    if block_site:
+        dash_S = ' -S'
+    else:
+        dash_S = ''
+    contents = script_template % dict(
+        python=_safe_arg(executable),
+        dash_S=dash_S,
+        path=path,
+        module_name=module_name,
+        attrs=attrs,
+        arguments=arguments,
+        initialization=initialization,
+        relative_paths_setup=relative_paths_setup,
+        )
+    return _write_script(dest, contents, 'script')
+
 if is_jython and jython_os_name == 'linux':
-    script_header = '#!/usr/bin/env %(python)s'
+    script_header = '#!/usr/bin/env %(python)s%(dash_S)s'
 else:
-    script_header = '#!%(python)s'
+    script_header = '#!%(python)s%(dash_S)s'
 
+sys_path_template = '''\
+import sys
+sys.path[0:0] = [
+    %s,
+    ]
+'''
 
 script_template = script_header + '''\
-
 %(relative_paths_setup)s
+
 import sys
 sys.path[0:0] = [
-  %(path)s,
-  ]
+    %(path)s,
+    ]
+
 %(initialization)s
 import %(module_name)s
 
@@ -1083,47 +1253,25 @@
     %(module_name)s.%(attrs)s(%(arguments)s)
 '''
 
+# These are used only by the older ``scripts`` function.
 
 def _pyscript(path, dest, executable, rsetup):
-    generated = []
-    script = dest
-    if is_win32:
-        dest += '-script.py'
-
     contents = py_script_template % dict(
-        python = _safe_arg(executable),
-        path = path,
-        relative_paths_setup = rsetup,
+        python=_safe_arg(executable),
+        dash_S='',
+        path=path,
+        relative_paths_setup=rsetup,
         )
-    changed = not (os.path.exists(dest) and open(dest).read() == contents)
-
-    if is_win32:
-        # generate exe file and give the script a magic name:
-        exe = script + '.exe'
-        open(exe, 'wb').write(
-            pkg_resources.resource_string('setuptools', 'cli.exe')
-            )
-        generated.append(exe)
-
-    if changed:
-        open(dest, 'w').write(contents)
-        try:
-            os.chmod(dest,0755)
-        except (AttributeError, os.error):
-            pass
-        logger.info("Generated interpreter %r.", script)
-
-    generated.append(dest)
-    return generated
+    return _write_script(dest, contents, 'interpreter')
 
 py_script_template = script_header + '''\
-
 %(relative_paths_setup)s
+
 import sys
 
 sys.path[0:0] = [
-  %(path)s,
-  ]
+    %(path)s,
+    ]
 
 _interactive = True
 if len(sys.argv) > 1:
@@ -1151,6 +1299,274 @@
     __import__("code").interact(banner="", local=globals())
 '''
 
+# These are used only by the newer ``sitepackage_safe_scripts`` function.
+
+def _get_system_paths(executable):
+    """Return lists of standard lib and site paths for executable.
+    """
+    # We want to get a list of the site packages, which is not easy.
+    # The canonical way to do this is to use
+    # distutils.sysconfig.get_python_lib(), but that only returns a
+    # single path, which does not reflect reality for many system
+    # Pythons, which have multiple additions.  Instead, we start Python
+    # with -S, which does not import site.py and set up the extra paths
+    # like site-packages or (Ubuntu/Debian) dist-packages and
+    # python-support. We then compare that sys.path with the normal one
+    # (minus user packages if this is Python 2.6, because we don't
+    # support those (yet?).  The set of the normal one minus the set of
+    # the ones in ``python -S`` is the set of packages that are
+    # effectively site-packages.
+    #
+    # The given executable might not be the current executable, so it is
+    # appropriate to do another subprocess to figure out what the
+    # additional site-package paths are. Moreover, even if this
+    # executable *is* the current executable, this code might be run in
+    # the context of code that has manipulated the sys.path--for
+    # instance, to add local zc.buildout or setuptools eggs.
+    def get_sys_path(*args, **kwargs):
+        cmd = [executable]
+        cmd.extend(args)
+        cmd.extend([
+            "-c", "import sys, os;"
+            "print repr([os.path.normpath(p) for p in sys.path if p])"])
+        # Windows needs some (as yet to be determined) part of the real env.
+        env = os.environ.copy()
+        env.update(kwargs)
+        _proc = subprocess.Popen(
+            cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, env=env)
+        stdout, stderr = _proc.communicate();
+        if _proc.returncode:
+            raise RuntimeError(
+                'error trying to get system packages:\n%s' % (stderr,))
+        res = eval(stdout.strip())
+        try:
+            res.remove('.')
+        except ValueError:
+            pass
+        return res
+    stdlib = get_sys_path('-S') # stdlib only
+    no_user_paths = get_sys_path(PYTHONNOUSERSITE='x')
+    site_paths = [p for p in no_user_paths if p not in stdlib]
+    return (stdlib, site_paths)
+
+def _get_module_file(executable, name):
+    """Return a module's file path.
+
+    - executable is a path to the desired Python executable.
+    - name is the name of the (pure, not C) Python module.
+    """
+    cmd = [executable, "-c",
+           "import imp; "
+           "fp, path, desc = imp.find_module(%r); "
+           "fp.close; "
+           "print path" % (name,)]
+    _proc = subprocess.Popen(
+        cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+    stdout, stderr = _proc.communicate();
+    if _proc.returncode:
+        logger.info(
+            'Could not find file for module %s:\n%s', name, stderr)
+        return None
+    # else: ...
+    res = stdout.strip()
+    if res.endswith('.pyc') or res.endswith('.pyo'):
+        raise RuntimeError('Cannot find uncompiled version of %s' % (name,))
+    if not os.path.exists(res):
+        raise RuntimeError(
+            'File does not exist for module %s:\n%s' % (name, res))
+    return res
+
+def _generate_sitecustomize(dest, executable, initialization='',
+                            exec_sitecustomize=False):
+    """Write a sitecustomize file with optional custom initialization.
+
+    The created script will execute the underlying Python's
+    sitecustomize if exec_sitecustomize is True.
+    """
+    sitecustomize_path = os.path.join(dest, 'sitecustomize.py')
+    sitecustomize = open(sitecustomize_path, 'w')
+    if initialization:
+        sitecustomize.write(initialization + '\n')
+    if exec_sitecustomize:
+        real_sitecustomize_path = _get_module_file(
+            executable, 'sitecustomize')
+        if real_sitecustomize_path:
+            real_sitecustomize = open(real_sitecustomize_path, 'r')
+            sitecustomize.write(
+                '\n# The following is from\n# %s\n' %
+                (real_sitecustomize_path,))
+            sitecustomize.write(real_sitecustomize.read())
+            real_sitecustomize.close()
+    sitecustomize.close()
+    return sitecustomize_path
+
+def _generate_site(dest, working_set, executable, extra_paths=(),
+                   add_site_packages=False, relative_paths=False):
+    """Write a site.py file with eggs from working_set.
+
+    extra_paths will be added to the path.  If add_site_packages is True,
+    paths from the underlying Python will be added.
+    """
+    path = _get_path(working_set, extra_paths)
+    site_path = os.path.join(dest, 'site.py')
+    egg_path_string, preamble = _relative_path_and_setup(
+        site_path, path, relative_paths, indent_level=2, omit_os_import=True)
+    if preamble:
+        preamble = '\n'.join(
+            [(line and '    %s' % (line,) or line)
+             for line in preamble.split('\n')])
+    original_path_setup = ''
+    if add_site_packages:
+        stdlib, site_paths = _get_system_paths(executable)
+        original_path_setup = original_path_snippet % (
+            _format_paths((repr(p) for p in site_paths), 2),)
+        distribution = working_set.find(
+            pkg_resources.Requirement.parse('setuptools'))
+        if distribution is not None:
+            # We need to worry about namespace packages.
+            if relative_paths:
+                location = _relativitize(
+                    distribution.location,
+                    os.path.normcase(os.path.abspath(site_path)),
+                    relative_paths)
+            else:
+                location = repr(distribution.location)
+            preamble += namespace_add_site_packages_setup % (location,)
+            original_path_setup = (
+                addsitedir_namespace_originalpackages_snippet +
+                original_path_setup)
+    addsitepackages_marker = 'def addsitepackages('
+    enableusersite_marker = 'ENABLE_USER_SITE = '
+    successful_rewrite = False
+    real_site_path = _get_module_file(executable, 'site')
+    real_site = open(real_site_path, 'r')
+    site = open(site_path, 'w')
+    try:
+        for line in real_site.readlines():
+            if line.startswith(enableusersite_marker):
+                site.write(enableusersite_marker)
+                site.write('False # buildout does not support user sites.\n')
+            elif line.startswith(addsitepackages_marker):
+                site.write(addsitepackages_script % (
+                    preamble, egg_path_string, original_path_setup))
+                site.write(line[len(addsitepackages_marker):])
+                successful_rewrite = True
+            else:
+                site.write(line)
+    finally:
+        site.close()
+        real_site.close()
+    if not successful_rewrite:
+        raise RuntimeError('Buildout did not successfully rewrite site.py')
+    return site_path
+
+namespace_add_site_packages_setup = '''
+    setuptools_path = %s
+    sys.path.append(setuptools_path)
+    known_paths.add(os.path.normcase(setuptools_path))
+    import pkg_resources'''
+
+addsitedir_namespace_originalpackages_snippet = '''
+            pkg_resources.working_set.add_entry(sitedir)'''
+
+original_path_snippet = '''
+    original_paths = [
+        %s
+        ]
+    for path in original_paths:
+        addsitedir(path, known_paths)'''
+
+addsitepackages_script = '''\
+def addsitepackages(known_paths):
+    """Add site packages, as determined by zc.buildout.
+
+    See original_addsitepackages, below, for the original version."""%s
+    buildout_paths = [
+        %s
+        ]
+    for path in buildout_paths:
+        sitedir, sitedircase = makepath(path)
+        if not sitedircase in known_paths and os.path.exists(sitedir):
+            sys.path.append(sitedir)
+            known_paths.add(sitedircase)%s
+    return known_paths
+
+def original_addsitepackages('''
+
+def _generate_interpreter(name, dest, executable, site_py_dest,
+                          relative_paths=False):
+    """Write an interpreter script, using the site.py approach."""
+    full_name = os.path.join(dest, name)
+    site_py_dest_string, rpsetup = _relative_path_and_setup(
+        full_name, [site_py_dest], relative_paths, omit_os_import=True)
+    if rpsetup:
+        rpsetup += "\n"
+    if sys.platform == 'win32':
+        windows_import = '\nimport subprocess'
+        # os.exec* is a mess on Windows, particularly if the path
+        # to the executable has spaces and the Python is using MSVCRT.
+        # The standard fix is to surround the executable's path with quotes,
+        # but that has been unreliable in testing.
+        #
+        # Here's a demonstration of the problem.  Given a Python
+        # compiled with a MSVCRT-based compiler, such as the free Visual
+        # C++ 2008 Express Edition, and an executable path with spaces
+        # in it such as the below, we see the following.
+        #
+        # >>> import os
+        # >>> p0 = 'C:\\Documents and Settings\\Administrator\\My Documents\\Downloads\\Python-2.6.4\\PCbuild\\python.exe'
+        # >>> os.path.exists(p0)
+        # True
+        # >>> os.execv(p0, [])
+        # Traceback (most recent call last):
+        #  File "<stdin>", line 1, in <module>
+        # OSError: [Errno 22] Invalid argument
+        #
+        # That seems like a standard problem.  The standard solution is
+        # to quote the path (see, for instance
+        # http://bugs.python.org/issue436259).  However, this solution,
+        # and other variations, fail:
+        #
+        # >>> p1 = '"C:\\Documents and Settings\\Administrator\\My Documents\\Downloads\\Python-2.6.4\\PCbuild\\python.exe"'
+        # >>> os.execv(p1, [])
+        # Traceback (most recent call last):
+        #   File "<stdin>", line 1, in <module>
+        # OSError: [Errno 22] Invalid argument
+        #
+        # We simply use subprocess instead, since it handles everything
+        # nicely, and the transparency of exec* (that is, not running,
+        # perhaps unexpectedly, in a subprocess) is arguably not a
+        # necessity, at least for many use cases.
+        execute = 'subprocess.call(argv, env=environ)'
+    else:
+        windows_import = ''
+        execute = 'os.execve(sys.executable, argv, environ)'
+    contents = interpreter_template % dict(
+        python=_safe_arg(executable),
+        dash_S=' -S',
+        site_dest=site_py_dest_string,
+        relative_paths_setup=rpsetup,
+        windows_import=windows_import,
+        execute=execute,
+        )
+    return _write_script(full_name, contents, 'interpreter')
+
+interpreter_template = script_header + '''
+import os
+import sys%(windows_import)s
+%(relative_paths_setup)s
+argv = [sys.executable] + sys.argv[1:]
+environ = os.environ.copy()
+path = %(site_dest)s
+if environ.get('PYTHONPATH'):
+    path = os.pathsep.join([path, environ['PYTHONPATH']])
+environ['PYTHONPATH'] = path
+%(execute)s
+'''
+
+# End of script generation code.
+############################################################################
+
 runsetup_template = """
 import sys
 sys.path.insert(0, %(setupdir)r)

=== modified file 'src/zc/buildout/easy_install.txt'
--- src/zc/buildout/easy_install.txt	2009-11-11 21:21:11 +0000
+++ src/zc/buildout/easy_install.txt	2010-02-23 20:41:17 +0000
@@ -521,25 +521,38 @@
 Script generation
 -----------------
 
-The easy_install module provides support for creating scripts from
-eggs.  It provides a function similar to setuptools except that it
-provides facilities for baking a script's path into the script.  This
-has two advantages:
+The easy_install module provides support for creating scripts from eggs.
+It provides two competing functions.  One, ``scripts``, is a
+well-established approach to generating reliable scripts with a "clean"
+Python--e.g., one that does not have any packages in its site-packages.
+The other, ``sitepackage_safe_scripts``, is newer, a bit trickier, and is
+designed to work with a Python that has code in its site-packages, such
+as a system Python.
+
+Both are similar to setuptools except that they provides facilities for
+baking a script's path into the script.  This has two advantages:
 
 - The eggs to be used by a script are not chosen at run time, making
   startup faster and, more importantly, deterministic.
 
-- The script doesn't have to import pkg_resources because the logic
-  that pkg_resources would execute at run time is executed at
-  script-creation time.
-
-The scripts method can be used to generate scripts. Let's create a
-destination directory for it to place them in:
-
-    >>> import tempfile
+- The script doesn't have to import pkg_resources because the logic that
+  pkg_resources would execute at run time is executed at script-creation
+  time.  (There is an exception in ``sitepackage_safe_scripts`` if you
+  want to have your Python's site packages available, as discussed
+  below, but even in that case pkg_resources is only partially
+  activated, which can be a significant time savings.)
+
+
+The ``scripts`` function
+~~~~~~~~~~~~~~~~~~~~~~~~
+
+The ``scripts`` function is the first way to generate scripts that we'll
+examine. It is the earlier approach that the package offered.  Let's
+create a destination directory for it to place them in:
+
     >>> bin = tmpdir('bin')
 
-Now, we'll use the scripts method to generate scripts in this directory
+Now, we'll use the scripts function to generate scripts in this directory
 from the demo egg:
 
     >>> import sys
@@ -736,8 +749,8 @@
     >>> print system(os.path.join(bin, 'run')),
     3 1
 
-Including extra paths in scripts
---------------------------------
+The ``scripts`` function: Including extra paths in scripts
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 We can pass a keyword argument, extra paths, to cause additional paths
 to be included in the a generated script:
@@ -762,8 +775,8 @@
     if __name__ == '__main__':
         eggrecipedemo.main()
 
-Providing script arguments
---------------------------
+The ``scripts`` function: Providing script arguments
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 An "argument" keyword argument can be used to pass arguments to an
 entry point.  The value passed is a source string to be placed between the
@@ -786,8 +799,8 @@
     if __name__ == '__main__':
         eggrecipedemo.main(1, 2)
 
-Passing initialization code
----------------------------
+The ``scripts`` function: Passing initialization code
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 You can also pass script initialization code:
 
@@ -812,8 +825,8 @@
     if __name__ == '__main__':
         eggrecipedemo.main(1, 2)
 
-Relative paths
---------------
+The ``scripts`` function: Relative paths
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 Sometimes, you want to be able to move a buildout directory around and
 have scripts still work without having to rebuild them.  We can
@@ -836,7 +849,7 @@
     ...    interpreter='py',
     ...    relative_paths=bo)
 
-    >>> cat(bo, 'bin', 'run')
+    >>> cat(bo, 'bin', 'run') # doctest: +NORMALIZE_WHITESPACE
     #!/usr/local/bin/python2.4
     <BLANKLINE>
     import os
@@ -868,7 +881,7 @@
 
 We specified an interpreter and its paths are adjusted too:
 
-    >>> cat(bo, 'bin', 'py')
+    >>> cat(bo, 'bin', 'py') # doctest: +NORMALIZE_WHITESPACE
     #!/usr/local/bin/python2.4
     <BLANKLINE>
     import os
@@ -911,6 +924,557 @@
         del _interactive
         __import__("code").interact(banner="", local=globals())
 
+The ``sitepackage_safe_scripts`` function
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The newer function for creating scripts is ``sitepackage_safe_scripts``.
+ It has the same basic functionality as the ``scripts`` function: it can
+create scripts to run arbitrary entry points, and to run a Python
+interpreter.  The following are the differences from a user's
+perspective.
+
+- It can be used safely with a Python that has packages installed itself,
+  such as a system-installed Python.
+
+- In contrast to the interpreter generated by the ``scripts`` method, which
+  supports only a small subset of the usual Python executable's options,
+  the interpreter generated by ``sitepackage_safe_scripts`` supports all
+  of them. This makes it possible to use as full Python replacement for
+  scripts that need the distributions specified in your buildout.
+
+- Both the interpreter and the entry point scripts allow you to include the
+  site packages, and/or the sitecustomize, of the Python executable, if
+  desired.
+
+It works by creating site.py and sitecustomize.py files that set up the
+desired paths and initialization.  These must be placed within an otherwise
+empty directory.  Typically this is in a recipe's parts directory.
+
+Here's the simplest example, building an interpreter script.
+
+    >>> interpreter_dir = tmpdir('interpreter')
+    >>> interpreter_parts_dir = os.path.join(
+    ...     interpreter_dir, 'parts', 'interpreter')
+    >>> interpreter_bin_dir = os.path.join(interpreter_dir, 'bin')
+    >>> mkdir(interpreter_bin_dir)
+    >>> mkdir(interpreter_dir, 'eggs')
+    >>> mkdir(interpreter_dir, 'parts')
+    >>> mkdir(interpreter_parts_dir)
+
+    >>> ws = zc.buildout.easy_install.install(
+    ...     ['demo'], join(interpreter_dir, 'eggs'), links=[link_server],
+    ...     index=link_server+'index/')
+    >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts(
+    ...     interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir,
+    ...     interpreter='py')
+
+Depending on whether the machine being used is running Windows or not, this
+produces either three or four files.  In both cases, we have site.py and
+sitecustomize.py generated in the parts/interpreter directory.  For Windows,
+we have py.exe and py-script.py; for other operating systems, we have py.
+
+    >>> sitecustomize_path = os.path.join(
+    ...     interpreter_parts_dir, 'sitecustomize.py')
+    >>> site_path = os.path.join(interpreter_parts_dir, 'site.py')
+    >>> interpreter_path = os.path.join(interpreter_bin_dir, 'py')
+    >>> if sys.platform == 'win32':
+    ...     py_path = os.path.join(interpreter_bin_dir, 'py-script.py')
+    ...     expected = [sitecustomize_path,
+    ...                 site_path,
+    ...                 os.path.join(interpreter_bin_dir, 'py.exe'),
+    ...                 py_path]
+    ... else:
+    ...     py_path = interpreter_path
+    ...     expected = [sitecustomize_path, site_path, py_path]
+    ...
+    >>> assert generated == expected, repr((generated, expected))
+
+We didn't ask for any initialization, and we didn't ask to use the underlying
+sitecustomization, so sitecustomize.py is empty.
+
+    >>> cat(sitecustomize_path)
+
+The interpreter script is simple.  It puts the directory with the
+site.py and sitecustomize.py on the PYTHONPATH and (re)starts Python.
+
+    >>> cat(py_path)
+    #!/usr/bin/python -S
+    import os
+    import sys
+    <BLANKLINE>
+    argv = [sys.executable] + sys.argv[1:]
+    environ = os.environ.copy()
+    path = '/interpreter/parts/interpreter'
+    if environ.get('PYTHONPATH'):
+        path = os.pathsep.join([path, environ['PYTHONPATH']])
+    environ['PYTHONPATH'] = path
+    os.execve(sys.executable, argv, environ)
+
+The site.py file is a modified version of the underlying Python's site.py.
+The most important modification is that it has a different version of the
+addsitepackages function.  It sets up the Python path, similarly to the
+behavior of the function it replaces.  The following shows the part that
+buildout inserts, in the simplest case.
+
+    >>> sys.stdout.write('#\n'); cat(site_path)
+    ... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE
+    #...
+    def addsitepackages(known_paths):
+        """Add site packages, as determined by zc.buildout.
+    <BLANKLINE>
+        See original_addsitepackages, below, for the original version."""
+        buildout_paths = [
+            '/interpreter/eggs/demo-0.3-pyN.N.egg',
+            '/interpreter/eggs/demoneeded-1.1-pyN.N.egg'
+            ]
+        for path in buildout_paths:
+            sitedir, sitedircase = makepath(path)
+            if not sitedircase in known_paths and os.path.exists(sitedir):
+                sys.path.append(sitedir)
+                known_paths.add(sitedircase)
+        return known_paths
+    <BLANKLINE>
+    def original_addsitepackages(known_paths):...
+
+Here are some examples of the interpreter in use.
+
+    >>> print call_py(interpreter_path, "print 16+26")
+    42
+    <BLANKLINE>
+    >>> res = call_py(interpreter_path, "import sys; print sys.path")
+    >>> print res # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE
+    ['',
+     '/interpreter/parts/interpreter',
+     ...,
+     '/interpreter/eggs/demo-0.3-pyN.N.egg',
+     '/interpreter/eggs/demoneeded-1.1-pyN.N.egg']
+    <BLANKLINE>
+    >>> clean_paths = eval(res.strip()) # This is used later for comparison.
+
+If you provide initialization, it goes in sitecustomize.py.
+
+    >>> def reset_interpreter():
+    ...     # This is necessary because, in our tests, the timestamps of the
+    ...     # .pyc files are not outdated when we want them to be.
+    ...     rmdir(interpreter_bin_dir)
+    ...     mkdir(interpreter_bin_dir)
+    ...     rmdir(interpreter_parts_dir)
+    ...     mkdir(interpreter_parts_dir)
+    ...
+    >>> reset_interpreter()
+
+    >>> initialization_string = """\
+    ... import os
+    ... os.environ['FOO'] = 'bar baz bing shazam'"""
+    >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts(
+    ...     interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir,
+    ...     interpreter='py', initialization=initialization_string)
+    >>> cat(sitecustomize_path)
+    import os
+    os.environ['FOO'] = 'bar baz bing shazam'
+    >>> print call_py(interpreter_path, "import os; print os.environ['FOO']")
+    bar baz bing shazam
+    <BLANKLINE>
+
+If you use relative paths, this affects the interpreter and site.py.  (This is
+again the UNIX version; the Windows version uses subprocess instead of
+os.execve.)
+
+    >>> reset_interpreter()
+    >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts(
+    ...     interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir,
+    ...     interpreter='py', relative_paths=interpreter_dir)
+    >>> cat(py_path)
+    #!/usr/bin/python -S
+    import os
+    import sys
+    <BLANKLINE>
+    join = os.path.join
+    base = os.path.dirname(os.path.abspath(os.path.realpath(__file__)))
+    base = os.path.dirname(base)
+    <BLANKLINE>
+    argv = [sys.executable] + sys.argv[1:]
+    environ = os.environ.copy()
+    path = join(base, 'parts/interpreter')
+    if environ.get('PYTHONPATH'):
+        path = os.pathsep.join([path, environ['PYTHONPATH']])
+    environ['PYTHONPATH'] = path
+    os.execve(sys.executable, argv, environ)
+
+For site.py, we again show only the pertinent parts.  Notice that the egg
+paths join a base to a path, as with the use of this argument in the
+``scripts`` function.
+
+    >>> sys.stdout.write('#\n'); cat(site_path) # doctest: +ELLIPSIS
+    #...
+    def addsitepackages(known_paths):
+        """Add site packages, as determined by zc.buildout.
+    <BLANKLINE>
+        See original_addsitepackages, below, for the original version."""
+        join = os.path.join
+        base = os.path.dirname(os.path.abspath(os.path.realpath(__file__)))
+        base = os.path.dirname(base)
+        base = os.path.dirname(base)
+        buildout_paths = [
+            join(base, 'eggs/demo-0.3-pyN.N.egg'),
+            join(base, 'eggs/demoneeded-1.1-pyN.N.egg')
+            ]...
+
+The paths resolve in practice as you would expect.
+
+    >>> print call_py(interpreter_path,
+    ...               "import sys, pprint; pprint.pprint(sys.path)")
+    ... # doctest: +ELLIPSIS
+    ['',
+     '/interpreter/parts/interpreter',
+     ...,
+     '/interpreter/eggs/demo-0.3-pyN.N.egg',
+     '/interpreter/eggs/demoneeded-1.1-pyN.N.egg']
+    <BLANKLINE>
+
+The ``extra_paths`` argument affects the path in site.py.  Notice that
+/interpreter/other is added after the eggs.
+
+    >>> reset_interpreter()
+    >>> mkdir(interpreter_dir, 'other')
+    >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts(
+    ...     interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir,
+    ...     interpreter='py', extra_paths=[join(interpreter_dir, 'other')])
+    >>> sys.stdout.write('#\n'); cat(site_path) # doctest: +ELLIPSIS
+    #...
+    def addsitepackages(known_paths):
+        """Add site packages, as determined by zc.buildout.
+    <BLANKLINE>
+        See original_addsitepackages, below, for the original version."""
+        buildout_paths = [
+            '/interpreter/eggs/demo-0.3-pyN.N.egg',
+            '/interpreter/eggs/demoneeded-1.1-pyN.N.egg',
+            '/interpreter/other'
+            ]...
+
+    >>> print call_py(interpreter_path,
+    ...               "import sys, pprint; pprint.pprint(sys.path)")
+    ... # doctest: +ELLIPSIS
+    ['',
+     '/interpreter/parts/interpreter',
+     ...,
+     '/interpreter/eggs/demo-0.3-pyN.N.egg',
+     '/interpreter/eggs/demoneeded-1.1-pyN.N.egg',
+     '/interpreter/other']
+    <BLANKLINE>
+
+The ``sitepackage_safe_scripts`` function: using site-packages
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The ``sitepackage_safe_scripts`` function supports including site
+packages.  This has some advantages and some serious dangers.
+
+A typical reason to include site-packages is that it is easier to
+install one or more dependencies in your Python than it is with
+buildout.  Some packages, such as lxml or Python PostgreSQL integration,
+have dependencies that can be much easier to build and/or install using
+other mechanisms, such as your operating system's package manager.  By
+installing some core packages into your Python's site-packages, this can
+significantly simplify some application installations.
+
+However, doing this has a significant danger.  One of the primary goals
+of buildout is to provide repeatability.  Some packages (one of the
+better known Python openid packages, for instance) change their behavior
+depending on what packages are available.  If Python curl bindings are
+available, these may be preferred by the library.  If a certain XML
+package is installed, it may be preferred by the library.  These hidden
+choices may cause small or large behavior differences.  The fact that
+they can be rarely encountered can actually make it worse: you forget
+that this might be a problem, and debugging the differences can be
+difficult.  If you allow site-packages to be included in your buildout,
+and the Python you use is not managed precisely by your application (for
+instance, it is a system Python), you open yourself up to these
+possibilities.  Don't be unaware of the dangers.
+
+That explained, let's see how it works.  If you don't use namespace packages,
+this is very straightforward.
+
+    >>> reset_interpreter()
+    >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts(
+    ...     interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir,
+    ...     interpreter='py', add_site_packages=True)
+    >>> sys.stdout.write('#\n'); cat(site_path)
+    ... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE
+    #...
+    def addsitepackages(known_paths):
+        """Add site packages, as determined by zc.buildout.
+    <BLANKLINE>
+        See original_addsitepackages, below, for the original version."""
+        buildout_paths = [
+            '/interpreter/eggs/demo-0.3-pyN.N.egg',
+            '/interpreter/eggs/demoneeded-1.1-pyN.N.egg'
+            ]
+        for path in buildout_paths:
+            sitedir, sitedircase = makepath(path)
+            if not sitedircase in known_paths and os.path.exists(sitedir):
+                sys.path.append(sitedir)
+                known_paths.add(sitedircase)
+        original_paths = [
+            ...
+            ]
+        for path in original_paths:
+            addsitedir(path, known_paths)
+        return known_paths
+    <BLANKLINE>
+    def original_addsitepackages(known_paths):...
+
+It simply adds the original paths using addsitedir after the code to add the
+buildout paths.
+
+Here's an example of the new script in use.  Other documents and tests in
+this package give the feature a more thorough workout, but this should
+give you an idea of the feature.
+
+    >>> res = call_py(interpreter_path, "import sys; print sys.path")
+    >>> print res # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE
+    ['',
+     '/interpreter/parts/interpreter',
+     ...,
+     '/interpreter/eggs/demo-0.3-py2.4.egg',
+     '/interpreter/eggs/demoneeded-1.1-py2.4.egg',
+     ...]
+    <BLANKLINE>
+
+The clean_paths gathered earlier is a subset of this full list of paths.
+
+    >>> full_paths = eval(res.strip())
+    >>> len(clean_paths) < len(full_paths)
+    True
+    >>> set(os.path.normpath(p) for p in clean_paths).issubset(
+    ...     os.path.normpath(p) for p in full_paths)
+    True
+
+Unfortunately, because of how setuptools namespace packages are implemented
+differently for operating system packages (debs or rpms) as opposed to
+standard setuptools installation, there's a slightly trickier dance if you
+use them.  To show this we'll needs some extra eggs that use namespaces.
+We'll use the ``tellmy.fortune`` package, which we'll need to make an initial
+call to another text fixture to create.
+
+    >>> from zc.buildout.tests import create_sample_namespace_eggs
+    >>> namespace_eggs = tmpdir('namespace_eggs')
+    >>> create_sample_namespace_eggs(namespace_eggs)
+
+    >>> reset_interpreter()
+    >>> ws = zc.buildout.easy_install.install(
+    ...     ['demo', 'tellmy.fortune'], join(interpreter_dir, 'eggs'),
+    ...     links=[link_server, namespace_eggs], index=link_server+'index/')
+    >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts(
+    ...     interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir,
+    ...     interpreter='py', add_site_packages=True)
+    >>> sys.stdout.write('#\n'); cat(site_path)
+    ... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE
+    #...
+    def addsitepackages(known_paths):
+        """Add site packages, as determined by zc.buildout.
+    <BLANKLINE>
+        See original_addsitepackages, below, for the original version."""
+        setuptools_path = '...setuptools...'
+        sys.path.append(setuptools_path)
+        known_paths.add(os.path.normcase(setuptools_path))
+        import pkg_resources
+        buildout_paths = [
+            '/interpreter/eggs/demo-0.3-pyN.N.egg',
+            '/interpreter/eggs/tellmy.fortune-1.0-pyN.N.egg',
+            '...setuptools...',
+            '/interpreter/eggs/demoneeded-1.1-pyN.N.egg'
+            ]
+        for path in buildout_paths:
+            sitedir, sitedircase = makepath(path)
+            if not sitedircase in known_paths and os.path.exists(sitedir):
+                sys.path.append(sitedir)
+                known_paths.add(sitedircase)
+                pkg_resources.working_set.add_entry(sitedir)
+        original_paths = [
+            ...
+            ]
+        for path in original_paths:
+            addsitedir(path, known_paths)
+        return known_paths
+    <BLANKLINE>
+    def original_addsitepackages(known_paths):...
+
+    >>> print call_py(interpreter_path, "import sys; print sys.path")
+    ... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE
+    ['',
+     '/interpreter/parts/interpreter',
+     ...,
+     '...setuptools...',
+     '/interpreter/eggs/demo-0.3-pyN.N.egg',
+     '/interpreter/eggs/tellmy.fortune-1.0-pyN.N.egg',
+     '/interpreter/eggs/demoneeded-1.1-pyN.N.egg',
+     ...]
+
+As you can see, the script now first imports pkg_resources.  Then we
+need to process egg files specially to look for namespace packages there
+*before* we process process lines in .pth files that use the "import"
+feature--lines that might be part of the setuptools namespace package
+implementation for system packages, as mentioned above, and that must
+come after processing egg namespaces.
+
+The most complex that this function gets is if you use namespace packages,
+include site-packages, and use relative paths.  For completeness, we'll look
+at that result.
+
+    >>> reset_interpreter()
+    >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts(
+    ...     interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir,
+    ...     interpreter='py', add_site_packages=True,
+    ...     relative_paths=interpreter_dir)
+    >>> sys.stdout.write('#\n'); cat(site_path)
+    ... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE
+    #...
+    def addsitepackages(known_paths):
+        """Add site packages, as determined by zc.buildout.
+    <BLANKLINE>
+        See original_addsitepackages, below, for the original version."""
+        join = os.path.join
+        base = os.path.dirname(os.path.abspath(os.path.realpath(__file__)))
+        base = os.path.dirname(base)
+        base = os.path.dirname(base)
+        setuptools_path = '...setuptools...'
+        sys.path.append(setuptools_path)
+        known_paths.add(os.path.normcase(setuptools_path))
+        import pkg_resources
+        buildout_paths = [
+            join(base, 'eggs/demo-0.3-pyN.N.egg'),
+            join(base, 'eggs/tellmy.fortune-1.0-pyN.N.egg'),
+            '...setuptools...',
+            join(base, 'eggs/demoneeded-1.1-pyN.N.egg')
+            ]
+        for path in buildout_paths:
+            sitedir, sitedircase = makepath(path)
+            if not sitedircase in known_paths and os.path.exists(sitedir):
+                sys.path.append(sitedir)
+                known_paths.add(sitedircase)
+                pkg_resources.working_set.add_entry(sitedir)
+        original_paths = [
+            ...
+            ]
+        for path in original_paths:
+            addsitedir(path, known_paths)
+        return known_paths
+    <BLANKLINE>
+    def original_addsitepackages(known_paths):...
+
+    >>> print call_py(interpreter_path, "import sys; print sys.path")
+    ... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE
+    ['',
+     '/interpreter/parts/interpreter',
+     ...,
+     '...setuptools...',
+     '/interpreter/eggs/demo-0.3-pyN.N.egg',
+     '/interpreter/eggs/tellmy.fortune-1.0-pyN.N.egg',
+     '/interpreter/eggs/demoneeded-1.1-pyN.N.egg',
+     ...]
+
+The ``exec_sitecustomize`` argument does the same thing for the
+sitecustomize module--it allows you to include the code from the
+sitecustomize module in the underlying Python if you set the argument to
+True.  The z3c.recipe.scripts package sets up the full environment necessary
+to demonstrate this piece.
+
+The ``sitepackage_safe_scripts`` function: writing scripts for entry points
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+All of the examples so far for this function have been creating
+interpreters.  The function can also write scripts for entry
+points.  They are almost identical to the scripts that we saw for the
+``scripts`` function except that they ``import site`` after setting the
+sys.path to include our custom site.py and sitecustomize.py files.  These
+files then initialize the Python environment as we have already seen.  Let's
+see a simple example.
+
+    >>> reset_interpreter()
+    >>> ws = zc.buildout.easy_install.install(
+    ...     ['demo'], join(interpreter_dir, 'eggs'), links=[link_server],
+    ...     index=link_server+'index/')
+    >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts(
+    ...     interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir,
+    ...     reqs=['demo'])
+
+As before, in Windows, 2 files are generated for each script.  A script
+file, ending in '-script.py', and an exe file that allows the script
+to be invoked directly without having to specify the Python
+interpreter and without having to provide a '.py' suffix.  This is in addition
+to the site.py and sitecustomize.py files that are generated as with our
+interpreter examples above.
+
+    >>> if sys.platform == 'win32':
+    ...     demo_path = os.path.join(interpreter_bin_dir, 'demo-script.py')
+    ...     expected = [sitecustomize_path,
+    ...                 site_path,
+    ...                 os.path.join(interpreter_bin_dir, 'demo.exe'),
+    ...                 demo_path]
+    ... else:
+    ...     demo_path = os.path.join(interpreter_bin_dir, 'demo')
+    ...     expected = [sitecustomize_path, site_path, demo_path]
+    ...
+    >>> assert generated == expected, repr((generated, expected))
+
+The demo script runs the entry point defined in the demo egg:
+
+    >>> cat(demo_path) # doctest: +NORMALIZE_WHITESPACE
+    #!/usr/local/bin/python2.4 -S
+    <BLANKLINE>
+    import sys
+    sys.path[0:0] = [
+        '/interpreter/parts/interpreter',
+        ]
+    <BLANKLINE>
+    <BLANKLINE>
+    import site # imports custom buildout-generated site.py
+    <BLANKLINE>
+    import eggrecipedemo
+    <BLANKLINE>
+    if __name__ == '__main__':
+        eggrecipedemo.main()
+
+    >>> demo_call = join(interpreter_bin_dir, 'demo')
+    >>> if sys.platform == 'win32':
+    ...     demo_call = '"%s"' % demo_call
+    >>> print system(demo_call)
+    3 1
+    <BLANKLINE>
+
+There are a few differences from the ``scripts`` function.  First, the
+``reqs`` argument (an iterable of string requirements or entry point
+tuples) is a keyword argument here.  We see that in the example above.
+Second, the ``arguments`` argument is now named ``script_arguments`` to
+try and clarify that it does not affect interpreters. While the
+``initialization`` argument continues to affect both the interpreters
+and the entry point scripts, if you have initialization that is only
+pertinent to the entry point scripts, you can use the
+``script_initialization`` argument.
+
+Let's see ``script_arguments`` and ``script_initialization`` in action.
+
+    >>> reset_interpreter()
+    >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts(
+    ...     interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir,
+    ...     reqs=['demo'], script_arguments='1, 2',
+    ...    script_initialization='import os\nos.chdir("foo")')
+
+    >>> cat(demo_path) # doctest: +NORMALIZE_WHITESPACE
+    #!/usr/local/bin/python2.4 -S
+    import sys
+    sys.path[0:0] = [
+      '/interpreter/parts/interpreter',
+      ]
+    <BLANKLINE>
+    import site # imports custom buildout-generated site.py
+    import os
+    os.chdir("foo")
+    <BLANKLINE>
+    import eggrecipedemo
+    <BLANKLINE>
+    if __name__ == '__main__':
+        eggrecipedemo.main(1, 2)
 
 Handling custom build options for extensions provided in source distributions
 -----------------------------------------------------------------------------

=== modified file 'src/zc/buildout/testing.py'
--- src/zc/buildout/testing.py	2010-02-23 20:41:17 +0000
+++ src/zc/buildout/testing.py	2010-02-23 20:41:17 +0000
@@ -28,6 +28,7 @@
 import subprocess
 import sys
 import tempfile
+import textwrap
 import threading
 import time
 import urllib2
@@ -105,6 +106,16 @@
     e.close()
     return result
 
+def call_py(interpreter, cmd, flags=None):
+    if sys.platform == 'win32':
+        args = ['"%s"' % arg for arg in (interpreter, flags, cmd) if arg]
+        args.insert(-1, '"-c"')
+        return system('"%s"' % ' '.join(args))
+    else:
+        cmd = repr(cmd)
+        return system(
+            ' '.join(arg for arg in (interpreter, flags, '-c', cmd) if arg))
+
 def get(url):
     return urllib2.urlopen(url).read()
 
@@ -116,7 +127,11 @@
     args = [zc.buildout.easy_install._safe_arg(arg)
             for arg in args]
     args.insert(0, '-q')
-    args.append(dict(os.environ, PYTHONPATH=setuptools_location))
+    env = dict(os.environ)
+    if executable == sys.executable:
+        env['PYTHONPATH'] = setuptools_location
+    # else pass an executable that has setuptools! See testselectingpython.py.
+    args.append(env)
 
     here = os.getcwd()
     try:
@@ -135,6 +150,11 @@
 def bdist_egg(setup, executable, dest):
     _runsetup(setup, executable, 'bdist_egg', '-d', dest)
 
+def sys_install(setup, dest):
+    _runsetup(setup, sys.executable, 'install', '--install-purelib', dest,
+              '--record', os.path.join(dest, '__added_files__'),
+              '--single-version-externally-managed')
+
 def find_python(version):
     e = os.environ.get('PYTHON%s' % version)
     if e is not None:
@@ -202,6 +222,24 @@
         time.sleep(0.01)
     raise ValueError('Timed out waiting for: '+label)
 
+def make_buildout():
+    # Create a basic buildout.cfg to avoid a warning from buildout:
+    open('buildout.cfg', 'w').write(
+        "[buildout]\nparts =\n"
+        )
+    # Use the buildout bootstrap command to create a buildout
+    zc.buildout.buildout.Buildout(
+        'buildout.cfg',
+        [('buildout', 'log-level', 'WARNING'),
+         # trick bootstrap into putting the buildout develop egg
+         # in the eggs dir.
+         ('buildout', 'develop-eggs-directory', 'eggs'),
+         ]
+        ).bootstrap([])
+    # Create the develop-eggs dir, which didn't get created the usual
+    # way due to the trick above:
+    os.mkdir('develop-eggs')
+
 def buildoutSetUp(test):
 
     test.globs['__tear_downs'] = __tear_downs = []
@@ -255,27 +293,7 @@
     sample = tmpdir('sample-buildout')
 
     os.chdir(sample)
-
-    # Create a basic buildout.cfg to avoid a warning from buildout:
-    open('buildout.cfg', 'w').write(
-        "[buildout]\nparts =\n"
-        )
-
-    # Use the buildout bootstrap command to create a buildout
-    zc.buildout.buildout.Buildout(
-        'buildout.cfg',
-        [('buildout', 'log-level', 'WARNING'),
-         # trick bootstrap into putting the buildout develop egg
-         # in the eggs dir.
-         ('buildout', 'develop-eggs-directory', 'eggs'),
-         ]
-        ).bootstrap([])
-
-
-
-    # Create the develop-eggs dir, which didn't get created the usual
-    # way due to the trick above:
-    os.mkdir('develop-eggs')
+    make_buildout()
 
     def start_server(path):
         port, thread = _start_server(path, name=path)
@@ -283,6 +301,50 @@
         register_teardown(lambda: stop_server(url, thread))
         return url
 
+    def make_py(initialization=''):
+        """Returns paths to new executable and to its site-packages.
+        """
+        buildout = tmpdir('executable_buildout')
+        site_packages_dir = os.path.join(buildout, 'site-packages')
+        mkdir(site_packages_dir)
+        old_wd = os.getcwd()
+        os.chdir(buildout)
+        make_buildout()
+        # Normally we don't process .pth files in extra-paths.  We want to
+        # in this case so that we can test with setuptools system installs
+        # (--single-version-externally-managed), which use .pth files.
+        initialization = (
+            ('import sys\n'
+             'import site\n'
+             'known_paths = set(sys.path)\n'
+             'site_packages_dir = %r\n'
+             'site.addsitedir(site_packages_dir, known_paths)\n'
+            ) % (site_packages_dir,)) + initialization
+        initialization = '\n'.join(
+            '  ' + line for line in initialization.split('\n'))
+        install_develop(
+            'zc.recipe.egg', os.path.join(buildout, 'develop-eggs'))
+        install_develop(
+            'z3c.recipe.scripts', os.path.join(buildout, 'develop-eggs'))
+        write('buildout.cfg', textwrap.dedent('''\
+            [buildout]
+            parts = py
+
+            [py]
+            recipe = z3c.recipe.scripts
+            interpreter = py
+            initialization =
+            %(initialization)s
+            extra-paths = %(site-packages)s
+            eggs = setuptools
+            ''') % {
+                'initialization': initialization,
+                'site-packages': site_packages_dir})
+        system(os.path.join(buildout, 'bin', 'buildout'))
+        os.chdir(old_wd)
+        return (
+            os.path.join(buildout, 'bin', 'py'), site_packages_dir)
+
     test.globs.update(dict(
         sample_buildout = sample,
         ls = ls,
@@ -293,6 +355,7 @@
         tmpdir = tmpdir,
         write = write,
         system = system,
+        call_py = call_py,
         get = get,
         cd = (lambda *path: os.chdir(os.path.join(*path))),
         join = os.path.join,
@@ -301,6 +364,7 @@
         start_server = start_server,
         buildout = os.path.join(sample, 'bin', 'buildout'),
         wait_until = wait_until,
+        make_py = make_py
         ))
 
     zc.buildout.easy_install.prefer_final(prefer_final)

=== modified file 'src/zc/buildout/tests.py'
--- src/zc/buildout/tests.py	2010-02-23 20:41:17 +0000
+++ src/zc/buildout/tests.py	2010-02-23 20:41:17 +0000
@@ -53,6 +53,7 @@
 
     >>> ls('develop-eggs')
     -  foo.egg-link
+    -  z3c.recipe.scripts.egg-link
     -  zc.recipe.egg.egg-link
 
     """
@@ -84,6 +85,7 @@
 
     >>> ls('develop-eggs')
     -  foo.egg-link
+    -  z3c.recipe.scripts.egg-link
     -  zc.recipe.egg.egg-link
 
     >>> print system(join('bin', 'buildout')+' -vvv'), # doctest: +ELLIPSIS
@@ -668,6 +670,7 @@
 
     >>> ls('develop-eggs')
     -  foox.egg-link
+    -  z3c.recipe.scripts.egg-link
     -  zc.recipe.egg.egg-link
 
 Create another:
@@ -692,6 +695,7 @@
     >>> ls('develop-eggs')
     -  foox.egg-link
     -  fooy.egg-link
+    -  z3c.recipe.scripts.egg-link
     -  zc.recipe.egg.egg-link
 
 Remove one:
@@ -709,6 +713,7 @@
 
     >>> ls('develop-eggs')
     -  fooy.egg-link
+    -  z3c.recipe.scripts.egg-link
     -  zc.recipe.egg.egg-link
 
 Remove the other:
@@ -723,6 +728,7 @@
 All gone
 
     >>> ls('develop-eggs')
+    -  z3c.recipe.scripts.egg-link
     -  zc.recipe.egg.egg-link
     '''
 
@@ -797,6 +803,7 @@
     ...            + join(sample_buildout, 'eggs'))
 
     >>> ls('develop-eggs')
+    -  z3c.recipe.scripts.egg-link
     -  zc.recipe.egg.egg-link
 
     >>> ls('eggs') # doctest: +ELLIPSIS
@@ -1769,6 +1776,235 @@
     1 2
     """
 
+def versions_section_ignored_for_dependency_in_favor_of_site_packages():
+    r"""
+This is a test for a bugfix.
+
+The error showed itself when at least two dependencies were in a shared
+location like site-packages, and the first one met the "versions" setting.  The
+first dependency would be added, but subsequent dependencies from the same
+location (e.g., site-packages) would use the version of the package found in
+the shared location, ignoring the version setting.
+
+We begin with a Python that has demoneeded version 1.1 installed and a
+demo version 0.3, all in a site-packages-like shared directory.  We need
+to create this.  ``eggrecipedemo.main()`` shows the number after the dot
+(that is, ``X`` in ``1.X``), for the demo package and the demoneeded
+package, so this demonstrates that our Python does in fact have demo
+version 0.3 and demoneeded version 1.1.
+
+    >>> py_path = make_py_with_system_install(make_py, sample_eggs)
+    >>> print call_py(
+    ...     py_path,
+    ...     "import tellmy.version; print tellmy.version.__version__"),
+    1.1
+
+Now here's a setup that would expose the bug, using the
+zc.buildout.easy_install API.
+
+    >>> example_dest = tmpdir('example_dest')
+    >>> workingset = zc.buildout.easy_install.install(
+    ...     ['tellmy.version'], example_dest, links=[sample_eggs],
+    ...     executable=py_path,
+    ...     index=None,
+    ...     versions={'tellmy.version': '1.0'})
+    >>> for dist in workingset:
+    ...     res = str(dist)
+    ...     if res.startswith('tellmy.version'):
+    ...         print res
+    ...         break
+    tellmy.version 1.0
+
+Before the bugfix, the desired tellmy.version distribution would have
+been blocked the one in site-packages.
+"""
+
+def handle_namespace_package_in_both_site_packages_and_buildout_eggs():
+    r"""
+If you have the same namespace package in both site-packages and in
+buildout, we need to be very careful that faux-Python-executables and
+scripts generated by easy_install.sitepackage_safe_scripts correctly
+combine the two. We show this with the local recipe that uses the
+function, z3c.recipe.scripts.
+
+To demonstrate this, we will create three packages: tellmy.version 1.0,
+tellmy.version 1.1, and tellmy.fortune 1.0.  tellmy.version 1.1 is installed.
+
+    >>> py_path = make_py_with_system_install(make_py, sample_eggs)
+    >>> print call_py(
+    ...     py_path,
+    ...     "import tellmy.version; print tellmy.version.__version__")
+    1.1
+    <BLANKLINE>
+
+Now we will create a buildout that creates a script and a faux-Python script.
+We want to see that both can successfully import the specified versions of
+tellmy.version and tellmy.fortune.
+
+    >>> write('buildout.cfg',
+    ... '''
+    ... [buildout]
+    ... parts = eggs
+    ... find-links = %(link_server)s
+    ...
+    ... [primed_python]
+    ... executable = %(py_path)s
+    ...
+    ... [eggs]
+    ... recipe = z3c.recipe.scripts
+    ... python = primed_python
+    ... interpreter = py
+    ... add-site-packages = true
+    ... eggs = tellmy.version == 1.0
+    ...        tellmy.fortune == 1.0
+    ...        demo
+    ... script-initialization =
+    ...     import tellmy.version
+    ...     print tellmy.version.__version__
+    ...     import tellmy.fortune
+    ...     print tellmy.fortune.__version__
+    ... ''' % globals())
+
+    >>> print system(buildout)
+    Installing eggs.
+    Getting distribution for 'tellmy.version==1.0'.
+    Got tellmy.version 1.0.
+    Getting distribution for 'tellmy.fortune==1.0'.
+    Got tellmy.fortune 1.0.
+    Getting distribution for 'demo'.
+    Got demo 0.4c1.
+    Getting distribution for 'demoneeded'.
+    Got demoneeded 1.2c1.
+    Generated script '/sample-buildout/bin/demo'.
+    Generated interpreter '/sample-buildout/bin/py'.
+    <BLANKLINE>
+
+Finally, we are ready for the actual test.  Prior to the bug fix that
+this tests, the results of both calls below was the following::
+
+    1.1
+    Traceback (most recent call last):
+      ...
+    ImportError: No module named fortune
+    <BLANKLINE>
+
+In other words, we got the site-packages version of tellmy.version, and
+we could not import tellmy.fortune at all.  The following are the correct
+results for the interpreter and for the script.
+
+    >>> print call_py(
+    ...     join('bin', 'py'),
+    ...     "import tellmy.version; " +
+    ...     "print tellmy.version.__version__; " +
+    ...     "import tellmy.fortune; " +
+    ...     "print tellmy.fortune.__version__") # doctest: +ELLIPSIS
+    1.0
+    1.0...
+
+    >>> print system(join('bin', 'demo'))
+    1.0
+    1.0
+    4 2
+    <BLANKLINE>
+    """
+
+def handle_sys_path_version_hack():
+    r"""
+This is a test for a bugfix.
+
+If you use a Python that has a different version of one of your
+dependencies, and the new package tries to do sys.path tricks in the
+setup.py to get a __version__, and it uses namespace packages, the older
+package will be loaded first, making the setup version the wrong number.
+While very arguably packages simply shouldn't do this, some do, and we
+don't want buildout to fall over when they do.
+
+To demonstrate this, we will need to create a distribution that has one of
+these unpleasant tricks, and a Python that has an older version installed.
+
+    >>> py_path, site_packages_path = make_py()
+    >>> for version in ('1.0', '1.1'):
+    ...     tmp = tempfile.mkdtemp()
+    ...     try:
+    ...         write(tmp, 'README.txt', '')
+    ...         mkdir(tmp, 'src')
+    ...         mkdir(tmp, 'src', 'tellmy')
+    ...         write(tmp, 'src', 'tellmy', '__init__.py',
+    ...             "__import__("
+    ...             "'pkg_resources').declare_namespace(__name__)\n")
+    ...         mkdir(tmp, 'src', 'tellmy', 'version')
+    ...         write(tmp, 'src', 'tellmy', 'version',
+    ...               '__init__.py', '__version__=%r\n' % version)
+    ...         write(
+    ...             tmp, 'setup.py',
+    ...             "from setuptools import setup\n"
+    ...             "import sys\n"
+    ...             "sys.path.insert(0, 'src')\n"
+    ...             "from tellmy.version import __version__\n"
+    ...             "setup(\n"
+    ...             " name='tellmy.version',\n"
+    ...             " package_dir = {'': 'src'},\n"
+    ...             " packages = ['tellmy', 'tellmy.version'],\n"
+    ...             " install_requires = ['setuptools'],\n"
+    ...             " namespace_packages=['tellmy'],\n"
+    ...             " zip_safe=True, version=__version__,\n"
+    ...             " author='bob', url='bob', author_email='bob')\n"
+    ...             )
+    ...         zc.buildout.testing.sdist(tmp, sample_eggs)
+    ...         if version == '1.0':
+    ...             # We install the 1.0 version in site packages the way a
+    ...             # system packaging system (debs, rpms) would do it.
+    ...             zc.buildout.testing.sys_install(tmp, site_packages_path)
+    ...     finally:
+    ...         shutil.rmtree(tmp)
+    >>> print call_py(
+    ...     py_path,
+    ...     "import tellmy.version; print tellmy.version.__version__")
+    1.0
+    <BLANKLINE>
+    >>> write('buildout.cfg',
+    ... '''
+    ... [buildout]
+    ... parts = eggs
+    ... find-links = %(sample_eggs)s
+    ...
+    ... [primed_python]
+    ... executable = %(py_path)s
+    ...
+    ... [eggs]
+    ... recipe = zc.recipe.egg:eggs
+    ... python = primed_python
+    ... eggs = tellmy.version == 1.1
+    ... ''' % globals())
+
+Before the bugfix, running this buildout would generate this error:
+
+    Installing eggs.
+    Getting distribution for 'tellmy.version==1.1'.
+    Installing tellmy.version 1.1
+    Caused installation of a distribution:
+    tellmy.version 1.0
+    with a different version.
+    Got None.
+    While:
+      Installing eggs.
+    Error: There is a version conflict.
+    We already have: tellmy.version 1.0
+    <BLANKLINE>
+
+You can see the copiously commented fix for this in easy_install.py (see
+zc.buildout.easy_install.Installer._call_easy_install and particularly
+the comment leading up to zc.buildout.easy_install._easy_install_cmd).
+Now the install works correctly, as seen here.
+
+    >>> print system(buildout)
+    Installing eggs.
+    Getting distribution for 'tellmy.version==1.1'.
+    Got tellmy.version 1.1.
+    <BLANKLINE>
+
+    """
+
 if sys.version_info > (2, 4):
     def test_exit_codes():
         """
@@ -2367,6 +2603,7 @@
 
     >>> ls('develop-eggs')
     -  foo.egg-link
+    -  z3c.recipe.scripts.egg-link
     -  zc.recipe.egg.egg-link
 
     """
@@ -2654,6 +2891,47 @@
 
 ######################################################################
 
+def make_py_with_system_install(make_py, sample_eggs):
+    py_path, site_packages_path = make_py()
+    create_sample_namespace_eggs(sample_eggs, site_packages_path)
+    return py_path
+
+def create_sample_namespace_eggs(dest, site_packages_path=None):
+    from zc.buildout.testing import write, mkdir
+    for pkg, version in (('version', '1.0'), ('version', '1.1'),
+                         ('fortune', '1.0')):
+        tmp = tempfile.mkdtemp()
+        try:
+            write(tmp, 'README.txt', '')
+            mkdir(tmp, 'src')
+            mkdir(tmp, 'src', 'tellmy')
+            write(tmp, 'src', 'tellmy', '__init__.py',
+                "__import__("
+                "'pkg_resources').declare_namespace(__name__)\n")
+            mkdir(tmp, 'src', 'tellmy', pkg)
+            write(tmp, 'src', 'tellmy', pkg,
+                  '__init__.py', '__version__=%r\n' % version)
+            write(
+                tmp, 'setup.py',
+                "from setuptools import setup\n"
+                "setup(\n"
+                " name='tellmy.%(pkg)s',\n"
+                " package_dir = {'': 'src'},\n"
+                " packages = ['tellmy', 'tellmy.%(pkg)s'],\n"
+                " install_requires = ['setuptools'],\n"
+                " namespace_packages=['tellmy'],\n"
+                " zip_safe=True, version=%(version)r,\n"
+                " author='bob', url='bob', author_email='bob')\n"
+                % locals()
+                )
+            zc.buildout.testing.sdist(tmp, dest)
+            if (site_packages_path and pkg == 'version' and version == '1.1'):
+                # We install the 1.1 version in site packages the way a
+                # system packaging system (debs, rpms) would do it.
+                zc.buildout.testing.sys_install(tmp, site_packages_path)
+        finally:
+            shutil.rmtree(tmp)
+
 def create_sample_eggs(test, executable=sys.executable):
     write = test.globs['write']
     dest = test.globs['sample_eggs']
@@ -2776,6 +3054,7 @@
         test.globs['sample_eggs'])
     test.globs['update_extdemo'] = lambda : add_source_dist(test, 1.5)
     zc.buildout.testing.install_develop('zc.recipe.egg', test)
+    zc.buildout.testing.install_develop('z3c.recipe.scripts', test)
 
 egg_parse = re.compile('([0-9a-zA-Z_.]+)-([0-9a-zA-Z_.]+)-py(\d[.]\d).egg$'
                        ).match
@@ -2934,6 +3213,10 @@
                (re.compile('[-d]  setuptools-\S+[.]egg'), 'setuptools.egg'),
                (re.compile(r'\\[\\]?'), '/'),
                (re.compile(r'\#!\S+\bpython\S*'), '#!/usr/bin/python'),
+               # Normalize generate_script's Windows interpreter to UNIX:
+               (re.compile(r'\nimport subprocess\n'), '\n'),
+               (re.compile('subprocess\\.call\\(argv, env=environ\\)'),
+                'os.execve(sys.executable, argv, environ)'),
                ]+(sys.version_info < (2, 5) and [
                   (re.compile('.*No module named runpy.*', re.S), ''),
                   (re.compile('.*usage: pdb.py scriptfile .*', re.S), ''),
@@ -3043,6 +3326,8 @@
                zc.buildout.testing.normalize_script,
                normalize_bang,
                (re.compile('Downloading.*setuptools.*egg\n'), ''),
+               (re.compile('options:'), 'Options:'),
+               (re.compile('usage:'), 'Usage:'),
                ]),
             ))
 

=== modified file 'src/zc/buildout/testselectingpython.py'
--- src/zc/buildout/testselectingpython.py	2009-11-27 13:20:19 +0000
+++ src/zc/buildout/testselectingpython.py	2010-02-23 20:41:17 +0000
@@ -11,7 +11,7 @@
 # FOR A PARTICULAR PURPOSE.
 #
 ##############################################################################
-import os, re, sys, unittest
+import os, re, subprocess, sys, textwrap, unittest
 from zope.testing import doctest, renormalizing
 import zc.buildout.tests
 import zc.buildout.testing
@@ -42,6 +42,33 @@
 
 def multi_python(test):
     other_executable = zc.buildout.testing.find_python(other_version)
+    command = textwrap.dedent('''\
+        try:
+            import setuptools
+        except ImportError:
+            import sys
+            sys.exit(1)
+        ''')
+    if subprocess.call([other_executable, '-c', command],
+                       env=os.environ):
+        # the other executable does not have setuptools.  Get setuptools.
+        # We will do this using the same tools we are testing, for better or
+        # worse.  Alternatively, we could try using bootstrap.
+        executable_dir = test.globs['tmpdir']('executable_dir')
+        executable_parts = os.path.join(executable_dir, 'parts')
+        test.globs['mkdir'](executable_parts)
+        ws = zc.buildout.easy_install.install(
+            ['setuptools'], executable_dir,
+            index='http://www.python.org/pypi/',
+            always_unzip=True, executable=other_executable)
+        zc.buildout.easy_install.sitepackage_safe_scripts(
+            executable_dir, ws, other_executable, executable_parts,
+            reqs=['setuptools'], interpreter='py')
+        original_executable = other_executable
+        other_executable = os.path.join(executable_dir, 'py')
+        assert not subprocess.call(
+            [other_executable, '-c', command], env=os.environ), (
+            'test set up failed')
     sample_eggs = test.globs['tmpdir']('sample_eggs')
     os.mkdir(os.path.join(sample_eggs, 'index'))
     test.globs['sample_eggs'] = sample_eggs

=== modified file 'src/zc/buildout/update.txt'
--- src/zc/buildout/update.txt	2009-11-06 22:33:23 +0000
+++ src/zc/buildout/update.txt	2010-02-23 20:41:17 +0000
@@ -81,6 +81,7 @@
 Our buildout script has been updated to use the new eggs:
 
     >>> cat(sample_buildout, 'bin', 'buildout')
+    ... # doctest: +NORMALIZE_WHITESPACE
     #!/usr/local/bin/python2.4
     <BLANKLINE>
     import sys

=== added directory 'z3c.recipe.scripts_'
=== added file 'z3c.recipe.scripts_/CHANGES.txt'
--- z3c.recipe.scripts_/CHANGES.txt	1970-01-01 00:00:00 +0000
+++ z3c.recipe.scripts_/CHANGES.txt	2010-02-23 20:41:17 +0000
@@ -0,0 +1,7 @@
+Change History
+**************
+
+1.0.0
+=====
+
+Initial public version.

=== added file 'z3c.recipe.scripts_/README.txt'
--- z3c.recipe.scripts_/README.txt	1970-01-01 00:00:00 +0000
+++ z3c.recipe.scripts_/README.txt	2010-02-23 20:41:17 +0000
@@ -0,0 +1,10 @@
+********************************
+Buildout Script Recipe
+********************************
+
+.. contents::
+
+The script recipe installs eggs into a buildout eggs directory, exactly
+like zc.recipe.egg, and then generates scripts in a buildout bin
+directory with egg paths baked into them.
+

=== added file 'z3c.recipe.scripts_/setup.py'
--- z3c.recipe.scripts_/setup.py	1970-01-01 00:00:00 +0000
+++ z3c.recipe.scripts_/setup.py	2010-02-23 20:41:17 +0000
@@ -0,0 +1,76 @@
+##############################################################################
+#
+# Copyright (c) 2007 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""Setup for z3c.recipe.scripts package
+
+$Id: setup.py 106736 2009-12-18 02:33:08Z gary $
+"""
+
+version = '1.0.0dev'
+
+import os
+from setuptools import setup, find_packages
+
+def read(*rnames):
+    return open(os.path.join(os.path.dirname(__file__), *rnames)).read()
+
+name = "z3c.recipe.scripts"
+setup(
+    name = name,
+    version = version,
+    author = "Gary Poster",
+    author_email = "[e-mail]",
+    description = "Recipe for installing Python scripts",
+    long_description = (
+        read('README.txt')
+        + '\n' +
+        read('CHANGES.txt')
+        + '\n' +
+        'Detailed Documentation\n'
+        '**********************\n'
+        + '\n' +
+        read('src', 'z3c', 'recipe', 'scripts', 'README.txt')
+        + '\n' +
+        'Download\n'
+        '*********\n'
+        ),
+    keywords = "development build",
+    classifiers = [
+       'Development Status :: 5 - Production/Stable',
+       'Framework :: Buildout',
+       'Intended Audience :: Developers',
+       'License :: OSI Approved :: Zope Public License',
+       'Topic :: Software Development :: Build Tools',
+       'Topic :: Software Development :: Libraries :: Python Modules',
+       ],
+    url='http://cheeseshop.python.org/pypi/z3c.recipe.scripts',
+    license = "ZPL 2.1",
+
+    packages = find_packages('src'),
+    package_dir = {'':'src'},
+    namespace_packages = ['z3c', 'z3c.recipe'],
+    install_requires = [
+        'zc.buildout >=1.5.0dev',
+        'zc.recipe.egg >=1.2.3dev',
+        'setuptools'],
+    tests_require = ['zope.testing'],
+    test_suite = name+'.tests.test_suite',
+    entry_points = {'zc.buildout': ['default = %s:Scripts' % name,
+                                    'script = %s:Scripts' % name,
+                                    'scripts = %s:Scripts' % name,
+                                    'interpreter = %s:Interpreter' % name,
+                                    ]
+                    },
+    include_package_data = True,
+    zip_safe=False,
+    )

=== added directory 'z3c.recipe.scripts_/src'
=== added directory 'z3c.recipe.scripts_/src/z3c'
=== added file 'z3c.recipe.scripts_/src/z3c/__init__.py'
--- z3c.recipe.scripts_/src/z3c/__init__.py	1970-01-01 00:00:00 +0000
+++ z3c.recipe.scripts_/src/z3c/__init__.py	2010-02-23 20:41:17 +0000
@@ -0,0 +1,1 @@
+__import__('pkg_resources').declare_namespace(__name__)

=== added directory 'z3c.recipe.scripts_/src/z3c/recipe'
=== added file 'z3c.recipe.scripts_/src/z3c/recipe/__init__.py'
--- z3c.recipe.scripts_/src/z3c/recipe/__init__.py	1970-01-01 00:00:00 +0000
+++ z3c.recipe.scripts_/src/z3c/recipe/__init__.py	2010-02-23 20:41:17 +0000
@@ -0,0 +1,1 @@
+__import__('pkg_resources').declare_namespace(__name__)

=== added directory 'z3c.recipe.scripts_/src/z3c/recipe/scripts'
=== added file 'z3c.recipe.scripts_/src/z3c/recipe/scripts/README.txt'
--- z3c.recipe.scripts_/src/z3c/recipe/scripts/README.txt	1970-01-01 00:00:00 +0000
+++ z3c.recipe.scripts_/src/z3c/recipe/scripts/README.txt	2010-02-23 20:41:17 +0000
@@ -0,0 +1,402 @@
+Script and interpreter generation
+=================================
+
+This recipe is very similar to zc.recipe.egg, and if you are familiar with its
+options, you will be able to use this one easily.
+
+The script and interpreter generation in this recipe are improved from
+those provided by zc.recipe.egg in two basic ways.
+
+- The interpreter generated by the script supports all interpreter
+  options, as opposed to the subset provided by zc.recipe.egg.
+
+- Both scripts and interpreters from this recipe can optionally choose
+  to include site-packages, and even sitecustomize.
+
+The recipe takes several options.  First, here's the list of the options
+that overlap from the standard zc.recipe.eggs scripts recipe.  After
+this, we'll list the new options and describe them.
+
+* eggs
+* find-links
+* index
+* python
+* extra-paths
+* entry-points
+* scripts
+* dependent-scripts
+* interpreter
+* arguments
+* initialization
+* relative-paths
+
+In addition to these, the recipe offers these new options.  They are
+introduced here, and described more in depth below.
+
+add-site-packages
+    You can choose to have the site-packages of the underlying Python
+    available to your script or interpreter, in addition to the packages
+    from your eggs.  See the section on this option for motivations and
+    warnings.
+
+extends
+    You can extend another section using this value.  It is intended to be
+    used by extending a section that uses this package's scripts recipe.
+    In this manner, you can avoid repeating yourself.
+
+exec-sitecustomize
+    Normally the Python's real sitecustomize module is not processed.
+    If you want it to be processed, set this value to 'true'.  This will
+    be honored irrespective of the setting for add-site-packages.
+
+script-initialization
+    The standard initialization code affects both an interpreter and scripts.
+    The code in script-initialization is used only for the generated scripts.
+
+Finally, the "interpreter" entry point ignores ``script-initialization``,
+``scripts``, and ``arguments``, and provides yet another additional option.
+
+name
+    While, by default, the interpreter recipe takes the name of the
+    section to be the desired interpreter name, you can specify the
+    interpreter name here instead.
+
+Script generation
+-----------------
+
+Generating a basic script looks virtually identical to using zc.recipe.egg.
+
+(Note that the find-links and index values are typically not needed; they
+are included to help make this document run as a test successfully.)
+
+    >>> write(sample_buildout, 'buildout.cfg',
+    ... """
+    ... [buildout]
+    ... parts = demo
+    ...
+    ... [demo]
+    ... recipe = z3c.recipe.scripts
+    ... eggs = demo<0.3
+    ... find-links = %(server)s
+    ... index = %(server)s/index
+    ... """ % dict(server=link_server))
+
+    >>> print system(buildout),
+    Installing demo.
+    Getting distribution for 'demo<0.3'.
+    Got demo 0.2.
+    Getting distribution for 'demoneeded'.
+    Got demoneeded 1.2c1.
+    Generated script '/sample-buildout/bin/demo'.
+
+    >>> print system(join(sample_buildout, 'bin', 'demo')),
+    2 2
+
+Interpreter generation
+----------------------
+
+As with zc.recipe.egg, you can generate an interpreter with the default
+script recipe shown above by supplying the "interpreter" option.
+This example will create both an entry point script and an interpreter.
+
+    >>> write(sample_buildout, 'buildout.cfg',
+    ... """
+    ... [buildout]
+    ... parts = demo
+    ...
+    ... [demo]
+    ... recipe = z3c.recipe.scripts
+    ... eggs = demo<0.3
+    ... find-links = %(server)s
+    ... index = %(server)s/index
+    ... interpreter = py
+    ... """ % dict(server=link_server))
+
+    >>> print system(buildout),
+    Uninstalling demo.
+    Installing demo.
+    Generated script '/sample-buildout/bin/demo'.
+    Generated interpreter '/sample-buildout/bin/py'.
+
+You can also generate an interpreter alone with the ``interpreter`` recipe.
+
+    >>> write(sample_buildout, 'buildout.cfg',
+    ... """
+    ... [buildout]
+    ... parts = py
+    ...
+    ... [py]
+    ... recipe = z3c.recipe.scripts:interpreter
+    ... eggs = demo<0.3
+    ... find-links = %(server)s
+    ... index = %(server)s/index
+    ... """ % dict(server=link_server))
+
+    >>> print system(buildout),
+    Uninstalling demo.
+    Installing py.
+    Generated interpreter '/sample-buildout/bin/py'.
+
+In both cases, the bin/py script works by restarting Python after
+specifying a special path in PYTHONPATH.  This example shows the UNIX version;
+the Windows version actually uses subprocess instead.
+
+    >>> cat(sample_buildout, 'bin', 'py') # doctest: +NORMALIZE_WHITESPACE
+    #!/usr/bin/python2.4 -S
+    <BLANKLINE>
+    import os
+    import sys
+    <BLANKLINE>
+    argv = [sys.executable] + sys.argv[1:]
+    environ = os.environ.copy()
+    path = '/sample-buildout/parts/py'
+    if environ.get('PYTHONPATH'):
+        path = os.pathsep.join([path, environ['PYTHONPATH']])
+    environ['PYTHONPATH'] = path
+    os.execve(sys.executable, argv, environ)
+
+The path is a directory that contains two files: our own site.py and
+sitecustomize.py.  The site.py is modified from the underlying Python's
+site.py, and is responsible for setting up our paths. The
+sitecustomize.py is responsible for running the initialization code
+provided.
+
+    >>> ls(sample_buildout, 'parts', 'py')
+    -  site.py
+    -  sitecustomize.py
+
+Here's an example of using the generated interpreter.
+
+    >>> print system(join(sample_buildout, 'bin', 'py') +
+    ...              ' -c "import sys, pprint; pprint.pprint(sys.path[-2:])"')
+    ['/sample-buildout/eggs/demo-0.2-pyN.N.egg',
+     '/sample-buildout/eggs/demoneeded-1.2c1-pyN.N.egg']
+    <BLANKLINE>
+
+Including site-packages and sitecustomize
+-----------------------------------------
+
+As introduced above, this recipe supports including site packages.  This has
+some advantages and some serious dangers.
+
+A typical reason to include site-packages is that it is easier to
+install one or more dependencies in your Python than it is with
+buildout.  Some packages, such as lxml or Python PostgreSQL integration,
+have dependencies that can be much easier to build and/or install using
+other mechanisms, such as your operating system's package manager.  By
+installing some core packages into your Python's site-packages, this can
+significantly simplify some application installations.
+
+However, doing this has a significant danger.  One of the primary goals
+of buildout is to provide repeatability.  Some packages (one of the
+better known Python openid packages, for instance) change their behavior
+depending on what packages are available.  If Python curl bindings are
+available, these may be preferred by the library.  If a certain XML
+package is installed, it may be preferred by the library.  These hidden
+choices may cause small or large behavior differences.  The fact that
+they can be rarely encountered can actually make it worse: you forget
+that this might be a problem, and debugging the differences can be
+difficult.  If you allow site-packages to be included in your buildout,
+and the Python you use is not managed precisely by your application (for
+instance, it is a system Python), you open yourself up to these
+possibilities.  Don't be unaware of the dangers.
+
+To show off these features, we need to use buildout with a Python
+executable with some extra paths to show ``add-site-packages``; and one
+guaranteed to have a sitecustomize module to show
+``exec-sitecustomize``.  We'll make one using a test fixture called
+``make_py``. The os.environ change below will go into the sitecustomize,
+and the site_packages_path will be in the Python's path.
+
+    >>> py_path, site_packages_path = make_py(initialization='''\
+    ... import os
+    ... os.environ['zc.buildout'] = 'foo bar baz shazam'
+    ... ''')
+    >>> print site_packages_path
+    /executable_buildout/site-packages
+
+Now let's take a look at add-site-packages.
+
+    >>> write(sample_buildout, 'buildout.cfg',
+    ... """
+    ... [buildout]
+    ... parts = py
+    ... executable = %(py_path)s
+    ...
+    ... [py]
+    ... recipe = z3c.recipe.scripts:interpreter
+    ... add-site-packages = true
+    ... eggs = demo<0.3
+    ... find-links = %(server)s
+    ... index = %(server)s/index
+    ... """ % dict(server=link_server, py_path=py_path))
+
+    >>> print system(buildout),
+    Uninstalling py.
+    Installing py.
+    Generated interpreter '/sample-buildout/bin/py'.
+
+    >>> print system(join(sample_buildout, 'bin', 'py') +
+    ...              ''' -c "import sys, pprint; pprint.pprint(sys.path)"''')
+    ... # doctest: +ELLIPSIS
+    ['',
+     '/sample-buildout/parts/py',
+     ...,
+     '/sample-buildout/eggs/demo-0.2-pyN.N.egg',
+     '/sample-buildout/eggs/demoneeded-1.2c1-pyN.N.egg',
+     '/executable_buildout/eggs/setuptools-X-pyN.N.egg',
+     '/executable_buildout/site-packages']
+    <BLANKLINE>
+
+Next we will use the exec-sitecustomize option.  It simply copies
+Python's underlying sitecustomize module, if it exists, to the local
+version.  The os.environ change shown above in the make_py call will go
+into the sitecustomize.
+
+    >>> write(sample_buildout, 'buildout.cfg',
+    ... """
+    ... [buildout]
+    ... parts = py
+    ... executable = %(py_path)s
+    ...
+    ... [py]
+    ... recipe = z3c.recipe.scripts:interpreter
+    ... exec-sitecustomize = true
+    ... eggs = demo<0.3
+    ... find-links = %(server)s
+    ... index = %(server)s/index
+    ... """ % dict(server=link_server, py_path=py_path))
+
+    >>> print system(buildout),
+    Uninstalling py.
+    Installing py.
+    Generated interpreter '/sample-buildout/bin/py'.
+
+    >>> cat(sample_buildout, 'parts', 'py', 'sitecustomize.py')
+    ... # doctest: +NORMALIZE_WHITESPACE +ELLIPSIS
+    <BLANKLINE>
+    # The following is from
+    # /executable_buildout/parts/py/sitecustomize.py
+    ...
+    import os
+    os.environ['zc.buildout'] = 'foo bar baz shazam'
+
+    >>> print system(join(sample_buildout, 'bin', 'py') +
+    ...              ''' -c "import os; print os.environ['zc.buildout']"''')
+    foo bar baz shazam
+    <BLANKLINE>
+
+Options
+-------
+
+We'll focus now on the options that are different than zc.recipe.egg.
+
+Let's look at the ``extends`` option first.
+
+    >>> write(sample_buildout, 'buildout.cfg',
+    ... """
+    ... [buildout]
+    ... parts = demo python
+    ...
+    ... [demo]
+    ... recipe = z3c.recipe.scripts
+    ... eggs = demo<0.3
+    ... find-links = %(server)s
+    ... index = %(server)s/index
+    ...
+    ... [python]
+    ... recipe = z3c.recipe.scripts:interpreter
+    ... extends = demo
+    ... initialization =
+    ...     import os
+    ...     os.environ['zc.buildout'] = 'foo bar baz shazam'
+    ... """ % dict(server=link_server))
+
+That makes it easier to specify some initialization for the interpreter
+that is different than a script, while duplicating other configuration.
+
+Now let's put it in action.
+
+    >>> print system(buildout),
+    Uninstalling py.
+    Installing demo.
+    Generated script '/sample-buildout/bin/demo'.
+    Installing python.
+    Generated interpreter '/sample-buildout/bin/python'.
+
+    >>> print system(join(sample_buildout, 'bin', 'python') +
+    ...              ' -c "import sys, pprint; pprint.pprint(sys.path[-2:])"')
+    ['/sample-buildout/eggs/demo-0.2-pyN.N.egg',
+     '/sample-buildout/eggs/demoneeded-1.2c1-pyN.N.egg']
+    <BLANKLINE>
+    >>> print system(join(sample_buildout, 'bin', 'python') +
+    ...              ''' -c "import os; print os.environ['zc.buildout']"'''),
+    foo bar baz shazam
+
+Note that the parts/py directory has been cleaned up, and parts/python has
+been created.
+
+    >>> ls(sample_buildout, 'parts')
+    d  demo
+    d  python
+
+If you want to have initialization that only affects scripts, not the
+interpreter, you can use script-initialization.  Here's a demonstration.
+
+    >>> write(sample_buildout, 'buildout.cfg',
+    ... """
+    ... [buildout]
+    ... parts = demo
+    ...
+    ... [demo]
+    ... recipe = z3c.recipe.scripts
+    ... eggs = demo<0.3
+    ... find-links = %(server)s
+    ... index = %(server)s/index
+    ... interpreter = py
+    ... script-initialization =
+    ...     print "Hi from the script"
+    ... """ % dict(server=link_server))
+
+    >>> print system(buildout),
+    Uninstalling python.
+    Uninstalling demo.
+    Installing demo.
+    Generated script '/sample-buildout/bin/demo'.
+    Generated interpreter '/sample-buildout/bin/py'.
+
+    >>> print system(join(sample_buildout, 'bin', 'py') +
+    ...              ''' -c "print 'Hi from the interpreter'"'''),
+    Hi from the interpreter
+
+    >>> print system(join(sample_buildout, 'bin', 'demo')),
+    Hi from the script
+    2 2
+
+The last new option is ``name``.  This simply changes the name of the
+interpreter, so that you are not forced to use the name of the section.
+
+    >>> write(sample_buildout, 'buildout.cfg',
+    ... """
+    ... [buildout]
+    ... parts = interpreter
+    ...
+    ... [interpreter]
+    ... name = python2
+    ... recipe = z3c.recipe.scripts:interpreter
+    ... eggs = demo<0.3
+    ... find-links = %(server)s
+    ... index = %(server)s/index
+    ... """ % dict(server=link_server))
+
+    >>> print system(buildout),
+    Uninstalling demo.
+    Installing interpreter.
+    Generated interpreter '/sample-buildout/bin/python2'.
+
+    >>> print system(join(sample_buildout, 'bin', 'python2') +
+    ...              ' -c "print 42"')
+    42
+    <BLANKLINE>
+
+The other options all identical to zc.recipe.egg.

=== added file 'z3c.recipe.scripts_/src/z3c/recipe/scripts/__init__.py'
--- z3c.recipe.scripts_/src/z3c/recipe/scripts/__init__.py	1970-01-01 00:00:00 +0000
+++ z3c.recipe.scripts_/src/z3c/recipe/scripts/__init__.py	2010-02-23 20:41:17 +0000
@@ -0,0 +1,1 @@
+from z3c.recipe.scripts.scripts import Scripts, Interpreter

=== added file 'z3c.recipe.scripts_/src/z3c/recipe/scripts/scripts.py'
--- z3c.recipe.scripts_/src/z3c/recipe/scripts/scripts.py	1970-01-01 00:00:00 +0000
+++ z3c.recipe.scripts_/src/z3c/recipe/scripts/scripts.py	2010-02-23 20:41:17 +0000
@@ -0,0 +1,101 @@
+##############################################################################
+#
+# Copyright (c) 2009-2010 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""Install scripts from eggs.
+"""
+import os
+import zc.buildout
+import zc.buildout.easy_install
+from zc.recipe.egg.egg import ScriptBase
+
+
+class Base(ScriptBase):
+
+    def __init__(self, buildout, name, options):
+        if 'extends' in options:
+            options.update(buildout[options['extends']])
+        super(Base, self).__init__(buildout, name, options)
+        self.default_eggs = '' # Disables feature from zc.recipe.egg.
+        b_options = buildout['buildout']
+        options['parts-directory'] = os.path.join(
+            b_options['parts-directory'], self.name)
+
+        value = options.setdefault(
+            'add-site-packages',
+            b_options.get('add-site-packages', 'false'))
+        if value not in ('true', 'false'):
+            raise zc.buildout.UserError(
+                "Invalid value for add-site-packages option: %s" %
+                (value,))
+        self.add_site_packages = (value == 'true')
+
+        value = options.setdefault(
+            'exec-sitecustomize',
+            b_options.get('exec-sitecustomize', 'false'))
+        if value not in ('true', 'false'):
+            raise zc.buildout.UserError(
+                "Invalid value for exec-sitecustomize option: %s" %
+                (value,))
+        self.exec_sitecustomize = (value == 'true')
+
+
+class Interpreter(Base):
+
+    def __init__(self, buildout, name, options):
+        super(Interpreter, self).__init__(buildout, name, options)
+
+        options.setdefault('name', name)
+
+    def install(self):
+        reqs, ws = self.working_set()
+        options = self.options
+        generated = []
+        if not os.path.exists(options['parts-directory']):
+            os.mkdir(options['parts-directory'])
+            generated.append(options['parts-directory'])
+        generated.extend(zc.buildout.easy_install.sitepackage_safe_scripts(
+            options['bin-directory'], ws, options['executable'],
+            options['parts-directory'],
+            interpreter=options['name'],
+            extra_paths=self.extra_paths,
+            initialization=options.get('initialization', ''),
+            add_site_packages=self.add_site_packages,
+            exec_sitecustomize=self.exec_sitecustomize,
+            relative_paths=self._relative_paths,
+            ))
+        return generated
+
+    update = install
+
+
+class Scripts(Base):
+
+    def _install(self, reqs, ws, scripts):
+        options = self.options
+        generated = []
+        if not os.path.exists(options['parts-directory']):
+            os.mkdir(options['parts-directory'])
+            generated.append(options['parts-directory'])
+        generated.extend(zc.buildout.easy_install.sitepackage_safe_scripts(
+            options['bin-directory'], ws, options['executable'],
+            options['parts-directory'], reqs=reqs, scripts=scripts,
+            interpreter=options.get('interpreter'),
+            extra_paths=self.extra_paths,
+            initialization=options.get('initialization', ''),
+            add_site_packages=self.add_site_packages,
+            exec_sitecustomize=self.exec_sitecustomize,
+            relative_paths=self._relative_paths,
+            script_arguments=options.get('arguments', ''),
+            script_initialization=options.get('script-initialization', '')
+            ))
+        return generated

=== added file 'z3c.recipe.scripts_/src/z3c/recipe/scripts/tests.py'
--- z3c.recipe.scripts_/src/z3c/recipe/scripts/tests.py	1970-01-01 00:00:00 +0000
+++ z3c.recipe.scripts_/src/z3c/recipe/scripts/tests.py	2010-02-23 20:41:17 +0000
@@ -0,0 +1,293 @@
+##############################################################################
+#
+# Copyright (c) 2006 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+
+import os, re, shutil, sys
+import zc.buildout.tests
+import zc.buildout.testselectingpython
+import zc.buildout.testing
+
+import unittest
+from zope.testing import doctest, renormalizing
+
+# We do not explicitly test the recipe support for the ``eggs``,
+# ``find-links``, and ``index`` options because they are used for most or
+# all of the examples.  The README tests ``extends``,
+# ``include-site-customization`` and ``name``.  That leaves ``python``,
+# ``extra-paths``, ``initialization``, ``relative-paths``, and
+# ``add-site-packages``.
+
+def supports_python_option():
+    """
+This simply shows that the ``python`` option can specify another section to
+find the ``executable``.  (The ``python`` option defaults to looking in the
+``buildout`` section.)  We do this by creating a custom Python that will have
+some initialization that we can look for.
+
+    >>> py_path, site_packages_path = make_py(initialization='''
+    ... import os
+    ... os.environ['zc.buildout'] = 'foo bar baz shazam'
+    ... ''')
+
+    >>> write(sample_buildout, 'buildout.cfg',
+    ... '''
+    ... [buildout]
+    ... parts = py
+    ...
+    ... [custom_python]
+    ... executable = %(py_path)s
+    ...
+    ... [py]
+    ... recipe = z3c.recipe.scripts:interpreter
+    ... exec-sitecustomize = true
+    ... eggs = demo<0.3
+    ... find-links = %(server)s
+    ... index = %(server)s/index
+    ... python = custom_python
+    ... ''' % dict(server=link_server, py_path=py_path))
+
+    >>> print system(buildout),
+    Installing py.
+    Getting distribution for 'demo<0.3'.
+    Got demo 0.2.
+    Getting distribution for 'demoneeded'.
+    Got demoneeded 1.2c1.
+    Generated interpreter '/sample-buildout/bin/py'.
+
+    >>> print system(join(sample_buildout, 'bin', 'py') +
+    ...              ''' -c "import os; print os.environ['zc.buildout']"'''),
+    foo bar baz shazam
+"""
+
+def interpreter_recipe_supports_extra_paths_option():
+    """
+This shows that specifying extra-paths will affect sys.path.
+
+This recipe will not add paths that do not exist, so we create them.
+
+    >>> mkdir(sample_buildout, 'foo')
+    >>> mkdir(sample_buildout, 'foo', 'bar')
+    >>> mkdir(sample_buildout, 'spam')
+
+    >>> write(sample_buildout, 'buildout.cfg',
+    ... '''
+    ... [buildout]
+    ... parts = py
+    ...
+    ... [py]
+    ... recipe = z3c.recipe.scripts:interpreter
+    ... find-links = %(server)s
+    ... index = %(server)s/index
+    ... extra-paths =
+    ...    ${buildout:directory}/foo/bar
+    ...    ${buildout:directory}/spam
+    ... ''' % dict(server=link_server))
+
+    >>> print system(buildout),
+    Installing py.
+    Generated interpreter '/sample-buildout/bin/py'.
+    >>> print system(join(sample_buildout, 'bin', 'py') +
+    ...              ''' -c "import sys;print 'path' + ' '.join(sys.path)"''')
+    ... # doctest:+ELLIPSIS
+    path.../foo/bar /sample-buildout/spam...
+
+"""
+
+def interpreter_recipe_supports_initialization_option():
+    """
+This simply shows that the ``initialization`` option can specify code to
+run on initialization.
+
+    >>> write(sample_buildout, 'buildout.cfg',
+    ... '''
+    ... [buildout]
+    ... parts = py
+    ...
+    ... [py]
+    ... recipe = z3c.recipe.scripts:interpreter
+    ... initialization =
+    ...     import os
+    ...     os.environ['zc.buildout'] = 'foo bar baz shazam'
+    ... eggs = demo<0.3
+    ... find-links = %(server)s
+    ... index = %(server)s/index
+    ... ''' % dict(server=link_server))
+
+    >>> print system(buildout),
+    Installing py.
+    Getting distribution for 'demo<0.3'.
+    Got demo 0.2.
+    Getting distribution for 'demoneeded'.
+    Got demoneeded 1.2c1.
+    Generated interpreter '/sample-buildout/bin/py'.
+
+    >>> cat(sample_buildout, 'parts', 'py', 'sitecustomize.py')
+    ... # doctest: +NORMALIZE_WHITESPACE
+    <BLANKLINE>
+    import os
+    os.environ['zc.buildout'] = 'foo bar baz shazam'
+    >>> print system(join(sample_buildout, 'bin', 'py') +
+    ...              ''' -c "import os; print os.environ['zc.buildout']"'''),
+    foo bar baz shazam
+
+This also works with the exec-sitecustomize option, processing local
+initialization, and then the Python's initialization.  We show this with a
+custom Python.
+
+    >>> py_path, site_packages_path = make_py(initialization='''
+    ... import os
+    ... os.environ['zc.buildout'] = 'foo bar baz shazam'
+    ... ''')
+
+    >>> write(sample_buildout, 'buildout.cfg',
+    ... '''
+    ... [buildout]
+    ... parts = py
+    ...
+    ... [custom_python]
+    ... executable = %(py_path)s
+    ...
+    ... [py]
+    ... recipe = z3c.recipe.scripts:interpreter
+    ... initialization =
+    ...     import os
+    ...     os.environ['zc.recipe.egg'] = 'baLOOba'
+    ... exec-sitecustomize = true
+    ... eggs = demo<0.3
+    ... find-links = %(server)s
+    ... index = %(server)s/index
+    ... python = custom_python
+    ... ''' % dict(server=link_server, py_path=py_path))
+
+    >>> print system(buildout),
+    Uninstalling py.
+    Installing py.
+    Generated interpreter '/sample-buildout/bin/py'.
+
+    >>> cat(sample_buildout, 'parts', 'py', 'sitecustomize.py')
+    ... # doctest: +NORMALIZE_WHITESPACE +ELLIPSIS
+    <BLANKLINE>
+    import os
+    os.environ['zc.recipe.egg'] = 'baLOOba'
+    <BLANKLINE>
+    # The following is from
+    # /executable_buildout/parts/py/sitecustomize.py
+    ...
+    import os
+    os.environ['zc.buildout'] = 'foo bar baz shazam'
+
+    >>> print system(join(sample_buildout, 'bin', 'py') + ' -c ' +
+    ...              '''"import os; print os.environ['zc.recipe.egg']"'''),
+    baLOOba
+    >>> print system(join(sample_buildout, 'bin', 'py') +
+    ...              ''' -c "import os; print os.environ['zc.buildout']"'''),
+    foo bar baz shazam
+
+"""
+
+def interpreter_recipe_supports_relative_paths_option():
+    """
+This shows that the relative-paths option affects the code for inserting
+paths into sys.path.
+
+    >>> write(sample_buildout, 'buildout.cfg',
+    ... '''
+    ... [buildout]
+    ... parts = py
+    ...
+    ... [py]
+    ... recipe = z3c.recipe.scripts:interpreter
+    ... find-links = %(server)s
+    ... index = %(server)s/index
+    ... relative-paths = true
+    ... extra-paths =
+    ...    /foo/bar
+    ...    ${buildout:directory}/spam
+    ... ''' % dict(server=link_server))
+
+    >>> print system(buildout),
+    Installing py.
+    Generated interpreter '/sample-buildout/bin/py'.
+
+Let's look at the site.py that was generated:
+
+    >>> import sys
+    >>> sys.stdout.write('#'); cat(sample_buildout, 'parts', 'py', 'site.py')
+    ... # doctest: +NORMALIZE_WHITESPACE +ELLIPSIS
+    #...
+    def addsitepackages(known_paths):
+        "..."
+        join = os.path.join
+        base = os.path.dirname(os.path.abspath(os.path.realpath(__file__)))
+        base = os.path.dirname(base)
+        base = os.path.dirname(base)
+        buildout_paths = [
+            '/foo/bar',
+            join(base, 'spam')
+            ]...
+
+
+"""
+
+def setUp(test):
+    zc.buildout.tests.easy_install_SetUp(test)
+    zc.buildout.testing.install_develop('zc.recipe.egg', test)
+    zc.buildout.testing.install_develop('z3c.recipe.scripts', test)
+
+def setUpSelecting(test):
+    zc.buildout.testselectingpython.setup(test)
+    zc.buildout.testing.install_develop('zc.recipe.egg', test)
+    zc.buildout.testing.install_develop('z3c.recipe.scripts', test)
+
+def test_suite():
+    suite = unittest.TestSuite((
+        doctest.DocFileSuite(
+            'README.txt',
+            setUp=setUp, tearDown=zc.buildout.testing.buildoutTearDown,
+            checker=renormalizing.RENormalizing([
+               zc.buildout.testing.normalize_path,
+               zc.buildout.testing.normalize_endings,
+               zc.buildout.testing.normalize_script,
+               zc.buildout.testing.normalize_egg_py,
+               zc.buildout.tests.normalize_bang,
+               (re.compile(r'zc.buildout(-\S+)?[.]egg(-link)?'),
+                'zc.buildout.egg'),
+               (re.compile('[-d]  setuptools-[^-]+-'), 'setuptools-X-'),
+               (re.compile(r'setuptools-[\w.]+-py'), 'setuptools-X-py'),
+               (re.compile(r'eggs\\\\demo'), 'eggs/demo'),
+               (re.compile(r'[a-zA-Z]:\\\\foo\\\\bar'), '/foo/bar'),
+               (re.compile(r'\#!\S+\bpython\S*'), '#!/usr/bin/python'),
+               # Normalize generate_script's Windows interpreter to UNIX:
+               (re.compile(r'\nimport subprocess\n'), '\n'),
+               (re.compile('subprocess\\.call\\(argv, env=environ\\)'),
+                'os.execve(sys.executable, argv, environ)'),
+               ])
+            ),
+        doctest.DocTestSuite(
+            setUp=setUp,
+            tearDown=zc.buildout.testing.buildoutTearDown,
+            checker=renormalizing.RENormalizing([
+                zc.buildout.testing.normalize_path,
+                zc.buildout.testing.normalize_endings,
+                zc.buildout.testing.normalize_egg_py,
+                (re.compile(r'[a-zA-Z]:\\\\foo\\\\bar'), '/foo/bar'),
+                ]),
+            ),
+
+        ))
+
+    return suite
+
+if __name__ == '__main__':
+    unittest.main(defaultTest='test_suite')
+

=== modified file 'zc.recipe.egg_/setup.py'
--- zc.recipe.egg_/setup.py	2009-08-06 13:49:47 +0000
+++ zc.recipe.egg_/setup.py	2010-02-23 20:41:17 +0000
@@ -16,7 +16,7 @@
 $Id$
 """
 
-version = '0'
+version = '1.2.3dev'
 
 import os
 from setuptools import setup, find_packages
@@ -66,7 +66,7 @@
     package_dir = {'':'src'},
     namespace_packages = ['zc', 'zc.recipe'],
     install_requires = [
-        'zc.buildout >=1.2.0',
+        'zc.buildout >=1.5.0dev',
         'setuptools'],
     tests_require = ['zope.testing'],
     test_suite = name+'.tests.test_suite',

=== modified file 'zc.recipe.egg_/src/zc/recipe/egg/README.txt'
--- zc.recipe.egg_/src/zc/recipe/egg/README.txt	2010-02-23 20:41:17 +0000
+++ zc.recipe.egg_/src/zc/recipe/egg/README.txt	2010-02-23 20:41:17 +0000
@@ -154,6 +154,8 @@
 interpreter
    The name of a script to generate that allows access to a Python
    interpreter that has the path set based on the eggs installed.
+   (See the ``z3c.recipe.scripts`` recipe for a more full-featured
+   interpreter.)
 
 extra-paths
    Extra paths to include in a generated script.
@@ -577,7 +579,7 @@
     -  demo
     -  other
 
-    >>> cat(sample_buildout, 'bin', 'other')
+    >>> cat(sample_buildout, 'bin', 'other') # doctest: +NORMALIZE_WHITESPACE
     #!/usr/local/bin/python2.4
     <BLANKLINE>
     import sys
@@ -640,3 +642,4 @@
     Uninstalling bigdemo.
     Installing demo.
     Generated script '/sample-buildout/bin/foo'.
+

=== modified file 'zc.recipe.egg_/src/zc/recipe/egg/api.txt'
--- zc.recipe.egg_/src/zc/recipe/egg/api.txt	2010-02-23 20:41:17 +0000
+++ zc.recipe.egg_/src/zc/recipe/egg/api.txt	2010-02-23 20:41:17 +0000
@@ -117,6 +117,7 @@
     extras = other
     find-links = http://localhost:27071/
     index = http://localhost:27071/index
+    python = buildout
     recipe = sample
 
 If we use the extra-paths option:

=== modified file 'zc.recipe.egg_/src/zc/recipe/egg/custom.txt'
--- zc.recipe.egg_/src/zc/recipe/egg/custom.txt	2010-02-23 20:41:17 +0000
+++ zc.recipe.egg_/src/zc/recipe/egg/custom.txt	2010-02-23 20:41:17 +0000
@@ -150,6 +150,7 @@
 
     >>> ls(sample_buildout, 'develop-eggs')
     d  extdemo-1.4-py2.4-unix-i686.egg
+    -  z3c.recipe.scripts.egg-link
     -  zc.recipe.egg.egg-link
 
 Note that no scripts or dependencies are installed.  To install
@@ -231,6 +232,7 @@
     >>> ls(sample_buildout, 'develop-eggs')
     -  demo.egg-link
     d  extdemo-1.4-py2.4-unix-i686.egg
+    -  z3c.recipe.scripts.egg-link
     -  zc.recipe.egg.egg-link
 
 But if we run the buildout in the default on-line and newest modes, we
@@ -248,6 +250,7 @@
     -  demo.egg-link
     d  extdemo-1.4-py2.4-linux-i686.egg
     d  extdemo-1.5-py2.4-linux-i686.egg
+    -  z3c.recipe.scripts.egg-link
     -  zc.recipe.egg.egg-link
 
 Controlling the version used
@@ -287,6 +290,7 @@
     >>> ls(sample_buildout, 'develop-eggs')
     -  demo.egg-link
     d  extdemo-1.4-py2.4-linux-i686.egg
+    -  z3c.recipe.scripts.egg-link
     -  zc.recipe.egg.egg-link
 
 
@@ -553,6 +557,7 @@
     >>> ls('develop-eggs')
     -  demo.egg-link
     -  extdemo.egg-link
+    -  z3c.recipe.scripts.egg-link
     -  zc.recipe.egg.egg-link
 
 and the extdemo now has a built extension:

=== modified file 'zc.recipe.egg_/src/zc/recipe/egg/egg.py'
--- zc.recipe.egg_/src/zc/recipe/egg/egg.py	2010-02-23 20:41:17 +0000
+++ zc.recipe.egg_/src/zc/recipe/egg/egg.py	2010-02-23 20:41:17 +0000
@@ -19,11 +19,12 @@
 import logging, os, re, zipfile
 import zc.buildout.easy_install
 
+
 class Eggs(object):
 
     def __init__(self, buildout, name, options):
         self.buildout = buildout
-        self.name = name
+        self.name = self.default_eggs = name
         self.options = options
         b_options = buildout['buildout']
         links = options.get('find-links', b_options['find-links'])
@@ -52,7 +53,7 @@
         # verify that this is None, 'true' or 'false'
         get_bool(options, 'unzip')
 
-        python = options.get('python', b_options['python'])
+        python = options.setdefault('python', b_options['python'])
         options['executable'] = buildout[python]['executable']
 
     def working_set(self, extra=()):
@@ -65,15 +66,16 @@
 
         distributions = [
             r.strip()
-            for r in options.get('eggs', self.name).split('\n')
+            for r in options.get('eggs', self.default_eggs).split('\n')
             if r.strip()]
         orig_distributions = distributions[:]
         distributions.extend(extra)
 
-        if self.buildout['buildout'].get('offline') == 'true':
+        if b_options.get('offline') == 'true':
             ws = zc.buildout.easy_install.working_set(
                 distributions, options['executable'],
-                [options['develop-eggs-directory'], options['eggs-directory']]
+                [options['develop-eggs-directory'],
+                 options['eggs-directory']],
                 )
         else:
             kw = {}
@@ -85,7 +87,7 @@
                 index=self.index,
                 executable=options['executable'],
                 path=[options['develop-eggs-directory']],
-                newest=self.buildout['buildout'].get('newest') == 'true',
+                newest=b_options.get('newest') == 'true',
                 allow_hosts=self.allow_hosts,
                 **kw)
 
@@ -97,16 +99,19 @@
 
     update = install
 
-class Scripts(Eggs):
+
+class ScriptBase(Eggs):
 
     def __init__(self, buildout, name, options):
-        super(Scripts, self).__init__(buildout, name, options)
-
-        options['bin-directory'] = buildout['buildout']['bin-directory']
+        super(ScriptBase, self).__init__(buildout, name, options)
+
+        b_options = buildout['buildout']
+
+        options['bin-directory'] = b_options['bin-directory']
         options['_b'] = options['bin-directory'] # backward compat.
 
         self.extra_paths = [
-            os.path.join(buildout['buildout']['directory'], p.strip())
+            os.path.join(b_options['directory'], p.strip())
             for p in options.get('extra-paths', '').split('\n')
             if p.strip()
             ]
@@ -115,11 +120,9 @@
 
 
         relative_paths = options.get(
-            'relative-paths',
-            buildout['buildout'].get('relative-paths', 'false')
-            )
+            'relative-paths', b_options.get('relative-paths', 'false'))
         if relative_paths == 'true':
-            options['buildout-directory'] = buildout['buildout']['directory']
+            options['buildout-directory'] = b_options['directory']
             self._relative_paths = options['buildout-directory']
         else:
             self._relative_paths = ''
@@ -128,12 +131,13 @@
     parse_entry_point = re.compile(
         '([^=]+)=(\w+(?:[.]\w+)*):(\w+(?:[.]\w+)*)$'
         ).match
+
     def install(self):
         reqs, ws = self.working_set()
         options = self.options
 
         scripts = options.get('scripts')
-        if scripts or scripts is None:
+        if scripts or scripts is None or options.get('interpreter'):
             if scripts is not None:
                 scripts = scripts.split()
                 scripts = dict([
@@ -157,22 +161,32 @@
                     name = dist.project_name
                     if name != 'setuptools' and name not in reqs:
                         reqs.append(name)
-
-            return zc.buildout.easy_install.scripts(
-                reqs, ws, options['executable'],
-                options['bin-directory'],
-                scripts=scripts,
-                extra_paths=self.extra_paths,
-                interpreter=options.get('interpreter'),
-                initialization=options.get('initialization', ''),
-                arguments=options.get('arguments', ''),
-                relative_paths=self._relative_paths,
-                )
-
+            return self._install(reqs, ws, scripts)
         return ()
 
     update = install
 
+    def _install(self, reqs, ws, scripts):
+        # Subclasses implement this.
+        raise NotImplementedError()
+
+
+class Scripts(ScriptBase):
+
+    def _install(self, reqs, ws, scripts):
+        options = self.options
+        return zc.buildout.easy_install.scripts(
+            reqs, ws, options['executable'],
+            options['bin-directory'],
+            scripts=scripts,
+            extra_paths=self.extra_paths,
+            interpreter=options.get('interpreter'),
+            initialization=options.get('initialization', ''),
+            arguments=options.get('arguments', ''),
+            relative_paths=self._relative_paths
+            )
+
+
 def get_bool(options, name, default=False):
     value = options.get(name)
     if not value:

=== modified file 'zc.recipe.egg_/src/zc/recipe/egg/selecting-python.txt'
--- zc.recipe.egg_/src/zc/recipe/egg/selecting-python.txt	2010-02-23 20:41:17 +0000
+++ zc.recipe.egg_/src/zc/recipe/egg/selecting-python.txt	2010-02-23 20:41:17 +0000
@@ -35,7 +35,7 @@
     ... index = http://www.python.org/pypi/
     ...
     ... [python2.4]
-    ... executable = %(python23)s
+    ... executable = %(python24)s
     ...
     ... [demo]
     ... recipe = zc.recipe.egg
@@ -43,7 +43,7 @@
     ... find-links = %(server)s
     ... python = python2.4
     ... interpreter = py-demo
-    ... """ % dict(server=link_server, python23=other_executable))
+    ... """ % dict(server=link_server, python24=other_executable))
 
 Now, if we run the buildout:
 
@@ -55,8 +55,6 @@
    Getting distribution for 'demo<0.3'.
    Got demo 0.2.
    Getting distribution for 'demoneeded'.
-   Getting distribution for 'setuptools'.
-   Got setuptools 0.6.
    Got demoneeded 1.2c1.
    Generated script '/sample-buildout/bin/demo'.
    Generated interpreter '/sample-buildout/bin/py-demo'.
@@ -66,7 +64,6 @@
     >>> ls(sample_buildout, 'eggs')
     -  demo-0.2-py2.4.egg
     -  demoneeded-1.2c1-py2.4.egg
-    d  setuptools-0.6-py2.4.egg
     d  setuptools-0.6-py2.5.egg
     -  zc.buildout-1.0-py2.5.egg