Merge lp:~gary/zc.buildout/python-support-8-support-subprocess into lp:zc.buildout

[Gary Poster]

This branch does two things.

1) It makes it simple to start Python processes from scripts started with the new recipe: PYTHONPATH is set so that everything works by default.

2) It makes bootstrap.py (and dev.py) yet more robust in the face of system Pythons.

#1 affects #2.

[Gary Poster (gary)]

The conflicts in this branch are resolved in https://code.edge.launchpad.net/~gary/zc.buildout/python-support (see https://code.edge.launchpad.net/~gary/zc.buildout/python-support/+merge/21733 ).

[Francis J. Lacoste (flacoste)]

> === modified file 'bootstrap/bootstrap.py'
> +# In order to be more robust in the face of system Pythons, we want to run
> +# with site-packages loaded. This is somewhat tricky, in particular because

With or without? It seems that you want to run with it, but remove any all
namespace from it? Anyway the comment could be clearer given the
non-obviousness of the code.

> +# Python 2.6's distutils imports site, so starting with the -S flag is not
> +# sufficient.
> +if 'site' in sys.modules:
> + # We will restart with python -S.
> + args = sys.argv[:]
> + args[0:0] = [sys.executable, '-S']
> + args = map(quote, args)
> + os.execv(sys.executable, args)
> +clean_path = sys.path[:]
> +import site
> +sys.path[:] = clean_path
> +for k, v in sys.modules.items():
> + if (hasattr(v, '__path__') and
> + len(v.__path__)==1 and
> + not os.path.exists(os.path.join(v.__path__[0],'__init__.py'))):
> + # This is a namespace package. Remove it.
> + sys.modules.pop(k)
> +

> === modified file 'dev.py'
> +# In order to be more robust in the face of system Pythons, we want to run
> +# with site-packages loaded. This is somewhat tricky, in particular because
> +# Python 2.6's distutils imports site, so starting with the -S flag is not
> +# sufficient.

Since this is copy and paste from the other location, my other comment applies
here also.

> +if 'site' in sys.modules:
> + # We will restart with python -S.
> + args = sys.argv[:]
> + args[0:0] = [sys.executable, '-S']
> + args = map(quote, args)
> + os.execv(sys.executable, args)
> +clean_path = sys.path[:]
> +import site
> +sys.path[:] = clean_path
> +for k, v in sys.modules.items():
> + if (hasattr(v, '__path__') and
> + len(v.__path__)==1 and
> + not os.path.exists(os.path.join(v.__path__[0],'__init__.py'))):
> + # This is a namespace package. Remove it.
> + sys.modules.pop(k)
> +
> is_jython = sys.platform.startswith('java')

> === modified file 'src/zc/buildout/buildout.py'

> - args.insert(0, zc.buildout.easy_install._safe_arg (sys.executable))
> -
> + args.insert(0, zc.buildout.easy_install._safe_arg(sys.executable))
> + env = os.environ.copy()
> + env['PYTHONPATH'] = partsdir
> if is_jython:
> - sys.exit(subprocess.Popen([sys.executable] + list(args)).wait())
> + sys.exit(
> + subprocess.Popen(
> + [sys.executable] + list(args), env=env).wait())
> else:
> - sys.exit(os.spawnv(os.P_WAIT, sys.executable, args))
> + sys.exit(os.spawnve(os.P_WAIT, sys.executable, args, env))
>

The intent here is to run the script with only partsdir in the path?

> === modified file 'src/zc/buildout/easy_install.py'
> --- src/zc/buildout/easy_install.py 2010-03-19 16:04:20 +0000
> +++ src/zc/buildout/easy_install.py 2010-03-19 16:04:20 +0000
> @@ -99,6 +99,7 @@
> "print repr([os.path.normpath(p) for p in sys.path if p])"])
> # Windows needs some (as yet to be determined) part of the real env.
> env = os.environ.copy()
> + env.pop('PYTHONPATH', None)

Care to explain w...

review: Needs Information

[lp:~gary/zc.buildout/python-support-8-support-subprocess]

564. By Gary Poster on 2010-03-19

add explanatory comments; extend test to show that scripts honor explicit PYTHONPATH

[Gary Poster (gary)]

Good call on all comments. In the new revision, I answered your questions in the new code comments, and added a test as you described.

(Yes, we want to run without site-packages loaded.)

[Francis J. Lacoste (flacoste)]

All good!

review: Approve

Unmerged revisions

564. By Gary Poster on 2010-03-19

add explanatory comments; extend test to show that scripts honor explicit PYTHONPATH

563. By Gary Poster on 2010-03-19

fix intermittent test failure in update.txt

562. By Gary Poster on 2010-03-19

fixes for bootstrap and a system Python; changes based on learning what would be necessary to be able to develop buildout with a system Python (zc.recipe.testing would also need to use sitepackage_safe_scripts)

561. By Gary Poster on 2010-03-11

set up PYTHONPATH for scripts too, so subprocesses are good to go by default.

560. By Gary Poster on 2010-02-24

merge from gary-6 <- gary-5

559. By Gary Poster on 2010-02-24

with these changes, I can build zc.buildout and run its tests successfully with my system Python. To make it fully robust, zc.recipe.test probably would need to use sitepackage_safe_scripts, but this works for now.

558. By Gary Poster on 2010-02-24

add test for recent fix for buildout

557. By Gary Poster on 2010-02-24

make the buildout script itself safe for a Python with site packages.

556. By Gary Poster on 2010-02-23

merge from gary-5 <- gary-4

555. By Gary Poster on 2010-02-23

merge from gary-5 <- gary-4

=== modified file 'bootstrap/bootstrap.py'
--- bootstrap/bootstrap.py	2010-03-19 19:13:23 +0000
+++ bootstrap/bootstrap.py	2010-03-19 19:13:23 +0000
@@ -23,6 +23,39 @@
 import os, shutil, sys, tempfile, textwrap, urllib, urllib2
 from optparse import OptionParser
 
+if sys.platform == 'win32':
+    def quote(c):
+        if ' ' in c:
+            return '"%s"' % c # work around spawn lamosity on windows
+        else:
+            return c
+else:
+    quote = str
+
+# In order to be more robust in the face of system Pythons, we want to
+# run without site-packages loaded.  This is somewhat tricky, in
+# particular because Python 2.6's distutils imports site, so starting
+# with the -S flag is not sufficient.  However, we'll start with that:
+if 'site' in sys.modules:
+    # We will restart with python -S.
+    args = sys.argv[:]
+    args[0:0] = [sys.executable, '-S']
+    args = map(quote, args)
+    os.execv(sys.executable, args)
+# Now we are running with -S.  We'll get the clean sys.path, import site
+# because distutils will do it later, and then reset the path and clean
+# out any namespace packages from site-packages that might have been
+# loaded by .pth files.
+clean_path = sys.path[:]
+import site
+sys.path[:] = clean_path
+for k, v in sys.modules.items():
+    if (hasattr(v, '__path__') and
+        len(v.__path__)==1 and
+        not os.path.exists(os.path.join(v.__path__[0],'__init__.py'))):
+        # This is a namespace package.  Remove it.
+        sys.modules.pop(k)
+
 is_jython = sys.platform.startswith('java')
 
 setuptools_source = 'http://peak.telecommunity.com/dist/ez_setup.py'
@@ -128,16 +161,6 @@
         if path not in pkg_resources.working_set.entries:
             pkg_resources.working_set.add_entry(path)
 
-if sys.platform == 'win32':
-    def quote(c):
-        if ' ' in c:
-            return '"%s"' % c # work around spawn lamosity on windows
-        else:
-            return c
-else:
-    def quote (c):
-        return c
-
 cmd = [quote(sys.executable),
        '-c',
        quote('from setuptools.command.easy_install import main; main()'),
 
=== modified file 'buildout.cfg'
--- buildout.cfg	2010-03-19 19:13:23 +0000
+++ buildout.cfg	2010-03-19 19:13:23 +0000
@@ -3,14 +3,14 @@
 parts = test oltest py
 
 [py]
-recipe = zc.recipe.egg
+recipe = z3c.recipe.scripts
 eggs = zc.buildout
        zope.testing
 interpreter = py
 
 [test]
 recipe = zc.recipe.testrunner
-eggs = 
+eggs =
   zc.buildout
   zc.recipe.egg
   z3c.recipe.scripts
@@ -18,7 +18,7 @@
 # Tests that can be run wo a network
 [oltest]
 recipe = zc.recipe.testrunner
-eggs = 
+eggs =
   zc.buildout
   zc.recipe.egg
   z3c.recipe.scripts
 
=== modified file 'dev.py'
--- dev.py	2010-03-19 19:13:23 +0000
+++ dev.py	2010-03-19 19:13:23 +0000
@@ -21,6 +21,39 @@
 
 import os, shutil, sys, subprocess, urllib2
 
+if sys.platform == 'win32':
+    def quote(c):
+        if ' ' in c:
+            return '"%s"' % c # work around spawn lamosity on windows
+        else:
+            return c
+else:
+    quote = str
+
+# In order to be more robust in the face of system Pythons, we want to
+# run without site-packages loaded.  This is somewhat tricky, in
+# particular because Python 2.6's distutils imports site, so starting
+# with the -S flag is not sufficient.  However, we'll start with that:
+if 'site' in sys.modules:
+    # We will restart with python -S.
+    args = sys.argv[:]
+    args[0:0] = [sys.executable, '-S']
+    args = map(quote, args)
+    os.execv(sys.executable, args)
+# Now we are running with -S.  We'll get the clean sys.path, import site
+# because distutils will do it later, and then reset the path and clean
+# out any namespace packages from site-packages that might have been
+# loaded by .pth files.
+clean_path = sys.path[:]
+import site
+sys.path[:] = clean_path
+for k, v in sys.modules.items():
+    if (hasattr(v, '__path__') and
+        len(v.__path__)==1 and
+        not os.path.exists(os.path.join(v.__path__[0],'__init__.py'))):
+        # This is a namespace package.  Remove it.
+        sys.modules.pop(k)
+
 is_jython = sys.platform.startswith('java')
 
 for d in 'eggs', 'develop-eggs', 'bin':
@@ -49,14 +82,20 @@
 env['PYTHONPATH'] = os.path.dirname(pkg_resources.__file__)
 subprocess.Popen(
     [sys.executable] +
-    ['setup.py', '-q', 'develop', '-m', '-x', '-d', 'develop-eggs'],
+    ['-S', 'setup.py', '-q', 'develop', '-m', '-x', '-d', 'develop-eggs'],
     env=env).wait()
 
 pkg_resources.working_set.add_entry('src')
 
 import zc.buildout.easy_install
-zc.buildout.easy_install.scripts(
-    ['zc.buildout'], pkg_resources.working_set , sys.executable, 'bin')
+if not os.path.exists('parts'):
+    os.mkdir('parts')
+partsdir = os.path.join('parts', 'buildout')
+if not os.path.exists(partsdir):
+    os.mkdir(partsdir)
+zc.buildout.easy_install.sitepackage_safe_scripts(
+    'bin', pkg_resources.working_set, sys.executable, partsdir,
+    reqs=['zc.buildout'])
 
 bin_buildout = os.path.join('bin', 'buildout')
 
@@ -64,4 +103,5 @@
     # Jython needs the script to be called twice via sys.executable
     assert subprocess.Popen([sys.executable] + [bin_buildout]).wait() == 0
 
+
 sys.exit(subprocess.Popen(bin_buildout).wait())
 
=== modified file 'src/zc/buildout/buildout.py'
--- src/zc/buildout/buildout.py	2010-03-19 19:13:23 +0000
+++ src/zc/buildout/buildout.py	2010-03-19 19:13:23 +0000
@@ -375,7 +375,9 @@
         if options.get('offline') == 'true':
             ws = zc.buildout.easy_install.working_set(
                 distributions, options['executable'],
-                [options['develop-eggs-directory'], options['eggs-directory']]
+                [options['develop-eggs-directory'],
+                 options['eggs-directory']],
+                include_site_packages=False,
                 )
         else:
             ws = zc.buildout.easy_install.install(
@@ -385,7 +387,8 @@
                 executable=options['executable'],
                 path=[options['develop-eggs-directory']],
                 newest=self.newest,
-                allow_hosts=self._allow_hosts
+                allow_hosts=self._allow_hosts,
+                include_site_packages=False,
                 )
 
         # Now copy buildout and setuptools eggs, and record destination eggs:
@@ -851,16 +854,19 @@
         if not self.newest:
             return
 
+        options = self['buildout']
+
         ws = zc.buildout.easy_install.install(
             [
-            (spec + ' ' + self['buildout'].get(spec+'-version', '')).strip()
+            (spec + ' ' + options.get(spec+'-version', '')).strip()
             for spec in ('zc.buildout', 'setuptools')
             ],
-            self['buildout']['eggs-directory'],
-            links = self['buildout'].get('find-links', '').split(),
-            index = self['buildout'].get('index'),
-            path = [self['buildout']['develop-eggs-directory']],
-            allow_hosts = self._allow_hosts
+            options['eggs-directory'],
+            links = options.get('find-links', '').split(),
+            index = options.get('index'),
+            path = [options['develop-eggs-directory']],
+            allow_hosts = self._allow_hosts,
+            include_site_packages=False
             )
 
         upgraded = []
@@ -876,7 +882,7 @@
         __doing__ = 'Upgrading.'
 
         should_run = realpath(
-            os.path.join(os.path.abspath(self['buildout']['bin-directory']),
+            os.path.join(os.path.abspath(options['bin-directory']),
                          'buildout')
             )
         if sys.platform == 'win32':
@@ -908,21 +914,34 @@
 
         # the new dist is different, so we've upgraded.
         # Update the scripts and return True
-        zc.buildout.easy_install.scripts(
-            ['zc.buildout'], ws, sys.executable,
-            self['buildout']['bin-directory'],
-            )
+        partsdir = os.path.join(options['parts-directory'], 'buildout')
+        if os.path.exists(partsdir):
+            # This is primarily for unit tests, in which .py files change too
+            # fast for Python to know to regenerate the .pyc/.pyo files.
+            shutil.rmtree(partsdir)
+        os.mkdir(partsdir)
+        zc.buildout.easy_install.sitepackage_safe_scripts(
+            options['bin-directory'], ws, sys.executable, partsdir,
+            reqs=['zc.buildout'])
 
         # Restart
         args = map(zc.buildout.easy_install._safe_arg, sys.argv)
         if not __debug__:
             args.insert(0, '-O')
-        args.insert(0, zc.buildout.easy_install._safe_arg (sys.executable))
-
+        args.insert(0, zc.buildout.easy_install._safe_arg(sys.executable))
+        # We want to make sure that our new site.py is used for rerunning
+        # buildout, so we put the partsdir in PYTHONPATH for our restart.
+        # This overrides any set PYTHONPATH, but since we generally are
+        # trying to run with a completely "clean" python (only the standard
+        # library) then that should be fine.
+        env = os.environ.copy()
+        env['PYTHONPATH'] = partsdir
         if is_jython:
-            sys.exit(subprocess.Popen([sys.executable] + list(args)).wait())
+            sys.exit(
+                subprocess.Popen(
+                    [sys.executable] + list(args), env=env).wait())
         else:
-            sys.exit(os.spawnv(os.P_WAIT, sys.executable, args))
+            sys.exit(os.spawnve(os.P_WAIT, sys.executable, args, env))
 
     def _load_extensions(self):
         __doing__ = 'Loading extensions.'
@@ -943,7 +962,8 @@
                 working_set=pkg_resources.working_set,
                 links = self['buildout'].get('find-links', '').split(),
                 index = self['buildout'].get('index'),
-                newest=self.newest, allow_hosts=self._allow_hosts)
+                newest=self.newest, allow_hosts=self._allow_hosts,
+                include_site_packages=False)
 
             # Clear cache because extensions might now let us read pages we
             # couldn't read before.
@@ -1060,7 +1080,8 @@
                 path=path,
                 working_set=pkg_resources.working_set,
                 newest=buildout.newest,
-                allow_hosts=buildout._allow_hosts
+                allow_hosts=buildout._allow_hosts,
+                include_site_packages=False,
                 )
 
         __doing__ = 'Loading %s recipe entry %s:%s.', group, spec, entry
 
=== modified file 'src/zc/buildout/easy_install.py'
--- src/zc/buildout/easy_install.py	2010-03-19 19:13:23 +0000
+++ src/zc/buildout/easy_install.py	2010-03-19 19:13:23 +0000
@@ -99,6 +99,10 @@
             "print repr([os.path.normpath(p) for p in sys.path if p])"])
         # Windows needs some (as yet to be determined) part of the real env.
         env = os.environ.copy()
+        # We need to make sure that PYTHONPATH, which will often be set
+        # to include a custom buildout-generated site.py, is not set, or
+        # else we will not get an accurate sys.path for the executable.
+        env.pop('PYTHONPATH', None)
         env.update(kwargs)
         _proc = subprocess.Popen(
             cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, env=env)
@@ -337,9 +341,6 @@
                     self._site_packages))
         if self._include_site_packages:
             path.extend(self._site_packages)
-        # else we could try to still include the buildout_and_setuptools_path
-        # if the elements are not in site_packages, but we're not bothering
-        # with this optimization for now, in the name of code simplicity.
         if dest is not None and dest not in path:
             path.insert(0, dest)
         self._path = path
@@ -1209,9 +1210,9 @@
     generated.append(_generate_site(
         site_py_dest, working_set, executable, extra_paths,
         include_site_packages, relative_paths))
-    script_initialization = (
-        '\nimport site # imports custom buildout-generated site.py\n%s' % (
-            script_initialization,))
+    script_initialization = _script_initialization_template % dict(
+        site_py_dest=site_py_dest,
+        script_initialization=script_initialization)
     if not script_initialization.endswith('\n'):
         script_initialization += '\n'
     generated.extend(_generate_scripts(
@@ -1222,6 +1223,15 @@
             interpreter, dest, executable, site_py_dest, relative_paths))
     return generated
 
+_script_initialization_template = '''
+import site # imports custom buildout-generated site.py
+import os
+path = %(site_py_dest)r
+if os.environ.get('PYTHONPATH'):
+    path = os.pathsep.join([path, os.environ['PYTHONPATH']])
+os.environ['PYTHONPATH'] = path
+%(script_initialization)s'''
+
 # Utilities for the script generation functions.
 
 # These are shared by both ``scripts`` and ``sitepackage_safe_scripts``
@@ -1504,8 +1514,14 @@
            "fp, path, desc = imp.find_module(%r); "
            "fp.close; "
            "print path" % (name,)]
+    env = os.environ.copy()
+    # We need to make sure that PYTHONPATH, which will often be set to
+    # include a custom buildout-generated site.py, is not set, or else
+    # we will not get an accurate value for the "real" site.py and
+    # sitecustomize.py.
+    env.pop('PYTHONPATH', None)
     _proc = subprocess.Popen(
-        cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+        cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, env=env)
     stdout, stderr = _proc.communicate();
     if _proc.returncode:
         logger.info(
@@ -1601,7 +1617,9 @@
         site.close()
         real_site.close()
     if not successful_rewrite:
-        raise RuntimeError('Buildout did not successfully rewrite site.py')
+        raise RuntimeError(
+            'Buildout did not successfully rewrite %s to %s' %
+            (real_site_path, site_path))
     return site_path
 
 namespace_include_site_packages_setup = '''
 
=== modified file 'src/zc/buildout/easy_install.txt'
--- src/zc/buildout/easy_install.txt	2010-03-19 19:13:23 +0000
+++ src/zc/buildout/easy_install.txt	2010-03-19 19:13:23 +0000
@@ -1499,6 +1499,11 @@
     <BLANKLINE>
     <BLANKLINE>
     import site # imports custom buildout-generated site.py
+    import os
+    path = '/interpreter/parts/interpreter'
+    if os.environ.get('PYTHONPATH'):
+        path = os.pathsep.join([path, os.environ['PYTHONPATH']])
+    os.environ['PYTHONPATH'] = path
     <BLANKLINE>
     import eggrecipedemo
     <BLANKLINE>
@@ -1528,7 +1533,7 @@
     >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts(
     ...     interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir,
     ...     reqs=['demo'], script_arguments='1, 2',
-    ...    script_initialization='import os\nos.chdir("foo")')
+    ...     script_initialization='import os\nos.chdir("foo")')
 
     >>> cat(demo_path) # doctest: +NORMALIZE_WHITESPACE
     #!/usr/local/bin/python2.4 -S
@@ -1539,6 +1544,11 @@
     <BLANKLINE>
     import site # imports custom buildout-generated site.py
     import os
+    path = '/interpreter/parts/interpreter'
+    if os.environ.get('PYTHONPATH'):
+        path = os.pathsep.join([path, os.environ['PYTHONPATH']])
+    os.environ['PYTHONPATH'] = path
+    import os
     os.chdir("foo")
     <BLANKLINE>
     import eggrecipedemo
 
=== modified file 'src/zc/buildout/tests.py'
--- src/zc/buildout/tests.py	2010-03-19 19:13:23 +0000
+++ src/zc/buildout/tests.py	2010-03-19 19:13:23 +0000
@@ -2254,8 +2254,95 @@
 
     """
 
+def subprocesses_have_same_environment_by_default():
+    """
+The scripts generated by sitepackage_safe_scripts set the PYTHONPATH so that,
+if the environment is maintained (the default behavior), subprocesses get
+the same Python packages.
+
+First, we set up a script and an interpreter.
+
+    >>> interpreter_dir = tmpdir('interpreter')
+    >>> interpreter_parts_dir = os.path.join(
+    ...     interpreter_dir, 'parts', 'interpreter')
+    >>> interpreter_bin_dir = os.path.join(interpreter_dir, 'bin')
+    >>> mkdir(interpreter_bin_dir)
+    >>> mkdir(interpreter_dir, 'eggs')
+    >>> mkdir(interpreter_dir, 'parts')
+    >>> mkdir(interpreter_parts_dir)
+    >>> ws = zc.buildout.easy_install.install(
+    ...     ['demo'], join(interpreter_dir, 'eggs'), links=[link_server],
+    ...     index=link_server+'index/')
+    >>> test = (
+    ...     "import subprocess, sys; subprocess.call("
+    ...     "[sys.executable, '-c', "
+    ...     "'import eggrecipedemo; print eggrecipedemo.x'])")
+    >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts(
+    ...     interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir,
+    ...     reqs=['demo'], interpreter='py',
+    ...     script_initialization=test + '; sys.exit(0)')
+
+This works for the script.
+
+    >>> print system(join(interpreter_bin_dir, 'demo'))
+    3
+    <BLANKLINE>
+
+This also works for the generated interpreter.
+
+    >>> print call_py(join(interpreter_bin_dir, 'py'), test)
+    3
+    <BLANKLINE>
+
+If you have a PYTHONPATH in your environment, it will be honored, after
+the buildout-generated path.
+
+    >>> original_pythonpath = os.environ.get('PYTHONPATH')
+    >>> os.environ['PYTHONPATH'] = 'foo'
+    >>> test = (
+    ...     "import subprocess, sys; subprocess.call("
+    ...     "[sys.executable, '-c', "
+    ...     "'import sys, pprint; pprint.pprint(sys.path)'])")
+    >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts(
+    ...     interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir,
+    ...     reqs=['demo'], interpreter='py',
+    ...     script_initialization=test + '; sys.exit(0)')
+
+This works for the script.  As you can see, /sample_buildout/foo is included
+right after the "parts" directory that contains site.py and sitecustomize.py.
+You can also see, actually more easily than in the other example, that we
+have the desired eggs available.
+
+    >>> print system(join(interpreter_bin_dir, 'demo')), # doctest: +ELLIPSIS
+    ['',
+     '/interpreter/parts/interpreter',
+     '/sample-buildout/foo',
+     ...
+     '/interpreter/eggs/demo-0.3-pyN.N.egg',
+     '/interpreter/eggs/demoneeded-1.1-pyN.N.egg']
+
+This also works for the generated interpreter, with identical results.
+
+    >>> print call_py(join(interpreter_bin_dir, 'py'), test),
+    ... # doctest: +ELLIPSIS
+    ['',
+     '/interpreter/parts/interpreter',
+     '/sample-buildout/foo',
+     ...
+     '/interpreter/eggs/demo-0.3-pyN.N.egg',
+     '/interpreter/eggs/demoneeded-1.1-pyN.N.egg']
+
+    >>> # Cleanup
+    >>> if original_pythonpath:
+    ...     os.environ['PYTHONPATH'] = original_pythonpath
+    ... else:
+    ...     del os.environ['PYTHONPATH']
+    ...
+
+    """
+
 def bootstrap_makes_buildout_that_works_with_system_python():
-    """
+    r"""
 In order to work smoothly with a system Python, bootstrapping creates
 the buildout script with
 zc.buildout.easy_install.sitepackage_safe_scripts. If it did not, a
@@ -2274,15 +2361,11 @@
     >>> write(sample_buildout, 'recipes', 'dummy.py',
     ... '''
     ... import logging, os, zc.buildout
-    ...
     ... class Dummy:
-    ...
     ...     def __init__(self, buildout, name, options):
     ...         pass
-    ...
     ...     def install(self):
     ...         return ()
-    ...
     ...     def update(self):
     ...         pass
     ... ''')
@@ -2340,6 +2423,67 @@
     Installing dummy.
     <BLANKLINE>
 
+Here's the same story with a namespace package, which has some additional
+complications behind the scenes.  First, a recipe, in the "tellmy" namespace.
+
+    >>> mkdir(sample_buildout, 'ns')
+    >>> mkdir(sample_buildout, 'ns', 'tellmy')
+    >>> write(sample_buildout, 'ns', 'tellmy', '__init__.py',
+    ...       "__import__('pkg_resources').declare_namespace(__name__)\n")
+    >>> mkdir(sample_buildout, 'ns', 'tellmy', 'recipes')
+    >>> write(sample_buildout, 'ns', 'tellmy', 'recipes', '__init__.py', ' ')
+    >>> write(sample_buildout, 'ns', 'tellmy', 'recipes', 'dummy.py',
+    ... '''
+    ... import logging, os, zc.buildout
+    ... class Dummy:
+    ...     def __init__(self, buildout, name, options):
+    ...         pass
+    ...     def install(self):
+    ...         return ()
+    ...     def update(self):
+    ...         pass
+    ... ''')
+    >>> write(sample_buildout, 'ns', 'setup.py',
+    ... '''
+    ... from setuptools import setup
+    ... setup(
+    ...     name="tellmy.recipes",
+    ...     packages=['tellmy', 'tellmy.recipes'],
+    ...     install_requires=['setuptools'],
+    ...     namespace_packages=['tellmy'],
+    ...     entry_points = {'zc.buildout':
+    ...                     ['dummy = tellmy.recipes.dummy:Dummy']},
+    ...     )
+    ... ''')
+
+Now, a buildout that uses it.
+
+    >>> create_sample_namespace_eggs(sample_eggs, site_packages_path)
+    >>> rmdir('develop-eggs')
+    >>> from zc.buildout.testing import make_buildout
+    >>> make_buildout(executable=py_path)
+    >>> write(sample_buildout, 'buildout.cfg',
+    ... '''
+    ... [buildout]
+    ... develop = ns
+    ...           recipes
+    ... parts = dummy
+    ... find-links = %(link_server)s
+    ... executable = %(py_path)s
+    ...
+    ... [dummy]
+    ... recipe = tellmy.recipes:dummy
+    ... ''' % globals())
+
+Now we actually run the buildout.
+
+    >>> print system(buildout)
+    Develop: '/sample-buildout/ns'
+    Develop: '/sample-buildout/recipes'
+    Uninstalling dummy.
+    Installing dummy.
+    <BLANKLINE>
+
     """
 
 if sys.version_info > (2, 4):
 
=== modified file 'src/zc/buildout/update.txt'
--- src/zc/buildout/update.txt	2010-03-19 19:13:23 +0000
+++ src/zc/buildout/update.txt	2010-03-19 19:13:23 +0000
@@ -78,22 +78,26 @@
     zc.buildout 99.99
     setuptools 99.99
 
-Our buildout script has been updated to use the new eggs:
+Our buildout script's site.py has been updated to use the new eggs:
 
-    >>> cat(sample_buildout, 'bin', 'buildout')
-    ... # doctest: +NORMALIZE_WHITESPACE
-    #!/usr/local/bin/python2.4
-    <BLANKLINE>
-    import sys
-    sys.path[0:0] = [
-      '/sample-buildout/eggs/zc.buildout-99.99-py2.4.egg',
-      '/sample-buildout/eggs/setuptools-99.99-py2.4.egg',
-      ]
-    <BLANKLINE>
-    import zc.buildout.buildout
-    <BLANKLINE>
-    if __name__ == '__main__':
-        zc.buildout.buildout.main()
+    >>> cat(sample_buildout, 'parts', 'buildout', 'site.py')
+    ... # doctest: +NORMALIZE_WHITESPACE +ELLIPSIS
+    "...
+    def addsitepackages(known_paths):
+        """Add site packages, as determined by zc.buildout.
+    <BLANKLINE>
+        See original_addsitepackages, below, for the original version."""
+        buildout_paths = [
+            '/sample-buildout/eggs/zc.buildout-99.99-pyN.N.egg',
+            '/sample-buildout/eggs/setuptools-99.99-pyN.N.egg'
+            ]
+        for path in buildout_paths:
+            sitedir, sitedircase = makepath(path)
+            if not sitedircase in known_paths and os.path.exists(sitedir):
+                sys.path.append(sitedir)
+                known_paths.add(sitedircase)
+        return known_paths
+    ...
 
 Now, let's recreate the sample buildout. If we specify constraints on
 the versions of zc.buildout and setuptools to use, running the
@@ -120,7 +124,6 @@
       zc.buildout version 1.0.0,
       setuptools version 0.6;
     restarting.
-    Generated script '/sample-buildout/bin/buildout'.
     Develop: '/sample-buildout/showversions'
     Updating show-versions.
     zc.buildout 1.0.0