Merge lp:~gary/zc.buildout/python-support-9-relative-paths into lp:~gary/zc.buildout/python-support

[Gary Poster]

This change introduces the ``relative-paths`` option, which Jim Fulton added to zc.recipe.egg and I carried over to z3c.recipe.scripts, to buildout proper. It affects the creation of the buildout script, as the test shows.

I am proposing it for merging into python-support, which is the zc.buildout fork I am currently maintaining for Launchpad.

[Francis J. Lacoste (flacoste)]

Looks good but there are two things I don't understand.

1) From where the unzip and use-dependancy-links documentation comes from?

2) Why do you need to move import site to after the path = sys.path[0] initialization.

And I don't see any implementation related to relative-paths? Is the diff missing something?

[Francis J. Lacoste (flacoste)]

review: Needs Information

[Gary Poster (gary)]

> Looks good but there are two things I don't understand.
>
> 1) From where the unzip and use-dependancy-links documentation comes from?

This is a darn good question. I looked into it a bit and I don't have a full answer.

The documentation comes from previous branches in this work--maybe as far back as python-support-2. The question in my mind is...why was this not merged before? I don't know. You can see it is a part of the previous branches in lines 2417-2435 of http://bazaar.launchpad.net/%7Egary/zc.buildout/python-support-8-support-subprocess/annotate/head%3A/src/zc/buildout/buildout.txt . It should have been merged long before.

Jim has not merged the pertinent branch, so I don't see how it could be some re-insertion.

The only thing I can figure is that it is some bzr + svn oddity.

In any case, the documentation should in fact be there, and it is not duplicated. It's a good change, even if it shouldn't really be here.

>
> 2) Why do you need to move import site to after the path = sys.path[0]
> initialization.

It's a safety measure. One of the main points of the site module is to manipulate the path. Let's get the path we want as soon as possible, before the site module has its way with the path.

> And I don't see any implementation related to relative-paths? Is the diff
> missing something?

No. The implementation has...four parts, I would say.

1) We set the default value here (line 21 in the diff as of this writing: ``'relative-paths': 'false',``

2) We take the value in consideration for the only script that buildout generates itself, bin/buildout, in this branch (lines 29-39 of the current diff).

3) We offer recipes a ``relative_paths`` argument to the ``scripts`` and ``site_package_safe_scripts`` functions provided by zc.buildout.easy_install. This was already done.

4) Recipes that generate files that might have paths should use the ``relative_paths`` argument described above. zc.recipe.egg and z3c.recipe.scripts already do this. Other recipes do not (e.g., z3c.recipe.i18n).

Therefore, the big change we have here is that buildout actually starts to recognize the option in itself that zc.recipe.egg had already been looking for.

[Francis J. Lacoste (flacoste)]

review: Approve

=== modified file 'src/zc/buildout/bootstrap.txt'
--- src/zc/buildout/bootstrap.txt	2010-02-13 00:39:20 +0000
+++ src/zc/buildout/bootstrap.txt	2010-04-19 19:29:12 +0000
@@ -47,7 +47,7 @@
     X...
     d  zc.buildout-...egg
 
-Now trying the `--version` option, that let you define a version for
+Now we will try the `--version` option, which lets you define a version for
 `zc.buildout`. If not provided, bootstrap will look for the latest one.
 
 Let's try with an unknown version::

=== modified file 'src/zc/buildout/buildout.py'
--- src/zc/buildout/buildout.py	2010-03-19 19:14:02 +0000
+++ src/zc/buildout/buildout.py	2010-04-19 19:29:12 +0000
@@ -129,6 +129,7 @@
     'parts-directory': 'parts',
     'prefer-final': 'false',
     'python': 'buildout',
+    'relative-paths': 'false',
     'socket-timeout': '',
     'unzip': 'false',
     'use-dependency-links': 'true',
@@ -411,9 +412,16 @@
         partsdir = os.path.join(options['parts-directory'], 'buildout')
         if not os.path.exists(partsdir):
             os.mkdir(partsdir)
+        # (Honor the relative-paths option.)
+        relative_paths = options.get('relative-paths', 'false')
+        if relative_paths == 'true':
+            relative_paths = options['directory']
+        else:
+            assert relative_paths == 'false'
+            relative_paths = ''
         zc.buildout.easy_install.sitepackage_safe_scripts(
             options['bin-directory'], ws, options['executable'], partsdir,
-            reqs=['zc.buildout'])
+            reqs=['zc.buildout'], relative_paths=relative_paths)
 
     init = bootstrap
 

=== modified file 'src/zc/buildout/buildout.txt'
--- src/zc/buildout/buildout.txt	2010-03-19 18:12:23 +0000
+++ src/zc/buildout/buildout.txt	2010-04-19 19:29:12 +0000
@@ -767,8 +767,9 @@
         DEFAULT_VALUE
     python= buildout
         DEFAULT_VALUE
+    relative-paths= false
+        DEFAULT_VALUE
     socket-timeout=
-        DEFAULT_VALUE
     unzip= false
         DEFAULT_VALUE
     use-dependency-links= true
@@ -2286,6 +2287,7 @@
     parts-directory = /sample-buildout/parts
     prefer-final = false
     python = buildout
+    relative-paths = false
     socket-timeout =
     unzip = false
     use-dependency-links = true
@@ -2349,6 +2351,33 @@
    Python executable.  By default, the buildout section defines the
    default Python as the Python used to run the buildout.
 
+relative-paths
+    The paths generated by zc.buildout are absolute by default, and this
+    option is ``false``.  However, if you set this value to be ``true``,
+    bin/buildout will be generated with code that makes the paths relative.
+    Some recipes, such as zc.recipe.egg and z3c.recipe.scripts, honor this
+    value as well.
+
+unzip
+    By default, zc.buildout doesn't unzip zip-safe eggs ("unzip = false").
+    This follows the policy followed by setuptools itself.  Experience shows
+    this policy to to be inconvenient.  Zipped eggs make debugging more
+    difficult and often import more slowly.  You can include an unzip option in
+    the buildout section to change the default unzipping policy ("unzip =
+    true").
+
+use-dependency-links
+    By default buildout will obey the setuptools dependency_links metadata
+    when it looks for dependencies. This behavior can be controlled with
+    the use-dependency-links buildout option::
+
+      [buildout]
+      ...
+      use-dependency-links = false
+
+    The option defaults to true. If you set it to false, then dependency
+    links are only looked for in the locations specified by find-links.
+
 verbosity
    A log-level adjustment.  Typically, this is set via the -q and -v
    command-line options.
@@ -2395,8 +2424,56 @@
 directory if the original buildout had develop eggs for either
 buildout or setuptools.)
 
-Note that the buildout script was installed but not run.  To run
-the buildout, we'd have to run the installed buildout script.
+If relative-paths is ``true``, the buildout script uses relative paths.
+
+    >>> write(sample_bootstrapped, 'setup.cfg',
+    ... '''
+    ... [buildout]
+    ... relative-paths = true
+    ... parts =
+    ... ''')
+
+    >>> print system(buildout
+    ...              +' -c'+os.path.join(sample_bootstrapped, 'setup.cfg')
+    ...              +' bootstrap'),
+    Generated script '/sample-bootstrapped/bin/buildout'.
+
+    >>> buildout_script = join(sample_bootstrapped, 'bin', 'buildout')
+    >>> import sys
+    >>> if sys.platform.startswith('win'):
+    ...     buildout_script += '-script.py'
+    >>> print open(buildout_script).read() # doctest: +ELLIPSIS
+    #!... -S
+    <BLANKLINE>
+    import os
+    <BLANKLINE>
+    join = os.path.join
+    base = os.path.dirname(os.path.abspath(os.path.realpath(__file__)))
+    base = os.path.dirname(base)
+    <BLANKLINE>
+    import sys
+    sys.path[0:0] = [
+        join(base, 'parts/buildout'),
+        ]
+    <BLANKLINE>
+    <BLANKLINE>
+    import os
+    path = sys.path[0]
+    if os.environ.get('PYTHONPATH'):
+        path = os.pathsep.join([path, os.environ['PYTHONPATH']])
+    os.environ['PYTHONPATH'] = path
+    import site # imports custom buildout-generated site.py
+    <BLANKLINE>
+    import zc.buildout.buildout
+    <BLANKLINE>
+    if __name__ == '__main__':
+        zc.buildout.buildout.main()
+    <BLANKLINE>
+
+
+Note that, in the above two examples, the buildout script was installed
+but not run.  To run the buildout, we'd have to run the installed
+buildout script.
 
 If we have an existing buildout that already has a buildout.cfg, we'll
 normally use the bootstrap command instead of init.  It will complain

=== modified file 'src/zc/buildout/easy_install.py'
--- src/zc/buildout/easy_install.py	2010-03-19 19:14:02 +0000
+++ src/zc/buildout/easy_install.py	2010-04-19 19:29:12 +0000
@@ -1199,12 +1199,12 @@
     return generated
 
 _script_initialization_template = '''
-import site # imports custom buildout-generated site.py
 import os
-path = %(site_py_dest)r
+path = sys.path[0]
 if os.environ.get('PYTHONPATH'):
     path = os.pathsep.join([path, os.environ['PYTHONPATH']])
 os.environ['PYTHONPATH'] = path
+import site # imports custom buildout-generated site.py
 %(script_initialization)s'''
 
 # Utilities for the script generation functions.

=== modified file 'src/zc/buildout/easy_install.txt'
--- src/zc/buildout/easy_install.txt	2010-03-11 02:49:36 +0000
+++ src/zc/buildout/easy_install.txt	2010-04-19 19:29:12 +0000
@@ -1498,12 +1498,12 @@
         ]
     <BLANKLINE>
     <BLANKLINE>
-    import site # imports custom buildout-generated site.py
     import os
-    path = '/interpreter/parts/interpreter'
+    path = sys.path[0]
     if os.environ.get('PYTHONPATH'):
         path = os.pathsep.join([path, os.environ['PYTHONPATH']])
     os.environ['PYTHONPATH'] = path
+    import site # imports custom buildout-generated site.py
     <BLANKLINE>
     import eggrecipedemo
     <BLANKLINE>
@@ -1542,12 +1542,12 @@
       '/interpreter/parts/interpreter',
       ]
     <BLANKLINE>
-    import site # imports custom buildout-generated site.py
     import os
-    path = '/interpreter/parts/interpreter'
+    path = sys.path[0]
     if os.environ.get('PYTHONPATH'):
         path = os.pathsep.join([path, os.environ['PYTHONPATH']])
     os.environ['PYTHONPATH'] = path
+    import site # imports custom buildout-generated site.py
     import os
     os.chdir("foo")
     <BLANKLINE>