python-virtualenv doesn't work with Python 2.6.x

Bug #339904 reported by Krzysztof Klimonda
64
This bug affects 6 people
Affects Status Importance Assigned to Milestone
linux (Ubuntu)
Invalid
Undecided
Unassigned
python-setuptools (Ubuntu)
Fix Released
Undecided
Unassigned
python-virtualenv (Ubuntu)
Fix Released
High
Matthias Klose
python2.6 (Ubuntu)
Fix Released
Medium
Unassigned
python3.0 (Ubuntu)
Fix Released
Undecided
Unassigned

Bug Description

Binary package hint: python-virtualenv

kklimonda@ubuntu:~$ lsb_release -rd
Description: Ubuntu jaunty (development branch)
Release: 9.04
kklimonda@ubuntu:~$ virtualenv --version
1.0.1dev
kklimonda@ubuntu:~$ apt-cache policy python-virtualenv |grep Installed
  Installed: 1.1-1

Why does virtualenv return version 1.0.1dev when package is 1.1-1?
Also the newest virtualenv version is 1.3.2 and version 1.3.1 includes full python 2.6 support (I guess that's the real culprit as those missing modules are part of it).

kklimonda@ubuntu:~$ virtualenv test
New python executable in test/bin/python
'import site' failed; use -v for traceback
Installing setuptools...........
  Complete output from command test/bin/python -c "#!python
\"\"\"Bootstrap setuptoo...

" --always-copy -U setuptools:
  'import site' failed; use -v for traceback
Traceback (most recent call last):
  File "<string>", line 53, in <module>
  File "/home/kklimonda/test/lib/python2.6/os.py", line 49, in <module>
    import posixpath as path
  File "/home/kklimonda/test/lib/python2.6/posixpath.py", line 16, in <module>
    import warnings
ImportError: No module named warnings
----------------------------------------
...Installing setuptools...done.
Traceback (most recent call last):
  File "/usr/bin/virtualenv", line 8, in <module>
    load_entry_point('virtualenv==1.1', 'console_scripts', 'virtualenv')()
  File "/usr/lib/python2.6/dist-packages/virtualenv.py", line 350, in main
    unzip_setuptools=options.unzip_setuptools)
  File "/usr/lib/python2.6/dist-packages/virtualenv.py", line 578, in create_environment
    install_setuptools(py_executable, unzip=unzip_setuptools)
  File "/usr/lib/python2.6/dist-packages/virtualenv.py", line 266, in install_setuptools
    cwd=cwd)
  File "/usr/lib/python2.6/dist-packages/virtualenv.py", line 411, in call_subprocess
    % (cmd_desc, proc.returncode))
OSError: Command test/bin/python -c "#!python
\"\"\"Bootstrap setuptoo...

" --always-copy -U setuptools failed with error code 1
kklimonda@ubuntu:~$

Even if I link all required python modules (warnings.py*, linecache.py*, _abcoll.py*, abc.py) I get another error:

File "<string>", line 7, in <module>
(...a lot of those lines...)
File "<string>", line 7, in <module>
File "/home/kklimonda/test/lib/python2.6/posixpath.py", line 121, in dirname
    if head and head != '/'*len(head):
RuntimeError: maximum recursion depth exceeded in cmp
(...)
" --always-copy -U setuptools failed with error code 1
kklimonda@ubuntu:~$

Revision history for this message
Markus Korn (thekorn) wrote :

this issue is unrelated to the linux kernel, closing the linux task

Changed in linux (Ubuntu):
status: New → Invalid
Revision history for this message
Markus Korn (thekorn) wrote :

Thank you for reporting this bug, I'm able to reproduce this bug on my jaunty system

Changed in python-virtualenv (Ubuntu):
status: New → Confirmed
Revision history for this message
Markus Korn (thekorn) wrote :

I'm going to work on an update for this package over the weekend

Changed in python-virtualenv (Ubuntu):
assignee: nobody → thekorn
status: Confirmed → In Progress
Revision history for this message
Markus Korn (thekorn) wrote :

unassigned myself as python-virtualenv fails to build on jaunty and I don't have enough packaging skills to fix this. Will report a separate bug about this.

Changed in python-virtualenv (Ubuntu):
assignee: thekorn → nobody
Markus Korn (thekorn)
Changed in python-virtualenv (Ubuntu):
status: In Progress → Confirmed
Revision history for this message
Joe "Rotund" Tennies (joe-tennies) wrote :

I feel this is a big problem as python2.6 is the default /usr/bin/python

This becomes a big issue as it crashes by default.

Revision history for this message
Gavin Carothers (gavin.carothers) wrote :
Download full text (5.2 KiB)

Using a newer virtualenv, 1.3.3 does NOT solve this issue.

gavin@asfolath ~/P/butcher> virtualenv -v .
Creating ./lib/python2.6
Symlinking Python bootstrap modules
  Symlinking ./lib/python2.6/types.pyc
  Symlinking ./lib/python2.6/copy_reg.pyc
  Symlinking ./lib/python2.6/sre_constants.py
  Symlinking ./lib/python2.6/os.pyc
  Symlinking ./lib/python2.6/stat.py
  Symlinking ./lib/python2.6/warnings.pyc
  Symlinking ./lib/python2.6/encodings
  Symlinking ./lib/python2.6/sre.pyc
  Symlinking ./lib/python2.6/fnmatch.py
  Symlinking ./lib/python2.6/re.py
  Symlinking ./lib/python2.6/sre_compile.py
  Symlinking ./lib/python2.6/linecache.pyc
  Symlinking ./lib/python2.6/UserDict.pyc
  Symlinking ./lib/python2.6/lib-dynload
  Symlinking ./lib/python2.6/types.py
  Symlinking ./lib/python2.6/config
  Symlinking ./lib/python2.6/copy_reg.py
  Symlinking ./lib/python2.6/genericpath.py
  Symlinking ./lib/python2.6/sre_parse.pyc
  Symlinking ./lib/python2.6/abc.py
  Symlinking ./lib/python2.6/sre_constants.pyc
  Symlinking ./lib/python2.6/re.pyc
  Symlinking ./lib/python2.6/sre.py
  Symlinking ./lib/python2.6/locale.pyc
  Symlinking ./lib/python2.6/fnmatch.pyc
  Symlinking ./lib/python2.6/genericpath.pyc
  Symlinking ./lib/python2.6/warnings.py
  Symlinking ./lib/python2.6/stat.pyc
  Symlinking ./lib/python2.6/os.py
  Symlinking ./lib/python2.6/codecs.pyc
  Symlinking ./lib/python2.6/ntpath.py
  Symlinking ./lib/python2.6/abc.pyc
  Symlinking ./lib/python2.6/locale.py
  Symlinking ./lib/python2.6/_abcoll.pyc
  Symlinking ./lib/python2.6/posixpath.pyc
  Symlinking ./lib/python2.6/sre_compile.pyc
  Symlinking ./lib/python2.6/ntpath.pyc
  Symlinking ./lib/python2.6/UserDict.py
  Symlinking ./lib/python2.6/posixpath.py
  Symlinking ./lib/python2.6/linecache.py
  Symlinking ./lib/python2.6/sre_parse.py
  Symlinking ./lib/python2.6/_abcoll.py
  Symlinking ./lib/python2.6/codecs.py
Creating ./lib/python2.6/site-packages
Writing ./lib/python2.6/site.py
Writing ./lib/python2.6/orig-prefix.txt
Creating parent directories for ./include
Symlinking ./include/python2.6
Creating ./bin
New python executable in ./bin/python
Changed mode of ./bin/python to 0755
Testing executable with ./bin/python -c "import sys; print sys.prefix"
Got sys.prefix result: '/home/gavin/Programing/butcher'
Creating ./lib/python2.6/distutils
Writing ./lib/python2.6/distutils/__init__.py
Writing ./lib/python2.6/distutils/distutils.cfg
Using existing Setuptools egg: /usr/local/lib/python2.6/dist-packages/virtualenv-1.3.3-py2.6.egg/support-files/setuptools-0.6c9-py2.6.egg
Installing setuptools...
  error: can't create or remove files in install directory
  The following error occurred while trying to add or remove files in the
  installation directory:
      [Errno 2] No such file or directory: '/home/gavin/Programing/butcher/local/lib/python2.6/dist-packages/test-easy-install-7933.pth'
  The installation directory you specified (via --install-dir, --prefix, or
  the distutils default setting) was:
      /home/gavin/Programing/butcher/local/lib/python2.6/dist-packages/
  This directory does not currently exist. Please create it and try again, or
  choose a different installation directory (using ...

Read more...

Revision history for this message
Christopher Denter (dennda) wrote :

I'm having the same *serious* bug (python programmers need virtualenv...).

It does work with 2.5 though.
Someone mentioned it fails due to some python 2.6 patchery done by ubuntu/debian. (Something dist-packages / site-packages related).

Fix needed!

Revision history for this message
Florian Apolloner (apollo13) wrote :

I can confirm this bug. A fix would be really nice (but in a way that virtualenv, doesn't need to get patched...)

Revision history for this message
Krzysztof Klimonda (kklimonda) wrote :

Quoting Python 2.6 Ubuntu changelog:
"* For locally installed packages, create a directory
/usr/local/lib/python2.6/dist-packages. This is the default for
installations done with distutils and setuptools. Third party stuff
packaged within the distribution goes to /usr/lib/python2.6/dist-packages.
There is no /usr/lib/python2.6/site-packages in the file system and
on sys.path. No package within the distribution must not install
anything in this location.
"
I guess what happens is:
* virtualenv creates only $ENV/usr/lib/python2.6/site-packages because it doesn't check where Python distribution expects local packages to go.
* then virtualenv invokes setuptools which looks for $ENV/usr/local/lib/python2.6/dist-packages/ and bails out.

Correct me if I'm wrong, this is what i've deducted after short work with virtualenv-1.3.3.

virtualenv still works with python 2.5 because dist-packages and whole split into /usr/local/ vs. /usr/ was made in Python 2.6.. Well, according to changelogs.

Revision history for this message
Krzysztof Klimonda (kklimonda) wrote :

Hmm...
python-setuptools package is heavily patched to work with debian/ubuntu python 2.6 installation.

I guess that this part of python-setuptools_0.6c9-0ubuntu3.diff.gz should be "ported" to all setuptools eggs that are bundled with python-virtualenv/virtualenv tarballs..
--- python-setuptools-0.6c9.orig/setuptools/command/easy_install.py
+++ python-setuptools-0.6c9/setuptools/command/easy_install.py
@@ -1105,15 +1105,26 @@

- INSTALL_SCHEMES = dict(
+ if sys.version[:3] in ('2.3', '2.4', '2.5'):
+
+ INSTALL_SCHEMES = dict(
         posix = dict(
- install_dir = '$base/lib/python$py_version_short/site-packages',
- script_dir = '$base/bin',
+ install_dir = '$base/local/lib/python$py_version_short/site-packages',
+ script_dir = '$base/local/bin',
         ),
- )
+ )
+
+ else:
+
+ INSTALL_SCHEMES = dict(
+ posix = dict(
+ install_dir = '$base/local/lib/python$py_version_short/dist-packages',
+ script_dir = '$base/local/bin',
+ ),
+ )

     DEFAULT_SCHEME = dict(
- install_dir = '$base/Lib/site-packages',
+ install_dir = '$base/Lib/dist-packages',
         script_dir = '$base/Scripts',
     )

@@ -1157,10 +1168,18 @@
             if sys.platform in ('os2emx', 'riscos'):
                 sitedirs.append(os.path.join(prefix, "Lib", "site-packages"))
             elif os.sep == '/':
- sitedirs.extend([os.path.join(prefix,
+ if sys.version[:3] in ('2.3', '2.4', '2.5'):
+ sdir = "site-packages"
+ else:
+ sdir = "dist-packages"
+ sitedirs.extend([os.path.join(prefix,
+ "local/lib",
+ "python" + sys.version[:3],
+ sdir),
+ os.path.join(prefix,
                                          "lib",
                                          "python" + sys.version[:3],
- "site-packages"),
+ sdir),
                             os.path.join(prefix, "lib", "site-python")])
             else:
                 sitedirs.extend(

I'll try to mess with it in next few hours after I'm back at home if no one else fix it in the meantime..

Revision history for this message
Gavin Carothers (gavin.carothers) wrote :

Patching setuptools was not a great idea packagers. Given the common usages of ez_setup.py and other methods of installing setuptools.

Anyway, quick work around to get a working python2.5 virtualenv:
virtualenv --python=python2.5 py25-ve

No need for any symlink monkeying or package patching.

Revision history for this message
Krzysztof Klimonda (kklimonda) wrote :

This workaround doesn't work with virtualenv from ubuntu repositories.
Maybe a bug should be filled to bump python-virtualenv version to 1.3.x ?
(After all without another patch virtualenv 1.1 doesn't work with python 2.6)

Revision history for this message
Krzysztof Klimonda (kklimonda) wrote :

Maybe "the right way"(tm) of fixing it would be to make virtualenv use python-setuptools from ubuntu repository?
(still not at home.. just random thoughts ;) )

Revision history for this message
William Grant (wgrant) wrote :

This shouldn't be too hard to fix (I patched it locally myself last week). I'll look at it this weekend.

Changed in python-virtualenv (Ubuntu):
assignee: nobody → wgrant
importance: Undecided → High
milestone: none → ubuntu-9.04
status: Confirmed → Triaged
Revision history for this message
William Grant (wgrant) wrote :

I recall that my fix was to pass --prefix when running setuptools inside virtualenv - that overrides the default install layout.

Matthias Klose (doko)
Changed in python2.6 (Ubuntu):
importance: Undecided → Medium
status: New → Triaged
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package python2.6 - 2.6.1-1ubuntu10

---------------
python2.6 (2.6.1-1ubuntu10) jaunty; urgency=low

  [Matthias Klose]
  * Always use the `unix_prefix' scheme for setup.py install in a virtualenv
    setup. LP: #339904.

  [Marc Deslauriers]
  * debian/pyhtml2devhelp.py: Update parsing logic.

 -- Matthias Klose <email address hidden> Sat, 04 Apr 2009 11:11:32 +0200

Changed in python2.6 (Ubuntu):
status: Triaged → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package python-virtualenv - 1.3.3-1ubuntu1

---------------
python-virtualenv (1.3.3-1ubuntu1) jaunty; urgency=low

  * Correctly set up sys.path for python versions >= 2.6. LP: #339904.
  * Always use the default name for the site directory in virtualenv.
  * Fix build failure. LP: #347203.

python-virtualenv (1.3.3-1) unstable; urgency=low

  * New upstream version... Closes: #518189.
  * ...which adds a "--relocatable" option. Closes: #503168.
  * Update Standards-Version.
  * Fix package name in man page. Closes: #508724.

 -- Matthias Klose <email address hidden> Sat, 04 Apr 2009 11:36:07 +0200

Changed in python-virtualenv (Ubuntu):
status: Triaged → Fix Released
Revision history for this message
Krzysztof Klimonda (kklimonda) wrote :

It wasn't fixed - at least for now. Maybe you are still cooking the right release ;).
imho the best way of fixing it would require some code changes in virtualenv to make it use setuptools provided by ubuntu instead of eggs.

Matthias Klose (doko)
Changed in python-virtualenv (Ubuntu):
assignee: wgrant → doko
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package python-setuptools - 0.6c9-0ubuntu4

---------------
python-setuptools (0.6c9-0ubuntu4) jaunty; urgency=low

  * Update the installation schemes in easy_install to follow the modified
    distutils install command:
    - When the --prefix option is used for setup.py install, Use the
      `posix' scheme. LP: #344410.
    - Use the `deb_system' scheme if --install-layout=deb is specified.
    - Use the the `unix_local' scheme if neither --install-layout=deb
      nor --prefix is specified.
  * Always use the `posix' scheme for setup.py install in a virtualenv
    setup. LP: #339904.
  * Error out when easy_install tries to install into /usr, unless the
    (new) option --force-installation-into-system-dir is given (heh, this
    option name is even longer than --single-version-externally-managed).

 -- Matthias Klose <email address hidden> Sat, 04 Apr 2009 13:13:59 +0200

Changed in python-setuptools (Ubuntu):
status: New → Fix Released
Revision history for this message
Krzysztof Klimonda (kklimonda) wrote :

Ok, here is my patch.. it works on python 2.5 and python 2.6 (as python 2.4 is unsupported by Ubuntu 9.04), uses systemtools from ubuntu instead of eggs.

Revision history for this message
Krzysztof Klimonda (kklimonda) wrote :

And whole shebang.. um, i mean debdiff.

Revision history for this message
Matthias Klose (doko) wrote :

I'm not sure what this patch is trying to do. please could you explain what doesn't work with the current version, and what the patch is supposed to do? Proposing to open a separate report for this.

Revision history for this message
Krzysztof Klimonda (kklimonda) wrote :

When installed whole set (new python2.6 and python-virtualenv) everything works fine. I had this fix made before last update and then due to some mirror sync problems i got only virtualenv.
I'm not sure if the change made to python2.6 was better than fixing virtualenv itself. Checking for "real_prefix" seems a bit hackish to me. But maybe that's only because when I choose a path I stick to it ;)

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package python3.0 - 3.0.1-0ubuntu11

---------------
python3.0 (3.0.1-0ubuntu11) jaunty; urgency=low

  [Matthias Klose]
  * Update to 20090405 from the release30-maint branch.
  * Always use the `unix_prefix' scheme for setup.py install in a virtualenv
    setup. LP: #339904.
  * Revert inverting the supported/unsupported check for rtinstall.
    LP: #354858, #354228.
  * Py_DECREF: Add `do { ... } while (0)' to avoid compiler warnings.
  * Fix build failure building the documentation.

  [Marc Deslauriers]
  * debian/pyhtml2devhelp.py: Update parsing logic.

 -- Matthias Klose <email address hidden> Sun, 05 Apr 2009 15:52:57 +0200

Changed in python3.0 (Ubuntu):
status: New → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.