Merge lp:~jseutter/testtools/py3_again into lp:~testtools-committers/testtools/trunk

[Jerry Seutter]

This is a new MP based off https://code.launchpad.net/~jseutter/testtools/py3/+merge/105505 that does not have files needlessly removed and added again.

This change adds support for building and installing using python 3. Previously this package could be used in both python 2 and python 3, but a "python setup.py install" using python 3 would fail when trying to compile _compat2x.py.

get_revno() will now silently fail if bzrlib is not present, as bzrlib is not yet available in python 3.

[Robert Collins (lifeless)]

the change to raise is invalid - it causes the wrong exception to be raised. AFAIK there is no way to do it compatibly with python 3, which is why that file isn't menat to be compiled on python3.

We don't want get_revno() to silently fail if bzrlib is not present - get_revno is required for non-final versions.

The change to not catch SyntaxError makes sense, but I'm sorry, the other bits do not. I'll cherrypick that component into trunk tomorrowish. I'm marking this needs-fixing so noone just-merges it, but you don't need to do furhter work - I'll cherrypick the bits out. If you want to figure out how to not install _compat2x.py on python3, that would be excellent, and probably fix the bytecompilation error.

>>> try:
... try:
... raise ValueError("foo")
... except ValueError:
... raise sys.exc_info()
... except ValueError, e:
... print e, type(e)
...
 <type 'exceptions.ValueError'>
>>> try:
... try:
... raise ValueError("foo")
... except ValueError:
... c,o,t = sys.exc_info()
... raise c,o,t
... except ValueError, e:
... print e, type(e)
...
foo <type 'exceptions.ValueError'>

review: Needs Fixing

[Barry Warsaw (barry)]

On May 13, 2012, at 05:42 AM, Robert Collins wrote:

>the change to raise is invalid - it causes the wrong exception to be
>raised. AFAIK there is no way to do it compatibly with python 3, which is why
>that file isn't menat to be compiled on python3.

You could use the six package, which supports a version compatible reraise():

http://packages.python.org/six/#syntax-compatibility

The implementation is fairly hairy, so it's probably not worth lifting that
from the package.

[Jerry Seutter (jseutter)]

> On May 13, 2012, at 05:42 AM, Robert Collins wrote:
>
> >the change to raise is invalid - it causes the wrong exception to be
> >raised. AFAIK there is no way to do it compatibly with python 3, which is why
> >that file isn't menat to be compiled on python3.
>
> You could use the six package, which supports a version compatible reraise():
>
> http://packages.python.org/six/#syntax-compatibility
>
> The implementation is fairly hairy, so it's probably not worth lifting that
> from the package.

I wish I understood Python packaging better so I could tell Python not to compile/install certain .py files. In practice python 2 and 3 are going to be installed in different places so there isn't much point in deploying both files all the time. I'll see if I can figure out how to do this.

Thanks for identifying the bug I introduced.

[Barry Warsaw (barry)]

FWIW, I think this is a reasonable workaround for when bzrlib isn't available. At least it's no worse than just returning None unconditionally.

=== modified file 'setup.py'
--- setup.py 2012-05-12 05:21:17 +0000
+++ setup.py 2012-05-16 17:49:49 +0000
@@ -13,6 +13,11 @@
         import bzrlib.errors
         import bzrlib.workingtree
     except ImportError:
+ from subprocess import Popen, PIPE
+ proc = Popen(['bzr', 'revno'], stdout=PIPE)
+ stdout, stderr = proc.communicate()
+ if proc.returncode == 0:
+ return int(stdout.strip())
         return None
     try:
         t = bzrlib.workingtree.WorkingTree.open_containing(__file__)[0]

[Barry Warsaw (barry)]

Yay for Launchpad formatting.

BTW, what's the status of this? Getting a Python3 compatible testtools is on the critical path for a Python 3 compatible launchpadlib.

[Barry Warsaw (barry)]

As far as the byte compilation errors, the easiest thing to do is just ignore them. It won't matter anyway, since as you say, the module won't get imported in Python 3 anyway. If you *really* wanted to fix it I suppose you could hack the MANIFEST file after it gets generated (maybe by a custom distutils subclass), I doubt it's worth the effort.

[Barry Warsaw (barry)]

My branch building on Jerry's. It reverts the _compat2x.py change and includes the setup.py change.

lp:~barry/testtools/py3

[Barry Warsaw (barry)]

On May 16, 2012, at 05:29 PM, Jerry Seutter wrote:

>I wish I understood Python packaging better so I could tell Python not to
>compile/install certain .py files. In practice python 2 and 3 are going to
>be installed in different places so there isn't much point in deploying both
>files all the time. I'll see if I can figure out how to do this.

I just spent an hour trying a bunch of different ugly kludges to avoid byte
compiling _compat2x.py when installing with Python 3, with no luck. I've sent
a message off to the python-porting mailing list, though if I don't get a good
answer there, I'll re-post it to distutils-sig. I'll follow up here if I get
any good advice; it would make for a nice item in my porting quick reference.

[Jerry Seutter (jseutter)]

I've merged in Barry's changes from lp:~barry/testtools/py3.

[lp:~jseutter/testtools/py3_again]

262. By Jerry Seutter on 2012-05-18

Merging in Barry's fixes to setup.py and _compat2x.py.

Unmerged revisions

262. By Jerry Seutter on 2012-05-18

Merging in Barry's fixes to setup.py and _compat2x.py.

261. By Jerry Seutter on 2012-05-12

Adding Python 3 support.

=== modified file 'setup.py'
--- setup.py	2011-11-25 18:24:10 +0000
+++ setup.py	2012-05-18 05:55:21 +0000
@@ -9,8 +9,16 @@
 
 
 def get_revno():
-    import bzrlib.errors
-    import bzrlib.workingtree
+    try:
+        import bzrlib.errors
+        import bzrlib.workingtree
+    except ImportError:
+        from subprocess import Popen, PIPE
+        proc = Popen(['bzr', 'revno'], stdout=PIPE)
+        stdout, stderr = proc.communicate()
+        if proc.returncode == 0:
+            return int(stdout.strip())
+        return None
     try:
         t = bzrlib.workingtree.WorkingTree.open_containing(__file__)[0]
     except (bzrlib.errors.NotBranchError, bzrlib.errors.NoWorkingTree):

=== modified file 'testtools/_compat2x.py'
--- testtools/_compat2x.py	2011-07-26 23:08:51 +0000
+++ testtools/_compat2x.py	2012-05-18 05:55:21 +0000
@@ -14,4 +14,3 @@
 def reraise(exc_class, exc_obj, exc_tb, _marker=object()):
     """Re-raise an exception received from sys.exc_info() or similar."""
     raise exc_class, exc_obj, exc_tb
-

=== modified file 'testtools/compat.py'
--- testtools/compat.py	2011-12-05 15:21:33 +0000
+++ testtools/compat.py	2012-05-18 05:55:21 +0000
@@ -32,10 +32,10 @@
 BytesIO = try_imports(['StringIO.StringIO', 'io.BytesIO'])
 StringIO = try_imports(['StringIO.StringIO', 'io.StringIO'])
 
-try:
+if sys.version_info < (3, 0):
     from testtools import _compat2x as _compat
     _compat
-except SyntaxError:
+else:
     from testtools import _compat3x as _compat
 
 reraise = _compat.reraise