Merge lp:~barry/oneconf/lp1088536 into lp:oneconf

[Barry Warsaw]

Update packaging for Python 3 port.

[Barry Warsaw (barry)]

I'm turning this back to a Work In Progress. The issue is that Software Center imports oneconf modules directly, and because it's port to Python 3 is blocked on Xapian (sigh), SC for now still needs Python 2 versions of the modules.

What I propose is to split oneconf into three binary packages:

python-oneconf - the Python 2 version of the core library, usable by SC until its port is complete
python3-oneconf - the Python 3 version of the core library
oneconf - the bin scripts and other stuff

[Didier Roche-Tolomelli (didrocks)]

That's fine with me :)

[Barry Warsaw (barry)]

Okay, I've implemented the package split, so now Software Center can depend on `python-oneconf` while oneconf itself can depend on `python3-oneconf`.

I think this is ready for review now, but please do your own testing! Note that I've enabled two out of three of the unittests during build time. I couldn't figure out how to get test_daemon to run properly in the chroot, but if you have ideas please let me know.

[Didier Roche-Tolomelli (didrocks)]

This looks good to me!
small things to change IMHO:
- can you move debhelper to version 9 instead of 8? Let's use the latest and greatest :)
- I think short description should be different between binary package, maybe adding pythonX lib?
- We should add the tests (and so test.test_daemon) as an autopkg tests to ensure depends don't break oneconf :)

Thanks a lot!

review: Needs Fixing

[lp:~barry/oneconf/lp1088536]

273. By Barry Warsaw on 2013-01-14

- debhelper 9
- Include the Python version in the short description

274. By Barry Warsaw on 2013-01-15

Preliminary DEP 8 autopkgtests.

275. By Barry Warsaw on 2013-01-15

* Don't write files if we're testing.
* Use all test dependencies.

276. By Barry Warsaw on 2013-01-15

Better; don't write files if not building.

277. By Barry Warsaw on 2013-01-15

Add restrictions

[Barry Warsaw (barry)]

On Jan 14, 2013, at 11:15 PM, Didier Roche wrote:

>- can you move debhelper to version 9 instead of 8? Let's use the latest and
>- greatest :)

Done. Easy.

>- I think short description should be different between binary package, maybe
>- adding pythonX lib?

Done. Easy.

>- We should add the tests (and so test.test_daemon) as an autopkg tests to
>- ensure depends don't break oneconf :)

Not so easy. :)

I think most of my problem is not really having written any autopkgtests
previously, so I'm glad for the opportunity to learn how to do this.
Unfortunately, it seems there are plenty of obstacles, such as the long
turnaround times for tweaking the test, and the (current) need for a writable
build directory to run setup.py (since it writes version.py and the dbus .conf
file, not to mention wanting to delete the .egg directories).

Anyway, I'll keep playing with this and see if I can get a usable
autopkgtest.

[lp:~barry/oneconf/lp1088536]

278. By Barry Warsaw on 2013-01-16

* Complete the working autotests:
 - we need a writable build tree
 - add xvfb to the dependencies so we can run the test_daemon with an X display
 - disable one test that fails under autopkgtest

279. By Barry Warsaw on 2013-01-16

debian/tests/*: Enable DEP 8 autopkgtests.

[Barry Warsaw (barry)]

I got the DEP 8 tests working, although I had to disable one unittest in test_daemon.py. I wasn't able to get this test to pass both under binary package build time and under autopkgtests. Please take a look, but it seemed not terribly important, so I just commented it out.

This should address all the issues with the packaging. I'm happy to merge and upload it after your positive review (of course, it'll have to get approved by the archive admins due to the new binary packages).

[Didier Roche-Tolomelli (didrocks)]

That's perfect! Thanks a bunch again Barry, feel free to go ahead and push it :)

review: Approve

[Barry Warsaw (barry)]

On Jan 16, 2013, at 05:21 PM, Didier Roche wrote:

>That's perfect! Thanks a bunch again Barry, feel free to go ahead and push it
>:)

Thanks! I'll push it as soon as I submit an mp for LP: #1100408 (S-C Depends:
python-oneconf).

=== modified file 'debian/changelog'
--- debian/changelog	2012-06-08 13:30:08 +0000
+++ debian/changelog	2013-01-16 03:10:27 +0000
@@ -1,3 +1,26 @@
+oneconf (0.3) raring; urgency=low
+
+  * Port to Python 3 and split the packaging:
+    - debian/compat: Bump to 8.
+    - debian/control:
+      + debhelper >= 8
+      + Add python3-* build dependencies.
+      + Add new binary packages: oneconf-common, python-oneconf,
+        python3-oneconf, oneconf (the latter has the executables).
+      + Remove python-imaging since PIL is not available for Python 3.
+      + Switch to python3-oauthlib.
+    - debian/*.install: Added for new binary package layout.
+    - debian/*.links: Added.
+    - debian/rules:
+      + Build for both Python 2 and Python 3.
+      + Run the test suite where possible.
+    - debian/tests/*: Enable DEP 8 autopkgtests.
+    - oneconf/version.py:
+      + Bump to 0.3
+      + Codename: raring
+
+ -- Barry Warsaw <[e-mail]>  Tue, 15 Jan 2013 10:52:09 -0500
+
 oneconf (0.2.9.1) quantal; urgency=low
 
   * handle a regression when the wallpaper isn't reachable + test case
 
=== modified file 'debian/compat'
--- debian/compat	2010-07-29 15:08:16 +0000
+++ debian/compat	2013-01-16 03:10:27 +0000
@@ -1,1 +1,1 @@
-7
+9
 
=== modified file 'debian/control'
--- debian/control	2012-04-12 14:45:36 +0000
+++ debian/control	2013-01-16 03:10:27 +0000
@@ -1,27 +1,106 @@
 Source: oneconf
 Section: python
 Priority: extra
-Build-Depends: debhelper (>= 7.0.50),
+Build-Depends: debhelper (>= 9),
                lsb-release,
                gettext,
                intltool,
+               xvfb,
                python,
+               python-setuptools,
                python-distutils-extra,
                python-nose,
+               python-mock,
+               python-xdg,
+               python-gi,
+               python-piston-mini-client,
+               python-dbus,
+               python3,
+               python3-setuptools,
+               python3-distutils-extra,
+               python3-nose,
+               python3-mock,
+               python3-xdg,
+               python3-gi,
+               python3-piston-mini-client,
+               python3-dbus
 Maintainer: Didier Roche <[e-mail]>
-Standards-Version: 3.9.3
+Standards-Version: 3.9.4
+X-Python-Version: >= 2.7
+X-Python3-Version: >= 3.3
+XS-Testsuite: autopkgtest
+
+Package: oneconf-common
+Architecture: all
+Depends: ${misc:Depends},
+         ${python:Depends},
+         python-dbus,
+         python-oauthlib,
+         python-gi,
+         python-apt,
+         python-piston-mini-client,
+         python-httplib2,
+         python-xdg
+Description: synchronize your configuration data over the network
+ OneConf provides the ability to sync your computer's configuration data over
+ the network.
+ .
+ It integrates nicely with the Ubuntu Software Center to compare sets of
+ installed software between computers. A command line tool also provides for
+ the same functionality.
+ .
+ This is the packaging containing common files.
+
+Package: python-oneconf
+Architecture: all
+Depends: ${misc:Depends},
+         ${python:Depends},
+         oneconf-common,
+         python-dbus,
+         python-oauthlib,
+         python-gi,
+         python-apt,
+         python-piston-mini-client,
+         python-httplib2,
+         python-xdg
+Description: synchronize your configuration data over the network (Python 2)
+ OneConf provides the ability to sync your computer's configuration data over
+ the network.
+ .
+ It integrates nicely with the Ubuntu Software Center to compare sets of
+ installed software between computers. A command line tool also provides for
+ the same functionality.
+ .
+ This is the Python 2 support library.
+
+Package: python3-oneconf
+Architecture: all
+Depends: ${misc:Depends},
+         ${python3:Depends},
+         oneconf-common,
+         python3-dbus,
+         python3-oauthlib,
+         python3-gi,
+         python3-apt,
+         python3-piston-mini-client,
+         python3-httplib2,
+         python3-xdg
+Description: synchronize your configuration data over the network (Python 3)
+ OneConf provides the ability to sync your computer's configuration data over
+ the network.
+ .
+ It integrates nicely with the Ubuntu Software Center to compare sets of
+ installed software between computers. A command line tool also provides for
+ the same functionality.
+ .
+ This is the Python 3 support library.
 
 Package: oneconf
 Architecture: all
 Depends: ${misc:Depends},
-         ${python:Depends},
-         python-dbus,
-         python-oauth,
-         python-gi,
-         python-apt,
-         ubuntu-sso-client,
-         python-piston-mini-client (>= 0.3+bzr32-0ubuntu1),
-         python-imaging,
+         oneconf-common,
+         python3-oneconf,
+         ubuntu-sso-client
 Recommends: software-center (>= 4.1.21),
             update-notifier (>= 0.103),
 Description: synchronize your configuration data over the network
 
=== added file 'debian/oneconf-common.install'
--- debian/oneconf-common.install	1970-01-01 00:00:00 +0000
+++ debian/oneconf-common.install	2013-01-16 03:10:27 +0000
@@ -0,0 +1,1 @@
+usr/share
 
=== added file 'debian/oneconf.install'
--- debian/oneconf.install	1970-01-01 00:00:00 +0000
+++ debian/oneconf.install	2013-01-16 03:10:27 +0000
@@ -0,0 +1,1 @@
+usr/bin
 
=== added file 'debian/oneconf.links'
--- debian/oneconf.links	1970-01-01 00:00:00 +0000
+++ debian/oneconf.links	2013-01-16 03:10:27 +0000
@@ -0,0 +1,2 @@
+usr/share/oneconf/oneconf-query usr/bin/oneconf-query
+usr/share/oneconf/oneconf-update usr/share/update-notifier/plugins/cache-changed/oneconf-update
 
=== added file 'debian/python-oneconf.install'
--- debian/python-oneconf.install	1970-01-01 00:00:00 +0000
+++ debian/python-oneconf.install	2013-01-16 03:10:27 +0000
@@ -0,0 +1,1 @@
+usr/lib/python2.*/dist-packages
 
=== added file 'debian/python3-oneconf.install'
--- debian/python3-oneconf.install	1970-01-01 00:00:00 +0000
+++ debian/python3-oneconf.install	2013-01-16 03:10:27 +0000
@@ -0,0 +1,1 @@
+usr/lib/python3
 
=== modified file 'debian/rules'
--- debian/rules	2011-09-01 08:47:01 +0000
+++ debian/rules	2013-01-16 03:10:27 +0000
@@ -1,22 +1,51 @@
 #!/usr/bin/make -f
+
+#DH_VERBOSE=1
 share:=usr/share/oneconf
 
+# Prevent setuptools/distribute from accessing the internet.
+export http_proxy = http://127.0.9.1:9
+
+PYTHON2=$(shell pyversions -vr)
+PYTHON3=$(shell py3versions -vr)
+
 %:
-	dh --with=python2 $@
-
-# Install the modules in a private directory, as our applications are the
-# only ones using them.
-override_dh_auto_install:
+	dh $@ --with python2,python3
+
+ifeq (,$(filter nocheck,$(DEB_BUILD_OPTIONS)))
+test-python%:
+	# The tests don't work well when run as one unit, so run them
+	# individually.
+	python$* setup.py nosetests -vv --test test.test_mainfeatures
+	python$* setup.py nosetests -vv --test test.test_syncing
+	# I don't know of a good way to run this test in the schroot, even
+	# using xvfb to give us an X display.  Rely on DEP 8 tests to cover
+	# these tests.
+	#xvfb-run python$* setup.py nosetests -vv --test test.test_daemon
+
+override_dh_auto_test: $(PYTHON2:%=test-python%) $(PYTHON3:%=test-python%)
+endif
+
+build-python%:
+	python$* setup.py build
+
+override_dh_auto_build: $(PYTHON3:%=build-python%)
+	dh_auto_build
+
+install-python%:
+	python$* setup.py install --root=$(CURDIR)/debian/tmp --install-layout=deb
+
+override_dh_auto_install: $(PYTHON3:%=install-python%)
 	dh_auto_install
-	# Move the scripts to $(SHARE) and relink some again later.
-	mv debian/oneconf/usr/*bin/* debian/oneconf/$(share)
-	rmdir debian/oneconf/usr/*bin
+	# Move the scripts to $(SHARE) and relink some again later.  The
+	# scripts will be included in the "oneconf" binary package
+	mkdir -p debian/oneconf/$(share)
+	mv debian/tmp/usr/*bin/* debian/oneconf/$(share)
 
+override_dh_auto_clean:
+	dh_auto_clean
+	rm -rf build
+	rm -rf *.egg-info
 
 override_dh_install:
 	dh_install --fail-missing
-
-	dh_link $(share)/oneconf-query usr/bin/oneconf-query \
-		$(share)/oneconf-update usr/share/update-notifier/plugins/cache-changed/oneconf-update
-
-.PHONY: override_dh_auto_install
 
=== added directory 'debian/tests'
=== added file 'debian/tests/control'
--- debian/tests/control	1970-01-01 00:00:00 +0000
+++ debian/tests/control	2013-01-16 03:10:27 +0000
@@ -0,0 +1,3 @@
+Tests: unittests
+Depends: @, xvfb, python3-nose, python3-mock
+Restrictions: rw-build-tree build-needed
 
=== added file 'debian/tests/unittests'
--- debian/tests/unittests	1970-01-01 00:00:00 +0000
+++ debian/tests/unittests	2013-01-16 03:10:27 +0000
@@ -0,0 +1,13 @@
+#!/bin/sh
+# autopkgtest check: Run the upstream unittests.
+# (C) 2013 Canonical Ltd.
+# Author: Barry Warsaw <[e-mail]>
+
+set -ex
+
+# Run the tests individually, since running them via the tests/run script does
+# not always complete.
+
+python3 setup.py nosetests -vv --test test.test_mainfeatures
+python3 setup.py nosetests -vv --test test.test_syncing
+xvfb-run python3 setup.py nosetests -vv --test test.test_daemon
 
=== modified file 'oneconf/version.py'
--- oneconf/version.py	2012-11-21 20:01:42 +0000
+++ oneconf/version.py	2013-01-16 03:10:27 +0000
@@ -1,5 +1,5 @@
 
-VERSION='0.2.9'
-CODENAME='quantal'
+VERSION='0.3'
+CODENAME='raring'
 DISTRO='Ubuntu'
 RELEASE='13.04'
 
=== modified file 'setup.py'
--- setup.py	2012-11-29 23:17:54 +0000
+++ setup.py	2013-01-16 03:10:27 +0000
@@ -4,6 +4,7 @@
 from DistUtilsExtra.command import *
 
 import re
+import sys
 import glob
 from codecs import open
 from subprocess import Popen, PIPE
@@ -22,21 +23,24 @@
                 stdout=PIPE,
                 universal_newlines=True).communicate()[0].strip()
 
-with open('oneconf/version.py', 'w', encoding='utf-8') as fp:
-    fp.write("""
+#should be replaced by $USR
+oneconf_service_path = "/usr/share/oneconf/oneconf-service"
+
+# Only write the files if we're building.
+if any(argv for argv in sys.argv if 'build' in argv):
+    with open('oneconf/version.py', 'w', encoding='utf-8') as fp:
+        fp.write("""\
 VERSION='%s'
 CODENAME='%s'
 DISTRO='%s'
 RELEASE='%s'
 """ % (VERSION, CODENAME, DISTRO, RELEASE))
-
-#should be replaced by $USR
-oneconf_service_path = "/usr/share/oneconf/oneconf-service"
-
-with open('misc/com.ubuntu.OneConf.service', 'w', encoding='utf-8') as fp:
-    fp.write("""[D-BUS Service]
+    with open('misc/com.ubuntu.OneConf.service', 'w', encoding='utf-8') as fp:
+        fp.write("""\
+[D-BUS Service]
 Name=com.ubuntu.OneConf
-Exec=%s""" % oneconf_service_path)
+Exec=%s
+""" % oneconf_service_path)
 
 # real setup
 setup(name="oneconf", version=VERSION,
 
=== modified file 'test/test_daemon.py'
--- test/test_daemon.py	2012-12-13 23:37:23 +0000
+++ test/test_daemon.py	2013-01-16 03:10:27 +0000
@@ -105,10 +105,12 @@
             daemon2 = subprocess.Popen(["./oneconf-service"],
                                        stdout=devnull, stderr=devnull)
             daemon2.wait() # let it proceeding quitting
-            time_stop = time.time()
+            #time_stop = time.time()
             self.assertFalse(self.daemon_still_there(daemon2.pid))
-            self.assertTrue(
-                time_stop - self.time_start < MIN_TIME_WITHOUT_ACTIVITY)
+            # This assertion is unreliable, especially under DEP 8
+            # autopkgtests and possibly other limited environments.
+            ## self.assertLess(time_stop - self.time_start,
+            ##                 MIN_TIME_WITHOUT_ACTIVITY)
         finally:
             if close:
                 devnull.close()