Merge ~bryce/ubuntu/+source/nose:fix-lp1859880-py3-sphinx into ubuntu/+source/nose:ubuntu/devel

[Bryce Harrington]

Several packages are blocked in proposed migration due to having nose as a dependency for their testsuites. Nose has been ported to python3, but the issue is that its code docs are being generated using the python2 version of sphinx.

I considered a few different ways to fix this:

1. Disable code docs. Nose is no longer maintained and upstream recommends using nose2 or pytest instead, so providing code docs may not be very important anymore, as developers should probably not use it anymore. However, the module still works well enough and is widely used so this seems too brute force.
2. Carry a debian patch to convert codebase to python3. With Ubuntu moving towards python3-only, there is little need for preserving python2 support. However, such a patch would touch a lot of code files and thus be brittle and require regeneration if debian changes things.
3. Generate docs from the build output rather than the source.
4. Add 2to3 as a dependency, and run it on the codebase prior to calling sphinx.

I experimented with #3 for a while, but didn't find a clean and robust way to do it. I'm not certain it would produce exactly the same results as generation from source, although that might not matter in practice.

Option #4, running 2to3, works cleanly and is minimally invasive. The main trade-off is it adds another dependency for the package, but this seems a fair trade.

A PPA with this package is available from:
  https://launchpad.net/~bryce/+archive/ubuntu/nose-fix-lp1859880-py3-sphinx/+packages

[Dan Watkins (oddbloke)]

Being nosy (and hopefully not screwing up review slots by leaving a comment).

[Bryce Harrington (bryce)]

Thanks, I've updated the branch with that fix.

[Andreas Hasenack (ahasenack)]

python-nose has reverse-dependencies that need to be accounted for:

$ reverse-depends -s python-nose
Reverse-Suggests
* pypi2deb
* python-matplotlib [amd64 arm64 armhf ppc64el s390x]

Reverse-Depends
* python-freshen
* python-nose-json

You need a plan for these reverse deps, as just removing python-nose would break their installation and your upload won't migrate.

review: Needs Information

[Bryce Harrington (bryce)]

* python-nose-json has no rdepends itself, and could be just dropped. There is a python3-nose-json already.

* python-freshen doesn't have separate py2/py3 packages, and may work fine as is with python3, or with minimal adjustments. I will try locally updating it to py3.

* pypi2deb Suggests both python-nose and python3-nose. Since it's a Suggests it probably does not require any change, but it could perhaps have its py2 Suggests dropped.

* python-matplotlib has numerous python2 dependencies (only), however in this case python-nose is just a Suggests, so presumably won't cause a migration block?

Does the above sound adequate as a plan? Do you spot any other issues in the merge apart from these potential dependency issues?

[Andreas Hasenack (ahasenack)]

> * python-nose-json has no rdepends itself, and could be just dropped. There
> is a python3-nose-json already.

ok

>
> * python-freshen doesn't have separate py2/py3 packages, and may work fine as
> is with python3, or with minimal adjustments. I will try locally updating it
> to py3.

Last commit in https://github.com/rlisagor/freshen was in 2015

> * pypi2deb Suggests both python-nose and python3-nose. Since it's a Suggests
> it probably does not require any change, but it could perhaps have its py2
> Suggests dropped.

I don't recall if a missing suggests stops migration.

> * python-matplotlib has numerous python2 dependencies (only), however in this
> case python-nose is just a Suggests, so presumably won't cause a migration
> block?

I'm not sure

>
> Does the above sound adequate as a plan? Do you spot any other issues in the
> merge apart from these potential dependency issues?

The other check would be in the seeds, but given these are py2 dependencies, I doubt they are seeded, but would check anyway.

Finally, I suggest to bring this up in standup. Specifically about removing a package that is a Suggests-level dependency of others. It does sound wrong to keep the "Suggests" in them, if this is the route we are going to take.

[Andreas Hasenack (ahasenack)]

This also relies on the fact that the "Suggests" dependency is correct in all these cases, i.e., nothing breaks by default without the suggested package.

[Bryce Harrington (bryce)]

(For focal, nose was permitted to migrate without fixing the py2 dependence. I think that was probably a 'kicking-the-can-down-the-road' move but what's done is done. This sphinx fix is probably still relevant for improving the package, but the MP would either need to be redone to preserve py2 functionality, or more effort would need taken to remove the remaining py2 dependencies.)

review: Disapprove

Unmerged commits

76002b1... by Bryce Harrington on 2020-01-16

changelog

3b1a3f7... by Bryce Harrington on 2020-01-16

update-maintainer

5b63066... by Bryce Harrington on 2020-01-16

d/control: Use the python3 version of the coverage module

4c2ba26... by Bryce Harrington on 2020-01-16

d/control, d/rules: Use the python3 version of sphinx

Convert codebase to python3 before generating code docs

nose's root codebase has some py2-isms, which are fixed up during
setup.py's build via setup3lib.py and setuptool's 2to3 mechanisms.
Unfortunately, sphinx doesn't see these changes and tries to build docs
from the original (python2) codebase.

Previously, the python2 version of sphinx was being used to generate the
docs, so this was not a problem. However, when using python3-sphinx, it
errors when it encounters python2 source. Address this by running 2to3
over the codebase before calling sphinx.

665ffe5... by Bryce Harrington on 2020-01-15

d/rules, d/control: Drop python2 binary package

diff --git a/debian/changelog b/debian/changelog
index e1949b7..d289d22 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,12 @@
+nose (1.3.7-4ubuntu1) focal; urgency=medium
+
+  * Drop python2 support (LP: #1859880)
+    - d/control: Switch to py3 versions of coverage and sphinx modules
+    - d/rules: Force 2to3 when generating code docs with sphinx
+    - d/rules, d/control: Drop py2 binary package
+
+ -- Bryce Harrington <[e-mail]>  Thu, 16 Jan 2020 20:20:35 +0000
+
 nose (1.3.7-4build1) focal; urgency=medium
 
   * No-change rebuild to generate dependencies on python2.
diff --git a/debian/control b/debian/control
index 752b5c2..c6e7e46 100644
--- a/debian/control
+++ b/debian/control
@@ -1,7 +1,8 @@
 Source: nose
 Section: python
 Priority: optional
-Maintainer: Debian Python Modules Team <[e-mail]>
+Maintainer: Ubuntu Developers <[e-mail]>
+XSBC-Original-Maintainer: Debian Python Modules Team <[e-mail]>
 Uploaders:
  Torsten Marek <[e-mail]>,
  Dmitry Shachnev <[e-mail]>,
@@ -12,13 +13,11 @@ Build-Depends:
  libjs-jquery-hotkeys,
  libjs-jquery-isonscreen,
  libjs-jquery-tablesorter,
- python-all (>= 2.6.6-3~),
- python-setuptools (>= 0.6a9),
- python-sphinx (>= 1.0.7+dfsg),
- python-coverage,
+ python3-coverage,
+ python3-sphinx (>= 1.0.7+dfsg),
  python3-all,
- python3-setuptools
-X-Python-Version: >= 2.5
+ python3-setuptools,
+ 2to3
 X-Python3-Version: >= 3.2
 Standards-Version: 4.1.3
 Homepage: https://nose.readthedocs.io/en/latest/
@@ -37,16 +36,6 @@ Description: documentation for discovery and running for Python's unittest
  .
  This package provides the documentation for nose.
 
-Package: python-nose
-Architecture: all
-Multi-Arch: foreign
-Depends: python-pkg-resources, ${misc:Depends}, ${python:Depends}
-Suggests: python-coverage, python-nose-doc
-Description: test discovery and running of Python's unittest
- nose provides an alternate test discovery and running process for
- unittest, one that is intended to mimic the behavior of py.test as
- much as is reasonably possible without resorting to too much magic
-
 Package: python3-nose
 Architecture: all
 Multi-Arch: foreign
diff --git a/debian/rules b/debian/rules
index c48a026..c8720ed 100755
--- a/debian/rules
+++ b/debian/rules
@@ -1,30 +1,27 @@
 #!/usr/bin/make -f
 
-DESTDIR2 := $(CURDIR)/debian/python-nose
 DESTDIR3 := $(CURDIR)/debian/python3-nose
 
 export PYBUILD_NAME = nose
-export PYBUILD_TEST_ARGS_python2 = {interpreter} setup.py test
 export PYBUILD_TEST_ARGS_python3 = {interpreter} setup.py build_tests && {interpreter} selftest.py -v
 
 %:
-	dh $@ --with python2,python3,sphinxdoc --buildsystem=pybuild
+	dh $@ --with python3,sphinxdoc --buildsystem=pybuild
 
 override_dh_auto_build:
 	dh_auto_build
-	python setup.py build_sphinx
+	2to3 --write --no-diffs --nobackups nose
+	python3 setup.py build_sphinx
 
 override_dh_auto_test:
 	dh_auto_test -- --system=custom
 
 override_dh_auto_install:
 	dh_auto_install
-	sed -i -e 's,#!/usr/bin/python.*,#!/usr/bin/python,' $(DESTDIR2)/usr/bin/nosetests
 	cd $(DESTDIR3)/usr/bin/ && mv nosetests nosetests3
 	rm -f $(DESTDIR3)/usr/bin/nosetests-3.*
 	sed -i -e 's,#!/usr/bin/python3.*,#!/usr/bin/python3,' $(DESTDIR3)/usr/bin/nosetests3
 
 override_dh_installman:
-	dh_installman -p python-nose nosetests.1
 	dh_installman -p python3-nose nosetests.1
 	cd $(DESTDIR3)/usr/share/man/man1 && mv nosetests.1 nosetests3.1