Merge lp:~abentley/juju-ci-tools/python-canary into lp:juju-ci-tools

[Aaron Bentley]

Use a canary to install python so that make test installs updated deps

This branch updates make test to autodetect python dependency changes.

It uses a canary to do this, and introduces a new python-deps target.

Every time one of the files the canary depends on changes, make test will rerun pipdeps.

[Curtis Hovey (sinzui)]

I think this breaks "make test" on windows and the osx slave. I do run this command to be certain the scripts can run on those hosts. Can we the apt-calls for non-ubuntu?

[Martin Packman (gz)]

I think this is okay, because only python-deps happens for tests, and just the pip install should be cross-platform correct for windows and osx?

[Aaron Bentley (abentley)]

@martin As written, the python-deps starts by installing pip using apt, so it would break on Windows.

[lp:~abentley/juju-ci-tools/python-canary]

1284. By Aaron Bentley on 2016-02-18

Merged trunk into python-canary.

1285. By Aaron Bentley on 2016-02-24

Make pip one of the equivs packages.

[Aaron Bentley (abentley)]

> I think this breaks "make test" on windows and the osx slave. I do run this
> command to be certain the scripts can run on those hosts. Can we the apt-calls
> for non-ubuntu?

I've changed it so that pip is one of the packages installed by install-deps, instead of being handled separately. Basically, if you don't install-deps, you're responsible for installing pip.

This gives PYTHON_PACKAGE_CANARY a dependency on apt-deps, but this is an *ordering* dependency-- if make was going to run apt-deps anyhow, it runs it before PYTHON_PACKAGE_CANARY. But it won't run apt-deps unless another target (like install-deps) is required.

So:
  make install-deps:
    1. make apt-deps
    2. make python-deps

  make test
    1. make python-deps
    2. python -m unittest ...

I think this addresses the concern of unwanted apt operations on Windows and Mac and also fixes the problem that pip wasn't being handled the same way as other debian packages.

Does that work for you?

[Aaron Bentley (abentley)]

Turns out I didn't understand how ordering-only dependencies work. Sigh.

Unmerged revisions

1285. By Aaron Bentley on 2016-02-24

Make pip one of the equivs packages.

1284. By Aaron Bentley on 2016-02-18

Merged trunk into python-canary.

1283. By Aaron Bentley on 2016-02-18

Use a canary to install python so that make test installs updated deps

=== added file '.python-canary'
=== modified file 'Makefile'
--- Makefile	2015-12-11 20:38:31 +0000
+++ Makefile	2016-02-24 15:03:51 +0000
@@ -1,5 +1,5 @@
 p=test*.py
-test:
+test: python-deps
 	TMPDIR=/tmp python -m unittest discover -vv ./tests -p "$(p)"
 lint:
 	flake8 $$(find -name '*.py')
@@ -10,15 +10,19 @@
 	find . -name '*.pyc' -delete
 apt-update:
 	sudo apt-get -qq update
-juju-ci-tools.common_0.1.0-0_all.deb: apt-update
+juju-ci-tools.common_0.1.0-1_all.deb: apt-update
 	sudo apt-get install -y equivs
 	equivs-build juju-ci-tools-common
-install-deps: juju-ci-tools.common_0.1.0-0_all.deb apt-update
+PYTHON_PACKAGE_CANARY := .python-canary
+$(PYTHON_PACKAGE_CANARY): requirements.txt pipdeps.py | apt-deps
+	./pipdeps.py install
+	touch $(PYTHON_PACKAGE_CANARY)
+python-deps: $(PYTHON_PACKAGE_CANARY)
+install-deps: apt-deps python-deps
+apt-deps: juju-ci-tools.common_0.1.0-1_all.deb apt-update
 	sudo dpkg -i juju-ci-tools.common_0.1.0-0_all.deb || true
 	sudo apt-get install -y -f
 	sudo apt-get install -y juju-local juju juju-quickstart juju-deployer
-	sudo apt-get install -y python-pip
-	./pipdeps.py install
 name=NAMEHERE
 assess_file=assess_$(name).py
 test_assess_file=tests/test_assess_$(name).py
@@ -26,4 +30,4 @@
 	install -m 755 template_assess.py.tmpl $(assess_file)
 	install -m 644 template_test.py.tmpl $(test_assess_file)
 	sed -i -e "s/TEMPLATE/$(name)/g" $(assess_file) $(test_assess_file)
-.PHONY: lint test cover clean new-assess apt-update install-deps
+.PHONY: lint test cover clean new-assess apt-update install-deps python-deps apt-deps

=== modified file 'juju-ci-tools-common'
--- juju-ci-tools-common	2015-12-18 17:30:38 +0000
+++ juju-ci-tools-common	2016-02-24 15:03:51 +0000
@@ -3,8 +3,8 @@
 Standards-Version: 3.9.2
 
 Package: juju-ci-tools.common
-Version: 0.1.0-0
-Depends: s3cmd, euca2ools, uvtool-libvirt, uvtool, simplestreams, distro-info-data
+Version: 0.1.0-1
+Depends: s3cmd, euca2ools, uvtool-libvirt, uvtool, simplestreams, distro-info-data, python-pip
 Architecture: all
 Description: juju-ci-tools-common
  A metapackage providing the dependencies for juju-reports