Merge ~smoser/cloud-init:bug/package-build-no-stylechecks into cloud-init:master

[Scott Moser]

tox/build: do not package depend on style requirements.

When the style/checking dependencies were updated in
test-requirements.txt, the debian package build dependencies created
by ./packages/bddeb were also updated. Pycodestyle was added to the list
in order to pin its version. That broke the package build for 16.04. The
reason for this is simply that python3-pycodestyle is not available in
16.04.

The change here is to remove style dependencies from test-requirements,
and add them to the tox environments directly. We had previously changed
the package build process to not run pep8 or flake8 simply to avoid having
to code to N different versions of style checkers (3bcb72c593f).

The link between package build and test-requirements still exists, though.
So future breakage can occur if any package is added to
test-requirements.txt (or requirements.txt) if the target distro release
does not have a python3-<packagename> in its archive.

There is also a bit of a tox.ini cleanup here, in that we do not have to
explictly list '-rrequirements.txt' as the setup.py pulls those in. And
lastly, we drop the -rtest-requirements.txt from the base 'testenv', and
add these test requirements only to environments that need to run test.

Finally, a change to packages/debian/control.in to drop the build
dependencies that were listed for style checking and also a dependency
on iproute2 which was a bad unit test that has been previously fixed.

[Server Team CI bot (server-team-bot)]

FAILED: Continuous integration, rev:e99f746105fc605e99eb792d3b17dae9240c099a
No commit message was specified in the merge proposal. Click on the following link and set the commit message (if you want a jenkins rebuild you need to trigger it yourself):
https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/324541/+edit-commit-message

https://jenkins.ubuntu.com/server/job/cloud-init-ci/389/
Executed test runs:
    FAILURE: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-amd64/389/console
    FAILURE: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-arm64/389/console
    FAILURE: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-ppc64el/389/console
    FAILURE: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-s390x/389/console
    FAILURE: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=vm-i386/389/console

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/389/rebuild

review: Needs Fixing (continuous-integration)

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/390/
Executed test runs:
    SUCCESS: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-amd64/390
    SUCCESS: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-arm64/390
    SUCCESS: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-ppc64el/390
    SUCCESS: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-s390x/390
    SUCCESS: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=vm-i386/390

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/390/rebuild

review: Approve (continuous-integration)

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:28732bbc4c9a85d48447a4f2a830a415c42ba813
https://jenkins.ubuntu.com/server/job/cloud-init-ci/391/
Executed test runs:
    SUCCESS: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-amd64/391
    SUCCESS: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-arm64/391
    SUCCESS: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-ppc64el/391
    SUCCESS: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-s390x/391
    SUCCESS: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=vm-i386/391

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/391/rebuild

review: Approve (continuous-integration)

[Joshua Powers (powersj)]

+1 passes CI, build test working again, all tox envs, except centos6, look good. Thanks for doing this.

The tox centos6 env, tested on centos 6 lxd, fail with the following, and I can get these cleaned up in a separate merge.

======================================================================
ERROR: Test read_user_data_callback() no files are found.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/root/cloud-init/tests/unittests/test_datasource/test_altcloud.py", line 422, in test_callback_none
    self.assertIsNone(dsac.read_user_data_callback(self.mount_dir))
AttributeError: 'TestReadUserDataCallback' object has no attribute 'assertIsNone'

======================================================================
FAIL: test_scripts_shebanged (tests.unittests.test_datasource.test_smartos.TestSmartOSDataSource)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/root/cloud-init/tests/unittests/test_datasource/test_smartos.py", line 465, in test_scripts_shebanged
    self.assertEqual(shebang, "#!/bin/bash")
AssertionError: '/bin/true' != '#!/bin/bash'
- /bin/true
+ #!/bin/bash

[Joshua Powers (powersj)]

review: Approve

diff --git a/packages/debian/control.in b/packages/debian/control.in
index b58561e..6c39d53 100644
--- a/packages/debian/control.in
+++ b/packages/debian/control.in
@@ -6,10 +6,6 @@ Maintainer: Scott Moser <[e-mail]>
 Build-Depends: debhelper (>= 9),
                dh-python,
                dh-systemd,
-               iproute2,
-               pep8,
-               pyflakes,
-               python3-pyflakes | pyflakes (<< 1.1.0-2),
                ${python},
                ${test_requires},
                ${requires}
diff --git a/test-requirements.txt b/test-requirements.txt
index 1b39ea5..d9d41b5 100644
--- a/test-requirements.txt
+++ b/test-requirements.txt
@@ -11,9 +11,3 @@ coverage
 # Only really needed on older versions of python
 contextlib2
 setuptools
-
-# Used for syle checking
-pycodestyle==2.3.1
-pyflakes==1.5.0
-flake8==3.3.0
-hacking==0.13.0
diff --git a/tox.ini b/tox.ini
index 03bb5f1..6276662 100644
--- a/tox.ini
+++ b/tox.ini
@@ -4,13 +4,16 @@ recreate = True
 
 [testenv]
 commands = python -m nose {posargs:tests/unittests}
-deps = -r{toxinidir}/test-requirements.txt
-       -r{toxinidir}/requirements.txt
 setenv =
     LC_ALL = en_US.utf-8
 
 [testenv:flake8]
 basepython = python3
+deps =
+    pycodestyle==2.3.1
+    pyflakes==1.5.0
+    flake8==3.3.0
+    hacking==0.13.0
 commands = {envpython} -m flake8 {posargs:cloudinit/ tests/ tools/}
 
 # https://github.com/gabrielfalcao/HTTPretty/issues/223
@@ -23,11 +26,17 @@ commands = {envpython} -m pylint {posargs:cloudinit}
 
 [testenv:py3]
 basepython = python3
+deps = -r{toxinidir}/test-requirements.txt
 commands = {envpython} -m nose {posargs:--with-coverage \
            --cover-erase --cover-branches --cover-inclusive \
            --cover-package=cloudinit tests/unittests}
 
+[testenv:py27]
+basepython = python2.7
+deps = -r{toxinidir}/test-requirements.txt
+
 [testenv:py26]
+deps = -r{toxinidir}/test-requirements.txt
 commands = nosetests {posargs:tests/unittests}
 setenv =
     LC_ALL = C
@@ -39,8 +48,7 @@ exclude = .venv,.tox,dist,doc,*egg,.git,build,tools
 
 [testenv:doc]
 basepython = python3
-deps = {[testenv]deps}
-       sphinx
+deps = sphinx
 commands = {envpython} -m sphinx {posargs:doc/rtd doc/rtd_html}
 
 [testenv:xenial]
@@ -63,10 +71,6 @@ deps =
     nose==1.3.7
     unittest2==1.1.0
     contextlib2==0.5.1
-    pep8==1.7.0
-    pyflakes==1.1.0
-    flake8==2.5.4
-    hacking==0.10.2
 
 [testenv:centos6]
 basepython = python2.6