Merge lp:~james-page/charms/trusty/cinder/tox into lp:~openstack-charmers-archive/charms/trusty/cinder/next

[James Page]

Support for execution of lint and tests using tox

[lp:~james-page/charms/trusty/cinder/tox]

102. By James Page on 2015-07-16

Fixup lint target

[uosci-testing-bot (uosci-testing-bot)]

charm_lint_check #6281 cinder-next for james-page mp265008
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/6281/

[uosci-testing-bot (uosci-testing-bot)]

charm_unit_test #5913 cinder-next for james-page mp265008
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/5913/

[lp:~james-page/charms/trusty/cinder/tox]

103. By James Page on 2015-07-16

Switchover to using tox executing unit and lint tests

[uosci-testing-bot (uosci-testing-bot)]

charm_unit_test #5914 cinder-next for james-page mp265008
    UNIT FAIL: unit-test missing

UNIT Results (max last 2 lines):
INFO:root:Search string not found in makefile target commands.
ERROR:root:No make target was executed.

Full unit test output: http://paste.ubuntu.com/11888272/
Build: http://10.245.162.77:8080/job/charm_unit_test/5914/

[uosci-testing-bot (uosci-testing-bot)]

charm_lint_check #6282 cinder-next for james-page mp265008
    LINT FAIL: lint-test missing

LINT Results (max last 2 lines):
INFO:root:Search string not found in makefile target commands.
ERROR:root:No make target was executed.

Full lint test output: http://paste.ubuntu.com/11888273/
Build: http://10.245.162.77:8080/job/charm_lint_check/6282/

[uosci-testing-bot (uosci-testing-bot)]

charm_amulet_test #5135 cinder-next for james-page mp265008
    AMULET OK: passed

Build: http://10.245.162.77:8080/job/charm_amulet_test/5135/

[uosci-testing-bot (uosci-testing-bot)]

charm_amulet_test #5136 cinder-next for james-page mp265008
    AMULET OK: passed

Build: http://10.245.162.77:8080/job/charm_amulet_test/5136/

[Billy Olsen (billy-olsen)]

Code changes lgtm and I'd like to give it a thumbs up, but on my recently-reinstaled (aka cleanish) vivid system I get this --> http://paste.ubuntu.com/11957838/.

I suspect one of the dependencies might be slightly off ?? I'll spend more time looking at it next week since I don't think this is required for 15.07 release.

review: Needs Information

[lp:~james-page/charms/trusty/cinder/tox]

104. By James Page on 2015-09-10

Rebase

105. By James Page on 2015-09-14

Add a bit more patching when loading openstack_upgrade

[uosci-testing-bot (uosci-testing-bot)]

charm_lint_check #9930 cinder-next for james-page mp265008
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/9930/

[uosci-testing-bot (uosci-testing-bot)]

charm_unit_test #9149 cinder-next for james-page mp265008
    UNIT FAIL: unit-test failed

UNIT Results (max last 2 lines):
make: *** [test] Error 1
ERROR:root:Make target returned non-zero.

Full unit test output: http://paste.ubuntu.com/12407902/
Build: http://10.245.162.77:8080/job/charm_unit_test/9149/

[lp:~james-page/charms/trusty/cinder/tox]

106. By James Page on 2015-09-14

Tidy, fixup failing tests

[uosci-testing-bot (uosci-testing-bot)]

charm_lint_check #9933 cinder-next for james-page mp265008
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/9933/

[uosci-testing-bot (uosci-testing-bot)]

charm_unit_test #9152 cinder-next for james-page mp265008
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/9152/

[lp:~james-page/charms/trusty/cinder/tox]

107. By James Page on 2015-09-14

Use context helpers throughout

108. By James Page on 2015-09-14

Tidy lint

[James Page (james-page)]

I still see:

==============================
Failed 1 tests - output below:
==============================

unit_tests.test_cluster_hooks.TestClusterHooks.test_cluster_hook
----------------------------------------------------------------

Captured traceback:
~~~~~~~~~~~~~~~~~~~
    Traceback (most recent call last):
      File "/home/jamespage/src/charms/tox/cinder/.tox/py27-trusty/local/lib/python2.7/site-packages/mock.py", line 1201, in patched
        return func(*args, **keywargs)
      File "/home/jamespage/src/charms/tox/cinder/unit_tests/test_cluster_hooks.py", line 82, in test_cluster_hook
        self.assertEquals(ex, service.call_args_list)
      File "/usr/lib/python2.7/unittest/case.py", line 513, in assertEqual
        assertion_func(first, second, msg=msg)
      File "/usr/lib/python2.7/unittest/case.py", line 506, in _baseAssertEqual
        raise self.failureException(msg)
    AssertionError: [call('stop', 'cinder-api'), call('stop', 'cinder-volume'), call('stop', 'cinder-scheduler'), call('stop', 'haproxy'), call('stop', 'apache2'), call('start', 'cinder-api'), call('start', 'cinder-volume'), call('start', 'cinder-scheduler'), call('start', 'haproxy'), call('start', 'apache2')] != []

When run under tox (ok under make test); for the life of me I can't figure out how its working in the current codebase.

[lp:~james-page/charms/trusty/cinder/tox]

109. By James Page on 2015-09-14

Make sure that RESTART_MAP is patched consistently

[James Page (james-page)]

OK - figured that out - cinder_utils.restart_map was not being assigned a return value consistently across all import patching.

I don't really like the cross test implications of doing this outside of the test class.

[uosci-testing-bot (uosci-testing-bot)]

charm_amulet_test #6415 cinder-next for james-page mp265008
    AMULET OK: passed

Build: http://10.245.162.77:8080/job/charm_amulet_test/6415/

[James Page (james-page)]

review: Resubmit

[uosci-testing-bot (uosci-testing-bot)]

charm_lint_check #9935 cinder-next for james-page mp265008
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/9935/

[uosci-testing-bot (uosci-testing-bot)]

charm_unit_test #9153 cinder-next for james-page mp265008
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/9153/

[uosci-testing-bot (uosci-testing-bot)]

charm_amulet_test #6417 cinder-next for james-page mp265008
    AMULET OK: passed

Build: http://10.245.162.77:8080/job/charm_amulet_test/6417/

[uosci-testing-bot (uosci-testing-bot)]

charm_amulet_test #6418 cinder-next for james-page mp265008
    AMULET OK: passed

Build: http://10.245.162.77:8080/job/charm_amulet_test/6418/

[lp:~james-page/charms/trusty/cinder/tox]

110. By James Page on 2015-10-29

Rebase

[uosci-testing-bot (uosci-testing-bot)]

charm_lint_check #12852 cinder-next for james-page mp265008
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/12852/

[uosci-testing-bot (uosci-testing-bot)]

charm_unit_test #11932 cinder-next for james-page mp265008
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/11932/

[uosci-testing-bot (uosci-testing-bot)]

charm_amulet_test #7645 cinder-next for james-page mp265008
    AMULET OK: passed

Build: http://10.245.162.77:8080/job/charm_amulet_test/7645/

[lp:~james-page/charms/trusty/cinder/tox]

111. By James Page on 2015-10-30

resync tox bits

[uosci-testing-bot (uosci-testing-bot)]

charm_lint_check #12917 cinder-next for james-page mp265008
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/12917/

[uosci-testing-bot (uosci-testing-bot)]

charm_unit_test #11997 cinder-next for james-page mp265008
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/11997/

[uosci-testing-bot (uosci-testing-bot)]

charm_amulet_test #7660 cinder-next for james-page mp265008
    AMULET OK: passed

Build: http://10.245.162.77:8080/job/charm_amulet_test/7660/

[Corey Bryant (corey.bryant)]

It looks like this mp is missing (test-)requirements.txt files.

review: Needs Fixing

[Corey Bryant (corey.bryant)]

We probably want to also run flake8 against the actions/ directory.

[lp:~james-page/charms/trusty/cinder/tox]

112. By James Page on 2015-11-03

Add missing files, lint actions

[James Page (james-page)]

Missing files added and actions now being lint'ed

review: Resubmit

[Corey Bryant (corey.bryant)]

Looks good and tox tests run successfully.

review: Approve

=== modified file '.bzrignore'
--- .bzrignore	2014-07-02 08:13:36 +0000
+++ .bzrignore	2015-11-03 11:00:52 +0000
@@ -1,2 +1,4 @@
 bin
 .coverage
+.testrepository
+.tox

=== added file '.testr.conf'
--- .testr.conf	1970-01-01 00:00:00 +0000
+++ .testr.conf	2015-11-03 11:00:52 +0000
@@ -0,0 +1,8 @@
+[DEFAULT]
+test_command=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \
+             OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \
+             OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-60} \
+             ${PYTHON:-python} -m subunit.run discover -t ./ ./unit_tests $LISTOPT $IDOPTION
+
+test_id_option=--load-list $IDFILE
+test_list_option=--list

=== added file 'requirements.txt'
--- requirements.txt	1970-01-01 00:00:00 +0000
+++ requirements.txt	2015-11-03 11:00:52 +0000
@@ -0,0 +1,11 @@
+# The order of packages is significant, because pip processes them in the order
+# of appearance. Changing the order has an impact on the overall integration
+# process, which may cause wedges in the gate later.
+PyYAML>=3.1.0
+simplejson>=2.2.0
+netifaces>=0.10.4
+netaddr>=0.7.12,!=0.7.16
+Jinja2>=2.6  # BSD License (3 clause)
+six>=1.9.0
+dnspython>=1.12.0
+psutil>=1.1.1,<2.0.0

=== added file 'test-requirements.txt'
--- test-requirements.txt	1970-01-01 00:00:00 +0000
+++ test-requirements.txt	2015-11-03 11:00:52 +0000
@@ -0,0 +1,8 @@
+# The order of packages is significant, because pip processes them in the order
+# of appearance. Changing the order has an impact on the overall integration
+# process, which may cause wedges in the gate later.
+coverage>=3.6
+mock>=1.2
+flake8>=2.2.4,<=2.4.1
+os-testr>=0.4.1
+charm-tools

=== added file 'tox.ini'
--- tox.ini	1970-01-01 00:00:00 +0000
+++ tox.ini	2015-11-03 11:00:52 +0000
@@ -0,0 +1,29 @@
+[tox]
+envlist = lint,py27
+skipsdist = True
+
+[testenv]
+setenv = VIRTUAL_ENV={envdir}
+         PYTHONHASHSEED=0
+install_command =
+  pip install --allow-unverified python-apt {opts} {packages}
+commands = ostestr {posargs}
+
+[testenv:py27]
+basepython = python2.7
+deps = -r{toxinidir}/requirements.txt
+       -r{toxinidir}/test-requirements.txt
+
+[testenv:lint]
+basepython = python2.7
+deps = -r{toxinidir}/requirements.txt
+       -r{toxinidir}/test-requirements.txt
+commands = flake8 {posargs} hooks unit_tests tests actions
+           charm proof
+
+[testenv:venv]
+commands = {posargs}
+
+[flake8]
+ignore = E402,E226
+exclude = hooks/charmhelpers

=== modified file 'unit_tests/test_actions_git_reinstall.py'
--- unit_tests/test_actions_git_reinstall.py	2015-04-15 16:26:17 +0000
+++ unit_tests/test_actions_git_reinstall.py	2015-11-03 11:00:52 +0000
@@ -1,25 +1,14 @@
-from mock import patch, MagicMock
+from mock import patch
 import os
 
 os.environ['JUJU_UNIT_NAME'] = 'cinder'
 
 from test_utils import RESTART_MAP
-import cinder_utils as utils
-
-# Need to do some early patching to get the module loaded.
-_restart_map = utils.restart_map
-_register_configs = utils.register_configs
-
-utils.restart_map = MagicMock()
-utils.restart_map.return_value = RESTART_MAP
-utils.register_configs = MagicMock()
-
-import git_reinstall
-
-# Unpatch it now that its loaded.
-utils.restart_map = _restart_map
-utils.register_configs = _register_configs
-
+
+with patch('cinder_utils.register_configs') as register_configs:
+    with patch('cinder_utils.restart_map') as restart_map:
+        restart_map.return_value = RESTART_MAP
+        import git_reinstall
 
 from test_utils import (
     CharmTestCase

=== modified file 'unit_tests/test_actions_openstack_upgrade.py'
--- unit_tests/test_actions_openstack_upgrade.py	2015-09-22 21:03:02 +0000
+++ unit_tests/test_actions_openstack_upgrade.py	2015-11-03 11:00:52 +0000
@@ -3,13 +3,16 @@
 
 os.environ['JUJU_UNIT_NAME'] = 'cinder'
 
-with patch('cinder_utils.register_configs') as register_configs:
-    import openstack_upgrade
-
 from test_utils import (
-    CharmTestCase
+    CharmTestCase,
+    RESTART_MAP
 )
 
+with patch('cinder_utils.register_configs') as register_configs:
+    with patch('cinder_utils.restart_map') as restart_map:
+        restart_map.return_value = RESTART_MAP
+        import openstack_upgrade
+
 TO_PATCH = [
     'config_changed',
     'do_openstack_upgrade',

=== modified file 'unit_tests/test_cinder_hooks.py'
--- unit_tests/test_cinder_hooks.py	2015-09-12 06:32:34 +0000
+++ unit_tests/test_cinder_hooks.py	2015-11-03 11:00:52 +0000
@@ -6,28 +6,18 @@
 )
 import yaml
 
-import cinder_utils as utils
 from test_utils import (
     CharmTestCase,
-    RESTART_MAP
+    RESTART_MAP,
 )
 
-# Need to do some early patching to get the module loaded.
-_restart_map = utils.restart_map
-_register_configs = utils.register_configs
-
-utils.restart_map = MagicMock()
-utils.restart_map.return_value = RESTART_MAP
-utils.register_configs = MagicMock()
-
-import cinder_hooks as hooks
-hooks.hooks._config_save = False
-
-hooks.hooks._config_save = False
-
-# Unpatch it now that its loaded.
-utils.restart_map = _restart_map
-utils.register_configs = _register_configs
+with patch('cinder_utils.register_configs') as register_configs:
+    with patch('cinder_utils.restart_map') as restart_map:
+        restart_map.return_value = RESTART_MAP
+        import cinder_hooks as hooks
+
+hooks.hooks._config_save = False
+import cinder_utils as utils
 
 TO_PATCH = [
     'check_call',
@@ -366,9 +356,10 @@
         self.CONFIGS.complete_contexts.return_value = ['https']
         self.relation_ids.return_value = ['identity-service:0']
         hooks.configure_https()
-        calls = [call('a2dissite', 'openstack_https_frontend'),
-                 call('service', 'apache2', 'reload')]
-        self.check_call.assert_called_has_calls(calls)
+        self.check_call.assert_called_with(['a2ensite',
+                                            'openstack_https_frontend'])
+        self.service_reload.assert_called_with('apache2',
+                                               restart_on_failure=True)
         identity_joined.assert_called_with(rid='identity-service:0')
 
     @patch.object(hooks, 'identity_joined')
@@ -377,9 +368,10 @@
         self.CONFIGS.complete_contexts.return_value = []
         self.relation_ids.return_value = ['identity-service:0']
         hooks.configure_https()
-        calls = [call('a2dissite', 'openstack_https_frontend'),
-                 call('service', 'apache2', 'reload')]
-        self.check_call.assert_called_has_calls(calls)
+        self.check_call.assert_called_with(['a2dissite',
+                                            'openstack_https_frontend'])
+        self.service_reload.assert_called_with('apache2',
+                                               restart_on_failure=True)
         identity_joined.assert_called_with(rid='identity-service:0')
 
     def test_image_service_changed(self):
@@ -425,7 +417,7 @@
         self.test_config.set('prefer-ipv6', True)
         self.test_config.set('vip', 'dummy_vip')
         hooks.hooks.execute(['hooks/shared-db-relation-joined'])
-        self.sync_db_with_multi_ipv6_addresses.assert_called_with_once(
+        self.sync_db_with_multi_ipv6_addresses.assert_called_with(
             'cinder', 'cinder')
 
     def test_db_joined_with_postgresql(self):

=== modified file 'unit_tests/test_cluster_hooks.py'
--- unit_tests/test_cluster_hooks.py	2015-06-19 16:29:55 +0000
+++ unit_tests/test_cluster_hooks.py	2015-11-03 11:00:52 +0000
@@ -1,28 +1,20 @@
 import os
-from mock import MagicMock, patch, call
-
-os.environ['JUJU_UNIT_NAME'] = 'cinder'
-import cinder_utils as utils
-
-# Need to do some early patching to get the module loaded.
-# _restart_map = utils.restart_map
-_register_configs = utils.register_configs
-_service_enabled = utils.service_enabled
-utils.register_configs = MagicMock()
-utils.service_enabled = MagicMock()
-
-import cinder_hooks as hooks
-hooks.hooks._config_save = False
-
-# Unpatch it now that its loaded.
-utils.register_configs = _register_configs
-utils.service_enabled = _service_enabled
+from mock import patch, call
 
 from test_utils import (
     CharmTestCase,
     RESTART_MAP,
 )
 
+os.environ['JUJU_UNIT_NAME'] = 'cinder'
+
+with patch('cinder_utils.register_configs') as register_configs:
+    with patch('cinder_utils.restart_map') as restart_map:
+        restart_map.return_value = RESTART_MAP
+        import cinder_hooks as hooks
+
+hooks.hooks._config_save = False
+
 TO_PATCH = [
     # cinder_utils
     'determine_packages',