Merge lp:~corey.bryant/charms/trusty/keystone/action-managed-upgrade into lp:~openstack-charmers-archive/charms/trusty/keystone/next

[Corey Bryant]

[uosci-testing-bot (uosci-testing-bot)]

charm_lint_check #10050 keystone-next for corey.bryant mp271143
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/10050/

[uosci-testing-bot (uosci-testing-bot)]

charm_unit_test #9217 keystone-next for corey.bryant mp271143
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/9217/

[uosci-testing-bot (uosci-testing-bot)]

charm_amulet_test #6437 keystone-next for corey.bryant mp271143
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/12418280/
Build: http://10.245.162.77:8080/job/charm_amulet_test/6437/

[uosci-testing-bot (uosci-testing-bot)]

charm_unit_test #9224 keystone-next for corey.bryant mp271143
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/9224/

[uosci-testing-bot (uosci-testing-bot)]

charm_lint_check #10060 keystone-next for corey.bryant mp271143
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/10060/

[uosci-testing-bot (uosci-testing-bot)]

charm_amulet_test #6447 keystone-next for corey.bryant mp271143
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/12420117/
Build: http://10.245.162.77:8080/job/charm_amulet_test/6447/

[Billy Olsen (billy-olsen)]

When I ran the upgrade action, I required a subsequent config-changed manual hook run in order to get the upgrade to work properly. The initial run of the openstack-ugprade action did upgrade all of the necessary packages, however the template rendering engine rendered kilo, then rendered icehouse templates (from which I was ugprading from). Digging into this a bit, I suspect its due to the way that the charmhelpers.contrib.openstack.utils.get_os_codename_package works. Since this is determined as part of register_configs(), which happens prior to the upgrade, the package versions are kept at the icehouse level which causes the icehouse templates to be rendered. I noticed this because I experienced bug https://bugs.launchpad.net/charms/+source/keystone/+bug/1417211 upon upgrade, then realized that this is due to rendering the icehouse template as can be seen at http://paste.ubuntu.com/12517976/.

review: Needs Fixing

[Billy Olsen (billy-olsen)]

Hmm, I see the same configs setting is on other MPs for the action ugprade as well, however they do *NOT* experience the kilo/icehouse/kilo issue. I think this warrants further investigation as to what specifically the root cause of this is.

[uosci-testing-bot (uosci-testing-bot)]

charm_lint_check #10468 keystone-next for corey.bryant mp271143
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/10468/

[uosci-testing-bot (uosci-testing-bot)]

charm_unit_test #9659 keystone-next for corey.bryant mp271143
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/9659/

[uosci-testing-bot (uosci-testing-bot)]

charm_amulet_test #6611 keystone-next for corey.bryant mp271143
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/12526596/
Build: http://10.245.162.77:8080/job/charm_amulet_test/6611/

[James Page (james-page)]

review: Needs Resubmitting

[uosci-testing-bot (uosci-testing-bot)]

charm_lint_check #10696 keystone-next for corey.bryant mp271143
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/10696/

[uosci-testing-bot (uosci-testing-bot)]

charm_unit_test #9877 keystone-next for corey.bryant mp271143
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/9877/

[uosci-testing-bot (uosci-testing-bot)]

charm_amulet_test #6740 keystone-next for corey.bryant mp271143
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/12545931/
Build: http://10.245.162.77:8080/job/charm_amulet_test/6740/

[uosci-testing-bot (uosci-testing-bot)]

charm_amulet_test #6745 keystone-next for corey.bryant mp271143
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/12546291/
Build: http://10.245.162.77:8080/job/charm_amulet_test/6745/

[uosci-testing-bot (uosci-testing-bot)]

charm_amulet_test #6753 keystone-next for corey.bryant mp271143
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/12547504/
Build: http://10.245.162.77:8080/job/charm_amulet_test/6753/

[Corey Bryant (corey.bryant)]

Note that in my latest commit to this mp I reverted charm-helper changes to service_pause() and service_resume() which was causing amulet tests to fail. ack is looking into fixing those tests, but for now we should be ok merging this mp without those updates.

[uosci-testing-bot (uosci-testing-bot)]

charm_lint_check #10775 keystone-next for corey.bryant mp271143
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/10775/

[uosci-testing-bot (uosci-testing-bot)]

charm_unit_test #9952 keystone-next for corey.bryant mp271143
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/9952/

[uosci-testing-bot (uosci-testing-bot)]

charm_amulet_test #6780 keystone-next for corey.bryant mp271143
    AMULET OK: passed

Build: http://10.245.162.77:8080/job/charm_amulet_test/6780/

[Corey Bryant (corey.bryant)]

In response to Billy's find above, we've moved all of the action-managed upgrade code to import CONFIGS instead of calling register_configs() (see commit #185 below). Having 2 calls to register_configs() was causing issues.

[uosci-testing-bot (uosci-testing-bot)]

charm_lint_check #11539 keystone-next for corey.bryant mp271143
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/11539/

[uosci-testing-bot (uosci-testing-bot)]

charm_unit_test #10731 keystone-next for corey.bryant mp271143
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/10731/

[David Ames (thedac)]

Missing the openstack-upgrade symlink.
Still approved. I'll add the symlink at merge time.

review: Approve

[uosci-testing-bot (uosci-testing-bot)]

charm_amulet_test #7242 keystone-next for corey.bryant mp271143
    AMULET OK: passed

Build: http://10.245.162.77:8080/job/charm_amulet_test/7242/

=== modified file 'actions.yaml'
--- actions.yaml	2015-08-28 06:58:11 +0000
+++ actions.yaml	2015-10-07 16:21:02 +0000
@@ -11,3 +11,7 @@
     Resume keystone services.
     If the keystone deployment is clustered using the hacluster charm, the
     corresponding hacluster unit on the node must be resumed as well.
+openstack-upgrade:
+  description: |
+    Perform openstack upgrades. Config option action-managed-upgrade must be
+    set to True.

=== added file 'actions/openstack_upgrade.py'
--- actions/openstack_upgrade.py	1970-01-01 00:00:00 +0000
+++ actions/openstack_upgrade.py	2015-10-07 16:21:02 +0000
@@ -0,0 +1,37 @@
+#!/usr/bin/python
+import sys
+
+sys.path.append('hooks/')
+
+from charmhelpers.contrib.openstack.utils import (
+    do_action_openstack_upgrade,
+)
+
+from keystone_hooks import (
+    config_changed,
+    CONFIGS,
+)
+
+from keystone_utils import (
+    do_openstack_upgrade,
+)
+
+
+def openstack_upgrade():
+    """Perform action-managed OpenStack upgrade.
+
+    Upgrades packages to the configured openstack-origin version and sets
+    the corresponding action status as a result.
+
+    If the charm was installed from source we cannot upgrade it.
+    For backwards compatibility a config flag (action-managed-upgrade) must
+    be set for this code to run, otherwise a full service level upgrade will
+    fire on config-changed."""
+
+    if (do_action_openstack_upgrade('keystone',
+                                    do_openstack_upgrade,
+                                    CONFIGS)):
+        config_changed()
+
+if __name__ == '__main__':
+    openstack_upgrade()

=== modified file 'config.yaml'
--- config.yaml	2015-07-17 01:29:31 +0000
+++ config.yaml	2015-10-07 16:21:02 +0000
@@ -298,4 +298,13 @@
     description: |
       A comma-separated list of nagios servicegroups.
       If left empty, the nagios_context will be used as the servicegroup
-
+  action-managed-upgrade:
+    type: boolean
+    default: False
+    description: |
+      If True enables openstack upgrades for this charm via juju actions.
+      You will still need to set openstack-origin to the new repository but
+      instead of an upgrade running automatically across all units, it will
+      wait for you to execute the openstack-upgrade action for this charm on
+      each unit. If False it will revert to existing behavior of upgrading
+      all units on config change.

=== modified file 'hooks/keystone_hooks.py'
--- hooks/keystone_hooks.py	2015-09-30 07:01:00 +0000
+++ hooks/keystone_hooks.py	2015-10-07 16:21:02 +0000
@@ -153,7 +153,7 @@
         if config_value_changed('openstack-origin-git'):
             status_set('maintenance', 'Running Git install')
             git_install(config('openstack-origin-git'))
-    else:
+    elif not config('action-managed-upgrade'):
         if openstack_upgrade_available('keystone'):
             status_set('maintenance', 'Running openstack upgrade')
             do_openstack_upgrade(configs=CONFIGS)

=== added file 'unit_tests/test_actions_openstack_upgrade.py'
--- unit_tests/test_actions_openstack_upgrade.py	1970-01-01 00:00:00 +0000
+++ unit_tests/test_actions_openstack_upgrade.py	2015-10-07 16:21:02 +0000
@@ -0,0 +1,56 @@
+from mock import patch
+import os
+
+os.environ['JUJU_UNIT_NAME'] = 'keystone'
+
+with patch('keystone_utils.register_configs') as register_configs:
+    import openstack_upgrade
+    import keystone_hooks as hooks
+
+from test_utils import (
+    CharmTestCase
+)
+
+TO_PATCH = [
+    'config_changed',
+    'do_openstack_upgrade',
+]
+
+
+class TestCinderUpgradeActions(CharmTestCase):
+
+    def setUp(self):
+        super(TestCinderUpgradeActions, self).setUp(openstack_upgrade,
+                                                    TO_PATCH)
+
+    @patch.object(hooks, 'register_configs')
+    @patch('charmhelpers.contrib.openstack.utils.config')
+    @patch('charmhelpers.contrib.openstack.utils.action_set')
+    @patch('charmhelpers.contrib.openstack.utils.git_install_requested')
+    @patch('charmhelpers.contrib.openstack.utils.openstack_upgrade_available')
+    def test_openstack_upgrade_true(self, upgrade_avail, git_requested,
+                                    action_set, config, reg_configs):
+        git_requested.return_value = False
+        upgrade_avail.return_value = True
+        config.return_value = True
+
+        openstack_upgrade.openstack_upgrade()
+
+        self.assertTrue(self.do_openstack_upgrade.called)
+        self.assertTrue(self.config_changed.called)
+
+    @patch.object(hooks, 'register_configs')
+    @patch('charmhelpers.contrib.openstack.utils.config')
+    @patch('charmhelpers.contrib.openstack.utils.action_set')
+    @patch('charmhelpers.contrib.openstack.utils.git_install_requested')
+    @patch('charmhelpers.contrib.openstack.utils.openstack_upgrade_available')
+    def test_openstack_upgrade_false(self, upgrade_avail, git_requested,
+                                     action_set, config, reg_configs):
+        git_requested.return_value = False
+        upgrade_avail.return_value = True
+        config.return_value = False
+
+        openstack_upgrade.openstack_upgrade()
+
+        self.assertFalse(self.do_openstack_upgrade.called)
+        self.assertFalse(self.config_changed.called)

=== modified file 'unit_tests/test_keystone_hooks.py'
--- unit_tests/test_keystone_hooks.py	2015-09-30 14:48:53 +0000
+++ unit_tests/test_keystone_hooks.py	2015-10-07 16:21:02 +0000
@@ -540,6 +540,37 @@
         self.assertFalse(self.openstack_upgrade_available.called)
         self.assertFalse(self.do_openstack_upgrade.called)
 
+    @patch.object(hooks, 'git_install_requested')
+    @patch.object(hooks, 'config_value_changed')
+    @patch.object(hooks, 'ensure_ssl_dir')
+    @patch.object(hooks, 'configure_https')
+    @patch.object(hooks, 'is_pki_enabled')
+    @patch.object(hooks, 'is_ssl_cert_master')
+    @patch.object(hooks, 'peer_units')
+    @patch.object(unison, 'get_homedir')
+    @patch.object(unison, 'ensure_user')
+    @patch('keystone_utils.ensure_ssl_cert_master')
+    def test_config_changed_with_openstack_upgrade_action(self,
+                                                          ensure_ssl_cert,
+                                                          ensure_user,
+                                                          get_home,
+                                                          peer_units, is_ssl,
+                                                          is_pki, config_https,
+                                                          ensure_ssl_dir,
+                                                          config_value_changed,
+                                                          git_requested):
+        ensure_ssl_cert.return_value = False
+        is_pki.return_value = False
+        peer_units.return_value = []
+
+        git_requested.return_value = False
+        self.openstack_upgrade_available.return_value = True
+        self.test_config.set('action-managed-upgrade', True)
+
+        hooks.config_changed()
+
+        self.assertFalse(self.do_openstack_upgrade.called)
+
     @patch('keystone_utils.log')
     @patch('keystone_utils.ensure_ssl_cert_master')
     @patch.object(hooks, 'hashlib')