Merge lp:~corey.bryant/charms/trusty/keystone/git-1531612 into lp:~openstack-charmers-archive/charms/trusty/keystone/next

[Corey Bryant]

[uosci-testing-bot (uosci-testing-bot)]

charm_lint_check #16803 keystone-next for corey.bryant mp281958
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/16803/

[uosci-testing-bot (uosci-testing-bot)]

charm_unit_test #15697 keystone-next for corey.bryant mp281958
    UNIT FAIL: unit-test failed

UNIT Results (max last 2 lines):
make: *** [test] Error 1
ERROR:root:Make target returned non-zero.

Full unit test output: http://paste.ubuntu.com/14434924/
Build: http://10.245.162.77:8080/job/charm_unit_test/15697/

[uosci-testing-bot (uosci-testing-bot)]

charm_amulet_test #8578 keystone-next for corey.bryant mp281958
    AMULET OK: passed

Build: http://10.245.162.77:8080/job/charm_amulet_test/8578/

[lp:~corey.bryant/charms/trusty/keystone/git-1531612]

196. By Liam Young on 2016-01-08

[hopem, r=gnuoy] Fix upgrade breakage whereby if upgrading from
version of charm that did not support
db-initialised peer setting db ops get stuck
waiting infinitely for db to be intialised.

Closes-Bug: 1519035

[uosci-testing-bot (uosci-testing-bot)]

charm_lint_check #16836 keystone-next for corey.bryant mp281958
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/16836/

[uosci-testing-bot (uosci-testing-bot)]

charm_unit_test #15722 keystone-next for corey.bryant mp281958
    UNIT FAIL: unit-test failed

UNIT Results (max last 2 lines):
make: *** [test] Error 1
ERROR:root:Make target returned non-zero.

Full unit test output: http://paste.ubuntu.com/14438187/
Build: http://10.245.162.77:8080/job/charm_unit_test/15722/

[uosci-testing-bot (uosci-testing-bot)]

charm_lint_check #16842 keystone-next for corey.bryant mp281958
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/16842/

[uosci-testing-bot (uosci-testing-bot)]

charm_unit_test #15729 keystone-next for corey.bryant mp281958
    UNIT FAIL: unit-test failed

UNIT Results (max last 2 lines):
make: *** [test] Error 1
ERROR:root:Make target returned non-zero.

Full unit test output: http://paste.ubuntu.com/14438584/
Build: http://10.245.162.77:8080/job/charm_unit_test/15729/

[uosci-testing-bot (uosci-testing-bot)]

charm_lint_check #16850 keystone-next for corey.bryant mp281958
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/16850/

[uosci-testing-bot (uosci-testing-bot)]

charm_unit_test #15736 keystone-next for corey.bryant mp281958
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/15736/

[uosci-testing-bot (uosci-testing-bot)]

charm_amulet_test #8605 keystone-next for corey.bryant mp281958
    AMULET OK: passed

Build: http://10.245.162.77:8080/job/charm_amulet_test/8605/

[lp:~corey.bryant/charms/trusty/keystone/git-1531612]

197. By Corey Bryant on 2016-01-11

[corey.bryant,r=osci] Sync charm-helpers.

Enable sync of payload.archive, sync charm-helpers, and fixup unit test failures.

[uosci-testing-bot (uosci-testing-bot)]

charm_lint_check #17080 keystone-next for corey.bryant mp281958
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/17080/

[uosci-testing-bot (uosci-testing-bot)]

charm_unit_test #15955 keystone-next for corey.bryant mp281958
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/15955/

[uosci-testing-bot (uosci-testing-bot)]

charm_amulet_test #8690 keystone-next for corey.bryant mp281958
    AMULET OK: passed

Build: http://10.245.162.77:8080/job/charm_amulet_test/8690/

=== modified file 'charm-helpers-hooks.yaml'
--- charm-helpers-hooks.yaml	2015-08-06 11:52:27 +0000
+++ charm-helpers-hooks.yaml	2016-01-11 14:23:26 +0000
@@ -11,7 +11,7 @@
         - cluster
     - contrib.python
     - contrib.unison
-    - payload.execd
+    - payload
     - contrib.peerstorage
     - contrib.network.ip
     - contrib.python.packages

=== modified file 'charmhelpers/contrib/openstack/context.py'
--- charmhelpers/contrib/openstack/context.py	2016-01-04 21:27:51 +0000
+++ charmhelpers/contrib/openstack/context.py	2016-01-11 14:23:26 +0000
@@ -57,6 +57,7 @@
     get_nic_hwaddr,
     mkdir,
     write_file,
+    pwgen,
 )
 from charmhelpers.contrib.hahelpers.cluster import (
     determine_apache_port,
@@ -87,6 +88,8 @@
     is_bridge_member,
 )
 from charmhelpers.contrib.openstack.utils import get_host_ip
+from charmhelpers.core.unitdata import kv
+
 CA_CERT_PATH = '/usr/local/share/ca-certificates/keystone_juju_ca_cert.crt'
 ADDRESS_TYPES = ['admin', 'internal', 'public']
 
@@ -636,11 +639,18 @@
             ctxt['ipv6'] = True
             ctxt['local_host'] = 'ip6-localhost'
             ctxt['haproxy_host'] = '::'
-            ctxt['stat_port'] = ':::8888'
         else:
             ctxt['local_host'] = '127.0.0.1'
             ctxt['haproxy_host'] = '0.0.0.0'
-            ctxt['stat_port'] = ':8888'
+
+        ctxt['stat_port'] = '8888'
+
+        db = kv()
+        ctxt['stat_password'] = db.get('stat-password')
+        if not ctxt['stat_password']:
+            ctxt['stat_password'] = db.set('stat-password',
+                                           pwgen(32))
+            db.flush()
 
         for frontend in cluster_hosts:
             if (len(cluster_hosts[frontend]['backends']) > 1 or

=== modified file 'charmhelpers/contrib/openstack/templates/haproxy.cfg'
--- charmhelpers/contrib/openstack/templates/haproxy.cfg	2015-12-07 23:14:00 +0000
+++ charmhelpers/contrib/openstack/templates/haproxy.cfg	2016-01-11 14:23:26 +0000
@@ -33,13 +33,14 @@
     timeout server 30000
 {%- endif %}
 
-listen stats {{ stat_port }}
+listen stats
+    bind {{ local_host }}:{{ stat_port }}
     mode http
     stats enable
     stats hide-version
     stats realm Haproxy\ Statistics
     stats uri /
-    stats auth admin:password
+    stats auth admin:{{ stat_password }}
 
 {% if frontends -%}
 {% for service, ports in service_ports.items() -%}

=== modified file 'charmhelpers/contrib/openstack/utils.py'
--- charmhelpers/contrib/openstack/utils.py	2016-01-05 11:46:02 +0000
+++ charmhelpers/contrib/openstack/utils.py	2016-01-11 14:23:26 +0000
@@ -593,7 +593,7 @@
     return yaml.load(projects_yaml)
 
 
-def git_clone_and_install(projects_yaml, core_project, depth=1):
+def git_clone_and_install(projects_yaml, core_project):
     """
     Clone/install all specified OpenStack repositories.
 
@@ -643,6 +643,9 @@
     for p in projects['repositories']:
         repo = p['repository']
         branch = p['branch']
+        depth = '1'
+        if 'depth' in p.keys():
+            depth = p['depth']
         if p['name'] == 'requirements':
             repo_dir = _git_clone_and_install_single(repo, branch, depth,
                                                      parent_dir, http_proxy,
@@ -687,19 +690,13 @@
     """
     Clone and install a single git repository.
     """
-    dest_dir = os.path.join(parent_dir, os.path.basename(repo))
-
     if not os.path.exists(parent_dir):
         juju_log('Directory already exists at {}. '
                  'No need to create directory.'.format(parent_dir))
         os.mkdir(parent_dir)
 
-    if not os.path.exists(dest_dir):
-        juju_log('Cloning git repo: {}, branch: {}'.format(repo, branch))
-        repo_dir = install_remote(repo, dest=parent_dir, branch=branch,
-                                  depth=depth)
-    else:
-        repo_dir = dest_dir
+    juju_log('Cloning git repo: {}, branch: {}'.format(repo, branch))
+    repo_dir = install_remote(repo, dest=parent_dir, branch=branch, depth=depth)
 
     venv = os.path.join(parent_dir, 'venv')
 

=== modified file 'charmhelpers/fetch/giturl.py'
--- charmhelpers/fetch/giturl.py	2016-01-04 21:27:51 +0000
+++ charmhelpers/fetch/giturl.py	2016-01-11 14:23:26 +0000
@@ -22,7 +22,6 @@
     filter_installed_packages,
     apt_install,
 )
-from charmhelpers.core.host import mkdir
 
 if filter_installed_packages(['git']) != []:
     apt_install(['git'])
@@ -50,8 +49,8 @@
             cmd = ['git', '-C', dest, 'pull', source, branch]
         else:
             cmd = ['git', 'clone', source, dest, '--branch', branch]
-        if depth:
-            cmd.extend(['--depth', depth])
+            if depth:
+                cmd.extend(['--depth', depth])
         check_call(cmd)
 
     def install(self, source, branch="master", dest=None, depth=None):
@@ -62,8 +61,6 @@
         else:
             dest_dir = os.path.join(os.environ.get('CHARM_DIR'), "fetched",
                                     branch_name)
-        if not os.path.exists(dest_dir):
-            mkdir(dest_dir, perms=0o755)
         try:
             self.clone(source, dest_dir, branch, depth)
         except OSError as e:

=== added file 'charmhelpers/payload/archive.py'
--- charmhelpers/payload/archive.py	1970-01-01 00:00:00 +0000
+++ charmhelpers/payload/archive.py	2016-01-11 14:23:26 +0000
@@ -0,0 +1,73 @@
+# Copyright 2014-2015 Canonical Limited.
+#
+# This file is part of charm-helpers.
+#
+# charm-helpers is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Lesser General Public License version 3 as
+# published by the Free Software Foundation.
+#
+# charm-helpers is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public License
+# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
+
+import os
+import tarfile
+import zipfile
+from charmhelpers.core import (
+    host,
+    hookenv,
+)
+
+
+class ArchiveError(Exception):
+    pass
+
+
+def get_archive_handler(archive_name):
+    if os.path.isfile(archive_name):
+        if tarfile.is_tarfile(archive_name):
+            return extract_tarfile
+        elif zipfile.is_zipfile(archive_name):
+            return extract_zipfile
+    else:
+        # look at the file name
+        for ext in ('.tar', '.tar.gz', '.tgz', 'tar.bz2', '.tbz2', '.tbz'):
+            if archive_name.endswith(ext):
+                return extract_tarfile
+        for ext in ('.zip', '.jar'):
+            if archive_name.endswith(ext):
+                return extract_zipfile
+
+
+def archive_dest_default(archive_name):
+    archive_file = os.path.basename(archive_name)
+    return os.path.join(hookenv.charm_dir(), "archives", archive_file)
+
+
+def extract(archive_name, destpath=None):
+    handler = get_archive_handler(archive_name)
+    if handler:
+        if not destpath:
+            destpath = archive_dest_default(archive_name)
+        if not os.path.isdir(destpath):
+            host.mkdir(destpath)
+        handler(archive_name, destpath)
+        return destpath
+    else:
+        raise ArchiveError("No handler for archive")
+
+
+def extract_tarfile(archive_name, destpath):
+    "Unpack a tar archive, optionally compressed"
+    archive = tarfile.open(archive_name)
+    archive.extractall(destpath)
+
+
+def extract_zipfile(archive_name, destpath):
+    "Unpack a zip file"
+    archive = zipfile.ZipFile(archive_name)
+    archive.extractall(destpath)

=== modified file 'unit_tests/test_keystone_contexts.py'
--- unit_tests/test_keystone_contexts.py	2015-12-07 14:04:38 +0000
+++ unit_tests/test_keystone_contexts.py	2016-01-11 14:23:26 +0000
@@ -91,10 +91,11 @@
     @patch('charmhelpers.contrib.openstack.context.related_units')
     @patch('charmhelpers.contrib.openstack.context.relation_get')
     @patch('charmhelpers.contrib.openstack.context.log')
+    @patch('charmhelpers.contrib.openstack.context.kv')
     @patch('__builtin__.open')
     def test_haproxy_context_service_enabled(
-        self, mock_open, mock_log, mock_relation_get, mock_related_units,
-            mock_unit_get, mock_relation_ids, mock_config,
+        self, mock_open, mock_kv, mock_log, mock_relation_get,
+            mock_related_units, mock_unit_get, mock_relation_ids, mock_config,
             mock_get_address_in_network, mock_get_netmask_for_address,
             mock_api_port):
         os.environ['JUJU_UNIT_NAME'] = 'keystone'
@@ -108,6 +109,7 @@
         mock_get_netmask_for_address.return_value = '255.255.255.0'
         self.determine_apache_port.return_value = '34'
         mock_api_port.return_value = '12'
+        mock_kv().get.return_value = 'abcdefghijklmnopqrstuvwxyz123456'
 
         ctxt = context.HAProxyContext()
 
@@ -118,7 +120,8 @@
                               'public_port': '12'},
              'local_host': '127.0.0.1',
              'haproxy_host': '0.0.0.0',
-             'stat_port': ':8888',
+             'stat_port': '8888',
+             'stat_password': 'abcdefghijklmnopqrstuvwxyz123456',
              'service_ports': {'admin-port': ['12', '34'],
                                'public-port': ['12', '34']},
              'default_backend': '1.2.3.4',