Merge lp:~springfield-team/charms/trusty/nova-cloud-controller/n1kv into lp:~openstack-charmers-archive/charms/trusty/nova-cloud-controller/next

[Shiv Prasad Rao]

[uosci-testing-bot (uosci-testing-bot)]

UOSCI bot says:
charm_lint_check #1166 nova-cloud-controller-next for shivrao mp242447
    LINT OK: passed

LINT Results (max last 5 lines):
  I: config.yaml: option os-admin-network has no default value
  I: config.yaml: option haproxy-client-timeout has no default value
  I: config.yaml: option ssl_cert has no default value
  I: config.yaml: option nvp-l3-uuid has no default value
  I: config.yaml: option os-internal-network has no default value

Full lint test output: http://paste.ubuntu.com/9137369/
Build: http://10.98.191.181:8080/job/charm_lint_check/1166/

[uosci-testing-bot (uosci-testing-bot)]

UOSCI bot says:
charm_unit_test #1000 nova-cloud-controller-next for shivrao mp242447
    UNIT OK: passed

UNIT Results (max last 5 lines):
  hooks/nova_cc_hooks 442 145 67% 132-135, 148-149, 173, 184-185, 189, 221, 226-231, 242, 322-325, 333-336, 342-345, 355-371, 380-382, 392-406, 410-419, 505, 515, 519-520, 573, 579-589, 594-605, 615-625, 630-669, 679-694, 702-706, 731-740, 764, 769-777, 803, 853-856
  hooks/nova_cc_utils 445 112 75% 296-301, 312-315, 325-326, 382, 384, 430-432, 436, 450-458, 465-470, 474-488, 544, 593-595, 600-603, 608, 612, 636-637, 651-653, 674-675, 681-704, 708-714, 718-724, 730, 736, 743, 754-758, 843, 903-909, 913-915, 919-922, 926-938
  TOTAL 1038 366 65%
  Ran 96 tests in 8.559s
  OK

Full unit test output: http://paste.ubuntu.com/9137388/
Build: http://10.98.191.181:8080/job/charm_unit_test/1000/

[uosci-testing-bot (uosci-testing-bot)]

UOSCI bot says:
charm_amulet_test #508 nova-cloud-controller-next for shivrao mp242447
    AMULET FAIL: amulet-test failed

AMULET Results (max last 5 lines):
  WARNING cannot delete security group "juju-osci-sv05-0". Used by another environmenERROR:root:Make target returned non-zero.
t?
  juju-test INFO : Results: 0 passed, 2 failed, 1 errored
  ERROR subprocess encountered error code 2
  make: *** [test] Error 2

Full amulet test output: http://paste.ubuntu.com/9137940/
Build: http://10.98.191.181:8080/job/charm_amulet_test/508/

[lp:~springfield-team/charms/trusty/nova-cloud-controller/n1kv]

128. By Jorge Niedbalski on 2014-11-21

[all] resync with /next && charm helpers "make sync"

[uosci-testing-bot (uosci-testing-bot)]

UOSCI bot says:
charm_lint_check #1168 nova-cloud-controller-next for shivrao mp242447
    LINT OK: passed

LINT Results (max last 5 lines):
  I: config.yaml: option os-admin-network has no default value
  I: config.yaml: option haproxy-client-timeout has no default value
  I: config.yaml: option ssl_cert has no default value
  I: config.yaml: option nvp-l3-uuid has no default value
  I: config.yaml: option os-internal-network has no default value

Full lint test output: http://paste.ubuntu.com/9150464/
Build: http://10.98.191.181:8080/job/charm_lint_check/1168/

[uosci-testing-bot (uosci-testing-bot)]

UOSCI bot says:
charm_unit_test #1002 nova-cloud-controller-next for shivrao mp242447
    UNIT OK: passed

UNIT Results (max last 5 lines):
  hooks/nova_cc_hooks 442 145 67% 132-135, 148-149, 173, 184-185, 189, 221, 226-231, 242, 322-325, 333-336, 342-345, 355-371, 380-382, 392-406, 410-419, 505, 515, 519-520, 573, 579-589, 594-605, 615-625, 630-669, 679-694, 702-706, 731-740, 764, 769-777, 803, 853-856
  hooks/nova_cc_utils 445 112 75% 296-301, 312-315, 325-326, 382, 384, 430-432, 436, 450-458, 465-470, 474-488, 544, 593-595, 600-603, 608, 612, 636-637, 651-653, 674-675, 681-704, 708-714, 718-724, 730, 736, 743, 754-758, 843, 903-909, 913-915, 919-922, 926-938
  TOTAL 1038 366 65%
  Ran 96 tests in 8.973s
  OK

Full unit test output: http://paste.ubuntu.com/9150468/
Build: http://10.98.191.181:8080/job/charm_unit_test/1002/

[uosci-testing-bot (uosci-testing-bot)]

UOSCI bot says:
charm_amulet_test #510 nova-cloud-controller-next for shivrao mp242447
    AMULET FAIL: amulet-test failed

AMULET Results (max last 5 lines):
  ERROR waited for 10m0s without being able to connect: ssh: connect to host 10.215.3.235 port 22: No route to host
  juju-test.conductor WARNING : Could not bootstrap osci-sv05, got Bootstrap returned with an exit > 0. Skipping
  juju-test INFO : Results: 0 passed, 0 failed, 3 errored
  ERROR subprocess encountered error code 124
  make: *** [test] Error 124

Full amulet test output: http://paste.ubuntu.com/9151027/
Build: http://10.98.191.181:8080/job/charm_amulet_test/510/

[lp:~springfield-team/charms/trusty/nova-cloud-controller/n1kv]

129. By Shiv Prasad Rao on 2014-12-08

Changes for n1kv

130. By Edward Hope-Morley on 2014-12-15

[hopem] synced /next

Unmerged revisions

130. By Edward Hope-Morley on 2014-12-15

[hopem] synced /next

129. By Shiv Prasad Rao on 2014-12-08

Changes for n1kv

128. By Jorge Niedbalski on 2014-11-21

[all] resync with /next && charm helpers "make sync"

127. By Shiv Prasad Rao on 2014-11-21

Additional changes for n1kv

126. By Shiv Prasad Rao on 2014-11-18

Cisco Nexus 1000V changes

=== modified file 'hooks/charmhelpers/contrib/network/ip.py'
--- hooks/charmhelpers/contrib/network/ip.py	2014-10-09 10:31:45 +0000
+++ hooks/charmhelpers/contrib/network/ip.py	2014-11-21 15:33:38 +0000
@@ -1,15 +1,12 @@
 import glob
 import re
 import subprocess
-import sys
 
 from functools import partial
 
 from charmhelpers.core.hookenv import unit_get
 from charmhelpers.fetch import apt_install
 from charmhelpers.core.hookenv import (
-    WARNING,
-    ERROR,
     log
 )
 
@@ -34,31 +31,28 @@
                          network)
 
 
+def no_ip_found_error_out(network):
+    errmsg = ("No IP address found in network: %s" % network)
+    raise ValueError(errmsg)
+
+
 def get_address_in_network(network, fallback=None, fatal=False):
-    """
-    Get an IPv4 or IPv6 address within the network from the host.
+    """Get an IPv4 or IPv6 address within the network from the host.
 
     :param network (str): CIDR presentation format. For example,
         '192.168.1.0/24'.
     :param fallback (str): If no address is found, return fallback.
     :param fatal (boolean): If no address is found, fallback is not
         set and fatal is True then exit(1).
-
     """
-
-    def not_found_error_out():
-        log("No IP address found in network: %s" % network,
-            level=ERROR)
-        sys.exit(1)
-
     if network is None:
         if fallback is not None:
             return fallback
+
+        if fatal:
+            no_ip_found_error_out(network)
         else:
-            if fatal:
-                not_found_error_out()
-            else:
-                return None
+            return None
 
     _validate_cidr(network)
     network = netaddr.IPNetwork(network)
@@ -70,6 +64,7 @@
             cidr = netaddr.IPNetwork("%s/%s" % (addr, netmask))
             if cidr in network:
                 return str(cidr.ip)
+
         if network.version == 6 and netifaces.AF_INET6 in addresses:
             for addr in addresses[netifaces.AF_INET6]:
                 if not addr['addr'].startswith('fe80'):
@@ -82,20 +77,20 @@
         return fallback
 
     if fatal:
-        not_found_error_out()
+        no_ip_found_error_out(network)
 
     return None
 
 
 def is_ipv6(address):
-    '''Determine whether provided address is IPv6 or not'''
+    """Determine whether provided address is IPv6 or not."""
     try:
         address = netaddr.IPAddress(address)
     except netaddr.AddrFormatError:
         # probably a hostname - so not an address at all!
         return False
-    else:
-        return address.version == 6
+
+    return address.version == 6
 
 
 def is_address_in_network(network, address):
@@ -113,11 +108,13 @@
     except (netaddr.core.AddrFormatError, ValueError):
         raise ValueError("Network (%s) is not in CIDR presentation format" %
                          network)
+
     try:
         address = netaddr.IPAddress(address)
     except (netaddr.core.AddrFormatError, ValueError):
         raise ValueError("Address (%s) is not in correct presentation format" %
                          address)
+
     if address in network:
         return True
     else:
@@ -147,6 +144,7 @@
                     return iface
                 else:
                     return addresses[netifaces.AF_INET][0][key]
+
         if address.version == 6 and netifaces.AF_INET6 in addresses:
             for addr in addresses[netifaces.AF_INET6]:
                 if not addr['addr'].startswith('fe80'):
@@ -160,41 +158,42 @@
                             return str(cidr).split('/')[1]
                         else:
                             return addr[key]
+
     return None
 
 
 get_iface_for_address = partial(_get_for_address, key='iface')
 
+
 get_netmask_for_address = partial(_get_for_address, key='netmask')
 
 
 def format_ipv6_addr(address):
-    """
-    IPv6 needs to be wrapped with [] in url link to parse correctly.
+    """If address is IPv6, wrap it in '[]' otherwise return None.
+
+    This is required by most configuration files when specifying IPv6
+    addresses.
     """
     if is_ipv6(address):
-        address = "[%s]" % address
-    else:
-        log("Not a valid ipv6 address: %s" % address, level=WARNING)
-        address = None
+        return "[%s]" % address
 
-    return address
+    return None
 
 
 def get_iface_addr(iface='eth0', inet_type='AF_INET', inc_aliases=False,
                    fatal=True, exc_list=None):
-    """
-    Return the assigned IP address for a given interface, if any, or [].
-    """
+    """Return the assigned IP address for a given interface, if any."""
     # Extract nic if passed /dev/ethX
     if '/' in iface:
         iface = iface.split('/')[-1]
+
     if not exc_list:
         exc_list = []
+
     try:
         inet_num = getattr(netifaces, inet_type)
     except AttributeError:
-        raise Exception('Unknown inet type ' + str(inet_type))
+        raise Exception("Unknown inet type '%s'" % str(inet_type))
 
     interfaces = netifaces.interfaces()
     if inc_aliases:
@@ -202,15 +201,18 @@
         for _iface in interfaces:
             if iface == _iface or _iface.split(':')[0] == iface:
                 ifaces.append(_iface)
+
         if fatal and not ifaces:
             raise Exception("Invalid interface '%s'" % iface)
+
         ifaces.sort()
     else:
         if iface not in interfaces:
             if fatal:
-                raise Exception("%s not found " % (iface))
+                raise Exception("Interface '%s' not found " % (iface))
             else:
                 return []
+
         else:
             ifaces = [iface]
 
@@ -221,11 +223,14 @@
             for entry in net_info[inet_num]:
                 if 'addr' in entry and entry['addr'] not in exc_list:
                     addresses.append(entry['addr'])
+
     if fatal and not addresses:
         raise Exception("Interface '%s' doesn't have any %s addresses." %
                         (iface, inet_type))
+
     return addresses
 
+
 get_ipv4_addr = partial(get_iface_addr, inet_type='AF_INET')
 
 
@@ -241,6 +246,7 @@
                 raw = re.match(ll_key, _addr)
                 if raw:
                     _addr = raw.group(1)
+
                 if _addr == addr:
                     log("Address '%s' is configured on iface '%s'" %
                         (addr, iface))
@@ -251,8 +257,9 @@
 
 
 def sniff_iface(f):
-    """If no iface provided, inject net iface inferred from unit private
-    address.
+    """Ensure decorated function is called with a value for iface.
+
+    If no iface provided, inject net iface inferred from unit private address.
     """
     def iface_sniffer(*args, **kwargs):
         if not kwargs.get('iface', None):
@@ -317,33 +324,28 @@
                 return addrs
 
     if fatal:
-        raise Exception("Interface '%s' doesn't have a scope global "
+        raise Exception("Interface '%s' does not have a scope global "
                         "non-temporary ipv6 address." % iface)
 
     return []
 
 
 def get_bridges(vnic_dir='/sys/devices/virtual/net'):
-    """
-    Return a list of bridges on the system or []
-    """
-    b_rgex = vnic_dir + '/*/bridge'
-    return [x.replace(vnic_dir, '').split('/')[1] for x in glob.glob(b_rgex)]
+    """Return a list of bridges on the system."""
+    b_regex = "%s/*/bridge" % vnic_dir
+    return [x.replace(vnic_dir, '').split('/')[1] for x in glob.glob(b_regex)]
 
 
 def get_bridge_nics(bridge, vnic_dir='/sys/devices/virtual/net'):
-    """
-    Return a list of nics comprising a given bridge on the system or []
-    """
-    brif_rgex = "%s/%s/brif/*" % (vnic_dir, bridge)
-    return [x.split('/')[-1] for x in glob.glob(brif_rgex)]
+    """Return a list of nics comprising a given bridge on the system."""
+    brif_regex = "%s/%s/brif/*" % (vnic_dir, bridge)
+    return [x.split('/')[-1] for x in glob.glob(brif_regex)]
 
 
 def is_bridge_member(nic):
-    """
-    Check if a given nic is a member of a bridge
-    """
+    """Check if a given nic is a member of a bridge."""
     for bridge in get_bridges():
         if nic in get_bridge_nics(bridge):
             return True
+
     return False
 
=== modified file 'hooks/charmhelpers/contrib/openstack/context.py'
--- hooks/charmhelpers/contrib/openstack/context.py	2014-10-13 16:18:58 +0000
+++ hooks/charmhelpers/contrib/openstack/context.py	2014-11-21 15:33:38 +0000
@@ -3,18 +3,15 @@
 import time
 
 from base64 import b64decode
-
-from subprocess import (
-    check_call
-)
+from subprocess import check_call
 
 from charmhelpers.fetch import (
     apt_install,
     filter_installed_packages,
 )
-
 from charmhelpers.core.hookenv import (
     config,
+    is_relation_made,
     local_unit,
     log,
     relation_get,
@@ -23,43 +20,40 @@
     relation_set,
     unit_get,
     unit_private_ip,
+    DEBUG,
+    INFO,
+    WARNING,
     ERROR,
-    INFO
 )
-
 from charmhelpers.core.host import (
     mkdir,
-    write_file
+    write_file,
 )
-
 from charmhelpers.contrib.hahelpers.cluster import (
     determine_apache_port,
     determine_api_port,
     https,
-    is_clustered
+    is_clustered,
 )
-
 from charmhelpers.contrib.hahelpers.apache import (
     get_cert,
     get_ca_cert,
     install_ca_cert,
 )
-
 from charmhelpers.contrib.openstack.neutron import (
     neutron_plugin_attribute,
 )
-
 from charmhelpers.contrib.network.ip import (
     get_address_in_network,
     get_ipv6_addr,
     get_netmask_for_address,
     format_ipv6_addr,
-    is_address_in_network
+    is_address_in_network,
 )
-
 from charmhelpers.contrib.openstack.utils import get_host_ip
 
 CA_CERT_PATH = '/usr/local/share/ca-certificates/keystone_juju_ca_cert.crt'
+ADDRESS_TYPES = ['admin', 'internal', 'public']
 
 
 class OSContextError(Exception):
@@ -67,7 +61,7 @@
 
 
 def ensure_packages(packages):
-    '''Install but do not upgrade required plugin packages'''
+    """Install but do not upgrade required plugin packages."""
     required = filter_installed_packages(packages)
     if required:
         apt_install(required, fatal=True)
@@ -78,17 +72,24 @@
     for k, v in ctxt.iteritems():
         if v is None or v == '':
             _missing.append(k)
+
     if _missing:
-        log('Missing required data: %s' % ' '.join(_missing), level='INFO')
+        log('Missing required data: %s' % ' '.join(_missing), level=INFO)
         return False
+
     return True
 
 
 def config_flags_parser(config_flags):
+    """Parses config flags string into dict.
+
+    The provided config_flags string may be a list of comma-separated values
+    which themselves may be comma-separated list of values.
+    """
     if config_flags.find('==') >= 0:
-        log("config_flags is not in expected format (key=value)",
-            level=ERROR)
+        log("config_flags is not in expected format (key=value)", level=ERROR)
         raise OSContextError
+
     # strip the following from each value.
     post_strippers = ' ,'
     # we strip any leading/trailing '=' or ' ' from the string then
@@ -111,17 +112,18 @@
             # if this not the first entry, expect an embedded key.
             index = current.rfind(',')
             if index < 0:
-                log("invalid config value(s) at index %s" % (i),
-                    level=ERROR)
+                log("Invalid config value(s) at index %s" % (i), level=ERROR)
                 raise OSContextError
             key = current[index + 1:]
 
         # Add to collection.
         flags[key.strip(post_strippers)] = value.rstrip(post_strippers)
+
     return flags
 
 
 class OSContextGenerator(object):
+    """Base class for all context generators."""
     interfaces = []
 
     def __call__(self):
@@ -133,11 +135,11 @@
 
     def __init__(self,
                  database=None, user=None, relation_prefix=None, ssl_dir=None):
-        '''
-        Allows inspecting relation for settings prefixed with relation_prefix.
-        This is useful for parsing access for multiple databases returned via
-        the shared-db interface (eg, nova_password, quantum_password)
-        '''
+        """Allows inspecting relation for settings prefixed with
+        relation_prefix. This is useful for parsing access for multiple
+        databases returned via the shared-db interface (eg, nova_password,
+        quantum_password)
+        """
         self.relation_prefix = relation_prefix
         self.database = database
         self.user = user
@@ -147,9 +149,8 @@
         self.database = self.database or config('database')
         self.user = self.user or config('database-user')
         if None in [self.database, self.user]:
-            log('Could not generate shared_db context. '
-                'Missing required charm config options. '
-                '(database name and user)')
+            log("Could not generate shared_db context. Missing required charm "
+                "config options. (database name and user)", level=ERROR)
             raise OSContextError
 
         ctxt = {}
@@ -202,23 +203,24 @@
     def __call__(self):
         self.database = self.database or config('database')
         if self.database is None:
-            log('Could not generate postgresql_db context. '
-                'Missing required charm config options. '
-                '(database name)')
+            log('Could not generate postgresql_db context. Missing required '
+                'charm config options. (database name)', level=ERROR)
             raise OSContextError
+
         ctxt = {}
-
         for rid in relation_ids(self.interfaces[0]):
             for unit in related_units(rid):
-                ctxt = {
-                    'database_host': relation_get('host', rid=rid, unit=unit),
-                    'database': self.database,
-                    'database_user': relation_get('user', rid=rid, unit=unit),
-                    'database_password': relation_get('password', rid=rid, unit=unit),
-                    'database_type': 'postgresql',
-                }
+                rel_host = relation_get('host', rid=rid, unit=unit)
+                rel_user = relation_get('user', rid=rid, unit=unit)
+                rel_passwd = relation_get('password', rid=rid, unit=unit)
+                ctxt = {'database_host': rel_host,
+                        'database': self.database,
+                        'database_user': rel_user,
+                        'database_password': rel_passwd,
+                        'database_type': 'postgresql'}
                 if context_complete(ctxt):
                     return ctxt
+
         return {}
 
 
@@ -227,23 +229,29 @@
         ca_path = os.path.join(ssl_dir, 'db-client.ca')
         with open(ca_path, 'w') as fh:
             fh.write(b64decode(rdata['ssl_ca']))
+
         ctxt['database_ssl_ca'] = ca_path
     elif 'ssl_ca' in rdata:
-        log("Charm not setup for ssl support but ssl ca found")
+        log("Charm not setup for ssl support but ssl ca found", level=INFO)
         return ctxt
+
     if 'ssl_cert' in rdata:
         cert_path = os.path.join(
             ssl_dir, 'db-client.cert')
         if not os.path.exists(cert_path):
-            log("Waiting 1m for ssl client cert validity")
+            log("Waiting 1m for ssl client cert validity", level=INFO)
             time.sleep(60)
+
         with open(cert_path, 'w') as fh:
             fh.write(b64decode(rdata['ssl_cert']))
+
         ctxt['database_ssl_cert'] = cert_path
         key_path = os.path.join(ssl_dir, 'db-client.key')
         with open(key_path, 'w') as fh:
             fh.write(b64decode(rdata['ssl_key']))
+
         ctxt['database_ssl_key'] = key_path
+
     return ctxt
 
 
@@ -251,9 +259,8 @@
     interfaces = ['identity-service']
 
     def __call__(self):
-        log('Generating template context for identity-service')
+        log('Generating template context for identity-service', level=DEBUG)
         ctxt = {}
-
         for rid in relation_ids('identity-service'):
             for unit in related_units(rid):
                 rdata = relation_get(rid=rid, unit=unit)
@@ -261,26 +268,24 @@
                 serv_host = format_ipv6_addr(serv_host) or serv_host
                 auth_host = rdata.get('auth_host')
                 auth_host = format_ipv6_addr(auth_host) or auth_host
-
-                ctxt = {
-                    'service_port': rdata.get('service_port'),
-                    'service_host': serv_host,
-                    'auth_host': auth_host,
-                    'auth_port': rdata.get('auth_port'),
-                    'admin_tenant_name': rdata.get('service_tenant'),
-                    'admin_user': rdata.get('service_username'),
-                    'admin_password': rdata.get('service_password'),
-                    'service_protocol':
-                    rdata.get('service_protocol') or 'http',
-                    'auth_protocol':
-                    rdata.get('auth_protocol') or 'http',
-                }
+                svc_protocol = rdata.get('service_protocol') or 'http'
+                auth_protocol = rdata.get('auth_protocol') or 'http'
+                ctxt = {'service_port': rdata.get('service_port'),
+                        'service_host': serv_host,
+                        'auth_host': auth_host,
+                        'auth_port': rdata.get('auth_port'),
+                        'admin_tenant_name': rdata.get('service_tenant'),
+                        'admin_user': rdata.get('service_username'),
+                        'admin_password': rdata.get('service_password'),
+                        'service_protocol': svc_protocol,
+                        'auth_protocol': auth_protocol}
                 if context_complete(ctxt):
                     # NOTE(jamespage) this is required for >= icehouse
                     # so a missing value just indicates keystone needs
                     # upgrading
                     ctxt['admin_tenant_id'] = rdata.get('service_tenant_id')
                     return ctxt
+
         return {}
 
 
@@ -293,21 +298,23 @@
         self.interfaces = [rel_name]
 
     def __call__(self):
-        log('Generating template context for amqp')
+        log('Generating template context for amqp', level=DEBUG)
         conf = config()
-        user_setting = 'rabbit-user'
-        vhost_setting = 'rabbit-vhost'
         if self.relation_prefix:
-            user_setting = self.relation_prefix + '-rabbit-user'
-            vhost_setting = self.relation_prefix + '-rabbit-vhost'
+            user_setting = '%s-rabbit-user' % (self.relation_prefix)
+            vhost_setting = '%s-rabbit-vhost' % (self.relation_prefix)
+        else:
+            user_setting = 'rabbit-user'
+            vhost_setting = 'rabbit-vhost'
 
         try:
             username = conf[user_setting]
             vhost = conf[vhost_setting]
         except KeyError as e:
-            log('Could not generate shared_db context. '
-                'Missing required charm config options: %s.' % e)
+            log('Could not generate shared_db context. Missing required charm '
+                'config options: %s.' % e, level=ERROR)
             raise OSContextError
+
         ctxt = {}
         for rid in relation_ids(self.rel_name):
             ha_vip_only = False
@@ -321,6 +328,7 @@
                     host = relation_get('private-address', rid=rid, unit=unit)
                     host = format_ipv6_addr(host) or host
                     ctxt['rabbitmq_host'] = host
+
                 ctxt.update({
                     'rabbitmq_user': username,
                     'rabbitmq_password': relation_get('password', rid=rid,
@@ -331,6 +339,7 @@
                 ssl_port = relation_get('ssl_port', rid=rid, unit=unit)
                 if ssl_port:
                     ctxt['rabbit_ssl_port'] = ssl_port
+
                 ssl_ca = relation_get('ssl_ca', rid=rid, unit=unit)
                 if ssl_ca:
                     ctxt['rabbit_ssl_ca'] = ssl_ca
@@ -344,41 +353,45 @@
                 if context_complete(ctxt):
                     if 'rabbit_ssl_ca' in ctxt:
                         if not self.ssl_dir:
-                            log(("Charm not setup for ssl support "
-                                 "but ssl ca found"))
+                            log("Charm not setup for ssl support but ssl ca "
+                                "found", level=INFO)
                             break
+
                         ca_path = os.path.join(
                             self.ssl_dir, 'rabbit-client-ca.pem')
                         with open(ca_path, 'w') as fh:
                             fh.write(b64decode(ctxt['rabbit_ssl_ca']))
                             ctxt['rabbit_ssl_ca'] = ca_path
+
                     # Sufficient information found = break out!
                     break
+
             # Used for active/active rabbitmq >= grizzly
-            if ('clustered' not in ctxt or ha_vip_only) \
-                    and len(related_units(rid)) > 1:
+            if (('clustered' not in ctxt or ha_vip_only) and
+                    len(related_units(rid)) > 1):
                 rabbitmq_hosts = []
                 for unit in related_units(rid):
                     host = relation_get('private-address', rid=rid, unit=unit)
                     host = format_ipv6_addr(host) or host
                     rabbitmq_hosts.append(host)
+
                 ctxt['rabbitmq_hosts'] = ','.join(rabbitmq_hosts)
+
         if not context_complete(ctxt):
             return {}
-        else:
-            return ctxt
+
+        return ctxt
 
 
 class CephContext(OSContextGenerator):
+    """Generates context for /etc/ceph/ceph.conf templates."""
     interfaces = ['ceph']
 
     def __call__(self):
-        '''This generates context for /etc/ceph/ceph.conf templates'''
         if not relation_ids('ceph'):
             return {}
 
-        log('Generating template context for ceph')
-
+        log('Generating template context for ceph', level=DEBUG)
         mon_hosts = []
         auth = None
         key = None
@@ -387,18 +400,18 @@
             for unit in related_units(rid):
                 auth = relation_get('auth', rid=rid, unit=unit)
                 key = relation_get('key', rid=rid, unit=unit)
-                ceph_addr = \
-                    relation_get('ceph-public-address', rid=rid, unit=unit) or \
-                    relation_get('private-address', rid=rid, unit=unit)
+                ceph_pub_addr = relation_get('ceph-public-address', rid=rid,
+                                             unit=unit)
+                unit_priv_addr = relation_get('private-address', rid=rid,
+                                              unit=unit)
+                ceph_addr = ceph_pub_addr or unit_priv_addr
                 ceph_addr = format_ipv6_addr(ceph_addr) or ceph_addr
                 mon_hosts.append(ceph_addr)
 
-        ctxt = {
-            'mon_hosts': ' '.join(mon_hosts),
-            'auth': auth,
-            'key': key,
-            'use_syslog': use_syslog
-        }
+        ctxt = {'mon_hosts': ' '.join(mon_hosts),
+                'auth': auth,
+                'key': key,
+                'use_syslog': use_syslog}
 
         if not os.path.isdir('/etc/ceph'):
             os.mkdir('/etc/ceph')
@@ -407,79 +420,65 @@
             return {}
 
         ensure_packages(['ceph-common'])
-
         return ctxt
 
 
-ADDRESS_TYPES = ['admin', 'internal', 'public']
-
-
 class HAProxyContext(OSContextGenerator):
+    """Provides half a context for the haproxy template, which describes
+    all peers to be included in the cluster.  Each charm needs to include
+    its own context generator that describes the port mapping.
+    """
     interfaces = ['cluster']
 
     def __call__(self):
-        '''
-        Builds half a context for the haproxy template, which describes
-        all peers to be included in the cluster.  Each charm needs to include
-        its own context generator that describes the port mapping.
-        '''
         if not relation_ids('cluster'):
             return {}
 
-        l_unit = local_unit().replace('/', '-')
-
         if config('prefer-ipv6'):
             addr = get_ipv6_addr(exc_list=[config('vip')])[0]
         else:
             addr = get_host_ip(unit_get('private-address'))
 
+        l_unit = local_unit().replace('/', '-')
         cluster_hosts = {}
 
         # NOTE(jamespage): build out map of configured network endpoints
         # and associated backends
         for addr_type in ADDRESS_TYPES:
-            laddr = get_address_in_network(
-                config('os-{}-network'.format(addr_type)))
+            cfg_opt = 'os-{}-network'.format(addr_type)
+            laddr = get_address_in_network(config(cfg_opt))
             if laddr:
-                cluster_hosts[laddr] = {}
-                cluster_hosts[laddr]['network'] = "{}/{}".format(
-                    laddr,
-                    get_netmask_for_address(laddr)
-                )
-                cluster_hosts[laddr]['backends'] = {}
-                cluster_hosts[laddr]['backends'][l_unit] = laddr
+                netmask = get_netmask_for_address(laddr)
+                cluster_hosts[laddr] = {'network': "{}/{}".format(laddr,
+                                                                  netmask),
+                                        'backends': {l_unit: laddr}}
                 for rid in relation_ids('cluster'):
                     for unit in related_units(rid):
-                        _unit = unit.replace('/', '-')
                         _laddr = relation_get('{}-address'.format(addr_type),
                                               rid=rid, unit=unit)
                         if _laddr:
+                            _unit = unit.replace('/', '-')
                             cluster_hosts[laddr]['backends'][_unit] = _laddr
 
         # NOTE(jamespage) no split configurations found, just use
         # private addresses
         if not cluster_hosts:
-            cluster_hosts[addr] = {}
-            cluster_hosts[addr]['network'] = "{}/{}".format(
-                addr,
-                get_netmask_for_address(addr)
-            )
-            cluster_hosts[addr]['backends'] = {}
-            cluster_hosts[addr]['backends'][l_unit] = addr
+            netmask = get_netmask_for_address(addr)
+            cluster_hosts[addr] = {'network': "{}/{}".format(addr, netmask),
+                                   'backends': {l_unit: addr}}
             for rid in relation_ids('cluster'):
                 for unit in related_units(rid):
-                    _unit = unit.replace('/', '-')
                     _laddr = relation_get('private-address',
                                           rid=rid, unit=unit)
                     if _laddr:
+                        _unit = unit.replace('/', '-')
                         cluster_hosts[addr]['backends'][_unit] = _laddr
 
-        ctxt = {
-            'frontends': cluster_hosts,
-        }
+        ctxt = {'frontends': cluster_hosts}
 
         if config('haproxy-server-timeout'):
             ctxt['haproxy_server_timeout'] = config('haproxy-server-timeout')
+
         if config('haproxy-client-timeout'):
             ctxt['haproxy_client_timeout'] = config('haproxy-client-timeout')
 
@@ -495,11 +494,15 @@
         for frontend in cluster_hosts:
             if len(cluster_hosts[frontend]['backends']) > 1:
                 # Enable haproxy when we have enough peers.
-                log('Ensuring haproxy enabled in /etc/default/haproxy.')
+                log('Ensuring haproxy enabled in /etc/default/haproxy.',
+                    level=DEBUG)
                 with open('/etc/default/haproxy', 'w') as out:
                     out.write('ENABLED=1\n')
+
                 return ctxt
-        log('HAProxy context is incomplete, this unit has no peers.')
+
+        log('HAProxy context is incomplete, this unit has no peers.',
+            level=INFO)
         return {}
 
 
@@ -507,29 +510,28 @@
     interfaces = ['image-service']
 
     def __call__(self):
-        '''
-        Obtains the glance API server from the image-service relation.  Useful
-        in nova and cinder (currently).
-        '''
-        log('Generating template context for image-service.')
+        """Obtains the glance API server from the image-service relation.
+        Useful in nova and cinder (currently).
+        """
+        log('Generating template context for image-service.', level=DEBUG)
         rids = relation_ids('image-service')
         if not rids:
             return {}
+
         for rid in rids:
             for unit in related_units(rid):
                 api_server = relation_get('glance-api-server',
                                           rid=rid, unit=unit)
                 if api_server:
                     return {'glance_api_servers': api_server}
-        log('ImageService context is incomplete. '
-            'Missing required relation data.')
+
+        log("ImageService context is incomplete. Missing required relation "
+            "data.", level=INFO)
         return {}
 
 
 class ApacheSSLContext(OSContextGenerator):
-
-    """
-    Generates a context for an apache vhost configuration that configures
+    """Generates a context for an apache vhost configuration that configures
     HTTPS reverse proxying for one or many endpoints.  Generated context
     looks something like::
 
@@ -563,6 +565,7 @@
         else:
             cert_filename = 'cert'
             key_filename = 'key'
+
         write_file(path=os.path.join(ssl_dir, cert_filename),
                    content=b64decode(cert))
         write_file(path=os.path.join(ssl_dir, key_filename),
@@ -574,7 +577,8 @@
             install_ca_cert(b64decode(ca_cert))
 
     def canonical_names(self):
-        '''Figure out which canonical names clients will access this service'''
+        """Figure out which canonical names clients will access this service.
+        """
         cns = []
         for r_id in relation_ids('identity-service'):
             for unit in related_units(r_id):
@@ -582,47 +586,71 @@
                 for k in rdata:
                     if k.startswith('ssl_key_'):
                         cns.append(k.lstrip('ssl_key_'))
+
         return list(set(cns))
 
+    def get_network_addresses(self):
+        """For each network configured, return corresponding address and vip
+           (if available).
+
+        Returns a list of tuples of the form:
+
+            [(address_in_net_a, vip_in_net_a),
+             (address_in_net_b, vip_in_net_b),
+             ...]
+
+            or, if no vip(s) available:
+
+            [(address_in_net_a, address_in_net_a),
+             (address_in_net_b, address_in_net_b),
+             ...]
+        """
+        addresses = []
+        if config('vip'):
+            vips = config('vip').split()
+        else:
+            vips = []
+
+        for net_type in ['os-internal-network', 'os-admin-network',
+                         'os-public-network']:
+            addr = get_address_in_network(config(net_type),
+                                          unit_get('private-address'))
+            if len(vips) > 1 and is_clustered():
+                if not config(net_type):
+                    log("Multiple networks configured but net_type "
+                        "is None (%s)." % net_type, level=WARNING)
+                    continue
+
+                for vip in vips:
+                    if is_address_in_network(config(net_type), vip):
+                        addresses.append((addr, vip))
+                        break
+
+            elif is_clustered() and config('vip'):
+                addresses.append((addr, config('vip')))
+            else:
+                addresses.append((addr, addr))
+
+        return addresses
+
     def __call__(self):
         if isinstance(self.external_ports, basestring):
             self.external_ports = [self.external_ports]
-        if (not self.external_ports or not https()):
+
+        if not self.external_ports or not https():
             return {}
 
         self.configure_ca()
         self.enable_modules()
 
-        ctxt = {
-            'namespace': self.service_namespace,
-            'endpoints': [],
-            'ext_ports': []
-        }
+        ctxt = {'namespace': self.service_namespace,
+                'endpoints': [],
+                'ext_ports': []}
 
         for cn in self.canonical_names():
             self.configure_cert(cn)
 
-        addresses = []
-        vips = []
-        if config('vip'):
-            vips = config('vip').split()
-
-        for network_type in ['os-internal-network',
-                             'os-admin-network',
-                             'os-public-network']:
-            address = get_address_in_network(config(network_type),
-                                             unit_get('private-address'))
-            if len(vips) > 0 and is_clustered():
-                for vip in vips:
-                    if is_address_in_network(config(network_type),
-                                             vip):
-                        addresses.append((address, vip))
-                        break
-            elif is_clustered():
-                addresses.append((address, config('vip')))
-            else:
-                addresses.append((address, address))
-
+        addresses = self.get_network_addresses()
         for address, endpoint in set(addresses):
             for api_port in self.external_ports:
                 ext_port = determine_apache_port(api_port)
@@ -630,6 +658,7 @@
                 portmap = (address, endpoint, int(ext_port), int(int_port))
                 ctxt['endpoints'].append(portmap)
                 ctxt['ext_ports'].append(int(ext_port))
+
         ctxt['ext_ports'] = list(set(ctxt['ext_ports']))
         return ctxt
 
@@ -647,21 +676,23 @@
 
     @property
     def packages(self):
-        return neutron_plugin_attribute(
-            self.plugin, 'packages', self.network_manager)
+        return neutron_plugin_attribute(self.plugin, 'packages',
+                                        self.network_manager)
 
     @property
     def neutron_security_groups(self):
         return None
 
     def _ensure_packages(self):
-        [ensure_packages(pkgs) for pkgs in self.packages]
+        for pkgs in self.packages:
+            ensure_packages(pkgs)
 
     def _save_flag_file(self):
         if self.network_manager == 'quantum':
             _file = '/etc/nova/quantum_plugin.conf'
         else:
             _file = '/etc/nova/neutron_plugin.conf'
+
         with open(_file, 'wb') as out:
             out.write(self.plugin + '\n')
 
@@ -670,13 +701,11 @@
                                           self.network_manager)
         config = neutron_plugin_attribute(self.plugin, 'config',
                                           self.network_manager)
-        ovs_ctxt = {
-            'core_plugin': driver,
-            'neutron_plugin': 'ovs',
-            'neutron_security_groups': self.neutron_security_groups,
-            'local_ip': unit_private_ip(),
-            'config': config
-        }
+        ovs_ctxt = {'core_plugin': driver,
+                    'neutron_plugin': 'ovs',
+                    'neutron_security_groups': self.neutron_security_groups,
+                    'local_ip': unit_private_ip(),
+                    'config': config}
 
         return ovs_ctxt
 
@@ -685,13 +714,11 @@
                                           self.network_manager)
         config = neutron_plugin_attribute(self.plugin, 'config',
                                           self.network_manager)
-        nvp_ctxt = {
-            'core_plugin': driver,
-            'neutron_plugin': 'nvp',
-            'neutron_security_groups': self.neutron_security_groups,
-            'local_ip': unit_private_ip(),
-            'config': config
-        }
+        nvp_ctxt = {'core_plugin': driver,
+                    'neutron_plugin': 'nvp',
+                    'neutron_security_groups': self.neutron_security_groups,
+                    'local_ip': unit_private_ip(),
+                    'config': config}
 
         return nvp_ctxt
 
@@ -700,35 +727,50 @@
                                           self.network_manager)
         n1kv_config = neutron_plugin_attribute(self.plugin, 'config',
                                                self.network_manager)
-        n1kv_ctxt = {
-            'core_plugin': driver,
-            'neutron_plugin': 'n1kv',
-            'neutron_security_groups': self.neutron_security_groups,
-            'local_ip': unit_private_ip(),
-            'config': n1kv_config,
-            'vsm_ip': config('n1kv-vsm-ip'),
-            'vsm_username': config('n1kv-vsm-username'),
-            'vsm_password': config('n1kv-vsm-password'),
-            'restrict_policy_profiles': config(
-                'n1kv_restrict_policy_profiles'),
-        }
+        n1kv_user_config_flags = config('n1kv-config-flags')
+        restrict_policy_profiles = config('n1kv-restrict-policy-profiles')
+        n1kv_ctxt = {'core_plugin': driver,
+                     'neutron_plugin': 'n1kv',
+                     'neutron_security_groups': self.neutron_security_groups,
+                     'local_ip': unit_private_ip(),
+                     'config': n1kv_config,
+                     'vsm_ip': config('n1kv-vsm-ip'),
+                     'vsm_username': config('n1kv-vsm-username'),
+                     'vsm_password': config('n1kv-vsm-password'),
+                     'restrict_policy_profiles': restrict_policy_profiles}
+
+        if n1kv_user_config_flags:
+            flags = config_flags_parser(n1kv_user_config_flags)
+            n1kv_ctxt['user_config_flags'] = flags
 
         return n1kv_ctxt
 
+    def calico_ctxt(self):
+        driver = neutron_plugin_attribute(self.plugin, 'driver',
+                                          self.network_manager)
+        config = neutron_plugin_attribute(self.plugin, 'config',
+                                          self.network_manager)
+        calico_ctxt = {'core_plugin': driver,
+                       'neutron_plugin': 'Calico',
+                       'neutron_security_groups': self.neutron_security_groups,
+                       'local_ip': unit_private_ip(),
+                       'config': config}
+
+        return calico_ctxt
+
     def neutron_ctxt(self):
         if https():
             proto = 'https'
         else:
             proto = 'http'
+
         if is_clustered():
             host = config('vip')
         else:
             host = unit_get('private-address')
-        url = '%s://%s:%s' % (proto, host, '9696')
-        ctxt = {
-            'network_manager': self.network_manager,
-            'neutron_url': url,
-        }
+
+        ctxt = {'network_manager': self.network_manager,
+                'neutron_url': '%s://%s:%s' % (proto, host, '9696')}
         return ctxt
 
     def __call__(self):
@@ -748,6 +790,8 @@
             ctxt.update(self.nvp_ctxt())
         elif self.plugin == 'n1kv':
             ctxt.update(self.n1kv_ctxt())
+        elif self.plugin == 'Calico':
+            ctxt.update(self.calico_ctxt())
 
         alchemy_flags = config('neutron-alchemy-flags')
         if alchemy_flags:
@@ -759,23 +803,40 @@
 
 
 class OSConfigFlagContext(OSContextGenerator):
-
-    """
-    Responsible for adding user-defined config-flags in charm config to a
-    template context.
+    """Provides support for user-defined config flags.
+
+    Users can define a comma-seperated list of key=value pairs
+    in the charm configuration and apply them at any point in
+    any file by using a template flag.
+
+    Sometimes users might want config flags inserted within a
+    specific section so this class allows users to specify the
+    template flag name, allowing for multiple template flags
+    (sections) within the same context.
 
     NOTE: the value of config-flags may be a comma-separated list of
           key=value pairs and some Openstack config files support
           comma-separated lists as values.
     """
 
+    def __init__(self, charm_flag='config-flags',
+                 template_flag='user_config_flags'):
+        """
+        :param charm_flag: config flags in charm configuration.
+        :param template_flag: insert point for user-defined flags in template
+                              file.
+        """
+        super(OSConfigFlagContext, self).__init__()
+        self._charm_flag = charm_flag
+        self._template_flag = template_flag
+
     def __call__(self):
-        config_flags = config('config-flags')
+        config_flags = config(self._charm_flag)
         if not config_flags:
             return {}
 
-        flags = config_flags_parser(config_flags)
-        return {'user_config_flags': flags}
+        return {self._template_flag:
+                config_flags_parser(config_flags)}
 
 
 class SubordinateConfigContext(OSContextGenerator):
@@ -819,7 +880,6 @@
                 },
             }
         }
-
     """
 
     def __init__(self, service, config_file, interface):
@@ -849,26 +909,28 @@
 
                     if self.service not in sub_config:
                         log('Found subordinate_config on %s but it contained'
-                            'nothing for %s service' % (rid, self.service))
+                            'nothing for %s service' % (rid, self.service),
+                            level=INFO)
                         continue
 
                     sub_config = sub_config[self.service]
                     if self.config_file not in sub_config:
                         log('Found subordinate_config on %s but it contained'
-                            'nothing for %s' % (rid, self.config_file))
+                            'nothing for %s' % (rid, self.config_file),
+                            level=INFO)
                         continue
 
                     sub_config = sub_config[self.config_file]
                     for k, v in sub_config.iteritems():
                         if k == 'sections':
                             for section, config_dict in v.iteritems():
-                                log("adding section '%s'" % (section))
+                                log("Adding section '%s'" % (section),
+                                    level=DEBUG)
                                 ctxt[k][section] = config_dict
                         else:
                             ctxt[k] = v
 
-        log("%d section(s) found" % (len(ctxt['sections'])), level=INFO)
-
+        log("%d section(s) found" % (len(ctxt['sections'])), level=DEBUG)
         return ctxt
 
 
@@ -880,15 +942,14 @@
             False if config('debug') is None else config('debug')
         ctxt['verbose'] = \
             False if config('verbose') is None else config('verbose')
+
         return ctxt
 
 
 class SyslogContext(OSContextGenerator):
 
     def __call__(self):
-        ctxt = {
-            'use_syslog': config('use-syslog')
-        }
+        ctxt = {'use_syslog': config('use-syslog')}
         return ctxt
 
 
@@ -896,13 +957,9 @@
 
     def __call__(self):
         if config('prefer-ipv6'):
-            return {
-                'bind_host': '::'
-            }
+            return {'bind_host': '::'}
         else:
-            return {
-                'bind_host': '0.0.0.0'
-            }
+            return {'bind_host': '0.0.0.0'}
 
 
 class WorkerConfigContext(OSContextGenerator):
@@ -914,11 +971,42 @@
         except ImportError:
             apt_install('python-psutil', fatal=True)
             from psutil import NUM_CPUS
+
         return NUM_CPUS
 
     def __call__(self):
-        multiplier = config('worker-multiplier') or 1
-        ctxt = {
-            "workers": self.num_cpus * multiplier
-        }
+        multiplier = config('worker-multiplier') or 0
+        ctxt = {"workers": self.num_cpus * multiplier}
+        return ctxt
+
+
+class ZeroMQContext(OSContextGenerator):
+    interfaces = ['zeromq-configuration']
+
+    def __call__(self):
+        ctxt = {}
+        if is_relation_made('zeromq-configuration', 'host'):
+            for rid in relation_ids('zeromq-configuration'):
+                    for unit in related_units(rid):
+                        ctxt['zmq_nonce'] = relation_get('nonce', unit, rid)
+                        ctxt['zmq_host'] = relation_get('host', unit, rid)
+
+        return ctxt
+
+
+class NotificationDriverContext(OSContextGenerator):
+
+    def __init__(self, zmq_relation='zeromq-configuration',
+                 amqp_relation='amqp'):
+        """
+        :param zmq_relation: Name of Zeromq relation to check
+        """
+        self.zmq_relation = zmq_relation
+        self.amqp_relation = amqp_relation
+
+    def __call__(self):
+        ctxt = {'notifications': 'False'}
+        if is_relation_made(self.amqp_relation):
+            ctxt['notifications'] = "True"
+
         return ctxt
 
=== modified file 'hooks/charmhelpers/contrib/openstack/neutron.py'
--- hooks/charmhelpers/contrib/openstack/neutron.py	2014-07-28 14:41:41 +0000
+++ hooks/charmhelpers/contrib/openstack/neutron.py	2014-11-21 15:33:38 +0000
@@ -138,10 +138,25 @@
                                         relation_prefix='neutron',
                                         ssl_dir=NEUTRON_CONF_DIR)],
             'services': [],
-            'packages': [['neutron-plugin-cisco']],
+            'packages': [[headers_package()] + determine_dkms_package(),
+                         ['neutron-plugin-cisco']],
             'server_packages': ['neutron-server',
                                 'neutron-plugin-cisco'],
             'server_services': ['neutron-server']
+        },
+        'Calico': {
+            'config': '/etc/neutron/plugins/ml2/ml2_conf.ini',
+            'driver': 'neutron.plugins.ml2.plugin.Ml2Plugin',
+            'contexts': [
+                context.SharedDBContext(user=config('neutron-database-user'),
+                                        database=config('neutron-database'),
+                                        relation_prefix='neutron',
+                                        ssl_dir=NEUTRON_CONF_DIR)],
+            'services': ['calico-compute', 'bird', 'neutron-dhcp-agent'],
+            'packages': [[headers_package()] + determine_dkms_package(),
+                         ['calico-compute', 'bird', 'neutron-dhcp-agent']],
+            'server_packages': ['neutron-server', 'calico-control'],
+            'server_services': ['neutron-server']
         }
     }
     if release >= 'icehouse':
@@ -162,7 +177,8 @@
     elif manager == 'neutron':
         plugins = neutron_plugins()
     else:
-        log('Error: Network manager does not support plugins.')
+        log("Network manager '%s' does not support plugins." % (manager),
+            level=ERROR)
         raise Exception
 
     try:
 
=== modified file 'hooks/charmhelpers/contrib/openstack/utils.py'
--- hooks/charmhelpers/contrib/openstack/utils.py	2014-10-06 21:03:50 +0000
+++ hooks/charmhelpers/contrib/openstack/utils.py	2014-11-21 15:33:38 +0000
@@ -2,6 +2,7 @@
 
 # Common python helper functions used for OpenStack charms.
 from collections import OrderedDict
+from functools import wraps
 
 import subprocess
 import json
@@ -468,6 +469,14 @@
         return result.split('.')[0]
 
 
+def get_matchmaker_map(mm_file='/etc/oslo/matchmaker_ring.json'):
+    mm_map = {}
+    if os.path.isfile(mm_file):
+        with open(mm_file, 'r') as f:
+            mm_map = json.load(f)
+    return mm_map
+
+
 def sync_db_with_multi_ipv6_addresses(database, database_user,
                                       relation_prefix=None):
     hosts = get_ipv6_addr(dynamic_only=False)
@@ -484,3 +493,18 @@
 
     for rid in relation_ids('shared-db'):
         relation_set(relation_id=rid, **kwargs)
+
+
+def os_requires_version(ostack_release, pkg):
+    """
+    Decorator for hook to specify minimum supported release
+    """
+    def wrap(f):
+        @wraps(f)
+        def wrapped_f(*args):
+            if os_release(pkg) < ostack_release:
+                raise Exception("This hook is not supported on releases"
+                                " before %s" % ostack_release)
+            f(*args)
+        return wrapped_f
+    return wrap
 
=== modified file 'hooks/charmhelpers/contrib/storage/linux/ceph.py'
--- hooks/charmhelpers/contrib/storage/linux/ceph.py	2014-07-28 14:41:41 +0000
+++ hooks/charmhelpers/contrib/storage/linux/ceph.py	2014-11-21 15:33:38 +0000
@@ -16,19 +16,18 @@
 from subprocess import (
     check_call,
     check_output,
-    CalledProcessError
+    CalledProcessError,
 )
-
 from charmhelpers.core.hookenv import (
     relation_get,
     relation_ids,
     related_units,
     log,
+    DEBUG,
     INFO,
     WARNING,
-    ERROR
+    ERROR,
 )
-
 from charmhelpers.core.host import (
     mount,
     mounts,
@@ -37,7 +36,6 @@
     service_running,
     umount,
 )
-
 from charmhelpers.fetch import (
     apt_install,
 )
@@ -56,69 +54,60 @@
 
 
 def install():
-    ''' Basic Ceph client installation '''
+    """Basic Ceph client installation."""
     ceph_dir = "/etc/ceph"
     if not os.path.exists(ceph_dir):
         os.mkdir(ceph_dir)
+
     apt_install('ceph-common', fatal=True)
 
 
 def rbd_exists(service, pool, rbd_img):
-    ''' Check to see if a RADOS block device exists '''
+    """Check to see if a RADOS block device exists."""
     try:
-        out = check_output(['rbd', 'list', '--id', service,
-                            '--pool', pool])
+        out = check_output(['rbd', 'list', '--id', service, '--pool', pool])
     except CalledProcessError:
         return False
-    else:
-        return rbd_img in out
+
+    return rbd_img in out
 
 
 def create_rbd_image(service, pool, image, sizemb):
-    ''' Create a new RADOS block device '''
-    cmd = [
-        'rbd',
-        'create',
-        image,
-        '--size',
-        str(sizemb),
-        '--id',
-        service,
-        '--pool',
-        pool
-    ]
+    """Create a new RADOS block device."""
+    cmd = ['rbd', 'create', image, '--size', str(sizemb), '--id', service,
+           '--pool', pool]
     check_call(cmd)
 
 
 def pool_exists(service, name):
-    ''' Check to see if a RADOS pool already exists '''
+    """Check to see if a RADOS pool already exists."""
     try:
         out = check_output(['rados', '--id', service, 'lspools'])
     except CalledProcessError:
         return False
-    else:
-        return name in out
+
+    return name in out
 
 
 def get_osds(service):
-    '''
-    Return a list of all Ceph Object Storage Daemons
-    currently in the cluster
-    '''
+    """Return a list of all Ceph Object Storage Daemons currently in the
+    cluster.
+    """
     version = ceph_version()
     if version and version >= '0.56':
         return json.loads(check_output(['ceph', '--id', service,
                                         'osd', 'ls', '--format=json']))
-    else:
-        return None
-
-
-def create_pool(service, name, replicas=2):
-    ''' Create a new RADOS pool '''
+
+    return None
+
+
+def create_pool(service, name, replicas=3):
+    """Create a new RADOS pool."""
     if pool_exists(service, name):
         log("Ceph pool {} already exists, skipping creation".format(name),
             level=WARNING)
         return
+
     # Calculate the number of placement groups based
     # on upstream recommended best practices.
     osds = get_osds(service)
@@ -128,27 +117,19 @@
         # NOTE(james-page): Default to 200 for older ceph versions
         # which don't support OSD query from cli
         pgnum = 200
-    cmd = [
-        'ceph', '--id', service,
-        'osd', 'pool', 'create',
-        name, str(pgnum)
-    ]
+
+    cmd = ['ceph', '--id', service, 'osd', 'pool', 'create', name, str(pgnum)]
     check_call(cmd)
-    cmd = [
-        'ceph', '--id', service,
-        'osd', 'pool', 'set', name,
-        'size', str(replicas)
-    ]
+
+    cmd = ['ceph', '--id', service, 'osd', 'pool', 'set', name, 'size',
+           str(replicas)]
     check_call(cmd)
 
 
 def delete_pool(service, name):
-    ''' Delete a RADOS pool from ceph '''
-    cmd = [
-        'ceph', '--id', service,
-        'osd', 'pool', 'delete',
-        name, '--yes-i-really-really-mean-it'
-    ]
+    """Delete a RADOS pool from ceph."""
+    cmd = ['ceph', '--id', service, 'osd', 'pool', 'delete', name,
+           '--yes-i-really-really-mean-it']
     check_call(cmd)
 
 
@@ -161,44 +142,43 @@
 
 
 def create_keyring(service, key):
-    ''' Create a new Ceph keyring containing key'''
+    """Create a new Ceph keyring containing key."""
     keyring = _keyring_path(service)
     if os.path.exists(keyring):
-        log('ceph: Keyring exists at %s.' % keyring, level=WARNING)
+        log('Ceph keyring exists at %s.' % keyring, level=WARNING)
         return
-    cmd = [
-        'ceph-authtool',
-        keyring,
-        '--create-keyring',
-        '--name=client.{}'.format(service),
-        '--add-key={}'.format(key)
-    ]
+
+    cmd = ['ceph-authtool', keyring, '--create-keyring',
+           '--name=client.{}'.format(service), '--add-key={}'.format(key)]
     check_call(cmd)
-    log('ceph: Created new ring at %s.' % keyring, level=INFO)
+    log('Created new ceph keyring at %s.' % keyring, level=DEBUG)
 
 
 def create_key_file(service, key):
-    ''' Create a file containing key '''
+    """Create a file containing key."""
     keyfile = _keyfile_path(service)
     if os.path.exists(keyfile):
-        log('ceph: Keyfile exists at %s.' % keyfile, level=WARNING)
+        log('Keyfile exists at %s.' % keyfile, level=WARNING)
         return
+
     with open(keyfile, 'w') as fd:
         fd.write(key)
-    log('ceph: Created new keyfile at %s.' % keyfile, level=INFO)
+
+    log('Created new keyfile at %s.' % keyfile, level=INFO)
 
 
 def get_ceph_nodes():
-    ''' Query named relation 'ceph' to detemine current nodes '''
+    """Query named relation 'ceph' to determine current nodes."""
     hosts = []
     for r_id in relation_ids('ceph'):
         for unit in related_units(r_id):
             hosts.append(relation_get('private-address', unit=unit, rid=r_id))
+
     return hosts
 
 
 def configure(service, key, auth, use_syslog):
-    ''' Perform basic configuration of Ceph '''
+    """Perform basic configuration of Ceph."""
     create_keyring(service, key)
     create_key_file(service, key)
     hosts = get_ceph_nodes()
@@ -211,17 +191,17 @@
 
 
 def image_mapped(name):
-    ''' Determine whether a RADOS block device is mapped locally '''
+    """Determine whether a RADOS block device is mapped locally."""
     try:
         out = check_output(['rbd', 'showmapped'])
     except CalledProcessError:
         return False
-    else:
-        return name in out
+
+    return name in out
 
 
 def map_block_storage(service, pool, image):
-    ''' Map a RADOS block device for local use '''
+    """Map a RADOS block device for local use."""
     cmd = [
         'rbd',
         'map',
@@ -235,31 +215,32 @@
 
 
 def filesystem_mounted(fs):
-    ''' Determine whether a filesytems is already mounted '''
+    """Determine whether a filesytems is already mounted."""
     return fs in [f for f, m in mounts()]
 
 
 def make_filesystem(blk_device, fstype='ext4', timeout=10):
-    ''' Make a new filesystem on the specified block device '''
+    """Make a new filesystem on the specified block device."""
     count = 0
     e_noent = os.errno.ENOENT
     while not os.path.exists(blk_device):
         if count >= timeout:
-            log('ceph: gave up waiting on block device %s' % blk_device,
+            log('Gave up waiting on block device %s' % blk_device,
                 level=ERROR)
             raise IOError(e_noent, os.strerror(e_noent), blk_device)
-        log('ceph: waiting for block device %s to appear' % blk_device,
-            level=INFO)
+
+        log('Waiting for block device %s to appear' % blk_device,
+            level=DEBUG)
         count += 1
         time.sleep(1)
     else:
-        log('ceph: Formatting block device %s as filesystem %s.' %
+        log('Formatting block device %s as filesystem %s.' %
             (blk_device, fstype), level=INFO)
         check_call(['mkfs', '-t', fstype, blk_device])
 
 
 def place_data_on_block_device(blk_device, data_src_dst):
-    ''' Migrate data in data_src_dst to blk_device and then remount '''
+    """Migrate data in data_src_dst to blk_device and then remount."""
     # mount block device into /mnt
     mount(blk_device, '/mnt')
     # copy data to /mnt
@@ -279,8 +260,8 @@
 
 # TODO: re-use
 def modprobe(module):
-    ''' Load a kernel module and configure for auto-load on reboot '''
-    log('ceph: Loading kernel module', level=INFO)
+    """Load a kernel module and configure for auto-load on reboot."""
+    log('Loading kernel module', level=INFO)
     cmd = ['modprobe', module]
     check_call(cmd)
     with open('/etc/modules', 'r+') as modules:
@@ -289,7 +270,7 @@
 
 
 def copy_files(src, dst, symlinks=False, ignore=None):
-    ''' Copy files from src to dst '''
+    """Copy files from src to dst."""
     for item in os.listdir(src):
         s = os.path.join(src, item)
         d = os.path.join(dst, item)
@@ -300,9 +281,9 @@
 
 
 def ensure_ceph_storage(service, pool, rbd_img, sizemb, mount_point,
-                        blk_device, fstype, system_services=[]):
-    """
-    NOTE: This function must only be called from a single service unit for
+                        blk_device, fstype, system_services=[],
+                        replicas=3):
+    """NOTE: This function must only be called from a single service unit for
     the same rbd_img otherwise data loss will occur.
 
     Ensures given pool and RBD image exists, is mapped to a block device,
@@ -316,15 +297,16 @@
     """
     # Ensure pool, RBD image, RBD mappings are in place.
     if not pool_exists(service, pool):
-        log('ceph: Creating new pool {}.'.format(pool))
-        create_pool(service, pool)
+        log('Creating new pool {}.'.format(pool), level=INFO)
+        create_pool(service, pool, replicas=replicas)
 
     if not rbd_exists(service, pool, rbd_img):
-        log('ceph: Creating RBD image ({}).'.format(rbd_img))
+        log('Creating RBD image ({}).'.format(rbd_img), level=INFO)
         create_rbd_image(service, pool, rbd_img, sizemb)
 
     if not image_mapped(rbd_img):
-        log('ceph: Mapping RBD Image {} as a Block Device.'.format(rbd_img))
+        log('Mapping RBD Image {} as a Block Device.'.format(rbd_img),
+            level=INFO)
         map_block_storage(service, pool, rbd_img)
 
     # make file system
@@ -339,42 +321,44 @@
 
         for svc in system_services:
             if service_running(svc):
-                log('ceph: Stopping services {} prior to migrating data.'
-                    .format(svc))
+                log('Stopping services {} prior to migrating data.'
+                    .format(svc), level=DEBUG)
                 service_stop(svc)
 
         place_data_on_block_device(blk_device, mount_point)
 
         for svc in system_services:
-            log('ceph: Starting service {} after migrating data.'
-                .format(svc))
+            log('Starting service {} after migrating data.'
+                .format(svc), level=DEBUG)
             service_start(svc)
 
 
 def ensure_ceph_keyring(service, user=None, group=None):
-    '''
-    Ensures a ceph keyring is created for a named service
-    and optionally ensures user and group ownership.
+    """Ensures a ceph keyring is created for a named service and optionally
+    ensures user and group ownership.
 
     Returns False if no ceph key is available in relation state.
-    '''
+    """
     key = None
     for rid in relation_ids('ceph'):
         for unit in related_units(rid):
             key = relation_get('key', rid=rid, unit=unit)
             if key:
                 break
+
     if not key:
         return False
+
     create_keyring(service=service, key=key)
     keyring = _keyring_path(service)
     if user and group:
         check_call(['chown', '%s.%s' % (user, group), keyring])
+
     return True
 
 
 def ceph_version():
-    ''' Retrieve the local version of ceph '''
+    """Retrieve the local version of ceph."""
     if os.path.exists('/usr/bin/ceph'):
         cmd = ['ceph', '-v']
         output = check_output(cmd)
 
=== modified file 'hooks/charmhelpers/core/hookenv.py'
--- hooks/charmhelpers/core/hookenv.py	2014-10-06 21:03:50 +0000
+++ hooks/charmhelpers/core/hookenv.py	2014-11-21 15:33:38 +0000
@@ -214,6 +214,12 @@
         except KeyError:
             return (self._prev_dict or {})[key]
 
+    def keys(self):
+        prev_keys = []
+        if self._prev_dict is not None:
+            prev_keys = self._prev_dict.keys()
+        return list(set(prev_keys + dict.keys(self)))
+
     def load_previous(self, path=None):
         """Load previous copy of config from disk.
 
 
=== modified file 'hooks/charmhelpers/core/host.py'
--- hooks/charmhelpers/core/host.py	2014-10-06 21:03:50 +0000
+++ hooks/charmhelpers/core/host.py	2014-11-21 15:33:38 +0000
@@ -6,13 +6,13 @@
 #  Matthew Wedgwood <[e-mail]>
 
 import os
+import re
 import pwd
 import grp
 import random
 import string
 import subprocess
 import hashlib
-import shutil
 from contextlib import contextmanager
 
 from collections import OrderedDict
@@ -317,7 +317,13 @@
         ip_output = (line for line in ip_output if line)
         for line in ip_output:
             if line.split()[1].startswith(int_type):
-                interfaces.append(line.split()[1].replace(":", ""))
+                matched = re.search('.*: (bond[0-9]+\.[0-9]+)@.*', line)
+                if matched:
+                    interface = matched.groups()[0]
+                else:
+                    interface = line.split()[1].replace(":", "")
+                interfaces.append(interface)
+
     return interfaces
 
 
 
=== modified file 'hooks/charmhelpers/core/services/__init__.py'
--- hooks/charmhelpers/core/services/__init__.py	2014-08-13 13:12:02 +0000
+++ hooks/charmhelpers/core/services/__init__.py	2014-11-21 15:33:38 +0000
@@ -1,2 +1,2 @@
-from .base import *
-from .helpers import *
+from .base import *  # NOQA
+from .helpers import *  # NOQA
 
=== modified file 'hooks/charmhelpers/fetch/__init__.py'
--- hooks/charmhelpers/fetch/__init__.py	2014-10-06 21:03:50 +0000
+++ hooks/charmhelpers/fetch/__init__.py	2014-11-21 15:33:38 +0000
@@ -72,6 +72,7 @@
 FETCH_HANDLERS = (
     'charmhelpers.fetch.archiveurl.ArchiveUrlFetchHandler',
     'charmhelpers.fetch.bzrurl.BzrUrlFetchHandler',
+    'charmhelpers.fetch.giturl.GitUrlFetchHandler',
 )
 
 APT_NO_LOCK = 100  # The return code for "couldn't acquire lock" in APT.
@@ -218,6 +219,7 @@
         pocket for the release.
         'cloud:' may be used to activate official cloud archive pockets,
         such as 'cloud:icehouse'
+        'distro' may be used as a noop
 
     @param key: A key to be added to the system's APT keyring and used
     to verify the signatures on packages. Ideally, this should be an
@@ -251,8 +253,10 @@
         release = lsb_release()['DISTRIB_CODENAME']
         with open('/etc/apt/sources.list.d/proposed.list', 'w') as apt:
             apt.write(PROPOSED_POCKET.format(release))
+    elif source == 'distro':
+        pass
     else:
-        raise SourceConfigError("Unknown source: {!r}".format(source))
+        log("Unknown source: {!r}".format(source))
 
     if key:
         if '-----BEGIN PGP PUBLIC KEY BLOCK-----' in key:
 
=== added file 'hooks/charmhelpers/fetch/giturl.py'
--- hooks/charmhelpers/fetch/giturl.py	1970-01-01 00:00:00 +0000
+++ hooks/charmhelpers/fetch/giturl.py	2014-11-21 15:33:38 +0000
@@ -0,0 +1,44 @@
+import os
+from charmhelpers.fetch import (
+    BaseFetchHandler,
+    UnhandledSource
+)
+from charmhelpers.core.host import mkdir
+
+try:
+    from git import Repo
+except ImportError:
+    from charmhelpers.fetch import apt_install
+    apt_install("python-git")
+    from git import Repo
+
+
+class GitUrlFetchHandler(BaseFetchHandler):
+    """Handler for git branches via generic and github URLs"""
+    def can_handle(self, source):
+        url_parts = self.parse_url(source)
+        #TODO (mattyw) no support for ssh git@ yet
+        if url_parts.scheme not in ('http', 'https', 'git'):
+            return False
+        else:
+            return True
+
+    def clone(self, source, dest, branch):
+        if not self.can_handle(source):
+            raise UnhandledSource("Cannot handle {}".format(source))
+
+        repo = Repo.clone_from(source, dest)
+        repo.git.checkout(branch)
+
+    def install(self, source, branch="master"):
+        url_parts = self.parse_url(source)
+        branch_name = url_parts.path.strip("/").split("/")[-1]
+        dest_dir = os.path.join(os.environ.get('CHARM_DIR'), "fetched",
+                                branch_name)
+        if not os.path.exists(dest_dir):
+            mkdir(dest_dir, perms=0755)
+        try:
+            self.clone(source, dest_dir, branch)
+        except OSError as e:
+            raise UnhandledSource(e.strerror)
+        return dest_dir
 
=== added file 'templates/icehouse/cisco_plugins.ini'
--- templates/icehouse/cisco_plugins.ini	1970-01-01 00:00:00 +0000
+++ templates/icehouse/cisco_plugins.ini	2014-11-21 15:33:38 +0000
@@ -0,0 +1,43 @@
+###############################################################################
+# [ WARNING ]
+# Configuration file maintained by Juju. Local changes may be overwritten.
+###############################################################################
+[cisco_plugins]
+
+[cisco]
+
+[cisco_n1k]
+integration_bridge = br-int
+default_policy_profile = default-pp
+network_node_policy_profile = default-pp
+{% if openstack_release != 'havana' -%}
+http_timeout = 120
+# (BoolOpt) Specify whether plugin should attempt to synchronize with the VSM
+# when neutron is started.
+# Default value: False, indicating no full sync will be performed.
+#
+enable_sync_on_start = False
+{% endif -%}
+restrict_policy_profiles = {{ restrict_policy_profiles }}
+{% if n1kv_user_config_flags -%}
+{% for key, value in n1kv_user_config_flags.iteritems() -%}
+{{ key }} = {{ value }}
+{% endfor -%}
+{% endif -%}
+
+[CISCO_PLUGINS]
+vswitch_plugin = neutron.plugins.cisco.n1kv.n1kv_neutron_plugin.N1kvNeutronPluginV2
+
+[N1KV:{{ vsm_ip }}]
+password = {{ vsm_password }}
+username = {{ vsm_username }}
+
+{% include "parts/section-database" %}
+
+[securitygroup]
+{% if neutron_security_groups -%}
+firewall_driver = neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver
+enable_security_group = True
+{% else -%}
+firewall_driver = neutron.agent.firewall.NoopFirewallDriver
+{% endif -%}
 
=== modified file 'templates/icehouse/nova.conf'
--- templates/icehouse/nova.conf	2014-10-07 11:37:20 +0000
+++ templates/icehouse/nova.conf	2014-11-21 15:33:38 +0000
@@ -76,6 +76,18 @@
 {% endif -%}
 {% endif -%}
 
+{% if neutron_plugin and neutron_plugin == 'n1kv' -%}
+libvirt_user_virtio_for_bridges =  True
+nova_firewall_driver = nova.virt.firewall.NoopFirewallDriver
+{% if neutron_security_groups -%}
+security_group_api = {{ network_manager }}
+nova_firewall_driver = nova.virt.firewall.NoopFirewallDriver
+{% endif -%}
+{% if external_network -%}
+default_floating_pool = {{ external_network }}
+{% endif -%}
+{% endif -%}
+
 {% if network_manager_config -%}
 {% for key, value in network_manager_config.iteritems() -%}
 {{ key }} = {{ value }}
 
=== modified file 'tests/basic_deployment.py'
--- tests/basic_deployment.py	2014-10-14 15:17:57 +0000
+++ tests/basic_deployment.py	2014-11-21 15:33:38 +0000
@@ -19,7 +19,7 @@
 class NovaCCBasicDeployment(OpenStackAmuletDeployment):
     """Amulet tests on a basic nova cloud controller deployment."""
 
-    def __init__(self, series=None, openstack=None, source=None, stable=False):
+    def __init__(self, series=None, openstack=None, source=None, stable=True):
         """Deploy the entire test environment."""
         super(NovaCCBasicDeployment, self).__init__(series, openstack, source, stable)
         self._add_services()