Merge lp:~plumgrid-team/charms/trusty/plumgrid-director/trunk into lp:charms/trusty/plumgrid-director

[Bilal Baqar]

[lp:~plumgrid-team/charms/trusty/plumgrid-director/trunk]

24. By Bilal Baqar on 2016-01-15

Merge: config is pushed in install hook now so that config-changed hook is run completely

[Review Queue (review-queue)]

This item has failed automated testing! Results available here http://juju-ci.vapour.ws:8080/job/charm-bundle-test-lxc/2161/

review: Needs Fixing (automated testing)

[Review Queue (review-queue)]

This item has failed automated testing! Results available here http://juju-ci.vapour.ws:8080/job/charm-bundle-test-aws/2141/

review: Needs Fixing (automated testing)

[lp:~plumgrid-team/charms/trusty/plumgrid-director/trunk]

25. By Bilal Baqar on 2016-01-26

Merge: setting hostname of lxc to the hostname of the node

[Charles Butler (lazypower)]

Greetings Bilal,

This branch doesn't appear to apply cleanly. Can you take a look and resolve the merge conflicts?

review: Needs Fixing

[Bilal Baqar (bbaqar)]

Hey Charles

Thanks for taking the time out to review the merge proposal. I ll deal with the conflicts.

[lp:~plumgrid-team/charms/trusty/plumgrid-director/trunk]

26. By Bilal Baqar on 2016-03-03

[Merge] - Fixing lint

27. By Bilal Baqar on 2016-03-14

Made the following changes:
1. Reordered file and module imports
2. Sorted director IPs
3. Added unit fqdn in /etc/hosts of plumgrid-lxc
4. Loading plumgrid specific iptables on install
5. Added temporary upgrade hook to load iptables
6. stop_pg() is being used in restart_pg()
7. persistant iptables

28. By Bilal Baqar on 2016-03-25

Improved config-changed hook to perform steps according to the config changed

29. By Bilal Baqar on 2016-03-28

OPSVM Changes - Ticket: [SOL-830]
- Getting OPSVM IP from charm configs
- Making OPSVM specific changes
- Setting OPSVM IP in relation with edge/gw
- Cleaned code in various functions
- Added restart_on_change decorater function that restarts plumgrid service only when there has been any change in the configuration files
- Removed restart of plumgrid service when only two directors are available
- Fixed unit tests accordingly

30. By Bilal Baqar on 2016-04-08

Adding status messages in charms - Ticket:[SOL-949]

31. By Bilal Baqar on 2016-04-25

Merge: Liberty/Mitaka support

32. By Bilal Baqar on 2016-05-18

Merge - Mitaka changes
- Created new relation with neutron-api-plumgrid
- getting pg creds in config
- nginx conf changes for middleware

33. By Bilal Baqar on 2016-05-28

5.1 changes
- configure-pg-sources added
- updated templates

34. By Junaid Ali on 2016-06-19

L3 fabric changes

35. By Junaid Ali on 2016-07-28

Changes:
    - Default value for management interface removed as
      'juju-br0' has been changed to 'br-eth0' in Juju 2.0
    - Updated get_mgmt_interface method

Unmerged revisions

35. By Junaid Ali on 2016-07-28

Changes:
    - Default value for management interface removed as
      'juju-br0' has been changed to 'br-eth0' in Juju 2.0
    - Updated get_mgmt_interface method

=== modified file 'Makefile'
--- Makefile	2015-08-24 16:18:48 +0000
+++ Makefile	2016-04-08 08:30:24 +0000
@@ -7,7 +7,7 @@
         netaddr jinja2
 
 lint: virtualenv
-	.venv/bin/flake8 --exclude hooks/charmhelpers hooks unit_tests tests
+	.venv/bin/flake8 --exclude hooks/charmhelpers hooks unit_tests tests --ignore E402
 	@charm proof
 
 unit_test: virtualenv

=== modified file 'config.yaml'
--- config.yaml	2015-08-16 19:04:53 +0000
+++ config.yaml	2016-04-08 08:30:24 +0000
@@ -7,6 +7,28 @@
     default: 'null'
     type: string
     description: Public SSH key of PLUMgrid LCM which is running PG-Tools.
+  mgmt-interface:
+    type: string
+    default: 'juju-br0'
+    description: The interface connected to PLUMgrid Managment network.
+  os-data-network:
+    type: string
+    default:
+    description: |
+      The IP address and netmask of the OpenStack Data network (e.g.,
+      192.168.0.0/24)
+      .
+      This network will be used for tenant network traffic in overlay
+      networks.
+  fabric-interfaces:
+    default: 'MANAGEMENT'
+    type: string
+    description: |
+       Interfaces that will provide fabric connectivity on the director nodes.
+       Provided in form of json in a string. These interfaces have to be connected
+       to the os-data-network specified in the config. Default value is MANAGEMENT which
+       will configure the management interface as the fabric interface on each
+       director.
   network-device-mtu:
     type: string
     default: '1580'
@@ -19,7 +41,21 @@
     default: null
     type: string
     description: Provide the respective keys of the install sources.
+  plumgrid-build:
+    default: 'latest'
+    type: string
+    description: |
+       Provide the build version of PLUMgrid packages that needs to be installed
+  iovisor-build:
+    default: 'latest'
+    type: string
+    description: |
+       Provide the build version of iovisor package that needs to be installed
   plumgrid-license-key:
     default: null
     type: string
     description: Provide the PLUMgrid ONS License key.
+  opsvm-ip:
+    default: 127.0.0.1
+    type: string
+    description: IP address of the PLUMgrid Operations VM Management interface.

=== modified file 'hooks/charmhelpers/contrib/openstack/neutron.py'
--- hooks/charmhelpers/contrib/openstack/neutron.py	2015-07-29 18:07:31 +0000
+++ hooks/charmhelpers/contrib/openstack/neutron.py	2016-04-08 08:30:24 +0000
@@ -204,8 +204,8 @@
                                         database=config('database'),
                                         ssl_dir=NEUTRON_CONF_DIR)],
             'services': [],
-            'packages': [['plumgrid-lxc'],
-                         ['iovisor-dkms']],
+            'packages': ['plumgrid-lxc',
+                         'iovisor-dkms'],
             'server_packages': ['neutron-server',
                                 'neutron-plugin-plumgrid'],
             'server_services': ['neutron-server']

=== modified file 'hooks/pg_dir_context.py'
--- hooks/pg_dir_context.py	2015-08-24 16:18:48 +0000
+++ hooks/pg_dir_context.py	2016-04-08 08:30:24 +0000
@@ -3,6 +3,9 @@
 # This file contains the class that generates context
 # for PLUMgrid template files.
 
+import re
+from charmhelpers.contrib.openstack import context
+from charmhelpers.contrib.openstack.utils import get_host_ip
 from charmhelpers.core.hookenv import (
     config,
     unit_get,
@@ -12,12 +15,15 @@
     related_units,
     relation_get,
 )
-from charmhelpers.contrib.openstack import context
-from charmhelpers.contrib.openstack.utils import get_host_ip
-from charmhelpers.contrib.network.ip import get_address_in_network
+from charmhelpers.contrib.network.ip import (
+    is_ip,
+    get_address_in_network,
+)
 
-import re
-from socket import gethostname as get_unit_hostname
+from socket import (
+    gethostname,
+    getfqdn
+)
 
 
 def _pg_dir_ips():
@@ -25,12 +31,11 @@
     Inspects plumgrid-director peer relation and returns the
     ips of the peer directors
     '''
-    pg_dir_ips = []
-    for rid in relation_ids('director'):
-        for unit in related_units(rid):
-            rdata = relation_get(rid=rid, unit=unit)
-            pg_dir_ips.append(rdata['private-address'])
-    return pg_dir_ips
+    return [get_host_ip(rdata['private-address'])
+            for rid in relation_ids("director")
+            for rdata in
+            (relation_get(rid=rid, unit=unit) for unit in related_units(rid))
+            if rdata]
 
 
 class PGDirContext(context.NeutronContext):
@@ -71,24 +76,29 @@
         pg_dir_ips = _pg_dir_ips()
         pg_dir_ips.append(str(get_address_in_network(network=None,
                           fallback=get_host_ip(unit_get('private-address')))))
+        pg_dir_ips = sorted(pg_dir_ips)
         pg_ctxt['director_ips'] = pg_dir_ips
-        pg_dir_ips_string = ''
-        single_ip = True
-        for ip in pg_dir_ips:
-            if single_ip:
-                pg_dir_ips_string = str(ip)
-                single_ip = False
-            else:
-                pg_dir_ips_string = pg_dir_ips_string + ',' + str(ip)
-        pg_ctxt['director_ips_string'] = pg_dir_ips_string
-        pg_ctxt['virtual_ip'] = conf['plumgrid-virtual-ip']
-        pg_ctxt['pg_hostname'] = "pg-director"
-        from pg_dir_utils import check_interface_type
-        interface_type = check_interface_type()
-        pg_ctxt['interface'] = interface_type
-        pg_ctxt['label'] = get_unit_hostname()
+        dir_count = len(pg_dir_ips)
+        pg_ctxt['director_ips_string'] = (str(pg_dir_ips[0]) + ',' +
+                                          str(pg_dir_ips[1]) + ',' +
+                                          str(pg_dir_ips[2])
+                                          if dir_count == 3 else
+                                          str(pg_dir_ips[0]))
+        PG_VIP = conf['plumgrid-virtual-ip']
+        if is_ip(PG_VIP):
+            pg_ctxt['virtual_ip'] = PG_VIP
+        else:
+            raise ValueError('Invalid PLUMgrid Virtual IP Provided')
+        unit_hostname = gethostname()
+        pg_ctxt['pg_hostname'] = unit_hostname
+        pg_ctxt['pg_fqdn'] = getfqdn()
+        from pg_dir_utils import get_mgmt_interface, get_fabric_interface
+        pg_ctxt['interface'] = get_mgmt_interface()
+        pg_ctxt['fabric_interface'] = get_fabric_interface()
+        pg_ctxt['label'] = unit_hostname
         pg_ctxt['fabric_mode'] = 'host'
         virtual_ip_array = re.split('\.', conf['plumgrid-virtual-ip'])
         pg_ctxt['virtual_router_id'] = virtual_ip_array[3]
+        pg_ctxt['opsvm_ip'] = conf['opsvm-ip']
 
         return pg_ctxt

=== modified file 'hooks/pg_dir_hooks.py'
--- hooks/pg_dir_hooks.py	2015-08-24 16:18:48 +0000
+++ hooks/pg_dir_hooks.py	2016-04-08 08:30:24 +0000
@@ -7,22 +7,28 @@
 
 import sys
 import time
+from charmhelpers.core.host import service_running
+from charmhelpers.contrib.network.ip import is_ip
+
 from charmhelpers.core.hookenv import (
     Hooks,
     UnregisteredHookError,
     log,
     config,
+    relation_set,
+    relation_ids,
+    status_set
 )
 
 from charmhelpers.fetch import (
     apt_install,
-    apt_purge,
     configure_sources,
 )
 
 from pg_dir_utils import (
     register_configs,
     restart_pg,
+    restart_map,
     stop_pg,
     determine_packages,
     load_iovisor,
@@ -30,6 +36,10 @@
     ensure_mtu,
     add_lcm_key,
     post_pg_license,
+    fabric_interface_changed,
+    load_iptables,
+    restart_on_change,
+    director_cluster_ready
 )
 
 hooks = Hooks()
@@ -41,22 +51,39 @@
     '''
     Install hook is run when the charm is first deployed on a node.
     '''
+    status_set('maintenance', 'Executing pre-install')
+    load_iptables()
     configure_sources(update=True)
+    status_set('maintenance', 'Installing apt packages')
     pkgs = determine_packages()
     for pkg in pkgs:
         apt_install(pkg, options=['--force-yes'], fatal=True)
     load_iovisor()
     ensure_mtu()
-    add_lcm_key()
+    CONFIGS.write_all()
 
 
 @hooks.hook('director-relation-joined')
+@restart_on_change(restart_map())
 def dir_joined():
     '''
     This hook is run when a unit of director is added.
     '''
-    CONFIGS.write_all()
-    restart_pg()
+    if director_cluster_ready():
+        ensure_mtu()
+        CONFIGS.write_all()
+
+
[e-mail]('plumgrid-relation-joined')
+def plumgrid_joined(relation_id=None):
+    '''
+    This hook is run when relation with edge or gateway is created.
+    '''
+    opsvm_ip = config('opsvm-ip')
+    if not is_ip(opsvm_ip):
+        raise ValueError('Incorrect OPSVM IP specified')
+    else:
+        relation_set(relation_id=relation_id, opsvm_ip=opsvm_ip)
 
 
 @hooks.hook('config-changed')
@@ -65,22 +92,41 @@
     This hook is run when a config parameter is changed.
     It also runs on node reboot.
     '''
-    if post_pg_license():
-        log("PLUMgrid License Posted")
-        return 1
-    if add_lcm_key():
-        log("PLUMgrid LCM Key added")
-        return 1
-    stop_pg()
-    configure_sources(update=True)
-    pkgs = determine_packages()
-    for pkg in pkgs:
-        apt_install(pkg, options=['--force-yes'], fatal=True)
-    load_iovisor()
+    charm_config = config()
+    if charm_config.changed('lcm-ssh-key'):
+        if add_lcm_key():
+            log("PLUMgrid LCM Key added")
+    if charm_config.changed('plumgrid-license-key'):
+        if post_pg_license():
+            log("PLUMgrid License Posted")
+    if charm_config.changed('fabric-interfaces'):
+        if not fabric_interface_changed():
+            log("Fabric interface already set")
+        else:
+            stop_pg()
+    if charm_config.changed('plumgrid-virtual-ip'):
+        CONFIGS.write_all()
+        stop_pg()
+    if (charm_config.changed('install_sources') or
+        charm_config.changed('plumgrid-build') or
+        charm_config.changed('install_keys') or
+            charm_config.changed('iovisor-build')):
+        status_set('maintenance', 'Upgrading apt packages')
+        stop_pg()
+        configure_sources(update=True)
+        pkgs = determine_packages()
+        for pkg in pkgs:
+            apt_install(pkg, options=['--force-yes'], fatal=True)
+            remove_iovisor()
+            load_iovisor()
+    if charm_config.changed('opsvm-ip'):
+        for rid in relation_ids('plumgrid'):
+            plumgrid_joined(rid)
+        stop_pg()
     ensure_mtu()
-    add_lcm_key()
     CONFIGS.write_all()
-    restart_pg()
+    if not service_running('plumgrid'):
+        restart_pg()
 
 
 @hooks.hook('start')
@@ -93,20 +139,34 @@
         while (count < 10):
             if post_pg_license():
                 break
-            count = count + 1
+            count += 1
             time.sleep(15)
 
 
[e-mail]('upgrade-charm')
+@restart_on_change(restart_map())
+def upgrade_charm():
+    '''
+    This hook is run when the charm is upgraded
+    '''
+    ensure_mtu()
+    CONFIGS.write_all()
+
+
 @hooks.hook('stop')
 def stop():
     '''
     This hook is run when the charm is destroyed.
     '''
     stop_pg()
-    remove_iovisor()
-    pkgs = determine_packages()
-    for pkg in pkgs:
-        apt_purge(pkg, fatal=False)
+
+
[e-mail]('update-status')
+def update_status():
+    if service_running('plumgrid'):
+        status_set('active', 'Unit is ready')
+    else:
+        status_set('blocked', 'plumgrid service not running')
 
 
 def main():

=== modified file 'hooks/pg_dir_utils.py'
--- hooks/pg_dir_utils.py	2015-08-24 16:18:48 +0000
+++ hooks/pg_dir_utils.py	2016-04-08 08:30:24 +0000
@@ -2,45 +2,60 @@
 
 # This file contains functions used by the hooks to deploy PLUMgrid Director.
 
+import pg_dir_context
+import subprocess
+import time
+import os
+import json
+from collections import OrderedDict
+from socket import gethostname as get_unit_hostname
+from copy import deepcopy
 from charmhelpers.contrib.openstack.neutron import neutron_plugin_attribute
-from copy import deepcopy
+from charmhelpers.contrib.openstack import templating
+from charmhelpers.contrib.storage.linux.ceph import modprobe
 from charmhelpers.core.hookenv import (
     log,
     config,
+    unit_get,
+    status_set
 )
-from charmhelpers.contrib.openstack import templating
-from charmhelpers.core.host import set_nic_mtu
-from collections import OrderedDict
-from charmhelpers.contrib.storage.linux.ceph import modprobe
-from charmhelpers.contrib.openstack.utils import (
-    os_release,
+from charmhelpers.contrib.network.ip import (
+    get_iface_from_addr,
+    get_bridges,
+    get_bridge_nics,
+    is_ip,
+    is_address_in_network,
+    get_iface_addr
 )
 from charmhelpers.core.host import (
     service_start,
     service_stop,
-)
-import pg_dir_context
-import subprocess
-import time
-import os
-import re
-import json
+    service_running,
+    path_hash,
+    set_nic_mtu
+)
+from charmhelpers.fetch import (
+    apt_cache,
+    apt_install
+)
+from charmhelpers.contrib.openstack.utils import (
+    os_release,
+)
 
 LXC_CONF = '/etc/libvirt/lxc.conf'
 TEMPLATES = 'templates/'
 PG_LXC_DATA_PATH = '/var/lib/libvirt/filesystems/plumgrid-data'
 PG_LXC_PATH = '/var/lib/libvirt/filesystems/plumgrid'
-
 PG_CONF = '%s/conf/pg/plumgrid.conf' % PG_LXC_DATA_PATH
 PG_KA_CONF = '%s/conf/etc/keepalived.conf' % PG_LXC_DATA_PATH
 PG_DEF_CONF = '%s/conf/pg/nginx.conf' % PG_LXC_DATA_PATH
 PG_HN_CONF = '%s/conf/etc/hostname' % PG_LXC_DATA_PATH
 PG_HS_CONF = '%s/conf/etc/hosts' % PG_LXC_DATA_PATH
 PG_IFCS_CONF = '%s/conf/pg/ifcs.conf' % PG_LXC_DATA_PATH
+OPS_CONF = '%s/conf/etc/00-pg.conf' % PG_LXC_DATA_PATH
 AUTH_KEY_PATH = '%s/root/.ssh/authorized_keys' % PG_LXC_DATA_PATH
 TEMP_LICENSE_FILE = '/tmp/license'
 
-
 BASE_RESOURCE_MAP = OrderedDict([
     (PG_KA_CONF, {
         'services': ['plumgrid'],
@@ -62,6 +77,10 @@
         'services': ['plumgrid'],
         'contexts': [pg_dir_context.PGDirContext()],
     }),
+    (OPS_CONF, {
+        'services': ['plumgrid'],
+        'contexts': [pg_dir_context.PGDirContext()],
+    }),
     (PG_IFCS_CONF, {
         'services': [],
         'contexts': [pg_dir_context.PGDirContext()],
@@ -74,7 +93,25 @@
     Returns list of packages required by PLUMgrid director as specified
     in the neutron_plugins dictionary in charmhelpers.
     '''
-    return neutron_plugin_attribute('plumgrid', 'packages', 'neutron')
+    pkgs = []
+    tag = 'latest'
+    for pkg in neutron_plugin_attribute('plumgrid', 'packages', 'neutron'):
+        if 'plumgrid' in pkg:
+            tag = config('plumgrid-build')
+        elif pkg == 'iovisor-dkms':
+            tag = config('iovisor-build')
+
+        if tag == 'latest':
+            pkgs.append(pkg)
+        else:
+            if tag in [i.ver_str for i in apt_cache()[pkg].version_list]:
+                pkgs.append('%s=%s' % (pkg, tag))
+            else:
+                error_msg = \
+                    "Build version '%s' for package '%s' not available" \
+                    % (tag, pkg)
+                raise ValueError(error_msg)
+    return pkgs
 
 
 def register_configs(release=None):
@@ -111,11 +148,20 @@
     '''
     Stops and Starts PLUMgrid service after flushing iptables.
     '''
-    service_stop('plumgrid')
-    time.sleep(2)
-    _exec_cmd(cmd=['iptables', '-F'])
+    stop_pg()
     service_start('plumgrid')
-    time.sleep(5)
+    time.sleep(3)
+    if not service_running('plumgrid'):
+        if service_running('libvirt-bin'):
+            raise ValueError("plumgrid service couldn't be started")
+        else:
+            if service_start('libvirt-bin'):
+                time.sleep(3)
+                if not service_running('plumgrid'):
+                    raise ValueError("plumgrid service couldn't be started")
+            else:
+                raise ValueError("libvirt-bin service couldn't be started")
+    status_set('active', 'Unit is ready')
 
 
 def stop_pg():
@@ -138,39 +184,95 @@
     Removes iovisor kernel module.
     '''
     _exec_cmd(cmd=['rmmod', 'iovisor'],
-              error_msg='Error Loading IOVisor Kernel Module')
-
-
-def check_interface_type():
-    '''
-    Checks the interface. Support added for AWS deployments. There are 2
-    possible interfaces "juju-br0" and "eth0". The default being juju-br0
-    '''
-    log("Checking Interface Type")
-    default_interface = "juju-br0"
-    AWS_interface = "eth0"
-    shell_output = subprocess.check_output(['brctl', 'show', 'juju-br0'])
-    output = re.split(' |\n|\t', shell_output)
-    if output[10] == '':
-        return AWS_interface
-    else:
-        return default_interface
+              error_msg='Error Removing IOVisor Kernel Module')
+    time.sleep(1)
+
+
+def interface_exists(interface):
+    '''
+    Checks if interface exists on node.
+    '''
+    try:
+        subprocess.check_call(['ip', 'link', 'show', interface],
+                              stdout=open(os.devnull, 'w'),
+                              stderr=subprocess.STDOUT)
+    except subprocess.CalledProcessError:
+        return False
+    return True
+
+
+def get_mgmt_interface():
+    '''
+    Returns the managment interface.
+    '''
+    mgmt_interface = config('mgmt-interface')
+    if interface_exists(mgmt_interface):
+        return mgmt_interface
+    else:
+        log('Provided managment interface %s does not exist'
+            % mgmt_interface)
+        return get_iface_from_addr(unit_get('private-address'))
+
+
+def fabric_interface_changed():
+    '''
+    Returns true if interface for node changed.
+    '''
+    fabric_interface = get_fabric_interface()
+    try:
+        with open(PG_IFCS_CONF, 'r') as ifcs:
+            for line in ifcs:
+                if 'fabric_core' in line:
+                    if line.split()[0] == fabric_interface:
+                        return False
+    except IOError:
+        return True
+    return True
+
+
+def get_fabric_interface():
+    '''
+    Returns the fabric interface.
+    '''
+    fabric_interfaces = config('fabric-interfaces')
+    if fabric_interfaces == 'MANAGEMENT':
+        return get_mgmt_interface()
+    else:
+        try:
+            all_fabric_interfaces = json.loads(fabric_interfaces)
+        except ValueError:
+            raise ValueError('Invalid json provided for fabric interfaces')
+        hostname = get_unit_hostname()
+        if hostname in all_fabric_interfaces:
+            node_fabric_interface = all_fabric_interfaces[hostname]
+        elif 'DEFAULT' in all_fabric_interfaces:
+            node_fabric_interface = all_fabric_interfaces['DEFAULT']
+        else:
+            raise ValueError('No fabric interface provided for node')
+        if interface_exists(node_fabric_interface):
+            if is_address_in_network(config('os-data-network'),
+                                     get_iface_addr(node_fabric_interface)[0]):
+                return node_fabric_interface
+            else:
+                raise ValueError('Fabric interface not in fabric network')
+        else:
+            log('Provided fabric interface %s does not exist'
+                % node_fabric_interface)
+            raise ValueError('Provided fabric interface does not exist')
+        return node_fabric_interface
 
 
 def ensure_mtu():
     '''
     Ensures required MTU of the underlying networking of the node.
     '''
-    log("Changing MTU of juju-br0 and all attached interfaces")
     interface_mtu = config('network-device-mtu')
-    interface_type = check_interface_type()
-    if interface_type == "juju-br0":
-        cmd = subprocess.check_output(["brctl", "show", interface_type])
-        words = cmd.split()
-        for word in words:
-            if 'eth' in word:
-                set_nic_mtu(word, interface_mtu)
-    set_nic_mtu(interface_type, interface_mtu)
+    fabric_interface = get_fabric_interface()
+    if fabric_interface in get_bridges():
+        attached_interfaces = get_bridge_nics(fabric_interface)
+        for interface in attached_interfaces:
+            set_nic_mtu(interface, interface_mtu)
+    set_nic_mtu(fabric_interface, interface_mtu)
 
 
 def _exec_cmd(cmd=None, error_msg='Command exited with ERRORs', fatal=False):
@@ -229,6 +331,8 @@
         log('PLUMgrid License Key not specified')
         return 0
     PG_VIP = config('plumgrid-virtual-ip')
+    if not is_ip(PG_VIP):
+        raise ValueError('Invalid IP Provided')
     LICENSE_POST_PATH = 'https://%s/0/tenant_manager/license_key' % PG_VIP
     LICENSE_GET_PATH = 'https://%s/0/tenant_manager/licenses' % PG_VIP
     PG_CURL = '%s/opt/pg/scripts/pg_curl.sh' % PG_LXC_PATH
@@ -239,8 +343,7 @@
         'plumgrid:plumgrid',
         LICENSE_POST_PATH,
         '-d',
-        json.dumps(license)
-        ]
+        json.dumps(license)]
     licence_get_cmd = [PG_CURL, '-u', 'plumgrid:plumgrid', LICENSE_GET_PATH]
     try:
         old_license = subprocess.check_output(licence_get_cmd)
@@ -254,3 +357,76 @@
         log('No change in PLUMgrid License')
         return 0
     return 1
+
+
+def load_iptables():
+    '''
+    Loads iptables rules to allow all PLUMgrid communication.
+    '''
+    network = get_cidr_from_iface(get_mgmt_interface())
+    if network:
+        _exec_cmd(['sudo', 'iptables', '-A', 'INPUT', '-p', 'tcp',
+                   '-j', 'ACCEPT', '-s', network, '-d',
+                   network, '-m', 'state', '--state', 'NEW'])
+        _exec_cmd(['sudo', 'iptables', '-A', 'INPUT', '-p', 'udp', '-j',
+                   'ACCEPT', '-s', network, '-d', network,
+                   '-m', 'state', '--state', 'NEW'])
+        _exec_cmd(['sudo', 'iptables', '-I', 'INPUT', '-s', network,
+                   '-d', '224.0.0.18/32', '-j', 'ACCEPT'])
+    _exec_cmd(['sudo', 'iptables', '-I', 'INPUT', '-p', 'vrrp', '-j',
+               'ACCEPT'])
+    _exec_cmd(['sudo', 'iptables', '-A', 'INPUT', '-p', 'tcp', '-j',
+               'ACCEPT', '-d', config('plumgrid-virtual-ip'), '-m',
+               'state', '--state', 'NEW'])
+    apt_install('iptables-persistent')
+
+
+def get_cidr_from_iface(interface):
+    '''
+    Determines Network CIDR from interface.
+    '''
+    if not interface:
+        return None
+    apt_install('ohai')
+    try:
+        os_info = subprocess.check_output(['ohai', '-l', 'fatal'])
+    except OSError:
+        log('Unable to get operating system information')
+        return None
+    try:
+        os_info_json = json.loads(os_info)
+    except ValueError:
+        log('Unable to determine network')
+        return None
+    device = os_info_json['network']['interfaces'].get(interface)
+    if device is not None:
+        if device.get('routes'):
+            routes = device['routes']
+            for net in routes:
+                if 'scope' in net:
+                    return net.get('destination')
+        else:
+            return None
+    else:
+        return None
+
+
+def director_cluster_ready():
+    dirs_count = len(pg_dir_context._pg_dir_ips())
+    return True if dirs_count == 2 else False
+
+
+def restart_on_change(restart_map):
+    """
+    Restart services based on configuration files changing
+    """
+    def wrap(f):
+        def wrapped_f(*args, **kwargs):
+            checksums = {path: path_hash(path) for path in restart_map}
+            f(*args, **kwargs)
+            for path in restart_map:
+                if path_hash(path) != checksums[path]:
+                    restart_pg()
+                    break
+        return wrapped_f
+    return wrap

=== added symlink 'hooks/plumgrid-relation-joined'
=== target is u'pg_dir_hooks.py'
=== added symlink 'hooks/update-status'
=== target is u'pg_dir_hooks.py'
=== added symlink 'hooks/upgrade-charm'
=== target is u'pg_dir_hooks.py'
=== added file 'templates/kilo/00-pg.conf'
--- templates/kilo/00-pg.conf	1970-01-01 00:00:00 +0000
+++ templates/kilo/00-pg.conf	2016-04-08 08:30:24 +0000
@@ -0,0 +1,2 @@
+$template ls_json,"{{'{'}}{{'%'}}timestamp:::date-rfc3339,jsonf:@timestamp%,%source:::jsonf:@source_host%,%msg:::json%}"
+:syslogtag,isequal,"pg:" @{{ opsvm_ip }}:6000;ls_json

=== modified file 'templates/kilo/hosts'
--- templates/kilo/hosts	2015-07-29 18:11:14 +0000
+++ templates/kilo/hosts	2016-04-08 08:30:24 +0000
@@ -1,5 +1,5 @@
 127.0.0.1   localhost
-127.0.1.1   {{ pg_hostname }}
+127.0.1.1   {{ pg_fqdn }}   {{ pg_hostname }}
 
 # The following lines are desirable for IPv6 capable hosts
 ::1     ip6-localhost ip6-loopback

=== modified file 'templates/kilo/ifcs.conf'
--- templates/kilo/ifcs.conf	2015-07-29 18:11:14 +0000
+++ templates/kilo/ifcs.conf	2016-04-08 08:30:24 +0000
@@ -1,2 +1,2 @@
-{{ interface }} = fabric_core host
+{{ fabric_interface }} = fabric_core host
 

=== modified file 'templates/kilo/nginx.conf'
--- templates/kilo/nginx.conf	2015-08-10 10:06:33 +0000
+++ templates/kilo/nginx.conf	2016-04-08 08:30:24 +0000
@@ -12,6 +12,10 @@
   server {{ virtual_ip }}:3000;
 }
 
+upstream pgMW {
+  server {{ opsvm_ip }}:4000;
+}
+
 map $http_upgrade $connection_upgrade {
   default upgrade;
   '' close;
@@ -58,6 +62,19 @@
     proxy_set_header Host $host;
   }
 
+  location /mwv0 {
+    proxy_pass http://pgMW;
+    proxy_redirect off;
+    proxy_http_version 1.1;
+    proxy_set_header Upgrade $http_upgrade;
+    proxy_set_header Connection "upgrade";
+    proxy_set_header Host $host;
+  }
+
+  location /cloudApex/ {
+    index index.html;
+  }
+
   location /vtap/ {
     alias /opt/pg/vtap;
   }

=== added file 'tests/files/plumgrid-director-dense.yaml'
--- tests/files/plumgrid-director-dense.yaml	1970-01-01 00:00:00 +0000
+++ tests/files/plumgrid-director-dense.yaml	2016-04-08 08:30:24 +0000
@@ -0,0 +1,133 @@
+test:
+  series: 'trusty'
+  relations:
+  - - mysql
+    - keystone
+  - - nova-cloud-controller
+    - mysql
+  - - nova-cloud-controller
+    - rabbitmq-server
+  - - nova-cloud-controller
+    - glance
+  - - nova-cloud-controller
+    - keystone
+  - - nova-compute
+    - nova-cloud-controller
+  - - nova-compute
+    - mysql
+  - - nova-compute
+    - rabbitmq-server
+  - - nova-compute
+    - glance
+  - - glance
+    - mysql
+  - - glance
+    - keystone
+  - - glance
+    - cinder
+  - - mysql
+    - cinder
+  - - cinder
+    - rabbitmq-server
+  - - cinder
+    - nova-cloud-controller
+  - - cinder
+    - keystone
+  - - openstack-dashboard
+    - keystone
+  - - neutron-api
+    - mysql
+  - - neutron-api
+    - keystone
+  - - neutron-api
+    - rabbitmq-server
+  - - neutron-api
+    - nova-cloud-controller
+  - - neutron-api
+    - neutron-api-plumgrid
+  - - neutron-api-plumgrid
+    - plumgrid-edge
+  - - plumgrid-director
+    - plumgrid-edge
+  - - nova-compute
+    - plumgrid-edge
+  - - plumgrid-director
+    - plumgrid-gateway
+  services:
+    mysql:
+      charm: cs:trusty/mysql
+      num_units: 1
+      to: 'lxc:plumgrid-director=0'
+    rabbitmq-server:
+      charm: cs:trusty/rabbitmq-server
+      num_units: 1
+      to: 'lxc:plumgrid-director=0'
+    keystone:
+      charm: cs:trusty/keystone
+      num_units: 1
+      options:
+        admin-password: plumgrid
+        openstack-origin: cloud:trusty-kilo
+      to: 'lxc:plumgrid-director=0'
+    nova-cloud-controller:
+      charm: cs:trusty/nova-cloud-controller
+      num_units: 1
+      options:
+        console-access-protocol: novnc
+        network-manager: Neutron
+        openstack-origin: cloud:trusty-kilo
+        quantum-security-groups: 'yes'
+      to: 'lxc:plumgrid-director=0'
+    glance:
+      charm: cs:trusty/glance
+      num_units: 1
+      options:
+        openstack-origin: cloud:trusty-kilo
+      to: 'lxc:plumgrid-director=0'
+    openstack-dashboard:
+      charm: cs:trusty/openstack-dashboard
+      num_units: 1
+      options:
+        openstack-origin: cloud:trusty-kilo
+      to: 'lxc:plumgrid-director=0'
+    cinder:
+      charm: cs:trusty/cinder
+      num_units: 1
+      options:
+        openstack-origin: cloud:trusty-kilo
+      to: 'lxc:plumgrid-director=0'
+    neutron-api:
+      charm: cs:~plumgrid-team/trusty/neutron-api
+      num_units: 1
+      options:
+        neutron-plugin: plumgrid
+        neutron-security-groups: false
+        openstack-origin: cloud:trusty-kilo
+        plumgrid-password: plumgrid
+        plumgrid-username: plumgrid
+        plumgrid-virtual-ip: 192.168.100.250
+      to: 'lxc:plumgrid-director=0'
+    neutron-api-plumgrid:
+      charm: cs:~plumgrid-team/trusty/neutron-api-plumgrid
+      options:
+        enable-metadata: True
+    plumgrid-director:
+      charm: cs:~plumgrid-team/trusty/plumgrid-director
+      num_units: 1
+      constraints: "root-disk=30G mem=8G cpu-cores=8"
+      options:
+        plumgrid-virtual-ip: 192.168.100.250
+    nova-compute:
+      charm: cs:~plumgrid-team/trusty/nova-compute
+      num_units: 1
+      options:
+        enable-live-migration: true
+        enable-resize: true
+        migration-auth-type: ssh
+        openstack-origin: cloud:trusty-kilo
+      to: '0'
+    plumgrid-edge:
+      charm: cs:~plumgrid-team/trusty/plumgrid-edge
+    plumgrid-gateway:
+      charm: cs:~plumgrid-team/trusty/plumgrid-gateway
+      num_units: 1

=== removed file 'tests/files/plumgrid-director-dense.yaml'
--- tests/files/plumgrid-director-dense.yaml	2015-09-01 17:31:26 +0000
+++ tests/files/plumgrid-director-dense.yaml	1970-01-01 00:00:00 +0000
@@ -1,133 +0,0 @@
-test:
-  series: 'trusty'
-  relations:
-  - - mysql
-    - keystone
-  - - nova-cloud-controller
-    - mysql
-  - - nova-cloud-controller
-    - rabbitmq-server
-  - - nova-cloud-controller
-    - glance
-  - - nova-cloud-controller
-    - keystone
-  - - nova-compute
-    - nova-cloud-controller
-  - - nova-compute
-    - mysql
-  - - nova-compute
-    - rabbitmq-server
-  - - nova-compute
-    - glance
-  - - glance
-    - mysql
-  - - glance
-    - keystone
-  - - glance
-    - cinder
-  - - mysql
-    - cinder
-  - - cinder
-    - rabbitmq-server
-  - - cinder
-    - nova-cloud-controller
-  - - cinder
-    - keystone
-  - - openstack-dashboard
-    - keystone
-  - - neutron-api
-    - mysql
-  - - neutron-api
-    - keystone
-  - - neutron-api
-    - rabbitmq-server
-  - - neutron-api
-    - nova-cloud-controller
-  - - neutron-api
-    - neutron-api-plumgrid
-  - - neutron-api-plumgrid
-    - plumgrid-edge
-  - - plumgrid-director
-    - plumgrid-edge
-  - - nova-compute
-    - plumgrid-edge
-  - - plumgrid-director
-    - plumgrid-gateway
-  services:
-    mysql:
-      charm: cs:trusty/mysql
-      num_units: 1
-      to: 'lxc:plumgrid-director=0'
-    rabbitmq-server:
-      charm: cs:trusty/rabbitmq-server
-      num_units: 1
-      to: 'lxc:plumgrid-director=0'
-    keystone:
-      charm: cs:trusty/keystone
-      num_units: 1
-      options:
-        admin-password: plumgrid
-        openstack-origin: cloud:trusty-kilo
-      to: 'lxc:plumgrid-director=0'
-    nova-cloud-controller:
-      charm: cs:trusty/nova-cloud-controller
-      num_units: 1
-      options:
-        console-access-protocol: novnc
-        network-manager: Neutron
-        openstack-origin: cloud:trusty-kilo
-        quantum-security-groups: 'yes'
-      to: 'lxc:plumgrid-director=0'
-    glance:
-      charm: cs:trusty/glance
-      num_units: 1
-      options:
-        openstack-origin: cloud:trusty-kilo
-      to: 'lxc:plumgrid-director=0'
-    openstack-dashboard:
-      charm: cs:trusty/openstack-dashboard
-      num_units: 1
-      options:
-        openstack-origin: cloud:trusty-kilo
-      to: 'lxc:plumgrid-director=0'
-    cinder:
-      charm: cs:trusty/cinder
-      num_units: 1
-      options:
-        openstack-origin: cloud:trusty-kilo
-      to: 'lxc:plumgrid-director=0'
-    neutron-api:
-      charm: cs:~plumgrid-team/trusty/neutron-api
-      num_units: 1
-      options:
-        neutron-plugin: plumgrid
-        neutron-security-groups: false
-        openstack-origin: cloud:trusty-kilo
-        plumgrid-password: plumgrid
-        plumgrid-username: plumgrid
-        plumgrid-virtual-ip: 192.168.100.250
-      to: 'lxc:plumgrid-director=0'
-    neutron-api-plumgrid:
-      charm: cs:~plumgrid-team/trusty/neutron-api-plumgrid
-      options:
-        enable-metadata: True
-    plumgrid-director:
-      charm: cs:~plumgrid-team/trusty/plumgrid-director
-      num_units: 1
-      constraints: "root-disk=30G mem=8G cpu-cores=8"
-      options:
-        plumgrid-virtual-ip: 192.168.100.250
-    nova-compute:
-      charm: cs:~plumgrid-team/trusty/nova-compute
-      num_units: 1
-      options:
-        enable-live-migration: true
-        enable-resize: true
-        migration-auth-type: ssh
-        openstack-origin: cloud:trusty-kilo
-      to: '0'
-    plumgrid-edge:
-      charm: cs:~plumgrid-team/trusty/plumgrid-edge
-    plumgrid-gateway:
-      charm: cs:~plumgrid-team/trusty/plumgrid-gateway
-      num_units: 1

=== added file 'tests/test.yaml'
--- tests/test.yaml	1970-01-01 00:00:00 +0000
+++ tests/test.yaml	2016-04-08 08:30:24 +0000
@@ -0,0 +1,2 @@
+makefile:
+  - lint

=== removed file 'tests/test.yaml'
--- tests/test.yaml	2015-09-01 17:31:26 +0000
+++ tests/test.yaml	1970-01-01 00:00:00 +0000
@@ -1,2 +0,0 @@
-makefile:
-  - lint

=== modified file 'unit_tests/test_pg_dir_context.py'
--- unit_tests/test_pg_dir_context.py	2015-08-24 16:18:48 +0000
+++ unit_tests/test_pg_dir_context.py	2016-04-08 08:30:24 +0000
@@ -8,7 +8,8 @@
     'config',
     'unit_get',
     'get_host_ip',
-    'get_unit_hostname',
+    'gethostname',
+    'getfqdn'
 ]
 
 
@@ -45,19 +46,21 @@
                   'neutron_plugin_attribute')
     @patch.object(charmhelpers.contrib.openstack.context, 'unit_private_ip')
     @patch.object(context, '_pg_dir_ips')
-    @patch.object(utils, 'check_interface_type')
-    def test_neutroncc_context_api_rel(self, _int_type, _pg_dir_ips,
-                                       _unit_priv_ip, _npa, _ens_pkgs,
-                                       _save_ff, _https, _is_clus,
-                                       _unit_get, _config, _runits, _rids,
-                                       _rget):
+    @patch.object(utils, 'get_mgmt_interface')
+    @patch.object(utils, 'get_fabric_interface')
+    def test_neutroncc_context_api_rel(self, _fabric_int, _mgmt_int,
+                                       _pg_dir_ips, _unit_priv_ip, _npa,
+                                       _ens_pkgs, _save_ff, _https,
+                                       _is_clus, _unit_get, _config,
+                                       _runits, _rids, _rget):
         def mock_npa(plugin, section, manager):
             if section == "driver":
                 return "neutron.randomdriver"
             if section == "config":
                 return "neutron.randomconfig"
 
-        config = {'plumgrid-virtual-ip': "192.168.100.250"}
+        config = {'plumgrid-virtual-ip': "192.168.100.250",
+                  'opsvm-ip': '127.0.0.1'}
 
         def mock_config(key=None):
             if key:
@@ -70,10 +73,12 @@
         _npa.side_effect = mock_npa
         _unit_get.return_value = '192.168.100.201'
         _unit_priv_ip.return_value = '192.168.100.201'
-        self.get_unit_hostname.return_value = 'node0'
+        self.gethostname.return_value = 'node0'
+        self.getfqdn.return_value = 'node0.maas'
         self.get_host_ip.return_value = '192.168.100.201'
         _pg_dir_ips.return_value = ['192.168.100.202', '192.168.100.203']
-        _int_type.return_value = 'juju-br0'
+        _mgmt_int.return_value = 'juju-br0'
+        _fabric_int.return_value = 'juju-br0'
         napi_ctxt = context.PGDirContext()
         expect = {
             'config': 'neutron.randomconfig',
@@ -84,14 +89,17 @@
             'neutron_security_groups': None,
             'neutron_url': 'https://None:9696',
             'virtual_ip': '192.168.100.250',
-            'pg_hostname': 'pg-director',
+            'pg_hostname': 'node0',
+            'pg_fqdn': 'node0.maas',
             'interface': 'juju-br0',
+            'fabric_interface': 'juju-br0',
             'label': 'node0',
             'fabric_mode': 'host',
             'virtual_router_id': '250',
-            'director_ips': ['192.168.100.202', '192.168.100.203',
-                             '192.168.100.201'],
+            'director_ips': ['192.168.100.201', '192.168.100.202',
+                             '192.168.100.203'],
             'director_ips_string':
-            '192.168.100.202,192.168.100.203,192.168.100.201',
+            '192.168.100.201,192.168.100.202,192.168.100.203',
+            'opsvm_ip': '127.0.0.1',
         }
         self.assertEquals(expect, napi_ctxt())

=== modified file 'unit_tests/test_pg_dir_hooks.py'
--- unit_tests/test_pg_dir_hooks.py	2015-08-24 16:18:48 +0000
+++ unit_tests/test_pg_dir_hooks.py	2016-04-08 08:30:24 +0000
@@ -1,5 +1,7 @@
 from mock import MagicMock, patch, call
+
 from test_utils import CharmTestCase
+
 with patch('charmhelpers.core.hookenv.config') as config:
     config.return_value = 'neutron'
     import pg_dir_utils as utils
@@ -18,7 +20,6 @@
 TO_PATCH = [
     'remove_iovisor',
     'apt_install',
-    'apt_purge',
     'CONFIGS',
     'log',
     'configure_sources',
@@ -29,7 +30,8 @@
     'add_lcm_key',
     'determine_packages',
     'post_pg_license',
-    'config'
+    'config',
+    'load_iptables'
 ]
 NEUTRON_CONF_DIR = "/etc/neutron"
 
@@ -58,33 +60,11 @@
         ])
         self.load_iovisor.assert_called_with()
         self.ensure_mtu.assert_called_with()
-        self.add_lcm_key.assert_called_with()
-
-    def test_config_changed_hook(self):
-        _pkgs = ['plumgrid-lxc', 'iovisor-dkms']
-        self.add_lcm_key.return_value = 0
-        self.post_pg_license.return_value = 0
-        self.determine_packages.return_value = [_pkgs]
-        self._call_hook('config-changed')
-        self.stop_pg.assert_called_with()
-        self.configure_sources.assert_called_with(update=True)
-        self.apt_install.assert_has_calls([
-            call(_pkgs, fatal=True,
-                 options=['--force-yes']),
-        ])
-        self.load_iovisor.assert_called_with()
-        self.ensure_mtu.assert_called_with()
-
-        self.CONFIGS.write_all.assert_called_with()
-        self.restart_pg.assert_called_with()
 
     def test_start(self):
         self._call_hook('start')
         self.test_config.set('plumgrid-license-key', None)
 
     def test_stop(self):
-        _pkgs = ['plumgrid-lxc', 'iovisor-dkms']
         self._call_hook('stop')
         self.stop_pg.assert_called_with()
-        self.remove_iovisor.assert_called_with()
-        self.determine_packages.return_value = _pkgs

=== modified file 'unit_tests/test_pg_dir_utils.py'
--- unit_tests/test_pg_dir_utils.py	2015-08-24 16:18:48 +0000
+++ unit_tests/test_pg_dir_utils.py	2016-04-08 08:30:24 +0000
@@ -55,7 +55,8 @@
                  nutils.PG_DEF_CONF,
                  nutils.PG_HN_CONF,
                  nutils.PG_HS_CONF,
-                 nutils.PG_IFCS_CONF]
+                 nutils.PG_IFCS_CONF,
+                 nutils.OPS_CONF]
         self.assertItemsEqual(_regconfs.configs, confs)
 
     def test_resource_map(self):
@@ -73,6 +74,7 @@
             (nutils.PG_DEF_CONF, ['plumgrid']),
             (nutils.PG_HN_CONF, ['plumgrid']),
             (nutils.PG_HS_CONF, ['plumgrid']),
+            (nutils.OPS_CONF, ['plumgrid']),
             (nutils.PG_IFCS_CONF, []),
         ])
         self.assertEqual(expect, _restart_map)