Merge lp:~jamesbeedy/charms/trusty/nova-compute/add-iface-support into lp:~openstack-charmers-archive/charms/trusty/nova-compute/next

[james beedy]

Adds ability to configure user defined network interfaces.

Added template for network interface config file in templates called ethN.
Added gen_interfaces() func to nova_compute_utils.py, called from
install hook.
Added import of yaml package to nova_compute_utils.py to facilitate
the parsing of string -> dict in gen_interfaces.
Added 'network-interfaces' and 'up-network-interfaces' parameters to
config.yaml.
Modified README.md to reflect docs on new params.

Partially fixes bug #1505064

[lp:~jamesbeedy/charms/trusty/nova-compute/add-iface-support]

176. By james beedy on 2015-10-12

Fix typo.

177. By james beedy on 2015-10-12

Minor fixes

Removed unneeded double quotes in config.yaml.
Fixed block level indentation in README.md.

178. By james beedy on 2015-10-12

Corrected indentation.

Removed extra space from beginning of line.

179. By james beedy on 2015-10-12

Fix variable name.

Modified 'replace_me' to reflect correct name 'replace_iface_stanza'.

180. By james beedy on 2015-10-12

Fixed typo in variable name.

181. By james beedy on 2015-10-12

Refactored variable names, fixed spacing, refactored interface stanza
logic to handle the only possible cases.

Removed unneeded configs options from network-interfaces
possible parameters.

182. By james beedy on 2015-10-12

Modified templates/ethN to reflect available params.

Removed interface-gateway, interface-nameservers, and
interface-dns-search from available params.

183. By james beedy on 2015-10-12

Modified comment to be more explicit in gen_interfaces.

184. By james beedy on 2015-10-12

Fix typo

185. By james beedy on 2015-10-12

Fix line length.

186. By james beedy on 2015-10-12

Repositioned iface_source_stanza_all outside of for loop.

187. By james beedy on 2015-10-12

Clearify gen_interfaces() description comment.

[uosci-testing-bot (uosci-testing-bot)]

charm_lint_check #11712 nova-compute-next for jamesbeedy mp274104
    LINT FAIL: lint-test failed

LINT Results (max last 2 lines):
make: *** [lint] Error 1
ERROR:root:Make target returned non-zero.

Full lint test output: http://paste.ubuntu.com/12763471/
Build: http://10.245.162.77:8080/job/charm_lint_check/11712/

[uosci-testing-bot (uosci-testing-bot)]

charm_unit_test #10895 nova-compute-next for jamesbeedy mp274104
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/10895/

[lp:~jamesbeedy/charms/trusty/nova-compute/add-iface-support]

188. By james beedy on 2015-10-12

Fix python negation syntax

189. By james beedy on 2015-10-12

Insert whitespace after ','

190. By james beedy on 2015-10-12

Removed trailing blank line in ethN

191. By james beedy on 2015-10-12

Removed trailing whitespace

[uosci-testing-bot (uosci-testing-bot)]

charm_amulet_test #7279 nova-compute-next for jamesbeedy mp274104
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/12764369/
Build: http://10.245.162.77:8080/job/charm_amulet_test/7279/

Unmerged revisions

191. By james beedy on 2015-10-12

Removed trailing whitespace

190. By james beedy on 2015-10-12

Removed trailing blank line in ethN

189. By james beedy on 2015-10-12

Insert whitespace after ','

188. By james beedy on 2015-10-12

Fix python negation syntax

187. By james beedy on 2015-10-12

Clearify gen_interfaces() description comment.

186. By james beedy on 2015-10-12

Repositioned iface_source_stanza_all outside of for loop.

185. By james beedy on 2015-10-12

Fix line length.

184. By james beedy on 2015-10-12

Fix typo

183. By james beedy on 2015-10-12

Modified comment to be more explicit in gen_interfaces.

182. By james beedy on 2015-10-12

Modified templates/ethN to reflect available params.

Removed interface-gateway, interface-nameservers, and
interface-dns-search from available params.

=== modified file 'README.md'
--- README.md	2015-04-30 12:09:23 +0000
+++ README.md	2015-10-12 08:18:56 +0000
@@ -119,3 +119,33 @@
         - {name: nova,
            repository: 'git://github.com/openstack/nova',
            branch: master}
+
+Network Interfaces
+=====================
+
+The minimum network-interfaces parameters needed to configure
+additional interfaces are:
+
+    network-interfaces: include-file://network-interfaces.yaml
+
+    network-interfaces.yaml
+        {interfaces:[
+          {interface-label: ethN,
+           interface-mode: dhcp}]}
+
+An example of configuring multiple interfaces might look like:
+
+    network-interfaces: include-file://network-interfaces.yaml
+
+    network-interfaces.yaml
+        {interfaces:[
+          {interface-label: eth1,
+           interface-mode: static,
+           interface-address: 192.168.1.10,
+           interface-netmask: 255.255.255.0},
+          {interface-label: eth2,
+	   interface-mode: dhcp},
+          {interface-label: eth3,
+           interface-mode: manual},
+          {interface-label: br-vlan,
+           interface-mode: manual}]}
 
=== modified file 'config.yaml'
--- config.yaml	2015-10-09 15:02:40 +0000
+++ config.yaml	2015-10-12 08:18:56 +0000
@@ -291,3 +291,18 @@
       wait for you to execute the openstack-upgrade action for this charm on
       each unit. If False it will revert to existing behavior of upgrading
       all units on config change.
+  network-interfaces:
+    type: string
+    default:
+    description: |
+      String list of network interface dicts comprised of at least an
+      'interface-label', and 'interface-mode'. For interfaces specifying
+      "inerface-mode: static", a minimum of 'interface-address' and 
+      'interface-netmask' must also be supplied. Additional optional
+      parameters for 'static' interfaces include:
+          * interface-broadcast
+      For more details see README.md.
+  up-network-interfaces:
+    type: boolean
+    default: False
+    description: Bring up user defined network interfaces.
 
=== modified file 'hooks/nova_compute_hooks.py'
--- hooks/nova_compute_hooks.py	2015-10-09 15:02:40 +0000
+++ hooks/nova_compute_hooks.py	2015-10-12 08:18:56 +0000
@@ -70,6 +70,7 @@
     install_hugepages,
     REQUIRED_INTERFACES,
     check_optional_relations,
+    gen_interfaces,
 )
 
 from charmhelpers.contrib.network.ip import (
@@ -103,6 +104,9 @@
 
     status_set('maintenance', 'Git install')
     git_install(config('openstack-origin-git'))
+    if config('network-interfaces'):
+        status_set('maintenance', 'Configuring network interfaces')
+        gen_interfaces(config('network-interfaces'))
 
 
 @hooks.hook('config-changed')
 
=== modified file 'hooks/nova_compute_utils.py'
--- hooks/nova_compute_utils.py	2015-10-06 16:58:22 +0000
+++ hooks/nova_compute_utils.py	2015-10-12 08:18:56 +0000
@@ -2,6 +2,7 @@
 import shutil
 import pwd
 import subprocess
+import yaml
 
 from base64 import b64decode
 from copy import deepcopy
@@ -235,6 +236,8 @@
     },
 }
 
+DEFAULT_NETWORK_IFACE_CONF = '/etc/network/interfaces'
+NETWORK_IFACE_CONF_DIR = '/etc/network/interfaces.d'
 
 # Maps virt-type config to a compute package(s).
 VIRT_TYPES = {
@@ -851,3 +854,52 @@
         return status_get()
     else:
         return 'unknown', 'No optional relations'
+
+
+def gen_interfaces(interfaces):
+    """Generate interface files and replace stale stanzas in
+    /etc/network/interfaces with sourcing of new ethN.cfg file.
+    """
+    ifaces = yaml.load(interfaces)['interfaces']
+    iface_source_stanza_all = 'source /etc/network/interfaces.d/*.cfg'
+    for iface in ifaces:
+        net_iface_context = {}
+        iface_conf = os.path.join(NETWORK_IFACE_CONF_DIR,
+                                  ('%s.cfg' % iface['interface-label']))
+        iface_source_stanza_new = 'source %s' % iface_conf
+        iface_dhcp_stanza_stale = ('iface %s inet dhcp' % 
+                                   iface['interface-label'])
+        
+        # Dynamically generate context depending on interface mode
+        if iface['interface-mode'] in ['dhcp', 'manual']:
+            net_iface_context = {
+                'interface_label': iface['interface-label'],
+                'interface_mode': iface['interface-mode'],
+            }
+        else:
+            for key in iface.keys():
+                net_iface_context[key.replace('-','_')] = iface[key]
+        
+        with open(DEFAULT_NETWORK_IFACE_CONF, 'r') as iface_file:
+            filedata = iface_file.read()
+
+        # Check for existing stanzas matching the current interface.
+        # There are two possibilities that could exist for stanzas of 
+        # a non-primary interface of a juju deployed node depending on
+        # the provider used. This check ensures affinity for each 
+        # stanza <-> interface, so as we don't end up with multiple
+        # definitions.
+        if iface_dhcp_stanza_stale in filedata:
+            filedata = filedata.replace(iface_dhcp_stanza_stale, 
+                                        iface_source_stanza_new)
+            with open(DEFAULT_NETWORK_IFACE_CONF, 'w') as iface_file:
+                iface_file.write(filedata)        
+        elif not iface_source_stanza_all in filedata:
+            with open(DEFAULT_NETWORK_IFACE_CONF, 'a') as iface_file:
+                iface_file.write(iface_source_stanza_new)
+        
+        # Render iface config file, bring up network interface
+        # if up-network-interfaces == True
+        render('ethN', iface_conf, net_iface_context, perms=0o644)
+        if config('up-network-interfaces'):
+            check_call(['ifconfig', iface['interface-label'], 'up'])
 
=== added file 'templates/ethN'
--- templates/ethN	1970-01-01 00:00:00 +0000
+++ templates/ethN	2015-10-12 08:18:56 +0000
@@ -0,0 +1,13 @@
+auto {{ interface_label }}
+iface {{ interface_label }} inet {{ interface_mode }}
+{% if interface_mode == 'static' -%}
+address {{ interface_address }}
+netmask {{ interface_netmask }}
+{% if interface_broadcast -%}
+broadcast {{ interface_broadcast }}
+{% endif -%}
+{% endif -%}
+up ip l set {{ interface_label }} up
+down ip l set {{ interface_label }} down
+post-up  ethtool  -K  {{ interface_label }}  gso off  gro off || true
+