Merge ~raharper/cloud-init:netconfig-v2-passthrough into cloud-init:master

Proposed by Ryan Harper
Status: Rejected
Rejected by: Scott Moser
Proposed branch: ~raharper/cloud-init:netconfig-v2-passthrough
Merge into: cloud-init:master
Diff against target: 1917 lines (+1471/-41)
15 files modified
cloudinit/distros/__init__.py (+1/-1)
cloudinit/distros/debian.py (+11/-7)
cloudinit/distros/rhel.py (+1/-0)
cloudinit/net/__init__.py (+5/-0)
cloudinit/net/eni.py (+3/-12)
cloudinit/net/netplan.py (+383/-0)
cloudinit/net/network_state.py (+307/-11)
cloudinit/net/renderer.py (+1/-0)
cloudinit/net/renderers.py (+3/-1)
cloudinit/net/sysconfig.py (+19/-5)
systemd/cloud-init.service (+1/-0)
systemd/systemd-networkd-wait-online.path (+5/-0)
tests/unittests/test_distros/test_netconfig.py (+347/-4)
tests/unittests/test_net.py (+301/-0)
tools/net-convert.py (+83/-0)
Reviewer Review Type Date Requested Status
Server Team CI bot continuous-integration Needs Fixing
Scott Moser Needs Fixing
Review via email: mp+319259@code.launchpad.net

Description of the change

cloudinit.net: add v2 parsing, and v2 rendering

Network configuration version2 format is implemented in a package
called netplan (nplan)[1] which allows consolidated network config
for multiple network controllers.

- Detect and determine if Ubuntu Distro object will render eni or netplan
- Allow v2 configs to be passed directly to netplan
- Allow any network state (parsed from any format cloud-init supports) to
  render to v2 if system supports netplan.
- Move eni's _subnet_is_ipv6 to common code for use by other renderers
- Fix to base distro class for looking up path to
  system_info/network/renderers
- Make sysconfig renderer always emit /etc/syconfig/network configuration

1. https://lists.ubuntu.com/archives/ubuntu-devel/2016-July/039464.html

To post a comment you must log in.
Revision history for this message
Scott Moser (smoser) wrote :

comments inline.

Revision history for this message
Ryan Harper (raharper) wrote :

Thanks for the review

Revision history for this message
Scott Moser (smoser) wrote :

some small things.
i've not yet looked at the tests

review: Needs Fixing
Revision history for this message
Ryan Harper (raharper) wrote :

Thanks for the review. I plan to rebase to trunk and pickup the renderer priority branch you have as well as a fix for https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1669860 and I'll drop the related partial-fix in this branch.

Ideally the renderer priority branch should include some unittest to handle default policy selection, renderer availability, and setting policy; as well as broken config (provide a policy with nothing available, we should still have a valid renderer).

Revision history for this message
Server Team CI bot (server-team-bot) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
Ryan Harper (raharper) wrote :

I lacked the git-fu to force a proper rebase against itself, so I've pushed a new branch for review:

https://code.launchpad.net/~raharper/cloud-init/+git/cloud-init/+merge/320291

Revision history for this message
Scott Moser (smoser) wrote :

Marked this rejected.
It was merged at link above.

Unmerged commits

4cf22b3... by Ryan Harper

test_net: Remove duplicate unittests left from merge

d5a6353... by Ryan Harper

Merge branch 'merge-netconfig-v2-pass' into netconfig-v2-passthrough

Merge in a rebase of 'netconfig-v2-pass onto master' into netconfig-v2-pass

773df13... by Ryan Harper

test_netconfig: Fix config path for setting network renderer priorities, fix mock for available checks

ad519eb... by Ryan Harper

fix cfg path for network renderer priorty

2ac9242... by Ryan Harper

Mock out an ensure_dir which attempted to create dir

e66c337... by Ryan Harper

net: add renderers for automatically selecting the renderer.

previously, the distro had hard coded which network renderer it would
use. This adds support for just picking the right renderer based
on what is available.

Now, that can be set via a 'priority' in system_info, but should
generally work. That config looks like:
 system_info:
   network:
     renderers: ["eni", "sysconfig"]

8a53515... by Ryan Harper

Integrate use of network renderers into distros and update unittests

022cadc... by Ryan Harper

unittests for netplan, including postcommands, style fixes

08c41d6... by Ryan Harper

Add v2 passthrough test to distro netconfig

e778ea5... by Ryan Harper

Test eni and netplan rendering from distro; fix issue with dhcp4 attr in netplan

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1diff --git a/cloudinit/distros/__init__.py b/cloudinit/distros/__init__.py
2index 803ac74..22ae998 100755
3--- a/cloudinit/distros/__init__.py
4+++ b/cloudinit/distros/__init__.py
5@@ -73,7 +73,7 @@ class Distro(object):
6
7 def _supported_write_network_config(self, network_config):
8 priority = util.get_cfg_by_path(
9- self._cfg, ('network', 'renderers'), None)
10+ self._cfg, ('system_info', 'network', 'renderers'), None)
11
12 name, render_cls = renderers.select(priority=priority)
13 LOG.debug("Selected renderer '%s' from priority list: %s",
14diff --git a/cloudinit/distros/debian.py b/cloudinit/distros/debian.py
15index 1101f02..a069bc2 100644
16--- a/cloudinit/distros/debian.py
17+++ b/cloudinit/distros/debian.py
18@@ -36,17 +36,20 @@ ENI_HEADER = """# This file is generated from information provided by
19 # network: {config: disabled}
20 """
21
22-NETWORK_CONF_FN = "/etc/network/interfaces.d/50-cloud-init.cfg"
23-
24
25 class Distro(distros.Distro):
26 hostname_conf_fn = "/etc/hostname"
27 locale_conf_fn = "/etc/default/locale"
28+ network_conf_fn = {
29+ "eni": "/etc/network/interfaces.d/50-cloud-init.cfg",
30+ "netplan": "/etc/netplan/50-cloud-init.yaml"
31+ }
32 renderer_configs = {
33- 'eni': {
34- 'eni_path': NETWORK_CONF_FN,
35- 'eni_header': ENI_HEADER,
36- }
37+ "eni": {"eni_path": network_conf_fn["eni"],
38+ "eni_header": ENI_HEADER},
39+ "netplan": {"netplan_path": network_conf_fn["netplan"],
40+ "netplan_header": ENI_HEADER,
41+ "postcmds": True}
42 }
43
44 def __init__(self, name, cfg, paths):
45@@ -75,7 +78,8 @@ class Distro(distros.Distro):
46 self.package_command('install', pkgs=pkglist)
47
48 def _write_network(self, settings):
49- util.write_file(NETWORK_CONF_FN, settings)
50+ # this is always going to be legacy based
51+ util.write_file(self.network_conf_fn["eni"], settings)
52 return ['all']
53
54 def _write_network_config(self, netconfig):
55diff --git a/cloudinit/distros/rhel.py b/cloudinit/distros/rhel.py
56index 372c7d0..0eef69b 100644
57--- a/cloudinit/distros/rhel.py
58+++ b/cloudinit/distros/rhel.py
59@@ -39,6 +39,7 @@ class Distro(distros.Distro):
60 resolve_conf_fn = "/etc/resolv.conf"
61 tz_local_fn = "/etc/localtime"
62 usr_lib_exec = "/usr/libexec"
63+ network_render_config = {'sysconfig': None}
64
65 def __init__(self, name, cfg, paths):
66 distros.Distro.__init__(self, name, cfg, paths)
67diff --git a/cloudinit/net/__init__.py b/cloudinit/net/__init__.py
68index 1cf98ef..1297808 100755
69--- a/cloudinit/net/__init__.py
70+++ b/cloudinit/net/__init__.py
71@@ -14,6 +14,7 @@ from cloudinit import util
72
73 LOG = logging.getLogger(__name__)
74 SYS_CLASS_NET = "/sys/class/net/"
75+SYS_DEV_VIRT_NET = "/sys/devices/virtual/net/"
76 DEFAULT_PRIMARY_INTERFACE = 'eth0'
77
78
79@@ -205,7 +206,11 @@ def _get_current_rename_info(check_downable=True):
80 """Collect information necessary for rename_interfaces."""
81 names = get_devicelist()
82 bymac = {}
83+ virtual = os.listdir(SYS_DEV_VIRT_NET)
84 for n in names:
85+ # do not attempt to rename virtual interfaces
86+ if n in virtual:
87+ continue
88 bymac[get_interface_mac(n)] = {
89 'name': n, 'up': is_up(n), 'downable': None}
90
91diff --git a/cloudinit/net/eni.py b/cloudinit/net/eni.py
92index 9d39a2b..6bb744e 100644
93--- a/cloudinit/net/eni.py
94+++ b/cloudinit/net/eni.py
95@@ -8,6 +8,7 @@ import re
96 from . import ParserError
97
98 from . import renderer
99+from .network_state import subnet_is_ipv6
100
101 from cloudinit import util
102
103@@ -111,16 +112,6 @@ def _iface_start_entry(iface, index, render_hwaddress=False):
104 return lines
105
106
107-def _subnet_is_ipv6(subnet):
108- # 'static6' or 'dhcp6'
109- if subnet['type'].endswith('6'):
110- # This is a request for DHCPv6.
111- return True
112- elif subnet['type'] == 'static' and ":" in subnet['address']:
113- return True
114- return False
115-
116-
117 def _parse_deb_config_data(ifaces, contents, src_dir, src_path):
118 """Parses the file contents, placing result into ifaces.
119
120@@ -370,7 +361,7 @@ class Renderer(renderer.Renderer):
121 iface['mode'] = subnet['type']
122 iface['control'] = subnet.get('control', 'auto')
123 subnet_inet = 'inet'
124- if _subnet_is_ipv6(subnet):
125+ if subnet_is_ipv6(subnet):
126 subnet_inet += '6'
127 iface['inet'] = subnet_inet
128 if subnet['type'].startswith('dhcp'):
129@@ -486,7 +477,7 @@ class Renderer(renderer.Renderer):
130 def network_state_to_eni(network_state, header=None, render_hwaddress=False):
131 # render the provided network state, return a string of equivalent eni
132 eni_path = 'etc/network/interfaces'
133- renderer = Renderer({
134+ renderer = Renderer(config={
135 'eni_path': eni_path,
136 'eni_header': header,
137 'links_path_prefix': None,
138diff --git a/cloudinit/net/netplan.py b/cloudinit/net/netplan.py
139new file mode 100644
140index 0000000..49f3120
141--- /dev/null
142+++ b/cloudinit/net/netplan.py
143@@ -0,0 +1,383 @@
144+# This file is part of cloud-init. See LICENSE file ...
145+
146+import copy
147+import os
148+from textwrap import indent
149+
150+from . import renderer
151+from .network_state import subnet_is_ipv6
152+
153+from cloudinit import util
154+from cloudinit.net import SYS_CLASS_NET, get_devicelist
155+
156+
157+NET_CONFIG_TO_V2 = {
158+ 'bond': {'bond-ad-select': 'ad-select',
159+ 'bond-arp-interval': 'arp-interval',
160+ 'bond-arp-ip-target': 'arp-ip-target',
161+ 'bond-arp-validate': 'arp-validate',
162+ 'bond-downdelay': 'down-delay',
163+ 'bond-fail-over-mac': 'fail-over-mac-policy',
164+ 'bond-lacp-rate': 'lacp-rate',
165+ 'bond-miimon': 'mii-monitor-interval',
166+ 'bond-min-links': 'min-links',
167+ 'bond-mode': 'mode',
168+ 'bond-num-grat-arp': 'gratuitious-arp',
169+ 'bond-primary-reselect': 'primary-reselect-policy',
170+ 'bond-updelay': 'up-delay',
171+ 'bond-xmit_hash_policy': 'transmit_hash_policy'},
172+ 'bridge': {'bridge_ageing': 'ageing-time',
173+ 'bridge_bridgeprio': 'priority',
174+ 'bridge_fd': 'forward-delay',
175+ 'bridge_gcint': None,
176+ 'bridge_hello': 'hello-time',
177+ 'bridge_maxage': 'max-age',
178+ 'bridge_maxwait': None,
179+ 'bridge_pathcost': 'path-cost',
180+ 'bridge_portprio': None,
181+ 'bridge_waitport': None}}
182+
183+
184+def _get_params_dict_by_match(config, match):
185+ return dict((key, value) for (key, value) in config.items()
186+ if key.startswith(match))
187+
188+
189+def _extract_addresses(config, entry):
190+ """ This method parse a cloudinit.net.network_state dictionary (config) and
191+ maps netstate keys/values into a dictionary (entry) to represent
192+ netplan yaml.
193+
194+ An example config dictionary might look like:
195+
196+ {'mac_address': '52:54:00:12:34:00',
197+ 'name': 'interface0',
198+ 'subnets': [
199+ {'address': '192.168.1.2/24',
200+ 'mtu': 1501,
201+ 'type': 'static'},
202+ {'address': '2001:4800:78ff:1b:be76:4eff:fe06:1000",
203+ 'mtu': 1480,
204+ 'netmask': 64,
205+ 'type': 'static'}],
206+ 'type: physical'
207+ }
208+
209+ An entry dictionary looks like:
210+
211+ {'set-name': 'interface0',
212+ 'match': {'macaddress': '52:54:00:12:34:00'},
213+ 'mtu': 1501}
214+
215+ After modification returns
216+
217+ {'set-name': 'interface0',
218+ 'match': {'macaddress': '52:54:00:12:34:00'},
219+ 'mtu': 1501,
220+ 'address': ['192.168.1.2/24', '2001:4800:78ff:1b:be76:4eff:fe06:1000"],
221+ 'mtu6': 1480}
222+
223+ """
224+
225+ def _listify(obj, token=' '):
226+ """ Helper to convert strings to list of strings, handle single
227+ string"""
228+ if not obj or type(obj) not in [str]:
229+ return obj
230+ if token in obj:
231+ return obj.split(token)
232+ else:
233+ return [obj, ]
234+
235+ addresses = []
236+ routes = []
237+ nameservers = []
238+ searchdomains = []
239+ subnets = config.get('subnets', [])
240+ if subnets is None:
241+ subnets = []
242+ for subnet in subnets:
243+ sn_type = subnet.get('type')
244+ if sn_type.startswith('dhcp'):
245+ if sn_type == 'dhcp':
246+ sn_type += '4'
247+ entry.update({sn_type: True})
248+ elif sn_type in ['static']:
249+ addr = "%s" % subnet.get('address')
250+ if 'netmask' in subnet:
251+ addr += "/%s" % subnet.get('netmask')
252+ if 'gateway' in subnet and subnet.get('gateway'):
253+ gateway = subnet.get('gateway')
254+ if ":" in gateway:
255+ entry.update({'gateway6': gateway})
256+ else:
257+ entry.update({'gateway4': gateway})
258+ if 'dns_nameservers' in subnet:
259+ nameservers += _listify(subnet.get('dns_nameservers', []))
260+ if 'dns_search' in subnet:
261+ searchdomains += _listify(subnet.get('dns_search', []))
262+ if 'mtu' in subnet:
263+ mtukey = 'mtu'
264+ if subnet_is_ipv6(subnet):
265+ mtukey += '6'
266+ entry.update({mtukey: subnet.get('mtu')})
267+ for route in subnet.get('routes', []):
268+ to_net = "%s/%s" % (route.get('network'),
269+ route.get('netmask'))
270+ route = {
271+ 'via': route.get('gateway'),
272+ 'to': to_net,
273+ }
274+ if 'metric' in route:
275+ route.update({'metric': route.get('metric', 100)})
276+ routes.append(route)
277+
278+ addresses.append(addr)
279+
280+ if len(addresses) > 0:
281+ entry.update({'addresses': addresses})
282+ if len(routes) > 0:
283+ entry.update({'routes': routes})
284+ if len(nameservers) > 0:
285+ ns = {'addresses': nameservers}
286+ entry.update({'nameservers': ns})
287+ if len(searchdomains) > 0:
288+ ns = entry.get('nameservers', {})
289+ ns.update({'search': searchdomains})
290+ entry.update({'nameservers': ns})
291+
292+
293+def _extract_bond_slaves_by_name(interfaces, entry, bond_master):
294+ bond_slave_names = sorted([name for (name, cfg) in interfaces.items()
295+ if cfg.get('bond-master', None) == bond_master])
296+ if len(bond_slave_names) > 0:
297+ entry.update({'interfaces': bond_slave_names})
298+
299+
300+class Renderer(renderer.Renderer):
301+ """Renders network information in a /etc/netplan/network.yaml format."""
302+
303+ NETPLAN_GENERATE = ['netplan', 'generate']
304+
305+ def __init__(self, config=None):
306+ if not config:
307+ config = {}
308+ self.netplan_path = config.get('netplan_path',
309+ 'etc/netplan/50-cloud-init.yaml')
310+ self.netplan_header = config.get('netplan_header', None)
311+ self._postcmds = config.get('postcmds', False)
312+
313+ def render_network_state(self, target, network_state):
314+ # check network state for version
315+ # if v2, then extract network_state.config
316+ # else render_v2_from_state
317+ fpnplan = os.path.join(target, self.netplan_path)
318+ util.ensure_dir(os.path.dirname(fpnplan))
319+ header = self.netplan_header if self.netplan_header else ""
320+
321+ if network_state.version > 1:
322+ # pass-through original config
323+ content = util.yaml_dumps({'network': network_state.config},
324+ explicit_start=False,
325+ explicit_end=False)
326+ else:
327+ # render from state
328+ content = self._render_content(network_state)
329+ # ensure we poke udev to run net_setup_link
330+ if not header.endswith("\n"):
331+ header += "\n"
332+ util.write_file(fpnplan, header + content)
333+
334+ self._netplan_generate(run=self._postcmds)
335+ self._net_setup_link(run=self._postcmds)
336+
337+ def _netplan_generate(self, run=False):
338+ if not run:
339+ return
340+ util.subp(self.NETPLAN_GENERATE, capture=True)
341+
342+ def _net_setup_link(self, run=False):
343+ """ To ensure device link properties are applied, we poke
344+ udev to re-evaluate networkd .link files and call
345+ the setup_link udev builtin command
346+ """
347+ if not run:
348+ return
349+ setup_lnk = ['udevadm', 'test-builtin', 'net_setup_link']
350+ for cmd in [setup_lnk + [SYS_CLASS_NET + iface]
351+ for iface in get_devicelist() if
352+ os.path.islink(SYS_CLASS_NET + iface)]:
353+ util.subp(cmd, capture=True)
354+
355+ def _render_content(self, network_state):
356+ ethernets = {}
357+ wifis = {}
358+ bridges = {}
359+ bonds = {}
360+ vlans = {}
361+ content = []
362+
363+ interfaces = network_state._network_state.get('interfaces', [])
364+
365+ nameservers = network_state.dns_nameservers
366+ searchdomains = network_state.dns_searchdomains
367+
368+ for config in network_state.iter_interfaces():
369+ ifname = config.get('name')
370+ # filter None entries up front so we can do simple if key in dict
371+ ifcfg = dict((key, value) for (key, value) in config.items()
372+ if value)
373+
374+ if_type = ifcfg.get('type')
375+ if if_type == 'physical':
376+ # required_keys = ['name', 'mac_address']
377+ eth = {
378+ 'set-name': ifname,
379+ 'match': ifcfg.get('match', None),
380+ }
381+ if eth['match'] is None:
382+ macaddr = ifcfg.get('mac_address', None)
383+ if macaddr is not None:
384+ eth['match'] = {'macaddress': macaddr.lower()}
385+ else:
386+ del eth['match']
387+ del eth['set-name']
388+ if 'mtu' in ifcfg:
389+ eth['mtu'] = ifcfg.get('mtu')
390+
391+ _extract_addresses(ifcfg, eth)
392+ ethernets.update({ifname: eth})
393+
394+ elif if_type == 'bond':
395+ # required_keys = ['name', 'bond_interfaces']
396+ bond = {}
397+ bond_config = {}
398+ # extract bond params and drop the bond_ prefix as it's
399+ # redundent in v2 yaml format
400+ v2_bond_map = NET_CONFIG_TO_V2.get('bond')
401+ for match in ['bond_', 'bond-']:
402+ bond_params = _get_params_dict_by_match(ifcfg, match)
403+ for (param, value) in bond_params.items():
404+ newname = v2_bond_map.get(param)
405+ if newname is None:
406+ continue
407+ bond_config.update({newname: value})
408+
409+ if len(bond_config) > 0:
410+ bond.update({'parameters': bond_config})
411+ slave_interfaces = ifcfg.get('bond-slaves')
412+ if slave_interfaces == 'none':
413+ _extract_bond_slaves_by_name(interfaces, bond, ifname)
414+ _extract_addresses(ifcfg, bond)
415+ bonds.update({ifname: bond})
416+
417+ elif if_type == 'bridge':
418+ # required_keys = ['name', 'bridge_ports']
419+ ports = sorted(copy.copy(ifcfg.get('bridge_ports')))
420+ bridge = {
421+ 'interfaces': ports,
422+ }
423+ # extract bridge params and drop the bridge prefix as it's
424+ # redundent in v2 yaml format
425+ match_prefix = 'bridge_'
426+ params = _get_params_dict_by_match(ifcfg, match_prefix)
427+ br_config = {}
428+
429+ # v2 yaml uses different names for the keys
430+ # and at least one value format change
431+ v2_bridge_map = NET_CONFIG_TO_V2.get('bridge')
432+ for (param, value) in params.items():
433+ newname = v2_bridge_map.get(param)
434+ if newname is None:
435+ continue
436+ br_config.update({newname: value})
437+ if newname == 'path-cost':
438+ # <interface> <cost> -> <interface>: int(<cost>)
439+ newvalue = {}
440+ for costval in value:
441+ (port, cost) = costval.split()
442+ newvalue[port] = int(cost)
443+ br_config.update({newname: newvalue})
444+ if len(br_config) > 0:
445+ bridge.update({'parameters': br_config})
446+ _extract_addresses(ifcfg, bridge)
447+ bridges.update({ifname: bridge})
448+
449+ elif if_type == 'vlan':
450+ # required_keys = ['name', 'vlan_id', 'vlan-raw-device']
451+ vlan = {
452+ 'id': ifcfg.get('vlan_id'),
453+ 'link': ifcfg.get('vlan-raw-device')
454+ }
455+
456+ _extract_addresses(ifcfg, vlan)
457+ vlans.update({ifname: vlan})
458+
459+ # inject global nameserver values under each physical interface
460+ if nameservers:
461+ for _eth, cfg in ethernets.items():
462+ nscfg = cfg.get('nameservers', {})
463+ addresses = nscfg.get('addresses', [])
464+ addresses += nameservers
465+ nscfg.update({'addresses': addresses})
466+ cfg.update({'nameservers': nscfg})
467+
468+ if searchdomains:
469+ for _eth, cfg in ethernets.items():
470+ nscfg = cfg.get('nameservers', {})
471+ search = nscfg.get('search', [])
472+ search += searchdomains
473+ nscfg.update({'search': search})
474+ cfg.update({'nameservers': nscfg})
475+
476+ # workaround yaml dictionary key sorting when dumping
477+ def _render_section(name, section):
478+ if section:
479+ dump = util.yaml_dumps({name: section},
480+ explicit_start=False,
481+ explicit_end=False)
482+ txt = indent(dump, ' ' * 4)
483+ return [txt]
484+ return []
485+
486+ content.append("network:\n version: 2\n")
487+ content += _render_section('ethernets', ethernets)
488+ content += _render_section('wifis', wifis)
489+ content += _render_section('bonds', bonds)
490+ content += _render_section('bridges', bridges)
491+ content += _render_section('vlans', vlans)
492+
493+ return "".join(content)
494+
495+
496+def network_state_to_netplan(network_state, header=None):
497+ # render the provided network state, return a string of equivalent eni
498+ netplan_path = 'etc/network/50-cloud-init.yaml'
499+ renderer = Renderer({
500+ 'netplan_path': netplan_path,
501+ 'netplan_header': header,
502+ })
503+ if not header:
504+ header = ""
505+ if not header.endswith("\n"):
506+ header += "\n"
507+ contents = renderer._render_content(network_state)
508+ return header + contents
509+
510+
511+def available(target=None):
512+ expected = ['netplan']
513+ search = ['/usr/sbin', '/sbin']
514+ for p in expected:
515+ if not util.which(p, search=search, target=target):
516+ return False
517+ return True
518+
519+ expected_paths = ['etc/netplan/']
520+ for p in expected_paths:
521+ if not os.path.isfile(util.target_path(target, p)):
522+ return False
523+ return True
524+
525+
526+# vi: ts=4 expandtab
527diff --git a/cloudinit/net/network_state.py b/cloudinit/net/network_state.py
528index 90b2835..2469f7c 100644
529--- a/cloudinit/net/network_state.py
530+++ b/cloudinit/net/network_state.py
531@@ -1,4 +1,4 @@
532-# Copyright (C) 2013-2014 Canonical Ltd.
533+# Copyright (C) 2017 Canonical Ltd.
534 #
535 # Author: Ryan Harper <ryan.harper@canonical.com>
536 #
537@@ -18,6 +18,10 @@ NETWORK_STATE_VERSION = 1
538 NETWORK_STATE_REQUIRED_KEYS = {
539 1: ['version', 'config', 'network_state'],
540 }
541+NETWORK_V2_KEY_FILTER = [
542+ 'addresses', 'dhcp4', 'dhcp6', 'gateway4', 'gateway6', 'interfaces',
543+ 'match', 'mtu', 'nameservers', 'renderer', 'set-name', 'wakeonlan'
544+]
545
546
547 def parse_net_config_data(net_config, skip_broken=True):
548@@ -26,11 +30,18 @@ def parse_net_config_data(net_config, skip_broken=True):
549 :param net_config: curtin network config dict
550 """
551 state = None
552- if 'version' in net_config and 'config' in net_config:
553- nsi = NetworkStateInterpreter(version=net_config.get('version'),
554- config=net_config.get('config'))
555+ version = net_config.get('version')
556+ config = net_config.get('config')
557+ if version == 2:
558+ # v2 does not have explicit 'config' key so we
559+ # pass the whole net-config as-is
560+ config = net_config
561+
562+ if version and config:
563+ nsi = NetworkStateInterpreter(version=version, config=config)
564 nsi.parse_config(skip_broken=skip_broken)
565- state = nsi.network_state
566+ state = nsi.get_network_state()
567+
568 return state
569
570
571@@ -103,14 +114,21 @@ class CommandHandlerMeta(type):
572
573 class NetworkState(object):
574
575- def __init__(self, network_state, version=NETWORK_STATE_VERSION):
576+ def __init__(self, network_state, version=NETWORK_STATE_VERSION,
577+ config=None):
578 self._network_state = copy.deepcopy(network_state)
579 self._version = version
580+ self._config = copy.deepcopy(config)
581+ self.use_ipv6 = network_state.get('use_ipv6', False)
582
583 @property
584 def version(self):
585 return self._version
586
587+ @property
588+ def config(self):
589+ return self._config
590+
591 def iter_routes(self, filter_func=None):
592 for route in self._network_state.get('routes', []):
593 if filter_func is not None:
594@@ -152,7 +170,8 @@ class NetworkStateInterpreter(object):
595 'dns': {
596 'nameservers': [],
597 'search': [],
598- }
599+ },
600+ 'use_ipv6': False,
601 }
602
603 def __init__(self, version=NETWORK_STATE_VERSION, config=None):
604@@ -163,7 +182,16 @@ class NetworkStateInterpreter(object):
605
606 @property
607 def network_state(self):
608- return NetworkState(self._network_state, version=self._version)
609+ return NetworkState(self._network_state, version=self._version,
610+ config=self._config)
611+
612+ @property
613+ def use_ipv6(self):
614+ return self._network_state.get('use_ipv6')
615+
616+ @use_ipv6.setter
617+ def use_ipv6(self, val):
618+ self._network_state.update({'use_ipv6': val})
619
620 def dump(self):
621 state = {
622@@ -192,8 +220,19 @@ class NetworkStateInterpreter(object):
623 def dump_network_state(self):
624 return util.yaml_dumps(self._network_state)
625
626+ def as_dict(self):
627+ return {'version': self.version, 'config': self.config}
628+
629+ def get_network_state(self):
630+ ns = self.network_state
631+ return ns
632+
633 def parse_config(self, skip_broken=True):
634- # rebuild network state
635+ if self._version == 1:
636+ self.parse_config_v1(skip_broken=skip_broken)
637+ self._parsed = True
638+
639+ def parse_config_v1(self, skip_broken=True):
640 for command in self._config:
641 command_type = command['type']
642 try:
643@@ -211,6 +250,25 @@ class NetworkStateInterpreter(object):
644 exc_info=True)
645 LOG.debug(self.dump_network_state())
646
647+ def parse_config_v2(self, skip_broken=True):
648+ for command_type, command in self._config.get('network').items():
649+
650+ try:
651+ handler = self.command_handlers[command_type]
652+ except KeyError:
653+ raise RuntimeError("No handler found for"
654+ " command '%s'" % command_type)
655+ try:
656+ handler(self, command)
657+ self._v2_common(self, command)
658+ except InvalidCommand:
659+ if not skip_broken:
660+ raise
661+ else:
662+ LOG.warn("Skipping invalid command: %s", command,
663+ exc_info=True)
664+ LOG.debug(self.dump_network_state())
665+
666 @ensure_command_keys(['name'])
667 def handle_loopback(self, command):
668 return self.handle_physical(command)
669@@ -238,11 +296,16 @@ class NetworkStateInterpreter(object):
670 if subnets:
671 for subnet in subnets:
672 if subnet['type'] == 'static':
673+ if ':' in subnet['address']:
674+ self.use_ipv6 = True
675 if 'netmask' in subnet and ':' in subnet['address']:
676 subnet['netmask'] = mask2cidr(subnet['netmask'])
677 for route in subnet.get('routes', []):
678 if 'netmask' in route:
679 route['netmask'] = mask2cidr(route['netmask'])
680+ elif subnet['type'].endswith('6'):
681+ self.use_ipv6 = True
682+
683 iface.update({
684 'name': command.get('name'),
685 'type': command.get('type'),
686@@ -327,7 +390,7 @@ class NetworkStateInterpreter(object):
687 bond_if.update({param: val})
688 self._network_state['interfaces'].update({ifname: bond_if})
689
690- @ensure_command_keys(['name', 'bridge_interfaces', 'params'])
691+ @ensure_command_keys(['name', 'bridge_interfaces'])
692 def handle_bridge(self, command):
693 '''
694 auto br0
695@@ -373,7 +436,7 @@ class NetworkStateInterpreter(object):
696 self.handle_physical(command)
697 iface = interfaces.get(command.get('name'), {})
698 iface['bridge_ports'] = command['bridge_interfaces']
699- for param, val in command.get('params').items():
700+ for param, val in command.get('params', {}).items():
701 iface.update({param: val})
702
703 interfaces.update({iface['name']: iface})
704@@ -407,6 +470,239 @@ class NetworkStateInterpreter(object):
705 }
706 routes.append(route)
707
708+ # V2 handlers
709+ def handle_bonds(self, command):
710+ '''
711+ v2_command = {
712+ bond0: {
713+ 'interfaces': ['interface0', 'interface1'],
714+ 'miimon': 100,
715+ 'mode': '802.3ad',
716+ 'xmit_hash_policy': 'layer3+4'},
717+ bond1: {
718+ 'bond-slaves': ['interface2', 'interface7'],
719+ 'mode': 1
720+ }
721+ }
722+
723+ v1_command = {
724+ 'type': 'bond'
725+ 'name': 'bond0',
726+ 'bond_interfaces': [interface0, interface1],
727+ 'params': {
728+ 'bond-mode': '802.3ad',
729+ 'bond_miimon: 100,
730+ 'bond_xmit_hash_policy': 'layer3+4',
731+ }
732+ }
733+
734+ '''
735+ self._handle_bond_bridge(command, cmd_type='bond')
736+
737+ def handle_bridges(self, command):
738+
739+ '''
740+ v2_command = {
741+ br0: {
742+ 'interfaces': ['interface0', 'interface1'],
743+ 'fd': 0,
744+ 'stp': 'off',
745+ 'maxwait': 0,
746+ }
747+ }
748+
749+ v1_command = {
750+ 'type': 'bridge'
751+ 'name': 'br0',
752+ 'bridge_interfaces': [interface0, interface1],
753+ 'params': {
754+ 'bridge_stp': 'off',
755+ 'bridge_fd: 0,
756+ 'bridge_maxwait': 0
757+ }
758+ }
759+
760+ '''
761+ self._handle_bond_bridge(command, cmd_type='bridge')
762+
763+ def handle_ethernets(self, command):
764+ '''
765+ ethernets:
766+ eno1:
767+ match:
768+ macaddress: 00:11:22:33:44:55
769+ wakeonlan: true
770+ dhcp4: true
771+ dhcp6: false
772+ addresses:
773+ - 192.168.14.2/24
774+ - 2001:1::1/64
775+ gateway4: 192.168.14.1
776+ gateway6: 2001:1::2
777+ nameservers:
778+ search: [foo.local, bar.local]
779+ addresses: [8.8.8.8, 8.8.4.4]
780+ lom:
781+ match:
782+ driver: ixgbe
783+ set-name: lom1
784+ dhcp6: true
785+ switchports:
786+ match:
787+ name: enp2*
788+ mtu: 1280
789+
790+ command = {
791+ 'type': 'physical',
792+ 'mac_address': 'c0:d6:9f:2c:e8:80',
793+ 'name': 'eth0',
794+ 'subnets': [
795+ {'type': 'dhcp4'}
796+ ]
797+ }
798+ '''
799+ for eth, cfg in command.items():
800+ phy_cmd = {
801+ 'type': 'physical',
802+ 'name': cfg.get('set-name', eth),
803+ }
804+ mac_address = cfg.get('match', {}).get('macaddress', None)
805+ if not mac_address:
806+ LOG.warning('NetworkState Version2: missing macaddress')
807+
808+ for key in ['mtu', 'match', 'wakeonlan']:
809+ if key in cfg:
810+ phy_cmd.update({key: cfg.get(key)})
811+
812+ subnets = self._v2_to_v1_ipcfg(cfg)
813+ if len(subnets) > 0:
814+ phy_cmd.update({'subnets': subnets})
815+
816+ LOG.debug('v2(ethernets) -> v1(physical):\n%s', phy_cmd)
817+ self.handle_physical(phy_cmd)
818+
819+ def handle_vlans(self, command):
820+ '''
821+ v2_vlans = {
822+ 'eth0.123': {
823+ 'id': 123,
824+ 'link': 'eth0',
825+ 'dhcp4': True,
826+ }
827+ }
828+
829+ v1_command = {
830+ 'type': 'vlan',
831+ 'name': 'eth0.123',
832+ 'vlan_link': 'eth0',
833+ 'vlan_id': 123,
834+ 'subnets': [{'type': 'dhcp4'}],
835+ }
836+ '''
837+ for vlan, cfg in command.items():
838+ vlan_cmd = {
839+ 'type': 'vlan',
840+ 'name': vlan,
841+ 'vlan_id': cfg.get('id'),
842+ 'vlan_link': cfg.get('link'),
843+ }
844+ subnets = self._v2_to_v1_ipcfg(cfg)
845+ if len(subnets) > 0:
846+ vlan_cmd.update({'subnets': subnets})
847+ LOG.debug('v2(vlans) -> v1(vlan):\n%s', vlan_cmd)
848+ self.handle_vlan(vlan_cmd)
849+
850+ def handle_wifis(self, command):
851+ raise NotImplemented('NetworkState V2: Skipping wifi configuration')
852+
853+ def _v2_common(self, cfg):
854+ LOG.debug('v2_common: handling config:\n%s', cfg)
855+ if 'nameservers' in cfg:
856+ search = cfg.get('nameservers').get('search', [])
857+ dns = cfg.get('nameservers').get('addresses', [])
858+ name_cmd = {'type': 'nameserver'}
859+ if len(search) > 0:
860+ name_cmd.update({'search': search})
861+ if len(dns) > 0:
862+ name_cmd.update({'addresses': dns})
863+ LOG.debug('v2(nameserver) -> v1(nameserver):\n%s', name_cmd)
864+ self.handle_nameserver(name_cmd)
865+
866+ def _handle_bond_bridge(self, command, cmd_type=None):
867+ """Common handler for bond and bridge types"""
868+ for item_name, item_cfg in command.items():
869+ item_params = dict((key, value) for (key, value) in
870+ item_cfg.items() if key not in
871+ NETWORK_V2_KEY_FILTER)
872+ v1_cmd = {
873+ 'type': cmd_type,
874+ 'name': item_name,
875+ cmd_type + '_interfaces': item_cfg.get('interfaces'),
876+ 'params': item_params,
877+ }
878+ subnets = self._v2_to_v1_ipcfg(item_cfg)
879+ if len(subnets) > 0:
880+ v1_cmd.update({'subnets': subnets})
881+
882+ LOG.debug('v2(%ss) -> v1(%s):\n%s', cmd_type, cmd_type, v1_cmd)
883+ self.handle_bridge(v1_cmd)
884+
885+ def _v2_to_v1_ipcfg(self, cfg):
886+ """Common ipconfig extraction from v2 to v1 subnets array."""
887+
888+ subnets = []
889+ if 'dhcp4' in cfg:
890+ subnets.append({'type': 'dhcp4'})
891+ if 'dhcp6' in cfg:
892+ self.use_ipv6 = True
893+ subnets.append({'type': 'dhcp6'})
894+
895+ gateway4 = None
896+ gateway6 = None
897+ for address in cfg.get('addresses', []):
898+ subnet = {
899+ 'type': 'static',
900+ 'address': address,
901+ }
902+
903+ routes = []
904+ for route in cfg.get('routes', []):
905+ route_addr = route.get('to')
906+ if "/" in route_addr:
907+ route_addr, route_cidr = route_addr.split("/")
908+ route_netmask = cidr2mask(route_cidr)
909+ subnet_route = {
910+ 'address': route_addr,
911+ 'netmask': route_netmask,
912+ 'gateway': route.get('via')
913+ }
914+ routes.append(subnet_route)
915+ if len(routes) > 0:
916+ subnet.update({'routes': routes})
917+
918+ if ":" in address:
919+ if 'gateway6' in cfg and gateway6 is None:
920+ gateway6 = cfg.get('gateway6')
921+ subnet.update({'gateway': gateway6})
922+ else:
923+ if 'gateway4' in cfg and gateway4 is None:
924+ gateway4 = cfg.get('gateway4')
925+ subnet.update({'gateway': gateway4})
926+
927+ subnets.append(subnet)
928+ return subnets
929+
930+
931+def subnet_is_ipv6(subnet):
932+ """ Common helper for checking network_state subnets for ipv6"""
933+ # 'static6' or 'dhcp6'
934+ if subnet['type'].endswith('6'):
935+ # This is a request for DHCPv6.
936+ return True
937+ elif subnet['type'] == 'static' and ":" in subnet['address']:
938+ return True
939+ return False
940+
941
942 def cidr2mask(cidr):
943 mask = [0, 0, 0, 0]
944diff --git a/cloudinit/net/renderer.py b/cloudinit/net/renderer.py
945index a5b2b57..77bdc62 100644
946--- a/cloudinit/net/renderer.py
947+++ b/cloudinit/net/renderer.py
948@@ -41,4 +41,5 @@ class Renderer(object):
949 return self.render_network_state(
950 network_state=parse_net_config_data(network_config), target=target)
951
952+
953 # vi: ts=4 expandtab
954diff --git a/cloudinit/net/renderers.py b/cloudinit/net/renderers.py
955index 5ad8455..5117b4a 100644
956--- a/cloudinit/net/renderers.py
957+++ b/cloudinit/net/renderers.py
958@@ -1,15 +1,17 @@
959 # This file is part of cloud-init. See LICENSE file for license information.
960
961 from . import eni
962+from . import netplan
963 from . import RendererNotFoundError
964 from . import sysconfig
965
966 NAME_TO_RENDERER = {
967 "eni": eni,
968+ "netplan": netplan,
969 "sysconfig": sysconfig,
970 }
971
972-DEFAULT_PRIORITY = ["eni", "sysconfig"]
973+DEFAULT_PRIORITY = ["eni", "sysconfig", "netplan"]
974
975
976 def search(priority=None, target=None, first=False):
977diff --git a/cloudinit/net/sysconfig.py b/cloudinit/net/sysconfig.py
978index 117b515..23ac2e3 100644
979--- a/cloudinit/net/sysconfig.py
980+++ b/cloudinit/net/sysconfig.py
981@@ -9,6 +9,7 @@ from cloudinit.distros.parsers import resolv_conf
982 from cloudinit import util
983
984 from . import renderer
985+from .network_state import subnet_is_ipv6
986
987
988 def _make_header(sep='#'):
989@@ -194,7 +195,7 @@ class Renderer(renderer.Renderer):
990 def __init__(self, config=None):
991 if not config:
992 config = {}
993- self.sysconf_dir = config.get('sysconf_dir', 'etc/sysconfig/')
994+ self.sysconf_dir = config.get('sysconf_dir', 'etc/sysconfig')
995 self.netrules_path = config.get(
996 'netrules_path', 'etc/udev/rules.d/70-persistent-net.rules')
997 self.dns_path = config.get('dns_path', 'etc/resolv.conf')
998@@ -220,7 +221,7 @@ class Renderer(renderer.Renderer):
999 iface_cfg['BOOTPROTO'] = 'dhcp'
1000 elif subnet_type == 'static':
1001 iface_cfg['BOOTPROTO'] = 'static'
1002- if subnet.get('ipv6'):
1003+ if subnet_is_ipv6(subnet):
1004 iface_cfg['IPV6ADDR'] = subnet['address']
1005 iface_cfg['IPV6INIT'] = True
1006 else:
1007@@ -390,19 +391,32 @@ class Renderer(renderer.Renderer):
1008 return contents
1009
1010 def render_network_state(self, network_state, target=None):
1011+ file_mode = 0o644
1012 base_sysconf_dir = util.target_path(target, self.sysconf_dir)
1013 for path, data in self._render_sysconfig(base_sysconf_dir,
1014 network_state).items():
1015- util.write_file(path, data)
1016+ util.write_file(path, data, file_mode)
1017 if self.dns_path:
1018 dns_path = util.target_path(target, self.dns_path)
1019 resolv_content = self._render_dns(network_state,
1020 existing_dns_path=dns_path)
1021- util.write_file(dns_path, resolv_content)
1022+ util.write_file(dns_path, resolv_content, file_mode)
1023 if self.netrules_path:
1024 netrules_content = self._render_persistent_net(network_state)
1025 netrules_path = util.target_path(target, self.netrules_path)
1026- util.write_file(netrules_path, netrules_content)
1027+ util.write_file(netrules_path, netrules_content, file_mode)
1028+
1029+ # always write /etc/sysconfig/network configuration
1030+ sysconfig_path = util.target_path(target, "etc/sysconfig/network")
1031+ netcfg = [
1032+ ('# Created by cloud-init on instance boot automatically, '
1033+ 'do not edit.'),
1034+ 'NETWORKING=yes',
1035+ ]
1036+ if network_state.use_ipv6:
1037+ netcfg.append('NETWORKING_IPV6=yes')
1038+ netcfg.append('IPV6_AUTOCONF=no')
1039+ util.write_file(sysconfig_path, "\n".join(netcfg) + "\n", file_mode)
1040
1041
1042 def available(target=None):
1043diff --git a/systemd/cloud-init.service b/systemd/cloud-init.service
1044index fb3b918..39acc20 100644
1045--- a/systemd/cloud-init.service
1046+++ b/systemd/cloud-init.service
1047@@ -5,6 +5,7 @@ Wants=cloud-init-local.service
1048 Wants=sshd-keygen.service
1049 Wants=sshd.service
1050 After=cloud-init-local.service
1051+After=systemd-networkd-wait-online.service
1052 After=networking.service
1053 Before=network-online.target
1054 Before=sshd-keygen.service
1055diff --git a/systemd/systemd-networkd-wait-online.path b/systemd/systemd-networkd-wait-online.path
1056new file mode 100644
1057index 0000000..64940b8
1058--- /dev/null
1059+++ b/systemd/systemd-networkd-wait-online.path
1060@@ -0,0 +1,5 @@
1061+[Unit]
1062+Description=Trigger systemd-networkd-wait-online if netplan runs/updates
1063+
1064+[Path]
1065+PathChanged=/run/systemd/generator/netplan.stamp
1066diff --git a/tests/unittests/test_distros/test_netconfig.py b/tests/unittests/test_distros/test_netconfig.py
1067index bde3bb5..2188779 100644
1068--- a/tests/unittests/test_distros/test_netconfig.py
1069+++ b/tests/unittests/test_distros/test_netconfig.py
1070@@ -19,6 +19,7 @@ from cloudinit.distros.parsers.sys_conf import SysConf
1071 from cloudinit import helpers
1072 from cloudinit import settings
1073 from cloudinit import util
1074+from cloudinit.net import eni
1075
1076
1077 BASE_NET_CFG = '''
1078@@ -28,10 +29,10 @@ iface lo inet loopback
1079 auto eth0
1080 iface eth0 inet static
1081 address 192.168.1.5
1082- netmask 255.255.255.0
1083- network 192.168.0.0
1084 broadcast 192.168.1.0
1085 gateway 192.168.1.254
1086+ netmask 255.255.255.0
1087+ network 192.168.0.0
1088
1089 auto eth1
1090 iface eth1 inet dhcp
1091@@ -67,6 +68,100 @@ iface eth1 inet6 static
1092 gateway 2607:f0d0:1002:0011::1
1093 '''
1094
1095+V1_NET_CFG = {'config': [{'name': 'eth0',
1096+
1097+ 'subnets': [{'address': '192.168.1.5',
1098+ 'broadcast': '192.168.1.0',
1099+ 'gateway': '192.168.1.254',
1100+ 'netmask': '255.255.255.0',
1101+ 'type': 'static'}],
1102+ 'type': 'physical'},
1103+ {'name': 'eth1',
1104+ 'subnets': [{'control': 'auto', 'type': 'dhcp4'}],
1105+ 'type': 'physical'}],
1106+ 'version': 1}
1107+
1108+V1_NET_CFG_OUTPUT = """
1109+# This file is generated from information provided by
1110+# the datasource. Changes to it will not persist across an instance.
1111+# To disable cloud-init's network configuration capabilities, write a file
1112+# /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
1113+# network: {config: disabled}
1114+auto lo
1115+iface lo inet loopback
1116+
1117+auto eth0
1118+iface eth0 inet static
1119+ address 192.168.1.5
1120+ broadcast 192.168.1.0
1121+ gateway 192.168.1.254
1122+ netmask 255.255.255.0
1123+
1124+auto eth1
1125+iface eth1 inet dhcp
1126+"""
1127+
1128+V1_NET_CFG_IPV6 = {'config': [{'name': 'eth0',
1129+ 'subnets': [{'address':
1130+ '2607:f0d0:1002:0011::2',
1131+ 'gateway':
1132+ '2607:f0d0:1002:0011::1',
1133+ 'netmask': '64',
1134+ 'type': 'static'}],
1135+ 'type': 'physical'},
1136+ {'name': 'eth1',
1137+ 'subnets': [{'control': 'auto',
1138+ 'type': 'dhcp4'}],
1139+ 'type': 'physical'}],
1140+ 'version': 1}
1141+
1142+
1143+V1_TO_V2_NET_CFG_OUTPUT = """
1144+# This file is generated from information provided by
1145+# the datasource. Changes to it will not persist across an instance.
1146+# To disable cloud-init's network configuration capabilities, write a file
1147+# /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
1148+# network: {config: disabled}
1149+network:
1150+ version: 2
1151+ ethernets:
1152+ eth0:
1153+ addresses:
1154+ - 192.168.1.5/255.255.255.0
1155+ gateway4: 192.168.1.254
1156+ eth1:
1157+ dhcp4: true
1158+"""
1159+
1160+V2_NET_CFG = {
1161+ 'ethernets': {
1162+ 'eth7': {
1163+ 'addresses': ['192.168.1.5/255.255.255.0'],
1164+ 'gateway4': '192.168.1.254'},
1165+ 'eth9': {
1166+ 'dhcp4': True}
1167+ },
1168+ 'version': 2
1169+}
1170+
1171+
1172+V2_TO_V2_NET_CFG_OUTPUT = """
1173+# This file is generated from information provided by
1174+# the datasource. Changes to it will not persist across an instance.
1175+# To disable cloud-init's network configuration capabilities, write a file
1176+# /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
1177+# network: {config: disabled}
1178+network:
1179+ ethernets:
1180+ eth7:
1181+ addresses:
1182+ - 192.168.1.5/255.255.255.0
1183+ gateway4: 192.168.1.254
1184+ eth9:
1185+ dhcp4: true
1186+ version: 2
1187+"""
1188+
1189
1190 class WriteBuffer(object):
1191 def __init__(self):
1192@@ -83,10 +178,12 @@ class WriteBuffer(object):
1193
1194 class TestNetCfgDistro(TestCase):
1195
1196- def _get_distro(self, dname):
1197+ def _get_distro(self, dname, renderers=None):
1198 cls = distros.fetch(dname)
1199 cfg = settings.CFG_BUILTIN
1200 cfg['system_info']['distro'] = dname
1201+ if renderers:
1202+ cfg['system_info']['network'] = {'renderers': renderers}
1203 paths = helpers.Paths({})
1204 return cls(dname, cfg, paths)
1205
1206@@ -116,6 +213,107 @@ class TestNetCfgDistro(TestCase):
1207 self.assertEqual(str(write_buf).strip(), BASE_NET_CFG.strip())
1208 self.assertEqual(write_buf.mode, 0o644)
1209
1210+ def test_apply_network_config_eni_ub(self):
1211+ ub_distro = self._get_distro('ubuntu')
1212+ with ExitStack() as mocks:
1213+ write_bufs = {}
1214+
1215+ def replace_write(filename, content, mode=0o644, omode="wb"):
1216+ buf = WriteBuffer()
1217+ buf.mode = mode
1218+ buf.omode = omode
1219+ buf.write(content)
1220+ write_bufs[filename] = buf
1221+
1222+ # eni availability checks
1223+ mocks.enter_context(
1224+ mock.patch.object(util, 'which', return_value=True))
1225+ mocks.enter_context(
1226+ mock.patch.object(eni, 'available', return_value=True))
1227+ mocks.enter_context(
1228+ mock.patch.object(util, 'ensure_dir'))
1229+ mocks.enter_context(
1230+ mock.patch.object(util, 'write_file', replace_write))
1231+ mocks.enter_context(
1232+ mock.patch.object(os.path, 'isfile', return_value=True))
1233+
1234+ ub_distro.apply_network_config(V1_NET_CFG, False)
1235+
1236+ self.assertEqual(len(write_bufs), 2)
1237+ eni_name = '/etc/network/interfaces.d/50-cloud-init.cfg'
1238+ self.assertIn(eni_name, write_bufs)
1239+ write_buf = write_bufs[eni_name]
1240+ self.assertEqual(str(write_buf).strip(), V1_NET_CFG_OUTPUT.strip())
1241+ self.assertEqual(write_buf.mode, 0o644)
1242+
1243+ def test_apply_network_config_v1_to_netplan_ub(self):
1244+ renderers = ['netplan']
1245+ ub_distro = self._get_distro('ubuntu', renderers=renderers)
1246+ with ExitStack() as mocks:
1247+ write_bufs = {}
1248+
1249+ def replace_write(filename, content, mode=0o644, omode="wb"):
1250+ buf = WriteBuffer()
1251+ buf.mode = mode
1252+ buf.omode = omode
1253+ buf.write(content)
1254+ write_bufs[filename] = buf
1255+
1256+ mocks.enter_context(
1257+ mock.patch.object(util, 'which', return_value=True))
1258+ mocks.enter_context(
1259+ mock.patch.object(util, 'write_file', replace_write))
1260+ mocks.enter_context(
1261+ mock.patch.object(util, 'ensure_dir'))
1262+ mocks.enter_context(
1263+ mock.patch.object(util, 'subp', return_value=(0, 0)))
1264+ mocks.enter_context(
1265+ mock.patch.object(os.path, 'isfile', return_value=True))
1266+
1267+ ub_distro.apply_network_config(V1_NET_CFG, False)
1268+
1269+ self.assertEqual(len(write_bufs), 1)
1270+ netplan_name = '/etc/netplan/50-cloud-init.yaml'
1271+ self.assertIn(netplan_name, write_bufs)
1272+ write_buf = write_bufs[netplan_name]
1273+ self.assertEqual(str(write_buf).strip(),
1274+ V1_TO_V2_NET_CFG_OUTPUT.strip())
1275+ self.assertEqual(write_buf.mode, 0o644)
1276+
1277+ def test_apply_network_config_v2_passthrough_ub(self):
1278+ renderers = ['netplan']
1279+ ub_distro = self._get_distro('ubuntu', renderers=renderers)
1280+ with ExitStack() as mocks:
1281+ write_bufs = {}
1282+
1283+ def replace_write(filename, content, mode=0o644, omode="wb"):
1284+ buf = WriteBuffer()
1285+ buf.mode = mode
1286+ buf.omode = omode
1287+ buf.write(content)
1288+ write_bufs[filename] = buf
1289+
1290+ mocks.enter_context(
1291+ mock.patch.object(util, 'which', return_value=True))
1292+ mocks.enter_context(
1293+ mock.patch.object(util, 'write_file', replace_write))
1294+ mocks.enter_context(
1295+ mock.patch.object(util, 'ensure_dir'))
1296+ mocks.enter_context(
1297+ mock.patch.object(util, 'subp', return_value=(0, 0)))
1298+ mocks.enter_context(
1299+ mock.patch.object(os.path, 'isfile', return_value=True))
1300+
1301+ ub_distro.apply_network_config(V2_NET_CFG, False)
1302+
1303+ self.assertEqual(len(write_bufs), 1)
1304+ netplan_name = '/etc/netplan/50-cloud-init.yaml'
1305+ self.assertIn(netplan_name, write_bufs)
1306+ write_buf = write_bufs[netplan_name]
1307+ self.assertEqual(str(write_buf).strip(),
1308+ V2_TO_V2_NET_CFG_OUTPUT.strip())
1309+ self.assertEqual(write_buf.mode, 0o644)
1310+
1311 def assertCfgEquals(self, blob1, blob2):
1312 b1 = dict(SysConf(blob1.strip().splitlines()))
1313 b2 = dict(SysConf(blob2.strip().splitlines()))
1314@@ -195,6 +393,79 @@ NETWORKING=yes
1315 self.assertCfgEquals(expected_buf, str(write_buf))
1316 self.assertEqual(write_buf.mode, 0o644)
1317
1318+ def test_apply_network_config_rh(self):
1319+ renderers = ['sysconfig']
1320+ rh_distro = self._get_distro('rhel', renderers=renderers)
1321+
1322+ write_bufs = {}
1323+
1324+ def replace_write(filename, content, mode=0o644, omode="wb"):
1325+ buf = WriteBuffer()
1326+ buf.mode = mode
1327+ buf.omode = omode
1328+ buf.write(content)
1329+ write_bufs[filename] = buf
1330+
1331+ with ExitStack() as mocks:
1332+ # sysconfig availability checks
1333+ mocks.enter_context(
1334+ mock.patch.object(util, 'which', return_value=True))
1335+ mocks.enter_context(
1336+ mock.patch.object(util, 'write_file', replace_write))
1337+ mocks.enter_context(
1338+ mock.patch.object(util, 'load_file', return_value=''))
1339+ mocks.enter_context(
1340+ mock.patch.object(os.path, 'isfile', return_value=True))
1341+
1342+ rh_distro.apply_network_config(V1_NET_CFG, False)
1343+
1344+ self.assertEqual(len(write_bufs), 5)
1345+
1346+ # eth0
1347+ self.assertIn('/etc/sysconfig/network-scripts/ifcfg-eth0',
1348+ write_bufs)
1349+ write_buf = write_bufs['/etc/sysconfig/network-scripts/ifcfg-eth0']
1350+ expected_buf = '''
1351+# Created by cloud-init on instance boot automatically, do not edit.
1352+#
1353+BOOTPROTO=static
1354+DEVICE=eth0
1355+IPADDR=192.168.1.5
1356+NETMASK=255.255.255.0
1357+NM_CONTROLLED=no
1358+ONBOOT=yes
1359+TYPE=Ethernet
1360+USERCTL=no
1361+'''
1362+ self.assertCfgEquals(expected_buf, str(write_buf))
1363+ self.assertEqual(write_buf.mode, 0o644)
1364+
1365+ # eth1
1366+ self.assertIn('/etc/sysconfig/network-scripts/ifcfg-eth1',
1367+ write_bufs)
1368+ write_buf = write_bufs['/etc/sysconfig/network-scripts/ifcfg-eth1']
1369+ expected_buf = '''
1370+# Created by cloud-init on instance boot automatically, do not edit.
1371+#
1372+BOOTPROTO=dhcp
1373+DEVICE=eth1
1374+NM_CONTROLLED=no
1375+ONBOOT=yes
1376+TYPE=Ethernet
1377+USERCTL=no
1378+'''
1379+ self.assertCfgEquals(expected_buf, str(write_buf))
1380+ self.assertEqual(write_buf.mode, 0o644)
1381+
1382+ self.assertIn('/etc/sysconfig/network', write_bufs)
1383+ write_buf = write_bufs['/etc/sysconfig/network']
1384+ expected_buf = '''
1385+# Created by cloud-init v. 0.7
1386+NETWORKING=yes
1387+'''
1388+ self.assertCfgEquals(expected_buf, str(write_buf))
1389+ self.assertEqual(write_buf.mode, 0o644)
1390+
1391 def test_write_ipv6_rhel(self):
1392 rh_distro = self._get_distro('rhel')
1393
1394@@ -213,7 +484,7 @@ NETWORKING=yes
1395 mocks.enter_context(
1396 mock.patch.object(util, 'load_file', return_value=''))
1397 mocks.enter_context(
1398- mock.patch.object(os.path, 'isfile', return_value=False))
1399+ mock.patch.object(os.path, 'isfile', return_value=True))
1400
1401 rh_distro.apply_network(BASE_NET_CFG_IPV6, False)
1402
1403@@ -274,6 +545,78 @@ IPV6_AUTOCONF=no
1404 self.assertCfgEquals(expected_buf, str(write_buf))
1405 self.assertEqual(write_buf.mode, 0o644)
1406
1407+ def test_apply_network_config_ipv6_rh(self):
1408+ renderers = ['sysconfig']
1409+ rh_distro = self._get_distro('rhel', renderers=renderers)
1410+
1411+ write_bufs = {}
1412+
1413+ def replace_write(filename, content, mode=0o644, omode="wb"):
1414+ buf = WriteBuffer()
1415+ buf.mode = mode
1416+ buf.omode = omode
1417+ buf.write(content)
1418+ write_bufs[filename] = buf
1419+
1420+ with ExitStack() as mocks:
1421+ mocks.enter_context(
1422+ mock.patch.object(util, 'which', return_value=True))
1423+ mocks.enter_context(
1424+ mock.patch.object(util, 'write_file', replace_write))
1425+ mocks.enter_context(
1426+ mock.patch.object(util, 'load_file', return_value=''))
1427+ mocks.enter_context(
1428+ mock.patch.object(os.path, 'isfile', return_value=True))
1429+
1430+ rh_distro.apply_network_config(V1_NET_CFG_IPV6, False)
1431+
1432+ self.assertEqual(len(write_bufs), 5)
1433+
1434+ self.assertIn('/etc/sysconfig/network-scripts/ifcfg-eth0',
1435+ write_bufs)
1436+ write_buf = write_bufs['/etc/sysconfig/network-scripts/ifcfg-eth0']
1437+ expected_buf = '''
1438+# Created by cloud-init on instance boot automatically, do not edit.
1439+#
1440+BOOTPROTO=static
1441+DEVICE=eth0
1442+IPV6ADDR=2607:f0d0:1002:0011::2
1443+IPV6INIT=yes
1444+NETMASK=64
1445+NM_CONTROLLED=no
1446+ONBOOT=yes
1447+TYPE=Ethernet
1448+USERCTL=no
1449+'''
1450+ self.assertCfgEquals(expected_buf, str(write_buf))
1451+ self.assertEqual(write_buf.mode, 0o644)
1452+ self.assertIn('/etc/sysconfig/network-scripts/ifcfg-eth1',
1453+ write_bufs)
1454+ write_buf = write_bufs['/etc/sysconfig/network-scripts/ifcfg-eth1']
1455+ expected_buf = '''
1456+# Created by cloud-init on instance boot automatically, do not edit.
1457+#
1458+BOOTPROTO=dhcp
1459+DEVICE=eth1
1460+NM_CONTROLLED=no
1461+ONBOOT=yes
1462+TYPE=Ethernet
1463+USERCTL=no
1464+'''
1465+ self.assertCfgEquals(expected_buf, str(write_buf))
1466+ self.assertEqual(write_buf.mode, 0o644)
1467+
1468+ self.assertIn('/etc/sysconfig/network', write_bufs)
1469+ write_buf = write_bufs['/etc/sysconfig/network']
1470+ expected_buf = '''
1471+# Created by cloud-init v. 0.7
1472+NETWORKING=yes
1473+NETWORKING_IPV6=yes
1474+IPV6_AUTOCONF=no
1475+'''
1476+ self.assertCfgEquals(expected_buf, str(write_buf))
1477+ self.assertEqual(write_buf.mode, 0o644)
1478+
1479 def test_simple_write_freebsd(self):
1480 fbsd_distro = self._get_distro('freebsd')
1481
1482diff --git a/tests/unittests/test_net.py b/tests/unittests/test_net.py
1483index 902204a..4f07d80 100644
1484--- a/tests/unittests/test_net.py
1485+++ b/tests/unittests/test_net.py
1486@@ -3,6 +3,7 @@
1487 from cloudinit import net
1488 from cloudinit.net import cmdline
1489 from cloudinit.net import eni
1490+from cloudinit.net import netplan
1491 from cloudinit.net import network_state
1492 from cloudinit.net import renderers
1493 from cloudinit.net import sysconfig
1494@@ -408,6 +409,41 @@ NETWORK_CONFIGS = {
1495 post-up route add default gw 65.61.151.37 || true
1496 pre-down route del default gw 65.61.151.37 || true
1497 """).rstrip(' '),
1498+ 'expected_netplan': textwrap.dedent("""
1499+ network:
1500+ version: 2
1501+ ethernets:
1502+ eth1:
1503+ match:
1504+ macaddress: cf:d6:af:48:e8:80
1505+ nameservers:
1506+ addresses:
1507+ - 1.2.3.4
1508+ - 5.6.7.8
1509+ search:
1510+ - wark.maas
1511+ set-name: eth1
1512+ eth99:
1513+ addresses:
1514+ - 192.168.21.3/24
1515+ dhcp4: true
1516+ match:
1517+ macaddress: c0:d6:9f:2c:e8:80
1518+ nameservers:
1519+ addresses:
1520+ - 8.8.8.8
1521+ - 8.8.4.4
1522+ - 1.2.3.4
1523+ - 5.6.7.8
1524+ search:
1525+ - barley.maas
1526+ - sach.maas
1527+ - wark.maas
1528+ routes:
1529+ - to: 0.0.0.0/0.0.0.0
1530+ via: 65.61.151.37
1531+ set-name: eth99
1532+ """).rstrip(' '),
1533 'yaml': textwrap.dedent("""
1534 version: 1
1535 config:
1536@@ -450,6 +486,14 @@ NETWORK_CONFIGS = {
1537 # control-alias iface0
1538 iface iface0 inet6 dhcp
1539 """).rstrip(' '),
1540+ 'expected_netplan': textwrap.dedent("""
1541+ network:
1542+ version: 2
1543+ ethernets:
1544+ iface0:
1545+ dhcp4: true
1546+ dhcp6: true
1547+ """).rstrip(' '),
1548 'yaml': textwrap.dedent("""\
1549 version: 1
1550 config:
1551@@ -524,6 +568,126 @@ iface eth0.101 inet static
1552 post-up route add -net 10.0.0.0 netmask 255.0.0.0 gw 11.0.0.1 metric 3 || true
1553 pre-down route del -net 10.0.0.0 netmask 255.0.0.0 gw 11.0.0.1 metric 3 || true
1554 """),
1555+ 'expected_netplan': textwrap.dedent("""
1556+ network:
1557+ version: 2
1558+ ethernets:
1559+ eth0:
1560+ match:
1561+ macaddress: c0:d6:9f:2c:e8:80
1562+ nameservers:
1563+ addresses:
1564+ - 8.8.8.8
1565+ - 4.4.4.4
1566+ - 8.8.4.4
1567+ search:
1568+ - barley.maas
1569+ - wark.maas
1570+ - foobar.maas
1571+ set-name: eth0
1572+ eth1:
1573+ match:
1574+ macaddress: aa:d6:9f:2c:e8:80
1575+ nameservers:
1576+ addresses:
1577+ - 8.8.8.8
1578+ - 4.4.4.4
1579+ - 8.8.4.4
1580+ search:
1581+ - barley.maas
1582+ - wark.maas
1583+ - foobar.maas
1584+ set-name: eth1
1585+ eth2:
1586+ match:
1587+ macaddress: c0:bb:9f:2c:e8:80
1588+ nameservers:
1589+ addresses:
1590+ - 8.8.8.8
1591+ - 4.4.4.4
1592+ - 8.8.4.4
1593+ search:
1594+ - barley.maas
1595+ - wark.maas
1596+ - foobar.maas
1597+ set-name: eth2
1598+ eth3:
1599+ match:
1600+ macaddress: 66:bb:9f:2c:e8:80
1601+ nameservers:
1602+ addresses:
1603+ - 8.8.8.8
1604+ - 4.4.4.4
1605+ - 8.8.4.4
1606+ search:
1607+ - barley.maas
1608+ - wark.maas
1609+ - foobar.maas
1610+ set-name: eth3
1611+ eth4:
1612+ match:
1613+ macaddress: 98:bb:9f:2c:e8:80
1614+ nameservers:
1615+ addresses:
1616+ - 8.8.8.8
1617+ - 4.4.4.4
1618+ - 8.8.4.4
1619+ search:
1620+ - barley.maas
1621+ - wark.maas
1622+ - foobar.maas
1623+ set-name: eth4
1624+ eth5:
1625+ dhcp4: true
1626+ match:
1627+ macaddress: 98:bb:9f:2c:e8:8a
1628+ nameservers:
1629+ addresses:
1630+ - 8.8.8.8
1631+ - 4.4.4.4
1632+ - 8.8.4.4
1633+ search:
1634+ - barley.maas
1635+ - wark.maas
1636+ - foobar.maas
1637+ set-name: eth5
1638+ bonds:
1639+ bond0:
1640+ dhcp6: true
1641+ interfaces:
1642+ - eth1
1643+ - eth2
1644+ parameters:
1645+ mode: active-backup
1646+ bridges:
1647+ br0:
1648+ addresses:
1649+ - 192.168.14.2/24
1650+ - 2001:1::1/64
1651+ interfaces:
1652+ - eth3
1653+ - eth4
1654+ vlans:
1655+ bond0.200:
1656+ dhcp4: true
1657+ id: 200
1658+ link: bond0
1659+ eth0.101:
1660+ addresses:
1661+ - 192.168.0.2/24
1662+ - 192.168.2.10/24
1663+ gateway4: 192.168.0.1
1664+ id: 101
1665+ link: eth0
1666+ nameservers:
1667+ addresses:
1668+ - 192.168.0.10
1669+ - 10.23.23.134
1670+ search:
1671+ - barley.maas
1672+ - sacchromyces.maas
1673+ - brettanomyces.maas
1674+ """).rstrip(' '),
1675 'yaml': textwrap.dedent("""
1676 version: 1
1677 config:
1678@@ -808,6 +972,99 @@ iface eth0 inet dhcp
1679 expected, dir2dict(tmp_dir)['/etc/network/interfaces'])
1680
1681
1682+class TestNetplanNetRendering(CiTestCase):
1683+
1684+ @mock.patch("cloudinit.net.sys_dev_path")
1685+ @mock.patch("cloudinit.net.read_sys_net")
1686+ @mock.patch("cloudinit.net.get_devicelist")
1687+ def test_default_generation(self, mock_get_devicelist,
1688+ mock_read_sys_net,
1689+ mock_sys_dev_path):
1690+ tmp_dir = self.tmp_dir()
1691+ _setup_test(tmp_dir, mock_get_devicelist,
1692+ mock_read_sys_net, mock_sys_dev_path)
1693+
1694+ network_cfg = net.generate_fallback_config()
1695+ ns = network_state.parse_net_config_data(network_cfg,
1696+ skip_broken=False)
1697+
1698+ render_dir = os.path.join(tmp_dir, "render")
1699+ os.makedirs(render_dir)
1700+
1701+ render_target = 'netplan.yaml'
1702+ renderer = netplan.Renderer(
1703+ {'netplan_path': render_target, 'postcmds': False})
1704+ renderer.render_network_state(render_dir, ns)
1705+
1706+ self.assertTrue(os.path.exists(os.path.join(render_dir,
1707+ render_target)))
1708+ with open(os.path.join(render_dir, render_target)) as fh:
1709+ contents = fh.read()
1710+ print(contents)
1711+
1712+ expected = """
1713+network:
1714+ version: 2
1715+ ethernets:
1716+ eth1000:
1717+ dhcp4: true
1718+ match:
1719+ macaddress: 07-1c-c6-75-a4-be
1720+ set-name: eth1000
1721+"""
1722+ self.assertEqual(expected.lstrip(), contents.lstrip())
1723+
1724+
1725+class TestNetplanPostcommands(CiTestCase):
1726+ mycfg = {
1727+ 'config': [{"type": "physical", "name": "eth0",
1728+ "mac_address": "c0:d6:9f:2c:e8:80",
1729+ "subnets": [{"type": "dhcp"}]}],
1730+ 'version': 1}
1731+
1732+ @mock.patch.object(netplan.Renderer, '_netplan_generate')
1733+ @mock.patch.object(netplan.Renderer, '_net_setup_link')
1734+ def test_netplan_render_calls_postcmds(self, mock_netplan_generate,
1735+ mock_net_setup_link):
1736+ tmp_dir = self.tmp_dir()
1737+ ns = network_state.parse_net_config_data(self.mycfg,
1738+ skip_broken=False)
1739+
1740+ render_dir = os.path.join(tmp_dir, "render")
1741+ os.makedirs(render_dir)
1742+
1743+ render_target = 'netplan.yaml'
1744+ renderer = netplan.Renderer(
1745+ {'netplan_path': render_target, 'postcmds': True})
1746+ renderer.render_network_state(render_dir, ns)
1747+
1748+ mock_netplan_generate.assert_called_with(run=True)
1749+ mock_net_setup_link.assert_called_with(run=True)
1750+
1751+ @mock.patch.object(netplan, "get_devicelist")
1752+ @mock.patch('cloudinit.util.subp')
1753+ def test_netplan_postcmds(self, mock_subp, mock_devlist):
1754+ mock_devlist.side_effect = [['lo']]
1755+ tmp_dir = self.tmp_dir()
1756+ ns = network_state.parse_net_config_data(self.mycfg,
1757+ skip_broken=False)
1758+
1759+ render_dir = os.path.join(tmp_dir, "render")
1760+ os.makedirs(render_dir)
1761+
1762+ render_target = 'netplan.yaml'
1763+ renderer = netplan.Renderer(
1764+ {'netplan_path': render_target, 'postcmds': True})
1765+ renderer.render_network_state(render_dir, ns)
1766+
1767+ expected = [
1768+ mock.call(['netplan', 'generate'], capture=True),
1769+ mock.call(['udevadm', 'test-builtin', 'net_setup_link',
1770+ '/sys/class/net/lo'], capture=True),
1771+ ]
1772+ mock_subp.assert_has_calls(expected)
1773+
1774+
1775 class TestEniNetworkStateToEni(CiTestCase):
1776 mycfg = {
1777 'config': [{"type": "physical", "name": "eth0",
1778@@ -953,6 +1210,50 @@ class TestCmdlineReadKernelConfig(CiTestCase):
1779 self.assertEqual(found['config'], expected)
1780
1781
1782+class TestNetplanRoundTrip(CiTestCase):
1783+ def _render_and_read(self, network_config=None, state=None,
1784+ netplan_path=None, dir=None):
1785+ if dir is None:
1786+ dir = self.tmp_dir()
1787+
1788+ if network_config:
1789+ ns = network_state.parse_net_config_data(network_config)
1790+ elif state:
1791+ ns = state
1792+ else:
1793+ raise ValueError("Expected data or state, got neither")
1794+
1795+ if netplan_path is None:
1796+ netplan_path = 'etc/netplan/50-cloud-init.yaml'
1797+
1798+ renderer = netplan.Renderer(
1799+ config={'netplan_path': netplan_path})
1800+
1801+ renderer.render_network_state(dir, ns)
1802+ return dir2dict(dir)
1803+
1804+ def testsimple_render_small_netplan(self):
1805+ entry = NETWORK_CONFIGS['small']
1806+ files = self._render_and_read(network_config=yaml.load(entry['yaml']))
1807+ self.assertEqual(
1808+ entry['expected_netplan'].splitlines(),
1809+ files['/etc/netplan/50-cloud-init.yaml'].splitlines())
1810+
1811+ def testsimple_render_v4_and_v6(self):
1812+ entry = NETWORK_CONFIGS['v4_and_v6']
1813+ files = self._render_and_read(network_config=yaml.load(entry['yaml']))
1814+ self.assertEqual(
1815+ entry['expected_netplan'].splitlines(),
1816+ files['/etc/netplan/50-cloud-init.yaml'].splitlines())
1817+
1818+ def testsimple_render_all(self):
1819+ entry = NETWORK_CONFIGS['all']
1820+ files = self._render_and_read(network_config=yaml.load(entry['yaml']))
1821+ self.assertEqual(
1822+ entry['expected_netplan'].splitlines(),
1823+ files['/etc/netplan/50-cloud-init.yaml'].splitlines())
1824+
1825+
1826 class TestEniRoundTrip(CiTestCase):
1827 def _render_and_read(self, network_config=None, state=None, eni_path=None,
1828 links_prefix=None, netrules_path=None, dir=None):
1829diff --git a/tools/net-convert.py b/tools/net-convert.py
1830new file mode 100755
1831index 0000000..1424bb0
1832--- /dev/null
1833+++ b/tools/net-convert.py
1834@@ -0,0 +1,83 @@
1835+#!/usr/bin/python3
1836+#
1837+# This file is part of cloud-init. See LICENSE file ...
1838+
1839+import argparse
1840+import json
1841+import os
1842+import yaml
1843+
1844+from cloudinit.sources.helpers import openstack
1845+
1846+from cloudinit.net import eni
1847+from cloudinit.net import network_state
1848+from cloudinit.net import netplan
1849+from cloudinit.net import sysconfig
1850+
1851+
1852+def main():
1853+ parser = argparse.ArgumentParser()
1854+ parser.add_argument("--network-data", "-p", type=open,
1855+ metavar="PATH", required=True)
1856+ parser.add_argument("--kind", "-k",
1857+ choices=['eni', 'network_data.json', 'yaml'],
1858+ required=True)
1859+ parser.add_argument("-d", "--directory",
1860+ metavar="PATH",
1861+ help="directory to place output in",
1862+ required=True)
1863+ parser.add_argument("-m", "--mac",
1864+ metavar="name,mac",
1865+ action='append',
1866+ help="interface name to mac mapping")
1867+ parser.add_argument("--output-kind", "-ok",
1868+ choices=['eni', 'netplan', 'sysconfig'],
1869+ required=True)
1870+ args = parser.parse_args()
1871+
1872+ if not os.path.isdir(args.directory):
1873+ os.makedirs(args.directory)
1874+
1875+ if args.mac:
1876+ known_macs = {}
1877+ for item in args.mac:
1878+ iface_name, iface_mac = item.split(",", 1)
1879+ known_macs[iface_mac] = iface_name
1880+ else:
1881+ known_macs = None
1882+
1883+ net_data = args.network_data.read()
1884+ if args.kind == "eni":
1885+ pre_ns = eni.convert_eni_data(net_data)
1886+ ns = network_state.parse_net_config_data(pre_ns)
1887+ elif args.kind == "yaml":
1888+ pre_ns = yaml.load(net_data)
1889+ if 'network' in pre_ns:
1890+ pre_ns = pre_ns.get('network')
1891+ print("Input YAML")
1892+ print(yaml.dump(pre_ns, default_flow_style=False, indent=4))
1893+ ns = network_state.parse_net_config_data(pre_ns)
1894+ else:
1895+ pre_ns = openstack.convert_net_json(
1896+ json.loads(net_data), known_macs=known_macs)
1897+ ns = network_state.parse_net_config_data(pre_ns)
1898+
1899+ if not ns:
1900+ raise RuntimeError("No valid network_state object created from"
1901+ "input data")
1902+
1903+ print("\nInternal State")
1904+ print(yaml.dump(ns, default_flow_style=False, indent=4))
1905+ if args.output_kind == "eni":
1906+ r_cls = eni.Renderer
1907+ elif args.output_kind == "netplan":
1908+ r_cls = netplan.Renderer
1909+ else:
1910+ r_cls = sysconfig.Renderer
1911+
1912+ r = r_cls()
1913+ r.render_network_state(ns, target=args.directory)
1914+
1915+
1916+if __name__ == '__main__':
1917+ main()

Subscribers

People subscribed via source and target branches