cloud-init

Merge ~raharper/cloud-init:rebased-netconfig-v2-passthrough into cloud-init:master

  1. Git
  2. lp:~raharper/cloud-init
  3. rebased-netconfig-v2-passthrough
  4. Merge into master
Proposed by Ryan Harper on 2017-03-19
Status: Merged
Merged at revision: ef18b8ac4cf7e3dfd98830fbdb298380a192a0fc
Proposed branch: ~raharper/cloud-init:rebased-netconfig-v2-passthrough
Merge into: cloud-init:master
Diff against target: 1811 lines (+1443/-36)
10 files modified
cloudinit/distros/debian.py (+11/-5)
cloudinit/net/eni.py (+3/-12)
cloudinit/net/netplan.py (+380/-0)
cloudinit/net/network_state.py (+299/-9)
cloudinit/net/renderers.py (+3/-1)
cloudinit/net/sysconfig.py (+15/-5)
systemd/cloud-init.service (+1/-0)
tests/unittests/test_distros/test_netconfig.py (+347/-4)
tests/unittests/test_net.py (+301/-0)
tools/net-convert.py (+83/-0)
Reviewer Review Type Date Requested Status
Server Team CI bot continuous-integration Needs Fixing on 2017-03-20
cloud-init commiters 2017-03-19 Pending
Review via email: mp+320291@code.launchpad.net

Description of the Change

cloudinit.net: add network config v2 parsing and rendering

Network configuration version 2 format is implemented in a package
called netplan (nplan)[1] which allows consolidated network config
for multiple network controllers.

- Add a new netplan renderer
- Update default policy, placing eni and sysconfig first
  This requires explicit policy to enable netplan over eni
  on systems which have both (Yakkety, Zesty, UC16)
- Allow any network state (parsed from any format cloud-init supports) to
  render to v2 if system supports netplan.
- Move eni's _subnet_is_ipv6 to common code for use by other renderers
- Make sysconfig renderer always emit /etc/syconfig/network configuration
- Update cloud-init.service systemd unit to also wait on systemd-networkd-wait-online.service

1. https://lists.ubuntu.com/archives/ubuntu-devel/2016-July/039464.html

To post a comment you must log in.
~raharper/cloud-init:rebased-netconfig-v2-passthrough updated on 2017-03-19
f006666... by Ryan Harper on 2017-03-19

(squash): eni.py whitespace damage

Server Team CI bot (server-team-bot) wrote :
review: Needs Fixing (continuous-integration)
Server Team CI bot (server-team-bot) wrote :
review: Needs Fixing (continuous-integration)
Scott Moser (smoser) wrote :

Ryan, this looks good, thanks.

I'll take a look first thing monday morning on it.

Scott Moser (smoser) wrote :

i have some small comments.
don't worry about fixing the 'handle_' comment, but i do worry about the confusion.

Ryan Harper (raharper) :
~raharper/cloud-init:rebased-netconfig-v2-passthrough updated on 2017-03-20
3ddea40... by Ryan Harper on 2017-03-20

unittests/test_distro/test_netconfig: pass 'system_info' dict into Distro class

2b63ec0... by Ryan Harper on 2017-03-20

distros/debian.py: update comment in _write_network method

Ryan Harper (raharper) wrote :

I'm mostly done fixing up the small issues; one larger discussion to be had on what to do about v2 passthrough and if we get v2 but need to render eni or sysconfig.

~raharper/cloud-init:rebased-netconfig-v2-passthrough updated on 2017-03-20
4ba0163... by Ryan Harper on 2017-03-20

net.sysconfig.py: Use _make_header() when writing sysconfig/network

9bcc80e... by Ryan Harper on 2017-03-20

Disable v2 passthrough, instead parse so we can render network config cross distro

Server Team CI bot (server-team-bot) wrote :
review: Needs Fixing (continuous-integration)
~raharper/cloud-init:rebased-netconfig-v2-passthrough updated on 2017-03-20
9a10ab5... by Ryan Harper on 2017-03-20

network_state: v2 don't warn/error on missing mac_address in v2 input

1c1fc3a... by Ryan Harper on 2017-03-20

netplan: drop textwrap indent module, and reformat some comments/docstrings

Server Team CI bot (server-team-bot) wrote :
review: Needs Fixing (continuous-integration)

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1diff --git a/cloudinit/distros/debian.py b/cloudinit/distros/debian.py
2index 1101f02..3f0f9d5 100644
3--- a/cloudinit/distros/debian.py
4+++ b/cloudinit/distros/debian.py
5@@ -42,11 +42,16 @@ NETWORK_CONF_FN = "/etc/network/interfaces.d/50-cloud-init.cfg"
6 class Distro(distros.Distro):
7 hostname_conf_fn = "/etc/hostname"
8 locale_conf_fn = "/etc/default/locale"
9+ network_conf_fn = {
10+ "eni": "/etc/network/interfaces.d/50-cloud-init.cfg",
11+ "netplan": "/etc/netplan/50-cloud-init.yaml"
12+ }
13 renderer_configs = {
14- 'eni': {
15- 'eni_path': NETWORK_CONF_FN,
16- 'eni_header': ENI_HEADER,
17- }
18+ "eni": {"eni_path": network_conf_fn["eni"],
19+ "eni_header": ENI_HEADER},
20+ "netplan": {"netplan_path": network_conf_fn["netplan"],
21+ "netplan_header": ENI_HEADER,
22+ "postcmds": True}
23 }
24
25 def __init__(self, name, cfg, paths):
26@@ -75,7 +80,8 @@ class Distro(distros.Distro):
27 self.package_command('install', pkgs=pkglist)
28
29 def _write_network(self, settings):
30- util.write_file(NETWORK_CONF_FN, settings)
31+ # this is a legacy method, it will always write eni
32+ util.write_file(self.network_conf_fn["eni"], settings)
33 return ['all']
34
35 def _write_network_config(self, netconfig):
36diff --git a/cloudinit/net/eni.py b/cloudinit/net/eni.py
37index f471e05..9819d4f 100644
38--- a/cloudinit/net/eni.py
39+++ b/cloudinit/net/eni.py
40@@ -8,6 +8,7 @@ import re
41 from . import ParserError
42
43 from . import renderer
44+from .network_state import subnet_is_ipv6
45
46 from cloudinit import util
47
48@@ -111,16 +112,6 @@ def _iface_start_entry(iface, index, render_hwaddress=False):
49 return lines
50
51
52-def _subnet_is_ipv6(subnet):
53- # 'static6' or 'dhcp6'
54- if subnet['type'].endswith('6'):
55- # This is a request for DHCPv6.
56- return True
57- elif subnet['type'] == 'static' and ":" in subnet['address']:
58- return True
59- return False
60-
61-
62 def _parse_deb_config_data(ifaces, contents, src_dir, src_path):
63 """Parses the file contents, placing result into ifaces.
64
65@@ -370,7 +361,7 @@ class Renderer(renderer.Renderer):
66 iface['mode'] = subnet['type']
67 iface['control'] = subnet.get('control', 'auto')
68 subnet_inet = 'inet'
69- if _subnet_is_ipv6(subnet):
70+ if subnet_is_ipv6(subnet):
71 subnet_inet += '6'
72 iface['inet'] = subnet_inet
73 if subnet['type'].startswith('dhcp'):
74@@ -486,7 +477,7 @@ class Renderer(renderer.Renderer):
75 def network_state_to_eni(network_state, header=None, render_hwaddress=False):
76 # render the provided network state, return a string of equivalent eni
77 eni_path = 'etc/network/interfaces'
78- renderer = Renderer({
79+ renderer = Renderer(config={
80 'eni_path': eni_path,
81 'eni_header': header,
82 'links_path_prefix': None,
83diff --git a/cloudinit/net/netplan.py b/cloudinit/net/netplan.py
84new file mode 100644
85index 0000000..7b3a6e8
86--- /dev/null
87+++ b/cloudinit/net/netplan.py
88@@ -0,0 +1,380 @@
89+# This file is part of cloud-init. See LICENSE file ...
90+
91+import copy
92+import os
93+
94+from . import renderer
95+from .network_state import subnet_is_ipv6
96+
97+from cloudinit import util
98+from cloudinit.net import SYS_CLASS_NET, get_devicelist
99+
100+
101+NET_CONFIG_TO_V2 = {
102+ 'bond': {'bond-ad-select': 'ad-select',
103+ 'bond-arp-interval': 'arp-interval',
104+ 'bond-arp-ip-target': 'arp-ip-target',
105+ 'bond-arp-validate': 'arp-validate',
106+ 'bond-downdelay': 'down-delay',
107+ 'bond-fail-over-mac': 'fail-over-mac-policy',
108+ 'bond-lacp-rate': 'lacp-rate',
109+ 'bond-miimon': 'mii-monitor-interval',
110+ 'bond-min-links': 'min-links',
111+ 'bond-mode': 'mode',
112+ 'bond-num-grat-arp': 'gratuitious-arp',
113+ 'bond-primary-reselect': 'primary-reselect-policy',
114+ 'bond-updelay': 'up-delay',
115+ 'bond-xmit_hash_policy': 'transmit_hash_policy'},
116+ 'bridge': {'bridge_ageing': 'ageing-time',
117+ 'bridge_bridgeprio': 'priority',
118+ 'bridge_fd': 'forward-delay',
119+ 'bridge_gcint': None,
120+ 'bridge_hello': 'hello-time',
121+ 'bridge_maxage': 'max-age',
122+ 'bridge_maxwait': None,
123+ 'bridge_pathcost': 'path-cost',
124+ 'bridge_portprio': None,
125+ 'bridge_waitport': None}}
126+
127+
128+def indent(text, prefix):
129+ lines = []
130+ for line in text.splitlines(True):
131+ lines.append(prefix + line)
132+ return ''.join(lines)
133+
134+
135+def _get_params_dict_by_match(config, match):
136+ return dict((key, value) for (key, value) in config.items()
137+ if key.startswith(match))
138+
139+
140+def _extract_addresses(config, entry):
141+ """This method parse a cloudinit.net.network_state dictionary (config) and
142+ maps netstate keys/values into a dictionary (entry) to represent
143+ netplan yaml.
144+
145+ An example config dictionary might look like:
146+
147+ {'mac_address': '52:54:00:12:34:00',
148+ 'name': 'interface0',
149+ 'subnets': [
150+ {'address': '192.168.1.2/24',
151+ 'mtu': 1501,
152+ 'type': 'static'},
153+ {'address': '2001:4800:78ff:1b:be76:4eff:fe06:1000",
154+ 'mtu': 1480,
155+ 'netmask': 64,
156+ 'type': 'static'}],
157+ 'type: physical'
158+ }
159+
160+ An entry dictionary looks like:
161+
162+ {'set-name': 'interface0',
163+ 'match': {'macaddress': '52:54:00:12:34:00'},
164+ 'mtu': 1501}
165+
166+ After modification returns
167+
168+ {'set-name': 'interface0',
169+ 'match': {'macaddress': '52:54:00:12:34:00'},
170+ 'mtu': 1501,
171+ 'address': ['192.168.1.2/24', '2001:4800:78ff:1b:be76:4eff:fe06:1000"],
172+ 'mtu6': 1480}
173+
174+ """
175+
176+ def _listify(obj, token=' '):
177+ "Helper to convert strings to list of strings, handle single string"
178+ if not obj or type(obj) not in [str]:
179+ return obj
180+ if token in obj:
181+ return obj.split(token)
182+ else:
183+ return [obj, ]
184+
185+ addresses = []
186+ routes = []
187+ nameservers = []
188+ searchdomains = []
189+ subnets = config.get('subnets', [])
190+ if subnets is None:
191+ subnets = []
192+ for subnet in subnets:
193+ sn_type = subnet.get('type')
194+ if sn_type.startswith('dhcp'):
195+ if sn_type == 'dhcp':
196+ sn_type += '4'
197+ entry.update({sn_type: True})
198+ elif sn_type in ['static']:
199+ addr = "%s" % subnet.get('address')
200+ if 'netmask' in subnet:
201+ addr += "/%s" % subnet.get('netmask')
202+ if 'gateway' in subnet and subnet.get('gateway'):
203+ gateway = subnet.get('gateway')
204+ if ":" in gateway:
205+ entry.update({'gateway6': gateway})
206+ else:
207+ entry.update({'gateway4': gateway})
208+ if 'dns_nameservers' in subnet:
209+ nameservers += _listify(subnet.get('dns_nameservers', []))
210+ if 'dns_search' in subnet:
211+ searchdomains += _listify(subnet.get('dns_search', []))
212+ if 'mtu' in subnet:
213+ mtukey = 'mtu'
214+ if subnet_is_ipv6(subnet):
215+ mtukey += '6'
216+ entry.update({mtukey: subnet.get('mtu')})
217+ for route in subnet.get('routes', []):
218+ to_net = "%s/%s" % (route.get('network'),
219+ route.get('netmask'))
220+ route = {
221+ 'via': route.get('gateway'),
222+ 'to': to_net,
223+ }
224+ if 'metric' in route:
225+ route.update({'metric': route.get('metric', 100)})
226+ routes.append(route)
227+
228+ addresses.append(addr)
229+
230+ if len(addresses) > 0:
231+ entry.update({'addresses': addresses})
232+ if len(routes) > 0:
233+ entry.update({'routes': routes})
234+ if len(nameservers) > 0:
235+ ns = {'addresses': nameservers}
236+ entry.update({'nameservers': ns})
237+ if len(searchdomains) > 0:
238+ ns = entry.get('nameservers', {})
239+ ns.update({'search': searchdomains})
240+ entry.update({'nameservers': ns})
241+
242+
243+def _extract_bond_slaves_by_name(interfaces, entry, bond_master):
244+ bond_slave_names = sorted([name for (name, cfg) in interfaces.items()
245+ if cfg.get('bond-master', None) == bond_master])
246+ if len(bond_slave_names) > 0:
247+ entry.update({'interfaces': bond_slave_names})
248+
249+
250+class Renderer(renderer.Renderer):
251+ """Renders network information in a /etc/netplan/network.yaml format."""
252+
253+ NETPLAN_GENERATE = ['netplan', 'generate']
254+
255+ def __init__(self, config=None):
256+ if not config:
257+ config = {}
258+ self.netplan_path = config.get('netplan_path',
259+ 'etc/netplan/50-cloud-init.yaml')
260+ self.netplan_header = config.get('netplan_header', None)
261+ self._postcmds = config.get('postcmds', False)
262+
263+ def render_network_state(self, target, network_state):
264+ # check network state for version
265+ # if v2, then extract network_state.config
266+ # else render_v2_from_state
267+ fpnplan = os.path.join(target, self.netplan_path)
268+ util.ensure_dir(os.path.dirname(fpnplan))
269+ header = self.netplan_header if self.netplan_header else ""
270+
271+ # render from state
272+ content = self._render_content(network_state)
273+
274+ # ensure we poke udev to run net_setup_link
275+ if not header.endswith("\n"):
276+ header += "\n"
277+ util.write_file(fpnplan, header + content)
278+
279+ self._netplan_generate(run=self._postcmds)
280+ self._net_setup_link(run=self._postcmds)
281+
282+ def _netplan_generate(self, run=False):
283+ if not run:
284+ print("netplan postcmd disabled")
285+ return
286+ util.subp(self.NETPLAN_GENERATE, capture=True)
287+
288+ def _net_setup_link(self, run=False):
289+ """To ensure device link properties are applied, we poke
290+ udev to re-evaluate networkd .link files and call
291+ the setup_link udev builtin command
292+ """
293+ if not run:
294+ print("netsetup postcmd disabled")
295+ return
296+ setup_lnk = ['udevadm', 'test-builtin', 'net_setup_link']
297+ for cmd in [setup_lnk + [SYS_CLASS_NET + iface]
298+ for iface in get_devicelist() if
299+ os.path.islink(SYS_CLASS_NET + iface)]:
300+ print(cmd)
301+ util.subp(cmd, capture=True)
302+
303+ def _render_content(self, network_state):
304+ print('rendering v2 for victory!')
305+ ethernets = {}
306+ wifis = {}
307+ bridges = {}
308+ bonds = {}
309+ vlans = {}
310+ content = []
311+
312+ interfaces = network_state._network_state.get('interfaces', [])
313+
314+ nameservers = network_state.dns_nameservers
315+ searchdomains = network_state.dns_searchdomains
316+
317+ for config in network_state.iter_interfaces():
318+ ifname = config.get('name')
319+ # filter None entries up front so we can do simple if key in dict
320+ ifcfg = dict((key, value) for (key, value) in config.items()
321+ if value)
322+
323+ if_type = ifcfg.get('type')
324+ if if_type == 'physical':
325+ # required_keys = ['name', 'mac_address']
326+ eth = {
327+ 'set-name': ifname,
328+ 'match': ifcfg.get('match', None),
329+ }
330+ if eth['match'] is None:
331+ macaddr = ifcfg.get('mac_address', None)
332+ if macaddr is not None:
333+ eth['match'] = {'macaddress': macaddr.lower()}
334+ else:
335+ del eth['match']
336+ del eth['set-name']
337+ if 'mtu' in ifcfg:
338+ eth['mtu'] = ifcfg.get('mtu')
339+
340+ _extract_addresses(ifcfg, eth)
341+ ethernets.update({ifname: eth})
342+
343+ elif if_type == 'bond':
344+ # required_keys = ['name', 'bond_interfaces']
345+ bond = {}
346+ bond_config = {}
347+ # extract bond params and drop the bond_ prefix as it's
348+ # redundent in v2 yaml format
349+ v2_bond_map = NET_CONFIG_TO_V2.get('bond')
350+ for match in ['bond_', 'bond-']:
351+ bond_params = _get_params_dict_by_match(ifcfg, match)
352+ for (param, value) in bond_params.items():
353+ newname = v2_bond_map.get(param)
354+ if newname is None:
355+ continue
356+ bond_config.update({newname: value})
357+
358+ if len(bond_config) > 0:
359+ bond.update({'parameters': bond_config})
360+ slave_interfaces = ifcfg.get('bond-slaves')
361+ if slave_interfaces == 'none':
362+ _extract_bond_slaves_by_name(interfaces, bond, ifname)
363+ _extract_addresses(ifcfg, bond)
364+ bonds.update({ifname: bond})
365+
366+ elif if_type == 'bridge':
367+ # required_keys = ['name', 'bridge_ports']
368+ ports = sorted(copy.copy(ifcfg.get('bridge_ports')))
369+ bridge = {
370+ 'interfaces': ports,
371+ }
372+ # extract bridge params and drop the bridge prefix as it's
373+ # redundent in v2 yaml format
374+ match_prefix = 'bridge_'
375+ params = _get_params_dict_by_match(ifcfg, match_prefix)
376+ br_config = {}
377+
378+ # v2 yaml uses different names for the keys
379+ # and at least one value format change
380+ v2_bridge_map = NET_CONFIG_TO_V2.get('bridge')
381+ for (param, value) in params.items():
382+ newname = v2_bridge_map.get(param)
383+ if newname is None:
384+ continue
385+ br_config.update({newname: value})
386+ if newname == 'path-cost':
387+ # <interface> <cost> -> <interface>: int(<cost>)
388+ newvalue = {}
389+ for costval in value:
390+ (port, cost) = costval.split()
391+ newvalue[port] = int(cost)
392+ br_config.update({newname: newvalue})
393+ if len(br_config) > 0:
394+ bridge.update({'parameters': br_config})
395+ _extract_addresses(ifcfg, bridge)
396+ bridges.update({ifname: bridge})
397+
398+ elif if_type == 'vlan':
399+ # required_keys = ['name', 'vlan_id', 'vlan-raw-device']
400+ vlan = {
401+ 'id': ifcfg.get('vlan_id'),
402+ 'link': ifcfg.get('vlan-raw-device')
403+ }
404+
405+ _extract_addresses(ifcfg, vlan)
406+ vlans.update({ifname: vlan})
407+
408+ # inject global nameserver values under each physical interface
409+ if nameservers:
410+ for _eth, cfg in ethernets.items():
411+ nscfg = cfg.get('nameservers', {})
412+ addresses = nscfg.get('addresses', [])
413+ addresses += nameservers
414+ nscfg.update({'addresses': addresses})
415+ cfg.update({'nameservers': nscfg})
416+
417+ if searchdomains:
418+ for _eth, cfg in ethernets.items():
419+ nscfg = cfg.get('nameservers', {})
420+ search = nscfg.get('search', [])
421+ search += searchdomains
422+ nscfg.update({'search': search})
423+ cfg.update({'nameservers': nscfg})
424+
425+ # workaround yaml dictionary key sorting when dumping
426+ def _render_section(name, section):
427+ if section:
428+ dump = util.yaml_dumps({name: section},
429+ explicit_start=False,
430+ explicit_end=False)
431+ txt = indent(dump, ' ' * 4)
432+ return [txt]
433+ return []
434+
435+ content.append("network:\n version: 2\n")
436+ content += _render_section('ethernets', ethernets)
437+ content += _render_section('wifis', wifis)
438+ content += _render_section('bonds', bonds)
439+ content += _render_section('bridges', bridges)
440+ content += _render_section('vlans', vlans)
441+
442+ return "".join(content)
443+
444+
445+def available(target=None):
446+ expected = ['netplan']
447+ search = ['/usr/sbin', '/sbin']
448+ for p in expected:
449+ if not util.which(p, search=search, target=target):
450+ return False
451+ return True
452+
453+
454+def network_state_to_netplan(network_state, header=None):
455+ # render the provided network state, return a string of equivalent eni
456+ netplan_path = 'etc/network/50-cloud-init.yaml'
457+ renderer = Renderer({
458+ 'netplan_path': netplan_path,
459+ 'netplan_header': header,
460+ })
461+ if not header:
462+ header = ""
463+ if not header.endswith("\n"):
464+ header += "\n"
465+ contents = renderer._render_content(network_state)
466+ return header + contents
467+
468+# vi: ts=4 expandtab
469diff --git a/cloudinit/net/network_state.py b/cloudinit/net/network_state.py
470index 90b2835..c14aae2 100644
471--- a/cloudinit/net/network_state.py
472+++ b/cloudinit/net/network_state.py
473@@ -1,4 +1,4 @@
474-# Copyright (C) 2013-2014 Canonical Ltd.
475+# Copyright (C) 2017 Canonical Ltd.
476 #
477 # Author: Ryan Harper <ryan.harper@canonical.com>
478 #
479@@ -18,6 +18,10 @@ NETWORK_STATE_VERSION = 1
480 NETWORK_STATE_REQUIRED_KEYS = {
481 1: ['version', 'config', 'network_state'],
482 }
483+NETWORK_V2_KEY_FILTER = [
484+ 'addresses', 'dhcp4', 'dhcp6', 'gateway4', 'gateway6', 'interfaces',
485+ 'match', 'mtu', 'nameservers', 'renderer', 'set-name', 'wakeonlan'
486+]
487
488
489 def parse_net_config_data(net_config, skip_broken=True):
490@@ -26,11 +30,18 @@ def parse_net_config_data(net_config, skip_broken=True):
491 :param net_config: curtin network config dict
492 """
493 state = None
494- if 'version' in net_config and 'config' in net_config:
495- nsi = NetworkStateInterpreter(version=net_config.get('version'),
496- config=net_config.get('config'))
497+ version = net_config.get('version')
498+ config = net_config.get('config')
499+ if version == 2:
500+ # v2 does not have explicit 'config' key so we
501+ # pass the whole net-config as-is
502+ config = net_config
503+
504+ if version and config:
505+ nsi = NetworkStateInterpreter(version=version, config=config)
506 nsi.parse_config(skip_broken=skip_broken)
507- state = nsi.network_state
508+ state = nsi.get_network_state()
509+
510 return state
511
512
513@@ -106,6 +117,7 @@ class NetworkState(object):
514 def __init__(self, network_state, version=NETWORK_STATE_VERSION):
515 self._network_state = copy.deepcopy(network_state)
516 self._version = version
517+ self.use_ipv6 = network_state.get('use_ipv6', False)
518
519 @property
520 def version(self):
521@@ -152,7 +164,8 @@ class NetworkStateInterpreter(object):
522 'dns': {
523 'nameservers': [],
524 'search': [],
525- }
526+ },
527+ 'use_ipv6': False,
528 }
529
530 def __init__(self, version=NETWORK_STATE_VERSION, config=None):
531@@ -165,6 +178,14 @@ class NetworkStateInterpreter(object):
532 def network_state(self):
533 return NetworkState(self._network_state, version=self._version)
534
535+ @property
536+ def use_ipv6(self):
537+ return self._network_state.get('use_ipv6')
538+
539+ @use_ipv6.setter
540+ def use_ipv6(self, val):
541+ self._network_state.update({'use_ipv6': val})
542+
543 def dump(self):
544 state = {
545 'version': self._version,
546@@ -192,8 +213,22 @@ class NetworkStateInterpreter(object):
547 def dump_network_state(self):
548 return util.yaml_dumps(self._network_state)
549
550+ def as_dict(self):
551+ return {'version': self.version, 'config': self.config}
552+
553+ def get_network_state(self):
554+ ns = self.network_state
555+ return ns
556+
557 def parse_config(self, skip_broken=True):
558- # rebuild network state
559+ if self._version == 1:
560+ self.parse_config_v1(skip_broken=skip_broken)
561+ self._parsed = True
562+ elif self._version == 2:
563+ self.parse_config_v2(skip_broken=skip_broken)
564+ self._parsed = True
565+
566+ def parse_config_v1(self, skip_broken=True):
567 for command in self._config:
568 command_type = command['type']
569 try:
570@@ -211,6 +246,26 @@ class NetworkStateInterpreter(object):
571 exc_info=True)
572 LOG.debug(self.dump_network_state())
573
574+ def parse_config_v2(self, skip_broken=True):
575+ for command_type, command in self._config.items():
576+ if command_type == 'version':
577+ continue
578+ try:
579+ handler = self.command_handlers[command_type]
580+ except KeyError:
581+ raise RuntimeError("No handler found for"
582+ " command '%s'" % command_type)
583+ try:
584+ handler(self, command)
585+ self._v2_common(command)
586+ except InvalidCommand:
587+ if not skip_broken:
588+ raise
589+ else:
590+ LOG.warn("Skipping invalid command: %s", command,
591+ exc_info=True)
592+ LOG.debug(self.dump_network_state())
593+
594 @ensure_command_keys(['name'])
595 def handle_loopback(self, command):
596 return self.handle_physical(command)
597@@ -238,11 +293,16 @@ class NetworkStateInterpreter(object):
598 if subnets:
599 for subnet in subnets:
600 if subnet['type'] == 'static':
601+ if ':' in subnet['address']:
602+ self.use_ipv6 = True
603 if 'netmask' in subnet and ':' in subnet['address']:
604 subnet['netmask'] = mask2cidr(subnet['netmask'])
605 for route in subnet.get('routes', []):
606 if 'netmask' in route:
607 route['netmask'] = mask2cidr(route['netmask'])
608+ elif subnet['type'].endswith('6'):
609+ self.use_ipv6 = True
610+
611 iface.update({
612 'name': command.get('name'),
613 'type': command.get('type'),
614@@ -327,7 +387,7 @@ class NetworkStateInterpreter(object):
615 bond_if.update({param: val})
616 self._network_state['interfaces'].update({ifname: bond_if})
617
618- @ensure_command_keys(['name', 'bridge_interfaces', 'params'])
619+ @ensure_command_keys(['name', 'bridge_interfaces'])
620 def handle_bridge(self, command):
621 '''
622 auto br0
623@@ -373,7 +433,7 @@ class NetworkStateInterpreter(object):
624 self.handle_physical(command)
625 iface = interfaces.get(command.get('name'), {})
626 iface['bridge_ports'] = command['bridge_interfaces']
627- for param, val in command.get('params').items():
628+ for param, val in command.get('params', {}).items():
629 iface.update({param: val})
630
631 interfaces.update({iface['name']: iface})
632@@ -407,6 +467,236 @@ class NetworkStateInterpreter(object):
633 }
634 routes.append(route)
635
636+ # V2 handlers
637+ def handle_bonds(self, command):
638+ '''
639+ v2_command = {
640+ bond0: {
641+ 'interfaces': ['interface0', 'interface1'],
642+ 'miimon': 100,
643+ 'mode': '802.3ad',
644+ 'xmit_hash_policy': 'layer3+4'},
645+ bond1: {
646+ 'bond-slaves': ['interface2', 'interface7'],
647+ 'mode': 1
648+ }
649+ }
650+
651+ v1_command = {
652+ 'type': 'bond'
653+ 'name': 'bond0',
654+ 'bond_interfaces': [interface0, interface1],
655+ 'params': {
656+ 'bond-mode': '802.3ad',
657+ 'bond_miimon: 100,
658+ 'bond_xmit_hash_policy': 'layer3+4',
659+ }
660+ }
661+
662+ '''
663+ self._handle_bond_bridge(command, cmd_type='bond')
664+
665+ def handle_bridges(self, command):
666+
667+ '''
668+ v2_command = {
669+ br0: {
670+ 'interfaces': ['interface0', 'interface1'],
671+ 'fd': 0,
672+ 'stp': 'off',
673+ 'maxwait': 0,
674+ }
675+ }
676+
677+ v1_command = {
678+ 'type': 'bridge'
679+ 'name': 'br0',
680+ 'bridge_interfaces': [interface0, interface1],
681+ 'params': {
682+ 'bridge_stp': 'off',
683+ 'bridge_fd: 0,
684+ 'bridge_maxwait': 0
685+ }
686+ }
687+
688+ '''
689+ self._handle_bond_bridge(command, cmd_type='bridge')
690+
691+ def handle_ethernets(self, command):
692+ '''
693+ ethernets:
694+ eno1:
695+ match:
696+ macaddress: 00:11:22:33:44:55
697+ wakeonlan: true
698+ dhcp4: true
699+ dhcp6: false
700+ addresses:
701+ - 192.168.14.2/24
702+ - 2001:1::1/64
703+ gateway4: 192.168.14.1
704+ gateway6: 2001:1::2
705+ nameservers:
706+ search: [foo.local, bar.local]
707+ addresses: [8.8.8.8, 8.8.4.4]
708+ lom:
709+ match:
710+ driver: ixgbe
711+ set-name: lom1
712+ dhcp6: true
713+ switchports:
714+ match:
715+ name: enp2*
716+ mtu: 1280
717+
718+ command = {
719+ 'type': 'physical',
720+ 'mac_address': 'c0:d6:9f:2c:e8:80',
721+ 'name': 'eth0',
722+ 'subnets': [
723+ {'type': 'dhcp4'}
724+ ]
725+ }
726+ '''
727+ for eth, cfg in command.items():
728+ phy_cmd = {
729+ 'type': 'physical',
730+ 'name': cfg.get('set-name', eth),
731+ }
732+
733+ for key in ['mtu', 'match', 'wakeonlan']:
734+ if key in cfg:
735+ phy_cmd.update({key: cfg.get(key)})
736+
737+ subnets = self._v2_to_v1_ipcfg(cfg)
738+ if len(subnets) > 0:
739+ phy_cmd.update({'subnets': subnets})
740+
741+ LOG.debug('v2(ethernets) -> v1(physical):\n%s', phy_cmd)
742+ self.handle_physical(phy_cmd)
743+
744+ def handle_vlans(self, command):
745+ '''
746+ v2_vlans = {
747+ 'eth0.123': {
748+ 'id': 123,
749+ 'link': 'eth0',
750+ 'dhcp4': True,
751+ }
752+ }
753+
754+ v1_command = {
755+ 'type': 'vlan',
756+ 'name': 'eth0.123',
757+ 'vlan_link': 'eth0',
758+ 'vlan_id': 123,
759+ 'subnets': [{'type': 'dhcp4'}],
760+ }
761+ '''
762+ for vlan, cfg in command.items():
763+ vlan_cmd = {
764+ 'type': 'vlan',
765+ 'name': vlan,
766+ 'vlan_id': cfg.get('id'),
767+ 'vlan_link': cfg.get('link'),
768+ }
769+ subnets = self._v2_to_v1_ipcfg(cfg)
770+ if len(subnets) > 0:
771+ vlan_cmd.update({'subnets': subnets})
772+ LOG.debug('v2(vlans) -> v1(vlan):\n%s', vlan_cmd)
773+ self.handle_vlan(vlan_cmd)
774+
775+ def handle_wifis(self, command):
776+ raise NotImplemented('NetworkState V2: Skipping wifi configuration')
777+
778+ def _v2_common(self, cfg):
779+ LOG.debug('v2_common: handling config:\n%s', cfg)
780+ if 'nameservers' in cfg:
781+ search = cfg.get('nameservers').get('search', [])
782+ dns = cfg.get('nameservers').get('addresses', [])
783+ name_cmd = {'type': 'nameserver'}
784+ if len(search) > 0:
785+ name_cmd.update({'search': search})
786+ if len(dns) > 0:
787+ name_cmd.update({'addresses': dns})
788+ LOG.debug('v2(nameserver) -> v1(nameserver):\n%s', name_cmd)
789+ self.handle_nameserver(name_cmd)
790+
791+ def _handle_bond_bridge(self, command, cmd_type=None):
792+ """Common handler for bond and bridge types"""
793+ for item_name, item_cfg in command.items():
794+ item_params = dict((key, value) for (key, value) in
795+ item_cfg.items() if key not in
796+ NETWORK_V2_KEY_FILTER)
797+ v1_cmd = {
798+ 'type': cmd_type,
799+ 'name': item_name,
800+ cmd_type + '_interfaces': item_cfg.get('interfaces'),
801+ 'params': item_params,
802+ }
803+ subnets = self._v2_to_v1_ipcfg(item_cfg)
804+ if len(subnets) > 0:
805+ v1_cmd.update({'subnets': subnets})
806+
807+ LOG.debug('v2(%ss) -> v1(%s):\n%s', cmd_type, cmd_type, v1_cmd)
808+ self.handle_bridge(v1_cmd)
809+
810+ def _v2_to_v1_ipcfg(self, cfg):
811+ """Common ipconfig extraction from v2 to v1 subnets array."""
812+
813+ subnets = []
814+ if 'dhcp4' in cfg:
815+ subnets.append({'type': 'dhcp4'})
816+ if 'dhcp6' in cfg:
817+ self.use_ipv6 = True
818+ subnets.append({'type': 'dhcp6'})
819+
820+ gateway4 = None
821+ gateway6 = None
822+ for address in cfg.get('addresses', []):
823+ subnet = {
824+ 'type': 'static',
825+ 'address': address,
826+ }
827+
828+ routes = []
829+ for route in cfg.get('routes', []):
830+ route_addr = route.get('to')
831+ if "/" in route_addr:
832+ route_addr, route_cidr = route_addr.split("/")
833+ route_netmask = cidr2mask(route_cidr)
834+ subnet_route = {
835+ 'address': route_addr,
836+ 'netmask': route_netmask,
837+ 'gateway': route.get('via')
838+ }
839+ routes.append(subnet_route)
840+ if len(routes) > 0:
841+ subnet.update({'routes': routes})
842+
843+ if ":" in address:
844+ if 'gateway6' in cfg and gateway6 is None:
845+ gateway6 = cfg.get('gateway6')
846+ subnet.update({'gateway': gateway6})
847+ else:
848+ if 'gateway4' in cfg and gateway4 is None:
849+ gateway4 = cfg.get('gateway4')
850+ subnet.update({'gateway': gateway4})
851+
852+ subnets.append(subnet)
853+ return subnets
854+
855+
856+def subnet_is_ipv6(subnet):
857+ """ Common helper for checking network_state subnets for ipv6"""
858+ # 'static6' or 'dhcp6'
859+ if subnet['type'].endswith('6'):
860+ # This is a request for DHCPv6.
861+ return True
862+ elif subnet['type'] == 'static' and ":" in subnet['address']:
863+ return True
864+ return False
865+
866
867 def cidr2mask(cidr):
868 mask = [0, 0, 0, 0]
869diff --git a/cloudinit/net/renderers.py b/cloudinit/net/renderers.py
870index 5ad8455..5117b4a 100644
871--- a/cloudinit/net/renderers.py
872+++ b/cloudinit/net/renderers.py
873@@ -1,15 +1,17 @@
874 # This file is part of cloud-init. See LICENSE file for license information.
875
876 from . import eni
877+from . import netplan
878 from . import RendererNotFoundError
879 from . import sysconfig
880
881 NAME_TO_RENDERER = {
882 "eni": eni,
883+ "netplan": netplan,
884 "sysconfig": sysconfig,
885 }
886
887-DEFAULT_PRIORITY = ["eni", "sysconfig"]
888+DEFAULT_PRIORITY = ["eni", "sysconfig", "netplan"]
889
890
891 def search(priority=None, target=None, first=False):
892diff --git a/cloudinit/net/sysconfig.py b/cloudinit/net/sysconfig.py
893index 117b515..504e4d0 100644
894--- a/cloudinit/net/sysconfig.py
895+++ b/cloudinit/net/sysconfig.py
896@@ -9,6 +9,7 @@ from cloudinit.distros.parsers import resolv_conf
897 from cloudinit import util
898
899 from . import renderer
900+from .network_state import subnet_is_ipv6
901
902
903 def _make_header(sep='#'):
904@@ -194,7 +195,7 @@ class Renderer(renderer.Renderer):
905 def __init__(self, config=None):
906 if not config:
907 config = {}
908- self.sysconf_dir = config.get('sysconf_dir', 'etc/sysconfig/')
909+ self.sysconf_dir = config.get('sysconf_dir', 'etc/sysconfig')
910 self.netrules_path = config.get(
911 'netrules_path', 'etc/udev/rules.d/70-persistent-net.rules')
912 self.dns_path = config.get('dns_path', 'etc/resolv.conf')
913@@ -220,7 +221,7 @@ class Renderer(renderer.Renderer):
914 iface_cfg['BOOTPROTO'] = 'dhcp'
915 elif subnet_type == 'static':
916 iface_cfg['BOOTPROTO'] = 'static'
917- if subnet.get('ipv6'):
918+ if subnet_is_ipv6(subnet):
919 iface_cfg['IPV6ADDR'] = subnet['address']
920 iface_cfg['IPV6INIT'] = True
921 else:
922@@ -390,19 +391,28 @@ class Renderer(renderer.Renderer):
923 return contents
924
925 def render_network_state(self, network_state, target=None):
926+ file_mode = 0o644
927 base_sysconf_dir = util.target_path(target, self.sysconf_dir)
928 for path, data in self._render_sysconfig(base_sysconf_dir,
929 network_state).items():
930- util.write_file(path, data)
931+ util.write_file(path, data, file_mode)
932 if self.dns_path:
933 dns_path = util.target_path(target, self.dns_path)
934 resolv_content = self._render_dns(network_state,
935 existing_dns_path=dns_path)
936- util.write_file(dns_path, resolv_content)
937+ util.write_file(dns_path, resolv_content, file_mode)
938 if self.netrules_path:
939 netrules_content = self._render_persistent_net(network_state)
940 netrules_path = util.target_path(target, self.netrules_path)
941- util.write_file(netrules_path, netrules_content)
942+ util.write_file(netrules_path, netrules_content, file_mode)
943+
944+ # always write /etc/sysconfig/network configuration
945+ sysconfig_path = util.target_path(target, "etc/sysconfig/network")
946+ netcfg = [_make_header(), 'NETWORKING=yes']
947+ if network_state.use_ipv6:
948+ netcfg.append('NETWORKING_IPV6=yes')
949+ netcfg.append('IPV6_AUTOCONF=no')
950+ util.write_file(sysconfig_path, "\n".join(netcfg) + "\n", file_mode)
951
952
953 def available(target=None):
954diff --git a/systemd/cloud-init.service b/systemd/cloud-init.service
955index fb3b918..39acc20 100644
956--- a/systemd/cloud-init.service
957+++ b/systemd/cloud-init.service
958@@ -5,6 +5,7 @@ Wants=cloud-init-local.service
959 Wants=sshd-keygen.service
960 Wants=sshd.service
961 After=cloud-init-local.service
962+After=systemd-networkd-wait-online.service
963 After=networking.service
964 Before=network-online.target
965 Before=sshd-keygen.service
966diff --git a/tests/unittests/test_distros/test_netconfig.py b/tests/unittests/test_distros/test_netconfig.py
967index bde3bb5..d187c9d 100644
968--- a/tests/unittests/test_distros/test_netconfig.py
969+++ b/tests/unittests/test_distros/test_netconfig.py
970@@ -19,6 +19,7 @@ from cloudinit.distros.parsers.sys_conf import SysConf
971 from cloudinit import helpers
972 from cloudinit import settings
973 from cloudinit import util
974+from cloudinit.net import eni
975
976
977 BASE_NET_CFG = '''
978@@ -28,10 +29,10 @@ iface lo inet loopback
979 auto eth0
980 iface eth0 inet static
981 address 192.168.1.5
982- netmask 255.255.255.0
983- network 192.168.0.0
984 broadcast 192.168.1.0
985 gateway 192.168.1.254
986+ netmask 255.255.255.0
987+ network 192.168.0.0
988
989 auto eth1
990 iface eth1 inet dhcp
991@@ -67,6 +68,100 @@ iface eth1 inet6 static
992 gateway 2607:f0d0:1002:0011::1
993 '''
994
995+V1_NET_CFG = {'config': [{'name': 'eth0',
996+
997+ 'subnets': [{'address': '192.168.1.5',
998+ 'broadcast': '192.168.1.0',
999+ 'gateway': '192.168.1.254',
1000+ 'netmask': '255.255.255.0',
1001+ 'type': 'static'}],
1002+ 'type': 'physical'},
1003+ {'name': 'eth1',
1004+ 'subnets': [{'control': 'auto', 'type': 'dhcp4'}],
1005+ 'type': 'physical'}],
1006+ 'version': 1}
1007+
1008+V1_NET_CFG_OUTPUT = """
1009+# This file is generated from information provided by
1010+# the datasource. Changes to it will not persist across an instance.
1011+# To disable cloud-init's network configuration capabilities, write a file
1012+# /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
1013+# network: {config: disabled}
1014+auto lo
1015+iface lo inet loopback
1016+
1017+auto eth0
1018+iface eth0 inet static
1019+ address 192.168.1.5
1020+ broadcast 192.168.1.0
1021+ gateway 192.168.1.254
1022+ netmask 255.255.255.0
1023+
1024+auto eth1
1025+iface eth1 inet dhcp
1026+"""
1027+
1028+V1_NET_CFG_IPV6 = {'config': [{'name': 'eth0',
1029+ 'subnets': [{'address':
1030+ '2607:f0d0:1002:0011::2',
1031+ 'gateway':
1032+ '2607:f0d0:1002:0011::1',
1033+ 'netmask': '64',
1034+ 'type': 'static'}],
1035+ 'type': 'physical'},
1036+ {'name': 'eth1',
1037+ 'subnets': [{'control': 'auto',
1038+ 'type': 'dhcp4'}],
1039+ 'type': 'physical'}],
1040+ 'version': 1}
1041+
1042+
1043+V1_TO_V2_NET_CFG_OUTPUT = """
1044+# This file is generated from information provided by
1045+# the datasource. Changes to it will not persist across an instance.
1046+# To disable cloud-init's network configuration capabilities, write a file
1047+# /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
1048+# network: {config: disabled}
1049+network:
1050+ version: 2
1051+ ethernets:
1052+ eth0:
1053+ addresses:
1054+ - 192.168.1.5/255.255.255.0
1055+ gateway4: 192.168.1.254
1056+ eth1:
1057+ dhcp4: true
1058+"""
1059+
1060+V2_NET_CFG = {
1061+ 'ethernets': {
1062+ 'eth7': {
1063+ 'addresses': ['192.168.1.5/255.255.255.0'],
1064+ 'gateway4': '192.168.1.254'},
1065+ 'eth9': {
1066+ 'dhcp4': True}
1067+ },
1068+ 'version': 2
1069+}
1070+
1071+
1072+V2_TO_V2_NET_CFG_OUTPUT = """
1073+# This file is generated from information provided by
1074+# the datasource. Changes to it will not persist across an instance.
1075+# To disable cloud-init's network configuration capabilities, write a file
1076+# /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
1077+# network: {config: disabled}
1078+network:
1079+ version: 2
1080+ ethernets:
1081+ eth7:
1082+ addresses:
1083+ - 192.168.1.5/255.255.255.0
1084+ gateway4: 192.168.1.254
1085+ eth9:
1086+ dhcp4: true
1087+"""
1088+
1089
1090 class WriteBuffer(object):
1091 def __init__(self):
1092@@ -83,12 +178,14 @@ class WriteBuffer(object):
1093
1094 class TestNetCfgDistro(TestCase):
1095
1096- def _get_distro(self, dname):
1097+ def _get_distro(self, dname, renderers=None):
1098 cls = distros.fetch(dname)
1099 cfg = settings.CFG_BUILTIN
1100 cfg['system_info']['distro'] = dname
1101+ if renderers:
1102+ cfg['system_info']['network'] = {'renderers': renderers}
1103 paths = helpers.Paths({})
1104- return cls(dname, cfg, paths)
1105+ return cls(dname, cfg.get('system_info'), paths)
1106
1107 def test_simple_write_ub(self):
1108 ub_distro = self._get_distro('ubuntu')
1109@@ -116,6 +213,107 @@ class TestNetCfgDistro(TestCase):
1110 self.assertEqual(str(write_buf).strip(), BASE_NET_CFG.strip())
1111 self.assertEqual(write_buf.mode, 0o644)
1112
1113+ def test_apply_network_config_eni_ub(self):
1114+ ub_distro = self._get_distro('ubuntu')
1115+ with ExitStack() as mocks:
1116+ write_bufs = {}
1117+
1118+ def replace_write(filename, content, mode=0o644, omode="wb"):
1119+ buf = WriteBuffer()
1120+ buf.mode = mode
1121+ buf.omode = omode
1122+ buf.write(content)
1123+ write_bufs[filename] = buf
1124+
1125+ # eni availability checks
1126+ mocks.enter_context(
1127+ mock.patch.object(util, 'which', return_value=True))
1128+ mocks.enter_context(
1129+ mock.patch.object(eni, 'available', return_value=True))
1130+ mocks.enter_context(
1131+ mock.patch.object(util, 'ensure_dir'))
1132+ mocks.enter_context(
1133+ mock.patch.object(util, 'write_file', replace_write))
1134+ mocks.enter_context(
1135+ mock.patch.object(os.path, 'isfile', return_value=False))
1136+
1137+ ub_distro.apply_network_config(V1_NET_CFG, False)
1138+
1139+ self.assertEqual(len(write_bufs), 2)
1140+ eni_name = '/etc/network/interfaces.d/50-cloud-init.cfg'
1141+ self.assertIn(eni_name, write_bufs)
1142+ write_buf = write_bufs[eni_name]
1143+ self.assertEqual(str(write_buf).strip(), V1_NET_CFG_OUTPUT.strip())
1144+ self.assertEqual(write_buf.mode, 0o644)
1145+
1146+ def test_apply_network_config_v1_to_netplan_ub(self):
1147+ renderers = ['netplan']
1148+ ub_distro = self._get_distro('ubuntu', renderers=renderers)
1149+ with ExitStack() as mocks:
1150+ write_bufs = {}
1151+
1152+ def replace_write(filename, content, mode=0o644, omode="wb"):
1153+ buf = WriteBuffer()
1154+ buf.mode = mode
1155+ buf.omode = omode
1156+ buf.write(content)
1157+ write_bufs[filename] = buf
1158+
1159+ mocks.enter_context(
1160+ mock.patch.object(util, 'which', return_value=True))
1161+ mocks.enter_context(
1162+ mock.patch.object(util, 'write_file', replace_write))
1163+ mocks.enter_context(
1164+ mock.patch.object(util, 'ensure_dir'))
1165+ mocks.enter_context(
1166+ mock.patch.object(util, 'subp', return_value=(0, 0)))
1167+ mocks.enter_context(
1168+ mock.patch.object(os.path, 'isfile', return_value=False))
1169+
1170+ ub_distro.apply_network_config(V1_NET_CFG, False)
1171+
1172+ self.assertEqual(len(write_bufs), 1)
1173+ netplan_name = '/etc/netplan/50-cloud-init.yaml'
1174+ self.assertIn(netplan_name, write_bufs)
1175+ write_buf = write_bufs[netplan_name]
1176+ self.assertEqual(str(write_buf).strip(),
1177+ V1_TO_V2_NET_CFG_OUTPUT.strip())
1178+ self.assertEqual(write_buf.mode, 0o644)
1179+
1180+ def test_apply_network_config_v2_passthrough_ub(self):
1181+ renderers = ['netplan']
1182+ ub_distro = self._get_distro('ubuntu', renderers=renderers)
1183+ with ExitStack() as mocks:
1184+ write_bufs = {}
1185+
1186+ def replace_write(filename, content, mode=0o644, omode="wb"):
1187+ buf = WriteBuffer()
1188+ buf.mode = mode
1189+ buf.omode = omode
1190+ buf.write(content)
1191+ write_bufs[filename] = buf
1192+
1193+ mocks.enter_context(
1194+ mock.patch.object(util, 'which', return_value=True))
1195+ mocks.enter_context(
1196+ mock.patch.object(util, 'write_file', replace_write))
1197+ mocks.enter_context(
1198+ mock.patch.object(util, 'ensure_dir'))
1199+ mocks.enter_context(
1200+ mock.patch.object(util, 'subp', return_value=(0, 0)))
1201+ mocks.enter_context(
1202+ mock.patch.object(os.path, 'isfile', return_value=False))
1203+
1204+ ub_distro.apply_network_config(V2_NET_CFG, False)
1205+
1206+ self.assertEqual(len(write_bufs), 1)
1207+ netplan_name = '/etc/netplan/50-cloud-init.yaml'
1208+ self.assertIn(netplan_name, write_bufs)
1209+ write_buf = write_bufs[netplan_name]
1210+ self.assertEqual(str(write_buf).strip(),
1211+ V2_TO_V2_NET_CFG_OUTPUT.strip())
1212+ self.assertEqual(write_buf.mode, 0o644)
1213+
1214 def assertCfgEquals(self, blob1, blob2):
1215 b1 = dict(SysConf(blob1.strip().splitlines()))
1216 b2 = dict(SysConf(blob2.strip().splitlines()))
1217@@ -195,6 +393,79 @@ NETWORKING=yes
1218 self.assertCfgEquals(expected_buf, str(write_buf))
1219 self.assertEqual(write_buf.mode, 0o644)
1220
1221+ def test_apply_network_config_rh(self):
1222+ renderers = ['sysconfig']
1223+ rh_distro = self._get_distro('rhel', renderers=renderers)
1224+
1225+ write_bufs = {}
1226+
1227+ def replace_write(filename, content, mode=0o644, omode="wb"):
1228+ buf = WriteBuffer()
1229+ buf.mode = mode
1230+ buf.omode = omode
1231+ buf.write(content)
1232+ write_bufs[filename] = buf
1233+
1234+ with ExitStack() as mocks:
1235+ # sysconfig availability checks
1236+ mocks.enter_context(
1237+ mock.patch.object(util, 'which', return_value=True))
1238+ mocks.enter_context(
1239+ mock.patch.object(util, 'write_file', replace_write))
1240+ mocks.enter_context(
1241+ mock.patch.object(util, 'load_file', return_value=''))
1242+ mocks.enter_context(
1243+ mock.patch.object(os.path, 'isfile', return_value=True))
1244+
1245+ rh_distro.apply_network_config(V1_NET_CFG, False)
1246+
1247+ self.assertEqual(len(write_bufs), 5)
1248+
1249+ # eth0
1250+ self.assertIn('/etc/sysconfig/network-scripts/ifcfg-eth0',
1251+ write_bufs)
1252+ write_buf = write_bufs['/etc/sysconfig/network-scripts/ifcfg-eth0']
1253+ expected_buf = '''
1254+# Created by cloud-init on instance boot automatically, do not edit.
1255+#
1256+BOOTPROTO=static
1257+DEVICE=eth0
1258+IPADDR=192.168.1.5
1259+NETMASK=255.255.255.0
1260+NM_CONTROLLED=no
1261+ONBOOT=yes
1262+TYPE=Ethernet
1263+USERCTL=no
1264+'''
1265+ self.assertCfgEquals(expected_buf, str(write_buf))
1266+ self.assertEqual(write_buf.mode, 0o644)
1267+
1268+ # eth1
1269+ self.assertIn('/etc/sysconfig/network-scripts/ifcfg-eth1',
1270+ write_bufs)
1271+ write_buf = write_bufs['/etc/sysconfig/network-scripts/ifcfg-eth1']
1272+ expected_buf = '''
1273+# Created by cloud-init on instance boot automatically, do not edit.
1274+#
1275+BOOTPROTO=dhcp
1276+DEVICE=eth1
1277+NM_CONTROLLED=no
1278+ONBOOT=yes
1279+TYPE=Ethernet
1280+USERCTL=no
1281+'''
1282+ self.assertCfgEquals(expected_buf, str(write_buf))
1283+ self.assertEqual(write_buf.mode, 0o644)
1284+
1285+ self.assertIn('/etc/sysconfig/network', write_bufs)
1286+ write_buf = write_bufs['/etc/sysconfig/network']
1287+ expected_buf = '''
1288+# Created by cloud-init v. 0.7
1289+NETWORKING=yes
1290+'''
1291+ self.assertCfgEquals(expected_buf, str(write_buf))
1292+ self.assertEqual(write_buf.mode, 0o644)
1293+
1294 def test_write_ipv6_rhel(self):
1295 rh_distro = self._get_distro('rhel')
1296
1297@@ -274,6 +545,78 @@ IPV6_AUTOCONF=no
1298 self.assertCfgEquals(expected_buf, str(write_buf))
1299 self.assertEqual(write_buf.mode, 0o644)
1300
1301+ def test_apply_network_config_ipv6_rh(self):
1302+ renderers = ['sysconfig']
1303+ rh_distro = self._get_distro('rhel', renderers=renderers)
1304+
1305+ write_bufs = {}
1306+
1307+ def replace_write(filename, content, mode=0o644, omode="wb"):
1308+ buf = WriteBuffer()
1309+ buf.mode = mode
1310+ buf.omode = omode
1311+ buf.write(content)
1312+ write_bufs[filename] = buf
1313+
1314+ with ExitStack() as mocks:
1315+ mocks.enter_context(
1316+ mock.patch.object(util, 'which', return_value=True))
1317+ mocks.enter_context(
1318+ mock.patch.object(util, 'write_file', replace_write))
1319+ mocks.enter_context(
1320+ mock.patch.object(util, 'load_file', return_value=''))
1321+ mocks.enter_context(
1322+ mock.patch.object(os.path, 'isfile', return_value=True))
1323+
1324+ rh_distro.apply_network_config(V1_NET_CFG_IPV6, False)
1325+
1326+ self.assertEqual(len(write_bufs), 5)
1327+
1328+ self.assertIn('/etc/sysconfig/network-scripts/ifcfg-eth0',
1329+ write_bufs)
1330+ write_buf = write_bufs['/etc/sysconfig/network-scripts/ifcfg-eth0']
1331+ expected_buf = '''
1332+# Created by cloud-init on instance boot automatically, do not edit.
1333+#
1334+BOOTPROTO=static
1335+DEVICE=eth0
1336+IPV6ADDR=2607:f0d0:1002:0011::2
1337+IPV6INIT=yes
1338+NETMASK=64
1339+NM_CONTROLLED=no
1340+ONBOOT=yes
1341+TYPE=Ethernet
1342+USERCTL=no
1343+'''
1344+ self.assertCfgEquals(expected_buf, str(write_buf))
1345+ self.assertEqual(write_buf.mode, 0o644)
1346+ self.assertIn('/etc/sysconfig/network-scripts/ifcfg-eth1',
1347+ write_bufs)
1348+ write_buf = write_bufs['/etc/sysconfig/network-scripts/ifcfg-eth1']
1349+ expected_buf = '''
1350+# Created by cloud-init on instance boot automatically, do not edit.
1351+#
1352+BOOTPROTO=dhcp
1353+DEVICE=eth1
1354+NM_CONTROLLED=no
1355+ONBOOT=yes
1356+TYPE=Ethernet
1357+USERCTL=no
1358+'''
1359+ self.assertCfgEquals(expected_buf, str(write_buf))
1360+ self.assertEqual(write_buf.mode, 0o644)
1361+
1362+ self.assertIn('/etc/sysconfig/network', write_bufs)
1363+ write_buf = write_bufs['/etc/sysconfig/network']
1364+ expected_buf = '''
1365+# Created by cloud-init v. 0.7
1366+NETWORKING=yes
1367+NETWORKING_IPV6=yes
1368+IPV6_AUTOCONF=no
1369+'''
1370+ self.assertCfgEquals(expected_buf, str(write_buf))
1371+ self.assertEqual(write_buf.mode, 0o644)
1372+
1373 def test_simple_write_freebsd(self):
1374 fbsd_distro = self._get_distro('freebsd')
1375
1376diff --git a/tests/unittests/test_net.py b/tests/unittests/test_net.py
1377index 902204a..4f07d80 100644
1378--- a/tests/unittests/test_net.py
1379+++ b/tests/unittests/test_net.py
1380@@ -3,6 +3,7 @@
1381 from cloudinit import net
1382 from cloudinit.net import cmdline
1383 from cloudinit.net import eni
1384+from cloudinit.net import netplan
1385 from cloudinit.net import network_state
1386 from cloudinit.net import renderers
1387 from cloudinit.net import sysconfig
1388@@ -408,6 +409,41 @@ NETWORK_CONFIGS = {
1389 post-up route add default gw 65.61.151.37 || true
1390 pre-down route del default gw 65.61.151.37 || true
1391 """).rstrip(' '),
1392+ 'expected_netplan': textwrap.dedent("""
1393+ network:
1394+ version: 2
1395+ ethernets:
1396+ eth1:
1397+ match:
1398+ macaddress: cf:d6:af:48:e8:80
1399+ nameservers:
1400+ addresses:
1401+ - 1.2.3.4
1402+ - 5.6.7.8
1403+ search:
1404+ - wark.maas
1405+ set-name: eth1
1406+ eth99:
1407+ addresses:
1408+ - 192.168.21.3/24
1409+ dhcp4: true
1410+ match:
1411+ macaddress: c0:d6:9f:2c:e8:80
1412+ nameservers:
1413+ addresses:
1414+ - 8.8.8.8
1415+ - 8.8.4.4
1416+ - 1.2.3.4
1417+ - 5.6.7.8
1418+ search:
1419+ - barley.maas
1420+ - sach.maas
1421+ - wark.maas
1422+ routes:
1423+ - to: 0.0.0.0/0.0.0.0
1424+ via: 65.61.151.37
1425+ set-name: eth99
1426+ """).rstrip(' '),
1427 'yaml': textwrap.dedent("""
1428 version: 1
1429 config:
1430@@ -450,6 +486,14 @@ NETWORK_CONFIGS = {
1431 # control-alias iface0
1432 iface iface0 inet6 dhcp
1433 """).rstrip(' '),
1434+ 'expected_netplan': textwrap.dedent("""
1435+ network:
1436+ version: 2
1437+ ethernets:
1438+ iface0:
1439+ dhcp4: true
1440+ dhcp6: true
1441+ """).rstrip(' '),
1442 'yaml': textwrap.dedent("""\
1443 version: 1
1444 config:
1445@@ -524,6 +568,126 @@ iface eth0.101 inet static
1446 post-up route add -net 10.0.0.0 netmask 255.0.0.0 gw 11.0.0.1 metric 3 || true
1447 pre-down route del -net 10.0.0.0 netmask 255.0.0.0 gw 11.0.0.1 metric 3 || true
1448 """),
1449+ 'expected_netplan': textwrap.dedent("""
1450+ network:
1451+ version: 2
1452+ ethernets:
1453+ eth0:
1454+ match:
1455+ macaddress: c0:d6:9f:2c:e8:80
1456+ nameservers:
1457+ addresses:
1458+ - 8.8.8.8
1459+ - 4.4.4.4
1460+ - 8.8.4.4
1461+ search:
1462+ - barley.maas
1463+ - wark.maas
1464+ - foobar.maas
1465+ set-name: eth0
1466+ eth1:
1467+ match:
1468+ macaddress: aa:d6:9f:2c:e8:80
1469+ nameservers:
1470+ addresses:
1471+ - 8.8.8.8
1472+ - 4.4.4.4
1473+ - 8.8.4.4
1474+ search:
1475+ - barley.maas
1476+ - wark.maas
1477+ - foobar.maas
1478+ set-name: eth1
1479+ eth2:
1480+ match:
1481+ macaddress: c0:bb:9f:2c:e8:80
1482+ nameservers:
1483+ addresses:
1484+ - 8.8.8.8
1485+ - 4.4.4.4
1486+ - 8.8.4.4
1487+ search:
1488+ - barley.maas
1489+ - wark.maas
1490+ - foobar.maas
1491+ set-name: eth2
1492+ eth3:
1493+ match:
1494+ macaddress: 66:bb:9f:2c:e8:80
1495+ nameservers:
1496+ addresses:
1497+ - 8.8.8.8
1498+ - 4.4.4.4
1499+ - 8.8.4.4
1500+ search:
1501+ - barley.maas
1502+ - wark.maas
1503+ - foobar.maas
1504+ set-name: eth3
1505+ eth4:
1506+ match:
1507+ macaddress: 98:bb:9f:2c:e8:80
1508+ nameservers:
1509+ addresses:
1510+ - 8.8.8.8
1511+ - 4.4.4.4
1512+ - 8.8.4.4
1513+ search:
1514+ - barley.maas
1515+ - wark.maas
1516+ - foobar.maas
1517+ set-name: eth4
1518+ eth5:
1519+ dhcp4: true
1520+ match:
1521+ macaddress: 98:bb:9f:2c:e8:8a
1522+ nameservers:
1523+ addresses:
1524+ - 8.8.8.8
1525+ - 4.4.4.4
1526+ - 8.8.4.4
1527+ search:
1528+ - barley.maas
1529+ - wark.maas
1530+ - foobar.maas
1531+ set-name: eth5
1532+ bonds:
1533+ bond0:
1534+ dhcp6: true
1535+ interfaces:
1536+ - eth1
1537+ - eth2
1538+ parameters:
1539+ mode: active-backup
1540+ bridges:
1541+ br0:
1542+ addresses:
1543+ - 192.168.14.2/24
1544+ - 2001:1::1/64
1545+ interfaces:
1546+ - eth3
1547+ - eth4
1548+ vlans:
1549+ bond0.200:
1550+ dhcp4: true
1551+ id: 200
1552+ link: bond0
1553+ eth0.101:
1554+ addresses:
1555+ - 192.168.0.2/24
1556+ - 192.168.2.10/24
1557+ gateway4: 192.168.0.1
1558+ id: 101
1559+ link: eth0
1560+ nameservers:
1561+ addresses:
1562+ - 192.168.0.10
1563+ - 10.23.23.134
1564+ search:
1565+ - barley.maas
1566+ - sacchromyces.maas
1567+ - brettanomyces.maas
1568+ """).rstrip(' '),
1569 'yaml': textwrap.dedent("""
1570 version: 1
1571 config:
1572@@ -808,6 +972,99 @@ iface eth0 inet dhcp
1573 expected, dir2dict(tmp_dir)['/etc/network/interfaces'])
1574
1575
1576+class TestNetplanNetRendering(CiTestCase):
1577+
1578+ @mock.patch("cloudinit.net.sys_dev_path")
1579+ @mock.patch("cloudinit.net.read_sys_net")
1580+ @mock.patch("cloudinit.net.get_devicelist")
1581+ def test_default_generation(self, mock_get_devicelist,
1582+ mock_read_sys_net,
1583+ mock_sys_dev_path):
1584+ tmp_dir = self.tmp_dir()
1585+ _setup_test(tmp_dir, mock_get_devicelist,
1586+ mock_read_sys_net, mock_sys_dev_path)
1587+
1588+ network_cfg = net.generate_fallback_config()
1589+ ns = network_state.parse_net_config_data(network_cfg,
1590+ skip_broken=False)
1591+
1592+ render_dir = os.path.join(tmp_dir, "render")
1593+ os.makedirs(render_dir)
1594+
1595+ render_target = 'netplan.yaml'
1596+ renderer = netplan.Renderer(
1597+ {'netplan_path': render_target, 'postcmds': False})
1598+ renderer.render_network_state(render_dir, ns)
1599+
1600+ self.assertTrue(os.path.exists(os.path.join(render_dir,
1601+ render_target)))
1602+ with open(os.path.join(render_dir, render_target)) as fh:
1603+ contents = fh.read()
1604+ print(contents)
1605+
1606+ expected = """
1607+network:
1608+ version: 2
1609+ ethernets:
1610+ eth1000:
1611+ dhcp4: true
1612+ match:
1613+ macaddress: 07-1c-c6-75-a4-be
1614+ set-name: eth1000
1615+"""
1616+ self.assertEqual(expected.lstrip(), contents.lstrip())
1617+
1618+
1619+class TestNetplanPostcommands(CiTestCase):
1620+ mycfg = {
1621+ 'config': [{"type": "physical", "name": "eth0",
1622+ "mac_address": "c0:d6:9f:2c:e8:80",
1623+ "subnets": [{"type": "dhcp"}]}],
1624+ 'version': 1}
1625+
1626+ @mock.patch.object(netplan.Renderer, '_netplan_generate')
1627+ @mock.patch.object(netplan.Renderer, '_net_setup_link')
1628+ def test_netplan_render_calls_postcmds(self, mock_netplan_generate,
1629+ mock_net_setup_link):
1630+ tmp_dir = self.tmp_dir()
1631+ ns = network_state.parse_net_config_data(self.mycfg,
1632+ skip_broken=False)
1633+
1634+ render_dir = os.path.join(tmp_dir, "render")
1635+ os.makedirs(render_dir)
1636+
1637+ render_target = 'netplan.yaml'
1638+ renderer = netplan.Renderer(
1639+ {'netplan_path': render_target, 'postcmds': True})
1640+ renderer.render_network_state(render_dir, ns)
1641+
1642+ mock_netplan_generate.assert_called_with(run=True)
1643+ mock_net_setup_link.assert_called_with(run=True)
1644+
1645+ @mock.patch.object(netplan, "get_devicelist")
1646+ @mock.patch('cloudinit.util.subp')
1647+ def test_netplan_postcmds(self, mock_subp, mock_devlist):
1648+ mock_devlist.side_effect = [['lo']]
1649+ tmp_dir = self.tmp_dir()
1650+ ns = network_state.parse_net_config_data(self.mycfg,
1651+ skip_broken=False)
1652+
1653+ render_dir = os.path.join(tmp_dir, "render")
1654+ os.makedirs(render_dir)
1655+
1656+ render_target = 'netplan.yaml'
1657+ renderer = netplan.Renderer(
1658+ {'netplan_path': render_target, 'postcmds': True})
1659+ renderer.render_network_state(render_dir, ns)
1660+
1661+ expected = [
1662+ mock.call(['netplan', 'generate'], capture=True),
1663+ mock.call(['udevadm', 'test-builtin', 'net_setup_link',
1664+ '/sys/class/net/lo'], capture=True),
1665+ ]
1666+ mock_subp.assert_has_calls(expected)
1667+
1668+
1669 class TestEniNetworkStateToEni(CiTestCase):
1670 mycfg = {
1671 'config': [{"type": "physical", "name": "eth0",
1672@@ -953,6 +1210,50 @@ class TestCmdlineReadKernelConfig(CiTestCase):
1673 self.assertEqual(found['config'], expected)
1674
1675
1676+class TestNetplanRoundTrip(CiTestCase):
1677+ def _render_and_read(self, network_config=None, state=None,
1678+ netplan_path=None, dir=None):
1679+ if dir is None:
1680+ dir = self.tmp_dir()
1681+
1682+ if network_config:
1683+ ns = network_state.parse_net_config_data(network_config)
1684+ elif state:
1685+ ns = state
1686+ else:
1687+ raise ValueError("Expected data or state, got neither")
1688+
1689+ if netplan_path is None:
1690+ netplan_path = 'etc/netplan/50-cloud-init.yaml'
1691+
1692+ renderer = netplan.Renderer(
1693+ config={'netplan_path': netplan_path})
1694+
1695+ renderer.render_network_state(dir, ns)
1696+ return dir2dict(dir)
1697+
1698+ def testsimple_render_small_netplan(self):
1699+ entry = NETWORK_CONFIGS['small']
1700+ files = self._render_and_read(network_config=yaml.load(entry['yaml']))
1701+ self.assertEqual(
1702+ entry['expected_netplan'].splitlines(),
1703+ files['/etc/netplan/50-cloud-init.yaml'].splitlines())
1704+
1705+ def testsimple_render_v4_and_v6(self):
1706+ entry = NETWORK_CONFIGS['v4_and_v6']
1707+ files = self._render_and_read(network_config=yaml.load(entry['yaml']))
1708+ self.assertEqual(
1709+ entry['expected_netplan'].splitlines(),
1710+ files['/etc/netplan/50-cloud-init.yaml'].splitlines())
1711+
1712+ def testsimple_render_all(self):
1713+ entry = NETWORK_CONFIGS['all']
1714+ files = self._render_and_read(network_config=yaml.load(entry['yaml']))
1715+ self.assertEqual(
1716+ entry['expected_netplan'].splitlines(),
1717+ files['/etc/netplan/50-cloud-init.yaml'].splitlines())
1718+
1719+
1720 class TestEniRoundTrip(CiTestCase):
1721 def _render_and_read(self, network_config=None, state=None, eni_path=None,
1722 links_prefix=None, netrules_path=None, dir=None):
1723diff --git a/tools/net-convert.py b/tools/net-convert.py
1724new file mode 100755
1725index 0000000..1424bb0
1726--- /dev/null
1727+++ b/tools/net-convert.py
1728@@ -0,0 +1,83 @@
1729+#!/usr/bin/python3
1730+#
1731+# This file is part of cloud-init. See LICENSE file ...
1732+
1733+import argparse
1734+import json
1735+import os
1736+import yaml
1737+
1738+from cloudinit.sources.helpers import openstack
1739+
1740+from cloudinit.net import eni
1741+from cloudinit.net import network_state
1742+from cloudinit.net import netplan
1743+from cloudinit.net import sysconfig
1744+
1745+
1746+def main():
1747+ parser = argparse.ArgumentParser()
1748+ parser.add_argument("--network-data", "-p", type=open,
1749+ metavar="PATH", required=True)
1750+ parser.add_argument("--kind", "-k",
1751+ choices=['eni', 'network_data.json', 'yaml'],
1752+ required=True)
1753+ parser.add_argument("-d", "--directory",
1754+ metavar="PATH",
1755+ help="directory to place output in",
1756+ required=True)
1757+ parser.add_argument("-m", "--mac",
1758+ metavar="name,mac",
1759+ action='append',
1760+ help="interface name to mac mapping")
1761+ parser.add_argument("--output-kind", "-ok",
1762+ choices=['eni', 'netplan', 'sysconfig'],
1763+ required=True)
1764+ args = parser.parse_args()
1765+
1766+ if not os.path.isdir(args.directory):
1767+ os.makedirs(args.directory)
1768+
1769+ if args.mac:
1770+ known_macs = {}
1771+ for item in args.mac:
1772+ iface_name, iface_mac = item.split(",", 1)
1773+ known_macs[iface_mac] = iface_name
1774+ else:
1775+ known_macs = None
1776+
1777+ net_data = args.network_data.read()
1778+ if args.kind == "eni":
1779+ pre_ns = eni.convert_eni_data(net_data)
1780+ ns = network_state.parse_net_config_data(pre_ns)
1781+ elif args.kind == "yaml":
1782+ pre_ns = yaml.load(net_data)
1783+ if 'network' in pre_ns:
1784+ pre_ns = pre_ns.get('network')
1785+ print("Input YAML")
1786+ print(yaml.dump(pre_ns, default_flow_style=False, indent=4))
1787+ ns = network_state.parse_net_config_data(pre_ns)
1788+ else:
1789+ pre_ns = openstack.convert_net_json(
1790+ json.loads(net_data), known_macs=known_macs)
1791+ ns = network_state.parse_net_config_data(pre_ns)
1792+
1793+ if not ns:
1794+ raise RuntimeError("No valid network_state object created from"
1795+ "input data")
1796+
1797+ print("\nInternal State")
1798+ print(yaml.dump(ns, default_flow_style=False, indent=4))
1799+ if args.output_kind == "eni":
1800+ r_cls = eni.Renderer
1801+ elif args.output_kind == "netplan":
1802+ r_cls = netplan.Renderer
1803+ else:
1804+ r_cls = sysconfig.Renderer
1805+
1806+ r = r_cls()
1807+ r.render_network_state(ns, target=args.directory)
1808+
1809+
1810+if __name__ == '__main__':
1811+ main()

Subscribers

People subscribed via source and target branches