Merge ~raharper/cloud-init:rebased-netconfig-v2-passthrough into cloud-init:master
- Git
- lp:~raharper/cloud-init
- rebased-netconfig-v2-passthrough
- Merge into master
Status: | Merged |
---|---|
Merged at revision: | ef18b8ac4cf7e3dfd98830fbdb298380a192a0fc |
Proposed branch: | ~raharper/cloud-init:rebased-netconfig-v2-passthrough |
Merge into: | cloud-init:master |
Diff against target: |
1811 lines (+1443/-36) 10 files modified
cloudinit/distros/debian.py (+11/-5) cloudinit/net/eni.py (+3/-12) cloudinit/net/netplan.py (+380/-0) cloudinit/net/network_state.py (+299/-9) cloudinit/net/renderers.py (+3/-1) cloudinit/net/sysconfig.py (+15/-5) systemd/cloud-init.service (+1/-0) tests/unittests/test_distros/test_netconfig.py (+347/-4) tests/unittests/test_net.py (+301/-0) tools/net-convert.py (+83/-0) |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Server Team CI bot | continuous-integration | Needs Fixing | |
cloud-init Commiters | Pending | ||
Review via email: mp+320291@code.launchpad.net |
Commit message
Description of the change
cloudinit.net: add network config v2 parsing and rendering
Network configuration version 2 format is implemented in a package
called netplan (nplan)[1] which allows consolidated network config
for multiple network controllers.
- Add a new netplan renderer
- Update default policy, placing eni and sysconfig first
This requires explicit policy to enable netplan over eni
on systems which have both (Yakkety, Zesty, UC16)
- Allow any network state (parsed from any format cloud-init supports) to
render to v2 if system supports netplan.
- Move eni's _subnet_is_ipv6 to common code for use by other renderers
- Make sysconfig renderer always emit /etc/syconfig/
- Update cloud-init.service systemd unit to also wait on systemd-
1. https:/
- f006666... by Ryan Harper
-
(squash): eni.py whitespace damage
Server Team CI bot (server-team-bot) wrote : | # |
Server Team CI bot (server-team-bot) wrote : | # |
FAILED: Continuous integration, rev:f006666a84e
https:/
Executed test runs:
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
Click here to trigger a rebuild:
https:/
Scott Moser (smoser) wrote : | # |
Ryan, this looks good, thanks.
I'll take a look first thing monday morning on it.
Scott Moser (smoser) wrote : | # |
i have some small comments.
don't worry about fixing the 'handle_' comment, but i do worry about the confusion.
Ryan Harper (raharper) : | # |
- 3ddea40... by Ryan Harper
-
unittests/
test_distro/ test_netconfig: pass 'system_info' dict into Distro class - 2b63ec0... by Ryan Harper
-
distros/debian.py: update comment in _write_network method
Ryan Harper (raharper) wrote : | # |
I'm mostly done fixing up the small issues; one larger discussion to be had on what to do about v2 passthrough and if we get v2 but need to render eni or sysconfig.
- 4ba0163... by Ryan Harper
-
net.sysconfig.py: Use _make_header() when writing sysconfig/network
- 9bcc80e... by Ryan Harper
-
Disable v2 passthrough, instead parse so we can render network config cross distro
Server Team CI bot (server-team-bot) wrote : | # |
FAILED: Continuous integration, rev:9bcc80e67b7
https:/
Executed test runs:
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
Click here to trigger a rebuild:
https:/
- 9a10ab5... by Ryan Harper
-
network_state: v2 don't warn/error on missing mac_address in v2 input
- 1c1fc3a... by Ryan Harper
-
netplan: drop textwrap indent module, and reformat some comments/docstrings
Server Team CI bot (server-team-bot) wrote : | # |
FAILED: Continuous integration, rev:1c1fc3a8841
https:/
Executed test runs:
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
Click here to trigger a rebuild:
https:/
Preview Diff
1 | diff --git a/cloudinit/distros/debian.py b/cloudinit/distros/debian.py |
2 | index 1101f02..3f0f9d5 100644 |
3 | --- a/cloudinit/distros/debian.py |
4 | +++ b/cloudinit/distros/debian.py |
5 | @@ -42,11 +42,16 @@ NETWORK_CONF_FN = "/etc/network/interfaces.d/50-cloud-init.cfg" |
6 | class Distro(distros.Distro): |
7 | hostname_conf_fn = "/etc/hostname" |
8 | locale_conf_fn = "/etc/default/locale" |
9 | + network_conf_fn = { |
10 | + "eni": "/etc/network/interfaces.d/50-cloud-init.cfg", |
11 | + "netplan": "/etc/netplan/50-cloud-init.yaml" |
12 | + } |
13 | renderer_configs = { |
14 | - 'eni': { |
15 | - 'eni_path': NETWORK_CONF_FN, |
16 | - 'eni_header': ENI_HEADER, |
17 | - } |
18 | + "eni": {"eni_path": network_conf_fn["eni"], |
19 | + "eni_header": ENI_HEADER}, |
20 | + "netplan": {"netplan_path": network_conf_fn["netplan"], |
21 | + "netplan_header": ENI_HEADER, |
22 | + "postcmds": True} |
23 | } |
24 | |
25 | def __init__(self, name, cfg, paths): |
26 | @@ -75,7 +80,8 @@ class Distro(distros.Distro): |
27 | self.package_command('install', pkgs=pkglist) |
28 | |
29 | def _write_network(self, settings): |
30 | - util.write_file(NETWORK_CONF_FN, settings) |
31 | + # this is a legacy method, it will always write eni |
32 | + util.write_file(self.network_conf_fn["eni"], settings) |
33 | return ['all'] |
34 | |
35 | def _write_network_config(self, netconfig): |
36 | diff --git a/cloudinit/net/eni.py b/cloudinit/net/eni.py |
37 | index f471e05..9819d4f 100644 |
38 | --- a/cloudinit/net/eni.py |
39 | +++ b/cloudinit/net/eni.py |
40 | @@ -8,6 +8,7 @@ import re |
41 | from . import ParserError |
42 | |
43 | from . import renderer |
44 | +from .network_state import subnet_is_ipv6 |
45 | |
46 | from cloudinit import util |
47 | |
48 | @@ -111,16 +112,6 @@ def _iface_start_entry(iface, index, render_hwaddress=False): |
49 | return lines |
50 | |
51 | |
52 | -def _subnet_is_ipv6(subnet): |
53 | - # 'static6' or 'dhcp6' |
54 | - if subnet['type'].endswith('6'): |
55 | - # This is a request for DHCPv6. |
56 | - return True |
57 | - elif subnet['type'] == 'static' and ":" in subnet['address']: |
58 | - return True |
59 | - return False |
60 | - |
61 | - |
62 | def _parse_deb_config_data(ifaces, contents, src_dir, src_path): |
63 | """Parses the file contents, placing result into ifaces. |
64 | |
65 | @@ -370,7 +361,7 @@ class Renderer(renderer.Renderer): |
66 | iface['mode'] = subnet['type'] |
67 | iface['control'] = subnet.get('control', 'auto') |
68 | subnet_inet = 'inet' |
69 | - if _subnet_is_ipv6(subnet): |
70 | + if subnet_is_ipv6(subnet): |
71 | subnet_inet += '6' |
72 | iface['inet'] = subnet_inet |
73 | if subnet['type'].startswith('dhcp'): |
74 | @@ -486,7 +477,7 @@ class Renderer(renderer.Renderer): |
75 | def network_state_to_eni(network_state, header=None, render_hwaddress=False): |
76 | # render the provided network state, return a string of equivalent eni |
77 | eni_path = 'etc/network/interfaces' |
78 | - renderer = Renderer({ |
79 | + renderer = Renderer(config={ |
80 | 'eni_path': eni_path, |
81 | 'eni_header': header, |
82 | 'links_path_prefix': None, |
83 | diff --git a/cloudinit/net/netplan.py b/cloudinit/net/netplan.py |
84 | new file mode 100644 |
85 | index 0000000..7b3a6e8 |
86 | --- /dev/null |
87 | +++ b/cloudinit/net/netplan.py |
88 | @@ -0,0 +1,380 @@ |
89 | +# This file is part of cloud-init. See LICENSE file ... |
90 | + |
91 | +import copy |
92 | +import os |
93 | + |
94 | +from . import renderer |
95 | +from .network_state import subnet_is_ipv6 |
96 | + |
97 | +from cloudinit import util |
98 | +from cloudinit.net import SYS_CLASS_NET, get_devicelist |
99 | + |
100 | + |
101 | +NET_CONFIG_TO_V2 = { |
102 | + 'bond': {'bond-ad-select': 'ad-select', |
103 | + 'bond-arp-interval': 'arp-interval', |
104 | + 'bond-arp-ip-target': 'arp-ip-target', |
105 | + 'bond-arp-validate': 'arp-validate', |
106 | + 'bond-downdelay': 'down-delay', |
107 | + 'bond-fail-over-mac': 'fail-over-mac-policy', |
108 | + 'bond-lacp-rate': 'lacp-rate', |
109 | + 'bond-miimon': 'mii-monitor-interval', |
110 | + 'bond-min-links': 'min-links', |
111 | + 'bond-mode': 'mode', |
112 | + 'bond-num-grat-arp': 'gratuitious-arp', |
113 | + 'bond-primary-reselect': 'primary-reselect-policy', |
114 | + 'bond-updelay': 'up-delay', |
115 | + 'bond-xmit_hash_policy': 'transmit_hash_policy'}, |
116 | + 'bridge': {'bridge_ageing': 'ageing-time', |
117 | + 'bridge_bridgeprio': 'priority', |
118 | + 'bridge_fd': 'forward-delay', |
119 | + 'bridge_gcint': None, |
120 | + 'bridge_hello': 'hello-time', |
121 | + 'bridge_maxage': 'max-age', |
122 | + 'bridge_maxwait': None, |
123 | + 'bridge_pathcost': 'path-cost', |
124 | + 'bridge_portprio': None, |
125 | + 'bridge_waitport': None}} |
126 | + |
127 | + |
128 | +def indent(text, prefix): |
129 | + lines = [] |
130 | + for line in text.splitlines(True): |
131 | + lines.append(prefix + line) |
132 | + return ''.join(lines) |
133 | + |
134 | + |
135 | +def _get_params_dict_by_match(config, match): |
136 | + return dict((key, value) for (key, value) in config.items() |
137 | + if key.startswith(match)) |
138 | + |
139 | + |
140 | +def _extract_addresses(config, entry): |
141 | + """This method parse a cloudinit.net.network_state dictionary (config) and |
142 | + maps netstate keys/values into a dictionary (entry) to represent |
143 | + netplan yaml. |
144 | + |
145 | + An example config dictionary might look like: |
146 | + |
147 | + {'mac_address': '52:54:00:12:34:00', |
148 | + 'name': 'interface0', |
149 | + 'subnets': [ |
150 | + {'address': '192.168.1.2/24', |
151 | + 'mtu': 1501, |
152 | + 'type': 'static'}, |
153 | + {'address': '2001:4800:78ff:1b:be76:4eff:fe06:1000", |
154 | + 'mtu': 1480, |
155 | + 'netmask': 64, |
156 | + 'type': 'static'}], |
157 | + 'type: physical' |
158 | + } |
159 | + |
160 | + An entry dictionary looks like: |
161 | + |
162 | + {'set-name': 'interface0', |
163 | + 'match': {'macaddress': '52:54:00:12:34:00'}, |
164 | + 'mtu': 1501} |
165 | + |
166 | + After modification returns |
167 | + |
168 | + {'set-name': 'interface0', |
169 | + 'match': {'macaddress': '52:54:00:12:34:00'}, |
170 | + 'mtu': 1501, |
171 | + 'address': ['192.168.1.2/24', '2001:4800:78ff:1b:be76:4eff:fe06:1000"], |
172 | + 'mtu6': 1480} |
173 | + |
174 | + """ |
175 | + |
176 | + def _listify(obj, token=' '): |
177 | + "Helper to convert strings to list of strings, handle single string" |
178 | + if not obj or type(obj) not in [str]: |
179 | + return obj |
180 | + if token in obj: |
181 | + return obj.split(token) |
182 | + else: |
183 | + return [obj, ] |
184 | + |
185 | + addresses = [] |
186 | + routes = [] |
187 | + nameservers = [] |
188 | + searchdomains = [] |
189 | + subnets = config.get('subnets', []) |
190 | + if subnets is None: |
191 | + subnets = [] |
192 | + for subnet in subnets: |
193 | + sn_type = subnet.get('type') |
194 | + if sn_type.startswith('dhcp'): |
195 | + if sn_type == 'dhcp': |
196 | + sn_type += '4' |
197 | + entry.update({sn_type: True}) |
198 | + elif sn_type in ['static']: |
199 | + addr = "%s" % subnet.get('address') |
200 | + if 'netmask' in subnet: |
201 | + addr += "/%s" % subnet.get('netmask') |
202 | + if 'gateway' in subnet and subnet.get('gateway'): |
203 | + gateway = subnet.get('gateway') |
204 | + if ":" in gateway: |
205 | + entry.update({'gateway6': gateway}) |
206 | + else: |
207 | + entry.update({'gateway4': gateway}) |
208 | + if 'dns_nameservers' in subnet: |
209 | + nameservers += _listify(subnet.get('dns_nameservers', [])) |
210 | + if 'dns_search' in subnet: |
211 | + searchdomains += _listify(subnet.get('dns_search', [])) |
212 | + if 'mtu' in subnet: |
213 | + mtukey = 'mtu' |
214 | + if subnet_is_ipv6(subnet): |
215 | + mtukey += '6' |
216 | + entry.update({mtukey: subnet.get('mtu')}) |
217 | + for route in subnet.get('routes', []): |
218 | + to_net = "%s/%s" % (route.get('network'), |
219 | + route.get('netmask')) |
220 | + route = { |
221 | + 'via': route.get('gateway'), |
222 | + 'to': to_net, |
223 | + } |
224 | + if 'metric' in route: |
225 | + route.update({'metric': route.get('metric', 100)}) |
226 | + routes.append(route) |
227 | + |
228 | + addresses.append(addr) |
229 | + |
230 | + if len(addresses) > 0: |
231 | + entry.update({'addresses': addresses}) |
232 | + if len(routes) > 0: |
233 | + entry.update({'routes': routes}) |
234 | + if len(nameservers) > 0: |
235 | + ns = {'addresses': nameservers} |
236 | + entry.update({'nameservers': ns}) |
237 | + if len(searchdomains) > 0: |
238 | + ns = entry.get('nameservers', {}) |
239 | + ns.update({'search': searchdomains}) |
240 | + entry.update({'nameservers': ns}) |
241 | + |
242 | + |
243 | +def _extract_bond_slaves_by_name(interfaces, entry, bond_master): |
244 | + bond_slave_names = sorted([name for (name, cfg) in interfaces.items() |
245 | + if cfg.get('bond-master', None) == bond_master]) |
246 | + if len(bond_slave_names) > 0: |
247 | + entry.update({'interfaces': bond_slave_names}) |
248 | + |
249 | + |
250 | +class Renderer(renderer.Renderer): |
251 | + """Renders network information in a /etc/netplan/network.yaml format.""" |
252 | + |
253 | + NETPLAN_GENERATE = ['netplan', 'generate'] |
254 | + |
255 | + def __init__(self, config=None): |
256 | + if not config: |
257 | + config = {} |
258 | + self.netplan_path = config.get('netplan_path', |
259 | + 'etc/netplan/50-cloud-init.yaml') |
260 | + self.netplan_header = config.get('netplan_header', None) |
261 | + self._postcmds = config.get('postcmds', False) |
262 | + |
263 | + def render_network_state(self, target, network_state): |
264 | + # check network state for version |
265 | + # if v2, then extract network_state.config |
266 | + # else render_v2_from_state |
267 | + fpnplan = os.path.join(target, self.netplan_path) |
268 | + util.ensure_dir(os.path.dirname(fpnplan)) |
269 | + header = self.netplan_header if self.netplan_header else "" |
270 | + |
271 | + # render from state |
272 | + content = self._render_content(network_state) |
273 | + |
274 | + # ensure we poke udev to run net_setup_link |
275 | + if not header.endswith("\n"): |
276 | + header += "\n" |
277 | + util.write_file(fpnplan, header + content) |
278 | + |
279 | + self._netplan_generate(run=self._postcmds) |
280 | + self._net_setup_link(run=self._postcmds) |
281 | + |
282 | + def _netplan_generate(self, run=False): |
283 | + if not run: |
284 | + print("netplan postcmd disabled") |
285 | + return |
286 | + util.subp(self.NETPLAN_GENERATE, capture=True) |
287 | + |
288 | + def _net_setup_link(self, run=False): |
289 | + """To ensure device link properties are applied, we poke |
290 | + udev to re-evaluate networkd .link files and call |
291 | + the setup_link udev builtin command |
292 | + """ |
293 | + if not run: |
294 | + print("netsetup postcmd disabled") |
295 | + return |
296 | + setup_lnk = ['udevadm', 'test-builtin', 'net_setup_link'] |
297 | + for cmd in [setup_lnk + [SYS_CLASS_NET + iface] |
298 | + for iface in get_devicelist() if |
299 | + os.path.islink(SYS_CLASS_NET + iface)]: |
300 | + print(cmd) |
301 | + util.subp(cmd, capture=True) |
302 | + |
303 | + def _render_content(self, network_state): |
304 | + print('rendering v2 for victory!') |
305 | + ethernets = {} |
306 | + wifis = {} |
307 | + bridges = {} |
308 | + bonds = {} |
309 | + vlans = {} |
310 | + content = [] |
311 | + |
312 | + interfaces = network_state._network_state.get('interfaces', []) |
313 | + |
314 | + nameservers = network_state.dns_nameservers |
315 | + searchdomains = network_state.dns_searchdomains |
316 | + |
317 | + for config in network_state.iter_interfaces(): |
318 | + ifname = config.get('name') |
319 | + # filter None entries up front so we can do simple if key in dict |
320 | + ifcfg = dict((key, value) for (key, value) in config.items() |
321 | + if value) |
322 | + |
323 | + if_type = ifcfg.get('type') |
324 | + if if_type == 'physical': |
325 | + # required_keys = ['name', 'mac_address'] |
326 | + eth = { |
327 | + 'set-name': ifname, |
328 | + 'match': ifcfg.get('match', None), |
329 | + } |
330 | + if eth['match'] is None: |
331 | + macaddr = ifcfg.get('mac_address', None) |
332 | + if macaddr is not None: |
333 | + eth['match'] = {'macaddress': macaddr.lower()} |
334 | + else: |
335 | + del eth['match'] |
336 | + del eth['set-name'] |
337 | + if 'mtu' in ifcfg: |
338 | + eth['mtu'] = ifcfg.get('mtu') |
339 | + |
340 | + _extract_addresses(ifcfg, eth) |
341 | + ethernets.update({ifname: eth}) |
342 | + |
343 | + elif if_type == 'bond': |
344 | + # required_keys = ['name', 'bond_interfaces'] |
345 | + bond = {} |
346 | + bond_config = {} |
347 | + # extract bond params and drop the bond_ prefix as it's |
348 | + # redundent in v2 yaml format |
349 | + v2_bond_map = NET_CONFIG_TO_V2.get('bond') |
350 | + for match in ['bond_', 'bond-']: |
351 | + bond_params = _get_params_dict_by_match(ifcfg, match) |
352 | + for (param, value) in bond_params.items(): |
353 | + newname = v2_bond_map.get(param) |
354 | + if newname is None: |
355 | + continue |
356 | + bond_config.update({newname: value}) |
357 | + |
358 | + if len(bond_config) > 0: |
359 | + bond.update({'parameters': bond_config}) |
360 | + slave_interfaces = ifcfg.get('bond-slaves') |
361 | + if slave_interfaces == 'none': |
362 | + _extract_bond_slaves_by_name(interfaces, bond, ifname) |
363 | + _extract_addresses(ifcfg, bond) |
364 | + bonds.update({ifname: bond}) |
365 | + |
366 | + elif if_type == 'bridge': |
367 | + # required_keys = ['name', 'bridge_ports'] |
368 | + ports = sorted(copy.copy(ifcfg.get('bridge_ports'))) |
369 | + bridge = { |
370 | + 'interfaces': ports, |
371 | + } |
372 | + # extract bridge params and drop the bridge prefix as it's |
373 | + # redundent in v2 yaml format |
374 | + match_prefix = 'bridge_' |
375 | + params = _get_params_dict_by_match(ifcfg, match_prefix) |
376 | + br_config = {} |
377 | + |
378 | + # v2 yaml uses different names for the keys |
379 | + # and at least one value format change |
380 | + v2_bridge_map = NET_CONFIG_TO_V2.get('bridge') |
381 | + for (param, value) in params.items(): |
382 | + newname = v2_bridge_map.get(param) |
383 | + if newname is None: |
384 | + continue |
385 | + br_config.update({newname: value}) |
386 | + if newname == 'path-cost': |
387 | + # <interface> <cost> -> <interface>: int(<cost>) |
388 | + newvalue = {} |
389 | + for costval in value: |
390 | + (port, cost) = costval.split() |
391 | + newvalue[port] = int(cost) |
392 | + br_config.update({newname: newvalue}) |
393 | + if len(br_config) > 0: |
394 | + bridge.update({'parameters': br_config}) |
395 | + _extract_addresses(ifcfg, bridge) |
396 | + bridges.update({ifname: bridge}) |
397 | + |
398 | + elif if_type == 'vlan': |
399 | + # required_keys = ['name', 'vlan_id', 'vlan-raw-device'] |
400 | + vlan = { |
401 | + 'id': ifcfg.get('vlan_id'), |
402 | + 'link': ifcfg.get('vlan-raw-device') |
403 | + } |
404 | + |
405 | + _extract_addresses(ifcfg, vlan) |
406 | + vlans.update({ifname: vlan}) |
407 | + |
408 | + # inject global nameserver values under each physical interface |
409 | + if nameservers: |
410 | + for _eth, cfg in ethernets.items(): |
411 | + nscfg = cfg.get('nameservers', {}) |
412 | + addresses = nscfg.get('addresses', []) |
413 | + addresses += nameservers |
414 | + nscfg.update({'addresses': addresses}) |
415 | + cfg.update({'nameservers': nscfg}) |
416 | + |
417 | + if searchdomains: |
418 | + for _eth, cfg in ethernets.items(): |
419 | + nscfg = cfg.get('nameservers', {}) |
420 | + search = nscfg.get('search', []) |
421 | + search += searchdomains |
422 | + nscfg.update({'search': search}) |
423 | + cfg.update({'nameservers': nscfg}) |
424 | + |
425 | + # workaround yaml dictionary key sorting when dumping |
426 | + def _render_section(name, section): |
427 | + if section: |
428 | + dump = util.yaml_dumps({name: section}, |
429 | + explicit_start=False, |
430 | + explicit_end=False) |
431 | + txt = indent(dump, ' ' * 4) |
432 | + return [txt] |
433 | + return [] |
434 | + |
435 | + content.append("network:\n version: 2\n") |
436 | + content += _render_section('ethernets', ethernets) |
437 | + content += _render_section('wifis', wifis) |
438 | + content += _render_section('bonds', bonds) |
439 | + content += _render_section('bridges', bridges) |
440 | + content += _render_section('vlans', vlans) |
441 | + |
442 | + return "".join(content) |
443 | + |
444 | + |
445 | +def available(target=None): |
446 | + expected = ['netplan'] |
447 | + search = ['/usr/sbin', '/sbin'] |
448 | + for p in expected: |
449 | + if not util.which(p, search=search, target=target): |
450 | + return False |
451 | + return True |
452 | + |
453 | + |
454 | +def network_state_to_netplan(network_state, header=None): |
455 | + # render the provided network state, return a string of equivalent eni |
456 | + netplan_path = 'etc/network/50-cloud-init.yaml' |
457 | + renderer = Renderer({ |
458 | + 'netplan_path': netplan_path, |
459 | + 'netplan_header': header, |
460 | + }) |
461 | + if not header: |
462 | + header = "" |
463 | + if not header.endswith("\n"): |
464 | + header += "\n" |
465 | + contents = renderer._render_content(network_state) |
466 | + return header + contents |
467 | + |
468 | +# vi: ts=4 expandtab |
469 | diff --git a/cloudinit/net/network_state.py b/cloudinit/net/network_state.py |
470 | index 90b2835..c14aae2 100644 |
471 | --- a/cloudinit/net/network_state.py |
472 | +++ b/cloudinit/net/network_state.py |
473 | @@ -1,4 +1,4 @@ |
474 | -# Copyright (C) 2013-2014 Canonical Ltd. |
475 | +# Copyright (C) 2017 Canonical Ltd. |
476 | # |
477 | # Author: Ryan Harper <ryan.harper@canonical.com> |
478 | # |
479 | @@ -18,6 +18,10 @@ NETWORK_STATE_VERSION = 1 |
480 | NETWORK_STATE_REQUIRED_KEYS = { |
481 | 1: ['version', 'config', 'network_state'], |
482 | } |
483 | +NETWORK_V2_KEY_FILTER = [ |
484 | + 'addresses', 'dhcp4', 'dhcp6', 'gateway4', 'gateway6', 'interfaces', |
485 | + 'match', 'mtu', 'nameservers', 'renderer', 'set-name', 'wakeonlan' |
486 | +] |
487 | |
488 | |
489 | def parse_net_config_data(net_config, skip_broken=True): |
490 | @@ -26,11 +30,18 @@ def parse_net_config_data(net_config, skip_broken=True): |
491 | :param net_config: curtin network config dict |
492 | """ |
493 | state = None |
494 | - if 'version' in net_config and 'config' in net_config: |
495 | - nsi = NetworkStateInterpreter(version=net_config.get('version'), |
496 | - config=net_config.get('config')) |
497 | + version = net_config.get('version') |
498 | + config = net_config.get('config') |
499 | + if version == 2: |
500 | + # v2 does not have explicit 'config' key so we |
501 | + # pass the whole net-config as-is |
502 | + config = net_config |
503 | + |
504 | + if version and config: |
505 | + nsi = NetworkStateInterpreter(version=version, config=config) |
506 | nsi.parse_config(skip_broken=skip_broken) |
507 | - state = nsi.network_state |
508 | + state = nsi.get_network_state() |
509 | + |
510 | return state |
511 | |
512 | |
513 | @@ -106,6 +117,7 @@ class NetworkState(object): |
514 | def __init__(self, network_state, version=NETWORK_STATE_VERSION): |
515 | self._network_state = copy.deepcopy(network_state) |
516 | self._version = version |
517 | + self.use_ipv6 = network_state.get('use_ipv6', False) |
518 | |
519 | @property |
520 | def version(self): |
521 | @@ -152,7 +164,8 @@ class NetworkStateInterpreter(object): |
522 | 'dns': { |
523 | 'nameservers': [], |
524 | 'search': [], |
525 | - } |
526 | + }, |
527 | + 'use_ipv6': False, |
528 | } |
529 | |
530 | def __init__(self, version=NETWORK_STATE_VERSION, config=None): |
531 | @@ -165,6 +178,14 @@ class NetworkStateInterpreter(object): |
532 | def network_state(self): |
533 | return NetworkState(self._network_state, version=self._version) |
534 | |
535 | + @property |
536 | + def use_ipv6(self): |
537 | + return self._network_state.get('use_ipv6') |
538 | + |
539 | + @use_ipv6.setter |
540 | + def use_ipv6(self, val): |
541 | + self._network_state.update({'use_ipv6': val}) |
542 | + |
543 | def dump(self): |
544 | state = { |
545 | 'version': self._version, |
546 | @@ -192,8 +213,22 @@ class NetworkStateInterpreter(object): |
547 | def dump_network_state(self): |
548 | return util.yaml_dumps(self._network_state) |
549 | |
550 | + def as_dict(self): |
551 | + return {'version': self.version, 'config': self.config} |
552 | + |
553 | + def get_network_state(self): |
554 | + ns = self.network_state |
555 | + return ns |
556 | + |
557 | def parse_config(self, skip_broken=True): |
558 | - # rebuild network state |
559 | + if self._version == 1: |
560 | + self.parse_config_v1(skip_broken=skip_broken) |
561 | + self._parsed = True |
562 | + elif self._version == 2: |
563 | + self.parse_config_v2(skip_broken=skip_broken) |
564 | + self._parsed = True |
565 | + |
566 | + def parse_config_v1(self, skip_broken=True): |
567 | for command in self._config: |
568 | command_type = command['type'] |
569 | try: |
570 | @@ -211,6 +246,26 @@ class NetworkStateInterpreter(object): |
571 | exc_info=True) |
572 | LOG.debug(self.dump_network_state()) |
573 | |
574 | + def parse_config_v2(self, skip_broken=True): |
575 | + for command_type, command in self._config.items(): |
576 | + if command_type == 'version': |
577 | + continue |
578 | + try: |
579 | + handler = self.command_handlers[command_type] |
580 | + except KeyError: |
581 | + raise RuntimeError("No handler found for" |
582 | + " command '%s'" % command_type) |
583 | + try: |
584 | + handler(self, command) |
585 | + self._v2_common(command) |
586 | + except InvalidCommand: |
587 | + if not skip_broken: |
588 | + raise |
589 | + else: |
590 | + LOG.warn("Skipping invalid command: %s", command, |
591 | + exc_info=True) |
592 | + LOG.debug(self.dump_network_state()) |
593 | + |
594 | @ensure_command_keys(['name']) |
595 | def handle_loopback(self, command): |
596 | return self.handle_physical(command) |
597 | @@ -238,11 +293,16 @@ class NetworkStateInterpreter(object): |
598 | if subnets: |
599 | for subnet in subnets: |
600 | if subnet['type'] == 'static': |
601 | + if ':' in subnet['address']: |
602 | + self.use_ipv6 = True |
603 | if 'netmask' in subnet and ':' in subnet['address']: |
604 | subnet['netmask'] = mask2cidr(subnet['netmask']) |
605 | for route in subnet.get('routes', []): |
606 | if 'netmask' in route: |
607 | route['netmask'] = mask2cidr(route['netmask']) |
608 | + elif subnet['type'].endswith('6'): |
609 | + self.use_ipv6 = True |
610 | + |
611 | iface.update({ |
612 | 'name': command.get('name'), |
613 | 'type': command.get('type'), |
614 | @@ -327,7 +387,7 @@ class NetworkStateInterpreter(object): |
615 | bond_if.update({param: val}) |
616 | self._network_state['interfaces'].update({ifname: bond_if}) |
617 | |
618 | - @ensure_command_keys(['name', 'bridge_interfaces', 'params']) |
619 | + @ensure_command_keys(['name', 'bridge_interfaces']) |
620 | def handle_bridge(self, command): |
621 | ''' |
622 | auto br0 |
623 | @@ -373,7 +433,7 @@ class NetworkStateInterpreter(object): |
624 | self.handle_physical(command) |
625 | iface = interfaces.get(command.get('name'), {}) |
626 | iface['bridge_ports'] = command['bridge_interfaces'] |
627 | - for param, val in command.get('params').items(): |
628 | + for param, val in command.get('params', {}).items(): |
629 | iface.update({param: val}) |
630 | |
631 | interfaces.update({iface['name']: iface}) |
632 | @@ -407,6 +467,236 @@ class NetworkStateInterpreter(object): |
633 | } |
634 | routes.append(route) |
635 | |
636 | + # V2 handlers |
637 | + def handle_bonds(self, command): |
638 | + ''' |
639 | + v2_command = { |
640 | + bond0: { |
641 | + 'interfaces': ['interface0', 'interface1'], |
642 | + 'miimon': 100, |
643 | + 'mode': '802.3ad', |
644 | + 'xmit_hash_policy': 'layer3+4'}, |
645 | + bond1: { |
646 | + 'bond-slaves': ['interface2', 'interface7'], |
647 | + 'mode': 1 |
648 | + } |
649 | + } |
650 | + |
651 | + v1_command = { |
652 | + 'type': 'bond' |
653 | + 'name': 'bond0', |
654 | + 'bond_interfaces': [interface0, interface1], |
655 | + 'params': { |
656 | + 'bond-mode': '802.3ad', |
657 | + 'bond_miimon: 100, |
658 | + 'bond_xmit_hash_policy': 'layer3+4', |
659 | + } |
660 | + } |
661 | + |
662 | + ''' |
663 | + self._handle_bond_bridge(command, cmd_type='bond') |
664 | + |
665 | + def handle_bridges(self, command): |
666 | + |
667 | + ''' |
668 | + v2_command = { |
669 | + br0: { |
670 | + 'interfaces': ['interface0', 'interface1'], |
671 | + 'fd': 0, |
672 | + 'stp': 'off', |
673 | + 'maxwait': 0, |
674 | + } |
675 | + } |
676 | + |
677 | + v1_command = { |
678 | + 'type': 'bridge' |
679 | + 'name': 'br0', |
680 | + 'bridge_interfaces': [interface0, interface1], |
681 | + 'params': { |
682 | + 'bridge_stp': 'off', |
683 | + 'bridge_fd: 0, |
684 | + 'bridge_maxwait': 0 |
685 | + } |
686 | + } |
687 | + |
688 | + ''' |
689 | + self._handle_bond_bridge(command, cmd_type='bridge') |
690 | + |
691 | + def handle_ethernets(self, command): |
692 | + ''' |
693 | + ethernets: |
694 | + eno1: |
695 | + match: |
696 | + macaddress: 00:11:22:33:44:55 |
697 | + wakeonlan: true |
698 | + dhcp4: true |
699 | + dhcp6: false |
700 | + addresses: |
701 | + - 192.168.14.2/24 |
702 | + - 2001:1::1/64 |
703 | + gateway4: 192.168.14.1 |
704 | + gateway6: 2001:1::2 |
705 | + nameservers: |
706 | + search: [foo.local, bar.local] |
707 | + addresses: [8.8.8.8, 8.8.4.4] |
708 | + lom: |
709 | + match: |
710 | + driver: ixgbe |
711 | + set-name: lom1 |
712 | + dhcp6: true |
713 | + switchports: |
714 | + match: |
715 | + name: enp2* |
716 | + mtu: 1280 |
717 | + |
718 | + command = { |
719 | + 'type': 'physical', |
720 | + 'mac_address': 'c0:d6:9f:2c:e8:80', |
721 | + 'name': 'eth0', |
722 | + 'subnets': [ |
723 | + {'type': 'dhcp4'} |
724 | + ] |
725 | + } |
726 | + ''' |
727 | + for eth, cfg in command.items(): |
728 | + phy_cmd = { |
729 | + 'type': 'physical', |
730 | + 'name': cfg.get('set-name', eth), |
731 | + } |
732 | + |
733 | + for key in ['mtu', 'match', 'wakeonlan']: |
734 | + if key in cfg: |
735 | + phy_cmd.update({key: cfg.get(key)}) |
736 | + |
737 | + subnets = self._v2_to_v1_ipcfg(cfg) |
738 | + if len(subnets) > 0: |
739 | + phy_cmd.update({'subnets': subnets}) |
740 | + |
741 | + LOG.debug('v2(ethernets) -> v1(physical):\n%s', phy_cmd) |
742 | + self.handle_physical(phy_cmd) |
743 | + |
744 | + def handle_vlans(self, command): |
745 | + ''' |
746 | + v2_vlans = { |
747 | + 'eth0.123': { |
748 | + 'id': 123, |
749 | + 'link': 'eth0', |
750 | + 'dhcp4': True, |
751 | + } |
752 | + } |
753 | + |
754 | + v1_command = { |
755 | + 'type': 'vlan', |
756 | + 'name': 'eth0.123', |
757 | + 'vlan_link': 'eth0', |
758 | + 'vlan_id': 123, |
759 | + 'subnets': [{'type': 'dhcp4'}], |
760 | + } |
761 | + ''' |
762 | + for vlan, cfg in command.items(): |
763 | + vlan_cmd = { |
764 | + 'type': 'vlan', |
765 | + 'name': vlan, |
766 | + 'vlan_id': cfg.get('id'), |
767 | + 'vlan_link': cfg.get('link'), |
768 | + } |
769 | + subnets = self._v2_to_v1_ipcfg(cfg) |
770 | + if len(subnets) > 0: |
771 | + vlan_cmd.update({'subnets': subnets}) |
772 | + LOG.debug('v2(vlans) -> v1(vlan):\n%s', vlan_cmd) |
773 | + self.handle_vlan(vlan_cmd) |
774 | + |
775 | + def handle_wifis(self, command): |
776 | + raise NotImplemented('NetworkState V2: Skipping wifi configuration') |
777 | + |
778 | + def _v2_common(self, cfg): |
779 | + LOG.debug('v2_common: handling config:\n%s', cfg) |
780 | + if 'nameservers' in cfg: |
781 | + search = cfg.get('nameservers').get('search', []) |
782 | + dns = cfg.get('nameservers').get('addresses', []) |
783 | + name_cmd = {'type': 'nameserver'} |
784 | + if len(search) > 0: |
785 | + name_cmd.update({'search': search}) |
786 | + if len(dns) > 0: |
787 | + name_cmd.update({'addresses': dns}) |
788 | + LOG.debug('v2(nameserver) -> v1(nameserver):\n%s', name_cmd) |
789 | + self.handle_nameserver(name_cmd) |
790 | + |
791 | + def _handle_bond_bridge(self, command, cmd_type=None): |
792 | + """Common handler for bond and bridge types""" |
793 | + for item_name, item_cfg in command.items(): |
794 | + item_params = dict((key, value) for (key, value) in |
795 | + item_cfg.items() if key not in |
796 | + NETWORK_V2_KEY_FILTER) |
797 | + v1_cmd = { |
798 | + 'type': cmd_type, |
799 | + 'name': item_name, |
800 | + cmd_type + '_interfaces': item_cfg.get('interfaces'), |
801 | + 'params': item_params, |
802 | + } |
803 | + subnets = self._v2_to_v1_ipcfg(item_cfg) |
804 | + if len(subnets) > 0: |
805 | + v1_cmd.update({'subnets': subnets}) |
806 | + |
807 | + LOG.debug('v2(%ss) -> v1(%s):\n%s', cmd_type, cmd_type, v1_cmd) |
808 | + self.handle_bridge(v1_cmd) |
809 | + |
810 | + def _v2_to_v1_ipcfg(self, cfg): |
811 | + """Common ipconfig extraction from v2 to v1 subnets array.""" |
812 | + |
813 | + subnets = [] |
814 | + if 'dhcp4' in cfg: |
815 | + subnets.append({'type': 'dhcp4'}) |
816 | + if 'dhcp6' in cfg: |
817 | + self.use_ipv6 = True |
818 | + subnets.append({'type': 'dhcp6'}) |
819 | + |
820 | + gateway4 = None |
821 | + gateway6 = None |
822 | + for address in cfg.get('addresses', []): |
823 | + subnet = { |
824 | + 'type': 'static', |
825 | + 'address': address, |
826 | + } |
827 | + |
828 | + routes = [] |
829 | + for route in cfg.get('routes', []): |
830 | + route_addr = route.get('to') |
831 | + if "/" in route_addr: |
832 | + route_addr, route_cidr = route_addr.split("/") |
833 | + route_netmask = cidr2mask(route_cidr) |
834 | + subnet_route = { |
835 | + 'address': route_addr, |
836 | + 'netmask': route_netmask, |
837 | + 'gateway': route.get('via') |
838 | + } |
839 | + routes.append(subnet_route) |
840 | + if len(routes) > 0: |
841 | + subnet.update({'routes': routes}) |
842 | + |
843 | + if ":" in address: |
844 | + if 'gateway6' in cfg and gateway6 is None: |
845 | + gateway6 = cfg.get('gateway6') |
846 | + subnet.update({'gateway': gateway6}) |
847 | + else: |
848 | + if 'gateway4' in cfg and gateway4 is None: |
849 | + gateway4 = cfg.get('gateway4') |
850 | + subnet.update({'gateway': gateway4}) |
851 | + |
852 | + subnets.append(subnet) |
853 | + return subnets |
854 | + |
855 | + |
856 | +def subnet_is_ipv6(subnet): |
857 | + """ Common helper for checking network_state subnets for ipv6""" |
858 | + # 'static6' or 'dhcp6' |
859 | + if subnet['type'].endswith('6'): |
860 | + # This is a request for DHCPv6. |
861 | + return True |
862 | + elif subnet['type'] == 'static' and ":" in subnet['address']: |
863 | + return True |
864 | + return False |
865 | + |
866 | |
867 | def cidr2mask(cidr): |
868 | mask = [0, 0, 0, 0] |
869 | diff --git a/cloudinit/net/renderers.py b/cloudinit/net/renderers.py |
870 | index 5ad8455..5117b4a 100644 |
871 | --- a/cloudinit/net/renderers.py |
872 | +++ b/cloudinit/net/renderers.py |
873 | @@ -1,15 +1,17 @@ |
874 | # This file is part of cloud-init. See LICENSE file for license information. |
875 | |
876 | from . import eni |
877 | +from . import netplan |
878 | from . import RendererNotFoundError |
879 | from . import sysconfig |
880 | |
881 | NAME_TO_RENDERER = { |
882 | "eni": eni, |
883 | + "netplan": netplan, |
884 | "sysconfig": sysconfig, |
885 | } |
886 | |
887 | -DEFAULT_PRIORITY = ["eni", "sysconfig"] |
888 | +DEFAULT_PRIORITY = ["eni", "sysconfig", "netplan"] |
889 | |
890 | |
891 | def search(priority=None, target=None, first=False): |
892 | diff --git a/cloudinit/net/sysconfig.py b/cloudinit/net/sysconfig.py |
893 | index 117b515..504e4d0 100644 |
894 | --- a/cloudinit/net/sysconfig.py |
895 | +++ b/cloudinit/net/sysconfig.py |
896 | @@ -9,6 +9,7 @@ from cloudinit.distros.parsers import resolv_conf |
897 | from cloudinit import util |
898 | |
899 | from . import renderer |
900 | +from .network_state import subnet_is_ipv6 |
901 | |
902 | |
903 | def _make_header(sep='#'): |
904 | @@ -194,7 +195,7 @@ class Renderer(renderer.Renderer): |
905 | def __init__(self, config=None): |
906 | if not config: |
907 | config = {} |
908 | - self.sysconf_dir = config.get('sysconf_dir', 'etc/sysconfig/') |
909 | + self.sysconf_dir = config.get('sysconf_dir', 'etc/sysconfig') |
910 | self.netrules_path = config.get( |
911 | 'netrules_path', 'etc/udev/rules.d/70-persistent-net.rules') |
912 | self.dns_path = config.get('dns_path', 'etc/resolv.conf') |
913 | @@ -220,7 +221,7 @@ class Renderer(renderer.Renderer): |
914 | iface_cfg['BOOTPROTO'] = 'dhcp' |
915 | elif subnet_type == 'static': |
916 | iface_cfg['BOOTPROTO'] = 'static' |
917 | - if subnet.get('ipv6'): |
918 | + if subnet_is_ipv6(subnet): |
919 | iface_cfg['IPV6ADDR'] = subnet['address'] |
920 | iface_cfg['IPV6INIT'] = True |
921 | else: |
922 | @@ -390,19 +391,28 @@ class Renderer(renderer.Renderer): |
923 | return contents |
924 | |
925 | def render_network_state(self, network_state, target=None): |
926 | + file_mode = 0o644 |
927 | base_sysconf_dir = util.target_path(target, self.sysconf_dir) |
928 | for path, data in self._render_sysconfig(base_sysconf_dir, |
929 | network_state).items(): |
930 | - util.write_file(path, data) |
931 | + util.write_file(path, data, file_mode) |
932 | if self.dns_path: |
933 | dns_path = util.target_path(target, self.dns_path) |
934 | resolv_content = self._render_dns(network_state, |
935 | existing_dns_path=dns_path) |
936 | - util.write_file(dns_path, resolv_content) |
937 | + util.write_file(dns_path, resolv_content, file_mode) |
938 | if self.netrules_path: |
939 | netrules_content = self._render_persistent_net(network_state) |
940 | netrules_path = util.target_path(target, self.netrules_path) |
941 | - util.write_file(netrules_path, netrules_content) |
942 | + util.write_file(netrules_path, netrules_content, file_mode) |
943 | + |
944 | + # always write /etc/sysconfig/network configuration |
945 | + sysconfig_path = util.target_path(target, "etc/sysconfig/network") |
946 | + netcfg = [_make_header(), 'NETWORKING=yes'] |
947 | + if network_state.use_ipv6: |
948 | + netcfg.append('NETWORKING_IPV6=yes') |
949 | + netcfg.append('IPV6_AUTOCONF=no') |
950 | + util.write_file(sysconfig_path, "\n".join(netcfg) + "\n", file_mode) |
951 | |
952 | |
953 | def available(target=None): |
954 | diff --git a/systemd/cloud-init.service b/systemd/cloud-init.service |
955 | index fb3b918..39acc20 100644 |
956 | --- a/systemd/cloud-init.service |
957 | +++ b/systemd/cloud-init.service |
958 | @@ -5,6 +5,7 @@ Wants=cloud-init-local.service |
959 | Wants=sshd-keygen.service |
960 | Wants=sshd.service |
961 | After=cloud-init-local.service |
962 | +After=systemd-networkd-wait-online.service |
963 | After=networking.service |
964 | Before=network-online.target |
965 | Before=sshd-keygen.service |
966 | diff --git a/tests/unittests/test_distros/test_netconfig.py b/tests/unittests/test_distros/test_netconfig.py |
967 | index bde3bb5..d187c9d 100644 |
968 | --- a/tests/unittests/test_distros/test_netconfig.py |
969 | +++ b/tests/unittests/test_distros/test_netconfig.py |
970 | @@ -19,6 +19,7 @@ from cloudinit.distros.parsers.sys_conf import SysConf |
971 | from cloudinit import helpers |
972 | from cloudinit import settings |
973 | from cloudinit import util |
974 | +from cloudinit.net import eni |
975 | |
976 | |
977 | BASE_NET_CFG = ''' |
978 | @@ -28,10 +29,10 @@ iface lo inet loopback |
979 | auto eth0 |
980 | iface eth0 inet static |
981 | address 192.168.1.5 |
982 | - netmask 255.255.255.0 |
983 | - network 192.168.0.0 |
984 | broadcast 192.168.1.0 |
985 | gateway 192.168.1.254 |
986 | + netmask 255.255.255.0 |
987 | + network 192.168.0.0 |
988 | |
989 | auto eth1 |
990 | iface eth1 inet dhcp |
991 | @@ -67,6 +68,100 @@ iface eth1 inet6 static |
992 | gateway 2607:f0d0:1002:0011::1 |
993 | ''' |
994 | |
995 | +V1_NET_CFG = {'config': [{'name': 'eth0', |
996 | + |
997 | + 'subnets': [{'address': '192.168.1.5', |
998 | + 'broadcast': '192.168.1.0', |
999 | + 'gateway': '192.168.1.254', |
1000 | + 'netmask': '255.255.255.0', |
1001 | + 'type': 'static'}], |
1002 | + 'type': 'physical'}, |
1003 | + {'name': 'eth1', |
1004 | + 'subnets': [{'control': 'auto', 'type': 'dhcp4'}], |
1005 | + 'type': 'physical'}], |
1006 | + 'version': 1} |
1007 | + |
1008 | +V1_NET_CFG_OUTPUT = """ |
1009 | +# This file is generated from information provided by |
1010 | +# the datasource. Changes to it will not persist across an instance. |
1011 | +# To disable cloud-init's network configuration capabilities, write a file |
1012 | +# /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following: |
1013 | +# network: {config: disabled} |
1014 | +auto lo |
1015 | +iface lo inet loopback |
1016 | + |
1017 | +auto eth0 |
1018 | +iface eth0 inet static |
1019 | + address 192.168.1.5 |
1020 | + broadcast 192.168.1.0 |
1021 | + gateway 192.168.1.254 |
1022 | + netmask 255.255.255.0 |
1023 | + |
1024 | +auto eth1 |
1025 | +iface eth1 inet dhcp |
1026 | +""" |
1027 | + |
1028 | +V1_NET_CFG_IPV6 = {'config': [{'name': 'eth0', |
1029 | + 'subnets': [{'address': |
1030 | + '2607:f0d0:1002:0011::2', |
1031 | + 'gateway': |
1032 | + '2607:f0d0:1002:0011::1', |
1033 | + 'netmask': '64', |
1034 | + 'type': 'static'}], |
1035 | + 'type': 'physical'}, |
1036 | + {'name': 'eth1', |
1037 | + 'subnets': [{'control': 'auto', |
1038 | + 'type': 'dhcp4'}], |
1039 | + 'type': 'physical'}], |
1040 | + 'version': 1} |
1041 | + |
1042 | + |
1043 | +V1_TO_V2_NET_CFG_OUTPUT = """ |
1044 | +# This file is generated from information provided by |
1045 | +# the datasource. Changes to it will not persist across an instance. |
1046 | +# To disable cloud-init's network configuration capabilities, write a file |
1047 | +# /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following: |
1048 | +# network: {config: disabled} |
1049 | +network: |
1050 | + version: 2 |
1051 | + ethernets: |
1052 | + eth0: |
1053 | + addresses: |
1054 | + - 192.168.1.5/255.255.255.0 |
1055 | + gateway4: 192.168.1.254 |
1056 | + eth1: |
1057 | + dhcp4: true |
1058 | +""" |
1059 | + |
1060 | +V2_NET_CFG = { |
1061 | + 'ethernets': { |
1062 | + 'eth7': { |
1063 | + 'addresses': ['192.168.1.5/255.255.255.0'], |
1064 | + 'gateway4': '192.168.1.254'}, |
1065 | + 'eth9': { |
1066 | + 'dhcp4': True} |
1067 | + }, |
1068 | + 'version': 2 |
1069 | +} |
1070 | + |
1071 | + |
1072 | +V2_TO_V2_NET_CFG_OUTPUT = """ |
1073 | +# This file is generated from information provided by |
1074 | +# the datasource. Changes to it will not persist across an instance. |
1075 | +# To disable cloud-init's network configuration capabilities, write a file |
1076 | +# /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following: |
1077 | +# network: {config: disabled} |
1078 | +network: |
1079 | + version: 2 |
1080 | + ethernets: |
1081 | + eth7: |
1082 | + addresses: |
1083 | + - 192.168.1.5/255.255.255.0 |
1084 | + gateway4: 192.168.1.254 |
1085 | + eth9: |
1086 | + dhcp4: true |
1087 | +""" |
1088 | + |
1089 | |
1090 | class WriteBuffer(object): |
1091 | def __init__(self): |
1092 | @@ -83,12 +178,14 @@ class WriteBuffer(object): |
1093 | |
1094 | class TestNetCfgDistro(TestCase): |
1095 | |
1096 | - def _get_distro(self, dname): |
1097 | + def _get_distro(self, dname, renderers=None): |
1098 | cls = distros.fetch(dname) |
1099 | cfg = settings.CFG_BUILTIN |
1100 | cfg['system_info']['distro'] = dname |
1101 | + if renderers: |
1102 | + cfg['system_info']['network'] = {'renderers': renderers} |
1103 | paths = helpers.Paths({}) |
1104 | - return cls(dname, cfg, paths) |
1105 | + return cls(dname, cfg.get('system_info'), paths) |
1106 | |
1107 | def test_simple_write_ub(self): |
1108 | ub_distro = self._get_distro('ubuntu') |
1109 | @@ -116,6 +213,107 @@ class TestNetCfgDistro(TestCase): |
1110 | self.assertEqual(str(write_buf).strip(), BASE_NET_CFG.strip()) |
1111 | self.assertEqual(write_buf.mode, 0o644) |
1112 | |
1113 | + def test_apply_network_config_eni_ub(self): |
1114 | + ub_distro = self._get_distro('ubuntu') |
1115 | + with ExitStack() as mocks: |
1116 | + write_bufs = {} |
1117 | + |
1118 | + def replace_write(filename, content, mode=0o644, omode="wb"): |
1119 | + buf = WriteBuffer() |
1120 | + buf.mode = mode |
1121 | + buf.omode = omode |
1122 | + buf.write(content) |
1123 | + write_bufs[filename] = buf |
1124 | + |
1125 | + # eni availability checks |
1126 | + mocks.enter_context( |
1127 | + mock.patch.object(util, 'which', return_value=True)) |
1128 | + mocks.enter_context( |
1129 | + mock.patch.object(eni, 'available', return_value=True)) |
1130 | + mocks.enter_context( |
1131 | + mock.patch.object(util, 'ensure_dir')) |
1132 | + mocks.enter_context( |
1133 | + mock.patch.object(util, 'write_file', replace_write)) |
1134 | + mocks.enter_context( |
1135 | + mock.patch.object(os.path, 'isfile', return_value=False)) |
1136 | + |
1137 | + ub_distro.apply_network_config(V1_NET_CFG, False) |
1138 | + |
1139 | + self.assertEqual(len(write_bufs), 2) |
1140 | + eni_name = '/etc/network/interfaces.d/50-cloud-init.cfg' |
1141 | + self.assertIn(eni_name, write_bufs) |
1142 | + write_buf = write_bufs[eni_name] |
1143 | + self.assertEqual(str(write_buf).strip(), V1_NET_CFG_OUTPUT.strip()) |
1144 | + self.assertEqual(write_buf.mode, 0o644) |
1145 | + |
1146 | + def test_apply_network_config_v1_to_netplan_ub(self): |
1147 | + renderers = ['netplan'] |
1148 | + ub_distro = self._get_distro('ubuntu', renderers=renderers) |
1149 | + with ExitStack() as mocks: |
1150 | + write_bufs = {} |
1151 | + |
1152 | + def replace_write(filename, content, mode=0o644, omode="wb"): |
1153 | + buf = WriteBuffer() |
1154 | + buf.mode = mode |
1155 | + buf.omode = omode |
1156 | + buf.write(content) |
1157 | + write_bufs[filename] = buf |
1158 | + |
1159 | + mocks.enter_context( |
1160 | + mock.patch.object(util, 'which', return_value=True)) |
1161 | + mocks.enter_context( |
1162 | + mock.patch.object(util, 'write_file', replace_write)) |
1163 | + mocks.enter_context( |
1164 | + mock.patch.object(util, 'ensure_dir')) |
1165 | + mocks.enter_context( |
1166 | + mock.patch.object(util, 'subp', return_value=(0, 0))) |
1167 | + mocks.enter_context( |
1168 | + mock.patch.object(os.path, 'isfile', return_value=False)) |
1169 | + |
1170 | + ub_distro.apply_network_config(V1_NET_CFG, False) |
1171 | + |
1172 | + self.assertEqual(len(write_bufs), 1) |
1173 | + netplan_name = '/etc/netplan/50-cloud-init.yaml' |
1174 | + self.assertIn(netplan_name, write_bufs) |
1175 | + write_buf = write_bufs[netplan_name] |
1176 | + self.assertEqual(str(write_buf).strip(), |
1177 | + V1_TO_V2_NET_CFG_OUTPUT.strip()) |
1178 | + self.assertEqual(write_buf.mode, 0o644) |
1179 | + |
1180 | + def test_apply_network_config_v2_passthrough_ub(self): |
1181 | + renderers = ['netplan'] |
1182 | + ub_distro = self._get_distro('ubuntu', renderers=renderers) |
1183 | + with ExitStack() as mocks: |
1184 | + write_bufs = {} |
1185 | + |
1186 | + def replace_write(filename, content, mode=0o644, omode="wb"): |
1187 | + buf = WriteBuffer() |
1188 | + buf.mode = mode |
1189 | + buf.omode = omode |
1190 | + buf.write(content) |
1191 | + write_bufs[filename] = buf |
1192 | + |
1193 | + mocks.enter_context( |
1194 | + mock.patch.object(util, 'which', return_value=True)) |
1195 | + mocks.enter_context( |
1196 | + mock.patch.object(util, 'write_file', replace_write)) |
1197 | + mocks.enter_context( |
1198 | + mock.patch.object(util, 'ensure_dir')) |
1199 | + mocks.enter_context( |
1200 | + mock.patch.object(util, 'subp', return_value=(0, 0))) |
1201 | + mocks.enter_context( |
1202 | + mock.patch.object(os.path, 'isfile', return_value=False)) |
1203 | + |
1204 | + ub_distro.apply_network_config(V2_NET_CFG, False) |
1205 | + |
1206 | + self.assertEqual(len(write_bufs), 1) |
1207 | + netplan_name = '/etc/netplan/50-cloud-init.yaml' |
1208 | + self.assertIn(netplan_name, write_bufs) |
1209 | + write_buf = write_bufs[netplan_name] |
1210 | + self.assertEqual(str(write_buf).strip(), |
1211 | + V2_TO_V2_NET_CFG_OUTPUT.strip()) |
1212 | + self.assertEqual(write_buf.mode, 0o644) |
1213 | + |
1214 | def assertCfgEquals(self, blob1, blob2): |
1215 | b1 = dict(SysConf(blob1.strip().splitlines())) |
1216 | b2 = dict(SysConf(blob2.strip().splitlines())) |
1217 | @@ -195,6 +393,79 @@ NETWORKING=yes |
1218 | self.assertCfgEquals(expected_buf, str(write_buf)) |
1219 | self.assertEqual(write_buf.mode, 0o644) |
1220 | |
1221 | + def test_apply_network_config_rh(self): |
1222 | + renderers = ['sysconfig'] |
1223 | + rh_distro = self._get_distro('rhel', renderers=renderers) |
1224 | + |
1225 | + write_bufs = {} |
1226 | + |
1227 | + def replace_write(filename, content, mode=0o644, omode="wb"): |
1228 | + buf = WriteBuffer() |
1229 | + buf.mode = mode |
1230 | + buf.omode = omode |
1231 | + buf.write(content) |
1232 | + write_bufs[filename] = buf |
1233 | + |
1234 | + with ExitStack() as mocks: |
1235 | + # sysconfig availability checks |
1236 | + mocks.enter_context( |
1237 | + mock.patch.object(util, 'which', return_value=True)) |
1238 | + mocks.enter_context( |
1239 | + mock.patch.object(util, 'write_file', replace_write)) |
1240 | + mocks.enter_context( |
1241 | + mock.patch.object(util, 'load_file', return_value='')) |
1242 | + mocks.enter_context( |
1243 | + mock.patch.object(os.path, 'isfile', return_value=True)) |
1244 | + |
1245 | + rh_distro.apply_network_config(V1_NET_CFG, False) |
1246 | + |
1247 | + self.assertEqual(len(write_bufs), 5) |
1248 | + |
1249 | + # eth0 |
1250 | + self.assertIn('/etc/sysconfig/network-scripts/ifcfg-eth0', |
1251 | + write_bufs) |
1252 | + write_buf = write_bufs['/etc/sysconfig/network-scripts/ifcfg-eth0'] |
1253 | + expected_buf = ''' |
1254 | +# Created by cloud-init on instance boot automatically, do not edit. |
1255 | +# |
1256 | +BOOTPROTO=static |
1257 | +DEVICE=eth0 |
1258 | +IPADDR=192.168.1.5 |
1259 | +NETMASK=255.255.255.0 |
1260 | +NM_CONTROLLED=no |
1261 | +ONBOOT=yes |
1262 | +TYPE=Ethernet |
1263 | +USERCTL=no |
1264 | +''' |
1265 | + self.assertCfgEquals(expected_buf, str(write_buf)) |
1266 | + self.assertEqual(write_buf.mode, 0o644) |
1267 | + |
1268 | + # eth1 |
1269 | + self.assertIn('/etc/sysconfig/network-scripts/ifcfg-eth1', |
1270 | + write_bufs) |
1271 | + write_buf = write_bufs['/etc/sysconfig/network-scripts/ifcfg-eth1'] |
1272 | + expected_buf = ''' |
1273 | +# Created by cloud-init on instance boot automatically, do not edit. |
1274 | +# |
1275 | +BOOTPROTO=dhcp |
1276 | +DEVICE=eth1 |
1277 | +NM_CONTROLLED=no |
1278 | +ONBOOT=yes |
1279 | +TYPE=Ethernet |
1280 | +USERCTL=no |
1281 | +''' |
1282 | + self.assertCfgEquals(expected_buf, str(write_buf)) |
1283 | + self.assertEqual(write_buf.mode, 0o644) |
1284 | + |
1285 | + self.assertIn('/etc/sysconfig/network', write_bufs) |
1286 | + write_buf = write_bufs['/etc/sysconfig/network'] |
1287 | + expected_buf = ''' |
1288 | +# Created by cloud-init v. 0.7 |
1289 | +NETWORKING=yes |
1290 | +''' |
1291 | + self.assertCfgEquals(expected_buf, str(write_buf)) |
1292 | + self.assertEqual(write_buf.mode, 0o644) |
1293 | + |
1294 | def test_write_ipv6_rhel(self): |
1295 | rh_distro = self._get_distro('rhel') |
1296 | |
1297 | @@ -274,6 +545,78 @@ IPV6_AUTOCONF=no |
1298 | self.assertCfgEquals(expected_buf, str(write_buf)) |
1299 | self.assertEqual(write_buf.mode, 0o644) |
1300 | |
1301 | + def test_apply_network_config_ipv6_rh(self): |
1302 | + renderers = ['sysconfig'] |
1303 | + rh_distro = self._get_distro('rhel', renderers=renderers) |
1304 | + |
1305 | + write_bufs = {} |
1306 | + |
1307 | + def replace_write(filename, content, mode=0o644, omode="wb"): |
1308 | + buf = WriteBuffer() |
1309 | + buf.mode = mode |
1310 | + buf.omode = omode |
1311 | + buf.write(content) |
1312 | + write_bufs[filename] = buf |
1313 | + |
1314 | + with ExitStack() as mocks: |
1315 | + mocks.enter_context( |
1316 | + mock.patch.object(util, 'which', return_value=True)) |
1317 | + mocks.enter_context( |
1318 | + mock.patch.object(util, 'write_file', replace_write)) |
1319 | + mocks.enter_context( |
1320 | + mock.patch.object(util, 'load_file', return_value='')) |
1321 | + mocks.enter_context( |
1322 | + mock.patch.object(os.path, 'isfile', return_value=True)) |
1323 | + |
1324 | + rh_distro.apply_network_config(V1_NET_CFG_IPV6, False) |
1325 | + |
1326 | + self.assertEqual(len(write_bufs), 5) |
1327 | + |
1328 | + self.assertIn('/etc/sysconfig/network-scripts/ifcfg-eth0', |
1329 | + write_bufs) |
1330 | + write_buf = write_bufs['/etc/sysconfig/network-scripts/ifcfg-eth0'] |
1331 | + expected_buf = ''' |
1332 | +# Created by cloud-init on instance boot automatically, do not edit. |
1333 | +# |
1334 | +BOOTPROTO=static |
1335 | +DEVICE=eth0 |
1336 | +IPV6ADDR=2607:f0d0:1002:0011::2 |
1337 | +IPV6INIT=yes |
1338 | +NETMASK=64 |
1339 | +NM_CONTROLLED=no |
1340 | +ONBOOT=yes |
1341 | +TYPE=Ethernet |
1342 | +USERCTL=no |
1343 | +''' |
1344 | + self.assertCfgEquals(expected_buf, str(write_buf)) |
1345 | + self.assertEqual(write_buf.mode, 0o644) |
1346 | + self.assertIn('/etc/sysconfig/network-scripts/ifcfg-eth1', |
1347 | + write_bufs) |
1348 | + write_buf = write_bufs['/etc/sysconfig/network-scripts/ifcfg-eth1'] |
1349 | + expected_buf = ''' |
1350 | +# Created by cloud-init on instance boot automatically, do not edit. |
1351 | +# |
1352 | +BOOTPROTO=dhcp |
1353 | +DEVICE=eth1 |
1354 | +NM_CONTROLLED=no |
1355 | +ONBOOT=yes |
1356 | +TYPE=Ethernet |
1357 | +USERCTL=no |
1358 | +''' |
1359 | + self.assertCfgEquals(expected_buf, str(write_buf)) |
1360 | + self.assertEqual(write_buf.mode, 0o644) |
1361 | + |
1362 | + self.assertIn('/etc/sysconfig/network', write_bufs) |
1363 | + write_buf = write_bufs['/etc/sysconfig/network'] |
1364 | + expected_buf = ''' |
1365 | +# Created by cloud-init v. 0.7 |
1366 | +NETWORKING=yes |
1367 | +NETWORKING_IPV6=yes |
1368 | +IPV6_AUTOCONF=no |
1369 | +''' |
1370 | + self.assertCfgEquals(expected_buf, str(write_buf)) |
1371 | + self.assertEqual(write_buf.mode, 0o644) |
1372 | + |
1373 | def test_simple_write_freebsd(self): |
1374 | fbsd_distro = self._get_distro('freebsd') |
1375 | |
1376 | diff --git a/tests/unittests/test_net.py b/tests/unittests/test_net.py |
1377 | index 902204a..4f07d80 100644 |
1378 | --- a/tests/unittests/test_net.py |
1379 | +++ b/tests/unittests/test_net.py |
1380 | @@ -3,6 +3,7 @@ |
1381 | from cloudinit import net |
1382 | from cloudinit.net import cmdline |
1383 | from cloudinit.net import eni |
1384 | +from cloudinit.net import netplan |
1385 | from cloudinit.net import network_state |
1386 | from cloudinit.net import renderers |
1387 | from cloudinit.net import sysconfig |
1388 | @@ -408,6 +409,41 @@ NETWORK_CONFIGS = { |
1389 | post-up route add default gw 65.61.151.37 || true |
1390 | pre-down route del default gw 65.61.151.37 || true |
1391 | """).rstrip(' '), |
1392 | + 'expected_netplan': textwrap.dedent(""" |
1393 | + network: |
1394 | + version: 2 |
1395 | + ethernets: |
1396 | + eth1: |
1397 | + match: |
1398 | + macaddress: cf:d6:af:48:e8:80 |
1399 | + nameservers: |
1400 | + addresses: |
1401 | + - 1.2.3.4 |
1402 | + - 5.6.7.8 |
1403 | + search: |
1404 | + - wark.maas |
1405 | + set-name: eth1 |
1406 | + eth99: |
1407 | + addresses: |
1408 | + - 192.168.21.3/24 |
1409 | + dhcp4: true |
1410 | + match: |
1411 | + macaddress: c0:d6:9f:2c:e8:80 |
1412 | + nameservers: |
1413 | + addresses: |
1414 | + - 8.8.8.8 |
1415 | + - 8.8.4.4 |
1416 | + - 1.2.3.4 |
1417 | + - 5.6.7.8 |
1418 | + search: |
1419 | + - barley.maas |
1420 | + - sach.maas |
1421 | + - wark.maas |
1422 | + routes: |
1423 | + - to: 0.0.0.0/0.0.0.0 |
1424 | + via: 65.61.151.37 |
1425 | + set-name: eth99 |
1426 | + """).rstrip(' '), |
1427 | 'yaml': textwrap.dedent(""" |
1428 | version: 1 |
1429 | config: |
1430 | @@ -450,6 +486,14 @@ NETWORK_CONFIGS = { |
1431 | # control-alias iface0 |
1432 | iface iface0 inet6 dhcp |
1433 | """).rstrip(' '), |
1434 | + 'expected_netplan': textwrap.dedent(""" |
1435 | + network: |
1436 | + version: 2 |
1437 | + ethernets: |
1438 | + iface0: |
1439 | + dhcp4: true |
1440 | + dhcp6: true |
1441 | + """).rstrip(' '), |
1442 | 'yaml': textwrap.dedent("""\ |
1443 | version: 1 |
1444 | config: |
1445 | @@ -524,6 +568,126 @@ iface eth0.101 inet static |
1446 | post-up route add -net 10.0.0.0 netmask 255.0.0.0 gw 11.0.0.1 metric 3 || true |
1447 | pre-down route del -net 10.0.0.0 netmask 255.0.0.0 gw 11.0.0.1 metric 3 || true |
1448 | """), |
1449 | + 'expected_netplan': textwrap.dedent(""" |
1450 | + network: |
1451 | + version: 2 |
1452 | + ethernets: |
1453 | + eth0: |
1454 | + match: |
1455 | + macaddress: c0:d6:9f:2c:e8:80 |
1456 | + nameservers: |
1457 | + addresses: |
1458 | + - 8.8.8.8 |
1459 | + - 4.4.4.4 |
1460 | + - 8.8.4.4 |
1461 | + search: |
1462 | + - barley.maas |
1463 | + - wark.maas |
1464 | + - foobar.maas |
1465 | + set-name: eth0 |
1466 | + eth1: |
1467 | + match: |
1468 | + macaddress: aa:d6:9f:2c:e8:80 |
1469 | + nameservers: |
1470 | + addresses: |
1471 | + - 8.8.8.8 |
1472 | + - 4.4.4.4 |
1473 | + - 8.8.4.4 |
1474 | + search: |
1475 | + - barley.maas |
1476 | + - wark.maas |
1477 | + - foobar.maas |
1478 | + set-name: eth1 |
1479 | + eth2: |
1480 | + match: |
1481 | + macaddress: c0:bb:9f:2c:e8:80 |
1482 | + nameservers: |
1483 | + addresses: |
1484 | + - 8.8.8.8 |
1485 | + - 4.4.4.4 |
1486 | + - 8.8.4.4 |
1487 | + search: |
1488 | + - barley.maas |
1489 | + - wark.maas |
1490 | + - foobar.maas |
1491 | + set-name: eth2 |
1492 | + eth3: |
1493 | + match: |
1494 | + macaddress: 66:bb:9f:2c:e8:80 |
1495 | + nameservers: |
1496 | + addresses: |
1497 | + - 8.8.8.8 |
1498 | + - 4.4.4.4 |
1499 | + - 8.8.4.4 |
1500 | + search: |
1501 | + - barley.maas |
1502 | + - wark.maas |
1503 | + - foobar.maas |
1504 | + set-name: eth3 |
1505 | + eth4: |
1506 | + match: |
1507 | + macaddress: 98:bb:9f:2c:e8:80 |
1508 | + nameservers: |
1509 | + addresses: |
1510 | + - 8.8.8.8 |
1511 | + - 4.4.4.4 |
1512 | + - 8.8.4.4 |
1513 | + search: |
1514 | + - barley.maas |
1515 | + - wark.maas |
1516 | + - foobar.maas |
1517 | + set-name: eth4 |
1518 | + eth5: |
1519 | + dhcp4: true |
1520 | + match: |
1521 | + macaddress: 98:bb:9f:2c:e8:8a |
1522 | + nameservers: |
1523 | + addresses: |
1524 | + - 8.8.8.8 |
1525 | + - 4.4.4.4 |
1526 | + - 8.8.4.4 |
1527 | + search: |
1528 | + - barley.maas |
1529 | + - wark.maas |
1530 | + - foobar.maas |
1531 | + set-name: eth5 |
1532 | + bonds: |
1533 | + bond0: |
1534 | + dhcp6: true |
1535 | + interfaces: |
1536 | + - eth1 |
1537 | + - eth2 |
1538 | + parameters: |
1539 | + mode: active-backup |
1540 | + bridges: |
1541 | + br0: |
1542 | + addresses: |
1543 | + - 192.168.14.2/24 |
1544 | + - 2001:1::1/64 |
1545 | + interfaces: |
1546 | + - eth3 |
1547 | + - eth4 |
1548 | + vlans: |
1549 | + bond0.200: |
1550 | + dhcp4: true |
1551 | + id: 200 |
1552 | + link: bond0 |
1553 | + eth0.101: |
1554 | + addresses: |
1555 | + - 192.168.0.2/24 |
1556 | + - 192.168.2.10/24 |
1557 | + gateway4: 192.168.0.1 |
1558 | + id: 101 |
1559 | + link: eth0 |
1560 | + nameservers: |
1561 | + addresses: |
1562 | + - 192.168.0.10 |
1563 | + - 10.23.23.134 |
1564 | + search: |
1565 | + - barley.maas |
1566 | + - sacchromyces.maas |
1567 | + - brettanomyces.maas |
1568 | + """).rstrip(' '), |
1569 | 'yaml': textwrap.dedent(""" |
1570 | version: 1 |
1571 | config: |
1572 | @@ -808,6 +972,99 @@ iface eth0 inet dhcp |
1573 | expected, dir2dict(tmp_dir)['/etc/network/interfaces']) |
1574 | |
1575 | |
1576 | +class TestNetplanNetRendering(CiTestCase): |
1577 | + |
1578 | + @mock.patch("cloudinit.net.sys_dev_path") |
1579 | + @mock.patch("cloudinit.net.read_sys_net") |
1580 | + @mock.patch("cloudinit.net.get_devicelist") |
1581 | + def test_default_generation(self, mock_get_devicelist, |
1582 | + mock_read_sys_net, |
1583 | + mock_sys_dev_path): |
1584 | + tmp_dir = self.tmp_dir() |
1585 | + _setup_test(tmp_dir, mock_get_devicelist, |
1586 | + mock_read_sys_net, mock_sys_dev_path) |
1587 | + |
1588 | + network_cfg = net.generate_fallback_config() |
1589 | + ns = network_state.parse_net_config_data(network_cfg, |
1590 | + skip_broken=False) |
1591 | + |
1592 | + render_dir = os.path.join(tmp_dir, "render") |
1593 | + os.makedirs(render_dir) |
1594 | + |
1595 | + render_target = 'netplan.yaml' |
1596 | + renderer = netplan.Renderer( |
1597 | + {'netplan_path': render_target, 'postcmds': False}) |
1598 | + renderer.render_network_state(render_dir, ns) |
1599 | + |
1600 | + self.assertTrue(os.path.exists(os.path.join(render_dir, |
1601 | + render_target))) |
1602 | + with open(os.path.join(render_dir, render_target)) as fh: |
1603 | + contents = fh.read() |
1604 | + print(contents) |
1605 | + |
1606 | + expected = """ |
1607 | +network: |
1608 | + version: 2 |
1609 | + ethernets: |
1610 | + eth1000: |
1611 | + dhcp4: true |
1612 | + match: |
1613 | + macaddress: 07-1c-c6-75-a4-be |
1614 | + set-name: eth1000 |
1615 | +""" |
1616 | + self.assertEqual(expected.lstrip(), contents.lstrip()) |
1617 | + |
1618 | + |
1619 | +class TestNetplanPostcommands(CiTestCase): |
1620 | + mycfg = { |
1621 | + 'config': [{"type": "physical", "name": "eth0", |
1622 | + "mac_address": "c0:d6:9f:2c:e8:80", |
1623 | + "subnets": [{"type": "dhcp"}]}], |
1624 | + 'version': 1} |
1625 | + |
1626 | + @mock.patch.object(netplan.Renderer, '_netplan_generate') |
1627 | + @mock.patch.object(netplan.Renderer, '_net_setup_link') |
1628 | + def test_netplan_render_calls_postcmds(self, mock_netplan_generate, |
1629 | + mock_net_setup_link): |
1630 | + tmp_dir = self.tmp_dir() |
1631 | + ns = network_state.parse_net_config_data(self.mycfg, |
1632 | + skip_broken=False) |
1633 | + |
1634 | + render_dir = os.path.join(tmp_dir, "render") |
1635 | + os.makedirs(render_dir) |
1636 | + |
1637 | + render_target = 'netplan.yaml' |
1638 | + renderer = netplan.Renderer( |
1639 | + {'netplan_path': render_target, 'postcmds': True}) |
1640 | + renderer.render_network_state(render_dir, ns) |
1641 | + |
1642 | + mock_netplan_generate.assert_called_with(run=True) |
1643 | + mock_net_setup_link.assert_called_with(run=True) |
1644 | + |
1645 | + @mock.patch.object(netplan, "get_devicelist") |
1646 | + @mock.patch('cloudinit.util.subp') |
1647 | + def test_netplan_postcmds(self, mock_subp, mock_devlist): |
1648 | + mock_devlist.side_effect = [['lo']] |
1649 | + tmp_dir = self.tmp_dir() |
1650 | + ns = network_state.parse_net_config_data(self.mycfg, |
1651 | + skip_broken=False) |
1652 | + |
1653 | + render_dir = os.path.join(tmp_dir, "render") |
1654 | + os.makedirs(render_dir) |
1655 | + |
1656 | + render_target = 'netplan.yaml' |
1657 | + renderer = netplan.Renderer( |
1658 | + {'netplan_path': render_target, 'postcmds': True}) |
1659 | + renderer.render_network_state(render_dir, ns) |
1660 | + |
1661 | + expected = [ |
1662 | + mock.call(['netplan', 'generate'], capture=True), |
1663 | + mock.call(['udevadm', 'test-builtin', 'net_setup_link', |
1664 | + '/sys/class/net/lo'], capture=True), |
1665 | + ] |
1666 | + mock_subp.assert_has_calls(expected) |
1667 | + |
1668 | + |
1669 | class TestEniNetworkStateToEni(CiTestCase): |
1670 | mycfg = { |
1671 | 'config': [{"type": "physical", "name": "eth0", |
1672 | @@ -953,6 +1210,50 @@ class TestCmdlineReadKernelConfig(CiTestCase): |
1673 | self.assertEqual(found['config'], expected) |
1674 | |
1675 | |
1676 | +class TestNetplanRoundTrip(CiTestCase): |
1677 | + def _render_and_read(self, network_config=None, state=None, |
1678 | + netplan_path=None, dir=None): |
1679 | + if dir is None: |
1680 | + dir = self.tmp_dir() |
1681 | + |
1682 | + if network_config: |
1683 | + ns = network_state.parse_net_config_data(network_config) |
1684 | + elif state: |
1685 | + ns = state |
1686 | + else: |
1687 | + raise ValueError("Expected data or state, got neither") |
1688 | + |
1689 | + if netplan_path is None: |
1690 | + netplan_path = 'etc/netplan/50-cloud-init.yaml' |
1691 | + |
1692 | + renderer = netplan.Renderer( |
1693 | + config={'netplan_path': netplan_path}) |
1694 | + |
1695 | + renderer.render_network_state(dir, ns) |
1696 | + return dir2dict(dir) |
1697 | + |
1698 | + def testsimple_render_small_netplan(self): |
1699 | + entry = NETWORK_CONFIGS['small'] |
1700 | + files = self._render_and_read(network_config=yaml.load(entry['yaml'])) |
1701 | + self.assertEqual( |
1702 | + entry['expected_netplan'].splitlines(), |
1703 | + files['/etc/netplan/50-cloud-init.yaml'].splitlines()) |
1704 | + |
1705 | + def testsimple_render_v4_and_v6(self): |
1706 | + entry = NETWORK_CONFIGS['v4_and_v6'] |
1707 | + files = self._render_and_read(network_config=yaml.load(entry['yaml'])) |
1708 | + self.assertEqual( |
1709 | + entry['expected_netplan'].splitlines(), |
1710 | + files['/etc/netplan/50-cloud-init.yaml'].splitlines()) |
1711 | + |
1712 | + def testsimple_render_all(self): |
1713 | + entry = NETWORK_CONFIGS['all'] |
1714 | + files = self._render_and_read(network_config=yaml.load(entry['yaml'])) |
1715 | + self.assertEqual( |
1716 | + entry['expected_netplan'].splitlines(), |
1717 | + files['/etc/netplan/50-cloud-init.yaml'].splitlines()) |
1718 | + |
1719 | + |
1720 | class TestEniRoundTrip(CiTestCase): |
1721 | def _render_and_read(self, network_config=None, state=None, eni_path=None, |
1722 | links_prefix=None, netrules_path=None, dir=None): |
1723 | diff --git a/tools/net-convert.py b/tools/net-convert.py |
1724 | new file mode 100755 |
1725 | index 0000000..1424bb0 |
1726 | --- /dev/null |
1727 | +++ b/tools/net-convert.py |
1728 | @@ -0,0 +1,83 @@ |
1729 | +#!/usr/bin/python3 |
1730 | +# |
1731 | +# This file is part of cloud-init. See LICENSE file ... |
1732 | + |
1733 | +import argparse |
1734 | +import json |
1735 | +import os |
1736 | +import yaml |
1737 | + |
1738 | +from cloudinit.sources.helpers import openstack |
1739 | + |
1740 | +from cloudinit.net import eni |
1741 | +from cloudinit.net import network_state |
1742 | +from cloudinit.net import netplan |
1743 | +from cloudinit.net import sysconfig |
1744 | + |
1745 | + |
1746 | +def main(): |
1747 | + parser = argparse.ArgumentParser() |
1748 | + parser.add_argument("--network-data", "-p", type=open, |
1749 | + metavar="PATH", required=True) |
1750 | + parser.add_argument("--kind", "-k", |
1751 | + choices=['eni', 'network_data.json', 'yaml'], |
1752 | + required=True) |
1753 | + parser.add_argument("-d", "--directory", |
1754 | + metavar="PATH", |
1755 | + help="directory to place output in", |
1756 | + required=True) |
1757 | + parser.add_argument("-m", "--mac", |
1758 | + metavar="name,mac", |
1759 | + action='append', |
1760 | + help="interface name to mac mapping") |
1761 | + parser.add_argument("--output-kind", "-ok", |
1762 | + choices=['eni', 'netplan', 'sysconfig'], |
1763 | + required=True) |
1764 | + args = parser.parse_args() |
1765 | + |
1766 | + if not os.path.isdir(args.directory): |
1767 | + os.makedirs(args.directory) |
1768 | + |
1769 | + if args.mac: |
1770 | + known_macs = {} |
1771 | + for item in args.mac: |
1772 | + iface_name, iface_mac = item.split(",", 1) |
1773 | + known_macs[iface_mac] = iface_name |
1774 | + else: |
1775 | + known_macs = None |
1776 | + |
1777 | + net_data = args.network_data.read() |
1778 | + if args.kind == "eni": |
1779 | + pre_ns = eni.convert_eni_data(net_data) |
1780 | + ns = network_state.parse_net_config_data(pre_ns) |
1781 | + elif args.kind == "yaml": |
1782 | + pre_ns = yaml.load(net_data) |
1783 | + if 'network' in pre_ns: |
1784 | + pre_ns = pre_ns.get('network') |
1785 | + print("Input YAML") |
1786 | + print(yaml.dump(pre_ns, default_flow_style=False, indent=4)) |
1787 | + ns = network_state.parse_net_config_data(pre_ns) |
1788 | + else: |
1789 | + pre_ns = openstack.convert_net_json( |
1790 | + json.loads(net_data), known_macs=known_macs) |
1791 | + ns = network_state.parse_net_config_data(pre_ns) |
1792 | + |
1793 | + if not ns: |
1794 | + raise RuntimeError("No valid network_state object created from" |
1795 | + "input data") |
1796 | + |
1797 | + print("\nInternal State") |
1798 | + print(yaml.dump(ns, default_flow_style=False, indent=4)) |
1799 | + if args.output_kind == "eni": |
1800 | + r_cls = eni.Renderer |
1801 | + elif args.output_kind == "netplan": |
1802 | + r_cls = netplan.Renderer |
1803 | + else: |
1804 | + r_cls = sysconfig.Renderer |
1805 | + |
1806 | + r = r_cls() |
1807 | + r.render_network_state(ns, target=args.directory) |
1808 | + |
1809 | + |
1810 | +if __name__ == '__main__': |
1811 | + main() |
FAILED: Continuous integration, rev:c43f407e2ac 41e3e474dbfd023 fc6325919819a9 /jenkins. ubuntu. com/server/ job/cloud- init-ci/ 139/ /jenkins. ubuntu. com/server/ job/cloud- init-ci/ nodes=metal- amd64/139/ console /jenkins. ubuntu. com/server/ job/cloud- init-ci/ nodes=metal- arm64/139/ console /jenkins. ubuntu. com/server/ job/cloud- init-ci/ nodes=metal- ppc64el/ 139/console /jenkins. ubuntu. com/server/ job/cloud- init-ci/ nodes=metal- s390x/139/ console /jenkins. ubuntu. com/server/ job/cloud- init-ci/ nodes=vm- i386/139/ console
https:/
Executed test runs:
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
Click here to trigger a rebuild: /jenkins. ubuntu. com/server/ job/cloud- init-ci/ 139/rebuild
https:/