Merge ~raharper/cloud-init:netconfig-v2-passthrough into cloud-init:master
- Git
- lp:~raharper/cloud-init
- netconfig-v2-passthrough
- Merge into master
Status: | Rejected |
---|---|
Rejected by: | Scott Moser |
Proposed branch: | ~raharper/cloud-init:netconfig-v2-passthrough |
Merge into: | cloud-init:master |
Diff against target: |
1917 lines (+1471/-41) 15 files modified
cloudinit/distros/__init__.py (+1/-1) cloudinit/distros/debian.py (+11/-7) cloudinit/distros/rhel.py (+1/-0) cloudinit/net/__init__.py (+5/-0) cloudinit/net/eni.py (+3/-12) cloudinit/net/netplan.py (+383/-0) cloudinit/net/network_state.py (+307/-11) cloudinit/net/renderer.py (+1/-0) cloudinit/net/renderers.py (+3/-1) cloudinit/net/sysconfig.py (+19/-5) systemd/cloud-init.service (+1/-0) systemd/systemd-networkd-wait-online.path (+5/-0) tests/unittests/test_distros/test_netconfig.py (+347/-4) tests/unittests/test_net.py (+301/-0) tools/net-convert.py (+83/-0) |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Server Team CI bot | continuous-integration | Needs Fixing | |
Scott Moser | Needs Fixing | ||
Review via email: mp+319259@code.launchpad.net |
Commit message
Description of the change
cloudinit.net: add v2 parsing, and v2 rendering
Network configuration version2 format is implemented in a package
called netplan (nplan)[1] which allows consolidated network config
for multiple network controllers.
- Detect and determine if Ubuntu Distro object will render eni or netplan
- Allow v2 configs to be passed directly to netplan
- Allow any network state (parsed from any format cloud-init supports) to
render to v2 if system supports netplan.
- Move eni's _subnet_is_ipv6 to common code for use by other renderers
- Fix to base distro class for looking up path to
system_
- Make sysconfig renderer always emit /etc/syconfig/
1. https:/
Scott Moser (smoser) wrote : | # |
Ryan Harper (raharper) wrote : | # |
Thanks for the review
Scott Moser (smoser) wrote : | # |
some small things.
i've not yet looked at the tests
Ryan Harper (raharper) wrote : | # |
Thanks for the review. I plan to rebase to trunk and pickup the renderer priority branch you have as well as a fix for https:/
Ideally the renderer priority branch should include some unittest to handle default policy selection, renderer availability, and setting policy; as well as broken config (provide a policy with nothing available, we should still have a valid renderer).
Server Team CI bot (server-team-bot) wrote : | # |
FAILED: Continuous integration, rev:4cf22b3f6cb
https:/
Executed test runs:
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
Click here to trigger a rebuild:
https:/
Ryan Harper (raharper) wrote : | # |
I lacked the git-fu to force a proper rebase against itself, so I've pushed a new branch for review:
https:/
Scott Moser (smoser) wrote : | # |
Marked this rejected.
It was merged at link above.
Unmerged commits
- 4cf22b3... by Ryan Harper
-
test_net: Remove duplicate unittests left from merge
- d5a6353... by Ryan Harper
-
Merge branch 'merge-
netconfig- v2-pass' into netconfig- v2-passthrough Merge in a rebase of 'netconfig-v2-pass onto master' into netconfig-v2-pass
- 773df13... by Ryan Harper
-
test_netconfig: Fix config path for setting network renderer priorities, fix mock for available checks
- ad519eb... by Ryan Harper
-
fix cfg path for network renderer priorty
- 2ac9242... by Ryan Harper
-
Mock out an ensure_dir which attempted to create dir
- e66c337... by Ryan Harper
-
net: add renderers for automatically selecting the renderer.
previously, the distro had hard coded which network renderer it would
use. This adds support for just picking the right renderer based
on what is available.Now, that can be set via a 'priority' in system_info, but should
generally work. That config looks like:
system_info:
network:
renderers: ["eni", "sysconfig"] - 8a53515... by Ryan Harper
-
Integrate use of network renderers into distros and update unittests
- 022cadc... by Ryan Harper
-
unittests for netplan, including postcommands, style fixes
- 08c41d6... by Ryan Harper
-
Add v2 passthrough test to distro netconfig
- e778ea5... by Ryan Harper
-
Test eni and netplan rendering from distro; fix issue with dhcp4 attr in netplan
Preview Diff
1 | diff --git a/cloudinit/distros/__init__.py b/cloudinit/distros/__init__.py |
2 | index 803ac74..22ae998 100755 |
3 | --- a/cloudinit/distros/__init__.py |
4 | +++ b/cloudinit/distros/__init__.py |
5 | @@ -73,7 +73,7 @@ class Distro(object): |
6 | |
7 | def _supported_write_network_config(self, network_config): |
8 | priority = util.get_cfg_by_path( |
9 | - self._cfg, ('network', 'renderers'), None) |
10 | + self._cfg, ('system_info', 'network', 'renderers'), None) |
11 | |
12 | name, render_cls = renderers.select(priority=priority) |
13 | LOG.debug("Selected renderer '%s' from priority list: %s", |
14 | diff --git a/cloudinit/distros/debian.py b/cloudinit/distros/debian.py |
15 | index 1101f02..a069bc2 100644 |
16 | --- a/cloudinit/distros/debian.py |
17 | +++ b/cloudinit/distros/debian.py |
18 | @@ -36,17 +36,20 @@ ENI_HEADER = """# This file is generated from information provided by |
19 | # network: {config: disabled} |
20 | """ |
21 | |
22 | -NETWORK_CONF_FN = "/etc/network/interfaces.d/50-cloud-init.cfg" |
23 | - |
24 | |
25 | class Distro(distros.Distro): |
26 | hostname_conf_fn = "/etc/hostname" |
27 | locale_conf_fn = "/etc/default/locale" |
28 | + network_conf_fn = { |
29 | + "eni": "/etc/network/interfaces.d/50-cloud-init.cfg", |
30 | + "netplan": "/etc/netplan/50-cloud-init.yaml" |
31 | + } |
32 | renderer_configs = { |
33 | - 'eni': { |
34 | - 'eni_path': NETWORK_CONF_FN, |
35 | - 'eni_header': ENI_HEADER, |
36 | - } |
37 | + "eni": {"eni_path": network_conf_fn["eni"], |
38 | + "eni_header": ENI_HEADER}, |
39 | + "netplan": {"netplan_path": network_conf_fn["netplan"], |
40 | + "netplan_header": ENI_HEADER, |
41 | + "postcmds": True} |
42 | } |
43 | |
44 | def __init__(self, name, cfg, paths): |
45 | @@ -75,7 +78,8 @@ class Distro(distros.Distro): |
46 | self.package_command('install', pkgs=pkglist) |
47 | |
48 | def _write_network(self, settings): |
49 | - util.write_file(NETWORK_CONF_FN, settings) |
50 | + # this is always going to be legacy based |
51 | + util.write_file(self.network_conf_fn["eni"], settings) |
52 | return ['all'] |
53 | |
54 | def _write_network_config(self, netconfig): |
55 | diff --git a/cloudinit/distros/rhel.py b/cloudinit/distros/rhel.py |
56 | index 372c7d0..0eef69b 100644 |
57 | --- a/cloudinit/distros/rhel.py |
58 | +++ b/cloudinit/distros/rhel.py |
59 | @@ -39,6 +39,7 @@ class Distro(distros.Distro): |
60 | resolve_conf_fn = "/etc/resolv.conf" |
61 | tz_local_fn = "/etc/localtime" |
62 | usr_lib_exec = "/usr/libexec" |
63 | + network_render_config = {'sysconfig': None} |
64 | |
65 | def __init__(self, name, cfg, paths): |
66 | distros.Distro.__init__(self, name, cfg, paths) |
67 | diff --git a/cloudinit/net/__init__.py b/cloudinit/net/__init__.py |
68 | index 1cf98ef..1297808 100755 |
69 | --- a/cloudinit/net/__init__.py |
70 | +++ b/cloudinit/net/__init__.py |
71 | @@ -14,6 +14,7 @@ from cloudinit import util |
72 | |
73 | LOG = logging.getLogger(__name__) |
74 | SYS_CLASS_NET = "/sys/class/net/" |
75 | +SYS_DEV_VIRT_NET = "/sys/devices/virtual/net/" |
76 | DEFAULT_PRIMARY_INTERFACE = 'eth0' |
77 | |
78 | |
79 | @@ -205,7 +206,11 @@ def _get_current_rename_info(check_downable=True): |
80 | """Collect information necessary for rename_interfaces.""" |
81 | names = get_devicelist() |
82 | bymac = {} |
83 | + virtual = os.listdir(SYS_DEV_VIRT_NET) |
84 | for n in names: |
85 | + # do not attempt to rename virtual interfaces |
86 | + if n in virtual: |
87 | + continue |
88 | bymac[get_interface_mac(n)] = { |
89 | 'name': n, 'up': is_up(n), 'downable': None} |
90 | |
91 | diff --git a/cloudinit/net/eni.py b/cloudinit/net/eni.py |
92 | index 9d39a2b..6bb744e 100644 |
93 | --- a/cloudinit/net/eni.py |
94 | +++ b/cloudinit/net/eni.py |
95 | @@ -8,6 +8,7 @@ import re |
96 | from . import ParserError |
97 | |
98 | from . import renderer |
99 | +from .network_state import subnet_is_ipv6 |
100 | |
101 | from cloudinit import util |
102 | |
103 | @@ -111,16 +112,6 @@ def _iface_start_entry(iface, index, render_hwaddress=False): |
104 | return lines |
105 | |
106 | |
107 | -def _subnet_is_ipv6(subnet): |
108 | - # 'static6' or 'dhcp6' |
109 | - if subnet['type'].endswith('6'): |
110 | - # This is a request for DHCPv6. |
111 | - return True |
112 | - elif subnet['type'] == 'static' and ":" in subnet['address']: |
113 | - return True |
114 | - return False |
115 | - |
116 | - |
117 | def _parse_deb_config_data(ifaces, contents, src_dir, src_path): |
118 | """Parses the file contents, placing result into ifaces. |
119 | |
120 | @@ -370,7 +361,7 @@ class Renderer(renderer.Renderer): |
121 | iface['mode'] = subnet['type'] |
122 | iface['control'] = subnet.get('control', 'auto') |
123 | subnet_inet = 'inet' |
124 | - if _subnet_is_ipv6(subnet): |
125 | + if subnet_is_ipv6(subnet): |
126 | subnet_inet += '6' |
127 | iface['inet'] = subnet_inet |
128 | if subnet['type'].startswith('dhcp'): |
129 | @@ -486,7 +477,7 @@ class Renderer(renderer.Renderer): |
130 | def network_state_to_eni(network_state, header=None, render_hwaddress=False): |
131 | # render the provided network state, return a string of equivalent eni |
132 | eni_path = 'etc/network/interfaces' |
133 | - renderer = Renderer({ |
134 | + renderer = Renderer(config={ |
135 | 'eni_path': eni_path, |
136 | 'eni_header': header, |
137 | 'links_path_prefix': None, |
138 | diff --git a/cloudinit/net/netplan.py b/cloudinit/net/netplan.py |
139 | new file mode 100644 |
140 | index 0000000..49f3120 |
141 | --- /dev/null |
142 | +++ b/cloudinit/net/netplan.py |
143 | @@ -0,0 +1,383 @@ |
144 | +# This file is part of cloud-init. See LICENSE file ... |
145 | + |
146 | +import copy |
147 | +import os |
148 | +from textwrap import indent |
149 | + |
150 | +from . import renderer |
151 | +from .network_state import subnet_is_ipv6 |
152 | + |
153 | +from cloudinit import util |
154 | +from cloudinit.net import SYS_CLASS_NET, get_devicelist |
155 | + |
156 | + |
157 | +NET_CONFIG_TO_V2 = { |
158 | + 'bond': {'bond-ad-select': 'ad-select', |
159 | + 'bond-arp-interval': 'arp-interval', |
160 | + 'bond-arp-ip-target': 'arp-ip-target', |
161 | + 'bond-arp-validate': 'arp-validate', |
162 | + 'bond-downdelay': 'down-delay', |
163 | + 'bond-fail-over-mac': 'fail-over-mac-policy', |
164 | + 'bond-lacp-rate': 'lacp-rate', |
165 | + 'bond-miimon': 'mii-monitor-interval', |
166 | + 'bond-min-links': 'min-links', |
167 | + 'bond-mode': 'mode', |
168 | + 'bond-num-grat-arp': 'gratuitious-arp', |
169 | + 'bond-primary-reselect': 'primary-reselect-policy', |
170 | + 'bond-updelay': 'up-delay', |
171 | + 'bond-xmit_hash_policy': 'transmit_hash_policy'}, |
172 | + 'bridge': {'bridge_ageing': 'ageing-time', |
173 | + 'bridge_bridgeprio': 'priority', |
174 | + 'bridge_fd': 'forward-delay', |
175 | + 'bridge_gcint': None, |
176 | + 'bridge_hello': 'hello-time', |
177 | + 'bridge_maxage': 'max-age', |
178 | + 'bridge_maxwait': None, |
179 | + 'bridge_pathcost': 'path-cost', |
180 | + 'bridge_portprio': None, |
181 | + 'bridge_waitport': None}} |
182 | + |
183 | + |
184 | +def _get_params_dict_by_match(config, match): |
185 | + return dict((key, value) for (key, value) in config.items() |
186 | + if key.startswith(match)) |
187 | + |
188 | + |
189 | +def _extract_addresses(config, entry): |
190 | + """ This method parse a cloudinit.net.network_state dictionary (config) and |
191 | + maps netstate keys/values into a dictionary (entry) to represent |
192 | + netplan yaml. |
193 | + |
194 | + An example config dictionary might look like: |
195 | + |
196 | + {'mac_address': '52:54:00:12:34:00', |
197 | + 'name': 'interface0', |
198 | + 'subnets': [ |
199 | + {'address': '192.168.1.2/24', |
200 | + 'mtu': 1501, |
201 | + 'type': 'static'}, |
202 | + {'address': '2001:4800:78ff:1b:be76:4eff:fe06:1000", |
203 | + 'mtu': 1480, |
204 | + 'netmask': 64, |
205 | + 'type': 'static'}], |
206 | + 'type: physical' |
207 | + } |
208 | + |
209 | + An entry dictionary looks like: |
210 | + |
211 | + {'set-name': 'interface0', |
212 | + 'match': {'macaddress': '52:54:00:12:34:00'}, |
213 | + 'mtu': 1501} |
214 | + |
215 | + After modification returns |
216 | + |
217 | + {'set-name': 'interface0', |
218 | + 'match': {'macaddress': '52:54:00:12:34:00'}, |
219 | + 'mtu': 1501, |
220 | + 'address': ['192.168.1.2/24', '2001:4800:78ff:1b:be76:4eff:fe06:1000"], |
221 | + 'mtu6': 1480} |
222 | + |
223 | + """ |
224 | + |
225 | + def _listify(obj, token=' '): |
226 | + """ Helper to convert strings to list of strings, handle single |
227 | + string""" |
228 | + if not obj or type(obj) not in [str]: |
229 | + return obj |
230 | + if token in obj: |
231 | + return obj.split(token) |
232 | + else: |
233 | + return [obj, ] |
234 | + |
235 | + addresses = [] |
236 | + routes = [] |
237 | + nameservers = [] |
238 | + searchdomains = [] |
239 | + subnets = config.get('subnets', []) |
240 | + if subnets is None: |
241 | + subnets = [] |
242 | + for subnet in subnets: |
243 | + sn_type = subnet.get('type') |
244 | + if sn_type.startswith('dhcp'): |
245 | + if sn_type == 'dhcp': |
246 | + sn_type += '4' |
247 | + entry.update({sn_type: True}) |
248 | + elif sn_type in ['static']: |
249 | + addr = "%s" % subnet.get('address') |
250 | + if 'netmask' in subnet: |
251 | + addr += "/%s" % subnet.get('netmask') |
252 | + if 'gateway' in subnet and subnet.get('gateway'): |
253 | + gateway = subnet.get('gateway') |
254 | + if ":" in gateway: |
255 | + entry.update({'gateway6': gateway}) |
256 | + else: |
257 | + entry.update({'gateway4': gateway}) |
258 | + if 'dns_nameservers' in subnet: |
259 | + nameservers += _listify(subnet.get('dns_nameservers', [])) |
260 | + if 'dns_search' in subnet: |
261 | + searchdomains += _listify(subnet.get('dns_search', [])) |
262 | + if 'mtu' in subnet: |
263 | + mtukey = 'mtu' |
264 | + if subnet_is_ipv6(subnet): |
265 | + mtukey += '6' |
266 | + entry.update({mtukey: subnet.get('mtu')}) |
267 | + for route in subnet.get('routes', []): |
268 | + to_net = "%s/%s" % (route.get('network'), |
269 | + route.get('netmask')) |
270 | + route = { |
271 | + 'via': route.get('gateway'), |
272 | + 'to': to_net, |
273 | + } |
274 | + if 'metric' in route: |
275 | + route.update({'metric': route.get('metric', 100)}) |
276 | + routes.append(route) |
277 | + |
278 | + addresses.append(addr) |
279 | + |
280 | + if len(addresses) > 0: |
281 | + entry.update({'addresses': addresses}) |
282 | + if len(routes) > 0: |
283 | + entry.update({'routes': routes}) |
284 | + if len(nameservers) > 0: |
285 | + ns = {'addresses': nameservers} |
286 | + entry.update({'nameservers': ns}) |
287 | + if len(searchdomains) > 0: |
288 | + ns = entry.get('nameservers', {}) |
289 | + ns.update({'search': searchdomains}) |
290 | + entry.update({'nameservers': ns}) |
291 | + |
292 | + |
293 | +def _extract_bond_slaves_by_name(interfaces, entry, bond_master): |
294 | + bond_slave_names = sorted([name for (name, cfg) in interfaces.items() |
295 | + if cfg.get('bond-master', None) == bond_master]) |
296 | + if len(bond_slave_names) > 0: |
297 | + entry.update({'interfaces': bond_slave_names}) |
298 | + |
299 | + |
300 | +class Renderer(renderer.Renderer): |
301 | + """Renders network information in a /etc/netplan/network.yaml format.""" |
302 | + |
303 | + NETPLAN_GENERATE = ['netplan', 'generate'] |
304 | + |
305 | + def __init__(self, config=None): |
306 | + if not config: |
307 | + config = {} |
308 | + self.netplan_path = config.get('netplan_path', |
309 | + 'etc/netplan/50-cloud-init.yaml') |
310 | + self.netplan_header = config.get('netplan_header', None) |
311 | + self._postcmds = config.get('postcmds', False) |
312 | + |
313 | + def render_network_state(self, target, network_state): |
314 | + # check network state for version |
315 | + # if v2, then extract network_state.config |
316 | + # else render_v2_from_state |
317 | + fpnplan = os.path.join(target, self.netplan_path) |
318 | + util.ensure_dir(os.path.dirname(fpnplan)) |
319 | + header = self.netplan_header if self.netplan_header else "" |
320 | + |
321 | + if network_state.version > 1: |
322 | + # pass-through original config |
323 | + content = util.yaml_dumps({'network': network_state.config}, |
324 | + explicit_start=False, |
325 | + explicit_end=False) |
326 | + else: |
327 | + # render from state |
328 | + content = self._render_content(network_state) |
329 | + # ensure we poke udev to run net_setup_link |
330 | + if not header.endswith("\n"): |
331 | + header += "\n" |
332 | + util.write_file(fpnplan, header + content) |
333 | + |
334 | + self._netplan_generate(run=self._postcmds) |
335 | + self._net_setup_link(run=self._postcmds) |
336 | + |
337 | + def _netplan_generate(self, run=False): |
338 | + if not run: |
339 | + return |
340 | + util.subp(self.NETPLAN_GENERATE, capture=True) |
341 | + |
342 | + def _net_setup_link(self, run=False): |
343 | + """ To ensure device link properties are applied, we poke |
344 | + udev to re-evaluate networkd .link files and call |
345 | + the setup_link udev builtin command |
346 | + """ |
347 | + if not run: |
348 | + return |
349 | + setup_lnk = ['udevadm', 'test-builtin', 'net_setup_link'] |
350 | + for cmd in [setup_lnk + [SYS_CLASS_NET + iface] |
351 | + for iface in get_devicelist() if |
352 | + os.path.islink(SYS_CLASS_NET + iface)]: |
353 | + util.subp(cmd, capture=True) |
354 | + |
355 | + def _render_content(self, network_state): |
356 | + ethernets = {} |
357 | + wifis = {} |
358 | + bridges = {} |
359 | + bonds = {} |
360 | + vlans = {} |
361 | + content = [] |
362 | + |
363 | + interfaces = network_state._network_state.get('interfaces', []) |
364 | + |
365 | + nameservers = network_state.dns_nameservers |
366 | + searchdomains = network_state.dns_searchdomains |
367 | + |
368 | + for config in network_state.iter_interfaces(): |
369 | + ifname = config.get('name') |
370 | + # filter None entries up front so we can do simple if key in dict |
371 | + ifcfg = dict((key, value) for (key, value) in config.items() |
372 | + if value) |
373 | + |
374 | + if_type = ifcfg.get('type') |
375 | + if if_type == 'physical': |
376 | + # required_keys = ['name', 'mac_address'] |
377 | + eth = { |
378 | + 'set-name': ifname, |
379 | + 'match': ifcfg.get('match', None), |
380 | + } |
381 | + if eth['match'] is None: |
382 | + macaddr = ifcfg.get('mac_address', None) |
383 | + if macaddr is not None: |
384 | + eth['match'] = {'macaddress': macaddr.lower()} |
385 | + else: |
386 | + del eth['match'] |
387 | + del eth['set-name'] |
388 | + if 'mtu' in ifcfg: |
389 | + eth['mtu'] = ifcfg.get('mtu') |
390 | + |
391 | + _extract_addresses(ifcfg, eth) |
392 | + ethernets.update({ifname: eth}) |
393 | + |
394 | + elif if_type == 'bond': |
395 | + # required_keys = ['name', 'bond_interfaces'] |
396 | + bond = {} |
397 | + bond_config = {} |
398 | + # extract bond params and drop the bond_ prefix as it's |
399 | + # redundent in v2 yaml format |
400 | + v2_bond_map = NET_CONFIG_TO_V2.get('bond') |
401 | + for match in ['bond_', 'bond-']: |
402 | + bond_params = _get_params_dict_by_match(ifcfg, match) |
403 | + for (param, value) in bond_params.items(): |
404 | + newname = v2_bond_map.get(param) |
405 | + if newname is None: |
406 | + continue |
407 | + bond_config.update({newname: value}) |
408 | + |
409 | + if len(bond_config) > 0: |
410 | + bond.update({'parameters': bond_config}) |
411 | + slave_interfaces = ifcfg.get('bond-slaves') |
412 | + if slave_interfaces == 'none': |
413 | + _extract_bond_slaves_by_name(interfaces, bond, ifname) |
414 | + _extract_addresses(ifcfg, bond) |
415 | + bonds.update({ifname: bond}) |
416 | + |
417 | + elif if_type == 'bridge': |
418 | + # required_keys = ['name', 'bridge_ports'] |
419 | + ports = sorted(copy.copy(ifcfg.get('bridge_ports'))) |
420 | + bridge = { |
421 | + 'interfaces': ports, |
422 | + } |
423 | + # extract bridge params and drop the bridge prefix as it's |
424 | + # redundent in v2 yaml format |
425 | + match_prefix = 'bridge_' |
426 | + params = _get_params_dict_by_match(ifcfg, match_prefix) |
427 | + br_config = {} |
428 | + |
429 | + # v2 yaml uses different names for the keys |
430 | + # and at least one value format change |
431 | + v2_bridge_map = NET_CONFIG_TO_V2.get('bridge') |
432 | + for (param, value) in params.items(): |
433 | + newname = v2_bridge_map.get(param) |
434 | + if newname is None: |
435 | + continue |
436 | + br_config.update({newname: value}) |
437 | + if newname == 'path-cost': |
438 | + # <interface> <cost> -> <interface>: int(<cost>) |
439 | + newvalue = {} |
440 | + for costval in value: |
441 | + (port, cost) = costval.split() |
442 | + newvalue[port] = int(cost) |
443 | + br_config.update({newname: newvalue}) |
444 | + if len(br_config) > 0: |
445 | + bridge.update({'parameters': br_config}) |
446 | + _extract_addresses(ifcfg, bridge) |
447 | + bridges.update({ifname: bridge}) |
448 | + |
449 | + elif if_type == 'vlan': |
450 | + # required_keys = ['name', 'vlan_id', 'vlan-raw-device'] |
451 | + vlan = { |
452 | + 'id': ifcfg.get('vlan_id'), |
453 | + 'link': ifcfg.get('vlan-raw-device') |
454 | + } |
455 | + |
456 | + _extract_addresses(ifcfg, vlan) |
457 | + vlans.update({ifname: vlan}) |
458 | + |
459 | + # inject global nameserver values under each physical interface |
460 | + if nameservers: |
461 | + for _eth, cfg in ethernets.items(): |
462 | + nscfg = cfg.get('nameservers', {}) |
463 | + addresses = nscfg.get('addresses', []) |
464 | + addresses += nameservers |
465 | + nscfg.update({'addresses': addresses}) |
466 | + cfg.update({'nameservers': nscfg}) |
467 | + |
468 | + if searchdomains: |
469 | + for _eth, cfg in ethernets.items(): |
470 | + nscfg = cfg.get('nameservers', {}) |
471 | + search = nscfg.get('search', []) |
472 | + search += searchdomains |
473 | + nscfg.update({'search': search}) |
474 | + cfg.update({'nameservers': nscfg}) |
475 | + |
476 | + # workaround yaml dictionary key sorting when dumping |
477 | + def _render_section(name, section): |
478 | + if section: |
479 | + dump = util.yaml_dumps({name: section}, |
480 | + explicit_start=False, |
481 | + explicit_end=False) |
482 | + txt = indent(dump, ' ' * 4) |
483 | + return [txt] |
484 | + return [] |
485 | + |
486 | + content.append("network:\n version: 2\n") |
487 | + content += _render_section('ethernets', ethernets) |
488 | + content += _render_section('wifis', wifis) |
489 | + content += _render_section('bonds', bonds) |
490 | + content += _render_section('bridges', bridges) |
491 | + content += _render_section('vlans', vlans) |
492 | + |
493 | + return "".join(content) |
494 | + |
495 | + |
496 | +def network_state_to_netplan(network_state, header=None): |
497 | + # render the provided network state, return a string of equivalent eni |
498 | + netplan_path = 'etc/network/50-cloud-init.yaml' |
499 | + renderer = Renderer({ |
500 | + 'netplan_path': netplan_path, |
501 | + 'netplan_header': header, |
502 | + }) |
503 | + if not header: |
504 | + header = "" |
505 | + if not header.endswith("\n"): |
506 | + header += "\n" |
507 | + contents = renderer._render_content(network_state) |
508 | + return header + contents |
509 | + |
510 | + |
511 | +def available(target=None): |
512 | + expected = ['netplan'] |
513 | + search = ['/usr/sbin', '/sbin'] |
514 | + for p in expected: |
515 | + if not util.which(p, search=search, target=target): |
516 | + return False |
517 | + return True |
518 | + |
519 | + expected_paths = ['etc/netplan/'] |
520 | + for p in expected_paths: |
521 | + if not os.path.isfile(util.target_path(target, p)): |
522 | + return False |
523 | + return True |
524 | + |
525 | + |
526 | +# vi: ts=4 expandtab |
527 | diff --git a/cloudinit/net/network_state.py b/cloudinit/net/network_state.py |
528 | index 90b2835..2469f7c 100644 |
529 | --- a/cloudinit/net/network_state.py |
530 | +++ b/cloudinit/net/network_state.py |
531 | @@ -1,4 +1,4 @@ |
532 | -# Copyright (C) 2013-2014 Canonical Ltd. |
533 | +# Copyright (C) 2017 Canonical Ltd. |
534 | # |
535 | # Author: Ryan Harper <ryan.harper@canonical.com> |
536 | # |
537 | @@ -18,6 +18,10 @@ NETWORK_STATE_VERSION = 1 |
538 | NETWORK_STATE_REQUIRED_KEYS = { |
539 | 1: ['version', 'config', 'network_state'], |
540 | } |
541 | +NETWORK_V2_KEY_FILTER = [ |
542 | + 'addresses', 'dhcp4', 'dhcp6', 'gateway4', 'gateway6', 'interfaces', |
543 | + 'match', 'mtu', 'nameservers', 'renderer', 'set-name', 'wakeonlan' |
544 | +] |
545 | |
546 | |
547 | def parse_net_config_data(net_config, skip_broken=True): |
548 | @@ -26,11 +30,18 @@ def parse_net_config_data(net_config, skip_broken=True): |
549 | :param net_config: curtin network config dict |
550 | """ |
551 | state = None |
552 | - if 'version' in net_config and 'config' in net_config: |
553 | - nsi = NetworkStateInterpreter(version=net_config.get('version'), |
554 | - config=net_config.get('config')) |
555 | + version = net_config.get('version') |
556 | + config = net_config.get('config') |
557 | + if version == 2: |
558 | + # v2 does not have explicit 'config' key so we |
559 | + # pass the whole net-config as-is |
560 | + config = net_config |
561 | + |
562 | + if version and config: |
563 | + nsi = NetworkStateInterpreter(version=version, config=config) |
564 | nsi.parse_config(skip_broken=skip_broken) |
565 | - state = nsi.network_state |
566 | + state = nsi.get_network_state() |
567 | + |
568 | return state |
569 | |
570 | |
571 | @@ -103,14 +114,21 @@ class CommandHandlerMeta(type): |
572 | |
573 | class NetworkState(object): |
574 | |
575 | - def __init__(self, network_state, version=NETWORK_STATE_VERSION): |
576 | + def __init__(self, network_state, version=NETWORK_STATE_VERSION, |
577 | + config=None): |
578 | self._network_state = copy.deepcopy(network_state) |
579 | self._version = version |
580 | + self._config = copy.deepcopy(config) |
581 | + self.use_ipv6 = network_state.get('use_ipv6', False) |
582 | |
583 | @property |
584 | def version(self): |
585 | return self._version |
586 | |
587 | + @property |
588 | + def config(self): |
589 | + return self._config |
590 | + |
591 | def iter_routes(self, filter_func=None): |
592 | for route in self._network_state.get('routes', []): |
593 | if filter_func is not None: |
594 | @@ -152,7 +170,8 @@ class NetworkStateInterpreter(object): |
595 | 'dns': { |
596 | 'nameservers': [], |
597 | 'search': [], |
598 | - } |
599 | + }, |
600 | + 'use_ipv6': False, |
601 | } |
602 | |
603 | def __init__(self, version=NETWORK_STATE_VERSION, config=None): |
604 | @@ -163,7 +182,16 @@ class NetworkStateInterpreter(object): |
605 | |
606 | @property |
607 | def network_state(self): |
608 | - return NetworkState(self._network_state, version=self._version) |
609 | + return NetworkState(self._network_state, version=self._version, |
610 | + config=self._config) |
611 | + |
612 | + @property |
613 | + def use_ipv6(self): |
614 | + return self._network_state.get('use_ipv6') |
615 | + |
616 | + @use_ipv6.setter |
617 | + def use_ipv6(self, val): |
618 | + self._network_state.update({'use_ipv6': val}) |
619 | |
620 | def dump(self): |
621 | state = { |
622 | @@ -192,8 +220,19 @@ class NetworkStateInterpreter(object): |
623 | def dump_network_state(self): |
624 | return util.yaml_dumps(self._network_state) |
625 | |
626 | + def as_dict(self): |
627 | + return {'version': self.version, 'config': self.config} |
628 | + |
629 | + def get_network_state(self): |
630 | + ns = self.network_state |
631 | + return ns |
632 | + |
633 | def parse_config(self, skip_broken=True): |
634 | - # rebuild network state |
635 | + if self._version == 1: |
636 | + self.parse_config_v1(skip_broken=skip_broken) |
637 | + self._parsed = True |
638 | + |
639 | + def parse_config_v1(self, skip_broken=True): |
640 | for command in self._config: |
641 | command_type = command['type'] |
642 | try: |
643 | @@ -211,6 +250,25 @@ class NetworkStateInterpreter(object): |
644 | exc_info=True) |
645 | LOG.debug(self.dump_network_state()) |
646 | |
647 | + def parse_config_v2(self, skip_broken=True): |
648 | + for command_type, command in self._config.get('network').items(): |
649 | + |
650 | + try: |
651 | + handler = self.command_handlers[command_type] |
652 | + except KeyError: |
653 | + raise RuntimeError("No handler found for" |
654 | + " command '%s'" % command_type) |
655 | + try: |
656 | + handler(self, command) |
657 | + self._v2_common(self, command) |
658 | + except InvalidCommand: |
659 | + if not skip_broken: |
660 | + raise |
661 | + else: |
662 | + LOG.warn("Skipping invalid command: %s", command, |
663 | + exc_info=True) |
664 | + LOG.debug(self.dump_network_state()) |
665 | + |
666 | @ensure_command_keys(['name']) |
667 | def handle_loopback(self, command): |
668 | return self.handle_physical(command) |
669 | @@ -238,11 +296,16 @@ class NetworkStateInterpreter(object): |
670 | if subnets: |
671 | for subnet in subnets: |
672 | if subnet['type'] == 'static': |
673 | + if ':' in subnet['address']: |
674 | + self.use_ipv6 = True |
675 | if 'netmask' in subnet and ':' in subnet['address']: |
676 | subnet['netmask'] = mask2cidr(subnet['netmask']) |
677 | for route in subnet.get('routes', []): |
678 | if 'netmask' in route: |
679 | route['netmask'] = mask2cidr(route['netmask']) |
680 | + elif subnet['type'].endswith('6'): |
681 | + self.use_ipv6 = True |
682 | + |
683 | iface.update({ |
684 | 'name': command.get('name'), |
685 | 'type': command.get('type'), |
686 | @@ -327,7 +390,7 @@ class NetworkStateInterpreter(object): |
687 | bond_if.update({param: val}) |
688 | self._network_state['interfaces'].update({ifname: bond_if}) |
689 | |
690 | - @ensure_command_keys(['name', 'bridge_interfaces', 'params']) |
691 | + @ensure_command_keys(['name', 'bridge_interfaces']) |
692 | def handle_bridge(self, command): |
693 | ''' |
694 | auto br0 |
695 | @@ -373,7 +436,7 @@ class NetworkStateInterpreter(object): |
696 | self.handle_physical(command) |
697 | iface = interfaces.get(command.get('name'), {}) |
698 | iface['bridge_ports'] = command['bridge_interfaces'] |
699 | - for param, val in command.get('params').items(): |
700 | + for param, val in command.get('params', {}).items(): |
701 | iface.update({param: val}) |
702 | |
703 | interfaces.update({iface['name']: iface}) |
704 | @@ -407,6 +470,239 @@ class NetworkStateInterpreter(object): |
705 | } |
706 | routes.append(route) |
707 | |
708 | + # V2 handlers |
709 | + def handle_bonds(self, command): |
710 | + ''' |
711 | + v2_command = { |
712 | + bond0: { |
713 | + 'interfaces': ['interface0', 'interface1'], |
714 | + 'miimon': 100, |
715 | + 'mode': '802.3ad', |
716 | + 'xmit_hash_policy': 'layer3+4'}, |
717 | + bond1: { |
718 | + 'bond-slaves': ['interface2', 'interface7'], |
719 | + 'mode': 1 |
720 | + } |
721 | + } |
722 | + |
723 | + v1_command = { |
724 | + 'type': 'bond' |
725 | + 'name': 'bond0', |
726 | + 'bond_interfaces': [interface0, interface1], |
727 | + 'params': { |
728 | + 'bond-mode': '802.3ad', |
729 | + 'bond_miimon: 100, |
730 | + 'bond_xmit_hash_policy': 'layer3+4', |
731 | + } |
732 | + } |
733 | + |
734 | + ''' |
735 | + self._handle_bond_bridge(command, cmd_type='bond') |
736 | + |
737 | + def handle_bridges(self, command): |
738 | + |
739 | + ''' |
740 | + v2_command = { |
741 | + br0: { |
742 | + 'interfaces': ['interface0', 'interface1'], |
743 | + 'fd': 0, |
744 | + 'stp': 'off', |
745 | + 'maxwait': 0, |
746 | + } |
747 | + } |
748 | + |
749 | + v1_command = { |
750 | + 'type': 'bridge' |
751 | + 'name': 'br0', |
752 | + 'bridge_interfaces': [interface0, interface1], |
753 | + 'params': { |
754 | + 'bridge_stp': 'off', |
755 | + 'bridge_fd: 0, |
756 | + 'bridge_maxwait': 0 |
757 | + } |
758 | + } |
759 | + |
760 | + ''' |
761 | + self._handle_bond_bridge(command, cmd_type='bridge') |
762 | + |
763 | + def handle_ethernets(self, command): |
764 | + ''' |
765 | + ethernets: |
766 | + eno1: |
767 | + match: |
768 | + macaddress: 00:11:22:33:44:55 |
769 | + wakeonlan: true |
770 | + dhcp4: true |
771 | + dhcp6: false |
772 | + addresses: |
773 | + - 192.168.14.2/24 |
774 | + - 2001:1::1/64 |
775 | + gateway4: 192.168.14.1 |
776 | + gateway6: 2001:1::2 |
777 | + nameservers: |
778 | + search: [foo.local, bar.local] |
779 | + addresses: [8.8.8.8, 8.8.4.4] |
780 | + lom: |
781 | + match: |
782 | + driver: ixgbe |
783 | + set-name: lom1 |
784 | + dhcp6: true |
785 | + switchports: |
786 | + match: |
787 | + name: enp2* |
788 | + mtu: 1280 |
789 | + |
790 | + command = { |
791 | + 'type': 'physical', |
792 | + 'mac_address': 'c0:d6:9f:2c:e8:80', |
793 | + 'name': 'eth0', |
794 | + 'subnets': [ |
795 | + {'type': 'dhcp4'} |
796 | + ] |
797 | + } |
798 | + ''' |
799 | + for eth, cfg in command.items(): |
800 | + phy_cmd = { |
801 | + 'type': 'physical', |
802 | + 'name': cfg.get('set-name', eth), |
803 | + } |
804 | + mac_address = cfg.get('match', {}).get('macaddress', None) |
805 | + if not mac_address: |
806 | + LOG.warning('NetworkState Version2: missing macaddress') |
807 | + |
808 | + for key in ['mtu', 'match', 'wakeonlan']: |
809 | + if key in cfg: |
810 | + phy_cmd.update({key: cfg.get(key)}) |
811 | + |
812 | + subnets = self._v2_to_v1_ipcfg(cfg) |
813 | + if len(subnets) > 0: |
814 | + phy_cmd.update({'subnets': subnets}) |
815 | + |
816 | + LOG.debug('v2(ethernets) -> v1(physical):\n%s', phy_cmd) |
817 | + self.handle_physical(phy_cmd) |
818 | + |
819 | + def handle_vlans(self, command): |
820 | + ''' |
821 | + v2_vlans = { |
822 | + 'eth0.123': { |
823 | + 'id': 123, |
824 | + 'link': 'eth0', |
825 | + 'dhcp4': True, |
826 | + } |
827 | + } |
828 | + |
829 | + v1_command = { |
830 | + 'type': 'vlan', |
831 | + 'name': 'eth0.123', |
832 | + 'vlan_link': 'eth0', |
833 | + 'vlan_id': 123, |
834 | + 'subnets': [{'type': 'dhcp4'}], |
835 | + } |
836 | + ''' |
837 | + for vlan, cfg in command.items(): |
838 | + vlan_cmd = { |
839 | + 'type': 'vlan', |
840 | + 'name': vlan, |
841 | + 'vlan_id': cfg.get('id'), |
842 | + 'vlan_link': cfg.get('link'), |
843 | + } |
844 | + subnets = self._v2_to_v1_ipcfg(cfg) |
845 | + if len(subnets) > 0: |
846 | + vlan_cmd.update({'subnets': subnets}) |
847 | + LOG.debug('v2(vlans) -> v1(vlan):\n%s', vlan_cmd) |
848 | + self.handle_vlan(vlan_cmd) |
849 | + |
850 | + def handle_wifis(self, command): |
851 | + raise NotImplemented('NetworkState V2: Skipping wifi configuration') |
852 | + |
853 | + def _v2_common(self, cfg): |
854 | + LOG.debug('v2_common: handling config:\n%s', cfg) |
855 | + if 'nameservers' in cfg: |
856 | + search = cfg.get('nameservers').get('search', []) |
857 | + dns = cfg.get('nameservers').get('addresses', []) |
858 | + name_cmd = {'type': 'nameserver'} |
859 | + if len(search) > 0: |
860 | + name_cmd.update({'search': search}) |
861 | + if len(dns) > 0: |
862 | + name_cmd.update({'addresses': dns}) |
863 | + LOG.debug('v2(nameserver) -> v1(nameserver):\n%s', name_cmd) |
864 | + self.handle_nameserver(name_cmd) |
865 | + |
866 | + def _handle_bond_bridge(self, command, cmd_type=None): |
867 | + """Common handler for bond and bridge types""" |
868 | + for item_name, item_cfg in command.items(): |
869 | + item_params = dict((key, value) for (key, value) in |
870 | + item_cfg.items() if key not in |
871 | + NETWORK_V2_KEY_FILTER) |
872 | + v1_cmd = { |
873 | + 'type': cmd_type, |
874 | + 'name': item_name, |
875 | + cmd_type + '_interfaces': item_cfg.get('interfaces'), |
876 | + 'params': item_params, |
877 | + } |
878 | + subnets = self._v2_to_v1_ipcfg(item_cfg) |
879 | + if len(subnets) > 0: |
880 | + v1_cmd.update({'subnets': subnets}) |
881 | + |
882 | + LOG.debug('v2(%ss) -> v1(%s):\n%s', cmd_type, cmd_type, v1_cmd) |
883 | + self.handle_bridge(v1_cmd) |
884 | + |
885 | + def _v2_to_v1_ipcfg(self, cfg): |
886 | + """Common ipconfig extraction from v2 to v1 subnets array.""" |
887 | + |
888 | + subnets = [] |
889 | + if 'dhcp4' in cfg: |
890 | + subnets.append({'type': 'dhcp4'}) |
891 | + if 'dhcp6' in cfg: |
892 | + self.use_ipv6 = True |
893 | + subnets.append({'type': 'dhcp6'}) |
894 | + |
895 | + gateway4 = None |
896 | + gateway6 = None |
897 | + for address in cfg.get('addresses', []): |
898 | + subnet = { |
899 | + 'type': 'static', |
900 | + 'address': address, |
901 | + } |
902 | + |
903 | + routes = [] |
904 | + for route in cfg.get('routes', []): |
905 | + route_addr = route.get('to') |
906 | + if "/" in route_addr: |
907 | + route_addr, route_cidr = route_addr.split("/") |
908 | + route_netmask = cidr2mask(route_cidr) |
909 | + subnet_route = { |
910 | + 'address': route_addr, |
911 | + 'netmask': route_netmask, |
912 | + 'gateway': route.get('via') |
913 | + } |
914 | + routes.append(subnet_route) |
915 | + if len(routes) > 0: |
916 | + subnet.update({'routes': routes}) |
917 | + |
918 | + if ":" in address: |
919 | + if 'gateway6' in cfg and gateway6 is None: |
920 | + gateway6 = cfg.get('gateway6') |
921 | + subnet.update({'gateway': gateway6}) |
922 | + else: |
923 | + if 'gateway4' in cfg and gateway4 is None: |
924 | + gateway4 = cfg.get('gateway4') |
925 | + subnet.update({'gateway': gateway4}) |
926 | + |
927 | + subnets.append(subnet) |
928 | + return subnets |
929 | + |
930 | + |
931 | +def subnet_is_ipv6(subnet): |
932 | + """ Common helper for checking network_state subnets for ipv6""" |
933 | + # 'static6' or 'dhcp6' |
934 | + if subnet['type'].endswith('6'): |
935 | + # This is a request for DHCPv6. |
936 | + return True |
937 | + elif subnet['type'] == 'static' and ":" in subnet['address']: |
938 | + return True |
939 | + return False |
940 | + |
941 | |
942 | def cidr2mask(cidr): |
943 | mask = [0, 0, 0, 0] |
944 | diff --git a/cloudinit/net/renderer.py b/cloudinit/net/renderer.py |
945 | index a5b2b57..77bdc62 100644 |
946 | --- a/cloudinit/net/renderer.py |
947 | +++ b/cloudinit/net/renderer.py |
948 | @@ -41,4 +41,5 @@ class Renderer(object): |
949 | return self.render_network_state( |
950 | network_state=parse_net_config_data(network_config), target=target) |
951 | |
952 | + |
953 | # vi: ts=4 expandtab |
954 | diff --git a/cloudinit/net/renderers.py b/cloudinit/net/renderers.py |
955 | index 5ad8455..5117b4a 100644 |
956 | --- a/cloudinit/net/renderers.py |
957 | +++ b/cloudinit/net/renderers.py |
958 | @@ -1,15 +1,17 @@ |
959 | # This file is part of cloud-init. See LICENSE file for license information. |
960 | |
961 | from . import eni |
962 | +from . import netplan |
963 | from . import RendererNotFoundError |
964 | from . import sysconfig |
965 | |
966 | NAME_TO_RENDERER = { |
967 | "eni": eni, |
968 | + "netplan": netplan, |
969 | "sysconfig": sysconfig, |
970 | } |
971 | |
972 | -DEFAULT_PRIORITY = ["eni", "sysconfig"] |
973 | +DEFAULT_PRIORITY = ["eni", "sysconfig", "netplan"] |
974 | |
975 | |
976 | def search(priority=None, target=None, first=False): |
977 | diff --git a/cloudinit/net/sysconfig.py b/cloudinit/net/sysconfig.py |
978 | index 117b515..23ac2e3 100644 |
979 | --- a/cloudinit/net/sysconfig.py |
980 | +++ b/cloudinit/net/sysconfig.py |
981 | @@ -9,6 +9,7 @@ from cloudinit.distros.parsers import resolv_conf |
982 | from cloudinit import util |
983 | |
984 | from . import renderer |
985 | +from .network_state import subnet_is_ipv6 |
986 | |
987 | |
988 | def _make_header(sep='#'): |
989 | @@ -194,7 +195,7 @@ class Renderer(renderer.Renderer): |
990 | def __init__(self, config=None): |
991 | if not config: |
992 | config = {} |
993 | - self.sysconf_dir = config.get('sysconf_dir', 'etc/sysconfig/') |
994 | + self.sysconf_dir = config.get('sysconf_dir', 'etc/sysconfig') |
995 | self.netrules_path = config.get( |
996 | 'netrules_path', 'etc/udev/rules.d/70-persistent-net.rules') |
997 | self.dns_path = config.get('dns_path', 'etc/resolv.conf') |
998 | @@ -220,7 +221,7 @@ class Renderer(renderer.Renderer): |
999 | iface_cfg['BOOTPROTO'] = 'dhcp' |
1000 | elif subnet_type == 'static': |
1001 | iface_cfg['BOOTPROTO'] = 'static' |
1002 | - if subnet.get('ipv6'): |
1003 | + if subnet_is_ipv6(subnet): |
1004 | iface_cfg['IPV6ADDR'] = subnet['address'] |
1005 | iface_cfg['IPV6INIT'] = True |
1006 | else: |
1007 | @@ -390,19 +391,32 @@ class Renderer(renderer.Renderer): |
1008 | return contents |
1009 | |
1010 | def render_network_state(self, network_state, target=None): |
1011 | + file_mode = 0o644 |
1012 | base_sysconf_dir = util.target_path(target, self.sysconf_dir) |
1013 | for path, data in self._render_sysconfig(base_sysconf_dir, |
1014 | network_state).items(): |
1015 | - util.write_file(path, data) |
1016 | + util.write_file(path, data, file_mode) |
1017 | if self.dns_path: |
1018 | dns_path = util.target_path(target, self.dns_path) |
1019 | resolv_content = self._render_dns(network_state, |
1020 | existing_dns_path=dns_path) |
1021 | - util.write_file(dns_path, resolv_content) |
1022 | + util.write_file(dns_path, resolv_content, file_mode) |
1023 | if self.netrules_path: |
1024 | netrules_content = self._render_persistent_net(network_state) |
1025 | netrules_path = util.target_path(target, self.netrules_path) |
1026 | - util.write_file(netrules_path, netrules_content) |
1027 | + util.write_file(netrules_path, netrules_content, file_mode) |
1028 | + |
1029 | + # always write /etc/sysconfig/network configuration |
1030 | + sysconfig_path = util.target_path(target, "etc/sysconfig/network") |
1031 | + netcfg = [ |
1032 | + ('# Created by cloud-init on instance boot automatically, ' |
1033 | + 'do not edit.'), |
1034 | + 'NETWORKING=yes', |
1035 | + ] |
1036 | + if network_state.use_ipv6: |
1037 | + netcfg.append('NETWORKING_IPV6=yes') |
1038 | + netcfg.append('IPV6_AUTOCONF=no') |
1039 | + util.write_file(sysconfig_path, "\n".join(netcfg) + "\n", file_mode) |
1040 | |
1041 | |
1042 | def available(target=None): |
1043 | diff --git a/systemd/cloud-init.service b/systemd/cloud-init.service |
1044 | index fb3b918..39acc20 100644 |
1045 | --- a/systemd/cloud-init.service |
1046 | +++ b/systemd/cloud-init.service |
1047 | @@ -5,6 +5,7 @@ Wants=cloud-init-local.service |
1048 | Wants=sshd-keygen.service |
1049 | Wants=sshd.service |
1050 | After=cloud-init-local.service |
1051 | +After=systemd-networkd-wait-online.service |
1052 | After=networking.service |
1053 | Before=network-online.target |
1054 | Before=sshd-keygen.service |
1055 | diff --git a/systemd/systemd-networkd-wait-online.path b/systemd/systemd-networkd-wait-online.path |
1056 | new file mode 100644 |
1057 | index 0000000..64940b8 |
1058 | --- /dev/null |
1059 | +++ b/systemd/systemd-networkd-wait-online.path |
1060 | @@ -0,0 +1,5 @@ |
1061 | +[Unit] |
1062 | +Description=Trigger systemd-networkd-wait-online if netplan runs/updates |
1063 | + |
1064 | +[Path] |
1065 | +PathChanged=/run/systemd/generator/netplan.stamp |
1066 | diff --git a/tests/unittests/test_distros/test_netconfig.py b/tests/unittests/test_distros/test_netconfig.py |
1067 | index bde3bb5..2188779 100644 |
1068 | --- a/tests/unittests/test_distros/test_netconfig.py |
1069 | +++ b/tests/unittests/test_distros/test_netconfig.py |
1070 | @@ -19,6 +19,7 @@ from cloudinit.distros.parsers.sys_conf import SysConf |
1071 | from cloudinit import helpers |
1072 | from cloudinit import settings |
1073 | from cloudinit import util |
1074 | +from cloudinit.net import eni |
1075 | |
1076 | |
1077 | BASE_NET_CFG = ''' |
1078 | @@ -28,10 +29,10 @@ iface lo inet loopback |
1079 | auto eth0 |
1080 | iface eth0 inet static |
1081 | address 192.168.1.5 |
1082 | - netmask 255.255.255.0 |
1083 | - network 192.168.0.0 |
1084 | broadcast 192.168.1.0 |
1085 | gateway 192.168.1.254 |
1086 | + netmask 255.255.255.0 |
1087 | + network 192.168.0.0 |
1088 | |
1089 | auto eth1 |
1090 | iface eth1 inet dhcp |
1091 | @@ -67,6 +68,100 @@ iface eth1 inet6 static |
1092 | gateway 2607:f0d0:1002:0011::1 |
1093 | ''' |
1094 | |
1095 | +V1_NET_CFG = {'config': [{'name': 'eth0', |
1096 | + |
1097 | + 'subnets': [{'address': '192.168.1.5', |
1098 | + 'broadcast': '192.168.1.0', |
1099 | + 'gateway': '192.168.1.254', |
1100 | + 'netmask': '255.255.255.0', |
1101 | + 'type': 'static'}], |
1102 | + 'type': 'physical'}, |
1103 | + {'name': 'eth1', |
1104 | + 'subnets': [{'control': 'auto', 'type': 'dhcp4'}], |
1105 | + 'type': 'physical'}], |
1106 | + 'version': 1} |
1107 | + |
1108 | +V1_NET_CFG_OUTPUT = """ |
1109 | +# This file is generated from information provided by |
1110 | +# the datasource. Changes to it will not persist across an instance. |
1111 | +# To disable cloud-init's network configuration capabilities, write a file |
1112 | +# /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following: |
1113 | +# network: {config: disabled} |
1114 | +auto lo |
1115 | +iface lo inet loopback |
1116 | + |
1117 | +auto eth0 |
1118 | +iface eth0 inet static |
1119 | + address 192.168.1.5 |
1120 | + broadcast 192.168.1.0 |
1121 | + gateway 192.168.1.254 |
1122 | + netmask 255.255.255.0 |
1123 | + |
1124 | +auto eth1 |
1125 | +iface eth1 inet dhcp |
1126 | +""" |
1127 | + |
1128 | +V1_NET_CFG_IPV6 = {'config': [{'name': 'eth0', |
1129 | + 'subnets': [{'address': |
1130 | + '2607:f0d0:1002:0011::2', |
1131 | + 'gateway': |
1132 | + '2607:f0d0:1002:0011::1', |
1133 | + 'netmask': '64', |
1134 | + 'type': 'static'}], |
1135 | + 'type': 'physical'}, |
1136 | + {'name': 'eth1', |
1137 | + 'subnets': [{'control': 'auto', |
1138 | + 'type': 'dhcp4'}], |
1139 | + 'type': 'physical'}], |
1140 | + 'version': 1} |
1141 | + |
1142 | + |
1143 | +V1_TO_V2_NET_CFG_OUTPUT = """ |
1144 | +# This file is generated from information provided by |
1145 | +# the datasource. Changes to it will not persist across an instance. |
1146 | +# To disable cloud-init's network configuration capabilities, write a file |
1147 | +# /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following: |
1148 | +# network: {config: disabled} |
1149 | +network: |
1150 | + version: 2 |
1151 | + ethernets: |
1152 | + eth0: |
1153 | + addresses: |
1154 | + - 192.168.1.5/255.255.255.0 |
1155 | + gateway4: 192.168.1.254 |
1156 | + eth1: |
1157 | + dhcp4: true |
1158 | +""" |
1159 | + |
1160 | +V2_NET_CFG = { |
1161 | + 'ethernets': { |
1162 | + 'eth7': { |
1163 | + 'addresses': ['192.168.1.5/255.255.255.0'], |
1164 | + 'gateway4': '192.168.1.254'}, |
1165 | + 'eth9': { |
1166 | + 'dhcp4': True} |
1167 | + }, |
1168 | + 'version': 2 |
1169 | +} |
1170 | + |
1171 | + |
1172 | +V2_TO_V2_NET_CFG_OUTPUT = """ |
1173 | +# This file is generated from information provided by |
1174 | +# the datasource. Changes to it will not persist across an instance. |
1175 | +# To disable cloud-init's network configuration capabilities, write a file |
1176 | +# /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following: |
1177 | +# network: {config: disabled} |
1178 | +network: |
1179 | + ethernets: |
1180 | + eth7: |
1181 | + addresses: |
1182 | + - 192.168.1.5/255.255.255.0 |
1183 | + gateway4: 192.168.1.254 |
1184 | + eth9: |
1185 | + dhcp4: true |
1186 | + version: 2 |
1187 | +""" |
1188 | + |
1189 | |
1190 | class WriteBuffer(object): |
1191 | def __init__(self): |
1192 | @@ -83,10 +178,12 @@ class WriteBuffer(object): |
1193 | |
1194 | class TestNetCfgDistro(TestCase): |
1195 | |
1196 | - def _get_distro(self, dname): |
1197 | + def _get_distro(self, dname, renderers=None): |
1198 | cls = distros.fetch(dname) |
1199 | cfg = settings.CFG_BUILTIN |
1200 | cfg['system_info']['distro'] = dname |
1201 | + if renderers: |
1202 | + cfg['system_info']['network'] = {'renderers': renderers} |
1203 | paths = helpers.Paths({}) |
1204 | return cls(dname, cfg, paths) |
1205 | |
1206 | @@ -116,6 +213,107 @@ class TestNetCfgDistro(TestCase): |
1207 | self.assertEqual(str(write_buf).strip(), BASE_NET_CFG.strip()) |
1208 | self.assertEqual(write_buf.mode, 0o644) |
1209 | |
1210 | + def test_apply_network_config_eni_ub(self): |
1211 | + ub_distro = self._get_distro('ubuntu') |
1212 | + with ExitStack() as mocks: |
1213 | + write_bufs = {} |
1214 | + |
1215 | + def replace_write(filename, content, mode=0o644, omode="wb"): |
1216 | + buf = WriteBuffer() |
1217 | + buf.mode = mode |
1218 | + buf.omode = omode |
1219 | + buf.write(content) |
1220 | + write_bufs[filename] = buf |
1221 | + |
1222 | + # eni availability checks |
1223 | + mocks.enter_context( |
1224 | + mock.patch.object(util, 'which', return_value=True)) |
1225 | + mocks.enter_context( |
1226 | + mock.patch.object(eni, 'available', return_value=True)) |
1227 | + mocks.enter_context( |
1228 | + mock.patch.object(util, 'ensure_dir')) |
1229 | + mocks.enter_context( |
1230 | + mock.patch.object(util, 'write_file', replace_write)) |
1231 | + mocks.enter_context( |
1232 | + mock.patch.object(os.path, 'isfile', return_value=True)) |
1233 | + |
1234 | + ub_distro.apply_network_config(V1_NET_CFG, False) |
1235 | + |
1236 | + self.assertEqual(len(write_bufs), 2) |
1237 | + eni_name = '/etc/network/interfaces.d/50-cloud-init.cfg' |
1238 | + self.assertIn(eni_name, write_bufs) |
1239 | + write_buf = write_bufs[eni_name] |
1240 | + self.assertEqual(str(write_buf).strip(), V1_NET_CFG_OUTPUT.strip()) |
1241 | + self.assertEqual(write_buf.mode, 0o644) |
1242 | + |
1243 | + def test_apply_network_config_v1_to_netplan_ub(self): |
1244 | + renderers = ['netplan'] |
1245 | + ub_distro = self._get_distro('ubuntu', renderers=renderers) |
1246 | + with ExitStack() as mocks: |
1247 | + write_bufs = {} |
1248 | + |
1249 | + def replace_write(filename, content, mode=0o644, omode="wb"): |
1250 | + buf = WriteBuffer() |
1251 | + buf.mode = mode |
1252 | + buf.omode = omode |
1253 | + buf.write(content) |
1254 | + write_bufs[filename] = buf |
1255 | + |
1256 | + mocks.enter_context( |
1257 | + mock.patch.object(util, 'which', return_value=True)) |
1258 | + mocks.enter_context( |
1259 | + mock.patch.object(util, 'write_file', replace_write)) |
1260 | + mocks.enter_context( |
1261 | + mock.patch.object(util, 'ensure_dir')) |
1262 | + mocks.enter_context( |
1263 | + mock.patch.object(util, 'subp', return_value=(0, 0))) |
1264 | + mocks.enter_context( |
1265 | + mock.patch.object(os.path, 'isfile', return_value=True)) |
1266 | + |
1267 | + ub_distro.apply_network_config(V1_NET_CFG, False) |
1268 | + |
1269 | + self.assertEqual(len(write_bufs), 1) |
1270 | + netplan_name = '/etc/netplan/50-cloud-init.yaml' |
1271 | + self.assertIn(netplan_name, write_bufs) |
1272 | + write_buf = write_bufs[netplan_name] |
1273 | + self.assertEqual(str(write_buf).strip(), |
1274 | + V1_TO_V2_NET_CFG_OUTPUT.strip()) |
1275 | + self.assertEqual(write_buf.mode, 0o644) |
1276 | + |
1277 | + def test_apply_network_config_v2_passthrough_ub(self): |
1278 | + renderers = ['netplan'] |
1279 | + ub_distro = self._get_distro('ubuntu', renderers=renderers) |
1280 | + with ExitStack() as mocks: |
1281 | + write_bufs = {} |
1282 | + |
1283 | + def replace_write(filename, content, mode=0o644, omode="wb"): |
1284 | + buf = WriteBuffer() |
1285 | + buf.mode = mode |
1286 | + buf.omode = omode |
1287 | + buf.write(content) |
1288 | + write_bufs[filename] = buf |
1289 | + |
1290 | + mocks.enter_context( |
1291 | + mock.patch.object(util, 'which', return_value=True)) |
1292 | + mocks.enter_context( |
1293 | + mock.patch.object(util, 'write_file', replace_write)) |
1294 | + mocks.enter_context( |
1295 | + mock.patch.object(util, 'ensure_dir')) |
1296 | + mocks.enter_context( |
1297 | + mock.patch.object(util, 'subp', return_value=(0, 0))) |
1298 | + mocks.enter_context( |
1299 | + mock.patch.object(os.path, 'isfile', return_value=True)) |
1300 | + |
1301 | + ub_distro.apply_network_config(V2_NET_CFG, False) |
1302 | + |
1303 | + self.assertEqual(len(write_bufs), 1) |
1304 | + netplan_name = '/etc/netplan/50-cloud-init.yaml' |
1305 | + self.assertIn(netplan_name, write_bufs) |
1306 | + write_buf = write_bufs[netplan_name] |
1307 | + self.assertEqual(str(write_buf).strip(), |
1308 | + V2_TO_V2_NET_CFG_OUTPUT.strip()) |
1309 | + self.assertEqual(write_buf.mode, 0o644) |
1310 | + |
1311 | def assertCfgEquals(self, blob1, blob2): |
1312 | b1 = dict(SysConf(blob1.strip().splitlines())) |
1313 | b2 = dict(SysConf(blob2.strip().splitlines())) |
1314 | @@ -195,6 +393,79 @@ NETWORKING=yes |
1315 | self.assertCfgEquals(expected_buf, str(write_buf)) |
1316 | self.assertEqual(write_buf.mode, 0o644) |
1317 | |
1318 | + def test_apply_network_config_rh(self): |
1319 | + renderers = ['sysconfig'] |
1320 | + rh_distro = self._get_distro('rhel', renderers=renderers) |
1321 | + |
1322 | + write_bufs = {} |
1323 | + |
1324 | + def replace_write(filename, content, mode=0o644, omode="wb"): |
1325 | + buf = WriteBuffer() |
1326 | + buf.mode = mode |
1327 | + buf.omode = omode |
1328 | + buf.write(content) |
1329 | + write_bufs[filename] = buf |
1330 | + |
1331 | + with ExitStack() as mocks: |
1332 | + # sysconfig availability checks |
1333 | + mocks.enter_context( |
1334 | + mock.patch.object(util, 'which', return_value=True)) |
1335 | + mocks.enter_context( |
1336 | + mock.patch.object(util, 'write_file', replace_write)) |
1337 | + mocks.enter_context( |
1338 | + mock.patch.object(util, 'load_file', return_value='')) |
1339 | + mocks.enter_context( |
1340 | + mock.patch.object(os.path, 'isfile', return_value=True)) |
1341 | + |
1342 | + rh_distro.apply_network_config(V1_NET_CFG, False) |
1343 | + |
1344 | + self.assertEqual(len(write_bufs), 5) |
1345 | + |
1346 | + # eth0 |
1347 | + self.assertIn('/etc/sysconfig/network-scripts/ifcfg-eth0', |
1348 | + write_bufs) |
1349 | + write_buf = write_bufs['/etc/sysconfig/network-scripts/ifcfg-eth0'] |
1350 | + expected_buf = ''' |
1351 | +# Created by cloud-init on instance boot automatically, do not edit. |
1352 | +# |
1353 | +BOOTPROTO=static |
1354 | +DEVICE=eth0 |
1355 | +IPADDR=192.168.1.5 |
1356 | +NETMASK=255.255.255.0 |
1357 | +NM_CONTROLLED=no |
1358 | +ONBOOT=yes |
1359 | +TYPE=Ethernet |
1360 | +USERCTL=no |
1361 | +''' |
1362 | + self.assertCfgEquals(expected_buf, str(write_buf)) |
1363 | + self.assertEqual(write_buf.mode, 0o644) |
1364 | + |
1365 | + # eth1 |
1366 | + self.assertIn('/etc/sysconfig/network-scripts/ifcfg-eth1', |
1367 | + write_bufs) |
1368 | + write_buf = write_bufs['/etc/sysconfig/network-scripts/ifcfg-eth1'] |
1369 | + expected_buf = ''' |
1370 | +# Created by cloud-init on instance boot automatically, do not edit. |
1371 | +# |
1372 | +BOOTPROTO=dhcp |
1373 | +DEVICE=eth1 |
1374 | +NM_CONTROLLED=no |
1375 | +ONBOOT=yes |
1376 | +TYPE=Ethernet |
1377 | +USERCTL=no |
1378 | +''' |
1379 | + self.assertCfgEquals(expected_buf, str(write_buf)) |
1380 | + self.assertEqual(write_buf.mode, 0o644) |
1381 | + |
1382 | + self.assertIn('/etc/sysconfig/network', write_bufs) |
1383 | + write_buf = write_bufs['/etc/sysconfig/network'] |
1384 | + expected_buf = ''' |
1385 | +# Created by cloud-init v. 0.7 |
1386 | +NETWORKING=yes |
1387 | +''' |
1388 | + self.assertCfgEquals(expected_buf, str(write_buf)) |
1389 | + self.assertEqual(write_buf.mode, 0o644) |
1390 | + |
1391 | def test_write_ipv6_rhel(self): |
1392 | rh_distro = self._get_distro('rhel') |
1393 | |
1394 | @@ -213,7 +484,7 @@ NETWORKING=yes |
1395 | mocks.enter_context( |
1396 | mock.patch.object(util, 'load_file', return_value='')) |
1397 | mocks.enter_context( |
1398 | - mock.patch.object(os.path, 'isfile', return_value=False)) |
1399 | + mock.patch.object(os.path, 'isfile', return_value=True)) |
1400 | |
1401 | rh_distro.apply_network(BASE_NET_CFG_IPV6, False) |
1402 | |
1403 | @@ -274,6 +545,78 @@ IPV6_AUTOCONF=no |
1404 | self.assertCfgEquals(expected_buf, str(write_buf)) |
1405 | self.assertEqual(write_buf.mode, 0o644) |
1406 | |
1407 | + def test_apply_network_config_ipv6_rh(self): |
1408 | + renderers = ['sysconfig'] |
1409 | + rh_distro = self._get_distro('rhel', renderers=renderers) |
1410 | + |
1411 | + write_bufs = {} |
1412 | + |
1413 | + def replace_write(filename, content, mode=0o644, omode="wb"): |
1414 | + buf = WriteBuffer() |
1415 | + buf.mode = mode |
1416 | + buf.omode = omode |
1417 | + buf.write(content) |
1418 | + write_bufs[filename] = buf |
1419 | + |
1420 | + with ExitStack() as mocks: |
1421 | + mocks.enter_context( |
1422 | + mock.patch.object(util, 'which', return_value=True)) |
1423 | + mocks.enter_context( |
1424 | + mock.patch.object(util, 'write_file', replace_write)) |
1425 | + mocks.enter_context( |
1426 | + mock.patch.object(util, 'load_file', return_value='')) |
1427 | + mocks.enter_context( |
1428 | + mock.patch.object(os.path, 'isfile', return_value=True)) |
1429 | + |
1430 | + rh_distro.apply_network_config(V1_NET_CFG_IPV6, False) |
1431 | + |
1432 | + self.assertEqual(len(write_bufs), 5) |
1433 | + |
1434 | + self.assertIn('/etc/sysconfig/network-scripts/ifcfg-eth0', |
1435 | + write_bufs) |
1436 | + write_buf = write_bufs['/etc/sysconfig/network-scripts/ifcfg-eth0'] |
1437 | + expected_buf = ''' |
1438 | +# Created by cloud-init on instance boot automatically, do not edit. |
1439 | +# |
1440 | +BOOTPROTO=static |
1441 | +DEVICE=eth0 |
1442 | +IPV6ADDR=2607:f0d0:1002:0011::2 |
1443 | +IPV6INIT=yes |
1444 | +NETMASK=64 |
1445 | +NM_CONTROLLED=no |
1446 | +ONBOOT=yes |
1447 | +TYPE=Ethernet |
1448 | +USERCTL=no |
1449 | +''' |
1450 | + self.assertCfgEquals(expected_buf, str(write_buf)) |
1451 | + self.assertEqual(write_buf.mode, 0o644) |
1452 | + self.assertIn('/etc/sysconfig/network-scripts/ifcfg-eth1', |
1453 | + write_bufs) |
1454 | + write_buf = write_bufs['/etc/sysconfig/network-scripts/ifcfg-eth1'] |
1455 | + expected_buf = ''' |
1456 | +# Created by cloud-init on instance boot automatically, do not edit. |
1457 | +# |
1458 | +BOOTPROTO=dhcp |
1459 | +DEVICE=eth1 |
1460 | +NM_CONTROLLED=no |
1461 | +ONBOOT=yes |
1462 | +TYPE=Ethernet |
1463 | +USERCTL=no |
1464 | +''' |
1465 | + self.assertCfgEquals(expected_buf, str(write_buf)) |
1466 | + self.assertEqual(write_buf.mode, 0o644) |
1467 | + |
1468 | + self.assertIn('/etc/sysconfig/network', write_bufs) |
1469 | + write_buf = write_bufs['/etc/sysconfig/network'] |
1470 | + expected_buf = ''' |
1471 | +# Created by cloud-init v. 0.7 |
1472 | +NETWORKING=yes |
1473 | +NETWORKING_IPV6=yes |
1474 | +IPV6_AUTOCONF=no |
1475 | +''' |
1476 | + self.assertCfgEquals(expected_buf, str(write_buf)) |
1477 | + self.assertEqual(write_buf.mode, 0o644) |
1478 | + |
1479 | def test_simple_write_freebsd(self): |
1480 | fbsd_distro = self._get_distro('freebsd') |
1481 | |
1482 | diff --git a/tests/unittests/test_net.py b/tests/unittests/test_net.py |
1483 | index 902204a..4f07d80 100644 |
1484 | --- a/tests/unittests/test_net.py |
1485 | +++ b/tests/unittests/test_net.py |
1486 | @@ -3,6 +3,7 @@ |
1487 | from cloudinit import net |
1488 | from cloudinit.net import cmdline |
1489 | from cloudinit.net import eni |
1490 | +from cloudinit.net import netplan |
1491 | from cloudinit.net import network_state |
1492 | from cloudinit.net import renderers |
1493 | from cloudinit.net import sysconfig |
1494 | @@ -408,6 +409,41 @@ NETWORK_CONFIGS = { |
1495 | post-up route add default gw 65.61.151.37 || true |
1496 | pre-down route del default gw 65.61.151.37 || true |
1497 | """).rstrip(' '), |
1498 | + 'expected_netplan': textwrap.dedent(""" |
1499 | + network: |
1500 | + version: 2 |
1501 | + ethernets: |
1502 | + eth1: |
1503 | + match: |
1504 | + macaddress: cf:d6:af:48:e8:80 |
1505 | + nameservers: |
1506 | + addresses: |
1507 | + - 1.2.3.4 |
1508 | + - 5.6.7.8 |
1509 | + search: |
1510 | + - wark.maas |
1511 | + set-name: eth1 |
1512 | + eth99: |
1513 | + addresses: |
1514 | + - 192.168.21.3/24 |
1515 | + dhcp4: true |
1516 | + match: |
1517 | + macaddress: c0:d6:9f:2c:e8:80 |
1518 | + nameservers: |
1519 | + addresses: |
1520 | + - 8.8.8.8 |
1521 | + - 8.8.4.4 |
1522 | + - 1.2.3.4 |
1523 | + - 5.6.7.8 |
1524 | + search: |
1525 | + - barley.maas |
1526 | + - sach.maas |
1527 | + - wark.maas |
1528 | + routes: |
1529 | + - to: 0.0.0.0/0.0.0.0 |
1530 | + via: 65.61.151.37 |
1531 | + set-name: eth99 |
1532 | + """).rstrip(' '), |
1533 | 'yaml': textwrap.dedent(""" |
1534 | version: 1 |
1535 | config: |
1536 | @@ -450,6 +486,14 @@ NETWORK_CONFIGS = { |
1537 | # control-alias iface0 |
1538 | iface iface0 inet6 dhcp |
1539 | """).rstrip(' '), |
1540 | + 'expected_netplan': textwrap.dedent(""" |
1541 | + network: |
1542 | + version: 2 |
1543 | + ethernets: |
1544 | + iface0: |
1545 | + dhcp4: true |
1546 | + dhcp6: true |
1547 | + """).rstrip(' '), |
1548 | 'yaml': textwrap.dedent("""\ |
1549 | version: 1 |
1550 | config: |
1551 | @@ -524,6 +568,126 @@ iface eth0.101 inet static |
1552 | post-up route add -net 10.0.0.0 netmask 255.0.0.0 gw 11.0.0.1 metric 3 || true |
1553 | pre-down route del -net 10.0.0.0 netmask 255.0.0.0 gw 11.0.0.1 metric 3 || true |
1554 | """), |
1555 | + 'expected_netplan': textwrap.dedent(""" |
1556 | + network: |
1557 | + version: 2 |
1558 | + ethernets: |
1559 | + eth0: |
1560 | + match: |
1561 | + macaddress: c0:d6:9f:2c:e8:80 |
1562 | + nameservers: |
1563 | + addresses: |
1564 | + - 8.8.8.8 |
1565 | + - 4.4.4.4 |
1566 | + - 8.8.4.4 |
1567 | + search: |
1568 | + - barley.maas |
1569 | + - wark.maas |
1570 | + - foobar.maas |
1571 | + set-name: eth0 |
1572 | + eth1: |
1573 | + match: |
1574 | + macaddress: aa:d6:9f:2c:e8:80 |
1575 | + nameservers: |
1576 | + addresses: |
1577 | + - 8.8.8.8 |
1578 | + - 4.4.4.4 |
1579 | + - 8.8.4.4 |
1580 | + search: |
1581 | + - barley.maas |
1582 | + - wark.maas |
1583 | + - foobar.maas |
1584 | + set-name: eth1 |
1585 | + eth2: |
1586 | + match: |
1587 | + macaddress: c0:bb:9f:2c:e8:80 |
1588 | + nameservers: |
1589 | + addresses: |
1590 | + - 8.8.8.8 |
1591 | + - 4.4.4.4 |
1592 | + - 8.8.4.4 |
1593 | + search: |
1594 | + - barley.maas |
1595 | + - wark.maas |
1596 | + - foobar.maas |
1597 | + set-name: eth2 |
1598 | + eth3: |
1599 | + match: |
1600 | + macaddress: 66:bb:9f:2c:e8:80 |
1601 | + nameservers: |
1602 | + addresses: |
1603 | + - 8.8.8.8 |
1604 | + - 4.4.4.4 |
1605 | + - 8.8.4.4 |
1606 | + search: |
1607 | + - barley.maas |
1608 | + - wark.maas |
1609 | + - foobar.maas |
1610 | + set-name: eth3 |
1611 | + eth4: |
1612 | + match: |
1613 | + macaddress: 98:bb:9f:2c:e8:80 |
1614 | + nameservers: |
1615 | + addresses: |
1616 | + - 8.8.8.8 |
1617 | + - 4.4.4.4 |
1618 | + - 8.8.4.4 |
1619 | + search: |
1620 | + - barley.maas |
1621 | + - wark.maas |
1622 | + - foobar.maas |
1623 | + set-name: eth4 |
1624 | + eth5: |
1625 | + dhcp4: true |
1626 | + match: |
1627 | + macaddress: 98:bb:9f:2c:e8:8a |
1628 | + nameservers: |
1629 | + addresses: |
1630 | + - 8.8.8.8 |
1631 | + - 4.4.4.4 |
1632 | + - 8.8.4.4 |
1633 | + search: |
1634 | + - barley.maas |
1635 | + - wark.maas |
1636 | + - foobar.maas |
1637 | + set-name: eth5 |
1638 | + bonds: |
1639 | + bond0: |
1640 | + dhcp6: true |
1641 | + interfaces: |
1642 | + - eth1 |
1643 | + - eth2 |
1644 | + parameters: |
1645 | + mode: active-backup |
1646 | + bridges: |
1647 | + br0: |
1648 | + addresses: |
1649 | + - 192.168.14.2/24 |
1650 | + - 2001:1::1/64 |
1651 | + interfaces: |
1652 | + - eth3 |
1653 | + - eth4 |
1654 | + vlans: |
1655 | + bond0.200: |
1656 | + dhcp4: true |
1657 | + id: 200 |
1658 | + link: bond0 |
1659 | + eth0.101: |
1660 | + addresses: |
1661 | + - 192.168.0.2/24 |
1662 | + - 192.168.2.10/24 |
1663 | + gateway4: 192.168.0.1 |
1664 | + id: 101 |
1665 | + link: eth0 |
1666 | + nameservers: |
1667 | + addresses: |
1668 | + - 192.168.0.10 |
1669 | + - 10.23.23.134 |
1670 | + search: |
1671 | + - barley.maas |
1672 | + - sacchromyces.maas |
1673 | + - brettanomyces.maas |
1674 | + """).rstrip(' '), |
1675 | 'yaml': textwrap.dedent(""" |
1676 | version: 1 |
1677 | config: |
1678 | @@ -808,6 +972,99 @@ iface eth0 inet dhcp |
1679 | expected, dir2dict(tmp_dir)['/etc/network/interfaces']) |
1680 | |
1681 | |
1682 | +class TestNetplanNetRendering(CiTestCase): |
1683 | + |
1684 | + @mock.patch("cloudinit.net.sys_dev_path") |
1685 | + @mock.patch("cloudinit.net.read_sys_net") |
1686 | + @mock.patch("cloudinit.net.get_devicelist") |
1687 | + def test_default_generation(self, mock_get_devicelist, |
1688 | + mock_read_sys_net, |
1689 | + mock_sys_dev_path): |
1690 | + tmp_dir = self.tmp_dir() |
1691 | + _setup_test(tmp_dir, mock_get_devicelist, |
1692 | + mock_read_sys_net, mock_sys_dev_path) |
1693 | + |
1694 | + network_cfg = net.generate_fallback_config() |
1695 | + ns = network_state.parse_net_config_data(network_cfg, |
1696 | + skip_broken=False) |
1697 | + |
1698 | + render_dir = os.path.join(tmp_dir, "render") |
1699 | + os.makedirs(render_dir) |
1700 | + |
1701 | + render_target = 'netplan.yaml' |
1702 | + renderer = netplan.Renderer( |
1703 | + {'netplan_path': render_target, 'postcmds': False}) |
1704 | + renderer.render_network_state(render_dir, ns) |
1705 | + |
1706 | + self.assertTrue(os.path.exists(os.path.join(render_dir, |
1707 | + render_target))) |
1708 | + with open(os.path.join(render_dir, render_target)) as fh: |
1709 | + contents = fh.read() |
1710 | + print(contents) |
1711 | + |
1712 | + expected = """ |
1713 | +network: |
1714 | + version: 2 |
1715 | + ethernets: |
1716 | + eth1000: |
1717 | + dhcp4: true |
1718 | + match: |
1719 | + macaddress: 07-1c-c6-75-a4-be |
1720 | + set-name: eth1000 |
1721 | +""" |
1722 | + self.assertEqual(expected.lstrip(), contents.lstrip()) |
1723 | + |
1724 | + |
1725 | +class TestNetplanPostcommands(CiTestCase): |
1726 | + mycfg = { |
1727 | + 'config': [{"type": "physical", "name": "eth0", |
1728 | + "mac_address": "c0:d6:9f:2c:e8:80", |
1729 | + "subnets": [{"type": "dhcp"}]}], |
1730 | + 'version': 1} |
1731 | + |
1732 | + @mock.patch.object(netplan.Renderer, '_netplan_generate') |
1733 | + @mock.patch.object(netplan.Renderer, '_net_setup_link') |
1734 | + def test_netplan_render_calls_postcmds(self, mock_netplan_generate, |
1735 | + mock_net_setup_link): |
1736 | + tmp_dir = self.tmp_dir() |
1737 | + ns = network_state.parse_net_config_data(self.mycfg, |
1738 | + skip_broken=False) |
1739 | + |
1740 | + render_dir = os.path.join(tmp_dir, "render") |
1741 | + os.makedirs(render_dir) |
1742 | + |
1743 | + render_target = 'netplan.yaml' |
1744 | + renderer = netplan.Renderer( |
1745 | + {'netplan_path': render_target, 'postcmds': True}) |
1746 | + renderer.render_network_state(render_dir, ns) |
1747 | + |
1748 | + mock_netplan_generate.assert_called_with(run=True) |
1749 | + mock_net_setup_link.assert_called_with(run=True) |
1750 | + |
1751 | + @mock.patch.object(netplan, "get_devicelist") |
1752 | + @mock.patch('cloudinit.util.subp') |
1753 | + def test_netplan_postcmds(self, mock_subp, mock_devlist): |
1754 | + mock_devlist.side_effect = [['lo']] |
1755 | + tmp_dir = self.tmp_dir() |
1756 | + ns = network_state.parse_net_config_data(self.mycfg, |
1757 | + skip_broken=False) |
1758 | + |
1759 | + render_dir = os.path.join(tmp_dir, "render") |
1760 | + os.makedirs(render_dir) |
1761 | + |
1762 | + render_target = 'netplan.yaml' |
1763 | + renderer = netplan.Renderer( |
1764 | + {'netplan_path': render_target, 'postcmds': True}) |
1765 | + renderer.render_network_state(render_dir, ns) |
1766 | + |
1767 | + expected = [ |
1768 | + mock.call(['netplan', 'generate'], capture=True), |
1769 | + mock.call(['udevadm', 'test-builtin', 'net_setup_link', |
1770 | + '/sys/class/net/lo'], capture=True), |
1771 | + ] |
1772 | + mock_subp.assert_has_calls(expected) |
1773 | + |
1774 | + |
1775 | class TestEniNetworkStateToEni(CiTestCase): |
1776 | mycfg = { |
1777 | 'config': [{"type": "physical", "name": "eth0", |
1778 | @@ -953,6 +1210,50 @@ class TestCmdlineReadKernelConfig(CiTestCase): |
1779 | self.assertEqual(found['config'], expected) |
1780 | |
1781 | |
1782 | +class TestNetplanRoundTrip(CiTestCase): |
1783 | + def _render_and_read(self, network_config=None, state=None, |
1784 | + netplan_path=None, dir=None): |
1785 | + if dir is None: |
1786 | + dir = self.tmp_dir() |
1787 | + |
1788 | + if network_config: |
1789 | + ns = network_state.parse_net_config_data(network_config) |
1790 | + elif state: |
1791 | + ns = state |
1792 | + else: |
1793 | + raise ValueError("Expected data or state, got neither") |
1794 | + |
1795 | + if netplan_path is None: |
1796 | + netplan_path = 'etc/netplan/50-cloud-init.yaml' |
1797 | + |
1798 | + renderer = netplan.Renderer( |
1799 | + config={'netplan_path': netplan_path}) |
1800 | + |
1801 | + renderer.render_network_state(dir, ns) |
1802 | + return dir2dict(dir) |
1803 | + |
1804 | + def testsimple_render_small_netplan(self): |
1805 | + entry = NETWORK_CONFIGS['small'] |
1806 | + files = self._render_and_read(network_config=yaml.load(entry['yaml'])) |
1807 | + self.assertEqual( |
1808 | + entry['expected_netplan'].splitlines(), |
1809 | + files['/etc/netplan/50-cloud-init.yaml'].splitlines()) |
1810 | + |
1811 | + def testsimple_render_v4_and_v6(self): |
1812 | + entry = NETWORK_CONFIGS['v4_and_v6'] |
1813 | + files = self._render_and_read(network_config=yaml.load(entry['yaml'])) |
1814 | + self.assertEqual( |
1815 | + entry['expected_netplan'].splitlines(), |
1816 | + files['/etc/netplan/50-cloud-init.yaml'].splitlines()) |
1817 | + |
1818 | + def testsimple_render_all(self): |
1819 | + entry = NETWORK_CONFIGS['all'] |
1820 | + files = self._render_and_read(network_config=yaml.load(entry['yaml'])) |
1821 | + self.assertEqual( |
1822 | + entry['expected_netplan'].splitlines(), |
1823 | + files['/etc/netplan/50-cloud-init.yaml'].splitlines()) |
1824 | + |
1825 | + |
1826 | class TestEniRoundTrip(CiTestCase): |
1827 | def _render_and_read(self, network_config=None, state=None, eni_path=None, |
1828 | links_prefix=None, netrules_path=None, dir=None): |
1829 | diff --git a/tools/net-convert.py b/tools/net-convert.py |
1830 | new file mode 100755 |
1831 | index 0000000..1424bb0 |
1832 | --- /dev/null |
1833 | +++ b/tools/net-convert.py |
1834 | @@ -0,0 +1,83 @@ |
1835 | +#!/usr/bin/python3 |
1836 | +# |
1837 | +# This file is part of cloud-init. See LICENSE file ... |
1838 | + |
1839 | +import argparse |
1840 | +import json |
1841 | +import os |
1842 | +import yaml |
1843 | + |
1844 | +from cloudinit.sources.helpers import openstack |
1845 | + |
1846 | +from cloudinit.net import eni |
1847 | +from cloudinit.net import network_state |
1848 | +from cloudinit.net import netplan |
1849 | +from cloudinit.net import sysconfig |
1850 | + |
1851 | + |
1852 | +def main(): |
1853 | + parser = argparse.ArgumentParser() |
1854 | + parser.add_argument("--network-data", "-p", type=open, |
1855 | + metavar="PATH", required=True) |
1856 | + parser.add_argument("--kind", "-k", |
1857 | + choices=['eni', 'network_data.json', 'yaml'], |
1858 | + required=True) |
1859 | + parser.add_argument("-d", "--directory", |
1860 | + metavar="PATH", |
1861 | + help="directory to place output in", |
1862 | + required=True) |
1863 | + parser.add_argument("-m", "--mac", |
1864 | + metavar="name,mac", |
1865 | + action='append', |
1866 | + help="interface name to mac mapping") |
1867 | + parser.add_argument("--output-kind", "-ok", |
1868 | + choices=['eni', 'netplan', 'sysconfig'], |
1869 | + required=True) |
1870 | + args = parser.parse_args() |
1871 | + |
1872 | + if not os.path.isdir(args.directory): |
1873 | + os.makedirs(args.directory) |
1874 | + |
1875 | + if args.mac: |
1876 | + known_macs = {} |
1877 | + for item in args.mac: |
1878 | + iface_name, iface_mac = item.split(",", 1) |
1879 | + known_macs[iface_mac] = iface_name |
1880 | + else: |
1881 | + known_macs = None |
1882 | + |
1883 | + net_data = args.network_data.read() |
1884 | + if args.kind == "eni": |
1885 | + pre_ns = eni.convert_eni_data(net_data) |
1886 | + ns = network_state.parse_net_config_data(pre_ns) |
1887 | + elif args.kind == "yaml": |
1888 | + pre_ns = yaml.load(net_data) |
1889 | + if 'network' in pre_ns: |
1890 | + pre_ns = pre_ns.get('network') |
1891 | + print("Input YAML") |
1892 | + print(yaml.dump(pre_ns, default_flow_style=False, indent=4)) |
1893 | + ns = network_state.parse_net_config_data(pre_ns) |
1894 | + else: |
1895 | + pre_ns = openstack.convert_net_json( |
1896 | + json.loads(net_data), known_macs=known_macs) |
1897 | + ns = network_state.parse_net_config_data(pre_ns) |
1898 | + |
1899 | + if not ns: |
1900 | + raise RuntimeError("No valid network_state object created from" |
1901 | + "input data") |
1902 | + |
1903 | + print("\nInternal State") |
1904 | + print(yaml.dump(ns, default_flow_style=False, indent=4)) |
1905 | + if args.output_kind == "eni": |
1906 | + r_cls = eni.Renderer |
1907 | + elif args.output_kind == "netplan": |
1908 | + r_cls = netplan.Renderer |
1909 | + else: |
1910 | + r_cls = sysconfig.Renderer |
1911 | + |
1912 | + r = r_cls() |
1913 | + r.render_network_state(ns, target=args.directory) |
1914 | + |
1915 | + |
1916 | +if __name__ == '__main__': |
1917 | + main() |
comments inline.