Merge ~raharper/cloud-init:ubuntu/devel/newupstream-20181214 into cloud-init:ubuntu/devel
- Git
- lp:~raharper/cloud-init
- ubuntu/devel/newupstream-20181214
- Merge into ubuntu/devel
Proposed by
Ryan Harper
Status: | Merged | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Merged at revision: | 47a4d44a244aefb55e2a192bd069ae830c6d176e | ||||||||||||
Proposed branch: | ~raharper/cloud-init:ubuntu/devel/newupstream-20181214 | ||||||||||||
Merge into: | cloud-init:ubuntu/devel | ||||||||||||
Diff against target: |
1245 lines (+556/-145) 23 files modified
ChangeLog (+54/-0) bash_completion/cloud-init (+4/-1) cloudinit/cmd/devel/net_convert.py (+10/-5) cloudinit/cmd/main.py (+4/-16) cloudinit/config/cc_write_files.py (+6/-1) cloudinit/dhclient_hook.py (+72/-38) cloudinit/net/eni.py (+15/-14) cloudinit/net/netplan.py (+3/-3) cloudinit/net/sysconfig.py (+21/-4) cloudinit/sources/DataSourceAzure.py (+2/-2) cloudinit/sources/DataSourceNoCloud.py (+31/-1) cloudinit/sources/helpers/vmware/imc/config_nic.py (+2/-3) cloudinit/tests/test_dhclient_hook.py (+105/-0) cloudinit/version.py (+1/-1) config/cloud.cfg.tmpl (+11/-1) debian/changelog (+16/-0) tests/cloud_tests/releases.yaml (+16/-0) tests/unittests/test_cli.py (+8/-8) tests/unittests/test_datasource/test_nocloud.py (+66/-34) tests/unittests/test_handler/test_handler_write_files.py (+12/-0) tests/unittests/test_net.py (+44/-6) tests/unittests/test_vmware_config_file.py (+52/-6) tox.ini (+1/-1) |
||||||||||||
Related bugs: |
|
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Server Team CI bot | continuous-integration | Approve | |
cloud-init Commiters | Pending | ||
Review via email: mp+360957@code.launchpad.net |
Commit message
cloud-init (18.5-1-
* New upstream snapshot.
- Update to pylint 2.2.2.
- Release 18.5 (LP: #1808380)
- tests: add Disco release [Joshua Powers]
- net: render 'metric' values in per-subnet routes (LP: #1805871)
- write_files: add support for appending to files. [James Baxter]
- config: On ubuntu select cloud archive mirrors for armel, armhf, arm64.
(LP: #1805854)
- dhclient-hook: cleanups, tests and fix a bug on 'down' event.
- NoCloud: Allow top level 'network' key in network-config. (LP: #1798117)
- ovf: Fix ovf network config generation gateway/routes (LP: #1806103)
-- Ryan Harper <email address hidden> Fri, 14 Dec 2018 14:45:46 -0600
Description of the change
To post a comment you must log in.
Revision history for this message
Server Team CI bot (server-team-bot) wrote : | # |
review:
Approve
(continuous-integration)
Preview Diff
[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1 | diff --git a/ChangeLog b/ChangeLog |
2 | index 9c043b0..8fa6fdd 100644 |
3 | --- a/ChangeLog |
4 | +++ b/ChangeLog |
5 | @@ -1,3 +1,57 @@ |
6 | +18.5: |
7 | + - tests: add Disco release [Joshua Powers] |
8 | + - net: render 'metric' values in per-subnet routes (LP: #1805871) |
9 | + - write_files: add support for appending to files. [James Baxter] |
10 | + - config: On ubuntu select cloud archive mirrors for armel, armhf, arm64. |
11 | + (LP: #1805854) |
12 | + - dhclient-hook: cleanups, tests and fix a bug on 'down' event. |
13 | + - NoCloud: Allow top level 'network' key in network-config. (LP: #1798117) |
14 | + - ovf: Fix ovf network config generation gateway/routes (LP: #1806103) |
15 | + - azure: detect vnet migration via netlink media change event |
16 | + [Tamilmani Manoharan] |
17 | + - Azure: fix copy/paste error in error handling when reading azure ovf. |
18 | + [Adam DePue] |
19 | + - tests: fix incorrect order of mocks in test_handle_zfs_root. |
20 | + - doc: Change dns_nameserver property to dns_nameservers. [Tomer Cohen] |
21 | + - OVF: identify label iso9660 filesystems with label 'OVF ENV'. |
22 | + - logs: collect-logs ignore instance-data-sensitive.json on non-root user |
23 | + (LP: #1805201) |
24 | + - net: Ephemeral*Network: add connectivity check via URL |
25 | + - azure: _poll_imds only retry on 404. Fail on Timeout (LP: #1803598) |
26 | + - resizefs: Prefix discovered devpath with '/dev/' when path does not |
27 | + exist [Igor Galić] |
28 | + - azure: retry imds polling on requests.Timeout (LP: #1800223) |
29 | + - azure: Accept variation in error msg from mount for ntfs volumes |
30 | + [Jason Zions] (LP: #1799338) |
31 | + - azure: fix regression introduced when persisting ephemeral dhcp lease |
32 | + [asakkurr] |
33 | + - azure: add udev rules to create cloud-init Gen2 disk name symlinks |
34 | + (LP: #1797480) |
35 | + - tests: ec2 mock missing httpretty user-data and instance-identity routes |
36 | + - azure: remove /etc/netplan/90-hotplug-azure.yaml when net from IMDS |
37 | + - azure: report ready to fabric after reprovision and reduce logging |
38 | + [asakkurr] (LP: #1799594) |
39 | + - query: better error when missing read permission on instance-data |
40 | + - instance-data: fallback to instance-data.json if sensitive is absent. |
41 | + (LP: #1798189) |
42 | + - docs: remove colon from network v1 config example. [Tomer Cohen] |
43 | + - Add cloud-id binary to packages for SUSE [Jason Zions] |
44 | + - systemd: On SUSE ensure cloud-init.service runs before wicked |
45 | + [Robert Schweikert] (LP: #1799709) |
46 | + - update detection of openSUSE variants [Robert Schweikert] |
47 | + - azure: Add apply_network_config option to disable network from IMDS |
48 | + (LP: #1798424) |
49 | + - Correct spelling in an error message (udevadm). [Katie McLaughlin] |
50 | + - tests: meta_data key changed to meta-data in ec2 instance-data.json |
51 | + (LP: #1797231) |
52 | + - tests: fix kvm integration test to assert flexible config-disk path |
53 | + (LP: #1797199) |
54 | + - tools: Add cloud-id command line utility |
55 | + - instance-data: Add standard keys platform and subplatform. Refactor ec2. |
56 | + - net: ignore nics that have "zero" mac address. (LP: #1796917) |
57 | + - tests: fix apt_configure_primary to be more flexible |
58 | + - Ubuntu: update sources.list to comment out deb-src entries. (LP: #74747) |
59 | + |
60 | 18.4: |
61 | - add rtd example docs about new standardized keys |
62 | - use ds._crawled_metadata instance attribute if set when writing |
63 | diff --git a/bash_completion/cloud-init b/bash_completion/cloud-init |
64 | index 8c25032..a9577e9 100644 |
65 | --- a/bash_completion/cloud-init |
66 | +++ b/bash_completion/cloud-init |
67 | @@ -30,7 +30,10 @@ _cloudinit_complete() |
68 | devel) |
69 | COMPREPLY=($(compgen -W "--help schema net-convert" -- $cur_word)) |
70 | ;; |
71 | - dhclient-hook|features) |
72 | + dhclient-hook) |
73 | + COMPREPLY=($(compgen -W "--help up down" -- $cur_word)) |
74 | + ;; |
75 | + features) |
76 | COMPREPLY=($(compgen -W "--help" -- $cur_word)) |
77 | ;; |
78 | init) |
79 | diff --git a/cloudinit/cmd/devel/net_convert.py b/cloudinit/cmd/devel/net_convert.py |
80 | index a0f58a0..1ad7e0b 100755 |
81 | --- a/cloudinit/cmd/devel/net_convert.py |
82 | +++ b/cloudinit/cmd/devel/net_convert.py |
83 | @@ -9,6 +9,7 @@ import yaml |
84 | |
85 | from cloudinit.sources.helpers import openstack |
86 | from cloudinit.sources import DataSourceAzure as azure |
87 | +from cloudinit.sources import DataSourceOVF as ovf |
88 | |
89 | from cloudinit import distros |
90 | from cloudinit.net import eni, netplan, network_state, sysconfig |
91 | @@ -31,7 +32,7 @@ def get_parser(parser=None): |
92 | metavar="PATH", required=True) |
93 | parser.add_argument("-k", "--kind", |
94 | choices=['eni', 'network_data.json', 'yaml', |
95 | - 'azure-imds'], |
96 | + 'azure-imds', 'vmware-imc'], |
97 | required=True) |
98 | parser.add_argument("-d", "--directory", |
99 | metavar="PATH", |
100 | @@ -76,7 +77,6 @@ def handle_args(name, args): |
101 | net_data = args.network_data.read() |
102 | if args.kind == "eni": |
103 | pre_ns = eni.convert_eni_data(net_data) |
104 | - ns = network_state.parse_net_config_data(pre_ns) |
105 | elif args.kind == "yaml": |
106 | pre_ns = yaml.load(net_data) |
107 | if 'network' in pre_ns: |
108 | @@ -85,15 +85,16 @@ def handle_args(name, args): |
109 | sys.stderr.write('\n'.join( |
110 | ["Input YAML", |
111 | yaml.dump(pre_ns, default_flow_style=False, indent=4), ""])) |
112 | - ns = network_state.parse_net_config_data(pre_ns) |
113 | elif args.kind == 'network_data.json': |
114 | pre_ns = openstack.convert_net_json( |
115 | json.loads(net_data), known_macs=known_macs) |
116 | - ns = network_state.parse_net_config_data(pre_ns) |
117 | elif args.kind == 'azure-imds': |
118 | pre_ns = azure.parse_network_config(json.loads(net_data)) |
119 | - ns = network_state.parse_net_config_data(pre_ns) |
120 | + elif args.kind == 'vmware-imc': |
121 | + config = ovf.Config(ovf.ConfigFile(args.network_data.name)) |
122 | + pre_ns = ovf.get_network_config_from_conf(config, False) |
123 | |
124 | + ns = network_state.parse_net_config_data(pre_ns) |
125 | if not ns: |
126 | raise RuntimeError("No valid network_state object created from" |
127 | "input data") |
128 | @@ -111,6 +112,10 @@ def handle_args(name, args): |
129 | elif args.output_kind == "netplan": |
130 | r_cls = netplan.Renderer |
131 | config = distro.renderer_configs.get('netplan') |
132 | + # don't run netplan generate/apply |
133 | + config['postcmds'] = False |
134 | + # trim leading slash |
135 | + config['netplan_path'] = config['netplan_path'][1:] |
136 | else: |
137 | r_cls = sysconfig.Renderer |
138 | config = distro.renderer_configs.get('sysconfig') |
139 | diff --git a/cloudinit/cmd/main.py b/cloudinit/cmd/main.py |
140 | index 5a43702..933c019 100644 |
141 | --- a/cloudinit/cmd/main.py |
142 | +++ b/cloudinit/cmd/main.py |
143 | @@ -41,7 +41,7 @@ from cloudinit.settings import (PER_INSTANCE, PER_ALWAYS, PER_ONCE, |
144 | from cloudinit import atomic_helper |
145 | |
146 | from cloudinit.config import cc_set_hostname |
147 | -from cloudinit.dhclient_hook import LogDhclient |
148 | +from cloudinit import dhclient_hook |
149 | |
150 | |
151 | # Welcome message template |
152 | @@ -586,12 +586,6 @@ def main_single(name, args): |
153 | return 0 |
154 | |
155 | |
156 | -def dhclient_hook(name, args): |
157 | - record = LogDhclient(args) |
158 | - record.check_hooks_dir() |
159 | - record.record() |
160 | - |
161 | - |
162 | def status_wrapper(name, args, data_d=None, link_d=None): |
163 | if data_d is None: |
164 | data_d = os.path.normpath("/var/lib/cloud/data") |
165 | @@ -795,15 +789,9 @@ def main(sysv_args=None): |
166 | 'query', |
167 | help='Query standardized instance metadata from the command line.') |
168 | |
169 | - parser_dhclient = subparsers.add_parser('dhclient-hook', |
170 | - help=('run the dhclient hook' |
171 | - 'to record network info')) |
172 | - parser_dhclient.add_argument("net_action", |
173 | - help=('action taken on the interface')) |
174 | - parser_dhclient.add_argument("net_interface", |
175 | - help=('the network interface being acted' |
176 | - ' upon')) |
177 | - parser_dhclient.set_defaults(action=('dhclient_hook', dhclient_hook)) |
178 | + parser_dhclient = subparsers.add_parser( |
179 | + dhclient_hook.NAME, help=dhclient_hook.__doc__) |
180 | + dhclient_hook.get_parser(parser_dhclient) |
181 | |
182 | parser_features = subparsers.add_parser('features', |
183 | help=('list defined features')) |
184 | diff --git a/cloudinit/config/cc_write_files.py b/cloudinit/config/cc_write_files.py |
185 | index 31d1db6..0b6546e 100644 |
186 | --- a/cloudinit/config/cc_write_files.py |
187 | +++ b/cloudinit/config/cc_write_files.py |
188 | @@ -49,6 +49,10 @@ binary gzip data can be specified and will be decoded before being written. |
189 | ... |
190 | path: /bin/arch |
191 | permissions: '0555' |
192 | + - content: | |
193 | + 15 * * * * root ship_logs |
194 | + path: /etc/crontab |
195 | + append: true |
196 | """ |
197 | |
198 | import base64 |
199 | @@ -113,7 +117,8 @@ def write_files(name, files): |
200 | contents = extract_contents(f_info.get('content', ''), extractions) |
201 | (u, g) = util.extract_usergroup(f_info.get('owner', DEFAULT_OWNER)) |
202 | perms = decode_perms(f_info.get('permissions'), DEFAULT_PERMS) |
203 | - util.write_file(path, contents, mode=perms) |
204 | + omode = 'ab' if util.get_cfg_option_bool(f_info, 'append') else 'wb' |
205 | + util.write_file(path, contents, omode=omode, mode=perms) |
206 | util.chownbyname(path, u, g) |
207 | |
208 | |
209 | diff --git a/cloudinit/dhclient_hook.py b/cloudinit/dhclient_hook.py |
210 | index 7f02d7f..72b51b6 100644 |
211 | --- a/cloudinit/dhclient_hook.py |
212 | +++ b/cloudinit/dhclient_hook.py |
213 | @@ -1,5 +1,8 @@ |
214 | # This file is part of cloud-init. See LICENSE file for license information. |
215 | |
216 | +"""Run the dhclient hook to record network info.""" |
217 | + |
218 | +import argparse |
219 | import os |
220 | |
221 | from cloudinit import atomic_helper |
222 | @@ -8,44 +11,75 @@ from cloudinit import stages |
223 | |
224 | LOG = logging.getLogger(__name__) |
225 | |
226 | +NAME = "dhclient-hook" |
227 | +UP = "up" |
228 | +DOWN = "down" |
229 | +EVENTS = (UP, DOWN) |
230 | + |
231 | + |
232 | +def _get_hooks_dir(): |
233 | + i = stages.Init() |
234 | + return os.path.join(i.paths.get_runpath(), 'dhclient.hooks') |
235 | + |
236 | + |
237 | +def _filter_env_vals(info): |
238 | + """Given info (os.environ), return a dictionary with |
239 | + lower case keys for each entry starting with DHCP4_ or new_.""" |
240 | + new_info = {} |
241 | + for k, v in info.items(): |
242 | + if k.startswith("DHCP4_") or k.startswith("new_"): |
243 | + key = (k.replace('DHCP4_', '').replace('new_', '')).lower() |
244 | + new_info[key] = v |
245 | + return new_info |
246 | + |
247 | + |
248 | +def run_hook(interface, event, data_d=None, env=None): |
249 | + if event not in EVENTS: |
250 | + raise ValueError("Unexpected event '%s'. Expected one of: %s" % |
251 | + (event, EVENTS)) |
252 | + if data_d is None: |
253 | + data_d = _get_hooks_dir() |
254 | + if env is None: |
255 | + env = os.environ |
256 | + hook_file = os.path.join(data_d, interface + ".json") |
257 | + |
258 | + if event == UP: |
259 | + if not os.path.exists(data_d): |
260 | + os.makedirs(data_d) |
261 | + atomic_helper.write_json(hook_file, _filter_env_vals(env)) |
262 | + LOG.debug("Wrote dhclient options in %s", hook_file) |
263 | + elif event == DOWN: |
264 | + if os.path.exists(hook_file): |
265 | + os.remove(hook_file) |
266 | + LOG.debug("Removed dhclient options file %s", hook_file) |
267 | + |
268 | + |
269 | +def get_parser(parser=None): |
270 | + if parser is None: |
271 | + parser = argparse.ArgumentParser(prog=NAME, description=__doc__) |
272 | + parser.add_argument( |
273 | + "event", help='event taken on the interface', choices=EVENTS) |
274 | + parser.add_argument( |
275 | + "interface", help='the network interface being acted upon') |
276 | + # cloud-init main uses 'action' |
277 | + parser.set_defaults(action=(NAME, handle_args)) |
278 | + return parser |
279 | + |
280 | + |
281 | +def handle_args(name, args, data_d=None): |
282 | + """Handle the Namespace args. |
283 | + Takes 'name' as passed by cloud-init main. not used here.""" |
284 | + return run_hook(interface=args.interface, event=args.event, data_d=data_d) |
285 | + |
286 | + |
287 | +if __name__ == '__main__': |
288 | + import sys |
289 | + parser = get_parser() |
290 | + args = parser.parse_args(args=sys.argv[1:]) |
291 | + return_value = handle_args( |
292 | + NAME, args, data_d=os.environ.get('_CI_DHCP_HOOK_DATA_D')) |
293 | + if return_value: |
294 | + sys.exit(return_value) |
295 | |
296 | -class LogDhclient(object): |
297 | - |
298 | - def __init__(self, cli_args): |
299 | - self.hooks_dir = self._get_hooks_dir() |
300 | - self.net_interface = cli_args.net_interface |
301 | - self.net_action = cli_args.net_action |
302 | - self.hook_file = os.path.join(self.hooks_dir, |
303 | - self.net_interface + ".json") |
304 | - |
305 | - @staticmethod |
306 | - def _get_hooks_dir(): |
307 | - i = stages.Init() |
308 | - return os.path.join(i.paths.get_runpath(), 'dhclient.hooks') |
309 | - |
310 | - def check_hooks_dir(self): |
311 | - if not os.path.exists(self.hooks_dir): |
312 | - os.makedirs(self.hooks_dir) |
313 | - else: |
314 | - # If the action is down and the json file exists, we need to |
315 | - # delete the file |
316 | - if self.net_action is 'down' and os.path.exists(self.hook_file): |
317 | - os.remove(self.hook_file) |
318 | - |
319 | - @staticmethod |
320 | - def get_vals(info): |
321 | - new_info = {} |
322 | - for k, v in info.items(): |
323 | - if k.startswith("DHCP4_") or k.startswith("new_"): |
324 | - key = (k.replace('DHCP4_', '').replace('new_', '')).lower() |
325 | - new_info[key] = v |
326 | - return new_info |
327 | - |
328 | - def record(self): |
329 | - envs = os.environ |
330 | - if self.hook_file is None: |
331 | - return |
332 | - atomic_helper.write_json(self.hook_file, self.get_vals(envs)) |
333 | - LOG.debug("Wrote dhclient options in %s", self.hook_file) |
334 | |
335 | # vi: ts=4 expandtab |
336 | diff --git a/cloudinit/net/eni.py b/cloudinit/net/eni.py |
337 | index c6f631a..6423632 100644 |
338 | --- a/cloudinit/net/eni.py |
339 | +++ b/cloudinit/net/eni.py |
340 | @@ -371,22 +371,23 @@ class Renderer(renderer.Renderer): |
341 | 'gateway': 'gw', |
342 | 'metric': 'metric', |
343 | } |
344 | + |
345 | + default_gw = '' |
346 | if route['network'] == '0.0.0.0' and route['netmask'] == '0.0.0.0': |
347 | - default_gw = " default gw %s" % route['gateway'] |
348 | - content.append(up + default_gw + or_true) |
349 | - content.append(down + default_gw + or_true) |
350 | + default_gw = ' default' |
351 | elif route['network'] == '::' and route['prefix'] == 0: |
352 | - # ipv6! |
353 | - default_gw = " -A inet6 default gw %s" % route['gateway'] |
354 | - content.append(up + default_gw + or_true) |
355 | - content.append(down + default_gw + or_true) |
356 | - else: |
357 | - route_line = "" |
358 | - for k in ['network', 'netmask', 'gateway', 'metric']: |
359 | - if k in route: |
360 | - route_line += " %s %s" % (mapping[k], route[k]) |
361 | - content.append(up + route_line + or_true) |
362 | - content.append(down + route_line + or_true) |
363 | + default_gw = ' -A inet6 default' |
364 | + |
365 | + route_line = '' |
366 | + for k in ['network', 'netmask', 'gateway', 'metric']: |
367 | + if default_gw and k in ['network', 'netmask']: |
368 | + continue |
369 | + if k == 'gateway': |
370 | + route_line += '%s %s %s' % (default_gw, mapping[k], route[k]) |
371 | + elif k in route: |
372 | + route_line += ' %s %s' % (mapping[k], route[k]) |
373 | + content.append(up + route_line + or_true) |
374 | + content.append(down + route_line + or_true) |
375 | return content |
376 | |
377 | def _render_iface(self, iface, render_hwaddress=False): |
378 | diff --git a/cloudinit/net/netplan.py b/cloudinit/net/netplan.py |
379 | index bc1087f..21517fd 100644 |
380 | --- a/cloudinit/net/netplan.py |
381 | +++ b/cloudinit/net/netplan.py |
382 | @@ -114,13 +114,13 @@ def _extract_addresses(config, entry, ifname): |
383 | for route in subnet.get('routes', []): |
384 | to_net = "%s/%s" % (route.get('network'), |
385 | route.get('prefix')) |
386 | - route = { |
387 | + new_route = { |
388 | 'via': route.get('gateway'), |
389 | 'to': to_net, |
390 | } |
391 | if 'metric' in route: |
392 | - route.update({'metric': route.get('metric', 100)}) |
393 | - routes.append(route) |
394 | + new_route.update({'metric': route.get('metric', 100)}) |
395 | + routes.append(new_route) |
396 | |
397 | addresses.append(addr) |
398 | |
399 | diff --git a/cloudinit/net/sysconfig.py b/cloudinit/net/sysconfig.py |
400 | index 9c16d3a..17293e1 100644 |
401 | --- a/cloudinit/net/sysconfig.py |
402 | +++ b/cloudinit/net/sysconfig.py |
403 | @@ -156,13 +156,23 @@ class Route(ConfigMap): |
404 | _quote_value(gateway_value))) |
405 | buf.write("%s=%s\n" % ('NETMASK' + str(reindex), |
406 | _quote_value(netmask_value))) |
407 | + metric_key = 'METRIC' + index |
408 | + if metric_key in self._conf: |
409 | + metric_value = str(self._conf['METRIC' + index]) |
410 | + buf.write("%s=%s\n" % ('METRIC' + str(reindex), |
411 | + _quote_value(metric_value))) |
412 | elif proto == "ipv6" and self.is_ipv6_route(address_value): |
413 | netmask_value = str(self._conf['NETMASK' + index]) |
414 | gateway_value = str(self._conf['GATEWAY' + index]) |
415 | - buf.write("%s/%s via %s dev %s\n" % (address_value, |
416 | - netmask_value, |
417 | - gateway_value, |
418 | - self._route_name)) |
419 | + metric_value = ( |
420 | + 'metric ' + str(self._conf['METRIC' + index]) |
421 | + if 'METRIC' + index in self._conf else '') |
422 | + buf.write( |
423 | + "%s/%s via %s %s dev %s\n" % (address_value, |
424 | + netmask_value, |
425 | + gateway_value, |
426 | + metric_value, |
427 | + self._route_name)) |
428 | |
429 | return buf.getvalue() |
430 | |
431 | @@ -370,6 +380,9 @@ class Renderer(renderer.Renderer): |
432 | else: |
433 | iface_cfg['GATEWAY'] = subnet['gateway'] |
434 | |
435 | + if 'metric' in subnet: |
436 | + iface_cfg['METRIC'] = subnet['metric'] |
437 | + |
438 | if 'dns_search' in subnet: |
439 | iface_cfg['DOMAIN'] = ' '.join(subnet['dns_search']) |
440 | |
441 | @@ -414,15 +427,19 @@ class Renderer(renderer.Renderer): |
442 | else: |
443 | iface_cfg['GATEWAY'] = route['gateway'] |
444 | route_cfg.has_set_default_ipv4 = True |
445 | + if 'metric' in route: |
446 | + iface_cfg['METRIC'] = route['metric'] |
447 | |
448 | else: |
449 | gw_key = 'GATEWAY%s' % route_cfg.last_idx |
450 | nm_key = 'NETMASK%s' % route_cfg.last_idx |
451 | addr_key = 'ADDRESS%s' % route_cfg.last_idx |
452 | + metric_key = 'METRIC%s' % route_cfg.last_idx |
453 | route_cfg.last_idx += 1 |
454 | # add default routes only to ifcfg files, not |
455 | # to route-* or route6-* |
456 | for (old_key, new_key) in [('gateway', gw_key), |
457 | + ('metric', metric_key), |
458 | ('netmask', nm_key), |
459 | ('network', addr_key)]: |
460 | if old_key in route: |
461 | diff --git a/cloudinit/sources/DataSourceAzure.py b/cloudinit/sources/DataSourceAzure.py |
462 | index e076d5d..46efca4 100644 |
463 | --- a/cloudinit/sources/DataSourceAzure.py |
464 | +++ b/cloudinit/sources/DataSourceAzure.py |
465 | @@ -980,8 +980,8 @@ def read_azure_ovf(contents): |
466 | raise NonAzureDataSource("No LinuxProvisioningConfigurationSet") |
467 | if len(lpcs_nodes) > 1: |
468 | raise BrokenAzureDataSource("found '%d' %ss" % |
469 | - ("LinuxProvisioningConfigurationSet", |
470 | - len(lpcs_nodes))) |
471 | + (len(lpcs_nodes), |
472 | + "LinuxProvisioningConfigurationSet")) |
473 | lpcs = lpcs_nodes[0] |
474 | |
475 | if not lpcs.hasChildNodes(): |
476 | diff --git a/cloudinit/sources/DataSourceNoCloud.py b/cloudinit/sources/DataSourceNoCloud.py |
477 | index 9010f06..6860f0c 100644 |
478 | --- a/cloudinit/sources/DataSourceNoCloud.py |
479 | +++ b/cloudinit/sources/DataSourceNoCloud.py |
480 | @@ -311,6 +311,35 @@ def parse_cmdline_data(ds_id, fill, cmdline=None): |
481 | return True |
482 | |
483 | |
484 | +def _maybe_remove_top_network(cfg): |
485 | + """If network-config contains top level 'network' key, then remove it. |
486 | + |
487 | + Some providers of network configuration may provide a top level |
488 | + 'network' key (LP: #1798117) even though it is not necessary. |
489 | + |
490 | + Be friendly and remove it if it really seems so. |
491 | + |
492 | + Return the original value if no change or the updated value if changed.""" |
493 | + nullval = object() |
494 | + network_val = cfg.get('network', nullval) |
495 | + if network_val is nullval: |
496 | + return cfg |
497 | + bmsg = 'Top level network key in network-config %s: %s' |
498 | + if not isinstance(network_val, dict): |
499 | + LOG.debug(bmsg, "was not a dict", cfg) |
500 | + return cfg |
501 | + if len(list(cfg.keys())) != 1: |
502 | + LOG.debug(bmsg, "had multiple top level keys", cfg) |
503 | + return cfg |
504 | + if network_val.get('config') == "disabled": |
505 | + LOG.debug(bmsg, "was config/disabled", cfg) |
506 | + elif not all(('config' in network_val, 'version' in network_val)): |
507 | + LOG.debug(bmsg, "but missing 'config' or 'version'", cfg) |
508 | + return cfg |
509 | + LOG.debug(bmsg, "fixed by removing shifting network.", cfg) |
510 | + return network_val |
511 | + |
512 | + |
513 | def _merge_new_seed(cur, seeded): |
514 | ret = cur.copy() |
515 | |
516 | @@ -320,7 +349,8 @@ def _merge_new_seed(cur, seeded): |
517 | ret['meta-data'] = util.mergemanydict([cur['meta-data'], newmd]) |
518 | |
519 | if seeded.get('network-config'): |
520 | - ret['network-config'] = util.load_yaml(seeded['network-config']) |
521 | + ret['network-config'] = _maybe_remove_top_network( |
522 | + util.load_yaml(seeded.get('network-config'))) |
523 | |
524 | if 'user-data' in seeded: |
525 | ret['user-data'] = seeded['user-data'] |
526 | diff --git a/cloudinit/sources/helpers/vmware/imc/config_nic.py b/cloudinit/sources/helpers/vmware/imc/config_nic.py |
527 | index e1890e2..77cbf3b 100644 |
528 | --- a/cloudinit/sources/helpers/vmware/imc/config_nic.py |
529 | +++ b/cloudinit/sources/helpers/vmware/imc/config_nic.py |
530 | @@ -165,9 +165,8 @@ class NicConfigurator(object): |
531 | |
532 | # Add routes if there is no primary nic |
533 | if not self._primaryNic and v4.gateways: |
534 | - route_list.extend(self.gen_ipv4_route(nic, |
535 | - v4.gateways, |
536 | - v4.netmask)) |
537 | + subnet.update( |
538 | + {'routes': self.gen_ipv4_route(nic, v4.gateways, v4.netmask)}) |
539 | |
540 | return ([subnet], route_list) |
541 | |
542 | diff --git a/cloudinit/tests/test_dhclient_hook.py b/cloudinit/tests/test_dhclient_hook.py |
543 | new file mode 100644 |
544 | index 0000000..7aab8dd |
545 | --- /dev/null |
546 | +++ b/cloudinit/tests/test_dhclient_hook.py |
547 | @@ -0,0 +1,105 @@ |
548 | +# This file is part of cloud-init. See LICENSE file for license information. |
549 | + |
550 | +"""Tests for cloudinit.dhclient_hook.""" |
551 | + |
552 | +from cloudinit import dhclient_hook as dhc |
553 | +from cloudinit.tests.helpers import CiTestCase, dir2dict, populate_dir |
554 | + |
555 | +import argparse |
556 | +import json |
557 | +import mock |
558 | +import os |
559 | + |
560 | + |
561 | +class TestDhclientHook(CiTestCase): |
562 | + |
563 | + ex_env = { |
564 | + 'interface': 'eth0', |
565 | + 'new_dhcp_lease_time': '3600', |
566 | + 'new_host_name': 'x1', |
567 | + 'new_ip_address': '10.145.210.163', |
568 | + 'new_subnet_mask': '255.255.255.0', |
569 | + 'old_host_name': 'x1', |
570 | + 'PATH': '/usr/sbin:/usr/bin:/sbin:/bin', |
571 | + 'pid': '614', |
572 | + 'reason': 'BOUND', |
573 | + } |
574 | + |
575 | + # some older versions of dhclient put the same content, |
576 | + # but in upper case with DHCP4_ instead of new_ |
577 | + ex_env_dhcp4 = { |
578 | + 'REASON': 'BOUND', |
579 | + 'DHCP4_dhcp_lease_time': '3600', |
580 | + 'DHCP4_host_name': 'x1', |
581 | + 'DHCP4_ip_address': '10.145.210.163', |
582 | + 'DHCP4_subnet_mask': '255.255.255.0', |
583 | + 'INTERFACE': 'eth0', |
584 | + 'PATH': '/usr/sbin:/usr/bin:/sbin:/bin', |
585 | + 'pid': '614', |
586 | + } |
587 | + |
588 | + expected = { |
589 | + 'dhcp_lease_time': '3600', |
590 | + 'host_name': 'x1', |
591 | + 'ip_address': '10.145.210.163', |
592 | + 'subnet_mask': '255.255.255.0'} |
593 | + |
594 | + def setUp(self): |
595 | + super(TestDhclientHook, self).setUp() |
596 | + self.tmp = self.tmp_dir() |
597 | + |
598 | + def test_handle_args(self): |
599 | + """quick test of call to handle_args.""" |
600 | + nic = 'eth0' |
601 | + args = argparse.Namespace(event=dhc.UP, interface=nic) |
602 | + with mock.patch.dict("os.environ", clear=True, values=self.ex_env): |
603 | + dhc.handle_args(dhc.NAME, args, data_d=self.tmp) |
604 | + found = dir2dict(self.tmp + os.path.sep) |
605 | + self.assertEqual([nic + ".json"], list(found.keys())) |
606 | + self.assertEqual(self.expected, json.loads(found[nic + ".json"])) |
607 | + |
608 | + def test_run_hook_up_creates_dir(self): |
609 | + """If dir does not exist, run_hook should create it.""" |
610 | + subd = self.tmp_path("subdir", self.tmp) |
611 | + nic = 'eth1' |
612 | + dhc.run_hook(nic, 'up', data_d=subd, env=self.ex_env) |
613 | + self.assertEqual( |
614 | + set([nic + ".json"]), set(dir2dict(subd + os.path.sep))) |
615 | + |
616 | + def test_run_hook_up(self): |
617 | + """Test expected use of run_hook_up.""" |
618 | + nic = 'eth0' |
619 | + dhc.run_hook(nic, 'up', data_d=self.tmp, env=self.ex_env) |
620 | + found = dir2dict(self.tmp + os.path.sep) |
621 | + self.assertEqual([nic + ".json"], list(found.keys())) |
622 | + self.assertEqual(self.expected, json.loads(found[nic + ".json"])) |
623 | + |
624 | + def test_run_hook_up_dhcp4_prefix(self): |
625 | + """Test run_hook filters correctly with older DHCP4_ data.""" |
626 | + nic = 'eth0' |
627 | + dhc.run_hook(nic, 'up', data_d=self.tmp, env=self.ex_env_dhcp4) |
628 | + found = dir2dict(self.tmp + os.path.sep) |
629 | + self.assertEqual([nic + ".json"], list(found.keys())) |
630 | + self.assertEqual(self.expected, json.loads(found[nic + ".json"])) |
631 | + |
632 | + def test_run_hook_down_deletes(self): |
633 | + """down should delete the created json file.""" |
634 | + nic = 'eth1' |
635 | + populate_dir( |
636 | + self.tmp, {nic + ".json": "{'abcd'}", 'myfile.txt': 'text'}) |
637 | + dhc.run_hook(nic, 'down', data_d=self.tmp, env={'old_host_name': 'x1'}) |
638 | + self.assertEqual( |
639 | + set(['myfile.txt']), |
640 | + set(dir2dict(self.tmp + os.path.sep))) |
641 | + |
642 | + def test_get_parser(self): |
643 | + """Smoke test creation of get_parser.""" |
644 | + # cloud-init main uses 'action'. |
645 | + event, interface = (dhc.UP, 'mynic0') |
646 | + self.assertEqual( |
647 | + argparse.Namespace(event=event, interface=interface, |
648 | + action=(dhc.NAME, dhc.handle_args)), |
649 | + dhc.get_parser().parse_args([event, interface])) |
650 | + |
651 | + |
652 | +# vi: ts=4 expandtab |
653 | diff --git a/cloudinit/version.py b/cloudinit/version.py |
654 | index 844a02e..a2c5d43 100644 |
655 | --- a/cloudinit/version.py |
656 | +++ b/cloudinit/version.py |
657 | @@ -4,7 +4,7 @@ |
658 | # |
659 | # This file is part of cloud-init. See LICENSE file for license information. |
660 | |
661 | -__VERSION__ = "18.4" |
662 | +__VERSION__ = "18.5" |
663 | _PACKAGED_VERSION = '@@PACKAGED_VERSION@@' |
664 | |
665 | FEATURES = [ |
666 | diff --git a/config/cloud.cfg.tmpl b/config/cloud.cfg.tmpl |
667 | index 1fef133..7513176 100644 |
668 | --- a/config/cloud.cfg.tmpl |
669 | +++ b/config/cloud.cfg.tmpl |
670 | @@ -167,7 +167,17 @@ system_info: |
671 | - http://%(availability_zone)s.clouds.archive.ubuntu.com/ubuntu/ |
672 | - http://%(region)s.clouds.archive.ubuntu.com/ubuntu/ |
673 | security: [] |
674 | - - arches: [armhf, armel, default] |
675 | + - arches: [arm64, armel, armhf] |
676 | + failsafe: |
677 | + primary: http://ports.ubuntu.com/ubuntu-ports |
678 | + security: http://ports.ubuntu.com/ubuntu-ports |
679 | + search: |
680 | + primary: |
681 | + - http://%(ec2_region)s.ec2.ports.ubuntu.com/ubuntu-ports/ |
682 | + - http://%(availability_zone)s.clouds.ports.ubuntu.com/ubuntu-ports/ |
683 | + - http://%(region)s.clouds.ports.ubuntu.com/ubuntu-ports/ |
684 | + security: [] |
685 | + - arches: [default] |
686 | failsafe: |
687 | primary: http://ports.ubuntu.com/ubuntu-ports |
688 | security: http://ports.ubuntu.com/ubuntu-ports |
689 | diff --git a/debian/changelog b/debian/changelog |
690 | index 283bcd8..09a0034 100644 |
691 | --- a/debian/changelog |
692 | +++ b/debian/changelog |
693 | @@ -1,3 +1,19 @@ |
694 | +cloud-init (18.5-1-g5b065316-0ubuntu1) disco; urgency=medium |
695 | + |
696 | + * New upstream snapshot. |
697 | + - Update to pylint 2.2.2. |
698 | + - Release 18.5 (LP: #1808380) |
699 | + - tests: add Disco release [Joshua Powers] |
700 | + - net: render 'metric' values in per-subnet routes (LP: #1805871) |
701 | + - write_files: add support for appending to files. [James Baxter] |
702 | + - config: On ubuntu select cloud archive mirrors for armel, armhf, arm64. |
703 | + (LP: #1805854) |
704 | + - dhclient-hook: cleanups, tests and fix a bug on 'down' event. |
705 | + - NoCloud: Allow top level 'network' key in network-config. (LP: #1798117) |
706 | + - ovf: Fix ovf network config generation gateway/routes (LP: #1806103) |
707 | + |
708 | + -- Ryan Harper <ryan.harper@canonical.com> Fri, 14 Dec 2018 14:45:46 -0600 |
709 | + |
710 | cloud-init (18.4-31-gbf791715-0ubuntu1) disco; urgency=medium |
711 | |
712 | * New upstream snapshot. |
713 | diff --git a/tests/cloud_tests/releases.yaml b/tests/cloud_tests/releases.yaml |
714 | index defae02..ec5da72 100644 |
715 | --- a/tests/cloud_tests/releases.yaml |
716 | +++ b/tests/cloud_tests/releases.yaml |
717 | @@ -129,6 +129,22 @@ features: |
718 | |
719 | releases: |
720 | # UBUNTU ================================================================= |
721 | + disco: |
722 | + # EOL: Jan 2020 |
723 | + default: |
724 | + enabled: true |
725 | + release: disco |
726 | + version: 19.04 |
727 | + os: ubuntu |
728 | + feature_groups: |
729 | + - base |
730 | + - debian_base |
731 | + - ubuntu_specific |
732 | + lxd: |
733 | + sstreams_server: https://cloud-images.ubuntu.com/daily |
734 | + alias: disco |
735 | + setup_overrides: null |
736 | + override_templates: false |
737 | cosmic: |
738 | # EOL: Jul 2019 |
739 | default: |
740 | diff --git a/tests/unittests/test_cli.py b/tests/unittests/test_cli.py |
741 | index 199d69b..d283f13 100644 |
742 | --- a/tests/unittests/test_cli.py |
743 | +++ b/tests/unittests/test_cli.py |
744 | @@ -246,18 +246,18 @@ class TestCLI(test_helpers.FilesystemMockingTestCase): |
745 | self.assertEqual('cc_ntp', parseargs.name) |
746 | self.assertFalse(parseargs.report) |
747 | |
748 | - @mock.patch('cloudinit.cmd.main.dhclient_hook') |
749 | - def test_dhclient_hook_subcommand(self, m_dhclient_hook): |
750 | + @mock.patch('cloudinit.cmd.main.dhclient_hook.handle_args') |
751 | + def test_dhclient_hook_subcommand(self, m_handle_args): |
752 | """The subcommand 'dhclient-hook' calls dhclient_hook with args.""" |
753 | - self._call_main(['cloud-init', 'dhclient-hook', 'net_action', 'eth0']) |
754 | - (name, parseargs) = m_dhclient_hook.call_args_list[0][0] |
755 | - self.assertEqual('dhclient_hook', name) |
756 | + self._call_main(['cloud-init', 'dhclient-hook', 'up', 'eth0']) |
757 | + (name, parseargs) = m_handle_args.call_args_list[0][0] |
758 | + self.assertEqual('dhclient-hook', name) |
759 | self.assertEqual('dhclient-hook', parseargs.subcommand) |
760 | - self.assertEqual('dhclient_hook', parseargs.action[0]) |
761 | + self.assertEqual('dhclient-hook', parseargs.action[0]) |
762 | self.assertFalse(parseargs.debug) |
763 | self.assertFalse(parseargs.force) |
764 | - self.assertEqual('net_action', parseargs.net_action) |
765 | - self.assertEqual('eth0', parseargs.net_interface) |
766 | + self.assertEqual('up', parseargs.event) |
767 | + self.assertEqual('eth0', parseargs.interface) |
768 | |
769 | @mock.patch('cloudinit.cmd.main.main_features') |
770 | def test_features_hook_subcommand(self, m_features): |
771 | diff --git a/tests/unittests/test_datasource/test_nocloud.py b/tests/unittests/test_datasource/test_nocloud.py |
772 | index b6468b6..3429272 100644 |
773 | --- a/tests/unittests/test_datasource/test_nocloud.py |
774 | +++ b/tests/unittests/test_datasource/test_nocloud.py |
775 | @@ -1,7 +1,10 @@ |
776 | # This file is part of cloud-init. See LICENSE file for license information. |
777 | |
778 | from cloudinit import helpers |
779 | -from cloudinit.sources import DataSourceNoCloud |
780 | +from cloudinit.sources.DataSourceNoCloud import ( |
781 | + DataSourceNoCloud as dsNoCloud, |
782 | + _maybe_remove_top_network, |
783 | + parse_cmdline_data) |
784 | from cloudinit import util |
785 | from cloudinit.tests.helpers import CiTestCase, populate_dir, mock, ExitStack |
786 | |
787 | @@ -40,9 +43,7 @@ class TestNoCloudDataSource(CiTestCase): |
788 | 'datasource': {'NoCloud': {'fs_label': None}} |
789 | } |
790 | |
791 | - ds = DataSourceNoCloud.DataSourceNoCloud |
792 | - |
793 | - dsrc = ds(sys_cfg=sys_cfg, distro=None, paths=self.paths) |
794 | + dsrc = dsNoCloud(sys_cfg=sys_cfg, distro=None, paths=self.paths) |
795 | ret = dsrc.get_data() |
796 | self.assertEqual(dsrc.userdata_raw, ud) |
797 | self.assertEqual(dsrc.metadata, md) |
798 | @@ -63,9 +64,7 @@ class TestNoCloudDataSource(CiTestCase): |
799 | 'datasource': {'NoCloud': {'fs_label': None}} |
800 | } |
801 | |
802 | - ds = DataSourceNoCloud.DataSourceNoCloud |
803 | - |
804 | - dsrc = ds(sys_cfg=sys_cfg, distro=None, paths=self.paths) |
805 | + dsrc = dsNoCloud(sys_cfg=sys_cfg, distro=None, paths=self.paths) |
806 | self.assertTrue(dsrc.get_data()) |
807 | self.assertEqual(dsrc.platform_type, 'nocloud') |
808 | self.assertEqual( |
809 | @@ -73,8 +72,6 @@ class TestNoCloudDataSource(CiTestCase): |
810 | |
811 | def test_fs_label(self, m_is_lxd): |
812 | # find_devs_with should not be called ff fs_label is None |
813 | - ds = DataSourceNoCloud.DataSourceNoCloud |
814 | - |
815 | class PsuedoException(Exception): |
816 | pass |
817 | |
818 | @@ -84,12 +81,12 @@ class TestNoCloudDataSource(CiTestCase): |
819 | |
820 | # by default, NoCloud should search for filesystems by label |
821 | sys_cfg = {'datasource': {'NoCloud': {}}} |
822 | - dsrc = ds(sys_cfg=sys_cfg, distro=None, paths=self.paths) |
823 | + dsrc = dsNoCloud(sys_cfg=sys_cfg, distro=None, paths=self.paths) |
824 | self.assertRaises(PsuedoException, dsrc.get_data) |
825 | |
826 | # but disabling searching should just end up with None found |
827 | sys_cfg = {'datasource': {'NoCloud': {'fs_label': None}}} |
828 | - dsrc = ds(sys_cfg=sys_cfg, distro=None, paths=self.paths) |
829 | + dsrc = dsNoCloud(sys_cfg=sys_cfg, distro=None, paths=self.paths) |
830 | ret = dsrc.get_data() |
831 | self.assertFalse(ret) |
832 | |
833 | @@ -97,13 +94,10 @@ class TestNoCloudDataSource(CiTestCase): |
834 | # no source should be found if no cmdline, config, and fs_label=None |
835 | sys_cfg = {'datasource': {'NoCloud': {'fs_label': None}}} |
836 | |
837 | - ds = DataSourceNoCloud.DataSourceNoCloud |
838 | - dsrc = ds(sys_cfg=sys_cfg, distro=None, paths=self.paths) |
839 | + dsrc = dsNoCloud(sys_cfg=sys_cfg, distro=None, paths=self.paths) |
840 | self.assertFalse(dsrc.get_data()) |
841 | |
842 | def test_seed_in_config(self, m_is_lxd): |
843 | - ds = DataSourceNoCloud.DataSourceNoCloud |
844 | - |
845 | data = { |
846 | 'fs_label': None, |
847 | 'meta-data': yaml.safe_dump({'instance-id': 'IID'}), |
848 | @@ -111,7 +105,7 @@ class TestNoCloudDataSource(CiTestCase): |
849 | } |
850 | |
851 | sys_cfg = {'datasource': {'NoCloud': data}} |
852 | - dsrc = ds(sys_cfg=sys_cfg, distro=None, paths=self.paths) |
853 | + dsrc = dsNoCloud(sys_cfg=sys_cfg, distro=None, paths=self.paths) |
854 | ret = dsrc.get_data() |
855 | self.assertEqual(dsrc.userdata_raw, b"USER_DATA_RAW") |
856 | self.assertEqual(dsrc.metadata.get('instance-id'), 'IID') |
857 | @@ -130,9 +124,7 @@ class TestNoCloudDataSource(CiTestCase): |
858 | 'datasource': {'NoCloud': {'fs_label': None}} |
859 | } |
860 | |
861 | - ds = DataSourceNoCloud.DataSourceNoCloud |
862 | - |
863 | - dsrc = ds(sys_cfg=sys_cfg, distro=None, paths=self.paths) |
864 | + dsrc = dsNoCloud(sys_cfg=sys_cfg, distro=None, paths=self.paths) |
865 | ret = dsrc.get_data() |
866 | self.assertEqual(dsrc.userdata_raw, ud) |
867 | self.assertEqual(dsrc.metadata, md) |
868 | @@ -145,9 +137,7 @@ class TestNoCloudDataSource(CiTestCase): |
869 | |
870 | sys_cfg = {'datasource': {'NoCloud': {'fs_label': None}}} |
871 | |
872 | - ds = DataSourceNoCloud.DataSourceNoCloud |
873 | - |
874 | - dsrc = ds(sys_cfg=sys_cfg, distro=None, paths=self.paths) |
875 | + dsrc = dsNoCloud(sys_cfg=sys_cfg, distro=None, paths=self.paths) |
876 | ret = dsrc.get_data() |
877 | self.assertEqual(dsrc.userdata_raw, b"ud") |
878 | self.assertFalse(dsrc.vendordata) |
879 | @@ -174,9 +164,7 @@ class TestNoCloudDataSource(CiTestCase): |
880 | |
881 | sys_cfg = {'datasource': {'NoCloud': {'fs_label': None}}} |
882 | |
883 | - ds = DataSourceNoCloud.DataSourceNoCloud |
884 | - |
885 | - dsrc = ds(sys_cfg=sys_cfg, distro=None, paths=self.paths) |
886 | + dsrc = dsNoCloud(sys_cfg=sys_cfg, distro=None, paths=self.paths) |
887 | ret = dsrc.get_data() |
888 | self.assertTrue(ret) |
889 | # very simple check just for the strings above |
890 | @@ -195,9 +183,23 @@ class TestNoCloudDataSource(CiTestCase): |
891 | |
892 | sys_cfg = {'datasource': {'NoCloud': {'fs_label': None}}} |
893 | |
894 | - ds = DataSourceNoCloud.DataSourceNoCloud |
895 | + dsrc = dsNoCloud(sys_cfg=sys_cfg, distro=None, paths=self.paths) |
896 | + ret = dsrc.get_data() |
897 | + self.assertTrue(ret) |
898 | + self.assertEqual(netconf, dsrc.network_config) |
899 | + |
900 | + def test_metadata_network_config_with_toplevel_network(self, m_is_lxd): |
901 | + """network-config may have 'network' top level key.""" |
902 | + netconf = {'config': 'disabled'} |
903 | + populate_dir( |
904 | + os.path.join(self.paths.seed_dir, "nocloud"), |
905 | + {'user-data': b"ud", |
906 | + 'meta-data': "instance-id: IID\n", |
907 | + 'network-config': yaml.dump({'network': netconf}) + "\n"}) |
908 | + |
909 | + sys_cfg = {'datasource': {'NoCloud': {'fs_label': None}}} |
910 | |
911 | - dsrc = ds(sys_cfg=sys_cfg, distro=None, paths=self.paths) |
912 | + dsrc = dsNoCloud(sys_cfg=sys_cfg, distro=None, paths=self.paths) |
913 | ret = dsrc.get_data() |
914 | self.assertTrue(ret) |
915 | self.assertEqual(netconf, dsrc.network_config) |
916 | @@ -228,9 +230,7 @@ class TestNoCloudDataSource(CiTestCase): |
917 | |
918 | sys_cfg = {'datasource': {'NoCloud': {'fs_label': None}}} |
919 | |
920 | - ds = DataSourceNoCloud.DataSourceNoCloud |
921 | - |
922 | - dsrc = ds(sys_cfg=sys_cfg, distro=None, paths=self.paths) |
923 | + dsrc = dsNoCloud(sys_cfg=sys_cfg, distro=None, paths=self.paths) |
924 | ret = dsrc.get_data() |
925 | self.assertTrue(ret) |
926 | self.assertEqual(netconf, dsrc.network_config) |
927 | @@ -258,8 +258,7 @@ class TestParseCommandLineData(CiTestCase): |
928 | for (fmt, expected) in pairs: |
929 | fill = {} |
930 | cmdline = fmt % {'ds_id': ds_id} |
931 | - ret = DataSourceNoCloud.parse_cmdline_data(ds_id=ds_id, fill=fill, |
932 | - cmdline=cmdline) |
933 | + ret = parse_cmdline_data(ds_id=ds_id, fill=fill, cmdline=cmdline) |
934 | self.assertEqual(expected, fill) |
935 | self.assertTrue(ret) |
936 | |
937 | @@ -276,10 +275,43 @@ class TestParseCommandLineData(CiTestCase): |
938 | |
939 | for cmdline in cmdlines: |
940 | fill = {} |
941 | - ret = DataSourceNoCloud.parse_cmdline_data(ds_id=ds_id, fill=fill, |
942 | - cmdline=cmdline) |
943 | + ret = parse_cmdline_data(ds_id=ds_id, fill=fill, cmdline=cmdline) |
944 | self.assertEqual(fill, {}) |
945 | self.assertFalse(ret) |
946 | |
947 | |
948 | +class TestMaybeRemoveToplevelNetwork(CiTestCase): |
949 | + """test _maybe_remove_top_network function.""" |
950 | + basecfg = [{'type': 'physical', 'name': 'interface0', |
951 | + 'subnets': [{'type': 'dhcp'}]}] |
952 | + |
953 | + def test_should_remove_safely(self): |
954 | + mcfg = {'config': self.basecfg, 'version': 1} |
955 | + self.assertEqual(mcfg, _maybe_remove_top_network({'network': mcfg})) |
956 | + |
957 | + def test_no_remove_if_other_keys(self): |
958 | + """should not shift if other keys at top level.""" |
959 | + mcfg = {'network': {'config': self.basecfg, 'version': 1}, |
960 | + 'unknown_keyname': 'keyval'} |
961 | + self.assertEqual(mcfg, _maybe_remove_top_network(mcfg)) |
962 | + |
963 | + def test_no_remove_if_non_dict(self): |
964 | + """should not shift if not a dict.""" |
965 | + mcfg = {'network': '"content here'} |
966 | + self.assertEqual(mcfg, _maybe_remove_top_network(mcfg)) |
967 | + |
968 | + def test_no_remove_if_missing_config_or_version(self): |
969 | + """should not shift unless network entry has config and version.""" |
970 | + mcfg = {'network': {'config': self.basecfg}} |
971 | + self.assertEqual(mcfg, _maybe_remove_top_network(mcfg)) |
972 | + |
973 | + mcfg = {'network': {'version': 1}} |
974 | + self.assertEqual(mcfg, _maybe_remove_top_network(mcfg)) |
975 | + |
976 | + def test_remove_with_config_disabled(self): |
977 | + """network/config=disabled should be shifted.""" |
978 | + mcfg = {'config': 'disabled'} |
979 | + self.assertEqual(mcfg, _maybe_remove_top_network({'network': mcfg})) |
980 | + |
981 | + |
982 | # vi: ts=4 expandtab |
983 | diff --git a/tests/unittests/test_handler/test_handler_write_files.py b/tests/unittests/test_handler/test_handler_write_files.py |
984 | index 7fa8fd2..bc8756c 100644 |
985 | --- a/tests/unittests/test_handler/test_handler_write_files.py |
986 | +++ b/tests/unittests/test_handler/test_handler_write_files.py |
987 | @@ -52,6 +52,18 @@ class TestWriteFiles(FilesystemMockingTestCase): |
988 | "test_simple", [{"content": expected, "path": filename}]) |
989 | self.assertEqual(util.load_file(filename), expected) |
990 | |
991 | + def test_append(self): |
992 | + self.patchUtils(self.tmp) |
993 | + existing = "hello " |
994 | + added = "world\n" |
995 | + expected = existing + added |
996 | + filename = "/tmp/append.file" |
997 | + util.write_file(filename, existing) |
998 | + write_files( |
999 | + "test_append", |
1000 | + [{"content": added, "path": filename, "append": "true"}]) |
1001 | + self.assertEqual(util.load_file(filename), expected) |
1002 | + |
1003 | def test_yaml_binary(self): |
1004 | self.patchUtils(self.tmp) |
1005 | data = util.load_yaml(YAML_TEXT) |
1006 | diff --git a/tests/unittests/test_net.py b/tests/unittests/test_net.py |
1007 | index 8e38373..195f261 100644 |
1008 | --- a/tests/unittests/test_net.py |
1009 | +++ b/tests/unittests/test_net.py |
1010 | @@ -488,8 +488,8 @@ NETWORK_CONFIGS = { |
1011 | address 192.168.21.3/24 |
1012 | dns-nameservers 8.8.8.8 8.8.4.4 |
1013 | dns-search barley.maas sach.maas |
1014 | - post-up route add default gw 65.61.151.37 || true |
1015 | - pre-down route del default gw 65.61.151.37 || true |
1016 | + post-up route add default gw 65.61.151.37 metric 10000 || true |
1017 | + pre-down route del default gw 65.61.151.37 metric 10000 || true |
1018 | """).rstrip(' '), |
1019 | 'expected_netplan': textwrap.dedent(""" |
1020 | network: |
1021 | @@ -513,7 +513,8 @@ NETWORK_CONFIGS = { |
1022 | - barley.maas |
1023 | - sach.maas |
1024 | routes: |
1025 | - - to: 0.0.0.0/0 |
1026 | + - metric: 10000 |
1027 | + to: 0.0.0.0/0 |
1028 | via: 65.61.151.37 |
1029 | set-name: eth99 |
1030 | """).rstrip(' '), |
1031 | @@ -537,6 +538,7 @@ NETWORK_CONFIGS = { |
1032 | HWADDR=c0:d6:9f:2c:e8:80 |
1033 | IPADDR=192.168.21.3 |
1034 | NETMASK=255.255.255.0 |
1035 | + METRIC=10000 |
1036 | NM_CONTROLLED=no |
1037 | ONBOOT=yes |
1038 | TYPE=Ethernet |
1039 | @@ -561,7 +563,7 @@ NETWORK_CONFIGS = { |
1040 | - gateway: 65.61.151.37 |
1041 | netmask: 0.0.0.0 |
1042 | network: 0.0.0.0 |
1043 | - metric: 2 |
1044 | + metric: 10000 |
1045 | - type: physical |
1046 | name: eth1 |
1047 | mac_address: "cf:d6:af:48:e8:80" |
1048 | @@ -1161,6 +1163,13 @@ pre-down route del -net 10.0.0.0 netmask 255.0.0.0 gw 11.0.0.1 metric 3 || true |
1049 | - gateway: 192.168.0.3 |
1050 | netmask: 255.255.255.0 |
1051 | network: 10.1.3.0 |
1052 | + - gateway: 2001:67c:1562:1 |
1053 | + network: 2001:67c:1 |
1054 | + netmask: ffff:ffff:0 |
1055 | + - gateway: 3001:67c:1562:1 |
1056 | + network: 3001:67c:1 |
1057 | + netmask: ffff:ffff:0 |
1058 | + metric: 10000 |
1059 | - type: static |
1060 | address: 192.168.1.2/24 |
1061 | - type: static |
1062 | @@ -1197,6 +1206,11 @@ pre-down route del -net 10.0.0.0 netmask 255.0.0.0 gw 11.0.0.1 metric 3 || true |
1063 | routes: |
1064 | - to: 10.1.3.0/24 |
1065 | via: 192.168.0.3 |
1066 | + - to: 2001:67c:1/32 |
1067 | + via: 2001:67c:1562:1 |
1068 | + - metric: 10000 |
1069 | + to: 3001:67c:1/32 |
1070 | + via: 3001:67c:1562:1 |
1071 | """), |
1072 | 'yaml-v2': textwrap.dedent(""" |
1073 | version: 2 |
1074 | @@ -1228,6 +1242,11 @@ pre-down route del -net 10.0.0.0 netmask 255.0.0.0 gw 11.0.0.1 metric 3 || true |
1075 | routes: |
1076 | - to: 10.1.3.0/24 |
1077 | via: 192.168.0.3 |
1078 | + - to: 2001:67c:1562:8007::1/64 |
1079 | + via: 2001:67c:1562:8007::aac:40b2 |
1080 | + - metric: 10000 |
1081 | + to: 3001:67c:1562:8007::1/64 |
1082 | + via: 3001:67c:1562:8007::aac:40b2 |
1083 | """), |
1084 | 'expected_netplan-v2': textwrap.dedent(""" |
1085 | network: |
1086 | @@ -1249,6 +1268,11 @@ pre-down route del -net 10.0.0.0 netmask 255.0.0.0 gw 11.0.0.1 metric 3 || true |
1087 | routes: |
1088 | - to: 10.1.3.0/24 |
1089 | via: 192.168.0.3 |
1090 | + - to: 2001:67c:1562:8007::1/64 |
1091 | + via: 2001:67c:1562:8007::aac:40b2 |
1092 | + - metric: 10000 |
1093 | + to: 3001:67c:1562:8007::1/64 |
1094 | + via: 3001:67c:1562:8007::aac:40b2 |
1095 | ethernets: |
1096 | eth0: |
1097 | match: |
1098 | @@ -1349,6 +1373,10 @@ pre-down route del -net 10.0.0.0 netmask 255.0.0.0 gw 11.0.0.1 metric 3 || true |
1099 | USERCTL=no |
1100 | """), |
1101 | 'route6-bond0': textwrap.dedent("""\ |
1102 | + # Created by cloud-init on instance boot automatically, do not edit. |
1103 | + # |
1104 | + 2001:67c:1/ffff:ffff:0 via 2001:67c:1562:1 dev bond0 |
1105 | + 3001:67c:1/ffff:ffff:0 via 3001:67c:1562:1 metric 10000 dev bond0 |
1106 | """), |
1107 | 'route-bond0': textwrap.dedent("""\ |
1108 | ADDRESS0=10.1.3.0 |
1109 | @@ -1879,14 +1907,24 @@ class TestRhelSysConfigRendering(CiTestCase): |
1110 | return dir2dict(dir) |
1111 | |
1112 | def _compare_files_to_expected(self, expected, found): |
1113 | + |
1114 | + def _try_load(f): |
1115 | + ''' Attempt to load shell content, otherwise return as-is ''' |
1116 | + try: |
1117 | + return util.load_shell_content(f) |
1118 | + except ValueError: |
1119 | + pass |
1120 | + # route6- * files aren't shell content, but iproute2 params |
1121 | + return f |
1122 | + |
1123 | orig_maxdiff = self.maxDiff |
1124 | expected_d = dict( |
1125 | - (os.path.join(self.scripts_dir, k), util.load_shell_content(v)) |
1126 | + (os.path.join(self.scripts_dir, k), _try_load(v)) |
1127 | for k, v in expected.items()) |
1128 | |
1129 | # only compare the files in scripts_dir |
1130 | scripts_found = dict( |
1131 | - (k, util.load_shell_content(v)) for k, v in found.items() |
1132 | + (k, _try_load(v)) for k, v in found.items() |
1133 | if k.startswith(self.scripts_dir)) |
1134 | try: |
1135 | self.maxDiff = None |
1136 | diff --git a/tests/unittests/test_vmware_config_file.py b/tests/unittests/test_vmware_config_file.py |
1137 | index 602dedb..f47335e 100644 |
1138 | --- a/tests/unittests/test_vmware_config_file.py |
1139 | +++ b/tests/unittests/test_vmware_config_file.py |
1140 | @@ -263,7 +263,7 @@ class TestVmwareConfigFile(CiTestCase): |
1141 | nicConfigurator = NicConfigurator(config.nics, False) |
1142 | nics_cfg_list = nicConfigurator.generate() |
1143 | |
1144 | - self.assertEqual(5, len(nics_cfg_list), "number of elements") |
1145 | + self.assertEqual(2, len(nics_cfg_list), "number of elements") |
1146 | |
1147 | nic1 = {'name': 'NIC1'} |
1148 | nic2 = {'name': 'NIC2'} |
1149 | @@ -275,8 +275,6 @@ class TestVmwareConfigFile(CiTestCase): |
1150 | nic1.update(cfg) |
1151 | elif cfg.get('name') == nic2.get('name'): |
1152 | nic2.update(cfg) |
1153 | - elif cfg_type == 'route': |
1154 | - route_list.append(cfg) |
1155 | |
1156 | self.assertEqual('physical', nic1.get('type'), 'type of NIC1') |
1157 | self.assertEqual('NIC1', nic1.get('name'), 'name of NIC1') |
1158 | @@ -297,6 +295,9 @@ class TestVmwareConfigFile(CiTestCase): |
1159 | static6_subnet.append(subnet) |
1160 | else: |
1161 | self.assertEqual(True, False, 'Unknown type') |
1162 | + if 'route' in subnet: |
1163 | + for route in subnet.get('routes'): |
1164 | + route_list.append(route) |
1165 | |
1166 | self.assertEqual(1, len(static_subnet), 'Number of static subnet') |
1167 | self.assertEqual(1, len(static6_subnet), 'Number of static6 subnet') |
1168 | @@ -351,6 +352,8 @@ class TestVmwareConfigFile(CiTestCase): |
1169 | class TestVmwareNetConfig(CiTestCase): |
1170 | """Test conversion of vmware config to cloud-init config.""" |
1171 | |
1172 | + maxDiff = None |
1173 | + |
1174 | def _get_NicConfigurator(self, text): |
1175 | fp = None |
1176 | try: |
1177 | @@ -420,9 +423,52 @@ class TestVmwareNetConfig(CiTestCase): |
1178 | 'mac_address': '00:50:56:a6:8c:08', |
1179 | 'subnets': [ |
1180 | {'control': 'auto', 'type': 'static', |
1181 | - 'address': '10.20.87.154', 'netmask': '255.255.252.0'}]}, |
1182 | - {'type': 'route', 'destination': '10.20.84.0/22', |
1183 | - 'gateway': '10.20.87.253', 'metric': 10000}], |
1184 | + 'address': '10.20.87.154', 'netmask': '255.255.252.0', |
1185 | + 'routes': |
1186 | + [{'type': 'route', 'destination': '10.20.84.0/22', |
1187 | + 'gateway': '10.20.87.253', 'metric': 10000}]}]}], |
1188 | + nc.generate()) |
1189 | + |
1190 | + def test_cust_non_primary_nic_with_gateway_(self): |
1191 | + """A customer non primary nic set can have a gateway.""" |
1192 | + config = textwrap.dedent("""\ |
1193 | + [NETWORK] |
1194 | + NETWORKING = yes |
1195 | + BOOTPROTO = dhcp |
1196 | + HOSTNAME = static-debug-vm |
1197 | + DOMAINNAME = cluster.local |
1198 | + |
1199 | + [NIC-CONFIG] |
1200 | + NICS = NIC1 |
1201 | + |
1202 | + [NIC1] |
1203 | + MACADDR = 00:50:56:ac:d1:8a |
1204 | + ONBOOT = yes |
1205 | + IPv4_MODE = BACKWARDS_COMPATIBLE |
1206 | + BOOTPROTO = static |
1207 | + IPADDR = 100.115.223.75 |
1208 | + NETMASK = 255.255.255.0 |
1209 | + GATEWAY = 100.115.223.254 |
1210 | + |
1211 | + |
1212 | + [DNS] |
1213 | + DNSFROMDHCP=no |
1214 | + |
1215 | + NAMESERVER|1 = 8.8.8.8 |
1216 | + |
1217 | + [DATETIME] |
1218 | + UTC = yes |
1219 | + """) |
1220 | + nc = self._get_NicConfigurator(config) |
1221 | + self.assertEqual( |
1222 | + [{'type': 'physical', 'name': 'NIC1', |
1223 | + 'mac_address': '00:50:56:ac:d1:8a', |
1224 | + 'subnets': [ |
1225 | + {'control': 'auto', 'type': 'static', |
1226 | + 'address': '100.115.223.75', 'netmask': '255.255.255.0', |
1227 | + 'routes': |
1228 | + [{'type': 'route', 'destination': '100.115.223.0/24', |
1229 | + 'gateway': '100.115.223.254', 'metric': 10000}]}]}], |
1230 | nc.generate()) |
1231 | |
1232 | def test_a_primary_nic_with_gateway(self): |
1233 | diff --git a/tox.ini b/tox.ini |
1234 | index 2fb3209..d983348 100644 |
1235 | --- a/tox.ini |
1236 | +++ b/tox.ini |
1237 | @@ -21,7 +21,7 @@ setenv = |
1238 | basepython = python3 |
1239 | deps = |
1240 | # requirements |
1241 | - pylint==1.8.1 |
1242 | + pylint==2.2.2 |
1243 | # test-requirements because unit tests are now present in cloudinit tree |
1244 | -r{toxinidir}/test-requirements.txt |
1245 | commands = {envpython} -m pylint {posargs:cloudinit tests tools} |
PASSED: Continuous integration, rev:47a4d44a244 aefb55e2a192bd0 69ae830c6d176e /jenkins. ubuntu. com/server/ job/cloud- init-ci/ 490/
https:/
Executed test runs:
SUCCESS: Checkout
SUCCESS: Unit & Style Tests
SUCCESS: Ubuntu LTS: Build
SUCCESS: Ubuntu LTS: Integration
IN_PROGRESS: Declarative: Post Actions
Click here to trigger a rebuild: /jenkins. ubuntu. com/server/ job/cloud- init-ci/ 490/rebuild
https:/