Merge ~raharper/cloud-init:ubuntu/devel/newupstream-20181214 into cloud-init:ubuntu/devel

Proposed by Ryan Harper on 2018-12-14
Status: Merged
Merged at revision: 47a4d44a244aefb55e2a192bd069ae830c6d176e
Proposed branch: ~raharper/cloud-init:ubuntu/devel/newupstream-20181214
Merge into: cloud-init:ubuntu/devel
Diff against target: 1245 lines (+556/-145)
23 files modified
ChangeLog (+54/-0)
bash_completion/cloud-init (+4/-1)
cloudinit/cmd/devel/net_convert.py (+10/-5)
cloudinit/cmd/main.py (+4/-16)
cloudinit/config/cc_write_files.py (+6/-1)
cloudinit/dhclient_hook.py (+72/-38)
cloudinit/net/eni.py (+15/-14)
cloudinit/net/netplan.py (+3/-3)
cloudinit/net/sysconfig.py (+21/-4)
cloudinit/sources/DataSourceAzure.py (+2/-2)
cloudinit/sources/DataSourceNoCloud.py (+31/-1)
cloudinit/sources/helpers/vmware/imc/config_nic.py (+2/-3)
cloudinit/tests/test_dhclient_hook.py (+105/-0)
cloudinit/version.py (+1/-1)
config/cloud.cfg.tmpl (+11/-1)
debian/changelog (+16/-0)
tests/cloud_tests/releases.yaml (+16/-0)
tests/unittests/test_cli.py (+8/-8)
tests/unittests/test_datasource/test_nocloud.py (+66/-34)
tests/unittests/test_handler/test_handler_write_files.py (+12/-0)
tests/unittests/test_net.py (+44/-6)
tests/unittests/test_vmware_config_file.py (+52/-6)
tox.ini (+1/-1)
Reviewer Review Type Date Requested Status
Server Team CI bot continuous-integration Approve on 2018-12-14
cloud-init Commiters 2018-12-14 Pending
Review via email: mp+360957@code.launchpad.net

Commit message

cloud-init (18.5-1-g5b065316-0ubuntu1) disco; urgency=medium

  * New upstream snapshot.
    - Update to pylint 2.2.2.
    - Release 18.5 (LP: #1808380)
    - tests: add Disco release [Joshua Powers]
    - net: render 'metric' values in per-subnet routes (LP: #1805871)
    - write_files: add support for appending to files. [James Baxter]
    - config: On ubuntu select cloud archive mirrors for armel, armhf, arm64.
      (LP: #1805854)
    - dhclient-hook: cleanups, tests and fix a bug on 'down' event.
    - NoCloud: Allow top level 'network' key in network-config. (LP: #1798117)
    - ovf: Fix ovf network config generation gateway/routes (LP: #1806103)

 -- Ryan Harper <email address hidden> Fri, 14 Dec 2018 14:45:46 -0600

To post a comment you must log in.

PASSED: Continuous integration, rev:47a4d44a244aefb55e2a192bd069ae830c6d176e
https://jenkins.ubuntu.com/server/job/cloud-init-ci/490/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/490/rebuild

review: Approve (continuous-integration)

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1diff --git a/ChangeLog b/ChangeLog
2index 9c043b0..8fa6fdd 100644
3--- a/ChangeLog
4+++ b/ChangeLog
5@@ -1,3 +1,57 @@
6+18.5:
7+ - tests: add Disco release [Joshua Powers]
8+ - net: render 'metric' values in per-subnet routes (LP: #1805871)
9+ - write_files: add support for appending to files. [James Baxter]
10+ - config: On ubuntu select cloud archive mirrors for armel, armhf, arm64.
11+ (LP: #1805854)
12+ - dhclient-hook: cleanups, tests and fix a bug on 'down' event.
13+ - NoCloud: Allow top level 'network' key in network-config. (LP: #1798117)
14+ - ovf: Fix ovf network config generation gateway/routes (LP: #1806103)
15+ - azure: detect vnet migration via netlink media change event
16+ [Tamilmani Manoharan]
17+ - Azure: fix copy/paste error in error handling when reading azure ovf.
18+ [Adam DePue]
19+ - tests: fix incorrect order of mocks in test_handle_zfs_root.
20+ - doc: Change dns_nameserver property to dns_nameservers. [Tomer Cohen]
21+ - OVF: identify label iso9660 filesystems with label 'OVF ENV'.
22+ - logs: collect-logs ignore instance-data-sensitive.json on non-root user
23+ (LP: #1805201)
24+ - net: Ephemeral*Network: add connectivity check via URL
25+ - azure: _poll_imds only retry on 404. Fail on Timeout (LP: #1803598)
26+ - resizefs: Prefix discovered devpath with '/dev/' when path does not
27+ exist [Igor Galić]
28+ - azure: retry imds polling on requests.Timeout (LP: #1800223)
29+ - azure: Accept variation in error msg from mount for ntfs volumes
30+ [Jason Zions] (LP: #1799338)
31+ - azure: fix regression introduced when persisting ephemeral dhcp lease
32+ [asakkurr]
33+ - azure: add udev rules to create cloud-init Gen2 disk name symlinks
34+ (LP: #1797480)
35+ - tests: ec2 mock missing httpretty user-data and instance-identity routes
36+ - azure: remove /etc/netplan/90-hotplug-azure.yaml when net from IMDS
37+ - azure: report ready to fabric after reprovision and reduce logging
38+ [asakkurr] (LP: #1799594)
39+ - query: better error when missing read permission on instance-data
40+ - instance-data: fallback to instance-data.json if sensitive is absent.
41+ (LP: #1798189)
42+ - docs: remove colon from network v1 config example. [Tomer Cohen]
43+ - Add cloud-id binary to packages for SUSE [Jason Zions]
44+ - systemd: On SUSE ensure cloud-init.service runs before wicked
45+ [Robert Schweikert] (LP: #1799709)
46+ - update detection of openSUSE variants [Robert Schweikert]
47+ - azure: Add apply_network_config option to disable network from IMDS
48+ (LP: #1798424)
49+ - Correct spelling in an error message (udevadm). [Katie McLaughlin]
50+ - tests: meta_data key changed to meta-data in ec2 instance-data.json
51+ (LP: #1797231)
52+ - tests: fix kvm integration test to assert flexible config-disk path
53+ (LP: #1797199)
54+ - tools: Add cloud-id command line utility
55+ - instance-data: Add standard keys platform and subplatform. Refactor ec2.
56+ - net: ignore nics that have "zero" mac address. (LP: #1796917)
57+ - tests: fix apt_configure_primary to be more flexible
58+ - Ubuntu: update sources.list to comment out deb-src entries. (LP: #74747)
59+
60 18.4:
61 - add rtd example docs about new standardized keys
62 - use ds._crawled_metadata instance attribute if set when writing
63diff --git a/bash_completion/cloud-init b/bash_completion/cloud-init
64index 8c25032..a9577e9 100644
65--- a/bash_completion/cloud-init
66+++ b/bash_completion/cloud-init
67@@ -30,7 +30,10 @@ _cloudinit_complete()
68 devel)
69 COMPREPLY=($(compgen -W "--help schema net-convert" -- $cur_word))
70 ;;
71- dhclient-hook|features)
72+ dhclient-hook)
73+ COMPREPLY=($(compgen -W "--help up down" -- $cur_word))
74+ ;;
75+ features)
76 COMPREPLY=($(compgen -W "--help" -- $cur_word))
77 ;;
78 init)
79diff --git a/cloudinit/cmd/devel/net_convert.py b/cloudinit/cmd/devel/net_convert.py
80index a0f58a0..1ad7e0b 100755
81--- a/cloudinit/cmd/devel/net_convert.py
82+++ b/cloudinit/cmd/devel/net_convert.py
83@@ -9,6 +9,7 @@ import yaml
84
85 from cloudinit.sources.helpers import openstack
86 from cloudinit.sources import DataSourceAzure as azure
87+from cloudinit.sources import DataSourceOVF as ovf
88
89 from cloudinit import distros
90 from cloudinit.net import eni, netplan, network_state, sysconfig
91@@ -31,7 +32,7 @@ def get_parser(parser=None):
92 metavar="PATH", required=True)
93 parser.add_argument("-k", "--kind",
94 choices=['eni', 'network_data.json', 'yaml',
95- 'azure-imds'],
96+ 'azure-imds', 'vmware-imc'],
97 required=True)
98 parser.add_argument("-d", "--directory",
99 metavar="PATH",
100@@ -76,7 +77,6 @@ def handle_args(name, args):
101 net_data = args.network_data.read()
102 if args.kind == "eni":
103 pre_ns = eni.convert_eni_data(net_data)
104- ns = network_state.parse_net_config_data(pre_ns)
105 elif args.kind == "yaml":
106 pre_ns = yaml.load(net_data)
107 if 'network' in pre_ns:
108@@ -85,15 +85,16 @@ def handle_args(name, args):
109 sys.stderr.write('\n'.join(
110 ["Input YAML",
111 yaml.dump(pre_ns, default_flow_style=False, indent=4), ""]))
112- ns = network_state.parse_net_config_data(pre_ns)
113 elif args.kind == 'network_data.json':
114 pre_ns = openstack.convert_net_json(
115 json.loads(net_data), known_macs=known_macs)
116- ns = network_state.parse_net_config_data(pre_ns)
117 elif args.kind == 'azure-imds':
118 pre_ns = azure.parse_network_config(json.loads(net_data))
119- ns = network_state.parse_net_config_data(pre_ns)
120+ elif args.kind == 'vmware-imc':
121+ config = ovf.Config(ovf.ConfigFile(args.network_data.name))
122+ pre_ns = ovf.get_network_config_from_conf(config, False)
123
124+ ns = network_state.parse_net_config_data(pre_ns)
125 if not ns:
126 raise RuntimeError("No valid network_state object created from"
127 "input data")
128@@ -111,6 +112,10 @@ def handle_args(name, args):
129 elif args.output_kind == "netplan":
130 r_cls = netplan.Renderer
131 config = distro.renderer_configs.get('netplan')
132+ # don't run netplan generate/apply
133+ config['postcmds'] = False
134+ # trim leading slash
135+ config['netplan_path'] = config['netplan_path'][1:]
136 else:
137 r_cls = sysconfig.Renderer
138 config = distro.renderer_configs.get('sysconfig')
139diff --git a/cloudinit/cmd/main.py b/cloudinit/cmd/main.py
140index 5a43702..933c019 100644
141--- a/cloudinit/cmd/main.py
142+++ b/cloudinit/cmd/main.py
143@@ -41,7 +41,7 @@ from cloudinit.settings import (PER_INSTANCE, PER_ALWAYS, PER_ONCE,
144 from cloudinit import atomic_helper
145
146 from cloudinit.config import cc_set_hostname
147-from cloudinit.dhclient_hook import LogDhclient
148+from cloudinit import dhclient_hook
149
150
151 # Welcome message template
152@@ -586,12 +586,6 @@ def main_single(name, args):
153 return 0
154
155
156-def dhclient_hook(name, args):
157- record = LogDhclient(args)
158- record.check_hooks_dir()
159- record.record()
160-
161-
162 def status_wrapper(name, args, data_d=None, link_d=None):
163 if data_d is None:
164 data_d = os.path.normpath("/var/lib/cloud/data")
165@@ -795,15 +789,9 @@ def main(sysv_args=None):
166 'query',
167 help='Query standardized instance metadata from the command line.')
168
169- parser_dhclient = subparsers.add_parser('dhclient-hook',
170- help=('run the dhclient hook'
171- 'to record network info'))
172- parser_dhclient.add_argument("net_action",
173- help=('action taken on the interface'))
174- parser_dhclient.add_argument("net_interface",
175- help=('the network interface being acted'
176- ' upon'))
177- parser_dhclient.set_defaults(action=('dhclient_hook', dhclient_hook))
178+ parser_dhclient = subparsers.add_parser(
179+ dhclient_hook.NAME, help=dhclient_hook.__doc__)
180+ dhclient_hook.get_parser(parser_dhclient)
181
182 parser_features = subparsers.add_parser('features',
183 help=('list defined features'))
184diff --git a/cloudinit/config/cc_write_files.py b/cloudinit/config/cc_write_files.py
185index 31d1db6..0b6546e 100644
186--- a/cloudinit/config/cc_write_files.py
187+++ b/cloudinit/config/cc_write_files.py
188@@ -49,6 +49,10 @@ binary gzip data can be specified and will be decoded before being written.
189 ...
190 path: /bin/arch
191 permissions: '0555'
192+ - content: |
193+ 15 * * * * root ship_logs
194+ path: /etc/crontab
195+ append: true
196 """
197
198 import base64
199@@ -113,7 +117,8 @@ def write_files(name, files):
200 contents = extract_contents(f_info.get('content', ''), extractions)
201 (u, g) = util.extract_usergroup(f_info.get('owner', DEFAULT_OWNER))
202 perms = decode_perms(f_info.get('permissions'), DEFAULT_PERMS)
203- util.write_file(path, contents, mode=perms)
204+ omode = 'ab' if util.get_cfg_option_bool(f_info, 'append') else 'wb'
205+ util.write_file(path, contents, omode=omode, mode=perms)
206 util.chownbyname(path, u, g)
207
208
209diff --git a/cloudinit/dhclient_hook.py b/cloudinit/dhclient_hook.py
210index 7f02d7f..72b51b6 100644
211--- a/cloudinit/dhclient_hook.py
212+++ b/cloudinit/dhclient_hook.py
213@@ -1,5 +1,8 @@
214 # This file is part of cloud-init. See LICENSE file for license information.
215
216+"""Run the dhclient hook to record network info."""
217+
218+import argparse
219 import os
220
221 from cloudinit import atomic_helper
222@@ -8,44 +11,75 @@ from cloudinit import stages
223
224 LOG = logging.getLogger(__name__)
225
226+NAME = "dhclient-hook"
227+UP = "up"
228+DOWN = "down"
229+EVENTS = (UP, DOWN)
230+
231+
232+def _get_hooks_dir():
233+ i = stages.Init()
234+ return os.path.join(i.paths.get_runpath(), 'dhclient.hooks')
235+
236+
237+def _filter_env_vals(info):
238+ """Given info (os.environ), return a dictionary with
239+ lower case keys for each entry starting with DHCP4_ or new_."""
240+ new_info = {}
241+ for k, v in info.items():
242+ if k.startswith("DHCP4_") or k.startswith("new_"):
243+ key = (k.replace('DHCP4_', '').replace('new_', '')).lower()
244+ new_info[key] = v
245+ return new_info
246+
247+
248+def run_hook(interface, event, data_d=None, env=None):
249+ if event not in EVENTS:
250+ raise ValueError("Unexpected event '%s'. Expected one of: %s" %
251+ (event, EVENTS))
252+ if data_d is None:
253+ data_d = _get_hooks_dir()
254+ if env is None:
255+ env = os.environ
256+ hook_file = os.path.join(data_d, interface + ".json")
257+
258+ if event == UP:
259+ if not os.path.exists(data_d):
260+ os.makedirs(data_d)
261+ atomic_helper.write_json(hook_file, _filter_env_vals(env))
262+ LOG.debug("Wrote dhclient options in %s", hook_file)
263+ elif event == DOWN:
264+ if os.path.exists(hook_file):
265+ os.remove(hook_file)
266+ LOG.debug("Removed dhclient options file %s", hook_file)
267+
268+
269+def get_parser(parser=None):
270+ if parser is None:
271+ parser = argparse.ArgumentParser(prog=NAME, description=__doc__)
272+ parser.add_argument(
273+ "event", help='event taken on the interface', choices=EVENTS)
274+ parser.add_argument(
275+ "interface", help='the network interface being acted upon')
276+ # cloud-init main uses 'action'
277+ parser.set_defaults(action=(NAME, handle_args))
278+ return parser
279+
280+
281+def handle_args(name, args, data_d=None):
282+ """Handle the Namespace args.
283+ Takes 'name' as passed by cloud-init main. not used here."""
284+ return run_hook(interface=args.interface, event=args.event, data_d=data_d)
285+
286+
287+if __name__ == '__main__':
288+ import sys
289+ parser = get_parser()
290+ args = parser.parse_args(args=sys.argv[1:])
291+ return_value = handle_args(
292+ NAME, args, data_d=os.environ.get('_CI_DHCP_HOOK_DATA_D'))
293+ if return_value:
294+ sys.exit(return_value)
295
296-class LogDhclient(object):
297-
298- def __init__(self, cli_args):
299- self.hooks_dir = self._get_hooks_dir()
300- self.net_interface = cli_args.net_interface
301- self.net_action = cli_args.net_action
302- self.hook_file = os.path.join(self.hooks_dir,
303- self.net_interface + ".json")
304-
305- @staticmethod
306- def _get_hooks_dir():
307- i = stages.Init()
308- return os.path.join(i.paths.get_runpath(), 'dhclient.hooks')
309-
310- def check_hooks_dir(self):
311- if not os.path.exists(self.hooks_dir):
312- os.makedirs(self.hooks_dir)
313- else:
314- # If the action is down and the json file exists, we need to
315- # delete the file
316- if self.net_action is 'down' and os.path.exists(self.hook_file):
317- os.remove(self.hook_file)
318-
319- @staticmethod
320- def get_vals(info):
321- new_info = {}
322- for k, v in info.items():
323- if k.startswith("DHCP4_") or k.startswith("new_"):
324- key = (k.replace('DHCP4_', '').replace('new_', '')).lower()
325- new_info[key] = v
326- return new_info
327-
328- def record(self):
329- envs = os.environ
330- if self.hook_file is None:
331- return
332- atomic_helper.write_json(self.hook_file, self.get_vals(envs))
333- LOG.debug("Wrote dhclient options in %s", self.hook_file)
334
335 # vi: ts=4 expandtab
336diff --git a/cloudinit/net/eni.py b/cloudinit/net/eni.py
337index c6f631a..6423632 100644
338--- a/cloudinit/net/eni.py
339+++ b/cloudinit/net/eni.py
340@@ -371,22 +371,23 @@ class Renderer(renderer.Renderer):
341 'gateway': 'gw',
342 'metric': 'metric',
343 }
344+
345+ default_gw = ''
346 if route['network'] == '0.0.0.0' and route['netmask'] == '0.0.0.0':
347- default_gw = " default gw %s" % route['gateway']
348- content.append(up + default_gw + or_true)
349- content.append(down + default_gw + or_true)
350+ default_gw = ' default'
351 elif route['network'] == '::' and route['prefix'] == 0:
352- # ipv6!
353- default_gw = " -A inet6 default gw %s" % route['gateway']
354- content.append(up + default_gw + or_true)
355- content.append(down + default_gw + or_true)
356- else:
357- route_line = ""
358- for k in ['network', 'netmask', 'gateway', 'metric']:
359- if k in route:
360- route_line += " %s %s" % (mapping[k], route[k])
361- content.append(up + route_line + or_true)
362- content.append(down + route_line + or_true)
363+ default_gw = ' -A inet6 default'
364+
365+ route_line = ''
366+ for k in ['network', 'netmask', 'gateway', 'metric']:
367+ if default_gw and k in ['network', 'netmask']:
368+ continue
369+ if k == 'gateway':
370+ route_line += '%s %s %s' % (default_gw, mapping[k], route[k])
371+ elif k in route:
372+ route_line += ' %s %s' % (mapping[k], route[k])
373+ content.append(up + route_line + or_true)
374+ content.append(down + route_line + or_true)
375 return content
376
377 def _render_iface(self, iface, render_hwaddress=False):
378diff --git a/cloudinit/net/netplan.py b/cloudinit/net/netplan.py
379index bc1087f..21517fd 100644
380--- a/cloudinit/net/netplan.py
381+++ b/cloudinit/net/netplan.py
382@@ -114,13 +114,13 @@ def _extract_addresses(config, entry, ifname):
383 for route in subnet.get('routes', []):
384 to_net = "%s/%s" % (route.get('network'),
385 route.get('prefix'))
386- route = {
387+ new_route = {
388 'via': route.get('gateway'),
389 'to': to_net,
390 }
391 if 'metric' in route:
392- route.update({'metric': route.get('metric', 100)})
393- routes.append(route)
394+ new_route.update({'metric': route.get('metric', 100)})
395+ routes.append(new_route)
396
397 addresses.append(addr)
398
399diff --git a/cloudinit/net/sysconfig.py b/cloudinit/net/sysconfig.py
400index 9c16d3a..17293e1 100644
401--- a/cloudinit/net/sysconfig.py
402+++ b/cloudinit/net/sysconfig.py
403@@ -156,13 +156,23 @@ class Route(ConfigMap):
404 _quote_value(gateway_value)))
405 buf.write("%s=%s\n" % ('NETMASK' + str(reindex),
406 _quote_value(netmask_value)))
407+ metric_key = 'METRIC' + index
408+ if metric_key in self._conf:
409+ metric_value = str(self._conf['METRIC' + index])
410+ buf.write("%s=%s\n" % ('METRIC' + str(reindex),
411+ _quote_value(metric_value)))
412 elif proto == "ipv6" and self.is_ipv6_route(address_value):
413 netmask_value = str(self._conf['NETMASK' + index])
414 gateway_value = str(self._conf['GATEWAY' + index])
415- buf.write("%s/%s via %s dev %s\n" % (address_value,
416- netmask_value,
417- gateway_value,
418- self._route_name))
419+ metric_value = (
420+ 'metric ' + str(self._conf['METRIC' + index])
421+ if 'METRIC' + index in self._conf else '')
422+ buf.write(
423+ "%s/%s via %s %s dev %s\n" % (address_value,
424+ netmask_value,
425+ gateway_value,
426+ metric_value,
427+ self._route_name))
428
429 return buf.getvalue()
430
431@@ -370,6 +380,9 @@ class Renderer(renderer.Renderer):
432 else:
433 iface_cfg['GATEWAY'] = subnet['gateway']
434
435+ if 'metric' in subnet:
436+ iface_cfg['METRIC'] = subnet['metric']
437+
438 if 'dns_search' in subnet:
439 iface_cfg['DOMAIN'] = ' '.join(subnet['dns_search'])
440
441@@ -414,15 +427,19 @@ class Renderer(renderer.Renderer):
442 else:
443 iface_cfg['GATEWAY'] = route['gateway']
444 route_cfg.has_set_default_ipv4 = True
445+ if 'metric' in route:
446+ iface_cfg['METRIC'] = route['metric']
447
448 else:
449 gw_key = 'GATEWAY%s' % route_cfg.last_idx
450 nm_key = 'NETMASK%s' % route_cfg.last_idx
451 addr_key = 'ADDRESS%s' % route_cfg.last_idx
452+ metric_key = 'METRIC%s' % route_cfg.last_idx
453 route_cfg.last_idx += 1
454 # add default routes only to ifcfg files, not
455 # to route-* or route6-*
456 for (old_key, new_key) in [('gateway', gw_key),
457+ ('metric', metric_key),
458 ('netmask', nm_key),
459 ('network', addr_key)]:
460 if old_key in route:
461diff --git a/cloudinit/sources/DataSourceAzure.py b/cloudinit/sources/DataSourceAzure.py
462index e076d5d..46efca4 100644
463--- a/cloudinit/sources/DataSourceAzure.py
464+++ b/cloudinit/sources/DataSourceAzure.py
465@@ -980,8 +980,8 @@ def read_azure_ovf(contents):
466 raise NonAzureDataSource("No LinuxProvisioningConfigurationSet")
467 if len(lpcs_nodes) > 1:
468 raise BrokenAzureDataSource("found '%d' %ss" %
469- ("LinuxProvisioningConfigurationSet",
470- len(lpcs_nodes)))
471+ (len(lpcs_nodes),
472+ "LinuxProvisioningConfigurationSet"))
473 lpcs = lpcs_nodes[0]
474
475 if not lpcs.hasChildNodes():
476diff --git a/cloudinit/sources/DataSourceNoCloud.py b/cloudinit/sources/DataSourceNoCloud.py
477index 9010f06..6860f0c 100644
478--- a/cloudinit/sources/DataSourceNoCloud.py
479+++ b/cloudinit/sources/DataSourceNoCloud.py
480@@ -311,6 +311,35 @@ def parse_cmdline_data(ds_id, fill, cmdline=None):
481 return True
482
483
484+def _maybe_remove_top_network(cfg):
485+ """If network-config contains top level 'network' key, then remove it.
486+
487+ Some providers of network configuration may provide a top level
488+ 'network' key (LP: #1798117) even though it is not necessary.
489+
490+ Be friendly and remove it if it really seems so.
491+
492+ Return the original value if no change or the updated value if changed."""
493+ nullval = object()
494+ network_val = cfg.get('network', nullval)
495+ if network_val is nullval:
496+ return cfg
497+ bmsg = 'Top level network key in network-config %s: %s'
498+ if not isinstance(network_val, dict):
499+ LOG.debug(bmsg, "was not a dict", cfg)
500+ return cfg
501+ if len(list(cfg.keys())) != 1:
502+ LOG.debug(bmsg, "had multiple top level keys", cfg)
503+ return cfg
504+ if network_val.get('config') == "disabled":
505+ LOG.debug(bmsg, "was config/disabled", cfg)
506+ elif not all(('config' in network_val, 'version' in network_val)):
507+ LOG.debug(bmsg, "but missing 'config' or 'version'", cfg)
508+ return cfg
509+ LOG.debug(bmsg, "fixed by removing shifting network.", cfg)
510+ return network_val
511+
512+
513 def _merge_new_seed(cur, seeded):
514 ret = cur.copy()
515
516@@ -320,7 +349,8 @@ def _merge_new_seed(cur, seeded):
517 ret['meta-data'] = util.mergemanydict([cur['meta-data'], newmd])
518
519 if seeded.get('network-config'):
520- ret['network-config'] = util.load_yaml(seeded['network-config'])
521+ ret['network-config'] = _maybe_remove_top_network(
522+ util.load_yaml(seeded.get('network-config')))
523
524 if 'user-data' in seeded:
525 ret['user-data'] = seeded['user-data']
526diff --git a/cloudinit/sources/helpers/vmware/imc/config_nic.py b/cloudinit/sources/helpers/vmware/imc/config_nic.py
527index e1890e2..77cbf3b 100644
528--- a/cloudinit/sources/helpers/vmware/imc/config_nic.py
529+++ b/cloudinit/sources/helpers/vmware/imc/config_nic.py
530@@ -165,9 +165,8 @@ class NicConfigurator(object):
531
532 # Add routes if there is no primary nic
533 if not self._primaryNic and v4.gateways:
534- route_list.extend(self.gen_ipv4_route(nic,
535- v4.gateways,
536- v4.netmask))
537+ subnet.update(
538+ {'routes': self.gen_ipv4_route(nic, v4.gateways, v4.netmask)})
539
540 return ([subnet], route_list)
541
542diff --git a/cloudinit/tests/test_dhclient_hook.py b/cloudinit/tests/test_dhclient_hook.py
543new file mode 100644
544index 0000000..7aab8dd
545--- /dev/null
546+++ b/cloudinit/tests/test_dhclient_hook.py
547@@ -0,0 +1,105 @@
548+# This file is part of cloud-init. See LICENSE file for license information.
549+
550+"""Tests for cloudinit.dhclient_hook."""
551+
552+from cloudinit import dhclient_hook as dhc
553+from cloudinit.tests.helpers import CiTestCase, dir2dict, populate_dir
554+
555+import argparse
556+import json
557+import mock
558+import os
559+
560+
561+class TestDhclientHook(CiTestCase):
562+
563+ ex_env = {
564+ 'interface': 'eth0',
565+ 'new_dhcp_lease_time': '3600',
566+ 'new_host_name': 'x1',
567+ 'new_ip_address': '10.145.210.163',
568+ 'new_subnet_mask': '255.255.255.0',
569+ 'old_host_name': 'x1',
570+ 'PATH': '/usr/sbin:/usr/bin:/sbin:/bin',
571+ 'pid': '614',
572+ 'reason': 'BOUND',
573+ }
574+
575+ # some older versions of dhclient put the same content,
576+ # but in upper case with DHCP4_ instead of new_
577+ ex_env_dhcp4 = {
578+ 'REASON': 'BOUND',
579+ 'DHCP4_dhcp_lease_time': '3600',
580+ 'DHCP4_host_name': 'x1',
581+ 'DHCP4_ip_address': '10.145.210.163',
582+ 'DHCP4_subnet_mask': '255.255.255.0',
583+ 'INTERFACE': 'eth0',
584+ 'PATH': '/usr/sbin:/usr/bin:/sbin:/bin',
585+ 'pid': '614',
586+ }
587+
588+ expected = {
589+ 'dhcp_lease_time': '3600',
590+ 'host_name': 'x1',
591+ 'ip_address': '10.145.210.163',
592+ 'subnet_mask': '255.255.255.0'}
593+
594+ def setUp(self):
595+ super(TestDhclientHook, self).setUp()
596+ self.tmp = self.tmp_dir()
597+
598+ def test_handle_args(self):
599+ """quick test of call to handle_args."""
600+ nic = 'eth0'
601+ args = argparse.Namespace(event=dhc.UP, interface=nic)
602+ with mock.patch.dict("os.environ", clear=True, values=self.ex_env):
603+ dhc.handle_args(dhc.NAME, args, data_d=self.tmp)
604+ found = dir2dict(self.tmp + os.path.sep)
605+ self.assertEqual([nic + ".json"], list(found.keys()))
606+ self.assertEqual(self.expected, json.loads(found[nic + ".json"]))
607+
608+ def test_run_hook_up_creates_dir(self):
609+ """If dir does not exist, run_hook should create it."""
610+ subd = self.tmp_path("subdir", self.tmp)
611+ nic = 'eth1'
612+ dhc.run_hook(nic, 'up', data_d=subd, env=self.ex_env)
613+ self.assertEqual(
614+ set([nic + ".json"]), set(dir2dict(subd + os.path.sep)))
615+
616+ def test_run_hook_up(self):
617+ """Test expected use of run_hook_up."""
618+ nic = 'eth0'
619+ dhc.run_hook(nic, 'up', data_d=self.tmp, env=self.ex_env)
620+ found = dir2dict(self.tmp + os.path.sep)
621+ self.assertEqual([nic + ".json"], list(found.keys()))
622+ self.assertEqual(self.expected, json.loads(found[nic + ".json"]))
623+
624+ def test_run_hook_up_dhcp4_prefix(self):
625+ """Test run_hook filters correctly with older DHCP4_ data."""
626+ nic = 'eth0'
627+ dhc.run_hook(nic, 'up', data_d=self.tmp, env=self.ex_env_dhcp4)
628+ found = dir2dict(self.tmp + os.path.sep)
629+ self.assertEqual([nic + ".json"], list(found.keys()))
630+ self.assertEqual(self.expected, json.loads(found[nic + ".json"]))
631+
632+ def test_run_hook_down_deletes(self):
633+ """down should delete the created json file."""
634+ nic = 'eth1'
635+ populate_dir(
636+ self.tmp, {nic + ".json": "{'abcd'}", 'myfile.txt': 'text'})
637+ dhc.run_hook(nic, 'down', data_d=self.tmp, env={'old_host_name': 'x1'})
638+ self.assertEqual(
639+ set(['myfile.txt']),
640+ set(dir2dict(self.tmp + os.path.sep)))
641+
642+ def test_get_parser(self):
643+ """Smoke test creation of get_parser."""
644+ # cloud-init main uses 'action'.
645+ event, interface = (dhc.UP, 'mynic0')
646+ self.assertEqual(
647+ argparse.Namespace(event=event, interface=interface,
648+ action=(dhc.NAME, dhc.handle_args)),
649+ dhc.get_parser().parse_args([event, interface]))
650+
651+
652+# vi: ts=4 expandtab
653diff --git a/cloudinit/version.py b/cloudinit/version.py
654index 844a02e..a2c5d43 100644
655--- a/cloudinit/version.py
656+++ b/cloudinit/version.py
657@@ -4,7 +4,7 @@
658 #
659 # This file is part of cloud-init. See LICENSE file for license information.
660
661-__VERSION__ = "18.4"
662+__VERSION__ = "18.5"
663 _PACKAGED_VERSION = '@@PACKAGED_VERSION@@'
664
665 FEATURES = [
666diff --git a/config/cloud.cfg.tmpl b/config/cloud.cfg.tmpl
667index 1fef133..7513176 100644
668--- a/config/cloud.cfg.tmpl
669+++ b/config/cloud.cfg.tmpl
670@@ -167,7 +167,17 @@ system_info:
671 - http://%(availability_zone)s.clouds.archive.ubuntu.com/ubuntu/
672 - http://%(region)s.clouds.archive.ubuntu.com/ubuntu/
673 security: []
674- - arches: [armhf, armel, default]
675+ - arches: [arm64, armel, armhf]
676+ failsafe:
677+ primary: http://ports.ubuntu.com/ubuntu-ports
678+ security: http://ports.ubuntu.com/ubuntu-ports
679+ search:
680+ primary:
681+ - http://%(ec2_region)s.ec2.ports.ubuntu.com/ubuntu-ports/
682+ - http://%(availability_zone)s.clouds.ports.ubuntu.com/ubuntu-ports/
683+ - http://%(region)s.clouds.ports.ubuntu.com/ubuntu-ports/
684+ security: []
685+ - arches: [default]
686 failsafe:
687 primary: http://ports.ubuntu.com/ubuntu-ports
688 security: http://ports.ubuntu.com/ubuntu-ports
689diff --git a/debian/changelog b/debian/changelog
690index 283bcd8..09a0034 100644
691--- a/debian/changelog
692+++ b/debian/changelog
693@@ -1,3 +1,19 @@
694+cloud-init (18.5-1-g5b065316-0ubuntu1) disco; urgency=medium
695+
696+ * New upstream snapshot.
697+ - Update to pylint 2.2.2.
698+ - Release 18.5 (LP: #1808380)
699+ - tests: add Disco release [Joshua Powers]
700+ - net: render 'metric' values in per-subnet routes (LP: #1805871)
701+ - write_files: add support for appending to files. [James Baxter]
702+ - config: On ubuntu select cloud archive mirrors for armel, armhf, arm64.
703+ (LP: #1805854)
704+ - dhclient-hook: cleanups, tests and fix a bug on 'down' event.
705+ - NoCloud: Allow top level 'network' key in network-config. (LP: #1798117)
706+ - ovf: Fix ovf network config generation gateway/routes (LP: #1806103)
707+
708+ -- Ryan Harper <ryan.harper@canonical.com> Fri, 14 Dec 2018 14:45:46 -0600
709+
710 cloud-init (18.4-31-gbf791715-0ubuntu1) disco; urgency=medium
711
712 * New upstream snapshot.
713diff --git a/tests/cloud_tests/releases.yaml b/tests/cloud_tests/releases.yaml
714index defae02..ec5da72 100644
715--- a/tests/cloud_tests/releases.yaml
716+++ b/tests/cloud_tests/releases.yaml
717@@ -129,6 +129,22 @@ features:
718
719 releases:
720 # UBUNTU =================================================================
721+ disco:
722+ # EOL: Jan 2020
723+ default:
724+ enabled: true
725+ release: disco
726+ version: 19.04
727+ os: ubuntu
728+ feature_groups:
729+ - base
730+ - debian_base
731+ - ubuntu_specific
732+ lxd:
733+ sstreams_server: https://cloud-images.ubuntu.com/daily
734+ alias: disco
735+ setup_overrides: null
736+ override_templates: false
737 cosmic:
738 # EOL: Jul 2019
739 default:
740diff --git a/tests/unittests/test_cli.py b/tests/unittests/test_cli.py
741index 199d69b..d283f13 100644
742--- a/tests/unittests/test_cli.py
743+++ b/tests/unittests/test_cli.py
744@@ -246,18 +246,18 @@ class TestCLI(test_helpers.FilesystemMockingTestCase):
745 self.assertEqual('cc_ntp', parseargs.name)
746 self.assertFalse(parseargs.report)
747
748- @mock.patch('cloudinit.cmd.main.dhclient_hook')
749- def test_dhclient_hook_subcommand(self, m_dhclient_hook):
750+ @mock.patch('cloudinit.cmd.main.dhclient_hook.handle_args')
751+ def test_dhclient_hook_subcommand(self, m_handle_args):
752 """The subcommand 'dhclient-hook' calls dhclient_hook with args."""
753- self._call_main(['cloud-init', 'dhclient-hook', 'net_action', 'eth0'])
754- (name, parseargs) = m_dhclient_hook.call_args_list[0][0]
755- self.assertEqual('dhclient_hook', name)
756+ self._call_main(['cloud-init', 'dhclient-hook', 'up', 'eth0'])
757+ (name, parseargs) = m_handle_args.call_args_list[0][0]
758+ self.assertEqual('dhclient-hook', name)
759 self.assertEqual('dhclient-hook', parseargs.subcommand)
760- self.assertEqual('dhclient_hook', parseargs.action[0])
761+ self.assertEqual('dhclient-hook', parseargs.action[0])
762 self.assertFalse(parseargs.debug)
763 self.assertFalse(parseargs.force)
764- self.assertEqual('net_action', parseargs.net_action)
765- self.assertEqual('eth0', parseargs.net_interface)
766+ self.assertEqual('up', parseargs.event)
767+ self.assertEqual('eth0', parseargs.interface)
768
769 @mock.patch('cloudinit.cmd.main.main_features')
770 def test_features_hook_subcommand(self, m_features):
771diff --git a/tests/unittests/test_datasource/test_nocloud.py b/tests/unittests/test_datasource/test_nocloud.py
772index b6468b6..3429272 100644
773--- a/tests/unittests/test_datasource/test_nocloud.py
774+++ b/tests/unittests/test_datasource/test_nocloud.py
775@@ -1,7 +1,10 @@
776 # This file is part of cloud-init. See LICENSE file for license information.
777
778 from cloudinit import helpers
779-from cloudinit.sources import DataSourceNoCloud
780+from cloudinit.sources.DataSourceNoCloud import (
781+ DataSourceNoCloud as dsNoCloud,
782+ _maybe_remove_top_network,
783+ parse_cmdline_data)
784 from cloudinit import util
785 from cloudinit.tests.helpers import CiTestCase, populate_dir, mock, ExitStack
786
787@@ -40,9 +43,7 @@ class TestNoCloudDataSource(CiTestCase):
788 'datasource': {'NoCloud': {'fs_label': None}}
789 }
790
791- ds = DataSourceNoCloud.DataSourceNoCloud
792-
793- dsrc = ds(sys_cfg=sys_cfg, distro=None, paths=self.paths)
794+ dsrc = dsNoCloud(sys_cfg=sys_cfg, distro=None, paths=self.paths)
795 ret = dsrc.get_data()
796 self.assertEqual(dsrc.userdata_raw, ud)
797 self.assertEqual(dsrc.metadata, md)
798@@ -63,9 +64,7 @@ class TestNoCloudDataSource(CiTestCase):
799 'datasource': {'NoCloud': {'fs_label': None}}
800 }
801
802- ds = DataSourceNoCloud.DataSourceNoCloud
803-
804- dsrc = ds(sys_cfg=sys_cfg, distro=None, paths=self.paths)
805+ dsrc = dsNoCloud(sys_cfg=sys_cfg, distro=None, paths=self.paths)
806 self.assertTrue(dsrc.get_data())
807 self.assertEqual(dsrc.platform_type, 'nocloud')
808 self.assertEqual(
809@@ -73,8 +72,6 @@ class TestNoCloudDataSource(CiTestCase):
810
811 def test_fs_label(self, m_is_lxd):
812 # find_devs_with should not be called ff fs_label is None
813- ds = DataSourceNoCloud.DataSourceNoCloud
814-
815 class PsuedoException(Exception):
816 pass
817
818@@ -84,12 +81,12 @@ class TestNoCloudDataSource(CiTestCase):
819
820 # by default, NoCloud should search for filesystems by label
821 sys_cfg = {'datasource': {'NoCloud': {}}}
822- dsrc = ds(sys_cfg=sys_cfg, distro=None, paths=self.paths)
823+ dsrc = dsNoCloud(sys_cfg=sys_cfg, distro=None, paths=self.paths)
824 self.assertRaises(PsuedoException, dsrc.get_data)
825
826 # but disabling searching should just end up with None found
827 sys_cfg = {'datasource': {'NoCloud': {'fs_label': None}}}
828- dsrc = ds(sys_cfg=sys_cfg, distro=None, paths=self.paths)
829+ dsrc = dsNoCloud(sys_cfg=sys_cfg, distro=None, paths=self.paths)
830 ret = dsrc.get_data()
831 self.assertFalse(ret)
832
833@@ -97,13 +94,10 @@ class TestNoCloudDataSource(CiTestCase):
834 # no source should be found if no cmdline, config, and fs_label=None
835 sys_cfg = {'datasource': {'NoCloud': {'fs_label': None}}}
836
837- ds = DataSourceNoCloud.DataSourceNoCloud
838- dsrc = ds(sys_cfg=sys_cfg, distro=None, paths=self.paths)
839+ dsrc = dsNoCloud(sys_cfg=sys_cfg, distro=None, paths=self.paths)
840 self.assertFalse(dsrc.get_data())
841
842 def test_seed_in_config(self, m_is_lxd):
843- ds = DataSourceNoCloud.DataSourceNoCloud
844-
845 data = {
846 'fs_label': None,
847 'meta-data': yaml.safe_dump({'instance-id': 'IID'}),
848@@ -111,7 +105,7 @@ class TestNoCloudDataSource(CiTestCase):
849 }
850
851 sys_cfg = {'datasource': {'NoCloud': data}}
852- dsrc = ds(sys_cfg=sys_cfg, distro=None, paths=self.paths)
853+ dsrc = dsNoCloud(sys_cfg=sys_cfg, distro=None, paths=self.paths)
854 ret = dsrc.get_data()
855 self.assertEqual(dsrc.userdata_raw, b"USER_DATA_RAW")
856 self.assertEqual(dsrc.metadata.get('instance-id'), 'IID')
857@@ -130,9 +124,7 @@ class TestNoCloudDataSource(CiTestCase):
858 'datasource': {'NoCloud': {'fs_label': None}}
859 }
860
861- ds = DataSourceNoCloud.DataSourceNoCloud
862-
863- dsrc = ds(sys_cfg=sys_cfg, distro=None, paths=self.paths)
864+ dsrc = dsNoCloud(sys_cfg=sys_cfg, distro=None, paths=self.paths)
865 ret = dsrc.get_data()
866 self.assertEqual(dsrc.userdata_raw, ud)
867 self.assertEqual(dsrc.metadata, md)
868@@ -145,9 +137,7 @@ class TestNoCloudDataSource(CiTestCase):
869
870 sys_cfg = {'datasource': {'NoCloud': {'fs_label': None}}}
871
872- ds = DataSourceNoCloud.DataSourceNoCloud
873-
874- dsrc = ds(sys_cfg=sys_cfg, distro=None, paths=self.paths)
875+ dsrc = dsNoCloud(sys_cfg=sys_cfg, distro=None, paths=self.paths)
876 ret = dsrc.get_data()
877 self.assertEqual(dsrc.userdata_raw, b"ud")
878 self.assertFalse(dsrc.vendordata)
879@@ -174,9 +164,7 @@ class TestNoCloudDataSource(CiTestCase):
880
881 sys_cfg = {'datasource': {'NoCloud': {'fs_label': None}}}
882
883- ds = DataSourceNoCloud.DataSourceNoCloud
884-
885- dsrc = ds(sys_cfg=sys_cfg, distro=None, paths=self.paths)
886+ dsrc = dsNoCloud(sys_cfg=sys_cfg, distro=None, paths=self.paths)
887 ret = dsrc.get_data()
888 self.assertTrue(ret)
889 # very simple check just for the strings above
890@@ -195,9 +183,23 @@ class TestNoCloudDataSource(CiTestCase):
891
892 sys_cfg = {'datasource': {'NoCloud': {'fs_label': None}}}
893
894- ds = DataSourceNoCloud.DataSourceNoCloud
895+ dsrc = dsNoCloud(sys_cfg=sys_cfg, distro=None, paths=self.paths)
896+ ret = dsrc.get_data()
897+ self.assertTrue(ret)
898+ self.assertEqual(netconf, dsrc.network_config)
899+
900+ def test_metadata_network_config_with_toplevel_network(self, m_is_lxd):
901+ """network-config may have 'network' top level key."""
902+ netconf = {'config': 'disabled'}
903+ populate_dir(
904+ os.path.join(self.paths.seed_dir, "nocloud"),
905+ {'user-data': b"ud",
906+ 'meta-data': "instance-id: IID\n",
907+ 'network-config': yaml.dump({'network': netconf}) + "\n"})
908+
909+ sys_cfg = {'datasource': {'NoCloud': {'fs_label': None}}}
910
911- dsrc = ds(sys_cfg=sys_cfg, distro=None, paths=self.paths)
912+ dsrc = dsNoCloud(sys_cfg=sys_cfg, distro=None, paths=self.paths)
913 ret = dsrc.get_data()
914 self.assertTrue(ret)
915 self.assertEqual(netconf, dsrc.network_config)
916@@ -228,9 +230,7 @@ class TestNoCloudDataSource(CiTestCase):
917
918 sys_cfg = {'datasource': {'NoCloud': {'fs_label': None}}}
919
920- ds = DataSourceNoCloud.DataSourceNoCloud
921-
922- dsrc = ds(sys_cfg=sys_cfg, distro=None, paths=self.paths)
923+ dsrc = dsNoCloud(sys_cfg=sys_cfg, distro=None, paths=self.paths)
924 ret = dsrc.get_data()
925 self.assertTrue(ret)
926 self.assertEqual(netconf, dsrc.network_config)
927@@ -258,8 +258,7 @@ class TestParseCommandLineData(CiTestCase):
928 for (fmt, expected) in pairs:
929 fill = {}
930 cmdline = fmt % {'ds_id': ds_id}
931- ret = DataSourceNoCloud.parse_cmdline_data(ds_id=ds_id, fill=fill,
932- cmdline=cmdline)
933+ ret = parse_cmdline_data(ds_id=ds_id, fill=fill, cmdline=cmdline)
934 self.assertEqual(expected, fill)
935 self.assertTrue(ret)
936
937@@ -276,10 +275,43 @@ class TestParseCommandLineData(CiTestCase):
938
939 for cmdline in cmdlines:
940 fill = {}
941- ret = DataSourceNoCloud.parse_cmdline_data(ds_id=ds_id, fill=fill,
942- cmdline=cmdline)
943+ ret = parse_cmdline_data(ds_id=ds_id, fill=fill, cmdline=cmdline)
944 self.assertEqual(fill, {})
945 self.assertFalse(ret)
946
947
948+class TestMaybeRemoveToplevelNetwork(CiTestCase):
949+ """test _maybe_remove_top_network function."""
950+ basecfg = [{'type': 'physical', 'name': 'interface0',
951+ 'subnets': [{'type': 'dhcp'}]}]
952+
953+ def test_should_remove_safely(self):
954+ mcfg = {'config': self.basecfg, 'version': 1}
955+ self.assertEqual(mcfg, _maybe_remove_top_network({'network': mcfg}))
956+
957+ def test_no_remove_if_other_keys(self):
958+ """should not shift if other keys at top level."""
959+ mcfg = {'network': {'config': self.basecfg, 'version': 1},
960+ 'unknown_keyname': 'keyval'}
961+ self.assertEqual(mcfg, _maybe_remove_top_network(mcfg))
962+
963+ def test_no_remove_if_non_dict(self):
964+ """should not shift if not a dict."""
965+ mcfg = {'network': '"content here'}
966+ self.assertEqual(mcfg, _maybe_remove_top_network(mcfg))
967+
968+ def test_no_remove_if_missing_config_or_version(self):
969+ """should not shift unless network entry has config and version."""
970+ mcfg = {'network': {'config': self.basecfg}}
971+ self.assertEqual(mcfg, _maybe_remove_top_network(mcfg))
972+
973+ mcfg = {'network': {'version': 1}}
974+ self.assertEqual(mcfg, _maybe_remove_top_network(mcfg))
975+
976+ def test_remove_with_config_disabled(self):
977+ """network/config=disabled should be shifted."""
978+ mcfg = {'config': 'disabled'}
979+ self.assertEqual(mcfg, _maybe_remove_top_network({'network': mcfg}))
980+
981+
982 # vi: ts=4 expandtab
983diff --git a/tests/unittests/test_handler/test_handler_write_files.py b/tests/unittests/test_handler/test_handler_write_files.py
984index 7fa8fd2..bc8756c 100644
985--- a/tests/unittests/test_handler/test_handler_write_files.py
986+++ b/tests/unittests/test_handler/test_handler_write_files.py
987@@ -52,6 +52,18 @@ class TestWriteFiles(FilesystemMockingTestCase):
988 "test_simple", [{"content": expected, "path": filename}])
989 self.assertEqual(util.load_file(filename), expected)
990
991+ def test_append(self):
992+ self.patchUtils(self.tmp)
993+ existing = "hello "
994+ added = "world\n"
995+ expected = existing + added
996+ filename = "/tmp/append.file"
997+ util.write_file(filename, existing)
998+ write_files(
999+ "test_append",
1000+ [{"content": added, "path": filename, "append": "true"}])
1001+ self.assertEqual(util.load_file(filename), expected)
1002+
1003 def test_yaml_binary(self):
1004 self.patchUtils(self.tmp)
1005 data = util.load_yaml(YAML_TEXT)
1006diff --git a/tests/unittests/test_net.py b/tests/unittests/test_net.py
1007index 8e38373..195f261 100644
1008--- a/tests/unittests/test_net.py
1009+++ b/tests/unittests/test_net.py
1010@@ -488,8 +488,8 @@ NETWORK_CONFIGS = {
1011 address 192.168.21.3/24
1012 dns-nameservers 8.8.8.8 8.8.4.4
1013 dns-search barley.maas sach.maas
1014- post-up route add default gw 65.61.151.37 || true
1015- pre-down route del default gw 65.61.151.37 || true
1016+ post-up route add default gw 65.61.151.37 metric 10000 || true
1017+ pre-down route del default gw 65.61.151.37 metric 10000 || true
1018 """).rstrip(' '),
1019 'expected_netplan': textwrap.dedent("""
1020 network:
1021@@ -513,7 +513,8 @@ NETWORK_CONFIGS = {
1022 - barley.maas
1023 - sach.maas
1024 routes:
1025- - to: 0.0.0.0/0
1026+ - metric: 10000
1027+ to: 0.0.0.0/0
1028 via: 65.61.151.37
1029 set-name: eth99
1030 """).rstrip(' '),
1031@@ -537,6 +538,7 @@ NETWORK_CONFIGS = {
1032 HWADDR=c0:d6:9f:2c:e8:80
1033 IPADDR=192.168.21.3
1034 NETMASK=255.255.255.0
1035+ METRIC=10000
1036 NM_CONTROLLED=no
1037 ONBOOT=yes
1038 TYPE=Ethernet
1039@@ -561,7 +563,7 @@ NETWORK_CONFIGS = {
1040 - gateway: 65.61.151.37
1041 netmask: 0.0.0.0
1042 network: 0.0.0.0
1043- metric: 2
1044+ metric: 10000
1045 - type: physical
1046 name: eth1
1047 mac_address: "cf:d6:af:48:e8:80"
1048@@ -1161,6 +1163,13 @@ pre-down route del -net 10.0.0.0 netmask 255.0.0.0 gw 11.0.0.1 metric 3 || true
1049 - gateway: 192.168.0.3
1050 netmask: 255.255.255.0
1051 network: 10.1.3.0
1052+ - gateway: 2001:67c:1562:1
1053+ network: 2001:67c:1
1054+ netmask: ffff:ffff:0
1055+ - gateway: 3001:67c:1562:1
1056+ network: 3001:67c:1
1057+ netmask: ffff:ffff:0
1058+ metric: 10000
1059 - type: static
1060 address: 192.168.1.2/24
1061 - type: static
1062@@ -1197,6 +1206,11 @@ pre-down route del -net 10.0.0.0 netmask 255.0.0.0 gw 11.0.0.1 metric 3 || true
1063 routes:
1064 - to: 10.1.3.0/24
1065 via: 192.168.0.3
1066+ - to: 2001:67c:1/32
1067+ via: 2001:67c:1562:1
1068+ - metric: 10000
1069+ to: 3001:67c:1/32
1070+ via: 3001:67c:1562:1
1071 """),
1072 'yaml-v2': textwrap.dedent("""
1073 version: 2
1074@@ -1228,6 +1242,11 @@ pre-down route del -net 10.0.0.0 netmask 255.0.0.0 gw 11.0.0.1 metric 3 || true
1075 routes:
1076 - to: 10.1.3.0/24
1077 via: 192.168.0.3
1078+ - to: 2001:67c:1562:8007::1/64
1079+ via: 2001:67c:1562:8007::aac:40b2
1080+ - metric: 10000
1081+ to: 3001:67c:1562:8007::1/64
1082+ via: 3001:67c:1562:8007::aac:40b2
1083 """),
1084 'expected_netplan-v2': textwrap.dedent("""
1085 network:
1086@@ -1249,6 +1268,11 @@ pre-down route del -net 10.0.0.0 netmask 255.0.0.0 gw 11.0.0.1 metric 3 || true
1087 routes:
1088 - to: 10.1.3.0/24
1089 via: 192.168.0.3
1090+ - to: 2001:67c:1562:8007::1/64
1091+ via: 2001:67c:1562:8007::aac:40b2
1092+ - metric: 10000
1093+ to: 3001:67c:1562:8007::1/64
1094+ via: 3001:67c:1562:8007::aac:40b2
1095 ethernets:
1096 eth0:
1097 match:
1098@@ -1349,6 +1373,10 @@ pre-down route del -net 10.0.0.0 netmask 255.0.0.0 gw 11.0.0.1 metric 3 || true
1099 USERCTL=no
1100 """),
1101 'route6-bond0': textwrap.dedent("""\
1102+ # Created by cloud-init on instance boot automatically, do not edit.
1103+ #
1104+ 2001:67c:1/ffff:ffff:0 via 2001:67c:1562:1 dev bond0
1105+ 3001:67c:1/ffff:ffff:0 via 3001:67c:1562:1 metric 10000 dev bond0
1106 """),
1107 'route-bond0': textwrap.dedent("""\
1108 ADDRESS0=10.1.3.0
1109@@ -1879,14 +1907,24 @@ class TestRhelSysConfigRendering(CiTestCase):
1110 return dir2dict(dir)
1111
1112 def _compare_files_to_expected(self, expected, found):
1113+
1114+ def _try_load(f):
1115+ ''' Attempt to load shell content, otherwise return as-is '''
1116+ try:
1117+ return util.load_shell_content(f)
1118+ except ValueError:
1119+ pass
1120+ # route6- * files aren't shell content, but iproute2 params
1121+ return f
1122+
1123 orig_maxdiff = self.maxDiff
1124 expected_d = dict(
1125- (os.path.join(self.scripts_dir, k), util.load_shell_content(v))
1126+ (os.path.join(self.scripts_dir, k), _try_load(v))
1127 for k, v in expected.items())
1128
1129 # only compare the files in scripts_dir
1130 scripts_found = dict(
1131- (k, util.load_shell_content(v)) for k, v in found.items()
1132+ (k, _try_load(v)) for k, v in found.items()
1133 if k.startswith(self.scripts_dir))
1134 try:
1135 self.maxDiff = None
1136diff --git a/tests/unittests/test_vmware_config_file.py b/tests/unittests/test_vmware_config_file.py
1137index 602dedb..f47335e 100644
1138--- a/tests/unittests/test_vmware_config_file.py
1139+++ b/tests/unittests/test_vmware_config_file.py
1140@@ -263,7 +263,7 @@ class TestVmwareConfigFile(CiTestCase):
1141 nicConfigurator = NicConfigurator(config.nics, False)
1142 nics_cfg_list = nicConfigurator.generate()
1143
1144- self.assertEqual(5, len(nics_cfg_list), "number of elements")
1145+ self.assertEqual(2, len(nics_cfg_list), "number of elements")
1146
1147 nic1 = {'name': 'NIC1'}
1148 nic2 = {'name': 'NIC2'}
1149@@ -275,8 +275,6 @@ class TestVmwareConfigFile(CiTestCase):
1150 nic1.update(cfg)
1151 elif cfg.get('name') == nic2.get('name'):
1152 nic2.update(cfg)
1153- elif cfg_type == 'route':
1154- route_list.append(cfg)
1155
1156 self.assertEqual('physical', nic1.get('type'), 'type of NIC1')
1157 self.assertEqual('NIC1', nic1.get('name'), 'name of NIC1')
1158@@ -297,6 +295,9 @@ class TestVmwareConfigFile(CiTestCase):
1159 static6_subnet.append(subnet)
1160 else:
1161 self.assertEqual(True, False, 'Unknown type')
1162+ if 'route' in subnet:
1163+ for route in subnet.get('routes'):
1164+ route_list.append(route)
1165
1166 self.assertEqual(1, len(static_subnet), 'Number of static subnet')
1167 self.assertEqual(1, len(static6_subnet), 'Number of static6 subnet')
1168@@ -351,6 +352,8 @@ class TestVmwareConfigFile(CiTestCase):
1169 class TestVmwareNetConfig(CiTestCase):
1170 """Test conversion of vmware config to cloud-init config."""
1171
1172+ maxDiff = None
1173+
1174 def _get_NicConfigurator(self, text):
1175 fp = None
1176 try:
1177@@ -420,9 +423,52 @@ class TestVmwareNetConfig(CiTestCase):
1178 'mac_address': '00:50:56:a6:8c:08',
1179 'subnets': [
1180 {'control': 'auto', 'type': 'static',
1181- 'address': '10.20.87.154', 'netmask': '255.255.252.0'}]},
1182- {'type': 'route', 'destination': '10.20.84.0/22',
1183- 'gateway': '10.20.87.253', 'metric': 10000}],
1184+ 'address': '10.20.87.154', 'netmask': '255.255.252.0',
1185+ 'routes':
1186+ [{'type': 'route', 'destination': '10.20.84.0/22',
1187+ 'gateway': '10.20.87.253', 'metric': 10000}]}]}],
1188+ nc.generate())
1189+
1190+ def test_cust_non_primary_nic_with_gateway_(self):
1191+ """A customer non primary nic set can have a gateway."""
1192+ config = textwrap.dedent("""\
1193+ [NETWORK]
1194+ NETWORKING = yes
1195+ BOOTPROTO = dhcp
1196+ HOSTNAME = static-debug-vm
1197+ DOMAINNAME = cluster.local
1198+
1199+ [NIC-CONFIG]
1200+ NICS = NIC1
1201+
1202+ [NIC1]
1203+ MACADDR = 00:50:56:ac:d1:8a
1204+ ONBOOT = yes
1205+ IPv4_MODE = BACKWARDS_COMPATIBLE
1206+ BOOTPROTO = static
1207+ IPADDR = 100.115.223.75
1208+ NETMASK = 255.255.255.0
1209+ GATEWAY = 100.115.223.254
1210+
1211+
1212+ [DNS]
1213+ DNSFROMDHCP=no
1214+
1215+ NAMESERVER|1 = 8.8.8.8
1216+
1217+ [DATETIME]
1218+ UTC = yes
1219+ """)
1220+ nc = self._get_NicConfigurator(config)
1221+ self.assertEqual(
1222+ [{'type': 'physical', 'name': 'NIC1',
1223+ 'mac_address': '00:50:56:ac:d1:8a',
1224+ 'subnets': [
1225+ {'control': 'auto', 'type': 'static',
1226+ 'address': '100.115.223.75', 'netmask': '255.255.255.0',
1227+ 'routes':
1228+ [{'type': 'route', 'destination': '100.115.223.0/24',
1229+ 'gateway': '100.115.223.254', 'metric': 10000}]}]}],
1230 nc.generate())
1231
1232 def test_a_primary_nic_with_gateway(self):
1233diff --git a/tox.ini b/tox.ini
1234index 2fb3209..d983348 100644
1235--- a/tox.ini
1236+++ b/tox.ini
1237@@ -21,7 +21,7 @@ setenv =
1238 basepython = python3
1239 deps =
1240 # requirements
1241- pylint==1.8.1
1242+ pylint==2.2.2
1243 # test-requirements because unit tests are now present in cloudinit tree
1244 -r{toxinidir}/test-requirements.txt
1245 commands = {envpython} -m pylint {posargs:cloudinit tests tools}

Subscribers

People subscribed via source and target branches