Merge ~raharper/cloud-init:ubuntu/devel/newupstream-20181214 into cloud-init:ubuntu/devel

Proposed by Ryan Harper
Status: Merged
Merged at revision: 47a4d44a244aefb55e2a192bd069ae830c6d176e
Proposed branch: ~raharper/cloud-init:ubuntu/devel/newupstream-20181214
Merge into: cloud-init:ubuntu/devel
Diff against target: 1245 lines (+556/-145)
23 files modified
ChangeLog (+54/-0)
bash_completion/cloud-init (+4/-1)
cloudinit/cmd/devel/net_convert.py (+10/-5)
cloudinit/cmd/main.py (+4/-16)
cloudinit/config/cc_write_files.py (+6/-1)
cloudinit/dhclient_hook.py (+72/-38)
cloudinit/net/eni.py (+15/-14)
cloudinit/net/netplan.py (+3/-3)
cloudinit/net/sysconfig.py (+21/-4)
cloudinit/sources/DataSourceAzure.py (+2/-2)
cloudinit/sources/DataSourceNoCloud.py (+31/-1)
cloudinit/sources/helpers/vmware/imc/config_nic.py (+2/-3)
cloudinit/tests/test_dhclient_hook.py (+105/-0)
cloudinit/version.py (+1/-1)
config/cloud.cfg.tmpl (+11/-1)
debian/changelog (+16/-0)
tests/cloud_tests/releases.yaml (+16/-0)
tests/unittests/test_cli.py (+8/-8)
tests/unittests/test_datasource/test_nocloud.py (+66/-34)
tests/unittests/test_handler/test_handler_write_files.py (+12/-0)
tests/unittests/test_net.py (+44/-6)
tests/unittests/test_vmware_config_file.py (+52/-6)
tox.ini (+1/-1)
Reviewer Review Type Date Requested Status
Server Team CI bot continuous-integration Approve
cloud-init Commiters Pending
Review via email: mp+360957@code.launchpad.net

Commit message

cloud-init (18.5-1-g5b065316-0ubuntu1) disco; urgency=medium

  * New upstream snapshot.
    - Update to pylint 2.2.2.
    - Release 18.5 (LP: #1808380)
    - tests: add Disco release [Joshua Powers]
    - net: render 'metric' values in per-subnet routes (LP: #1805871)
    - write_files: add support for appending to files. [James Baxter]
    - config: On ubuntu select cloud archive mirrors for armel, armhf, arm64.
      (LP: #1805854)
    - dhclient-hook: cleanups, tests and fix a bug on 'down' event.
    - NoCloud: Allow top level 'network' key in network-config. (LP: #1798117)
    - ovf: Fix ovf network config generation gateway/routes (LP: #1806103)

 -- Ryan Harper <email address hidden> Fri, 14 Dec 2018 14:45:46 -0600

To post a comment you must log in.

PASSED: Continuous integration, rev:47a4d44a244aefb55e2a192bd069ae830c6d176e
https://jenkins.ubuntu.com/server/job/cloud-init-ci/490/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/490/rebuild

review: Approve (continuous-integration)

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
diff --git a/ChangeLog b/ChangeLog
index 9c043b0..8fa6fdd 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,57 @@
118.5:
2 - tests: add Disco release [Joshua Powers]
3 - net: render 'metric' values in per-subnet routes (LP: #1805871)
4 - write_files: add support for appending to files. [James Baxter]
5 - config: On ubuntu select cloud archive mirrors for armel, armhf, arm64.
6 (LP: #1805854)
7 - dhclient-hook: cleanups, tests and fix a bug on 'down' event.
8 - NoCloud: Allow top level 'network' key in network-config. (LP: #1798117)
9 - ovf: Fix ovf network config generation gateway/routes (LP: #1806103)
10 - azure: detect vnet migration via netlink media change event
11 [Tamilmani Manoharan]
12 - Azure: fix copy/paste error in error handling when reading azure ovf.
13 [Adam DePue]
14 - tests: fix incorrect order of mocks in test_handle_zfs_root.
15 - doc: Change dns_nameserver property to dns_nameservers. [Tomer Cohen]
16 - OVF: identify label iso9660 filesystems with label 'OVF ENV'.
17 - logs: collect-logs ignore instance-data-sensitive.json on non-root user
18 (LP: #1805201)
19 - net: Ephemeral*Network: add connectivity check via URL
20 - azure: _poll_imds only retry on 404. Fail on Timeout (LP: #1803598)
21 - resizefs: Prefix discovered devpath with '/dev/' when path does not
22 exist [Igor Galić]
23 - azure: retry imds polling on requests.Timeout (LP: #1800223)
24 - azure: Accept variation in error msg from mount for ntfs volumes
25 [Jason Zions] (LP: #1799338)
26 - azure: fix regression introduced when persisting ephemeral dhcp lease
27 [asakkurr]
28 - azure: add udev rules to create cloud-init Gen2 disk name symlinks
29 (LP: #1797480)
30 - tests: ec2 mock missing httpretty user-data and instance-identity routes
31 - azure: remove /etc/netplan/90-hotplug-azure.yaml when net from IMDS
32 - azure: report ready to fabric after reprovision and reduce logging
33 [asakkurr] (LP: #1799594)
34 - query: better error when missing read permission on instance-data
35 - instance-data: fallback to instance-data.json if sensitive is absent.
36 (LP: #1798189)
37 - docs: remove colon from network v1 config example. [Tomer Cohen]
38 - Add cloud-id binary to packages for SUSE [Jason Zions]
39 - systemd: On SUSE ensure cloud-init.service runs before wicked
40 [Robert Schweikert] (LP: #1799709)
41 - update detection of openSUSE variants [Robert Schweikert]
42 - azure: Add apply_network_config option to disable network from IMDS
43 (LP: #1798424)
44 - Correct spelling in an error message (udevadm). [Katie McLaughlin]
45 - tests: meta_data key changed to meta-data in ec2 instance-data.json
46 (LP: #1797231)
47 - tests: fix kvm integration test to assert flexible config-disk path
48 (LP: #1797199)
49 - tools: Add cloud-id command line utility
50 - instance-data: Add standard keys platform and subplatform. Refactor ec2.
51 - net: ignore nics that have "zero" mac address. (LP: #1796917)
52 - tests: fix apt_configure_primary to be more flexible
53 - Ubuntu: update sources.list to comment out deb-src entries. (LP: #74747)
54
118.4:5518.4:
2 - add rtd example docs about new standardized keys56 - add rtd example docs about new standardized keys
3 - use ds._crawled_metadata instance attribute if set when writing57 - use ds._crawled_metadata instance attribute if set when writing
diff --git a/bash_completion/cloud-init b/bash_completion/cloud-init
index 8c25032..a9577e9 100644
--- a/bash_completion/cloud-init
+++ b/bash_completion/cloud-init
@@ -30,7 +30,10 @@ _cloudinit_complete()
30 devel)30 devel)
31 COMPREPLY=($(compgen -W "--help schema net-convert" -- $cur_word))31 COMPREPLY=($(compgen -W "--help schema net-convert" -- $cur_word))
32 ;;32 ;;
33 dhclient-hook|features)33 dhclient-hook)
34 COMPREPLY=($(compgen -W "--help up down" -- $cur_word))
35 ;;
36 features)
34 COMPREPLY=($(compgen -W "--help" -- $cur_word))37 COMPREPLY=($(compgen -W "--help" -- $cur_word))
35 ;;38 ;;
36 init)39 init)
diff --git a/cloudinit/cmd/devel/net_convert.py b/cloudinit/cmd/devel/net_convert.py
index a0f58a0..1ad7e0b 100755
--- a/cloudinit/cmd/devel/net_convert.py
+++ b/cloudinit/cmd/devel/net_convert.py
@@ -9,6 +9,7 @@ import yaml
99
10from cloudinit.sources.helpers import openstack10from cloudinit.sources.helpers import openstack
11from cloudinit.sources import DataSourceAzure as azure11from cloudinit.sources import DataSourceAzure as azure
12from cloudinit.sources import DataSourceOVF as ovf
1213
13from cloudinit import distros14from cloudinit import distros
14from cloudinit.net import eni, netplan, network_state, sysconfig15from cloudinit.net import eni, netplan, network_state, sysconfig
@@ -31,7 +32,7 @@ def get_parser(parser=None):
31 metavar="PATH", required=True)32 metavar="PATH", required=True)
32 parser.add_argument("-k", "--kind",33 parser.add_argument("-k", "--kind",
33 choices=['eni', 'network_data.json', 'yaml',34 choices=['eni', 'network_data.json', 'yaml',
34 'azure-imds'],35 'azure-imds', 'vmware-imc'],
35 required=True)36 required=True)
36 parser.add_argument("-d", "--directory",37 parser.add_argument("-d", "--directory",
37 metavar="PATH",38 metavar="PATH",
@@ -76,7 +77,6 @@ def handle_args(name, args):
76 net_data = args.network_data.read()77 net_data = args.network_data.read()
77 if args.kind == "eni":78 if args.kind == "eni":
78 pre_ns = eni.convert_eni_data(net_data)79 pre_ns = eni.convert_eni_data(net_data)
79 ns = network_state.parse_net_config_data(pre_ns)
80 elif args.kind == "yaml":80 elif args.kind == "yaml":
81 pre_ns = yaml.load(net_data)81 pre_ns = yaml.load(net_data)
82 if 'network' in pre_ns:82 if 'network' in pre_ns:
@@ -85,15 +85,16 @@ def handle_args(name, args):
85 sys.stderr.write('\n'.join(85 sys.stderr.write('\n'.join(
86 ["Input YAML",86 ["Input YAML",
87 yaml.dump(pre_ns, default_flow_style=False, indent=4), ""]))87 yaml.dump(pre_ns, default_flow_style=False, indent=4), ""]))
88 ns = network_state.parse_net_config_data(pre_ns)
89 elif args.kind == 'network_data.json':88 elif args.kind == 'network_data.json':
90 pre_ns = openstack.convert_net_json(89 pre_ns = openstack.convert_net_json(
91 json.loads(net_data), known_macs=known_macs)90 json.loads(net_data), known_macs=known_macs)
92 ns = network_state.parse_net_config_data(pre_ns)
93 elif args.kind == 'azure-imds':91 elif args.kind == 'azure-imds':
94 pre_ns = azure.parse_network_config(json.loads(net_data))92 pre_ns = azure.parse_network_config(json.loads(net_data))
95 ns = network_state.parse_net_config_data(pre_ns)93 elif args.kind == 'vmware-imc':
94 config = ovf.Config(ovf.ConfigFile(args.network_data.name))
95 pre_ns = ovf.get_network_config_from_conf(config, False)
9696
97 ns = network_state.parse_net_config_data(pre_ns)
97 if not ns:98 if not ns:
98 raise RuntimeError("No valid network_state object created from"99 raise RuntimeError("No valid network_state object created from"
99 "input data")100 "input data")
@@ -111,6 +112,10 @@ def handle_args(name, args):
111 elif args.output_kind == "netplan":112 elif args.output_kind == "netplan":
112 r_cls = netplan.Renderer113 r_cls = netplan.Renderer
113 config = distro.renderer_configs.get('netplan')114 config = distro.renderer_configs.get('netplan')
115 # don't run netplan generate/apply
116 config['postcmds'] = False
117 # trim leading slash
118 config['netplan_path'] = config['netplan_path'][1:]
114 else:119 else:
115 r_cls = sysconfig.Renderer120 r_cls = sysconfig.Renderer
116 config = distro.renderer_configs.get('sysconfig')121 config = distro.renderer_configs.get('sysconfig')
diff --git a/cloudinit/cmd/main.py b/cloudinit/cmd/main.py
index 5a43702..933c019 100644
--- a/cloudinit/cmd/main.py
+++ b/cloudinit/cmd/main.py
@@ -41,7 +41,7 @@ from cloudinit.settings import (PER_INSTANCE, PER_ALWAYS, PER_ONCE,
41from cloudinit import atomic_helper41from cloudinit import atomic_helper
4242
43from cloudinit.config import cc_set_hostname43from cloudinit.config import cc_set_hostname
44from cloudinit.dhclient_hook import LogDhclient44from cloudinit import dhclient_hook
4545
4646
47# Welcome message template47# Welcome message template
@@ -586,12 +586,6 @@ def main_single(name, args):
586 return 0586 return 0
587587
588588
589def dhclient_hook(name, args):
590 record = LogDhclient(args)
591 record.check_hooks_dir()
592 record.record()
593
594
595def status_wrapper(name, args, data_d=None, link_d=None):589def status_wrapper(name, args, data_d=None, link_d=None):
596 if data_d is None:590 if data_d is None:
597 data_d = os.path.normpath("/var/lib/cloud/data")591 data_d = os.path.normpath("/var/lib/cloud/data")
@@ -795,15 +789,9 @@ def main(sysv_args=None):
795 'query',789 'query',
796 help='Query standardized instance metadata from the command line.')790 help='Query standardized instance metadata from the command line.')
797791
798 parser_dhclient = subparsers.add_parser('dhclient-hook',792 parser_dhclient = subparsers.add_parser(
799 help=('run the dhclient hook'793 dhclient_hook.NAME, help=dhclient_hook.__doc__)
800 'to record network info'))794 dhclient_hook.get_parser(parser_dhclient)
801 parser_dhclient.add_argument("net_action",
802 help=('action taken on the interface'))
803 parser_dhclient.add_argument("net_interface",
804 help=('the network interface being acted'
805 ' upon'))
806 parser_dhclient.set_defaults(action=('dhclient_hook', dhclient_hook))
807795
808 parser_features = subparsers.add_parser('features',796 parser_features = subparsers.add_parser('features',
809 help=('list defined features'))797 help=('list defined features'))
diff --git a/cloudinit/config/cc_write_files.py b/cloudinit/config/cc_write_files.py
index 31d1db6..0b6546e 100644
--- a/cloudinit/config/cc_write_files.py
+++ b/cloudinit/config/cc_write_files.py
@@ -49,6 +49,10 @@ binary gzip data can be specified and will be decoded before being written.
49 ...49 ...
50 path: /bin/arch50 path: /bin/arch
51 permissions: '0555'51 permissions: '0555'
52 - content: |
53 15 * * * * root ship_logs
54 path: /etc/crontab
55 append: true
52"""56"""
5357
54import base6458import base64
@@ -113,7 +117,8 @@ def write_files(name, files):
113 contents = extract_contents(f_info.get('content', ''), extractions)117 contents = extract_contents(f_info.get('content', ''), extractions)
114 (u, g) = util.extract_usergroup(f_info.get('owner', DEFAULT_OWNER))118 (u, g) = util.extract_usergroup(f_info.get('owner', DEFAULT_OWNER))
115 perms = decode_perms(f_info.get('permissions'), DEFAULT_PERMS)119 perms = decode_perms(f_info.get('permissions'), DEFAULT_PERMS)
116 util.write_file(path, contents, mode=perms)120 omode = 'ab' if util.get_cfg_option_bool(f_info, 'append') else 'wb'
121 util.write_file(path, contents, omode=omode, mode=perms)
117 util.chownbyname(path, u, g)122 util.chownbyname(path, u, g)
118123
119124
diff --git a/cloudinit/dhclient_hook.py b/cloudinit/dhclient_hook.py
index 7f02d7f..72b51b6 100644
--- a/cloudinit/dhclient_hook.py
+++ b/cloudinit/dhclient_hook.py
@@ -1,5 +1,8 @@
1# This file is part of cloud-init. See LICENSE file for license information.1# This file is part of cloud-init. See LICENSE file for license information.
22
3"""Run the dhclient hook to record network info."""
4
5import argparse
3import os6import os
47
5from cloudinit import atomic_helper8from cloudinit import atomic_helper
@@ -8,44 +11,75 @@ from cloudinit import stages
811
9LOG = logging.getLogger(__name__)12LOG = logging.getLogger(__name__)
1013
14NAME = "dhclient-hook"
15UP = "up"
16DOWN = "down"
17EVENTS = (UP, DOWN)
18
19
20def _get_hooks_dir():
21 i = stages.Init()
22 return os.path.join(i.paths.get_runpath(), 'dhclient.hooks')
23
24
25def _filter_env_vals(info):
26 """Given info (os.environ), return a dictionary with
27 lower case keys for each entry starting with DHCP4_ or new_."""
28 new_info = {}
29 for k, v in info.items():
30 if k.startswith("DHCP4_") or k.startswith("new_"):
31 key = (k.replace('DHCP4_', '').replace('new_', '')).lower()
32 new_info[key] = v
33 return new_info
34
35
36def run_hook(interface, event, data_d=None, env=None):
37 if event not in EVENTS:
38 raise ValueError("Unexpected event '%s'. Expected one of: %s" %
39 (event, EVENTS))
40 if data_d is None:
41 data_d = _get_hooks_dir()
42 if env is None:
43 env = os.environ
44 hook_file = os.path.join(data_d, interface + ".json")
45
46 if event == UP:
47 if not os.path.exists(data_d):
48 os.makedirs(data_d)
49 atomic_helper.write_json(hook_file, _filter_env_vals(env))
50 LOG.debug("Wrote dhclient options in %s", hook_file)
51 elif event == DOWN:
52 if os.path.exists(hook_file):
53 os.remove(hook_file)
54 LOG.debug("Removed dhclient options file %s", hook_file)
55
56
57def get_parser(parser=None):
58 if parser is None:
59 parser = argparse.ArgumentParser(prog=NAME, description=__doc__)
60 parser.add_argument(
61 "event", help='event taken on the interface', choices=EVENTS)
62 parser.add_argument(
63 "interface", help='the network interface being acted upon')
64 # cloud-init main uses 'action'
65 parser.set_defaults(action=(NAME, handle_args))
66 return parser
67
68
69def handle_args(name, args, data_d=None):
70 """Handle the Namespace args.
71 Takes 'name' as passed by cloud-init main. not used here."""
72 return run_hook(interface=args.interface, event=args.event, data_d=data_d)
73
74
75if __name__ == '__main__':
76 import sys
77 parser = get_parser()
78 args = parser.parse_args(args=sys.argv[1:])
79 return_value = handle_args(
80 NAME, args, data_d=os.environ.get('_CI_DHCP_HOOK_DATA_D'))
81 if return_value:
82 sys.exit(return_value)
1183
12class LogDhclient(object):
13
14 def __init__(self, cli_args):
15 self.hooks_dir = self._get_hooks_dir()
16 self.net_interface = cli_args.net_interface
17 self.net_action = cli_args.net_action
18 self.hook_file = os.path.join(self.hooks_dir,
19 self.net_interface + ".json")
20
21 @staticmethod
22 def _get_hooks_dir():
23 i = stages.Init()
24 return os.path.join(i.paths.get_runpath(), 'dhclient.hooks')
25
26 def check_hooks_dir(self):
27 if not os.path.exists(self.hooks_dir):
28 os.makedirs(self.hooks_dir)
29 else:
30 # If the action is down and the json file exists, we need to
31 # delete the file
32 if self.net_action is 'down' and os.path.exists(self.hook_file):
33 os.remove(self.hook_file)
34
35 @staticmethod
36 def get_vals(info):
37 new_info = {}
38 for k, v in info.items():
39 if k.startswith("DHCP4_") or k.startswith("new_"):
40 key = (k.replace('DHCP4_', '').replace('new_', '')).lower()
41 new_info[key] = v
42 return new_info
43
44 def record(self):
45 envs = os.environ
46 if self.hook_file is None:
47 return
48 atomic_helper.write_json(self.hook_file, self.get_vals(envs))
49 LOG.debug("Wrote dhclient options in %s", self.hook_file)
5084
51# vi: ts=4 expandtab85# vi: ts=4 expandtab
diff --git a/cloudinit/net/eni.py b/cloudinit/net/eni.py
index c6f631a..6423632 100644
--- a/cloudinit/net/eni.py
+++ b/cloudinit/net/eni.py
@@ -371,22 +371,23 @@ class Renderer(renderer.Renderer):
371 'gateway': 'gw',371 'gateway': 'gw',
372 'metric': 'metric',372 'metric': 'metric',
373 }373 }
374
375 default_gw = ''
374 if route['network'] == '0.0.0.0' and route['netmask'] == '0.0.0.0':376 if route['network'] == '0.0.0.0' and route['netmask'] == '0.0.0.0':
375 default_gw = " default gw %s" % route['gateway']377 default_gw = ' default'
376 content.append(up + default_gw + or_true)
377 content.append(down + default_gw + or_true)
378 elif route['network'] == '::' and route['prefix'] == 0:378 elif route['network'] == '::' and route['prefix'] == 0:
379 # ipv6!379 default_gw = ' -A inet6 default'
380 default_gw = " -A inet6 default gw %s" % route['gateway']380
381 content.append(up + default_gw + or_true)381 route_line = ''
382 content.append(down + default_gw + or_true)382 for k in ['network', 'netmask', 'gateway', 'metric']:
383 else:383 if default_gw and k in ['network', 'netmask']:
384 route_line = ""384 continue
385 for k in ['network', 'netmask', 'gateway', 'metric']:385 if k == 'gateway':
386 if k in route:386 route_line += '%s %s %s' % (default_gw, mapping[k], route[k])
387 route_line += " %s %s" % (mapping[k], route[k])387 elif k in route:
388 content.append(up + route_line + or_true)388 route_line += ' %s %s' % (mapping[k], route[k])
389 content.append(down + route_line + or_true)389 content.append(up + route_line + or_true)
390 content.append(down + route_line + or_true)
390 return content391 return content
391392
392 def _render_iface(self, iface, render_hwaddress=False):393 def _render_iface(self, iface, render_hwaddress=False):
diff --git a/cloudinit/net/netplan.py b/cloudinit/net/netplan.py
index bc1087f..21517fd 100644
--- a/cloudinit/net/netplan.py
+++ b/cloudinit/net/netplan.py
@@ -114,13 +114,13 @@ def _extract_addresses(config, entry, ifname):
114 for route in subnet.get('routes', []):114 for route in subnet.get('routes', []):
115 to_net = "%s/%s" % (route.get('network'),115 to_net = "%s/%s" % (route.get('network'),
116 route.get('prefix'))116 route.get('prefix'))
117 route = {117 new_route = {
118 'via': route.get('gateway'),118 'via': route.get('gateway'),
119 'to': to_net,119 'to': to_net,
120 }120 }
121 if 'metric' in route:121 if 'metric' in route:
122 route.update({'metric': route.get('metric', 100)})122 new_route.update({'metric': route.get('metric', 100)})
123 routes.append(route)123 routes.append(new_route)
124124
125 addresses.append(addr)125 addresses.append(addr)
126126
diff --git a/cloudinit/net/sysconfig.py b/cloudinit/net/sysconfig.py
index 9c16d3a..17293e1 100644
--- a/cloudinit/net/sysconfig.py
+++ b/cloudinit/net/sysconfig.py
@@ -156,13 +156,23 @@ class Route(ConfigMap):
156 _quote_value(gateway_value)))156 _quote_value(gateway_value)))
157 buf.write("%s=%s\n" % ('NETMASK' + str(reindex),157 buf.write("%s=%s\n" % ('NETMASK' + str(reindex),
158 _quote_value(netmask_value)))158 _quote_value(netmask_value)))
159 metric_key = 'METRIC' + index
160 if metric_key in self._conf:
161 metric_value = str(self._conf['METRIC' + index])
162 buf.write("%s=%s\n" % ('METRIC' + str(reindex),
163 _quote_value(metric_value)))
159 elif proto == "ipv6" and self.is_ipv6_route(address_value):164 elif proto == "ipv6" and self.is_ipv6_route(address_value):
160 netmask_value = str(self._conf['NETMASK' + index])165 netmask_value = str(self._conf['NETMASK' + index])
161 gateway_value = str(self._conf['GATEWAY' + index])166 gateway_value = str(self._conf['GATEWAY' + index])
162 buf.write("%s/%s via %s dev %s\n" % (address_value,167 metric_value = (
163 netmask_value,168 'metric ' + str(self._conf['METRIC' + index])
164 gateway_value,169 if 'METRIC' + index in self._conf else '')
165 self._route_name))170 buf.write(
171 "%s/%s via %s %s dev %s\n" % (address_value,
172 netmask_value,
173 gateway_value,
174 metric_value,
175 self._route_name))
166176
167 return buf.getvalue()177 return buf.getvalue()
168178
@@ -370,6 +380,9 @@ class Renderer(renderer.Renderer):
370 else:380 else:
371 iface_cfg['GATEWAY'] = subnet['gateway']381 iface_cfg['GATEWAY'] = subnet['gateway']
372382
383 if 'metric' in subnet:
384 iface_cfg['METRIC'] = subnet['metric']
385
373 if 'dns_search' in subnet:386 if 'dns_search' in subnet:
374 iface_cfg['DOMAIN'] = ' '.join(subnet['dns_search'])387 iface_cfg['DOMAIN'] = ' '.join(subnet['dns_search'])
375388
@@ -414,15 +427,19 @@ class Renderer(renderer.Renderer):
414 else:427 else:
415 iface_cfg['GATEWAY'] = route['gateway']428 iface_cfg['GATEWAY'] = route['gateway']
416 route_cfg.has_set_default_ipv4 = True429 route_cfg.has_set_default_ipv4 = True
430 if 'metric' in route:
431 iface_cfg['METRIC'] = route['metric']
417432
418 else:433 else:
419 gw_key = 'GATEWAY%s' % route_cfg.last_idx434 gw_key = 'GATEWAY%s' % route_cfg.last_idx
420 nm_key = 'NETMASK%s' % route_cfg.last_idx435 nm_key = 'NETMASK%s' % route_cfg.last_idx
421 addr_key = 'ADDRESS%s' % route_cfg.last_idx436 addr_key = 'ADDRESS%s' % route_cfg.last_idx
437 metric_key = 'METRIC%s' % route_cfg.last_idx
422 route_cfg.last_idx += 1438 route_cfg.last_idx += 1
423 # add default routes only to ifcfg files, not439 # add default routes only to ifcfg files, not
424 # to route-* or route6-*440 # to route-* or route6-*
425 for (old_key, new_key) in [('gateway', gw_key),441 for (old_key, new_key) in [('gateway', gw_key),
442 ('metric', metric_key),
426 ('netmask', nm_key),443 ('netmask', nm_key),
427 ('network', addr_key)]:444 ('network', addr_key)]:
428 if old_key in route:445 if old_key in route:
diff --git a/cloudinit/sources/DataSourceAzure.py b/cloudinit/sources/DataSourceAzure.py
index e076d5d..46efca4 100644
--- a/cloudinit/sources/DataSourceAzure.py
+++ b/cloudinit/sources/DataSourceAzure.py
@@ -980,8 +980,8 @@ def read_azure_ovf(contents):
980 raise NonAzureDataSource("No LinuxProvisioningConfigurationSet")980 raise NonAzureDataSource("No LinuxProvisioningConfigurationSet")
981 if len(lpcs_nodes) > 1:981 if len(lpcs_nodes) > 1:
982 raise BrokenAzureDataSource("found '%d' %ss" %982 raise BrokenAzureDataSource("found '%d' %ss" %
983 ("LinuxProvisioningConfigurationSet",983 (len(lpcs_nodes),
984 len(lpcs_nodes)))984 "LinuxProvisioningConfigurationSet"))
985 lpcs = lpcs_nodes[0]985 lpcs = lpcs_nodes[0]
986986
987 if not lpcs.hasChildNodes():987 if not lpcs.hasChildNodes():
diff --git a/cloudinit/sources/DataSourceNoCloud.py b/cloudinit/sources/DataSourceNoCloud.py
index 9010f06..6860f0c 100644
--- a/cloudinit/sources/DataSourceNoCloud.py
+++ b/cloudinit/sources/DataSourceNoCloud.py
@@ -311,6 +311,35 @@ def parse_cmdline_data(ds_id, fill, cmdline=None):
311 return True311 return True
312312
313313
314def _maybe_remove_top_network(cfg):
315 """If network-config contains top level 'network' key, then remove it.
316
317 Some providers of network configuration may provide a top level
318 'network' key (LP: #1798117) even though it is not necessary.
319
320 Be friendly and remove it if it really seems so.
321
322 Return the original value if no change or the updated value if changed."""
323 nullval = object()
324 network_val = cfg.get('network', nullval)
325 if network_val is nullval:
326 return cfg
327 bmsg = 'Top level network key in network-config %s: %s'
328 if not isinstance(network_val, dict):
329 LOG.debug(bmsg, "was not a dict", cfg)
330 return cfg
331 if len(list(cfg.keys())) != 1:
332 LOG.debug(bmsg, "had multiple top level keys", cfg)
333 return cfg
334 if network_val.get('config') == "disabled":
335 LOG.debug(bmsg, "was config/disabled", cfg)
336 elif not all(('config' in network_val, 'version' in network_val)):
337 LOG.debug(bmsg, "but missing 'config' or 'version'", cfg)
338 return cfg
339 LOG.debug(bmsg, "fixed by removing shifting network.", cfg)
340 return network_val
341
342
314def _merge_new_seed(cur, seeded):343def _merge_new_seed(cur, seeded):
315 ret = cur.copy()344 ret = cur.copy()
316345
@@ -320,7 +349,8 @@ def _merge_new_seed(cur, seeded):
320 ret['meta-data'] = util.mergemanydict([cur['meta-data'], newmd])349 ret['meta-data'] = util.mergemanydict([cur['meta-data'], newmd])
321350
322 if seeded.get('network-config'):351 if seeded.get('network-config'):
323 ret['network-config'] = util.load_yaml(seeded['network-config'])352 ret['network-config'] = _maybe_remove_top_network(
353 util.load_yaml(seeded.get('network-config')))
324354
325 if 'user-data' in seeded:355 if 'user-data' in seeded:
326 ret['user-data'] = seeded['user-data']356 ret['user-data'] = seeded['user-data']
diff --git a/cloudinit/sources/helpers/vmware/imc/config_nic.py b/cloudinit/sources/helpers/vmware/imc/config_nic.py
index e1890e2..77cbf3b 100644
--- a/cloudinit/sources/helpers/vmware/imc/config_nic.py
+++ b/cloudinit/sources/helpers/vmware/imc/config_nic.py
@@ -165,9 +165,8 @@ class NicConfigurator(object):
165165
166 # Add routes if there is no primary nic166 # Add routes if there is no primary nic
167 if not self._primaryNic and v4.gateways:167 if not self._primaryNic and v4.gateways:
168 route_list.extend(self.gen_ipv4_route(nic,168 subnet.update(
169 v4.gateways,169 {'routes': self.gen_ipv4_route(nic, v4.gateways, v4.netmask)})
170 v4.netmask))
171170
172 return ([subnet], route_list)171 return ([subnet], route_list)
173172
diff --git a/cloudinit/tests/test_dhclient_hook.py b/cloudinit/tests/test_dhclient_hook.py
174new file mode 100644173new file mode 100644
index 0000000..7aab8dd
--- /dev/null
+++ b/cloudinit/tests/test_dhclient_hook.py
@@ -0,0 +1,105 @@
1# This file is part of cloud-init. See LICENSE file for license information.
2
3"""Tests for cloudinit.dhclient_hook."""
4
5from cloudinit import dhclient_hook as dhc
6from cloudinit.tests.helpers import CiTestCase, dir2dict, populate_dir
7
8import argparse
9import json
10import mock
11import os
12
13
14class TestDhclientHook(CiTestCase):
15
16 ex_env = {
17 'interface': 'eth0',
18 'new_dhcp_lease_time': '3600',
19 'new_host_name': 'x1',
20 'new_ip_address': '10.145.210.163',
21 'new_subnet_mask': '255.255.255.0',
22 'old_host_name': 'x1',
23 'PATH': '/usr/sbin:/usr/bin:/sbin:/bin',
24 'pid': '614',
25 'reason': 'BOUND',
26 }
27
28 # some older versions of dhclient put the same content,
29 # but in upper case with DHCP4_ instead of new_
30 ex_env_dhcp4 = {
31 'REASON': 'BOUND',
32 'DHCP4_dhcp_lease_time': '3600',
33 'DHCP4_host_name': 'x1',
34 'DHCP4_ip_address': '10.145.210.163',
35 'DHCP4_subnet_mask': '255.255.255.0',
36 'INTERFACE': 'eth0',
37 'PATH': '/usr/sbin:/usr/bin:/sbin:/bin',
38 'pid': '614',
39 }
40
41 expected = {
42 'dhcp_lease_time': '3600',
43 'host_name': 'x1',
44 'ip_address': '10.145.210.163',
45 'subnet_mask': '255.255.255.0'}
46
47 def setUp(self):
48 super(TestDhclientHook, self).setUp()
49 self.tmp = self.tmp_dir()
50
51 def test_handle_args(self):
52 """quick test of call to handle_args."""
53 nic = 'eth0'
54 args = argparse.Namespace(event=dhc.UP, interface=nic)
55 with mock.patch.dict("os.environ", clear=True, values=self.ex_env):
56 dhc.handle_args(dhc.NAME, args, data_d=self.tmp)
57 found = dir2dict(self.tmp + os.path.sep)
58 self.assertEqual([nic + ".json"], list(found.keys()))
59 self.assertEqual(self.expected, json.loads(found[nic + ".json"]))
60
61 def test_run_hook_up_creates_dir(self):
62 """If dir does not exist, run_hook should create it."""
63 subd = self.tmp_path("subdir", self.tmp)
64 nic = 'eth1'
65 dhc.run_hook(nic, 'up', data_d=subd, env=self.ex_env)
66 self.assertEqual(
67 set([nic + ".json"]), set(dir2dict(subd + os.path.sep)))
68
69 def test_run_hook_up(self):
70 """Test expected use of run_hook_up."""
71 nic = 'eth0'
72 dhc.run_hook(nic, 'up', data_d=self.tmp, env=self.ex_env)
73 found = dir2dict(self.tmp + os.path.sep)
74 self.assertEqual([nic + ".json"], list(found.keys()))
75 self.assertEqual(self.expected, json.loads(found[nic + ".json"]))
76
77 def test_run_hook_up_dhcp4_prefix(self):
78 """Test run_hook filters correctly with older DHCP4_ data."""
79 nic = 'eth0'
80 dhc.run_hook(nic, 'up', data_d=self.tmp, env=self.ex_env_dhcp4)
81 found = dir2dict(self.tmp + os.path.sep)
82 self.assertEqual([nic + ".json"], list(found.keys()))
83 self.assertEqual(self.expected, json.loads(found[nic + ".json"]))
84
85 def test_run_hook_down_deletes(self):
86 """down should delete the created json file."""
87 nic = 'eth1'
88 populate_dir(
89 self.tmp, {nic + ".json": "{'abcd'}", 'myfile.txt': 'text'})
90 dhc.run_hook(nic, 'down', data_d=self.tmp, env={'old_host_name': 'x1'})
91 self.assertEqual(
92 set(['myfile.txt']),
93 set(dir2dict(self.tmp + os.path.sep)))
94
95 def test_get_parser(self):
96 """Smoke test creation of get_parser."""
97 # cloud-init main uses 'action'.
98 event, interface = (dhc.UP, 'mynic0')
99 self.assertEqual(
100 argparse.Namespace(event=event, interface=interface,
101 action=(dhc.NAME, dhc.handle_args)),
102 dhc.get_parser().parse_args([event, interface]))
103
104
105# vi: ts=4 expandtab
diff --git a/cloudinit/version.py b/cloudinit/version.py
index 844a02e..a2c5d43 100644
--- a/cloudinit/version.py
+++ b/cloudinit/version.py
@@ -4,7 +4,7 @@
4#4#
5# This file is part of cloud-init. See LICENSE file for license information.5# This file is part of cloud-init. See LICENSE file for license information.
66
7__VERSION__ = "18.4"7__VERSION__ = "18.5"
8_PACKAGED_VERSION = '@@PACKAGED_VERSION@@'8_PACKAGED_VERSION = '@@PACKAGED_VERSION@@'
99
10FEATURES = [10FEATURES = [
diff --git a/config/cloud.cfg.tmpl b/config/cloud.cfg.tmpl
index 1fef133..7513176 100644
--- a/config/cloud.cfg.tmpl
+++ b/config/cloud.cfg.tmpl
@@ -167,7 +167,17 @@ system_info:
167 - http://%(availability_zone)s.clouds.archive.ubuntu.com/ubuntu/167 - http://%(availability_zone)s.clouds.archive.ubuntu.com/ubuntu/
168 - http://%(region)s.clouds.archive.ubuntu.com/ubuntu/168 - http://%(region)s.clouds.archive.ubuntu.com/ubuntu/
169 security: []169 security: []
170 - arches: [armhf, armel, default]170 - arches: [arm64, armel, armhf]
171 failsafe:
172 primary: http://ports.ubuntu.com/ubuntu-ports
173 security: http://ports.ubuntu.com/ubuntu-ports
174 search:
175 primary:
176 - http://%(ec2_region)s.ec2.ports.ubuntu.com/ubuntu-ports/
177 - http://%(availability_zone)s.clouds.ports.ubuntu.com/ubuntu-ports/
178 - http://%(region)s.clouds.ports.ubuntu.com/ubuntu-ports/
179 security: []
180 - arches: [default]
171 failsafe:181 failsafe:
172 primary: http://ports.ubuntu.com/ubuntu-ports182 primary: http://ports.ubuntu.com/ubuntu-ports
173 security: http://ports.ubuntu.com/ubuntu-ports183 security: http://ports.ubuntu.com/ubuntu-ports
diff --git a/debian/changelog b/debian/changelog
index 283bcd8..09a0034 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,19 @@
1cloud-init (18.5-1-g5b065316-0ubuntu1) disco; urgency=medium
2
3 * New upstream snapshot.
4 - Update to pylint 2.2.2.
5 - Release 18.5 (LP: #1808380)
6 - tests: add Disco release [Joshua Powers]
7 - net: render 'metric' values in per-subnet routes (LP: #1805871)
8 - write_files: add support for appending to files. [James Baxter]
9 - config: On ubuntu select cloud archive mirrors for armel, armhf, arm64.
10 (LP: #1805854)
11 - dhclient-hook: cleanups, tests and fix a bug on 'down' event.
12 - NoCloud: Allow top level 'network' key in network-config. (LP: #1798117)
13 - ovf: Fix ovf network config generation gateway/routes (LP: #1806103)
14
15 -- Ryan Harper <ryan.harper@canonical.com> Fri, 14 Dec 2018 14:45:46 -0600
16
1cloud-init (18.4-31-gbf791715-0ubuntu1) disco; urgency=medium17cloud-init (18.4-31-gbf791715-0ubuntu1) disco; urgency=medium
218
3 * New upstream snapshot.19 * New upstream snapshot.
diff --git a/tests/cloud_tests/releases.yaml b/tests/cloud_tests/releases.yaml
index defae02..ec5da72 100644
--- a/tests/cloud_tests/releases.yaml
+++ b/tests/cloud_tests/releases.yaml
@@ -129,6 +129,22 @@ features:
129129
130releases:130releases:
131 # UBUNTU =================================================================131 # UBUNTU =================================================================
132 disco:
133 # EOL: Jan 2020
134 default:
135 enabled: true
136 release: disco
137 version: 19.04
138 os: ubuntu
139 feature_groups:
140 - base
141 - debian_base
142 - ubuntu_specific
143 lxd:
144 sstreams_server: https://cloud-images.ubuntu.com/daily
145 alias: disco
146 setup_overrides: null
147 override_templates: false
132 cosmic:148 cosmic:
133 # EOL: Jul 2019149 # EOL: Jul 2019
134 default:150 default:
diff --git a/tests/unittests/test_cli.py b/tests/unittests/test_cli.py
index 199d69b..d283f13 100644
--- a/tests/unittests/test_cli.py
+++ b/tests/unittests/test_cli.py
@@ -246,18 +246,18 @@ class TestCLI(test_helpers.FilesystemMockingTestCase):
246 self.assertEqual('cc_ntp', parseargs.name)246 self.assertEqual('cc_ntp', parseargs.name)
247 self.assertFalse(parseargs.report)247 self.assertFalse(parseargs.report)
248248
249 @mock.patch('cloudinit.cmd.main.dhclient_hook')249 @mock.patch('cloudinit.cmd.main.dhclient_hook.handle_args')
250 def test_dhclient_hook_subcommand(self, m_dhclient_hook):250 def test_dhclient_hook_subcommand(self, m_handle_args):
251 """The subcommand 'dhclient-hook' calls dhclient_hook with args."""251 """The subcommand 'dhclient-hook' calls dhclient_hook with args."""
252 self._call_main(['cloud-init', 'dhclient-hook', 'net_action', 'eth0'])252 self._call_main(['cloud-init', 'dhclient-hook', 'up', 'eth0'])
253 (name, parseargs) = m_dhclient_hook.call_args_list[0][0]253 (name, parseargs) = m_handle_args.call_args_list[0][0]
254 self.assertEqual('dhclient_hook', name)254 self.assertEqual('dhclient-hook', name)
255 self.assertEqual('dhclient-hook', parseargs.subcommand)255 self.assertEqual('dhclient-hook', parseargs.subcommand)
256 self.assertEqual('dhclient_hook', parseargs.action[0])256 self.assertEqual('dhclient-hook', parseargs.action[0])
257 self.assertFalse(parseargs.debug)257 self.assertFalse(parseargs.debug)
258 self.assertFalse(parseargs.force)258 self.assertFalse(parseargs.force)
259 self.assertEqual('net_action', parseargs.net_action)259 self.assertEqual('up', parseargs.event)
260 self.assertEqual('eth0', parseargs.net_interface)260 self.assertEqual('eth0', parseargs.interface)
261261
262 @mock.patch('cloudinit.cmd.main.main_features')262 @mock.patch('cloudinit.cmd.main.main_features')
263 def test_features_hook_subcommand(self, m_features):263 def test_features_hook_subcommand(self, m_features):
diff --git a/tests/unittests/test_datasource/test_nocloud.py b/tests/unittests/test_datasource/test_nocloud.py
index b6468b6..3429272 100644
--- a/tests/unittests/test_datasource/test_nocloud.py
+++ b/tests/unittests/test_datasource/test_nocloud.py
@@ -1,7 +1,10 @@
1# This file is part of cloud-init. See LICENSE file for license information.1# This file is part of cloud-init. See LICENSE file for license information.
22
3from cloudinit import helpers3from cloudinit import helpers
4from cloudinit.sources import DataSourceNoCloud4from cloudinit.sources.DataSourceNoCloud import (
5 DataSourceNoCloud as dsNoCloud,
6 _maybe_remove_top_network,
7 parse_cmdline_data)
5from cloudinit import util8from cloudinit import util
6from cloudinit.tests.helpers import CiTestCase, populate_dir, mock, ExitStack9from cloudinit.tests.helpers import CiTestCase, populate_dir, mock, ExitStack
710
@@ -40,9 +43,7 @@ class TestNoCloudDataSource(CiTestCase):
40 'datasource': {'NoCloud': {'fs_label': None}}43 'datasource': {'NoCloud': {'fs_label': None}}
41 }44 }
4245
43 ds = DataSourceNoCloud.DataSourceNoCloud46 dsrc = dsNoCloud(sys_cfg=sys_cfg, distro=None, paths=self.paths)
44
45 dsrc = ds(sys_cfg=sys_cfg, distro=None, paths=self.paths)
46 ret = dsrc.get_data()47 ret = dsrc.get_data()
47 self.assertEqual(dsrc.userdata_raw, ud)48 self.assertEqual(dsrc.userdata_raw, ud)
48 self.assertEqual(dsrc.metadata, md)49 self.assertEqual(dsrc.metadata, md)
@@ -63,9 +64,7 @@ class TestNoCloudDataSource(CiTestCase):
63 'datasource': {'NoCloud': {'fs_label': None}}64 'datasource': {'NoCloud': {'fs_label': None}}
64 }65 }
6566
66 ds = DataSourceNoCloud.DataSourceNoCloud67 dsrc = dsNoCloud(sys_cfg=sys_cfg, distro=None, paths=self.paths)
67
68 dsrc = ds(sys_cfg=sys_cfg, distro=None, paths=self.paths)
69 self.assertTrue(dsrc.get_data())68 self.assertTrue(dsrc.get_data())
70 self.assertEqual(dsrc.platform_type, 'nocloud')69 self.assertEqual(dsrc.platform_type, 'nocloud')
71 self.assertEqual(70 self.assertEqual(
@@ -73,8 +72,6 @@ class TestNoCloudDataSource(CiTestCase):
7372
74 def test_fs_label(self, m_is_lxd):73 def test_fs_label(self, m_is_lxd):
75 # find_devs_with should not be called ff fs_label is None74 # find_devs_with should not be called ff fs_label is None
76 ds = DataSourceNoCloud.DataSourceNoCloud
77
78 class PsuedoException(Exception):75 class PsuedoException(Exception):
79 pass76 pass
8077
@@ -84,12 +81,12 @@ class TestNoCloudDataSource(CiTestCase):
8481
85 # by default, NoCloud should search for filesystems by label82 # by default, NoCloud should search for filesystems by label
86 sys_cfg = {'datasource': {'NoCloud': {}}}83 sys_cfg = {'datasource': {'NoCloud': {}}}
87 dsrc = ds(sys_cfg=sys_cfg, distro=None, paths=self.paths)84 dsrc = dsNoCloud(sys_cfg=sys_cfg, distro=None, paths=self.paths)
88 self.assertRaises(PsuedoException, dsrc.get_data)85 self.assertRaises(PsuedoException, dsrc.get_data)
8986
90 # but disabling searching should just end up with None found87 # but disabling searching should just end up with None found
91 sys_cfg = {'datasource': {'NoCloud': {'fs_label': None}}}88 sys_cfg = {'datasource': {'NoCloud': {'fs_label': None}}}
92 dsrc = ds(sys_cfg=sys_cfg, distro=None, paths=self.paths)89 dsrc = dsNoCloud(sys_cfg=sys_cfg, distro=None, paths=self.paths)
93 ret = dsrc.get_data()90 ret = dsrc.get_data()
94 self.assertFalse(ret)91 self.assertFalse(ret)
9592
@@ -97,13 +94,10 @@ class TestNoCloudDataSource(CiTestCase):
97 # no source should be found if no cmdline, config, and fs_label=None94 # no source should be found if no cmdline, config, and fs_label=None
98 sys_cfg = {'datasource': {'NoCloud': {'fs_label': None}}}95 sys_cfg = {'datasource': {'NoCloud': {'fs_label': None}}}
9996
100 ds = DataSourceNoCloud.DataSourceNoCloud97 dsrc = dsNoCloud(sys_cfg=sys_cfg, distro=None, paths=self.paths)
101 dsrc = ds(sys_cfg=sys_cfg, distro=None, paths=self.paths)
102 self.assertFalse(dsrc.get_data())98 self.assertFalse(dsrc.get_data())
10399
104 def test_seed_in_config(self, m_is_lxd):100 def test_seed_in_config(self, m_is_lxd):
105 ds = DataSourceNoCloud.DataSourceNoCloud
106
107 data = {101 data = {
108 'fs_label': None,102 'fs_label': None,
109 'meta-data': yaml.safe_dump({'instance-id': 'IID'}),103 'meta-data': yaml.safe_dump({'instance-id': 'IID'}),
@@ -111,7 +105,7 @@ class TestNoCloudDataSource(CiTestCase):
111 }105 }
112106
113 sys_cfg = {'datasource': {'NoCloud': data}}107 sys_cfg = {'datasource': {'NoCloud': data}}
114 dsrc = ds(sys_cfg=sys_cfg, distro=None, paths=self.paths)108 dsrc = dsNoCloud(sys_cfg=sys_cfg, distro=None, paths=self.paths)
115 ret = dsrc.get_data()109 ret = dsrc.get_data()
116 self.assertEqual(dsrc.userdata_raw, b"USER_DATA_RAW")110 self.assertEqual(dsrc.userdata_raw, b"USER_DATA_RAW")
117 self.assertEqual(dsrc.metadata.get('instance-id'), 'IID')111 self.assertEqual(dsrc.metadata.get('instance-id'), 'IID')
@@ -130,9 +124,7 @@ class TestNoCloudDataSource(CiTestCase):
130 'datasource': {'NoCloud': {'fs_label': None}}124 'datasource': {'NoCloud': {'fs_label': None}}
131 }125 }
132126
133 ds = DataSourceNoCloud.DataSourceNoCloud127 dsrc = dsNoCloud(sys_cfg=sys_cfg, distro=None, paths=self.paths)
134
135 dsrc = ds(sys_cfg=sys_cfg, distro=None, paths=self.paths)
136 ret = dsrc.get_data()128 ret = dsrc.get_data()
137 self.assertEqual(dsrc.userdata_raw, ud)129 self.assertEqual(dsrc.userdata_raw, ud)
138 self.assertEqual(dsrc.metadata, md)130 self.assertEqual(dsrc.metadata, md)
@@ -145,9 +137,7 @@ class TestNoCloudDataSource(CiTestCase):
145137
146 sys_cfg = {'datasource': {'NoCloud': {'fs_label': None}}}138 sys_cfg = {'datasource': {'NoCloud': {'fs_label': None}}}
147139
148 ds = DataSourceNoCloud.DataSourceNoCloud140 dsrc = dsNoCloud(sys_cfg=sys_cfg, distro=None, paths=self.paths)
149
150 dsrc = ds(sys_cfg=sys_cfg, distro=None, paths=self.paths)
151 ret = dsrc.get_data()141 ret = dsrc.get_data()
152 self.assertEqual(dsrc.userdata_raw, b"ud")142 self.assertEqual(dsrc.userdata_raw, b"ud")
153 self.assertFalse(dsrc.vendordata)143 self.assertFalse(dsrc.vendordata)
@@ -174,9 +164,7 @@ class TestNoCloudDataSource(CiTestCase):
174164
175 sys_cfg = {'datasource': {'NoCloud': {'fs_label': None}}}165 sys_cfg = {'datasource': {'NoCloud': {'fs_label': None}}}
176166
177 ds = DataSourceNoCloud.DataSourceNoCloud167 dsrc = dsNoCloud(sys_cfg=sys_cfg, distro=None, paths=self.paths)
178
179 dsrc = ds(sys_cfg=sys_cfg, distro=None, paths=self.paths)
180 ret = dsrc.get_data()168 ret = dsrc.get_data()
181 self.assertTrue(ret)169 self.assertTrue(ret)
182 # very simple check just for the strings above170 # very simple check just for the strings above
@@ -195,9 +183,23 @@ class TestNoCloudDataSource(CiTestCase):
195183
196 sys_cfg = {'datasource': {'NoCloud': {'fs_label': None}}}184 sys_cfg = {'datasource': {'NoCloud': {'fs_label': None}}}
197185
198 ds = DataSourceNoCloud.DataSourceNoCloud186 dsrc = dsNoCloud(sys_cfg=sys_cfg, distro=None, paths=self.paths)
187 ret = dsrc.get_data()
188 self.assertTrue(ret)
189 self.assertEqual(netconf, dsrc.network_config)
190
191 def test_metadata_network_config_with_toplevel_network(self, m_is_lxd):
192 """network-config may have 'network' top level key."""
193 netconf = {'config': 'disabled'}
194 populate_dir(
195 os.path.join(self.paths.seed_dir, "nocloud"),
196 {'user-data': b"ud",
197 'meta-data': "instance-id: IID\n",
198 'network-config': yaml.dump({'network': netconf}) + "\n"})
199
200 sys_cfg = {'datasource': {'NoCloud': {'fs_label': None}}}
199201
200 dsrc = ds(sys_cfg=sys_cfg, distro=None, paths=self.paths)202 dsrc = dsNoCloud(sys_cfg=sys_cfg, distro=None, paths=self.paths)
201 ret = dsrc.get_data()203 ret = dsrc.get_data()
202 self.assertTrue(ret)204 self.assertTrue(ret)
203 self.assertEqual(netconf, dsrc.network_config)205 self.assertEqual(netconf, dsrc.network_config)
@@ -228,9 +230,7 @@ class TestNoCloudDataSource(CiTestCase):
228230
229 sys_cfg = {'datasource': {'NoCloud': {'fs_label': None}}}231 sys_cfg = {'datasource': {'NoCloud': {'fs_label': None}}}
230232
231 ds = DataSourceNoCloud.DataSourceNoCloud233 dsrc = dsNoCloud(sys_cfg=sys_cfg, distro=None, paths=self.paths)
232
233 dsrc = ds(sys_cfg=sys_cfg, distro=None, paths=self.paths)
234 ret = dsrc.get_data()234 ret = dsrc.get_data()
235 self.assertTrue(ret)235 self.assertTrue(ret)
236 self.assertEqual(netconf, dsrc.network_config)236 self.assertEqual(netconf, dsrc.network_config)
@@ -258,8 +258,7 @@ class TestParseCommandLineData(CiTestCase):
258 for (fmt, expected) in pairs:258 for (fmt, expected) in pairs:
259 fill = {}259 fill = {}
260 cmdline = fmt % {'ds_id': ds_id}260 cmdline = fmt % {'ds_id': ds_id}
261 ret = DataSourceNoCloud.parse_cmdline_data(ds_id=ds_id, fill=fill,261 ret = parse_cmdline_data(ds_id=ds_id, fill=fill, cmdline=cmdline)
262 cmdline=cmdline)
263 self.assertEqual(expected, fill)262 self.assertEqual(expected, fill)
264 self.assertTrue(ret)263 self.assertTrue(ret)
265264
@@ -276,10 +275,43 @@ class TestParseCommandLineData(CiTestCase):
276275
277 for cmdline in cmdlines:276 for cmdline in cmdlines:
278 fill = {}277 fill = {}
279 ret = DataSourceNoCloud.parse_cmdline_data(ds_id=ds_id, fill=fill,278 ret = parse_cmdline_data(ds_id=ds_id, fill=fill, cmdline=cmdline)
280 cmdline=cmdline)
281 self.assertEqual(fill, {})279 self.assertEqual(fill, {})
282 self.assertFalse(ret)280 self.assertFalse(ret)
283281
284282
283class TestMaybeRemoveToplevelNetwork(CiTestCase):
284 """test _maybe_remove_top_network function."""
285 basecfg = [{'type': 'physical', 'name': 'interface0',
286 'subnets': [{'type': 'dhcp'}]}]
287
288 def test_should_remove_safely(self):
289 mcfg = {'config': self.basecfg, 'version': 1}
290 self.assertEqual(mcfg, _maybe_remove_top_network({'network': mcfg}))
291
292 def test_no_remove_if_other_keys(self):
293 """should not shift if other keys at top level."""
294 mcfg = {'network': {'config': self.basecfg, 'version': 1},
295 'unknown_keyname': 'keyval'}
296 self.assertEqual(mcfg, _maybe_remove_top_network(mcfg))
297
298 def test_no_remove_if_non_dict(self):
299 """should not shift if not a dict."""
300 mcfg = {'network': '"content here'}
301 self.assertEqual(mcfg, _maybe_remove_top_network(mcfg))
302
303 def test_no_remove_if_missing_config_or_version(self):
304 """should not shift unless network entry has config and version."""
305 mcfg = {'network': {'config': self.basecfg}}
306 self.assertEqual(mcfg, _maybe_remove_top_network(mcfg))
307
308 mcfg = {'network': {'version': 1}}
309 self.assertEqual(mcfg, _maybe_remove_top_network(mcfg))
310
311 def test_remove_with_config_disabled(self):
312 """network/config=disabled should be shifted."""
313 mcfg = {'config': 'disabled'}
314 self.assertEqual(mcfg, _maybe_remove_top_network({'network': mcfg}))
315
316
285# vi: ts=4 expandtab317# vi: ts=4 expandtab
diff --git a/tests/unittests/test_handler/test_handler_write_files.py b/tests/unittests/test_handler/test_handler_write_files.py
index 7fa8fd2..bc8756c 100644
--- a/tests/unittests/test_handler/test_handler_write_files.py
+++ b/tests/unittests/test_handler/test_handler_write_files.py
@@ -52,6 +52,18 @@ class TestWriteFiles(FilesystemMockingTestCase):
52 "test_simple", [{"content": expected, "path": filename}])52 "test_simple", [{"content": expected, "path": filename}])
53 self.assertEqual(util.load_file(filename), expected)53 self.assertEqual(util.load_file(filename), expected)
5454
55 def test_append(self):
56 self.patchUtils(self.tmp)
57 existing = "hello "
58 added = "world\n"
59 expected = existing + added
60 filename = "/tmp/append.file"
61 util.write_file(filename, existing)
62 write_files(
63 "test_append",
64 [{"content": added, "path": filename, "append": "true"}])
65 self.assertEqual(util.load_file(filename), expected)
66
55 def test_yaml_binary(self):67 def test_yaml_binary(self):
56 self.patchUtils(self.tmp)68 self.patchUtils(self.tmp)
57 data = util.load_yaml(YAML_TEXT)69 data = util.load_yaml(YAML_TEXT)
diff --git a/tests/unittests/test_net.py b/tests/unittests/test_net.py
index 8e38373..195f261 100644
--- a/tests/unittests/test_net.py
+++ b/tests/unittests/test_net.py
@@ -488,8 +488,8 @@ NETWORK_CONFIGS = {
488 address 192.168.21.3/24488 address 192.168.21.3/24
489 dns-nameservers 8.8.8.8 8.8.4.4489 dns-nameservers 8.8.8.8 8.8.4.4
490 dns-search barley.maas sach.maas490 dns-search barley.maas sach.maas
491 post-up route add default gw 65.61.151.37 || true491 post-up route add default gw 65.61.151.37 metric 10000 || true
492 pre-down route del default gw 65.61.151.37 || true492 pre-down route del default gw 65.61.151.37 metric 10000 || true
493 """).rstrip(' '),493 """).rstrip(' '),
494 'expected_netplan': textwrap.dedent("""494 'expected_netplan': textwrap.dedent("""
495 network:495 network:
@@ -513,7 +513,8 @@ NETWORK_CONFIGS = {
513 - barley.maas513 - barley.maas
514 - sach.maas514 - sach.maas
515 routes:515 routes:
516 - to: 0.0.0.0/0516 - metric: 10000
517 to: 0.0.0.0/0
517 via: 65.61.151.37518 via: 65.61.151.37
518 set-name: eth99519 set-name: eth99
519 """).rstrip(' '),520 """).rstrip(' '),
@@ -537,6 +538,7 @@ NETWORK_CONFIGS = {
537 HWADDR=c0:d6:9f:2c:e8:80538 HWADDR=c0:d6:9f:2c:e8:80
538 IPADDR=192.168.21.3539 IPADDR=192.168.21.3
539 NETMASK=255.255.255.0540 NETMASK=255.255.255.0
541 METRIC=10000
540 NM_CONTROLLED=no542 NM_CONTROLLED=no
541 ONBOOT=yes543 ONBOOT=yes
542 TYPE=Ethernet544 TYPE=Ethernet
@@ -561,7 +563,7 @@ NETWORK_CONFIGS = {
561 - gateway: 65.61.151.37563 - gateway: 65.61.151.37
562 netmask: 0.0.0.0564 netmask: 0.0.0.0
563 network: 0.0.0.0565 network: 0.0.0.0
564 metric: 2566 metric: 10000
565 - type: physical567 - type: physical
566 name: eth1568 name: eth1
567 mac_address: "cf:d6:af:48:e8:80"569 mac_address: "cf:d6:af:48:e8:80"
@@ -1161,6 +1163,13 @@ pre-down route del -net 10.0.0.0 netmask 255.0.0.0 gw 11.0.0.1 metric 3 || true
1161 - gateway: 192.168.0.31163 - gateway: 192.168.0.3
1162 netmask: 255.255.255.01164 netmask: 255.255.255.0
1163 network: 10.1.3.01165 network: 10.1.3.0
1166 - gateway: 2001:67c:1562:1
1167 network: 2001:67c:1
1168 netmask: ffff:ffff:0
1169 - gateway: 3001:67c:1562:1
1170 network: 3001:67c:1
1171 netmask: ffff:ffff:0
1172 metric: 10000
1164 - type: static1173 - type: static
1165 address: 192.168.1.2/241174 address: 192.168.1.2/24
1166 - type: static1175 - type: static
@@ -1197,6 +1206,11 @@ pre-down route del -net 10.0.0.0 netmask 255.0.0.0 gw 11.0.0.1 metric 3 || true
1197 routes:1206 routes:
1198 - to: 10.1.3.0/241207 - to: 10.1.3.0/24
1199 via: 192.168.0.31208 via: 192.168.0.3
1209 - to: 2001:67c:1/32
1210 via: 2001:67c:1562:1
1211 - metric: 10000
1212 to: 3001:67c:1/32
1213 via: 3001:67c:1562:1
1200 """),1214 """),
1201 'yaml-v2': textwrap.dedent("""1215 'yaml-v2': textwrap.dedent("""
1202 version: 21216 version: 2
@@ -1228,6 +1242,11 @@ pre-down route del -net 10.0.0.0 netmask 255.0.0.0 gw 11.0.0.1 metric 3 || true
1228 routes:1242 routes:
1229 - to: 10.1.3.0/241243 - to: 10.1.3.0/24
1230 via: 192.168.0.31244 via: 192.168.0.3
1245 - to: 2001:67c:1562:8007::1/64
1246 via: 2001:67c:1562:8007::aac:40b2
1247 - metric: 10000
1248 to: 3001:67c:1562:8007::1/64
1249 via: 3001:67c:1562:8007::aac:40b2
1231 """),1250 """),
1232 'expected_netplan-v2': textwrap.dedent("""1251 'expected_netplan-v2': textwrap.dedent("""
1233 network:1252 network:
@@ -1249,6 +1268,11 @@ pre-down route del -net 10.0.0.0 netmask 255.0.0.0 gw 11.0.0.1 metric 3 || true
1249 routes:1268 routes:
1250 - to: 10.1.3.0/241269 - to: 10.1.3.0/24
1251 via: 192.168.0.31270 via: 192.168.0.3
1271 - to: 2001:67c:1562:8007::1/64
1272 via: 2001:67c:1562:8007::aac:40b2
1273 - metric: 10000
1274 to: 3001:67c:1562:8007::1/64
1275 via: 3001:67c:1562:8007::aac:40b2
1252 ethernets:1276 ethernets:
1253 eth0:1277 eth0:
1254 match:1278 match:
@@ -1349,6 +1373,10 @@ pre-down route del -net 10.0.0.0 netmask 255.0.0.0 gw 11.0.0.1 metric 3 || true
1349 USERCTL=no1373 USERCTL=no
1350 """),1374 """),
1351 'route6-bond0': textwrap.dedent("""\1375 'route6-bond0': textwrap.dedent("""\
1376 # Created by cloud-init on instance boot automatically, do not edit.
1377 #
1378 2001:67c:1/ffff:ffff:0 via 2001:67c:1562:1 dev bond0
1379 3001:67c:1/ffff:ffff:0 via 3001:67c:1562:1 metric 10000 dev bond0
1352 """),1380 """),
1353 'route-bond0': textwrap.dedent("""\1381 'route-bond0': textwrap.dedent("""\
1354 ADDRESS0=10.1.3.01382 ADDRESS0=10.1.3.0
@@ -1879,14 +1907,24 @@ class TestRhelSysConfigRendering(CiTestCase):
1879 return dir2dict(dir)1907 return dir2dict(dir)
18801908
1881 def _compare_files_to_expected(self, expected, found):1909 def _compare_files_to_expected(self, expected, found):
1910
1911 def _try_load(f):
1912 ''' Attempt to load shell content, otherwise return as-is '''
1913 try:
1914 return util.load_shell_content(f)
1915 except ValueError:
1916 pass
1917 # route6- * files aren't shell content, but iproute2 params
1918 return f
1919
1882 orig_maxdiff = self.maxDiff1920 orig_maxdiff = self.maxDiff
1883 expected_d = dict(1921 expected_d = dict(
1884 (os.path.join(self.scripts_dir, k), util.load_shell_content(v))1922 (os.path.join(self.scripts_dir, k), _try_load(v))
1885 for k, v in expected.items())1923 for k, v in expected.items())
18861924
1887 # only compare the files in scripts_dir1925 # only compare the files in scripts_dir
1888 scripts_found = dict(1926 scripts_found = dict(
1889 (k, util.load_shell_content(v)) for k, v in found.items()1927 (k, _try_load(v)) for k, v in found.items()
1890 if k.startswith(self.scripts_dir))1928 if k.startswith(self.scripts_dir))
1891 try:1929 try:
1892 self.maxDiff = None1930 self.maxDiff = None
diff --git a/tests/unittests/test_vmware_config_file.py b/tests/unittests/test_vmware_config_file.py
index 602dedb..f47335e 100644
--- a/tests/unittests/test_vmware_config_file.py
+++ b/tests/unittests/test_vmware_config_file.py
@@ -263,7 +263,7 @@ class TestVmwareConfigFile(CiTestCase):
263 nicConfigurator = NicConfigurator(config.nics, False)263 nicConfigurator = NicConfigurator(config.nics, False)
264 nics_cfg_list = nicConfigurator.generate()264 nics_cfg_list = nicConfigurator.generate()
265265
266 self.assertEqual(5, len(nics_cfg_list), "number of elements")266 self.assertEqual(2, len(nics_cfg_list), "number of elements")
267267
268 nic1 = {'name': 'NIC1'}268 nic1 = {'name': 'NIC1'}
269 nic2 = {'name': 'NIC2'}269 nic2 = {'name': 'NIC2'}
@@ -275,8 +275,6 @@ class TestVmwareConfigFile(CiTestCase):
275 nic1.update(cfg)275 nic1.update(cfg)
276 elif cfg.get('name') == nic2.get('name'):276 elif cfg.get('name') == nic2.get('name'):
277 nic2.update(cfg)277 nic2.update(cfg)
278 elif cfg_type == 'route':
279 route_list.append(cfg)
280278
281 self.assertEqual('physical', nic1.get('type'), 'type of NIC1')279 self.assertEqual('physical', nic1.get('type'), 'type of NIC1')
282 self.assertEqual('NIC1', nic1.get('name'), 'name of NIC1')280 self.assertEqual('NIC1', nic1.get('name'), 'name of NIC1')
@@ -297,6 +295,9 @@ class TestVmwareConfigFile(CiTestCase):
297 static6_subnet.append(subnet)295 static6_subnet.append(subnet)
298 else:296 else:
299 self.assertEqual(True, False, 'Unknown type')297 self.assertEqual(True, False, 'Unknown type')
298 if 'route' in subnet:
299 for route in subnet.get('routes'):
300 route_list.append(route)
300301
301 self.assertEqual(1, len(static_subnet), 'Number of static subnet')302 self.assertEqual(1, len(static_subnet), 'Number of static subnet')
302 self.assertEqual(1, len(static6_subnet), 'Number of static6 subnet')303 self.assertEqual(1, len(static6_subnet), 'Number of static6 subnet')
@@ -351,6 +352,8 @@ class TestVmwareConfigFile(CiTestCase):
351class TestVmwareNetConfig(CiTestCase):352class TestVmwareNetConfig(CiTestCase):
352 """Test conversion of vmware config to cloud-init config."""353 """Test conversion of vmware config to cloud-init config."""
353354
355 maxDiff = None
356
354 def _get_NicConfigurator(self, text):357 def _get_NicConfigurator(self, text):
355 fp = None358 fp = None
356 try:359 try:
@@ -420,9 +423,52 @@ class TestVmwareNetConfig(CiTestCase):
420 'mac_address': '00:50:56:a6:8c:08',423 'mac_address': '00:50:56:a6:8c:08',
421 'subnets': [424 'subnets': [
422 {'control': 'auto', 'type': 'static',425 {'control': 'auto', 'type': 'static',
423 'address': '10.20.87.154', 'netmask': '255.255.252.0'}]},426 'address': '10.20.87.154', 'netmask': '255.255.252.0',
424 {'type': 'route', 'destination': '10.20.84.0/22',427 'routes':
425 'gateway': '10.20.87.253', 'metric': 10000}],428 [{'type': 'route', 'destination': '10.20.84.0/22',
429 'gateway': '10.20.87.253', 'metric': 10000}]}]}],
430 nc.generate())
431
432 def test_cust_non_primary_nic_with_gateway_(self):
433 """A customer non primary nic set can have a gateway."""
434 config = textwrap.dedent("""\
435 [NETWORK]
436 NETWORKING = yes
437 BOOTPROTO = dhcp
438 HOSTNAME = static-debug-vm
439 DOMAINNAME = cluster.local
440
441 [NIC-CONFIG]
442 NICS = NIC1
443
444 [NIC1]
445 MACADDR = 00:50:56:ac:d1:8a
446 ONBOOT = yes
447 IPv4_MODE = BACKWARDS_COMPATIBLE
448 BOOTPROTO = static
449 IPADDR = 100.115.223.75
450 NETMASK = 255.255.255.0
451 GATEWAY = 100.115.223.254
452
453
454 [DNS]
455 DNSFROMDHCP=no
456
457 NAMESERVER|1 = 8.8.8.8
458
459 [DATETIME]
460 UTC = yes
461 """)
462 nc = self._get_NicConfigurator(config)
463 self.assertEqual(
464 [{'type': 'physical', 'name': 'NIC1',
465 'mac_address': '00:50:56:ac:d1:8a',
466 'subnets': [
467 {'control': 'auto', 'type': 'static',
468 'address': '100.115.223.75', 'netmask': '255.255.255.0',
469 'routes':
470 [{'type': 'route', 'destination': '100.115.223.0/24',
471 'gateway': '100.115.223.254', 'metric': 10000}]}]}],
426 nc.generate())472 nc.generate())
427473
428 def test_a_primary_nic_with_gateway(self):474 def test_a_primary_nic_with_gateway(self):
diff --git a/tox.ini b/tox.ini
index 2fb3209..d983348 100644
--- a/tox.ini
+++ b/tox.ini
@@ -21,7 +21,7 @@ setenv =
21basepython = python321basepython = python3
22deps =22deps =
23 # requirements23 # requirements
24 pylint==1.8.124 pylint==2.2.2
25 # test-requirements because unit tests are now present in cloudinit tree25 # test-requirements because unit tests are now present in cloudinit tree
26 -r{toxinidir}/test-requirements.txt26 -r{toxinidir}/test-requirements.txt
27commands = {envpython} -m pylint {posargs:cloudinit tests tools}27commands = {envpython} -m pylint {posargs:cloudinit tests tools}

Subscribers

People subscribed via source and target branches