cloud-init

Merge ~chad.smith/cloud-init:feature/azure-network-per-boot into cloud-init:master

  1. Git
  2. lp:~chad.smith/cloud-init
  3. feature/azure-network-per-boot
  4. Merge into master
Proposed by Chad Smith
Status: Merged
Approved by: Scott Moser
Approved revision: d838fd9f1ecb6b96d73727d73634ce19531a3891
Merge reported by: Server Team CI bot
Merged at revision: not available
Proposed branch: ~chad.smith/cloud-init:feature/azure-network-per-boot
Merge into: cloud-init:master
Diff against target: 960 lines (+605/-59)
3 files modified
cloudinit/cmd/devel/net_convert.py (+7/-2)
cloudinit/sources/DataSourceAzure.py (+227/-29)
tests/unittests/test_datasource/test_azure.py (+371/-28)
Reviewer Review Type Date Requested Status
Server Team CI bot continuous-integration Approve
Scott Moser Approve
Review via email: mp+352660@code.launchpad.net

Commit message

azure: allow azure to generate network configuration from IMDS per boot.

Azure datasource now queries IMDS metadata service for network
configuration at link local address
http://169.254.169.254/metadata/instance?api-version=2017-12-01. The
azure metadata service presents a list of macs and allocated ip addresses
associated with this instance. Azure will now also regenerate network
configuration on every boot because it subscribes to EventType.BOOT
maintenance events as well as the 'first boot'
EventType.BOOT_NEW_INSTANCE.

For testing add azure-imds --kind to cloud-init devel net_convert tool
for debugging IMDS metadata.

Also refactor _get_data into 3 discrete methods:
  - is_platform_viable: check quickly whether the datasource is
    potentially compatible with the platform on which is is running
  - crawl_metadata: walk all potential metadata candidates, returning a
    structured dict of all metadata and userdata. Raise InvalidMetaData on
    error.
  - _get_data: call crawl_metadata and process results or error. Cache
    instance data on class attributes: metadata, userdata_raw etc.

Description of the change

As a note: current supported Ubuntu images contain some magic udev rules and/or netplan config which automatically setup dhcp on any nic eth1 or greater which shows up on an azure instance during boot. With this functionality in cloud-init proper, we can drop the static script definitions from supported ubuntu images because cloud-init will own the network config on any new device that shows up across boot.

to test:
create an Azure vm;
ssh azcurl -H Metadata:true "http://169.254.169.254/metadata/instance?api-version=2017-12-01" > azure.json

python3 -m cloudinit.cmd.main devel net_convert --kind azure-imds --network-data azure.json -d output -O netplan

To post a comment you must log in.
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

PASSED: Continuous integration, rev:ee224a69788844fc474e7d75f7d96c94f76c5b32
https://jenkins.ubuntu.com/server/job/cloud-init-ci/207/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    SUCCESS: MAAS Compatability Testing
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/207/rebuild

review: Approve (continuous-integration)
Revision history for this message
Scott Moser (smoser) :
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

FAILED: Continuous integration, rev:c1152c7080468bef048156754fefeb6cb4e6bcfa
https://jenkins.ubuntu.com/server/job/cloud-init-ci/214/
Executed test runs:
    SUCCESS: Checkout
    FAILED: Unit & Style Tests

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/214/rebuild

review: Needs Fixing (continuous-integration)
Revision history for this message
Chad Smith (chad.smith) :
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

FAILED: Continuous integration, rev:59c62bd868f941bb4450558257e36910fd409d54
https://jenkins.ubuntu.com/server/job/cloud-init-ci/217/
Executed test runs:
    SUCCESS: Checkout
    FAILED: Unit & Style Tests

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/217/rebuild

review: Needs Fixing (continuous-integration)
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

FAILED: Continuous integration, rev:59c62bd868f941bb4450558257e36910fd409d54
https://jenkins.ubuntu.com/server/job/cloud-init-ci/219/
Executed test runs:
    SUCCESS: Checkout
    FAILED: Unit & Style Tests

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/219/rebuild

review: Needs Fixing (continuous-integration)
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

FAILED: Continuous integration, rev:59c62bd868f941bb4450558257e36910fd409d54
https://jenkins.ubuntu.com/server/job/cloud-init-ci/220/
Executed test runs:
    SUCCESS: Checkout
    FAILED: Unit & Style Tests

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/220/rebuild

review: Needs Fixing (continuous-integration)
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

FAILED: Continuous integration, rev:a89229b914b4b78986a59581817d56f566bda16f
https://jenkins.ubuntu.com/server/job/cloud-init-ci/221/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    FAILED: Ubuntu LTS: Build

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/221/rebuild

review: Needs Fixing (continuous-integration)
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

FAILED: Continuous integration, rev:8f970e614a7f7f6ae84cc4f24380785ba74f95aa
https://jenkins.ubuntu.com/server/job/cloud-init-ci/222/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    FAILED: Ubuntu LTS: Build

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/222/rebuild

review: Needs Fixing (continuous-integration)
Revision history for this message
Scott Moser (smoser) wrote :

I think i'm fine with this as long as we get a c-i run.
I do have one comment in line. but other than that if you think its good, i think its good.

Revision history for this message
Server Team CI bot (server-team-bot) wrote :

PASSED: Continuous integration, rev:c762ca7f5c642a28375f6a93762f57a6334c27eb
https://jenkins.ubuntu.com/server/job/cloud-init-ci/229/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    SUCCESS: MAAS Compatability Testing
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/229/rebuild

review: Approve (continuous-integration)
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

PASSED: Continuous integration, rev:d838fd9f1ecb6b96d73727d73634ce19531a3891
https://jenkins.ubuntu.com/server/job/cloud-init-ci/232/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    SUCCESS: MAAS Compatability Testing
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/232/rebuild

review: Approve (continuous-integration)
Revision history for this message
Scott Moser (smoser) wrote :

Per Chad:

16:39 <@blackboxsw> smoser: I'd like to take your suggestion to bubble up network setup into Azure.get_data as a tech-debt item, there's their poll_imds use-case which wants to re-try dhcp attempts across IMDS metadata updates due to a user-triggered IP address change. I think this will need broader discussion with Azure and/or access to a test env that exhibits the imds metadata change to verify that a single dhcp. I'm not sure if we
16:39 <@blackboxsw> want to block the Azure network-config-per-boot branch on that refactor or ot.

I'm good with that.

Approve.

review: Approve
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

FAILED: Autolanding.
More details in the following jenkins job:
https://jenkins.ubuntu.com/server/job/cloud-init-autoland-test/23/
Executed test runs:
    SUCCESS: Checkout
    FAILED: Unit & Style Tests

review: Needs Fixing (continuous-integration)
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

FAILED: Autolanding.
More details in the following jenkins job:
https://jenkins.ubuntu.com/server/job/cloud-init-autoland-test/24/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    FAILED: Ubuntu LTS: Integration

review: Needs Fixing (continuous-integration)
Revision history for this message
Server Team CI bot (server-team-bot) :
review: Approve (continuous-integration)

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1diff --git a/cloudinit/cmd/devel/net_convert.py b/cloudinit/cmd/devel/net_convert.py
2index 1ec08a3..271dc5e 100755
3--- a/cloudinit/cmd/devel/net_convert.py
4+++ b/cloudinit/cmd/devel/net_convert.py
5@@ -8,6 +8,7 @@ import sys
6 import yaml
7
8 from cloudinit.sources.helpers import openstack
9+from cloudinit.sources import DataSourceAzure as azure
10
11 from cloudinit.net import eni, netplan, network_state, sysconfig
12 from cloudinit import log
13@@ -28,7 +29,8 @@ def get_parser(parser=None):
14 parser.add_argument("-p", "--network-data", type=open,
15 metavar="PATH", required=True)
16 parser.add_argument("-k", "--kind",
17- choices=['eni', 'network_data.json', 'yaml'],
18+ choices=['eni', 'network_data.json', 'yaml',
19+ 'azure-imds'],
20 required=True)
21 parser.add_argument("-d", "--directory",
22 metavar="PATH",
23@@ -78,10 +80,13 @@ def handle_args(name, args):
24 ["Input YAML",
25 yaml.dump(pre_ns, default_flow_style=False, indent=4), ""]))
26 ns = network_state.parse_net_config_data(pre_ns)
27- else:
28+ elif args.kind == 'network_data.json':
29 pre_ns = openstack.convert_net_json(
30 json.loads(net_data), known_macs=known_macs)
31 ns = network_state.parse_net_config_data(pre_ns)
32+ elif args.kind == 'azure-imds':
33+ pre_ns = azure.parse_network_config(json.loads(net_data))
34+ ns = network_state.parse_net_config_data(pre_ns)
35
36 if not ns:
37 raise RuntimeError("No valid network_state object created from"
38diff --git a/cloudinit/sources/DataSourceAzure.py b/cloudinit/sources/DataSourceAzure.py
39index 7007d9e..783445e 100644
40--- a/cloudinit/sources/DataSourceAzure.py
41+++ b/cloudinit/sources/DataSourceAzure.py
42@@ -8,6 +8,7 @@ import base64
43 import contextlib
44 import crypt
45 from functools import partial
46+import json
47 import os
48 import os.path
49 import re
50@@ -17,6 +18,7 @@ import xml.etree.ElementTree as ET
51
52 from cloudinit import log as logging
53 from cloudinit import net
54+from cloudinit.event import EventType
55 from cloudinit.net.dhcp import EphemeralDHCPv4
56 from cloudinit import sources
57 from cloudinit.sources.helpers.azure import get_metadata_from_fabric
58@@ -49,7 +51,17 @@ DEFAULT_FS = 'ext4'
59 AZURE_CHASSIS_ASSET_TAG = '7783-7084-3265-9085-8269-3286-77'
60 REPROVISION_MARKER_FILE = "/var/lib/cloud/data/poll_imds"
61 REPORTED_READY_MARKER_FILE = "/var/lib/cloud/data/reported_ready"
62-IMDS_URL = "http://169.254.169.254/metadata/reprovisiondata"
63+AGENT_SEED_DIR = '/var/lib/waagent'
64+IMDS_URL = "http://169.254.169.254/metadata/"
65+
66+# List of static scripts and network config artifacts created by
67+# stock ubuntu suported images.
68+UBUNTU_EXTENDED_NETWORK_SCRIPTS = [
69+ '/etc/netplan/90-azure-hotplug.yaml',
70+ '/usr/local/sbin/ephemeral_eth.sh',
71+ '/etc/udev/rules.d/10-net-device-added.rules',
72+ '/run/network/interfaces.ephemeral.d',
73+]
74
75
76 def find_storvscid_from_sysctl_pnpinfo(sysctl_out, deviceid):
77@@ -185,7 +197,7 @@ if util.is_FreeBSD():
78
79 BUILTIN_DS_CONFIG = {
80 'agent_command': AGENT_START_BUILTIN,
81- 'data_dir': "/var/lib/waagent",
82+ 'data_dir': AGENT_SEED_DIR,
83 'set_hostname': True,
84 'hostname_bounce': {
85 'interface': DEFAULT_PRIMARY_NIC,
86@@ -252,6 +264,7 @@ class DataSourceAzure(sources.DataSource):
87
88 dsname = 'Azure'
89 _negotiated = False
90+ _metadata_imds = sources.UNSET
91
92 def __init__(self, sys_cfg, distro, paths):
93 sources.DataSource.__init__(self, sys_cfg, distro, paths)
94@@ -263,6 +276,8 @@ class DataSourceAzure(sources.DataSource):
95 BUILTIN_DS_CONFIG])
96 self.dhclient_lease_file = self.ds_cfg.get('dhclient_lease_file')
97 self._network_config = None
98+ # Regenerate network config new_instance boot and every boot
99+ self.update_events['network'].add(EventType.BOOT)
100
101 def __str__(self):
102 root = sources.DataSource.__str__(self)
103@@ -336,15 +351,17 @@ class DataSourceAzure(sources.DataSource):
104 metadata['public-keys'] = key_value or pubkeys_from_crt_files(fp_files)
105 return metadata
106
107- def _get_data(self):
108+ def crawl_metadata(self):
109+ """Walk all instance metadata sources returning a dict on success.
110+
111+ @return: A dictionary of any metadata content for this instance.
112+ @raise: InvalidMetaDataException when the expected metadata service is
113+ unavailable, broken or disabled.
114+ """
115+ crawled_data = {}
116 # azure removes/ejects the cdrom containing the ovf-env.xml
117 # file on reboot. So, in order to successfully reboot we
118 # need to look in the datadir and consider that valid
119- asset_tag = util.read_dmi_data('chassis-asset-tag')
120- if asset_tag != AZURE_CHASSIS_ASSET_TAG:
121- LOG.debug("Non-Azure DMI asset tag '%s' discovered.", asset_tag)
122- return False
123-
124 ddir = self.ds_cfg['data_dir']
125
126 candidates = [self.seed_dir]
127@@ -373,41 +390,84 @@ class DataSourceAzure(sources.DataSource):
128 except NonAzureDataSource:
129 continue
130 except BrokenAzureDataSource as exc:
131- raise exc
132+ msg = 'BrokenAzureDataSource: %s' % exc
133+ raise sources.InvalidMetaDataException(msg)
134 except util.MountFailedError:
135 LOG.warning("%s was not mountable", cdev)
136 continue
137
138 if reprovision or self._should_reprovision(ret):
139 ret = self._reprovision()
140- (md, self.userdata_raw, cfg, files) = ret
141+ imds_md = get_metadata_from_imds(
142+ self.fallback_interface, retries=3)
143+ (md, userdata_raw, cfg, files) = ret
144 self.seed = cdev
145- self.metadata = util.mergemanydict([md, DEFAULT_METADATA])
146- self.cfg = util.mergemanydict([cfg, BUILTIN_CLOUD_CONFIG])
147+ crawled_data.update({
148+ 'cfg': cfg,
149+ 'files': files,
150+ 'metadata': util.mergemanydict(
151+ [md, {'imds': imds_md}]),
152+ 'userdata_raw': userdata_raw})
153 found = cdev
154
155 LOG.debug("found datasource in %s", cdev)
156 break
157
158 if not found:
159- return False
160+ raise sources.InvalidMetaDataException('No Azure metadata found')
161
162 if found == ddir:
163 LOG.debug("using files cached in %s", ddir)
164
165 seed = _get_random_seed()
166 if seed:
167- self.metadata['random_seed'] = seed
168+ crawled_data['metadata']['random_seed'] = seed
169+ crawled_data['metadata']['instance-id'] = util.read_dmi_data(
170+ 'system-uuid')
171+ return crawled_data
172+
173+ def _is_platform_viable(self):
174+ """Check platform environment to report if this datasource may run."""
175+ return _is_platform_viable(self.seed_dir)
176+
177+ def clear_cached_attrs(self, attr_defaults=()):
178+ """Reset any cached class attributes to defaults."""
179+ super(DataSourceAzure, self).clear_cached_attrs(attr_defaults)
180+ self._metadata_imds = sources.UNSET
181+
182+ def _get_data(self):
183+ """Crawl and process datasource metadata caching metadata as attrs.
184+
185+ @return: True on success, False on error, invalid or disabled
186+ datasource.
187+ """
188+ if not self._is_platform_viable():
189+ return False
190+ try:
191+ crawled_data = util.log_time(
192+ logfunc=LOG.debug, msg='Crawl of metadata service',
193+ func=self.crawl_metadata)
194+ except sources.InvalidMetaDataException as e:
195+ LOG.warning('Could not crawl Azure metadata: %s', e)
196+ return False
197+ if self.distro and self.distro.name == 'ubuntu':
198+ maybe_remove_ubuntu_network_config_scripts()
199+
200+ # Process crawled data and augment with various config defaults
201+ self.cfg = util.mergemanydict(
202+ [crawled_data['cfg'], BUILTIN_CLOUD_CONFIG])
203+ self._metadata_imds = crawled_data['metadata']['imds']
204+ self.metadata = util.mergemanydict(
205+ [crawled_data['metadata'], DEFAULT_METADATA])
206+ self.userdata_raw = crawled_data['userdata_raw']
207
208 user_ds_cfg = util.get_cfg_by_path(self.cfg, DS_CFG_PATH, {})
209 self.ds_cfg = util.mergemanydict([user_ds_cfg, self.ds_cfg])
210
211 # walinux agent writes files world readable, but expects
212 # the directory to be protected.
213- write_files(ddir, files, dirmode=0o700)
214-
215- self.metadata['instance-id'] = util.read_dmi_data('system-uuid')
216-
217+ write_files(
218+ self.ds_cfg['data_dir'], crawled_data['files'], dirmode=0o700)
219 return True
220
221 def device_name_to_device(self, name):
222@@ -436,7 +496,7 @@ class DataSourceAzure(sources.DataSource):
223 def _poll_imds(self):
224 """Poll IMDS for the new provisioning data until we get a valid
225 response. Then return the returned JSON object."""
226- url = IMDS_URL + "?api-version=2017-04-02"
227+ url = IMDS_URL + "reprovisiondata?api-version=2017-04-02"
228 headers = {"Metadata": "true"}
229 report_ready = bool(not os.path.isfile(REPORTED_READY_MARKER_FILE))
230 LOG.debug("Start polling IMDS")
231@@ -487,7 +547,7 @@ class DataSourceAzure(sources.DataSource):
232 jump back into the polling loop in order to retrieve the ovf_env."""
233 if not ret:
234 return False
235- (_md, self.userdata_raw, cfg, _files) = ret
236+ (_md, _userdata_raw, cfg, _files) = ret
237 path = REPROVISION_MARKER_FILE
238 if (cfg.get('PreprovisionedVm') is True or
239 os.path.isfile(path)):
240@@ -543,22 +603,15 @@ class DataSourceAzure(sources.DataSource):
241 @property
242 def network_config(self):
243 """Generate a network config like net.generate_fallback_network() with
244- the following execptions.
245+ the following exceptions.
246
247 1. Probe the drivers of the net-devices present and inject them in
248 the network configuration under params: driver: <driver> value
249 2. Generate a fallback network config that does not include any of
250 the blacklisted devices.
251 """
252- blacklist = ['mlx4_core']
253 if not self._network_config:
254- LOG.debug('Azure: generating fallback configuration')
255- # generate a network config, blacklist picking any mlx4_core devs
256- netconfig = net.generate_fallback_config(
257- blacklist_drivers=blacklist, config_driver=True)
258-
259- self._network_config = netconfig
260-
261+ self._network_config = parse_network_config(self._metadata_imds)
262 return self._network_config
263
264
265@@ -1025,6 +1078,151 @@ def load_azure_ds_dir(source_dir):
266 return (md, ud, cfg, {'ovf-env.xml': contents})
267
268
269+def parse_network_config(imds_metadata):
270+ """Convert imds_metadata dictionary to network v2 configuration.
271+
272+ Parses network configuration from imds metadata if present or generate
273+ fallback network config excluding mlx4_core devices.
274+
275+ @param: imds_metadata: Dict of content read from IMDS network service.
276+ @return: Dictionary containing network version 2 standard configuration.
277+ """
278+ if imds_metadata != sources.UNSET and imds_metadata:
279+ netconfig = {'version': 2, 'ethernets': {}}
280+ LOG.debug('Azure: generating network configuration from IMDS')
281+ network_metadata = imds_metadata['network']
282+ for idx, intf in enumerate(network_metadata['interface']):
283+ nicname = 'eth{idx}'.format(idx=idx)
284+ dev_config = {}
285+ for addr4 in intf['ipv4']['ipAddress']:
286+ privateIpv4 = addr4['privateIpAddress']
287+ if privateIpv4:
288+ if dev_config.get('dhcp4', False):
289+ # Append static address config for nic > 1
290+ netPrefix = intf['ipv4']['subnet'][0].get(
291+ 'prefix', '24')
292+ if not dev_config.get('addresses'):
293+ dev_config['addresses'] = []
294+ dev_config['addresses'].append(
295+ '{ip}/{prefix}'.format(
296+ ip=privateIpv4, prefix=netPrefix))
297+ else:
298+ dev_config['dhcp4'] = True
299+ for addr6 in intf['ipv6']['ipAddress']:
300+ privateIpv6 = addr6['privateIpAddress']
301+ if privateIpv6:
302+ dev_config['dhcp6'] = True
303+ break
304+ if dev_config:
305+ mac = ':'.join(re.findall(r'..', intf['macAddress']))
306+ dev_config.update(
307+ {'match': {'macaddress': mac.lower()},
308+ 'set-name': nicname})
309+ netconfig['ethernets'][nicname] = dev_config
310+ else:
311+ blacklist = ['mlx4_core']
312+ LOG.debug('Azure: generating fallback configuration')
313+ # generate a network config, blacklist picking mlx4_core devs
314+ netconfig = net.generate_fallback_config(
315+ blacklist_drivers=blacklist, config_driver=True)
316+ return netconfig
317+
318+
319+def get_metadata_from_imds(fallback_nic, retries):
320+ """Query Azure's network metadata service, returning a dictionary.
321+
322+ If network is not up, setup ephemeral dhcp on fallback_nic to talk to the
323+ IMDS. For more info on IMDS:
324+ https://docs.microsoft.com/en-us/azure/virtual-machines/windows/instance-metadata-service
325+
326+ @param fallback_nic: String. The name of the nic which requires active
327+ network in order to query IMDS.
328+ @param retries: The number of retries of the IMDS_URL.
329+
330+ @return: A dict of instance metadata containing compute and network
331+ info.
332+ """
333+ kwargs = {'logfunc': LOG.debug,
334+ 'msg': 'Crawl of Azure Instance Metadata Service (IMDS)',
335+ 'func': _get_metadata_from_imds, 'args': (retries,)}
336+ if net.is_up(fallback_nic):
337+ return util.log_time(**kwargs)
338+ else:
339+ with EphemeralDHCPv4(fallback_nic):
340+ return util.log_time(**kwargs)
341+
342+
343+def _get_metadata_from_imds(retries):
344+
345+ def retry_on_url_error(msg, exception):
346+ if isinstance(exception, UrlError) and exception.code == 404:
347+ return True # Continue retries
348+ return False # Stop retries on all other exceptions
349+
350+ url = IMDS_URL + "instance?api-version=2017-12-01"
351+ headers = {"Metadata": "true"}
352+ try:
353+ response = readurl(
354+ url, timeout=1, headers=headers, retries=retries,
355+ exception_cb=retry_on_url_error)
356+ except Exception as e:
357+ LOG.debug('Ignoring IMDS instance metadata: %s', e)
358+ return {}
359+ try:
360+ return util.load_json(str(response))
361+ except json.decoder.JSONDecodeError:
362+ LOG.warning(
363+ 'Ignoring non-json IMDS instance metadata: %s', str(response))
364+ return {}
365+
366+
367+def maybe_remove_ubuntu_network_config_scripts(paths=None):
368+ """Remove Azure-specific ubuntu network config for non-primary nics.
369+
370+ @param paths: List of networking scripts or directories to remove when
371+ present.
372+
373+ In certain supported ubuntu images, static udev rules or netplan yaml
374+ config is delivered in the base ubuntu image to support dhcp on any
375+ additional interfaces which get attached by a customer at some point
376+ after initial boot. Since the Azure datasource can now regenerate
377+ network configuration as metadata reports these new devices, we no longer
378+ want the udev rules or netplan's 90-azure-hotplug.yaml to configure
379+ networking on eth1 or greater as it might collide with cloud-init's
380+ configuration.
381+
382+ Remove the any existing extended network scripts if the datasource is
383+ enabled to write network per-boot.
384+ """
385+ if not paths:
386+ paths = UBUNTU_EXTENDED_NETWORK_SCRIPTS
387+ logged = False
388+ for path in paths:
389+ if os.path.exists(path):
390+ if not logged:
391+ LOG.info(
392+ 'Removing Ubuntu extended network scripts because'
393+ ' cloud-init updates Azure network configuration on the'
394+ ' following event: %s.',
395+ EventType.BOOT)
396+ logged = True
397+ if os.path.isdir(path):
398+ util.del_dir(path)
399+ else:
400+ util.del_file(path)
401+
402+
403+def _is_platform_viable(seed_dir):
404+ """Check platform environment to report if this datasource may run."""
405+ asset_tag = util.read_dmi_data('chassis-asset-tag')
406+ if asset_tag == AZURE_CHASSIS_ASSET_TAG:
407+ return True
408+ LOG.debug("Non-Azure DMI asset tag '%s' discovered.", asset_tag)
409+ if os.path.exists(os.path.join(seed_dir, 'ovf-env.xml')):
410+ return True
411+ return False
412+
413+
414 class BrokenAzureDataSource(Exception):
415 pass
416
417diff --git a/tests/unittests/test_datasource/test_azure.py b/tests/unittests/test_datasource/test_azure.py
418index e82716e..4e428b7 100644
419--- a/tests/unittests/test_datasource/test_azure.py
420+++ b/tests/unittests/test_datasource/test_azure.py
421@@ -1,15 +1,21 @@
422 # This file is part of cloud-init. See LICENSE file for license information.
423
424+from cloudinit import distros
425 from cloudinit import helpers
426-from cloudinit.sources import DataSourceAzure as dsaz
427+from cloudinit import url_helper
428+from cloudinit.sources import (
429+ UNSET, DataSourceAzure as dsaz, InvalidMetaDataException)
430 from cloudinit.util import (b64e, decode_binary, load_file, write_file,
431 find_freebsd_part, get_path_dev_freebsd,
432 MountFailedError)
433 from cloudinit.version import version_string as vs
434-from cloudinit.tests.helpers import (CiTestCase, TestCase, populate_dir, mock,
435- ExitStack, PY26, SkipTest)
436+from cloudinit.tests.helpers import (
437+ HttprettyTestCase, CiTestCase, populate_dir, mock, wrap_and_call,
438+ ExitStack, PY26, SkipTest)
439
440 import crypt
441+import httpretty
442+import json
443 import os
444 import stat
445 import xml.etree.ElementTree as ET
446@@ -77,6 +83,106 @@ def construct_valid_ovf_env(data=None, pubkeys=None,
447 return content
448
449
450+NETWORK_METADATA = {
451+ "network": {
452+ "interface": [
453+ {
454+ "macAddress": "000D3A047598",
455+ "ipv6": {
456+ "ipAddress": []
457+ },
458+ "ipv4": {
459+ "subnet": [
460+ {
461+ "prefix": "24",
462+ "address": "10.0.0.0"
463+ }
464+ ],
465+ "ipAddress": [
466+ {
467+ "privateIpAddress": "10.0.0.4",
468+ "publicIpAddress": "104.46.124.81"
469+ }
470+ ]
471+ }
472+ }
473+ ]
474+ }
475+}
476+
477+
478+class TestGetMetadataFromIMDS(HttprettyTestCase):
479+
480+ with_logs = True
481+
482+ def setUp(self):
483+ super(TestGetMetadataFromIMDS, self).setUp()
484+ self.network_md_url = dsaz.IMDS_URL + "instance?api-version=2017-12-01"
485+
486+ @mock.patch('cloudinit.sources.DataSourceAzure.readurl')
487+ @mock.patch('cloudinit.sources.DataSourceAzure.EphemeralDHCPv4')
488+ @mock.patch('cloudinit.sources.DataSourceAzure.net.is_up')
489+ def test_get_metadata_does_not_dhcp_if_network_is_up(
490+ self, m_net_is_up, m_dhcp, m_readurl):
491+ """Do not perform DHCP setup when nic is already up."""
492+ m_net_is_up.return_value = True
493+ m_readurl.return_value = url_helper.StringResponse(
494+ json.dumps(NETWORK_METADATA).encode('utf-8'))
495+ self.assertEqual(
496+ NETWORK_METADATA,
497+ dsaz.get_metadata_from_imds('eth9', retries=3))
498+
499+ m_net_is_up.assert_called_with('eth9')
500+ m_dhcp.assert_not_called()
501+ self.assertIn(
502+ "Crawl of Azure Instance Metadata Service (IMDS) took", # log_time
503+ self.logs.getvalue())
504+
505+ @mock.patch('cloudinit.sources.DataSourceAzure.readurl')
506+ @mock.patch('cloudinit.sources.DataSourceAzure.EphemeralDHCPv4')
507+ @mock.patch('cloudinit.sources.DataSourceAzure.net.is_up')
508+ def test_get_metadata_performs_dhcp_when_network_is_down(
509+ self, m_net_is_up, m_dhcp, m_readurl):
510+ """Perform DHCP setup when nic is not up."""
511+ m_net_is_up.return_value = False
512+ m_readurl.return_value = url_helper.StringResponse(
513+ json.dumps(NETWORK_METADATA).encode('utf-8'))
514+
515+ self.assertEqual(
516+ NETWORK_METADATA,
517+ dsaz.get_metadata_from_imds('eth9', retries=2))
518+
519+ m_net_is_up.assert_called_with('eth9')
520+ m_dhcp.assert_called_with('eth9')
521+ self.assertIn(
522+ "Crawl of Azure Instance Metadata Service (IMDS) took", # log_time
523+ self.logs.getvalue())
524+
525+ m_readurl.assert_called_with(
526+ self.network_md_url, exception_cb=mock.ANY,
527+ headers={'Metadata': 'true'}, retries=2, timeout=1)
528+
529+ @mock.patch('cloudinit.url_helper.time.sleep')
530+ @mock.patch('cloudinit.sources.DataSourceAzure.net.is_up')
531+ def test_get_metadata_from_imds_empty_when_no_imds_present(
532+ self, m_net_is_up, m_sleep):
533+ """Return empty dict when IMDS network metadata is absent."""
534+ httpretty.register_uri(
535+ httpretty.GET,
536+ dsaz.IMDS_URL + 'instance?api-version=2017-12-01',
537+ body={}, status=404)
538+
539+ m_net_is_up.return_value = True # skips dhcp
540+
541+ self.assertEqual({}, dsaz.get_metadata_from_imds('eth9', retries=2))
542+
543+ m_net_is_up.assert_called_with('eth9')
544+ self.assertEqual([mock.call(1), mock.call(1)], m_sleep.call_args_list)
545+ self.assertIn(
546+ "Crawl of Azure Instance Metadata Service (IMDS) took", # log_time
547+ self.logs.getvalue())
548+
549+
550 class TestAzureDataSource(CiTestCase):
551
552 with_logs = True
553@@ -95,8 +201,19 @@ class TestAzureDataSource(CiTestCase):
554 self.patches = ExitStack()
555 self.addCleanup(self.patches.close)
556
557- self.patches.enter_context(mock.patch.object(dsaz, '_get_random_seed'))
558-
559+ self.patches.enter_context(mock.patch.object(
560+ dsaz, '_get_random_seed', return_value='wild'))
561+ self.m_get_metadata_from_imds = self.patches.enter_context(
562+ mock.patch.object(
563+ dsaz, 'get_metadata_from_imds',
564+ mock.MagicMock(return_value=NETWORK_METADATA)))
565+ self.m_fallback_nic = self.patches.enter_context(
566+ mock.patch('cloudinit.sources.net.find_fallback_nic',
567+ return_value='eth9'))
568+ self.m_remove_ubuntu_network_scripts = self.patches.enter_context(
569+ mock.patch.object(
570+ dsaz, 'maybe_remove_ubuntu_network_config_scripts',
571+ mock.MagicMock()))
572 super(TestAzureDataSource, self).setUp()
573
574 def apply_patches(self, patches):
575@@ -137,7 +254,7 @@ scbus-1 on xpt0 bus 0
576 ])
577 return dsaz
578
579- def _get_ds(self, data, agent_command=None):
580+ def _get_ds(self, data, agent_command=None, distro=None):
581
582 def dsdevs():
583 return data.get('dsdevs', [])
584@@ -186,8 +303,11 @@ scbus-1 on xpt0 bus 0
585 side_effect=_wait_for_files)),
586 ])
587
588+ if distro is not None:
589+ distro_cls = distros.fetch(distro)
590+ distro = distro_cls(distro, data.get('sys_cfg', {}), self.paths)
591 dsrc = dsaz.DataSourceAzure(
592- data.get('sys_cfg', {}), distro=None, paths=self.paths)
593+ data.get('sys_cfg', {}), distro=distro, paths=self.paths)
594 if agent_command is not None:
595 dsrc.ds_cfg['agent_command'] = agent_command
596
597@@ -260,29 +380,20 @@ fdescfs /dev/fd fdescfs rw 0 0
598 res = get_path_dev_freebsd('/etc', mnt_list)
599 self.assertIsNotNone(res)
600
601- @mock.patch('cloudinit.sources.DataSourceAzure.util.read_dmi_data')
602- def test_non_azure_dmi_chassis_asset_tag(self, m_read_dmi_data):
603- """Report non-azure when DMI's chassis asset tag doesn't match.
604-
605- Return False when the asset tag doesn't match Azure's static
606- AZURE_CHASSIS_ASSET_TAG.
607- """
608+ @mock.patch('cloudinit.sources.DataSourceAzure._is_platform_viable')
609+ def test_call_is_platform_viable_seed(self, m_is_platform_viable):
610+ """Check seed_dir using _is_platform_viable and return False."""
611 # Return a non-matching asset tag value
612- nonazure_tag = dsaz.AZURE_CHASSIS_ASSET_TAG + 'X'
613- m_read_dmi_data.return_value = nonazure_tag
614+ m_is_platform_viable.return_value = False
615 dsrc = dsaz.DataSourceAzure(
616 {}, distro=None, paths=self.paths)
617 self.assertFalse(dsrc.get_data())
618- self.assertEqual(
619- "DEBUG: Non-Azure DMI asset tag '{0}' discovered.\n".format(
620- nonazure_tag),
621- self.logs.getvalue())
622+ m_is_platform_viable.assert_called_with(dsrc.seed_dir)
623
624 def test_basic_seed_dir(self):
625 odata = {'HostName': "myhost", 'UserName': "myuser"}
626 data = {'ovfcontent': construct_valid_ovf_env(data=odata),
627 'sys_cfg': {}}
628-
629 dsrc = self._get_ds(data)
630 ret = dsrc.get_data()
631 self.assertTrue(ret)
632@@ -291,6 +402,82 @@ fdescfs /dev/fd fdescfs rw 0 0
633 self.assertTrue(os.path.isfile(
634 os.path.join(self.waagent_d, 'ovf-env.xml')))
635
636+ def test_get_data_non_ubuntu_will_not_remove_network_scripts(self):
637+ """get_data on non-Ubuntu will not remove ubuntu net scripts."""
638+ odata = {'HostName': "myhost", 'UserName': "myuser"}
639+ data = {'ovfcontent': construct_valid_ovf_env(data=odata),
640+ 'sys_cfg': {}}
641+
642+ dsrc = self._get_ds(data, distro='debian')
643+ dsrc.get_data()
644+ self.m_remove_ubuntu_network_scripts.assert_not_called()
645+
646+ def test_get_data_on_ubuntu_will_remove_network_scripts(self):
647+ """get_data will remove ubuntu net scripts on Ubuntu distro."""
648+ odata = {'HostName': "myhost", 'UserName': "myuser"}
649+ data = {'ovfcontent': construct_valid_ovf_env(data=odata),
650+ 'sys_cfg': {}}
651+
652+ dsrc = self._get_ds(data, distro='ubuntu')
653+ dsrc.get_data()
654+ self.m_remove_ubuntu_network_scripts.assert_called_once_with()
655+
656+ def test_crawl_metadata_returns_structured_data_and_caches_nothing(self):
657+ """Return all structured metadata and cache no class attributes."""
658+ yaml_cfg = "{agent_command: my_command}\n"
659+ odata = {'HostName': "myhost", 'UserName': "myuser",
660+ 'UserData': {'text': 'FOOBAR', 'encoding': 'plain'},
661+ 'dscfg': {'text': yaml_cfg, 'encoding': 'plain'}}
662+ data = {'ovfcontent': construct_valid_ovf_env(data=odata),
663+ 'sys_cfg': {}}
664+ dsrc = self._get_ds(data)
665+ expected_cfg = {
666+ 'PreprovisionedVm': False,
667+ 'datasource': {'Azure': {'agent_command': 'my_command'}},
668+ 'system_info': {'default_user': {'name': u'myuser'}}}
669+ expected_metadata = {
670+ 'azure_data': {
671+ 'configurationsettype': 'LinuxProvisioningConfiguration'},
672+ 'imds': {'network': {'interface': [{
673+ 'ipv4': {'ipAddress': [
674+ {'privateIpAddress': '10.0.0.4',
675+ 'publicIpAddress': '104.46.124.81'}],
676+ 'subnet': [{'address': '10.0.0.0', 'prefix': '24'}]},
677+ 'ipv6': {'ipAddress': []},
678+ 'macAddress': '000D3A047598'}]}},
679+ 'instance-id': 'test-instance-id',
680+ 'local-hostname': u'myhost',
681+ 'random_seed': 'wild'}
682+
683+ crawled_metadata = dsrc.crawl_metadata()
684+
685+ self.assertItemsEqual(
686+ crawled_metadata.keys(),
687+ ['cfg', 'files', 'metadata', 'userdata_raw'])
688+ self.assertEqual(crawled_metadata['cfg'], expected_cfg)
689+ self.assertEqual(
690+ list(crawled_metadata['files'].keys()), ['ovf-env.xml'])
691+ self.assertIn(
692+ b'<HostName>myhost</HostName>',
693+ crawled_metadata['files']['ovf-env.xml'])
694+ self.assertEqual(crawled_metadata['metadata'], expected_metadata)
695+ self.assertEqual(crawled_metadata['userdata_raw'], 'FOOBAR')
696+ self.assertEqual(dsrc.userdata_raw, None)
697+ self.assertEqual(dsrc.metadata, {})
698+ self.assertEqual(dsrc._metadata_imds, UNSET)
699+ self.assertFalse(os.path.isfile(
700+ os.path.join(self.waagent_d, 'ovf-env.xml')))
701+
702+ def test_crawl_metadata_raises_invalid_metadata_on_error(self):
703+ """crawl_metadata raises an exception on invalid ovf-env.xml."""
704+ data = {'ovfcontent': "BOGUS", 'sys_cfg': {}}
705+ dsrc = self._get_ds(data)
706+ error_msg = ('BrokenAzureDataSource: Invalid ovf-env.xml:'
707+ ' syntax error: line 1, column 0')
708+ with self.assertRaises(InvalidMetaDataException) as cm:
709+ dsrc.crawl_metadata()
710+ self.assertEqual(str(cm.exception), error_msg)
711+
712 def test_waagent_d_has_0700_perms(self):
713 # we expect /var/lib/waagent to be created 0700
714 dsrc = self._get_ds({'ovfcontent': construct_valid_ovf_env()})
715@@ -314,6 +501,20 @@ fdescfs /dev/fd fdescfs rw 0 0
716 self.assertTrue(ret)
717 self.assertEqual(data['agent_invoked'], cfg['agent_command'])
718
719+ def test_network_config_set_from_imds(self):
720+ """Datasource.network_config returns IMDS network data."""
721+ odata = {}
722+ data = {'ovfcontent': construct_valid_ovf_env(data=odata)}
723+ expected_network_config = {
724+ 'ethernets': {
725+ 'eth0': {'set-name': 'eth0',
726+ 'match': {'macaddress': '00:0d:3a:04:75:98'},
727+ 'dhcp4': True}},
728+ 'version': 2}
729+ dsrc = self._get_ds(data)
730+ dsrc.get_data()
731+ self.assertEqual(expected_network_config, dsrc.network_config)
732+
733 def test_user_cfg_set_agent_command(self):
734 # set dscfg in via base64 encoded yaml
735 cfg = {'agent_command': "my_command"}
736@@ -579,12 +780,34 @@ fdescfs /dev/fd fdescfs rw 0 0
737 self.assertEqual(
738 [mock.call("/dev/cd0")], m_check_fbsd_cdrom.call_args_list)
739
740+ @mock.patch('cloudinit.net.generate_fallback_config')
741+ def test_imds_network_config(self, mock_fallback):
742+ """Network config is generated from IMDS network data when present."""
743+ odata = {'HostName': "myhost", 'UserName': "myuser"}
744+ data = {'ovfcontent': construct_valid_ovf_env(data=odata),
745+ 'sys_cfg': {}}
746+
747+ dsrc = self._get_ds(data)
748+ ret = dsrc.get_data()
749+ self.assertTrue(ret)
750+
751+ expected_cfg = {
752+ 'ethernets': {
753+ 'eth0': {'dhcp4': True,
754+ 'match': {'macaddress': '00:0d:3a:04:75:98'},
755+ 'set-name': 'eth0'}},
756+ 'version': 2}
757+
758+ self.assertEqual(expected_cfg, dsrc.network_config)
759+ mock_fallback.assert_not_called()
760+
761 @mock.patch('cloudinit.net.get_interface_mac')
762 @mock.patch('cloudinit.net.get_devicelist')
763 @mock.patch('cloudinit.net.device_driver')
764 @mock.patch('cloudinit.net.generate_fallback_config')
765- def test_network_config(self, mock_fallback, mock_dd,
766- mock_devlist, mock_get_mac):
767+ def test_fallback_network_config(self, mock_fallback, mock_dd,
768+ mock_devlist, mock_get_mac):
769+ """On absent IMDS network data, generate network fallback config."""
770 odata = {'HostName': "myhost", 'UserName': "myuser"}
771 data = {'ovfcontent': construct_valid_ovf_env(data=odata),
772 'sys_cfg': {}}
773@@ -605,6 +828,8 @@ fdescfs /dev/fd fdescfs rw 0 0
774 mock_get_mac.return_value = '00:11:22:33:44:55'
775
776 dsrc = self._get_ds(data)
777+ # Represent empty response from network imds
778+ self.m_get_metadata_from_imds.return_value = {}
779 ret = dsrc.get_data()
780 self.assertTrue(ret)
781
782@@ -617,8 +842,9 @@ fdescfs /dev/fd fdescfs rw 0 0
783 @mock.patch('cloudinit.net.get_devicelist')
784 @mock.patch('cloudinit.net.device_driver')
785 @mock.patch('cloudinit.net.generate_fallback_config')
786- def test_network_config_blacklist(self, mock_fallback, mock_dd,
787- mock_devlist, mock_get_mac):
788+ def test_fallback_network_config_blacklist(self, mock_fallback, mock_dd,
789+ mock_devlist, mock_get_mac):
790+ """On absent network metadata, blacklist mlx from fallback config."""
791 odata = {'HostName': "myhost", 'UserName': "myuser"}
792 data = {'ovfcontent': construct_valid_ovf_env(data=odata),
793 'sys_cfg': {}}
794@@ -649,6 +875,8 @@ fdescfs /dev/fd fdescfs rw 0 0
795 mock_get_mac.return_value = '00:11:22:33:44:55'
796
797 dsrc = self._get_ds(data)
798+ # Represent empty response from network imds
799+ self.m_get_metadata_from_imds.return_value = {}
800 ret = dsrc.get_data()
801 self.assertTrue(ret)
802
803@@ -689,9 +917,12 @@ class TestAzureBounce(CiTestCase):
804 mock.patch.object(dsaz, 'get_metadata_from_fabric',
805 mock.MagicMock(return_value={})))
806 self.patches.enter_context(
807- mock.patch.object(dsaz.util, 'which', lambda x: True))
808+ mock.patch.object(dsaz, 'get_metadata_from_imds',
809+ mock.MagicMock(return_value={})))
810 self.patches.enter_context(
811- mock.patch.object(dsaz, '_get_random_seed'))
812+ mock.patch.object(dsaz.util, 'which', lambda x: True))
813+ self.patches.enter_context(mock.patch.object(
814+ dsaz, '_get_random_seed', return_value='wild'))
815
816 def _dmi_mocks(key):
817 if key == 'system-uuid':
818@@ -719,9 +950,12 @@ class TestAzureBounce(CiTestCase):
819 mock.patch.object(dsaz, 'set_hostname'))
820 self.subp = self.patches.enter_context(
821 mock.patch('cloudinit.sources.DataSourceAzure.util.subp'))
822+ self.find_fallback_nic = self.patches.enter_context(
823+ mock.patch('cloudinit.net.find_fallback_nic', return_value='eth9'))
824
825 def tearDown(self):
826 self.patches.close()
827+ super(TestAzureBounce, self).tearDown()
828
829 def _get_ds(self, ovfcontent=None, agent_command=None):
830 if ovfcontent is not None:
831@@ -927,7 +1161,7 @@ class TestLoadAzureDsDir(CiTestCase):
832 str(context_manager.exception))
833
834
835-class TestReadAzureOvf(TestCase):
836+class TestReadAzureOvf(CiTestCase):
837
838 def test_invalid_xml_raises_non_azure_ds(self):
839 invalid_xml = "<foo>" + construct_valid_ovf_env(data={})
840@@ -1188,6 +1422,25 @@ class TestCanDevBeReformatted(CiTestCase):
841 "(datasource.Azure.never_destroy_ntfs)", msg)
842
843
844+class TestClearCachedData(CiTestCase):
845+
846+ def test_clear_cached_attrs_clears_imds(self):
847+ """All class attributes are reset to defaults, including imds data."""
848+ tmp = self.tmp_dir()
849+ paths = helpers.Paths(
850+ {'cloud_dir': tmp, 'run_dir': tmp})
851+ dsrc = dsaz.DataSourceAzure({}, distro=None, paths=paths)
852+ clean_values = [dsrc.metadata, dsrc.userdata, dsrc._metadata_imds]
853+ dsrc.metadata = 'md'
854+ dsrc.userdata = 'ud'
855+ dsrc._metadata_imds = 'imds'
856+ dsrc._dirty_cache = True
857+ dsrc.clear_cached_attrs()
858+ self.assertEqual(
859+ [dsrc.metadata, dsrc.userdata, dsrc._metadata_imds],
860+ clean_values)
861+
862+
863 class TestAzureNetExists(CiTestCase):
864
865 def test_azure_net_must_exist_for_legacy_objpkl(self):
866@@ -1398,4 +1651,94 @@ class TestAzureDataSourcePreprovisioning(CiTestCase):
867 self.assertEqual(m_net.call_count, 1)
868
869
870+class TestRemoveUbuntuNetworkConfigScripts(CiTestCase):
871+
872+ with_logs = True
873+
874+ def setUp(self):
875+ super(TestRemoveUbuntuNetworkConfigScripts, self).setUp()
876+ self.tmp = self.tmp_dir()
877+
878+ def test_remove_network_scripts_removes_both_files_and_directories(self):
879+ """Any files or directories in paths are removed when present."""
880+ file1 = self.tmp_path('file1', dir=self.tmp)
881+ subdir = self.tmp_path('sub1', dir=self.tmp)
882+ subfile = self.tmp_path('leaf1', dir=subdir)
883+ write_file(file1, 'file1content')
884+ write_file(subfile, 'leafcontent')
885+ dsaz.maybe_remove_ubuntu_network_config_scripts(paths=[subdir, file1])
886+
887+ for path in (file1, subdir, subfile):
888+ self.assertFalse(os.path.exists(path),
889+ 'Found unremoved: %s' % path)
890+
891+ expected_logs = [
892+ 'INFO: Removing Ubuntu extended network scripts because cloud-init'
893+ ' updates Azure network configuration on the following event:'
894+ ' System boot.',
895+ 'Recursively deleting %s' % subdir,
896+ 'Attempting to remove %s' % file1]
897+ for log in expected_logs:
898+ self.assertIn(log, self.logs.getvalue())
899+
900+ def test_remove_network_scripts_only_attempts_removal_if_path_exists(self):
901+ """Any files or directories absent are skipped without error."""
902+ dsaz.maybe_remove_ubuntu_network_config_scripts(paths=[
903+ self.tmp_path('nodirhere/', dir=self.tmp),
904+ self.tmp_path('notfilehere', dir=self.tmp)])
905+ self.assertNotIn('/not/a', self.logs.getvalue()) # No delete logs
906+
907+ @mock.patch('cloudinit.sources.DataSourceAzure.os.path.exists')
908+ def test_remove_network_scripts_default_removes_stock_scripts(self,
909+ m_exists):
910+ """Azure's stock ubuntu image scripts and artifacts are removed."""
911+ # Report path absent on all to avoid delete operation
912+ m_exists.return_value = False
913+ dsaz.maybe_remove_ubuntu_network_config_scripts()
914+ calls = m_exists.call_args_list
915+ for path in dsaz.UBUNTU_EXTENDED_NETWORK_SCRIPTS:
916+ self.assertIn(mock.call(path), calls)
917+
918+
919+class TestWBIsPlatformViable(CiTestCase):
920+ """White box tests for _is_platform_viable."""
921+ with_logs = True
922+
923+ @mock.patch('cloudinit.sources.DataSourceAzure.util.read_dmi_data')
924+ def test_true_on_non_azure_chassis(self, m_read_dmi_data):
925+ """Return True if DMI chassis-asset-tag is AZURE_CHASSIS_ASSET_TAG."""
926+ m_read_dmi_data.return_value = dsaz.AZURE_CHASSIS_ASSET_TAG
927+ self.assertTrue(dsaz._is_platform_viable('doesnotmatter'))
928+
929+ @mock.patch('cloudinit.sources.DataSourceAzure.os.path.exists')
930+ @mock.patch('cloudinit.sources.DataSourceAzure.util.read_dmi_data')
931+ def test_true_on_azure_ovf_env_in_seed_dir(self, m_read_dmi_data, m_exist):
932+ """Return True if ovf-env.xml exists in known seed dirs."""
933+ # Non-matching Azure chassis-asset-tag
934+ m_read_dmi_data.return_value = dsaz.AZURE_CHASSIS_ASSET_TAG + 'X'
935+
936+ m_exist.return_value = True
937+ self.assertTrue(dsaz._is_platform_viable('/some/seed/dir'))
938+ m_exist.called_once_with('/other/seed/dir')
939+
940+ def test_false_on_no_matching_azure_criteria(self):
941+ """Report non-azure on unmatched asset tag, ovf-env absent and no dev.
942+
943+ Return False when the asset tag doesn't match Azure's static
944+ AZURE_CHASSIS_ASSET_TAG, no ovf-env.xml files exist in known seed dirs
945+ and no devices have a label starting with prefix 'rd_rdfe_'.
946+ """
947+ self.assertFalse(wrap_and_call(
948+ 'cloudinit.sources.DataSourceAzure',
949+ {'os.path.exists': False,
950+ # Non-matching Azure chassis-asset-tag
951+ 'util.read_dmi_data': dsaz.AZURE_CHASSIS_ASSET_TAG + 'X',
952+ 'util.which': None},
953+ dsaz._is_platform_viable, 'doesnotmatter'))
954+ self.assertIn(
955+ "DEBUG: Non-Azure DMI asset tag '{0}' discovered.\n".format(
956+ dsaz.AZURE_CHASSIS_ASSET_TAG + 'X'),
957+ self.logs.getvalue())
958+
959+
960 # vi: ts=4 expandtab

Subscribers

People subscribed via source and target branches