Merge ~smoser/cloud-init:feature/datasource-ibmcloud into cloud-init:master
- Git
- lp:~smoser/cloud-init
- feature/datasource-ibmcloud
- Merge into master
Status: | Rejected |
---|---|
Rejected by: | Chad Smith |
Proposed branch: | ~smoser/cloud-init:feature/datasource-ibmcloud |
Merge into: | cloud-init:master |
Diff against target: |
1044 lines (+838/-14) 8 files modified
cloudinit/sources/DataSourceConfigDrive.py (+10/-0) cloudinit/sources/DataSourceIBMCloud.py (+311/-0) cloudinit/sources/helpers/openstack.py (+2/-2) cloudinit/tests/test_util.py (+72/-0) cloudinit/util.py (+24/-0) tests/unittests/test_datasource/test_ibmcloud.py (+262/-0) tests/unittests/test_ds_identify.py (+101/-3) tools/ds-identify (+56/-9) |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Chad Smith | Approve | ||
Server Team CI bot | continuous-integration | Approve | |
Review via email: mp+341774@code.launchpad.net |
Commit message
IBMCloud: Initial IBM Cloud datasource.
This adds a specific IBM Cloud datasource.
IBM Cloud is identified by:
a.) running on xen
b.) one of a LABEL=METADATA disk or a LABEL=config-2 disk with
UUID=9796-932E
The datasource contains its own config-drive reader that reads
only the currently supported portion of config-drive needed for
ibm cloud.
During the provisioning boot, cloud-init is disabled.
See the docstring in DataSourceIBMCl
Description of the change
TODO:
* we could further limit the search for ibmcloud datasource to devices
that have a LABEL=SWAP-xvdb1.
/dev/xvdb1: LABEL="SWAP-xvdb1" UUID="d51fcca0-
Here is some information collected from 16.04 images I've launched.
OS_CODE (user-data) http://
TEMPLATE_
TEMPLATE_
Here are some hacky scripts in a gist that i put together while doing this:
https:/
Server Team CI bot (server-team-bot) wrote : | # |
Server Team CI bot (server-team-bot) wrote : | # |
PASSED: Continuous integration, rev:25f99a5de7d
https:/
Executed test runs:
SUCCESS: Checkout
SUCCESS: Unit & Style Tests
SUCCESS: Ubuntu LTS: Build
SUCCESS: Ubuntu LTS: Integration
SUCCESS: MAAS Compatability Testing
IN_PROGRESS: Declarative: Post Actions
Click here to trigger a rebuild:
https:/
Server Team CI bot (server-team-bot) wrote : | # |
FAILED: Continuous integration, rev:466ad1cda93
https:/
Executed test runs:
SUCCESS: Checkout
FAILED: Unit & Style Tests
Click here to trigger a rebuild:
https:/
Server Team CI bot (server-team-bot) wrote : | # |
FAILED: Continuous integration, rev:56f4558535f
https:/
Executed test runs:
SUCCESS: Checkout
FAILED: Unit & Style Tests
Click here to trigger a rebuild:
https:/
Server Team CI bot (server-team-bot) wrote : | # |
FAILED: Continuous integration, rev:b046fdac982
https:/
Executed test runs:
SUCCESS: Checkout
FAILED: Unit & Style Tests
Click here to trigger a rebuild:
https:/
Server Team CI bot (server-team-bot) wrote : | # |
PASSED: Continuous integration, rev:f44314ac0f5
https:/
Executed test runs:
SUCCESS: Checkout
SUCCESS: Unit & Style Tests
SUCCESS: Ubuntu LTS: Build
SUCCESS: Ubuntu LTS: Integration
SUCCESS: MAAS Compatability Testing
IN_PROGRESS: Declarative: Post Actions
Click here to trigger a rebuild:
https:/
Chad Smith (chad.smith) : | # |
Ryan Harper (raharper) wrote : | # |
some inline comments, questions.
Server Team CI bot (server-team-bot) wrote : | # |
PASSED: Continuous integration, rev:02e2951824f
https:/
Executed test runs:
SUCCESS: Checkout
SUCCESS: Unit & Style Tests
SUCCESS: Ubuntu LTS: Build
SUCCESS: Ubuntu LTS: Integration
SUCCESS: MAAS Compatability Testing
IN_PROGRESS: Declarative: Post Actions
Click here to trigger a rebuild:
https:/
Scott Moser (smoser) wrote : | # |
addressed feedbacks
Scott Moser (smoser) wrote : | # |
addressed feedbacks.
Server Team CI bot (server-team-bot) wrote : | # |
PASSED: Continuous integration, rev:26072b9d5e1
https:/
Executed test runs:
SUCCESS: Checkout
SUCCESS: Unit & Style Tests
SUCCESS: Ubuntu LTS: Build
SUCCESS: Ubuntu LTS: Integration
SUCCESS: MAAS Compatability Testing
IN_PROGRESS: Declarative: Post Actions
Click here to trigger a rebuild:
https:/
Server Team CI bot (server-team-bot) wrote : | # |
PASSED: Continuous integration, rev:bf1b4f90f6c
https:/
Executed test runs:
SUCCESS: Checkout
SUCCESS: Unit & Style Tests
SUCCESS: Ubuntu LTS: Build
SUCCESS: Ubuntu LTS: Integration
SUCCESS: MAAS Compatability Testing
IN_PROGRESS: Declarative: Post Actions
Click here to trigger a rebuild:
https:/
Server Team CI bot (server-team-bot) wrote : | # |
PASSED: Continuous integration, rev:f60dc12c816
https:/
Executed test runs:
SUCCESS: Checkout
SUCCESS: Unit & Style Tests
SUCCESS: Ubuntu LTS: Build
SUCCESS: Ubuntu LTS: Integration
SUCCESS: MAAS Compatability Testing
IN_PROGRESS: Declarative: Post Actions
Click here to trigger a rebuild:
https:/
Chad Smith (chad.smith) wrote : | # |
Minor nits and suggestions, looks great. Tested ds-id & datasource against openstack cloud making sure we didn't get false positives etc.
Scott Moser (smoser) wrote : | # |
I'll get to your suggestions later.
thanks for the review.
Chad Smith (chad.smith) wrote : | # |
Shepherded this in as branch https:/
There was an error fetching revisions from git servers. Please try again in a few minutes. If the problem persists, contact Launchpad support.
Preview Diff
1 | diff --git a/cloudinit/sources/DataSourceConfigDrive.py b/cloudinit/sources/DataSourceConfigDrive.py |
2 | index b8db626..c7b5fe5 100644 |
3 | --- a/cloudinit/sources/DataSourceConfigDrive.py |
4 | +++ b/cloudinit/sources/DataSourceConfigDrive.py |
5 | @@ -14,6 +14,7 @@ from cloudinit import util |
6 | |
7 | from cloudinit.net import eni |
8 | |
9 | +from cloudinit.sources.DataSourceIBMCloud import get_ibm_platform |
10 | from cloudinit.sources.helpers import openstack |
11 | |
12 | LOG = logging.getLogger(__name__) |
13 | @@ -255,6 +256,15 @@ def find_candidate_devs(probe_optical=True): |
14 | # an unpartitioned block device (ex sda, not sda1) |
15 | devices = [d for d in candidates |
16 | if d in by_label or not util.is_partition(d)] |
17 | + |
18 | + if devices: |
19 | + # IBMCloud uses config-2 label, but limited to a single UUID. |
20 | + ibm_platform, ibm_path = get_ibm_platform() |
21 | + if ibm_path in devices: |
22 | + devices.remove(ibm_path) |
23 | + LOG.debug("IBMCloud device '%s' (%s) removed from candidate list", |
24 | + ibm_path, ibm_platform) |
25 | + |
26 | return devices |
27 | |
28 | |
29 | diff --git a/cloudinit/sources/DataSourceIBMCloud.py b/cloudinit/sources/DataSourceIBMCloud.py |
30 | new file mode 100644 |
31 | index 0000000..e687539 |
32 | --- /dev/null |
33 | +++ b/cloudinit/sources/DataSourceIBMCloud.py |
34 | @@ -0,0 +1,311 @@ |
35 | +# This file is part of cloud-init. See LICENSE file for license information. |
36 | +"""Datasource for IBMCloud. |
37 | + |
38 | +IBMCloud is also know as SoftLayer or BlueMix. |
39 | +IBMCloud hypervisor is xen (2018-03-10). |
40 | + |
41 | +There are 2 different api exposed launch methods. |
42 | + * template: This is the legacy method of launching instances. |
43 | + When booting from an image template, the system boots first into |
44 | + a "provisioning" mode. There, host <-> guest mechanisms are utilized |
45 | + to execute code in the guest and provision it. |
46 | + |
47 | + Cloud-init will disable itself when it detects that it is in the |
48 | + provisioning mode. It detects this by the presence of |
49 | + a file '/root/provisioningConfiguration.cfg'. |
50 | + |
51 | + When provided with user-data, the "first boot" will contain a |
52 | + ConfigDrive-like disk labeled with 'METADATA'. If there is no user-data |
53 | + provided, then there is no data-source. |
54 | + |
55 | + Cloud-init never does any network configuration in this mode. |
56 | + |
57 | + * os_code: Essentially "launch by OS Code" (Operating System Code). |
58 | + This is a more modern approach. There is no specific "provisioning" boot. |
59 | + Instead, cloud-init does all the customization. With or without |
60 | + user-data provided, an OpenStack ConfigDrive like disk is attached. |
61 | + |
62 | + Only disks with label 'config-2' and UUID '9796-932E' are considered. |
63 | + This is to avoid this datasource claiming ConfigDrive. This does |
64 | + mean that 1 in 8^16 (~4 billion) Xen ConfigDrive systems will be |
65 | + incorrectly identified as IBMCloud. |
66 | + |
67 | +TODO: |
68 | + * is uuid (/sys/hypervisor/uuid) stable for life of an instance? |
69 | + it seems it is not the same as data's uuid in the os_code case |
70 | + but is in the template case. |
71 | + |
72 | +""" |
73 | +import base64 |
74 | +import json |
75 | +import os |
76 | + |
77 | +from cloudinit import log as logging |
78 | +from cloudinit import sources |
79 | +from cloudinit.sources.helpers import openstack |
80 | +from cloudinit import util |
81 | + |
82 | +LOG = logging.getLogger(__name__) |
83 | + |
84 | +IBM_CONFIG_UUID = "9796-932E" |
85 | + |
86 | + |
87 | +class Platforms(object): |
88 | + TEMPLATE_LIVE_METADATA = "Template/Live/Metadata" |
89 | + TEMPLATE_LIVE_NODATA = "UNABLE TO BE IDENTIFIED." |
90 | + TEMPLATE_PROVISIONING_METADATA = "Template/Provisioning/Metadata" |
91 | + TEMPLATE_PROVISIONING_NODATA = "Template/Provisioning/No-Metadata" |
92 | + OS_CODE = "OS-Code/Live" |
93 | + |
94 | + |
95 | +PROVISIONING = ( |
96 | + Platforms.TEMPLATE_PROVISIONING_METADATA, |
97 | + Platforms.TEMPLATE_PROVISIONING_NODATA) |
98 | + |
99 | + |
100 | +class DataSourceIBMCloud(sources.DataSource): |
101 | + |
102 | + dsname = 'IBMCloud' |
103 | + system_uuid = None |
104 | + |
105 | + def __init__(self, sys_cfg, distro, paths): |
106 | + super(DataSourceIBMCloud, self).__init__(sys_cfg, distro, paths) |
107 | + self.source = None |
108 | + self._network_config = None |
109 | + self.network_json = None |
110 | + self.platform = None |
111 | + |
112 | + def __str__(self): |
113 | + root = super(DataSourceIBMCloud, self).__str__() |
114 | + mstr = "%s [%s %s]" % (root, self.platform, self.source) |
115 | + return mstr |
116 | + |
117 | + def _get_data(self): |
118 | + results = read_md() |
119 | + if results is None: |
120 | + return False |
121 | + |
122 | + self.source = results['source'] |
123 | + self.platform = results['platform'] |
124 | + self.metadata = results['metadata'] |
125 | + self.userdata_raw = results.get('userdata') |
126 | + self.network_json = results.get('networkdata') |
127 | + vd = results.get('vendordata') |
128 | + self.vendordata_pure = vd |
129 | + self.system_uuid = results['system-uuid'] |
130 | + try: |
131 | + self.vendordata_raw = sources.convert_vendordata(vd) |
132 | + except ValueError as e: |
133 | + LOG.warning("Invalid content in vendor-data: %s", e) |
134 | + self.vendordata_raw = None |
135 | + |
136 | + return True |
137 | + |
138 | + def check_instance_id(self, sys_cfg): |
139 | + """quickly (local check only) if self.instance_id is still valid |
140 | + |
141 | + in Template mode, the system uuid (/sys/hypervisor/uuid) is the |
142 | + same as found in the METADATA disk. But that is not true in OS_CODE |
143 | + mode. So we read the system_uuid and keep that for later compare.""" |
144 | + if self.system_uuid is None: |
145 | + return False |
146 | + return self.system_uuid == _read_system_uuid() |
147 | + |
148 | + @property |
149 | + def network_config(self): |
150 | + if self.platform != Platforms.OS_CODE: |
151 | + # If deployed from template, an agent in the provisioning |
152 | + # environment handles networking configuration. Not cloud-init. |
153 | + return {'config': 'disabled', 'version': 1} |
154 | + if self._network_config is None: |
155 | + if self.network_json is not None: |
156 | + LOG.debug("network config provided via network_json") |
157 | + self._network_config = openstack.convert_net_json( |
158 | + self.network_json, known_macs=None) |
159 | + else: |
160 | + LOG.debug("no network configuration available.") |
161 | + return self._network_config |
162 | + |
163 | + |
164 | +def _read_system_uuid(): |
165 | + uuid_path = "/sys/hypervisor/uuid" |
166 | + if not os.path.isfile(uuid_path): |
167 | + return None |
168 | + return util.load_file(uuid_path).strip().lower() |
169 | + |
170 | + |
171 | +def _is_xen(): |
172 | + return os.path.exists("/proc/xen") |
173 | + |
174 | + |
175 | +def _is_ibm_provisioning(): |
176 | + return os.path.exists("/root/provisioningConfiguration.cfg") |
177 | + |
178 | + |
179 | +def get_ibm_platform(): |
180 | + """Return a tuple (Platform, path) |
181 | + |
182 | + If this is Not IBM cloud, then the return value is (None, None). |
183 | + An instance in provisioning mode is considered running on IBM cloud.""" |
184 | + label_mdata = "METADATA" |
185 | + label_cfg2 = "CONFIG-2" |
186 | + not_found = (None, None) |
187 | + |
188 | + if not _is_xen(): |
189 | + return not_found |
190 | + |
191 | + # fslabels contains only the first entry with a given label. |
192 | + fslabels = {} |
193 | + try: |
194 | + devs = util.blkid() |
195 | + except util.ProcessExecutionError as e: |
196 | + LOG.warning("Failed to run blkid: %s", e) |
197 | + return (None, None) |
198 | + |
199 | + for dev in sorted(devs.keys()): |
200 | + data = devs[dev] |
201 | + label = data.get("LABEL", "").upper() |
202 | + uuid = data.get("UUID", "").upper() |
203 | + if label not in (label_mdata, label_cfg2): |
204 | + continue |
205 | + if label in fslabels: |
206 | + LOG.warning("Duplicate fslabel '%s'. existing=%s current=%s", |
207 | + label, fslabels[label], data) |
208 | + continue |
209 | + if label == label_cfg2 and uuid != IBM_CONFIG_UUID: |
210 | + LOG.debug("Skipping %s with LABEL=%s due to uuid != %s: %s", |
211 | + dev, label, uuid, data) |
212 | + continue |
213 | + fslabels[label] = data |
214 | + |
215 | + metadata_path = fslabels.get(label_mdata, {}).get('DEVNAME') |
216 | + cfg2_path = fslabels.get(label_cfg2, {}).get('DEVNAME') |
217 | + |
218 | + if cfg2_path: |
219 | + return (Platforms.OS_CODE, cfg2_path) |
220 | + elif metadata_path: |
221 | + if _is_ibm_provisioning(): |
222 | + return (Platforms.TEMPLATE_PROVISIONING_METADATA, metadata_path) |
223 | + else: |
224 | + return (Platforms.TEMPLATE_LIVE_METADATA, metadata_path) |
225 | + elif _is_ibm_provisioning(): |
226 | + return (Platforms.TEMPLATE_PROVISIONING_NODATA, None) |
227 | + return not_found |
228 | + |
229 | + |
230 | +def read_md(): |
231 | + """Read data from IBM Cloud. |
232 | + |
233 | + @return: None if not running on IBM Cloud. |
234 | + dictionary with guaranteed fields: metadata, version |
235 | + and optional fields: userdata, vendordata, networkdata. |
236 | + Also includes the system uuid from /sys/hypervisor/uuid.""" |
237 | + platform, path = get_ibm_platform() |
238 | + if platform is None: |
239 | + LOG.debug("This is not an IBMCloud platform.") |
240 | + return None |
241 | + elif platform in PROVISIONING: |
242 | + LOG.debug("Cloud-init is disabled during provisioning: %s.", |
243 | + platform) |
244 | + return None |
245 | + |
246 | + ret = {'platform': platform, 'source': path, |
247 | + 'system-uuid': _read_system_uuid()} |
248 | + |
249 | + try: |
250 | + if os.path.isdir(path): |
251 | + results = metadata_from_dir(path) |
252 | + else: |
253 | + results = util.mount_cb(path, metadata_from_dir) |
254 | + except BrokenMetadata as e: |
255 | + raise RuntimeError( |
256 | + "Failed reading IBM config disk (platform=%s path=%s): %s" % |
257 | + (platform, path, e)) |
258 | + |
259 | + ret.update(results) |
260 | + return ret |
261 | + |
262 | + |
263 | +class BrokenMetadata(IOError): |
264 | + pass |
265 | + |
266 | + |
267 | +def metadata_from_dir(source_dir): |
268 | + def opath(fname): |
269 | + return os.path.join("openstack", "latest", fname) |
270 | + |
271 | + def load_json_bytes(blob): |
272 | + return json.loads(blob.decode('utf-8')) |
273 | + |
274 | + files = [ |
275 | + # tuples of (results_name, path, translator) |
276 | + ('metadata_raw', opath('meta_data.json'), load_json_bytes), |
277 | + ('userdata', opath('user_data'), None), |
278 | + ('vendordata', opath('vendor_data.json'), load_json_bytes), |
279 | + ('networkdata', opath('network_data.json'), load_json_bytes), |
280 | + ] |
281 | + |
282 | + results = {} |
283 | + for (name, path, transl) in files: |
284 | + fpath = os.path.join(source_dir, path) |
285 | + raw = None |
286 | + try: |
287 | + raw = util.load_file(fpath, decode=False) |
288 | + except IOError as e: |
289 | + LOG.debug("Failed reading path '%s': %s", fpath, e) |
290 | + |
291 | + if raw is None or transl is None: |
292 | + data = raw |
293 | + else: |
294 | + try: |
295 | + data = transl(raw) |
296 | + except Exception as e: |
297 | + raise BrokenMetadata("Failed decoding %s: %s" % (path, e)) |
298 | + |
299 | + results[name] = data |
300 | + |
301 | + if results.get('metadata_raw') is None: |
302 | + raise BrokenMetadata( |
303 | + "%s missing required file 'meta_data.json'" % source_dir) |
304 | + |
305 | + results['metadata'] = {} |
306 | + |
307 | + md_raw = results['metadata_raw'] |
308 | + md = results['metadata'] |
309 | + if 'random_seed' in md_raw: |
310 | + try: |
311 | + md['random_seed'] = base64.b64decode(md_raw['random_seed']) |
312 | + except (ValueError, TypeError) as e: |
313 | + raise BrokenMetadata( |
314 | + "Badly formatted metadata random_seed entry: %s" % e) |
315 | + |
316 | + renames = ( |
317 | + ('public_keys', 'public-keys'), ('hostname', 'local-hostname'), |
318 | + ('uuid', 'instance-id')) |
319 | + for mdname, newname in renames: |
320 | + if mdname in md_raw: |
321 | + md[newname] = md_raw[mdname] |
322 | + |
323 | + return results |
324 | + |
325 | + |
326 | +# Used to match classes to dependencies |
327 | +datasources = [ |
328 | + (DataSourceIBMCloud, (sources.DEP_FILESYSTEM,)), |
329 | +] |
330 | + |
331 | + |
332 | +# Return a list of data sources that match this set of dependencies |
333 | +def get_datasource_list(depends): |
334 | + return sources.list_from_depends(depends, datasources) |
335 | + |
336 | + |
337 | +if __name__ == "__main__": |
338 | + import argparse |
339 | + |
340 | + parser = argparse.ArgumentParser(description='Query IBM Cloud Metadata') |
341 | + args = parser.parse_args() |
342 | + data = read_md() |
343 | + print(util.json_dumps(data)) |
344 | + |
345 | +# vi: ts=4 expandtab |
346 | diff --git a/cloudinit/sources/helpers/openstack.py b/cloudinit/sources/helpers/openstack.py |
347 | index 26f3168..82821a2 100644 |
348 | --- a/cloudinit/sources/helpers/openstack.py |
349 | +++ b/cloudinit/sources/helpers/openstack.py |
350 | @@ -329,9 +329,9 @@ class BaseReader(object): |
351 | |
352 | |
353 | class ConfigDriveReader(BaseReader): |
354 | - def __init__(self, base_path): |
355 | + def __init__(self, base_path, versions=None): |
356 | super(ConfigDriveReader, self).__init__(base_path) |
357 | - self._versions = None |
358 | + self._versions = versions |
359 | |
360 | def _path_join(self, base, *add_ons): |
361 | components = [base] + list(add_ons) |
362 | diff --git a/cloudinit/tests/test_util.py b/cloudinit/tests/test_util.py |
363 | index d30643d..3f37dbb 100644 |
364 | --- a/cloudinit/tests/test_util.py |
365 | +++ b/cloudinit/tests/test_util.py |
366 | @@ -3,6 +3,7 @@ |
367 | """Tests for cloudinit.util""" |
368 | |
369 | import logging |
370 | +from textwrap import dedent |
371 | |
372 | import cloudinit.util as util |
373 | |
374 | @@ -140,4 +141,75 @@ class TestGetHostnameFqdn(CiTestCase): |
375 | [{'fqdn': True, 'metadata_only': True}, |
376 | {'metadata_only': True}], mycloud.calls) |
377 | |
378 | + |
379 | +class TestBlkid(CiTestCase): |
380 | + ids = { |
381 | + "id01": "1111-1111", |
382 | + "id02": "22222222-2222", |
383 | + "id03": "33333333-3333", |
384 | + "id04": "44444444-4444", |
385 | + "id05": "55555555-5555-5555-5555-555555555555", |
386 | + "id06": "66666666-6666-6666-6666-666666666666", |
387 | + "id07": "52894610484658920398", |
388 | + "id08": "86753098675309867530", |
389 | + "id09": "99999999-9999-9999-9999-999999999999", |
390 | + } |
391 | + |
392 | + blkid_out = dedent("""\ |
393 | + /dev/loop0: TYPE="squashfs" |
394 | + /dev/loop1: TYPE="squashfs" |
395 | + /dev/loop2: TYPE="squashfs" |
396 | + /dev/loop3: TYPE="squashfs" |
397 | + /dev/sda1: UUID="{id01}" TYPE="vfat" PARTUUID="{id02}" |
398 | + /dev/sda2: UUID="{id03}" TYPE="ext4" PARTUUID="{id04}" |
399 | + /dev/sda3: UUID="{id05}" TYPE="ext4" PARTUUID="{id06}" |
400 | + /dev/sda4: LABEL="default" UUID="{id07}" UUID_SUB="{id08}" """ |
401 | + """TYPE="zfs_member" PARTUUID="{id09}" |
402 | + /dev/loop4: TYPE="squashfs" |
403 | + """) |
404 | + |
405 | + maxDiff = None |
406 | + |
407 | + def _get_expected(self): |
408 | + return ({ |
409 | + "/dev/loop0": {"DEVNAME": "/dev/loop0", "TYPE": "squashfs"}, |
410 | + "/dev/loop1": {"DEVNAME": "/dev/loop1", "TYPE": "squashfs"}, |
411 | + "/dev/loop2": {"DEVNAME": "/dev/loop2", "TYPE": "squashfs"}, |
412 | + "/dev/loop3": {"DEVNAME": "/dev/loop3", "TYPE": "squashfs"}, |
413 | + "/dev/loop4": {"DEVNAME": "/dev/loop4", "TYPE": "squashfs"}, |
414 | + "/dev/sda1": {"DEVNAME": "/dev/sda1", "TYPE": "vfat", |
415 | + "UUID": self.ids["id01"], |
416 | + "PARTUUID": self.ids["id02"]}, |
417 | + "/dev/sda2": {"DEVNAME": "/dev/sda2", "TYPE": "ext4", |
418 | + "UUID": self.ids["id03"], |
419 | + "PARTUUID": self.ids["id04"]}, |
420 | + "/dev/sda3": {"DEVNAME": "/dev/sda3", "TYPE": "ext4", |
421 | + "UUID": self.ids["id05"], |
422 | + "PARTUUID": self.ids["id06"]}, |
423 | + "/dev/sda4": {"DEVNAME": "/dev/sda4", "TYPE": "zfs_member", |
424 | + "LABEL": "default", |
425 | + "UUID": self.ids["id07"], |
426 | + "UUID_SUB": self.ids["id08"], |
427 | + "PARTUUID": self.ids["id09"]}, |
428 | + }) |
429 | + |
430 | + @mock.patch("cloudinit.util.subp") |
431 | + def test_functional_blkid(self, m_subp): |
432 | + m_subp.return_value = ( |
433 | + self.blkid_out.format(**self.ids), "") |
434 | + self.assertEqual(self._get_expected(), util.blkid()) |
435 | + m_subp.assert_called_with(["blkid", "-o", "full"], capture=True, |
436 | + decode="replace") |
437 | + |
438 | + @mock.patch("cloudinit.util.subp") |
439 | + def test_blkid_no_cache_uses_no_cache(self, m_subp): |
440 | + """blkid should turn off cache if disable_cache is true.""" |
441 | + m_subp.return_value = ( |
442 | + self.blkid_out.format(**self.ids), "") |
443 | + self.assertEqual(self._get_expected(), |
444 | + util.blkid(disable_cache=True)) |
445 | + m_subp.assert_called_with(["blkid", "-o", "full", "-c", "/dev/null"], |
446 | + capture=True, decode="replace") |
447 | + |
448 | + |
449 | # vi: ts=4 expandtab |
450 | diff --git a/cloudinit/util.py b/cloudinit/util.py |
451 | index cae8b19..7ae62cc 100644 |
452 | --- a/cloudinit/util.py |
453 | +++ b/cloudinit/util.py |
454 | @@ -1237,6 +1237,30 @@ def find_devs_with(criteria=None, oformat='device', |
455 | return entries |
456 | |
457 | |
458 | +def blkid(devs=None, disable_cache=False): |
459 | + if devs is None: |
460 | + devs = [] |
461 | + else: |
462 | + devs = list(devs) |
463 | + |
464 | + cmd = ['blkid', '-o', 'full'] |
465 | + if disable_cache: |
466 | + cmd.extend(['-c', '/dev/null']) |
467 | + cmd.extend(devs) |
468 | + |
469 | + # we have to decode with 'replace' as shelx.split (called by |
470 | + # load_shell_content) can't take bytes. So this is potentially |
471 | + # lossy of non-utf-8 chars in blkid output. |
472 | + out, _ = subp(cmd, capture=True, decode="replace") |
473 | + ret = {} |
474 | + for line in out.splitlines(): |
475 | + dev, _, data = line.partition(":") |
476 | + ret[dev] = load_shell_content(data) |
477 | + ret[dev]["DEVNAME"] = dev |
478 | + |
479 | + return ret |
480 | + |
481 | + |
482 | def peek_file(fname, max_bytes): |
483 | LOG.debug("Peeking at %s (max_bytes=%s)", fname, max_bytes) |
484 | with open(fname, 'rb') as ifh: |
485 | diff --git a/tests/unittests/test_datasource/test_ibmcloud.py b/tests/unittests/test_datasource/test_ibmcloud.py |
486 | new file mode 100644 |
487 | index 0000000..894bfb5 |
488 | --- /dev/null |
489 | +++ b/tests/unittests/test_datasource/test_ibmcloud.py |
490 | @@ -0,0 +1,262 @@ |
491 | +# This file is part of cloud-init. See LICENSE file for license information. |
492 | + |
493 | +from cloudinit.sources import DataSourceIBMCloud as ibm |
494 | +from cloudinit.tests import helpers as test_helpers |
495 | + |
496 | +import base64 |
497 | +import copy |
498 | +import json |
499 | +import mock |
500 | +from textwrap import dedent |
501 | + |
502 | +D_PATH = "cloudinit.sources.DataSourceIBMCloud." |
503 | + |
504 | + |
505 | +class TestIBMCloud(test_helpers.CiTestCase): |
506 | + """Test the datasource.""" |
507 | + def setUp(self): |
508 | + super(TestIBMCloud, self).setUp() |
509 | + pass |
510 | + |
511 | + |
512 | +@mock.patch(D_PATH + "_is_xen", return_value=True) |
513 | +@mock.patch(D_PATH + "_is_ibm_provisioning") |
514 | +@mock.patch(D_PATH + "util.blkid") |
515 | +class TestGetIBMPlatform(test_helpers.CiTestCase): |
516 | + """Test the get_ibm_platform helper.""" |
517 | + |
518 | + blkid_base = { |
519 | + "/dev/xvda1": { |
520 | + "DEVNAME": "/dev/xvda1", "LABEL": "cloudimg-bootfs", |
521 | + "TYPE": "ext3"}, |
522 | + "/dev/xvda2": { |
523 | + "DEVNAME": "/dev/xvda2", "LABEL": "cloudimg-rootfs", |
524 | + "TYPE": "ext4"}, |
525 | + } |
526 | + |
527 | + blkid_metadata_disk = { |
528 | + "/dev/xvdh1": { |
529 | + "DEVNAME": "/dev/xvdh1", "LABEL": "METADATA", "TYPE": "vfat", |
530 | + "SEC_TYPE": "msdos", "UUID": "681B-8C5D", |
531 | + "PARTUUID": "3d631e09-01"}, |
532 | + } |
533 | + |
534 | + blkid_oscode_disk = { |
535 | + "/dev/xvdh": { |
536 | + "DEVNAME": "/dev/xvdh", "LABEL": "config-2", "TYPE": "vfat", |
537 | + "SEC_TYPE": "msdos", "UUID": ibm.IBM_CONFIG_UUID} |
538 | + } |
539 | + |
540 | + def setUp(self): |
541 | + self.blkid_metadata = copy.deepcopy(self.blkid_base) |
542 | + self.blkid_metadata.update(copy.deepcopy(self.blkid_metadata_disk)) |
543 | + |
544 | + self.blkid_oscode = copy.deepcopy(self.blkid_base) |
545 | + self.blkid_oscode.update(copy.deepcopy(self.blkid_oscode_disk)) |
546 | + |
547 | + def test_id_template_live_metadata(self, m_blkid, m_is_prov, _m_xen): |
548 | + """identify TEMPLATE_LIVE_METADATA.""" |
549 | + m_blkid.return_value = self.blkid_metadata |
550 | + m_is_prov.return_value = False |
551 | + self.assertEqual( |
552 | + (ibm.Platforms.TEMPLATE_LIVE_METADATA, "/dev/xvdh1"), |
553 | + ibm.get_ibm_platform()) |
554 | + |
555 | + def test_id_template_prov_metadata(self, m_blkid, m_is_prov, _m_xen): |
556 | + """identify TEMPLATE_PROVISIONING_METADATA.""" |
557 | + m_blkid.return_value = self.blkid_metadata |
558 | + m_is_prov.return_value = True |
559 | + self.assertEqual( |
560 | + (ibm.Platforms.TEMPLATE_PROVISIONING_METADATA, "/dev/xvdh1"), |
561 | + ibm.get_ibm_platform()) |
562 | + |
563 | + def test_id_template_prov_nodata(self, m_blkid, m_is_prov, _m_xen): |
564 | + """identify TEMPLATE_PROVISIONING_NODATA.""" |
565 | + m_blkid.return_value = self.blkid_base |
566 | + m_is_prov.return_value = True |
567 | + self.assertEqual( |
568 | + (ibm.Platforms.TEMPLATE_PROVISIONING_NODATA, None), |
569 | + ibm.get_ibm_platform()) |
570 | + |
571 | + def test_id_os_code(self, m_blkid, m_is_prov, _m_xen): |
572 | + """Identify OS_CODE.""" |
573 | + m_blkid.return_value = self.blkid_oscode |
574 | + m_is_prov.return_value = False |
575 | + self.assertEqual((ibm.Platforms.OS_CODE, "/dev/xvdh"), |
576 | + ibm.get_ibm_platform()) |
577 | + |
578 | + def test_id_os_code_must_match_uuid(self, m_blkid, m_is_prov, _m_xen): |
579 | + """Test against false positive on openstack with non-ibm UUID.""" |
580 | + blkid = self.blkid_oscode |
581 | + blkid["/dev/xvdh"]["UUID"] = "9999-9999" |
582 | + m_blkid.return_value = blkid |
583 | + m_is_prov.return_value = False |
584 | + self.assertEqual((None, None), ibm.get_ibm_platform()) |
585 | + |
586 | + |
587 | +@mock.patch(D_PATH + "_read_system_uuid", return_value=None) |
588 | +@mock.patch(D_PATH + "get_ibm_platform") |
589 | +class TestReadMD(test_helpers.CiTestCase): |
590 | + """Test the read_datasource helper.""" |
591 | + |
592 | + template_md = { |
593 | + "files": [], |
594 | + "network_config": {"content_path": "/content/interfaces"}, |
595 | + "hostname": "ci-fond-ram", |
596 | + "name": "ci-fond-ram", |
597 | + "domain": "testing.ci.cloud-init.org", |
598 | + "meta": {"dsmode": "net"}, |
599 | + "uuid": "8e636730-9f5d-c4a5-327c-d7123c46e82f", |
600 | + "public_keys": {"1091307": "ssh-rsa AAAAB3NzaC1...Hw== ci-pubkey"}, |
601 | + } |
602 | + |
603 | + oscode_md = { |
604 | + "hostname": "ci-grand-gannet.testing.ci.cloud-init.org", |
605 | + "name": "ci-grand-gannet", |
606 | + "uuid": "2f266908-8e6c-4818-9b5c-42e9cc66a785", |
607 | + "random_seed": "bm90LXJhbmRvbQo=", |
608 | + "crypt_key": "ssh-rsa AAAAB3NzaC1yc2..n6z/", |
609 | + "configuration_token": "eyJhbGciOi..M3ZA", |
610 | + "public_keys": {"1091307": "ssh-rsa AAAAB3N..Hw== ci-pubkey"}, |
611 | + } |
612 | + |
613 | + content_interfaces = dedent("""\ |
614 | + auto lo |
615 | + iface lo inet loopback |
616 | + |
617 | + auto eth0 |
618 | + allow-hotplug eth0 |
619 | + iface eth0 inet static |
620 | + address 10.82.43.5 |
621 | + netmask 255.255.255.192 |
622 | + """) |
623 | + |
624 | + userdata = b"#!/bin/sh\necho hi mom\n" |
625 | + # meta.js file gets json encoded userdata as a list. |
626 | + meta_js = '["#!/bin/sh\necho hi mom\n"]' |
627 | + vendor_data = { |
628 | + "cloud-init": "#!/bin/bash\necho 'root:$6$5ab01p1m1' | chpasswd -e"} |
629 | + |
630 | + network_data = { |
631 | + "links": [ |
632 | + {"id": "interface_29402281", "name": "eth0", "mtu": None, |
633 | + "type": "phy", "ethernet_mac_address": "06:00:f1:bd:da:25"}, |
634 | + {"id": "interface_29402279", "name": "eth1", "mtu": None, |
635 | + "type": "phy", "ethernet_mac_address": "06:98:5e:d0:7f:86"} |
636 | + ], |
637 | + "networks": [ |
638 | + {"id": "network_109887563", "link": "interface_29402281", |
639 | + "type": "ipv4", "ip_address": "10.82.43.2", |
640 | + "netmask": "255.255.255.192", |
641 | + "routes": [ |
642 | + {"network": "10.0.0.0", "netmask": "255.0.0.0", |
643 | + "gateway": "10.82.43.1"}, |
644 | + {"network": "161.26.0.0", "netmask": "255.255.0.0", |
645 | + "gateway": "10.82.43.1"}]}, |
646 | + {"id": "network_109887551", "link": "interface_29402279", |
647 | + "type": "ipv4", "ip_address": "108.168.194.252", |
648 | + "netmask": "255.255.255.248", |
649 | + "routes": [ |
650 | + {"network": "0.0.0.0", "netmask": "0.0.0.0", |
651 | + "gateway": "108.168.194.249"}]} |
652 | + ], |
653 | + "services": [ |
654 | + {"type": "dns", "address": "10.0.80.11"}, |
655 | + {"type": "dns", "address": "10.0.80.12"} |
656 | + ], |
657 | + } |
658 | + |
659 | + sysuuid = '7f79ebf5-d791-43c3-a723-854e8389d59f' |
660 | + |
661 | + def _get_expected_metadata(self, os_md): |
662 | + """return expected 'metadata' for data loaded fro meta_data.json.""" |
663 | + os_md = copy.deepcopy(os_md) |
664 | + renames = ( |
665 | + ('hostname', 'local-hostname'), |
666 | + ('uuid', 'instance-id'), |
667 | + ('public_keys', 'public-keys')) |
668 | + ret = {} |
669 | + for osname, mdname in renames: |
670 | + if osname in os_md: |
671 | + ret[mdname] = os_md[osname] |
672 | + if 'random_seed' in os_md: |
673 | + ret['random_seed'] = base64.b64decode(os_md['random_seed']) |
674 | + |
675 | + return ret |
676 | + |
677 | + def test_provisioning_md(self, m_platform, m_sysuuid): |
678 | + """Provisioning env with a metadata disk should return None.""" |
679 | + m_platform.return_value = ( |
680 | + ibm.Platforms.TEMPLATE_PROVISIONING_METADATA, "/dev/xvdh") |
681 | + self.assertIsNone(ibm.read_md()) |
682 | + |
683 | + def test_provisioning_no_metadata(self, m_platform, m_sysuuid): |
684 | + """Provisioning env with no metadata disk should return None.""" |
685 | + m_platform.return_value = ( |
686 | + ibm.Platforms.TEMPLATE_PROVISIONING_NODATA, None) |
687 | + self.assertIsNone(ibm.read_md()) |
688 | + |
689 | + def test_provisioning_not_ibm(self, m_platform, m_sysuuid): |
690 | + """Provisioning env but not identified as IBM should return None.""" |
691 | + m_platform.return_value = (None, None) |
692 | + self.assertIsNone(ibm.read_md()) |
693 | + |
694 | + def test_template_live(self, m_platform, m_sysuuid): |
695 | + """Template live environment should be identified.""" |
696 | + tmpdir = self.tmp_dir() |
697 | + m_platform.return_value = ( |
698 | + ibm.Platforms.TEMPLATE_LIVE_METADATA, tmpdir) |
699 | + m_sysuuid.return_value = self.sysuuid |
700 | + |
701 | + test_helpers.populate_dir(tmpdir, { |
702 | + 'openstack/latest/meta_data.json': json.dumps(self.template_md), |
703 | + 'openstack/latest/user_data': self.userdata, |
704 | + 'openstack/content/interfaces': self.content_interfaces, |
705 | + 'meta.js': self.meta_js}) |
706 | + |
707 | + ret = ibm.read_md() |
708 | + self.assertEqual(ibm.Platforms.TEMPLATE_LIVE_METADATA, |
709 | + ret['platform']) |
710 | + self.assertEqual(tmpdir, ret['source']) |
711 | + self.assertEqual(self.userdata, ret['userdata']) |
712 | + self.assertEqual(self._get_expected_metadata(self.template_md), |
713 | + ret['metadata']) |
714 | + self.assertEqual(self.sysuuid, ret['system-uuid']) |
715 | + |
716 | + def test_os_code_live(self, m_platform, m_sysuuid): |
717 | + """Verify an os_code metadata path.""" |
718 | + tmpdir = self.tmp_dir() |
719 | + m_platform.return_value = (ibm.Platforms.OS_CODE, tmpdir) |
720 | + netdata = json.dumps(self.network_data) |
721 | + test_helpers.populate_dir(tmpdir, { |
722 | + 'openstack/latest/meta_data.json': json.dumps(self.oscode_md), |
723 | + 'openstack/latest/user_data': self.userdata, |
724 | + 'openstack/latest/vendor_data.json': json.dumps(self.vendor_data), |
725 | + 'openstack/latest/network_data.json': netdata, |
726 | + }) |
727 | + |
728 | + ret = ibm.read_md() |
729 | + self.assertEqual(ibm.Platforms.OS_CODE, ret['platform']) |
730 | + self.assertEqual(tmpdir, ret['source']) |
731 | + self.assertEqual(self.userdata, ret['userdata']) |
732 | + self.assertEqual(self._get_expected_metadata(self.oscode_md), |
733 | + ret['metadata']) |
734 | + |
735 | + def test_os_code_live_no_userdata(self, m_platform, m_sysuuid): |
736 | + """Verify os_code without user-data.""" |
737 | + tmpdir = self.tmp_dir() |
738 | + m_platform.return_value = (ibm.Platforms.OS_CODE, tmpdir) |
739 | + test_helpers.populate_dir(tmpdir, { |
740 | + 'openstack/latest/meta_data.json': json.dumps(self.oscode_md), |
741 | + 'openstack/latest/vendor_data.json': json.dumps(self.vendor_data), |
742 | + }) |
743 | + |
744 | + ret = ibm.read_md() |
745 | + self.assertEqual(ibm.Platforms.OS_CODE, ret['platform']) |
746 | + self.assertEqual(tmpdir, ret['source']) |
747 | + self.assertIsNone(ret['userdata']) |
748 | + self.assertEqual(self._get_expected_metadata(self.oscode_md), |
749 | + ret['metadata']) |
750 | + |
751 | + |
752 | +# vi: ts=4 expandtab |
753 | diff --git a/tests/unittests/test_ds_identify.py b/tests/unittests/test_ds_identify.py |
754 | index 85999b7..5364398 100644 |
755 | --- a/tests/unittests/test_ds_identify.py |
756 | +++ b/tests/unittests/test_ds_identify.py |
757 | @@ -9,6 +9,8 @@ from cloudinit import util |
758 | from cloudinit.tests.helpers import ( |
759 | CiTestCase, dir2dict, populate_dir) |
760 | |
761 | +from cloudinit.sources import DataSourceIBMCloud as dsibm |
762 | + |
763 | UNAME_MYSYS = ("Linux bart 4.4.0-62-generic #83-Ubuntu " |
764 | "SMP Wed Jan 18 14:10:15 UTC 2017 x86_64 GNU/Linux") |
765 | UNAME_PPC64EL = ("Linux diamond 4.4.0-83-generic #106-Ubuntu SMP " |
766 | @@ -37,8 +39,8 @@ BLKID_UEFI_UBUNTU = [ |
767 | |
768 | POLICY_FOUND_ONLY = "search,found=all,maybe=none,notfound=disabled" |
769 | POLICY_FOUND_OR_MAYBE = "search,found=all,maybe=all,notfound=disabled" |
770 | -DI_DEFAULT_POLICY = "search,found=all,maybe=all,notfound=enabled" |
771 | -DI_DEFAULT_POLICY_NO_DMI = "search,found=all,maybe=all,notfound=disabled" |
772 | +DI_DEFAULT_POLICY = "search,found=all,maybe=all,notfound=disabled" |
773 | +DI_DEFAULT_POLICY_NO_DMI = "search,found=all,maybe=all,notfound=enabled" |
774 | DI_EC2_STRICT_ID_DEFAULT = "true" |
775 | OVF_MATCH_STRING = 'http://schemas.dmtf.org/ovf/environment/1' |
776 | |
777 | @@ -64,6 +66,9 @@ P_SYS_VENDOR = "sys/class/dmi/id/sys_vendor" |
778 | P_SEED_DIR = "var/lib/cloud/seed" |
779 | P_DSID_CFG = "etc/cloud/ds-identify.cfg" |
780 | |
781 | +IBM_PROVISIONING_CHECK_PATH = "/root/provisioningConfiguration.cfg" |
782 | +IBM_CONFIG_UUID = "9796-932E" |
783 | + |
784 | MOCK_VIRT_IS_KVM = {'name': 'detect_virt', 'RET': 'kvm', 'ret': 0} |
785 | MOCK_VIRT_IS_VMWARE = {'name': 'detect_virt', 'RET': 'vmware', 'ret': 0} |
786 | MOCK_VIRT_IS_XEN = {'name': 'detect_virt', 'RET': 'xen', 'ret': 0} |
787 | @@ -239,6 +244,57 @@ class TestDsIdentify(CiTestCase): |
788 | self._test_ds_found('ConfigDriveUpper') |
789 | return |
790 | |
791 | + def test_ibmcloud_template_userdata_in_provisioning(self): |
792 | + """Template provisioned with user-data during provisioning stage. |
793 | + |
794 | + Template provisioning with user-data has METADATA disk, |
795 | + datasource should return not found.""" |
796 | + data = copy.deepcopy(VALID_CFG['IBMCloud-metadata']) |
797 | + data['files'] = {IBM_PROVISIONING_CHECK_PATH: 'xxx'} |
798 | + return self._check_via_dict(data, RC_NOT_FOUND) |
799 | + |
800 | + def test_ibmcloud_template_userdata(self): |
801 | + """Template provisioned with user-data first boot. |
802 | + |
803 | + Template provisioning with user-data has METADATA disk. |
804 | + datasource should return found.""" |
805 | + self._test_ds_found('IBMCloud-metadata') |
806 | + |
807 | + def test_ibmcloud_template_no_userdata_in_provisioning(self): |
808 | + """Template provisioned with no user-data during provisioning. |
809 | + |
810 | + no disks attached. Datasource should return not found.""" |
811 | + data = copy.deepcopy(VALID_CFG['IBMCloud-nodisks']) |
812 | + data['files'] = {IBM_PROVISIONING_CHECK_PATH: 'xxx'} |
813 | + return self._check_via_dict(data, RC_NOT_FOUND) |
814 | + |
815 | + def test_ibmcloud_template_no_userdata(self): |
816 | + """Template provisioned with no user-data first boot. |
817 | + |
818 | + no disks attached. Datasource should return found.""" |
819 | + self._check_via_dict(VALID_CFG['IBMCloud-nodisks'], RC_NOT_FOUND) |
820 | + |
821 | + def test_ibmcloud_os_code(self): |
822 | + """Launched by os code always has config-2 disk.""" |
823 | + self._test_ds_found('IBMCloud-config-2') |
824 | + |
825 | + def test_ibmcloud_os_code_different_uuid(self): |
826 | + """IBM cloud config-2 disks must be explicit match on UUID. |
827 | + |
828 | + If the UUID is not 9796-932E then we actually expect ConfigDrive.""" |
829 | + data = copy.deepcopy(VALID_CFG['IBMCloud-config-2']) |
830 | + offset = None |
831 | + for m, d in enumerate(data['mocks']): |
832 | + if d.get('name') == "blkid": |
833 | + offset = m |
834 | + break |
835 | + if not offset: |
836 | + raise ValueError("Expected to find 'blkid' mock, but did not.") |
837 | + data['mocks'][offset]['out'] = d['out'].replace(dsibm.IBM_CONFIG_UUID, |
838 | + "DEAD-BEEF") |
839 | + self._check_via_dict( |
840 | + data, rc=RC_FOUND, dslist=['ConfigDrive', DS_NONE]) |
841 | + |
842 | def test_policy_disabled(self): |
843 | """A Builtin policy of 'disabled' should return not found. |
844 | |
845 | @@ -452,7 +508,7 @@ VALID_CFG = { |
846 | }, |
847 | 'Ec2-xen': { |
848 | 'ds': 'Ec2', |
849 | - 'mocks': [{'name': 'detect_virt', 'RET': 'xen', 'ret': 0}], |
850 | + 'mocks': [MOCK_VIRT_IS_XEN], |
851 | 'files': { |
852 | 'sys/hypervisor/uuid': 'ec2c6e2f-5fac-4fc7-9c82-74127ec14bbb\n' |
853 | }, |
854 | @@ -579,6 +635,48 @@ VALID_CFG = { |
855 | 'ds': 'Hetzner', |
856 | 'files': {P_SYS_VENDOR: 'Hetzner\n'}, |
857 | }, |
858 | + 'IBMCloud-metadata': { |
859 | + 'ds': 'IBMCloud', |
860 | + 'mocks': [ |
861 | + MOCK_VIRT_IS_XEN, |
862 | + {'name': 'blkid', 'ret': 0, |
863 | + 'out': blkid_out( |
864 | + [{'DEVNAME': 'xvda1', 'TYPE': 'vfat', 'PARTUUID': uuid4()}, |
865 | + {'DEVNAME': 'xvda2', 'TYPE': 'ext4', |
866 | + 'LABEL': 'cloudimg-rootfs', 'PARTUUID': uuid4()}, |
867 | + {'DEVNAME': 'xvdb', 'TYPE': 'vfat', 'LABEL': 'METADATA'}]), |
868 | + }, |
869 | + ], |
870 | + }, |
871 | + 'IBMCloud-config-2': { |
872 | + 'ds': 'IBMCloud', |
873 | + 'mocks': [ |
874 | + MOCK_VIRT_IS_XEN, |
875 | + {'name': 'blkid', 'ret': 0, |
876 | + 'out': blkid_out( |
877 | + [{'DEVNAME': 'xvda1', 'TYPE': 'ext3', 'PARTUUID': uuid4(), |
878 | + 'UUID': uuid4(), 'LABEL': 'cloudimg-bootfs'}, |
879 | + {'DEVNAME': 'xvdb', 'TYPE': 'vfat', 'LABEL': 'config-2', |
880 | + 'UUID': dsibm.IBM_CONFIG_UUID}, |
881 | + {'DEVNAME': 'xvda2', 'TYPE': 'ext4', |
882 | + 'LABEL': 'cloudimg-rootfs', 'PARTUUID': uuid4(), |
883 | + 'UUID': uuid4()}, |
884 | + ]), |
885 | + }, |
886 | + ], |
887 | + }, |
888 | + 'IBMCloud-nodisks': { |
889 | + 'ds': 'IBMCloud', |
890 | + 'mocks': [ |
891 | + MOCK_VIRT_IS_XEN, |
892 | + {'name': 'blkid', 'ret': 0, |
893 | + 'out': blkid_out( |
894 | + [{'DEVNAME': 'xvda1', 'TYPE': 'vfat', 'PARTUUID': uuid4()}, |
895 | + {'DEVNAME': 'xvda2', 'TYPE': 'ext4', |
896 | + 'LABEL': 'cloudimg-rootfs', 'PARTUUID': uuid4()}]), |
897 | + }, |
898 | + ], |
899 | + }, |
900 | } |
901 | |
902 | # vi: ts=4 expandtab |
903 | diff --git a/tools/ds-identify b/tools/ds-identify |
904 | index e2552c8..9a2db5c 100755 |
905 | --- a/tools/ds-identify |
906 | +++ b/tools/ds-identify |
907 | @@ -92,6 +92,7 @@ DI_DMI_SYS_VENDOR="" |
908 | DI_DMI_PRODUCT_SERIAL="" |
909 | DI_DMI_PRODUCT_UUID="" |
910 | DI_FS_LABELS="" |
911 | +DI_FS_UUIDS="" |
912 | DI_ISO9660_DEVS="" |
913 | DI_KERNEL_CMDLINE="" |
914 | DI_VIRT="" |
915 | @@ -114,7 +115,7 @@ DI_DSNAME="" |
916 | # be searched if there is no setting found in config. |
917 | DI_DSLIST_DEFAULT="MAAS ConfigDrive NoCloud AltCloud Azure Bigstep \ |
918 | CloudSigma CloudStack DigitalOcean AliYun Ec2 GCE OpenNebula OpenStack \ |
919 | -OVF SmartOS Scaleway Hetzner" |
920 | +OVF SmartOS Scaleway Hetzner IBMCloud" |
921 | DI_DSLIST="" |
922 | DI_MODE="" |
923 | DI_ON_FOUND="" |
924 | @@ -123,6 +124,8 @@ DI_ON_NOTFOUND="" |
925 | |
926 | DI_EC2_STRICT_ID_DEFAULT="true" |
927 | |
928 | +_IS_IBM_CLOUD="" |
929 | + |
930 | error() { |
931 | set -- "ERROR:" "$@"; |
932 | debug 0 "$@" |
933 | @@ -196,7 +199,7 @@ read_fs_info() { |
934 | return |
935 | fi |
936 | local oifs="$IFS" line="" delim="," |
937 | - local ret=0 out="" labels="" dev="" label="" ftype="" isodevs="" |
938 | + local ret=0 out="" labels="" dev="" label="" ftype="" isodevs="" uuids="" |
939 | out=$(blkid -c /dev/null -o export) || { |
940 | ret=$? |
941 | error "failed running [$ret]: blkid -c /dev/null -o export" |
942 | @@ -219,12 +222,14 @@ read_fs_info() { |
943 | LABEL=*) label="${line#LABEL=}"; |
944 | labels="${labels}${line#LABEL=}${delim}";; |
945 | TYPE=*) ftype=${line#TYPE=};; |
946 | + UUID=*) uuids="${uuids}${line#UUID=}$delim";; |
947 | esac |
948 | done |
949 | [ -n "$dev" -a "$ftype" = "iso9660" ] && |
950 | isodevs="${isodevs} ${dev}=$label" |
951 | |
952 | DI_FS_LABELS="${labels%${delim}}" |
953 | + DI_FS_UUIDS="${uuids%${delim}}" |
954 | DI_ISO9660_DEVS="${isodevs# }" |
955 | } |
956 | |
957 | @@ -437,14 +442,25 @@ dmi_sys_vendor_is() { |
958 | [ "${DI_DMI_SYS_VENDOR}" = "$1" ] |
959 | } |
960 | |
961 | -has_fs_with_label() { |
962 | - local label="$1" |
963 | - case ",${DI_FS_LABELS}," in |
964 | - *,$label,*) return 0;; |
965 | +has_fs_with_uuid() { |
966 | + case ",${DI_FS_UUIDS}," in |
967 | + *,$1,*) return 0;; |
968 | esac |
969 | return 1 |
970 | } |
971 | |
972 | +has_fs_with_label() { |
973 | + # has_fs_with_label(label1[ ,label2 ..]) |
974 | + # return 0 if a there is a filesystem that matches any of the labels. |
975 | + local label="" |
976 | + for label in "$@"; do |
977 | + case ",${DI_FS_LABELS}," in |
978 | + *,$label,*) return 0;; |
979 | + esac |
980 | + done |
981 | + return 1 |
982 | +} |
983 | + |
984 | nocase_equal() { |
985 | # nocase_equal(a, b) |
986 | # return 0 if case insenstive comparision a.lower() == b.lower() |
987 | @@ -583,6 +599,8 @@ dscheck_NoCloud() { |
988 | case " ${DI_DMI_PRODUCT_SERIAL} " in |
989 | *\ ds=nocloud*) return ${DS_FOUND};; |
990 | esac |
991 | + |
992 | + is_ibm_cloud && return ${DS_NOT_FOUND} |
993 | for d in nocloud nocloud-net; do |
994 | check_seed_dir "$d" meta-data user-data && return ${DS_FOUND} |
995 | check_writable_seed_dir "$d" meta-data user-data && return ${DS_FOUND} |
996 | @@ -594,9 +612,8 @@ dscheck_NoCloud() { |
997 | } |
998 | |
999 | check_configdrive_v2() { |
1000 | - if has_fs_with_label "config-2"; then |
1001 | - return ${DS_FOUND} |
1002 | - elif has_fs_with_label "CONFIG-2"; then |
1003 | + is_ibm_cloud && return ${DS_NOT_FOUND} |
1004 | + if has_fs_with_label CONFIG-2 config-2; then |
1005 | return ${DS_FOUND} |
1006 | fi |
1007 | # look in /config-drive <vlc>/seed/config_drive for a directory |
1008 | @@ -988,6 +1005,36 @@ dscheck_Hetzner() { |
1009 | return ${DS_NOT_FOUND} |
1010 | } |
1011 | |
1012 | +is_ibm_provisioning() { |
1013 | + [ -f "${PATH_ROOT}/root/provisioningConfiguration.cfg" ] |
1014 | +} |
1015 | + |
1016 | +is_ibm_cloud() { |
1017 | + cached "${_IS_IBM_CLOUD}" && return ${_IS_IBM_CLOUD} |
1018 | + local ret=1 |
1019 | + if [ "$DI_VIRT" = "xen" ]; then |
1020 | + if is_ibm_provisioning; then |
1021 | + ret=0 |
1022 | + elif has_fs_with_label METADATA metadata; then |
1023 | + ret=0 |
1024 | + elif has_fs_with_uuid 9796-932E && |
1025 | + has_fs_with_label CONFIG-2 config-2; then |
1026 | + ret=0 |
1027 | + fi |
1028 | + fi |
1029 | + _IS_IBM_CLOUD=$ret |
1030 | + return $ret |
1031 | +} |
1032 | + |
1033 | +dscheck_IBMCloud() { |
1034 | + if is_ibm_provisioning; then |
1035 | + debug 1 "cloud-init disabled during provisioning on IBMCloud" |
1036 | + return ${DS_NOT_FOUND} |
1037 | + fi |
1038 | + is_ibm_cloud && return ${DS_FOUND} |
1039 | + return ${DS_NOT_FOUND} |
1040 | +} |
1041 | + |
1042 | collect_info() { |
1043 | read_virt |
1044 | read_pid1_product_name |
FAILED: Continuous integration, rev:1ebcb6a85ca 6865f1aa6657ce3 2e10b5436702a4 /jenkins. ubuntu. com/server/ job/cloud- init-ci/ 893/
https:/
Executed test runs:
SUCCESS: Checkout
FAILED: Unit & Style Tests
Click here to trigger a rebuild: /jenkins. ubuntu. com/server/ job/cloud- init-ci/ 893/rebuild
https:/