Merge ~chad.smith/cloud-init:ubuntu/bionic into cloud-init:ubuntu/bionic
- Git
- lp:~chad.smith/cloud-init
- ubuntu/bionic
- Merge into ubuntu/bionic
Status: | Merged |
---|---|
Merged at revision: | ef4aa258cb508c9be6343fcbefa0735066f4ad21 |
Proposed branch: | ~chad.smith/cloud-init:ubuntu/bionic |
Merge into: | cloud-init:ubuntu/bionic |
Diff against target: |
6964 lines (+3990/-570) 81 files modified
.github/pull_request_template.md (+9/-0) ChangeLog (+36/-0) cloudinit/analyze/__main__.py (+86/-2) cloudinit/analyze/show.py (+192/-10) cloudinit/analyze/tests/test_boot.py (+170/-0) cloudinit/apport.py (+1/-0) cloudinit/config/cc_apt_configure.py (+3/-1) cloudinit/config/cc_lxd.py (+1/-1) cloudinit/config/cc_set_passwords.py (+34/-19) cloudinit/config/cc_ssh.py (+55/-0) cloudinit/config/cc_ubuntu_drivers.py (+49/-1) cloudinit/config/tests/test_ssh.py (+166/-0) cloudinit/config/tests/test_ubuntu_drivers.py (+81/-18) cloudinit/distros/__init__.py (+22/-22) cloudinit/distros/arch.py (+14/-0) cloudinit/distros/debian.py (+2/-2) cloudinit/distros/freebsd.py (+16/-16) cloudinit/distros/opensuse.py (+2/-0) cloudinit/distros/parsers/sys_conf.py (+7/-0) cloudinit/distros/ubuntu.py (+15/-0) cloudinit/net/__init__.py (+112/-43) cloudinit/net/cmdline.py (+16/-9) cloudinit/net/dhcp.py (+90/-0) cloudinit/net/network_state.py (+12/-4) cloudinit/net/sysconfig.py (+12/-0) cloudinit/net/tests/test_dhcp.py (+119/-1) cloudinit/net/tests/test_init.py (+262/-9) cloudinit/settings.py (+1/-0) cloudinit/sources/DataSourceAzure.py (+141/-32) cloudinit/sources/DataSourceCloudSigma.py (+2/-6) cloudinit/sources/DataSourceExoscale.py (+258/-0) cloudinit/sources/DataSourceGCE.py (+20/-2) cloudinit/sources/DataSourceHetzner.py (+3/-0) cloudinit/sources/DataSourceOVF.py (+6/-1) cloudinit/sources/DataSourceOracle.py (+99/-7) cloudinit/sources/__init__.py (+27/-0) cloudinit/sources/helpers/azure.py (+152/-8) cloudinit/sources/helpers/vmware/imc/config_custom_script.py (+42/-101) cloudinit/sources/tests/test_oracle.py (+228/-11) cloudinit/stages.py (+50/-15) cloudinit/tests/helpers.py (+2/-1) cloudinit/tests/test_stages.py (+132/-19) cloudinit/url_helper.py (+5/-4) cloudinit/version.py (+1/-1) debian/changelog (+60/-3) debian/cloud-init.templates (+3/-3) debian/patches/ubuntu-advantage-revert-tip.patch (+4/-8) doc/examples/cloud-config-datasources.txt (+1/-1) doc/examples/cloud-config-user-groups.txt (+1/-0) doc/rtd/conf.py (+0/-5) doc/rtd/topics/analyze.rst (+84/-0) doc/rtd/topics/capabilities.rst (+1/-0) doc/rtd/topics/datasources.rst (+1/-0) doc/rtd/topics/datasources/exoscale.rst (+68/-0) doc/rtd/topics/datasources/oracle.rst (+24/-1) doc/rtd/topics/debugging.rst (+13/-0) doc/rtd/topics/format.rst (+13/-12) doc/rtd/topics/network-config-format-v2.rst (+1/-1) doc/rtd/topics/network-config.rst (+5/-4) integration-requirements.txt (+2/-1) systemd/cloud-init-generator.tmpl (+6/-1) templates/ntp.conf.debian.tmpl (+2/-1) tests/cloud_tests/platforms.yaml (+1/-0) tests/cloud_tests/platforms/nocloudkvm/instance.py (+9/-4) tests/cloud_tests/platforms/platforms.py (+1/-1) tests/cloud_tests/setup_image.py (+2/-1) tests/unittests/test_datasource/test_azure.py (+112/-15) tests/unittests/test_datasource/test_common.py (+13/-0) tests/unittests/test_datasource/test_ec2.py (+2/-1) tests/unittests/test_datasource/test_exoscale.py (+203/-0) tests/unittests/test_datasource/test_gce.py (+18/-0) tests/unittests/test_distros/test_netconfig.py (+86/-0) tests/unittests/test_ds_identify.py (+25/-0) tests/unittests/test_handler/test_handler_apt_source_v3.py (+11/-0) tests/unittests/test_handler/test_handler_ntp.py (+15/-10) tests/unittests/test_net.py (+197/-23) tests/unittests/test_reporting_hyperv.py (+65/-0) tests/unittests/test_vmware/test_custom_script.py (+63/-53) tools/build-on-freebsd (+40/-33) tools/ds-identify (+32/-14) tools/xkvm (+53/-8) |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Ryan Harper | Approve | ||
Server Team CI bot | continuous-integration | Approve | |
Review via email: mp+371686@code.launchpad.net |
Commit message
Upstream snapshot for SRU into bionic
Also enables Exoscale in debian/
Description of the change
Server Team CI bot (server-team-bot) wrote : | # |
Server Team CI bot (server-team-bot) wrote : | # |
PASSED: Continuous integration, rev:5b00861163e
https:/
Executed test runs:
SUCCESS: Checkout
SUCCESS: Unit & Style Tests
SUCCESS: Ubuntu LTS: Build
SUCCESS: Ubuntu LTS: Integration
IN_PROGRESS: Declarative: Post Actions
Click here to trigger a rebuild:
https:/
Server Team CI bot (server-team-bot) wrote : | # |
PASSED: Continuous integration, rev:ef4aa258cb5
https:/
Executed test runs:
SUCCESS: Checkout
SUCCESS: Unit & Style Tests
SUCCESS: Ubuntu LTS: Build
SUCCESS: Ubuntu LTS: Integration
IN_PROGRESS: Declarative: Post Actions
Click here to trigger a rebuild:
https:/
Ryan Harper (raharper) wrote : | # |
Looks good. Thanks.
Preview Diff
1 | diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md |
2 | new file mode 100644 |
3 | index 0000000..170a71e |
4 | --- /dev/null |
5 | +++ b/.github/pull_request_template.md |
6 | @@ -0,0 +1,9 @@ |
7 | +***This GitHub repo is only a mirror. Do not submit pull requests |
8 | +here!*** |
9 | + |
10 | +Thank you for taking the time to write and submit a change to |
11 | +cloud-init! Please follow [our hacking |
12 | +guide](https://cloudinit.readthedocs.io/en/latest/topics/hacking.html) |
13 | +to submit your change to cloud-init's [Launchpad git |
14 | +repository](https://code.launchpad.net/cloud-init/), where cloud-init |
15 | +development happens. |
16 | diff --git a/ChangeLog b/ChangeLog |
17 | index bf48fd4..a98f8c2 100644 |
18 | --- a/ChangeLog |
19 | +++ b/ChangeLog |
20 | @@ -1,3 +1,39 @@ |
21 | +19.2: |
22 | + - net: add rfc3442 (classless static routes) to EphemeralDHCP |
23 | + (LP: #1821102) |
24 | + - templates/ntp.conf.debian.tmpl: fix missing newline for pools |
25 | + (LP: #1836598) |
26 | + - Support netplan renderer in Arch Linux [Conrad Hoffmann] |
27 | + - Fix typo in publicly viewable documentation. [David Medberry] |
28 | + - Add a cdrom size checker for OVF ds to ds-identify |
29 | + [Pengpeng Sun] (LP: #1806701) |
30 | + - VMWare: Trigger the post customization script via cc_scripts module. |
31 | + [Xiaofeng Wang] (LP: #1833192) |
32 | + - Cloud-init analyze module: Added ability to analyze boot events. |
33 | + [Sam Gilson] |
34 | + - Update debian eni network configuration location, retain Ubuntu setting |
35 | + [Janos Lenart] |
36 | + - net: skip bond interfaces in get_interfaces |
37 | + [Stanislav Makar] (LP: #1812857) |
38 | + - Fix a couple of issues raised by a coverity scan |
39 | + - Add missing dsname for Hetzner Cloud datasource [Markus Schade] |
40 | + - doc: indicate that netplan is default in Ubuntu now |
41 | + - azure: add region and AZ properties from imds compute location metadata |
42 | + - sysconfig: support more bonding options [Penghui Liao] |
43 | + - cloud-init-generator: use libexec path to ds-identify on redhat systems |
44 | + (LP: #1833264) |
45 | + - tools/build-on-freebsd: update to python3 [Gonéri Le Bouder] |
46 | + - Allow identification of OpenStack by Asset Tag |
47 | + [Mark T. Voelker] (LP: #1669875) |
48 | + - Fix spelling error making 'an Ubuntu' consistent. [Brian Murray] |
49 | + - run-container: centos: comment out the repo mirrorlist [Paride Legovini] |
50 | + - netplan: update netplan key mappings for gratuitous-arp (LP: #1827238) |
51 | + - freebsd: fix the name of cloudcfg VARIANT [Gonéri Le Bouder] |
52 | + - freebsd: ability to grow root file system [Gonéri Le Bouder] |
53 | + - freebsd: NoCloud data source support [Gonéri Le Bouder] (LP: #1645824) |
54 | + - Azure: Return static fallback address as if failed to find endpoint |
55 | + [Jason Zions (MSFT)] |
56 | + |
57 | 19.1: |
58 | - freebsd: add chpasswd pkg in the image [Gonéri Le Bouder] |
59 | - tests: add Eoan release [Paride Legovini] |
60 | diff --git a/cloudinit/analyze/__main__.py b/cloudinit/analyze/__main__.py |
61 | index f861365..99e5c20 100644 |
62 | --- a/cloudinit/analyze/__main__.py |
63 | +++ b/cloudinit/analyze/__main__.py |
64 | @@ -7,7 +7,7 @@ import re |
65 | import sys |
66 | |
67 | from cloudinit.util import json_dumps |
68 | - |
69 | +from datetime import datetime |
70 | from . import dump |
71 | from . import show |
72 | |
73 | @@ -52,9 +52,93 @@ def get_parser(parser=None): |
74 | dest='outfile', default='-', |
75 | help='specify where to write output. ') |
76 | parser_dump.set_defaults(action=('dump', analyze_dump)) |
77 | + parser_boot = subparsers.add_parser( |
78 | + 'boot', help='Print list of boot times for kernel and cloud-init') |
79 | + parser_boot.add_argument('-i', '--infile', action='store', |
80 | + dest='infile', default='/var/log/cloud-init.log', |
81 | + help='specify where to read input. ') |
82 | + parser_boot.add_argument('-o', '--outfile', action='store', |
83 | + dest='outfile', default='-', |
84 | + help='specify where to write output.') |
85 | + parser_boot.set_defaults(action=('boot', analyze_boot)) |
86 | return parser |
87 | |
88 | |
89 | +def analyze_boot(name, args): |
90 | + """Report a list of how long different boot operations took. |
91 | + |
92 | + For Example: |
93 | + -- Most Recent Boot Record -- |
94 | + Kernel Started at: <time> |
95 | + Kernel ended boot at: <time> |
96 | + Kernel time to boot (seconds): <time> |
97 | + Cloud-init activated by systemd at: <time> |
98 | + Time between Kernel end boot and Cloud-init activation (seconds):<time> |
99 | + Cloud-init start: <time> |
100 | + """ |
101 | + infh, outfh = configure_io(args) |
102 | + kernel_info = show.dist_check_timestamp() |
103 | + status_code, kernel_start, kernel_end, ci_sysd_start = \ |
104 | + kernel_info |
105 | + kernel_start_timestamp = datetime.utcfromtimestamp(kernel_start) |
106 | + kernel_end_timestamp = datetime.utcfromtimestamp(kernel_end) |
107 | + ci_sysd_start_timestamp = datetime.utcfromtimestamp(ci_sysd_start) |
108 | + try: |
109 | + last_init_local = \ |
110 | + [e for e in _get_events(infh) if e['name'] == 'init-local' and |
111 | + 'starting search' in e['description']][-1] |
112 | + ci_start = datetime.utcfromtimestamp(last_init_local['timestamp']) |
113 | + except IndexError: |
114 | + ci_start = 'Could not find init-local log-line in cloud-init.log' |
115 | + status_code = show.FAIL_CODE |
116 | + |
117 | + FAILURE_MSG = 'Your Linux distro or container does not support this ' \ |
118 | + 'functionality.\n' \ |
119 | + 'You must be running a Kernel Telemetry supported ' \ |
120 | + 'distro.\nPlease check ' \ |
121 | + 'https://cloudinit.readthedocs.io/en/latest' \ |
122 | + '/topics/analyze.html for more ' \ |
123 | + 'information on supported distros.\n' |
124 | + |
125 | + SUCCESS_MSG = '-- Most Recent Boot Record --\n' \ |
126 | + ' Kernel Started at: {k_s_t}\n' \ |
127 | + ' Kernel ended boot at: {k_e_t}\n' \ |
128 | + ' Kernel time to boot (seconds): {k_r}\n' \ |
129 | + ' Cloud-init activated by systemd at: {ci_sysd_t}\n' \ |
130 | + ' Time between Kernel end boot and Cloud-init ' \ |
131 | + 'activation (seconds): {bt_r}\n' \ |
132 | + ' Cloud-init start: {ci_start}\n' |
133 | + |
134 | + CONTAINER_MSG = '-- Most Recent Container Boot Record --\n' \ |
135 | + ' Container started at: {k_s_t}\n' \ |
136 | + ' Cloud-init activated by systemd at: {ci_sysd_t}\n' \ |
137 | + ' Cloud-init start: {ci_start}\n' \ |
138 | + |
139 | + status_map = { |
140 | + show.FAIL_CODE: FAILURE_MSG, |
141 | + show.CONTAINER_CODE: CONTAINER_MSG, |
142 | + show.SUCCESS_CODE: SUCCESS_MSG |
143 | + } |
144 | + |
145 | + kernel_runtime = kernel_end - kernel_start |
146 | + between_process_runtime = ci_sysd_start - kernel_end |
147 | + |
148 | + kwargs = { |
149 | + 'k_s_t': kernel_start_timestamp, |
150 | + 'k_e_t': kernel_end_timestamp, |
151 | + 'k_r': kernel_runtime, |
152 | + 'bt_r': between_process_runtime, |
153 | + 'k_e': kernel_end, |
154 | + 'k_s': kernel_start, |
155 | + 'ci_sysd': ci_sysd_start, |
156 | + 'ci_sysd_t': ci_sysd_start_timestamp, |
157 | + 'ci_start': ci_start |
158 | + } |
159 | + |
160 | + outfh.write(status_map[status_code].format(**kwargs)) |
161 | + return status_code |
162 | + |
163 | + |
164 | def analyze_blame(name, args): |
165 | """Report a list of records sorted by largest time delta. |
166 | |
167 | @@ -119,7 +203,7 @@ def analyze_dump(name, args): |
168 | |
169 | def _get_events(infile): |
170 | rawdata = None |
171 | - events, rawdata = show.load_events(infile, None) |
172 | + events, rawdata = show.load_events_infile(infile) |
173 | if not events: |
174 | events, _ = dump.dump_events(rawdata=rawdata) |
175 | return events |
176 | diff --git a/cloudinit/analyze/show.py b/cloudinit/analyze/show.py |
177 | index 3e778b8..511b808 100644 |
178 | --- a/cloudinit/analyze/show.py |
179 | +++ b/cloudinit/analyze/show.py |
180 | @@ -8,8 +8,11 @@ import base64 |
181 | import datetime |
182 | import json |
183 | import os |
184 | +import time |
185 | +import sys |
186 | |
187 | from cloudinit import util |
188 | +from cloudinit.distros import uses_systemd |
189 | |
190 | # An event: |
191 | ''' |
192 | @@ -49,6 +52,10 @@ format_key = { |
193 | |
194 | formatting_help = " ".join(["{0}: {1}".format(k.replace('%', '%%'), v) |
195 | for k, v in format_key.items()]) |
196 | +SUCCESS_CODE = 'successful' |
197 | +FAIL_CODE = 'failure' |
198 | +CONTAINER_CODE = 'container' |
199 | +TIMESTAMP_UNKNOWN = (FAIL_CODE, -1, -1, -1) |
200 | |
201 | |
202 | def format_record(msg, event): |
203 | @@ -125,9 +132,175 @@ def total_time_record(total_time): |
204 | return 'Total Time: %3.5f seconds\n' % total_time |
205 | |
206 | |
207 | +class SystemctlReader(object): |
208 | + ''' |
209 | + Class for dealing with all systemctl subp calls in a consistent manner. |
210 | + ''' |
211 | + def __init__(self, property, parameter=None): |
212 | + self.epoch = None |
213 | + self.args = ['/bin/systemctl', 'show'] |
214 | + if parameter: |
215 | + self.args.append(parameter) |
216 | + self.args.extend(['-p', property]) |
217 | + # Don't want the init of our object to break. Instead of throwing |
218 | + # an exception, set an error code that gets checked when data is |
219 | + # requested from the object |
220 | + self.failure = self.subp() |
221 | + |
222 | + def subp(self): |
223 | + ''' |
224 | + Make a subp call based on set args and handle errors by setting |
225 | + failure code |
226 | + |
227 | + :return: whether the subp call failed or not |
228 | + ''' |
229 | + try: |
230 | + value, err = util.subp(self.args, capture=True) |
231 | + if err: |
232 | + return err |
233 | + self.epoch = value |
234 | + return None |
235 | + except Exception as systemctl_fail: |
236 | + return systemctl_fail |
237 | + |
238 | + def parse_epoch_as_float(self): |
239 | + ''' |
240 | + If subp call succeeded, return the timestamp from subp as a float. |
241 | + |
242 | + :return: timestamp as a float |
243 | + ''' |
244 | + # subp has 2 ways to fail: it either fails and throws an exception, |
245 | + # or returns an error code. Raise an exception here in order to make |
246 | + # sure both scenarios throw exceptions |
247 | + if self.failure: |
248 | + raise RuntimeError('Subprocess call to systemctl has failed, ' |
249 | + 'returning error code ({})' |
250 | + .format(self.failure)) |
251 | + # Output from systemctl show has the format Property=Value. |
252 | + # For example, UserspaceMonotonic=1929304 |
253 | + timestamp = self.epoch.split('=')[1] |
254 | + # Timestamps reported by systemctl are in microseconds, converting |
255 | + return float(timestamp) / 1000000 |
256 | + |
257 | + |
258 | +def dist_check_timestamp(): |
259 | + ''' |
260 | + Determine which init system a particular linux distro is using. |
261 | + Each init system (systemd, upstart, etc) has a different way of |
262 | + providing timestamps. |
263 | + |
264 | + :return: timestamps of kernelboot, kernelendboot, and cloud-initstart |
265 | + or TIMESTAMP_UNKNOWN if the timestamps cannot be retrieved. |
266 | + ''' |
267 | + |
268 | + if uses_systemd(): |
269 | + return gather_timestamps_using_systemd() |
270 | + |
271 | + # Use dmesg to get timestamps if the distro does not have systemd |
272 | + if util.is_FreeBSD() or 'gentoo' in \ |
273 | + util.system_info()['system'].lower(): |
274 | + return gather_timestamps_using_dmesg() |
275 | + |
276 | + # this distro doesn't fit anything that is supported by cloud-init. just |
277 | + # return error codes |
278 | + return TIMESTAMP_UNKNOWN |
279 | + |
280 | + |
281 | +def gather_timestamps_using_dmesg(): |
282 | + ''' |
283 | + Gather timestamps that corresponds to kernel begin initialization, |
284 | + kernel finish initialization using dmesg as opposed to systemctl |
285 | + |
286 | + :return: the two timestamps plus a dummy timestamp to keep consistency |
287 | + with gather_timestamps_using_systemd |
288 | + ''' |
289 | + try: |
290 | + data, _ = util.subp(['dmesg'], capture=True) |
291 | + split_entries = data[0].splitlines() |
292 | + for i in split_entries: |
293 | + if i.decode('UTF-8').find('user') != -1: |
294 | + splitup = i.decode('UTF-8').split() |
295 | + stripped = splitup[1].strip(']') |
296 | + |
297 | + # kernel timestamp from dmesg is equal to 0, |
298 | + # with the userspace timestamp relative to it. |
299 | + user_space_timestamp = float(stripped) |
300 | + kernel_start = float(time.time()) - float(util.uptime()) |
301 | + kernel_end = kernel_start + user_space_timestamp |
302 | + |
303 | + # systemd wont start cloud-init in this case, |
304 | + # so we cannot get that timestamp |
305 | + return SUCCESS_CODE, kernel_start, kernel_end, \ |
306 | + kernel_end |
307 | + |
308 | + except Exception: |
309 | + pass |
310 | + return TIMESTAMP_UNKNOWN |
311 | + |
312 | + |
313 | +def gather_timestamps_using_systemd(): |
314 | + ''' |
315 | + Gather timestamps that corresponds to kernel begin initialization, |
316 | + kernel finish initialization. and cloud-init systemd unit activation |
317 | + |
318 | + :return: the three timestamps |
319 | + ''' |
320 | + kernel_start = float(time.time()) - float(util.uptime()) |
321 | + try: |
322 | + delta_k_end = SystemctlReader('UserspaceTimestampMonotonic')\ |
323 | + .parse_epoch_as_float() |
324 | + delta_ci_s = SystemctlReader('InactiveExitTimestampMonotonic', |
325 | + 'cloud-init-local').parse_epoch_as_float() |
326 | + base_time = kernel_start |
327 | + status = SUCCESS_CODE |
328 | + # lxc based containers do not set their monotonic zero point to be when |
329 | + # the container starts, instead keep using host boot as zero point |
330 | + # time.CLOCK_MONOTONIC_RAW is only available in python 3.3 |
331 | + if util.is_container(): |
332 | + # clock.monotonic also uses host boot as zero point |
333 | + if sys.version_info >= (3, 3): |
334 | + base_time = float(time.time()) - float(time.monotonic()) |
335 | + # TODO: lxcfs automatically truncates /proc/uptime to seconds |
336 | + # in containers when https://github.com/lxc/lxcfs/issues/292 |
337 | + # is fixed, util.uptime() should be used instead of stat on |
338 | + try: |
339 | + file_stat = os.stat('/proc/1/cmdline') |
340 | + kernel_start = file_stat.st_atime |
341 | + except OSError as err: |
342 | + raise RuntimeError('Could not determine container boot ' |
343 | + 'time from /proc/1/cmdline. ({})' |
344 | + .format(err)) |
345 | + status = CONTAINER_CODE |
346 | + else: |
347 | + status = FAIL_CODE |
348 | + kernel_end = base_time + delta_k_end |
349 | + cloudinit_sysd = base_time + delta_ci_s |
350 | + |
351 | + except Exception as e: |
352 | + # Except ALL exceptions as Systemctl reader can throw many different |
353 | + # errors, but any failure in systemctl means that timestamps cannot be |
354 | + # obtained |
355 | + print(e) |
356 | + return TIMESTAMP_UNKNOWN |
357 | + return status, kernel_start, kernel_end, cloudinit_sysd |
358 | + |
359 | + |
360 | def generate_records(events, blame_sort=False, |
361 | print_format="(%n) %d seconds in %I%D", |
362 | dump_files=False, log_datafiles=False): |
363 | + ''' |
364 | + Take in raw events and create parent-child dependencies between events |
365 | + in order to order events in chronological order. |
366 | + |
367 | + :param events: JSONs from dump that represents events taken from logs |
368 | + :param blame_sort: whether to sort by timestamp or by time taken. |
369 | + :param print_format: formatting to represent event, time stamp, |
370 | + and time taken by the event in one line |
371 | + :param dump_files: whether to dump files into JSONs |
372 | + :param log_datafiles: whether or not to log events generated |
373 | + |
374 | + :return: boot records ordered chronologically |
375 | + ''' |
376 | |
377 | sorted_events = sorted(events, key=lambda x: x['timestamp']) |
378 | records = [] |
379 | @@ -189,19 +362,28 @@ def generate_records(events, blame_sort=False, |
380 | |
381 | |
382 | def show_events(events, print_format): |
383 | + ''' |
384 | + A passthrough method that makes it easier to call generate_records() |
385 | + |
386 | + :param events: JSONs from dump that represents events taken from logs |
387 | + :param print_format: formatting to represent event, time stamp, |
388 | + and time taken by the event in one line |
389 | + |
390 | + :return: boot records ordered chronologically |
391 | + ''' |
392 | return generate_records(events, print_format=print_format) |
393 | |
394 | |
395 | -def load_events(infile, rawdata=None): |
396 | - if rawdata: |
397 | - data = rawdata.read() |
398 | - else: |
399 | - data = infile.read() |
400 | +def load_events_infile(infile): |
401 | + ''' |
402 | + Takes in a log file, read it, and convert to json. |
403 | + |
404 | + :param infile: The Log file to be read |
405 | |
406 | - j = None |
407 | + :return: json version of logfile, raw file |
408 | + ''' |
409 | + data = infile.read() |
410 | try: |
411 | - j = json.loads(data) |
412 | + return json.loads(data), data |
413 | except ValueError: |
414 | - pass |
415 | - |
416 | - return j, data |
417 | + return None, data |
418 | diff --git a/cloudinit/analyze/tests/test_boot.py b/cloudinit/analyze/tests/test_boot.py |
419 | new file mode 100644 |
420 | index 0000000..706e2cc |
421 | --- /dev/null |
422 | +++ b/cloudinit/analyze/tests/test_boot.py |
423 | @@ -0,0 +1,170 @@ |
424 | +import os |
425 | +from cloudinit.analyze.__main__ import (analyze_boot, get_parser) |
426 | +from cloudinit.tests.helpers import CiTestCase, mock |
427 | +from cloudinit.analyze.show import dist_check_timestamp, SystemctlReader, \ |
428 | + FAIL_CODE, CONTAINER_CODE |
429 | + |
430 | +err_code = (FAIL_CODE, -1, -1, -1) |
431 | + |
432 | + |
433 | +class TestDistroChecker(CiTestCase): |
434 | + |
435 | + @mock.patch('cloudinit.util.system_info', return_value={'dist': ('', '', |
436 | + ''), |
437 | + 'system': ''}) |
438 | + @mock.patch('platform.linux_distribution', return_value=('', '', '')) |
439 | + @mock.patch('cloudinit.util.is_FreeBSD', return_value=False) |
440 | + def test_blank_distro(self, m_sys_info, m_linux_distribution, m_free_bsd): |
441 | + self.assertEqual(err_code, dist_check_timestamp()) |
442 | + |
443 | + @mock.patch('cloudinit.util.system_info', return_value={'dist': ('', '', |
444 | + '')}) |
445 | + @mock.patch('platform.linux_distribution', return_value=('', '', '')) |
446 | + @mock.patch('cloudinit.util.is_FreeBSD', return_value=True) |
447 | + def test_freebsd_gentoo_cant_find(self, m_sys_info, |
448 | + m_linux_distribution, m_is_FreeBSD): |
449 | + self.assertEqual(err_code, dist_check_timestamp()) |
450 | + |
451 | + @mock.patch('cloudinit.util.subp', return_value=(0, 1)) |
452 | + def test_subp_fails(self, m_subp): |
453 | + self.assertEqual(err_code, dist_check_timestamp()) |
454 | + |
455 | + |
456 | +class TestSystemCtlReader(CiTestCase): |
457 | + |
458 | + def test_systemctl_invalid_property(self): |
459 | + reader = SystemctlReader('dummyProperty') |
460 | + with self.assertRaises(RuntimeError): |
461 | + reader.parse_epoch_as_float() |
462 | + |
463 | + def test_systemctl_invalid_parameter(self): |
464 | + reader = SystemctlReader('dummyProperty', 'dummyParameter') |
465 | + with self.assertRaises(RuntimeError): |
466 | + reader.parse_epoch_as_float() |
467 | + |
468 | + @mock.patch('cloudinit.util.subp', return_value=('U=1000000', None)) |
469 | + def test_systemctl_works_correctly_threshold(self, m_subp): |
470 | + reader = SystemctlReader('dummyProperty', 'dummyParameter') |
471 | + self.assertEqual(1.0, reader.parse_epoch_as_float()) |
472 | + thresh = 1.0 - reader.parse_epoch_as_float() |
473 | + self.assertTrue(thresh < 1e-6) |
474 | + self.assertTrue(thresh > (-1 * 1e-6)) |
475 | + |
476 | + @mock.patch('cloudinit.util.subp', return_value=('U=0', None)) |
477 | + def test_systemctl_succeed_zero(self, m_subp): |
478 | + reader = SystemctlReader('dummyProperty', 'dummyParameter') |
479 | + self.assertEqual(0.0, reader.parse_epoch_as_float()) |
480 | + |
481 | + @mock.patch('cloudinit.util.subp', return_value=('U=1', None)) |
482 | + def test_systemctl_succeed_distinct(self, m_subp): |
483 | + reader = SystemctlReader('dummyProperty', 'dummyParameter') |
484 | + val1 = reader.parse_epoch_as_float() |
485 | + m_subp.return_value = ('U=2', None) |
486 | + reader2 = SystemctlReader('dummyProperty', 'dummyParameter') |
487 | + val2 = reader2.parse_epoch_as_float() |
488 | + self.assertNotEqual(val1, val2) |
489 | + |
490 | + @mock.patch('cloudinit.util.subp', return_value=('100', None)) |
491 | + def test_systemctl_epoch_not_splittable(self, m_subp): |
492 | + reader = SystemctlReader('dummyProperty', 'dummyParameter') |
493 | + with self.assertRaises(IndexError): |
494 | + reader.parse_epoch_as_float() |
495 | + |
496 | + @mock.patch('cloudinit.util.subp', return_value=('U=foobar', None)) |
497 | + def test_systemctl_cannot_convert_epoch_to_float(self, m_subp): |
498 | + reader = SystemctlReader('dummyProperty', 'dummyParameter') |
499 | + with self.assertRaises(ValueError): |
500 | + reader.parse_epoch_as_float() |
501 | + |
502 | + |
503 | +class TestAnalyzeBoot(CiTestCase): |
504 | + |
505 | + def set_up_dummy_file_ci(self, path, log_path): |
506 | + infh = open(path, 'w+') |
507 | + infh.write('2019-07-08 17:40:49,601 - util.py[DEBUG]: Cloud-init v. ' |
508 | + '19.1-1-gbaa47854-0ubuntu1~18.04.1 running \'init-local\' ' |
509 | + 'at Mon, 08 Jul 2019 17:40:49 +0000. Up 18.84 seconds.') |
510 | + infh.close() |
511 | + outfh = open(log_path, 'w+') |
512 | + outfh.close() |
513 | + |
514 | + def set_up_dummy_file(self, path, log_path): |
515 | + infh = open(path, 'w+') |
516 | + infh.write('dummy data') |
517 | + infh.close() |
518 | + outfh = open(log_path, 'w+') |
519 | + outfh.close() |
520 | + |
521 | + def remove_dummy_file(self, path, log_path): |
522 | + if os.path.isfile(path): |
523 | + os.remove(path) |
524 | + if os.path.isfile(log_path): |
525 | + os.remove(log_path) |
526 | + |
527 | + @mock.patch('cloudinit.analyze.show.dist_check_timestamp', |
528 | + return_value=err_code) |
529 | + def test_boot_invalid_distro(self, m_dist_check_timestamp): |
530 | + |
531 | + path = os.path.dirname(os.path.abspath(__file__)) |
532 | + log_path = path + '/boot-test.log' |
533 | + path += '/dummy.log' |
534 | + self.set_up_dummy_file(path, log_path) |
535 | + |
536 | + parser = get_parser() |
537 | + args = parser.parse_args(args=['boot', '-i', path, '-o', |
538 | + log_path]) |
539 | + name_default = '' |
540 | + analyze_boot(name_default, args) |
541 | + # now args have been tested, go into outfile and make sure error |
542 | + # message is in the outfile |
543 | + outfh = open(args.outfile, 'r') |
544 | + data = outfh.read() |
545 | + err_string = 'Your Linux distro or container does not support this ' \ |
546 | + 'functionality.\nYou must be running a Kernel ' \ |
547 | + 'Telemetry supported distro.\nPlease check ' \ |
548 | + 'https://cloudinit.readthedocs.io/en/latest/topics' \ |
549 | + '/analyze.html for more information on supported ' \ |
550 | + 'distros.\n' |
551 | + |
552 | + self.remove_dummy_file(path, log_path) |
553 | + self.assertEqual(err_string, data) |
554 | + |
555 | + @mock.patch("cloudinit.util.is_container", return_value=True) |
556 | + @mock.patch('cloudinit.util.subp', return_value=('U=1000000', None)) |
557 | + def test_container_no_ci_log_line(self, m_is_container, m_subp): |
558 | + path = os.path.dirname(os.path.abspath(__file__)) |
559 | + log_path = path + '/boot-test.log' |
560 | + path += '/dummy.log' |
561 | + self.set_up_dummy_file(path, log_path) |
562 | + |
563 | + parser = get_parser() |
564 | + args = parser.parse_args(args=['boot', '-i', path, '-o', |
565 | + log_path]) |
566 | + name_default = '' |
567 | + |
568 | + finish_code = analyze_boot(name_default, args) |
569 | + |
570 | + self.remove_dummy_file(path, log_path) |
571 | + self.assertEqual(FAIL_CODE, finish_code) |
572 | + |
573 | + @mock.patch("cloudinit.util.is_container", return_value=True) |
574 | + @mock.patch('cloudinit.util.subp', return_value=('U=1000000', None)) |
575 | + @mock.patch('cloudinit.analyze.__main__._get_events', return_value=[{ |
576 | + 'name': 'init-local', 'description': 'starting search', 'timestamp': |
577 | + 100000}]) |
578 | + @mock.patch('cloudinit.analyze.show.dist_check_timestamp', |
579 | + return_value=(CONTAINER_CODE, 1, 1, 1)) |
580 | + def test_container_ci_log_line(self, m_is_container, m_subp, m_get, m_g): |
581 | + path = os.path.dirname(os.path.abspath(__file__)) |
582 | + log_path = path + '/boot-test.log' |
583 | + path += '/dummy.log' |
584 | + self.set_up_dummy_file_ci(path, log_path) |
585 | + |
586 | + parser = get_parser() |
587 | + args = parser.parse_args(args=['boot', '-i', path, '-o', |
588 | + log_path]) |
589 | + name_default = '' |
590 | + finish_code = analyze_boot(name_default, args) |
591 | + |
592 | + self.remove_dummy_file(path, log_path) |
593 | + self.assertEqual(CONTAINER_CODE, finish_code) |
594 | diff --git a/cloudinit/apport.py b/cloudinit/apport.py |
595 | index 22cb7fd..003ff1f 100644 |
596 | --- a/cloudinit/apport.py |
597 | +++ b/cloudinit/apport.py |
598 | @@ -23,6 +23,7 @@ KNOWN_CLOUD_NAMES = [ |
599 | 'CloudStack', |
600 | 'DigitalOcean', |
601 | 'GCE - Google Compute Engine', |
602 | + 'Exoscale', |
603 | 'Hetzner Cloud', |
604 | 'IBM - (aka SoftLayer or BlueMix)', |
605 | 'LXD', |
606 | diff --git a/cloudinit/config/cc_apt_configure.py b/cloudinit/config/cc_apt_configure.py |
607 | index 919d199..f01e2aa 100644 |
608 | --- a/cloudinit/config/cc_apt_configure.py |
609 | +++ b/cloudinit/config/cc_apt_configure.py |
610 | @@ -332,6 +332,8 @@ def apply_apt(cfg, cloud, target): |
611 | |
612 | |
613 | def debconf_set_selections(selections, target=None): |
614 | + if not selections.endswith(b'\n'): |
615 | + selections += b'\n' |
616 | util.subp(['debconf-set-selections'], data=selections, target=target, |
617 | capture=True) |
618 | |
619 | @@ -374,7 +376,7 @@ def apply_debconf_selections(cfg, target=None): |
620 | |
621 | selections = '\n'.join( |
622 | [selsets[key] for key in sorted(selsets.keys())]) |
623 | - debconf_set_selections(selections.encode() + b"\n", target=target) |
624 | + debconf_set_selections(selections.encode(), target=target) |
625 | |
626 | # get a complete list of packages listed in input |
627 | pkgs_cfgd = set() |
628 | diff --git a/cloudinit/config/cc_lxd.py b/cloudinit/config/cc_lxd.py |
629 | index 71d13ed..d983077 100644 |
630 | --- a/cloudinit/config/cc_lxd.py |
631 | +++ b/cloudinit/config/cc_lxd.py |
632 | @@ -152,7 +152,7 @@ def handle(name, cfg, cloud, log, args): |
633 | |
634 | if cmd_attach: |
635 | log.debug("Setting up default lxd bridge: %s" % |
636 | - " ".join(cmd_create)) |
637 | + " ".join(cmd_attach)) |
638 | _lxc(cmd_attach) |
639 | |
640 | elif bridge_cfg: |
641 | diff --git a/cloudinit/config/cc_set_passwords.py b/cloudinit/config/cc_set_passwords.py |
642 | index 4585e4d..cf9b5ab 100755 |
643 | --- a/cloudinit/config/cc_set_passwords.py |
644 | +++ b/cloudinit/config/cc_set_passwords.py |
645 | @@ -9,27 +9,40 @@ |
646 | """ |
647 | Set Passwords |
648 | ------------- |
649 | -**Summary:** Set user passwords |
650 | - |
651 | -Set system passwords and enable or disable ssh password authentication. |
652 | -The ``chpasswd`` config key accepts a dictionary containing a single one of two |
653 | -keys, either ``expire`` or ``list``. If ``expire`` is specified and is set to |
654 | -``false``, then the ``password`` global config key is used as the password for |
655 | -all user accounts. If the ``expire`` key is specified and is set to ``true`` |
656 | -then user passwords will be expired, preventing the default system passwords |
657 | -from being used. |
658 | - |
659 | -If the ``list`` key is provided, a list of |
660 | -``username:password`` pairs can be specified. The usernames specified |
661 | -must already exist on the system, or have been created using the |
662 | -``cc_users_groups`` module. A password can be randomly generated using |
663 | -``username:RANDOM`` or ``username:R``. A hashed password can be specified |
664 | -using ``username:$6$salt$hash``. Password ssh authentication can be |
665 | -enabled, disabled, or left to system defaults using ``ssh_pwauth``. |
666 | +**Summary:** Set user passwords and enable/disable SSH password authentication |
667 | + |
668 | +This module consumes three top-level config keys: ``ssh_pwauth``, ``chpasswd`` |
669 | +and ``password``. |
670 | + |
671 | +The ``ssh_pwauth`` config key determines whether or not sshd will be configured |
672 | +to accept password authentication. True values will enable password auth, |
673 | +false values will disable password auth, and the literal string ``unchanged`` |
674 | +will leave it unchanged. Setting no value will also leave the current setting |
675 | +on-disk unchanged. |
676 | + |
677 | +The ``chpasswd`` config key accepts a dictionary containing either or both of |
678 | +``expire`` and ``list``. |
679 | + |
680 | +If the ``list`` key is provided, it should contain a list of |
681 | +``username:password`` pairs. This can be either a YAML list (of strings), or a |
682 | +multi-line string with one pair per line. Each user will have the |
683 | +corresponding password set. A password can be randomly generated by specifying |
684 | +``RANDOM`` or ``R`` as a user's password. A hashed password, created by a tool |
685 | +like ``mkpasswd``, can be specified; a regex |
686 | +(``r'\\$(1|2a|2y|5|6)(\\$.+){2}'``) is used to determine if a password value |
687 | +should be treated as a hash. |
688 | |
689 | .. note:: |
690 | - if using ``expire: true`` then a ssh authkey should be specified or it may |
691 | - not be possible to login to the system |
692 | + The users specified must already exist on the system. Users will have been |
693 | + created by the ``cc_users_groups`` module at this point. |
694 | + |
695 | +By default, all users on the system will have their passwords expired (meaning |
696 | +that they will have to be reset the next time the user logs in). To disable |
697 | +this behaviour, set ``expire`` under ``chpasswd`` to a false value. |
698 | + |
699 | +If a ``list`` of user/password pairs is not specified under ``chpasswd``, then |
700 | +the value of the ``password`` config key will be used to set the default user's |
701 | +password. |
702 | |
703 | **Internal name:** ``cc_set_passwords`` |
704 | |
705 | @@ -160,6 +173,8 @@ def handle(_name, cfg, cloud, log, args): |
706 | hashed_users = [] |
707 | randlist = [] |
708 | users = [] |
709 | + # N.B. This regex is included in the documentation (i.e. the module |
710 | + # docstring), so any changes to it should be reflected there. |
711 | prog = re.compile(r'\$(1|2a|2y|5|6)(\$.+){2}') |
712 | for line in plist: |
713 | u, p = line.split(':', 1) |
714 | diff --git a/cloudinit/config/cc_ssh.py b/cloudinit/config/cc_ssh.py |
715 | index f8f7cb3..fdd8f4d 100755 |
716 | --- a/cloudinit/config/cc_ssh.py |
717 | +++ b/cloudinit/config/cc_ssh.py |
718 | @@ -91,6 +91,9 @@ public keys. |
719 | ssh_authorized_keys: |
720 | - ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAGEA3FSyQwBI6Z+nCSjUU ... |
721 | - ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA3I7VUf2l5gSn5uavROsc5HRDpZ ... |
722 | + ssh_publish_hostkeys: |
723 | + enabled: <true/false> (Defaults to true) |
724 | + blacklist: <list of key types> (Defaults to [dsa]) |
725 | """ |
726 | |
727 | import glob |
728 | @@ -104,6 +107,10 @@ from cloudinit import util |
729 | |
730 | GENERATE_KEY_NAMES = ['rsa', 'dsa', 'ecdsa', 'ed25519'] |
731 | KEY_FILE_TPL = '/etc/ssh/ssh_host_%s_key' |
732 | +PUBLISH_HOST_KEYS = True |
733 | +# Don't publish the dsa hostkey by default since OpenSSH recommends not using |
734 | +# it. |
735 | +HOST_KEY_PUBLISH_BLACKLIST = ['dsa'] |
736 | |
737 | CONFIG_KEY_TO_FILE = {} |
738 | PRIV_TO_PUB = {} |
739 | @@ -176,6 +183,23 @@ def handle(_name, cfg, cloud, log, _args): |
740 | util.logexc(log, "Failed generating key type %s to " |
741 | "file %s", keytype, keyfile) |
742 | |
743 | + if "ssh_publish_hostkeys" in cfg: |
744 | + host_key_blacklist = util.get_cfg_option_list( |
745 | + cfg["ssh_publish_hostkeys"], "blacklist", |
746 | + HOST_KEY_PUBLISH_BLACKLIST) |
747 | + publish_hostkeys = util.get_cfg_option_bool( |
748 | + cfg["ssh_publish_hostkeys"], "enabled", PUBLISH_HOST_KEYS) |
749 | + else: |
750 | + host_key_blacklist = HOST_KEY_PUBLISH_BLACKLIST |
751 | + publish_hostkeys = PUBLISH_HOST_KEYS |
752 | + |
753 | + if publish_hostkeys: |
754 | + hostkeys = get_public_host_keys(blacklist=host_key_blacklist) |
755 | + try: |
756 | + cloud.datasource.publish_host_keys(hostkeys) |
757 | + except Exception: |
758 | + util.logexc(log, "Publishing host keys failed!") |
759 | + |
760 | try: |
761 | (users, _groups) = ug_util.normalize_users_groups(cfg, cloud.distro) |
762 | (user, _user_config) = ug_util.extract_default(users) |
763 | @@ -209,4 +233,35 @@ def apply_credentials(keys, user, disable_root, disable_root_opts): |
764 | |
765 | ssh_util.setup_user_keys(keys, 'root', options=key_prefix) |
766 | |
767 | + |
768 | +def get_public_host_keys(blacklist=None): |
769 | + """Read host keys from /etc/ssh/*.pub files and return them as a list. |
770 | + |
771 | + @param blacklist: List of key types to ignore. e.g. ['dsa', 'rsa'] |
772 | + @returns: List of keys, each formatted as a two-element tuple. |
773 | + e.g. [('ssh-rsa', 'AAAAB3Nz...'), ('ssh-ed25519', 'AAAAC3Nx...')] |
774 | + """ |
775 | + public_key_file_tmpl = '%s.pub' % (KEY_FILE_TPL,) |
776 | + key_list = [] |
777 | + blacklist_files = [] |
778 | + if blacklist: |
779 | + # Convert blacklist to filenames: |
780 | + # 'dsa' -> '/etc/ssh/ssh_host_dsa_key.pub' |
781 | + blacklist_files = [public_key_file_tmpl % (key_type,) |
782 | + for key_type in blacklist] |
783 | + # Get list of public key files and filter out blacklisted files. |
784 | + file_list = [hostfile for hostfile |
785 | + in glob.glob(public_key_file_tmpl % ('*',)) |
786 | + if hostfile not in blacklist_files] |
787 | + |
788 | + # Read host key files, retrieve first two fields as a tuple and |
789 | + # append that tuple to key_list. |
790 | + for file_name in file_list: |
791 | + file_contents = util.load_file(file_name) |
792 | + key_data = file_contents.split() |
793 | + if key_data and len(key_data) > 1: |
794 | + key_list.append(tuple(key_data[:2])) |
795 | + return key_list |
796 | + |
797 | + |
798 | # vi: ts=4 expandtab |
799 | diff --git a/cloudinit/config/cc_ubuntu_drivers.py b/cloudinit/config/cc_ubuntu_drivers.py |
800 | index 91feb60..297451d 100644 |
801 | --- a/cloudinit/config/cc_ubuntu_drivers.py |
802 | +++ b/cloudinit/config/cc_ubuntu_drivers.py |
803 | @@ -2,12 +2,14 @@ |
804 | |
805 | """Ubuntu Drivers: Interact with third party drivers in Ubuntu.""" |
806 | |
807 | +import os |
808 | from textwrap import dedent |
809 | |
810 | from cloudinit.config.schema import ( |
811 | get_schema_doc, validate_cloudconfig_schema) |
812 | from cloudinit import log as logging |
813 | from cloudinit.settings import PER_INSTANCE |
814 | +from cloudinit import temp_utils |
815 | from cloudinit import type_utils |
816 | from cloudinit import util |
817 | |
818 | @@ -64,6 +66,33 @@ OLD_UBUNTU_DRIVERS_STDERR_NEEDLE = ( |
819 | __doc__ = get_schema_doc(schema) # Supplement python help() |
820 | |
821 | |
822 | +# Use a debconf template to configure a global debconf variable |
823 | +# (linux/nvidia/latelink) setting this to "true" allows the |
824 | +# 'linux-restricted-modules' deb to accept the NVIDIA EULA and the package |
825 | +# will automatically link the drivers to the running kernel. |
826 | + |
827 | +# EOL_XENIAL: can then drop this script and use python3-debconf which is only |
828 | +# available in Bionic and later. Can't use python3-debconf currently as it |
829 | +# isn't in Xenial and doesn't yet support X_LOADTEMPLATEFILE debconf command. |
830 | + |
831 | +NVIDIA_DEBCONF_CONTENT = """\ |
832 | +Template: linux/nvidia/latelink |
833 | +Type: boolean |
834 | +Default: true |
835 | +Description: Late-link NVIDIA kernel modules? |
836 | + Enable this to link the NVIDIA kernel modules in cloud-init and |
837 | + make them available for use. |
838 | +""" |
839 | + |
840 | +NVIDIA_DRIVER_LATELINK_DEBCONF_SCRIPT = """\ |
841 | +#!/bin/sh |
842 | +# Allow cloud-init to trigger EULA acceptance via registering a debconf |
843 | +# template to set linux/nvidia/latelink true |
844 | +. /usr/share/debconf/confmodule |
845 | +db_x_loadtemplatefile "$1" cloud-init |
846 | +""" |
847 | + |
848 | + |
849 | def install_drivers(cfg, pkg_install_func): |
850 | if not isinstance(cfg, dict): |
851 | raise TypeError( |
852 | @@ -89,9 +118,28 @@ def install_drivers(cfg, pkg_install_func): |
853 | if version_cfg: |
854 | driver_arg += ':{}'.format(version_cfg) |
855 | |
856 | - LOG.debug("Installing NVIDIA drivers (%s=%s, version=%s)", |
857 | + LOG.debug("Installing and activating NVIDIA drivers (%s=%s, version=%s)", |
858 | cfgpath, nv_acc, version_cfg if version_cfg else 'latest') |
859 | |
860 | + # Register and set debconf selection linux/nvidia/latelink = true |
861 | + tdir = temp_utils.mkdtemp(needs_exe=True) |
862 | + debconf_file = os.path.join(tdir, 'nvidia.template') |
863 | + debconf_script = os.path.join(tdir, 'nvidia-debconf.sh') |
864 | + try: |
865 | + util.write_file(debconf_file, NVIDIA_DEBCONF_CONTENT) |
866 | + util.write_file( |
867 | + debconf_script, |
868 | + util.encode_text(NVIDIA_DRIVER_LATELINK_DEBCONF_SCRIPT), |
869 | + mode=0o755) |
870 | + util.subp([debconf_script, debconf_file]) |
871 | + except Exception as e: |
872 | + util.logexc( |
873 | + LOG, "Failed to register NVIDIA debconf template: %s", str(e)) |
874 | + raise |
875 | + finally: |
876 | + if os.path.isdir(tdir): |
877 | + util.del_dir(tdir) |
878 | + |
879 | try: |
880 | util.subp(['ubuntu-drivers', 'install', '--gpgpu', driver_arg]) |
881 | except util.ProcessExecutionError as exc: |
882 | diff --git a/cloudinit/config/tests/test_ssh.py b/cloudinit/config/tests/test_ssh.py |
883 | index c8a4271..e778984 100644 |
884 | --- a/cloudinit/config/tests/test_ssh.py |
885 | +++ b/cloudinit/config/tests/test_ssh.py |
886 | @@ -1,5 +1,6 @@ |
887 | # This file is part of cloud-init. See LICENSE file for license information. |
888 | |
889 | +import os.path |
890 | |
891 | from cloudinit.config import cc_ssh |
892 | from cloudinit import ssh_util |
893 | @@ -12,6 +13,25 @@ MODPATH = "cloudinit.config.cc_ssh." |
894 | class TestHandleSsh(CiTestCase): |
895 | """Test cc_ssh handling of ssh config.""" |
896 | |
897 | + def _publish_hostkey_test_setup(self): |
898 | + self.test_hostkeys = { |
899 | + 'dsa': ('ssh-dss', 'AAAAB3NzaC1kc3MAAACB'), |
900 | + 'ecdsa': ('ecdsa-sha2-nistp256', 'AAAAE2VjZ'), |
901 | + 'ed25519': ('ssh-ed25519', 'AAAAC3NzaC1lZDI'), |
902 | + 'rsa': ('ssh-rsa', 'AAAAB3NzaC1yc2EAAA'), |
903 | + } |
904 | + self.test_hostkey_files = [] |
905 | + hostkey_tmpdir = self.tmp_dir() |
906 | + for key_type in ['dsa', 'ecdsa', 'ed25519', 'rsa']: |
907 | + key_data = self.test_hostkeys[key_type] |
908 | + filename = 'ssh_host_%s_key.pub' % key_type |
909 | + filepath = os.path.join(hostkey_tmpdir, filename) |
910 | + self.test_hostkey_files.append(filepath) |
911 | + with open(filepath, 'w') as f: |
912 | + f.write(' '.join(key_data)) |
913 | + |
914 | + cc_ssh.KEY_FILE_TPL = os.path.join(hostkey_tmpdir, 'ssh_host_%s_key') |
915 | + |
916 | def test_apply_credentials_with_user(self, m_setup_keys): |
917 | """Apply keys for the given user and root.""" |
918 | keys = ["key1"] |
919 | @@ -64,6 +84,7 @@ class TestHandleSsh(CiTestCase): |
920 | # Mock os.path.exits to True to short-circuit the key writing logic |
921 | m_path_exists.return_value = True |
922 | m_nug.return_value = ([], {}) |
923 | + cc_ssh.PUBLISH_HOST_KEYS = False |
924 | cloud = self.tmp_cloud( |
925 | distro='ubuntu', metadata={'public-keys': keys}) |
926 | cc_ssh.handle("name", cfg, cloud, None, None) |
927 | @@ -149,3 +170,148 @@ class TestHandleSsh(CiTestCase): |
928 | self.assertEqual([mock.call(set(keys), user), |
929 | mock.call(set(keys), "root", options="")], |
930 | m_setup_keys.call_args_list) |
931 | + |
932 | + @mock.patch(MODPATH + "glob.glob") |
933 | + @mock.patch(MODPATH + "ug_util.normalize_users_groups") |
934 | + @mock.patch(MODPATH + "os.path.exists") |
935 | + def test_handle_publish_hostkeys_default( |
936 | + self, m_path_exists, m_nug, m_glob, m_setup_keys): |
937 | + """Test handle with various configs for ssh_publish_hostkeys.""" |
938 | + self._publish_hostkey_test_setup() |
939 | + cc_ssh.PUBLISH_HOST_KEYS = True |
940 | + keys = ["key1"] |
941 | + user = "clouduser" |
942 | + # Return no matching keys for first glob, test keys for second. |
943 | + m_glob.side_effect = iter([ |
944 | + [], |
945 | + self.test_hostkey_files, |
946 | + ]) |
947 | + # Mock os.path.exits to True to short-circuit the key writing logic |
948 | + m_path_exists.return_value = True |
949 | + m_nug.return_value = ({user: {"default": user}}, {}) |
950 | + cloud = self.tmp_cloud( |
951 | + distro='ubuntu', metadata={'public-keys': keys}) |
952 | + cloud.datasource.publish_host_keys = mock.Mock() |
953 | + |
954 | + cfg = {} |
955 | + expected_call = [self.test_hostkeys[key_type] for key_type |
956 | + in ['ecdsa', 'ed25519', 'rsa']] |
957 | + cc_ssh.handle("name", cfg, cloud, None, None) |
958 | + self.assertEqual([mock.call(expected_call)], |
959 | + cloud.datasource.publish_host_keys.call_args_list) |
960 | + |
961 | + @mock.patch(MODPATH + "glob.glob") |
962 | + @mock.patch(MODPATH + "ug_util.normalize_users_groups") |
963 | + @mock.patch(MODPATH + "os.path.exists") |
964 | + def test_handle_publish_hostkeys_config_enable( |
965 | + self, m_path_exists, m_nug, m_glob, m_setup_keys): |
966 | + """Test handle with various configs for ssh_publish_hostkeys.""" |
967 | + self._publish_hostkey_test_setup() |
968 | + cc_ssh.PUBLISH_HOST_KEYS = False |
969 | + keys = ["key1"] |
970 | + user = "clouduser" |
971 | + # Return no matching keys for first glob, test keys for second. |
972 | + m_glob.side_effect = iter([ |
973 | + [], |
974 | + self.test_hostkey_files, |
975 | + ]) |
976 | + # Mock os.path.exits to True to short-circuit the key writing logic |
977 | + m_path_exists.return_value = True |
978 | + m_nug.return_value = ({user: {"default": user}}, {}) |
979 | + cloud = self.tmp_cloud( |
980 | + distro='ubuntu', metadata={'public-keys': keys}) |
981 | + cloud.datasource.publish_host_keys = mock.Mock() |
982 | + |
983 | + cfg = {'ssh_publish_hostkeys': {'enabled': True}} |
984 | + expected_call = [self.test_hostkeys[key_type] for key_type |
985 | + in ['ecdsa', 'ed25519', 'rsa']] |
986 | + cc_ssh.handle("name", cfg, cloud, None, None) |
987 | + self.assertEqual([mock.call(expected_call)], |
988 | + cloud.datasource.publish_host_keys.call_args_list) |
989 | + |
990 | + @mock.patch(MODPATH + "glob.glob") |
991 | + @mock.patch(MODPATH + "ug_util.normalize_users_groups") |
992 | + @mock.patch(MODPATH + "os.path.exists") |
993 | + def test_handle_publish_hostkeys_config_disable( |
994 | + self, m_path_exists, m_nug, m_glob, m_setup_keys): |
995 | + """Test handle with various configs for ssh_publish_hostkeys.""" |
996 | + self._publish_hostkey_test_setup() |
997 | + cc_ssh.PUBLISH_HOST_KEYS = True |
998 | + keys = ["key1"] |
999 | + user = "clouduser" |
1000 | + # Return no matching keys for first glob, test keys for second. |
1001 | + m_glob.side_effect = iter([ |
1002 | + [], |
1003 | + self.test_hostkey_files, |
1004 | + ]) |
1005 | + # Mock os.path.exits to True to short-circuit the key writing logic |
1006 | + m_path_exists.return_value = True |
1007 | + m_nug.return_value = ({user: {"default": user}}, {}) |
1008 | + cloud = self.tmp_cloud( |
1009 | + distro='ubuntu', metadata={'public-keys': keys}) |
1010 | + cloud.datasource.publish_host_keys = mock.Mock() |
1011 | + |
1012 | + cfg = {'ssh_publish_hostkeys': {'enabled': False}} |
1013 | + cc_ssh.handle("name", cfg, cloud, None, None) |
1014 | + self.assertFalse(cloud.datasource.publish_host_keys.call_args_list) |
1015 | + cloud.datasource.publish_host_keys.assert_not_called() |
1016 | + |
1017 | + @mock.patch(MODPATH + "glob.glob") |
1018 | + @mock.patch(MODPATH + "ug_util.normalize_users_groups") |
1019 | + @mock.patch(MODPATH + "os.path.exists") |
1020 | + def test_handle_publish_hostkeys_config_blacklist( |
1021 | + self, m_path_exists, m_nug, m_glob, m_setup_keys): |
1022 | + """Test handle with various configs for ssh_publish_hostkeys.""" |
1023 | + self._publish_hostkey_test_setup() |
1024 | + cc_ssh.PUBLISH_HOST_KEYS = True |
1025 | + keys = ["key1"] |
1026 | + user = "clouduser" |
1027 | + # Return no matching keys for first glob, test keys for second. |
1028 | + m_glob.side_effect = iter([ |
1029 | + [], |
1030 | + self.test_hostkey_files, |
1031 | + ]) |
1032 | + # Mock os.path.exits to True to short-circuit the key writing logic |
1033 | + m_path_exists.return_value = True |
1034 | + m_nug.return_value = ({user: {"default": user}}, {}) |
1035 | + cloud = self.tmp_cloud( |
1036 | + distro='ubuntu', metadata={'public-keys': keys}) |
1037 | + cloud.datasource.publish_host_keys = mock.Mock() |
1038 | + |
1039 | + cfg = {'ssh_publish_hostkeys': {'enabled': True, |
1040 | + 'blacklist': ['dsa', 'rsa']}} |
1041 | + expected_call = [self.test_hostkeys[key_type] for key_type |
1042 | + in ['ecdsa', 'ed25519']] |
1043 | + cc_ssh.handle("name", cfg, cloud, None, None) |
1044 | + self.assertEqual([mock.call(expected_call)], |
1045 | + cloud.datasource.publish_host_keys.call_args_list) |
1046 | + |
1047 | + @mock.patch(MODPATH + "glob.glob") |
1048 | + @mock.patch(MODPATH + "ug_util.normalize_users_groups") |
1049 | + @mock.patch(MODPATH + "os.path.exists") |
1050 | + def test_handle_publish_hostkeys_empty_blacklist( |
1051 | + self, m_path_exists, m_nug, m_glob, m_setup_keys): |
1052 | + """Test handle with various configs for ssh_publish_hostkeys.""" |
1053 | + self._publish_hostkey_test_setup() |
1054 | + cc_ssh.PUBLISH_HOST_KEYS = True |
1055 | + keys = ["key1"] |
1056 | + user = "clouduser" |
1057 | + # Return no matching keys for first glob, test keys for second. |
1058 | + m_glob.side_effect = iter([ |
1059 | + [], |
1060 | + self.test_hostkey_files, |
1061 | + ]) |
1062 | + # Mock os.path.exits to True to short-circuit the key writing logic |
1063 | + m_path_exists.return_value = True |
1064 | + m_nug.return_value = ({user: {"default": user}}, {}) |
1065 | + cloud = self.tmp_cloud( |
1066 | + distro='ubuntu', metadata={'public-keys': keys}) |
1067 | + cloud.datasource.publish_host_keys = mock.Mock() |
1068 | + |
1069 | + cfg = {'ssh_publish_hostkeys': {'enabled': True, |
1070 | + 'blacklist': []}} |
1071 | + expected_call = [self.test_hostkeys[key_type] for key_type |
1072 | + in ['dsa', 'ecdsa', 'ed25519', 'rsa']] |
1073 | + cc_ssh.handle("name", cfg, cloud, None, None) |
1074 | + self.assertEqual([mock.call(expected_call)], |
1075 | + cloud.datasource.publish_host_keys.call_args_list) |
1076 | diff --git a/cloudinit/config/tests/test_ubuntu_drivers.py b/cloudinit/config/tests/test_ubuntu_drivers.py |
1077 | index efba4ce..4695269 100644 |
1078 | --- a/cloudinit/config/tests/test_ubuntu_drivers.py |
1079 | +++ b/cloudinit/config/tests/test_ubuntu_drivers.py |
1080 | @@ -1,6 +1,7 @@ |
1081 | # This file is part of cloud-init. See LICENSE file for license information. |
1082 | |
1083 | import copy |
1084 | +import os |
1085 | |
1086 | from cloudinit.tests.helpers import CiTestCase, skipUnlessJsonSchema, mock |
1087 | from cloudinit.config.schema import ( |
1088 | @@ -9,11 +10,27 @@ from cloudinit.config import cc_ubuntu_drivers as drivers |
1089 | from cloudinit.util import ProcessExecutionError |
1090 | |
1091 | MPATH = "cloudinit.config.cc_ubuntu_drivers." |
1092 | +M_TMP_PATH = MPATH + "temp_utils.mkdtemp" |
1093 | OLD_UBUNTU_DRIVERS_ERROR_STDERR = ( |
1094 | "ubuntu-drivers: error: argument <command>: invalid choice: 'install' " |
1095 | "(choose from 'list', 'autoinstall', 'devices', 'debug')\n") |
1096 | |
1097 | |
1098 | +class AnyTempScriptAndDebconfFile(object): |
1099 | + |
1100 | + def __init__(self, tmp_dir, debconf_file): |
1101 | + self.tmp_dir = tmp_dir |
1102 | + self.debconf_file = debconf_file |
1103 | + |
1104 | + def __eq__(self, cmd): |
1105 | + if not len(cmd) == 2: |
1106 | + return False |
1107 | + script, debconf_file = cmd |
1108 | + if bool(script.startswith(self.tmp_dir) and script.endswith('.sh')): |
1109 | + return debconf_file == self.debconf_file |
1110 | + return False |
1111 | + |
1112 | + |
1113 | class TestUbuntuDrivers(CiTestCase): |
1114 | cfg_accepted = {'drivers': {'nvidia': {'license-accepted': True}}} |
1115 | install_gpgpu = ['ubuntu-drivers', 'install', '--gpgpu', 'nvidia'] |
1116 | @@ -28,16 +45,23 @@ class TestUbuntuDrivers(CiTestCase): |
1117 | {'drivers': {'nvidia': {'license-accepted': "TRUE"}}}, |
1118 | schema=drivers.schema, strict=True) |
1119 | |
1120 | + @mock.patch(M_TMP_PATH) |
1121 | @mock.patch(MPATH + "util.subp", return_value=('', '')) |
1122 | @mock.patch(MPATH + "util.which", return_value=False) |
1123 | - def _assert_happy_path_taken(self, config, m_which, m_subp): |
1124 | + def _assert_happy_path_taken( |
1125 | + self, config, m_which, m_subp, m_tmp): |
1126 | """Positive path test through handle. Package should be installed.""" |
1127 | + tdir = self.tmp_dir() |
1128 | + debconf_file = os.path.join(tdir, 'nvidia.template') |
1129 | + m_tmp.return_value = tdir |
1130 | myCloud = mock.MagicMock() |
1131 | drivers.handle('ubuntu_drivers', config, myCloud, None, None) |
1132 | self.assertEqual([mock.call(['ubuntu-drivers-common'])], |
1133 | myCloud.distro.install_packages.call_args_list) |
1134 | - self.assertEqual([mock.call(self.install_gpgpu)], |
1135 | - m_subp.call_args_list) |
1136 | + self.assertEqual( |
1137 | + [mock.call(AnyTempScriptAndDebconfFile(tdir, debconf_file)), |
1138 | + mock.call(self.install_gpgpu)], |
1139 | + m_subp.call_args_list) |
1140 | |
1141 | def test_handle_does_package_install(self): |
1142 | self._assert_happy_path_taken(self.cfg_accepted) |
1143 | @@ -48,19 +72,33 @@ class TestUbuntuDrivers(CiTestCase): |
1144 | new_config['drivers']['nvidia']['license-accepted'] = true_value |
1145 | self._assert_happy_path_taken(new_config) |
1146 | |
1147 | - @mock.patch(MPATH + "util.subp", side_effect=ProcessExecutionError( |
1148 | - stdout='No drivers found for installation.\n', exit_code=1)) |
1149 | + @mock.patch(M_TMP_PATH) |
1150 | + @mock.patch(MPATH + "util.subp") |
1151 | @mock.patch(MPATH + "util.which", return_value=False) |
1152 | - def test_handle_raises_error_if_no_drivers_found(self, m_which, m_subp): |
1153 | + def test_handle_raises_error_if_no_drivers_found( |
1154 | + self, m_which, m_subp, m_tmp): |
1155 | """If ubuntu-drivers doesn't install any drivers, raise an error.""" |
1156 | + tdir = self.tmp_dir() |
1157 | + debconf_file = os.path.join(tdir, 'nvidia.template') |
1158 | + m_tmp.return_value = tdir |
1159 | myCloud = mock.MagicMock() |
1160 | + |
1161 | + def fake_subp(cmd): |
1162 | + if cmd[0].startswith(tdir): |
1163 | + return |
1164 | + raise ProcessExecutionError( |
1165 | + stdout='No drivers found for installation.\n', exit_code=1) |
1166 | + m_subp.side_effect = fake_subp |
1167 | + |
1168 | with self.assertRaises(Exception): |
1169 | drivers.handle( |
1170 | 'ubuntu_drivers', self.cfg_accepted, myCloud, None, None) |
1171 | self.assertEqual([mock.call(['ubuntu-drivers-common'])], |
1172 | myCloud.distro.install_packages.call_args_list) |
1173 | - self.assertEqual([mock.call(self.install_gpgpu)], |
1174 | - m_subp.call_args_list) |
1175 | + self.assertEqual( |
1176 | + [mock.call(AnyTempScriptAndDebconfFile(tdir, debconf_file)), |
1177 | + mock.call(self.install_gpgpu)], |
1178 | + m_subp.call_args_list) |
1179 | self.assertIn('ubuntu-drivers found no drivers for installation', |
1180 | self.logs.getvalue()) |
1181 | |
1182 | @@ -108,18 +146,25 @@ class TestUbuntuDrivers(CiTestCase): |
1183 | myLog.debug.call_args_list[0][0][0]) |
1184 | self.assertEqual(0, m_install_drivers.call_count) |
1185 | |
1186 | + @mock.patch(M_TMP_PATH) |
1187 | @mock.patch(MPATH + "util.subp", return_value=('', '')) |
1188 | @mock.patch(MPATH + "util.which", return_value=True) |
1189 | - def test_install_drivers_no_install_if_present(self, m_which, m_subp): |
1190 | + def test_install_drivers_no_install_if_present( |
1191 | + self, m_which, m_subp, m_tmp): |
1192 | """If 'ubuntu-drivers' is present, no package install should occur.""" |
1193 | + tdir = self.tmp_dir() |
1194 | + debconf_file = os.path.join(tdir, 'nvidia.template') |
1195 | + m_tmp.return_value = tdir |
1196 | pkg_install = mock.MagicMock() |
1197 | drivers.install_drivers(self.cfg_accepted['drivers'], |
1198 | pkg_install_func=pkg_install) |
1199 | self.assertEqual(0, pkg_install.call_count) |
1200 | self.assertEqual([mock.call('ubuntu-drivers')], |
1201 | m_which.call_args_list) |
1202 | - self.assertEqual([mock.call(self.install_gpgpu)], |
1203 | - m_subp.call_args_list) |
1204 | + self.assertEqual( |
1205 | + [mock.call(AnyTempScriptAndDebconfFile(tdir, debconf_file)), |
1206 | + mock.call(self.install_gpgpu)], |
1207 | + m_subp.call_args_list) |
1208 | |
1209 | def test_install_drivers_rejects_invalid_config(self): |
1210 | """install_drivers should raise TypeError if not given a config dict""" |
1211 | @@ -128,20 +173,33 @@ class TestUbuntuDrivers(CiTestCase): |
1212 | drivers.install_drivers("mystring", pkg_install_func=pkg_install) |
1213 | self.assertEqual(0, pkg_install.call_count) |
1214 | |
1215 | - @mock.patch(MPATH + "util.subp", side_effect=ProcessExecutionError( |
1216 | - stderr=OLD_UBUNTU_DRIVERS_ERROR_STDERR, exit_code=2)) |
1217 | + @mock.patch(M_TMP_PATH) |
1218 | + @mock.patch(MPATH + "util.subp") |
1219 | @mock.patch(MPATH + "util.which", return_value=False) |
1220 | def test_install_drivers_handles_old_ubuntu_drivers_gracefully( |
1221 | - self, m_which, m_subp): |
1222 | + self, m_which, m_subp, m_tmp): |
1223 | """Older ubuntu-drivers versions should emit message and raise error""" |
1224 | + tdir = self.tmp_dir() |
1225 | + debconf_file = os.path.join(tdir, 'nvidia.template') |
1226 | + m_tmp.return_value = tdir |
1227 | myCloud = mock.MagicMock() |
1228 | + |
1229 | + def fake_subp(cmd): |
1230 | + if cmd[0].startswith(tdir): |
1231 | + return |
1232 | + raise ProcessExecutionError( |
1233 | + stderr=OLD_UBUNTU_DRIVERS_ERROR_STDERR, exit_code=2) |
1234 | + m_subp.side_effect = fake_subp |
1235 | + |
1236 | with self.assertRaises(Exception): |
1237 | drivers.handle( |
1238 | 'ubuntu_drivers', self.cfg_accepted, myCloud, None, None) |
1239 | self.assertEqual([mock.call(['ubuntu-drivers-common'])], |
1240 | myCloud.distro.install_packages.call_args_list) |
1241 | - self.assertEqual([mock.call(self.install_gpgpu)], |
1242 | - m_subp.call_args_list) |
1243 | + self.assertEqual( |
1244 | + [mock.call(AnyTempScriptAndDebconfFile(tdir, debconf_file)), |
1245 | + mock.call(self.install_gpgpu)], |
1246 | + m_subp.call_args_list) |
1247 | self.assertIn('WARNING: the available version of ubuntu-drivers is' |
1248 | ' too old to perform requested driver installation', |
1249 | self.logs.getvalue()) |
1250 | @@ -153,16 +211,21 @@ class TestUbuntuDriversWithVersion(TestUbuntuDrivers): |
1251 | 'drivers': {'nvidia': {'license-accepted': True, 'version': '123'}}} |
1252 | install_gpgpu = ['ubuntu-drivers', 'install', '--gpgpu', 'nvidia:123'] |
1253 | |
1254 | + @mock.patch(M_TMP_PATH) |
1255 | @mock.patch(MPATH + "util.subp", return_value=('', '')) |
1256 | @mock.patch(MPATH + "util.which", return_value=False) |
1257 | - def test_version_none_uses_latest(self, m_which, m_subp): |
1258 | + def test_version_none_uses_latest(self, m_which, m_subp, m_tmp): |
1259 | + tdir = self.tmp_dir() |
1260 | + debconf_file = os.path.join(tdir, 'nvidia.template') |
1261 | + m_tmp.return_value = tdir |
1262 | myCloud = mock.MagicMock() |
1263 | version_none_cfg = { |
1264 | 'drivers': {'nvidia': {'license-accepted': True, 'version': None}}} |
1265 | drivers.handle( |
1266 | 'ubuntu_drivers', version_none_cfg, myCloud, None, None) |
1267 | self.assertEqual( |
1268 | - [mock.call(['ubuntu-drivers', 'install', '--gpgpu', 'nvidia'])], |
1269 | + [mock.call(AnyTempScriptAndDebconfFile(tdir, debconf_file)), |
1270 | + mock.call(['ubuntu-drivers', 'install', '--gpgpu', 'nvidia'])], |
1271 | m_subp.call_args_list) |
1272 | |
1273 | def test_specifying_a_version_doesnt_override_license_acceptance(self): |
1274 | diff --git a/cloudinit/distros/__init__.py b/cloudinit/distros/__init__.py |
1275 | index 20c994d..00bdee3 100644 |
1276 | --- a/cloudinit/distros/__init__.py |
1277 | +++ b/cloudinit/distros/__init__.py |
1278 | @@ -396,16 +396,16 @@ class Distro(object): |
1279 | else: |
1280 | create_groups = True |
1281 | |
1282 | - adduser_cmd = ['useradd', name] |
1283 | - log_adduser_cmd = ['useradd', name] |
1284 | + useradd_cmd = ['useradd', name] |
1285 | + log_useradd_cmd = ['useradd', name] |
1286 | if util.system_is_snappy(): |
1287 | - adduser_cmd.append('--extrausers') |
1288 | - log_adduser_cmd.append('--extrausers') |
1289 | + useradd_cmd.append('--extrausers') |
1290 | + log_useradd_cmd.append('--extrausers') |
1291 | |
1292 | # Since we are creating users, we want to carefully validate the |
1293 | # inputs. If something goes wrong, we can end up with a system |
1294 | # that nobody can login to. |
1295 | - adduser_opts = { |
1296 | + useradd_opts = { |
1297 | "gecos": '--comment', |
1298 | "homedir": '--home', |
1299 | "primary_group": '--gid', |
1300 | @@ -418,7 +418,7 @@ class Distro(object): |
1301 | "selinux_user": '--selinux-user', |
1302 | } |
1303 | |
1304 | - adduser_flags = { |
1305 | + useradd_flags = { |
1306 | "no_user_group": '--no-user-group', |
1307 | "system": '--system', |
1308 | "no_log_init": '--no-log-init', |
1309 | @@ -453,32 +453,32 @@ class Distro(object): |
1310 | # Check the values and create the command |
1311 | for key, val in sorted(kwargs.items()): |
1312 | |
1313 | - if key in adduser_opts and val and isinstance(val, str): |
1314 | - adduser_cmd.extend([adduser_opts[key], val]) |
1315 | + if key in useradd_opts and val and isinstance(val, str): |
1316 | + useradd_cmd.extend([useradd_opts[key], val]) |
1317 | |
1318 | # Redact certain fields from the logs |
1319 | if key in redact_opts: |
1320 | - log_adduser_cmd.extend([adduser_opts[key], 'REDACTED']) |
1321 | + log_useradd_cmd.extend([useradd_opts[key], 'REDACTED']) |
1322 | else: |
1323 | - log_adduser_cmd.extend([adduser_opts[key], val]) |
1324 | + log_useradd_cmd.extend([useradd_opts[key], val]) |
1325 | |
1326 | - elif key in adduser_flags and val: |
1327 | - adduser_cmd.append(adduser_flags[key]) |
1328 | - log_adduser_cmd.append(adduser_flags[key]) |
1329 | + elif key in useradd_flags and val: |
1330 | + useradd_cmd.append(useradd_flags[key]) |
1331 | + log_useradd_cmd.append(useradd_flags[key]) |
1332 | |
1333 | # Don't create the home directory if directed so or if the user is a |
1334 | # system user |
1335 | if kwargs.get('no_create_home') or kwargs.get('system'): |
1336 | - adduser_cmd.append('-M') |
1337 | - log_adduser_cmd.append('-M') |
1338 | + useradd_cmd.append('-M') |
1339 | + log_useradd_cmd.append('-M') |
1340 | else: |
1341 | - adduser_cmd.append('-m') |
1342 | - log_adduser_cmd.append('-m') |
1343 | + useradd_cmd.append('-m') |
1344 | + log_useradd_cmd.append('-m') |
1345 | |
1346 | # Run the command |
1347 | LOG.debug("Adding user %s", name) |
1348 | try: |
1349 | - util.subp(adduser_cmd, logstring=log_adduser_cmd) |
1350 | + util.subp(useradd_cmd, logstring=log_useradd_cmd) |
1351 | except Exception as e: |
1352 | util.logexc(LOG, "Failed to create user %s", name) |
1353 | raise e |
1354 | @@ -490,15 +490,15 @@ class Distro(object): |
1355 | |
1356 | snapuser = kwargs.get('snapuser') |
1357 | known = kwargs.get('known', False) |
1358 | - adduser_cmd = ["snap", "create-user", "--sudoer", "--json"] |
1359 | + create_user_cmd = ["snap", "create-user", "--sudoer", "--json"] |
1360 | if known: |
1361 | - adduser_cmd.append("--known") |
1362 | - adduser_cmd.append(snapuser) |
1363 | + create_user_cmd.append("--known") |
1364 | + create_user_cmd.append(snapuser) |
1365 | |
1366 | # Run the command |
1367 | LOG.debug("Adding snap user %s", name) |
1368 | try: |
1369 | - (out, err) = util.subp(adduser_cmd, logstring=adduser_cmd, |
1370 | + (out, err) = util.subp(create_user_cmd, logstring=create_user_cmd, |
1371 | capture=True) |
1372 | LOG.debug("snap create-user returned: %s:%s", out, err) |
1373 | jobj = util.load_json(out) |
1374 | diff --git a/cloudinit/distros/arch.py b/cloudinit/distros/arch.py |
1375 | index b814c8b..9f89c5f 100644 |
1376 | --- a/cloudinit/distros/arch.py |
1377 | +++ b/cloudinit/distros/arch.py |
1378 | @@ -12,6 +12,8 @@ from cloudinit import util |
1379 | from cloudinit.distros import net_util |
1380 | from cloudinit.distros.parsers.hostname import HostnameConf |
1381 | |
1382 | +from cloudinit.net.renderers import RendererNotFoundError |
1383 | + |
1384 | from cloudinit.settings import PER_INSTANCE |
1385 | |
1386 | import os |
1387 | @@ -24,6 +26,11 @@ class Distro(distros.Distro): |
1388 | network_conf_dir = "/etc/netctl" |
1389 | resolve_conf_fn = "/etc/resolv.conf" |
1390 | init_cmd = ['systemctl'] # init scripts |
1391 | + renderer_configs = { |
1392 | + "netplan": {"netplan_path": "/etc/netplan/50-cloud-init.yaml", |
1393 | + "netplan_header": "# generated by cloud-init\n", |
1394 | + "postcmds": True} |
1395 | + } |
1396 | |
1397 | def __init__(self, name, cfg, paths): |
1398 | distros.Distro.__init__(self, name, cfg, paths) |
1399 | @@ -50,6 +57,13 @@ class Distro(distros.Distro): |
1400 | self.update_package_sources() |
1401 | self.package_command('', pkgs=pkglist) |
1402 | |
1403 | + def _write_network_config(self, netconfig): |
1404 | + try: |
1405 | + return self._supported_write_network_config(netconfig) |
1406 | + except RendererNotFoundError: |
1407 | + # Fall back to old _write_network |
1408 | + raise NotImplementedError |
1409 | + |
1410 | def _write_network(self, settings): |
1411 | entries = net_util.translate_network(settings) |
1412 | LOG.debug("Translated ubuntu style network settings %s into %s", |
1413 | diff --git a/cloudinit/distros/debian.py b/cloudinit/distros/debian.py |
1414 | index d517fb8..0ad93ff 100644 |
1415 | --- a/cloudinit/distros/debian.py |
1416 | +++ b/cloudinit/distros/debian.py |
1417 | @@ -36,14 +36,14 @@ ENI_HEADER = """# This file is generated from information provided by |
1418 | # network: {config: disabled} |
1419 | """ |
1420 | |
1421 | -NETWORK_CONF_FN = "/etc/network/interfaces.d/50-cloud-init.cfg" |
1422 | +NETWORK_CONF_FN = "/etc/network/interfaces.d/50-cloud-init" |
1423 | LOCALE_CONF_FN = "/etc/default/locale" |
1424 | |
1425 | |
1426 | class Distro(distros.Distro): |
1427 | hostname_conf_fn = "/etc/hostname" |
1428 | network_conf_fn = { |
1429 | - "eni": "/etc/network/interfaces.d/50-cloud-init.cfg", |
1430 | + "eni": "/etc/network/interfaces.d/50-cloud-init", |
1431 | "netplan": "/etc/netplan/50-cloud-init.yaml" |
1432 | } |
1433 | renderer_configs = { |
1434 | diff --git a/cloudinit/distros/freebsd.py b/cloudinit/distros/freebsd.py |
1435 | index ff22d56..f7825fd 100644 |
1436 | --- a/cloudinit/distros/freebsd.py |
1437 | +++ b/cloudinit/distros/freebsd.py |
1438 | @@ -185,10 +185,10 @@ class Distro(distros.Distro): |
1439 | LOG.info("User %s already exists, skipping.", name) |
1440 | return False |
1441 | |
1442 | - adduser_cmd = ['pw', 'useradd', '-n', name] |
1443 | - log_adduser_cmd = ['pw', 'useradd', '-n', name] |
1444 | + pw_useradd_cmd = ['pw', 'useradd', '-n', name] |
1445 | + log_pw_useradd_cmd = ['pw', 'useradd', '-n', name] |
1446 | |
1447 | - adduser_opts = { |
1448 | + pw_useradd_opts = { |
1449 | "homedir": '-d', |
1450 | "gecos": '-c', |
1451 | "primary_group": '-g', |
1452 | @@ -196,34 +196,34 @@ class Distro(distros.Distro): |
1453 | "shell": '-s', |
1454 | "inactive": '-E', |
1455 | } |
1456 | - adduser_flags = { |
1457 | + pw_useradd_flags = { |
1458 | "no_user_group": '--no-user-group', |
1459 | "system": '--system', |
1460 | "no_log_init": '--no-log-init', |
1461 | } |
1462 | |
1463 | for key, val in kwargs.items(): |
1464 | - if (key in adduser_opts and val and |
1465 | + if (key in pw_useradd_opts and val and |
1466 | isinstance(val, six.string_types)): |
1467 | - adduser_cmd.extend([adduser_opts[key], val]) |
1468 | + pw_useradd_cmd.extend([pw_useradd_opts[key], val]) |
1469 | |
1470 | - elif key in adduser_flags and val: |
1471 | - adduser_cmd.append(adduser_flags[key]) |
1472 | - log_adduser_cmd.append(adduser_flags[key]) |
1473 | + elif key in pw_useradd_flags and val: |
1474 | + pw_useradd_cmd.append(pw_useradd_flags[key]) |
1475 | + log_pw_useradd_cmd.append(pw_useradd_flags[key]) |
1476 | |
1477 | if 'no_create_home' in kwargs or 'system' in kwargs: |
1478 | - adduser_cmd.append('-d/nonexistent') |
1479 | - log_adduser_cmd.append('-d/nonexistent') |
1480 | + pw_useradd_cmd.append('-d/nonexistent') |
1481 | + log_pw_useradd_cmd.append('-d/nonexistent') |
1482 | else: |
1483 | - adduser_cmd.append('-d/usr/home/%s' % name) |
1484 | - adduser_cmd.append('-m') |
1485 | - log_adduser_cmd.append('-d/usr/home/%s' % name) |
1486 | - log_adduser_cmd.append('-m') |
1487 | + pw_useradd_cmd.append('-d/usr/home/%s' % name) |
1488 | + pw_useradd_cmd.append('-m') |
1489 | + log_pw_useradd_cmd.append('-d/usr/home/%s' % name) |
1490 | + log_pw_useradd_cmd.append('-m') |
1491 | |
1492 | # Run the command |
1493 | LOG.info("Adding user %s", name) |
1494 | try: |
1495 | - util.subp(adduser_cmd, logstring=log_adduser_cmd) |
1496 | + util.subp(pw_useradd_cmd, logstring=log_pw_useradd_cmd) |
1497 | except Exception as e: |
1498 | util.logexc(LOG, "Failed to create user %s", name) |
1499 | raise e |
1500 | diff --git a/cloudinit/distros/opensuse.py b/cloudinit/distros/opensuse.py |
1501 | index 1bfe047..e41e2f7 100644 |
1502 | --- a/cloudinit/distros/opensuse.py |
1503 | +++ b/cloudinit/distros/opensuse.py |
1504 | @@ -38,6 +38,8 @@ class Distro(distros.Distro): |
1505 | 'sysconfig': { |
1506 | 'control': 'etc/sysconfig/network/config', |
1507 | 'iface_templates': '%(base)s/network/ifcfg-%(name)s', |
1508 | + 'netrules_path': ( |
1509 | + 'etc/udev/rules.d/85-persistent-net-cloud-init.rules'), |
1510 | 'route_templates': { |
1511 | 'ipv4': '%(base)s/network/ifroute-%(name)s', |
1512 | 'ipv6': '%(base)s/network/ifroute-%(name)s', |
1513 | diff --git a/cloudinit/distros/parsers/sys_conf.py b/cloudinit/distros/parsers/sys_conf.py |
1514 | index c27b5d5..44df17d 100644 |
1515 | --- a/cloudinit/distros/parsers/sys_conf.py |
1516 | +++ b/cloudinit/distros/parsers/sys_conf.py |
1517 | @@ -43,6 +43,13 @@ def _contains_shell_variable(text): |
1518 | |
1519 | |
1520 | class SysConf(configobj.ConfigObj): |
1521 | + """A configobj.ConfigObj subclass specialised for sysconfig files. |
1522 | + |
1523 | + :param contents: |
1524 | + The sysconfig file to parse, in a format accepted by |
1525 | + ``configobj.ConfigObj.__init__`` (i.e. "a filename, file like object, |
1526 | + or list of lines"). |
1527 | + """ |
1528 | def __init__(self, contents): |
1529 | configobj.ConfigObj.__init__(self, contents, |
1530 | interpolation=False, |
1531 | diff --git a/cloudinit/distros/ubuntu.py b/cloudinit/distros/ubuntu.py |
1532 | index 6815410..e5fcbc5 100644 |
1533 | --- a/cloudinit/distros/ubuntu.py |
1534 | +++ b/cloudinit/distros/ubuntu.py |
1535 | @@ -21,6 +21,21 @@ LOG = logging.getLogger(__name__) |
1536 | |
1537 | class Distro(debian.Distro): |
1538 | |
1539 | + def __init__(self, name, cfg, paths): |
1540 | + super(Distro, self).__init__(name, cfg, paths) |
1541 | + # Ubuntu specific network cfg locations |
1542 | + self.network_conf_fn = { |
1543 | + "eni": "/etc/network/interfaces.d/50-cloud-init.cfg", |
1544 | + "netplan": "/etc/netplan/50-cloud-init.yaml" |
1545 | + } |
1546 | + self.renderer_configs = { |
1547 | + "eni": {"eni_path": self.network_conf_fn["eni"], |
1548 | + "eni_header": debian.ENI_HEADER}, |
1549 | + "netplan": {"netplan_path": self.network_conf_fn["netplan"], |
1550 | + "netplan_header": debian.ENI_HEADER, |
1551 | + "postcmds": True} |
1552 | + } |
1553 | + |
1554 | @property |
1555 | def preferred_ntp_clients(self): |
1556 | """The preferred ntp client is dependent on the version.""" |
1557 | diff --git a/cloudinit/net/__init__.py b/cloudinit/net/__init__.py |
1558 | index 3642fb1..ea707c0 100644 |
1559 | --- a/cloudinit/net/__init__.py |
1560 | +++ b/cloudinit/net/__init__.py |
1561 | @@ -9,6 +9,7 @@ import errno |
1562 | import logging |
1563 | import os |
1564 | import re |
1565 | +from functools import partial |
1566 | |
1567 | from cloudinit.net.network_state import mask_to_net_prefix |
1568 | from cloudinit import util |
1569 | @@ -264,46 +265,29 @@ def find_fallback_nic(blacklist_drivers=None): |
1570 | |
1571 | |
1572 | def generate_fallback_config(blacklist_drivers=None, config_driver=None): |
1573 | - """Determine which attached net dev is most likely to have a connection and |
1574 | - generate network state to run dhcp on that interface""" |
1575 | - |
1576 | + """Generate network cfg v2 for dhcp on the NIC most likely connected.""" |
1577 | if not config_driver: |
1578 | config_driver = False |
1579 | |
1580 | target_name = find_fallback_nic(blacklist_drivers=blacklist_drivers) |
1581 | - if target_name: |
1582 | - target_mac = read_sys_net_safe(target_name, 'address') |
1583 | - nconf = {'config': [], 'version': 1} |
1584 | - cfg = {'type': 'physical', 'name': target_name, |
1585 | - 'mac_address': target_mac, 'subnets': [{'type': 'dhcp'}]} |
1586 | - # inject the device driver name, dev_id into config if enabled and |
1587 | - # device has a valid device driver value |
1588 | - if config_driver: |
1589 | - driver = device_driver(target_name) |
1590 | - if driver: |
1591 | - cfg['params'] = { |
1592 | - 'driver': driver, |
1593 | - 'device_id': device_devid(target_name), |
1594 | - } |
1595 | - nconf['config'].append(cfg) |
1596 | - return nconf |
1597 | - else: |
1598 | + if not target_name: |
1599 | # can't read any interfaces addresses (or there are none); give up |
1600 | return None |
1601 | + target_mac = read_sys_net_safe(target_name, 'address') |
1602 | + cfg = {'dhcp4': True, 'set-name': target_name, |
1603 | + 'match': {'macaddress': target_mac.lower()}} |
1604 | + if config_driver: |
1605 | + driver = device_driver(target_name) |
1606 | + if driver: |
1607 | + cfg['match']['driver'] = driver |
1608 | + nconf = {'ethernets': {target_name: cfg}, 'version': 2} |
1609 | + return nconf |
1610 | |
1611 | |
1612 | -def apply_network_config_names(netcfg, strict_present=True, strict_busy=True): |
1613 | - """read the network config and rename devices accordingly. |
1614 | - if strict_present is false, then do not raise exception if no devices |
1615 | - match. if strict_busy is false, then do not raise exception if the |
1616 | - device cannot be renamed because it is currently configured. |
1617 | - |
1618 | - renames are only attempted for interfaces of type 'physical'. It is |
1619 | - expected that the network system will create other devices with the |
1620 | - correct name in place.""" |
1621 | +def extract_physdevs(netcfg): |
1622 | |
1623 | def _version_1(netcfg): |
1624 | - renames = [] |
1625 | + physdevs = [] |
1626 | for ent in netcfg.get('config', {}): |
1627 | if ent.get('type') != 'physical': |
1628 | continue |
1629 | @@ -317,11 +301,11 @@ def apply_network_config_names(netcfg, strict_present=True, strict_busy=True): |
1630 | driver = device_driver(name) |
1631 | if not device_id: |
1632 | device_id = device_devid(name) |
1633 | - renames.append([mac, name, driver, device_id]) |
1634 | - return renames |
1635 | + physdevs.append([mac, name, driver, device_id]) |
1636 | + return physdevs |
1637 | |
1638 | def _version_2(netcfg): |
1639 | - renames = [] |
1640 | + physdevs = [] |
1641 | for ent in netcfg.get('ethernets', {}).values(): |
1642 | # only rename if configured to do so |
1643 | name = ent.get('set-name') |
1644 | @@ -337,16 +321,69 @@ def apply_network_config_names(netcfg, strict_present=True, strict_busy=True): |
1645 | driver = device_driver(name) |
1646 | if not device_id: |
1647 | device_id = device_devid(name) |
1648 | - renames.append([mac, name, driver, device_id]) |
1649 | - return renames |
1650 | + physdevs.append([mac, name, driver, device_id]) |
1651 | + return physdevs |
1652 | + |
1653 | + version = netcfg.get('version') |
1654 | + if version == 1: |
1655 | + return _version_1(netcfg) |
1656 | + elif version == 2: |
1657 | + return _version_2(netcfg) |
1658 | + |
1659 | + raise RuntimeError('Unknown network config version: %s' % version) |
1660 | + |
1661 | + |
1662 | +def wait_for_physdevs(netcfg, strict=True): |
1663 | + physdevs = extract_physdevs(netcfg) |
1664 | + |
1665 | + # set of expected iface names and mac addrs |
1666 | + expected_ifaces = dict([(iface[0], iface[1]) for iface in physdevs]) |
1667 | + expected_macs = set(expected_ifaces.keys()) |
1668 | + |
1669 | + # set of current macs |
1670 | + present_macs = get_interfaces_by_mac().keys() |
1671 | + |
1672 | + # compare the set of expected mac address values to |
1673 | + # the current macs present; we only check MAC as cloud-init |
1674 | + # has not yet renamed interfaces and the netcfg may include |
1675 | + # such renames. |
1676 | + for _ in range(0, 5): |
1677 | + if expected_macs.issubset(present_macs): |
1678 | + LOG.debug('net: all expected physical devices present') |
1679 | + return |
1680 | |
1681 | - if netcfg.get('version') == 1: |
1682 | - return _rename_interfaces(_version_1(netcfg)) |
1683 | - elif netcfg.get('version') == 2: |
1684 | - return _rename_interfaces(_version_2(netcfg)) |
1685 | + missing = expected_macs.difference(present_macs) |
1686 | + LOG.debug('net: waiting for expected net devices: %s', missing) |
1687 | + for mac in missing: |
1688 | + # trigger a settle, unless this interface exists |
1689 | + syspath = sys_dev_path(expected_ifaces[mac]) |
1690 | + settle = partial(util.udevadm_settle, exists=syspath) |
1691 | + msg = 'Waiting for udev events to settle or %s exists' % syspath |
1692 | + util.log_time(LOG.debug, msg, func=settle) |
1693 | |
1694 | - raise RuntimeError('Failed to apply network config names. Found bad' |
1695 | - ' network config version: %s' % netcfg.get('version')) |
1696 | + # update present_macs after settles |
1697 | + present_macs = get_interfaces_by_mac().keys() |
1698 | + |
1699 | + msg = 'Not all expected physical devices present: %s' % missing |
1700 | + LOG.warning(msg) |
1701 | + if strict: |
1702 | + raise RuntimeError(msg) |
1703 | + |
1704 | + |
1705 | +def apply_network_config_names(netcfg, strict_present=True, strict_busy=True): |
1706 | + """read the network config and rename devices accordingly. |
1707 | + if strict_present is false, then do not raise exception if no devices |
1708 | + match. if strict_busy is false, then do not raise exception if the |
1709 | + device cannot be renamed because it is currently configured. |
1710 | + |
1711 | + renames are only attempted for interfaces of type 'physical'. It is |
1712 | + expected that the network system will create other devices with the |
1713 | + correct name in place.""" |
1714 | + |
1715 | + try: |
1716 | + _rename_interfaces(extract_physdevs(netcfg)) |
1717 | + except RuntimeError as e: |
1718 | + raise RuntimeError('Failed to apply network config names: %s' % e) |
1719 | |
1720 | |
1721 | def interface_has_own_mac(ifname, strict=False): |
1722 | @@ -622,6 +659,8 @@ def get_interfaces(): |
1723 | continue |
1724 | if is_vlan(name): |
1725 | continue |
1726 | + if is_bond(name): |
1727 | + continue |
1728 | mac = get_interface_mac(name) |
1729 | # some devices may not have a mac (tun0) |
1730 | if not mac: |
1731 | @@ -677,7 +716,7 @@ class EphemeralIPv4Network(object): |
1732 | """ |
1733 | |
1734 | def __init__(self, interface, ip, prefix_or_mask, broadcast, router=None, |
1735 | - connectivity_url=None): |
1736 | + connectivity_url=None, static_routes=None): |
1737 | """Setup context manager and validate call signature. |
1738 | |
1739 | @param interface: Name of the network interface to bring up. |
1740 | @@ -688,6 +727,7 @@ class EphemeralIPv4Network(object): |
1741 | @param router: Optionally the default gateway IP. |
1742 | @param connectivity_url: Optionally, a URL to verify if a usable |
1743 | connection already exists. |
1744 | + @param static_routes: Optionally a list of static routes from DHCP |
1745 | """ |
1746 | if not all([interface, ip, prefix_or_mask, broadcast]): |
1747 | raise ValueError( |
1748 | @@ -704,6 +744,7 @@ class EphemeralIPv4Network(object): |
1749 | self.ip = ip |
1750 | self.broadcast = broadcast |
1751 | self.router = router |
1752 | + self.static_routes = static_routes |
1753 | self.cleanup_cmds = [] # List of commands to run to cleanup state. |
1754 | |
1755 | def __enter__(self): |
1756 | @@ -716,7 +757,21 @@ class EphemeralIPv4Network(object): |
1757 | return |
1758 | |
1759 | self._bringup_device() |
1760 | - if self.router: |
1761 | + |
1762 | + # rfc3442 requires us to ignore the router config *if* classless static |
1763 | + # routes are provided. |
1764 | + # |
1765 | + # https://tools.ietf.org/html/rfc3442 |
1766 | + # |
1767 | + # If the DHCP server returns both a Classless Static Routes option and |
1768 | + # a Router option, the DHCP client MUST ignore the Router option. |
1769 | + # |
1770 | + # Similarly, if the DHCP server returns both a Classless Static Routes |
1771 | + # option and a Static Routes option, the DHCP client MUST ignore the |
1772 | + # Static Routes option. |
1773 | + if self.static_routes: |
1774 | + self._bringup_static_routes() |
1775 | + elif self.router: |
1776 | self._bringup_router() |
1777 | |
1778 | def __exit__(self, excp_type, excp_value, excp_traceback): |
1779 | @@ -760,6 +815,20 @@ class EphemeralIPv4Network(object): |
1780 | ['ip', '-family', 'inet', 'addr', 'del', cidr, 'dev', |
1781 | self.interface]) |
1782 | |
1783 | + def _bringup_static_routes(self): |
1784 | + # static_routes = [("169.254.169.254/32", "130.56.248.255"), |
1785 | + # ("0.0.0.0/0", "130.56.240.1")] |
1786 | + for net_address, gateway in self.static_routes: |
1787 | + via_arg = [] |
1788 | + if gateway != "0.0.0.0/0": |
1789 | + via_arg = ['via', gateway] |
1790 | + util.subp( |
1791 | + ['ip', '-4', 'route', 'add', net_address] + via_arg + |
1792 | + ['dev', self.interface], capture=True) |
1793 | + self.cleanup_cmds.insert( |
1794 | + 0, ['ip', '-4', 'route', 'del', net_address] + via_arg + |
1795 | + ['dev', self.interface]) |
1796 | + |
1797 | def _bringup_router(self): |
1798 | """Perform the ip commands to fully setup the router if needed.""" |
1799 | # Check if a default route exists and exit if it does |
1800 | diff --git a/cloudinit/net/cmdline.py b/cloudinit/net/cmdline.py |
1801 | index f89a0f7..556a10f 100755 |
1802 | --- a/cloudinit/net/cmdline.py |
1803 | +++ b/cloudinit/net/cmdline.py |
1804 | @@ -177,21 +177,13 @@ def _is_initramfs_netconfig(files, cmdline): |
1805 | return False |
1806 | |
1807 | |
1808 | -def read_kernel_cmdline_config(files=None, mac_addrs=None, cmdline=None): |
1809 | +def read_initramfs_config(files=None, mac_addrs=None, cmdline=None): |
1810 | if cmdline is None: |
1811 | cmdline = util.get_cmdline() |
1812 | |
1813 | if files is None: |
1814 | files = _get_klibc_net_cfg_files() |
1815 | |
1816 | - if 'network-config=' in cmdline: |
1817 | - data64 = None |
1818 | - for tok in cmdline.split(): |
1819 | - if tok.startswith("network-config="): |
1820 | - data64 = tok.split("=", 1)[1] |
1821 | - if data64: |
1822 | - return util.load_yaml(_b64dgz(data64)) |
1823 | - |
1824 | if not _is_initramfs_netconfig(files, cmdline): |
1825 | return None |
1826 | |
1827 | @@ -204,4 +196,19 @@ def read_kernel_cmdline_config(files=None, mac_addrs=None, cmdline=None): |
1828 | |
1829 | return config_from_klibc_net_cfg(files=files, mac_addrs=mac_addrs) |
1830 | |
1831 | + |
1832 | +def read_kernel_cmdline_config(cmdline=None): |
1833 | + if cmdline is None: |
1834 | + cmdline = util.get_cmdline() |
1835 | + |
1836 | + if 'network-config=' in cmdline: |
1837 | + data64 = None |
1838 | + for tok in cmdline.split(): |
1839 | + if tok.startswith("network-config="): |
1840 | + data64 = tok.split("=", 1)[1] |
1841 | + if data64: |
1842 | + return util.load_yaml(_b64dgz(data64)) |
1843 | + |
1844 | + return None |
1845 | + |
1846 | # vi: ts=4 expandtab |
1847 | diff --git a/cloudinit/net/dhcp.py b/cloudinit/net/dhcp.py |
1848 | index c98a97c..1737991 100644 |
1849 | --- a/cloudinit/net/dhcp.py |
1850 | +++ b/cloudinit/net/dhcp.py |
1851 | @@ -92,10 +92,14 @@ class EphemeralDHCPv4(object): |
1852 | nmap = {'interface': 'interface', 'ip': 'fixed-address', |
1853 | 'prefix_or_mask': 'subnet-mask', |
1854 | 'broadcast': 'broadcast-address', |
1855 | + 'static_routes': 'rfc3442-classless-static-routes', |
1856 | 'router': 'routers'} |
1857 | kwargs = dict([(k, self.lease.get(v)) for k, v in nmap.items()]) |
1858 | if not kwargs['broadcast']: |
1859 | kwargs['broadcast'] = bcip(kwargs['prefix_or_mask'], kwargs['ip']) |
1860 | + if kwargs['static_routes']: |
1861 | + kwargs['static_routes'] = ( |
1862 | + parse_static_routes(kwargs['static_routes'])) |
1863 | if self.connectivity_url: |
1864 | kwargs['connectivity_url'] = self.connectivity_url |
1865 | ephipv4 = EphemeralIPv4Network(**kwargs) |
1866 | @@ -272,4 +276,90 @@ def networkd_get_option_from_leases(keyname, leases_d=None): |
1867 | return data[keyname] |
1868 | return None |
1869 | |
1870 | + |
1871 | +def parse_static_routes(rfc3442): |
1872 | + """ parse rfc3442 format and return a list containing tuple of strings. |
1873 | + |
1874 | + The tuple is composed of the network_address (including net length) and |
1875 | + gateway for a parsed static route. |
1876 | + |
1877 | + @param rfc3442: string in rfc3442 format |
1878 | + @returns: list of tuple(str, str) for all valid parsed routes until the |
1879 | + first parsing error. |
1880 | + |
1881 | + E.g. |
1882 | + sr = parse_state_routes("32,169,254,169,254,130,56,248,255,0,130,56,240,1") |
1883 | + sr = [ |
1884 | + ("169.254.169.254/32", "130.56.248.255"), ("0.0.0.0/0", "130.56.240.1") |
1885 | + ] |
1886 | + |
1887 | + Python version of isc-dhclient's hooks: |
1888 | + /etc/dhcp/dhclient-exit-hooks.d/rfc3442-classless-routes |
1889 | + """ |
1890 | + # raw strings from dhcp lease may end in semi-colon |
1891 | + rfc3442 = rfc3442.rstrip(";") |
1892 | + tokens = rfc3442.split(',') |
1893 | + static_routes = [] |
1894 | + |
1895 | + def _trunc_error(cidr, required, remain): |
1896 | + msg = ("RFC3442 string malformed. Current route has CIDR of %s " |
1897 | + "and requires %s significant octets, but only %s remain. " |
1898 | + "Verify DHCP rfc3442-classless-static-routes value: %s" |
1899 | + % (cidr, required, remain, rfc3442)) |
1900 | + LOG.error(msg) |
1901 | + |
1902 | + current_idx = 0 |
1903 | + for idx, tok in enumerate(tokens): |
1904 | + if idx < current_idx: |
1905 | + continue |
1906 | + net_length = int(tok) |
1907 | + if net_length in range(25, 33): |
1908 | + req_toks = 9 |
1909 | + if len(tokens[idx:]) < req_toks: |
1910 | + _trunc_error(net_length, req_toks, len(tokens[idx:])) |
1911 | + return static_routes |
1912 | + net_address = ".".join(tokens[idx+1:idx+5]) |
1913 | + gateway = ".".join(tokens[idx+5:idx+req_toks]) |
1914 | + current_idx = idx + req_toks |
1915 | + elif net_length in range(17, 25): |
1916 | + req_toks = 8 |
1917 | + if len(tokens[idx:]) < req_toks: |
1918 | + _trunc_error(net_length, req_toks, len(tokens[idx:])) |
1919 | + return static_routes |
1920 | + net_address = ".".join(tokens[idx+1:idx+4] + ["0"]) |
1921 | + gateway = ".".join(tokens[idx+4:idx+req_toks]) |
1922 | + current_idx = idx + req_toks |
1923 | + elif net_length in range(9, 17): |
1924 | + req_toks = 7 |
1925 | + if len(tokens[idx:]) < req_toks: |
1926 | + _trunc_error(net_length, req_toks, len(tokens[idx:])) |
1927 | + return static_routes |
1928 | + net_address = ".".join(tokens[idx+1:idx+3] + ["0", "0"]) |
1929 | + gateway = ".".join(tokens[idx+3:idx+req_toks]) |
1930 | + current_idx = idx + req_toks |
1931 | + elif net_length in range(1, 9): |
1932 | + req_toks = 6 |
1933 | + if len(tokens[idx:]) < req_toks: |
1934 | + _trunc_error(net_length, req_toks, len(tokens[idx:])) |
1935 | + return static_routes |
1936 | + net_address = ".".join(tokens[idx+1:idx+2] + ["0", "0", "0"]) |
1937 | + gateway = ".".join(tokens[idx+2:idx+req_toks]) |
1938 | + current_idx = idx + req_toks |
1939 | + elif net_length == 0: |
1940 | + req_toks = 5 |
1941 | + if len(tokens[idx:]) < req_toks: |
1942 | + _trunc_error(net_length, req_toks, len(tokens[idx:])) |
1943 | + return static_routes |
1944 | + net_address = "0.0.0.0" |
1945 | + gateway = ".".join(tokens[idx+1:idx+req_toks]) |
1946 | + current_idx = idx + req_toks |
1947 | + else: |
1948 | + LOG.error('Parsed invalid net length "%s". Verify DHCP ' |
1949 | + 'rfc3442-classless-static-routes value.', net_length) |
1950 | + return static_routes |
1951 | + |
1952 | + static_routes.append(("%s/%s" % (net_address, net_length), gateway)) |
1953 | + |
1954 | + return static_routes |
1955 | + |
1956 | # vi: ts=4 expandtab |
1957 | diff --git a/cloudinit/net/network_state.py b/cloudinit/net/network_state.py |
1958 | index 3702130..c0c415d 100644 |
1959 | --- a/cloudinit/net/network_state.py |
1960 | +++ b/cloudinit/net/network_state.py |
1961 | @@ -596,6 +596,7 @@ class NetworkStateInterpreter(object): |
1962 | eno1: |
1963 | match: |
1964 | macaddress: 00:11:22:33:44:55 |
1965 | + driver: hv_netsvc |
1966 | wakeonlan: true |
1967 | dhcp4: true |
1968 | dhcp6: false |
1969 | @@ -631,15 +632,18 @@ class NetworkStateInterpreter(object): |
1970 | 'type': 'physical', |
1971 | 'name': cfg.get('set-name', eth), |
1972 | } |
1973 | - mac_address = cfg.get('match', {}).get('macaddress', None) |
1974 | + match = cfg.get('match', {}) |
1975 | + mac_address = match.get('macaddress', None) |
1976 | if not mac_address: |
1977 | LOG.debug('NetworkState Version2: missing "macaddress" info ' |
1978 | 'in config entry: %s: %s', eth, str(cfg)) |
1979 | - phy_cmd.update({'mac_address': mac_address}) |
1980 | - |
1981 | + phy_cmd['mac_address'] = mac_address |
1982 | + driver = match.get('driver', None) |
1983 | + if driver: |
1984 | + phy_cmd['params'] = {'driver': driver} |
1985 | for key in ['mtu', 'match', 'wakeonlan']: |
1986 | if key in cfg: |
1987 | - phy_cmd.update({key: cfg.get(key)}) |
1988 | + phy_cmd[key] = cfg[key] |
1989 | |
1990 | subnets = self._v2_to_v1_ipcfg(cfg) |
1991 | if len(subnets) > 0: |
1992 | @@ -673,6 +677,8 @@ class NetworkStateInterpreter(object): |
1993 | 'vlan_id': cfg.get('id'), |
1994 | 'vlan_link': cfg.get('link'), |
1995 | } |
1996 | + if 'mtu' in cfg: |
1997 | + vlan_cmd['mtu'] = cfg['mtu'] |
1998 | subnets = self._v2_to_v1_ipcfg(cfg) |
1999 | if len(subnets) > 0: |
2000 | vlan_cmd.update({'subnets': subnets}) |
2001 | @@ -722,6 +728,8 @@ class NetworkStateInterpreter(object): |
2002 | 'params': dict((v2key_to_v1[k], v) for k, v in |
2003 | item_params.get('parameters', {}).items()) |
2004 | } |
2005 | + if 'mtu' in item_cfg: |
2006 | + v1_cmd['mtu'] = item_cfg['mtu'] |
2007 | subnets = self._v2_to_v1_ipcfg(item_cfg) |
2008 | if len(subnets) > 0: |
2009 | v1_cmd.update({'subnets': subnets}) |
2010 | diff --git a/cloudinit/net/sysconfig.py b/cloudinit/net/sysconfig.py |
2011 | index a47da0a..be5dede 100644 |
2012 | --- a/cloudinit/net/sysconfig.py |
2013 | +++ b/cloudinit/net/sysconfig.py |
2014 | @@ -284,6 +284,18 @@ class Renderer(renderer.Renderer): |
2015 | ('bond_mode', "mode=%s"), |
2016 | ('bond_xmit_hash_policy', "xmit_hash_policy=%s"), |
2017 | ('bond_miimon', "miimon=%s"), |
2018 | + ('bond_min_links', "min_links=%s"), |
2019 | + ('bond_arp_interval', "arp_interval=%s"), |
2020 | + ('bond_arp_ip_target', "arp_ip_target=%s"), |
2021 | + ('bond_arp_validate', "arp_validate=%s"), |
2022 | + ('bond_ad_select', "ad_select=%s"), |
2023 | + ('bond_num_grat_arp', "num_grat_arp=%s"), |
2024 | + ('bond_downdelay', "downdelay=%s"), |
2025 | + ('bond_updelay', "updelay=%s"), |
2026 | + ('bond_lacp_rate', "lacp_rate=%s"), |
2027 | + ('bond_fail_over_mac', "fail_over_mac=%s"), |
2028 | + ('bond_primary', "primary=%s"), |
2029 | + ('bond_primary_reselect', "primary_reselect=%s"), |
2030 | ]) |
2031 | |
2032 | bridge_opts_keys = tuple([ |
2033 | diff --git a/cloudinit/net/tests/test_dhcp.py b/cloudinit/net/tests/test_dhcp.py |
2034 | index 5139024..91f503c 100644 |
2035 | --- a/cloudinit/net/tests/test_dhcp.py |
2036 | +++ b/cloudinit/net/tests/test_dhcp.py |
2037 | @@ -8,7 +8,8 @@ from textwrap import dedent |
2038 | import cloudinit.net as net |
2039 | from cloudinit.net.dhcp import ( |
2040 | InvalidDHCPLeaseFileError, maybe_perform_dhcp_discovery, |
2041 | - parse_dhcp_lease_file, dhcp_discovery, networkd_load_leases) |
2042 | + parse_dhcp_lease_file, dhcp_discovery, networkd_load_leases, |
2043 | + parse_static_routes) |
2044 | from cloudinit.util import ensure_file, write_file |
2045 | from cloudinit.tests.helpers import ( |
2046 | CiTestCase, HttprettyTestCase, mock, populate_dir, wrap_and_call) |
2047 | @@ -64,6 +65,123 @@ class TestParseDHCPLeasesFile(CiTestCase): |
2048 | self.assertItemsEqual(expected, parse_dhcp_lease_file(lease_file)) |
2049 | |
2050 | |
2051 | +class TestDHCPRFC3442(CiTestCase): |
2052 | + |
2053 | + def test_parse_lease_finds_rfc3442_classless_static_routes(self): |
2054 | + """parse_dhcp_lease_file returns rfc3442-classless-static-routes.""" |
2055 | + lease_file = self.tmp_path('leases') |
2056 | + content = dedent(""" |
2057 | + lease { |
2058 | + interface "wlp3s0"; |
2059 | + fixed-address 192.168.2.74; |
2060 | + option subnet-mask 255.255.255.0; |
2061 | + option routers 192.168.2.1; |
2062 | + option rfc3442-classless-static-routes 0,130,56,240,1; |
2063 | + renew 4 2017/07/27 18:02:30; |
2064 | + expire 5 2017/07/28 07:08:15; |
2065 | + } |
2066 | + """) |
2067 | + expected = [ |
2068 | + {'interface': 'wlp3s0', 'fixed-address': '192.168.2.74', |
2069 | + 'subnet-mask': '255.255.255.0', 'routers': '192.168.2.1', |
2070 | + 'rfc3442-classless-static-routes': '0,130,56,240,1', |
2071 | + 'renew': '4 2017/07/27 18:02:30', |
2072 | + 'expire': '5 2017/07/28 07:08:15'}] |
2073 | + write_file(lease_file, content) |
2074 | + self.assertItemsEqual(expected, parse_dhcp_lease_file(lease_file)) |
2075 | + |
2076 | + @mock.patch('cloudinit.net.dhcp.EphemeralIPv4Network') |
2077 | + @mock.patch('cloudinit.net.dhcp.maybe_perform_dhcp_discovery') |
2078 | + def test_obtain_lease_parses_static_routes(self, m_maybe, m_ipv4): |
2079 | + """EphemeralDHPCv4 parses rfc3442 routes for EphemeralIPv4Network""" |
2080 | + lease = [ |
2081 | + {'interface': 'wlp3s0', 'fixed-address': '192.168.2.74', |
2082 | + 'subnet-mask': '255.255.255.0', 'routers': '192.168.2.1', |
2083 | + 'rfc3442-classless-static-routes': '0,130,56,240,1', |
2084 | + 'renew': '4 2017/07/27 18:02:30', |
2085 | + 'expire': '5 2017/07/28 07:08:15'}] |
2086 | + m_maybe.return_value = lease |
2087 | + eph = net.dhcp.EphemeralDHCPv4() |
2088 | + eph.obtain_lease() |
2089 | + expected_kwargs = { |
2090 | + 'interface': 'wlp3s0', |
2091 | + 'ip': '192.168.2.74', |
2092 | + 'prefix_or_mask': '255.255.255.0', |
2093 | + 'broadcast': '192.168.2.255', |
2094 | + 'static_routes': [('0.0.0.0/0', '130.56.240.1')], |
2095 | + 'router': '192.168.2.1'} |
2096 | + m_ipv4.assert_called_with(**expected_kwargs) |
2097 | + |
2098 | + |
2099 | +class TestDHCPParseStaticRoutes(CiTestCase): |
2100 | + |
2101 | + with_logs = True |
2102 | + |
2103 | + def parse_static_routes_empty_string(self): |
2104 | + self.assertEqual([], parse_static_routes("")) |
2105 | + |
2106 | + def test_parse_static_routes_invalid_input_returns_empty_list(self): |
2107 | + rfc3442 = "32,169,254,169,254,130,56,248" |
2108 | + self.assertEqual([], parse_static_routes(rfc3442)) |
2109 | + |
2110 | + def test_parse_static_routes_bogus_width_returns_empty_list(self): |
2111 | + rfc3442 = "33,169,254,169,254,130,56,248" |
2112 | + self.assertEqual([], parse_static_routes(rfc3442)) |
2113 | + |
2114 | + def test_parse_static_routes_single_ip(self): |
2115 | + rfc3442 = "32,169,254,169,254,130,56,248,255" |
2116 | + self.assertEqual([('169.254.169.254/32', '130.56.248.255')], |
2117 | + parse_static_routes(rfc3442)) |
2118 | + |
2119 | + def test_parse_static_routes_single_ip_handles_trailing_semicolon(self): |
2120 | + rfc3442 = "32,169,254,169,254,130,56,248,255;" |
2121 | + self.assertEqual([('169.254.169.254/32', '130.56.248.255')], |
2122 | + parse_static_routes(rfc3442)) |
2123 | + |
2124 | + def test_parse_static_routes_default_route(self): |
2125 | + rfc3442 = "0,130,56,240,1" |
2126 | + self.assertEqual([('0.0.0.0/0', '130.56.240.1')], |
2127 | + parse_static_routes(rfc3442)) |
2128 | + |
2129 | + def test_parse_static_routes_class_c_b_a(self): |
2130 | + class_c = "24,192,168,74,192,168,0,4" |
2131 | + class_b = "16,172,16,172,16,0,4" |
2132 | + class_a = "8,10,10,0,0,4" |
2133 | + rfc3442 = ",".join([class_c, class_b, class_a]) |
2134 | + self.assertEqual(sorted([ |
2135 | + ("192.168.74.0/24", "192.168.0.4"), |
2136 | + ("172.16.0.0/16", "172.16.0.4"), |
2137 | + ("10.0.0.0/8", "10.0.0.4") |
2138 | + ]), sorted(parse_static_routes(rfc3442))) |
2139 | + |
2140 | + def test_parse_static_routes_logs_error_truncated(self): |
2141 | + bad_rfc3442 = { |
2142 | + "class_c": "24,169,254,169,10", |
2143 | + "class_b": "16,172,16,10", |
2144 | + "class_a": "8,10,10", |
2145 | + "gateway": "0,0", |
2146 | + "netlen": "33,0", |
2147 | + } |
2148 | + for rfc3442 in bad_rfc3442.values(): |
2149 | + self.assertEqual([], parse_static_routes(rfc3442)) |
2150 | + |
2151 | + logs = self.logs.getvalue() |
2152 | + self.assertEqual(len(bad_rfc3442.keys()), len(logs.splitlines())) |
2153 | + |
2154 | + def test_parse_static_routes_returns_valid_routes_until_parse_err(self): |
2155 | + class_c = "24,192,168,74,192,168,0,4" |
2156 | + class_b = "16,172,16,172,16,0,4" |
2157 | + class_a_error = "8,10,10,0,0" |
2158 | + rfc3442 = ",".join([class_c, class_b, class_a_error]) |
2159 | + self.assertEqual(sorted([ |
2160 | + ("192.168.74.0/24", "192.168.0.4"), |
2161 | + ("172.16.0.0/16", "172.16.0.4"), |
2162 | + ]), sorted(parse_static_routes(rfc3442))) |
2163 | + |
2164 | + logs = self.logs.getvalue() |
2165 | + self.assertIn(rfc3442, logs.splitlines()[0]) |
2166 | + |
2167 | + |
2168 | class TestDHCPDiscoveryClean(CiTestCase): |
2169 | with_logs = True |
2170 | |
2171 | diff --git a/cloudinit/net/tests/test_init.py b/cloudinit/net/tests/test_init.py |
2172 | index 6d2affe..d2e38f0 100644 |
2173 | --- a/cloudinit/net/tests/test_init.py |
2174 | +++ b/cloudinit/net/tests/test_init.py |
2175 | @@ -212,9 +212,9 @@ class TestGenerateFallbackConfig(CiTestCase): |
2176 | mac = 'aa:bb:cc:aa:bb:cc' |
2177 | write_file(os.path.join(self.sysdir, 'eth1', 'address'), mac) |
2178 | expected = { |
2179 | - 'config': [{'type': 'physical', 'mac_address': mac, |
2180 | - 'name': 'eth1', 'subnets': [{'type': 'dhcp'}]}], |
2181 | - 'version': 1} |
2182 | + 'ethernets': {'eth1': {'match': {'macaddress': mac}, |
2183 | + 'dhcp4': True, 'set-name': 'eth1'}}, |
2184 | + 'version': 2} |
2185 | self.assertEqual(expected, net.generate_fallback_config()) |
2186 | |
2187 | def test_generate_fallback_finds_dormant_eth_with_mac(self): |
2188 | @@ -223,9 +223,9 @@ class TestGenerateFallbackConfig(CiTestCase): |
2189 | mac = 'aa:bb:cc:aa:bb:cc' |
2190 | write_file(os.path.join(self.sysdir, 'eth0', 'address'), mac) |
2191 | expected = { |
2192 | - 'config': [{'type': 'physical', 'mac_address': mac, |
2193 | - 'name': 'eth0', 'subnets': [{'type': 'dhcp'}]}], |
2194 | - 'version': 1} |
2195 | + 'ethernets': {'eth0': {'match': {'macaddress': mac}, 'dhcp4': True, |
2196 | + 'set-name': 'eth0'}}, |
2197 | + 'version': 2} |
2198 | self.assertEqual(expected, net.generate_fallback_config()) |
2199 | |
2200 | def test_generate_fallback_finds_eth_by_operstate(self): |
2201 | @@ -233,9 +233,10 @@ class TestGenerateFallbackConfig(CiTestCase): |
2202 | mac = 'aa:bb:cc:aa:bb:cc' |
2203 | write_file(os.path.join(self.sysdir, 'eth0', 'address'), mac) |
2204 | expected = { |
2205 | - 'config': [{'type': 'physical', 'mac_address': mac, |
2206 | - 'name': 'eth0', 'subnets': [{'type': 'dhcp'}]}], |
2207 | - 'version': 1} |
2208 | + 'ethernets': { |
2209 | + 'eth0': {'dhcp4': True, 'match': {'macaddress': mac}, |
2210 | + 'set-name': 'eth0'}}, |
2211 | + 'version': 2} |
2212 | valid_operstates = ['dormant', 'down', 'lowerlayerdown', 'unknown'] |
2213 | for state in valid_operstates: |
2214 | write_file(os.path.join(self.sysdir, 'eth0', 'operstate'), state) |
2215 | @@ -549,6 +550,45 @@ class TestEphemeralIPV4Network(CiTestCase): |
2216 | self.assertEqual(expected_setup_calls, m_subp.call_args_list) |
2217 | m_subp.assert_has_calls(expected_teardown_calls) |
2218 | |
2219 | + def test_ephemeral_ipv4_network_with_rfc3442_static_routes(self, m_subp): |
2220 | + params = { |
2221 | + 'interface': 'eth0', 'ip': '192.168.2.2', |
2222 | + 'prefix_or_mask': '255.255.255.0', 'broadcast': '192.168.2.255', |
2223 | + 'static_routes': [('169.254.169.254/32', '192.168.2.1'), |
2224 | + ('0.0.0.0/0', '192.168.2.1')], |
2225 | + 'router': '192.168.2.1'} |
2226 | + expected_setup_calls = [ |
2227 | + mock.call( |
2228 | + ['ip', '-family', 'inet', 'addr', 'add', '192.168.2.2/24', |
2229 | + 'broadcast', '192.168.2.255', 'dev', 'eth0'], |
2230 | + capture=True, update_env={'LANG': 'C'}), |
2231 | + mock.call( |
2232 | + ['ip', '-family', 'inet', 'link', 'set', 'dev', 'eth0', 'up'], |
2233 | + capture=True), |
2234 | + mock.call( |
2235 | + ['ip', '-4', 'route', 'add', '169.254.169.254/32', |
2236 | + 'via', '192.168.2.1', 'dev', 'eth0'], capture=True), |
2237 | + mock.call( |
2238 | + ['ip', '-4', 'route', 'add', '0.0.0.0/0', |
2239 | + 'via', '192.168.2.1', 'dev', 'eth0'], capture=True)] |
2240 | + expected_teardown_calls = [ |
2241 | + mock.call( |
2242 | + ['ip', '-4', 'route', 'del', '0.0.0.0/0', |
2243 | + 'via', '192.168.2.1', 'dev', 'eth0'], capture=True), |
2244 | + mock.call( |
2245 | + ['ip', '-4', 'route', 'del', '169.254.169.254/32', |
2246 | + 'via', '192.168.2.1', 'dev', 'eth0'], capture=True), |
2247 | + mock.call( |
2248 | + ['ip', '-family', 'inet', 'link', 'set', 'dev', |
2249 | + 'eth0', 'down'], capture=True), |
2250 | + mock.call( |
2251 | + ['ip', '-family', 'inet', 'addr', 'del', |
2252 | + '192.168.2.2/24', 'dev', 'eth0'], capture=True) |
2253 | + ] |
2254 | + with net.EphemeralIPv4Network(**params): |
2255 | + self.assertEqual(expected_setup_calls, m_subp.call_args_list) |
2256 | + m_subp.assert_has_calls(expected_setup_calls + expected_teardown_calls) |
2257 | + |
2258 | |
2259 | class TestApplyNetworkCfgNames(CiTestCase): |
2260 | V1_CONFIG = textwrap.dedent("""\ |
2261 | @@ -669,3 +709,216 @@ class TestHasURLConnectivity(HttprettyTestCase): |
2262 | httpretty.register_uri(httpretty.GET, self.url, body={}, status=404) |
2263 | self.assertFalse( |
2264 | net.has_url_connectivity(self.url), 'Expected False on url fail') |
2265 | + |
2266 | + |
2267 | +def _mk_v1_phys(mac, name, driver, device_id): |
2268 | + v1_cfg = {'type': 'physical', 'name': name, 'mac_address': mac} |
2269 | + params = {} |
2270 | + if driver: |
2271 | + params.update({'driver': driver}) |
2272 | + if device_id: |
2273 | + params.update({'device_id': device_id}) |
2274 | + |
2275 | + if params: |
2276 | + v1_cfg.update({'params': params}) |
2277 | + |
2278 | + return v1_cfg |
2279 | + |
2280 | + |
2281 | +def _mk_v2_phys(mac, name, driver=None, device_id=None): |
2282 | + v2_cfg = {'set-name': name, 'match': {'macaddress': mac}} |
2283 | + if driver: |
2284 | + v2_cfg['match'].update({'driver': driver}) |
2285 | + if device_id: |
2286 | + v2_cfg['match'].update({'device_id': device_id}) |
2287 | + |
2288 | + return v2_cfg |
2289 | + |
2290 | + |
2291 | +class TestExtractPhysdevs(CiTestCase): |
2292 | + |
2293 | + def setUp(self): |
2294 | + super(TestExtractPhysdevs, self).setUp() |
2295 | + self.add_patch('cloudinit.net.device_driver', 'm_driver') |
2296 | + self.add_patch('cloudinit.net.device_devid', 'm_devid') |
2297 | + |
2298 | + def test_extract_physdevs_looks_up_driver_v1(self): |
2299 | + driver = 'virtio' |
2300 | + self.m_driver.return_value = driver |
2301 | + physdevs = [ |
2302 | + ['aa:bb:cc:dd:ee:ff', 'eth0', None, '0x1000'], |
2303 | + ] |
2304 | + netcfg = { |
2305 | + 'version': 1, |
2306 | + 'config': [_mk_v1_phys(*args) for args in physdevs], |
2307 | + } |
2308 | + # insert the driver value for verification |
2309 | + physdevs[0][2] = driver |
2310 | + self.assertEqual(sorted(physdevs), |
2311 | + sorted(net.extract_physdevs(netcfg))) |
2312 | + self.m_driver.assert_called_with('eth0') |
2313 | + |
2314 | + def test_extract_physdevs_looks_up_driver_v2(self): |
2315 | + driver = 'virtio' |
2316 | + self.m_driver.return_value = driver |
2317 | + physdevs = [ |
2318 | + ['aa:bb:cc:dd:ee:ff', 'eth0', None, '0x1000'], |
2319 | + ] |
2320 | + netcfg = { |
2321 | + 'version': 2, |
2322 | + 'ethernets': {args[1]: _mk_v2_phys(*args) for args in physdevs}, |
2323 | + } |
2324 | + # insert the driver value for verification |
2325 | + physdevs[0][2] = driver |
2326 | + self.assertEqual(sorted(physdevs), |
2327 | + sorted(net.extract_physdevs(netcfg))) |
2328 | + self.m_driver.assert_called_with('eth0') |
2329 | + |
2330 | + def test_extract_physdevs_looks_up_devid_v1(self): |
2331 | + devid = '0x1000' |
2332 | + self.m_devid.return_value = devid |
2333 | + physdevs = [ |
2334 | + ['aa:bb:cc:dd:ee:ff', 'eth0', 'virtio', None], |
2335 | + ] |
2336 | + netcfg = { |
2337 | + 'version': 1, |
2338 | + 'config': [_mk_v1_phys(*args) for args in physdevs], |
2339 | + } |
2340 | + # insert the driver value for verification |
2341 | + physdevs[0][3] = devid |
2342 | + self.assertEqual(sorted(physdevs), |
2343 | + sorted(net.extract_physdevs(netcfg))) |
2344 | + self.m_devid.assert_called_with('eth0') |
2345 | + |
2346 | + def test_extract_physdevs_looks_up_devid_v2(self): |
2347 | + devid = '0x1000' |
2348 | + self.m_devid.return_value = devid |
2349 | + physdevs = [ |
2350 | + ['aa:bb:cc:dd:ee:ff', 'eth0', 'virtio', None], |
2351 | + ] |
2352 | + netcfg = { |
2353 | + 'version': 2, |
2354 | + 'ethernets': {args[1]: _mk_v2_phys(*args) for args in physdevs}, |
2355 | + } |
2356 | + # insert the driver value for verification |
2357 | + physdevs[0][3] = devid |
2358 | + self.assertEqual(sorted(physdevs), |
2359 | + sorted(net.extract_physdevs(netcfg))) |
2360 | + self.m_devid.assert_called_with('eth0') |
2361 | + |
2362 | + def test_get_v1_type_physical(self): |
2363 | + physdevs = [ |
2364 | + ['aa:bb:cc:dd:ee:ff', 'eth0', 'virtio', '0x1000'], |
2365 | + ['00:11:22:33:44:55', 'ens3', 'e1000', '0x1643'], |
2366 | + ['09:87:65:43:21:10', 'ens0p1', 'mlx4_core', '0:0:1000'], |
2367 | + ] |
2368 | + netcfg = { |
2369 | + 'version': 1, |
2370 | + 'config': [_mk_v1_phys(*args) for args in physdevs], |
2371 | + } |
2372 | + self.assertEqual(sorted(physdevs), |
2373 | + sorted(net.extract_physdevs(netcfg))) |
2374 | + |
2375 | + def test_get_v2_type_physical(self): |
2376 | + physdevs = [ |
2377 | + ['aa:bb:cc:dd:ee:ff', 'eth0', 'virtio', '0x1000'], |
2378 | + ['00:11:22:33:44:55', 'ens3', 'e1000', '0x1643'], |
2379 | + ['09:87:65:43:21:10', 'ens0p1', 'mlx4_core', '0:0:1000'], |
2380 | + ] |
2381 | + netcfg = { |
2382 | + 'version': 2, |
2383 | + 'ethernets': {args[1]: _mk_v2_phys(*args) for args in physdevs}, |
2384 | + } |
2385 | + self.assertEqual(sorted(physdevs), |
2386 | + sorted(net.extract_physdevs(netcfg))) |
2387 | + |
2388 | + def test_get_v2_type_physical_skips_if_no_set_name(self): |
2389 | + netcfg = { |
2390 | + 'version': 2, |
2391 | + 'ethernets': { |
2392 | + 'ens3': { |
2393 | + 'match': {'macaddress': '00:11:22:33:44:55'}, |
2394 | + } |
2395 | + } |
2396 | + } |
2397 | + self.assertEqual([], net.extract_physdevs(netcfg)) |
2398 | + |
2399 | + def test_runtime_error_on_unknown_netcfg_version(self): |
2400 | + with self.assertRaises(RuntimeError): |
2401 | + net.extract_physdevs({'version': 3, 'awesome_config': []}) |
2402 | + |
2403 | + |
2404 | +class TestWaitForPhysdevs(CiTestCase): |
2405 | + |
2406 | + with_logs = True |
2407 | + |
2408 | + def setUp(self): |
2409 | + super(TestWaitForPhysdevs, self).setUp() |
2410 | + self.add_patch('cloudinit.net.get_interfaces_by_mac', |
2411 | + 'm_get_iface_mac') |
2412 | + self.add_patch('cloudinit.util.udevadm_settle', 'm_udev_settle') |
2413 | + |
2414 | + def test_wait_for_physdevs_skips_settle_if_all_present(self): |
2415 | + physdevs = [ |
2416 | + ['aa:bb:cc:dd:ee:ff', 'eth0', 'virtio', '0x1000'], |
2417 | + ['00:11:22:33:44:55', 'ens3', 'e1000', '0x1643'], |
2418 | + ] |
2419 | + netcfg = { |
2420 | + 'version': 2, |
2421 | + 'ethernets': {args[1]: _mk_v2_phys(*args) |
2422 | + for args in physdevs}, |
2423 | + } |
2424 | + self.m_get_iface_mac.side_effect = iter([ |
2425 | + {'aa:bb:cc:dd:ee:ff': 'eth0', |
2426 | + '00:11:22:33:44:55': 'ens3'}, |
2427 | + ]) |
2428 | + net.wait_for_physdevs(netcfg) |
2429 | + self.assertEqual(0, self.m_udev_settle.call_count) |
2430 | + |
2431 | + def test_wait_for_physdevs_calls_udev_settle_on_missing(self): |
2432 | + physdevs = [ |
2433 | + ['aa:bb:cc:dd:ee:ff', 'eth0', 'virtio', '0x1000'], |
2434 | + ['00:11:22:33:44:55', 'ens3', 'e1000', '0x1643'], |
2435 | + ] |
2436 | + netcfg = { |
2437 | + 'version': 2, |
2438 | + 'ethernets': {args[1]: _mk_v2_phys(*args) |
2439 | + for args in physdevs}, |
2440 | + } |
2441 | + self.m_get_iface_mac.side_effect = iter([ |
2442 | + {'aa:bb:cc:dd:ee:ff': 'eth0'}, # first call ens3 is missing |
2443 | + {'aa:bb:cc:dd:ee:ff': 'eth0', |
2444 | + '00:11:22:33:44:55': 'ens3'}, # second call has both |
2445 | + ]) |
2446 | + net.wait_for_physdevs(netcfg) |
2447 | + self.m_udev_settle.assert_called_with(exists=net.sys_dev_path('ens3')) |
2448 | + |
2449 | + def test_wait_for_physdevs_raise_runtime_error_if_missing_and_strict(self): |
2450 | + physdevs = [ |
2451 | + ['aa:bb:cc:dd:ee:ff', 'eth0', 'virtio', '0x1000'], |
2452 | + ['00:11:22:33:44:55', 'ens3', 'e1000', '0x1643'], |
2453 | + ] |
2454 | + netcfg = { |
2455 | + 'version': 2, |
2456 | + 'ethernets': {args[1]: _mk_v2_phys(*args) |
2457 | + for args in physdevs}, |
2458 | + } |
2459 | + self.m_get_iface_mac.return_value = {} |
2460 | + with self.assertRaises(RuntimeError): |
2461 | + net.wait_for_physdevs(netcfg) |
2462 | + |
2463 | + self.assertEqual(5 * len(physdevs), self.m_udev_settle.call_count) |
2464 | + |
2465 | + def test_wait_for_physdevs_no_raise_if_not_strict(self): |
2466 | + physdevs = [ |
2467 | + ['aa:bb:cc:dd:ee:ff', 'eth0', 'virtio', '0x1000'], |
2468 | + ['00:11:22:33:44:55', 'ens3', 'e1000', '0x1643'], |
2469 | + ] |
2470 | + netcfg = { |
2471 | + 'version': 2, |
2472 | + 'ethernets': {args[1]: _mk_v2_phys(*args) |
2473 | + for args in physdevs}, |
2474 | + } |
2475 | + self.m_get_iface_mac.return_value = {} |
2476 | + net.wait_for_physdevs(netcfg, strict=False) |
2477 | + self.assertEqual(5 * len(physdevs), self.m_udev_settle.call_count) |
2478 | diff --git a/cloudinit/settings.py b/cloudinit/settings.py |
2479 | index b1ebaad..2060d81 100644 |
2480 | --- a/cloudinit/settings.py |
2481 | +++ b/cloudinit/settings.py |
2482 | @@ -39,6 +39,7 @@ CFG_BUILTIN = { |
2483 | 'Hetzner', |
2484 | 'IBMCloud', |
2485 | 'Oracle', |
2486 | + 'Exoscale', |
2487 | # At the end to act as a 'catch' when none of the above work... |
2488 | 'None', |
2489 | ], |
2490 | diff --git a/cloudinit/sources/DataSourceAzure.py b/cloudinit/sources/DataSourceAzure.py |
2491 | index b7440c1..4984fa8 100755 |
2492 | --- a/cloudinit/sources/DataSourceAzure.py |
2493 | +++ b/cloudinit/sources/DataSourceAzure.py |
2494 | @@ -26,9 +26,14 @@ from cloudinit.url_helper import UrlError, readurl, retry_on_url_exc |
2495 | from cloudinit import util |
2496 | from cloudinit.reporting import events |
2497 | |
2498 | -from cloudinit.sources.helpers.azure import (azure_ds_reporter, |
2499 | - azure_ds_telemetry_reporter, |
2500 | - get_metadata_from_fabric) |
2501 | +from cloudinit.sources.helpers.azure import ( |
2502 | + azure_ds_reporter, |
2503 | + azure_ds_telemetry_reporter, |
2504 | + get_metadata_from_fabric, |
2505 | + get_boot_telemetry, |
2506 | + get_system_info, |
2507 | + report_diagnostic_event, |
2508 | + EphemeralDHCPv4WithReporting) |
2509 | |
2510 | LOG = logging.getLogger(__name__) |
2511 | |
2512 | @@ -354,7 +359,7 @@ class DataSourceAzure(sources.DataSource): |
2513 | bname = str(pk['fingerprint'] + ".crt") |
2514 | fp_files += [os.path.join(ddir, bname)] |
2515 | LOG.debug("ssh authentication: " |
2516 | - "using fingerprint from fabirc") |
2517 | + "using fingerprint from fabric") |
2518 | |
2519 | with events.ReportEventStack( |
2520 | name="waiting-for-ssh-public-key", |
2521 | @@ -419,12 +424,17 @@ class DataSourceAzure(sources.DataSource): |
2522 | ret = load_azure_ds_dir(cdev) |
2523 | |
2524 | except NonAzureDataSource: |
2525 | + report_diagnostic_event( |
2526 | + "Did not find Azure data source in %s" % cdev) |
2527 | continue |
2528 | except BrokenAzureDataSource as exc: |
2529 | msg = 'BrokenAzureDataSource: %s' % exc |
2530 | + report_diagnostic_event(msg) |
2531 | raise sources.InvalidMetaDataException(msg) |
2532 | except util.MountFailedError: |
2533 | - LOG.warning("%s was not mountable", cdev) |
2534 | + msg = '%s was not mountable' % cdev |
2535 | + report_diagnostic_event(msg) |
2536 | + LOG.warning(msg) |
2537 | continue |
2538 | |
2539 | perform_reprovision = reprovision or self._should_reprovision(ret) |
2540 | @@ -432,6 +442,7 @@ class DataSourceAzure(sources.DataSource): |
2541 | if util.is_FreeBSD(): |
2542 | msg = "Free BSD is not supported for PPS VMs" |
2543 | LOG.error(msg) |
2544 | + report_diagnostic_event(msg) |
2545 | raise sources.InvalidMetaDataException(msg) |
2546 | ret = self._reprovision() |
2547 | imds_md = get_metadata_from_imds( |
2548 | @@ -450,7 +461,9 @@ class DataSourceAzure(sources.DataSource): |
2549 | break |
2550 | |
2551 | if not found: |
2552 | - raise sources.InvalidMetaDataException('No Azure metadata found') |
2553 | + msg = 'No Azure metadata found' |
2554 | + report_diagnostic_event(msg) |
2555 | + raise sources.InvalidMetaDataException(msg) |
2556 | |
2557 | if found == ddir: |
2558 | LOG.debug("using files cached in %s", ddir) |
2559 | @@ -469,9 +482,14 @@ class DataSourceAzure(sources.DataSource): |
2560 | self._report_ready(lease=self._ephemeral_dhcp_ctx.lease) |
2561 | self._ephemeral_dhcp_ctx.clean_network() # Teardown ephemeral |
2562 | else: |
2563 | - with EphemeralDHCPv4() as lease: |
2564 | - self._report_ready(lease=lease) |
2565 | - |
2566 | + try: |
2567 | + with EphemeralDHCPv4WithReporting( |
2568 | + azure_ds_reporter) as lease: |
2569 | + self._report_ready(lease=lease) |
2570 | + except Exception as e: |
2571 | + report_diagnostic_event( |
2572 | + "exception while reporting ready: %s" % e) |
2573 | + raise |
2574 | return crawled_data |
2575 | |
2576 | def _is_platform_viable(self): |
2577 | @@ -493,6 +511,16 @@ class DataSourceAzure(sources.DataSource): |
2578 | if not self._is_platform_viable(): |
2579 | return False |
2580 | try: |
2581 | + get_boot_telemetry() |
2582 | + except Exception as e: |
2583 | + LOG.warning("Failed to get boot telemetry: %s", e) |
2584 | + |
2585 | + try: |
2586 | + get_system_info() |
2587 | + except Exception as e: |
2588 | + LOG.warning("Failed to get system information: %s", e) |
2589 | + |
2590 | + try: |
2591 | crawled_data = util.log_time( |
2592 | logfunc=LOG.debug, msg='Crawl of metadata service', |
2593 | func=self.crawl_metadata) |
2594 | @@ -551,27 +579,55 @@ class DataSourceAzure(sources.DataSource): |
2595 | headers = {"Metadata": "true"} |
2596 | nl_sock = None |
2597 | report_ready = bool(not os.path.isfile(REPORTED_READY_MARKER_FILE)) |
2598 | + self.imds_logging_threshold = 1 |
2599 | + self.imds_poll_counter = 1 |
2600 | + dhcp_attempts = 0 |
2601 | + vnet_switched = False |
2602 | + return_val = None |
2603 | |
2604 | def exc_cb(msg, exception): |
2605 | if isinstance(exception, UrlError) and exception.code == 404: |
2606 | + if self.imds_poll_counter == self.imds_logging_threshold: |
2607 | + # Reducing the logging frequency as we are polling IMDS |
2608 | + self.imds_logging_threshold *= 2 |
2609 | + LOG.debug("Call to IMDS with arguments %s failed " |
2610 | + "with status code %s after %s retries", |
2611 | + msg, exception.code, self.imds_poll_counter) |
2612 | + LOG.debug("Backing off logging threshold for the same " |
2613 | + "exception to %d", self.imds_logging_threshold) |
2614 | + self.imds_poll_counter += 1 |
2615 | return True |
2616 | + |
2617 | # If we get an exception while trying to call IMDS, we |
2618 | # call DHCP and setup the ephemeral network to acquire the new IP. |
2619 | + LOG.debug("Call to IMDS with arguments %s failed with " |
2620 | + "status code %s", msg, exception.code) |
2621 | + report_diagnostic_event("polling IMDS failed with exception %s" |
2622 | + % exception.code) |
2623 | return False |
2624 | |
2625 | LOG.debug("Wait for vnetswitch to happen") |
2626 | while True: |
2627 | try: |
2628 | - # Save our EphemeralDHCPv4 context so we avoid repeated dhcp |
2629 | - self._ephemeral_dhcp_ctx = EphemeralDHCPv4() |
2630 | - lease = self._ephemeral_dhcp_ctx.obtain_lease() |
2631 | + # Save our EphemeralDHCPv4 context to avoid repeated dhcp |
2632 | + with events.ReportEventStack( |
2633 | + name="obtain-dhcp-lease", |
2634 | + description="obtain dhcp lease", |
2635 | + parent=azure_ds_reporter): |
2636 | + self._ephemeral_dhcp_ctx = EphemeralDHCPv4() |
2637 | + lease = self._ephemeral_dhcp_ctx.obtain_lease() |
2638 | + |
2639 | + if vnet_switched: |
2640 | + dhcp_attempts += 1 |
2641 | if report_ready: |
2642 | try: |
2643 | nl_sock = netlink.create_bound_netlink_socket() |
2644 | except netlink.NetlinkCreateSocketError as e: |
2645 | + report_diagnostic_event(e) |
2646 | LOG.warning(e) |
2647 | self._ephemeral_dhcp_ctx.clean_network() |
2648 | - return |
2649 | + break |
2650 | + |
2651 | path = REPORTED_READY_MARKER_FILE |
2652 | LOG.info( |
2653 | "Creating a marker file to report ready: %s", path) |
2654 | @@ -579,17 +635,33 @@ class DataSourceAzure(sources.DataSource): |
2655 | pid=os.getpid(), time=time())) |
2656 | self._report_ready(lease=lease) |
2657 | report_ready = False |
2658 | - try: |
2659 | - netlink.wait_for_media_disconnect_connect( |
2660 | - nl_sock, lease['interface']) |
2661 | - except AssertionError as error: |
2662 | - LOG.error(error) |
2663 | - return |
2664 | + |
2665 | + with events.ReportEventStack( |
2666 | + name="wait-for-media-disconnect-connect", |
2667 | + description="wait for vnet switch", |
2668 | + parent=azure_ds_reporter): |
2669 | + try: |
2670 | + netlink.wait_for_media_disconnect_connect( |
2671 | + nl_sock, lease['interface']) |
2672 | + except AssertionError as error: |
2673 | + report_diagnostic_event(error) |
2674 | + LOG.error(error) |
2675 | + break |
2676 | + |
2677 | + vnet_switched = True |
2678 | self._ephemeral_dhcp_ctx.clean_network() |
2679 | else: |
2680 | - return readurl(url, timeout=IMDS_TIMEOUT_IN_SECONDS, |
2681 | - headers=headers, exception_cb=exc_cb, |
2682 | - infinite=True, log_req_resp=False).contents |
2683 | + with events.ReportEventStack( |
2684 | + name="get-reprovision-data-from-imds", |
2685 | + description="get reprovision data from imds", |
2686 | + parent=azure_ds_reporter): |
2687 | + return_val = readurl(url, |
2688 | + timeout=IMDS_TIMEOUT_IN_SECONDS, |
2689 | + headers=headers, |
2690 | + exception_cb=exc_cb, |
2691 | + infinite=True, |
2692 | + log_req_resp=False).contents |
2693 | + break |
2694 | except UrlError: |
2695 | # Teardown our EphemeralDHCPv4 context on failure as we retry |
2696 | self._ephemeral_dhcp_ctx.clean_network() |
2697 | @@ -598,6 +670,14 @@ class DataSourceAzure(sources.DataSource): |
2698 | if nl_sock: |
2699 | nl_sock.close() |
2700 | |
2701 | + if vnet_switched: |
2702 | + report_diagnostic_event("attempted dhcp %d times after reuse" % |
2703 | + dhcp_attempts) |
2704 | + report_diagnostic_event("polled imds %d times after reuse" % |
2705 | + self.imds_poll_counter) |
2706 | + |
2707 | + return return_val |
2708 | + |
2709 | @azure_ds_telemetry_reporter |
2710 | def _report_ready(self, lease): |
2711 | """Tells the fabric provisioning has completed """ |
2712 | @@ -666,9 +746,12 @@ class DataSourceAzure(sources.DataSource): |
2713 | self.ds_cfg['agent_command']) |
2714 | try: |
2715 | fabric_data = metadata_func() |
2716 | - except Exception: |
2717 | + except Exception as e: |
2718 | + report_diagnostic_event( |
2719 | + "Error communicating with Azure fabric; You may experience " |
2720 | + "connectivity issues: %s" % e) |
2721 | LOG.warning( |
2722 | - "Error communicating with Azure fabric; You may experience." |
2723 | + "Error communicating with Azure fabric; You may experience " |
2724 | "connectivity issues.", exc_info=True) |
2725 | return False |
2726 | |
2727 | @@ -684,6 +767,11 @@ class DataSourceAzure(sources.DataSource): |
2728 | return |
2729 | |
2730 | @property |
2731 | + def availability_zone(self): |
2732 | + return self.metadata.get( |
2733 | + 'imds', {}).get('compute', {}).get('platformFaultDomain') |
2734 | + |
2735 | + @property |
2736 | def network_config(self): |
2737 | """Generate a network config like net.generate_fallback_network() with |
2738 | the following exceptions. |
2739 | @@ -701,6 +789,10 @@ class DataSourceAzure(sources.DataSource): |
2740 | self._network_config = parse_network_config(nc_src) |
2741 | return self._network_config |
2742 | |
2743 | + @property |
2744 | + def region(self): |
2745 | + return self.metadata.get('imds', {}).get('compute', {}).get('location') |
2746 | + |
2747 | |
2748 | def _partitions_on_device(devpath, maxnum=16): |
2749 | # return a list of tuples (ptnum, path) for each part on devpath |
2750 | @@ -1018,7 +1110,9 @@ def read_azure_ovf(contents): |
2751 | try: |
2752 | dom = minidom.parseString(contents) |
2753 | except Exception as e: |
2754 | - raise BrokenAzureDataSource("Invalid ovf-env.xml: %s" % e) |
2755 | + error_str = "Invalid ovf-env.xml: %s" % e |
2756 | + report_diagnostic_event(error_str) |
2757 | + raise BrokenAzureDataSource(error_str) |
2758 | |
2759 | results = find_child(dom.documentElement, |
2760 | lambda n: n.localName == "ProvisioningSection") |
2761 | @@ -1232,7 +1326,7 @@ def parse_network_config(imds_metadata): |
2762 | privateIpv4 = addr4['privateIpAddress'] |
2763 | if privateIpv4: |
2764 | if dev_config.get('dhcp4', False): |
2765 | - # Append static address config for nic > 1 |
2766 | + # Append static address config for ip > 1 |
2767 | netPrefix = intf['ipv4']['subnet'][0].get( |
2768 | 'prefix', '24') |
2769 | if not dev_config.get('addresses'): |
2770 | @@ -1242,6 +1336,11 @@ def parse_network_config(imds_metadata): |
2771 | ip=privateIpv4, prefix=netPrefix)) |
2772 | else: |
2773 | dev_config['dhcp4'] = True |
2774 | + # non-primary interfaces should have a higher |
2775 | + # route-metric (cost) so default routes prefer |
2776 | + # primary nic due to lower route-metric value |
2777 | + dev_config['dhcp4-overrides'] = { |
2778 | + 'route-metric': (idx + 1) * 100} |
2779 | for addr6 in intf['ipv6']['ipAddress']: |
2780 | privateIpv6 = addr6['privateIpAddress'] |
2781 | if privateIpv6: |
2782 | @@ -1285,8 +1384,13 @@ def get_metadata_from_imds(fallback_nic, retries): |
2783 | if net.is_up(fallback_nic): |
2784 | return util.log_time(**kwargs) |
2785 | else: |
2786 | - with EphemeralDHCPv4(fallback_nic): |
2787 | - return util.log_time(**kwargs) |
2788 | + try: |
2789 | + with EphemeralDHCPv4WithReporting( |
2790 | + azure_ds_reporter, fallback_nic): |
2791 | + return util.log_time(**kwargs) |
2792 | + except Exception as e: |
2793 | + report_diagnostic_event("exception while getting metadata: %s" % e) |
2794 | + raise |
2795 | |
2796 | |
2797 | @azure_ds_telemetry_reporter |
2798 | @@ -1299,11 +1403,14 @@ def _get_metadata_from_imds(retries): |
2799 | url, timeout=IMDS_TIMEOUT_IN_SECONDS, headers=headers, |
2800 | retries=retries, exception_cb=retry_on_url_exc) |
2801 | except Exception as e: |
2802 | - LOG.debug('Ignoring IMDS instance metadata: %s', e) |
2803 | + msg = 'Ignoring IMDS instance metadata: %s' % e |
2804 | + report_diagnostic_event(msg) |
2805 | + LOG.debug(msg) |
2806 | return {} |
2807 | try: |
2808 | return util.load_json(str(response)) |
2809 | - except json.decoder.JSONDecodeError: |
2810 | + except json.decoder.JSONDecodeError as e: |
2811 | + report_diagnostic_event('non-json imds response' % e) |
2812 | LOG.warning( |
2813 | 'Ignoring non-json IMDS instance metadata: %s', str(response)) |
2814 | return {} |
2815 | @@ -1356,8 +1463,10 @@ def _is_platform_viable(seed_dir): |
2816 | asset_tag = util.read_dmi_data('chassis-asset-tag') |
2817 | if asset_tag == AZURE_CHASSIS_ASSET_TAG: |
2818 | return True |
2819 | - LOG.debug("Non-Azure DMI asset tag '%s' discovered.", asset_tag) |
2820 | - evt.description = "Non-Azure DMI asset tag '%s' discovered.", asset_tag |
2821 | + msg = "Non-Azure DMI asset tag '%s' discovered." % asset_tag |
2822 | + LOG.debug(msg) |
2823 | + evt.description = msg |
2824 | + report_diagnostic_event(msg) |
2825 | if os.path.exists(os.path.join(seed_dir, 'ovf-env.xml')): |
2826 | return True |
2827 | return False |
2828 | diff --git a/cloudinit/sources/DataSourceCloudSigma.py b/cloudinit/sources/DataSourceCloudSigma.py |
2829 | index 2955d3f..df88f67 100644 |
2830 | --- a/cloudinit/sources/DataSourceCloudSigma.py |
2831 | +++ b/cloudinit/sources/DataSourceCloudSigma.py |
2832 | @@ -42,12 +42,8 @@ class DataSourceCloudSigma(sources.DataSource): |
2833 | if not sys_product_name: |
2834 | LOG.debug("system-product-name not available in dmi data") |
2835 | return False |
2836 | - else: |
2837 | - LOG.debug("detected hypervisor as %s", sys_product_name) |
2838 | - return 'cloudsigma' in sys_product_name.lower() |
2839 | - |
2840 | - LOG.warning("failed to query dmi data for system product name") |
2841 | - return False |
2842 | + LOG.debug("detected hypervisor as %s", sys_product_name) |
2843 | + return 'cloudsigma' in sys_product_name.lower() |
2844 | |
2845 | def _get_data(self): |
2846 | """ |
2847 | diff --git a/cloudinit/sources/DataSourceExoscale.py b/cloudinit/sources/DataSourceExoscale.py |
2848 | new file mode 100644 |
2849 | index 0000000..52e7f6f |
2850 | --- /dev/null |
2851 | +++ b/cloudinit/sources/DataSourceExoscale.py |
2852 | @@ -0,0 +1,258 @@ |
2853 | +# Author: Mathieu Corbin <mathieu.corbin@exoscale.com> |
2854 | +# Author: Christopher Glass <christopher.glass@exoscale.com> |
2855 | +# |
2856 | +# This file is part of cloud-init. See LICENSE file for license information. |
2857 | + |
2858 | +from cloudinit import ec2_utils as ec2 |
2859 | +from cloudinit import log as logging |
2860 | +from cloudinit import sources |
2861 | +from cloudinit import url_helper |
2862 | +from cloudinit import util |
2863 | + |
2864 | +LOG = logging.getLogger(__name__) |
2865 | + |
2866 | +METADATA_URL = "http://169.254.169.254" |
2867 | +API_VERSION = "1.0" |
2868 | +PASSWORD_SERVER_PORT = 8080 |
2869 | + |
2870 | +URL_TIMEOUT = 10 |
2871 | +URL_RETRIES = 6 |
2872 | + |
2873 | +EXOSCALE_DMI_NAME = "Exoscale" |
2874 | + |
2875 | +BUILTIN_DS_CONFIG = { |
2876 | + # We run the set password config module on every boot in order to enable |
2877 | + # resetting the instance's password via the exoscale console (and a |
2878 | + # subsequent instance reboot). |
2879 | + 'cloud_config_modules': [["set-passwords", "always"]] |
2880 | +} |
2881 | + |
2882 | + |
2883 | +class DataSourceExoscale(sources.DataSource): |
2884 | + |
2885 | + dsname = 'Exoscale' |
2886 | + |
2887 | + def __init__(self, sys_cfg, distro, paths): |
2888 | + super(DataSourceExoscale, self).__init__(sys_cfg, distro, paths) |
2889 | + LOG.debug("Initializing the Exoscale datasource") |
2890 | + |
2891 | + self.metadata_url = self.ds_cfg.get('metadata_url', METADATA_URL) |
2892 | + self.api_version = self.ds_cfg.get('api_version', API_VERSION) |
2893 | + self.password_server_port = int( |
2894 | + self.ds_cfg.get('password_server_port', PASSWORD_SERVER_PORT)) |
2895 | + self.url_timeout = self.ds_cfg.get('timeout', URL_TIMEOUT) |
2896 | + self.url_retries = self.ds_cfg.get('retries', URL_RETRIES) |
2897 | + |
2898 | + self.extra_config = BUILTIN_DS_CONFIG |
2899 | + |
2900 | + def wait_for_metadata_service(self): |
2901 | + """Wait for the metadata service to be reachable.""" |
2902 | + |
2903 | + metadata_url = "{}/{}/meta-data/instance-id".format( |
2904 | + self.metadata_url, self.api_version) |
2905 | + |
2906 | + url = url_helper.wait_for_url( |
2907 | + urls=[metadata_url], |
2908 | + max_wait=self.url_max_wait, |
2909 | + timeout=self.url_timeout, |
2910 | + status_cb=LOG.critical) |
2911 | + |
2912 | + return bool(url) |
2913 | + |
2914 | + def crawl_metadata(self): |
2915 | + """ |
2916 | + Crawl the metadata service when available. |
2917 | + |
2918 | + @returns: Dictionary of crawled metadata content. |
2919 | + """ |
2920 | + metadata_ready = util.log_time( |
2921 | + logfunc=LOG.info, |
2922 | + msg='waiting for the metadata service', |
2923 | + func=self.wait_for_metadata_service) |
2924 | + |
2925 | + if not metadata_ready: |
2926 | + return {} |
2927 | + |
2928 | + return read_metadata(self.metadata_url, self.api_version, |
2929 | + self.password_server_port, self.url_timeout, |
2930 | + self.url_retries) |
2931 | + |
2932 | + def _get_data(self): |
2933 | + """Fetch the user data, the metadata and the VM password |
2934 | + from the metadata service. |
2935 | + |
2936 | + Please refer to the datasource documentation for details on how the |
2937 | + metadata server and password server are crawled. |
2938 | + """ |
2939 | + if not self._is_platform_viable(): |
2940 | + return False |
2941 | + |
2942 | + data = util.log_time( |
2943 | + logfunc=LOG.debug, |
2944 | + msg='Crawl of metadata service', |
2945 | + func=self.crawl_metadata) |
2946 | + |
2947 | + if not data: |
2948 | + return False |
2949 | + |
2950 | + self.userdata_raw = data['user-data'] |
2951 | + self.metadata = data['meta-data'] |
2952 | + password = data.get('password') |
2953 | + |
2954 | + password_config = {} |
2955 | + if password: |
2956 | + # Since we have a password, let's make sure we are allowed to use |
2957 | + # it by allowing ssh_pwauth. |
2958 | + # The password module's default behavior is to leave the |
2959 | + # configuration as-is in this regard, so that means it will either |
2960 | + # leave the password always disabled if no password is ever set, or |
2961 | + # leave the password login enabled if we set it once. |
2962 | + password_config = { |
2963 | + 'ssh_pwauth': True, |
2964 | + 'password': password, |
2965 | + 'chpasswd': { |
2966 | + 'expire': False, |
2967 | + }, |
2968 | + } |
2969 | + |
2970 | + # builtin extra_config overrides password_config |
2971 | + self.extra_config = util.mergemanydict( |
2972 | + [self.extra_config, password_config]) |
2973 | + |
2974 | + return True |
2975 | + |
2976 | + def get_config_obj(self): |
2977 | + return self.extra_config |
2978 | + |
2979 | + def _is_platform_viable(self): |
2980 | + return util.read_dmi_data('system-product-name').startswith( |
2981 | + EXOSCALE_DMI_NAME) |
2982 | + |
2983 | + |
2984 | +# Used to match classes to dependencies |
2985 | +datasources = [ |
2986 | + (DataSourceExoscale, (sources.DEP_FILESYSTEM, sources.DEP_NETWORK)), |
2987 | +] |
2988 | + |
2989 | + |
2990 | +# Return a list of data sources that match this set of dependencies |
2991 | +def get_datasource_list(depends): |
2992 | + return sources.list_from_depends(depends, datasources) |
2993 | + |
2994 | + |
2995 | +def get_password(metadata_url=METADATA_URL, |
2996 | + api_version=API_VERSION, |
2997 | + password_server_port=PASSWORD_SERVER_PORT, |
2998 | + url_timeout=URL_TIMEOUT, |
2999 | + url_retries=URL_RETRIES): |
3000 | + """Obtain the VM's password if set. |
3001 | + |
3002 | + Once fetched the password is marked saved. Future calls to this method may |
3003 | + return empty string or 'saved_password'.""" |
3004 | + password_url = "{}:{}/{}/".format(metadata_url, password_server_port, |
3005 | + api_version) |
3006 | + response = url_helper.read_file_or_url( |
3007 | + password_url, |
3008 | + ssl_details=None, |
3009 | + headers={"DomU_Request": "send_my_password"}, |
3010 | + timeout=url_timeout, |
3011 | + retries=url_retries) |
3012 | + password = response.contents.decode('utf-8') |
3013 | + # the password is empty or already saved |
3014 | + # Note: the original metadata server would answer an additional |
3015 | + # 'bad_request' status, but the Exoscale implementation does not. |
3016 | + if password in ['', 'saved_password']: |
3017 | + return None |
3018 | + # save the password |
3019 | + url_helper.read_file_or_url( |
3020 | + password_url, |
3021 | + ssl_details=None, |
3022 | + headers={"DomU_Request": "saved_password"}, |
3023 | + timeout=url_timeout, |
3024 | + retries=url_retries) |
3025 | + return password |
3026 | + |
3027 | + |
3028 | +def read_metadata(metadata_url=METADATA_URL, |
3029 | + api_version=API_VERSION, |
3030 | + password_server_port=PASSWORD_SERVER_PORT, |
3031 | + url_timeout=URL_TIMEOUT, |
3032 | + url_retries=URL_RETRIES): |
3033 | + """Query the metadata server and return the retrieved data.""" |
3034 | + crawled_metadata = {} |
3035 | + crawled_metadata['_metadata_api_version'] = api_version |
3036 | + try: |
3037 | + crawled_metadata['user-data'] = ec2.get_instance_userdata( |
3038 | + api_version, |
3039 | + metadata_url, |
3040 | + timeout=url_timeout, |
3041 | + retries=url_retries) |
3042 | + crawled_metadata['meta-data'] = ec2.get_instance_metadata( |
3043 | + api_version, |
3044 | + metadata_url, |
3045 | + timeout=url_timeout, |
3046 | + retries=url_retries) |
3047 | + except Exception as e: |
3048 | + util.logexc(LOG, "failed reading from metadata url %s (%s)", |
3049 | + metadata_url, e) |
3050 | + return {} |
3051 | + |
3052 | + try: |
3053 | + crawled_metadata['password'] = get_password( |
3054 | + api_version=api_version, |
3055 | + metadata_url=metadata_url, |
3056 | + password_server_port=password_server_port, |
3057 | + url_retries=url_retries, |
3058 | + url_timeout=url_timeout) |
3059 | + except Exception as e: |
3060 | + util.logexc(LOG, "failed to read from password server url %s:%s (%s)", |
3061 | + metadata_url, password_server_port, e) |
3062 | + |
3063 | + return crawled_metadata |
3064 | + |
3065 | + |
3066 | +if __name__ == "__main__": |
3067 | + import argparse |
3068 | + |
3069 | + parser = argparse.ArgumentParser(description='Query Exoscale Metadata') |
3070 | + parser.add_argument( |
3071 | + "--endpoint", |
3072 | + metavar="URL", |
3073 | + help="The url of the metadata service.", |
3074 | + default=METADATA_URL) |
3075 | + parser.add_argument( |
3076 | + "--version", |
3077 | + metavar="VERSION", |
3078 | + help="The version of the metadata endpoint to query.", |
3079 | + default=API_VERSION) |
3080 | + parser.add_argument( |
3081 | + "--retries", |
3082 | + metavar="NUM", |
3083 | + type=int, |
3084 | + help="The number of retries querying the endpoint.", |
3085 | + default=URL_RETRIES) |
3086 | + parser.add_argument( |
3087 | + "--timeout", |
3088 | + metavar="NUM", |
3089 | + type=int, |
3090 | + help="The time in seconds to wait before timing out.", |
3091 | + default=URL_TIMEOUT) |
3092 | + parser.add_argument( |
3093 | + "--password-port", |
3094 | + metavar="PORT", |
3095 | + type=int, |
3096 | + help="The port on which the password endpoint listens", |
3097 | + default=PASSWORD_SERVER_PORT) |
3098 | + |
3099 | + args = parser.parse_args() |
3100 | + |
3101 | + data = read_metadata( |
3102 | + metadata_url=args.endpoint, |
3103 | + api_version=args.version, |
3104 | + password_server_port=args.password_port, |
3105 | + url_timeout=args.timeout, |
3106 | + url_retries=args.retries) |
3107 | + |
3108 | + print(util.json_dumps(data)) |
3109 | + |
3110 | +# vi: ts=4 expandtab |
3111 | diff --git a/cloudinit/sources/DataSourceGCE.py b/cloudinit/sources/DataSourceGCE.py |
3112 | index d816262..6cbfbba 100644 |
3113 | --- a/cloudinit/sources/DataSourceGCE.py |
3114 | +++ b/cloudinit/sources/DataSourceGCE.py |
3115 | @@ -18,10 +18,13 @@ LOG = logging.getLogger(__name__) |
3116 | MD_V1_URL = 'http://metadata.google.internal/computeMetadata/v1/' |
3117 | BUILTIN_DS_CONFIG = {'metadata_url': MD_V1_URL} |
3118 | REQUIRED_FIELDS = ('instance-id', 'availability-zone', 'local-hostname') |
3119 | +GUEST_ATTRIBUTES_URL = ('http://metadata.google.internal/computeMetadata/' |
3120 | + 'v1/instance/guest-attributes') |
3121 | +HOSTKEY_NAMESPACE = 'hostkeys' |
3122 | +HEADERS = {'Metadata-Flavor': 'Google'} |
3123 | |
3124 | |
3125 | class GoogleMetadataFetcher(object): |
3126 | - headers = {'Metadata-Flavor': 'Google'} |
3127 | |
3128 | def __init__(self, metadata_address): |
3129 | self.metadata_address = metadata_address |
3130 | @@ -32,7 +35,7 @@ class GoogleMetadataFetcher(object): |
3131 | url = self.metadata_address + path |
3132 | if is_recursive: |
3133 | url += '/?recursive=True' |
3134 | - resp = url_helper.readurl(url=url, headers=self.headers) |
3135 | + resp = url_helper.readurl(url=url, headers=HEADERS) |
3136 | except url_helper.UrlError as exc: |
3137 | msg = "url %s raised exception %s" |
3138 | LOG.debug(msg, path, exc) |
3139 | @@ -90,6 +93,10 @@ class DataSourceGCE(sources.DataSource): |
3140 | public_keys_data = self.metadata['public-keys-data'] |
3141 | return _parse_public_keys(public_keys_data, self.default_user) |
3142 | |
3143 | + def publish_host_keys(self, hostkeys): |
3144 | + for key in hostkeys: |
3145 | + _write_host_key_to_guest_attributes(*key) |
3146 | + |
3147 | def get_hostname(self, fqdn=False, resolve_ip=False, metadata_only=False): |
3148 | # GCE has long FDQN's and has asked for short hostnames. |
3149 | return self.metadata['local-hostname'].split('.')[0] |
3150 | @@ -103,6 +110,17 @@ class DataSourceGCE(sources.DataSource): |
3151 | return self.availability_zone.rsplit('-', 1)[0] |
3152 | |
3153 | |
3154 | +def _write_host_key_to_guest_attributes(key_type, key_value): |
3155 | + url = '%s/%s/%s' % (GUEST_ATTRIBUTES_URL, HOSTKEY_NAMESPACE, key_type) |
3156 | + key_value = key_value.encode('utf-8') |
3157 | + resp = url_helper.readurl(url=url, data=key_value, headers=HEADERS, |
3158 | + request_method='PUT', check_status=False) |
3159 | + if resp.ok(): |
3160 | + LOG.debug('Wrote %s host key to guest attributes.', key_type) |
3161 | + else: |
3162 | + LOG.debug('Unable to write %s host key to guest attributes.', key_type) |
3163 | + |
3164 | + |
3165 | def _has_expired(public_key): |
3166 | # Check whether an SSH key is expired. Public key input is a single SSH |
3167 | # public key in the GCE specific key format documented here: |
3168 | diff --git a/cloudinit/sources/DataSourceHetzner.py b/cloudinit/sources/DataSourceHetzner.py |
3169 | index 5c75b65..5029833 100644 |
3170 | --- a/cloudinit/sources/DataSourceHetzner.py |
3171 | +++ b/cloudinit/sources/DataSourceHetzner.py |
3172 | @@ -28,6 +28,9 @@ MD_WAIT_RETRY = 2 |
3173 | |
3174 | |
3175 | class DataSourceHetzner(sources.DataSource): |
3176 | + |
3177 | + dsname = 'Hetzner' |
3178 | + |
3179 | def __init__(self, sys_cfg, distro, paths): |
3180 | sources.DataSource.__init__(self, sys_cfg, distro, paths) |
3181 | self.distro = distro |
3182 | diff --git a/cloudinit/sources/DataSourceOVF.py b/cloudinit/sources/DataSourceOVF.py |
3183 | index 70e7a5c..dd941d2 100644 |
3184 | --- a/cloudinit/sources/DataSourceOVF.py |
3185 | +++ b/cloudinit/sources/DataSourceOVF.py |
3186 | @@ -148,6 +148,9 @@ class DataSourceOVF(sources.DataSource): |
3187 | product_marker, os.path.join(self.paths.cloud_dir, 'data')) |
3188 | special_customization = product_marker and not hasmarkerfile |
3189 | customscript = self._vmware_cust_conf.custom_script_name |
3190 | + ccScriptsDir = os.path.join( |
3191 | + self.paths.get_cpath("scripts"), |
3192 | + "per-instance") |
3193 | except Exception as e: |
3194 | _raise_error_status( |
3195 | "Error parsing the customization Config File", |
3196 | @@ -201,7 +204,9 @@ class DataSourceOVF(sources.DataSource): |
3197 | |
3198 | if customscript: |
3199 | try: |
3200 | - postcust = PostCustomScript(customscript, imcdirpath) |
3201 | + postcust = PostCustomScript(customscript, |
3202 | + imcdirpath, |
3203 | + ccScriptsDir) |
3204 | postcust.execute() |
3205 | except Exception as e: |
3206 | _raise_error_status( |
3207 | diff --git a/cloudinit/sources/DataSourceOracle.py b/cloudinit/sources/DataSourceOracle.py |
3208 | index 70b9c58..6e73f56 100644 |
3209 | --- a/cloudinit/sources/DataSourceOracle.py |
3210 | +++ b/cloudinit/sources/DataSourceOracle.py |
3211 | @@ -16,7 +16,7 @@ Notes: |
3212 | """ |
3213 | |
3214 | from cloudinit.url_helper import combine_url, readurl, UrlError |
3215 | -from cloudinit.net import dhcp |
3216 | +from cloudinit.net import dhcp, get_interfaces_by_mac |
3217 | from cloudinit import net |
3218 | from cloudinit import sources |
3219 | from cloudinit import util |
3220 | @@ -28,8 +28,80 @@ import re |
3221 | |
3222 | LOG = logging.getLogger(__name__) |
3223 | |
3224 | +BUILTIN_DS_CONFIG = { |
3225 | + # Don't use IMDS to configure secondary NICs by default |
3226 | + 'configure_secondary_nics': False, |
3227 | +} |
3228 | CHASSIS_ASSET_TAG = "OracleCloud.com" |
3229 | METADATA_ENDPOINT = "http://169.254.169.254/openstack/" |
3230 | +VNIC_METADATA_URL = 'http://169.254.169.254/opc/v1/vnics/' |
3231 | +# https://docs.cloud.oracle.com/iaas/Content/Network/Troubleshoot/connectionhang.htm#Overview, |
3232 | +# indicates that an MTU of 9000 is used within OCI |
3233 | +MTU = 9000 |
3234 | + |
3235 | + |
3236 | +def _add_network_config_from_opc_imds(network_config): |
3237 | + """ |
3238 | + Fetch data from Oracle's IMDS, generate secondary NIC config, merge it. |
3239 | + |
3240 | + The primary NIC configuration should not be modified based on the IMDS |
3241 | + values, as it should continue to be configured for DHCP. As such, this |
3242 | + takes an existing network_config dict which is expected to have the primary |
3243 | + NIC configuration already present. It will mutate the given dict to |
3244 | + include the secondary VNICs. |
3245 | + |
3246 | + :param network_config: |
3247 | + A v1 network config dict with the primary NIC already configured. This |
3248 | + dict will be mutated. |
3249 | + |
3250 | + :raises: |
3251 | + Exceptions are not handled within this function. Likely exceptions are |
3252 | + those raised by url_helper.readurl (if communicating with the IMDS |
3253 | + fails), ValueError/JSONDecodeError (if the IMDS returns invalid JSON), |
3254 | + and KeyError/IndexError (if the IMDS returns valid JSON with unexpected |
3255 | + contents). |
3256 | + """ |
3257 | + resp = readurl(VNIC_METADATA_URL) |
3258 | + vnics = json.loads(str(resp)) |
3259 | + |
3260 | + if 'nicIndex' in vnics[0]: |
3261 | + # TODO: Once configure_secondary_nics defaults to True, lower the level |
3262 | + # of this log message. (Currently, if we're running this code at all, |
3263 | + # someone has explicitly opted-in to secondary VNIC configuration, so |
3264 | + # we should warn them that it didn't happen. Once it's default, this |
3265 | + # would be emitted on every Bare Metal Machine launch, which means INFO |
3266 | + # or DEBUG would be more appropriate.) |
3267 | + LOG.warning( |
3268 | + 'VNIC metadata indicates this is a bare metal machine; skipping' |
3269 | + ' secondary VNIC configuration.' |
3270 | + ) |
3271 | + return |
3272 | + |
3273 | + interfaces_by_mac = get_interfaces_by_mac() |
3274 | + |
3275 | + for vnic_dict in vnics[1:]: |
3276 | + # We skip the first entry in the response because the primary interface |
3277 | + # is already configured by iSCSI boot; applying configuration from the |
3278 | + # IMDS is not required. |
3279 | + mac_address = vnic_dict['macAddr'].lower() |
3280 | + if mac_address not in interfaces_by_mac: |
3281 | + LOG.debug('Interface with MAC %s not found; skipping', mac_address) |
3282 | + continue |
3283 | + name = interfaces_by_mac[mac_address] |
3284 | + subnet = { |
3285 | + 'type': 'static', |
3286 | + 'address': vnic_dict['privateIp'], |
3287 | + 'netmask': vnic_dict['subnetCidrBlock'].split('/')[1], |
3288 | + 'gateway': vnic_dict['virtualRouterIp'], |
3289 | + 'control': 'manual', |
3290 | + } |
3291 | + network_config['config'].append({ |
3292 | + 'name': name, |
3293 | + 'type': 'physical', |
3294 | + 'mac_address': mac_address, |
3295 | + 'mtu': MTU, |
3296 | + 'subnets': [subnet], |
3297 | + }) |
3298 | |
3299 | |
3300 | class DataSourceOracle(sources.DataSource): |
3301 | @@ -37,8 +109,22 @@ class DataSourceOracle(sources.DataSource): |
3302 | dsname = 'Oracle' |
3303 | system_uuid = None |
3304 | vendordata_pure = None |
3305 | + network_config_sources = ( |
3306 | + sources.NetworkConfigSource.cmdline, |
3307 | + sources.NetworkConfigSource.ds, |
3308 | + sources.NetworkConfigSource.initramfs, |
3309 | + sources.NetworkConfigSource.system_cfg, |
3310 | + ) |
3311 | + |
3312 | _network_config = sources.UNSET |
3313 | |
3314 | + def __init__(self, sys_cfg, *args, **kwargs): |
3315 | + super(DataSourceOracle, self).__init__(sys_cfg, *args, **kwargs) |
3316 | + |
3317 | + self.ds_cfg = util.mergemanydict([ |
3318 | + util.get_cfg_by_path(sys_cfg, ['datasource', self.dsname], {}), |
3319 | + BUILTIN_DS_CONFIG]) |
3320 | + |
3321 | def _is_platform_viable(self): |
3322 | """Check platform environment to report if this datasource may run.""" |
3323 | return _is_platform_viable() |
3324 | @@ -48,7 +134,7 @@ class DataSourceOracle(sources.DataSource): |
3325 | return False |
3326 | |
3327 | # network may be configured if iscsi root. If that is the case |
3328 | - # then read_kernel_cmdline_config will return non-None. |
3329 | + # then read_initramfs_config will return non-None. |
3330 | if _is_iscsi_root(): |
3331 | data = self.crawl_metadata() |
3332 | else: |
3333 | @@ -118,11 +204,17 @@ class DataSourceOracle(sources.DataSource): |
3334 | We nonetheless return cmdline provided config if present |
3335 | and fallback to generate fallback.""" |
3336 | if self._network_config == sources.UNSET: |
3337 | - cmdline_cfg = cmdline.read_kernel_cmdline_config() |
3338 | - if cmdline_cfg: |
3339 | - self._network_config = cmdline_cfg |
3340 | - else: |
3341 | + self._network_config = cmdline.read_initramfs_config() |
3342 | + if not self._network_config: |
3343 | self._network_config = self.distro.generate_fallback_config() |
3344 | + if self.ds_cfg.get('configure_secondary_nics'): |
3345 | + try: |
3346 | + # Mutate self._network_config to include secondary VNICs |
3347 | + _add_network_config_from_opc_imds(self._network_config) |
3348 | + except Exception: |
3349 | + util.logexc( |
3350 | + LOG, |
3351 | + "Failed to fetch secondary network configuration!") |
3352 | return self._network_config |
3353 | |
3354 | |
3355 | @@ -137,7 +229,7 @@ def _is_platform_viable(): |
3356 | |
3357 | |
3358 | def _is_iscsi_root(): |
3359 | - return bool(cmdline.read_kernel_cmdline_config()) |
3360 | + return bool(cmdline.read_initramfs_config()) |
3361 | |
3362 | |
3363 | def _load_index(content): |
3364 | diff --git a/cloudinit/sources/__init__.py b/cloudinit/sources/__init__.py |
3365 | index e6966b3..a319322 100644 |
3366 | --- a/cloudinit/sources/__init__.py |
3367 | +++ b/cloudinit/sources/__init__.py |
3368 | @@ -66,6 +66,13 @@ CLOUD_ID_REGION_PREFIX_MAP = { |
3369 | 'china': ('azure-china', lambda c: c == 'azure'), # only change azure |
3370 | } |
3371 | |
3372 | +# NetworkConfigSource represents the canonical list of network config sources |
3373 | +# that cloud-init knows about. (Python 2.7 lacks PEP 435, so use a singleton |
3374 | +# namedtuple as an enum; see https://stackoverflow.com/a/6971002) |
3375 | +_NETCFG_SOURCE_NAMES = ('cmdline', 'ds', 'system_cfg', 'fallback', 'initramfs') |
3376 | +NetworkConfigSource = namedtuple('NetworkConfigSource', |
3377 | + _NETCFG_SOURCE_NAMES)(*_NETCFG_SOURCE_NAMES) |
3378 | + |
3379 | |
3380 | class DataSourceNotFoundException(Exception): |
3381 | pass |
3382 | @@ -153,6 +160,16 @@ class DataSource(object): |
3383 | # Track the discovered fallback nic for use in configuration generation. |
3384 | _fallback_interface = None |
3385 | |
3386 | + # The network configuration sources that should be considered for this data |
3387 | + # source. (The first source in this list that provides network |
3388 | + # configuration will be used without considering any that follow.) This |
3389 | + # should always be a subset of the members of NetworkConfigSource with no |
3390 | + # duplicate entries. |
3391 | + network_config_sources = (NetworkConfigSource.cmdline, |
3392 | + NetworkConfigSource.initramfs, |
3393 | + NetworkConfigSource.system_cfg, |
3394 | + NetworkConfigSource.ds) |
3395 | + |
3396 | # read_url_params |
3397 | url_max_wait = -1 # max_wait < 0 means do not wait |
3398 | url_timeout = 10 # timeout for each metadata url read attempt |
3399 | @@ -474,6 +491,16 @@ class DataSource(object): |
3400 | def get_public_ssh_keys(self): |
3401 | return normalize_pubkey_data(self.metadata.get('public-keys')) |
3402 | |
3403 | + def publish_host_keys(self, hostkeys): |
3404 | + """Publish the public SSH host keys (found in /etc/ssh/*.pub). |
3405 | + |
3406 | + @param hostkeys: List of host key tuples (key_type, key_value), |
3407 | + where key_type is the first field in the public key file |
3408 | + (e.g. 'ssh-rsa') and key_value is the key itself |
3409 | + (e.g. 'AAAAB3NzaC1y...'). |
3410 | + """ |
3411 | + pass |
3412 | + |
3413 | def _remap_device(self, short_name): |
3414 | # LP: #611137 |
3415 | # the metadata service may believe that devices are named 'sda' |
3416 | diff --git a/cloudinit/sources/helpers/azure.py b/cloudinit/sources/helpers/azure.py |
3417 | index 82c4c8c..f1fba17 100755 |
3418 | --- a/cloudinit/sources/helpers/azure.py |
3419 | +++ b/cloudinit/sources/helpers/azure.py |
3420 | @@ -16,7 +16,11 @@ from xml.etree import ElementTree |
3421 | |
3422 | from cloudinit import url_helper |
3423 | from cloudinit import util |
3424 | +from cloudinit import version |
3425 | +from cloudinit import distros |
3426 | from cloudinit.reporting import events |
3427 | +from cloudinit.net.dhcp import EphemeralDHCPv4 |
3428 | +from datetime import datetime |
3429 | |
3430 | LOG = logging.getLogger(__name__) |
3431 | |
3432 | @@ -24,6 +28,10 @@ LOG = logging.getLogger(__name__) |
3433 | # value is applied if the endpoint can't be found within a lease file |
3434 | DEFAULT_WIRESERVER_ENDPOINT = "a8:3f:81:10" |
3435 | |
3436 | +BOOT_EVENT_TYPE = 'boot-telemetry' |
3437 | +SYSTEMINFO_EVENT_TYPE = 'system-info' |
3438 | +DIAGNOSTIC_EVENT_TYPE = 'diagnostic' |
3439 | + |
3440 | azure_ds_reporter = events.ReportEventStack( |
3441 | name="azure-ds", |
3442 | description="initialize reporter for azure ds", |
3443 | @@ -40,6 +48,105 @@ def azure_ds_telemetry_reporter(func): |
3444 | return impl |
3445 | |
3446 | |
3447 | +@azure_ds_telemetry_reporter |
3448 | +def get_boot_telemetry(): |
3449 | + """Report timestamps related to kernel initialization and systemd |
3450 | + activation of cloud-init""" |
3451 | + if not distros.uses_systemd(): |
3452 | + raise RuntimeError( |
3453 | + "distro not using systemd, skipping boot telemetry") |
3454 | + |
3455 | + LOG.debug("Collecting boot telemetry") |
3456 | + try: |
3457 | + kernel_start = float(time.time()) - float(util.uptime()) |
3458 | + except ValueError: |
3459 | + raise RuntimeError("Failed to determine kernel start timestamp") |
3460 | + |
3461 | + try: |
3462 | + out, _ = util.subp(['/bin/systemctl', |
3463 | + 'show', '-p', |
3464 | + 'UserspaceTimestampMonotonic'], |
3465 | + capture=True) |
3466 | + tsm = None |
3467 | + if out and '=' in out: |
3468 | + tsm = out.split("=")[1] |
3469 | + |
3470 | + if not tsm: |
3471 | + raise RuntimeError("Failed to parse " |
3472 | + "UserspaceTimestampMonotonic from systemd") |
3473 | + |
3474 | + user_start = kernel_start + (float(tsm) / 1000000) |
3475 | + except util.ProcessExecutionError as e: |
3476 | + raise RuntimeError("Failed to get UserspaceTimestampMonotonic: %s" |
3477 | + % e) |
3478 | + except ValueError as e: |
3479 | + raise RuntimeError("Failed to parse " |
3480 | + "UserspaceTimestampMonotonic from systemd: %s" |
3481 | + % e) |
3482 | + |
3483 | + try: |
3484 | + out, _ = util.subp(['/bin/systemctl', 'show', |
3485 | + 'cloud-init-local', '-p', |
3486 | + 'InactiveExitTimestampMonotonic'], |
3487 | + capture=True) |
3488 | + tsm = None |
3489 | + if out and '=' in out: |
3490 | + tsm = out.split("=")[1] |
3491 | + if not tsm: |
3492 | + raise RuntimeError("Failed to parse " |
3493 | + "InactiveExitTimestampMonotonic from systemd") |
3494 | + |
3495 | + cloudinit_activation = kernel_start + (float(tsm) / 1000000) |
3496 | + except util.ProcessExecutionError as e: |
3497 | + raise RuntimeError("Failed to get InactiveExitTimestampMonotonic: %s" |
3498 | + % e) |
3499 | + except ValueError as e: |
3500 | + raise RuntimeError("Failed to parse " |
3501 | + "InactiveExitTimestampMonotonic from systemd: %s" |
3502 | + % e) |
3503 | + |
3504 | + evt = events.ReportingEvent( |
3505 | + BOOT_EVENT_TYPE, 'boot-telemetry', |
3506 | + "kernel_start=%s user_start=%s cloudinit_activation=%s" % |
3507 | + (datetime.utcfromtimestamp(kernel_start).isoformat() + 'Z', |
3508 | + datetime.utcfromtimestamp(user_start).isoformat() + 'Z', |
3509 | + datetime.utcfromtimestamp(cloudinit_activation).isoformat() + 'Z'), |
3510 | + events.DEFAULT_EVENT_ORIGIN) |
3511 | + events.report_event(evt) |
3512 | + |
3513 | + # return the event for unit testing purpose |
3514 | + return evt |
3515 | + |
3516 | + |
3517 | +@azure_ds_telemetry_reporter |
3518 | +def get_system_info(): |
3519 | + """Collect and report system information""" |
3520 | + info = util.system_info() |
3521 | + evt = events.ReportingEvent( |
3522 | + SYSTEMINFO_EVENT_TYPE, 'system information', |
3523 | + "cloudinit_version=%s, kernel_version=%s, variant=%s, " |
3524 | + "distro_name=%s, distro_version=%s, flavor=%s, " |
3525 | + "python_version=%s" % |
3526 | + (version.version_string(), info['release'], info['variant'], |
3527 | + info['dist'][0], info['dist'][1], info['dist'][2], |
3528 | + info['python']), events.DEFAULT_EVENT_ORIGIN) |
3529 | + events.report_event(evt) |
3530 | + |
3531 | + # return the event for unit testing purpose |
3532 | + return evt |
3533 | + |
3534 | + |
3535 | +def report_diagnostic_event(str): |
3536 | + """Report a diagnostic event""" |
3537 | + evt = events.ReportingEvent( |
3538 | + DIAGNOSTIC_EVENT_TYPE, 'diagnostic message', |
3539 | + str, events.DEFAULT_EVENT_ORIGIN) |
3540 | + events.report_event(evt) |
3541 | + |
3542 | + # return the event for unit testing purpose |
3543 | + return evt |
3544 | + |
3545 | + |
3546 | @contextmanager |
3547 | def cd(newdir): |
3548 | prevdir = os.getcwd() |
3549 | @@ -360,16 +467,19 @@ class WALinuxAgentShim(object): |
3550 | value = dhcp245 |
3551 | LOG.debug("Using Azure Endpoint from dhcp options") |
3552 | if value is None: |
3553 | + report_diagnostic_event("No Azure endpoint from dhcp options") |
3554 | LOG.debug('Finding Azure endpoint from networkd...') |
3555 | value = WALinuxAgentShim._networkd_get_value_from_leases() |
3556 | if value is None: |
3557 | # Option-245 stored in /run/cloud-init/dhclient.hooks/<ifc>.json |
3558 | # a dhclient exit hook that calls cloud-init-dhclient-hook |
3559 | + report_diagnostic_event("No Azure endpoint from networkd") |
3560 | LOG.debug('Finding Azure endpoint from hook json...') |
3561 | dhcp_options = WALinuxAgentShim._load_dhclient_json() |
3562 | value = WALinuxAgentShim._get_value_from_dhcpoptions(dhcp_options) |
3563 | if value is None: |
3564 | # Fallback and check the leases file if unsuccessful |
3565 | + report_diagnostic_event("No Azure endpoint from dhclient logs") |
3566 | LOG.debug("Unable to find endpoint in dhclient logs. " |
3567 | " Falling back to check lease files") |
3568 | if fallback_lease_file is None: |
3569 | @@ -381,11 +491,15 @@ class WALinuxAgentShim(object): |
3570 | value = WALinuxAgentShim._get_value_from_leases_file( |
3571 | fallback_lease_file) |
3572 | if value is None: |
3573 | - LOG.warning("No lease found; using default endpoint") |
3574 | + msg = "No lease found; using default endpoint" |
3575 | + report_diagnostic_event(msg) |
3576 | + LOG.warning(msg) |
3577 | value = DEFAULT_WIRESERVER_ENDPOINT |
3578 | |
3579 | endpoint_ip_address = WALinuxAgentShim.get_ip_from_lease_value(value) |
3580 | - LOG.debug('Azure endpoint found at %s', endpoint_ip_address) |
3581 | + msg = 'Azure endpoint found at %s' % endpoint_ip_address |
3582 | + report_diagnostic_event(msg) |
3583 | + LOG.debug(msg) |
3584 | return endpoint_ip_address |
3585 | |
3586 | @azure_ds_telemetry_reporter |
3587 | @@ -399,16 +513,19 @@ class WALinuxAgentShim(object): |
3588 | try: |
3589 | response = http_client.get( |
3590 | 'http://{0}/machine/?comp=goalstate'.format(self.endpoint)) |
3591 | - except Exception: |
3592 | + except Exception as e: |
3593 | if attempts < 10: |
3594 | time.sleep(attempts + 1) |
3595 | else: |
3596 | + report_diagnostic_event( |
3597 | + "failed to register with Azure: %s" % e) |
3598 | raise |
3599 | else: |
3600 | break |
3601 | attempts += 1 |
3602 | LOG.debug('Successfully fetched GoalState XML.') |
3603 | goal_state = GoalState(response.contents, http_client) |
3604 | + report_diagnostic_event("container_id %s" % goal_state.container_id) |
3605 | ssh_keys = [] |
3606 | if goal_state.certificates_xml is not None and pubkey_info is not None: |
3607 | LOG.debug('Certificate XML found; parsing out public keys.') |
3608 | @@ -449,11 +566,20 @@ class WALinuxAgentShim(object): |
3609 | container_id=goal_state.container_id, |
3610 | instance_id=goal_state.instance_id, |
3611 | ) |
3612 | - http_client.post( |
3613 | - "http://{0}/machine?comp=health".format(self.endpoint), |
3614 | - data=document, |
3615 | - extra_headers={'Content-Type': 'text/xml; charset=utf-8'}, |
3616 | - ) |
3617 | + # Host will collect kvps when cloud-init reports ready. |
3618 | + # some kvps might still be in the queue. We yield the scheduler |
3619 | + # to make sure we process all kvps up till this point. |
3620 | + time.sleep(0) |
3621 | + try: |
3622 | + http_client.post( |
3623 | + "http://{0}/machine?comp=health".format(self.endpoint), |
3624 | + data=document, |
3625 | + extra_headers={'Content-Type': 'text/xml; charset=utf-8'}, |
3626 | + ) |
3627 | + except Exception as e: |
3628 | + report_diagnostic_event("exception while reporting ready: %s" % e) |
3629 | + raise |
3630 | + |
3631 | LOG.info('Reported ready to Azure fabric.') |
3632 | |
3633 | |
3634 | @@ -467,4 +593,22 @@ def get_metadata_from_fabric(fallback_lease_file=None, dhcp_opts=None, |
3635 | finally: |
3636 | shim.clean_up() |
3637 | |
3638 | + |
3639 | +class EphemeralDHCPv4WithReporting(object): |
3640 | + def __init__(self, reporter, nic=None): |
3641 | + self.reporter = reporter |
3642 | + self.ephemeralDHCPv4 = EphemeralDHCPv4(iface=nic) |
3643 | + |
3644 | + def __enter__(self): |
3645 | + with events.ReportEventStack( |
3646 | + name="obtain-dhcp-lease", |
3647 | + description="obtain dhcp lease", |
3648 | + parent=self.reporter): |
3649 | + return self.ephemeralDHCPv4.__enter__() |
3650 | + |
3651 | + def __exit__(self, excp_type, excp_value, excp_traceback): |
3652 | + self.ephemeralDHCPv4.__exit__( |
3653 | + excp_type, excp_value, excp_traceback) |
3654 | + |
3655 | + |
3656 | # vi: ts=4 expandtab |
3657 | diff --git a/cloudinit/sources/helpers/vmware/imc/config_custom_script.py b/cloudinit/sources/helpers/vmware/imc/config_custom_script.py |
3658 | index a7d4ad9..9f14770 100644 |
3659 | --- a/cloudinit/sources/helpers/vmware/imc/config_custom_script.py |
3660 | +++ b/cloudinit/sources/helpers/vmware/imc/config_custom_script.py |
3661 | @@ -1,5 +1,5 @@ |
3662 | # Copyright (C) 2017 Canonical Ltd. |
3663 | -# Copyright (C) 2017 VMware Inc. |
3664 | +# Copyright (C) 2017-2019 VMware Inc. |
3665 | # |
3666 | # Author: Maitreyee Saikia <msaikia@vmware.com> |
3667 | # |
3668 | @@ -8,7 +8,6 @@ |
3669 | import logging |
3670 | import os |
3671 | import stat |
3672 | -from textwrap import dedent |
3673 | |
3674 | from cloudinit import util |
3675 | |
3676 | @@ -20,12 +19,15 @@ class CustomScriptNotFound(Exception): |
3677 | |
3678 | |
3679 | class CustomScriptConstant(object): |
3680 | - RC_LOCAL = "/etc/rc.local" |
3681 | - POST_CUST_TMP_DIR = "/root/.customization" |
3682 | - POST_CUST_RUN_SCRIPT_NAME = "post-customize-guest.sh" |
3683 | - POST_CUST_RUN_SCRIPT = os.path.join(POST_CUST_TMP_DIR, |
3684 | - POST_CUST_RUN_SCRIPT_NAME) |
3685 | - POST_REBOOT_PENDING_MARKER = "/.guest-customization-post-reboot-pending" |
3686 | + CUSTOM_TMP_DIR = "/root/.customization" |
3687 | + |
3688 | + # The user defined custom script |
3689 | + CUSTOM_SCRIPT_NAME = "customize.sh" |
3690 | + CUSTOM_SCRIPT = os.path.join(CUSTOM_TMP_DIR, |
3691 | + CUSTOM_SCRIPT_NAME) |
3692 | + POST_CUSTOM_PENDING_MARKER = "/.guest-customization-post-reboot-pending" |
3693 | + # The cc_scripts_per_instance script to launch custom script |
3694 | + POST_CUSTOM_SCRIPT_NAME = "post-customize-guest.sh" |
3695 | |
3696 | |
3697 | class RunCustomScript(object): |
3698 | @@ -39,10 +41,19 @@ class RunCustomScript(object): |
3699 | raise CustomScriptNotFound("Script %s not found!! " |
3700 | "Cannot execute custom script!" |
3701 | % self.scriptpath) |
3702 | + |
3703 | + util.ensure_dir(CustomScriptConstant.CUSTOM_TMP_DIR) |
3704 | + |
3705 | + LOG.debug("Copying custom script to %s", |
3706 | + CustomScriptConstant.CUSTOM_SCRIPT) |
3707 | + util.copy(self.scriptpath, CustomScriptConstant.CUSTOM_SCRIPT) |
3708 | + |
3709 | # Strip any CR characters from the decoded script |
3710 | - util.load_file(self.scriptpath).replace("\r", "") |
3711 | - st = os.stat(self.scriptpath) |
3712 | - os.chmod(self.scriptpath, st.st_mode | stat.S_IEXEC) |
3713 | + content = util.load_file( |
3714 | + CustomScriptConstant.CUSTOM_SCRIPT).replace("\r", "") |
3715 | + util.write_file(CustomScriptConstant.CUSTOM_SCRIPT, |
3716 | + content, |
3717 | + mode=0o544) |
3718 | |
3719 | |
3720 | class PreCustomScript(RunCustomScript): |
3721 | @@ -50,104 +61,34 @@ class PreCustomScript(RunCustomScript): |
3722 | """Executing custom script with precustomization argument.""" |
3723 | LOG.debug("Executing pre-customization script") |
3724 | self.prepare_script() |
3725 | - util.subp(["/bin/sh", self.scriptpath, "precustomization"]) |
3726 | + util.subp([CustomScriptConstant.CUSTOM_SCRIPT, "precustomization"]) |
3727 | |
3728 | |
3729 | class PostCustomScript(RunCustomScript): |
3730 | - def __init__(self, scriptname, directory): |
3731 | + def __init__(self, scriptname, directory, ccScriptsDir): |
3732 | super(PostCustomScript, self).__init__(scriptname, directory) |
3733 | - # Determine when to run custom script. When postreboot is True, |
3734 | - # the user uploaded script will run as part of rc.local after |
3735 | - # the machine reboots. This is determined by presence of rclocal. |
3736 | - # When postreboot is False, script will run as part of cloud-init. |
3737 | - self.postreboot = False |
3738 | - |
3739 | - def _install_post_reboot_agent(self, rclocal): |
3740 | - """ |
3741 | - Install post-reboot agent for running custom script after reboot. |
3742 | - As part of this process, we are editing the rclocal file to run a |
3743 | - VMware script, which in turn is resposible for handling the user |
3744 | - script. |
3745 | - @param: path to rc local. |
3746 | - """ |
3747 | - LOG.debug("Installing post-reboot customization from %s to %s", |
3748 | - self.directory, rclocal) |
3749 | - if not self.has_previous_agent(rclocal): |
3750 | - LOG.info("Adding post-reboot customization agent to rc.local") |
3751 | - new_content = dedent(""" |
3752 | - # Run post-reboot guest customization |
3753 | - /bin/sh %s |
3754 | - exit 0 |
3755 | - """) % CustomScriptConstant.POST_CUST_RUN_SCRIPT |
3756 | - existing_rclocal = util.load_file(rclocal).replace('exit 0\n', '') |
3757 | - st = os.stat(rclocal) |
3758 | - # "x" flag should be set |
3759 | - mode = st.st_mode | stat.S_IEXEC |
3760 | - util.write_file(rclocal, existing_rclocal + new_content, mode) |
3761 | - |
3762 | - else: |
3763 | - # We don't need to update rclocal file everytime a customization |
3764 | - # is requested. It just needs to be done for the first time. |
3765 | - LOG.info("Post-reboot guest customization agent is already " |
3766 | - "registered in rc.local") |
3767 | - LOG.debug("Installing post-reboot customization agent finished: %s", |
3768 | - self.postreboot) |
3769 | - |
3770 | - def has_previous_agent(self, rclocal): |
3771 | - searchstring = "# Run post-reboot guest customization" |
3772 | - if searchstring in open(rclocal).read(): |
3773 | - return True |
3774 | - return False |
3775 | - |
3776 | - def find_rc_local(self): |
3777 | - """ |
3778 | - Determine if rc local is present. |
3779 | - """ |
3780 | - rclocal = "" |
3781 | - if os.path.exists(CustomScriptConstant.RC_LOCAL): |
3782 | - LOG.debug("rc.local detected.") |
3783 | - # resolving in case of symlink |
3784 | - rclocal = os.path.realpath(CustomScriptConstant.RC_LOCAL) |
3785 | - LOG.debug("rc.local resolved to %s", rclocal) |
3786 | - else: |
3787 | - LOG.warning("Can't find rc.local, post-customization " |
3788 | - "will be run before reboot") |
3789 | - return rclocal |
3790 | - |
3791 | - def install_agent(self): |
3792 | - rclocal = self.find_rc_local() |
3793 | - if rclocal: |
3794 | - self._install_post_reboot_agent(rclocal) |
3795 | - self.postreboot = True |
3796 | + self.ccScriptsDir = ccScriptsDir |
3797 | + self.ccScriptPath = os.path.join( |
3798 | + ccScriptsDir, |
3799 | + CustomScriptConstant.POST_CUSTOM_SCRIPT_NAME) |
3800 | |
3801 | def execute(self): |
3802 | """ |
3803 | - This method executes post-customization script before or after reboot |
3804 | - based on the presence of rc local. |
3805 | + This method copy the post customize run script to |
3806 | + cc_scripts_per_instance directory and let this |
3807 | + module to run post custom script. |
3808 | """ |
3809 | self.prepare_script() |
3810 | - self.install_agent() |
3811 | - if not self.postreboot: |
3812 | - LOG.warning("Executing post-customization script inline") |
3813 | - util.subp(["/bin/sh", self.scriptpath, "postcustomization"]) |
3814 | - else: |
3815 | - LOG.debug("Scheduling custom script to run post reboot") |
3816 | - if not os.path.isdir(CustomScriptConstant.POST_CUST_TMP_DIR): |
3817 | - os.mkdir(CustomScriptConstant.POST_CUST_TMP_DIR) |
3818 | - # Script "post-customize-guest.sh" and user uploaded script are |
3819 | - # are present in the same directory and needs to copied to a temp |
3820 | - # directory to be executed post reboot. User uploaded script is |
3821 | - # saved as customize.sh in the temp directory. |
3822 | - # post-customize-guest.sh excutes customize.sh after reboot. |
3823 | - LOG.debug("Copying post-customization script") |
3824 | - util.copy(self.scriptpath, |
3825 | - CustomScriptConstant.POST_CUST_TMP_DIR + "/customize.sh") |
3826 | - LOG.debug("Copying script to run post-customization script") |
3827 | - util.copy( |
3828 | - os.path.join(self.directory, |
3829 | - CustomScriptConstant.POST_CUST_RUN_SCRIPT_NAME), |
3830 | - CustomScriptConstant.POST_CUST_RUN_SCRIPT) |
3831 | - LOG.info("Creating post-reboot pending marker") |
3832 | - util.ensure_file(CustomScriptConstant.POST_REBOOT_PENDING_MARKER) |
3833 | + |
3834 | + LOG.debug("Copying post customize run script to %s", |
3835 | + self.ccScriptPath) |
3836 | + util.copy( |
3837 | + os.path.join(self.directory, |
3838 | + CustomScriptConstant.POST_CUSTOM_SCRIPT_NAME), |
3839 | + self.ccScriptPath) |
3840 | + st = os.stat(self.ccScriptPath) |
3841 | + os.chmod(self.ccScriptPath, st.st_mode | stat.S_IEXEC) |
3842 | + LOG.info("Creating post customization pending marker") |
3843 | + util.ensure_file(CustomScriptConstant.POST_CUSTOM_PENDING_MARKER) |
3844 | |
3845 | # vi: ts=4 expandtab |
3846 | diff --git a/cloudinit/sources/tests/test_oracle.py b/cloudinit/sources/tests/test_oracle.py |
3847 | index 97d6294..3ddf7df 100644 |
3848 | --- a/cloudinit/sources/tests/test_oracle.py |
3849 | +++ b/cloudinit/sources/tests/test_oracle.py |
3850 | @@ -1,7 +1,7 @@ |
3851 | # This file is part of cloud-init. See LICENSE file for license information. |
3852 | |
3853 | from cloudinit.sources import DataSourceOracle as oracle |
3854 | -from cloudinit.sources import BrokenMetadata |
3855 | +from cloudinit.sources import BrokenMetadata, NetworkConfigSource |
3856 | from cloudinit import helpers |
3857 | |
3858 | from cloudinit.tests import helpers as test_helpers |
3859 | @@ -18,10 +18,52 @@ import uuid |
3860 | DS_PATH = "cloudinit.sources.DataSourceOracle" |
3861 | MD_VER = "2013-10-17" |
3862 | |
3863 | +# `curl -L http://169.254.169.254/opc/v1/vnics/` on a Oracle Bare Metal Machine |
3864 | +# with a secondary VNIC attached (vnicId truncated for Python line length) |
3865 | +OPC_BM_SECONDARY_VNIC_RESPONSE = """\ |
3866 | +[ { |
3867 | + "vnicId" : "ocid1.vnic.oc1.phx.abyhqljtyvcucqkhdqmgjszebxe4hrb!!TRUNCATED||", |
3868 | + "privateIp" : "10.0.0.8", |
3869 | + "vlanTag" : 0, |
3870 | + "macAddr" : "90:e2:ba:d4:f1:68", |
3871 | + "virtualRouterIp" : "10.0.0.1", |
3872 | + "subnetCidrBlock" : "10.0.0.0/24", |
3873 | + "nicIndex" : 0 |
3874 | +}, { |
3875 | + "vnicId" : "ocid1.vnic.oc1.phx.abyhqljtfmkxjdy2sqidndiwrsg63zf!!TRUNCATED||", |
3876 | + "privateIp" : "10.0.4.5", |
3877 | + "vlanTag" : 1, |
3878 | + "macAddr" : "02:00:17:05:CF:51", |
3879 | + "virtualRouterIp" : "10.0.4.1", |
3880 | + "subnetCidrBlock" : "10.0.4.0/24", |
3881 | + "nicIndex" : 0 |
3882 | +} ]""" |
3883 | + |
3884 | +# `curl -L http://169.254.169.254/opc/v1/vnics/` on a Oracle Virtual Machine |
3885 | +# with a secondary VNIC attached |
3886 | +OPC_VM_SECONDARY_VNIC_RESPONSE = """\ |
3887 | +[ { |
3888 | + "vnicId" : "ocid1.vnic.oc1.phx.abyhqljtch72z5pd76cc2636qeqh7z_truncated", |
3889 | + "privateIp" : "10.0.0.230", |
3890 | + "vlanTag" : 1039, |
3891 | + "macAddr" : "02:00:17:05:D1:DB", |
3892 | + "virtualRouterIp" : "10.0.0.1", |
3893 | + "subnetCidrBlock" : "10.0.0.0/24" |
3894 | +}, { |
3895 | + "vnicId" : "ocid1.vnic.oc1.phx.abyhqljt4iew3gwmvrwrhhf3bp5drj_truncated", |
3896 | + "privateIp" : "10.0.0.231", |
3897 | + "vlanTag" : 1041, |
3898 | + "macAddr" : "00:00:17:02:2B:B1", |
3899 | + "virtualRouterIp" : "10.0.0.1", |
3900 | + "subnetCidrBlock" : "10.0.0.0/24" |
3901 | +} ]""" |
3902 | + |
3903 | |
3904 | class TestDataSourceOracle(test_helpers.CiTestCase): |
3905 | """Test datasource DataSourceOracle.""" |
3906 | |
3907 | + with_logs = True |
3908 | + |
3909 | ds_class = oracle.DataSourceOracle |
3910 | |
3911 | my_uuid = str(uuid.uuid4()) |
3912 | @@ -79,6 +121,16 @@ class TestDataSourceOracle(test_helpers.CiTestCase): |
3913 | self.assertEqual( |
3914 | 'metadata (http://169.254.169.254/openstack/)', ds.subplatform) |
3915 | |
3916 | + def test_sys_cfg_can_enable_configure_secondary_nics(self): |
3917 | + # Confirm that behaviour is toggled by sys_cfg |
3918 | + ds, _mocks = self._get_ds() |
3919 | + self.assertFalse(ds.ds_cfg['configure_secondary_nics']) |
3920 | + |
3921 | + sys_cfg = { |
3922 | + 'datasource': {'Oracle': {'configure_secondary_nics': True}}} |
3923 | + ds, _mocks = self._get_ds(sys_cfg=sys_cfg) |
3924 | + self.assertTrue(ds.ds_cfg['configure_secondary_nics']) |
3925 | + |
3926 | @mock.patch(DS_PATH + "._is_iscsi_root", return_value=True) |
3927 | def test_without_userdata(self, m_is_iscsi_root): |
3928 | """If no user-data is provided, it should not be in return dict.""" |
3929 | @@ -133,9 +185,12 @@ class TestDataSourceOracle(test_helpers.CiTestCase): |
3930 | self.assertEqual(self.my_md['uuid'], ds.get_instance_id()) |
3931 | self.assertEqual(my_userdata, ds.userdata_raw) |
3932 | |
3933 | - @mock.patch(DS_PATH + ".cmdline.read_kernel_cmdline_config") |
3934 | + @mock.patch(DS_PATH + "._add_network_config_from_opc_imds", |
3935 | + side_effect=lambda network_config: network_config) |
3936 | + @mock.patch(DS_PATH + ".cmdline.read_initramfs_config") |
3937 | @mock.patch(DS_PATH + "._is_iscsi_root", return_value=True) |
3938 | - def test_network_cmdline(self, m_is_iscsi_root, m_cmdline_config): |
3939 | + def test_network_cmdline(self, m_is_iscsi_root, m_initramfs_config, |
3940 | + _m_add_network_config_from_opc_imds): |
3941 | """network_config should read kernel cmdline.""" |
3942 | distro = mock.MagicMock() |
3943 | ds, _ = self._get_ds(distro=distro, patches={ |
3944 | @@ -145,15 +200,18 @@ class TestDataSourceOracle(test_helpers.CiTestCase): |
3945 | MD_VER: {'system_uuid': self.my_uuid, |
3946 | 'meta_data': self.my_md}}}}) |
3947 | ncfg = {'version': 1, 'config': [{'a': 'b'}]} |
3948 | - m_cmdline_config.return_value = ncfg |
3949 | + m_initramfs_config.return_value = ncfg |
3950 | self.assertTrue(ds._get_data()) |
3951 | self.assertEqual(ncfg, ds.network_config) |
3952 | - m_cmdline_config.assert_called_once_with() |
3953 | + self.assertEqual([mock.call()], m_initramfs_config.call_args_list) |
3954 | self.assertFalse(distro.generate_fallback_config.called) |
3955 | |
3956 | - @mock.patch(DS_PATH + ".cmdline.read_kernel_cmdline_config") |
3957 | + @mock.patch(DS_PATH + "._add_network_config_from_opc_imds", |
3958 | + side_effect=lambda network_config: network_config) |
3959 | + @mock.patch(DS_PATH + ".cmdline.read_initramfs_config") |
3960 | @mock.patch(DS_PATH + "._is_iscsi_root", return_value=True) |
3961 | - def test_network_fallback(self, m_is_iscsi_root, m_cmdline_config): |
3962 | + def test_network_fallback(self, m_is_iscsi_root, m_initramfs_config, |
3963 | + _m_add_network_config_from_opc_imds): |
3964 | """test that fallback network is generated if no kernel cmdline.""" |
3965 | distro = mock.MagicMock() |
3966 | ds, _ = self._get_ds(distro=distro, patches={ |
3967 | @@ -163,18 +221,95 @@ class TestDataSourceOracle(test_helpers.CiTestCase): |
3968 | MD_VER: {'system_uuid': self.my_uuid, |
3969 | 'meta_data': self.my_md}}}}) |
3970 | ncfg = {'version': 1, 'config': [{'a': 'b'}]} |
3971 | - m_cmdline_config.return_value = None |
3972 | + m_initramfs_config.return_value = None |
3973 | self.assertTrue(ds._get_data()) |
3974 | ncfg = {'version': 1, 'config': [{'distro1': 'value'}]} |
3975 | distro.generate_fallback_config.return_value = ncfg |
3976 | self.assertEqual(ncfg, ds.network_config) |
3977 | - m_cmdline_config.assert_called_once_with() |
3978 | + self.assertEqual([mock.call()], m_initramfs_config.call_args_list) |
3979 | distro.generate_fallback_config.assert_called_once_with() |
3980 | - self.assertEqual(1, m_cmdline_config.call_count) |
3981 | |
3982 | # test that the result got cached, and the methods not re-called. |
3983 | self.assertEqual(ncfg, ds.network_config) |
3984 | - self.assertEqual(1, m_cmdline_config.call_count) |
3985 | + self.assertEqual(1, m_initramfs_config.call_count) |
3986 | + |
3987 | + @mock.patch(DS_PATH + "._add_network_config_from_opc_imds") |
3988 | + @mock.patch(DS_PATH + ".cmdline.read_initramfs_config", |
3989 | + return_value={'some': 'config'}) |
3990 | + @mock.patch(DS_PATH + "._is_iscsi_root", return_value=True) |
3991 | + def test_secondary_nics_added_to_network_config_if_enabled( |
3992 | + self, _m_is_iscsi_root, _m_initramfs_config, |
3993 | + m_add_network_config_from_opc_imds): |
3994 | + |
3995 | + needle = object() |
3996 | + |
3997 | + def network_config_side_effect(network_config): |
3998 | + network_config['secondary_added'] = needle |
3999 | + |
4000 | + m_add_network_config_from_opc_imds.side_effect = ( |
4001 | + network_config_side_effect) |
4002 | + |
4003 | + distro = mock.MagicMock() |
4004 | + ds, _ = self._get_ds(distro=distro, patches={ |
4005 | + '_is_platform_viable': {'return_value': True}, |
4006 | + 'crawl_metadata': { |
4007 | + 'return_value': { |
4008 | + MD_VER: {'system_uuid': self.my_uuid, |
4009 | + 'meta_data': self.my_md}}}}) |
4010 | + ds.ds_cfg['configure_secondary_nics'] = True |
4011 | + self.assertEqual(needle, ds.network_config['secondary_added']) |
4012 | + |
4013 | + @mock.patch(DS_PATH + "._add_network_config_from_opc_imds") |
4014 | + @mock.patch(DS_PATH + ".cmdline.read_initramfs_config", |
4015 | + return_value={'some': 'config'}) |
4016 | + @mock.patch(DS_PATH + "._is_iscsi_root", return_value=True) |
4017 | + def test_secondary_nics_not_added_to_network_config_by_default( |
4018 | + self, _m_is_iscsi_root, _m_initramfs_config, |
4019 | + m_add_network_config_from_opc_imds): |
4020 | + |
4021 | + def network_config_side_effect(network_config): |
4022 | + network_config['secondary_added'] = True |
4023 | + |
4024 | + m_add_network_config_from_opc_imds.side_effect = ( |
4025 | + network_config_side_effect) |
4026 | + |
4027 | + distro = mock.MagicMock() |
4028 | + ds, _ = self._get_ds(distro=distro, patches={ |
4029 | + '_is_platform_viable': {'return_value': True}, |
4030 | + 'crawl_metadata': { |
4031 | + 'return_value': { |
4032 | + MD_VER: {'system_uuid': self.my_uuid, |
4033 | + 'meta_data': self.my_md}}}}) |
4034 | + self.assertNotIn('secondary_added', ds.network_config) |
4035 | + |
4036 | + @mock.patch(DS_PATH + "._add_network_config_from_opc_imds") |
4037 | + @mock.patch(DS_PATH + ".cmdline.read_initramfs_config") |
4038 | + @mock.patch(DS_PATH + "._is_iscsi_root", return_value=True) |
4039 | + def test_secondary_nic_failure_isnt_blocking( |
4040 | + self, _m_is_iscsi_root, m_initramfs_config, |
4041 | + m_add_network_config_from_opc_imds): |
4042 | + |
4043 | + m_add_network_config_from_opc_imds.side_effect = Exception() |
4044 | + |
4045 | + distro = mock.MagicMock() |
4046 | + ds, _ = self._get_ds(distro=distro, patches={ |
4047 | + '_is_platform_viable': {'return_value': True}, |
4048 | + 'crawl_metadata': { |
4049 | + 'return_value': { |
4050 | + MD_VER: {'system_uuid': self.my_uuid, |
4051 | + 'meta_data': self.my_md}}}}) |
4052 | + ds.ds_cfg['configure_secondary_nics'] = True |
4053 | + self.assertEqual(ds.network_config, m_initramfs_config.return_value) |
4054 | + self.assertIn('Failed to fetch secondary network configuration', |
4055 | + self.logs.getvalue()) |
4056 | + |
4057 | + def test_ds_network_cfg_preferred_over_initramfs(self): |
4058 | + """Ensure that DS net config is preferred over initramfs config""" |
4059 | + network_config_sources = oracle.DataSourceOracle.network_config_sources |
4060 | + self.assertLess( |
4061 | + network_config_sources.index(NetworkConfigSource.ds), |
4062 | + network_config_sources.index(NetworkConfigSource.initramfs) |
4063 | + ) |
4064 | |
4065 | |
4066 | @mock.patch(DS_PATH + "._read_system_uuid", return_value=str(uuid.uuid4())) |
4067 | @@ -336,4 +471,86 @@ class TestLoadIndex(test_helpers.CiTestCase): |
4068 | oracle._load_index("\n".join(["meta_data.json", "user_data"]))) |
4069 | |
4070 | |
4071 | +class TestNetworkConfigFromOpcImds(test_helpers.CiTestCase): |
4072 | + |
4073 | + with_logs = True |
4074 | + |
4075 | + def setUp(self): |
4076 | + super(TestNetworkConfigFromOpcImds, self).setUp() |
4077 | + self.add_patch(DS_PATH + '.readurl', 'm_readurl') |
4078 | + self.add_patch(DS_PATH + '.get_interfaces_by_mac', |
4079 | + 'm_get_interfaces_by_mac') |
4080 | + |
4081 | + def test_failure_to_readurl(self): |
4082 | + # readurl failures should just bubble out to the caller |
4083 | + self.m_readurl.side_effect = Exception('oh no') |
4084 | + with self.assertRaises(Exception) as excinfo: |
4085 | + oracle._add_network_config_from_opc_imds({}) |
4086 | + self.assertEqual(str(excinfo.exception), 'oh no') |
4087 | + |
4088 | + def test_empty_response(self): |
4089 | + # empty response error should just bubble out to the caller |
4090 | + self.m_readurl.return_value = '' |
4091 | + with self.assertRaises(Exception): |
4092 | + oracle._add_network_config_from_opc_imds([]) |
4093 | + |
4094 | + def test_invalid_json(self): |
4095 | + # invalid JSON error should just bubble out to the caller |
4096 | + self.m_readurl.return_value = '{' |
4097 | + with self.assertRaises(Exception): |
4098 | + oracle._add_network_config_from_opc_imds([]) |
4099 | + |
4100 | + def test_no_secondary_nics_does_not_mutate_input(self): |
4101 | + self.m_readurl.return_value = json.dumps([{}]) |
4102 | + # We test this by passing in a non-dict to ensure that no dict |
4103 | + # operations are used; failure would be seen as exceptions |
4104 | + oracle._add_network_config_from_opc_imds(object()) |
4105 | + |
4106 | + def test_bare_metal_machine_skipped(self): |
4107 | + # nicIndex in the first entry indicates a bare metal machine |
4108 | + self.m_readurl.return_value = OPC_BM_SECONDARY_VNIC_RESPONSE |
4109 | + # We test this by passing in a non-dict to ensure that no dict |
4110 | + # operations are used |
4111 | + self.assertFalse(oracle._add_network_config_from_opc_imds(object())) |
4112 | + self.assertIn('bare metal machine', self.logs.getvalue()) |
4113 | + |
4114 | + def test_missing_mac_skipped(self): |
4115 | + self.m_readurl.return_value = OPC_VM_SECONDARY_VNIC_RESPONSE |
4116 | + self.m_get_interfaces_by_mac.return_value = {} |
4117 | + |
4118 | + network_config = {'version': 1, 'config': [{'primary': 'nic'}]} |
4119 | + oracle._add_network_config_from_opc_imds(network_config) |
4120 | + |
4121 | + self.assertEqual(1, len(network_config['config'])) |
4122 | + self.assertIn( |
4123 | + 'Interface with MAC 00:00:17:02:2b:b1 not found; skipping', |
4124 | + self.logs.getvalue()) |
4125 | + |
4126 | + def test_secondary_nic(self): |
4127 | + self.m_readurl.return_value = OPC_VM_SECONDARY_VNIC_RESPONSE |
4128 | + mac_addr, nic_name = '00:00:17:02:2b:b1', 'ens3' |
4129 | + self.m_get_interfaces_by_mac.return_value = { |
4130 | + mac_addr: nic_name, |
4131 | + } |
4132 | + |
4133 | + network_config = {'version': 1, 'config': [{'primary': 'nic'}]} |
4134 | + oracle._add_network_config_from_opc_imds(network_config) |
4135 | + |
4136 | + # The input is mutated |
4137 | + self.assertEqual(2, len(network_config['config'])) |
4138 | + |
4139 | + secondary_nic_cfg = network_config['config'][1] |
4140 | + self.assertEqual(nic_name, secondary_nic_cfg['name']) |
4141 | + self.assertEqual('physical', secondary_nic_cfg['type']) |
4142 | + self.assertEqual(mac_addr, secondary_nic_cfg['mac_address']) |
4143 | + self.assertEqual(9000, secondary_nic_cfg['mtu']) |
4144 | + |
4145 | + self.assertEqual(1, len(secondary_nic_cfg['subnets'])) |
4146 | + subnet_cfg = secondary_nic_cfg['subnets'][0] |
4147 | + # These values are hard-coded in OPC_VM_SECONDARY_VNIC_RESPONSE |
4148 | + self.assertEqual('10.0.0.231', subnet_cfg['address']) |
4149 | + self.assertEqual('24', subnet_cfg['netmask']) |
4150 | + self.assertEqual('10.0.0.1', subnet_cfg['gateway']) |
4151 | + self.assertEqual('manual', subnet_cfg['control']) |
4152 | + |
4153 | # vi: ts=4 expandtab |
4154 | diff --git a/cloudinit/stages.py b/cloudinit/stages.py |
4155 | index da7d349..5012988 100644 |
4156 | --- a/cloudinit/stages.py |
4157 | +++ b/cloudinit/stages.py |
4158 | @@ -24,6 +24,7 @@ from cloudinit.handlers.shell_script import ShellScriptPartHandler |
4159 | from cloudinit.handlers.upstart_job import UpstartJobPartHandler |
4160 | |
4161 | from cloudinit.event import EventType |
4162 | +from cloudinit.sources import NetworkConfigSource |
4163 | |
4164 | from cloudinit import cloud |
4165 | from cloudinit import config |
4166 | @@ -630,32 +631,54 @@ class Init(object): |
4167 | if os.path.exists(disable_file): |
4168 | return (None, disable_file) |
4169 | |
4170 | - cmdline_cfg = ('cmdline', cmdline.read_kernel_cmdline_config()) |
4171 | - dscfg = ('ds', None) |
4172 | + available_cfgs = { |
4173 | + NetworkConfigSource.cmdline: cmdline.read_kernel_cmdline_config(), |
4174 | + NetworkConfigSource.initramfs: cmdline.read_initramfs_config(), |
4175 | + NetworkConfigSource.ds: None, |
4176 | + NetworkConfigSource.system_cfg: self.cfg.get('network'), |
4177 | + } |
4178 | + |
4179 | if self.datasource and hasattr(self.datasource, 'network_config'): |
4180 | - dscfg = ('ds', self.datasource.network_config) |
4181 | - sys_cfg = ('system_cfg', self.cfg.get('network')) |
4182 | + available_cfgs[NetworkConfigSource.ds] = ( |
4183 | + self.datasource.network_config) |
4184 | |
4185 | - for loc, ncfg in (cmdline_cfg, sys_cfg, dscfg): |
4186 | + if self.datasource: |
4187 | + order = self.datasource.network_config_sources |
4188 | + else: |
4189 | + order = sources.DataSource.network_config_sources |
4190 | + for cfg_source in order: |
4191 | + if not hasattr(NetworkConfigSource, cfg_source): |
4192 | + LOG.warning('data source specifies an invalid network' |
4193 | + ' cfg_source: %s', cfg_source) |
4194 | + continue |
4195 | + if cfg_source not in available_cfgs: |
4196 | + LOG.warning('data source specifies an unavailable network' |
4197 | + ' cfg_source: %s', cfg_source) |
4198 | + continue |
4199 | + ncfg = available_cfgs[cfg_source] |
4200 | if net.is_disabled_cfg(ncfg): |
4201 | - LOG.debug("network config disabled by %s", loc) |
4202 | - return (None, loc) |
4203 | + LOG.debug("network config disabled by %s", cfg_source) |
4204 | + return (None, cfg_source) |
4205 | if ncfg: |
4206 | - return (ncfg, loc) |
4207 | - return (self.distro.generate_fallback_config(), "fallback") |
4208 | - |
4209 | - def apply_network_config(self, bring_up): |
4210 | - netcfg, src = self._find_networking_config() |
4211 | - if netcfg is None: |
4212 | - LOG.info("network config is disabled by %s", src) |
4213 | - return |
4214 | + return (ncfg, cfg_source) |
4215 | + return (self.distro.generate_fallback_config(), |
4216 | + NetworkConfigSource.fallback) |
4217 | |
4218 | + def _apply_netcfg_names(self, netcfg): |
4219 | try: |
4220 | LOG.debug("applying net config names for %s", netcfg) |
4221 | self.distro.apply_network_config_names(netcfg) |
4222 | except Exception as e: |
4223 | LOG.warning("Failed to rename devices: %s", e) |
4224 | |
4225 | + def apply_network_config(self, bring_up): |
4226 | + # get a network config |
4227 | + netcfg, src = self._find_networking_config() |
4228 | + if netcfg is None: |
4229 | + LOG.info("network config is disabled by %s", src) |
4230 | + return |
4231 | + |
4232 | + # request an update if needed/available |
4233 | if self.datasource is not NULL_DATA_SOURCE: |
4234 | if not self.is_new_instance(): |
4235 | if not self.datasource.update_metadata([EventType.BOOT]): |
4236 | @@ -663,8 +686,20 @@ class Init(object): |
4237 | "No network config applied. Neither a new instance" |
4238 | " nor datasource network update on '%s' event", |
4239 | EventType.BOOT) |
4240 | + # nothing new, but ensure proper names |
4241 | + self._apply_netcfg_names(netcfg) |
4242 | return |
4243 | + else: |
4244 | + # refresh netcfg after update |
4245 | + netcfg, src = self._find_networking_config() |
4246 | + |
4247 | + # ensure all physical devices in config are present |
4248 | + net.wait_for_physdevs(netcfg) |
4249 | + |
4250 | + # apply renames from config |
4251 | + self._apply_netcfg_names(netcfg) |
4252 | |
4253 | + # rendering config |
4254 | LOG.info("Applying network configuration from %s bringup=%s: %s", |
4255 | src, bring_up, netcfg) |
4256 | try: |
4257 | diff --git a/cloudinit/tests/helpers.py b/cloudinit/tests/helpers.py |
4258 | index f41180f..23fddd0 100644 |
4259 | --- a/cloudinit/tests/helpers.py |
4260 | +++ b/cloudinit/tests/helpers.py |
4261 | @@ -198,7 +198,8 @@ class CiTestCase(TestCase): |
4262 | prefix="ci-%s." % self.__class__.__name__) |
4263 | else: |
4264 | tmpd = tempfile.mkdtemp(dir=dir) |
4265 | - self.addCleanup(functools.partial(shutil.rmtree, tmpd)) |
4266 | + self.addCleanup( |
4267 | + functools.partial(shutil.rmtree, tmpd, ignore_errors=True)) |
4268 | return tmpd |
4269 | |
4270 | def tmp_path(self, path, dir=None): |
4271 | diff --git a/cloudinit/tests/test_stages.py b/cloudinit/tests/test_stages.py |
4272 | index 94b6b25..d5c9c0e 100644 |
4273 | --- a/cloudinit/tests/test_stages.py |
4274 | +++ b/cloudinit/tests/test_stages.py |
4275 | @@ -6,6 +6,7 @@ import os |
4276 | |
4277 | from cloudinit import stages |
4278 | from cloudinit import sources |
4279 | +from cloudinit.sources import NetworkConfigSource |
4280 | |
4281 | from cloudinit.event import EventType |
4282 | from cloudinit.util import write_file |
4283 | @@ -37,6 +38,7 @@ class FakeDataSource(sources.DataSource): |
4284 | |
4285 | class TestInit(CiTestCase): |
4286 | with_logs = True |
4287 | + allowed_subp = False |
4288 | |
4289 | def setUp(self): |
4290 | super(TestInit, self).setUp() |
4291 | @@ -57,84 +59,189 @@ class TestInit(CiTestCase): |
4292 | (None, disable_file), |
4293 | self.init._find_networking_config()) |
4294 | |
4295 | + @mock.patch('cloudinit.stages.cmdline.read_initramfs_config') |
4296 | @mock.patch('cloudinit.stages.cmdline.read_kernel_cmdline_config') |
4297 | - def test_wb__find_networking_config_disabled_by_kernel(self, m_cmdline): |
4298 | + def test_wb__find_networking_config_disabled_by_kernel( |
4299 | + self, m_cmdline, m_initramfs): |
4300 | """find_networking_config returns when disabled by kernel cmdline.""" |
4301 | m_cmdline.return_value = {'config': 'disabled'} |
4302 | + m_initramfs.return_value = {'config': ['fake_initrd']} |
4303 | self.assertEqual( |
4304 | - (None, 'cmdline'), |
4305 | + (None, NetworkConfigSource.cmdline), |
4306 | self.init._find_networking_config()) |
4307 | self.assertEqual('DEBUG: network config disabled by cmdline\n', |
4308 | self.logs.getvalue()) |
4309 | |
4310 | + @mock.patch('cloudinit.stages.cmdline.read_initramfs_config') |
4311 | @mock.patch('cloudinit.stages.cmdline.read_kernel_cmdline_config') |
4312 | - def test_wb__find_networking_config_disabled_by_datasrc(self, m_cmdline): |
4313 | + def test_wb__find_networking_config_disabled_by_initrd( |
4314 | + self, m_cmdline, m_initramfs): |
4315 | + """find_networking_config returns when disabled by kernel cmdline.""" |
4316 | + m_cmdline.return_value = {} |
4317 | + m_initramfs.return_value = {'config': 'disabled'} |
4318 | + self.assertEqual( |
4319 | + (None, NetworkConfigSource.initramfs), |
4320 | + self.init._find_networking_config()) |
4321 | + self.assertEqual('DEBUG: network config disabled by initramfs\n', |
4322 | + self.logs.getvalue()) |
4323 | + |
4324 | + @mock.patch('cloudinit.stages.cmdline.read_initramfs_config') |
4325 | + @mock.patch('cloudinit.stages.cmdline.read_kernel_cmdline_config') |
4326 | + def test_wb__find_networking_config_disabled_by_datasrc( |
4327 | + self, m_cmdline, m_initramfs): |
4328 | """find_networking_config returns when disabled by datasource cfg.""" |
4329 | m_cmdline.return_value = {} # Kernel doesn't disable networking |
4330 | + m_initramfs.return_value = {} # initramfs doesn't disable networking |
4331 | self.init._cfg = {'system_info': {'paths': {'cloud_dir': self.tmpdir}}, |
4332 | 'network': {}} # system config doesn't disable |
4333 | |
4334 | self.init.datasource = FakeDataSource( |
4335 | network_config={'config': 'disabled'}) |
4336 | self.assertEqual( |
4337 | - (None, 'ds'), |
4338 | + (None, NetworkConfigSource.ds), |
4339 | self.init._find_networking_config()) |
4340 | self.assertEqual('DEBUG: network config disabled by ds\n', |
4341 | self.logs.getvalue()) |
4342 | |
4343 | + @mock.patch('cloudinit.stages.cmdline.read_initramfs_config') |
4344 | @mock.patch('cloudinit.stages.cmdline.read_kernel_cmdline_config') |
4345 | - def test_wb__find_networking_config_disabled_by_sysconfig(self, m_cmdline): |
4346 | + def test_wb__find_networking_config_disabled_by_sysconfig( |
4347 | + self, m_cmdline, m_initramfs): |
4348 | """find_networking_config returns when disabled by system config.""" |
4349 | m_cmdline.return_value = {} # Kernel doesn't disable networking |
4350 | + m_initramfs.return_value = {} # initramfs doesn't disable networking |
4351 | self.init._cfg = {'system_info': {'paths': {'cloud_dir': self.tmpdir}}, |
4352 | 'network': {'config': 'disabled'}} |
4353 | self.assertEqual( |
4354 | - (None, 'system_cfg'), |
4355 | + (None, NetworkConfigSource.system_cfg), |
4356 | self.init._find_networking_config()) |
4357 | self.assertEqual('DEBUG: network config disabled by system_cfg\n', |
4358 | self.logs.getvalue()) |
4359 | |
4360 | + @mock.patch('cloudinit.stages.cmdline.read_initramfs_config') |
4361 | + @mock.patch('cloudinit.stages.cmdline.read_kernel_cmdline_config') |
4362 | + def test__find_networking_config_uses_datasrc_order( |
4363 | + self, m_cmdline, m_initramfs): |
4364 | + """find_networking_config should check sources in DS defined order""" |
4365 | + # cmdline and initramfs, which would normally be preferred over other |
4366 | + # sources, disable networking; in this case, though, the DS moves them |
4367 | + # later so its own config is preferred |
4368 | + m_cmdline.return_value = {'config': 'disabled'} |
4369 | + m_initramfs.return_value = {'config': 'disabled'} |
4370 | + |
4371 | + ds_net_cfg = {'config': {'needle': True}} |
4372 | + self.init.datasource = FakeDataSource(network_config=ds_net_cfg) |
4373 | + self.init.datasource.network_config_sources = [ |
4374 | + NetworkConfigSource.ds, NetworkConfigSource.system_cfg, |
4375 | + NetworkConfigSource.cmdline, NetworkConfigSource.initramfs] |
4376 | + |
4377 | + self.assertEqual( |
4378 | + (ds_net_cfg, NetworkConfigSource.ds), |
4379 | + self.init._find_networking_config()) |
4380 | + |
4381 | + @mock.patch('cloudinit.stages.cmdline.read_initramfs_config') |
4382 | + @mock.patch('cloudinit.stages.cmdline.read_kernel_cmdline_config') |
4383 | + def test__find_networking_config_warns_if_datasrc_uses_invalid_src( |
4384 | + self, m_cmdline, m_initramfs): |
4385 | + """find_networking_config should check sources in DS defined order""" |
4386 | + ds_net_cfg = {'config': {'needle': True}} |
4387 | + self.init.datasource = FakeDataSource(network_config=ds_net_cfg) |
4388 | + self.init.datasource.network_config_sources = [ |
4389 | + 'invalid_src', NetworkConfigSource.ds] |
4390 | + |
4391 | + self.assertEqual( |
4392 | + (ds_net_cfg, NetworkConfigSource.ds), |
4393 | + self.init._find_networking_config()) |
4394 | + self.assertIn('WARNING: data source specifies an invalid network' |
4395 | + ' cfg_source: invalid_src', |
4396 | + self.logs.getvalue()) |
4397 | + |
4398 | + @mock.patch('cloudinit.stages.cmdline.read_initramfs_config') |
4399 | @mock.patch('cloudinit.stages.cmdline.read_kernel_cmdline_config') |
4400 | - def test_wb__find_networking_config_returns_kernel(self, m_cmdline): |
4401 | + def test__find_networking_config_warns_if_datasrc_uses_unavailable_src( |
4402 | + self, m_cmdline, m_initramfs): |
4403 | + """find_networking_config should check sources in DS defined order""" |
4404 | + ds_net_cfg = {'config': {'needle': True}} |
4405 | + self.init.datasource = FakeDataSource(network_config=ds_net_cfg) |
4406 | + self.init.datasource.network_config_sources = [ |
4407 | + NetworkConfigSource.fallback, NetworkConfigSource.ds] |
4408 | + |
4409 | + self.assertEqual( |
4410 | + (ds_net_cfg, NetworkConfigSource.ds), |
4411 | + self.init._find_networking_config()) |
4412 | + self.assertIn('WARNING: data source specifies an unavailable network' |
4413 | + ' cfg_source: fallback', |
4414 | + self.logs.getvalue()) |
4415 | + |
4416 | + @mock.patch('cloudinit.stages.cmdline.read_initramfs_config') |
4417 | + @mock.patch('cloudinit.stages.cmdline.read_kernel_cmdline_config') |
4418 | + def test_wb__find_networking_config_returns_kernel( |
4419 | + self, m_cmdline, m_initramfs): |
4420 | """find_networking_config returns kernel cmdline config if present.""" |
4421 | expected_cfg = {'config': ['fakekernel']} |
4422 | m_cmdline.return_value = expected_cfg |
4423 | + m_initramfs.return_value = {'config': ['fake_initrd']} |
4424 | self.init._cfg = {'system_info': {'paths': {'cloud_dir': self.tmpdir}}, |
4425 | 'network': {'config': ['fakesys_config']}} |
4426 | self.init.datasource = FakeDataSource( |
4427 | network_config={'config': ['fakedatasource']}) |
4428 | self.assertEqual( |
4429 | - (expected_cfg, 'cmdline'), |
4430 | + (expected_cfg, NetworkConfigSource.cmdline), |
4431 | self.init._find_networking_config()) |
4432 | |
4433 | + @mock.patch('cloudinit.stages.cmdline.read_initramfs_config') |
4434 | @mock.patch('cloudinit.stages.cmdline.read_kernel_cmdline_config') |
4435 | - def test_wb__find_networking_config_returns_system_cfg(self, m_cmdline): |
4436 | + def test_wb__find_networking_config_returns_initramfs( |
4437 | + self, m_cmdline, m_initramfs): |
4438 | + """find_networking_config returns kernel cmdline config if present.""" |
4439 | + expected_cfg = {'config': ['fake_initrd']} |
4440 | + m_cmdline.return_value = {} |
4441 | + m_initramfs.return_value = expected_cfg |
4442 | + self.init._cfg = {'system_info': {'paths': {'cloud_dir': self.tmpdir}}, |
4443 | + 'network': {'config': ['fakesys_config']}} |
4444 | + self.init.datasource = FakeDataSource( |
4445 | + network_config={'config': ['fakedatasource']}) |
4446 | + self.assertEqual( |
4447 | + (expected_cfg, NetworkConfigSource.initramfs), |
4448 | + self.init._find_networking_config()) |
4449 | + |
4450 | + @mock.patch('cloudinit.stages.cmdline.read_initramfs_config') |
4451 | + @mock.patch('cloudinit.stages.cmdline.read_kernel_cmdline_config') |
4452 | + def test_wb__find_networking_config_returns_system_cfg( |
4453 | + self, m_cmdline, m_initramfs): |
4454 | """find_networking_config returns system config when present.""" |
4455 | m_cmdline.return_value = {} # No kernel network config |
4456 | + m_initramfs.return_value = {} # no initramfs network config |
4457 | expected_cfg = {'config': ['fakesys_config']} |
4458 | self.init._cfg = {'system_info': {'paths': {'cloud_dir': self.tmpdir}}, |
4459 | 'network': expected_cfg} |
4460 | self.init.datasource = FakeDataSource( |
4461 | network_config={'config': ['fakedatasource']}) |
4462 | self.assertEqual( |
4463 | - (expected_cfg, 'system_cfg'), |
4464 | + (expected_cfg, NetworkConfigSource.system_cfg), |
4465 | self.init._find_networking_config()) |
4466 | |
4467 | + @mock.patch('cloudinit.stages.cmdline.read_initramfs_config') |
4468 | @mock.patch('cloudinit.stages.cmdline.read_kernel_cmdline_config') |
4469 | - def test_wb__find_networking_config_returns_datasrc_cfg(self, m_cmdline): |
4470 | + def test_wb__find_networking_config_returns_datasrc_cfg( |
4471 | + self, m_cmdline, m_initramfs): |
4472 | """find_networking_config returns datasource net config if present.""" |
4473 | m_cmdline.return_value = {} # No kernel network config |
4474 | + m_initramfs.return_value = {} # no initramfs network config |
4475 | # No system config for network in setUp |
4476 | expected_cfg = {'config': ['fakedatasource']} |
4477 | self.init.datasource = FakeDataSource(network_config=expected_cfg) |
4478 | self.assertEqual( |
4479 | - (expected_cfg, 'ds'), |
4480 | + (expected_cfg, NetworkConfigSource.ds), |
4481 | self.init._find_networking_config()) |
4482 | |
4483 | + @mock.patch('cloudinit.stages.cmdline.read_initramfs_config') |
4484 | @mock.patch('cloudinit.stages.cmdline.read_kernel_cmdline_config') |
4485 | - def test_wb__find_networking_config_returns_fallback(self, m_cmdline): |
4486 | + def test_wb__find_networking_config_returns_fallback( |
4487 | + self, m_cmdline, m_initramfs): |
4488 | """find_networking_config returns fallback config if not defined.""" |
4489 | m_cmdline.return_value = {} # Kernel doesn't disable networking |
4490 | + m_initramfs.return_value = {} # no initramfs network config |
4491 | # Neither datasource nor system_info disable or provide network |
4492 | |
4493 | fake_cfg = {'config': [{'type': 'physical', 'name': 'eth9'}], |
4494 | @@ -147,7 +254,7 @@ class TestInit(CiTestCase): |
4495 | distro = self.init.distro |
4496 | distro.generate_fallback_config = fake_generate_fallback |
4497 | self.assertEqual( |
4498 | - (fake_cfg, 'fallback'), |
4499 | + (fake_cfg, NetworkConfigSource.fallback), |
4500 | self.init._find_networking_config()) |
4501 | self.assertNotIn('network config disabled', self.logs.getvalue()) |
4502 | |
4503 | @@ -166,8 +273,9 @@ class TestInit(CiTestCase): |
4504 | 'INFO: network config is disabled by %s' % disable_file, |
4505 | self.logs.getvalue()) |
4506 | |
4507 | + @mock.patch('cloudinit.net.get_interfaces_by_mac') |
4508 | @mock.patch('cloudinit.distros.ubuntu.Distro') |
4509 | - def test_apply_network_on_new_instance(self, m_ubuntu): |
4510 | + def test_apply_network_on_new_instance(self, m_ubuntu, m_macs): |
4511 | """Call distro apply_network_config methods on is_new_instance.""" |
4512 | net_cfg = { |
4513 | 'version': 1, 'config': [ |
4514 | @@ -175,7 +283,9 @@ class TestInit(CiTestCase): |
4515 | 'name': 'eth9', 'mac_address': '42:42:42:42:42:42'}]} |
4516 | |
4517 | def fake_network_config(): |
4518 | - return net_cfg, 'fallback' |
4519 | + return net_cfg, NetworkConfigSource.fallback |
4520 | + |
4521 | + m_macs.return_value = {'42:42:42:42:42:42': 'eth9'} |
4522 | |
4523 | self.init._find_networking_config = fake_network_config |
4524 | self.init.apply_network_config(True) |
4525 | @@ -195,7 +305,7 @@ class TestInit(CiTestCase): |
4526 | 'name': 'eth9', 'mac_address': '42:42:42:42:42:42'}]} |
4527 | |
4528 | def fake_network_config(): |
4529 | - return net_cfg, 'fallback' |
4530 | + return net_cfg, NetworkConfigSource.fallback |
4531 | |
4532 | self.init._find_networking_config = fake_network_config |
4533 | self.init.apply_network_config(True) |
4534 | @@ -206,8 +316,9 @@ class TestInit(CiTestCase): |
4535 | " nor datasource network update on '%s' event" % EventType.BOOT, |
4536 | self.logs.getvalue()) |
4537 | |
4538 | + @mock.patch('cloudinit.net.get_interfaces_by_mac') |
4539 | @mock.patch('cloudinit.distros.ubuntu.Distro') |
4540 | - def test_apply_network_on_datasource_allowed_event(self, m_ubuntu): |
4541 | + def test_apply_network_on_datasource_allowed_event(self, m_ubuntu, m_macs): |
4542 | """Apply network if datasource.update_metadata permits BOOT event.""" |
4543 | old_instance_id = os.path.join( |
4544 | self.init.paths.get_cpath('data'), 'instance-id') |
4545 | @@ -218,7 +329,9 @@ class TestInit(CiTestCase): |
4546 | 'name': 'eth9', 'mac_address': '42:42:42:42:42:42'}]} |
4547 | |
4548 | def fake_network_config(): |
4549 | - return net_cfg, 'fallback' |
4550 | + return net_cfg, NetworkConfigSource.fallback |
4551 | + |
4552 | + m_macs.return_value = {'42:42:42:42:42:42': 'eth9'} |
4553 | |
4554 | self.init._find_networking_config = fake_network_config |
4555 | self.init.datasource = FakeDataSource(paths=self.init.paths) |
4556 | diff --git a/cloudinit/url_helper.py b/cloudinit/url_helper.py |
4557 | index 0af0d9e..44ee61d 100644 |
4558 | --- a/cloudinit/url_helper.py |
4559 | +++ b/cloudinit/url_helper.py |
4560 | @@ -199,18 +199,19 @@ def _get_ssl_args(url, ssl_details): |
4561 | def readurl(url, data=None, timeout=None, retries=0, sec_between=1, |
4562 | headers=None, headers_cb=None, ssl_details=None, |
4563 | check_status=True, allow_redirects=True, exception_cb=None, |
4564 | - session=None, infinite=False, log_req_resp=True): |
4565 | + session=None, infinite=False, log_req_resp=True, |
4566 | + request_method=None): |
4567 | url = _cleanurl(url) |
4568 | req_args = { |
4569 | 'url': url, |
4570 | } |
4571 | req_args.update(_get_ssl_args(url, ssl_details)) |
4572 | req_args['allow_redirects'] = allow_redirects |
4573 | - req_args['method'] = 'GET' |
4574 | + if not request_method: |
4575 | + request_method = 'POST' if data else 'GET' |
4576 | + req_args['method'] = request_method |
4577 | if timeout is not None: |
4578 | req_args['timeout'] = max(float(timeout), 0) |
4579 | - if data: |
4580 | - req_args['method'] = 'POST' |
4581 | # It doesn't seem like config |
4582 | # was added in older library versions (or newer ones either), thus we |
4583 | # need to manually do the retries if it wasn't... |
4584 | diff --git a/cloudinit/version.py b/cloudinit/version.py |
4585 | index ddcd436..b04b11f 100644 |
4586 | --- a/cloudinit/version.py |
4587 | +++ b/cloudinit/version.py |
4588 | @@ -4,7 +4,7 @@ |
4589 | # |
4590 | # This file is part of cloud-init. See LICENSE file for license information. |
4591 | |
4592 | -__VERSION__ = "19.1" |
4593 | +__VERSION__ = "19.2" |
4594 | _PACKAGED_VERSION = '@@PACKAGED_VERSION@@' |
4595 | |
4596 | FEATURES = [ |
4597 | diff --git a/debian/changelog b/debian/changelog |
4598 | index 032f711..8ae019f 100644 |
4599 | --- a/debian/changelog |
4600 | +++ b/debian/changelog |
4601 | @@ -1,9 +1,66 @@ |
4602 | -cloud-init (19.1-1-gbaa47854-0ubuntu1~18.04.2) UNRELEASED; urgency=medium |
4603 | +cloud-init (19.2-21-ge6383719-0ubuntu1~18.04.1) bionic; urgency=medium |
4604 | |
4605 | * refresh patches: |
4606 | + debian/patches/ubuntu-advantage-revert-tip.patch |
4607 | - |
4608 | - -- Chad Smith <chad.smith@canonical.com> Tue, 04 Jun 2019 15:01:41 -0600 |
4609 | + * refresh patches: |
4610 | + + debian/patches/ubuntu-advantage-revert-tip.patch |
4611 | + * debian/cloud-init.templates: enable Exoscale cloud. |
4612 | + * New upstream snapshot. (LP: #1841099) |
4613 | + - ubuntu-drivers: call db_x_loadtemplatefile to accept NVIDIA EULA |
4614 | + - Add missing #cloud-config comment on first example in documentation. |
4615 | + [Florian Müller] |
4616 | + - ubuntu-drivers: emit latelink=true debconf to accept nvidia eula |
4617 | + - DataSourceOracle: prefer DS network config over initramfs |
4618 | + - format.rst: add text/jinja2 to list of content types (+ cleanups) |
4619 | + - Add GitHub pull request template to point people at hacking doc |
4620 | + - cloudinit/distros/parsers/sys_conf: add docstring to SysConf |
4621 | + - pyflakes: remove unused variable [Joshua Powers] |
4622 | + - Azure: Record boot timestamps, system information, and diagnostic events |
4623 | + [Anh Vo] |
4624 | + - DataSourceOracle: configure secondary NICs on Virtual Machines |
4625 | + - distros: fix confusing variable names |
4626 | + - azure/net: generate_fallback_nic emits network v2 config instead of v1 |
4627 | + - Add support for publishing host keys to GCE guest attributes |
4628 | + [Rick Wright] |
4629 | + - New data source for the Exoscale.com cloud platform [Chris Glass] |
4630 | + - doc: remove intersphinx extension |
4631 | + - cc_set_passwords: rewrite documentation |
4632 | + - net/cmdline: split interfaces_by_mac and init network config |
4633 | + determination |
4634 | + - stages: allow data sources to override network config source order |
4635 | + - cloud_tests: updates and fixes |
4636 | + - Fix bug rendering MTU on bond or vlan when input was netplan. |
4637 | + [Scott Moser] |
4638 | + - net: update net sequence, include wait on netdevs, opensuse netrules path |
4639 | + - Release 19.2 |
4640 | + - net: add rfc3442 (classless static routes) to EphemeralDHCP |
4641 | + - templates/ntp.conf.debian.tmpl: fix missing newline for pools |
4642 | + - Support netplan renderer in Arch Linux [Conrad Hoffmann] |
4643 | + - Fix typo in publicly viewable documentation. [David Medberry] |
4644 | + - Add a cdrom size checker for OVF ds to ds-identify [Pengpeng Sun] |
4645 | + - VMWare: Trigger the post customization script via cc_scripts module. |
4646 | + [Xiaofeng Wang] |
4647 | + - Cloud-init analyze module: Added ability to analyze boot events. |
4648 | + [Sam Gilson] |
4649 | + - Update debian eni network configuration location, retain Ubuntu setting |
4650 | + [Janos Lenart] |
4651 | + - net: skip bond interfaces in get_interfaces [Stanislav Makar] |
4652 | + - Fix a couple of issues raised by a coverity scan |
4653 | + - Add missing dsname for Hetzner Cloud datasource [Markus Schade] |
4654 | + - doc: indicate that netplan is default in Ubuntu now |
4655 | + - azure: add region and AZ properties from imds compute location metadata |
4656 | + - sysconfig: support more bonding options [Penghui Liao] |
4657 | + - cloud-init-generator: use libexec path to ds-identify on redhat systems |
4658 | + - tools/build-on-freebsd: update to python3 [Gonéri Le Bouder] |
4659 | + - Allow identification of OpenStack by Asset Tag [Mark T. Voelker] |
4660 | + - Fix spelling error making 'an Ubuntu' consistent. [Brian Murray] |
4661 | + - run-container: centos: comment out the repo mirrorlist [Paride Legovini] |
4662 | + - netplan: update netplan key mappings for gratuitous-arp |
4663 | + - freebsd: fix the name of cloudcfg VARIANT [Gonéri Le Bouder] |
4664 | + - freebsd: ability to grow root file system [Gonéri Le Bouder] |
4665 | + - freebsd: NoCloud data source support [Gonéri Le Bouder] |
4666 | + |
4667 | + -- Chad Smith <chad.smith@canonical.com> Thu, 22 Aug 2019 12:56:36 -0600 |
4668 | |
4669 | cloud-init (19.1-1-gbaa47854-0ubuntu1~18.04.1) bionic; urgency=medium |
4670 | |
4671 | diff --git a/debian/cloud-init.templates b/debian/cloud-init.templates |
4672 | index ef3c3a7..8d37ee5 100644 |
4673 | --- a/debian/cloud-init.templates |
4674 | +++ b/debian/cloud-init.templates |
4675 | @@ -1,8 +1,8 @@ |
4676 | Template: cloud-init/datasources |
4677 | Type: multiselect |
4678 | -Default: NoCloud, ConfigDrive, OpenNebula, DigitalOcean, Azure, AltCloud, OVF, MAAS, GCE, OpenStack, CloudSigma, SmartOS, Bigstep, Scaleway, AliYun, Ec2, CloudStack, Hetzner, IBMCloud, None |
4679 | -Choices-C: NoCloud, ConfigDrive, OpenNebula, DigitalOcean, Azure, AltCloud, OVF, MAAS, GCE, OpenStack, CloudSigma, SmartOS, Bigstep, Scaleway, AliYun, Ec2, CloudStack, Hetzner, IBMCloud, None |
4680 | -Choices: NoCloud: Reads info from /var/lib/cloud/seed only, ConfigDrive: Reads data from Openstack Config Drive, OpenNebula: read from OpenNebula context disk, DigitalOcean: reads data from Droplet datasource, Azure: read from MS Azure cdrom. Requires walinux-agent, AltCloud: config disks for RHEVm and vSphere, OVF: Reads data from OVF Transports, MAAS: Reads data from Ubuntu MAAS, GCE: google compute metadata service, OpenStack: native openstack metadata service, CloudSigma: metadata over serial for cloudsigma.com, SmartOS: Read from SmartOS metadata service, Bigstep: Bigstep metadata service, Scaleway: Scaleway metadata service, AliYun: Alibaba metadata service, Ec2: reads data from EC2 Metadata service, CloudStack: Read from CloudStack metadata service, Hetzner: Hetzner Cloud, IBMCloud: IBM Cloud. Previously softlayer or bluemix., None: Failsafe datasource |
4681 | +Default: NoCloud, ConfigDrive, OpenNebula, DigitalOcean, Azure, AltCloud, OVF, MAAS, GCE, OpenStack, CloudSigma, SmartOS, Bigstep, Scaleway, AliYun, Ec2, CloudStack, Hetzner, IBMCloud, Exoscale, None |
4682 | +Choices-C: NoCloud, ConfigDrive, OpenNebula, DigitalOcean, Azure, AltCloud, OVF, MAAS, GCE, OpenStack, CloudSigma, SmartOS, Bigstep, Scaleway, AliYun, Ec2, CloudStack, Hetzner, IBMCloud, Exoscale, None |
4683 | +Choices: NoCloud: Reads info from /var/lib/cloud/seed only, ConfigDrive: Reads data from Openstack Config Drive, OpenNebula: read from OpenNebula context disk, DigitalOcean: reads data from Droplet datasource, Azure: read from MS Azure cdrom. Requires walinux-agent, AltCloud: config disks for RHEVm and vSphere, OVF: Reads data from OVF Transports, MAAS: Reads data from Ubuntu MAAS, GCE: google compute metadata service, OpenStack: native openstack metadata service, CloudSigma: metadata over serial for cloudsigma.com, SmartOS: Read from SmartOS metadata service, Bigstep: Bigstep metadata service, Scaleway: Scaleway metadata service, AliYun: Alibaba metadata service, Ec2: reads data from EC2 Metadata service, CloudStack: Read from CloudStack metadata service, Hetzner: Hetzner Cloud, IBMCloud: IBM Cloud. Previously softlayer or bluemix., Exoscale: Exoscale metadata service, None: Failsafe datasource |
4684 | Description: Which data sources should be searched? |
4685 | Cloud-init supports searching different "Data Sources" for information |
4686 | that it uses to configure a cloud instance. |
4687 | diff --git a/debian/patches/ubuntu-advantage-revert-tip.patch b/debian/patches/ubuntu-advantage-revert-tip.patch |
4688 | index b956067..6d8b888 100644 |
4689 | --- a/debian/patches/ubuntu-advantage-revert-tip.patch |
4690 | +++ b/debian/patches/ubuntu-advantage-revert-tip.patch |
4691 | @@ -9,10 +9,8 @@ Forwarded: not-needed |
4692 | Last-Update: 2019-05-10 |
4693 | --- |
4694 | This patch header follows DEP-3: http://dep.debian.net/deps/dep3/ |
4695 | -Index: cloud-init/cloudinit/config/cc_ubuntu_advantage.py |
4696 | -=================================================================== |
4697 | ---- cloud-init.orig/cloudinit/config/cc_ubuntu_advantage.py |
4698 | -+++ cloud-init/cloudinit/config/cc_ubuntu_advantage.py |
4699 | +--- a/cloudinit/config/cc_ubuntu_advantage.py |
4700 | ++++ b/cloudinit/config/cc_ubuntu_advantage.py |
4701 | @@ -1,143 +1,150 @@ |
4702 | +# Copyright (C) 2018 Canonical Ltd. |
4703 | +# |
4704 | @@ -294,10 +292,8 @@ Index: cloud-init/cloudinit/config/cc_ubuntu_advantage.py |
4705 | + run_commands(cfgin.get('commands', [])) |
4706 | |
4707 | # vi: ts=4 expandtab |
4708 | -Index: cloud-init/cloudinit/config/tests/test_ubuntu_advantage.py |
4709 | -=================================================================== |
4710 | ---- cloud-init.orig/cloudinit/config/tests/test_ubuntu_advantage.py |
4711 | -+++ cloud-init/cloudinit/config/tests/test_ubuntu_advantage.py |
4712 | +--- a/cloudinit/config/tests/test_ubuntu_advantage.py |
4713 | ++++ b/cloudinit/config/tests/test_ubuntu_advantage.py |
4714 | @@ -1,7 +1,10 @@ |
4715 | # This file is part of cloud-init. See LICENSE file for license information. |
4716 | |
4717 | diff --git a/doc/examples/cloud-config-datasources.txt b/doc/examples/cloud-config-datasources.txt |
4718 | index 2651c02..52a2476 100644 |
4719 | --- a/doc/examples/cloud-config-datasources.txt |
4720 | +++ b/doc/examples/cloud-config-datasources.txt |
4721 | @@ -38,7 +38,7 @@ datasource: |
4722 | # these are optional, but allow you to basically provide a datasource |
4723 | # right here |
4724 | user-data: | |
4725 | - # This is the user-data verbatum |
4726 | + # This is the user-data verbatim |
4727 | meta-data: |
4728 | instance-id: i-87018aed |
4729 | local-hostname: myhost.internal |
4730 | diff --git a/doc/examples/cloud-config-user-groups.txt b/doc/examples/cloud-config-user-groups.txt |
4731 | index 6a363b7..f588bfb 100644 |
4732 | --- a/doc/examples/cloud-config-user-groups.txt |
4733 | +++ b/doc/examples/cloud-config-user-groups.txt |
4734 | @@ -1,3 +1,4 @@ |
4735 | +#cloud-config |
4736 | # Add groups to the system |
4737 | # The following example adds the ubuntu group with members 'root' and 'sys' |
4738 | # and the empty group cloud-users. |
4739 | diff --git a/doc/rtd/conf.py b/doc/rtd/conf.py |
4740 | index 50eb05c..4174477 100644 |
4741 | --- a/doc/rtd/conf.py |
4742 | +++ b/doc/rtd/conf.py |
4743 | @@ -27,16 +27,11 @@ project = 'Cloud-Init' |
4744 | # Add any Sphinx extension module names here, as strings. They can be |
4745 | # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom ones. |
4746 | extensions = [ |
4747 | - 'sphinx.ext.intersphinx', |
4748 | 'sphinx.ext.autodoc', |
4749 | 'sphinx.ext.autosectionlabel', |
4750 | 'sphinx.ext.viewcode', |
4751 | ] |
4752 | |
4753 | -intersphinx_mapping = { |
4754 | - 'sphinx': ('http://sphinx.pocoo.org', None) |
4755 | -} |
4756 | - |
4757 | # The suffix of source filenames. |
4758 | source_suffix = '.rst' |
4759 | |
4760 | diff --git a/doc/rtd/topics/analyze.rst b/doc/rtd/topics/analyze.rst |
4761 | new file mode 100644 |
4762 | index 0000000..5cf38bd |
4763 | --- /dev/null |
4764 | +++ b/doc/rtd/topics/analyze.rst |
4765 | @@ -0,0 +1,84 @@ |
4766 | +************************* |
4767 | +Cloud-init Analyze Module |
4768 | +************************* |
4769 | + |
4770 | +Overview |
4771 | +======== |
4772 | +The analyze module was added to cloud-init in order to help analyze cloud-init boot time |
4773 | +performance. It is loosely based on systemd-analyze where there are 4 main actions: |
4774 | +show, blame, dump, and boot. |
4775 | + |
4776 | +The 'show' action is similar to 'systemd-analyze critical-chain' which prints a list of units, the |
4777 | +time they started and how long they took. For cloud-init, we have four stages, and within each stage |
4778 | +a number of modules may run depending on configuration. ‘cloudinit-analyze show’ will, for each |
4779 | +boot, print this information and a summary total time, per boot. |
4780 | + |
4781 | +The 'blame' action matches 'systemd-analyze blame' where it prints, in descending order, |
4782 | +the units that took the longest to run. This output is highly useful for examining where cloud-init |
4783 | +is spending its time during execution. |
4784 | + |
4785 | +The 'dump' action simply dumps the cloud-init logs that the analyze module is performing |
4786 | +the analysis on and returns a list of dictionaries that can be consumed for other reporting needs. |
4787 | + |
4788 | +The 'boot' action prints out kernel related timestamps that are not included in any of the |
4789 | +cloud-init logs. There are three different timestamps that are presented to the user: |
4790 | +kernel start, kernel finish boot, and cloud-init start. This was added for additional |
4791 | +clarity into the boot process that cloud-init does not have control over, to aid in debugging of |
4792 | +performance issues related to cloudinit startup or tracking regression. |
4793 | + |
4794 | +Usage |
4795 | +===== |
4796 | +Using each of the printing formats is as easy as running one of the following bash commands: |
4797 | + |
4798 | +.. code-block:: shell-session |
4799 | + |
4800 | + cloud-init analyze show |
4801 | + cloud-init analyze blame |
4802 | + cloud-init analyze dump |
4803 | + cloud-init analyze boot |
4804 | + |
4805 | +Cloud-init analyze boot Timestamp Gathering |
4806 | +=========================================== |
4807 | +The following boot related timestamps are gathered on demand when cloud-init analyze boot runs: |
4808 | +- Kernel Startup, which is inferred from system uptime |
4809 | +- Kernel Finishes Initialization, which is inferred from systemd UserSpaceMonotonicTimestamp property |
4810 | +- Cloud-init activation, which is inferred from the property InactiveExitTimestamp of the cloud-init |
4811 | +local systemd unit. |
4812 | + |
4813 | +In order to gather the necessary timestamps using systemd, running the commands |
4814 | + |
4815 | +.. code-block:: shell-session |
4816 | + |
4817 | + systemctl show -p UserspaceTimestampMonotonic |
4818 | + systemctl show cloud-init-local -p InactiveExitTimestampMonotonic |
4819 | + |
4820 | +will gather the UserspaceTimestamp and InactiveExitTimestamp. |
4821 | +The UserspaceTimestamp tracks when the init system starts, which is used as an indicator of kernel |
4822 | +finishing initialization. The InactiveExitTimestamp tracks when a particular systemd unit transitions |
4823 | +from the Inactive to Active state, which can be used to mark the beginning of systemd's activation |
4824 | +of cloud-init. |
4825 | + |
4826 | +Currently this only works for distros that use systemd as the init process. We will be expanding |
4827 | +support for other distros in the future and this document will be updated accordingly. |
4828 | + |
4829 | +If systemd is not present on the system, dmesg is used to attempt to find an event that logs the |
4830 | +beginning of the init system. However, with this method only the first two timestamps are able to be found; |
4831 | +dmesg does not monitor userspace processes, so no cloud-init start timestamps are emitted like when |
4832 | +using systemd. |
4833 | + |
4834 | +List of Cloud-init analyze boot supported distros |
4835 | +================================================= |
4836 | +- Arch |
4837 | +- CentOS |
4838 | +- Debian |
4839 | +- Fedora |
4840 | +- OpenSuSE |
4841 | +- Red Hat Enterprise Linux |
4842 | +- Ubuntu |
4843 | +- SUSE Linux Enterprise Server |
4844 | +- CoreOS |
4845 | + |
4846 | +List of Cloud-init analyze boot unsupported distros |
4847 | +=================================================== |
4848 | +- FreeBSD |
4849 | +- Gentoo |
4850 | \ No newline at end of file |
4851 | diff --git a/doc/rtd/topics/capabilities.rst b/doc/rtd/topics/capabilities.rst |
4852 | index 0d8b894..6d85a99 100644 |
4853 | --- a/doc/rtd/topics/capabilities.rst |
4854 | +++ b/doc/rtd/topics/capabilities.rst |
4855 | @@ -217,6 +217,7 @@ Get detailed reports of where cloud-init spends most of its time. See |
4856 | * **dump** Machine-readable JSON dump of all cloud-init tracked events. |
4857 | * **show** show time-ordered report of the cost of operations during each |
4858 | boot stage. |
4859 | +* **boot** show timestamps from kernel initialization, kernel finish initialization, and cloud-init start. |
4860 | |
4861 | .. _cli_devel: |
4862 | |
4863 | diff --git a/doc/rtd/topics/datasources.rst b/doc/rtd/topics/datasources.rst |
4864 | index 648c606..2148cd5 100644 |
4865 | --- a/doc/rtd/topics/datasources.rst |
4866 | +++ b/doc/rtd/topics/datasources.rst |
4867 | @@ -155,6 +155,7 @@ Follow for more information. |
4868 | datasources/configdrive.rst |
4869 | datasources/digitalocean.rst |
4870 | datasources/ec2.rst |
4871 | + datasources/exoscale.rst |
4872 | datasources/maas.rst |
4873 | datasources/nocloud.rst |
4874 | datasources/opennebula.rst |
4875 | diff --git a/doc/rtd/topics/datasources/exoscale.rst b/doc/rtd/topics/datasources/exoscale.rst |
4876 | new file mode 100644 |
4877 | index 0000000..27aec9c |
4878 | --- /dev/null |
4879 | +++ b/doc/rtd/topics/datasources/exoscale.rst |
4880 | @@ -0,0 +1,68 @@ |
4881 | +.. _datasource_exoscale: |
4882 | + |
4883 | +Exoscale |
4884 | +======== |
4885 | + |
4886 | +This datasource supports reading from the metadata server used on the |
4887 | +`Exoscale platform <https://exoscale.com>`_. |
4888 | + |
4889 | +Use of the Exoscale datasource is recommended to benefit from new features of |
4890 | +the Exoscale platform. |
4891 | + |
4892 | +The datasource relies on the availability of a compatible metadata server |
4893 | +(``http://169.254.169.254`` is used by default) and its companion password |
4894 | +server, reachable at the same address (by default on port 8080). |
4895 | + |
4896 | +Crawling of metadata |
4897 | +-------------------- |
4898 | + |
4899 | +The metadata service and password server are crawled slightly differently: |
4900 | + |
4901 | + * The "metadata service" is crawled every boot. |
4902 | + * The password server is also crawled every boot (the Exoscale datasource |
4903 | + forces the password module to run with "frequency always"). |
4904 | + |
4905 | +In the password server case, the following rules apply in order to enable the |
4906 | +"restore instance password" functionality: |
4907 | + |
4908 | + * If a password is returned by the password server, it is then marked "saved" |
4909 | + by the cloud-init datasource. Subsequent boots will skip setting the password |
4910 | + (the password server will return "saved_password"). |
4911 | + * When the instance password is reset (via the Exoscale UI), the password |
4912 | + server will return the non-empty password at next boot, therefore causing |
4913 | + cloud-init to reset the instance's password. |
4914 | + |
4915 | +Configuration |
4916 | +------------- |
4917 | + |
4918 | +Users of this datasource are discouraged from changing the default settings |
4919 | +unless instructed to by Exoscale support. |
4920 | + |
4921 | +The following settings are available and can be set for the datasource in system |
4922 | +configuration (in `/etc/cloud/cloud.cfg.d/`). |
4923 | + |
4924 | +The settings available are: |
4925 | + |
4926 | + * **metadata_url**: The URL for the metadata service (defaults to |
4927 | + ``http://169.254.169.254``) |
4928 | + * **api_version**: The API version path on which to query the instance metadata |
4929 | + (defaults to ``1.0``) |
4930 | + * **password_server_port**: The port (on the metadata server) on which the |
4931 | + password server listens (defaults to ``8080``). |
4932 | + * **timeout**: the timeout value provided to urlopen for each individual http |
4933 | + request. (defaults to ``10``) |
4934 | + * **retries**: The number of retries that should be done for an http request |
4935 | + (defaults to ``6``) |
4936 | + |
4937 | + |
4938 | +An example configuration with the default values is provided below: |
4939 | + |
4940 | +.. sourcecode:: yaml |
4941 | + |
4942 | + datasource: |
4943 | + Exoscale: |
4944 | + metadata_url: "http://169.254.169.254" |
4945 | + api_version: "1.0" |
4946 | + password_server_port: 8080 |
4947 | + timeout: 10 |
4948 | + retries: 6 |
4949 | diff --git a/doc/rtd/topics/datasources/oracle.rst b/doc/rtd/topics/datasources/oracle.rst |
4950 | index f2383ce..98c4657 100644 |
4951 | --- a/doc/rtd/topics/datasources/oracle.rst |
4952 | +++ b/doc/rtd/topics/datasources/oracle.rst |
4953 | @@ -8,7 +8,7 @@ This datasource reads metadata, vendor-data and user-data from |
4954 | |
4955 | Oracle Platform |
4956 | --------------- |
4957 | -OCI provides bare metal and virtual machines. In both cases, |
4958 | +OCI provides bare metal and virtual machines. In both cases, |
4959 | the platform identifies itself via DMI data in the chassis asset tag |
4960 | with the string 'OracleCloud.com'. |
4961 | |
4962 | @@ -22,5 +22,28 @@ Cloud-init has a specific datasource for Oracle in order to: |
4963 | implementation. |
4964 | |
4965 | |
4966 | +Configuration |
4967 | +------------- |
4968 | + |
4969 | +The following configuration can be set for the datasource in system |
4970 | +configuration (in ``/etc/cloud/cloud.cfg`` or ``/etc/cloud/cloud.cfg.d/``). |
4971 | + |
4972 | +The settings that may be configured are: |
4973 | + |
4974 | +* **configure_secondary_nics**: A boolean, defaulting to False. If set |
4975 | + to True on an OCI Virtual Machine, cloud-init will fetch networking |
4976 | + metadata from Oracle's IMDS and use it to configure the non-primary |
4977 | + network interface controllers in the system. If set to True on an |
4978 | + OCI Bare Metal Machine, it will have no effect (though this may |
4979 | + change in the future). |
4980 | + |
4981 | +An example configuration with the default values is provided below: |
4982 | + |
4983 | +.. sourcecode:: yaml |
4984 | + |
4985 | + datasource: |
4986 | + Oracle: |
4987 | + configure_secondary_nics: false |
4988 | + |
4989 | .. _Oracle Compute Infrastructure: https://cloud.oracle.com/ |
4990 | .. vi: textwidth=78 |
4991 | diff --git a/doc/rtd/topics/debugging.rst b/doc/rtd/topics/debugging.rst |
4992 | index 51363ea..e13d915 100644 |
4993 | --- a/doc/rtd/topics/debugging.rst |
4994 | +++ b/doc/rtd/topics/debugging.rst |
4995 | @@ -68,6 +68,19 @@ subcommands default to reading /var/log/cloud-init.log. |
4996 | 00.00100s (modules-final/config-rightscale_userdata) |
4997 | ... |
4998 | |
4999 | +* ``analyze boot`` Make subprocess calls to the kernel in order to get relevant |
5000 | + pre-cloud-init timestamps, such as the kernel start, kernel finish boot, and cloud-init start. |
PASSED: Continuous integration, rev:5463fec28e7 9740fff2382c504 f94756a6eda6e2 /jenkins. ubuntu. com/server/ job/cloud- init-ci/ 1071/
https:/
Executed test runs:
SUCCESS: Checkout
SUCCESS: Unit & Style Tests
SUCCESS: Ubuntu LTS: Build
SUCCESS: Ubuntu LTS: Integration
IN_PROGRESS: Declarative: Post Actions
Click here to trigger a rebuild: /jenkins. ubuntu. com/server/ job/cloud- init-ci/ 1071//rebuild
https:/