Merge ~chad.smith/cloud-init:set-hostname-before-network into cloud-init:master

Proposed by Chad Smith
Status: Merged
Approved by: Scott Moser
Approved revision: 148b8ab1ae0a21bb9e58a94818a234d0aaaf3548
Merge reported by: Chad Smith
Merged at revision: 133ad2cb327ad17b7b81319fac8f9f14577c04df
Proposed branch: ~chad.smith/cloud-init:set-hostname-before-network
Merge into: cloud-init:master
Diff against target: 668 lines (+449/-22)
9 files modified
cloudinit/cloud.py (+3/-2)
cloudinit/cmd/main.py (+25/-0)
cloudinit/cmd/tests/test_main.py (+161/-0)
cloudinit/config/cc_set_hostname.py (+35/-6)
cloudinit/sources/__init__.py (+17/-4)
cloudinit/sources/tests/test_init.py (+69/-1)
cloudinit/tests/test_util.py (+74/-0)
cloudinit/util.py (+12/-5)
tests/unittests/test_handler/test_handler_set_hostname.py (+53/-4)
Reviewer Review Type Date Requested Status
Scott Moser Approve
Server Team CI bot continuous-integration Approve
Review via email: mp+339720@code.launchpad.net

Commit message

set_hostname: When present in metadata, set it before network bringup.

When instance meta-data provides hostname information, run
cc_set_hostname in the init-local or init-net stage before network
comes up.

Prevent an initial DHCP request which leaks the stock cloud-image default
hostname before the meta-data provided hostname was processed.
A leaked cloud-image hostname adversely affects Dynamic DNS which
would reallocate 'ubuntu' hostname in DNS to every instance brought up by
cloud-init. These instances would only update DNS to the cloud-init
configured hostname upon DHCP lease renewal.

This branch extends the get_hostname methods in datasource, cloud and
util to limit results to metadata_only to avoid extra cost of querying
the distro for hostname information if metadata does not provide that
information.

LP: #1746455

Description of the change

See commit message

To post a comment you must log in.
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

FAILED: Continuous integration, rev:a015a634f18594be2bcf78824eb64311d1d52845
https://jenkins.ubuntu.com/server/job/cloud-init-ci/790/
Executed test runs:
    SUCCESS: Checkout
    FAILED: Unit & Style Tests

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/790/rebuild

review: Needs Fixing (continuous-integration)
Revision history for this message
Scott Moser (smoser) :
Revision history for this message
Ryan Harper (raharper) wrote :

I've a bit of a quibble with the description. Mostly because cloud-init has _no_ control over how DNS or DHCP servers react to dhcp client requests coming from an instance.

What we (cloud-init) can do is set instance specified hostnames as _soon_ as we know it, which may be of help to some situations where Dynamic DNS via dhcp client hostname value.

This is an environment specific issue and having cloud-init set hostname as soon as possible does not ensure that there won't be issues with DNS, DHCP outside of cloud-init control.

Some inline comments too.

8211b97... by Chad Smith

lints and flake8

Revision history for this message
Server Team CI bot (server-team-bot) wrote :

PASSED: Continuous integration, rev:2baccbbf6c4421ef8e166e7a35f20be442d73578
https://jenkins.ubuntu.com/server/job/cloud-init-ci/793/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    SUCCESS: MAAS Compatability Testing
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/793/rebuild

review: Approve (continuous-integration)
Revision history for this message
Chad Smith (chad.smith) :
Revision history for this message
Chad Smith (chad.smith) :
Revision history for this message
Scott Moser (smoser) wrote :

My only comment at this point is wheter or not we should bother with calling the config module and writing the marker file and such at that point.

the other option is to just set it.

The other thoughts (i guess i did have more than one)
a.) we have had issues with setting hostname not being possible early in boot due to that being a systemd service (hostnamectl) .
b.) we want to somehow make sure that we only set this hostname on first instance boot. We don't want to overwrite what a user configured manually.

I think that your use of config module probably handles that, but just want to write those thoughts down.

Revision history for this message
Ryan Harper (raharper) wrote :

Sorry for the noise on the get_hostname part; I missed it getting sent through the datasource object.

6bd8875... by Chad Smith

SethostnameError is now raised by cc_set_hostname. Handle that exception in init-local main. Call cc_set_hostname if hostname changed in init-net.

37d5b61... by Chad Smith

UserDataProcess class now returns a tuple of (processed_messages, processing_errors). Datsources now requery userdata or vendordata on previous errors instead of using the cache.

227b353... by Chad Smith

flakes

Revision history for this message
Server Team CI bot (server-team-bot) wrote :

PASSED: Continuous integration, rev:2b8f6fa52c9f4dbd21a0f6fa572c6660c78f8972
https://jenkins.ubuntu.com/server/job/cloud-init-ci/798/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    SUCCESS: MAAS Compatability Testing
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/798/rebuild

review: Approve (continuous-integration)
Revision history for this message
Scott Moser (smoser) wrote :

hm...
Does UserDataProcessor.process() actually get part handlers and such called?

If so, we're absolutely changing their contract. Now they'd be running without a network.

I'm really sorry this is so hard. :-(

6ebb0d1... by Chad Smith

address review comments

cc8a6ec... by Chad Smith

revert UserDataProcess changes

92d62c6... by Chad Smith

don't call update in local stage as it tries to process userdata. Only react to metadata in local stage

Revision history for this message
Server Team CI bot (server-team-bot) wrote :

PASSED: Continuous integration, rev:254d7a22e42b2179f9a4450bcfe6feed056828ea
https://jenkins.ubuntu.com/server/job/cloud-init-ci/811/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    SUCCESS: MAAS Compatability Testing
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/811/rebuild

review: Approve (continuous-integration)
e51081d... by Chad Smith

unit tests for cc_set_hostname and get_hostname_fqdn

Revision history for this message
Server Team CI bot (server-team-bot) wrote :

PASSED: Continuous integration, rev:cf810a94cbfe3cb46b21e40f7c50d6a77ea22d0f
https://jenkins.ubuntu.com/server/job/cloud-init-ci/812/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    SUCCESS: MAAS Compatability Testing
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/812/rebuild

review: Approve (continuous-integration)
29fc8fe... by Chad Smith

add unit test for SetHostnameError

Revision history for this message
Server Team CI bot (server-team-bot) wrote :

PASSED: Continuous integration, rev:711843e2e24f279ec7dbe38fef89ea95305011fc
https://jenkins.ubuntu.com/server/job/cloud-init-ci/813/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    SUCCESS: MAAS Compatability Testing
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/813/rebuild

review: Approve (continuous-integration)
Revision history for this message
Scott Moser (smoser) wrote :

Some questions inline.
I think your commit message also needs updating to address the
fact that you're (I htink) only looking at meta-data, not user-data.
Specifically:
  "When instance metadata/user-data provides hostname information,"

review: Needs Fixing
4a3814d... by Chad Smith

address review: fixup incorrect docstring comments on metadata_only parameter, comment about different uses of set-hostname artifact file versus prevvious-hostname used by cc_update_hostname

d9ff4f2... by Chad Smith

add unit tests for main_init calling set_hostname on metadata

148b8ab... by Chad Smith

flakes

Revision history for this message
Server Team CI bot (server-team-bot) wrote :

PASSED: Continuous integration, rev:fe9ba4daf5997fcbe5d6b81a59becc5d95f3dfc9
https://jenkins.ubuntu.com/server/job/cloud-init-ci/844/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    SUCCESS: MAAS Compatability Testing
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/844/rebuild

review: Approve (continuous-integration)
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

PASSED: Continuous integration, rev:148b8ab1ae0a21bb9e58a94818a234d0aaaf3548
https://jenkins.ubuntu.com/server/job/cloud-init-ci/845/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    SUCCESS: MAAS Compatability Testing
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/845/rebuild

review: Approve (continuous-integration)
Revision history for this message
Scott Moser (smoser) wrote :

Approved. I'd like to get this into bionic tomorrow.

review: Approve
Revision history for this message
Chad Smith (chad.smith) wrote :

An upstream commit landed for this bug.

To view that commit see the following URL:
https://git.launchpad.net/cloud-init/commit/?id=133ad2cb

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
diff --git a/cloudinit/cloud.py b/cloudinit/cloud.py
index ba61678..6d12c43 100644
--- a/cloudinit/cloud.py
+++ b/cloudinit/cloud.py
@@ -78,8 +78,9 @@ class Cloud(object):
78 def get_locale(self):78 def get_locale(self):
79 return self.datasource.get_locale()79 return self.datasource.get_locale()
8080
81 def get_hostname(self, fqdn=False):81 def get_hostname(self, fqdn=False, metadata_only=False):
82 return self.datasource.get_hostname(fqdn=fqdn)82 return self.datasource.get_hostname(
83 fqdn=fqdn, metadata_only=metadata_only)
8384
84 def device_name_to_device(self, name):85 def device_name_to_device(self, name):
85 return self.datasource.device_name_to_device(name)86 return self.datasource.device_name_to_device(name)
diff --git a/cloudinit/cmd/main.py b/cloudinit/cmd/main.py
index fcddd75..3f2dbb9 100644
--- a/cloudinit/cmd/main.py
+++ b/cloudinit/cmd/main.py
@@ -40,6 +40,7 @@ from cloudinit.settings import (PER_INSTANCE, PER_ALWAYS, PER_ONCE,
4040
41from cloudinit import atomic_helper41from cloudinit import atomic_helper
4242
43from cloudinit.config import cc_set_hostname
43from cloudinit.dhclient_hook import LogDhclient44from cloudinit.dhclient_hook import LogDhclient
4445
4546
@@ -352,6 +353,11 @@ def main_init(name, args):
352 LOG.debug("[%s] %s will now be targeting instance id: %s. new=%s",353 LOG.debug("[%s] %s will now be targeting instance id: %s. new=%s",
353 mode, name, iid, init.is_new_instance())354 mode, name, iid, init.is_new_instance())
354355
356 if mode == sources.DSMODE_LOCAL:
357 # Before network comes up, set any configured hostname to allow
358 # dhcp clients to advertize this hostname to any DDNS services
359 # LP: #1746455.
360 _maybe_set_hostname(init, stage='local', retry_stage='network')
355 init.apply_network_config(bring_up=bool(mode != sources.DSMODE_LOCAL))361 init.apply_network_config(bring_up=bool(mode != sources.DSMODE_LOCAL))
356362
357 if mode == sources.DSMODE_LOCAL:363 if mode == sources.DSMODE_LOCAL:
@@ -368,6 +374,7 @@ def main_init(name, args):
368 init.setup_datasource()374 init.setup_datasource()
369 # update fully realizes user-data (pulling in #include if necessary)375 # update fully realizes user-data (pulling in #include if necessary)
370 init.update()376 init.update()
377 _maybe_set_hostname(init, stage='init-net', retry_stage='modules:config')
371 # Stage 7378 # Stage 7
372 try:379 try:
373 # Attempt to consume the data per instance.380 # Attempt to consume the data per instance.
@@ -681,6 +688,24 @@ def status_wrapper(name, args, data_d=None, link_d=None):
681 return len(v1[mode]['errors'])688 return len(v1[mode]['errors'])
682689
683690
691def _maybe_set_hostname(init, stage, retry_stage):
692 """Call set-hostname if metadata, vendordata or userdata provides it.
693
694 @param stage: String representing current stage in which we are running.
695 @param retry_stage: String represented logs upon error setting hostname.
696 """
697 cloud = init.cloudify()
698 (hostname, _fqdn) = util.get_hostname_fqdn(
699 init.cfg, cloud, metadata_only=True)
700 if hostname: # meta-data or user-data hostname content
701 try:
702 cc_set_hostname.handle('set-hostname', init.cfg, cloud, LOG, None)
703 except cc_set_hostname.SetHostnameError as e:
704 LOG.debug(
705 'Failed setting hostname in %s stage. Will'
706 ' retry in %s stage. Error: %s.', stage, retry_stage, str(e))
707
708
684def main_features(name, args):709def main_features(name, args):
685 sys.stdout.write('\n'.join(sorted(version.FEATURES)) + '\n')710 sys.stdout.write('\n'.join(sorted(version.FEATURES)) + '\n')
686711
diff --git a/cloudinit/cmd/tests/test_main.py b/cloudinit/cmd/tests/test_main.py
687new file mode 100644712new file mode 100644
index 0000000..dbe421c
--- /dev/null
+++ b/cloudinit/cmd/tests/test_main.py
@@ -0,0 +1,161 @@
1# This file is part of cloud-init. See LICENSE file for license information.
2
3from collections import namedtuple
4import copy
5import os
6from six import StringIO
7
8from cloudinit.cmd import main
9from cloudinit.util import (
10 ensure_dir, load_file, write_file, yaml_dumps)
11from cloudinit.tests.helpers import (
12 FilesystemMockingTestCase, wrap_and_call)
13
14mypaths = namedtuple('MyPaths', 'run_dir')
15myargs = namedtuple('MyArgs', 'debug files force local reporter subcommand')
16
17
18class TestMain(FilesystemMockingTestCase):
19
20 with_logs = True
21
22 def setUp(self):
23 super(TestMain, self).setUp()
24 self.new_root = self.tmp_dir()
25 self.cloud_dir = self.tmp_path('var/lib/cloud/', dir=self.new_root)
26 os.makedirs(self.cloud_dir)
27 self.replicateTestRoot('simple_ubuntu', self.new_root)
28 self.cfg = {
29 'datasource_list': ['None'],
30 'runcmd': ['ls /etc'], # test ALL_DISTROS
31 'system_info': {'paths': {'cloud_dir': self.cloud_dir,
32 'run_dir': self.new_root}},
33 'write_files': [
34 {
35 'path': '/etc/blah.ini',
36 'content': 'blah',
37 'permissions': 0o755,
38 },
39 ],
40 'cloud_init_modules': ['write-files', 'runcmd'],
41 }
42 cloud_cfg = yaml_dumps(self.cfg)
43 ensure_dir(os.path.join(self.new_root, 'etc', 'cloud'))
44 self.cloud_cfg_file = os.path.join(
45 self.new_root, 'etc', 'cloud', 'cloud.cfg')
46 write_file(self.cloud_cfg_file, cloud_cfg)
47 self.patchOS(self.new_root)
48 self.patchUtils(self.new_root)
49 self.stderr = StringIO()
50 self.patchStdoutAndStderr(stderr=self.stderr)
51
52 def test_main_init_run_net_stops_on_file_no_net(self):
53 """When no-net file is present, main_init does not process modules."""
54 stop_file = os.path.join(self.cloud_dir, 'data', 'no-net') # stop file
55 write_file(stop_file, '')
56 cmdargs = myargs(
57 debug=False, files=None, force=False, local=False, reporter=None,
58 subcommand='init')
59 (item1, item2) = wrap_and_call(
60 'cloudinit.cmd.main',
61 {'util.close_stdin': True,
62 'netinfo.debug_info': 'my net debug info',
63 'util.fixup_output': ('outfmt', 'errfmt')},
64 main.main_init, 'init', cmdargs)
65 # We should not run write_files module
66 self.assertFalse(
67 os.path.exists(os.path.join(self.new_root, 'etc/blah.ini')),
68 'Unexpected run of write_files module produced blah.ini')
69 self.assertEqual([], item2)
70 # Instancify is called
71 instance_id_path = 'var/lib/cloud/data/instance-id'
72 self.assertFalse(
73 os.path.exists(os.path.join(self.new_root, instance_id_path)),
74 'Unexpected call to datasource.instancify produced instance-id')
75 expected_logs = [
76 "Exiting. stop file ['{stop_file}'] existed\n".format(
77 stop_file=stop_file),
78 'my net debug info' # netinfo.debug_info
79 ]
80 for log in expected_logs:
81 self.assertIn(log, self.stderr.getvalue())
82
83 def test_main_init_run_net_runs_modules(self):
84 """Modules like write_files are run in 'net' mode."""
85 cmdargs = myargs(
86 debug=False, files=None, force=False, local=False, reporter=None,
87 subcommand='init')
88 (item1, item2) = wrap_and_call(
89 'cloudinit.cmd.main',
90 {'util.close_stdin': True,
91 'netinfo.debug_info': 'my net debug info',
92 'util.fixup_output': ('outfmt', 'errfmt')},
93 main.main_init, 'init', cmdargs)
94 self.assertEqual([], item2)
95 # Instancify is called
96 instance_id_path = 'var/lib/cloud/data/instance-id'
97 self.assertEqual(
98 'iid-datasource-none\n',
99 os.path.join(load_file(
100 os.path.join(self.new_root, instance_id_path))))
101 # modules are run (including write_files)
102 self.assertEqual(
103 'blah', load_file(os.path.join(self.new_root, 'etc/blah.ini')))
104 expected_logs = [
105 'network config is disabled by fallback', # apply_network_config
106 'my net debug info', # netinfo.debug_info
107 'no previous run detected'
108 ]
109 for log in expected_logs:
110 self.assertIn(log, self.stderr.getvalue())
111
112 def test_main_init_run_net_calls_set_hostname_when_metadata_present(self):
113 """When local-hostname metadata is present, call cc_set_hostname."""
114 self.cfg['datasource'] = {
115 'None': {'metadata': {'local-hostname': 'md-hostname'}}}
116 cloud_cfg = yaml_dumps(self.cfg)
117 write_file(self.cloud_cfg_file, cloud_cfg)
118 cmdargs = myargs(
119 debug=False, files=None, force=False, local=False, reporter=None,
120 subcommand='init')
121
122 def set_hostname(name, cfg, cloud, log, args):
123 self.assertEqual('set-hostname', name)
124 updated_cfg = copy.deepcopy(self.cfg)
125 updated_cfg.update(
126 {'def_log_file': '/var/log/cloud-init.log',
127 'log_cfgs': [],
128 'syslog_fix_perms': ['syslog:adm', 'root:adm', 'root:wheel'],
129 'vendor_data': {'enabled': True, 'prefix': []}})
130 updated_cfg.pop('system_info')
131
132 self.assertEqual(updated_cfg, cfg)
133 self.assertEqual(main.LOG, log)
134 self.assertIsNone(args)
135
136 (item1, item2) = wrap_and_call(
137 'cloudinit.cmd.main',
138 {'util.close_stdin': True,
139 'netinfo.debug_info': 'my net debug info',
140 'cc_set_hostname.handle': {'side_effect': set_hostname},
141 'util.fixup_output': ('outfmt', 'errfmt')},
142 main.main_init, 'init', cmdargs)
143 self.assertEqual([], item2)
144 # Instancify is called
145 instance_id_path = 'var/lib/cloud/data/instance-id'
146 self.assertEqual(
147 'iid-datasource-none\n',
148 os.path.join(load_file(
149 os.path.join(self.new_root, instance_id_path))))
150 # modules are run (including write_files)
151 self.assertEqual(
152 'blah', load_file(os.path.join(self.new_root, 'etc/blah.ini')))
153 expected_logs = [
154 'network config is disabled by fallback', # apply_network_config
155 'my net debug info', # netinfo.debug_info
156 'no previous run detected'
157 ]
158 for log in expected_logs:
159 self.assertIn(log, self.stderr.getvalue())
160
161# vi: ts=4 expandtab
diff --git a/cloudinit/config/cc_set_hostname.py b/cloudinit/config/cc_set_hostname.py
index aa3dfe5..3d2b2da 100644
--- a/cloudinit/config/cc_set_hostname.py
+++ b/cloudinit/config/cc_set_hostname.py
@@ -32,22 +32,51 @@ will be used.
32 hostname: <fqdn/hostname>32 hostname: <fqdn/hostname>
33"""33"""
3434
35import os
36
37
38from cloudinit.atomic_helper import write_json
35from cloudinit import util39from cloudinit import util
3640
3741
42class SetHostnameError(Exception):
43 """Raised when the distro runs into an exception when setting hostname.
44
45 This may happen if we attempt to set the hostname early in cloud-init's
46 init-local timeframe as certain services may not be running yet.
47 """
48 pass
49
50
38def handle(name, cfg, cloud, log, _args):51def handle(name, cfg, cloud, log, _args):
39 if util.get_cfg_option_bool(cfg, "preserve_hostname", False):52 if util.get_cfg_option_bool(cfg, "preserve_hostname", False):
40 log.debug(("Configuration option 'preserve_hostname' is set,"53 log.debug(("Configuration option 'preserve_hostname' is set,"
41 " not setting the hostname in module %s"), name)54 " not setting the hostname in module %s"), name)
42 return55 return
43
44 (hostname, fqdn) = util.get_hostname_fqdn(cfg, cloud)56 (hostname, fqdn) = util.get_hostname_fqdn(cfg, cloud)
57 # Check for previous successful invocation of set-hostname
58
59 # set-hostname artifact file accounts for both hostname and fqdn
60 # deltas. As such, it's format is different than cc_update_hostname's
61 # previous-hostname file which only contains the base hostname.
62 # TODO consolidate previous-hostname and set-hostname artifact files and
63 # distro._read_hostname implementation so we only validate one artifact.
64 prev_fn = os.path.join(cloud.get_cpath('data'), "set-hostname")
65 prev_hostname = {}
66 if os.path.exists(prev_fn):
67 prev_hostname = util.load_json(util.load_file(prev_fn))
68 hostname_changed = (hostname != prev_hostname.get('hostname') or
69 fqdn != prev_hostname.get('fqdn'))
70 if not hostname_changed:
71 log.debug('No hostname changes. Skipping set-hostname')
72 return
73 log.debug("Setting the hostname to %s (%s)", fqdn, hostname)
45 try:74 try:
46 log.debug("Setting the hostname to %s (%s)", fqdn, hostname)
47 cloud.distro.set_hostname(hostname, fqdn)75 cloud.distro.set_hostname(hostname, fqdn)
48 except Exception:76 except Exception as e:
49 util.logexc(log, "Failed to set the hostname to %s (%s)", fqdn,77 msg = "Failed to set the hostname to %s (%s)" % (fqdn, hostname)
50 hostname)78 util.logexc(log, msg)
51 raise79 raise SetHostnameError("%s: %s" % (msg, e))
80 write_json(prev_fn, {'hostname': hostname, 'fqdn': fqdn})
5281
53# vi: ts=4 expandtab82# vi: ts=4 expandtab
diff --git a/cloudinit/sources/__init__.py b/cloudinit/sources/__init__.py
index a05ca2f..df0b374 100644
--- a/cloudinit/sources/__init__.py
+++ b/cloudinit/sources/__init__.py
@@ -276,21 +276,34 @@ class DataSource(object):
276 return "iid-datasource"276 return "iid-datasource"
277 return str(self.metadata['instance-id'])277 return str(self.metadata['instance-id'])
278278
279 def get_hostname(self, fqdn=False, resolve_ip=False):279 def get_hostname(self, fqdn=False, resolve_ip=False, metadata_only=False):
280 """Get hostname or fqdn from the datasource. Look it up if desired.
281
282 @param fqdn: Boolean, set True to return hostname with domain.
283 @param resolve_ip: Boolean, set True to attempt to resolve an ipv4
284 address provided in local-hostname meta-data.
285 @param metadata_only: Boolean, set True to avoid looking up hostname
286 if meta-data doesn't have local-hostname present.
287
288 @return: hostname or qualified hostname. Optionally return None when
289 metadata_only is True and local-hostname data is not available.
290 """
280 defdomain = "localdomain"291 defdomain = "localdomain"
281 defhost = "localhost"292 defhost = "localhost"
282 domain = defdomain293 domain = defdomain
283294
284 if not self.metadata or 'local-hostname' not in self.metadata:295 if not self.metadata or 'local-hostname' not in self.metadata:
296 if metadata_only:
297 return None
285 # this is somewhat questionable really.298 # this is somewhat questionable really.
286 # the cloud datasource was asked for a hostname299 # the cloud datasource was asked for a hostname
287 # and didn't have one. raising error might be more appropriate300 # and didn't have one. raising error might be more appropriate
288 # but instead, basically look up the existing hostname301 # but instead, basically look up the existing hostname
289 toks = []302 toks = []
290 hostname = util.get_hostname()303 hostname = util.get_hostname()
291 fqdn = util.get_fqdn_from_hosts(hostname)304 hosts_fqdn = util.get_fqdn_from_hosts(hostname)
292 if fqdn and fqdn.find(".") > 0:305 if hosts_fqdn and hosts_fqdn.find(".") > 0:
293 toks = str(fqdn).split(".")306 toks = str(hosts_fqdn).split(".")
294 elif hostname and hostname.find(".") > 0:307 elif hostname and hostname.find(".") > 0:
295 toks = str(hostname).split(".")308 toks = str(hostname).split(".")
296 elif hostname:309 elif hostname:
diff --git a/cloudinit/sources/tests/test_init.py b/cloudinit/sources/tests/test_init.py
index af15115..5065083 100644
--- a/cloudinit/sources/tests/test_init.py
+++ b/cloudinit/sources/tests/test_init.py
@@ -7,7 +7,7 @@ import stat
7from cloudinit.helpers import Paths7from cloudinit.helpers import Paths
8from cloudinit.sources import (8from cloudinit.sources import (
9 INSTANCE_JSON_FILE, DataSource)9 INSTANCE_JSON_FILE, DataSource)
10from cloudinit.tests.helpers import CiTestCase, skipIf10from cloudinit.tests.helpers import CiTestCase, skipIf, mock
11from cloudinit.user_data import UserDataProcessor11from cloudinit.user_data import UserDataProcessor
12from cloudinit import util12from cloudinit import util
1313
@@ -108,6 +108,74 @@ class TestDataSource(CiTestCase):
108 self.assertEqual('userdata_raw', datasource.userdata_raw)108 self.assertEqual('userdata_raw', datasource.userdata_raw)
109 self.assertEqual('vendordata_raw', datasource.vendordata_raw)109 self.assertEqual('vendordata_raw', datasource.vendordata_raw)
110110
111 def test_get_hostname_strips_local_hostname_without_domain(self):
112 """Datasource.get_hostname strips metadata local-hostname of domain."""
113 tmp = self.tmp_dir()
114 datasource = DataSourceTestSubclassNet(
115 self.sys_cfg, self.distro, Paths({'run_dir': tmp}))
116 self.assertTrue(datasource.get_data())
117 self.assertEqual(
118 'test-subclass-hostname', datasource.metadata['local-hostname'])
119 self.assertEqual('test-subclass-hostname', datasource.get_hostname())
120 datasource.metadata['local-hostname'] = 'hostname.my.domain.com'
121 self.assertEqual('hostname', datasource.get_hostname())
122
123 def test_get_hostname_with_fqdn_returns_local_hostname_with_domain(self):
124 """Datasource.get_hostname with fqdn set gets qualified hostname."""
125 tmp = self.tmp_dir()
126 datasource = DataSourceTestSubclassNet(
127 self.sys_cfg, self.distro, Paths({'run_dir': tmp}))
128 self.assertTrue(datasource.get_data())
129 datasource.metadata['local-hostname'] = 'hostname.my.domain.com'
130 self.assertEqual(
131 'hostname.my.domain.com', datasource.get_hostname(fqdn=True))
132
133 def test_get_hostname_without_metadata_uses_system_hostname(self):
134 """Datasource.gethostname runs util.get_hostname when no metadata."""
135 tmp = self.tmp_dir()
136 datasource = DataSourceTestSubclassNet(
137 self.sys_cfg, self.distro, Paths({'run_dir': tmp}))
138 self.assertEqual({}, datasource.metadata)
139 mock_fqdn = 'cloudinit.sources.util.get_fqdn_from_hosts'
140 with mock.patch('cloudinit.sources.util.get_hostname') as m_gethost:
141 with mock.patch(mock_fqdn) as m_fqdn:
142 m_gethost.return_value = 'systemhostname.domain.com'
143 m_fqdn.return_value = None # No maching fqdn in /etc/hosts
144 self.assertEqual('systemhostname', datasource.get_hostname())
145 self.assertEqual(
146 'systemhostname.domain.com',
147 datasource.get_hostname(fqdn=True))
148
149 def test_get_hostname_without_metadata_returns_none(self):
150 """Datasource.gethostname returns None when metadata_only and no MD."""
151 tmp = self.tmp_dir()
152 datasource = DataSourceTestSubclassNet(
153 self.sys_cfg, self.distro, Paths({'run_dir': tmp}))
154 self.assertEqual({}, datasource.metadata)
155 mock_fqdn = 'cloudinit.sources.util.get_fqdn_from_hosts'
156 with mock.patch('cloudinit.sources.util.get_hostname') as m_gethost:
157 with mock.patch(mock_fqdn) as m_fqdn:
158 self.assertIsNone(datasource.get_hostname(metadata_only=True))
159 self.assertIsNone(
160 datasource.get_hostname(fqdn=True, metadata_only=True))
161 self.assertEqual([], m_gethost.call_args_list)
162 self.assertEqual([], m_fqdn.call_args_list)
163
164 def test_get_hostname_without_metadata_prefers_etc_hosts(self):
165 """Datasource.gethostname prefers /etc/hosts to util.get_hostname."""
166 tmp = self.tmp_dir()
167 datasource = DataSourceTestSubclassNet(
168 self.sys_cfg, self.distro, Paths({'run_dir': tmp}))
169 self.assertEqual({}, datasource.metadata)
170 mock_fqdn = 'cloudinit.sources.util.get_fqdn_from_hosts'
171 with mock.patch('cloudinit.sources.util.get_hostname') as m_gethost:
172 with mock.patch(mock_fqdn) as m_fqdn:
173 m_gethost.return_value = 'systemhostname.domain.com'
174 m_fqdn.return_value = 'fqdnhostname.domain.com'
175 self.assertEqual('fqdnhostname', datasource.get_hostname())
176 self.assertEqual('fqdnhostname.domain.com',
177 datasource.get_hostname(fqdn=True))
178
111 def test_get_data_write_json_instance_data(self):179 def test_get_data_write_json_instance_data(self):
112 """get_data writes INSTANCE_JSON_FILE to run_dir as readonly root."""180 """get_data writes INSTANCE_JSON_FILE to run_dir as readonly root."""
113 tmp = self.tmp_dir()181 tmp = self.tmp_dir()
diff --git a/cloudinit/tests/test_util.py b/cloudinit/tests/test_util.py
index c3e2e40..d30643d 100644
--- a/cloudinit/tests/test_util.py
+++ b/cloudinit/tests/test_util.py
@@ -16,6 +16,25 @@ MOUNT_INFO = [
16]16]
1717
1818
19class FakeCloud(object):
20
21 def __init__(self, hostname, fqdn):
22 self.hostname = hostname
23 self.fqdn = fqdn
24 self.calls = []
25
26 def get_hostname(self, fqdn=None, metadata_only=None):
27 myargs = {}
28 if fqdn is not None:
29 myargs['fqdn'] = fqdn
30 if metadata_only is not None:
31 myargs['metadata_only'] = metadata_only
32 self.calls.append(myargs)
33 if fqdn:
34 return self.fqdn
35 return self.hostname
36
37
19class TestUtil(CiTestCase):38class TestUtil(CiTestCase):
2039
21 def test_parse_mount_info_no_opts_no_arg(self):40 def test_parse_mount_info_no_opts_no_arg(self):
@@ -67,3 +86,58 @@ class TestShellify(CiTestCase):
67 "'echo' 'hi' 'sis'", ""]),86 "'echo' 'hi' 'sis'", ""]),
68 util.shellify(["echo hi mom", ["echo", "hi dad"],87 util.shellify(["echo hi mom", ["echo", "hi dad"],
69 ('echo', 'hi', 'sis')]))88 ('echo', 'hi', 'sis')]))
89
90
91class TestGetHostnameFqdn(CiTestCase):
92
93 def test_get_hostname_fqdn_from_only_cfg_fqdn(self):
94 """When cfg only has the fqdn key, derive hostname and fqdn from it."""
95 hostname, fqdn = util.get_hostname_fqdn(
96 cfg={'fqdn': 'myhost.domain.com'}, cloud=None)
97 self.assertEqual('myhost', hostname)
98 self.assertEqual('myhost.domain.com', fqdn)
99
100 def test_get_hostname_fqdn_from_cfg_fqdn_and_hostname(self):
101 """When cfg has both fqdn and hostname keys, return them."""
102 hostname, fqdn = util.get_hostname_fqdn(
103 cfg={'fqdn': 'myhost.domain.com', 'hostname': 'other'}, cloud=None)
104 self.assertEqual('other', hostname)
105 self.assertEqual('myhost.domain.com', fqdn)
106
107 def test_get_hostname_fqdn_from_cfg_hostname_with_domain(self):
108 """When cfg has only hostname key which represents a fqdn, use that."""
109 hostname, fqdn = util.get_hostname_fqdn(
110 cfg={'hostname': 'myhost.domain.com'}, cloud=None)
111 self.assertEqual('myhost', hostname)
112 self.assertEqual('myhost.domain.com', fqdn)
113
114 def test_get_hostname_fqdn_from_cfg_hostname_without_domain(self):
115 """When cfg has a hostname without a '.' query cloud.get_hostname."""
116 mycloud = FakeCloud('cloudhost', 'cloudhost.mycloud.com')
117 hostname, fqdn = util.get_hostname_fqdn(
118 cfg={'hostname': 'myhost'}, cloud=mycloud)
119 self.assertEqual('myhost', hostname)
120 self.assertEqual('cloudhost.mycloud.com', fqdn)
121 self.assertEqual(
122 [{'fqdn': True, 'metadata_only': False}], mycloud.calls)
123
124 def test_get_hostname_fqdn_from_without_fqdn_or_hostname(self):
125 """When cfg has neither hostname nor fqdn cloud.get_hostname."""
126 mycloud = FakeCloud('cloudhost', 'cloudhost.mycloud.com')
127 hostname, fqdn = util.get_hostname_fqdn(cfg={}, cloud=mycloud)
128 self.assertEqual('cloudhost', hostname)
129 self.assertEqual('cloudhost.mycloud.com', fqdn)
130 self.assertEqual(
131 [{'fqdn': True, 'metadata_only': False},
132 {'metadata_only': False}], mycloud.calls)
133
134 def test_get_hostname_fqdn_from_passes_metadata_only_to_cloud(self):
135 """Calls to cloud.get_hostname pass the metadata_only parameter."""
136 mycloud = FakeCloud('cloudhost', 'cloudhost.mycloud.com')
137 hostname, fqdn = util.get_hostname_fqdn(
138 cfg={}, cloud=mycloud, metadata_only=True)
139 self.assertEqual(
140 [{'fqdn': True, 'metadata_only': True},
141 {'metadata_only': True}], mycloud.calls)
142
143# vi: ts=4 expandtab
diff --git a/cloudinit/util.py b/cloudinit/util.py
index 083a8ef..4504f05 100644
--- a/cloudinit/util.py
+++ b/cloudinit/util.py
@@ -1025,9 +1025,16 @@ def dos2unix(contents):
1025 return contents.replace('\r\n', '\n')1025 return contents.replace('\r\n', '\n')
10261026
10271027
1028def get_hostname_fqdn(cfg, cloud):1028def get_hostname_fqdn(cfg, cloud, metadata_only=False):
1029 # return the hostname and fqdn from 'cfg'. If not found in cfg,1029 """Get hostname and fqdn from config if present and fallback to cloud.
1030 # then fall back to data from cloud1030
1031 @param cfg: Dictionary of merged user-data configuration (from init.cfg).
1032 @param cloud: Cloud instance from init.cloudify().
1033 @param metadata_only: Boolean, set True to only query cloud meta-data,
1034 returning None if not present in meta-data.
1035 @return: a Tuple of strings <hostname>, <fqdn>. Values can be none when
1036 metadata_only is True and no cfg or metadata provides hostname info.
1037 """
1031 if "fqdn" in cfg:1038 if "fqdn" in cfg:
1032 # user specified a fqdn. Default hostname then is based off that1039 # user specified a fqdn. Default hostname then is based off that
1033 fqdn = cfg['fqdn']1040 fqdn = cfg['fqdn']
@@ -1041,11 +1048,11 @@ def get_hostname_fqdn(cfg, cloud):
1041 else:1048 else:
1042 # no fqdn set, get fqdn from cloud.1049 # no fqdn set, get fqdn from cloud.
1043 # get hostname from cfg if available otherwise cloud1050 # get hostname from cfg if available otherwise cloud
1044 fqdn = cloud.get_hostname(fqdn=True)1051 fqdn = cloud.get_hostname(fqdn=True, metadata_only=metadata_only)
1045 if "hostname" in cfg:1052 if "hostname" in cfg:
1046 hostname = cfg['hostname']1053 hostname = cfg['hostname']
1047 else:1054 else:
1048 hostname = cloud.get_hostname()1055 hostname = cloud.get_hostname(metadata_only=metadata_only)
1049 return (hostname, fqdn)1056 return (hostname, fqdn)
10501057
10511058
diff --git a/tests/unittests/test_handler/test_handler_set_hostname.py b/tests/unittests/test_handler/test_handler_set_hostname.py
index abdc17e..d09ec23 100644
--- a/tests/unittests/test_handler/test_handler_set_hostname.py
+++ b/tests/unittests/test_handler/test_handler_set_hostname.py
@@ -11,6 +11,7 @@ from cloudinit.tests import helpers as t_help
1111
12from configobj import ConfigObj12from configobj import ConfigObj
13import logging13import logging
14import os
14import shutil15import shutil
15from six import BytesIO16from six import BytesIO
16import tempfile17import tempfile
@@ -19,14 +20,18 @@ LOG = logging.getLogger(__name__)
1920
2021
21class TestHostname(t_help.FilesystemMockingTestCase):22class TestHostname(t_help.FilesystemMockingTestCase):
23
24 with_logs = True
25
22 def setUp(self):26 def setUp(self):
23 super(TestHostname, self).setUp()27 super(TestHostname, self).setUp()
24 self.tmp = tempfile.mkdtemp()28 self.tmp = tempfile.mkdtemp()
29 util.ensure_dir(os.path.join(self.tmp, 'data'))
25 self.addCleanup(shutil.rmtree, self.tmp)30 self.addCleanup(shutil.rmtree, self.tmp)
2631
27 def _fetch_distro(self, kind):32 def _fetch_distro(self, kind):
28 cls = distros.fetch(kind)33 cls = distros.fetch(kind)
29 paths = helpers.Paths({})34 paths = helpers.Paths({'cloud_dir': self.tmp})
30 return cls(kind, {}, paths)35 return cls(kind, {}, paths)
3136
32 def test_write_hostname_rhel(self):37 def test_write_hostname_rhel(self):
@@ -34,7 +39,7 @@ class TestHostname(t_help.FilesystemMockingTestCase):
34 'hostname': 'blah.blah.blah.yahoo.com',39 'hostname': 'blah.blah.blah.yahoo.com',
35 }40 }
36 distro = self._fetch_distro('rhel')41 distro = self._fetch_distro('rhel')
37 paths = helpers.Paths({})42 paths = helpers.Paths({'cloud_dir': self.tmp})
38 ds = None43 ds = None
39 cc = cloud.Cloud(ds, paths, {}, distro, None)44 cc = cloud.Cloud(ds, paths, {}, distro, None)
40 self.patchUtils(self.tmp)45 self.patchUtils(self.tmp)
@@ -51,7 +56,7 @@ class TestHostname(t_help.FilesystemMockingTestCase):
51 'hostname': 'blah.blah.blah.yahoo.com',56 'hostname': 'blah.blah.blah.yahoo.com',
52 }57 }
53 distro = self._fetch_distro('debian')58 distro = self._fetch_distro('debian')
54 paths = helpers.Paths({})59 paths = helpers.Paths({'cloud_dir': self.tmp})
55 ds = None60 ds = None
56 cc = cloud.Cloud(ds, paths, {}, distro, None)61 cc = cloud.Cloud(ds, paths, {}, distro, None)
57 self.patchUtils(self.tmp)62 self.patchUtils(self.tmp)
@@ -65,7 +70,7 @@ class TestHostname(t_help.FilesystemMockingTestCase):
65 'hostname': 'blah.blah.blah.suse.com',70 'hostname': 'blah.blah.blah.suse.com',
66 }71 }
67 distro = self._fetch_distro('sles')72 distro = self._fetch_distro('sles')
68 paths = helpers.Paths({})73 paths = helpers.Paths({'cloud_dir': self.tmp})
69 ds = None74 ds = None
70 cc = cloud.Cloud(ds, paths, {}, distro, None)75 cc = cloud.Cloud(ds, paths, {}, distro, None)
71 self.patchUtils(self.tmp)76 self.patchUtils(self.tmp)
@@ -74,4 +79,48 @@ class TestHostname(t_help.FilesystemMockingTestCase):
74 contents = util.load_file(distro.hostname_conf_fn)79 contents = util.load_file(distro.hostname_conf_fn)
75 self.assertEqual('blah', contents.strip())80 self.assertEqual('blah', contents.strip())
7681
82 def test_multiple_calls_skips_unchanged_hostname(self):
83 """Only new hostname or fqdn values will generate a hostname call."""
84 distro = self._fetch_distro('debian')
85 paths = helpers.Paths({'cloud_dir': self.tmp})
86 ds = None
87 cc = cloud.Cloud(ds, paths, {}, distro, None)
88 self.patchUtils(self.tmp)
89 cc_set_hostname.handle(
90 'cc_set_hostname', {'hostname': 'hostname1.me.com'}, cc, LOG, [])
91 contents = util.load_file("/etc/hostname")
92 self.assertEqual('hostname1', contents.strip())
93 cc_set_hostname.handle(
94 'cc_set_hostname', {'hostname': 'hostname1.me.com'}, cc, LOG, [])
95 self.assertIn(
96 'DEBUG: No hostname changes. Skipping set-hostname\n',
97 self.logs.getvalue())
98 cc_set_hostname.handle(
99 'cc_set_hostname', {'hostname': 'hostname2.me.com'}, cc, LOG, [])
100 contents = util.load_file("/etc/hostname")
101 self.assertEqual('hostname2', contents.strip())
102 self.assertIn(
103 'Non-persistently setting the system hostname to hostname2',
104 self.logs.getvalue())
105
106 def test_error_on_distro_set_hostname_errors(self):
107 """Raise SetHostnameError on exceptions from distro.set_hostname."""
108 distro = self._fetch_distro('debian')
109
110 def set_hostname_error(hostname, fqdn):
111 raise Exception("OOPS on: %s" % fqdn)
112
113 distro.set_hostname = set_hostname_error
114 paths = helpers.Paths({'cloud_dir': self.tmp})
115 ds = None
116 cc = cloud.Cloud(ds, paths, {}, distro, None)
117 self.patchUtils(self.tmp)
118 with self.assertRaises(cc_set_hostname.SetHostnameError) as ctx_mgr:
119 cc_set_hostname.handle(
120 'somename', {'hostname': 'hostname1.me.com'}, cc, LOG, [])
121 self.assertEqual(
122 'Failed to set the hostname to hostname1.me.com (hostname1):'
123 ' OOPS on: hostname1.me.com',
124 str(ctx_mgr.exception))
125
77# vi: ts=4 expandtab126# vi: ts=4 expandtab

Subscribers

People subscribed via source and target branches