cloud-init

Merge ~raharper/cloud-init:feature/update-ntp-spec into cloud-init:master

  1. Git
  2. lp:~raharper/cloud-init
  3. feature/update-ntp-spec
  4. Merge into master
Proposed by Ryan Harper
Status: Merged
Approved by: Scott Moser
Approved revision: 30904e28f1d1b4df41fcf81b0fcfbd491bf8f7e7
Merge reported by: Scott Moser
Merged at revision: c6dff581a9c253170d5e3f12fb83d16a8dec8257
Proposed branch: ~raharper/cloud-init:feature/update-ntp-spec
Merge into: cloud-init:master
Diff against target: 2162 lines (+1369/-381)
21 files modified
cloudinit/config/cc_ntp.py (+407/-78)
cloudinit/distros/__init__.py (+12/-0)
cloudinit/distros/opensuse.py (+24/-0)
cloudinit/distros/ubuntu.py (+19/-0)
config/cloud.cfg.tmpl (+2/-0)
templates/chrony.conf.debian.tmpl (+39/-0)
templates/chrony.conf.fedora.tmpl (+48/-0)
templates/chrony.conf.opensuse.tmpl (+38/-0)
templates/chrony.conf.rhel.tmpl (+45/-0)
templates/chrony.conf.sles.tmpl (+38/-0)
templates/chrony.conf.ubuntu.tmpl (+42/-0)
tests/cloud_tests/testcases/modules/ntp.yaml (+1/-0)
tests/cloud_tests/testcases/modules/ntp_chrony.py (+15/-0)
tests/cloud_tests/testcases/modules/ntp_chrony.yaml (+17/-0)
tests/cloud_tests/testcases/modules/ntp_pools.yaml (+1/-0)
tests/cloud_tests/testcases/modules/ntp_servers.yaml (+1/-0)
tests/cloud_tests/testcases/modules/ntp_timesyncd.py (+15/-0)
tests/cloud_tests/testcases/modules/ntp_timesyncd.yaml (+15/-0)
tests/unittests/test_distros/test_netconfig.py (+6/-0)
tests/unittests/test_distros/test_user_data_normalize.py (+6/-0)
tests/unittests/test_handler/test_handler_ntp.py (+578/-303)
Reviewer Review Type Date Requested Status
Server Team CI bot continuous-integration Approve
Scott Moser Approve
Chad Smith Approve
Review via email: mp+339438@code.launchpad.net

Commit message

Implement ntp client spec with auto support for distro selection

Add a base NTP client configuration dictionary and allow Distro
specific changes to be merged. Add a select client function which
implements logic to preferr installed clients over clients which
need to be installed. Also allow distributions to override the
cloud-init defaults.

LP: #1749722

Description of the change

Implement ntp client spec with auto support for distro selection

Add a base NTP client configuration dictionary and allow Distro
specific changes to be merged. Add a select client function which
implements logic to preferr installed clients over clients which
need to be installed. Also allow distributions to override the
cloud-init defaults.

LP: #1749722

To post a comment you must log in.
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

FAILED: Continuous integration, rev:dad6bd58753bd7af4b4eacea08af818f07bd4284
https://jenkins.ubuntu.com/server/job/cloud-init-ci/791/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    FAILED: Ubuntu LTS: Integration

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/791/rebuild

review: Needs Fixing (continuous-integration)
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

FAILED: Continuous integration, rev:51bb59d9f1fde767b2edeefb5f68cdd9c6820189
https://jenkins.ubuntu.com/server/job/cloud-init-ci/792/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    FAILED: Ubuntu LTS: Integration

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/792/rebuild

review: Needs Fixing (continuous-integration)
Revision history for this message
Scott Moser (smoser) wrote :

fix descriptions where they are "".

I comments in line. I'm not sure if integration with the distro is beneficial or not compared to just knowing about distros in the ntp module itself.

Revision history for this message
Ryan Harper (raharper) :
Revision history for this message
Scott Moser (smoser) :
Revision history for this message
Chad Smith (chad.smith) wrote :

minor inline comments, I'll have a pastebin of the updated schema dictionary as I see it shaping up.

Revision history for this message
Chad Smith (chad.smith) wrote :

More minor inline comments and updated schema @ http://paste.ubuntu.com/p/2by6yyn8dT/

Revision history for this message
Ryan Harper (raharper) wrote :

OK, I think I've addressed most of the comments here, except I still need to:

 - switch to temp_utils.mkstemp() for the custom template; I didn't want to fight temp_utils for the unittest, but it's the Right Thing(TM) so that's up next.

I think the current commit log will show you what things I did address, so please look that over and see if I missed anything.

Revision history for this message
Server Team CI bot (server-team-bot) wrote :

FAILED: Continuous integration, rev:6a893344a4a328a5cf565d9d780f9baf677999b0
https://jenkins.ubuntu.com/server/job/cloud-init-ci/931/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    FAILED: Ubuntu LTS: Integration

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/931/rebuild

review: Needs Fixing (continuous-integration)
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

FAILED: Continuous integration, rev:adf6244a6a0403e6204f8536aab4e9278a814e25
https://jenkins.ubuntu.com/server/job/cloud-init-ci/938/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    FAILED: Ubuntu LTS: Integration

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/938/rebuild

review: Needs Fixing (continuous-integration)
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

FAILED: Continuous integration, rev:59349c84f49de903be13db67dc71d6871dce9bfb
https://jenkins.ubuntu.com/server/job/cloud-init-ci/943/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    FAILED: MAAS Compatability Testing

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/943/rebuild

review: Needs Fixing (continuous-integration)
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

FAILED: Continuous integration, rev:7f1fbcf0ad85e918939b22f30236ef2a296c8afe
https://jenkins.ubuntu.com/server/job/cloud-init-ci/944/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    FAILED: MAAS Compatability Testing

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/944/rebuild

review: Needs Fixing (continuous-integration)
Revision history for this message
Ryan Harper (raharper) wrote :

Most of this has already been discussed, but saving my comments for history.

Revision history for this message
Server Team CI bot (server-team-bot) wrote :

FAILED: Continuous integration, rev:5086bd7ac25ac0975f9fbfbfa7eabb0e372f75b0
https://jenkins.ubuntu.com/server/job/cloud-init-ci/948/
Executed test runs:
    SUCCESS: Checkout
    FAILED: Unit & Style Tests

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/948/rebuild

review: Needs Fixing (continuous-integration)
Revision history for this message
Ryan Harper (raharper) wrote :

The branch should pass CI now, updates cloud-ci ntp test-cases to specify the ntp client, added tests for timesyncd and chrony.

I've a couple in-line comments that I think need some discussion before we can conclude the branch is ready.

Revision history for this message
Server Team CI bot (server-team-bot) wrote :

PASSED: Continuous integration, rev:8ffcc66c74e2e7a622495575839b486f43a0222d
https://jenkins.ubuntu.com/server/job/cloud-init-ci/949/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    SUCCESS: MAAS Compatability Testing
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/949/rebuild

review: Approve (continuous-integration)
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

FAILED: Continuous integration, rev:b94f8d7e2b3df1f72b2da69ccad1e115e260819a
https://jenkins.ubuntu.com/server/job/cloud-init-ci/957/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    FAILED: Ubuntu LTS: Build

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/957/rebuild

review: Needs Fixing (continuous-integration)
Revision history for this message
Ryan Harper (raharper) wrote :

Refactored all logic into cc_ntp module deferring only to distros for a preferred ordering of ntp clients.

This passes all unit/ci-cloud tests locally. Next step is testing it in different datasources/clouds.

Revision history for this message
Server Team CI bot (server-team-bot) wrote :

FAILED: Continuous integration, rev:3d28f5286612316cc210d23a59604e71d294d03d
https://jenkins.ubuntu.com/server/job/cloud-init-ci/959/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    FAILED: Ubuntu LTS: Build

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/959/rebuild

review: Needs Fixing (continuous-integration)
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

PASSED: Continuous integration, rev:ce33a01512109e6a08cabdba1df3373922caf4bc
https://jenkins.ubuntu.com/server/job/cloud-init-ci/960/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    SUCCESS: MAAS Compatability Testing
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/960/rebuild

review: Approve (continuous-integration)
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

PASSED: Continuous integration, rev:2230bd0046a795da56e3958b761dc4c6472a4d35
https://jenkins.ubuntu.com/server/job/cloud-init-ci/961/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    SUCCESS: MAAS Compatability Testing
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/961/rebuild

review: Approve (continuous-integration)
Revision history for this message
Chad Smith (chad.smith) :
Revision history for this message
Chad Smith (chad.smith) :
Revision history for this message
Ryan Harper (raharper) wrote :

Thanks for the review, Ill fix what you pointed out, check what docs look like, drop the zfs fix and rebase to master.

Revision history for this message
Chad Smith (chad.smith) wrote :

round 2 inline comments. please rebase too to reduce the diff delta with zfs changes which have landed.

Revision history for this message
Server Team CI bot (server-team-bot) wrote :

PASSED: Continuous integration, rev:d9fde55e463da3ada89bc3c7c76a4cea7f6fa927
https://jenkins.ubuntu.com/server/job/cloud-init-ci/964/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    SUCCESS: MAAS Compatability Testing
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/964/rebuild

review: Approve (continuous-integration)
Revision history for this message
Ryan Harper (raharper) wrote :

Some feedback, I'll pick up some additional suggestions around cloud-test

Revision history for this message
Chad Smith (chad.smith) :
Revision history for this message
Ryan Harper (raharper) wrote :

Will grab some of these as well, open questions inline.

Revision history for this message
Server Team CI bot (server-team-bot) wrote :

FAILED: Continuous integration, rev:2dbc5d8638ff717b6a0eb891ac2b804fd1c2aed2
https://jenkins.ubuntu.com/server/job/cloud-init-ci/965/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    FAILED: Ubuntu LTS: Integration

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/965/rebuild

review: Needs Fixing (continuous-integration)
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

FAILED: Continuous integration, rev:2dbc5d8638ff717b6a0eb891ac2b804fd1c2aed2
https://jenkins.ubuntu.com/server/job/cloud-init-ci/966/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    FAILED: Ubuntu LTS: Integration

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/966/rebuild

review: Needs Fixing (continuous-integration)
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

FAILED: Continuous integration, rev:2dbc5d8638ff717b6a0eb891ac2b804fd1c2aed2
https://jenkins.ubuntu.com/server/job/cloud-init-ci/974/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    FAILED: Ubuntu LTS: Integration

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/974/rebuild

review: Needs Fixing (continuous-integration)
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

PASSED: Continuous integration, rev:815ce15352f26e752386b0738f707af29d589623
https://jenkins.ubuntu.com/server/job/cloud-init-ci/976/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    SUCCESS: MAAS Compatability Testing
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/976/rebuild

review: Approve (continuous-integration)
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

FAILED: Continuous integration, rev:9162c2e0a4040607af4cce666ebe94970c7337e4
https://jenkins.ubuntu.com/server/job/cloud-init-ci/978/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    FAILED: MAAS Compatability Testing

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/978/rebuild

review: Needs Fixing (continuous-integration)
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

FAILED: Continuous integration, rev:46ae12701ae617ff92a83d17bc1fee5cafcd9316
https://jenkins.ubuntu.com/server/job/cloud-init-ci/981/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    FAILED: MAAS Compatability Testing

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/981/rebuild

review: Needs Fixing (continuous-integration)
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

PASSED: Continuous integration, rev:0f45961973a9778db7e32c07682de5d5cebcb87e
https://jenkins.ubuntu.com/server/job/cloud-init-ci/982/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    SUCCESS: MAAS Compatability Testing
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/982/rebuild

review: Approve (continuous-integration)
Revision history for this message
Chad Smith (chad.smith) wrote :

I feel like we could do a bit better validation on the user-provided config keys with a simple validate_config function that'd raise understandable errors on misconfig for certain keys:

Here are some examples of easy and hard tracebacks to understand, as the person can't just run cloud-init status --long to see the error message, they have to walk through /var/log/cloud-init.log and read through a traceback stack to get at what may be the problem.

What do you think about a bit more validation on required config values?

https://pastebin.ubuntu.com/p/4X9TJg3NSz/

Revision history for this message
Ryan Harper (raharper) wrote :

If you've got some changes to make the validation check tighter, I'm happy to have those; and I agree that if status --long can emit exactly the error that's really nice.

Revision history for this message
Scott Moser (smoser) wrote :

Update the commit message on the merge proposal tomake sure it is in sync
with the current code.

- I think the '_set_preferred_ntp_client' client is kind of wierd.
  could we get the same function with an attribute ?
  Then it would only be called if used. Rather than being called on init.

- 'handle' in cc_ntp is longer than i'd like. the less that function
  does the easier things are to test. but I wont insist on that.

Revision history for this message
Chad Smith (chad.smith) wrote :

> If you've got some changes to make the validation check tighter, I'm happy to
> have those; and I agree that if status --long can emit exactly the error
> that's really nice.

Here's some tested code I ran on lxc's with sparse/incorrect user-data. it tightens up the error reporting so cloud-init status --long gives you an actionable traceback.
http://paste.ubuntu.com/p/dcKcYrM5ty/

Revision history for this message
Chad Smith (chad.smith) wrote :

oops lints/flakes fixed http://paste.ubuntu.com/p/5Zv2WTX9Rf/

Revision history for this message
Chad Smith (chad.smith) :
review: Approve
~raharper/cloud-init:feature/update-ntp-spec updated
1f840b0... by Ryan Harper

Switch preferred ntp client to distro property, drop unneeded mocks

1a6cb7d... by Ryan Harper

Add Chad's additional schema validations

Revision history for this message
Ryan Harper (raharper) wrote :

I think I've addressed the feedback,

- switched to distro property
- pulled in Chad's tighter schema checks

Revision history for this message
Server Team CI bot (server-team-bot) wrote :

PASSED: Continuous integration, rev:1a6cb7d532e4c0d0f7241d79c6650ab094b7fea5
https://jenkins.ubuntu.com/server/job/cloud-init-ci/991/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    SUCCESS: MAAS Compatability Testing
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/991/rebuild

review: Approve (continuous-integration)
~raharper/cloud-init:feature/update-ntp-spec updated
beeb8c6... by Ryan Harper

Drop utf-8 apostrophe for ascii

f5bcbdc... by Ryan Harper

Drop util.decode_text(); adjust unittests

Revision history for this message
Server Team CI bot (server-team-bot) wrote :

PASSED: Continuous integration, rev:f5bcbdc4002680ea74edf5ca8b1c25b9a9ad78d5
https://jenkins.ubuntu.com/server/job/cloud-init-ci/992/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    SUCCESS: MAAS Compatability Testing
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/992/rebuild

review: Approve (continuous-integration)
Revision history for this message
Scott Moser (smoser) wrote :

some minor nits.
thanks for addressing feedback quickly.

~raharper/cloud-init:feature/update-ntp-spec updated
8c87e14... by Ryan Harper

Drop use of copy.deepcopy for a simple copy via list()

30904e2... by Ryan Harper

Use /bin/sh where possible in cloud_tests for ntp

Revision history for this message
Ryan Harper (raharper) wrote :

Updated

Revision history for this message
Scott Moser (smoser) :
review: Approve
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

PASSED: Continuous integration, rev:30904e28f1d1b4df41fcf81b0fcfbd491bf8f7e7
https://jenkins.ubuntu.com/server/job/cloud-init-ci/994/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    SUCCESS: MAAS Compatability Testing
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/994/rebuild

review: Approve (continuous-integration)
Revision history for this message
Scott Moser (smoser) wrote :

An upstream commit landed for this bug.

To view that commit see the following URL:
https://git.launchpad.net/cloud-init/commit/?id=c6dff581

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1diff --git a/cloudinit/config/cc_ntp.py b/cloudinit/config/cc_ntp.py
2index cbd0237..9e074bd 100644
3--- a/cloudinit/config/cc_ntp.py
4+++ b/cloudinit/config/cc_ntp.py
5@@ -10,20 +10,95 @@ from cloudinit.config.schema import (
6 get_schema_doc, validate_cloudconfig_schema)
7 from cloudinit import log as logging
8 from cloudinit.settings import PER_INSTANCE
9+from cloudinit import temp_utils
10 from cloudinit import templater
11 from cloudinit import type_utils
12 from cloudinit import util
13
14+import copy
15 import os
16+import six
17 from textwrap import dedent
18
19 LOG = logging.getLogger(__name__)
20
21 frequency = PER_INSTANCE
22 NTP_CONF = '/etc/ntp.conf'
23-TIMESYNCD_CONF = '/etc/systemd/timesyncd.conf.d/cloud-init.conf'
24 NR_POOL_SERVERS = 4
25-distros = ['centos', 'debian', 'fedora', 'opensuse', 'sles', 'ubuntu']
26+distros = ['centos', 'debian', 'fedora', 'opensuse', 'rhel', 'sles', 'ubuntu']
27+
28+NTP_CLIENT_CONFIG = {
29+ 'chrony': {
30+ 'check_exe': 'chronyd',
31+ 'confpath': '/etc/chrony.conf',
32+ 'packages': ['chrony'],
33+ 'service_name': 'chrony',
34+ 'template_name': 'chrony.conf.{distro}',
35+ 'template': None,
36+ },
37+ 'ntp': {
38+ 'check_exe': 'ntpd',
39+ 'confpath': NTP_CONF,
40+ 'packages': ['ntp'],
41+ 'service_name': 'ntp',
42+ 'template_name': 'ntp.conf.{distro}',
43+ 'template': None,
44+ },
45+ 'ntpdate': {
46+ 'check_exe': 'ntpdate',
47+ 'confpath': NTP_CONF,
48+ 'packages': ['ntpdate'],
49+ 'service_name': 'ntpdate',
50+ 'template_name': 'ntp.conf.{distro}',
51+ 'template': None,
52+ },
53+ 'systemd-timesyncd': {
54+ 'check_exe': '/lib/systemd/systemd-timesyncd',
55+ 'confpath': '/etc/systemd/timesyncd.conf.d/cloud-init.conf',
56+ 'packages': [],
57+ 'service_name': 'systemd-timesyncd',
58+ 'template_name': 'timesyncd.conf',
59+ 'template': None,
60+ },
61+}
62+
63+# This is Distro-specific configuration overrides of the base config
64+DISTRO_CLIENT_CONFIG = {
65+ 'debian': {
66+ 'chrony': {
67+ 'confpath': '/etc/chrony/chrony.conf',
68+ },
69+ },
70+ 'opensuse': {
71+ 'chrony': {
72+ 'service_name': 'chronyd',
73+ },
74+ 'ntp': {
75+ 'confpath': '/etc/ntp.conf',
76+ 'service_name': 'ntpd',
77+ },
78+ 'systemd-timesyncd': {
79+ 'check_exe': '/usr/lib/systemd/systemd-timesyncd',
80+ },
81+ },
82+ 'sles': {
83+ 'chrony': {
84+ 'service_name': 'chronyd',
85+ },
86+ 'ntp': {
87+ 'confpath': '/etc/ntp.conf',
88+ 'service_name': 'ntpd',
89+ },
90+ 'systemd-timesyncd': {
91+ 'check_exe': '/usr/lib/systemd/systemd-timesyncd',
92+ },
93+ },
94+ 'ubuntu': {
95+ 'chrony': {
96+ 'confpath': '/etc/chrony/chrony.conf',
97+ },
98+ },
99+}
100
101
102 # The schema definition for each cloud-config module is a strict contract for
103@@ -48,7 +123,34 @@ schema = {
104 'distros': distros,
105 'examples': [
106 dedent("""\
107+ # Override ntp with chrony configuration on Ubuntu
108+ ntp:
109+ enabled: true
110+ ntp_client: chrony # Uses cloud-init default chrony configuration
111+ """),
112+ dedent("""\
113+ # Provide a custom ntp client configuration
114 ntp:
115+ enabled: true
116+ ntp_client: myntpclient
117+ config:
118+ confpath: /etc/myntpclient/myntpclient.conf
119+ check_exe: myntpclientd
120+ packages:
121+ - myntpclient
122+ service_name: myntpclient
123+ template: |
124+ ## template:jinja
125+ # My NTP Client config
126+ {% if pools -%}# pools{% endif %}
127+ {% for pool in pools -%}
128+ pool {{pool}} iburst
129+ {% endfor %}
130+ {%- if servers %}# servers
131+ {% endif %}
132+ {% for server in servers -%}
133+ server {{server}} iburst
134+ {% endfor %}
135 pools: [0.int.pool.ntp.org, 1.int.pool.ntp.org, ntp.myorg.org]
136 servers:
137 - ntp.server.local
138@@ -83,79 +185,159 @@ schema = {
139 List of ntp servers. If both pools and servers are
140 empty, 4 default pool servers will be provided with
141 the format ``{0-3}.{distro}.pool.ntp.org``.""")
142- }
143+ },
144+ 'ntp_client': {
145+ 'type': 'string',
146+ 'default': 'auto',
147+ 'description': dedent("""\
148+ Name of an NTP client to use to configure system NTP.
149+ When unprovided or 'auto' the default client preferred
150+ by the distribution will be used. The following
151+ built-in client names can be used to override existing
152+ configuration defaults: chrony, ntp, ntpdate,
153+ systemd-timesyncd."""),
154+ },
155+ 'enabled': {
156+ 'type': 'boolean',
157+ 'default': True,
158+ 'description': dedent("""\
159+ Attempt to enable ntp clients if set to True. If set
160+ to False, ntp client will not be configured or
161+ installed"""),
162+ },
163+ 'config': {
164+ 'description': dedent("""\
165+ Configuration settings or overrides for the
166+ ``ntp_client`` specified."""),
167+ 'type': ['object'],
168+ 'properties': {
169+ 'confpath': {
170+ 'type': 'string',
171+ 'description': dedent("""\
172+ The path to where the ``ntp_client``
173+ configuration is written."""),
174+ },
175+ 'check_exe': {
176+ 'type': 'string',
177+ 'description': dedent("""\
178+ The executable name for the ``ntp_client``.
179+ For example, ntp service ``check_exe`` is
180+ 'ntpd' because it runs the ntpd binary."""),
181+ },
182+ 'packages': {
183+ 'type': 'array',
184+ 'items': {
185+ 'type': 'string',
186+ },
187+ 'uniqueItems': True,
188+ 'description': dedent("""\
189+ List of packages needed to be installed for the
190+ selected ``ntp_client``."""),
191+ },
192+ 'service_name': {
193+ 'type': 'string',
194+ 'description': dedent("""\
195+ The systemd or sysvinit service name used to
196+ start and stop the ``ntp_client``
197+ service."""),
198+ },
199+ 'template': {
200+ 'type': 'string',
201+ 'description': dedent("""\
202+ Inline template allowing users to define their
203+ own ``ntp_client`` configuration template.
204+ The value must start with '## template:jinja'
205+ to enable use of templating support.
206+ """),
207+ },
208+ },
209+ # Don't use REQUIRED_NTP_CONFIG_KEYS to allow for override
210+ # of builtin client values.
211+ 'required': [],
212+ 'minProperties': 1, # If we have config, define something
213+ 'additionalProperties': False
214+ },
215 },
216 'required': [],
217 'additionalProperties': False
218 }
219 }
220 }
221-
222-__doc__ = get_schema_doc(schema) # Supplement python help()
223+REQUIRED_NTP_CONFIG_KEYS = frozenset([
224+ 'check_exe', 'confpath', 'packages', 'service_name'])
225
226
227-def handle(name, cfg, cloud, log, _args):
228- """Enable and configure ntp."""
229- if 'ntp' not in cfg:
230- LOG.debug(
231- "Skipping module named %s, not present or disabled by cfg", name)
232- return
233- ntp_cfg = cfg['ntp']
234- if ntp_cfg is None:
235- ntp_cfg = {} # Allow empty config which will install the package
236+__doc__ = get_schema_doc(schema) # Supplement python help()
237
238- # TODO drop this when validate_cloudconfig_schema is strict=True
239- if not isinstance(ntp_cfg, (dict)):
240- raise RuntimeError(
241- "'ntp' key existed in config, but not a dictionary type,"
242- " is a {_type} instead".format(_type=type_utils.obj_name(ntp_cfg)))
243
244- validate_cloudconfig_schema(cfg, schema)
245- if ntp_installable():
246- service_name = 'ntp'
247- confpath = NTP_CONF
248- template_name = None
249- packages = ['ntp']
250- check_exe = 'ntpd'
251- else:
252- service_name = 'systemd-timesyncd'
253- confpath = TIMESYNCD_CONF
254- template_name = 'timesyncd.conf'
255- packages = []
256- check_exe = '/lib/systemd/systemd-timesyncd'
257-
258- rename_ntp_conf()
259- # ensure when ntp is installed it has a configuration file
260- # to use instead of starting up with packaged defaults
261- write_ntp_config_template(ntp_cfg, cloud, confpath, template=template_name)
262- install_ntp(cloud.distro.install_packages, packages=packages,
263- check_exe=check_exe)
264+def distro_ntp_client_configs(distro):
265+ """Construct a distro-specific ntp client config dictionary by merging
266+ distro specific changes into base config.
267
268- try:
269- reload_ntp(service_name, systemd=cloud.distro.uses_systemd())
270- except util.ProcessExecutionError as e:
271- LOG.exception("Failed to reload/start ntp service: %s", e)
272- raise
273+ @param distro: String providing the distro class name.
274+ @returns: Dict of distro configurations for ntp clients.
275+ """
276+ dcfg = DISTRO_CLIENT_CONFIG
277+ cfg = copy.copy(NTP_CLIENT_CONFIG)
278+ if distro in dcfg:
279+ cfg = util.mergemanydict([cfg, dcfg[distro]], reverse=True)
280+ return cfg
281
282
283-def ntp_installable():
284- """Check if we can install ntp package
285+def select_ntp_client(ntp_client, distro):
286+ """Determine which ntp client is to be used, consulting the distro
287+ for its preference.
288
289- Ubuntu-Core systems do not have an ntp package available, so
290- we always return False. Other systems require package managers to install
291- the ntp package If we fail to find one of the package managers, then we
292- cannot install ntp.
293+ @param ntp_client: String name of the ntp client to use.
294+ @param distro: Distro class instance.
295+ @returns: Dict of the selected ntp client or {} if none selected.
296 """
297- if util.system_is_snappy():
298- return False
299
300- if any(map(util.which, ['apt-get', 'dnf', 'yum', 'zypper'])):
301- return True
302+ # construct distro-specific ntp_client_config dict
303+ distro_cfg = distro_ntp_client_configs(distro.name)
304+
305+ # user specified client, return its config
306+ if ntp_client and ntp_client != 'auto':
307+ LOG.debug('Selected NTP client "%s" via user-data configuration',
308+ ntp_client)
309+ return distro_cfg.get(ntp_client, {})
310+
311+ # default to auto if unset in distro
312+ distro_ntp_client = distro.get_option('ntp_client', 'auto')
313+
314+ clientcfg = {}
315+ if distro_ntp_client == "auto":
316+ for client in distro.preferred_ntp_clients:
317+ cfg = distro_cfg.get(client)
318+ if util.which(cfg.get('check_exe')):
319+ LOG.debug('Selected NTP client "%s", already installed',
320+ client)
321+ clientcfg = cfg
322+ break
323+
324+ if not clientcfg:
325+ client = distro.preferred_ntp_clients[0]
326+ LOG.debug(
327+ 'Selected distro preferred NTP client "%s", not yet installed',
328+ client)
329+ clientcfg = distro_cfg.get(client)
330+ else:
331+ LOG.debug('Selected NTP client "%s" via distro system config',
332+ distro_ntp_client)
333+ clientcfg = distro_cfg.get(distro_ntp_client, {})
334+
335+ return clientcfg
336
337- return False
338
339+def install_ntp_client(install_func, packages=None, check_exe="ntpd"):
340+ """Install ntp client package if not already installed.
341
342-def install_ntp(install_func, packages=None, check_exe="ntpd"):
343+ @param install_func: function. This parameter is invoked with the contents
344+ of the packages parameter.
345+ @param packages: list. This parameter defaults to ['ntp'].
346+ @param check_exe: string. The name of a binary that indicates the package
347+ the specified package is already installed.
348+ """
349 if util.which(check_exe):
350 return
351 if packages is None:
352@@ -164,15 +346,23 @@ def install_ntp(install_func, packages=None, check_exe="ntpd"):
353 install_func(packages)
354
355
356-def rename_ntp_conf(config=None):
357- """Rename any existing ntp.conf file"""
358- if config is None: # For testing
359- config = NTP_CONF
360- if os.path.exists(config):
361- util.rename(config, config + ".dist")
362+def rename_ntp_conf(confpath=None):
363+ """Rename any existing ntp client config file
364+
365+ @param confpath: string. Specify a path to an existing ntp client
366+ configuration file.
367+ """
368+ if os.path.exists(confpath):
369+ util.rename(confpath, confpath + ".dist")
370
371
372 def generate_server_names(distro):
373+ """Generate a list of server names to populate an ntp client configuration
374+ file.
375+
376+ @param distro: string. Specify the distro name
377+ @returns: list: A list of strings representing ntp servers for this distro.
378+ """
379 names = []
380 pool_distro = distro
381 # For legal reasons x.pool.sles.ntp.org does not exist,
382@@ -185,34 +375,60 @@ def generate_server_names(distro):
383 return names
384
385
386-def write_ntp_config_template(cfg, cloud, path, template=None):
387- servers = cfg.get('servers', [])
388- pools = cfg.get('pools', [])
389+def write_ntp_config_template(distro_name, servers=None, pools=None,
390+ path=None, template_fn=None, template=None):
391+ """Render a ntp client configuration for the specified client.
392+
393+ @param distro_name: string. The distro class name.
394+ @param servers: A list of strings specifying ntp servers. Defaults to empty
395+ list.
396+ @param pools: A list of strings specifying ntp pools. Defaults to empty
397+ list.
398+ @param path: A string to specify where to write the rendered template.
399+ @param template_fn: A string to specify the template source file.
400+ @param template: A string specifying the contents of the template. This
401+ content will be written to a temporary file before being used to render
402+ the configuration file.
403+
404+ @raises: ValueError when path is None.
405+ @raises: ValueError when template_fn is None and template is None.
406+ """
407+ if not servers:
408+ servers = []
409+ if not pools:
410+ pools = []
411
412 if len(servers) == 0 and len(pools) == 0:
413- pools = generate_server_names(cloud.distro.name)
414+ pools = generate_server_names(distro_name)
415 LOG.debug(
416 'Adding distro default ntp pool servers: %s', ','.join(pools))
417
418- params = {
419- 'servers': servers,
420- 'pools': pools,
421- }
422+ if not path:
423+ raise ValueError('Invalid value for path parameter')
424
425- if template is None:
426- template = 'ntp.conf.%s' % cloud.distro.name
427+ if not template_fn and not template:
428+ raise ValueError('Not template_fn or template provided')
429
430- template_fn = cloud.get_template_filename(template)
431- if not template_fn:
432- template_fn = cloud.get_template_filename('ntp.conf')
433- if not template_fn:
434- raise RuntimeError(
435- 'No template found, not rendering {path}'.format(path=path))
436+ params = {'servers': servers, 'pools': pools}
437+ if template:
438+ tfile = temp_utils.mkstemp(prefix='template_name-', suffix=".tmpl")
439+ template_fn = tfile[1] # filepath is second item in tuple
440+ util.write_file(template_fn, content=template)
441
442 templater.render_to_file(template_fn, path, params)
443+ # clean up temporary template
444+ if template:
445+ util.del_file(template_fn)
446
447
448 def reload_ntp(service, systemd=False):
449+ """Restart or reload an ntp system service.
450+
451+ @param service: A string specifying the name of the service to be affected.
452+ @param systemd: A boolean indicating if the distro uses systemd, defaults
453+ to False.
454+ @returns: A tuple of stdout, stderr results from executing the action.
455+ """
456 if systemd:
457 cmd = ['systemctl', 'reload-or-restart', service]
458 else:
459@@ -220,4 +436,117 @@ def reload_ntp(service, systemd=False):
460 util.subp(cmd, capture=True)
461
462
463+def supplemental_schema_validation(ntp_config):
464+ """Validate user-provided ntp:config option values.
465+
466+ This function supplements flexible jsonschema validation with specific
467+ value checks to aid in triage of invalid user-provided configuration.
468+
469+ @param ntp_config: Dictionary of configuration value under 'ntp'.
470+
471+ @raises: ValueError describing invalid values provided.
472+ """
473+ errors = []
474+ missing = REQUIRED_NTP_CONFIG_KEYS.difference(set(ntp_config.keys()))
475+ if missing:
476+ keys = ', '.join(sorted(missing))
477+ errors.append(
478+ 'Missing required ntp:config keys: {keys}'.format(keys=keys))
479+ elif not any([ntp_config.get('template'),
480+ ntp_config.get('template_name')]):
481+ errors.append(
482+ 'Either ntp:config:template or ntp:config:template_name values'
483+ ' are required')
484+ for key, value in sorted(ntp_config.items()):
485+ keypath = 'ntp:config:' + key
486+ if key == 'confpath':
487+ if not all([value, isinstance(value, six.string_types)]):
488+ errors.append(
489+ 'Expected a config file path {keypath}.'
490+ ' Found ({value})'.format(keypath=keypath, value=value))
491+ elif key == 'packages':
492+ if not isinstance(value, list):
493+ errors.append(
494+ 'Expected a list of required package names for {keypath}.'
495+ ' Found ({value})'.format(keypath=keypath, value=value))
496+ elif key in ('template', 'template_name'):
497+ if value is None: # Either template or template_name can be none
498+ continue
499+ if not isinstance(value, six.string_types):
500+ errors.append(
501+ 'Expected a string type for {keypath}.'
502+ ' Found ({value})'.format(keypath=keypath, value=value))
503+ elif not isinstance(value, six.string_types):
504+ errors.append(
505+ 'Expected a string type for {keypath}.'
506+ ' Found ({value})'.format(keypath=keypath, value=value))
507+
508+ if errors:
509+ raise ValueError(r'Invalid ntp configuration:\n{errors}'.format(
510+ errors='\n'.join(errors)))
511+
512+
513+def handle(name, cfg, cloud, log, _args):
514+ """Enable and configure ntp."""
515+ if 'ntp' not in cfg:
516+ LOG.debug(
517+ "Skipping module named %s, not present or disabled by cfg", name)
518+ return
519+ ntp_cfg = cfg['ntp']
520+ if ntp_cfg is None:
521+ ntp_cfg = {} # Allow empty config which will install the package
522+
523+ # TODO drop this when validate_cloudconfig_schema is strict=True
524+ if not isinstance(ntp_cfg, (dict)):
525+ raise RuntimeError(
526+ "'ntp' key existed in config, but not a dictionary type,"
527+ " is a {_type} instead".format(_type=type_utils.obj_name(ntp_cfg)))
528+
529+ validate_cloudconfig_schema(cfg, schema)
530+
531+ # Allow users to explicitly enable/disable
532+ enabled = ntp_cfg.get('enabled', True)
533+ if util.is_false(enabled):
534+ LOG.debug("Skipping module named %s, disabled by cfg", name)
535+ return
536+
537+ # Select which client is going to be used and get the configuration
538+ ntp_client_config = select_ntp_client(ntp_cfg.get('ntp_client'),
539+ cloud.distro)
540+
541+ # Allow user ntp config to override distro configurations
542+ ntp_client_config = util.mergemanydict(
543+ [ntp_client_config, ntp_cfg.get('config', {})], reverse=True)
544+
545+ supplemental_schema_validation(ntp_client_config)
546+ rename_ntp_conf(confpath=ntp_client_config.get('confpath'))
547+
548+ template_fn = None
549+ if not ntp_client_config.get('template'):
550+ template_name = (
551+ ntp_client_config.get('template_name').replace('{distro}',
552+ cloud.distro.name))
553+ template_fn = cloud.get_template_filename(template_name)
554+ if not template_fn:
555+ msg = ('No template found, not rendering %s' %
556+ ntp_client_config.get('template_name'))
557+ raise RuntimeError(msg)
558+
559+ write_ntp_config_template(cloud.distro.name,
560+ servers=ntp_cfg.get('servers', []),
561+ pools=ntp_cfg.get('pools', []),
562+ path=ntp_client_config.get('confpath'),
563+ template_fn=template_fn,
564+ template=ntp_client_config.get('template'))
565+
566+ install_ntp_client(cloud.distro.install_packages,
567+ packages=ntp_client_config['packages'],
568+ check_exe=ntp_client_config['check_exe'])
569+ try:
570+ reload_ntp(ntp_client_config['service_name'],
571+ systemd=cloud.distro.uses_systemd())
572+ except util.ProcessExecutionError as e:
573+ LOG.exception("Failed to reload/start ntp service: %s", e)
574+ raise
575+
576 # vi: ts=4 expandtab
577diff --git a/cloudinit/distros/__init__.py b/cloudinit/distros/__init__.py
578index 55260ea..6c22b07 100755
579--- a/cloudinit/distros/__init__.py
580+++ b/cloudinit/distros/__init__.py
581@@ -49,6 +49,9 @@ LOG = logging.getLogger(__name__)
582 # It could break when Amazon adds new regions and new AZs.
583 _EC2_AZ_RE = re.compile('^[a-z][a-z]-(?:[a-z]+-)+[0-9][a-z]$')
584
585+# Default NTP Client Configurations
586+PREFERRED_NTP_CLIENTS = ['chrony', 'systemd-timesyncd', 'ntp', 'ntpdate']
587+
588
589 @six.add_metaclass(abc.ABCMeta)
590 class Distro(object):
591@@ -60,6 +63,7 @@ class Distro(object):
592 tz_zone_dir = "/usr/share/zoneinfo"
593 init_cmd = ['service'] # systemctl, service etc
594 renderer_configs = {}
595+ _preferred_ntp_clients = None
596
597 def __init__(self, name, cfg, paths):
598 self._paths = paths
599@@ -339,6 +343,14 @@ class Distro(object):
600 contents.write("%s\n" % (eh))
601 util.write_file(self.hosts_fn, contents.getvalue(), mode=0o644)
602
603+ @property
604+ def preferred_ntp_clients(self):
605+ """Allow distro to determine the preferred ntp client list"""
606+ if not self._preferred_ntp_clients:
607+ self._preferred_ntp_clients = list(PREFERRED_NTP_CLIENTS)
608+
609+ return self._preferred_ntp_clients
610+
611 def _bring_up_interface(self, device_name):
612 cmd = ['ifup', device_name]
613 LOG.debug("Attempting to run bring up interface %s using command %s",
614diff --git a/cloudinit/distros/opensuse.py b/cloudinit/distros/opensuse.py
615index 162dfa0..9f90e95 100644
616--- a/cloudinit/distros/opensuse.py
617+++ b/cloudinit/distros/opensuse.py
618@@ -208,4 +208,28 @@ class Distro(distros.Distro):
619 nameservers, searchservers)
620 return dev_names
621
622+ @property
623+ def preferred_ntp_clients(self):
624+ """The preferred ntp client is dependent on the version."""
625+
626+ """Allow distro to determine the preferred ntp client list"""
627+ if not self._preferred_ntp_clients:
628+ distro_info = util.system_info()['dist']
629+ name = distro_info[0]
630+ major_ver = int(distro_info[1].split('.')[0])
631+
632+ # This is horribly complicated because of a case of
633+ # "we do not care if versions should be increasing syndrome"
634+ if (
635+ (major_ver >= 15 and 'openSUSE' not in name) or
636+ (major_ver >= 15 and 'openSUSE' in name and major_ver != 42)
637+ ):
638+ self._preferred_ntp_clients = ['chrony',
639+ 'systemd-timesyncd', 'ntp']
640+ else:
641+ self._preferred_ntp_clients = ['ntp',
642+ 'systemd-timesyncd', 'chrony']
643+
644+ return self._preferred_ntp_clients
645+
646 # vi: ts=4 expandtab
647diff --git a/cloudinit/distros/ubuntu.py b/cloudinit/distros/ubuntu.py
648index 82ca34f..fdc1f62 100644
649--- a/cloudinit/distros/ubuntu.py
650+++ b/cloudinit/distros/ubuntu.py
651@@ -10,12 +10,31 @@
652 # This file is part of cloud-init. See LICENSE file for license information.
653
654 from cloudinit.distros import debian
655+from cloudinit.distros import PREFERRED_NTP_CLIENTS
656 from cloudinit import log as logging
657+from cloudinit import util
658+
659+import copy
660
661 LOG = logging.getLogger(__name__)
662
663
664 class Distro(debian.Distro):
665+
666+ @property
667+ def preferred_ntp_clients(self):
668+ """The preferred ntp client is dependent on the version."""
669+ if not self._preferred_ntp_clients:
670+ (name, version, codename) = util.system_info()['dist']
671+ # Xenial cloud-init only installed ntp, UbuntuCore has timesyncd.
672+ if codename == "xenial" and not util.system_is_snappy():
673+ self._preferred_ntp_clients = ['ntp']
674+ else:
675+ self._preferred_ntp_clients = (
676+ copy.deepcopy(PREFERRED_NTP_CLIENTS))
677+ return self._preferred_ntp_clients
678+
679 pass
680
681+
682 # vi: ts=4 expandtab
683diff --git a/config/cloud.cfg.tmpl b/config/cloud.cfg.tmpl
684index 3129d4e..5619de3 100644
685--- a/config/cloud.cfg.tmpl
686+++ b/config/cloud.cfg.tmpl
687@@ -151,6 +151,8 @@ system_info:
688 groups: [adm, audio, cdrom, dialout, dip, floppy, lxd, netdev, plugdev, sudo, video]
689 sudo: ["ALL=(ALL) NOPASSWD:ALL"]
690 shell: /bin/bash
691+ # Automatically discover the best ntp_client
692+ ntp_client: auto
693 # Other config here will be given to the distro class and/or path classes
694 paths:
695 cloud_dir: /var/lib/cloud/
696diff --git a/templates/chrony.conf.debian.tmpl b/templates/chrony.conf.debian.tmpl
697new file mode 100644
698index 0000000..e72bfee
699--- /dev/null
700+++ b/templates/chrony.conf.debian.tmpl
701@@ -0,0 +1,39 @@
702+## template:jinja
703+# Welcome to the chrony configuration file. See chrony.conf(5) for more
704+# information about usuable directives.
705+{% if pools %}# pools
706+{% endif %}
707+{% for pool in pools -%}
708+pool {{pool}} iburst
709+{% endfor %}
710+{%- if servers %}# servers
711+{% endif %}
712+{% for server in servers -%}
713+server {{server}} iburst
714+{% endfor %}
715+
716+# This directive specify the location of the file containing ID/key pairs for
717+# NTP authentication.
718+keyfile /etc/chrony/chrony.keys
719+
720+# This directive specify the file into which chronyd will store the rate
721+# information.
722+driftfile /var/lib/chrony/chrony.drift
723+
724+# Uncomment the following line to turn logging on.
725+#log tracking measurements statistics
726+
727+# Log files location.
728+logdir /var/log/chrony
729+
730+# Stop bad estimates upsetting machine clock.
731+maxupdateskew 100.0
732+
733+# This directive enables kernel synchronisation (every 11 minutes) of the
734+# real-time clock. Note that it can't be used along with the 'rtcfile' directive.
735+rtcsync
736+
737+# Step the system clock instead of slewing it if the adjustment is larger than
738+# one second, but only in the first three clock updates.
739+makestep 1 3
740+
741diff --git a/templates/chrony.conf.fedora.tmpl b/templates/chrony.conf.fedora.tmpl
742new file mode 100644
743index 0000000..8551f79
744--- /dev/null
745+++ b/templates/chrony.conf.fedora.tmpl
746@@ -0,0 +1,48 @@
747+## template:jinja
748+# Use public servers from the pool.ntp.org project.
749+# Please consider joining the pool (http://www.pool.ntp.org/join.html).
750+{% if pools %}# pools
751+{% endif %}
752+{% for pool in pools -%}
753+pool {{pool}} iburst
754+{% endfor %}
755+{%- if servers %}# servers
756+{% endif %}
757+{% for server in servers -%}
758+server {{server}} iburst
759+{% endfor %}
760+
761+# Record the rate at which the system clock gains/losses time.
762+driftfile /var/lib/chrony/drift
763+
764+# Allow the system clock to be stepped in the first three updates
765+# if its offset is larger than 1 second.
766+makestep 1.0 3
767+
768+# Enable kernel synchronization of the real-time clock (RTC).
769+rtcsync
770+
771+# Enable hardware timestamping on all interfaces that support it.
772+#hwtimestamp *
773+
774+# Increase the minimum number of selectable sources required to adjust
775+# the system clock.
776+#minsources 2
777+
778+# Allow NTP client access from local network.
779+#allow 192.168.0.0/16
780+
781+# Serve time even if not synchronized to a time source.
782+#local stratum 10
783+
784+# Specify file containing keys for NTP authentication.
785+#keyfile /etc/chrony.keys
786+
787+# Get TAI-UTC offset and leap seconds from the system tz database.
788+leapsectz right/UTC
789+
790+# Specify directory for log files.
791+logdir /var/log/chrony
792+
793+# Select which information is logged.
794+#log measurements statistics tracking
795diff --git a/templates/chrony.conf.opensuse.tmpl b/templates/chrony.conf.opensuse.tmpl
796new file mode 100644
797index 0000000..a3d3e0e
798--- /dev/null
799+++ b/templates/chrony.conf.opensuse.tmpl
800@@ -0,0 +1,38 @@
801+## template:jinja
802+# Use public servers from the pool.ntp.org project.
803+# Please consider joining the pool (http://www.pool.ntp.org/join.html).
804+{% if pools %}# pools
805+{% endif %}
806+{% for pool in pools -%}
807+pool {{pool}} iburst
808+{% endfor %}
809+{%- if servers %}# servers
810+{% endif %}
811+{% for server in servers -%}
812+server {{server}} iburst
813+{% endfor %}
814+
815+# Record the rate at which the system clock gains/losses time.
816+driftfile /var/lib/chrony/drift
817+
818+# In first three updates step the system clock instead of slew
819+# if the adjustment is larger than 1 second.
820+makestep 1.0 3
821+
822+# Enable kernel synchronization of the real-time clock (RTC).
823+rtcsync
824+
825+# Allow NTP client access from local network.
826+#allow 192.168/16
827+
828+# Serve time even if not synchronized to any NTP server.
829+#local stratum 10
830+
831+# Specify file containing keys for NTP authentication.
832+#keyfile /etc/chrony.keys
833+
834+# Specify directory for log files.
835+logdir /var/log/chrony
836+
837+# Select which information is logged.
838+#log measurements statistics tracking
839diff --git a/templates/chrony.conf.rhel.tmpl b/templates/chrony.conf.rhel.tmpl
840new file mode 100644
841index 0000000..5b3542e
842--- /dev/null
843+++ b/templates/chrony.conf.rhel.tmpl
844@@ -0,0 +1,45 @@
845+## template:jinja
846+# Use public servers from the pool.ntp.org project.
847+# Please consider joining the pool (http://www.pool.ntp.org/join.html).
848+{% if pools %}# pools
849+{% endif %}
850+{% for pool in pools -%}
851+pool {{pool}} iburst
852+{% endfor %}
853+{%- if servers %}# servers
854+{% endif %}
855+{% for server in servers -%}
856+server {{server}} iburst
857+{% endfor %}
858+
859+# Record the rate at which the system clock gains/losses time.
860+driftfile /var/lib/chrony/drift
861+
862+# Allow the system clock to be stepped in the first three updates
863+# if its offset is larger than 1 second.
864+makestep 1.0 3
865+
866+# Enable kernel synchronization of the real-time clock (RTC).
867+rtcsync
868+
869+# Enable hardware timestamping on all interfaces that support it.
870+#hwtimestamp *
871+
872+# Increase the minimum number of selectable sources required to adjust
873+# the system clock.
874+#minsources 2
875+
876+# Allow NTP client access from local network.
877+#allow 192.168.0.0/16
878+
879+# Serve time even if not synchronized to a time source.
880+#local stratum 10
881+
882+# Specify file containing keys for NTP authentication.
883+#keyfile /etc/chrony.keys
884+
885+# Specify directory for log files.
886+logdir /var/log/chrony
887+
888+# Select which information is logged.
889+#log measurements statistics tracking
890diff --git a/templates/chrony.conf.sles.tmpl b/templates/chrony.conf.sles.tmpl
891new file mode 100644
892index 0000000..a3d3e0e
893--- /dev/null
894+++ b/templates/chrony.conf.sles.tmpl
895@@ -0,0 +1,38 @@
896+## template:jinja
897+# Use public servers from the pool.ntp.org project.
898+# Please consider joining the pool (http://www.pool.ntp.org/join.html).
899+{% if pools %}# pools
900+{% endif %}
901+{% for pool in pools -%}
902+pool {{pool}} iburst
903+{% endfor %}
904+{%- if servers %}# servers
905+{% endif %}
906+{% for server in servers -%}
907+server {{server}} iburst
908+{% endfor %}
909+
910+# Record the rate at which the system clock gains/losses time.
911+driftfile /var/lib/chrony/drift
912+
913+# In first three updates step the system clock instead of slew
914+# if the adjustment is larger than 1 second.
915+makestep 1.0 3
916+
917+# Enable kernel synchronization of the real-time clock (RTC).
918+rtcsync
919+
920+# Allow NTP client access from local network.
921+#allow 192.168/16
922+
923+# Serve time even if not synchronized to any NTP server.
924+#local stratum 10
925+
926+# Specify file containing keys for NTP authentication.
927+#keyfile /etc/chrony.keys
928+
929+# Specify directory for log files.
930+logdir /var/log/chrony
931+
932+# Select which information is logged.
933+#log measurements statistics tracking
934diff --git a/templates/chrony.conf.ubuntu.tmpl b/templates/chrony.conf.ubuntu.tmpl
935new file mode 100644
936index 0000000..da7f16a
937--- /dev/null
938+++ b/templates/chrony.conf.ubuntu.tmpl
939@@ -0,0 +1,42 @@
940+## template:jinja
941+# Welcome to the chrony configuration file. See chrony.conf(5) for more
942+# information about usuable directives.
943+
944+# Use servers from the NTP Pool Project. Approved by Ubuntu Technical Board
945+# on 2011-02-08 (LP: #104525). See http://www.pool.ntp.org/join.html for
946+# more information.
947+{% if pools %}# pools
948+{% endif %}
949+{% for pool in pools -%}
950+pool {{pool}} iburst
951+{% endfor %}
952+{%- if servers %}# servers
953+{% endif %}
954+{% for server in servers -%}
955+server {{server}} iburst
956+{% endfor %}
957+
958+# This directive specify the location of the file containing ID/key pairs for
959+# NTP authentication.
960+keyfile /etc/chrony/chrony.keys
961+
962+# This directive specify the file into which chronyd will store the rate
963+# information.
964+driftfile /var/lib/chrony/chrony.drift
965+
966+# Uncomment the following line to turn logging on.
967+#log tracking measurements statistics
968+
969+# Log files location.
970+logdir /var/log/chrony
971+
972+# Stop bad estimates upsetting machine clock.
973+maxupdateskew 100.0
974+
975+# This directive enables kernel synchronisation (every 11 minutes) of the
976+# real-time clock. Note that it can't be used along with the 'rtcfile' directive.
977+rtcsync
978+
979+# Step the system clock instead of slewing it if the adjustment is larger than
980+# one second, but only in the first three clock updates.
981+makestep 1 3
982diff --git a/tests/cloud_tests/testcases/modules/ntp.yaml b/tests/cloud_tests/testcases/modules/ntp.yaml
983index 2530d72..7ea0707 100644
984--- a/tests/cloud_tests/testcases/modules/ntp.yaml
985+++ b/tests/cloud_tests/testcases/modules/ntp.yaml
986@@ -4,6 +4,7 @@
987 cloud_config: |
988 #cloud-config
989 ntp:
990+ ntp_client: ntp
991 pools: []
992 servers: []
993 collect_scripts:
994diff --git a/tests/cloud_tests/testcases/modules/ntp_chrony.py b/tests/cloud_tests/testcases/modules/ntp_chrony.py
995new file mode 100644
996index 0000000..461630a
997--- /dev/null
998+++ b/tests/cloud_tests/testcases/modules/ntp_chrony.py
999@@ -0,0 +1,15 @@
1000+# This file is part of cloud-init. See LICENSE file for license information.
1001+
1002+"""cloud-init Integration Test Verify Script."""
1003+from tests.cloud_tests.testcases import base
1004+
1005+
1006+class TestNtpChrony(base.CloudTestCase):
1007+ """Test ntp module with chrony client"""
1008+
1009+ def test_chrony_entires(self):
1010+ """Test chrony config entries"""
1011+ out = self.get_data_file('chrony_conf')
1012+ self.assertIn('.pool.ntp.org', out)
1013+
1014+# vi: ts=4 expandtab
1015diff --git a/tests/cloud_tests/testcases/modules/ntp_chrony.yaml b/tests/cloud_tests/testcases/modules/ntp_chrony.yaml
1016new file mode 100644
1017index 0000000..120735e
1018--- /dev/null
1019+++ b/tests/cloud_tests/testcases/modules/ntp_chrony.yaml
1020@@ -0,0 +1,17 @@
1021+#
1022+# ntp enabled, chrony selected, check conf file
1023+# as chrony won't start in a container
1024+#
1025+cloud_config: |
1026+ #cloud-config
1027+ ntp:
1028+ enabled: true
1029+ ntp_client: chrony
1030+collect_scripts:
1031+ chrony_conf: |
1032+ #!/bin/sh
1033+ set -- /etc/chrony.conf /etc/chrony/chrony.conf
1034+ for p in "$@"; do
1035+ [ -e "$p" ] && { cat "$p"; exit; }
1036+ done
1037+# vi: ts=4 expandtab
1038diff --git a/tests/cloud_tests/testcases/modules/ntp_pools.yaml b/tests/cloud_tests/testcases/modules/ntp_pools.yaml
1039index d490b22..60fa0fd 100644
1040--- a/tests/cloud_tests/testcases/modules/ntp_pools.yaml
1041+++ b/tests/cloud_tests/testcases/modules/ntp_pools.yaml
1042@@ -9,6 +9,7 @@ required_features:
1043 cloud_config: |
1044 #cloud-config
1045 ntp:
1046+ ntp_client: ntp
1047 pools:
1048 - 0.cloud-init.mypool
1049 - 1.cloud-init.mypool
1050diff --git a/tests/cloud_tests/testcases/modules/ntp_servers.yaml b/tests/cloud_tests/testcases/modules/ntp_servers.yaml
1051index 6b13b70..ee63667 100644
1052--- a/tests/cloud_tests/testcases/modules/ntp_servers.yaml
1053+++ b/tests/cloud_tests/testcases/modules/ntp_servers.yaml
1054@@ -6,6 +6,7 @@ required_features:
1055 cloud_config: |
1056 #cloud-config
1057 ntp:
1058+ ntp_client: ntp
1059 servers:
1060 - 172.16.15.14
1061 - 172.16.17.18
1062diff --git a/tests/cloud_tests/testcases/modules/ntp_timesyncd.py b/tests/cloud_tests/testcases/modules/ntp_timesyncd.py
1063new file mode 100644
1064index 0000000..eca750b
1065--- /dev/null
1066+++ b/tests/cloud_tests/testcases/modules/ntp_timesyncd.py
1067@@ -0,0 +1,15 @@
1068+# This file is part of cloud-init. See LICENSE file for license information.
1069+
1070+"""cloud-init Integration Test Verify Script."""
1071+from tests.cloud_tests.testcases import base
1072+
1073+
1074+class TestNtpTimesyncd(base.CloudTestCase):
1075+ """Test ntp module with systemd-timesyncd client"""
1076+
1077+ def test_timesyncd_entries(self):
1078+ """Test timesyncd config entries"""
1079+ out = self.get_data_file('timesyncd_conf')
1080+ self.assertIn('.pool.ntp.org', out)
1081+
1082+# vi: ts=4 expandtab
1083diff --git a/tests/cloud_tests/testcases/modules/ntp_timesyncd.yaml b/tests/cloud_tests/testcases/modules/ntp_timesyncd.yaml
1084new file mode 100644
1085index 0000000..ee47a74
1086--- /dev/null
1087+++ b/tests/cloud_tests/testcases/modules/ntp_timesyncd.yaml
1088@@ -0,0 +1,15 @@
1089+#
1090+# ntp enabled, systemd-timesyncd selected, check conf file
1091+# as systemd-timesyncd won't start in a container
1092+#
1093+cloud_config: |
1094+ #cloud-config
1095+ ntp:
1096+ enabled: true
1097+ ntp_client: systemd-timesyncd
1098+collect_scripts:
1099+ timesyncd_conf: |
1100+ #!/bin/sh
1101+ cat /etc/systemd/timesyncd.conf.d/cloud-init.conf
1102+
1103+# vi: ts=4 expandtab
1104diff --git a/tests/unittests/test_distros/test_netconfig.py b/tests/unittests/test_distros/test_netconfig.py
1105index 1c2e45f..7765e40 100644
1106--- a/tests/unittests/test_distros/test_netconfig.py
1107+++ b/tests/unittests/test_distros/test_netconfig.py
1108@@ -189,6 +189,12 @@ hn0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
1109 status: active
1110 """
1111
1112+ def setUp(self):
1113+ super(TestNetCfgDistro, self).setUp()
1114+ self.add_patch('cloudinit.util.system_is_snappy', 'm_snappy')
1115+ self.add_patch('cloudinit.util.system_info', 'm_sysinfo')
1116+ self.m_sysinfo.return_value = {'dist': ('Distro', '99.1', 'Codename')}
1117+
1118 def _get_distro(self, dname, renderers=None):
1119 cls = distros.fetch(dname)
1120 cfg = settings.CFG_BUILTIN
1121diff --git a/tests/unittests/test_distros/test_user_data_normalize.py b/tests/unittests/test_distros/test_user_data_normalize.py
1122index 0fa9cdb..fa4b6cf 100644
1123--- a/tests/unittests/test_distros/test_user_data_normalize.py
1124+++ b/tests/unittests/test_distros/test_user_data_normalize.py
1125@@ -22,6 +22,12 @@ bcfg = {
1126
1127 class TestUGNormalize(TestCase):
1128
1129+ def setUp(self):
1130+ super(TestUGNormalize, self).setUp()
1131+ self.add_patch('cloudinit.util.system_is_snappy', 'm_snappy')
1132+ self.add_patch('cloudinit.util.system_info', 'm_sysinfo')
1133+ self.m_sysinfo.return_value = {'dist': ('Distro', '99.1', 'Codename')}
1134+
1135 def _make_distro(self, dtype, def_user=None):
1136 cfg = dict(settings.CFG_BUILTIN)
1137 cfg['system_info']['distro'] = dtype
1138diff --git a/tests/unittests/test_handler/test_handler_ntp.py b/tests/unittests/test_handler/test_handler_ntp.py
1139index 695897c..1b3ca57 100644
1140--- a/tests/unittests/test_handler/test_handler_ntp.py
1141+++ b/tests/unittests/test_handler/test_handler_ntp.py
1142@@ -4,20 +4,21 @@ from cloudinit.config import cc_ntp
1143 from cloudinit.sources import DataSourceNone
1144 from cloudinit import (distros, helpers, cloud, util)
1145 from cloudinit.tests.helpers import (
1146- FilesystemMockingTestCase, mock, skipUnlessJsonSchema)
1147+ CiTestCase, FilesystemMockingTestCase, mock, skipUnlessJsonSchema)
1148
1149
1150+import copy
1151 import os
1152 from os.path import dirname
1153 import shutil
1154
1155-NTP_TEMPLATE = b"""\
1156+NTP_TEMPLATE = """\
1157 ## template: jinja
1158 servers {{servers}}
1159 pools {{pools}}
1160 """
1161
1162-TIMESYNCD_TEMPLATE = b"""\
1163+TIMESYNCD_TEMPLATE = """\
1164 ## template:jinja
1165 [Time]
1166 {% if servers or pools -%}
1167@@ -32,56 +33,88 @@ class TestNtp(FilesystemMockingTestCase):
1168
1169 def setUp(self):
1170 super(TestNtp, self).setUp()
1171- self.subp = util.subp
1172 self.new_root = self.tmp_dir()
1173+ self.add_patch('cloudinit.util.system_is_snappy', 'm_snappy')
1174+ self.m_snappy.return_value = False
1175+ self.add_patch('cloudinit.util.system_info', 'm_sysinfo')
1176+ self.m_sysinfo.return_value = {'dist': ('Distro', '99.1', 'Codename')}
1177
1178- def _get_cloud(self, distro):
1179- self.patchUtils(self.new_root)
1180+ def _get_cloud(self, distro, sys_cfg=None):
1181+ self.new_root = self.reRoot(root=self.new_root)
1182 paths = helpers.Paths({'templates_dir': self.new_root})
1183 cls = distros.fetch(distro)
1184- mydist = cls(distro, {}, paths)
1185- myds = DataSourceNone.DataSourceNone({}, mydist, paths)
1186- return cloud.Cloud(myds, paths, {}, mydist, None)
1187+ if not sys_cfg:
1188+ sys_cfg = {}
1189+ mydist = cls(distro, sys_cfg, paths)
1190+ myds = DataSourceNone.DataSourceNone(sys_cfg, mydist, paths)
1191+ return cloud.Cloud(myds, paths, sys_cfg, mydist, None)
1192+
1193+ def _get_template_path(self, template_name, distro, basepath=None):
1194+ # ntp.conf.{distro} -> ntp.conf.debian.tmpl
1195+ template_fn = '{0}.tmpl'.format(
1196+ template_name.replace('{distro}', distro))
1197+ if not basepath:
1198+ basepath = self.new_root
1199+ path = os.path.join(basepath, template_fn)
1200+ return path
1201+
1202+ def _generate_template(self, template=None):
1203+ if not template:
1204+ template = NTP_TEMPLATE
1205+ confpath = os.path.join(self.new_root, 'client.conf')
1206+ template_fn = os.path.join(self.new_root, 'client.conf.tmpl')
1207+ util.write_file(template_fn, content=template)
1208+ return (confpath, template_fn)
1209+
1210+ def _mock_ntp_client_config(self, client=None, distro=None):
1211+ if not client:
1212+ client = 'ntp'
1213+ if not distro:
1214+ distro = 'ubuntu'
1215+ dcfg = cc_ntp.distro_ntp_client_configs(distro)
1216+ if client == 'systemd-timesyncd':
1217+ template = TIMESYNCD_TEMPLATE
1218+ else:
1219+ template = NTP_TEMPLATE
1220+ (confpath, template_fn) = self._generate_template(template=template)
1221+ ntpconfig = copy.deepcopy(dcfg[client])
1222+ ntpconfig['confpath'] = confpath
1223+ ntpconfig['template_name'] = os.path.basename(confpath)
1224+ return ntpconfig
1225
1226 @mock.patch("cloudinit.config.cc_ntp.util")
1227 def test_ntp_install(self, mock_util):
1228- """ntp_install installs via install_func when check_exe is absent."""
1229+ """ntp_install_client runs install_func when check_exe is absent."""
1230 mock_util.which.return_value = None # check_exe not found.
1231 install_func = mock.MagicMock()
1232- cc_ntp.install_ntp(install_func, packages=['ntpx'], check_exe='ntpdx')
1233-
1234+ cc_ntp.install_ntp_client(install_func,
1235+ packages=['ntpx'], check_exe='ntpdx')
1236 mock_util.which.assert_called_with('ntpdx')
1237 install_func.assert_called_once_with(['ntpx'])
1238
1239 @mock.patch("cloudinit.config.cc_ntp.util")
1240 def test_ntp_install_not_needed(self, mock_util):
1241- """ntp_install doesn't attempt install when check_exe is found."""
1242- mock_util.which.return_value = ["/usr/sbin/ntpd"] # check_exe found.
1243+ """ntp_install_client doesn't install when check_exe is found."""
1244+ client = 'chrony'
1245+ mock_util.which.return_value = [client] # check_exe found.
1246 install_func = mock.MagicMock()
1247- cc_ntp.install_ntp(install_func, packages=['ntp'], check_exe='ntpd')
1248+ cc_ntp.install_ntp_client(install_func, packages=[client],
1249+ check_exe=client)
1250 install_func.assert_not_called()
1251
1252 @mock.patch("cloudinit.config.cc_ntp.util")
1253 def test_ntp_install_no_op_with_empty_pkg_list(self, mock_util):
1254- """ntp_install calls install_func with empty list"""
1255+ """ntp_install_client runs install_func with empty list"""
1256 mock_util.which.return_value = None # check_exe not found
1257 install_func = mock.MagicMock()
1258- cc_ntp.install_ntp(install_func, packages=[], check_exe='timesyncd')
1259+ cc_ntp.install_ntp_client(install_func, packages=[],
1260+ check_exe='timesyncd')
1261 install_func.assert_called_once_with([])
1262
1263- def test_ntp_rename_ntp_conf(self):
1264- """When NTP_CONF exists, rename_ntp moves it."""
1265- ntpconf = self.tmp_path("ntp.conf", self.new_root)
1266- util.write_file(ntpconf, "")
1267- with mock.patch("cloudinit.config.cc_ntp.NTP_CONF", ntpconf):
1268- cc_ntp.rename_ntp_conf()
1269- self.assertFalse(os.path.exists(ntpconf))
1270- self.assertTrue(os.path.exists("{0}.dist".format(ntpconf)))
1271-
1272 @mock.patch("cloudinit.config.cc_ntp.util")
1273 def test_reload_ntp_defaults(self, mock_util):
1274 """Test service is restarted/reloaded (defaults)"""
1275- service = 'ntp'
1276+ service = 'ntp_service_name'
1277 cmd = ['service', service, 'restart']
1278 cc_ntp.reload_ntp(service)
1279 mock_util.subp.assert_called_with(cmd, capture=True)
1280@@ -89,193 +122,171 @@ class TestNtp(FilesystemMockingTestCase):
1281 @mock.patch("cloudinit.config.cc_ntp.util")
1282 def test_reload_ntp_systemd(self, mock_util):
1283 """Test service is restarted/reloaded (systemd)"""
1284- service = 'ntp'
1285- cmd = ['systemctl', 'reload-or-restart', service]
1286+ service = 'ntp_service_name'
1287 cc_ntp.reload_ntp(service, systemd=True)
1288- mock_util.subp.assert_called_with(cmd, capture=True)
1289-
1290- @mock.patch("cloudinit.config.cc_ntp.util")
1291- def test_reload_ntp_systemd_timesycnd(self, mock_util):
1292- """Test service is restarted/reloaded (systemd/timesyncd)"""
1293- service = 'systemd-timesycnd'
1294 cmd = ['systemctl', 'reload-or-restart', service]
1295- cc_ntp.reload_ntp(service, systemd=True)
1296 mock_util.subp.assert_called_with(cmd, capture=True)
1297
1298+ def test_ntp_rename_ntp_conf(self):
1299+ """When NTP_CONF exists, rename_ntp moves it."""
1300+ ntpconf = self.tmp_path("ntp.conf", self.new_root)
1301+ util.write_file(ntpconf, "")
1302+ cc_ntp.rename_ntp_conf(confpath=ntpconf)
1303+ self.assertFalse(os.path.exists(ntpconf))
1304+ self.assertTrue(os.path.exists("{0}.dist".format(ntpconf)))
1305+
1306 def test_ntp_rename_ntp_conf_skip_missing(self):
1307 """When NTP_CONF doesn't exist rename_ntp doesn't create a file."""
1308 ntpconf = self.tmp_path("ntp.conf", self.new_root)
1309 self.assertFalse(os.path.exists(ntpconf))
1310- with mock.patch("cloudinit.config.cc_ntp.NTP_CONF", ntpconf):
1311- cc_ntp.rename_ntp_conf()
1312+ cc_ntp.rename_ntp_conf(confpath=ntpconf)
1313 self.assertFalse(os.path.exists("{0}.dist".format(ntpconf)))
1314 self.assertFalse(os.path.exists(ntpconf))
1315
1316- def test_write_ntp_config_template_from_ntp_conf_tmpl_with_servers(self):
1317- """write_ntp_config_template reads content from ntp.conf.tmpl.
1318-
1319- It reads ntp.conf.tmpl if present and renders the value from servers
1320- key. When no pools key is defined, template is rendered using an empty
1321- list for pools.
1322- """
1323- distro = 'ubuntu'
1324- cfg = {
1325- 'servers': ['192.168.2.1', '192.168.2.2']
1326- }
1327- mycloud = self._get_cloud(distro)
1328- ntp_conf = self.tmp_path("ntp.conf", self.new_root) # Doesn't exist
1329- # Create ntp.conf.tmpl
1330- with open('{0}.tmpl'.format(ntp_conf), 'wb') as stream:
1331- stream.write(NTP_TEMPLATE)
1332- with mock.patch('cloudinit.config.cc_ntp.NTP_CONF', ntp_conf):
1333- cc_ntp.write_ntp_config_template(cfg, mycloud, ntp_conf)
1334- content = util.read_file_or_url('file://' + ntp_conf).contents
1335+ def test_write_ntp_config_template_uses_ntp_conf_distro_no_servers(self):
1336+ """write_ntp_config_template reads from $client.conf.distro.tmpl"""
1337+ servers = []
1338+ pools = ['10.0.0.1', '10.0.0.2']
1339+ (confpath, template_fn) = self._generate_template()
1340+ mock_path = 'cloudinit.config.cc_ntp.temp_utils._TMPDIR'
1341+ with mock.patch(mock_path, self.new_root):
1342+ cc_ntp.write_ntp_config_template('ubuntu',
1343+ servers=servers, pools=pools,
1344+ path=confpath,
1345+ template_fn=template_fn,
1346+ template=None)
1347+ content = util.read_file_or_url('file://' + confpath).contents
1348 self.assertEqual(
1349- "servers ['192.168.2.1', '192.168.2.2']\npools []\n",
1350- content.decode())
1351+ "servers []\npools ['10.0.0.1', '10.0.0.2']\n", content.decode())
1352
1353- def test_write_ntp_config_template_uses_ntp_conf_distro_no_servers(self):
1354- """write_ntp_config_template reads content from ntp.conf.distro.tmpl.
1355+ def test_write_ntp_config_template_defaults_pools_w_empty_lists(self):
1356+ """write_ntp_config_template defaults pools servers upon empty config.
1357
1358- It reads ntp.conf.<distro>.tmpl before attempting ntp.conf.tmpl. It
1359- renders the value from the keys servers and pools. When no
1360- servers value is present, template is rendered using an empty list.
1361+ When both pools and servers are empty, default NR_POOL_SERVERS get
1362+ configured.
1363 """
1364 distro = 'ubuntu'
1365- cfg = {
1366- 'pools': ['10.0.0.1', '10.0.0.2']
1367- }
1368- mycloud = self._get_cloud(distro)
1369- ntp_conf = self.tmp_path('ntp.conf', self.new_root) # Doesn't exist
1370- # Create ntp.conf.tmpl which isn't read
1371- with open('{0}.tmpl'.format(ntp_conf), 'wb') as stream:
1372- stream.write(b'NOT READ: ntp.conf.<distro>.tmpl is primary')
1373- # Create ntp.conf.tmpl.<distro>
1374- with open('{0}.{1}.tmpl'.format(ntp_conf, distro), 'wb') as stream:
1375- stream.write(NTP_TEMPLATE)
1376- with mock.patch('cloudinit.config.cc_ntp.NTP_CONF', ntp_conf):
1377- cc_ntp.write_ntp_config_template(cfg, mycloud, ntp_conf)
1378- content = util.read_file_or_url('file://' + ntp_conf).contents
1379+ pools = cc_ntp.generate_server_names(distro)
1380+ servers = []
1381+ (confpath, template_fn) = self._generate_template()
1382+ mock_path = 'cloudinit.config.cc_ntp.temp_utils._TMPDIR'
1383+ with mock.patch(mock_path, self.new_root):
1384+ cc_ntp.write_ntp_config_template(distro,
1385+ servers=servers, pools=pools,
1386+ path=confpath,
1387+ template_fn=template_fn,
1388+ template=None)
1389+ content = util.read_file_or_url('file://' + confpath).contents
1390 self.assertEqual(
1391- "servers []\npools ['10.0.0.1', '10.0.0.2']\n",
1392+ "servers []\npools {0}\n".format(pools),
1393 content.decode())
1394
1395- def test_write_ntp_config_template_defaults_pools_when_empty_lists(self):
1396- """write_ntp_config_template defaults pools servers upon empty config.
1397+ def test_defaults_pools_empty_lists_sles(self):
1398+ """write_ntp_config_template defaults opensuse pools upon empty config.
1399
1400 When both pools and servers are empty, default NR_POOL_SERVERS get
1401 configured.
1402 """
1403- distro = 'ubuntu'
1404- mycloud = self._get_cloud(distro)
1405- ntp_conf = self.tmp_path('ntp.conf', self.new_root) # Doesn't exist
1406- # Create ntp.conf.tmpl
1407- with open('{0}.tmpl'.format(ntp_conf), 'wb') as stream:
1408- stream.write(NTP_TEMPLATE)
1409- with mock.patch('cloudinit.config.cc_ntp.NTP_CONF', ntp_conf):
1410- cc_ntp.write_ntp_config_template({}, mycloud, ntp_conf)
1411- content = util.read_file_or_url('file://' + ntp_conf).contents
1412- default_pools = [
1413- "{0}.{1}.pool.ntp.org".format(x, distro)
1414- for x in range(0, cc_ntp.NR_POOL_SERVERS)]
1415+ distro = 'sles'
1416+ default_pools = cc_ntp.generate_server_names(distro)
1417+ (confpath, template_fn) = self._generate_template()
1418+
1419+ cc_ntp.write_ntp_config_template(distro,
1420+ servers=[], pools=[],
1421+ path=confpath,
1422+ template_fn=template_fn,
1423+ template=None)
1424+ content = util.read_file_or_url('file://' + confpath).contents
1425+ for pool in default_pools:
1426+ self.assertIn('opensuse', pool)
1427 self.assertEqual(
1428- "servers []\npools {0}\n".format(default_pools),
1429- content.decode())
1430+ "servers []\npools {0}\n".format(default_pools), content.decode())
1431 self.assertIn(
1432 "Adding distro default ntp pool servers: {0}".format(
1433 ",".join(default_pools)),
1434 self.logs.getvalue())
1435
1436- @mock.patch("cloudinit.config.cc_ntp.ntp_installable")
1437- def test_ntp_handler_mocked_template(self, m_ntp_install):
1438- """Test ntp handler renders ubuntu ntp.conf template."""
1439- pools = ['0.mycompany.pool.ntp.org', '3.mycompany.pool.ntp.org']
1440- servers = ['192.168.23.3', '192.168.23.4']
1441- cfg = {
1442- 'ntp': {
1443- 'pools': pools,
1444- 'servers': servers
1445- }
1446- }
1447- mycloud = self._get_cloud('ubuntu')
1448- ntp_conf = self.tmp_path('ntp.conf', self.new_root) # Doesn't exist
1449- m_ntp_install.return_value = True
1450-
1451- # Create ntp.conf.tmpl
1452- with open('{0}.tmpl'.format(ntp_conf), 'wb') as stream:
1453- stream.write(NTP_TEMPLATE)
1454- with mock.patch('cloudinit.config.cc_ntp.NTP_CONF', ntp_conf):
1455- with mock.patch.object(util, 'which', return_value=None):
1456- cc_ntp.handle('notimportant', cfg, mycloud, None, None)
1457-
1458- content = util.read_file_or_url('file://' + ntp_conf).contents
1459- self.assertEqual(
1460- 'servers {0}\npools {1}\n'.format(servers, pools),
1461- content.decode())
1462-
1463- @mock.patch("cloudinit.config.cc_ntp.util")
1464- def test_ntp_handler_mocked_template_snappy(self, m_util):
1465- """Test ntp handler renders timesycnd.conf template on snappy."""
1466+ def test_timesyncd_template(self):
1467+ """Test timesycnd template is correct"""
1468 pools = ['0.mycompany.pool.ntp.org', '3.mycompany.pool.ntp.org']
1469 servers = ['192.168.23.3', '192.168.23.4']
1470- cfg = {
1471- 'ntp': {
1472- 'pools': pools,
1473- 'servers': servers
1474- }
1475- }
1476- mycloud = self._get_cloud('ubuntu')
1477- m_util.system_is_snappy.return_value = True
1478-
1479- # Create timesyncd.conf.tmpl
1480- tsyncd_conf = self.tmp_path("timesyncd.conf", self.new_root)
1481- template = '{0}.tmpl'.format(tsyncd_conf)
1482- with open(template, 'wb') as stream:
1483- stream.write(TIMESYNCD_TEMPLATE)
1484-
1485- with mock.patch('cloudinit.config.cc_ntp.TIMESYNCD_CONF', tsyncd_conf):
1486- cc_ntp.handle('notimportant', cfg, mycloud, None, None)
1487-
1488- content = util.read_file_or_url('file://' + tsyncd_conf).contents
1489+ (confpath, template_fn) = self._generate_template(
1490+ template=TIMESYNCD_TEMPLATE)
1491+ cc_ntp.write_ntp_config_template('ubuntu',
1492+ servers=servers, pools=pools,
1493+ path=confpath,
1494+ template_fn=template_fn,
1495+ template=None)
1496+ content = util.read_file_or_url('file://' + confpath).contents
1497 self.assertEqual(
1498 "[Time]\nNTP=%s %s \n" % (" ".join(servers), " ".join(pools)),
1499 content.decode())
1500
1501- def test_ntp_handler_real_distro_templates(self):
1502- """Test ntp handler renders the shipped distro ntp.conf templates."""
1503+ def test_distro_ntp_client_configs(self):
1504+ """Test we have updated ntp client configs on different distros"""
1505+ delta = copy.deepcopy(cc_ntp.DISTRO_CLIENT_CONFIG)
1506+ base = copy.deepcopy(cc_ntp.NTP_CLIENT_CONFIG)
1507+ # confirm no-delta distros match the base config
1508+ for distro in cc_ntp.distros:
1509+ if distro not in delta:
1510+ result = cc_ntp.distro_ntp_client_configs(distro)
1511+ self.assertEqual(base, result)
1512+ # for distros with delta, ensure the merged config values match
1513+ # what is set in the delta
1514+ for distro in delta.keys():
1515+ result = cc_ntp.distro_ntp_client_configs(distro)
1516+ for client in delta[distro].keys():
1517+ for key in delta[distro][client].keys():
1518+ self.assertEqual(delta[distro][client][key],
1519+ result[client][key])
1520+
1521+ def test_ntp_handler_real_distro_ntp_templates(self):
1522+ """Test ntp handler renders the shipped distro ntp client templates."""
1523 pools = ['0.mycompany.pool.ntp.org', '3.mycompany.pool.ntp.org']
1524 servers = ['192.168.23.3', '192.168.23.4']
1525- cfg = {
1526- 'ntp': {
1527- 'pools': pools,
1528- 'servers': servers
1529- }
1530- }
1531- ntp_conf = self.tmp_path('ntp.conf', self.new_root) # Doesn't exist
1532- for distro in ('debian', 'ubuntu', 'fedora', 'rhel', 'sles'):
1533- mycloud = self._get_cloud(distro)
1534- root_dir = dirname(dirname(os.path.realpath(util.__file__)))
1535- tmpl_file = os.path.join(
1536- '{0}/templates/ntp.conf.{1}.tmpl'.format(root_dir, distro))
1537- # Create a copy in our tmp_dir
1538- shutil.copy(
1539- tmpl_file,
1540- os.path.join(self.new_root, 'ntp.conf.%s.tmpl' % distro))
1541- with mock.patch('cloudinit.config.cc_ntp.NTP_CONF', ntp_conf):
1542- with mock.patch.object(util, 'which', return_value=[True]):
1543- cc_ntp.handle('notimportant', cfg, mycloud, None, None)
1544-
1545- content = util.read_file_or_url('file://' + ntp_conf).contents
1546- expected_servers = '\n'.join([
1547- 'server {0} iburst'.format(server) for server in servers])
1548- self.assertIn(
1549- expected_servers, content.decode(),
1550- 'failed to render ntp.conf for distro:{0}'.format(distro))
1551- expected_pools = '\n'.join([
1552- 'pool {0} iburst'.format(pool) for pool in pools])
1553- self.assertIn(
1554- expected_pools, content.decode(),
1555- 'failed to render ntp.conf for distro:{0}'.format(distro))
1556+ for client in ['ntp', 'systemd-timesyncd', 'chrony']:
1557+ for distro in cc_ntp.distros:
1558+ distro_cfg = cc_ntp.distro_ntp_client_configs(distro)
1559+ ntpclient = distro_cfg[client]
1560+ confpath = (
1561+ os.path.join(self.new_root, ntpclient.get('confpath')[1:]))
1562+ template = ntpclient.get('template_name')
1563+ # find sourcetree template file
1564+ root_dir = (
1565+ dirname(dirname(os.path.realpath(util.__file__))) +
1566+ '/templates')
1567+ source_fn = self._get_template_path(template, distro,
1568+ basepath=root_dir)
1569+ template_fn = self._get_template_path(template, distro)
1570+ # don't fail if cloud-init doesn't have a template for
1571+ # a distro,client pair
1572+ if not os.path.exists(source_fn):
1573+ continue
1574+ # Create a copy in our tmp_dir
1575+ shutil.copy(source_fn, template_fn)
1576+ cc_ntp.write_ntp_config_template(distro, servers=servers,
1577+ pools=pools, path=confpath,
1578+ template_fn=template_fn)
1579+ content = util.read_file_or_url('file://' + confpath).contents
1580+ if client in ['ntp', 'chrony']:
1581+ expected_servers = '\n'.join([
1582+ 'server {0} iburst'.format(srv) for srv in servers])
1583+ print('distro=%s client=%s' % (distro, client))
1584+ self.assertIn(expected_servers, content.decode(),
1585+ ('failed to render {0} conf'
1586+ ' for distro:{1}'.format(client, distro)))
1587+ expected_pools = '\n'.join([
1588+ 'pool {0} iburst'.format(pool) for pool in pools])
1589+ self.assertIn(expected_pools, content.decode(),
1590+ ('failed to render {0} conf'
1591+ ' for distro:{1}'.format(client, distro)))
1592+ elif client == 'systemd-timesyncd':
1593+ expected_content = (
1594+ "# cloud-init generated file\n" +
1595+ "# See timesyncd.conf(5) for details.\n\n" +
1596+ "[Time]\nNTP=%s %s \n" % (" ".join(servers),
1597+ " ".join(pools)))
1598+ self.assertEqual(expected_content, content.decode())
1599
1600 def test_no_ntpcfg_does_nothing(self):
1601 """When no ntp section is defined handler logs a warning and noops."""
1602@@ -285,95 +296,99 @@ class TestNtp(FilesystemMockingTestCase):
1603 'not present or disabled by cfg\n',
1604 self.logs.getvalue())
1605
1606- def test_ntp_handler_schema_validation_allows_empty_ntp_config(self):
1607+ @mock.patch('cloudinit.config.cc_ntp.select_ntp_client')
1608+ def test_ntp_handler_schema_validation_allows_empty_ntp_config(self,
1609+ m_select):
1610 """Ntp schema validation allows for an empty ntp: configuration."""
1611 valid_empty_configs = [{'ntp': {}}, {'ntp': None}]
1612- distro = 'ubuntu'
1613- cc = self._get_cloud(distro)
1614- ntp_conf = os.path.join(self.new_root, 'ntp.conf')
1615- with open('{0}.tmpl'.format(ntp_conf), 'wb') as stream:
1616- stream.write(NTP_TEMPLATE)
1617 for valid_empty_config in valid_empty_configs:
1618- with mock.patch('cloudinit.config.cc_ntp.NTP_CONF', ntp_conf):
1619- cc_ntp.handle('cc_ntp', valid_empty_config, cc, None, [])
1620- with open(ntp_conf) as stream:
1621- content = stream.read()
1622- default_pools = [
1623- "{0}.{1}.pool.ntp.org".format(x, distro)
1624- for x in range(0, cc_ntp.NR_POOL_SERVERS)]
1625- self.assertEqual(
1626- "servers []\npools {0}\n".format(default_pools),
1627- content)
1628- self.assertNotIn('Invalid config:', self.logs.getvalue())
1629+ for distro in cc_ntp.distros:
1630+ mycloud = self._get_cloud(distro)
1631+ ntpconfig = self._mock_ntp_client_config(distro=distro)
1632+ confpath = ntpconfig['confpath']
1633+ m_select.return_value = ntpconfig
1634+ cc_ntp.handle('cc_ntp', valid_empty_config, mycloud, None, [])
1635+ content = util.read_file_or_url('file://' + confpath).contents
1636+ pools = cc_ntp.generate_server_names(mycloud.distro.name)
1637+ self.assertEqual(
1638+ "servers []\npools {0}\n".format(pools), content.decode())
1639+ self.assertNotIn('Invalid config:', self.logs.getvalue())
1640
1641 @skipUnlessJsonSchema()
1642- def test_ntp_handler_schema_validation_warns_non_string_item_type(self):
1643+ @mock.patch('cloudinit.config.cc_ntp.select_ntp_client')
1644+ def test_ntp_handler_schema_validation_warns_non_string_item_type(self,
1645+ m_sel):
1646 """Ntp schema validation warns of non-strings in pools or servers.
1647
1648 Schema validation is not strict, so ntp config is still be rendered.
1649 """
1650 invalid_config = {'ntp': {'pools': [123], 'servers': ['valid', None]}}
1651- cc = self._get_cloud('ubuntu')
1652- ntp_conf = os.path.join(self.new_root, 'ntp.conf')
1653- with open('{0}.tmpl'.format(ntp_conf), 'wb') as stream:
1654- stream.write(NTP_TEMPLATE)
1655- with mock.patch('cloudinit.config.cc_ntp.NTP_CONF', ntp_conf):
1656- cc_ntp.handle('cc_ntp', invalid_config, cc, None, [])
1657- self.assertIn(
1658- "Invalid config:\nntp.pools.0: 123 is not of type 'string'\n"
1659- "ntp.servers.1: None is not of type 'string'",
1660- self.logs.getvalue())
1661- with open(ntp_conf) as stream:
1662- content = stream.read()
1663- self.assertEqual("servers ['valid', None]\npools [123]\n", content)
1664+ for distro in cc_ntp.distros:
1665+ mycloud = self._get_cloud(distro)
1666+ ntpconfig = self._mock_ntp_client_config(distro=distro)
1667+ confpath = ntpconfig['confpath']
1668+ m_sel.return_value = ntpconfig
1669+ cc_ntp.handle('cc_ntp', invalid_config, mycloud, None, [])
1670+ self.assertIn(
1671+ "Invalid config:\nntp.pools.0: 123 is not of type 'string'\n"
1672+ "ntp.servers.1: None is not of type 'string'",
1673+ self.logs.getvalue())
1674+ content = util.read_file_or_url('file://' + confpath).contents
1675+ self.assertEqual("servers ['valid', None]\npools [123]\n",
1676+ content.decode())
1677
1678 @skipUnlessJsonSchema()
1679- def test_ntp_handler_schema_validation_warns_of_non_array_type(self):
1680+ @mock.patch('cloudinit.config.cc_ntp.select_ntp_client')
1681+ def test_ntp_handler_schema_validation_warns_of_non_array_type(self,
1682+ m_select):
1683 """Ntp schema validation warns of non-array pools or servers types.
1684
1685 Schema validation is not strict, so ntp config is still be rendered.
1686 """
1687 invalid_config = {'ntp': {'pools': 123, 'servers': 'non-array'}}
1688- cc = self._get_cloud('ubuntu')
1689- ntp_conf = os.path.join(self.new_root, 'ntp.conf')
1690- with open('{0}.tmpl'.format(ntp_conf), 'wb') as stream:
1691- stream.write(NTP_TEMPLATE)
1692- with mock.patch('cloudinit.config.cc_ntp.NTP_CONF', ntp_conf):
1693- cc_ntp.handle('cc_ntp', invalid_config, cc, None, [])
1694- self.assertIn(
1695- "Invalid config:\nntp.pools: 123 is not of type 'array'\n"
1696- "ntp.servers: 'non-array' is not of type 'array'",
1697- self.logs.getvalue())
1698- with open(ntp_conf) as stream:
1699- content = stream.read()
1700- self.assertEqual("servers non-array\npools 123\n", content)
1701+
1702+ for distro in cc_ntp.distros:
1703+ mycloud = self._get_cloud(distro)
1704+ ntpconfig = self._mock_ntp_client_config(distro=distro)
1705+ confpath = ntpconfig['confpath']
1706+ m_select.return_value = ntpconfig
1707+ cc_ntp.handle('cc_ntp', invalid_config, mycloud, None, [])
1708+ self.assertIn(
1709+ "Invalid config:\nntp.pools: 123 is not of type 'array'\n"
1710+ "ntp.servers: 'non-array' is not of type 'array'",
1711+ self.logs.getvalue())
1712+ content = util.read_file_or_url('file://' + confpath).contents
1713+ self.assertEqual("servers non-array\npools 123\n",
1714+ content.decode())
1715
1716 @skipUnlessJsonSchema()
1717- def test_ntp_handler_schema_validation_warns_invalid_key_present(self):
1718+ @mock.patch('cloudinit.config.cc_ntp.select_ntp_client')
1719+ def test_ntp_handler_schema_validation_warns_invalid_key_present(self,
1720+ m_select):
1721 """Ntp schema validation warns of invalid keys present in ntp config.
1722
1723 Schema validation is not strict, so ntp config is still be rendered.
1724 """
1725 invalid_config = {
1726 'ntp': {'invalidkey': 1, 'pools': ['0.mycompany.pool.ntp.org']}}
1727- cc = self._get_cloud('ubuntu')
1728- ntp_conf = os.path.join(self.new_root, 'ntp.conf')
1729- with open('{0}.tmpl'.format(ntp_conf), 'wb') as stream:
1730- stream.write(NTP_TEMPLATE)
1731- with mock.patch('cloudinit.config.cc_ntp.NTP_CONF', ntp_conf):
1732- cc_ntp.handle('cc_ntp', invalid_config, cc, None, [])
1733- self.assertIn(
1734- "Invalid config:\nntp: Additional properties are not allowed "
1735- "('invalidkey' was unexpected)",
1736- self.logs.getvalue())
1737- with open(ntp_conf) as stream:
1738- content = stream.read()
1739- self.assertEqual(
1740- "servers []\npools ['0.mycompany.pool.ntp.org']\n",
1741- content)
1742+ for distro in cc_ntp.distros:
1743+ mycloud = self._get_cloud(distro)
1744+ ntpconfig = self._mock_ntp_client_config(distro=distro)
1745+ confpath = ntpconfig['confpath']
1746+ m_select.return_value = ntpconfig
1747+ cc_ntp.handle('cc_ntp', invalid_config, mycloud, None, [])
1748+ self.assertIn(
1749+ "Invalid config:\nntp: Additional properties are not allowed "
1750+ "('invalidkey' was unexpected)",
1751+ self.logs.getvalue())
1752+ content = util.read_file_or_url('file://' + confpath).contents
1753+ self.assertEqual(
1754+ "servers []\npools ['0.mycompany.pool.ntp.org']\n",
1755+ content.decode())
1756
1757 @skipUnlessJsonSchema()
1758- def test_ntp_handler_schema_validation_warns_of_duplicates(self):
1759+ @mock.patch('cloudinit.config.cc_ntp.select_ntp_client')
1760+ def test_ntp_handler_schema_validation_warns_of_duplicates(self, m_select):
1761 """Ntp schema validation warns of duplicates in servers or pools.
1762
1763 Schema validation is not strict, so ntp config is still be rendered.
1764@@ -381,74 +396,334 @@ class TestNtp(FilesystemMockingTestCase):
1765 invalid_config = {
1766 'ntp': {'pools': ['0.mypool.org', '0.mypool.org'],
1767 'servers': ['10.0.0.1', '10.0.0.1']}}
1768- cc = self._get_cloud('ubuntu')
1769- ntp_conf = os.path.join(self.new_root, 'ntp.conf')
1770- with open('{0}.tmpl'.format(ntp_conf), 'wb') as stream:
1771- stream.write(NTP_TEMPLATE)
1772- with mock.patch('cloudinit.config.cc_ntp.NTP_CONF', ntp_conf):
1773- cc_ntp.handle('cc_ntp', invalid_config, cc, None, [])
1774- self.assertIn(
1775- "Invalid config:\nntp.pools: ['0.mypool.org', '0.mypool.org'] has "
1776- "non-unique elements\nntp.servers: ['10.0.0.1', '10.0.0.1'] has "
1777- "non-unique elements",
1778- self.logs.getvalue())
1779- with open(ntp_conf) as stream:
1780- content = stream.read()
1781- self.assertEqual(
1782- "servers ['10.0.0.1', '10.0.0.1']\n"
1783- "pools ['0.mypool.org', '0.mypool.org']\n",
1784- content)
1785+ for distro in cc_ntp.distros:
1786+ mycloud = self._get_cloud(distro)
1787+ ntpconfig = self._mock_ntp_client_config(distro=distro)
1788+ confpath = ntpconfig['confpath']
1789+ m_select.return_value = ntpconfig
1790+ cc_ntp.handle('cc_ntp', invalid_config, mycloud, None, [])
1791+ self.assertIn(
1792+ "Invalid config:\nntp.pools: ['0.mypool.org', '0.mypool.org']"
1793+ " has non-unique elements\nntp.servers: "
1794+ "['10.0.0.1', '10.0.0.1'] has non-unique elements",
1795+ self.logs.getvalue())
1796+ content = util.read_file_or_url('file://' + confpath).contents
1797+ self.assertEqual(
1798+ "servers ['10.0.0.1', '10.0.0.1']\n"
1799+ "pools ['0.mypool.org', '0.mypool.org']\n", content.decode())
1800
1801- @mock.patch("cloudinit.config.cc_ntp.ntp_installable")
1802- def test_ntp_handler_timesyncd(self, m_ntp_install):
1803+ @mock.patch('cloudinit.config.cc_ntp.select_ntp_client')
1804+ def test_ntp_handler_timesyncd(self, m_select):
1805 """Test ntp handler configures timesyncd"""
1806- m_ntp_install.return_value = False
1807- distro = 'ubuntu'
1808- cfg = {
1809- 'servers': ['192.168.2.1', '192.168.2.2'],
1810- 'pools': ['0.mypool.org'],
1811- }
1812- mycloud = self._get_cloud(distro)
1813- tsyncd_conf = self.tmp_path("timesyncd.conf", self.new_root)
1814- # Create timesyncd.conf.tmpl
1815- template = '{0}.tmpl'.format(tsyncd_conf)
1816- print(template)
1817- with open(template, 'wb') as stream:
1818- stream.write(TIMESYNCD_TEMPLATE)
1819- with mock.patch('cloudinit.config.cc_ntp.TIMESYNCD_CONF', tsyncd_conf):
1820- cc_ntp.write_ntp_config_template(cfg, mycloud, tsyncd_conf,
1821- template='timesyncd.conf')
1822-
1823- content = util.read_file_or_url('file://' + tsyncd_conf).contents
1824- self.assertEqual(
1825- "[Time]\nNTP=192.168.2.1 192.168.2.2 0.mypool.org \n",
1826- content.decode())
1827+ servers = ['192.168.2.1', '192.168.2.2']
1828+ pools = ['0.mypool.org']
1829+ cfg = {'ntp': {'servers': servers, 'pools': pools}}
1830+ client = 'systemd-timesyncd'
1831+ for distro in cc_ntp.distros:
1832+ mycloud = self._get_cloud(distro)
1833+ ntpconfig = self._mock_ntp_client_config(distro=distro,
1834+ client=client)
1835+ confpath = ntpconfig['confpath']
1836+ m_select.return_value = ntpconfig
1837+ cc_ntp.handle('cc_ntp', cfg, mycloud, None, [])
1838+ content = util.read_file_or_url('file://' + confpath).contents
1839+ self.assertEqual(
1840+ "[Time]\nNTP=192.168.2.1 192.168.2.2 0.mypool.org \n",
1841+ content.decode())
1842+
1843+ @mock.patch('cloudinit.config.cc_ntp.select_ntp_client')
1844+ def test_ntp_handler_enabled_false(self, m_select):
1845+ """Test ntp handler does not run if enabled: false """
1846+ cfg = {'ntp': {'enabled': False}}
1847+ for distro in cc_ntp.distros:
1848+ mycloud = self._get_cloud(distro)
1849+ cc_ntp.handle('notimportant', cfg, mycloud, None, None)
1850+ self.assertEqual(0, m_select.call_count)
1851+
1852+ @mock.patch('cloudinit.config.cc_ntp.select_ntp_client')
1853+ @mock.patch("cloudinit.distros.Distro.uses_systemd")
1854+ def test_ntp_the_whole_package(self, m_sysd, m_select):
1855+ """Test enabled config renders template, and restarts service """
1856+ cfg = {'ntp': {'enabled': True}}
1857+ for distro in cc_ntp.distros:
1858+ mycloud = self._get_cloud(distro)
1859+ ntpconfig = self._mock_ntp_client_config(distro=distro)
1860+ confpath = ntpconfig['confpath']
1861+ service_name = ntpconfig['service_name']
1862+ m_select.return_value = ntpconfig
1863+ pools = cc_ntp.generate_server_names(mycloud.distro.name)
1864+ # force uses systemd path
1865+ m_sysd.return_value = True
1866+ with mock.patch('cloudinit.config.cc_ntp.util') as m_util:
1867+ # allow use of util.mergemanydict
1868+ m_util.mergemanydict.side_effect = util.mergemanydict
1869+ # default client is present
1870+ m_util.which.return_value = True
1871+ # use the config 'enabled' value
1872+ m_util.is_false.return_value = util.is_false(
1873+ cfg['ntp']['enabled'])
1874+ cc_ntp.handle('notimportant', cfg, mycloud, None, None)
1875+ m_util.subp.assert_called_with(
1876+ ['systemctl', 'reload-or-restart',
1877+ service_name], capture=True)
1878+ content = util.read_file_or_url('file://' + confpath).contents
1879+ self.assertEqual(
1880+ "servers []\npools {0}\n".format(pools),
1881+ content.decode())
1882+
1883+ def test_opensuse_picks_chrony(self):
1884+ """Test opensuse picks chrony or ntp on certain distro versions"""
1885+ # < 15.0 => ntp
1886+ self.m_sysinfo.return_value = {'dist':
1887+ ('openSUSE', '13.2', 'Harlequin')}
1888+ mycloud = self._get_cloud('opensuse')
1889+ expected_client = mycloud.distro.preferred_ntp_clients[0]
1890+ self.assertEqual('ntp', expected_client)
1891+
1892+ # >= 15.0 and not openSUSE => chrony
1893+ self.m_sysinfo.return_value = {'dist':
1894+ ('SLES', '15.0',
1895+ 'SUSE Linux Enterprise Server 15')}
1896+ mycloud = self._get_cloud('sles')
1897+ expected_client = mycloud.distro.preferred_ntp_clients[0]
1898+ self.assertEqual('chrony', expected_client)
1899+
1900+ # >= 15.0 and openSUSE and ver != 42 => chrony
1901+ self.m_sysinfo.return_value = {'dist': ('openSUSE Tumbleweed',
1902+ '20180326',
1903+ 'timbleweed')}
1904+ mycloud = self._get_cloud('opensuse')
1905+ expected_client = mycloud.distro.preferred_ntp_clients[0]
1906+ self.assertEqual('chrony', expected_client)
1907+
1908+ def test_ubuntu_xenial_picks_ntp(self):
1909+ """Test Ubuntu picks ntp on xenial release"""
1910+
1911+ self.m_sysinfo.return_value = {'dist': ('Ubuntu', '16.04', 'xenial')}
1912+ mycloud = self._get_cloud('ubuntu')
1913+ expected_client = mycloud.distro.preferred_ntp_clients[0]
1914+ self.assertEqual('ntp', expected_client)
1915
1916- def test_write_ntp_config_template_defaults_pools_empty_lists_sles(self):
1917- """write_ntp_config_template defaults pools servers upon empty config.
1918+ @mock.patch('cloudinit.config.cc_ntp.util.which')
1919+ def test_snappy_system_picks_timesyncd(self, m_which):
1920+ """Test snappy systems prefer installed clients"""
1921
1922- When both pools and servers are empty, default NR_POOL_SERVERS get
1923- configured.
1924- """
1925- distro = 'sles'
1926- mycloud = self._get_cloud(distro)
1927- ntp_conf = self.tmp_path('ntp.conf', self.new_root) # Doesn't exist
1928- # Create ntp.conf.tmpl
1929- with open('{0}.tmpl'.format(ntp_conf), 'wb') as stream:
1930- stream.write(NTP_TEMPLATE)
1931- with mock.patch('cloudinit.config.cc_ntp.NTP_CONF', ntp_conf):
1932- cc_ntp.write_ntp_config_template({}, mycloud, ntp_conf)
1933- content = util.read_file_or_url('file://' + ntp_conf).contents
1934- default_pools = [
1935- "{0}.opensuse.pool.ntp.org".format(x)
1936- for x in range(0, cc_ntp.NR_POOL_SERVERS)]
1937- self.assertEqual(
1938- "servers []\npools {0}\n".format(default_pools),
1939- content.decode())
1940- self.assertIn(
1941- "Adding distro default ntp pool servers: {0}".format(
1942- ",".join(default_pools)),
1943- self.logs.getvalue())
1944+ # we are on ubuntu-core here
1945+ self.m_snappy.return_value = True
1946
1947+ # ubuntu core systems will have timesyncd installed
1948+ m_which.side_effect = iter([None, '/lib/systemd/systemd-timesyncd',
1949+ None, None, None])
1950+ distro = 'ubuntu'
1951+ mycloud = self._get_cloud(distro)
1952+ distro_configs = cc_ntp.distro_ntp_client_configs(distro)
1953+ expected_client = 'systemd-timesyncd'
1954+ expected_cfg = distro_configs[expected_client]
1955+ expected_calls = []
1956+ # we only get to timesyncd
1957+ for client in mycloud.distro.preferred_ntp_clients[0:2]:
1958+ cfg = distro_configs[client]
1959+ expected_calls.append(mock.call(cfg['check_exe']))
1960+ result = cc_ntp.select_ntp_client(None, mycloud.distro)
1961+ m_which.assert_has_calls(expected_calls)
1962+ self.assertEqual(sorted(expected_cfg), sorted(cfg))
1963+ self.assertEqual(sorted(expected_cfg), sorted(result))
1964+
1965+ @mock.patch('cloudinit.config.cc_ntp.util.which')
1966+ def test_ntp_distro_searches_all_preferred_clients(self, m_which):
1967+ """Test select_ntp_client search all distro perferred clients """
1968+ # nothing is installed
1969+ m_which.return_value = None
1970+ for distro in cc_ntp.distros:
1971+ mycloud = self._get_cloud(distro)
1972+ distro_configs = cc_ntp.distro_ntp_client_configs(distro)
1973+ expected_client = mycloud.distro.preferred_ntp_clients[0]
1974+ expected_cfg = distro_configs[expected_client]
1975+ expected_calls = []
1976+ for client in mycloud.distro.preferred_ntp_clients:
1977+ cfg = distro_configs[client]
1978+ expected_calls.append(mock.call(cfg['check_exe']))
1979+ cc_ntp.select_ntp_client({}, mycloud.distro)
1980+ m_which.assert_has_calls(expected_calls)
1981+ self.assertEqual(sorted(expected_cfg), sorted(cfg))
1982+
1983+ @mock.patch('cloudinit.config.cc_ntp.util.which')
1984+ def test_user_cfg_ntp_client_auto_uses_distro_clients(self, m_which):
1985+ """Test user_cfg.ntp_client='auto' defaults to distro search"""
1986+ # nothing is installed
1987+ m_which.return_value = None
1988+ for distro in cc_ntp.distros:
1989+ mycloud = self._get_cloud(distro)
1990+ distro_configs = cc_ntp.distro_ntp_client_configs(distro)
1991+ expected_client = mycloud.distro.preferred_ntp_clients[0]
1992+ expected_cfg = distro_configs[expected_client]
1993+ expected_calls = []
1994+ for client in mycloud.distro.preferred_ntp_clients:
1995+ cfg = distro_configs[client]
1996+ expected_calls.append(mock.call(cfg['check_exe']))
1997+ cc_ntp.select_ntp_client('auto', mycloud.distro)
1998+ m_which.assert_has_calls(expected_calls)
1999+ self.assertEqual(sorted(expected_cfg), sorted(cfg))
2000+
2001+ @mock.patch('cloudinit.config.cc_ntp.write_ntp_config_template')
2002+ @mock.patch('cloudinit.cloud.Cloud.get_template_filename')
2003+ @mock.patch('cloudinit.config.cc_ntp.util.which')
2004+ def test_ntp_custom_client_overrides_installed_clients(self, m_which,
2005+ m_tmpfn, m_write):
2006+ """Test user client is installed despite other clients present """
2007+ client = 'ntpdate'
2008+ cfg = {'ntp': {'ntp_client': client}}
2009+ for distro in cc_ntp.distros:
2010+ # client is not installed
2011+ m_which.side_effect = iter([None])
2012+ mycloud = self._get_cloud(distro)
2013+ with mock.patch.object(mycloud.distro,
2014+ 'install_packages') as m_install:
2015+ cc_ntp.handle('notimportant', cfg, mycloud, None, None)
2016+ m_install.assert_called_with([client])
2017+ m_which.assert_called_with(client)
2018+
2019+ @mock.patch('cloudinit.config.cc_ntp.util.which')
2020+ def test_ntp_system_config_overrides_distro_builtin_clients(self, m_which):
2021+ """Test distro system_config overrides builtin preferred ntp clients"""
2022+ system_client = 'chrony'
2023+ sys_cfg = {'ntp_client': system_client}
2024+ # no clients installed
2025+ m_which.return_value = None
2026+ for distro in cc_ntp.distros:
2027+ mycloud = self._get_cloud(distro, sys_cfg=sys_cfg)
2028+ distro_configs = cc_ntp.distro_ntp_client_configs(distro)
2029+ expected_cfg = distro_configs[system_client]
2030+ result = cc_ntp.select_ntp_client(None, mycloud.distro)
2031+ self.assertEqual(sorted(expected_cfg), sorted(result))
2032+ m_which.assert_has_calls([])
2033+
2034+ @mock.patch('cloudinit.config.cc_ntp.util.which')
2035+ def test_ntp_user_config_overrides_system_cfg(self, m_which):
2036+ """Test user-data overrides system_config ntp_client"""
2037+ system_client = 'chrony'
2038+ sys_cfg = {'ntp_client': system_client}
2039+ user_client = 'systemd-timesyncd'
2040+ # no clients installed
2041+ m_which.return_value = None
2042+ for distro in cc_ntp.distros:
2043+ mycloud = self._get_cloud(distro, sys_cfg=sys_cfg)
2044+ distro_configs = cc_ntp.distro_ntp_client_configs(distro)
2045+ expected_cfg = distro_configs[user_client]
2046+ result = cc_ntp.select_ntp_client(user_client, mycloud.distro)
2047+ self.assertEqual(sorted(expected_cfg), sorted(result))
2048+ m_which.assert_has_calls([])
2049+
2050+ @mock.patch('cloudinit.config.cc_ntp.reload_ntp')
2051+ @mock.patch('cloudinit.config.cc_ntp.install_ntp_client')
2052+ def test_ntp_user_provided_config_with_template(self, m_install, m_reload):
2053+ custom = r'\n#MyCustomTemplate'
2054+ user_template = NTP_TEMPLATE + custom
2055+ confpath = os.path.join(self.new_root, 'etc/myntp/myntp.conf')
2056+ cfg = {
2057+ 'ntp': {
2058+ 'pools': ['mypool.org'],
2059+ 'ntp_client': 'myntpd',
2060+ 'config': {
2061+ 'check_exe': 'myntpd',
2062+ 'confpath': confpath,
2063+ 'packages': ['myntp'],
2064+ 'service_name': 'myntp',
2065+ 'template': user_template,
2066+ }
2067+ }
2068+ }
2069+ for distro in cc_ntp.distros:
2070+ mycloud = self._get_cloud(distro)
2071+ mock_path = 'cloudinit.config.cc_ntp.temp_utils._TMPDIR'
2072+ with mock.patch(mock_path, self.new_root):
2073+ cc_ntp.handle('notimportant', cfg, mycloud, None, None)
2074+ content = util.read_file_or_url('file://' + confpath).contents
2075+ self.assertEqual(
2076+ "servers []\npools ['mypool.org']\n%s" % custom,
2077+ content.decode())
2078+
2079+ @mock.patch('cloudinit.config.cc_ntp.supplemental_schema_validation')
2080+ @mock.patch('cloudinit.config.cc_ntp.reload_ntp')
2081+ @mock.patch('cloudinit.config.cc_ntp.install_ntp_client')
2082+ @mock.patch('cloudinit.config.cc_ntp.select_ntp_client')
2083+ def test_ntp_user_provided_config_template_only(self, m_select, m_install,
2084+ m_reload, m_schema):
2085+ """Test custom template for default client"""
2086+ custom = r'\n#MyCustomTemplate'
2087+ user_template = NTP_TEMPLATE + custom
2088+ client = 'chrony'
2089+ cfg = {
2090+ 'pools': ['mypool.org'],
2091+ 'ntp_client': client,
2092+ 'config': {
2093+ 'template': user_template,
2094+ }
2095+ }
2096+ expected_merged_cfg = {
2097+ 'check_exe': 'chronyd',
2098+ 'confpath': '{tmpdir}/client.conf'.format(tmpdir=self.new_root),
2099+ 'template_name': 'client.conf', 'template': user_template,
2100+ 'service_name': 'chrony', 'packages': ['chrony']}
2101+ for distro in cc_ntp.distros:
2102+ mycloud = self._get_cloud(distro)
2103+ ntpconfig = self._mock_ntp_client_config(client=client,
2104+ distro=distro)
2105+ confpath = ntpconfig['confpath']
2106+ m_select.return_value = ntpconfig
2107+ mock_path = 'cloudinit.config.cc_ntp.temp_utils._TMPDIR'
2108+ with mock.patch(mock_path, self.new_root):
2109+ cc_ntp.handle('notimportant',
2110+ {'ntp': cfg}, mycloud, None, None)
2111+ content = util.read_file_or_url('file://' + confpath).contents
2112+ self.assertEqual(
2113+ "servers []\npools ['mypool.org']\n%s" % custom,
2114+ content.decode())
2115+ m_schema.assert_called_with(expected_merged_cfg)
2116+
2117+
2118+class TestSupplementalSchemaValidation(CiTestCase):
2119+
2120+ def test_error_on_missing_keys(self):
2121+ """ValueError raised reporting any missing required ntp:config keys"""
2122+ cfg = {}
2123+ match = (r'Invalid ntp configuration:\\nMissing required ntp:config'
2124+ ' keys: check_exe, confpath, packages, service_name')
2125+ with self.assertRaisesRegex(ValueError, match):
2126+ cc_ntp.supplemental_schema_validation(cfg)
2127+
2128+ def test_error_requiring_either_template_or_template_name(self):
2129+ """ValueError raised if both template not template_name are None."""
2130+ cfg = {'confpath': 'someconf', 'check_exe': '', 'service_name': '',
2131+ 'template': None, 'template_name': None, 'packages': []}
2132+ match = (r'Invalid ntp configuration:\\nEither ntp:config:template'
2133+ ' or ntp:config:template_name values are required')
2134+ with self.assertRaisesRegex(ValueError, match):
2135+ cc_ntp.supplemental_schema_validation(cfg)
2136+
2137+ def test_error_on_non_list_values(self):
2138+ """ValueError raised when packages is not of type list."""
2139+ cfg = {'confpath': 'someconf', 'check_exe': '', 'service_name': '',
2140+ 'template': 'asdf', 'template_name': None, 'packages': 'NOPE'}
2141+ match = (r'Invalid ntp configuration:\\nExpected a list of required'
2142+ ' package names for ntp:config:packages. Found \(NOPE\)')
2143+ with self.assertRaisesRegex(ValueError, match):
2144+ cc_ntp.supplemental_schema_validation(cfg)
2145+
2146+ def test_error_on_non_string_values(self):
2147+ """ValueError raised for any values expected as string type."""
2148+ cfg = {'confpath': 1, 'check_exe': 2, 'service_name': 3,
2149+ 'template': 4, 'template_name': 5, 'packages': []}
2150+ errors = [
2151+ 'Expected a config file path ntp:config:confpath. Found (1)',
2152+ 'Expected a string type for ntp:config:check_exe. Found (2)',
2153+ 'Expected a string type for ntp:config:service_name. Found (3)',
2154+ 'Expected a string type for ntp:config:template. Found (4)',
2155+ 'Expected a string type for ntp:config:template_name. Found (5)']
2156+ with self.assertRaises(ValueError) as context_mgr:
2157+ cc_ntp.supplemental_schema_validation(cfg)
2158+ error_msg = str(context_mgr.exception)
2159+ for error in errors:
2160+ self.assertIn(error, error_msg)
2161
2162 # vi: ts=4 expandtab

Subscribers

People subscribed via source and target branches