cc_ntp fails to work when deploying ubuntu-core

Bug #1686485 reported by Blake Rouse
18
This bug affects 3 people
Affects Status Importance Assigned to Milestone
MAAS
Invalid
Undecided
Unassigned
cloud-init
Fix Released
High
Unassigned

Bug Description

When deploying Ubuntu Core with MAAS I am seeing this error in /var/log/cloud-init.log:

2017-04-26 18:11:45,172 - cc_apt_configure.py[DEBUG]: Nothing to do: No apt config and running on snappy
2017-04-26 18:11:45,172 - handlers.py[DEBUG]: finish: modules-config/config-apt-configure: SUCCESS: config-apt-configure ran successfully
2017-04-26 18:11:45,172 - stages.py[DEBUG]: Running module ntp (<module 'cloudinit.config.cc_ntp' from '/usr/lib/python3/dist-packages/cloudinit/config/cc_ntp.py'>) with frequency once-per-instance
2017-04-26 18:11:45,172 - handlers.py[DEBUG]: start: modules-config/config-ntp: running config-ntp with frequency once-per-instance
2017-04-26 18:11:45,173 - util.py[DEBUG]: Writing to /var/lib/cloud/instances/mpcgqp/sem/config_ntp - wb: [420] 24 bytes
2017-04-26 18:11:45,173 - helpers.py[DEBUG]: Running config-ntp using lock (<FileLock using file '/var/lib/cloud/instances/mpcgqp/sem/config_ntp'>)
2017-04-26 18:11:45,175 - util.py[DEBUG]: Writing to /var/lib/cloud/instances/mpcgqp/sem/update_sources - wb: [420] 24 bytes
2017-04-26 18:11:45,176 - helpers.py[DEBUG]: Running update-sources using lock (<FileLock using file '/var/lib/cloud/instances/mpcgqp/sem/update_sources'>)
2017-04-26 18:11:45,176 - util.py[DEBUG]: Running command ['apt-get', '--option=Dpkg::Options::=--force-confold', '--option=Dpkg::options::=--force-unsafe-io', '--assume-yes', '--quiet', 'update'] with allowed return codes [0] (shell=False, capture=False)
2017-04-26 18:11:45,186 - util.py[DEBUG]: apt-update [apt-get --option=Dpkg::Options::=--force-confold --option=Dpkg::options::=--force-unsafe-io --assume-yes --quiet update] took 0.010 seconds
2017-04-26 18:11:45,186 - util.py[DEBUG]: Running command ['apt-get', '--option=Dpkg::Options::=--force-confold', '--option=Dpkg::options::=--force-unsafe-io', '--assume-yes', '--quiet', 'install', 'ntp'] with allowed return codes [0] (shell=False, capture=False)
2017-04-26 18:11:45,191 - util.py[DEBUG]: apt-install [apt-get --option=Dpkg::Options::=--force-confold --option=Dpkg::options::=--force-unsafe-io --assume-yes --quiet install ntp] took 0.005 seconds
2017-04-26 18:11:45,193 - util.py[DEBUG]: Reading from /etc/cloud/templates/ntp.conf.ubuntu.tmpl (quiet=False)
2017-04-26 18:11:45,193 - util.py[DEBUG]: Read 2509 bytes from /etc/cloud/templates/ntp.conf.ubuntu.tmpl
2017-04-26 18:11:45,193 - templater.py[DEBUG]: Rendering content of '/etc/cloud/templates/ntp.conf.ubuntu.tmpl' using renderer jinja
2017-04-26 18:11:45,197 - util.py[DEBUG]: Writing to /etc/ntp.conf - wb: [420] 2330 bytes
2017-04-26 18:11:45,200 - handlers.py[DEBUG]: finish: modules-config/config-ntp: FAIL: running config-ntp with frequency once-per-instance
2017-04-26 18:11:45,200 - util.py[WARNING]: Running module ntp (<module 'cloudinit.config.cc_ntp' from '/usr/lib/python3/dist-packages/cloudinit/config/cc_ntp.py'>) failed
2017-04-26 18:11:45,202 - util.py[DEBUG]: Running module ntp (<module 'cloudinit.config.cc_ntp' from '/usr/lib/python3/dist-packages/cloudinit/config/cc_ntp.py'>) failed
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/cloudinit/stages.py", line 787, in _run_modules
    freq=freq)
  File "/usr/lib/python3/dist-packages/cloudinit/cloud.py", line 54, in run
    return self._runners.run(name, functor, args, freq, clear_on_fail)
  File "/usr/lib/python3/dist-packages/cloudinit/helpers.py", line 187, in run
    results = functor(*args)
  File "/usr/lib/python3/dist-packages/cloudinit/config/cc_ntp.py", line 80, in handle
    write_ntp_config_template(ntp_cfg, cloud)
  File "/usr/lib/python3/dist-packages/cloudinit/config/cc_ntp.py", line 126, in write_ntp_config_template
    templater.render_to_file(template_fn, NTP_CONF, params)
  File "/usr/lib/python3/dist-packages/cloudinit/templater.py", line 131, in render_to_file
    util.write_file(outfn, contents, mode=mode)
  File "/usr/lib/python3/dist-packages/cloudinit/util.py", line 1711, in write_file
    with open(filename, omode) as fh:
OSError: [Errno 30] Read-only file system: '/etc/ntp.conf'

Note: This doesn't break deployment. Deployment still succeeds, except for ntp syncing is not setup to point to MAAS.

Related branches

Changed in maas:
status: New → Triaged
importance: Undecided → High
milestone: none → 2.2.1
Revision history for this message
Ryan Harper (raharper) wrote :

A quick look at the core image, and ntp is not installed by default. The system does include systemd's timesyncd which can be configured but will require changes to cloud-init.

Revision history for this message
Lee Trager (ltrager) wrote :

NTP isn't in the base Ubuntu image either, but timesyncd is. If cloud-init must gain timesyncd support for Ubuntu-Core it would be worth discussing using it for Ubuntu, CentOS, and any other OS with timesyncd installed.

Changed in maas:
milestone: 2.2.1 → 2.2.x
Revision history for this message
David Britton (dpb) wrote :

+1 on ltrager's comment. since systemd is a common denominator here, we should just move that way.

Changed in maas:
status: Triaged → Incomplete
milestone: 2.2.x → none
importance: High → Undecided
milestone: none → 2.3.0
Revision history for this message
Ryan Harper (raharper) wrote :

Well, systemd-timesyncd is somewhat awkward for validation. Unlike ntp, there's no runtime way to query the deamon as to which ntp servers it actually connected with; we're left with parsing all of the configs on the disk, and then parsing the unit output.

# timedatectl --help
timedatectl [OPTIONS...] COMMAND ...

Query or change system time and date settings.

  -h --help Show this help message
     --version Show package version
     --no-pager Do not pipe output into a pager
     --no-ask-password Do not prompt for password
  -H --host=[USER@]HOST Operate on remote host
  -M --machine=CONTAINER Operate on local container
     --adjust-system-clock Adjust system clock when changing local RTC mode

Commands:
  status Show current time settings
  set-time TIME Set system time
  set-timezone ZONE Set system time zone
  list-timezones Show known time zones
  set-local-rtc BOOL Control whether RTC is in local time
  set-ntp BOOL Enable or disable network time synchronization

Status is bland:

# timedatectl status
      Local time: Mon 2017-07-31 20:59:17 UTC
  Universal time: Mon 2017-07-31 20:59:17 UTC
        RTC time: Mon 2017-07-31 20:59:17
       Time zone: Etc/UTC (UTC, +0000)
 Network time on: yes
NTP synchronized: yes
 RTC in local TZ: no

Via unit log, I can see which server it used:

# journalctl -o short-precise -u systemd-timesyncd.service
-- Logs begin at Mon 2017-07-31 20:45:48 UTC, end at Mon 2017-07-31 20:59:17 UTC. --
Jul 31 20:45:50.153145 ubuntu systemd[1]: Starting Network Time Synchronization...
Jul 31 20:45:50.224368 ubuntu systemd[1]: Started Network Time Synchronization.
Jul 31 20:46:21.526041 x1 systemd-timesyncd[607]: Synchronized to time server 91.189.89.198:123 (ntp.ubuntu.com).
Jul 31 20:51:23.469719 x1 systemd[1]: Stopping Network Time Synchronization...
Jul 31 20:51:23.481443 x1 systemd[1]: Stopped Network Time Synchronization.
Jul 31 20:51:23.495047 x1 systemd[1]: Starting Network Time Synchronization...
Jul 31 20:51:23.519878 x1 systemd[1]: Started Network Time Synchronization.
Jul 31 20:51:33.689329 x1 systemd-timesyncd[1384]: Timed out waiting for reply from 10.0.2.2:123 (10.0.2.2).
Jul 31 20:51:43.939276 x1 systemd-timesyncd[1384]: Timed out waiting for reply from 10.0.2.3:123 (10.0.2.3).

Some manual testing with timesyncd on Ubuntu Core shows, it also needs an update to allow /var/lib/systemd/clock file to be writable; this is where the current time data is kept on disk for keeping clock current upon reboot.

Ryan Harper (raharper)
Changed in cloud-init:
importance: Undecided → High
status: New → In Progress
Scott Moser (smoser)
Changed in cloud-init:
status: In Progress → Fix Committed
Revision history for this message
Andres Rodriguez (andreserl) wrote :

@Scott, @Ryan,

Why not just make timesyncd the default instead of installing 'ntp' ?

Changed in maas:
status: Incomplete → Invalid
milestone: 2.3.0 → none
Revision history for this message
Scott Moser (smoser) wrote : Fixed in Cloud-init 17.1

This bug is believed to be fixed in cloud-init in 17.1. If this is still a problem for you, please make a comment and set the state back to New

Thank you.

Changed in cloud-init:
status: Fix Committed → Fix Released
Revision history for this message
James Falcon (falcojr) wrote :
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.