Merge lp:~fo0bar/turku/turku-agent-gonogo into lp:turku

[Ryan Finnie]

[lp:~fo0bar/turku/turku-agent-gonogo]

52. By Ryan Finnie on 2020-03-23

Allow gonogo_program to be stored in config, allow --gonogo-program to be shlex-split

Unmerged revisions

52. By Ryan Finnie on 2020-03-23

Allow gonogo_program to be stored in config, allow --gonogo-program to be shlex-split

51. By Colin Watson on 2019-05-23

On Upstart, check if turku-agent-rsyncd is already running before starting it.

Reviewed-on: https://code.launchpad.net/~cjwatson/turku/turku-agent-fix-rsyncd-interaction/+merge/367847
Reviewed-by: Haw Loeung <email address hidden>

50. By Colin Watson on 2019-05-23

Don't restart rsyncd when updating its configuration.

Reviewed-on: https://code.launchpad.net/~cjwatson/turku/turku-agent-fix-rsyncd-interaction/+merge/367418
Reviewed-by: Haw Loeung <email address hidden>

49. By Ryan Finnie on 2019-04-22

Fix Python 3 .values conversion with --restore

48. By Ryan Finnie on 2019-04-22

Remove early legacy config migrations/mitigations

47. By Ryan Finnie on 2019-04-22

Convert to Python 3

Program functionality is identical to previous, but will now need
to be installed with python3.

46. By Ryan Finnie on 2019-04-21

turku-agent-ping: Add --gonogo-program

45. By Ryan Finnie on 2018-06-02

Fix incorrect description turku-update-config.timer

44. By Ryan Finnie on 2018-06-02

Add systemd timers for turku-agent-ping and turku-update-config

43. By Ryan Finnie on 2016-04-19

[jacekn] Make sure update_config errors are not ignored. This allows for hooks to fail on bad configuration.

=== added file '.bzrignore'
--- .bzrignore	1970-01-01 00:00:00 +0000
+++ .bzrignore	2019-06-15 00:58:25 +0000
@@ -0,0 +1,5 @@
+*.pyc
+./build/
+./dist/
+./MANIFEST
+./*.egg-info/

=== renamed file '.bzrignore' => '.bzrignore.moved'
=== added file 'MANIFEST.in'
--- MANIFEST.in	1970-01-01 00:00:00 +0000
+++ MANIFEST.in	2019-06-15 00:58:25 +0000
@@ -0,0 +1,3 @@
+include turku-agent.cron
+include turku-agent-rsyncd.conf
+include turku-agent-rsyncd.service

=== added file 'README'
--- README	1970-01-01 00:00:00 +0000
+++ README	2019-06-15 00:58:25 +0000
@@ -0,0 +1,4 @@
+Turku backups - client agent
+Copyright 2015 Canonical Ltd.
+
+https://launchpad.net/turku

=== added file 'setup.py'
--- setup.py	1970-01-01 00:00:00 +0000
+++ setup.py	2019-06-15 00:58:25 +0000
@@ -0,0 +1,39 @@
+#!/usr/bin/env python3
+
+# Turku backups - client agent
+# Copyright 2015 Canonical Ltd.
+#
+# This program is free software: you can redistribute it and/or modify it
+# under the terms of the GNU General Public License version 3, as published by
+# the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful, but WITHOUT
+# ANY WARRANTY; without even the implied warranties of MERCHANTABILITY,
+# SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+# General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along with
+# this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import sys
+from setuptools import setup
+
+assert(sys.version_info > (3, 4))
+
+
+setup(
+    name='turku_agent',
+    description='Turku backups - client agent',
+    version='0.2.0',
+    author='Ryan Finnie',
+    author_email='[e-mail]',
+    url='https://launchpad.net/turku',
+    packages=['turku_agent'],
+    entry_points={
+        'console_scripts': [
+            'turku-agent-ping = turku_agent.ping:main',
+            'turku-agent-rsyncd-wrapper = turku_agent.rsyncd_wrapper:main',
+            'turku-update-config = turku_agent.update_config:main',
+        ],
+    },
+)

=== renamed file 'setup.py' => 'setup.py.moved'
=== added file 'turku-agent-ping'
--- turku-agent-ping	1970-01-01 00:00:00 +0000
+++ turku-agent-ping	2019-06-15 00:58:25 +0000
@@ -0,0 +1,20 @@
+#!/usr/bin/env python3
+
+# Turku backups - client agent
+# Copyright 2015 Canonical Ltd.
+#
+# This program is free software: you can redistribute it and/or modify it
+# under the terms of the GNU General Public License version 3, as published by
+# the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful, but WITHOUT
+# ANY WARRANTY; without even the implied warranties of MERCHANTABILITY,
+# SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+# General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along with
+# this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import sys
+from turku_agent import ping
+sys.exit(ping.main())

=== added file 'turku-agent-ping.service'
--- turku-agent-ping.service	1970-01-01 00:00:00 +0000
+++ turku-agent-ping.service	2019-06-15 00:58:25 +0000
@@ -0,0 +1,6 @@
+[Unit]
+Description=turku-agent-ping
+
+[Service]
+Type=oneshot
+ExecStart=/usr/bin/env turku-agent-ping

=== added file 'turku-agent-ping.timer'
--- turku-agent-ping.timer	1970-01-01 00:00:00 +0000
+++ turku-agent-ping.timer	2019-06-15 00:58:25 +0000
@@ -0,0 +1,10 @@
+[Unit]
+Description=turku-agent-ping
+
+[Timer]
+OnCalendar=*:0/5
+RandomizedDelaySec=5m
+Persistent=true
+
+[Install]
+WantedBy=timers.target

=== added file 'turku-agent-rsyncd-wrapper'
--- turku-agent-rsyncd-wrapper	1970-01-01 00:00:00 +0000
+++ turku-agent-rsyncd-wrapper	2019-06-15 00:58:25 +0000
@@ -0,0 +1,20 @@
+#!/usr/bin/env python3
+
+# Turku backups - client agent
+# Copyright 2015 Canonical Ltd.
+#
+# This program is free software: you can redistribute it and/or modify it
+# under the terms of the GNU General Public License version 3, as published by
+# the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful, but WITHOUT
+# ANY WARRANTY; without even the implied warranties of MERCHANTABILITY,
+# SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+# General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along with
+# this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import sys
+from turku_agent import rsyncd_wrapper
+sys.exit(rsyncd_wrapper.main())

=== added file 'turku-agent-rsyncd.conf'
--- turku-agent-rsyncd.conf	1970-01-01 00:00:00 +0000
+++ turku-agent-rsyncd.conf	2019-06-15 00:58:25 +0000
@@ -0,0 +1,8 @@
+description "turku rsync daemon"
+
+start on runlevel [2345]
+stop on runlevel [!2345]
+
+respawn
+
+exec /usr/bin/env turku-agent-rsyncd-wrapper

=== added file 'turku-agent-rsyncd.init-debian'
--- turku-agent-rsyncd.init-debian	1970-01-01 00:00:00 +0000
+++ turku-agent-rsyncd.init-debian	2019-06-15 00:58:25 +0000
@@ -0,0 +1,55 @@
+#! /bin/sh
+
+### BEGIN INIT INFO
+# Provides:          turku-agent-rsyncd
+# Required-Start:    $remote_fs $syslog
+# Required-Stop:     $remote_fs $syslog
+# Should-Start:      $named
+# Default-Start:     2 3 4 5
+# Default-Stop:      
+# Short-Description: turku rsync daemon
+# Description:       turku rsync daemon
+### END INIT INFO
+
+set -e
+
+PID_FILE=/var/run/turku-agent-rsyncd.pid
+
+. /lib/lsb/init-functions
+
+export PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
+
+case "$1" in
+  start)
+    log_daemon_msg "Starting turku rsync daemon" "turku-agent-rsyncd"
+    start-stop-daemon --start --quiet --background --make-pidfile \
+      --pidfile $PID_FILE --startas /usr/bin/env -- turku-agent-rsyncd-wrapper
+    log_end_msg $?
+    ;;
+  stop)
+    log_daemon_msg "Stopping turku rsync daemon" "turku-agent-rsyncd"
+    start-stop-daemon --stop --quiet --oknodo --pidfile $PID_FILE --retry 5
+    log_end_msg $?
+    rm -f $PID_FILE
+    ;;
+
+  reload|force-reload)
+    log_daemon_msg "Reloading turku rsync daemon" "turku-agent-rsyncd"
+    log_end_msg 0
+    ;;
+
+  restart)
+    "$0" stop
+    "$0" start
+    ;;
+
+  status)
+    status_of_proc -p $PID_FILE rsync turku-agent-rsyncd
+    exit $? # notreached due to set -e
+    ;;
+  *)
+    echo "Usage: /etc/init.d/turku-agent-rsyncd {start|stop|reload|force-reload|restart|status}"
+    exit 1
+esac
+
+exit 0

=== added file 'turku-agent-rsyncd.service'
--- turku-agent-rsyncd.service	1970-01-01 00:00:00 +0000
+++ turku-agent-rsyncd.service	2019-06-15 00:58:25 +0000
@@ -0,0 +1,10 @@
+[Unit]
+Description=turku rsyncd daemon
+ConditionPathExists=/var/lib/turku-agent/rsyncd.conf
+
+[Service]
+ExecStart=/usr/bin/env turku-agent-rsyncd-wrapper
+Restart=always
+
+[Install]
+WantedBy=multi-user.target

=== added file 'turku-agent.cron'
--- turku-agent.cron	1970-01-01 00:00:00 +0000
+++ turku-agent.cron	2019-06-15 00:58:25 +0000
@@ -0,0 +1,3 @@
+PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
+*/5 * * * * root sh -c 'systemctl is-active basic.target 2>/dev/null >/dev/null || turku-agent-ping --wait=300 >/dev/null 2>/dev/null'
+0 0,12 * * * root sh -c 'systemctl is-active basic.target 2>/dev/null >/dev/null || turku-update-config --wait=7200 >/dev/null 2>/dev/null'

=== added file 'turku-update-config'
--- turku-update-config	1970-01-01 00:00:00 +0000
+++ turku-update-config	2019-06-15 00:58:25 +0000
@@ -0,0 +1,20 @@
+#!/usr/bin/env python3
+
+# Turku backups - client agent
+# Copyright 2015 Canonical Ltd.
+#
+# This program is free software: you can redistribute it and/or modify it
+# under the terms of the GNU General Public License version 3, as published by
+# the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful, but WITHOUT
+# ANY WARRANTY; without even the implied warranties of MERCHANTABILITY,
+# SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+# General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along with
+# this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import sys
+from turku_agent import update_config
+sys.exit(update_config.main())

=== added file 'turku-update-config.service'
--- turku-update-config.service	1970-01-01 00:00:00 +0000
+++ turku-update-config.service	2019-06-15 00:58:25 +0000
@@ -0,0 +1,6 @@
+[Unit]
+Description=turku-update-config
+
+[Service]
+Type=oneshot
+ExecStart=/usr/bin/env turku-update-config

=== added file 'turku-update-config.timer'
--- turku-update-config.timer	1970-01-01 00:00:00 +0000
+++ turku-update-config.timer	2019-06-15 00:58:25 +0000
@@ -0,0 +1,10 @@
+[Unit]
+Description=turku-update-config
+
+[Timer]
+OnCalendar=0,12:0
+RandomizedDelaySec=12h
+Persistent=true
+
+[Install]
+WantedBy=timers.target

=== added directory 'turku_agent'
=== added file 'turku_agent/__init__.py'
=== added file 'turku_agent/ping.py'
--- turku_agent/ping.py	1970-01-01 00:00:00 +0000
+++ turku_agent/ping.py	2019-06-15 00:58:25 +0000
@@ -0,0 +1,211 @@
+#!/usr/bin/env python3
+
+# Turku backups - client agent
+# Copyright 2015 Canonical Ltd.
+#
+# This program is free software: you can redistribute it and/or modify it
+# under the terms of the GNU General Public License version 3, as published by
+# the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful, but WITHOUT
+# ANY WARRANTY; without even the implied warranties of MERCHANTABILITY,
+# SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+# General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along with
+# this program.  If not, see <http://www.gnu.org/licenses/>.
+
+
+import os
+import json
+import random
+import shlex
+import subprocess
+import tempfile
+import time
+from .utils import load_config, acquire_lock, api_call
+
+
+def parse_args():
+    import argparse
+
+    parser = argparse.ArgumentParser(
+        formatter_class=argparse.ArgumentDefaultsHelpFormatter)
+    parser.add_argument('--config-dir', '-c', type=str, default='/etc/turku-agent')
+    parser.add_argument('--wait', '-w', type=float)
+    parser.add_argument('--restore', action='store_true')
+    parser.add_argument('--restore-storage', type=str, default=None)
+    parser.add_argument('--gonogo-program', type=str, default=None)
+    return parser.parse_args()
+
+
+def call_ssh(config, storage, ssh_req):
+    # Write the server host public key
+    t = tempfile.NamedTemporaryFile()
+    for key in storage['ssh_ping_host_keys']:
+        t.write(('%s %s\n' % (storage['ssh_ping_host'], key)).encode('UTF-8'))
+    t.flush()
+
+    # Call ssh
+    ssh_command = config['ssh_command']
+    ssh_command += [
+        '-T',
+        '-o', 'BatchMode=yes',
+        '-o', 'UserKnownHostsFile=%s' % t.name,
+        '-o', 'StrictHostKeyChecking=yes',
+        '-o', 'CheckHostIP=no',
+        '-i', config['ssh_private_key_file'],
+        '-R', '%d:%s:%d' % (ssh_req['port'], config['rsyncd_local_address'], config['rsyncd_local_port']),
+        '-p', str(storage['ssh_ping_port']),
+        '-l', storage['ssh_ping_user'],
+        storage['ssh_ping_host'],
+        'turku-ping-remote',
+    ]
+    p = subprocess.Popen(ssh_command, stdin=subprocess.PIPE)
+
+    # Write the ssh request
+    p.stdin.write((json.dumps(ssh_req) + '\n.\n').encode('UTF-8'))
+    p.stdin.flush()
+
+    # Wait for the server to close the SSH connection
+    try:
+        p.wait()
+    except KeyboardInterrupt:
+        pass
+
+    # Cleanup
+    t.close()
+
+
+def main():
+    args = parse_args()
+
+    # Sleep a random amount of time if requested
+    if args.wait:
+        time.sleep(random.uniform(0, args.wait))
+
+    config = load_config(args.config_dir)
+
+    # Basic checks
+    for i in ('ssh_private_key_file', 'machine_uuid', 'machine_secret', 'api_url'):
+        if i not in config:
+            return
+    if not os.path.isfile(config['ssh_private_key_file']):
+        return
+
+    # If a go/no-go program is defined, run it and only go if it exits 0.
+    if args.gonogo_program:
+        gonogo_program = shlex.split(args.gonogo_program)
+    elif config['gonogo_program']:
+        gonogo_program = config['gonogo_program']
+    else:
+        gonogo_program = []
+    if gonogo_program:
+        try:
+            subprocess.check_call(gonogo_program)
+        except (subprocess.CalledProcessError, OSError):
+            return
+
+    lock = acquire_lock(os.path.join(config['lock_dir'], 'turku-agent-ping.lock'))
+
+    restore_mode = args.restore
+
+    # Check with the API server
+    api_out = {}
+
+    machine_merge_map = (
+        ('machine_uuid', 'uuid'),
+        ('machine_secret', 'secret'),
+    )
+    api_out['machine'] = {}
+    for a, b in machine_merge_map:
+        if a in config:
+            api_out['machine'][b] = config[a]
+
+    if restore_mode:
+        print('Entering restore mode.')
+        print()
+        api_reply = api_call(config['api_url'], 'agent_ping_restore', api_out)
+
+        sources_by_storage = {}
+        for source_name in api_reply['machine']['sources']:
+            source = api_reply['machine']['sources'][source_name]
+            if source_name not in config['sources']:
+                continue
+            if 'storage' not in source:
+                continue
+            if source['storage']['name'] not in sources_by_storage:
+                sources_by_storage[source['storage']['name']] = {}
+            sources_by_storage[source['storage']['name']][source_name] = source
+
+        if len(sources_by_storage) == 0:
+            print('Cannot find any appropraite sources.')
+            return
+        print('This machine\'s sources are on the following storage units:')
+        for storage_name in sources_by_storage:
+            print('    %s' % storage_name)
+            for source_name in sources_by_storage[storage_name]:
+                print('        %s' % source_name)
+        print()
+        if len(sources_by_storage) == 1:
+            storage = list(list(sources_by_storage.values())[0].values())[0]['storage']
+        elif args.restore_storage:
+            if args.restore_storage in sources_by_storage:
+                storage = sources_by_storage[args.restore_storage]['storage']
+            else:
+                print('Cannot find appropriate storage "%s"' % args.restore_storage)
+                return
+        else:
+            print('Multiple storages found.  Please use --restore-storage to specify one.')
+            return
+
+        ssh_req = {
+            'verbose': True,
+            'action': 'restore',
+            'port': random.randint(49152, 65535),
+        }
+        print('Storage unit: %s' % storage['name'])
+        if 'restore_path' in config:
+            print('Local destination path: %s' % config['restore_path'])
+            print('Sample restore usage from storage unit:')
+            print(
+                '    RSYNC_PASSWORD=%s rsync -avzP --numeric-ids ${P?}/ rsync://%s@127.0.0.1:%s/%s/' % (
+                    config['restore_password'],
+                    config['restore_username'],
+                    ssh_req['port'], config['restore_module']
+                )
+            )
+            print()
+        call_ssh(config, storage, ssh_req)
+    else:
+        api_reply = api_call(config['api_url'], 'agent_ping_checkin', api_out)
+
+        if 'scheduled_sources' not in api_reply:
+            return
+        sources_by_storage = {}
+        for source_name in api_reply['machine']['scheduled_sources']:
+            source = api_reply['machine']['scheduled_sources'][source_name]
+            if source_name not in config['sources']:
+                continue
+            if 'storage' not in source:
+                continue
+            if source['storage']['name'] not in sources_by_storage:
+                sources_by_storage[source['storage']['name']] = {}
+            sources_by_storage[source['storage']['name']][source_name] = source
+
+        for storage_name in sources_by_storage:
+            ssh_req = {
+                'verbose': True,
+                'action': 'checkin',
+                'port': random.randint(49152, 65535),
+                'sources': {},
+            }
+            for source in sources_by_storage[storage_name]:
+                ssh_req['sources'][source] = {
+                    'username': config['sources'][source]['username'],
+                    'password': config['sources'][source]['password'],
+                }
+            call_ssh(config, list(sources_by_storage[storage_name].values())[0]['storage'], ssh_req)
+
+    # Cleanup
+    lock.close()

=== added file 'turku_agent/rsyncd_wrapper.py'
--- turku_agent/rsyncd_wrapper.py	1970-01-01 00:00:00 +0000
+++ turku_agent/rsyncd_wrapper.py	2019-06-15 00:58:25 +0000
@@ -0,0 +1,42 @@
+#!/usr/bin/env python3
+
+# Turku backups - client agent
+# Copyright 2015 Canonical Ltd.
+#
+# This program is free software: you can redistribute it and/or modify it
+# under the terms of the GNU General Public License version 3, as published by
+# the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful, but WITHOUT
+# ANY WARRANTY; without even the implied warranties of MERCHANTABILITY,
+# SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+# General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along with
+# this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import os
+from .utils import load_config
+
+
+def parse_args():
+    import argparse
+
+    parser = argparse.ArgumentParser(
+        formatter_class=argparse.ArgumentDefaultsHelpFormatter)
+    parser.add_argument('--config-dir', '-c', type=str, default='/etc/turku-agent')
+    parser.add_argument('--detach', action='store_true')
+    return parser.parse_known_args()
+
+
+def main():
+    args, rest = parse_args()
+
+    config = load_config(args.config_dir)
+    rsyncd_command = config['rsyncd_command']
+    if not args.detach:
+        rsyncd_command.append('--no-detach')
+    rsyncd_command.append('--daemon')
+    rsyncd_command.append('--config=%s' % os.path.join(config['var_dir'], 'rsyncd.conf'))
+    rsyncd_command += rest
+    os.execvp(rsyncd_command[0], rsyncd_command)

=== added file 'turku_agent/update_config.py'
--- turku_agent/update_config.py	1970-01-01 00:00:00 +0000
+++ turku_agent/update_config.py	2019-06-15 00:58:25 +0000
@@ -0,0 +1,179 @@
+#!/usr/bin/env python3
+
+# Turku backups - client agent
+# Copyright 2015 Canonical Ltd.
+#
+# This program is free software: you can redistribute it and/or modify it
+# under the terms of the GNU General Public License version 3, as published by
+# the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful, but WITHOUT
+# ANY WARRANTY; without even the implied warranties of MERCHANTABILITY,
+# SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+# General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along with
+# this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import random
+import os
+import subprocess
+import time
+import logging
+from .utils import json_dump_p, json_dumps_p, load_config, fill_config, acquire_lock, api_call
+
+
+class IncompleteConfigError(Exception):
+    pass
+
+
+def parse_args():
+    import argparse
+
+    parser = argparse.ArgumentParser(
+        formatter_class=argparse.ArgumentDefaultsHelpFormatter)
+    parser.add_argument('--config-dir', '-c', type=str, default='/etc/turku-agent')
+    parser.add_argument('--wait', '-w', type=float)
+    parser.add_argument('--debug', action='store_true')
+    return parser.parse_args()
+
+
+def write_conf_files(config):
+    # Build rsyncd.conf
+    built_rsyncd_conf = (
+        'address = %s\n' % config['rsyncd_local_address'] +
+        'port = %d\n' % config['rsyncd_local_port'] +
+        'log file = /dev/stdout\n' +
+        'uid = root\n' +
+        'gid = root\n' +
+        'list = false\n\n'
+    )
+    rsyncd_secrets = []
+    rsyncd_secrets.append((config['restore_username'], config['restore_password']))
+    built_rsyncd_conf += (
+        '[%s]\n' +
+        '    path = %s\n' +
+        '    auth users = %s\n' +
+        '    secrets file = %s\n' +
+        '    read only = false\n\n'
+    ) % (
+        config['restore_module'],
+        config['restore_path'],
+        config['restore_username'],
+        os.path.join(config['var_dir'], 'rsyncd.secrets'),
+    )
+    for s in config['sources']:
+        sd = config['sources'][s]
+        rsyncd_secrets.append((sd['username'], sd['password']))
+        built_rsyncd_conf += (
+            '[%s]\n' +
+            '    path = %s\n' +
+            '    auth users = %s\n' +
+            '    secrets file = %s\n' +
+            '    read only = true\n\n'
+        ) % (
+            s,
+            sd['path'],
+            sd['username'],
+            os.path.join(config['var_dir'], 'rsyncd.secrets'),
+        )
+    with open(os.path.join(config['var_dir'], 'rsyncd.conf'), 'w') as f:
+        f.write(built_rsyncd_conf)
+
+    # Build rsyncd.secrets
+    built_rsyncd_secrets = ''
+    for (username, password) in rsyncd_secrets:
+        built_rsyncd_secrets += username + ':' + password + '\n'
+    with open(os.path.join(config['var_dir'], 'rsyncd.secrets'), 'w') as f:
+        os.fchmod(f.fileno(), 0o600)
+        f.write(built_rsyncd_secrets)
+
+
+def init_is_upstart():
+    try:
+        return 'upstart' in subprocess.check_output(
+            ['initctl', 'version'],
+            stderr=subprocess.DEVNULL, universal_newlines=True)
+    except (FileNotFoundError, subprocess.CalledProcessError):
+        return False
+
+
+def start_services():
+    # Start rsyncd if it isn't already running.
+    # Note that we do *not* need to reload rsyncd when changing rsyncd.conf,
+    # as it rereads it on every client connection; but we may need to start
+    # it as it won't start if its configuration file doesn't exist.
+    if init_is_upstart():
+        # With Upstart, start will fail if the service is already running,
+        # so we need to check for that first.
+        try:
+            if 'start/running' in subprocess.check_output(
+                    ['status', 'turku-agent-rsyncd'],
+                    stderr=subprocess.STDOUT, universal_newlines=True):
+                return
+        except subprocess.CalledProcessError:
+            pass
+    subprocess.check_call(['service', 'turku-agent-rsyncd', 'start'])
+
+
+def send_config(config):
+    required_keys = ['api_url']
+    if 'api_auth' not in config:
+        required_keys += ['api_auth_name', 'api_auth_secret']
+    for k in required_keys:
+        if k not in config:
+            raise IncompleteConfigError('Required config "%s" not found.' % k)
+
+    api_out = {}
+    if ('api_auth_name' in config) and ('api_auth_secret' in config):
+        # name/secret style
+        api_out['auth'] = {
+            'name': config['api_auth_name'],
+            'secret': config['api_auth_secret'],
+        }
+    else:
+        # nameless secret style
+        api_out['auth'] = config['api_auth']
+
+    # Merge the following options into the machine section
+    machine_merge_map = (
+        ('machine_uuid', 'uuid'),
+        ('machine_secret', 'secret'),
+        ('environment_name', 'environment_name'),
+        ('service_name', 'service_name'),
+        ('unit_name', 'unit_name'),
+        ('ssh_public_key', 'ssh_public_key'),
+        ('published', 'published'),
+    )
+    api_out['machine'] = {}
+    for a, b in machine_merge_map:
+        if a in config:
+            api_out['machine'][b] = config[a]
+
+    api_out['machine']['sources'] = config['sources']
+
+    api_reply = api_call(config['api_url'], 'update_config', api_out)
+
+
+def main():
+    args = parse_args()
+    # Sleep a random amount of time if requested
+    if args.wait:
+        time.sleep(random.uniform(0, args.wait))
+
+    config = load_config(args.config_dir)
+    lock = acquire_lock(os.path.join(config['lock_dir'], 'turku-update-config.lock'))
+    fill_config(config)
+    if args.debug:
+        print(json_dumps_p(config))
+    write_conf_files(config)
+    try:
+        send_config(config)
+    except Exception as e:
+        if args.debug:
+            raise
+        logging.exception(e)
+        return 1
+    start_services()
+
+    lock.close()

=== added file 'turku_agent/utils.py'
--- turku_agent/utils.py	1970-01-01 00:00:00 +0000
+++ turku_agent/utils.py	2019-06-15 00:58:25 +0000
@@ -0,0 +1,332 @@
+#!/usr/bin/env python3
+
+# Turku backups - client agent
+# Copyright 2015 Canonical Ltd.
+#
+# This program is free software: you can redistribute it and/or modify it
+# under the terms of the GNU General Public License version 3, as published by
+# the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful, but WITHOUT
+# ANY WARRANTY; without even the implied warranties of MERCHANTABILITY,
+# SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+# General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along with
+# this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import uuid
+import string
+import random
+import json
+import os
+import copy
+import subprocess
+import platform
+import urllib.parse
+import http.client
+
+
+class RuntimeLock():
+    name = None
+    file = None
+
+    def __init__(self, name):
+        import fcntl
+        file = open(name, 'w')
+        try:
+            fcntl.lockf(file, fcntl.LOCK_EX | fcntl.LOCK_NB)
+        except IOError as e:
+            import errno
+            if e.errno in (errno.EACCES, errno.EAGAIN):
+                raise
+        file.write('%10s\n' % os.getpid())
+        file.flush()
+        file.seek(0)
+        self.name = name
+        self.file = file
+
+    def close(self):
+        if self.file:
+            self.file.close()
+            self.file = None
+            os.unlink(self.name)
+
+    def __del__(self):
+        self.close()
+
+    def __enter__(self):
+        self.file.__enter__()
+        return self
+
+    def __exit__(self, exc, value, tb):
+        result = self.file.__exit__(exc, value, tb)
+        self.close()
+        return result
+
+
+def acquire_lock(name):
+    return RuntimeLock(name)
+
+
+def json_dump_p(obj, f):
+    """Calls json.dump with standard (pretty) formatting"""
+    return json.dump(obj, f, sort_keys=True, indent=4, separators=(',', ': '))
+
+
+def json_dumps_p(obj):
+    """Calls json.dumps with standard (pretty) formatting"""
+    return json.dumps(obj, sort_keys=True, indent=4, separators=(',', ': '))
+
+
+def json_load_file(file):
+    with open(file) as f:
+        try:
+            return json.load(f)
+        except ValueError as e:
+            e.args += (file,)
+            raise
+
+
+def dict_merge(s, m):
+    """Recursively merge one dict into another."""
+    if not isinstance(m, dict):
+        return m
+    out = copy.deepcopy(s)
+    for k, v in list(m.items()):
+        if k in out and isinstance(out[k], dict):
+            out[k] = dict_merge(out[k], v)
+        else:
+            out[k] = copy.deepcopy(v)
+    return out
+
+
+def load_config(config_dir):
+    config = {}
+    config['config_dir'] = config_dir
+
+    config_d = os.path.join(config['config_dir'], 'config.d')
+    sources_d = os.path.join(config['config_dir'], 'sources.d')
+
+    # Merge in config.d/*.json to the root level
+    config_files = []
+    if os.path.isdir(config_d):
+        config_files = [
+            os.path.join(config_d, fn)
+            for fn in os.listdir(config_d)
+            if fn.endswith('.json')
+            and os.path.isfile(os.path.join(config_d, fn))
+            and os.access(os.path.join(config_d, fn), os.R_OK)
+        ]
+    config_files.sort()
+    for file in config_files:
+        config = dict_merge(config, json_load_file(file))
+
+    if 'var_dir' not in config:
+        config['var_dir'] = '/var/lib/turku-agent'
+
+    var_config_d = os.path.join(config['var_dir'], 'config.d')
+
+    # Load /var config.d files
+    var_config = {}
+    var_config_files = []
+    if os.path.isdir(var_config_d):
+        var_config_files = [
+            os.path.join(var_config_d, fn)
+            for fn in os.listdir(var_config_d)
+            if fn.endswith('.json')
+            and os.path.isfile(os.path.join(var_config_d, fn))
+            and os.access(os.path.join(var_config_d, fn), os.R_OK)
+        ]
+    var_config_files.sort()
+    for file in var_config_files:
+        var_config = dict_merge(var_config, json_load_file(file))
+    # /etc gets priority over /var
+    var_config = dict_merge(var_config, config)
+    config = var_config
+
+    if 'lock_dir' not in config:
+        config['lock_dir'] = '/var/lock'
+
+    if 'rsyncd_command' not in config:
+        config['rsyncd_command'] = ['rsync']
+
+    if 'rsyncd_local_address' not in config:
+        config['rsyncd_local_address'] = '127.0.0.1'
+
+    if 'rsyncd_local_port' not in config:
+        config['rsyncd_local_port'] = 27873
+
+    if 'ssh_command' not in config:
+        config['ssh_command'] = ['ssh']
+
+    if 'gonogo_program' not in config:
+        config['gonogo_program'] = []
+
+    var_sources_d = os.path.join(config['var_dir'], 'sources.d')
+
+    # Validate the unit name
+    if 'unit_name' not in config:
+        config['unit_name'] = platform.node()
+        # If this isn't in the on-disk config, don't write it; just
+        # generate it every time
+
+    # Pull the SSH public key
+    if os.path.isfile(os.path.join(config['var_dir'], 'ssh_key.pub')):
+        with open(os.path.join(config['var_dir'], 'ssh_key.pub')) as f:
+            config['ssh_public_key'] = f.read().rstrip()
+        config['ssh_public_key_file'] = os.path.join(config['var_dir'], 'ssh_key.pub')
+        config['ssh_private_key_file'] = os.path.join(config['var_dir'], 'ssh_key')
+
+    sources_config = {}
+    # Merge in sources.d/*.json to the sources dict
+    sources_files = []
+    if os.path.isdir(sources_d):
+        sources_files = [
+            os.path.join(sources_d, fn)
+            for fn in os.listdir(sources_d)
+            if fn.endswith('.json')
+            and os.path.isfile(os.path.join(sources_d, fn))
+            and os.access(os.path.join(sources_d, fn), os.R_OK)
+        ]
+    sources_files.sort()
+    var_sources_files = []
+    if os.path.isdir(var_sources_d):
+        var_sources_files = [
+            os.path.join(var_sources_d, fn)
+            for fn in os.listdir(var_sources_d)
+            if fn.endswith('.json')
+            and os.path.isfile(os.path.join(var_sources_d, fn))
+            and os.access(os.path.join(var_sources_d, fn), os.R_OK)
+        ]
+    var_sources_files.sort()
+    sources_files += var_sources_files
+    for file in sources_files:
+        sources_config = dict_merge(sources_config, json_load_file(file))
+
+    # Check for required sources options
+    for s in list(sources_config.keys()):
+        if 'path' not in sources_config[s]:
+            del sources_config[s]
+
+    config['sources'] = sources_config
+
+    return config
+
+
+def fill_config(config):
+    config_d = os.path.join(config['config_dir'], 'config.d')
+    sources_d = os.path.join(config['config_dir'], 'sources.d')
+    var_config_d = os.path.join(config['var_dir'], 'config.d')
+    var_sources_d = os.path.join(config['var_dir'], 'sources.d')
+
+    # Create required directories
+    for d in (config_d, sources_d, var_config_d, var_sources_d):
+        if not os.path.isdir(d):
+            os.makedirs(d)
+
+    # Validate the machine UUID/secret
+    write_uuid_data = False
+    if 'machine_uuid' not in config:
+        config['machine_uuid'] = str(uuid.uuid4())
+        write_uuid_data = True
+    if 'machine_secret' not in config:
+        config['machine_secret'] = ''.join(
+            random.choice(string.ascii_letters + string.digits)
+            for i in range(30)
+        )
+        write_uuid_data = True
+    # Write out the machine UUID/secret if needed
+    if write_uuid_data:
+        with open(os.path.join(var_config_d, '10-machine_uuid.json'), 'w') as f:
+            os.fchmod(f.fileno(), 0o600)
+            json_dump_p({
+                'machine_uuid': config['machine_uuid'],
+                'machine_secret': config['machine_secret'],
+            }, f)
+
+    # Restoration configuration
+    write_restore_data = False
+    if 'restore_path' not in config:
+        config['restore_path'] = '/var/backups/turku-agent/restore'
+        write_restore_data = True
+    if 'restore_module' not in config:
+        config['restore_module'] = 'turku-restore'
+        write_restore_data = True
+    if 'restore_username' not in config:
+        config['restore_username'] = str(uuid.uuid4())
+        write_restore_data = True
+    if 'restore_password' not in config:
+        config['restore_password'] = ''.join(
+            random.choice(string.ascii_letters + string.digits)
+            for i in range(30)
+        )
+        write_restore_data = True
+    if write_restore_data:
+        with open(os.path.join(var_config_d, '10-restore.json'), 'w') as f:
+            os.fchmod(f.fileno(), 0o600)
+            restore_out = {
+                'restore_path': config['restore_path'],
+                'restore_module': config['restore_module'],
+                'restore_username': config['restore_username'],
+                'restore_password': config['restore_password'],
+            }
+            json_dump_p(restore_out, f)
+    if not os.path.isdir(config['restore_path']):
+        os.makedirs(config['restore_path'])
+
+    # Generate the SSH keypair if it doesn't exist
+    if 'ssh_private_key_file' not in config:
+        subprocess.check_call([
+            'ssh-keygen', '-t', 'rsa', '-N', '', '-C', 'turku',
+            '-f', os.path.join(config['var_dir'], 'ssh_key')
+        ])
+        with open(os.path.join(config['var_dir'], 'ssh_key.pub')) as f:
+            config['ssh_public_key'] = f.read().rstrip()
+        config['ssh_public_key_file'] = os.path.join(config['var_dir'], 'ssh_key.pub')
+        config['ssh_private_key_file'] = os.path.join(config['var_dir'], 'ssh_key')
+
+    for s in config['sources']:
+        # Check for missing usernames/passwords
+        if not ('username' in config['sources'][s] or 'password' in config['sources'][s]):
+            sources_secrets_d = os.path.join(config['config_dir'], 'sources_secrets.d')
+            if 'username' not in config['sources'][s]:
+                config['sources'][s]['username'] = str(uuid.uuid4())
+            if 'password' not in config['sources'][s]:
+                config['sources'][s]['password'] = ''.join(
+                    random.choice(string.ascii_letters + string.digits)
+                    for i in range(30)
+                )
+            with open(os.path.join(var_sources_d, '10-' + s + '.json'), 'w') as f:
+                os.fchmod(f.fileno(), 0o600)
+                json_dump_p({
+                    s: {
+                        'username': config['sources'][s]['username'],
+                        'password': config['sources'][s]['password'],
+                    }
+                }, f)
+
+
+def api_call(api_url, cmd, post_data, timeout=5):
+    url = urllib.parse.urlparse(api_url)
+    if url.scheme == 'https':
+        h = http.client.HTTPSConnection(url.netloc, timeout=timeout)
+    else:
+        h = http.client.HTTPConnection(url.netloc, timeout=timeout)
+    out = json.dumps(post_data)
+    h.putrequest('POST', '%s/%s' % (url.path, cmd))
+    h.putheader('Content-Type', 'application/json')
+    h.putheader('Content-Length', len(out))
+    h.putheader('Accept', 'application/json')
+    h.endheaders()
+    h.send(out.encode('UTF-8'))
+
+    res = h.getresponse()
+    if not res.status == http.client.OK:
+        raise Exception('Received error %d (%s) from API server' % (res.status, res.reason))
+    if not res.getheader('content-type') == 'application/json':
+        raise Exception('Received invalid reply from API server')
+    try:
+        return json.loads(res.read().decode('UTF-8'))
+    except ValueError:
+        raise Exception('Received invalid reply from API server')