Merge lp:~allenap/maas/respawn-cluster-until-it-works into lp:~maas-maintainers/maas/packaging

Proposed by Gavin Panella
Status: Merged
Approved by: Gavin Panella
Approved revision: 328
Merged at revision: 327
Proposed branch: lp:~allenap/maas/respawn-cluster-until-it-works
Merge into: lp:~maas-maintainers/maas/packaging
Diff against target: 53 lines (+27/-11)
1 file modified
debian/maas-cluster-controller.maas-clusterd.upstart (+27/-11)
To merge this branch: bzr merge lp:~allenap/maas/respawn-cluster-until-it-works
Reviewer Review Type Date Requested Status
Raphaël Badin (community) Approve
Julian Edwards (community) Approve
Andres Rodriguez Pending
Review via email:

Commit message

Force a delay and respawn of the cluster controller when the shared-secret has not yet been installed.

Description of the change

There's a race when installing the MAAS packages on a new installation *or* when upgrading. If the cluster is started before the region, it will find no shared-secret and crash. This is because the pre-start script in /etc/init/maas-clusterd.conf checks for the secret. However, the secret is generated by the region when it starts. As long as the region starts before the cluster, all is well.

I propose moving the pre-start check for the secret into the script section. If the script does not exist, print out an explanatory message, sleep for 5 seconds, then die. Upstart will then respawn the job. If the region has by now started then the cluster controller will be started too.

To post a comment you must log in.
Revision history for this message
Julian Edwards (julian-edwards) :
review: Approve
Revision history for this message
Raphaël Badin (rvb) :
review: Approve
328. By Gavin Panella

Explain how to find the secret on the region.

Revision history for this message
Gavin Panella (allenap) wrote :

Thanks comrades!

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'debian/maas-cluster-controller.maas-clusterd.upstart'
2--- debian/maas-cluster-controller.maas-clusterd.upstart 2014-10-10 12:27:35 +0000
3+++ debian/maas-cluster-controller.maas-clusterd.upstart 2014-10-15 11:12:32 +0000
4@@ -18,22 +18,38 @@
5 stop
6 exit 0
7 fi
8- if ! maas-provision check-for-shared-secret >/dev/null; then
9- echo 'A shared-secret has not been installed for this cluster.' >&2
10- echo 'Obtain the secret from the region controller and install' >&2
11- echo 'with `maas-provision install-shared-secret`.' >&2
12- stop
13- exit 0
14- fi
15 end script
17 script
18- # Prepare settings.
19+ # Exit immediately on error, and treat unset variables as errors. In
20+ # sh/dash, unfortunately, the following does not cause the use of an unset
21+ # variable to halt the script, it merely prints an error and sets $?.
22+ set -o errexit -o nounset
23+ # Load the configuration file.
25- # Allow the tftpd process to read CLUSTER_UUID as set in that config
26- # file.
27+ # The MAAS cluster controller needs both CLUSTER_UUID and MAAS_URL to
28+ # operate.
29 export CLUSTER_UUID
30 export MAAS_URL
31+ # Check for the shared-secret. If it's not here, sleep for a while, then
32+ # exit and allow respawn to do its thing.
33+ if ! maas-provision check-for-shared-secret >/dev/null; then
34+ fmt -w 72 <<-'EOF' >&2
35+ A shared secret has not been installed for this cluster. Obtain
36+ the secret from the region (find it in /var/lib/maas/secret once
37+ the region controller has started for the first time) and
38+ install it with `maas-provision install-shared-secret`.
40+ However, if this machine is also serving as the region, ensure
41+ that the region controller is started (`sudo service apache2
42+ start` typically). It will create the shared secret, which
43+ maas-clusterd will then find when it respawns in 5 seconds.
45+ EOF
46+ exec sleep 5
47+ fi
48 # To add options to your daemon, edit the line below:
49- exec /usr/bin/authbind --deep /usr/bin/twistd -n --uid=maas --gid=maas --pidfile=/run/ --logfile=/dev/null maas-pserv --config-file=/etc/maas/pserv.yaml
50+ exec /usr/bin/authbind --deep /usr/bin/twistd \
51+ --nodaemon --uid=maas --gid=maas --pidfile=/run/ \
52+ --logfile=/dev/null maas-pserv --config-file=/etc/maas/pserv.yaml
53 end script


People subscribed via source and target branches