Merge lp:~allenap/maas/respawn-cluster-until-it-works into lp:~maas-maintainers/maas/packaging

[Gavin Panella]

Force a delay and respawn of the cluster controller when the shared-secret has not yet been installed.

There's a race when installing the MAAS packages on a new installation *or* when upgrading. If the cluster is started before the region, it will find no shared-secret and crash. This is because the pre-start script in /etc/init/maas-clusterd.conf checks for the secret. However, the secret is generated by the region when it starts. As long as the region starts before the cluster, all is well.

I propose moving the pre-start check for the secret into the script section. If the script does not exist, print out an explanatory message, sleep for 5 seconds, then die. Upstart will then respawn the job. If the region has by now started then the cluster controller will be started too.

[Julian Edwards (julian-edwards)]

review: Approve

[Raphaël Badin (rvb)]

review: Approve

[lp:~allenap/maas/respawn-cluster-until-it-works]

328. By Gavin Panella on 2014-10-15

Explain how to find the secret on the region.

[Gavin Panella (allenap)]

Thanks comrades!

=== modified file 'debian/maas-cluster-controller.maas-clusterd.upstart'
--- debian/maas-cluster-controller.maas-clusterd.upstart	2014-10-10 12:27:35 +0000
+++ debian/maas-cluster-controller.maas-clusterd.upstart	2014-10-15 11:12:32 +0000
@@ -18,22 +18,38 @@
         stop
         exit 0
     fi
-    if ! maas-provision check-for-shared-secret >/dev/null; then
-       echo 'A shared-secret has not been installed for this cluster.' >&2
-       echo 'Obtain the secret from the region controller and install' >&2
-       echo 'with `maas-provision install-shared-secret`.' >&2
-       stop
-       exit 0
-    fi
 end script
 
 script
-    # Prepare settings.
+    # Exit immediately on error, and treat unset variables as errors. In
+    # sh/dash, unfortunately, the following does not cause the use of an unset
+    # variable to halt the script, it merely prints an error and sets $?.
+    set -o errexit -o nounset
+    # Load the configuration file.
     . $CONFIG_FILE
-    # Allow the tftpd process to read CLUSTER_UUID as set in that config
-    # file.
+    # The MAAS cluster controller needs both CLUSTER_UUID and MAAS_URL to
+    # operate.
     export CLUSTER_UUID
     export MAAS_URL
+    # Check for the shared-secret. If it's not here, sleep for a while, then
+    # exit and allow respawn to do its thing.
+    if ! maas-provision check-for-shared-secret >/dev/null; then
+        fmt -w 72 <<-'EOF' >&2
+	A shared secret has not been installed for this cluster. Obtain
+	the secret from the region (find it in /var/lib/maas/secret once
+	the region controller has started for the first time) and
+	install it with `maas-provision install-shared-secret`.
+
+	However, if this machine is also serving as the region, ensure
+	that the region controller is started (`sudo service apache2
+	start` typically). It will create the shared secret, which
+	maas-clusterd will then find when it respawns in 5 seconds.
+
+	EOF
+        exec sleep 5
+    fi
     # To add options to your daemon, edit the line below:
-    exec /usr/bin/authbind --deep /usr/bin/twistd -n --uid=maas --gid=maas --pidfile=/run/maas-cluster.pid --logfile=/dev/null maas-pserv --config-file=/etc/maas/pserv.yaml
+    exec /usr/bin/authbind --deep /usr/bin/twistd \
+        --nodaemon --uid=maas --gid=maas --pidfile=/run/maas-cluster.pid \
+        --logfile=/dev/null maas-pserv --config-file=/etc/maas/pserv.yaml
 end script