Merge lp:~aisrael/juju-deployer/workaround-lp-1421315 into lp:juju-deployer

[Adam Israel]

I recently ran into a bug lp:1421315 while reviewing a merge request (lp:238283)

juju-deployer would throw an exception due to the service not being available in the watch status output. This fix expands on the original workaround, to wait 5.1 seconds for status (which is supposed to be available in 5 seconds) to be available. In this merge, the check is wrapped in a while block that will attempt to read the status for the specific unit, for up to 60 seconds, before failing.

I have tested this against the above merge request, and it eliminated all related testing errors.

[Adam Israel (aisrael)]

The merge that this was affecting is: https://code.launchpad.net/~stub/charms/precise/postgresql/enable-integration-tests/+merge/238283

[Tim Van Steenburgh (tvansteenburgh)]

+1 LGTM.

review: Approve

=== modified file 'deployer/action/importer.py'
--- deployer/action/importer.py	2015-03-24 17:38:56 +0000
+++ deployer/action/importer.py	2015-03-27 00:25:46 +0000
@@ -23,8 +23,22 @@
         env_status = self.env.status()
         reloaded = False
 
+        # Workaround an issue where watch output doesn't include subordinate
+        # services right away, and add_unit would fail while attempting to add
+        # units to a non-existant service. See lp:1421315 for details.
+        cur_units = 0
         for svc in self.deployment.get_services():
-            cur_units = len(env_status['services'][svc.name].get('units', ()))
+            delay = time.time() + 60
+            while delay > time.time():
+                if svc.name in env_status['services']:
+                    cur_units = len(
+                        env_status['services'][svc.name].get('units', ())
+                    )
+                    if cur_units > 0:
+                        break
+                time.sleep(5)
+                env_status = self.env.status()
+
             delta = (svc.num_units - cur_units)
 
             if delta <= 0:
@@ -45,9 +59,14 @@
             if svc.unit_placement:
                 # Reload status once after non placed services units are done.
                 if reloaded is False:
-                    # Crappy workaround juju-core api inconsistency
-                    time.sleep(5.1)
-                    env_status = self.env.status()
+                    # Improved crappy workaround juju-core api inconsistency
+                    delay = time.time() + 60
+                    while delay > time.time():
+                        time.sleep(5.1)
+                        env_status = self.env.status()
+                        if svc.name in env_status['services']:
+                            break
+
                     reloaded = True
 
                 placement = self.deployment.get_unit_placement(svc, env_status)
@@ -70,7 +89,7 @@
             foo: bar
         1:
           constraints: "mem=4G"
-        
+
         This method first attempts to create any machines in the 'machines'
         section of the bundle specification with the given constraints and
         annotations.  Then, if there are any machines requested for containers
@@ -81,7 +100,7 @@
             self.log.info("Creating machines...")
             for machine_name, spec in machines.items():
                 self.machines[machine_name] = self.env.add_machine(
-                        series=spec.get('series', 
+                        series=spec.get('series',
                                         self.deployment.data['series']),
                         constraints=spec.get('constraints'))
                 annotations = spec.get('annotations')