wait() waits many hrs, or even infinity

juju-wait will wait indefinitely for a settled status, and may keep waiting even though a unit is in an error state.

This is preventing us from using the new-ish mojo built-in wait, as we can't have our metal tied up for several hours in CI on failing jobs.

The mojo repo contains an old copy of juju-wait in tree.

Quite a while back we committed an optional max_wait to juju-wait address that, and have been using that as a post-deploy phase with juju-wait trunk to successfully wait for things to settle.

Mojo should consider re-freshing the juju-wait code, and plumbing the max_wait option through to be usable in a manifest.

This issue is likely scarce in production deploys. We run into it as we use Mojo to do charm testing, and sometimes those charms have issues. We need for the tooling to be as tunable and resilient to that as possible, as is possible with juju-wait from trunk.

Example:

#### mojo's juju-wait starts here

00:58:23.997 2016-09-22 08:12:33 [INFO] Waiting for environment to reach steady state
02:58:48.609 2016-09-22 10:12:58 [INFO] All units idle since 2016-09-22 10:12:25.640469Z (ceilometer/0, ceph-osd/0, ceph/0, ceph/1, ceph/2, cinder/0, glance/0, heat/0, keystone/0, mongodb/0, mysql/0, neutron-api/0, neutron-gateway/0, nova-cloud-controller/0, nova-compute/0, nova-compute/1, nova-compute/2, openstack-dashboard/0, rabbitmq-server/0, swift-proxy/0, swift-storage-z1/0, swift-storage-z2/0, swift-storage-z3/0)
02:58:48.609 2016-09-22 10:12:58 [INFO] Environment has reached steady state

#### mojo's juju-wait claims ready state, even though multiple units are in an error state

02:58:48.610 2016-09-22 10:12:58 [INFO] Manifest comment:
02:58:48.610
02:58:48.610 #############################################################################
02:58:48.610 Check juju statuses are green and that hooks have finished
02:58:48.610 #############################################################################
02:58:48.610
02:58:48.610
02:58:48.610 2016-09-22 10:12:58 [INFO] Pulling secrets from /srv/mojo/LOCAL/mojo-openstack-specs/specs/full_stack/stable_deploy_baremetal/mitaka to /srv/mojo/mojo-openstack-specs/xenial/osci-mojo/local
02:58:48.610 2016-09-22 10:12:58 [WARNING] Automatic secrets phase ran but secrets directory /srv/mojo/LOCAL/mojo-openstack-specs/specs/full_stack/stable_deploy_baremetal/mitaka does not exist!
02:58:48.611 2016-09-22 10:12:58 [INFO] Running script check_juju.py
03:43:51.354 2016-09-22 10:58:00 [WARNING] No debug log matching debug-logs found. Using default.

#### juju-wait trunk starts here

03:43:52.266 2016-09-22 10:58:01 [ERROR] INFO:root:Calling juju-wait
03:43:52.266 DEBUG:root:swift-storage-z1/0 workload status is error since 2016-09-22 10:12:34Z
03:43:52.266 DEBUG:root:swift-storage-z2/0 workload status is error since 2016-09-22 10:12:32Z
03:43:52.266 DEBUG:root:swift-storage-z3/0 workload status is error since 2016-09-22 10:12:30Z