Merge lp:~ev/uci-engine/requires-ready-deployment into lp:uci-engine

[Evan]

Require a ready deployment before running the integration tests.

Require a ready deployment before running the integration tests.

[PS Jenkins bot (ps-jenkins)]

PASSED: Continuous integration, rev:898
http://s-jenkins.ubuntu-ci:8080/job/uci-engine-ci/1719/
Executed test runs:

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/uci-engine-ci/1719/rebuild

review: Approve (continuous-integration)

[Francis Ginther (fginther)]

This should make the issue of a not-ready deployment more obvious.

One inline comment.

review: Needs Fixing

[Evan (ev)]

Good call. Fixed.

[PS Jenkins bot (ps-jenkins)]

PASSED: Continuous integration, rev:899
http://s-jenkins.ubuntu-ci:8080/job/uci-engine-ci/1721/
Executed test runs:

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/uci-engine-ci/1721/rebuild

review: Approve (continuous-integration)

[Francis Ginther (fginther)]

Approved

review: Approve

[Ubuntu CI Bot (uci-bot)]

The attempt to merge lp:~ev/uci-engine/requires-ready-deployment into lp:uci-engine failed. Below is the output from the failed tests.

Running cm...
Updating source dependencies...
Updating source dependencies...
Updating source dependencies...
Updating source dependencies...
Updating source dependencies...
Updating source dependencies...
uploading webui-content.tgz to swift
Updating source dependencies...
Checking juju status
Private PPAs: disabled
Preparing local branch upload...
Uploading local branch, fingerprint ce92644bf713fd75458271a884f112468c90ec3b
Building charm: lander
Building charm: wsgi-app
Building charm: rabbitmq-worker
Building charm: chroot-builder
Building charm: key-secret-subordinate
Building charm: webui
Building charm: system-image-server
Installing keys from bzr+ssh://bazaar.launchpad.net/~ci-engineering-private/+junk/ci-airline-dev-keys/
Running juju-deployer -v -c /tmp/tmp4tIrwB/deployer/branch-source-builder.yaml -c /tmp/tmp4tIrwB/deployer/britney-proxy.yaml -c /tmp/tmp4tIrwB/deployer/coverage-retriever.yaml -c /tmp/tmp4tIrwB/deployer/gatekeeper.yaml -c /tmp/tmp4tIrwB/deployer/image-builder.yaml -c /tmp/tmp4tIrwB/deployer/lander.yaml -c /tmp/tmp4tIrwB/deployer/nf-stats-service.yaml -c /tmp/tmp4tIrwB/deployer/ppa-creator.yaml -c /tmp/tmp4tIrwB/deployer/publisher.yaml -c /tmp/tmp4tIrwB/deployer/relations.yaml -c /tmp/tmp4tIrwB/deployer/test-runner.yaml -c /tmp/tmp4tIrwB/deployer/ticket-system.yaml -c /tmp/tmp4tIrwB/deployer/validator.yaml -c /tmp/tmp4tIrwB/deployer/webui.yaml ci-airline
Tests running...
ci-utils.ci_utils.tests.test_amqp.TestAMQP.testConnectFailed ... OK (0.002 secs)
ci-utils.ci_utils.tests.test_amqp.TestAMQP.testProcessQueue ... OK (0.002 secs)
ci-utils.ci_utils.tests.test_amqp.TestAMQP.testRunForever ... OK (0.103 secs)
ci-utils.ci_utils.tests.test_amqp.TestAMQP.testSent ... OK (0.004 secs)
ci-utils.ci_utils.tests.test_amqp.TestProgressTrigger.testProgress ... OK (0.002 secs)
ci-utils.ci_utils.tests.test_amqp_worker.TestAMQPWorker.testCancel ... OK (0.106 secs)
ci-utils.ci_utils.tests.test_amqp_worker.TestAMQPWorker.testNoQueue ... OK (0.003 secs)
ci-utils.ci_utils.tests.test_amqp_worker.TestAMQPWorker.testNoTicket ... OK (0.005 secs)
ci-utils.ci_utils.tests.test_amqp_worker.TestAMQPWorker.testOnMessageCalledProcessError ... OK (0.007 secs)
ci-utils.ci_utils.tests.test_amqp_worker.TestAMQPWorker.testOnMessageFail ... OK (0.002 secs)
ci-utils.ci_utils.tests.test_amqp_worker.TestAMQPWorker.testOnMessageKilled ... OK (0.002 secs)
ci-utils.ci_utils.tests.test_amqp_worker.TestAMQPWorker.testOnMessageSimple ... OK (0.002 secs)
ci-utils.ci_utils.tests.test_amqp_worker.TestAMQPWorker.testOnMessageUnexpected ... OK (0.003 secs)
ci-utils.ci_utils.tests.test_amqp_worker.TestAMQPWorker.testSaveLastRun ... OK (0.001 secs)
ci-utils.ci_utils.tests.test_amqp_worker.TestTimer.testCBRuns ... OK (0.021 secs)
ci-utils.ci_utils.tests.test_amqp_worker.TestTimer.testCanCancel ... OK (0.000 secs)
ci-utils.ci_utils.tests.test_data_store.TestDataStoreConfig.test_invalid_auth_config ... OK (0.000 secs)
ci-utils.ci_utils.tests.test_data_store.TestDataStoreConfig.test_valid_auth_config ... OK (0.000 secs)
ci-utils.ci_utils.test...

[Joe Talbott (joetalbott)]

On Mon, Nov 17, 2014 at 08:21:04PM -0000, Evan Dandrea wrote:
> Evan Dandrea has proposed merging lp:~ev/uci-engine/requires-ready-deployment into lp:uci-engine.
>
> Requested reviews:
> Canonical CI Engineering (canonical-ci-engineering)
>
> For more details, see:
> https://code.launchpad.net/~ev/uci-engine/requires-ready-deployment/+merge/242023
>
> Require a ready deployment before running the integration tests.
>

See my comments in-line. Nothing to halt landing this MP just food for
thought.

[snip]
> +class ReadyDeployment(features.Feature):
> + def _probe(self):

I think it was Ned Batchelder that did a talk where he admonished using
a class with a single method in it and asked "why not just use a
method?"

> + print 'here we go'

I think this can be removed, right?

> + try:
> + out = subprocess.check_output(['juju', 'status'])
> + except subprocess.CalledProcessError:
> + return False

We should probably log something here.

> +
> + status = yaml.safe_load(out)
> + for service, service_data in status['services'].iteritems():
> + for unit, unit_data in service_data['units'].iteritems():
> + agent_state = unit_data['agent-state']
> + if agent_state in ('pending', 'error'):
> + msg = '{} in {} state.'.format(unit, agent_state)
> + print >>sys.stderr, msg

I seem to recall this method of writing to stderr being frowned upon.
For python3 compatibility you can do something like:

  from future import print_function

  print(msg, file=sys.stderr)

[snip]

Unmerged revisions

899. By Evan on 2014-11-17

Treat all non-started states as an error.

898. By Evan on 2014-11-17

Remove debugging statement.

897. By Evan on 2014-11-17

Add a feature requirement of a ready deployment.

=== modified file 'ci-utils/ci_utils/testing/features.py'
--- ci-utils/ci_utils/testing/features.py	2014-10-15 10:35:04 +0000
+++ ci-utils/ci_utils/testing/features.py	2014-11-17 23:53:11 +0000
@@ -14,6 +14,7 @@
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 import errno
 import os
+import sys
 import subprocess
 
 import swiftclient
@@ -122,6 +123,27 @@
 nova_creds = NovaCredentials()
 
 
+class ReadyDeployment(features.Feature):
+    def _probe(self):
+        try:
+            out = subprocess.check_output(['juju', 'status'])
+        except subprocess.CalledProcessError:
+            return False
+
+        status = yaml.safe_load(out)
+        for service, service_data in status['services'].iteritems():
+            for unit, unit_data in service_data['units'].iteritems():
+                agent_state = unit_data['agent-state']
+                if agent_state != 'started':
+                    msg = '{} in {} state.'.format(unit, agent_state)
+                    print >>sys.stderr, msg
+                    return False
+        return True
+
+
+ready_deployment = ReadyDeployment()
+
+
 class BootstrappedJuju(features.Feature):
 
     def __init__(self):

=== modified file 'tests/deployers.py'
--- tests/deployers.py	2014-11-06 13:40:54 +0000
+++ tests/deployers.py	2014-11-17 23:53:11 +0000
@@ -92,8 +92,7 @@
     return ip, int(port)
 
 
-# TODO ev 2014-06-16 @features.requires(features.settled_units)
[e-mail](features.bootstrapped_juju)
[e-mail](features.ready_deployment)
 class DeployerTest(testing.TestCaseWithGnupg):
     '''Base class for building juju deployer based tests.'''