Merge lp:~vila/uci-engine/nova-failure into lp:uci-engine

Proposed by Vincent Ladeuil
Status: Rejected
Rejected by: Vincent Ladeuil
Proposed branch: lp:~vila/uci-engine/nova-failure
Merge into: lp:uci-engine
Diff against target: 127 lines (+23/-19)
4 files modified
ci-utils/ci_utils/testing/features.py (+1/-9)
test_runner/tstrun/testbed.py (+14/-3)
test_runner/tstrun/tests/test_testbed.py (+3/-3)
tests/test_britney.py (+5/-4)
To merge this branch: bzr merge lp:~vila/uci-engine/nova-failure
Reviewer Review Type Date Requested Status
Francis Ginther Approve
Para Siva (community) Approve
Review via email: mp+245764@code.launchpad.net

Commit message

Turn nova exceptions into TestBedException to identify infra failures

Description of the change

Most of the testbed related issues were already raising TestBedException except for the nova issues.

With this patch *all* testbed issues (nova or not) can be identified at higher levels with that single exception.

To post a comment you must log in.
Revision history for this message
Para Siva (psivaa) wrote :

One question inline, about removing the 'skipping' of the integration tests altogether. Apart from that, it looks good.

review: Approve
Revision history for this message
Francis Ginther (fginther) wrote :

Looks good. It looks odd to re-enable the Swift feature with this MP, but I have no good reason to create a separate MP just for that. Ultimately if there is trouble landing this due to the Swift tests having problems in tarmac, then we can rework in that event.

review: Approve
Revision history for this message
Vincent Ladeuil (vila) wrote :

Oops, got confused when submitting this, thanks for the heads-up !

> Ultimately if there is trouble landing this due to the Swift tests having problems in tarmac, then we can rework in that event.

/me nods

Yup, good summary, I'll try to land and will revisit a more focused MP if I encounter issues.

Revision history for this message
Para Siva (psivaa) wrote :

Just clarified my comment. +1

Revision history for this message
Ubuntu CI Bot (uci-bot) wrote :
Download full text (117.8 KiB)

The attempt to merge lp:~vila/uci-engine/nova-failure into lp:uci-engine failed. Below is the output from the failed tests.

Checking juju status
Private PPAs: disabled
Preparing local branch upload...
Uploading local branch, fingerprint 87df68656e30adc88219166e790e1cc78ebbc61b
Installing keys from bzr+ssh://bazaar.launchpad.net/~ci-engineering-private/+junk/ci-airline-dev-keys/
Running juju-deployer -v -c /tmp/tmpI70Mjg/deployer/relations.yaml -c /tmp/tmpI70Mjg/deployer/services.yaml ci-airline
Tests running...
ci-utils.ci_utils.tests.test_amqp.TestAMQP.testConnectFailed ... OK (0.002 secs)
ci-utils.ci_utils.tests.test_amqp.TestAMQP.testProcessQueue ... OK (0.003 secs)
ci-utils.ci_utils.tests.test_amqp.TestAMQP.testRunForever ... OK (0.102 secs)
ci-utils.ci_utils.tests.test_amqp.TestAMQP.testSent ... OK (0.002 secs)
ci-utils.ci_utils.tests.test_amqp.TestProgressTrigger.testProgress ... OK (0.001 secs)
ci-utils.ci_utils.tests.test_amqp_worker.TestAMQPWorker.testCancel ... OK (0.108 secs)
ci-utils.ci_utils.tests.test_amqp_worker.TestAMQPWorker.testNoQueue ... OK (0.005 secs)
ci-utils.ci_utils.tests.test_amqp_worker.TestAMQPWorker.testNoTicket ... OK (0.004 secs)
ci-utils.ci_utils.tests.test_amqp_worker.TestAMQPWorker.testOnMessageCalledProcessError ... OK (0.004 secs)
ci-utils.ci_utils.tests.test_amqp_worker.TestAMQPWorker.testOnMessageFail ... OK (0.003 secs)
ci-utils.ci_utils.tests.test_amqp_worker.TestAMQPWorker.testOnMessageKilled ... OK (0.003 secs)
ci-utils.ci_utils.tests.test_amqp_worker.TestAMQPWorker.testOnMessageSimple ... OK (0.003 secs)
ci-utils.ci_utils.tests.test_amqp_worker.TestAMQPWorker.testOnMessageUnexpected ... OK (0.003 secs)
ci-utils.ci_utils.tests.test_amqp_worker.TestAMQPWorker.testSaveLastRun ... OK (0.001 secs)
ci-utils.ci_utils.tests.test_amqp_worker.TestTimer.testCBRuns ... OK (0.021 secs)
ci-utils.ci_utils.tests.test_amqp_worker.TestTimer.testCanCancel ... OK (0.001 secs)
ci-utils.ci_utils.tests.test_data_store.TestDataStoreConfig.test_invalid_auth_config ... OK (0.000 secs)
ci-utils.ci_utils.tests.test_data_store.TestDataStoreConfig.test_valid_auth_config ... OK (0.000 secs)
ci-utils.ci_utils.tests.test_data_store.TestDataStoreFileName.test_abspath ... OK (0.000 secs)
ci-utils.ci_utils.tests.test_data_store.TestDataStoreFileName.test_basename ... OK (0.000 secs)
ci-utils.ci_utils.tests.test_data_store.TestDataStoreFileName.test_relpath ... OK (0.000 secs)
ci-utils.ci_utils.tests.test_deployer.TestDeployer.test_load_empty_single_config ... OK (0.004 secs)
ci-utils.ci_utils.tests.test_deployer.TestDeployer.test_load_multiple_config ... OK (0.007 secs)
ci-utils.ci_utils.tests.test_deployer.TestDeployer.test_load_single_config ... OK (0.004 secs)
ci-utils.ci_utils.tests.test_features.TestJujuControlBucket.test_no_control_bucket ... OK (0.003 secs)
ci-utils.ci_utils.tests.test_features.TestJujuGetEnvsFromYaml.test_broken_environments_file ... OK (0.001 secs)
ci-utils.ci_utils.tests.test_features.TestJujuGetEnvsFromYaml.test_inconsistent_environments_file ... OK (0.003 secs)
ci-utils.ci_utils.tests.test_features.TestJujuNova.test_get_nova_id_malformed ... OK (0.000 secs)
ci-utils.ci_utils.tests.test_features.TestJujuNov...

Revision history for this message
Vincent Ladeuil (vila) wrote :
Download full text (4.8 KiB)

FAIL: tests.test_test_runner.TestTestRunner.test_process_ticket

TestBedException: None of [m1.smaller,m1.small,standard.small] can be found

FAIL: tests.test_test_runner.TestTestRunner.test_run_tests

TestBedException: None of [m1.smaller,m1.small,standard.small] can be found

Huh ?

======================================================================
FAIL: tests.test_webui.WebUITest.test_edit_ticket
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/tarmac/branch.ODIjlD/bin/../tests/test_webui.py", line 291, in test_edit_ticket
    wait_for(get_element_by_css, 'a.yui3-pagview-link-page-active')
  File "/dev/shm/venv-a55Hp9/local/lib/python2.7/site-packages/sst-0.2.4-py2.7.egg/sst/actions.py", line 145, in inner
    return func(*args, **kwargs)
  File "/dev/shm/venv-a55Hp9/local/lib/python2.7/site-packages/sst-0.2.4-py2.7.egg/sst/actions.py", line 852, in wait_for
    return _wait_for(condition, False, _TIMEOUT, _POLL, *args, **kwargs)
  File "/dev/shm/venv-a55Hp9/local/lib/python2.7/site-packages/sst-0.2.4-py2.7.egg/sst/actions.py", line 816, in _wait_for
    _raise(error)
  File "/dev/shm/venv-a55Hp9/local/lib/python2.7/site-packages/sst-0.2.4-py2.7.egg/sst/actions.py", line 118, in _raise
    raise AssertionError(msg)
AssertionError: Timed out waiting for: get_element_by_css
Error during wait: Could not identify element: 0 elements found
======================================================================
FAIL: tests.test_webui.WebUITest.test_edit_ticket_sources_and_mps
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/tarmac/branch.ODIjlD/bin/../tests/test_webui.py", line 354, in test_edit_ticket_sources_and_mps
    wait_for(get_element_by_css, 'a.yui3-pagview-link-page-active')
  File "/dev/shm/venv-a55Hp9/local/lib/python2.7/site-packages/sst-0.2.4-py2.7.egg/sst/actions.py", line 145, in inner
    return func(*args, **kwargs)
  File "/dev/shm/venv-a55Hp9/local/lib/python2.7/site-packages/sst-0.2.4-py2.7.egg/sst/actions.py", line 852, in wait_for
    return _wait_for(condition, False, _TIMEOUT, _POLL, *args, **kwargs)
  File "/dev/shm/venv-a55Hp9/local/lib/python2.7/site-packages/sst-0.2.4-py2.7.egg/sst/actions.py", line 816, in _wait_for
    _raise(error)
  File "/dev/shm/venv-a55Hp9/local/lib/python2.7/site-packages/sst-0.2.4-py2.7.egg/sst/actions.py", line 118, in _raise
    raise AssertionError(msg)
AssertionError: Timed out waiting for: get_element_by_css
Error during wait: Could not identify element: 0 elements found
======================================================================
FAIL: tests.test_webui.WebUITest.test_edit_ticket_with_invalid_mps
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/tarmac/branch.ODIjlD/bin/../tests/test_webui.py", line 490, in test_edit_ticket_with_invalid_mps
    wait_for(get_element_by_css, 'a.yui3-pagview-link-page-active')
  File "/dev/shm/venv-a55Hp9/local/lib/python2.7/site-packages/sst-0.2.4-py2.7.egg/sst/actions.py", line 145, in inner
    return func(*args, **kwargs)
  File "/dev...

Read more...

Unmerged revisions

914. By Vincent Ladeuil

Raise TestBedException for nova related failures too so all testbed related issues can be detected at higher levels.

913. By Vincent Ladeuil

Fix missed fallout from an old refactoring moving the config part out of britney/post_request.py.

912. By Vincent Ladeuil

Activate Swift and Nova features again so integration tests are not skipped anymore.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'ci-utils/ci_utils/testing/features.py'
--- ci-utils/ci_utils/testing/features.py 2014-11-26 08:09:56 +0000
+++ ci-utils/ci_utils/testing/features.py 2015-01-07 16:35:28 +0000
@@ -1,5 +1,5 @@
1# Ubuntu CI Engine1# Ubuntu CI Engine
2# Copyright 2014 Canonical Ltd.2# Copyright 2014, 2015 Canonical Ltd.
33
4# This program is free software: you can redistribute it and/or modify it4# This program is free software: you can redistribute it and/or modify it
5# under the terms of the GNU Affero General Public License version 3, as5# under the terms of the GNU Affero General Public License version 3, as
@@ -36,10 +36,6 @@
36class Swift(features.Feature):36class Swift(features.Feature):
3737
38 def _probe(self):38 def _probe(self):
39 # FIXME: swift needs to handle more transient failures, tests are
40 # currently failing when swift becomes too slow. Disable all tests that
41 # require a "perfect" swift -- vila 2014-11-26
42 return False
43 conn = self.get_connection()39 conn = self.get_connection()
44 if conn is None:40 if conn is None:
45 return False41 return False
@@ -71,10 +67,6 @@
71class NovaCompute(features.Feature):67class NovaCompute(features.Feature):
7268
73 def _probe(self):69 def _probe(self):
74 # FIXME: nova needs to handle more transient failures, tests are
75 # currently failing for various known reasons. Disable all tests that
76 # require a "perfect" nova. -- vila 2014-11-26
77 return False
78 client = self.get_client()70 client = self.get_client()
79 if client is None:71 if client is None:
80 return False72 return False
8173
=== modified file 'test_runner/tstrun/testbed.py'
--- test_runner/tstrun/testbed.py 2014-10-17 12:28:22 +0000
+++ test_runner/tstrun/testbed.py 2015-01-07 16:35:28 +0000
@@ -1,6 +1,6 @@
1#!/usr/bin/env python1#!/usr/bin/env python
2# Ubuntu CI Engine2# Ubuntu CI Engine
3# Copyright 2014 Canonical Ltd.3# Copyright 2014, 2015 Canonical Ltd.
44
5# This program is free software: you can redistribute it and/or modify it5# This program is free software: you can redistribute it and/or modify it
6# under the terms of the GNU Affero General Public License version 3, as6# under the terms of the GNU Affero General Public License version 3, as
@@ -178,8 +178,19 @@
178 msg = ('Rate limit reached for {},'178 msg = ('Rate limit reached for {},'
179 ' will sleep for {} seconds')179 ' will sleep for {} seconds')
180 self.logger.exception(msg.format(func.__name__, nap_time))180 self.logger.exception(msg.format(func.__name__, nap_time))
181 time.sleep(nap_time)181 except:
182 return func(*args, **kwargs) # Retry once182 # All other exceptions are raised
183 self.logger.exception('{} failed'.format(func.__name__))
184 raise TestBedException('{} failed'.format(func.__name__))
185 try:
186 time.sleep(nap_time)
187 return func(*args, **kwargs) # Retry once
188 except:
189 # FIXME: This duplicates the two lines above but this method will
190 # be refactored to retry more than once in the future. The
191 # duplication can be fixed then -- vila 2015-01-07
192 self.logger.exception('{} failed'.format(func.__name__))
193 raise TestBedException('{} failed'.format(func.__name__))
183194
184 def flavors_list(self):195 def flavors_list(self):
185 return self.retry(self.nova.flavors.list)196 return self.retry(self.nova.flavors.list)
186197
=== modified file 'test_runner/tstrun/tests/test_testbed.py'
--- test_runner/tstrun/tests/test_testbed.py 2014-11-04 12:07:12 +0000
+++ test_runner/tstrun/tests/test_testbed.py 2015-01-07 16:35:28 +0000
@@ -1,5 +1,5 @@
1# Ubuntu CI Engine1# Ubuntu CI Engine
2# Copyright 2014 Canonical Ltd.2# Copyright 2014, 2015 Canonical Ltd.
33
4# This program is free software: you can redistribute it and/or modify it4# This program is free software: you can redistribute it and/or modify it
5# under the terms of the GNU Affero General Public License version 3, as5# under the terms of the GNU Affero General Public License version 3, as
@@ -128,9 +128,9 @@
128 # This mimics what will happen when we encounter unknown transient128 # This mimics what will happen when we encounter unknown transient
129 # failures we want to catch: an exception will bubble up and we'll have129 # failures we want to catch: an exception will bubble up and we'll have
130 # to add it to NovaClient.retry().130 # to add it to NovaClient.retry().
131 with self.assertRaises(AssertionError) as cm:131 with self.assertRaises(testbed.TestBedException) as cm:
132 tb.find_flavor()132 tb.find_flavor()
133 self.assertEqual('Boom!', unicode(cm.exception))133 self.assertEqual('fail failed', '{}'.format(cm.exception))
134134
135135
136@features.requires(features.nova_creds)136@features.requires(features.nova_creds)
137137
=== modified file 'tests/test_britney.py'
--- tests/test_britney.py 2014-11-03 12:59:48 +0000
+++ tests/test_britney.py 2015-01-07 16:35:28 +0000
@@ -1,6 +1,6 @@
1#!/usr/bin/env python1#!/usr/bin/env python
2# Ubuntu Continuous Integration Engine2# Ubuntu Continuous Integration Engine
3# Copyright 2014 Canonical Ltd.3# Copyright 2014, 2015 Canonical Ltd.
44
5# This program is free software: you can redistribute it and/or modify it5# This program is free software: you can redistribute it and/or modify it
6# under the terms of the GNU Affero General Public License version 3, as6# under the terms of the GNU Affero General Public License version 3, as
@@ -37,6 +37,7 @@
3737
3838
39from britney import (39from britney import (
40 config,
40 post_request,41 post_request,
41 process_results,42 process_results,
42)43)
@@ -186,15 +187,15 @@
186 password = 's3cr3t'187 password = 's3cr3t'
187 rabbit_ip = get_rabbit_ip(status)188 rabbit_ip = get_rabbit_ip(status)
188 deployers.expose_rabbit(self, rabbit_ip, user, password)189 deployers.expose_rabbit(self, rabbit_ip, user, password)
189 os.makedirs(post_request.user_config_dir())190 os.makedirs(config.user_config_dir())
190 self.config = post_request.BritneyStack()191 self.config = config.BritneyStack()
191 self.config.store._load_from_string('''\192 self.config.store._load_from_string('''\
192uci.rabbit.host = {rabbit_ip}193uci.rabbit.host = {rabbit_ip}
193uci.rabbit.user = {user}194uci.rabbit.user = {user}
194uci.rabbit.password = {password}195uci.rabbit.password = {password}
195 '''.format(**locals()))196 '''.format(**locals()))
196 fixtures.patch(self, amqp_utils, 'get_config',197 fixtures.patch(self, amqp_utils, 'get_config',
197 post_request.amqp_config(self.config))198 config.amqp_config(self.config))
198 self.out = StringIO()199 self.out = StringIO()
199 self.err = StringIO()200 self.err = StringIO()
200201

Subscribers

People subscribed via source and target branches