Merge lp:~1chb1n/charms/trusty/ceilometer/next.1601-test-update1 into lp:~openstack-charmers-archive/charms/trusty/ceilometer/next

Proposed by Ryan Beisner on 2016-01-08
Status: Merged
Merged at revision: 112
Proposed branch: lp:~1chb1n/charms/trusty/ceilometer/next.1601-test-update1
Merge into: lp:~openstack-charmers-archive/charms/trusty/ceilometer/next
Diff against target: 254 lines (+118/-76)
6 files modified
Makefile (+1/-0)
tests/019-basic-trusty-mitaka (+11/-0)
tests/019-basic-vivid-kilo (+0/-9)
tests/021-basic-xenial-mitaka (+9/-0)
tests/README (+96/-66)
tests/tests.yaml (+1/-1)
To merge this branch: bzr merge lp:~1chb1n/charms/trusty/ceilometer/next.1601-test-update1
Reviewer Review Type Date Requested Status
OpenStack Charmers 2016-01-08 Pending
Review via email: mp+282053@code.launchpad.net

Description of the change

Cosmetic updates to tests/dir and Makefile to standardize *os-charms, prep for 16.01. No changes to test code, hooks or charmhelpers.

To post a comment you must log in.

charm_unit_test #15731 ceilometer-next for 1chb1n mp282053
    UNIT FAIL: unit-test failed

UNIT Results (max last 2 lines):
Makefile:15: *** missing separator (did you mean TAB instead of 8 spaces?). Stop.
ERROR:root:Make target returned non-zero.

Full unit test output: http://paste.ubuntu.com/14440496/
Build: http://10.245.162.77:8080/job/charm_unit_test/15731/

charm_lint_check #16845 ceilometer-next for 1chb1n mp282053
    LINT FAIL: lint-test failed

LINT Results (max last 2 lines):
Makefile:15: *** missing separator (did you mean TAB instead of 8 spaces?). Stop.
ERROR:root:Make target returned non-zero.

Full lint test output: http://paste.ubuntu.com/14440497/
Build: http://10.245.162.77:8080/job/charm_lint_check/16845/

charm_lint_check #16858 ceilometer-next for 1chb1n mp282053
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/16858/

charm_unit_test #15746 ceilometer-next for 1chb1n mp282053
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/15746/

charm_lint_check #16861 ceilometer-next for 1chb1n mp282053
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/16861/

112. By Ryan Beisner on 2016-01-08

remove amulet tests for unsupported releases

113. By Ryan Beisner on 2016-01-08

Move 00-setup to prevent extra, unnecessary bootstrap in test runs.

114. By Ryan Beisner on 2016-01-08

Re-number amulet test file names; add missing combos as not-executable for now.

115. By Ryan Beisner on 2016-01-08

Update tests/README

116. By Ryan Beisner on 2016-01-08

Update bundletester testplan yaml file

charm_lint_check #16900 ceilometer-next for 1chb1n mp282053
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/16900/

charm_unit_test #15786 ceilometer-next for 1chb1n mp282053
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/15786/

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'Makefile'
2--- Makefile 2015-10-16 11:59:37 +0000
3+++ Makefile 2016-01-08 21:46:24 +0000
4@@ -12,6 +12,7 @@
5
6 functional_test:
7 @echo Starting Amulet tests...
8+ @tests/setup/00-setup
9 @juju test -v -p AMULET_HTTP_PROXY,AMULET_OS_VIP --timeout 2700
10
11 bin/charm_helpers_sync.py:
12
13=== renamed file 'tests/020-basic-trusty-liberty' => 'tests/018-basic-trusty-liberty'
14=== added file 'tests/019-basic-trusty-mitaka'
15--- tests/019-basic-trusty-mitaka 1970-01-01 00:00:00 +0000
16+++ tests/019-basic-trusty-mitaka 2016-01-08 21:46:24 +0000
17@@ -0,0 +1,11 @@
18+#!/usr/bin/python
19+
20+"""Amulet tests on a basic ceilometer deployment on trusty-liberty."""
21+
22+from basic_deployment import CeilometerBasicDeployment
23+
24+if __name__ == '__main__':
25+ deployment = CeilometerBasicDeployment(series='trusty',
26+ openstack='cloud:trusty-liberty',
27+ source='cloud:trusty-updates/liberty')
28+ deployment.run_tests()
29
30=== removed file 'tests/019-basic-vivid-kilo'
31--- tests/019-basic-vivid-kilo 2015-06-12 14:56:01 +0000
32+++ tests/019-basic-vivid-kilo 1970-01-01 00:00:00 +0000
33@@ -1,9 +0,0 @@
34-#!/usr/bin/python
35-
36-"""Amulet tests on a basic ceilometer deployment on vivid-kilo."""
37-
38-from basic_deployment import CeilometerBasicDeployment
39-
40-if __name__ == '__main__':
41- deployment = CeilometerBasicDeployment(series='vivid')
42- deployment.run_tests()
43
44=== renamed file 'tests/021-basic-wily-liberty' => 'tests/020-basic-wily-liberty'
45=== added file 'tests/021-basic-xenial-mitaka'
46--- tests/021-basic-xenial-mitaka 1970-01-01 00:00:00 +0000
47+++ tests/021-basic-xenial-mitaka 2016-01-08 21:46:24 +0000
48@@ -0,0 +1,9 @@
49+#!/usr/bin/python
50+
51+"""Amulet tests on a basic ceilometer deployment on xenial-mitaka."""
52+
53+from basic_deployment import CeilometerBasicDeployment
54+
55+if __name__ == '__main__':
56+ deployment = CeilometerBasicDeployment(series='xenial')
57+ deployment.run_tests()
58
59=== modified file 'tests/README'
60--- tests/README 2015-06-12 14:56:01 +0000
61+++ tests/README 2016-01-08 21:46:24 +0000
62@@ -1,83 +1,113 @@
63-This directory provides Amulet tests that focus on verification of
64-ceilometer deployments.
65-
66-test_* methods are called in lexical sort order.
67-
68-Test name convention to ensure desired test order:
69+This directory provides Amulet tests to verify basic deployment functionality
70+from the perspective of this charm, its requirements and its features, as
71+exercised in a subset of the full OpenStack deployment test bundle topology.
72+
73+Reference: lp:openstack-charm-testing for full test bundles.
74+
75+A single topology and configuration is defined and deployed, once for each of
76+the defined Ubuntu:OpenStack release combos. The ongoing goal is for this
77+charm to always possess tests and combo definitions for all currently-supported
78+release combinations of U:OS.
79+
80+test_* methods are called in lexical sort order, as with most runners. However,
81+each individual test method should be idempotent and expected to pass regardless
82+of run order or Ubuntu:OpenStack combo. When writing or modifying tests,
83+ensure that every individual test is not dependent on another test_ method.
84+
85+Test naming convention, purely for code organization purposes:
86 1xx service and endpoint checks
87 2xx relation checks
88 3xx config checks
89 4xx functional checks
90- 9xx restarts and other final checks
91+ 9xx restarts, config changes, actions and other final checks
92
93-In order to run tests, you'll need charm-tools installed (in addition to
94-juju, of course):
95+In order to run tests, charm-tools and juju must be installed:
96 sudo add-apt-repository ppa:juju/stable
97 sudo apt-get update
98- sudo apt-get install charm-tools
99-
100-Common uses of ceilometer relations in bundles:
101- - - ceilometer
102- - keystone:identity-service
103- - [ ceilometer, rabbitmq-server ]
104- - [ ceilometer, mongodb ]
105- - [ ceilometer-agent, nova-compute ]
106- - [ ceilometer-agent, ceilometer ]
107-
108-More detailed relations of ceilometer in a common deployment:
109- relations:
110- amqp:
111- - rabbitmq-server
112- ceilometer-service:
113- - ceilometer-agent
114- cluster:
115- - ceilometer
116- identity-service:
117- - keystone
118- shared-db:
119- - mongodb
120-
121-If you use a web proxy server to access the web, you'll need to set the
122-AMULET_HTTP_PROXY environment variable to the http URL of the proxy server.
123+ sudo apt-get install charm-tools juju juju-deployer amulet
124+
125+Alternatively, tests may be exercised with proposed or development versions
126+of juju and related tools:
127+
128+ # juju proposed version
129+ sudo add-apt-repository ppa:juju/proposed
130+ sudo apt-get update
131+ sudo apt-get install charm-tools juju juju-deployer
132+
133+ # juju development version
134+ sudo add-apt-repository ppa:juju/devel
135+ sudo apt-get update
136+ sudo apt-get install charm-tools juju juju-deployer
137+
138+Some tests may need to download files. If a web proxy server is required in
139+the environment, the AMULET_HTTP_PROXY environment variable must be set and
140+passed into the juju test command. This is unrelated to juju's http proxy
141+settings or behavior.
142
143 The following examples demonstrate different ways that tests can be executed.
144 All examples are run from the charm's root directory.
145
146- * To run all tests (starting with 00-setup):
147-
148- make test
149-
150- * To run a specific test module (or modules):
151-
152- juju test -v -p AMULET_HTTP_PROXY 15-basic-trusty-icehouse
153-
154- * To run a specific test module (or modules), and keep the environment
155- deployed after a failure:
156-
157- juju test --set-e -v -p AMULET_HTTP_PROXY 15-basic-trusty-icehouse
158+ * To run all +x tests in the tests directory:
159+
160+ bzr branch lp:charms/trusty/foo
161+ cd foo
162+ make functional_test
163+
164+ * To run the tests against a specific release combo as defined in tests/:
165+
166+ bzr branch lp:charms/trusty/foo
167+ cd foo
168+ juju test -v -p AMULET_HTTP_PROXY 015-basic-trusty-icehouse
169+
170+ * To run tests and keep the juju environment deployed after a failure:
171+
172+ bzr branch lp:charms/trusty/foo
173+ cd foo
174+ juju test --set-e -v -p AMULET_HTTP_PROXY 015-basic-trusty-icehouse
175
176 * To re-run a test module against an already deployed environment (one
177 that was deployed by a previous call to 'juju test --set-e'):
178
179- ./tests/15-basic-trusty-icehouse
180-
181-For debugging and test development purposes, all code should be idempotent.
182-In other words, the code should have the ability to be re-run without changing
183-the results beyond the initial run. This enables editing and re-running of a
184-test module against an already deployed environment, as described above.
185-
186-Manual debugging tips:
187-
188- * Set the following env vars before using the OpenStack CLI as admin:
189- export OS_AUTH_URL=http://`juju-deployer -f keystone 2>&1 | tail -n 1`:5000/v2.0
190- export OS_TENANT_NAME=admin
191+ ./tests/015-basic-trusty-icehouse
192+
193+ * Even with --set-e, `juju test` will tear down the deployment when all
194+ tests pass. The following work flow may be more effective when
195+ iterating on test writing.
196+
197+ bzr branch lp:charms/trusty/foo
198+ cd foo
199+ ./tests/setup/00-setup
200+ juju bootstrap
201+ ./tests/015-basic-trusty-icehouse
202+ # make some changes, run tests again
203+ ./tests/015-basic-trusty-icehouse
204+ # make some changes, run tests again
205+ ./tests/015-basic-trusty-icehouse
206+
207+ * There may be test definitions in the tests/ dir which are not set +x
208+ executable. This is generally true for deprecated releases, or for
209+ upcoming releases which are not yet validated and enabled. To enable
210+ and run these tests:
211+ bzr branch lp:charms/trusty/foo
212+ cd foo
213+ ls tests
214+ chmod +x tests/017-basic-trusty-kilo
215+ ./tests/setup/00-setup
216+ juju bootstrap
217+ ./tests/017-basic-trusty-kilo
218+
219+
220+Additional notes:
221+
222+ * Use DEBUG to turn on debug logging, use ERROR otherwise.
223+ u = OpenStackAmuletUtils(ERROR)
224+ u = OpenStackAmuletUtils(DEBUG)
225+
226+ * To interact with the deployed environment:
227 export OS_USERNAME=admin
228 export OS_PASSWORD=openstack
229- export OS_REGION_NAME=RegionOne
230-
231- * Set the following env vars before using the OpenStack CLI as demoUser:
232- export OS_AUTH_URL=http://`juju-deployer -f keystone 2>&1 | tail -n 1`:5000/v2.0
233- export OS_TENANT_NAME=demoTenant
234- export OS_USERNAME=demoUser
235- export OS_PASSWORD=password
236- export OS_REGION_NAME=RegionOne
237+ export OS_TENANT_NAME=admin
238+ export OS_REGION_NAME=RegionOne
239+ export OS_AUTH_URL=${OS_AUTH_PROTOCOL:-http}://`juju-deployer -e trusty -f keystone`:5000/v2.0
240+ keystone user-list
241+ glance image-list
242
243=== added directory 'tests/setup'
244=== renamed file 'tests/00-setup' => 'tests/setup/00-setup'
245=== modified file 'tests/tests.yaml'
246--- tests/tests.yaml 2015-10-06 17:24:49 +0000
247+++ tests/tests.yaml 2016-01-08 21:46:24 +0000
248@@ -1,5 +1,5 @@
249 bootstrap: true
250-reset: true
251+reset: false
252 virtualenv: true
253 makefile:
254 - lint

Subscribers

People subscribed via source and target branches