Merge lp:~1chb1n/charms/trusty/ceilometer/next.1601-test-update1 into lp:~openstack-charmers-archive/charms/trusty/ceilometer/next
- Trusty Tahr (14.04)
- next.1601-test-update1
- Merge into next
Status: | Merged |
---|---|
Merged at revision: | 112 |
Proposed branch: | lp:~1chb1n/charms/trusty/ceilometer/next.1601-test-update1 |
Merge into: | lp:~openstack-charmers-archive/charms/trusty/ceilometer/next |
Diff against target: |
254 lines (+118/-76) 6 files modified
Makefile (+1/-0) tests/019-basic-trusty-mitaka (+11/-0) tests/019-basic-vivid-kilo (+0/-9) tests/021-basic-xenial-mitaka (+9/-0) tests/README (+96/-66) tests/tests.yaml (+1/-1) |
To merge this branch: | bzr merge lp:~1chb1n/charms/trusty/ceilometer/next.1601-test-update1 |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
OpenStack Charmers | Pending | ||
Review via email: mp+282053@code.launchpad.net |
Commit message
Description of the change
Cosmetic updates to tests/dir and Makefile to standardize *os-charms, prep for 16.01. No changes to test code, hooks or charmhelpers.
uosci-testing-bot (uosci-testing-bot) wrote : | # |
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_lint_check #16845 ceilometer-next for 1chb1n mp282053
LINT FAIL: lint-test failed
LINT Results (max last 2 lines):
Makefile:15: *** missing separator (did you mean TAB instead of 8 spaces?). Stop.
ERROR:root:Make target returned non-zero.
Full lint test output: http://
Build: http://
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_lint_check #16858 ceilometer-next for 1chb1n mp282053
LINT OK: passed
Build: http://
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_unit_test #15746 ceilometer-next for 1chb1n mp282053
UNIT OK: passed
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_lint_check #16861 ceilometer-next for 1chb1n mp282053
LINT OK: passed
Build: http://
- 112. By Ryan Beisner
-
remove amulet tests for unsupported releases
- 113. By Ryan Beisner
-
Move 00-setup to prevent extra, unnecessary bootstrap in test runs.
- 114. By Ryan Beisner
-
Re-number amulet test file names; add missing combos as not-executable for now.
- 115. By Ryan Beisner
-
Update tests/README
- 116. By Ryan Beisner
-
Update bundletester testplan yaml file
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_lint_check #16900 ceilometer-next for 1chb1n mp282053
LINT OK: passed
Build: http://
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_unit_test #15786 ceilometer-next for 1chb1n mp282053
UNIT OK: passed
Preview Diff
1 | === modified file 'Makefile' |
2 | --- Makefile 2015-10-16 11:59:37 +0000 |
3 | +++ Makefile 2016-01-08 21:46:24 +0000 |
4 | @@ -12,6 +12,7 @@ |
5 | |
6 | functional_test: |
7 | @echo Starting Amulet tests... |
8 | + @tests/setup/00-setup |
9 | @juju test -v -p AMULET_HTTP_PROXY,AMULET_OS_VIP --timeout 2700 |
10 | |
11 | bin/charm_helpers_sync.py: |
12 | |
13 | === renamed file 'tests/020-basic-trusty-liberty' => 'tests/018-basic-trusty-liberty' |
14 | === added file 'tests/019-basic-trusty-mitaka' |
15 | --- tests/019-basic-trusty-mitaka 1970-01-01 00:00:00 +0000 |
16 | +++ tests/019-basic-trusty-mitaka 2016-01-08 21:46:24 +0000 |
17 | @@ -0,0 +1,11 @@ |
18 | +#!/usr/bin/python |
19 | + |
20 | +"""Amulet tests on a basic ceilometer deployment on trusty-liberty.""" |
21 | + |
22 | +from basic_deployment import CeilometerBasicDeployment |
23 | + |
24 | +if __name__ == '__main__': |
25 | + deployment = CeilometerBasicDeployment(series='trusty', |
26 | + openstack='cloud:trusty-liberty', |
27 | + source='cloud:trusty-updates/liberty') |
28 | + deployment.run_tests() |
29 | |
30 | === removed file 'tests/019-basic-vivid-kilo' |
31 | --- tests/019-basic-vivid-kilo 2015-06-12 14:56:01 +0000 |
32 | +++ tests/019-basic-vivid-kilo 1970-01-01 00:00:00 +0000 |
33 | @@ -1,9 +0,0 @@ |
34 | -#!/usr/bin/python |
35 | - |
36 | -"""Amulet tests on a basic ceilometer deployment on vivid-kilo.""" |
37 | - |
38 | -from basic_deployment import CeilometerBasicDeployment |
39 | - |
40 | -if __name__ == '__main__': |
41 | - deployment = CeilometerBasicDeployment(series='vivid') |
42 | - deployment.run_tests() |
43 | |
44 | === renamed file 'tests/021-basic-wily-liberty' => 'tests/020-basic-wily-liberty' |
45 | === added file 'tests/021-basic-xenial-mitaka' |
46 | --- tests/021-basic-xenial-mitaka 1970-01-01 00:00:00 +0000 |
47 | +++ tests/021-basic-xenial-mitaka 2016-01-08 21:46:24 +0000 |
48 | @@ -0,0 +1,9 @@ |
49 | +#!/usr/bin/python |
50 | + |
51 | +"""Amulet tests on a basic ceilometer deployment on xenial-mitaka.""" |
52 | + |
53 | +from basic_deployment import CeilometerBasicDeployment |
54 | + |
55 | +if __name__ == '__main__': |
56 | + deployment = CeilometerBasicDeployment(series='xenial') |
57 | + deployment.run_tests() |
58 | |
59 | === modified file 'tests/README' |
60 | --- tests/README 2015-06-12 14:56:01 +0000 |
61 | +++ tests/README 2016-01-08 21:46:24 +0000 |
62 | @@ -1,83 +1,113 @@ |
63 | -This directory provides Amulet tests that focus on verification of |
64 | -ceilometer deployments. |
65 | - |
66 | -test_* methods are called in lexical sort order. |
67 | - |
68 | -Test name convention to ensure desired test order: |
69 | +This directory provides Amulet tests to verify basic deployment functionality |
70 | +from the perspective of this charm, its requirements and its features, as |
71 | +exercised in a subset of the full OpenStack deployment test bundle topology. |
72 | + |
73 | +Reference: lp:openstack-charm-testing for full test bundles. |
74 | + |
75 | +A single topology and configuration is defined and deployed, once for each of |
76 | +the defined Ubuntu:OpenStack release combos. The ongoing goal is for this |
77 | +charm to always possess tests and combo definitions for all currently-supported |
78 | +release combinations of U:OS. |
79 | + |
80 | +test_* methods are called in lexical sort order, as with most runners. However, |
81 | +each individual test method should be idempotent and expected to pass regardless |
82 | +of run order or Ubuntu:OpenStack combo. When writing or modifying tests, |
83 | +ensure that every individual test is not dependent on another test_ method. |
84 | + |
85 | +Test naming convention, purely for code organization purposes: |
86 | 1xx service and endpoint checks |
87 | 2xx relation checks |
88 | 3xx config checks |
89 | 4xx functional checks |
90 | - 9xx restarts and other final checks |
91 | + 9xx restarts, config changes, actions and other final checks |
92 | |
93 | -In order to run tests, you'll need charm-tools installed (in addition to |
94 | -juju, of course): |
95 | +In order to run tests, charm-tools and juju must be installed: |
96 | sudo add-apt-repository ppa:juju/stable |
97 | sudo apt-get update |
98 | - sudo apt-get install charm-tools |
99 | - |
100 | -Common uses of ceilometer relations in bundles: |
101 | - - - ceilometer |
102 | - - keystone:identity-service |
103 | - - [ ceilometer, rabbitmq-server ] |
104 | - - [ ceilometer, mongodb ] |
105 | - - [ ceilometer-agent, nova-compute ] |
106 | - - [ ceilometer-agent, ceilometer ] |
107 | - |
108 | -More detailed relations of ceilometer in a common deployment: |
109 | - relations: |
110 | - amqp: |
111 | - - rabbitmq-server |
112 | - ceilometer-service: |
113 | - - ceilometer-agent |
114 | - cluster: |
115 | - - ceilometer |
116 | - identity-service: |
117 | - - keystone |
118 | - shared-db: |
119 | - - mongodb |
120 | - |
121 | -If you use a web proxy server to access the web, you'll need to set the |
122 | -AMULET_HTTP_PROXY environment variable to the http URL of the proxy server. |
123 | + sudo apt-get install charm-tools juju juju-deployer amulet |
124 | + |
125 | +Alternatively, tests may be exercised with proposed or development versions |
126 | +of juju and related tools: |
127 | + |
128 | + # juju proposed version |
129 | + sudo add-apt-repository ppa:juju/proposed |
130 | + sudo apt-get update |
131 | + sudo apt-get install charm-tools juju juju-deployer |
132 | + |
133 | + # juju development version |
134 | + sudo add-apt-repository ppa:juju/devel |
135 | + sudo apt-get update |
136 | + sudo apt-get install charm-tools juju juju-deployer |
137 | + |
138 | +Some tests may need to download files. If a web proxy server is required in |
139 | +the environment, the AMULET_HTTP_PROXY environment variable must be set and |
140 | +passed into the juju test command. This is unrelated to juju's http proxy |
141 | +settings or behavior. |
142 | |
143 | The following examples demonstrate different ways that tests can be executed. |
144 | All examples are run from the charm's root directory. |
145 | |
146 | - * To run all tests (starting with 00-setup): |
147 | - |
148 | - make test |
149 | - |
150 | - * To run a specific test module (or modules): |
151 | - |
152 | - juju test -v -p AMULET_HTTP_PROXY 15-basic-trusty-icehouse |
153 | - |
154 | - * To run a specific test module (or modules), and keep the environment |
155 | - deployed after a failure: |
156 | - |
157 | - juju test --set-e -v -p AMULET_HTTP_PROXY 15-basic-trusty-icehouse |
158 | + * To run all +x tests in the tests directory: |
159 | + |
160 | + bzr branch lp:charms/trusty/foo |
161 | + cd foo |
162 | + make functional_test |
163 | + |
164 | + * To run the tests against a specific release combo as defined in tests/: |
165 | + |
166 | + bzr branch lp:charms/trusty/foo |
167 | + cd foo |
168 | + juju test -v -p AMULET_HTTP_PROXY 015-basic-trusty-icehouse |
169 | + |
170 | + * To run tests and keep the juju environment deployed after a failure: |
171 | + |
172 | + bzr branch lp:charms/trusty/foo |
173 | + cd foo |
174 | + juju test --set-e -v -p AMULET_HTTP_PROXY 015-basic-trusty-icehouse |
175 | |
176 | * To re-run a test module against an already deployed environment (one |
177 | that was deployed by a previous call to 'juju test --set-e'): |
178 | |
179 | - ./tests/15-basic-trusty-icehouse |
180 | - |
181 | -For debugging and test development purposes, all code should be idempotent. |
182 | -In other words, the code should have the ability to be re-run without changing |
183 | -the results beyond the initial run. This enables editing and re-running of a |
184 | -test module against an already deployed environment, as described above. |
185 | - |
186 | -Manual debugging tips: |
187 | - |
188 | - * Set the following env vars before using the OpenStack CLI as admin: |
189 | - export OS_AUTH_URL=http://`juju-deployer -f keystone 2>&1 | tail -n 1`:5000/v2.0 |
190 | - export OS_TENANT_NAME=admin |
191 | + ./tests/015-basic-trusty-icehouse |
192 | + |
193 | + * Even with --set-e, `juju test` will tear down the deployment when all |
194 | + tests pass. The following work flow may be more effective when |
195 | + iterating on test writing. |
196 | + |
197 | + bzr branch lp:charms/trusty/foo |
198 | + cd foo |
199 | + ./tests/setup/00-setup |
200 | + juju bootstrap |
201 | + ./tests/015-basic-trusty-icehouse |
202 | + # make some changes, run tests again |
203 | + ./tests/015-basic-trusty-icehouse |
204 | + # make some changes, run tests again |
205 | + ./tests/015-basic-trusty-icehouse |
206 | + |
207 | + * There may be test definitions in the tests/ dir which are not set +x |
208 | + executable. This is generally true for deprecated releases, or for |
209 | + upcoming releases which are not yet validated and enabled. To enable |
210 | + and run these tests: |
211 | + bzr branch lp:charms/trusty/foo |
212 | + cd foo |
213 | + ls tests |
214 | + chmod +x tests/017-basic-trusty-kilo |
215 | + ./tests/setup/00-setup |
216 | + juju bootstrap |
217 | + ./tests/017-basic-trusty-kilo |
218 | + |
219 | + |
220 | +Additional notes: |
221 | + |
222 | + * Use DEBUG to turn on debug logging, use ERROR otherwise. |
223 | + u = OpenStackAmuletUtils(ERROR) |
224 | + u = OpenStackAmuletUtils(DEBUG) |
225 | + |
226 | + * To interact with the deployed environment: |
227 | export OS_USERNAME=admin |
228 | export OS_PASSWORD=openstack |
229 | - export OS_REGION_NAME=RegionOne |
230 | - |
231 | - * Set the following env vars before using the OpenStack CLI as demoUser: |
232 | - export OS_AUTH_URL=http://`juju-deployer -f keystone 2>&1 | tail -n 1`:5000/v2.0 |
233 | - export OS_TENANT_NAME=demoTenant |
234 | - export OS_USERNAME=demoUser |
235 | - export OS_PASSWORD=password |
236 | - export OS_REGION_NAME=RegionOne |
237 | + export OS_TENANT_NAME=admin |
238 | + export OS_REGION_NAME=RegionOne |
239 | + export OS_AUTH_URL=${OS_AUTH_PROTOCOL:-http}://`juju-deployer -e trusty -f keystone`:5000/v2.0 |
240 | + keystone user-list |
241 | + glance image-list |
242 | |
243 | === added directory 'tests/setup' |
244 | === renamed file 'tests/00-setup' => 'tests/setup/00-setup' |
245 | === modified file 'tests/tests.yaml' |
246 | --- tests/tests.yaml 2015-10-06 17:24:49 +0000 |
247 | +++ tests/tests.yaml 2016-01-08 21:46:24 +0000 |
248 | @@ -1,5 +1,5 @@ |
249 | bootstrap: true |
250 | -reset: true |
251 | +reset: false |
252 | virtualenv: true |
253 | makefile: |
254 | - lint |
charm_unit_test #15731 ceilometer-next for 1chb1n mp282053
UNIT FAIL: unit-test failed
UNIT Results (max last 2 lines):
Makefile:15: *** missing separator (did you mean TAB instead of 8 spaces?). Stop.
ERROR:root:Make target returned non-zero.
Full unit test output: http:// paste.ubuntu. com/14440496/ 10.245. 162.77: 8080/job/ charm_unit_ test/15731/
Build: http://