Merge lp:~1chb1n/charms/trusty/cinder/next.1601-test-update1 into lp:~openstack-charmers-archive/charms/trusty/cinder/next

Proposed by Ryan Beisner on 2016-01-08
Status: Merged
Merged at revision: 142
Proposed branch: lp:~1chb1n/charms/trusty/cinder/next.1601-test-update1
Merge into: lp:~openstack-charmers-archive/charms/trusty/cinder/next
Diff against target: 300 lines (+129/-81)
9 files modified
Makefile (+1/-0)
tests/018-basic-trusty-liberty (+11/-0)
tests/019-basic-trusty-mitaka (+11/-0)
tests/019-basic-vivid-kilo (+0/-9)
tests/020-basic-wily-liberty (+9/-0)
tests/021-basic-xenial-mitaka (+9/-0)
tests/054-basic-vivid-kilo-git (+0/-9)
tests/README (+86/-62)
tests/tests.yaml (+2/-1)
To merge this branch: bzr merge lp:~1chb1n/charms/trusty/cinder/next.1601-test-update1
Reviewer Review Type Date Requested Status
Liam Young 2016-01-08 Approve on 2016-01-12
Review via email: mp+282068@code.launchpad.net

Description of the change

Cosmetic updates to tests/dir and Makefile to standardize *os-charms, prep for 16.01. No changes to test code, hooks or charmhelpers.

To post a comment you must log in.

charm_lint_check #16877 cinder-next for 1chb1n mp282068
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/16877/

charm_unit_test #15767 cinder-next for 1chb1n mp282068
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/15767/

141. By Ryan Beisner on 2016-01-08

remove amulet tests for unsupported releases

142. By Ryan Beisner on 2016-01-08

Move 00-setup to prevent extra, unnecessary bootstrap in test runs.

143. By Ryan Beisner on 2016-01-08

Re-number amulet test file names; add missing combos as not-executable for now.

144. By Ryan Beisner on 2016-01-08

Update tests/README

145. By Ryan Beisner on 2016-01-08

Update bundletester testplan yaml file

charm_lint_check #16889 cinder-next for 1chb1n mp282068
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/16889/

charm_unit_test #15780 cinder-next for 1chb1n mp282068
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/15780/

Liam Young (gnuoy) wrote :

Approved

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'Makefile'
2--- Makefile 2015-10-29 04:46:01 +0000
3+++ Makefile 2016-01-08 21:46:50 +0000
4@@ -13,6 +13,7 @@
5
6 functional_test:
7 @echo Starting amulet deployment tests...
8+ @tests/setup/00-setup
9 @juju test -v -p AMULET_HTTP_PROXY,AMULET_OS_VIP --timeout 2700
10
11 bin/charm_helpers_sync.py:
12
13=== added file 'tests/018-basic-trusty-liberty'
14--- tests/018-basic-trusty-liberty 1970-01-01 00:00:00 +0000
15+++ tests/018-basic-trusty-liberty 2016-01-08 21:46:50 +0000
16@@ -0,0 +1,11 @@
17+#!/usr/bin/python
18+
19+"""Amulet tests on a basic cinder deployment on trusty-liberty."""
20+
21+from basic_deployment import CinderBasicDeployment
22+
23+if __name__ == '__main__':
24+ deployment = CinderBasicDeployment(series='trusty',
25+ openstack='cloud:trusty-liberty',
26+ source='cloud:trusty-updates/liberty')
27+ deployment.run_tests()
28
29=== added file 'tests/019-basic-trusty-mitaka'
30--- tests/019-basic-trusty-mitaka 1970-01-01 00:00:00 +0000
31+++ tests/019-basic-trusty-mitaka 2016-01-08 21:46:50 +0000
32@@ -0,0 +1,11 @@
33+#!/usr/bin/python
34+
35+"""Amulet tests on a basic cinder deployment on trusty-liberty."""
36+
37+from basic_deployment import CinderBasicDeployment
38+
39+if __name__ == '__main__':
40+ deployment = CinderBasicDeployment(series='trusty',
41+ openstack='cloud:trusty-liberty',
42+ source='cloud:trusty-updates/liberty')
43+ deployment.run_tests()
44
45=== removed file 'tests/019-basic-vivid-kilo'
46--- tests/019-basic-vivid-kilo 2015-06-23 23:52:07 +0000
47+++ tests/019-basic-vivid-kilo 1970-01-01 00:00:00 +0000
48@@ -1,9 +0,0 @@
49-#!/usr/bin/python
50-
51-"""Amulet tests on a basic Cinder deployment on vivid-kilo."""
52-
53-from basic_deployment import CinderBasicDeployment
54-
55-if __name__ == '__main__':
56- deployment = CinderBasicDeployment(series='vivid')
57- deployment.run_tests()
58
59=== added file 'tests/020-basic-wily-liberty'
60--- tests/020-basic-wily-liberty 1970-01-01 00:00:00 +0000
61+++ tests/020-basic-wily-liberty 2016-01-08 21:46:50 +0000
62@@ -0,0 +1,9 @@
63+#!/usr/bin/python
64+
65+"""Amulet tests on a basic Cinder deployment on wily-liberty."""
66+
67+from basic_deployment import CinderBasicDeployment
68+
69+if __name__ == '__main__':
70+ deployment = CinderBasicDeployment(series='wily')
71+ deployment.run_tests()
72
73=== added file 'tests/021-basic-xenial-mitaka'
74--- tests/021-basic-xenial-mitaka 1970-01-01 00:00:00 +0000
75+++ tests/021-basic-xenial-mitaka 2016-01-08 21:46:50 +0000
76@@ -0,0 +1,9 @@
77+#!/usr/bin/python
78+
79+"""Amulet tests on a basic Cinder deployment on xenial-mitaka."""
80+
81+from basic_deployment import CinderBasicDeployment
82+
83+if __name__ == '__main__':
84+ deployment = CinderBasicDeployment(series='xenial')
85+ deployment.run_tests()
86
87=== removed file 'tests/054-basic-vivid-kilo-git'
88--- tests/054-basic-vivid-kilo-git 2015-06-23 23:52:07 +0000
89+++ tests/054-basic-vivid-kilo-git 1970-01-01 00:00:00 +0000
90@@ -1,9 +0,0 @@
91-#!/usr/bin/python
92-
93-"""Amulet tests on a basic Cinder git deployment on vivid-kilo."""
94-
95-from basic_deployment import CinderBasicDeployment
96-
97-if __name__ == '__main__':
98- deployment = CinderBasicDeployment(series='vivid', git=True)
99- deployment.run_tests()
100
101=== modified file 'tests/README'
102--- tests/README 2015-06-23 23:52:07 +0000
103+++ tests/README 2016-01-08 21:46:50 +0000
104@@ -1,89 +1,113 @@
105-This directory provides Amulet tests that focus on verification of Cinder
106-deployments.
107-
108-test_* methods are called in lexical sort order.
109-
110-Test name convention to ensure desired test order:
111+This directory provides Amulet tests to verify basic deployment functionality
112+from the perspective of this charm, its requirements and its features, as
113+exercised in a subset of the full OpenStack deployment test bundle topology.
114+
115+Reference: lp:openstack-charm-testing for full test bundles.
116+
117+A single topology and configuration is defined and deployed, once for each of
118+the defined Ubuntu:OpenStack release combos. The ongoing goal is for this
119+charm to always possess tests and combo definitions for all currently-supported
120+release combinations of U:OS.
121+
122+test_* methods are called in lexical sort order, as with most runners. However,
123+each individual test method should be idempotent and expected to pass regardless
124+of run order or Ubuntu:OpenStack combo. When writing or modifying tests,
125+ensure that every individual test is not dependent on another test_ method.
126+
127+Test naming convention, purely for code organization purposes:
128 1xx service and endpoint checks
129 2xx relation checks
130 3xx config checks
131 4xx functional checks
132- 9xx restarts and other final checks
133+ 9xx restarts, config changes, actions and other final checks
134
135-In order to run tests, you'll need charm-tools installed (in addition to
136-juju, of course):
137+In order to run tests, charm-tools and juju must be installed:
138 sudo add-apt-repository ppa:juju/stable
139 sudo apt-get update
140- sudo apt-get install charm-tools
141-
142-If you use a web proxy server to access the web, you'll need to set the
143-AMULET_HTTP_PROXY environment variable to the http URL of the proxy server.
144+ sudo apt-get install charm-tools juju juju-deployer amulet
145+
146+Alternatively, tests may be exercised with proposed or development versions
147+of juju and related tools:
148+
149+ # juju proposed version
150+ sudo add-apt-repository ppa:juju/proposed
151+ sudo apt-get update
152+ sudo apt-get install charm-tools juju juju-deployer
153+
154+ # juju development version
155+ sudo add-apt-repository ppa:juju/devel
156+ sudo apt-get update
157+ sudo apt-get install charm-tools juju juju-deployer
158+
159+Some tests may need to download files. If a web proxy server is required in
160+the environment, the AMULET_HTTP_PROXY environment variable must be set and
161+passed into the juju test command. This is unrelated to juju's http proxy
162+settings or behavior.
163
164 The following examples demonstrate different ways that tests can be executed.
165 All examples are run from the charm's root directory.
166
167- * To run all tests (starting with 00-setup):
168-
169- make test
170-
171- * To run a specific test module (or modules):
172-
173- juju test -v -p AMULET_HTTP_PROXY 15-basic-trusty-icehouse
174-
175- * To run a specific test module (or modules), and keep the environment
176- deployed after a failure:
177-
178- juju test --set-e -v -p AMULET_HTTP_PROXY 15-basic-trusty-icehouse
179+ * To run all +x tests in the tests directory:
180+
181+ bzr branch lp:charms/trusty/foo
182+ cd foo
183+ make functional_test
184+
185+ * To run the tests against a specific release combo as defined in tests/:
186+
187+ bzr branch lp:charms/trusty/foo
188+ cd foo
189+ juju test -v -p AMULET_HTTP_PROXY 015-basic-trusty-icehouse
190+
191+ * To run tests and keep the juju environment deployed after a failure:
192+
193+ bzr branch lp:charms/trusty/foo
194+ cd foo
195+ juju test --set-e -v -p AMULET_HTTP_PROXY 015-basic-trusty-icehouse
196
197 * To re-run a test module against an already deployed environment (one
198 that was deployed by a previous call to 'juju test --set-e'):
199
200- ./tests/15-basic-trusty-icehouse
201-
202-
203-For debugging and test development purposes, all code should be idempotent.
204-In other words, the code should have the ability to be re-run without changing
205-the results beyond the initial run. This enables editing and re-running of a
206-test module against an already deployed environment, as described above.
207-
208-Additional testing and debug information:
209+ ./tests/015-basic-trusty-icehouse
210+
211+ * Even with --set-e, `juju test` will tear down the deployment when all
212+ tests pass. The following work flow may be more effective when
213+ iterating on test writing.
214+
215+ bzr branch lp:charms/trusty/foo
216+ cd foo
217+ ./tests/setup/00-setup
218+ juju bootstrap
219+ ./tests/015-basic-trusty-icehouse
220+ # make some changes, run tests again
221+ ./tests/015-basic-trusty-icehouse
222+ # make some changes, run tests again
223+ ./tests/015-basic-trusty-icehouse
224+
225+ * There may be test definitions in the tests/ dir which are not set +x
226+ executable. This is generally true for deprecated releases, or for
227+ upcoming releases which are not yet validated and enabled. To enable
228+ and run these tests:
229+ bzr branch lp:charms/trusty/foo
230+ cd foo
231+ ls tests
232+ chmod +x tests/017-basic-trusty-kilo
233+ ./tests/setup/00-setup
234+ juju bootstrap
235+ ./tests/017-basic-trusty-kilo
236+
237+
238+Additional notes:
239
240 * Use DEBUG to turn on debug logging, use ERROR otherwise.
241-
242 u = OpenStackAmuletUtils(ERROR)
243 u = OpenStackAmuletUtils(DEBUG)
244
245 * To interact with the deployed environment:
246-
247 export OS_USERNAME=admin
248 export OS_PASSWORD=openstack
249 export OS_TENANT_NAME=admin
250 export OS_REGION_NAME=RegionOne
251 export OS_AUTH_URL=${OS_AUTH_PROTOCOL:-http}://`juju-deployer -e trusty -f keystone`:5000/v2.0
252 keystone user-list
253- cinder list
254-
255- * Handy commands for watching these tests run or manually poking around:
256-
257- watch "cinder list && cinder snapshot-list && glance image-list"
258- watch "juju get cinder | egrep 'verbose|debug|true|false'"
259- watch "juju ssh cinder/0 'sudo lvs && sudo status cinder-api && sudo status cinder-volume'"
260- juju set cinder verbose=false && juju set cinder debug=false
261-
262- * Be aware that method names that start with 'test_' are called in alpha/num
263- order. Test methods which are order-sensitive can be numbered to ensure
264- that they run in the expected order. ex:
265-
266- test_000_earlytest
267- test_300_sometest
268- test_999_latertest
269-
270- * Preserving the deployed environment:
271- Even with juju --set-e, amulet will tear down the juju environment
272- when all tests pass. This force_fail 'test' can be used in basic_deployment.py
273- to simulate a failed test and keep the environment.
274-
275- def test_zzzz_fake_fail(self):
276- '''Force a fake fail to keep juju environment after a successful test run'''
277- # Useful in test writing, when used with: juju test --set-e
278- amulet.raise_status(amulet.FAIL, msg='using fake fail to keep juju environment')
279+ glance image-list
280
281=== added directory 'tests/setup'
282=== renamed file 'tests/00-setup' => 'tests/setup/00-setup'
283=== modified file 'tests/tests.yaml'
284--- tests/tests.yaml 2015-10-06 14:11:42 +0000
285+++ tests/tests.yaml 2016-01-08 21:46:50 +0000
286@@ -1,5 +1,5 @@
287 bootstrap: true
288-reset: true
289+reset: false
290 virtualenv: true
291 makefile:
292 - lint
293@@ -9,6 +9,7 @@
294 packages:
295 - amulet
296 - distro-info-data
297+ - python-ceilometerclient
298 - python-cinderclient
299 - python-distro-info
300 - python-glanceclient

Subscribers

People subscribed via source and target branches