Merge lp:~1chb1n/charms/trusty/nova-cloud-controller/next.1601-test-update1 into lp:~openstack-charmers-archive/charms/trusty/nova-cloud-controller/next

Proposed by Ryan Beisner
Status: Merged
Merged at revision: 211
Proposed branch: lp:~1chb1n/charms/trusty/nova-cloud-controller/next.1601-test-update1
Merge into: lp:~openstack-charmers-archive/charms/trusty/nova-cloud-controller/next
Diff against target: 245 lines (+121/-57)
6 files modified
Makefile (+1/-0)
tests/019-basic-trusty-mitaka (+12/-0)
tests/019-basic-vivid-kilo (+0/-10)
tests/021-basic-xenial-mitaka (+10/-0)
tests/README (+96/-46)
tests/tests.yaml (+2/-1)
To merge this branch: bzr merge lp:~1chb1n/charms/trusty/nova-cloud-controller/next.1601-test-update1
Reviewer Review Type Date Requested Status
Liam Young (community) Approve
Review via email: mp+282072@code.launchpad.net

Description of the change

Cosmetic updates to tests/dir and Makefile to standardize *os-charms, prep for 16.01. No changes to test code, hooks or charmhelpers.

To post a comment you must log in.
Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #16876 nova-cloud-controller-next for 1chb1n mp282072
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/16876/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_unit_test #15766 nova-cloud-controller-next for 1chb1n mp282072
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/15766/

209. By Ryan Beisner

remove amulet tests for unsupported releases

210. By Ryan Beisner

Move 00-setup to prevent extra, unnecessary bootstrap in test runs.

211. By Ryan Beisner

Re-number amulet test file names; add missing combos as not-executable for now.

212. By Ryan Beisner

Update tests/README

213. By Ryan Beisner

Update bundletester testplan yaml file

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_unit_test #15775 nova-cloud-controller-next for 1chb1n mp282072
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/15775/

Revision history for this message
Liam Young (gnuoy) wrote :

Approved

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'Makefile'
2--- Makefile 2015-10-06 15:04:00 +0000
3+++ Makefile 2016-01-08 21:47:29 +0000
4@@ -13,6 +13,7 @@
5
6 functional_test:
7 @echo Starting Amulet tests...
8+ @tests/setup/00-setup
9 @juju test -v -p AMULET_HTTP_PROXY,AMULET_OS_VIP --timeout 2700
10
11 bin/charm_helpers_sync.py:
12
13=== renamed file 'tests/020-basic-trusty-liberty' => 'tests/018-basic-trusty-liberty'
14=== added file 'tests/019-basic-trusty-mitaka'
15--- tests/019-basic-trusty-mitaka 1970-01-01 00:00:00 +0000
16+++ tests/019-basic-trusty-mitaka 2016-01-08 21:47:29 +0000
17@@ -0,0 +1,12 @@
18+#!/usr/bin/python
19+
20+"""Amulet tests on a basic nova cloud controller deployment on
21+ trusty-liberty."""
22+
23+from basic_deployment import NovaCCBasicDeployment
24+
25+if __name__ == '__main__':
26+ deployment = NovaCCBasicDeployment(series='trusty',
27+ openstack='cloud:trusty-liberty',
28+ source='cloud:trusty-updates/liberty')
29+ deployment.run_tests()
30
31=== removed file 'tests/019-basic-vivid-kilo'
32--- tests/019-basic-vivid-kilo 2015-10-07 18:03:38 +0000
33+++ tests/019-basic-vivid-kilo 1970-01-01 00:00:00 +0000
34@@ -1,10 +0,0 @@
35-#!/usr/bin/python
36-
37-"""Amulet tests on a basic nova cloud controller deployment on
38- vivid-kilo."""
39-
40-from basic_deployment import NovaCCBasicDeployment
41-
42-if __name__ == '__main__':
43- deployment = NovaCCBasicDeployment(series='vivid')
44- deployment.run_tests()
45
46=== renamed file 'tests/021-basic-wily-liberty' => 'tests/020-basic-wily-liberty'
47=== added file 'tests/021-basic-xenial-mitaka'
48--- tests/021-basic-xenial-mitaka 1970-01-01 00:00:00 +0000
49+++ tests/021-basic-xenial-mitaka 2016-01-08 21:47:29 +0000
50@@ -0,0 +1,10 @@
51+#!/usr/bin/python
52+
53+"""Amulet tests on a basic nova cloud controller deployment on
54+ xenial-mitaka."""
55+
56+from basic_deployment import NovaCCBasicDeployment
57+
58+if __name__ == '__main__':
59+ deployment = NovaCCBasicDeployment(series='xenial')
60+ deployment.run_tests()
61
62=== modified file 'tests/README'
63--- tests/README 2015-10-07 18:03:38 +0000
64+++ tests/README 2016-01-08 21:47:29 +0000
65@@ -1,63 +1,113 @@
66-This directory provides Amulet tests that focus on verification of Nova Cloud
67-Controller deployments.
68-
69-test_* methods are called in lexical sort order, although each individual test
70-should be idempotent, and expected to pass regardless of run order.
71-
72-Test name convention to ensure desired test order:
73+This directory provides Amulet tests to verify basic deployment functionality
74+from the perspective of this charm, its requirements and its features, as
75+exercised in a subset of the full OpenStack deployment test bundle topology.
76+
77+Reference: lp:openstack-charm-testing for full test bundles.
78+
79+A single topology and configuration is defined and deployed, once for each of
80+the defined Ubuntu:OpenStack release combos. The ongoing goal is for this
81+charm to always possess tests and combo definitions for all currently-supported
82+release combinations of U:OS.
83+
84+test_* methods are called in lexical sort order, as with most runners. However,
85+each individual test method should be idempotent and expected to pass regardless
86+of run order or Ubuntu:OpenStack combo. When writing or modifying tests,
87+ensure that every individual test is not dependent on another test_ method.
88+
89+Test naming convention, purely for code organization purposes:
90 1xx service and endpoint checks
91 2xx relation checks
92 3xx config checks
93 4xx functional checks
94- 9xx restarts and other final checks
95+ 9xx restarts, config changes, actions and other final checks
96
97-In order to run tests, you'll need charm-tools installed (in addition to
98-juju, of course):
99+In order to run tests, charm-tools and juju must be installed:
100 sudo add-apt-repository ppa:juju/stable
101 sudo apt-get update
102- sudo apt-get install charm-tools
103-
104-If you use a web proxy server to access the web, you'll need to set the
105-AMULET_HTTP_PROXY environment variable to the http URL of the proxy server.
106+ sudo apt-get install charm-tools juju juju-deployer amulet
107+
108+Alternatively, tests may be exercised with proposed or development versions
109+of juju and related tools:
110+
111+ # juju proposed version
112+ sudo add-apt-repository ppa:juju/proposed
113+ sudo apt-get update
114+ sudo apt-get install charm-tools juju juju-deployer
115+
116+ # juju development version
117+ sudo add-apt-repository ppa:juju/devel
118+ sudo apt-get update
119+ sudo apt-get install charm-tools juju juju-deployer
120+
121+Some tests may need to download files. If a web proxy server is required in
122+the environment, the AMULET_HTTP_PROXY environment variable must be set and
123+passed into the juju test command. This is unrelated to juju's http proxy
124+settings or behavior.
125
126 The following examples demonstrate different ways that tests can be executed.
127 All examples are run from the charm's root directory.
128
129- * To run all tests (starting with 00-setup):
130-
131- make test
132-
133- * To run a specific test module (or modules):
134-
135- juju test -v -p AMULET_HTTP_PROXY 15-basic-trusty-icehouse
136-
137- * To run a specific test module (or modules), and keep the environment
138- deployed after a failure:
139-
140- juju test --set-e -v -p AMULET_HTTP_PROXY 15-basic-trusty-icehouse
141+ * To run all +x tests in the tests directory:
142+
143+ bzr branch lp:charms/trusty/foo
144+ cd foo
145+ make functional_test
146+
147+ * To run the tests against a specific release combo as defined in tests/:
148+
149+ bzr branch lp:charms/trusty/foo
150+ cd foo
151+ juju test -v -p AMULET_HTTP_PROXY 015-basic-trusty-icehouse
152+
153+ * To run tests and keep the juju environment deployed after a failure:
154+
155+ bzr branch lp:charms/trusty/foo
156+ cd foo
157+ juju test --set-e -v -p AMULET_HTTP_PROXY 015-basic-trusty-icehouse
158
159 * To re-run a test module against an already deployed environment (one
160 that was deployed by a previous call to 'juju test --set-e'):
161
162- ./tests/15-basic-trusty-icehouse
163-
164-For debugging and test development purposes, all code should be idempotent.
165-In other words, the code should have the ability to be re-run without changing
166-the results beyond the initial run. This enables editing and re-running of a
167-test module against an already deployed environment, as described above.
168-
169-Manual debugging tips:
170-
171- * Set the following env vars before using the OpenStack CLI as admin:
172- export OS_AUTH_URL=http://`juju-deployer -f keystone 2>&1 | tail -n 1`:5000/v2.0
173- export OS_TENANT_NAME=admin
174+ ./tests/015-basic-trusty-icehouse
175+
176+ * Even with --set-e, `juju test` will tear down the deployment when all
177+ tests pass. The following work flow may be more effective when
178+ iterating on test writing.
179+
180+ bzr branch lp:charms/trusty/foo
181+ cd foo
182+ ./tests/setup/00-setup
183+ juju bootstrap
184+ ./tests/015-basic-trusty-icehouse
185+ # make some changes, run tests again
186+ ./tests/015-basic-trusty-icehouse
187+ # make some changes, run tests again
188+ ./tests/015-basic-trusty-icehouse
189+
190+ * There may be test definitions in the tests/ dir which are not set +x
191+ executable. This is generally true for deprecated releases, or for
192+ upcoming releases which are not yet validated and enabled. To enable
193+ and run these tests:
194+ bzr branch lp:charms/trusty/foo
195+ cd foo
196+ ls tests
197+ chmod +x tests/017-basic-trusty-kilo
198+ ./tests/setup/00-setup
199+ juju bootstrap
200+ ./tests/017-basic-trusty-kilo
201+
202+
203+Additional notes:
204+
205+ * Use DEBUG to turn on debug logging, use ERROR otherwise.
206+ u = OpenStackAmuletUtils(ERROR)
207+ u = OpenStackAmuletUtils(DEBUG)
208+
209+ * To interact with the deployed environment:
210 export OS_USERNAME=admin
211 export OS_PASSWORD=openstack
212- export OS_REGION_NAME=RegionOne
213-
214- * Set the following env vars before using the OpenStack CLI as demoUser:
215- export OS_AUTH_URL=http://`juju-deployer -f keystone 2>&1 | tail -n 1`:5000/v2.0
216- export OS_TENANT_NAME=demoTenant
217- export OS_USERNAME=demoUser
218- export OS_PASSWORD=password
219- export OS_REGION_NAME=RegionOne
220+ export OS_TENANT_NAME=admin
221+ export OS_REGION_NAME=RegionOne
222+ export OS_AUTH_URL=${OS_AUTH_PROTOCOL:-http}://`juju-deployer -e trusty -f keystone`:5000/v2.0
223+ keystone user-list
224+ glance image-list
225
226=== added directory 'tests/setup'
227=== renamed file 'tests/00-setup' => 'tests/setup/00-setup'
228=== modified file 'tests/tests.yaml'
229--- tests/tests.yaml 2015-10-06 14:11:50 +0000
230+++ tests/tests.yaml 2016-01-08 21:47:29 +0000
231@@ -1,5 +1,5 @@
232 bootstrap: true
233-reset: true
234+reset: false
235 virtualenv: true
236 makefile:
237 - lint
238@@ -9,6 +9,7 @@
239 packages:
240 - amulet
241 - distro-info-data
242+ - python-ceilometerclient
243 - python-cinderclient
244 - python-distro-info
245 - python-glanceclient

Subscribers

People subscribed via source and target branches