Merge lp:~1chb1n/charms/trusty/mysql/amulet-initial-enable into lp:charms/trusty/mysql
- Trusty Tahr (14.04)
- amulet-initial-enable
- Merge into trunk
Status: | Merged | ||||
---|---|---|---|---|---|
Merged at revision: | 145 | ||||
Proposed branch: | lp:~1chb1n/charms/trusty/mysql/amulet-initial-enable | ||||
Merge into: | lp:charms/trusty/mysql | ||||
Diff against target: |
1372 lines (+1209/-13) 22 files modified
Makefile (+6/-1) charm-helpers-tests.yaml (+6/-0) config.yaml (+1/-1) metadata.yaml (+1/-1) tests/00-setup (+18/-9) tests/010-configs (+5/-1) tests/014-basic-precise-icehouse (+11/-0) tests/015-basic-trusty-icehouse (+9/-0) tests/016-basic-trusty-juno (+11/-0) tests/017-basic-trusty-kilo (+11/-0) tests/018-basic-utopic-juno (+9/-0) tests/019-basic-vivid-kilo (+9/-0) tests/basic_deployment.py (+158/-0) tests/charmhelpers/__init__.py (+38/-0) tests/charmhelpers/contrib/__init__.py (+15/-0) tests/charmhelpers/contrib/amulet/__init__.py (+15/-0) tests/charmhelpers/contrib/amulet/deployment.py (+93/-0) tests/charmhelpers/contrib/amulet/utils.py (+323/-0) tests/charmhelpers/contrib/openstack/__init__.py (+15/-0) tests/charmhelpers/contrib/openstack/amulet/__init__.py (+15/-0) tests/charmhelpers/contrib/openstack/amulet/deployment.py (+146/-0) tests/charmhelpers/contrib/openstack/amulet/utils.py (+294/-0) |
||||
To merge this branch: | bzr merge lp:~1chb1n/charms/trusty/mysql/amulet-initial-enable | ||||
Related bugs: |
|
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
charmers | Pending | ||
Review via email: mp+256941@code.launchpad.net |
Commit message
Description of the change
Add basic functional test coverage for mysql on supported releases. These initial amulet tests exercise mysql services, config files, and relations with keystone.
This MP is dependent on a corresponding charm-helpers MP @ https:/
# Summary:
./
charm-helpers-
Makefile (added sync and functional_test targets)
tests/
00-setup (updated to work with all)
010-configs (renamed, fixed import issues)
014-basic-
015-basic-
016-basic-
017-basic-
018-basic-
019-basic-
basic_deployment.py (added, shared test definitions)
tests/charm-helpers (added and sync'd)
# Other housekeeping:
Addressed this: I: Categories are being deprecated in favor of tags. Please rename the "categories" field to "tags".
uosci-testing-bot (uosci-testing-bot) wrote : | # |
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_lint_check #3686 mysql for 1chb1n mp256941
LINT OK: passed
Ryan Beisner (1chb1n) wrote : | # |
FYI, unit test fail is:
https:/
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_amulet_test #3481 mysql for 1chb1n mp256941
AMULET FAIL: amulet-test failed
AMULET Results (max last 2 lines):
make: *** [functional_test] Error 124
ERROR:root:Make target returned non-zero.
Full amulet test output: http://
Build: http://
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_amulet_test #3486 mysql for 1chb1n mp256941
AMULET OK: passed
Build: http://
Corey Bryant (corey.bryant) wrote : | # |
Looks good! Just a few minor comments.
Ryan Beisner (1chb1n) wrote : | # |
Thanks! See inline.
- 156. By Ryan Beisner
-
cleanup
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_unit_test #3520 mysql for 1chb1n mp256941
UNIT FAIL: unit-test failed
UNIT Results (max last 2 lines):
make: *** [test] Error 1
ERROR:root:Make target returned non-zero.
Full unit test output: http://
Build: http://
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_lint_check #3733 mysql for 1chb1n mp256941
LINT OK: passed
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_amulet_test #3522 mysql for 1chb1n mp256941
AMULET OK: passed
Build: http://
Ryan Beisner (1chb1n) wrote : | # |
FYI, Amulet full results: http://
Preview Diff
1 | === modified file 'Makefile' |
2 | --- Makefile 2015-02-27 17:13:09 +0000 |
3 | +++ Makefile 2015-04-21 22:30:08 +0000 |
4 | @@ -7,7 +7,7 @@ |
5 | .venv/bin/pip install flake8 nose mock six |
6 | |
7 | lint: virtualenv |
8 | - .venv/bin/flake8 --exclude hooks/charmhelpers hooks |
9 | + .venv/bin/flake8 --exclude hooks/charmhelpers hooks unit_tests tests |
10 | @charm proof |
11 | |
12 | test: virtualenv |
13 | @@ -15,6 +15,10 @@ |
14 | @sudo apt-get install python-six |
15 | @.venv/bin/nosetests --nologcapture unit_tests |
16 | |
17 | +functional_test: |
18 | + @echo Starting Amulet tests... |
19 | + @juju test -v -p AMULET_HTTP_PROXY,AMULET_OS_VIP --timeout 2700 |
20 | + |
21 | bin/charm_helpers_sync.py: |
22 | @mkdir -p bin |
23 | @bzr cat lp:charm-helpers/tools/charm_helpers_sync/charm_helpers_sync.py \ |
24 | @@ -22,3 +26,4 @@ |
25 | |
26 | sync: bin/charm_helpers_sync.py |
27 | $(PYTHON) bin/charm_helpers_sync.py -c charm-helpers.yaml |
28 | + $(PYTHON) bin/charm_helpers_sync.py -c charm-helpers-tests.yaml |
29 | |
30 | === added file 'charm-helpers-tests.yaml' |
31 | --- charm-helpers-tests.yaml 1970-01-01 00:00:00 +0000 |
32 | +++ charm-helpers-tests.yaml 2015-04-21 22:30:08 +0000 |
33 | @@ -0,0 +1,6 @@ |
34 | +#branch: lp:charm-helpers |
35 | +branch: lp:~1chb1n/charm-helpers/amulet-mysql-enablements |
36 | +destination: tests/charmhelpers |
37 | +include: |
38 | + - contrib.amulet |
39 | + - contrib.openstack.amulet |
40 | |
41 | === modified file 'config.yaml' |
42 | --- config.yaml 2015-03-04 14:10:30 +0000 |
43 | +++ config.yaml 2015-04-21 22:30:08 +0000 |
44 | @@ -137,5 +137,5 @@ |
45 | Cron schedule used for backups. If empty backups are disabled |
46 | backup_retention_count: |
47 | default: 7 |
48 | - type: int |
49 | + type: int |
50 | description: Number of recent backups to retain. |
51 | |
52 | === modified file 'metadata.yaml' |
53 | --- metadata.yaml 2015-01-06 13:19:28 +0000 |
54 | +++ metadata.yaml 2015-04-21 22:30:08 +0000 |
55 | @@ -6,7 +6,7 @@ |
56 | server. SQL (Structured Query Language) is the most popular database query |
57 | language in the world. The main goals of MySQL are speed, robustness and |
58 | ease of use. |
59 | -categories: |
60 | +tags: |
61 | - databases |
62 | provides: |
63 | db: |
64 | |
65 | === modified file 'tests/00-setup' |
66 | --- tests/00-setup 2015-02-24 19:02:13 +0000 |
67 | +++ tests/00-setup 2015-04-21 22:30:08 +0000 |
68 | @@ -1,12 +1,21 @@ |
69 | #!/bin/bash |
70 | |
71 | -set -x |
72 | - |
73 | -# Check if amulet is installed before addign repository and updating apt-get. |
74 | -dpkg -s amulet |
75 | -if [ $? -ne 0 ]; then |
76 | - sudo add-apt-repository -y ppa:juju/stable |
77 | - sudo apt-get update -qq |
78 | - sudo apt-get install -y amulet |
79 | +set -ex |
80 | + |
81 | +sudo add-apt-repository --yes ppa:juju/stable |
82 | +sudo apt-get update --yes |
83 | +sudo apt-get install --yes python-amulet \ |
84 | + python3-pip \ |
85 | + python-keystoneclient |
86 | + |
87 | +# Enable http proxy if AMULET_HTTP_PROXY is set |
88 | +if [[ -n "$AMULET_HTTP_PROXY" ]]; then |
89 | + export HTTP_PROXY=$AMULET_HTTP_PROXY |
90 | + export HTTPS_PROXY=$(echo $AMULET_HTTP_PROXY | sed 's/http/https/g') |
91 | + export http_proxy=$HTTP_PROXY |
92 | + export https_proxy=$HTTPS_PROXY |
93 | + env | egrep "proxy|PROXY" |
94 | fi |
95 | -sudo apt-get install python3-pymysql |
96 | + |
97 | +# Setup for generic amulet test (Not in main < Vivid, pip it instead) |
98 | +sudo -E pip3 install PyMySQL |
99 | |
100 | === renamed file 'tests/15-configs' => 'tests/010-configs' |
101 | --- tests/15-configs 2015-03-04 18:00:46 +0000 |
102 | +++ tests/010-configs 2015-04-21 22:30:08 +0000 |
103 | @@ -1,7 +1,11 @@ |
104 | #!/usr/bin/python3 |
105 | |
106 | import amulet |
107 | -import pymysql |
108 | + |
109 | +try: |
110 | + import pymysql |
111 | +except ImportError: |
112 | + import pymysql3 |
113 | |
114 | |
115 | max_connections = 10 |
116 | |
117 | === added file 'tests/014-basic-precise-icehouse' |
118 | --- tests/014-basic-precise-icehouse 1970-01-01 00:00:00 +0000 |
119 | +++ tests/014-basic-precise-icehouse 2015-04-21 22:30:08 +0000 |
120 | @@ -0,0 +1,11 @@ |
121 | +#!/usr/bin/python |
122 | + |
123 | +"""Amulet tests on a basic mysql deployment on precise-icehouse.""" |
124 | + |
125 | +from basic_deployment import MySQLBasicDeployment |
126 | + |
127 | +if __name__ == '__main__': |
128 | + deployment = MySQLBasicDeployment(series='precise', |
129 | + openstack='cloud:precise-icehouse', |
130 | + source='cloud:precise-updates/icehouse') |
131 | + deployment.run_tests() |
132 | |
133 | === added file 'tests/015-basic-trusty-icehouse' |
134 | --- tests/015-basic-trusty-icehouse 1970-01-01 00:00:00 +0000 |
135 | +++ tests/015-basic-trusty-icehouse 2015-04-21 22:30:08 +0000 |
136 | @@ -0,0 +1,9 @@ |
137 | +#!/usr/bin/python |
138 | + |
139 | +"""Amulet tests on a basic mysql deployment on trusty-icehouse.""" |
140 | + |
141 | +from basic_deployment import MySQLBasicDeployment |
142 | + |
143 | +if __name__ == '__main__': |
144 | + deployment = MySQLBasicDeployment(series='trusty') |
145 | + deployment.run_tests() |
146 | |
147 | === added file 'tests/016-basic-trusty-juno' |
148 | --- tests/016-basic-trusty-juno 1970-01-01 00:00:00 +0000 |
149 | +++ tests/016-basic-trusty-juno 2015-04-21 22:30:08 +0000 |
150 | @@ -0,0 +1,11 @@ |
151 | +#!/usr/bin/python |
152 | + |
153 | +"""Amulet tests on a basic mysql deployment on trusty-juno.""" |
154 | + |
155 | +from basic_deployment import MySQLBasicDeployment |
156 | + |
157 | +if __name__ == '__main__': |
158 | + deployment = MySQLBasicDeployment(series='trusty', |
159 | + openstack='cloud:trusty-juno', |
160 | + source='cloud:trusty-updates/juno') |
161 | + deployment.run_tests() |
162 | |
163 | === added file 'tests/017-basic-trusty-kilo' |
164 | --- tests/017-basic-trusty-kilo 1970-01-01 00:00:00 +0000 |
165 | +++ tests/017-basic-trusty-kilo 2015-04-21 22:30:08 +0000 |
166 | @@ -0,0 +1,11 @@ |
167 | +#!/usr/bin/python |
168 | + |
169 | +"""Amulet tests on a basic mysql deployment on trusty-kilo.""" |
170 | + |
171 | +from basic_deployment import MySQLBasicDeployment |
172 | + |
173 | +if __name__ == '__main__': |
174 | + deployment = MySQLBasicDeployment(series='trusty', |
175 | + openstack='cloud:trusty-kilo', |
176 | + source='cloud:trusty-updates/kilo') |
177 | + deployment.run_tests() |
178 | |
179 | === added file 'tests/018-basic-utopic-juno' |
180 | --- tests/018-basic-utopic-juno 1970-01-01 00:00:00 +0000 |
181 | +++ tests/018-basic-utopic-juno 2015-04-21 22:30:08 +0000 |
182 | @@ -0,0 +1,9 @@ |
183 | +#!/usr/bin/python |
184 | + |
185 | +"""Amulet tests on a basic mysql deployment on utopic-juno.""" |
186 | + |
187 | +from basic_deployment import MySQLBasicDeployment |
188 | + |
189 | +if __name__ == '__main__': |
190 | + deployment = MySQLBasicDeployment(series='utopic') |
191 | + deployment.run_tests() |
192 | |
193 | === added file 'tests/019-basic-vivid-kilo' |
194 | --- tests/019-basic-vivid-kilo 1970-01-01 00:00:00 +0000 |
195 | +++ tests/019-basic-vivid-kilo 2015-04-21 22:30:08 +0000 |
196 | @@ -0,0 +1,9 @@ |
197 | +#!/usr/bin/python |
198 | + |
199 | +"""Amulet tests on a basic mysql deployment on vivid-kilo.""" |
200 | + |
201 | +from basic_deployment import MySQLBasicDeployment |
202 | + |
203 | +if __name__ == '__main__': |
204 | + deployment = MySQLBasicDeployment(series='vivid') |
205 | + deployment.run_tests() |
206 | |
207 | === added file 'tests/basic_deployment.py' |
208 | --- tests/basic_deployment.py 1970-01-01 00:00:00 +0000 |
209 | +++ tests/basic_deployment.py 2015-04-21 22:30:08 +0000 |
210 | @@ -0,0 +1,158 @@ |
211 | +#!/usr/bin/python |
212 | + |
213 | +import amulet |
214 | + |
215 | +from charmhelpers.contrib.openstack.amulet.deployment import ( |
216 | + OpenStackAmuletDeployment |
217 | +) |
218 | + |
219 | +from charmhelpers.contrib.openstack.amulet.utils import ( # noqa |
220 | + OpenStackAmuletUtils, |
221 | + DEBUG, |
222 | + ERROR |
223 | +) |
224 | + |
225 | +# Use DEBUG to turn on debug logging |
226 | +u = OpenStackAmuletUtils(DEBUG) |
227 | + |
228 | + |
229 | +class MySQLBasicDeployment(OpenStackAmuletDeployment): |
230 | + """Amulet tests on a basic MySQL deployment.""" |
231 | + |
232 | + def __init__(self, series=None, openstack=None, source=None, |
233 | + git=False, stable=False): |
234 | + """Deploy the test environment.""" |
235 | + super(MySQLBasicDeployment, self).__init__(series, openstack, |
236 | + source, stable) |
237 | + self.git = git |
238 | + self._add_services() |
239 | + self._add_relations() |
240 | + self._configure_services() |
241 | + self._deploy() |
242 | + self._initialize_tests() |
243 | + |
244 | + def _add_services(self): |
245 | + """Add services |
246 | + |
247 | + Add the services that we're testing, where MySQL is local, |
248 | + and the rest of the service are from lp branches that are |
249 | + compatible with the local charm (e.g. stable or next). |
250 | + """ |
251 | + this_service = {'name': 'mysql'} |
252 | + other_services = [{'name': 'keystone'}] |
253 | + super(MySQLBasicDeployment, self)._add_services(this_service, |
254 | + other_services) |
255 | + |
256 | + def _add_relations(self): |
257 | + """Add all of the relations for the services.""" |
258 | + relations = {'mysql:shared-db': 'keystone:shared-db'} |
259 | + super(MySQLBasicDeployment, self)._add_relations(relations) |
260 | + |
261 | + def _configure_services(self): |
262 | + """Configure all of the services.""" |
263 | + |
264 | + mysql_config = {} |
265 | + keystone_config = {'admin-password': 'openstack', |
266 | + 'admin-token': 'ubuntutesting'} |
267 | + |
268 | + configs = {'mysql': mysql_config, |
269 | + 'keystone': keystone_config} |
270 | + super(MySQLBasicDeployment, self)._configure_services(configs) |
271 | + |
272 | + def _initialize_tests(self): |
273 | + """Perform final initialization before tests get run.""" |
274 | + # Access the sentries for inspecting service units |
275 | + self.mysql_sentry = self.d.sentry.unit['mysql/0'] |
276 | + self.keystone_sentry = self.d.sentry.unit['keystone/0'] |
277 | + |
278 | + # Authenticate keystone admin |
279 | + self.keystone = u.authenticate_keystone_admin(self.keystone_sentry, |
280 | + user='admin', |
281 | + password='openstack', |
282 | + tenant='admin') |
283 | + |
284 | + def test_100_services(self): |
285 | + """Verify the expected services are running on the corresponding |
286 | + service units.""" |
287 | + commands = { |
288 | + self.mysql_sentry: ['status mysql'], |
289 | + self.keystone_sentry: ['status keystone'] |
290 | + } |
291 | + ret = u.validate_services(commands) |
292 | + if ret: |
293 | + amulet.raise_status(amulet.FAIL, msg=ret) |
294 | + |
295 | + def test_120_mysql_keystone_database_query(self): |
296 | + """Verify that the user table in the keystone mysql database |
297 | + contains an admin user with a specific email address.""" |
298 | + |
299 | + cmd = ("export FOO=$(sudo cat /var/lib/mysql/mysql.passwd);" |
300 | + "mysql -u root -p$FOO -e " |
301 | + "\"SELECT extra FROM keystone.user WHERE name='admin';\"") |
302 | + |
303 | + output, retcode = self.mysql_sentry.run(cmd) |
304 | + u.log.debug('command: `{}` returned {}'.format(cmd, retcode)) |
305 | + u.log.debug('output:\n{}'.format(output)) |
306 | + |
307 | + if retcode: |
308 | + msg = "command `{}` returned {}".format(cmd, str(retcode)) |
309 | + amulet.raise_status(amulet.FAIL, msg=msg) |
310 | + |
311 | + if "juju@localhost" not in output: |
312 | + msg = ("keystone mysql database query produced " |
313 | + "unexpected data:\n{}".format(output)) |
314 | + amulet.raise_status(amulet.FAIL, msg=msg) |
315 | + |
316 | + def test_150_mysql_shared_db_relation(self): |
317 | + """Verify the mysql shared-db relation data""" |
318 | + unit = self.mysql_sentry |
319 | + relation = ['shared-db', 'keystone:shared-db'] |
320 | + expected_data = { |
321 | + 'allowed_units': 'keystone/0', |
322 | + 'private-address': u.valid_ip, |
323 | + 'password': u.not_null, |
324 | + 'db_host': u.valid_ip |
325 | + } |
326 | + ret = u.validate_relation_data(unit, relation, expected_data) |
327 | + if ret: |
328 | + message = u.relation_error('mysql shared-db', ret) |
329 | + amulet.raise_status(amulet.FAIL, msg=message) |
330 | + |
331 | + def test_200_mysql_default_config(self): |
332 | + """Verify some important confg data in the mysql config file's |
333 | + mysqld section.""" |
334 | + unit = self.mysql_sentry |
335 | + conf = '/etc/mysql/my.cnf' |
336 | + relation = unit.relation('shared-db', 'keystone:shared-db') |
337 | + u.log.debug('relation: {}'.format(relation)) |
338 | + expected = {'user': 'mysql', |
339 | + 'socket': '/var/run/mysqld/mysqld.sock', |
340 | + 'port': '3306', |
341 | + 'basedir': '/usr', |
342 | + 'datadir': '/var/lib/mysql', |
343 | + 'myisam-recover': 'BACKUP', |
344 | + 'query_cache_size': '0', |
345 | + 'query_cache_type': '0', |
346 | + 'tmpdir': '/tmp', |
347 | + 'bind-address': '0.0.0.0', |
348 | + 'log_error': '/var/log/mysql/error.log', |
349 | + 'character-set-server': 'utf8'} |
350 | + |
351 | + ret = u.validate_config_data(unit, conf, 'mysqld', expected) |
352 | + if ret: |
353 | + message = "mysql config error: {}".format(ret) |
354 | + amulet.raise_status(amulet.FAIL, msg=message) |
355 | + |
356 | + def test_900_restart_on_config_change(self): |
357 | + """Verify that mysql is restarted when the config is changed. |
358 | + """ |
359 | + |
360 | + self.d.configure('mysql', {'dataset-size': '50%'}) |
361 | + |
362 | + if not u.service_restarted(self.mysql_sentry, 'mysql', |
363 | + '/etc/mysql/my.cnf', |
364 | + sleep_time=30): |
365 | + self.d.configure('mysql', {'dataset-size': '80%'}) |
366 | + message = "mysql service didn't restart after config change" |
367 | + amulet.raise_status(amulet.FAIL, msg=message) |
368 | + self.d.configure('mysql', {'dataset-size': '80%'}) |
369 | |
370 | === added directory 'tests/charmhelpers' |
371 | === added file 'tests/charmhelpers/__init__.py' |
372 | --- tests/charmhelpers/__init__.py 1970-01-01 00:00:00 +0000 |
373 | +++ tests/charmhelpers/__init__.py 2015-04-21 22:30:08 +0000 |
374 | @@ -0,0 +1,38 @@ |
375 | +# Copyright 2014-2015 Canonical Limited. |
376 | +# |
377 | +# This file is part of charm-helpers. |
378 | +# |
379 | +# charm-helpers is free software: you can redistribute it and/or modify |
380 | +# it under the terms of the GNU Lesser General Public License version 3 as |
381 | +# published by the Free Software Foundation. |
382 | +# |
383 | +# charm-helpers is distributed in the hope that it will be useful, |
384 | +# but WITHOUT ANY WARRANTY; without even the implied warranty of |
385 | +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
386 | +# GNU Lesser General Public License for more details. |
387 | +# |
388 | +# You should have received a copy of the GNU Lesser General Public License |
389 | +# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
390 | + |
391 | +# Bootstrap charm-helpers, installing its dependencies if necessary using |
392 | +# only standard libraries. |
393 | +import subprocess |
394 | +import sys |
395 | + |
396 | +try: |
397 | + import six # flake8: noqa |
398 | +except ImportError: |
399 | + if sys.version_info.major == 2: |
400 | + subprocess.check_call(['apt-get', 'install', '-y', 'python-six']) |
401 | + else: |
402 | + subprocess.check_call(['apt-get', 'install', '-y', 'python3-six']) |
403 | + import six # flake8: noqa |
404 | + |
405 | +try: |
406 | + import yaml # flake8: noqa |
407 | +except ImportError: |
408 | + if sys.version_info.major == 2: |
409 | + subprocess.check_call(['apt-get', 'install', '-y', 'python-yaml']) |
410 | + else: |
411 | + subprocess.check_call(['apt-get', 'install', '-y', 'python3-yaml']) |
412 | + import yaml # flake8: noqa |
413 | |
414 | === added directory 'tests/charmhelpers/contrib' |
415 | === added file 'tests/charmhelpers/contrib/__init__.py' |
416 | --- tests/charmhelpers/contrib/__init__.py 1970-01-01 00:00:00 +0000 |
417 | +++ tests/charmhelpers/contrib/__init__.py 2015-04-21 22:30:08 +0000 |
418 | @@ -0,0 +1,15 @@ |
419 | +# Copyright 2014-2015 Canonical Limited. |
420 | +# |
421 | +# This file is part of charm-helpers. |
422 | +# |
423 | +# charm-helpers is free software: you can redistribute it and/or modify |
424 | +# it under the terms of the GNU Lesser General Public License version 3 as |
425 | +# published by the Free Software Foundation. |
426 | +# |
427 | +# charm-helpers is distributed in the hope that it will be useful, |
428 | +# but WITHOUT ANY WARRANTY; without even the implied warranty of |
429 | +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
430 | +# GNU Lesser General Public License for more details. |
431 | +# |
432 | +# You should have received a copy of the GNU Lesser General Public License |
433 | +# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
434 | |
435 | === added directory 'tests/charmhelpers/contrib/amulet' |
436 | === added file 'tests/charmhelpers/contrib/amulet/__init__.py' |
437 | --- tests/charmhelpers/contrib/amulet/__init__.py 1970-01-01 00:00:00 +0000 |
438 | +++ tests/charmhelpers/contrib/amulet/__init__.py 2015-04-21 22:30:08 +0000 |
439 | @@ -0,0 +1,15 @@ |
440 | +# Copyright 2014-2015 Canonical Limited. |
441 | +# |
442 | +# This file is part of charm-helpers. |
443 | +# |
444 | +# charm-helpers is free software: you can redistribute it and/or modify |
445 | +# it under the terms of the GNU Lesser General Public License version 3 as |
446 | +# published by the Free Software Foundation. |
447 | +# |
448 | +# charm-helpers is distributed in the hope that it will be useful, |
449 | +# but WITHOUT ANY WARRANTY; without even the implied warranty of |
450 | +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
451 | +# GNU Lesser General Public License for more details. |
452 | +# |
453 | +# You should have received a copy of the GNU Lesser General Public License |
454 | +# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
455 | |
456 | === added file 'tests/charmhelpers/contrib/amulet/deployment.py' |
457 | --- tests/charmhelpers/contrib/amulet/deployment.py 1970-01-01 00:00:00 +0000 |
458 | +++ tests/charmhelpers/contrib/amulet/deployment.py 2015-04-21 22:30:08 +0000 |
459 | @@ -0,0 +1,93 @@ |
460 | +# Copyright 2014-2015 Canonical Limited. |
461 | +# |
462 | +# This file is part of charm-helpers. |
463 | +# |
464 | +# charm-helpers is free software: you can redistribute it and/or modify |
465 | +# it under the terms of the GNU Lesser General Public License version 3 as |
466 | +# published by the Free Software Foundation. |
467 | +# |
468 | +# charm-helpers is distributed in the hope that it will be useful, |
469 | +# but WITHOUT ANY WARRANTY; without even the implied warranty of |
470 | +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
471 | +# GNU Lesser General Public License for more details. |
472 | +# |
473 | +# You should have received a copy of the GNU Lesser General Public License |
474 | +# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
475 | + |
476 | +import amulet |
477 | +import os |
478 | +import six |
479 | + |
480 | + |
481 | +class AmuletDeployment(object): |
482 | + """Amulet deployment. |
483 | + |
484 | + This class provides generic Amulet deployment and test runner |
485 | + methods. |
486 | + """ |
487 | + |
488 | + def __init__(self, series=None): |
489 | + """Initialize the deployment environment.""" |
490 | + self.series = None |
491 | + |
492 | + if series: |
493 | + self.series = series |
494 | + self.d = amulet.Deployment(series=self.series) |
495 | + else: |
496 | + self.d = amulet.Deployment() |
497 | + |
498 | + def _add_services(self, this_service, other_services): |
499 | + """Add services. |
500 | + |
501 | + Add services to the deployment where this_service is the local charm |
502 | + that we're testing and other_services are the other services that |
503 | + are being used in the local amulet tests. |
504 | + """ |
505 | + if this_service['name'] != os.path.basename(os.getcwd()): |
506 | + s = this_service['name'] |
507 | + msg = "The charm's root directory name needs to be {}".format(s) |
508 | + amulet.raise_status(amulet.FAIL, msg=msg) |
509 | + |
510 | + if 'units' not in this_service: |
511 | + this_service['units'] = 1 |
512 | + |
513 | + self.d.add(this_service['name'], units=this_service['units']) |
514 | + |
515 | + for svc in other_services: |
516 | + if 'location' in svc: |
517 | + branch_location = svc['location'] |
518 | + elif self.series: |
519 | + branch_location = 'cs:{}/{}'.format(self.series, svc['name']), |
520 | + else: |
521 | + branch_location = None |
522 | + |
523 | + if 'units' not in svc: |
524 | + svc['units'] = 1 |
525 | + |
526 | + self.d.add(svc['name'], charm=branch_location, units=svc['units']) |
527 | + |
528 | + def _add_relations(self, relations): |
529 | + """Add all of the relations for the services.""" |
530 | + for k, v in six.iteritems(relations): |
531 | + self.d.relate(k, v) |
532 | + |
533 | + def _configure_services(self, configs): |
534 | + """Configure all of the services.""" |
535 | + for service, config in six.iteritems(configs): |
536 | + self.d.configure(service, config) |
537 | + |
538 | + def _deploy(self): |
539 | + """Deploy environment and wait for all hooks to finish executing.""" |
540 | + try: |
541 | + self.d.setup(timeout=900) |
542 | + self.d.sentry.wait(timeout=900) |
543 | + except amulet.helpers.TimeoutError: |
544 | + amulet.raise_status(amulet.FAIL, msg="Deployment timed out") |
545 | + except Exception: |
546 | + raise |
547 | + |
548 | + def run_tests(self): |
549 | + """Run all of the methods that are prefixed with 'test_'.""" |
550 | + for test in dir(self): |
551 | + if test.startswith('test_'): |
552 | + getattr(self, test)() |
553 | |
554 | === added file 'tests/charmhelpers/contrib/amulet/utils.py' |
555 | --- tests/charmhelpers/contrib/amulet/utils.py 1970-01-01 00:00:00 +0000 |
556 | +++ tests/charmhelpers/contrib/amulet/utils.py 2015-04-21 22:30:08 +0000 |
557 | @@ -0,0 +1,323 @@ |
558 | +# Copyright 2014-2015 Canonical Limited. |
559 | +# |
560 | +# This file is part of charm-helpers. |
561 | +# |
562 | +# charm-helpers is free software: you can redistribute it and/or modify |
563 | +# it under the terms of the GNU Lesser General Public License version 3 as |
564 | +# published by the Free Software Foundation. |
565 | +# |
566 | +# charm-helpers is distributed in the hope that it will be useful, |
567 | +# but WITHOUT ANY WARRANTY; without even the implied warranty of |
568 | +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
569 | +# GNU Lesser General Public License for more details. |
570 | +# |
571 | +# You should have received a copy of the GNU Lesser General Public License |
572 | +# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
573 | + |
574 | +import ConfigParser |
575 | +import io |
576 | +import logging |
577 | +import re |
578 | +import sys |
579 | +import time |
580 | + |
581 | +import six |
582 | + |
583 | + |
584 | +class AmuletUtils(object): |
585 | + """Amulet utilities. |
586 | + |
587 | + This class provides common utility functions that are used by Amulet |
588 | + tests. |
589 | + """ |
590 | + |
591 | + def __init__(self, log_level=logging.ERROR): |
592 | + self.log = self.get_logger(level=log_level) |
593 | + |
594 | + def get_logger(self, name="amulet-logger", level=logging.DEBUG): |
595 | + """Get a logger object that will log to stdout.""" |
596 | + log = logging |
597 | + logger = log.getLogger(name) |
598 | + fmt = log.Formatter("%(asctime)s %(funcName)s " |
599 | + "%(levelname)s: %(message)s") |
600 | + |
601 | + handler = log.StreamHandler(stream=sys.stdout) |
602 | + handler.setLevel(level) |
603 | + handler.setFormatter(fmt) |
604 | + |
605 | + logger.addHandler(handler) |
606 | + logger.setLevel(level) |
607 | + |
608 | + return logger |
609 | + |
610 | + def valid_ip(self, ip): |
611 | + if re.match(r"^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}$", ip): |
612 | + return True |
613 | + else: |
614 | + return False |
615 | + |
616 | + def valid_url(self, url): |
617 | + p = re.compile( |
618 | + r'^(?:http|ftp)s?://' |
619 | + r'(?:(?:[A-Z0-9](?:[A-Z0-9-]{0,61}[A-Z0-9])?\.)+(?:[A-Z]{2,6}\.?|[A-Z0-9-]{2,}\.?)|' # noqa |
620 | + r'localhost|' |
621 | + r'\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})' |
622 | + r'(?::\d+)?' |
623 | + r'(?:/?|[/?]\S+)$', |
624 | + re.IGNORECASE) |
625 | + if p.match(url): |
626 | + return True |
627 | + else: |
628 | + return False |
629 | + |
630 | + def validate_services(self, commands): |
631 | + """Validate services. |
632 | + |
633 | + Verify the specified services are running on the corresponding |
634 | + service units. |
635 | + """ |
636 | + for k, v in six.iteritems(commands): |
637 | + for cmd in v: |
638 | + output, code = k.run(cmd) |
639 | + self.log.debug('{} `{}` returned ' |
640 | + '{}'.format(k.info['unit_name'], |
641 | + cmd, code)) |
642 | + if code != 0: |
643 | + return "command `{}` returned {}".format(cmd, str(code)) |
644 | + return None |
645 | + |
646 | + def _get_config(self, unit, filename): |
647 | + """Get a ConfigParser object for parsing a unit's config file.""" |
648 | + file_contents = unit.file_contents(filename) |
649 | + |
650 | + # NOTE(beisner): by default, ConfigParser does not handle options |
651 | + # with no value, such as the flags used in the mysql my.cnf file. |
652 | + # https://bugs.python.org/issue7005 |
653 | + config = ConfigParser.ConfigParser(allow_no_value=True) |
654 | + config.readfp(io.StringIO(file_contents)) |
655 | + return config |
656 | + |
657 | + def validate_config_data(self, sentry_unit, config_file, section, |
658 | + expected): |
659 | + """Validate config file data. |
660 | + |
661 | + Verify that the specified section of the config file contains |
662 | + the expected option key:value pairs. |
663 | + """ |
664 | + config = self._get_config(sentry_unit, config_file) |
665 | + |
666 | + if section != 'DEFAULT' and not config.has_section(section): |
667 | + return "section [{}] does not exist".format(section) |
668 | + |
669 | + for k in expected.keys(): |
670 | + if not config.has_option(section, k): |
671 | + return "section [{}] is missing option {}".format(section, k) |
672 | + if config.get(section, k) != expected[k]: |
673 | + return "section [{}] {}:{} != expected {}:{}".format( |
674 | + section, k, config.get(section, k), k, expected[k]) |
675 | + return None |
676 | + |
677 | + def _validate_dict_data(self, expected, actual): |
678 | + """Validate dictionary data. |
679 | + |
680 | + Compare expected dictionary data vs actual dictionary data. |
681 | + The values in the 'expected' dictionary can be strings, bools, ints, |
682 | + longs, or can be a function that evaluate a variable and returns a |
683 | + bool. |
684 | + """ |
685 | + self.log.debug('actual: {}'.format(repr(actual))) |
686 | + self.log.debug('expected: {}'.format(repr(expected))) |
687 | + |
688 | + for k, v in six.iteritems(expected): |
689 | + if k in actual: |
690 | + if (isinstance(v, six.string_types) or |
691 | + isinstance(v, bool) or |
692 | + isinstance(v, six.integer_types)): |
693 | + if v != actual[k]: |
694 | + return "{}:{}".format(k, actual[k]) |
695 | + elif not v(actual[k]): |
696 | + return "{}:{}".format(k, actual[k]) |
697 | + else: |
698 | + return "key '{}' does not exist".format(k) |
699 | + return None |
700 | + |
701 | + def validate_relation_data(self, sentry_unit, relation, expected): |
702 | + """Validate actual relation data based on expected relation data.""" |
703 | + actual = sentry_unit.relation(relation[0], relation[1]) |
704 | + return self._validate_dict_data(expected, actual) |
705 | + |
706 | + def _validate_list_data(self, expected, actual): |
707 | + """Compare expected list vs actual list data.""" |
708 | + for e in expected: |
709 | + if e not in actual: |
710 | + return "expected item {} not found in actual list".format(e) |
711 | + return None |
712 | + |
713 | + def not_null(self, string): |
714 | + if string is not None: |
715 | + return True |
716 | + else: |
717 | + return False |
718 | + |
719 | + def _get_file_mtime(self, sentry_unit, filename): |
720 | + """Get last modification time of file.""" |
721 | + return sentry_unit.file_stat(filename)['mtime'] |
722 | + |
723 | + def _get_dir_mtime(self, sentry_unit, directory): |
724 | + """Get last modification time of directory.""" |
725 | + return sentry_unit.directory_stat(directory)['mtime'] |
726 | + |
727 | + def _get_proc_start_time(self, sentry_unit, service, pgrep_full=False): |
728 | + """Get process' start time. |
729 | + |
730 | + Determine start time of the process based on the last modification |
731 | + time of the /proc/pid directory. If pgrep_full is True, the process |
732 | + name is matched against the full command line. |
733 | + """ |
734 | + if pgrep_full: |
735 | + cmd = 'pgrep -o -f {}'.format(service) |
736 | + else: |
737 | + cmd = 'pgrep -o {}'.format(service) |
738 | + cmd = cmd + ' | grep -v pgrep || exit 0' |
739 | + cmd_out = sentry_unit.run(cmd) |
740 | + self.log.debug('CMDout: ' + str(cmd_out)) |
741 | + if cmd_out[0]: |
742 | + self.log.debug('Pid for %s %s' % (service, str(cmd_out[0]))) |
743 | + proc_dir = '/proc/{}'.format(cmd_out[0].strip()) |
744 | + return self._get_dir_mtime(sentry_unit, proc_dir) |
745 | + |
746 | + def service_restarted(self, sentry_unit, service, filename, |
747 | + pgrep_full=False, sleep_time=20): |
748 | + """Check if service was restarted. |
749 | + |
750 | + Compare a service's start time vs a file's last modification time |
751 | + (such as a config file for that service) to determine if the service |
752 | + has been restarted. |
753 | + """ |
754 | + time.sleep(sleep_time) |
755 | + if (self._get_proc_start_time(sentry_unit, service, pgrep_full) >= |
756 | + self._get_file_mtime(sentry_unit, filename)): |
757 | + return True |
758 | + else: |
759 | + return False |
760 | + |
761 | + def service_restarted_since(self, sentry_unit, mtime, service, |
762 | + pgrep_full=False, sleep_time=20, |
763 | + retry_count=2): |
764 | + """Check if service was been started after a given time. |
765 | + |
766 | + Args: |
767 | + sentry_unit (sentry): The sentry unit to check for the service on |
768 | + mtime (float): The epoch time to check against |
769 | + service (string): service name to look for in process table |
770 | + pgrep_full (boolean): Use full command line search mode with pgrep |
771 | + sleep_time (int): Seconds to sleep before looking for process |
772 | + retry_count (int): If service is not found, how many times to retry |
773 | + |
774 | + Returns: |
775 | + bool: True if service found and its start time it newer than mtime, |
776 | + False if service is older than mtime or if service was |
777 | + not found. |
778 | + """ |
779 | + self.log.debug('Checking %s restarted since %s' % (service, mtime)) |
780 | + time.sleep(sleep_time) |
781 | + proc_start_time = self._get_proc_start_time(sentry_unit, service, |
782 | + pgrep_full) |
783 | + while retry_count > 0 and not proc_start_time: |
784 | + self.log.debug('No pid file found for service %s, will retry %i ' |
785 | + 'more times' % (service, retry_count)) |
786 | + time.sleep(30) |
787 | + proc_start_time = self._get_proc_start_time(sentry_unit, service, |
788 | + pgrep_full) |
789 | + retry_count = retry_count - 1 |
790 | + |
791 | + if not proc_start_time: |
792 | + self.log.warn('No proc start time found, assuming service did ' |
793 | + 'not start') |
794 | + return False |
795 | + if proc_start_time >= mtime: |
796 | + self.log.debug('proc start time is newer than provided mtime' |
797 | + '(%s >= %s)' % (proc_start_time, mtime)) |
798 | + return True |
799 | + else: |
800 | + self.log.warn('proc start time (%s) is older than provided mtime ' |
801 | + '(%s), service did not restart' % (proc_start_time, |
802 | + mtime)) |
803 | + return False |
804 | + |
805 | + def config_updated_since(self, sentry_unit, filename, mtime, |
806 | + sleep_time=20): |
807 | + """Check if file was modified after a given time. |
808 | + |
809 | + Args: |
810 | + sentry_unit (sentry): The sentry unit to check the file mtime on |
811 | + filename (string): The file to check mtime of |
812 | + mtime (float): The epoch time to check against |
813 | + sleep_time (int): Seconds to sleep before looking for process |
814 | + |
815 | + Returns: |
816 | + bool: True if file was modified more recently than mtime, False if |
817 | + file was modified before mtime, |
818 | + """ |
819 | + self.log.debug('Checking %s updated since %s' % (filename, mtime)) |
820 | + time.sleep(sleep_time) |
821 | + file_mtime = self._get_file_mtime(sentry_unit, filename) |
822 | + if file_mtime >= mtime: |
823 | + self.log.debug('File mtime is newer than provided mtime ' |
824 | + '(%s >= %s)' % (file_mtime, mtime)) |
825 | + return True |
826 | + else: |
827 | + self.log.warn('File mtime %s is older than provided mtime %s' |
828 | + % (file_mtime, mtime)) |
829 | + return False |
830 | + |
831 | + def validate_service_config_changed(self, sentry_unit, mtime, service, |
832 | + filename, pgrep_full=False, |
833 | + sleep_time=20, retry_count=2): |
834 | + """Check service and file were updated after mtime |
835 | + |
836 | + Args: |
837 | + sentry_unit (sentry): The sentry unit to check for the service on |
838 | + mtime (float): The epoch time to check against |
839 | + service (string): service name to look for in process table |
840 | + filename (string): The file to check mtime of |
841 | + pgrep_full (boolean): Use full command line search mode with pgrep |
842 | + sleep_time (int): Seconds to sleep before looking for process |
843 | + retry_count (int): If service is not found, how many times to retry |
844 | + |
845 | + Typical Usage: |
846 | + u = OpenStackAmuletUtils(ERROR) |
847 | + ... |
848 | + mtime = u.get_sentry_time(self.cinder_sentry) |
849 | + self.d.configure('cinder', {'verbose': 'True', 'debug': 'True'}) |
850 | + if not u.validate_service_config_changed(self.cinder_sentry, |
851 | + mtime, |
852 | + 'cinder-api', |
853 | + '/etc/cinder/cinder.conf') |
854 | + amulet.raise_status(amulet.FAIL, msg='update failed') |
855 | + Returns: |
856 | + bool: True if both service and file where updated/restarted after |
857 | + mtime, False if service is older than mtime or if service was |
858 | + not found or if filename was modified before mtime. |
859 | + """ |
860 | + self.log.debug('Checking %s restarted since %s' % (service, mtime)) |
861 | + time.sleep(sleep_time) |
862 | + service_restart = self.service_restarted_since(sentry_unit, mtime, |
863 | + service, |
864 | + pgrep_full=pgrep_full, |
865 | + sleep_time=0, |
866 | + retry_count=retry_count) |
867 | + config_update = self.config_updated_since(sentry_unit, filename, mtime, |
868 | + sleep_time=0) |
869 | + return service_restart and config_update |
870 | + |
871 | + def get_sentry_time(self, sentry_unit): |
872 | + """Return current epoch time on a sentry""" |
873 | + cmd = "date +'%s'" |
874 | + return float(sentry_unit.run(cmd)[0]) |
875 | + |
876 | + def relation_error(self, name, data): |
877 | + return 'unexpected relation data in {} - {}'.format(name, data) |
878 | + |
879 | + def endpoint_error(self, name, data): |
880 | + return 'unexpected endpoint data in {} - {}'.format(name, data) |
881 | |
882 | === added directory 'tests/charmhelpers/contrib/openstack' |
883 | === added file 'tests/charmhelpers/contrib/openstack/__init__.py' |
884 | --- tests/charmhelpers/contrib/openstack/__init__.py 1970-01-01 00:00:00 +0000 |
885 | +++ tests/charmhelpers/contrib/openstack/__init__.py 2015-04-21 22:30:08 +0000 |
886 | @@ -0,0 +1,15 @@ |
887 | +# Copyright 2014-2015 Canonical Limited. |
888 | +# |
889 | +# This file is part of charm-helpers. |
890 | +# |
891 | +# charm-helpers is free software: you can redistribute it and/or modify |
892 | +# it under the terms of the GNU Lesser General Public License version 3 as |
893 | +# published by the Free Software Foundation. |
894 | +# |
895 | +# charm-helpers is distributed in the hope that it will be useful, |
896 | +# but WITHOUT ANY WARRANTY; without even the implied warranty of |
897 | +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
898 | +# GNU Lesser General Public License for more details. |
899 | +# |
900 | +# You should have received a copy of the GNU Lesser General Public License |
901 | +# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
902 | |
903 | === added directory 'tests/charmhelpers/contrib/openstack/amulet' |
904 | === added file 'tests/charmhelpers/contrib/openstack/amulet/__init__.py' |
905 | --- tests/charmhelpers/contrib/openstack/amulet/__init__.py 1970-01-01 00:00:00 +0000 |
906 | +++ tests/charmhelpers/contrib/openstack/amulet/__init__.py 2015-04-21 22:30:08 +0000 |
907 | @@ -0,0 +1,15 @@ |
908 | +# Copyright 2014-2015 Canonical Limited. |
909 | +# |
910 | +# This file is part of charm-helpers. |
911 | +# |
912 | +# charm-helpers is free software: you can redistribute it and/or modify |
913 | +# it under the terms of the GNU Lesser General Public License version 3 as |
914 | +# published by the Free Software Foundation. |
915 | +# |
916 | +# charm-helpers is distributed in the hope that it will be useful, |
917 | +# but WITHOUT ANY WARRANTY; without even the implied warranty of |
918 | +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
919 | +# GNU Lesser General Public License for more details. |
920 | +# |
921 | +# You should have received a copy of the GNU Lesser General Public License |
922 | +# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
923 | |
924 | === added file 'tests/charmhelpers/contrib/openstack/amulet/deployment.py' |
925 | --- tests/charmhelpers/contrib/openstack/amulet/deployment.py 1970-01-01 00:00:00 +0000 |
926 | +++ tests/charmhelpers/contrib/openstack/amulet/deployment.py 2015-04-21 22:30:08 +0000 |
927 | @@ -0,0 +1,146 @@ |
928 | +# Copyright 2014-2015 Canonical Limited. |
929 | +# |
930 | +# This file is part of charm-helpers. |
931 | +# |
932 | +# charm-helpers is free software: you can redistribute it and/or modify |
933 | +# it under the terms of the GNU Lesser General Public License version 3 as |
934 | +# published by the Free Software Foundation. |
935 | +# |
936 | +# charm-helpers is distributed in the hope that it will be useful, |
937 | +# but WITHOUT ANY WARRANTY; without even the implied warranty of |
938 | +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
939 | +# GNU Lesser General Public License for more details. |
940 | +# |
941 | +# You should have received a copy of the GNU Lesser General Public License |
942 | +# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
943 | + |
944 | +import six |
945 | +from collections import OrderedDict |
946 | +from charmhelpers.contrib.amulet.deployment import ( |
947 | + AmuletDeployment |
948 | +) |
949 | + |
950 | + |
951 | +class OpenStackAmuletDeployment(AmuletDeployment): |
952 | + """OpenStack amulet deployment. |
953 | + |
954 | + This class inherits from AmuletDeployment and has additional support |
955 | + that is specifically for use by OpenStack charms. |
956 | + """ |
957 | + |
958 | + def __init__(self, series=None, openstack=None, source=None, stable=True): |
959 | + """Initialize the deployment environment.""" |
960 | + super(OpenStackAmuletDeployment, self).__init__(series) |
961 | + self.openstack = openstack |
962 | + self.source = source |
963 | + self.stable = stable |
964 | + # Note(coreycb): this needs to be changed when new next branches come |
965 | + # out. |
966 | + self.current_next = "trusty" |
967 | + |
968 | + def _determine_branch_locations(self, other_services): |
969 | + """Determine the branch locations for the other services. |
970 | + |
971 | + Determine if the local branch being tested is derived from its |
972 | + stable or next (dev) branch, and based on this, use the corresonding |
973 | + stable or next branches for the other_services.""" |
974 | + base_charms = ['mysql', 'mongodb'] |
975 | + |
976 | + if self.series in ['precise', 'trusty']: |
977 | + base_series = self.series |
978 | + else: |
979 | + base_series = self.current_next |
980 | + |
981 | + if self.stable: |
982 | + for svc in other_services: |
983 | + temp = 'lp:charms/{}/{}' |
984 | + svc['location'] = temp.format(base_series, |
985 | + svc['name']) |
986 | + else: |
987 | + for svc in other_services: |
988 | + if svc['name'] in base_charms: |
989 | + temp = 'lp:charms/{}/{}' |
990 | + svc['location'] = temp.format(base_series, |
991 | + svc['name']) |
992 | + else: |
993 | + temp = 'lp:~openstack-charmers/charms/{}/{}/next' |
994 | + svc['location'] = temp.format(self.current_next, |
995 | + svc['name']) |
996 | + return other_services |
997 | + |
998 | + def _add_services(self, this_service, other_services): |
999 | + """Add services to the deployment and set openstack-origin/source.""" |
1000 | + other_services = self._determine_branch_locations(other_services) |
1001 | + |
1002 | + super(OpenStackAmuletDeployment, self)._add_services(this_service, |
1003 | + other_services) |
1004 | + |
1005 | + services = other_services |
1006 | + services.append(this_service) |
1007 | + use_source = ['mysql', 'mongodb', 'rabbitmq-server', 'ceph', |
1008 | + 'ceph-osd', 'ceph-radosgw'] |
1009 | + # Openstack subordinate charms do not expose an origin option as that |
1010 | + # is controlled by the principle |
1011 | + ignore = ['neutron-openvswitch'] |
1012 | + |
1013 | + if self.openstack: |
1014 | + for svc in services: |
1015 | + if svc['name'] not in use_source + ignore: |
1016 | + config = {'openstack-origin': self.openstack} |
1017 | + self.d.configure(svc['name'], config) |
1018 | + |
1019 | + if self.source: |
1020 | + for svc in services: |
1021 | + if svc['name'] in use_source and svc['name'] not in ignore: |
1022 | + config = {'source': self.source} |
1023 | + self.d.configure(svc['name'], config) |
1024 | + |
1025 | + def _configure_services(self, configs): |
1026 | + """Configure all of the services.""" |
1027 | + for service, config in six.iteritems(configs): |
1028 | + self.d.configure(service, config) |
1029 | + |
1030 | + def _get_openstack_release(self): |
1031 | + """Get openstack release. |
1032 | + |
1033 | + Return an integer representing the enum value of the openstack |
1034 | + release. |
1035 | + """ |
1036 | + # Must be ordered by OpenStack release (not by Ubuntu release): |
1037 | + (self.precise_essex, self.precise_folsom, self.precise_grizzly, |
1038 | + self.precise_havana, self.precise_icehouse, |
1039 | + self.trusty_icehouse, self.trusty_juno, self.utopic_juno, |
1040 | + self.trusty_kilo, self.vivid_kilo) = range(10) |
1041 | + |
1042 | + releases = { |
1043 | + ('precise', None): self.precise_essex, |
1044 | + ('precise', 'cloud:precise-folsom'): self.precise_folsom, |
1045 | + ('precise', 'cloud:precise-grizzly'): self.precise_grizzly, |
1046 | + ('precise', 'cloud:precise-havana'): self.precise_havana, |
1047 | + ('precise', 'cloud:precise-icehouse'): self.precise_icehouse, |
1048 | + ('trusty', None): self.trusty_icehouse, |
1049 | + ('trusty', 'cloud:trusty-juno'): self.trusty_juno, |
1050 | + ('trusty', 'cloud:trusty-kilo'): self.trusty_kilo, |
1051 | + ('utopic', None): self.utopic_juno, |
1052 | + ('vivid', None): self.vivid_kilo} |
1053 | + return releases[(self.series, self.openstack)] |
1054 | + |
1055 | + def _get_openstack_release_string(self): |
1056 | + """Get openstack release string. |
1057 | + |
1058 | + Return a string representing the openstack release. |
1059 | + """ |
1060 | + releases = OrderedDict([ |
1061 | + ('precise', 'essex'), |
1062 | + ('quantal', 'folsom'), |
1063 | + ('raring', 'grizzly'), |
1064 | + ('saucy', 'havana'), |
1065 | + ('trusty', 'icehouse'), |
1066 | + ('utopic', 'juno'), |
1067 | + ('vivid', 'kilo'), |
1068 | + ]) |
1069 | + if self.openstack: |
1070 | + os_origin = self.openstack.split(':')[1] |
1071 | + return os_origin.split('%s-' % self.series)[1].split('/')[0] |
1072 | + else: |
1073 | + return releases[self.series] |
1074 | |
1075 | === added file 'tests/charmhelpers/contrib/openstack/amulet/utils.py' |
1076 | --- tests/charmhelpers/contrib/openstack/amulet/utils.py 1970-01-01 00:00:00 +0000 |
1077 | +++ tests/charmhelpers/contrib/openstack/amulet/utils.py 2015-04-21 22:30:08 +0000 |
1078 | @@ -0,0 +1,294 @@ |
1079 | +# Copyright 2014-2015 Canonical Limited. |
1080 | +# |
1081 | +# This file is part of charm-helpers. |
1082 | +# |
1083 | +# charm-helpers is free software: you can redistribute it and/or modify |
1084 | +# it under the terms of the GNU Lesser General Public License version 3 as |
1085 | +# published by the Free Software Foundation. |
1086 | +# |
1087 | +# charm-helpers is distributed in the hope that it will be useful, |
1088 | +# but WITHOUT ANY WARRANTY; without even the implied warranty of |
1089 | +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
1090 | +# GNU Lesser General Public License for more details. |
1091 | +# |
1092 | +# You should have received a copy of the GNU Lesser General Public License |
1093 | +# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
1094 | + |
1095 | +import logging |
1096 | +import os |
1097 | +import time |
1098 | +import urllib |
1099 | + |
1100 | +import glanceclient.v1.client as glance_client |
1101 | +import keystoneclient.v2_0 as keystone_client |
1102 | +import novaclient.v1_1.client as nova_client |
1103 | + |
1104 | +import six |
1105 | + |
1106 | +from charmhelpers.contrib.amulet.utils import ( |
1107 | + AmuletUtils |
1108 | +) |
1109 | + |
1110 | +DEBUG = logging.DEBUG |
1111 | +ERROR = logging.ERROR |
1112 | + |
1113 | + |
1114 | +class OpenStackAmuletUtils(AmuletUtils): |
1115 | + """OpenStack amulet utilities. |
1116 | + |
1117 | + This class inherits from AmuletUtils and has additional support |
1118 | + that is specifically for use by OpenStack charms. |
1119 | + """ |
1120 | + |
1121 | + def __init__(self, log_level=ERROR): |
1122 | + """Initialize the deployment environment.""" |
1123 | + super(OpenStackAmuletUtils, self).__init__(log_level) |
1124 | + |
1125 | + def validate_endpoint_data(self, endpoints, admin_port, internal_port, |
1126 | + public_port, expected): |
1127 | + """Validate endpoint data. |
1128 | + |
1129 | + Validate actual endpoint data vs expected endpoint data. The ports |
1130 | + are used to find the matching endpoint. |
1131 | + """ |
1132 | + found = False |
1133 | + for ep in endpoints: |
1134 | + self.log.debug('endpoint: {}'.format(repr(ep))) |
1135 | + if (admin_port in ep.adminurl and |
1136 | + internal_port in ep.internalurl and |
1137 | + public_port in ep.publicurl): |
1138 | + found = True |
1139 | + actual = {'id': ep.id, |
1140 | + 'region': ep.region, |
1141 | + 'adminurl': ep.adminurl, |
1142 | + 'internalurl': ep.internalurl, |
1143 | + 'publicurl': ep.publicurl, |
1144 | + 'service_id': ep.service_id} |
1145 | + ret = self._validate_dict_data(expected, actual) |
1146 | + if ret: |
1147 | + return 'unexpected endpoint data - {}'.format(ret) |
1148 | + |
1149 | + if not found: |
1150 | + return 'endpoint not found' |
1151 | + |
1152 | + def validate_svc_catalog_endpoint_data(self, expected, actual): |
1153 | + """Validate service catalog endpoint data. |
1154 | + |
1155 | + Validate a list of actual service catalog endpoints vs a list of |
1156 | + expected service catalog endpoints. |
1157 | + """ |
1158 | + self.log.debug('actual: {}'.format(repr(actual))) |
1159 | + for k, v in six.iteritems(expected): |
1160 | + if k in actual: |
1161 | + ret = self._validate_dict_data(expected[k][0], actual[k][0]) |
1162 | + if ret: |
1163 | + return self.endpoint_error(k, ret) |
1164 | + else: |
1165 | + return "endpoint {} does not exist".format(k) |
1166 | + return ret |
1167 | + |
1168 | + def validate_tenant_data(self, expected, actual): |
1169 | + """Validate tenant data. |
1170 | + |
1171 | + Validate a list of actual tenant data vs list of expected tenant |
1172 | + data. |
1173 | + """ |
1174 | + self.log.debug('actual: {}'.format(repr(actual))) |
1175 | + for e in expected: |
1176 | + found = False |
1177 | + for act in actual: |
1178 | + a = {'enabled': act.enabled, 'description': act.description, |
1179 | + 'name': act.name, 'id': act.id} |
1180 | + if e['name'] == a['name']: |
1181 | + found = True |
1182 | + ret = self._validate_dict_data(e, a) |
1183 | + if ret: |
1184 | + return "unexpected tenant data - {}".format(ret) |
1185 | + if not found: |
1186 | + return "tenant {} does not exist".format(e['name']) |
1187 | + return ret |
1188 | + |
1189 | + def validate_role_data(self, expected, actual): |
1190 | + """Validate role data. |
1191 | + |
1192 | + Validate a list of actual role data vs a list of expected role |
1193 | + data. |
1194 | + """ |
1195 | + self.log.debug('actual: {}'.format(repr(actual))) |
1196 | + for e in expected: |
1197 | + found = False |
1198 | + for act in actual: |
1199 | + a = {'name': act.name, 'id': act.id} |
1200 | + if e['name'] == a['name']: |
1201 | + found = True |
1202 | + ret = self._validate_dict_data(e, a) |
1203 | + if ret: |
1204 | + return "unexpected role data - {}".format(ret) |
1205 | + if not found: |
1206 | + return "role {} does not exist".format(e['name']) |
1207 | + return ret |
1208 | + |
1209 | + def validate_user_data(self, expected, actual): |
1210 | + """Validate user data. |
1211 | + |
1212 | + Validate a list of actual user data vs a list of expected user |
1213 | + data. |
1214 | + """ |
1215 | + self.log.debug('actual: {}'.format(repr(actual))) |
1216 | + for e in expected: |
1217 | + found = False |
1218 | + for act in actual: |
1219 | + a = {'enabled': act.enabled, 'name': act.name, |
1220 | + 'email': act.email, 'tenantId': act.tenantId, |
1221 | + 'id': act.id} |
1222 | + if e['name'] == a['name']: |
1223 | + found = True |
1224 | + ret = self._validate_dict_data(e, a) |
1225 | + if ret: |
1226 | + return "unexpected user data - {}".format(ret) |
1227 | + if not found: |
1228 | + return "user {} does not exist".format(e['name']) |
1229 | + return ret |
1230 | + |
1231 | + def validate_flavor_data(self, expected, actual): |
1232 | + """Validate flavor data. |
1233 | + |
1234 | + Validate a list of actual flavors vs a list of expected flavors. |
1235 | + """ |
1236 | + self.log.debug('actual: {}'.format(repr(actual))) |
1237 | + act = [a.name for a in actual] |
1238 | + return self._validate_list_data(expected, act) |
1239 | + |
1240 | + def tenant_exists(self, keystone, tenant): |
1241 | + """Return True if tenant exists.""" |
1242 | + return tenant in [t.name for t in keystone.tenants.list()] |
1243 | + |
1244 | + def authenticate_keystone_admin(self, keystone_sentry, user, password, |
1245 | + tenant): |
1246 | + """Authenticates admin user with the keystone admin endpoint.""" |
1247 | + unit = keystone_sentry |
1248 | + service_ip = unit.relation('shared-db', |
1249 | + 'mysql:shared-db')['private-address'] |
1250 | + ep = "http://{}:35357/v2.0".format(service_ip.strip().decode('utf-8')) |
1251 | + return keystone_client.Client(username=user, password=password, |
1252 | + tenant_name=tenant, auth_url=ep) |
1253 | + |
1254 | + def authenticate_keystone_user(self, keystone, user, password, tenant): |
1255 | + """Authenticates a regular user with the keystone public endpoint.""" |
1256 | + ep = keystone.service_catalog.url_for(service_type='identity', |
1257 | + endpoint_type='publicURL') |
1258 | + return keystone_client.Client(username=user, password=password, |
1259 | + tenant_name=tenant, auth_url=ep) |
1260 | + |
1261 | + def authenticate_glance_admin(self, keystone): |
1262 | + """Authenticates admin user with glance.""" |
1263 | + ep = keystone.service_catalog.url_for(service_type='image', |
1264 | + endpoint_type='adminURL') |
1265 | + return glance_client.Client(ep, token=keystone.auth_token) |
1266 | + |
1267 | + def authenticate_nova_user(self, keystone, user, password, tenant): |
1268 | + """Authenticates a regular user with nova-api.""" |
1269 | + ep = keystone.service_catalog.url_for(service_type='identity', |
1270 | + endpoint_type='publicURL') |
1271 | + return nova_client.Client(username=user, api_key=password, |
1272 | + project_id=tenant, auth_url=ep) |
1273 | + |
1274 | + def create_cirros_image(self, glance, image_name): |
1275 | + """Download the latest cirros image and upload it to glance.""" |
1276 | + http_proxy = os.getenv('AMULET_HTTP_PROXY') |
1277 | + self.log.debug('AMULET_HTTP_PROXY: {}'.format(http_proxy)) |
1278 | + if http_proxy: |
1279 | + proxies = {'http': http_proxy} |
1280 | + opener = urllib.FancyURLopener(proxies) |
1281 | + else: |
1282 | + opener = urllib.FancyURLopener() |
1283 | + |
1284 | + f = opener.open("http://download.cirros-cloud.net/version/released") |
1285 | + version = f.read().strip() |
1286 | + cirros_img = "cirros-{}-x86_64-disk.img".format(version) |
1287 | + local_path = os.path.join('tests', cirros_img) |
1288 | + |
1289 | + if not os.path.exists(local_path): |
1290 | + cirros_url = "http://{}/{}/{}".format("download.cirros-cloud.net", |
1291 | + version, cirros_img) |
1292 | + opener.retrieve(cirros_url, local_path) |
1293 | + f.close() |
1294 | + |
1295 | + with open(local_path) as f: |
1296 | + image = glance.images.create(name=image_name, is_public=True, |
1297 | + disk_format='qcow2', |
1298 | + container_format='bare', data=f) |
1299 | + count = 1 |
1300 | + status = image.status |
1301 | + while status != 'active' and count < 10: |
1302 | + time.sleep(3) |
1303 | + image = glance.images.get(image.id) |
1304 | + status = image.status |
1305 | + self.log.debug('image status: {}'.format(status)) |
1306 | + count += 1 |
1307 | + |
1308 | + if status != 'active': |
1309 | + self.log.error('image creation timed out') |
1310 | + return None |
1311 | + |
1312 | + return image |
1313 | + |
1314 | + def delete_image(self, glance, image): |
1315 | + """Delete the specified image.""" |
1316 | + num_before = len(list(glance.images.list())) |
1317 | + glance.images.delete(image) |
1318 | + |
1319 | + count = 1 |
1320 | + num_after = len(list(glance.images.list())) |
1321 | + while num_after != (num_before - 1) and count < 10: |
1322 | + time.sleep(3) |
1323 | + num_after = len(list(glance.images.list())) |
1324 | + self.log.debug('number of images: {}'.format(num_after)) |
1325 | + count += 1 |
1326 | + |
1327 | + if num_after != (num_before - 1): |
1328 | + self.log.error('image deletion timed out') |
1329 | + return False |
1330 | + |
1331 | + return True |
1332 | + |
1333 | + def create_instance(self, nova, image_name, instance_name, flavor): |
1334 | + """Create the specified instance.""" |
1335 | + image = nova.images.find(name=image_name) |
1336 | + flavor = nova.flavors.find(name=flavor) |
1337 | + instance = nova.servers.create(name=instance_name, image=image, |
1338 | + flavor=flavor) |
1339 | + |
1340 | + count = 1 |
1341 | + status = instance.status |
1342 | + while status != 'ACTIVE' and count < 60: |
1343 | + time.sleep(3) |
1344 | + instance = nova.servers.get(instance.id) |
1345 | + status = instance.status |
1346 | + self.log.debug('instance status: {}'.format(status)) |
1347 | + count += 1 |
1348 | + |
1349 | + if status != 'ACTIVE': |
1350 | + self.log.error('instance creation timed out') |
1351 | + return None |
1352 | + |
1353 | + return instance |
1354 | + |
1355 | + def delete_instance(self, nova, instance): |
1356 | + """Delete the specified instance.""" |
1357 | + num_before = len(list(nova.servers.list())) |
1358 | + nova.servers.delete(instance) |
1359 | + |
1360 | + count = 1 |
1361 | + num_after = len(list(nova.servers.list())) |
1362 | + while num_after != (num_before - 1) and count < 10: |
1363 | + time.sleep(3) |
1364 | + num_after = len(list(nova.servers.list())) |
1365 | + self.log.debug('number of instances: {}'.format(num_after)) |
1366 | + count += 1 |
1367 | + |
1368 | + if num_after != (num_before - 1): |
1369 | + self.log.error('instance deletion timed out') |
1370 | + return False |
1371 | + |
1372 | + return True |
charm_unit_test #3473 mysql for 1chb1n mp256941
UNIT FAIL: unit-test failed
UNIT Results (max last 2 lines):
make: *** [test] Error 1
ERROR:root:Make target returned non-zero.
Full unit test output: http:// paste.ubuntu. com/10862036/ 10.245. 162.77: 8080/job/ charm_unit_ test/3473/
Build: http://