Merge lp:~evarlast/charms/trusty/mongodb/fix-dump-actions into lp:charms/trusty/mongodb
- Trusty Tahr (14.04)
- fix-dump-actions
- Merge into trunk
Status: | Merged |
---|---|
Merged at revision: | 80 |
Proposed branch: | lp:~evarlast/charms/trusty/mongodb/fix-dump-actions |
Merge into: | lp:charms/trusty/mongodb |
Diff against target: |
461 lines (+156/-59) 9 files modified
actions/backup.py (+5/-4) config.yaml (+1/-1) hooks/hooks.py (+1/-0) metadata.yaml (+1/-1) tests/01_deploy_single.py (+4/-1) tests/02_deploy_shard_test.py (+47/-19) tests/03_deploy_replicaset.py (+73/-19) tests/04_deploy_with_storage.py (+18/-11) tests/50_relate_ceilometer_test.py (+6/-3) |
To merge this branch: | bzr merge lp:~evarlast/charms/trusty/mongodb/fix-dump-actions |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Review Queue (community) | automated testing | Approve | |
Matt Bruzek (community) | Approve | ||
Jay R. Wren (community) | Needs Resubmitting | ||
Ryan Beisner (community) | Needs Fixing | ||
Matthew Williams (community) | Approve | ||
Tim Van Steenburgh (community) | Needs Fixing | ||
Review via email: mp+277191@code.launchpad.net |
Commit message
Description of the change
Dump wasn't working. Now it works and if args has spaces, they can be escaped with quotes.
e.g.
$ juju action do mongodb/0 dump args='-q "{a:1, b:2}"'
Action queued with id: 7b627526-
$ juju action fetch 7b
results:
command: mongodump -q "{a:1, b:2}"
uosci-testing-bot (uosci-testing-bot) wrote : | # |
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_unit_test #12564 mongodb for evarlast mp277191
UNIT OK: passed
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_amulet_test #7839 mongodb for evarlast mp277191
AMULET OK: passed
Build: http://
Tim Van Steenburgh (tvansteenburgh) wrote : | # |
Charm CI tests failed due to hardcoded unit numbers in the tests. Jenkins log: http://
To fix, replace occurrences of `d.sentry.
- 80. By Jay R. Wren
-
fix hardcoded unit numbers in the tests
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_unit_test #12615 mongodb for evarlast mp277191
UNIT OK: passed
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_lint_check #13484 mongodb for evarlast mp277191
LINT OK: passed
Build: http://
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_amulet_test #7845 mongodb for evarlast mp277191
AMULET FAIL: amulet-test failed
AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.
Full amulet test output: http://
Build: http://
Review Queue (review-queue) wrote : | # |
This item has failed automated testing! Results available here http://
- 81. By Jay R. Wren
-
use sentry[service][N] instead of sentry.unit[unit]
- 82. By Jay R. Wren
-
categories is now tags.
config key now defaults to ""
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_lint_check #13701 mongodb for evarlast mp277191
LINT OK: passed
Build: http://
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_unit_test #12771 mongodb for evarlast mp277191
UNIT OK: passed
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_amulet_test #7858 mongodb for evarlast mp277191
AMULET FAIL: amulet-test failed
AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.
Full amulet test output: http://
Build: http://
- 83. By Jay R. Wren
-
some ipv6 support
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_lint_check #13928 mongodb for evarlast mp277191
LINT FAIL: lint-test failed
LINT Results (max last 2 lines):
make: *** [lint] Error 1
ERROR:root:Make target returned non-zero.
Full lint test output: http://
Build: http://
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_unit_test #12984 mongodb for evarlast mp277191
UNIT OK: passed
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_amulet_test #7911 mongodb for evarlast mp277191
AMULET FAIL: amulet-test failed
AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.
Full amulet test output: http://
Build: http://
- 84. By Jay R. Wren
-
fix ipv6 url
- 85. By Jay R. Wren
-
fix ipv6 url
- 86. By Jay R. Wren
-
fix lint
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_lint_check #14010 mongodb for evarlast mp277191
LINT OK: passed
Build: http://
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_unit_test #13061 mongodb for evarlast mp277191
UNIT OK: passed
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_amulet_test #7939 mongodb for evarlast mp277191
AMULET OK: passed
Build: http://
- 87. By Jay R. Wren
-
fix broken amulet tests
we do not know which is master, cannot assume it is /0, so try 0 and if it fails try /1.
enable ipv6 in mongodb. We cannot know which provider is being used and if the public-address may be ipv6, so we must enable ipv6
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_lint_check #14066 mongodb for evarlast mp277191
LINT FAIL: lint-test failed
LINT Results (max last 2 lines):
make: *** [lint] Error 1
ERROR:root:Make target returned non-zero.
Full lint test output: http://
Build: http://
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_unit_test #13112 mongodb for evarlast mp277191
UNIT OK: passed
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_amulet_test #7941 mongodb for evarlast mp277191
AMULET FAIL: amulet-test failed
AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.
Full amulet test output: http://
Build: http://
- 88. By Jay R. Wren
-
use sentry.wait() instead of time.sleep - much faster.
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_lint_check #14136 mongodb for evarlast mp277191
LINT FAIL: lint-test failed
LINT Results (max last 2 lines):
make: *** [lint] Error 1
ERROR:root:Make target returned non-zero.
Full lint test output: http://
Build: http://
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_unit_test #13176 mongodb for evarlast mp277191
UNIT OK: passed
- 89. By Jay R. Wren
-
fix lint
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_lint_check #14137 mongodb for evarlast mp277191
LINT OK: passed
Build: http://
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_unit_test #13177 mongodb for evarlast mp277191
UNIT OK: passed
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_amulet_test #7957 mongodb for evarlast mp277191
AMULET FAIL: amulet-test failed
AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.
Full amulet test output: http://
Build: http://
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_amulet_test #7956 mongodb for evarlast mp277191
AMULET OK: passed
Build: http://
- 90. By Jay R. Wren
-
Use hostname used in relation.
Can't assume relation hostname is public-address. It isn't in many environments.
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_lint_check #14138 mongodb for evarlast mp277191
LINT OK: passed
Build: http://
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_unit_test #13178 mongodb for evarlast mp277191
UNIT OK: passed
- 91. By Jay R. Wren
-
use sentry.wait not time.sleep
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_amulet_test #7958 mongodb for evarlast mp277191
AMULET FAIL: amulet-test failed
AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.
Full amulet test output: http://
Build: http://
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_unit_test #13180 mongodb for evarlast mp277191
UNIT OK: passed
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_lint_check #14140 mongodb for evarlast mp277191
LINT OK: passed
Build: http://
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_amulet_test #7960 mongodb for evarlast mp277191
AMULET FAIL: amulet-test failed
AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.
Full amulet test output: http://
Build: http://
- 92. By Jay R. Wren
-
wait longer for replset
Don't retry on unit 1 when unit 0 is not primary.
Additional logging on fail.
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_lint_check #14141 mongodb for evarlast mp277191
LINT FAIL: lint-test failed
LINT Results (max last 2 lines):
make: *** [lint] Error 1
ERROR:root:Make target returned non-zero.
Full lint test output: http://
Build: http://
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_unit_test #13181 mongodb for evarlast mp277191
UNIT OK: passed
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_amulet_test #7961 mongodb for evarlast mp277191
AMULET FAIL: amulet-test failed
AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.
Full amulet test output: http://
Build: http://
- 93. By Jay R. Wren
-
fix lint
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_unit_test #13182 mongodb for evarlast mp277191
UNIT OK: passed
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_lint_check #14142 mongodb for evarlast mp277191
LINT OK: passed
Build: http://
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_amulet_test #7962 mongodb for evarlast mp277191
AMULET FAIL: amulet-test failed
AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.
Full amulet test output: http://
Build: http://
- 94. By Jay R. Wren
-
increase wait time for CI
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_lint_check #14253 mongodb for evarlast mp277191
LINT OK: passed
Build: http://
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_unit_test #13284 mongodb for evarlast mp277191
UNIT OK: passed
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_amulet_test #8016 mongodb for evarlast mp277191
AMULET FAIL: amulet-test failed
AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.
Full amulet test output: http://
Build: http://
- 95. By Jay R. Wren
-
increase wait time for CI
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_lint_check #14254 mongodb for evarlast mp277191
LINT OK: passed
Build: http://
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_unit_test #13285 mongodb for evarlast mp277191
UNIT OK: passed
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_amulet_test #8017 mongodb for evarlast mp277191
AMULET FAIL: amulet-test failed
AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.
Full amulet test output: http://
Build: http://
Ryan Beisner (1chb1n) wrote : | # |
Thank you for your work on this.
While I realize that the test race wasn't introduced by your proposed changes, please see:
https:/
It would be awesome to resolve that test issue along with this, as it's definitely presenting a pain point.
- 96. By Jay R. Wren
-
retry every 10s for replset validation
Jay R. Wren (evarlast) wrote : | # |
This should address https:/
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_lint_check #14310 mongodb for evarlast mp277191
LINT OK: passed
Build: http://
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_unit_test #13338 mongodb for evarlast mp277191
UNIT OK: passed
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_amulet_test #8028 mongodb for evarlast mp277191
AMULET OK: passed
Build: http://
Review Queue (review-queue) wrote : | # |
This item has failed automated testing! Results available here http://
Review Queue (review-queue) wrote : | # |
This item has failed automated testing! Results available here http://
Review Queue (review-queue) wrote : | # |
This item has failed automated testing! Results available here http://
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_lint_check #14635 mongodb for evarlast mp277191
LINT OK: passed
Build: http://
Review Queue (review-queue) wrote : | # |
This item has failed automated testing! Results available here http://
- 97. By Jay R. Wren
-
retry replSetGetStatus and use minimum unit number for master
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_lint_check #14756 mongodb for evarlast mp277191
LINT FAIL: lint-test failed
LINT Results (max last 2 lines):
make: *** [lint] Error 1
ERROR:root:Make target returned non-zero.
Full lint test output: http://
Build: http://
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_unit_test #13759 mongodb for evarlast mp277191
UNIT OK: passed
- 98. By Jay R. Wren
-
fix lint
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_lint_check #14757 mongodb for evarlast mp277191
LINT OK: passed
Build: http://
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_unit_test #13761 mongodb for evarlast mp277191
UNIT OK: passed
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_amulet_test #8078 mongodb for evarlast mp277191
AMULET OK: passed
Build: http://
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_amulet_test #8079 mongodb for evarlast mp277191
AMULET FAIL: amulet-test failed
AMULET Results (max last 2 lines):
make: *** [functional_test] Error 124
ERROR:root:Make target returned non-zero.
Full amulet test output: http://
Build: http://
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_amulet_test #8081 mongodb for evarlast mp277191
AMULET OK: passed
Build: http://
- 99. By Jay R. Wren
-
retry shard status interface test
sort units and use them based on order in 04_deploy_with_storage test
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_lint_check #14813 mongodb for evarlast mp277191
LINT OK: passed
Build: http://
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_unit_test #13813 mongodb for evarlast mp277191
UNIT OK: passed
Review Queue (review-queue) wrote : | # |
This item has failed automated testing! Results available here http://
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_amulet_test #8134 mongodb for evarlast mp277191
AMULET FAIL: amulet-test failed
AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.
Full amulet test output: http://
Build: http://
Review Queue (review-queue) wrote : | # |
This item has failed automated testing! Results available here http://
Review Queue (review-queue) wrote : | # |
This item has failed automated testing! Results available here http://
Review Queue (review-queue) wrote : | # |
This item has failed automated testing! Results available here http://
Review Queue (review-queue) wrote : | # |
This item has failed automated testing! Results available here http://
- 100. By Jay R. Wren
-
use constraints for storage test to prevent filling root volume
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_lint_check #15041 mongodb for evarlast mp277191
LINT OK: passed
Build: http://
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_unit_test #14029 mongodb for evarlast mp277191
UNIT OK: passed
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_amulet_test #8205 mongodb for evarlast mp277191
AMULET OK: passed
Build: http://
Review Queue (review-queue) wrote : | # |
This item has failed automated testing! Results available here http://
Review Queue (review-queue) wrote : | # |
This item has failed automated testing! Results available here http://
Review Queue (review-queue) wrote : | # |
This item has failed automated testing! Results available here http://
Review Queue (review-queue) wrote : | # |
This item has failed automated testing! Results available here http://
Jay R. Wren (evarlast) wrote : | # |
The only fail remaining is from a known bug, which I am not attempting to fix.
From the README:
Deploying three or more units at start can sometimes lead to unexpected race-conditions so it's best to start with two nodes.
http://
Please consider this for merging.
Matt Bruzek (mbruzek) wrote : | # |
Hello Jay,
While reviewing the queue today and noticed more than one mongodb charm merge proposals. Since we review the oldest first this proposal did not get reviewed first.
As Ryan points out in a bug 1518468 ( https:/
## 03_deploy_
The new code contains the comment “# We assume minimum unit number is master.” There should be a better way to get the master address in the test. Again you may be able to run commands on the units to determine their master relationship, or use a mongo python library to determine the master reliably.
The tests passed for me on AWS and I am going to merge this code. I would encourage you to change the test code to detect the state of the mongodb deployment from reliable sources that do not involve sleeping for arbitrary amounts of time. Such as running a command on the unit to tell when it is ready, or using `juju status-history unit/0` to determine ready states.
Review Queue (review-queue) wrote : | # |
This item has failed automated testing! Results available here http://
Review Queue (review-queue) wrote : | # |
The results (PASS) are in and available here: http://
Preview Diff
1 | === modified file 'actions/backup.py' | |||
2 | --- actions/backup.py 2015-10-08 08:12:35 +0000 | |||
3 | +++ actions/backup.py 2015-12-07 22:34:32 +0000 | |||
4 | @@ -1,3 +1,4 @@ | |||
5 | 1 | import shlex | ||
6 | 1 | import subprocess | 2 | import subprocess |
7 | 2 | import os | 3 | import os |
8 | 3 | try: | 4 | try: |
9 | @@ -14,7 +15,7 @@ | |||
10 | 14 | 15 | ||
11 | 15 | def execute(command, current_working_directory): | 16 | def execute(command, current_working_directory): |
12 | 16 | return subprocess.check_output( | 17 | return subprocess.check_output( |
14 | 17 | " ".join(command), | 18 | shlex.split(command), |
15 | 18 | cwd=current_working_directory, | 19 | cwd=current_working_directory, |
16 | 19 | ) | 20 | ) |
17 | 20 | 21 | ||
18 | @@ -38,7 +39,7 @@ | |||
19 | 38 | pass # Ignoring, the directory already exists | 39 | pass # Ignoring, the directory already exists |
20 | 39 | except Exception, e: | 40 | except Exception, e: |
21 | 40 | action_set({"directory creation exception": e}) | 41 | action_set({"directory creation exception": e}) |
23 | 41 | action_fail(e) | 42 | action_fail(str(e)) |
24 | 42 | return | 43 | return |
25 | 43 | 44 | ||
26 | 44 | command = cmd + " " + args | 45 | command = cmd + " " + args |
27 | @@ -50,7 +51,7 @@ | |||
28 | 50 | except subprocess.CalledProcessError, e: | 51 | except subprocess.CalledProcessError, e: |
29 | 51 | action_set({"error_code": e.returncode, | 52 | action_set({"error_code": e.returncode, |
30 | 52 | "exception": e, "output": e.output}) | 53 | "exception": e, "output": e.output}) |
32 | 53 | action_fail(e) | 54 | action_fail(str(e)) |
33 | 54 | except Exception, e: | 55 | except Exception, e: |
34 | 55 | action_set({"exception": e}) | 56 | action_set({"exception": e}) |
36 | 56 | action_fail(e) | 57 | action_fail(str(e)) |
37 | 57 | 58 | ||
38 | === modified file 'config.yaml' | |||
39 | --- config.yaml 2015-02-25 00:24:56 +0000 | |||
40 | +++ config.yaml 2015-12-07 22:34:32 +0000 | |||
41 | @@ -211,7 +211,7 @@ | |||
42 | 211 | option. | 211 | option. |
43 | 212 | key: | 212 | key: |
44 | 213 | type: string | 213 | type: string |
46 | 214 | default: | 214 | default: "" |
47 | 215 | description: > | 215 | description: > |
48 | 216 | Key ID to import to the apt keyring to support use with arbitary source | 216 | Key ID to import to the apt keyring to support use with arbitary source |
49 | 217 | configuration from outside of Launchpad archives or PPA's. | 217 | configuration from outside of Launchpad archives or PPA's. |
50 | 218 | 218 | ||
51 | === modified file 'hooks/hooks.py' | |||
52 | --- hooks/hooks.py 2015-09-09 12:59:18 +0000 | |||
53 | +++ hooks/hooks.py 2015-12-07 22:34:32 +0000 | |||
54 | @@ -254,6 +254,7 @@ | |||
55 | 254 | subprocess.call(['chown', '-R', 'mongodb:mongodb', config_data['dbpath']]) | 254 | subprocess.call(['chown', '-R', 'mongodb:mongodb', config_data['dbpath']]) |
56 | 255 | config.append("dbpath=%s" % config_data['dbpath']) | 255 | config.append("dbpath=%s" % config_data['dbpath']) |
57 | 256 | config.append("") | 256 | config.append("") |
58 | 257 | config.append("ipv6=true") | ||
59 | 257 | 258 | ||
60 | 258 | # logpath | 259 | # logpath |
61 | 259 | # Create the directory if not there already | 260 | # Create the directory if not there already |
62 | 260 | 261 | ||
63 | === modified file 'metadata.yaml' | |||
64 | --- metadata.yaml 2015-05-19 13:21:16 +0000 | |||
65 | +++ metadata.yaml 2015-12-07 22:34:32 +0000 | |||
66 | @@ -18,7 +18,7 @@ | |||
67 | 18 | - Auto-sharding for cloud-level scalability (Q209) | 18 | - Auto-sharding for cloud-level scalability (Q209) |
68 | 19 | High performance, scalability, and reasonable depth of functionality | 19 | High performance, scalability, and reasonable depth of functionality |
69 | 20 | are the goals for the project. | 20 | are the goals for the project. |
71 | 21 | categories: | 21 | tags: |
72 | 22 | - databases | 22 | - databases |
73 | 23 | provides: | 23 | provides: |
74 | 24 | nrpe-external-master: | 24 | nrpe-external-master: |
75 | 25 | 25 | ||
76 | === modified file 'tests/01_deploy_single.py' | |||
77 | --- tests/01_deploy_single.py 2014-12-10 15:36:14 +0000 | |||
78 | +++ tests/01_deploy_single.py 2015-12-07 22:34:32 +0000 | |||
79 | @@ -24,7 +24,10 @@ | |||
80 | 24 | # Validate connectivity from $WORLD | 24 | # Validate connectivity from $WORLD |
81 | 25 | ############################################################# | 25 | ############################################################# |
82 | 26 | def validate_world_connectivity(): | 26 | def validate_world_connectivity(): |
84 | 27 | client = MongoClient(d.sentry.unit['mongodb/0'].info['public-address']) | 27 | addy = d.sentry['mongodb'][0].info['public-address'] |
85 | 28 | if ":" in addy: | ||
86 | 29 | addy = "[{}]".format(addy) | ||
87 | 30 | client = MongoClient(addy) | ||
88 | 28 | 31 | ||
89 | 29 | db = client['test'] | 32 | db = client['test'] |
90 | 30 | # Can we successfully insert? | 33 | # Can we successfully insert? |
91 | 31 | 34 | ||
92 | === modified file 'tests/02_deploy_shard_test.py' | |||
93 | --- tests/02_deploy_shard_test.py 2014-12-09 23:45:14 +0000 | |||
94 | +++ tests/02_deploy_shard_test.py 2015-12-07 22:34:32 +0000 | |||
95 | @@ -2,6 +2,9 @@ | |||
96 | 2 | 2 | ||
97 | 3 | import amulet | 3 | import amulet |
98 | 4 | import requests | 4 | import requests |
99 | 5 | import sys | ||
100 | 6 | import time | ||
101 | 7 | import traceback | ||
102 | 5 | from pymongo import MongoClient | 8 | from pymongo import MongoClient |
103 | 6 | 9 | ||
104 | 7 | ######################################################### | 10 | ######################################################### |
105 | @@ -49,10 +52,10 @@ | |||
106 | 49 | raise | 52 | raise |
107 | 50 | 53 | ||
108 | 51 | sentry_dict = { | 54 | sentry_dict = { |
113 | 52 | 'config-sentry': d.sentry.unit['configsvr/0'], | 55 | 'config-sentry': d.sentry['configsvr'][0], |
114 | 53 | 'mongos-sentry': d.sentry.unit['mongos/0'], | 56 | 'mongos-sentry': d.sentry['mongos'][0], |
115 | 54 | 'shard1-sentry': d.sentry.unit['shard1/0'], | 57 | 'shard1-sentry': d.sentry['shard1'][0], |
116 | 55 | 'shard2-sentry': d.sentry.unit['shard2/0'] | 58 | 'shard2-sentry': d.sentry['shard2'][0] |
117 | 56 | } | 59 | } |
118 | 57 | 60 | ||
119 | 58 | 61 | ||
120 | @@ -60,10 +63,27 @@ | |||
121 | 60 | # Check presence of MongoDB GUI HEALTH Status | 63 | # Check presence of MongoDB GUI HEALTH Status |
122 | 61 | ############################################################# | 64 | ############################################################# |
123 | 62 | def validate_status_interface(): | 65 | def validate_status_interface(): |
128 | 63 | r = requests.get("http://{}:28017".format( | 66 | pubaddy = sentry_dict['config-sentry'].info['public-address'] |
129 | 64 | sentry_dict['config-sentry'].info['public-address']), | 67 | fmt = "http://{}:28017" |
130 | 65 | verify=False) | 68 | if ":" in pubaddy: |
131 | 66 | r.raise_for_status | 69 | fmt = "http://[{}]:28017" |
132 | 70 | time_between = 10 | ||
133 | 71 | tries = seconds / time_between | ||
134 | 72 | try: | ||
135 | 73 | r = requests.get(fmt.format(pubaddy), verify=False) | ||
136 | 74 | r.raise_for_status() | ||
137 | 75 | except requests.exception.ConnectionError as ex: | ||
138 | 76 | sys.stderr.write( | ||
139 | 77 | 'Connection error, sleep and retry... to {}: {}\n'. | ||
140 | 78 | format(pubaddy, ex)) | ||
141 | 79 | tb_lines = traceback.format_exception(ex.__class__, | ||
142 | 80 | ex, ex.__traceback__) | ||
143 | 81 | tb_text = ''.join(tb_lines) | ||
144 | 82 | sys.stderr.write(tb_text) | ||
145 | 83 | tries = tries - 1 | ||
146 | 84 | if tries < 0: | ||
147 | 85 | sys.stderr.write('retry limit caught, failing...\n') | ||
148 | 86 | time.sleep(time_between) | ||
149 | 67 | 87 | ||
150 | 68 | 88 | ||
151 | 69 | ############################################################# | 89 | ############################################################# |
152 | @@ -83,7 +103,10 @@ | |||
153 | 83 | # Validate connectivity from $WORLD | 103 | # Validate connectivity from $WORLD |
154 | 84 | ############################################################# | 104 | ############################################################# |
155 | 85 | def validate_world_connectivity(): | 105 | def validate_world_connectivity(): |
157 | 86 | client = MongoClient(d.sentry.unit['mongos/0'].info['public-address']) | 106 | pubaddy = d.sentry['mongos'][0].info['public-address'] |
158 | 107 | if ":" in pubaddy: | ||
159 | 108 | pubaddy = "[{}]".format(pubaddy) | ||
160 | 109 | client = MongoClient(pubaddy) | ||
161 | 87 | 110 | ||
162 | 88 | db = client['test'] | 111 | db = client['test'] |
163 | 89 | # Can we successfully insert? | 112 | # Can we successfully insert? |
164 | @@ -101,23 +124,28 @@ | |||
165 | 101 | ############################################################# | 124 | ############################################################# |
166 | 102 | # broken pending 1273312 | 125 | # broken pending 1273312 |
167 | 103 | def validate_relationships(): | 126 | def validate_relationships(): |
172 | 104 | d.sentry.unit['configsvr/0'].relation('configsvr', 'mongos:mongos-cfg') | 127 | d.sentry['configsvr'][0].relation('configsvr', 'mongos:mongos-cfg') |
173 | 105 | d.sentry.unit['shard1/0'].relation('database', 'mongos:mongos') | 128 | d.sentry['shard1'][0].relation('database', 'mongos:mongos') |
174 | 106 | d.sentry.unit['shard2/0'].relation('database', 'mongos:mongos') | 129 | d.sentry['shard2'][0].relation('database', 'mongos:mongos') |
175 | 107 | print(d.sentry.unit['shard1/0'].relation('database', 'mongos:mongos')) | 130 | print(d.sentry['shard1'][0].relation('database', 'mongos:mongos')) |
176 | 108 | 131 | ||
177 | 109 | 132 | ||
178 | 110 | def validate_manual_connection(): | 133 | def validate_manual_connection(): |
181 | 111 | output, code = d.sentry.unit['shard1/0'].run("mongo {}".format( | 134 | fmt = "mongo {}" |
182 | 112 | d.sentry.unit['mongos/0'].info['public-address'])) | 135 | addy = d.sentry['mongos'][0].info['public-address'] |
183 | 136 | if ":" in addy: | ||
184 | 137 | fmt = "mongo --ipv6 {}:27017" | ||
185 | 138 | jujuruncmd = fmt.format(addy) | ||
186 | 139 | output, code = d.sentry['shard1'][0].run(jujuruncmd) | ||
187 | 113 | if code != 0: | 140 | if code != 0: |
189 | 114 | message = "Manual Connection failed for unit shard1" | 141 | message = ("Manual Connection failed for unit shard1:{} code:{} cmd:{}" |
190 | 142 | .format(output, code, jujuruncmd)) | ||
191 | 115 | amulet.raise_status(amulet.SKIP, msg=message) | 143 | amulet.raise_status(amulet.SKIP, msg=message) |
192 | 116 | 144 | ||
195 | 117 | output, code = d.sentry.unit['shard2/0'].run("mongo {}".format( | 145 | output, code = d.sentry['shard2'][0].run(jujuruncmd) |
194 | 118 | d.sentry.unit['mongos/0'].info['public-address'])) | ||
196 | 119 | if code != 0: | 146 | if code != 0: |
198 | 120 | message = "Manual Connection failed for unit shard2" | 147 | message = ("Manual Connection failed for unit shard2:{} code:{} cmd:{}" |
199 | 148 | .format(output, code, jujuruncmd)) | ||
200 | 121 | amulet.raise_status(amulet.SKIP, msg=message) | 149 | amulet.raise_status(amulet.SKIP, msg=message) |
201 | 122 | 150 | ||
202 | 123 | 151 | ||
203 | 124 | 152 | ||
204 | === modified file 'tests/03_deploy_replicaset.py' | |||
205 | --- tests/03_deploy_replicaset.py 2015-02-25 01:49:08 +0000 | |||
206 | +++ tests/03_deploy_replicaset.py 2015-12-07 22:34:32 +0000 | |||
207 | @@ -1,8 +1,11 @@ | |||
208 | 1 | #!/usr/bin/env python3 | 1 | #!/usr/bin/env python3 |
209 | 2 | 2 | ||
210 | 3 | import amulet | 3 | import amulet |
211 | 4 | import sys | ||
212 | 5 | import time | ||
213 | 6 | import traceback | ||
214 | 4 | import requests | 7 | import requests |
216 | 5 | import time | 8 | import pymongo |
217 | 6 | from pymongo import MongoClient | 9 | from pymongo import MongoClient |
218 | 7 | from collections import Counter | 10 | from collections import Counter |
219 | 8 | 11 | ||
220 | @@ -13,9 +16,9 @@ | |||
221 | 13 | scale = 3 | 16 | scale = 3 |
222 | 14 | seconds = 1800 | 17 | seconds = 1800 |
223 | 15 | 18 | ||
225 | 16 | # amount of time to wait before testing for replicaset | 19 | # max amount of time to wait before testing for replicaset |
226 | 17 | # status | 20 | # status |
228 | 18 | wait_for_replicaset = 15 | 21 | wait_for_replicaset = 600 |
229 | 19 | 22 | ||
230 | 20 | ######################################################### | 23 | ######################################################### |
231 | 21 | # 3shard cluster configuration | 24 | # 3shard cluster configuration |
232 | @@ -36,9 +39,9 @@ | |||
233 | 36 | raise | 39 | raise |
234 | 37 | 40 | ||
235 | 38 | sentry_dict = { | 41 | sentry_dict = { |
239 | 39 | 'mongodb0-sentry': d.sentry.unit['mongodb/0'], | 42 | 'mongodb0-sentry': d.sentry['mongodb'][0], |
240 | 40 | 'mongodb1-sentry': d.sentry.unit['mongodb/1'], | 43 | 'mongodb1-sentry': d.sentry['mongodb'][1], |
241 | 41 | 'mongodb2-sentry': d.sentry.unit['mongodb/2'], | 44 | 'mongodb2-sentry': d.sentry['mongodb'][2], |
242 | 42 | } | 45 | } |
243 | 43 | 46 | ||
244 | 44 | 47 | ||
245 | @@ -46,9 +49,11 @@ | |||
246 | 46 | # Check presence of MongoDB GUI HEALTH Status | 49 | # Check presence of MongoDB GUI HEALTH Status |
247 | 47 | ############################################################# | 50 | ############################################################# |
248 | 48 | def validate_status_interface(): | 51 | def validate_status_interface(): |
252 | 49 | r = requests.get("http://{}:28017".format( | 52 | pubaddy = d.sentry['mongodb'][0].info['public-address'] |
253 | 50 | d.sentry.unit['mongodb/0'].info['public-address']), | 53 | fmt = "http://{}:28017" |
254 | 51 | verify=False) | 54 | if ":" in pubaddy: |
255 | 55 | fmt = "http://[{}]:28017" | ||
256 | 56 | r = requests.get(fmt.format(pubaddy), verify=False) | ||
257 | 52 | r.raise_for_status | 57 | r.raise_for_status |
258 | 53 | 58 | ||
259 | 54 | 59 | ||
260 | @@ -70,24 +75,47 @@ | |||
261 | 70 | ############################################################# | 75 | ############################################################# |
262 | 71 | def validate_replicaset_setup(): | 76 | def validate_replicaset_setup(): |
263 | 72 | 77 | ||
265 | 73 | time.sleep(wait_for_replicaset) | 78 | d.sentry.wait(seconds) |
266 | 74 | 79 | ||
267 | 75 | unit_status = [] | 80 | unit_status = [] |
268 | 81 | time_between = 10 | ||
269 | 82 | tries = wait_for_replicaset / time_between | ||
270 | 76 | 83 | ||
271 | 77 | for service in sentry_dict: | 84 | for service in sentry_dict: |
274 | 78 | client = MongoClient(sentry_dict[service].info['public-address']) | 85 | addy = sentry_dict[service].info['public-address'] |
275 | 79 | r = client.admin.command('replSetGetStatus') | 86 | if ":" in addy: |
276 | 87 | addy = "[{}]".format(addy) | ||
277 | 88 | while True: | ||
278 | 89 | try: | ||
279 | 90 | client = MongoClient(addy) | ||
280 | 91 | r = client.admin.command('replSetGetStatus') | ||
281 | 92 | break | ||
282 | 93 | except pymongo.errors.OperationFailure as ex: | ||
283 | 94 | sys.stderr.write( | ||
284 | 95 | 'OperationFailure, sleep and retry... to {}: {}\n'. | ||
285 | 96 | format(addy, ex)) | ||
286 | 97 | tb_lines = traceback.format_exception(ex.__class__, | ||
287 | 98 | ex, ex.__traceback__) | ||
288 | 99 | tb_text = ''.join(tb_lines) | ||
289 | 100 | sys.stderr.write(tb_text) | ||
290 | 101 | tries = tries - 1 | ||
291 | 102 | if tries < 0: | ||
292 | 103 | sys.stderr.write('retry limit caught, failing...\n') | ||
293 | 104 | break | ||
294 | 105 | time.sleep(time_between) | ||
295 | 80 | unit_status.append(r['myState']) | 106 | unit_status.append(r['myState']) |
296 | 81 | client.close() | 107 | client.close() |
297 | 82 | 108 | ||
298 | 83 | primaries = Counter(unit_status)[1] | 109 | primaries = Counter(unit_status)[1] |
299 | 84 | if primaries != 1: | 110 | if primaries != 1: |
301 | 85 | message = "Only one PRIMARY unit allowed! Found: %s" % (primaries) | 111 | message = "Only one PRIMARY unit allowed! Found: %s %s" % (primaries, |
302 | 112 | unit_status) | ||
303 | 86 | amulet.raise_status(amulet.FAIL, message) | 113 | amulet.raise_status(amulet.FAIL, message) |
304 | 87 | 114 | ||
305 | 88 | secondrs = Counter(unit_status)[2] | 115 | secondrs = Counter(unit_status)[2] |
306 | 89 | if secondrs != 2: | 116 | if secondrs != 2: |
308 | 90 | message = "Only two SECONDARY units allowed! (Found %s)" % (secondrs) | 117 | message = ("Only two SECONDARY units allowed! (Found %s) %s" % |
309 | 118 | (secondrs, unit_status)) | ||
310 | 91 | amulet.raise_status(amulet.FAIL, message) | 119 | amulet.raise_status(amulet.FAIL, message) |
311 | 92 | 120 | ||
312 | 93 | 121 | ||
313 | @@ -95,11 +123,37 @@ | |||
314 | 95 | # Validate connectivity from $WORLD | 123 | # Validate connectivity from $WORLD |
315 | 96 | ############################################################# | 124 | ############################################################# |
316 | 97 | def validate_world_connectivity(): | 125 | def validate_world_connectivity(): |
322 | 98 | client = MongoClient(d.sentry.unit['mongodb/0'].info['public-address']) | 126 | d.sentry['mongodb'] |
323 | 99 | 127 | ordered_units = sorted(d.sentry['mongodb'], key=lambda u: u.info['unit']) | |
324 | 100 | db = client['test'] | 128 | # We assume minimum unit number is master. |
325 | 101 | # Can we successfully insert? | 129 | addy = ordered_units[0].info['public-address'] |
326 | 102 | insert_id = db.amulet.insert({'assert': True}) | 130 | if ":" in addy: |
327 | 131 | addy = "[{}]".format(addy) | ||
328 | 132 | |||
329 | 133 | time_between = 10 | ||
330 | 134 | tries = wait_for_replicaset / time_between | ||
331 | 135 | insert_id = None | ||
332 | 136 | while True: | ||
333 | 137 | try: | ||
334 | 138 | client = MongoClient(addy) | ||
335 | 139 | db = client.test | ||
336 | 140 | # Can we successfully insert? | ||
337 | 141 | insert_id = db.amulet.insert({'assert': True}) | ||
338 | 142 | break | ||
339 | 143 | except pymongo.errors.AutoReconnect as ex: | ||
340 | 144 | sys.stderr.write( | ||
341 | 145 | 'AutoReconnect error, sleep and retry... to {}: {}\n'. | ||
342 | 146 | format(addy, ex)) | ||
343 | 147 | tb_lines = traceback.format_exception(ex.__class__, | ||
344 | 148 | ex, ex.__traceback__) | ||
345 | 149 | tb_text = ''.join(tb_lines) | ||
346 | 150 | sys.stderr.write(tb_text) | ||
347 | 151 | tries = tries - 1 | ||
348 | 152 | if tries < 0: | ||
349 | 153 | sys.stderr.write('retry limit caught, failing...\n') | ||
350 | 154 | break | ||
351 | 155 | time.sleep(time_between) | ||
352 | 156 | |||
353 | 103 | if insert_id is None: | 157 | if insert_id is None: |
354 | 104 | amulet.raise_status(amulet.FAIL, msg="Failed to insert test data") | 158 | amulet.raise_status(amulet.FAIL, msg="Failed to insert test data") |
355 | 105 | # Can we delete from a shard using the Mongos hub? | 159 | # Can we delete from a shard using the Mongos hub? |
356 | 106 | 160 | ||
357 | === modified file 'tests/04_deploy_with_storage.py' | |||
358 | --- tests/04_deploy_with_storage.py 2015-06-02 05:33:09 +0000 | |||
359 | +++ tests/04_deploy_with_storage.py 2015-12-07 22:34:32 +0000 | |||
360 | @@ -1,7 +1,6 @@ | |||
361 | 1 | #!/usr/bin/env python3 | 1 | #!/usr/bin/env python3 |
362 | 2 | 2 | ||
363 | 3 | import amulet | 3 | import amulet |
364 | 4 | import time | ||
365 | 5 | from pymongo import MongoClient | 4 | from pymongo import MongoClient |
366 | 6 | from collections import Counter | 5 | from collections import Counter |
367 | 7 | 6 | ||
368 | @@ -14,16 +13,17 @@ | |||
369 | 14 | 13 | ||
370 | 15 | # amount of time to wait before testing for replicaset | 14 | # amount of time to wait before testing for replicaset |
371 | 16 | # status | 15 | # status |
373 | 17 | wait_for_replicaset = 30 | 16 | wait_for_replicaset = 60*5 |
374 | 18 | # amount of time to wait for the data relation | 17 | # amount of time to wait for the data relation |
376 | 19 | wait_for_relation = 60*2 | 18 | wait_for_relation = 60*5 |
377 | 20 | 19 | ||
378 | 21 | ######################################################### | 20 | ######################################################### |
379 | 22 | # 3shard cluster configuration | 21 | # 3shard cluster configuration |
380 | 23 | ######################################################### | 22 | ######################################################### |
381 | 24 | d = amulet.Deployment(series='trusty') | 23 | d = amulet.Deployment(series='trusty') |
382 | 25 | 24 | ||
384 | 26 | d.add('mongodb', units=scale, series='trusty') | 25 | d.add('mongodb', units=scale, series='trusty', |
385 | 26 | constraints={'root-disk': '20480M'}) | ||
386 | 27 | d.add('storage', charm='cs:~chris-gondolin/trusty/storage-5', series='trusty') | 27 | d.add('storage', charm='cs:~chris-gondolin/trusty/storage-5', series='trusty') |
387 | 28 | d.configure('storage', {'provider': 'local'}) | 28 | d.configure('storage', {'provider': 'local'}) |
388 | 29 | 29 | ||
389 | @@ -32,16 +32,17 @@ | |||
390 | 32 | # Perform the setup for the deployment. | 32 | # Perform the setup for the deployment. |
391 | 33 | try: | 33 | try: |
392 | 34 | d.setup(seconds) | 34 | d.setup(seconds) |
394 | 35 | d.sentry.wait(seconds) | 35 | d.sentry.wait(wait_for_replicaset) |
395 | 36 | except amulet.helpers.TimeoutError: | 36 | except amulet.helpers.TimeoutError: |
396 | 37 | message = 'The environment did not setup in %d seconds.', seconds | 37 | message = 'The environment did not setup in %d seconds.', seconds |
397 | 38 | amulet.raise_status(amulet.SKIP, msg=message) | 38 | amulet.raise_status(amulet.SKIP, msg=message) |
398 | 39 | except: | 39 | except: |
399 | 40 | raise | 40 | raise |
400 | 41 | 41 | ||
401 | 42 | ordered_units = sorted(d.sentry['mongodb'], key=lambda u: u.info['unit']) | ||
402 | 42 | sentry_dict = { | 43 | sentry_dict = { |
405 | 43 | 'mongodb0-sentry': d.sentry.unit['mongodb/0'], | 44 | 'mongodb0-sentry': ordered_units[0], |
406 | 44 | 'mongodb1-sentry': d.sentry.unit['mongodb/1'], | 45 | 'mongodb1-sentry': ordered_units[1] |
407 | 45 | } | 46 | } |
408 | 46 | 47 | ||
409 | 47 | 48 | ||
410 | @@ -57,12 +58,15 @@ | |||
411 | 57 | ############################################################# | 58 | ############################################################# |
412 | 58 | def validate_replicaset_setup(): | 59 | def validate_replicaset_setup(): |
413 | 59 | 60 | ||
415 | 60 | time.sleep(wait_for_replicaset) | 61 | d.sentry.wait(seconds) |
416 | 61 | 62 | ||
417 | 62 | unit_status = [] | 63 | unit_status = [] |
418 | 63 | 64 | ||
419 | 64 | for service in sentry_dict: | 65 | for service in sentry_dict: |
421 | 65 | client = MongoClient(sentry_dict[service].info['public-address']) | 66 | addy = sentry_dict[service].info['public-address'] |
422 | 67 | if ":" in addy: | ||
423 | 68 | addy = "[{}]".format(addy) | ||
424 | 69 | client = MongoClient(addy) | ||
425 | 66 | r = client.admin.command('replSetGetStatus') | 70 | r = client.admin.command('replSetGetStatus') |
426 | 67 | unit_status.append(r['myState']) | 71 | unit_status.append(r['myState']) |
427 | 68 | client.close() | 72 | client.close() |
428 | @@ -81,7 +85,10 @@ | |||
429 | 81 | validate_status() | 85 | validate_status() |
430 | 82 | validate_replicaset_setup() | 86 | validate_replicaset_setup() |
431 | 83 | print("Adding storage relation, and sleeping for 2 min.") | 87 | print("Adding storage relation, and sleeping for 2 min.") |
434 | 84 | d.relate('mongodb:data', 'storage:data') | 88 | try: |
435 | 85 | time.sleep(wait_for_relation) | 89 | d.relate('mongodb:data', 'storage:data') |
436 | 90 | except OSError as e: | ||
437 | 91 | print("ignoring error:{}", e) | ||
438 | 92 | d.sentry.wait(wait_for_relation) | ||
439 | 86 | validate_status() | 93 | validate_status() |
440 | 87 | validate_replicaset_setup() | 94 | validate_replicaset_setup() |
441 | 88 | 95 | ||
442 | === modified file 'tests/50_relate_ceilometer_test.py' | |||
443 | --- tests/50_relate_ceilometer_test.py 2014-12-09 23:36:06 +0000 | |||
444 | +++ tests/50_relate_ceilometer_test.py 2015-12-07 22:34:32 +0000 | |||
445 | @@ -32,10 +32,13 @@ | |||
446 | 32 | getattr(self, test)() | 32 | getattr(self, test)() |
447 | 33 | 33 | ||
448 | 34 | def test_mongo_relation(self): | 34 | def test_mongo_relation(self): |
451 | 35 | unit = self.deploy.sentry.unit['ceilometer/0'] | 35 | unit = self.deploy.sentry['ceilometer'][0] |
452 | 36 | mongo = self.deploy.sentry.unit['mongodb/0'].info['public-address'] | 36 | mongo = self.deploy.sentry['mongodb'][0].info['public-address'] |
453 | 37 | mongo_reladdr = self.deploy.sentry['mongodb'][0].relation( | ||
454 | 38 | 'database', 'ceilometer:shared-db') | ||
455 | 37 | cont = unit.file_contents('/etc/ceilometer/ceilometer.conf') | 39 | cont = unit.file_contents('/etc/ceilometer/ceilometer.conf') |
457 | 38 | if mongo not in cont: | 40 | if (mongo not in cont and mongo_reladdr.get( |
458 | 41 | 'hostname', 'I SURE HOPE NOT') not in cont): | ||
459 | 39 | amulet.raise_status(amulet.FAIL, "Unable to verify ceilometer cfg") | 42 | amulet.raise_status(amulet.FAIL, "Unable to verify ceilometer cfg") |
460 | 40 | 43 | ||
461 | 41 | if __name__ == '__main__': | 44 | if __name__ == '__main__': |
charm_lint_check #13430 mongodb for evarlast mp277191
LINT OK: passed
Build: http:// 10.245. 162.77: 8080/job/ charm_lint_ check/13430/