Merge lp:~evarlast/charms/trusty/mongodb/fix-dump-actions into lp:charms/trusty/mongodb

Proposed by Jay R. Wren
Status: Merged
Merged at revision: 80
Proposed branch: lp:~evarlast/charms/trusty/mongodb/fix-dump-actions
Merge into: lp:charms/trusty/mongodb
Diff against target: 461 lines (+156/-59)
9 files modified
actions/backup.py (+5/-4)
config.yaml (+1/-1)
hooks/hooks.py (+1/-0)
metadata.yaml (+1/-1)
tests/01_deploy_single.py (+4/-1)
tests/02_deploy_shard_test.py (+47/-19)
tests/03_deploy_replicaset.py (+73/-19)
tests/04_deploy_with_storage.py (+18/-11)
tests/50_relate_ceilometer_test.py (+6/-3)
To merge this branch: bzr merge lp:~evarlast/charms/trusty/mongodb/fix-dump-actions
Reviewer Review Type Date Requested Status
Review Queue (community) automated testing Approve
Matt Bruzek (community) Approve
Jay R. Wren (community) Needs Resubmitting
Ryan Beisner (community) Needs Fixing
Matthew Williams (community) Approve
Tim Van Steenburgh (community) Needs Fixing
Review via email: mp+277191@code.launchpad.net

Description of the change

Dump wasn't working. Now it works and if args has spaces, they can be escaped with quotes.

e.g.

$ juju action do mongodb/0 dump args='-q "{a:1, b:2}"'
Action queued with id: 7b627526-8b0c-4f2e-8494-bdb13dabe23d
$ juju action fetch 7b
results:
  command: mongodump -q "{a:1, b:2}"

To post a comment you must log in.
Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #13430 mongodb for evarlast mp277191
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/13430/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_unit_test #12564 mongodb for evarlast mp277191
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/12564/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_amulet_test #7839 mongodb for evarlast mp277191
    AMULET OK: passed

Build: http://10.245.162.77:8080/job/charm_amulet_test/7839/

Revision history for this message
Tim Van Steenburgh (tvansteenburgh) wrote :

Charm CI tests failed due to hardcoded unit numbers in the tests. Jenkins log: http://juju-ci.vapour.ws:8080/job/charm-bundle-test-lxc/1432/console

To fix, replace occurrences of `d.sentry.unit['mongodb/0']` with `d.sentry['mongodb'][0]` in the test files (tests/*).

review: Needs Fixing
80. By Jay R. Wren

fix hardcoded unit numbers in the tests

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_unit_test #12615 mongodb for evarlast mp277191
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/12615/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #13484 mongodb for evarlast mp277191
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/13484/

Revision history for this message
Matthew Williams (mattyw) wrote :

LGTM

review: Approve
Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_amulet_test #7845 mongodb for evarlast mp277191
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/13240036/
Build: http://10.245.162.77:8080/job/charm_amulet_test/7845/

Revision history for this message
Review Queue (review-queue) wrote :

This item has failed automated testing! Results available here http://juju-ci.vapour.ws:8080/job/charm-bundle-test-aws/1421/

review: Needs Fixing (automated testing)
81. By Jay R. Wren

use sentry[service][N] instead of sentry.unit[unit]

82. By Jay R. Wren

categories is now tags.

config key now defaults to ""

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #13701 mongodb for evarlast mp277191
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/13701/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_unit_test #12771 mongodb for evarlast mp277191
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/12771/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_amulet_test #7858 mongodb for evarlast mp277191
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/13249011/
Build: http://10.245.162.77:8080/job/charm_amulet_test/7858/

83. By Jay R. Wren

some ipv6 support

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #13928 mongodb for evarlast mp277191
    LINT FAIL: lint-test failed

LINT Results (max last 2 lines):
make: *** [lint] Error 1
ERROR:root:Make target returned non-zero.

Full lint test output: http://paste.ubuntu.com/13318610/
Build: http://10.245.162.77:8080/job/charm_lint_check/13928/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_unit_test #12984 mongodb for evarlast mp277191
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/12984/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_amulet_test #7911 mongodb for evarlast mp277191
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/13318703/
Build: http://10.245.162.77:8080/job/charm_amulet_test/7911/

84. By Jay R. Wren

fix ipv6 url

85. By Jay R. Wren

fix ipv6 url

86. By Jay R. Wren

fix lint

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #14010 mongodb for evarlast mp277191
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/14010/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_unit_test #13061 mongodb for evarlast mp277191
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/13061/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_amulet_test #7939 mongodb for evarlast mp277191
    AMULET OK: passed

Build: http://10.245.162.77:8080/job/charm_amulet_test/7939/

87. By Jay R. Wren

fix broken amulet tests

we do not know which is master, cannot assume it is /0, so try 0 and if it fails try /1.

enable ipv6 in mongodb. We cannot know which provider is being used and if the public-address may be ipv6, so we must enable ipv6

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #14066 mongodb for evarlast mp277191
    LINT FAIL: lint-test failed

LINT Results (max last 2 lines):
make: *** [lint] Error 1
ERROR:root:Make target returned non-zero.

Full lint test output: http://paste.ubuntu.com/13340748/
Build: http://10.245.162.77:8080/job/charm_lint_check/14066/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_unit_test #13112 mongodb for evarlast mp277191
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/13112/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_amulet_test #7941 mongodb for evarlast mp277191
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/13341016/
Build: http://10.245.162.77:8080/job/charm_amulet_test/7941/

88. By Jay R. Wren

use sentry.wait() instead of time.sleep - much faster.

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #14136 mongodb for evarlast mp277191
    LINT FAIL: lint-test failed

LINT Results (max last 2 lines):
make: *** [lint] Error 1
ERROR:root:Make target returned non-zero.

Full lint test output: http://paste.ubuntu.com/13367320/
Build: http://10.245.162.77:8080/job/charm_lint_check/14136/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_unit_test #13176 mongodb for evarlast mp277191
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/13176/

89. By Jay R. Wren

fix lint

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #14137 mongodb for evarlast mp277191
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/14137/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_unit_test #13177 mongodb for evarlast mp277191
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/13177/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_amulet_test #7957 mongodb for evarlast mp277191
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/13368568/
Build: http://10.245.162.77:8080/job/charm_amulet_test/7957/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_amulet_test #7956 mongodb for evarlast mp277191
    AMULET OK: passed

Build: http://10.245.162.77:8080/job/charm_amulet_test/7956/

90. By Jay R. Wren

Use hostname used in relation.

Can't assume relation hostname is public-address. It isn't in many environments.

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #14138 mongodb for evarlast mp277191
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/14138/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_unit_test #13178 mongodb for evarlast mp277191
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/13178/

91. By Jay R. Wren

use sentry.wait not time.sleep

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_amulet_test #7958 mongodb for evarlast mp277191
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/13370168/
Build: http://10.245.162.77:8080/job/charm_amulet_test/7958/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_unit_test #13180 mongodb for evarlast mp277191
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/13180/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #14140 mongodb for evarlast mp277191
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/14140/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_amulet_test #7960 mongodb for evarlast mp277191
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/13370662/
Build: http://10.245.162.77:8080/job/charm_amulet_test/7960/

92. By Jay R. Wren

wait longer for replset

Don't retry on unit 1 when unit 0 is not primary.
Additional logging on fail.

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #14141 mongodb for evarlast mp277191
    LINT FAIL: lint-test failed

LINT Results (max last 2 lines):
make: *** [lint] Error 1
ERROR:root:Make target returned non-zero.

Full lint test output: http://paste.ubuntu.com/13372704/
Build: http://10.245.162.77:8080/job/charm_lint_check/14141/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_unit_test #13181 mongodb for evarlast mp277191
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/13181/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_amulet_test #7961 mongodb for evarlast mp277191
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/13373290/
Build: http://10.245.162.77:8080/job/charm_amulet_test/7961/

93. By Jay R. Wren

fix lint

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_unit_test #13182 mongodb for evarlast mp277191
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/13182/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #14142 mongodb for evarlast mp277191
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/14142/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_amulet_test #7962 mongodb for evarlast mp277191
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/13374149/
Build: http://10.245.162.77:8080/job/charm_amulet_test/7962/

94. By Jay R. Wren

increase wait time for CI

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #14253 mongodb for evarlast mp277191
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/14253/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_unit_test #13284 mongodb for evarlast mp277191
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/13284/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_amulet_test #8016 mongodb for evarlast mp277191
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/13479215/
Build: http://10.245.162.77:8080/job/charm_amulet_test/8016/

95. By Jay R. Wren

increase wait time for CI

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #14254 mongodb for evarlast mp277191
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/14254/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_unit_test #13285 mongodb for evarlast mp277191
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/13285/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_amulet_test #8017 mongodb for evarlast mp277191
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/13479787/
Build: http://10.245.162.77:8080/job/charm_amulet_test/8017/

Revision history for this message
Ryan Beisner (1chb1n) wrote :

Thank you for your work on this.

While I realize that the test race wasn't introduced by your proposed changes, please see:
https://bugs.launchpad.net/charms/+source/mongodb/+bug/1518468

It would be awesome to resolve that test issue along with this, as it's definitely presenting a pain point.

review: Needs Fixing
96. By Jay R. Wren

retry every 10s for replset validation

Revision history for this message
Jay R. Wren (evarlast) wrote :
Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #14310 mongodb for evarlast mp277191
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/14310/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_unit_test #13338 mongodb for evarlast mp277191
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/13338/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_amulet_test #8028 mongodb for evarlast mp277191
    AMULET OK: passed

Build: http://10.245.162.77:8080/job/charm_amulet_test/8028/

Revision history for this message
Review Queue (review-queue) wrote :

This item has failed automated testing! Results available here http://juju-ci.vapour.ws:8080/job/charm-bundle-test-lxc/1534/

review: Needs Fixing (automated testing)
Revision history for this message
Review Queue (review-queue) wrote :

This item has failed automated testing! Results available here http://juju-ci.vapour.ws:8080/job/charm-bundle-test-aws/1517/

review: Needs Fixing (automated testing)
Revision history for this message
Review Queue (review-queue) wrote :

This item has failed automated testing! Results available here http://juju-ci.vapour.ws:8080/job/charm-bundle-test-lxc/1536/

review: Needs Fixing (automated testing)
Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #14635 mongodb for evarlast mp277191
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/14635/

Revision history for this message
Review Queue (review-queue) wrote :

This item has failed automated testing! Results available here http://juju-ci.vapour.ws:8080/job/charm-bundle-test-lxc/1615/

review: Needs Fixing (automated testing)
97. By Jay R. Wren

retry replSetGetStatus and use minimum unit number for master

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #14756 mongodb for evarlast mp277191
    LINT FAIL: lint-test failed

LINT Results (max last 2 lines):
make: *** [lint] Error 1
ERROR:root:Make target returned non-zero.

Full lint test output: http://paste.ubuntu.com/13631511/
Build: http://10.245.162.77:8080/job/charm_lint_check/14756/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_unit_test #13759 mongodb for evarlast mp277191
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/13759/

98. By Jay R. Wren

fix lint

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #14757 mongodb for evarlast mp277191
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/14757/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_unit_test #13761 mongodb for evarlast mp277191
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/13761/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_amulet_test #8078 mongodb for evarlast mp277191
    AMULET OK: passed

Build: http://10.245.162.77:8080/job/charm_amulet_test/8078/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_amulet_test #8079 mongodb for evarlast mp277191
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [functional_test] Error 124
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/13632412/
Build: http://10.245.162.77:8080/job/charm_amulet_test/8079/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_amulet_test #8081 mongodb for evarlast mp277191
    AMULET OK: passed

Build: http://10.245.162.77:8080/job/charm_amulet_test/8081/

99. By Jay R. Wren

retry shard status interface test
sort units and use them based on order in 04_deploy_with_storage test

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #14813 mongodb for evarlast mp277191
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/14813/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_unit_test #13813 mongodb for evarlast mp277191
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/13813/

Revision history for this message
Review Queue (review-queue) wrote :

This item has failed automated testing! Results available here http://juju-ci.vapour.ws:8080/job/charm-bundle-test-lxc/1652/

review: Needs Fixing (automated testing)
Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_amulet_test #8134 mongodb for evarlast mp277191
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/13658235/
Build: http://10.245.162.77:8080/job/charm_amulet_test/8134/

Revision history for this message
Review Queue (review-queue) wrote :

This item has failed automated testing! Results available here http://juju-ci.vapour.ws:8080/job/charm-bundle-test-lxc/1654/

review: Needs Fixing (automated testing)
Revision history for this message
Review Queue (review-queue) wrote :

This item has failed automated testing! Results available here http://juju-ci.vapour.ws:8080/job/charm-bundle-test-aws/1634/

review: Needs Fixing (automated testing)
Revision history for this message
Review Queue (review-queue) wrote :

This item has failed automated testing! Results available here http://juju-ci.vapour.ws:8080/job/charm-bundle-test-lxc/1666/

review: Needs Fixing (automated testing)
Revision history for this message
Review Queue (review-queue) wrote :

This item has failed automated testing! Results available here http://juju-ci.vapour.ws:8080/job/charm-bundle-test-aws/1646/

review: Needs Fixing (automated testing)
100. By Jay R. Wren

use constraints for storage test to prevent filling root volume

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #15041 mongodb for evarlast mp277191
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/15041/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_unit_test #14029 mongodb for evarlast mp277191
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/14029/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_amulet_test #8205 mongodb for evarlast mp277191
    AMULET OK: passed

Build: http://10.245.162.77:8080/job/charm_amulet_test/8205/

Revision history for this message
Review Queue (review-queue) wrote :

This item has failed automated testing! Results available here http://juju-ci.vapour.ws:8080/job/charm-bundle-test-lxc/1716/

review: Needs Fixing (automated testing)
Revision history for this message
Review Queue (review-queue) wrote :

This item has failed automated testing! Results available here http://juju-ci.vapour.ws:8080/job/charm-bundle-test-aws/1694/

review: Needs Fixing (automated testing)
Revision history for this message
Review Queue (review-queue) wrote :

This item has failed automated testing! Results available here http://juju-ci.vapour.ws:8080/job/charm-bundle-test-lxc/1725/

review: Needs Fixing (automated testing)
Revision history for this message
Review Queue (review-queue) wrote :

This item has failed automated testing! Results available here http://juju-ci.vapour.ws:8080/job/charm-bundle-test-aws/1702/

review: Needs Fixing (automated testing)
Revision history for this message
Jay R. Wren (evarlast) wrote :

The only fail remaining is from a known bug, which I am not attempting to fix.

From the README:
Deploying three or more units at start can sometimes lead to unexpected race-conditions so it's best to start with two nodes.

http://review.juju.solutions/review/2357 Shows the AWS test passing and the LXC test fails because of the above known bug.

Please consider this for merging.

review: Needs Resubmitting
Revision history for this message
Matt Bruzek (mbruzek) wrote :

Hello Jay,

While reviewing the queue today and noticed more than one mongodb charm merge proposals. Since we review the oldest first this proposal did not get reviewed first.

As Ryan points out in a bug 1518468 ( https://bugs.launchpad.net/charms/+source/mongodb/+bug/1518468 ) using sleep funtions are not a deterministic way to build tests that run on multiple environments as the times can be very long on some (azure) and lower on others (aws). This merge proposal changes some sleep() calls to sentry.wait() but introduces several more time.sleep() commands and the latter seems to be the wrong way to go.

## 03_deploy_replicaset.py
The new code contains the comment “# We assume minimum unit number is master.” There should be a better way to get the master address in the test. Again you may be able to run commands on the units to determine their master relationship, or use a mongo python library to determine the master reliably.

The tests passed for me on AWS and I am going to merge this code. I would encourage you to change the test code to detect the state of the mongodb deployment from reliable sources that do not involve sleeping for arbitrary amounts of time. Such as running a command on the unit to tell when it is ready, or using `juju status-history unit/0` to determine ready states.

review: Approve
Revision history for this message
Review Queue (review-queue) wrote :

This item has failed automated testing! Results available here http://juju-ci.vapour.ws:8080/job/charm-bundle-test-lxc/1754/

review: Needs Fixing (automated testing)
Revision history for this message
Review Queue (review-queue) wrote :

The results (PASS) are in and available here: http://juju-ci.vapour.ws:8080/job/charm-bundle-test-aws/1731/

review: Approve (automated testing)

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'actions/backup.py'
--- actions/backup.py 2015-10-08 08:12:35 +0000
+++ actions/backup.py 2015-12-07 22:34:32 +0000
@@ -1,3 +1,4 @@
1import shlex
1import subprocess2import subprocess
2import os3import os
3try:4try:
@@ -14,7 +15,7 @@
1415
15def execute(command, current_working_directory):16def execute(command, current_working_directory):
16 return subprocess.check_output(17 return subprocess.check_output(
17 " ".join(command),18 shlex.split(command),
18 cwd=current_working_directory,19 cwd=current_working_directory,
19 )20 )
2021
@@ -38,7 +39,7 @@
38 pass # Ignoring, the directory already exists39 pass # Ignoring, the directory already exists
39 except Exception, e:40 except Exception, e:
40 action_set({"directory creation exception": e})41 action_set({"directory creation exception": e})
41 action_fail(e)42 action_fail(str(e))
42 return43 return
4344
44 command = cmd + " " + args45 command = cmd + " " + args
@@ -50,7 +51,7 @@
50 except subprocess.CalledProcessError, e:51 except subprocess.CalledProcessError, e:
51 action_set({"error_code": e.returncode,52 action_set({"error_code": e.returncode,
52 "exception": e, "output": e.output})53 "exception": e, "output": e.output})
53 action_fail(e)54 action_fail(str(e))
54 except Exception, e:55 except Exception, e:
55 action_set({"exception": e})56 action_set({"exception": e})
56 action_fail(e)57 action_fail(str(e))
5758
=== modified file 'config.yaml'
--- config.yaml 2015-02-25 00:24:56 +0000
+++ config.yaml 2015-12-07 22:34:32 +0000
@@ -211,7 +211,7 @@
211 option.211 option.
212 key:212 key:
213 type: string213 type: string
214 default:214 default: ""
215 description: >215 description: >
216 Key ID to import to the apt keyring to support use with arbitary source216 Key ID to import to the apt keyring to support use with arbitary source
217 configuration from outside of Launchpad archives or PPA's.217 configuration from outside of Launchpad archives or PPA's.
218218
=== modified file 'hooks/hooks.py'
--- hooks/hooks.py 2015-09-09 12:59:18 +0000
+++ hooks/hooks.py 2015-12-07 22:34:32 +0000
@@ -254,6 +254,7 @@
254 subprocess.call(['chown', '-R', 'mongodb:mongodb', config_data['dbpath']])254 subprocess.call(['chown', '-R', 'mongodb:mongodb', config_data['dbpath']])
255 config.append("dbpath=%s" % config_data['dbpath'])255 config.append("dbpath=%s" % config_data['dbpath'])
256 config.append("")256 config.append("")
257 config.append("ipv6=true")
257258
258 # logpath259 # logpath
259 # Create the directory if not there already260 # Create the directory if not there already
260261
=== modified file 'metadata.yaml'
--- metadata.yaml 2015-05-19 13:21:16 +0000
+++ metadata.yaml 2015-12-07 22:34:32 +0000
@@ -18,7 +18,7 @@
18 - Auto-sharding for cloud-level scalability (Q209)18 - Auto-sharding for cloud-level scalability (Q209)
19 High performance, scalability, and reasonable depth of functionality19 High performance, scalability, and reasonable depth of functionality
20 are the goals for the project.20 are the goals for the project.
21categories:21tags:
22 - databases22 - databases
23provides:23provides:
24 nrpe-external-master:24 nrpe-external-master:
2525
=== modified file 'tests/01_deploy_single.py'
--- tests/01_deploy_single.py 2014-12-10 15:36:14 +0000
+++ tests/01_deploy_single.py 2015-12-07 22:34:32 +0000
@@ -24,7 +24,10 @@
24# Validate connectivity from $WORLD24# Validate connectivity from $WORLD
25#############################################################25#############################################################
26def validate_world_connectivity():26def validate_world_connectivity():
27 client = MongoClient(d.sentry.unit['mongodb/0'].info['public-address'])27 addy = d.sentry['mongodb'][0].info['public-address']
28 if ":" in addy:
29 addy = "[{}]".format(addy)
30 client = MongoClient(addy)
2831
29 db = client['test']32 db = client['test']
30 # Can we successfully insert?33 # Can we successfully insert?
3134
=== modified file 'tests/02_deploy_shard_test.py'
--- tests/02_deploy_shard_test.py 2014-12-09 23:45:14 +0000
+++ tests/02_deploy_shard_test.py 2015-12-07 22:34:32 +0000
@@ -2,6 +2,9 @@
22
3import amulet3import amulet
4import requests4import requests
5import sys
6import time
7import traceback
5from pymongo import MongoClient8from pymongo import MongoClient
69
7#########################################################10#########################################################
@@ -49,10 +52,10 @@
49 raise52 raise
5053
51sentry_dict = {54sentry_dict = {
52 'config-sentry': d.sentry.unit['configsvr/0'],55 'config-sentry': d.sentry['configsvr'][0],
53 'mongos-sentry': d.sentry.unit['mongos/0'],56 'mongos-sentry': d.sentry['mongos'][0],
54 'shard1-sentry': d.sentry.unit['shard1/0'],57 'shard1-sentry': d.sentry['shard1'][0],
55 'shard2-sentry': d.sentry.unit['shard2/0']58 'shard2-sentry': d.sentry['shard2'][0]
56}59}
5760
5861
@@ -60,10 +63,27 @@
60# Check presence of MongoDB GUI HEALTH Status63# Check presence of MongoDB GUI HEALTH Status
61#############################################################64#############################################################
62def validate_status_interface():65def validate_status_interface():
63 r = requests.get("http://{}:28017".format(66 pubaddy = sentry_dict['config-sentry'].info['public-address']
64 sentry_dict['config-sentry'].info['public-address']),67 fmt = "http://{}:28017"
65 verify=False)68 if ":" in pubaddy:
66 r.raise_for_status69 fmt = "http://[{}]:28017"
70 time_between = 10
71 tries = seconds / time_between
72 try:
73 r = requests.get(fmt.format(pubaddy), verify=False)
74 r.raise_for_status()
75 except requests.exception.ConnectionError as ex:
76 sys.stderr.write(
77 'Connection error, sleep and retry... to {}: {}\n'.
78 format(pubaddy, ex))
79 tb_lines = traceback.format_exception(ex.__class__,
80 ex, ex.__traceback__)
81 tb_text = ''.join(tb_lines)
82 sys.stderr.write(tb_text)
83 tries = tries - 1
84 if tries < 0:
85 sys.stderr.write('retry limit caught, failing...\n')
86 time.sleep(time_between)
6787
6888
69#############################################################89#############################################################
@@ -83,7 +103,10 @@
83# Validate connectivity from $WORLD103# Validate connectivity from $WORLD
84#############################################################104#############################################################
85def validate_world_connectivity():105def validate_world_connectivity():
86 client = MongoClient(d.sentry.unit['mongos/0'].info['public-address'])106 pubaddy = d.sentry['mongos'][0].info['public-address']
107 if ":" in pubaddy:
108 pubaddy = "[{}]".format(pubaddy)
109 client = MongoClient(pubaddy)
87110
88 db = client['test']111 db = client['test']
89 # Can we successfully insert?112 # Can we successfully insert?
@@ -101,23 +124,28 @@
101#############################################################124#############################################################
102# broken pending 1273312125# broken pending 1273312
103def validate_relationships():126def validate_relationships():
104 d.sentry.unit['configsvr/0'].relation('configsvr', 'mongos:mongos-cfg')127 d.sentry['configsvr'][0].relation('configsvr', 'mongos:mongos-cfg')
105 d.sentry.unit['shard1/0'].relation('database', 'mongos:mongos')128 d.sentry['shard1'][0].relation('database', 'mongos:mongos')
106 d.sentry.unit['shard2/0'].relation('database', 'mongos:mongos')129 d.sentry['shard2'][0].relation('database', 'mongos:mongos')
107 print(d.sentry.unit['shard1/0'].relation('database', 'mongos:mongos'))130 print(d.sentry['shard1'][0].relation('database', 'mongos:mongos'))
108131
109132
110def validate_manual_connection():133def validate_manual_connection():
111 output, code = d.sentry.unit['shard1/0'].run("mongo {}".format(134 fmt = "mongo {}"
112 d.sentry.unit['mongos/0'].info['public-address']))135 addy = d.sentry['mongos'][0].info['public-address']
136 if ":" in addy:
137 fmt = "mongo --ipv6 {}:27017"
138 jujuruncmd = fmt.format(addy)
139 output, code = d.sentry['shard1'][0].run(jujuruncmd)
113 if code != 0:140 if code != 0:
114 message = "Manual Connection failed for unit shard1"141 message = ("Manual Connection failed for unit shard1:{} code:{} cmd:{}"
142 .format(output, code, jujuruncmd))
115 amulet.raise_status(amulet.SKIP, msg=message)143 amulet.raise_status(amulet.SKIP, msg=message)
116144
117 output, code = d.sentry.unit['shard2/0'].run("mongo {}".format(145 output, code = d.sentry['shard2'][0].run(jujuruncmd)
118 d.sentry.unit['mongos/0'].info['public-address']))
119 if code != 0:146 if code != 0:
120 message = "Manual Connection failed for unit shard2"147 message = ("Manual Connection failed for unit shard2:{} code:{} cmd:{}"
148 .format(output, code, jujuruncmd))
121 amulet.raise_status(amulet.SKIP, msg=message)149 amulet.raise_status(amulet.SKIP, msg=message)
122150
123151
124152
=== modified file 'tests/03_deploy_replicaset.py'
--- tests/03_deploy_replicaset.py 2015-02-25 01:49:08 +0000
+++ tests/03_deploy_replicaset.py 2015-12-07 22:34:32 +0000
@@ -1,8 +1,11 @@
1#!/usr/bin/env python31#!/usr/bin/env python3
22
3import amulet3import amulet
4import sys
5import time
6import traceback
4import requests7import requests
5import time8import pymongo
6from pymongo import MongoClient9from pymongo import MongoClient
7from collections import Counter10from collections import Counter
811
@@ -13,9 +16,9 @@
13scale = 316scale = 3
14seconds = 180017seconds = 1800
1518
16# amount of time to wait before testing for replicaset19# max amount of time to wait before testing for replicaset
17# status20# status
18wait_for_replicaset = 1521wait_for_replicaset = 600
1922
20#########################################################23#########################################################
21# 3shard cluster configuration24# 3shard cluster configuration
@@ -36,9 +39,9 @@
36 raise39 raise
3740
38sentry_dict = {41sentry_dict = {
39 'mongodb0-sentry': d.sentry.unit['mongodb/0'],42 'mongodb0-sentry': d.sentry['mongodb'][0],
40 'mongodb1-sentry': d.sentry.unit['mongodb/1'],43 'mongodb1-sentry': d.sentry['mongodb'][1],
41 'mongodb2-sentry': d.sentry.unit['mongodb/2'],44 'mongodb2-sentry': d.sentry['mongodb'][2],
42}45}
4346
4447
@@ -46,9 +49,11 @@
46# Check presence of MongoDB GUI HEALTH Status49# Check presence of MongoDB GUI HEALTH Status
47#############################################################50#############################################################
48def validate_status_interface():51def validate_status_interface():
49 r = requests.get("http://{}:28017".format(52 pubaddy = d.sentry['mongodb'][0].info['public-address']
50 d.sentry.unit['mongodb/0'].info['public-address']),53 fmt = "http://{}:28017"
51 verify=False)54 if ":" in pubaddy:
55 fmt = "http://[{}]:28017"
56 r = requests.get(fmt.format(pubaddy), verify=False)
52 r.raise_for_status57 r.raise_for_status
5358
5459
@@ -70,24 +75,47 @@
70#############################################################75#############################################################
71def validate_replicaset_setup():76def validate_replicaset_setup():
7277
73 time.sleep(wait_for_replicaset)78 d.sentry.wait(seconds)
7479
75 unit_status = []80 unit_status = []
81 time_between = 10
82 tries = wait_for_replicaset / time_between
7683
77 for service in sentry_dict:84 for service in sentry_dict:
78 client = MongoClient(sentry_dict[service].info['public-address'])85 addy = sentry_dict[service].info['public-address']
79 r = client.admin.command('replSetGetStatus')86 if ":" in addy:
87 addy = "[{}]".format(addy)
88 while True:
89 try:
90 client = MongoClient(addy)
91 r = client.admin.command('replSetGetStatus')
92 break
93 except pymongo.errors.OperationFailure as ex:
94 sys.stderr.write(
95 'OperationFailure, sleep and retry... to {}: {}\n'.
96 format(addy, ex))
97 tb_lines = traceback.format_exception(ex.__class__,
98 ex, ex.__traceback__)
99 tb_text = ''.join(tb_lines)
100 sys.stderr.write(tb_text)
101 tries = tries - 1
102 if tries < 0:
103 sys.stderr.write('retry limit caught, failing...\n')
104 break
105 time.sleep(time_between)
80 unit_status.append(r['myState'])106 unit_status.append(r['myState'])
81 client.close()107 client.close()
82108
83 primaries = Counter(unit_status)[1]109 primaries = Counter(unit_status)[1]
84 if primaries != 1:110 if primaries != 1:
85 message = "Only one PRIMARY unit allowed! Found: %s" % (primaries)111 message = "Only one PRIMARY unit allowed! Found: %s %s" % (primaries,
112 unit_status)
86 amulet.raise_status(amulet.FAIL, message)113 amulet.raise_status(amulet.FAIL, message)
87114
88 secondrs = Counter(unit_status)[2]115 secondrs = Counter(unit_status)[2]
89 if secondrs != 2:116 if secondrs != 2:
90 message = "Only two SECONDARY units allowed! (Found %s)" % (secondrs)117 message = ("Only two SECONDARY units allowed! (Found %s) %s" %
118 (secondrs, unit_status))
91 amulet.raise_status(amulet.FAIL, message)119 amulet.raise_status(amulet.FAIL, message)
92120
93121
@@ -95,11 +123,37 @@
95# Validate connectivity from $WORLD123# Validate connectivity from $WORLD
96#############################################################124#############################################################
97def validate_world_connectivity():125def validate_world_connectivity():
98 client = MongoClient(d.sentry.unit['mongodb/0'].info['public-address'])126 d.sentry['mongodb']
99127 ordered_units = sorted(d.sentry['mongodb'], key=lambda u: u.info['unit'])
100 db = client['test']128 # We assume minimum unit number is master.
101 # Can we successfully insert?129 addy = ordered_units[0].info['public-address']
102 insert_id = db.amulet.insert({'assert': True})130 if ":" in addy:
131 addy = "[{}]".format(addy)
132
133 time_between = 10
134 tries = wait_for_replicaset / time_between
135 insert_id = None
136 while True:
137 try:
138 client = MongoClient(addy)
139 db = client.test
140 # Can we successfully insert?
141 insert_id = db.amulet.insert({'assert': True})
142 break
143 except pymongo.errors.AutoReconnect as ex:
144 sys.stderr.write(
145 'AutoReconnect error, sleep and retry... to {}: {}\n'.
146 format(addy, ex))
147 tb_lines = traceback.format_exception(ex.__class__,
148 ex, ex.__traceback__)
149 tb_text = ''.join(tb_lines)
150 sys.stderr.write(tb_text)
151 tries = tries - 1
152 if tries < 0:
153 sys.stderr.write('retry limit caught, failing...\n')
154 break
155 time.sleep(time_between)
156
103 if insert_id is None:157 if insert_id is None:
104 amulet.raise_status(amulet.FAIL, msg="Failed to insert test data")158 amulet.raise_status(amulet.FAIL, msg="Failed to insert test data")
105 # Can we delete from a shard using the Mongos hub?159 # Can we delete from a shard using the Mongos hub?
106160
=== modified file 'tests/04_deploy_with_storage.py'
--- tests/04_deploy_with_storage.py 2015-06-02 05:33:09 +0000
+++ tests/04_deploy_with_storage.py 2015-12-07 22:34:32 +0000
@@ -1,7 +1,6 @@
1#!/usr/bin/env python31#!/usr/bin/env python3
22
3import amulet3import amulet
4import time
5from pymongo import MongoClient4from pymongo import MongoClient
6from collections import Counter5from collections import Counter
76
@@ -14,16 +13,17 @@
1413
15# amount of time to wait before testing for replicaset14# amount of time to wait before testing for replicaset
16# status15# status
17wait_for_replicaset = 3016wait_for_replicaset = 60*5
18# amount of time to wait for the data relation17# amount of time to wait for the data relation
19wait_for_relation = 60*218wait_for_relation = 60*5
2019
21#########################################################20#########################################################
22# 3shard cluster configuration21# 3shard cluster configuration
23#########################################################22#########################################################
24d = amulet.Deployment(series='trusty')23d = amulet.Deployment(series='trusty')
2524
26d.add('mongodb', units=scale, series='trusty')25d.add('mongodb', units=scale, series='trusty',
26 constraints={'root-disk': '20480M'})
27d.add('storage', charm='cs:~chris-gondolin/trusty/storage-5', series='trusty')27d.add('storage', charm='cs:~chris-gondolin/trusty/storage-5', series='trusty')
28d.configure('storage', {'provider': 'local'})28d.configure('storage', {'provider': 'local'})
2929
@@ -32,16 +32,17 @@
32# Perform the setup for the deployment.32# Perform the setup for the deployment.
33try:33try:
34 d.setup(seconds)34 d.setup(seconds)
35 d.sentry.wait(seconds)35 d.sentry.wait(wait_for_replicaset)
36except amulet.helpers.TimeoutError:36except amulet.helpers.TimeoutError:
37 message = 'The environment did not setup in %d seconds.', seconds37 message = 'The environment did not setup in %d seconds.', seconds
38 amulet.raise_status(amulet.SKIP, msg=message)38 amulet.raise_status(amulet.SKIP, msg=message)
39except:39except:
40 raise40 raise
4141
42ordered_units = sorted(d.sentry['mongodb'], key=lambda u: u.info['unit'])
42sentry_dict = {43sentry_dict = {
43 'mongodb0-sentry': d.sentry.unit['mongodb/0'],44 'mongodb0-sentry': ordered_units[0],
44 'mongodb1-sentry': d.sentry.unit['mongodb/1'],45 'mongodb1-sentry': ordered_units[1]
45}46}
4647
4748
@@ -57,12 +58,15 @@
57#############################################################58#############################################################
58def validate_replicaset_setup():59def validate_replicaset_setup():
5960
60 time.sleep(wait_for_replicaset)61 d.sentry.wait(seconds)
6162
62 unit_status = []63 unit_status = []
6364
64 for service in sentry_dict:65 for service in sentry_dict:
65 client = MongoClient(sentry_dict[service].info['public-address'])66 addy = sentry_dict[service].info['public-address']
67 if ":" in addy:
68 addy = "[{}]".format(addy)
69 client = MongoClient(addy)
66 r = client.admin.command('replSetGetStatus')70 r = client.admin.command('replSetGetStatus')
67 unit_status.append(r['myState'])71 unit_status.append(r['myState'])
68 client.close()72 client.close()
@@ -81,7 +85,10 @@
81validate_status()85validate_status()
82validate_replicaset_setup()86validate_replicaset_setup()
83print("Adding storage relation, and sleeping for 2 min.")87print("Adding storage relation, and sleeping for 2 min.")
84d.relate('mongodb:data', 'storage:data')88try:
85time.sleep(wait_for_relation)89 d.relate('mongodb:data', 'storage:data')
90except OSError as e:
91 print("ignoring error:{}", e)
92d.sentry.wait(wait_for_relation)
86validate_status()93validate_status()
87validate_replicaset_setup()94validate_replicaset_setup()
8895
=== modified file 'tests/50_relate_ceilometer_test.py'
--- tests/50_relate_ceilometer_test.py 2014-12-09 23:36:06 +0000
+++ tests/50_relate_ceilometer_test.py 2015-12-07 22:34:32 +0000
@@ -32,10 +32,13 @@
32 getattr(self, test)()32 getattr(self, test)()
3333
34 def test_mongo_relation(self):34 def test_mongo_relation(self):
35 unit = self.deploy.sentry.unit['ceilometer/0']35 unit = self.deploy.sentry['ceilometer'][0]
36 mongo = self.deploy.sentry.unit['mongodb/0'].info['public-address']36 mongo = self.deploy.sentry['mongodb'][0].info['public-address']
37 mongo_reladdr = self.deploy.sentry['mongodb'][0].relation(
38 'database', 'ceilometer:shared-db')
37 cont = unit.file_contents('/etc/ceilometer/ceilometer.conf')39 cont = unit.file_contents('/etc/ceilometer/ceilometer.conf')
38 if mongo not in cont:40 if (mongo not in cont and mongo_reladdr.get(
41 'hostname', 'I SURE HOPE NOT') not in cont):
39 amulet.raise_status(amulet.FAIL, "Unable to verify ceilometer cfg")42 amulet.raise_status(amulet.FAIL, "Unable to verify ceilometer cfg")
4043
41if __name__ == '__main__':44if __name__ == '__main__':

Subscribers

People subscribed via source and target branches