Juju Charms Collection
mongodb package

Merge lp:~mariosplivalo/charms/trusty/mongodb/replsets-fix-try into lp:charms/trusty/mongodb

  1. Trusty Tahr (14.04)
  2. replsets-fix-try
  3. Merge into trunk
Proposed by Mario Splivalo
Status: Superseded
Proposed branch: lp:~mariosplivalo/charms/trusty/mongodb/replsets-fix-try
Merge into: lp:charms/trusty/mongodb
Diff against target: 1531 lines (+1060/-139)
17 files modified
.bzrignore (+5/-0)
README.md (+18/-23)
charm-helpers-sync.yaml (+1/-0)
hooks/charmhelpers/contrib/hahelpers/cluster.py (+26/-12)
hooks/charmhelpers/contrib/python/packages.py (+80/-0)
hooks/charmhelpers/core/decorators.py (+41/-0)
hooks/charmhelpers/core/host.py (+7/-4)
hooks/charmhelpers/core/templating.py (+1/-1)
hooks/charmhelpers/fetch/__init__.py (+8/-1)
hooks/hooks.py (+298/-89)
setup.cfg (+6/-0)
test_requirements.txt (+1/-0)
tests/00_setup.sh (+0/-9)
tests/03_deploy_replicaset.py (+114/-0)
unit_tests/__init__.py (+2/-0)
unit_tests/test_hooks.py (+341/-0)
unit_tests/test_utils.py (+111/-0)
To merge this branch: bzr merge lp:~mariosplivalo/charms/trusty/mongodb/replsets-fix-try
Reviewer Review Type Date Requested Status
Jorge Niedbalski (community) Needs Fixing
Ryan Beisner (community) Needs Fixing
charmers Pending
Edward Hope-Morley Pending
Review via email: mp+246519@code.launchpad.net

This proposal supersedes a proposal from 2014-12-18.

This proposal has been superseded by a proposal from 2015-01-19.

Description of the change

This merge fixes lp #1379604. Mongodb charm now provides correct information to Ceilometer charm regarding MongoDB replicaset.

It also makes replicaset deploying more robust, fixing lp #1403698.

Also adds some unittest (for the changed code) and amulet tests for reploying replicaset.

To post a comment you must log in.
Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote : Posted in a previous version of this proposal

charm_unit_test #300 trusty-mongodb for mariosplivalo mp245059
    UNIT FAIL: unit-test failed

UNIT Results (max last 2 lines):
  Storing debug log for failure in /tmp/tmpeQOGBT
  make: *** [.venv] Error 1

Full unit test output: http://paste.ubuntu.com/9556195/
Build: http://10.245.162.77:8080/job/charm_unit_test/300/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote : Posted in a previous version of this proposal

charm_lint_check #272 trusty-mongodb for mariosplivalo mp245059
    LINT FAIL: lint-test failed

LINT Results (max last 2 lines):
  Storing debug log for failure in /var/lib/jenkins/.pip/pip.log
  make: *** [.venv] Error 1

Full lint test output: http://paste.ubuntu.com/9556196/
Build: http://10.245.162.77:8080/job/charm_lint_check/272/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote : Posted in a previous version of this proposal

charm_amulet_test #399 trusty-mongodb for mariosplivalo mp245059
    AMULET OK: passed

Build: http://10.245.162.77:8080/job/charm_amulet_test/399/

Revision history for this message
Billy Olsen (billy-olsen) wrote : Posted in a previous version of this proposal

Hey Mario - added a few comments of my own. A few things should be improved throughout here.

Revision history for this message
Jorge Niedbalski (niedbalski) wrote : Posted in a previous version of this proposal

Hello Mario,

Thanks for contributing to this charm. Next are my observations regarding to this first iteration on the review:

1) Lint has failures.

tests/03_deploy_replicaset.py:34:1: E302 expected 2 blank lines, found 1
tests/03_deploy_replicaset.py:45:20: F821 undefined name 'sentry_dict'
tests/03_deploy_replicaset.py:46:18: F821 undefined name 'sentry_dict'
tests/03_deploy_replicaset.py:77:1: E265 block comment should start with '# '
tests/03_deploy_replicaset.py:78:1: E265 block comment should start with '# '
tests/03_deploy_replicaset.py:79:1: E265 block comment should start with '# '
make: *** [lint] Error 1

2) Please change the makefile to just cover the hooks package.

 test: .venv
        @echo Starting unit tests...
- .venv/bin/nosetests -s --nologcapture --with-coverage $(EXTRA) unit_tests/
+ .venv/bin/nosetests -s --nologcapture --with-coverage $(EXTRA) --cover-package hooks unit_tests/

3) Please review the inline comments.

review: Needs Fixing
Revision history for this message
Mario Splivalo (mariosplivalo) wrote : Posted in a previous version of this proposal

Added my comments inline - stuff I haven't commented make perfect sense to me. Thank you, guys, for the inputs!

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote : Posted in a previous version of this proposal

charm_lint_check #680 trusty-mongodb for mariosplivalo mp245059
    LINT FAIL: lint-test failed

LINT Results (max last 2 lines):
  tests/03_deploy_replicaset.py:84:1: E265 block comment should start with '# '
  make: *** [lint] Error 1

Full lint test output: http://paste.ubuntu.com/9718870/
Build: http://10.245.162.77:8080/job/charm_lint_check/680/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote : Posted in a previous version of this proposal

charm_unit_test #709 trusty-mongodb for mariosplivalo mp245059
    UNIT FAIL: unit-test failed

UNIT Results (max last 2 lines):
  FAILED (errors=2)
  make: *** [test] Error 1

Full unit test output: http://paste.ubuntu.com/9718869/
Build: http://10.245.162.77:8080/job/charm_unit_test/709/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote : Posted in a previous version of this proposal

charm_amulet_test #866 trusty-mongodb for mariosplivalo mp245059
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
  ERROR subprocess encountered error code 1
  make: *** [functional_test] Error 1

Full amulet test output: http://paste.ubuntu.com/9719129/
Build: http://10.245.162.77:8080/job/charm_amulet_test/866/

Revision history for this message
Jorge Niedbalski (niedbalski) wrote : Posted in a previous version of this proposal

Hello Mario,

Aside from the following lint errors:

tests/03_deploy_replicaset.py:39:1: E302 expected 2 blank lines, found 1
tests/03_deploy_replicaset.py:50:20: F821 undefined name 'sentry_dict'
tests/03_deploy_replicaset.py:51:18: F821 undefined name 'sentry_dict'
tests/03_deploy_replicaset.py:82:1: E265 block comment should start with '# '
tests/03_deploy_replicaset.py:83:1: E265 block comment should start with '# '
tests/03_deploy_replicaset.py:84:1: E265 block comment should start with '# '
hooks/hooks.py:17:1: F401 'operator' imported but unused
hooks/hooks.py:33:1: F401 'fatal' imported but unused
hooks/hooks.py:49:1: F401 'local_unit' imported but unused
hooks/hooks.py:96:33: E113 unexpected indentation
make: *** [lint] Error 1

I made a few observations regarding to the code, I hope you can drive those in order to keep reviewing this proposal.

thanks.

review: Needs Fixing
Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote : Posted in a previous version of this proposal

charm_lint_check #701 trusty-mongodb for mariosplivalo mp245059
    LINT FAIL: lint-test failed

LINT Results (max last 2 lines):
  unit_tests/test_hooks.py:269:5: E303 too many blank lines (2)
  make: *** [lint] Error 1

Full lint test output: http://paste.ubuntu.com/9720638/
Build: http://10.245.162.77:8080/job/charm_lint_check/701/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote : Posted in a previous version of this proposal

charm_unit_test #730 trusty-mongodb for mariosplivalo mp245059
    UNIT FAIL: unit-test failed

UNIT Results (max last 2 lines):
  Storing debug log for failure in /var/lib/jenkins/.pip/pip.log
  make: *** [.venv] Error 1

Full unit test output: http://paste.ubuntu.com/9720639/
Build: http://10.245.162.77:8080/job/charm_unit_test/730/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote : Posted in a previous version of this proposal

charm_amulet_test #887 trusty-mongodb for mariosplivalo mp245059
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
  ERROR subprocess encountered error code 1
  make: *** [functional_test] Error 1

Full amulet test output: http://paste.ubuntu.com/9720732/
Build: http://10.245.162.77:8080/job/charm_amulet_test/887/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote : Posted in a previous version of this proposal

charm_lint_check #702 trusty-mongodb for mariosplivalo mp245059
    LINT FAIL: lint-test failed

LINT Results (max last 2 lines):
  Storing debug log for failure in /var/lib/jenkins/.pip/pip.log
  make: *** [.venv] Error 1

Full lint test output: http://paste.ubuntu.com/9720948/
Build: http://10.245.162.77:8080/job/charm_lint_check/702/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote : Posted in a previous version of this proposal

charm_unit_test #731 trusty-mongodb for mariosplivalo mp245059
    UNIT FAIL: unit-test failed

UNIT Results (max last 2 lines):
  Storing debug log for failure in /var/lib/jenkins/.pip/pip.log
  make: *** [.venv] Error 1

Full unit test output: http://paste.ubuntu.com/9720949/
Build: http://10.245.162.77:8080/job/charm_unit_test/731/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote : Posted in a previous version of this proposal

charm_amulet_test #888 trusty-mongodb for mariosplivalo mp245059
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
  ERROR subprocess encountered error code 1
  make: *** [functional_test] Error 1

Full amulet test output: http://paste.ubuntu.com/9721024/
Build: http://10.245.162.77:8080/job/charm_amulet_test/888/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote : Posted in a previous version of this proposal

charm_lint_check #704 trusty-mongodb for mariosplivalo mp245059
    LINT FAIL: lint-test failed

LINT Results (max last 2 lines):
  Storing debug log for failure in /var/lib/jenkins/.pip/pip.log
  make: *** [.venv] Error 1

Full lint test output: http://paste.ubuntu.com/9721529/
Build: http://10.245.162.77:8080/job/charm_lint_check/704/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote : Posted in a previous version of this proposal

charm_unit_test #733 trusty-mongodb for mariosplivalo mp245059
    UNIT FAIL: unit-test failed

UNIT Results (max last 2 lines):
  Storing debug log for failure in /var/lib/jenkins/.pip/pip.log
  make: *** [.venv] Error 1

Full unit test output: http://paste.ubuntu.com/9721530/
Build: http://10.245.162.77:8080/job/charm_unit_test/733/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote : Posted in a previous version of this proposal

charm_amulet_test #889 trusty-mongodb for mariosplivalo mp245059
    AMULET OK: passed

Build: http://10.245.162.77:8080/job/charm_amulet_test/889/

Revision history for this message
Jorge Niedbalski (niedbalski) wrote : Posted in a previous version of this proposal

A few more comments on the code. Please review.

review: Needs Fixing
Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_unit_test #813 trusty-mongodb for mariosplivalo mp246519
    UNIT FAIL: unit-test failed

UNIT Results (max last 2 lines):
  Storing debug log for failure in /var/lib/jenkins/.pip/pip.log
  make: *** [.venv] Error 1

Full unit test output: http://paste.ubuntu.com/9755506/
Build: http://10.245.162.77:8080/job/charm_unit_test/813/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #784 trusty-mongodb for mariosplivalo mp246519
    LINT FAIL: lint-test failed

LINT Results (max last 2 lines):
  Storing debug log for failure in /var/lib/jenkins/.pip/pip.log
  make: *** [.venv] Error 1

Full lint test output: http://paste.ubuntu.com/9755507/
Build: http://10.245.162.77:8080/job/charm_lint_check/784/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_amulet_test #969 trusty-mongodb for mariosplivalo mp246519
    AMULET OK: passed

Build: http://10.245.162.77:8080/job/charm_amulet_test/969/

Revision history for this message
Ryan Beisner (1chb1n) wrote :

Precise deploy tests fail with this MP with:

# from deployer output
2015-01-15 12:36:24 [DEBUG] deployer.import: Waiting for units before adding relations
2015-01-15 12:36:24 [ERROR] deployer.env: The following units had errors:
   unit: mongodb/0: machine: 9 agent-state: error details: hook failed: "install"
2015-01-15 12:36:24 [INFO] deployer.cli: Deployment stopped. run time: 431.20

# from mongodb juju unit log
2015-01-15 12:35:41 INFO install Setting up python-pymongo (2.1-1ubuntu0.1) ...
2015-01-15 12:35:41 INFO install Traceback (most recent call last):
2015-01-15 12:35:41 INFO install File "/var/lib/juju/agents/unit-mongodb-0/charm/hooks/install", line 38, in <module>
2015-01-15 12:35:41 INFO install from pymongo import MongoClient
2015-01-15 12:35:41 INFO install ImportError: cannot import name MongoClient
2015-01-15 12:35:41 ERROR juju.worker.uniter uniter.go:486 hook failed: exit status 1

Trusty, Juno deploy tests are happy.

review: Needs Fixing
Revision history for this message
Jorge Niedbalski (niedbalski) wrote :

Please make sure to include 'execd_preinstall' on the TO_PATCH list for /home/niedbalski/src/charms/trusty/mongodb/unit_tests/test_hooks.py

======================================================================
ERROR: test_install_hook (mongodb.unit_tests.test_hooks.MongoHooksTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/niedbalski/src/charms/trusty/mongodb/.venv/local/lib/python2.7/site-packages/mock.py", line 1201, in patched
    return func(*args, **keywargs)
  File "/home/niedbalski/src/charms/trusty/mongodb/unit_tests/test_hooks.py", line 154, in test_install_hook
    hooks.install_hook()
  File "hooks/hooks.py", line 915, in install_hook
    execd_preinstall()
  File "hooks/charmhelpers/payload/execd.py", line 50, in execd_preinstall
    execd_run('charm-pre-install', execd_dir=execd_dir)
  File "hooks/charmhelpers/payload/execd.py", line 38, in execd_run
    for submodule_path in execd_submodule_paths(command, execd_dir):
  File "hooks/charmhelpers/payload/execd.py", line 30, in execd_submodule_paths
    for module_path in execd_module_paths(execd_dir):
  File "hooks/charmhelpers/payload/execd.py", line 16, in execd_module_paths
    execd_dir = default_execd_dir()
  File "hooks/charmhelpers/payload/execd.py", line 10, in default_execd_dir
    return os.path.join(os.environ['CHARM_DIR'], 'exec.d')
  File "/home/niedbalski/src/charms/trusty/mongodb/.venv/lib/python2.7/UserDict.py", line 23, in __getitem__
    raise KeyError(key)
KeyError: 'CHARM_DIR'

review: Needs Fixing
Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_unit_test #887 trusty-mongodb for mariosplivalo mp246519
    UNIT FAIL: unit-test failed

UNIT Results (max last 2 lines):
  FAILED (errors=2)
  make: *** [test] Error 1

Full unit test output: http://paste.ubuntu.com/9788192/
Build: http://10.245.162.77:8080/job/charm_unit_test/887/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #858 trusty-mongodb for mariosplivalo mp246519
    LINT FAIL: lint-test failed

LINT Results (max last 2 lines):
  Storing debug log for failure in /var/lib/jenkins/.pip/pip.log
  make: *** [.venv] Error 1

Full lint test output: http://paste.ubuntu.com/9788195/
Build: http://10.245.162.77:8080/job/charm_lint_check/858/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_amulet_test #1082 trusty-mongodb for mariosplivalo mp246519
    AMULET OK: passed

Build: http://10.245.162.77:8080/job/charm_amulet_test/1082/

lp:~mariosplivalo/charms/trusty/mongodb/replsets-fix-try updated
106. By Mario Splivalo

lsb_release() was imported twice

Revision history for this message
Ryan Beisner (1chb1n) wrote :

FYI: mongodb install hook failed on Precise due to pip install timeout.

lp:~mariosplivalo/charms/trusty/mongodb/replsets-fix-try updated
107. By Mario Splivalo

Adding support for pip_proxy (working version)

108. By Mario Splivalo

Added support for pip_proxy charm config option.
This is necessary if the cloud environment can talk to the outside
world only via proxy.

109. By Mario Splivalo

Fixed wrong checking for pip_proxy option.

110. By Mario Splivalo

Fixed passing pip_proxy to pip_install

111. By Mario Splivalo

Code change to simplify unit_testing

112. By Mario Splivalo

Removed MongoClient usage in favor of Connection (both from pymongo).
pymongo.Connectin is deprecated in pymongo packaged with trusty, but it
is there and it behaves in the same manner as pymongo.Connection in
precise.

Hence, no need for pip_install and such...

113. By Mario Splivalo

Replaced python-portalocker with python-filelock. The later exists
in both precise and trusty.

114. By Mario Splivalo

Removed pip_proxy config option as it is no longer needed

115. By Mario Splivalo

Fix unit_tests (MongoClient->Connection)

116. By Mario Splivalo

Dependency change

117. By Mario Splivalo

Try not to destroy replicaset if primary is being removed

118. By Mario Splivalo

Remove replica-set-relation-departed symlink

119. By Mario Splivalo

Give more time for reelection to complete

120. By Mario Splivalo

Fix lint errors

121. By Mario Splivalo

Updated README.md to reflect proper removal of failed units

122. By Mario Splivalo

Updated README.md with information on how to fix degraded replicaset

Unmerged revisions

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file '.bzrignore'
2--- .bzrignore 2014-08-21 14:26:30 +0000
3+++ .bzrignore 2015-01-19 22:37:53 +0000
4@@ -1,4 +1,9 @@
5 .git
6+.project
7+.pydevproject
8+.coverage
9+.settings
10+.venv/
11 bin/*
12 scripts/charm-helpers-sync.py
13 exec.d/*
14
15=== modified file 'README.md'
16--- README.md 2014-11-20 15:42:07 +0000
17+++ README.md 2015-01-19 22:37:53 +0000
18@@ -12,27 +12,20 @@
19
20 ## Review the configurable options
21
22-The MongoDB charm allows for certain values to be configurable via a config.yaml file. The options provided are extensive, you should [review the options](https://jujucharms.com/fullscreen/search/precise/mongodb-20/?text=mongodb#bws-configuration).
23+The MongoDB charm allows for certain values to be configurable via a config.yaml file. The options provided are extensive, you should [review the options](https://jujucharms.com/fullscreen/search/precise/mongodb-20/?text=mongodb#bws-configuration).
24
25-Specifically the following options are important:
26+Specifically the following options are important:
27
28 - replicaset
29 - ie: myreplicaset
30 - Each replicaset has a unique name to distinguish it’s members from other replicasets available in the network.
31- - The default value of myset should be fine for most single cluster scenarios.
32+ - The default value of "myset" should be fine for most single cluster scenarios.
33
34 - web_admin_ui
35 - MongoDB comes with a basic but very informative web user interface that provides health
36 and status information on the database node as well as the cluster.
37 - The default value of yes will start the Admin web UI on port 28017.
38
39-- replicaset_master
40- - If this node is going to be joining an existing replicaset, you can specify a member of that cluster
41- ( preferably the master node ) so we can join the existing replicaset.
42- - The value should be in the form of host[:port]
43- - ie: hostname ( will connect to hostname on the default port of 27017 )
44- - ie: hostname:port ( will connect to hostname on port number <port> )
45-
46 Most of the options in config.yaml have been modeled after the default configuration file for mongodb (normally in /etc/mongodb.conf) and should be familiar to most mongodb admins. Each option in this charm have a brief description of what it does.
47
48 # Usage
49@@ -46,10 +39,11 @@
50
51 ## Replica Sets
52
53-Deploy the first MongoDB instance
54-
55- juju deploy mongodb
56- juju expose mongodb
57+Deploy the first two MongoDB instances that will form replicaset:
58+
59+ juju deployu mongodb -n 2
60+
61+Deploying three or more units at start can sometimes lead to unexpected race-conditions so it's best to start with two nodes.
62
63 Your deployment should look similar to this ( `juju status` ):
64
65@@ -91,18 +85,19 @@
66
67 juju set mongodb replicaset=<new_replicaset_name>
68
69-### Add one more nodes to your replicaset
70+### Add one or more nodes to your replicaset
71
72 juju add-unit mongodb
73-
74-
75-### Add multiple nodes to your replicaset
76-
77- juju add-unit mongodb -n5
78-
79+ juju add-unit mongodb -n2
80
81 We now have a working MongoDB replica-set.
82
83+Keep in mind that you need to have odd number of nodes for a properly formed replicaset.
84+
85+Replicaset can't function with only one available node - shall this happens the remaining node is switched to 'read-only' until at least one of the broken nodes is restored.
86+
87+More info can be found in MongoDB documentation at [their website](http://docs.mongodb.org/manual/replication/)
88+
89 ## Sharding (Scale Out Usage)
90
91 According the the MongoDB documentation found on [their website](http://docs.mongodb.org/manual/tutorial/deploy-shard-cluster/), one way of deploying a Shard Cluster is as follows:
92@@ -127,7 +122,7 @@
93 replicaset: configsvr
94
95 We'll save this one as `~/mongodb-shard.yaml`.
96-
97+
98 ### Bootstrap the environment
99 juju bootstrap
100
101@@ -210,7 +205,7 @@
102
103 ## MongoDB Contact Information
104
105-- [MongoDB website](http://mongodb.org)
106+- [MongoDB website](http://mongodb.org)
107 - [MongoDB documentation](http://www.mongodb.org/display/DOCS/Home)
108 - [MongoDB bug tracker](https://jira.mongodb.org/secure/Dashboard.jspa)
109 - [MongoDB user mailing list](https://groups.google.com/forum/#!forum/mongodb-user)
110
111=== modified file 'charm-helpers-sync.yaml'
112--- charm-helpers-sync.yaml 2015-01-13 17:51:52 +0000
113+++ charm-helpers-sync.yaml 2015-01-19 22:37:53 +0000
114@@ -4,4 +4,5 @@
115 - core
116 - fetch
117 - contrib.hahelpers.cluster
118+ - contrib.python.packages
119 - payload.execd
120
121=== modified file 'hooks/charmhelpers/contrib/hahelpers/cluster.py'
122--- hooks/charmhelpers/contrib/hahelpers/cluster.py 2014-12-09 23:58:57 +0000
123+++ hooks/charmhelpers/contrib/hahelpers/cluster.py 2015-01-19 22:37:53 +0000
124@@ -13,6 +13,7 @@
125
126 import subprocess
127 import os
128+
129 from socket import gethostname as get_unit_hostname
130
131 import six
132@@ -28,12 +29,19 @@
133 WARNING,
134 unit_get,
135 )
136+from charmhelpers.core.decorators import (
137+ retry_on_exception,
138+)
139
140
141 class HAIncompleteConfig(Exception):
142 pass
143
144
145+class CRMResourceNotFound(Exception):
146+ pass
147+
148+
149 def is_elected_leader(resource):
150 """
151 Returns True if the charm executing this is the elected cluster leader.
152@@ -68,24 +76,30 @@
153 return False
154
155
156-def is_crm_leader(resource):
157+@retry_on_exception(5, base_delay=2, exc_type=CRMResourceNotFound)
158+def is_crm_leader(resource, retry=False):
159 """
160 Returns True if the charm calling this is the elected corosync leader,
161 as returned by calling the external "crm" command.
162+
163+ We allow this operation to be retried to avoid the possibility of getting a
164+ false negative. See LP #1396246 for more info.
165 """
166- cmd = [
167- "crm", "resource",
168- "show", resource
169- ]
170+ cmd = ['crm', 'resource', 'show', resource]
171 try:
172- status = subprocess.check_output(cmd).decode('UTF-8')
173+ status = subprocess.check_output(cmd, stderr=subprocess.STDOUT)
174+ if not isinstance(status, six.text_type):
175+ status = six.text_type(status, "utf-8")
176 except subprocess.CalledProcessError:
177- return False
178- else:
179- if get_unit_hostname() in status:
180- return True
181- else:
182- return False
183+ status = None
184+
185+ if status and get_unit_hostname() in status:
186+ return True
187+
188+ if status and "resource %s is NOT running" % (resource) in status:
189+ raise CRMResourceNotFound("CRM resource %s not found" % (resource))
190+
191+ return False
192
193
194 def is_leader(resource):
195
196=== added directory 'hooks/charmhelpers/contrib/python'
197=== added file 'hooks/charmhelpers/contrib/python/__init__.py'
198=== added file 'hooks/charmhelpers/contrib/python/packages.py'
199--- hooks/charmhelpers/contrib/python/packages.py 1970-01-01 00:00:00 +0000
200+++ hooks/charmhelpers/contrib/python/packages.py 2015-01-19 22:37:53 +0000
201@@ -0,0 +1,80 @@
202+#!/usr/bin/env python
203+# coding: utf-8
204+
205+__author__ = "Jorge Niedbalski <jorge.niedbalski@canonical.com>"
206+
207+from charmhelpers.fetch import apt_install, apt_update
208+from charmhelpers.core.hookenv import log
209+
210+try:
211+ from pip import main as pip_execute
212+except ImportError:
213+ apt_update()
214+ apt_install('python-pip')
215+ from pip import main as pip_execute
216+
217+
218+def parse_options(given, available):
219+ """Given a set of options, check if available"""
220+ for key, value in sorted(given.items()):
221+ if key in available:
222+ yield "--{0}={1}".format(key, value)
223+
224+
225+def pip_install_requirements(requirements, **options):
226+ """Install a requirements file """
227+ command = ["install"]
228+
229+ available_options = ('proxy', 'src', 'log', )
230+ for option in parse_options(options, available_options):
231+ command.append(option)
232+
233+ command.append("-r {0}".format(requirements))
234+ log("Installing from file: {} with options: {}".format(requirements,
235+ command))
236+ pip_execute(command)
237+
238+
239+def pip_install(package, fatal=False, upgrade=False, **options):
240+ """Install a python package"""
241+ command = ["install"]
242+ available_options = ('proxy', 'src', 'log', "index-url")
243+ for option in parse_options(options, available_options):
244+ log("DEBUG: option: %s" % (option))
245+ command.append(option)
246+
247+ if upgrade:
248+ command.append('--upgrade')
249+
250+ if isinstance(package, list):
251+ command.extend(package)
252+ else:
253+ command.append(package)
254+
255+ log("Installing {} package with options: {}".format(package,
256+ command))
257+ pip_execute(command)
258+
259+
260+def pip_uninstall(package, **options):
261+ """Uninstall a python package"""
262+ command = ["uninstall", "-q", "-y"]
263+
264+ available_options = ('proxy', 'log', )
265+ for option in parse_options(options, available_options):
266+ command.append(option)
267+
268+ if isinstance(package, list):
269+ command.extend(package)
270+ else:
271+ command.append(package)
272+
273+ log("Uninstalling {} package with options: {}".format(package,
274+ command))
275+ pip_execute(command)
276+
277+
278+def pip_list():
279+ """Returns the list of current python installed packages
280+ """
281+ return pip_execute(["list"])
282
283=== added file 'hooks/charmhelpers/core/decorators.py'
284--- hooks/charmhelpers/core/decorators.py 1970-01-01 00:00:00 +0000
285+++ hooks/charmhelpers/core/decorators.py 2015-01-19 22:37:53 +0000
286@@ -0,0 +1,41 @@
287+#
288+# Copyright 2014 Canonical Ltd.
289+#
290+# Authors:
291+# Edward Hope-Morley <opentastic@gmail.com>
292+#
293+
294+import time
295+
296+from charmhelpers.core.hookenv import (
297+ log,
298+ INFO,
299+)
300+
301+
302+def retry_on_exception(num_retries, base_delay=0, exc_type=Exception):
303+ """If the decorated function raises exception exc_type, allow num_retries
304+ retry attempts before raise the exception.
305+ """
306+ def _retry_on_exception_inner_1(f):
307+ def _retry_on_exception_inner_2(*args, **kwargs):
308+ retries = num_retries
309+ multiplier = 1
310+ while True:
311+ try:
312+ return f(*args, **kwargs)
313+ except exc_type:
314+ if not retries:
315+ raise
316+
317+ delay = base_delay * multiplier
318+ multiplier += 1
319+ log("Retrying '%s' %d more times (delay=%s)" %
320+ (f.__name__, retries, delay), level=INFO)
321+ retries -= 1
322+ if delay:
323+ time.sleep(delay)
324+
325+ return _retry_on_exception_inner_2
326+
327+ return _retry_on_exception_inner_1
328
329=== modified file 'hooks/charmhelpers/core/host.py'
330--- hooks/charmhelpers/core/host.py 2014-12-09 23:58:57 +0000
331+++ hooks/charmhelpers/core/host.py 2015-01-19 22:37:53 +0000
332@@ -162,13 +162,16 @@
333 uid = pwd.getpwnam(owner).pw_uid
334 gid = grp.getgrnam(group).gr_gid
335 realpath = os.path.abspath(path)
336- if os.path.exists(realpath):
337- if force and not os.path.isdir(realpath):
338+ path_exists = os.path.exists(realpath)
339+ if path_exists and force:
340+ if not os.path.isdir(realpath):
341 log("Removing non-directory file {} prior to mkdir()".format(path))
342 os.unlink(realpath)
343- else:
344+ os.makedirs(realpath, perms)
345+ os.chown(realpath, uid, gid)
346+ elif not path_exists:
347 os.makedirs(realpath, perms)
348- os.chown(realpath, uid, gid)
349+ os.chown(realpath, uid, gid)
350
351
352 def write_file(path, content, owner='root', group='root', perms=0o444):
353
354=== modified file 'hooks/charmhelpers/core/templating.py'
355--- hooks/charmhelpers/core/templating.py 2014-12-09 23:58:57 +0000
356+++ hooks/charmhelpers/core/templating.py 2015-01-19 22:37:53 +0000
357@@ -48,5 +48,5 @@
358 level=hookenv.ERROR)
359 raise e
360 content = template.render(context)
361- host.mkdir(os.path.dirname(target))
362+ host.mkdir(os.path.dirname(target), owner, group)
363 host.write_file(target, content, owner, group, perms)
364
365=== modified file 'hooks/charmhelpers/fetch/__init__.py'
366--- hooks/charmhelpers/fetch/__init__.py 2014-12-09 23:58:57 +0000
367+++ hooks/charmhelpers/fetch/__init__.py 2015-01-19 22:37:53 +0000
368@@ -64,9 +64,16 @@
369 'trusty-juno/updates': 'trusty-updates/juno',
370 'trusty-updates/juno': 'trusty-updates/juno',
371 'juno/proposed': 'trusty-proposed/juno',
372- 'juno/proposed': 'trusty-proposed/juno',
373 'trusty-juno/proposed': 'trusty-proposed/juno',
374 'trusty-proposed/juno': 'trusty-proposed/juno',
375+ # Kilo
376+ 'kilo': 'trusty-updates/kilo',
377+ 'trusty-kilo': 'trusty-updates/kilo',
378+ 'trusty-kilo/updates': 'trusty-updates/kilo',
379+ 'trusty-updates/kilo': 'trusty-updates/kilo',
380+ 'kilo/proposed': 'trusty-proposed/kilo',
381+ 'trusty-kilo/proposed': 'trusty-proposed/kilo',
382+ 'trusty-proposed/kilo': 'trusty-proposed/kilo',
383 }
384
385 # The order of this list is very important. Handlers should be listed in from
386
387=== modified file 'hooks/hooks.py'
388--- hooks/hooks.py 2015-01-13 17:51:52 +0000
389+++ hooks/hooks.py 2015-01-19 22:37:53 +0000
390@@ -28,17 +28,50 @@
391 apt_install
392 )
393
394+import json
395+
396+from charmhelpers.core.host import lsb_release
397+
398+series = lsb_release()['DISTRIB_CODENAME']
399+
400+try:
401+ from pymongo import MongoClient
402+ from pymongo.errors import OperationFailure
403+except ImportError:
404+ if series == 'precise':
405+ from charmhelpers.contrib.python.packages import pip_install
406+ apt_install(['python-dev', 'build-essential'], fatal=True)
407+ pip_install('pymongo', fatal=True, upgrade=True)
408+ pip_install('portalocker', fatal=True, upgrade=True)
409+ else:
410+ apt_install("python-pymongo", fatal=True)
411+ apt_install("python-portalocker", fatal=True)
412+ from pymongo import MongoClient
413+ from pymongo.errors import OperationFailure
414+
415+try:
416+ from portalocker import Lock
417+except ImportError:
418+ if series == 'precise':
419+ from charmhelpers.contrib.python.packages import pip_install
420+ pip_install('portalocker', fatal=True, upgrade=True)
421+ else:
422+ apt_install("python-portalocker", fatal=True)
423+ from portalocker import Lock
424+
425 from charmhelpers.core.hookenv import (
426+ close_port,
427 config,
428+ open_port,
429 unit_get,
430 relation_get,
431 relation_set,
432 relations_of_type,
433 relation_id,
434 relation_ids,
435- open_port,
436- close_port,
437 Hooks,
438+ DEBUG,
439+ WARNING,
440 )
441
442 from charmhelpers.core.hookenv import log as juju_log
443@@ -50,6 +83,11 @@
444 lsb_release,
445 )
446
447+from charmhelpers.contrib.hahelpers.cluster import (
448+ oldest_peer,
449+ peer_units
450+)
451+
452 hooks = Hooks()
453
454 ###############################################################################
455@@ -61,6 +99,30 @@
456 default_wait_for = 10
457 default_max_tries = 5
458
459+INSTALL_PACKAGES = ['mongodb-server', 'python-yaml']
460+
461+INIT_LOCKFILE = '/tmp/mongodb-charm.lock'
462+
463+# number of seconds init_replset will pause while looping to check if
464+# replicaset is initialized
465+INIT_CHECK_DELAY = 1.5
466+
467+# number of times mongo_client_smart will try to execute given statement
468+MONGO_CLIENT_RETRIES = 10
469+
470+# These are MongoDB ReplicaSet states, for convenience:
471+MONGO_STARTUP = 0
472+MONGO_PRIMARY = 1
473+MONGO_SECONDARY = 2
474+MONGO_RECOVERING = 3
475+MONGO_FATAL = 4
476+MONGO_STARTUP2 = 5
477+MONGO_UNKNOWN = 6
478+MONGO_ARBITER = 7
479+MONGO_DOWN = 8
480+MONGO_ROLLBACK = 9
481+MONGO_REMOVED = 10
482+
483 ###############################################################################
484 # Supporting functions
485 ###############################################################################
486@@ -119,7 +181,7 @@
487
488
489 def update_file(filename=None, new_data=None, old_data=None):
490- juju_log("update_file: %s" % filename)
491+ juju_log("update_file: %s" % filename, level=DEBUG)
492 if filename is None or new_data is None:
493 retVal = False
494 try:
495@@ -131,7 +193,6 @@
496 juju_log(str(e))
497 retVal = False
498 finally:
499- juju_log("update_file %s returns: %s" % (filename, retVal))
500 return(retVal)
501
502
503@@ -161,6 +222,14 @@
504 return((None, None))
505
506
507+class MasterNotFoundException(Exception):
508+ pass
509+
510+
511+class TimeoutException(Exception):
512+ pass
513+
514+
515 ###############################################################################
516 # Charm support functions
517 ###############################################################################
518@@ -336,49 +405,161 @@
519 else:
520 cmd_line = 'mongo'
521 cmd_line += ' --host %s' % host
522- cmd_line += ' --eval \'%s\'' % command
523- juju_log("Executing: %s" % cmd_line)
524+ cmd_line += ' --eval \'printjson(%s)\'' % command
525+ juju_log("Executing: %s" % cmd_line, level=DEBUG)
526 return(subprocess.call(cmd_line, shell=True) == 0)
527
528
529-def init_replset(master_node=None):
530- if master_node is None:
531- juju_log("init_replset: master_node must be defined.")
532- retVal = False
533- else:
534- retVal = mongo_client(master_node, 'rs.initiate()')
535- juju_log("init_replset returns: %s" % retVal)
536+def mongo_client_smart(host='localhost', command=None):
537+ '''
538+ Rework of mongo_client function, it retries the command
539+ MONGO_CLIENT_RETRIES times
540+
541+ :param host: The host to connect to. Defaults to localhost
542+ :param command: The command to be executed. Can't be None
543+ :returns True if command succeeded, False if it failed
544+ '''
545+
546+ if command is None:
547+ return False
548+
549+ cmd_line = ['mongo', '--quiet', '--host', host,
550+ '--eval', 'printjson(%s)' % command]
551+ juju_log("mongo_client_smart executing: %s" % str(cmd_line), level=DEBUG)
552+
553+ for i in xrange(MONGO_CLIENT_RETRIES):
554+ try:
555+ cmd_output = subprocess.check_output(cmd_line)
556+ juju_log('mongo_client_smart executed, output: %s' %
557+ cmd_output)
558+ if json.loads(cmd_output)['ok'] == 1:
559+ return True
560+ except subprocess.CalledProcessError as err:
561+ juju_log('mongo_client_smart failed: %s' %
562+ err.output,
563+ level=DEBUG)
564+ pass
565+ finally:
566+ time.sleep(1.5)
567+
568+ # At this point, the command failed
569+ juju_log('mongo_client_smart failed executing: %s', level=WARNING)
570+ return False
571+
572+
573+def init_replset():
574+ config_data = config()
575+ # Use my IP at rs.initiate(), voids issues with invalid (and/or
576+ # not resolvable by peers) hostnames
577+
578+ rset = config_data['replicaset']
579+ addr = unit_get('private-address')
580+ port = config_data['port']
581+
582+ init = '{_id: "%s", members: [{_id: 0, host: "%s:%s"}]} ' % (rset,
583+ addr,
584+ port)
585+
586+ retVal = mongo_client('localhost', 'rs.initiate(%s)' % init)
587+ time.sleep(1) # give mongod some time to become primary
588+ c = MongoClient('localhost')
589+ while True:
590+ try:
591+ r = run_admin_command(c, 'replSetGetStatus')
592+ mongo_state = r['myState']
593+ juju_log('init_replset: myState: %s' % mongo_state)
594+ if mongo_state == MONGO_PRIMARY: # we're primary!
595+ break
596+ elif mongo_state in (MONGO_STARTUP,
597+ MONGO_STARTUP2,
598+ MONGO_SECONDARY
599+ ): # we are still initializing
600+ continue
601+ else:
602+ juju_log('init_replset: Unexpected replicaSet state: %s' %
603+ mongo_state)
604+ retVal = False
605+ break
606+ except OperationFailure as e:
607+ juju_log('init_replset: OperationFailure: %s' % e)
608+ if 'Received replSetInitiate' in str(e):
609+ continue
610+ else:
611+ raise
612+ finally:
613+ time.sleep(INIT_CHECK_DELAY)
614+
615+ juju_log("init_replset returns: %s" % retVal, level=DEBUG)
616 return(retVal)
617
618
619+def run_admin_command(client, cmdstr):
620+ """Runs an admin command against the client. Primary purpose is to
621+ simplify the unit testing.
622+ """
623+ return client.admin.command(cmdstr)
624+
625+
626 def join_replset(master_node=None, host=None):
627+ # TODO: This methoud shouldn't accept any arguments becase we don't care
628+ # about master_node - we always run replset admin commands on
629+ # localhost.
630+ # However, that might break other code calling this method.
631+ # This will wait charm rewrite.
632 juju_log("join_replset: master_node: %s, host: %s" %
633 (master_node, host))
634 if master_node is None or host is None:
635 retVal = False
636 else:
637- retVal = mongo_client(master_node, "rs.add(\"%s\")" % host)
638- juju_log("join_replset returns: %s" % retVal)
639+ retVal = mongo_client_smart('localhost', 'rs.add("%s")' % host)
640+ juju_log("join_replset returns: %s" % retVal, level=DEBUG)
641+ return(retVal)
642+
643+
644+def leave_replset(master_node=None, host=None):
645+ juju_log("leave_replset: master_node: %s, host: %s" %
646+ (master_node, host))
647+ if master_node is None or host is None:
648+ retVal = False
649+ else:
650+ retVal = mongo_client(master_node, 'rs.remove("%s")' % host)
651+ juju_log("leave_replset returns: %s" % retVal, level=DEBUG)
652 return(retVal)
653
654
655 def enable_replset(replicaset_name=None):
656+ retVal = False
657 if replicaset_name is None:
658- retVal = False
659+ juju_log('enable_replset: replicaset_name is None, exiting',
660+ level=DEBUG)
661 try:
662- mongodb_init_config = open(default_mongodb_init_config).read()
663- if re.search(' --replSet %s ' % replicaset_name,
664- mongodb_init_config, re.MULTILINE) is None:
665- mongodb_init_config = regex_sub([(' -- ',
666- ' -- --replSet %s ' %
667- replicaset_name)],
668- mongodb_init_config)
669- retVal = update_file(default_mongodb_init_config, mongodb_init_config)
670+ juju_log('enable_replset: trying to get lock on: %s' %
671+ default_mongodb_init_config)
672+ with Lock(INIT_LOCKFILE, mode='a', timeout=100,
673+ check_interval=.25, fail_when_locked=False):
674+ juju_log('enable_replset: lock acquired', level=DEBUG)
675+ with open(default_mongodb_init_config) as mongo_init_file:
676+ mongodb_init_config = mongo_init_file.read()
677+ if re.search(' --replSet %s ' % replicaset_name,
678+ mongodb_init_config, re.MULTILINE) is None:
679+ juju_log('enable_replset: --relset not preset,'
680+ ' enabling',
681+ level=DEBUG)
682+ mongodb_init_config = regex_sub([(' -- ',
683+ ' -- --replSet %s ' %
684+ replicaset_name)],
685+ mongodb_init_config)
686+ update_file(default_mongodb_init_config,
687+ mongodb_init_config)
688+ retVal = True
689+
690+ juju_log('enable_replset will return: %s' % str(retVal), level=DEBUG)
691+
692 except Exception, e:
693- juju_log(str(e))
694+ juju_log(str(e), level=WARNING)
695 retVal = False
696 finally:
697- return(retVal)
698+ return retVal
699
700
701 def update_daemon_options(daemon_options=None):
702@@ -752,8 +933,7 @@
703 arm64_trusty_quirk()
704
705 apt_update(fatal=True)
706- apt_install(packages=['mongodb-server', 'mongodb-clients', 'python-yaml'],
707- fatal=True)
708+ apt_install(packages=INSTALL_PACKAGES, fatal=True)
709
710
711 @hooks.hook('config-changed')
712@@ -928,28 +1108,34 @@
713 juju_log("my_hostname: %s" % my_hostname)
714 juju_log("my_port: %s" % my_port)
715 juju_log("my_replset: %s" % my_replset)
716-
717- return(relation_set(relation_id(), {
718- 'hostname': my_hostname,
719- 'port': my_port,
720- 'replset': my_replset,
721- 'type': 'database',
722- }))
723+ relation_data = {'hostname': my_hostname,
724+ 'port': my_port,
725+ 'type': 'database',
726+ }
727+
728+ if len(peer_units('replica-set')) > 1:
729+ relation_data['replset'] = my_replset
730+
731+ relation_set(relation_id(), relation_data)
732
733
734 @hooks.hook('replicaset-relation-joined')
735 def replica_set_relation_joined():
736- juju_log("replica_set_relation_joined")
737+ juju_log("replica_set_relation_joined-start")
738 my_hostname = unit_get('private-address')
739 my_port = config('port')
740 my_replset = config('replicaset')
741+
742 my_install_order = os.environ['JUJU_UNIT_NAME'].split('/')[1]
743 juju_log("my_hostname: %s" % my_hostname)
744 juju_log("my_port: %s" % my_port)
745 juju_log("my_replset: %s" % my_replset)
746 juju_log("my_install_order: %s" % my_install_order)
747- enable_replset(my_replset)
748- restart_mongod()
749+ # do not restart mongodb if config has not changed
750+ if enable_replset(my_replset):
751+ juju_log('Restarting mongodb after config change (enable replset)',
752+ level=DEBUG)
753+ restart_mongod()
754
755 relation_set(relation_id(), {
756 'hostname': my_hostname,
757@@ -958,61 +1144,84 @@
758 'install-order': my_install_order,
759 'type': 'replset',
760 })
761+ juju_log("replica_set_relation_joined-finish")
762+
763+
764+def am_i_primary():
765+ c = MongoClient('localhost')
766+ for i in xrange(10):
767+ try:
768+ r = run_admin_command(c, 'replSetGetStatus')
769+ juju_log('am_i_primary: replSetGetStatus returned: %s' % str(r),
770+ level=DEBUG)
771+ return r['myState'] == MONGO_PRIMARY
772+ except OperationFailure as e:
773+ juju_log('am_i_primary: OperationError: %s' % str(e), level=DEBUG)
774+ if 'replSetInitiate - should come online shortly' in str(e):
775+ # replSet initialization in progress
776+ continue
777+ elif 'EMPTYCONFIG' in str(e):
778+ # replication not
779+ return False
780+ else:
781+ raise
782+ finally:
783+ time.sleep(1.5)
784+
785+ # Raise an error if we exhausted the maximum amount of trials
786+ raise TimeoutException('Unable to determine if local unit is primary')
787
788
789 @hooks.hook('replicaset-relation-changed')
790 def replica_set_relation_changed():
791- juju_log("replica_set_relation_changed")
792- my_hostname = unit_get('private-address')
793- my_port = config('port')
794- my_install_order = os.environ['JUJU_UNIT_NAME'].split('/')[1]
795- my_replicaset_master = config('replicaset_master')
796-
797- # If we are joining an existing replicaset cluster, just join and leave.
798- if my_replicaset_master != "auto":
799- return(join_replset(my_replicaset_master, my_hostname))
800-
801- # Default to this node being the master
802- master_hostname = my_hostname
803- master_port = my_port
804- master_install_order = my_install_order
805-
806- # Check the nodes in the relation to find the master
807- for member in relations_of_type('replica-set'):
808- member = member['__unit__']
809- juju_log("replica_set_relation_changed: member: %s" % member)
810- hostname = relation_get('hostname', member)
811- port = relation_get('port', member)
812- inst_ordr = relation_get('install-order', member)
813- juju_log("replica_set_relation_changed: install_order: %s" % inst_ordr)
814- if inst_ordr is None:
815- juju_log("replica_set_relation_changed: install_order is None."
816- " relation is not ready")
817- break
818- if int(inst_ordr) < int(master_install_order):
819- master_hostname = hostname
820- master_port = port
821- master_install_order = inst_ordr
822-
823- # Initiate the replset
824- init_replset("%s:%s" % (master_hostname, master_port))
825-
826- # Add the rest of the nodes to the replset
827- for member in relations_of_type('replica-set'):
828- hostname = relation_get('hostname', member['__unit__'])
829- port = relation_get('port', member['__unit__'])
830- if master_hostname != hostname:
831- if hostname == my_hostname:
832- subprocess.call(['mongo', '--eval',
833- "rs.add(\"%s\")" % hostname])
834- else:
835- join_replset("%s:%s" % (master_hostname, master_port),
836- "%s:%s" % (hostname, port))
837-
838- # Add this node to the replset ( if needed )
839- if master_hostname != my_hostname:
840- join_replset("%s:%s" % (master_hostname, master_port),
841- "%s:%s" % (my_hostname, my_port))
842+ private_address = unit_get('private-address')
843+ remote_hostname = relation_get('hostname')
844+
845+ juju_log('replica_set_relation_changed-start')
846+ juju_log('local unit: %s, joining_unit: %s' % (private_address,
847+ remote_hostname),
848+ level=DEBUG)
849+
850+ if remote_hostname is None:
851+ juju_log('Joiner not ready yet... bailing out')
852+ return
853+
854+ # Initialize the replicaset - we do this only on the oldest unit in replset
855+ # TODO: figure a way how to avoid race conditions - when unit/1 actually
856+ # comes up before unit/0 does - happens rarely, but can happen
857+ # quickfix - deploy replset with only two units, use 'add-unit' to
858+ # add the rest
859+ if oldest_peer(peer_units('replica-set')):
860+ juju_log('Initializing replicaset')
861+ init_replset()
862+
863+ unit = "%s:%s" % (private_address, config('port'))
864+ unit_remote = "%s:%s" % (remote_hostname, relation_get('port'))
865+
866+ # If this is primary, add joined unit to replicaset
867+ if am_i_primary():
868+ juju_log('Adding new secondary... %s' % unit_remote, level=DEBUG)
869+ join_replset(unit, unit_remote)
870+
871+ juju_log('replica_set_relation_changed-finish')
872+
873+
874+@hooks.hook('replicaset-relation-departed')
875+def replica_set_relation_departed():
876+ juju_log('replica_set_relation_departed-start')
877+
878+ if not am_i_primary():
879+ juju_log('replica_set_relation_departed-finish')
880+ return
881+
882+ unit = "%s:%s" % (unit_get('private-address'),
883+ config('port'))
884+ unit_remote = "%s:%s" % (relation_get('hostname'),
885+ relation_get('port'))
886+ leave_replset(unit, unit_remote)
887+ juju_log('Removed %s from replicaset' % unit_remote)
888+
889+ juju_log('replica_set_relation_departed-finish')
890
891
892 @hooks.hook('data-relation-joined')
893
894=== added symlink 'hooks/replica-set-relation-departed'
895=== target is u'hooks.py'
896=== added file 'setup.cfg'
897--- setup.cfg 1970-01-01 00:00:00 +0000
898+++ setup.cfg 2015-01-19 22:37:53 +0000
899@@ -0,0 +1,6 @@
900+[nosetests]
901+verbosity=2
902+with-coverage=1
903+cover-erase=1
904+cover-package=hooks
905+
906
907=== modified file 'test_requirements.txt'
908--- test_requirements.txt 2014-12-09 16:08:35 +0000
909+++ test_requirements.txt 2015-01-19 22:37:53 +0000
910@@ -2,4 +2,5 @@
911 mock>=1.0.1
912 nose>=1.3.1
913 flake8
914+portalocker
915
916
917=== added file 'tests/00_setup.sh'
918--- tests/00_setup.sh 1970-01-01 00:00:00 +0000
919+++ tests/00_setup.sh 2015-01-19 22:37:53 +0000
920@@ -0,0 +1,9 @@
921+#!/bin/bash
922+
923+set -e
924+
925+sudo apt-get install python-setuptools -y
926+sudo add-apt-repository ppa:juju/stable -y
927+
928+sudo apt-get update
929+sudo apt-get install amulet python3 python3-requests python3-pymongo juju-core charm-tools python-mock python-pymongo -y
930
931=== removed file 'tests/00_setup.sh'
932--- tests/00_setup.sh 2014-12-09 15:37:16 +0000
933+++ tests/00_setup.sh 1970-01-01 00:00:00 +0000
934@@ -1,9 +0,0 @@
935-#!/bin/bash
936-
937-set -e
938-
939-sudo apt-get install python-setuptools -y
940-sudo add-apt-repository ppa:juju/stable -y
941-
942-sudo apt-get update
943-sudo apt-get install amulet python3 python3-requests python3-pymongo juju-core charm-tools python-mock python-pymongo -y
944
945=== added file 'tests/03_deploy_replicaset.py'
946--- tests/03_deploy_replicaset.py 1970-01-01 00:00:00 +0000
947+++ tests/03_deploy_replicaset.py 2015-01-19 22:37:53 +0000
948@@ -0,0 +1,114 @@
949+#!/usr/bin/env python3
950+
951+import amulet
952+import requests
953+import time
954+from pymongo import MongoClient
955+from collections import Counter
956+
957+
958+#########################################################
959+# Test Quick Config
960+#########################################################
961+scale = 3
962+seconds = 900
963+
964+# amount of time to wait before testing for replicaset
965+# status
966+wait_for_replicaset = 15
967+
968+#########################################################
969+# 3shard cluster configuration
970+#########################################################
971+d = amulet.Deployment(series='trusty')
972+
973+d.add('mongodb', charm='mongodb', units=scale)
974+d.expose('mongodb')
975+
976+# Perform the setup for the deployment.
977+try:
978+ d.setup(seconds)
979+ d.sentry.wait(seconds)
980+except amulet.helpers.TimeoutError:
981+ message = 'The environment did not setup in %d seconds.', seconds
982+ amulet.raise_status(amulet.SKIP, msg=message)
983+except:
984+ raise
985+
986+sentry_dict = {
987+ 'mongodb0-sentry': d.sentry.unit['mongodb/0'],
988+ 'mongodb1-sentry': d.sentry.unit['mongodb/1'],
989+ 'mongodb2-sentry': d.sentry.unit['mongodb/2'],
990+}
991+
992+
993+#############################################################
994+# Check presence of MongoDB GUI HEALTH Status
995+#############################################################
996+def validate_status_interface():
997+ r = requests.get("http://{}:28017".format(
998+ d.sentry.unit['mongodb/0'].info['public-address']),
999+ verify=False)
1000+ r.raise_for_status
1001+
1002+
1003+#############################################################
1004+# Validate that each unit has an active mongo service
1005+#############################################################
1006+def validate_running_services():
1007+ for service in sentry_dict:
1008+ output = sentry_dict[service].run('service mongodb status')
1009+ service_active = str(output).find('mongodb start/running')
1010+ if service_active == -1:
1011+ message = "Failed to find running MongoDB on host {}".format(
1012+ service)
1013+ amulet.raise_status(amulet.SKIP, msg=message)
1014+
1015+
1016+#############################################################
1017+# Validate proper replicaset setup
1018+#############################################################
1019+def validate_replicaset_setup():
1020+
1021+ time.sleep(wait_for_replicaset)
1022+
1023+ unit_status = []
1024+
1025+ for service in sentry_dict:
1026+ client = MongoClient(sentry_dict[service].info['public-address'])
1027+ r = client.admin.command('replSetGetStatus')
1028+ unit_status.append(r['myState'])
1029+ client.close()
1030+
1031+ primaries = Counter(unit_status)[1]
1032+ if primaries != 1:
1033+ message = "Only one PRIMARY unit allowed! Found: %s" % (primaries)
1034+ amulet.raise_status(amulet.FAIL, message)
1035+
1036+ secondrs = Counter(unit_status)[2]
1037+ if secondrs != 2:
1038+ message = "Only two SECONDARY units allowed! (Found %s)" % (secondrs)
1039+ amulet.raise_status(amulet.FAIL, message)
1040+
1041+
1042+#############################################################
1043+# Validate connectivity from $WORLD
1044+#############################################################
1045+def validate_world_connectivity():
1046+ client = MongoClient(d.sentry.unit['mongodb/0'].info['public-address'])
1047+
1048+ db = client['test']
1049+ # Can we successfully insert?
1050+ insert_id = db.amulet.insert({'assert': True})
1051+ if insert_id is None:
1052+ amulet.raise_status(amulet.FAIL, msg="Failed to insert test data")
1053+ # Can we delete from a shard using the Mongos hub?
1054+ result = db.amulet.remove(insert_id)
1055+ if result['err'] is not None:
1056+ amulet.raise_status(amulet.FAIL, msg="Failed to remove test data")
1057+
1058+
1059+validate_status_interface()
1060+validate_running_services()
1061+validate_replicaset_setup()
1062+validate_world_connectivity()
1063
1064=== added file 'unit_tests/__init__.py'
1065--- unit_tests/__init__.py 1970-01-01 00:00:00 +0000
1066+++ unit_tests/__init__.py 2015-01-19 22:37:53 +0000
1067@@ -0,0 +1,2 @@
1068+import sys
1069+sys.path.append('hooks')
1070
1071=== added file 'unit_tests/test_hooks.py'
1072--- unit_tests/test_hooks.py 1970-01-01 00:00:00 +0000
1073+++ unit_tests/test_hooks.py 2015-01-19 22:37:53 +0000
1074@@ -0,0 +1,341 @@
1075+from mock import patch, call
1076+
1077+import hooks
1078+
1079+from test_utils import CharmTestCase
1080+from pymongo.errors import OperationFailure
1081+from subprocess import CalledProcessError
1082+
1083+# Defines a set of functions to patch on the hooks object. Any of these
1084+# methods will be patched by default on the default invocations of the
1085+# hooks.some_func(). Invoking the the interface change relations will cause
1086+# the hooks context to be created outside of the normal mockery.
1087+TO_PATCH = [
1088+ 'relation_id',
1089+ 'relation_get',
1090+ 'relation_set',
1091+ 'unit_get',
1092+ 'juju_log',
1093+ 'config',
1094+]
1095+
1096+
1097+class MongoHooksTest(CharmTestCase):
1098+
1099+ def setUp(self):
1100+ super(MongoHooksTest, self).setUp(hooks, TO_PATCH)
1101+
1102+ # The self.config object can be used for direct invocations of the
1103+ # hooks methods. The side_effect of invoking the config object within
1104+ # the hooks object will return the value that is set in the test case's
1105+ # test_config dictionary
1106+ self.config.side_effect = self.test_config.get
1107+ self.relation_get.side_effect = self.test_relation.get
1108+
1109+ @patch.object(hooks, 'restart_mongod')
1110+ @patch.object(hooks, 'enable_replset')
1111+ # Note: patching the os.environ dictionary in-line here so there's no
1112+ # additional parameter sent into the function
1113+ @patch.dict('os.environ', JUJU_UNIT_NAME='fake-unit/0')
1114+ def test_replica_set_relation_joined(self, mock_enable_replset,
1115+ mock_restart):
1116+ self.unit_get.return_value = 'private.address'
1117+ self.test_config.set('port', '1234')
1118+ self.test_config.set('replicaset', 'fake-replicaset')
1119+ self.relation_id.return_value = 'fake-relation-id'
1120+
1121+ mock_enable_replset.return_value = False
1122+
1123+ hooks.replica_set_relation_joined()
1124+
1125+ # Verify that mongodb was NOT restarted since the replicaset we claimed
1126+ # was not enabled.
1127+ self.assertFalse(mock_restart.called)
1128+
1129+ exp_rel_vals = {'hostname': 'private.address',
1130+ 'port': '1234',
1131+ 'replset': 'fake-replicaset',
1132+ 'install-order': '0',
1133+ 'type': 'replset'}
1134+ # Check that the relation data was set as we expect it to be set.
1135+ self.relation_set.assert_called_with('fake-relation-id', exp_rel_vals)
1136+
1137+ mock_enable_replset.reset_mock()
1138+ self.relation_set.reset_mock()
1139+ mock_enable_replset.return_value = True
1140+
1141+ hooks.replica_set_relation_joined()
1142+
1143+ self.assertTrue(mock_restart.called)
1144+ self.relation_set.assert_called_with('fake-relation-id', exp_rel_vals)
1145+
1146+ @patch.object(hooks, 'run_admin_command')
1147+ @patch.object(hooks, 'MongoClient')
1148+ @patch.object(hooks, 'config')
1149+ @patch.object(hooks, 'mongo_client')
1150+ @patch('time.sleep')
1151+ def test_init_repl_set(self, mock_sleep, mock_mongo_client_fn,
1152+ mock_config, mock_mongo_client,
1153+ mock_run_admin_command):
1154+ mock_mongo_client_fn.return_value = False
1155+
1156+ mock_config.return_value = {'replicaset': 'foo',
1157+ 'private-address': 'mongo.local',
1158+ 'port': '12345'}
1159+
1160+ # Put the OK state (1) at the end and check the loop.
1161+ ret_values = [{'myState': x} for x in [0, 2, 5, 1]]
1162+ mock_run_admin_command.side_effect = ret_values
1163+
1164+ hooks.init_replset()
1165+
1166+ mock_run_admin_command.assert_called()
1167+ self.assertEqual(len(ret_values), mock_run_admin_command.call_count)
1168+ self.assertEqual(len(ret_values) + 1, mock_sleep.call_count)
1169+
1170+ mock_run_admin_command.reset_mock()
1171+ exc = [OperationFailure('Received replSetInitiate'),
1172+ OperationFailure('unhandled')]
1173+ mock_run_admin_command.side_effect = exc
1174+
1175+ try:
1176+ hooks.init_replset()
1177+ self.assertTrue(False, msg="Expected error")
1178+ except OperationFailure:
1179+ pass
1180+
1181+ mock_run_admin_command.assert_called()
1182+ self.assertEqual(2, mock_run_admin_command.call_count)
1183+
1184+ @patch.object(hooks, 'mongo_client_smart')
1185+ def test_join_replset(self, mock_mongo_client):
1186+ hooks.join_replset()
1187+ self.assertFalse(mock_mongo_client.called)
1188+
1189+ mock_mongo_client.reset_mock()
1190+ hooks.join_replset(master_node='mongo.local')
1191+ self.assertFalse(mock_mongo_client.called)
1192+
1193+ mock_mongo_client.reset_mock()
1194+ hooks.join_replset(host='fake-host')
1195+ self.assertFalse(mock_mongo_client.called)
1196+
1197+ mock_mongo_client.reset_mock()
1198+ hooks.join_replset(master_node='mongo.local', host='fake-host')
1199+ mock_mongo_client.assert_called_with('localhost',
1200+ 'rs.add("fake-host")')
1201+
1202+ @patch.object(hooks, 'mongo_client')
1203+ def test_leave_replset(self, mock_mongo_client):
1204+ hooks.leave_replset()
1205+ self.assertFalse(mock_mongo_client.called)
1206+
1207+ mock_mongo_client.reset_mock()
1208+ hooks.leave_replset(master_node='mongo.local')
1209+ self.assertFalse(mock_mongo_client.called)
1210+
1211+ mock_mongo_client.reset_mock()
1212+ hooks.leave_replset(host='fake-host')
1213+ self.assertFalse(mock_mongo_client.called)
1214+
1215+ mock_mongo_client.reset_mock()
1216+ hooks.leave_replset('mongo.local', 'fake-host')
1217+ mock_mongo_client.assert_called_with('mongo.local',
1218+ 'rs.remove("fake-host")')
1219+
1220+ @patch.object(hooks, 'apt_install')
1221+ @patch.object(hooks, 'apt_update')
1222+ @patch.object(hooks, 'add_source')
1223+ @patch.dict('os.environ', CHARM_DIR='/tmp/charm/dir')
1224+ def test_install_hook(self, mock_add_source, mock_apt_update,
1225+ mock_apt_install):
1226+ self.test_config.set('source', 'fake-source')
1227+ self.test_config.set('key', 'fake-key')
1228+
1229+ hooks.install_hook()
1230+ mock_add_source.assert_called_with('fake-source', 'fake-key')
1231+ mock_apt_update.assert_called_with(fatal=True)
1232+ mock_apt_install.assert_called_with(packages=hooks.INSTALL_PACKAGES,
1233+ fatal=True)
1234+
1235+ @patch.object(hooks, 'run_admin_command')
1236+ @patch.object(hooks, 'MongoClient')
1237+ @patch('time.sleep')
1238+ def test_am_i_primary(self, mock_sleep, mock_mongo_client,
1239+ mock_run_admin_cmd):
1240+ mock_run_admin_cmd.side_effect = [{'myState': x} for x in xrange(5)]
1241+ expected_results = [True if x == 1 else False for x in xrange(5)]
1242+
1243+ # Check expected return values each time...
1244+ for exp in expected_results:
1245+ rv = hooks.am_i_primary()
1246+ self.assertEqual(exp, rv)
1247+
1248+ @patch.object(hooks, 'run_admin_command')
1249+ @patch.object(hooks, 'MongoClient')
1250+ @patch('time.sleep')
1251+ def test_am_i_primary_too_many_attempts(self, mock_sleep,
1252+ mock_mongo_client,
1253+ mock_run_admin_cmd):
1254+ msg = 'replSetInitiate - should come online shortly'
1255+ mock_run_admin_cmd.side_effect = [OperationFailure(msg)
1256+ for x in xrange(10)]
1257+
1258+ try:
1259+ hooks.am_i_primary()
1260+ self.assertTrue(False, 'Expected failure.')
1261+ except hooks.TimeoutException:
1262+ self.assertEqual(mock_run_admin_cmd.call_count, 10)
1263+ pass
1264+
1265+ @patch.object(hooks, 'run_admin_command')
1266+ @patch.object(hooks, 'MongoClient')
1267+ @patch('time.sleep')
1268+ def test_am_i_primary_operation_failures(self, mock_sleep,
1269+ mock_mongo_client,
1270+ mock_run_admin_cmd):
1271+ mock_run_admin_cmd.side_effect = OperationFailure('EMPTYCONFIG')
1272+
1273+ rv = hooks.am_i_primary()
1274+ mock_run_admin_cmd.assert_called()
1275+ self.assertFalse(rv)
1276+
1277+ mock_run_admin_cmd.reset_mock()
1278+ mock_run_admin_cmd.side_effect = OperationFailure('unexpected failure')
1279+ try:
1280+ hooks.am_i_primary()
1281+ self.assertFalse(True, "Expected OperationFailure to be raised")
1282+ except OperationFailure:
1283+ mock_run_admin_cmd.assert_called()
1284+
1285+ @patch('time.sleep')
1286+ @patch('subprocess.check_output')
1287+ def test_mongo_client_smart_no_command(self, mock_check_output,
1288+ mock_sleep):
1289+ rv = hooks.mongo_client_smart()
1290+ self.assertFalse(rv)
1291+ self.assertEqual(0, mock_check_output.call_count)
1292+
1293+ mock_check_output.reset_mock()
1294+ mock_check_output.return_value = '{"ok": 1}'
1295+
1296+ rv = hooks.mongo_client_smart(command='fake-cmd')
1297+ self.assertTrue(rv)
1298+ mock_check_output.assert_called_once_with(['mongo', '--quiet',
1299+ '--host', 'localhost',
1300+ '--eval',
1301+ 'printjson(fake-cmd)'])
1302+
1303+ @patch('time.sleep')
1304+ @patch('subprocess.check_output')
1305+ def test_mongo_client_smart_error_cases(self, mock_ck_output, mock_sleep):
1306+ mock_ck_output.side_effect = [CalledProcessError(1, 'cmd',
1307+ output='fake-error')
1308+ for x in xrange(11)]
1309+ rv = hooks.mongo_client_smart(command='fake-cmd')
1310+ self.assertFalse(rv)
1311+
1312+ @patch('subprocess.call')
1313+ def test_mongo_client(self, mock_subprocess):
1314+ rv = hooks.mongo_client()
1315+ self.assertFalse(rv)
1316+ self.assertEqual(0, mock_subprocess.call_count)
1317+
1318+ mock_subprocess.reset_mock()
1319+ rv = hooks.mongo_client(host='fake-host')
1320+ self.assertFalse(rv)
1321+ self.assertEqual(0, mock_subprocess.call_count)
1322+
1323+ mock_subprocess.reset_mock()
1324+ rv = hooks.mongo_client(command='fake-command')
1325+ self.assertFalse(rv)
1326+ self.assertEqual(0, mock_subprocess.call_count)
1327+
1328+ mock_subprocess.reset_mock()
1329+ mock_subprocess.return_value = 0
1330+ rv = hooks.mongo_client(host='fake-host', command='fake-command')
1331+ expected_cmd = ("mongo --host %s --eval 'printjson(%s)'"
1332+ % ('fake-host', 'fake-command'))
1333+ mock_subprocess.assert_called_once_with(expected_cmd, shell=True)
1334+ self.assertTrue(rv)
1335+
1336+ mock_subprocess.reset_mock()
1337+ mock_subprocess.return_value = 1
1338+ rv = hooks.mongo_client(host='fake-host', command='fake-command')
1339+ expected_cmd = ("mongo --host %s --eval 'printjson(%s)'"
1340+ % ('fake-host', 'fake-command'))
1341+ mock_subprocess.assert_called_once_with(expected_cmd, shell=True)
1342+ self.assertFalse(rv)
1343+
1344+ @patch.object(hooks, 'am_i_primary')
1345+ @patch.object(hooks, 'init_replset')
1346+ @patch.object(hooks, 'relation_get')
1347+ @patch.object(hooks, 'peer_units')
1348+ @patch.object(hooks, 'oldest_peer')
1349+ @patch.object(hooks, 'join_replset')
1350+ @patch.object(hooks, 'unit_get')
1351+ def test_replica_set_relation_changed(self, mock_unit_get,
1352+ mock_join_replset, mock_oldest_peer,
1353+ mock_peer_units, mock_relation_get,
1354+ mock_init_replset, mock_is_primary):
1355+ # set the unit_get('private-address')
1356+ mock_unit_get.return_value = 'juju-local-unit-0.local'
1357+ mock_relation_get.return_value = None
1358+
1359+ # Test when remote hostname is None, should not join
1360+ hooks.replica_set_relation_changed()
1361+ self.assertEqual(0, mock_join_replset.call_count)
1362+
1363+ # Test remote hostname is valid, but master is somehow not defined
1364+ mock_join_replset.reset_mock()
1365+ mock_relation_get.return_value = 'juju-local-unit-0'
1366+
1367+ hooks.replica_set_relation_changed()
1368+
1369+ self.assertEqual(1, mock_join_replset.call_count)
1370+
1371+ # Test when not oldest peer, don't init replica set
1372+ mock_init_replset.reset_mock()
1373+ mock_oldest_peer.reset_mock()
1374+ mock_peer_units.return_value = ['mongodb/1', 'mongodb/2']
1375+ mock_oldest_peer.return_value = False
1376+
1377+ hooks.replica_set_relation_changed()
1378+
1379+ self.assertEqual(mock_init_replset.call_count, 0)
1380+
1381+ # Test when its also the PRIMARY
1382+ mock_relation_get.reset_mock()
1383+ mock_relation_get.side_effect = ['juju-remote-unit-0', '12345']
1384+ mock_oldest_peer.reset_mock()
1385+ mock_oldest_peer.return_value = False
1386+ mock_is_primary.reset_mock()
1387+ mock_is_primary.return_value = True
1388+
1389+ hooks.replica_set_relation_changed()
1390+ call1 = call('juju-local-unit-0.local:27017',
1391+ 'juju-remote-unit-0:12345')
1392+ mock_join_replset.assert_has_calls(call1)
1393+
1394+ @patch.object(hooks, 'unit_get')
1395+ @patch.object(hooks, 'leave_replset')
1396+ @patch.object(hooks, 'am_i_primary')
1397+ def test_replica_set_relation_departed(self, mock_am_i_primary,
1398+ mock_leave_replset, mock_unit_get):
1399+ mock_am_i_primary.return_value = False
1400+ hooks.replica_set_relation_departed()
1401+
1402+ self.assertEqual(0, mock_leave_replset.call_count)
1403+
1404+ mock_am_i_primary.reset_mock()
1405+ mock_am_i_primary.return_value = True
1406+ mock_unit_get.return_value = 'juju-local'
1407+
1408+ self.test_relation.set({'hostname': 'juju-remote',
1409+ 'port': '27017'})
1410+ mock_leave_replset.reset_mock()
1411+
1412+ hooks.replica_set_relation_departed()
1413+
1414+ call1 = call('juju-local:27017', 'juju-remote:27017')
1415+ mock_leave_replset.assert_has_calls(call1)
1416
1417=== added file 'unit_tests/test_utils.py'
1418--- unit_tests/test_utils.py 1970-01-01 00:00:00 +0000
1419+++ unit_tests/test_utils.py 2015-01-19 22:37:53 +0000
1420@@ -0,0 +1,111 @@
1421+import logging
1422+import unittest
1423+import os
1424+import yaml
1425+import io
1426+
1427+from contextlib import contextmanager
1428+from mock import patch
1429+
1430+
1431+@contextmanager
1432+def mock_open(filename, contents=None):
1433+ ''' Slightly simpler mock of open to return contents for filename '''
1434+ def mock_file(*args):
1435+ if args[0] == filename:
1436+ return io.StringIO(contents)
1437+ else:
1438+ return open(*args)
1439+ with patch('__builtin__.open', mock_file):
1440+ yield
1441+
1442+
1443+def load_config():
1444+ '''
1445+ Walk backwords from __file__ looking for config.yaml, load and return the
1446+ 'options' section'
1447+ '''
1448+ config = None
1449+ f = __file__
1450+ while config is None:
1451+ d = os.path.dirname(f)
1452+ if os.path.isfile(os.path.join(d, 'config.yaml')):
1453+ config = os.path.join(d, 'config.yaml')
1454+ break
1455+ f = d
1456+
1457+ if not config:
1458+ logging.error('Could not find config.yaml in any parent directory '
1459+ 'of %s. ' % file)
1460+ raise Exception
1461+
1462+ return yaml.safe_load(open(config).read())['options']
1463+
1464+
1465+def get_default_config():
1466+ '''
1467+ Load default charm config from config.yaml return as a dict.
1468+ If no default is set in config.yaml, its value is None.
1469+ '''
1470+ default_config = {}
1471+ config = load_config()
1472+ for k, v in config.iteritems():
1473+ if 'default' in v:
1474+ default_config[k] = v['default']
1475+ else:
1476+ default_config[k] = None
1477+ return default_config
1478+
1479+
1480+class CharmTestCase(unittest.TestCase):
1481+ def setUp(self, obj, patches):
1482+ super(CharmTestCase, self).setUp()
1483+ self.patches = patches
1484+ self.obj = obj
1485+ self.test_config = TestConfig()
1486+ self.test_relation = TestRelation()
1487+ self.patch_all()
1488+
1489+ def patch(self, method):
1490+ _m = patch.object(self.obj, method)
1491+ mock = _m.start()
1492+ self.addCleanup(_m.stop)
1493+ return mock
1494+
1495+ def patch_all(self):
1496+ for method in self.patches:
1497+ setattr(self, method, self.patch(method))
1498+
1499+
1500+class TestConfig(object):
1501+ def __init__(self):
1502+ self.config = get_default_config()
1503+
1504+ def get(self, attr):
1505+ try:
1506+ return self.config[attr]
1507+ except KeyError:
1508+ return None
1509+
1510+ def get_all(self):
1511+ return self.config
1512+
1513+ def set(self, attr, value):
1514+ if attr not in self.config:
1515+ raise KeyError
1516+ self.config[attr] = value
1517+
1518+
1519+class TestRelation(object):
1520+ def __init__(self, relation_data={}):
1521+ self.relation_data = relation_data
1522+
1523+ def set(self, relation_data):
1524+ self.relation_data = relation_data
1525+
1526+ def get(self, attr=None, unit=None, rid=None):
1527+ if attr is None:
1528+ return self.relation_data
1529+ elif attr in self.relation_data:
1530+ return self.relation_data.get(attr)
1531+ return None

Subscribers

People subscribed via source and target branches