Merge lp:~niedbalski/charms/trusty/rabbitmq-server/fix-1442443 into lp:charms/trusty/rabbitmq-server
- Trusty Tahr (14.04)
- fix-1442443
- Merge into trunk
Status: | Superseded | ||||
---|---|---|---|---|---|
Proposed branch: | lp:~niedbalski/charms/trusty/rabbitmq-server/fix-1442443 | ||||
Merge into: | lp:charms/trusty/rabbitmq-server | ||||
Diff against target: |
746 lines (+373/-168) 11 files modified
config.yaml (+7/-1) hooks/rabbit_utils.py (+37/-39) hooks/rabbitmq_context.py (+125/-0) hooks/rabbitmq_server_relations.py (+7/-113) hooks/ssl_utils.py (+55/-0) templates/rabbitmq.config (+25/-11) tests/00_setup.sh (+3/-0) tests/50_test_cluster_partition.py (+31/-0) unit_tests/test_rabbit_utils.py (+1/-1) unit_tests/test_rabbitmq_context.py (+79/-0) unit_tests/test_rabbitmq_server_relations.py (+3/-3) |
||||
To merge this branch: | bzr merge lp:~niedbalski/charms/trusty/rabbitmq-server/fix-1442443 | ||||
Related bugs: |
|
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Billy Olsen | Needs Fixing | ||
OpenStack Charmers | Pending | ||
Review via email: mp+255770@code.launchpad.net |
This proposal has been superseded by a proposal from 2015-04-10.
Commit message
Description of the change
- Added the cluster_
- Refactored the ssl handling code.
- Added a generic config loader
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_lint_check #3202 rabbitmq-server for niedbalski mp255770
LINT OK: passed
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_unit_test #2990 rabbitmq-server for niedbalski mp255770
UNIT OK: passed
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_amulet_test #3019 rabbitmq-server for niedbalski mp255770
AMULET FAIL: amulet-test failed
AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.
Full amulet test output: http://
Build: http://
- 89. By Liam Young
-
[gnuoy,trivial] Pre-release charmhelper sync
- 90. By James Page
-
[wolsen,
r=james- page] Fixup SSL tests to not fail when certs expire! - 91. By Jorge Niedbalski
-
Rebase commit
- 92. By Jorge Niedbalski
-
Rebase
- 93. By Jorge Niedbalski
-
addressing @dosaboy comments
- 94. By Jorge Niedbalski
-
addressing @dosaboy comments
- 95. By Jorge Niedbalski
-
addressing @dosaboy comments
- 96. By Jorge Niedbalski
-
addressing comments
- 97. By Jorge Niedbalski
-
Addressed @beisner comments
- 98. By Jorge Niedbalski
-
Addressed @beisner comments
Unmerged revisions
- 98. By Jorge Niedbalski
-
Addressed @beisner comments
- 97. By Jorge Niedbalski
-
Addressed @beisner comments
- 96. By Jorge Niedbalski
-
addressing comments
- 95. By Jorge Niedbalski
-
addressing @dosaboy comments
- 94. By Jorge Niedbalski
-
addressing @dosaboy comments
- 93. By Jorge Niedbalski
-
addressing @dosaboy comments
- 92. By Jorge Niedbalski
-
Rebase
- 91. By Jorge Niedbalski
-
Rebase commit
Preview Diff
1 | === modified file 'config.yaml' | |||
2 | --- config.yaml 2015-03-03 06:05:14 +0000 | |||
3 | +++ config.yaml 2015-04-10 02:40:31 +0000 | |||
4 | @@ -8,7 +8,7 @@ | |||
5 | 8 | type: string | 8 | type: string |
6 | 9 | default: "off" | 9 | default: "off" |
7 | 10 | description: | | 10 | description: | |
9 | 11 | Enable SSL connections on rabbitmq, valid values are 'off', 'on', 'only'. If ssl_key, | 11 | Enable SSL connections on rabbitmq, valid values are 'off', 'on', 'only'. If ssl_key, |
10 | 12 | ssl_cert, ssl_ca are provided then then those values will be used. Otherwise | 12 | ssl_cert, ssl_ca are provided then then those values will be used. Otherwise |
11 | 13 | the service will act as its own certificate authority and pass its ca cert to clients. | 13 | the service will act as its own certificate authority and pass its ca cert to clients. |
12 | 14 | For HA or clustered rabbits ssl key/cert must be provided. | 14 | For HA or clustered rabbits ssl key/cert must be provided. |
13 | @@ -92,6 +92,12 @@ | |||
14 | 92 | description: | | 92 | description: | |
15 | 93 | When set to true the 'ha-mode: all' policy is applied to all the exchages | 93 | When set to true the 'ha-mode: all' policy is applied to all the exchages |
16 | 94 | that match the expression '^(?!amq\.).*' | 94 | that match the expression '^(?!amq\.).*' |
17 | 95 | cluster-partition-handling: | ||
18 | 96 | type: string | ||
19 | 97 | default: ignore | ||
20 | 98 | description: | | ||
21 | 99 | How to respond to cluster partitions. | ||
22 | 100 | See http://www.rabbitmq.com/partitions.html for further details. | ||
23 | 95 | rbd-size: | 101 | rbd-size: |
24 | 96 | type: string | 102 | type: string |
25 | 97 | default: 5G | 103 | default: 5G |
26 | 98 | 104 | ||
27 | === modified file 'hooks/rabbit_utils.py' | |||
28 | --- hooks/rabbit_utils.py 2015-03-24 20:17:01 +0000 | |||
29 | +++ hooks/rabbit_utils.py 2015-04-10 02:40:31 +0000 | |||
30 | @@ -1,6 +1,4 @@ | |||
31 | 1 | import os | 1 | import os |
32 | 2 | import pwd | ||
33 | 3 | import grp | ||
34 | 4 | import re | 2 | import re |
35 | 5 | import socket | 3 | import socket |
36 | 6 | import sys | 4 | import sys |
37 | @@ -8,6 +6,13 @@ | |||
38 | 8 | import glob | 6 | import glob |
39 | 9 | import tempfile | 7 | import tempfile |
40 | 10 | 8 | ||
41 | 9 | from rabbitmq_context import ( | ||
42 | 10 | RabbitMQSSLContext, | ||
43 | 11 | RabbitMQClusterContext | ||
44 | 12 | ) | ||
45 | 13 | |||
46 | 14 | from charmhelpers.core.templating import render | ||
47 | 15 | |||
48 | 11 | from charmhelpers.contrib.openstack.utils import ( | 16 | from charmhelpers.contrib.openstack.utils import ( |
49 | 12 | get_hostname, | 17 | get_hostname, |
50 | 13 | ) | 18 | ) |
51 | @@ -30,8 +35,6 @@ | |||
52 | 30 | cmp_pkgrevno | 35 | cmp_pkgrevno |
53 | 31 | ) | 36 | ) |
54 | 32 | 37 | ||
55 | 33 | from charmhelpers.core.templating import render | ||
56 | 34 | |||
57 | 35 | from charmhelpers.contrib.peerstorage import ( | 38 | from charmhelpers.contrib.peerstorage import ( |
58 | 36 | peer_store, | 39 | peer_store, |
59 | 37 | peer_retrieve | 40 | peer_retrieve |
60 | @@ -57,7 +60,10 @@ | |||
61 | 57 | # the charm doesn't concern itself with template specifics etc. | 60 | # the charm doesn't concern itself with template specifics etc. |
62 | 58 | CONFIG_FILES = OrderedDict([ | 61 | CONFIG_FILES = OrderedDict([ |
63 | 59 | (RABBITMQ_CONF, { | 62 | (RABBITMQ_CONF, { |
65 | 60 | 'hook_contexts': None, | 63 | 'hook_contexts': [ |
66 | 64 | RabbitMQSSLContext(), | ||
67 | 65 | RabbitMQClusterContext(), | ||
68 | 66 | ], | ||
69 | 61 | 'services': ['rabbitmq-server'] | 67 | 'services': ['rabbitmq-server'] |
70 | 62 | }), | 68 | }), |
71 | 63 | (ENV_CONF, { | 69 | (ENV_CONF, { |
72 | @@ -71,6 +77,32 @@ | |||
73 | 71 | ]) | 77 | ]) |
74 | 72 | 78 | ||
75 | 73 | 79 | ||
76 | 80 | class ConfigRenderer(): | ||
77 | 81 | |||
78 | 82 | def __init__(self, config): | ||
79 | 83 | self.config_data = {} | ||
80 | 84 | |||
81 | 85 | for config_path, data in config.items(): | ||
82 | 86 | if 'hook_contexts' in data and data['hook_contexts']: | ||
83 | 87 | ctxt = {} | ||
84 | 88 | for svc_context in data['hook_contexts']: | ||
85 | 89 | ctxt.update(svc_context()) | ||
86 | 90 | self.config_data[config_path] = ctxt | ||
87 | 91 | |||
88 | 92 | def write(self, config_path): | ||
89 | 93 | data = self.config_data.get(config_path, None) | ||
90 | 94 | if data: | ||
91 | 95 | log("writing config file: %s , data: %s" % (config_path, | ||
92 | 96 | str(data)), level='DEBUG') | ||
93 | 97 | |||
94 | 98 | render(os.path.basename(config_path), config_path, | ||
95 | 99 | data, perms=0o644) | ||
96 | 100 | |||
97 | 101 | def write_all(self): | ||
98 | 102 | for service in self.config_data.keys(): | ||
99 | 103 | self.write(service) | ||
100 | 104 | |||
101 | 105 | |||
102 | 74 | class RabbitmqError(Exception): | 106 | class RabbitmqError(Exception): |
103 | 75 | pass | 107 | pass |
104 | 76 | 108 | ||
105 | @@ -390,40 +422,6 @@ | |||
106 | 390 | def disable_plugin(plugin): | 422 | def disable_plugin(plugin): |
107 | 391 | _manage_plugin(plugin, 'disable') | 423 | _manage_plugin(plugin, 'disable') |
108 | 392 | 424 | ||
109 | 393 | ssl_key_file = "/etc/rabbitmq/rabbit-server-privkey.pem" | ||
110 | 394 | ssl_cert_file = "/etc/rabbitmq/rabbit-server-cert.pem" | ||
111 | 395 | ssl_ca_file = "/etc/rabbitmq/rabbit-server-ca.pem" | ||
112 | 396 | |||
113 | 397 | |||
114 | 398 | def enable_ssl(ssl_key, ssl_cert, ssl_port, | ||
115 | 399 | ssl_ca=None, ssl_only=False, ssl_client=None): | ||
116 | 400 | uid = pwd.getpwnam("root").pw_uid | ||
117 | 401 | gid = grp.getgrnam("rabbitmq").gr_gid | ||
118 | 402 | |||
119 | 403 | for contents, path in ( | ||
120 | 404 | (ssl_key, ssl_key_file), | ||
121 | 405 | (ssl_cert, ssl_cert_file), | ||
122 | 406 | (ssl_ca, ssl_ca_file)): | ||
123 | 407 | if not contents: | ||
124 | 408 | continue | ||
125 | 409 | with open(path, 'w') as fh: | ||
126 | 410 | fh.write(contents) | ||
127 | 411 | os.chmod(path, 0o640) | ||
128 | 412 | os.chown(path, uid, gid) | ||
129 | 413 | |||
130 | 414 | data = { | ||
131 | 415 | "ssl_port": ssl_port, | ||
132 | 416 | "ssl_cert_file": ssl_cert_file, | ||
133 | 417 | "ssl_key_file": ssl_key_file, | ||
134 | 418 | "ssl_client": ssl_client, | ||
135 | 419 | "ssl_ca_file": "", | ||
136 | 420 | "ssl_only": ssl_only} | ||
137 | 421 | |||
138 | 422 | if ssl_ca: | ||
139 | 423 | data["ssl_ca_file"] = ssl_ca_file | ||
140 | 424 | |||
141 | 425 | render(os.path.basename(RABBITMQ_CONF), RABBITMQ_CONF, data, perms=0o644) | ||
142 | 426 | |||
143 | 427 | 425 | ||
144 | 428 | def execute(cmd, die=False, echo=False): | 426 | def execute(cmd, die=False, echo=False): |
145 | 429 | """ Executes a command | 427 | """ Executes a command |
146 | 430 | 428 | ||
147 | === added file 'hooks/rabbitmq_context.py' | |||
148 | --- hooks/rabbitmq_context.py 1970-01-01 00:00:00 +0000 | |||
149 | +++ hooks/rabbitmq_context.py 2015-04-10 02:40:31 +0000 | |||
150 | @@ -0,0 +1,125 @@ | |||
151 | 1 | from charmhelpers.contrib.ssl.service import ServiceCA | ||
152 | 2 | |||
153 | 3 | from charmhelpers.core.hookenv import ( | ||
154 | 4 | open_port, | ||
155 | 5 | close_port, | ||
156 | 6 | config, | ||
157 | 7 | log, | ||
158 | 8 | ERROR, | ||
159 | 9 | ) | ||
160 | 10 | |||
161 | 11 | import sys | ||
162 | 12 | import pwd | ||
163 | 13 | import grp | ||
164 | 14 | import os | ||
165 | 15 | import base64 | ||
166 | 16 | |||
167 | 17 | import ssl_utils | ||
168 | 18 | |||
169 | 19 | ssl_key_file = "/etc/rabbitmq/rabbit-server-privkey.pem" | ||
170 | 20 | ssl_cert_file = "/etc/rabbitmq/rabbit-server-cert.pem" | ||
171 | 21 | ssl_ca_file = "/etc/rabbitmq/rabbit-server-ca.pem" | ||
172 | 22 | |||
173 | 23 | |||
174 | 24 | def convert_from_base64(v): | ||
175 | 25 | # Rabbit originally supported pem encoded key/cert in config, play | ||
176 | 26 | # nice on upgrades as we now expect base64 encoded key/cert/ca. | ||
177 | 27 | if not v: | ||
178 | 28 | return v | ||
179 | 29 | if v.startswith('-----BEGIN'): | ||
180 | 30 | return v | ||
181 | 31 | try: | ||
182 | 32 | return base64.b64decode(v) | ||
183 | 33 | except TypeError: | ||
184 | 34 | return v | ||
185 | 35 | |||
186 | 36 | |||
187 | 37 | class RabbitMQSSLContext(object): | ||
188 | 38 | |||
189 | 39 | def enable_ssl(self, ssl_key, ssl_cert, ssl_port, | ||
190 | 40 | ssl_ca=None, ssl_only=False, ssl_client=None): | ||
191 | 41 | |||
192 | 42 | uid = pwd.getpwnam("root").pw_uid | ||
193 | 43 | gid = grp.getgrnam("rabbitmq").gr_gid | ||
194 | 44 | |||
195 | 45 | for contents, path in ( | ||
196 | 46 | (ssl_key, ssl_key_file), | ||
197 | 47 | (ssl_cert, ssl_cert_file), | ||
198 | 48 | (ssl_ca, ssl_ca_file)): | ||
199 | 49 | |||
200 | 50 | if not contents: | ||
201 | 51 | continue | ||
202 | 52 | |||
203 | 53 | with open(path, 'w') as fh: | ||
204 | 54 | fh.write(contents) | ||
205 | 55 | |||
206 | 56 | os.chmod(path, 0o640) | ||
207 | 57 | os.chown(path, uid, gid) | ||
208 | 58 | |||
209 | 59 | data = { | ||
210 | 60 | "ssl_port": ssl_port, | ||
211 | 61 | "ssl_cert_file": ssl_cert_file, | ||
212 | 62 | "ssl_key_file": ssl_key_file, | ||
213 | 63 | "ssl_client": ssl_client, | ||
214 | 64 | "ssl_ca_file": "", | ||
215 | 65 | "ssl_only": ssl_only | ||
216 | 66 | } | ||
217 | 67 | |||
218 | 68 | if ssl_ca: | ||
219 | 69 | data["ssl_ca_file"] = ssl_ca_file | ||
220 | 70 | |||
221 | 71 | return data | ||
222 | 72 | |||
223 | 73 | def __call__(self): | ||
224 | 74 | """ | ||
225 | 75 | The legacy config support adds some additional complications. | ||
226 | 76 | |||
227 | 77 | ssl_enabled = True, ssl = off -> ssl enabled | ||
228 | 78 | ssl_enabled = False, ssl = on -> ssl enabled | ||
229 | 79 | """ | ||
230 | 80 | ssl_mode, external_ca = ssl_utils.get_ssl_mode() | ||
231 | 81 | |||
232 | 82 | ctxt = { | ||
233 | 83 | 'ssl_mode': ssl_mode, | ||
234 | 84 | } | ||
235 | 85 | |||
236 | 86 | if ssl_mode == 'off': | ||
237 | 87 | close_port(config('ssl_port')) | ||
238 | 88 | ssl_utils.reconfigure_client_ssl() | ||
239 | 89 | return ctxt | ||
240 | 90 | |||
241 | 91 | ssl_key = convert_from_base64(config('ssl_key')) | ||
242 | 92 | ssl_cert = convert_from_base64(config('ssl_cert')) | ||
243 | 93 | ssl_ca = convert_from_base64(config('ssl_ca')) | ||
244 | 94 | ssl_port = config('ssl_port') | ||
245 | 95 | |||
246 | 96 | # If external managed certs then we need all the fields. | ||
247 | 97 | if ( | ||
248 | 98 | ssl_mode in ('on', 'only') and | ||
249 | 99 | any((ssl_key, ssl_cert)) and | ||
250 | 100 | not all((ssl_key, ssl_cert)) | ||
251 | 101 | ): | ||
252 | 102 | log('If ssl_key or ssl_cert are specified both are required.', | ||
253 | 103 | level=ERROR) | ||
254 | 104 | sys.exit(1) | ||
255 | 105 | |||
256 | 106 | if not external_ca: | ||
257 | 107 | ssl_cert, ssl_key, ssl_ca = ServiceCA.get_service_cert() | ||
258 | 108 | |||
259 | 109 | ctxt.update(self.enable_ssl( | ||
260 | 110 | ssl_key, ssl_cert, ssl_port, ssl_ca, | ||
261 | 111 | ssl_only=(ssl_mode == "only"), ssl_client=False | ||
262 | 112 | )) | ||
263 | 113 | |||
264 | 114 | ssl_utils.reconfigure_client_ssl(True) | ||
265 | 115 | open_port(ssl_port) | ||
266 | 116 | |||
267 | 117 | return ctxt | ||
268 | 118 | |||
269 | 119 | |||
270 | 120 | class RabbitMQClusterContext(object): | ||
271 | 121 | |||
272 | 122 | def __call__(self): | ||
273 | 123 | return { | ||
274 | 124 | 'cluster_partition_handling': config('cluster-partition-handling'), | ||
275 | 125 | } | ||
276 | 0 | 126 | ||
277 | === modified file 'hooks/rabbitmq_server_relations.py' | |||
278 | --- hooks/rabbitmq_server_relations.py 2015-03-06 06:39:19 +0000 | |||
279 | +++ hooks/rabbitmq_server_relations.py 2015-04-10 02:40:31 +0000 | |||
280 | @@ -1,5 +1,5 @@ | |||
281 | 1 | #!/usr/bin/python | 1 | #!/usr/bin/python |
283 | 2 | import base64 | 2 | |
284 | 3 | import os | 3 | import os |
285 | 4 | import shutil | 4 | import shutil |
286 | 5 | import sys | 5 | import sys |
287 | @@ -8,6 +8,8 @@ | |||
288 | 8 | import socket | 8 | import socket |
289 | 9 | 9 | ||
290 | 10 | import rabbit_utils as rabbit | 10 | import rabbit_utils as rabbit |
291 | 11 | import ssl_utils | ||
292 | 12 | |||
293 | 11 | from lib.utils import ( | 13 | from lib.utils import ( |
294 | 12 | chown, chmod, | 14 | chown, chmod, |
295 | 13 | is_newer, | 15 | is_newer, |
296 | @@ -46,7 +48,6 @@ | |||
297 | 46 | related_units, | 48 | related_units, |
298 | 47 | service_name, | 49 | service_name, |
299 | 48 | local_unit, | 50 | local_unit, |
300 | 49 | relations_of_type, | ||
301 | 50 | config, | 51 | config, |
302 | 51 | unit_get, | 52 | unit_get, |
303 | 52 | is_relation_made, | 53 | is_relation_made, |
304 | @@ -61,7 +62,6 @@ | |||
305 | 61 | service_restart, | 62 | service_restart, |
306 | 62 | ) | 63 | ) |
307 | 63 | from charmhelpers.contrib.charmsupport import nrpe | 64 | from charmhelpers.contrib.charmsupport import nrpe |
308 | 64 | from charmhelpers.contrib.ssl.service import ServiceCA | ||
309 | 65 | 65 | ||
310 | 66 | from charmhelpers.contrib.peerstorage import ( | 66 | from charmhelpers.contrib.peerstorage import ( |
311 | 67 | peer_echo, | 67 | peer_echo, |
312 | @@ -176,7 +176,7 @@ | |||
313 | 176 | get_address_in_network(config('access-network'), | 176 | get_address_in_network(config('access-network'), |
314 | 177 | unit_get('private-address')) | 177 | unit_get('private-address')) |
315 | 178 | 178 | ||
317 | 179 | configure_client_ssl(relation_settings) | 179 | ssl_utils.configure_client_ssl(relation_settings) |
318 | 180 | 180 | ||
319 | 181 | if is_clustered(): | 181 | if is_clustered(): |
320 | 182 | relation_settings['clustered'] = 'true' | 182 | relation_settings['clustered'] = 'true' |
321 | @@ -302,19 +302,6 @@ | |||
322 | 302 | service_restart('rabbitmq-server') | 302 | service_restart('rabbitmq-server') |
323 | 303 | 303 | ||
324 | 304 | 304 | ||
325 | 305 | @hooks.hook('cluster-relation-departed') | ||
326 | 306 | def cluster_departed(): | ||
327 | 307 | if is_relation_made('ha') and \ | ||
328 | 308 | config('ha-vip-only') is False: | ||
329 | 309 | log('hacluster relation is present, skipping native ' | ||
330 | 310 | 'rabbitmq cluster config.') | ||
331 | 311 | return | ||
332 | 312 | if not is_newer(): | ||
333 | 313 | log('cluster_joined: Relation lesser.') | ||
334 | 314 | return | ||
335 | 315 | rabbit.break_cluster() | ||
336 | 316 | |||
337 | 317 | |||
338 | 318 | @hooks.hook('ha-relation-joined') | 305 | @hooks.hook('ha-relation-joined') |
339 | 319 | def ha_joined(): | 306 | def ha_joined(): |
340 | 320 | corosync_bindiface = config('ha-bindiface') | 307 | corosync_bindiface = config('ha-bindiface') |
341 | @@ -526,103 +513,10 @@ | |||
342 | 526 | MAN_PLUGIN = 'rabbitmq_management' | 513 | MAN_PLUGIN = 'rabbitmq_management' |
343 | 527 | 514 | ||
344 | 528 | 515 | ||
345 | 529 | def configure_client_ssl(relation_data): | ||
346 | 530 | """Configure client with ssl | ||
347 | 531 | """ | ||
348 | 532 | ssl_mode, external_ca = _get_ssl_mode() | ||
349 | 533 | if ssl_mode == 'off': | ||
350 | 534 | return | ||
351 | 535 | relation_data['ssl_port'] = config('ssl_port') | ||
352 | 536 | if external_ca: | ||
353 | 537 | if config('ssl_ca'): | ||
354 | 538 | relation_data['ssl_ca'] = base64.b64encode( | ||
355 | 539 | config('ssl_ca')) | ||
356 | 540 | return | ||
357 | 541 | ca = ServiceCA.get_ca() | ||
358 | 542 | relation_data['ssl_ca'] = base64.b64encode(ca.get_ca_bundle()) | ||
359 | 543 | |||
360 | 544 | |||
361 | 545 | def _get_ssl_mode(): | ||
362 | 546 | ssl_mode = config('ssl') | ||
363 | 547 | external_ca = False | ||
364 | 548 | # Legacy config boolean option | ||
365 | 549 | ssl_on = config('ssl_enabled') | ||
366 | 550 | if ssl_mode == 'off' and ssl_on is False: | ||
367 | 551 | ssl_mode = 'off' | ||
368 | 552 | elif ssl_mode == 'off' and ssl_on: | ||
369 | 553 | ssl_mode = 'on' | ||
370 | 554 | ssl_key = config('ssl_key') | ||
371 | 555 | ssl_cert = config('ssl_cert') | ||
372 | 556 | if all((ssl_key, ssl_cert)): | ||
373 | 557 | external_ca = True | ||
374 | 558 | return ssl_mode, external_ca | ||
375 | 559 | |||
376 | 560 | |||
377 | 561 | def _convert_from_base64(v): | ||
378 | 562 | # Rabbit originally supported pem encoded key/cert in config, play | ||
379 | 563 | # nice on upgrades as we now expect base64 encoded key/cert/ca. | ||
380 | 564 | if not v: | ||
381 | 565 | return v | ||
382 | 566 | if v.startswith('-----BEGIN'): | ||
383 | 567 | return v | ||
384 | 568 | try: | ||
385 | 569 | return base64.b64decode(v) | ||
386 | 570 | except TypeError: | ||
387 | 571 | return v | ||
388 | 572 | |||
389 | 573 | |||
390 | 574 | def reconfigure_client_ssl(ssl_enabled=False): | ||
391 | 575 | ssl_config_keys = set(('ssl_key', 'ssl_cert', 'ssl_ca')) | ||
392 | 576 | for rid in relation_ids('amqp'): | ||
393 | 577 | rdata = relation_get(rid=rid, unit=os.environ['JUJU_UNIT_NAME']) | ||
394 | 578 | if not ssl_enabled and ssl_config_keys.intersection(rdata): | ||
395 | 579 | # No clean way to remove entirely, but blank them. | ||
396 | 580 | relation_set(relation_id=rid, ssl_key='', ssl_cert='', ssl_ca='') | ||
397 | 581 | elif ssl_enabled and not ssl_config_keys.intersection(rdata): | ||
398 | 582 | configure_client_ssl(rdata) | ||
399 | 583 | relation_set(relation_id=rid, **rdata) | ||
400 | 584 | |||
401 | 585 | |||
402 | 586 | def configure_rabbit_ssl(): | ||
403 | 587 | """ | ||
404 | 588 | The legacy config support adds some additional complications. | ||
405 | 589 | |||
406 | 590 | ssl_enabled = True, ssl = off -> ssl enabled | ||
407 | 591 | ssl_enabled = False, ssl = on -> ssl enabled | ||
408 | 592 | """ | ||
409 | 593 | ssl_mode, external_ca = _get_ssl_mode() | ||
410 | 594 | |||
411 | 595 | if ssl_mode == 'off': | ||
412 | 596 | if os.path.exists(rabbit.RABBITMQ_CONF): | ||
413 | 597 | os.remove(rabbit.RABBITMQ_CONF) | ||
414 | 598 | close_port(config('ssl_port')) | ||
415 | 599 | reconfigure_client_ssl() | ||
416 | 600 | return | ||
417 | 601 | ssl_key = _convert_from_base64(config('ssl_key')) | ||
418 | 602 | ssl_cert = _convert_from_base64(config('ssl_cert')) | ||
419 | 603 | ssl_ca = _convert_from_base64(config('ssl_ca')) | ||
420 | 604 | ssl_port = config('ssl_port') | ||
421 | 605 | |||
422 | 606 | # If external managed certs then we need all the fields. | ||
423 | 607 | if (ssl_mode in ('on', 'only') and any((ssl_key, ssl_cert)) and | ||
424 | 608 | not all((ssl_key, ssl_cert))): | ||
425 | 609 | log('If ssl_key or ssl_cert are specified both are required.', | ||
426 | 610 | level=ERROR) | ||
427 | 611 | sys.exit(1) | ||
428 | 612 | |||
429 | 613 | if not external_ca: | ||
430 | 614 | ssl_cert, ssl_key, ssl_ca = ServiceCA.get_service_cert() | ||
431 | 615 | |||
432 | 616 | rabbit.enable_ssl( | ||
433 | 617 | ssl_key, ssl_cert, ssl_port, ssl_ca, | ||
434 | 618 | ssl_only=(ssl_mode == "only"), ssl_client=False) | ||
435 | 619 | reconfigure_client_ssl(True) | ||
436 | 620 | open_port(ssl_port) | ||
437 | 621 | |||
438 | 622 | |||
439 | 623 | @hooks.hook('config-changed') | 516 | @hooks.hook('config-changed') |
440 | 624 | @restart_on_change(rabbit.restart_map()) | 517 | @restart_on_change(rabbit.restart_map()) |
441 | 625 | def config_changed(): | 518 | def config_changed(): |
442 | 519 | |||
443 | 626 | if config('prefer-ipv6'): | 520 | if config('prefer-ipv6'): |
444 | 627 | rabbit.assert_charm_supports_ipv6() | 521 | rabbit.assert_charm_supports_ipv6() |
445 | 628 | 522 | ||
446 | @@ -652,9 +546,9 @@ | |||
447 | 652 | rabbit.disable_plugin(MAN_PLUGIN) | 546 | rabbit.disable_plugin(MAN_PLUGIN) |
448 | 653 | close_port(55672) | 547 | close_port(55672) |
449 | 654 | 548 | ||
450 | 655 | configure_rabbit_ssl() | ||
451 | 656 | |||
452 | 657 | rabbit.set_all_mirroring_queues(config('mirroring-queues')) | 549 | rabbit.set_all_mirroring_queues(config('mirroring-queues')) |
453 | 550 | rabbit.ConfigRenderer( | ||
454 | 551 | rabbit.CONFIG_FILES).write_all() | ||
455 | 658 | 552 | ||
456 | 659 | if is_relation_made("ha"): | 553 | if is_relation_made("ha"): |
457 | 660 | ha_is_active_active = config("ha-vip-only") | 554 | ha_is_active_active = config("ha-vip-only") |
458 | 661 | 555 | ||
459 | === added file 'hooks/ssl_utils.py' | |||
460 | --- hooks/ssl_utils.py 1970-01-01 00:00:00 +0000 | |||
461 | +++ hooks/ssl_utils.py 2015-04-10 02:40:31 +0000 | |||
462 | @@ -0,0 +1,55 @@ | |||
463 | 1 | from charmhelpers.contrib.ssl.service import ServiceCA | ||
464 | 2 | |||
465 | 3 | from charmhelpers.core.hookenv import ( | ||
466 | 4 | config, | ||
467 | 5 | relation_ids, | ||
468 | 6 | relation_set, | ||
469 | 7 | relation_get, | ||
470 | 8 | ) | ||
471 | 9 | |||
472 | 10 | import base64 | ||
473 | 11 | import os | ||
474 | 12 | |||
475 | 13 | |||
476 | 14 | def get_ssl_mode(): | ||
477 | 15 | ssl_mode = config('ssl') | ||
478 | 16 | external_ca = False | ||
479 | 17 | # Legacy config boolean option | ||
480 | 18 | ssl_on = config('ssl_enabled') | ||
481 | 19 | if ssl_mode == 'off' and ssl_on is False: | ||
482 | 20 | ssl_mode = 'off' | ||
483 | 21 | elif ssl_mode == 'off' and ssl_on: | ||
484 | 22 | ssl_mode = 'on' | ||
485 | 23 | ssl_key = config('ssl_key') | ||
486 | 24 | ssl_cert = config('ssl_cert') | ||
487 | 25 | if all((ssl_key, ssl_cert)): | ||
488 | 26 | external_ca = True | ||
489 | 27 | return ssl_mode, external_ca | ||
490 | 28 | |||
491 | 29 | |||
492 | 30 | def configure_client_ssl(relation_data): | ||
493 | 31 | """Configure client with ssl | ||
494 | 32 | """ | ||
495 | 33 | ssl_mode, external_ca = get_ssl_mode() | ||
496 | 34 | if ssl_mode == 'off': | ||
497 | 35 | return | ||
498 | 36 | relation_data['ssl_port'] = config('ssl_port') | ||
499 | 37 | if external_ca: | ||
500 | 38 | if config('ssl_ca'): | ||
501 | 39 | relation_data['ssl_ca'] = base64.b64encode( | ||
502 | 40 | config('ssl_ca')) | ||
503 | 41 | return | ||
504 | 42 | ca = ServiceCA.get_ca() | ||
505 | 43 | relation_data['ssl_ca'] = base64.b64encode(ca.get_ca_bundle()) | ||
506 | 44 | |||
507 | 45 | |||
508 | 46 | def reconfigure_client_ssl(ssl_enabled=False): | ||
509 | 47 | ssl_config_keys = set(('ssl_key', 'ssl_cert', 'ssl_ca')) | ||
510 | 48 | for rid in relation_ids('amqp'): | ||
511 | 49 | rdata = relation_get(rid=rid, unit=os.environ['JUJU_UNIT_NAME']) | ||
512 | 50 | if not ssl_enabled and ssl_config_keys.intersection(rdata): | ||
513 | 51 | # No clean way to remove entirely, but blank them. | ||
514 | 52 | relation_set(relation_id=rid, ssl_key='', ssl_cert='', ssl_ca='') | ||
515 | 53 | elif ssl_enabled and not ssl_config_keys.intersection(rdata): | ||
516 | 54 | configure_client_ssl(rdata) | ||
517 | 55 | relation_set(relation_id=rid, **rdata) | ||
518 | 0 | 56 | ||
519 | === modified file 'templates/rabbitmq.config' | |||
520 | --- templates/rabbitmq.config 2014-05-23 08:13:05 +0000 | |||
521 | +++ templates/rabbitmq.config 2015-04-10 02:40:31 +0000 | |||
522 | @@ -1,21 +1,35 @@ | |||
523 | 1 | [ | 1 | [ |
526 | 2 | {rabbit, [ | 2 | {rabbit, [ |
527 | 3 | {% if ssl_only %} | 3 | {% if ssl_only %} |
528 | 4 | {tcp_listeners, []}, | 4 | {tcp_listeners, []}, |
529 | 5 | {% else %} | 5 | {% else %} |
530 | 6 | {tcp_listeners, [5672]}, | 6 | {tcp_listeners, [5672]}, |
531 | 7 | {% endif %} | 7 | {% endif %} |
533 | 8 | {ssl_listeners, [{{ ssl_port }}]}, | 8 | {% if ssl_port %} |
534 | 9 | {ssl_listeners, [{{ ssl_port }}]}, | ||
535 | 10 | {% endif %} | ||
536 | 11 | {% if ssl_mode == "on" %} | ||
537 | 9 | {ssl_options, [ | 12 | {ssl_options, [ |
538 | 10 | {verify, verify_peer}, | 13 | {verify, verify_peer}, |
541 | 11 | {% if ssl_client %} | 14 | {% if ssl_client %} |
542 | 12 | {fail_if_no_peer_cert, true}, | 15 | {fail_if_no_peer_cert, true}, |
543 | 13 | {% else %} | 16 | {% else %} |
544 | 14 | {fail_if_no_peer_cert, false}, | 17 | {fail_if_no_peer_cert, false}, |
550 | 15 | {% endif %}{% if ssl_ca_file %} | 18 | {% endif %} |
551 | 16 | {cacertfile, "{{ ssl_ca_file }}"}, {% endif %} | 19 | {% if ssl_ca_file %} |
552 | 17 | {certfile, "{{ ssl_cert_file }}"}, | 20 | {cacertfile, "{{ ssl_ca_file }}"}, |
553 | 18 | {keyfile, "{{ ssl_key_file }}"} | 21 | {% endif %} |
554 | 19 | ]} | 22 | {% if ssl_cert_file %} |
555 | 23 | {certfile, "{{ ssl_cert_file }}"}, | ||
556 | 24 | {% endif %} | ||
557 | 25 | {% if ssl_key_file %} | ||
558 | 26 | {keyfile, "{{ ssl_key_file }}"} | ||
559 | 27 | {% endif %} | ||
560 | 28 | ]}, | ||
561 | 29 | {% endif %} | ||
562 | 30 | |||
563 | 31 | {% if cluster_partition_handling %} | ||
564 | 32 | {cluster_partition_handling, {{ cluster_partition_handling }}} | ||
565 | 33 | {% endif %} | ||
566 | 20 | ]} | 34 | ]} |
567 | 21 | ]. | ||
568 | 22 | \ No newline at end of file | 35 | \ No newline at end of file |
569 | 36 | ]. | ||
570 | 23 | 37 | ||
571 | === modified file 'tests/00_setup.sh' | |||
572 | --- tests/00_setup.sh 2015-03-12 16:42:09 +0000 | |||
573 | +++ tests/00_setup.sh 2015-04-10 02:40:31 +0000 | |||
574 | @@ -15,4 +15,7 @@ | |||
575 | 15 | # Install any additional python packages, or software here. | 15 | # Install any additional python packages, or software here. |
576 | 16 | sudo apt-get install -y python python-pika python3-requests python3-setuptools | 16 | sudo apt-get install -y python python-pika python3-requests python3-setuptools |
577 | 17 | 17 | ||
578 | 18 | # Set http proxy if amulet is using one. | ||
579 | 19 | [[ -n "$AMULET_HTTP_PROXY" ]] && export http_proxy="$AMULET_HTTP_PROXY" && https_proxy="$AMULET_HTTP_PROXY" | ||
580 | 20 | |||
581 | 18 | sudo easy_install3 python3-pika | 21 | sudo easy_install3 python3-pika |
582 | 19 | 22 | ||
583 | === added file 'tests/50_test_cluster_partition.py' | |||
584 | --- tests/50_test_cluster_partition.py 1970-01-01 00:00:00 +0000 | |||
585 | +++ tests/50_test_cluster_partition.py 2015-04-10 02:40:31 +0000 | |||
586 | @@ -0,0 +1,31 @@ | |||
587 | 1 | #!/usr/bin/python | ||
588 | 2 | # | ||
589 | 3 | # This Amulet test deploys rabbitmq-server | ||
590 | 4 | # | ||
591 | 5 | # Note: We use python2, because pika doesn't support python3 | ||
592 | 6 | import amulet | ||
593 | 7 | |||
594 | 8 | # The number of seconds to wait for the environment to setup. | ||
595 | 9 | seconds = 1200 | ||
596 | 10 | d = amulet.Deployment(series="trusty") | ||
597 | 11 | |||
598 | 12 | d.add('rabbitmq-server', units=1) | ||
599 | 13 | # Create a configuration. | ||
600 | 14 | configuration = {'cluster-partition-handling': "autoheal"} | ||
601 | 15 | d.configure('rabbitmq-server', configuration) | ||
602 | 16 | |||
603 | 17 | d.expose('rabbitmq-server') | ||
604 | 18 | try: | ||
605 | 19 | d.setup(timeout=seconds) | ||
606 | 20 | d.sentry.wait(seconds) | ||
607 | 21 | except amulet.helpers.TimeoutError: | ||
608 | 22 | message = 'The environment did not setup in %d seconds.' % seconds | ||
609 | 23 | amulet.raise_status(amulet.SKIP, msg=message) | ||
610 | 24 | except: | ||
611 | 25 | raise | ||
612 | 26 | |||
613 | 27 | rabbit_unit = d.sentry.unit['rabbitmq-server/0'] | ||
614 | 28 | output, code = rabbit_unit.run("grep autoheal /etc/rabbitmq/rabbitmq.conf") | ||
615 | 29 | |||
616 | 30 | if code != 0 or output == "": | ||
617 | 31 | amulet.raise_status(amulet.FAIL, msg=message) | ||
618 | 0 | 32 | ||
619 | === modified file 'unit_tests/test_rabbit_utils.py' | |||
620 | --- unit_tests/test_rabbit_utils.py 2014-10-08 15:57:57 +0000 | |||
621 | +++ unit_tests/test_rabbit_utils.py 2015-04-10 02:40:31 +0000 | |||
622 | @@ -4,8 +4,8 @@ | |||
623 | 4 | import tempfile | 4 | import tempfile |
624 | 5 | import sys | 5 | import sys |
625 | 6 | 6 | ||
626 | 7 | import rabbit_utils | ||
627 | 7 | sys.modules['MySQLdb'] = mock.Mock() | 8 | sys.modules['MySQLdb'] = mock.Mock() |
628 | 8 | import rabbit_utils | ||
629 | 9 | 9 | ||
630 | 10 | 10 | ||
631 | 11 | class UtilsTests(unittest.TestCase): | 11 | class UtilsTests(unittest.TestCase): |
632 | 12 | 12 | ||
633 | === added file 'unit_tests/test_rabbitmq_context.py' | |||
634 | --- unit_tests/test_rabbitmq_context.py 1970-01-01 00:00:00 +0000 | |||
635 | +++ unit_tests/test_rabbitmq_context.py 2015-04-10 02:40:31 +0000 | |||
636 | @@ -0,0 +1,79 @@ | |||
637 | 1 | import rabbitmq_context | ||
638 | 2 | |||
639 | 3 | import mock | ||
640 | 4 | import unittest | ||
641 | 5 | |||
642 | 6 | |||
643 | 7 | class TestRabbitMQSSLContext(unittest.TestCase): | ||
644 | 8 | |||
645 | 9 | @mock.patch("rabbitmq_context.config") | ||
646 | 10 | @mock.patch("rabbitmq_context.close_port") | ||
647 | 11 | @mock.patch("rabbitmq_context.ssl_utils.reconfigure_client_ssl") | ||
648 | 12 | @mock.patch("rabbitmq_context.ssl_utils.get_ssl_mode") | ||
649 | 13 | def test_context_ssl_off(self, get_ssl_mode, reconfig_ssl, close_port, | ||
650 | 14 | config): | ||
651 | 15 | get_ssl_mode.return_value = ("off", "off") | ||
652 | 16 | self.assertEqual(rabbitmq_context.RabbitMQSSLContext().__call__(), { | ||
653 | 17 | "ssl_mode": "off" | ||
654 | 18 | }) | ||
655 | 19 | |||
656 | 20 | close_port.assert_called_once() | ||
657 | 21 | reconfig_ssl.assert_called_once() | ||
658 | 22 | |||
659 | 23 | @mock.patch("rabbitmq_context.open_port") | ||
660 | 24 | @mock.patch("rabbitmq_context.os.chmod") | ||
661 | 25 | @mock.patch("rabbitmq_context.os.chown") | ||
662 | 26 | @mock.patch("rabbitmq_context.pwd.getpwnam") | ||
663 | 27 | @mock.patch("rabbitmq_context.grp.getgrnam") | ||
664 | 28 | @mock.patch("rabbitmq_context.config") | ||
665 | 29 | @mock.patch("rabbitmq_context.close_port") | ||
666 | 30 | @mock.patch("rabbitmq_context.ssl_utils.reconfigure_client_ssl") | ||
667 | 31 | @mock.patch("rabbitmq_context.ssl_utils.get_ssl_mode") | ||
668 | 32 | def test_context_ssl_on(self, get_ssl_mode, reconfig_ssl, close_port, | ||
669 | 33 | config, gr, pw, chown, chmod, open_port): | ||
670 | 34 | |||
671 | 35 | get_ssl_mode.return_value = ("on", "on") | ||
672 | 36 | |||
673 | 37 | def config_get(n): | ||
674 | 38 | return None | ||
675 | 39 | |||
676 | 40 | config.side_effect = config_get | ||
677 | 41 | |||
678 | 42 | def pw(name): | ||
679 | 43 | class Uid(object): | ||
680 | 44 | pw_uid = 1 | ||
681 | 45 | gr_gid = 100 | ||
682 | 46 | return Uid() | ||
683 | 47 | |||
684 | 48 | pw.side_effect = pw | ||
685 | 49 | gr.side_effect = pw | ||
686 | 50 | |||
687 | 51 | m = mock.mock_open() | ||
688 | 52 | with mock.patch('rabbitmq_context.open', m, create=True): | ||
689 | 53 | self.assertEqual( | ||
690 | 54 | rabbitmq_context.RabbitMQSSLContext().__call__(), { | ||
691 | 55 | "ssl_port": None, | ||
692 | 56 | "ssl_cert_file": "/etc/rabbitmq/rabbit-server-cert.pem", | ||
693 | 57 | "ssl_key_file": '/etc/rabbitmq/rabbit-server-privkey.pem', | ||
694 | 58 | "ssl_client": False, | ||
695 | 59 | "ssl_ca_file": "", | ||
696 | 60 | "ssl_only": False, | ||
697 | 61 | "ssl_mode": "on", | ||
698 | 62 | }) | ||
699 | 63 | |||
700 | 64 | reconfig_ssl.assert_called_once() | ||
701 | 65 | open_port.assert_called_once() | ||
702 | 66 | |||
703 | 67 | |||
704 | 68 | class TestRabbitMQClusterContext(unittest.TestCase): | ||
705 | 69 | |||
706 | 70 | @mock.patch("rabbitmq_context.config") | ||
707 | 71 | def test_context_ssl_off(self, config): | ||
708 | 72 | config.return_value = "ignore" | ||
709 | 73 | |||
710 | 74 | self.assertEqual( | ||
711 | 75 | rabbitmq_context.RabbitMQClusterContext().__call__(), { | ||
712 | 76 | 'cluster_partition_handling': "ignore" | ||
713 | 77 | }) | ||
714 | 78 | |||
715 | 79 | config.assert_called_once_with("cluster-partition-handling") | ||
716 | 0 | 80 | ||
717 | === modified file 'unit_tests/test_rabbitmq_server_relations.py' | |||
718 | --- unit_tests/test_rabbitmq_server_relations.py 2015-01-23 08:23:05 +0000 | |||
719 | +++ unit_tests/test_rabbitmq_server_relations.py 2015-04-10 02:40:31 +0000 | |||
720 | @@ -2,7 +2,7 @@ | |||
721 | 2 | from testtools import TestCase | 2 | from testtools import TestCase |
722 | 3 | from mock import patch, MagicMock | 3 | from mock import patch, MagicMock |
723 | 4 | 4 | ||
725 | 5 | os.environ['JUJU_UNIT_NAME'] = 'UNIT_TEST/0' | 5 | os.environ['JUJU_UNIT_NAME'] = 'UNIT_TEST/0' # noqa - needed for import |
726 | 6 | import rabbitmq_server_relations | 6 | import rabbitmq_server_relations |
727 | 7 | 7 | ||
728 | 8 | 8 | ||
729 | @@ -37,7 +37,7 @@ | |||
730 | 37 | @patch('rabbitmq_server_relations.relation_set') | 37 | @patch('rabbitmq_server_relations.relation_set') |
731 | 38 | @patch('apt_pkg.Cache') | 38 | @patch('apt_pkg.Cache') |
732 | 39 | @patch('rabbitmq_server_relations.is_clustered') | 39 | @patch('rabbitmq_server_relations.is_clustered') |
734 | 40 | @patch('rabbitmq_server_relations.configure_client_ssl') | 40 | @patch('rabbitmq_server_relations.ssl_utils.configure_client_ssl') |
735 | 41 | @patch('rabbitmq_server_relations.unit_get') | 41 | @patch('rabbitmq_server_relations.unit_get') |
736 | 42 | @patch('rabbitmq_server_relations.relation_get') | 42 | @patch('rabbitmq_server_relations.relation_get') |
737 | 43 | @patch('rabbitmq_server_relations.is_elected_leader') | 43 | @patch('rabbitmq_server_relations.is_elected_leader') |
738 | @@ -87,7 +87,7 @@ | |||
739 | 87 | @patch('rabbitmq_server_relations.relation_set') | 87 | @patch('rabbitmq_server_relations.relation_set') |
740 | 88 | @patch('apt_pkg.Cache') | 88 | @patch('apt_pkg.Cache') |
741 | 89 | @patch('rabbitmq_server_relations.is_clustered') | 89 | @patch('rabbitmq_server_relations.is_clustered') |
743 | 90 | @patch('rabbitmq_server_relations.configure_client_ssl') | 90 | @patch('rabbitmq_server_relations.ssl_utils.configure_client_ssl') |
744 | 91 | @patch('rabbitmq_server_relations.unit_get') | 91 | @patch('rabbitmq_server_relations.unit_get') |
745 | 92 | @patch('rabbitmq_server_relations.relation_get') | 92 | @patch('rabbitmq_server_relations.relation_get') |
746 | 93 | @patch('rabbitmq_server_relations.is_elected_leader') | 93 | @patch('rabbitmq_server_relations.is_elected_leader') |
Jorge,
I think some of the refactor in here is a definite improvement and makes things cleaner, thanks! I've got a few inline comments included, but the biggest thing is that the rabbitmq-server charm now has a /next and /trunk branch to match the rest of the flow of the openstack charms (since this has recently come under ownership of the ~openstack-charmers team). Can you retarget for /next?
Also, I think the README file should be updated with some discussions regarding high availability.
I'll run through some tests on this but wanted to drop some feedback now.