Merge lp:~niedbalski/charms/trusty/rabbitmq-server/fix-1442443 into lp:~openstack-charmers-archive/charms/trusty/rabbitmq-server/next
- Trusty Tahr (14.04)
- fix-1442443
- Merge into next
Status: | Superseded | ||||
---|---|---|---|---|---|
Proposed branch: | lp:~niedbalski/charms/trusty/rabbitmq-server/fix-1442443 | ||||
Merge into: | lp:~openstack-charmers-archive/charms/trusty/rabbitmq-server/next | ||||
Diff against target: |
707 lines (+392/-152) 9 files modified
config.yaml (+23/-1) hooks/rabbit_utils.py (+37/-39) hooks/rabbitmq_context.py (+122/-0) hooks/rabbitmq_server_relations.py (+7/-99) hooks/ssl_utils.py (+66/-0) templates/rabbitmq.config (+25/-11) tests/50_test_cluster_partition.py (+31/-0) unit_tests/test_rabbitmq_context.py (+79/-0) unit_tests/test_rabbitmq_server_relations.py (+2/-2) |
||||
To merge this branch: | bzr merge lp:~niedbalski/charms/trusty/rabbitmq-server/fix-1442443 | ||||
Related bugs: |
|
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
James Page | Needs Fixing | ||
Billy Olsen | Pending | ||
Review via email: mp+255872@code.launchpad.net |
This proposal supersedes a proposal from 2015-04-10.
This proposal has been superseded by a proposal from 2015-04-16.
Commit message
Description of the change
- Added the cluster_
- Refactored the ssl handling code.
- Added a generic config loader
Billy Olsen (billy-olsen) wrote : Posted in a previous version of this proposal | # |
uosci-testing-bot (uosci-testing-bot) wrote : Posted in a previous version of this proposal | # |
charm_lint_check #3202 rabbitmq-server for niedbalski mp255770
LINT OK: passed
uosci-testing-bot (uosci-testing-bot) wrote : Posted in a previous version of this proposal | # |
charm_unit_test #2990 rabbitmq-server for niedbalski mp255770
UNIT OK: passed
uosci-testing-bot (uosci-testing-bot) wrote : Posted in a previous version of this proposal | # |
charm_amulet_test #3019 rabbitmq-server for niedbalski mp255770
AMULET FAIL: amulet-test failed
AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.
Full amulet test output: http://
Build: http://
uosci-testing-bot (uosci-testing-bot) wrote : Posted in a previous version of this proposal | # |
charm_lint_check #3210 rabbitmq-
LINT OK: passed
uosci-testing-bot (uosci-testing-bot) wrote : Posted in a previous version of this proposal | # |
charm_unit_test #2998 rabbitmq-
UNIT OK: passed
uosci-testing-bot (uosci-testing-bot) wrote : Posted in a previous version of this proposal | # |
charm_amulet_test #3027 rabbitmq-
AMULET FAIL: amulet-test failed
AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.
Full amulet test output: http://
Build: http://
Edward Hope-Morley (hopem) : Posted in a previous version of this proposal | # |
uosci-testing-bot (uosci-testing-bot) wrote : Posted in a previous version of this proposal | # |
charm_unit_test #3005 rabbitmq-
UNIT OK: passed
uosci-testing-bot (uosci-testing-bot) wrote : Posted in a previous version of this proposal | # |
charm_lint_check #3217 rabbitmq-
LINT OK: passed
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_lint_check #3224 rabbitmq-
LINT OK: passed
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_unit_test #3012 rabbitmq-
UNIT OK: passed
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_amulet_test #3041 rabbitmq-
AMULET FAIL: amulet-test failed
AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.
Full amulet test output: http://
Build: http://
James Page (james-page) wrote : | # |
Please can you review the amulet test failures - they appear to be related to SSL.
Jorge Niedbalski (niedbalski) wrote : | # |
@james-page:
Seems that this tests has been failing since Feb 23 (I tested rev 86), because the certificates being used in amulet tests expired:
verify error:num=
notAfter=Feb 23 20:08:18 2015 GMT
This is not directly implied to my proposed change, and seems that https:/
I would appreciate if you can review this proposal.
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_unit_test #3086 rabbitmq-
UNIT OK: passed
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_lint_check #3298 rabbitmq-
LINT OK: passed
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_amulet_test #3115 rabbitmq-
AMULET FAIL: amulet-test failed
AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.
Full amulet test output: http://
Build: http://
Billy Olsen (billy-olsen) wrote : | # |
With these changes, this LGTM. I tested some of the partitioning logic here and things seem fine by me. It appears that an unknown value for cluster_
The fix for LP#1436014 is already in the queue and I don't think it should particularly hold this up, however osci bails as soon as the first test fails and I can't see the full output.
I'm running the amulet tests against your change + the fix for 1436014 and will mark it based on the findings of that.
As always, thanks for the submission Jorge!
- 89. By Liam Young
-
[gnuoy,trivial] Pre-release charmhelper sync
- 90. By James Page
-
[wolsen,
r=james- page] Fixup SSL tests to not fail when certs expire! - 91. By Jorge Niedbalski
-
Rebase commit
- 92. By Jorge Niedbalski
-
Rebase
- 93. By Jorge Niedbalski
-
addressing @dosaboy comments
- 94. By Jorge Niedbalski
-
addressing @dosaboy comments
- 95. By Jorge Niedbalski
-
addressing @dosaboy comments
- 96. By Jorge Niedbalski
-
addressing comments
- 97. By Jorge Niedbalski
-
Addressed @beisner comments
- 98. By Jorge Niedbalski
-
Addressed @beisner comments
Unmerged revisions
Preview Diff
1 | === modified file 'config.yaml' |
2 | --- config.yaml 2015-03-03 06:05:14 +0000 |
3 | +++ config.yaml 2015-04-13 17:16:27 +0000 |
4 | @@ -8,7 +8,7 @@ |
5 | type: string |
6 | default: "off" |
7 | description: | |
8 | - Enable SSL connections on rabbitmq, valid values are 'off', 'on', 'only'. If ssl_key, |
9 | + Enable SSL connections on rabbitmq, valid values are 'off', 'on', 'only'. If ssl_key, |
10 | ssl_cert, ssl_ca are provided then then those values will be used. Otherwise |
11 | the service will act as its own certificate authority and pass its ca cert to clients. |
12 | For HA or clustered rabbits ssl key/cert must be provided. |
13 | @@ -92,6 +92,28 @@ |
14 | description: | |
15 | When set to true the 'ha-mode: all' policy is applied to all the exchages |
16 | that match the expression '^(?!amq\.).*' |
17 | + cluster-partition-handling: |
18 | + type: string |
19 | + default: ignore |
20 | + description: | |
21 | + RabbitMQ offers three ways to deal with network partitions automatically. |
22 | + Available modes: |
23 | + |
24 | + ignore - Your network is reliable. All your nodes are in a rack, |
25 | + connected with a switch, and that switch is also the route to the outside world. |
26 | + You don't want to run any risk of any of your cluster shutting down if any other part of it fails |
27 | + (or you have a two node cluster). |
28 | + |
29 | + pause_minority - Your network is maybe less reliable. You have clustered across 3 AZs in EC2, |
30 | + and you assume that only one AZ will fail at once. |
31 | + In that scenario you want the remaining two AZs to continue working and the nodes from the failed AZ |
32 | + to rejoin automatically and without fuss when the AZ comes back. |
33 | + |
34 | + autoheal - Your network may not be reliable. |
35 | + You are more concerned with continuity of service than with data integrity. |
36 | + You may have a two node cluster. |
37 | + |
38 | + For more information see http://www.rabbitmq.com/partitions.html. |
39 | rbd-size: |
40 | type: string |
41 | default: 5G |
42 | |
43 | === modified file 'hooks/rabbit_utils.py' |
44 | --- hooks/rabbit_utils.py 2015-03-26 13:07:43 +0000 |
45 | +++ hooks/rabbit_utils.py 2015-04-13 17:16:27 +0000 |
46 | @@ -1,6 +1,4 @@ |
47 | import os |
48 | -import pwd |
49 | -import grp |
50 | import re |
51 | import socket |
52 | import sys |
53 | @@ -8,6 +6,13 @@ |
54 | import glob |
55 | import tempfile |
56 | |
57 | +from rabbitmq_context import ( |
58 | + RabbitMQSSLContext, |
59 | + RabbitMQClusterContext |
60 | +) |
61 | + |
62 | +from charmhelpers.core.templating import render |
63 | + |
64 | from charmhelpers.contrib.openstack.utils import ( |
65 | get_hostname, |
66 | ) |
67 | @@ -30,8 +35,6 @@ |
68 | cmp_pkgrevno |
69 | ) |
70 | |
71 | -from charmhelpers.core.templating import render |
72 | - |
73 | from charmhelpers.contrib.peerstorage import ( |
74 | peer_store, |
75 | peer_retrieve |
76 | @@ -57,7 +60,10 @@ |
77 | # the charm doesn't concern itself with template specifics etc. |
78 | CONFIG_FILES = OrderedDict([ |
79 | (RABBITMQ_CONF, { |
80 | - 'hook_contexts': None, |
81 | + 'hook_contexts': [ |
82 | + RabbitMQSSLContext(), |
83 | + RabbitMQClusterContext(), |
84 | + ], |
85 | 'services': ['rabbitmq-server'] |
86 | }), |
87 | (ENV_CONF, { |
88 | @@ -71,6 +77,32 @@ |
89 | ]) |
90 | |
91 | |
92 | +class ConfigRenderer(): |
93 | + |
94 | + def __init__(self, config): |
95 | + self.config_data = {} |
96 | + |
97 | + for config_path, data in config.items(): |
98 | + if 'hook_contexts' in data and data['hook_contexts']: |
99 | + ctxt = {} |
100 | + for svc_context in data['hook_contexts']: |
101 | + ctxt.update(svc_context()) |
102 | + self.config_data[config_path] = ctxt |
103 | + |
104 | + def write(self, config_path): |
105 | + data = self.config_data.get(config_path, None) |
106 | + if data: |
107 | + log("writing config file: %s , data: %s" % (config_path, |
108 | + str(data)), level='DEBUG') |
109 | + |
110 | + render(os.path.basename(config_path), config_path, |
111 | + data, perms=0o644) |
112 | + |
113 | + def write_all(self): |
114 | + for service in self.config_data.keys(): |
115 | + self.write(service) |
116 | + |
117 | + |
118 | class RabbitmqError(Exception): |
119 | pass |
120 | |
121 | @@ -390,40 +422,6 @@ |
122 | def disable_plugin(plugin): |
123 | _manage_plugin(plugin, 'disable') |
124 | |
125 | -ssl_key_file = "/etc/rabbitmq/rabbit-server-privkey.pem" |
126 | -ssl_cert_file = "/etc/rabbitmq/rabbit-server-cert.pem" |
127 | -ssl_ca_file = "/etc/rabbitmq/rabbit-server-ca.pem" |
128 | - |
129 | - |
130 | -def enable_ssl(ssl_key, ssl_cert, ssl_port, |
131 | - ssl_ca=None, ssl_only=False, ssl_client=None): |
132 | - uid = pwd.getpwnam("root").pw_uid |
133 | - gid = grp.getgrnam("rabbitmq").gr_gid |
134 | - |
135 | - for contents, path in ( |
136 | - (ssl_key, ssl_key_file), |
137 | - (ssl_cert, ssl_cert_file), |
138 | - (ssl_ca, ssl_ca_file)): |
139 | - if not contents: |
140 | - continue |
141 | - with open(path, 'w') as fh: |
142 | - fh.write(contents) |
143 | - os.chmod(path, 0o640) |
144 | - os.chown(path, uid, gid) |
145 | - |
146 | - data = { |
147 | - "ssl_port": ssl_port, |
148 | - "ssl_cert_file": ssl_cert_file, |
149 | - "ssl_key_file": ssl_key_file, |
150 | - "ssl_client": ssl_client, |
151 | - "ssl_ca_file": "", |
152 | - "ssl_only": ssl_only} |
153 | - |
154 | - if ssl_ca: |
155 | - data["ssl_ca_file"] = ssl_ca_file |
156 | - |
157 | - render(os.path.basename(RABBITMQ_CONF), RABBITMQ_CONF, data, perms=0o644) |
158 | - |
159 | |
160 | def execute(cmd, die=False, echo=False): |
161 | """ Executes a command |
162 | |
163 | === added file 'hooks/rabbitmq_context.py' |
164 | --- hooks/rabbitmq_context.py 1970-01-01 00:00:00 +0000 |
165 | +++ hooks/rabbitmq_context.py 2015-04-13 17:16:27 +0000 |
166 | @@ -0,0 +1,122 @@ |
167 | +from charmhelpers.contrib.ssl.service import ServiceCA |
168 | + |
169 | +from charmhelpers.core.hookenv import ( |
170 | + open_port, |
171 | + close_port, |
172 | + config, |
173 | + log, |
174 | + ERROR, |
175 | +) |
176 | + |
177 | +import sys |
178 | +import pwd |
179 | +import grp |
180 | +import os |
181 | +import base64 |
182 | + |
183 | +import ssl_utils |
184 | + |
185 | +ssl_key_file = "/etc/rabbitmq/rabbit-server-privkey.pem" |
186 | +ssl_cert_file = "/etc/rabbitmq/rabbit-server-cert.pem" |
187 | +ssl_ca_file = "/etc/rabbitmq/rabbit-server-ca.pem" |
188 | + |
189 | + |
190 | +def convert_from_base64(v): |
191 | + # Rabbit originally supported pem encoded key/cert in config, play |
192 | + # nice on upgrades as we now expect base64 encoded key/cert/ca. |
193 | + if not v: |
194 | + return v |
195 | + if v.startswith('-----BEGIN'): |
196 | + return v |
197 | + try: |
198 | + return base64.b64decode(v) |
199 | + except TypeError: |
200 | + return v |
201 | + |
202 | + |
203 | +class RabbitMQSSLContext(object): |
204 | + |
205 | + def enable_ssl(self, ssl_key, ssl_cert, ssl_port, |
206 | + ssl_ca=None, ssl_only=False, ssl_client=None): |
207 | + |
208 | + uid = pwd.getpwnam("root").pw_uid |
209 | + gid = grp.getgrnam("rabbitmq").gr_gid |
210 | + |
211 | + for contents, path in ( |
212 | + (ssl_key, ssl_key_file), |
213 | + (ssl_cert, ssl_cert_file), |
214 | + (ssl_ca, ssl_ca_file)): |
215 | + |
216 | + if not contents: |
217 | + continue |
218 | + |
219 | + with open(path, 'w') as fh: |
220 | + fh.write(contents) |
221 | + |
222 | + os.chmod(path, 0o640) |
223 | + os.chown(path, uid, gid) |
224 | + |
225 | + data = { |
226 | + "ssl_port": ssl_port, |
227 | + "ssl_cert_file": ssl_cert_file, |
228 | + "ssl_key_file": ssl_key_file, |
229 | + "ssl_client": ssl_client, |
230 | + "ssl_ca_file": "", |
231 | + "ssl_only": ssl_only |
232 | + } |
233 | + |
234 | + if ssl_ca: |
235 | + data["ssl_ca_file"] = ssl_ca_file |
236 | + |
237 | + return data |
238 | + |
239 | + def __call__(self): |
240 | + """ |
241 | + The legacy config support adds some additional complications. |
242 | + |
243 | + ssl_enabled = True, ssl = off -> ssl enabled |
244 | + ssl_enabled = False, ssl = on -> ssl enabled |
245 | + """ |
246 | + ssl_mode, external_ca = ssl_utils.get_ssl_mode() |
247 | + |
248 | + ctxt = { |
249 | + 'ssl_mode': ssl_mode, |
250 | + } |
251 | + |
252 | + if ssl_mode == 'off': |
253 | + close_port(config('ssl_port')) |
254 | + ssl_utils.reconfigure_client_ssl() |
255 | + return ctxt |
256 | + |
257 | + ssl_key = convert_from_base64(config('ssl_key')) |
258 | + ssl_cert = convert_from_base64(config('ssl_cert')) |
259 | + ssl_ca = convert_from_base64(config('ssl_ca')) |
260 | + ssl_port = config('ssl_port') |
261 | + |
262 | + # If external managed certs then we need all the fields. |
263 | + if (ssl_mode in ('on', 'only') and any((ssl_key, ssl_cert)) and |
264 | + not all((ssl_key, ssl_cert))): |
265 | + log('If ssl_key or ssl_cert are specified both are required.', |
266 | + level=ERROR) |
267 | + sys.exit(1) |
268 | + |
269 | + if not external_ca: |
270 | + ssl_cert, ssl_key, ssl_ca = ServiceCA.get_service_cert() |
271 | + |
272 | + ctxt.update(self.enable_ssl( |
273 | + ssl_key, ssl_cert, ssl_port, ssl_ca, |
274 | + ssl_only=(ssl_mode == "only"), ssl_client=False |
275 | + )) |
276 | + |
277 | + ssl_utils.reconfigure_client_ssl(True) |
278 | + open_port(ssl_port) |
279 | + |
280 | + return ctxt |
281 | + |
282 | + |
283 | +class RabbitMQClusterContext(object): |
284 | + |
285 | + def __call__(self): |
286 | + return { |
287 | + 'cluster_partition_handling': config('cluster-partition-handling'), |
288 | + } |
289 | |
290 | === modified file 'hooks/rabbitmq_server_relations.py' |
291 | --- hooks/rabbitmq_server_relations.py 2015-04-01 09:33:46 +0000 |
292 | +++ hooks/rabbitmq_server_relations.py 2015-04-13 17:16:27 +0000 |
293 | @@ -1,5 +1,5 @@ |
294 | #!/usr/bin/python |
295 | -import base64 |
296 | + |
297 | import os |
298 | import shutil |
299 | import sys |
300 | @@ -8,6 +8,8 @@ |
301 | import socket |
302 | |
303 | import rabbit_utils as rabbit |
304 | +import ssl_utils |
305 | + |
306 | from lib.utils import ( |
307 | chown, chmod, |
308 | is_newer, |
309 | @@ -60,7 +62,6 @@ |
310 | service_restart, |
311 | ) |
312 | from charmhelpers.contrib.charmsupport import nrpe |
313 | -from charmhelpers.contrib.ssl.service import ServiceCA |
314 | |
315 | from charmhelpers.contrib.peerstorage import ( |
316 | peer_echo, |
317 | @@ -175,7 +176,7 @@ |
318 | get_address_in_network(config('access-network'), |
319 | unit_get('private-address')) |
320 | |
321 | - configure_client_ssl(relation_settings) |
322 | + ssl_utils.configure_client_ssl(relation_settings) |
323 | |
324 | if is_clustered(): |
325 | relation_settings['clustered'] = 'true' |
326 | @@ -512,103 +513,10 @@ |
327 | MAN_PLUGIN = 'rabbitmq_management' |
328 | |
329 | |
330 | -def configure_client_ssl(relation_data): |
331 | - """Configure client with ssl |
332 | - """ |
333 | - ssl_mode, external_ca = _get_ssl_mode() |
334 | - if ssl_mode == 'off': |
335 | - return |
336 | - relation_data['ssl_port'] = config('ssl_port') |
337 | - if external_ca: |
338 | - if config('ssl_ca'): |
339 | - relation_data['ssl_ca'] = base64.b64encode( |
340 | - config('ssl_ca')) |
341 | - return |
342 | - ca = ServiceCA.get_ca() |
343 | - relation_data['ssl_ca'] = base64.b64encode(ca.get_ca_bundle()) |
344 | - |
345 | - |
346 | -def _get_ssl_mode(): |
347 | - ssl_mode = config('ssl') |
348 | - external_ca = False |
349 | - # Legacy config boolean option |
350 | - ssl_on = config('ssl_enabled') |
351 | - if ssl_mode == 'off' and ssl_on is False: |
352 | - ssl_mode = 'off' |
353 | - elif ssl_mode == 'off' and ssl_on: |
354 | - ssl_mode = 'on' |
355 | - ssl_key = config('ssl_key') |
356 | - ssl_cert = config('ssl_cert') |
357 | - if all((ssl_key, ssl_cert)): |
358 | - external_ca = True |
359 | - return ssl_mode, external_ca |
360 | - |
361 | - |
362 | -def _convert_from_base64(v): |
363 | - # Rabbit originally supported pem encoded key/cert in config, play |
364 | - # nice on upgrades as we now expect base64 encoded key/cert/ca. |
365 | - if not v: |
366 | - return v |
367 | - if v.startswith('-----BEGIN'): |
368 | - return v |
369 | - try: |
370 | - return base64.b64decode(v) |
371 | - except TypeError: |
372 | - return v |
373 | - |
374 | - |
375 | -def reconfigure_client_ssl(ssl_enabled=False): |
376 | - ssl_config_keys = set(('ssl_key', 'ssl_cert', 'ssl_ca')) |
377 | - for rid in relation_ids('amqp'): |
378 | - rdata = relation_get(rid=rid, unit=os.environ['JUJU_UNIT_NAME']) |
379 | - if not ssl_enabled and ssl_config_keys.intersection(rdata): |
380 | - # No clean way to remove entirely, but blank them. |
381 | - relation_set(relation_id=rid, ssl_key='', ssl_cert='', ssl_ca='') |
382 | - elif ssl_enabled and not ssl_config_keys.intersection(rdata): |
383 | - configure_client_ssl(rdata) |
384 | - relation_set(relation_id=rid, **rdata) |
385 | - |
386 | - |
387 | -def configure_rabbit_ssl(): |
388 | - """ |
389 | - The legacy config support adds some additional complications. |
390 | - |
391 | - ssl_enabled = True, ssl = off -> ssl enabled |
392 | - ssl_enabled = False, ssl = on -> ssl enabled |
393 | - """ |
394 | - ssl_mode, external_ca = _get_ssl_mode() |
395 | - |
396 | - if ssl_mode == 'off': |
397 | - if os.path.exists(rabbit.RABBITMQ_CONF): |
398 | - os.remove(rabbit.RABBITMQ_CONF) |
399 | - close_port(config('ssl_port')) |
400 | - reconfigure_client_ssl() |
401 | - return |
402 | - ssl_key = _convert_from_base64(config('ssl_key')) |
403 | - ssl_cert = _convert_from_base64(config('ssl_cert')) |
404 | - ssl_ca = _convert_from_base64(config('ssl_ca')) |
405 | - ssl_port = config('ssl_port') |
406 | - |
407 | - # If external managed certs then we need all the fields. |
408 | - if (ssl_mode in ('on', 'only') and any((ssl_key, ssl_cert)) and |
409 | - not all((ssl_key, ssl_cert))): |
410 | - log('If ssl_key or ssl_cert are specified both are required.', |
411 | - level=ERROR) |
412 | - sys.exit(1) |
413 | - |
414 | - if not external_ca: |
415 | - ssl_cert, ssl_key, ssl_ca = ServiceCA.get_service_cert() |
416 | - |
417 | - rabbit.enable_ssl( |
418 | - ssl_key, ssl_cert, ssl_port, ssl_ca, |
419 | - ssl_only=(ssl_mode == "only"), ssl_client=False) |
420 | - reconfigure_client_ssl(True) |
421 | - open_port(ssl_port) |
422 | - |
423 | - |
424 | @hooks.hook('config-changed') |
425 | @restart_on_change(rabbit.restart_map()) |
426 | def config_changed(): |
427 | + |
428 | if config('prefer-ipv6'): |
429 | rabbit.assert_charm_supports_ipv6() |
430 | |
431 | @@ -638,9 +546,9 @@ |
432 | rabbit.disable_plugin(MAN_PLUGIN) |
433 | close_port(55672) |
434 | |
435 | - configure_rabbit_ssl() |
436 | - |
437 | rabbit.set_all_mirroring_queues(config('mirroring-queues')) |
438 | + rabbit.ConfigRenderer( |
439 | + rabbit.CONFIG_FILES).write_all() |
440 | |
441 | if is_relation_made("ha"): |
442 | ha_is_active_active = config("ha-vip-only") |
443 | |
444 | === added file 'hooks/ssl_utils.py' |
445 | --- hooks/ssl_utils.py 1970-01-01 00:00:00 +0000 |
446 | +++ hooks/ssl_utils.py 2015-04-13 17:16:27 +0000 |
447 | @@ -0,0 +1,66 @@ |
448 | +from charmhelpers.contrib.ssl.service import ServiceCA |
449 | + |
450 | +from charmhelpers.core.hookenv import ( |
451 | + config, |
452 | + relation_ids, |
453 | + relation_set, |
454 | + relation_get, |
455 | + log, |
456 | + WARNING, |
457 | + local_unit, |
458 | +) |
459 | + |
460 | +import base64 |
461 | + |
462 | + |
463 | +def get_ssl_mode(): |
464 | + ssl_mode = config('ssl') |
465 | + external_ca = False |
466 | + # Legacy config boolean option |
467 | + ssl_enabled = config('ssl_enabled') |
468 | + |
469 | + if ssl_enabled: |
470 | + log("Deprecated ssl_enabled config option is True -\ |
471 | + this should be set to False and 'ssl' config set to 'on' instead.", |
472 | + level=WARNING) |
473 | + ssl_mode = 'on' |
474 | + elif ssl_mode == 'on': |
475 | + ssl_mode = 'on' |
476 | + else: |
477 | + ssl_mode = 'off' |
478 | + |
479 | + ssl_key = config('ssl_key') |
480 | + ssl_cert = config('ssl_cert') |
481 | + |
482 | + if all((ssl_key, ssl_cert)): |
483 | + external_ca = True |
484 | + |
485 | + return ssl_mode, external_ca |
486 | + |
487 | + |
488 | +def configure_client_ssl(relation_data): |
489 | + """Configure client with ssl |
490 | + """ |
491 | + ssl_mode, external_ca = get_ssl_mode() |
492 | + if ssl_mode == 'off': |
493 | + return |
494 | + relation_data['ssl_port'] = config('ssl_port') |
495 | + if external_ca: |
496 | + if config('ssl_ca'): |
497 | + relation_data['ssl_ca'] = base64.b64encode( |
498 | + config('ssl_ca')) |
499 | + return |
500 | + ca = ServiceCA.get_ca() |
501 | + relation_data['ssl_ca'] = base64.b64encode(ca.get_ca_bundle()) |
502 | + |
503 | + |
504 | +def reconfigure_client_ssl(ssl_enabled=False): |
505 | + ssl_config_keys = set(('ssl_key', 'ssl_cert', 'ssl_ca')) |
506 | + for rid in relation_ids('amqp'): |
507 | + rdata = relation_get(rid=rid, unit=local_unit()) |
508 | + if not ssl_enabled and ssl_config_keys.intersection(rdata): |
509 | + # No clean way to remove entirely, but blank them. |
510 | + relation_set(relation_id=rid, ssl_key='', ssl_cert='', ssl_ca='') |
511 | + elif ssl_enabled and not ssl_config_keys.intersection(rdata): |
512 | + configure_client_ssl(rdata) |
513 | + relation_set(relation_id=rid, **rdata) |
514 | |
515 | === modified file 'templates/rabbitmq.config' |
516 | --- templates/rabbitmq.config 2014-05-23 08:13:05 +0000 |
517 | +++ templates/rabbitmq.config 2015-04-13 17:16:27 +0000 |
518 | @@ -1,21 +1,35 @@ |
519 | [ |
520 | - {rabbit, [ |
521 | -{% if ssl_only %} |
522 | + {rabbit, [ |
523 | +{% if ssl_only %} |
524 | {tcp_listeners, []}, |
525 | {% else %} |
526 | {tcp_listeners, [5672]}, |
527 | {% endif %} |
528 | - {ssl_listeners, [{{ ssl_port }}]}, |
529 | +{% if ssl_port %} |
530 | + {ssl_listeners, [{{ ssl_port }}]}, |
531 | +{% endif %} |
532 | +{% if ssl_mode == "on" %} |
533 | {ssl_options, [ |
534 | {verify, verify_peer}, |
535 | -{% if ssl_client %} |
536 | - {fail_if_no_peer_cert, true}, |
537 | +{% if ssl_client %} |
538 | + {fail_if_no_peer_cert, true}, |
539 | {% else %} |
540 | {fail_if_no_peer_cert, false}, |
541 | -{% endif %}{% if ssl_ca_file %} |
542 | - {cacertfile, "{{ ssl_ca_file }}"}, {% endif %} |
543 | - {certfile, "{{ ssl_cert_file }}"}, |
544 | - {keyfile, "{{ ssl_key_file }}"} |
545 | - ]} |
546 | +{% endif %} |
547 | + {% if ssl_ca_file %} |
548 | + {cacertfile, "{{ ssl_ca_file }}"}, |
549 | + {% endif %} |
550 | + {% if ssl_cert_file %} |
551 | + {certfile, "{{ ssl_cert_file }}"}, |
552 | + {% endif %} |
553 | + {% if ssl_key_file %} |
554 | + {keyfile, "{{ ssl_key_file }}"} |
555 | + {% endif %} |
556 | + ]}, |
557 | +{% endif %} |
558 | + |
559 | + {% if cluster_partition_handling %} |
560 | + {cluster_partition_handling, {{ cluster_partition_handling }}} |
561 | + {% endif %} |
562 | ]} |
563 | -]. |
564 | \ No newline at end of file |
565 | +]. |
566 | |
567 | === added file 'tests/50_test_cluster_partition.py' |
568 | --- tests/50_test_cluster_partition.py 1970-01-01 00:00:00 +0000 |
569 | +++ tests/50_test_cluster_partition.py 2015-04-13 17:16:27 +0000 |
570 | @@ -0,0 +1,31 @@ |
571 | +#!/usr/bin/python |
572 | +# |
573 | +# This Amulet test deploys rabbitmq-server |
574 | +# |
575 | +# Note: We use python2, because pika doesn't support python3 |
576 | +import amulet |
577 | + |
578 | +# The number of seconds to wait for the environment to setup. |
579 | +seconds = 1200 |
580 | +d = amulet.Deployment(series="trusty") |
581 | + |
582 | +d.add('rabbitmq-server', units=1) |
583 | +# Create a configuration. |
584 | +configuration = {'cluster-partition-handling': "autoheal"} |
585 | +d.configure('rabbitmq-server', configuration) |
586 | + |
587 | +d.expose('rabbitmq-server') |
588 | +try: |
589 | + d.setup(timeout=seconds) |
590 | + d.sentry.wait(seconds) |
591 | +except amulet.helpers.TimeoutError: |
592 | + message = 'The environment did not setup in %d seconds.' % seconds |
593 | + amulet.raise_status(amulet.SKIP, msg=message) |
594 | +except: |
595 | + raise |
596 | + |
597 | +rabbit_unit = d.sentry.unit['rabbitmq-server/0'] |
598 | +output, code = rabbit_unit.run("grep autoheal /etc/rabbitmq/rabbitmq.config") |
599 | + |
600 | +if code != 0 or output == "": |
601 | + amulet.raise_status(amulet.FAIL, msg="didn't find autoheal") |
602 | |
603 | === added file 'unit_tests/test_rabbitmq_context.py' |
604 | --- unit_tests/test_rabbitmq_context.py 1970-01-01 00:00:00 +0000 |
605 | +++ unit_tests/test_rabbitmq_context.py 2015-04-13 17:16:27 +0000 |
606 | @@ -0,0 +1,79 @@ |
607 | +import rabbitmq_context |
608 | + |
609 | +import mock |
610 | +import unittest |
611 | + |
612 | + |
613 | +class TestRabbitMQSSLContext(unittest.TestCase): |
614 | + |
615 | + @mock.patch("rabbitmq_context.config") |
616 | + @mock.patch("rabbitmq_context.close_port") |
617 | + @mock.patch("rabbitmq_context.ssl_utils.reconfigure_client_ssl") |
618 | + @mock.patch("rabbitmq_context.ssl_utils.get_ssl_mode") |
619 | + def test_context_ssl_off(self, get_ssl_mode, reconfig_ssl, close_port, |
620 | + config): |
621 | + get_ssl_mode.return_value = ("off", "off") |
622 | + self.assertEqual(rabbitmq_context.RabbitMQSSLContext().__call__(), { |
623 | + "ssl_mode": "off" |
624 | + }) |
625 | + |
626 | + close_port.assert_called_once() |
627 | + reconfig_ssl.assert_called_once() |
628 | + |
629 | + @mock.patch("rabbitmq_context.open_port") |
630 | + @mock.patch("rabbitmq_context.os.chmod") |
631 | + @mock.patch("rabbitmq_context.os.chown") |
632 | + @mock.patch("rabbitmq_context.pwd.getpwnam") |
633 | + @mock.patch("rabbitmq_context.grp.getgrnam") |
634 | + @mock.patch("rabbitmq_context.config") |
635 | + @mock.patch("rabbitmq_context.close_port") |
636 | + @mock.patch("rabbitmq_context.ssl_utils.reconfigure_client_ssl") |
637 | + @mock.patch("rabbitmq_context.ssl_utils.get_ssl_mode") |
638 | + def test_context_ssl_on(self, get_ssl_mode, reconfig_ssl, close_port, |
639 | + config, gr, pw, chown, chmod, open_port): |
640 | + |
641 | + get_ssl_mode.return_value = ("on", "on") |
642 | + |
643 | + def config_get(n): |
644 | + return None |
645 | + |
646 | + config.side_effect = config_get |
647 | + |
648 | + def pw(name): |
649 | + class Uid(object): |
650 | + pw_uid = 1 |
651 | + gr_gid = 100 |
652 | + return Uid() |
653 | + |
654 | + pw.side_effect = pw |
655 | + gr.side_effect = pw |
656 | + |
657 | + m = mock.mock_open() |
658 | + with mock.patch('rabbitmq_context.open', m, create=True): |
659 | + self.assertEqual( |
660 | + rabbitmq_context.RabbitMQSSLContext().__call__(), { |
661 | + "ssl_port": None, |
662 | + "ssl_cert_file": "/etc/rabbitmq/rabbit-server-cert.pem", |
663 | + "ssl_key_file": '/etc/rabbitmq/rabbit-server-privkey.pem', |
664 | + "ssl_client": False, |
665 | + "ssl_ca_file": "", |
666 | + "ssl_only": False, |
667 | + "ssl_mode": "on", |
668 | + }) |
669 | + |
670 | + reconfig_ssl.assert_called_once() |
671 | + open_port.assert_called_once() |
672 | + |
673 | + |
674 | +class TestRabbitMQClusterContext(unittest.TestCase): |
675 | + |
676 | + @mock.patch("rabbitmq_context.config") |
677 | + def test_context_ssl_off(self, config): |
678 | + config.return_value = "ignore" |
679 | + |
680 | + self.assertEqual( |
681 | + rabbitmq_context.RabbitMQClusterContext().__call__(), { |
682 | + 'cluster_partition_handling': "ignore" |
683 | + }) |
684 | + |
685 | + config.assert_called_once_with("cluster-partition-handling") |
686 | |
687 | === modified file 'unit_tests/test_rabbitmq_server_relations.py' |
688 | --- unit_tests/test_rabbitmq_server_relations.py 2015-03-03 22:59:25 +0000 |
689 | +++ unit_tests/test_rabbitmq_server_relations.py 2015-04-13 17:16:27 +0000 |
690 | @@ -37,7 +37,7 @@ |
691 | @patch('rabbitmq_server_relations.relation_set') |
692 | @patch('apt_pkg.Cache') |
693 | @patch('rabbitmq_server_relations.is_clustered') |
694 | - @patch('rabbitmq_server_relations.configure_client_ssl') |
695 | + @patch('rabbitmq_server_relations.ssl_utils.configure_client_ssl') |
696 | @patch('rabbitmq_server_relations.unit_get') |
697 | @patch('rabbitmq_server_relations.relation_get') |
698 | @patch('rabbitmq_server_relations.is_elected_leader') |
699 | @@ -87,7 +87,7 @@ |
700 | @patch('rabbitmq_server_relations.relation_set') |
701 | @patch('apt_pkg.Cache') |
702 | @patch('rabbitmq_server_relations.is_clustered') |
703 | - @patch('rabbitmq_server_relations.configure_client_ssl') |
704 | + @patch('rabbitmq_server_relations.ssl_utils.configure_client_ssl') |
705 | @patch('rabbitmq_server_relations.unit_get') |
706 | @patch('rabbitmq_server_relations.relation_get') |
707 | @patch('rabbitmq_server_relations.is_elected_leader') |
Jorge,
I think some of the refactor in here is a definite improvement and makes things cleaner, thanks! I've got a few inline comments included, but the biggest thing is that the rabbitmq-server charm now has a /next and /trunk branch to match the rest of the flow of the openstack charms (since this has recently come under ownership of the ~openstack-charmers team). Can you retarget for /next?
Also, I think the README file should be updated with some discussions regarding high availability.
I'll run through some tests on this but wanted to drop some feedback now.