Merge lp:~johnsca/charms/trusty/cf-go-router/port-conflicts into lp:~cf-charmers/charms/trusty/cf-go-router/trunk
- Trusty Tahr (14.04)
- port-conflicts
- Merge into trunk
Status: | Merged |
---|---|
Merged at revision: | 31 |
Proposed branch: | lp:~johnsca/charms/trusty/cf-go-router/port-conflicts |
Merge into: | lp:~cf-charmers/charms/trusty/cf-go-router/trunk |
Diff against target: |
365 lines (+140/-76) 8 files modified
config.yaml (+4/-1) hooks/charmhelpers/contrib/cloudfoundry/contexts.py (+17/-6) hooks/charmhelpers/core/services.py (+93/-56) hooks/config.py (+3/-2) hooks/logrouter-relation-changed (+6/-0) hooks/tests/test_hooks.py (+4/-4) metadata.yaml (+2/-0) templates/gorouter.yml (+11/-7) |
To merge this branch: | bzr merge lp:~johnsca/charms/trusty/cf-go-router/port-conflicts |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Cloud Foundry Charmers | Pending | ||
Review via email: mp+222671@code.launchpad.net |
Commit message
Description of the change
Resolve port conflicts in cf-go-router
Cory Johns (johnsca) wrote : | # |
- 38. By Cory Johns
-
Removed varz_port option as it doesn't add much value to change
Cory Johns (johnsca) wrote : | # |
Please take a look.
Benjamin Saller (bcsaller) wrote : | # |
Thanks, LGTM with changes
At min we should use the config['port'] the action for the rest might be
a card unless you disagree with the approach.
https:/
File hooks/config.py (right):
https:/
hooks/config.py:39: 'ports': [80],
if this is a config option (and it can be because we expose this one)
then this can't be hard-coded here.
config['port']
The harder issue is that we then need to track the previous value if we
are to close the old port.
We (and it doesn't have to be now) could use a config context that
called callbacks here as callback(new_value, old_value=None) so we have
a chance to do something with the old saved context. seem reasonable?
- 39. By Cory Johns
-
Removed hard-coded port and updated charm-helpers for port changing awareness logic
Cory Johns (johnsca) wrote : | # |
Please take a look.
Cory Johns (johnsca) wrote : | # |
On 2014/06/11 22:23:56, benjamin.saller wrote:
> hooks/config.py:39: 'ports': [80],
> if this is a config option (and it can be because we expose this one)
then this
> can't be hard-coded here.
> The harder issue is that we then need to track the previous value if
we are to
> close the old port.
Thanks, good catches. I added the logic to close the old port when
switching to charm-helpers because it should always happen, and I feel
like it should be transparent to the charm author.
Benjamin Saller (bcsaller) wrote : | # |
LGTM
Nice fix for the port handling
https:/
File hooks/charmhelp
https:/
hooks/charmhelp
Cool, thanks
https:/
File hooks/config.py (right):
https:/
hooks/config.py:39: 'ports': [hookenv.
funny that we use the hookenv.
didn't recall that it even did per-key lookup :)
Cory Johns (johnsca) wrote : | # |
*** Submitted:
Resolve port conflicts in cf-go-router
R=benjamin.saller
CC=
https:/
Preview Diff
1 | === modified file 'config.yaml' |
2 | --- config.yaml 2014-05-27 21:50:52 +0000 |
3 | +++ config.yaml 2014-06-12 14:01:13 +0000 |
4 | @@ -25,4 +25,7 @@ |
5 | description: | |
6 | Key ID to import to the apt keyring to support use with arbitary source |
7 | configuration from outside of Launchpad archives or PPA's. |
8 | - |
9 | + port: |
10 | + type: int |
11 | + default: 80 |
12 | + description: Port for the router to listen on. |
13 | |
14 | === modified file 'hooks/charmhelpers/contrib/cloudfoundry/contexts.py' |
15 | --- hooks/charmhelpers/contrib/cloudfoundry/contexts.py 2014-05-27 21:50:52 +0000 |
16 | +++ hooks/charmhelpers/contrib/cloudfoundry/contexts.py 2014-06-12 14:01:13 +0000 |
17 | @@ -33,7 +33,7 @@ |
18 | |
19 | class NatsRelation(RelationContext): |
20 | interface = 'nats' |
21 | - required_keys = ['nats_port', 'nats_address', 'nats_user', 'nats_password'] |
22 | + required_keys = ['address', 'port', 'user', 'password'] |
23 | |
24 | |
25 | class MysqlRelation(RelationContext): |
26 | @@ -44,9 +44,10 @@ |
27 | def get_data(self): |
28 | RelationContext.get_data(self) |
29 | if self.is_ready(): |
30 | - if 'port' not in self['db']: |
31 | - self['db']['port'] = '3306' |
32 | - self['db']['dsn'] = self.dsn_template.format(**self['db']) |
33 | + for unit in self['db']: |
34 | + if 'port' not in unit: |
35 | + unit['port'] = '3306' |
36 | + unit['dsn'] = self.dsn_template.format(**unit) |
37 | |
38 | |
39 | class RouterRelation(RelationContext): |
40 | @@ -56,9 +57,19 @@ |
41 | |
42 | class LogRouterRelation(RelationContext): |
43 | interface = 'logrouter' |
44 | - required_keys = ['shared-secret', 'logrouter-address'] |
45 | + required_keys = ['shared_secret', 'address', 'incoming_port', 'outgoing_port'] |
46 | |
47 | |
48 | class LoggregatorRelation(RelationContext): |
49 | interface = 'loggregator' |
50 | - required_keys = ['shared_secret', 'loggregator_address'] |
51 | + required_keys = ['address', 'incoming_port', 'outgoing_port'] |
52 | + |
53 | + |
54 | +class EtcdRelation(RelationContext): |
55 | + interface = 'etcd' |
56 | + required_keys = ['hostname', 'port'] |
57 | + |
58 | + |
59 | +class CloudControllerRelation(RelationContext): |
60 | + interface = 'cc' |
61 | + required_keys = ['hostname', 'port', 'user', 'password'] |
62 | |
63 | === modified file 'hooks/charmhelpers/core/services.py' |
64 | --- hooks/charmhelpers/core/services.py 2014-05-29 17:27:32 +0000 |
65 | +++ hooks/charmhelpers/core/services.py 2014-06-12 14:01:13 +0000 |
66 | @@ -11,6 +11,14 @@ |
67 | """ |
68 | Register a list of services, given their definitions. |
69 | |
70 | + Traditional charm authoring is focused on implementing hooks. That is, |
71 | + the charm author is thinking in terms of "What hook am I handling; what |
72 | + does this hook need to do?" However, in most cases, the real question |
73 | + should be "Do I have the information I need to configure and start this |
74 | + piece of software and, if so, what are the steps for doing so." The |
75 | + ServiceManager framework tries to bring the focus to the data and the |
76 | + setup tasks, in the most declarative way possible. |
77 | + |
78 | Service definitions are dicts in the following formats (all keys except |
79 | 'service' are optional): |
80 | |
81 | @@ -67,28 +75,28 @@ |
82 | a mongodb relation and which runs a custom `db_migrate` function prior to |
83 | restarting the service, and a Runit serivce called spadesd. |
84 | |
85 | - >>> manager = services.ServiceManager([ |
86 | - ... { |
87 | - ... 'service': 'bingod', |
88 | - ... 'ports': [80, 443], |
89 | - ... 'required_data': [MongoRelation(), config()], |
90 | - ... 'data_ready': [ |
91 | - ... services.template(source='bingod.conf'), |
92 | - ... services.template(source='bingod.ini', |
93 | - ... target='/etc/bingod.ini', |
94 | - ... owner='bingo', perms=0400), |
95 | - ... ], |
96 | - ... }, |
97 | - ... { |
98 | - ... 'service': 'spadesd', |
99 | - ... 'data_ready': services.template(source='spadesd_run.j2', |
100 | - ... target='/etc/sv/spadesd/run', |
101 | - ... perms=0555), |
102 | - ... 'start': runit_start, |
103 | - ... 'stop': runit_stop, |
104 | - ... }, |
105 | - ... ]) |
106 | - ... manager.manage() |
107 | + manager = services.ServiceManager([ |
108 | + { |
109 | + 'service': 'bingod', |
110 | + 'ports': [80, 443], |
111 | + 'required_data': [MongoRelation(), config(), {'my': 'data'}], |
112 | + 'data_ready': [ |
113 | + services.template(source='bingod.conf'), |
114 | + services.template(source='bingod.ini', |
115 | + target='/etc/bingod.ini', |
116 | + owner='bingo', perms=0400), |
117 | + ], |
118 | + }, |
119 | + { |
120 | + 'service': 'spadesd', |
121 | + 'data_ready': services.template(source='spadesd_run.j2', |
122 | + target='/etc/sv/spadesd/run', |
123 | + perms=0555), |
124 | + 'start': runit_start, |
125 | + 'stop': runit_stop, |
126 | + }, |
127 | + ]) |
128 | + manager.manage() |
129 | """ |
130 | self.services = {} |
131 | for service in services or []: |
132 | @@ -213,55 +221,73 @@ |
133 | interface = None |
134 | required_keys = [] |
135 | |
136 | + def __init__(self, *args, **kwargs): |
137 | + super(RelationContext, self).__init__(*args, **kwargs) |
138 | + self.get_data() |
139 | + |
140 | def __bool__(self): |
141 | """ |
142 | - Updates the data and returns True if all of the required_keys are available. |
143 | + Returns True if all of the required_keys are available. |
144 | """ |
145 | - self.get_data() |
146 | return self.is_ready() |
147 | |
148 | __nonzero__ = __bool__ |
149 | |
150 | + def __repr__(self): |
151 | + return super(RelationContext, self).__repr__() |
152 | + |
153 | def is_ready(self): |
154 | """ |
155 | - Returns True if all of the required_keys are available. |
156 | - """ |
157 | - return set(self.get(self.interface, {}).keys()).issuperset(set(self.required_keys)) |
158 | + Returns True if all of the `required_keys` are available from any units. |
159 | + """ |
160 | + ready = len(self.get(self.interface, [])) > 0 |
161 | + if not ready: |
162 | + hookenv.log('Incomplete relation: {}'.format(self.__class__.__name__), hookenv.DEBUG) |
163 | + return ready |
164 | + |
165 | + def _is_ready(self, unit_data): |
166 | + """ |
167 | + Helper method that tests a set of relation data and returns True if |
168 | + all of the `required_keys` are present. |
169 | + """ |
170 | + return set(unit_data.keys()).issuperset(set(self.required_keys)) |
171 | |
172 | def get_data(self): |
173 | """ |
174 | - Retrieve the relation data and store it under `self[self.interface]`. |
175 | - |
176 | - If there are more than one units related on the desired interface, |
177 | - then each unit will have its data stored under `self[self.interface][unit_id]` |
178 | - and one of the units with complete information will chosen at random |
179 | - to fill the values at `self[self.interface]`. |
180 | - |
181 | - |
182 | - For example: |
183 | - |
184 | - { |
185 | - 'foo': 'bar', |
186 | - 'unit/0': { |
187 | - 'foo': 'bar', |
188 | - }, |
189 | - 'unit/1': { |
190 | - 'foo': 'baz', |
191 | - }, |
192 | - } |
193 | + Retrieve the relation data for each unit involved in a realtion and, |
194 | + if complete, store it in a list under `self[self.interface]`. This |
195 | + is automatically called when the RelationContext is instantiated. |
196 | + |
197 | + The units are sorted lexographically first by the service ID, then by |
198 | + the unit ID. Thus, if an interface has two other services, 'db:1' |
199 | + and 'db:2', with 'db:1' having two units, 'wordpress/0' and 'wordpress/1', |
200 | + and 'db:2' having one unit, 'mediawiki/0', all of which have a complete |
201 | + set of data, the relation data for the units will be stored in the |
202 | + order: 'wordpress/0', 'wordpress/1', 'mediawiki/0'. |
203 | + |
204 | + If you only care about a single unit on the relation, you can just |
205 | + access it as `{{ interface[0]['key'] }}`. However, if you can at all |
206 | + support multiple units on a relation, you should iterate over the list, |
207 | + like: |
208 | + |
209 | + {% for unit in interface -%} |
210 | + {{ unit['key'] }}{% if not loop.last %},{% endif %} |
211 | + {%- endfor %} |
212 | + |
213 | + Note that since all sets of relation data from all related services and |
214 | + units are in a single list, if you need to know which service or unit a |
215 | + set of data came from, you'll need to extend this class to preserve |
216 | + that information. |
217 | """ |
218 | if not hookenv.relation_ids(self.interface): |
219 | return |
220 | |
221 | - ns = self.setdefault(self.interface, {}) |
222 | - required = set(self.required_keys) |
223 | - for rid in hookenv.relation_ids(self.interface): |
224 | - for unit in hookenv.related_units(rid): |
225 | + ns = self.setdefault(self.interface, []) |
226 | + for rid in sorted(hookenv.relation_ids(self.interface)): |
227 | + for unit in sorted(hookenv.related_units(rid)): |
228 | reldata = hookenv.relation_get(rid=rid, unit=unit) |
229 | - unit_ns = ns.setdefault(unit, {}) |
230 | - unit_ns.update(reldata) |
231 | - if set(reldata.keys()).issuperset(required): |
232 | - ns.update(reldata) |
233 | + if self._is_ready(reldata): |
234 | + ns.append(reldata) |
235 | |
236 | |
237 | class ManagerCallback(object): |
238 | @@ -308,7 +334,18 @@ |
239 | """ |
240 | def __call__(self, manager, service_name, event_name): |
241 | service = manager.get_service(service_name) |
242 | - for port in service.get('ports', []): |
243 | + new_ports = service.get('ports', []) |
244 | + port_file = os.path.join(hookenv.charm_dir(), '.{}.ports'.format(service_name)) |
245 | + if os.path.exists(port_file): |
246 | + with open(port_file) as fp: |
247 | + old_ports = fp.read().split(',') |
248 | + for old_port in old_ports: |
249 | + old_port = int(old_port) |
250 | + if old_port not in new_ports: |
251 | + hookenv.close_port(old_port) |
252 | + with open(port_file, 'w') as fp: |
253 | + fp.write(','.join(str(port) for port in new_ports)) |
254 | + for port in new_ports: |
255 | if event_name == 'start': |
256 | hookenv.open_port(port) |
257 | elif event_name == 'stop': |
258 | @@ -316,6 +353,6 @@ |
259 | |
260 | |
261 | # Convenience aliases |
262 | -template = TemplateCallback |
263 | +render_template = template = TemplateCallback |
264 | open_ports = PortManagerCallback() |
265 | close_ports = PortManagerCallback() |
266 | |
267 | === modified file 'hooks/config.py' |
268 | --- hooks/config.py 2014-06-03 03:50:30 +0000 |
269 | +++ hooks/config.py 2014-06-12 14:01:13 +0000 |
270 | @@ -36,10 +36,11 @@ |
271 | SERVICES = [ |
272 | { |
273 | 'service': 'gorouter', |
274 | - 'ports': [80, 443], |
275 | + 'ports': [hookenv.config('port')], |
276 | 'required_data': [ |
277 | - {'domain': domain()}, |
278 | + {'domain': domain(), 'config': hookenv.config(), 'varz_port': 8081}, |
279 | contexts.NatsRelation(), |
280 | + contexts.LogRouterRelation(), |
281 | ], |
282 | 'data_ready': [ |
283 | services.template(source='gorouter.conf', |
284 | |
285 | === added file 'hooks/logrouter-relation-changed' |
286 | --- hooks/logrouter-relation-changed 1970-01-01 00:00:00 +0000 |
287 | +++ hooks/logrouter-relation-changed 2014-06-12 14:01:13 +0000 |
288 | @@ -0,0 +1,6 @@ |
289 | +#!/usr/bin/env python |
290 | +from charmhelpers.core import services |
291 | +import config |
292 | + |
293 | +manager = services.ServiceManager(config.SERVICES) |
294 | +manager.manage() |
295 | |
296 | === modified file 'hooks/tests/test_hooks.py' |
297 | --- hooks/tests/test_hooks.py 2014-06-03 03:50:30 +0000 |
298 | +++ hooks/tests/test_hooks.py 2014-06-12 14:01:13 +0000 |
299 | @@ -14,10 +14,10 @@ |
300 | @mock.patch('hooks.hooks.host.write_file') |
301 | def test_emit_router_config(self, write_file): |
302 | emit_router_config({ |
303 | - 'nats_address': 'addy', |
304 | - 'nats_port': 123, |
305 | - 'nats_password': 'password', |
306 | - 'nats_user': 'user'}) |
307 | + 'address': 'addy', |
308 | + 'port': 123, |
309 | + 'password': 'password', |
310 | + 'user': 'user'}) |
311 | |
312 | args, kw = write_file.call_args_list[0] |
313 | self.assertEqual( |
314 | |
315 | === modified file 'metadata.yaml' |
316 | --- metadata.yaml 2014-05-12 07:19:21 +0000 |
317 | +++ metadata.yaml 2014-06-12 14:01:13 +0000 |
318 | @@ -13,3 +13,5 @@ |
319 | requires: |
320 | nats: |
321 | interface: nats |
322 | + logrouter: |
323 | + interface: logrouter |
324 | |
325 | === modified file 'templates/gorouter.yml' |
326 | --- templates/gorouter.yml 2014-04-17 15:12:52 +0000 |
327 | +++ templates/gorouter.yml 2014-06-12 14:01:13 +0000 |
328 | @@ -1,11 +1,11 @@ |
329 | nats: |
330 | - - host: "{{ nats['nats_address'] }}" |
331 | - port: {{ nats['nats_port'] }} |
332 | - user: "{{ nats['nats_user'] }}" |
333 | - pass: "{{ nats['nats_password'] }}" |
334 | + - host: "{{ nats[0]['address'] }}" |
335 | + port: {{ nats[0]['port'] }} |
336 | + user: "{{ nats[0]['user'] }}" |
337 | + pass: "{{ nats[0]['password'] }}" |
338 | |
339 | status: |
340 | - port: 8081 |
341 | + port: {{ varz_port }} |
342 | user: user |
343 | pass: password |
344 | |
345 | @@ -13,7 +13,11 @@ |
346 | file: /var/vcap/sys/log/gorouter/gorouter.log |
347 | level: info |
348 | |
349 | -port: 80 |
350 | +loggregatorConfig: |
351 | + url: {{ logrouter[0]['address'] }}:{{ logrouter[0]['incoming_port'] }} |
352 | + shared_secret: {{ logrouter[0]['shared_secret'] }} |
353 | + |
354 | +port: {{ config['port'] }} |
355 | index: 1 |
356 | pidfile: /var/vcap/sys/run/gorouter/gorouter.pid |
357 | go_max_procs: 8 |
358 | @@ -23,6 +27,6 @@ |
359 | publish_start_message_interval: 30 |
360 | prune_stale_droplets_interval: 30 |
361 | droplet_stale_threshold: 120 |
362 | -publish_active_apps_interval: 0 |
363 | +publish_active_apps_interval: 0 |
364 | |
365 | endpoint_timeout: 60 |
Reviewers: mp+222671_ code.launchpad. net,
Message:
Please take a look.
Description:
Resolve port conflicts in cf-go-router
https:/ /code.launchpad .net/~johnsca/ charms/ trusty/ cf-go-router/ port-conflicts/ +merge/ 222671
(do not edit description out of merge proposal)
Please review this at https:/ /codereview. appspot. com/103270043/
Affected files (+134, -75 lines): ers/contrib/ cloudfoundry/ contexts. py ers/core/ services. py -relation- changed test_hooks. py gorouter. yml
A [revision details]
M config.yaml
M hooks/charmhelp
M hooks/charmhelp
M hooks/config.py
A hooks/logrouter
M hooks/tests/
M metadata.yaml
M templates/