Merge lp:~tvansteenburgh/charm-helpers/config-implicit-save into lp:charm-helpers
- config-implicit-save
- Merge into devel
Status: | Merged | ||||
---|---|---|---|---|---|
Merged at revision: | 200 | ||||
Proposed branch: | lp:~tvansteenburgh/charm-helpers/config-implicit-save | ||||
Merge into: | lp:charm-helpers | ||||
Diff against target: |
278 lines (+87/-27) 6 files modified
charmhelpers/core/hookenv.py (+28/-15) charmhelpers/core/services/base.py (+3/-0) docs/examples/config.rst (+0/-8) tests/contrib/ansible/test_ansible.py (+5/-2) tests/core/test_hookenv.py (+31/-0) tests/core/test_services.py (+20/-2) |
||||
To merge this branch: | bzr merge lp:~tvansteenburgh/charm-helpers/config-implicit-save | ||||
Related bugs: |
|
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Stuart Bishop (community) | Approve | ||
Review via email: mp+224678@code.launchpad.net |
Commit message
Description of the change
Make Config implicitly save itself instead of requiring charm author to config.save() in every hook.
Stuart Bishop (stub) wrote : | # |
Tim Van Steenburgh (tvansteenburgh) wrote : | # |
> I don't think using the destructor is the right thing to do. It will cause the
> changes to be persisted even if the hook fails.
>
> The sanest place I can think of to save the config only on successful
> termination is in the @hooks.hook decorator.
Good observation. Doing the save in the @hooks.hook will be difficult since there is not a global Config object.
Can you think of a downside of doing this instead:
```
def __del__(self):
if self.implicit_save and not sys.exc_type:
self.save()
```
Stuart Bishop (stub) wrote : | # |
There actually *is* a global config object, because the config() method is using the @cached decorator. This could even be made explicit.
I didn't know you could check for sys.exc_type to catch this case (I imagine you would need to ignore SystemExit?). In any case, I don't think it is a good idea here; I have hooks that catch exceptions, log nice error messages, and sys.exit(1).
Stuart Bishop (stub) wrote : | # |
... although if you *can* sniff for sys.exc_* reliably, and see a SystemExit, it may be good enough to save only if a) no exception b) its a SystemExit with a 0 return code
Tim Van Steenburgh (tvansteenburgh) wrote : | # |
One other idea: do the save in Config.
I like this for two reasons:
1. No sys.exc_type voodoo
2. Keeps code local to the Config object, no coupling to or dependency on @hooks.hook
The downsides are:
1. May save() multiple times. Performance hit for writing file multiple times.
2. save()s are immediate, no rollback if exception occurs.
I think #1 is minor, and #2 is actually consistent with the way many charms have handled persisting config thus far - by writing dot files.
I think the pros outweigh the cons but would appreciate other opinions.
Tim Van Steenburgh (tvansteenburgh) wrote : | # |
In retrospect I guess this is no different than the original idea of doing the save in the destructor. :/
- 175. By Tim Van Steenburgh
-
Merge from trunk
- 176. By Tim Van Steenburgh
-
Change how Config is implicitly saved
Call Config.save() automatically after hook execution if the hook
code is running in the context of a @hook decorator or the
Services Framework. - 177. By Tim Van Steenburgh
-
Merge from lp:charm-helpers
- 178. By Tim Van Steenburgh
-
Remove no-op tests
Tim Van Steenburgh (tvansteenburgh) wrote : | # |
After discussion with stub and cory_fu, the following changes were made:
* Don't implicitly save in __del__ since that will fire even after exceptions occur.
Instead:
* Save in Hooks.execute(), if the hook code is running in the context of a @hook decorated function, or
* Save in ServiceManager.
In other scenarios, client code would need to call Config.save() explicitly. Implicit saves can still be disabled by setting `Config.
The important changes are on r176. The other commits are just merges from trunk and cleanup.
Stuart Bishop (stub) wrote : | # |
All looks good to me. Thanks for this - it will let me drop a chunk of code in the PostgreSQL charm I've been wanting to drop.
Preview Diff
1 | === modified file 'charmhelpers/core/hookenv.py' | |||
2 | --- charmhelpers/core/hookenv.py 2014-08-19 14:04:37 +0000 | |||
3 | +++ charmhelpers/core/hookenv.py 2014-08-26 15:16:14 +0000 | |||
4 | @@ -156,12 +156,15 @@ | |||
5 | 156 | 156 | ||
6 | 157 | 157 | ||
7 | 158 | class Config(dict): | 158 | class Config(dict): |
14 | 159 | """A Juju charm config dictionary that can write itself to | 159 | """A dictionary representation of the charm's config.yaml, with some |
15 | 160 | disk (as json) and track which values have changed since | 160 | extra features: |
16 | 161 | the previous hook invocation. | 161 | |
17 | 162 | 162 | - See which values in the dictionary have changed since the previous hook. | |
18 | 163 | Do not instantiate this object directly - instead call | 163 | - For values that have changed, see what the previous value was. |
19 | 164 | ``hookenv.config()`` | 164 | - Store arbitrary data for use in a later hook. |
20 | 165 | |||
21 | 166 | NOTE: Do not instantiate this object directly - instead call | ||
22 | 167 | ``hookenv.config()``, which will return an instance of :class:`Config`. | ||
23 | 165 | 168 | ||
24 | 166 | Example usage:: | 169 | Example usage:: |
25 | 167 | 170 | ||
26 | @@ -170,8 +173,8 @@ | |||
27 | 170 | >>> config = hookenv.config() | 173 | >>> config = hookenv.config() |
28 | 171 | >>> config['foo'] | 174 | >>> config['foo'] |
29 | 172 | 'bar' | 175 | 'bar' |
30 | 176 | >>> # store a new key/value for later use | ||
31 | 173 | >>> config['mykey'] = 'myval' | 177 | >>> config['mykey'] = 'myval' |
32 | 174 | >>> config.save() | ||
33 | 175 | 178 | ||
34 | 176 | 179 | ||
35 | 177 | >>> # user runs `juju set mycharm foo=baz` | 180 | >>> # user runs `juju set mycharm foo=baz` |
36 | @@ -188,22 +191,23 @@ | |||
37 | 188 | >>> # keys/values that we add are preserved across hooks | 191 | >>> # keys/values that we add are preserved across hooks |
38 | 189 | >>> config['mykey'] | 192 | >>> config['mykey'] |
39 | 190 | 'myval' | 193 | 'myval' |
40 | 191 | >>> # don't forget to save at the end of hook! | ||
41 | 192 | >>> config.save() | ||
42 | 193 | 194 | ||
43 | 194 | """ | 195 | """ |
44 | 195 | CONFIG_FILE_NAME = '.juju-persistent-config' | 196 | CONFIG_FILE_NAME = '.juju-persistent-config' |
45 | 196 | 197 | ||
46 | 197 | def __init__(self, *args, **kw): | 198 | def __init__(self, *args, **kw): |
47 | 198 | super(Config, self).__init__(*args, **kw) | 199 | super(Config, self).__init__(*args, **kw) |
48 | 200 | self.implicit_save = True | ||
49 | 199 | self._prev_dict = None | 201 | self._prev_dict = None |
50 | 200 | self.path = os.path.join(charm_dir(), Config.CONFIG_FILE_NAME) | 202 | self.path = os.path.join(charm_dir(), Config.CONFIG_FILE_NAME) |
51 | 201 | if os.path.exists(self.path): | 203 | if os.path.exists(self.path): |
52 | 202 | self.load_previous() | 204 | self.load_previous() |
53 | 203 | 205 | ||
54 | 204 | def load_previous(self, path=None): | 206 | def load_previous(self, path=None): |
57 | 205 | """Load previous copy of config from disk so that current values | 207 | """Load previous copy of config from disk. |
58 | 206 | can be compared to previous values. | 208 | |
59 | 209 | In normal usage you don't need to call this method directly - it | ||
60 | 210 | is called automatically at object initialization. | ||
61 | 207 | 211 | ||
62 | 208 | :param path: | 212 | :param path: |
63 | 209 | 213 | ||
64 | @@ -218,8 +222,8 @@ | |||
65 | 218 | self._prev_dict = json.load(f) | 222 | self._prev_dict = json.load(f) |
66 | 219 | 223 | ||
67 | 220 | def changed(self, key): | 224 | def changed(self, key): |
70 | 221 | """Return true if the value for this key has changed since | 225 | """Return True if the current value for this key is different from |
71 | 222 | the last save. | 226 | the previous value. |
72 | 223 | 227 | ||
73 | 224 | """ | 228 | """ |
74 | 225 | if self._prev_dict is None: | 229 | if self._prev_dict is None: |
75 | @@ -228,7 +232,7 @@ | |||
76 | 228 | 232 | ||
77 | 229 | def previous(self, key): | 233 | def previous(self, key): |
78 | 230 | """Return previous value for this key, or None if there | 234 | """Return previous value for this key, or None if there |
80 | 231 | is no "previous" value. | 235 | is no previous value. |
81 | 232 | 236 | ||
82 | 233 | """ | 237 | """ |
83 | 234 | if self._prev_dict: | 238 | if self._prev_dict: |
84 | @@ -238,7 +242,13 @@ | |||
85 | 238 | def save(self): | 242 | def save(self): |
86 | 239 | """Save this config to disk. | 243 | """Save this config to disk. |
87 | 240 | 244 | ||
89 | 241 | Preserves items in _prev_dict that do not exist in self. | 245 | If the charm is using the :mod:`Services Framework <services.base>` |
90 | 246 | or :meth:'@hook <Hooks.hook>' decorator, this | ||
91 | 247 | is called automatically at the end of successful hook execution. | ||
92 | 248 | Otherwise, it should be called directly by user code. | ||
93 | 249 | |||
94 | 250 | To disable automatic saves, set ``implicit_save=False`` on this | ||
95 | 251 | instance. | ||
96 | 242 | 252 | ||
97 | 243 | """ | 253 | """ |
98 | 244 | if self._prev_dict: | 254 | if self._prev_dict: |
99 | @@ -478,6 +488,9 @@ | |||
100 | 478 | hook_name = os.path.basename(args[0]) | 488 | hook_name = os.path.basename(args[0]) |
101 | 479 | if hook_name in self._hooks: | 489 | if hook_name in self._hooks: |
102 | 480 | self._hooks[hook_name]() | 490 | self._hooks[hook_name]() |
103 | 491 | cfg = config() | ||
104 | 492 | if cfg.implicit_save: | ||
105 | 493 | cfg.save() | ||
106 | 481 | else: | 494 | else: |
107 | 482 | raise UnregisteredHookError(hook_name) | 495 | raise UnregisteredHookError(hook_name) |
108 | 483 | 496 | ||
109 | 484 | 497 | ||
110 | === modified file 'charmhelpers/core/services/base.py' | |||
111 | --- charmhelpers/core/services/base.py 2014-08-18 17:21:31 +0000 | |||
112 | +++ charmhelpers/core/services/base.py 2014-08-26 15:16:14 +0000 | |||
113 | @@ -118,6 +118,9 @@ | |||
114 | 118 | else: | 118 | else: |
115 | 119 | self.provide_data() | 119 | self.provide_data() |
116 | 120 | self.reconfigure_services() | 120 | self.reconfigure_services() |
117 | 121 | cfg = hookenv.config() | ||
118 | 122 | if cfg.implicit_save: | ||
119 | 123 | cfg.save() | ||
120 | 121 | 124 | ||
121 | 122 | def provide_data(self): | 125 | def provide_data(self): |
122 | 123 | """ | 126 | """ |
123 | 124 | 127 | ||
124 | === modified file 'docs/examples/config.rst' | |||
125 | --- docs/examples/config.rst 2014-06-12 17:55:56 +0000 | |||
126 | +++ docs/examples/config.rst 2014-08-26 15:16:14 +0000 | |||
127 | @@ -38,8 +38,6 @@ | |||
128 | 38 | 38 | ||
129 | 39 | assert config['app-name'] == 'My App' | 39 | assert config['app-name'] == 'My App' |
130 | 40 | 40 | ||
131 | 41 | config.save() | ||
132 | 42 | |||
133 | 43 | Checking if a config value has changed | 41 | Checking if a config value has changed |
134 | 44 | -------------------------------------- | 42 | -------------------------------------- |
135 | 45 | 43 | ||
136 | @@ -64,8 +62,6 @@ | |||
137 | 64 | assert config['app-name'] == 'My New App' | 62 | assert config['app-name'] == 'My New App' |
138 | 65 | assert config.previous('app-name') == 'My App' | 63 | assert config.previous('app-name') == 'My App' |
139 | 66 | 64 | ||
140 | 67 | config.save() | ||
141 | 68 | |||
142 | 69 | Saving arbitrary key/value data | 65 | Saving arbitrary key/value data |
143 | 70 | ------------------------------- | 66 | ------------------------------- |
144 | 71 | 67 | ||
145 | @@ -85,12 +81,8 @@ | |||
146 | 85 | 81 | ||
147 | 86 | config['mykey'] = 'myval' | 82 | config['mykey'] = 'myval' |
148 | 87 | 83 | ||
149 | 88 | config.save() | ||
150 | 89 | |||
151 | 90 | @hooks.hook('config-changed') | 84 | @hooks.hook('config-changed') |
152 | 91 | def config_changed(): | 85 | def config_changed(): |
153 | 92 | config = hookenv.config() | 86 | config = hookenv.config() |
154 | 93 | 87 | ||
155 | 94 | assert config['mykey'] == 'myval' | 88 | assert config['mykey'] == 'myval' |
156 | 95 | |||
157 | 96 | config.save() | ||
158 | 97 | 89 | ||
159 | === modified file 'tests/contrib/ansible/test_ansible.py' | |||
160 | --- tests/contrib/ansible/test_ansible.py 2014-07-30 13:30:43 +0000 | |||
161 | +++ tests/contrib/ansible/test_ansible.py 2014-08-26 15:16:14 +0000 | |||
162 | @@ -11,6 +11,7 @@ | |||
163 | 11 | 11 | ||
164 | 12 | 12 | ||
165 | 13 | import charmhelpers.contrib.ansible | 13 | import charmhelpers.contrib.ansible |
166 | 14 | from charmhelpers.core import hookenv | ||
167 | 14 | 15 | ||
168 | 15 | 16 | ||
169 | 16 | class InstallAnsibleSupportTestCase(unittest.TestCase): | 17 | class InstallAnsibleSupportTestCase(unittest.TestCase): |
170 | @@ -185,7 +186,8 @@ | |||
171 | 185 | 'ansible-playbook', '-c', 'local', 'playbooks/complete-state.yaml', | 186 | 'ansible-playbook', '-c', 'local', 'playbooks/complete-state.yaml', |
172 | 186 | '--tags', 'install,somethingelse']) | 187 | '--tags', 'install,somethingelse']) |
173 | 187 | 188 | ||
175 | 188 | def test_hooks_executes_playbook_with_tag(self): | 189 | @mock.patch.object(hookenv, 'config') |
176 | 190 | def test_hooks_executes_playbook_with_tag(self, config): | ||
177 | 189 | hooks = charmhelpers.contrib.ansible.AnsibleHooks('my/playbook.yaml') | 191 | hooks = charmhelpers.contrib.ansible.AnsibleHooks('my/playbook.yaml') |
178 | 190 | foo = mock.MagicMock() | 192 | foo = mock.MagicMock() |
179 | 191 | hooks.register('foo', foo) | 193 | hooks.register('foo', foo) |
180 | @@ -197,7 +199,8 @@ | |||
181 | 197 | 'ansible-playbook', '-c', 'local', 'my/playbook.yaml', | 199 | 'ansible-playbook', '-c', 'local', 'my/playbook.yaml', |
182 | 198 | '--tags', 'foo']) | 200 | '--tags', 'foo']) |
183 | 199 | 201 | ||
185 | 200 | def test_specifying_ansible_handled_hooks(self): | 202 | @mock.patch.object(hookenv, 'config') |
186 | 203 | def test_specifying_ansible_handled_hooks(self, config): | ||
187 | 201 | hooks = charmhelpers.contrib.ansible.AnsibleHooks( | 204 | hooks = charmhelpers.contrib.ansible.AnsibleHooks( |
188 | 202 | 'my/playbook.yaml', default_hooks=['start', 'stop']) | 205 | 'my/playbook.yaml', default_hooks=['start', 'stop']) |
189 | 203 | 206 | ||
190 | 204 | 207 | ||
191 | === modified file 'tests/core/test_hookenv.py' | |||
192 | --- tests/core/test_hookenv.py 2014-05-06 20:42:06 +0000 | |||
193 | +++ tests/core/test_hookenv.py 2014-08-26 15:16:14 +0000 | |||
194 | @@ -932,6 +932,37 @@ | |||
195 | 932 | 932 | ||
196 | 933 | 933 | ||
197 | 934 | class HooksTest(TestCase): | 934 | class HooksTest(TestCase): |
198 | 935 | def setUp(self): | ||
199 | 936 | super(HooksTest, self).setUp() | ||
200 | 937 | self.config = patch.object(hookenv, 'config') | ||
201 | 938 | self.config.start() | ||
202 | 939 | |||
203 | 940 | def tearDown(self): | ||
204 | 941 | super(HooksTest, self).tearDown() | ||
205 | 942 | self.config.stop() | ||
206 | 943 | |||
207 | 944 | def test_config_saved_after_execute(self): | ||
208 | 945 | config = hookenv.config() | ||
209 | 946 | config.implicit_save = True | ||
210 | 947 | |||
211 | 948 | foo = MagicMock() | ||
212 | 949 | hooks = hookenv.Hooks() | ||
213 | 950 | hooks.register('foo', foo) | ||
214 | 951 | hooks.execute(['foo', 'some', 'other', 'args']) | ||
215 | 952 | |||
216 | 953 | self.assertTrue(config.save.called) | ||
217 | 954 | |||
218 | 955 | def test_config_not_saved_after_execute(self): | ||
219 | 956 | config = hookenv.config() | ||
220 | 957 | config.implicit_save = False | ||
221 | 958 | |||
222 | 959 | foo = MagicMock() | ||
223 | 960 | hooks = hookenv.Hooks() | ||
224 | 961 | hooks.register('foo', foo) | ||
225 | 962 | hooks.execute(['foo', 'some', 'other', 'args']) | ||
226 | 963 | |||
227 | 964 | self.assertFalse(config.save.called) | ||
228 | 965 | |||
229 | 935 | def test_runs_a_registered_function(self): | 966 | def test_runs_a_registered_function(self): |
230 | 936 | foo = MagicMock() | 967 | foo = MagicMock() |
231 | 937 | hooks = hookenv.Hooks() | 968 | hooks = hookenv.Hooks() |
232 | 938 | 969 | ||
233 | === modified file 'tests/core/test_services.py' | |||
234 | --- tests/core/test_services.py 2014-07-10 14:43:59 +0000 | |||
235 | +++ tests/core/test_services.py 2014-08-26 15:16:14 +0000 | |||
236 | @@ -30,7 +30,8 @@ | |||
237 | 30 | @mock.patch.object(services.ServiceManager, 'reconfigure_services') | 30 | @mock.patch.object(services.ServiceManager, 'reconfigure_services') |
238 | 31 | @mock.patch.object(services.ServiceManager, 'stop_services') | 31 | @mock.patch.object(services.ServiceManager, 'stop_services') |
239 | 32 | @mock.patch.object(hookenv, 'hook_name') | 32 | @mock.patch.object(hookenv, 'hook_name') |
241 | 33 | def test_manage_stop(self, hook_name, stop_services, reconfigure_services): | 33 | @mock.patch.object(hookenv, 'config') |
242 | 34 | def test_manage_stop(self, config, hook_name, stop_services, reconfigure_services): | ||
243 | 34 | manager = services.ServiceManager() | 35 | manager = services.ServiceManager() |
244 | 35 | hook_name.return_value = 'stop' | 36 | hook_name.return_value = 'stop' |
245 | 36 | manager.manage() | 37 | manager.manage() |
246 | @@ -41,7 +42,8 @@ | |||
247 | 41 | @mock.patch.object(services.ServiceManager, 'reconfigure_services') | 42 | @mock.patch.object(services.ServiceManager, 'reconfigure_services') |
248 | 42 | @mock.patch.object(services.ServiceManager, 'stop_services') | 43 | @mock.patch.object(services.ServiceManager, 'stop_services') |
249 | 43 | @mock.patch.object(hookenv, 'hook_name') | 44 | @mock.patch.object(hookenv, 'hook_name') |
251 | 44 | def test_manage_other(self, hook_name, stop_services, reconfigure_services, provide_data): | 45 | @mock.patch.object(hookenv, 'config') |
252 | 46 | def test_manage_other(self, config, hook_name, stop_services, reconfigure_services, provide_data): | ||
253 | 45 | manager = services.ServiceManager() | 47 | manager = services.ServiceManager() |
254 | 46 | hook_name.return_value = 'config-changed' | 48 | hook_name.return_value = 'config-changed' |
255 | 47 | manager.manage() | 49 | manager.manage() |
256 | @@ -49,6 +51,22 @@ | |||
257 | 49 | reconfigure_services.assert_called_once_with() | 51 | reconfigure_services.assert_called_once_with() |
258 | 50 | provide_data.assert_called_once_with() | 52 | provide_data.assert_called_once_with() |
259 | 51 | 53 | ||
260 | 54 | @mock.patch.object(hookenv, 'config') | ||
261 | 55 | def test_manage_config_saved(self, config): | ||
262 | 56 | config = config.return_value | ||
263 | 57 | config.implicit_save = True | ||
264 | 58 | manager = services.ServiceManager() | ||
265 | 59 | manager.manage() | ||
266 | 60 | self.assertTrue(config.save.called) | ||
267 | 61 | |||
268 | 62 | @mock.patch.object(hookenv, 'config') | ||
269 | 63 | def test_manage_config_not_saved(self, config): | ||
270 | 64 | config = config.return_value | ||
271 | 65 | config.implicit_save = False | ||
272 | 66 | manager = services.ServiceManager() | ||
273 | 67 | manager.manage() | ||
274 | 68 | self.assertFalse(config.save.called) | ||
275 | 69 | |||
276 | 52 | @mock.patch.object(services.ServiceManager, 'save_ready') | 70 | @mock.patch.object(services.ServiceManager, 'save_ready') |
277 | 53 | @mock.patch.object(services.ServiceManager, 'fire_event') | 71 | @mock.patch.object(services.ServiceManager, 'fire_event') |
278 | 54 | @mock.patch.object(services.ServiceManager, 'is_ready') | 72 | @mock.patch.object(services.ServiceManager, 'is_ready') |
I don't think using the destructor is the right thing to do. It will cause the changes to be persisted even if the hook fails.
The sanest place I can think of to save the config only on successful termination is in the @hooks.hook decorator.