Merge lp:~ivoks/charms/trusty/elasticsearch/refresh into lp:charms/trusty/elasticsearch
- Trusty Tahr (14.04)
- refresh
- Merge into trunk
Status: | Needs review |
---|---|
Proposed branch: | lp:~ivoks/charms/trusty/elasticsearch/refresh |
Merge into: | lp:charms/trusty/elasticsearch |
Diff against target: |
5480 lines (+3717/-626) 42 files modified
README.md (+3/-3) charm-helpers.yaml (+1/-0) config.yaml (+1/-1) hooks/charmhelpers/__init__.py (+36/-0) hooks/charmhelpers/contrib/__init__.py (+13/-0) hooks/charmhelpers/contrib/ansible/__init__.py (+86/-5) hooks/charmhelpers/contrib/templating/__init__.py (+13/-0) hooks/charmhelpers/contrib/templating/contexts.py (+25/-4) hooks/charmhelpers/core/__init__.py (+13/-0) hooks/charmhelpers/core/decorators.py (+55/-0) hooks/charmhelpers/core/files.py (+43/-0) hooks/charmhelpers/core/fstab.py (+28/-12) hooks/charmhelpers/core/hookenv.py (+594/-58) hooks/charmhelpers/core/host.py (+677/-146) hooks/charmhelpers/core/host_factory/centos.py (+72/-0) hooks/charmhelpers/core/host_factory/ubuntu.py (+88/-0) hooks/charmhelpers/core/hugepage.py (+69/-0) hooks/charmhelpers/core/kernel.py (+72/-0) hooks/charmhelpers/core/kernel_factory/centos.py (+17/-0) hooks/charmhelpers/core/kernel_factory/ubuntu.py (+13/-0) hooks/charmhelpers/core/services/__init__.py (+14/-0) hooks/charmhelpers/core/services/base.py (+57/-19) hooks/charmhelpers/core/services/helpers.py (+64/-13) hooks/charmhelpers/core/strutils.py (+123/-0) hooks/charmhelpers/core/sysctl.py (+26/-6) hooks/charmhelpers/core/templating.py (+44/-11) hooks/charmhelpers/core/unitdata.py (+518/-0) hooks/charmhelpers/fetch/__init__.py (+58/-275) hooks/charmhelpers/fetch/archiveurl.py (+75/-18) hooks/charmhelpers/fetch/bzrurl.py (+52/-26) hooks/charmhelpers/fetch/centos.py (+171/-0) hooks/charmhelpers/fetch/giturl.py (+45/-20) hooks/charmhelpers/fetch/snap.py (+122/-0) hooks/charmhelpers/fetch/ubuntu.py (+364/-0) hooks/charmhelpers/osplatform.py (+25/-0) hooks/charmhelpers/payload/__init__.py (+14/-0) hooks/charmhelpers/payload/execd.py (+17/-2) hooks/hooks.py (+1/-1) metadata.yaml (+3/-0) tasks/install-elasticsearch.yml (+1/-1) templates/elasticsearch.yml (+2/-3) unit_tests/test_hooks.py (+2/-2) |
To merge this branch: | bzr merge lp:~ivoks/charms/trusty/elasticsearch/refresh |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
charmers | Pending | ||
Review via email: mp+321247@code.launchpad.net |
Commit message
Description of the change
Elasticsearch charm refreshment. Adds support for newer versions of Elasticsearch, drops obsolete workarounds and adds support for 16.04.
- 57. By Ante Karamatić
-
Allow elasticsearch to start when there's only one unit
discovery.
zen.ping. unicast. hosts cannot be empty, so it should not
be configured when there are no peers - 58. By Ante Karamatić
-
len() doesn't exist in ansible
Check if relations.peer is not empty instead
Unmerged revisions
- 58. By Ante Karamatić
-
len() doesn't exist in ansible
Check if relations.peer is not empty instead
- 57. By Ante Karamatić
-
Allow elasticsearch to start when there's only one unit
discovery.
zen.ping. unicast. hosts cannot be empty, so it should not
be configured when there are no peers - 56. By Ante Karamatić
-
Update README.md
- 55. By Ante Karamatić
-
Drop support for precise
Precise will EOL in few weeks and it doesn't have python3. There's
no justification to invest time to workaround python2/python3 problem. - 54. By Ante Karamatić
-
Adjust tests for new charmhelpers
- 53. By Ante Karamatić
-
Drop obsolete config options in the template
Removed two config options that are not needed any more and
only break newer versions of Elasticsearch - 52. By Ante Karamatić
-
Allow distribution to select JRE
By switching to default-
jre-headless we make sure available JRE
is always installable. This doesn't resolve the requirement
on the user to know which Elasticsearch works with wich version
of JRE - 51. By Ante Karamatić
-
Add series in metadata.yaml and add support for xenial
Previous 2 commits enable successful deployment of Elasticsearch
on Xenial and this commit only declares that in metadata - 50. By Ante Karamatić
-
Switch hooks to python3
This is a requirement for deployment in Ubuntu 16.04 or later
- 49. By Ante Karamatić
-
Change default repo for elasticsearch
New repo is in line with upstream's instructions and also pulls in
latest 5.x version, instead of old 2.x
Preview Diff
1 | === modified file 'README.md' | |||
2 | --- README.md 2014-10-31 03:43:31 +0000 | |||
3 | +++ README.md 2017-05-11 14:56:57 +0000 | |||
4 | @@ -14,11 +14,11 @@ | |||
5 | 14 | 14 | ||
6 | 15 | You can simply deploy one node with: | 15 | You can simply deploy one node with: |
7 | 16 | 16 | ||
9 | 17 | juju deploy elasticsearch | 17 | juju deploy cs:~ivoks/elasticsearch |
10 | 18 | 18 | ||
11 | 19 | You can also deploy and relate the Kibana dashboard: | 19 | You can also deploy and relate the Kibana dashboard: |
12 | 20 | 20 | ||
14 | 21 | juju deploy kibana | 21 | juju deploy cs:~ivoks/kibana |
15 | 22 | juju add-relation kibana elasticsearch | 22 | juju add-relation kibana elasticsearch |
16 | 23 | juju expose kibana | 23 | juju expose kibana |
17 | 24 | 24 | ||
18 | @@ -29,7 +29,7 @@ | |||
19 | 29 | 29 | ||
20 | 30 | Deploy three or more units with: | 30 | Deploy three or more units with: |
21 | 31 | 31 | ||
23 | 32 | juju deploy -n3 elasticsearch | 32 | juju deploy -n3 cs:~ivoks/elasticsearch |
24 | 33 | 33 | ||
25 | 34 | And when they have started you can inspect the cluster health: | 34 | And when they have started you can inspect the cluster health: |
26 | 35 | 35 | ||
27 | 36 | 36 | ||
28 | === modified file 'charm-helpers.yaml' | |||
29 | --- charm-helpers.yaml 2014-04-08 14:58:22 +0000 | |||
30 | +++ charm-helpers.yaml 2017-05-11 14:56:57 +0000 | |||
31 | @@ -6,3 +6,4 @@ | |||
32 | 6 | - contrib.ansible|inc=* | 6 | - contrib.ansible|inc=* |
33 | 7 | - contrib.templating.contexts | 7 | - contrib.templating.contexts |
34 | 8 | - payload.execd|inc=* | 8 | - payload.execd|inc=* |
35 | 9 | - osplatform | ||
36 | 9 | 10 | ||
37 | === modified file 'config.yaml' | |||
38 | --- config.yaml 2016-03-08 16:38:17 +0000 | |||
39 | +++ config.yaml 2017-05-11 14:56:57 +0000 | |||
40 | @@ -1,7 +1,7 @@ | |||
41 | 1 | options: | 1 | options: |
42 | 2 | apt-repository: | 2 | apt-repository: |
43 | 3 | type: string | 3 | type: string |
45 | 4 | default: "deb http://packages.elastic.co/elasticsearch/2.x/debian stable main" | 4 | default: "deb https://artifacts.elastic.co/packages/5.x/apt stable main" |
46 | 5 | description: | | 5 | description: | |
47 | 6 | A deb-line for the apt archive which contains the elasticsearch package. | 6 | A deb-line for the apt archive which contains the elasticsearch package. |
48 | 7 | This is necessary until elasticsearch gets into the debian/ubuntu archives. | 7 | This is necessary until elasticsearch gets into the debian/ubuntu archives. |
49 | 8 | 8 | ||
50 | === modified file 'hooks/charmhelpers/__init__.py' | |||
51 | --- hooks/charmhelpers/__init__.py 2014-02-06 12:54:59 +0000 | |||
52 | +++ hooks/charmhelpers/__init__.py 2017-05-11 14:56:57 +0000 | |||
53 | @@ -0,0 +1,36 @@ | |||
54 | 1 | # Copyright 2014-2015 Canonical Limited. | ||
55 | 2 | # | ||
56 | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); | ||
57 | 4 | # you may not use this file except in compliance with the License. | ||
58 | 5 | # You may obtain a copy of the License at | ||
59 | 6 | # | ||
60 | 7 | # http://www.apache.org/licenses/LICENSE-2.0 | ||
61 | 8 | # | ||
62 | 9 | # Unless required by applicable law or agreed to in writing, software | ||
63 | 10 | # distributed under the License is distributed on an "AS IS" BASIS, | ||
64 | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
65 | 12 | # See the License for the specific language governing permissions and | ||
66 | 13 | # limitations under the License. | ||
67 | 14 | |||
68 | 15 | # Bootstrap charm-helpers, installing its dependencies if necessary using | ||
69 | 16 | # only standard libraries. | ||
70 | 17 | import subprocess | ||
71 | 18 | import sys | ||
72 | 19 | |||
73 | 20 | try: | ||
74 | 21 | import six # flake8: noqa | ||
75 | 22 | except ImportError: | ||
76 | 23 | if sys.version_info.major == 2: | ||
77 | 24 | subprocess.check_call(['apt-get', 'install', '-y', 'python-six']) | ||
78 | 25 | else: | ||
79 | 26 | subprocess.check_call(['apt-get', 'install', '-y', 'python3-six']) | ||
80 | 27 | import six # flake8: noqa | ||
81 | 28 | |||
82 | 29 | try: | ||
83 | 30 | import yaml # flake8: noqa | ||
84 | 31 | except ImportError: | ||
85 | 32 | if sys.version_info.major == 2: | ||
86 | 33 | subprocess.check_call(['apt-get', 'install', '-y', 'python-yaml']) | ||
87 | 34 | else: | ||
88 | 35 | subprocess.check_call(['apt-get', 'install', '-y', 'python3-yaml']) | ||
89 | 36 | import yaml # flake8: noqa | ||
90 | 0 | 37 | ||
91 | === modified file 'hooks/charmhelpers/contrib/__init__.py' | |||
92 | --- hooks/charmhelpers/contrib/__init__.py 2014-02-06 12:54:59 +0000 | |||
93 | +++ hooks/charmhelpers/contrib/__init__.py 2017-05-11 14:56:57 +0000 | |||
94 | @@ -0,0 +1,13 @@ | |||
95 | 1 | # Copyright 2014-2015 Canonical Limited. | ||
96 | 2 | # | ||
97 | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); | ||
98 | 4 | # you may not use this file except in compliance with the License. | ||
99 | 5 | # You may obtain a copy of the License at | ||
100 | 6 | # | ||
101 | 7 | # http://www.apache.org/licenses/LICENSE-2.0 | ||
102 | 8 | # | ||
103 | 9 | # Unless required by applicable law or agreed to in writing, software | ||
104 | 10 | # distributed under the License is distributed on an "AS IS" BASIS, | ||
105 | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
106 | 12 | # See the License for the specific language governing permissions and | ||
107 | 13 | # limitations under the License. | ||
108 | 0 | 14 | ||
109 | === modified file 'hooks/charmhelpers/contrib/ansible/__init__.py' | |||
110 | --- hooks/charmhelpers/contrib/ansible/__init__.py 2014-10-24 16:35:00 +0000 | |||
111 | +++ hooks/charmhelpers/contrib/ansible/__init__.py 2017-05-11 14:56:57 +0000 | |||
112 | @@ -1,3 +1,17 @@ | |||
113 | 1 | # Copyright 2014-2015 Canonical Limited. | ||
114 | 2 | # | ||
115 | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); | ||
116 | 4 | # you may not use this file except in compliance with the License. | ||
117 | 5 | # You may obtain a copy of the License at | ||
118 | 6 | # | ||
119 | 7 | # http://www.apache.org/licenses/LICENSE-2.0 | ||
120 | 8 | # | ||
121 | 9 | # Unless required by applicable law or agreed to in writing, software | ||
122 | 10 | # distributed under the License is distributed on an "AS IS" BASIS, | ||
123 | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
124 | 12 | # See the License for the specific language governing permissions and | ||
125 | 13 | # limitations under the License. | ||
126 | 14 | |||
127 | 1 | # Copyright 2013 Canonical Ltd. | 15 | # Copyright 2013 Canonical Ltd. |
128 | 2 | # | 16 | # |
129 | 3 | # Authors: | 17 | # Authors: |
130 | @@ -59,9 +73,36 @@ | |||
131 | 59 | .. _playbooks: http://www.ansibleworks.com/docs/playbooks.html | 73 | .. _playbooks: http://www.ansibleworks.com/docs/playbooks.html |
132 | 60 | .. _modules: http://www.ansibleworks.com/docs/modules.html | 74 | .. _modules: http://www.ansibleworks.com/docs/modules.html |
133 | 61 | 75 | ||
134 | 76 | A further feature os the ansible hooks is to provide a light weight "action" | ||
135 | 77 | scripting tool. This is a decorator that you apply to a function, and that | ||
136 | 78 | function can now receive cli args, and can pass extra args to the playbook. | ||
137 | 79 | |||
138 | 80 | e.g. | ||
139 | 81 | |||
140 | 82 | |||
141 | 83 | @hooks.action() | ||
142 | 84 | def some_action(amount, force="False"): | ||
143 | 85 | "Usage: some-action AMOUNT [force=True]" # <-- shown on error | ||
144 | 86 | # process the arguments | ||
145 | 87 | # do some calls | ||
146 | 88 | # return extra-vars to be passed to ansible-playbook | ||
147 | 89 | return { | ||
148 | 90 | 'amount': int(amount), | ||
149 | 91 | 'type': force, | ||
150 | 92 | } | ||
151 | 93 | |||
152 | 94 | You can now create a symlink to hooks.py that can be invoked like a hook, but | ||
153 | 95 | with cli params: | ||
154 | 96 | |||
155 | 97 | # link actions/some-action to hooks/hooks.py | ||
156 | 98 | |||
157 | 99 | actions/some-action amount=10 force=true | ||
158 | 100 | |||
159 | 62 | """ | 101 | """ |
160 | 63 | import os | 102 | import os |
161 | 103 | import stat | ||
162 | 64 | import subprocess | 104 | import subprocess |
163 | 105 | import functools | ||
164 | 65 | 106 | ||
165 | 66 | import charmhelpers.contrib.templating.contexts | 107 | import charmhelpers.contrib.templating.contexts |
166 | 67 | import charmhelpers.core.host | 108 | import charmhelpers.core.host |
167 | @@ -96,12 +137,13 @@ | |||
168 | 96 | hosts_file.write('localhost ansible_connection=local') | 137 | hosts_file.write('localhost ansible_connection=local') |
169 | 97 | 138 | ||
170 | 98 | 139 | ||
172 | 99 | def apply_playbook(playbook, tags=None): | 140 | def apply_playbook(playbook, tags=None, extra_vars=None): |
173 | 100 | tags = tags or [] | 141 | tags = tags or [] |
174 | 101 | tags = ",".join(tags) | 142 | tags = ",".join(tags) |
175 | 102 | charmhelpers.contrib.templating.contexts.juju_state_to_yaml( | 143 | charmhelpers.contrib.templating.contexts.juju_state_to_yaml( |
176 | 103 | ansible_vars_path, namespace_separator='__', | 144 | ansible_vars_path, namespace_separator='__', |
178 | 104 | allow_hyphens_in_keys=False) | 145 | allow_hyphens_in_keys=False, mode=(stat.S_IRUSR | stat.S_IWUSR)) |
179 | 146 | |||
180 | 105 | # we want ansible's log output to be unbuffered | 147 | # we want ansible's log output to be unbuffered |
181 | 106 | env = os.environ.copy() | 148 | env = os.environ.copy() |
182 | 107 | env['PYTHONUNBUFFERED'] = "1" | 149 | env['PYTHONUNBUFFERED'] = "1" |
183 | @@ -113,6 +155,9 @@ | |||
184 | 113 | ] | 155 | ] |
185 | 114 | if tags: | 156 | if tags: |
186 | 115 | call.extend(['--tags', '{}'.format(tags)]) | 157 | call.extend(['--tags', '{}'.format(tags)]) |
187 | 158 | if extra_vars: | ||
188 | 159 | extra = ["%s=%s" % (k, v) for k, v in extra_vars.items()] | ||
189 | 160 | call.extend(['--extra-vars', " ".join(extra)]) | ||
190 | 116 | subprocess.check_call(call, env=env) | 161 | subprocess.check_call(call, env=env) |
191 | 117 | 162 | ||
192 | 118 | 163 | ||
193 | @@ -156,16 +201,52 @@ | |||
194 | 156 | """Register any hooks handled by ansible.""" | 201 | """Register any hooks handled by ansible.""" |
195 | 157 | super(AnsibleHooks, self).__init__() | 202 | super(AnsibleHooks, self).__init__() |
196 | 158 | 203 | ||
197 | 204 | self._actions = {} | ||
198 | 159 | self.playbook_path = playbook_path | 205 | self.playbook_path = playbook_path |
199 | 160 | 206 | ||
200 | 161 | default_hooks = default_hooks or [] | 207 | default_hooks = default_hooks or [] |
202 | 162 | noop = lambda *args, **kwargs: None | 208 | |
203 | 209 | def noop(*args, **kwargs): | ||
204 | 210 | pass | ||
205 | 211 | |||
206 | 163 | for hook in default_hooks: | 212 | for hook in default_hooks: |
207 | 164 | self.register(hook, noop) | 213 | self.register(hook, noop) |
208 | 165 | 214 | ||
209 | 215 | def register_action(self, name, function): | ||
210 | 216 | """Register a hook""" | ||
211 | 217 | self._actions[name] = function | ||
212 | 218 | |||
213 | 166 | def execute(self, args): | 219 | def execute(self, args): |
214 | 167 | """Execute the hook followed by the playbook using the hook as tag.""" | 220 | """Execute the hook followed by the playbook using the hook as tag.""" |
215 | 168 | super(AnsibleHooks, self).execute(args) | ||
216 | 169 | hook_name = os.path.basename(args[0]) | 221 | hook_name = os.path.basename(args[0]) |
217 | 222 | extra_vars = None | ||
218 | 223 | if hook_name in self._actions: | ||
219 | 224 | extra_vars = self._actions[hook_name](args[1:]) | ||
220 | 225 | else: | ||
221 | 226 | super(AnsibleHooks, self).execute(args) | ||
222 | 227 | |||
223 | 170 | charmhelpers.contrib.ansible.apply_playbook( | 228 | charmhelpers.contrib.ansible.apply_playbook( |
225 | 171 | self.playbook_path, tags=[hook_name]) | 229 | self.playbook_path, tags=[hook_name], extra_vars=extra_vars) |
226 | 230 | |||
227 | 231 | def action(self, *action_names): | ||
228 | 232 | """Decorator, registering them as actions""" | ||
229 | 233 | def action_wrapper(decorated): | ||
230 | 234 | |||
231 | 235 | @functools.wraps(decorated) | ||
232 | 236 | def wrapper(argv): | ||
233 | 237 | kwargs = dict(arg.split('=') for arg in argv) | ||
234 | 238 | try: | ||
235 | 239 | return decorated(**kwargs) | ||
236 | 240 | except TypeError as e: | ||
237 | 241 | if decorated.__doc__: | ||
238 | 242 | e.args += (decorated.__doc__,) | ||
239 | 243 | raise | ||
240 | 244 | |||
241 | 245 | self.register_action(decorated.__name__, wrapper) | ||
242 | 246 | if '_' in decorated.__name__: | ||
243 | 247 | self.register_action( | ||
244 | 248 | decorated.__name__.replace('_', '-'), wrapper) | ||
245 | 249 | |||
246 | 250 | return wrapper | ||
247 | 251 | |||
248 | 252 | return action_wrapper | ||
249 | 172 | 253 | ||
250 | === modified file 'hooks/charmhelpers/contrib/templating/__init__.py' | |||
251 | --- hooks/charmhelpers/contrib/templating/__init__.py 2014-02-06 12:54:59 +0000 | |||
252 | +++ hooks/charmhelpers/contrib/templating/__init__.py 2017-05-11 14:56:57 +0000 | |||
253 | @@ -0,0 +1,13 @@ | |||
254 | 1 | # Copyright 2014-2015 Canonical Limited. | ||
255 | 2 | # | ||
256 | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); | ||
257 | 4 | # you may not use this file except in compliance with the License. | ||
258 | 5 | # You may obtain a copy of the License at | ||
259 | 6 | # | ||
260 | 7 | # http://www.apache.org/licenses/LICENSE-2.0 | ||
261 | 8 | # | ||
262 | 9 | # Unless required by applicable law or agreed to in writing, software | ||
263 | 10 | # distributed under the License is distributed on an "AS IS" BASIS, | ||
264 | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
265 | 12 | # See the License for the specific language governing permissions and | ||
266 | 13 | # limitations under the License. | ||
267 | 0 | 14 | ||
268 | === modified file 'hooks/charmhelpers/contrib/templating/contexts.py' | |||
269 | --- hooks/charmhelpers/contrib/templating/contexts.py 2014-10-24 16:35:00 +0000 | |||
270 | +++ hooks/charmhelpers/contrib/templating/contexts.py 2017-05-11 14:56:57 +0000 | |||
271 | @@ -1,3 +1,17 @@ | |||
272 | 1 | # Copyright 2014-2015 Canonical Limited. | ||
273 | 2 | # | ||
274 | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); | ||
275 | 4 | # you may not use this file except in compliance with the License. | ||
276 | 5 | # You may obtain a copy of the License at | ||
277 | 6 | # | ||
278 | 7 | # http://www.apache.org/licenses/LICENSE-2.0 | ||
279 | 8 | # | ||
280 | 9 | # Unless required by applicable law or agreed to in writing, software | ||
281 | 10 | # distributed under the License is distributed on an "AS IS" BASIS, | ||
282 | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
283 | 12 | # See the License for the specific language governing permissions and | ||
284 | 13 | # limitations under the License. | ||
285 | 14 | |||
286 | 1 | # Copyright 2013 Canonical Ltd. | 15 | # Copyright 2013 Canonical Ltd. |
287 | 2 | # | 16 | # |
288 | 3 | # Authors: | 17 | # Authors: |
289 | @@ -6,6 +20,8 @@ | |||
290 | 6 | import os | 20 | import os |
291 | 7 | import yaml | 21 | import yaml |
292 | 8 | 22 | ||
293 | 23 | import six | ||
294 | 24 | |||
295 | 9 | import charmhelpers.core.hookenv | 25 | import charmhelpers.core.hookenv |
296 | 10 | 26 | ||
297 | 11 | 27 | ||
298 | @@ -62,7 +78,7 @@ | |||
299 | 62 | 78 | ||
300 | 63 | 79 | ||
301 | 64 | def juju_state_to_yaml(yaml_path, namespace_separator=':', | 80 | def juju_state_to_yaml(yaml_path, namespace_separator=':', |
303 | 65 | allow_hyphens_in_keys=True): | 81 | allow_hyphens_in_keys=True, mode=None): |
304 | 66 | """Update the juju config and state in a yaml file. | 82 | """Update the juju config and state in a yaml file. |
305 | 67 | 83 | ||
306 | 68 | This includes any current relation-get data, and the charm | 84 | This includes any current relation-get data, and the charm |
307 | @@ -92,9 +108,9 @@ | |||
308 | 92 | 108 | ||
309 | 93 | # Don't use non-standard tags for unicode which will not | 109 | # Don't use non-standard tags for unicode which will not |
310 | 94 | # work when salt uses yaml.load_safe. | 110 | # work when salt uses yaml.load_safe. |
314 | 95 | yaml.add_representer(unicode, lambda dumper, | 111 | yaml.add_representer(six.text_type, |
315 | 96 | value: dumper.represent_scalar( | 112 | lambda dumper, value: dumper.represent_scalar( |
316 | 97 | u'tag:yaml.org,2002:str', value)) | 113 | six.u('tag:yaml.org,2002:str'), value)) |
317 | 98 | 114 | ||
318 | 99 | yaml_dir = os.path.dirname(yaml_path) | 115 | yaml_dir = os.path.dirname(yaml_path) |
319 | 100 | if not os.path.exists(yaml_dir): | 116 | if not os.path.exists(yaml_dir): |
320 | @@ -104,8 +120,13 @@ | |||
321 | 104 | with open(yaml_path, "r") as existing_vars_file: | 120 | with open(yaml_path, "r") as existing_vars_file: |
322 | 105 | existing_vars = yaml.load(existing_vars_file.read()) | 121 | existing_vars = yaml.load(existing_vars_file.read()) |
323 | 106 | else: | 122 | else: |
324 | 123 | with open(yaml_path, "w+"): | ||
325 | 124 | pass | ||
326 | 107 | existing_vars = {} | 125 | existing_vars = {} |
327 | 108 | 126 | ||
328 | 127 | if mode is not None: | ||
329 | 128 | os.chmod(yaml_path, mode) | ||
330 | 129 | |||
331 | 109 | if not allow_hyphens_in_keys: | 130 | if not allow_hyphens_in_keys: |
332 | 110 | config = dict_keys_without_hyphens(config) | 131 | config = dict_keys_without_hyphens(config) |
333 | 111 | existing_vars.update(config) | 132 | existing_vars.update(config) |
334 | 112 | 133 | ||
335 | === modified file 'hooks/charmhelpers/core/__init__.py' | |||
336 | --- hooks/charmhelpers/core/__init__.py 2014-02-06 12:54:59 +0000 | |||
337 | +++ hooks/charmhelpers/core/__init__.py 2017-05-11 14:56:57 +0000 | |||
338 | @@ -0,0 +1,13 @@ | |||
339 | 1 | # Copyright 2014-2015 Canonical Limited. | ||
340 | 2 | # | ||
341 | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); | ||
342 | 4 | # you may not use this file except in compliance with the License. | ||
343 | 5 | # You may obtain a copy of the License at | ||
344 | 6 | # | ||
345 | 7 | # http://www.apache.org/licenses/LICENSE-2.0 | ||
346 | 8 | # | ||
347 | 9 | # Unless required by applicable law or agreed to in writing, software | ||
348 | 10 | # distributed under the License is distributed on an "AS IS" BASIS, | ||
349 | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
350 | 12 | # See the License for the specific language governing permissions and | ||
351 | 13 | # limitations under the License. | ||
352 | 0 | 14 | ||
353 | === added file 'hooks/charmhelpers/core/decorators.py' | |||
354 | --- hooks/charmhelpers/core/decorators.py 1970-01-01 00:00:00 +0000 | |||
355 | +++ hooks/charmhelpers/core/decorators.py 2017-05-11 14:56:57 +0000 | |||
356 | @@ -0,0 +1,55 @@ | |||
357 | 1 | # Copyright 2014-2015 Canonical Limited. | ||
358 | 2 | # | ||
359 | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); | ||
360 | 4 | # you may not use this file except in compliance with the License. | ||
361 | 5 | # You may obtain a copy of the License at | ||
362 | 6 | # | ||
363 | 7 | # http://www.apache.org/licenses/LICENSE-2.0 | ||
364 | 8 | # | ||
365 | 9 | # Unless required by applicable law or agreed to in writing, software | ||
366 | 10 | # distributed under the License is distributed on an "AS IS" BASIS, | ||
367 | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
368 | 12 | # See the License for the specific language governing permissions and | ||
369 | 13 | # limitations under the License. | ||
370 | 14 | |||
371 | 15 | # | ||
372 | 16 | # Copyright 2014 Canonical Ltd. | ||
373 | 17 | # | ||
374 | 18 | # Authors: | ||
375 | 19 | # Edward Hope-Morley <opentastic@gmail.com> | ||
376 | 20 | # | ||
377 | 21 | |||
378 | 22 | import time | ||
379 | 23 | |||
380 | 24 | from charmhelpers.core.hookenv import ( | ||
381 | 25 | log, | ||
382 | 26 | INFO, | ||
383 | 27 | ) | ||
384 | 28 | |||
385 | 29 | |||
386 | 30 | def retry_on_exception(num_retries, base_delay=0, exc_type=Exception): | ||
387 | 31 | """If the decorated function raises exception exc_type, allow num_retries | ||
388 | 32 | retry attempts before raise the exception. | ||
389 | 33 | """ | ||
390 | 34 | def _retry_on_exception_inner_1(f): | ||
391 | 35 | def _retry_on_exception_inner_2(*args, **kwargs): | ||
392 | 36 | retries = num_retries | ||
393 | 37 | multiplier = 1 | ||
394 | 38 | while True: | ||
395 | 39 | try: | ||
396 | 40 | return f(*args, **kwargs) | ||
397 | 41 | except exc_type: | ||
398 | 42 | if not retries: | ||
399 | 43 | raise | ||
400 | 44 | |||
401 | 45 | delay = base_delay * multiplier | ||
402 | 46 | multiplier += 1 | ||
403 | 47 | log("Retrying '%s' %d more times (delay=%s)" % | ||
404 | 48 | (f.__name__, retries, delay), level=INFO) | ||
405 | 49 | retries -= 1 | ||
406 | 50 | if delay: | ||
407 | 51 | time.sleep(delay) | ||
408 | 52 | |||
409 | 53 | return _retry_on_exception_inner_2 | ||
410 | 54 | |||
411 | 55 | return _retry_on_exception_inner_1 | ||
412 | 0 | 56 | ||
413 | === added file 'hooks/charmhelpers/core/files.py' | |||
414 | --- hooks/charmhelpers/core/files.py 1970-01-01 00:00:00 +0000 | |||
415 | +++ hooks/charmhelpers/core/files.py 2017-05-11 14:56:57 +0000 | |||
416 | @@ -0,0 +1,43 @@ | |||
417 | 1 | #!/usr/bin/env python | ||
418 | 2 | # -*- coding: utf-8 -*- | ||
419 | 3 | |||
420 | 4 | # Copyright 2014-2015 Canonical Limited. | ||
421 | 5 | # | ||
422 | 6 | # Licensed under the Apache License, Version 2.0 (the "License"); | ||
423 | 7 | # you may not use this file except in compliance with the License. | ||
424 | 8 | # You may obtain a copy of the License at | ||
425 | 9 | # | ||
426 | 10 | # http://www.apache.org/licenses/LICENSE-2.0 | ||
427 | 11 | # | ||
428 | 12 | # Unless required by applicable law or agreed to in writing, software | ||
429 | 13 | # distributed under the License is distributed on an "AS IS" BASIS, | ||
430 | 14 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
431 | 15 | # See the License for the specific language governing permissions and | ||
432 | 16 | # limitations under the License. | ||
433 | 17 | |||
434 | 18 | __author__ = 'Jorge Niedbalski <niedbalski@ubuntu.com>' | ||
435 | 19 | |||
436 | 20 | import os | ||
437 | 21 | import subprocess | ||
438 | 22 | |||
439 | 23 | |||
440 | 24 | def sed(filename, before, after, flags='g'): | ||
441 | 25 | """ | ||
442 | 26 | Search and replaces the given pattern on filename. | ||
443 | 27 | |||
444 | 28 | :param filename: relative or absolute file path. | ||
445 | 29 | :param before: expression to be replaced (see 'man sed') | ||
446 | 30 | :param after: expression to replace with (see 'man sed') | ||
447 | 31 | :param flags: sed-compatible regex flags in example, to make | ||
448 | 32 | the search and replace case insensitive, specify ``flags="i"``. | ||
449 | 33 | The ``g`` flag is always specified regardless, so you do not | ||
450 | 34 | need to remember to include it when overriding this parameter. | ||
451 | 35 | :returns: If the sed command exit code was zero then return, | ||
452 | 36 | otherwise raise CalledProcessError. | ||
453 | 37 | """ | ||
454 | 38 | expression = r's/{0}/{1}/{2}'.format(before, | ||
455 | 39 | after, flags) | ||
456 | 40 | |||
457 | 41 | return subprocess.check_call(["sed", "-i", "-r", "-e", | ||
458 | 42 | expression, | ||
459 | 43 | os.path.expanduser(filename)]) | ||
460 | 0 | 44 | ||
461 | === modified file 'hooks/charmhelpers/core/fstab.py' | |||
462 | --- hooks/charmhelpers/core/fstab.py 2014-10-24 16:35:00 +0000 | |||
463 | +++ hooks/charmhelpers/core/fstab.py 2017-05-11 14:56:57 +0000 | |||
464 | @@ -1,12 +1,27 @@ | |||
465 | 1 | #!/usr/bin/env python | 1 | #!/usr/bin/env python |
466 | 2 | # -*- coding: utf-8 -*- | 2 | # -*- coding: utf-8 -*- |
467 | 3 | 3 | ||
468 | 4 | # Copyright 2014-2015 Canonical Limited. | ||
469 | 5 | # | ||
470 | 6 | # Licensed under the Apache License, Version 2.0 (the "License"); | ||
471 | 7 | # you may not use this file except in compliance with the License. | ||
472 | 8 | # You may obtain a copy of the License at | ||
473 | 9 | # | ||
474 | 10 | # http://www.apache.org/licenses/LICENSE-2.0 | ||
475 | 11 | # | ||
476 | 12 | # Unless required by applicable law or agreed to in writing, software | ||
477 | 13 | # distributed under the License is distributed on an "AS IS" BASIS, | ||
478 | 14 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
479 | 15 | # See the License for the specific language governing permissions and | ||
480 | 16 | # limitations under the License. | ||
481 | 17 | |||
482 | 18 | import io | ||
483 | 19 | import os | ||
484 | 20 | |||
485 | 4 | __author__ = 'Jorge Niedbalski R. <jorge.niedbalski@canonical.com>' | 21 | __author__ = 'Jorge Niedbalski R. <jorge.niedbalski@canonical.com>' |
486 | 5 | 22 | ||
491 | 6 | import os | 23 | |
492 | 7 | 24 | class Fstab(io.FileIO): | |
489 | 8 | |||
490 | 9 | class Fstab(file): | ||
493 | 10 | """This class extends file in order to implement a file reader/writer | 25 | """This class extends file in order to implement a file reader/writer |
494 | 11 | for file `/etc/fstab` | 26 | for file `/etc/fstab` |
495 | 12 | """ | 27 | """ |
496 | @@ -24,8 +39,8 @@ | |||
497 | 24 | options = "defaults" | 39 | options = "defaults" |
498 | 25 | 40 | ||
499 | 26 | self.options = options | 41 | self.options = options |
502 | 27 | self.d = d | 42 | self.d = int(d) |
503 | 28 | self.p = p | 43 | self.p = int(p) |
504 | 29 | 44 | ||
505 | 30 | def __eq__(self, o): | 45 | def __eq__(self, o): |
506 | 31 | return str(self) == str(o) | 46 | return str(self) == str(o) |
507 | @@ -45,7 +60,7 @@ | |||
508 | 45 | self._path = path | 60 | self._path = path |
509 | 46 | else: | 61 | else: |
510 | 47 | self._path = self.DEFAULT_PATH | 62 | self._path = self.DEFAULT_PATH |
512 | 48 | file.__init__(self, self._path, 'r+') | 63 | super(Fstab, self).__init__(self._path, 'rb+') |
513 | 49 | 64 | ||
514 | 50 | def _hydrate_entry(self, line): | 65 | def _hydrate_entry(self, line): |
515 | 51 | # NOTE: use split with no arguments to split on any | 66 | # NOTE: use split with no arguments to split on any |
516 | @@ -58,8 +73,9 @@ | |||
517 | 58 | def entries(self): | 73 | def entries(self): |
518 | 59 | self.seek(0) | 74 | self.seek(0) |
519 | 60 | for line in self.readlines(): | 75 | for line in self.readlines(): |
520 | 76 | line = line.decode('us-ascii') | ||
521 | 61 | try: | 77 | try: |
523 | 62 | if not line.startswith("#"): | 78 | if line.strip() and not line.strip().startswith("#"): |
524 | 63 | yield self._hydrate_entry(line) | 79 | yield self._hydrate_entry(line) |
525 | 64 | except ValueError: | 80 | except ValueError: |
526 | 65 | pass | 81 | pass |
527 | @@ -75,18 +91,18 @@ | |||
528 | 75 | if self.get_entry_by_attr('device', entry.device): | 91 | if self.get_entry_by_attr('device', entry.device): |
529 | 76 | return False | 92 | return False |
530 | 77 | 93 | ||
532 | 78 | self.write(str(entry) + '\n') | 94 | self.write((str(entry) + '\n').encode('us-ascii')) |
533 | 79 | self.truncate() | 95 | self.truncate() |
534 | 80 | return entry | 96 | return entry |
535 | 81 | 97 | ||
536 | 82 | def remove_entry(self, entry): | 98 | def remove_entry(self, entry): |
537 | 83 | self.seek(0) | 99 | self.seek(0) |
538 | 84 | 100 | ||
540 | 85 | lines = self.readlines() | 101 | lines = [l.decode('us-ascii') for l in self.readlines()] |
541 | 86 | 102 | ||
542 | 87 | found = False | 103 | found = False |
543 | 88 | for index, line in enumerate(lines): | 104 | for index, line in enumerate(lines): |
545 | 89 | if not line.startswith("#"): | 105 | if line.strip() and not line.strip().startswith("#"): |
546 | 90 | if self._hydrate_entry(line) == entry: | 106 | if self._hydrate_entry(line) == entry: |
547 | 91 | found = True | 107 | found = True |
548 | 92 | break | 108 | break |
549 | @@ -97,7 +113,7 @@ | |||
550 | 97 | lines.remove(line) | 113 | lines.remove(line) |
551 | 98 | 114 | ||
552 | 99 | self.seek(0) | 115 | self.seek(0) |
554 | 100 | self.write(''.join(lines)) | 116 | self.write(''.join(lines).encode('us-ascii')) |
555 | 101 | self.truncate() | 117 | self.truncate() |
556 | 102 | return True | 118 | return True |
557 | 103 | 119 | ||
558 | 104 | 120 | ||
559 | === modified file 'hooks/charmhelpers/core/hookenv.py' | |||
560 | --- hooks/charmhelpers/core/hookenv.py 2014-10-24 16:35:00 +0000 | |||
561 | +++ hooks/charmhelpers/core/hookenv.py 2017-05-11 14:56:57 +0000 | |||
562 | @@ -1,17 +1,43 @@ | |||
563 | 1 | # Copyright 2014-2015 Canonical Limited. | ||
564 | 2 | # | ||
565 | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); | ||
566 | 4 | # you may not use this file except in compliance with the License. | ||
567 | 5 | # You may obtain a copy of the License at | ||
568 | 6 | # | ||
569 | 7 | # http://www.apache.org/licenses/LICENSE-2.0 | ||
570 | 8 | # | ||
571 | 9 | # Unless required by applicable law or agreed to in writing, software | ||
572 | 10 | # distributed under the License is distributed on an "AS IS" BASIS, | ||
573 | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
574 | 12 | # See the License for the specific language governing permissions and | ||
575 | 13 | # limitations under the License. | ||
576 | 14 | |||
577 | 1 | "Interactions with the Juju environment" | 15 | "Interactions with the Juju environment" |
578 | 2 | # Copyright 2013 Canonical Ltd. | 16 | # Copyright 2013 Canonical Ltd. |
579 | 3 | # | 17 | # |
580 | 4 | # Authors: | 18 | # Authors: |
581 | 5 | # Charm Helpers Developers <juju@lists.ubuntu.com> | 19 | # Charm Helpers Developers <juju@lists.ubuntu.com> |
582 | 6 | 20 | ||
583 | 21 | from __future__ import print_function | ||
584 | 22 | import copy | ||
585 | 23 | from distutils.version import LooseVersion | ||
586 | 24 | from functools import wraps | ||
587 | 25 | import glob | ||
588 | 7 | import os | 26 | import os |
589 | 8 | import json | 27 | import json |
590 | 9 | import yaml | 28 | import yaml |
591 | 10 | import subprocess | 29 | import subprocess |
592 | 11 | import sys | 30 | import sys |
594 | 12 | import UserDict | 31 | import errno |
595 | 32 | import tempfile | ||
596 | 13 | from subprocess import CalledProcessError | 33 | from subprocess import CalledProcessError |
597 | 14 | 34 | ||
598 | 35 | import six | ||
599 | 36 | if not six.PY3: | ||
600 | 37 | from UserDict import UserDict | ||
601 | 38 | else: | ||
602 | 39 | from collections import UserDict | ||
603 | 40 | |||
604 | 15 | CRITICAL = "CRITICAL" | 41 | CRITICAL = "CRITICAL" |
605 | 16 | ERROR = "ERROR" | 42 | ERROR = "ERROR" |
606 | 17 | WARNING = "WARNING" | 43 | WARNING = "WARNING" |
607 | @@ -35,15 +61,18 @@ | |||
608 | 35 | 61 | ||
609 | 36 | will cache the result of unit_get + 'test' for future calls. | 62 | will cache the result of unit_get + 'test' for future calls. |
610 | 37 | """ | 63 | """ |
611 | 64 | @wraps(func) | ||
612 | 38 | def wrapper(*args, **kwargs): | 65 | def wrapper(*args, **kwargs): |
613 | 39 | global cache | 66 | global cache |
614 | 40 | key = str((func, args, kwargs)) | 67 | key = str((func, args, kwargs)) |
615 | 41 | try: | 68 | try: |
616 | 42 | return cache[key] | 69 | return cache[key] |
617 | 43 | except KeyError: | 70 | except KeyError: |
621 | 44 | res = func(*args, **kwargs) | 71 | pass # Drop out of the exception handler scope. |
622 | 45 | cache[key] = res | 72 | res = func(*args, **kwargs) |
623 | 46 | return res | 73 | cache[key] = res |
624 | 74 | return res | ||
625 | 75 | wrapper._wrapped = func | ||
626 | 47 | return wrapper | 76 | return wrapper |
627 | 48 | 77 | ||
628 | 49 | 78 | ||
629 | @@ -63,16 +92,29 @@ | |||
630 | 63 | command = ['juju-log'] | 92 | command = ['juju-log'] |
631 | 64 | if level: | 93 | if level: |
632 | 65 | command += ['-l', level] | 94 | command += ['-l', level] |
633 | 95 | if not isinstance(message, six.string_types): | ||
634 | 96 | message = repr(message) | ||
635 | 66 | command += [message] | 97 | command += [message] |
640 | 67 | subprocess.call(command) | 98 | # Missing juju-log should not cause failures in unit tests |
641 | 68 | 99 | # Send log output to stderr | |
642 | 69 | 100 | try: | |
643 | 70 | class Serializable(UserDict.IterableUserDict): | 101 | subprocess.call(command) |
644 | 102 | except OSError as e: | ||
645 | 103 | if e.errno == errno.ENOENT: | ||
646 | 104 | if level: | ||
647 | 105 | message = "{}: {}".format(level, message) | ||
648 | 106 | message = "juju-log: {}".format(message) | ||
649 | 107 | print(message, file=sys.stderr) | ||
650 | 108 | else: | ||
651 | 109 | raise | ||
652 | 110 | |||
653 | 111 | |||
654 | 112 | class Serializable(UserDict): | ||
655 | 71 | """Wrapper, an object that can be serialized to yaml or json""" | 113 | """Wrapper, an object that can be serialized to yaml or json""" |
656 | 72 | 114 | ||
657 | 73 | def __init__(self, obj): | 115 | def __init__(self, obj): |
658 | 74 | # wrap the object | 116 | # wrap the object |
660 | 75 | UserDict.IterableUserDict.__init__(self) | 117 | UserDict.__init__(self) |
661 | 76 | self.data = obj | 118 | self.data = obj |
662 | 77 | 119 | ||
663 | 78 | def __getattr__(self, attr): | 120 | def __getattr__(self, attr): |
664 | @@ -130,9 +172,19 @@ | |||
665 | 130 | return os.environ.get('JUJU_RELATION', None) | 172 | return os.environ.get('JUJU_RELATION', None) |
666 | 131 | 173 | ||
667 | 132 | 174 | ||
671 | 133 | def relation_id(): | 175 | @cached |
672 | 134 | """The relation ID for the current relation hook""" | 176 | def relation_id(relation_name=None, service_or_unit=None): |
673 | 135 | return os.environ.get('JUJU_RELATION_ID', None) | 177 | """The relation ID for the current or a specified relation""" |
674 | 178 | if not relation_name and not service_or_unit: | ||
675 | 179 | return os.environ.get('JUJU_RELATION_ID', None) | ||
676 | 180 | elif relation_name and service_or_unit: | ||
677 | 181 | service_name = service_or_unit.split('/')[0] | ||
678 | 182 | for relid in relation_ids(relation_name): | ||
679 | 183 | remote_service = remote_service_name(relid) | ||
680 | 184 | if remote_service == service_name: | ||
681 | 185 | return relid | ||
682 | 186 | else: | ||
683 | 187 | raise ValueError('Must specify neither or both of relation_name and service_or_unit') | ||
684 | 136 | 188 | ||
685 | 137 | 189 | ||
686 | 138 | def local_unit(): | 190 | def local_unit(): |
687 | @@ -142,7 +194,7 @@ | |||
688 | 142 | 194 | ||
689 | 143 | def remote_unit(): | 195 | def remote_unit(): |
690 | 144 | """The remote unit for the current relation hook""" | 196 | """The remote unit for the current relation hook""" |
692 | 145 | return os.environ['JUJU_REMOTE_UNIT'] | 197 | return os.environ.get('JUJU_REMOTE_UNIT', None) |
693 | 146 | 198 | ||
694 | 147 | 199 | ||
695 | 148 | def service_name(): | 200 | def service_name(): |
696 | @@ -150,9 +202,20 @@ | |||
697 | 150 | return local_unit().split('/')[0] | 202 | return local_unit().split('/')[0] |
698 | 151 | 203 | ||
699 | 152 | 204 | ||
700 | 205 | @cached | ||
701 | 206 | def remote_service_name(relid=None): | ||
702 | 207 | """The remote service name for a given relation-id (or the current relation)""" | ||
703 | 208 | if relid is None: | ||
704 | 209 | unit = remote_unit() | ||
705 | 210 | else: | ||
706 | 211 | units = related_units(relid) | ||
707 | 212 | unit = units[0] if units else None | ||
708 | 213 | return unit.split('/')[0] if unit else None | ||
709 | 214 | |||
710 | 215 | |||
711 | 153 | def hook_name(): | 216 | def hook_name(): |
712 | 154 | """The name of the currently executing hook""" | 217 | """The name of the currently executing hook""" |
714 | 155 | return os.path.basename(sys.argv[0]) | 218 | return os.environ.get('JUJU_HOOK_NAME', os.path.basename(sys.argv[0])) |
715 | 156 | 219 | ||
716 | 157 | 220 | ||
717 | 158 | class Config(dict): | 221 | class Config(dict): |
718 | @@ -202,23 +265,7 @@ | |||
719 | 202 | self.path = os.path.join(charm_dir(), Config.CONFIG_FILE_NAME) | 265 | self.path = os.path.join(charm_dir(), Config.CONFIG_FILE_NAME) |
720 | 203 | if os.path.exists(self.path): | 266 | if os.path.exists(self.path): |
721 | 204 | self.load_previous() | 267 | self.load_previous() |
739 | 205 | 268 | atexit(self._implicit_save) | |
723 | 206 | def __getitem__(self, key): | ||
724 | 207 | """For regular dict lookups, check the current juju config first, | ||
725 | 208 | then the previous (saved) copy. This ensures that user-saved values | ||
726 | 209 | will be returned by a dict lookup. | ||
727 | 210 | |||
728 | 211 | """ | ||
729 | 212 | try: | ||
730 | 213 | return dict.__getitem__(self, key) | ||
731 | 214 | except KeyError: | ||
732 | 215 | return (self._prev_dict or {})[key] | ||
733 | 216 | |||
734 | 217 | def keys(self): | ||
735 | 218 | prev_keys = [] | ||
736 | 219 | if self._prev_dict is not None: | ||
737 | 220 | prev_keys = self._prev_dict.keys() | ||
738 | 221 | return list(set(prev_keys + dict.keys(self))) | ||
740 | 222 | 269 | ||
741 | 223 | def load_previous(self, path=None): | 270 | def load_previous(self, path=None): |
742 | 224 | """Load previous copy of config from disk. | 271 | """Load previous copy of config from disk. |
743 | @@ -237,6 +284,9 @@ | |||
744 | 237 | self.path = path or self.path | 284 | self.path = path or self.path |
745 | 238 | with open(self.path) as f: | 285 | with open(self.path) as f: |
746 | 239 | self._prev_dict = json.load(f) | 286 | self._prev_dict = json.load(f) |
747 | 287 | for k, v in copy.deepcopy(self._prev_dict).items(): | ||
748 | 288 | if k not in self: | ||
749 | 289 | self[k] = v | ||
750 | 240 | 290 | ||
751 | 241 | def changed(self, key): | 291 | def changed(self, key): |
752 | 242 | """Return True if the current value for this key is different from | 292 | """Return True if the current value for this key is different from |
753 | @@ -268,13 +318,13 @@ | |||
754 | 268 | instance. | 318 | instance. |
755 | 269 | 319 | ||
756 | 270 | """ | 320 | """ |
757 | 271 | if self._prev_dict: | ||
758 | 272 | for k, v in self._prev_dict.iteritems(): | ||
759 | 273 | if k not in self: | ||
760 | 274 | self[k] = v | ||
761 | 275 | with open(self.path, 'w') as f: | 321 | with open(self.path, 'w') as f: |
762 | 276 | json.dump(self, f) | 322 | json.dump(self, f) |
763 | 277 | 323 | ||
764 | 324 | def _implicit_save(self): | ||
765 | 325 | if self.implicit_save: | ||
766 | 326 | self.save() | ||
767 | 327 | |||
768 | 278 | 328 | ||
769 | 279 | @cached | 329 | @cached |
770 | 280 | def config(scope=None): | 330 | def config(scope=None): |
771 | @@ -282,9 +332,12 @@ | |||
772 | 282 | config_cmd_line = ['config-get'] | 332 | config_cmd_line = ['config-get'] |
773 | 283 | if scope is not None: | 333 | if scope is not None: |
774 | 284 | config_cmd_line.append(scope) | 334 | config_cmd_line.append(scope) |
775 | 335 | else: | ||
776 | 336 | config_cmd_line.append('--all') | ||
777 | 285 | config_cmd_line.append('--format=json') | 337 | config_cmd_line.append('--format=json') |
778 | 286 | try: | 338 | try: |
780 | 287 | config_data = json.loads(subprocess.check_output(config_cmd_line)) | 339 | config_data = json.loads( |
781 | 340 | subprocess.check_output(config_cmd_line).decode('UTF-8')) | ||
782 | 288 | if scope is not None: | 341 | if scope is not None: |
783 | 289 | return config_data | 342 | return config_data |
784 | 290 | return Config(config_data) | 343 | return Config(config_data) |
785 | @@ -303,10 +356,10 @@ | |||
786 | 303 | if unit: | 356 | if unit: |
787 | 304 | _args.append(unit) | 357 | _args.append(unit) |
788 | 305 | try: | 358 | try: |
790 | 306 | return json.loads(subprocess.check_output(_args)) | 359 | return json.loads(subprocess.check_output(_args).decode('UTF-8')) |
791 | 307 | except ValueError: | 360 | except ValueError: |
792 | 308 | return None | 361 | return None |
794 | 309 | except CalledProcessError, e: | 362 | except CalledProcessError as e: |
795 | 310 | if e.returncode == 2: | 363 | if e.returncode == 2: |
796 | 311 | return None | 364 | return None |
797 | 312 | raise | 365 | raise |
798 | @@ -316,18 +369,49 @@ | |||
799 | 316 | """Set relation information for the current unit""" | 369 | """Set relation information for the current unit""" |
800 | 317 | relation_settings = relation_settings if relation_settings else {} | 370 | relation_settings = relation_settings if relation_settings else {} |
801 | 318 | relation_cmd_line = ['relation-set'] | 371 | relation_cmd_line = ['relation-set'] |
802 | 372 | accepts_file = "--file" in subprocess.check_output( | ||
803 | 373 | relation_cmd_line + ["--help"], universal_newlines=True) | ||
804 | 319 | if relation_id is not None: | 374 | if relation_id is not None: |
805 | 320 | relation_cmd_line.extend(('-r', relation_id)) | 375 | relation_cmd_line.extend(('-r', relation_id)) |
812 | 321 | for k, v in (relation_settings.items() + kwargs.items()): | 376 | settings = relation_settings.copy() |
813 | 322 | if v is None: | 377 | settings.update(kwargs) |
814 | 323 | relation_cmd_line.append('{}='.format(k)) | 378 | for key, value in settings.items(): |
815 | 324 | else: | 379 | # Force value to be a string: it always should, but some call |
816 | 325 | relation_cmd_line.append('{}={}'.format(k, v)) | 380 | # sites pass in things like dicts or numbers. |
817 | 326 | subprocess.check_call(relation_cmd_line) | 381 | if value is not None: |
818 | 382 | settings[key] = "{}".format(value) | ||
819 | 383 | if accepts_file: | ||
820 | 384 | # --file was introduced in Juju 1.23.2. Use it by default if | ||
821 | 385 | # available, since otherwise we'll break if the relation data is | ||
822 | 386 | # too big. Ideally we should tell relation-set to read the data from | ||
823 | 387 | # stdin, but that feature is broken in 1.23.2: Bug #1454678. | ||
824 | 388 | with tempfile.NamedTemporaryFile(delete=False) as settings_file: | ||
825 | 389 | settings_file.write(yaml.safe_dump(settings).encode("utf-8")) | ||
826 | 390 | subprocess.check_call( | ||
827 | 391 | relation_cmd_line + ["--file", settings_file.name]) | ||
828 | 392 | os.remove(settings_file.name) | ||
829 | 393 | else: | ||
830 | 394 | for key, value in settings.items(): | ||
831 | 395 | if value is None: | ||
832 | 396 | relation_cmd_line.append('{}='.format(key)) | ||
833 | 397 | else: | ||
834 | 398 | relation_cmd_line.append('{}={}'.format(key, value)) | ||
835 | 399 | subprocess.check_call(relation_cmd_line) | ||
836 | 327 | # Flush cache of any relation-gets for local unit | 400 | # Flush cache of any relation-gets for local unit |
837 | 328 | flush(local_unit()) | 401 | flush(local_unit()) |
838 | 329 | 402 | ||
839 | 330 | 403 | ||
840 | 404 | def relation_clear(r_id=None): | ||
841 | 405 | ''' Clears any relation data already set on relation r_id ''' | ||
842 | 406 | settings = relation_get(rid=r_id, | ||
843 | 407 | unit=local_unit()) | ||
844 | 408 | for setting in settings: | ||
845 | 409 | if setting not in ['public-address', 'private-address']: | ||
846 | 410 | settings[setting] = None | ||
847 | 411 | relation_set(relation_id=r_id, | ||
848 | 412 | **settings) | ||
849 | 413 | |||
850 | 414 | |||
851 | 331 | @cached | 415 | @cached |
852 | 332 | def relation_ids(reltype=None): | 416 | def relation_ids(reltype=None): |
853 | 333 | """A list of relation_ids""" | 417 | """A list of relation_ids""" |
854 | @@ -335,7 +419,8 @@ | |||
855 | 335 | relid_cmd_line = ['relation-ids', '--format=json'] | 419 | relid_cmd_line = ['relation-ids', '--format=json'] |
856 | 336 | if reltype is not None: | 420 | if reltype is not None: |
857 | 337 | relid_cmd_line.append(reltype) | 421 | relid_cmd_line.append(reltype) |
859 | 338 | return json.loads(subprocess.check_output(relid_cmd_line)) or [] | 422 | return json.loads( |
860 | 423 | subprocess.check_output(relid_cmd_line).decode('UTF-8')) or [] | ||
861 | 339 | return [] | 424 | return [] |
862 | 340 | 425 | ||
863 | 341 | 426 | ||
864 | @@ -346,7 +431,8 @@ | |||
865 | 346 | units_cmd_line = ['relation-list', '--format=json'] | 431 | units_cmd_line = ['relation-list', '--format=json'] |
866 | 347 | if relid is not None: | 432 | if relid is not None: |
867 | 348 | units_cmd_line.extend(('-r', relid)) | 433 | units_cmd_line.extend(('-r', relid)) |
869 | 349 | return json.loads(subprocess.check_output(units_cmd_line)) or [] | 434 | return json.loads( |
870 | 435 | subprocess.check_output(units_cmd_line).decode('UTF-8')) or [] | ||
871 | 350 | 436 | ||
872 | 351 | 437 | ||
873 | 352 | @cached | 438 | @cached |
874 | @@ -386,21 +472,101 @@ | |||
875 | 386 | 472 | ||
876 | 387 | 473 | ||
877 | 388 | @cached | 474 | @cached |
878 | 475 | def metadata(): | ||
879 | 476 | """Get the current charm metadata.yaml contents as a python object""" | ||
880 | 477 | with open(os.path.join(charm_dir(), 'metadata.yaml')) as md: | ||
881 | 478 | return yaml.safe_load(md) | ||
882 | 479 | |||
883 | 480 | |||
884 | 481 | @cached | ||
885 | 389 | def relation_types(): | 482 | def relation_types(): |
886 | 390 | """Get a list of relation types supported by this charm""" | 483 | """Get a list of relation types supported by this charm""" |
887 | 391 | charmdir = os.environ.get('CHARM_DIR', '') | ||
888 | 392 | mdf = open(os.path.join(charmdir, 'metadata.yaml')) | ||
889 | 393 | md = yaml.safe_load(mdf) | ||
890 | 394 | rel_types = [] | 484 | rel_types = [] |
891 | 485 | md = metadata() | ||
892 | 395 | for key in ('provides', 'requires', 'peers'): | 486 | for key in ('provides', 'requires', 'peers'): |
893 | 396 | section = md.get(key) | 487 | section = md.get(key) |
894 | 397 | if section: | 488 | if section: |
895 | 398 | rel_types.extend(section.keys()) | 489 | rel_types.extend(section.keys()) |
896 | 399 | mdf.close() | ||
897 | 400 | return rel_types | 490 | return rel_types |
898 | 401 | 491 | ||
899 | 402 | 492 | ||
900 | 403 | @cached | 493 | @cached |
901 | 494 | def peer_relation_id(): | ||
902 | 495 | '''Get the peers relation id if a peers relation has been joined, else None.''' | ||
903 | 496 | md = metadata() | ||
904 | 497 | section = md.get('peers') | ||
905 | 498 | if section: | ||
906 | 499 | for key in section: | ||
907 | 500 | relids = relation_ids(key) | ||
908 | 501 | if relids: | ||
909 | 502 | return relids[0] | ||
910 | 503 | return None | ||
911 | 504 | |||
912 | 505 | |||
913 | 506 | @cached | ||
914 | 507 | def relation_to_interface(relation_name): | ||
915 | 508 | """ | ||
916 | 509 | Given the name of a relation, return the interface that relation uses. | ||
917 | 510 | |||
918 | 511 | :returns: The interface name, or ``None``. | ||
919 | 512 | """ | ||
920 | 513 | return relation_to_role_and_interface(relation_name)[1] | ||
921 | 514 | |||
922 | 515 | |||
923 | 516 | @cached | ||
924 | 517 | def relation_to_role_and_interface(relation_name): | ||
925 | 518 | """ | ||
926 | 519 | Given the name of a relation, return the role and the name of the interface | ||
927 | 520 | that relation uses (where role is one of ``provides``, ``requires``, or ``peers``). | ||
928 | 521 | |||
929 | 522 | :returns: A tuple containing ``(role, interface)``, or ``(None, None)``. | ||
930 | 523 | """ | ||
931 | 524 | _metadata = metadata() | ||
932 | 525 | for role in ('provides', 'requires', 'peers'): | ||
933 | 526 | interface = _metadata.get(role, {}).get(relation_name, {}).get('interface') | ||
934 | 527 | if interface: | ||
935 | 528 | return role, interface | ||
936 | 529 | return None, None | ||
937 | 530 | |||
938 | 531 | |||
939 | 532 | @cached | ||
940 | 533 | def role_and_interface_to_relations(role, interface_name): | ||
941 | 534 | """ | ||
942 | 535 | Given a role and interface name, return a list of relation names for the | ||
943 | 536 | current charm that use that interface under that role (where role is one | ||
944 | 537 | of ``provides``, ``requires``, or ``peers``). | ||
945 | 538 | |||
946 | 539 | :returns: A list of relation names. | ||
947 | 540 | """ | ||
948 | 541 | _metadata = metadata() | ||
949 | 542 | results = [] | ||
950 | 543 | for relation_name, relation in _metadata.get(role, {}).items(): | ||
951 | 544 | if relation['interface'] == interface_name: | ||
952 | 545 | results.append(relation_name) | ||
953 | 546 | return results | ||
954 | 547 | |||
955 | 548 | |||
956 | 549 | @cached | ||
957 | 550 | def interface_to_relations(interface_name): | ||
958 | 551 | """ | ||
959 | 552 | Given an interface, return a list of relation names for the current | ||
960 | 553 | charm that use that interface. | ||
961 | 554 | |||
962 | 555 | :returns: A list of relation names. | ||
963 | 556 | """ | ||
964 | 557 | results = [] | ||
965 | 558 | for role in ('provides', 'requires', 'peers'): | ||
966 | 559 | results.extend(role_and_interface_to_relations(role, interface_name)) | ||
967 | 560 | return results | ||
968 | 561 | |||
969 | 562 | |||
970 | 563 | @cached | ||
971 | 564 | def charm_name(): | ||
972 | 565 | """Get the name of the current charm as is specified on metadata.yaml""" | ||
973 | 566 | return metadata().get('name') | ||
974 | 567 | |||
975 | 568 | |||
976 | 569 | @cached | ||
977 | 404 | def relations(): | 570 | def relations(): |
978 | 405 | """Get a nested dictionary of relation data for all related units""" | 571 | """Get a nested dictionary of relation data for all related units""" |
979 | 406 | rels = {} | 572 | rels = {} |
980 | @@ -450,21 +616,72 @@ | |||
981 | 450 | subprocess.check_call(_args) | 616 | subprocess.check_call(_args) |
982 | 451 | 617 | ||
983 | 452 | 618 | ||
984 | 619 | def open_ports(start, end, protocol="TCP"): | ||
985 | 620 | """Opens a range of service network ports""" | ||
986 | 621 | _args = ['open-port'] | ||
987 | 622 | _args.append('{}-{}/{}'.format(start, end, protocol)) | ||
988 | 623 | subprocess.check_call(_args) | ||
989 | 624 | |||
990 | 625 | |||
991 | 626 | def close_ports(start, end, protocol="TCP"): | ||
992 | 627 | """Close a range of service network ports""" | ||
993 | 628 | _args = ['close-port'] | ||
994 | 629 | _args.append('{}-{}/{}'.format(start, end, protocol)) | ||
995 | 630 | subprocess.check_call(_args) | ||
996 | 631 | |||
997 | 632 | |||
998 | 453 | @cached | 633 | @cached |
999 | 454 | def unit_get(attribute): | 634 | def unit_get(attribute): |
1000 | 455 | """Get the unit ID for the remote unit""" | 635 | """Get the unit ID for the remote unit""" |
1001 | 456 | _args = ['unit-get', '--format=json', attribute] | 636 | _args = ['unit-get', '--format=json', attribute] |
1002 | 457 | try: | 637 | try: |
1004 | 458 | return json.loads(subprocess.check_output(_args)) | 638 | return json.loads(subprocess.check_output(_args).decode('UTF-8')) |
1005 | 459 | except ValueError: | 639 | except ValueError: |
1006 | 460 | return None | 640 | return None |
1007 | 461 | 641 | ||
1008 | 462 | 642 | ||
1009 | 643 | def unit_public_ip(): | ||
1010 | 644 | """Get this unit's public IP address""" | ||
1011 | 645 | return unit_get('public-address') | ||
1012 | 646 | |||
1013 | 647 | |||
1014 | 463 | def unit_private_ip(): | 648 | def unit_private_ip(): |
1015 | 464 | """Get this unit's private IP address""" | 649 | """Get this unit's private IP address""" |
1016 | 465 | return unit_get('private-address') | 650 | return unit_get('private-address') |
1017 | 466 | 651 | ||
1018 | 467 | 652 | ||
1019 | 653 | @cached | ||
1020 | 654 | def storage_get(attribute=None, storage_id=None): | ||
1021 | 655 | """Get storage attributes""" | ||
1022 | 656 | _args = ['storage-get', '--format=json'] | ||
1023 | 657 | if storage_id: | ||
1024 | 658 | _args.extend(('-s', storage_id)) | ||
1025 | 659 | if attribute: | ||
1026 | 660 | _args.append(attribute) | ||
1027 | 661 | try: | ||
1028 | 662 | return json.loads(subprocess.check_output(_args).decode('UTF-8')) | ||
1029 | 663 | except ValueError: | ||
1030 | 664 | return None | ||
1031 | 665 | |||
1032 | 666 | |||
1033 | 667 | @cached | ||
1034 | 668 | def storage_list(storage_name=None): | ||
1035 | 669 | """List the storage IDs for the unit""" | ||
1036 | 670 | _args = ['storage-list', '--format=json'] | ||
1037 | 671 | if storage_name: | ||
1038 | 672 | _args.append(storage_name) | ||
1039 | 673 | try: | ||
1040 | 674 | return json.loads(subprocess.check_output(_args).decode('UTF-8')) | ||
1041 | 675 | except ValueError: | ||
1042 | 676 | return None | ||
1043 | 677 | except OSError as e: | ||
1044 | 678 | import errno | ||
1045 | 679 | if e.errno == errno.ENOENT: | ||
1046 | 680 | # storage-list does not exist | ||
1047 | 681 | return [] | ||
1048 | 682 | raise | ||
1049 | 683 | |||
1050 | 684 | |||
1051 | 468 | class UnregisteredHookError(Exception): | 685 | class UnregisteredHookError(Exception): |
1052 | 469 | """Raised when an undefined hook is called""" | 686 | """Raised when an undefined hook is called""" |
1053 | 470 | pass | 687 | pass |
1054 | @@ -492,10 +709,14 @@ | |||
1055 | 492 | hooks.execute(sys.argv) | 709 | hooks.execute(sys.argv) |
1056 | 493 | """ | 710 | """ |
1057 | 494 | 711 | ||
1059 | 495 | def __init__(self, config_save=True): | 712 | def __init__(self, config_save=None): |
1060 | 496 | super(Hooks, self).__init__() | 713 | super(Hooks, self).__init__() |
1061 | 497 | self._hooks = {} | 714 | self._hooks = {} |
1063 | 498 | self._config_save = config_save | 715 | |
1064 | 716 | # For unknown reasons, we allow the Hooks constructor to override | ||
1065 | 717 | # config().implicit_save. | ||
1066 | 718 | if config_save is not None: | ||
1067 | 719 | config().implicit_save = config_save | ||
1068 | 499 | 720 | ||
1069 | 500 | def register(self, name, function): | 721 | def register(self, name, function): |
1070 | 501 | """Register a hook""" | 722 | """Register a hook""" |
1071 | @@ -503,13 +724,16 @@ | |||
1072 | 503 | 724 | ||
1073 | 504 | def execute(self, args): | 725 | def execute(self, args): |
1074 | 505 | """Execute a registered hook based on args[0]""" | 726 | """Execute a registered hook based on args[0]""" |
1075 | 727 | _run_atstart() | ||
1076 | 506 | hook_name = os.path.basename(args[0]) | 728 | hook_name = os.path.basename(args[0]) |
1077 | 507 | if hook_name in self._hooks: | 729 | if hook_name in self._hooks: |
1083 | 508 | self._hooks[hook_name]() | 730 | try: |
1084 | 509 | if self._config_save: | 731 | self._hooks[hook_name]() |
1085 | 510 | cfg = config() | 732 | except SystemExit as x: |
1086 | 511 | if cfg.implicit_save: | 733 | if x.code is None or x.code == 0: |
1087 | 512 | cfg.save() | 734 | _run_atexit() |
1088 | 735 | raise | ||
1089 | 736 | _run_atexit() | ||
1090 | 513 | else: | 737 | else: |
1091 | 514 | raise UnregisteredHookError(hook_name) | 738 | raise UnregisteredHookError(hook_name) |
1092 | 515 | 739 | ||
1093 | @@ -530,3 +754,315 @@ | |||
1094 | 530 | def charm_dir(): | 754 | def charm_dir(): |
1095 | 531 | """Return the root directory of the current charm""" | 755 | """Return the root directory of the current charm""" |
1096 | 532 | return os.environ.get('CHARM_DIR') | 756 | return os.environ.get('CHARM_DIR') |
1097 | 757 | |||
1098 | 758 | |||
1099 | 759 | @cached | ||
1100 | 760 | def action_get(key=None): | ||
1101 | 761 | """Gets the value of an action parameter, or all key/value param pairs""" | ||
1102 | 762 | cmd = ['action-get'] | ||
1103 | 763 | if key is not None: | ||
1104 | 764 | cmd.append(key) | ||
1105 | 765 | cmd.append('--format=json') | ||
1106 | 766 | action_data = json.loads(subprocess.check_output(cmd).decode('UTF-8')) | ||
1107 | 767 | return action_data | ||
1108 | 768 | |||
1109 | 769 | |||
1110 | 770 | def action_set(values): | ||
1111 | 771 | """Sets the values to be returned after the action finishes""" | ||
1112 | 772 | cmd = ['action-set'] | ||
1113 | 773 | for k, v in list(values.items()): | ||
1114 | 774 | cmd.append('{}={}'.format(k, v)) | ||
1115 | 775 | subprocess.check_call(cmd) | ||
1116 | 776 | |||
1117 | 777 | |||
1118 | 778 | def action_fail(message): | ||
1119 | 779 | """Sets the action status to failed and sets the error message. | ||
1120 | 780 | |||
1121 | 781 | The results set by action_set are preserved.""" | ||
1122 | 782 | subprocess.check_call(['action-fail', message]) | ||
1123 | 783 | |||
1124 | 784 | |||
1125 | 785 | def action_name(): | ||
1126 | 786 | """Get the name of the currently executing action.""" | ||
1127 | 787 | return os.environ.get('JUJU_ACTION_NAME') | ||
1128 | 788 | |||
1129 | 789 | |||
1130 | 790 | def action_uuid(): | ||
1131 | 791 | """Get the UUID of the currently executing action.""" | ||
1132 | 792 | return os.environ.get('JUJU_ACTION_UUID') | ||
1133 | 793 | |||
1134 | 794 | |||
1135 | 795 | def action_tag(): | ||
1136 | 796 | """Get the tag for the currently executing action.""" | ||
1137 | 797 | return os.environ.get('JUJU_ACTION_TAG') | ||
1138 | 798 | |||
1139 | 799 | |||
1140 | 800 | def status_set(workload_state, message): | ||
1141 | 801 | """Set the workload state with a message | ||
1142 | 802 | |||
1143 | 803 | Use status-set to set the workload state with a message which is visible | ||
1144 | 804 | to the user via juju status. If the status-set command is not found then | ||
1145 | 805 | assume this is juju < 1.23 and juju-log the message unstead. | ||
1146 | 806 | |||
1147 | 807 | workload_state -- valid juju workload state. | ||
1148 | 808 | message -- status update message | ||
1149 | 809 | """ | ||
1150 | 810 | valid_states = ['maintenance', 'blocked', 'waiting', 'active'] | ||
1151 | 811 | if workload_state not in valid_states: | ||
1152 | 812 | raise ValueError( | ||
1153 | 813 | '{!r} is not a valid workload state'.format(workload_state) | ||
1154 | 814 | ) | ||
1155 | 815 | cmd = ['status-set', workload_state, message] | ||
1156 | 816 | try: | ||
1157 | 817 | ret = subprocess.call(cmd) | ||
1158 | 818 | if ret == 0: | ||
1159 | 819 | return | ||
1160 | 820 | except OSError as e: | ||
1161 | 821 | if e.errno != errno.ENOENT: | ||
1162 | 822 | raise | ||
1163 | 823 | log_message = 'status-set failed: {} {}'.format(workload_state, | ||
1164 | 824 | message) | ||
1165 | 825 | log(log_message, level='INFO') | ||
1166 | 826 | |||
1167 | 827 | |||
1168 | 828 | def status_get(): | ||
1169 | 829 | """Retrieve the previously set juju workload state and message | ||
1170 | 830 | |||
1171 | 831 | If the status-get command is not found then assume this is juju < 1.23 and | ||
1172 | 832 | return 'unknown', "" | ||
1173 | 833 | |||
1174 | 834 | """ | ||
1175 | 835 | cmd = ['status-get', "--format=json", "--include-data"] | ||
1176 | 836 | try: | ||
1177 | 837 | raw_status = subprocess.check_output(cmd) | ||
1178 | 838 | except OSError as e: | ||
1179 | 839 | if e.errno == errno.ENOENT: | ||
1180 | 840 | return ('unknown', "") | ||
1181 | 841 | else: | ||
1182 | 842 | raise | ||
1183 | 843 | else: | ||
1184 | 844 | status = json.loads(raw_status.decode("UTF-8")) | ||
1185 | 845 | return (status["status"], status["message"]) | ||
1186 | 846 | |||
1187 | 847 | |||
1188 | 848 | def translate_exc(from_exc, to_exc): | ||
1189 | 849 | def inner_translate_exc1(f): | ||
1190 | 850 | @wraps(f) | ||
1191 | 851 | def inner_translate_exc2(*args, **kwargs): | ||
1192 | 852 | try: | ||
1193 | 853 | return f(*args, **kwargs) | ||
1194 | 854 | except from_exc: | ||
1195 | 855 | raise to_exc | ||
1196 | 856 | |||
1197 | 857 | return inner_translate_exc2 | ||
1198 | 858 | |||
1199 | 859 | return inner_translate_exc1 | ||
1200 | 860 | |||
1201 | 861 | |||
1202 | 862 | def application_version_set(version): | ||
1203 | 863 | """Charm authors may trigger this command from any hook to output what | ||
1204 | 864 | version of the application is running. This could be a package version, | ||
1205 | 865 | for instance postgres version 9.5. It could also be a build number or | ||
1206 | 866 | version control revision identifier, for instance git sha 6fb7ba68. """ | ||
1207 | 867 | |||
1208 | 868 | cmd = ['application-version-set'] | ||
1209 | 869 | cmd.append(version) | ||
1210 | 870 | try: | ||
1211 | 871 | subprocess.check_call(cmd) | ||
1212 | 872 | except OSError: | ||
1213 | 873 | log("Application Version: {}".format(version)) | ||
1214 | 874 | |||
1215 | 875 | |||
1216 | 876 | @translate_exc(from_exc=OSError, to_exc=NotImplementedError) | ||
1217 | 877 | def is_leader(): | ||
1218 | 878 | """Does the current unit hold the juju leadership | ||
1219 | 879 | |||
1220 | 880 | Uses juju to determine whether the current unit is the leader of its peers | ||
1221 | 881 | """ | ||
1222 | 882 | cmd = ['is-leader', '--format=json'] | ||
1223 | 883 | return json.loads(subprocess.check_output(cmd).decode('UTF-8')) | ||
1224 | 884 | |||
1225 | 885 | |||
1226 | 886 | @translate_exc(from_exc=OSError, to_exc=NotImplementedError) | ||
1227 | 887 | def leader_get(attribute=None): | ||
1228 | 888 | """Juju leader get value(s)""" | ||
1229 | 889 | cmd = ['leader-get', '--format=json'] + [attribute or '-'] | ||
1230 | 890 | return json.loads(subprocess.check_output(cmd).decode('UTF-8')) | ||
1231 | 891 | |||
1232 | 892 | |||
1233 | 893 | @translate_exc(from_exc=OSError, to_exc=NotImplementedError) | ||
1234 | 894 | def leader_set(settings=None, **kwargs): | ||
1235 | 895 | """Juju leader set value(s)""" | ||
1236 | 896 | # Don't log secrets. | ||
1237 | 897 | # log("Juju leader-set '%s'" % (settings), level=DEBUG) | ||
1238 | 898 | cmd = ['leader-set'] | ||
1239 | 899 | settings = settings or {} | ||
1240 | 900 | settings.update(kwargs) | ||
1241 | 901 | for k, v in settings.items(): | ||
1242 | 902 | if v is None: | ||
1243 | 903 | cmd.append('{}='.format(k)) | ||
1244 | 904 | else: | ||
1245 | 905 | cmd.append('{}={}'.format(k, v)) | ||
1246 | 906 | subprocess.check_call(cmd) | ||
1247 | 907 | |||
1248 | 908 | |||
1249 | 909 | @translate_exc(from_exc=OSError, to_exc=NotImplementedError) | ||
1250 | 910 | def payload_register(ptype, klass, pid): | ||
1251 | 911 | """ is used while a hook is running to let Juju know that a | ||
1252 | 912 | payload has been started.""" | ||
1253 | 913 | cmd = ['payload-register'] | ||
1254 | 914 | for x in [ptype, klass, pid]: | ||
1255 | 915 | cmd.append(x) | ||
1256 | 916 | subprocess.check_call(cmd) | ||
1257 | 917 | |||
1258 | 918 | |||
1259 | 919 | @translate_exc(from_exc=OSError, to_exc=NotImplementedError) | ||
1260 | 920 | def payload_unregister(klass, pid): | ||
1261 | 921 | """ is used while a hook is running to let Juju know | ||
1262 | 922 | that a payload has been manually stopped. The <class> and <id> provided | ||
1263 | 923 | must match a payload that has been previously registered with juju using | ||
1264 | 924 | payload-register.""" | ||
1265 | 925 | cmd = ['payload-unregister'] | ||
1266 | 926 | for x in [klass, pid]: | ||
1267 | 927 | cmd.append(x) | ||
1268 | 928 | subprocess.check_call(cmd) | ||
1269 | 929 | |||
1270 | 930 | |||
1271 | 931 | @translate_exc(from_exc=OSError, to_exc=NotImplementedError) | ||
1272 | 932 | def payload_status_set(klass, pid, status): | ||
1273 | 933 | """is used to update the current status of a registered payload. | ||
1274 | 934 | The <class> and <id> provided must match a payload that has been previously | ||
1275 | 935 | registered with juju using payload-register. The <status> must be one of the | ||
1276 | 936 | follow: starting, started, stopping, stopped""" | ||
1277 | 937 | cmd = ['payload-status-set'] | ||
1278 | 938 | for x in [klass, pid, status]: | ||
1279 | 939 | cmd.append(x) | ||
1280 | 940 | subprocess.check_call(cmd) | ||
1281 | 941 | |||
1282 | 942 | |||
1283 | 943 | @translate_exc(from_exc=OSError, to_exc=NotImplementedError) | ||
1284 | 944 | def resource_get(name): | ||
1285 | 945 | """used to fetch the resource path of the given name. | ||
1286 | 946 | |||
1287 | 947 | <name> must match a name of defined resource in metadata.yaml | ||
1288 | 948 | |||
1289 | 949 | returns either a path or False if resource not available | ||
1290 | 950 | """ | ||
1291 | 951 | if not name: | ||
1292 | 952 | return False | ||
1293 | 953 | |||
1294 | 954 | cmd = ['resource-get', name] | ||
1295 | 955 | try: | ||
1296 | 956 | return subprocess.check_output(cmd).decode('UTF-8') | ||
1297 | 957 | except subprocess.CalledProcessError: | ||
1298 | 958 | return False | ||
1299 | 959 | |||
1300 | 960 | |||
1301 | 961 | @cached | ||
1302 | 962 | def juju_version(): | ||
1303 | 963 | """Full version string (eg. '1.23.3.1-trusty-amd64')""" | ||
1304 | 964 | # Per https://bugs.launchpad.net/juju-core/+bug/1455368/comments/1 | ||
1305 | 965 | jujud = glob.glob('/var/lib/juju/tools/machine-*/jujud')[0] | ||
1306 | 966 | return subprocess.check_output([jujud, 'version'], | ||
1307 | 967 | universal_newlines=True).strip() | ||
1308 | 968 | |||
1309 | 969 | |||
1310 | 970 | @cached | ||
1311 | 971 | def has_juju_version(minimum_version): | ||
1312 | 972 | """Return True if the Juju version is at least the provided version""" | ||
1313 | 973 | return LooseVersion(juju_version()) >= LooseVersion(minimum_version) | ||
1314 | 974 | |||
1315 | 975 | |||
1316 | 976 | _atexit = [] | ||
1317 | 977 | _atstart = [] | ||
1318 | 978 | |||
1319 | 979 | |||
1320 | 980 | def atstart(callback, *args, **kwargs): | ||
1321 | 981 | '''Schedule a callback to run before the main hook. | ||
1322 | 982 | |||
1323 | 983 | Callbacks are run in the order they were added. | ||
1324 | 984 | |||
1325 | 985 | This is useful for modules and classes to perform initialization | ||
1326 | 986 | and inject behavior. In particular: | ||
1327 | 987 | |||
1328 | 988 | - Run common code before all of your hooks, such as logging | ||
1329 | 989 | the hook name or interesting relation data. | ||
1330 | 990 | - Defer object or module initialization that requires a hook | ||
1331 | 991 | context until we know there actually is a hook context, | ||
1332 | 992 | making testing easier. | ||
1333 | 993 | - Rather than requiring charm authors to include boilerplate to | ||
1334 | 994 | invoke your helper's behavior, have it run automatically if | ||
1335 | 995 | your object is instantiated or module imported. | ||
1336 | 996 | |||
1337 | 997 | This is not at all useful after your hook framework as been launched. | ||
1338 | 998 | ''' | ||
1339 | 999 | global _atstart | ||
1340 | 1000 | _atstart.append((callback, args, kwargs)) | ||
1341 | 1001 | |||
1342 | 1002 | |||
1343 | 1003 | def atexit(callback, *args, **kwargs): | ||
1344 | 1004 | '''Schedule a callback to run on successful hook completion. | ||
1345 | 1005 | |||
1346 | 1006 | Callbacks are run in the reverse order that they were added.''' | ||
1347 | 1007 | _atexit.append((callback, args, kwargs)) | ||
1348 | 1008 | |||
1349 | 1009 | |||
1350 | 1010 | def _run_atstart(): | ||
1351 | 1011 | '''Hook frameworks must invoke this before running the main hook body.''' | ||
1352 | 1012 | global _atstart | ||
1353 | 1013 | for callback, args, kwargs in _atstart: | ||
1354 | 1014 | callback(*args, **kwargs) | ||
1355 | 1015 | del _atstart[:] | ||
1356 | 1016 | |||
1357 | 1017 | |||
1358 | 1018 | def _run_atexit(): | ||
1359 | 1019 | '''Hook frameworks must invoke this after the main hook body has | ||
1360 | 1020 | successfully completed. Do not invoke it if the hook fails.''' | ||
1361 | 1021 | global _atexit | ||
1362 | 1022 | for callback, args, kwargs in reversed(_atexit): | ||
1363 | 1023 | callback(*args, **kwargs) | ||
1364 | 1024 | del _atexit[:] | ||
1365 | 1025 | |||
1366 | 1026 | |||
1367 | 1027 | @translate_exc(from_exc=OSError, to_exc=NotImplementedError) | ||
1368 | 1028 | def network_get_primary_address(binding): | ||
1369 | 1029 | ''' | ||
1370 | 1030 | Retrieve the primary network address for a named binding | ||
1371 | 1031 | |||
1372 | 1032 | :param binding: string. The name of a relation of extra-binding | ||
1373 | 1033 | :return: string. The primary IP address for the named binding | ||
1374 | 1034 | :raise: NotImplementedError if run on Juju < 2.0 | ||
1375 | 1035 | ''' | ||
1376 | 1036 | cmd = ['network-get', '--primary-address', binding] | ||
1377 | 1037 | return subprocess.check_output(cmd).decode('UTF-8').strip() | ||
1378 | 1038 | |||
1379 | 1039 | |||
1380 | 1040 | def add_metric(*args, **kwargs): | ||
1381 | 1041 | """Add metric values. Values may be expressed with keyword arguments. For | ||
1382 | 1042 | metric names containing dashes, these may be expressed as one or more | ||
1383 | 1043 | 'key=value' positional arguments. May only be called from the collect-metrics | ||
1384 | 1044 | hook.""" | ||
1385 | 1045 | _args = ['add-metric'] | ||
1386 | 1046 | _kvpairs = [] | ||
1387 | 1047 | _kvpairs.extend(args) | ||
1388 | 1048 | _kvpairs.extend(['{}={}'.format(k, v) for k, v in kwargs.items()]) | ||
1389 | 1049 | _args.extend(sorted(_kvpairs)) | ||
1390 | 1050 | try: | ||
1391 | 1051 | subprocess.check_call(_args) | ||
1392 | 1052 | return | ||
1393 | 1053 | except EnvironmentError as e: | ||
1394 | 1054 | if e.errno != errno.ENOENT: | ||
1395 | 1055 | raise | ||
1396 | 1056 | log_message = 'add-metric failed: {}'.format(' '.join(_kvpairs)) | ||
1397 | 1057 | log(log_message, level='INFO') | ||
1398 | 1058 | |||
1399 | 1059 | |||
1400 | 1060 | def meter_status(): | ||
1401 | 1061 | """Get the meter status, if running in the meter-status-changed hook.""" | ||
1402 | 1062 | return os.environ.get('JUJU_METER_STATUS') | ||
1403 | 1063 | |||
1404 | 1064 | |||
1405 | 1065 | def meter_info(): | ||
1406 | 1066 | """Get the meter status information, if running in the meter-status-changed | ||
1407 | 1067 | hook.""" | ||
1408 | 1068 | return os.environ.get('JUJU_METER_INFO') | ||
1409 | 533 | 1069 | ||
1410 | === modified file 'hooks/charmhelpers/core/host.py' | |||
1411 | --- hooks/charmhelpers/core/host.py 2014-10-24 16:35:00 +0000 | |||
1412 | +++ hooks/charmhelpers/core/host.py 2017-05-11 14:56:57 +0000 | |||
1413 | @@ -1,3 +1,17 @@ | |||
1414 | 1 | # Copyright 2014-2015 Canonical Limited. | ||
1415 | 2 | # | ||
1416 | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); | ||
1417 | 4 | # you may not use this file except in compliance with the License. | ||
1418 | 5 | # You may obtain a copy of the License at | ||
1419 | 6 | # | ||
1420 | 7 | # http://www.apache.org/licenses/LICENSE-2.0 | ||
1421 | 8 | # | ||
1422 | 9 | # Unless required by applicable law or agreed to in writing, software | ||
1423 | 10 | # distributed under the License is distributed on an "AS IS" BASIS, | ||
1424 | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
1425 | 12 | # See the License for the specific language governing permissions and | ||
1426 | 13 | # limitations under the License. | ||
1427 | 14 | |||
1428 | 1 | """Tools for working with the host system""" | 15 | """Tools for working with the host system""" |
1429 | 2 | # Copyright 2012 Canonical Ltd. | 16 | # Copyright 2012 Canonical Ltd. |
1430 | 3 | # | 17 | # |
1431 | @@ -8,80 +22,328 @@ | |||
1432 | 8 | import os | 22 | import os |
1433 | 9 | import re | 23 | import re |
1434 | 10 | import pwd | 24 | import pwd |
1435 | 25 | import glob | ||
1436 | 11 | import grp | 26 | import grp |
1437 | 12 | import random | 27 | import random |
1438 | 13 | import string | 28 | import string |
1439 | 14 | import subprocess | 29 | import subprocess |
1440 | 15 | import hashlib | 30 | import hashlib |
1441 | 31 | import functools | ||
1442 | 32 | import itertools | ||
1443 | 33 | import six | ||
1444 | 34 | |||
1445 | 16 | from contextlib import contextmanager | 35 | from contextlib import contextmanager |
1446 | 17 | |||
1447 | 18 | from collections import OrderedDict | 36 | from collections import OrderedDict |
1465 | 19 | 37 | from .hookenv import log | |
1466 | 20 | from hookenv import log | 38 | from .fstab import Fstab |
1467 | 21 | from fstab import Fstab | 39 | from charmhelpers.osplatform import get_platform |
1468 | 22 | 40 | ||
1469 | 23 | 41 | __platform__ = get_platform() | |
1470 | 24 | def service_start(service_name): | 42 | if __platform__ == "ubuntu": |
1471 | 25 | """Start a system service""" | 43 | from charmhelpers.core.host_factory.ubuntu import ( |
1472 | 26 | return service('start', service_name) | 44 | service_available, |
1473 | 27 | 45 | add_new_group, | |
1474 | 28 | 46 | lsb_release, | |
1475 | 29 | def service_stop(service_name): | 47 | cmp_pkgrevno, |
1476 | 30 | """Stop a system service""" | 48 | CompareHostReleases, |
1477 | 31 | return service('stop', service_name) | 49 | ) # flake8: noqa -- ignore F401 for this import |
1478 | 32 | 50 | elif __platform__ == "centos": | |
1479 | 33 | 51 | from charmhelpers.core.host_factory.centos import ( | |
1480 | 34 | def service_restart(service_name): | 52 | service_available, |
1481 | 35 | """Restart a system service""" | 53 | add_new_group, |
1482 | 54 | lsb_release, | ||
1483 | 55 | cmp_pkgrevno, | ||
1484 | 56 | CompareHostReleases, | ||
1485 | 57 | ) # flake8: noqa -- ignore F401 for this import | ||
1486 | 58 | |||
1487 | 59 | UPDATEDB_PATH = '/etc/updatedb.conf' | ||
1488 | 60 | |||
1489 | 61 | def service_start(service_name, **kwargs): | ||
1490 | 62 | """Start a system service. | ||
1491 | 63 | |||
1492 | 64 | The specified service name is managed via the system level init system. | ||
1493 | 65 | Some init systems (e.g. upstart) require that additional arguments be | ||
1494 | 66 | provided in order to directly control service instances whereas other init | ||
1495 | 67 | systems allow for addressing instances of a service directly by name (e.g. | ||
1496 | 68 | systemd). | ||
1497 | 69 | |||
1498 | 70 | The kwargs allow for the additional parameters to be passed to underlying | ||
1499 | 71 | init systems for those systems which require/allow for them. For example, | ||
1500 | 72 | the ceph-osd upstart script requires the id parameter to be passed along | ||
1501 | 73 | in order to identify which running daemon should be reloaded. The follow- | ||
1502 | 74 | ing example stops the ceph-osd service for instance id=4: | ||
1503 | 75 | |||
1504 | 76 | service_stop('ceph-osd', id=4) | ||
1505 | 77 | |||
1506 | 78 | :param service_name: the name of the service to stop | ||
1507 | 79 | :param **kwargs: additional parameters to pass to the init system when | ||
1508 | 80 | managing services. These will be passed as key=value | ||
1509 | 81 | parameters to the init system's commandline. kwargs | ||
1510 | 82 | are ignored for systemd enabled systems. | ||
1511 | 83 | """ | ||
1512 | 84 | return service('start', service_name, **kwargs) | ||
1513 | 85 | |||
1514 | 86 | |||
1515 | 87 | def service_stop(service_name, **kwargs): | ||
1516 | 88 | """Stop a system service. | ||
1517 | 89 | |||
1518 | 90 | The specified service name is managed via the system level init system. | ||
1519 | 91 | Some init systems (e.g. upstart) require that additional arguments be | ||
1520 | 92 | provided in order to directly control service instances whereas other init | ||
1521 | 93 | systems allow for addressing instances of a service directly by name (e.g. | ||
1522 | 94 | systemd). | ||
1523 | 95 | |||
1524 | 96 | The kwargs allow for the additional parameters to be passed to underlying | ||
1525 | 97 | init systems for those systems which require/allow for them. For example, | ||
1526 | 98 | the ceph-osd upstart script requires the id parameter to be passed along | ||
1527 | 99 | in order to identify which running daemon should be reloaded. The follow- | ||
1528 | 100 | ing example stops the ceph-osd service for instance id=4: | ||
1529 | 101 | |||
1530 | 102 | service_stop('ceph-osd', id=4) | ||
1531 | 103 | |||
1532 | 104 | :param service_name: the name of the service to stop | ||
1533 | 105 | :param **kwargs: additional parameters to pass to the init system when | ||
1534 | 106 | managing services. These will be passed as key=value | ||
1535 | 107 | parameters to the init system's commandline. kwargs | ||
1536 | 108 | are ignored for systemd enabled systems. | ||
1537 | 109 | """ | ||
1538 | 110 | return service('stop', service_name, **kwargs) | ||
1539 | 111 | |||
1540 | 112 | |||
1541 | 113 | def service_restart(service_name, **kwargs): | ||
1542 | 114 | """Restart a system service. | ||
1543 | 115 | |||
1544 | 116 | The specified service name is managed via the system level init system. | ||
1545 | 117 | Some init systems (e.g. upstart) require that additional arguments be | ||
1546 | 118 | provided in order to directly control service instances whereas other init | ||
1547 | 119 | systems allow for addressing instances of a service directly by name (e.g. | ||
1548 | 120 | systemd). | ||
1549 | 121 | |||
1550 | 122 | The kwargs allow for the additional parameters to be passed to underlying | ||
1551 | 123 | init systems for those systems which require/allow for them. For example, | ||
1552 | 124 | the ceph-osd upstart script requires the id parameter to be passed along | ||
1553 | 125 | in order to identify which running daemon should be restarted. The follow- | ||
1554 | 126 | ing example restarts the ceph-osd service for instance id=4: | ||
1555 | 127 | |||
1556 | 128 | service_restart('ceph-osd', id=4) | ||
1557 | 129 | |||
1558 | 130 | :param service_name: the name of the service to restart | ||
1559 | 131 | :param **kwargs: additional parameters to pass to the init system when | ||
1560 | 132 | managing services. These will be passed as key=value | ||
1561 | 133 | parameters to the init system's commandline. kwargs | ||
1562 | 134 | are ignored for init systems not allowing additional | ||
1563 | 135 | parameters via the commandline (systemd). | ||
1564 | 136 | """ | ||
1565 | 36 | return service('restart', service_name) | 137 | return service('restart', service_name) |
1566 | 37 | 138 | ||
1567 | 38 | 139 | ||
1569 | 39 | def service_reload(service_name, restart_on_failure=False): | 140 | def service_reload(service_name, restart_on_failure=False, **kwargs): |
1570 | 40 | """Reload a system service, optionally falling back to restart if | 141 | """Reload a system service, optionally falling back to restart if |
1573 | 41 | reload fails""" | 142 | reload fails. |
1574 | 42 | service_result = service('reload', service_name) | 143 | |
1575 | 144 | The specified service name is managed via the system level init system. | ||
1576 | 145 | Some init systems (e.g. upstart) require that additional arguments be | ||
1577 | 146 | provided in order to directly control service instances whereas other init | ||
1578 | 147 | systems allow for addressing instances of a service directly by name (e.g. | ||
1579 | 148 | systemd). | ||
1580 | 149 | |||
1581 | 150 | The kwargs allow for the additional parameters to be passed to underlying | ||
1582 | 151 | init systems for those systems which require/allow for them. For example, | ||
1583 | 152 | the ceph-osd upstart script requires the id parameter to be passed along | ||
1584 | 153 | in order to identify which running daemon should be reloaded. The follow- | ||
1585 | 154 | ing example restarts the ceph-osd service for instance id=4: | ||
1586 | 155 | |||
1587 | 156 | service_reload('ceph-osd', id=4) | ||
1588 | 157 | |||
1589 | 158 | :param service_name: the name of the service to reload | ||
1590 | 159 | :param restart_on_failure: boolean indicating whether to fallback to a | ||
1591 | 160 | restart if the reload fails. | ||
1592 | 161 | :param **kwargs: additional parameters to pass to the init system when | ||
1593 | 162 | managing services. These will be passed as key=value | ||
1594 | 163 | parameters to the init system's commandline. kwargs | ||
1595 | 164 | are ignored for init systems not allowing additional | ||
1596 | 165 | parameters via the commandline (systemd). | ||
1597 | 166 | """ | ||
1598 | 167 | service_result = service('reload', service_name, **kwargs) | ||
1599 | 43 | if not service_result and restart_on_failure: | 168 | if not service_result and restart_on_failure: |
1601 | 44 | service_result = service('restart', service_name) | 169 | service_result = service('restart', service_name, **kwargs) |
1602 | 45 | return service_result | 170 | return service_result |
1603 | 46 | 171 | ||
1604 | 47 | 172 | ||
1608 | 48 | def service(action, service_name): | 173 | def service_pause(service_name, init_dir="/etc/init", initd_dir="/etc/init.d", |
1609 | 49 | """Control a system service""" | 174 | **kwargs): |
1610 | 50 | cmd = ['service', service_name, action] | 175 | """Pause a system service. |
1611 | 176 | |||
1612 | 177 | Stop it, and prevent it from starting again at boot. | ||
1613 | 178 | |||
1614 | 179 | :param service_name: the name of the service to pause | ||
1615 | 180 | :param init_dir: path to the upstart init directory | ||
1616 | 181 | :param initd_dir: path to the sysv init directory | ||
1617 | 182 | :param **kwargs: additional parameters to pass to the init system when | ||
1618 | 183 | managing services. These will be passed as key=value | ||
1619 | 184 | parameters to the init system's commandline. kwargs | ||
1620 | 185 | are ignored for init systems which do not support | ||
1621 | 186 | key=value arguments via the commandline. | ||
1622 | 187 | """ | ||
1623 | 188 | stopped = True | ||
1624 | 189 | if service_running(service_name, **kwargs): | ||
1625 | 190 | stopped = service_stop(service_name, **kwargs) | ||
1626 | 191 | upstart_file = os.path.join(init_dir, "{}.conf".format(service_name)) | ||
1627 | 192 | sysv_file = os.path.join(initd_dir, service_name) | ||
1628 | 193 | if init_is_systemd(): | ||
1629 | 194 | service('disable', service_name) | ||
1630 | 195 | elif os.path.exists(upstart_file): | ||
1631 | 196 | override_path = os.path.join( | ||
1632 | 197 | init_dir, '{}.override'.format(service_name)) | ||
1633 | 198 | with open(override_path, 'w') as fh: | ||
1634 | 199 | fh.write("manual\n") | ||
1635 | 200 | elif os.path.exists(sysv_file): | ||
1636 | 201 | subprocess.check_call(["update-rc.d", service_name, "disable"]) | ||
1637 | 202 | else: | ||
1638 | 203 | raise ValueError( | ||
1639 | 204 | "Unable to detect {0} as SystemD, Upstart {1} or" | ||
1640 | 205 | " SysV {2}".format( | ||
1641 | 206 | service_name, upstart_file, sysv_file)) | ||
1642 | 207 | return stopped | ||
1643 | 208 | |||
1644 | 209 | |||
1645 | 210 | def service_resume(service_name, init_dir="/etc/init", | ||
1646 | 211 | initd_dir="/etc/init.d", **kwargs): | ||
1647 | 212 | """Resume a system service. | ||
1648 | 213 | |||
1649 | 214 | Reenable starting again at boot. Start the service. | ||
1650 | 215 | |||
1651 | 216 | :param service_name: the name of the service to resume | ||
1652 | 217 | :param init_dir: the path to the init dir | ||
1653 | 218 | :param initd dir: the path to the initd dir | ||
1654 | 219 | :param **kwargs: additional parameters to pass to the init system when | ||
1655 | 220 | managing services. These will be passed as key=value | ||
1656 | 221 | parameters to the init system's commandline. kwargs | ||
1657 | 222 | are ignored for systemd enabled systems. | ||
1658 | 223 | """ | ||
1659 | 224 | upstart_file = os.path.join(init_dir, "{}.conf".format(service_name)) | ||
1660 | 225 | sysv_file = os.path.join(initd_dir, service_name) | ||
1661 | 226 | if init_is_systemd(): | ||
1662 | 227 | service('enable', service_name) | ||
1663 | 228 | elif os.path.exists(upstart_file): | ||
1664 | 229 | override_path = os.path.join( | ||
1665 | 230 | init_dir, '{}.override'.format(service_name)) | ||
1666 | 231 | if os.path.exists(override_path): | ||
1667 | 232 | os.unlink(override_path) | ||
1668 | 233 | elif os.path.exists(sysv_file): | ||
1669 | 234 | subprocess.check_call(["update-rc.d", service_name, "enable"]) | ||
1670 | 235 | else: | ||
1671 | 236 | raise ValueError( | ||
1672 | 237 | "Unable to detect {0} as SystemD, Upstart {1} or" | ||
1673 | 238 | " SysV {2}".format( | ||
1674 | 239 | service_name, upstart_file, sysv_file)) | ||
1675 | 240 | started = service_running(service_name, **kwargs) | ||
1676 | 241 | |||
1677 | 242 | if not started: | ||
1678 | 243 | started = service_start(service_name, **kwargs) | ||
1679 | 244 | return started | ||
1680 | 245 | |||
1681 | 246 | |||
1682 | 247 | def service(action, service_name, **kwargs): | ||
1683 | 248 | """Control a system service. | ||
1684 | 249 | |||
1685 | 250 | :param action: the action to take on the service | ||
1686 | 251 | :param service_name: the name of the service to perform th action on | ||
1687 | 252 | :param **kwargs: additional params to be passed to the service command in | ||
1688 | 253 | the form of key=value. | ||
1689 | 254 | """ | ||
1690 | 255 | if init_is_systemd(): | ||
1691 | 256 | cmd = ['systemctl', action, service_name] | ||
1692 | 257 | else: | ||
1693 | 258 | cmd = ['service', service_name, action] | ||
1694 | 259 | for key, value in six.iteritems(kwargs): | ||
1695 | 260 | parameter = '%s=%s' % (key, value) | ||
1696 | 261 | cmd.append(parameter) | ||
1697 | 51 | return subprocess.call(cmd) == 0 | 262 | return subprocess.call(cmd) == 0 |
1698 | 52 | 263 | ||
1699 | 53 | 264 | ||
1725 | 54 | def service_running(service): | 265 | _UPSTART_CONF = "/etc/init/{}.conf" |
1726 | 55 | """Determine whether a system service is running""" | 266 | _INIT_D_CONF = "/etc/init.d/{}" |
1727 | 56 | try: | 267 | |
1728 | 57 | output = subprocess.check_output(['service', service, 'status'], stderr=subprocess.STDOUT) | 268 | |
1729 | 58 | except subprocess.CalledProcessError: | 269 | def service_running(service_name, **kwargs): |
1730 | 59 | return False | 270 | """Determine whether a system service is running. |
1731 | 60 | else: | 271 | |
1732 | 61 | if ("start/running" in output or "is running" in output): | 272 | :param service_name: the name of the service |
1733 | 62 | return True | 273 | :param **kwargs: additional args to pass to the service command. This is |
1734 | 63 | else: | 274 | used to pass additional key=value arguments to the |
1735 | 64 | return False | 275 | service command line for managing specific instance |
1736 | 65 | 276 | units (e.g. service ceph-osd status id=2). The kwargs | |
1737 | 66 | 277 | are ignored in systemd services. | |
1738 | 67 | def service_available(service_name): | 278 | """ |
1739 | 68 | """Determine whether a system service is available""" | 279 | if init_is_systemd(): |
1740 | 69 | try: | 280 | return service('is-active', service_name) |
1741 | 70 | subprocess.check_output(['service', service_name, 'status'], stderr=subprocess.STDOUT) | 281 | else: |
1742 | 71 | except subprocess.CalledProcessError as e: | 282 | if os.path.exists(_UPSTART_CONF.format(service_name)): |
1743 | 72 | return 'unrecognized service' not in e.output | 283 | try: |
1744 | 73 | else: | 284 | cmd = ['status', service_name] |
1745 | 74 | return True | 285 | for key, value in six.iteritems(kwargs): |
1746 | 75 | 286 | parameter = '%s=%s' % (key, value) | |
1747 | 76 | 287 | cmd.append(parameter) | |
1748 | 77 | def adduser(username, password=None, shell='/bin/bash', system_user=False): | 288 | output = subprocess.check_output(cmd, |
1749 | 78 | """Add a user to the system""" | 289 | stderr=subprocess.STDOUT).decode('UTF-8') |
1750 | 290 | except subprocess.CalledProcessError: | ||
1751 | 291 | return False | ||
1752 | 292 | else: | ||
1753 | 293 | # This works for upstart scripts where the 'service' command | ||
1754 | 294 | # returns a consistent string to represent running | ||
1755 | 295 | # 'start/running' | ||
1756 | 296 | if ("start/running" in output or | ||
1757 | 297 | "is running" in output or | ||
1758 | 298 | "up and running" in output): | ||
1759 | 299 | return True | ||
1760 | 300 | elif os.path.exists(_INIT_D_CONF.format(service_name)): | ||
1761 | 301 | # Check System V scripts init script return codes | ||
1762 | 302 | return service('status', service_name) | ||
1763 | 303 | return False | ||
1764 | 304 | |||
1765 | 305 | |||
1766 | 306 | SYSTEMD_SYSTEM = '/run/systemd/system' | ||
1767 | 307 | |||
1768 | 308 | |||
1769 | 309 | def init_is_systemd(): | ||
1770 | 310 | """Return True if the host system uses systemd, False otherwise.""" | ||
1771 | 311 | if lsb_release()['DISTRIB_CODENAME'] == 'trusty': | ||
1772 | 312 | return False | ||
1773 | 313 | return os.path.isdir(SYSTEMD_SYSTEM) | ||
1774 | 314 | |||
1775 | 315 | |||
1776 | 316 | def adduser(username, password=None, shell='/bin/bash', | ||
1777 | 317 | system_user=False, primary_group=None, | ||
1778 | 318 | secondary_groups=None, uid=None, home_dir=None): | ||
1779 | 319 | """Add a user to the system. | ||
1780 | 320 | |||
1781 | 321 | Will log but otherwise succeed if the user already exists. | ||
1782 | 322 | |||
1783 | 323 | :param str username: Username to create | ||
1784 | 324 | :param str password: Password for user; if ``None``, create a system user | ||
1785 | 325 | :param str shell: The default shell for the user | ||
1786 | 326 | :param bool system_user: Whether to create a login or system user | ||
1787 | 327 | :param str primary_group: Primary group for user; defaults to username | ||
1788 | 328 | :param list secondary_groups: Optional list of additional groups | ||
1789 | 329 | :param int uid: UID for user being created | ||
1790 | 330 | :param str home_dir: Home directory for user | ||
1791 | 331 | |||
1792 | 332 | :returns: The password database entry struct, as returned by `pwd.getpwnam` | ||
1793 | 333 | """ | ||
1794 | 79 | try: | 334 | try: |
1795 | 80 | user_info = pwd.getpwnam(username) | 335 | user_info = pwd.getpwnam(username) |
1796 | 81 | log('user {0} already exists!'.format(username)) | 336 | log('user {0} already exists!'.format(username)) |
1797 | 337 | if uid: | ||
1798 | 338 | user_info = pwd.getpwuid(int(uid)) | ||
1799 | 339 | log('user with uid {0} already exists!'.format(uid)) | ||
1800 | 82 | except KeyError: | 340 | except KeyError: |
1801 | 83 | log('creating user {0}'.format(username)) | 341 | log('creating user {0}'.format(username)) |
1802 | 84 | cmd = ['useradd'] | 342 | cmd = ['useradd'] |
1803 | 343 | if uid: | ||
1804 | 344 | cmd.extend(['--uid', str(uid)]) | ||
1805 | 345 | if home_dir: | ||
1806 | 346 | cmd.extend(['--home', str(home_dir)]) | ||
1807 | 85 | if system_user or password is None: | 347 | if system_user or password is None: |
1808 | 86 | cmd.append('--system') | 348 | cmd.append('--system') |
1809 | 87 | else: | 349 | else: |
1810 | @@ -90,32 +352,104 @@ | |||
1811 | 90 | '--shell', shell, | 352 | '--shell', shell, |
1812 | 91 | '--password', password, | 353 | '--password', password, |
1813 | 92 | ]) | 354 | ]) |
1814 | 355 | if not primary_group: | ||
1815 | 356 | try: | ||
1816 | 357 | grp.getgrnam(username) | ||
1817 | 358 | primary_group = username # avoid "group exists" error | ||
1818 | 359 | except KeyError: | ||
1819 | 360 | pass | ||
1820 | 361 | if primary_group: | ||
1821 | 362 | cmd.extend(['-g', primary_group]) | ||
1822 | 363 | if secondary_groups: | ||
1823 | 364 | cmd.extend(['-G', ','.join(secondary_groups)]) | ||
1824 | 93 | cmd.append(username) | 365 | cmd.append(username) |
1825 | 94 | subprocess.check_call(cmd) | 366 | subprocess.check_call(cmd) |
1826 | 95 | user_info = pwd.getpwnam(username) | 367 | user_info = pwd.getpwnam(username) |
1827 | 96 | return user_info | 368 | return user_info |
1828 | 97 | 369 | ||
1829 | 98 | 370 | ||
1830 | 371 | def user_exists(username): | ||
1831 | 372 | """Check if a user exists""" | ||
1832 | 373 | try: | ||
1833 | 374 | pwd.getpwnam(username) | ||
1834 | 375 | user_exists = True | ||
1835 | 376 | except KeyError: | ||
1836 | 377 | user_exists = False | ||
1837 | 378 | return user_exists | ||
1838 | 379 | |||
1839 | 380 | |||
1840 | 381 | def uid_exists(uid): | ||
1841 | 382 | """Check if a uid exists""" | ||
1842 | 383 | try: | ||
1843 | 384 | pwd.getpwuid(uid) | ||
1844 | 385 | uid_exists = True | ||
1845 | 386 | except KeyError: | ||
1846 | 387 | uid_exists = False | ||
1847 | 388 | return uid_exists | ||
1848 | 389 | |||
1849 | 390 | |||
1850 | 391 | def group_exists(groupname): | ||
1851 | 392 | """Check if a group exists""" | ||
1852 | 393 | try: | ||
1853 | 394 | grp.getgrnam(groupname) | ||
1854 | 395 | group_exists = True | ||
1855 | 396 | except KeyError: | ||
1856 | 397 | group_exists = False | ||
1857 | 398 | return group_exists | ||
1858 | 399 | |||
1859 | 400 | |||
1860 | 401 | def gid_exists(gid): | ||
1861 | 402 | """Check if a gid exists""" | ||
1862 | 403 | try: | ||
1863 | 404 | grp.getgrgid(gid) | ||
1864 | 405 | gid_exists = True | ||
1865 | 406 | except KeyError: | ||
1866 | 407 | gid_exists = False | ||
1867 | 408 | return gid_exists | ||
1868 | 409 | |||
1869 | 410 | |||
1870 | 411 | def add_group(group_name, system_group=False, gid=None): | ||
1871 | 412 | """Add a group to the system | ||
1872 | 413 | |||
1873 | 414 | Will log but otherwise succeed if the group already exists. | ||
1874 | 415 | |||
1875 | 416 | :param str group_name: group to create | ||
1876 | 417 | :param bool system_group: Create system group | ||
1877 | 418 | :param int gid: GID for user being created | ||
1878 | 419 | |||
1879 | 420 | :returns: The password database entry struct, as returned by `grp.getgrnam` | ||
1880 | 421 | """ | ||
1881 | 422 | try: | ||
1882 | 423 | group_info = grp.getgrnam(group_name) | ||
1883 | 424 | log('group {0} already exists!'.format(group_name)) | ||
1884 | 425 | if gid: | ||
1885 | 426 | group_info = grp.getgrgid(gid) | ||
1886 | 427 | log('group with gid {0} already exists!'.format(gid)) | ||
1887 | 428 | except KeyError: | ||
1888 | 429 | log('creating group {0}'.format(group_name)) | ||
1889 | 430 | add_new_group(group_name, system_group, gid) | ||
1890 | 431 | group_info = grp.getgrnam(group_name) | ||
1891 | 432 | return group_info | ||
1892 | 433 | |||
1893 | 434 | |||
1894 | 99 | def add_user_to_group(username, group): | 435 | def add_user_to_group(username, group): |
1895 | 100 | """Add a user to a group""" | 436 | """Add a user to a group""" |
1901 | 101 | cmd = [ | 437 | cmd = ['gpasswd', '-a', username, group] |
1897 | 102 | 'gpasswd', '-a', | ||
1898 | 103 | username, | ||
1899 | 104 | group | ||
1900 | 105 | ] | ||
1902 | 106 | log("Adding user {} to group {}".format(username, group)) | 438 | log("Adding user {} to group {}".format(username, group)) |
1903 | 107 | subprocess.check_call(cmd) | 439 | subprocess.check_call(cmd) |
1904 | 108 | 440 | ||
1905 | 109 | 441 | ||
1907 | 110 | def rsync(from_path, to_path, flags='-r', options=None): | 442 | def rsync(from_path, to_path, flags='-r', options=None, timeout=None): |
1908 | 111 | """Replicate the contents of a path""" | 443 | """Replicate the contents of a path""" |
1909 | 112 | options = options or ['--delete', '--executability'] | 444 | options = options or ['--delete', '--executability'] |
1910 | 113 | cmd = ['/usr/bin/rsync', flags] | 445 | cmd = ['/usr/bin/rsync', flags] |
1911 | 446 | if timeout: | ||
1912 | 447 | cmd = ['timeout', str(timeout)] + cmd | ||
1913 | 114 | cmd.extend(options) | 448 | cmd.extend(options) |
1914 | 115 | cmd.append(from_path) | 449 | cmd.append(from_path) |
1915 | 116 | cmd.append(to_path) | 450 | cmd.append(to_path) |
1916 | 117 | log(" ".join(cmd)) | 451 | log(" ".join(cmd)) |
1918 | 118 | return subprocess.check_output(cmd).strip() | 452 | return subprocess.check_output(cmd, stderr=subprocess.STDOUT).decode('UTF-8').strip() |
1919 | 119 | 453 | ||
1920 | 120 | 454 | ||
1921 | 121 | def symlink(source, destination): | 455 | def symlink(source, destination): |
1922 | @@ -130,42 +464,43 @@ | |||
1923 | 130 | subprocess.check_call(cmd) | 464 | subprocess.check_call(cmd) |
1924 | 131 | 465 | ||
1925 | 132 | 466 | ||
1927 | 133 | def mkdir(path, owner='root', group='root', perms=0555, force=False): | 467 | def mkdir(path, owner='root', group='root', perms=0o555, force=False): |
1928 | 134 | """Create a directory""" | 468 | """Create a directory""" |
1929 | 135 | log("Making dir {} {}:{} {:o}".format(path, owner, group, | 469 | log("Making dir {} {}:{} {:o}".format(path, owner, group, |
1930 | 136 | perms)) | 470 | perms)) |
1931 | 137 | uid = pwd.getpwnam(owner).pw_uid | 471 | uid = pwd.getpwnam(owner).pw_uid |
1932 | 138 | gid = grp.getgrnam(group).gr_gid | 472 | gid = grp.getgrnam(group).gr_gid |
1933 | 139 | realpath = os.path.abspath(path) | 473 | realpath = os.path.abspath(path) |
1936 | 140 | if os.path.exists(realpath): | 474 | path_exists = os.path.exists(realpath) |
1937 | 141 | if force and not os.path.isdir(realpath): | 475 | if path_exists and force: |
1938 | 476 | if not os.path.isdir(realpath): | ||
1939 | 142 | log("Removing non-directory file {} prior to mkdir()".format(path)) | 477 | log("Removing non-directory file {} prior to mkdir()".format(path)) |
1940 | 143 | os.unlink(realpath) | 478 | os.unlink(realpath) |
1942 | 144 | else: | 479 | os.makedirs(realpath, perms) |
1943 | 480 | elif not path_exists: | ||
1944 | 145 | os.makedirs(realpath, perms) | 481 | os.makedirs(realpath, perms) |
1945 | 146 | os.chown(realpath, uid, gid) | 482 | os.chown(realpath, uid, gid) |
1950 | 147 | 483 | os.chmod(realpath, perms) | |
1951 | 148 | 484 | ||
1952 | 149 | def write_file(path, content, owner='root', group='root', perms=0444): | 485 | |
1953 | 150 | """Create or overwrite a file with the contents of a string""" | 486 | def write_file(path, content, owner='root', group='root', perms=0o444): |
1954 | 487 | """Create or overwrite a file with the contents of a byte string.""" | ||
1955 | 151 | log("Writing file {} {}:{} {:o}".format(path, owner, group, perms)) | 488 | log("Writing file {} {}:{} {:o}".format(path, owner, group, perms)) |
1956 | 152 | uid = pwd.getpwnam(owner).pw_uid | 489 | uid = pwd.getpwnam(owner).pw_uid |
1957 | 153 | gid = grp.getgrnam(group).gr_gid | 490 | gid = grp.getgrnam(group).gr_gid |
1959 | 154 | with open(path, 'w') as target: | 491 | with open(path, 'wb') as target: |
1960 | 155 | os.fchown(target.fileno(), uid, gid) | 492 | os.fchown(target.fileno(), uid, gid) |
1961 | 156 | os.fchmod(target.fileno(), perms) | 493 | os.fchmod(target.fileno(), perms) |
1962 | 157 | target.write(content) | 494 | target.write(content) |
1963 | 158 | 495 | ||
1964 | 159 | 496 | ||
1965 | 160 | def fstab_remove(mp): | 497 | def fstab_remove(mp): |
1968 | 161 | """Remove the given mountpoint entry from /etc/fstab | 498 | """Remove the given mountpoint entry from /etc/fstab""" |
1967 | 162 | """ | ||
1969 | 163 | return Fstab.remove_by_mountpoint(mp) | 499 | return Fstab.remove_by_mountpoint(mp) |
1970 | 164 | 500 | ||
1971 | 165 | 501 | ||
1972 | 166 | def fstab_add(dev, mp, fs, options=None): | 502 | def fstab_add(dev, mp, fs, options=None): |
1975 | 167 | """Adds the given device entry to the /etc/fstab file | 503 | """Adds the given device entry to the /etc/fstab file""" |
1974 | 168 | """ | ||
1976 | 169 | return Fstab.add(dev, mp, fs, options=options) | 504 | return Fstab.add(dev, mp, fs, options=options) |
1977 | 170 | 505 | ||
1978 | 171 | 506 | ||
1979 | @@ -177,7 +512,7 @@ | |||
1980 | 177 | cmd_args.extend([device, mountpoint]) | 512 | cmd_args.extend([device, mountpoint]) |
1981 | 178 | try: | 513 | try: |
1982 | 179 | subprocess.check_output(cmd_args) | 514 | subprocess.check_output(cmd_args) |
1984 | 180 | except subprocess.CalledProcessError, e: | 515 | except subprocess.CalledProcessError as e: |
1985 | 181 | log('Error mounting {} at {}\n{}'.format(device, mountpoint, e.output)) | 516 | log('Error mounting {} at {}\n{}'.format(device, mountpoint, e.output)) |
1986 | 182 | return False | 517 | return False |
1987 | 183 | 518 | ||
1988 | @@ -191,7 +526,7 @@ | |||
1989 | 191 | cmd_args = ['umount', mountpoint] | 526 | cmd_args = ['umount', mountpoint] |
1990 | 192 | try: | 527 | try: |
1991 | 193 | subprocess.check_output(cmd_args) | 528 | subprocess.check_output(cmd_args) |
1993 | 194 | except subprocess.CalledProcessError, e: | 529 | except subprocess.CalledProcessError as e: |
1994 | 195 | log('Error unmounting {}\n{}'.format(mountpoint, e.output)) | 530 | log('Error unmounting {}\n{}'.format(mountpoint, e.output)) |
1995 | 196 | return False | 531 | return False |
1996 | 197 | 532 | ||
1997 | @@ -209,25 +544,48 @@ | |||
1998 | 209 | return system_mounts | 544 | return system_mounts |
1999 | 210 | 545 | ||
2000 | 211 | 546 | ||
2001 | 547 | def fstab_mount(mountpoint): | ||
2002 | 548 | """Mount filesystem using fstab""" | ||
2003 | 549 | cmd_args = ['mount', mountpoint] | ||
2004 | 550 | try: | ||
2005 | 551 | subprocess.check_output(cmd_args) | ||
2006 | 552 | except subprocess.CalledProcessError as e: | ||
2007 | 553 | log('Error unmounting {}\n{}'.format(mountpoint, e.output)) | ||
2008 | 554 | return False | ||
2009 | 555 | return True | ||
2010 | 556 | |||
2011 | 557 | |||
2012 | 212 | def file_hash(path, hash_type='md5'): | 558 | def file_hash(path, hash_type='md5'): |
2015 | 213 | """ | 559 | """Generate a hash checksum of the contents of 'path' or None if not found. |
2014 | 214 | Generate a hash checksum of the contents of 'path' or None if not found. | ||
2016 | 215 | 560 | ||
2017 | 216 | :param str hash_type: Any hash alrgorithm supported by :mod:`hashlib`, | 561 | :param str hash_type: Any hash alrgorithm supported by :mod:`hashlib`, |
2018 | 217 | such as md5, sha1, sha256, sha512, etc. | 562 | such as md5, sha1, sha256, sha512, etc. |
2019 | 218 | """ | 563 | """ |
2020 | 219 | if os.path.exists(path): | 564 | if os.path.exists(path): |
2021 | 220 | h = getattr(hashlib, hash_type)() | 565 | h = getattr(hashlib, hash_type)() |
2024 | 221 | with open(path, 'r') as source: | 566 | with open(path, 'rb') as source: |
2025 | 222 | h.update(source.read()) # IGNORE:E1101 - it does have update | 567 | h.update(source.read()) |
2026 | 223 | return h.hexdigest() | 568 | return h.hexdigest() |
2027 | 224 | else: | 569 | else: |
2028 | 225 | return None | 570 | return None |
2029 | 226 | 571 | ||
2030 | 227 | 572 | ||
2031 | 573 | def path_hash(path): | ||
2032 | 574 | """Generate a hash checksum of all files matching 'path'. Standard | ||
2033 | 575 | wildcards like '*' and '?' are supported, see documentation for the 'glob' | ||
2034 | 576 | module for more information. | ||
2035 | 577 | |||
2036 | 578 | :return: dict: A { filename: hash } dictionary for all matched files. | ||
2037 | 579 | Empty if none found. | ||
2038 | 580 | """ | ||
2039 | 581 | return { | ||
2040 | 582 | filename: file_hash(filename) | ||
2041 | 583 | for filename in glob.iglob(path) | ||
2042 | 584 | } | ||
2043 | 585 | |||
2044 | 586 | |||
2045 | 228 | def check_hash(path, checksum, hash_type='md5'): | 587 | def check_hash(path, checksum, hash_type='md5'): |
2048 | 229 | """ | 588 | """Validate a file using a cryptographic checksum. |
2047 | 230 | Validate a file using a cryptographic checksum. | ||
2049 | 231 | 589 | ||
2050 | 232 | :param str checksum: Value of the checksum used to validate the file. | 590 | :param str checksum: Value of the checksum used to validate the file. |
2051 | 233 | :param str hash_type: Hash algorithm used to generate `checksum`. | 591 | :param str hash_type: Hash algorithm used to generate `checksum`. |
2052 | @@ -242,100 +600,184 @@ | |||
2053 | 242 | 600 | ||
2054 | 243 | 601 | ||
2055 | 244 | class ChecksumError(ValueError): | 602 | class ChecksumError(ValueError): |
2056 | 603 | """A class derived from Value error to indicate the checksum failed.""" | ||
2057 | 245 | pass | 604 | pass |
2058 | 246 | 605 | ||
2059 | 247 | 606 | ||
2061 | 248 | def restart_on_change(restart_map, stopstart=False): | 607 | def restart_on_change(restart_map, stopstart=False, restart_functions=None): |
2062 | 249 | """Restart services based on configuration files changing | 608 | """Restart services based on configuration files changing |
2063 | 250 | 609 | ||
2064 | 251 | This function is used a decorator, for example:: | 610 | This function is used a decorator, for example:: |
2065 | 252 | 611 | ||
2066 | 253 | @restart_on_change({ | 612 | @restart_on_change({ |
2067 | 254 | '/etc/ceph/ceph.conf': [ 'cinder-api', 'cinder-volume' ] | 613 | '/etc/ceph/ceph.conf': [ 'cinder-api', 'cinder-volume' ] |
2068 | 614 | '/etc/apache/sites-enabled/*': [ 'apache2' ] | ||
2069 | 255 | }) | 615 | }) |
2071 | 256 | def ceph_client_changed(): | 616 | def config_changed(): |
2072 | 257 | pass # your code here | 617 | pass # your code here |
2073 | 258 | 618 | ||
2074 | 259 | In this example, the cinder-api and cinder-volume services | 619 | In this example, the cinder-api and cinder-volume services |
2075 | 260 | would be restarted if /etc/ceph/ceph.conf is changed by the | 620 | would be restarted if /etc/ceph/ceph.conf is changed by the |
2077 | 261 | ceph_client_changed function. | 621 | ceph_client_changed function. The apache2 service would be |
2078 | 622 | restarted if any file matching the pattern got changed, created | ||
2079 | 623 | or removed. Standard wildcards are supported, see documentation | ||
2080 | 624 | for the 'glob' module for more information. | ||
2081 | 625 | |||
2082 | 626 | @param restart_map: {path_file_name: [service_name, ...] | ||
2083 | 627 | @param stopstart: DEFAULT false; whether to stop, start OR restart | ||
2084 | 628 | @param restart_functions: nonstandard functions to use to restart services | ||
2085 | 629 | {svc: func, ...} | ||
2086 | 630 | @returns result from decorated function | ||
2087 | 262 | """ | 631 | """ |
2088 | 263 | def wrap(f): | 632 | def wrap(f): |
2106 | 264 | def wrapped_f(*args): | 633 | @functools.wraps(f) |
2107 | 265 | checksums = {} | 634 | def wrapped_f(*args, **kwargs): |
2108 | 266 | for path in restart_map: | 635 | return restart_on_change_helper( |
2109 | 267 | checksums[path] = file_hash(path) | 636 | (lambda: f(*args, **kwargs)), restart_map, stopstart, |
2110 | 268 | f(*args) | 637 | restart_functions) |
2094 | 269 | restarts = [] | ||
2095 | 270 | for path in restart_map: | ||
2096 | 271 | if checksums[path] != file_hash(path): | ||
2097 | 272 | restarts += restart_map[path] | ||
2098 | 273 | services_list = list(OrderedDict.fromkeys(restarts)) | ||
2099 | 274 | if not stopstart: | ||
2100 | 275 | for service_name in services_list: | ||
2101 | 276 | service('restart', service_name) | ||
2102 | 277 | else: | ||
2103 | 278 | for action in ['stop', 'start']: | ||
2104 | 279 | for service_name in services_list: | ||
2105 | 280 | service(action, service_name) | ||
2111 | 281 | return wrapped_f | 638 | return wrapped_f |
2112 | 282 | return wrap | 639 | return wrap |
2113 | 283 | 640 | ||
2114 | 284 | 641 | ||
2123 | 285 | def lsb_release(): | 642 | def restart_on_change_helper(lambda_f, restart_map, stopstart=False, |
2124 | 286 | """Return /etc/lsb-release in a dict""" | 643 | restart_functions=None): |
2125 | 287 | d = {} | 644 | """Helper function to perform the restart_on_change function. |
2126 | 288 | with open('/etc/lsb-release', 'r') as lsb: | 645 | |
2127 | 289 | for l in lsb: | 646 | This is provided for decorators to restart services if files described |
2128 | 290 | k, v = l.split('=') | 647 | in the restart_map have changed after an invocation of lambda_f(). |
2129 | 291 | d[k.strip()] = v.strip() | 648 | |
2130 | 292 | return d | 649 | @param lambda_f: function to call. |
2131 | 650 | @param restart_map: {file: [service, ...]} | ||
2132 | 651 | @param stopstart: whether to stop, start or restart a service | ||
2133 | 652 | @param restart_functions: nonstandard functions to use to restart services | ||
2134 | 653 | {svc: func, ...} | ||
2135 | 654 | @returns result of lambda_f() | ||
2136 | 655 | """ | ||
2137 | 656 | if restart_functions is None: | ||
2138 | 657 | restart_functions = {} | ||
2139 | 658 | checksums = {path: path_hash(path) for path in restart_map} | ||
2140 | 659 | r = lambda_f() | ||
2141 | 660 | # create a list of lists of the services to restart | ||
2142 | 661 | restarts = [restart_map[path] | ||
2143 | 662 | for path in restart_map | ||
2144 | 663 | if path_hash(path) != checksums[path]] | ||
2145 | 664 | # create a flat list of ordered services without duplicates from lists | ||
2146 | 665 | services_list = list(OrderedDict.fromkeys(itertools.chain(*restarts))) | ||
2147 | 666 | if services_list: | ||
2148 | 667 | actions = ('stop', 'start') if stopstart else ('restart',) | ||
2149 | 668 | for service_name in services_list: | ||
2150 | 669 | if service_name in restart_functions: | ||
2151 | 670 | restart_functions[service_name](service_name) | ||
2152 | 671 | else: | ||
2153 | 672 | for action in actions: | ||
2154 | 673 | service(action, service_name) | ||
2155 | 674 | return r | ||
2156 | 293 | 675 | ||
2157 | 294 | 676 | ||
2158 | 295 | def pwgen(length=None): | 677 | def pwgen(length=None): |
2159 | 296 | """Generate a random pasword.""" | 678 | """Generate a random pasword.""" |
2160 | 297 | if length is None: | 679 | if length is None: |
2161 | 680 | # A random length is ok to use a weak PRNG | ||
2162 | 298 | length = random.choice(range(35, 45)) | 681 | length = random.choice(range(35, 45)) |
2163 | 299 | alphanumeric_chars = [ | 682 | alphanumeric_chars = [ |
2165 | 300 | l for l in (string.letters + string.digits) | 683 | l for l in (string.ascii_letters + string.digits) |
2166 | 301 | if l not in 'l0QD1vAEIOUaeiou'] | 684 | if l not in 'l0QD1vAEIOUaeiou'] |
2167 | 685 | # Use a crypto-friendly PRNG (e.g. /dev/urandom) for making the | ||
2168 | 686 | # actual password | ||
2169 | 687 | random_generator = random.SystemRandom() | ||
2170 | 302 | random_chars = [ | 688 | random_chars = [ |
2172 | 303 | random.choice(alphanumeric_chars) for _ in range(length)] | 689 | random_generator.choice(alphanumeric_chars) for _ in range(length)] |
2173 | 304 | return(''.join(random_chars)) | 690 | return(''.join(random_chars)) |
2174 | 305 | 691 | ||
2175 | 306 | 692 | ||
2179 | 307 | def list_nics(nic_type): | 693 | def is_phy_iface(interface): |
2180 | 308 | '''Return a list of nics of given type(s)''' | 694 | """Returns True if interface is not virtual, otherwise False.""" |
2181 | 309 | if isinstance(nic_type, basestring): | 695 | if interface: |
2182 | 696 | sys_net = '/sys/class/net' | ||
2183 | 697 | if os.path.isdir(sys_net): | ||
2184 | 698 | for iface in glob.glob(os.path.join(sys_net, '*')): | ||
2185 | 699 | if '/virtual/' in os.path.realpath(iface): | ||
2186 | 700 | continue | ||
2187 | 701 | |||
2188 | 702 | if interface == os.path.basename(iface): | ||
2189 | 703 | return True | ||
2190 | 704 | |||
2191 | 705 | return False | ||
2192 | 706 | |||
2193 | 707 | |||
2194 | 708 | def get_bond_master(interface): | ||
2195 | 709 | """Returns bond master if interface is bond slave otherwise None. | ||
2196 | 710 | |||
2197 | 711 | NOTE: the provided interface is expected to be physical | ||
2198 | 712 | """ | ||
2199 | 713 | if interface: | ||
2200 | 714 | iface_path = '/sys/class/net/%s' % (interface) | ||
2201 | 715 | if os.path.exists(iface_path): | ||
2202 | 716 | if '/virtual/' in os.path.realpath(iface_path): | ||
2203 | 717 | return None | ||
2204 | 718 | |||
2205 | 719 | master = os.path.join(iface_path, 'master') | ||
2206 | 720 | if os.path.exists(master): | ||
2207 | 721 | master = os.path.realpath(master) | ||
2208 | 722 | # make sure it is a bond master | ||
2209 | 723 | if os.path.exists(os.path.join(master, 'bonding')): | ||
2210 | 724 | return os.path.basename(master) | ||
2211 | 725 | |||
2212 | 726 | return None | ||
2213 | 727 | |||
2214 | 728 | |||
2215 | 729 | def list_nics(nic_type=None): | ||
2216 | 730 | """Return a list of nics of given type(s)""" | ||
2217 | 731 | if isinstance(nic_type, six.string_types): | ||
2218 | 310 | int_types = [nic_type] | 732 | int_types = [nic_type] |
2219 | 311 | else: | 733 | else: |
2220 | 312 | int_types = nic_type | 734 | int_types = nic_type |
2221 | 735 | |||
2222 | 313 | interfaces = [] | 736 | interfaces = [] |
2227 | 314 | for int_type in int_types: | 737 | if nic_type: |
2228 | 315 | cmd = ['ip', 'addr', 'show', 'label', int_type + '*'] | 738 | for int_type in int_types: |
2229 | 316 | ip_output = subprocess.check_output(cmd).split('\n') | 739 | cmd = ['ip', 'addr', 'show', 'label', int_type + '*'] |
2230 | 317 | ip_output = (line for line in ip_output if line) | 740 | ip_output = subprocess.check_output(cmd).decode('UTF-8') |
2231 | 741 | ip_output = ip_output.split('\n') | ||
2232 | 742 | ip_output = (line for line in ip_output if line) | ||
2233 | 743 | for line in ip_output: | ||
2234 | 744 | if line.split()[1].startswith(int_type): | ||
2235 | 745 | matched = re.search('.*: (' + int_type + | ||
2236 | 746 | r'[0-9]+\.[0-9]+)@.*', line) | ||
2237 | 747 | if matched: | ||
2238 | 748 | iface = matched.groups()[0] | ||
2239 | 749 | else: | ||
2240 | 750 | iface = line.split()[1].replace(":", "") | ||
2241 | 751 | |||
2242 | 752 | if iface not in interfaces: | ||
2243 | 753 | interfaces.append(iface) | ||
2244 | 754 | else: | ||
2245 | 755 | cmd = ['ip', 'a'] | ||
2246 | 756 | ip_output = subprocess.check_output(cmd).decode('UTF-8').split('\n') | ||
2247 | 757 | ip_output = (line.strip() for line in ip_output if line) | ||
2248 | 758 | |||
2249 | 759 | key = re.compile('^[0-9]+:\s+(.+):') | ||
2250 | 318 | for line in ip_output: | 760 | for line in ip_output: |
2258 | 319 | if line.split()[1].startswith(int_type): | 761 | matched = re.search(key, line) |
2259 | 320 | matched = re.search('.*: (bond[0-9]+\.[0-9]+)@.*', line) | 762 | if matched: |
2260 | 321 | if matched: | 763 | iface = matched.group(1) |
2261 | 322 | interface = matched.groups()[0] | 764 | iface = iface.partition("@")[0] |
2262 | 323 | else: | 765 | if iface not in interfaces: |
2263 | 324 | interface = line.split()[1].replace(":", "") | 766 | interfaces.append(iface) |
2257 | 325 | interfaces.append(interface) | ||
2264 | 326 | 767 | ||
2265 | 327 | return interfaces | 768 | return interfaces |
2266 | 328 | 769 | ||
2267 | 329 | 770 | ||
2268 | 330 | def set_nic_mtu(nic, mtu): | 771 | def set_nic_mtu(nic, mtu): |
2270 | 331 | '''Set MTU on a network interface''' | 772 | """Set the Maximum Transmission Unit (MTU) on a network interface.""" |
2271 | 332 | cmd = ['ip', 'link', 'set', nic, 'mtu', mtu] | 773 | cmd = ['ip', 'link', 'set', nic, 'mtu', mtu] |
2272 | 333 | subprocess.check_call(cmd) | 774 | subprocess.check_call(cmd) |
2273 | 334 | 775 | ||
2274 | 335 | 776 | ||
2275 | 336 | def get_nic_mtu(nic): | 777 | def get_nic_mtu(nic): |
2276 | 778 | """Return the Maximum Transmission Unit (MTU) for a network interface.""" | ||
2277 | 337 | cmd = ['ip', 'addr', 'show', nic] | 779 | cmd = ['ip', 'addr', 'show', nic] |
2279 | 338 | ip_output = subprocess.check_output(cmd).split('\n') | 780 | ip_output = subprocess.check_output(cmd).decode('UTF-8').split('\n') |
2280 | 339 | mtu = "" | 781 | mtu = "" |
2281 | 340 | for line in ip_output: | 782 | for line in ip_output: |
2282 | 341 | words = line.split() | 783 | words = line.split() |
2283 | @@ -345,8 +787,9 @@ | |||
2284 | 345 | 787 | ||
2285 | 346 | 788 | ||
2286 | 347 | def get_nic_hwaddr(nic): | 789 | def get_nic_hwaddr(nic): |
2287 | 790 | """Return the Media Access Control (MAC) for a network interface.""" | ||
2288 | 348 | cmd = ['ip', '-o', '-0', 'addr', 'show', nic] | 791 | cmd = ['ip', '-o', '-0', 'addr', 'show', nic] |
2290 | 349 | ip_output = subprocess.check_output(cmd) | 792 | ip_output = subprocess.check_output(cmd).decode('UTF-8') |
2291 | 350 | hwaddr = "" | 793 | hwaddr = "" |
2292 | 351 | words = ip_output.split() | 794 | words = ip_output.split() |
2293 | 352 | if 'link/ether' in words: | 795 | if 'link/ether' in words: |
2294 | @@ -354,38 +797,126 @@ | |||
2295 | 354 | return hwaddr | 797 | return hwaddr |
2296 | 355 | 798 | ||
2297 | 356 | 799 | ||
2298 | 357 | def cmp_pkgrevno(package, revno, pkgcache=None): | ||
2299 | 358 | '''Compare supplied revno with the revno of the installed package | ||
2300 | 359 | |||
2301 | 360 | * 1 => Installed revno is greater than supplied arg | ||
2302 | 361 | * 0 => Installed revno is the same as supplied arg | ||
2303 | 362 | * -1 => Installed revno is less than supplied arg | ||
2304 | 363 | |||
2305 | 364 | ''' | ||
2306 | 365 | import apt_pkg | ||
2307 | 366 | from charmhelpers.fetch import apt_cache | ||
2308 | 367 | if not pkgcache: | ||
2309 | 368 | pkgcache = apt_cache() | ||
2310 | 369 | pkg = pkgcache[package] | ||
2311 | 370 | return apt_pkg.version_compare(pkg.current_ver.ver_str, revno) | ||
2312 | 371 | |||
2313 | 372 | |||
2314 | 373 | @contextmanager | 800 | @contextmanager |
2316 | 374 | def chdir(d): | 801 | def chdir(directory): |
2317 | 802 | """Change the current working directory to a different directory for a code | ||
2318 | 803 | block and return the previous directory after the block exits. Useful to | ||
2319 | 804 | run commands from a specificed directory. | ||
2320 | 805 | |||
2321 | 806 | :param str directory: The directory path to change to for this context. | ||
2322 | 807 | """ | ||
2323 | 375 | cur = os.getcwd() | 808 | cur = os.getcwd() |
2324 | 376 | try: | 809 | try: |
2326 | 377 | yield os.chdir(d) | 810 | yield os.chdir(directory) |
2327 | 378 | finally: | 811 | finally: |
2328 | 379 | os.chdir(cur) | 812 | os.chdir(cur) |
2329 | 380 | 813 | ||
2330 | 381 | 814 | ||
2332 | 382 | def chownr(path, owner, group): | 815 | def chownr(path, owner, group, follow_links=True, chowntopdir=False): |
2333 | 816 | """Recursively change user and group ownership of files and directories | ||
2334 | 817 | in given path. Doesn't chown path itself by default, only its children. | ||
2335 | 818 | |||
2336 | 819 | :param str path: The string path to start changing ownership. | ||
2337 | 820 | :param str owner: The owner string to use when looking up the uid. | ||
2338 | 821 | :param str group: The group string to use when looking up the gid. | ||
2339 | 822 | :param bool follow_links: Also follow and chown links if True | ||
2340 | 823 | :param bool chowntopdir: Also chown path itself if True | ||
2341 | 824 | """ | ||
2342 | 383 | uid = pwd.getpwnam(owner).pw_uid | 825 | uid = pwd.getpwnam(owner).pw_uid |
2343 | 384 | gid = grp.getgrnam(group).gr_gid | 826 | gid = grp.getgrnam(group).gr_gid |
2344 | 827 | if follow_links: | ||
2345 | 828 | chown = os.chown | ||
2346 | 829 | else: | ||
2347 | 830 | chown = os.lchown | ||
2348 | 385 | 831 | ||
2350 | 386 | for root, dirs, files in os.walk(path): | 832 | if chowntopdir: |
2351 | 833 | broken_symlink = os.path.lexists(path) and not os.path.exists(path) | ||
2352 | 834 | if not broken_symlink: | ||
2353 | 835 | chown(path, uid, gid) | ||
2354 | 836 | for root, dirs, files in os.walk(path, followlinks=follow_links): | ||
2355 | 387 | for name in dirs + files: | 837 | for name in dirs + files: |
2356 | 388 | full = os.path.join(root, name) | 838 | full = os.path.join(root, name) |
2357 | 389 | broken_symlink = os.path.lexists(full) and not os.path.exists(full) | 839 | broken_symlink = os.path.lexists(full) and not os.path.exists(full) |
2358 | 390 | if not broken_symlink: | 840 | if not broken_symlink: |
2360 | 391 | os.chown(full, uid, gid) | 841 | chown(full, uid, gid) |
2361 | 842 | |||
2362 | 843 | |||
2363 | 844 | def lchownr(path, owner, group): | ||
2364 | 845 | """Recursively change user and group ownership of files and directories | ||
2365 | 846 | in a given path, not following symbolic links. See the documentation for | ||
2366 | 847 | 'os.lchown' for more information. | ||
2367 | 848 | |||
2368 | 849 | :param str path: The string path to start changing ownership. | ||
2369 | 850 | :param str owner: The owner string to use when looking up the uid. | ||
2370 | 851 | :param str group: The group string to use when looking up the gid. | ||
2371 | 852 | """ | ||
2372 | 853 | chownr(path, owner, group, follow_links=False) | ||
2373 | 854 | |||
2374 | 855 | |||
2375 | 856 | def owner(path): | ||
2376 | 857 | """Returns a tuple containing the username & groupname owning the path. | ||
2377 | 858 | |||
2378 | 859 | :param str path: the string path to retrieve the ownership | ||
2379 | 860 | :return tuple(str, str): A (username, groupname) tuple containing the | ||
2380 | 861 | name of the user and group owning the path. | ||
2381 | 862 | :raises OSError: if the specified path does not exist | ||
2382 | 863 | """ | ||
2383 | 864 | stat = os.stat(path) | ||
2384 | 865 | username = pwd.getpwuid(stat.st_uid)[0] | ||
2385 | 866 | groupname = grp.getgrgid(stat.st_gid)[0] | ||
2386 | 867 | return username, groupname | ||
2387 | 868 | |||
2388 | 869 | |||
2389 | 870 | def get_total_ram(): | ||
2390 | 871 | """The total amount of system RAM in bytes. | ||
2391 | 872 | |||
2392 | 873 | This is what is reported by the OS, and may be overcommitted when | ||
2393 | 874 | there are multiple containers hosted on the same machine. | ||
2394 | 875 | """ | ||
2395 | 876 | with open('/proc/meminfo', 'r') as f: | ||
2396 | 877 | for line in f.readlines(): | ||
2397 | 878 | if line: | ||
2398 | 879 | key, value, unit = line.split() | ||
2399 | 880 | if key == 'MemTotal:': | ||
2400 | 881 | assert unit == 'kB', 'Unknown unit' | ||
2401 | 882 | return int(value) * 1024 # Classic, not KiB. | ||
2402 | 883 | raise NotImplementedError() | ||
2403 | 884 | |||
2404 | 885 | |||
2405 | 886 | UPSTART_CONTAINER_TYPE = '/run/container_type' | ||
2406 | 887 | |||
2407 | 888 | |||
2408 | 889 | def is_container(): | ||
2409 | 890 | """Determine whether unit is running in a container | ||
2410 | 891 | |||
2411 | 892 | @return: boolean indicating if unit is in a container | ||
2412 | 893 | """ | ||
2413 | 894 | if init_is_systemd(): | ||
2414 | 895 | # Detect using systemd-detect-virt | ||
2415 | 896 | return subprocess.call(['systemd-detect-virt', | ||
2416 | 897 | '--container']) == 0 | ||
2417 | 898 | else: | ||
2418 | 899 | # Detect using upstart container file marker | ||
2419 | 900 | return os.path.exists(UPSTART_CONTAINER_TYPE) | ||
2420 | 901 | |||
2421 | 902 | |||
2422 | 903 | def add_to_updatedb_prunepath(path, updatedb_path=UPDATEDB_PATH): | ||
2423 | 904 | with open(updatedb_path, 'r+') as f_id: | ||
2424 | 905 | updatedb_text = f_id.read() | ||
2425 | 906 | output = updatedb(updatedb_text, path) | ||
2426 | 907 | f_id.seek(0) | ||
2427 | 908 | f_id.write(output) | ||
2428 | 909 | f_id.truncate() | ||
2429 | 910 | |||
2430 | 911 | |||
2431 | 912 | def updatedb(updatedb_text, new_path): | ||
2432 | 913 | lines = [line for line in updatedb_text.split("\n")] | ||
2433 | 914 | for i, line in enumerate(lines): | ||
2434 | 915 | if line.startswith("PRUNEPATHS="): | ||
2435 | 916 | paths_line = line.split("=")[1].replace('"', '') | ||
2436 | 917 | paths = paths_line.split(" ") | ||
2437 | 918 | if new_path not in paths: | ||
2438 | 919 | paths.append(new_path) | ||
2439 | 920 | lines[i] = 'PRUNEPATHS="{}"'.format(' '.join(paths)) | ||
2440 | 921 | output = "\n".join(lines) | ||
2441 | 922 | return output | ||
2442 | 392 | 923 | ||
2443 | === added directory 'hooks/charmhelpers/core/host_factory' | |||
2444 | === added file 'hooks/charmhelpers/core/host_factory/__init__.py' | |||
2445 | === added file 'hooks/charmhelpers/core/host_factory/centos.py' | |||
2446 | --- hooks/charmhelpers/core/host_factory/centos.py 1970-01-01 00:00:00 +0000 | |||
2447 | +++ hooks/charmhelpers/core/host_factory/centos.py 2017-05-11 14:56:57 +0000 | |||
2448 | @@ -0,0 +1,72 @@ | |||
2449 | 1 | import subprocess | ||
2450 | 2 | import yum | ||
2451 | 3 | import os | ||
2452 | 4 | |||
2453 | 5 | from charmhelpers.core.strutils import BasicStringComparator | ||
2454 | 6 | |||
2455 | 7 | |||
2456 | 8 | class CompareHostReleases(BasicStringComparator): | ||
2457 | 9 | """Provide comparisons of Host releases. | ||
2458 | 10 | |||
2459 | 11 | Use in the form of | ||
2460 | 12 | |||
2461 | 13 | if CompareHostReleases(release) > 'trusty': | ||
2462 | 14 | # do something with mitaka | ||
2463 | 15 | """ | ||
2464 | 16 | |||
2465 | 17 | def __init__(self, item): | ||
2466 | 18 | raise NotImplementedError( | ||
2467 | 19 | "CompareHostReleases() is not implemented for CentOS") | ||
2468 | 20 | |||
2469 | 21 | |||
2470 | 22 | def service_available(service_name): | ||
2471 | 23 | # """Determine whether a system service is available.""" | ||
2472 | 24 | if os.path.isdir('/run/systemd/system'): | ||
2473 | 25 | cmd = ['systemctl', 'is-enabled', service_name] | ||
2474 | 26 | else: | ||
2475 | 27 | cmd = ['service', service_name, 'is-enabled'] | ||
2476 | 28 | return subprocess.call(cmd) == 0 | ||
2477 | 29 | |||
2478 | 30 | |||
2479 | 31 | def add_new_group(group_name, system_group=False, gid=None): | ||
2480 | 32 | cmd = ['groupadd'] | ||
2481 | 33 | if gid: | ||
2482 | 34 | cmd.extend(['--gid', str(gid)]) | ||
2483 | 35 | if system_group: | ||
2484 | 36 | cmd.append('-r') | ||
2485 | 37 | cmd.append(group_name) | ||
2486 | 38 | subprocess.check_call(cmd) | ||
2487 | 39 | |||
2488 | 40 | |||
2489 | 41 | def lsb_release(): | ||
2490 | 42 | """Return /etc/os-release in a dict.""" | ||
2491 | 43 | d = {} | ||
2492 | 44 | with open('/etc/os-release', 'r') as lsb: | ||
2493 | 45 | for l in lsb: | ||
2494 | 46 | s = l.split('=') | ||
2495 | 47 | if len(s) != 2: | ||
2496 | 48 | continue | ||
2497 | 49 | d[s[0].strip()] = s[1].strip() | ||
2498 | 50 | return d | ||
2499 | 51 | |||
2500 | 52 | |||
2501 | 53 | def cmp_pkgrevno(package, revno, pkgcache=None): | ||
2502 | 54 | """Compare supplied revno with the revno of the installed package. | ||
2503 | 55 | |||
2504 | 56 | * 1 => Installed revno is greater than supplied arg | ||
2505 | 57 | * 0 => Installed revno is the same as supplied arg | ||
2506 | 58 | * -1 => Installed revno is less than supplied arg | ||
2507 | 59 | |||
2508 | 60 | This function imports YumBase function if the pkgcache argument | ||
2509 | 61 | is None. | ||
2510 | 62 | """ | ||
2511 | 63 | if not pkgcache: | ||
2512 | 64 | y = yum.YumBase() | ||
2513 | 65 | packages = y.doPackageLists() | ||
2514 | 66 | pkgcache = {i.Name: i.version for i in packages['installed']} | ||
2515 | 67 | pkg = pkgcache[package] | ||
2516 | 68 | if pkg > revno: | ||
2517 | 69 | return 1 | ||
2518 | 70 | if pkg < revno: | ||
2519 | 71 | return -1 | ||
2520 | 72 | return 0 | ||
2521 | 0 | 73 | ||
2522 | === added file 'hooks/charmhelpers/core/host_factory/ubuntu.py' | |||
2523 | --- hooks/charmhelpers/core/host_factory/ubuntu.py 1970-01-01 00:00:00 +0000 | |||
2524 | +++ hooks/charmhelpers/core/host_factory/ubuntu.py 2017-05-11 14:56:57 +0000 | |||
2525 | @@ -0,0 +1,88 @@ | |||
2526 | 1 | import subprocess | ||
2527 | 2 | |||
2528 | 3 | from charmhelpers.core.strutils import BasicStringComparator | ||
2529 | 4 | |||
2530 | 5 | |||
2531 | 6 | UBUNTU_RELEASES = ( | ||
2532 | 7 | 'lucid', | ||
2533 | 8 | 'maverick', | ||
2534 | 9 | 'natty', | ||
2535 | 10 | 'oneiric', | ||
2536 | 11 | 'precise', | ||
2537 | 12 | 'quantal', | ||
2538 | 13 | 'raring', | ||
2539 | 14 | 'saucy', | ||
2540 | 15 | 'trusty', | ||
2541 | 16 | 'utopic', | ||
2542 | 17 | 'vivid', | ||
2543 | 18 | 'wily', | ||
2544 | 19 | 'xenial', | ||
2545 | 20 | 'yakkety', | ||
2546 | 21 | 'zesty', | ||
2547 | 22 | ) | ||
2548 | 23 | |||
2549 | 24 | |||
2550 | 25 | class CompareHostReleases(BasicStringComparator): | ||
2551 | 26 | """Provide comparisons of Ubuntu releases. | ||
2552 | 27 | |||
2553 | 28 | Use in the form of | ||
2554 | 29 | |||
2555 | 30 | if CompareHostReleases(release) > 'trusty': | ||
2556 | 31 | # do something with mitaka | ||
2557 | 32 | """ | ||
2558 | 33 | _list = UBUNTU_RELEASES | ||
2559 | 34 | |||
2560 | 35 | |||
2561 | 36 | def service_available(service_name): | ||
2562 | 37 | """Determine whether a system service is available""" | ||
2563 | 38 | try: | ||
2564 | 39 | subprocess.check_output( | ||
2565 | 40 | ['service', service_name, 'status'], | ||
2566 | 41 | stderr=subprocess.STDOUT).decode('UTF-8') | ||
2567 | 42 | except subprocess.CalledProcessError as e: | ||
2568 | 43 | return b'unrecognized service' not in e.output | ||
2569 | 44 | else: | ||
2570 | 45 | return True | ||
2571 | 46 | |||
2572 | 47 | |||
2573 | 48 | def add_new_group(group_name, system_group=False, gid=None): | ||
2574 | 49 | cmd = ['addgroup'] | ||
2575 | 50 | if gid: | ||
2576 | 51 | cmd.extend(['--gid', str(gid)]) | ||
2577 | 52 | if system_group: | ||
2578 | 53 | cmd.append('--system') | ||
2579 | 54 | else: | ||
2580 | 55 | cmd.extend([ | ||
2581 | 56 | '--group', | ||
2582 | 57 | ]) | ||
2583 | 58 | cmd.append(group_name) | ||
2584 | 59 | subprocess.check_call(cmd) | ||
2585 | 60 | |||
2586 | 61 | |||
2587 | 62 | def lsb_release(): | ||
2588 | 63 | """Return /etc/lsb-release in a dict""" | ||
2589 | 64 | d = {} | ||
2590 | 65 | with open('/etc/lsb-release', 'r') as lsb: | ||
2591 | 66 | for l in lsb: | ||
2592 | 67 | k, v = l.split('=') | ||
2593 | 68 | d[k.strip()] = v.strip() | ||
2594 | 69 | return d | ||
2595 | 70 | |||
2596 | 71 | |||
2597 | 72 | def cmp_pkgrevno(package, revno, pkgcache=None): | ||
2598 | 73 | """Compare supplied revno with the revno of the installed package. | ||
2599 | 74 | |||
2600 | 75 | * 1 => Installed revno is greater than supplied arg | ||
2601 | 76 | * 0 => Installed revno is the same as supplied arg | ||
2602 | 77 | * -1 => Installed revno is less than supplied arg | ||
2603 | 78 | |||
2604 | 79 | This function imports apt_cache function from charmhelpers.fetch if | ||
2605 | 80 | the pkgcache argument is None. Be sure to add charmhelpers.fetch if | ||
2606 | 81 | you call this function, or pass an apt_pkg.Cache() instance. | ||
2607 | 82 | """ | ||
2608 | 83 | import apt_pkg | ||
2609 | 84 | if not pkgcache: | ||
2610 | 85 | from charmhelpers.fetch import apt_cache | ||
2611 | 86 | pkgcache = apt_cache() | ||
2612 | 87 | pkg = pkgcache[package] | ||
2613 | 88 | return apt_pkg.version_compare(pkg.current_ver.ver_str, revno) | ||
2614 | 0 | 89 | ||
2615 | === added file 'hooks/charmhelpers/core/hugepage.py' | |||
2616 | --- hooks/charmhelpers/core/hugepage.py 1970-01-01 00:00:00 +0000 | |||
2617 | +++ hooks/charmhelpers/core/hugepage.py 2017-05-11 14:56:57 +0000 | |||
2618 | @@ -0,0 +1,69 @@ | |||
2619 | 1 | # -*- coding: utf-8 -*- | ||
2620 | 2 | |||
2621 | 3 | # Copyright 2014-2015 Canonical Limited. | ||
2622 | 4 | # | ||
2623 | 5 | # Licensed under the Apache License, Version 2.0 (the "License"); | ||
2624 | 6 | # you may not use this file except in compliance with the License. | ||
2625 | 7 | # You may obtain a copy of the License at | ||
2626 | 8 | # | ||
2627 | 9 | # http://www.apache.org/licenses/LICENSE-2.0 | ||
2628 | 10 | # | ||
2629 | 11 | # Unless required by applicable law or agreed to in writing, software | ||
2630 | 12 | # distributed under the License is distributed on an "AS IS" BASIS, | ||
2631 | 13 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
2632 | 14 | # See the License for the specific language governing permissions and | ||
2633 | 15 | # limitations under the License. | ||
2634 | 16 | |||
2635 | 17 | import yaml | ||
2636 | 18 | from charmhelpers.core import fstab | ||
2637 | 19 | from charmhelpers.core import sysctl | ||
2638 | 20 | from charmhelpers.core.host import ( | ||
2639 | 21 | add_group, | ||
2640 | 22 | add_user_to_group, | ||
2641 | 23 | fstab_mount, | ||
2642 | 24 | mkdir, | ||
2643 | 25 | ) | ||
2644 | 26 | from charmhelpers.core.strutils import bytes_from_string | ||
2645 | 27 | from subprocess import check_output | ||
2646 | 28 | |||
2647 | 29 | |||
2648 | 30 | def hugepage_support(user, group='hugetlb', nr_hugepages=256, | ||
2649 | 31 | max_map_count=65536, mnt_point='/run/hugepages/kvm', | ||
2650 | 32 | pagesize='2MB', mount=True, set_shmmax=False): | ||
2651 | 33 | """Enable hugepages on system. | ||
2652 | 34 | |||
2653 | 35 | Args: | ||
2654 | 36 | user (str) -- Username to allow access to hugepages to | ||
2655 | 37 | group (str) -- Group name to own hugepages | ||
2656 | 38 | nr_hugepages (int) -- Number of pages to reserve | ||
2657 | 39 | max_map_count (int) -- Number of Virtual Memory Areas a process can own | ||
2658 | 40 | mnt_point (str) -- Directory to mount hugepages on | ||
2659 | 41 | pagesize (str) -- Size of hugepages | ||
2660 | 42 | mount (bool) -- Whether to Mount hugepages | ||
2661 | 43 | """ | ||
2662 | 44 | group_info = add_group(group) | ||
2663 | 45 | gid = group_info.gr_gid | ||
2664 | 46 | add_user_to_group(user, group) | ||
2665 | 47 | if max_map_count < 2 * nr_hugepages: | ||
2666 | 48 | max_map_count = 2 * nr_hugepages | ||
2667 | 49 | sysctl_settings = { | ||
2668 | 50 | 'vm.nr_hugepages': nr_hugepages, | ||
2669 | 51 | 'vm.max_map_count': max_map_count, | ||
2670 | 52 | 'vm.hugetlb_shm_group': gid, | ||
2671 | 53 | } | ||
2672 | 54 | if set_shmmax: | ||
2673 | 55 | shmmax_current = int(check_output(['sysctl', '-n', 'kernel.shmmax'])) | ||
2674 | 56 | shmmax_minsize = bytes_from_string(pagesize) * nr_hugepages | ||
2675 | 57 | if shmmax_minsize > shmmax_current: | ||
2676 | 58 | sysctl_settings['kernel.shmmax'] = shmmax_minsize | ||
2677 | 59 | sysctl.create(yaml.dump(sysctl_settings), '/etc/sysctl.d/10-hugepage.conf') | ||
2678 | 60 | mkdir(mnt_point, owner='root', group='root', perms=0o755, force=False) | ||
2679 | 61 | lfstab = fstab.Fstab() | ||
2680 | 62 | fstab_entry = lfstab.get_entry_by_attr('mountpoint', mnt_point) | ||
2681 | 63 | if fstab_entry: | ||
2682 | 64 | lfstab.remove_entry(fstab_entry) | ||
2683 | 65 | entry = lfstab.Entry('nodev', mnt_point, 'hugetlbfs', | ||
2684 | 66 | 'mode=1770,gid={},pagesize={}'.format(gid, pagesize), 0, 0) | ||
2685 | 67 | lfstab.add_entry(entry) | ||
2686 | 68 | if mount: | ||
2687 | 69 | fstab_mount(mnt_point) | ||
2688 | 0 | 70 | ||
2689 | === added file 'hooks/charmhelpers/core/kernel.py' | |||
2690 | --- hooks/charmhelpers/core/kernel.py 1970-01-01 00:00:00 +0000 | |||
2691 | +++ hooks/charmhelpers/core/kernel.py 2017-05-11 14:56:57 +0000 | |||
2692 | @@ -0,0 +1,72 @@ | |||
2693 | 1 | #!/usr/bin/env python | ||
2694 | 2 | # -*- coding: utf-8 -*- | ||
2695 | 3 | |||
2696 | 4 | # Copyright 2014-2015 Canonical Limited. | ||
2697 | 5 | # | ||
2698 | 6 | # Licensed under the Apache License, Version 2.0 (the "License"); | ||
2699 | 7 | # you may not use this file except in compliance with the License. | ||
2700 | 8 | # You may obtain a copy of the License at | ||
2701 | 9 | # | ||
2702 | 10 | # http://www.apache.org/licenses/LICENSE-2.0 | ||
2703 | 11 | # | ||
2704 | 12 | # Unless required by applicable law or agreed to in writing, software | ||
2705 | 13 | # distributed under the License is distributed on an "AS IS" BASIS, | ||
2706 | 14 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
2707 | 15 | # See the License for the specific language governing permissions and | ||
2708 | 16 | # limitations under the License. | ||
2709 | 17 | |||
2710 | 18 | import re | ||
2711 | 19 | import subprocess | ||
2712 | 20 | |||
2713 | 21 | from charmhelpers.osplatform import get_platform | ||
2714 | 22 | from charmhelpers.core.hookenv import ( | ||
2715 | 23 | log, | ||
2716 | 24 | INFO | ||
2717 | 25 | ) | ||
2718 | 26 | |||
2719 | 27 | __platform__ = get_platform() | ||
2720 | 28 | if __platform__ == "ubuntu": | ||
2721 | 29 | from charmhelpers.core.kernel_factory.ubuntu import ( | ||
2722 | 30 | persistent_modprobe, | ||
2723 | 31 | update_initramfs, | ||
2724 | 32 | ) # flake8: noqa -- ignore F401 for this import | ||
2725 | 33 | elif __platform__ == "centos": | ||
2726 | 34 | from charmhelpers.core.kernel_factory.centos import ( | ||
2727 | 35 | persistent_modprobe, | ||
2728 | 36 | update_initramfs, | ||
2729 | 37 | ) # flake8: noqa -- ignore F401 for this import | ||
2730 | 38 | |||
2731 | 39 | __author__ = "Jorge Niedbalski <jorge.niedbalski@canonical.com>" | ||
2732 | 40 | |||
2733 | 41 | |||
2734 | 42 | def modprobe(module, persist=True): | ||
2735 | 43 | """Load a kernel module and configure for auto-load on reboot.""" | ||
2736 | 44 | cmd = ['modprobe', module] | ||
2737 | 45 | |||
2738 | 46 | log('Loading kernel module %s' % module, level=INFO) | ||
2739 | 47 | |||
2740 | 48 | subprocess.check_call(cmd) | ||
2741 | 49 | if persist: | ||
2742 | 50 | persistent_modprobe(module) | ||
2743 | 51 | |||
2744 | 52 | |||
2745 | 53 | def rmmod(module, force=False): | ||
2746 | 54 | """Remove a module from the linux kernel""" | ||
2747 | 55 | cmd = ['rmmod'] | ||
2748 | 56 | if force: | ||
2749 | 57 | cmd.append('-f') | ||
2750 | 58 | cmd.append(module) | ||
2751 | 59 | log('Removing kernel module %s' % module, level=INFO) | ||
2752 | 60 | return subprocess.check_call(cmd) | ||
2753 | 61 | |||
2754 | 62 | |||
2755 | 63 | def lsmod(): | ||
2756 | 64 | """Shows what kernel modules are currently loaded""" | ||
2757 | 65 | return subprocess.check_output(['lsmod'], | ||
2758 | 66 | universal_newlines=True) | ||
2759 | 67 | |||
2760 | 68 | |||
2761 | 69 | def is_module_loaded(module): | ||
2762 | 70 | """Checks if a kernel module is already loaded""" | ||
2763 | 71 | matches = re.findall('^%s[ ]+' % module, lsmod(), re.M) | ||
2764 | 72 | return len(matches) > 0 | ||
2765 | 0 | 73 | ||
2766 | === added directory 'hooks/charmhelpers/core/kernel_factory' | |||
2767 | === added file 'hooks/charmhelpers/core/kernel_factory/__init__.py' | |||
2768 | === added file 'hooks/charmhelpers/core/kernel_factory/centos.py' | |||
2769 | --- hooks/charmhelpers/core/kernel_factory/centos.py 1970-01-01 00:00:00 +0000 | |||
2770 | +++ hooks/charmhelpers/core/kernel_factory/centos.py 2017-05-11 14:56:57 +0000 | |||
2771 | @@ -0,0 +1,17 @@ | |||
2772 | 1 | import subprocess | ||
2773 | 2 | import os | ||
2774 | 3 | |||
2775 | 4 | |||
2776 | 5 | def persistent_modprobe(module): | ||
2777 | 6 | """Load a kernel module and configure for auto-load on reboot.""" | ||
2778 | 7 | if not os.path.exists('/etc/rc.modules'): | ||
2779 | 8 | open('/etc/rc.modules', 'a') | ||
2780 | 9 | os.chmod('/etc/rc.modules', 111) | ||
2781 | 10 | with open('/etc/rc.modules', 'r+') as modules: | ||
2782 | 11 | if module not in modules.read(): | ||
2783 | 12 | modules.write('modprobe %s\n' % module) | ||
2784 | 13 | |||
2785 | 14 | |||
2786 | 15 | def update_initramfs(version='all'): | ||
2787 | 16 | """Updates an initramfs image.""" | ||
2788 | 17 | return subprocess.check_call(["dracut", "-f", version]) | ||
2789 | 0 | 18 | ||
2790 | === added file 'hooks/charmhelpers/core/kernel_factory/ubuntu.py' | |||
2791 | --- hooks/charmhelpers/core/kernel_factory/ubuntu.py 1970-01-01 00:00:00 +0000 | |||
2792 | +++ hooks/charmhelpers/core/kernel_factory/ubuntu.py 2017-05-11 14:56:57 +0000 | |||
2793 | @@ -0,0 +1,13 @@ | |||
2794 | 1 | import subprocess | ||
2795 | 2 | |||
2796 | 3 | |||
2797 | 4 | def persistent_modprobe(module): | ||
2798 | 5 | """Load a kernel module and configure for auto-load on reboot.""" | ||
2799 | 6 | with open('/etc/modules', 'r+') as modules: | ||
2800 | 7 | if module not in modules.read(): | ||
2801 | 8 | modules.write(module + "\n") | ||
2802 | 9 | |||
2803 | 10 | |||
2804 | 11 | def update_initramfs(version='all'): | ||
2805 | 12 | """Updates an initramfs image.""" | ||
2806 | 13 | return subprocess.check_call(["update-initramfs", "-k", version, "-u"]) | ||
2807 | 0 | 14 | ||
2808 | === modified file 'hooks/charmhelpers/core/services/__init__.py' | |||
2809 | --- hooks/charmhelpers/core/services/__init__.py 2014-10-24 16:35:00 +0000 | |||
2810 | +++ hooks/charmhelpers/core/services/__init__.py 2017-05-11 14:56:57 +0000 | |||
2811 | @@ -1,2 +1,16 @@ | |||
2812 | 1 | # Copyright 2014-2015 Canonical Limited. | ||
2813 | 2 | # | ||
2814 | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); | ||
2815 | 4 | # you may not use this file except in compliance with the License. | ||
2816 | 5 | # You may obtain a copy of the License at | ||
2817 | 6 | # | ||
2818 | 7 | # http://www.apache.org/licenses/LICENSE-2.0 | ||
2819 | 8 | # | ||
2820 | 9 | # Unless required by applicable law or agreed to in writing, software | ||
2821 | 10 | # distributed under the License is distributed on an "AS IS" BASIS, | ||
2822 | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
2823 | 12 | # See the License for the specific language governing permissions and | ||
2824 | 13 | # limitations under the License. | ||
2825 | 14 | |||
2826 | 1 | from .base import * # NOQA | 15 | from .base import * # NOQA |
2827 | 2 | from .helpers import * # NOQA | 16 | from .helpers import * # NOQA |
2828 | 3 | 17 | ||
2829 | === modified file 'hooks/charmhelpers/core/services/base.py' | |||
2830 | --- hooks/charmhelpers/core/services/base.py 2014-10-24 16:35:00 +0000 | |||
2831 | +++ hooks/charmhelpers/core/services/base.py 2017-05-11 14:56:57 +0000 | |||
2832 | @@ -1,7 +1,21 @@ | |||
2833 | 1 | # Copyright 2014-2015 Canonical Limited. | ||
2834 | 2 | # | ||
2835 | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); | ||
2836 | 4 | # you may not use this file except in compliance with the License. | ||
2837 | 5 | # You may obtain a copy of the License at | ||
2838 | 6 | # | ||
2839 | 7 | # http://www.apache.org/licenses/LICENSE-2.0 | ||
2840 | 8 | # | ||
2841 | 9 | # Unless required by applicable law or agreed to in writing, software | ||
2842 | 10 | # distributed under the License is distributed on an "AS IS" BASIS, | ||
2843 | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
2844 | 12 | # See the License for the specific language governing permissions and | ||
2845 | 13 | # limitations under the License. | ||
2846 | 14 | |||
2847 | 1 | import os | 15 | import os |
2848 | 2 | import re | ||
2849 | 3 | import json | 16 | import json |
2851 | 4 | from collections import Iterable | 17 | from inspect import getargspec |
2852 | 18 | from collections import Iterable, OrderedDict | ||
2853 | 5 | 19 | ||
2854 | 6 | from charmhelpers.core import host | 20 | from charmhelpers.core import host |
2855 | 7 | from charmhelpers.core import hookenv | 21 | from charmhelpers.core import hookenv |
2856 | @@ -103,7 +117,7 @@ | |||
2857 | 103 | """ | 117 | """ |
2858 | 104 | self._ready_file = os.path.join(hookenv.charm_dir(), 'READY-SERVICES.json') | 118 | self._ready_file = os.path.join(hookenv.charm_dir(), 'READY-SERVICES.json') |
2859 | 105 | self._ready = None | 119 | self._ready = None |
2861 | 106 | self.services = {} | 120 | self.services = OrderedDict() |
2862 | 107 | for service in services or []: | 121 | for service in services or []: |
2863 | 108 | service_name = service['service'] | 122 | service_name = service['service'] |
2864 | 109 | self.services[service_name] = service | 123 | self.services[service_name] = service |
2865 | @@ -112,15 +126,18 @@ | |||
2866 | 112 | """ | 126 | """ |
2867 | 113 | Handle the current hook by doing The Right Thing with the registered services. | 127 | Handle the current hook by doing The Right Thing with the registered services. |
2868 | 114 | """ | 128 | """ |
2878 | 115 | hook_name = hookenv.hook_name() | 129 | hookenv._run_atstart() |
2879 | 116 | if hook_name == 'stop': | 130 | try: |
2880 | 117 | self.stop_services() | 131 | hook_name = hookenv.hook_name() |
2881 | 118 | else: | 132 | if hook_name == 'stop': |
2882 | 119 | self.provide_data() | 133 | self.stop_services() |
2883 | 120 | self.reconfigure_services() | 134 | else: |
2884 | 121 | cfg = hookenv.config() | 135 | self.reconfigure_services() |
2885 | 122 | if cfg.implicit_save: | 136 | self.provide_data() |
2886 | 123 | cfg.save() | 137 | except SystemExit as x: |
2887 | 138 | if x.code is None or x.code == 0: | ||
2888 | 139 | hookenv._run_atexit() | ||
2889 | 140 | hookenv._run_atexit() | ||
2890 | 124 | 141 | ||
2891 | 125 | def provide_data(self): | 142 | def provide_data(self): |
2892 | 126 | """ | 143 | """ |
2893 | @@ -129,15 +146,36 @@ | |||
2894 | 129 | A provider must have a `name` attribute, which indicates which relation | 146 | A provider must have a `name` attribute, which indicates which relation |
2895 | 130 | to set data on, and a `provide_data()` method, which returns a dict of | 147 | to set data on, and a `provide_data()` method, which returns a dict of |
2896 | 131 | data to set. | 148 | data to set. |
2897 | 149 | |||
2898 | 150 | The `provide_data()` method can optionally accept two parameters: | ||
2899 | 151 | |||
2900 | 152 | * ``remote_service`` The name of the remote service that the data will | ||
2901 | 153 | be provided to. The `provide_data()` method will be called once | ||
2902 | 154 | for each connected service (not unit). This allows the method to | ||
2903 | 155 | tailor its data to the given service. | ||
2904 | 156 | * ``service_ready`` Whether or not the service definition had all of | ||
2905 | 157 | its requirements met, and thus the ``data_ready`` callbacks run. | ||
2906 | 158 | |||
2907 | 159 | Note that the ``provided_data`` methods are now called **after** the | ||
2908 | 160 | ``data_ready`` callbacks are run. This gives the ``data_ready`` callbacks | ||
2909 | 161 | a chance to generate any data necessary for the providing to the remote | ||
2910 | 162 | services. | ||
2911 | 132 | """ | 163 | """ |
2914 | 133 | hook_name = hookenv.hook_name() | 164 | for service_name, service in self.services.items(): |
2915 | 134 | for service in self.services.values(): | 165 | service_ready = self.is_ready(service_name) |
2916 | 135 | for provider in service.get('provided_data', []): | 166 | for provider in service.get('provided_data', []): |
2922 | 136 | if re.match(r'{}-relation-(joined|changed)'.format(provider.name), hook_name): | 167 | for relid in hookenv.relation_ids(provider.name): |
2923 | 137 | data = provider.provide_data() | 168 | units = hookenv.related_units(relid) |
2924 | 138 | _ready = provider._is_ready(data) if hasattr(provider, '_is_ready') else data | 169 | if not units: |
2925 | 139 | if _ready: | 170 | continue |
2926 | 140 | hookenv.relation_set(None, data) | 171 | remote_service = units[0].split('/')[0] |
2927 | 172 | argspec = getargspec(provider.provide_data) | ||
2928 | 173 | if len(argspec.args) > 1: | ||
2929 | 174 | data = provider.provide_data(remote_service, service_ready) | ||
2930 | 175 | else: | ||
2931 | 176 | data = provider.provide_data() | ||
2932 | 177 | if data: | ||
2933 | 178 | hookenv.relation_set(relid, data) | ||
2934 | 141 | 179 | ||
2935 | 142 | def reconfigure_services(self, *service_names): | 180 | def reconfigure_services(self, *service_names): |
2936 | 143 | """ | 181 | """ |
2937 | 144 | 182 | ||
2938 | === modified file 'hooks/charmhelpers/core/services/helpers.py' | |||
2939 | --- hooks/charmhelpers/core/services/helpers.py 2014-10-24 16:35:00 +0000 | |||
2940 | +++ hooks/charmhelpers/core/services/helpers.py 2017-05-11 14:56:57 +0000 | |||
2941 | @@ -1,6 +1,22 @@ | |||
2942 | 1 | # Copyright 2014-2015 Canonical Limited. | ||
2943 | 2 | # | ||
2944 | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); | ||
2945 | 4 | # you may not use this file except in compliance with the License. | ||
2946 | 5 | # You may obtain a copy of the License at | ||
2947 | 6 | # | ||
2948 | 7 | # http://www.apache.org/licenses/LICENSE-2.0 | ||
2949 | 8 | # | ||
2950 | 9 | # Unless required by applicable law or agreed to in writing, software | ||
2951 | 10 | # distributed under the License is distributed on an "AS IS" BASIS, | ||
2952 | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
2953 | 12 | # See the License for the specific language governing permissions and | ||
2954 | 13 | # limitations under the License. | ||
2955 | 14 | |||
2956 | 1 | import os | 15 | import os |
2957 | 2 | import yaml | 16 | import yaml |
2958 | 17 | |||
2959 | 3 | from charmhelpers.core import hookenv | 18 | from charmhelpers.core import hookenv |
2960 | 19 | from charmhelpers.core import host | ||
2961 | 4 | from charmhelpers.core import templating | 20 | from charmhelpers.core import templating |
2962 | 5 | 21 | ||
2963 | 6 | from charmhelpers.core.services.base import ManagerCallback | 22 | from charmhelpers.core.services.base import ManagerCallback |
2964 | @@ -29,12 +45,14 @@ | |||
2965 | 29 | """ | 45 | """ |
2966 | 30 | name = None | 46 | name = None |
2967 | 31 | interface = None | 47 | interface = None |
2968 | 32 | required_keys = [] | ||
2969 | 33 | 48 | ||
2970 | 34 | def __init__(self, name=None, additional_required_keys=None): | 49 | def __init__(self, name=None, additional_required_keys=None): |
2971 | 50 | if not hasattr(self, 'required_keys'): | ||
2972 | 51 | self.required_keys = [] | ||
2973 | 52 | |||
2974 | 35 | if name is not None: | 53 | if name is not None: |
2975 | 36 | self.name = name | 54 | self.name = name |
2977 | 37 | if additional_required_keys is not None: | 55 | if additional_required_keys: |
2978 | 38 | self.required_keys.extend(additional_required_keys) | 56 | self.required_keys.extend(additional_required_keys) |
2979 | 39 | self.get_data() | 57 | self.get_data() |
2980 | 40 | 58 | ||
2981 | @@ -118,7 +136,10 @@ | |||
2982 | 118 | """ | 136 | """ |
2983 | 119 | name = 'db' | 137 | name = 'db' |
2984 | 120 | interface = 'mysql' | 138 | interface = 'mysql' |
2986 | 121 | required_keys = ['host', 'user', 'password', 'database'] | 139 | |
2987 | 140 | def __init__(self, *args, **kwargs): | ||
2988 | 141 | self.required_keys = ['host', 'user', 'password', 'database'] | ||
2989 | 142 | RelationContext.__init__(self, *args, **kwargs) | ||
2990 | 122 | 143 | ||
2991 | 123 | 144 | ||
2992 | 124 | class HttpRelation(RelationContext): | 145 | class HttpRelation(RelationContext): |
2993 | @@ -130,7 +151,10 @@ | |||
2994 | 130 | """ | 151 | """ |
2995 | 131 | name = 'website' | 152 | name = 'website' |
2996 | 132 | interface = 'http' | 153 | interface = 'http' |
2998 | 133 | required_keys = ['host', 'port'] | 154 | |
2999 | 155 | def __init__(self, *args, **kwargs): | ||
3000 | 156 | self.required_keys = ['host', 'port'] | ||
3001 | 157 | RelationContext.__init__(self, *args, **kwargs) | ||
3002 | 134 | 158 | ||
3003 | 135 | def provide_data(self): | 159 | def provide_data(self): |
3004 | 136 | return { | 160 | return { |
3005 | @@ -196,7 +220,7 @@ | |||
3006 | 196 | if not os.path.isabs(file_name): | 220 | if not os.path.isabs(file_name): |
3007 | 197 | file_name = os.path.join(hookenv.charm_dir(), file_name) | 221 | file_name = os.path.join(hookenv.charm_dir(), file_name) |
3008 | 198 | with open(file_name, 'w') as file_stream: | 222 | with open(file_name, 'w') as file_stream: |
3010 | 199 | os.fchmod(file_stream.fileno(), 0600) | 223 | os.fchmod(file_stream.fileno(), 0o600) |
3011 | 200 | yaml.dump(config_data, file_stream) | 224 | yaml.dump(config_data, file_stream) |
3012 | 201 | 225 | ||
3013 | 202 | def read_context(self, file_name): | 226 | def read_context(self, file_name): |
3014 | @@ -211,28 +235,55 @@ | |||
3015 | 211 | 235 | ||
3016 | 212 | class TemplateCallback(ManagerCallback): | 236 | class TemplateCallback(ManagerCallback): |
3017 | 213 | """ | 237 | """ |
3022 | 214 | Callback class that will render a Jinja2 template, for use as a ready action. | 238 | Callback class that will render a Jinja2 template, for use as a ready |
3023 | 215 | 239 | action. | |
3024 | 216 | :param str source: The template source file, relative to `$CHARM_DIR/templates` | 240 | |
3025 | 217 | :param str target: The target to write the rendered template to | 241 | :param str source: The template source file, relative to |
3026 | 242 | `$CHARM_DIR/templates` | ||
3027 | 243 | |||
3028 | 244 | :param str target: The target to write the rendered template to (or None) | ||
3029 | 218 | :param str owner: The owner of the rendered file | 245 | :param str owner: The owner of the rendered file |
3030 | 219 | :param str group: The group of the rendered file | 246 | :param str group: The group of the rendered file |
3031 | 220 | :param int perms: The permissions of the rendered file | 247 | :param int perms: The permissions of the rendered file |
3032 | 248 | :param partial on_change_action: functools partial to be executed when | ||
3033 | 249 | rendered file changes | ||
3034 | 250 | :param jinja2 loader template_loader: A jinja2 template loader | ||
3035 | 251 | |||
3036 | 252 | :return str: The rendered template | ||
3037 | 221 | """ | 253 | """ |
3039 | 222 | def __init__(self, source, target, owner='root', group='root', perms=0444): | 254 | def __init__(self, source, target, |
3040 | 255 | owner='root', group='root', perms=0o444, | ||
3041 | 256 | on_change_action=None, template_loader=None): | ||
3042 | 223 | self.source = source | 257 | self.source = source |
3043 | 224 | self.target = target | 258 | self.target = target |
3044 | 225 | self.owner = owner | 259 | self.owner = owner |
3045 | 226 | self.group = group | 260 | self.group = group |
3046 | 227 | self.perms = perms | 261 | self.perms = perms |
3047 | 262 | self.on_change_action = on_change_action | ||
3048 | 263 | self.template_loader = template_loader | ||
3049 | 228 | 264 | ||
3050 | 229 | def __call__(self, manager, service_name, event_name): | 265 | def __call__(self, manager, service_name, event_name): |
3051 | 266 | pre_checksum = '' | ||
3052 | 267 | if self.on_change_action and os.path.isfile(self.target): | ||
3053 | 268 | pre_checksum = host.file_hash(self.target) | ||
3054 | 230 | service = manager.get_service(service_name) | 269 | service = manager.get_service(service_name) |
3056 | 231 | context = {} | 270 | context = {'ctx': {}} |
3057 | 232 | for ctx in service.get('required_data', []): | 271 | for ctx in service.get('required_data', []): |
3058 | 233 | context.update(ctx) | 272 | context.update(ctx) |
3061 | 234 | templating.render(self.source, self.target, context, | 273 | context['ctx'].update(ctx) |
3062 | 235 | self.owner, self.group, self.perms) | 274 | |
3063 | 275 | result = templating.render(self.source, self.target, context, | ||
3064 | 276 | self.owner, self.group, self.perms, | ||
3065 | 277 | template_loader=self.template_loader) | ||
3066 | 278 | if self.on_change_action: | ||
3067 | 279 | if pre_checksum == host.file_hash(self.target): | ||
3068 | 280 | hookenv.log( | ||
3069 | 281 | 'No change detected: {}'.format(self.target), | ||
3070 | 282 | hookenv.DEBUG) | ||
3071 | 283 | else: | ||
3072 | 284 | self.on_change_action() | ||
3073 | 285 | |||
3074 | 286 | return result | ||
3075 | 236 | 287 | ||
3076 | 237 | 288 | ||
3077 | 238 | # Convenience aliases for templates | 289 | # Convenience aliases for templates |
3078 | 239 | 290 | ||
3079 | === added file 'hooks/charmhelpers/core/strutils.py' | |||
3080 | --- hooks/charmhelpers/core/strutils.py 1970-01-01 00:00:00 +0000 | |||
3081 | +++ hooks/charmhelpers/core/strutils.py 2017-05-11 14:56:57 +0000 | |||
3082 | @@ -0,0 +1,123 @@ | |||
3083 | 1 | #!/usr/bin/env python | ||
3084 | 2 | # -*- coding: utf-8 -*- | ||
3085 | 3 | |||
3086 | 4 | # Copyright 2014-2015 Canonical Limited. | ||
3087 | 5 | # | ||
3088 | 6 | # Licensed under the Apache License, Version 2.0 (the "License"); | ||
3089 | 7 | # you may not use this file except in compliance with the License. | ||
3090 | 8 | # You may obtain a copy of the License at | ||
3091 | 9 | # | ||
3092 | 10 | # http://www.apache.org/licenses/LICENSE-2.0 | ||
3093 | 11 | # | ||
3094 | 12 | # Unless required by applicable law or agreed to in writing, software | ||
3095 | 13 | # distributed under the License is distributed on an "AS IS" BASIS, | ||
3096 | 14 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
3097 | 15 | # See the License for the specific language governing permissions and | ||
3098 | 16 | # limitations under the License. | ||
3099 | 17 | |||
3100 | 18 | import six | ||
3101 | 19 | import re | ||
3102 | 20 | |||
3103 | 21 | |||
3104 | 22 | def bool_from_string(value): | ||
3105 | 23 | """Interpret string value as boolean. | ||
3106 | 24 | |||
3107 | 25 | Returns True if value translates to True otherwise False. | ||
3108 | 26 | """ | ||
3109 | 27 | if isinstance(value, six.string_types): | ||
3110 | 28 | value = six.text_type(value) | ||
3111 | 29 | else: | ||
3112 | 30 | msg = "Unable to interpret non-string value '%s' as boolean" % (value) | ||
3113 | 31 | raise ValueError(msg) | ||
3114 | 32 | |||
3115 | 33 | value = value.strip().lower() | ||
3116 | 34 | |||
3117 | 35 | if value in ['y', 'yes', 'true', 't', 'on']: | ||
3118 | 36 | return True | ||
3119 | 37 | elif value in ['n', 'no', 'false', 'f', 'off']: | ||
3120 | 38 | return False | ||
3121 | 39 | |||
3122 | 40 | msg = "Unable to interpret string value '%s' as boolean" % (value) | ||
3123 | 41 | raise ValueError(msg) | ||
3124 | 42 | |||
3125 | 43 | |||
3126 | 44 | def bytes_from_string(value): | ||
3127 | 45 | """Interpret human readable string value as bytes. | ||
3128 | 46 | |||
3129 | 47 | Returns int | ||
3130 | 48 | """ | ||
3131 | 49 | BYTE_POWER = { | ||
3132 | 50 | 'K': 1, | ||
3133 | 51 | 'KB': 1, | ||
3134 | 52 | 'M': 2, | ||
3135 | 53 | 'MB': 2, | ||
3136 | 54 | 'G': 3, | ||
3137 | 55 | 'GB': 3, | ||
3138 | 56 | 'T': 4, | ||
3139 | 57 | 'TB': 4, | ||
3140 | 58 | 'P': 5, | ||
3141 | 59 | 'PB': 5, | ||
3142 | 60 | } | ||
3143 | 61 | if isinstance(value, six.string_types): | ||
3144 | 62 | value = six.text_type(value) | ||
3145 | 63 | else: | ||
3146 | 64 | msg = "Unable to interpret non-string value '%s' as boolean" % (value) | ||
3147 | 65 | raise ValueError(msg) | ||
3148 | 66 | matches = re.match("([0-9]+)([a-zA-Z]+)", value) | ||
3149 | 67 | if not matches: | ||
3150 | 68 | msg = "Unable to interpret string value '%s' as bytes" % (value) | ||
3151 | 69 | raise ValueError(msg) | ||
3152 | 70 | return int(matches.group(1)) * (1024 ** BYTE_POWER[matches.group(2)]) | ||
3153 | 71 | |||
3154 | 72 | |||
3155 | 73 | class BasicStringComparator(object): | ||
3156 | 74 | """Provides a class that will compare strings from an iterator type object. | ||
3157 | 75 | Used to provide > and < comparisons on strings that may not necessarily be | ||
3158 | 76 | alphanumerically ordered. e.g. OpenStack or Ubuntu releases AFTER the | ||
3159 | 77 | z-wrap. | ||
3160 | 78 | """ | ||
3161 | 79 | |||
3162 | 80 | _list = None | ||
3163 | 81 | |||
3164 | 82 | def __init__(self, item): | ||
3165 | 83 | if self._list is None: | ||
3166 | 84 | raise Exception("Must define the _list in the class definition!") | ||
3167 | 85 | try: | ||
3168 | 86 | self.index = self._list.index(item) | ||
3169 | 87 | except Exception: | ||
3170 | 88 | raise KeyError("Item '{}' is not in list '{}'" | ||
3171 | 89 | .format(item, self._list)) | ||
3172 | 90 | |||
3173 | 91 | def __eq__(self, other): | ||
3174 | 92 | assert isinstance(other, str) or isinstance(other, self.__class__) | ||
3175 | 93 | return self.index == self._list.index(other) | ||
3176 | 94 | |||
3177 | 95 | def __ne__(self, other): | ||
3178 | 96 | return not self.__eq__(other) | ||
3179 | 97 | |||
3180 | 98 | def __lt__(self, other): | ||
3181 | 99 | assert isinstance(other, str) or isinstance(other, self.__class__) | ||
3182 | 100 | return self.index < self._list.index(other) | ||
3183 | 101 | |||
3184 | 102 | def __ge__(self, other): | ||
3185 | 103 | return not self.__lt__(other) | ||
3186 | 104 | |||
3187 | 105 | def __gt__(self, other): | ||
3188 | 106 | assert isinstance(other, str) or isinstance(other, self.__class__) | ||
3189 | 107 | return self.index > self._list.index(other) | ||
3190 | 108 | |||
3191 | 109 | def __le__(self, other): | ||
3192 | 110 | return not self.__gt__(other) | ||
3193 | 111 | |||
3194 | 112 | def __str__(self): | ||
3195 | 113 | """Always give back the item at the index so it can be used in | ||
3196 | 114 | comparisons like: | ||
3197 | 115 | |||
3198 | 116 | s_mitaka = CompareOpenStack('mitaka') | ||
3199 | 117 | s_newton = CompareOpenstack('newton') | ||
3200 | 118 | |||
3201 | 119 | assert s_newton > s_mitaka | ||
3202 | 120 | |||
3203 | 121 | @returns: <string> | ||
3204 | 122 | """ | ||
3205 | 123 | return self._list[self.index] | ||
3206 | 0 | 124 | ||
3207 | === modified file 'hooks/charmhelpers/core/sysctl.py' | |||
3208 | --- hooks/charmhelpers/core/sysctl.py 2014-10-24 16:35:00 +0000 | |||
3209 | +++ hooks/charmhelpers/core/sysctl.py 2017-05-11 14:56:57 +0000 | |||
3210 | @@ -1,7 +1,19 @@ | |||
3211 | 1 | #!/usr/bin/env python | 1 | #!/usr/bin/env python |
3212 | 2 | # -*- coding: utf-8 -*- | 2 | # -*- coding: utf-8 -*- |
3213 | 3 | 3 | ||
3215 | 4 | __author__ = 'Jorge Niedbalski R. <jorge.niedbalski@canonical.com>' | 4 | # Copyright 2014-2015 Canonical Limited. |
3216 | 5 | # | ||
3217 | 6 | # Licensed under the Apache License, Version 2.0 (the "License"); | ||
3218 | 7 | # you may not use this file except in compliance with the License. | ||
3219 | 8 | # You may obtain a copy of the License at | ||
3220 | 9 | # | ||
3221 | 10 | # http://www.apache.org/licenses/LICENSE-2.0 | ||
3222 | 11 | # | ||
3223 | 12 | # Unless required by applicable law or agreed to in writing, software | ||
3224 | 13 | # distributed under the License is distributed on an "AS IS" BASIS, | ||
3225 | 14 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
3226 | 15 | # See the License for the specific language governing permissions and | ||
3227 | 16 | # limitations under the License. | ||
3228 | 5 | 17 | ||
3229 | 6 | import yaml | 18 | import yaml |
3230 | 7 | 19 | ||
3231 | @@ -10,25 +22,33 @@ | |||
3232 | 10 | from charmhelpers.core.hookenv import ( | 22 | from charmhelpers.core.hookenv import ( |
3233 | 11 | log, | 23 | log, |
3234 | 12 | DEBUG, | 24 | DEBUG, |
3235 | 25 | ERROR, | ||
3236 | 13 | ) | 26 | ) |
3237 | 14 | 27 | ||
3238 | 28 | __author__ = 'Jorge Niedbalski R. <jorge.niedbalski@canonical.com>' | ||
3239 | 29 | |||
3240 | 15 | 30 | ||
3241 | 16 | def create(sysctl_dict, sysctl_file): | 31 | def create(sysctl_dict, sysctl_file): |
3242 | 17 | """Creates a sysctl.conf file from a YAML associative array | 32 | """Creates a sysctl.conf file from a YAML associative array |
3243 | 18 | 33 | ||
3246 | 19 | :param sysctl_dict: a dict of sysctl options eg { 'kernel.max_pid': 1337 } | 34 | :param sysctl_dict: a YAML-formatted string of sysctl options eg "{ 'kernel.max_pid': 1337 }" |
3247 | 20 | :type sysctl_dict: dict | 35 | :type sysctl_dict: str |
3248 | 21 | :param sysctl_file: path to the sysctl file to be saved | 36 | :param sysctl_file: path to the sysctl file to be saved |
3249 | 22 | :type sysctl_file: str or unicode | 37 | :type sysctl_file: str or unicode |
3250 | 23 | :returns: None | 38 | :returns: None |
3251 | 24 | """ | 39 | """ |
3253 | 25 | sysctl_dict = yaml.load(sysctl_dict) | 40 | try: |
3254 | 41 | sysctl_dict_parsed = yaml.safe_load(sysctl_dict) | ||
3255 | 42 | except yaml.YAMLError: | ||
3256 | 43 | log("Error parsing YAML sysctl_dict: {}".format(sysctl_dict), | ||
3257 | 44 | level=ERROR) | ||
3258 | 45 | return | ||
3259 | 26 | 46 | ||
3260 | 27 | with open(sysctl_file, "w") as fd: | 47 | with open(sysctl_file, "w") as fd: |
3262 | 28 | for key, value in sysctl_dict.items(): | 48 | for key, value in sysctl_dict_parsed.items(): |
3263 | 29 | fd.write("{}={}\n".format(key, value)) | 49 | fd.write("{}={}\n".format(key, value)) |
3264 | 30 | 50 | ||
3266 | 31 | log("Updating sysctl_file: %s values: %s" % (sysctl_file, sysctl_dict), | 51 | log("Updating sysctl_file: %s values: %s" % (sysctl_file, sysctl_dict_parsed), |
3267 | 32 | level=DEBUG) | 52 | level=DEBUG) |
3268 | 33 | 53 | ||
3269 | 34 | check_call(["sysctl", "-p", sysctl_file]) | 54 | check_call(["sysctl", "-p", sysctl_file]) |
3270 | 35 | 55 | ||
3271 | === modified file 'hooks/charmhelpers/core/templating.py' | |||
3272 | --- hooks/charmhelpers/core/templating.py 2014-10-24 16:35:00 +0000 | |||
3273 | +++ hooks/charmhelpers/core/templating.py 2017-05-11 14:56:57 +0000 | |||
3274 | @@ -1,16 +1,33 @@ | |||
3275 | 1 | # Copyright 2014-2015 Canonical Limited. | ||
3276 | 2 | # | ||
3277 | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); | ||
3278 | 4 | # you may not use this file except in compliance with the License. | ||
3279 | 5 | # You may obtain a copy of the License at | ||
3280 | 6 | # | ||
3281 | 7 | # http://www.apache.org/licenses/LICENSE-2.0 | ||
3282 | 8 | # | ||
3283 | 9 | # Unless required by applicable law or agreed to in writing, software | ||
3284 | 10 | # distributed under the License is distributed on an "AS IS" BASIS, | ||
3285 | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
3286 | 12 | # See the License for the specific language governing permissions and | ||
3287 | 13 | # limitations under the License. | ||
3288 | 14 | |||
3289 | 1 | import os | 15 | import os |
3290 | 16 | import sys | ||
3291 | 2 | 17 | ||
3292 | 3 | from charmhelpers.core import host | 18 | from charmhelpers.core import host |
3293 | 4 | from charmhelpers.core import hookenv | 19 | from charmhelpers.core import hookenv |
3294 | 5 | 20 | ||
3295 | 6 | 21 | ||
3297 | 7 | def render(source, target, context, owner='root', group='root', perms=0444, templates_dir=None): | 22 | def render(source, target, context, owner='root', group='root', |
3298 | 23 | perms=0o444, templates_dir=None, encoding='UTF-8', template_loader=None): | ||
3299 | 8 | """ | 24 | """ |
3300 | 9 | Render a template. | 25 | Render a template. |
3301 | 10 | 26 | ||
3302 | 11 | The `source` path, if not absolute, is relative to the `templates_dir`. | 27 | The `source` path, if not absolute, is relative to the `templates_dir`. |
3303 | 12 | 28 | ||
3305 | 13 | The `target` path should be absolute. | 29 | The `target` path should be absolute. It can also be `None`, in which |
3306 | 30 | case no file will be written. | ||
3307 | 14 | 31 | ||
3308 | 15 | The context should be a dict containing the values to be replaced in the | 32 | The context should be a dict containing the values to be replaced in the |
3309 | 16 | template. | 33 | template. |
3310 | @@ -19,8 +36,12 @@ | |||
3311 | 19 | 36 | ||
3312 | 20 | If omitted, `templates_dir` defaults to the `templates` folder in the charm. | 37 | If omitted, `templates_dir` defaults to the `templates` folder in the charm. |
3313 | 21 | 38 | ||
3316 | 22 | Note: Using this requires python-jinja2; if it is not installed, calling | 39 | The rendered template will be written to the file as well as being returned |
3317 | 23 | this will attempt to use charmhelpers.fetch.apt_install to install it. | 40 | as a string. |
3318 | 41 | |||
3319 | 42 | Note: Using this requires python-jinja2 or python3-jinja2; if it is not | ||
3320 | 43 | installed, calling this will attempt to use charmhelpers.fetch.apt_install | ||
3321 | 44 | to install it. | ||
3322 | 24 | """ | 45 | """ |
3323 | 25 | try: | 46 | try: |
3324 | 26 | from jinja2 import FileSystemLoader, Environment, exceptions | 47 | from jinja2 import FileSystemLoader, Environment, exceptions |
3325 | @@ -32,20 +53,32 @@ | |||
3326 | 32 | 'charmhelpers.fetch to install it', | 53 | 'charmhelpers.fetch to install it', |
3327 | 33 | level=hookenv.ERROR) | 54 | level=hookenv.ERROR) |
3328 | 34 | raise | 55 | raise |
3330 | 35 | apt_install('python-jinja2', fatal=True) | 56 | if sys.version_info.major == 2: |
3331 | 57 | apt_install('python-jinja2', fatal=True) | ||
3332 | 58 | else: | ||
3333 | 59 | apt_install('python3-jinja2', fatal=True) | ||
3334 | 36 | from jinja2 import FileSystemLoader, Environment, exceptions | 60 | from jinja2 import FileSystemLoader, Environment, exceptions |
3335 | 37 | 61 | ||
3339 | 38 | if templates_dir is None: | 62 | if template_loader: |
3340 | 39 | templates_dir = os.path.join(hookenv.charm_dir(), 'templates') | 63 | template_env = Environment(loader=template_loader) |
3341 | 40 | loader = Environment(loader=FileSystemLoader(templates_dir)) | 64 | else: |
3342 | 65 | if templates_dir is None: | ||
3343 | 66 | templates_dir = os.path.join(hookenv.charm_dir(), 'templates') | ||
3344 | 67 | template_env = Environment(loader=FileSystemLoader(templates_dir)) | ||
3345 | 41 | try: | 68 | try: |
3346 | 42 | source = source | 69 | source = source |
3348 | 43 | template = loader.get_template(source) | 70 | template = template_env.get_template(source) |
3349 | 44 | except exceptions.TemplateNotFound as e: | 71 | except exceptions.TemplateNotFound as e: |
3350 | 45 | hookenv.log('Could not load template %s from %s.' % | 72 | hookenv.log('Could not load template %s from %s.' % |
3351 | 46 | (source, templates_dir), | 73 | (source, templates_dir), |
3352 | 47 | level=hookenv.ERROR) | 74 | level=hookenv.ERROR) |
3353 | 48 | raise e | 75 | raise e |
3354 | 49 | content = template.render(context) | 76 | content = template.render(context) |
3357 | 50 | host.mkdir(os.path.dirname(target)) | 77 | if target is not None: |
3358 | 51 | host.write_file(target, content, owner, group, perms) | 78 | target_dir = os.path.dirname(target) |
3359 | 79 | if not os.path.exists(target_dir): | ||
3360 | 80 | # This is a terrible default directory permission, as the file | ||
3361 | 81 | # or its siblings will often contain secrets. | ||
3362 | 82 | host.mkdir(os.path.dirname(target), owner, group, perms=0o755) | ||
3363 | 83 | host.write_file(target, content.encode(encoding), owner, group, perms) | ||
3364 | 84 | return content | ||
3365 | 52 | 85 | ||
3366 | === added file 'hooks/charmhelpers/core/unitdata.py' | |||
3367 | --- hooks/charmhelpers/core/unitdata.py 1970-01-01 00:00:00 +0000 | |||
3368 | +++ hooks/charmhelpers/core/unitdata.py 2017-05-11 14:56:57 +0000 | |||
3369 | @@ -0,0 +1,518 @@ | |||
3370 | 1 | #!/usr/bin/env python | ||
3371 | 2 | # -*- coding: utf-8 -*- | ||
3372 | 3 | # | ||
3373 | 4 | # Copyright 2014-2015 Canonical Limited. | ||
3374 | 5 | # | ||
3375 | 6 | # Licensed under the Apache License, Version 2.0 (the "License"); | ||
3376 | 7 | # you may not use this file except in compliance with the License. | ||
3377 | 8 | # You may obtain a copy of the License at | ||
3378 | 9 | # | ||
3379 | 10 | # http://www.apache.org/licenses/LICENSE-2.0 | ||
3380 | 11 | # | ||
3381 | 12 | # Unless required by applicable law or agreed to in writing, software | ||
3382 | 13 | # distributed under the License is distributed on an "AS IS" BASIS, | ||
3383 | 14 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
3384 | 15 | # See the License for the specific language governing permissions and | ||
3385 | 16 | # limitations under the License. | ||
3386 | 17 | # | ||
3387 | 18 | # Authors: | ||
3388 | 19 | # Kapil Thangavelu <kapil.foss@gmail.com> | ||
3389 | 20 | # | ||
3390 | 21 | """ | ||
3391 | 22 | Intro | ||
3392 | 23 | ----- | ||
3393 | 24 | |||
3394 | 25 | A simple way to store state in units. This provides a key value | ||
3395 | 26 | storage with support for versioned, transactional operation, | ||
3396 | 27 | and can calculate deltas from previous values to simplify unit logic | ||
3397 | 28 | when processing changes. | ||
3398 | 29 | |||
3399 | 30 | |||
3400 | 31 | Hook Integration | ||
3401 | 32 | ---------------- | ||
3402 | 33 | |||
3403 | 34 | There are several extant frameworks for hook execution, including | ||
3404 | 35 | |||
3405 | 36 | - charmhelpers.core.hookenv.Hooks | ||
3406 | 37 | - charmhelpers.core.services.ServiceManager | ||
3407 | 38 | |||
3408 | 39 | The storage classes are framework agnostic, one simple integration is | ||
3409 | 40 | via the HookData contextmanager. It will record the current hook | ||
3410 | 41 | execution environment (including relation data, config data, etc.), | ||
3411 | 42 | setup a transaction and allow easy access to the changes from | ||
3412 | 43 | previously seen values. One consequence of the integration is the | ||
3413 | 44 | reservation of particular keys ('rels', 'unit', 'env', 'config', | ||
3414 | 45 | 'charm_revisions') for their respective values. | ||
3415 | 46 | |||
3416 | 47 | Here's a fully worked integration example using hookenv.Hooks:: | ||
3417 | 48 | |||
3418 | 49 | from charmhelper.core import hookenv, unitdata | ||
3419 | 50 | |||
3420 | 51 | hook_data = unitdata.HookData() | ||
3421 | 52 | db = unitdata.kv() | ||
3422 | 53 | hooks = hookenv.Hooks() | ||
3423 | 54 | |||
3424 | 55 | @hooks.hook | ||
3425 | 56 | def config_changed(): | ||
3426 | 57 | # Print all changes to configuration from previously seen | ||
3427 | 58 | # values. | ||
3428 | 59 | for changed, (prev, cur) in hook_data.conf.items(): | ||
3429 | 60 | print('config changed', changed, | ||
3430 | 61 | 'previous value', prev, | ||
3431 | 62 | 'current value', cur) | ||
3432 | 63 | |||
3433 | 64 | # Get some unit specific bookeeping | ||
3434 | 65 | if not db.get('pkg_key'): | ||
3435 | 66 | key = urllib.urlopen('https://example.com/pkg_key').read() | ||
3436 | 67 | db.set('pkg_key', key) | ||
3437 | 68 | |||
3438 | 69 | # Directly access all charm config as a mapping. | ||
3439 | 70 | conf = db.getrange('config', True) | ||
3440 | 71 | |||
3441 | 72 | # Directly access all relation data as a mapping | ||
3442 | 73 | rels = db.getrange('rels', True) | ||
3443 | 74 | |||
3444 | 75 | if __name__ == '__main__': | ||
3445 | 76 | with hook_data(): | ||
3446 | 77 | hook.execute() | ||
3447 | 78 | |||
3448 | 79 | |||
3449 | 80 | A more basic integration is via the hook_scope context manager which simply | ||
3450 | 81 | manages transaction scope (and records hook name, and timestamp):: | ||
3451 | 82 | |||
3452 | 83 | >>> from unitdata import kv | ||
3453 | 84 | >>> db = kv() | ||
3454 | 85 | >>> with db.hook_scope('install'): | ||
3455 | 86 | ... # do work, in transactional scope. | ||
3456 | 87 | ... db.set('x', 1) | ||
3457 | 88 | >>> db.get('x') | ||
3458 | 89 | 1 | ||
3459 | 90 | |||
3460 | 91 | |||
3461 | 92 | Usage | ||
3462 | 93 | ----- | ||
3463 | 94 | |||
3464 | 95 | Values are automatically json de/serialized to preserve basic typing | ||
3465 | 96 | and complex data struct capabilities (dicts, lists, ints, booleans, etc). | ||
3466 | 97 | |||
3467 | 98 | Individual values can be manipulated via get/set:: | ||
3468 | 99 | |||
3469 | 100 | >>> kv.set('y', True) | ||
3470 | 101 | >>> kv.get('y') | ||
3471 | 102 | True | ||
3472 | 103 | |||
3473 | 104 | # We can set complex values (dicts, lists) as a single key. | ||
3474 | 105 | >>> kv.set('config', {'a': 1, 'b': True'}) | ||
3475 | 106 | |||
3476 | 107 | # Also supports returning dictionaries as a record which | ||
3477 | 108 | # provides attribute access. | ||
3478 | 109 | >>> config = kv.get('config', record=True) | ||
3479 | 110 | >>> config.b | ||
3480 | 111 | True | ||
3481 | 112 | |||
3482 | 113 | |||
3483 | 114 | Groups of keys can be manipulated with update/getrange:: | ||
3484 | 115 | |||
3485 | 116 | >>> kv.update({'z': 1, 'y': 2}, prefix="gui.") | ||
3486 | 117 | >>> kv.getrange('gui.', strip=True) | ||
3487 | 118 | {'z': 1, 'y': 2} | ||
3488 | 119 | |||
3489 | 120 | When updating values, its very helpful to understand which values | ||
3490 | 121 | have actually changed and how have they changed. The storage | ||
3491 | 122 | provides a delta method to provide for this:: | ||
3492 | 123 | |||
3493 | 124 | >>> data = {'debug': True, 'option': 2} | ||
3494 | 125 | >>> delta = kv.delta(data, 'config.') | ||
3495 | 126 | >>> delta.debug.previous | ||
3496 | 127 | None | ||
3497 | 128 | >>> delta.debug.current | ||
3498 | 129 | True | ||
3499 | 130 | >>> delta | ||
3500 | 131 | {'debug': (None, True), 'option': (None, 2)} | ||
3501 | 132 | |||
3502 | 133 | Note the delta method does not persist the actual change, it needs to | ||
3503 | 134 | be explicitly saved via 'update' method:: | ||
3504 | 135 | |||
3505 | 136 | >>> kv.update(data, 'config.') | ||
3506 | 137 | |||
3507 | 138 | Values modified in the context of a hook scope retain historical values | ||
3508 | 139 | associated to the hookname. | ||
3509 | 140 | |||
3510 | 141 | >>> with db.hook_scope('config-changed'): | ||
3511 | 142 | ... db.set('x', 42) | ||
3512 | 143 | >>> db.gethistory('x') | ||
3513 | 144 | [(1, u'x', 1, u'install', u'2015-01-21T16:49:30.038372'), | ||
3514 | 145 | (2, u'x', 42, u'config-changed', u'2015-01-21T16:49:30.038786')] | ||
3515 | 146 | |||
3516 | 147 | """ | ||
3517 | 148 | |||
3518 | 149 | import collections | ||
3519 | 150 | import contextlib | ||
3520 | 151 | import datetime | ||
3521 | 152 | import itertools | ||
3522 | 153 | import json | ||
3523 | 154 | import os | ||
3524 | 155 | import pprint | ||
3525 | 156 | import sqlite3 | ||
3526 | 157 | import sys | ||
3527 | 158 | |||
3528 | 159 | __author__ = 'Kapil Thangavelu <kapil.foss@gmail.com>' | ||
3529 | 160 | |||
3530 | 161 | |||
3531 | 162 | class Storage(object): | ||
3532 | 163 | """Simple key value database for local unit state within charms. | ||
3533 | 164 | |||
3534 | 165 | Modifications are not persisted unless :meth:`flush` is called. | ||
3535 | 166 | |||
3536 | 167 | To support dicts, lists, integer, floats, and booleans values | ||
3537 | 168 | are automatically json encoded/decoded. | ||
3538 | 169 | """ | ||
3539 | 170 | def __init__(self, path=None): | ||
3540 | 171 | self.db_path = path | ||
3541 | 172 | if path is None: | ||
3542 | 173 | if 'UNIT_STATE_DB' in os.environ: | ||
3543 | 174 | self.db_path = os.environ['UNIT_STATE_DB'] | ||
3544 | 175 | else: | ||
3545 | 176 | self.db_path = os.path.join( | ||
3546 | 177 | os.environ.get('CHARM_DIR', ''), '.unit-state.db') | ||
3547 | 178 | self.conn = sqlite3.connect('%s' % self.db_path) | ||
3548 | 179 | self.cursor = self.conn.cursor() | ||
3549 | 180 | self.revision = None | ||
3550 | 181 | self._closed = False | ||
3551 | 182 | self._init() | ||
3552 | 183 | |||
3553 | 184 | def close(self): | ||
3554 | 185 | if self._closed: | ||
3555 | 186 | return | ||
3556 | 187 | self.flush(False) | ||
3557 | 188 | self.cursor.close() | ||
3558 | 189 | self.conn.close() | ||
3559 | 190 | self._closed = True | ||
3560 | 191 | |||
3561 | 192 | def get(self, key, default=None, record=False): | ||
3562 | 193 | self.cursor.execute('select data from kv where key=?', [key]) | ||
3563 | 194 | result = self.cursor.fetchone() | ||
3564 | 195 | if not result: | ||
3565 | 196 | return default | ||
3566 | 197 | if record: | ||
3567 | 198 | return Record(json.loads(result[0])) | ||
3568 | 199 | return json.loads(result[0]) | ||
3569 | 200 | |||
3570 | 201 | def getrange(self, key_prefix, strip=False): | ||
3571 | 202 | """ | ||
3572 | 203 | Get a range of keys starting with a common prefix as a mapping of | ||
3573 | 204 | keys to values. | ||
3574 | 205 | |||
3575 | 206 | :param str key_prefix: Common prefix among all keys | ||
3576 | 207 | :param bool strip: Optionally strip the common prefix from the key | ||
3577 | 208 | names in the returned dict | ||
3578 | 209 | :return dict: A (possibly empty) dict of key-value mappings | ||
3579 | 210 | """ | ||
3580 | 211 | self.cursor.execute("select key, data from kv where key like ?", | ||
3581 | 212 | ['%s%%' % key_prefix]) | ||
3582 | 213 | result = self.cursor.fetchall() | ||
3583 | 214 | |||
3584 | 215 | if not result: | ||
3585 | 216 | return {} | ||
3586 | 217 | if not strip: | ||
3587 | 218 | key_prefix = '' | ||
3588 | 219 | return dict([ | ||
3589 | 220 | (k[len(key_prefix):], json.loads(v)) for k, v in result]) | ||
3590 | 221 | |||
3591 | 222 | def update(self, mapping, prefix=""): | ||
3592 | 223 | """ | ||
3593 | 224 | Set the values of multiple keys at once. | ||
3594 | 225 | |||
3595 | 226 | :param dict mapping: Mapping of keys to values | ||
3596 | 227 | :param str prefix: Optional prefix to apply to all keys in `mapping` | ||
3597 | 228 | before setting | ||
3598 | 229 | """ | ||
3599 | 230 | for k, v in mapping.items(): | ||
3600 | 231 | self.set("%s%s" % (prefix, k), v) | ||
3601 | 232 | |||
3602 | 233 | def unset(self, key): | ||
3603 | 234 | """ | ||
3604 | 235 | Remove a key from the database entirely. | ||
3605 | 236 | """ | ||
3606 | 237 | self.cursor.execute('delete from kv where key=?', [key]) | ||
3607 | 238 | if self.revision and self.cursor.rowcount: | ||
3608 | 239 | self.cursor.execute( | ||
3609 | 240 | 'insert into kv_revisions values (?, ?, ?)', | ||
3610 | 241 | [key, self.revision, json.dumps('DELETED')]) | ||
3611 | 242 | |||
3612 | 243 | def unsetrange(self, keys=None, prefix=""): | ||
3613 | 244 | """ | ||
3614 | 245 | Remove a range of keys starting with a common prefix, from the database | ||
3615 | 246 | entirely. | ||
3616 | 247 | |||
3617 | 248 | :param list keys: List of keys to remove. | ||
3618 | 249 | :param str prefix: Optional prefix to apply to all keys in ``keys`` | ||
3619 | 250 | before removing. | ||
3620 | 251 | """ | ||
3621 | 252 | if keys is not None: | ||
3622 | 253 | keys = ['%s%s' % (prefix, key) for key in keys] | ||
3623 | 254 | self.cursor.execute('delete from kv where key in (%s)' % ','.join(['?'] * len(keys)), keys) | ||
3624 | 255 | if self.revision and self.cursor.rowcount: | ||
3625 | 256 | self.cursor.execute( | ||
3626 | 257 | 'insert into kv_revisions values %s' % ','.join(['(?, ?, ?)'] * len(keys)), | ||
3627 | 258 | list(itertools.chain.from_iterable((key, self.revision, json.dumps('DELETED')) for key in keys))) | ||
3628 | 259 | else: | ||
3629 | 260 | self.cursor.execute('delete from kv where key like ?', | ||
3630 | 261 | ['%s%%' % prefix]) | ||
3631 | 262 | if self.revision and self.cursor.rowcount: | ||
3632 | 263 | self.cursor.execute( | ||
3633 | 264 | 'insert into kv_revisions values (?, ?, ?)', | ||
3634 | 265 | ['%s%%' % prefix, self.revision, json.dumps('DELETED')]) | ||
3635 | 266 | |||
3636 | 267 | def set(self, key, value): | ||
3637 | 268 | """ | ||
3638 | 269 | Set a value in the database. | ||
3639 | 270 | |||
3640 | 271 | :param str key: Key to set the value for | ||
3641 | 272 | :param value: Any JSON-serializable value to be set | ||
3642 | 273 | """ | ||
3643 | 274 | serialized = json.dumps(value) | ||
3644 | 275 | |||
3645 | 276 | self.cursor.execute('select data from kv where key=?', [key]) | ||
3646 | 277 | exists = self.cursor.fetchone() | ||
3647 | 278 | |||
3648 | 279 | # Skip mutations to the same value | ||
3649 | 280 | if exists: | ||
3650 | 281 | if exists[0] == serialized: | ||
3651 | 282 | return value | ||
3652 | 283 | |||
3653 | 284 | if not exists: | ||
3654 | 285 | self.cursor.execute( | ||
3655 | 286 | 'insert into kv (key, data) values (?, ?)', | ||
3656 | 287 | (key, serialized)) | ||
3657 | 288 | else: | ||
3658 | 289 | self.cursor.execute(''' | ||
3659 | 290 | update kv | ||
3660 | 291 | set data = ? | ||
3661 | 292 | where key = ?''', [serialized, key]) | ||
3662 | 293 | |||
3663 | 294 | # Save | ||
3664 | 295 | if not self.revision: | ||
3665 | 296 | return value | ||
3666 | 297 | |||
3667 | 298 | self.cursor.execute( | ||
3668 | 299 | 'select 1 from kv_revisions where key=? and revision=?', | ||
3669 | 300 | [key, self.revision]) | ||
3670 | 301 | exists = self.cursor.fetchone() | ||
3671 | 302 | |||
3672 | 303 | if not exists: | ||
3673 | 304 | self.cursor.execute( | ||
3674 | 305 | '''insert into kv_revisions ( | ||
3675 | 306 | revision, key, data) values (?, ?, ?)''', | ||
3676 | 307 | (self.revision, key, serialized)) | ||
3677 | 308 | else: | ||
3678 | 309 | self.cursor.execute( | ||
3679 | 310 | ''' | ||
3680 | 311 | update kv_revisions | ||
3681 | 312 | set data = ? | ||
3682 | 313 | where key = ? | ||
3683 | 314 | and revision = ?''', | ||
3684 | 315 | [serialized, key, self.revision]) | ||
3685 | 316 | |||
3686 | 317 | return value | ||
3687 | 318 | |||
3688 | 319 | def delta(self, mapping, prefix): | ||
3689 | 320 | """ | ||
3690 | 321 | return a delta containing values that have changed. | ||
3691 | 322 | """ | ||
3692 | 323 | previous = self.getrange(prefix, strip=True) | ||
3693 | 324 | if not previous: | ||
3694 | 325 | pk = set() | ||
3695 | 326 | else: | ||
3696 | 327 | pk = set(previous.keys()) | ||
3697 | 328 | ck = set(mapping.keys()) | ||
3698 | 329 | delta = DeltaSet() | ||
3699 | 330 | |||
3700 | 331 | # added | ||
3701 | 332 | for k in ck.difference(pk): | ||
3702 | 333 | delta[k] = Delta(None, mapping[k]) | ||
3703 | 334 | |||
3704 | 335 | # removed | ||
3705 | 336 | for k in pk.difference(ck): | ||
3706 | 337 | delta[k] = Delta(previous[k], None) | ||
3707 | 338 | |||
3708 | 339 | # changed | ||
3709 | 340 | for k in pk.intersection(ck): | ||
3710 | 341 | c = mapping[k] | ||
3711 | 342 | p = previous[k] | ||
3712 | 343 | if c != p: | ||
3713 | 344 | delta[k] = Delta(p, c) | ||
3714 | 345 | |||
3715 | 346 | return delta | ||
3716 | 347 | |||
3717 | 348 | @contextlib.contextmanager | ||
3718 | 349 | def hook_scope(self, name=""): | ||
3719 | 350 | """Scope all future interactions to the current hook execution | ||
3720 | 351 | revision.""" | ||
3721 | 352 | assert not self.revision | ||
3722 | 353 | self.cursor.execute( | ||
3723 | 354 | 'insert into hooks (hook, date) values (?, ?)', | ||
3724 | 355 | (name or sys.argv[0], | ||
3725 | 356 | datetime.datetime.utcnow().isoformat())) | ||
3726 | 357 | self.revision = self.cursor.lastrowid | ||
3727 | 358 | try: | ||
3728 | 359 | yield self.revision | ||
3729 | 360 | self.revision = None | ||
3730 | 361 | except: | ||
3731 | 362 | self.flush(False) | ||
3732 | 363 | self.revision = None | ||
3733 | 364 | raise | ||
3734 | 365 | else: | ||
3735 | 366 | self.flush() | ||
3736 | 367 | |||
3737 | 368 | def flush(self, save=True): | ||
3738 | 369 | if save: | ||
3739 | 370 | self.conn.commit() | ||
3740 | 371 | elif self._closed: | ||
3741 | 372 | return | ||
3742 | 373 | else: | ||
3743 | 374 | self.conn.rollback() | ||
3744 | 375 | |||
3745 | 376 | def _init(self): | ||
3746 | 377 | self.cursor.execute(''' | ||
3747 | 378 | create table if not exists kv ( | ||
3748 | 379 | key text, | ||
3749 | 380 | data text, | ||
3750 | 381 | primary key (key) | ||
3751 | 382 | )''') | ||
3752 | 383 | self.cursor.execute(''' | ||
3753 | 384 | create table if not exists kv_revisions ( | ||
3754 | 385 | key text, | ||
3755 | 386 | revision integer, | ||
3756 | 387 | data text, | ||
3757 | 388 | primary key (key, revision) | ||
3758 | 389 | )''') | ||
3759 | 390 | self.cursor.execute(''' | ||
3760 | 391 | create table if not exists hooks ( | ||
3761 | 392 | version integer primary key autoincrement, | ||
3762 | 393 | hook text, | ||
3763 | 394 | date text | ||
3764 | 395 | )''') | ||
3765 | 396 | self.conn.commit() | ||
3766 | 397 | |||
3767 | 398 | def gethistory(self, key, deserialize=False): | ||
3768 | 399 | self.cursor.execute( | ||
3769 | 400 | ''' | ||
3770 | 401 | select kv.revision, kv.key, kv.data, h.hook, h.date | ||
3771 | 402 | from kv_revisions kv, | ||
3772 | 403 | hooks h | ||
3773 | 404 | where kv.key=? | ||
3774 | 405 | and kv.revision = h.version | ||
3775 | 406 | ''', [key]) | ||
3776 | 407 | if deserialize is False: | ||
3777 | 408 | return self.cursor.fetchall() | ||
3778 | 409 | return map(_parse_history, self.cursor.fetchall()) | ||
3779 | 410 | |||
3780 | 411 | def debug(self, fh=sys.stderr): | ||
3781 | 412 | self.cursor.execute('select * from kv') | ||
3782 | 413 | pprint.pprint(self.cursor.fetchall(), stream=fh) | ||
3783 | 414 | self.cursor.execute('select * from kv_revisions') | ||
3784 | 415 | pprint.pprint(self.cursor.fetchall(), stream=fh) | ||
3785 | 416 | |||
3786 | 417 | |||
3787 | 418 | def _parse_history(d): | ||
3788 | 419 | return (d[0], d[1], json.loads(d[2]), d[3], | ||
3789 | 420 | datetime.datetime.strptime(d[-1], "%Y-%m-%dT%H:%M:%S.%f")) | ||
3790 | 421 | |||
3791 | 422 | |||
3792 | 423 | class HookData(object): | ||
3793 | 424 | """Simple integration for existing hook exec frameworks. | ||
3794 | 425 | |||
3795 | 426 | Records all unit information, and stores deltas for processing | ||
3796 | 427 | by the hook. | ||
3797 | 428 | |||
3798 | 429 | Sample:: | ||
3799 | 430 | |||
3800 | 431 | from charmhelper.core import hookenv, unitdata | ||
3801 | 432 | |||
3802 | 433 | changes = unitdata.HookData() | ||
3803 | 434 | db = unitdata.kv() | ||
3804 | 435 | hooks = hookenv.Hooks() | ||
3805 | 436 | |||
3806 | 437 | @hooks.hook | ||
3807 | 438 | def config_changed(): | ||
3808 | 439 | # View all changes to configuration | ||
3809 | 440 | for changed, (prev, cur) in changes.conf.items(): | ||
3810 | 441 | print('config changed', changed, | ||
3811 | 442 | 'previous value', prev, | ||
3812 | 443 | 'current value', cur) | ||
3813 | 444 | |||
3814 | 445 | # Get some unit specific bookeeping | ||
3815 | 446 | if not db.get('pkg_key'): | ||
3816 | 447 | key = urllib.urlopen('https://example.com/pkg_key').read() | ||
3817 | 448 | db.set('pkg_key', key) | ||
3818 | 449 | |||
3819 | 450 | if __name__ == '__main__': | ||
3820 | 451 | with changes(): | ||
3821 | 452 | hook.execute() | ||
3822 | 453 | |||
3823 | 454 | """ | ||
3824 | 455 | def __init__(self): | ||
3825 | 456 | self.kv = kv() | ||
3826 | 457 | self.conf = None | ||
3827 | 458 | self.rels = None | ||
3828 | 459 | |||
3829 | 460 | @contextlib.contextmanager | ||
3830 | 461 | def __call__(self): | ||
3831 | 462 | from charmhelpers.core import hookenv | ||
3832 | 463 | hook_name = hookenv.hook_name() | ||
3833 | 464 | |||
3834 | 465 | with self.kv.hook_scope(hook_name): | ||
3835 | 466 | self._record_charm_version(hookenv.charm_dir()) | ||
3836 | 467 | delta_config, delta_relation = self._record_hook(hookenv) | ||
3837 | 468 | yield self.kv, delta_config, delta_relation | ||
3838 | 469 | |||
3839 | 470 | def _record_charm_version(self, charm_dir): | ||
3840 | 471 | # Record revisions.. charm revisions are meaningless | ||
3841 | 472 | # to charm authors as they don't control the revision. | ||
3842 | 473 | # so logic dependnent on revision is not particularly | ||
3843 | 474 | # useful, however it is useful for debugging analysis. | ||
3844 | 475 | charm_rev = open( | ||
3845 | 476 | os.path.join(charm_dir, 'revision')).read().strip() | ||
3846 | 477 | charm_rev = charm_rev or '0' | ||
3847 | 478 | revs = self.kv.get('charm_revisions', []) | ||
3848 | 479 | if charm_rev not in revs: | ||
3849 | 480 | revs.append(charm_rev.strip() or '0') | ||
3850 | 481 | self.kv.set('charm_revisions', revs) | ||
3851 | 482 | |||
3852 | 483 | def _record_hook(self, hookenv): | ||
3853 | 484 | data = hookenv.execution_environment() | ||
3854 | 485 | self.conf = conf_delta = self.kv.delta(data['conf'], 'config') | ||
3855 | 486 | self.rels = rels_delta = self.kv.delta(data['rels'], 'rels') | ||
3856 | 487 | self.kv.set('env', dict(data['env'])) | ||
3857 | 488 | self.kv.set('unit', data['unit']) | ||
3858 | 489 | self.kv.set('relid', data.get('relid')) | ||
3859 | 490 | return conf_delta, rels_delta | ||
3860 | 491 | |||
3861 | 492 | |||
3862 | 493 | class Record(dict): | ||
3863 | 494 | |||
3864 | 495 | __slots__ = () | ||
3865 | 496 | |||
3866 | 497 | def __getattr__(self, k): | ||
3867 | 498 | if k in self: | ||
3868 | 499 | return self[k] | ||
3869 | 500 | raise AttributeError(k) | ||
3870 | 501 | |||
3871 | 502 | |||
3872 | 503 | class DeltaSet(Record): | ||
3873 | 504 | |||
3874 | 505 | __slots__ = () | ||
3875 | 506 | |||
3876 | 507 | |||
3877 | 508 | Delta = collections.namedtuple('Delta', ['previous', 'current']) | ||
3878 | 509 | |||
3879 | 510 | |||
3880 | 511 | _KV = None | ||
3881 | 512 | |||
3882 | 513 | |||
3883 | 514 | def kv(): | ||
3884 | 515 | global _KV | ||
3885 | 516 | if _KV is None: | ||
3886 | 517 | _KV = Storage() | ||
3887 | 518 | return _KV | ||
3888 | 0 | 519 | ||
3889 | === modified file 'hooks/charmhelpers/fetch/__init__.py' | |||
3890 | --- hooks/charmhelpers/fetch/__init__.py 2014-10-24 16:35:00 +0000 | |||
3891 | +++ hooks/charmhelpers/fetch/__init__.py 2017-05-11 14:56:57 +0000 | |||
3892 | @@ -1,71 +1,31 @@ | |||
3893 | 1 | # Copyright 2014-2015 Canonical Limited. | ||
3894 | 2 | # | ||
3895 | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); | ||
3896 | 4 | # you may not use this file except in compliance with the License. | ||
3897 | 5 | # You may obtain a copy of the License at | ||
3898 | 6 | # | ||
3899 | 7 | # http://www.apache.org/licenses/LICENSE-2.0 | ||
3900 | 8 | # | ||
3901 | 9 | # Unless required by applicable law or agreed to in writing, software | ||
3902 | 10 | # distributed under the License is distributed on an "AS IS" BASIS, | ||
3903 | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
3904 | 12 | # See the License for the specific language governing permissions and | ||
3905 | 13 | # limitations under the License. | ||
3906 | 14 | |||
3907 | 1 | import importlib | 15 | import importlib |
3910 | 2 | from tempfile import NamedTemporaryFile | 16 | from charmhelpers.osplatform import get_platform |
3909 | 3 | import time | ||
3911 | 4 | from yaml import safe_load | 17 | from yaml import safe_load |
3912 | 5 | from charmhelpers.core.host import ( | ||
3913 | 6 | lsb_release | ||
3914 | 7 | ) | ||
3915 | 8 | from urlparse import ( | ||
3916 | 9 | urlparse, | ||
3917 | 10 | urlunparse, | ||
3918 | 11 | ) | ||
3919 | 12 | import subprocess | ||
3920 | 13 | from charmhelpers.core.hookenv import ( | 18 | from charmhelpers.core.hookenv import ( |
3921 | 14 | config, | 19 | config, |
3922 | 15 | log, | 20 | log, |
3923 | 16 | ) | 21 | ) |
3976 | 17 | import os | 22 | |
3977 | 18 | 23 | import six | |
3978 | 19 | 24 | if six.PY3: | |
3979 | 20 | CLOUD_ARCHIVE = """# Ubuntu Cloud Archive | 25 | from urllib.parse import urlparse, urlunparse |
3980 | 21 | deb http://ubuntu-cloud.archive.canonical.com/ubuntu {} main | 26 | else: |
3981 | 22 | """ | 27 | from urlparse import urlparse, urlunparse |
3982 | 23 | PROPOSED_POCKET = """# Proposed | 28 | |
3931 | 24 | deb http://archive.ubuntu.com/ubuntu {}-proposed main universe multiverse restricted | ||
3932 | 25 | """ | ||
3933 | 26 | CLOUD_ARCHIVE_POCKETS = { | ||
3934 | 27 | # Folsom | ||
3935 | 28 | 'folsom': 'precise-updates/folsom', | ||
3936 | 29 | 'precise-folsom': 'precise-updates/folsom', | ||
3937 | 30 | 'precise-folsom/updates': 'precise-updates/folsom', | ||
3938 | 31 | 'precise-updates/folsom': 'precise-updates/folsom', | ||
3939 | 32 | 'folsom/proposed': 'precise-proposed/folsom', | ||
3940 | 33 | 'precise-folsom/proposed': 'precise-proposed/folsom', | ||
3941 | 34 | 'precise-proposed/folsom': 'precise-proposed/folsom', | ||
3942 | 35 | # Grizzly | ||
3943 | 36 | 'grizzly': 'precise-updates/grizzly', | ||
3944 | 37 | 'precise-grizzly': 'precise-updates/grizzly', | ||
3945 | 38 | 'precise-grizzly/updates': 'precise-updates/grizzly', | ||
3946 | 39 | 'precise-updates/grizzly': 'precise-updates/grizzly', | ||
3947 | 40 | 'grizzly/proposed': 'precise-proposed/grizzly', | ||
3948 | 41 | 'precise-grizzly/proposed': 'precise-proposed/grizzly', | ||
3949 | 42 | 'precise-proposed/grizzly': 'precise-proposed/grizzly', | ||
3950 | 43 | # Havana | ||
3951 | 44 | 'havana': 'precise-updates/havana', | ||
3952 | 45 | 'precise-havana': 'precise-updates/havana', | ||
3953 | 46 | 'precise-havana/updates': 'precise-updates/havana', | ||
3954 | 47 | 'precise-updates/havana': 'precise-updates/havana', | ||
3955 | 48 | 'havana/proposed': 'precise-proposed/havana', | ||
3956 | 49 | 'precise-havana/proposed': 'precise-proposed/havana', | ||
3957 | 50 | 'precise-proposed/havana': 'precise-proposed/havana', | ||
3958 | 51 | # Icehouse | ||
3959 | 52 | 'icehouse': 'precise-updates/icehouse', | ||
3960 | 53 | 'precise-icehouse': 'precise-updates/icehouse', | ||
3961 | 54 | 'precise-icehouse/updates': 'precise-updates/icehouse', | ||
3962 | 55 | 'precise-updates/icehouse': 'precise-updates/icehouse', | ||
3963 | 56 | 'icehouse/proposed': 'precise-proposed/icehouse', | ||
3964 | 57 | 'precise-icehouse/proposed': 'precise-proposed/icehouse', | ||
3965 | 58 | 'precise-proposed/icehouse': 'precise-proposed/icehouse', | ||
3966 | 59 | # Juno | ||
3967 | 60 | 'juno': 'trusty-updates/juno', | ||
3968 | 61 | 'trusty-juno': 'trusty-updates/juno', | ||
3969 | 62 | 'trusty-juno/updates': 'trusty-updates/juno', | ||
3970 | 63 | 'trusty-updates/juno': 'trusty-updates/juno', | ||
3971 | 64 | 'juno/proposed': 'trusty-proposed/juno', | ||
3972 | 65 | 'juno/proposed': 'trusty-proposed/juno', | ||
3973 | 66 | 'trusty-juno/proposed': 'trusty-proposed/juno', | ||
3974 | 67 | 'trusty-proposed/juno': 'trusty-proposed/juno', | ||
3975 | 68 | } | ||
3983 | 69 | 29 | ||
3984 | 70 | # The order of this list is very important. Handlers should be listed in from | 30 | # The order of this list is very important. Handlers should be listed in from |
3985 | 71 | # least- to most-specific URL matching. | 31 | # least- to most-specific URL matching. |
3986 | @@ -75,10 +35,6 @@ | |||
3987 | 75 | 'charmhelpers.fetch.giturl.GitUrlFetchHandler', | 35 | 'charmhelpers.fetch.giturl.GitUrlFetchHandler', |
3988 | 76 | ) | 36 | ) |
3989 | 77 | 37 | ||
3990 | 78 | APT_NO_LOCK = 100 # The return code for "couldn't acquire lock" in APT. | ||
3991 | 79 | APT_NO_LOCK_RETRY_DELAY = 10 # Wait 10 seconds between apt lock checks. | ||
3992 | 80 | APT_NO_LOCK_RETRY_COUNT = 30 # Retry to acquire the lock X times. | ||
3993 | 81 | |||
3994 | 82 | 38 | ||
3995 | 83 | class SourceConfigError(Exception): | 39 | class SourceConfigError(Exception): |
3996 | 84 | pass | 40 | pass |
3997 | @@ -116,172 +72,38 @@ | |||
3998 | 116 | return urlunparse(parts) | 72 | return urlunparse(parts) |
3999 | 117 | 73 | ||
4000 | 118 | 74 | ||
4157 | 119 | def filter_installed_packages(packages): | 75 | __platform__ = get_platform() |
4158 | 120 | """Returns a list of packages that require installation""" | 76 | module = "charmhelpers.fetch.%s" % __platform__ |
4159 | 121 | cache = apt_cache() | 77 | fetch = importlib.import_module(module) |
4160 | 122 | _pkgs = [] | 78 | |
4161 | 123 | for package in packages: | 79 | filter_installed_packages = fetch.filter_installed_packages |
4162 | 124 | try: | 80 | install = fetch.install |
4163 | 125 | p = cache[package] | 81 | upgrade = fetch.upgrade |
4164 | 126 | p.current_ver or _pkgs.append(package) | 82 | update = fetch.update |
4165 | 127 | except KeyError: | 83 | purge = fetch.purge |
4166 | 128 | log('Package {} has no installation candidate.'.format(package), | 84 | add_source = fetch.add_source |
4167 | 129 | level='WARNING') | 85 | |
4168 | 130 | _pkgs.append(package) | 86 | if __platform__ == "ubuntu": |
4169 | 131 | return _pkgs | 87 | apt_cache = fetch.apt_cache |
4170 | 132 | 88 | apt_install = fetch.install | |
4171 | 133 | 89 | apt_update = fetch.update | |
4172 | 134 | def apt_cache(in_memory=True): | 90 | apt_upgrade = fetch.upgrade |
4173 | 135 | """Build and return an apt cache""" | 91 | apt_purge = fetch.purge |
4174 | 136 | import apt_pkg | 92 | apt_mark = fetch.apt_mark |
4175 | 137 | apt_pkg.init() | 93 | apt_hold = fetch.apt_hold |
4176 | 138 | if in_memory: | 94 | apt_unhold = fetch.apt_unhold |
4177 | 139 | apt_pkg.config.set("Dir::Cache::pkgcache", "") | 95 | get_upstream_version = fetch.get_upstream_version |
4178 | 140 | apt_pkg.config.set("Dir::Cache::srcpkgcache", "") | 96 | elif __platform__ == "centos": |
4179 | 141 | return apt_pkg.Cache() | 97 | yum_search = fetch.yum_search |
4024 | 142 | |||
4025 | 143 | |||
4026 | 144 | def apt_install(packages, options=None, fatal=False): | ||
4027 | 145 | """Install one or more packages""" | ||
4028 | 146 | if options is None: | ||
4029 | 147 | options = ['--option=Dpkg::Options::=--force-confold'] | ||
4030 | 148 | |||
4031 | 149 | cmd = ['apt-get', '--assume-yes'] | ||
4032 | 150 | cmd.extend(options) | ||
4033 | 151 | cmd.append('install') | ||
4034 | 152 | if isinstance(packages, basestring): | ||
4035 | 153 | cmd.append(packages) | ||
4036 | 154 | else: | ||
4037 | 155 | cmd.extend(packages) | ||
4038 | 156 | log("Installing {} with options: {}".format(packages, | ||
4039 | 157 | options)) | ||
4040 | 158 | _run_apt_command(cmd, fatal) | ||
4041 | 159 | |||
4042 | 160 | |||
4043 | 161 | def apt_upgrade(options=None, fatal=False, dist=False): | ||
4044 | 162 | """Upgrade all packages""" | ||
4045 | 163 | if options is None: | ||
4046 | 164 | options = ['--option=Dpkg::Options::=--force-confold'] | ||
4047 | 165 | |||
4048 | 166 | cmd = ['apt-get', '--assume-yes'] | ||
4049 | 167 | cmd.extend(options) | ||
4050 | 168 | if dist: | ||
4051 | 169 | cmd.append('dist-upgrade') | ||
4052 | 170 | else: | ||
4053 | 171 | cmd.append('upgrade') | ||
4054 | 172 | log("Upgrading with options: {}".format(options)) | ||
4055 | 173 | _run_apt_command(cmd, fatal) | ||
4056 | 174 | |||
4057 | 175 | |||
4058 | 176 | def apt_update(fatal=False): | ||
4059 | 177 | """Update local apt cache""" | ||
4060 | 178 | cmd = ['apt-get', 'update'] | ||
4061 | 179 | _run_apt_command(cmd, fatal) | ||
4062 | 180 | |||
4063 | 181 | |||
4064 | 182 | def apt_purge(packages, fatal=False): | ||
4065 | 183 | """Purge one or more packages""" | ||
4066 | 184 | cmd = ['apt-get', '--assume-yes', 'purge'] | ||
4067 | 185 | if isinstance(packages, basestring): | ||
4068 | 186 | cmd.append(packages) | ||
4069 | 187 | else: | ||
4070 | 188 | cmd.extend(packages) | ||
4071 | 189 | log("Purging {}".format(packages)) | ||
4072 | 190 | _run_apt_command(cmd, fatal) | ||
4073 | 191 | |||
4074 | 192 | |||
4075 | 193 | def apt_hold(packages, fatal=False): | ||
4076 | 194 | """Hold one or more packages""" | ||
4077 | 195 | cmd = ['apt-mark', 'hold'] | ||
4078 | 196 | if isinstance(packages, basestring): | ||
4079 | 197 | cmd.append(packages) | ||
4080 | 198 | else: | ||
4081 | 199 | cmd.extend(packages) | ||
4082 | 200 | log("Holding {}".format(packages)) | ||
4083 | 201 | |||
4084 | 202 | if fatal: | ||
4085 | 203 | subprocess.check_call(cmd) | ||
4086 | 204 | else: | ||
4087 | 205 | subprocess.call(cmd) | ||
4088 | 206 | |||
4089 | 207 | |||
4090 | 208 | def add_source(source, key=None): | ||
4091 | 209 | """Add a package source to this system. | ||
4092 | 210 | |||
4093 | 211 | @param source: a URL or sources.list entry, as supported by | ||
4094 | 212 | add-apt-repository(1). Examples:: | ||
4095 | 213 | |||
4096 | 214 | ppa:charmers/example | ||
4097 | 215 | deb https://stub:key@private.example.com/ubuntu trusty main | ||
4098 | 216 | |||
4099 | 217 | In addition: | ||
4100 | 218 | 'proposed:' may be used to enable the standard 'proposed' | ||
4101 | 219 | pocket for the release. | ||
4102 | 220 | 'cloud:' may be used to activate official cloud archive pockets, | ||
4103 | 221 | such as 'cloud:icehouse' | ||
4104 | 222 | 'distro' may be used as a noop | ||
4105 | 223 | |||
4106 | 224 | @param key: A key to be added to the system's APT keyring and used | ||
4107 | 225 | to verify the signatures on packages. Ideally, this should be an | ||
4108 | 226 | ASCII format GPG public key including the block headers. A GPG key | ||
4109 | 227 | id may also be used, but be aware that only insecure protocols are | ||
4110 | 228 | available to retrieve the actual public key from a public keyserver | ||
4111 | 229 | placing your Juju environment at risk. ppa and cloud archive keys | ||
4112 | 230 | are securely added automtically, so sould not be provided. | ||
4113 | 231 | """ | ||
4114 | 232 | if source is None: | ||
4115 | 233 | log('Source is not present. Skipping') | ||
4116 | 234 | return | ||
4117 | 235 | |||
4118 | 236 | if (source.startswith('ppa:') or | ||
4119 | 237 | source.startswith('http') or | ||
4120 | 238 | source.startswith('deb ') or | ||
4121 | 239 | source.startswith('cloud-archive:')): | ||
4122 | 240 | subprocess.check_call(['add-apt-repository', '--yes', source]) | ||
4123 | 241 | elif source.startswith('cloud:'): | ||
4124 | 242 | apt_install(filter_installed_packages(['ubuntu-cloud-keyring']), | ||
4125 | 243 | fatal=True) | ||
4126 | 244 | pocket = source.split(':')[-1] | ||
4127 | 245 | if pocket not in CLOUD_ARCHIVE_POCKETS: | ||
4128 | 246 | raise SourceConfigError( | ||
4129 | 247 | 'Unsupported cloud: source option %s' % | ||
4130 | 248 | pocket) | ||
4131 | 249 | actual_pocket = CLOUD_ARCHIVE_POCKETS[pocket] | ||
4132 | 250 | with open('/etc/apt/sources.list.d/cloud-archive.list', 'w') as apt: | ||
4133 | 251 | apt.write(CLOUD_ARCHIVE.format(actual_pocket)) | ||
4134 | 252 | elif source == 'proposed': | ||
4135 | 253 | release = lsb_release()['DISTRIB_CODENAME'] | ||
4136 | 254 | with open('/etc/apt/sources.list.d/proposed.list', 'w') as apt: | ||
4137 | 255 | apt.write(PROPOSED_POCKET.format(release)) | ||
4138 | 256 | elif source == 'distro': | ||
4139 | 257 | pass | ||
4140 | 258 | else: | ||
4141 | 259 | log("Unknown source: {!r}".format(source)) | ||
4142 | 260 | |||
4143 | 261 | if key: | ||
4144 | 262 | if '-----BEGIN PGP PUBLIC KEY BLOCK-----' in key: | ||
4145 | 263 | with NamedTemporaryFile() as key_file: | ||
4146 | 264 | key_file.write(key) | ||
4147 | 265 | key_file.flush() | ||
4148 | 266 | key_file.seek(0) | ||
4149 | 267 | subprocess.check_call(['apt-key', 'add', '-'], stdin=key_file) | ||
4150 | 268 | else: | ||
4151 | 269 | # Note that hkp: is in no way a secure protocol. Using a | ||
4152 | 270 | # GPG key id is pointless from a security POV unless you | ||
4153 | 271 | # absolutely trust your network and DNS. | ||
4154 | 272 | subprocess.check_call(['apt-key', 'adv', '--keyserver', | ||
4155 | 273 | 'hkp://keyserver.ubuntu.com:80', '--recv', | ||
4156 | 274 | key]) | ||
4180 | 275 | 98 | ||
4181 | 276 | 99 | ||
4182 | 277 | def configure_sources(update=False, | 100 | def configure_sources(update=False, |
4183 | 278 | sources_var='install_sources', | 101 | sources_var='install_sources', |
4184 | 279 | keys_var='install_keys'): | 102 | keys_var='install_keys'): |
4187 | 280 | """ | 103 | """Configure multiple sources from charm configuration. |
4186 | 281 | Configure multiple sources from charm configuration. | ||
4188 | 282 | 104 | ||
4189 | 283 | The lists are encoded as yaml fragments in the configuration. | 105 | The lists are encoded as yaml fragments in the configuration. |
4191 | 284 | The frament needs to be included as a string. Sources and their | 106 | The fragment needs to be included as a string. Sources and their |
4192 | 285 | corresponding keys are of the types supported by add_source(). | 107 | corresponding keys are of the types supported by add_source(). |
4193 | 286 | 108 | ||
4194 | 287 | Example config: | 109 | Example config: |
4195 | @@ -297,14 +119,14 @@ | |||
4196 | 297 | sources = safe_load((config(sources_var) or '').strip()) or [] | 119 | sources = safe_load((config(sources_var) or '').strip()) or [] |
4197 | 298 | keys = safe_load((config(keys_var) or '').strip()) or None | 120 | keys = safe_load((config(keys_var) or '').strip()) or None |
4198 | 299 | 121 | ||
4200 | 300 | if isinstance(sources, basestring): | 122 | if isinstance(sources, six.string_types): |
4201 | 301 | sources = [sources] | 123 | sources = [sources] |
4202 | 302 | 124 | ||
4203 | 303 | if keys is None: | 125 | if keys is None: |
4204 | 304 | for source in sources: | 126 | for source in sources: |
4205 | 305 | add_source(source, None) | 127 | add_source(source, None) |
4206 | 306 | else: | 128 | else: |
4208 | 307 | if isinstance(keys, basestring): | 129 | if isinstance(keys, six.string_types): |
4209 | 308 | keys = [keys] | 130 | keys = [keys] |
4210 | 309 | 131 | ||
4211 | 310 | if len(sources) != len(keys): | 132 | if len(sources) != len(keys): |
4212 | @@ -313,12 +135,11 @@ | |||
4213 | 313 | for source, key in zip(sources, keys): | 135 | for source, key in zip(sources, keys): |
4214 | 314 | add_source(source, key) | 136 | add_source(source, key) |
4215 | 315 | if update: | 137 | if update: |
4217 | 316 | apt_update(fatal=True) | 138 | fetch.update(fatal=True) |
4218 | 317 | 139 | ||
4219 | 318 | 140 | ||
4220 | 319 | def install_remote(source, *args, **kwargs): | 141 | def install_remote(source, *args, **kwargs): |
4223 | 320 | """ | 142 | """Install a file tree from a remote source. |
4222 | 321 | Install a file tree from a remote source | ||
4224 | 322 | 143 | ||
4225 | 323 | The specified source should be a url of the form: | 144 | The specified source should be a url of the form: |
4226 | 324 | scheme://[host]/path[#[option=value][&...]] | 145 | scheme://[host]/path[#[option=value][&...]] |
4227 | @@ -341,18 +162,17 @@ | |||
4228 | 341 | # We ONLY check for True here because can_handle may return a string | 162 | # We ONLY check for True here because can_handle may return a string |
4229 | 342 | # explaining why it can't handle a given source. | 163 | # explaining why it can't handle a given source. |
4230 | 343 | handlers = [h for h in plugins() if h.can_handle(source) is True] | 164 | handlers = [h for h in plugins() if h.can_handle(source) is True] |
4231 | 344 | installed_to = None | ||
4232 | 345 | for handler in handlers: | 165 | for handler in handlers: |
4233 | 346 | try: | 166 | try: |
4240 | 347 | installed_to = handler.install(source, *args, **kwargs) | 167 | return handler.install(source, *args, **kwargs) |
4241 | 348 | except UnhandledSource: | 168 | except UnhandledSource as e: |
4242 | 349 | pass | 169 | log('Install source attempt unsuccessful: {}'.format(e), |
4243 | 350 | if not installed_to: | 170 | level='WARNING') |
4244 | 351 | raise UnhandledSource("No handler found for source {}".format(source)) | 171 | raise UnhandledSource("No handler found for source {}".format(source)) |
4239 | 352 | return installed_to | ||
4245 | 353 | 172 | ||
4246 | 354 | 173 | ||
4247 | 355 | def install_from_config(config_var_name): | 174 | def install_from_config(config_var_name): |
4248 | 175 | """Install a file from config.""" | ||
4249 | 356 | charm_config = config() | 176 | charm_config = config() |
4250 | 357 | source = charm_config[config_var_name] | 177 | source = charm_config[config_var_name] |
4251 | 358 | return install_remote(source) | 178 | return install_remote(source) |
4252 | @@ -369,46 +189,9 @@ | |||
4253 | 369 | importlib.import_module(package), | 189 | importlib.import_module(package), |
4254 | 370 | classname) | 190 | classname) |
4255 | 371 | plugin_list.append(handler_class()) | 191 | plugin_list.append(handler_class()) |
4257 | 372 | except (ImportError, AttributeError): | 192 | except NotImplementedError: |
4258 | 373 | # Skip missing plugins so that they can be ommitted from | 193 | # Skip missing plugins so that they can be ommitted from |
4259 | 374 | # installation if desired | 194 | # installation if desired |
4260 | 375 | log("FetchHandler {} not found, skipping plugin".format( | 195 | log("FetchHandler {} not found, skipping plugin".format( |
4261 | 376 | handler_name)) | 196 | handler_name)) |
4262 | 377 | return plugin_list | 197 | return plugin_list |
4263 | 378 | |||
4264 | 379 | |||
4265 | 380 | def _run_apt_command(cmd, fatal=False): | ||
4266 | 381 | """ | ||
4267 | 382 | Run an APT command, checking output and retrying if the fatal flag is set | ||
4268 | 383 | to True. | ||
4269 | 384 | |||
4270 | 385 | :param: cmd: str: The apt command to run. | ||
4271 | 386 | :param: fatal: bool: Whether the command's output should be checked and | ||
4272 | 387 | retried. | ||
4273 | 388 | """ | ||
4274 | 389 | env = os.environ.copy() | ||
4275 | 390 | |||
4276 | 391 | if 'DEBIAN_FRONTEND' not in env: | ||
4277 | 392 | env['DEBIAN_FRONTEND'] = 'noninteractive' | ||
4278 | 393 | |||
4279 | 394 | if fatal: | ||
4280 | 395 | retry_count = 0 | ||
4281 | 396 | result = None | ||
4282 | 397 | |||
4283 | 398 | # If the command is considered "fatal", we need to retry if the apt | ||
4284 | 399 | # lock was not acquired. | ||
4285 | 400 | |||
4286 | 401 | while result is None or result == APT_NO_LOCK: | ||
4287 | 402 | try: | ||
4288 | 403 | result = subprocess.check_call(cmd, env=env) | ||
4289 | 404 | except subprocess.CalledProcessError, e: | ||
4290 | 405 | retry_count = retry_count + 1 | ||
4291 | 406 | if retry_count > APT_NO_LOCK_RETRY_COUNT: | ||
4292 | 407 | raise | ||
4293 | 408 | result = e.returncode | ||
4294 | 409 | log("Couldn't acquire DPKG lock. Will retry in {} seconds." | ||
4295 | 410 | "".format(APT_NO_LOCK_RETRY_DELAY)) | ||
4296 | 411 | time.sleep(APT_NO_LOCK_RETRY_DELAY) | ||
4297 | 412 | |||
4298 | 413 | else: | ||
4299 | 414 | subprocess.call(cmd, env=env) | ||
4300 | 415 | 198 | ||
4301 | === modified file 'hooks/charmhelpers/fetch/archiveurl.py' | |||
4302 | --- hooks/charmhelpers/fetch/archiveurl.py 2014-10-24 16:35:00 +0000 | |||
4303 | +++ hooks/charmhelpers/fetch/archiveurl.py 2017-05-11 14:56:57 +0000 | |||
4304 | @@ -1,8 +1,20 @@ | |||
4305 | 1 | # Copyright 2014-2015 Canonical Limited. | ||
4306 | 2 | # | ||
4307 | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); | ||
4308 | 4 | # you may not use this file except in compliance with the License. | ||
4309 | 5 | # You may obtain a copy of the License at | ||
4310 | 6 | # | ||
4311 | 7 | # http://www.apache.org/licenses/LICENSE-2.0 | ||
4312 | 8 | # | ||
4313 | 9 | # Unless required by applicable law or agreed to in writing, software | ||
4314 | 10 | # distributed under the License is distributed on an "AS IS" BASIS, | ||
4315 | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
4316 | 12 | # See the License for the specific language governing permissions and | ||
4317 | 13 | # limitations under the License. | ||
4318 | 14 | |||
4319 | 1 | import os | 15 | import os |
4320 | 2 | import urllib2 | ||
4321 | 3 | from urllib import urlretrieve | ||
4322 | 4 | import urlparse | ||
4323 | 5 | import hashlib | 16 | import hashlib |
4324 | 17 | import re | ||
4325 | 6 | 18 | ||
4326 | 7 | from charmhelpers.fetch import ( | 19 | from charmhelpers.fetch import ( |
4327 | 8 | BaseFetchHandler, | 20 | BaseFetchHandler, |
4328 | @@ -14,6 +26,41 @@ | |||
4329 | 14 | ) | 26 | ) |
4330 | 15 | from charmhelpers.core.host import mkdir, check_hash | 27 | from charmhelpers.core.host import mkdir, check_hash |
4331 | 16 | 28 | ||
4332 | 29 | import six | ||
4333 | 30 | if six.PY3: | ||
4334 | 31 | from urllib.request import ( | ||
4335 | 32 | build_opener, install_opener, urlopen, urlretrieve, | ||
4336 | 33 | HTTPPasswordMgrWithDefaultRealm, HTTPBasicAuthHandler, | ||
4337 | 34 | ) | ||
4338 | 35 | from urllib.parse import urlparse, urlunparse, parse_qs | ||
4339 | 36 | from urllib.error import URLError | ||
4340 | 37 | else: | ||
4341 | 38 | from urllib import urlretrieve | ||
4342 | 39 | from urllib2 import ( | ||
4343 | 40 | build_opener, install_opener, urlopen, | ||
4344 | 41 | HTTPPasswordMgrWithDefaultRealm, HTTPBasicAuthHandler, | ||
4345 | 42 | URLError | ||
4346 | 43 | ) | ||
4347 | 44 | from urlparse import urlparse, urlunparse, parse_qs | ||
4348 | 45 | |||
4349 | 46 | |||
4350 | 47 | def splituser(host): | ||
4351 | 48 | '''urllib.splituser(), but six's support of this seems broken''' | ||
4352 | 49 | _userprog = re.compile('^(.*)@(.*)$') | ||
4353 | 50 | match = _userprog.match(host) | ||
4354 | 51 | if match: | ||
4355 | 52 | return match.group(1, 2) | ||
4356 | 53 | return None, host | ||
4357 | 54 | |||
4358 | 55 | |||
4359 | 56 | def splitpasswd(user): | ||
4360 | 57 | '''urllib.splitpasswd(), but six's support of this is missing''' | ||
4361 | 58 | _passwdprog = re.compile('^([^:]*):(.*)$', re.S) | ||
4362 | 59 | match = _passwdprog.match(user) | ||
4363 | 60 | if match: | ||
4364 | 61 | return match.group(1, 2) | ||
4365 | 62 | return user, None | ||
4366 | 63 | |||
4367 | 17 | 64 | ||
4368 | 18 | class ArchiveUrlFetchHandler(BaseFetchHandler): | 65 | class ArchiveUrlFetchHandler(BaseFetchHandler): |
4369 | 19 | """ | 66 | """ |
4370 | @@ -28,6 +75,8 @@ | |||
4371 | 28 | def can_handle(self, source): | 75 | def can_handle(self, source): |
4372 | 29 | url_parts = self.parse_url(source) | 76 | url_parts = self.parse_url(source) |
4373 | 30 | if url_parts.scheme not in ('http', 'https', 'ftp', 'file'): | 77 | if url_parts.scheme not in ('http', 'https', 'ftp', 'file'): |
4374 | 78 | # XXX: Why is this returning a boolean and a string? It's | ||
4375 | 79 | # doomed to fail since "bool(can_handle('foo://'))" will be True. | ||
4376 | 31 | return "Wrong source type" | 80 | return "Wrong source type" |
4377 | 32 | if get_archive_handler(self.base_url(source)): | 81 | if get_archive_handler(self.base_url(source)): |
4378 | 33 | return True | 82 | return True |
4379 | @@ -42,22 +91,22 @@ | |||
4380 | 42 | """ | 91 | """ |
4381 | 43 | # propogate all exceptions | 92 | # propogate all exceptions |
4382 | 44 | # URLError, OSError, etc | 93 | # URLError, OSError, etc |
4384 | 45 | proto, netloc, path, params, query, fragment = urlparse.urlparse(source) | 94 | proto, netloc, path, params, query, fragment = urlparse(source) |
4385 | 46 | if proto in ('http', 'https'): | 95 | if proto in ('http', 'https'): |
4387 | 47 | auth, barehost = urllib2.splituser(netloc) | 96 | auth, barehost = splituser(netloc) |
4388 | 48 | if auth is not None: | 97 | if auth is not None: |
4392 | 49 | source = urlparse.urlunparse((proto, barehost, path, params, query, fragment)) | 98 | source = urlunparse((proto, barehost, path, params, query, fragment)) |
4393 | 50 | username, password = urllib2.splitpasswd(auth) | 99 | username, password = splitpasswd(auth) |
4394 | 51 | passman = urllib2.HTTPPasswordMgrWithDefaultRealm() | 100 | passman = HTTPPasswordMgrWithDefaultRealm() |
4395 | 52 | # Realm is set to None in add_password to force the username and password | 101 | # Realm is set to None in add_password to force the username and password |
4396 | 53 | # to be used whatever the realm | 102 | # to be used whatever the realm |
4397 | 54 | passman.add_password(None, source, username, password) | 103 | passman.add_password(None, source, username, password) |
4402 | 55 | authhandler = urllib2.HTTPBasicAuthHandler(passman) | 104 | authhandler = HTTPBasicAuthHandler(passman) |
4403 | 56 | opener = urllib2.build_opener(authhandler) | 105 | opener = build_opener(authhandler) |
4404 | 57 | urllib2.install_opener(opener) | 106 | install_opener(opener) |
4405 | 58 | response = urllib2.urlopen(source) | 107 | response = urlopen(source) |
4406 | 59 | try: | 108 | try: |
4408 | 60 | with open(dest, 'w') as dest_file: | 109 | with open(dest, 'wb') as dest_file: |
4409 | 61 | dest_file.write(response.read()) | 110 | dest_file.write(response.read()) |
4410 | 62 | except Exception as e: | 111 | except Exception as e: |
4411 | 63 | if os.path.isfile(dest): | 112 | if os.path.isfile(dest): |
4412 | @@ -91,18 +140,26 @@ | |||
4413 | 91 | url_parts = self.parse_url(source) | 140 | url_parts = self.parse_url(source) |
4414 | 92 | dest_dir = os.path.join(os.environ.get('CHARM_DIR'), 'fetched') | 141 | dest_dir = os.path.join(os.environ.get('CHARM_DIR'), 'fetched') |
4415 | 93 | if not os.path.exists(dest_dir): | 142 | if not os.path.exists(dest_dir): |
4417 | 94 | mkdir(dest_dir, perms=0755) | 143 | mkdir(dest_dir, perms=0o755) |
4418 | 95 | dld_file = os.path.join(dest_dir, os.path.basename(url_parts.path)) | 144 | dld_file = os.path.join(dest_dir, os.path.basename(url_parts.path)) |
4419 | 96 | try: | 145 | try: |
4420 | 97 | self.download(source, dld_file) | 146 | self.download(source, dld_file) |
4422 | 98 | except urllib2.URLError as e: | 147 | except URLError as e: |
4423 | 99 | raise UnhandledSource(e.reason) | 148 | raise UnhandledSource(e.reason) |
4424 | 100 | except OSError as e: | 149 | except OSError as e: |
4425 | 101 | raise UnhandledSource(e.strerror) | 150 | raise UnhandledSource(e.strerror) |
4427 | 102 | options = urlparse.parse_qs(url_parts.fragment) | 151 | options = parse_qs(url_parts.fragment) |
4428 | 103 | for key, value in options.items(): | 152 | for key, value in options.items(): |
4431 | 104 | if key in hashlib.algorithms: | 153 | if not six.PY3: |
4432 | 105 | check_hash(dld_file, value, key) | 154 | algorithms = hashlib.algorithms |
4433 | 155 | else: | ||
4434 | 156 | algorithms = hashlib.algorithms_available | ||
4435 | 157 | if key in algorithms: | ||
4436 | 158 | if len(value) != 1: | ||
4437 | 159 | raise TypeError( | ||
4438 | 160 | "Expected 1 hash value, not %d" % len(value)) | ||
4439 | 161 | expected = value[0] | ||
4440 | 162 | check_hash(dld_file, expected, key) | ||
4441 | 106 | if checksum: | 163 | if checksum: |
4442 | 107 | check_hash(dld_file, checksum, hash_type) | 164 | check_hash(dld_file, checksum, hash_type) |
4443 | 108 | return extract(dld_file, dest) | 165 | return extract(dld_file, dest) |
4444 | 109 | 166 | ||
4445 | === modified file 'hooks/charmhelpers/fetch/bzrurl.py' | |||
4446 | --- hooks/charmhelpers/fetch/bzrurl.py 2014-10-24 16:35:00 +0000 | |||
4447 | +++ hooks/charmhelpers/fetch/bzrurl.py 2017-05-11 14:56:57 +0000 | |||
4448 | @@ -1,50 +1,76 @@ | |||
4449 | 1 | # Copyright 2014-2015 Canonical Limited. | ||
4450 | 2 | # | ||
4451 | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); | ||
4452 | 4 | # you may not use this file except in compliance with the License. | ||
4453 | 5 | # You may obtain a copy of the License at | ||
4454 | 6 | # | ||
4455 | 7 | # http://www.apache.org/licenses/LICENSE-2.0 | ||
4456 | 8 | # | ||
4457 | 9 | # Unless required by applicable law or agreed to in writing, software | ||
4458 | 10 | # distributed under the License is distributed on an "AS IS" BASIS, | ||
4459 | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
4460 | 12 | # See the License for the specific language governing permissions and | ||
4461 | 13 | # limitations under the License. | ||
4462 | 14 | |||
4463 | 1 | import os | 15 | import os |
4464 | 16 | from subprocess import check_call | ||
4465 | 2 | from charmhelpers.fetch import ( | 17 | from charmhelpers.fetch import ( |
4466 | 3 | BaseFetchHandler, | 18 | BaseFetchHandler, |
4468 | 4 | UnhandledSource | 19 | UnhandledSource, |
4469 | 20 | filter_installed_packages, | ||
4470 | 21 | install, | ||
4471 | 5 | ) | 22 | ) |
4472 | 6 | from charmhelpers.core.host import mkdir | 23 | from charmhelpers.core.host import mkdir |
4473 | 7 | 24 | ||
4480 | 8 | try: | 25 | |
4481 | 9 | from bzrlib.branch import Branch | 26 | if filter_installed_packages(['bzr']) != []: |
4482 | 10 | except ImportError: | 27 | install(['bzr']) |
4483 | 11 | from charmhelpers.fetch import apt_install | 28 | if filter_installed_packages(['bzr']) != []: |
4484 | 12 | apt_install("python-bzrlib") | 29 | raise NotImplementedError('Unable to install bzr') |
4479 | 13 | from bzrlib.branch import Branch | ||
4485 | 14 | 30 | ||
4486 | 15 | 31 | ||
4487 | 16 | class BzrUrlFetchHandler(BaseFetchHandler): | 32 | class BzrUrlFetchHandler(BaseFetchHandler): |
4489 | 17 | """Handler for bazaar branches via generic and lp URLs""" | 33 | """Handler for bazaar branches via generic and lp URLs.""" |
4490 | 34 | |||
4491 | 18 | def can_handle(self, source): | 35 | def can_handle(self, source): |
4492 | 19 | url_parts = self.parse_url(source) | 36 | url_parts = self.parse_url(source) |
4494 | 20 | if url_parts.scheme not in ('bzr+ssh', 'lp'): | 37 | if url_parts.scheme not in ('bzr+ssh', 'lp', ''): |
4495 | 21 | return False | 38 | return False |
4496 | 39 | elif not url_parts.scheme: | ||
4497 | 40 | return os.path.exists(os.path.join(source, '.bzr')) | ||
4498 | 22 | else: | 41 | else: |
4499 | 23 | return True | 42 | return True |
4500 | 24 | 43 | ||
4504 | 25 | def branch(self, source, dest): | 44 | def branch(self, source, dest, revno=None): |
4502 | 26 | url_parts = self.parse_url(source) | ||
4503 | 27 | # If we use lp:branchname scheme we need to load plugins | ||
4505 | 28 | if not self.can_handle(source): | 45 | if not self.can_handle(source): |
4506 | 29 | raise UnhandledSource("Cannot handle {}".format(source)) | 46 | raise UnhandledSource("Cannot handle {}".format(source)) |
4515 | 30 | if url_parts.scheme == "lp": | 47 | cmd_opts = [] |
4516 | 31 | from bzrlib.plugin import load_plugins | 48 | if revno: |
4517 | 32 | load_plugins() | 49 | cmd_opts += ['-r', str(revno)] |
4518 | 33 | try: | 50 | if os.path.exists(dest): |
4519 | 34 | remote_branch = Branch.open(source) | 51 | cmd = ['bzr', 'pull'] |
4520 | 35 | remote_branch.bzrdir.sprout(dest).open_branch() | 52 | cmd += cmd_opts |
4521 | 36 | except Exception as e: | 53 | cmd += ['--overwrite', '-d', dest, source] |
4522 | 37 | raise e | 54 | else: |
4523 | 55 | cmd = ['bzr', 'branch'] | ||
4524 | 56 | cmd += cmd_opts | ||
4525 | 57 | cmd += [source, dest] | ||
4526 | 58 | check_call(cmd) | ||
4527 | 38 | 59 | ||
4529 | 39 | def install(self, source): | 60 | def install(self, source, dest=None, revno=None): |
4530 | 40 | url_parts = self.parse_url(source) | 61 | url_parts = self.parse_url(source) |
4531 | 41 | branch_name = url_parts.path.strip("/").split("/")[-1] | 62 | branch_name = url_parts.path.strip("/").split("/")[-1] |
4536 | 42 | dest_dir = os.path.join(os.environ.get('CHARM_DIR'), "fetched", | 63 | if dest: |
4537 | 43 | branch_name) | 64 | dest_dir = os.path.join(dest, branch_name) |
4538 | 44 | if not os.path.exists(dest_dir): | 65 | else: |
4539 | 45 | mkdir(dest_dir, perms=0755) | 66 | dest_dir = os.path.join(os.environ.get('CHARM_DIR'), "fetched", |
4540 | 67 | branch_name) | ||
4541 | 68 | |||
4542 | 69 | if dest and not os.path.exists(dest): | ||
4543 | 70 | mkdir(dest, perms=0o755) | ||
4544 | 71 | |||
4545 | 46 | try: | 72 | try: |
4547 | 47 | self.branch(source, dest_dir) | 73 | self.branch(source, dest_dir, revno) |
4548 | 48 | except OSError as e: | 74 | except OSError as e: |
4549 | 49 | raise UnhandledSource(e.strerror) | 75 | raise UnhandledSource(e.strerror) |
4550 | 50 | return dest_dir | 76 | return dest_dir |
4551 | 51 | 77 | ||
4552 | === added file 'hooks/charmhelpers/fetch/centos.py' | |||
4553 | --- hooks/charmhelpers/fetch/centos.py 1970-01-01 00:00:00 +0000 | |||
4554 | +++ hooks/charmhelpers/fetch/centos.py 2017-05-11 14:56:57 +0000 | |||
4555 | @@ -0,0 +1,171 @@ | |||
4556 | 1 | # Copyright 2014-2015 Canonical Limited. | ||
4557 | 2 | # | ||
4558 | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); | ||
4559 | 4 | # you may not use this file except in compliance with the License. | ||
4560 | 5 | # You may obtain a copy of the License at | ||
4561 | 6 | # | ||
4562 | 7 | # http://www.apache.org/licenses/LICENSE-2.0 | ||
4563 | 8 | # | ||
4564 | 9 | # Unless required by applicable law or agreed to in writing, software | ||
4565 | 10 | # distributed under the License is distributed on an "AS IS" BASIS, | ||
4566 | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
4567 | 12 | # See the License for the specific language governing permissions and | ||
4568 | 13 | # limitations under the License. | ||
4569 | 14 | |||
4570 | 15 | import subprocess | ||
4571 | 16 | import os | ||
4572 | 17 | import time | ||
4573 | 18 | import six | ||
4574 | 19 | import yum | ||
4575 | 20 | |||
4576 | 21 | from tempfile import NamedTemporaryFile | ||
4577 | 22 | from charmhelpers.core.hookenv import log | ||
4578 | 23 | |||
4579 | 24 | YUM_NO_LOCK = 1 # The return code for "couldn't acquire lock" in YUM. | ||
4580 | 25 | YUM_NO_LOCK_RETRY_DELAY = 10 # Wait 10 seconds between apt lock checks. | ||
4581 | 26 | YUM_NO_LOCK_RETRY_COUNT = 30 # Retry to acquire the lock X times. | ||
4582 | 27 | |||
4583 | 28 | |||
4584 | 29 | def filter_installed_packages(packages): | ||
4585 | 30 | """Return a list of packages that require installation.""" | ||
4586 | 31 | yb = yum.YumBase() | ||
4587 | 32 | package_list = yb.doPackageLists() | ||
4588 | 33 | temp_cache = {p.base_package_name: 1 for p in package_list['installed']} | ||
4589 | 34 | |||
4590 | 35 | _pkgs = [p for p in packages if not temp_cache.get(p, False)] | ||
4591 | 36 | return _pkgs | ||
4592 | 37 | |||
4593 | 38 | |||
4594 | 39 | def install(packages, options=None, fatal=False): | ||
4595 | 40 | """Install one or more packages.""" | ||
4596 | 41 | cmd = ['yum', '--assumeyes'] | ||
4597 | 42 | if options is not None: | ||
4598 | 43 | cmd.extend(options) | ||
4599 | 44 | cmd.append('install') | ||
4600 | 45 | if isinstance(packages, six.string_types): | ||
4601 | 46 | cmd.append(packages) | ||
4602 | 47 | else: | ||
4603 | 48 | cmd.extend(packages) | ||
4604 | 49 | log("Installing {} with options: {}".format(packages, | ||
4605 | 50 | options)) | ||
4606 | 51 | _run_yum_command(cmd, fatal) | ||
4607 | 52 | |||
4608 | 53 | |||
4609 | 54 | def upgrade(options=None, fatal=False, dist=False): | ||
4610 | 55 | """Upgrade all packages.""" | ||
4611 | 56 | cmd = ['yum', '--assumeyes'] | ||
4612 | 57 | if options is not None: | ||
4613 | 58 | cmd.extend(options) | ||
4614 | 59 | cmd.append('upgrade') | ||
4615 | 60 | log("Upgrading with options: {}".format(options)) | ||
4616 | 61 | _run_yum_command(cmd, fatal) | ||
4617 | 62 | |||
4618 | 63 | |||
4619 | 64 | def update(fatal=False): | ||
4620 | 65 | """Update local yum cache.""" | ||
4621 | 66 | cmd = ['yum', '--assumeyes', 'update'] | ||
4622 | 67 | log("Update with fatal: {}".format(fatal)) | ||
4623 | 68 | _run_yum_command(cmd, fatal) | ||
4624 | 69 | |||
4625 | 70 | |||
4626 | 71 | def purge(packages, fatal=False): | ||
4627 | 72 | """Purge one or more packages.""" | ||
4628 | 73 | cmd = ['yum', '--assumeyes', 'remove'] | ||
4629 | 74 | if isinstance(packages, six.string_types): | ||
4630 | 75 | cmd.append(packages) | ||
4631 | 76 | else: | ||
4632 | 77 | cmd.extend(packages) | ||
4633 | 78 | log("Purging {}".format(packages)) | ||
4634 | 79 | _run_yum_command(cmd, fatal) | ||
4635 | 80 | |||
4636 | 81 | |||
4637 | 82 | def yum_search(packages): | ||
4638 | 83 | """Search for a package.""" | ||
4639 | 84 | output = {} | ||
4640 | 85 | cmd = ['yum', 'search'] | ||
4641 | 86 | if isinstance(packages, six.string_types): | ||
4642 | 87 | cmd.append(packages) | ||
4643 | 88 | else: | ||
4644 | 89 | cmd.extend(packages) | ||
4645 | 90 | log("Searching for {}".format(packages)) | ||
4646 | 91 | result = subprocess.check_output(cmd) | ||
4647 | 92 | for package in list(packages): | ||
4648 | 93 | output[package] = package in result | ||
4649 | 94 | return output | ||
4650 | 95 | |||
4651 | 96 | |||
4652 | 97 | def add_source(source, key=None): | ||
4653 | 98 | """Add a package source to this system. | ||
4654 | 99 | |||
4655 | 100 | @param source: a URL with a rpm package | ||
4656 | 101 | |||
4657 | 102 | @param key: A key to be added to the system's keyring and used | ||
4658 | 103 | to verify the signatures on packages. Ideally, this should be an | ||
4659 | 104 | ASCII format GPG public key including the block headers. A GPG key | ||
4660 | 105 | id may also be used, but be aware that only insecure protocols are | ||
4661 | 106 | available to retrieve the actual public key from a public keyserver | ||
4662 | 107 | placing your Juju environment at risk. | ||
4663 | 108 | """ | ||
4664 | 109 | if source is None: | ||
4665 | 110 | log('Source is not present. Skipping') | ||
4666 | 111 | return | ||
4667 | 112 | |||
4668 | 113 | if source.startswith('http'): | ||
4669 | 114 | directory = '/etc/yum.repos.d/' | ||
4670 | 115 | for filename in os.listdir(directory): | ||
4671 | 116 | with open(directory + filename, 'r') as rpm_file: | ||
4672 | 117 | if source in rpm_file.read(): | ||
4673 | 118 | break | ||
4674 | 119 | else: | ||
4675 | 120 | log("Add source: {!r}".format(source)) | ||
4676 | 121 | # write in the charms.repo | ||
4677 | 122 | with open(directory + 'Charms.repo', 'a') as rpm_file: | ||
4678 | 123 | rpm_file.write('[%s]\n' % source[7:].replace('/', '_')) | ||
4679 | 124 | rpm_file.write('name=%s\n' % source[7:]) | ||
4680 | 125 | rpm_file.write('baseurl=%s\n\n' % source) | ||
4681 | 126 | else: | ||
4682 | 127 | log("Unknown source: {!r}".format(source)) | ||
4683 | 128 | |||
4684 | 129 | if key: | ||
4685 | 130 | if '-----BEGIN PGP PUBLIC KEY BLOCK-----' in key: | ||
4686 | 131 | with NamedTemporaryFile('w+') as key_file: | ||
4687 | 132 | key_file.write(key) | ||
4688 | 133 | key_file.flush() | ||
4689 | 134 | key_file.seek(0) | ||
4690 | 135 | subprocess.check_call(['rpm', '--import', key_file]) | ||
4691 | 136 | else: | ||
4692 | 137 | subprocess.check_call(['rpm', '--import', key]) | ||
4693 | 138 | |||
4694 | 139 | |||
4695 | 140 | def _run_yum_command(cmd, fatal=False): | ||
4696 | 141 | """Run an YUM command. | ||
4697 | 142 | |||
4698 | 143 | Checks the output and retry if the fatal flag is set to True. | ||
4699 | 144 | |||
4700 | 145 | :param: cmd: str: The yum command to run. | ||
4701 | 146 | :param: fatal: bool: Whether the command's output should be checked and | ||
4702 | 147 | retried. | ||
4703 | 148 | """ | ||
4704 | 149 | env = os.environ.copy() | ||
4705 | 150 | |||
4706 | 151 | if fatal: | ||
4707 | 152 | retry_count = 0 | ||
4708 | 153 | result = None | ||
4709 | 154 | |||
4710 | 155 | # If the command is considered "fatal", we need to retry if the yum | ||
4711 | 156 | # lock was not acquired. | ||
4712 | 157 | |||
4713 | 158 | while result is None or result == YUM_NO_LOCK: | ||
4714 | 159 | try: | ||
4715 | 160 | result = subprocess.check_call(cmd, env=env) | ||
4716 | 161 | except subprocess.CalledProcessError as e: | ||
4717 | 162 | retry_count = retry_count + 1 | ||
4718 | 163 | if retry_count > YUM_NO_LOCK_RETRY_COUNT: | ||
4719 | 164 | raise | ||
4720 | 165 | result = e.returncode | ||
4721 | 166 | log("Couldn't acquire YUM lock. Will retry in {} seconds." | ||
4722 | 167 | "".format(YUM_NO_LOCK_RETRY_DELAY)) | ||
4723 | 168 | time.sleep(YUM_NO_LOCK_RETRY_DELAY) | ||
4724 | 169 | |||
4725 | 170 | else: | ||
4726 | 171 | subprocess.call(cmd, env=env) | ||
4727 | 0 | 172 | ||
4728 | === modified file 'hooks/charmhelpers/fetch/giturl.py' | |||
4729 | --- hooks/charmhelpers/fetch/giturl.py 2014-10-24 16:35:00 +0000 | |||
4730 | +++ hooks/charmhelpers/fetch/giturl.py 2017-05-11 14:56:57 +0000 | |||
4731 | @@ -1,44 +1,69 @@ | |||
4732 | 1 | # Copyright 2014-2015 Canonical Limited. | ||
4733 | 2 | # | ||
4734 | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); | ||
4735 | 4 | # you may not use this file except in compliance with the License. | ||
4736 | 5 | # You may obtain a copy of the License at | ||
4737 | 6 | # | ||
4738 | 7 | # http://www.apache.org/licenses/LICENSE-2.0 | ||
4739 | 8 | # | ||
4740 | 9 | # Unless required by applicable law or agreed to in writing, software | ||
4741 | 10 | # distributed under the License is distributed on an "AS IS" BASIS, | ||
4742 | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
4743 | 12 | # See the License for the specific language governing permissions and | ||
4744 | 13 | # limitations under the License. | ||
4745 | 14 | |||
4746 | 1 | import os | 15 | import os |
4747 | 16 | from subprocess import check_call, CalledProcessError | ||
4748 | 2 | from charmhelpers.fetch import ( | 17 | from charmhelpers.fetch import ( |
4749 | 3 | BaseFetchHandler, | 18 | BaseFetchHandler, |
4751 | 4 | UnhandledSource | 19 | UnhandledSource, |
4752 | 20 | filter_installed_packages, | ||
4753 | 21 | install, | ||
4754 | 5 | ) | 22 | ) |
4755 | 6 | from charmhelpers.core.host import mkdir | ||
4756 | 7 | 23 | ||
4763 | 8 | try: | 24 | if filter_installed_packages(['git']) != []: |
4764 | 9 | from git import Repo | 25 | install(['git']) |
4765 | 10 | except ImportError: | 26 | if filter_installed_packages(['git']) != []: |
4766 | 11 | from charmhelpers.fetch import apt_install | 27 | raise NotImplementedError('Unable to install git') |
4761 | 12 | apt_install("python-git") | ||
4762 | 13 | from git import Repo | ||
4767 | 14 | 28 | ||
4768 | 15 | 29 | ||
4769 | 16 | class GitUrlFetchHandler(BaseFetchHandler): | 30 | class GitUrlFetchHandler(BaseFetchHandler): |
4771 | 17 | """Handler for git branches via generic and github URLs""" | 31 | """Handler for git branches via generic and github URLs.""" |
4772 | 32 | |||
4773 | 18 | def can_handle(self, source): | 33 | def can_handle(self, source): |
4774 | 19 | url_parts = self.parse_url(source) | 34 | url_parts = self.parse_url(source) |
4777 | 20 | #TODO (mattyw) no support for ssh git@ yet | 35 | # TODO (mattyw) no support for ssh git@ yet |
4778 | 21 | if url_parts.scheme not in ('http', 'https', 'git'): | 36 | if url_parts.scheme not in ('http', 'https', 'git', ''): |
4779 | 22 | return False | 37 | return False |
4780 | 38 | elif not url_parts.scheme: | ||
4781 | 39 | return os.path.exists(os.path.join(source, '.git')) | ||
4782 | 23 | else: | 40 | else: |
4783 | 24 | return True | 41 | return True |
4784 | 25 | 42 | ||
4786 | 26 | def clone(self, source, dest, branch): | 43 | def clone(self, source, dest, branch="master", depth=None): |
4787 | 27 | if not self.can_handle(source): | 44 | if not self.can_handle(source): |
4788 | 28 | raise UnhandledSource("Cannot handle {}".format(source)) | 45 | raise UnhandledSource("Cannot handle {}".format(source)) |
4789 | 29 | 46 | ||
4792 | 30 | repo = Repo.clone_from(source, dest) | 47 | if os.path.exists(dest): |
4793 | 31 | repo.git.checkout(branch) | 48 | cmd = ['git', '-C', dest, 'pull', source, branch] |
4794 | 49 | else: | ||
4795 | 50 | cmd = ['git', 'clone', source, dest, '--branch', branch] | ||
4796 | 51 | if depth: | ||
4797 | 52 | cmd.extend(['--depth', depth]) | ||
4798 | 53 | check_call(cmd) | ||
4799 | 32 | 54 | ||
4801 | 33 | def install(self, source, branch="master"): | 55 | def install(self, source, branch="master", dest=None, depth=None): |
4802 | 34 | url_parts = self.parse_url(source) | 56 | url_parts = self.parse_url(source) |
4803 | 35 | branch_name = url_parts.path.strip("/").split("/")[-1] | 57 | branch_name = url_parts.path.strip("/").split("/")[-1] |
4808 | 36 | dest_dir = os.path.join(os.environ.get('CHARM_DIR'), "fetched", | 58 | if dest: |
4809 | 37 | branch_name) | 59 | dest_dir = os.path.join(dest, branch_name) |
4810 | 38 | if not os.path.exists(dest_dir): | 60 | else: |
4811 | 39 | mkdir(dest_dir, perms=0755) | 61 | dest_dir = os.path.join(os.environ.get('CHARM_DIR'), "fetched", |
4812 | 62 | branch_name) | ||
4813 | 40 | try: | 63 | try: |
4815 | 41 | self.clone(source, dest_dir, branch) | 64 | self.clone(source, dest_dir, branch, depth) |
4816 | 65 | except CalledProcessError as e: | ||
4817 | 66 | raise UnhandledSource(e) | ||
4818 | 42 | except OSError as e: | 67 | except OSError as e: |
4819 | 43 | raise UnhandledSource(e.strerror) | 68 | raise UnhandledSource(e.strerror) |
4820 | 44 | return dest_dir | 69 | return dest_dir |
4821 | 45 | 70 | ||
4822 | === added file 'hooks/charmhelpers/fetch/snap.py' | |||
4823 | --- hooks/charmhelpers/fetch/snap.py 1970-01-01 00:00:00 +0000 | |||
4824 | +++ hooks/charmhelpers/fetch/snap.py 2017-05-11 14:56:57 +0000 | |||
4825 | @@ -0,0 +1,122 @@ | |||
4826 | 1 | # Copyright 2014-2017 Canonical Limited. | ||
4827 | 2 | # | ||
4828 | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); | ||
4829 | 4 | # you may not use this file except in compliance with the License. | ||
4830 | 5 | # You may obtain a copy of the License at | ||
4831 | 6 | # | ||
4832 | 7 | # http://www.apache.org/licenses/LICENSE-2.0 | ||
4833 | 8 | # | ||
4834 | 9 | # Unless required by applicable law or agreed to in writing, software | ||
4835 | 10 | # distributed under the License is distributed on an "AS IS" BASIS, | ||
4836 | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
4837 | 12 | # See the License for the specific language governing permissions and | ||
4838 | 13 | # limitations under the License. | ||
4839 | 14 | """ | ||
4840 | 15 | Charm helpers snap for classic charms. | ||
4841 | 16 | |||
4842 | 17 | If writing reactive charms, use the snap layer: | ||
4843 | 18 | https://lists.ubuntu.com/archives/snapcraft/2016-September/001114.html | ||
4844 | 19 | """ | ||
4845 | 20 | import subprocess | ||
4846 | 21 | from os import environ | ||
4847 | 22 | from time import sleep | ||
4848 | 23 | from charmhelpers.core.hookenv import log | ||
4849 | 24 | |||
4850 | 25 | __author__ = 'Joseph Borg <joseph.borg@canonical.com>' | ||
4851 | 26 | |||
4852 | 27 | SNAP_NO_LOCK = 1 # The return code for "couldn't acquire lock" in Snap (hopefully this will be improved). | ||
4853 | 28 | SNAP_NO_LOCK_RETRY_DELAY = 10 # Wait X seconds between Snap lock checks. | ||
4854 | 29 | SNAP_NO_LOCK_RETRY_COUNT = 30 # Retry to acquire the lock X times. | ||
4855 | 30 | |||
4856 | 31 | |||
4857 | 32 | class CouldNotAcquireLockException(Exception): | ||
4858 | 33 | pass | ||
4859 | 34 | |||
4860 | 35 | |||
4861 | 36 | def _snap_exec(commands): | ||
4862 | 37 | """ | ||
4863 | 38 | Execute snap commands. | ||
4864 | 39 | |||
4865 | 40 | :param commands: List commands | ||
4866 | 41 | :return: Integer exit code | ||
4867 | 42 | """ | ||
4868 | 43 | assert type(commands) == list | ||
4869 | 44 | |||
4870 | 45 | retry_count = 0 | ||
4871 | 46 | return_code = None | ||
4872 | 47 | |||
4873 | 48 | while return_code is None or return_code == SNAP_NO_LOCK: | ||
4874 | 49 | try: | ||
4875 | 50 | return_code = subprocess.check_call(['snap'] + commands, env=environ) | ||
4876 | 51 | except subprocess.CalledProcessError as e: | ||
4877 | 52 | retry_count += + 1 | ||
4878 | 53 | if retry_count > SNAP_NO_LOCK_RETRY_COUNT: | ||
4879 | 54 | raise CouldNotAcquireLockException('Could not aquire lock after %s attempts' % SNAP_NO_LOCK_RETRY_COUNT) | ||
4880 | 55 | return_code = e.returncode | ||
4881 | 56 | log('Snap failed to acquire lock, trying again in %s seconds.' % SNAP_NO_LOCK_RETRY_DELAY, level='WARN') | ||
4882 | 57 | sleep(SNAP_NO_LOCK_RETRY_DELAY) | ||
4883 | 58 | |||
4884 | 59 | return return_code | ||
4885 | 60 | |||
4886 | 61 | |||
4887 | 62 | def snap_install(packages, *flags): | ||
4888 | 63 | """ | ||
4889 | 64 | Install a snap package. | ||
4890 | 65 | |||
4891 | 66 | :param packages: String or List String package name | ||
4892 | 67 | :param flags: List String flags to pass to install command | ||
4893 | 68 | :return: Integer return code from snap | ||
4894 | 69 | """ | ||
4895 | 70 | if type(packages) is not list: | ||
4896 | 71 | packages = [packages] | ||
4897 | 72 | |||
4898 | 73 | flags = list(flags) | ||
4899 | 74 | |||
4900 | 75 | message = 'Installing snap(s) "%s"' % ', '.join(packages) | ||
4901 | 76 | if flags: | ||
4902 | 77 | message += ' with option(s) "%s"' % ', '.join(flags) | ||
4903 | 78 | |||
4904 | 79 | log(message, level='INFO') | ||
4905 | 80 | return _snap_exec(['install'] + flags + packages) | ||
4906 | 81 | |||
4907 | 82 | |||
4908 | 83 | def snap_remove(packages, *flags): | ||
4909 | 84 | """ | ||
4910 | 85 | Remove a snap package. | ||
4911 | 86 | |||
4912 | 87 | :param packages: String or List String package name | ||
4913 | 88 | :param flags: List String flags to pass to remove command | ||
4914 | 89 | :return: Integer return code from snap | ||
4915 | 90 | """ | ||
4916 | 91 | if type(packages) is not list: | ||
4917 | 92 | packages = [packages] | ||
4918 | 93 | |||
4919 | 94 | flags = list(flags) | ||
4920 | 95 | |||
4921 | 96 | message = 'Removing snap(s) "%s"' % ', '.join(packages) | ||
4922 | 97 | if flags: | ||
4923 | 98 | message += ' with options "%s"' % ', '.join(flags) | ||
4924 | 99 | |||
4925 | 100 | log(message, level='INFO') | ||
4926 | 101 | return _snap_exec(['remove'] + flags + packages) | ||
4927 | 102 | |||
4928 | 103 | |||
4929 | 104 | def snap_refresh(packages, *flags): | ||
4930 | 105 | """ | ||
4931 | 106 | Refresh / Update snap package. | ||
4932 | 107 | |||
4933 | 108 | :param packages: String or List String package name | ||
4934 | 109 | :param flags: List String flags to pass to refresh command | ||
4935 | 110 | :return: Integer return code from snap | ||
4936 | 111 | """ | ||
4937 | 112 | if type(packages) is not list: | ||
4938 | 113 | packages = [packages] | ||
4939 | 114 | |||
4940 | 115 | flags = list(flags) | ||
4941 | 116 | |||
4942 | 117 | message = 'Refreshing snap(s) "%s"' % ', '.join(packages) | ||
4943 | 118 | if flags: | ||
4944 | 119 | message += ' with options "%s"' % ', '.join(flags) | ||
4945 | 120 | |||
4946 | 121 | log(message, level='INFO') | ||
4947 | 122 | return _snap_exec(['refresh'] + flags + packages) | ||
4948 | 0 | 123 | ||
4949 | === added file 'hooks/charmhelpers/fetch/ubuntu.py' | |||
4950 | --- hooks/charmhelpers/fetch/ubuntu.py 1970-01-01 00:00:00 +0000 | |||
4951 | +++ hooks/charmhelpers/fetch/ubuntu.py 2017-05-11 14:56:57 +0000 | |||
4952 | @@ -0,0 +1,364 @@ | |||
4953 | 1 | # Copyright 2014-2015 Canonical Limited. | ||
4954 | 2 | # | ||
4955 | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); | ||
4956 | 4 | # you may not use this file except in compliance with the License. | ||
4957 | 5 | # You may obtain a copy of the License at | ||
4958 | 6 | # | ||
4959 | 7 | # http://www.apache.org/licenses/LICENSE-2.0 | ||
4960 | 8 | # | ||
4961 | 9 | # Unless required by applicable law or agreed to in writing, software | ||
4962 | 10 | # distributed under the License is distributed on an "AS IS" BASIS, | ||
4963 | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
4964 | 12 | # See the License for the specific language governing permissions and | ||
4965 | 13 | # limitations under the License. | ||
4966 | 14 | |||
4967 | 15 | import os | ||
4968 | 16 | import six | ||
4969 | 17 | import time | ||
4970 | 18 | import subprocess | ||
4971 | 19 | |||
4972 | 20 | from tempfile import NamedTemporaryFile | ||
4973 | 21 | from charmhelpers.core.host import ( | ||
4974 | 22 | lsb_release | ||
4975 | 23 | ) | ||
4976 | 24 | from charmhelpers.core.hookenv import log | ||
4977 | 25 | from charmhelpers.fetch import SourceConfigError | ||
4978 | 26 | |||
4979 | 27 | CLOUD_ARCHIVE = """# Ubuntu Cloud Archive | ||
4980 | 28 | deb http://ubuntu-cloud.archive.canonical.com/ubuntu {} main | ||
4981 | 29 | """ | ||
4982 | 30 | |||
4983 | 31 | PROPOSED_POCKET = """# Proposed | ||
4984 | 32 | deb http://archive.ubuntu.com/ubuntu {}-proposed main universe multiverse restricted | ||
4985 | 33 | """ | ||
4986 | 34 | |||
4987 | 35 | CLOUD_ARCHIVE_POCKETS = { | ||
4988 | 36 | # Folsom | ||
4989 | 37 | 'folsom': 'precise-updates/folsom', | ||
4990 | 38 | 'precise-folsom': 'precise-updates/folsom', | ||
4991 | 39 | 'precise-folsom/updates': 'precise-updates/folsom', | ||
4992 | 40 | 'precise-updates/folsom': 'precise-updates/folsom', | ||
4993 | 41 | 'folsom/proposed': 'precise-proposed/folsom', | ||
4994 | 42 | 'precise-folsom/proposed': 'precise-proposed/folsom', | ||
4995 | 43 | 'precise-proposed/folsom': 'precise-proposed/folsom', | ||
4996 | 44 | # Grizzly | ||
4997 | 45 | 'grizzly': 'precise-updates/grizzly', | ||
4998 | 46 | 'precise-grizzly': 'precise-updates/grizzly', | ||
4999 | 47 | 'precise-grizzly/updates': 'precise-updates/grizzly', | ||
5000 | 48 | 'precise-updates/grizzly': 'precise-updates/grizzly', |
You might want to drop changes from README.md. This charm obviously doesn't work with outdated Kibana charm, so new Kibana charm is being proposed. That charm is available at https:/ /code.launchpad .net/~ivoks/ charms/ xenial/ kibana/ kibana- xenial and is pending charm publication in charmstore.