Merge lp:~bloodearnest/charm-haproxy/merge-trunk-snapstore into lp:~ubuntuone-pqm-team/charm-haproxy/snap-store
- merge-trunk-snapstore
- Merge into snap-store
Proposed by
Simon Davy
Status: | Work in progress |
---|---|
Proposed branch: | lp:~bloodearnest/charm-haproxy/merge-trunk-snapstore |
Merge into: | lp:~ubuntuone-pqm-team/charm-haproxy/snap-store |
Diff against target: |
5380 lines (+3150/-1126) 38 files modified
README.md (+6/-5) charm-helpers.yaml (+2/-1) config.yaml (+1/-1) hooks/charmhelpers/__init__.py (+11/-13) hooks/charmhelpers/contrib/__init__.py (+11/-13) hooks/charmhelpers/contrib/charmsupport/__init__.py (+11/-13) hooks/charmhelpers/contrib/charmsupport/nrpe.py (+106/-34) hooks/charmhelpers/contrib/charmsupport/volumes.py (+11/-13) hooks/charmhelpers/core/__init__.py (+11/-13) hooks/charmhelpers/core/decorators.py (+11/-13) hooks/charmhelpers/core/files.py (+43/-0) hooks/charmhelpers/core/fstab.py (+11/-13) hooks/charmhelpers/core/hookenv.py (+448/-63) hooks/charmhelpers/core/host.py (+625/-155) hooks/charmhelpers/core/host_factory/centos.py (+56/-0) hooks/charmhelpers/core/host_factory/ubuntu.py (+56/-0) hooks/charmhelpers/core/hugepage.py (+69/-0) hooks/charmhelpers/core/kernel.py (+72/-0) hooks/charmhelpers/core/kernel_factory/centos.py (+17/-0) hooks/charmhelpers/core/kernel_factory/ubuntu.py (+13/-0) hooks/charmhelpers/core/services/__init__.py (+11/-13) hooks/charmhelpers/core/services/base.py (+52/-30) hooks/charmhelpers/core/services/helpers.py (+42/-19) hooks/charmhelpers/core/strutils.py (+41/-13) hooks/charmhelpers/core/sysctl.py (+11/-13) hooks/charmhelpers/core/templating.py (+40/-24) hooks/charmhelpers/core/unitdata.py (+72/-31) hooks/charmhelpers/fetch/__init__.py (+46/-296) hooks/charmhelpers/fetch/archiveurl.py (+19/-15) hooks/charmhelpers/fetch/bzrurl.py (+48/-50) hooks/charmhelpers/fetch/centos.py (+171/-0) hooks/charmhelpers/fetch/giturl.py (+37/-39) hooks/charmhelpers/fetch/snap.py (+122/-0) hooks/charmhelpers/fetch/ubuntu.py (+364/-0) hooks/charmhelpers/osplatform.py (+25/-0) hooks/hooks.py (+1/-1) icon.svg (+456/-232) metadata.yaml (+1/-0) |
To merge this branch: | bzr merge lp:~bloodearnest/charm-haproxy/merge-trunk-snapstore |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Guillermo Gonzalez | Approve | ||
haproxy-team | Pending | ||
Review via email: mp+347844@code.launchpad.net |
Commit message
Merge trunk for autocert support
Description of the change
To post a comment you must log in.
Unmerged revisions
- 113. By Simon Davy
-
merge trunk r114, with updated charm-helpers and autocert support
Preview Diff
[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1 | === modified file 'README.md' |
2 | --- README.md 2015-06-10 14:38:26 +0000 |
3 | +++ README.md 2018-06-12 20:07:13 +0000 |
4 | @@ -1,6 +1,7 @@ |
5 | # Overview |
6 | |
7 | -This charm deploys a reverse proxy in front of other servies. You can use this to load balance existing deployments. |
8 | +This charm deploys a reverse proxy in front of other services. You can use |
9 | +this to load balance existing deployments. |
10 | |
11 | # Usage |
12 | |
13 | @@ -110,7 +111,7 @@ |
14 | charms written like apache2 that can act as a front-end for haproxy to take of |
15 | things like ssl encryption. When joining a service like apache2 on its |
16 | reverseproxy relation, haproxy's website relation will set an `all_services` |
17 | -varaible that conforms to the spec layed out in the apache2 charm. |
18 | +variable that conforms to the spec laid out in the apache2 charm. |
19 | |
20 | These settings can then be used when crafting your vhost template to make sure |
21 | traffic goes to the correct haproxy listener which will in turn forward the |
22 | @@ -136,7 +137,7 @@ |
23 | - { ... optionally more services here ... } |
24 | " |
25 | |
26 | -where the DEFAULT keyword means use the certificate set with `ssl_cert`/`ssl_key` (or |
27 | +The DEFAULT keyword means use the certificate set with `ssl_cert`/`ssl_key` (or |
28 | alternatively you can inline different base64-encode certificates). |
29 | |
30 | Note that in order to use SSL termination you need haproxy 1.5 or later, which |
31 | @@ -227,9 +228,9 @@ |
32 | ### active-active mode |
33 | |
34 | If the peering\_mode option is set to "active-active", then any haproxy unit |
35 | -will be independant from each other and will simply load-balance the traffic to |
36 | +will be independent from each other and will simply load-balance the traffic to |
37 | the backends. In this case, the indirection layer described above is not |
38 | -created in this case. |
39 | +created. |
40 | |
41 | This mode allows increasing the bandwidth to the backends by adding additional |
42 | units, at the cost of having less control over the number of connections that |
43 | |
44 | === modified file 'charm-helpers.yaml' |
45 | --- charm-helpers.yaml 2013-08-21 19:19:29 +0000 |
46 | +++ charm-helpers.yaml 2018-06-12 20:07:13 +0000 |
47 | @@ -1,4 +1,5 @@ |
48 | include: |
49 | - core |
50 | - fetch |
51 | - - contrib.charmsupport |
52 | \ No newline at end of file |
53 | + - contrib.charmsupport |
54 | + - osplatform |
55 | |
56 | === modified file 'config.yaml' |
57 | --- config.yaml 2016-11-28 04:37:08 +0000 |
58 | +++ config.yaml 2018-06-12 20:07:13 +0000 |
59 | @@ -49,7 +49,7 @@ |
60 | greater than 1024 bits are not supported by Java 7 and earlier clients. This |
61 | config key will be ignored if the installed haproxy package has no SSL support. |
62 | global_default_bind_ciphers: |
63 | - default: ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA |
64 | + default: ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:!DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:!DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:!CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA |
65 | type: string |
66 | description: | |
67 | Sets the default string describing the list of cipher algorithms |
68 | |
69 | === added file 'hooks/__init__.py' |
70 | === modified file 'hooks/charmhelpers/__init__.py' |
71 | --- hooks/charmhelpers/__init__.py 2015-02-09 12:53:57 +0000 |
72 | +++ hooks/charmhelpers/__init__.py 2018-06-12 20:07:13 +0000 |
73 | @@ -1,18 +1,16 @@ |
74 | # Copyright 2014-2015 Canonical Limited. |
75 | # |
76 | -# This file is part of charm-helpers. |
77 | -# |
78 | -# charm-helpers is free software: you can redistribute it and/or modify |
79 | -# it under the terms of the GNU Lesser General Public License version 3 as |
80 | -# published by the Free Software Foundation. |
81 | -# |
82 | -# charm-helpers is distributed in the hope that it will be useful, |
83 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
84 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
85 | -# GNU Lesser General Public License for more details. |
86 | -# |
87 | -# You should have received a copy of the GNU Lesser General Public License |
88 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
89 | +# Licensed under the Apache License, Version 2.0 (the "License"); |
90 | +# you may not use this file except in compliance with the License. |
91 | +# You may obtain a copy of the License at |
92 | +# |
93 | +# http://www.apache.org/licenses/LICENSE-2.0 |
94 | +# |
95 | +# Unless required by applicable law or agreed to in writing, software |
96 | +# distributed under the License is distributed on an "AS IS" BASIS, |
97 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
98 | +# See the License for the specific language governing permissions and |
99 | +# limitations under the License. |
100 | |
101 | # Bootstrap charm-helpers, installing its dependencies if necessary using |
102 | # only standard libraries. |
103 | |
104 | === modified file 'hooks/charmhelpers/contrib/__init__.py' |
105 | --- hooks/charmhelpers/contrib/__init__.py 2015-02-09 12:53:57 +0000 |
106 | +++ hooks/charmhelpers/contrib/__init__.py 2018-06-12 20:07:13 +0000 |
107 | @@ -1,15 +1,13 @@ |
108 | # Copyright 2014-2015 Canonical Limited. |
109 | # |
110 | -# This file is part of charm-helpers. |
111 | -# |
112 | -# charm-helpers is free software: you can redistribute it and/or modify |
113 | -# it under the terms of the GNU Lesser General Public License version 3 as |
114 | -# published by the Free Software Foundation. |
115 | -# |
116 | -# charm-helpers is distributed in the hope that it will be useful, |
117 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
118 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
119 | -# GNU Lesser General Public License for more details. |
120 | -# |
121 | -# You should have received a copy of the GNU Lesser General Public License |
122 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
123 | +# Licensed under the Apache License, Version 2.0 (the "License"); |
124 | +# you may not use this file except in compliance with the License. |
125 | +# You may obtain a copy of the License at |
126 | +# |
127 | +# http://www.apache.org/licenses/LICENSE-2.0 |
128 | +# |
129 | +# Unless required by applicable law or agreed to in writing, software |
130 | +# distributed under the License is distributed on an "AS IS" BASIS, |
131 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
132 | +# See the License for the specific language governing permissions and |
133 | +# limitations under the License. |
134 | |
135 | === modified file 'hooks/charmhelpers/contrib/charmsupport/__init__.py' |
136 | --- hooks/charmhelpers/contrib/charmsupport/__init__.py 2015-02-09 12:53:57 +0000 |
137 | +++ hooks/charmhelpers/contrib/charmsupport/__init__.py 2018-06-12 20:07:13 +0000 |
138 | @@ -1,15 +1,13 @@ |
139 | # Copyright 2014-2015 Canonical Limited. |
140 | # |
141 | -# This file is part of charm-helpers. |
142 | -# |
143 | -# charm-helpers is free software: you can redistribute it and/or modify |
144 | -# it under the terms of the GNU Lesser General Public License version 3 as |
145 | -# published by the Free Software Foundation. |
146 | -# |
147 | -# charm-helpers is distributed in the hope that it will be useful, |
148 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
149 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
150 | -# GNU Lesser General Public License for more details. |
151 | -# |
152 | -# You should have received a copy of the GNU Lesser General Public License |
153 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
154 | +# Licensed under the Apache License, Version 2.0 (the "License"); |
155 | +# you may not use this file except in compliance with the License. |
156 | +# You may obtain a copy of the License at |
157 | +# |
158 | +# http://www.apache.org/licenses/LICENSE-2.0 |
159 | +# |
160 | +# Unless required by applicable law or agreed to in writing, software |
161 | +# distributed under the License is distributed on an "AS IS" BASIS, |
162 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
163 | +# See the License for the specific language governing permissions and |
164 | +# limitations under the License. |
165 | |
166 | === modified file 'hooks/charmhelpers/contrib/charmsupport/nrpe.py' |
167 | --- hooks/charmhelpers/contrib/charmsupport/nrpe.py 2015-05-14 10:48:09 +0000 |
168 | +++ hooks/charmhelpers/contrib/charmsupport/nrpe.py 2018-06-12 20:07:13 +0000 |
169 | @@ -1,18 +1,16 @@ |
170 | # Copyright 2014-2015 Canonical Limited. |
171 | # |
172 | -# This file is part of charm-helpers. |
173 | -# |
174 | -# charm-helpers is free software: you can redistribute it and/or modify |
175 | -# it under the terms of the GNU Lesser General Public License version 3 as |
176 | -# published by the Free Software Foundation. |
177 | -# |
178 | -# charm-helpers is distributed in the hope that it will be useful, |
179 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
180 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
181 | -# GNU Lesser General Public License for more details. |
182 | -# |
183 | -# You should have received a copy of the GNU Lesser General Public License |
184 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
185 | +# Licensed under the Apache License, Version 2.0 (the "License"); |
186 | +# you may not use this file except in compliance with the License. |
187 | +# You may obtain a copy of the License at |
188 | +# |
189 | +# http://www.apache.org/licenses/LICENSE-2.0 |
190 | +# |
191 | +# Unless required by applicable law or agreed to in writing, software |
192 | +# distributed under the License is distributed on an "AS IS" BASIS, |
193 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
194 | +# See the License for the specific language governing permissions and |
195 | +# limitations under the License. |
196 | |
197 | """Compatibility with the nrpe-external-master charm""" |
198 | # Copyright 2012 Canonical Ltd. |
199 | @@ -40,6 +38,7 @@ |
200 | ) |
201 | |
202 | from charmhelpers.core.host import service |
203 | +from charmhelpers.core import host |
204 | |
205 | # This module adds compatibility with the nrpe-external-master and plain nrpe |
206 | # subordinate charms. To use it in your charm: |
207 | @@ -110,6 +109,13 @@ |
208 | # def local_monitors_relation_changed(): |
209 | # update_nrpe_config() |
210 | # |
211 | +# 4.a If your charm is a subordinate charm set primary=False |
212 | +# |
213 | +# from charmsupport.nrpe import NRPE |
214 | +# (...) |
215 | +# def update_nrpe_config(): |
216 | +# nrpe_compat = NRPE(primary=False) |
217 | +# |
218 | # 5. ln -s hooks.py nrpe-external-master-relation-changed |
219 | # ln -s hooks.py local-monitors-relation-changed |
220 | |
221 | @@ -148,6 +154,13 @@ |
222 | self.description = description |
223 | self.check_cmd = self._locate_cmd(check_cmd) |
224 | |
225 | + def _get_check_filename(self): |
226 | + return os.path.join(NRPE.nrpe_confdir, '{}.cfg'.format(self.command)) |
227 | + |
228 | + def _get_service_filename(self, hostname): |
229 | + return os.path.join(NRPE.nagios_exportdir, |
230 | + 'service__{}_{}.cfg'.format(hostname, self.command)) |
231 | + |
232 | def _locate_cmd(self, check_cmd): |
233 | search_path = ( |
234 | '/usr/lib/nagios/plugins', |
235 | @@ -163,9 +176,21 @@ |
236 | log('Check command not found: {}'.format(parts[0])) |
237 | return '' |
238 | |
239 | + def _remove_service_files(self): |
240 | + if not os.path.exists(NRPE.nagios_exportdir): |
241 | + return |
242 | + for f in os.listdir(NRPE.nagios_exportdir): |
243 | + if f.endswith('_{}.cfg'.format(self.command)): |
244 | + os.remove(os.path.join(NRPE.nagios_exportdir, f)) |
245 | + |
246 | + def remove(self, hostname): |
247 | + nrpe_check_file = self._get_check_filename() |
248 | + if os.path.exists(nrpe_check_file): |
249 | + os.remove(nrpe_check_file) |
250 | + self._remove_service_files() |
251 | + |
252 | def write(self, nagios_context, hostname, nagios_servicegroups): |
253 | - nrpe_check_file = '/etc/nagios/nrpe.d/{}.cfg'.format( |
254 | - self.command) |
255 | + nrpe_check_file = self._get_check_filename() |
256 | with open(nrpe_check_file, 'w') as nrpe_check_config: |
257 | nrpe_check_config.write("# check {}\n".format(self.shortname)) |
258 | nrpe_check_config.write("command[{}]={}\n".format( |
259 | @@ -180,9 +205,7 @@ |
260 | |
261 | def write_service_config(self, nagios_context, hostname, |
262 | nagios_servicegroups): |
263 | - for f in os.listdir(NRPE.nagios_exportdir): |
264 | - if re.search('.*{}.cfg'.format(self.command), f): |
265 | - os.remove(os.path.join(NRPE.nagios_exportdir, f)) |
266 | + self._remove_service_files() |
267 | |
268 | templ_vars = { |
269 | 'nagios_hostname': hostname, |
270 | @@ -192,8 +215,7 @@ |
271 | 'command': self.command, |
272 | } |
273 | nrpe_service_text = Check.service_template.format(**templ_vars) |
274 | - nrpe_service_file = '{}/service__{}_{}.cfg'.format( |
275 | - NRPE.nagios_exportdir, hostname, self.command) |
276 | + nrpe_service_file = self._get_service_filename(hostname) |
277 | with open(nrpe_service_file, 'w') as nrpe_service_config: |
278 | nrpe_service_config.write(str(nrpe_service_text)) |
279 | |
280 | @@ -205,10 +227,12 @@ |
281 | nagios_logdir = '/var/log/nagios' |
282 | nagios_exportdir = '/var/lib/nagios/export' |
283 | nrpe_confdir = '/etc/nagios/nrpe.d' |
284 | + homedir = '/var/lib/nagios' # home dir provided by nagios-nrpe-server |
285 | |
286 | - def __init__(self, hostname=None): |
287 | + def __init__(self, hostname=None, primary=True): |
288 | super(NRPE, self).__init__() |
289 | self.config = config() |
290 | + self.primary = primary |
291 | self.nagios_context = self.config['nagios_context'] |
292 | if 'nagios_servicegroups' in self.config and self.config['nagios_servicegroups']: |
293 | self.nagios_servicegroups = self.config['nagios_servicegroups'] |
294 | @@ -218,12 +242,38 @@ |
295 | if hostname: |
296 | self.hostname = hostname |
297 | else: |
298 | - self.hostname = "{}-{}".format(self.nagios_context, self.unit_name) |
299 | + nagios_hostname = get_nagios_hostname() |
300 | + if nagios_hostname: |
301 | + self.hostname = nagios_hostname |
302 | + else: |
303 | + self.hostname = "{}-{}".format(self.nagios_context, self.unit_name) |
304 | self.checks = [] |
305 | + # Iff in an nrpe-external-master relation hook, set primary status |
306 | + relation = relation_ids('nrpe-external-master') |
307 | + if relation: |
308 | + log("Setting charm primary status {}".format(primary)) |
309 | + for rid in relation_ids('nrpe-external-master'): |
310 | + relation_set(relation_id=rid, relation_settings={'primary': self.primary}) |
311 | |
312 | def add_check(self, *args, **kwargs): |
313 | self.checks.append(Check(*args, **kwargs)) |
314 | |
315 | + def remove_check(self, *args, **kwargs): |
316 | + if kwargs.get('shortname') is None: |
317 | + raise ValueError('shortname of check must be specified') |
318 | + |
319 | + # Use sensible defaults if they're not specified - these are not |
320 | + # actually used during removal, but they're required for constructing |
321 | + # the Check object; check_disk is chosen because it's part of the |
322 | + # nagios-plugins-basic package. |
323 | + if kwargs.get('check_cmd') is None: |
324 | + kwargs['check_cmd'] = 'check_disk' |
325 | + if kwargs.get('description') is None: |
326 | + kwargs['description'] = '' |
327 | + |
328 | + check = Check(*args, **kwargs) |
329 | + check.remove(self.hostname) |
330 | + |
331 | def write(self): |
332 | try: |
333 | nagios_uid = pwd.getpwnam('nagios').pw_uid |
334 | @@ -260,7 +310,7 @@ |
335 | :param str relation_name: Name of relation nrpe sub joined to |
336 | """ |
337 | for rel in relations_of_type(relation_name): |
338 | - if 'nagios_hostname' in rel: |
339 | + if 'nagios_host_context' in rel: |
340 | return rel['nagios_host_context'] |
341 | |
342 | |
343 | @@ -289,18 +339,30 @@ |
344 | return unit |
345 | |
346 | |
347 | -def add_init_service_checks(nrpe, services, unit_name): |
348 | +def add_init_service_checks(nrpe, services, unit_name, immediate_check=True): |
349 | """ |
350 | Add checks for each service in list |
351 | |
352 | :param NRPE nrpe: NRPE object to add check to |
353 | :param list services: List of services to check |
354 | :param str unit_name: Unit name to use in check description |
355 | + :param bool immediate_check: For sysv init, run the service check immediately |
356 | """ |
357 | for svc in services: |
358 | + # Don't add a check for these services from neutron-gateway |
359 | + if svc in ['ext-port', 'os-charm-phy-nic-mtu']: |
360 | + next |
361 | + |
362 | upstart_init = '/etc/init/%s.conf' % svc |
363 | sysv_init = '/etc/init.d/%s' % svc |
364 | - if os.path.exists(upstart_init): |
365 | + |
366 | + if host.init_is_systemd(): |
367 | + nrpe.add_check( |
368 | + shortname=svc, |
369 | + description='process check {%s}' % unit_name, |
370 | + check_cmd='check_systemd.py %s' % svc |
371 | + ) |
372 | + elif os.path.exists(upstart_init): |
373 | nrpe.add_check( |
374 | shortname=svc, |
375 | description='process check {%s}' % unit_name, |
376 | @@ -308,21 +370,31 @@ |
377 | ) |
378 | elif os.path.exists(sysv_init): |
379 | cronpath = '/etc/cron.d/nagios-service-check-%s' % svc |
380 | - cron_file = ('*/5 * * * * root ' |
381 | - '/usr/local/lib/nagios/plugins/check_exit_status.pl ' |
382 | - '-s /etc/init.d/%s status > ' |
383 | - '/var/lib/nagios/service-check-%s.txt\n' % (svc, |
384 | - svc) |
385 | - ) |
386 | + checkpath = '%s/service-check-%s.txt' % (nrpe.homedir, svc) |
387 | + croncmd = ( |
388 | + '/usr/local/lib/nagios/plugins/check_exit_status.pl ' |
389 | + '-s /etc/init.d/%s status' % svc |
390 | + ) |
391 | + cron_file = '*/5 * * * * root %s > %s\n' % (croncmd, checkpath) |
392 | f = open(cronpath, 'w') |
393 | f.write(cron_file) |
394 | f.close() |
395 | nrpe.add_check( |
396 | shortname=svc, |
397 | - description='process check {%s}' % unit_name, |
398 | - check_cmd='check_status_file.py -f ' |
399 | - '/var/lib/nagios/service-check-%s.txt' % svc, |
400 | + description='service check {%s}' % unit_name, |
401 | + check_cmd='check_status_file.py -f %s' % checkpath, |
402 | ) |
403 | + # if /var/lib/nagios doesn't exist open(checkpath, 'w') will fail |
404 | + # (LP: #1670223). |
405 | + if immediate_check and os.path.isdir(nrpe.homedir): |
406 | + f = open(checkpath, 'w') |
407 | + subprocess.call( |
408 | + croncmd.split(), |
409 | + stdout=f, |
410 | + stderr=subprocess.STDOUT |
411 | + ) |
412 | + f.close() |
413 | + os.chmod(checkpath, 0o644) |
414 | |
415 | |
416 | def copy_nrpe_checks(): |
417 | |
418 | === modified file 'hooks/charmhelpers/contrib/charmsupport/volumes.py' |
419 | --- hooks/charmhelpers/contrib/charmsupport/volumes.py 2015-02-09 12:53:57 +0000 |
420 | +++ hooks/charmhelpers/contrib/charmsupport/volumes.py 2018-06-12 20:07:13 +0000 |
421 | @@ -1,18 +1,16 @@ |
422 | # Copyright 2014-2015 Canonical Limited. |
423 | # |
424 | -# This file is part of charm-helpers. |
425 | -# |
426 | -# charm-helpers is free software: you can redistribute it and/or modify |
427 | -# it under the terms of the GNU Lesser General Public License version 3 as |
428 | -# published by the Free Software Foundation. |
429 | -# |
430 | -# charm-helpers is distributed in the hope that it will be useful, |
431 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
432 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
433 | -# GNU Lesser General Public License for more details. |
434 | -# |
435 | -# You should have received a copy of the GNU Lesser General Public License |
436 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
437 | +# Licensed under the Apache License, Version 2.0 (the "License"); |
438 | +# you may not use this file except in compliance with the License. |
439 | +# You may obtain a copy of the License at |
440 | +# |
441 | +# http://www.apache.org/licenses/LICENSE-2.0 |
442 | +# |
443 | +# Unless required by applicable law or agreed to in writing, software |
444 | +# distributed under the License is distributed on an "AS IS" BASIS, |
445 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
446 | +# See the License for the specific language governing permissions and |
447 | +# limitations under the License. |
448 | |
449 | ''' |
450 | Functions for managing volumes in juju units. One volume is supported per unit. |
451 | |
452 | === modified file 'hooks/charmhelpers/core/__init__.py' |
453 | --- hooks/charmhelpers/core/__init__.py 2015-02-09 12:53:57 +0000 |
454 | +++ hooks/charmhelpers/core/__init__.py 2018-06-12 20:07:13 +0000 |
455 | @@ -1,15 +1,13 @@ |
456 | # Copyright 2014-2015 Canonical Limited. |
457 | # |
458 | -# This file is part of charm-helpers. |
459 | -# |
460 | -# charm-helpers is free software: you can redistribute it and/or modify |
461 | -# it under the terms of the GNU Lesser General Public License version 3 as |
462 | -# published by the Free Software Foundation. |
463 | -# |
464 | -# charm-helpers is distributed in the hope that it will be useful, |
465 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
466 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
467 | -# GNU Lesser General Public License for more details. |
468 | -# |
469 | -# You should have received a copy of the GNU Lesser General Public License |
470 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
471 | +# Licensed under the Apache License, Version 2.0 (the "License"); |
472 | +# you may not use this file except in compliance with the License. |
473 | +# You may obtain a copy of the License at |
474 | +# |
475 | +# http://www.apache.org/licenses/LICENSE-2.0 |
476 | +# |
477 | +# Unless required by applicable law or agreed to in writing, software |
478 | +# distributed under the License is distributed on an "AS IS" BASIS, |
479 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
480 | +# See the License for the specific language governing permissions and |
481 | +# limitations under the License. |
482 | |
483 | === modified file 'hooks/charmhelpers/core/decorators.py' |
484 | --- hooks/charmhelpers/core/decorators.py 2015-02-09 12:58:07 +0000 |
485 | +++ hooks/charmhelpers/core/decorators.py 2018-06-12 20:07:13 +0000 |
486 | @@ -1,18 +1,16 @@ |
487 | # Copyright 2014-2015 Canonical Limited. |
488 | # |
489 | -# This file is part of charm-helpers. |
490 | -# |
491 | -# charm-helpers is free software: you can redistribute it and/or modify |
492 | -# it under the terms of the GNU Lesser General Public License version 3 as |
493 | -# published by the Free Software Foundation. |
494 | -# |
495 | -# charm-helpers is distributed in the hope that it will be useful, |
496 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
497 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
498 | -# GNU Lesser General Public License for more details. |
499 | -# |
500 | -# You should have received a copy of the GNU Lesser General Public License |
501 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
502 | +# Licensed under the Apache License, Version 2.0 (the "License"); |
503 | +# you may not use this file except in compliance with the License. |
504 | +# You may obtain a copy of the License at |
505 | +# |
506 | +# http://www.apache.org/licenses/LICENSE-2.0 |
507 | +# |
508 | +# Unless required by applicable law or agreed to in writing, software |
509 | +# distributed under the License is distributed on an "AS IS" BASIS, |
510 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
511 | +# See the License for the specific language governing permissions and |
512 | +# limitations under the License. |
513 | |
514 | # |
515 | # Copyright 2014 Canonical Ltd. |
516 | |
517 | === added file 'hooks/charmhelpers/core/files.py' |
518 | --- hooks/charmhelpers/core/files.py 1970-01-01 00:00:00 +0000 |
519 | +++ hooks/charmhelpers/core/files.py 2018-06-12 20:07:13 +0000 |
520 | @@ -0,0 +1,43 @@ |
521 | +#!/usr/bin/env python |
522 | +# -*- coding: utf-8 -*- |
523 | + |
524 | +# Copyright 2014-2015 Canonical Limited. |
525 | +# |
526 | +# Licensed under the Apache License, Version 2.0 (the "License"); |
527 | +# you may not use this file except in compliance with the License. |
528 | +# You may obtain a copy of the License at |
529 | +# |
530 | +# http://www.apache.org/licenses/LICENSE-2.0 |
531 | +# |
532 | +# Unless required by applicable law or agreed to in writing, software |
533 | +# distributed under the License is distributed on an "AS IS" BASIS, |
534 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
535 | +# See the License for the specific language governing permissions and |
536 | +# limitations under the License. |
537 | + |
538 | +__author__ = 'Jorge Niedbalski <niedbalski@ubuntu.com>' |
539 | + |
540 | +import os |
541 | +import subprocess |
542 | + |
543 | + |
544 | +def sed(filename, before, after, flags='g'): |
545 | + """ |
546 | + Search and replaces the given pattern on filename. |
547 | + |
548 | + :param filename: relative or absolute file path. |
549 | + :param before: expression to be replaced (see 'man sed') |
550 | + :param after: expression to replace with (see 'man sed') |
551 | + :param flags: sed-compatible regex flags in example, to make |
552 | + the search and replace case insensitive, specify ``flags="i"``. |
553 | + The ``g`` flag is always specified regardless, so you do not |
554 | + need to remember to include it when overriding this parameter. |
555 | + :returns: If the sed command exit code was zero then return, |
556 | + otherwise raise CalledProcessError. |
557 | + """ |
558 | + expression = r's/{0}/{1}/{2}'.format(before, |
559 | + after, flags) |
560 | + |
561 | + return subprocess.check_call(["sed", "-i", "-r", "-e", |
562 | + expression, |
563 | + os.path.expanduser(filename)]) |
564 | |
565 | === modified file 'hooks/charmhelpers/core/fstab.py' |
566 | --- hooks/charmhelpers/core/fstab.py 2015-05-14 10:48:09 +0000 |
567 | +++ hooks/charmhelpers/core/fstab.py 2018-06-12 20:07:13 +0000 |
568 | @@ -3,19 +3,17 @@ |
569 | |
570 | # Copyright 2014-2015 Canonical Limited. |
571 | # |
572 | -# This file is part of charm-helpers. |
573 | -# |
574 | -# charm-helpers is free software: you can redistribute it and/or modify |
575 | -# it under the terms of the GNU Lesser General Public License version 3 as |
576 | -# published by the Free Software Foundation. |
577 | -# |
578 | -# charm-helpers is distributed in the hope that it will be useful, |
579 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
580 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
581 | -# GNU Lesser General Public License for more details. |
582 | -# |
583 | -# You should have received a copy of the GNU Lesser General Public License |
584 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
585 | +# Licensed under the Apache License, Version 2.0 (the "License"); |
586 | +# you may not use this file except in compliance with the License. |
587 | +# You may obtain a copy of the License at |
588 | +# |
589 | +# http://www.apache.org/licenses/LICENSE-2.0 |
590 | +# |
591 | +# Unless required by applicable law or agreed to in writing, software |
592 | +# distributed under the License is distributed on an "AS IS" BASIS, |
593 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
594 | +# See the License for the specific language governing permissions and |
595 | +# limitations under the License. |
596 | |
597 | import io |
598 | import os |
599 | |
600 | === modified file 'hooks/charmhelpers/core/hookenv.py' |
601 | --- hooks/charmhelpers/core/hookenv.py 2015-05-14 10:48:09 +0000 |
602 | +++ hooks/charmhelpers/core/hookenv.py 2018-06-12 20:07:13 +0000 |
603 | @@ -1,18 +1,16 @@ |
604 | # Copyright 2014-2015 Canonical Limited. |
605 | # |
606 | -# This file is part of charm-helpers. |
607 | -# |
608 | -# charm-helpers is free software: you can redistribute it and/or modify |
609 | -# it under the terms of the GNU Lesser General Public License version 3 as |
610 | -# published by the Free Software Foundation. |
611 | -# |
612 | -# charm-helpers is distributed in the hope that it will be useful, |
613 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
614 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
615 | -# GNU Lesser General Public License for more details. |
616 | -# |
617 | -# You should have received a copy of the GNU Lesser General Public License |
618 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
619 | +# Licensed under the Apache License, Version 2.0 (the "License"); |
620 | +# you may not use this file except in compliance with the License. |
621 | +# You may obtain a copy of the License at |
622 | +# |
623 | +# http://www.apache.org/licenses/LICENSE-2.0 |
624 | +# |
625 | +# Unless required by applicable law or agreed to in writing, software |
626 | +# distributed under the License is distributed on an "AS IS" BASIS, |
627 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
628 | +# See the License for the specific language governing permissions and |
629 | +# limitations under the License. |
630 | |
631 | "Interactions with the Juju environment" |
632 | # Copyright 2013 Canonical Ltd. |
633 | @@ -21,7 +19,10 @@ |
634 | # Charm Helpers Developers <juju@lists.ubuntu.com> |
635 | |
636 | from __future__ import print_function |
637 | +import copy |
638 | +from distutils.version import LooseVersion |
639 | from functools import wraps |
640 | +import glob |
641 | import os |
642 | import json |
643 | import yaml |
644 | @@ -71,6 +72,7 @@ |
645 | res = func(*args, **kwargs) |
646 | cache[key] = res |
647 | return res |
648 | + wrapper._wrapped = func |
649 | return wrapper |
650 | |
651 | |
652 | @@ -170,9 +172,19 @@ |
653 | return os.environ.get('JUJU_RELATION', None) |
654 | |
655 | |
656 | -def relation_id(): |
657 | - """The relation ID for the current relation hook""" |
658 | - return os.environ.get('JUJU_RELATION_ID', None) |
659 | +@cached |
660 | +def relation_id(relation_name=None, service_or_unit=None): |
661 | + """The relation ID for the current or a specified relation""" |
662 | + if not relation_name and not service_or_unit: |
663 | + return os.environ.get('JUJU_RELATION_ID', None) |
664 | + elif relation_name and service_or_unit: |
665 | + service_name = service_or_unit.split('/')[0] |
666 | + for relid in relation_ids(relation_name): |
667 | + remote_service = remote_service_name(relid) |
668 | + if remote_service == service_name: |
669 | + return relid |
670 | + else: |
671 | + raise ValueError('Must specify neither or both of relation_name and service_or_unit') |
672 | |
673 | |
674 | def local_unit(): |
675 | @@ -190,9 +202,20 @@ |
676 | return local_unit().split('/')[0] |
677 | |
678 | |
679 | +@cached |
680 | +def remote_service_name(relid=None): |
681 | + """The remote service name for a given relation-id (or the current relation)""" |
682 | + if relid is None: |
683 | + unit = remote_unit() |
684 | + else: |
685 | + units = related_units(relid) |
686 | + unit = units[0] if units else None |
687 | + return unit.split('/')[0] if unit else None |
688 | + |
689 | + |
690 | def hook_name(): |
691 | """The name of the currently executing hook""" |
692 | - return os.path.basename(sys.argv[0]) |
693 | + return os.environ.get('JUJU_HOOK_NAME', os.path.basename(sys.argv[0])) |
694 | |
695 | |
696 | class Config(dict): |
697 | @@ -242,29 +265,7 @@ |
698 | self.path = os.path.join(charm_dir(), Config.CONFIG_FILE_NAME) |
699 | if os.path.exists(self.path): |
700 | self.load_previous() |
701 | - |
702 | - def __getitem__(self, key): |
703 | - """For regular dict lookups, check the current juju config first, |
704 | - then the previous (saved) copy. This ensures that user-saved values |
705 | - will be returned by a dict lookup. |
706 | - |
707 | - """ |
708 | - try: |
709 | - return dict.__getitem__(self, key) |
710 | - except KeyError: |
711 | - return (self._prev_dict or {})[key] |
712 | - |
713 | - def get(self, key, default=None): |
714 | - try: |
715 | - return self[key] |
716 | - except KeyError: |
717 | - return default |
718 | - |
719 | - def keys(self): |
720 | - prev_keys = [] |
721 | - if self._prev_dict is not None: |
722 | - prev_keys = self._prev_dict.keys() |
723 | - return list(set(prev_keys + list(dict.keys(self)))) |
724 | + atexit(self._implicit_save) |
725 | |
726 | def load_previous(self, path=None): |
727 | """Load previous copy of config from disk. |
728 | @@ -283,6 +284,9 @@ |
729 | self.path = path or self.path |
730 | with open(self.path) as f: |
731 | self._prev_dict = json.load(f) |
732 | + for k, v in copy.deepcopy(self._prev_dict).items(): |
733 | + if k not in self: |
734 | + self[k] = v |
735 | |
736 | def changed(self, key): |
737 | """Return True if the current value for this key is different from |
738 | @@ -314,13 +318,13 @@ |
739 | instance. |
740 | |
741 | """ |
742 | - if self._prev_dict: |
743 | - for k, v in six.iteritems(self._prev_dict): |
744 | - if k not in self: |
745 | - self[k] = v |
746 | with open(self.path, 'w') as f: |
747 | json.dump(self, f) |
748 | |
749 | + def _implicit_save(self): |
750 | + if self.implicit_save: |
751 | + self.save() |
752 | + |
753 | |
754 | @cached |
755 | def config(scope=None): |
756 | @@ -328,6 +332,8 @@ |
757 | config_cmd_line = ['config-get'] |
758 | if scope is not None: |
759 | config_cmd_line.append(scope) |
760 | + else: |
761 | + config_cmd_line.append('--all') |
762 | config_cmd_line.append('--format=json') |
763 | try: |
764 | config_data = json.loads( |
765 | @@ -364,11 +370,16 @@ |
766 | relation_settings = relation_settings if relation_settings else {} |
767 | relation_cmd_line = ['relation-set'] |
768 | accepts_file = "--file" in subprocess.check_output( |
769 | - relation_cmd_line + ["--help"]) |
770 | + relation_cmd_line + ["--help"], universal_newlines=True) |
771 | if relation_id is not None: |
772 | relation_cmd_line.extend(('-r', relation_id)) |
773 | settings = relation_settings.copy() |
774 | settings.update(kwargs) |
775 | + for key, value in settings.items(): |
776 | + # Force value to be a string: it always should, but some call |
777 | + # sites pass in things like dicts or numbers. |
778 | + if value is not None: |
779 | + settings[key] = "{}".format(value) |
780 | if accepts_file: |
781 | # --file was introduced in Juju 1.23.2. Use it by default if |
782 | # available, since otherwise we'll break if the relation data is |
783 | @@ -390,6 +401,17 @@ |
784 | flush(local_unit()) |
785 | |
786 | |
787 | +def relation_clear(r_id=None): |
788 | + ''' Clears any relation data already set on relation r_id ''' |
789 | + settings = relation_get(rid=r_id, |
790 | + unit=local_unit()) |
791 | + for setting in settings: |
792 | + if setting not in ['public-address', 'private-address']: |
793 | + settings[setting] = None |
794 | + relation_set(relation_id=r_id, |
795 | + **settings) |
796 | + |
797 | + |
798 | @cached |
799 | def relation_ids(reltype=None): |
800 | """A list of relation_ids""" |
801 | @@ -469,6 +491,76 @@ |
802 | |
803 | |
804 | @cached |
805 | +def peer_relation_id(): |
806 | + '''Get the peers relation id if a peers relation has been joined, else None.''' |
807 | + md = metadata() |
808 | + section = md.get('peers') |
809 | + if section: |
810 | + for key in section: |
811 | + relids = relation_ids(key) |
812 | + if relids: |
813 | + return relids[0] |
814 | + return None |
815 | + |
816 | + |
817 | +@cached |
818 | +def relation_to_interface(relation_name): |
819 | + """ |
820 | + Given the name of a relation, return the interface that relation uses. |
821 | + |
822 | + :returns: The interface name, or ``None``. |
823 | + """ |
824 | + return relation_to_role_and_interface(relation_name)[1] |
825 | + |
826 | + |
827 | +@cached |
828 | +def relation_to_role_and_interface(relation_name): |
829 | + """ |
830 | + Given the name of a relation, return the role and the name of the interface |
831 | + that relation uses (where role is one of ``provides``, ``requires``, or ``peers``). |
832 | + |
833 | + :returns: A tuple containing ``(role, interface)``, or ``(None, None)``. |
834 | + """ |
835 | + _metadata = metadata() |
836 | + for role in ('provides', 'requires', 'peers'): |
837 | + interface = _metadata.get(role, {}).get(relation_name, {}).get('interface') |
838 | + if interface: |
839 | + return role, interface |
840 | + return None, None |
841 | + |
842 | + |
843 | +@cached |
844 | +def role_and_interface_to_relations(role, interface_name): |
845 | + """ |
846 | + Given a role and interface name, return a list of relation names for the |
847 | + current charm that use that interface under that role (where role is one |
848 | + of ``provides``, ``requires``, or ``peers``). |
849 | + |
850 | + :returns: A list of relation names. |
851 | + """ |
852 | + _metadata = metadata() |
853 | + results = [] |
854 | + for relation_name, relation in _metadata.get(role, {}).items(): |
855 | + if relation['interface'] == interface_name: |
856 | + results.append(relation_name) |
857 | + return results |
858 | + |
859 | + |
860 | +@cached |
861 | +def interface_to_relations(interface_name): |
862 | + """ |
863 | + Given an interface, return a list of relation names for the current |
864 | + charm that use that interface. |
865 | + |
866 | + :returns: A list of relation names. |
867 | + """ |
868 | + results = [] |
869 | + for role in ('provides', 'requires', 'peers'): |
870 | + results.extend(role_and_interface_to_relations(role, interface_name)) |
871 | + return results |
872 | + |
873 | + |
874 | +@cached |
875 | def charm_name(): |
876 | """Get the name of the current charm as is specified on metadata.yaml""" |
877 | return metadata().get('name') |
878 | @@ -524,6 +616,20 @@ |
879 | subprocess.check_call(_args) |
880 | |
881 | |
882 | +def open_ports(start, end, protocol="TCP"): |
883 | + """Opens a range of service network ports""" |
884 | + _args = ['open-port'] |
885 | + _args.append('{}-{}/{}'.format(start, end, protocol)) |
886 | + subprocess.check_call(_args) |
887 | + |
888 | + |
889 | +def close_ports(start, end, protocol="TCP"): |
890 | + """Close a range of service network ports""" |
891 | + _args = ['close-port'] |
892 | + _args.append('{}-{}/{}'.format(start, end, protocol)) |
893 | + subprocess.check_call(_args) |
894 | + |
895 | + |
896 | @cached |
897 | def unit_get(attribute): |
898 | """Get the unit ID for the remote unit""" |
899 | @@ -544,6 +650,38 @@ |
900 | return unit_get('private-address') |
901 | |
902 | |
903 | +@cached |
904 | +def storage_get(attribute=None, storage_id=None): |
905 | + """Get storage attributes""" |
906 | + _args = ['storage-get', '--format=json'] |
907 | + if storage_id: |
908 | + _args.extend(('-s', storage_id)) |
909 | + if attribute: |
910 | + _args.append(attribute) |
911 | + try: |
912 | + return json.loads(subprocess.check_output(_args).decode('UTF-8')) |
913 | + except ValueError: |
914 | + return None |
915 | + |
916 | + |
917 | +@cached |
918 | +def storage_list(storage_name=None): |
919 | + """List the storage IDs for the unit""" |
920 | + _args = ['storage-list', '--format=json'] |
921 | + if storage_name: |
922 | + _args.append(storage_name) |
923 | + try: |
924 | + return json.loads(subprocess.check_output(_args).decode('UTF-8')) |
925 | + except ValueError: |
926 | + return None |
927 | + except OSError as e: |
928 | + import errno |
929 | + if e.errno == errno.ENOENT: |
930 | + # storage-list does not exist |
931 | + return [] |
932 | + raise |
933 | + |
934 | + |
935 | class UnregisteredHookError(Exception): |
936 | """Raised when an undefined hook is called""" |
937 | pass |
938 | @@ -571,10 +709,14 @@ |
939 | hooks.execute(sys.argv) |
940 | """ |
941 | |
942 | - def __init__(self, config_save=True): |
943 | + def __init__(self, config_save=None): |
944 | super(Hooks, self).__init__() |
945 | self._hooks = {} |
946 | - self._config_save = config_save |
947 | + |
948 | + # For unknown reasons, we allow the Hooks constructor to override |
949 | + # config().implicit_save. |
950 | + if config_save is not None: |
951 | + config().implicit_save = config_save |
952 | |
953 | def register(self, name, function): |
954 | """Register a hook""" |
955 | @@ -582,13 +724,16 @@ |
956 | |
957 | def execute(self, args): |
958 | """Execute a registered hook based on args[0]""" |
959 | + _run_atstart() |
960 | hook_name = os.path.basename(args[0]) |
961 | if hook_name in self._hooks: |
962 | - self._hooks[hook_name]() |
963 | - if self._config_save: |
964 | - cfg = config() |
965 | - if cfg.implicit_save: |
966 | - cfg.save() |
967 | + try: |
968 | + self._hooks[hook_name]() |
969 | + except SystemExit as x: |
970 | + if x.code is None or x.code == 0: |
971 | + _run_atexit() |
972 | + raise |
973 | + _run_atexit() |
974 | else: |
975 | raise UnregisteredHookError(hook_name) |
976 | |
977 | @@ -637,6 +782,21 @@ |
978 | subprocess.check_call(['action-fail', message]) |
979 | |
980 | |
981 | +def action_name(): |
982 | + """Get the name of the currently executing action.""" |
983 | + return os.environ.get('JUJU_ACTION_NAME') |
984 | + |
985 | + |
986 | +def action_uuid(): |
987 | + """Get the UUID of the currently executing action.""" |
988 | + return os.environ.get('JUJU_ACTION_UUID') |
989 | + |
990 | + |
991 | +def action_tag(): |
992 | + """Get the tag for the currently executing action.""" |
993 | + return os.environ.get('JUJU_ACTION_TAG') |
994 | + |
995 | + |
996 | def status_set(workload_state, message): |
997 | """Set the workload state with a message |
998 | |
999 | @@ -666,18 +826,243 @@ |
1000 | |
1001 | |
1002 | def status_get(): |
1003 | - """Retrieve the previously set juju workload state |
1004 | - |
1005 | - If the status-set command is not found then assume this is juju < 1.23 and |
1006 | - return 'unknown' |
1007 | + """Retrieve the previously set juju workload state and message |
1008 | + |
1009 | + If the status-get command is not found then assume this is juju < 1.23 and |
1010 | + return 'unknown', "" |
1011 | + |
1012 | """ |
1013 | - cmd = ['status-get'] |
1014 | + cmd = ['status-get', "--format=json", "--include-data"] |
1015 | try: |
1016 | - raw_status = subprocess.check_output(cmd, universal_newlines=True) |
1017 | - status = raw_status.rstrip() |
1018 | - return status |
1019 | + raw_status = subprocess.check_output(cmd) |
1020 | except OSError as e: |
1021 | if e.errno == errno.ENOENT: |
1022 | - return 'unknown' |
1023 | - else: |
1024 | - raise |
1025 | + return ('unknown', "") |
1026 | + else: |
1027 | + raise |
1028 | + else: |
1029 | + status = json.loads(raw_status.decode("UTF-8")) |
1030 | + return (status["status"], status["message"]) |
1031 | + |
1032 | + |
1033 | +def translate_exc(from_exc, to_exc): |
1034 | + def inner_translate_exc1(f): |
1035 | + @wraps(f) |
1036 | + def inner_translate_exc2(*args, **kwargs): |
1037 | + try: |
1038 | + return f(*args, **kwargs) |
1039 | + except from_exc: |
1040 | + raise to_exc |
1041 | + |
1042 | + return inner_translate_exc2 |
1043 | + |
1044 | + return inner_translate_exc1 |
1045 | + |
1046 | + |
1047 | +def application_version_set(version): |
1048 | + """Charm authors may trigger this command from any hook to output what |
1049 | + version of the application is running. This could be a package version, |
1050 | + for instance postgres version 9.5. It could also be a build number or |
1051 | + version control revision identifier, for instance git sha 6fb7ba68. """ |
1052 | + |
1053 | + cmd = ['application-version-set'] |
1054 | + cmd.append(version) |
1055 | + try: |
1056 | + subprocess.check_call(cmd) |
1057 | + except OSError: |
1058 | + log("Application Version: {}".format(version)) |
1059 | + |
1060 | + |
1061 | +@translate_exc(from_exc=OSError, to_exc=NotImplementedError) |
1062 | +def is_leader(): |
1063 | + """Does the current unit hold the juju leadership |
1064 | + |
1065 | + Uses juju to determine whether the current unit is the leader of its peers |
1066 | + """ |
1067 | + cmd = ['is-leader', '--format=json'] |
1068 | + return json.loads(subprocess.check_output(cmd).decode('UTF-8')) |
1069 | + |
1070 | + |
1071 | +@translate_exc(from_exc=OSError, to_exc=NotImplementedError) |
1072 | +def leader_get(attribute=None): |
1073 | + """Juju leader get value(s)""" |
1074 | + cmd = ['leader-get', '--format=json'] + [attribute or '-'] |
1075 | + return json.loads(subprocess.check_output(cmd).decode('UTF-8')) |
1076 | + |
1077 | + |
1078 | +@translate_exc(from_exc=OSError, to_exc=NotImplementedError) |
1079 | +def leader_set(settings=None, **kwargs): |
1080 | + """Juju leader set value(s)""" |
1081 | + # Don't log secrets. |
1082 | + # log("Juju leader-set '%s'" % (settings), level=DEBUG) |
1083 | + cmd = ['leader-set'] |
1084 | + settings = settings or {} |
1085 | + settings.update(kwargs) |
1086 | + for k, v in settings.items(): |
1087 | + if v is None: |
1088 | + cmd.append('{}='.format(k)) |
1089 | + else: |
1090 | + cmd.append('{}={}'.format(k, v)) |
1091 | + subprocess.check_call(cmd) |
1092 | + |
1093 | + |
1094 | +@translate_exc(from_exc=OSError, to_exc=NotImplementedError) |
1095 | +def payload_register(ptype, klass, pid): |
1096 | + """ is used while a hook is running to let Juju know that a |
1097 | + payload has been started.""" |
1098 | + cmd = ['payload-register'] |
1099 | + for x in [ptype, klass, pid]: |
1100 | + cmd.append(x) |
1101 | + subprocess.check_call(cmd) |
1102 | + |
1103 | + |
1104 | +@translate_exc(from_exc=OSError, to_exc=NotImplementedError) |
1105 | +def payload_unregister(klass, pid): |
1106 | + """ is used while a hook is running to let Juju know |
1107 | + that a payload has been manually stopped. The <class> and <id> provided |
1108 | + must match a payload that has been previously registered with juju using |
1109 | + payload-register.""" |
1110 | + cmd = ['payload-unregister'] |
1111 | + for x in [klass, pid]: |
1112 | + cmd.append(x) |
1113 | + subprocess.check_call(cmd) |
1114 | + |
1115 | + |
1116 | +@translate_exc(from_exc=OSError, to_exc=NotImplementedError) |
1117 | +def payload_status_set(klass, pid, status): |
1118 | + """is used to update the current status of a registered payload. |
1119 | + The <class> and <id> provided must match a payload that has been previously |
1120 | + registered with juju using payload-register. The <status> must be one of the |
1121 | + follow: starting, started, stopping, stopped""" |
1122 | + cmd = ['payload-status-set'] |
1123 | + for x in [klass, pid, status]: |
1124 | + cmd.append(x) |
1125 | + subprocess.check_call(cmd) |
1126 | + |
1127 | + |
1128 | +@translate_exc(from_exc=OSError, to_exc=NotImplementedError) |
1129 | +def resource_get(name): |
1130 | + """used to fetch the resource path of the given name. |
1131 | + |
1132 | + <name> must match a name of defined resource in metadata.yaml |
1133 | + |
1134 | + returns either a path or False if resource not available |
1135 | + """ |
1136 | + if not name: |
1137 | + return False |
1138 | + |
1139 | + cmd = ['resource-get', name] |
1140 | + try: |
1141 | + return subprocess.check_output(cmd).decode('UTF-8') |
1142 | + except subprocess.CalledProcessError: |
1143 | + return False |
1144 | + |
1145 | + |
1146 | +@cached |
1147 | +def juju_version(): |
1148 | + """Full version string (eg. '1.23.3.1-trusty-amd64')""" |
1149 | + # Per https://bugs.launchpad.net/juju-core/+bug/1455368/comments/1 |
1150 | + jujud = glob.glob('/var/lib/juju/tools/machine-*/jujud')[0] |
1151 | + return subprocess.check_output([jujud, 'version'], |
1152 | + universal_newlines=True).strip() |
1153 | + |
1154 | + |
1155 | +@cached |
1156 | +def has_juju_version(minimum_version): |
1157 | + """Return True if the Juju version is at least the provided version""" |
1158 | + return LooseVersion(juju_version()) >= LooseVersion(minimum_version) |
1159 | + |
1160 | + |
1161 | +_atexit = [] |
1162 | +_atstart = [] |
1163 | + |
1164 | + |
1165 | +def atstart(callback, *args, **kwargs): |
1166 | + '''Schedule a callback to run before the main hook. |
1167 | + |
1168 | + Callbacks are run in the order they were added. |
1169 | + |
1170 | + This is useful for modules and classes to perform initialization |
1171 | + and inject behavior. In particular: |
1172 | + |
1173 | + - Run common code before all of your hooks, such as logging |
1174 | + the hook name or interesting relation data. |
1175 | + - Defer object or module initialization that requires a hook |
1176 | + context until we know there actually is a hook context, |
1177 | + making testing easier. |
1178 | + - Rather than requiring charm authors to include boilerplate to |
1179 | + invoke your helper's behavior, have it run automatically if |
1180 | + your object is instantiated or module imported. |
1181 | + |
1182 | + This is not at all useful after your hook framework as been launched. |
1183 | + ''' |
1184 | + global _atstart |
1185 | + _atstart.append((callback, args, kwargs)) |
1186 | + |
1187 | + |
1188 | +def atexit(callback, *args, **kwargs): |
1189 | + '''Schedule a callback to run on successful hook completion. |
1190 | + |
1191 | + Callbacks are run in the reverse order that they were added.''' |
1192 | + _atexit.append((callback, args, kwargs)) |
1193 | + |
1194 | + |
1195 | +def _run_atstart(): |
1196 | + '''Hook frameworks must invoke this before running the main hook body.''' |
1197 | + global _atstart |
1198 | + for callback, args, kwargs in _atstart: |
1199 | + callback(*args, **kwargs) |
1200 | + del _atstart[:] |
1201 | + |
1202 | + |
1203 | +def _run_atexit(): |
1204 | + '''Hook frameworks must invoke this after the main hook body has |
1205 | + successfully completed. Do not invoke it if the hook fails.''' |
1206 | + global _atexit |
1207 | + for callback, args, kwargs in reversed(_atexit): |
1208 | + callback(*args, **kwargs) |
1209 | + del _atexit[:] |
1210 | + |
1211 | + |
1212 | +@translate_exc(from_exc=OSError, to_exc=NotImplementedError) |
1213 | +def network_get_primary_address(binding): |
1214 | + ''' |
1215 | + Retrieve the primary network address for a named binding |
1216 | + |
1217 | + :param binding: string. The name of a relation of extra-binding |
1218 | + :return: string. The primary IP address for the named binding |
1219 | + :raise: NotImplementedError if run on Juju < 2.0 |
1220 | + ''' |
1221 | + cmd = ['network-get', '--primary-address', binding] |
1222 | + return subprocess.check_output(cmd).decode('UTF-8').strip() |
1223 | + |
1224 | + |
1225 | +def add_metric(*args, **kwargs): |
1226 | + """Add metric values. Values may be expressed with keyword arguments. For |
1227 | + metric names containing dashes, these may be expressed as one or more |
1228 | + 'key=value' positional arguments. May only be called from the collect-metrics |
1229 | + hook.""" |
1230 | + _args = ['add-metric'] |
1231 | + _kvpairs = [] |
1232 | + _kvpairs.extend(args) |
1233 | + _kvpairs.extend(['{}={}'.format(k, v) for k, v in kwargs.items()]) |
1234 | + _args.extend(sorted(_kvpairs)) |
1235 | + try: |
1236 | + subprocess.check_call(_args) |
1237 | + return |
1238 | + except EnvironmentError as e: |
1239 | + if e.errno != errno.ENOENT: |
1240 | + raise |
1241 | + log_message = 'add-metric failed: {}'.format(' '.join(_kvpairs)) |
1242 | + log(log_message, level='INFO') |
1243 | + |
1244 | + |
1245 | +def meter_status(): |
1246 | + """Get the meter status, if running in the meter-status-changed hook.""" |
1247 | + return os.environ.get('JUJU_METER_STATUS') |
1248 | + |
1249 | + |
1250 | +def meter_info(): |
1251 | + """Get the meter status information, if running in the meter-status-changed |
1252 | + hook.""" |
1253 | + return os.environ.get('JUJU_METER_INFO') |
1254 | |
1255 | === modified file 'hooks/charmhelpers/core/host.py' |
1256 | --- hooks/charmhelpers/core/host.py 2015-05-14 10:48:09 +0000 |
1257 | +++ hooks/charmhelpers/core/host.py 2018-06-12 20:07:13 +0000 |
1258 | @@ -1,18 +1,16 @@ |
1259 | # Copyright 2014-2015 Canonical Limited. |
1260 | # |
1261 | -# This file is part of charm-helpers. |
1262 | -# |
1263 | -# charm-helpers is free software: you can redistribute it and/or modify |
1264 | -# it under the terms of the GNU Lesser General Public License version 3 as |
1265 | -# published by the Free Software Foundation. |
1266 | -# |
1267 | -# charm-helpers is distributed in the hope that it will be useful, |
1268 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
1269 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
1270 | -# GNU Lesser General Public License for more details. |
1271 | -# |
1272 | -# You should have received a copy of the GNU Lesser General Public License |
1273 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
1274 | +# Licensed under the Apache License, Version 2.0 (the "License"); |
1275 | +# you may not use this file except in compliance with the License. |
1276 | +# You may obtain a copy of the License at |
1277 | +# |
1278 | +# http://www.apache.org/licenses/LICENSE-2.0 |
1279 | +# |
1280 | +# Unless required by applicable law or agreed to in writing, software |
1281 | +# distributed under the License is distributed on an "AS IS" BASIS, |
1282 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
1283 | +# See the License for the specific language governing permissions and |
1284 | +# limitations under the License. |
1285 | |
1286 | """Tools for working with the host system""" |
1287 | # Copyright 2012 Canonical Ltd. |
1288 | @@ -24,85 +22,326 @@ |
1289 | import os |
1290 | import re |
1291 | import pwd |
1292 | +import glob |
1293 | import grp |
1294 | import random |
1295 | import string |
1296 | import subprocess |
1297 | import hashlib |
1298 | +import functools |
1299 | +import itertools |
1300 | +import six |
1301 | + |
1302 | from contextlib import contextmanager |
1303 | from collections import OrderedDict |
1304 | - |
1305 | -import six |
1306 | - |
1307 | from .hookenv import log |
1308 | from .fstab import Fstab |
1309 | - |
1310 | - |
1311 | -def service_start(service_name): |
1312 | - """Start a system service""" |
1313 | - return service('start', service_name) |
1314 | - |
1315 | - |
1316 | -def service_stop(service_name): |
1317 | - """Stop a system service""" |
1318 | - return service('stop', service_name) |
1319 | - |
1320 | - |
1321 | -def service_restart(service_name): |
1322 | - """Restart a system service""" |
1323 | +from charmhelpers.osplatform import get_platform |
1324 | + |
1325 | +__platform__ = get_platform() |
1326 | +if __platform__ == "ubuntu": |
1327 | + from charmhelpers.core.host_factory.ubuntu import ( |
1328 | + service_available, |
1329 | + add_new_group, |
1330 | + lsb_release, |
1331 | + cmp_pkgrevno, |
1332 | + ) # flake8: noqa -- ignore F401 for this import |
1333 | +elif __platform__ == "centos": |
1334 | + from charmhelpers.core.host_factory.centos import ( |
1335 | + service_available, |
1336 | + add_new_group, |
1337 | + lsb_release, |
1338 | + cmp_pkgrevno, |
1339 | + ) # flake8: noqa -- ignore F401 for this import |
1340 | + |
1341 | +UPDATEDB_PATH = '/etc/updatedb.conf' |
1342 | + |
1343 | +def service_start(service_name, **kwargs): |
1344 | + """Start a system service. |
1345 | + |
1346 | + The specified service name is managed via the system level init system. |
1347 | + Some init systems (e.g. upstart) require that additional arguments be |
1348 | + provided in order to directly control service instances whereas other init |
1349 | + systems allow for addressing instances of a service directly by name (e.g. |
1350 | + systemd). |
1351 | + |
1352 | + The kwargs allow for the additional parameters to be passed to underlying |
1353 | + init systems for those systems which require/allow for them. For example, |
1354 | + the ceph-osd upstart script requires the id parameter to be passed along |
1355 | + in order to identify which running daemon should be reloaded. The follow- |
1356 | + ing example stops the ceph-osd service for instance id=4: |
1357 | + |
1358 | + service_stop('ceph-osd', id=4) |
1359 | + |
1360 | + :param service_name: the name of the service to stop |
1361 | + :param **kwargs: additional parameters to pass to the init system when |
1362 | + managing services. These will be passed as key=value |
1363 | + parameters to the init system's commandline. kwargs |
1364 | + are ignored for systemd enabled systems. |
1365 | + """ |
1366 | + return service('start', service_name, **kwargs) |
1367 | + |
1368 | + |
1369 | +def service_stop(service_name, **kwargs): |
1370 | + """Stop a system service. |
1371 | + |
1372 | + The specified service name is managed via the system level init system. |
1373 | + Some init systems (e.g. upstart) require that additional arguments be |
1374 | + provided in order to directly control service instances whereas other init |
1375 | + systems allow for addressing instances of a service directly by name (e.g. |
1376 | + systemd). |
1377 | + |
1378 | + The kwargs allow for the additional parameters to be passed to underlying |
1379 | + init systems for those systems which require/allow for them. For example, |
1380 | + the ceph-osd upstart script requires the id parameter to be passed along |
1381 | + in order to identify which running daemon should be reloaded. The follow- |
1382 | + ing example stops the ceph-osd service for instance id=4: |
1383 | + |
1384 | + service_stop('ceph-osd', id=4) |
1385 | + |
1386 | + :param service_name: the name of the service to stop |
1387 | + :param **kwargs: additional parameters to pass to the init system when |
1388 | + managing services. These will be passed as key=value |
1389 | + parameters to the init system's commandline. kwargs |
1390 | + are ignored for systemd enabled systems. |
1391 | + """ |
1392 | + return service('stop', service_name, **kwargs) |
1393 | + |
1394 | + |
1395 | +def service_restart(service_name, **kwargs): |
1396 | + """Restart a system service. |
1397 | + |
1398 | + The specified service name is managed via the system level init system. |
1399 | + Some init systems (e.g. upstart) require that additional arguments be |
1400 | + provided in order to directly control service instances whereas other init |
1401 | + systems allow for addressing instances of a service directly by name (e.g. |
1402 | + systemd). |
1403 | + |
1404 | + The kwargs allow for the additional parameters to be passed to underlying |
1405 | + init systems for those systems which require/allow for them. For example, |
1406 | + the ceph-osd upstart script requires the id parameter to be passed along |
1407 | + in order to identify which running daemon should be restarted. The follow- |
1408 | + ing example restarts the ceph-osd service for instance id=4: |
1409 | + |
1410 | + service_restart('ceph-osd', id=4) |
1411 | + |
1412 | + :param service_name: the name of the service to restart |
1413 | + :param **kwargs: additional parameters to pass to the init system when |
1414 | + managing services. These will be passed as key=value |
1415 | + parameters to the init system's commandline. kwargs |
1416 | + are ignored for init systems not allowing additional |
1417 | + parameters via the commandline (systemd). |
1418 | + """ |
1419 | return service('restart', service_name) |
1420 | |
1421 | |
1422 | -def service_reload(service_name, restart_on_failure=False): |
1423 | +def service_reload(service_name, restart_on_failure=False, **kwargs): |
1424 | """Reload a system service, optionally falling back to restart if |
1425 | - reload fails""" |
1426 | - service_result = service('reload', service_name) |
1427 | + reload fails. |
1428 | + |
1429 | + The specified service name is managed via the system level init system. |
1430 | + Some init systems (e.g. upstart) require that additional arguments be |
1431 | + provided in order to directly control service instances whereas other init |
1432 | + systems allow for addressing instances of a service directly by name (e.g. |
1433 | + systemd). |
1434 | + |
1435 | + The kwargs allow for the additional parameters to be passed to underlying |
1436 | + init systems for those systems which require/allow for them. For example, |
1437 | + the ceph-osd upstart script requires the id parameter to be passed along |
1438 | + in order to identify which running daemon should be reloaded. The follow- |
1439 | + ing example restarts the ceph-osd service for instance id=4: |
1440 | + |
1441 | + service_reload('ceph-osd', id=4) |
1442 | + |
1443 | + :param service_name: the name of the service to reload |
1444 | + :param restart_on_failure: boolean indicating whether to fallback to a |
1445 | + restart if the reload fails. |
1446 | + :param **kwargs: additional parameters to pass to the init system when |
1447 | + managing services. These will be passed as key=value |
1448 | + parameters to the init system's commandline. kwargs |
1449 | + are ignored for init systems not allowing additional |
1450 | + parameters via the commandline (systemd). |
1451 | + """ |
1452 | + service_result = service('reload', service_name, **kwargs) |
1453 | if not service_result and restart_on_failure: |
1454 | - service_result = service('restart', service_name) |
1455 | + service_result = service('restart', service_name, **kwargs) |
1456 | return service_result |
1457 | |
1458 | |
1459 | -def service(action, service_name): |
1460 | - """Control a system service""" |
1461 | - cmd = ['service', service_name, action] |
1462 | +def service_pause(service_name, init_dir="/etc/init", initd_dir="/etc/init.d", |
1463 | + **kwargs): |
1464 | + """Pause a system service. |
1465 | + |
1466 | + Stop it, and prevent it from starting again at boot. |
1467 | + |
1468 | + :param service_name: the name of the service to pause |
1469 | + :param init_dir: path to the upstart init directory |
1470 | + :param initd_dir: path to the sysv init directory |
1471 | + :param **kwargs: additional parameters to pass to the init system when |
1472 | + managing services. These will be passed as key=value |
1473 | + parameters to the init system's commandline. kwargs |
1474 | + are ignored for init systems which do not support |
1475 | + key=value arguments via the commandline. |
1476 | + """ |
1477 | + stopped = True |
1478 | + if service_running(service_name, **kwargs): |
1479 | + stopped = service_stop(service_name, **kwargs) |
1480 | + upstart_file = os.path.join(init_dir, "{}.conf".format(service_name)) |
1481 | + sysv_file = os.path.join(initd_dir, service_name) |
1482 | + if init_is_systemd(): |
1483 | + service('disable', service_name) |
1484 | + elif os.path.exists(upstart_file): |
1485 | + override_path = os.path.join( |
1486 | + init_dir, '{}.override'.format(service_name)) |
1487 | + with open(override_path, 'w') as fh: |
1488 | + fh.write("manual\n") |
1489 | + elif os.path.exists(sysv_file): |
1490 | + subprocess.check_call(["update-rc.d", service_name, "disable"]) |
1491 | + else: |
1492 | + raise ValueError( |
1493 | + "Unable to detect {0} as SystemD, Upstart {1} or" |
1494 | + " SysV {2}".format( |
1495 | + service_name, upstart_file, sysv_file)) |
1496 | + return stopped |
1497 | + |
1498 | + |
1499 | +def service_resume(service_name, init_dir="/etc/init", |
1500 | + initd_dir="/etc/init.d", **kwargs): |
1501 | + """Resume a system service. |
1502 | + |
1503 | + Reenable starting again at boot. Start the service. |
1504 | + |
1505 | + :param service_name: the name of the service to resume |
1506 | + :param init_dir: the path to the init dir |
1507 | + :param initd dir: the path to the initd dir |
1508 | + :param **kwargs: additional parameters to pass to the init system when |
1509 | + managing services. These will be passed as key=value |
1510 | + parameters to the init system's commandline. kwargs |
1511 | + are ignored for systemd enabled systems. |
1512 | + """ |
1513 | + upstart_file = os.path.join(init_dir, "{}.conf".format(service_name)) |
1514 | + sysv_file = os.path.join(initd_dir, service_name) |
1515 | + if init_is_systemd(): |
1516 | + service('enable', service_name) |
1517 | + elif os.path.exists(upstart_file): |
1518 | + override_path = os.path.join( |
1519 | + init_dir, '{}.override'.format(service_name)) |
1520 | + if os.path.exists(override_path): |
1521 | + os.unlink(override_path) |
1522 | + elif os.path.exists(sysv_file): |
1523 | + subprocess.check_call(["update-rc.d", service_name, "enable"]) |
1524 | + else: |
1525 | + raise ValueError( |
1526 | + "Unable to detect {0} as SystemD, Upstart {1} or" |
1527 | + " SysV {2}".format( |
1528 | + service_name, upstart_file, sysv_file)) |
1529 | + started = service_running(service_name, **kwargs) |
1530 | + |
1531 | + if not started: |
1532 | + started = service_start(service_name, **kwargs) |
1533 | + return started |
1534 | + |
1535 | + |
1536 | +def service(action, service_name, **kwargs): |
1537 | + """Control a system service. |
1538 | + |
1539 | + :param action: the action to take on the service |
1540 | + :param service_name: the name of the service to perform th action on |
1541 | + :param **kwargs: additional params to be passed to the service command in |
1542 | + the form of key=value. |
1543 | + """ |
1544 | + if init_is_systemd(): |
1545 | + cmd = ['systemctl', action, service_name] |
1546 | + else: |
1547 | + cmd = ['service', service_name, action] |
1548 | + for key, value in six.iteritems(kwargs): |
1549 | + parameter = '%s=%s' % (key, value) |
1550 | + cmd.append(parameter) |
1551 | return subprocess.call(cmd) == 0 |
1552 | |
1553 | |
1554 | -def service_running(service): |
1555 | - """Determine whether a system service is running""" |
1556 | - try: |
1557 | - output = subprocess.check_output( |
1558 | - ['service', service, 'status'], |
1559 | - stderr=subprocess.STDOUT).decode('UTF-8') |
1560 | - except subprocess.CalledProcessError: |
1561 | - return False |
1562 | - else: |
1563 | - if ("start/running" in output or "is running" in output): |
1564 | - return True |
1565 | - else: |
1566 | - return False |
1567 | - |
1568 | - |
1569 | -def service_available(service_name): |
1570 | - """Determine whether a system service is available""" |
1571 | - try: |
1572 | - subprocess.check_output( |
1573 | - ['service', service_name, 'status'], |
1574 | - stderr=subprocess.STDOUT).decode('UTF-8') |
1575 | - except subprocess.CalledProcessError as e: |
1576 | - return b'unrecognized service' not in e.output |
1577 | - else: |
1578 | - return True |
1579 | - |
1580 | - |
1581 | -def adduser(username, password=None, shell='/bin/bash', system_user=False): |
1582 | - """Add a user to the system""" |
1583 | +_UPSTART_CONF = "/etc/init/{}.conf" |
1584 | +_INIT_D_CONF = "/etc/init.d/{}" |
1585 | + |
1586 | + |
1587 | +def service_running(service_name, **kwargs): |
1588 | + """Determine whether a system service is running. |
1589 | + |
1590 | + :param service_name: the name of the service |
1591 | + :param **kwargs: additional args to pass to the service command. This is |
1592 | + used to pass additional key=value arguments to the |
1593 | + service command line for managing specific instance |
1594 | + units (e.g. service ceph-osd status id=2). The kwargs |
1595 | + are ignored in systemd services. |
1596 | + """ |
1597 | + if init_is_systemd(): |
1598 | + return service('is-active', service_name) |
1599 | + else: |
1600 | + if os.path.exists(_UPSTART_CONF.format(service_name)): |
1601 | + try: |
1602 | + cmd = ['status', service_name] |
1603 | + for key, value in six.iteritems(kwargs): |
1604 | + parameter = '%s=%s' % (key, value) |
1605 | + cmd.append(parameter) |
1606 | + output = subprocess.check_output(cmd, |
1607 | + stderr=subprocess.STDOUT).decode('UTF-8') |
1608 | + except subprocess.CalledProcessError: |
1609 | + return False |
1610 | + else: |
1611 | + # This works for upstart scripts where the 'service' command |
1612 | + # returns a consistent string to represent running |
1613 | + # 'start/running' |
1614 | + if ("start/running" in output or |
1615 | + "is running" in output or |
1616 | + "up and running" in output): |
1617 | + return True |
1618 | + elif os.path.exists(_INIT_D_CONF.format(service_name)): |
1619 | + # Check System V scripts init script return codes |
1620 | + return service('status', service_name) |
1621 | + return False |
1622 | + |
1623 | + |
1624 | +SYSTEMD_SYSTEM = '/run/systemd/system' |
1625 | + |
1626 | + |
1627 | +def init_is_systemd(): |
1628 | + """Return True if the host system uses systemd, False otherwise.""" |
1629 | + if lsb_release()['DISTRIB_CODENAME'] == 'trusty': |
1630 | + return False |
1631 | + return os.path.isdir(SYSTEMD_SYSTEM) |
1632 | + |
1633 | + |
1634 | +def adduser(username, password=None, shell='/bin/bash', |
1635 | + system_user=False, primary_group=None, |
1636 | + secondary_groups=None, uid=None, home_dir=None): |
1637 | + """Add a user to the system. |
1638 | + |
1639 | + Will log but otherwise succeed if the user already exists. |
1640 | + |
1641 | + :param str username: Username to create |
1642 | + :param str password: Password for user; if ``None``, create a system user |
1643 | + :param str shell: The default shell for the user |
1644 | + :param bool system_user: Whether to create a login or system user |
1645 | + :param str primary_group: Primary group for user; defaults to username |
1646 | + :param list secondary_groups: Optional list of additional groups |
1647 | + :param int uid: UID for user being created |
1648 | + :param str home_dir: Home directory for user |
1649 | + |
1650 | + :returns: The password database entry struct, as returned by `pwd.getpwnam` |
1651 | + """ |
1652 | try: |
1653 | user_info = pwd.getpwnam(username) |
1654 | log('user {0} already exists!'.format(username)) |
1655 | + if uid: |
1656 | + user_info = pwd.getpwuid(int(uid)) |
1657 | + log('user with uid {0} already exists!'.format(uid)) |
1658 | except KeyError: |
1659 | log('creating user {0}'.format(username)) |
1660 | cmd = ['useradd'] |
1661 | + if uid: |
1662 | + cmd.extend(['--uid', str(uid)]) |
1663 | + if home_dir: |
1664 | + cmd.extend(['--home', str(home_dir)]) |
1665 | if system_user or password is None: |
1666 | cmd.append('--system') |
1667 | else: |
1668 | @@ -111,52 +350,104 @@ |
1669 | '--shell', shell, |
1670 | '--password', password, |
1671 | ]) |
1672 | + if not primary_group: |
1673 | + try: |
1674 | + grp.getgrnam(username) |
1675 | + primary_group = username # avoid "group exists" error |
1676 | + except KeyError: |
1677 | + pass |
1678 | + if primary_group: |
1679 | + cmd.extend(['-g', primary_group]) |
1680 | + if secondary_groups: |
1681 | + cmd.extend(['-G', ','.join(secondary_groups)]) |
1682 | cmd.append(username) |
1683 | subprocess.check_call(cmd) |
1684 | user_info = pwd.getpwnam(username) |
1685 | return user_info |
1686 | |
1687 | |
1688 | -def add_group(group_name, system_group=False): |
1689 | - """Add a group to the system""" |
1690 | +def user_exists(username): |
1691 | + """Check if a user exists""" |
1692 | + try: |
1693 | + pwd.getpwnam(username) |
1694 | + user_exists = True |
1695 | + except KeyError: |
1696 | + user_exists = False |
1697 | + return user_exists |
1698 | + |
1699 | + |
1700 | +def uid_exists(uid): |
1701 | + """Check if a uid exists""" |
1702 | + try: |
1703 | + pwd.getpwuid(uid) |
1704 | + uid_exists = True |
1705 | + except KeyError: |
1706 | + uid_exists = False |
1707 | + return uid_exists |
1708 | + |
1709 | + |
1710 | +def group_exists(groupname): |
1711 | + """Check if a group exists""" |
1712 | + try: |
1713 | + grp.getgrnam(groupname) |
1714 | + group_exists = True |
1715 | + except KeyError: |
1716 | + group_exists = False |
1717 | + return group_exists |
1718 | + |
1719 | + |
1720 | +def gid_exists(gid): |
1721 | + """Check if a gid exists""" |
1722 | + try: |
1723 | + grp.getgrgid(gid) |
1724 | + gid_exists = True |
1725 | + except KeyError: |
1726 | + gid_exists = False |
1727 | + return gid_exists |
1728 | + |
1729 | + |
1730 | +def add_group(group_name, system_group=False, gid=None): |
1731 | + """Add a group to the system |
1732 | + |
1733 | + Will log but otherwise succeed if the group already exists. |
1734 | + |
1735 | + :param str group_name: group to create |
1736 | + :param bool system_group: Create system group |
1737 | + :param int gid: GID for user being created |
1738 | + |
1739 | + :returns: The password database entry struct, as returned by `grp.getgrnam` |
1740 | + """ |
1741 | try: |
1742 | group_info = grp.getgrnam(group_name) |
1743 | log('group {0} already exists!'.format(group_name)) |
1744 | + if gid: |
1745 | + group_info = grp.getgrgid(gid) |
1746 | + log('group with gid {0} already exists!'.format(gid)) |
1747 | except KeyError: |
1748 | log('creating group {0}'.format(group_name)) |
1749 | - cmd = ['addgroup'] |
1750 | - if system_group: |
1751 | - cmd.append('--system') |
1752 | - else: |
1753 | - cmd.extend([ |
1754 | - '--group', |
1755 | - ]) |
1756 | - cmd.append(group_name) |
1757 | - subprocess.check_call(cmd) |
1758 | + add_new_group(group_name, system_group, gid) |
1759 | group_info = grp.getgrnam(group_name) |
1760 | return group_info |
1761 | |
1762 | |
1763 | def add_user_to_group(username, group): |
1764 | """Add a user to a group""" |
1765 | - cmd = [ |
1766 | - 'gpasswd', '-a', |
1767 | - username, |
1768 | - group |
1769 | - ] |
1770 | + cmd = ['gpasswd', '-a', username, group] |
1771 | log("Adding user {} to group {}".format(username, group)) |
1772 | subprocess.check_call(cmd) |
1773 | |
1774 | |
1775 | -def rsync(from_path, to_path, flags='-r', options=None): |
1776 | +def rsync(from_path, to_path, flags='-r', options=None, timeout=None): |
1777 | """Replicate the contents of a path""" |
1778 | options = options or ['--delete', '--executability'] |
1779 | cmd = ['/usr/bin/rsync', flags] |
1780 | + if timeout: |
1781 | + cmd = ['timeout', str(timeout)] + cmd |
1782 | cmd.extend(options) |
1783 | cmd.append(from_path) |
1784 | cmd.append(to_path) |
1785 | log(" ".join(cmd)) |
1786 | - return subprocess.check_output(cmd).decode('UTF-8').strip() |
1787 | + return subprocess.check_output(cmd, stderr=subprocess.STDOUT).decode('UTF-8').strip() |
1788 | |
1789 | |
1790 | def symlink(source, destination): |
1791 | @@ -202,14 +493,12 @@ |
1792 | |
1793 | |
1794 | def fstab_remove(mp): |
1795 | - """Remove the given mountpoint entry from /etc/fstab |
1796 | - """ |
1797 | + """Remove the given mountpoint entry from /etc/fstab""" |
1798 | return Fstab.remove_by_mountpoint(mp) |
1799 | |
1800 | |
1801 | def fstab_add(dev, mp, fs, options=None): |
1802 | - """Adds the given device entry to the /etc/fstab file |
1803 | - """ |
1804 | + """Adds the given device entry to the /etc/fstab file""" |
1805 | return Fstab.add(dev, mp, fs, options=options) |
1806 | |
1807 | |
1808 | @@ -253,9 +542,19 @@ |
1809 | return system_mounts |
1810 | |
1811 | |
1812 | +def fstab_mount(mountpoint): |
1813 | + """Mount filesystem using fstab""" |
1814 | + cmd_args = ['mount', mountpoint] |
1815 | + try: |
1816 | + subprocess.check_output(cmd_args) |
1817 | + except subprocess.CalledProcessError as e: |
1818 | + log('Error unmounting {}\n{}'.format(mountpoint, e.output)) |
1819 | + return False |
1820 | + return True |
1821 | + |
1822 | + |
1823 | def file_hash(path, hash_type='md5'): |
1824 | - """ |
1825 | - Generate a hash checksum of the contents of 'path' or None if not found. |
1826 | + """Generate a hash checksum of the contents of 'path' or None if not found. |
1827 | |
1828 | :param str hash_type: Any hash alrgorithm supported by :mod:`hashlib`, |
1829 | such as md5, sha1, sha256, sha512, etc. |
1830 | @@ -269,9 +568,22 @@ |
1831 | return None |
1832 | |
1833 | |
1834 | +def path_hash(path): |
1835 | + """Generate a hash checksum of all files matching 'path'. Standard |
1836 | + wildcards like '*' and '?' are supported, see documentation for the 'glob' |
1837 | + module for more information. |
1838 | + |
1839 | + :return: dict: A { filename: hash } dictionary for all matched files. |
1840 | + Empty if none found. |
1841 | + """ |
1842 | + return { |
1843 | + filename: file_hash(filename) |
1844 | + for filename in glob.iglob(path) |
1845 | + } |
1846 | + |
1847 | + |
1848 | def check_hash(path, checksum, hash_type='md5'): |
1849 | - """ |
1850 | - Validate a file using a cryptographic checksum. |
1851 | + """Validate a file using a cryptographic checksum. |
1852 | |
1853 | :param str checksum: Value of the checksum used to validate the file. |
1854 | :param str hash_type: Hash algorithm used to generate `checksum`. |
1855 | @@ -286,54 +598,78 @@ |
1856 | |
1857 | |
1858 | class ChecksumError(ValueError): |
1859 | + """A class derived from Value error to indicate the checksum failed.""" |
1860 | pass |
1861 | |
1862 | |
1863 | -def restart_on_change(restart_map, stopstart=False): |
1864 | +def restart_on_change(restart_map, stopstart=False, restart_functions=None): |
1865 | """Restart services based on configuration files changing |
1866 | |
1867 | This function is used a decorator, for example:: |
1868 | |
1869 | @restart_on_change({ |
1870 | '/etc/ceph/ceph.conf': [ 'cinder-api', 'cinder-volume' ] |
1871 | + '/etc/apache/sites-enabled/*': [ 'apache2' ] |
1872 | }) |
1873 | - def ceph_client_changed(): |
1874 | + def config_changed(): |
1875 | pass # your code here |
1876 | |
1877 | In this example, the cinder-api and cinder-volume services |
1878 | would be restarted if /etc/ceph/ceph.conf is changed by the |
1879 | - ceph_client_changed function. |
1880 | + ceph_client_changed function. The apache2 service would be |
1881 | + restarted if any file matching the pattern got changed, created |
1882 | + or removed. Standard wildcards are supported, see documentation |
1883 | + for the 'glob' module for more information. |
1884 | + |
1885 | + @param restart_map: {path_file_name: [service_name, ...] |
1886 | + @param stopstart: DEFAULT false; whether to stop, start OR restart |
1887 | + @param restart_functions: nonstandard functions to use to restart services |
1888 | + {svc: func, ...} |
1889 | + @returns result from decorated function |
1890 | """ |
1891 | def wrap(f): |
1892 | + @functools.wraps(f) |
1893 | def wrapped_f(*args, **kwargs): |
1894 | - checksums = {} |
1895 | - for path in restart_map: |
1896 | - checksums[path] = file_hash(path) |
1897 | - f(*args, **kwargs) |
1898 | - restarts = [] |
1899 | - for path in restart_map: |
1900 | - if checksums[path] != file_hash(path): |
1901 | - restarts += restart_map[path] |
1902 | - services_list = list(OrderedDict.fromkeys(restarts)) |
1903 | - if not stopstart: |
1904 | - for service_name in services_list: |
1905 | - service('restart', service_name) |
1906 | - else: |
1907 | - for action in ['stop', 'start']: |
1908 | - for service_name in services_list: |
1909 | - service(action, service_name) |
1910 | + return restart_on_change_helper( |
1911 | + (lambda: f(*args, **kwargs)), restart_map, stopstart, |
1912 | + restart_functions) |
1913 | return wrapped_f |
1914 | return wrap |
1915 | |
1916 | |
1917 | -def lsb_release(): |
1918 | - """Return /etc/lsb-release in a dict""" |
1919 | - d = {} |
1920 | - with open('/etc/lsb-release', 'r') as lsb: |
1921 | - for l in lsb: |
1922 | - k, v = l.split('=') |
1923 | - d[k.strip()] = v.strip() |
1924 | - return d |
1925 | +def restart_on_change_helper(lambda_f, restart_map, stopstart=False, |
1926 | + restart_functions=None): |
1927 | + """Helper function to perform the restart_on_change function. |
1928 | + |
1929 | + This is provided for decorators to restart services if files described |
1930 | + in the restart_map have changed after an invocation of lambda_f(). |
1931 | + |
1932 | + @param lambda_f: function to call. |
1933 | + @param restart_map: {file: [service, ...]} |
1934 | + @param stopstart: whether to stop, start or restart a service |
1935 | + @param restart_functions: nonstandard functions to use to restart services |
1936 | + {svc: func, ...} |
1937 | + @returns result of lambda_f() |
1938 | + """ |
1939 | + if restart_functions is None: |
1940 | + restart_functions = {} |
1941 | + checksums = {path: path_hash(path) for path in restart_map} |
1942 | + r = lambda_f() |
1943 | + # create a list of lists of the services to restart |
1944 | + restarts = [restart_map[path] |
1945 | + for path in restart_map |
1946 | + if path_hash(path) != checksums[path]] |
1947 | + # create a flat list of ordered services without duplicates from lists |
1948 | + services_list = list(OrderedDict.fromkeys(itertools.chain(*restarts))) |
1949 | + if services_list: |
1950 | + actions = ('stop', 'start') if stopstart else ('restart',) |
1951 | + for service_name in services_list: |
1952 | + if service_name in restart_functions: |
1953 | + restart_functions[service_name](service_name) |
1954 | + else: |
1955 | + for action in actions: |
1956 | + service(action, service_name) |
1957 | + return r |
1958 | |
1959 | |
1960 | def pwgen(length=None): |
1961 | @@ -352,36 +688,92 @@ |
1962 | return(''.join(random_chars)) |
1963 | |
1964 | |
1965 | -def list_nics(nic_type): |
1966 | - '''Return a list of nics of given type(s)''' |
1967 | +def is_phy_iface(interface): |
1968 | + """Returns True if interface is not virtual, otherwise False.""" |
1969 | + if interface: |
1970 | + sys_net = '/sys/class/net' |
1971 | + if os.path.isdir(sys_net): |
1972 | + for iface in glob.glob(os.path.join(sys_net, '*')): |
1973 | + if '/virtual/' in os.path.realpath(iface): |
1974 | + continue |
1975 | + |
1976 | + if interface == os.path.basename(iface): |
1977 | + return True |
1978 | + |
1979 | + return False |
1980 | + |
1981 | + |
1982 | +def get_bond_master(interface): |
1983 | + """Returns bond master if interface is bond slave otherwise None. |
1984 | + |
1985 | + NOTE: the provided interface is expected to be physical |
1986 | + """ |
1987 | + if interface: |
1988 | + iface_path = '/sys/class/net/%s' % (interface) |
1989 | + if os.path.exists(iface_path): |
1990 | + if '/virtual/' in os.path.realpath(iface_path): |
1991 | + return None |
1992 | + |
1993 | + master = os.path.join(iface_path, 'master') |
1994 | + if os.path.exists(master): |
1995 | + master = os.path.realpath(master) |
1996 | + # make sure it is a bond master |
1997 | + if os.path.exists(os.path.join(master, 'bonding')): |
1998 | + return os.path.basename(master) |
1999 | + |
2000 | + return None |
2001 | + |
2002 | + |
2003 | +def list_nics(nic_type=None): |
2004 | + """Return a list of nics of given type(s)""" |
2005 | if isinstance(nic_type, six.string_types): |
2006 | int_types = [nic_type] |
2007 | else: |
2008 | int_types = nic_type |
2009 | + |
2010 | interfaces = [] |
2011 | - for int_type in int_types: |
2012 | - cmd = ['ip', 'addr', 'show', 'label', int_type + '*'] |
2013 | + if nic_type: |
2014 | + for int_type in int_types: |
2015 | + cmd = ['ip', 'addr', 'show', 'label', int_type + '*'] |
2016 | + ip_output = subprocess.check_output(cmd).decode('UTF-8') |
2017 | + ip_output = ip_output.split('\n') |
2018 | + ip_output = (line for line in ip_output if line) |
2019 | + for line in ip_output: |
2020 | + if line.split()[1].startswith(int_type): |
2021 | + matched = re.search('.*: (' + int_type + |
2022 | + r'[0-9]+\.[0-9]+)@.*', line) |
2023 | + if matched: |
2024 | + iface = matched.groups()[0] |
2025 | + else: |
2026 | + iface = line.split()[1].replace(":", "") |
2027 | + |
2028 | + if iface not in interfaces: |
2029 | + interfaces.append(iface) |
2030 | + else: |
2031 | + cmd = ['ip', 'a'] |
2032 | ip_output = subprocess.check_output(cmd).decode('UTF-8').split('\n') |
2033 | - ip_output = (line for line in ip_output if line) |
2034 | + ip_output = (line.strip() for line in ip_output if line) |
2035 | + |
2036 | + key = re.compile('^[0-9]+:\s+(.+):') |
2037 | for line in ip_output: |
2038 | - if line.split()[1].startswith(int_type): |
2039 | - matched = re.search('.*: (' + int_type + r'[0-9]+\.[0-9]+)@.*', line) |
2040 | - if matched: |
2041 | - interface = matched.groups()[0] |
2042 | - else: |
2043 | - interface = line.split()[1].replace(":", "") |
2044 | - interfaces.append(interface) |
2045 | + matched = re.search(key, line) |
2046 | + if matched: |
2047 | + iface = matched.group(1) |
2048 | + iface = iface.partition("@")[0] |
2049 | + if iface not in interfaces: |
2050 | + interfaces.append(iface) |
2051 | |
2052 | return interfaces |
2053 | |
2054 | |
2055 | def set_nic_mtu(nic, mtu): |
2056 | - '''Set MTU on a network interface''' |
2057 | + """Set the Maximum Transmission Unit (MTU) on a network interface.""" |
2058 | cmd = ['ip', 'link', 'set', nic, 'mtu', mtu] |
2059 | subprocess.check_call(cmd) |
2060 | |
2061 | |
2062 | def get_nic_mtu(nic): |
2063 | + """Return the Maximum Transmission Unit (MTU) for a network interface.""" |
2064 | cmd = ['ip', 'addr', 'show', nic] |
2065 | ip_output = subprocess.check_output(cmd).decode('UTF-8').split('\n') |
2066 | mtu = "" |
2067 | @@ -393,6 +785,7 @@ |
2068 | |
2069 | |
2070 | def get_nic_hwaddr(nic): |
2071 | + """Return the Media Access Control (MAC) for a network interface.""" |
2072 | cmd = ['ip', '-o', '-0', 'addr', 'show', nic] |
2073 | ip_output = subprocess.check_output(cmd).decode('UTF-8') |
2074 | hwaddr = "" |
2075 | @@ -402,35 +795,31 @@ |
2076 | return hwaddr |
2077 | |
2078 | |
2079 | -def cmp_pkgrevno(package, revno, pkgcache=None): |
2080 | - '''Compare supplied revno with the revno of the installed package |
2081 | - |
2082 | - * 1 => Installed revno is greater than supplied arg |
2083 | - * 0 => Installed revno is the same as supplied arg |
2084 | - * -1 => Installed revno is less than supplied arg |
2085 | - |
2086 | - This function imports apt_cache function from charmhelpers.fetch if |
2087 | - the pkgcache argument is None. Be sure to add charmhelpers.fetch if |
2088 | - you call this function, or pass an apt_pkg.Cache() instance. |
2089 | - ''' |
2090 | - import apt_pkg |
2091 | - if not pkgcache: |
2092 | - from charmhelpers.fetch import apt_cache |
2093 | - pkgcache = apt_cache() |
2094 | - pkg = pkgcache[package] |
2095 | - return apt_pkg.version_compare(pkg.current_ver.ver_str, revno) |
2096 | - |
2097 | - |
2098 | @contextmanager |
2099 | -def chdir(d): |
2100 | +def chdir(directory): |
2101 | + """Change the current working directory to a different directory for a code |
2102 | + block and return the previous directory after the block exits. Useful to |
2103 | + run commands from a specificed directory. |
2104 | + |
2105 | + :param str directory: The directory path to change to for this context. |
2106 | + """ |
2107 | cur = os.getcwd() |
2108 | try: |
2109 | - yield os.chdir(d) |
2110 | + yield os.chdir(directory) |
2111 | finally: |
2112 | os.chdir(cur) |
2113 | |
2114 | |
2115 | -def chownr(path, owner, group, follow_links=True): |
2116 | +def chownr(path, owner, group, follow_links=True, chowntopdir=False): |
2117 | + """Recursively change user and group ownership of files and directories |
2118 | + in given path. Doesn't chown path itself by default, only its children. |
2119 | + |
2120 | + :param str path: The string path to start changing ownership. |
2121 | + :param str owner: The owner string to use when looking up the uid. |
2122 | + :param str group: The group string to use when looking up the gid. |
2123 | + :param bool follow_links: Also follow and chown links if True |
2124 | + :param bool chowntopdir: Also chown path itself if True |
2125 | + """ |
2126 | uid = pwd.getpwnam(owner).pw_uid |
2127 | gid = grp.getgrnam(group).gr_gid |
2128 | if follow_links: |
2129 | @@ -438,7 +827,11 @@ |
2130 | else: |
2131 | chown = os.lchown |
2132 | |
2133 | - for root, dirs, files in os.walk(path): |
2134 | + if chowntopdir: |
2135 | + broken_symlink = os.path.lexists(path) and not os.path.exists(path) |
2136 | + if not broken_symlink: |
2137 | + chown(path, uid, gid) |
2138 | + for root, dirs, files in os.walk(path, followlinks=follow_links): |
2139 | for name in dirs + files: |
2140 | full = os.path.join(root, name) |
2141 | broken_symlink = os.path.lexists(full) and not os.path.exists(full) |
2142 | @@ -447,4 +840,81 @@ |
2143 | |
2144 | |
2145 | def lchownr(path, owner, group): |
2146 | + """Recursively change user and group ownership of files and directories |
2147 | + in a given path, not following symbolic links. See the documentation for |
2148 | + 'os.lchown' for more information. |
2149 | + |
2150 | + :param str path: The string path to start changing ownership. |
2151 | + :param str owner: The owner string to use when looking up the uid. |
2152 | + :param str group: The group string to use when looking up the gid. |
2153 | + """ |
2154 | chownr(path, owner, group, follow_links=False) |
2155 | + |
2156 | + |
2157 | +def owner(path): |
2158 | + """Returns a tuple containing the username & groupname owning the path. |
2159 | + |
2160 | + :param str path: the string path to retrieve the ownership |
2161 | + :return tuple(str, str): A (username, groupname) tuple containing the |
2162 | + name of the user and group owning the path. |
2163 | + :raises OSError: if the specified path does not exist |
2164 | + """ |
2165 | + stat = os.stat(path) |
2166 | + username = pwd.getpwuid(stat.st_uid)[0] |
2167 | + groupname = grp.getgrgid(stat.st_gid)[0] |
2168 | + return username, groupname |
2169 | + |
2170 | + |
2171 | +def get_total_ram(): |
2172 | + """The total amount of system RAM in bytes. |
2173 | + |
2174 | + This is what is reported by the OS, and may be overcommitted when |
2175 | + there are multiple containers hosted on the same machine. |
2176 | + """ |
2177 | + with open('/proc/meminfo', 'r') as f: |
2178 | + for line in f.readlines(): |
2179 | + if line: |
2180 | + key, value, unit = line.split() |
2181 | + if key == 'MemTotal:': |
2182 | + assert unit == 'kB', 'Unknown unit' |
2183 | + return int(value) * 1024 # Classic, not KiB. |
2184 | + raise NotImplementedError() |
2185 | + |
2186 | + |
2187 | +UPSTART_CONTAINER_TYPE = '/run/container_type' |
2188 | + |
2189 | + |
2190 | +def is_container(): |
2191 | + """Determine whether unit is running in a container |
2192 | + |
2193 | + @return: boolean indicating if unit is in a container |
2194 | + """ |
2195 | + if init_is_systemd(): |
2196 | + # Detect using systemd-detect-virt |
2197 | + return subprocess.call(['systemd-detect-virt', |
2198 | + '--container']) == 0 |
2199 | + else: |
2200 | + # Detect using upstart container file marker |
2201 | + return os.path.exists(UPSTART_CONTAINER_TYPE) |
2202 | + |
2203 | + |
2204 | +def add_to_updatedb_prunepath(path, updatedb_path=UPDATEDB_PATH): |
2205 | + with open(updatedb_path, 'r+') as f_id: |
2206 | + updatedb_text = f_id.read() |
2207 | + output = updatedb(updatedb_text, path) |
2208 | + f_id.seek(0) |
2209 | + f_id.write(output) |
2210 | + f_id.truncate() |
2211 | + |
2212 | + |
2213 | +def updatedb(updatedb_text, new_path): |
2214 | + lines = [line for line in updatedb_text.split("\n")] |
2215 | + for i, line in enumerate(lines): |
2216 | + if line.startswith("PRUNEPATHS="): |
2217 | + paths_line = line.split("=")[1].replace('"', '') |
2218 | + paths = paths_line.split(" ") |
2219 | + if new_path not in paths: |
2220 | + paths.append(new_path) |
2221 | + lines[i] = 'PRUNEPATHS="{}"'.format(' '.join(paths)) |
2222 | + output = "\n".join(lines) |
2223 | + return output |
2224 | |
2225 | === added directory 'hooks/charmhelpers/core/host_factory' |
2226 | === added file 'hooks/charmhelpers/core/host_factory/__init__.py' |
2227 | === added file 'hooks/charmhelpers/core/host_factory/centos.py' |
2228 | --- hooks/charmhelpers/core/host_factory/centos.py 1970-01-01 00:00:00 +0000 |
2229 | +++ hooks/charmhelpers/core/host_factory/centos.py 2018-06-12 20:07:13 +0000 |
2230 | @@ -0,0 +1,56 @@ |
2231 | +import subprocess |
2232 | +import yum |
2233 | +import os |
2234 | + |
2235 | + |
2236 | +def service_available(service_name): |
2237 | + # """Determine whether a system service is available.""" |
2238 | + if os.path.isdir('/run/systemd/system'): |
2239 | + cmd = ['systemctl', 'is-enabled', service_name] |
2240 | + else: |
2241 | + cmd = ['service', service_name, 'is-enabled'] |
2242 | + return subprocess.call(cmd) == 0 |
2243 | + |
2244 | + |
2245 | +def add_new_group(group_name, system_group=False, gid=None): |
2246 | + cmd = ['groupadd'] |
2247 | + if gid: |
2248 | + cmd.extend(['--gid', str(gid)]) |
2249 | + if system_group: |
2250 | + cmd.append('-r') |
2251 | + cmd.append(group_name) |
2252 | + subprocess.check_call(cmd) |
2253 | + |
2254 | + |
2255 | +def lsb_release(): |
2256 | + """Return /etc/os-release in a dict.""" |
2257 | + d = {} |
2258 | + with open('/etc/os-release', 'r') as lsb: |
2259 | + for l in lsb: |
2260 | + s = l.split('=') |
2261 | + if len(s) != 2: |
2262 | + continue |
2263 | + d[s[0].strip()] = s[1].strip() |
2264 | + return d |
2265 | + |
2266 | + |
2267 | +def cmp_pkgrevno(package, revno, pkgcache=None): |
2268 | + """Compare supplied revno with the revno of the installed package. |
2269 | + |
2270 | + * 1 => Installed revno is greater than supplied arg |
2271 | + * 0 => Installed revno is the same as supplied arg |
2272 | + * -1 => Installed revno is less than supplied arg |
2273 | + |
2274 | + This function imports YumBase function if the pkgcache argument |
2275 | + is None. |
2276 | + """ |
2277 | + if not pkgcache: |
2278 | + y = yum.YumBase() |
2279 | + packages = y.doPackageLists() |
2280 | + pkgcache = {i.Name: i.version for i in packages['installed']} |
2281 | + pkg = pkgcache[package] |
2282 | + if pkg > revno: |
2283 | + return 1 |
2284 | + if pkg < revno: |
2285 | + return -1 |
2286 | + return 0 |
2287 | |
2288 | === added file 'hooks/charmhelpers/core/host_factory/ubuntu.py' |
2289 | --- hooks/charmhelpers/core/host_factory/ubuntu.py 1970-01-01 00:00:00 +0000 |
2290 | +++ hooks/charmhelpers/core/host_factory/ubuntu.py 2018-06-12 20:07:13 +0000 |
2291 | @@ -0,0 +1,56 @@ |
2292 | +import subprocess |
2293 | + |
2294 | + |
2295 | +def service_available(service_name): |
2296 | + """Determine whether a system service is available""" |
2297 | + try: |
2298 | + subprocess.check_output( |
2299 | + ['service', service_name, 'status'], |
2300 | + stderr=subprocess.STDOUT).decode('UTF-8') |
2301 | + except subprocess.CalledProcessError as e: |
2302 | + return b'unrecognized service' not in e.output |
2303 | + else: |
2304 | + return True |
2305 | + |
2306 | + |
2307 | +def add_new_group(group_name, system_group=False, gid=None): |
2308 | + cmd = ['addgroup'] |
2309 | + if gid: |
2310 | + cmd.extend(['--gid', str(gid)]) |
2311 | + if system_group: |
2312 | + cmd.append('--system') |
2313 | + else: |
2314 | + cmd.extend([ |
2315 | + '--group', |
2316 | + ]) |
2317 | + cmd.append(group_name) |
2318 | + subprocess.check_call(cmd) |
2319 | + |
2320 | + |
2321 | +def lsb_release(): |
2322 | + """Return /etc/lsb-release in a dict""" |
2323 | + d = {} |
2324 | + with open('/etc/lsb-release', 'r') as lsb: |
2325 | + for l in lsb: |
2326 | + k, v = l.split('=') |
2327 | + d[k.strip()] = v.strip() |
2328 | + return d |
2329 | + |
2330 | + |
2331 | +def cmp_pkgrevno(package, revno, pkgcache=None): |
2332 | + """Compare supplied revno with the revno of the installed package. |
2333 | + |
2334 | + * 1 => Installed revno is greater than supplied arg |
2335 | + * 0 => Installed revno is the same as supplied arg |
2336 | + * -1 => Installed revno is less than supplied arg |
2337 | + |
2338 | + This function imports apt_cache function from charmhelpers.fetch if |
2339 | + the pkgcache argument is None. Be sure to add charmhelpers.fetch if |
2340 | + you call this function, or pass an apt_pkg.Cache() instance. |
2341 | + """ |
2342 | + import apt_pkg |
2343 | + if not pkgcache: |
2344 | + from charmhelpers.fetch import apt_cache |
2345 | + pkgcache = apt_cache() |
2346 | + pkg = pkgcache[package] |
2347 | + return apt_pkg.version_compare(pkg.current_ver.ver_str, revno) |
2348 | |
2349 | === added file 'hooks/charmhelpers/core/hugepage.py' |
2350 | --- hooks/charmhelpers/core/hugepage.py 1970-01-01 00:00:00 +0000 |
2351 | +++ hooks/charmhelpers/core/hugepage.py 2018-06-12 20:07:13 +0000 |
2352 | @@ -0,0 +1,69 @@ |
2353 | +# -*- coding: utf-8 -*- |
2354 | + |
2355 | +# Copyright 2014-2015 Canonical Limited. |
2356 | +# |
2357 | +# Licensed under the Apache License, Version 2.0 (the "License"); |
2358 | +# you may not use this file except in compliance with the License. |
2359 | +# You may obtain a copy of the License at |
2360 | +# |
2361 | +# http://www.apache.org/licenses/LICENSE-2.0 |
2362 | +# |
2363 | +# Unless required by applicable law or agreed to in writing, software |
2364 | +# distributed under the License is distributed on an "AS IS" BASIS, |
2365 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
2366 | +# See the License for the specific language governing permissions and |
2367 | +# limitations under the License. |
2368 | + |
2369 | +import yaml |
2370 | +from charmhelpers.core import fstab |
2371 | +from charmhelpers.core import sysctl |
2372 | +from charmhelpers.core.host import ( |
2373 | + add_group, |
2374 | + add_user_to_group, |
2375 | + fstab_mount, |
2376 | + mkdir, |
2377 | +) |
2378 | +from charmhelpers.core.strutils import bytes_from_string |
2379 | +from subprocess import check_output |
2380 | + |
2381 | + |
2382 | +def hugepage_support(user, group='hugetlb', nr_hugepages=256, |
2383 | + max_map_count=65536, mnt_point='/run/hugepages/kvm', |
2384 | + pagesize='2MB', mount=True, set_shmmax=False): |
2385 | + """Enable hugepages on system. |
2386 | + |
2387 | + Args: |
2388 | + user (str) -- Username to allow access to hugepages to |
2389 | + group (str) -- Group name to own hugepages |
2390 | + nr_hugepages (int) -- Number of pages to reserve |
2391 | + max_map_count (int) -- Number of Virtual Memory Areas a process can own |
2392 | + mnt_point (str) -- Directory to mount hugepages on |
2393 | + pagesize (str) -- Size of hugepages |
2394 | + mount (bool) -- Whether to Mount hugepages |
2395 | + """ |
2396 | + group_info = add_group(group) |
2397 | + gid = group_info.gr_gid |
2398 | + add_user_to_group(user, group) |
2399 | + if max_map_count < 2 * nr_hugepages: |
2400 | + max_map_count = 2 * nr_hugepages |
2401 | + sysctl_settings = { |
2402 | + 'vm.nr_hugepages': nr_hugepages, |
2403 | + 'vm.max_map_count': max_map_count, |
2404 | + 'vm.hugetlb_shm_group': gid, |
2405 | + } |
2406 | + if set_shmmax: |
2407 | + shmmax_current = int(check_output(['sysctl', '-n', 'kernel.shmmax'])) |
2408 | + shmmax_minsize = bytes_from_string(pagesize) * nr_hugepages |
2409 | + if shmmax_minsize > shmmax_current: |
2410 | + sysctl_settings['kernel.shmmax'] = shmmax_minsize |
2411 | + sysctl.create(yaml.dump(sysctl_settings), '/etc/sysctl.d/10-hugepage.conf') |
2412 | + mkdir(mnt_point, owner='root', group='root', perms=0o755, force=False) |
2413 | + lfstab = fstab.Fstab() |
2414 | + fstab_entry = lfstab.get_entry_by_attr('mountpoint', mnt_point) |
2415 | + if fstab_entry: |
2416 | + lfstab.remove_entry(fstab_entry) |
2417 | + entry = lfstab.Entry('nodev', mnt_point, 'hugetlbfs', |
2418 | + 'mode=1770,gid={},pagesize={}'.format(gid, pagesize), 0, 0) |
2419 | + lfstab.add_entry(entry) |
2420 | + if mount: |
2421 | + fstab_mount(mnt_point) |
2422 | |
2423 | === added file 'hooks/charmhelpers/core/kernel.py' |
2424 | --- hooks/charmhelpers/core/kernel.py 1970-01-01 00:00:00 +0000 |
2425 | +++ hooks/charmhelpers/core/kernel.py 2018-06-12 20:07:13 +0000 |
2426 | @@ -0,0 +1,72 @@ |
2427 | +#!/usr/bin/env python |
2428 | +# -*- coding: utf-8 -*- |
2429 | + |
2430 | +# Copyright 2014-2015 Canonical Limited. |
2431 | +# |
2432 | +# Licensed under the Apache License, Version 2.0 (the "License"); |
2433 | +# you may not use this file except in compliance with the License. |
2434 | +# You may obtain a copy of the License at |
2435 | +# |
2436 | +# http://www.apache.org/licenses/LICENSE-2.0 |
2437 | +# |
2438 | +# Unless required by applicable law or agreed to in writing, software |
2439 | +# distributed under the License is distributed on an "AS IS" BASIS, |
2440 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
2441 | +# See the License for the specific language governing permissions and |
2442 | +# limitations under the License. |
2443 | + |
2444 | +import re |
2445 | +import subprocess |
2446 | + |
2447 | +from charmhelpers.osplatform import get_platform |
2448 | +from charmhelpers.core.hookenv import ( |
2449 | + log, |
2450 | + INFO |
2451 | +) |
2452 | + |
2453 | +__platform__ = get_platform() |
2454 | +if __platform__ == "ubuntu": |
2455 | + from charmhelpers.core.kernel_factory.ubuntu import ( |
2456 | + persistent_modprobe, |
2457 | + update_initramfs, |
2458 | + ) # flake8: noqa -- ignore F401 for this import |
2459 | +elif __platform__ == "centos": |
2460 | + from charmhelpers.core.kernel_factory.centos import ( |
2461 | + persistent_modprobe, |
2462 | + update_initramfs, |
2463 | + ) # flake8: noqa -- ignore F401 for this import |
2464 | + |
2465 | +__author__ = "Jorge Niedbalski <jorge.niedbalski@canonical.com>" |
2466 | + |
2467 | + |
2468 | +def modprobe(module, persist=True): |
2469 | + """Load a kernel module and configure for auto-load on reboot.""" |
2470 | + cmd = ['modprobe', module] |
2471 | + |
2472 | + log('Loading kernel module %s' % module, level=INFO) |
2473 | + |
2474 | + subprocess.check_call(cmd) |
2475 | + if persist: |
2476 | + persistent_modprobe(module) |
2477 | + |
2478 | + |
2479 | +def rmmod(module, force=False): |
2480 | + """Remove a module from the linux kernel""" |
2481 | + cmd = ['rmmod'] |
2482 | + if force: |
2483 | + cmd.append('-f') |
2484 | + cmd.append(module) |
2485 | + log('Removing kernel module %s' % module, level=INFO) |
2486 | + return subprocess.check_call(cmd) |
2487 | + |
2488 | + |
2489 | +def lsmod(): |
2490 | + """Shows what kernel modules are currently loaded""" |
2491 | + return subprocess.check_output(['lsmod'], |
2492 | + universal_newlines=True) |
2493 | + |
2494 | + |
2495 | +def is_module_loaded(module): |
2496 | + """Checks if a kernel module is already loaded""" |
2497 | + matches = re.findall('^%s[ ]+' % module, lsmod(), re.M) |
2498 | + return len(matches) > 0 |
2499 | |
2500 | === added directory 'hooks/charmhelpers/core/kernel_factory' |
2501 | === added file 'hooks/charmhelpers/core/kernel_factory/__init__.py' |
2502 | === added file 'hooks/charmhelpers/core/kernel_factory/centos.py' |
2503 | --- hooks/charmhelpers/core/kernel_factory/centos.py 1970-01-01 00:00:00 +0000 |
2504 | +++ hooks/charmhelpers/core/kernel_factory/centos.py 2018-06-12 20:07:13 +0000 |
2505 | @@ -0,0 +1,17 @@ |
2506 | +import subprocess |
2507 | +import os |
2508 | + |
2509 | + |
2510 | +def persistent_modprobe(module): |
2511 | + """Load a kernel module and configure for auto-load on reboot.""" |
2512 | + if not os.path.exists('/etc/rc.modules'): |
2513 | + open('/etc/rc.modules', 'a') |
2514 | + os.chmod('/etc/rc.modules', 111) |
2515 | + with open('/etc/rc.modules', 'r+') as modules: |
2516 | + if module not in modules.read(): |
2517 | + modules.write('modprobe %s\n' % module) |
2518 | + |
2519 | + |
2520 | +def update_initramfs(version='all'): |
2521 | + """Updates an initramfs image.""" |
2522 | + return subprocess.check_call(["dracut", "-f", version]) |
2523 | |
2524 | === added file 'hooks/charmhelpers/core/kernel_factory/ubuntu.py' |
2525 | --- hooks/charmhelpers/core/kernel_factory/ubuntu.py 1970-01-01 00:00:00 +0000 |
2526 | +++ hooks/charmhelpers/core/kernel_factory/ubuntu.py 2018-06-12 20:07:13 +0000 |
2527 | @@ -0,0 +1,13 @@ |
2528 | +import subprocess |
2529 | + |
2530 | + |
2531 | +def persistent_modprobe(module): |
2532 | + """Load a kernel module and configure for auto-load on reboot.""" |
2533 | + with open('/etc/modules', 'r+') as modules: |
2534 | + if module not in modules.read(): |
2535 | + modules.write(module + "\n") |
2536 | + |
2537 | + |
2538 | +def update_initramfs(version='all'): |
2539 | + """Updates an initramfs image.""" |
2540 | + return subprocess.check_call(["update-initramfs", "-k", version, "-u"]) |
2541 | |
2542 | === modified file 'hooks/charmhelpers/core/services/__init__.py' |
2543 | --- hooks/charmhelpers/core/services/__init__.py 2015-02-09 12:58:07 +0000 |
2544 | +++ hooks/charmhelpers/core/services/__init__.py 2018-06-12 20:07:13 +0000 |
2545 | @@ -1,18 +1,16 @@ |
2546 | # Copyright 2014-2015 Canonical Limited. |
2547 | # |
2548 | -# This file is part of charm-helpers. |
2549 | -# |
2550 | -# charm-helpers is free software: you can redistribute it and/or modify |
2551 | -# it under the terms of the GNU Lesser General Public License version 3 as |
2552 | -# published by the Free Software Foundation. |
2553 | -# |
2554 | -# charm-helpers is distributed in the hope that it will be useful, |
2555 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
2556 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
2557 | -# GNU Lesser General Public License for more details. |
2558 | -# |
2559 | -# You should have received a copy of the GNU Lesser General Public License |
2560 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
2561 | +# Licensed under the Apache License, Version 2.0 (the "License"); |
2562 | +# you may not use this file except in compliance with the License. |
2563 | +# You may obtain a copy of the License at |
2564 | +# |
2565 | +# http://www.apache.org/licenses/LICENSE-2.0 |
2566 | +# |
2567 | +# Unless required by applicable law or agreed to in writing, software |
2568 | +# distributed under the License is distributed on an "AS IS" BASIS, |
2569 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
2570 | +# See the License for the specific language governing permissions and |
2571 | +# limitations under the License. |
2572 | |
2573 | from .base import * # NOQA |
2574 | from .helpers import * # NOQA |
2575 | |
2576 | === modified file 'hooks/charmhelpers/core/services/base.py' |
2577 | --- hooks/charmhelpers/core/services/base.py 2015-05-14 10:48:09 +0000 |
2578 | +++ hooks/charmhelpers/core/services/base.py 2018-06-12 20:07:13 +0000 |
2579 | @@ -1,22 +1,20 @@ |
2580 | # Copyright 2014-2015 Canonical Limited. |
2581 | # |
2582 | -# This file is part of charm-helpers. |
2583 | -# |
2584 | -# charm-helpers is free software: you can redistribute it and/or modify |
2585 | -# it under the terms of the GNU Lesser General Public License version 3 as |
2586 | -# published by the Free Software Foundation. |
2587 | -# |
2588 | -# charm-helpers is distributed in the hope that it will be useful, |
2589 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
2590 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
2591 | -# GNU Lesser General Public License for more details. |
2592 | -# |
2593 | -# You should have received a copy of the GNU Lesser General Public License |
2594 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
2595 | +# Licensed under the Apache License, Version 2.0 (the "License"); |
2596 | +# you may not use this file except in compliance with the License. |
2597 | +# You may obtain a copy of the License at |
2598 | +# |
2599 | +# http://www.apache.org/licenses/LICENSE-2.0 |
2600 | +# |
2601 | +# Unless required by applicable law or agreed to in writing, software |
2602 | +# distributed under the License is distributed on an "AS IS" BASIS, |
2603 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
2604 | +# See the License for the specific language governing permissions and |
2605 | +# limitations under the License. |
2606 | |
2607 | import os |
2608 | -import re |
2609 | import json |
2610 | +from inspect import getargspec |
2611 | from collections import Iterable, OrderedDict |
2612 | |
2613 | from charmhelpers.core import host |
2614 | @@ -128,15 +126,18 @@ |
2615 | """ |
2616 | Handle the current hook by doing The Right Thing with the registered services. |
2617 | """ |
2618 | - hook_name = hookenv.hook_name() |
2619 | - if hook_name == 'stop': |
2620 | - self.stop_services() |
2621 | - else: |
2622 | - self.provide_data() |
2623 | - self.reconfigure_services() |
2624 | - cfg = hookenv.config() |
2625 | - if cfg.implicit_save: |
2626 | - cfg.save() |
2627 | + hookenv._run_atstart() |
2628 | + try: |
2629 | + hook_name = hookenv.hook_name() |
2630 | + if hook_name == 'stop': |
2631 | + self.stop_services() |
2632 | + else: |
2633 | + self.reconfigure_services() |
2634 | + self.provide_data() |
2635 | + except SystemExit as x: |
2636 | + if x.code is None or x.code == 0: |
2637 | + hookenv._run_atexit() |
2638 | + hookenv._run_atexit() |
2639 | |
2640 | def provide_data(self): |
2641 | """ |
2642 | @@ -145,15 +146,36 @@ |
2643 | A provider must have a `name` attribute, which indicates which relation |
2644 | to set data on, and a `provide_data()` method, which returns a dict of |
2645 | data to set. |
2646 | + |
2647 | + The `provide_data()` method can optionally accept two parameters: |
2648 | + |
2649 | + * ``remote_service`` The name of the remote service that the data will |
2650 | + be provided to. The `provide_data()` method will be called once |
2651 | + for each connected service (not unit). This allows the method to |
2652 | + tailor its data to the given service. |
2653 | + * ``service_ready`` Whether or not the service definition had all of |
2654 | + its requirements met, and thus the ``data_ready`` callbacks run. |
2655 | + |
2656 | + Note that the ``provided_data`` methods are now called **after** the |
2657 | + ``data_ready`` callbacks are run. This gives the ``data_ready`` callbacks |
2658 | + a chance to generate any data necessary for the providing to the remote |
2659 | + services. |
2660 | """ |
2661 | - hook_name = hookenv.hook_name() |
2662 | - for service in self.services.values(): |
2663 | + for service_name, service in self.services.items(): |
2664 | + service_ready = self.is_ready(service_name) |
2665 | for provider in service.get('provided_data', []): |
2666 | - if re.match(r'{}-relation-(joined|changed)'.format(provider.name), hook_name): |
2667 | - data = provider.provide_data() |
2668 | - _ready = provider._is_ready(data) if hasattr(provider, '_is_ready') else data |
2669 | - if _ready: |
2670 | - hookenv.relation_set(None, data) |
2671 | + for relid in hookenv.relation_ids(provider.name): |
2672 | + units = hookenv.related_units(relid) |
2673 | + if not units: |
2674 | + continue |
2675 | + remote_service = units[0].split('/')[0] |
2676 | + argspec = getargspec(provider.provide_data) |
2677 | + if len(argspec.args) > 1: |
2678 | + data = provider.provide_data(remote_service, service_ready) |
2679 | + else: |
2680 | + data = provider.provide_data() |
2681 | + if data: |
2682 | + hookenv.relation_set(relid, data) |
2683 | |
2684 | def reconfigure_services(self, *service_names): |
2685 | """ |
2686 | |
2687 | === modified file 'hooks/charmhelpers/core/services/helpers.py' |
2688 | --- hooks/charmhelpers/core/services/helpers.py 2015-05-14 10:48:09 +0000 |
2689 | +++ hooks/charmhelpers/core/services/helpers.py 2018-06-12 20:07:13 +0000 |
2690 | @@ -1,22 +1,22 @@ |
2691 | # Copyright 2014-2015 Canonical Limited. |
2692 | # |
2693 | -# This file is part of charm-helpers. |
2694 | -# |
2695 | -# charm-helpers is free software: you can redistribute it and/or modify |
2696 | -# it under the terms of the GNU Lesser General Public License version 3 as |
2697 | -# published by the Free Software Foundation. |
2698 | -# |
2699 | -# charm-helpers is distributed in the hope that it will be useful, |
2700 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
2701 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
2702 | -# GNU Lesser General Public License for more details. |
2703 | -# |
2704 | -# You should have received a copy of the GNU Lesser General Public License |
2705 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
2706 | +# Licensed under the Apache License, Version 2.0 (the "License"); |
2707 | +# you may not use this file except in compliance with the License. |
2708 | +# You may obtain a copy of the License at |
2709 | +# |
2710 | +# http://www.apache.org/licenses/LICENSE-2.0 |
2711 | +# |
2712 | +# Unless required by applicable law or agreed to in writing, software |
2713 | +# distributed under the License is distributed on an "AS IS" BASIS, |
2714 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
2715 | +# See the License for the specific language governing permissions and |
2716 | +# limitations under the License. |
2717 | |
2718 | import os |
2719 | import yaml |
2720 | + |
2721 | from charmhelpers.core import hookenv |
2722 | +from charmhelpers.core import host |
2723 | from charmhelpers.core import templating |
2724 | |
2725 | from charmhelpers.core.services.base import ManagerCallback |
2726 | @@ -239,28 +239,51 @@ |
2727 | action. |
2728 | |
2729 | :param str source: The template source file, relative to |
2730 | - `$CHARM_DIR/templates` |
2731 | + `$CHARM_DIR/templates` |
2732 | |
2733 | - :param str target: The target to write the rendered template to |
2734 | + :param str target: The target to write the rendered template to (or None) |
2735 | :param str owner: The owner of the rendered file |
2736 | :param str group: The group of the rendered file |
2737 | :param int perms: The permissions of the rendered file |
2738 | + :param partial on_change_action: functools partial to be executed when |
2739 | + rendered file changes |
2740 | + :param jinja2 loader template_loader: A jinja2 template loader |
2741 | + |
2742 | + :return str: The rendered template |
2743 | """ |
2744 | def __init__(self, source, target, |
2745 | - owner='root', group='root', perms=0o444): |
2746 | + owner='root', group='root', perms=0o444, |
2747 | + on_change_action=None, template_loader=None): |
2748 | self.source = source |
2749 | self.target = target |
2750 | self.owner = owner |
2751 | self.group = group |
2752 | self.perms = perms |
2753 | + self.on_change_action = on_change_action |
2754 | + self.template_loader = template_loader |
2755 | |
2756 | def __call__(self, manager, service_name, event_name): |
2757 | + pre_checksum = '' |
2758 | + if self.on_change_action and os.path.isfile(self.target): |
2759 | + pre_checksum = host.file_hash(self.target) |
2760 | service = manager.get_service(service_name) |
2761 | - context = {} |
2762 | + context = {'ctx': {}} |
2763 | for ctx in service.get('required_data', []): |
2764 | context.update(ctx) |
2765 | - templating.render(self.source, self.target, context, |
2766 | - self.owner, self.group, self.perms) |
2767 | + context['ctx'].update(ctx) |
2768 | + |
2769 | + result = templating.render(self.source, self.target, context, |
2770 | + self.owner, self.group, self.perms, |
2771 | + template_loader=self.template_loader) |
2772 | + if self.on_change_action: |
2773 | + if pre_checksum == host.file_hash(self.target): |
2774 | + hookenv.log( |
2775 | + 'No change detected: {}'.format(self.target), |
2776 | + hookenv.DEBUG) |
2777 | + else: |
2778 | + self.on_change_action() |
2779 | + |
2780 | + return result |
2781 | |
2782 | |
2783 | # Convenience aliases for templates |
2784 | |
2785 | === modified file 'hooks/charmhelpers/core/strutils.py' |
2786 | --- hooks/charmhelpers/core/strutils.py 2015-05-14 10:48:09 +0000 |
2787 | +++ hooks/charmhelpers/core/strutils.py 2018-06-12 20:07:13 +0000 |
2788 | @@ -3,21 +3,20 @@ |
2789 | |
2790 | # Copyright 2014-2015 Canonical Limited. |
2791 | # |
2792 | -# This file is part of charm-helpers. |
2793 | -# |
2794 | -# charm-helpers is free software: you can redistribute it and/or modify |
2795 | -# it under the terms of the GNU Lesser General Public License version 3 as |
2796 | -# published by the Free Software Foundation. |
2797 | -# |
2798 | -# charm-helpers is distributed in the hope that it will be useful, |
2799 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
2800 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
2801 | -# GNU Lesser General Public License for more details. |
2802 | -# |
2803 | -# You should have received a copy of the GNU Lesser General Public License |
2804 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
2805 | +# Licensed under the Apache License, Version 2.0 (the "License"); |
2806 | +# you may not use this file except in compliance with the License. |
2807 | +# You may obtain a copy of the License at |
2808 | +# |
2809 | +# http://www.apache.org/licenses/LICENSE-2.0 |
2810 | +# |
2811 | +# Unless required by applicable law or agreed to in writing, software |
2812 | +# distributed under the License is distributed on an "AS IS" BASIS, |
2813 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
2814 | +# See the License for the specific language governing permissions and |
2815 | +# limitations under the License. |
2816 | |
2817 | import six |
2818 | +import re |
2819 | |
2820 | |
2821 | def bool_from_string(value): |
2822 | @@ -40,3 +39,32 @@ |
2823 | |
2824 | msg = "Unable to interpret string value '%s' as boolean" % (value) |
2825 | raise ValueError(msg) |
2826 | + |
2827 | + |
2828 | +def bytes_from_string(value): |
2829 | + """Interpret human readable string value as bytes. |
2830 | + |
2831 | + Returns int |
2832 | + """ |
2833 | + BYTE_POWER = { |
2834 | + 'K': 1, |
2835 | + 'KB': 1, |
2836 | + 'M': 2, |
2837 | + 'MB': 2, |
2838 | + 'G': 3, |
2839 | + 'GB': 3, |
2840 | + 'T': 4, |
2841 | + 'TB': 4, |
2842 | + 'P': 5, |
2843 | + 'PB': 5, |
2844 | + } |
2845 | + if isinstance(value, six.string_types): |
2846 | + value = six.text_type(value) |
2847 | + else: |
2848 | + msg = "Unable to interpret non-string value '%s' as boolean" % (value) |
2849 | + raise ValueError(msg) |
2850 | + matches = re.match("([0-9]+)([a-zA-Z]+)", value) |
2851 | + if not matches: |
2852 | + msg = "Unable to interpret string value '%s' as bytes" % (value) |
2853 | + raise ValueError(msg) |
2854 | + return int(matches.group(1)) * (1024 ** BYTE_POWER[matches.group(2)]) |
2855 | |
2856 | === modified file 'hooks/charmhelpers/core/sysctl.py' |
2857 | --- hooks/charmhelpers/core/sysctl.py 2015-05-14 10:48:09 +0000 |
2858 | +++ hooks/charmhelpers/core/sysctl.py 2018-06-12 20:07:13 +0000 |
2859 | @@ -3,19 +3,17 @@ |
2860 | |
2861 | # Copyright 2014-2015 Canonical Limited. |
2862 | # |
2863 | -# This file is part of charm-helpers. |
2864 | -# |
2865 | -# charm-helpers is free software: you can redistribute it and/or modify |
2866 | -# it under the terms of the GNU Lesser General Public License version 3 as |
2867 | -# published by the Free Software Foundation. |
2868 | -# |
2869 | -# charm-helpers is distributed in the hope that it will be useful, |
2870 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
2871 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
2872 | -# GNU Lesser General Public License for more details. |
2873 | -# |
2874 | -# You should have received a copy of the GNU Lesser General Public License |
2875 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
2876 | +# Licensed under the Apache License, Version 2.0 (the "License"); |
2877 | +# you may not use this file except in compliance with the License. |
2878 | +# You may obtain a copy of the License at |
2879 | +# |
2880 | +# http://www.apache.org/licenses/LICENSE-2.0 |
2881 | +# |
2882 | +# Unless required by applicable law or agreed to in writing, software |
2883 | +# distributed under the License is distributed on an "AS IS" BASIS, |
2884 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
2885 | +# See the License for the specific language governing permissions and |
2886 | +# limitations under the License. |
2887 | |
2888 | import yaml |
2889 | |
2890 | |
2891 | === modified file 'hooks/charmhelpers/core/templating.py' |
2892 | --- hooks/charmhelpers/core/templating.py 2015-02-09 12:58:07 +0000 |
2893 | +++ hooks/charmhelpers/core/templating.py 2018-06-12 20:07:13 +0000 |
2894 | @@ -1,33 +1,33 @@ |
2895 | # Copyright 2014-2015 Canonical Limited. |
2896 | # |
2897 | -# This file is part of charm-helpers. |
2898 | -# |
2899 | -# charm-helpers is free software: you can redistribute it and/or modify |
2900 | -# it under the terms of the GNU Lesser General Public License version 3 as |
2901 | -# published by the Free Software Foundation. |
2902 | -# |
2903 | -# charm-helpers is distributed in the hope that it will be useful, |
2904 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
2905 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
2906 | -# GNU Lesser General Public License for more details. |
2907 | -# |
2908 | -# You should have received a copy of the GNU Lesser General Public License |
2909 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
2910 | +# Licensed under the Apache License, Version 2.0 (the "License"); |
2911 | +# you may not use this file except in compliance with the License. |
2912 | +# You may obtain a copy of the License at |
2913 | +# |
2914 | +# http://www.apache.org/licenses/LICENSE-2.0 |
2915 | +# |
2916 | +# Unless required by applicable law or agreed to in writing, software |
2917 | +# distributed under the License is distributed on an "AS IS" BASIS, |
2918 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
2919 | +# See the License for the specific language governing permissions and |
2920 | +# limitations under the License. |
2921 | |
2922 | import os |
2923 | +import sys |
2924 | |
2925 | from charmhelpers.core import host |
2926 | from charmhelpers.core import hookenv |
2927 | |
2928 | |
2929 | def render(source, target, context, owner='root', group='root', |
2930 | - perms=0o444, templates_dir=None, encoding='UTF-8'): |
2931 | + perms=0o444, templates_dir=None, encoding='UTF-8', template_loader=None): |
2932 | """ |
2933 | Render a template. |
2934 | |
2935 | The `source` path, if not absolute, is relative to the `templates_dir`. |
2936 | |
2937 | - The `target` path should be absolute. |
2938 | + The `target` path should be absolute. It can also be `None`, in which |
2939 | + case no file will be written. |
2940 | |
2941 | The context should be a dict containing the values to be replaced in the |
2942 | template. |
2943 | @@ -36,8 +36,12 @@ |
2944 | |
2945 | If omitted, `templates_dir` defaults to the `templates` folder in the charm. |
2946 | |
2947 | - Note: Using this requires python-jinja2; if it is not installed, calling |
2948 | - this will attempt to use charmhelpers.fetch.apt_install to install it. |
2949 | + The rendered template will be written to the file as well as being returned |
2950 | + as a string. |
2951 | + |
2952 | + Note: Using this requires python-jinja2 or python3-jinja2; if it is not |
2953 | + installed, calling this will attempt to use charmhelpers.fetch.apt_install |
2954 | + to install it. |
2955 | """ |
2956 | try: |
2957 | from jinja2 import FileSystemLoader, Environment, exceptions |
2958 | @@ -49,20 +53,32 @@ |
2959 | 'charmhelpers.fetch to install it', |
2960 | level=hookenv.ERROR) |
2961 | raise |
2962 | - apt_install('python-jinja2', fatal=True) |
2963 | + if sys.version_info.major == 2: |
2964 | + apt_install('python-jinja2', fatal=True) |
2965 | + else: |
2966 | + apt_install('python3-jinja2', fatal=True) |
2967 | from jinja2 import FileSystemLoader, Environment, exceptions |
2968 | |
2969 | - if templates_dir is None: |
2970 | - templates_dir = os.path.join(hookenv.charm_dir(), 'templates') |
2971 | - loader = Environment(loader=FileSystemLoader(templates_dir)) |
2972 | + if template_loader: |
2973 | + template_env = Environment(loader=template_loader) |
2974 | + else: |
2975 | + if templates_dir is None: |
2976 | + templates_dir = os.path.join(hookenv.charm_dir(), 'templates') |
2977 | + template_env = Environment(loader=FileSystemLoader(templates_dir)) |
2978 | try: |
2979 | source = source |
2980 | - template = loader.get_template(source) |
2981 | + template = template_env.get_template(source) |
2982 | except exceptions.TemplateNotFound as e: |
2983 | hookenv.log('Could not load template %s from %s.' % |
2984 | (source, templates_dir), |
2985 | level=hookenv.ERROR) |
2986 | raise e |
2987 | content = template.render(context) |
2988 | - host.mkdir(os.path.dirname(target), owner, group, perms=0o755) |
2989 | - host.write_file(target, content.encode(encoding), owner, group, perms) |
2990 | + if target is not None: |
2991 | + target_dir = os.path.dirname(target) |
2992 | + if not os.path.exists(target_dir): |
2993 | + # This is a terrible default directory permission, as the file |
2994 | + # or its siblings will often contain secrets. |
2995 | + host.mkdir(os.path.dirname(target), owner, group, perms=0o755) |
2996 | + host.write_file(target, content.encode(encoding), owner, group, perms) |
2997 | + return content |
2998 | |
2999 | === modified file 'hooks/charmhelpers/core/unitdata.py' |
3000 | --- hooks/charmhelpers/core/unitdata.py 2015-05-14 10:48:09 +0000 |
3001 | +++ hooks/charmhelpers/core/unitdata.py 2018-06-12 20:07:13 +0000 |
3002 | @@ -3,20 +3,17 @@ |
3003 | # |
3004 | # Copyright 2014-2015 Canonical Limited. |
3005 | # |
3006 | -# This file is part of charm-helpers. |
3007 | -# |
3008 | -# charm-helpers is free software: you can redistribute it and/or modify |
3009 | -# it under the terms of the GNU Lesser General Public License version 3 as |
3010 | -# published by the Free Software Foundation. |
3011 | -# |
3012 | -# charm-helpers is distributed in the hope that it will be useful, |
3013 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
3014 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
3015 | -# GNU Lesser General Public License for more details. |
3016 | -# |
3017 | -# You should have received a copy of the GNU Lesser General Public License |
3018 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
3019 | -# |
3020 | +# Licensed under the Apache License, Version 2.0 (the "License"); |
3021 | +# you may not use this file except in compliance with the License. |
3022 | +# You may obtain a copy of the License at |
3023 | +# |
3024 | +# http://www.apache.org/licenses/LICENSE-2.0 |
3025 | +# |
3026 | +# Unless required by applicable law or agreed to in writing, software |
3027 | +# distributed under the License is distributed on an "AS IS" BASIS, |
3028 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
3029 | +# See the License for the specific language governing permissions and |
3030 | +# limitations under the License. |
3031 | # |
3032 | # Authors: |
3033 | # Kapil Thangavelu <kapil.foss@gmail.com> |
3034 | @@ -152,6 +149,7 @@ |
3035 | import collections |
3036 | import contextlib |
3037 | import datetime |
3038 | +import itertools |
3039 | import json |
3040 | import os |
3041 | import pprint |
3042 | @@ -164,8 +162,7 @@ |
3043 | class Storage(object): |
3044 | """Simple key value database for local unit state within charms. |
3045 | |
3046 | - Modifications are automatically committed at hook exit. That's |
3047 | - currently regardless of exit code. |
3048 | + Modifications are not persisted unless :meth:`flush` is called. |
3049 | |
3050 | To support dicts, lists, integer, floats, and booleans values |
3051 | are automatically json encoded/decoded. |
3052 | @@ -173,8 +170,11 @@ |
3053 | def __init__(self, path=None): |
3054 | self.db_path = path |
3055 | if path is None: |
3056 | - self.db_path = os.path.join( |
3057 | - os.environ.get('CHARM_DIR', ''), '.unit-state.db') |
3058 | + if 'UNIT_STATE_DB' in os.environ: |
3059 | + self.db_path = os.environ['UNIT_STATE_DB'] |
3060 | + else: |
3061 | + self.db_path = os.path.join( |
3062 | + os.environ.get('CHARM_DIR', ''), '.unit-state.db') |
3063 | self.conn = sqlite3.connect('%s' % self.db_path) |
3064 | self.cursor = self.conn.cursor() |
3065 | self.revision = None |
3066 | @@ -189,15 +189,8 @@ |
3067 | self.conn.close() |
3068 | self._closed = True |
3069 | |
3070 | - def _scoped_query(self, stmt, params=None): |
3071 | - if params is None: |
3072 | - params = [] |
3073 | - return stmt, params |
3074 | - |
3075 | def get(self, key, default=None, record=False): |
3076 | - self.cursor.execute( |
3077 | - *self._scoped_query( |
3078 | - 'select data from kv where key=?', [key])) |
3079 | + self.cursor.execute('select data from kv where key=?', [key]) |
3080 | result = self.cursor.fetchone() |
3081 | if not result: |
3082 | return default |
3083 | @@ -206,33 +199,81 @@ |
3084 | return json.loads(result[0]) |
3085 | |
3086 | def getrange(self, key_prefix, strip=False): |
3087 | - stmt = "select key, data from kv where key like '%s%%'" % key_prefix |
3088 | - self.cursor.execute(*self._scoped_query(stmt)) |
3089 | + """ |
3090 | + Get a range of keys starting with a common prefix as a mapping of |
3091 | + keys to values. |
3092 | + |
3093 | + :param str key_prefix: Common prefix among all keys |
3094 | + :param bool strip: Optionally strip the common prefix from the key |
3095 | + names in the returned dict |
3096 | + :return dict: A (possibly empty) dict of key-value mappings |
3097 | + """ |
3098 | + self.cursor.execute("select key, data from kv where key like ?", |
3099 | + ['%s%%' % key_prefix]) |
3100 | result = self.cursor.fetchall() |
3101 | |
3102 | if not result: |
3103 | - return None |
3104 | + return {} |
3105 | if not strip: |
3106 | key_prefix = '' |
3107 | return dict([ |
3108 | (k[len(key_prefix):], json.loads(v)) for k, v in result]) |
3109 | |
3110 | def update(self, mapping, prefix=""): |
3111 | + """ |
3112 | + Set the values of multiple keys at once. |
3113 | + |
3114 | + :param dict mapping: Mapping of keys to values |
3115 | + :param str prefix: Optional prefix to apply to all keys in `mapping` |
3116 | + before setting |
3117 | + """ |
3118 | for k, v in mapping.items(): |
3119 | self.set("%s%s" % (prefix, k), v) |
3120 | |
3121 | def unset(self, key): |
3122 | + """ |
3123 | + Remove a key from the database entirely. |
3124 | + """ |
3125 | self.cursor.execute('delete from kv where key=?', [key]) |
3126 | if self.revision and self.cursor.rowcount: |
3127 | self.cursor.execute( |
3128 | 'insert into kv_revisions values (?, ?, ?)', |
3129 | [key, self.revision, json.dumps('DELETED')]) |
3130 | |
3131 | + def unsetrange(self, keys=None, prefix=""): |
3132 | + """ |
3133 | + Remove a range of keys starting with a common prefix, from the database |
3134 | + entirely. |
3135 | + |
3136 | + :param list keys: List of keys to remove. |
3137 | + :param str prefix: Optional prefix to apply to all keys in ``keys`` |
3138 | + before removing. |
3139 | + """ |
3140 | + if keys is not None: |
3141 | + keys = ['%s%s' % (prefix, key) for key in keys] |
3142 | + self.cursor.execute('delete from kv where key in (%s)' % ','.join(['?'] * len(keys)), keys) |
3143 | + if self.revision and self.cursor.rowcount: |
3144 | + self.cursor.execute( |
3145 | + 'insert into kv_revisions values %s' % ','.join(['(?, ?, ?)'] * len(keys)), |
3146 | + list(itertools.chain.from_iterable((key, self.revision, json.dumps('DELETED')) for key in keys))) |
3147 | + else: |
3148 | + self.cursor.execute('delete from kv where key like ?', |
3149 | + ['%s%%' % prefix]) |
3150 | + if self.revision and self.cursor.rowcount: |
3151 | + self.cursor.execute( |
3152 | + 'insert into kv_revisions values (?, ?, ?)', |
3153 | + ['%s%%' % prefix, self.revision, json.dumps('DELETED')]) |
3154 | + |
3155 | def set(self, key, value): |
3156 | + """ |
3157 | + Set a value in the database. |
3158 | + |
3159 | + :param str key: Key to set the value for |
3160 | + :param value: Any JSON-serializable value to be set |
3161 | + """ |
3162 | serialized = json.dumps(value) |
3163 | |
3164 | - self.cursor.execute( |
3165 | - 'select data from kv where key=?', [key]) |
3166 | + self.cursor.execute('select data from kv where key=?', [key]) |
3167 | exists = self.cursor.fetchone() |
3168 | |
3169 | # Skip mutations to the same value |
3170 | |
3171 | === modified file 'hooks/charmhelpers/fetch/__init__.py' |
3172 | --- hooks/charmhelpers/fetch/__init__.py 2016-02-12 18:30:48 +0000 |
3173 | +++ hooks/charmhelpers/fetch/__init__.py 2018-06-12 20:07:13 +0000 |
3174 | @@ -1,32 +1,24 @@ |
3175 | # Copyright 2014-2015 Canonical Limited. |
3176 | # |
3177 | -# This file is part of charm-helpers. |
3178 | -# |
3179 | -# charm-helpers is free software: you can redistribute it and/or modify |
3180 | -# it under the terms of the GNU Lesser General Public License version 3 as |
3181 | -# published by the Free Software Foundation. |
3182 | -# |
3183 | -# charm-helpers is distributed in the hope that it will be useful, |
3184 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
3185 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
3186 | -# GNU Lesser General Public License for more details. |
3187 | -# |
3188 | -# You should have received a copy of the GNU Lesser General Public License |
3189 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
3190 | +# Licensed under the Apache License, Version 2.0 (the "License"); |
3191 | +# you may not use this file except in compliance with the License. |
3192 | +# You may obtain a copy of the License at |
3193 | +# |
3194 | +# http://www.apache.org/licenses/LICENSE-2.0 |
3195 | +# |
3196 | +# Unless required by applicable law or agreed to in writing, software |
3197 | +# distributed under the License is distributed on an "AS IS" BASIS, |
3198 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
3199 | +# See the License for the specific language governing permissions and |
3200 | +# limitations under the License. |
3201 | |
3202 | import importlib |
3203 | -from tempfile import NamedTemporaryFile |
3204 | -import time |
3205 | +from charmhelpers.osplatform import get_platform |
3206 | from yaml import safe_load |
3207 | -from charmhelpers.core.host import ( |
3208 | - lsb_release |
3209 | -) |
3210 | -import subprocess |
3211 | from charmhelpers.core.hookenv import ( |
3212 | config, |
3213 | log, |
3214 | ) |
3215 | -import os |
3216 | |
3217 | import six |
3218 | if six.PY3: |
3219 | @@ -35,71 +27,6 @@ |
3220 | from urlparse import urlparse, urlunparse |
3221 | |
3222 | |
3223 | -CLOUD_ARCHIVE = """# Ubuntu Cloud Archive |
3224 | -deb http://ubuntu-cloud.archive.canonical.com/ubuntu {} main |
3225 | -""" |
3226 | -PROPOSED_POCKET = """# Proposed |
3227 | -deb http://archive.ubuntu.com/ubuntu {}-proposed main universe multiverse restricted |
3228 | -""" |
3229 | -CLOUD_ARCHIVE_POCKETS = { |
3230 | - # Folsom |
3231 | - 'folsom': 'precise-updates/folsom', |
3232 | - 'precise-folsom': 'precise-updates/folsom', |
3233 | - 'precise-folsom/updates': 'precise-updates/folsom', |
3234 | - 'precise-updates/folsom': 'precise-updates/folsom', |
3235 | - 'folsom/proposed': 'precise-proposed/folsom', |
3236 | - 'precise-folsom/proposed': 'precise-proposed/folsom', |
3237 | - 'precise-proposed/folsom': 'precise-proposed/folsom', |
3238 | - # Grizzly |
3239 | - 'grizzly': 'precise-updates/grizzly', |
3240 | - 'precise-grizzly': 'precise-updates/grizzly', |
3241 | - 'precise-grizzly/updates': 'precise-updates/grizzly', |
3242 | - 'precise-updates/grizzly': 'precise-updates/grizzly', |
3243 | - 'grizzly/proposed': 'precise-proposed/grizzly', |
3244 | - 'precise-grizzly/proposed': 'precise-proposed/grizzly', |
3245 | - 'precise-proposed/grizzly': 'precise-proposed/grizzly', |
3246 | - # Havana |
3247 | - 'havana': 'precise-updates/havana', |
3248 | - 'precise-havana': 'precise-updates/havana', |
3249 | - 'precise-havana/updates': 'precise-updates/havana', |
3250 | - 'precise-updates/havana': 'precise-updates/havana', |
3251 | - 'havana/proposed': 'precise-proposed/havana', |
3252 | - 'precise-havana/proposed': 'precise-proposed/havana', |
3253 | - 'precise-proposed/havana': 'precise-proposed/havana', |
3254 | - # Icehouse |
3255 | - 'icehouse': 'precise-updates/icehouse', |
3256 | - 'precise-icehouse': 'precise-updates/icehouse', |
3257 | - 'precise-icehouse/updates': 'precise-updates/icehouse', |
3258 | - 'precise-updates/icehouse': 'precise-updates/icehouse', |
3259 | - 'icehouse/proposed': 'precise-proposed/icehouse', |
3260 | - 'precise-icehouse/proposed': 'precise-proposed/icehouse', |
3261 | - 'precise-proposed/icehouse': 'precise-proposed/icehouse', |
3262 | - # Juno |
3263 | - 'juno': 'trusty-updates/juno', |
3264 | - 'trusty-juno': 'trusty-updates/juno', |
3265 | - 'trusty-juno/updates': 'trusty-updates/juno', |
3266 | - 'trusty-updates/juno': 'trusty-updates/juno', |
3267 | - 'juno/proposed': 'trusty-proposed/juno', |
3268 | - 'trusty-juno/proposed': 'trusty-proposed/juno', |
3269 | - 'trusty-proposed/juno': 'trusty-proposed/juno', |
3270 | - # Kilo |
3271 | - 'kilo': 'trusty-updates/kilo', |
3272 | - 'trusty-kilo': 'trusty-updates/kilo', |
3273 | - 'trusty-kilo/updates': 'trusty-updates/kilo', |
3274 | - 'trusty-updates/kilo': 'trusty-updates/kilo', |
3275 | - 'kilo/proposed': 'trusty-proposed/kilo', |
3276 | - 'trusty-kilo/proposed': 'trusty-proposed/kilo', |
3277 | - 'trusty-proposed/kilo': 'trusty-proposed/kilo', |
3278 | - # Liberty |
3279 | - 'liberty': 'trusty-updates/liberty', |
3280 | - 'trusty-liberty': 'trusty-updates/liberty', |
3281 | - 'trusty-liberty/updates': 'trusty-updates/liberty', |
3282 | - 'trusty-updates/liberty': 'trusty-updates/liberty', |
3283 | - 'liberty/proposed': 'trusty-proposed/liberty', |
3284 | - 'trusty-liberty/proposed': 'trusty-proposed/liberty', |
3285 | - 'trusty-proposed/liberty': 'trusty-proposed/liberty', |
3286 | -} |
3287 | - |
3288 | # The order of this list is very important. Handlers should be listed in from |
3289 | # least- to most-specific URL matching. |
3290 | FETCH_HANDLERS = ( |
3291 | @@ -108,10 +35,6 @@ |
3292 | 'charmhelpers.fetch.giturl.GitUrlFetchHandler', |
3293 | ) |
3294 | |
3295 | -APT_NO_LOCK = 100 # The return code for "couldn't acquire lock" in APT. |
3296 | -APT_NO_LOCK_RETRY_DELAY = 10 # Wait 10 seconds between apt lock checks. |
3297 | -APT_NO_LOCK_RETRY_COUNT = 30 # Retry to acquire the lock X times. |
3298 | - |
3299 | |
3300 | class SourceConfigError(Exception): |
3301 | pass |
3302 | @@ -149,172 +72,38 @@ |
3303 | return urlunparse(parts) |
3304 | |
3305 | |
3306 | -def filter_installed_packages(packages): |
3307 | - """Returns a list of packages that require installation""" |
3308 | - cache = apt_cache() |
3309 | - _pkgs = [] |
3310 | - for package in packages: |
3311 | - try: |
3312 | - p = cache[package] |
3313 | - p.current_ver or _pkgs.append(package) |
3314 | - except KeyError: |
3315 | - log('Package {} has no installation candidate.'.format(package), |
3316 | - level='WARNING') |
3317 | - _pkgs.append(package) |
3318 | - return _pkgs |
3319 | - |
3320 | - |
3321 | -def apt_cache(in_memory=True): |
3322 | - """Build and return an apt cache""" |
3323 | - from apt import apt_pkg |
3324 | - apt_pkg.init() |
3325 | - if in_memory: |
3326 | - apt_pkg.config.set("Dir::Cache::pkgcache", "") |
3327 | - apt_pkg.config.set("Dir::Cache::srcpkgcache", "") |
3328 | - return apt_pkg.Cache() |
3329 | - |
3330 | - |
3331 | -def apt_install(packages, options=None, fatal=False): |
3332 | - """Install one or more packages""" |
3333 | - if options is None: |
3334 | - options = ['--option=Dpkg::Options::=--force-confold'] |
3335 | - |
3336 | - cmd = ['apt-get', '--assume-yes'] |
3337 | - cmd.extend(options) |
3338 | - cmd.append('install') |
3339 | - if isinstance(packages, six.string_types): |
3340 | - cmd.append(packages) |
3341 | - else: |
3342 | - cmd.extend(packages) |
3343 | - log("Installing {} with options: {}".format(packages, |
3344 | - options)) |
3345 | - _run_apt_command(cmd, fatal) |
3346 | - |
3347 | - |
3348 | -def apt_upgrade(options=None, fatal=False, dist=False): |
3349 | - """Upgrade all packages""" |
3350 | - if options is None: |
3351 | - options = ['--option=Dpkg::Options::=--force-confold'] |
3352 | - |
3353 | - cmd = ['apt-get', '--assume-yes'] |
3354 | - cmd.extend(options) |
3355 | - if dist: |
3356 | - cmd.append('dist-upgrade') |
3357 | - else: |
3358 | - cmd.append('upgrade') |
3359 | - log("Upgrading with options: {}".format(options)) |
3360 | - _run_apt_command(cmd, fatal) |
3361 | - |
3362 | - |
3363 | -def apt_update(fatal=False): |
3364 | - """Update local apt cache""" |
3365 | - cmd = ['apt-get', 'update'] |
3366 | - _run_apt_command(cmd, fatal) |
3367 | - |
3368 | - |
3369 | -def apt_purge(packages, fatal=False): |
3370 | - """Purge one or more packages""" |
3371 | - cmd = ['apt-get', '--assume-yes', 'purge'] |
3372 | - if isinstance(packages, six.string_types): |
3373 | - cmd.append(packages) |
3374 | - else: |
3375 | - cmd.extend(packages) |
3376 | - log("Purging {}".format(packages)) |
3377 | - _run_apt_command(cmd, fatal) |
3378 | - |
3379 | - |
3380 | -def apt_hold(packages, fatal=False): |
3381 | - """Hold one or more packages""" |
3382 | - cmd = ['apt-mark', 'hold'] |
3383 | - if isinstance(packages, six.string_types): |
3384 | - cmd.append(packages) |
3385 | - else: |
3386 | - cmd.extend(packages) |
3387 | - log("Holding {}".format(packages)) |
3388 | - |
3389 | - if fatal: |
3390 | - subprocess.check_call(cmd) |
3391 | - else: |
3392 | - subprocess.call(cmd) |
3393 | - |
3394 | - |
3395 | -def add_source(source, key=None): |
3396 | - """Add a package source to this system. |
3397 | - |
3398 | - @param source: a URL or sources.list entry, as supported by |
3399 | - add-apt-repository(1). Examples:: |
3400 | - |
3401 | - ppa:charmers/example |
3402 | - deb https://stub:key@private.example.com/ubuntu trusty main |
3403 | - |
3404 | - In addition: |
3405 | - 'proposed:' may be used to enable the standard 'proposed' |
3406 | - pocket for the release. |
3407 | - 'cloud:' may be used to activate official cloud archive pockets, |
3408 | - such as 'cloud:icehouse' |
3409 | - 'distro' may be used as a noop |
3410 | - |
3411 | - @param key: A key to be added to the system's APT keyring and used |
3412 | - to verify the signatures on packages. Ideally, this should be an |
3413 | - ASCII format GPG public key including the block headers. A GPG key |
3414 | - id may also be used, but be aware that only insecure protocols are |
3415 | - available to retrieve the actual public key from a public keyserver |
3416 | - placing your Juju environment at risk. ppa and cloud archive keys |
3417 | - are securely added automtically, so sould not be provided. |
3418 | - """ |
3419 | - if source is None: |
3420 | - log('Source is not present. Skipping') |
3421 | - return |
3422 | - |
3423 | - if (source.startswith('ppa:') or |
3424 | - source.startswith('http') or |
3425 | - source.startswith('deb ') or |
3426 | - source.startswith('cloud-archive:')): |
3427 | - subprocess.check_call(['add-apt-repository', '--yes', source]) |
3428 | - elif source.startswith('cloud:'): |
3429 | - apt_install(filter_installed_packages(['ubuntu-cloud-keyring']), |
3430 | - fatal=True) |
3431 | - pocket = source.split(':')[-1] |
3432 | - if pocket not in CLOUD_ARCHIVE_POCKETS: |
3433 | - raise SourceConfigError( |
3434 | - 'Unsupported cloud: source option %s' % |
3435 | - pocket) |
3436 | - actual_pocket = CLOUD_ARCHIVE_POCKETS[pocket] |
3437 | - with open('/etc/apt/sources.list.d/cloud-archive.list', 'w') as apt: |
3438 | - apt.write(CLOUD_ARCHIVE.format(actual_pocket)) |
3439 | - elif source == 'proposed': |
3440 | - release = lsb_release()['DISTRIB_CODENAME'] |
3441 | - with open('/etc/apt/sources.list.d/proposed.list', 'w') as apt: |
3442 | - apt.write(PROPOSED_POCKET.format(release)) |
3443 | - elif source == 'distro': |
3444 | - pass |
3445 | - else: |
3446 | - log("Unknown source: {!r}".format(source)) |
3447 | - |
3448 | - if key: |
3449 | - if '-----BEGIN PGP PUBLIC KEY BLOCK-----' in key: |
3450 | - with NamedTemporaryFile('w+') as key_file: |
3451 | - key_file.write(key) |
3452 | - key_file.flush() |
3453 | - key_file.seek(0) |
3454 | - subprocess.check_call(['apt-key', 'add', '-'], stdin=key_file) |
3455 | - else: |
3456 | - # Note that hkp: is in no way a secure protocol. Using a |
3457 | - # GPG key id is pointless from a security POV unless you |
3458 | - # absolutely trust your network and DNS. |
3459 | - subprocess.check_call(['apt-key', 'adv', '--keyserver', |
3460 | - 'hkp://keyserver.ubuntu.com:80', '--recv', |
3461 | - key]) |
3462 | +__platform__ = get_platform() |
3463 | +module = "charmhelpers.fetch.%s" % __platform__ |
3464 | +fetch = importlib.import_module(module) |
3465 | + |
3466 | +filter_installed_packages = fetch.filter_installed_packages |
3467 | +install = fetch.install |
3468 | +upgrade = fetch.upgrade |
3469 | +update = fetch.update |
3470 | +purge = fetch.purge |
3471 | +add_source = fetch.add_source |
3472 | + |
3473 | +if __platform__ == "ubuntu": |
3474 | + apt_cache = fetch.apt_cache |
3475 | + apt_install = fetch.install |
3476 | + apt_update = fetch.update |
3477 | + apt_upgrade = fetch.upgrade |
3478 | + apt_purge = fetch.purge |
3479 | + apt_mark = fetch.apt_mark |
3480 | + apt_hold = fetch.apt_hold |
3481 | + apt_unhold = fetch.apt_unhold |
3482 | + get_upstream_version = fetch.get_upstream_version |
3483 | +elif __platform__ == "centos": |
3484 | + yum_search = fetch.yum_search |
3485 | |
3486 | |
3487 | def configure_sources(update=False, |
3488 | sources_var='install_sources', |
3489 | keys_var='install_keys'): |
3490 | - """ |
3491 | - Configure multiple sources from charm configuration. |
3492 | + """Configure multiple sources from charm configuration. |
3493 | |
3494 | The lists are encoded as yaml fragments in the configuration. |
3495 | - The frament needs to be included as a string. Sources and their |
3496 | + The fragment needs to be included as a string. Sources and their |
3497 | corresponding keys are of the types supported by add_source(). |
3498 | |
3499 | Example config: |
3500 | @@ -346,12 +135,11 @@ |
3501 | for source, key in zip(sources, keys): |
3502 | add_source(source, key) |
3503 | if update: |
3504 | - apt_update(fatal=True) |
3505 | + fetch.update(fatal=True) |
3506 | |
3507 | |
3508 | def install_remote(source, *args, **kwargs): |
3509 | - """ |
3510 | - Install a file tree from a remote source |
3511 | + """Install a file tree from a remote source. |
3512 | |
3513 | The specified source should be a url of the form: |
3514 | scheme://[host]/path[#[option=value][&...]] |
3515 | @@ -374,18 +162,17 @@ |
3516 | # We ONLY check for True here because can_handle may return a string |
3517 | # explaining why it can't handle a given source. |
3518 | handlers = [h for h in plugins() if h.can_handle(source) is True] |
3519 | - installed_to = None |
3520 | for handler in handlers: |
3521 | try: |
3522 | - installed_to = handler.install(source, *args, **kwargs) |
3523 | - except UnhandledSource: |
3524 | - pass |
3525 | - if not installed_to: |
3526 | - raise UnhandledSource("No handler found for source {}".format(source)) |
3527 | - return installed_to |
3528 | + return handler.install(source, *args, **kwargs) |
3529 | + except UnhandledSource as e: |
3530 | + log('Install source attempt unsuccessful: {}'.format(e), |
3531 | + level='WARNING') |
3532 | + raise UnhandledSource("No handler found for source {}".format(source)) |
3533 | |
3534 | |
3535 | def install_from_config(config_var_name): |
3536 | + """Install a file from config.""" |
3537 | charm_config = config() |
3538 | source = charm_config[config_var_name] |
3539 | return install_remote(source) |
3540 | @@ -402,46 +189,9 @@ |
3541 | importlib.import_module(package), |
3542 | classname) |
3543 | plugin_list.append(handler_class()) |
3544 | - except (ImportError, AttributeError): |
3545 | + except NotImplementedError: |
3546 | # Skip missing plugins so that they can be ommitted from |
3547 | # installation if desired |
3548 | log("FetchHandler {} not found, skipping plugin".format( |
3549 | handler_name)) |
3550 | return plugin_list |
3551 | - |
3552 | - |
3553 | -def _run_apt_command(cmd, fatal=False): |
3554 | - """ |
3555 | - Run an APT command, checking output and retrying if the fatal flag is set |
3556 | - to True. |
3557 | - |
3558 | - :param: cmd: str: The apt command to run. |
3559 | - :param: fatal: bool: Whether the command's output should be checked and |
3560 | - retried. |
3561 | - """ |
3562 | - env = os.environ.copy() |
3563 | - |
3564 | - if 'DEBIAN_FRONTEND' not in env: |
3565 | - env['DEBIAN_FRONTEND'] = 'noninteractive' |
3566 | - |
3567 | - if fatal: |
3568 | - retry_count = 0 |
3569 | - result = None |
3570 | - |
3571 | - # If the command is considered "fatal", we need to retry if the apt |
3572 | - # lock was not acquired. |
3573 | - |
3574 | - while result is None or result == APT_NO_LOCK: |
3575 | - try: |
3576 | - result = subprocess.check_call(cmd, env=env) |
3577 | - except subprocess.CalledProcessError as e: |
3578 | - retry_count = retry_count + 1 |
3579 | - if retry_count > APT_NO_LOCK_RETRY_COUNT: |
3580 | - raise |
3581 | - result = e.returncode |
3582 | - log("Couldn't acquire DPKG lock. Will retry in {} seconds." |
3583 | - "".format(APT_NO_LOCK_RETRY_DELAY)) |
3584 | - time.sleep(APT_NO_LOCK_RETRY_DELAY) |
3585 | - |
3586 | - else: |
3587 | - subprocess.call(cmd, env=env) |
3588 | |
3589 | === modified file 'hooks/charmhelpers/fetch/archiveurl.py' |
3590 | --- hooks/charmhelpers/fetch/archiveurl.py 2015-05-14 10:48:09 +0000 |
3591 | +++ hooks/charmhelpers/fetch/archiveurl.py 2018-06-12 20:07:13 +0000 |
3592 | @@ -1,18 +1,16 @@ |
3593 | # Copyright 2014-2015 Canonical Limited. |
3594 | # |
3595 | -# This file is part of charm-helpers. |
3596 | -# |
3597 | -# charm-helpers is free software: you can redistribute it and/or modify |
3598 | -# it under the terms of the GNU Lesser General Public License version 3 as |
3599 | -# published by the Free Software Foundation. |
3600 | -# |
3601 | -# charm-helpers is distributed in the hope that it will be useful, |
3602 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
3603 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
3604 | -# GNU Lesser General Public License for more details. |
3605 | -# |
3606 | -# You should have received a copy of the GNU Lesser General Public License |
3607 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
3608 | +# Licensed under the Apache License, Version 2.0 (the "License"); |
3609 | +# you may not use this file except in compliance with the License. |
3610 | +# You may obtain a copy of the License at |
3611 | +# |
3612 | +# http://www.apache.org/licenses/LICENSE-2.0 |
3613 | +# |
3614 | +# Unless required by applicable law or agreed to in writing, software |
3615 | +# distributed under the License is distributed on an "AS IS" BASIS, |
3616 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
3617 | +# See the License for the specific language governing permissions and |
3618 | +# limitations under the License. |
3619 | |
3620 | import os |
3621 | import hashlib |
3622 | @@ -77,6 +75,8 @@ |
3623 | def can_handle(self, source): |
3624 | url_parts = self.parse_url(source) |
3625 | if url_parts.scheme not in ('http', 'https', 'ftp', 'file'): |
3626 | + # XXX: Why is this returning a boolean and a string? It's |
3627 | + # doomed to fail since "bool(can_handle('foo://'))" will be True. |
3628 | return "Wrong source type" |
3629 | if get_archive_handler(self.base_url(source)): |
3630 | return True |
3631 | @@ -106,7 +106,7 @@ |
3632 | install_opener(opener) |
3633 | response = urlopen(source) |
3634 | try: |
3635 | - with open(dest, 'w') as dest_file: |
3636 | + with open(dest, 'wb') as dest_file: |
3637 | dest_file.write(response.read()) |
3638 | except Exception as e: |
3639 | if os.path.isfile(dest): |
3640 | @@ -155,7 +155,11 @@ |
3641 | else: |
3642 | algorithms = hashlib.algorithms_available |
3643 | if key in algorithms: |
3644 | - check_hash(dld_file, value, key) |
3645 | + if len(value) != 1: |
3646 | + raise TypeError( |
3647 | + "Expected 1 hash value, not %d" % len(value)) |
3648 | + expected = value[0] |
3649 | + check_hash(dld_file, expected, key) |
3650 | if checksum: |
3651 | check_hash(dld_file, checksum, hash_type) |
3652 | return extract(dld_file, dest) |
3653 | |
3654 | === modified file 'hooks/charmhelpers/fetch/bzrurl.py' |
3655 | --- hooks/charmhelpers/fetch/bzrurl.py 2015-02-09 12:53:57 +0000 |
3656 | +++ hooks/charmhelpers/fetch/bzrurl.py 2018-06-12 20:07:13 +0000 |
3657 | @@ -1,78 +1,76 @@ |
3658 | # Copyright 2014-2015 Canonical Limited. |
3659 | # |
3660 | -# This file is part of charm-helpers. |
3661 | -# |
3662 | -# charm-helpers is free software: you can redistribute it and/or modify |
3663 | -# it under the terms of the GNU Lesser General Public License version 3 as |
3664 | -# published by the Free Software Foundation. |
3665 | -# |
3666 | -# charm-helpers is distributed in the hope that it will be useful, |
3667 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
3668 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
3669 | -# GNU Lesser General Public License for more details. |
3670 | -# |
3671 | -# You should have received a copy of the GNU Lesser General Public License |
3672 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
3673 | +# Licensed under the Apache License, Version 2.0 (the "License"); |
3674 | +# you may not use this file except in compliance with the License. |
3675 | +# You may obtain a copy of the License at |
3676 | +# |
3677 | +# http://www.apache.org/licenses/LICENSE-2.0 |
3678 | +# |
3679 | +# Unless required by applicable law or agreed to in writing, software |
3680 | +# distributed under the License is distributed on an "AS IS" BASIS, |
3681 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
3682 | +# See the License for the specific language governing permissions and |
3683 | +# limitations under the License. |
3684 | |
3685 | import os |
3686 | +from subprocess import check_call |
3687 | from charmhelpers.fetch import ( |
3688 | BaseFetchHandler, |
3689 | - UnhandledSource |
3690 | + UnhandledSource, |
3691 | + filter_installed_packages, |
3692 | + install, |
3693 | ) |
3694 | from charmhelpers.core.host import mkdir |
3695 | |
3696 | -import six |
3697 | -if six.PY3: |
3698 | - raise ImportError('bzrlib does not support Python3') |
3699 | |
3700 | -try: |
3701 | - from bzrlib.branch import Branch |
3702 | - from bzrlib import bzrdir, workingtree, errors |
3703 | -except ImportError: |
3704 | - from charmhelpers.fetch import apt_install |
3705 | - apt_install("python-bzrlib") |
3706 | - from bzrlib.branch import Branch |
3707 | - from bzrlib import bzrdir, workingtree, errors |
3708 | +if filter_installed_packages(['bzr']) != []: |
3709 | + install(['bzr']) |
3710 | + if filter_installed_packages(['bzr']) != []: |
3711 | + raise NotImplementedError('Unable to install bzr') |
3712 | |
3713 | |
3714 | class BzrUrlFetchHandler(BaseFetchHandler): |
3715 | - """Handler for bazaar branches via generic and lp URLs""" |
3716 | + """Handler for bazaar branches via generic and lp URLs.""" |
3717 | + |
3718 | def can_handle(self, source): |
3719 | url_parts = self.parse_url(source) |
3720 | - if url_parts.scheme not in ('bzr+ssh', 'lp'): |
3721 | + if url_parts.scheme not in ('bzr+ssh', 'lp', ''): |
3722 | return False |
3723 | + elif not url_parts.scheme: |
3724 | + return os.path.exists(os.path.join(source, '.bzr')) |
3725 | else: |
3726 | return True |
3727 | |
3728 | - def branch(self, source, dest): |
3729 | - url_parts = self.parse_url(source) |
3730 | - # If we use lp:branchname scheme we need to load plugins |
3731 | + def branch(self, source, dest, revno=None): |
3732 | if not self.can_handle(source): |
3733 | raise UnhandledSource("Cannot handle {}".format(source)) |
3734 | - if url_parts.scheme == "lp": |
3735 | - from bzrlib.plugin import load_plugins |
3736 | - load_plugins() |
3737 | - try: |
3738 | - local_branch = bzrdir.BzrDir.create_branch_convenience(dest) |
3739 | - except errors.AlreadyControlDirError: |
3740 | - local_branch = Branch.open(dest) |
3741 | - try: |
3742 | - remote_branch = Branch.open(source) |
3743 | - remote_branch.push(local_branch) |
3744 | - tree = workingtree.WorkingTree.open(dest) |
3745 | - tree.update() |
3746 | - except Exception as e: |
3747 | - raise e |
3748 | + cmd_opts = [] |
3749 | + if revno: |
3750 | + cmd_opts += ['-r', str(revno)] |
3751 | + if os.path.exists(dest): |
3752 | + cmd = ['bzr', 'pull'] |
3753 | + cmd += cmd_opts |
3754 | + cmd += ['--overwrite', '-d', dest, source] |
3755 | + else: |
3756 | + cmd = ['bzr', 'branch'] |
3757 | + cmd += cmd_opts |
3758 | + cmd += [source, dest] |
3759 | + check_call(cmd) |
3760 | |
3761 | - def install(self, source): |
3762 | + def install(self, source, dest=None, revno=None): |
3763 | url_parts = self.parse_url(source) |
3764 | branch_name = url_parts.path.strip("/").split("/")[-1] |
3765 | - dest_dir = os.path.join(os.environ.get('CHARM_DIR'), "fetched", |
3766 | - branch_name) |
3767 | - if not os.path.exists(dest_dir): |
3768 | - mkdir(dest_dir, perms=0o755) |
3769 | + if dest: |
3770 | + dest_dir = os.path.join(dest, branch_name) |
3771 | + else: |
3772 | + dest_dir = os.path.join(os.environ.get('CHARM_DIR'), "fetched", |
3773 | + branch_name) |
3774 | + |
3775 | + if dest and not os.path.exists(dest): |
3776 | + mkdir(dest, perms=0o755) |
3777 | + |
3778 | try: |
3779 | - self.branch(source, dest_dir) |
3780 | + self.branch(source, dest_dir, revno) |
3781 | except OSError as e: |
3782 | raise UnhandledSource(e.strerror) |
3783 | return dest_dir |
3784 | |
3785 | === added file 'hooks/charmhelpers/fetch/centos.py' |
3786 | --- hooks/charmhelpers/fetch/centos.py 1970-01-01 00:00:00 +0000 |
3787 | +++ hooks/charmhelpers/fetch/centos.py 2018-06-12 20:07:13 +0000 |
3788 | @@ -0,0 +1,171 @@ |
3789 | +# Copyright 2014-2015 Canonical Limited. |
3790 | +# |
3791 | +# Licensed under the Apache License, Version 2.0 (the "License"); |
3792 | +# you may not use this file except in compliance with the License. |
3793 | +# You may obtain a copy of the License at |
3794 | +# |
3795 | +# http://www.apache.org/licenses/LICENSE-2.0 |
3796 | +# |
3797 | +# Unless required by applicable law or agreed to in writing, software |
3798 | +# distributed under the License is distributed on an "AS IS" BASIS, |
3799 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
3800 | +# See the License for the specific language governing permissions and |
3801 | +# limitations under the License. |
3802 | + |
3803 | +import subprocess |
3804 | +import os |
3805 | +import time |
3806 | +import six |
3807 | +import yum |
3808 | + |
3809 | +from tempfile import NamedTemporaryFile |
3810 | +from charmhelpers.core.hookenv import log |
3811 | + |
3812 | +YUM_NO_LOCK = 1 # The return code for "couldn't acquire lock" in YUM. |
3813 | +YUM_NO_LOCK_RETRY_DELAY = 10 # Wait 10 seconds between apt lock checks. |
3814 | +YUM_NO_LOCK_RETRY_COUNT = 30 # Retry to acquire the lock X times. |
3815 | + |
3816 | + |
3817 | +def filter_installed_packages(packages): |
3818 | + """Return a list of packages that require installation.""" |
3819 | + yb = yum.YumBase() |
3820 | + package_list = yb.doPackageLists() |
3821 | + temp_cache = {p.base_package_name: 1 for p in package_list['installed']} |
3822 | + |
3823 | + _pkgs = [p for p in packages if not temp_cache.get(p, False)] |
3824 | + return _pkgs |
3825 | + |
3826 | + |
3827 | +def install(packages, options=None, fatal=False): |
3828 | + """Install one or more packages.""" |
3829 | + cmd = ['yum', '--assumeyes'] |
3830 | + if options is not None: |
3831 | + cmd.extend(options) |
3832 | + cmd.append('install') |
3833 | + if isinstance(packages, six.string_types): |
3834 | + cmd.append(packages) |
3835 | + else: |
3836 | + cmd.extend(packages) |
3837 | + log("Installing {} with options: {}".format(packages, |
3838 | + options)) |
3839 | + _run_yum_command(cmd, fatal) |
3840 | + |
3841 | + |
3842 | +def upgrade(options=None, fatal=False, dist=False): |
3843 | + """Upgrade all packages.""" |
3844 | + cmd = ['yum', '--assumeyes'] |
3845 | + if options is not None: |
3846 | + cmd.extend(options) |
3847 | + cmd.append('upgrade') |
3848 | + log("Upgrading with options: {}".format(options)) |
3849 | + _run_yum_command(cmd, fatal) |
3850 | + |
3851 | + |
3852 | +def update(fatal=False): |
3853 | + """Update local yum cache.""" |
3854 | + cmd = ['yum', '--assumeyes', 'update'] |
3855 | + log("Update with fatal: {}".format(fatal)) |
3856 | + _run_yum_command(cmd, fatal) |
3857 | + |
3858 | + |
3859 | +def purge(packages, fatal=False): |
3860 | + """Purge one or more packages.""" |
3861 | + cmd = ['yum', '--assumeyes', 'remove'] |
3862 | + if isinstance(packages, six.string_types): |
3863 | + cmd.append(packages) |
3864 | + else: |
3865 | + cmd.extend(packages) |
3866 | + log("Purging {}".format(packages)) |
3867 | + _run_yum_command(cmd, fatal) |
3868 | + |
3869 | + |
3870 | +def yum_search(packages): |
3871 | + """Search for a package.""" |
3872 | + output = {} |
3873 | + cmd = ['yum', 'search'] |
3874 | + if isinstance(packages, six.string_types): |
3875 | + cmd.append(packages) |
3876 | + else: |
3877 | + cmd.extend(packages) |
3878 | + log("Searching for {}".format(packages)) |
3879 | + result = subprocess.check_output(cmd) |
3880 | + for package in list(packages): |
3881 | + output[package] = package in result |
3882 | + return output |
3883 | + |
3884 | + |
3885 | +def add_source(source, key=None): |
3886 | + """Add a package source to this system. |
3887 | + |
3888 | + @param source: a URL with a rpm package |
3889 | + |
3890 | + @param key: A key to be added to the system's keyring and used |
3891 | + to verify the signatures on packages. Ideally, this should be an |
3892 | + ASCII format GPG public key including the block headers. A GPG key |
3893 | + id may also be used, but be aware that only insecure protocols are |
3894 | + available to retrieve the actual public key from a public keyserver |
3895 | + placing your Juju environment at risk. |
3896 | + """ |
3897 | + if source is None: |
3898 | + log('Source is not present. Skipping') |
3899 | + return |
3900 | + |
3901 | + if source.startswith('http'): |
3902 | + directory = '/etc/yum.repos.d/' |
3903 | + for filename in os.listdir(directory): |
3904 | + with open(directory + filename, 'r') as rpm_file: |
3905 | + if source in rpm_file.read(): |
3906 | + break |
3907 | + else: |
3908 | + log("Add source: {!r}".format(source)) |
3909 | + # write in the charms.repo |
3910 | + with open(directory + 'Charms.repo', 'a') as rpm_file: |
3911 | + rpm_file.write('[%s]\n' % source[7:].replace('/', '_')) |
3912 | + rpm_file.write('name=%s\n' % source[7:]) |
3913 | + rpm_file.write('baseurl=%s\n\n' % source) |
3914 | + else: |
3915 | + log("Unknown source: {!r}".format(source)) |
3916 | + |
3917 | + if key: |
3918 | + if '-----BEGIN PGP PUBLIC KEY BLOCK-----' in key: |
3919 | + with NamedTemporaryFile('w+') as key_file: |
3920 | + key_file.write(key) |
3921 | + key_file.flush() |
3922 | + key_file.seek(0) |
3923 | + subprocess.check_call(['rpm', '--import', key_file]) |
3924 | + else: |
3925 | + subprocess.check_call(['rpm', '--import', key]) |
3926 | + |
3927 | + |
3928 | +def _run_yum_command(cmd, fatal=False): |
3929 | + """Run an YUM command. |
3930 | + |
3931 | + Checks the output and retry if the fatal flag is set to True. |
3932 | + |
3933 | + :param: cmd: str: The yum command to run. |
3934 | + :param: fatal: bool: Whether the command's output should be checked and |
3935 | + retried. |
3936 | + """ |
3937 | + env = os.environ.copy() |
3938 | + |
3939 | + if fatal: |
3940 | + retry_count = 0 |
3941 | + result = None |
3942 | + |
3943 | + # If the command is considered "fatal", we need to retry if the yum |
3944 | + # lock was not acquired. |
3945 | + |
3946 | + while result is None or result == YUM_NO_LOCK: |
3947 | + try: |
3948 | + result = subprocess.check_call(cmd, env=env) |
3949 | + except subprocess.CalledProcessError as e: |
3950 | + retry_count = retry_count + 1 |
3951 | + if retry_count > YUM_NO_LOCK_RETRY_COUNT: |
3952 | + raise |
3953 | + result = e.returncode |
3954 | + log("Couldn't acquire YUM lock. Will retry in {} seconds." |
3955 | + "".format(YUM_NO_LOCK_RETRY_DELAY)) |
3956 | + time.sleep(YUM_NO_LOCK_RETRY_DELAY) |
3957 | + |
3958 | + else: |
3959 | + subprocess.call(cmd, env=env) |
3960 | |
3961 | === modified file 'hooks/charmhelpers/fetch/giturl.py' |
3962 | --- hooks/charmhelpers/fetch/giturl.py 2015-05-14 10:48:09 +0000 |
3963 | +++ hooks/charmhelpers/fetch/giturl.py 2018-06-12 20:07:13 +0000 |
3964 | @@ -1,58 +1,58 @@ |
3965 | # Copyright 2014-2015 Canonical Limited. |
3966 | # |
3967 | -# This file is part of charm-helpers. |
3968 | -# |
3969 | -# charm-helpers is free software: you can redistribute it and/or modify |
3970 | -# it under the terms of the GNU Lesser General Public License version 3 as |
3971 | -# published by the Free Software Foundation. |
3972 | -# |
3973 | -# charm-helpers is distributed in the hope that it will be useful, |
3974 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
3975 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
3976 | -# GNU Lesser General Public License for more details. |
3977 | -# |
3978 | -# You should have received a copy of the GNU Lesser General Public License |
3979 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
3980 | +# Licensed under the Apache License, Version 2.0 (the "License"); |
3981 | +# you may not use this file except in compliance with the License. |
3982 | +# You may obtain a copy of the License at |
3983 | +# |
3984 | +# http://www.apache.org/licenses/LICENSE-2.0 |
3985 | +# |
3986 | +# Unless required by applicable law or agreed to in writing, software |
3987 | +# distributed under the License is distributed on an "AS IS" BASIS, |
3988 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
3989 | +# See the License for the specific language governing permissions and |
3990 | +# limitations under the License. |
3991 | |
3992 | import os |
3993 | +from subprocess import check_call, CalledProcessError |
3994 | from charmhelpers.fetch import ( |
3995 | BaseFetchHandler, |
3996 | - UnhandledSource |
3997 | + UnhandledSource, |
3998 | + filter_installed_packages, |
3999 | + install, |
4000 | ) |
4001 | -from charmhelpers.core.host import mkdir |
4002 | - |
4003 | -import six |
4004 | -if six.PY3: |
4005 | - raise ImportError('GitPython does not support Python 3') |
4006 | - |
4007 | -try: |
4008 | - from git import Repo |
4009 | -except ImportError: |
4010 | - from charmhelpers.fetch import apt_install |
4011 | - apt_install("python-git") |
4012 | - from git import Repo |
4013 | - |
4014 | -from git.exc import GitCommandError # noqa E402 |
4015 | + |
4016 | +if filter_installed_packages(['git']) != []: |
4017 | + install(['git']) |
4018 | + if filter_installed_packages(['git']) != []: |
4019 | + raise NotImplementedError('Unable to install git') |
4020 | |
4021 | |
4022 | class GitUrlFetchHandler(BaseFetchHandler): |
4023 | - """Handler for git branches via generic and github URLs""" |
4024 | + """Handler for git branches via generic and github URLs.""" |
4025 | + |
4026 | def can_handle(self, source): |
4027 | url_parts = self.parse_url(source) |
4028 | # TODO (mattyw) no support for ssh git@ yet |
4029 | - if url_parts.scheme not in ('http', 'https', 'git'): |
4030 | + if url_parts.scheme not in ('http', 'https', 'git', ''): |
4031 | return False |
4032 | + elif not url_parts.scheme: |
4033 | + return os.path.exists(os.path.join(source, '.git')) |
4034 | else: |
4035 | return True |
4036 | |
4037 | - def clone(self, source, dest, branch): |
4038 | + def clone(self, source, dest, branch="master", depth=None): |
4039 | if not self.can_handle(source): |
4040 | raise UnhandledSource("Cannot handle {}".format(source)) |
4041 | |
4042 | - repo = Repo.clone_from(source, dest) |
4043 | - repo.git.checkout(branch) |
4044 | + if os.path.exists(dest): |
4045 | + cmd = ['git', '-C', dest, 'pull', source, branch] |
4046 | + else: |
4047 | + cmd = ['git', 'clone', source, dest, '--branch', branch] |
4048 | + if depth: |
4049 | + cmd.extend(['--depth', depth]) |
4050 | + check_call(cmd) |
4051 | |
4052 | - def install(self, source, branch="master", dest=None): |
4053 | + def install(self, source, branch="master", dest=None, depth=None): |
4054 | url_parts = self.parse_url(source) |
4055 | branch_name = url_parts.path.strip("/").split("/")[-1] |
4056 | if dest: |
4057 | @@ -60,12 +60,10 @@ |
4058 | else: |
4059 | dest_dir = os.path.join(os.environ.get('CHARM_DIR'), "fetched", |
4060 | branch_name) |
4061 | - if not os.path.exists(dest_dir): |
4062 | - mkdir(dest_dir, perms=0o755) |
4063 | try: |
4064 | - self.clone(source, dest_dir, branch) |
4065 | - except GitCommandError as e: |
4066 | - raise UnhandledSource(e.message) |
4067 | + self.clone(source, dest_dir, branch, depth) |
4068 | + except CalledProcessError as e: |
4069 | + raise UnhandledSource(e) |
4070 | except OSError as e: |
4071 | raise UnhandledSource(e.strerror) |
4072 | return dest_dir |
4073 | |
4074 | === added file 'hooks/charmhelpers/fetch/snap.py' |
4075 | --- hooks/charmhelpers/fetch/snap.py 1970-01-01 00:00:00 +0000 |
4076 | +++ hooks/charmhelpers/fetch/snap.py 2018-06-12 20:07:13 +0000 |
4077 | @@ -0,0 +1,122 @@ |
4078 | +# Copyright 2014-2017 Canonical Limited. |
4079 | +# |
4080 | +# Licensed under the Apache License, Version 2.0 (the "License"); |
4081 | +# you may not use this file except in compliance with the License. |
4082 | +# You may obtain a copy of the License at |
4083 | +# |
4084 | +# http://www.apache.org/licenses/LICENSE-2.0 |
4085 | +# |
4086 | +# Unless required by applicable law or agreed to in writing, software |
4087 | +# distributed under the License is distributed on an "AS IS" BASIS, |
4088 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
4089 | +# See the License for the specific language governing permissions and |
4090 | +# limitations under the License. |
4091 | +""" |
4092 | +Charm helpers snap for classic charms. |
4093 | + |
4094 | +If writing reactive charms, use the snap layer: |
4095 | +https://lists.ubuntu.com/archives/snapcraft/2016-September/001114.html |
4096 | +""" |
4097 | +import subprocess |
4098 | +from os import environ |
4099 | +from time import sleep |
4100 | +from charmhelpers.core.hookenv import log |
4101 | + |
4102 | +__author__ = 'Joseph Borg <joseph.borg@canonical.com>' |
4103 | + |
4104 | +SNAP_NO_LOCK = 1 # The return code for "couldn't acquire lock" in Snap (hopefully this will be improved). |
4105 | +SNAP_NO_LOCK_RETRY_DELAY = 10 # Wait X seconds between Snap lock checks. |
4106 | +SNAP_NO_LOCK_RETRY_COUNT = 30 # Retry to acquire the lock X times. |
4107 | + |
4108 | + |
4109 | +class CouldNotAcquireLockException(Exception): |
4110 | + pass |
4111 | + |
4112 | + |
4113 | +def _snap_exec(commands): |
4114 | + """ |
4115 | + Execute snap commands. |
4116 | + |
4117 | + :param commands: List commands |
4118 | + :return: Integer exit code |
4119 | + """ |
4120 | + assert type(commands) == list |
4121 | + |
4122 | + retry_count = 0 |
4123 | + return_code = None |
4124 | + |
4125 | + while return_code is None or return_code == SNAP_NO_LOCK: |
4126 | + try: |
4127 | + return_code = subprocess.check_call(['snap'] + commands, env=environ) |
4128 | + except subprocess.CalledProcessError as e: |
4129 | + retry_count += + 1 |
4130 | + if retry_count > SNAP_NO_LOCK_RETRY_COUNT: |
4131 | + raise CouldNotAcquireLockException('Could not aquire lock after %s attempts' % SNAP_NO_LOCK_RETRY_COUNT) |
4132 | + return_code = e.returncode |
4133 | + log('Snap failed to acquire lock, trying again in %s seconds.' % SNAP_NO_LOCK_RETRY_DELAY, level='WARN') |
4134 | + sleep(SNAP_NO_LOCK_RETRY_DELAY) |
4135 | + |
4136 | + return return_code |
4137 | + |
4138 | + |
4139 | +def snap_install(packages, *flags): |
4140 | + """ |
4141 | + Install a snap package. |
4142 | + |
4143 | + :param packages: String or List String package name |
4144 | + :param flags: List String flags to pass to install command |
4145 | + :return: Integer return code from snap |
4146 | + """ |
4147 | + if type(packages) is not list: |
4148 | + packages = [packages] |
4149 | + |
4150 | + flags = list(flags) |
4151 | + |
4152 | + message = 'Installing snap(s) "%s"' % ', '.join(packages) |
4153 | + if flags: |
4154 | + message += ' with option(s) "%s"' % ', '.join(flags) |
4155 | + |
4156 | + log(message, level='INFO') |
4157 | + return _snap_exec(['install'] + flags + packages) |
4158 | + |
4159 | + |
4160 | +def snap_remove(packages, *flags): |
4161 | + """ |
4162 | + Remove a snap package. |
4163 | + |
4164 | + :param packages: String or List String package name |
4165 | + :param flags: List String flags to pass to remove command |
4166 | + :return: Integer return code from snap |
4167 | + """ |
4168 | + if type(packages) is not list: |
4169 | + packages = [packages] |
4170 | + |
4171 | + flags = list(flags) |
4172 | + |
4173 | + message = 'Removing snap(s) "%s"' % ', '.join(packages) |
4174 | + if flags: |
4175 | + message += ' with options "%s"' % ', '.join(flags) |
4176 | + |
4177 | + log(message, level='INFO') |
4178 | + return _snap_exec(['remove'] + flags + packages) |
4179 | + |
4180 | + |
4181 | +def snap_refresh(packages, *flags): |
4182 | + """ |
4183 | + Refresh / Update snap package. |
4184 | + |
4185 | + :param packages: String or List String package name |
4186 | + :param flags: List String flags to pass to refresh command |
4187 | + :return: Integer return code from snap |
4188 | + """ |
4189 | + if type(packages) is not list: |
4190 | + packages = [packages] |
4191 | + |
4192 | + flags = list(flags) |
4193 | + |
4194 | + message = 'Refreshing snap(s) "%s"' % ', '.join(packages) |
4195 | + if flags: |
4196 | + message += ' with options "%s"' % ', '.join(flags) |
4197 | + |
4198 | + log(message, level='INFO') |
4199 | + return _snap_exec(['refresh'] + flags + packages) |
4200 | |
4201 | === added file 'hooks/charmhelpers/fetch/ubuntu.py' |
4202 | --- hooks/charmhelpers/fetch/ubuntu.py 1970-01-01 00:00:00 +0000 |
4203 | +++ hooks/charmhelpers/fetch/ubuntu.py 2018-06-12 20:07:13 +0000 |
4204 | @@ -0,0 +1,364 @@ |
4205 | +# Copyright 2014-2015 Canonical Limited. |
4206 | +# |
4207 | +# Licensed under the Apache License, Version 2.0 (the "License"); |
4208 | +# you may not use this file except in compliance with the License. |
4209 | +# You may obtain a copy of the License at |
4210 | +# |
4211 | +# http://www.apache.org/licenses/LICENSE-2.0 |
4212 | +# |
4213 | +# Unless required by applicable law or agreed to in writing, software |
4214 | +# distributed under the License is distributed on an "AS IS" BASIS, |
4215 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
4216 | +# See the License for the specific language governing permissions and |
4217 | +# limitations under the License. |
4218 | + |
4219 | +import os |
4220 | +import six |
4221 | +import time |
4222 | +import subprocess |
4223 | + |
4224 | +from tempfile import NamedTemporaryFile |
4225 | +from charmhelpers.core.host import ( |
4226 | + lsb_release |
4227 | +) |
4228 | +from charmhelpers.core.hookenv import log |
4229 | +from charmhelpers.fetch import SourceConfigError |
4230 | + |
4231 | +CLOUD_ARCHIVE = """# Ubuntu Cloud Archive |
4232 | +deb http://ubuntu-cloud.archive.canonical.com/ubuntu {} main |
4233 | +""" |
4234 | + |
4235 | +PROPOSED_POCKET = """# Proposed |
4236 | +deb http://archive.ubuntu.com/ubuntu {}-proposed main universe multiverse restricted |
4237 | +""" |
4238 | + |
4239 | +CLOUD_ARCHIVE_POCKETS = { |
4240 | + # Folsom |
4241 | + 'folsom': 'precise-updates/folsom', |
4242 | + 'precise-folsom': 'precise-updates/folsom', |
4243 | + 'precise-folsom/updates': 'precise-updates/folsom', |
4244 | + 'precise-updates/folsom': 'precise-updates/folsom', |
4245 | + 'folsom/proposed': 'precise-proposed/folsom', |
4246 | + 'precise-folsom/proposed': 'precise-proposed/folsom', |
4247 | + 'precise-proposed/folsom': 'precise-proposed/folsom', |
4248 | + # Grizzly |
4249 | + 'grizzly': 'precise-updates/grizzly', |
4250 | + 'precise-grizzly': 'precise-updates/grizzly', |
4251 | + 'precise-grizzly/updates': 'precise-updates/grizzly', |
4252 | + 'precise-updates/grizzly': 'precise-updates/grizzly', |
4253 | + 'grizzly/proposed': 'precise-proposed/grizzly', |
4254 | + 'precise-grizzly/proposed': 'precise-proposed/grizzly', |
4255 | + 'precise-proposed/grizzly': 'precise-proposed/grizzly', |
4256 | + # Havana |
4257 | + 'havana': 'precise-updates/havana', |
4258 | + 'precise-havana': 'precise-updates/havana', |
4259 | + 'precise-havana/updates': 'precise-updates/havana', |
4260 | + 'precise-updates/havana': 'precise-updates/havana', |
4261 | + 'havana/proposed': 'precise-proposed/havana', |
4262 | + 'precise-havana/proposed': 'precise-proposed/havana', |
4263 | + 'precise-proposed/havana': 'precise-proposed/havana', |
4264 | + # Icehouse |
4265 | + 'icehouse': 'precise-updates/icehouse', |
4266 | + 'precise-icehouse': 'precise-updates/icehouse', |
4267 | + 'precise-icehouse/updates': 'precise-updates/icehouse', |
4268 | + 'precise-updates/icehouse': 'precise-updates/icehouse', |
4269 | + 'icehouse/proposed': 'precise-proposed/icehouse', |
4270 | + 'precise-icehouse/proposed': 'precise-proposed/icehouse', |
4271 | + 'precise-proposed/icehouse': 'precise-proposed/icehouse', |
4272 | + # Juno |
4273 | + 'juno': 'trusty-updates/juno', |
4274 | + 'trusty-juno': 'trusty-updates/juno', |
4275 | + 'trusty-juno/updates': 'trusty-updates/juno', |
4276 | + 'trusty-updates/juno': 'trusty-updates/juno', |
4277 | + 'juno/proposed': 'trusty-proposed/juno', |
4278 | + 'trusty-juno/proposed': 'trusty-proposed/juno', |
4279 | + 'trusty-proposed/juno': 'trusty-proposed/juno', |
4280 | + # Kilo |
4281 | + 'kilo': 'trusty-updates/kilo', |
4282 | + 'trusty-kilo': 'trusty-updates/kilo', |
4283 | + 'trusty-kilo/updates': 'trusty-updates/kilo', |
4284 | + 'trusty-updates/kilo': 'trusty-updates/kilo', |
4285 | + 'kilo/proposed': 'trusty-proposed/kilo', |
4286 | + 'trusty-kilo/proposed': 'trusty-proposed/kilo', |
4287 | + 'trusty-proposed/kilo': 'trusty-proposed/kilo', |
4288 | + # Liberty |
4289 | + 'liberty': 'trusty-updates/liberty', |
4290 | + 'trusty-liberty': 'trusty-updates/liberty', |
4291 | + 'trusty-liberty/updates': 'trusty-updates/liberty', |
4292 | + 'trusty-updates/liberty': 'trusty-updates/liberty', |
4293 | + 'liberty/proposed': 'trusty-proposed/liberty', |
4294 | + 'trusty-liberty/proposed': 'trusty-proposed/liberty', |
4295 | + 'trusty-proposed/liberty': 'trusty-proposed/liberty', |
4296 | + # Mitaka |
4297 | + 'mitaka': 'trusty-updates/mitaka', |
4298 | + 'trusty-mitaka': 'trusty-updates/mitaka', |
4299 | + 'trusty-mitaka/updates': 'trusty-updates/mitaka', |
4300 | + 'trusty-updates/mitaka': 'trusty-updates/mitaka', |
4301 | + 'mitaka/proposed': 'trusty-proposed/mitaka', |
4302 | + 'trusty-mitaka/proposed': 'trusty-proposed/mitaka', |
4303 | + 'trusty-proposed/mitaka': 'trusty-proposed/mitaka', |
4304 | + # Newton |
4305 | + 'newton': 'xenial-updates/newton', |
4306 | + 'xenial-newton': 'xenial-updates/newton', |
4307 | + 'xenial-newton/updates': 'xenial-updates/newton', |
4308 | + 'xenial-updates/newton': 'xenial-updates/newton', |
4309 | + 'newton/proposed': 'xenial-proposed/newton', |
4310 | + 'xenial-newton/proposed': 'xenial-proposed/newton', |
4311 | + 'xenial-proposed/newton': 'xenial-proposed/newton', |
4312 | + # Ocata |
4313 | + 'ocata': 'xenial-updates/ocata', |
4314 | + 'xenial-ocata': 'xenial-updates/ocata', |
4315 | + 'xenial-ocata/updates': 'xenial-updates/ocata', |
4316 | + 'xenial-updates/ocata': 'xenial-updates/ocata', |
4317 | + 'ocata/proposed': 'xenial-proposed/ocata', |
4318 | + 'xenial-ocata/proposed': 'xenial-proposed/ocata', |
4319 | + 'xenial-ocata/newton': 'xenial-proposed/ocata', |
4320 | +} |
4321 | + |
4322 | +APT_NO_LOCK = 100 # The return code for "couldn't acquire lock" in APT. |
4323 | +CMD_RETRY_DELAY = 10 # Wait 10 seconds between command retries. |
4324 | +CMD_RETRY_COUNT = 30 # Retry a failing fatal command X times. |
4325 | + |
4326 | + |
4327 | +def filter_installed_packages(packages): |
4328 | + """Return a list of packages that require installation.""" |
4329 | + cache = apt_cache() |
4330 | + _pkgs = [] |
4331 | + for package in packages: |
4332 | + try: |
4333 | + p = cache[package] |
4334 | + p.current_ver or _pkgs.append(package) |
4335 | + except KeyError: |
4336 | + log('Package {} has no installation candidate.'.format(package), |
4337 | + level='WARNING') |
4338 | + _pkgs.append(package) |
4339 | + return _pkgs |
4340 | + |
4341 | + |
4342 | +def apt_cache(in_memory=True, progress=None): |
4343 | + """Build and return an apt cache.""" |
4344 | + from apt import apt_pkg |
4345 | + apt_pkg.init() |
4346 | + if in_memory: |
4347 | + apt_pkg.config.set("Dir::Cache::pkgcache", "") |
4348 | + apt_pkg.config.set("Dir::Cache::srcpkgcache", "") |
4349 | + return apt_pkg.Cache(progress) |
4350 | + |
4351 | + |
4352 | +def install(packages, options=None, fatal=False): |
4353 | + """Install one or more packages.""" |
4354 | + if options is None: |
4355 | + options = ['--option=Dpkg::Options::=--force-confold'] |
4356 | + |
4357 | + cmd = ['apt-get', '--assume-yes'] |
4358 | + cmd.extend(options) |
4359 | + cmd.append('install') |
4360 | + if isinstance(packages, six.string_types): |
4361 | + cmd.append(packages) |
4362 | + else: |
4363 | + cmd.extend(packages) |
4364 | + log("Installing {} with options: {}".format(packages, |
4365 | + options)) |
4366 | + _run_apt_command(cmd, fatal) |
4367 | + |
4368 | + |
4369 | +def upgrade(options=None, fatal=False, dist=False): |
4370 | + """Upgrade all packages.""" |
4371 | + if options is None: |
4372 | + options = ['--option=Dpkg::Options::=--force-confold'] |
4373 | + |
4374 | + cmd = ['apt-get', '--assume-yes'] |
4375 | + cmd.extend(options) |
4376 | + if dist: |
4377 | + cmd.append('dist-upgrade') |
4378 | + else: |
4379 | + cmd.append('upgrade') |
4380 | + log("Upgrading with options: {}".format(options)) |
4381 | + _run_apt_command(cmd, fatal) |
4382 | + |
4383 | + |
4384 | +def update(fatal=False): |
4385 | + """Update local apt cache.""" |
4386 | + cmd = ['apt-get', 'update'] |
4387 | + _run_apt_command(cmd, fatal) |
4388 | + |
4389 | + |
4390 | +def purge(packages, fatal=False): |
4391 | + """Purge one or more packages.""" |
4392 | + cmd = ['apt-get', '--assume-yes', 'purge'] |
4393 | + if isinstance(packages, six.string_types): |
4394 | + cmd.append(packages) |
4395 | + else: |
4396 | + cmd.extend(packages) |
4397 | + log("Purging {}".format(packages)) |
4398 | + _run_apt_command(cmd, fatal) |
4399 | + |
4400 | + |
4401 | +def apt_mark(packages, mark, fatal=False): |
4402 | + """Flag one or more packages using apt-mark.""" |
4403 | + log("Marking {} as {}".format(packages, mark)) |
4404 | + cmd = ['apt-mark', mark] |
4405 | + if isinstance(packages, six.string_types): |
4406 | + cmd.append(packages) |
4407 | + else: |
4408 | + cmd.extend(packages) |
4409 | + |
4410 | + if fatal: |
4411 | + subprocess.check_call(cmd, universal_newlines=True) |
4412 | + else: |
4413 | + subprocess.call(cmd, universal_newlines=True) |
4414 | + |
4415 | + |
4416 | +def apt_hold(packages, fatal=False): |
4417 | + return apt_mark(packages, 'hold', fatal=fatal) |
4418 | + |
4419 | + |
4420 | +def apt_unhold(packages, fatal=False): |
4421 | + return apt_mark(packages, 'unhold', fatal=fatal) |
4422 | + |
4423 | + |
4424 | +def add_source(source, key=None): |
4425 | + """Add a package source to this system. |
4426 | + |
4427 | + @param source: a URL or sources.list entry, as supported by |
4428 | + add-apt-repository(1). Examples:: |
4429 | + |
4430 | + ppa:charmers/example |
4431 | + deb https://stub:key@private.example.com/ubuntu trusty main |
4432 | + |
4433 | + In addition: |
4434 | + 'proposed:' may be used to enable the standard 'proposed' |
4435 | + pocket for the release. |
4436 | + 'cloud:' may be used to activate official cloud archive pockets, |
4437 | + such as 'cloud:icehouse' |
4438 | + 'distro' may be used as a noop |
4439 | + |
4440 | + @param key: A key to be added to the system's APT keyring and used |
4441 | + to verify the signatures on packages. Ideally, this should be an |
4442 | + ASCII format GPG public key including the block headers. A GPG key |
4443 | + id may also be used, but be aware that only insecure protocols are |
4444 | + available to retrieve the actual public key from a public keyserver |
4445 | + placing your Juju environment at risk. ppa and cloud archive keys |
4446 | + are securely added automtically, so sould not be provided. |
4447 | + """ |
4448 | + if source is None: |
4449 | + log('Source is not present. Skipping') |
4450 | + return |
4451 | + |
4452 | + if (source.startswith('ppa:') or |
4453 | + source.startswith('http') or |
4454 | + source.startswith('deb ') or |
4455 | + source.startswith('cloud-archive:')): |
4456 | + cmd = ['add-apt-repository', '--yes', source] |
4457 | + _run_with_retries(cmd) |
4458 | + elif source.startswith('cloud:'): |
4459 | + install(filter_installed_packages(['ubuntu-cloud-keyring']), |
4460 | + fatal=True) |
4461 | + pocket = source.split(':')[-1] |
4462 | + if pocket not in CLOUD_ARCHIVE_POCKETS: |
4463 | + raise SourceConfigError( |
4464 | + 'Unsupported cloud: source option %s' % |
4465 | + pocket) |
4466 | + actual_pocket = CLOUD_ARCHIVE_POCKETS[pocket] |
4467 | + with open('/etc/apt/sources.list.d/cloud-archive.list', 'w') as apt: |
4468 | + apt.write(CLOUD_ARCHIVE.format(actual_pocket)) |
4469 | + elif source == 'proposed': |
4470 | + release = lsb_release()['DISTRIB_CODENAME'] |
4471 | + with open('/etc/apt/sources.list.d/proposed.list', 'w') as apt: |
4472 | + apt.write(PROPOSED_POCKET.format(release)) |
4473 | + elif source == 'distro': |
4474 | + pass |
4475 | + else: |
4476 | + log("Unknown source: {!r}".format(source)) |
4477 | + |
4478 | + if key: |
4479 | + if '-----BEGIN PGP PUBLIC KEY BLOCK-----' in key: |
4480 | + with NamedTemporaryFile('w+') as key_file: |
4481 | + key_file.write(key) |
4482 | + key_file.flush() |
4483 | + key_file.seek(0) |
4484 | + subprocess.check_call(['apt-key', 'add', '-'], stdin=key_file) |
4485 | + else: |
4486 | + # Note that hkp: is in no way a secure protocol. Using a |
4487 | + # GPG key id is pointless from a security POV unless you |
4488 | + # absolutely trust your network and DNS. |
4489 | + subprocess.check_call(['apt-key', 'adv', '--keyserver', |
4490 | + 'hkp://keyserver.ubuntu.com:80', '--recv', |
4491 | + key]) |
4492 | + |
4493 | + |
4494 | +def _run_with_retries(cmd, max_retries=CMD_RETRY_COUNT, retry_exitcodes=(1,), |
4495 | + retry_message="", cmd_env=None): |
4496 | + """Run a command and retry until success or max_retries is reached. |
4497 | + |
4498 | + :param: cmd: str: The apt command to run. |
4499 | + :param: max_retries: int: The number of retries to attempt on a fatal |
4500 | + command. Defaults to CMD_RETRY_COUNT. |
4501 | + :param: retry_exitcodes: tuple: Optional additional exit codes to retry. |
4502 | + Defaults to retry on exit code 1. |
4503 | + :param: retry_message: str: Optional log prefix emitted during retries. |
4504 | + :param: cmd_env: dict: Environment variables to add to the command run. |
4505 | + """ |
4506 | + |
4507 | + env = os.environ.copy() |
4508 | + if cmd_env: |
4509 | + env.update(cmd_env) |
4510 | + |
4511 | + if not retry_message: |
4512 | + retry_message = "Failed executing '{}'".format(" ".join(cmd)) |
4513 | + retry_message += ". Will retry in {} seconds".format(CMD_RETRY_DELAY) |
4514 | + |
4515 | + retry_count = 0 |
4516 | + result = None |
4517 | + |
4518 | + retry_results = (None,) + retry_exitcodes |
4519 | + while result in retry_results: |
4520 | + try: |
4521 | + result = subprocess.check_call(cmd, env=env) |
4522 | + except subprocess.CalledProcessError as e: |
4523 | + retry_count = retry_count + 1 |
4524 | + if retry_count > max_retries: |
4525 | + raise |
4526 | + result = e.returncode |
4527 | + log(retry_message) |
4528 | + time.sleep(CMD_RETRY_DELAY) |
4529 | + |
4530 | + |
4531 | +def _run_apt_command(cmd, fatal=False): |
4532 | + """Run an apt command with optional retries. |
4533 | + |
4534 | + :param: fatal: bool: Whether the command's output should be checked and |
4535 | + retried. |
4536 | + """ |
4537 | + # Provide DEBIAN_FRONTEND=noninteractive if not present in the environment. |
4538 | + cmd_env = { |
4539 | + 'DEBIAN_FRONTEND': os.environ.get('DEBIAN_FRONTEND', 'noninteractive')} |
4540 | + |
4541 | + if fatal: |
4542 | + _run_with_retries( |
4543 | + cmd, cmd_env=cmd_env, retry_exitcodes=(1, APT_NO_LOCK,), |
4544 | + retry_message="Couldn't acquire DPKG lock") |
4545 | + else: |
4546 | + env = os.environ.copy() |
4547 | + env.update(cmd_env) |
4548 | + subprocess.call(cmd, env=env) |
4549 | + |
4550 | + |
4551 | +def get_upstream_version(package): |
4552 | + """Determine upstream version based on installed package |
4553 | + |
4554 | + @returns None (if not installed) or the upstream version |
4555 | + """ |
4556 | + import apt_pkg |
4557 | + cache = apt_cache() |
4558 | + try: |
4559 | + pkg = cache[package] |
4560 | + except: |
4561 | + # the package is unknown to the current apt cache. |
4562 | + return None |
4563 | + |
4564 | + if not pkg.current_ver: |
4565 | + # package is known, but no version is currently installed. |
4566 | + return None |
4567 | + |
4568 | + return apt_pkg.upstream_version(pkg.current_ver.ver_str) |
4569 | |
4570 | === added file 'hooks/charmhelpers/osplatform.py' |
4571 | --- hooks/charmhelpers/osplatform.py 1970-01-01 00:00:00 +0000 |
4572 | +++ hooks/charmhelpers/osplatform.py 2018-06-12 20:07:13 +0000 |
4573 | @@ -0,0 +1,25 @@ |
4574 | +import platform |
4575 | + |
4576 | + |
4577 | +def get_platform(): |
4578 | + """Return the current OS platform. |
4579 | + |
4580 | + For example: if current os platform is Ubuntu then a string "ubuntu" |
4581 | + will be returned (which is the name of the module). |
4582 | + This string is used to decide which platform module should be imported. |
4583 | + """ |
4584 | + # linux_distribution is deprecated and will be removed in Python 3.7 |
4585 | + # Warings *not* disabled, as we certainly need to fix this. |
4586 | + tuple_platform = platform.linux_distribution() |
4587 | + current_platform = tuple_platform[0] |
4588 | + if "Ubuntu" in current_platform: |
4589 | + return "ubuntu" |
4590 | + elif "CentOS" in current_platform: |
4591 | + return "centos" |
4592 | + elif "debian" in current_platform: |
4593 | + # Stock Python does not detect Ubuntu and instead returns debian. |
4594 | + # Or at least it does in some build environments like Travis CI |
4595 | + return "ubuntu" |
4596 | + else: |
4597 | + raise RuntimeError("This module is not supported on {}." |
4598 | + .format(current_platform)) |
4599 | |
4600 | === modified file 'hooks/hooks.py' |
4601 | --- hooks/hooks.py 2017-07-25 07:56:09 +0000 |
4602 | +++ hooks/hooks.py 2018-06-12 20:07:13 +0000 |
4603 | @@ -786,7 +786,7 @@ |
4604 | # Write to disk the content of the given SSL certificates |
4605 | crts = service_config.get('crts', []) |
4606 | for i, crt in enumerate(crts): |
4607 | - if crt == "DEFAULT": |
4608 | + if crt == "DEFAULT" or crt == "EXTERNAL": |
4609 | continue |
4610 | content = base64.b64decode(crt) |
4611 | path = get_service_lib_path(service_name) |
4612 | |
4613 | === modified file 'icon.svg' |
4614 | --- icon.svg 2016-01-09 16:42:37 +0000 |
4615 | +++ icon.svg 2018-06-12 20:07:13 +0000 |
4616 | @@ -15,10 +15,21 @@ |
4617 | id="svg6517" |
4618 | version="1.1" |
4619 | inkscape:version="0.91+devel r" |
4620 | - sodipodi:docname="haproxy.svg" |
4621 | + sodipodi:docname="haproxy_circle.svg" |
4622 | viewBox="0 0 96 96"> |
4623 | <defs |
4624 | id="defs6519"> |
4625 | + <linearGradient |
4626 | + id="Background"> |
4627 | + <stop |
4628 | + id="stop4178" |
4629 | + offset="0" |
4630 | + style="stop-color:#dedede;stop-opacity:1" /> |
4631 | + <stop |
4632 | + id="stop4180" |
4633 | + offset="1" |
4634 | + style="stop-color:#ededed;stop-opacity:1" /> |
4635 | + </linearGradient> |
4636 | <filter |
4637 | style="color-interpolation-filters:sRGB" |
4638 | inkscape:label="Inner Shadow" |
4639 | @@ -68,7 +79,7 @@ |
4640 | id="feComposite954" /> |
4641 | <feGaussianBlur |
4642 | in="composite1" |
4643 | - stdDeviation="4" |
4644 | + stdDeviation="1" |
4645 | result="blur" |
4646 | id="feGaussianBlur956" /> |
4647 | <feOffset |
4648 | @@ -83,17 +94,6 @@ |
4649 | result="composite2" |
4650 | id="feComposite960" /> |
4651 | </filter> |
4652 | - <linearGradient |
4653 | - id="Background"> |
4654 | - <stop |
4655 | - id="stop4178" |
4656 | - offset="0" |
4657 | - style="stop-color:#22779e;stop-opacity:1" /> |
4658 | - <stop |
4659 | - id="stop4180" |
4660 | - offset="1" |
4661 | - style="stop-color:#2991c0;stop-opacity:1" /> |
4662 | - </linearGradient> |
4663 | <clipPath |
4664 | clipPathUnits="userSpaceOnUse" |
4665 | id="clipPath873"> |
4666 | @@ -110,128 +110,416 @@ |
4667 | sodipodi:nodetypes="sssssssss" /> |
4668 | </g> |
4669 | </clipPath> |
4670 | - <filter |
4671 | - inkscape:collect="always" |
4672 | - id="filter891" |
4673 | - inkscape:label="Badge Shadow"> |
4674 | - <feGaussianBlur |
4675 | - inkscape:collect="always" |
4676 | - stdDeviation="0.71999962" |
4677 | - id="feGaussianBlur893" /> |
4678 | - </filter> |
4679 | - <style |
4680 | - id="style867" |
4681 | - type="text/css"><![CDATA[ |
4682 | - .fil0 {fill:#1F1A17} |
4683 | - ]]></style> |
4684 | - <clipPath |
4685 | - id="clipPath16"> |
4686 | - <path |
4687 | - id="path18" |
4688 | - d="M -9,-9 H 605 V 222 H -9 Z" |
4689 | - inkscape:connector-curvature="0" /> |
4690 | - </clipPath> |
4691 | - <clipPath |
4692 | - id="clipPath116"> |
4693 | - <path |
4694 | - id="path118" |
4695 | - d="m 91.7368,146.3253 -9.7039,-1.577 -8.8548,-3.8814 -7.5206,-4.7308 -7.1566,-8.7335 -4.0431,-4.282 -3.9093,-1.4409 -1.034,2.5271 1.8079,2.6096 0.4062,3.6802 1.211,-0.0488 1.3232,-1.2069 -0.3569,3.7488 -1.4667,0.9839 0.0445,1.4286 -3.4744,-1.9655 -3.1462,-3.712 -0.6559,-3.3176 1.3453,-2.6567 1.2549,-4.5133 2.5521,-1.2084 2.6847,0.1318 2.5455,1.4791 -1.698,-8.6122 1.698,-9.5825 -1.8692,-4.4246 -6.1223,-6.5965 1.0885,-3.941 2.9002,-4.5669 5.4688,-3.8486 2.9007,-0.3969 3.225,-0.1094 -2.012,-8.2601 7.3993,-3.0326 9.2188,-1.2129 3.1535,2.0619 0.2427,5.5797 3.5178,5.8224 0.2426,4.6094 8.4909,-0.6066 7.8843,0.7279 -7.8843,-4.7307 1.3343,-5.701 4.9731,-7.763 4.8521,-2.0622 3.8814,1.5769 1.577,3.1538 8.1269,6.1861 1.5769,-1.3343 12.7363,-0.485 2.5473,2.0619 0.2426,3.6391 -0.849,1.5767 -0.6066,9.8251 -4.2454,8.4909 0.7276,3.7605 2.5475,-1.3343 7.1566,-6.6716 3.5175,-0.2424 3.8815,1.5769 3.8818,2.9109 1.9406,6.3077 11.4021,-0.7277 6.914,2.6686 5.5797,5.2157 4.0028,7.5206 0.9706,8.8546 -0.8493,10.3105 -2.1832,9.2185 -2.1836,2.9112 -3.0322,0.9706 -5.3373,-5.8224 -4.8518,-1.6982 -4.2455,7.0353 -4.2454,3.8815 -2.3049,1.4556 -9.2185,7.6419 -7.3993,4.0028 -7.3993,0.6066 -8.6119,-1.4556 -7.5206,-2.7899 -5.2158,-4.2454 -4.1241,-4.9734 -4.2454,-1.2129" |
4696 | - inkscape:connector-curvature="0" /> |
4697 | - </clipPath> |
4698 | - <clipPath |
4699 | - id="clipPath128"> |
4700 | - <path |
4701 | - id="path130" |
4702 | - d="m 91.7368,146.3253 -9.7039,-1.577 -8.8548,-3.8814 -7.5206,-4.7308 -7.1566,-8.7335 -4.0431,-4.282 -3.9093,-1.4409 -1.034,2.5271 1.8079,2.6096 0.4062,3.6802 1.211,-0.0488 1.3232,-1.2069 -0.3569,3.7488 -1.4667,0.9839 0.0445,1.4286 -3.4744,-1.9655 -3.1462,-3.712 -0.6559,-3.3176 1.3453,-2.6567 1.2549,-4.5133 2.5521,-1.2084 2.6847,0.1318 2.5455,1.4791 -1.698,-8.6122 1.698,-9.5825 -1.8692,-4.4246 -6.1223,-6.5965 1.0885,-3.941 2.9002,-4.5669 5.4688,-3.8486 2.9007,-0.3969 3.225,-0.1094 -2.012,-8.2601 7.3993,-3.0326 9.2188,-1.2129 3.1535,2.0619 0.2427,5.5797 3.5178,5.8224 0.2426,4.6094 8.4909,-0.6066 7.8843,0.7279 -7.8843,-4.7307 1.3343,-5.701 4.9731,-7.763 4.8521,-2.0622 3.8814,1.5769 1.577,3.1538 8.1269,6.1861 1.5769,-1.3343 12.7363,-0.485 2.5473,2.0619 0.2426,3.6391 -0.849,1.5767 -0.6066,9.8251 -4.2454,8.4909 0.7276,3.7605 2.5475,-1.3343 7.1566,-6.6716 3.5175,-0.2424 3.8815,1.5769 3.8818,2.9109 1.9406,6.3077 11.4021,-0.7277 6.914,2.6686 5.5797,5.2157 4.0028,7.5206 0.9706,8.8546 -0.8493,10.3105 -2.1832,9.2185 -2.1836,2.9112 -3.0322,0.9706 -5.3373,-5.8224 -4.8518,-1.6982 -4.2455,7.0353 -4.2454,3.8815 -2.3049,1.4556 -9.2185,7.6419 -7.3993,4.0028 -7.3993,0.6066 -8.6119,-1.4556 -7.5206,-2.7899 -5.2158,-4.2454 -4.1241,-4.9734 -4.2454,-1.2129" |
4703 | - inkscape:connector-curvature="0" /> |
4704 | - </clipPath> |
4705 | - <linearGradient |
4706 | - id="linearGradient3850" |
4707 | - inkscape:collect="always"> |
4708 | - <stop |
4709 | - id="stop3852" |
4710 | - offset="0" |
4711 | - style="stop-color:#000000;stop-opacity:1;" /> |
4712 | - <stop |
4713 | - id="stop3854" |
4714 | - offset="1" |
4715 | - style="stop-color:#000000;stop-opacity:0;" /> |
4716 | - </linearGradient> |
4717 | - <clipPath |
4718 | - clipPathUnits="userSpaceOnUse" |
4719 | - id="clipPath3095"> |
4720 | - <path |
4721 | - d="M 976.648,389.551 H 134.246 V 1229.55 H 976.648 V 389.551" |
4722 | - id="path3097" |
4723 | - inkscape:connector-curvature="0" /> |
4724 | - </clipPath> |
4725 | - <clipPath |
4726 | - clipPathUnits="userSpaceOnUse" |
4727 | - id="clipPath3195"> |
4728 | - <path |
4729 | - d="m 611.836,756.738 -106.34,105.207 c -8.473,8.289 -13.617,20.102 -13.598,33.379 L 598.301,790.207 c -0.031,-13.418 5.094,-25.031 13.535,-33.469" |
4730 | - id="path3197" |
4731 | - inkscape:connector-curvature="0" /> |
4732 | - </clipPath> |
4733 | - <clipPath |
4734 | - clipPathUnits="userSpaceOnUse" |
4735 | - id="clipPath3235"> |
4736 | - <path |
4737 | - d="m 1095.64,1501.81 c 35.46,-35.07 70.89,-70.11 106.35,-105.17 4.4,-4.38 7.11,-10.53 7.11,-17.55 l -106.37,105.21 c 0,7 -2.71,13.11 -7.09,17.51" |
4738 | - id="path3237" |
4739 | - inkscape:connector-curvature="0" /> |
4740 | - </clipPath> |
4741 | - <clipPath |
4742 | - id="clipPath4591" |
4743 | - clipPathUnits="userSpaceOnUse"> |
4744 | - <path |
4745 | - inkscape:connector-curvature="0" |
4746 | - d="m 1106.6009,730.43734 -0.036,21.648 c -0.01,3.50825 -2.8675,6.61375 -6.4037,6.92525 l -83.6503,7.33162 c -3.5205,0.30763 -6.3812,-2.29987 -6.3671,-5.8145 l 0.036,-21.6475 20.1171,-1.76662 -0.011,4.63775 c 0,1.83937 1.4844,3.19925 3.3262,3.0395 l 49.5274,-4.33975 c 1.8425,-0.166 3.3425,-1.78125 3.3538,-3.626 l 0.01,-4.63025 20.1,-1.7575" |
4747 | - style="fill:#ff00ff;fill-opacity:1;fill-rule:nonzero;stroke:none" |
4748 | - id="path4593" /> |
4749 | - </clipPath> |
4750 | - <radialGradient |
4751 | - gradientUnits="userSpaceOnUse" |
4752 | - gradientTransform="matrix(-1.4333926,-2.2742838,1.1731823,-0.73941125,-174.08025,98.374394)" |
4753 | - r="20.40658" |
4754 | - fy="93.399292" |
4755 | - fx="-26.508606" |
4756 | - cy="93.399292" |
4757 | - cx="-26.508606" |
4758 | - id="radialGradient3856" |
4759 | - xlink:href="#linearGradient3850" |
4760 | - inkscape:collect="always" /> |
4761 | - <linearGradient |
4762 | - gradientTransform="translate(-318.48033,212.32022)" |
4763 | - gradientUnits="userSpaceOnUse" |
4764 | - y2="993.19702" |
4765 | - x2="-51.879555" |
4766 | - y1="593.11615" |
4767 | - x1="348.20132" |
4768 | - id="linearGradient3895" |
4769 | - xlink:href="#linearGradient3850" |
4770 | - inkscape:collect="always" /> |
4771 | - <clipPath |
4772 | - id="clipPath3906" |
4773 | - clipPathUnits="userSpaceOnUse"> |
4774 | - <rect |
4775 | - transform="scale(1,-1)" |
4776 | - style="color:#000000;display:inline;overflow:visible;visibility:visible;opacity:0.8;fill:#ff00ff;stroke:none;stroke-width:4;marker:none;enable-background:accumulate" |
4777 | - id="rect3908" |
4778 | - width="1019.1371" |
4779 | - height="1019.1371" |
4780 | - x="357.9816" |
4781 | - y="-1725.8152" /> |
4782 | - </clipPath> |
4783 | - <clipPath |
4784 | - clipPathUnits="userSpaceOnUse" |
4785 | - id="clipPath4637"> |
4786 | - <path |
4787 | - sodipodi:nodetypes="sssssssss" |
4788 | - inkscape:connector-curvature="0" |
4789 | - id="path4639" |
4790 | - d="M -268,700.15563 V 666.4259 c 0,-27.24324 3.88785,-31.13513 31.10302,-31.13513 h 33.79408 c 27.21507,0 31.1029,3.89189 31.1029,31.13513 v 33.72973 c 0,27.24325 -3.88783,31.13514 -31.1029,31.13514 h -33.79408 c -27.21517,0 -31.10302,-3.89189 -31.10302,-31.13514 z" |
4791 | - style="display:inline;fill:#df382c;fill-opacity:1;stroke:none" /> |
4792 | + <linearGradient |
4793 | + id="linearGradient3804"> |
4794 | + <stop |
4795 | + id="stop3806" |
4796 | + offset="0" |
4797 | + style="stop-color:#ffff00;stop-opacity:1;" /> |
4798 | + <stop |
4799 | + id="stop3808" |
4800 | + offset="1" |
4801 | + style="stop-color:#a8650a;stop-opacity:0;" /> |
4802 | + </linearGradient> |
4803 | + <linearGradient |
4804 | + id="linearGradient3778"> |
4805 | + <stop |
4806 | + id="stop3780" |
4807 | + offset="0" |
4808 | + style="stop-color:#ffff00;stop-opacity:0.84536082;" /> |
4809 | + <stop |
4810 | + id="stop3782" |
4811 | + offset="1" |
4812 | + style="stop-color:#a8650a;stop-opacity:0;" /> |
4813 | + </linearGradient> |
4814 | + <linearGradient |
4815 | + id="linearGradient3725"> |
4816 | + <stop |
4817 | + id="stop3727" |
4818 | + offset="0" |
4819 | + style="stop-color:#0000ff;stop-opacity:1;" /> |
4820 | + <stop |
4821 | + id="stop3729" |
4822 | + offset="1" |
4823 | + style="stop-color:#000000;stop-opacity:0" /> |
4824 | + </linearGradient> |
4825 | + <linearGradient |
4826 | + id="linearGradient3715"> |
4827 | + <stop |
4828 | + style="stop-color:#3700d0;stop-opacity:1;" |
4829 | + offset="0" |
4830 | + id="stop3717" /> |
4831 | + <stop |
4832 | + style="stop-color: rgb(0, 0, 0); stop-opacity: 0;" |
4833 | + offset="1" |
4834 | + id="stop3719" /> |
4835 | + </linearGradient> |
4836 | + <linearGradient |
4837 | + id="linearGradient3705"> |
4838 | + <stop |
4839 | + style="stop-color:#ff0000;stop-opacity:0.54639173;" |
4840 | + offset="0" |
4841 | + id="stop3707" /> |
4842 | + <stop |
4843 | + style="stop-color:#ff0067;stop-opacity:0.42268041;" |
4844 | + offset="1" |
4845 | + id="stop3709" /> |
4846 | + </linearGradient> |
4847 | + <linearGradient |
4848 | + id="linearGradient3699"> |
4849 | + <stop |
4850 | + style="stop-color:#d80000;stop-opacity:1;" |
4851 | + offset="0" |
4852 | + id="stop3701" /> |
4853 | + <stop |
4854 | + style="stop-color: rgb(136, 0, 170); stop-opacity: 0;" |
4855 | + offset="1" |
4856 | + id="stop3703" /> |
4857 | + </linearGradient> |
4858 | + <linearGradient |
4859 | + id="linearGradient3693"> |
4860 | + <stop |
4861 | + style="stop-color:#4700aa;stop-opacity:1;" |
4862 | + offset="0" |
4863 | + id="stop3695" /> |
4864 | + <stop |
4865 | + style="stop-color: rgb(136, 0, 170); stop-opacity: 0;" |
4866 | + offset="1" |
4867 | + id="stop3697" /> |
4868 | + </linearGradient> |
4869 | + <linearGradient |
4870 | + id="linearGradient3687"> |
4871 | + <stop |
4872 | + style="stop-color:#cd00ff;stop-opacity:1;" |
4873 | + offset="0" |
4874 | + id="stop3689" /> |
4875 | + <stop |
4876 | + style="stop-color:#ff00ff;stop-opacity:0;" |
4877 | + offset="1" |
4878 | + id="stop3691" /> |
4879 | + </linearGradient> |
4880 | + <linearGradient |
4881 | + id="linearGradient3681"> |
4882 | + <stop |
4883 | + style="stop-color:#ffff00;stop-opacity:1;" |
4884 | + offset="0" |
4885 | + id="stop3683" /> |
4886 | + <stop |
4887 | + style="stop-color:#a8650a;stop-opacity:0.45360824;" |
4888 | + offset="1" |
4889 | + id="stop3685" /> |
4890 | + </linearGradient> |
4891 | + <linearGradient |
4892 | + id="linearGradient3265" |
4893 | + inkscape:collect="always"> |
4894 | + <stop |
4895 | + id="stop3267" |
4896 | + offset="0" |
4897 | + style="stop-color: rgb(128, 101, 10); stop-opacity: 1;" /> |
4898 | + <stop |
4899 | + id="stop3269" |
4900 | + offset="1" |
4901 | + style="stop-color: rgb(128, 101, 10); stop-opacity: 0;" /> |
4902 | + </linearGradient> |
4903 | + <linearGradient |
4904 | + id="linearGradient3255" |
4905 | + inkscape:collect="always"> |
4906 | + <stop |
4907 | + id="stop3257" |
4908 | + offset="0" |
4909 | + style="stop-color: rgb(6, 46, 5); stop-opacity: 1;" /> |
4910 | + <stop |
4911 | + id="stop3259" |
4912 | + offset="1" |
4913 | + style="stop-color: rgb(6, 46, 5); stop-opacity: 0;" /> |
4914 | + </linearGradient> |
4915 | + <linearGradient |
4916 | + id="linearGradient3238" |
4917 | + inkscape:collect="always"> |
4918 | + <stop |
4919 | + id="stop3240" |
4920 | + offset="0" |
4921 | + style="stop-color: rgb(0, 0, 0); stop-opacity: 1;" /> |
4922 | + <stop |
4923 | + id="stop3242" |
4924 | + offset="1" |
4925 | + style="stop-color: rgb(0, 0, 0); stop-opacity: 0;" /> |
4926 | + </linearGradient> |
4927 | + <linearGradient |
4928 | + id="linearGradient3230"> |
4929 | + <stop |
4930 | + id="stop3232" |
4931 | + offset="0" |
4932 | + style="stop-color:#ea009b;stop-opacity:1;" /> |
4933 | + <stop |
4934 | + id="stop3234" |
4935 | + offset="1" |
4936 | + style="stop-color: rgb(209, 249, 208); stop-opacity: 0;" /> |
4937 | + </linearGradient> |
4938 | + <linearGradient |
4939 | + id="linearGradient3218" |
4940 | + inkscape:collect="always"> |
4941 | + <stop |
4942 | + id="stop3220" |
4943 | + offset="0" |
4944 | + style="stop-color: rgb(136, 0, 170); stop-opacity: 1;" /> |
4945 | + <stop |
4946 | + id="stop3222" |
4947 | + offset="1" |
4948 | + style="stop-color: rgb(136, 0, 170); stop-opacity: 0;" /> |
4949 | + </linearGradient> |
4950 | + <linearGradient |
4951 | + id="linearGradient3208" |
4952 | + inkscape:collect="always"> |
4953 | + <stop |
4954 | + id="stop3210" |
4955 | + offset="0" |
4956 | + style="stop-color: rgb(36, 216, 31); stop-opacity: 1;" /> |
4957 | + <stop |
4958 | + id="stop3212" |
4959 | + offset="1" |
4960 | + style="stop-color: rgb(36, 216, 31); stop-opacity: 0;" /> |
4961 | + </linearGradient> |
4962 | + <linearGradient |
4963 | + id="linearGradient3184"> |
4964 | + <stop |
4965 | + id="stop3186" |
4966 | + offset="0" |
4967 | + style="stop-color:#4200d2;stop-opacity:1;" /> |
4968 | + <stop |
4969 | + id="stop3188" |
4970 | + offset="1" |
4971 | + style="stop-color:#d42aff;stop-opacity:0;" /> |
4972 | + </linearGradient> |
4973 | + <linearGradient |
4974 | + id="linearGradient3174" |
4975 | + inkscape:collect="always"> |
4976 | + <stop |
4977 | + id="stop3176" |
4978 | + offset="0" |
4979 | + style="stop-color: rgb(0, 255, 0); stop-opacity: 1;" /> |
4980 | + <stop |
4981 | + id="stop3178" |
4982 | + offset="1" |
4983 | + style="stop-color: rgb(0, 255, 0); stop-opacity: 0;" /> |
4984 | + </linearGradient> |
4985 | + <linearGradient |
4986 | + id="linearGradient3166" |
4987 | + inkscape:collect="always"> |
4988 | + <stop |
4989 | + id="stop3168" |
4990 | + offset="0" |
4991 | + style="stop-color: rgb(0, 0, 0); stop-opacity: 1;" /> |
4992 | + <stop |
4993 | + id="stop3170" |
4994 | + offset="1" |
4995 | + style="stop-color: rgb(0, 0, 0); stop-opacity: 0;" /> |
4996 | + </linearGradient> |
4997 | + <inkscape:perspective |
4998 | + id="perspective10" |
4999 | + inkscape:persp3d-origin="372.04724 : 350.78739 : 1" |
5000 | + inkscape:vp_z="744.09448 : 526.18109 : 1" |
The diff has been truncated for viewing.
LGTM, +1