Merge lp:~bloodearnest/charm-haproxy/merge-trunk-snapstore into lp:~ubuntuone-pqm-team/charm-haproxy/snap-store

Proposed by Simon Davy
Status: Work in progress
Proposed branch: lp:~bloodearnest/charm-haproxy/merge-trunk-snapstore
Merge into: lp:~ubuntuone-pqm-team/charm-haproxy/snap-store
Diff against target: 5380 lines (+3150/-1126)
38 files modified
README.md (+6/-5)
charm-helpers.yaml (+2/-1)
config.yaml (+1/-1)
hooks/charmhelpers/__init__.py (+11/-13)
hooks/charmhelpers/contrib/__init__.py (+11/-13)
hooks/charmhelpers/contrib/charmsupport/__init__.py (+11/-13)
hooks/charmhelpers/contrib/charmsupport/nrpe.py (+106/-34)
hooks/charmhelpers/contrib/charmsupport/volumes.py (+11/-13)
hooks/charmhelpers/core/__init__.py (+11/-13)
hooks/charmhelpers/core/decorators.py (+11/-13)
hooks/charmhelpers/core/files.py (+43/-0)
hooks/charmhelpers/core/fstab.py (+11/-13)
hooks/charmhelpers/core/hookenv.py (+448/-63)
hooks/charmhelpers/core/host.py (+625/-155)
hooks/charmhelpers/core/host_factory/centos.py (+56/-0)
hooks/charmhelpers/core/host_factory/ubuntu.py (+56/-0)
hooks/charmhelpers/core/hugepage.py (+69/-0)
hooks/charmhelpers/core/kernel.py (+72/-0)
hooks/charmhelpers/core/kernel_factory/centos.py (+17/-0)
hooks/charmhelpers/core/kernel_factory/ubuntu.py (+13/-0)
hooks/charmhelpers/core/services/__init__.py (+11/-13)
hooks/charmhelpers/core/services/base.py (+52/-30)
hooks/charmhelpers/core/services/helpers.py (+42/-19)
hooks/charmhelpers/core/strutils.py (+41/-13)
hooks/charmhelpers/core/sysctl.py (+11/-13)
hooks/charmhelpers/core/templating.py (+40/-24)
hooks/charmhelpers/core/unitdata.py (+72/-31)
hooks/charmhelpers/fetch/__init__.py (+46/-296)
hooks/charmhelpers/fetch/archiveurl.py (+19/-15)
hooks/charmhelpers/fetch/bzrurl.py (+48/-50)
hooks/charmhelpers/fetch/centos.py (+171/-0)
hooks/charmhelpers/fetch/giturl.py (+37/-39)
hooks/charmhelpers/fetch/snap.py (+122/-0)
hooks/charmhelpers/fetch/ubuntu.py (+364/-0)
hooks/charmhelpers/osplatform.py (+25/-0)
hooks/hooks.py (+1/-1)
icon.svg (+456/-232)
metadata.yaml (+1/-0)
To merge this branch: bzr merge lp:~bloodearnest/charm-haproxy/merge-trunk-snapstore
Reviewer Review Type Date Requested Status
Guillermo Gonzalez Approve
haproxy-team Pending
Review via email: mp+347844@code.launchpad.net

Commit message

Merge trunk for autocert support

To post a comment you must log in.
Revision history for this message
Guillermo Gonzalez (verterok) wrote :

LGTM, +1

review: Approve

Unmerged revisions

113. By Simon Davy

merge trunk r114, with updated charm-helpers and autocert support

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'README.md'
--- README.md 2015-06-10 14:38:26 +0000
+++ README.md 2018-06-12 20:07:13 +0000
@@ -1,6 +1,7 @@
1# Overview1# Overview
22
3This charm deploys a reverse proxy in front of other servies. You can use this to load balance existing deployments.3This charm deploys a reverse proxy in front of other services. You can use
4this to load balance existing deployments.
45
5# Usage6# Usage
67
@@ -110,7 +111,7 @@
110charms written like apache2 that can act as a front-end for haproxy to take of111charms written like apache2 that can act as a front-end for haproxy to take of
111things like ssl encryption. When joining a service like apache2 on its112things like ssl encryption. When joining a service like apache2 on its
112reverseproxy relation, haproxy's website relation will set an `all_services`113reverseproxy relation, haproxy's website relation will set an `all_services`
113varaible that conforms to the spec layed out in the apache2 charm.114variable that conforms to the spec laid out in the apache2 charm.
114115
115These settings can then be used when crafting your vhost template to make sure116These settings can then be used when crafting your vhost template to make sure
116traffic goes to the correct haproxy listener which will in turn forward the117traffic goes to the correct haproxy listener which will in turn forward the
@@ -136,7 +137,7 @@
136 - { ... optionally more services here ... }137 - { ... optionally more services here ... }
137 "138 "
138139
139where the DEFAULT keyword means use the certificate set with `ssl_cert`/`ssl_key` (or140The DEFAULT keyword means use the certificate set with `ssl_cert`/`ssl_key` (or
140alternatively you can inline different base64-encode certificates).141alternatively you can inline different base64-encode certificates).
141142
142Note that in order to use SSL termination you need haproxy 1.5 or later, which143Note that in order to use SSL termination you need haproxy 1.5 or later, which
@@ -227,9 +228,9 @@
227### active-active mode228### active-active mode
228229
229If the peering\_mode option is set to "active-active", then any haproxy unit230If the peering\_mode option is set to "active-active", then any haproxy unit
230will be independant from each other and will simply load-balance the traffic to231will be independent from each other and will simply load-balance the traffic to
231the backends. In this case, the indirection layer described above is not232the backends. In this case, the indirection layer described above is not
232created in this case.233created.
233234
234This mode allows increasing the bandwidth to the backends by adding additional235This mode allows increasing the bandwidth to the backends by adding additional
235units, at the cost of having less control over the number of connections that236units, at the cost of having less control over the number of connections that
236237
=== modified file 'charm-helpers.yaml'
--- charm-helpers.yaml 2013-08-21 19:19:29 +0000
+++ charm-helpers.yaml 2018-06-12 20:07:13 +0000
@@ -1,4 +1,5 @@
1include:1include:
2 - core2 - core
3 - fetch3 - fetch
4 - contrib.charmsupport
5\ No newline at end of file4\ No newline at end of file
5 - contrib.charmsupport
6 - osplatform
67
=== modified file 'config.yaml'
--- config.yaml 2016-11-28 04:37:08 +0000
+++ config.yaml 2018-06-12 20:07:13 +0000
@@ -49,7 +49,7 @@
49 greater than 1024 bits are not supported by Java 7 and earlier clients. This49 greater than 1024 bits are not supported by Java 7 and earlier clients. This
50 config key will be ignored if the installed haproxy package has no SSL support.50 config key will be ignored if the installed haproxy package has no SSL support.
51 global_default_bind_ciphers:51 global_default_bind_ciphers:
52 default: ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA52 default: ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:!DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:!DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:!CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA
53 type: string53 type: string
54 description: |54 description: |
55 Sets the default string describing the list of cipher algorithms55 Sets the default string describing the list of cipher algorithms
5656
=== added file 'hooks/__init__.py'
=== modified file 'hooks/charmhelpers/__init__.py'
--- hooks/charmhelpers/__init__.py 2015-02-09 12:53:57 +0000
+++ hooks/charmhelpers/__init__.py 2018-06-12 20:07:13 +0000
@@ -1,18 +1,16 @@
1# Copyright 2014-2015 Canonical Limited.1# Copyright 2014-2015 Canonical Limited.
2#2#
3# This file is part of charm-helpers.3# Licensed under the Apache License, Version 2.0 (the "License");
4#4# you may not use this file except in compliance with the License.
5# charm-helpers is free software: you can redistribute it and/or modify5# You may obtain a copy of the License at
6# it under the terms of the GNU Lesser General Public License version 3 as6#
7# published by the Free Software Foundation.7# http://www.apache.org/licenses/LICENSE-2.0
8#8#
9# charm-helpers is distributed in the hope that it will be useful,9# Unless required by applicable law or agreed to in writing, software
10# but WITHOUT ANY WARRANTY; without even the implied warranty of10# distributed under the License is distributed on an "AS IS" BASIS,
11# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the11# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12# GNU Lesser General Public License for more details.12# See the License for the specific language governing permissions and
13#13# limitations under the License.
14# You should have received a copy of the GNU Lesser General Public License
15# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1614
17# Bootstrap charm-helpers, installing its dependencies if necessary using15# Bootstrap charm-helpers, installing its dependencies if necessary using
18# only standard libraries.16# only standard libraries.
1917
=== modified file 'hooks/charmhelpers/contrib/__init__.py'
--- hooks/charmhelpers/contrib/__init__.py 2015-02-09 12:53:57 +0000
+++ hooks/charmhelpers/contrib/__init__.py 2018-06-12 20:07:13 +0000
@@ -1,15 +1,13 @@
1# Copyright 2014-2015 Canonical Limited.1# Copyright 2014-2015 Canonical Limited.
2#2#
3# This file is part of charm-helpers.3# Licensed under the Apache License, Version 2.0 (the "License");
4#4# you may not use this file except in compliance with the License.
5# charm-helpers is free software: you can redistribute it and/or modify5# You may obtain a copy of the License at
6# it under the terms of the GNU Lesser General Public License version 3 as6#
7# published by the Free Software Foundation.7# http://www.apache.org/licenses/LICENSE-2.0
8#8#
9# charm-helpers is distributed in the hope that it will be useful,9# Unless required by applicable law or agreed to in writing, software
10# but WITHOUT ANY WARRANTY; without even the implied warranty of10# distributed under the License is distributed on an "AS IS" BASIS,
11# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the11# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12# GNU Lesser General Public License for more details.12# See the License for the specific language governing permissions and
13#13# limitations under the License.
14# You should have received a copy of the GNU Lesser General Public License
15# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1614
=== modified file 'hooks/charmhelpers/contrib/charmsupport/__init__.py'
--- hooks/charmhelpers/contrib/charmsupport/__init__.py 2015-02-09 12:53:57 +0000
+++ hooks/charmhelpers/contrib/charmsupport/__init__.py 2018-06-12 20:07:13 +0000
@@ -1,15 +1,13 @@
1# Copyright 2014-2015 Canonical Limited.1# Copyright 2014-2015 Canonical Limited.
2#2#
3# This file is part of charm-helpers.3# Licensed under the Apache License, Version 2.0 (the "License");
4#4# you may not use this file except in compliance with the License.
5# charm-helpers is free software: you can redistribute it and/or modify5# You may obtain a copy of the License at
6# it under the terms of the GNU Lesser General Public License version 3 as6#
7# published by the Free Software Foundation.7# http://www.apache.org/licenses/LICENSE-2.0
8#8#
9# charm-helpers is distributed in the hope that it will be useful,9# Unless required by applicable law or agreed to in writing, software
10# but WITHOUT ANY WARRANTY; without even the implied warranty of10# distributed under the License is distributed on an "AS IS" BASIS,
11# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the11# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12# GNU Lesser General Public License for more details.12# See the License for the specific language governing permissions and
13#13# limitations under the License.
14# You should have received a copy of the GNU Lesser General Public License
15# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1614
=== modified file 'hooks/charmhelpers/contrib/charmsupport/nrpe.py'
--- hooks/charmhelpers/contrib/charmsupport/nrpe.py 2015-05-14 10:48:09 +0000
+++ hooks/charmhelpers/contrib/charmsupport/nrpe.py 2018-06-12 20:07:13 +0000
@@ -1,18 +1,16 @@
1# Copyright 2014-2015 Canonical Limited.1# Copyright 2014-2015 Canonical Limited.
2#2#
3# This file is part of charm-helpers.3# Licensed under the Apache License, Version 2.0 (the "License");
4#4# you may not use this file except in compliance with the License.
5# charm-helpers is free software: you can redistribute it and/or modify5# You may obtain a copy of the License at
6# it under the terms of the GNU Lesser General Public License version 3 as6#
7# published by the Free Software Foundation.7# http://www.apache.org/licenses/LICENSE-2.0
8#8#
9# charm-helpers is distributed in the hope that it will be useful,9# Unless required by applicable law or agreed to in writing, software
10# but WITHOUT ANY WARRANTY; without even the implied warranty of10# distributed under the License is distributed on an "AS IS" BASIS,
11# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the11# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12# GNU Lesser General Public License for more details.12# See the License for the specific language governing permissions and
13#13# limitations under the License.
14# You should have received a copy of the GNU Lesser General Public License
15# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1614
17"""Compatibility with the nrpe-external-master charm"""15"""Compatibility with the nrpe-external-master charm"""
18# Copyright 2012 Canonical Ltd.16# Copyright 2012 Canonical Ltd.
@@ -40,6 +38,7 @@
40)38)
4139
42from charmhelpers.core.host import service40from charmhelpers.core.host import service
41from charmhelpers.core import host
4342
44# This module adds compatibility with the nrpe-external-master and plain nrpe43# This module adds compatibility with the nrpe-external-master and plain nrpe
45# subordinate charms. To use it in your charm:44# subordinate charms. To use it in your charm:
@@ -110,6 +109,13 @@
110# def local_monitors_relation_changed():109# def local_monitors_relation_changed():
111# update_nrpe_config()110# update_nrpe_config()
112#111#
112# 4.a If your charm is a subordinate charm set primary=False
113#
114# from charmsupport.nrpe import NRPE
115# (...)
116# def update_nrpe_config():
117# nrpe_compat = NRPE(primary=False)
118#
113# 5. ln -s hooks.py nrpe-external-master-relation-changed119# 5. ln -s hooks.py nrpe-external-master-relation-changed
114# ln -s hooks.py local-monitors-relation-changed120# ln -s hooks.py local-monitors-relation-changed
115121
@@ -148,6 +154,13 @@
148 self.description = description154 self.description = description
149 self.check_cmd = self._locate_cmd(check_cmd)155 self.check_cmd = self._locate_cmd(check_cmd)
150156
157 def _get_check_filename(self):
158 return os.path.join(NRPE.nrpe_confdir, '{}.cfg'.format(self.command))
159
160 def _get_service_filename(self, hostname):
161 return os.path.join(NRPE.nagios_exportdir,
162 'service__{}_{}.cfg'.format(hostname, self.command))
163
151 def _locate_cmd(self, check_cmd):164 def _locate_cmd(self, check_cmd):
152 search_path = (165 search_path = (
153 '/usr/lib/nagios/plugins',166 '/usr/lib/nagios/plugins',
@@ -163,9 +176,21 @@
163 log('Check command not found: {}'.format(parts[0]))176 log('Check command not found: {}'.format(parts[0]))
164 return ''177 return ''
165178
179 def _remove_service_files(self):
180 if not os.path.exists(NRPE.nagios_exportdir):
181 return
182 for f in os.listdir(NRPE.nagios_exportdir):
183 if f.endswith('_{}.cfg'.format(self.command)):
184 os.remove(os.path.join(NRPE.nagios_exportdir, f))
185
186 def remove(self, hostname):
187 nrpe_check_file = self._get_check_filename()
188 if os.path.exists(nrpe_check_file):
189 os.remove(nrpe_check_file)
190 self._remove_service_files()
191
166 def write(self, nagios_context, hostname, nagios_servicegroups):192 def write(self, nagios_context, hostname, nagios_servicegroups):
167 nrpe_check_file = '/etc/nagios/nrpe.d/{}.cfg'.format(193 nrpe_check_file = self._get_check_filename()
168 self.command)
169 with open(nrpe_check_file, 'w') as nrpe_check_config:194 with open(nrpe_check_file, 'w') as nrpe_check_config:
170 nrpe_check_config.write("# check {}\n".format(self.shortname))195 nrpe_check_config.write("# check {}\n".format(self.shortname))
171 nrpe_check_config.write("command[{}]={}\n".format(196 nrpe_check_config.write("command[{}]={}\n".format(
@@ -180,9 +205,7 @@
180205
181 def write_service_config(self, nagios_context, hostname,206 def write_service_config(self, nagios_context, hostname,
182 nagios_servicegroups):207 nagios_servicegroups):
183 for f in os.listdir(NRPE.nagios_exportdir):208 self._remove_service_files()
184 if re.search('.*{}.cfg'.format(self.command), f):
185 os.remove(os.path.join(NRPE.nagios_exportdir, f))
186209
187 templ_vars = {210 templ_vars = {
188 'nagios_hostname': hostname,211 'nagios_hostname': hostname,
@@ -192,8 +215,7 @@
192 'command': self.command,215 'command': self.command,
193 }216 }
194 nrpe_service_text = Check.service_template.format(**templ_vars)217 nrpe_service_text = Check.service_template.format(**templ_vars)
195 nrpe_service_file = '{}/service__{}_{}.cfg'.format(218 nrpe_service_file = self._get_service_filename(hostname)
196 NRPE.nagios_exportdir, hostname, self.command)
197 with open(nrpe_service_file, 'w') as nrpe_service_config:219 with open(nrpe_service_file, 'w') as nrpe_service_config:
198 nrpe_service_config.write(str(nrpe_service_text))220 nrpe_service_config.write(str(nrpe_service_text))
199221
@@ -205,10 +227,12 @@
205 nagios_logdir = '/var/log/nagios'227 nagios_logdir = '/var/log/nagios'
206 nagios_exportdir = '/var/lib/nagios/export'228 nagios_exportdir = '/var/lib/nagios/export'
207 nrpe_confdir = '/etc/nagios/nrpe.d'229 nrpe_confdir = '/etc/nagios/nrpe.d'
230 homedir = '/var/lib/nagios' # home dir provided by nagios-nrpe-server
208231
209 def __init__(self, hostname=None):232 def __init__(self, hostname=None, primary=True):
210 super(NRPE, self).__init__()233 super(NRPE, self).__init__()
211 self.config = config()234 self.config = config()
235 self.primary = primary
212 self.nagios_context = self.config['nagios_context']236 self.nagios_context = self.config['nagios_context']
213 if 'nagios_servicegroups' in self.config and self.config['nagios_servicegroups']:237 if 'nagios_servicegroups' in self.config and self.config['nagios_servicegroups']:
214 self.nagios_servicegroups = self.config['nagios_servicegroups']238 self.nagios_servicegroups = self.config['nagios_servicegroups']
@@ -218,12 +242,38 @@
218 if hostname:242 if hostname:
219 self.hostname = hostname243 self.hostname = hostname
220 else:244 else:
221 self.hostname = "{}-{}".format(self.nagios_context, self.unit_name)245 nagios_hostname = get_nagios_hostname()
246 if nagios_hostname:
247 self.hostname = nagios_hostname
248 else:
249 self.hostname = "{}-{}".format(self.nagios_context, self.unit_name)
222 self.checks = []250 self.checks = []
251 # Iff in an nrpe-external-master relation hook, set primary status
252 relation = relation_ids('nrpe-external-master')
253 if relation:
254 log("Setting charm primary status {}".format(primary))
255 for rid in relation_ids('nrpe-external-master'):
256 relation_set(relation_id=rid, relation_settings={'primary': self.primary})
223257
224 def add_check(self, *args, **kwargs):258 def add_check(self, *args, **kwargs):
225 self.checks.append(Check(*args, **kwargs))259 self.checks.append(Check(*args, **kwargs))
226260
261 def remove_check(self, *args, **kwargs):
262 if kwargs.get('shortname') is None:
263 raise ValueError('shortname of check must be specified')
264
265 # Use sensible defaults if they're not specified - these are not
266 # actually used during removal, but they're required for constructing
267 # the Check object; check_disk is chosen because it's part of the
268 # nagios-plugins-basic package.
269 if kwargs.get('check_cmd') is None:
270 kwargs['check_cmd'] = 'check_disk'
271 if kwargs.get('description') is None:
272 kwargs['description'] = ''
273
274 check = Check(*args, **kwargs)
275 check.remove(self.hostname)
276
227 def write(self):277 def write(self):
228 try:278 try:
229 nagios_uid = pwd.getpwnam('nagios').pw_uid279 nagios_uid = pwd.getpwnam('nagios').pw_uid
@@ -260,7 +310,7 @@
260 :param str relation_name: Name of relation nrpe sub joined to310 :param str relation_name: Name of relation nrpe sub joined to
261 """311 """
262 for rel in relations_of_type(relation_name):312 for rel in relations_of_type(relation_name):
263 if 'nagios_hostname' in rel:313 if 'nagios_host_context' in rel:
264 return rel['nagios_host_context']314 return rel['nagios_host_context']
265315
266316
@@ -289,18 +339,30 @@
289 return unit339 return unit
290340
291341
292def add_init_service_checks(nrpe, services, unit_name):342def add_init_service_checks(nrpe, services, unit_name, immediate_check=True):
293 """343 """
294 Add checks for each service in list344 Add checks for each service in list
295345
296 :param NRPE nrpe: NRPE object to add check to346 :param NRPE nrpe: NRPE object to add check to
297 :param list services: List of services to check347 :param list services: List of services to check
298 :param str unit_name: Unit name to use in check description348 :param str unit_name: Unit name to use in check description
349 :param bool immediate_check: For sysv init, run the service check immediately
299 """350 """
300 for svc in services:351 for svc in services:
352 # Don't add a check for these services from neutron-gateway
353 if svc in ['ext-port', 'os-charm-phy-nic-mtu']:
354 next
355
301 upstart_init = '/etc/init/%s.conf' % svc356 upstart_init = '/etc/init/%s.conf' % svc
302 sysv_init = '/etc/init.d/%s' % svc357 sysv_init = '/etc/init.d/%s' % svc
303 if os.path.exists(upstart_init):358
359 if host.init_is_systemd():
360 nrpe.add_check(
361 shortname=svc,
362 description='process check {%s}' % unit_name,
363 check_cmd='check_systemd.py %s' % svc
364 )
365 elif os.path.exists(upstart_init):
304 nrpe.add_check(366 nrpe.add_check(
305 shortname=svc,367 shortname=svc,
306 description='process check {%s}' % unit_name,368 description='process check {%s}' % unit_name,
@@ -308,21 +370,31 @@
308 )370 )
309 elif os.path.exists(sysv_init):371 elif os.path.exists(sysv_init):
310 cronpath = '/etc/cron.d/nagios-service-check-%s' % svc372 cronpath = '/etc/cron.d/nagios-service-check-%s' % svc
311 cron_file = ('*/5 * * * * root '373 checkpath = '%s/service-check-%s.txt' % (nrpe.homedir, svc)
312 '/usr/local/lib/nagios/plugins/check_exit_status.pl '374 croncmd = (
313 '-s /etc/init.d/%s status > '375 '/usr/local/lib/nagios/plugins/check_exit_status.pl '
314 '/var/lib/nagios/service-check-%s.txt\n' % (svc,376 '-s /etc/init.d/%s status' % svc
315 svc)377 )
316 )378 cron_file = '*/5 * * * * root %s > %s\n' % (croncmd, checkpath)
317 f = open(cronpath, 'w')379 f = open(cronpath, 'w')
318 f.write(cron_file)380 f.write(cron_file)
319 f.close()381 f.close()
320 nrpe.add_check(382 nrpe.add_check(
321 shortname=svc,383 shortname=svc,
322 description='process check {%s}' % unit_name,384 description='service check {%s}' % unit_name,
323 check_cmd='check_status_file.py -f '385 check_cmd='check_status_file.py -f %s' % checkpath,
324 '/var/lib/nagios/service-check-%s.txt' % svc,
325 )386 )
387 # if /var/lib/nagios doesn't exist open(checkpath, 'w') will fail
388 # (LP: #1670223).
389 if immediate_check and os.path.isdir(nrpe.homedir):
390 f = open(checkpath, 'w')
391 subprocess.call(
392 croncmd.split(),
393 stdout=f,
394 stderr=subprocess.STDOUT
395 )
396 f.close()
397 os.chmod(checkpath, 0o644)
326398
327399
328def copy_nrpe_checks():400def copy_nrpe_checks():
329401
=== modified file 'hooks/charmhelpers/contrib/charmsupport/volumes.py'
--- hooks/charmhelpers/contrib/charmsupport/volumes.py 2015-02-09 12:53:57 +0000
+++ hooks/charmhelpers/contrib/charmsupport/volumes.py 2018-06-12 20:07:13 +0000
@@ -1,18 +1,16 @@
1# Copyright 2014-2015 Canonical Limited.1# Copyright 2014-2015 Canonical Limited.
2#2#
3# This file is part of charm-helpers.3# Licensed under the Apache License, Version 2.0 (the "License");
4#4# you may not use this file except in compliance with the License.
5# charm-helpers is free software: you can redistribute it and/or modify5# You may obtain a copy of the License at
6# it under the terms of the GNU Lesser General Public License version 3 as6#
7# published by the Free Software Foundation.7# http://www.apache.org/licenses/LICENSE-2.0
8#8#
9# charm-helpers is distributed in the hope that it will be useful,9# Unless required by applicable law or agreed to in writing, software
10# but WITHOUT ANY WARRANTY; without even the implied warranty of10# distributed under the License is distributed on an "AS IS" BASIS,
11# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the11# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12# GNU Lesser General Public License for more details.12# See the License for the specific language governing permissions and
13#13# limitations under the License.
14# You should have received a copy of the GNU Lesser General Public License
15# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1614
17'''15'''
18Functions for managing volumes in juju units. One volume is supported per unit.16Functions for managing volumes in juju units. One volume is supported per unit.
1917
=== modified file 'hooks/charmhelpers/core/__init__.py'
--- hooks/charmhelpers/core/__init__.py 2015-02-09 12:53:57 +0000
+++ hooks/charmhelpers/core/__init__.py 2018-06-12 20:07:13 +0000
@@ -1,15 +1,13 @@
1# Copyright 2014-2015 Canonical Limited.1# Copyright 2014-2015 Canonical Limited.
2#2#
3# This file is part of charm-helpers.3# Licensed under the Apache License, Version 2.0 (the "License");
4#4# you may not use this file except in compliance with the License.
5# charm-helpers is free software: you can redistribute it and/or modify5# You may obtain a copy of the License at
6# it under the terms of the GNU Lesser General Public License version 3 as6#
7# published by the Free Software Foundation.7# http://www.apache.org/licenses/LICENSE-2.0
8#8#
9# charm-helpers is distributed in the hope that it will be useful,9# Unless required by applicable law or agreed to in writing, software
10# but WITHOUT ANY WARRANTY; without even the implied warranty of10# distributed under the License is distributed on an "AS IS" BASIS,
11# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the11# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12# GNU Lesser General Public License for more details.12# See the License for the specific language governing permissions and
13#13# limitations under the License.
14# You should have received a copy of the GNU Lesser General Public License
15# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1614
=== modified file 'hooks/charmhelpers/core/decorators.py'
--- hooks/charmhelpers/core/decorators.py 2015-02-09 12:58:07 +0000
+++ hooks/charmhelpers/core/decorators.py 2018-06-12 20:07:13 +0000
@@ -1,18 +1,16 @@
1# Copyright 2014-2015 Canonical Limited.1# Copyright 2014-2015 Canonical Limited.
2#2#
3# This file is part of charm-helpers.3# Licensed under the Apache License, Version 2.0 (the "License");
4#4# you may not use this file except in compliance with the License.
5# charm-helpers is free software: you can redistribute it and/or modify5# You may obtain a copy of the License at
6# it under the terms of the GNU Lesser General Public License version 3 as6#
7# published by the Free Software Foundation.7# http://www.apache.org/licenses/LICENSE-2.0
8#8#
9# charm-helpers is distributed in the hope that it will be useful,9# Unless required by applicable law or agreed to in writing, software
10# but WITHOUT ANY WARRANTY; without even the implied warranty of10# distributed under the License is distributed on an "AS IS" BASIS,
11# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the11# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12# GNU Lesser General Public License for more details.12# See the License for the specific language governing permissions and
13#13# limitations under the License.
14# You should have received a copy of the GNU Lesser General Public License
15# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1614
17#15#
18# Copyright 2014 Canonical Ltd.16# Copyright 2014 Canonical Ltd.
1917
=== added file 'hooks/charmhelpers/core/files.py'
--- hooks/charmhelpers/core/files.py 1970-01-01 00:00:00 +0000
+++ hooks/charmhelpers/core/files.py 2018-06-12 20:07:13 +0000
@@ -0,0 +1,43 @@
1#!/usr/bin/env python
2# -*- coding: utf-8 -*-
3
4# Copyright 2014-2015 Canonical Limited.
5#
6# Licensed under the Apache License, Version 2.0 (the "License");
7# you may not use this file except in compliance with the License.
8# You may obtain a copy of the License at
9#
10# http://www.apache.org/licenses/LICENSE-2.0
11#
12# Unless required by applicable law or agreed to in writing, software
13# distributed under the License is distributed on an "AS IS" BASIS,
14# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
15# See the License for the specific language governing permissions and
16# limitations under the License.
17
18__author__ = 'Jorge Niedbalski <niedbalski@ubuntu.com>'
19
20import os
21import subprocess
22
23
24def sed(filename, before, after, flags='g'):
25 """
26 Search and replaces the given pattern on filename.
27
28 :param filename: relative or absolute file path.
29 :param before: expression to be replaced (see 'man sed')
30 :param after: expression to replace with (see 'man sed')
31 :param flags: sed-compatible regex flags in example, to make
32 the search and replace case insensitive, specify ``flags="i"``.
33 The ``g`` flag is always specified regardless, so you do not
34 need to remember to include it when overriding this parameter.
35 :returns: If the sed command exit code was zero then return,
36 otherwise raise CalledProcessError.
37 """
38 expression = r's/{0}/{1}/{2}'.format(before,
39 after, flags)
40
41 return subprocess.check_call(["sed", "-i", "-r", "-e",
42 expression,
43 os.path.expanduser(filename)])
044
=== modified file 'hooks/charmhelpers/core/fstab.py'
--- hooks/charmhelpers/core/fstab.py 2015-05-14 10:48:09 +0000
+++ hooks/charmhelpers/core/fstab.py 2018-06-12 20:07:13 +0000
@@ -3,19 +3,17 @@
33
4# Copyright 2014-2015 Canonical Limited.4# Copyright 2014-2015 Canonical Limited.
5#5#
6# This file is part of charm-helpers.6# Licensed under the Apache License, Version 2.0 (the "License");
7#7# you may not use this file except in compliance with the License.
8# charm-helpers is free software: you can redistribute it and/or modify8# You may obtain a copy of the License at
9# it under the terms of the GNU Lesser General Public License version 3 as9#
10# published by the Free Software Foundation.10# http://www.apache.org/licenses/LICENSE-2.0
11#11#
12# charm-helpers is distributed in the hope that it will be useful,12# Unless required by applicable law or agreed to in writing, software
13# but WITHOUT ANY WARRANTY; without even the implied warranty of13# distributed under the License is distributed on an "AS IS" BASIS,
14# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the14# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
15# GNU Lesser General Public License for more details.15# See the License for the specific language governing permissions and
16#16# limitations under the License.
17# You should have received a copy of the GNU Lesser General Public License
18# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1917
20import io18import io
21import os19import os
2220
=== modified file 'hooks/charmhelpers/core/hookenv.py'
--- hooks/charmhelpers/core/hookenv.py 2015-05-14 10:48:09 +0000
+++ hooks/charmhelpers/core/hookenv.py 2018-06-12 20:07:13 +0000
@@ -1,18 +1,16 @@
1# Copyright 2014-2015 Canonical Limited.1# Copyright 2014-2015 Canonical Limited.
2#2#
3# This file is part of charm-helpers.3# Licensed under the Apache License, Version 2.0 (the "License");
4#4# you may not use this file except in compliance with the License.
5# charm-helpers is free software: you can redistribute it and/or modify5# You may obtain a copy of the License at
6# it under the terms of the GNU Lesser General Public License version 3 as6#
7# published by the Free Software Foundation.7# http://www.apache.org/licenses/LICENSE-2.0
8#8#
9# charm-helpers is distributed in the hope that it will be useful,9# Unless required by applicable law or agreed to in writing, software
10# but WITHOUT ANY WARRANTY; without even the implied warranty of10# distributed under the License is distributed on an "AS IS" BASIS,
11# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the11# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12# GNU Lesser General Public License for more details.12# See the License for the specific language governing permissions and
13#13# limitations under the License.
14# You should have received a copy of the GNU Lesser General Public License
15# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1614
17"Interactions with the Juju environment"15"Interactions with the Juju environment"
18# Copyright 2013 Canonical Ltd.16# Copyright 2013 Canonical Ltd.
@@ -21,7 +19,10 @@
21# Charm Helpers Developers <juju@lists.ubuntu.com>19# Charm Helpers Developers <juju@lists.ubuntu.com>
2220
23from __future__ import print_function21from __future__ import print_function
22import copy
23from distutils.version import LooseVersion
24from functools import wraps24from functools import wraps
25import glob
25import os26import os
26import json27import json
27import yaml28import yaml
@@ -71,6 +72,7 @@
71 res = func(*args, **kwargs)72 res = func(*args, **kwargs)
72 cache[key] = res73 cache[key] = res
73 return res74 return res
75 wrapper._wrapped = func
74 return wrapper76 return wrapper
7577
7678
@@ -170,9 +172,19 @@
170 return os.environ.get('JUJU_RELATION', None)172 return os.environ.get('JUJU_RELATION', None)
171173
172174
173def relation_id():175@cached
174 """The relation ID for the current relation hook"""176def relation_id(relation_name=None, service_or_unit=None):
175 return os.environ.get('JUJU_RELATION_ID', None)177 """The relation ID for the current or a specified relation"""
178 if not relation_name and not service_or_unit:
179 return os.environ.get('JUJU_RELATION_ID', None)
180 elif relation_name and service_or_unit:
181 service_name = service_or_unit.split('/')[0]
182 for relid in relation_ids(relation_name):
183 remote_service = remote_service_name(relid)
184 if remote_service == service_name:
185 return relid
186 else:
187 raise ValueError('Must specify neither or both of relation_name and service_or_unit')
176188
177189
178def local_unit():190def local_unit():
@@ -190,9 +202,20 @@
190 return local_unit().split('/')[0]202 return local_unit().split('/')[0]
191203
192204
205@cached
206def remote_service_name(relid=None):
207 """The remote service name for a given relation-id (or the current relation)"""
208 if relid is None:
209 unit = remote_unit()
210 else:
211 units = related_units(relid)
212 unit = units[0] if units else None
213 return unit.split('/')[0] if unit else None
214
215
193def hook_name():216def hook_name():
194 """The name of the currently executing hook"""217 """The name of the currently executing hook"""
195 return os.path.basename(sys.argv[0])218 return os.environ.get('JUJU_HOOK_NAME', os.path.basename(sys.argv[0]))
196219
197220
198class Config(dict):221class Config(dict):
@@ -242,29 +265,7 @@
242 self.path = os.path.join(charm_dir(), Config.CONFIG_FILE_NAME)265 self.path = os.path.join(charm_dir(), Config.CONFIG_FILE_NAME)
243 if os.path.exists(self.path):266 if os.path.exists(self.path):
244 self.load_previous()267 self.load_previous()
245268 atexit(self._implicit_save)
246 def __getitem__(self, key):
247 """For regular dict lookups, check the current juju config first,
248 then the previous (saved) copy. This ensures that user-saved values
249 will be returned by a dict lookup.
250
251 """
252 try:
253 return dict.__getitem__(self, key)
254 except KeyError:
255 return (self._prev_dict or {})[key]
256
257 def get(self, key, default=None):
258 try:
259 return self[key]
260 except KeyError:
261 return default
262
263 def keys(self):
264 prev_keys = []
265 if self._prev_dict is not None:
266 prev_keys = self._prev_dict.keys()
267 return list(set(prev_keys + list(dict.keys(self))))
268269
269 def load_previous(self, path=None):270 def load_previous(self, path=None):
270 """Load previous copy of config from disk.271 """Load previous copy of config from disk.
@@ -283,6 +284,9 @@
283 self.path = path or self.path284 self.path = path or self.path
284 with open(self.path) as f:285 with open(self.path) as f:
285 self._prev_dict = json.load(f)286 self._prev_dict = json.load(f)
287 for k, v in copy.deepcopy(self._prev_dict).items():
288 if k not in self:
289 self[k] = v
286290
287 def changed(self, key):291 def changed(self, key):
288 """Return True if the current value for this key is different from292 """Return True if the current value for this key is different from
@@ -314,13 +318,13 @@
314 instance.318 instance.
315319
316 """320 """
317 if self._prev_dict:
318 for k, v in six.iteritems(self._prev_dict):
319 if k not in self:
320 self[k] = v
321 with open(self.path, 'w') as f:321 with open(self.path, 'w') as f:
322 json.dump(self, f)322 json.dump(self, f)
323323
324 def _implicit_save(self):
325 if self.implicit_save:
326 self.save()
327
324328
325@cached329@cached
326def config(scope=None):330def config(scope=None):
@@ -328,6 +332,8 @@
328 config_cmd_line = ['config-get']332 config_cmd_line = ['config-get']
329 if scope is not None:333 if scope is not None:
330 config_cmd_line.append(scope)334 config_cmd_line.append(scope)
335 else:
336 config_cmd_line.append('--all')
331 config_cmd_line.append('--format=json')337 config_cmd_line.append('--format=json')
332 try:338 try:
333 config_data = json.loads(339 config_data = json.loads(
@@ -364,11 +370,16 @@
364 relation_settings = relation_settings if relation_settings else {}370 relation_settings = relation_settings if relation_settings else {}
365 relation_cmd_line = ['relation-set']371 relation_cmd_line = ['relation-set']
366 accepts_file = "--file" in subprocess.check_output(372 accepts_file = "--file" in subprocess.check_output(
367 relation_cmd_line + ["--help"])373 relation_cmd_line + ["--help"], universal_newlines=True)
368 if relation_id is not None:374 if relation_id is not None:
369 relation_cmd_line.extend(('-r', relation_id))375 relation_cmd_line.extend(('-r', relation_id))
370 settings = relation_settings.copy()376 settings = relation_settings.copy()
371 settings.update(kwargs)377 settings.update(kwargs)
378 for key, value in settings.items():
379 # Force value to be a string: it always should, but some call
380 # sites pass in things like dicts or numbers.
381 if value is not None:
382 settings[key] = "{}".format(value)
372 if accepts_file:383 if accepts_file:
373 # --file was introduced in Juju 1.23.2. Use it by default if384 # --file was introduced in Juju 1.23.2. Use it by default if
374 # available, since otherwise we'll break if the relation data is385 # available, since otherwise we'll break if the relation data is
@@ -390,6 +401,17 @@
390 flush(local_unit())401 flush(local_unit())
391402
392403
404def relation_clear(r_id=None):
405 ''' Clears any relation data already set on relation r_id '''
406 settings = relation_get(rid=r_id,
407 unit=local_unit())
408 for setting in settings:
409 if setting not in ['public-address', 'private-address']:
410 settings[setting] = None
411 relation_set(relation_id=r_id,
412 **settings)
413
414
393@cached415@cached
394def relation_ids(reltype=None):416def relation_ids(reltype=None):
395 """A list of relation_ids"""417 """A list of relation_ids"""
@@ -469,6 +491,76 @@
469491
470492
471@cached493@cached
494def peer_relation_id():
495 '''Get the peers relation id if a peers relation has been joined, else None.'''
496 md = metadata()
497 section = md.get('peers')
498 if section:
499 for key in section:
500 relids = relation_ids(key)
501 if relids:
502 return relids[0]
503 return None
504
505
506@cached
507def relation_to_interface(relation_name):
508 """
509 Given the name of a relation, return the interface that relation uses.
510
511 :returns: The interface name, or ``None``.
512 """
513 return relation_to_role_and_interface(relation_name)[1]
514
515
516@cached
517def relation_to_role_and_interface(relation_name):
518 """
519 Given the name of a relation, return the role and the name of the interface
520 that relation uses (where role is one of ``provides``, ``requires``, or ``peers``).
521
522 :returns: A tuple containing ``(role, interface)``, or ``(None, None)``.
523 """
524 _metadata = metadata()
525 for role in ('provides', 'requires', 'peers'):
526 interface = _metadata.get(role, {}).get(relation_name, {}).get('interface')
527 if interface:
528 return role, interface
529 return None, None
530
531
532@cached
533def role_and_interface_to_relations(role, interface_name):
534 """
535 Given a role and interface name, return a list of relation names for the
536 current charm that use that interface under that role (where role is one
537 of ``provides``, ``requires``, or ``peers``).
538
539 :returns: A list of relation names.
540 """
541 _metadata = metadata()
542 results = []
543 for relation_name, relation in _metadata.get(role, {}).items():
544 if relation['interface'] == interface_name:
545 results.append(relation_name)
546 return results
547
548
549@cached
550def interface_to_relations(interface_name):
551 """
552 Given an interface, return a list of relation names for the current
553 charm that use that interface.
554
555 :returns: A list of relation names.
556 """
557 results = []
558 for role in ('provides', 'requires', 'peers'):
559 results.extend(role_and_interface_to_relations(role, interface_name))
560 return results
561
562
563@cached
472def charm_name():564def charm_name():
473 """Get the name of the current charm as is specified on metadata.yaml"""565 """Get the name of the current charm as is specified on metadata.yaml"""
474 return metadata().get('name')566 return metadata().get('name')
@@ -524,6 +616,20 @@
524 subprocess.check_call(_args)616 subprocess.check_call(_args)
525617
526618
619def open_ports(start, end, protocol="TCP"):
620 """Opens a range of service network ports"""
621 _args = ['open-port']
622 _args.append('{}-{}/{}'.format(start, end, protocol))
623 subprocess.check_call(_args)
624
625
626def close_ports(start, end, protocol="TCP"):
627 """Close a range of service network ports"""
628 _args = ['close-port']
629 _args.append('{}-{}/{}'.format(start, end, protocol))
630 subprocess.check_call(_args)
631
632
527@cached633@cached
528def unit_get(attribute):634def unit_get(attribute):
529 """Get the unit ID for the remote unit"""635 """Get the unit ID for the remote unit"""
@@ -544,6 +650,38 @@
544 return unit_get('private-address')650 return unit_get('private-address')
545651
546652
653@cached
654def storage_get(attribute=None, storage_id=None):
655 """Get storage attributes"""
656 _args = ['storage-get', '--format=json']
657 if storage_id:
658 _args.extend(('-s', storage_id))
659 if attribute:
660 _args.append(attribute)
661 try:
662 return json.loads(subprocess.check_output(_args).decode('UTF-8'))
663 except ValueError:
664 return None
665
666
667@cached
668def storage_list(storage_name=None):
669 """List the storage IDs for the unit"""
670 _args = ['storage-list', '--format=json']
671 if storage_name:
672 _args.append(storage_name)
673 try:
674 return json.loads(subprocess.check_output(_args).decode('UTF-8'))
675 except ValueError:
676 return None
677 except OSError as e:
678 import errno
679 if e.errno == errno.ENOENT:
680 # storage-list does not exist
681 return []
682 raise
683
684
547class UnregisteredHookError(Exception):685class UnregisteredHookError(Exception):
548 """Raised when an undefined hook is called"""686 """Raised when an undefined hook is called"""
549 pass687 pass
@@ -571,10 +709,14 @@
571 hooks.execute(sys.argv)709 hooks.execute(sys.argv)
572 """710 """
573711
574 def __init__(self, config_save=True):712 def __init__(self, config_save=None):
575 super(Hooks, self).__init__()713 super(Hooks, self).__init__()
576 self._hooks = {}714 self._hooks = {}
577 self._config_save = config_save715
716 # For unknown reasons, we allow the Hooks constructor to override
717 # config().implicit_save.
718 if config_save is not None:
719 config().implicit_save = config_save
578720
579 def register(self, name, function):721 def register(self, name, function):
580 """Register a hook"""722 """Register a hook"""
@@ -582,13 +724,16 @@
582724
583 def execute(self, args):725 def execute(self, args):
584 """Execute a registered hook based on args[0]"""726 """Execute a registered hook based on args[0]"""
727 _run_atstart()
585 hook_name = os.path.basename(args[0])728 hook_name = os.path.basename(args[0])
586 if hook_name in self._hooks:729 if hook_name in self._hooks:
587 self._hooks[hook_name]()730 try:
588 if self._config_save:731 self._hooks[hook_name]()
589 cfg = config()732 except SystemExit as x:
590 if cfg.implicit_save:733 if x.code is None or x.code == 0:
591 cfg.save()734 _run_atexit()
735 raise
736 _run_atexit()
592 else:737 else:
593 raise UnregisteredHookError(hook_name)738 raise UnregisteredHookError(hook_name)
594739
@@ -637,6 +782,21 @@
637 subprocess.check_call(['action-fail', message])782 subprocess.check_call(['action-fail', message])
638783
639784
785def action_name():
786 """Get the name of the currently executing action."""
787 return os.environ.get('JUJU_ACTION_NAME')
788
789
790def action_uuid():
791 """Get the UUID of the currently executing action."""
792 return os.environ.get('JUJU_ACTION_UUID')
793
794
795def action_tag():
796 """Get the tag for the currently executing action."""
797 return os.environ.get('JUJU_ACTION_TAG')
798
799
640def status_set(workload_state, message):800def status_set(workload_state, message):
641 """Set the workload state with a message801 """Set the workload state with a message
642802
@@ -666,18 +826,243 @@
666826
667827
668def status_get():828def status_get():
669 """Retrieve the previously set juju workload state829 """Retrieve the previously set juju workload state and message
670830
671 If the status-set command is not found then assume this is juju < 1.23 and831 If the status-get command is not found then assume this is juju < 1.23 and
672 return 'unknown'832 return 'unknown', ""
833
673 """834 """
674 cmd = ['status-get']835 cmd = ['status-get', "--format=json", "--include-data"]
675 try:836 try:
676 raw_status = subprocess.check_output(cmd, universal_newlines=True)837 raw_status = subprocess.check_output(cmd)
677 status = raw_status.rstrip()
678 return status
679 except OSError as e:838 except OSError as e:
680 if e.errno == errno.ENOENT:839 if e.errno == errno.ENOENT:
681 return 'unknown'840 return ('unknown', "")
682 else:841 else:
683 raise842 raise
843 else:
844 status = json.loads(raw_status.decode("UTF-8"))
845 return (status["status"], status["message"])
846
847
848def translate_exc(from_exc, to_exc):
849 def inner_translate_exc1(f):
850 @wraps(f)
851 def inner_translate_exc2(*args, **kwargs):
852 try:
853 return f(*args, **kwargs)
854 except from_exc:
855 raise to_exc
856
857 return inner_translate_exc2
858
859 return inner_translate_exc1
860
861
862def application_version_set(version):
863 """Charm authors may trigger this command from any hook to output what
864 version of the application is running. This could be a package version,
865 for instance postgres version 9.5. It could also be a build number or
866 version control revision identifier, for instance git sha 6fb7ba68. """
867
868 cmd = ['application-version-set']
869 cmd.append(version)
870 try:
871 subprocess.check_call(cmd)
872 except OSError:
873 log("Application Version: {}".format(version))
874
875
876@translate_exc(from_exc=OSError, to_exc=NotImplementedError)
877def is_leader():
878 """Does the current unit hold the juju leadership
879
880 Uses juju to determine whether the current unit is the leader of its peers
881 """
882 cmd = ['is-leader', '--format=json']
883 return json.loads(subprocess.check_output(cmd).decode('UTF-8'))
884
885
886@translate_exc(from_exc=OSError, to_exc=NotImplementedError)
887def leader_get(attribute=None):
888 """Juju leader get value(s)"""
889 cmd = ['leader-get', '--format=json'] + [attribute or '-']
890 return json.loads(subprocess.check_output(cmd).decode('UTF-8'))
891
892
893@translate_exc(from_exc=OSError, to_exc=NotImplementedError)
894def leader_set(settings=None, **kwargs):
895 """Juju leader set value(s)"""
896 # Don't log secrets.
897 # log("Juju leader-set '%s'" % (settings), level=DEBUG)
898 cmd = ['leader-set']
899 settings = settings or {}
900 settings.update(kwargs)
901 for k, v in settings.items():
902 if v is None:
903 cmd.append('{}='.format(k))
904 else:
905 cmd.append('{}={}'.format(k, v))
906 subprocess.check_call(cmd)
907
908
909@translate_exc(from_exc=OSError, to_exc=NotImplementedError)
910def payload_register(ptype, klass, pid):
911 """ is used while a hook is running to let Juju know that a
912 payload has been started."""
913 cmd = ['payload-register']
914 for x in [ptype, klass, pid]:
915 cmd.append(x)
916 subprocess.check_call(cmd)
917
918
919@translate_exc(from_exc=OSError, to_exc=NotImplementedError)
920def payload_unregister(klass, pid):
921 """ is used while a hook is running to let Juju know
922 that a payload has been manually stopped. The <class> and <id> provided
923 must match a payload that has been previously registered with juju using
924 payload-register."""
925 cmd = ['payload-unregister']
926 for x in [klass, pid]:
927 cmd.append(x)
928 subprocess.check_call(cmd)
929
930
931@translate_exc(from_exc=OSError, to_exc=NotImplementedError)
932def payload_status_set(klass, pid, status):
933 """is used to update the current status of a registered payload.
934 The <class> and <id> provided must match a payload that has been previously
935 registered with juju using payload-register. The <status> must be one of the
936 follow: starting, started, stopping, stopped"""
937 cmd = ['payload-status-set']
938 for x in [klass, pid, status]:
939 cmd.append(x)
940 subprocess.check_call(cmd)
941
942
943@translate_exc(from_exc=OSError, to_exc=NotImplementedError)
944def resource_get(name):
945 """used to fetch the resource path of the given name.
946
947 <name> must match a name of defined resource in metadata.yaml
948
949 returns either a path or False if resource not available
950 """
951 if not name:
952 return False
953
954 cmd = ['resource-get', name]
955 try:
956 return subprocess.check_output(cmd).decode('UTF-8')
957 except subprocess.CalledProcessError:
958 return False
959
960
961@cached
962def juju_version():
963 """Full version string (eg. '1.23.3.1-trusty-amd64')"""
964 # Per https://bugs.launchpad.net/juju-core/+bug/1455368/comments/1
965 jujud = glob.glob('/var/lib/juju/tools/machine-*/jujud')[0]
966 return subprocess.check_output([jujud, 'version'],
967 universal_newlines=True).strip()
968
969
970@cached
971def has_juju_version(minimum_version):
972 """Return True if the Juju version is at least the provided version"""
973 return LooseVersion(juju_version()) >= LooseVersion(minimum_version)
974
975
976_atexit = []
977_atstart = []
978
979
980def atstart(callback, *args, **kwargs):
981 '''Schedule a callback to run before the main hook.
982
983 Callbacks are run in the order they were added.
984
985 This is useful for modules and classes to perform initialization
986 and inject behavior. In particular:
987
988 - Run common code before all of your hooks, such as logging
989 the hook name or interesting relation data.
990 - Defer object or module initialization that requires a hook
991 context until we know there actually is a hook context,
992 making testing easier.
993 - Rather than requiring charm authors to include boilerplate to
994 invoke your helper's behavior, have it run automatically if
995 your object is instantiated or module imported.
996
997 This is not at all useful after your hook framework as been launched.
998 '''
999 global _atstart
1000 _atstart.append((callback, args, kwargs))
1001
1002
1003def atexit(callback, *args, **kwargs):
1004 '''Schedule a callback to run on successful hook completion.
1005
1006 Callbacks are run in the reverse order that they were added.'''
1007 _atexit.append((callback, args, kwargs))
1008
1009
1010def _run_atstart():
1011 '''Hook frameworks must invoke this before running the main hook body.'''
1012 global _atstart
1013 for callback, args, kwargs in _atstart:
1014 callback(*args, **kwargs)
1015 del _atstart[:]
1016
1017
1018def _run_atexit():
1019 '''Hook frameworks must invoke this after the main hook body has
1020 successfully completed. Do not invoke it if the hook fails.'''
1021 global _atexit
1022 for callback, args, kwargs in reversed(_atexit):
1023 callback(*args, **kwargs)
1024 del _atexit[:]
1025
1026
1027@translate_exc(from_exc=OSError, to_exc=NotImplementedError)
1028def network_get_primary_address(binding):
1029 '''
1030 Retrieve the primary network address for a named binding
1031
1032 :param binding: string. The name of a relation of extra-binding
1033 :return: string. The primary IP address for the named binding
1034 :raise: NotImplementedError if run on Juju < 2.0
1035 '''
1036 cmd = ['network-get', '--primary-address', binding]
1037 return subprocess.check_output(cmd).decode('UTF-8').strip()
1038
1039
1040def add_metric(*args, **kwargs):
1041 """Add metric values. Values may be expressed with keyword arguments. For
1042 metric names containing dashes, these may be expressed as one or more
1043 'key=value' positional arguments. May only be called from the collect-metrics
1044 hook."""
1045 _args = ['add-metric']
1046 _kvpairs = []
1047 _kvpairs.extend(args)
1048 _kvpairs.extend(['{}={}'.format(k, v) for k, v in kwargs.items()])
1049 _args.extend(sorted(_kvpairs))
1050 try:
1051 subprocess.check_call(_args)
1052 return
1053 except EnvironmentError as e:
1054 if e.errno != errno.ENOENT:
1055 raise
1056 log_message = 'add-metric failed: {}'.format(' '.join(_kvpairs))
1057 log(log_message, level='INFO')
1058
1059
1060def meter_status():
1061 """Get the meter status, if running in the meter-status-changed hook."""
1062 return os.environ.get('JUJU_METER_STATUS')
1063
1064
1065def meter_info():
1066 """Get the meter status information, if running in the meter-status-changed
1067 hook."""
1068 return os.environ.get('JUJU_METER_INFO')
6841069
=== modified file 'hooks/charmhelpers/core/host.py'
--- hooks/charmhelpers/core/host.py 2015-05-14 10:48:09 +0000
+++ hooks/charmhelpers/core/host.py 2018-06-12 20:07:13 +0000
@@ -1,18 +1,16 @@
1# Copyright 2014-2015 Canonical Limited.1# Copyright 2014-2015 Canonical Limited.
2#2#
3# This file is part of charm-helpers.3# Licensed under the Apache License, Version 2.0 (the "License");
4#4# you may not use this file except in compliance with the License.
5# charm-helpers is free software: you can redistribute it and/or modify5# You may obtain a copy of the License at
6# it under the terms of the GNU Lesser General Public License version 3 as6#
7# published by the Free Software Foundation.7# http://www.apache.org/licenses/LICENSE-2.0
8#8#
9# charm-helpers is distributed in the hope that it will be useful,9# Unless required by applicable law or agreed to in writing, software
10# but WITHOUT ANY WARRANTY; without even the implied warranty of10# distributed under the License is distributed on an "AS IS" BASIS,
11# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the11# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12# GNU Lesser General Public License for more details.12# See the License for the specific language governing permissions and
13#13# limitations under the License.
14# You should have received a copy of the GNU Lesser General Public License
15# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1614
17"""Tools for working with the host system"""15"""Tools for working with the host system"""
18# Copyright 2012 Canonical Ltd.16# Copyright 2012 Canonical Ltd.
@@ -24,85 +22,326 @@
24import os22import os
25import re23import re
26import pwd24import pwd
25import glob
27import grp26import grp
28import random27import random
29import string28import string
30import subprocess29import subprocess
31import hashlib30import hashlib
31import functools
32import itertools
33import six
34
32from contextlib import contextmanager35from contextlib import contextmanager
33from collections import OrderedDict36from collections import OrderedDict
34
35import six
36
37from .hookenv import log37from .hookenv import log
38from .fstab import Fstab38from .fstab import Fstab
3939from charmhelpers.osplatform import get_platform
4040
41def service_start(service_name):41__platform__ = get_platform()
42 """Start a system service"""42if __platform__ == "ubuntu":
43 return service('start', service_name)43 from charmhelpers.core.host_factory.ubuntu import (
4444 service_available,
4545 add_new_group,
46def service_stop(service_name):46 lsb_release,
47 """Stop a system service"""47 cmp_pkgrevno,
48 return service('stop', service_name)48 ) # flake8: noqa -- ignore F401 for this import
4949elif __platform__ == "centos":
5050 from charmhelpers.core.host_factory.centos import (
51def service_restart(service_name):51 service_available,
52 """Restart a system service"""52 add_new_group,
53 lsb_release,
54 cmp_pkgrevno,
55 ) # flake8: noqa -- ignore F401 for this import
56
57UPDATEDB_PATH = '/etc/updatedb.conf'
58
59def service_start(service_name, **kwargs):
60 """Start a system service.
61
62 The specified service name is managed via the system level init system.
63 Some init systems (e.g. upstart) require that additional arguments be
64 provided in order to directly control service instances whereas other init
65 systems allow for addressing instances of a service directly by name (e.g.
66 systemd).
67
68 The kwargs allow for the additional parameters to be passed to underlying
69 init systems for those systems which require/allow for them. For example,
70 the ceph-osd upstart script requires the id parameter to be passed along
71 in order to identify which running daemon should be reloaded. The follow-
72 ing example stops the ceph-osd service for instance id=4:
73
74 service_stop('ceph-osd', id=4)
75
76 :param service_name: the name of the service to stop
77 :param **kwargs: additional parameters to pass to the init system when
78 managing services. These will be passed as key=value
79 parameters to the init system's commandline. kwargs
80 are ignored for systemd enabled systems.
81 """
82 return service('start', service_name, **kwargs)
83
84
85def service_stop(service_name, **kwargs):
86 """Stop a system service.
87
88 The specified service name is managed via the system level init system.
89 Some init systems (e.g. upstart) require that additional arguments be
90 provided in order to directly control service instances whereas other init
91 systems allow for addressing instances of a service directly by name (e.g.
92 systemd).
93
94 The kwargs allow for the additional parameters to be passed to underlying
95 init systems for those systems which require/allow for them. For example,
96 the ceph-osd upstart script requires the id parameter to be passed along
97 in order to identify which running daemon should be reloaded. The follow-
98 ing example stops the ceph-osd service for instance id=4:
99
100 service_stop('ceph-osd', id=4)
101
102 :param service_name: the name of the service to stop
103 :param **kwargs: additional parameters to pass to the init system when
104 managing services. These will be passed as key=value
105 parameters to the init system's commandline. kwargs
106 are ignored for systemd enabled systems.
107 """
108 return service('stop', service_name, **kwargs)
109
110
111def service_restart(service_name, **kwargs):
112 """Restart a system service.
113
114 The specified service name is managed via the system level init system.
115 Some init systems (e.g. upstart) require that additional arguments be
116 provided in order to directly control service instances whereas other init
117 systems allow for addressing instances of a service directly by name (e.g.
118 systemd).
119
120 The kwargs allow for the additional parameters to be passed to underlying
121 init systems for those systems which require/allow for them. For example,
122 the ceph-osd upstart script requires the id parameter to be passed along
123 in order to identify which running daemon should be restarted. The follow-
124 ing example restarts the ceph-osd service for instance id=4:
125
126 service_restart('ceph-osd', id=4)
127
128 :param service_name: the name of the service to restart
129 :param **kwargs: additional parameters to pass to the init system when
130 managing services. These will be passed as key=value
131 parameters to the init system's commandline. kwargs
132 are ignored for init systems not allowing additional
133 parameters via the commandline (systemd).
134 """
53 return service('restart', service_name)135 return service('restart', service_name)
54136
55137
56def service_reload(service_name, restart_on_failure=False):138def service_reload(service_name, restart_on_failure=False, **kwargs):
57 """Reload a system service, optionally falling back to restart if139 """Reload a system service, optionally falling back to restart if
58 reload fails"""140 reload fails.
59 service_result = service('reload', service_name)141
142 The specified service name is managed via the system level init system.
143 Some init systems (e.g. upstart) require that additional arguments be
144 provided in order to directly control service instances whereas other init
145 systems allow for addressing instances of a service directly by name (e.g.
146 systemd).
147
148 The kwargs allow for the additional parameters to be passed to underlying
149 init systems for those systems which require/allow for them. For example,
150 the ceph-osd upstart script requires the id parameter to be passed along
151 in order to identify which running daemon should be reloaded. The follow-
152 ing example restarts the ceph-osd service for instance id=4:
153
154 service_reload('ceph-osd', id=4)
155
156 :param service_name: the name of the service to reload
157 :param restart_on_failure: boolean indicating whether to fallback to a
158 restart if the reload fails.
159 :param **kwargs: additional parameters to pass to the init system when
160 managing services. These will be passed as key=value
161 parameters to the init system's commandline. kwargs
162 are ignored for init systems not allowing additional
163 parameters via the commandline (systemd).
164 """
165 service_result = service('reload', service_name, **kwargs)
60 if not service_result and restart_on_failure:166 if not service_result and restart_on_failure:
61 service_result = service('restart', service_name)167 service_result = service('restart', service_name, **kwargs)
62 return service_result168 return service_result
63169
64170
65def service(action, service_name):171def service_pause(service_name, init_dir="/etc/init", initd_dir="/etc/init.d",
66 """Control a system service"""172 **kwargs):
67 cmd = ['service', service_name, action]173 """Pause a system service.
174
175 Stop it, and prevent it from starting again at boot.
176
177 :param service_name: the name of the service to pause
178 :param init_dir: path to the upstart init directory
179 :param initd_dir: path to the sysv init directory
180 :param **kwargs: additional parameters to pass to the init system when
181 managing services. These will be passed as key=value
182 parameters to the init system's commandline. kwargs
183 are ignored for init systems which do not support
184 key=value arguments via the commandline.
185 """
186 stopped = True
187 if service_running(service_name, **kwargs):
188 stopped = service_stop(service_name, **kwargs)
189 upstart_file = os.path.join(init_dir, "{}.conf".format(service_name))
190 sysv_file = os.path.join(initd_dir, service_name)
191 if init_is_systemd():
192 service('disable', service_name)
193 elif os.path.exists(upstart_file):
194 override_path = os.path.join(
195 init_dir, '{}.override'.format(service_name))
196 with open(override_path, 'w') as fh:
197 fh.write("manual\n")
198 elif os.path.exists(sysv_file):
199 subprocess.check_call(["update-rc.d", service_name, "disable"])
200 else:
201 raise ValueError(
202 "Unable to detect {0} as SystemD, Upstart {1} or"
203 " SysV {2}".format(
204 service_name, upstart_file, sysv_file))
205 return stopped
206
207
208def service_resume(service_name, init_dir="/etc/init",
209 initd_dir="/etc/init.d", **kwargs):
210 """Resume a system service.
211
212 Reenable starting again at boot. Start the service.
213
214 :param service_name: the name of the service to resume
215 :param init_dir: the path to the init dir
216 :param initd dir: the path to the initd dir
217 :param **kwargs: additional parameters to pass to the init system when
218 managing services. These will be passed as key=value
219 parameters to the init system's commandline. kwargs
220 are ignored for systemd enabled systems.
221 """
222 upstart_file = os.path.join(init_dir, "{}.conf".format(service_name))
223 sysv_file = os.path.join(initd_dir, service_name)
224 if init_is_systemd():
225 service('enable', service_name)
226 elif os.path.exists(upstart_file):
227 override_path = os.path.join(
228 init_dir, '{}.override'.format(service_name))
229 if os.path.exists(override_path):
230 os.unlink(override_path)
231 elif os.path.exists(sysv_file):
232 subprocess.check_call(["update-rc.d", service_name, "enable"])
233 else:
234 raise ValueError(
235 "Unable to detect {0} as SystemD, Upstart {1} or"
236 " SysV {2}".format(
237 service_name, upstart_file, sysv_file))
238 started = service_running(service_name, **kwargs)
239
240 if not started:
241 started = service_start(service_name, **kwargs)
242 return started
243
244
245def service(action, service_name, **kwargs):
246 """Control a system service.
247
248 :param action: the action to take on the service
249 :param service_name: the name of the service to perform th action on
250 :param **kwargs: additional params to be passed to the service command in
251 the form of key=value.
252 """
253 if init_is_systemd():
254 cmd = ['systemctl', action, service_name]
255 else:
256 cmd = ['service', service_name, action]
257 for key, value in six.iteritems(kwargs):
258 parameter = '%s=%s' % (key, value)
259 cmd.append(parameter)
68 return subprocess.call(cmd) == 0260 return subprocess.call(cmd) == 0
69261
70262
71def service_running(service):263_UPSTART_CONF = "/etc/init/{}.conf"
72 """Determine whether a system service is running"""264_INIT_D_CONF = "/etc/init.d/{}"
73 try:265
74 output = subprocess.check_output(266
75 ['service', service, 'status'],267def service_running(service_name, **kwargs):
76 stderr=subprocess.STDOUT).decode('UTF-8')268 """Determine whether a system service is running.
77 except subprocess.CalledProcessError:269
78 return False270 :param service_name: the name of the service
79 else:271 :param **kwargs: additional args to pass to the service command. This is
80 if ("start/running" in output or "is running" in output):272 used to pass additional key=value arguments to the
81 return True273 service command line for managing specific instance
82 else:274 units (e.g. service ceph-osd status id=2). The kwargs
83 return False275 are ignored in systemd services.
84276 """
85277 if init_is_systemd():
86def service_available(service_name):278 return service('is-active', service_name)
87 """Determine whether a system service is available"""279 else:
88 try:280 if os.path.exists(_UPSTART_CONF.format(service_name)):
89 subprocess.check_output(281 try:
90 ['service', service_name, 'status'],282 cmd = ['status', service_name]
91 stderr=subprocess.STDOUT).decode('UTF-8')283 for key, value in six.iteritems(kwargs):
92 except subprocess.CalledProcessError as e:284 parameter = '%s=%s' % (key, value)
93 return b'unrecognized service' not in e.output285 cmd.append(parameter)
94 else:286 output = subprocess.check_output(cmd,
95 return True287 stderr=subprocess.STDOUT).decode('UTF-8')
96288 except subprocess.CalledProcessError:
97289 return False
98def adduser(username, password=None, shell='/bin/bash', system_user=False):290 else:
99 """Add a user to the system"""291 # This works for upstart scripts where the 'service' command
292 # returns a consistent string to represent running
293 # 'start/running'
294 if ("start/running" in output or
295 "is running" in output or
296 "up and running" in output):
297 return True
298 elif os.path.exists(_INIT_D_CONF.format(service_name)):
299 # Check System V scripts init script return codes
300 return service('status', service_name)
301 return False
302
303
304SYSTEMD_SYSTEM = '/run/systemd/system'
305
306
307def init_is_systemd():
308 """Return True if the host system uses systemd, False otherwise."""
309 if lsb_release()['DISTRIB_CODENAME'] == 'trusty':
310 return False
311 return os.path.isdir(SYSTEMD_SYSTEM)
312
313
314def adduser(username, password=None, shell='/bin/bash',
315 system_user=False, primary_group=None,
316 secondary_groups=None, uid=None, home_dir=None):
317 """Add a user to the system.
318
319 Will log but otherwise succeed if the user already exists.
320
321 :param str username: Username to create
322 :param str password: Password for user; if ``None``, create a system user
323 :param str shell: The default shell for the user
324 :param bool system_user: Whether to create a login or system user
325 :param str primary_group: Primary group for user; defaults to username
326 :param list secondary_groups: Optional list of additional groups
327 :param int uid: UID for user being created
328 :param str home_dir: Home directory for user
329
330 :returns: The password database entry struct, as returned by `pwd.getpwnam`
331 """
100 try:332 try:
101 user_info = pwd.getpwnam(username)333 user_info = pwd.getpwnam(username)
102 log('user {0} already exists!'.format(username))334 log('user {0} already exists!'.format(username))
335 if uid:
336 user_info = pwd.getpwuid(int(uid))
337 log('user with uid {0} already exists!'.format(uid))
103 except KeyError:338 except KeyError:
104 log('creating user {0}'.format(username))339 log('creating user {0}'.format(username))
105 cmd = ['useradd']340 cmd = ['useradd']
341 if uid:
342 cmd.extend(['--uid', str(uid)])
343 if home_dir:
344 cmd.extend(['--home', str(home_dir)])
106 if system_user or password is None:345 if system_user or password is None:
107 cmd.append('--system')346 cmd.append('--system')
108 else:347 else:
@@ -111,52 +350,104 @@
111 '--shell', shell,350 '--shell', shell,
112 '--password', password,351 '--password', password,
113 ])352 ])
353 if not primary_group:
354 try:
355 grp.getgrnam(username)
356 primary_group = username # avoid "group exists" error
357 except KeyError:
358 pass
359 if primary_group:
360 cmd.extend(['-g', primary_group])
361 if secondary_groups:
362 cmd.extend(['-G', ','.join(secondary_groups)])
114 cmd.append(username)363 cmd.append(username)
115 subprocess.check_call(cmd)364 subprocess.check_call(cmd)
116 user_info = pwd.getpwnam(username)365 user_info = pwd.getpwnam(username)
117 return user_info366 return user_info
118367
119368
120def add_group(group_name, system_group=False):369def user_exists(username):
121 """Add a group to the system"""370 """Check if a user exists"""
371 try:
372 pwd.getpwnam(username)
373 user_exists = True
374 except KeyError:
375 user_exists = False
376 return user_exists
377
378
379def uid_exists(uid):
380 """Check if a uid exists"""
381 try:
382 pwd.getpwuid(uid)
383 uid_exists = True
384 except KeyError:
385 uid_exists = False
386 return uid_exists
387
388
389def group_exists(groupname):
390 """Check if a group exists"""
391 try:
392 grp.getgrnam(groupname)
393 group_exists = True
394 except KeyError:
395 group_exists = False
396 return group_exists
397
398
399def gid_exists(gid):
400 """Check if a gid exists"""
401 try:
402 grp.getgrgid(gid)
403 gid_exists = True
404 except KeyError:
405 gid_exists = False
406 return gid_exists
407
408
409def add_group(group_name, system_group=False, gid=None):
410 """Add a group to the system
411
412 Will log but otherwise succeed if the group already exists.
413
414 :param str group_name: group to create
415 :param bool system_group: Create system group
416 :param int gid: GID for user being created
417
418 :returns: The password database entry struct, as returned by `grp.getgrnam`
419 """
122 try:420 try:
123 group_info = grp.getgrnam(group_name)421 group_info = grp.getgrnam(group_name)
124 log('group {0} already exists!'.format(group_name))422 log('group {0} already exists!'.format(group_name))
423 if gid:
424 group_info = grp.getgrgid(gid)
425 log('group with gid {0} already exists!'.format(gid))
125 except KeyError:426 except KeyError:
126 log('creating group {0}'.format(group_name))427 log('creating group {0}'.format(group_name))
127 cmd = ['addgroup']428 add_new_group(group_name, system_group, gid)
128 if system_group:
129 cmd.append('--system')
130 else:
131 cmd.extend([
132 '--group',
133 ])
134 cmd.append(group_name)
135 subprocess.check_call(cmd)
136 group_info = grp.getgrnam(group_name)429 group_info = grp.getgrnam(group_name)
137 return group_info430 return group_info
138431
139432
140def add_user_to_group(username, group):433def add_user_to_group(username, group):
141 """Add a user to a group"""434 """Add a user to a group"""
142 cmd = [435 cmd = ['gpasswd', '-a', username, group]
143 'gpasswd', '-a',
144 username,
145 group
146 ]
147 log("Adding user {} to group {}".format(username, group))436 log("Adding user {} to group {}".format(username, group))
148 subprocess.check_call(cmd)437 subprocess.check_call(cmd)
149438
150439
151def rsync(from_path, to_path, flags='-r', options=None):440def rsync(from_path, to_path, flags='-r', options=None, timeout=None):
152 """Replicate the contents of a path"""441 """Replicate the contents of a path"""
153 options = options or ['--delete', '--executability']442 options = options or ['--delete', '--executability']
154 cmd = ['/usr/bin/rsync', flags]443 cmd = ['/usr/bin/rsync', flags]
444 if timeout:
445 cmd = ['timeout', str(timeout)] + cmd
155 cmd.extend(options)446 cmd.extend(options)
156 cmd.append(from_path)447 cmd.append(from_path)
157 cmd.append(to_path)448 cmd.append(to_path)
158 log(" ".join(cmd))449 log(" ".join(cmd))
159 return subprocess.check_output(cmd).decode('UTF-8').strip()450 return subprocess.check_output(cmd, stderr=subprocess.STDOUT).decode('UTF-8').strip()
160451
161452
162def symlink(source, destination):453def symlink(source, destination):
@@ -202,14 +493,12 @@
202493
203494
204def fstab_remove(mp):495def fstab_remove(mp):
205 """Remove the given mountpoint entry from /etc/fstab496 """Remove the given mountpoint entry from /etc/fstab"""
206 """
207 return Fstab.remove_by_mountpoint(mp)497 return Fstab.remove_by_mountpoint(mp)
208498
209499
210def fstab_add(dev, mp, fs, options=None):500def fstab_add(dev, mp, fs, options=None):
211 """Adds the given device entry to the /etc/fstab file501 """Adds the given device entry to the /etc/fstab file"""
212 """
213 return Fstab.add(dev, mp, fs, options=options)502 return Fstab.add(dev, mp, fs, options=options)
214503
215504
@@ -253,9 +542,19 @@
253 return system_mounts542 return system_mounts
254543
255544
545def fstab_mount(mountpoint):
546 """Mount filesystem using fstab"""
547 cmd_args = ['mount', mountpoint]
548 try:
549 subprocess.check_output(cmd_args)
550 except subprocess.CalledProcessError as e:
551 log('Error unmounting {}\n{}'.format(mountpoint, e.output))
552 return False
553 return True
554
555
256def file_hash(path, hash_type='md5'):556def file_hash(path, hash_type='md5'):
257 """557 """Generate a hash checksum of the contents of 'path' or None if not found.
258 Generate a hash checksum of the contents of 'path' or None if not found.
259558
260 :param str hash_type: Any hash alrgorithm supported by :mod:`hashlib`,559 :param str hash_type: Any hash alrgorithm supported by :mod:`hashlib`,
261 such as md5, sha1, sha256, sha512, etc.560 such as md5, sha1, sha256, sha512, etc.
@@ -269,9 +568,22 @@
269 return None568 return None
270569
271570
571def path_hash(path):
572 """Generate a hash checksum of all files matching 'path'. Standard
573 wildcards like '*' and '?' are supported, see documentation for the 'glob'
574 module for more information.
575
576 :return: dict: A { filename: hash } dictionary for all matched files.
577 Empty if none found.
578 """
579 return {
580 filename: file_hash(filename)
581 for filename in glob.iglob(path)
582 }
583
584
272def check_hash(path, checksum, hash_type='md5'):585def check_hash(path, checksum, hash_type='md5'):
273 """586 """Validate a file using a cryptographic checksum.
274 Validate a file using a cryptographic checksum.
275587
276 :param str checksum: Value of the checksum used to validate the file.588 :param str checksum: Value of the checksum used to validate the file.
277 :param str hash_type: Hash algorithm used to generate `checksum`.589 :param str hash_type: Hash algorithm used to generate `checksum`.
@@ -286,54 +598,78 @@
286598
287599
288class ChecksumError(ValueError):600class ChecksumError(ValueError):
601 """A class derived from Value error to indicate the checksum failed."""
289 pass602 pass
290603
291604
292def restart_on_change(restart_map, stopstart=False):605def restart_on_change(restart_map, stopstart=False, restart_functions=None):
293 """Restart services based on configuration files changing606 """Restart services based on configuration files changing
294607
295 This function is used a decorator, for example::608 This function is used a decorator, for example::
296609
297 @restart_on_change({610 @restart_on_change({
298 '/etc/ceph/ceph.conf': [ 'cinder-api', 'cinder-volume' ]611 '/etc/ceph/ceph.conf': [ 'cinder-api', 'cinder-volume' ]
612 '/etc/apache/sites-enabled/*': [ 'apache2' ]
299 })613 })
300 def ceph_client_changed():614 def config_changed():
301 pass # your code here615 pass # your code here
302616
303 In this example, the cinder-api and cinder-volume services617 In this example, the cinder-api and cinder-volume services
304 would be restarted if /etc/ceph/ceph.conf is changed by the618 would be restarted if /etc/ceph/ceph.conf is changed by the
305 ceph_client_changed function.619 ceph_client_changed function. The apache2 service would be
620 restarted if any file matching the pattern got changed, created
621 or removed. Standard wildcards are supported, see documentation
622 for the 'glob' module for more information.
623
624 @param restart_map: {path_file_name: [service_name, ...]
625 @param stopstart: DEFAULT false; whether to stop, start OR restart
626 @param restart_functions: nonstandard functions to use to restart services
627 {svc: func, ...}
628 @returns result from decorated function
306 """629 """
307 def wrap(f):630 def wrap(f):
631 @functools.wraps(f)
308 def wrapped_f(*args, **kwargs):632 def wrapped_f(*args, **kwargs):
309 checksums = {}633 return restart_on_change_helper(
310 for path in restart_map:634 (lambda: f(*args, **kwargs)), restart_map, stopstart,
311 checksums[path] = file_hash(path)635 restart_functions)
312 f(*args, **kwargs)
313 restarts = []
314 for path in restart_map:
315 if checksums[path] != file_hash(path):
316 restarts += restart_map[path]
317 services_list = list(OrderedDict.fromkeys(restarts))
318 if not stopstart:
319 for service_name in services_list:
320 service('restart', service_name)
321 else:
322 for action in ['stop', 'start']:
323 for service_name in services_list:
324 service(action, service_name)
325 return wrapped_f636 return wrapped_f
326 return wrap637 return wrap
327638
328639
329def lsb_release():640def restart_on_change_helper(lambda_f, restart_map, stopstart=False,
330 """Return /etc/lsb-release in a dict"""641 restart_functions=None):
331 d = {}642 """Helper function to perform the restart_on_change function.
332 with open('/etc/lsb-release', 'r') as lsb:643
333 for l in lsb:644 This is provided for decorators to restart services if files described
334 k, v = l.split('=')645 in the restart_map have changed after an invocation of lambda_f().
335 d[k.strip()] = v.strip()646
336 return d647 @param lambda_f: function to call.
648 @param restart_map: {file: [service, ...]}
649 @param stopstart: whether to stop, start or restart a service
650 @param restart_functions: nonstandard functions to use to restart services
651 {svc: func, ...}
652 @returns result of lambda_f()
653 """
654 if restart_functions is None:
655 restart_functions = {}
656 checksums = {path: path_hash(path) for path in restart_map}
657 r = lambda_f()
658 # create a list of lists of the services to restart
659 restarts = [restart_map[path]
660 for path in restart_map
661 if path_hash(path) != checksums[path]]
662 # create a flat list of ordered services without duplicates from lists
663 services_list = list(OrderedDict.fromkeys(itertools.chain(*restarts)))
664 if services_list:
665 actions = ('stop', 'start') if stopstart else ('restart',)
666 for service_name in services_list:
667 if service_name in restart_functions:
668 restart_functions[service_name](service_name)
669 else:
670 for action in actions:
671 service(action, service_name)
672 return r
337673
338674
339def pwgen(length=None):675def pwgen(length=None):
@@ -352,36 +688,92 @@
352 return(''.join(random_chars))688 return(''.join(random_chars))
353689
354690
355def list_nics(nic_type):691def is_phy_iface(interface):
356 '''Return a list of nics of given type(s)'''692 """Returns True if interface is not virtual, otherwise False."""
693 if interface:
694 sys_net = '/sys/class/net'
695 if os.path.isdir(sys_net):
696 for iface in glob.glob(os.path.join(sys_net, '*')):
697 if '/virtual/' in os.path.realpath(iface):
698 continue
699
700 if interface == os.path.basename(iface):
701 return True
702
703 return False
704
705
706def get_bond_master(interface):
707 """Returns bond master if interface is bond slave otherwise None.
708
709 NOTE: the provided interface is expected to be physical
710 """
711 if interface:
712 iface_path = '/sys/class/net/%s' % (interface)
713 if os.path.exists(iface_path):
714 if '/virtual/' in os.path.realpath(iface_path):
715 return None
716
717 master = os.path.join(iface_path, 'master')
718 if os.path.exists(master):
719 master = os.path.realpath(master)
720 # make sure it is a bond master
721 if os.path.exists(os.path.join(master, 'bonding')):
722 return os.path.basename(master)
723
724 return None
725
726
727def list_nics(nic_type=None):
728 """Return a list of nics of given type(s)"""
357 if isinstance(nic_type, six.string_types):729 if isinstance(nic_type, six.string_types):
358 int_types = [nic_type]730 int_types = [nic_type]
359 else:731 else:
360 int_types = nic_type732 int_types = nic_type
733
361 interfaces = []734 interfaces = []
362 for int_type in int_types:735 if nic_type:
363 cmd = ['ip', 'addr', 'show', 'label', int_type + '*']736 for int_type in int_types:
737 cmd = ['ip', 'addr', 'show', 'label', int_type + '*']
738 ip_output = subprocess.check_output(cmd).decode('UTF-8')
739 ip_output = ip_output.split('\n')
740 ip_output = (line for line in ip_output if line)
741 for line in ip_output:
742 if line.split()[1].startswith(int_type):
743 matched = re.search('.*: (' + int_type +
744 r'[0-9]+\.[0-9]+)@.*', line)
745 if matched:
746 iface = matched.groups()[0]
747 else:
748 iface = line.split()[1].replace(":", "")
749
750 if iface not in interfaces:
751 interfaces.append(iface)
752 else:
753 cmd = ['ip', 'a']
364 ip_output = subprocess.check_output(cmd).decode('UTF-8').split('\n')754 ip_output = subprocess.check_output(cmd).decode('UTF-8').split('\n')
365 ip_output = (line for line in ip_output if line)755 ip_output = (line.strip() for line in ip_output if line)
756
757 key = re.compile('^[0-9]+:\s+(.+):')
366 for line in ip_output:758 for line in ip_output:
367 if line.split()[1].startswith(int_type):759 matched = re.search(key, line)
368 matched = re.search('.*: (' + int_type + r'[0-9]+\.[0-9]+)@.*', line)760 if matched:
369 if matched:761 iface = matched.group(1)
370 interface = matched.groups()[0]762 iface = iface.partition("@")[0]
371 else:763 if iface not in interfaces:
372 interface = line.split()[1].replace(":", "")764 interfaces.append(iface)
373 interfaces.append(interface)
374765
375 return interfaces766 return interfaces
376767
377768
378def set_nic_mtu(nic, mtu):769def set_nic_mtu(nic, mtu):
379 '''Set MTU on a network interface'''770 """Set the Maximum Transmission Unit (MTU) on a network interface."""
380 cmd = ['ip', 'link', 'set', nic, 'mtu', mtu]771 cmd = ['ip', 'link', 'set', nic, 'mtu', mtu]
381 subprocess.check_call(cmd)772 subprocess.check_call(cmd)
382773
383774
384def get_nic_mtu(nic):775def get_nic_mtu(nic):
776 """Return the Maximum Transmission Unit (MTU) for a network interface."""
385 cmd = ['ip', 'addr', 'show', nic]777 cmd = ['ip', 'addr', 'show', nic]
386 ip_output = subprocess.check_output(cmd).decode('UTF-8').split('\n')778 ip_output = subprocess.check_output(cmd).decode('UTF-8').split('\n')
387 mtu = ""779 mtu = ""
@@ -393,6 +785,7 @@
393785
394786
395def get_nic_hwaddr(nic):787def get_nic_hwaddr(nic):
788 """Return the Media Access Control (MAC) for a network interface."""
396 cmd = ['ip', '-o', '-0', 'addr', 'show', nic]789 cmd = ['ip', '-o', '-0', 'addr', 'show', nic]
397 ip_output = subprocess.check_output(cmd).decode('UTF-8')790 ip_output = subprocess.check_output(cmd).decode('UTF-8')
398 hwaddr = ""791 hwaddr = ""
@@ -402,35 +795,31 @@
402 return hwaddr795 return hwaddr
403796
404797
405def cmp_pkgrevno(package, revno, pkgcache=None):
406 '''Compare supplied revno with the revno of the installed package
407
408 * 1 => Installed revno is greater than supplied arg
409 * 0 => Installed revno is the same as supplied arg
410 * -1 => Installed revno is less than supplied arg
411
412 This function imports apt_cache function from charmhelpers.fetch if
413 the pkgcache argument is None. Be sure to add charmhelpers.fetch if
414 you call this function, or pass an apt_pkg.Cache() instance.
415 '''
416 import apt_pkg
417 if not pkgcache:
418 from charmhelpers.fetch import apt_cache
419 pkgcache = apt_cache()
420 pkg = pkgcache[package]
421 return apt_pkg.version_compare(pkg.current_ver.ver_str, revno)
422
423
424@contextmanager798@contextmanager
425def chdir(d):799def chdir(directory):
800 """Change the current working directory to a different directory for a code
801 block and return the previous directory after the block exits. Useful to
802 run commands from a specificed directory.
803
804 :param str directory: The directory path to change to for this context.
805 """
426 cur = os.getcwd()806 cur = os.getcwd()
427 try:807 try:
428 yield os.chdir(d)808 yield os.chdir(directory)
429 finally:809 finally:
430 os.chdir(cur)810 os.chdir(cur)
431811
432812
433def chownr(path, owner, group, follow_links=True):813def chownr(path, owner, group, follow_links=True, chowntopdir=False):
814 """Recursively change user and group ownership of files and directories
815 in given path. Doesn't chown path itself by default, only its children.
816
817 :param str path: The string path to start changing ownership.
818 :param str owner: The owner string to use when looking up the uid.
819 :param str group: The group string to use when looking up the gid.
820 :param bool follow_links: Also follow and chown links if True
821 :param bool chowntopdir: Also chown path itself if True
822 """
434 uid = pwd.getpwnam(owner).pw_uid823 uid = pwd.getpwnam(owner).pw_uid
435 gid = grp.getgrnam(group).gr_gid824 gid = grp.getgrnam(group).gr_gid
436 if follow_links:825 if follow_links:
@@ -438,7 +827,11 @@
438 else:827 else:
439 chown = os.lchown828 chown = os.lchown
440829
441 for root, dirs, files in os.walk(path):830 if chowntopdir:
831 broken_symlink = os.path.lexists(path) and not os.path.exists(path)
832 if not broken_symlink:
833 chown(path, uid, gid)
834 for root, dirs, files in os.walk(path, followlinks=follow_links):
442 for name in dirs + files:835 for name in dirs + files:
443 full = os.path.join(root, name)836 full = os.path.join(root, name)
444 broken_symlink = os.path.lexists(full) and not os.path.exists(full)837 broken_symlink = os.path.lexists(full) and not os.path.exists(full)
@@ -447,4 +840,81 @@
447840
448841
449def lchownr(path, owner, group):842def lchownr(path, owner, group):
843 """Recursively change user and group ownership of files and directories
844 in a given path, not following symbolic links. See the documentation for
845 'os.lchown' for more information.
846
847 :param str path: The string path to start changing ownership.
848 :param str owner: The owner string to use when looking up the uid.
849 :param str group: The group string to use when looking up the gid.
850 """
450 chownr(path, owner, group, follow_links=False)851 chownr(path, owner, group, follow_links=False)
852
853
854def owner(path):
855 """Returns a tuple containing the username & groupname owning the path.
856
857 :param str path: the string path to retrieve the ownership
858 :return tuple(str, str): A (username, groupname) tuple containing the
859 name of the user and group owning the path.
860 :raises OSError: if the specified path does not exist
861 """
862 stat = os.stat(path)
863 username = pwd.getpwuid(stat.st_uid)[0]
864 groupname = grp.getgrgid(stat.st_gid)[0]
865 return username, groupname
866
867
868def get_total_ram():
869 """The total amount of system RAM in bytes.
870
871 This is what is reported by the OS, and may be overcommitted when
872 there are multiple containers hosted on the same machine.
873 """
874 with open('/proc/meminfo', 'r') as f:
875 for line in f.readlines():
876 if line:
877 key, value, unit = line.split()
878 if key == 'MemTotal:':
879 assert unit == 'kB', 'Unknown unit'
880 return int(value) * 1024 # Classic, not KiB.
881 raise NotImplementedError()
882
883
884UPSTART_CONTAINER_TYPE = '/run/container_type'
885
886
887def is_container():
888 """Determine whether unit is running in a container
889
890 @return: boolean indicating if unit is in a container
891 """
892 if init_is_systemd():
893 # Detect using systemd-detect-virt
894 return subprocess.call(['systemd-detect-virt',
895 '--container']) == 0
896 else:
897 # Detect using upstart container file marker
898 return os.path.exists(UPSTART_CONTAINER_TYPE)
899
900
901def add_to_updatedb_prunepath(path, updatedb_path=UPDATEDB_PATH):
902 with open(updatedb_path, 'r+') as f_id:
903 updatedb_text = f_id.read()
904 output = updatedb(updatedb_text, path)
905 f_id.seek(0)
906 f_id.write(output)
907 f_id.truncate()
908
909
910def updatedb(updatedb_text, new_path):
911 lines = [line for line in updatedb_text.split("\n")]
912 for i, line in enumerate(lines):
913 if line.startswith("PRUNEPATHS="):
914 paths_line = line.split("=")[1].replace('"', '')
915 paths = paths_line.split(" ")
916 if new_path not in paths:
917 paths.append(new_path)
918 lines[i] = 'PRUNEPATHS="{}"'.format(' '.join(paths))
919 output = "\n".join(lines)
920 return output
451921
=== added directory 'hooks/charmhelpers/core/host_factory'
=== added file 'hooks/charmhelpers/core/host_factory/__init__.py'
=== added file 'hooks/charmhelpers/core/host_factory/centos.py'
--- hooks/charmhelpers/core/host_factory/centos.py 1970-01-01 00:00:00 +0000
+++ hooks/charmhelpers/core/host_factory/centos.py 2018-06-12 20:07:13 +0000
@@ -0,0 +1,56 @@
1import subprocess
2import yum
3import os
4
5
6def service_available(service_name):
7 # """Determine whether a system service is available."""
8 if os.path.isdir('/run/systemd/system'):
9 cmd = ['systemctl', 'is-enabled', service_name]
10 else:
11 cmd = ['service', service_name, 'is-enabled']
12 return subprocess.call(cmd) == 0
13
14
15def add_new_group(group_name, system_group=False, gid=None):
16 cmd = ['groupadd']
17 if gid:
18 cmd.extend(['--gid', str(gid)])
19 if system_group:
20 cmd.append('-r')
21 cmd.append(group_name)
22 subprocess.check_call(cmd)
23
24
25def lsb_release():
26 """Return /etc/os-release in a dict."""
27 d = {}
28 with open('/etc/os-release', 'r') as lsb:
29 for l in lsb:
30 s = l.split('=')
31 if len(s) != 2:
32 continue
33 d[s[0].strip()] = s[1].strip()
34 return d
35
36
37def cmp_pkgrevno(package, revno, pkgcache=None):
38 """Compare supplied revno with the revno of the installed package.
39
40 * 1 => Installed revno is greater than supplied arg
41 * 0 => Installed revno is the same as supplied arg
42 * -1 => Installed revno is less than supplied arg
43
44 This function imports YumBase function if the pkgcache argument
45 is None.
46 """
47 if not pkgcache:
48 y = yum.YumBase()
49 packages = y.doPackageLists()
50 pkgcache = {i.Name: i.version for i in packages['installed']}
51 pkg = pkgcache[package]
52 if pkg > revno:
53 return 1
54 if pkg < revno:
55 return -1
56 return 0
057
=== added file 'hooks/charmhelpers/core/host_factory/ubuntu.py'
--- hooks/charmhelpers/core/host_factory/ubuntu.py 1970-01-01 00:00:00 +0000
+++ hooks/charmhelpers/core/host_factory/ubuntu.py 2018-06-12 20:07:13 +0000
@@ -0,0 +1,56 @@
1import subprocess
2
3
4def service_available(service_name):
5 """Determine whether a system service is available"""
6 try:
7 subprocess.check_output(
8 ['service', service_name, 'status'],
9 stderr=subprocess.STDOUT).decode('UTF-8')
10 except subprocess.CalledProcessError as e:
11 return b'unrecognized service' not in e.output
12 else:
13 return True
14
15
16def add_new_group(group_name, system_group=False, gid=None):
17 cmd = ['addgroup']
18 if gid:
19 cmd.extend(['--gid', str(gid)])
20 if system_group:
21 cmd.append('--system')
22 else:
23 cmd.extend([
24 '--group',
25 ])
26 cmd.append(group_name)
27 subprocess.check_call(cmd)
28
29
30def lsb_release():
31 """Return /etc/lsb-release in a dict"""
32 d = {}
33 with open('/etc/lsb-release', 'r') as lsb:
34 for l in lsb:
35 k, v = l.split('=')
36 d[k.strip()] = v.strip()
37 return d
38
39
40def cmp_pkgrevno(package, revno, pkgcache=None):
41 """Compare supplied revno with the revno of the installed package.
42
43 * 1 => Installed revno is greater than supplied arg
44 * 0 => Installed revno is the same as supplied arg
45 * -1 => Installed revno is less than supplied arg
46
47 This function imports apt_cache function from charmhelpers.fetch if
48 the pkgcache argument is None. Be sure to add charmhelpers.fetch if
49 you call this function, or pass an apt_pkg.Cache() instance.
50 """
51 import apt_pkg
52 if not pkgcache:
53 from charmhelpers.fetch import apt_cache
54 pkgcache = apt_cache()
55 pkg = pkgcache[package]
56 return apt_pkg.version_compare(pkg.current_ver.ver_str, revno)
057
=== added file 'hooks/charmhelpers/core/hugepage.py'
--- hooks/charmhelpers/core/hugepage.py 1970-01-01 00:00:00 +0000
+++ hooks/charmhelpers/core/hugepage.py 2018-06-12 20:07:13 +0000
@@ -0,0 +1,69 @@
1# -*- coding: utf-8 -*-
2
3# Copyright 2014-2015 Canonical Limited.
4#
5# Licensed under the Apache License, Version 2.0 (the "License");
6# you may not use this file except in compliance with the License.
7# You may obtain a copy of the License at
8#
9# http://www.apache.org/licenses/LICENSE-2.0
10#
11# Unless required by applicable law or agreed to in writing, software
12# distributed under the License is distributed on an "AS IS" BASIS,
13# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14# See the License for the specific language governing permissions and
15# limitations under the License.
16
17import yaml
18from charmhelpers.core import fstab
19from charmhelpers.core import sysctl
20from charmhelpers.core.host import (
21 add_group,
22 add_user_to_group,
23 fstab_mount,
24 mkdir,
25)
26from charmhelpers.core.strutils import bytes_from_string
27from subprocess import check_output
28
29
30def hugepage_support(user, group='hugetlb', nr_hugepages=256,
31 max_map_count=65536, mnt_point='/run/hugepages/kvm',
32 pagesize='2MB', mount=True, set_shmmax=False):
33 """Enable hugepages on system.
34
35 Args:
36 user (str) -- Username to allow access to hugepages to
37 group (str) -- Group name to own hugepages
38 nr_hugepages (int) -- Number of pages to reserve
39 max_map_count (int) -- Number of Virtual Memory Areas a process can own
40 mnt_point (str) -- Directory to mount hugepages on
41 pagesize (str) -- Size of hugepages
42 mount (bool) -- Whether to Mount hugepages
43 """
44 group_info = add_group(group)
45 gid = group_info.gr_gid
46 add_user_to_group(user, group)
47 if max_map_count < 2 * nr_hugepages:
48 max_map_count = 2 * nr_hugepages
49 sysctl_settings = {
50 'vm.nr_hugepages': nr_hugepages,
51 'vm.max_map_count': max_map_count,
52 'vm.hugetlb_shm_group': gid,
53 }
54 if set_shmmax:
55 shmmax_current = int(check_output(['sysctl', '-n', 'kernel.shmmax']))
56 shmmax_minsize = bytes_from_string(pagesize) * nr_hugepages
57 if shmmax_minsize > shmmax_current:
58 sysctl_settings['kernel.shmmax'] = shmmax_minsize
59 sysctl.create(yaml.dump(sysctl_settings), '/etc/sysctl.d/10-hugepage.conf')
60 mkdir(mnt_point, owner='root', group='root', perms=0o755, force=False)
61 lfstab = fstab.Fstab()
62 fstab_entry = lfstab.get_entry_by_attr('mountpoint', mnt_point)
63 if fstab_entry:
64 lfstab.remove_entry(fstab_entry)
65 entry = lfstab.Entry('nodev', mnt_point, 'hugetlbfs',
66 'mode=1770,gid={},pagesize={}'.format(gid, pagesize), 0, 0)
67 lfstab.add_entry(entry)
68 if mount:
69 fstab_mount(mnt_point)
070
=== added file 'hooks/charmhelpers/core/kernel.py'
--- hooks/charmhelpers/core/kernel.py 1970-01-01 00:00:00 +0000
+++ hooks/charmhelpers/core/kernel.py 2018-06-12 20:07:13 +0000
@@ -0,0 +1,72 @@
1#!/usr/bin/env python
2# -*- coding: utf-8 -*-
3
4# Copyright 2014-2015 Canonical Limited.
5#
6# Licensed under the Apache License, Version 2.0 (the "License");
7# you may not use this file except in compliance with the License.
8# You may obtain a copy of the License at
9#
10# http://www.apache.org/licenses/LICENSE-2.0
11#
12# Unless required by applicable law or agreed to in writing, software
13# distributed under the License is distributed on an "AS IS" BASIS,
14# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
15# See the License for the specific language governing permissions and
16# limitations under the License.
17
18import re
19import subprocess
20
21from charmhelpers.osplatform import get_platform
22from charmhelpers.core.hookenv import (
23 log,
24 INFO
25)
26
27__platform__ = get_platform()
28if __platform__ == "ubuntu":
29 from charmhelpers.core.kernel_factory.ubuntu import (
30 persistent_modprobe,
31 update_initramfs,
32 ) # flake8: noqa -- ignore F401 for this import
33elif __platform__ == "centos":
34 from charmhelpers.core.kernel_factory.centos import (
35 persistent_modprobe,
36 update_initramfs,
37 ) # flake8: noqa -- ignore F401 for this import
38
39__author__ = "Jorge Niedbalski <jorge.niedbalski@canonical.com>"
40
41
42def modprobe(module, persist=True):
43 """Load a kernel module and configure for auto-load on reboot."""
44 cmd = ['modprobe', module]
45
46 log('Loading kernel module %s' % module, level=INFO)
47
48 subprocess.check_call(cmd)
49 if persist:
50 persistent_modprobe(module)
51
52
53def rmmod(module, force=False):
54 """Remove a module from the linux kernel"""
55 cmd = ['rmmod']
56 if force:
57 cmd.append('-f')
58 cmd.append(module)
59 log('Removing kernel module %s' % module, level=INFO)
60 return subprocess.check_call(cmd)
61
62
63def lsmod():
64 """Shows what kernel modules are currently loaded"""
65 return subprocess.check_output(['lsmod'],
66 universal_newlines=True)
67
68
69def is_module_loaded(module):
70 """Checks if a kernel module is already loaded"""
71 matches = re.findall('^%s[ ]+' % module, lsmod(), re.M)
72 return len(matches) > 0
073
=== added directory 'hooks/charmhelpers/core/kernel_factory'
=== added file 'hooks/charmhelpers/core/kernel_factory/__init__.py'
=== added file 'hooks/charmhelpers/core/kernel_factory/centos.py'
--- hooks/charmhelpers/core/kernel_factory/centos.py 1970-01-01 00:00:00 +0000
+++ hooks/charmhelpers/core/kernel_factory/centos.py 2018-06-12 20:07:13 +0000
@@ -0,0 +1,17 @@
1import subprocess
2import os
3
4
5def persistent_modprobe(module):
6 """Load a kernel module and configure for auto-load on reboot."""
7 if not os.path.exists('/etc/rc.modules'):
8 open('/etc/rc.modules', 'a')
9 os.chmod('/etc/rc.modules', 111)
10 with open('/etc/rc.modules', 'r+') as modules:
11 if module not in modules.read():
12 modules.write('modprobe %s\n' % module)
13
14
15def update_initramfs(version='all'):
16 """Updates an initramfs image."""
17 return subprocess.check_call(["dracut", "-f", version])
018
=== added file 'hooks/charmhelpers/core/kernel_factory/ubuntu.py'
--- hooks/charmhelpers/core/kernel_factory/ubuntu.py 1970-01-01 00:00:00 +0000
+++ hooks/charmhelpers/core/kernel_factory/ubuntu.py 2018-06-12 20:07:13 +0000
@@ -0,0 +1,13 @@
1import subprocess
2
3
4def persistent_modprobe(module):
5 """Load a kernel module and configure for auto-load on reboot."""
6 with open('/etc/modules', 'r+') as modules:
7 if module not in modules.read():
8 modules.write(module + "\n")
9
10
11def update_initramfs(version='all'):
12 """Updates an initramfs image."""
13 return subprocess.check_call(["update-initramfs", "-k", version, "-u"])
014
=== modified file 'hooks/charmhelpers/core/services/__init__.py'
--- hooks/charmhelpers/core/services/__init__.py 2015-02-09 12:58:07 +0000
+++ hooks/charmhelpers/core/services/__init__.py 2018-06-12 20:07:13 +0000
@@ -1,18 +1,16 @@
1# Copyright 2014-2015 Canonical Limited.1# Copyright 2014-2015 Canonical Limited.
2#2#
3# This file is part of charm-helpers.3# Licensed under the Apache License, Version 2.0 (the "License");
4#4# you may not use this file except in compliance with the License.
5# charm-helpers is free software: you can redistribute it and/or modify5# You may obtain a copy of the License at
6# it under the terms of the GNU Lesser General Public License version 3 as6#
7# published by the Free Software Foundation.7# http://www.apache.org/licenses/LICENSE-2.0
8#8#
9# charm-helpers is distributed in the hope that it will be useful,9# Unless required by applicable law or agreed to in writing, software
10# but WITHOUT ANY WARRANTY; without even the implied warranty of10# distributed under the License is distributed on an "AS IS" BASIS,
11# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the11# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12# GNU Lesser General Public License for more details.12# See the License for the specific language governing permissions and
13#13# limitations under the License.
14# You should have received a copy of the GNU Lesser General Public License
15# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1614
17from .base import * # NOQA15from .base import * # NOQA
18from .helpers import * # NOQA16from .helpers import * # NOQA
1917
=== modified file 'hooks/charmhelpers/core/services/base.py'
--- hooks/charmhelpers/core/services/base.py 2015-05-14 10:48:09 +0000
+++ hooks/charmhelpers/core/services/base.py 2018-06-12 20:07:13 +0000
@@ -1,22 +1,20 @@
1# Copyright 2014-2015 Canonical Limited.1# Copyright 2014-2015 Canonical Limited.
2#2#
3# This file is part of charm-helpers.3# Licensed under the Apache License, Version 2.0 (the "License");
4#4# you may not use this file except in compliance with the License.
5# charm-helpers is free software: you can redistribute it and/or modify5# You may obtain a copy of the License at
6# it under the terms of the GNU Lesser General Public License version 3 as6#
7# published by the Free Software Foundation.7# http://www.apache.org/licenses/LICENSE-2.0
8#8#
9# charm-helpers is distributed in the hope that it will be useful,9# Unless required by applicable law or agreed to in writing, software
10# but WITHOUT ANY WARRANTY; without even the implied warranty of10# distributed under the License is distributed on an "AS IS" BASIS,
11# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the11# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12# GNU Lesser General Public License for more details.12# See the License for the specific language governing permissions and
13#13# limitations under the License.
14# You should have received a copy of the GNU Lesser General Public License
15# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1614
17import os15import os
18import re
19import json16import json
17from inspect import getargspec
20from collections import Iterable, OrderedDict18from collections import Iterable, OrderedDict
2119
22from charmhelpers.core import host20from charmhelpers.core import host
@@ -128,15 +126,18 @@
128 """126 """
129 Handle the current hook by doing The Right Thing with the registered services.127 Handle the current hook by doing The Right Thing with the registered services.
130 """128 """
131 hook_name = hookenv.hook_name()129 hookenv._run_atstart()
132 if hook_name == 'stop':130 try:
133 self.stop_services()131 hook_name = hookenv.hook_name()
134 else:132 if hook_name == 'stop':
135 self.provide_data()133 self.stop_services()
136 self.reconfigure_services()134 else:
137 cfg = hookenv.config()135 self.reconfigure_services()
138 if cfg.implicit_save:136 self.provide_data()
139 cfg.save()137 except SystemExit as x:
138 if x.code is None or x.code == 0:
139 hookenv._run_atexit()
140 hookenv._run_atexit()
140141
141 def provide_data(self):142 def provide_data(self):
142 """143 """
@@ -145,15 +146,36 @@
145 A provider must have a `name` attribute, which indicates which relation146 A provider must have a `name` attribute, which indicates which relation
146 to set data on, and a `provide_data()` method, which returns a dict of147 to set data on, and a `provide_data()` method, which returns a dict of
147 data to set.148 data to set.
149
150 The `provide_data()` method can optionally accept two parameters:
151
152 * ``remote_service`` The name of the remote service that the data will
153 be provided to. The `provide_data()` method will be called once
154 for each connected service (not unit). This allows the method to
155 tailor its data to the given service.
156 * ``service_ready`` Whether or not the service definition had all of
157 its requirements met, and thus the ``data_ready`` callbacks run.
158
159 Note that the ``provided_data`` methods are now called **after** the
160 ``data_ready`` callbacks are run. This gives the ``data_ready`` callbacks
161 a chance to generate any data necessary for the providing to the remote
162 services.
148 """163 """
149 hook_name = hookenv.hook_name()164 for service_name, service in self.services.items():
150 for service in self.services.values():165 service_ready = self.is_ready(service_name)
151 for provider in service.get('provided_data', []):166 for provider in service.get('provided_data', []):
152 if re.match(r'{}-relation-(joined|changed)'.format(provider.name), hook_name):167 for relid in hookenv.relation_ids(provider.name):
153 data = provider.provide_data()168 units = hookenv.related_units(relid)
154 _ready = provider._is_ready(data) if hasattr(provider, '_is_ready') else data169 if not units:
155 if _ready:170 continue
156 hookenv.relation_set(None, data)171 remote_service = units[0].split('/')[0]
172 argspec = getargspec(provider.provide_data)
173 if len(argspec.args) > 1:
174 data = provider.provide_data(remote_service, service_ready)
175 else:
176 data = provider.provide_data()
177 if data:
178 hookenv.relation_set(relid, data)
157179
158 def reconfigure_services(self, *service_names):180 def reconfigure_services(self, *service_names):
159 """181 """
160182
=== modified file 'hooks/charmhelpers/core/services/helpers.py'
--- hooks/charmhelpers/core/services/helpers.py 2015-05-14 10:48:09 +0000
+++ hooks/charmhelpers/core/services/helpers.py 2018-06-12 20:07:13 +0000
@@ -1,22 +1,22 @@
1# Copyright 2014-2015 Canonical Limited.1# Copyright 2014-2015 Canonical Limited.
2#2#
3# This file is part of charm-helpers.3# Licensed under the Apache License, Version 2.0 (the "License");
4#4# you may not use this file except in compliance with the License.
5# charm-helpers is free software: you can redistribute it and/or modify5# You may obtain a copy of the License at
6# it under the terms of the GNU Lesser General Public License version 3 as6#
7# published by the Free Software Foundation.7# http://www.apache.org/licenses/LICENSE-2.0
8#8#
9# charm-helpers is distributed in the hope that it will be useful,9# Unless required by applicable law or agreed to in writing, software
10# but WITHOUT ANY WARRANTY; without even the implied warranty of10# distributed under the License is distributed on an "AS IS" BASIS,
11# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the11# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12# GNU Lesser General Public License for more details.12# See the License for the specific language governing permissions and
13#13# limitations under the License.
14# You should have received a copy of the GNU Lesser General Public License
15# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1614
17import os15import os
18import yaml16import yaml
17
19from charmhelpers.core import hookenv18from charmhelpers.core import hookenv
19from charmhelpers.core import host
20from charmhelpers.core import templating20from charmhelpers.core import templating
2121
22from charmhelpers.core.services.base import ManagerCallback22from charmhelpers.core.services.base import ManagerCallback
@@ -239,28 +239,51 @@
239 action.239 action.
240240
241 :param str source: The template source file, relative to241 :param str source: The template source file, relative to
242 `$CHARM_DIR/templates`242 `$CHARM_DIR/templates`
243243
244 :param str target: The target to write the rendered template to244 :param str target: The target to write the rendered template to (or None)
245 :param str owner: The owner of the rendered file245 :param str owner: The owner of the rendered file
246 :param str group: The group of the rendered file246 :param str group: The group of the rendered file
247 :param int perms: The permissions of the rendered file247 :param int perms: The permissions of the rendered file
248 :param partial on_change_action: functools partial to be executed when
249 rendered file changes
250 :param jinja2 loader template_loader: A jinja2 template loader
251
252 :return str: The rendered template
248 """253 """
249 def __init__(self, source, target,254 def __init__(self, source, target,
250 owner='root', group='root', perms=0o444):255 owner='root', group='root', perms=0o444,
256 on_change_action=None, template_loader=None):
251 self.source = source257 self.source = source
252 self.target = target258 self.target = target
253 self.owner = owner259 self.owner = owner
254 self.group = group260 self.group = group
255 self.perms = perms261 self.perms = perms
262 self.on_change_action = on_change_action
263 self.template_loader = template_loader
256264
257 def __call__(self, manager, service_name, event_name):265 def __call__(self, manager, service_name, event_name):
266 pre_checksum = ''
267 if self.on_change_action and os.path.isfile(self.target):
268 pre_checksum = host.file_hash(self.target)
258 service = manager.get_service(service_name)269 service = manager.get_service(service_name)
259 context = {}270 context = {'ctx': {}}
260 for ctx in service.get('required_data', []):271 for ctx in service.get('required_data', []):
261 context.update(ctx)272 context.update(ctx)
262 templating.render(self.source, self.target, context,273 context['ctx'].update(ctx)
263 self.owner, self.group, self.perms)274
275 result = templating.render(self.source, self.target, context,
276 self.owner, self.group, self.perms,
277 template_loader=self.template_loader)
278 if self.on_change_action:
279 if pre_checksum == host.file_hash(self.target):
280 hookenv.log(
281 'No change detected: {}'.format(self.target),
282 hookenv.DEBUG)
283 else:
284 self.on_change_action()
285
286 return result
264287
265288
266# Convenience aliases for templates289# Convenience aliases for templates
267290
=== modified file 'hooks/charmhelpers/core/strutils.py'
--- hooks/charmhelpers/core/strutils.py 2015-05-14 10:48:09 +0000
+++ hooks/charmhelpers/core/strutils.py 2018-06-12 20:07:13 +0000
@@ -3,21 +3,20 @@
33
4# Copyright 2014-2015 Canonical Limited.4# Copyright 2014-2015 Canonical Limited.
5#5#
6# This file is part of charm-helpers.6# Licensed under the Apache License, Version 2.0 (the "License");
7#7# you may not use this file except in compliance with the License.
8# charm-helpers is free software: you can redistribute it and/or modify8# You may obtain a copy of the License at
9# it under the terms of the GNU Lesser General Public License version 3 as9#
10# published by the Free Software Foundation.10# http://www.apache.org/licenses/LICENSE-2.0
11#11#
12# charm-helpers is distributed in the hope that it will be useful,12# Unless required by applicable law or agreed to in writing, software
13# but WITHOUT ANY WARRANTY; without even the implied warranty of13# distributed under the License is distributed on an "AS IS" BASIS,
14# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the14# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
15# GNU Lesser General Public License for more details.15# See the License for the specific language governing permissions and
16#16# limitations under the License.
17# You should have received a copy of the GNU Lesser General Public License
18# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1917
20import six18import six
19import re
2120
2221
23def bool_from_string(value):22def bool_from_string(value):
@@ -40,3 +39,32 @@
4039
41 msg = "Unable to interpret string value '%s' as boolean" % (value)40 msg = "Unable to interpret string value '%s' as boolean" % (value)
42 raise ValueError(msg)41 raise ValueError(msg)
42
43
44def bytes_from_string(value):
45 """Interpret human readable string value as bytes.
46
47 Returns int
48 """
49 BYTE_POWER = {
50 'K': 1,
51 'KB': 1,
52 'M': 2,
53 'MB': 2,
54 'G': 3,
55 'GB': 3,
56 'T': 4,
57 'TB': 4,
58 'P': 5,
59 'PB': 5,
60 }
61 if isinstance(value, six.string_types):
62 value = six.text_type(value)
63 else:
64 msg = "Unable to interpret non-string value '%s' as boolean" % (value)
65 raise ValueError(msg)
66 matches = re.match("([0-9]+)([a-zA-Z]+)", value)
67 if not matches:
68 msg = "Unable to interpret string value '%s' as bytes" % (value)
69 raise ValueError(msg)
70 return int(matches.group(1)) * (1024 ** BYTE_POWER[matches.group(2)])
4371
=== modified file 'hooks/charmhelpers/core/sysctl.py'
--- hooks/charmhelpers/core/sysctl.py 2015-05-14 10:48:09 +0000
+++ hooks/charmhelpers/core/sysctl.py 2018-06-12 20:07:13 +0000
@@ -3,19 +3,17 @@
33
4# Copyright 2014-2015 Canonical Limited.4# Copyright 2014-2015 Canonical Limited.
5#5#
6# This file is part of charm-helpers.6# Licensed under the Apache License, Version 2.0 (the "License");
7#7# you may not use this file except in compliance with the License.
8# charm-helpers is free software: you can redistribute it and/or modify8# You may obtain a copy of the License at
9# it under the terms of the GNU Lesser General Public License version 3 as9#
10# published by the Free Software Foundation.10# http://www.apache.org/licenses/LICENSE-2.0
11#11#
12# charm-helpers is distributed in the hope that it will be useful,12# Unless required by applicable law or agreed to in writing, software
13# but WITHOUT ANY WARRANTY; without even the implied warranty of13# distributed under the License is distributed on an "AS IS" BASIS,
14# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the14# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
15# GNU Lesser General Public License for more details.15# See the License for the specific language governing permissions and
16#16# limitations under the License.
17# You should have received a copy of the GNU Lesser General Public License
18# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1917
20import yaml18import yaml
2119
2220
=== modified file 'hooks/charmhelpers/core/templating.py'
--- hooks/charmhelpers/core/templating.py 2015-02-09 12:58:07 +0000
+++ hooks/charmhelpers/core/templating.py 2018-06-12 20:07:13 +0000
@@ -1,33 +1,33 @@
1# Copyright 2014-2015 Canonical Limited.1# Copyright 2014-2015 Canonical Limited.
2#2#
3# This file is part of charm-helpers.3# Licensed under the Apache License, Version 2.0 (the "License");
4#4# you may not use this file except in compliance with the License.
5# charm-helpers is free software: you can redistribute it and/or modify5# You may obtain a copy of the License at
6# it under the terms of the GNU Lesser General Public License version 3 as6#
7# published by the Free Software Foundation.7# http://www.apache.org/licenses/LICENSE-2.0
8#8#
9# charm-helpers is distributed in the hope that it will be useful,9# Unless required by applicable law or agreed to in writing, software
10# but WITHOUT ANY WARRANTY; without even the implied warranty of10# distributed under the License is distributed on an "AS IS" BASIS,
11# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the11# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12# GNU Lesser General Public License for more details.12# See the License for the specific language governing permissions and
13#13# limitations under the License.
14# You should have received a copy of the GNU Lesser General Public License
15# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1614
17import os15import os
16import sys
1817
19from charmhelpers.core import host18from charmhelpers.core import host
20from charmhelpers.core import hookenv19from charmhelpers.core import hookenv
2120
2221
23def render(source, target, context, owner='root', group='root',22def render(source, target, context, owner='root', group='root',
24 perms=0o444, templates_dir=None, encoding='UTF-8'):23 perms=0o444, templates_dir=None, encoding='UTF-8', template_loader=None):
25 """24 """
26 Render a template.25 Render a template.
2726
28 The `source` path, if not absolute, is relative to the `templates_dir`.27 The `source` path, if not absolute, is relative to the `templates_dir`.
2928
30 The `target` path should be absolute.29 The `target` path should be absolute. It can also be `None`, in which
30 case no file will be written.
3131
32 The context should be a dict containing the values to be replaced in the32 The context should be a dict containing the values to be replaced in the
33 template.33 template.
@@ -36,8 +36,12 @@
3636
37 If omitted, `templates_dir` defaults to the `templates` folder in the charm.37 If omitted, `templates_dir` defaults to the `templates` folder in the charm.
3838
39 Note: Using this requires python-jinja2; if it is not installed, calling39 The rendered template will be written to the file as well as being returned
40 this will attempt to use charmhelpers.fetch.apt_install to install it.40 as a string.
41
42 Note: Using this requires python-jinja2 or python3-jinja2; if it is not
43 installed, calling this will attempt to use charmhelpers.fetch.apt_install
44 to install it.
41 """45 """
42 try:46 try:
43 from jinja2 import FileSystemLoader, Environment, exceptions47 from jinja2 import FileSystemLoader, Environment, exceptions
@@ -49,20 +53,32 @@
49 'charmhelpers.fetch to install it',53 'charmhelpers.fetch to install it',
50 level=hookenv.ERROR)54 level=hookenv.ERROR)
51 raise55 raise
52 apt_install('python-jinja2', fatal=True)56 if sys.version_info.major == 2:
57 apt_install('python-jinja2', fatal=True)
58 else:
59 apt_install('python3-jinja2', fatal=True)
53 from jinja2 import FileSystemLoader, Environment, exceptions60 from jinja2 import FileSystemLoader, Environment, exceptions
5461
55 if templates_dir is None:62 if template_loader:
56 templates_dir = os.path.join(hookenv.charm_dir(), 'templates')63 template_env = Environment(loader=template_loader)
57 loader = Environment(loader=FileSystemLoader(templates_dir))64 else:
65 if templates_dir is None:
66 templates_dir = os.path.join(hookenv.charm_dir(), 'templates')
67 template_env = Environment(loader=FileSystemLoader(templates_dir))
58 try:68 try:
59 source = source69 source = source
60 template = loader.get_template(source)70 template = template_env.get_template(source)
61 except exceptions.TemplateNotFound as e:71 except exceptions.TemplateNotFound as e:
62 hookenv.log('Could not load template %s from %s.' %72 hookenv.log('Could not load template %s from %s.' %
63 (source, templates_dir),73 (source, templates_dir),
64 level=hookenv.ERROR)74 level=hookenv.ERROR)
65 raise e75 raise e
66 content = template.render(context)76 content = template.render(context)
67 host.mkdir(os.path.dirname(target), owner, group, perms=0o755)77 if target is not None:
68 host.write_file(target, content.encode(encoding), owner, group, perms)78 target_dir = os.path.dirname(target)
79 if not os.path.exists(target_dir):
80 # This is a terrible default directory permission, as the file
81 # or its siblings will often contain secrets.
82 host.mkdir(os.path.dirname(target), owner, group, perms=0o755)
83 host.write_file(target, content.encode(encoding), owner, group, perms)
84 return content
6985
=== modified file 'hooks/charmhelpers/core/unitdata.py'
--- hooks/charmhelpers/core/unitdata.py 2015-05-14 10:48:09 +0000
+++ hooks/charmhelpers/core/unitdata.py 2018-06-12 20:07:13 +0000
@@ -3,20 +3,17 @@
3#3#
4# Copyright 2014-2015 Canonical Limited.4# Copyright 2014-2015 Canonical Limited.
5#5#
6# This file is part of charm-helpers.6# Licensed under the Apache License, Version 2.0 (the "License");
7#7# you may not use this file except in compliance with the License.
8# charm-helpers is free software: you can redistribute it and/or modify8# You may obtain a copy of the License at
9# it under the terms of the GNU Lesser General Public License version 3 as9#
10# published by the Free Software Foundation.10# http://www.apache.org/licenses/LICENSE-2.0
11#11#
12# charm-helpers is distributed in the hope that it will be useful,12# Unless required by applicable law or agreed to in writing, software
13# but WITHOUT ANY WARRANTY; without even the implied warranty of13# distributed under the License is distributed on an "AS IS" BASIS,
14# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the14# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
15# GNU Lesser General Public License for more details.15# See the License for the specific language governing permissions and
16#16# limitations under the License.
17# You should have received a copy of the GNU Lesser General Public License
18# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
19#
20#17#
21# Authors:18# Authors:
22# Kapil Thangavelu <kapil.foss@gmail.com>19# Kapil Thangavelu <kapil.foss@gmail.com>
@@ -152,6 +149,7 @@
152import collections149import collections
153import contextlib150import contextlib
154import datetime151import datetime
152import itertools
155import json153import json
156import os154import os
157import pprint155import pprint
@@ -164,8 +162,7 @@
164class Storage(object):162class Storage(object):
165 """Simple key value database for local unit state within charms.163 """Simple key value database for local unit state within charms.
166164
167 Modifications are automatically committed at hook exit. That's165 Modifications are not persisted unless :meth:`flush` is called.
168 currently regardless of exit code.
169166
170 To support dicts, lists, integer, floats, and booleans values167 To support dicts, lists, integer, floats, and booleans values
171 are automatically json encoded/decoded.168 are automatically json encoded/decoded.
@@ -173,8 +170,11 @@
173 def __init__(self, path=None):170 def __init__(self, path=None):
174 self.db_path = path171 self.db_path = path
175 if path is None:172 if path is None:
176 self.db_path = os.path.join(173 if 'UNIT_STATE_DB' in os.environ:
177 os.environ.get('CHARM_DIR', ''), '.unit-state.db')174 self.db_path = os.environ['UNIT_STATE_DB']
175 else:
176 self.db_path = os.path.join(
177 os.environ.get('CHARM_DIR', ''), '.unit-state.db')
178 self.conn = sqlite3.connect('%s' % self.db_path)178 self.conn = sqlite3.connect('%s' % self.db_path)
179 self.cursor = self.conn.cursor()179 self.cursor = self.conn.cursor()
180 self.revision = None180 self.revision = None
@@ -189,15 +189,8 @@
189 self.conn.close()189 self.conn.close()
190 self._closed = True190 self._closed = True
191191
192 def _scoped_query(self, stmt, params=None):
193 if params is None:
194 params = []
195 return stmt, params
196
197 def get(self, key, default=None, record=False):192 def get(self, key, default=None, record=False):
198 self.cursor.execute(193 self.cursor.execute('select data from kv where key=?', [key])
199 *self._scoped_query(
200 'select data from kv where key=?', [key]))
201 result = self.cursor.fetchone()194 result = self.cursor.fetchone()
202 if not result:195 if not result:
203 return default196 return default
@@ -206,33 +199,81 @@
206 return json.loads(result[0])199 return json.loads(result[0])
207200
208 def getrange(self, key_prefix, strip=False):201 def getrange(self, key_prefix, strip=False):
209 stmt = "select key, data from kv where key like '%s%%'" % key_prefix202 """
210 self.cursor.execute(*self._scoped_query(stmt))203 Get a range of keys starting with a common prefix as a mapping of
204 keys to values.
205
206 :param str key_prefix: Common prefix among all keys
207 :param bool strip: Optionally strip the common prefix from the key
208 names in the returned dict
209 :return dict: A (possibly empty) dict of key-value mappings
210 """
211 self.cursor.execute("select key, data from kv where key like ?",
212 ['%s%%' % key_prefix])
211 result = self.cursor.fetchall()213 result = self.cursor.fetchall()
212214
213 if not result:215 if not result:
214 return None216 return {}
215 if not strip:217 if not strip:
216 key_prefix = ''218 key_prefix = ''
217 return dict([219 return dict([
218 (k[len(key_prefix):], json.loads(v)) for k, v in result])220 (k[len(key_prefix):], json.loads(v)) for k, v in result])
219221
220 def update(self, mapping, prefix=""):222 def update(self, mapping, prefix=""):
223 """
224 Set the values of multiple keys at once.
225
226 :param dict mapping: Mapping of keys to values
227 :param str prefix: Optional prefix to apply to all keys in `mapping`
228 before setting
229 """
221 for k, v in mapping.items():230 for k, v in mapping.items():
222 self.set("%s%s" % (prefix, k), v)231 self.set("%s%s" % (prefix, k), v)
223232
224 def unset(self, key):233 def unset(self, key):
234 """
235 Remove a key from the database entirely.
236 """
225 self.cursor.execute('delete from kv where key=?', [key])237 self.cursor.execute('delete from kv where key=?', [key])
226 if self.revision and self.cursor.rowcount:238 if self.revision and self.cursor.rowcount:
227 self.cursor.execute(239 self.cursor.execute(
228 'insert into kv_revisions values (?, ?, ?)',240 'insert into kv_revisions values (?, ?, ?)',
229 [key, self.revision, json.dumps('DELETED')])241 [key, self.revision, json.dumps('DELETED')])
230242
243 def unsetrange(self, keys=None, prefix=""):
244 """
245 Remove a range of keys starting with a common prefix, from the database
246 entirely.
247
248 :param list keys: List of keys to remove.
249 :param str prefix: Optional prefix to apply to all keys in ``keys``
250 before removing.
251 """
252 if keys is not None:
253 keys = ['%s%s' % (prefix, key) for key in keys]
254 self.cursor.execute('delete from kv where key in (%s)' % ','.join(['?'] * len(keys)), keys)
255 if self.revision and self.cursor.rowcount:
256 self.cursor.execute(
257 'insert into kv_revisions values %s' % ','.join(['(?, ?, ?)'] * len(keys)),
258 list(itertools.chain.from_iterable((key, self.revision, json.dumps('DELETED')) for key in keys)))
259 else:
260 self.cursor.execute('delete from kv where key like ?',
261 ['%s%%' % prefix])
262 if self.revision and self.cursor.rowcount:
263 self.cursor.execute(
264 'insert into kv_revisions values (?, ?, ?)',
265 ['%s%%' % prefix, self.revision, json.dumps('DELETED')])
266
231 def set(self, key, value):267 def set(self, key, value):
268 """
269 Set a value in the database.
270
271 :param str key: Key to set the value for
272 :param value: Any JSON-serializable value to be set
273 """
232 serialized = json.dumps(value)274 serialized = json.dumps(value)
233275
234 self.cursor.execute(276 self.cursor.execute('select data from kv where key=?', [key])
235 'select data from kv where key=?', [key])
236 exists = self.cursor.fetchone()277 exists = self.cursor.fetchone()
237278
238 # Skip mutations to the same value279 # Skip mutations to the same value
239280
=== modified file 'hooks/charmhelpers/fetch/__init__.py'
--- hooks/charmhelpers/fetch/__init__.py 2016-02-12 18:30:48 +0000
+++ hooks/charmhelpers/fetch/__init__.py 2018-06-12 20:07:13 +0000
@@ -1,32 +1,24 @@
1# Copyright 2014-2015 Canonical Limited.1# Copyright 2014-2015 Canonical Limited.
2#2#
3# This file is part of charm-helpers.3# Licensed under the Apache License, Version 2.0 (the "License");
4#4# you may not use this file except in compliance with the License.
5# charm-helpers is free software: you can redistribute it and/or modify5# You may obtain a copy of the License at
6# it under the terms of the GNU Lesser General Public License version 3 as6#
7# published by the Free Software Foundation.7# http://www.apache.org/licenses/LICENSE-2.0
8#8#
9# charm-helpers is distributed in the hope that it will be useful,9# Unless required by applicable law or agreed to in writing, software
10# but WITHOUT ANY WARRANTY; without even the implied warranty of10# distributed under the License is distributed on an "AS IS" BASIS,
11# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the11# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12# GNU Lesser General Public License for more details.12# See the License for the specific language governing permissions and
13#13# limitations under the License.
14# You should have received a copy of the GNU Lesser General Public License
15# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1614
17import importlib15import importlib
18from tempfile import NamedTemporaryFile16from charmhelpers.osplatform import get_platform
19import time
20from yaml import safe_load17from yaml import safe_load
21from charmhelpers.core.host import (
22 lsb_release
23)
24import subprocess
25from charmhelpers.core.hookenv import (18from charmhelpers.core.hookenv import (
26 config,19 config,
27 log,20 log,
28)21)
29import os
3022
31import six23import six
32if six.PY3:24if six.PY3:
@@ -35,71 +27,6 @@
35 from urlparse import urlparse, urlunparse27 from urlparse import urlparse, urlunparse
3628
3729
38CLOUD_ARCHIVE = """# Ubuntu Cloud Archive
39deb http://ubuntu-cloud.archive.canonical.com/ubuntu {} main
40"""
41PROPOSED_POCKET = """# Proposed
42deb http://archive.ubuntu.com/ubuntu {}-proposed main universe multiverse restricted
43"""
44CLOUD_ARCHIVE_POCKETS = {
45 # Folsom
46 'folsom': 'precise-updates/folsom',
47 'precise-folsom': 'precise-updates/folsom',
48 'precise-folsom/updates': 'precise-updates/folsom',
49 'precise-updates/folsom': 'precise-updates/folsom',
50 'folsom/proposed': 'precise-proposed/folsom',
51 'precise-folsom/proposed': 'precise-proposed/folsom',
52 'precise-proposed/folsom': 'precise-proposed/folsom',
53 # Grizzly
54 'grizzly': 'precise-updates/grizzly',
55 'precise-grizzly': 'precise-updates/grizzly',
56 'precise-grizzly/updates': 'precise-updates/grizzly',
57 'precise-updates/grizzly': 'precise-updates/grizzly',
58 'grizzly/proposed': 'precise-proposed/grizzly',
59 'precise-grizzly/proposed': 'precise-proposed/grizzly',
60 'precise-proposed/grizzly': 'precise-proposed/grizzly',
61 # Havana
62 'havana': 'precise-updates/havana',
63 'precise-havana': 'precise-updates/havana',
64 'precise-havana/updates': 'precise-updates/havana',
65 'precise-updates/havana': 'precise-updates/havana',
66 'havana/proposed': 'precise-proposed/havana',
67 'precise-havana/proposed': 'precise-proposed/havana',
68 'precise-proposed/havana': 'precise-proposed/havana',
69 # Icehouse
70 'icehouse': 'precise-updates/icehouse',
71 'precise-icehouse': 'precise-updates/icehouse',
72 'precise-icehouse/updates': 'precise-updates/icehouse',
73 'precise-updates/icehouse': 'precise-updates/icehouse',
74 'icehouse/proposed': 'precise-proposed/icehouse',
75 'precise-icehouse/proposed': 'precise-proposed/icehouse',
76 'precise-proposed/icehouse': 'precise-proposed/icehouse',
77 # Juno
78 'juno': 'trusty-updates/juno',
79 'trusty-juno': 'trusty-updates/juno',
80 'trusty-juno/updates': 'trusty-updates/juno',
81 'trusty-updates/juno': 'trusty-updates/juno',
82 'juno/proposed': 'trusty-proposed/juno',
83 'trusty-juno/proposed': 'trusty-proposed/juno',
84 'trusty-proposed/juno': 'trusty-proposed/juno',
85 # Kilo
86 'kilo': 'trusty-updates/kilo',
87 'trusty-kilo': 'trusty-updates/kilo',
88 'trusty-kilo/updates': 'trusty-updates/kilo',
89 'trusty-updates/kilo': 'trusty-updates/kilo',
90 'kilo/proposed': 'trusty-proposed/kilo',
91 'trusty-kilo/proposed': 'trusty-proposed/kilo',
92 'trusty-proposed/kilo': 'trusty-proposed/kilo',
93 # Liberty
94 'liberty': 'trusty-updates/liberty',
95 'trusty-liberty': 'trusty-updates/liberty',
96 'trusty-liberty/updates': 'trusty-updates/liberty',
97 'trusty-updates/liberty': 'trusty-updates/liberty',
98 'liberty/proposed': 'trusty-proposed/liberty',
99 'trusty-liberty/proposed': 'trusty-proposed/liberty',
100 'trusty-proposed/liberty': 'trusty-proposed/liberty',
101}
102
103# The order of this list is very important. Handlers should be listed in from30# The order of this list is very important. Handlers should be listed in from
104# least- to most-specific URL matching.31# least- to most-specific URL matching.
105FETCH_HANDLERS = (32FETCH_HANDLERS = (
@@ -108,10 +35,6 @@
108 'charmhelpers.fetch.giturl.GitUrlFetchHandler',35 'charmhelpers.fetch.giturl.GitUrlFetchHandler',
109)36)
11037
111APT_NO_LOCK = 100 # The return code for "couldn't acquire lock" in APT.
112APT_NO_LOCK_RETRY_DELAY = 10 # Wait 10 seconds between apt lock checks.
113APT_NO_LOCK_RETRY_COUNT = 30 # Retry to acquire the lock X times.
114
11538
116class SourceConfigError(Exception):39class SourceConfigError(Exception):
117 pass40 pass
@@ -149,172 +72,38 @@
149 return urlunparse(parts)72 return urlunparse(parts)
15073
15174
152def filter_installed_packages(packages):75__platform__ = get_platform()
153 """Returns a list of packages that require installation"""76module = "charmhelpers.fetch.%s" % __platform__
154 cache = apt_cache()77fetch = importlib.import_module(module)
155 _pkgs = []78
156 for package in packages:79filter_installed_packages = fetch.filter_installed_packages
157 try:80install = fetch.install
158 p = cache[package]81upgrade = fetch.upgrade
159 p.current_ver or _pkgs.append(package)82update = fetch.update
160 except KeyError:83purge = fetch.purge
161 log('Package {} has no installation candidate.'.format(package),84add_source = fetch.add_source
162 level='WARNING')85
163 _pkgs.append(package)86if __platform__ == "ubuntu":
164 return _pkgs87 apt_cache = fetch.apt_cache
16588 apt_install = fetch.install
16689 apt_update = fetch.update
167def apt_cache(in_memory=True):90 apt_upgrade = fetch.upgrade
168 """Build and return an apt cache"""91 apt_purge = fetch.purge
169 from apt import apt_pkg92 apt_mark = fetch.apt_mark
170 apt_pkg.init()93 apt_hold = fetch.apt_hold
171 if in_memory:94 apt_unhold = fetch.apt_unhold
172 apt_pkg.config.set("Dir::Cache::pkgcache", "")95 get_upstream_version = fetch.get_upstream_version
173 apt_pkg.config.set("Dir::Cache::srcpkgcache", "")96elif __platform__ == "centos":
174 return apt_pkg.Cache()97 yum_search = fetch.yum_search
175
176
177def apt_install(packages, options=None, fatal=False):
178 """Install one or more packages"""
179 if options is None:
180 options = ['--option=Dpkg::Options::=--force-confold']
181
182 cmd = ['apt-get', '--assume-yes']
183 cmd.extend(options)
184 cmd.append('install')
185 if isinstance(packages, six.string_types):
186 cmd.append(packages)
187 else:
188 cmd.extend(packages)
189 log("Installing {} with options: {}".format(packages,
190 options))
191 _run_apt_command(cmd, fatal)
192
193
194def apt_upgrade(options=None, fatal=False, dist=False):
195 """Upgrade all packages"""
196 if options is None:
197 options = ['--option=Dpkg::Options::=--force-confold']
198
199 cmd = ['apt-get', '--assume-yes']
200 cmd.extend(options)
201 if dist:
202 cmd.append('dist-upgrade')
203 else:
204 cmd.append('upgrade')
205 log("Upgrading with options: {}".format(options))
206 _run_apt_command(cmd, fatal)
207
208
209def apt_update(fatal=False):
210 """Update local apt cache"""
211 cmd = ['apt-get', 'update']
212 _run_apt_command(cmd, fatal)
213
214
215def apt_purge(packages, fatal=False):
216 """Purge one or more packages"""
217 cmd = ['apt-get', '--assume-yes', 'purge']
218 if isinstance(packages, six.string_types):
219 cmd.append(packages)
220 else:
221 cmd.extend(packages)
222 log("Purging {}".format(packages))
223 _run_apt_command(cmd, fatal)
224
225
226def apt_hold(packages, fatal=False):
227 """Hold one or more packages"""
228 cmd = ['apt-mark', 'hold']
229 if isinstance(packages, six.string_types):
230 cmd.append(packages)
231 else:
232 cmd.extend(packages)
233 log("Holding {}".format(packages))
234
235 if fatal:
236 subprocess.check_call(cmd)
237 else:
238 subprocess.call(cmd)
239
240
241def add_source(source, key=None):
242 """Add a package source to this system.
243
244 @param source: a URL or sources.list entry, as supported by
245 add-apt-repository(1). Examples::
246
247 ppa:charmers/example
248 deb https://stub:key@private.example.com/ubuntu trusty main
249
250 In addition:
251 'proposed:' may be used to enable the standard 'proposed'
252 pocket for the release.
253 'cloud:' may be used to activate official cloud archive pockets,
254 such as 'cloud:icehouse'
255 'distro' may be used as a noop
256
257 @param key: A key to be added to the system's APT keyring and used
258 to verify the signatures on packages. Ideally, this should be an
259 ASCII format GPG public key including the block headers. A GPG key
260 id may also be used, but be aware that only insecure protocols are
261 available to retrieve the actual public key from a public keyserver
262 placing your Juju environment at risk. ppa and cloud archive keys
263 are securely added automtically, so sould not be provided.
264 """
265 if source is None:
266 log('Source is not present. Skipping')
267 return
268
269 if (source.startswith('ppa:') or
270 source.startswith('http') or
271 source.startswith('deb ') or
272 source.startswith('cloud-archive:')):
273 subprocess.check_call(['add-apt-repository', '--yes', source])
274 elif source.startswith('cloud:'):
275 apt_install(filter_installed_packages(['ubuntu-cloud-keyring']),
276 fatal=True)
277 pocket = source.split(':')[-1]
278 if pocket not in CLOUD_ARCHIVE_POCKETS:
279 raise SourceConfigError(
280 'Unsupported cloud: source option %s' %
281 pocket)
282 actual_pocket = CLOUD_ARCHIVE_POCKETS[pocket]
283 with open('/etc/apt/sources.list.d/cloud-archive.list', 'w') as apt:
284 apt.write(CLOUD_ARCHIVE.format(actual_pocket))
285 elif source == 'proposed':
286 release = lsb_release()['DISTRIB_CODENAME']
287 with open('/etc/apt/sources.list.d/proposed.list', 'w') as apt:
288 apt.write(PROPOSED_POCKET.format(release))
289 elif source == 'distro':
290 pass
291 else:
292 log("Unknown source: {!r}".format(source))
293
294 if key:
295 if '-----BEGIN PGP PUBLIC KEY BLOCK-----' in key:
296 with NamedTemporaryFile('w+') as key_file:
297 key_file.write(key)
298 key_file.flush()
299 key_file.seek(0)
300 subprocess.check_call(['apt-key', 'add', '-'], stdin=key_file)
301 else:
302 # Note that hkp: is in no way a secure protocol. Using a
303 # GPG key id is pointless from a security POV unless you
304 # absolutely trust your network and DNS.
305 subprocess.check_call(['apt-key', 'adv', '--keyserver',
306 'hkp://keyserver.ubuntu.com:80', '--recv',
307 key])
30898
30999
310def configure_sources(update=False,100def configure_sources(update=False,
311 sources_var='install_sources',101 sources_var='install_sources',
312 keys_var='install_keys'):102 keys_var='install_keys'):
313 """103 """Configure multiple sources from charm configuration.
314 Configure multiple sources from charm configuration.
315104
316 The lists are encoded as yaml fragments in the configuration.105 The lists are encoded as yaml fragments in the configuration.
317 The frament needs to be included as a string. Sources and their106 The fragment needs to be included as a string. Sources and their
318 corresponding keys are of the types supported by add_source().107 corresponding keys are of the types supported by add_source().
319108
320 Example config:109 Example config:
@@ -346,12 +135,11 @@
346 for source, key in zip(sources, keys):135 for source, key in zip(sources, keys):
347 add_source(source, key)136 add_source(source, key)
348 if update:137 if update:
349 apt_update(fatal=True)138 fetch.update(fatal=True)
350139
351140
352def install_remote(source, *args, **kwargs):141def install_remote(source, *args, **kwargs):
353 """142 """Install a file tree from a remote source.
354 Install a file tree from a remote source
355143
356 The specified source should be a url of the form:144 The specified source should be a url of the form:
357 scheme://[host]/path[#[option=value][&...]]145 scheme://[host]/path[#[option=value][&...]]
@@ -374,18 +162,17 @@
374 # We ONLY check for True here because can_handle may return a string162 # We ONLY check for True here because can_handle may return a string
375 # explaining why it can't handle a given source.163 # explaining why it can't handle a given source.
376 handlers = [h for h in plugins() if h.can_handle(source) is True]164 handlers = [h for h in plugins() if h.can_handle(source) is True]
377 installed_to = None
378 for handler in handlers:165 for handler in handlers:
379 try:166 try:
380 installed_to = handler.install(source, *args, **kwargs)167 return handler.install(source, *args, **kwargs)
381 except UnhandledSource:168 except UnhandledSource as e:
382 pass169 log('Install source attempt unsuccessful: {}'.format(e),
383 if not installed_to:170 level='WARNING')
384 raise UnhandledSource("No handler found for source {}".format(source))171 raise UnhandledSource("No handler found for source {}".format(source))
385 return installed_to
386172
387173
388def install_from_config(config_var_name):174def install_from_config(config_var_name):
175 """Install a file from config."""
389 charm_config = config()176 charm_config = config()
390 source = charm_config[config_var_name]177 source = charm_config[config_var_name]
391 return install_remote(source)178 return install_remote(source)
@@ -402,46 +189,9 @@
402 importlib.import_module(package),189 importlib.import_module(package),
403 classname)190 classname)
404 plugin_list.append(handler_class())191 plugin_list.append(handler_class())
405 except (ImportError, AttributeError):192 except NotImplementedError:
406 # Skip missing plugins so that they can be ommitted from193 # Skip missing plugins so that they can be ommitted from
407 # installation if desired194 # installation if desired
408 log("FetchHandler {} not found, skipping plugin".format(195 log("FetchHandler {} not found, skipping plugin".format(
409 handler_name))196 handler_name))
410 return plugin_list197 return plugin_list
411
412
413def _run_apt_command(cmd, fatal=False):
414 """
415 Run an APT command, checking output and retrying if the fatal flag is set
416 to True.
417
418 :param: cmd: str: The apt command to run.
419 :param: fatal: bool: Whether the command's output should be checked and
420 retried.
421 """
422 env = os.environ.copy()
423
424 if 'DEBIAN_FRONTEND' not in env:
425 env['DEBIAN_FRONTEND'] = 'noninteractive'
426
427 if fatal:
428 retry_count = 0
429 result = None
430
431 # If the command is considered "fatal", we need to retry if the apt
432 # lock was not acquired.
433
434 while result is None or result == APT_NO_LOCK:
435 try:
436 result = subprocess.check_call(cmd, env=env)
437 except subprocess.CalledProcessError as e:
438 retry_count = retry_count + 1
439 if retry_count > APT_NO_LOCK_RETRY_COUNT:
440 raise
441 result = e.returncode
442 log("Couldn't acquire DPKG lock. Will retry in {} seconds."
443 "".format(APT_NO_LOCK_RETRY_DELAY))
444 time.sleep(APT_NO_LOCK_RETRY_DELAY)
445
446 else:
447 subprocess.call(cmd, env=env)
448198
=== modified file 'hooks/charmhelpers/fetch/archiveurl.py'
--- hooks/charmhelpers/fetch/archiveurl.py 2015-05-14 10:48:09 +0000
+++ hooks/charmhelpers/fetch/archiveurl.py 2018-06-12 20:07:13 +0000
@@ -1,18 +1,16 @@
1# Copyright 2014-2015 Canonical Limited.1# Copyright 2014-2015 Canonical Limited.
2#2#
3# This file is part of charm-helpers.3# Licensed under the Apache License, Version 2.0 (the "License");
4#4# you may not use this file except in compliance with the License.
5# charm-helpers is free software: you can redistribute it and/or modify5# You may obtain a copy of the License at
6# it under the terms of the GNU Lesser General Public License version 3 as6#
7# published by the Free Software Foundation.7# http://www.apache.org/licenses/LICENSE-2.0
8#8#
9# charm-helpers is distributed in the hope that it will be useful,9# Unless required by applicable law or agreed to in writing, software
10# but WITHOUT ANY WARRANTY; without even the implied warranty of10# distributed under the License is distributed on an "AS IS" BASIS,
11# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the11# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12# GNU Lesser General Public License for more details.12# See the License for the specific language governing permissions and
13#13# limitations under the License.
14# You should have received a copy of the GNU Lesser General Public License
15# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1614
17import os15import os
18import hashlib16import hashlib
@@ -77,6 +75,8 @@
77 def can_handle(self, source):75 def can_handle(self, source):
78 url_parts = self.parse_url(source)76 url_parts = self.parse_url(source)
79 if url_parts.scheme not in ('http', 'https', 'ftp', 'file'):77 if url_parts.scheme not in ('http', 'https', 'ftp', 'file'):
78 # XXX: Why is this returning a boolean and a string? It's
79 # doomed to fail since "bool(can_handle('foo://'))" will be True.
80 return "Wrong source type"80 return "Wrong source type"
81 if get_archive_handler(self.base_url(source)):81 if get_archive_handler(self.base_url(source)):
82 return True82 return True
@@ -106,7 +106,7 @@
106 install_opener(opener)106 install_opener(opener)
107 response = urlopen(source)107 response = urlopen(source)
108 try:108 try:
109 with open(dest, 'w') as dest_file:109 with open(dest, 'wb') as dest_file:
110 dest_file.write(response.read())110 dest_file.write(response.read())
111 except Exception as e:111 except Exception as e:
112 if os.path.isfile(dest):112 if os.path.isfile(dest):
@@ -155,7 +155,11 @@
155 else:155 else:
156 algorithms = hashlib.algorithms_available156 algorithms = hashlib.algorithms_available
157 if key in algorithms:157 if key in algorithms:
158 check_hash(dld_file, value, key)158 if len(value) != 1:
159 raise TypeError(
160 "Expected 1 hash value, not %d" % len(value))
161 expected = value[0]
162 check_hash(dld_file, expected, key)
159 if checksum:163 if checksum:
160 check_hash(dld_file, checksum, hash_type)164 check_hash(dld_file, checksum, hash_type)
161 return extract(dld_file, dest)165 return extract(dld_file, dest)
162166
=== modified file 'hooks/charmhelpers/fetch/bzrurl.py'
--- hooks/charmhelpers/fetch/bzrurl.py 2015-02-09 12:53:57 +0000
+++ hooks/charmhelpers/fetch/bzrurl.py 2018-06-12 20:07:13 +0000
@@ -1,78 +1,76 @@
1# Copyright 2014-2015 Canonical Limited.1# Copyright 2014-2015 Canonical Limited.
2#2#
3# This file is part of charm-helpers.3# Licensed under the Apache License, Version 2.0 (the "License");
4#4# you may not use this file except in compliance with the License.
5# charm-helpers is free software: you can redistribute it and/or modify5# You may obtain a copy of the License at
6# it under the terms of the GNU Lesser General Public License version 3 as6#
7# published by the Free Software Foundation.7# http://www.apache.org/licenses/LICENSE-2.0
8#8#
9# charm-helpers is distributed in the hope that it will be useful,9# Unless required by applicable law or agreed to in writing, software
10# but WITHOUT ANY WARRANTY; without even the implied warranty of10# distributed under the License is distributed on an "AS IS" BASIS,
11# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the11# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12# GNU Lesser General Public License for more details.12# See the License for the specific language governing permissions and
13#13# limitations under the License.
14# You should have received a copy of the GNU Lesser General Public License
15# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1614
17import os15import os
16from subprocess import check_call
18from charmhelpers.fetch import (17from charmhelpers.fetch import (
19 BaseFetchHandler,18 BaseFetchHandler,
20 UnhandledSource19 UnhandledSource,
20 filter_installed_packages,
21 install,
21)22)
22from charmhelpers.core.host import mkdir23from charmhelpers.core.host import mkdir
2324
24import six
25if six.PY3:
26 raise ImportError('bzrlib does not support Python3')
2725
28try:26if filter_installed_packages(['bzr']) != []:
29 from bzrlib.branch import Branch27 install(['bzr'])
30 from bzrlib import bzrdir, workingtree, errors28 if filter_installed_packages(['bzr']) != []:
31except ImportError:29 raise NotImplementedError('Unable to install bzr')
32 from charmhelpers.fetch import apt_install
33 apt_install("python-bzrlib")
34 from bzrlib.branch import Branch
35 from bzrlib import bzrdir, workingtree, errors
3630
3731
38class BzrUrlFetchHandler(BaseFetchHandler):32class BzrUrlFetchHandler(BaseFetchHandler):
39 """Handler for bazaar branches via generic and lp URLs"""33 """Handler for bazaar branches via generic and lp URLs."""
34
40 def can_handle(self, source):35 def can_handle(self, source):
41 url_parts = self.parse_url(source)36 url_parts = self.parse_url(source)
42 if url_parts.scheme not in ('bzr+ssh', 'lp'):37 if url_parts.scheme not in ('bzr+ssh', 'lp', ''):
43 return False38 return False
39 elif not url_parts.scheme:
40 return os.path.exists(os.path.join(source, '.bzr'))
44 else:41 else:
45 return True42 return True
4643
47 def branch(self, source, dest):44 def branch(self, source, dest, revno=None):
48 url_parts = self.parse_url(source)
49 # If we use lp:branchname scheme we need to load plugins
50 if not self.can_handle(source):45 if not self.can_handle(source):
51 raise UnhandledSource("Cannot handle {}".format(source))46 raise UnhandledSource("Cannot handle {}".format(source))
52 if url_parts.scheme == "lp":47 cmd_opts = []
53 from bzrlib.plugin import load_plugins48 if revno:
54 load_plugins()49 cmd_opts += ['-r', str(revno)]
55 try:50 if os.path.exists(dest):
56 local_branch = bzrdir.BzrDir.create_branch_convenience(dest)51 cmd = ['bzr', 'pull']
57 except errors.AlreadyControlDirError:52 cmd += cmd_opts
58 local_branch = Branch.open(dest)53 cmd += ['--overwrite', '-d', dest, source]
59 try:54 else:
60 remote_branch = Branch.open(source)55 cmd = ['bzr', 'branch']
61 remote_branch.push(local_branch)56 cmd += cmd_opts
62 tree = workingtree.WorkingTree.open(dest)57 cmd += [source, dest]
63 tree.update()58 check_call(cmd)
64 except Exception as e:
65 raise e
6659
67 def install(self, source):60 def install(self, source, dest=None, revno=None):
68 url_parts = self.parse_url(source)61 url_parts = self.parse_url(source)
69 branch_name = url_parts.path.strip("/").split("/")[-1]62 branch_name = url_parts.path.strip("/").split("/")[-1]
70 dest_dir = os.path.join(os.environ.get('CHARM_DIR'), "fetched",63 if dest:
71 branch_name)64 dest_dir = os.path.join(dest, branch_name)
72 if not os.path.exists(dest_dir):65 else:
73 mkdir(dest_dir, perms=0o755)66 dest_dir = os.path.join(os.environ.get('CHARM_DIR'), "fetched",
67 branch_name)
68
69 if dest and not os.path.exists(dest):
70 mkdir(dest, perms=0o755)
71
74 try:72 try:
75 self.branch(source, dest_dir)73 self.branch(source, dest_dir, revno)
76 except OSError as e:74 except OSError as e:
77 raise UnhandledSource(e.strerror)75 raise UnhandledSource(e.strerror)
78 return dest_dir76 return dest_dir
7977
=== added file 'hooks/charmhelpers/fetch/centos.py'
--- hooks/charmhelpers/fetch/centos.py 1970-01-01 00:00:00 +0000
+++ hooks/charmhelpers/fetch/centos.py 2018-06-12 20:07:13 +0000
@@ -0,0 +1,171 @@
1# Copyright 2014-2015 Canonical Limited.
2#
3# Licensed under the Apache License, Version 2.0 (the "License");
4# you may not use this file except in compliance with the License.
5# You may obtain a copy of the License at
6#
7# http://www.apache.org/licenses/LICENSE-2.0
8#
9# Unless required by applicable law or agreed to in writing, software
10# distributed under the License is distributed on an "AS IS" BASIS,
11# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12# See the License for the specific language governing permissions and
13# limitations under the License.
14
15import subprocess
16import os
17import time
18import six
19import yum
20
21from tempfile import NamedTemporaryFile
22from charmhelpers.core.hookenv import log
23
24YUM_NO_LOCK = 1 # The return code for "couldn't acquire lock" in YUM.
25YUM_NO_LOCK_RETRY_DELAY = 10 # Wait 10 seconds between apt lock checks.
26YUM_NO_LOCK_RETRY_COUNT = 30 # Retry to acquire the lock X times.
27
28
29def filter_installed_packages(packages):
30 """Return a list of packages that require installation."""
31 yb = yum.YumBase()
32 package_list = yb.doPackageLists()
33 temp_cache = {p.base_package_name: 1 for p in package_list['installed']}
34
35 _pkgs = [p for p in packages if not temp_cache.get(p, False)]
36 return _pkgs
37
38
39def install(packages, options=None, fatal=False):
40 """Install one or more packages."""
41 cmd = ['yum', '--assumeyes']
42 if options is not None:
43 cmd.extend(options)
44 cmd.append('install')
45 if isinstance(packages, six.string_types):
46 cmd.append(packages)
47 else:
48 cmd.extend(packages)
49 log("Installing {} with options: {}".format(packages,
50 options))
51 _run_yum_command(cmd, fatal)
52
53
54def upgrade(options=None, fatal=False, dist=False):
55 """Upgrade all packages."""
56 cmd = ['yum', '--assumeyes']
57 if options is not None:
58 cmd.extend(options)
59 cmd.append('upgrade')
60 log("Upgrading with options: {}".format(options))
61 _run_yum_command(cmd, fatal)
62
63
64def update(fatal=False):
65 """Update local yum cache."""
66 cmd = ['yum', '--assumeyes', 'update']
67 log("Update with fatal: {}".format(fatal))
68 _run_yum_command(cmd, fatal)
69
70
71def purge(packages, fatal=False):
72 """Purge one or more packages."""
73 cmd = ['yum', '--assumeyes', 'remove']
74 if isinstance(packages, six.string_types):
75 cmd.append(packages)
76 else:
77 cmd.extend(packages)
78 log("Purging {}".format(packages))
79 _run_yum_command(cmd, fatal)
80
81
82def yum_search(packages):
83 """Search for a package."""
84 output = {}
85 cmd = ['yum', 'search']
86 if isinstance(packages, six.string_types):
87 cmd.append(packages)
88 else:
89 cmd.extend(packages)
90 log("Searching for {}".format(packages))
91 result = subprocess.check_output(cmd)
92 for package in list(packages):
93 output[package] = package in result
94 return output
95
96
97def add_source(source, key=None):
98 """Add a package source to this system.
99
100 @param source: a URL with a rpm package
101
102 @param key: A key to be added to the system's keyring and used
103 to verify the signatures on packages. Ideally, this should be an
104 ASCII format GPG public key including the block headers. A GPG key
105 id may also be used, but be aware that only insecure protocols are
106 available to retrieve the actual public key from a public keyserver
107 placing your Juju environment at risk.
108 """
109 if source is None:
110 log('Source is not present. Skipping')
111 return
112
113 if source.startswith('http'):
114 directory = '/etc/yum.repos.d/'
115 for filename in os.listdir(directory):
116 with open(directory + filename, 'r') as rpm_file:
117 if source in rpm_file.read():
118 break
119 else:
120 log("Add source: {!r}".format(source))
121 # write in the charms.repo
122 with open(directory + 'Charms.repo', 'a') as rpm_file:
123 rpm_file.write('[%s]\n' % source[7:].replace('/', '_'))
124 rpm_file.write('name=%s\n' % source[7:])
125 rpm_file.write('baseurl=%s\n\n' % source)
126 else:
127 log("Unknown source: {!r}".format(source))
128
129 if key:
130 if '-----BEGIN PGP PUBLIC KEY BLOCK-----' in key:
131 with NamedTemporaryFile('w+') as key_file:
132 key_file.write(key)
133 key_file.flush()
134 key_file.seek(0)
135 subprocess.check_call(['rpm', '--import', key_file])
136 else:
137 subprocess.check_call(['rpm', '--import', key])
138
139
140def _run_yum_command(cmd, fatal=False):
141 """Run an YUM command.
142
143 Checks the output and retry if the fatal flag is set to True.
144
145 :param: cmd: str: The yum command to run.
146 :param: fatal: bool: Whether the command's output should be checked and
147 retried.
148 """
149 env = os.environ.copy()
150
151 if fatal:
152 retry_count = 0
153 result = None
154
155 # If the command is considered "fatal", we need to retry if the yum
156 # lock was not acquired.
157
158 while result is None or result == YUM_NO_LOCK:
159 try:
160 result = subprocess.check_call(cmd, env=env)
161 except subprocess.CalledProcessError as e:
162 retry_count = retry_count + 1
163 if retry_count > YUM_NO_LOCK_RETRY_COUNT:
164 raise
165 result = e.returncode
166 log("Couldn't acquire YUM lock. Will retry in {} seconds."
167 "".format(YUM_NO_LOCK_RETRY_DELAY))
168 time.sleep(YUM_NO_LOCK_RETRY_DELAY)
169
170 else:
171 subprocess.call(cmd, env=env)
0172
=== modified file 'hooks/charmhelpers/fetch/giturl.py'
--- hooks/charmhelpers/fetch/giturl.py 2015-05-14 10:48:09 +0000
+++ hooks/charmhelpers/fetch/giturl.py 2018-06-12 20:07:13 +0000
@@ -1,58 +1,58 @@
1# Copyright 2014-2015 Canonical Limited.1# Copyright 2014-2015 Canonical Limited.
2#2#
3# This file is part of charm-helpers.3# Licensed under the Apache License, Version 2.0 (the "License");
4#4# you may not use this file except in compliance with the License.
5# charm-helpers is free software: you can redistribute it and/or modify5# You may obtain a copy of the License at
6# it under the terms of the GNU Lesser General Public License version 3 as6#
7# published by the Free Software Foundation.7# http://www.apache.org/licenses/LICENSE-2.0
8#8#
9# charm-helpers is distributed in the hope that it will be useful,9# Unless required by applicable law or agreed to in writing, software
10# but WITHOUT ANY WARRANTY; without even the implied warranty of10# distributed under the License is distributed on an "AS IS" BASIS,
11# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the11# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12# GNU Lesser General Public License for more details.12# See the License for the specific language governing permissions and
13#13# limitations under the License.
14# You should have received a copy of the GNU Lesser General Public License
15# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1614
17import os15import os
16from subprocess import check_call, CalledProcessError
18from charmhelpers.fetch import (17from charmhelpers.fetch import (
19 BaseFetchHandler,18 BaseFetchHandler,
20 UnhandledSource19 UnhandledSource,
20 filter_installed_packages,
21 install,
21)22)
22from charmhelpers.core.host import mkdir23
2324if filter_installed_packages(['git']) != []:
24import six25 install(['git'])
25if six.PY3:26 if filter_installed_packages(['git']) != []:
26 raise ImportError('GitPython does not support Python 3')27 raise NotImplementedError('Unable to install git')
27
28try:
29 from git import Repo
30except ImportError:
31 from charmhelpers.fetch import apt_install
32 apt_install("python-git")
33 from git import Repo
34
35from git.exc import GitCommandError # noqa E402
3628
3729
38class GitUrlFetchHandler(BaseFetchHandler):30class GitUrlFetchHandler(BaseFetchHandler):
39 """Handler for git branches via generic and github URLs"""31 """Handler for git branches via generic and github URLs."""
32
40 def can_handle(self, source):33 def can_handle(self, source):
41 url_parts = self.parse_url(source)34 url_parts = self.parse_url(source)
42 # TODO (mattyw) no support for ssh git@ yet35 # TODO (mattyw) no support for ssh git@ yet
43 if url_parts.scheme not in ('http', 'https', 'git'):36 if url_parts.scheme not in ('http', 'https', 'git', ''):
44 return False37 return False
38 elif not url_parts.scheme:
39 return os.path.exists(os.path.join(source, '.git'))
45 else:40 else:
46 return True41 return True
4742
48 def clone(self, source, dest, branch):43 def clone(self, source, dest, branch="master", depth=None):
49 if not self.can_handle(source):44 if not self.can_handle(source):
50 raise UnhandledSource("Cannot handle {}".format(source))45 raise UnhandledSource("Cannot handle {}".format(source))
5146
52 repo = Repo.clone_from(source, dest)47 if os.path.exists(dest):
53 repo.git.checkout(branch)48 cmd = ['git', '-C', dest, 'pull', source, branch]
49 else:
50 cmd = ['git', 'clone', source, dest, '--branch', branch]
51 if depth:
52 cmd.extend(['--depth', depth])
53 check_call(cmd)
5454
55 def install(self, source, branch="master", dest=None):55 def install(self, source, branch="master", dest=None, depth=None):
56 url_parts = self.parse_url(source)56 url_parts = self.parse_url(source)
57 branch_name = url_parts.path.strip("/").split("/")[-1]57 branch_name = url_parts.path.strip("/").split("/")[-1]
58 if dest:58 if dest:
@@ -60,12 +60,10 @@
60 else:60 else:
61 dest_dir = os.path.join(os.environ.get('CHARM_DIR'), "fetched",61 dest_dir = os.path.join(os.environ.get('CHARM_DIR'), "fetched",
62 branch_name)62 branch_name)
63 if not os.path.exists(dest_dir):
64 mkdir(dest_dir, perms=0o755)
65 try:63 try:
66 self.clone(source, dest_dir, branch)64 self.clone(source, dest_dir, branch, depth)
67 except GitCommandError as e:65 except CalledProcessError as e:
68 raise UnhandledSource(e.message)66 raise UnhandledSource(e)
69 except OSError as e:67 except OSError as e:
70 raise UnhandledSource(e.strerror)68 raise UnhandledSource(e.strerror)
71 return dest_dir69 return dest_dir
7270
=== added file 'hooks/charmhelpers/fetch/snap.py'
--- hooks/charmhelpers/fetch/snap.py 1970-01-01 00:00:00 +0000
+++ hooks/charmhelpers/fetch/snap.py 2018-06-12 20:07:13 +0000
@@ -0,0 +1,122 @@
1# Copyright 2014-2017 Canonical Limited.
2#
3# Licensed under the Apache License, Version 2.0 (the "License");
4# you may not use this file except in compliance with the License.
5# You may obtain a copy of the License at
6#
7# http://www.apache.org/licenses/LICENSE-2.0
8#
9# Unless required by applicable law or agreed to in writing, software
10# distributed under the License is distributed on an "AS IS" BASIS,
11# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12# See the License for the specific language governing permissions and
13# limitations under the License.
14"""
15Charm helpers snap for classic charms.
16
17If writing reactive charms, use the snap layer:
18https://lists.ubuntu.com/archives/snapcraft/2016-September/001114.html
19"""
20import subprocess
21from os import environ
22from time import sleep
23from charmhelpers.core.hookenv import log
24
25__author__ = 'Joseph Borg <joseph.borg@canonical.com>'
26
27SNAP_NO_LOCK = 1 # The return code for "couldn't acquire lock" in Snap (hopefully this will be improved).
28SNAP_NO_LOCK_RETRY_DELAY = 10 # Wait X seconds between Snap lock checks.
29SNAP_NO_LOCK_RETRY_COUNT = 30 # Retry to acquire the lock X times.
30
31
32class CouldNotAcquireLockException(Exception):
33 pass
34
35
36def _snap_exec(commands):
37 """
38 Execute snap commands.
39
40 :param commands: List commands
41 :return: Integer exit code
42 """
43 assert type(commands) == list
44
45 retry_count = 0
46 return_code = None
47
48 while return_code is None or return_code == SNAP_NO_LOCK:
49 try:
50 return_code = subprocess.check_call(['snap'] + commands, env=environ)
51 except subprocess.CalledProcessError as e:
52 retry_count += + 1
53 if retry_count > SNAP_NO_LOCK_RETRY_COUNT:
54 raise CouldNotAcquireLockException('Could not aquire lock after %s attempts' % SNAP_NO_LOCK_RETRY_COUNT)
55 return_code = e.returncode
56 log('Snap failed to acquire lock, trying again in %s seconds.' % SNAP_NO_LOCK_RETRY_DELAY, level='WARN')
57 sleep(SNAP_NO_LOCK_RETRY_DELAY)
58
59 return return_code
60
61
62def snap_install(packages, *flags):
63 """
64 Install a snap package.
65
66 :param packages: String or List String package name
67 :param flags: List String flags to pass to install command
68 :return: Integer return code from snap
69 """
70 if type(packages) is not list:
71 packages = [packages]
72
73 flags = list(flags)
74
75 message = 'Installing snap(s) "%s"' % ', '.join(packages)
76 if flags:
77 message += ' with option(s) "%s"' % ', '.join(flags)
78
79 log(message, level='INFO')
80 return _snap_exec(['install'] + flags + packages)
81
82
83def snap_remove(packages, *flags):
84 """
85 Remove a snap package.
86
87 :param packages: String or List String package name
88 :param flags: List String flags to pass to remove command
89 :return: Integer return code from snap
90 """
91 if type(packages) is not list:
92 packages = [packages]
93
94 flags = list(flags)
95
96 message = 'Removing snap(s) "%s"' % ', '.join(packages)
97 if flags:
98 message += ' with options "%s"' % ', '.join(flags)
99
100 log(message, level='INFO')
101 return _snap_exec(['remove'] + flags + packages)
102
103
104def snap_refresh(packages, *flags):
105 """
106 Refresh / Update snap package.
107
108 :param packages: String or List String package name
109 :param flags: List String flags to pass to refresh command
110 :return: Integer return code from snap
111 """
112 if type(packages) is not list:
113 packages = [packages]
114
115 flags = list(flags)
116
117 message = 'Refreshing snap(s) "%s"' % ', '.join(packages)
118 if flags:
119 message += ' with options "%s"' % ', '.join(flags)
120
121 log(message, level='INFO')
122 return _snap_exec(['refresh'] + flags + packages)
0123
=== added file 'hooks/charmhelpers/fetch/ubuntu.py'
--- hooks/charmhelpers/fetch/ubuntu.py 1970-01-01 00:00:00 +0000
+++ hooks/charmhelpers/fetch/ubuntu.py 2018-06-12 20:07:13 +0000
@@ -0,0 +1,364 @@
1# Copyright 2014-2015 Canonical Limited.
2#
3# Licensed under the Apache License, Version 2.0 (the "License");
4# you may not use this file except in compliance with the License.
5# You may obtain a copy of the License at
6#
7# http://www.apache.org/licenses/LICENSE-2.0
8#
9# Unless required by applicable law or agreed to in writing, software
10# distributed under the License is distributed on an "AS IS" BASIS,
11# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12# See the License for the specific language governing permissions and
13# limitations under the License.
14
15import os
16import six
17import time
18import subprocess
19
20from tempfile import NamedTemporaryFile
21from charmhelpers.core.host import (
22 lsb_release
23)
24from charmhelpers.core.hookenv import log
25from charmhelpers.fetch import SourceConfigError
26
27CLOUD_ARCHIVE = """# Ubuntu Cloud Archive
28deb http://ubuntu-cloud.archive.canonical.com/ubuntu {} main
29"""
30
31PROPOSED_POCKET = """# Proposed
32deb http://archive.ubuntu.com/ubuntu {}-proposed main universe multiverse restricted
33"""
34
35CLOUD_ARCHIVE_POCKETS = {
36 # Folsom
37 'folsom': 'precise-updates/folsom',
38 'precise-folsom': 'precise-updates/folsom',
39 'precise-folsom/updates': 'precise-updates/folsom',
40 'precise-updates/folsom': 'precise-updates/folsom',
41 'folsom/proposed': 'precise-proposed/folsom',
42 'precise-folsom/proposed': 'precise-proposed/folsom',
43 'precise-proposed/folsom': 'precise-proposed/folsom',
44 # Grizzly
45 'grizzly': 'precise-updates/grizzly',
46 'precise-grizzly': 'precise-updates/grizzly',
47 'precise-grizzly/updates': 'precise-updates/grizzly',
48 'precise-updates/grizzly': 'precise-updates/grizzly',
49 'grizzly/proposed': 'precise-proposed/grizzly',
50 'precise-grizzly/proposed': 'precise-proposed/grizzly',
51 'precise-proposed/grizzly': 'precise-proposed/grizzly',
52 # Havana
53 'havana': 'precise-updates/havana',
54 'precise-havana': 'precise-updates/havana',
55 'precise-havana/updates': 'precise-updates/havana',
56 'precise-updates/havana': 'precise-updates/havana',
57 'havana/proposed': 'precise-proposed/havana',
58 'precise-havana/proposed': 'precise-proposed/havana',
59 'precise-proposed/havana': 'precise-proposed/havana',
60 # Icehouse
61 'icehouse': 'precise-updates/icehouse',
62 'precise-icehouse': 'precise-updates/icehouse',
63 'precise-icehouse/updates': 'precise-updates/icehouse',
64 'precise-updates/icehouse': 'precise-updates/icehouse',
65 'icehouse/proposed': 'precise-proposed/icehouse',
66 'precise-icehouse/proposed': 'precise-proposed/icehouse',
67 'precise-proposed/icehouse': 'precise-proposed/icehouse',
68 # Juno
69 'juno': 'trusty-updates/juno',
70 'trusty-juno': 'trusty-updates/juno',
71 'trusty-juno/updates': 'trusty-updates/juno',
72 'trusty-updates/juno': 'trusty-updates/juno',
73 'juno/proposed': 'trusty-proposed/juno',
74 'trusty-juno/proposed': 'trusty-proposed/juno',
75 'trusty-proposed/juno': 'trusty-proposed/juno',
76 # Kilo
77 'kilo': 'trusty-updates/kilo',
78 'trusty-kilo': 'trusty-updates/kilo',
79 'trusty-kilo/updates': 'trusty-updates/kilo',
80 'trusty-updates/kilo': 'trusty-updates/kilo',
81 'kilo/proposed': 'trusty-proposed/kilo',
82 'trusty-kilo/proposed': 'trusty-proposed/kilo',
83 'trusty-proposed/kilo': 'trusty-proposed/kilo',
84 # Liberty
85 'liberty': 'trusty-updates/liberty',
86 'trusty-liberty': 'trusty-updates/liberty',
87 'trusty-liberty/updates': 'trusty-updates/liberty',
88 'trusty-updates/liberty': 'trusty-updates/liberty',
89 'liberty/proposed': 'trusty-proposed/liberty',
90 'trusty-liberty/proposed': 'trusty-proposed/liberty',
91 'trusty-proposed/liberty': 'trusty-proposed/liberty',
92 # Mitaka
93 'mitaka': 'trusty-updates/mitaka',
94 'trusty-mitaka': 'trusty-updates/mitaka',
95 'trusty-mitaka/updates': 'trusty-updates/mitaka',
96 'trusty-updates/mitaka': 'trusty-updates/mitaka',
97 'mitaka/proposed': 'trusty-proposed/mitaka',
98 'trusty-mitaka/proposed': 'trusty-proposed/mitaka',
99 'trusty-proposed/mitaka': 'trusty-proposed/mitaka',
100 # Newton
101 'newton': 'xenial-updates/newton',
102 'xenial-newton': 'xenial-updates/newton',
103 'xenial-newton/updates': 'xenial-updates/newton',
104 'xenial-updates/newton': 'xenial-updates/newton',
105 'newton/proposed': 'xenial-proposed/newton',
106 'xenial-newton/proposed': 'xenial-proposed/newton',
107 'xenial-proposed/newton': 'xenial-proposed/newton',
108 # Ocata
109 'ocata': 'xenial-updates/ocata',
110 'xenial-ocata': 'xenial-updates/ocata',
111 'xenial-ocata/updates': 'xenial-updates/ocata',
112 'xenial-updates/ocata': 'xenial-updates/ocata',
113 'ocata/proposed': 'xenial-proposed/ocata',
114 'xenial-ocata/proposed': 'xenial-proposed/ocata',
115 'xenial-ocata/newton': 'xenial-proposed/ocata',
116}
117
118APT_NO_LOCK = 100 # The return code for "couldn't acquire lock" in APT.
119CMD_RETRY_DELAY = 10 # Wait 10 seconds between command retries.
120CMD_RETRY_COUNT = 30 # Retry a failing fatal command X times.
121
122
123def filter_installed_packages(packages):
124 """Return a list of packages that require installation."""
125 cache = apt_cache()
126 _pkgs = []
127 for package in packages:
128 try:
129 p = cache[package]
130 p.current_ver or _pkgs.append(package)
131 except KeyError:
132 log('Package {} has no installation candidate.'.format(package),
133 level='WARNING')
134 _pkgs.append(package)
135 return _pkgs
136
137
138def apt_cache(in_memory=True, progress=None):
139 """Build and return an apt cache."""
140 from apt import apt_pkg
141 apt_pkg.init()
142 if in_memory:
143 apt_pkg.config.set("Dir::Cache::pkgcache", "")
144 apt_pkg.config.set("Dir::Cache::srcpkgcache", "")
145 return apt_pkg.Cache(progress)
146
147
148def install(packages, options=None, fatal=False):
149 """Install one or more packages."""
150 if options is None:
151 options = ['--option=Dpkg::Options::=--force-confold']
152
153 cmd = ['apt-get', '--assume-yes']
154 cmd.extend(options)
155 cmd.append('install')
156 if isinstance(packages, six.string_types):
157 cmd.append(packages)
158 else:
159 cmd.extend(packages)
160 log("Installing {} with options: {}".format(packages,
161 options))
162 _run_apt_command(cmd, fatal)
163
164
165def upgrade(options=None, fatal=False, dist=False):
166 """Upgrade all packages."""
167 if options is None:
168 options = ['--option=Dpkg::Options::=--force-confold']
169
170 cmd = ['apt-get', '--assume-yes']
171 cmd.extend(options)
172 if dist:
173 cmd.append('dist-upgrade')
174 else:
175 cmd.append('upgrade')
176 log("Upgrading with options: {}".format(options))
177 _run_apt_command(cmd, fatal)
178
179
180def update(fatal=False):
181 """Update local apt cache."""
182 cmd = ['apt-get', 'update']
183 _run_apt_command(cmd, fatal)
184
185
186def purge(packages, fatal=False):
187 """Purge one or more packages."""
188 cmd = ['apt-get', '--assume-yes', 'purge']
189 if isinstance(packages, six.string_types):
190 cmd.append(packages)
191 else:
192 cmd.extend(packages)
193 log("Purging {}".format(packages))
194 _run_apt_command(cmd, fatal)
195
196
197def apt_mark(packages, mark, fatal=False):
198 """Flag one or more packages using apt-mark."""
199 log("Marking {} as {}".format(packages, mark))
200 cmd = ['apt-mark', mark]
201 if isinstance(packages, six.string_types):
202 cmd.append(packages)
203 else:
204 cmd.extend(packages)
205
206 if fatal:
207 subprocess.check_call(cmd, universal_newlines=True)
208 else:
209 subprocess.call(cmd, universal_newlines=True)
210
211
212def apt_hold(packages, fatal=False):
213 return apt_mark(packages, 'hold', fatal=fatal)
214
215
216def apt_unhold(packages, fatal=False):
217 return apt_mark(packages, 'unhold', fatal=fatal)
218
219
220def add_source(source, key=None):
221 """Add a package source to this system.
222
223 @param source: a URL or sources.list entry, as supported by
224 add-apt-repository(1). Examples::
225
226 ppa:charmers/example
227 deb https://stub:key@private.example.com/ubuntu trusty main
228
229 In addition:
230 'proposed:' may be used to enable the standard 'proposed'
231 pocket for the release.
232 'cloud:' may be used to activate official cloud archive pockets,
233 such as 'cloud:icehouse'
234 'distro' may be used as a noop
235
236 @param key: A key to be added to the system's APT keyring and used
237 to verify the signatures on packages. Ideally, this should be an
238 ASCII format GPG public key including the block headers. A GPG key
239 id may also be used, but be aware that only insecure protocols are
240 available to retrieve the actual public key from a public keyserver
241 placing your Juju environment at risk. ppa and cloud archive keys
242 are securely added automtically, so sould not be provided.
243 """
244 if source is None:
245 log('Source is not present. Skipping')
246 return
247
248 if (source.startswith('ppa:') or
249 source.startswith('http') or
250 source.startswith('deb ') or
251 source.startswith('cloud-archive:')):
252 cmd = ['add-apt-repository', '--yes', source]
253 _run_with_retries(cmd)
254 elif source.startswith('cloud:'):
255 install(filter_installed_packages(['ubuntu-cloud-keyring']),
256 fatal=True)
257 pocket = source.split(':')[-1]
258 if pocket not in CLOUD_ARCHIVE_POCKETS:
259 raise SourceConfigError(
260 'Unsupported cloud: source option %s' %
261 pocket)
262 actual_pocket = CLOUD_ARCHIVE_POCKETS[pocket]
263 with open('/etc/apt/sources.list.d/cloud-archive.list', 'w') as apt:
264 apt.write(CLOUD_ARCHIVE.format(actual_pocket))
265 elif source == 'proposed':
266 release = lsb_release()['DISTRIB_CODENAME']
267 with open('/etc/apt/sources.list.d/proposed.list', 'w') as apt:
268 apt.write(PROPOSED_POCKET.format(release))
269 elif source == 'distro':
270 pass
271 else:
272 log("Unknown source: {!r}".format(source))
273
274 if key:
275 if '-----BEGIN PGP PUBLIC KEY BLOCK-----' in key:
276 with NamedTemporaryFile('w+') as key_file:
277 key_file.write(key)
278 key_file.flush()
279 key_file.seek(0)
280 subprocess.check_call(['apt-key', 'add', '-'], stdin=key_file)
281 else:
282 # Note that hkp: is in no way a secure protocol. Using a
283 # GPG key id is pointless from a security POV unless you
284 # absolutely trust your network and DNS.
285 subprocess.check_call(['apt-key', 'adv', '--keyserver',
286 'hkp://keyserver.ubuntu.com:80', '--recv',
287 key])
288
289
290def _run_with_retries(cmd, max_retries=CMD_RETRY_COUNT, retry_exitcodes=(1,),
291 retry_message="", cmd_env=None):
292 """Run a command and retry until success or max_retries is reached.
293
294 :param: cmd: str: The apt command to run.
295 :param: max_retries: int: The number of retries to attempt on a fatal
296 command. Defaults to CMD_RETRY_COUNT.
297 :param: retry_exitcodes: tuple: Optional additional exit codes to retry.
298 Defaults to retry on exit code 1.
299 :param: retry_message: str: Optional log prefix emitted during retries.
300 :param: cmd_env: dict: Environment variables to add to the command run.
301 """
302
303 env = os.environ.copy()
304 if cmd_env:
305 env.update(cmd_env)
306
307 if not retry_message:
308 retry_message = "Failed executing '{}'".format(" ".join(cmd))
309 retry_message += ". Will retry in {} seconds".format(CMD_RETRY_DELAY)
310
311 retry_count = 0
312 result = None
313
314 retry_results = (None,) + retry_exitcodes
315 while result in retry_results:
316 try:
317 result = subprocess.check_call(cmd, env=env)
318 except subprocess.CalledProcessError as e:
319 retry_count = retry_count + 1
320 if retry_count > max_retries:
321 raise
322 result = e.returncode
323 log(retry_message)
324 time.sleep(CMD_RETRY_DELAY)
325
326
327def _run_apt_command(cmd, fatal=False):
328 """Run an apt command with optional retries.
329
330 :param: fatal: bool: Whether the command's output should be checked and
331 retried.
332 """
333 # Provide DEBIAN_FRONTEND=noninteractive if not present in the environment.
334 cmd_env = {
335 'DEBIAN_FRONTEND': os.environ.get('DEBIAN_FRONTEND', 'noninteractive')}
336
337 if fatal:
338 _run_with_retries(
339 cmd, cmd_env=cmd_env, retry_exitcodes=(1, APT_NO_LOCK,),
340 retry_message="Couldn't acquire DPKG lock")
341 else:
342 env = os.environ.copy()
343 env.update(cmd_env)
344 subprocess.call(cmd, env=env)
345
346
347def get_upstream_version(package):
348 """Determine upstream version based on installed package
349
350 @returns None (if not installed) or the upstream version
351 """
352 import apt_pkg
353 cache = apt_cache()
354 try:
355 pkg = cache[package]
356 except:
357 # the package is unknown to the current apt cache.
358 return None
359
360 if not pkg.current_ver:
361 # package is known, but no version is currently installed.
362 return None
363
364 return apt_pkg.upstream_version(pkg.current_ver.ver_str)
0365
=== added file 'hooks/charmhelpers/osplatform.py'
--- hooks/charmhelpers/osplatform.py 1970-01-01 00:00:00 +0000
+++ hooks/charmhelpers/osplatform.py 2018-06-12 20:07:13 +0000
@@ -0,0 +1,25 @@
1import platform
2
3
4def get_platform():
5 """Return the current OS platform.
6
7 For example: if current os platform is Ubuntu then a string "ubuntu"
8 will be returned (which is the name of the module).
9 This string is used to decide which platform module should be imported.
10 """
11 # linux_distribution is deprecated and will be removed in Python 3.7
12 # Warings *not* disabled, as we certainly need to fix this.
13 tuple_platform = platform.linux_distribution()
14 current_platform = tuple_platform[0]
15 if "Ubuntu" in current_platform:
16 return "ubuntu"
17 elif "CentOS" in current_platform:
18 return "centos"
19 elif "debian" in current_platform:
20 # Stock Python does not detect Ubuntu and instead returns debian.
21 # Or at least it does in some build environments like Travis CI
22 return "ubuntu"
23 else:
24 raise RuntimeError("This module is not supported on {}."
25 .format(current_platform))
026
=== modified file 'hooks/hooks.py'
--- hooks/hooks.py 2017-07-25 07:56:09 +0000
+++ hooks/hooks.py 2018-06-12 20:07:13 +0000
@@ -786,7 +786,7 @@
786 # Write to disk the content of the given SSL certificates786 # Write to disk the content of the given SSL certificates
787 crts = service_config.get('crts', [])787 crts = service_config.get('crts', [])
788 for i, crt in enumerate(crts):788 for i, crt in enumerate(crts):
789 if crt == "DEFAULT":789 if crt == "DEFAULT" or crt == "EXTERNAL":
790 continue790 continue
791 content = base64.b64decode(crt)791 content = base64.b64decode(crt)
792 path = get_service_lib_path(service_name)792 path = get_service_lib_path(service_name)
793793
=== modified file 'icon.svg'
--- icon.svg 2016-01-09 16:42:37 +0000
+++ icon.svg 2018-06-12 20:07:13 +0000
@@ -15,10 +15,21 @@
15 id="svg6517"15 id="svg6517"
16 version="1.1"16 version="1.1"
17 inkscape:version="0.91+devel r"17 inkscape:version="0.91+devel r"
18 sodipodi:docname="haproxy.svg"18 sodipodi:docname="haproxy_circle.svg"
19 viewBox="0 0 96 96">19 viewBox="0 0 96 96">
20 <defs20 <defs
21 id="defs6519">21 id="defs6519">
22 <linearGradient
23 id="Background">
24 <stop
25 id="stop4178"
26 offset="0"
27 style="stop-color:#dedede;stop-opacity:1" />
28 <stop
29 id="stop4180"
30 offset="1"
31 style="stop-color:#ededed;stop-opacity:1" />
32 </linearGradient>
22 <filter33 <filter
23 style="color-interpolation-filters:sRGB"34 style="color-interpolation-filters:sRGB"
24 inkscape:label="Inner Shadow"35 inkscape:label="Inner Shadow"
@@ -68,7 +79,7 @@
68 id="feComposite954" />79 id="feComposite954" />
69 <feGaussianBlur80 <feGaussianBlur
70 in="composite1"81 in="composite1"
71 stdDeviation="4"82 stdDeviation="1"
72 result="blur"83 result="blur"
73 id="feGaussianBlur956" />84 id="feGaussianBlur956" />
74 <feOffset85 <feOffset
@@ -83,17 +94,6 @@
83 result="composite2"94 result="composite2"
84 id="feComposite960" />95 id="feComposite960" />
85 </filter>96 </filter>
86 <linearGradient
87 id="Background">
88 <stop
89 id="stop4178"
90 offset="0"
91 style="stop-color:#22779e;stop-opacity:1" />
92 <stop
93 id="stop4180"
94 offset="1"
95 style="stop-color:#2991c0;stop-opacity:1" />
96 </linearGradient>
97 <clipPath97 <clipPath
98 clipPathUnits="userSpaceOnUse"98 clipPathUnits="userSpaceOnUse"
99 id="clipPath873">99 id="clipPath873">
@@ -110,128 +110,416 @@
110 sodipodi:nodetypes="sssssssss" />110 sodipodi:nodetypes="sssssssss" />
111 </g>111 </g>
112 </clipPath>112 </clipPath>
113 <filter113 <linearGradient
114 inkscape:collect="always"114 id="linearGradient3804">
115 id="filter891"115 <stop
116 inkscape:label="Badge Shadow">116 id="stop3806"
117 <feGaussianBlur117 offset="0"
118 inkscape:collect="always"118 style="stop-color:#ffff00;stop-opacity:1;" />
119 stdDeviation="0.71999962"119 <stop
120 id="feGaussianBlur893" />120 id="stop3808"
121 </filter>121 offset="1"
122 <style122 style="stop-color:#a8650a;stop-opacity:0;" />
123 id="style867"123 </linearGradient>
124 type="text/css"><![CDATA[124 <linearGradient
125 .fil0 {fill:#1F1A17}125 id="linearGradient3778">
126 ]]></style>126 <stop
127 <clipPath127 id="stop3780"
128 id="clipPath16">128 offset="0"
129 <path129 style="stop-color:#ffff00;stop-opacity:0.84536082;" />
130 id="path18"130 <stop
131 d="M -9,-9 H 605 V 222 H -9 Z"131 id="stop3782"
132 inkscape:connector-curvature="0" />132 offset="1"
133 </clipPath>133 style="stop-color:#a8650a;stop-opacity:0;" />
134 <clipPath134 </linearGradient>
135 id="clipPath116">135 <linearGradient
136 <path136 id="linearGradient3725">
137 id="path118"137 <stop
138 d="m 91.7368,146.3253 -9.7039,-1.577 -8.8548,-3.8814 -7.5206,-4.7308 -7.1566,-8.7335 -4.0431,-4.282 -3.9093,-1.4409 -1.034,2.5271 1.8079,2.6096 0.4062,3.6802 1.211,-0.0488 1.3232,-1.2069 -0.3569,3.7488 -1.4667,0.9839 0.0445,1.4286 -3.4744,-1.9655 -3.1462,-3.712 -0.6559,-3.3176 1.3453,-2.6567 1.2549,-4.5133 2.5521,-1.2084 2.6847,0.1318 2.5455,1.4791 -1.698,-8.6122 1.698,-9.5825 -1.8692,-4.4246 -6.1223,-6.5965 1.0885,-3.941 2.9002,-4.5669 5.4688,-3.8486 2.9007,-0.3969 3.225,-0.1094 -2.012,-8.2601 7.3993,-3.0326 9.2188,-1.2129 3.1535,2.0619 0.2427,5.5797 3.5178,5.8224 0.2426,4.6094 8.4909,-0.6066 7.8843,0.7279 -7.8843,-4.7307 1.3343,-5.701 4.9731,-7.763 4.8521,-2.0622 3.8814,1.5769 1.577,3.1538 8.1269,6.1861 1.5769,-1.3343 12.7363,-0.485 2.5473,2.0619 0.2426,3.6391 -0.849,1.5767 -0.6066,9.8251 -4.2454,8.4909 0.7276,3.7605 2.5475,-1.3343 7.1566,-6.6716 3.5175,-0.2424 3.8815,1.5769 3.8818,2.9109 1.9406,6.3077 11.4021,-0.7277 6.914,2.6686 5.5797,5.2157 4.0028,7.5206 0.9706,8.8546 -0.8493,10.3105 -2.1832,9.2185 -2.1836,2.9112 -3.0322,0.9706 -5.3373,-5.8224 -4.8518,-1.6982 -4.2455,7.0353 -4.2454,3.8815 -2.3049,1.4556 -9.2185,7.6419 -7.3993,4.0028 -7.3993,0.6066 -8.6119,-1.4556 -7.5206,-2.7899 -5.2158,-4.2454 -4.1241,-4.9734 -4.2454,-1.2129"138 id="stop3727"
139 inkscape:connector-curvature="0" />139 offset="0"
140 </clipPath>140 style="stop-color:#0000ff;stop-opacity:1;" />
141 <clipPath141 <stop
142 id="clipPath128">142 id="stop3729"
143 <path143 offset="1"
144 id="path130"144 style="stop-color:#000000;stop-opacity:0" />
145 d="m 91.7368,146.3253 -9.7039,-1.577 -8.8548,-3.8814 -7.5206,-4.7308 -7.1566,-8.7335 -4.0431,-4.282 -3.9093,-1.4409 -1.034,2.5271 1.8079,2.6096 0.4062,3.6802 1.211,-0.0488 1.3232,-1.2069 -0.3569,3.7488 -1.4667,0.9839 0.0445,1.4286 -3.4744,-1.9655 -3.1462,-3.712 -0.6559,-3.3176 1.3453,-2.6567 1.2549,-4.5133 2.5521,-1.2084 2.6847,0.1318 2.5455,1.4791 -1.698,-8.6122 1.698,-9.5825 -1.8692,-4.4246 -6.1223,-6.5965 1.0885,-3.941 2.9002,-4.5669 5.4688,-3.8486 2.9007,-0.3969 3.225,-0.1094 -2.012,-8.2601 7.3993,-3.0326 9.2188,-1.2129 3.1535,2.0619 0.2427,5.5797 3.5178,5.8224 0.2426,4.6094 8.4909,-0.6066 7.8843,0.7279 -7.8843,-4.7307 1.3343,-5.701 4.9731,-7.763 4.8521,-2.0622 3.8814,1.5769 1.577,3.1538 8.1269,6.1861 1.5769,-1.3343 12.7363,-0.485 2.5473,2.0619 0.2426,3.6391 -0.849,1.5767 -0.6066,9.8251 -4.2454,8.4909 0.7276,3.7605 2.5475,-1.3343 7.1566,-6.6716 3.5175,-0.2424 3.8815,1.5769 3.8818,2.9109 1.9406,6.3077 11.4021,-0.7277 6.914,2.6686 5.5797,5.2157 4.0028,7.5206 0.9706,8.8546 -0.8493,10.3105 -2.1832,9.2185 -2.1836,2.9112 -3.0322,0.9706 -5.3373,-5.8224 -4.8518,-1.6982 -4.2455,7.0353 -4.2454,3.8815 -2.3049,1.4556 -9.2185,7.6419 -7.3993,4.0028 -7.3993,0.6066 -8.6119,-1.4556 -7.5206,-2.7899 -5.2158,-4.2454 -4.1241,-4.9734 -4.2454,-1.2129"145 </linearGradient>
146 inkscape:connector-curvature="0" />146 <linearGradient
147 </clipPath>147 id="linearGradient3715">
148 <linearGradient148 <stop
149 id="linearGradient3850"149 style="stop-color:#3700d0;stop-opacity:1;"
150 inkscape:collect="always">150 offset="0"
151 <stop151 id="stop3717" />
152 id="stop3852"152 <stop
153 offset="0"153 style="stop-color: rgb(0, 0, 0); stop-opacity: 0;"
154 style="stop-color:#000000;stop-opacity:1;" />154 offset="1"
155 <stop155 id="stop3719" />
156 id="stop3854"156 </linearGradient>
157 offset="1"157 <linearGradient
158 style="stop-color:#000000;stop-opacity:0;" />158 id="linearGradient3705">
159 </linearGradient>159 <stop
160 <clipPath160 style="stop-color:#ff0000;stop-opacity:0.54639173;"
161 clipPathUnits="userSpaceOnUse"161 offset="0"
162 id="clipPath3095">162 id="stop3707" />
163 <path163 <stop
164 d="M 976.648,389.551 H 134.246 V 1229.55 H 976.648 V 389.551"164 style="stop-color:#ff0067;stop-opacity:0.42268041;"
165 id="path3097"165 offset="1"
166 inkscape:connector-curvature="0" />166 id="stop3709" />
167 </clipPath>167 </linearGradient>
168 <clipPath168 <linearGradient
169 clipPathUnits="userSpaceOnUse"169 id="linearGradient3699">
170 id="clipPath3195">170 <stop
171 <path171 style="stop-color:#d80000;stop-opacity:1;"
172 d="m 611.836,756.738 -106.34,105.207 c -8.473,8.289 -13.617,20.102 -13.598,33.379 L 598.301,790.207 c -0.031,-13.418 5.094,-25.031 13.535,-33.469"172 offset="0"
173 id="path3197"173 id="stop3701" />
174 inkscape:connector-curvature="0" />174 <stop
175 </clipPath>175 style="stop-color: rgb(136, 0, 170); stop-opacity: 0;"
176 <clipPath176 offset="1"
177 clipPathUnits="userSpaceOnUse"177 id="stop3703" />
178 id="clipPath3235">178 </linearGradient>
179 <path179 <linearGradient
180 d="m 1095.64,1501.81 c 35.46,-35.07 70.89,-70.11 106.35,-105.17 4.4,-4.38 7.11,-10.53 7.11,-17.55 l -106.37,105.21 c 0,7 -2.71,13.11 -7.09,17.51"180 id="linearGradient3693">
181 id="path3237"181 <stop
182 inkscape:connector-curvature="0" />182 style="stop-color:#4700aa;stop-opacity:1;"
183 </clipPath>183 offset="0"
184 <clipPath184 id="stop3695" />
185 id="clipPath4591"185 <stop
186 clipPathUnits="userSpaceOnUse">186 style="stop-color: rgb(136, 0, 170); stop-opacity: 0;"
187 <path187 offset="1"
188 inkscape:connector-curvature="0"188 id="stop3697" />
189 d="m 1106.6009,730.43734 -0.036,21.648 c -0.01,3.50825 -2.8675,6.61375 -6.4037,6.92525 l -83.6503,7.33162 c -3.5205,0.30763 -6.3812,-2.29987 -6.3671,-5.8145 l 0.036,-21.6475 20.1171,-1.76662 -0.011,4.63775 c 0,1.83937 1.4844,3.19925 3.3262,3.0395 l 49.5274,-4.33975 c 1.8425,-0.166 3.3425,-1.78125 3.3538,-3.626 l 0.01,-4.63025 20.1,-1.7575"189 </linearGradient>
190 style="fill:#ff00ff;fill-opacity:1;fill-rule:nonzero;stroke:none"190 <linearGradient
191 id="path4593" />191 id="linearGradient3687">
192 </clipPath>192 <stop
193 <radialGradient193 style="stop-color:#cd00ff;stop-opacity:1;"
194 gradientUnits="userSpaceOnUse"194 offset="0"
195 gradientTransform="matrix(-1.4333926,-2.2742838,1.1731823,-0.73941125,-174.08025,98.374394)"195 id="stop3689" />
196 r="20.40658"196 <stop
197 fy="93.399292"197 style="stop-color:#ff00ff;stop-opacity:0;"
198 fx="-26.508606"198 offset="1"
199 cy="93.399292"199 id="stop3691" />
200 cx="-26.508606"200 </linearGradient>
201 id="radialGradient3856"201 <linearGradient
202 xlink:href="#linearGradient3850"202 id="linearGradient3681">
203 inkscape:collect="always" />203 <stop
204 <linearGradient204 style="stop-color:#ffff00;stop-opacity:1;"
205 gradientTransform="translate(-318.48033,212.32022)"205 offset="0"
206 gradientUnits="userSpaceOnUse"206 id="stop3683" />
207 y2="993.19702"207 <stop
208 x2="-51.879555"208 style="stop-color:#a8650a;stop-opacity:0.45360824;"
209 y1="593.11615"209 offset="1"
210 x1="348.20132"210 id="stop3685" />
211 id="linearGradient3895"211 </linearGradient>
212 xlink:href="#linearGradient3850"212 <linearGradient
213 inkscape:collect="always" />213 id="linearGradient3265"
214 <clipPath214 inkscape:collect="always">
215 id="clipPath3906"215 <stop
216 clipPathUnits="userSpaceOnUse">216 id="stop3267"
217 <rect217 offset="0"
218 transform="scale(1,-1)"218 style="stop-color: rgb(128, 101, 10); stop-opacity: 1;" />
219 style="color:#000000;display:inline;overflow:visible;visibility:visible;opacity:0.8;fill:#ff00ff;stroke:none;stroke-width:4;marker:none;enable-background:accumulate"219 <stop
220 id="rect3908"220 id="stop3269"
221 width="1019.1371"221 offset="1"
222 height="1019.1371"222 style="stop-color: rgb(128, 101, 10); stop-opacity: 0;" />
223 x="357.9816"223 </linearGradient>
224 y="-1725.8152" />224 <linearGradient
225 </clipPath>225 id="linearGradient3255"
226 <clipPath226 inkscape:collect="always">
227 clipPathUnits="userSpaceOnUse"227 <stop
228 id="clipPath4637">228 id="stop3257"
229 <path229 offset="0"
230 sodipodi:nodetypes="sssssssss"230 style="stop-color: rgb(6, 46, 5); stop-opacity: 1;" />
231 inkscape:connector-curvature="0"231 <stop
232 id="path4639"232 id="stop3259"
233 d="M -268,700.15563 V 666.4259 c 0,-27.24324 3.88785,-31.13513 31.10302,-31.13513 h 33.79408 c 27.21507,0 31.1029,3.89189 31.1029,31.13513 v 33.72973 c 0,27.24325 -3.88783,31.13514 -31.1029,31.13514 h -33.79408 c -27.21517,0 -31.10302,-3.89189 -31.10302,-31.13514 z"233 offset="1"
234 style="display:inline;fill:#df382c;fill-opacity:1;stroke:none" />234 style="stop-color: rgb(6, 46, 5); stop-opacity: 0;" />
235 </linearGradient>
236 <linearGradient
237 id="linearGradient3238"
238 inkscape:collect="always">
239 <stop
240 id="stop3240"
241 offset="0"
242 style="stop-color: rgb(0, 0, 0); stop-opacity: 1;" />
243 <stop
244 id="stop3242"
245 offset="1"
246 style="stop-color: rgb(0, 0, 0); stop-opacity: 0;" />
247 </linearGradient>
248 <linearGradient
249 id="linearGradient3230">
250 <stop
251 id="stop3232"
252 offset="0"
253 style="stop-color:#ea009b;stop-opacity:1;" />
254 <stop
255 id="stop3234"
256 offset="1"
257 style="stop-color: rgb(209, 249, 208); stop-opacity: 0;" />
258 </linearGradient>
259 <linearGradient
260 id="linearGradient3218"
261 inkscape:collect="always">
262 <stop
263 id="stop3220"
264 offset="0"
265 style="stop-color: rgb(136, 0, 170); stop-opacity: 1;" />
266 <stop
267 id="stop3222"
268 offset="1"
269 style="stop-color: rgb(136, 0, 170); stop-opacity: 0;" />
270 </linearGradient>
271 <linearGradient
272 id="linearGradient3208"
273 inkscape:collect="always">
274 <stop
275 id="stop3210"
276 offset="0"
277 style="stop-color: rgb(36, 216, 31); stop-opacity: 1;" />
278 <stop
279 id="stop3212"
280 offset="1"
281 style="stop-color: rgb(36, 216, 31); stop-opacity: 0;" />
282 </linearGradient>
283 <linearGradient
284 id="linearGradient3184">
285 <stop
286 id="stop3186"
287 offset="0"
288 style="stop-color:#4200d2;stop-opacity:1;" />
289 <stop
290 id="stop3188"
291 offset="1"
292 style="stop-color:#d42aff;stop-opacity:0;" />
293 </linearGradient>
294 <linearGradient
295 id="linearGradient3174"
296 inkscape:collect="always">
297 <stop
298 id="stop3176"
299 offset="0"
300 style="stop-color: rgb(0, 255, 0); stop-opacity: 1;" />
301 <stop
302 id="stop3178"
303 offset="1"
304 style="stop-color: rgb(0, 255, 0); stop-opacity: 0;" />
305 </linearGradient>
306 <linearGradient
307 id="linearGradient3166"
308 inkscape:collect="always">
309 <stop
310 id="stop3168"
311 offset="0"
312 style="stop-color: rgb(0, 0, 0); stop-opacity: 1;" />
313 <stop
314 id="stop3170"
315 offset="1"
316 style="stop-color: rgb(0, 0, 0); stop-opacity: 0;" />
317 </linearGradient>
318 <inkscape:perspective
319 id="perspective10"
320 inkscape:persp3d-origin="372.04724 : 350.78739 : 1"
321 inkscape:vp_z="744.09448 : 526.18109 : 1"
The diff has been truncated for viewing.

Subscribers

People subscribed via source and target branches