Merge lp:~bloodearnest/charm-haproxy/merge-trunk-snapstore into lp:~ubuntuone-pqm-team/charm-haproxy/snap-store
- merge-trunk-snapstore
- Merge into snap-store
Proposed by
Simon Davy
Status: | Work in progress |
---|---|
Proposed branch: | lp:~bloodearnest/charm-haproxy/merge-trunk-snapstore |
Merge into: | lp:~ubuntuone-pqm-team/charm-haproxy/snap-store |
Diff against target: |
5380 lines (+3150/-1126) 38 files modified
README.md (+6/-5) charm-helpers.yaml (+2/-1) config.yaml (+1/-1) hooks/charmhelpers/__init__.py (+11/-13) hooks/charmhelpers/contrib/__init__.py (+11/-13) hooks/charmhelpers/contrib/charmsupport/__init__.py (+11/-13) hooks/charmhelpers/contrib/charmsupport/nrpe.py (+106/-34) hooks/charmhelpers/contrib/charmsupport/volumes.py (+11/-13) hooks/charmhelpers/core/__init__.py (+11/-13) hooks/charmhelpers/core/decorators.py (+11/-13) hooks/charmhelpers/core/files.py (+43/-0) hooks/charmhelpers/core/fstab.py (+11/-13) hooks/charmhelpers/core/hookenv.py (+448/-63) hooks/charmhelpers/core/host.py (+625/-155) hooks/charmhelpers/core/host_factory/centos.py (+56/-0) hooks/charmhelpers/core/host_factory/ubuntu.py (+56/-0) hooks/charmhelpers/core/hugepage.py (+69/-0) hooks/charmhelpers/core/kernel.py (+72/-0) hooks/charmhelpers/core/kernel_factory/centos.py (+17/-0) hooks/charmhelpers/core/kernel_factory/ubuntu.py (+13/-0) hooks/charmhelpers/core/services/__init__.py (+11/-13) hooks/charmhelpers/core/services/base.py (+52/-30) hooks/charmhelpers/core/services/helpers.py (+42/-19) hooks/charmhelpers/core/strutils.py (+41/-13) hooks/charmhelpers/core/sysctl.py (+11/-13) hooks/charmhelpers/core/templating.py (+40/-24) hooks/charmhelpers/core/unitdata.py (+72/-31) hooks/charmhelpers/fetch/__init__.py (+46/-296) hooks/charmhelpers/fetch/archiveurl.py (+19/-15) hooks/charmhelpers/fetch/bzrurl.py (+48/-50) hooks/charmhelpers/fetch/centos.py (+171/-0) hooks/charmhelpers/fetch/giturl.py (+37/-39) hooks/charmhelpers/fetch/snap.py (+122/-0) hooks/charmhelpers/fetch/ubuntu.py (+364/-0) hooks/charmhelpers/osplatform.py (+25/-0) hooks/hooks.py (+1/-1) icon.svg (+456/-232) metadata.yaml (+1/-0) |
To merge this branch: | bzr merge lp:~bloodearnest/charm-haproxy/merge-trunk-snapstore |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Guillermo Gonzalez | Approve | ||
haproxy-team | Pending | ||
Review via email: mp+347844@code.launchpad.net |
Commit message
Merge trunk for autocert support
Description of the change
To post a comment you must log in.
Unmerged revisions
- 113. By Simon Davy
-
merge trunk r114, with updated charm-helpers and autocert support
Preview Diff
[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1 | === modified file 'README.md' | |||
2 | --- README.md 2015-06-10 14:38:26 +0000 | |||
3 | +++ README.md 2018-06-12 20:07:13 +0000 | |||
4 | @@ -1,6 +1,7 @@ | |||
5 | 1 | # Overview | 1 | # Overview |
6 | 2 | 2 | ||
8 | 3 | This charm deploys a reverse proxy in front of other servies. You can use this to load balance existing deployments. | 3 | This charm deploys a reverse proxy in front of other services. You can use |
9 | 4 | this to load balance existing deployments. | ||
10 | 4 | 5 | ||
11 | 5 | # Usage | 6 | # Usage |
12 | 6 | 7 | ||
13 | @@ -110,7 +111,7 @@ | |||
14 | 110 | charms written like apache2 that can act as a front-end for haproxy to take of | 111 | charms written like apache2 that can act as a front-end for haproxy to take of |
15 | 111 | things like ssl encryption. When joining a service like apache2 on its | 112 | things like ssl encryption. When joining a service like apache2 on its |
16 | 112 | reverseproxy relation, haproxy's website relation will set an `all_services` | 113 | reverseproxy relation, haproxy's website relation will set an `all_services` |
18 | 113 | varaible that conforms to the spec layed out in the apache2 charm. | 114 | variable that conforms to the spec laid out in the apache2 charm. |
19 | 114 | 115 | ||
20 | 115 | These settings can then be used when crafting your vhost template to make sure | 116 | These settings can then be used when crafting your vhost template to make sure |
21 | 116 | traffic goes to the correct haproxy listener which will in turn forward the | 117 | traffic goes to the correct haproxy listener which will in turn forward the |
22 | @@ -136,7 +137,7 @@ | |||
23 | 136 | - { ... optionally more services here ... } | 137 | - { ... optionally more services here ... } |
24 | 137 | " | 138 | " |
25 | 138 | 139 | ||
27 | 139 | where the DEFAULT keyword means use the certificate set with `ssl_cert`/`ssl_key` (or | 140 | The DEFAULT keyword means use the certificate set with `ssl_cert`/`ssl_key` (or |
28 | 140 | alternatively you can inline different base64-encode certificates). | 141 | alternatively you can inline different base64-encode certificates). |
29 | 141 | 142 | ||
30 | 142 | Note that in order to use SSL termination you need haproxy 1.5 or later, which | 143 | Note that in order to use SSL termination you need haproxy 1.5 or later, which |
31 | @@ -227,9 +228,9 @@ | |||
32 | 227 | ### active-active mode | 228 | ### active-active mode |
33 | 228 | 229 | ||
34 | 229 | If the peering\_mode option is set to "active-active", then any haproxy unit | 230 | If the peering\_mode option is set to "active-active", then any haproxy unit |
36 | 230 | will be independant from each other and will simply load-balance the traffic to | 231 | will be independent from each other and will simply load-balance the traffic to |
37 | 231 | the backends. In this case, the indirection layer described above is not | 232 | the backends. In this case, the indirection layer described above is not |
39 | 232 | created in this case. | 233 | created. |
40 | 233 | 234 | ||
41 | 234 | This mode allows increasing the bandwidth to the backends by adding additional | 235 | This mode allows increasing the bandwidth to the backends by adding additional |
42 | 235 | units, at the cost of having less control over the number of connections that | 236 | units, at the cost of having less control over the number of connections that |
43 | 236 | 237 | ||
44 | === modified file 'charm-helpers.yaml' | |||
45 | --- charm-helpers.yaml 2013-08-21 19:19:29 +0000 | |||
46 | +++ charm-helpers.yaml 2018-06-12 20:07:13 +0000 | |||
47 | @@ -1,4 +1,5 @@ | |||
48 | 1 | include: | 1 | include: |
49 | 2 | - core | 2 | - core |
50 | 3 | - fetch | 3 | - fetch |
51 | 4 | - contrib.charmsupport | ||
52 | 5 | \ No newline at end of file | 4 | \ No newline at end of file |
53 | 5 | - contrib.charmsupport | ||
54 | 6 | - osplatform | ||
55 | 6 | 7 | ||
56 | === modified file 'config.yaml' | |||
57 | --- config.yaml 2016-11-28 04:37:08 +0000 | |||
58 | +++ config.yaml 2018-06-12 20:07:13 +0000 | |||
59 | @@ -49,7 +49,7 @@ | |||
60 | 49 | greater than 1024 bits are not supported by Java 7 and earlier clients. This | 49 | greater than 1024 bits are not supported by Java 7 and earlier clients. This |
61 | 50 | config key will be ignored if the installed haproxy package has no SSL support. | 50 | config key will be ignored if the installed haproxy package has no SSL support. |
62 | 51 | global_default_bind_ciphers: | 51 | global_default_bind_ciphers: |
64 | 52 | default: ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA | 52 | default: ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:!DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:!DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:!CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA |
65 | 53 | type: string | 53 | type: string |
66 | 54 | description: | | 54 | description: | |
67 | 55 | Sets the default string describing the list of cipher algorithms | 55 | Sets the default string describing the list of cipher algorithms |
68 | 56 | 56 | ||
69 | === added file 'hooks/__init__.py' | |||
70 | === modified file 'hooks/charmhelpers/__init__.py' | |||
71 | --- hooks/charmhelpers/__init__.py 2015-02-09 12:53:57 +0000 | |||
72 | +++ hooks/charmhelpers/__init__.py 2018-06-12 20:07:13 +0000 | |||
73 | @@ -1,18 +1,16 @@ | |||
74 | 1 | # Copyright 2014-2015 Canonical Limited. | 1 | # Copyright 2014-2015 Canonical Limited. |
75 | 2 | # | 2 | # |
89 | 3 | # This file is part of charm-helpers. | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); |
90 | 4 | # | 4 | # you may not use this file except in compliance with the License. |
91 | 5 | # charm-helpers is free software: you can redistribute it and/or modify | 5 | # You may obtain a copy of the License at |
92 | 6 | # it under the terms of the GNU Lesser General Public License version 3 as | 6 | # |
93 | 7 | # published by the Free Software Foundation. | 7 | # http://www.apache.org/licenses/LICENSE-2.0 |
94 | 8 | # | 8 | # |
95 | 9 | # charm-helpers is distributed in the hope that it will be useful, | 9 | # Unless required by applicable law or agreed to in writing, software |
96 | 10 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 10 | # distributed under the License is distributed on an "AS IS" BASIS, |
97 | 11 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
98 | 12 | # GNU Lesser General Public License for more details. | 12 | # See the License for the specific language governing permissions and |
99 | 13 | # | 13 | # limitations under the License. |
87 | 14 | # You should have received a copy of the GNU Lesser General Public License | ||
88 | 15 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
100 | 16 | 14 | ||
101 | 17 | # Bootstrap charm-helpers, installing its dependencies if necessary using | 15 | # Bootstrap charm-helpers, installing its dependencies if necessary using |
102 | 18 | # only standard libraries. | 16 | # only standard libraries. |
103 | 19 | 17 | ||
104 | === modified file 'hooks/charmhelpers/contrib/__init__.py' | |||
105 | --- hooks/charmhelpers/contrib/__init__.py 2015-02-09 12:53:57 +0000 | |||
106 | +++ hooks/charmhelpers/contrib/__init__.py 2018-06-12 20:07:13 +0000 | |||
107 | @@ -1,15 +1,13 @@ | |||
108 | 1 | # Copyright 2014-2015 Canonical Limited. | 1 | # Copyright 2014-2015 Canonical Limited. |
109 | 2 | # | 2 | # |
123 | 3 | # This file is part of charm-helpers. | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); |
124 | 4 | # | 4 | # you may not use this file except in compliance with the License. |
125 | 5 | # charm-helpers is free software: you can redistribute it and/or modify | 5 | # You may obtain a copy of the License at |
126 | 6 | # it under the terms of the GNU Lesser General Public License version 3 as | 6 | # |
127 | 7 | # published by the Free Software Foundation. | 7 | # http://www.apache.org/licenses/LICENSE-2.0 |
128 | 8 | # | 8 | # |
129 | 9 | # charm-helpers is distributed in the hope that it will be useful, | 9 | # Unless required by applicable law or agreed to in writing, software |
130 | 10 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 10 | # distributed under the License is distributed on an "AS IS" BASIS, |
131 | 11 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
132 | 12 | # GNU Lesser General Public License for more details. | 12 | # See the License for the specific language governing permissions and |
133 | 13 | # | 13 | # limitations under the License. |
121 | 14 | # You should have received a copy of the GNU Lesser General Public License | ||
122 | 15 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
134 | 16 | 14 | ||
135 | === modified file 'hooks/charmhelpers/contrib/charmsupport/__init__.py' | |||
136 | --- hooks/charmhelpers/contrib/charmsupport/__init__.py 2015-02-09 12:53:57 +0000 | |||
137 | +++ hooks/charmhelpers/contrib/charmsupport/__init__.py 2018-06-12 20:07:13 +0000 | |||
138 | @@ -1,15 +1,13 @@ | |||
139 | 1 | # Copyright 2014-2015 Canonical Limited. | 1 | # Copyright 2014-2015 Canonical Limited. |
140 | 2 | # | 2 | # |
154 | 3 | # This file is part of charm-helpers. | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); |
155 | 4 | # | 4 | # you may not use this file except in compliance with the License. |
156 | 5 | # charm-helpers is free software: you can redistribute it and/or modify | 5 | # You may obtain a copy of the License at |
157 | 6 | # it under the terms of the GNU Lesser General Public License version 3 as | 6 | # |
158 | 7 | # published by the Free Software Foundation. | 7 | # http://www.apache.org/licenses/LICENSE-2.0 |
159 | 8 | # | 8 | # |
160 | 9 | # charm-helpers is distributed in the hope that it will be useful, | 9 | # Unless required by applicable law or agreed to in writing, software |
161 | 10 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 10 | # distributed under the License is distributed on an "AS IS" BASIS, |
162 | 11 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
163 | 12 | # GNU Lesser General Public License for more details. | 12 | # See the License for the specific language governing permissions and |
164 | 13 | # | 13 | # limitations under the License. |
152 | 14 | # You should have received a copy of the GNU Lesser General Public License | ||
153 | 15 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
165 | 16 | 14 | ||
166 | === modified file 'hooks/charmhelpers/contrib/charmsupport/nrpe.py' | |||
167 | --- hooks/charmhelpers/contrib/charmsupport/nrpe.py 2015-05-14 10:48:09 +0000 | |||
168 | +++ hooks/charmhelpers/contrib/charmsupport/nrpe.py 2018-06-12 20:07:13 +0000 | |||
169 | @@ -1,18 +1,16 @@ | |||
170 | 1 | # Copyright 2014-2015 Canonical Limited. | 1 | # Copyright 2014-2015 Canonical Limited. |
171 | 2 | # | 2 | # |
185 | 3 | # This file is part of charm-helpers. | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); |
186 | 4 | # | 4 | # you may not use this file except in compliance with the License. |
187 | 5 | # charm-helpers is free software: you can redistribute it and/or modify | 5 | # You may obtain a copy of the License at |
188 | 6 | # it under the terms of the GNU Lesser General Public License version 3 as | 6 | # |
189 | 7 | # published by the Free Software Foundation. | 7 | # http://www.apache.org/licenses/LICENSE-2.0 |
190 | 8 | # | 8 | # |
191 | 9 | # charm-helpers is distributed in the hope that it will be useful, | 9 | # Unless required by applicable law or agreed to in writing, software |
192 | 10 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 10 | # distributed under the License is distributed on an "AS IS" BASIS, |
193 | 11 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
194 | 12 | # GNU Lesser General Public License for more details. | 12 | # See the License for the specific language governing permissions and |
195 | 13 | # | 13 | # limitations under the License. |
183 | 14 | # You should have received a copy of the GNU Lesser General Public License | ||
184 | 15 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
196 | 16 | 14 | ||
197 | 17 | """Compatibility with the nrpe-external-master charm""" | 15 | """Compatibility with the nrpe-external-master charm""" |
198 | 18 | # Copyright 2012 Canonical Ltd. | 16 | # Copyright 2012 Canonical Ltd. |
199 | @@ -40,6 +38,7 @@ | |||
200 | 40 | ) | 38 | ) |
201 | 41 | 39 | ||
202 | 42 | from charmhelpers.core.host import service | 40 | from charmhelpers.core.host import service |
203 | 41 | from charmhelpers.core import host | ||
204 | 43 | 42 | ||
205 | 44 | # This module adds compatibility with the nrpe-external-master and plain nrpe | 43 | # This module adds compatibility with the nrpe-external-master and plain nrpe |
206 | 45 | # subordinate charms. To use it in your charm: | 44 | # subordinate charms. To use it in your charm: |
207 | @@ -110,6 +109,13 @@ | |||
208 | 110 | # def local_monitors_relation_changed(): | 109 | # def local_monitors_relation_changed(): |
209 | 111 | # update_nrpe_config() | 110 | # update_nrpe_config() |
210 | 112 | # | 111 | # |
211 | 112 | # 4.a If your charm is a subordinate charm set primary=False | ||
212 | 113 | # | ||
213 | 114 | # from charmsupport.nrpe import NRPE | ||
214 | 115 | # (...) | ||
215 | 116 | # def update_nrpe_config(): | ||
216 | 117 | # nrpe_compat = NRPE(primary=False) | ||
217 | 118 | # | ||
218 | 113 | # 5. ln -s hooks.py nrpe-external-master-relation-changed | 119 | # 5. ln -s hooks.py nrpe-external-master-relation-changed |
219 | 114 | # ln -s hooks.py local-monitors-relation-changed | 120 | # ln -s hooks.py local-monitors-relation-changed |
220 | 115 | 121 | ||
221 | @@ -148,6 +154,13 @@ | |||
222 | 148 | self.description = description | 154 | self.description = description |
223 | 149 | self.check_cmd = self._locate_cmd(check_cmd) | 155 | self.check_cmd = self._locate_cmd(check_cmd) |
224 | 150 | 156 | ||
225 | 157 | def _get_check_filename(self): | ||
226 | 158 | return os.path.join(NRPE.nrpe_confdir, '{}.cfg'.format(self.command)) | ||
227 | 159 | |||
228 | 160 | def _get_service_filename(self, hostname): | ||
229 | 161 | return os.path.join(NRPE.nagios_exportdir, | ||
230 | 162 | 'service__{}_{}.cfg'.format(hostname, self.command)) | ||
231 | 163 | |||
232 | 151 | def _locate_cmd(self, check_cmd): | 164 | def _locate_cmd(self, check_cmd): |
233 | 152 | search_path = ( | 165 | search_path = ( |
234 | 153 | '/usr/lib/nagios/plugins', | 166 | '/usr/lib/nagios/plugins', |
235 | @@ -163,9 +176,21 @@ | |||
236 | 163 | log('Check command not found: {}'.format(parts[0])) | 176 | log('Check command not found: {}'.format(parts[0])) |
237 | 164 | return '' | 177 | return '' |
238 | 165 | 178 | ||
239 | 179 | def _remove_service_files(self): | ||
240 | 180 | if not os.path.exists(NRPE.nagios_exportdir): | ||
241 | 181 | return | ||
242 | 182 | for f in os.listdir(NRPE.nagios_exportdir): | ||
243 | 183 | if f.endswith('_{}.cfg'.format(self.command)): | ||
244 | 184 | os.remove(os.path.join(NRPE.nagios_exportdir, f)) | ||
245 | 185 | |||
246 | 186 | def remove(self, hostname): | ||
247 | 187 | nrpe_check_file = self._get_check_filename() | ||
248 | 188 | if os.path.exists(nrpe_check_file): | ||
249 | 189 | os.remove(nrpe_check_file) | ||
250 | 190 | self._remove_service_files() | ||
251 | 191 | |||
252 | 166 | def write(self, nagios_context, hostname, nagios_servicegroups): | 192 | def write(self, nagios_context, hostname, nagios_servicegroups): |
255 | 167 | nrpe_check_file = '/etc/nagios/nrpe.d/{}.cfg'.format( | 193 | nrpe_check_file = self._get_check_filename() |
254 | 168 | self.command) | ||
256 | 169 | with open(nrpe_check_file, 'w') as nrpe_check_config: | 194 | with open(nrpe_check_file, 'w') as nrpe_check_config: |
257 | 170 | nrpe_check_config.write("# check {}\n".format(self.shortname)) | 195 | nrpe_check_config.write("# check {}\n".format(self.shortname)) |
258 | 171 | nrpe_check_config.write("command[{}]={}\n".format( | 196 | nrpe_check_config.write("command[{}]={}\n".format( |
259 | @@ -180,9 +205,7 @@ | |||
260 | 180 | 205 | ||
261 | 181 | def write_service_config(self, nagios_context, hostname, | 206 | def write_service_config(self, nagios_context, hostname, |
262 | 182 | nagios_servicegroups): | 207 | nagios_servicegroups): |
266 | 183 | for f in os.listdir(NRPE.nagios_exportdir): | 208 | self._remove_service_files() |
264 | 184 | if re.search('.*{}.cfg'.format(self.command), f): | ||
265 | 185 | os.remove(os.path.join(NRPE.nagios_exportdir, f)) | ||
267 | 186 | 209 | ||
268 | 187 | templ_vars = { | 210 | templ_vars = { |
269 | 188 | 'nagios_hostname': hostname, | 211 | 'nagios_hostname': hostname, |
270 | @@ -192,8 +215,7 @@ | |||
271 | 192 | 'command': self.command, | 215 | 'command': self.command, |
272 | 193 | } | 216 | } |
273 | 194 | nrpe_service_text = Check.service_template.format(**templ_vars) | 217 | nrpe_service_text = Check.service_template.format(**templ_vars) |
276 | 195 | nrpe_service_file = '{}/service__{}_{}.cfg'.format( | 218 | nrpe_service_file = self._get_service_filename(hostname) |
275 | 196 | NRPE.nagios_exportdir, hostname, self.command) | ||
277 | 197 | with open(nrpe_service_file, 'w') as nrpe_service_config: | 219 | with open(nrpe_service_file, 'w') as nrpe_service_config: |
278 | 198 | nrpe_service_config.write(str(nrpe_service_text)) | 220 | nrpe_service_config.write(str(nrpe_service_text)) |
279 | 199 | 221 | ||
280 | @@ -205,10 +227,12 @@ | |||
281 | 205 | nagios_logdir = '/var/log/nagios' | 227 | nagios_logdir = '/var/log/nagios' |
282 | 206 | nagios_exportdir = '/var/lib/nagios/export' | 228 | nagios_exportdir = '/var/lib/nagios/export' |
283 | 207 | nrpe_confdir = '/etc/nagios/nrpe.d' | 229 | nrpe_confdir = '/etc/nagios/nrpe.d' |
284 | 230 | homedir = '/var/lib/nagios' # home dir provided by nagios-nrpe-server | ||
285 | 208 | 231 | ||
287 | 209 | def __init__(self, hostname=None): | 232 | def __init__(self, hostname=None, primary=True): |
288 | 210 | super(NRPE, self).__init__() | 233 | super(NRPE, self).__init__() |
289 | 211 | self.config = config() | 234 | self.config = config() |
290 | 235 | self.primary = primary | ||
291 | 212 | self.nagios_context = self.config['nagios_context'] | 236 | self.nagios_context = self.config['nagios_context'] |
292 | 213 | if 'nagios_servicegroups' in self.config and self.config['nagios_servicegroups']: | 237 | if 'nagios_servicegroups' in self.config and self.config['nagios_servicegroups']: |
293 | 214 | self.nagios_servicegroups = self.config['nagios_servicegroups'] | 238 | self.nagios_servicegroups = self.config['nagios_servicegroups'] |
294 | @@ -218,12 +242,38 @@ | |||
295 | 218 | if hostname: | 242 | if hostname: |
296 | 219 | self.hostname = hostname | 243 | self.hostname = hostname |
297 | 220 | else: | 244 | else: |
299 | 221 | self.hostname = "{}-{}".format(self.nagios_context, self.unit_name) | 245 | nagios_hostname = get_nagios_hostname() |
300 | 246 | if nagios_hostname: | ||
301 | 247 | self.hostname = nagios_hostname | ||
302 | 248 | else: | ||
303 | 249 | self.hostname = "{}-{}".format(self.nagios_context, self.unit_name) | ||
304 | 222 | self.checks = [] | 250 | self.checks = [] |
305 | 251 | # Iff in an nrpe-external-master relation hook, set primary status | ||
306 | 252 | relation = relation_ids('nrpe-external-master') | ||
307 | 253 | if relation: | ||
308 | 254 | log("Setting charm primary status {}".format(primary)) | ||
309 | 255 | for rid in relation_ids('nrpe-external-master'): | ||
310 | 256 | relation_set(relation_id=rid, relation_settings={'primary': self.primary}) | ||
311 | 223 | 257 | ||
312 | 224 | def add_check(self, *args, **kwargs): | 258 | def add_check(self, *args, **kwargs): |
313 | 225 | self.checks.append(Check(*args, **kwargs)) | 259 | self.checks.append(Check(*args, **kwargs)) |
314 | 226 | 260 | ||
315 | 261 | def remove_check(self, *args, **kwargs): | ||
316 | 262 | if kwargs.get('shortname') is None: | ||
317 | 263 | raise ValueError('shortname of check must be specified') | ||
318 | 264 | |||
319 | 265 | # Use sensible defaults if they're not specified - these are not | ||
320 | 266 | # actually used during removal, but they're required for constructing | ||
321 | 267 | # the Check object; check_disk is chosen because it's part of the | ||
322 | 268 | # nagios-plugins-basic package. | ||
323 | 269 | if kwargs.get('check_cmd') is None: | ||
324 | 270 | kwargs['check_cmd'] = 'check_disk' | ||
325 | 271 | if kwargs.get('description') is None: | ||
326 | 272 | kwargs['description'] = '' | ||
327 | 273 | |||
328 | 274 | check = Check(*args, **kwargs) | ||
329 | 275 | check.remove(self.hostname) | ||
330 | 276 | |||
331 | 227 | def write(self): | 277 | def write(self): |
332 | 228 | try: | 278 | try: |
333 | 229 | nagios_uid = pwd.getpwnam('nagios').pw_uid | 279 | nagios_uid = pwd.getpwnam('nagios').pw_uid |
334 | @@ -260,7 +310,7 @@ | |||
335 | 260 | :param str relation_name: Name of relation nrpe sub joined to | 310 | :param str relation_name: Name of relation nrpe sub joined to |
336 | 261 | """ | 311 | """ |
337 | 262 | for rel in relations_of_type(relation_name): | 312 | for rel in relations_of_type(relation_name): |
339 | 263 | if 'nagios_hostname' in rel: | 313 | if 'nagios_host_context' in rel: |
340 | 264 | return rel['nagios_host_context'] | 314 | return rel['nagios_host_context'] |
341 | 265 | 315 | ||
342 | 266 | 316 | ||
343 | @@ -289,18 +339,30 @@ | |||
344 | 289 | return unit | 339 | return unit |
345 | 290 | 340 | ||
346 | 291 | 341 | ||
348 | 292 | def add_init_service_checks(nrpe, services, unit_name): | 342 | def add_init_service_checks(nrpe, services, unit_name, immediate_check=True): |
349 | 293 | """ | 343 | """ |
350 | 294 | Add checks for each service in list | 344 | Add checks for each service in list |
351 | 295 | 345 | ||
352 | 296 | :param NRPE nrpe: NRPE object to add check to | 346 | :param NRPE nrpe: NRPE object to add check to |
353 | 297 | :param list services: List of services to check | 347 | :param list services: List of services to check |
354 | 298 | :param str unit_name: Unit name to use in check description | 348 | :param str unit_name: Unit name to use in check description |
355 | 349 | :param bool immediate_check: For sysv init, run the service check immediately | ||
356 | 299 | """ | 350 | """ |
357 | 300 | for svc in services: | 351 | for svc in services: |
358 | 352 | # Don't add a check for these services from neutron-gateway | ||
359 | 353 | if svc in ['ext-port', 'os-charm-phy-nic-mtu']: | ||
360 | 354 | next | ||
361 | 355 | |||
362 | 301 | upstart_init = '/etc/init/%s.conf' % svc | 356 | upstart_init = '/etc/init/%s.conf' % svc |
363 | 302 | sysv_init = '/etc/init.d/%s' % svc | 357 | sysv_init = '/etc/init.d/%s' % svc |
365 | 303 | if os.path.exists(upstart_init): | 358 | |
366 | 359 | if host.init_is_systemd(): | ||
367 | 360 | nrpe.add_check( | ||
368 | 361 | shortname=svc, | ||
369 | 362 | description='process check {%s}' % unit_name, | ||
370 | 363 | check_cmd='check_systemd.py %s' % svc | ||
371 | 364 | ) | ||
372 | 365 | elif os.path.exists(upstart_init): | ||
373 | 304 | nrpe.add_check( | 366 | nrpe.add_check( |
374 | 305 | shortname=svc, | 367 | shortname=svc, |
375 | 306 | description='process check {%s}' % unit_name, | 368 | description='process check {%s}' % unit_name, |
376 | @@ -308,21 +370,31 @@ | |||
377 | 308 | ) | 370 | ) |
378 | 309 | elif os.path.exists(sysv_init): | 371 | elif os.path.exists(sysv_init): |
379 | 310 | cronpath = '/etc/cron.d/nagios-service-check-%s' % svc | 372 | cronpath = '/etc/cron.d/nagios-service-check-%s' % svc |
386 | 311 | cron_file = ('*/5 * * * * root ' | 373 | checkpath = '%s/service-check-%s.txt' % (nrpe.homedir, svc) |
387 | 312 | '/usr/local/lib/nagios/plugins/check_exit_status.pl ' | 374 | croncmd = ( |
388 | 313 | '-s /etc/init.d/%s status > ' | 375 | '/usr/local/lib/nagios/plugins/check_exit_status.pl ' |
389 | 314 | '/var/lib/nagios/service-check-%s.txt\n' % (svc, | 376 | '-s /etc/init.d/%s status' % svc |
390 | 315 | svc) | 377 | ) |
391 | 316 | ) | 378 | cron_file = '*/5 * * * * root %s > %s\n' % (croncmd, checkpath) |
392 | 317 | f = open(cronpath, 'w') | 379 | f = open(cronpath, 'w') |
393 | 318 | f.write(cron_file) | 380 | f.write(cron_file) |
394 | 319 | f.close() | 381 | f.close() |
395 | 320 | nrpe.add_check( | 382 | nrpe.add_check( |
396 | 321 | shortname=svc, | 383 | shortname=svc, |
400 | 322 | description='process check {%s}' % unit_name, | 384 | description='service check {%s}' % unit_name, |
401 | 323 | check_cmd='check_status_file.py -f ' | 385 | check_cmd='check_status_file.py -f %s' % checkpath, |
399 | 324 | '/var/lib/nagios/service-check-%s.txt' % svc, | ||
402 | 325 | ) | 386 | ) |
403 | 387 | # if /var/lib/nagios doesn't exist open(checkpath, 'w') will fail | ||
404 | 388 | # (LP: #1670223). | ||
405 | 389 | if immediate_check and os.path.isdir(nrpe.homedir): | ||
406 | 390 | f = open(checkpath, 'w') | ||
407 | 391 | subprocess.call( | ||
408 | 392 | croncmd.split(), | ||
409 | 393 | stdout=f, | ||
410 | 394 | stderr=subprocess.STDOUT | ||
411 | 395 | ) | ||
412 | 396 | f.close() | ||
413 | 397 | os.chmod(checkpath, 0o644) | ||
414 | 326 | 398 | ||
415 | 327 | 399 | ||
416 | 328 | def copy_nrpe_checks(): | 400 | def copy_nrpe_checks(): |
417 | 329 | 401 | ||
418 | === modified file 'hooks/charmhelpers/contrib/charmsupport/volumes.py' | |||
419 | --- hooks/charmhelpers/contrib/charmsupport/volumes.py 2015-02-09 12:53:57 +0000 | |||
420 | +++ hooks/charmhelpers/contrib/charmsupport/volumes.py 2018-06-12 20:07:13 +0000 | |||
421 | @@ -1,18 +1,16 @@ | |||
422 | 1 | # Copyright 2014-2015 Canonical Limited. | 1 | # Copyright 2014-2015 Canonical Limited. |
423 | 2 | # | 2 | # |
437 | 3 | # This file is part of charm-helpers. | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); |
438 | 4 | # | 4 | # you may not use this file except in compliance with the License. |
439 | 5 | # charm-helpers is free software: you can redistribute it and/or modify | 5 | # You may obtain a copy of the License at |
440 | 6 | # it under the terms of the GNU Lesser General Public License version 3 as | 6 | # |
441 | 7 | # published by the Free Software Foundation. | 7 | # http://www.apache.org/licenses/LICENSE-2.0 |
442 | 8 | # | 8 | # |
443 | 9 | # charm-helpers is distributed in the hope that it will be useful, | 9 | # Unless required by applicable law or agreed to in writing, software |
444 | 10 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 10 | # distributed under the License is distributed on an "AS IS" BASIS, |
445 | 11 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
446 | 12 | # GNU Lesser General Public License for more details. | 12 | # See the License for the specific language governing permissions and |
447 | 13 | # | 13 | # limitations under the License. |
435 | 14 | # You should have received a copy of the GNU Lesser General Public License | ||
436 | 15 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
448 | 16 | 14 | ||
449 | 17 | ''' | 15 | ''' |
450 | 18 | Functions for managing volumes in juju units. One volume is supported per unit. | 16 | Functions for managing volumes in juju units. One volume is supported per unit. |
451 | 19 | 17 | ||
452 | === modified file 'hooks/charmhelpers/core/__init__.py' | |||
453 | --- hooks/charmhelpers/core/__init__.py 2015-02-09 12:53:57 +0000 | |||
454 | +++ hooks/charmhelpers/core/__init__.py 2018-06-12 20:07:13 +0000 | |||
455 | @@ -1,15 +1,13 @@ | |||
456 | 1 | # Copyright 2014-2015 Canonical Limited. | 1 | # Copyright 2014-2015 Canonical Limited. |
457 | 2 | # | 2 | # |
471 | 3 | # This file is part of charm-helpers. | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); |
472 | 4 | # | 4 | # you may not use this file except in compliance with the License. |
473 | 5 | # charm-helpers is free software: you can redistribute it and/or modify | 5 | # You may obtain a copy of the License at |
474 | 6 | # it under the terms of the GNU Lesser General Public License version 3 as | 6 | # |
475 | 7 | # published by the Free Software Foundation. | 7 | # http://www.apache.org/licenses/LICENSE-2.0 |
476 | 8 | # | 8 | # |
477 | 9 | # charm-helpers is distributed in the hope that it will be useful, | 9 | # Unless required by applicable law or agreed to in writing, software |
478 | 10 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 10 | # distributed under the License is distributed on an "AS IS" BASIS, |
479 | 11 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
480 | 12 | # GNU Lesser General Public License for more details. | 12 | # See the License for the specific language governing permissions and |
481 | 13 | # | 13 | # limitations under the License. |
469 | 14 | # You should have received a copy of the GNU Lesser General Public License | ||
470 | 15 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
482 | 16 | 14 | ||
483 | === modified file 'hooks/charmhelpers/core/decorators.py' | |||
484 | --- hooks/charmhelpers/core/decorators.py 2015-02-09 12:58:07 +0000 | |||
485 | +++ hooks/charmhelpers/core/decorators.py 2018-06-12 20:07:13 +0000 | |||
486 | @@ -1,18 +1,16 @@ | |||
487 | 1 | # Copyright 2014-2015 Canonical Limited. | 1 | # Copyright 2014-2015 Canonical Limited. |
488 | 2 | # | 2 | # |
502 | 3 | # This file is part of charm-helpers. | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); |
503 | 4 | # | 4 | # you may not use this file except in compliance with the License. |
504 | 5 | # charm-helpers is free software: you can redistribute it and/or modify | 5 | # You may obtain a copy of the License at |
505 | 6 | # it under the terms of the GNU Lesser General Public License version 3 as | 6 | # |
506 | 7 | # published by the Free Software Foundation. | 7 | # http://www.apache.org/licenses/LICENSE-2.0 |
507 | 8 | # | 8 | # |
508 | 9 | # charm-helpers is distributed in the hope that it will be useful, | 9 | # Unless required by applicable law or agreed to in writing, software |
509 | 10 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 10 | # distributed under the License is distributed on an "AS IS" BASIS, |
510 | 11 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
511 | 12 | # GNU Lesser General Public License for more details. | 12 | # See the License for the specific language governing permissions and |
512 | 13 | # | 13 | # limitations under the License. |
500 | 14 | # You should have received a copy of the GNU Lesser General Public License | ||
501 | 15 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
513 | 16 | 14 | ||
514 | 17 | # | 15 | # |
515 | 18 | # Copyright 2014 Canonical Ltd. | 16 | # Copyright 2014 Canonical Ltd. |
516 | 19 | 17 | ||
517 | === added file 'hooks/charmhelpers/core/files.py' | |||
518 | --- hooks/charmhelpers/core/files.py 1970-01-01 00:00:00 +0000 | |||
519 | +++ hooks/charmhelpers/core/files.py 2018-06-12 20:07:13 +0000 | |||
520 | @@ -0,0 +1,43 @@ | |||
521 | 1 | #!/usr/bin/env python | ||
522 | 2 | # -*- coding: utf-8 -*- | ||
523 | 3 | |||
524 | 4 | # Copyright 2014-2015 Canonical Limited. | ||
525 | 5 | # | ||
526 | 6 | # Licensed under the Apache License, Version 2.0 (the "License"); | ||
527 | 7 | # you may not use this file except in compliance with the License. | ||
528 | 8 | # You may obtain a copy of the License at | ||
529 | 9 | # | ||
530 | 10 | # http://www.apache.org/licenses/LICENSE-2.0 | ||
531 | 11 | # | ||
532 | 12 | # Unless required by applicable law or agreed to in writing, software | ||
533 | 13 | # distributed under the License is distributed on an "AS IS" BASIS, | ||
534 | 14 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
535 | 15 | # See the License for the specific language governing permissions and | ||
536 | 16 | # limitations under the License. | ||
537 | 17 | |||
538 | 18 | __author__ = 'Jorge Niedbalski <niedbalski@ubuntu.com>' | ||
539 | 19 | |||
540 | 20 | import os | ||
541 | 21 | import subprocess | ||
542 | 22 | |||
543 | 23 | |||
544 | 24 | def sed(filename, before, after, flags='g'): | ||
545 | 25 | """ | ||
546 | 26 | Search and replaces the given pattern on filename. | ||
547 | 27 | |||
548 | 28 | :param filename: relative or absolute file path. | ||
549 | 29 | :param before: expression to be replaced (see 'man sed') | ||
550 | 30 | :param after: expression to replace with (see 'man sed') | ||
551 | 31 | :param flags: sed-compatible regex flags in example, to make | ||
552 | 32 | the search and replace case insensitive, specify ``flags="i"``. | ||
553 | 33 | The ``g`` flag is always specified regardless, so you do not | ||
554 | 34 | need to remember to include it when overriding this parameter. | ||
555 | 35 | :returns: If the sed command exit code was zero then return, | ||
556 | 36 | otherwise raise CalledProcessError. | ||
557 | 37 | """ | ||
558 | 38 | expression = r's/{0}/{1}/{2}'.format(before, | ||
559 | 39 | after, flags) | ||
560 | 40 | |||
561 | 41 | return subprocess.check_call(["sed", "-i", "-r", "-e", | ||
562 | 42 | expression, | ||
563 | 43 | os.path.expanduser(filename)]) | ||
564 | 0 | 44 | ||
565 | === modified file 'hooks/charmhelpers/core/fstab.py' | |||
566 | --- hooks/charmhelpers/core/fstab.py 2015-05-14 10:48:09 +0000 | |||
567 | +++ hooks/charmhelpers/core/fstab.py 2018-06-12 20:07:13 +0000 | |||
568 | @@ -3,19 +3,17 @@ | |||
569 | 3 | 3 | ||
570 | 4 | # Copyright 2014-2015 Canonical Limited. | 4 | # Copyright 2014-2015 Canonical Limited. |
571 | 5 | # | 5 | # |
585 | 6 | # This file is part of charm-helpers. | 6 | # Licensed under the Apache License, Version 2.0 (the "License"); |
586 | 7 | # | 7 | # you may not use this file except in compliance with the License. |
587 | 8 | # charm-helpers is free software: you can redistribute it and/or modify | 8 | # You may obtain a copy of the License at |
588 | 9 | # it under the terms of the GNU Lesser General Public License version 3 as | 9 | # |
589 | 10 | # published by the Free Software Foundation. | 10 | # http://www.apache.org/licenses/LICENSE-2.0 |
590 | 11 | # | 11 | # |
591 | 12 | # charm-helpers is distributed in the hope that it will be useful, | 12 | # Unless required by applicable law or agreed to in writing, software |
592 | 13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 13 | # distributed under the License is distributed on an "AS IS" BASIS, |
593 | 14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 14 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
594 | 15 | # GNU Lesser General Public License for more details. | 15 | # See the License for the specific language governing permissions and |
595 | 16 | # | 16 | # limitations under the License. |
583 | 17 | # You should have received a copy of the GNU Lesser General Public License | ||
584 | 18 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
596 | 19 | 17 | ||
597 | 20 | import io | 18 | import io |
598 | 21 | import os | 19 | import os |
599 | 22 | 20 | ||
600 | === modified file 'hooks/charmhelpers/core/hookenv.py' | |||
601 | --- hooks/charmhelpers/core/hookenv.py 2015-05-14 10:48:09 +0000 | |||
602 | +++ hooks/charmhelpers/core/hookenv.py 2018-06-12 20:07:13 +0000 | |||
603 | @@ -1,18 +1,16 @@ | |||
604 | 1 | # Copyright 2014-2015 Canonical Limited. | 1 | # Copyright 2014-2015 Canonical Limited. |
605 | 2 | # | 2 | # |
619 | 3 | # This file is part of charm-helpers. | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); |
620 | 4 | # | 4 | # you may not use this file except in compliance with the License. |
621 | 5 | # charm-helpers is free software: you can redistribute it and/or modify | 5 | # You may obtain a copy of the License at |
622 | 6 | # it under the terms of the GNU Lesser General Public License version 3 as | 6 | # |
623 | 7 | # published by the Free Software Foundation. | 7 | # http://www.apache.org/licenses/LICENSE-2.0 |
624 | 8 | # | 8 | # |
625 | 9 | # charm-helpers is distributed in the hope that it will be useful, | 9 | # Unless required by applicable law or agreed to in writing, software |
626 | 10 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 10 | # distributed under the License is distributed on an "AS IS" BASIS, |
627 | 11 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
628 | 12 | # GNU Lesser General Public License for more details. | 12 | # See the License for the specific language governing permissions and |
629 | 13 | # | 13 | # limitations under the License. |
617 | 14 | # You should have received a copy of the GNU Lesser General Public License | ||
618 | 15 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
630 | 16 | 14 | ||
631 | 17 | "Interactions with the Juju environment" | 15 | "Interactions with the Juju environment" |
632 | 18 | # Copyright 2013 Canonical Ltd. | 16 | # Copyright 2013 Canonical Ltd. |
633 | @@ -21,7 +19,10 @@ | |||
634 | 21 | # Charm Helpers Developers <juju@lists.ubuntu.com> | 19 | # Charm Helpers Developers <juju@lists.ubuntu.com> |
635 | 22 | 20 | ||
636 | 23 | from __future__ import print_function | 21 | from __future__ import print_function |
637 | 22 | import copy | ||
638 | 23 | from distutils.version import LooseVersion | ||
639 | 24 | from functools import wraps | 24 | from functools import wraps |
640 | 25 | import glob | ||
641 | 25 | import os | 26 | import os |
642 | 26 | import json | 27 | import json |
643 | 27 | import yaml | 28 | import yaml |
644 | @@ -71,6 +72,7 @@ | |||
645 | 71 | res = func(*args, **kwargs) | 72 | res = func(*args, **kwargs) |
646 | 72 | cache[key] = res | 73 | cache[key] = res |
647 | 73 | return res | 74 | return res |
648 | 75 | wrapper._wrapped = func | ||
649 | 74 | return wrapper | 76 | return wrapper |
650 | 75 | 77 | ||
651 | 76 | 78 | ||
652 | @@ -170,9 +172,19 @@ | |||
653 | 170 | return os.environ.get('JUJU_RELATION', None) | 172 | return os.environ.get('JUJU_RELATION', None) |
654 | 171 | 173 | ||
655 | 172 | 174 | ||
659 | 173 | def relation_id(): | 175 | @cached |
660 | 174 | """The relation ID for the current relation hook""" | 176 | def relation_id(relation_name=None, service_or_unit=None): |
661 | 175 | return os.environ.get('JUJU_RELATION_ID', None) | 177 | """The relation ID for the current or a specified relation""" |
662 | 178 | if not relation_name and not service_or_unit: | ||
663 | 179 | return os.environ.get('JUJU_RELATION_ID', None) | ||
664 | 180 | elif relation_name and service_or_unit: | ||
665 | 181 | service_name = service_or_unit.split('/')[0] | ||
666 | 182 | for relid in relation_ids(relation_name): | ||
667 | 183 | remote_service = remote_service_name(relid) | ||
668 | 184 | if remote_service == service_name: | ||
669 | 185 | return relid | ||
670 | 186 | else: | ||
671 | 187 | raise ValueError('Must specify neither or both of relation_name and service_or_unit') | ||
672 | 176 | 188 | ||
673 | 177 | 189 | ||
674 | 178 | def local_unit(): | 190 | def local_unit(): |
675 | @@ -190,9 +202,20 @@ | |||
676 | 190 | return local_unit().split('/')[0] | 202 | return local_unit().split('/')[0] |
677 | 191 | 203 | ||
678 | 192 | 204 | ||
679 | 205 | @cached | ||
680 | 206 | def remote_service_name(relid=None): | ||
681 | 207 | """The remote service name for a given relation-id (or the current relation)""" | ||
682 | 208 | if relid is None: | ||
683 | 209 | unit = remote_unit() | ||
684 | 210 | else: | ||
685 | 211 | units = related_units(relid) | ||
686 | 212 | unit = units[0] if units else None | ||
687 | 213 | return unit.split('/')[0] if unit else None | ||
688 | 214 | |||
689 | 215 | |||
690 | 193 | def hook_name(): | 216 | def hook_name(): |
691 | 194 | """The name of the currently executing hook""" | 217 | """The name of the currently executing hook""" |
693 | 195 | return os.path.basename(sys.argv[0]) | 218 | return os.environ.get('JUJU_HOOK_NAME', os.path.basename(sys.argv[0])) |
694 | 196 | 219 | ||
695 | 197 | 220 | ||
696 | 198 | class Config(dict): | 221 | class Config(dict): |
697 | @@ -242,29 +265,7 @@ | |||
698 | 242 | self.path = os.path.join(charm_dir(), Config.CONFIG_FILE_NAME) | 265 | self.path = os.path.join(charm_dir(), Config.CONFIG_FILE_NAME) |
699 | 243 | if os.path.exists(self.path): | 266 | if os.path.exists(self.path): |
700 | 244 | self.load_previous() | 267 | self.load_previous() |
724 | 245 | 268 | atexit(self._implicit_save) | |
702 | 246 | def __getitem__(self, key): | ||
703 | 247 | """For regular dict lookups, check the current juju config first, | ||
704 | 248 | then the previous (saved) copy. This ensures that user-saved values | ||
705 | 249 | will be returned by a dict lookup. | ||
706 | 250 | |||
707 | 251 | """ | ||
708 | 252 | try: | ||
709 | 253 | return dict.__getitem__(self, key) | ||
710 | 254 | except KeyError: | ||
711 | 255 | return (self._prev_dict or {})[key] | ||
712 | 256 | |||
713 | 257 | def get(self, key, default=None): | ||
714 | 258 | try: | ||
715 | 259 | return self[key] | ||
716 | 260 | except KeyError: | ||
717 | 261 | return default | ||
718 | 262 | |||
719 | 263 | def keys(self): | ||
720 | 264 | prev_keys = [] | ||
721 | 265 | if self._prev_dict is not None: | ||
722 | 266 | prev_keys = self._prev_dict.keys() | ||
723 | 267 | return list(set(prev_keys + list(dict.keys(self)))) | ||
725 | 268 | 269 | ||
726 | 269 | def load_previous(self, path=None): | 270 | def load_previous(self, path=None): |
727 | 270 | """Load previous copy of config from disk. | 271 | """Load previous copy of config from disk. |
728 | @@ -283,6 +284,9 @@ | |||
729 | 283 | self.path = path or self.path | 284 | self.path = path or self.path |
730 | 284 | with open(self.path) as f: | 285 | with open(self.path) as f: |
731 | 285 | self._prev_dict = json.load(f) | 286 | self._prev_dict = json.load(f) |
732 | 287 | for k, v in copy.deepcopy(self._prev_dict).items(): | ||
733 | 288 | if k not in self: | ||
734 | 289 | self[k] = v | ||
735 | 286 | 290 | ||
736 | 287 | def changed(self, key): | 291 | def changed(self, key): |
737 | 288 | """Return True if the current value for this key is different from | 292 | """Return True if the current value for this key is different from |
738 | @@ -314,13 +318,13 @@ | |||
739 | 314 | instance. | 318 | instance. |
740 | 315 | 319 | ||
741 | 316 | """ | 320 | """ |
742 | 317 | if self._prev_dict: | ||
743 | 318 | for k, v in six.iteritems(self._prev_dict): | ||
744 | 319 | if k not in self: | ||
745 | 320 | self[k] = v | ||
746 | 321 | with open(self.path, 'w') as f: | 321 | with open(self.path, 'w') as f: |
747 | 322 | json.dump(self, f) | 322 | json.dump(self, f) |
748 | 323 | 323 | ||
749 | 324 | def _implicit_save(self): | ||
750 | 325 | if self.implicit_save: | ||
751 | 326 | self.save() | ||
752 | 327 | |||
753 | 324 | 328 | ||
754 | 325 | @cached | 329 | @cached |
755 | 326 | def config(scope=None): | 330 | def config(scope=None): |
756 | @@ -328,6 +332,8 @@ | |||
757 | 328 | config_cmd_line = ['config-get'] | 332 | config_cmd_line = ['config-get'] |
758 | 329 | if scope is not None: | 333 | if scope is not None: |
759 | 330 | config_cmd_line.append(scope) | 334 | config_cmd_line.append(scope) |
760 | 335 | else: | ||
761 | 336 | config_cmd_line.append('--all') | ||
762 | 331 | config_cmd_line.append('--format=json') | 337 | config_cmd_line.append('--format=json') |
763 | 332 | try: | 338 | try: |
764 | 333 | config_data = json.loads( | 339 | config_data = json.loads( |
765 | @@ -364,11 +370,16 @@ | |||
766 | 364 | relation_settings = relation_settings if relation_settings else {} | 370 | relation_settings = relation_settings if relation_settings else {} |
767 | 365 | relation_cmd_line = ['relation-set'] | 371 | relation_cmd_line = ['relation-set'] |
768 | 366 | accepts_file = "--file" in subprocess.check_output( | 372 | accepts_file = "--file" in subprocess.check_output( |
770 | 367 | relation_cmd_line + ["--help"]) | 373 | relation_cmd_line + ["--help"], universal_newlines=True) |
771 | 368 | if relation_id is not None: | 374 | if relation_id is not None: |
772 | 369 | relation_cmd_line.extend(('-r', relation_id)) | 375 | relation_cmd_line.extend(('-r', relation_id)) |
773 | 370 | settings = relation_settings.copy() | 376 | settings = relation_settings.copy() |
774 | 371 | settings.update(kwargs) | 377 | settings.update(kwargs) |
775 | 378 | for key, value in settings.items(): | ||
776 | 379 | # Force value to be a string: it always should, but some call | ||
777 | 380 | # sites pass in things like dicts or numbers. | ||
778 | 381 | if value is not None: | ||
779 | 382 | settings[key] = "{}".format(value) | ||
780 | 372 | if accepts_file: | 383 | if accepts_file: |
781 | 373 | # --file was introduced in Juju 1.23.2. Use it by default if | 384 | # --file was introduced in Juju 1.23.2. Use it by default if |
782 | 374 | # available, since otherwise we'll break if the relation data is | 385 | # available, since otherwise we'll break if the relation data is |
783 | @@ -390,6 +401,17 @@ | |||
784 | 390 | flush(local_unit()) | 401 | flush(local_unit()) |
785 | 391 | 402 | ||
786 | 392 | 403 | ||
787 | 404 | def relation_clear(r_id=None): | ||
788 | 405 | ''' Clears any relation data already set on relation r_id ''' | ||
789 | 406 | settings = relation_get(rid=r_id, | ||
790 | 407 | unit=local_unit()) | ||
791 | 408 | for setting in settings: | ||
792 | 409 | if setting not in ['public-address', 'private-address']: | ||
793 | 410 | settings[setting] = None | ||
794 | 411 | relation_set(relation_id=r_id, | ||
795 | 412 | **settings) | ||
796 | 413 | |||
797 | 414 | |||
798 | 393 | @cached | 415 | @cached |
799 | 394 | def relation_ids(reltype=None): | 416 | def relation_ids(reltype=None): |
800 | 395 | """A list of relation_ids""" | 417 | """A list of relation_ids""" |
801 | @@ -469,6 +491,76 @@ | |||
802 | 469 | 491 | ||
803 | 470 | 492 | ||
804 | 471 | @cached | 493 | @cached |
805 | 494 | def peer_relation_id(): | ||
806 | 495 | '''Get the peers relation id if a peers relation has been joined, else None.''' | ||
807 | 496 | md = metadata() | ||
808 | 497 | section = md.get('peers') | ||
809 | 498 | if section: | ||
810 | 499 | for key in section: | ||
811 | 500 | relids = relation_ids(key) | ||
812 | 501 | if relids: | ||
813 | 502 | return relids[0] | ||
814 | 503 | return None | ||
815 | 504 | |||
816 | 505 | |||
817 | 506 | @cached | ||
818 | 507 | def relation_to_interface(relation_name): | ||
819 | 508 | """ | ||
820 | 509 | Given the name of a relation, return the interface that relation uses. | ||
821 | 510 | |||
822 | 511 | :returns: The interface name, or ``None``. | ||
823 | 512 | """ | ||
824 | 513 | return relation_to_role_and_interface(relation_name)[1] | ||
825 | 514 | |||
826 | 515 | |||
827 | 516 | @cached | ||
828 | 517 | def relation_to_role_and_interface(relation_name): | ||
829 | 518 | """ | ||
830 | 519 | Given the name of a relation, return the role and the name of the interface | ||
831 | 520 | that relation uses (where role is one of ``provides``, ``requires``, or ``peers``). | ||
832 | 521 | |||
833 | 522 | :returns: A tuple containing ``(role, interface)``, or ``(None, None)``. | ||
834 | 523 | """ | ||
835 | 524 | _metadata = metadata() | ||
836 | 525 | for role in ('provides', 'requires', 'peers'): | ||
837 | 526 | interface = _metadata.get(role, {}).get(relation_name, {}).get('interface') | ||
838 | 527 | if interface: | ||
839 | 528 | return role, interface | ||
840 | 529 | return None, None | ||
841 | 530 | |||
842 | 531 | |||
843 | 532 | @cached | ||
844 | 533 | def role_and_interface_to_relations(role, interface_name): | ||
845 | 534 | """ | ||
846 | 535 | Given a role and interface name, return a list of relation names for the | ||
847 | 536 | current charm that use that interface under that role (where role is one | ||
848 | 537 | of ``provides``, ``requires``, or ``peers``). | ||
849 | 538 | |||
850 | 539 | :returns: A list of relation names. | ||
851 | 540 | """ | ||
852 | 541 | _metadata = metadata() | ||
853 | 542 | results = [] | ||
854 | 543 | for relation_name, relation in _metadata.get(role, {}).items(): | ||
855 | 544 | if relation['interface'] == interface_name: | ||
856 | 545 | results.append(relation_name) | ||
857 | 546 | return results | ||
858 | 547 | |||
859 | 548 | |||
860 | 549 | @cached | ||
861 | 550 | def interface_to_relations(interface_name): | ||
862 | 551 | """ | ||
863 | 552 | Given an interface, return a list of relation names for the current | ||
864 | 553 | charm that use that interface. | ||
865 | 554 | |||
866 | 555 | :returns: A list of relation names. | ||
867 | 556 | """ | ||
868 | 557 | results = [] | ||
869 | 558 | for role in ('provides', 'requires', 'peers'): | ||
870 | 559 | results.extend(role_and_interface_to_relations(role, interface_name)) | ||
871 | 560 | return results | ||
872 | 561 | |||
873 | 562 | |||
874 | 563 | @cached | ||
875 | 472 | def charm_name(): | 564 | def charm_name(): |
876 | 473 | """Get the name of the current charm as is specified on metadata.yaml""" | 565 | """Get the name of the current charm as is specified on metadata.yaml""" |
877 | 474 | return metadata().get('name') | 566 | return metadata().get('name') |
878 | @@ -524,6 +616,20 @@ | |||
879 | 524 | subprocess.check_call(_args) | 616 | subprocess.check_call(_args) |
880 | 525 | 617 | ||
881 | 526 | 618 | ||
882 | 619 | def open_ports(start, end, protocol="TCP"): | ||
883 | 620 | """Opens a range of service network ports""" | ||
884 | 621 | _args = ['open-port'] | ||
885 | 622 | _args.append('{}-{}/{}'.format(start, end, protocol)) | ||
886 | 623 | subprocess.check_call(_args) | ||
887 | 624 | |||
888 | 625 | |||
889 | 626 | def close_ports(start, end, protocol="TCP"): | ||
890 | 627 | """Close a range of service network ports""" | ||
891 | 628 | _args = ['close-port'] | ||
892 | 629 | _args.append('{}-{}/{}'.format(start, end, protocol)) | ||
893 | 630 | subprocess.check_call(_args) | ||
894 | 631 | |||
895 | 632 | |||
896 | 527 | @cached | 633 | @cached |
897 | 528 | def unit_get(attribute): | 634 | def unit_get(attribute): |
898 | 529 | """Get the unit ID for the remote unit""" | 635 | """Get the unit ID for the remote unit""" |
899 | @@ -544,6 +650,38 @@ | |||
900 | 544 | return unit_get('private-address') | 650 | return unit_get('private-address') |
901 | 545 | 651 | ||
902 | 546 | 652 | ||
903 | 653 | @cached | ||
904 | 654 | def storage_get(attribute=None, storage_id=None): | ||
905 | 655 | """Get storage attributes""" | ||
906 | 656 | _args = ['storage-get', '--format=json'] | ||
907 | 657 | if storage_id: | ||
908 | 658 | _args.extend(('-s', storage_id)) | ||
909 | 659 | if attribute: | ||
910 | 660 | _args.append(attribute) | ||
911 | 661 | try: | ||
912 | 662 | return json.loads(subprocess.check_output(_args).decode('UTF-8')) | ||
913 | 663 | except ValueError: | ||
914 | 664 | return None | ||
915 | 665 | |||
916 | 666 | |||
917 | 667 | @cached | ||
918 | 668 | def storage_list(storage_name=None): | ||
919 | 669 | """List the storage IDs for the unit""" | ||
920 | 670 | _args = ['storage-list', '--format=json'] | ||
921 | 671 | if storage_name: | ||
922 | 672 | _args.append(storage_name) | ||
923 | 673 | try: | ||
924 | 674 | return json.loads(subprocess.check_output(_args).decode('UTF-8')) | ||
925 | 675 | except ValueError: | ||
926 | 676 | return None | ||
927 | 677 | except OSError as e: | ||
928 | 678 | import errno | ||
929 | 679 | if e.errno == errno.ENOENT: | ||
930 | 680 | # storage-list does not exist | ||
931 | 681 | return [] | ||
932 | 682 | raise | ||
933 | 683 | |||
934 | 684 | |||
935 | 547 | class UnregisteredHookError(Exception): | 685 | class UnregisteredHookError(Exception): |
936 | 548 | """Raised when an undefined hook is called""" | 686 | """Raised when an undefined hook is called""" |
937 | 549 | pass | 687 | pass |
938 | @@ -571,10 +709,14 @@ | |||
939 | 571 | hooks.execute(sys.argv) | 709 | hooks.execute(sys.argv) |
940 | 572 | """ | 710 | """ |
941 | 573 | 711 | ||
943 | 574 | def __init__(self, config_save=True): | 712 | def __init__(self, config_save=None): |
944 | 575 | super(Hooks, self).__init__() | 713 | super(Hooks, self).__init__() |
945 | 576 | self._hooks = {} | 714 | self._hooks = {} |
947 | 577 | self._config_save = config_save | 715 | |
948 | 716 | # For unknown reasons, we allow the Hooks constructor to override | ||
949 | 717 | # config().implicit_save. | ||
950 | 718 | if config_save is not None: | ||
951 | 719 | config().implicit_save = config_save | ||
952 | 578 | 720 | ||
953 | 579 | def register(self, name, function): | 721 | def register(self, name, function): |
954 | 580 | """Register a hook""" | 722 | """Register a hook""" |
955 | @@ -582,13 +724,16 @@ | |||
956 | 582 | 724 | ||
957 | 583 | def execute(self, args): | 725 | def execute(self, args): |
958 | 584 | """Execute a registered hook based on args[0]""" | 726 | """Execute a registered hook based on args[0]""" |
959 | 727 | _run_atstart() | ||
960 | 585 | hook_name = os.path.basename(args[0]) | 728 | hook_name = os.path.basename(args[0]) |
961 | 586 | if hook_name in self._hooks: | 729 | if hook_name in self._hooks: |
967 | 587 | self._hooks[hook_name]() | 730 | try: |
968 | 588 | if self._config_save: | 731 | self._hooks[hook_name]() |
969 | 589 | cfg = config() | 732 | except SystemExit as x: |
970 | 590 | if cfg.implicit_save: | 733 | if x.code is None or x.code == 0: |
971 | 591 | cfg.save() | 734 | _run_atexit() |
972 | 735 | raise | ||
973 | 736 | _run_atexit() | ||
974 | 592 | else: | 737 | else: |
975 | 593 | raise UnregisteredHookError(hook_name) | 738 | raise UnregisteredHookError(hook_name) |
976 | 594 | 739 | ||
977 | @@ -637,6 +782,21 @@ | |||
978 | 637 | subprocess.check_call(['action-fail', message]) | 782 | subprocess.check_call(['action-fail', message]) |
979 | 638 | 783 | ||
980 | 639 | 784 | ||
981 | 785 | def action_name(): | ||
982 | 786 | """Get the name of the currently executing action.""" | ||
983 | 787 | return os.environ.get('JUJU_ACTION_NAME') | ||
984 | 788 | |||
985 | 789 | |||
986 | 790 | def action_uuid(): | ||
987 | 791 | """Get the UUID of the currently executing action.""" | ||
988 | 792 | return os.environ.get('JUJU_ACTION_UUID') | ||
989 | 793 | |||
990 | 794 | |||
991 | 795 | def action_tag(): | ||
992 | 796 | """Get the tag for the currently executing action.""" | ||
993 | 797 | return os.environ.get('JUJU_ACTION_TAG') | ||
994 | 798 | |||
995 | 799 | |||
996 | 640 | def status_set(workload_state, message): | 800 | def status_set(workload_state, message): |
997 | 641 | """Set the workload state with a message | 801 | """Set the workload state with a message |
998 | 642 | 802 | ||
999 | @@ -666,18 +826,243 @@ | |||
1000 | 666 | 826 | ||
1001 | 667 | 827 | ||
1002 | 668 | def status_get(): | 828 | def status_get(): |
1007 | 669 | """Retrieve the previously set juju workload state | 829 | """Retrieve the previously set juju workload state and message |
1008 | 670 | 830 | ||
1009 | 671 | If the status-set command is not found then assume this is juju < 1.23 and | 831 | If the status-get command is not found then assume this is juju < 1.23 and |
1010 | 672 | return 'unknown' | 832 | return 'unknown', "" |
1011 | 833 | |||
1012 | 673 | """ | 834 | """ |
1014 | 674 | cmd = ['status-get'] | 835 | cmd = ['status-get', "--format=json", "--include-data"] |
1015 | 675 | try: | 836 | try: |
1019 | 676 | raw_status = subprocess.check_output(cmd, universal_newlines=True) | 837 | raw_status = subprocess.check_output(cmd) |
1017 | 677 | status = raw_status.rstrip() | ||
1018 | 678 | return status | ||
1020 | 679 | except OSError as e: | 838 | except OSError as e: |
1021 | 680 | if e.errno == errno.ENOENT: | 839 | if e.errno == errno.ENOENT: |
1025 | 681 | return 'unknown' | 840 | return ('unknown', "") |
1026 | 682 | else: | 841 | else: |
1027 | 683 | raise | 842 | raise |
1028 | 843 | else: | ||
1029 | 844 | status = json.loads(raw_status.decode("UTF-8")) | ||
1030 | 845 | return (status["status"], status["message"]) | ||
1031 | 846 | |||
1032 | 847 | |||
1033 | 848 | def translate_exc(from_exc, to_exc): | ||
1034 | 849 | def inner_translate_exc1(f): | ||
1035 | 850 | @wraps(f) | ||
1036 | 851 | def inner_translate_exc2(*args, **kwargs): | ||
1037 | 852 | try: | ||
1038 | 853 | return f(*args, **kwargs) | ||
1039 | 854 | except from_exc: | ||
1040 | 855 | raise to_exc | ||
1041 | 856 | |||
1042 | 857 | return inner_translate_exc2 | ||
1043 | 858 | |||
1044 | 859 | return inner_translate_exc1 | ||
1045 | 860 | |||
1046 | 861 | |||
1047 | 862 | def application_version_set(version): | ||
1048 | 863 | """Charm authors may trigger this command from any hook to output what | ||
1049 | 864 | version of the application is running. This could be a package version, | ||
1050 | 865 | for instance postgres version 9.5. It could also be a build number or | ||
1051 | 866 | version control revision identifier, for instance git sha 6fb7ba68. """ | ||
1052 | 867 | |||
1053 | 868 | cmd = ['application-version-set'] | ||
1054 | 869 | cmd.append(version) | ||
1055 | 870 | try: | ||
1056 | 871 | subprocess.check_call(cmd) | ||
1057 | 872 | except OSError: | ||
1058 | 873 | log("Application Version: {}".format(version)) | ||
1059 | 874 | |||
1060 | 875 | |||
1061 | 876 | @translate_exc(from_exc=OSError, to_exc=NotImplementedError) | ||
1062 | 877 | def is_leader(): | ||
1063 | 878 | """Does the current unit hold the juju leadership | ||
1064 | 879 | |||
1065 | 880 | Uses juju to determine whether the current unit is the leader of its peers | ||
1066 | 881 | """ | ||
1067 | 882 | cmd = ['is-leader', '--format=json'] | ||
1068 | 883 | return json.loads(subprocess.check_output(cmd).decode('UTF-8')) | ||
1069 | 884 | |||
1070 | 885 | |||
1071 | 886 | @translate_exc(from_exc=OSError, to_exc=NotImplementedError) | ||
1072 | 887 | def leader_get(attribute=None): | ||
1073 | 888 | """Juju leader get value(s)""" | ||
1074 | 889 | cmd = ['leader-get', '--format=json'] + [attribute or '-'] | ||
1075 | 890 | return json.loads(subprocess.check_output(cmd).decode('UTF-8')) | ||
1076 | 891 | |||
1077 | 892 | |||
1078 | 893 | @translate_exc(from_exc=OSError, to_exc=NotImplementedError) | ||
1079 | 894 | def leader_set(settings=None, **kwargs): | ||
1080 | 895 | """Juju leader set value(s)""" | ||
1081 | 896 | # Don't log secrets. | ||
1082 | 897 | # log("Juju leader-set '%s'" % (settings), level=DEBUG) | ||
1083 | 898 | cmd = ['leader-set'] | ||
1084 | 899 | settings = settings or {} | ||
1085 | 900 | settings.update(kwargs) | ||
1086 | 901 | for k, v in settings.items(): | ||
1087 | 902 | if v is None: | ||
1088 | 903 | cmd.append('{}='.format(k)) | ||
1089 | 904 | else: | ||
1090 | 905 | cmd.append('{}={}'.format(k, v)) | ||
1091 | 906 | subprocess.check_call(cmd) | ||
1092 | 907 | |||
1093 | 908 | |||
1094 | 909 | @translate_exc(from_exc=OSError, to_exc=NotImplementedError) | ||
1095 | 910 | def payload_register(ptype, klass, pid): | ||
1096 | 911 | """ is used while a hook is running to let Juju know that a | ||
1097 | 912 | payload has been started.""" | ||
1098 | 913 | cmd = ['payload-register'] | ||
1099 | 914 | for x in [ptype, klass, pid]: | ||
1100 | 915 | cmd.append(x) | ||
1101 | 916 | subprocess.check_call(cmd) | ||
1102 | 917 | |||
1103 | 918 | |||
1104 | 919 | @translate_exc(from_exc=OSError, to_exc=NotImplementedError) | ||
1105 | 920 | def payload_unregister(klass, pid): | ||
1106 | 921 | """ is used while a hook is running to let Juju know | ||
1107 | 922 | that a payload has been manually stopped. The <class> and <id> provided | ||
1108 | 923 | must match a payload that has been previously registered with juju using | ||
1109 | 924 | payload-register.""" | ||
1110 | 925 | cmd = ['payload-unregister'] | ||
1111 | 926 | for x in [klass, pid]: | ||
1112 | 927 | cmd.append(x) | ||
1113 | 928 | subprocess.check_call(cmd) | ||
1114 | 929 | |||
1115 | 930 | |||
1116 | 931 | @translate_exc(from_exc=OSError, to_exc=NotImplementedError) | ||
1117 | 932 | def payload_status_set(klass, pid, status): | ||
1118 | 933 | """is used to update the current status of a registered payload. | ||
1119 | 934 | The <class> and <id> provided must match a payload that has been previously | ||
1120 | 935 | registered with juju using payload-register. The <status> must be one of the | ||
1121 | 936 | follow: starting, started, stopping, stopped""" | ||
1122 | 937 | cmd = ['payload-status-set'] | ||
1123 | 938 | for x in [klass, pid, status]: | ||
1124 | 939 | cmd.append(x) | ||
1125 | 940 | subprocess.check_call(cmd) | ||
1126 | 941 | |||
1127 | 942 | |||
1128 | 943 | @translate_exc(from_exc=OSError, to_exc=NotImplementedError) | ||
1129 | 944 | def resource_get(name): | ||
1130 | 945 | """used to fetch the resource path of the given name. | ||
1131 | 946 | |||
1132 | 947 | <name> must match a name of defined resource in metadata.yaml | ||
1133 | 948 | |||
1134 | 949 | returns either a path or False if resource not available | ||
1135 | 950 | """ | ||
1136 | 951 | if not name: | ||
1137 | 952 | return False | ||
1138 | 953 | |||
1139 | 954 | cmd = ['resource-get', name] | ||
1140 | 955 | try: | ||
1141 | 956 | return subprocess.check_output(cmd).decode('UTF-8') | ||
1142 | 957 | except subprocess.CalledProcessError: | ||
1143 | 958 | return False | ||
1144 | 959 | |||
1145 | 960 | |||
1146 | 961 | @cached | ||
1147 | 962 | def juju_version(): | ||
1148 | 963 | """Full version string (eg. '1.23.3.1-trusty-amd64')""" | ||
1149 | 964 | # Per https://bugs.launchpad.net/juju-core/+bug/1455368/comments/1 | ||
1150 | 965 | jujud = glob.glob('/var/lib/juju/tools/machine-*/jujud')[0] | ||
1151 | 966 | return subprocess.check_output([jujud, 'version'], | ||
1152 | 967 | universal_newlines=True).strip() | ||
1153 | 968 | |||
1154 | 969 | |||
1155 | 970 | @cached | ||
1156 | 971 | def has_juju_version(minimum_version): | ||
1157 | 972 | """Return True if the Juju version is at least the provided version""" | ||
1158 | 973 | return LooseVersion(juju_version()) >= LooseVersion(minimum_version) | ||
1159 | 974 | |||
1160 | 975 | |||
1161 | 976 | _atexit = [] | ||
1162 | 977 | _atstart = [] | ||
1163 | 978 | |||
1164 | 979 | |||
1165 | 980 | def atstart(callback, *args, **kwargs): | ||
1166 | 981 | '''Schedule a callback to run before the main hook. | ||
1167 | 982 | |||
1168 | 983 | Callbacks are run in the order they were added. | ||
1169 | 984 | |||
1170 | 985 | This is useful for modules and classes to perform initialization | ||
1171 | 986 | and inject behavior. In particular: | ||
1172 | 987 | |||
1173 | 988 | - Run common code before all of your hooks, such as logging | ||
1174 | 989 | the hook name or interesting relation data. | ||
1175 | 990 | - Defer object or module initialization that requires a hook | ||
1176 | 991 | context until we know there actually is a hook context, | ||
1177 | 992 | making testing easier. | ||
1178 | 993 | - Rather than requiring charm authors to include boilerplate to | ||
1179 | 994 | invoke your helper's behavior, have it run automatically if | ||
1180 | 995 | your object is instantiated or module imported. | ||
1181 | 996 | |||
1182 | 997 | This is not at all useful after your hook framework as been launched. | ||
1183 | 998 | ''' | ||
1184 | 999 | global _atstart | ||
1185 | 1000 | _atstart.append((callback, args, kwargs)) | ||
1186 | 1001 | |||
1187 | 1002 | |||
1188 | 1003 | def atexit(callback, *args, **kwargs): | ||
1189 | 1004 | '''Schedule a callback to run on successful hook completion. | ||
1190 | 1005 | |||
1191 | 1006 | Callbacks are run in the reverse order that they were added.''' | ||
1192 | 1007 | _atexit.append((callback, args, kwargs)) | ||
1193 | 1008 | |||
1194 | 1009 | |||
1195 | 1010 | def _run_atstart(): | ||
1196 | 1011 | '''Hook frameworks must invoke this before running the main hook body.''' | ||
1197 | 1012 | global _atstart | ||
1198 | 1013 | for callback, args, kwargs in _atstart: | ||
1199 | 1014 | callback(*args, **kwargs) | ||
1200 | 1015 | del _atstart[:] | ||
1201 | 1016 | |||
1202 | 1017 | |||
1203 | 1018 | def _run_atexit(): | ||
1204 | 1019 | '''Hook frameworks must invoke this after the main hook body has | ||
1205 | 1020 | successfully completed. Do not invoke it if the hook fails.''' | ||
1206 | 1021 | global _atexit | ||
1207 | 1022 | for callback, args, kwargs in reversed(_atexit): | ||
1208 | 1023 | callback(*args, **kwargs) | ||
1209 | 1024 | del _atexit[:] | ||
1210 | 1025 | |||
1211 | 1026 | |||
1212 | 1027 | @translate_exc(from_exc=OSError, to_exc=NotImplementedError) | ||
1213 | 1028 | def network_get_primary_address(binding): | ||
1214 | 1029 | ''' | ||
1215 | 1030 | Retrieve the primary network address for a named binding | ||
1216 | 1031 | |||
1217 | 1032 | :param binding: string. The name of a relation of extra-binding | ||
1218 | 1033 | :return: string. The primary IP address for the named binding | ||
1219 | 1034 | :raise: NotImplementedError if run on Juju < 2.0 | ||
1220 | 1035 | ''' | ||
1221 | 1036 | cmd = ['network-get', '--primary-address', binding] | ||
1222 | 1037 | return subprocess.check_output(cmd).decode('UTF-8').strip() | ||
1223 | 1038 | |||
1224 | 1039 | |||
1225 | 1040 | def add_metric(*args, **kwargs): | ||
1226 | 1041 | """Add metric values. Values may be expressed with keyword arguments. For | ||
1227 | 1042 | metric names containing dashes, these may be expressed as one or more | ||
1228 | 1043 | 'key=value' positional arguments. May only be called from the collect-metrics | ||
1229 | 1044 | hook.""" | ||
1230 | 1045 | _args = ['add-metric'] | ||
1231 | 1046 | _kvpairs = [] | ||
1232 | 1047 | _kvpairs.extend(args) | ||
1233 | 1048 | _kvpairs.extend(['{}={}'.format(k, v) for k, v in kwargs.items()]) | ||
1234 | 1049 | _args.extend(sorted(_kvpairs)) | ||
1235 | 1050 | try: | ||
1236 | 1051 | subprocess.check_call(_args) | ||
1237 | 1052 | return | ||
1238 | 1053 | except EnvironmentError as e: | ||
1239 | 1054 | if e.errno != errno.ENOENT: | ||
1240 | 1055 | raise | ||
1241 | 1056 | log_message = 'add-metric failed: {}'.format(' '.join(_kvpairs)) | ||
1242 | 1057 | log(log_message, level='INFO') | ||
1243 | 1058 | |||
1244 | 1059 | |||
1245 | 1060 | def meter_status(): | ||
1246 | 1061 | """Get the meter status, if running in the meter-status-changed hook.""" | ||
1247 | 1062 | return os.environ.get('JUJU_METER_STATUS') | ||
1248 | 1063 | |||
1249 | 1064 | |||
1250 | 1065 | def meter_info(): | ||
1251 | 1066 | """Get the meter status information, if running in the meter-status-changed | ||
1252 | 1067 | hook.""" | ||
1253 | 1068 | return os.environ.get('JUJU_METER_INFO') | ||
1254 | 684 | 1069 | ||
1255 | === modified file 'hooks/charmhelpers/core/host.py' | |||
1256 | --- hooks/charmhelpers/core/host.py 2015-05-14 10:48:09 +0000 | |||
1257 | +++ hooks/charmhelpers/core/host.py 2018-06-12 20:07:13 +0000 | |||
1258 | @@ -1,18 +1,16 @@ | |||
1259 | 1 | # Copyright 2014-2015 Canonical Limited. | 1 | # Copyright 2014-2015 Canonical Limited. |
1260 | 2 | # | 2 | # |
1274 | 3 | # This file is part of charm-helpers. | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); |
1275 | 4 | # | 4 | # you may not use this file except in compliance with the License. |
1276 | 5 | # charm-helpers is free software: you can redistribute it and/or modify | 5 | # You may obtain a copy of the License at |
1277 | 6 | # it under the terms of the GNU Lesser General Public License version 3 as | 6 | # |
1278 | 7 | # published by the Free Software Foundation. | 7 | # http://www.apache.org/licenses/LICENSE-2.0 |
1279 | 8 | # | 8 | # |
1280 | 9 | # charm-helpers is distributed in the hope that it will be useful, | 9 | # Unless required by applicable law or agreed to in writing, software |
1281 | 10 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 10 | # distributed under the License is distributed on an "AS IS" BASIS, |
1282 | 11 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
1283 | 12 | # GNU Lesser General Public License for more details. | 12 | # See the License for the specific language governing permissions and |
1284 | 13 | # | 13 | # limitations under the License. |
1272 | 14 | # You should have received a copy of the GNU Lesser General Public License | ||
1273 | 15 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
1285 | 16 | 14 | ||
1286 | 17 | """Tools for working with the host system""" | 15 | """Tools for working with the host system""" |
1287 | 18 | # Copyright 2012 Canonical Ltd. | 16 | # Copyright 2012 Canonical Ltd. |
1288 | @@ -24,85 +22,326 @@ | |||
1289 | 24 | import os | 22 | import os |
1290 | 25 | import re | 23 | import re |
1291 | 26 | import pwd | 24 | import pwd |
1292 | 25 | import glob | ||
1293 | 27 | import grp | 26 | import grp |
1294 | 28 | import random | 27 | import random |
1295 | 29 | import string | 28 | import string |
1296 | 30 | import subprocess | 29 | import subprocess |
1297 | 31 | import hashlib | 30 | import hashlib |
1298 | 31 | import functools | ||
1299 | 32 | import itertools | ||
1300 | 33 | import six | ||
1301 | 34 | |||
1302 | 32 | from contextlib import contextmanager | 35 | from contextlib import contextmanager |
1303 | 33 | from collections import OrderedDict | 36 | from collections import OrderedDict |
1304 | 34 | |||
1305 | 35 | import six | ||
1306 | 36 | |||
1307 | 37 | from .hookenv import log | 37 | from .hookenv import log |
1308 | 38 | from .fstab import Fstab | 38 | from .fstab import Fstab |
1323 | 39 | 39 | from charmhelpers.osplatform import get_platform | |
1324 | 40 | 40 | ||
1325 | 41 | def service_start(service_name): | 41 | __platform__ = get_platform() |
1326 | 42 | """Start a system service""" | 42 | if __platform__ == "ubuntu": |
1327 | 43 | return service('start', service_name) | 43 | from charmhelpers.core.host_factory.ubuntu import ( |
1328 | 44 | 44 | service_available, | |
1329 | 45 | 45 | add_new_group, | |
1330 | 46 | def service_stop(service_name): | 46 | lsb_release, |
1331 | 47 | """Stop a system service""" | 47 | cmp_pkgrevno, |
1332 | 48 | return service('stop', service_name) | 48 | ) # flake8: noqa -- ignore F401 for this import |
1333 | 49 | 49 | elif __platform__ == "centos": | |
1334 | 50 | 50 | from charmhelpers.core.host_factory.centos import ( | |
1335 | 51 | def service_restart(service_name): | 51 | service_available, |
1336 | 52 | """Restart a system service""" | 52 | add_new_group, |
1337 | 53 | lsb_release, | ||
1338 | 54 | cmp_pkgrevno, | ||
1339 | 55 | ) # flake8: noqa -- ignore F401 for this import | ||
1340 | 56 | |||
1341 | 57 | UPDATEDB_PATH = '/etc/updatedb.conf' | ||
1342 | 58 | |||
1343 | 59 | def service_start(service_name, **kwargs): | ||
1344 | 60 | """Start a system service. | ||
1345 | 61 | |||
1346 | 62 | The specified service name is managed via the system level init system. | ||
1347 | 63 | Some init systems (e.g. upstart) require that additional arguments be | ||
1348 | 64 | provided in order to directly control service instances whereas other init | ||
1349 | 65 | systems allow for addressing instances of a service directly by name (e.g. | ||
1350 | 66 | systemd). | ||
1351 | 67 | |||
1352 | 68 | The kwargs allow for the additional parameters to be passed to underlying | ||
1353 | 69 | init systems for those systems which require/allow for them. For example, | ||
1354 | 70 | the ceph-osd upstart script requires the id parameter to be passed along | ||
1355 | 71 | in order to identify which running daemon should be reloaded. The follow- | ||
1356 | 72 | ing example stops the ceph-osd service for instance id=4: | ||
1357 | 73 | |||
1358 | 74 | service_stop('ceph-osd', id=4) | ||
1359 | 75 | |||
1360 | 76 | :param service_name: the name of the service to stop | ||
1361 | 77 | :param **kwargs: additional parameters to pass to the init system when | ||
1362 | 78 | managing services. These will be passed as key=value | ||
1363 | 79 | parameters to the init system's commandline. kwargs | ||
1364 | 80 | are ignored for systemd enabled systems. | ||
1365 | 81 | """ | ||
1366 | 82 | return service('start', service_name, **kwargs) | ||
1367 | 83 | |||
1368 | 84 | |||
1369 | 85 | def service_stop(service_name, **kwargs): | ||
1370 | 86 | """Stop a system service. | ||
1371 | 87 | |||
1372 | 88 | The specified service name is managed via the system level init system. | ||
1373 | 89 | Some init systems (e.g. upstart) require that additional arguments be | ||
1374 | 90 | provided in order to directly control service instances whereas other init | ||
1375 | 91 | systems allow for addressing instances of a service directly by name (e.g. | ||
1376 | 92 | systemd). | ||
1377 | 93 | |||
1378 | 94 | The kwargs allow for the additional parameters to be passed to underlying | ||
1379 | 95 | init systems for those systems which require/allow for them. For example, | ||
1380 | 96 | the ceph-osd upstart script requires the id parameter to be passed along | ||
1381 | 97 | in order to identify which running daemon should be reloaded. The follow- | ||
1382 | 98 | ing example stops the ceph-osd service for instance id=4: | ||
1383 | 99 | |||
1384 | 100 | service_stop('ceph-osd', id=4) | ||
1385 | 101 | |||
1386 | 102 | :param service_name: the name of the service to stop | ||
1387 | 103 | :param **kwargs: additional parameters to pass to the init system when | ||
1388 | 104 | managing services. These will be passed as key=value | ||
1389 | 105 | parameters to the init system's commandline. kwargs | ||
1390 | 106 | are ignored for systemd enabled systems. | ||
1391 | 107 | """ | ||
1392 | 108 | return service('stop', service_name, **kwargs) | ||
1393 | 109 | |||
1394 | 110 | |||
1395 | 111 | def service_restart(service_name, **kwargs): | ||
1396 | 112 | """Restart a system service. | ||
1397 | 113 | |||
1398 | 114 | The specified service name is managed via the system level init system. | ||
1399 | 115 | Some init systems (e.g. upstart) require that additional arguments be | ||
1400 | 116 | provided in order to directly control service instances whereas other init | ||
1401 | 117 | systems allow for addressing instances of a service directly by name (e.g. | ||
1402 | 118 | systemd). | ||
1403 | 119 | |||
1404 | 120 | The kwargs allow for the additional parameters to be passed to underlying | ||
1405 | 121 | init systems for those systems which require/allow for them. For example, | ||
1406 | 122 | the ceph-osd upstart script requires the id parameter to be passed along | ||
1407 | 123 | in order to identify which running daemon should be restarted. The follow- | ||
1408 | 124 | ing example restarts the ceph-osd service for instance id=4: | ||
1409 | 125 | |||
1410 | 126 | service_restart('ceph-osd', id=4) | ||
1411 | 127 | |||
1412 | 128 | :param service_name: the name of the service to restart | ||
1413 | 129 | :param **kwargs: additional parameters to pass to the init system when | ||
1414 | 130 | managing services. These will be passed as key=value | ||
1415 | 131 | parameters to the init system's commandline. kwargs | ||
1416 | 132 | are ignored for init systems not allowing additional | ||
1417 | 133 | parameters via the commandline (systemd). | ||
1418 | 134 | """ | ||
1419 | 53 | return service('restart', service_name) | 135 | return service('restart', service_name) |
1420 | 54 | 136 | ||
1421 | 55 | 137 | ||
1423 | 56 | def service_reload(service_name, restart_on_failure=False): | 138 | def service_reload(service_name, restart_on_failure=False, **kwargs): |
1424 | 57 | """Reload a system service, optionally falling back to restart if | 139 | """Reload a system service, optionally falling back to restart if |
1427 | 58 | reload fails""" | 140 | reload fails. |
1428 | 59 | service_result = service('reload', service_name) | 141 | |
1429 | 142 | The specified service name is managed via the system level init system. | ||
1430 | 143 | Some init systems (e.g. upstart) require that additional arguments be | ||
1431 | 144 | provided in order to directly control service instances whereas other init | ||
1432 | 145 | systems allow for addressing instances of a service directly by name (e.g. | ||
1433 | 146 | systemd). | ||
1434 | 147 | |||
1435 | 148 | The kwargs allow for the additional parameters to be passed to underlying | ||
1436 | 149 | init systems for those systems which require/allow for them. For example, | ||
1437 | 150 | the ceph-osd upstart script requires the id parameter to be passed along | ||
1438 | 151 | in order to identify which running daemon should be reloaded. The follow- | ||
1439 | 152 | ing example restarts the ceph-osd service for instance id=4: | ||
1440 | 153 | |||
1441 | 154 | service_reload('ceph-osd', id=4) | ||
1442 | 155 | |||
1443 | 156 | :param service_name: the name of the service to reload | ||
1444 | 157 | :param restart_on_failure: boolean indicating whether to fallback to a | ||
1445 | 158 | restart if the reload fails. | ||
1446 | 159 | :param **kwargs: additional parameters to pass to the init system when | ||
1447 | 160 | managing services. These will be passed as key=value | ||
1448 | 161 | parameters to the init system's commandline. kwargs | ||
1449 | 162 | are ignored for init systems not allowing additional | ||
1450 | 163 | parameters via the commandline (systemd). | ||
1451 | 164 | """ | ||
1452 | 165 | service_result = service('reload', service_name, **kwargs) | ||
1453 | 60 | if not service_result and restart_on_failure: | 166 | if not service_result and restart_on_failure: |
1455 | 61 | service_result = service('restart', service_name) | 167 | service_result = service('restart', service_name, **kwargs) |
1456 | 62 | return service_result | 168 | return service_result |
1457 | 63 | 169 | ||
1458 | 64 | 170 | ||
1462 | 65 | def service(action, service_name): | 171 | def service_pause(service_name, init_dir="/etc/init", initd_dir="/etc/init.d", |
1463 | 66 | """Control a system service""" | 172 | **kwargs): |
1464 | 67 | cmd = ['service', service_name, action] | 173 | """Pause a system service. |
1465 | 174 | |||
1466 | 175 | Stop it, and prevent it from starting again at boot. | ||
1467 | 176 | |||
1468 | 177 | :param service_name: the name of the service to pause | ||
1469 | 178 | :param init_dir: path to the upstart init directory | ||
1470 | 179 | :param initd_dir: path to the sysv init directory | ||
1471 | 180 | :param **kwargs: additional parameters to pass to the init system when | ||
1472 | 181 | managing services. These will be passed as key=value | ||
1473 | 182 | parameters to the init system's commandline. kwargs | ||
1474 | 183 | are ignored for init systems which do not support | ||
1475 | 184 | key=value arguments via the commandline. | ||
1476 | 185 | """ | ||
1477 | 186 | stopped = True | ||
1478 | 187 | if service_running(service_name, **kwargs): | ||
1479 | 188 | stopped = service_stop(service_name, **kwargs) | ||
1480 | 189 | upstart_file = os.path.join(init_dir, "{}.conf".format(service_name)) | ||
1481 | 190 | sysv_file = os.path.join(initd_dir, service_name) | ||
1482 | 191 | if init_is_systemd(): | ||
1483 | 192 | service('disable', service_name) | ||
1484 | 193 | elif os.path.exists(upstart_file): | ||
1485 | 194 | override_path = os.path.join( | ||
1486 | 195 | init_dir, '{}.override'.format(service_name)) | ||
1487 | 196 | with open(override_path, 'w') as fh: | ||
1488 | 197 | fh.write("manual\n") | ||
1489 | 198 | elif os.path.exists(sysv_file): | ||
1490 | 199 | subprocess.check_call(["update-rc.d", service_name, "disable"]) | ||
1491 | 200 | else: | ||
1492 | 201 | raise ValueError( | ||
1493 | 202 | "Unable to detect {0} as SystemD, Upstart {1} or" | ||
1494 | 203 | " SysV {2}".format( | ||
1495 | 204 | service_name, upstart_file, sysv_file)) | ||
1496 | 205 | return stopped | ||
1497 | 206 | |||
1498 | 207 | |||
1499 | 208 | def service_resume(service_name, init_dir="/etc/init", | ||
1500 | 209 | initd_dir="/etc/init.d", **kwargs): | ||
1501 | 210 | """Resume a system service. | ||
1502 | 211 | |||
1503 | 212 | Reenable starting again at boot. Start the service. | ||
1504 | 213 | |||
1505 | 214 | :param service_name: the name of the service to resume | ||
1506 | 215 | :param init_dir: the path to the init dir | ||
1507 | 216 | :param initd dir: the path to the initd dir | ||
1508 | 217 | :param **kwargs: additional parameters to pass to the init system when | ||
1509 | 218 | managing services. These will be passed as key=value | ||
1510 | 219 | parameters to the init system's commandline. kwargs | ||
1511 | 220 | are ignored for systemd enabled systems. | ||
1512 | 221 | """ | ||
1513 | 222 | upstart_file = os.path.join(init_dir, "{}.conf".format(service_name)) | ||
1514 | 223 | sysv_file = os.path.join(initd_dir, service_name) | ||
1515 | 224 | if init_is_systemd(): | ||
1516 | 225 | service('enable', service_name) | ||
1517 | 226 | elif os.path.exists(upstart_file): | ||
1518 | 227 | override_path = os.path.join( | ||
1519 | 228 | init_dir, '{}.override'.format(service_name)) | ||
1520 | 229 | if os.path.exists(override_path): | ||
1521 | 230 | os.unlink(override_path) | ||
1522 | 231 | elif os.path.exists(sysv_file): | ||
1523 | 232 | subprocess.check_call(["update-rc.d", service_name, "enable"]) | ||
1524 | 233 | else: | ||
1525 | 234 | raise ValueError( | ||
1526 | 235 | "Unable to detect {0} as SystemD, Upstart {1} or" | ||
1527 | 236 | " SysV {2}".format( | ||
1528 | 237 | service_name, upstart_file, sysv_file)) | ||
1529 | 238 | started = service_running(service_name, **kwargs) | ||
1530 | 239 | |||
1531 | 240 | if not started: | ||
1532 | 241 | started = service_start(service_name, **kwargs) | ||
1533 | 242 | return started | ||
1534 | 243 | |||
1535 | 244 | |||
1536 | 245 | def service(action, service_name, **kwargs): | ||
1537 | 246 | """Control a system service. | ||
1538 | 247 | |||
1539 | 248 | :param action: the action to take on the service | ||
1540 | 249 | :param service_name: the name of the service to perform th action on | ||
1541 | 250 | :param **kwargs: additional params to be passed to the service command in | ||
1542 | 251 | the form of key=value. | ||
1543 | 252 | """ | ||
1544 | 253 | if init_is_systemd(): | ||
1545 | 254 | cmd = ['systemctl', action, service_name] | ||
1546 | 255 | else: | ||
1547 | 256 | cmd = ['service', service_name, action] | ||
1548 | 257 | for key, value in six.iteritems(kwargs): | ||
1549 | 258 | parameter = '%s=%s' % (key, value) | ||
1550 | 259 | cmd.append(parameter) | ||
1551 | 68 | return subprocess.call(cmd) == 0 | 260 | return subprocess.call(cmd) == 0 |
1552 | 69 | 261 | ||
1553 | 70 | 262 | ||
1583 | 71 | def service_running(service): | 263 | _UPSTART_CONF = "/etc/init/{}.conf" |
1584 | 72 | """Determine whether a system service is running""" | 264 | _INIT_D_CONF = "/etc/init.d/{}" |
1585 | 73 | try: | 265 | |
1586 | 74 | output = subprocess.check_output( | 266 | |
1587 | 75 | ['service', service, 'status'], | 267 | def service_running(service_name, **kwargs): |
1588 | 76 | stderr=subprocess.STDOUT).decode('UTF-8') | 268 | """Determine whether a system service is running. |
1589 | 77 | except subprocess.CalledProcessError: | 269 | |
1590 | 78 | return False | 270 | :param service_name: the name of the service |
1591 | 79 | else: | 271 | :param **kwargs: additional args to pass to the service command. This is |
1592 | 80 | if ("start/running" in output or "is running" in output): | 272 | used to pass additional key=value arguments to the |
1593 | 81 | return True | 273 | service command line for managing specific instance |
1594 | 82 | else: | 274 | units (e.g. service ceph-osd status id=2). The kwargs |
1595 | 83 | return False | 275 | are ignored in systemd services. |
1596 | 84 | 276 | """ | |
1597 | 85 | 277 | if init_is_systemd(): | |
1598 | 86 | def service_available(service_name): | 278 | return service('is-active', service_name) |
1599 | 87 | """Determine whether a system service is available""" | 279 | else: |
1600 | 88 | try: | 280 | if os.path.exists(_UPSTART_CONF.format(service_name)): |
1601 | 89 | subprocess.check_output( | 281 | try: |
1602 | 90 | ['service', service_name, 'status'], | 282 | cmd = ['status', service_name] |
1603 | 91 | stderr=subprocess.STDOUT).decode('UTF-8') | 283 | for key, value in six.iteritems(kwargs): |
1604 | 92 | except subprocess.CalledProcessError as e: | 284 | parameter = '%s=%s' % (key, value) |
1605 | 93 | return b'unrecognized service' not in e.output | 285 | cmd.append(parameter) |
1606 | 94 | else: | 286 | output = subprocess.check_output(cmd, |
1607 | 95 | return True | 287 | stderr=subprocess.STDOUT).decode('UTF-8') |
1608 | 96 | 288 | except subprocess.CalledProcessError: | |
1609 | 97 | 289 | return False | |
1610 | 98 | def adduser(username, password=None, shell='/bin/bash', system_user=False): | 290 | else: |
1611 | 99 | """Add a user to the system""" | 291 | # This works for upstart scripts where the 'service' command |
1612 | 292 | # returns a consistent string to represent running | ||
1613 | 293 | # 'start/running' | ||
1614 | 294 | if ("start/running" in output or | ||
1615 | 295 | "is running" in output or | ||
1616 | 296 | "up and running" in output): | ||
1617 | 297 | return True | ||
1618 | 298 | elif os.path.exists(_INIT_D_CONF.format(service_name)): | ||
1619 | 299 | # Check System V scripts init script return codes | ||
1620 | 300 | return service('status', service_name) | ||
1621 | 301 | return False | ||
1622 | 302 | |||
1623 | 303 | |||
1624 | 304 | SYSTEMD_SYSTEM = '/run/systemd/system' | ||
1625 | 305 | |||
1626 | 306 | |||
1627 | 307 | def init_is_systemd(): | ||
1628 | 308 | """Return True if the host system uses systemd, False otherwise.""" | ||
1629 | 309 | if lsb_release()['DISTRIB_CODENAME'] == 'trusty': | ||
1630 | 310 | return False | ||
1631 | 311 | return os.path.isdir(SYSTEMD_SYSTEM) | ||
1632 | 312 | |||
1633 | 313 | |||
1634 | 314 | def adduser(username, password=None, shell='/bin/bash', | ||
1635 | 315 | system_user=False, primary_group=None, | ||
1636 | 316 | secondary_groups=None, uid=None, home_dir=None): | ||
1637 | 317 | """Add a user to the system. | ||
1638 | 318 | |||
1639 | 319 | Will log but otherwise succeed if the user already exists. | ||
1640 | 320 | |||
1641 | 321 | :param str username: Username to create | ||
1642 | 322 | :param str password: Password for user; if ``None``, create a system user | ||
1643 | 323 | :param str shell: The default shell for the user | ||
1644 | 324 | :param bool system_user: Whether to create a login or system user | ||
1645 | 325 | :param str primary_group: Primary group for user; defaults to username | ||
1646 | 326 | :param list secondary_groups: Optional list of additional groups | ||
1647 | 327 | :param int uid: UID for user being created | ||
1648 | 328 | :param str home_dir: Home directory for user | ||
1649 | 329 | |||
1650 | 330 | :returns: The password database entry struct, as returned by `pwd.getpwnam` | ||
1651 | 331 | """ | ||
1652 | 100 | try: | 332 | try: |
1653 | 101 | user_info = pwd.getpwnam(username) | 333 | user_info = pwd.getpwnam(username) |
1654 | 102 | log('user {0} already exists!'.format(username)) | 334 | log('user {0} already exists!'.format(username)) |
1655 | 335 | if uid: | ||
1656 | 336 | user_info = pwd.getpwuid(int(uid)) | ||
1657 | 337 | log('user with uid {0} already exists!'.format(uid)) | ||
1658 | 103 | except KeyError: | 338 | except KeyError: |
1659 | 104 | log('creating user {0}'.format(username)) | 339 | log('creating user {0}'.format(username)) |
1660 | 105 | cmd = ['useradd'] | 340 | cmd = ['useradd'] |
1661 | 341 | if uid: | ||
1662 | 342 | cmd.extend(['--uid', str(uid)]) | ||
1663 | 343 | if home_dir: | ||
1664 | 344 | cmd.extend(['--home', str(home_dir)]) | ||
1665 | 106 | if system_user or password is None: | 345 | if system_user or password is None: |
1666 | 107 | cmd.append('--system') | 346 | cmd.append('--system') |
1667 | 108 | else: | 347 | else: |
1668 | @@ -111,52 +350,104 @@ | |||
1669 | 111 | '--shell', shell, | 350 | '--shell', shell, |
1670 | 112 | '--password', password, | 351 | '--password', password, |
1671 | 113 | ]) | 352 | ]) |
1672 | 353 | if not primary_group: | ||
1673 | 354 | try: | ||
1674 | 355 | grp.getgrnam(username) | ||
1675 | 356 | primary_group = username # avoid "group exists" error | ||
1676 | 357 | except KeyError: | ||
1677 | 358 | pass | ||
1678 | 359 | if primary_group: | ||
1679 | 360 | cmd.extend(['-g', primary_group]) | ||
1680 | 361 | if secondary_groups: | ||
1681 | 362 | cmd.extend(['-G', ','.join(secondary_groups)]) | ||
1682 | 114 | cmd.append(username) | 363 | cmd.append(username) |
1683 | 115 | subprocess.check_call(cmd) | 364 | subprocess.check_call(cmd) |
1684 | 116 | user_info = pwd.getpwnam(username) | 365 | user_info = pwd.getpwnam(username) |
1685 | 117 | return user_info | 366 | return user_info |
1686 | 118 | 367 | ||
1687 | 119 | 368 | ||
1690 | 120 | def add_group(group_name, system_group=False): | 369 | def user_exists(username): |
1691 | 121 | """Add a group to the system""" | 370 | """Check if a user exists""" |
1692 | 371 | try: | ||
1693 | 372 | pwd.getpwnam(username) | ||
1694 | 373 | user_exists = True | ||
1695 | 374 | except KeyError: | ||
1696 | 375 | user_exists = False | ||
1697 | 376 | return user_exists | ||
1698 | 377 | |||
1699 | 378 | |||
1700 | 379 | def uid_exists(uid): | ||
1701 | 380 | """Check if a uid exists""" | ||
1702 | 381 | try: | ||
1703 | 382 | pwd.getpwuid(uid) | ||
1704 | 383 | uid_exists = True | ||
1705 | 384 | except KeyError: | ||
1706 | 385 | uid_exists = False | ||
1707 | 386 | return uid_exists | ||
1708 | 387 | |||
1709 | 388 | |||
1710 | 389 | def group_exists(groupname): | ||
1711 | 390 | """Check if a group exists""" | ||
1712 | 391 | try: | ||
1713 | 392 | grp.getgrnam(groupname) | ||
1714 | 393 | group_exists = True | ||
1715 | 394 | except KeyError: | ||
1716 | 395 | group_exists = False | ||
1717 | 396 | return group_exists | ||
1718 | 397 | |||
1719 | 398 | |||
1720 | 399 | def gid_exists(gid): | ||
1721 | 400 | """Check if a gid exists""" | ||
1722 | 401 | try: | ||
1723 | 402 | grp.getgrgid(gid) | ||
1724 | 403 | gid_exists = True | ||
1725 | 404 | except KeyError: | ||
1726 | 405 | gid_exists = False | ||
1727 | 406 | return gid_exists | ||
1728 | 407 | |||
1729 | 408 | |||
1730 | 409 | def add_group(group_name, system_group=False, gid=None): | ||
1731 | 410 | """Add a group to the system | ||
1732 | 411 | |||
1733 | 412 | Will log but otherwise succeed if the group already exists. | ||
1734 | 413 | |||
1735 | 414 | :param str group_name: group to create | ||
1736 | 415 | :param bool system_group: Create system group | ||
1737 | 416 | :param int gid: GID for user being created | ||
1738 | 417 | |||
1739 | 418 | :returns: The password database entry struct, as returned by `grp.getgrnam` | ||
1740 | 419 | """ | ||
1741 | 122 | try: | 420 | try: |
1742 | 123 | group_info = grp.getgrnam(group_name) | 421 | group_info = grp.getgrnam(group_name) |
1743 | 124 | log('group {0} already exists!'.format(group_name)) | 422 | log('group {0} already exists!'.format(group_name)) |
1744 | 423 | if gid: | ||
1745 | 424 | group_info = grp.getgrgid(gid) | ||
1746 | 425 | log('group with gid {0} already exists!'.format(gid)) | ||
1747 | 125 | except KeyError: | 426 | except KeyError: |
1748 | 126 | log('creating group {0}'.format(group_name)) | 427 | log('creating group {0}'.format(group_name)) |
1758 | 127 | cmd = ['addgroup'] | 428 | add_new_group(group_name, system_group, gid) |
1750 | 128 | if system_group: | ||
1751 | 129 | cmd.append('--system') | ||
1752 | 130 | else: | ||
1753 | 131 | cmd.extend([ | ||
1754 | 132 | '--group', | ||
1755 | 133 | ]) | ||
1756 | 134 | cmd.append(group_name) | ||
1757 | 135 | subprocess.check_call(cmd) | ||
1759 | 136 | group_info = grp.getgrnam(group_name) | 429 | group_info = grp.getgrnam(group_name) |
1760 | 137 | return group_info | 430 | return group_info |
1761 | 138 | 431 | ||
1762 | 139 | 432 | ||
1763 | 140 | def add_user_to_group(username, group): | 433 | def add_user_to_group(username, group): |
1764 | 141 | """Add a user to a group""" | 434 | """Add a user to a group""" |
1770 | 142 | cmd = [ | 435 | cmd = ['gpasswd', '-a', username, group] |
1766 | 143 | 'gpasswd', '-a', | ||
1767 | 144 | username, | ||
1768 | 145 | group | ||
1769 | 146 | ] | ||
1771 | 147 | log("Adding user {} to group {}".format(username, group)) | 436 | log("Adding user {} to group {}".format(username, group)) |
1772 | 148 | subprocess.check_call(cmd) | 437 | subprocess.check_call(cmd) |
1773 | 149 | 438 | ||
1774 | 150 | 439 | ||
1776 | 151 | def rsync(from_path, to_path, flags='-r', options=None): | 440 | def rsync(from_path, to_path, flags='-r', options=None, timeout=None): |
1777 | 152 | """Replicate the contents of a path""" | 441 | """Replicate the contents of a path""" |
1778 | 153 | options = options or ['--delete', '--executability'] | 442 | options = options or ['--delete', '--executability'] |
1779 | 154 | cmd = ['/usr/bin/rsync', flags] | 443 | cmd = ['/usr/bin/rsync', flags] |
1780 | 444 | if timeout: | ||
1781 | 445 | cmd = ['timeout', str(timeout)] + cmd | ||
1782 | 155 | cmd.extend(options) | 446 | cmd.extend(options) |
1783 | 156 | cmd.append(from_path) | 447 | cmd.append(from_path) |
1784 | 157 | cmd.append(to_path) | 448 | cmd.append(to_path) |
1785 | 158 | log(" ".join(cmd)) | 449 | log(" ".join(cmd)) |
1787 | 159 | return subprocess.check_output(cmd).decode('UTF-8').strip() | 450 | return subprocess.check_output(cmd, stderr=subprocess.STDOUT).decode('UTF-8').strip() |
1788 | 160 | 451 | ||
1789 | 161 | 452 | ||
1790 | 162 | def symlink(source, destination): | 453 | def symlink(source, destination): |
1791 | @@ -202,14 +493,12 @@ | |||
1792 | 202 | 493 | ||
1793 | 203 | 494 | ||
1794 | 204 | def fstab_remove(mp): | 495 | def fstab_remove(mp): |
1797 | 205 | """Remove the given mountpoint entry from /etc/fstab | 496 | """Remove the given mountpoint entry from /etc/fstab""" |
1796 | 206 | """ | ||
1798 | 207 | return Fstab.remove_by_mountpoint(mp) | 497 | return Fstab.remove_by_mountpoint(mp) |
1799 | 208 | 498 | ||
1800 | 209 | 499 | ||
1801 | 210 | def fstab_add(dev, mp, fs, options=None): | 500 | def fstab_add(dev, mp, fs, options=None): |
1804 | 211 | """Adds the given device entry to the /etc/fstab file | 501 | """Adds the given device entry to the /etc/fstab file""" |
1803 | 212 | """ | ||
1805 | 213 | return Fstab.add(dev, mp, fs, options=options) | 502 | return Fstab.add(dev, mp, fs, options=options) |
1806 | 214 | 503 | ||
1807 | 215 | 504 | ||
1808 | @@ -253,9 +542,19 @@ | |||
1809 | 253 | return system_mounts | 542 | return system_mounts |
1810 | 254 | 543 | ||
1811 | 255 | 544 | ||
1812 | 545 | def fstab_mount(mountpoint): | ||
1813 | 546 | """Mount filesystem using fstab""" | ||
1814 | 547 | cmd_args = ['mount', mountpoint] | ||
1815 | 548 | try: | ||
1816 | 549 | subprocess.check_output(cmd_args) | ||
1817 | 550 | except subprocess.CalledProcessError as e: | ||
1818 | 551 | log('Error unmounting {}\n{}'.format(mountpoint, e.output)) | ||
1819 | 552 | return False | ||
1820 | 553 | return True | ||
1821 | 554 | |||
1822 | 555 | |||
1823 | 256 | def file_hash(path, hash_type='md5'): | 556 | def file_hash(path, hash_type='md5'): |
1826 | 257 | """ | 557 | """Generate a hash checksum of the contents of 'path' or None if not found. |
1825 | 258 | Generate a hash checksum of the contents of 'path' or None if not found. | ||
1827 | 259 | 558 | ||
1828 | 260 | :param str hash_type: Any hash alrgorithm supported by :mod:`hashlib`, | 559 | :param str hash_type: Any hash alrgorithm supported by :mod:`hashlib`, |
1829 | 261 | such as md5, sha1, sha256, sha512, etc. | 560 | such as md5, sha1, sha256, sha512, etc. |
1830 | @@ -269,9 +568,22 @@ | |||
1831 | 269 | return None | 568 | return None |
1832 | 270 | 569 | ||
1833 | 271 | 570 | ||
1834 | 571 | def path_hash(path): | ||
1835 | 572 | """Generate a hash checksum of all files matching 'path'. Standard | ||
1836 | 573 | wildcards like '*' and '?' are supported, see documentation for the 'glob' | ||
1837 | 574 | module for more information. | ||
1838 | 575 | |||
1839 | 576 | :return: dict: A { filename: hash } dictionary for all matched files. | ||
1840 | 577 | Empty if none found. | ||
1841 | 578 | """ | ||
1842 | 579 | return { | ||
1843 | 580 | filename: file_hash(filename) | ||
1844 | 581 | for filename in glob.iglob(path) | ||
1845 | 582 | } | ||
1846 | 583 | |||
1847 | 584 | |||
1848 | 272 | def check_hash(path, checksum, hash_type='md5'): | 585 | def check_hash(path, checksum, hash_type='md5'): |
1851 | 273 | """ | 586 | """Validate a file using a cryptographic checksum. |
1850 | 274 | Validate a file using a cryptographic checksum. | ||
1852 | 275 | 587 | ||
1853 | 276 | :param str checksum: Value of the checksum used to validate the file. | 588 | :param str checksum: Value of the checksum used to validate the file. |
1854 | 277 | :param str hash_type: Hash algorithm used to generate `checksum`. | 589 | :param str hash_type: Hash algorithm used to generate `checksum`. |
1855 | @@ -286,54 +598,78 @@ | |||
1856 | 286 | 598 | ||
1857 | 287 | 599 | ||
1858 | 288 | class ChecksumError(ValueError): | 600 | class ChecksumError(ValueError): |
1859 | 601 | """A class derived from Value error to indicate the checksum failed.""" | ||
1860 | 289 | pass | 602 | pass |
1861 | 290 | 603 | ||
1862 | 291 | 604 | ||
1864 | 292 | def restart_on_change(restart_map, stopstart=False): | 605 | def restart_on_change(restart_map, stopstart=False, restart_functions=None): |
1865 | 293 | """Restart services based on configuration files changing | 606 | """Restart services based on configuration files changing |
1866 | 294 | 607 | ||
1867 | 295 | This function is used a decorator, for example:: | 608 | This function is used a decorator, for example:: |
1868 | 296 | 609 | ||
1869 | 297 | @restart_on_change({ | 610 | @restart_on_change({ |
1870 | 298 | '/etc/ceph/ceph.conf': [ 'cinder-api', 'cinder-volume' ] | 611 | '/etc/ceph/ceph.conf': [ 'cinder-api', 'cinder-volume' ] |
1871 | 612 | '/etc/apache/sites-enabled/*': [ 'apache2' ] | ||
1872 | 299 | }) | 613 | }) |
1874 | 300 | def ceph_client_changed(): | 614 | def config_changed(): |
1875 | 301 | pass # your code here | 615 | pass # your code here |
1876 | 302 | 616 | ||
1877 | 303 | In this example, the cinder-api and cinder-volume services | 617 | In this example, the cinder-api and cinder-volume services |
1878 | 304 | would be restarted if /etc/ceph/ceph.conf is changed by the | 618 | would be restarted if /etc/ceph/ceph.conf is changed by the |
1880 | 305 | ceph_client_changed function. | 619 | ceph_client_changed function. The apache2 service would be |
1881 | 620 | restarted if any file matching the pattern got changed, created | ||
1882 | 621 | or removed. Standard wildcards are supported, see documentation | ||
1883 | 622 | for the 'glob' module for more information. | ||
1884 | 623 | |||
1885 | 624 | @param restart_map: {path_file_name: [service_name, ...] | ||
1886 | 625 | @param stopstart: DEFAULT false; whether to stop, start OR restart | ||
1887 | 626 | @param restart_functions: nonstandard functions to use to restart services | ||
1888 | 627 | {svc: func, ...} | ||
1889 | 628 | @returns result from decorated function | ||
1890 | 306 | """ | 629 | """ |
1891 | 307 | def wrap(f): | 630 | def wrap(f): |
1892 | 631 | @functools.wraps(f) | ||
1893 | 308 | def wrapped_f(*args, **kwargs): | 632 | def wrapped_f(*args, **kwargs): |
1910 | 309 | checksums = {} | 633 | return restart_on_change_helper( |
1911 | 310 | for path in restart_map: | 634 | (lambda: f(*args, **kwargs)), restart_map, stopstart, |
1912 | 311 | checksums[path] = file_hash(path) | 635 | restart_functions) |
1897 | 312 | f(*args, **kwargs) | ||
1898 | 313 | restarts = [] | ||
1899 | 314 | for path in restart_map: | ||
1900 | 315 | if checksums[path] != file_hash(path): | ||
1901 | 316 | restarts += restart_map[path] | ||
1902 | 317 | services_list = list(OrderedDict.fromkeys(restarts)) | ||
1903 | 318 | if not stopstart: | ||
1904 | 319 | for service_name in services_list: | ||
1905 | 320 | service('restart', service_name) | ||
1906 | 321 | else: | ||
1907 | 322 | for action in ['stop', 'start']: | ||
1908 | 323 | for service_name in services_list: | ||
1909 | 324 | service(action, service_name) | ||
1913 | 325 | return wrapped_f | 636 | return wrapped_f |
1914 | 326 | return wrap | 637 | return wrap |
1915 | 327 | 638 | ||
1916 | 328 | 639 | ||
1925 | 329 | def lsb_release(): | 640 | def restart_on_change_helper(lambda_f, restart_map, stopstart=False, |
1926 | 330 | """Return /etc/lsb-release in a dict""" | 641 | restart_functions=None): |
1927 | 331 | d = {} | 642 | """Helper function to perform the restart_on_change function. |
1928 | 332 | with open('/etc/lsb-release', 'r') as lsb: | 643 | |
1929 | 333 | for l in lsb: | 644 | This is provided for decorators to restart services if files described |
1930 | 334 | k, v = l.split('=') | 645 | in the restart_map have changed after an invocation of lambda_f(). |
1931 | 335 | d[k.strip()] = v.strip() | 646 | |
1932 | 336 | return d | 647 | @param lambda_f: function to call. |
1933 | 648 | @param restart_map: {file: [service, ...]} | ||
1934 | 649 | @param stopstart: whether to stop, start or restart a service | ||
1935 | 650 | @param restart_functions: nonstandard functions to use to restart services | ||
1936 | 651 | {svc: func, ...} | ||
1937 | 652 | @returns result of lambda_f() | ||
1938 | 653 | """ | ||
1939 | 654 | if restart_functions is None: | ||
1940 | 655 | restart_functions = {} | ||
1941 | 656 | checksums = {path: path_hash(path) for path in restart_map} | ||
1942 | 657 | r = lambda_f() | ||
1943 | 658 | # create a list of lists of the services to restart | ||
1944 | 659 | restarts = [restart_map[path] | ||
1945 | 660 | for path in restart_map | ||
1946 | 661 | if path_hash(path) != checksums[path]] | ||
1947 | 662 | # create a flat list of ordered services without duplicates from lists | ||
1948 | 663 | services_list = list(OrderedDict.fromkeys(itertools.chain(*restarts))) | ||
1949 | 664 | if services_list: | ||
1950 | 665 | actions = ('stop', 'start') if stopstart else ('restart',) | ||
1951 | 666 | for service_name in services_list: | ||
1952 | 667 | if service_name in restart_functions: | ||
1953 | 668 | restart_functions[service_name](service_name) | ||
1954 | 669 | else: | ||
1955 | 670 | for action in actions: | ||
1956 | 671 | service(action, service_name) | ||
1957 | 672 | return r | ||
1958 | 337 | 673 | ||
1959 | 338 | 674 | ||
1960 | 339 | def pwgen(length=None): | 675 | def pwgen(length=None): |
1961 | @@ -352,36 +688,92 @@ | |||
1962 | 352 | return(''.join(random_chars)) | 688 | return(''.join(random_chars)) |
1963 | 353 | 689 | ||
1964 | 354 | 690 | ||
1967 | 355 | def list_nics(nic_type): | 691 | def is_phy_iface(interface): |
1968 | 356 | '''Return a list of nics of given type(s)''' | 692 | """Returns True if interface is not virtual, otherwise False.""" |
1969 | 693 | if interface: | ||
1970 | 694 | sys_net = '/sys/class/net' | ||
1971 | 695 | if os.path.isdir(sys_net): | ||
1972 | 696 | for iface in glob.glob(os.path.join(sys_net, '*')): | ||
1973 | 697 | if '/virtual/' in os.path.realpath(iface): | ||
1974 | 698 | continue | ||
1975 | 699 | |||
1976 | 700 | if interface == os.path.basename(iface): | ||
1977 | 701 | return True | ||
1978 | 702 | |||
1979 | 703 | return False | ||
1980 | 704 | |||
1981 | 705 | |||
1982 | 706 | def get_bond_master(interface): | ||
1983 | 707 | """Returns bond master if interface is bond slave otherwise None. | ||
1984 | 708 | |||
1985 | 709 | NOTE: the provided interface is expected to be physical | ||
1986 | 710 | """ | ||
1987 | 711 | if interface: | ||
1988 | 712 | iface_path = '/sys/class/net/%s' % (interface) | ||
1989 | 713 | if os.path.exists(iface_path): | ||
1990 | 714 | if '/virtual/' in os.path.realpath(iface_path): | ||
1991 | 715 | return None | ||
1992 | 716 | |||
1993 | 717 | master = os.path.join(iface_path, 'master') | ||
1994 | 718 | if os.path.exists(master): | ||
1995 | 719 | master = os.path.realpath(master) | ||
1996 | 720 | # make sure it is a bond master | ||
1997 | 721 | if os.path.exists(os.path.join(master, 'bonding')): | ||
1998 | 722 | return os.path.basename(master) | ||
1999 | 723 | |||
2000 | 724 | return None | ||
2001 | 725 | |||
2002 | 726 | |||
2003 | 727 | def list_nics(nic_type=None): | ||
2004 | 728 | """Return a list of nics of given type(s)""" | ||
2005 | 357 | if isinstance(nic_type, six.string_types): | 729 | if isinstance(nic_type, six.string_types): |
2006 | 358 | int_types = [nic_type] | 730 | int_types = [nic_type] |
2007 | 359 | else: | 731 | else: |
2008 | 360 | int_types = nic_type | 732 | int_types = nic_type |
2009 | 733 | |||
2010 | 361 | interfaces = [] | 734 | interfaces = [] |
2013 | 362 | for int_type in int_types: | 735 | if nic_type: |
2014 | 363 | cmd = ['ip', 'addr', 'show', 'label', int_type + '*'] | 736 | for int_type in int_types: |
2015 | 737 | cmd = ['ip', 'addr', 'show', 'label', int_type + '*'] | ||
2016 | 738 | ip_output = subprocess.check_output(cmd).decode('UTF-8') | ||
2017 | 739 | ip_output = ip_output.split('\n') | ||
2018 | 740 | ip_output = (line for line in ip_output if line) | ||
2019 | 741 | for line in ip_output: | ||
2020 | 742 | if line.split()[1].startswith(int_type): | ||
2021 | 743 | matched = re.search('.*: (' + int_type + | ||
2022 | 744 | r'[0-9]+\.[0-9]+)@.*', line) | ||
2023 | 745 | if matched: | ||
2024 | 746 | iface = matched.groups()[0] | ||
2025 | 747 | else: | ||
2026 | 748 | iface = line.split()[1].replace(":", "") | ||
2027 | 749 | |||
2028 | 750 | if iface not in interfaces: | ||
2029 | 751 | interfaces.append(iface) | ||
2030 | 752 | else: | ||
2031 | 753 | cmd = ['ip', 'a'] | ||
2032 | 364 | ip_output = subprocess.check_output(cmd).decode('UTF-8').split('\n') | 754 | ip_output = subprocess.check_output(cmd).decode('UTF-8').split('\n') |
2034 | 365 | ip_output = (line for line in ip_output if line) | 755 | ip_output = (line.strip() for line in ip_output if line) |
2035 | 756 | |||
2036 | 757 | key = re.compile('^[0-9]+:\s+(.+):') | ||
2037 | 366 | for line in ip_output: | 758 | for line in ip_output: |
2045 | 367 | if line.split()[1].startswith(int_type): | 759 | matched = re.search(key, line) |
2046 | 368 | matched = re.search('.*: (' + int_type + r'[0-9]+\.[0-9]+)@.*', line) | 760 | if matched: |
2047 | 369 | if matched: | 761 | iface = matched.group(1) |
2048 | 370 | interface = matched.groups()[0] | 762 | iface = iface.partition("@")[0] |
2049 | 371 | else: | 763 | if iface not in interfaces: |
2050 | 372 | interface = line.split()[1].replace(":", "") | 764 | interfaces.append(iface) |
2044 | 373 | interfaces.append(interface) | ||
2051 | 374 | 765 | ||
2052 | 375 | return interfaces | 766 | return interfaces |
2053 | 376 | 767 | ||
2054 | 377 | 768 | ||
2055 | 378 | def set_nic_mtu(nic, mtu): | 769 | def set_nic_mtu(nic, mtu): |
2057 | 379 | '''Set MTU on a network interface''' | 770 | """Set the Maximum Transmission Unit (MTU) on a network interface.""" |
2058 | 380 | cmd = ['ip', 'link', 'set', nic, 'mtu', mtu] | 771 | cmd = ['ip', 'link', 'set', nic, 'mtu', mtu] |
2059 | 381 | subprocess.check_call(cmd) | 772 | subprocess.check_call(cmd) |
2060 | 382 | 773 | ||
2061 | 383 | 774 | ||
2062 | 384 | def get_nic_mtu(nic): | 775 | def get_nic_mtu(nic): |
2063 | 776 | """Return the Maximum Transmission Unit (MTU) for a network interface.""" | ||
2064 | 385 | cmd = ['ip', 'addr', 'show', nic] | 777 | cmd = ['ip', 'addr', 'show', nic] |
2065 | 386 | ip_output = subprocess.check_output(cmd).decode('UTF-8').split('\n') | 778 | ip_output = subprocess.check_output(cmd).decode('UTF-8').split('\n') |
2066 | 387 | mtu = "" | 779 | mtu = "" |
2067 | @@ -393,6 +785,7 @@ | |||
2068 | 393 | 785 | ||
2069 | 394 | 786 | ||
2070 | 395 | def get_nic_hwaddr(nic): | 787 | def get_nic_hwaddr(nic): |
2071 | 788 | """Return the Media Access Control (MAC) for a network interface.""" | ||
2072 | 396 | cmd = ['ip', '-o', '-0', 'addr', 'show', nic] | 789 | cmd = ['ip', '-o', '-0', 'addr', 'show', nic] |
2073 | 397 | ip_output = subprocess.check_output(cmd).decode('UTF-8') | 790 | ip_output = subprocess.check_output(cmd).decode('UTF-8') |
2074 | 398 | hwaddr = "" | 791 | hwaddr = "" |
2075 | @@ -402,35 +795,31 @@ | |||
2076 | 402 | return hwaddr | 795 | return hwaddr |
2077 | 403 | 796 | ||
2078 | 404 | 797 | ||
2079 | 405 | def cmp_pkgrevno(package, revno, pkgcache=None): | ||
2080 | 406 | '''Compare supplied revno with the revno of the installed package | ||
2081 | 407 | |||
2082 | 408 | * 1 => Installed revno is greater than supplied arg | ||
2083 | 409 | * 0 => Installed revno is the same as supplied arg | ||
2084 | 410 | * -1 => Installed revno is less than supplied arg | ||
2085 | 411 | |||
2086 | 412 | This function imports apt_cache function from charmhelpers.fetch if | ||
2087 | 413 | the pkgcache argument is None. Be sure to add charmhelpers.fetch if | ||
2088 | 414 | you call this function, or pass an apt_pkg.Cache() instance. | ||
2089 | 415 | ''' | ||
2090 | 416 | import apt_pkg | ||
2091 | 417 | if not pkgcache: | ||
2092 | 418 | from charmhelpers.fetch import apt_cache | ||
2093 | 419 | pkgcache = apt_cache() | ||
2094 | 420 | pkg = pkgcache[package] | ||
2095 | 421 | return apt_pkg.version_compare(pkg.current_ver.ver_str, revno) | ||
2096 | 422 | |||
2097 | 423 | |||
2098 | 424 | @contextmanager | 798 | @contextmanager |
2100 | 425 | def chdir(d): | 799 | def chdir(directory): |
2101 | 800 | """Change the current working directory to a different directory for a code | ||
2102 | 801 | block and return the previous directory after the block exits. Useful to | ||
2103 | 802 | run commands from a specificed directory. | ||
2104 | 803 | |||
2105 | 804 | :param str directory: The directory path to change to for this context. | ||
2106 | 805 | """ | ||
2107 | 426 | cur = os.getcwd() | 806 | cur = os.getcwd() |
2108 | 427 | try: | 807 | try: |
2110 | 428 | yield os.chdir(d) | 808 | yield os.chdir(directory) |
2111 | 429 | finally: | 809 | finally: |
2112 | 430 | os.chdir(cur) | 810 | os.chdir(cur) |
2113 | 431 | 811 | ||
2114 | 432 | 812 | ||
2116 | 433 | def chownr(path, owner, group, follow_links=True): | 813 | def chownr(path, owner, group, follow_links=True, chowntopdir=False): |
2117 | 814 | """Recursively change user and group ownership of files and directories | ||
2118 | 815 | in given path. Doesn't chown path itself by default, only its children. | ||
2119 | 816 | |||
2120 | 817 | :param str path: The string path to start changing ownership. | ||
2121 | 818 | :param str owner: The owner string to use when looking up the uid. | ||
2122 | 819 | :param str group: The group string to use when looking up the gid. | ||
2123 | 820 | :param bool follow_links: Also follow and chown links if True | ||
2124 | 821 | :param bool chowntopdir: Also chown path itself if True | ||
2125 | 822 | """ | ||
2126 | 434 | uid = pwd.getpwnam(owner).pw_uid | 823 | uid = pwd.getpwnam(owner).pw_uid |
2127 | 435 | gid = grp.getgrnam(group).gr_gid | 824 | gid = grp.getgrnam(group).gr_gid |
2128 | 436 | if follow_links: | 825 | if follow_links: |
2129 | @@ -438,7 +827,11 @@ | |||
2130 | 438 | else: | 827 | else: |
2131 | 439 | chown = os.lchown | 828 | chown = os.lchown |
2132 | 440 | 829 | ||
2134 | 441 | for root, dirs, files in os.walk(path): | 830 | if chowntopdir: |
2135 | 831 | broken_symlink = os.path.lexists(path) and not os.path.exists(path) | ||
2136 | 832 | if not broken_symlink: | ||
2137 | 833 | chown(path, uid, gid) | ||
2138 | 834 | for root, dirs, files in os.walk(path, followlinks=follow_links): | ||
2139 | 442 | for name in dirs + files: | 835 | for name in dirs + files: |
2140 | 443 | full = os.path.join(root, name) | 836 | full = os.path.join(root, name) |
2141 | 444 | broken_symlink = os.path.lexists(full) and not os.path.exists(full) | 837 | broken_symlink = os.path.lexists(full) and not os.path.exists(full) |
2142 | @@ -447,4 +840,81 @@ | |||
2143 | 447 | 840 | ||
2144 | 448 | 841 | ||
2145 | 449 | def lchownr(path, owner, group): | 842 | def lchownr(path, owner, group): |
2146 | 843 | """Recursively change user and group ownership of files and directories | ||
2147 | 844 | in a given path, not following symbolic links. See the documentation for | ||
2148 | 845 | 'os.lchown' for more information. | ||
2149 | 846 | |||
2150 | 847 | :param str path: The string path to start changing ownership. | ||
2151 | 848 | :param str owner: The owner string to use when looking up the uid. | ||
2152 | 849 | :param str group: The group string to use when looking up the gid. | ||
2153 | 850 | """ | ||
2154 | 450 | chownr(path, owner, group, follow_links=False) | 851 | chownr(path, owner, group, follow_links=False) |
2155 | 852 | |||
2156 | 853 | |||
2157 | 854 | def owner(path): | ||
2158 | 855 | """Returns a tuple containing the username & groupname owning the path. | ||
2159 | 856 | |||
2160 | 857 | :param str path: the string path to retrieve the ownership | ||
2161 | 858 | :return tuple(str, str): A (username, groupname) tuple containing the | ||
2162 | 859 | name of the user and group owning the path. | ||
2163 | 860 | :raises OSError: if the specified path does not exist | ||
2164 | 861 | """ | ||
2165 | 862 | stat = os.stat(path) | ||
2166 | 863 | username = pwd.getpwuid(stat.st_uid)[0] | ||
2167 | 864 | groupname = grp.getgrgid(stat.st_gid)[0] | ||
2168 | 865 | return username, groupname | ||
2169 | 866 | |||
2170 | 867 | |||
2171 | 868 | def get_total_ram(): | ||
2172 | 869 | """The total amount of system RAM in bytes. | ||
2173 | 870 | |||
2174 | 871 | This is what is reported by the OS, and may be overcommitted when | ||
2175 | 872 | there are multiple containers hosted on the same machine. | ||
2176 | 873 | """ | ||
2177 | 874 | with open('/proc/meminfo', 'r') as f: | ||
2178 | 875 | for line in f.readlines(): | ||
2179 | 876 | if line: | ||
2180 | 877 | key, value, unit = line.split() | ||
2181 | 878 | if key == 'MemTotal:': | ||
2182 | 879 | assert unit == 'kB', 'Unknown unit' | ||
2183 | 880 | return int(value) * 1024 # Classic, not KiB. | ||
2184 | 881 | raise NotImplementedError() | ||
2185 | 882 | |||
2186 | 883 | |||
2187 | 884 | UPSTART_CONTAINER_TYPE = '/run/container_type' | ||
2188 | 885 | |||
2189 | 886 | |||
2190 | 887 | def is_container(): | ||
2191 | 888 | """Determine whether unit is running in a container | ||
2192 | 889 | |||
2193 | 890 | @return: boolean indicating if unit is in a container | ||
2194 | 891 | """ | ||
2195 | 892 | if init_is_systemd(): | ||
2196 | 893 | # Detect using systemd-detect-virt | ||
2197 | 894 | return subprocess.call(['systemd-detect-virt', | ||
2198 | 895 | '--container']) == 0 | ||
2199 | 896 | else: | ||
2200 | 897 | # Detect using upstart container file marker | ||
2201 | 898 | return os.path.exists(UPSTART_CONTAINER_TYPE) | ||
2202 | 899 | |||
2203 | 900 | |||
2204 | 901 | def add_to_updatedb_prunepath(path, updatedb_path=UPDATEDB_PATH): | ||
2205 | 902 | with open(updatedb_path, 'r+') as f_id: | ||
2206 | 903 | updatedb_text = f_id.read() | ||
2207 | 904 | output = updatedb(updatedb_text, path) | ||
2208 | 905 | f_id.seek(0) | ||
2209 | 906 | f_id.write(output) | ||
2210 | 907 | f_id.truncate() | ||
2211 | 908 | |||
2212 | 909 | |||
2213 | 910 | def updatedb(updatedb_text, new_path): | ||
2214 | 911 | lines = [line for line in updatedb_text.split("\n")] | ||
2215 | 912 | for i, line in enumerate(lines): | ||
2216 | 913 | if line.startswith("PRUNEPATHS="): | ||
2217 | 914 | paths_line = line.split("=")[1].replace('"', '') | ||
2218 | 915 | paths = paths_line.split(" ") | ||
2219 | 916 | if new_path not in paths: | ||
2220 | 917 | paths.append(new_path) | ||
2221 | 918 | lines[i] = 'PRUNEPATHS="{}"'.format(' '.join(paths)) | ||
2222 | 919 | output = "\n".join(lines) | ||
2223 | 920 | return output | ||
2224 | 451 | 921 | ||
2225 | === added directory 'hooks/charmhelpers/core/host_factory' | |||
2226 | === added file 'hooks/charmhelpers/core/host_factory/__init__.py' | |||
2227 | === added file 'hooks/charmhelpers/core/host_factory/centos.py' | |||
2228 | --- hooks/charmhelpers/core/host_factory/centos.py 1970-01-01 00:00:00 +0000 | |||
2229 | +++ hooks/charmhelpers/core/host_factory/centos.py 2018-06-12 20:07:13 +0000 | |||
2230 | @@ -0,0 +1,56 @@ | |||
2231 | 1 | import subprocess | ||
2232 | 2 | import yum | ||
2233 | 3 | import os | ||
2234 | 4 | |||
2235 | 5 | |||
2236 | 6 | def service_available(service_name): | ||
2237 | 7 | # """Determine whether a system service is available.""" | ||
2238 | 8 | if os.path.isdir('/run/systemd/system'): | ||
2239 | 9 | cmd = ['systemctl', 'is-enabled', service_name] | ||
2240 | 10 | else: | ||
2241 | 11 | cmd = ['service', service_name, 'is-enabled'] | ||
2242 | 12 | return subprocess.call(cmd) == 0 | ||
2243 | 13 | |||
2244 | 14 | |||
2245 | 15 | def add_new_group(group_name, system_group=False, gid=None): | ||
2246 | 16 | cmd = ['groupadd'] | ||
2247 | 17 | if gid: | ||
2248 | 18 | cmd.extend(['--gid', str(gid)]) | ||
2249 | 19 | if system_group: | ||
2250 | 20 | cmd.append('-r') | ||
2251 | 21 | cmd.append(group_name) | ||
2252 | 22 | subprocess.check_call(cmd) | ||
2253 | 23 | |||
2254 | 24 | |||
2255 | 25 | def lsb_release(): | ||
2256 | 26 | """Return /etc/os-release in a dict.""" | ||
2257 | 27 | d = {} | ||
2258 | 28 | with open('/etc/os-release', 'r') as lsb: | ||
2259 | 29 | for l in lsb: | ||
2260 | 30 | s = l.split('=') | ||
2261 | 31 | if len(s) != 2: | ||
2262 | 32 | continue | ||
2263 | 33 | d[s[0].strip()] = s[1].strip() | ||
2264 | 34 | return d | ||
2265 | 35 | |||
2266 | 36 | |||
2267 | 37 | def cmp_pkgrevno(package, revno, pkgcache=None): | ||
2268 | 38 | """Compare supplied revno with the revno of the installed package. | ||
2269 | 39 | |||
2270 | 40 | * 1 => Installed revno is greater than supplied arg | ||
2271 | 41 | * 0 => Installed revno is the same as supplied arg | ||
2272 | 42 | * -1 => Installed revno is less than supplied arg | ||
2273 | 43 | |||
2274 | 44 | This function imports YumBase function if the pkgcache argument | ||
2275 | 45 | is None. | ||
2276 | 46 | """ | ||
2277 | 47 | if not pkgcache: | ||
2278 | 48 | y = yum.YumBase() | ||
2279 | 49 | packages = y.doPackageLists() | ||
2280 | 50 | pkgcache = {i.Name: i.version for i in packages['installed']} | ||
2281 | 51 | pkg = pkgcache[package] | ||
2282 | 52 | if pkg > revno: | ||
2283 | 53 | return 1 | ||
2284 | 54 | if pkg < revno: | ||
2285 | 55 | return -1 | ||
2286 | 56 | return 0 | ||
2287 | 0 | 57 | ||
2288 | === added file 'hooks/charmhelpers/core/host_factory/ubuntu.py' | |||
2289 | --- hooks/charmhelpers/core/host_factory/ubuntu.py 1970-01-01 00:00:00 +0000 | |||
2290 | +++ hooks/charmhelpers/core/host_factory/ubuntu.py 2018-06-12 20:07:13 +0000 | |||
2291 | @@ -0,0 +1,56 @@ | |||
2292 | 1 | import subprocess | ||
2293 | 2 | |||
2294 | 3 | |||
2295 | 4 | def service_available(service_name): | ||
2296 | 5 | """Determine whether a system service is available""" | ||
2297 | 6 | try: | ||
2298 | 7 | subprocess.check_output( | ||
2299 | 8 | ['service', service_name, 'status'], | ||
2300 | 9 | stderr=subprocess.STDOUT).decode('UTF-8') | ||
2301 | 10 | except subprocess.CalledProcessError as e: | ||
2302 | 11 | return b'unrecognized service' not in e.output | ||
2303 | 12 | else: | ||
2304 | 13 | return True | ||
2305 | 14 | |||
2306 | 15 | |||
2307 | 16 | def add_new_group(group_name, system_group=False, gid=None): | ||
2308 | 17 | cmd = ['addgroup'] | ||
2309 | 18 | if gid: | ||
2310 | 19 | cmd.extend(['--gid', str(gid)]) | ||
2311 | 20 | if system_group: | ||
2312 | 21 | cmd.append('--system') | ||
2313 | 22 | else: | ||
2314 | 23 | cmd.extend([ | ||
2315 | 24 | '--group', | ||
2316 | 25 | ]) | ||
2317 | 26 | cmd.append(group_name) | ||
2318 | 27 | subprocess.check_call(cmd) | ||
2319 | 28 | |||
2320 | 29 | |||
2321 | 30 | def lsb_release(): | ||
2322 | 31 | """Return /etc/lsb-release in a dict""" | ||
2323 | 32 | d = {} | ||
2324 | 33 | with open('/etc/lsb-release', 'r') as lsb: | ||
2325 | 34 | for l in lsb: | ||
2326 | 35 | k, v = l.split('=') | ||
2327 | 36 | d[k.strip()] = v.strip() | ||
2328 | 37 | return d | ||
2329 | 38 | |||
2330 | 39 | |||
2331 | 40 | def cmp_pkgrevno(package, revno, pkgcache=None): | ||
2332 | 41 | """Compare supplied revno with the revno of the installed package. | ||
2333 | 42 | |||
2334 | 43 | * 1 => Installed revno is greater than supplied arg | ||
2335 | 44 | * 0 => Installed revno is the same as supplied arg | ||
2336 | 45 | * -1 => Installed revno is less than supplied arg | ||
2337 | 46 | |||
2338 | 47 | This function imports apt_cache function from charmhelpers.fetch if | ||
2339 | 48 | the pkgcache argument is None. Be sure to add charmhelpers.fetch if | ||
2340 | 49 | you call this function, or pass an apt_pkg.Cache() instance. | ||
2341 | 50 | """ | ||
2342 | 51 | import apt_pkg | ||
2343 | 52 | if not pkgcache: | ||
2344 | 53 | from charmhelpers.fetch import apt_cache | ||
2345 | 54 | pkgcache = apt_cache() | ||
2346 | 55 | pkg = pkgcache[package] | ||
2347 | 56 | return apt_pkg.version_compare(pkg.current_ver.ver_str, revno) | ||
2348 | 0 | 57 | ||
2349 | === added file 'hooks/charmhelpers/core/hugepage.py' | |||
2350 | --- hooks/charmhelpers/core/hugepage.py 1970-01-01 00:00:00 +0000 | |||
2351 | +++ hooks/charmhelpers/core/hugepage.py 2018-06-12 20:07:13 +0000 | |||
2352 | @@ -0,0 +1,69 @@ | |||
2353 | 1 | # -*- coding: utf-8 -*- | ||
2354 | 2 | |||
2355 | 3 | # Copyright 2014-2015 Canonical Limited. | ||
2356 | 4 | # | ||
2357 | 5 | # Licensed under the Apache License, Version 2.0 (the "License"); | ||
2358 | 6 | # you may not use this file except in compliance with the License. | ||
2359 | 7 | # You may obtain a copy of the License at | ||
2360 | 8 | # | ||
2361 | 9 | # http://www.apache.org/licenses/LICENSE-2.0 | ||
2362 | 10 | # | ||
2363 | 11 | # Unless required by applicable law or agreed to in writing, software | ||
2364 | 12 | # distributed under the License is distributed on an "AS IS" BASIS, | ||
2365 | 13 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
2366 | 14 | # See the License for the specific language governing permissions and | ||
2367 | 15 | # limitations under the License. | ||
2368 | 16 | |||
2369 | 17 | import yaml | ||
2370 | 18 | from charmhelpers.core import fstab | ||
2371 | 19 | from charmhelpers.core import sysctl | ||
2372 | 20 | from charmhelpers.core.host import ( | ||
2373 | 21 | add_group, | ||
2374 | 22 | add_user_to_group, | ||
2375 | 23 | fstab_mount, | ||
2376 | 24 | mkdir, | ||
2377 | 25 | ) | ||
2378 | 26 | from charmhelpers.core.strutils import bytes_from_string | ||
2379 | 27 | from subprocess import check_output | ||
2380 | 28 | |||
2381 | 29 | |||
2382 | 30 | def hugepage_support(user, group='hugetlb', nr_hugepages=256, | ||
2383 | 31 | max_map_count=65536, mnt_point='/run/hugepages/kvm', | ||
2384 | 32 | pagesize='2MB', mount=True, set_shmmax=False): | ||
2385 | 33 | """Enable hugepages on system. | ||
2386 | 34 | |||
2387 | 35 | Args: | ||
2388 | 36 | user (str) -- Username to allow access to hugepages to | ||
2389 | 37 | group (str) -- Group name to own hugepages | ||
2390 | 38 | nr_hugepages (int) -- Number of pages to reserve | ||
2391 | 39 | max_map_count (int) -- Number of Virtual Memory Areas a process can own | ||
2392 | 40 | mnt_point (str) -- Directory to mount hugepages on | ||
2393 | 41 | pagesize (str) -- Size of hugepages | ||
2394 | 42 | mount (bool) -- Whether to Mount hugepages | ||
2395 | 43 | """ | ||
2396 | 44 | group_info = add_group(group) | ||
2397 | 45 | gid = group_info.gr_gid | ||
2398 | 46 | add_user_to_group(user, group) | ||
2399 | 47 | if max_map_count < 2 * nr_hugepages: | ||
2400 | 48 | max_map_count = 2 * nr_hugepages | ||
2401 | 49 | sysctl_settings = { | ||
2402 | 50 | 'vm.nr_hugepages': nr_hugepages, | ||
2403 | 51 | 'vm.max_map_count': max_map_count, | ||
2404 | 52 | 'vm.hugetlb_shm_group': gid, | ||
2405 | 53 | } | ||
2406 | 54 | if set_shmmax: | ||
2407 | 55 | shmmax_current = int(check_output(['sysctl', '-n', 'kernel.shmmax'])) | ||
2408 | 56 | shmmax_minsize = bytes_from_string(pagesize) * nr_hugepages | ||
2409 | 57 | if shmmax_minsize > shmmax_current: | ||
2410 | 58 | sysctl_settings['kernel.shmmax'] = shmmax_minsize | ||
2411 | 59 | sysctl.create(yaml.dump(sysctl_settings), '/etc/sysctl.d/10-hugepage.conf') | ||
2412 | 60 | mkdir(mnt_point, owner='root', group='root', perms=0o755, force=False) | ||
2413 | 61 | lfstab = fstab.Fstab() | ||
2414 | 62 | fstab_entry = lfstab.get_entry_by_attr('mountpoint', mnt_point) | ||
2415 | 63 | if fstab_entry: | ||
2416 | 64 | lfstab.remove_entry(fstab_entry) | ||
2417 | 65 | entry = lfstab.Entry('nodev', mnt_point, 'hugetlbfs', | ||
2418 | 66 | 'mode=1770,gid={},pagesize={}'.format(gid, pagesize), 0, 0) | ||
2419 | 67 | lfstab.add_entry(entry) | ||
2420 | 68 | if mount: | ||
2421 | 69 | fstab_mount(mnt_point) | ||
2422 | 0 | 70 | ||
2423 | === added file 'hooks/charmhelpers/core/kernel.py' | |||
2424 | --- hooks/charmhelpers/core/kernel.py 1970-01-01 00:00:00 +0000 | |||
2425 | +++ hooks/charmhelpers/core/kernel.py 2018-06-12 20:07:13 +0000 | |||
2426 | @@ -0,0 +1,72 @@ | |||
2427 | 1 | #!/usr/bin/env python | ||
2428 | 2 | # -*- coding: utf-8 -*- | ||
2429 | 3 | |||
2430 | 4 | # Copyright 2014-2015 Canonical Limited. | ||
2431 | 5 | # | ||
2432 | 6 | # Licensed under the Apache License, Version 2.0 (the "License"); | ||
2433 | 7 | # you may not use this file except in compliance with the License. | ||
2434 | 8 | # You may obtain a copy of the License at | ||
2435 | 9 | # | ||
2436 | 10 | # http://www.apache.org/licenses/LICENSE-2.0 | ||
2437 | 11 | # | ||
2438 | 12 | # Unless required by applicable law or agreed to in writing, software | ||
2439 | 13 | # distributed under the License is distributed on an "AS IS" BASIS, | ||
2440 | 14 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
2441 | 15 | # See the License for the specific language governing permissions and | ||
2442 | 16 | # limitations under the License. | ||
2443 | 17 | |||
2444 | 18 | import re | ||
2445 | 19 | import subprocess | ||
2446 | 20 | |||
2447 | 21 | from charmhelpers.osplatform import get_platform | ||
2448 | 22 | from charmhelpers.core.hookenv import ( | ||
2449 | 23 | log, | ||
2450 | 24 | INFO | ||
2451 | 25 | ) | ||
2452 | 26 | |||
2453 | 27 | __platform__ = get_platform() | ||
2454 | 28 | if __platform__ == "ubuntu": | ||
2455 | 29 | from charmhelpers.core.kernel_factory.ubuntu import ( | ||
2456 | 30 | persistent_modprobe, | ||
2457 | 31 | update_initramfs, | ||
2458 | 32 | ) # flake8: noqa -- ignore F401 for this import | ||
2459 | 33 | elif __platform__ == "centos": | ||
2460 | 34 | from charmhelpers.core.kernel_factory.centos import ( | ||
2461 | 35 | persistent_modprobe, | ||
2462 | 36 | update_initramfs, | ||
2463 | 37 | ) # flake8: noqa -- ignore F401 for this import | ||
2464 | 38 | |||
2465 | 39 | __author__ = "Jorge Niedbalski <jorge.niedbalski@canonical.com>" | ||
2466 | 40 | |||
2467 | 41 | |||
2468 | 42 | def modprobe(module, persist=True): | ||
2469 | 43 | """Load a kernel module and configure for auto-load on reboot.""" | ||
2470 | 44 | cmd = ['modprobe', module] | ||
2471 | 45 | |||
2472 | 46 | log('Loading kernel module %s' % module, level=INFO) | ||
2473 | 47 | |||
2474 | 48 | subprocess.check_call(cmd) | ||
2475 | 49 | if persist: | ||
2476 | 50 | persistent_modprobe(module) | ||
2477 | 51 | |||
2478 | 52 | |||
2479 | 53 | def rmmod(module, force=False): | ||
2480 | 54 | """Remove a module from the linux kernel""" | ||
2481 | 55 | cmd = ['rmmod'] | ||
2482 | 56 | if force: | ||
2483 | 57 | cmd.append('-f') | ||
2484 | 58 | cmd.append(module) | ||
2485 | 59 | log('Removing kernel module %s' % module, level=INFO) | ||
2486 | 60 | return subprocess.check_call(cmd) | ||
2487 | 61 | |||
2488 | 62 | |||
2489 | 63 | def lsmod(): | ||
2490 | 64 | """Shows what kernel modules are currently loaded""" | ||
2491 | 65 | return subprocess.check_output(['lsmod'], | ||
2492 | 66 | universal_newlines=True) | ||
2493 | 67 | |||
2494 | 68 | |||
2495 | 69 | def is_module_loaded(module): | ||
2496 | 70 | """Checks if a kernel module is already loaded""" | ||
2497 | 71 | matches = re.findall('^%s[ ]+' % module, lsmod(), re.M) | ||
2498 | 72 | return len(matches) > 0 | ||
2499 | 0 | 73 | ||
2500 | === added directory 'hooks/charmhelpers/core/kernel_factory' | |||
2501 | === added file 'hooks/charmhelpers/core/kernel_factory/__init__.py' | |||
2502 | === added file 'hooks/charmhelpers/core/kernel_factory/centos.py' | |||
2503 | --- hooks/charmhelpers/core/kernel_factory/centos.py 1970-01-01 00:00:00 +0000 | |||
2504 | +++ hooks/charmhelpers/core/kernel_factory/centos.py 2018-06-12 20:07:13 +0000 | |||
2505 | @@ -0,0 +1,17 @@ | |||
2506 | 1 | import subprocess | ||
2507 | 2 | import os | ||
2508 | 3 | |||
2509 | 4 | |||
2510 | 5 | def persistent_modprobe(module): | ||
2511 | 6 | """Load a kernel module and configure for auto-load on reboot.""" | ||
2512 | 7 | if not os.path.exists('/etc/rc.modules'): | ||
2513 | 8 | open('/etc/rc.modules', 'a') | ||
2514 | 9 | os.chmod('/etc/rc.modules', 111) | ||
2515 | 10 | with open('/etc/rc.modules', 'r+') as modules: | ||
2516 | 11 | if module not in modules.read(): | ||
2517 | 12 | modules.write('modprobe %s\n' % module) | ||
2518 | 13 | |||
2519 | 14 | |||
2520 | 15 | def update_initramfs(version='all'): | ||
2521 | 16 | """Updates an initramfs image.""" | ||
2522 | 17 | return subprocess.check_call(["dracut", "-f", version]) | ||
2523 | 0 | 18 | ||
2524 | === added file 'hooks/charmhelpers/core/kernel_factory/ubuntu.py' | |||
2525 | --- hooks/charmhelpers/core/kernel_factory/ubuntu.py 1970-01-01 00:00:00 +0000 | |||
2526 | +++ hooks/charmhelpers/core/kernel_factory/ubuntu.py 2018-06-12 20:07:13 +0000 | |||
2527 | @@ -0,0 +1,13 @@ | |||
2528 | 1 | import subprocess | ||
2529 | 2 | |||
2530 | 3 | |||
2531 | 4 | def persistent_modprobe(module): | ||
2532 | 5 | """Load a kernel module and configure for auto-load on reboot.""" | ||
2533 | 6 | with open('/etc/modules', 'r+') as modules: | ||
2534 | 7 | if module not in modules.read(): | ||
2535 | 8 | modules.write(module + "\n") | ||
2536 | 9 | |||
2537 | 10 | |||
2538 | 11 | def update_initramfs(version='all'): | ||
2539 | 12 | """Updates an initramfs image.""" | ||
2540 | 13 | return subprocess.check_call(["update-initramfs", "-k", version, "-u"]) | ||
2541 | 0 | 14 | ||
2542 | === modified file 'hooks/charmhelpers/core/services/__init__.py' | |||
2543 | --- hooks/charmhelpers/core/services/__init__.py 2015-02-09 12:58:07 +0000 | |||
2544 | +++ hooks/charmhelpers/core/services/__init__.py 2018-06-12 20:07:13 +0000 | |||
2545 | @@ -1,18 +1,16 @@ | |||
2546 | 1 | # Copyright 2014-2015 Canonical Limited. | 1 | # Copyright 2014-2015 Canonical Limited. |
2547 | 2 | # | 2 | # |
2561 | 3 | # This file is part of charm-helpers. | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); |
2562 | 4 | # | 4 | # you may not use this file except in compliance with the License. |
2563 | 5 | # charm-helpers is free software: you can redistribute it and/or modify | 5 | # You may obtain a copy of the License at |
2564 | 6 | # it under the terms of the GNU Lesser General Public License version 3 as | 6 | # |
2565 | 7 | # published by the Free Software Foundation. | 7 | # http://www.apache.org/licenses/LICENSE-2.0 |
2566 | 8 | # | 8 | # |
2567 | 9 | # charm-helpers is distributed in the hope that it will be useful, | 9 | # Unless required by applicable law or agreed to in writing, software |
2568 | 10 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 10 | # distributed under the License is distributed on an "AS IS" BASIS, |
2569 | 11 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
2570 | 12 | # GNU Lesser General Public License for more details. | 12 | # See the License for the specific language governing permissions and |
2571 | 13 | # | 13 | # limitations under the License. |
2559 | 14 | # You should have received a copy of the GNU Lesser General Public License | ||
2560 | 15 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
2572 | 16 | 14 | ||
2573 | 17 | from .base import * # NOQA | 15 | from .base import * # NOQA |
2574 | 18 | from .helpers import * # NOQA | 16 | from .helpers import * # NOQA |
2575 | 19 | 17 | ||
2576 | === modified file 'hooks/charmhelpers/core/services/base.py' | |||
2577 | --- hooks/charmhelpers/core/services/base.py 2015-05-14 10:48:09 +0000 | |||
2578 | +++ hooks/charmhelpers/core/services/base.py 2018-06-12 20:07:13 +0000 | |||
2579 | @@ -1,22 +1,20 @@ | |||
2580 | 1 | # Copyright 2014-2015 Canonical Limited. | 1 | # Copyright 2014-2015 Canonical Limited. |
2581 | 2 | # | 2 | # |
2595 | 3 | # This file is part of charm-helpers. | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); |
2596 | 4 | # | 4 | # you may not use this file except in compliance with the License. |
2597 | 5 | # charm-helpers is free software: you can redistribute it and/or modify | 5 | # You may obtain a copy of the License at |
2598 | 6 | # it under the terms of the GNU Lesser General Public License version 3 as | 6 | # |
2599 | 7 | # published by the Free Software Foundation. | 7 | # http://www.apache.org/licenses/LICENSE-2.0 |
2600 | 8 | # | 8 | # |
2601 | 9 | # charm-helpers is distributed in the hope that it will be useful, | 9 | # Unless required by applicable law or agreed to in writing, software |
2602 | 10 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 10 | # distributed under the License is distributed on an "AS IS" BASIS, |
2603 | 11 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
2604 | 12 | # GNU Lesser General Public License for more details. | 12 | # See the License for the specific language governing permissions and |
2605 | 13 | # | 13 | # limitations under the License. |
2593 | 14 | # You should have received a copy of the GNU Lesser General Public License | ||
2594 | 15 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
2606 | 16 | 14 | ||
2607 | 17 | import os | 15 | import os |
2608 | 18 | import re | ||
2609 | 19 | import json | 16 | import json |
2610 | 17 | from inspect import getargspec | ||
2611 | 20 | from collections import Iterable, OrderedDict | 18 | from collections import Iterable, OrderedDict |
2612 | 21 | 19 | ||
2613 | 22 | from charmhelpers.core import host | 20 | from charmhelpers.core import host |
2614 | @@ -128,15 +126,18 @@ | |||
2615 | 128 | """ | 126 | """ |
2616 | 129 | Handle the current hook by doing The Right Thing with the registered services. | 127 | Handle the current hook by doing The Right Thing with the registered services. |
2617 | 130 | """ | 128 | """ |
2627 | 131 | hook_name = hookenv.hook_name() | 129 | hookenv._run_atstart() |
2628 | 132 | if hook_name == 'stop': | 130 | try: |
2629 | 133 | self.stop_services() | 131 | hook_name = hookenv.hook_name() |
2630 | 134 | else: | 132 | if hook_name == 'stop': |
2631 | 135 | self.provide_data() | 133 | self.stop_services() |
2632 | 136 | self.reconfigure_services() | 134 | else: |
2633 | 137 | cfg = hookenv.config() | 135 | self.reconfigure_services() |
2634 | 138 | if cfg.implicit_save: | 136 | self.provide_data() |
2635 | 139 | cfg.save() | 137 | except SystemExit as x: |
2636 | 138 | if x.code is None or x.code == 0: | ||
2637 | 139 | hookenv._run_atexit() | ||
2638 | 140 | hookenv._run_atexit() | ||
2639 | 140 | 141 | ||
2640 | 141 | def provide_data(self): | 142 | def provide_data(self): |
2641 | 142 | """ | 143 | """ |
2642 | @@ -145,15 +146,36 @@ | |||
2643 | 145 | A provider must have a `name` attribute, which indicates which relation | 146 | A provider must have a `name` attribute, which indicates which relation |
2644 | 146 | to set data on, and a `provide_data()` method, which returns a dict of | 147 | to set data on, and a `provide_data()` method, which returns a dict of |
2645 | 147 | data to set. | 148 | data to set. |
2646 | 149 | |||
2647 | 150 | The `provide_data()` method can optionally accept two parameters: | ||
2648 | 151 | |||
2649 | 152 | * ``remote_service`` The name of the remote service that the data will | ||
2650 | 153 | be provided to. The `provide_data()` method will be called once | ||
2651 | 154 | for each connected service (not unit). This allows the method to | ||
2652 | 155 | tailor its data to the given service. | ||
2653 | 156 | * ``service_ready`` Whether or not the service definition had all of | ||
2654 | 157 | its requirements met, and thus the ``data_ready`` callbacks run. | ||
2655 | 158 | |||
2656 | 159 | Note that the ``provided_data`` methods are now called **after** the | ||
2657 | 160 | ``data_ready`` callbacks are run. This gives the ``data_ready`` callbacks | ||
2658 | 161 | a chance to generate any data necessary for the providing to the remote | ||
2659 | 162 | services. | ||
2660 | 148 | """ | 163 | """ |
2663 | 149 | hook_name = hookenv.hook_name() | 164 | for service_name, service in self.services.items(): |
2664 | 150 | for service in self.services.values(): | 165 | service_ready = self.is_ready(service_name) |
2665 | 151 | for provider in service.get('provided_data', []): | 166 | for provider in service.get('provided_data', []): |
2671 | 152 | if re.match(r'{}-relation-(joined|changed)'.format(provider.name), hook_name): | 167 | for relid in hookenv.relation_ids(provider.name): |
2672 | 153 | data = provider.provide_data() | 168 | units = hookenv.related_units(relid) |
2673 | 154 | _ready = provider._is_ready(data) if hasattr(provider, '_is_ready') else data | 169 | if not units: |
2674 | 155 | if _ready: | 170 | continue |
2675 | 156 | hookenv.relation_set(None, data) | 171 | remote_service = units[0].split('/')[0] |
2676 | 172 | argspec = getargspec(provider.provide_data) | ||
2677 | 173 | if len(argspec.args) > 1: | ||
2678 | 174 | data = provider.provide_data(remote_service, service_ready) | ||
2679 | 175 | else: | ||
2680 | 176 | data = provider.provide_data() | ||
2681 | 177 | if data: | ||
2682 | 178 | hookenv.relation_set(relid, data) | ||
2683 | 157 | 179 | ||
2684 | 158 | def reconfigure_services(self, *service_names): | 180 | def reconfigure_services(self, *service_names): |
2685 | 159 | """ | 181 | """ |
2686 | 160 | 182 | ||
2687 | === modified file 'hooks/charmhelpers/core/services/helpers.py' | |||
2688 | --- hooks/charmhelpers/core/services/helpers.py 2015-05-14 10:48:09 +0000 | |||
2689 | +++ hooks/charmhelpers/core/services/helpers.py 2018-06-12 20:07:13 +0000 | |||
2690 | @@ -1,22 +1,22 @@ | |||
2691 | 1 | # Copyright 2014-2015 Canonical Limited. | 1 | # Copyright 2014-2015 Canonical Limited. |
2692 | 2 | # | 2 | # |
2706 | 3 | # This file is part of charm-helpers. | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); |
2707 | 4 | # | 4 | # you may not use this file except in compliance with the License. |
2708 | 5 | # charm-helpers is free software: you can redistribute it and/or modify | 5 | # You may obtain a copy of the License at |
2709 | 6 | # it under the terms of the GNU Lesser General Public License version 3 as | 6 | # |
2710 | 7 | # published by the Free Software Foundation. | 7 | # http://www.apache.org/licenses/LICENSE-2.0 |
2711 | 8 | # | 8 | # |
2712 | 9 | # charm-helpers is distributed in the hope that it will be useful, | 9 | # Unless required by applicable law or agreed to in writing, software |
2713 | 10 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 10 | # distributed under the License is distributed on an "AS IS" BASIS, |
2714 | 11 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
2715 | 12 | # GNU Lesser General Public License for more details. | 12 | # See the License for the specific language governing permissions and |
2716 | 13 | # | 13 | # limitations under the License. |
2704 | 14 | # You should have received a copy of the GNU Lesser General Public License | ||
2705 | 15 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
2717 | 16 | 14 | ||
2718 | 17 | import os | 15 | import os |
2719 | 18 | import yaml | 16 | import yaml |
2720 | 17 | |||
2721 | 19 | from charmhelpers.core import hookenv | 18 | from charmhelpers.core import hookenv |
2722 | 19 | from charmhelpers.core import host | ||
2723 | 20 | from charmhelpers.core import templating | 20 | from charmhelpers.core import templating |
2724 | 21 | 21 | ||
2725 | 22 | from charmhelpers.core.services.base import ManagerCallback | 22 | from charmhelpers.core.services.base import ManagerCallback |
2726 | @@ -239,28 +239,51 @@ | |||
2727 | 239 | action. | 239 | action. |
2728 | 240 | 240 | ||
2729 | 241 | :param str source: The template source file, relative to | 241 | :param str source: The template source file, relative to |
2731 | 242 | `$CHARM_DIR/templates` | 242 | `$CHARM_DIR/templates` |
2732 | 243 | 243 | ||
2734 | 244 | :param str target: The target to write the rendered template to | 244 | :param str target: The target to write the rendered template to (or None) |
2735 | 245 | :param str owner: The owner of the rendered file | 245 | :param str owner: The owner of the rendered file |
2736 | 246 | :param str group: The group of the rendered file | 246 | :param str group: The group of the rendered file |
2737 | 247 | :param int perms: The permissions of the rendered file | 247 | :param int perms: The permissions of the rendered file |
2738 | 248 | :param partial on_change_action: functools partial to be executed when | ||
2739 | 249 | rendered file changes | ||
2740 | 250 | :param jinja2 loader template_loader: A jinja2 template loader | ||
2741 | 251 | |||
2742 | 252 | :return str: The rendered template | ||
2743 | 248 | """ | 253 | """ |
2744 | 249 | def __init__(self, source, target, | 254 | def __init__(self, source, target, |
2746 | 250 | owner='root', group='root', perms=0o444): | 255 | owner='root', group='root', perms=0o444, |
2747 | 256 | on_change_action=None, template_loader=None): | ||
2748 | 251 | self.source = source | 257 | self.source = source |
2749 | 252 | self.target = target | 258 | self.target = target |
2750 | 253 | self.owner = owner | 259 | self.owner = owner |
2751 | 254 | self.group = group | 260 | self.group = group |
2752 | 255 | self.perms = perms | 261 | self.perms = perms |
2753 | 262 | self.on_change_action = on_change_action | ||
2754 | 263 | self.template_loader = template_loader | ||
2755 | 256 | 264 | ||
2756 | 257 | def __call__(self, manager, service_name, event_name): | 265 | def __call__(self, manager, service_name, event_name): |
2757 | 266 | pre_checksum = '' | ||
2758 | 267 | if self.on_change_action and os.path.isfile(self.target): | ||
2759 | 268 | pre_checksum = host.file_hash(self.target) | ||
2760 | 258 | service = manager.get_service(service_name) | 269 | service = manager.get_service(service_name) |
2762 | 259 | context = {} | 270 | context = {'ctx': {}} |
2763 | 260 | for ctx in service.get('required_data', []): | 271 | for ctx in service.get('required_data', []): |
2764 | 261 | context.update(ctx) | 272 | context.update(ctx) |
2767 | 262 | templating.render(self.source, self.target, context, | 273 | context['ctx'].update(ctx) |
2768 | 263 | self.owner, self.group, self.perms) | 274 | |
2769 | 275 | result = templating.render(self.source, self.target, context, | ||
2770 | 276 | self.owner, self.group, self.perms, | ||
2771 | 277 | template_loader=self.template_loader) | ||
2772 | 278 | if self.on_change_action: | ||
2773 | 279 | if pre_checksum == host.file_hash(self.target): | ||
2774 | 280 | hookenv.log( | ||
2775 | 281 | 'No change detected: {}'.format(self.target), | ||
2776 | 282 | hookenv.DEBUG) | ||
2777 | 283 | else: | ||
2778 | 284 | self.on_change_action() | ||
2779 | 285 | |||
2780 | 286 | return result | ||
2781 | 264 | 287 | ||
2782 | 265 | 288 | ||
2783 | 266 | # Convenience aliases for templates | 289 | # Convenience aliases for templates |
2784 | 267 | 290 | ||
2785 | === modified file 'hooks/charmhelpers/core/strutils.py' | |||
2786 | --- hooks/charmhelpers/core/strutils.py 2015-05-14 10:48:09 +0000 | |||
2787 | +++ hooks/charmhelpers/core/strutils.py 2018-06-12 20:07:13 +0000 | |||
2788 | @@ -3,21 +3,20 @@ | |||
2789 | 3 | 3 | ||
2790 | 4 | # Copyright 2014-2015 Canonical Limited. | 4 | # Copyright 2014-2015 Canonical Limited. |
2791 | 5 | # | 5 | # |
2805 | 6 | # This file is part of charm-helpers. | 6 | # Licensed under the Apache License, Version 2.0 (the "License"); |
2806 | 7 | # | 7 | # you may not use this file except in compliance with the License. |
2807 | 8 | # charm-helpers is free software: you can redistribute it and/or modify | 8 | # You may obtain a copy of the License at |
2808 | 9 | # it under the terms of the GNU Lesser General Public License version 3 as | 9 | # |
2809 | 10 | # published by the Free Software Foundation. | 10 | # http://www.apache.org/licenses/LICENSE-2.0 |
2810 | 11 | # | 11 | # |
2811 | 12 | # charm-helpers is distributed in the hope that it will be useful, | 12 | # Unless required by applicable law or agreed to in writing, software |
2812 | 13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 13 | # distributed under the License is distributed on an "AS IS" BASIS, |
2813 | 14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 14 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
2814 | 15 | # GNU Lesser General Public License for more details. | 15 | # See the License for the specific language governing permissions and |
2815 | 16 | # | 16 | # limitations under the License. |
2803 | 17 | # You should have received a copy of the GNU Lesser General Public License | ||
2804 | 18 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
2816 | 19 | 17 | ||
2817 | 20 | import six | 18 | import six |
2818 | 19 | import re | ||
2819 | 21 | 20 | ||
2820 | 22 | 21 | ||
2821 | 23 | def bool_from_string(value): | 22 | def bool_from_string(value): |
2822 | @@ -40,3 +39,32 @@ | |||
2823 | 40 | 39 | ||
2824 | 41 | msg = "Unable to interpret string value '%s' as boolean" % (value) | 40 | msg = "Unable to interpret string value '%s' as boolean" % (value) |
2825 | 42 | raise ValueError(msg) | 41 | raise ValueError(msg) |
2826 | 42 | |||
2827 | 43 | |||
2828 | 44 | def bytes_from_string(value): | ||
2829 | 45 | """Interpret human readable string value as bytes. | ||
2830 | 46 | |||
2831 | 47 | Returns int | ||
2832 | 48 | """ | ||
2833 | 49 | BYTE_POWER = { | ||
2834 | 50 | 'K': 1, | ||
2835 | 51 | 'KB': 1, | ||
2836 | 52 | 'M': 2, | ||
2837 | 53 | 'MB': 2, | ||
2838 | 54 | 'G': 3, | ||
2839 | 55 | 'GB': 3, | ||
2840 | 56 | 'T': 4, | ||
2841 | 57 | 'TB': 4, | ||
2842 | 58 | 'P': 5, | ||
2843 | 59 | 'PB': 5, | ||
2844 | 60 | } | ||
2845 | 61 | if isinstance(value, six.string_types): | ||
2846 | 62 | value = six.text_type(value) | ||
2847 | 63 | else: | ||
2848 | 64 | msg = "Unable to interpret non-string value '%s' as boolean" % (value) | ||
2849 | 65 | raise ValueError(msg) | ||
2850 | 66 | matches = re.match("([0-9]+)([a-zA-Z]+)", value) | ||
2851 | 67 | if not matches: | ||
2852 | 68 | msg = "Unable to interpret string value '%s' as bytes" % (value) | ||
2853 | 69 | raise ValueError(msg) | ||
2854 | 70 | return int(matches.group(1)) * (1024 ** BYTE_POWER[matches.group(2)]) | ||
2855 | 43 | 71 | ||
2856 | === modified file 'hooks/charmhelpers/core/sysctl.py' | |||
2857 | --- hooks/charmhelpers/core/sysctl.py 2015-05-14 10:48:09 +0000 | |||
2858 | +++ hooks/charmhelpers/core/sysctl.py 2018-06-12 20:07:13 +0000 | |||
2859 | @@ -3,19 +3,17 @@ | |||
2860 | 3 | 3 | ||
2861 | 4 | # Copyright 2014-2015 Canonical Limited. | 4 | # Copyright 2014-2015 Canonical Limited. |
2862 | 5 | # | 5 | # |
2876 | 6 | # This file is part of charm-helpers. | 6 | # Licensed under the Apache License, Version 2.0 (the "License"); |
2877 | 7 | # | 7 | # you may not use this file except in compliance with the License. |
2878 | 8 | # charm-helpers is free software: you can redistribute it and/or modify | 8 | # You may obtain a copy of the License at |
2879 | 9 | # it under the terms of the GNU Lesser General Public License version 3 as | 9 | # |
2880 | 10 | # published by the Free Software Foundation. | 10 | # http://www.apache.org/licenses/LICENSE-2.0 |
2881 | 11 | # | 11 | # |
2882 | 12 | # charm-helpers is distributed in the hope that it will be useful, | 12 | # Unless required by applicable law or agreed to in writing, software |
2883 | 13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 13 | # distributed under the License is distributed on an "AS IS" BASIS, |
2884 | 14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 14 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
2885 | 15 | # GNU Lesser General Public License for more details. | 15 | # See the License for the specific language governing permissions and |
2886 | 16 | # | 16 | # limitations under the License. |
2874 | 17 | # You should have received a copy of the GNU Lesser General Public License | ||
2875 | 18 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
2887 | 19 | 17 | ||
2888 | 20 | import yaml | 18 | import yaml |
2889 | 21 | 19 | ||
2890 | 22 | 20 | ||
2891 | === modified file 'hooks/charmhelpers/core/templating.py' | |||
2892 | --- hooks/charmhelpers/core/templating.py 2015-02-09 12:58:07 +0000 | |||
2893 | +++ hooks/charmhelpers/core/templating.py 2018-06-12 20:07:13 +0000 | |||
2894 | @@ -1,33 +1,33 @@ | |||
2895 | 1 | # Copyright 2014-2015 Canonical Limited. | 1 | # Copyright 2014-2015 Canonical Limited. |
2896 | 2 | # | 2 | # |
2910 | 3 | # This file is part of charm-helpers. | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); |
2911 | 4 | # | 4 | # you may not use this file except in compliance with the License. |
2912 | 5 | # charm-helpers is free software: you can redistribute it and/or modify | 5 | # You may obtain a copy of the License at |
2913 | 6 | # it under the terms of the GNU Lesser General Public License version 3 as | 6 | # |
2914 | 7 | # published by the Free Software Foundation. | 7 | # http://www.apache.org/licenses/LICENSE-2.0 |
2915 | 8 | # | 8 | # |
2916 | 9 | # charm-helpers is distributed in the hope that it will be useful, | 9 | # Unless required by applicable law or agreed to in writing, software |
2917 | 10 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 10 | # distributed under the License is distributed on an "AS IS" BASIS, |
2918 | 11 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
2919 | 12 | # GNU Lesser General Public License for more details. | 12 | # See the License for the specific language governing permissions and |
2920 | 13 | # | 13 | # limitations under the License. |
2908 | 14 | # You should have received a copy of the GNU Lesser General Public License | ||
2909 | 15 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
2921 | 16 | 14 | ||
2922 | 17 | import os | 15 | import os |
2923 | 16 | import sys | ||
2924 | 18 | 17 | ||
2925 | 19 | from charmhelpers.core import host | 18 | from charmhelpers.core import host |
2926 | 20 | from charmhelpers.core import hookenv | 19 | from charmhelpers.core import hookenv |
2927 | 21 | 20 | ||
2928 | 22 | 21 | ||
2929 | 23 | def render(source, target, context, owner='root', group='root', | 22 | def render(source, target, context, owner='root', group='root', |
2931 | 24 | perms=0o444, templates_dir=None, encoding='UTF-8'): | 23 | perms=0o444, templates_dir=None, encoding='UTF-8', template_loader=None): |
2932 | 25 | """ | 24 | """ |
2933 | 26 | Render a template. | 25 | Render a template. |
2934 | 27 | 26 | ||
2935 | 28 | The `source` path, if not absolute, is relative to the `templates_dir`. | 27 | The `source` path, if not absolute, is relative to the `templates_dir`. |
2936 | 29 | 28 | ||
2938 | 30 | The `target` path should be absolute. | 29 | The `target` path should be absolute. It can also be `None`, in which |
2939 | 30 | case no file will be written. | ||
2940 | 31 | 31 | ||
2941 | 32 | The context should be a dict containing the values to be replaced in the | 32 | The context should be a dict containing the values to be replaced in the |
2942 | 33 | template. | 33 | template. |
2943 | @@ -36,8 +36,12 @@ | |||
2944 | 36 | 36 | ||
2945 | 37 | If omitted, `templates_dir` defaults to the `templates` folder in the charm. | 37 | If omitted, `templates_dir` defaults to the `templates` folder in the charm. |
2946 | 38 | 38 | ||
2949 | 39 | Note: Using this requires python-jinja2; if it is not installed, calling | 39 | The rendered template will be written to the file as well as being returned |
2950 | 40 | this will attempt to use charmhelpers.fetch.apt_install to install it. | 40 | as a string. |
2951 | 41 | |||
2952 | 42 | Note: Using this requires python-jinja2 or python3-jinja2; if it is not | ||
2953 | 43 | installed, calling this will attempt to use charmhelpers.fetch.apt_install | ||
2954 | 44 | to install it. | ||
2955 | 41 | """ | 45 | """ |
2956 | 42 | try: | 46 | try: |
2957 | 43 | from jinja2 import FileSystemLoader, Environment, exceptions | 47 | from jinja2 import FileSystemLoader, Environment, exceptions |
2958 | @@ -49,20 +53,32 @@ | |||
2959 | 49 | 'charmhelpers.fetch to install it', | 53 | 'charmhelpers.fetch to install it', |
2960 | 50 | level=hookenv.ERROR) | 54 | level=hookenv.ERROR) |
2961 | 51 | raise | 55 | raise |
2963 | 52 | apt_install('python-jinja2', fatal=True) | 56 | if sys.version_info.major == 2: |
2964 | 57 | apt_install('python-jinja2', fatal=True) | ||
2965 | 58 | else: | ||
2966 | 59 | apt_install('python3-jinja2', fatal=True) | ||
2967 | 53 | from jinja2 import FileSystemLoader, Environment, exceptions | 60 | from jinja2 import FileSystemLoader, Environment, exceptions |
2968 | 54 | 61 | ||
2972 | 55 | if templates_dir is None: | 62 | if template_loader: |
2973 | 56 | templates_dir = os.path.join(hookenv.charm_dir(), 'templates') | 63 | template_env = Environment(loader=template_loader) |
2974 | 57 | loader = Environment(loader=FileSystemLoader(templates_dir)) | 64 | else: |
2975 | 65 | if templates_dir is None: | ||
2976 | 66 | templates_dir = os.path.join(hookenv.charm_dir(), 'templates') | ||
2977 | 67 | template_env = Environment(loader=FileSystemLoader(templates_dir)) | ||
2978 | 58 | try: | 68 | try: |
2979 | 59 | source = source | 69 | source = source |
2981 | 60 | template = loader.get_template(source) | 70 | template = template_env.get_template(source) |
2982 | 61 | except exceptions.TemplateNotFound as e: | 71 | except exceptions.TemplateNotFound as e: |
2983 | 62 | hookenv.log('Could not load template %s from %s.' % | 72 | hookenv.log('Could not load template %s from %s.' % |
2984 | 63 | (source, templates_dir), | 73 | (source, templates_dir), |
2985 | 64 | level=hookenv.ERROR) | 74 | level=hookenv.ERROR) |
2986 | 65 | raise e | 75 | raise e |
2987 | 66 | content = template.render(context) | 76 | content = template.render(context) |
2990 | 67 | host.mkdir(os.path.dirname(target), owner, group, perms=0o755) | 77 | if target is not None: |
2991 | 68 | host.write_file(target, content.encode(encoding), owner, group, perms) | 78 | target_dir = os.path.dirname(target) |
2992 | 79 | if not os.path.exists(target_dir): | ||
2993 | 80 | # This is a terrible default directory permission, as the file | ||
2994 | 81 | # or its siblings will often contain secrets. | ||
2995 | 82 | host.mkdir(os.path.dirname(target), owner, group, perms=0o755) | ||
2996 | 83 | host.write_file(target, content.encode(encoding), owner, group, perms) | ||
2997 | 84 | return content | ||
2998 | 69 | 85 | ||
2999 | === modified file 'hooks/charmhelpers/core/unitdata.py' | |||
3000 | --- hooks/charmhelpers/core/unitdata.py 2015-05-14 10:48:09 +0000 | |||
3001 | +++ hooks/charmhelpers/core/unitdata.py 2018-06-12 20:07:13 +0000 | |||
3002 | @@ -3,20 +3,17 @@ | |||
3003 | 3 | # | 3 | # |
3004 | 4 | # Copyright 2014-2015 Canonical Limited. | 4 | # Copyright 2014-2015 Canonical Limited. |
3005 | 5 | # | 5 | # |
3020 | 6 | # This file is part of charm-helpers. | 6 | # Licensed under the Apache License, Version 2.0 (the "License"); |
3021 | 7 | # | 7 | # you may not use this file except in compliance with the License. |
3022 | 8 | # charm-helpers is free software: you can redistribute it and/or modify | 8 | # You may obtain a copy of the License at |
3023 | 9 | # it under the terms of the GNU Lesser General Public License version 3 as | 9 | # |
3024 | 10 | # published by the Free Software Foundation. | 10 | # http://www.apache.org/licenses/LICENSE-2.0 |
3025 | 11 | # | 11 | # |
3026 | 12 | # charm-helpers is distributed in the hope that it will be useful, | 12 | # Unless required by applicable law or agreed to in writing, software |
3027 | 13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 13 | # distributed under the License is distributed on an "AS IS" BASIS, |
3028 | 14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 14 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
3029 | 15 | # GNU Lesser General Public License for more details. | 15 | # See the License for the specific language governing permissions and |
3030 | 16 | # | 16 | # limitations under the License. |
3017 | 17 | # You should have received a copy of the GNU Lesser General Public License | ||
3018 | 18 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
3019 | 19 | # | ||
3031 | 20 | # | 17 | # |
3032 | 21 | # Authors: | 18 | # Authors: |
3033 | 22 | # Kapil Thangavelu <kapil.foss@gmail.com> | 19 | # Kapil Thangavelu <kapil.foss@gmail.com> |
3034 | @@ -152,6 +149,7 @@ | |||
3035 | 152 | import collections | 149 | import collections |
3036 | 153 | import contextlib | 150 | import contextlib |
3037 | 154 | import datetime | 151 | import datetime |
3038 | 152 | import itertools | ||
3039 | 155 | import json | 153 | import json |
3040 | 156 | import os | 154 | import os |
3041 | 157 | import pprint | 155 | import pprint |
3042 | @@ -164,8 +162,7 @@ | |||
3043 | 164 | class Storage(object): | 162 | class Storage(object): |
3044 | 165 | """Simple key value database for local unit state within charms. | 163 | """Simple key value database for local unit state within charms. |
3045 | 166 | 164 | ||
3048 | 167 | Modifications are automatically committed at hook exit. That's | 165 | Modifications are not persisted unless :meth:`flush` is called. |
3047 | 168 | currently regardless of exit code. | ||
3049 | 169 | 166 | ||
3050 | 170 | To support dicts, lists, integer, floats, and booleans values | 167 | To support dicts, lists, integer, floats, and booleans values |
3051 | 171 | are automatically json encoded/decoded. | 168 | are automatically json encoded/decoded. |
3052 | @@ -173,8 +170,11 @@ | |||
3053 | 173 | def __init__(self, path=None): | 170 | def __init__(self, path=None): |
3054 | 174 | self.db_path = path | 171 | self.db_path = path |
3055 | 175 | if path is None: | 172 | if path is None: |
3058 | 176 | self.db_path = os.path.join( | 173 | if 'UNIT_STATE_DB' in os.environ: |
3059 | 177 | os.environ.get('CHARM_DIR', ''), '.unit-state.db') | 174 | self.db_path = os.environ['UNIT_STATE_DB'] |
3060 | 175 | else: | ||
3061 | 176 | self.db_path = os.path.join( | ||
3062 | 177 | os.environ.get('CHARM_DIR', ''), '.unit-state.db') | ||
3063 | 178 | self.conn = sqlite3.connect('%s' % self.db_path) | 178 | self.conn = sqlite3.connect('%s' % self.db_path) |
3064 | 179 | self.cursor = self.conn.cursor() | 179 | self.cursor = self.conn.cursor() |
3065 | 180 | self.revision = None | 180 | self.revision = None |
3066 | @@ -189,15 +189,8 @@ | |||
3067 | 189 | self.conn.close() | 189 | self.conn.close() |
3068 | 190 | self._closed = True | 190 | self._closed = True |
3069 | 191 | 191 | ||
3070 | 192 | def _scoped_query(self, stmt, params=None): | ||
3071 | 193 | if params is None: | ||
3072 | 194 | params = [] | ||
3073 | 195 | return stmt, params | ||
3074 | 196 | |||
3075 | 197 | def get(self, key, default=None, record=False): | 192 | def get(self, key, default=None, record=False): |
3079 | 198 | self.cursor.execute( | 193 | self.cursor.execute('select data from kv where key=?', [key]) |
3077 | 199 | *self._scoped_query( | ||
3078 | 200 | 'select data from kv where key=?', [key])) | ||
3080 | 201 | result = self.cursor.fetchone() | 194 | result = self.cursor.fetchone() |
3081 | 202 | if not result: | 195 | if not result: |
3082 | 203 | return default | 196 | return default |
3083 | @@ -206,33 +199,81 @@ | |||
3084 | 206 | return json.loads(result[0]) | 199 | return json.loads(result[0]) |
3085 | 207 | 200 | ||
3086 | 208 | def getrange(self, key_prefix, strip=False): | 201 | def getrange(self, key_prefix, strip=False): |
3089 | 209 | stmt = "select key, data from kv where key like '%s%%'" % key_prefix | 202 | """ |
3090 | 210 | self.cursor.execute(*self._scoped_query(stmt)) | 203 | Get a range of keys starting with a common prefix as a mapping of |
3091 | 204 | keys to values. | ||
3092 | 205 | |||
3093 | 206 | :param str key_prefix: Common prefix among all keys | ||
3094 | 207 | :param bool strip: Optionally strip the common prefix from the key | ||
3095 | 208 | names in the returned dict | ||
3096 | 209 | :return dict: A (possibly empty) dict of key-value mappings | ||
3097 | 210 | """ | ||
3098 | 211 | self.cursor.execute("select key, data from kv where key like ?", | ||
3099 | 212 | ['%s%%' % key_prefix]) | ||
3100 | 211 | result = self.cursor.fetchall() | 213 | result = self.cursor.fetchall() |
3101 | 212 | 214 | ||
3102 | 213 | if not result: | 215 | if not result: |
3104 | 214 | return None | 216 | return {} |
3105 | 215 | if not strip: | 217 | if not strip: |
3106 | 216 | key_prefix = '' | 218 | key_prefix = '' |
3107 | 217 | return dict([ | 219 | return dict([ |
3108 | 218 | (k[len(key_prefix):], json.loads(v)) for k, v in result]) | 220 | (k[len(key_prefix):], json.loads(v)) for k, v in result]) |
3109 | 219 | 221 | ||
3110 | 220 | def update(self, mapping, prefix=""): | 222 | def update(self, mapping, prefix=""): |
3111 | 223 | """ | ||
3112 | 224 | Set the values of multiple keys at once. | ||
3113 | 225 | |||
3114 | 226 | :param dict mapping: Mapping of keys to values | ||
3115 | 227 | :param str prefix: Optional prefix to apply to all keys in `mapping` | ||
3116 | 228 | before setting | ||
3117 | 229 | """ | ||
3118 | 221 | for k, v in mapping.items(): | 230 | for k, v in mapping.items(): |
3119 | 222 | self.set("%s%s" % (prefix, k), v) | 231 | self.set("%s%s" % (prefix, k), v) |
3120 | 223 | 232 | ||
3121 | 224 | def unset(self, key): | 233 | def unset(self, key): |
3122 | 234 | """ | ||
3123 | 235 | Remove a key from the database entirely. | ||
3124 | 236 | """ | ||
3125 | 225 | self.cursor.execute('delete from kv where key=?', [key]) | 237 | self.cursor.execute('delete from kv where key=?', [key]) |
3126 | 226 | if self.revision and self.cursor.rowcount: | 238 | if self.revision and self.cursor.rowcount: |
3127 | 227 | self.cursor.execute( | 239 | self.cursor.execute( |
3128 | 228 | 'insert into kv_revisions values (?, ?, ?)', | 240 | 'insert into kv_revisions values (?, ?, ?)', |
3129 | 229 | [key, self.revision, json.dumps('DELETED')]) | 241 | [key, self.revision, json.dumps('DELETED')]) |
3130 | 230 | 242 | ||
3131 | 243 | def unsetrange(self, keys=None, prefix=""): | ||
3132 | 244 | """ | ||
3133 | 245 | Remove a range of keys starting with a common prefix, from the database | ||
3134 | 246 | entirely. | ||
3135 | 247 | |||
3136 | 248 | :param list keys: List of keys to remove. | ||
3137 | 249 | :param str prefix: Optional prefix to apply to all keys in ``keys`` | ||
3138 | 250 | before removing. | ||
3139 | 251 | """ | ||
3140 | 252 | if keys is not None: | ||
3141 | 253 | keys = ['%s%s' % (prefix, key) for key in keys] | ||
3142 | 254 | self.cursor.execute('delete from kv where key in (%s)' % ','.join(['?'] * len(keys)), keys) | ||
3143 | 255 | if self.revision and self.cursor.rowcount: | ||
3144 | 256 | self.cursor.execute( | ||
3145 | 257 | 'insert into kv_revisions values %s' % ','.join(['(?, ?, ?)'] * len(keys)), | ||
3146 | 258 | list(itertools.chain.from_iterable((key, self.revision, json.dumps('DELETED')) for key in keys))) | ||
3147 | 259 | else: | ||
3148 | 260 | self.cursor.execute('delete from kv where key like ?', | ||
3149 | 261 | ['%s%%' % prefix]) | ||
3150 | 262 | if self.revision and self.cursor.rowcount: | ||
3151 | 263 | self.cursor.execute( | ||
3152 | 264 | 'insert into kv_revisions values (?, ?, ?)', | ||
3153 | 265 | ['%s%%' % prefix, self.revision, json.dumps('DELETED')]) | ||
3154 | 266 | |||
3155 | 231 | def set(self, key, value): | 267 | def set(self, key, value): |
3156 | 268 | """ | ||
3157 | 269 | Set a value in the database. | ||
3158 | 270 | |||
3159 | 271 | :param str key: Key to set the value for | ||
3160 | 272 | :param value: Any JSON-serializable value to be set | ||
3161 | 273 | """ | ||
3162 | 232 | serialized = json.dumps(value) | 274 | serialized = json.dumps(value) |
3163 | 233 | 275 | ||
3166 | 234 | self.cursor.execute( | 276 | self.cursor.execute('select data from kv where key=?', [key]) |
3165 | 235 | 'select data from kv where key=?', [key]) | ||
3167 | 236 | exists = self.cursor.fetchone() | 277 | exists = self.cursor.fetchone() |
3168 | 237 | 278 | ||
3169 | 238 | # Skip mutations to the same value | 279 | # Skip mutations to the same value |
3170 | 239 | 280 | ||
3171 | === modified file 'hooks/charmhelpers/fetch/__init__.py' | |||
3172 | --- hooks/charmhelpers/fetch/__init__.py 2016-02-12 18:30:48 +0000 | |||
3173 | +++ hooks/charmhelpers/fetch/__init__.py 2018-06-12 20:07:13 +0000 | |||
3174 | @@ -1,32 +1,24 @@ | |||
3175 | 1 | # Copyright 2014-2015 Canonical Limited. | 1 | # Copyright 2014-2015 Canonical Limited. |
3176 | 2 | # | 2 | # |
3190 | 3 | # This file is part of charm-helpers. | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); |
3191 | 4 | # | 4 | # you may not use this file except in compliance with the License. |
3192 | 5 | # charm-helpers is free software: you can redistribute it and/or modify | 5 | # You may obtain a copy of the License at |
3193 | 6 | # it under the terms of the GNU Lesser General Public License version 3 as | 6 | # |
3194 | 7 | # published by the Free Software Foundation. | 7 | # http://www.apache.org/licenses/LICENSE-2.0 |
3195 | 8 | # | 8 | # |
3196 | 9 | # charm-helpers is distributed in the hope that it will be useful, | 9 | # Unless required by applicable law or agreed to in writing, software |
3197 | 10 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 10 | # distributed under the License is distributed on an "AS IS" BASIS, |
3198 | 11 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
3199 | 12 | # GNU Lesser General Public License for more details. | 12 | # See the License for the specific language governing permissions and |
3200 | 13 | # | 13 | # limitations under the License. |
3188 | 14 | # You should have received a copy of the GNU Lesser General Public License | ||
3189 | 15 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
3201 | 16 | 14 | ||
3202 | 17 | import importlib | 15 | import importlib |
3205 | 18 | from tempfile import NamedTemporaryFile | 16 | from charmhelpers.osplatform import get_platform |
3204 | 19 | import time | ||
3206 | 20 | from yaml import safe_load | 17 | from yaml import safe_load |
3207 | 21 | from charmhelpers.core.host import ( | ||
3208 | 22 | lsb_release | ||
3209 | 23 | ) | ||
3210 | 24 | import subprocess | ||
3211 | 25 | from charmhelpers.core.hookenv import ( | 18 | from charmhelpers.core.hookenv import ( |
3212 | 26 | config, | 19 | config, |
3213 | 27 | log, | 20 | log, |
3214 | 28 | ) | 21 | ) |
3215 | 29 | import os | ||
3216 | 30 | 22 | ||
3217 | 31 | import six | 23 | import six |
3218 | 32 | if six.PY3: | 24 | if six.PY3: |
3219 | @@ -35,71 +27,6 @@ | |||
3220 | 35 | from urlparse import urlparse, urlunparse | 27 | from urlparse import urlparse, urlunparse |
3221 | 36 | 28 | ||
3222 | 37 | 29 | ||
3223 | 38 | CLOUD_ARCHIVE = """# Ubuntu Cloud Archive | ||
3224 | 39 | deb http://ubuntu-cloud.archive.canonical.com/ubuntu {} main | ||
3225 | 40 | """ | ||
3226 | 41 | PROPOSED_POCKET = """# Proposed | ||
3227 | 42 | deb http://archive.ubuntu.com/ubuntu {}-proposed main universe multiverse restricted | ||
3228 | 43 | """ | ||
3229 | 44 | CLOUD_ARCHIVE_POCKETS = { | ||
3230 | 45 | # Folsom | ||
3231 | 46 | 'folsom': 'precise-updates/folsom', | ||
3232 | 47 | 'precise-folsom': 'precise-updates/folsom', | ||
3233 | 48 | 'precise-folsom/updates': 'precise-updates/folsom', | ||
3234 | 49 | 'precise-updates/folsom': 'precise-updates/folsom', | ||
3235 | 50 | 'folsom/proposed': 'precise-proposed/folsom', | ||
3236 | 51 | 'precise-folsom/proposed': 'precise-proposed/folsom', | ||
3237 | 52 | 'precise-proposed/folsom': 'precise-proposed/folsom', | ||
3238 | 53 | # Grizzly | ||
3239 | 54 | 'grizzly': 'precise-updates/grizzly', | ||
3240 | 55 | 'precise-grizzly': 'precise-updates/grizzly', | ||
3241 | 56 | 'precise-grizzly/updates': 'precise-updates/grizzly', | ||
3242 | 57 | 'precise-updates/grizzly': 'precise-updates/grizzly', | ||
3243 | 58 | 'grizzly/proposed': 'precise-proposed/grizzly', | ||
3244 | 59 | 'precise-grizzly/proposed': 'precise-proposed/grizzly', | ||
3245 | 60 | 'precise-proposed/grizzly': 'precise-proposed/grizzly', | ||
3246 | 61 | # Havana | ||
3247 | 62 | 'havana': 'precise-updates/havana', | ||
3248 | 63 | 'precise-havana': 'precise-updates/havana', | ||
3249 | 64 | 'precise-havana/updates': 'precise-updates/havana', | ||
3250 | 65 | 'precise-updates/havana': 'precise-updates/havana', | ||
3251 | 66 | 'havana/proposed': 'precise-proposed/havana', | ||
3252 | 67 | 'precise-havana/proposed': 'precise-proposed/havana', | ||
3253 | 68 | 'precise-proposed/havana': 'precise-proposed/havana', | ||
3254 | 69 | # Icehouse | ||
3255 | 70 | 'icehouse': 'precise-updates/icehouse', | ||
3256 | 71 | 'precise-icehouse': 'precise-updates/icehouse', | ||
3257 | 72 | 'precise-icehouse/updates': 'precise-updates/icehouse', | ||
3258 | 73 | 'precise-updates/icehouse': 'precise-updates/icehouse', | ||
3259 | 74 | 'icehouse/proposed': 'precise-proposed/icehouse', | ||
3260 | 75 | 'precise-icehouse/proposed': 'precise-proposed/icehouse', | ||
3261 | 76 | 'precise-proposed/icehouse': 'precise-proposed/icehouse', | ||
3262 | 77 | # Juno | ||
3263 | 78 | 'juno': 'trusty-updates/juno', | ||
3264 | 79 | 'trusty-juno': 'trusty-updates/juno', | ||
3265 | 80 | 'trusty-juno/updates': 'trusty-updates/juno', | ||
3266 | 81 | 'trusty-updates/juno': 'trusty-updates/juno', | ||
3267 | 82 | 'juno/proposed': 'trusty-proposed/juno', | ||
3268 | 83 | 'trusty-juno/proposed': 'trusty-proposed/juno', | ||
3269 | 84 | 'trusty-proposed/juno': 'trusty-proposed/juno', | ||
3270 | 85 | # Kilo | ||
3271 | 86 | 'kilo': 'trusty-updates/kilo', | ||
3272 | 87 | 'trusty-kilo': 'trusty-updates/kilo', | ||
3273 | 88 | 'trusty-kilo/updates': 'trusty-updates/kilo', | ||
3274 | 89 | 'trusty-updates/kilo': 'trusty-updates/kilo', | ||
3275 | 90 | 'kilo/proposed': 'trusty-proposed/kilo', | ||
3276 | 91 | 'trusty-kilo/proposed': 'trusty-proposed/kilo', | ||
3277 | 92 | 'trusty-proposed/kilo': 'trusty-proposed/kilo', | ||
3278 | 93 | # Liberty | ||
3279 | 94 | 'liberty': 'trusty-updates/liberty', | ||
3280 | 95 | 'trusty-liberty': 'trusty-updates/liberty', | ||
3281 | 96 | 'trusty-liberty/updates': 'trusty-updates/liberty', | ||
3282 | 97 | 'trusty-updates/liberty': 'trusty-updates/liberty', | ||
3283 | 98 | 'liberty/proposed': 'trusty-proposed/liberty', | ||
3284 | 99 | 'trusty-liberty/proposed': 'trusty-proposed/liberty', | ||
3285 | 100 | 'trusty-proposed/liberty': 'trusty-proposed/liberty', | ||
3286 | 101 | } | ||
3287 | 102 | |||
3288 | 103 | # The order of this list is very important. Handlers should be listed in from | 30 | # The order of this list is very important. Handlers should be listed in from |
3289 | 104 | # least- to most-specific URL matching. | 31 | # least- to most-specific URL matching. |
3290 | 105 | FETCH_HANDLERS = ( | 32 | FETCH_HANDLERS = ( |
3291 | @@ -108,10 +35,6 @@ | |||
3292 | 108 | 'charmhelpers.fetch.giturl.GitUrlFetchHandler', | 35 | 'charmhelpers.fetch.giturl.GitUrlFetchHandler', |
3293 | 109 | ) | 36 | ) |
3294 | 110 | 37 | ||
3295 | 111 | APT_NO_LOCK = 100 # The return code for "couldn't acquire lock" in APT. | ||
3296 | 112 | APT_NO_LOCK_RETRY_DELAY = 10 # Wait 10 seconds between apt lock checks. | ||
3297 | 113 | APT_NO_LOCK_RETRY_COUNT = 30 # Retry to acquire the lock X times. | ||
3298 | 114 | |||
3299 | 115 | 38 | ||
3300 | 116 | class SourceConfigError(Exception): | 39 | class SourceConfigError(Exception): |
3301 | 117 | pass | 40 | pass |
3302 | @@ -149,172 +72,38 @@ | |||
3303 | 149 | return urlunparse(parts) | 72 | return urlunparse(parts) |
3304 | 150 | 73 | ||
3305 | 151 | 74 | ||
3462 | 152 | def filter_installed_packages(packages): | 75 | __platform__ = get_platform() |
3463 | 153 | """Returns a list of packages that require installation""" | 76 | module = "charmhelpers.fetch.%s" % __platform__ |
3464 | 154 | cache = apt_cache() | 77 | fetch = importlib.import_module(module) |
3465 | 155 | _pkgs = [] | 78 | |
3466 | 156 | for package in packages: | 79 | filter_installed_packages = fetch.filter_installed_packages |
3467 | 157 | try: | 80 | install = fetch.install |
3468 | 158 | p = cache[package] | 81 | upgrade = fetch.upgrade |
3469 | 159 | p.current_ver or _pkgs.append(package) | 82 | update = fetch.update |
3470 | 160 | except KeyError: | 83 | purge = fetch.purge |
3471 | 161 | log('Package {} has no installation candidate.'.format(package), | 84 | add_source = fetch.add_source |
3472 | 162 | level='WARNING') | 85 | |
3473 | 163 | _pkgs.append(package) | 86 | if __platform__ == "ubuntu": |
3474 | 164 | return _pkgs | 87 | apt_cache = fetch.apt_cache |
3475 | 165 | 88 | apt_install = fetch.install | |
3476 | 166 | 89 | apt_update = fetch.update | |
3477 | 167 | def apt_cache(in_memory=True): | 90 | apt_upgrade = fetch.upgrade |
3478 | 168 | """Build and return an apt cache""" | 91 | apt_purge = fetch.purge |
3479 | 169 | from apt import apt_pkg | 92 | apt_mark = fetch.apt_mark |
3480 | 170 | apt_pkg.init() | 93 | apt_hold = fetch.apt_hold |
3481 | 171 | if in_memory: | 94 | apt_unhold = fetch.apt_unhold |
3482 | 172 | apt_pkg.config.set("Dir::Cache::pkgcache", "") | 95 | get_upstream_version = fetch.get_upstream_version |
3483 | 173 | apt_pkg.config.set("Dir::Cache::srcpkgcache", "") | 96 | elif __platform__ == "centos": |
3484 | 174 | return apt_pkg.Cache() | 97 | yum_search = fetch.yum_search |
3329 | 175 | |||
3330 | 176 | |||
3331 | 177 | def apt_install(packages, options=None, fatal=False): | ||
3332 | 178 | """Install one or more packages""" | ||
3333 | 179 | if options is None: | ||
3334 | 180 | options = ['--option=Dpkg::Options::=--force-confold'] | ||
3335 | 181 | |||
3336 | 182 | cmd = ['apt-get', '--assume-yes'] | ||
3337 | 183 | cmd.extend(options) | ||
3338 | 184 | cmd.append('install') | ||
3339 | 185 | if isinstance(packages, six.string_types): | ||
3340 | 186 | cmd.append(packages) | ||
3341 | 187 | else: | ||
3342 | 188 | cmd.extend(packages) | ||
3343 | 189 | log("Installing {} with options: {}".format(packages, | ||
3344 | 190 | options)) | ||
3345 | 191 | _run_apt_command(cmd, fatal) | ||
3346 | 192 | |||
3347 | 193 | |||
3348 | 194 | def apt_upgrade(options=None, fatal=False, dist=False): | ||
3349 | 195 | """Upgrade all packages""" | ||
3350 | 196 | if options is None: | ||
3351 | 197 | options = ['--option=Dpkg::Options::=--force-confold'] | ||
3352 | 198 | |||
3353 | 199 | cmd = ['apt-get', '--assume-yes'] | ||
3354 | 200 | cmd.extend(options) | ||
3355 | 201 | if dist: | ||
3356 | 202 | cmd.append('dist-upgrade') | ||
3357 | 203 | else: | ||
3358 | 204 | cmd.append('upgrade') | ||
3359 | 205 | log("Upgrading with options: {}".format(options)) | ||
3360 | 206 | _run_apt_command(cmd, fatal) | ||
3361 | 207 | |||
3362 | 208 | |||
3363 | 209 | def apt_update(fatal=False): | ||
3364 | 210 | """Update local apt cache""" | ||
3365 | 211 | cmd = ['apt-get', 'update'] | ||
3366 | 212 | _run_apt_command(cmd, fatal) | ||
3367 | 213 | |||
3368 | 214 | |||
3369 | 215 | def apt_purge(packages, fatal=False): | ||
3370 | 216 | """Purge one or more packages""" | ||
3371 | 217 | cmd = ['apt-get', '--assume-yes', 'purge'] | ||
3372 | 218 | if isinstance(packages, six.string_types): | ||
3373 | 219 | cmd.append(packages) | ||
3374 | 220 | else: | ||
3375 | 221 | cmd.extend(packages) | ||
3376 | 222 | log("Purging {}".format(packages)) | ||
3377 | 223 | _run_apt_command(cmd, fatal) | ||
3378 | 224 | |||
3379 | 225 | |||
3380 | 226 | def apt_hold(packages, fatal=False): | ||
3381 | 227 | """Hold one or more packages""" | ||
3382 | 228 | cmd = ['apt-mark', 'hold'] | ||
3383 | 229 | if isinstance(packages, six.string_types): | ||
3384 | 230 | cmd.append(packages) | ||
3385 | 231 | else: | ||
3386 | 232 | cmd.extend(packages) | ||
3387 | 233 | log("Holding {}".format(packages)) | ||
3388 | 234 | |||
3389 | 235 | if fatal: | ||
3390 | 236 | subprocess.check_call(cmd) | ||
3391 | 237 | else: | ||
3392 | 238 | subprocess.call(cmd) | ||
3393 | 239 | |||
3394 | 240 | |||
3395 | 241 | def add_source(source, key=None): | ||
3396 | 242 | """Add a package source to this system. | ||
3397 | 243 | |||
3398 | 244 | @param source: a URL or sources.list entry, as supported by | ||
3399 | 245 | add-apt-repository(1). Examples:: | ||
3400 | 246 | |||
3401 | 247 | ppa:charmers/example | ||
3402 | 248 | deb https://stub:key@private.example.com/ubuntu trusty main | ||
3403 | 249 | |||
3404 | 250 | In addition: | ||
3405 | 251 | 'proposed:' may be used to enable the standard 'proposed' | ||
3406 | 252 | pocket for the release. | ||
3407 | 253 | 'cloud:' may be used to activate official cloud archive pockets, | ||
3408 | 254 | such as 'cloud:icehouse' | ||
3409 | 255 | 'distro' may be used as a noop | ||
3410 | 256 | |||
3411 | 257 | @param key: A key to be added to the system's APT keyring and used | ||
3412 | 258 | to verify the signatures on packages. Ideally, this should be an | ||
3413 | 259 | ASCII format GPG public key including the block headers. A GPG key | ||
3414 | 260 | id may also be used, but be aware that only insecure protocols are | ||
3415 | 261 | available to retrieve the actual public key from a public keyserver | ||
3416 | 262 | placing your Juju environment at risk. ppa and cloud archive keys | ||
3417 | 263 | are securely added automtically, so sould not be provided. | ||
3418 | 264 | """ | ||
3419 | 265 | if source is None: | ||
3420 | 266 | log('Source is not present. Skipping') | ||
3421 | 267 | return | ||
3422 | 268 | |||
3423 | 269 | if (source.startswith('ppa:') or | ||
3424 | 270 | source.startswith('http') or | ||
3425 | 271 | source.startswith('deb ') or | ||
3426 | 272 | source.startswith('cloud-archive:')): | ||
3427 | 273 | subprocess.check_call(['add-apt-repository', '--yes', source]) | ||
3428 | 274 | elif source.startswith('cloud:'): | ||
3429 | 275 | apt_install(filter_installed_packages(['ubuntu-cloud-keyring']), | ||
3430 | 276 | fatal=True) | ||
3431 | 277 | pocket = source.split(':')[-1] | ||
3432 | 278 | if pocket not in CLOUD_ARCHIVE_POCKETS: | ||
3433 | 279 | raise SourceConfigError( | ||
3434 | 280 | 'Unsupported cloud: source option %s' % | ||
3435 | 281 | pocket) | ||
3436 | 282 | actual_pocket = CLOUD_ARCHIVE_POCKETS[pocket] | ||
3437 | 283 | with open('/etc/apt/sources.list.d/cloud-archive.list', 'w') as apt: | ||
3438 | 284 | apt.write(CLOUD_ARCHIVE.format(actual_pocket)) | ||
3439 | 285 | elif source == 'proposed': | ||
3440 | 286 | release = lsb_release()['DISTRIB_CODENAME'] | ||
3441 | 287 | with open('/etc/apt/sources.list.d/proposed.list', 'w') as apt: | ||
3442 | 288 | apt.write(PROPOSED_POCKET.format(release)) | ||
3443 | 289 | elif source == 'distro': | ||
3444 | 290 | pass | ||
3445 | 291 | else: | ||
3446 | 292 | log("Unknown source: {!r}".format(source)) | ||
3447 | 293 | |||
3448 | 294 | if key: | ||
3449 | 295 | if '-----BEGIN PGP PUBLIC KEY BLOCK-----' in key: | ||
3450 | 296 | with NamedTemporaryFile('w+') as key_file: | ||
3451 | 297 | key_file.write(key) | ||
3452 | 298 | key_file.flush() | ||
3453 | 299 | key_file.seek(0) | ||
3454 | 300 | subprocess.check_call(['apt-key', 'add', '-'], stdin=key_file) | ||
3455 | 301 | else: | ||
3456 | 302 | # Note that hkp: is in no way a secure protocol. Using a | ||
3457 | 303 | # GPG key id is pointless from a security POV unless you | ||
3458 | 304 | # absolutely trust your network and DNS. | ||
3459 | 305 | subprocess.check_call(['apt-key', 'adv', '--keyserver', | ||
3460 | 306 | 'hkp://keyserver.ubuntu.com:80', '--recv', | ||
3461 | 307 | key]) | ||
3485 | 308 | 98 | ||
3486 | 309 | 99 | ||
3487 | 310 | def configure_sources(update=False, | 100 | def configure_sources(update=False, |
3488 | 311 | sources_var='install_sources', | 101 | sources_var='install_sources', |
3489 | 312 | keys_var='install_keys'): | 102 | keys_var='install_keys'): |
3492 | 313 | """ | 103 | """Configure multiple sources from charm configuration. |
3491 | 314 | Configure multiple sources from charm configuration. | ||
3493 | 315 | 104 | ||
3494 | 316 | The lists are encoded as yaml fragments in the configuration. | 105 | The lists are encoded as yaml fragments in the configuration. |
3496 | 317 | The frament needs to be included as a string. Sources and their | 106 | The fragment needs to be included as a string. Sources and their |
3497 | 318 | corresponding keys are of the types supported by add_source(). | 107 | corresponding keys are of the types supported by add_source(). |
3498 | 319 | 108 | ||
3499 | 320 | Example config: | 109 | Example config: |
3500 | @@ -346,12 +135,11 @@ | |||
3501 | 346 | for source, key in zip(sources, keys): | 135 | for source, key in zip(sources, keys): |
3502 | 347 | add_source(source, key) | 136 | add_source(source, key) |
3503 | 348 | if update: | 137 | if update: |
3505 | 349 | apt_update(fatal=True) | 138 | fetch.update(fatal=True) |
3506 | 350 | 139 | ||
3507 | 351 | 140 | ||
3508 | 352 | def install_remote(source, *args, **kwargs): | 141 | def install_remote(source, *args, **kwargs): |
3511 | 353 | """ | 142 | """Install a file tree from a remote source. |
3510 | 354 | Install a file tree from a remote source | ||
3512 | 355 | 143 | ||
3513 | 356 | The specified source should be a url of the form: | 144 | The specified source should be a url of the form: |
3514 | 357 | scheme://[host]/path[#[option=value][&...]] | 145 | scheme://[host]/path[#[option=value][&...]] |
3515 | @@ -374,18 +162,17 @@ | |||
3516 | 374 | # We ONLY check for True here because can_handle may return a string | 162 | # We ONLY check for True here because can_handle may return a string |
3517 | 375 | # explaining why it can't handle a given source. | 163 | # explaining why it can't handle a given source. |
3518 | 376 | handlers = [h for h in plugins() if h.can_handle(source) is True] | 164 | handlers = [h for h in plugins() if h.can_handle(source) is True] |
3519 | 377 | installed_to = None | ||
3520 | 378 | for handler in handlers: | 165 | for handler in handlers: |
3521 | 379 | try: | 166 | try: |
3528 | 380 | installed_to = handler.install(source, *args, **kwargs) | 167 | return handler.install(source, *args, **kwargs) |
3529 | 381 | except UnhandledSource: | 168 | except UnhandledSource as e: |
3530 | 382 | pass | 169 | log('Install source attempt unsuccessful: {}'.format(e), |
3531 | 383 | if not installed_to: | 170 | level='WARNING') |
3532 | 384 | raise UnhandledSource("No handler found for source {}".format(source)) | 171 | raise UnhandledSource("No handler found for source {}".format(source)) |
3527 | 385 | return installed_to | ||
3533 | 386 | 172 | ||
3534 | 387 | 173 | ||
3535 | 388 | def install_from_config(config_var_name): | 174 | def install_from_config(config_var_name): |
3536 | 175 | """Install a file from config.""" | ||
3537 | 389 | charm_config = config() | 176 | charm_config = config() |
3538 | 390 | source = charm_config[config_var_name] | 177 | source = charm_config[config_var_name] |
3539 | 391 | return install_remote(source) | 178 | return install_remote(source) |
3540 | @@ -402,46 +189,9 @@ | |||
3541 | 402 | importlib.import_module(package), | 189 | importlib.import_module(package), |
3542 | 403 | classname) | 190 | classname) |
3543 | 404 | plugin_list.append(handler_class()) | 191 | plugin_list.append(handler_class()) |
3545 | 405 | except (ImportError, AttributeError): | 192 | except NotImplementedError: |
3546 | 406 | # Skip missing plugins so that they can be ommitted from | 193 | # Skip missing plugins so that they can be ommitted from |
3547 | 407 | # installation if desired | 194 | # installation if desired |
3548 | 408 | log("FetchHandler {} not found, skipping plugin".format( | 195 | log("FetchHandler {} not found, skipping plugin".format( |
3549 | 409 | handler_name)) | 196 | handler_name)) |
3550 | 410 | return plugin_list | 197 | return plugin_list |
3551 | 411 | |||
3552 | 412 | |||
3553 | 413 | def _run_apt_command(cmd, fatal=False): | ||
3554 | 414 | """ | ||
3555 | 415 | Run an APT command, checking output and retrying if the fatal flag is set | ||
3556 | 416 | to True. | ||
3557 | 417 | |||
3558 | 418 | :param: cmd: str: The apt command to run. | ||
3559 | 419 | :param: fatal: bool: Whether the command's output should be checked and | ||
3560 | 420 | retried. | ||
3561 | 421 | """ | ||
3562 | 422 | env = os.environ.copy() | ||
3563 | 423 | |||
3564 | 424 | if 'DEBIAN_FRONTEND' not in env: | ||
3565 | 425 | env['DEBIAN_FRONTEND'] = 'noninteractive' | ||
3566 | 426 | |||
3567 | 427 | if fatal: | ||
3568 | 428 | retry_count = 0 | ||
3569 | 429 | result = None | ||
3570 | 430 | |||
3571 | 431 | # If the command is considered "fatal", we need to retry if the apt | ||
3572 | 432 | # lock was not acquired. | ||
3573 | 433 | |||
3574 | 434 | while result is None or result == APT_NO_LOCK: | ||
3575 | 435 | try: | ||
3576 | 436 | result = subprocess.check_call(cmd, env=env) | ||
3577 | 437 | except subprocess.CalledProcessError as e: | ||
3578 | 438 | retry_count = retry_count + 1 | ||
3579 | 439 | if retry_count > APT_NO_LOCK_RETRY_COUNT: | ||
3580 | 440 | raise | ||
3581 | 441 | result = e.returncode | ||
3582 | 442 | log("Couldn't acquire DPKG lock. Will retry in {} seconds." | ||
3583 | 443 | "".format(APT_NO_LOCK_RETRY_DELAY)) | ||
3584 | 444 | time.sleep(APT_NO_LOCK_RETRY_DELAY) | ||
3585 | 445 | |||
3586 | 446 | else: | ||
3587 | 447 | subprocess.call(cmd, env=env) | ||
3588 | 448 | 198 | ||
3589 | === modified file 'hooks/charmhelpers/fetch/archiveurl.py' | |||
3590 | --- hooks/charmhelpers/fetch/archiveurl.py 2015-05-14 10:48:09 +0000 | |||
3591 | +++ hooks/charmhelpers/fetch/archiveurl.py 2018-06-12 20:07:13 +0000 | |||
3592 | @@ -1,18 +1,16 @@ | |||
3593 | 1 | # Copyright 2014-2015 Canonical Limited. | 1 | # Copyright 2014-2015 Canonical Limited. |
3594 | 2 | # | 2 | # |
3608 | 3 | # This file is part of charm-helpers. | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); |
3609 | 4 | # | 4 | # you may not use this file except in compliance with the License. |
3610 | 5 | # charm-helpers is free software: you can redistribute it and/or modify | 5 | # You may obtain a copy of the License at |
3611 | 6 | # it under the terms of the GNU Lesser General Public License version 3 as | 6 | # |
3612 | 7 | # published by the Free Software Foundation. | 7 | # http://www.apache.org/licenses/LICENSE-2.0 |
3613 | 8 | # | 8 | # |
3614 | 9 | # charm-helpers is distributed in the hope that it will be useful, | 9 | # Unless required by applicable law or agreed to in writing, software |
3615 | 10 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 10 | # distributed under the License is distributed on an "AS IS" BASIS, |
3616 | 11 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
3617 | 12 | # GNU Lesser General Public License for more details. | 12 | # See the License for the specific language governing permissions and |
3618 | 13 | # | 13 | # limitations under the License. |
3606 | 14 | # You should have received a copy of the GNU Lesser General Public License | ||
3607 | 15 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
3619 | 16 | 14 | ||
3620 | 17 | import os | 15 | import os |
3621 | 18 | import hashlib | 16 | import hashlib |
3622 | @@ -77,6 +75,8 @@ | |||
3623 | 77 | def can_handle(self, source): | 75 | def can_handle(self, source): |
3624 | 78 | url_parts = self.parse_url(source) | 76 | url_parts = self.parse_url(source) |
3625 | 79 | if url_parts.scheme not in ('http', 'https', 'ftp', 'file'): | 77 | if url_parts.scheme not in ('http', 'https', 'ftp', 'file'): |
3626 | 78 | # XXX: Why is this returning a boolean and a string? It's | ||
3627 | 79 | # doomed to fail since "bool(can_handle('foo://'))" will be True. | ||
3628 | 80 | return "Wrong source type" | 80 | return "Wrong source type" |
3629 | 81 | if get_archive_handler(self.base_url(source)): | 81 | if get_archive_handler(self.base_url(source)): |
3630 | 82 | return True | 82 | return True |
3631 | @@ -106,7 +106,7 @@ | |||
3632 | 106 | install_opener(opener) | 106 | install_opener(opener) |
3633 | 107 | response = urlopen(source) | 107 | response = urlopen(source) |
3634 | 108 | try: | 108 | try: |
3636 | 109 | with open(dest, 'w') as dest_file: | 109 | with open(dest, 'wb') as dest_file: |
3637 | 110 | dest_file.write(response.read()) | 110 | dest_file.write(response.read()) |
3638 | 111 | except Exception as e: | 111 | except Exception as e: |
3639 | 112 | if os.path.isfile(dest): | 112 | if os.path.isfile(dest): |
3640 | @@ -155,7 +155,11 @@ | |||
3641 | 155 | else: | 155 | else: |
3642 | 156 | algorithms = hashlib.algorithms_available | 156 | algorithms = hashlib.algorithms_available |
3643 | 157 | if key in algorithms: | 157 | if key in algorithms: |
3645 | 158 | check_hash(dld_file, value, key) | 158 | if len(value) != 1: |
3646 | 159 | raise TypeError( | ||
3647 | 160 | "Expected 1 hash value, not %d" % len(value)) | ||
3648 | 161 | expected = value[0] | ||
3649 | 162 | check_hash(dld_file, expected, key) | ||
3650 | 159 | if checksum: | 163 | if checksum: |
3651 | 160 | check_hash(dld_file, checksum, hash_type) | 164 | check_hash(dld_file, checksum, hash_type) |
3652 | 161 | return extract(dld_file, dest) | 165 | return extract(dld_file, dest) |
3653 | 162 | 166 | ||
3654 | === modified file 'hooks/charmhelpers/fetch/bzrurl.py' | |||
3655 | --- hooks/charmhelpers/fetch/bzrurl.py 2015-02-09 12:53:57 +0000 | |||
3656 | +++ hooks/charmhelpers/fetch/bzrurl.py 2018-06-12 20:07:13 +0000 | |||
3657 | @@ -1,78 +1,76 @@ | |||
3658 | 1 | # Copyright 2014-2015 Canonical Limited. | 1 | # Copyright 2014-2015 Canonical Limited. |
3659 | 2 | # | 2 | # |
3673 | 3 | # This file is part of charm-helpers. | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); |
3674 | 4 | # | 4 | # you may not use this file except in compliance with the License. |
3675 | 5 | # charm-helpers is free software: you can redistribute it and/or modify | 5 | # You may obtain a copy of the License at |
3676 | 6 | # it under the terms of the GNU Lesser General Public License version 3 as | 6 | # |
3677 | 7 | # published by the Free Software Foundation. | 7 | # http://www.apache.org/licenses/LICENSE-2.0 |
3678 | 8 | # | 8 | # |
3679 | 9 | # charm-helpers is distributed in the hope that it will be useful, | 9 | # Unless required by applicable law or agreed to in writing, software |
3680 | 10 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 10 | # distributed under the License is distributed on an "AS IS" BASIS, |
3681 | 11 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
3682 | 12 | # GNU Lesser General Public License for more details. | 12 | # See the License for the specific language governing permissions and |
3683 | 13 | # | 13 | # limitations under the License. |
3671 | 14 | # You should have received a copy of the GNU Lesser General Public License | ||
3672 | 15 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
3684 | 16 | 14 | ||
3685 | 17 | import os | 15 | import os |
3686 | 16 | from subprocess import check_call | ||
3687 | 18 | from charmhelpers.fetch import ( | 17 | from charmhelpers.fetch import ( |
3688 | 19 | BaseFetchHandler, | 18 | BaseFetchHandler, |
3690 | 20 | UnhandledSource | 19 | UnhandledSource, |
3691 | 20 | filter_installed_packages, | ||
3692 | 21 | install, | ||
3693 | 21 | ) | 22 | ) |
3694 | 22 | from charmhelpers.core.host import mkdir | 23 | from charmhelpers.core.host import mkdir |
3695 | 23 | 24 | ||
3696 | 24 | import six | ||
3697 | 25 | if six.PY3: | ||
3698 | 26 | raise ImportError('bzrlib does not support Python3') | ||
3699 | 27 | 25 | ||
3708 | 28 | try: | 26 | if filter_installed_packages(['bzr']) != []: |
3709 | 29 | from bzrlib.branch import Branch | 27 | install(['bzr']) |
3710 | 30 | from bzrlib import bzrdir, workingtree, errors | 28 | if filter_installed_packages(['bzr']) != []: |
3711 | 31 | except ImportError: | 29 | raise NotImplementedError('Unable to install bzr') |
3704 | 32 | from charmhelpers.fetch import apt_install | ||
3705 | 33 | apt_install("python-bzrlib") | ||
3706 | 34 | from bzrlib.branch import Branch | ||
3707 | 35 | from bzrlib import bzrdir, workingtree, errors | ||
3712 | 36 | 30 | ||
3713 | 37 | 31 | ||
3714 | 38 | class BzrUrlFetchHandler(BaseFetchHandler): | 32 | class BzrUrlFetchHandler(BaseFetchHandler): |
3716 | 39 | """Handler for bazaar branches via generic and lp URLs""" | 33 | """Handler for bazaar branches via generic and lp URLs.""" |
3717 | 34 | |||
3718 | 40 | def can_handle(self, source): | 35 | def can_handle(self, source): |
3719 | 41 | url_parts = self.parse_url(source) | 36 | url_parts = self.parse_url(source) |
3721 | 42 | if url_parts.scheme not in ('bzr+ssh', 'lp'): | 37 | if url_parts.scheme not in ('bzr+ssh', 'lp', ''): |
3722 | 43 | return False | 38 | return False |
3723 | 39 | elif not url_parts.scheme: | ||
3724 | 40 | return os.path.exists(os.path.join(source, '.bzr')) | ||
3725 | 44 | else: | 41 | else: |
3726 | 45 | return True | 42 | return True |
3727 | 46 | 43 | ||
3731 | 47 | def branch(self, source, dest): | 44 | def branch(self, source, dest, revno=None): |
3729 | 48 | url_parts = self.parse_url(source) | ||
3730 | 49 | # If we use lp:branchname scheme we need to load plugins | ||
3732 | 50 | if not self.can_handle(source): | 45 | if not self.can_handle(source): |
3733 | 51 | raise UnhandledSource("Cannot handle {}".format(source)) | 46 | raise UnhandledSource("Cannot handle {}".format(source)) |
3748 | 52 | if url_parts.scheme == "lp": | 47 | cmd_opts = [] |
3749 | 53 | from bzrlib.plugin import load_plugins | 48 | if revno: |
3750 | 54 | load_plugins() | 49 | cmd_opts += ['-r', str(revno)] |
3751 | 55 | try: | 50 | if os.path.exists(dest): |
3752 | 56 | local_branch = bzrdir.BzrDir.create_branch_convenience(dest) | 51 | cmd = ['bzr', 'pull'] |
3753 | 57 | except errors.AlreadyControlDirError: | 52 | cmd += cmd_opts |
3754 | 58 | local_branch = Branch.open(dest) | 53 | cmd += ['--overwrite', '-d', dest, source] |
3755 | 59 | try: | 54 | else: |
3756 | 60 | remote_branch = Branch.open(source) | 55 | cmd = ['bzr', 'branch'] |
3757 | 61 | remote_branch.push(local_branch) | 56 | cmd += cmd_opts |
3758 | 62 | tree = workingtree.WorkingTree.open(dest) | 57 | cmd += [source, dest] |
3759 | 63 | tree.update() | 58 | check_call(cmd) |
3746 | 64 | except Exception as e: | ||
3747 | 65 | raise e | ||
3760 | 66 | 59 | ||
3762 | 67 | def install(self, source): | 60 | def install(self, source, dest=None, revno=None): |
3763 | 68 | url_parts = self.parse_url(source) | 61 | url_parts = self.parse_url(source) |
3764 | 69 | branch_name = url_parts.path.strip("/").split("/")[-1] | 62 | branch_name = url_parts.path.strip("/").split("/")[-1] |
3769 | 70 | dest_dir = os.path.join(os.environ.get('CHARM_DIR'), "fetched", | 63 | if dest: |
3770 | 71 | branch_name) | 64 | dest_dir = os.path.join(dest, branch_name) |
3771 | 72 | if not os.path.exists(dest_dir): | 65 | else: |
3772 | 73 | mkdir(dest_dir, perms=0o755) | 66 | dest_dir = os.path.join(os.environ.get('CHARM_DIR'), "fetched", |
3773 | 67 | branch_name) | ||
3774 | 68 | |||
3775 | 69 | if dest and not os.path.exists(dest): | ||
3776 | 70 | mkdir(dest, perms=0o755) | ||
3777 | 71 | |||
3778 | 74 | try: | 72 | try: |
3780 | 75 | self.branch(source, dest_dir) | 73 | self.branch(source, dest_dir, revno) |
3781 | 76 | except OSError as e: | 74 | except OSError as e: |
3782 | 77 | raise UnhandledSource(e.strerror) | 75 | raise UnhandledSource(e.strerror) |
3783 | 78 | return dest_dir | 76 | return dest_dir |
3784 | 79 | 77 | ||
3785 | === added file 'hooks/charmhelpers/fetch/centos.py' | |||
3786 | --- hooks/charmhelpers/fetch/centos.py 1970-01-01 00:00:00 +0000 | |||
3787 | +++ hooks/charmhelpers/fetch/centos.py 2018-06-12 20:07:13 +0000 | |||
3788 | @@ -0,0 +1,171 @@ | |||
3789 | 1 | # Copyright 2014-2015 Canonical Limited. | ||
3790 | 2 | # | ||
3791 | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); | ||
3792 | 4 | # you may not use this file except in compliance with the License. | ||
3793 | 5 | # You may obtain a copy of the License at | ||
3794 | 6 | # | ||
3795 | 7 | # http://www.apache.org/licenses/LICENSE-2.0 | ||
3796 | 8 | # | ||
3797 | 9 | # Unless required by applicable law or agreed to in writing, software | ||
3798 | 10 | # distributed under the License is distributed on an "AS IS" BASIS, | ||
3799 | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
3800 | 12 | # See the License for the specific language governing permissions and | ||
3801 | 13 | # limitations under the License. | ||
3802 | 14 | |||
3803 | 15 | import subprocess | ||
3804 | 16 | import os | ||
3805 | 17 | import time | ||
3806 | 18 | import six | ||
3807 | 19 | import yum | ||
3808 | 20 | |||
3809 | 21 | from tempfile import NamedTemporaryFile | ||
3810 | 22 | from charmhelpers.core.hookenv import log | ||
3811 | 23 | |||
3812 | 24 | YUM_NO_LOCK = 1 # The return code for "couldn't acquire lock" in YUM. | ||
3813 | 25 | YUM_NO_LOCK_RETRY_DELAY = 10 # Wait 10 seconds between apt lock checks. | ||
3814 | 26 | YUM_NO_LOCK_RETRY_COUNT = 30 # Retry to acquire the lock X times. | ||
3815 | 27 | |||
3816 | 28 | |||
3817 | 29 | def filter_installed_packages(packages): | ||
3818 | 30 | """Return a list of packages that require installation.""" | ||
3819 | 31 | yb = yum.YumBase() | ||
3820 | 32 | package_list = yb.doPackageLists() | ||
3821 | 33 | temp_cache = {p.base_package_name: 1 for p in package_list['installed']} | ||
3822 | 34 | |||
3823 | 35 | _pkgs = [p for p in packages if not temp_cache.get(p, False)] | ||
3824 | 36 | return _pkgs | ||
3825 | 37 | |||
3826 | 38 | |||
3827 | 39 | def install(packages, options=None, fatal=False): | ||
3828 | 40 | """Install one or more packages.""" | ||
3829 | 41 | cmd = ['yum', '--assumeyes'] | ||
3830 | 42 | if options is not None: | ||
3831 | 43 | cmd.extend(options) | ||
3832 | 44 | cmd.append('install') | ||
3833 | 45 | if isinstance(packages, six.string_types): | ||
3834 | 46 | cmd.append(packages) | ||
3835 | 47 | else: | ||
3836 | 48 | cmd.extend(packages) | ||
3837 | 49 | log("Installing {} with options: {}".format(packages, | ||
3838 | 50 | options)) | ||
3839 | 51 | _run_yum_command(cmd, fatal) | ||
3840 | 52 | |||
3841 | 53 | |||
3842 | 54 | def upgrade(options=None, fatal=False, dist=False): | ||
3843 | 55 | """Upgrade all packages.""" | ||
3844 | 56 | cmd = ['yum', '--assumeyes'] | ||
3845 | 57 | if options is not None: | ||
3846 | 58 | cmd.extend(options) | ||
3847 | 59 | cmd.append('upgrade') | ||
3848 | 60 | log("Upgrading with options: {}".format(options)) | ||
3849 | 61 | _run_yum_command(cmd, fatal) | ||
3850 | 62 | |||
3851 | 63 | |||
3852 | 64 | def update(fatal=False): | ||
3853 | 65 | """Update local yum cache.""" | ||
3854 | 66 | cmd = ['yum', '--assumeyes', 'update'] | ||
3855 | 67 | log("Update with fatal: {}".format(fatal)) | ||
3856 | 68 | _run_yum_command(cmd, fatal) | ||
3857 | 69 | |||
3858 | 70 | |||
3859 | 71 | def purge(packages, fatal=False): | ||
3860 | 72 | """Purge one or more packages.""" | ||
3861 | 73 | cmd = ['yum', '--assumeyes', 'remove'] | ||
3862 | 74 | if isinstance(packages, six.string_types): | ||
3863 | 75 | cmd.append(packages) | ||
3864 | 76 | else: | ||
3865 | 77 | cmd.extend(packages) | ||
3866 | 78 | log("Purging {}".format(packages)) | ||
3867 | 79 | _run_yum_command(cmd, fatal) | ||
3868 | 80 | |||
3869 | 81 | |||
3870 | 82 | def yum_search(packages): | ||
3871 | 83 | """Search for a package.""" | ||
3872 | 84 | output = {} | ||
3873 | 85 | cmd = ['yum', 'search'] | ||
3874 | 86 | if isinstance(packages, six.string_types): | ||
3875 | 87 | cmd.append(packages) | ||
3876 | 88 | else: | ||
3877 | 89 | cmd.extend(packages) | ||
3878 | 90 | log("Searching for {}".format(packages)) | ||
3879 | 91 | result = subprocess.check_output(cmd) | ||
3880 | 92 | for package in list(packages): | ||
3881 | 93 | output[package] = package in result | ||
3882 | 94 | return output | ||
3883 | 95 | |||
3884 | 96 | |||
3885 | 97 | def add_source(source, key=None): | ||
3886 | 98 | """Add a package source to this system. | ||
3887 | 99 | |||
3888 | 100 | @param source: a URL with a rpm package | ||
3889 | 101 | |||
3890 | 102 | @param key: A key to be added to the system's keyring and used | ||
3891 | 103 | to verify the signatures on packages. Ideally, this should be an | ||
3892 | 104 | ASCII format GPG public key including the block headers. A GPG key | ||
3893 | 105 | id may also be used, but be aware that only insecure protocols are | ||
3894 | 106 | available to retrieve the actual public key from a public keyserver | ||
3895 | 107 | placing your Juju environment at risk. | ||
3896 | 108 | """ | ||
3897 | 109 | if source is None: | ||
3898 | 110 | log('Source is not present. Skipping') | ||
3899 | 111 | return | ||
3900 | 112 | |||
3901 | 113 | if source.startswith('http'): | ||
3902 | 114 | directory = '/etc/yum.repos.d/' | ||
3903 | 115 | for filename in os.listdir(directory): | ||
3904 | 116 | with open(directory + filename, 'r') as rpm_file: | ||
3905 | 117 | if source in rpm_file.read(): | ||
3906 | 118 | break | ||
3907 | 119 | else: | ||
3908 | 120 | log("Add source: {!r}".format(source)) | ||
3909 | 121 | # write in the charms.repo | ||
3910 | 122 | with open(directory + 'Charms.repo', 'a') as rpm_file: | ||
3911 | 123 | rpm_file.write('[%s]\n' % source[7:].replace('/', '_')) | ||
3912 | 124 | rpm_file.write('name=%s\n' % source[7:]) | ||
3913 | 125 | rpm_file.write('baseurl=%s\n\n' % source) | ||
3914 | 126 | else: | ||
3915 | 127 | log("Unknown source: {!r}".format(source)) | ||
3916 | 128 | |||
3917 | 129 | if key: | ||
3918 | 130 | if '-----BEGIN PGP PUBLIC KEY BLOCK-----' in key: | ||
3919 | 131 | with NamedTemporaryFile('w+') as key_file: | ||
3920 | 132 | key_file.write(key) | ||
3921 | 133 | key_file.flush() | ||
3922 | 134 | key_file.seek(0) | ||
3923 | 135 | subprocess.check_call(['rpm', '--import', key_file]) | ||
3924 | 136 | else: | ||
3925 | 137 | subprocess.check_call(['rpm', '--import', key]) | ||
3926 | 138 | |||
3927 | 139 | |||
3928 | 140 | def _run_yum_command(cmd, fatal=False): | ||
3929 | 141 | """Run an YUM command. | ||
3930 | 142 | |||
3931 | 143 | Checks the output and retry if the fatal flag is set to True. | ||
3932 | 144 | |||
3933 | 145 | :param: cmd: str: The yum command to run. | ||
3934 | 146 | :param: fatal: bool: Whether the command's output should be checked and | ||
3935 | 147 | retried. | ||
3936 | 148 | """ | ||
3937 | 149 | env = os.environ.copy() | ||
3938 | 150 | |||
3939 | 151 | if fatal: | ||
3940 | 152 | retry_count = 0 | ||
3941 | 153 | result = None | ||
3942 | 154 | |||
3943 | 155 | # If the command is considered "fatal", we need to retry if the yum | ||
3944 | 156 | # lock was not acquired. | ||
3945 | 157 | |||
3946 | 158 | while result is None or result == YUM_NO_LOCK: | ||
3947 | 159 | try: | ||
3948 | 160 | result = subprocess.check_call(cmd, env=env) | ||
3949 | 161 | except subprocess.CalledProcessError as e: | ||
3950 | 162 | retry_count = retry_count + 1 | ||
3951 | 163 | if retry_count > YUM_NO_LOCK_RETRY_COUNT: | ||
3952 | 164 | raise | ||
3953 | 165 | result = e.returncode | ||
3954 | 166 | log("Couldn't acquire YUM lock. Will retry in {} seconds." | ||
3955 | 167 | "".format(YUM_NO_LOCK_RETRY_DELAY)) | ||
3956 | 168 | time.sleep(YUM_NO_LOCK_RETRY_DELAY) | ||
3957 | 169 | |||
3958 | 170 | else: | ||
3959 | 171 | subprocess.call(cmd, env=env) | ||
3960 | 0 | 172 | ||
3961 | === modified file 'hooks/charmhelpers/fetch/giturl.py' | |||
3962 | --- hooks/charmhelpers/fetch/giturl.py 2015-05-14 10:48:09 +0000 | |||
3963 | +++ hooks/charmhelpers/fetch/giturl.py 2018-06-12 20:07:13 +0000 | |||
3964 | @@ -1,58 +1,58 @@ | |||
3965 | 1 | # Copyright 2014-2015 Canonical Limited. | 1 | # Copyright 2014-2015 Canonical Limited. |
3966 | 2 | # | 2 | # |
3980 | 3 | # This file is part of charm-helpers. | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); |
3981 | 4 | # | 4 | # you may not use this file except in compliance with the License. |
3982 | 5 | # charm-helpers is free software: you can redistribute it and/or modify | 5 | # You may obtain a copy of the License at |
3983 | 6 | # it under the terms of the GNU Lesser General Public License version 3 as | 6 | # |
3984 | 7 | # published by the Free Software Foundation. | 7 | # http://www.apache.org/licenses/LICENSE-2.0 |
3985 | 8 | # | 8 | # |
3986 | 9 | # charm-helpers is distributed in the hope that it will be useful, | 9 | # Unless required by applicable law or agreed to in writing, software |
3987 | 10 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 10 | # distributed under the License is distributed on an "AS IS" BASIS, |
3988 | 11 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
3989 | 12 | # GNU Lesser General Public License for more details. | 12 | # See the License for the specific language governing permissions and |
3990 | 13 | # | 13 | # limitations under the License. |
3978 | 14 | # You should have received a copy of the GNU Lesser General Public License | ||
3979 | 15 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
3991 | 16 | 14 | ||
3992 | 17 | import os | 15 | import os |
3993 | 16 | from subprocess import check_call, CalledProcessError | ||
3994 | 18 | from charmhelpers.fetch import ( | 17 | from charmhelpers.fetch import ( |
3995 | 19 | BaseFetchHandler, | 18 | BaseFetchHandler, |
3997 | 20 | UnhandledSource | 19 | UnhandledSource, |
3998 | 20 | filter_installed_packages, | ||
3999 | 21 | install, | ||
4000 | 21 | ) | 22 | ) |
4015 | 22 | from charmhelpers.core.host import mkdir | 23 | |
4016 | 23 | 24 | if filter_installed_packages(['git']) != []: | |
4017 | 24 | import six | 25 | install(['git']) |
4018 | 25 | if six.PY3: | 26 | if filter_installed_packages(['git']) != []: |
4019 | 26 | raise ImportError('GitPython does not support Python 3') | 27 | raise NotImplementedError('Unable to install git') |
4006 | 27 | |||
4007 | 28 | try: | ||
4008 | 29 | from git import Repo | ||
4009 | 30 | except ImportError: | ||
4010 | 31 | from charmhelpers.fetch import apt_install | ||
4011 | 32 | apt_install("python-git") | ||
4012 | 33 | from git import Repo | ||
4013 | 34 | |||
4014 | 35 | from git.exc import GitCommandError # noqa E402 | ||
4020 | 36 | 28 | ||
4021 | 37 | 29 | ||
4022 | 38 | class GitUrlFetchHandler(BaseFetchHandler): | 30 | class GitUrlFetchHandler(BaseFetchHandler): |
4024 | 39 | """Handler for git branches via generic and github URLs""" | 31 | """Handler for git branches via generic and github URLs.""" |
4025 | 32 | |||
4026 | 40 | def can_handle(self, source): | 33 | def can_handle(self, source): |
4027 | 41 | url_parts = self.parse_url(source) | 34 | url_parts = self.parse_url(source) |
4028 | 42 | # TODO (mattyw) no support for ssh git@ yet | 35 | # TODO (mattyw) no support for ssh git@ yet |
4030 | 43 | if url_parts.scheme not in ('http', 'https', 'git'): | 36 | if url_parts.scheme not in ('http', 'https', 'git', ''): |
4031 | 44 | return False | 37 | return False |
4032 | 38 | elif not url_parts.scheme: | ||
4033 | 39 | return os.path.exists(os.path.join(source, '.git')) | ||
4034 | 45 | else: | 40 | else: |
4035 | 46 | return True | 41 | return True |
4036 | 47 | 42 | ||
4038 | 48 | def clone(self, source, dest, branch): | 43 | def clone(self, source, dest, branch="master", depth=None): |
4039 | 49 | if not self.can_handle(source): | 44 | if not self.can_handle(source): |
4040 | 50 | raise UnhandledSource("Cannot handle {}".format(source)) | 45 | raise UnhandledSource("Cannot handle {}".format(source)) |
4041 | 51 | 46 | ||
4044 | 52 | repo = Repo.clone_from(source, dest) | 47 | if os.path.exists(dest): |
4045 | 53 | repo.git.checkout(branch) | 48 | cmd = ['git', '-C', dest, 'pull', source, branch] |
4046 | 49 | else: | ||
4047 | 50 | cmd = ['git', 'clone', source, dest, '--branch', branch] | ||
4048 | 51 | if depth: | ||
4049 | 52 | cmd.extend(['--depth', depth]) | ||
4050 | 53 | check_call(cmd) | ||
4051 | 54 | 54 | ||
4053 | 55 | def install(self, source, branch="master", dest=None): | 55 | def install(self, source, branch="master", dest=None, depth=None): |
4054 | 56 | url_parts = self.parse_url(source) | 56 | url_parts = self.parse_url(source) |
4055 | 57 | branch_name = url_parts.path.strip("/").split("/")[-1] | 57 | branch_name = url_parts.path.strip("/").split("/")[-1] |
4056 | 58 | if dest: | 58 | if dest: |
4057 | @@ -60,12 +60,10 @@ | |||
4058 | 60 | else: | 60 | else: |
4059 | 61 | dest_dir = os.path.join(os.environ.get('CHARM_DIR'), "fetched", | 61 | dest_dir = os.path.join(os.environ.get('CHARM_DIR'), "fetched", |
4060 | 62 | branch_name) | 62 | branch_name) |
4061 | 63 | if not os.path.exists(dest_dir): | ||
4062 | 64 | mkdir(dest_dir, perms=0o755) | ||
4063 | 65 | try: | 63 | try: |
4067 | 66 | self.clone(source, dest_dir, branch) | 64 | self.clone(source, dest_dir, branch, depth) |
4068 | 67 | except GitCommandError as e: | 65 | except CalledProcessError as e: |
4069 | 68 | raise UnhandledSource(e.message) | 66 | raise UnhandledSource(e) |
4070 | 69 | except OSError as e: | 67 | except OSError as e: |
4071 | 70 | raise UnhandledSource(e.strerror) | 68 | raise UnhandledSource(e.strerror) |
4072 | 71 | return dest_dir | 69 | return dest_dir |
4073 | 72 | 70 | ||
4074 | === added file 'hooks/charmhelpers/fetch/snap.py' | |||
4075 | --- hooks/charmhelpers/fetch/snap.py 1970-01-01 00:00:00 +0000 | |||
4076 | +++ hooks/charmhelpers/fetch/snap.py 2018-06-12 20:07:13 +0000 | |||
4077 | @@ -0,0 +1,122 @@ | |||
4078 | 1 | # Copyright 2014-2017 Canonical Limited. | ||
4079 | 2 | # | ||
4080 | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); | ||
4081 | 4 | # you may not use this file except in compliance with the License. | ||
4082 | 5 | # You may obtain a copy of the License at | ||
4083 | 6 | # | ||
4084 | 7 | # http://www.apache.org/licenses/LICENSE-2.0 | ||
4085 | 8 | # | ||
4086 | 9 | # Unless required by applicable law or agreed to in writing, software | ||
4087 | 10 | # distributed under the License is distributed on an "AS IS" BASIS, | ||
4088 | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
4089 | 12 | # See the License for the specific language governing permissions and | ||
4090 | 13 | # limitations under the License. | ||
4091 | 14 | """ | ||
4092 | 15 | Charm helpers snap for classic charms. | ||
4093 | 16 | |||
4094 | 17 | If writing reactive charms, use the snap layer: | ||
4095 | 18 | https://lists.ubuntu.com/archives/snapcraft/2016-September/001114.html | ||
4096 | 19 | """ | ||
4097 | 20 | import subprocess | ||
4098 | 21 | from os import environ | ||
4099 | 22 | from time import sleep | ||
4100 | 23 | from charmhelpers.core.hookenv import log | ||
4101 | 24 | |||
4102 | 25 | __author__ = 'Joseph Borg <joseph.borg@canonical.com>' | ||
4103 | 26 | |||
4104 | 27 | SNAP_NO_LOCK = 1 # The return code for "couldn't acquire lock" in Snap (hopefully this will be improved). | ||
4105 | 28 | SNAP_NO_LOCK_RETRY_DELAY = 10 # Wait X seconds between Snap lock checks. | ||
4106 | 29 | SNAP_NO_LOCK_RETRY_COUNT = 30 # Retry to acquire the lock X times. | ||
4107 | 30 | |||
4108 | 31 | |||
4109 | 32 | class CouldNotAcquireLockException(Exception): | ||
4110 | 33 | pass | ||
4111 | 34 | |||
4112 | 35 | |||
4113 | 36 | def _snap_exec(commands): | ||
4114 | 37 | """ | ||
4115 | 38 | Execute snap commands. | ||
4116 | 39 | |||
4117 | 40 | :param commands: List commands | ||
4118 | 41 | :return: Integer exit code | ||
4119 | 42 | """ | ||
4120 | 43 | assert type(commands) == list | ||
4121 | 44 | |||
4122 | 45 | retry_count = 0 | ||
4123 | 46 | return_code = None | ||
4124 | 47 | |||
4125 | 48 | while return_code is None or return_code == SNAP_NO_LOCK: | ||
4126 | 49 | try: | ||
4127 | 50 | return_code = subprocess.check_call(['snap'] + commands, env=environ) | ||
4128 | 51 | except subprocess.CalledProcessError as e: | ||
4129 | 52 | retry_count += + 1 | ||
4130 | 53 | if retry_count > SNAP_NO_LOCK_RETRY_COUNT: | ||
4131 | 54 | raise CouldNotAcquireLockException('Could not aquire lock after %s attempts' % SNAP_NO_LOCK_RETRY_COUNT) | ||
4132 | 55 | return_code = e.returncode | ||
4133 | 56 | log('Snap failed to acquire lock, trying again in %s seconds.' % SNAP_NO_LOCK_RETRY_DELAY, level='WARN') | ||
4134 | 57 | sleep(SNAP_NO_LOCK_RETRY_DELAY) | ||
4135 | 58 | |||
4136 | 59 | return return_code | ||
4137 | 60 | |||
4138 | 61 | |||
4139 | 62 | def snap_install(packages, *flags): | ||
4140 | 63 | """ | ||
4141 | 64 | Install a snap package. | ||
4142 | 65 | |||
4143 | 66 | :param packages: String or List String package name | ||
4144 | 67 | :param flags: List String flags to pass to install command | ||
4145 | 68 | :return: Integer return code from snap | ||
4146 | 69 | """ | ||
4147 | 70 | if type(packages) is not list: | ||
4148 | 71 | packages = [packages] | ||
4149 | 72 | |||
4150 | 73 | flags = list(flags) | ||
4151 | 74 | |||
4152 | 75 | message = 'Installing snap(s) "%s"' % ', '.join(packages) | ||
4153 | 76 | if flags: | ||
4154 | 77 | message += ' with option(s) "%s"' % ', '.join(flags) | ||
4155 | 78 | |||
4156 | 79 | log(message, level='INFO') | ||
4157 | 80 | return _snap_exec(['install'] + flags + packages) | ||
4158 | 81 | |||
4159 | 82 | |||
4160 | 83 | def snap_remove(packages, *flags): | ||
4161 | 84 | """ | ||
4162 | 85 | Remove a snap package. | ||
4163 | 86 | |||
4164 | 87 | :param packages: String or List String package name | ||
4165 | 88 | :param flags: List String flags to pass to remove command | ||
4166 | 89 | :return: Integer return code from snap | ||
4167 | 90 | """ | ||
4168 | 91 | if type(packages) is not list: | ||
4169 | 92 | packages = [packages] | ||
4170 | 93 | |||
4171 | 94 | flags = list(flags) | ||
4172 | 95 | |||
4173 | 96 | message = 'Removing snap(s) "%s"' % ', '.join(packages) | ||
4174 | 97 | if flags: | ||
4175 | 98 | message += ' with options "%s"' % ', '.join(flags) | ||
4176 | 99 | |||
4177 | 100 | log(message, level='INFO') | ||
4178 | 101 | return _snap_exec(['remove'] + flags + packages) | ||
4179 | 102 | |||
4180 | 103 | |||
4181 | 104 | def snap_refresh(packages, *flags): | ||
4182 | 105 | """ | ||
4183 | 106 | Refresh / Update snap package. | ||
4184 | 107 | |||
4185 | 108 | :param packages: String or List String package name | ||
4186 | 109 | :param flags: List String flags to pass to refresh command | ||
4187 | 110 | :return: Integer return code from snap | ||
4188 | 111 | """ | ||
4189 | 112 | if type(packages) is not list: | ||
4190 | 113 | packages = [packages] | ||
4191 | 114 | |||
4192 | 115 | flags = list(flags) | ||
4193 | 116 | |||
4194 | 117 | message = 'Refreshing snap(s) "%s"' % ', '.join(packages) | ||
4195 | 118 | if flags: | ||
4196 | 119 | message += ' with options "%s"' % ', '.join(flags) | ||
4197 | 120 | |||
4198 | 121 | log(message, level='INFO') | ||
4199 | 122 | return _snap_exec(['refresh'] + flags + packages) | ||
4200 | 0 | 123 | ||
4201 | === added file 'hooks/charmhelpers/fetch/ubuntu.py' | |||
4202 | --- hooks/charmhelpers/fetch/ubuntu.py 1970-01-01 00:00:00 +0000 | |||
4203 | +++ hooks/charmhelpers/fetch/ubuntu.py 2018-06-12 20:07:13 +0000 | |||
4204 | @@ -0,0 +1,364 @@ | |||
4205 | 1 | # Copyright 2014-2015 Canonical Limited. | ||
4206 | 2 | # | ||
4207 | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); | ||
4208 | 4 | # you may not use this file except in compliance with the License. | ||
4209 | 5 | # You may obtain a copy of the License at | ||
4210 | 6 | # | ||
4211 | 7 | # http://www.apache.org/licenses/LICENSE-2.0 | ||
4212 | 8 | # | ||
4213 | 9 | # Unless required by applicable law or agreed to in writing, software | ||
4214 | 10 | # distributed under the License is distributed on an "AS IS" BASIS, | ||
4215 | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
4216 | 12 | # See the License for the specific language governing permissions and | ||
4217 | 13 | # limitations under the License. | ||
4218 | 14 | |||
4219 | 15 | import os | ||
4220 | 16 | import six | ||
4221 | 17 | import time | ||
4222 | 18 | import subprocess | ||
4223 | 19 | |||
4224 | 20 | from tempfile import NamedTemporaryFile | ||
4225 | 21 | from charmhelpers.core.host import ( | ||
4226 | 22 | lsb_release | ||
4227 | 23 | ) | ||
4228 | 24 | from charmhelpers.core.hookenv import log | ||
4229 | 25 | from charmhelpers.fetch import SourceConfigError | ||
4230 | 26 | |||
4231 | 27 | CLOUD_ARCHIVE = """# Ubuntu Cloud Archive | ||
4232 | 28 | deb http://ubuntu-cloud.archive.canonical.com/ubuntu {} main | ||
4233 | 29 | """ | ||
4234 | 30 | |||
4235 | 31 | PROPOSED_POCKET = """# Proposed | ||
4236 | 32 | deb http://archive.ubuntu.com/ubuntu {}-proposed main universe multiverse restricted | ||
4237 | 33 | """ | ||
4238 | 34 | |||
4239 | 35 | CLOUD_ARCHIVE_POCKETS = { | ||
4240 | 36 | # Folsom | ||
4241 | 37 | 'folsom': 'precise-updates/folsom', | ||
4242 | 38 | 'precise-folsom': 'precise-updates/folsom', | ||
4243 | 39 | 'precise-folsom/updates': 'precise-updates/folsom', | ||
4244 | 40 | 'precise-updates/folsom': 'precise-updates/folsom', | ||
4245 | 41 | 'folsom/proposed': 'precise-proposed/folsom', | ||
4246 | 42 | 'precise-folsom/proposed': 'precise-proposed/folsom', | ||
4247 | 43 | 'precise-proposed/folsom': 'precise-proposed/folsom', | ||
4248 | 44 | # Grizzly | ||
4249 | 45 | 'grizzly': 'precise-updates/grizzly', | ||
4250 | 46 | 'precise-grizzly': 'precise-updates/grizzly', | ||
4251 | 47 | 'precise-grizzly/updates': 'precise-updates/grizzly', | ||
4252 | 48 | 'precise-updates/grizzly': 'precise-updates/grizzly', | ||
4253 | 49 | 'grizzly/proposed': 'precise-proposed/grizzly', | ||
4254 | 50 | 'precise-grizzly/proposed': 'precise-proposed/grizzly', | ||
4255 | 51 | 'precise-proposed/grizzly': 'precise-proposed/grizzly', | ||
4256 | 52 | # Havana | ||
4257 | 53 | 'havana': 'precise-updates/havana', | ||
4258 | 54 | 'precise-havana': 'precise-updates/havana', | ||
4259 | 55 | 'precise-havana/updates': 'precise-updates/havana', | ||
4260 | 56 | 'precise-updates/havana': 'precise-updates/havana', | ||
4261 | 57 | 'havana/proposed': 'precise-proposed/havana', | ||
4262 | 58 | 'precise-havana/proposed': 'precise-proposed/havana', | ||
4263 | 59 | 'precise-proposed/havana': 'precise-proposed/havana', | ||
4264 | 60 | # Icehouse | ||
4265 | 61 | 'icehouse': 'precise-updates/icehouse', | ||
4266 | 62 | 'precise-icehouse': 'precise-updates/icehouse', | ||
4267 | 63 | 'precise-icehouse/updates': 'precise-updates/icehouse', | ||
4268 | 64 | 'precise-updates/icehouse': 'precise-updates/icehouse', | ||
4269 | 65 | 'icehouse/proposed': 'precise-proposed/icehouse', | ||
4270 | 66 | 'precise-icehouse/proposed': 'precise-proposed/icehouse', | ||
4271 | 67 | 'precise-proposed/icehouse': 'precise-proposed/icehouse', | ||
4272 | 68 | # Juno | ||
4273 | 69 | 'juno': 'trusty-updates/juno', | ||
4274 | 70 | 'trusty-juno': 'trusty-updates/juno', | ||
4275 | 71 | 'trusty-juno/updates': 'trusty-updates/juno', | ||
4276 | 72 | 'trusty-updates/juno': 'trusty-updates/juno', | ||
4277 | 73 | 'juno/proposed': 'trusty-proposed/juno', | ||
4278 | 74 | 'trusty-juno/proposed': 'trusty-proposed/juno', | ||
4279 | 75 | 'trusty-proposed/juno': 'trusty-proposed/juno', | ||
4280 | 76 | # Kilo | ||
4281 | 77 | 'kilo': 'trusty-updates/kilo', | ||
4282 | 78 | 'trusty-kilo': 'trusty-updates/kilo', | ||
4283 | 79 | 'trusty-kilo/updates': 'trusty-updates/kilo', | ||
4284 | 80 | 'trusty-updates/kilo': 'trusty-updates/kilo', | ||
4285 | 81 | 'kilo/proposed': 'trusty-proposed/kilo', | ||
4286 | 82 | 'trusty-kilo/proposed': 'trusty-proposed/kilo', | ||
4287 | 83 | 'trusty-proposed/kilo': 'trusty-proposed/kilo', | ||
4288 | 84 | # Liberty | ||
4289 | 85 | 'liberty': 'trusty-updates/liberty', | ||
4290 | 86 | 'trusty-liberty': 'trusty-updates/liberty', | ||
4291 | 87 | 'trusty-liberty/updates': 'trusty-updates/liberty', | ||
4292 | 88 | 'trusty-updates/liberty': 'trusty-updates/liberty', | ||
4293 | 89 | 'liberty/proposed': 'trusty-proposed/liberty', | ||
4294 | 90 | 'trusty-liberty/proposed': 'trusty-proposed/liberty', | ||
4295 | 91 | 'trusty-proposed/liberty': 'trusty-proposed/liberty', | ||
4296 | 92 | # Mitaka | ||
4297 | 93 | 'mitaka': 'trusty-updates/mitaka', | ||
4298 | 94 | 'trusty-mitaka': 'trusty-updates/mitaka', | ||
4299 | 95 | 'trusty-mitaka/updates': 'trusty-updates/mitaka', | ||
4300 | 96 | 'trusty-updates/mitaka': 'trusty-updates/mitaka', | ||
4301 | 97 | 'mitaka/proposed': 'trusty-proposed/mitaka', | ||
4302 | 98 | 'trusty-mitaka/proposed': 'trusty-proposed/mitaka', | ||
4303 | 99 | 'trusty-proposed/mitaka': 'trusty-proposed/mitaka', | ||
4304 | 100 | # Newton | ||
4305 | 101 | 'newton': 'xenial-updates/newton', | ||
4306 | 102 | 'xenial-newton': 'xenial-updates/newton', | ||
4307 | 103 | 'xenial-newton/updates': 'xenial-updates/newton', | ||
4308 | 104 | 'xenial-updates/newton': 'xenial-updates/newton', | ||
4309 | 105 | 'newton/proposed': 'xenial-proposed/newton', | ||
4310 | 106 | 'xenial-newton/proposed': 'xenial-proposed/newton', | ||
4311 | 107 | 'xenial-proposed/newton': 'xenial-proposed/newton', | ||
4312 | 108 | # Ocata | ||
4313 | 109 | 'ocata': 'xenial-updates/ocata', | ||
4314 | 110 | 'xenial-ocata': 'xenial-updates/ocata', | ||
4315 | 111 | 'xenial-ocata/updates': 'xenial-updates/ocata', | ||
4316 | 112 | 'xenial-updates/ocata': 'xenial-updates/ocata', | ||
4317 | 113 | 'ocata/proposed': 'xenial-proposed/ocata', | ||
4318 | 114 | 'xenial-ocata/proposed': 'xenial-proposed/ocata', | ||
4319 | 115 | 'xenial-ocata/newton': 'xenial-proposed/ocata', | ||
4320 | 116 | } | ||
4321 | 117 | |||
4322 | 118 | APT_NO_LOCK = 100 # The return code for "couldn't acquire lock" in APT. | ||
4323 | 119 | CMD_RETRY_DELAY = 10 # Wait 10 seconds between command retries. | ||
4324 | 120 | CMD_RETRY_COUNT = 30 # Retry a failing fatal command X times. | ||
4325 | 121 | |||
4326 | 122 | |||
4327 | 123 | def filter_installed_packages(packages): | ||
4328 | 124 | """Return a list of packages that require installation.""" | ||
4329 | 125 | cache = apt_cache() | ||
4330 | 126 | _pkgs = [] | ||
4331 | 127 | for package in packages: | ||
4332 | 128 | try: | ||
4333 | 129 | p = cache[package] | ||
4334 | 130 | p.current_ver or _pkgs.append(package) | ||
4335 | 131 | except KeyError: | ||
4336 | 132 | log('Package {} has no installation candidate.'.format(package), | ||
4337 | 133 | level='WARNING') | ||
4338 | 134 | _pkgs.append(package) | ||
4339 | 135 | return _pkgs | ||
4340 | 136 | |||
4341 | 137 | |||
4342 | 138 | def apt_cache(in_memory=True, progress=None): | ||
4343 | 139 | """Build and return an apt cache.""" | ||
4344 | 140 | from apt import apt_pkg | ||
4345 | 141 | apt_pkg.init() | ||
4346 | 142 | if in_memory: | ||
4347 | 143 | apt_pkg.config.set("Dir::Cache::pkgcache", "") | ||
4348 | 144 | apt_pkg.config.set("Dir::Cache::srcpkgcache", "") | ||
4349 | 145 | return apt_pkg.Cache(progress) | ||
4350 | 146 | |||
4351 | 147 | |||
4352 | 148 | def install(packages, options=None, fatal=False): | ||
4353 | 149 | """Install one or more packages.""" | ||
4354 | 150 | if options is None: | ||
4355 | 151 | options = ['--option=Dpkg::Options::=--force-confold'] | ||
4356 | 152 | |||
4357 | 153 | cmd = ['apt-get', '--assume-yes'] | ||
4358 | 154 | cmd.extend(options) | ||
4359 | 155 | cmd.append('install') | ||
4360 | 156 | if isinstance(packages, six.string_types): | ||
4361 | 157 | cmd.append(packages) | ||
4362 | 158 | else: | ||
4363 | 159 | cmd.extend(packages) | ||
4364 | 160 | log("Installing {} with options: {}".format(packages, | ||
4365 | 161 | options)) | ||
4366 | 162 | _run_apt_command(cmd, fatal) | ||
4367 | 163 | |||
4368 | 164 | |||
4369 | 165 | def upgrade(options=None, fatal=False, dist=False): | ||
4370 | 166 | """Upgrade all packages.""" | ||
4371 | 167 | if options is None: | ||
4372 | 168 | options = ['--option=Dpkg::Options::=--force-confold'] | ||
4373 | 169 | |||
4374 | 170 | cmd = ['apt-get', '--assume-yes'] | ||
4375 | 171 | cmd.extend(options) | ||
4376 | 172 | if dist: | ||
4377 | 173 | cmd.append('dist-upgrade') | ||
4378 | 174 | else: | ||
4379 | 175 | cmd.append('upgrade') | ||
4380 | 176 | log("Upgrading with options: {}".format(options)) | ||
4381 | 177 | _run_apt_command(cmd, fatal) | ||
4382 | 178 | |||
4383 | 179 | |||
4384 | 180 | def update(fatal=False): | ||
4385 | 181 | """Update local apt cache.""" | ||
4386 | 182 | cmd = ['apt-get', 'update'] | ||
4387 | 183 | _run_apt_command(cmd, fatal) | ||
4388 | 184 | |||
4389 | 185 | |||
4390 | 186 | def purge(packages, fatal=False): | ||
4391 | 187 | """Purge one or more packages.""" | ||
4392 | 188 | cmd = ['apt-get', '--assume-yes', 'purge'] | ||
4393 | 189 | if isinstance(packages, six.string_types): | ||
4394 | 190 | cmd.append(packages) | ||
4395 | 191 | else: | ||
4396 | 192 | cmd.extend(packages) | ||
4397 | 193 | log("Purging {}".format(packages)) | ||
4398 | 194 | _run_apt_command(cmd, fatal) | ||
4399 | 195 | |||
4400 | 196 | |||
4401 | 197 | def apt_mark(packages, mark, fatal=False): | ||
4402 | 198 | """Flag one or more packages using apt-mark.""" | ||
4403 | 199 | log("Marking {} as {}".format(packages, mark)) | ||
4404 | 200 | cmd = ['apt-mark', mark] | ||
4405 | 201 | if isinstance(packages, six.string_types): | ||
4406 | 202 | cmd.append(packages) | ||
4407 | 203 | else: | ||
4408 | 204 | cmd.extend(packages) | ||
4409 | 205 | |||
4410 | 206 | if fatal: | ||
4411 | 207 | subprocess.check_call(cmd, universal_newlines=True) | ||
4412 | 208 | else: | ||
4413 | 209 | subprocess.call(cmd, universal_newlines=True) | ||
4414 | 210 | |||
4415 | 211 | |||
4416 | 212 | def apt_hold(packages, fatal=False): | ||
4417 | 213 | return apt_mark(packages, 'hold', fatal=fatal) | ||
4418 | 214 | |||
4419 | 215 | |||
4420 | 216 | def apt_unhold(packages, fatal=False): | ||
4421 | 217 | return apt_mark(packages, 'unhold', fatal=fatal) | ||
4422 | 218 | |||
4423 | 219 | |||
4424 | 220 | def add_source(source, key=None): | ||
4425 | 221 | """Add a package source to this system. | ||
4426 | 222 | |||
4427 | 223 | @param source: a URL or sources.list entry, as supported by | ||
4428 | 224 | add-apt-repository(1). Examples:: | ||
4429 | 225 | |||
4430 | 226 | ppa:charmers/example | ||
4431 | 227 | deb https://stub:key@private.example.com/ubuntu trusty main | ||
4432 | 228 | |||
4433 | 229 | In addition: | ||
4434 | 230 | 'proposed:' may be used to enable the standard 'proposed' | ||
4435 | 231 | pocket for the release. | ||
4436 | 232 | 'cloud:' may be used to activate official cloud archive pockets, | ||
4437 | 233 | such as 'cloud:icehouse' | ||
4438 | 234 | 'distro' may be used as a noop | ||
4439 | 235 | |||
4440 | 236 | @param key: A key to be added to the system's APT keyring and used | ||
4441 | 237 | to verify the signatures on packages. Ideally, this should be an | ||
4442 | 238 | ASCII format GPG public key including the block headers. A GPG key | ||
4443 | 239 | id may also be used, but be aware that only insecure protocols are | ||
4444 | 240 | available to retrieve the actual public key from a public keyserver | ||
4445 | 241 | placing your Juju environment at risk. ppa and cloud archive keys | ||
4446 | 242 | are securely added automtically, so sould not be provided. | ||
4447 | 243 | """ | ||
4448 | 244 | if source is None: | ||
4449 | 245 | log('Source is not present. Skipping') | ||
4450 | 246 | return | ||
4451 | 247 | |||
4452 | 248 | if (source.startswith('ppa:') or | ||
4453 | 249 | source.startswith('http') or | ||
4454 | 250 | source.startswith('deb ') or | ||
4455 | 251 | source.startswith('cloud-archive:')): | ||
4456 | 252 | cmd = ['add-apt-repository', '--yes', source] | ||
4457 | 253 | _run_with_retries(cmd) | ||
4458 | 254 | elif source.startswith('cloud:'): | ||
4459 | 255 | install(filter_installed_packages(['ubuntu-cloud-keyring']), | ||
4460 | 256 | fatal=True) | ||
4461 | 257 | pocket = source.split(':')[-1] | ||
4462 | 258 | if pocket not in CLOUD_ARCHIVE_POCKETS: | ||
4463 | 259 | raise SourceConfigError( | ||
4464 | 260 | 'Unsupported cloud: source option %s' % | ||
4465 | 261 | pocket) | ||
4466 | 262 | actual_pocket = CLOUD_ARCHIVE_POCKETS[pocket] | ||
4467 | 263 | with open('/etc/apt/sources.list.d/cloud-archive.list', 'w') as apt: | ||
4468 | 264 | apt.write(CLOUD_ARCHIVE.format(actual_pocket)) | ||
4469 | 265 | elif source == 'proposed': | ||
4470 | 266 | release = lsb_release()['DISTRIB_CODENAME'] | ||
4471 | 267 | with open('/etc/apt/sources.list.d/proposed.list', 'w') as apt: | ||
4472 | 268 | apt.write(PROPOSED_POCKET.format(release)) | ||
4473 | 269 | elif source == 'distro': | ||
4474 | 270 | pass | ||
4475 | 271 | else: | ||
4476 | 272 | log("Unknown source: {!r}".format(source)) | ||
4477 | 273 | |||
4478 | 274 | if key: | ||
4479 | 275 | if '-----BEGIN PGP PUBLIC KEY BLOCK-----' in key: | ||
4480 | 276 | with NamedTemporaryFile('w+') as key_file: | ||
4481 | 277 | key_file.write(key) | ||
4482 | 278 | key_file.flush() | ||
4483 | 279 | key_file.seek(0) | ||
4484 | 280 | subprocess.check_call(['apt-key', 'add', '-'], stdin=key_file) | ||
4485 | 281 | else: | ||
4486 | 282 | # Note that hkp: is in no way a secure protocol. Using a | ||
4487 | 283 | # GPG key id is pointless from a security POV unless you | ||
4488 | 284 | # absolutely trust your network and DNS. | ||
4489 | 285 | subprocess.check_call(['apt-key', 'adv', '--keyserver', | ||
4490 | 286 | 'hkp://keyserver.ubuntu.com:80', '--recv', | ||
4491 | 287 | key]) | ||
4492 | 288 | |||
4493 | 289 | |||
4494 | 290 | def _run_with_retries(cmd, max_retries=CMD_RETRY_COUNT, retry_exitcodes=(1,), | ||
4495 | 291 | retry_message="", cmd_env=None): | ||
4496 | 292 | """Run a command and retry until success or max_retries is reached. | ||
4497 | 293 | |||
4498 | 294 | :param: cmd: str: The apt command to run. | ||
4499 | 295 | :param: max_retries: int: The number of retries to attempt on a fatal | ||
4500 | 296 | command. Defaults to CMD_RETRY_COUNT. | ||
4501 | 297 | :param: retry_exitcodes: tuple: Optional additional exit codes to retry. | ||
4502 | 298 | Defaults to retry on exit code 1. | ||
4503 | 299 | :param: retry_message: str: Optional log prefix emitted during retries. | ||
4504 | 300 | :param: cmd_env: dict: Environment variables to add to the command run. | ||
4505 | 301 | """ | ||
4506 | 302 | |||
4507 | 303 | env = os.environ.copy() | ||
4508 | 304 | if cmd_env: | ||
4509 | 305 | env.update(cmd_env) | ||
4510 | 306 | |||
4511 | 307 | if not retry_message: | ||
4512 | 308 | retry_message = "Failed executing '{}'".format(" ".join(cmd)) | ||
4513 | 309 | retry_message += ". Will retry in {} seconds".format(CMD_RETRY_DELAY) | ||
4514 | 310 | |||
4515 | 311 | retry_count = 0 | ||
4516 | 312 | result = None | ||
4517 | 313 | |||
4518 | 314 | retry_results = (None,) + retry_exitcodes | ||
4519 | 315 | while result in retry_results: | ||
4520 | 316 | try: | ||
4521 | 317 | result = subprocess.check_call(cmd, env=env) | ||
4522 | 318 | except subprocess.CalledProcessError as e: | ||
4523 | 319 | retry_count = retry_count + 1 | ||
4524 | 320 | if retry_count > max_retries: | ||
4525 | 321 | raise | ||
4526 | 322 | result = e.returncode | ||
4527 | 323 | log(retry_message) | ||
4528 | 324 | time.sleep(CMD_RETRY_DELAY) | ||
4529 | 325 | |||
4530 | 326 | |||
4531 | 327 | def _run_apt_command(cmd, fatal=False): | ||
4532 | 328 | """Run an apt command with optional retries. | ||
4533 | 329 | |||
4534 | 330 | :param: fatal: bool: Whether the command's output should be checked and | ||
4535 | 331 | retried. | ||
4536 | 332 | """ | ||
4537 | 333 | # Provide DEBIAN_FRONTEND=noninteractive if not present in the environment. | ||
4538 | 334 | cmd_env = { | ||
4539 | 335 | 'DEBIAN_FRONTEND': os.environ.get('DEBIAN_FRONTEND', 'noninteractive')} | ||
4540 | 336 | |||
4541 | 337 | if fatal: | ||
4542 | 338 | _run_with_retries( | ||
4543 | 339 | cmd, cmd_env=cmd_env, retry_exitcodes=(1, APT_NO_LOCK,), | ||
4544 | 340 | retry_message="Couldn't acquire DPKG lock") | ||
4545 | 341 | else: | ||
4546 | 342 | env = os.environ.copy() | ||
4547 | 343 | env.update(cmd_env) | ||
4548 | 344 | subprocess.call(cmd, env=env) | ||
4549 | 345 | |||
4550 | 346 | |||
4551 | 347 | def get_upstream_version(package): | ||
4552 | 348 | """Determine upstream version based on installed package | ||
4553 | 349 | |||
4554 | 350 | @returns None (if not installed) or the upstream version | ||
4555 | 351 | """ | ||
4556 | 352 | import apt_pkg | ||
4557 | 353 | cache = apt_cache() | ||
4558 | 354 | try: | ||
4559 | 355 | pkg = cache[package] | ||
4560 | 356 | except: | ||
4561 | 357 | # the package is unknown to the current apt cache. | ||
4562 | 358 | return None | ||
4563 | 359 | |||
4564 | 360 | if not pkg.current_ver: | ||
4565 | 361 | # package is known, but no version is currently installed. | ||
4566 | 362 | return None | ||
4567 | 363 | |||
4568 | 364 | return apt_pkg.upstream_version(pkg.current_ver.ver_str) | ||
4569 | 0 | 365 | ||
4570 | === added file 'hooks/charmhelpers/osplatform.py' | |||
4571 | --- hooks/charmhelpers/osplatform.py 1970-01-01 00:00:00 +0000 | |||
4572 | +++ hooks/charmhelpers/osplatform.py 2018-06-12 20:07:13 +0000 | |||
4573 | @@ -0,0 +1,25 @@ | |||
4574 | 1 | import platform | ||
4575 | 2 | |||
4576 | 3 | |||
4577 | 4 | def get_platform(): | ||
4578 | 5 | """Return the current OS platform. | ||
4579 | 6 | |||
4580 | 7 | For example: if current os platform is Ubuntu then a string "ubuntu" | ||
4581 | 8 | will be returned (which is the name of the module). | ||
4582 | 9 | This string is used to decide which platform module should be imported. | ||
4583 | 10 | """ | ||
4584 | 11 | # linux_distribution is deprecated and will be removed in Python 3.7 | ||
4585 | 12 | # Warings *not* disabled, as we certainly need to fix this. | ||
4586 | 13 | tuple_platform = platform.linux_distribution() | ||
4587 | 14 | current_platform = tuple_platform[0] | ||
4588 | 15 | if "Ubuntu" in current_platform: | ||
4589 | 16 | return "ubuntu" | ||
4590 | 17 | elif "CentOS" in current_platform: | ||
4591 | 18 | return "centos" | ||
4592 | 19 | elif "debian" in current_platform: | ||
4593 | 20 | # Stock Python does not detect Ubuntu and instead returns debian. | ||
4594 | 21 | # Or at least it does in some build environments like Travis CI | ||
4595 | 22 | return "ubuntu" | ||
4596 | 23 | else: | ||
4597 | 24 | raise RuntimeError("This module is not supported on {}." | ||
4598 | 25 | .format(current_platform)) | ||
4599 | 0 | 26 | ||
4600 | === modified file 'hooks/hooks.py' | |||
4601 | --- hooks/hooks.py 2017-07-25 07:56:09 +0000 | |||
4602 | +++ hooks/hooks.py 2018-06-12 20:07:13 +0000 | |||
4603 | @@ -786,7 +786,7 @@ | |||
4604 | 786 | # Write to disk the content of the given SSL certificates | 786 | # Write to disk the content of the given SSL certificates |
4605 | 787 | crts = service_config.get('crts', []) | 787 | crts = service_config.get('crts', []) |
4606 | 788 | for i, crt in enumerate(crts): | 788 | for i, crt in enumerate(crts): |
4608 | 789 | if crt == "DEFAULT": | 789 | if crt == "DEFAULT" or crt == "EXTERNAL": |
4609 | 790 | continue | 790 | continue |
4610 | 791 | content = base64.b64decode(crt) | 791 | content = base64.b64decode(crt) |
4611 | 792 | path = get_service_lib_path(service_name) | 792 | path = get_service_lib_path(service_name) |
4612 | 793 | 793 | ||
4613 | === modified file 'icon.svg' | |||
4614 | --- icon.svg 2016-01-09 16:42:37 +0000 | |||
4615 | +++ icon.svg 2018-06-12 20:07:13 +0000 | |||
4616 | @@ -15,10 +15,21 @@ | |||
4617 | 15 | id="svg6517" | 15 | id="svg6517" |
4618 | 16 | version="1.1" | 16 | version="1.1" |
4619 | 17 | inkscape:version="0.91+devel r" | 17 | inkscape:version="0.91+devel r" |
4621 | 18 | sodipodi:docname="haproxy.svg" | 18 | sodipodi:docname="haproxy_circle.svg" |
4622 | 19 | viewBox="0 0 96 96"> | 19 | viewBox="0 0 96 96"> |
4623 | 20 | <defs | 20 | <defs |
4624 | 21 | id="defs6519"> | 21 | id="defs6519"> |
4625 | 22 | <linearGradient | ||
4626 | 23 | id="Background"> | ||
4627 | 24 | <stop | ||
4628 | 25 | id="stop4178" | ||
4629 | 26 | offset="0" | ||
4630 | 27 | style="stop-color:#dedede;stop-opacity:1" /> | ||
4631 | 28 | <stop | ||
4632 | 29 | id="stop4180" | ||
4633 | 30 | offset="1" | ||
4634 | 31 | style="stop-color:#ededed;stop-opacity:1" /> | ||
4635 | 32 | </linearGradient> | ||
4636 | 22 | <filter | 33 | <filter |
4637 | 23 | style="color-interpolation-filters:sRGB" | 34 | style="color-interpolation-filters:sRGB" |
4638 | 24 | inkscape:label="Inner Shadow" | 35 | inkscape:label="Inner Shadow" |
4639 | @@ -68,7 +79,7 @@ | |||
4640 | 68 | id="feComposite954" /> | 79 | id="feComposite954" /> |
4641 | 69 | <feGaussianBlur | 80 | <feGaussianBlur |
4642 | 70 | in="composite1" | 81 | in="composite1" |
4644 | 71 | stdDeviation="4" | 82 | stdDeviation="1" |
4645 | 72 | result="blur" | 83 | result="blur" |
4646 | 73 | id="feGaussianBlur956" /> | 84 | id="feGaussianBlur956" /> |
4647 | 74 | <feOffset | 85 | <feOffset |
4648 | @@ -83,17 +94,6 @@ | |||
4649 | 83 | result="composite2" | 94 | result="composite2" |
4650 | 84 | id="feComposite960" /> | 95 | id="feComposite960" /> |
4651 | 85 | </filter> | 96 | </filter> |
4652 | 86 | <linearGradient | ||
4653 | 87 | id="Background"> | ||
4654 | 88 | <stop | ||
4655 | 89 | id="stop4178" | ||
4656 | 90 | offset="0" | ||
4657 | 91 | style="stop-color:#22779e;stop-opacity:1" /> | ||
4658 | 92 | <stop | ||
4659 | 93 | id="stop4180" | ||
4660 | 94 | offset="1" | ||
4661 | 95 | style="stop-color:#2991c0;stop-opacity:1" /> | ||
4662 | 96 | </linearGradient> | ||
4663 | 97 | <clipPath | 97 | <clipPath |
4664 | 98 | clipPathUnits="userSpaceOnUse" | 98 | clipPathUnits="userSpaceOnUse" |
4665 | 99 | id="clipPath873"> | 99 | id="clipPath873"> |
4666 | @@ -110,128 +110,416 @@ | |||
4667 | 110 | sodipodi:nodetypes="sssssssss" /> | 110 | sodipodi:nodetypes="sssssssss" /> |
4668 | 111 | </g> | 111 | </g> |
4669 | 112 | </clipPath> | 112 | </clipPath> |
4792 | 113 | <filter | 113 | <linearGradient |
4793 | 114 | inkscape:collect="always" | 114 | id="linearGradient3804"> |
4794 | 115 | id="filter891" | 115 | <stop |
4795 | 116 | inkscape:label="Badge Shadow"> | 116 | id="stop3806" |
4796 | 117 | <feGaussianBlur | 117 | offset="0" |
4797 | 118 | inkscape:collect="always" | 118 | style="stop-color:#ffff00;stop-opacity:1;" /> |
4798 | 119 | stdDeviation="0.71999962" | 119 | <stop |
4799 | 120 | id="feGaussianBlur893" /> | 120 | id="stop3808" |
4800 | 121 | </filter> | 121 | offset="1" |
4801 | 122 | <style | 122 | style="stop-color:#a8650a;stop-opacity:0;" /> |
4802 | 123 | id="style867" | 123 | </linearGradient> |
4803 | 124 | type="text/css"><![CDATA[ | 124 | <linearGradient |
4804 | 125 | .fil0 {fill:#1F1A17} | 125 | id="linearGradient3778"> |
4805 | 126 | ]]></style> | 126 | <stop |
4806 | 127 | <clipPath | 127 | id="stop3780" |
4807 | 128 | id="clipPath16"> | 128 | offset="0" |
4808 | 129 | <path | 129 | style="stop-color:#ffff00;stop-opacity:0.84536082;" /> |
4809 | 130 | id="path18" | 130 | <stop |
4810 | 131 | d="M -9,-9 H 605 V 222 H -9 Z" | 131 | id="stop3782" |
4811 | 132 | inkscape:connector-curvature="0" /> | 132 | offset="1" |
4812 | 133 | </clipPath> | 133 | style="stop-color:#a8650a;stop-opacity:0;" /> |
4813 | 134 | <clipPath | 134 | </linearGradient> |
4814 | 135 | id="clipPath116"> | 135 | <linearGradient |
4815 | 136 | <path | 136 | id="linearGradient3725"> |
4816 | 137 | id="path118" | 137 | <stop |
4817 | 138 | d="m 91.7368,146.3253 -9.7039,-1.577 -8.8548,-3.8814 -7.5206,-4.7308 -7.1566,-8.7335 -4.0431,-4.282 -3.9093,-1.4409 -1.034,2.5271 1.8079,2.6096 0.4062,3.6802 1.211,-0.0488 1.3232,-1.2069 -0.3569,3.7488 -1.4667,0.9839 0.0445,1.4286 -3.4744,-1.9655 -3.1462,-3.712 -0.6559,-3.3176 1.3453,-2.6567 1.2549,-4.5133 2.5521,-1.2084 2.6847,0.1318 2.5455,1.4791 -1.698,-8.6122 1.698,-9.5825 -1.8692,-4.4246 -6.1223,-6.5965 1.0885,-3.941 2.9002,-4.5669 5.4688,-3.8486 2.9007,-0.3969 3.225,-0.1094 -2.012,-8.2601 7.3993,-3.0326 9.2188,-1.2129 3.1535,2.0619 0.2427,5.5797 3.5178,5.8224 0.2426,4.6094 8.4909,-0.6066 7.8843,0.7279 -7.8843,-4.7307 1.3343,-5.701 4.9731,-7.763 4.8521,-2.0622 3.8814,1.5769 1.577,3.1538 8.1269,6.1861 1.5769,-1.3343 12.7363,-0.485 2.5473,2.0619 0.2426,3.6391 -0.849,1.5767 -0.6066,9.8251 -4.2454,8.4909 0.7276,3.7605 2.5475,-1.3343 7.1566,-6.6716 3.5175,-0.2424 3.8815,1.5769 3.8818,2.9109 1.9406,6.3077 11.4021,-0.7277 6.914,2.6686 5.5797,5.2157 4.0028,7.5206 0.9706,8.8546 -0.8493,10.3105 -2.1832,9.2185 -2.1836,2.9112 -3.0322,0.9706 -5.3373,-5.8224 -4.8518,-1.6982 -4.2455,7.0353 -4.2454,3.8815 -2.3049,1.4556 -9.2185,7.6419 -7.3993,4.0028 -7.3993,0.6066 -8.6119,-1.4556 -7.5206,-2.7899 -5.2158,-4.2454 -4.1241,-4.9734 -4.2454,-1.2129" | 138 | id="stop3727" |
4818 | 139 | inkscape:connector-curvature="0" /> | 139 | offset="0" |
4819 | 140 | </clipPath> | 140 | style="stop-color:#0000ff;stop-opacity:1;" /> |
4820 | 141 | <clipPath | 141 | <stop |
4821 | 142 | id="clipPath128"> | 142 | id="stop3729" |
4822 | 143 | <path | 143 | offset="1" |
4823 | 144 | id="path130" | 144 | style="stop-color:#000000;stop-opacity:0" /> |
4824 | 145 | d="m 91.7368,146.3253 -9.7039,-1.577 -8.8548,-3.8814 -7.5206,-4.7308 -7.1566,-8.7335 -4.0431,-4.282 -3.9093,-1.4409 -1.034,2.5271 1.8079,2.6096 0.4062,3.6802 1.211,-0.0488 1.3232,-1.2069 -0.3569,3.7488 -1.4667,0.9839 0.0445,1.4286 -3.4744,-1.9655 -3.1462,-3.712 -0.6559,-3.3176 1.3453,-2.6567 1.2549,-4.5133 2.5521,-1.2084 2.6847,0.1318 2.5455,1.4791 -1.698,-8.6122 1.698,-9.5825 -1.8692,-4.4246 -6.1223,-6.5965 1.0885,-3.941 2.9002,-4.5669 5.4688,-3.8486 2.9007,-0.3969 3.225,-0.1094 -2.012,-8.2601 7.3993,-3.0326 9.2188,-1.2129 3.1535,2.0619 0.2427,5.5797 3.5178,5.8224 0.2426,4.6094 8.4909,-0.6066 7.8843,0.7279 -7.8843,-4.7307 1.3343,-5.701 4.9731,-7.763 4.8521,-2.0622 3.8814,1.5769 1.577,3.1538 8.1269,6.1861 1.5769,-1.3343 12.7363,-0.485 2.5473,2.0619 0.2426,3.6391 -0.849,1.5767 -0.6066,9.8251 -4.2454,8.4909 0.7276,3.7605 2.5475,-1.3343 7.1566,-6.6716 3.5175,-0.2424 3.8815,1.5769 3.8818,2.9109 1.9406,6.3077 11.4021,-0.7277 6.914,2.6686 5.5797,5.2157 4.0028,7.5206 0.9706,8.8546 -0.8493,10.3105 -2.1832,9.2185 -2.1836,2.9112 -3.0322,0.9706 -5.3373,-5.8224 -4.8518,-1.6982 -4.2455,7.0353 -4.2454,3.8815 -2.3049,1.4556 -9.2185,7.6419 -7.3993,4.0028 -7.3993,0.6066 -8.6119,-1.4556 -7.5206,-2.7899 -5.2158,-4.2454 -4.1241,-4.9734 -4.2454,-1.2129" | 145 | </linearGradient> |
4825 | 146 | inkscape:connector-curvature="0" /> | 146 | <linearGradient |
4826 | 147 | </clipPath> | 147 | id="linearGradient3715"> |
4827 | 148 | <linearGradient | 148 | <stop |
4828 | 149 | id="linearGradient3850" | 149 | style="stop-color:#3700d0;stop-opacity:1;" |
4829 | 150 | inkscape:collect="always"> | 150 | offset="0" |
4830 | 151 | <stop | 151 | id="stop3717" /> |
4831 | 152 | id="stop3852" | 152 | <stop |
4832 | 153 | offset="0" | 153 | style="stop-color: rgb(0, 0, 0); stop-opacity: 0;" |
4833 | 154 | style="stop-color:#000000;stop-opacity:1;" /> | 154 | offset="1" |
4834 | 155 | <stop | 155 | id="stop3719" /> |
4835 | 156 | id="stop3854" | 156 | </linearGradient> |
4836 | 157 | offset="1" | 157 | <linearGradient |
4837 | 158 | style="stop-color:#000000;stop-opacity:0;" /> | 158 | id="linearGradient3705"> |
4838 | 159 | </linearGradient> | 159 | <stop |
4839 | 160 | <clipPath | 160 | style="stop-color:#ff0000;stop-opacity:0.54639173;" |
4840 | 161 | clipPathUnits="userSpaceOnUse" | 161 | offset="0" |
4841 | 162 | id="clipPath3095"> | 162 | id="stop3707" /> |
4842 | 163 | <path | 163 | <stop |
4843 | 164 | d="M 976.648,389.551 H 134.246 V 1229.55 H 976.648 V 389.551" | 164 | style="stop-color:#ff0067;stop-opacity:0.42268041;" |
4844 | 165 | id="path3097" | 165 | offset="1" |
4845 | 166 | inkscape:connector-curvature="0" /> | 166 | id="stop3709" /> |
4846 | 167 | </clipPath> | 167 | </linearGradient> |
4847 | 168 | <clipPath | 168 | <linearGradient |
4848 | 169 | clipPathUnits="userSpaceOnUse" | 169 | id="linearGradient3699"> |
4849 | 170 | id="clipPath3195"> | 170 | <stop |
4850 | 171 | <path | 171 | style="stop-color:#d80000;stop-opacity:1;" |
4851 | 172 | d="m 611.836,756.738 -106.34,105.207 c -8.473,8.289 -13.617,20.102 -13.598,33.379 L 598.301,790.207 c -0.031,-13.418 5.094,-25.031 13.535,-33.469" | 172 | offset="0" |
4852 | 173 | id="path3197" | 173 | id="stop3701" /> |
4853 | 174 | inkscape:connector-curvature="0" /> | 174 | <stop |
4854 | 175 | </clipPath> | 175 | style="stop-color: rgb(136, 0, 170); stop-opacity: 0;" |
4855 | 176 | <clipPath | 176 | offset="1" |
4856 | 177 | clipPathUnits="userSpaceOnUse" | 177 | id="stop3703" /> |
4857 | 178 | id="clipPath3235"> | 178 | </linearGradient> |
4858 | 179 | <path | 179 | <linearGradient |
4859 | 180 | d="m 1095.64,1501.81 c 35.46,-35.07 70.89,-70.11 106.35,-105.17 4.4,-4.38 7.11,-10.53 7.11,-17.55 l -106.37,105.21 c 0,7 -2.71,13.11 -7.09,17.51" | 180 | id="linearGradient3693"> |
4860 | 181 | id="path3237" | 181 | <stop |
4861 | 182 | inkscape:connector-curvature="0" /> | 182 | style="stop-color:#4700aa;stop-opacity:1;" |
4862 | 183 | </clipPath> | 183 | offset="0" |
4863 | 184 | <clipPath | 184 | id="stop3695" /> |
4864 | 185 | id="clipPath4591" | 185 | <stop |
4865 | 186 | clipPathUnits="userSpaceOnUse"> | 186 | style="stop-color: rgb(136, 0, 170); stop-opacity: 0;" |
4866 | 187 | <path | 187 | offset="1" |
4867 | 188 | inkscape:connector-curvature="0" | 188 | id="stop3697" /> |
4868 | 189 | d="m 1106.6009,730.43734 -0.036,21.648 c -0.01,3.50825 -2.8675,6.61375 -6.4037,6.92525 l -83.6503,7.33162 c -3.5205,0.30763 -6.3812,-2.29987 -6.3671,-5.8145 l 0.036,-21.6475 20.1171,-1.76662 -0.011,4.63775 c 0,1.83937 1.4844,3.19925 3.3262,3.0395 l 49.5274,-4.33975 c 1.8425,-0.166 3.3425,-1.78125 3.3538,-3.626 l 0.01,-4.63025 20.1,-1.7575" | 189 | </linearGradient> |
4869 | 190 | style="fill:#ff00ff;fill-opacity:1;fill-rule:nonzero;stroke:none" | 190 | <linearGradient |
4870 | 191 | id="path4593" /> | 191 | id="linearGradient3687"> |
4871 | 192 | </clipPath> | 192 | <stop |
4872 | 193 | <radialGradient | 193 | style="stop-color:#cd00ff;stop-opacity:1;" |
4873 | 194 | gradientUnits="userSpaceOnUse" | 194 | offset="0" |
4874 | 195 | gradientTransform="matrix(-1.4333926,-2.2742838,1.1731823,-0.73941125,-174.08025,98.374394)" | 195 | id="stop3689" /> |
4875 | 196 | r="20.40658" | 196 | <stop |
4876 | 197 | fy="93.399292" | 197 | style="stop-color:#ff00ff;stop-opacity:0;" |
4877 | 198 | fx="-26.508606" | 198 | offset="1" |
4878 | 199 | cy="93.399292" | 199 | id="stop3691" /> |
4879 | 200 | cx="-26.508606" | 200 | </linearGradient> |
4880 | 201 | id="radialGradient3856" | 201 | <linearGradient |
4881 | 202 | xlink:href="#linearGradient3850" | 202 | id="linearGradient3681"> |
4882 | 203 | inkscape:collect="always" /> | 203 | <stop |
4883 | 204 | <linearGradient | 204 | style="stop-color:#ffff00;stop-opacity:1;" |
4884 | 205 | gradientTransform="translate(-318.48033,212.32022)" | 205 | offset="0" |
4885 | 206 | gradientUnits="userSpaceOnUse" | 206 | id="stop3683" /> |
4886 | 207 | y2="993.19702" | 207 | <stop |
4887 | 208 | x2="-51.879555" | 208 | style="stop-color:#a8650a;stop-opacity:0.45360824;" |
4888 | 209 | y1="593.11615" | 209 | offset="1" |
4889 | 210 | x1="348.20132" | 210 | id="stop3685" /> |
4890 | 211 | id="linearGradient3895" | 211 | </linearGradient> |
4891 | 212 | xlink:href="#linearGradient3850" | 212 | <linearGradient |
4892 | 213 | inkscape:collect="always" /> | 213 | id="linearGradient3265" |
4893 | 214 | <clipPath | 214 | inkscape:collect="always"> |
4894 | 215 | id="clipPath3906" | 215 | <stop |
4895 | 216 | clipPathUnits="userSpaceOnUse"> | 216 | id="stop3267" |
4896 | 217 | <rect | 217 | offset="0" |
4897 | 218 | transform="scale(1,-1)" | 218 | style="stop-color: rgb(128, 101, 10); stop-opacity: 1;" /> |
4898 | 219 | style="color:#000000;display:inline;overflow:visible;visibility:visible;opacity:0.8;fill:#ff00ff;stroke:none;stroke-width:4;marker:none;enable-background:accumulate" | 219 | <stop |
4899 | 220 | id="rect3908" | 220 | id="stop3269" |
4900 | 221 | width="1019.1371" | 221 | offset="1" |
4901 | 222 | height="1019.1371" | 222 | style="stop-color: rgb(128, 101, 10); stop-opacity: 0;" /> |
4902 | 223 | x="357.9816" | 223 | </linearGradient> |
4903 | 224 | y="-1725.8152" /> | 224 | <linearGradient |
4904 | 225 | </clipPath> | 225 | id="linearGradient3255" |
4905 | 226 | <clipPath | 226 | inkscape:collect="always"> |
4906 | 227 | clipPathUnits="userSpaceOnUse" | 227 | <stop |
4907 | 228 | id="clipPath4637"> | 228 | id="stop3257" |
4908 | 229 | <path | 229 | offset="0" |
4909 | 230 | sodipodi:nodetypes="sssssssss" | 230 | style="stop-color: rgb(6, 46, 5); stop-opacity: 1;" /> |
4910 | 231 | inkscape:connector-curvature="0" | 231 | <stop |
4911 | 232 | id="path4639" | 232 | id="stop3259" |
4912 | 233 | d="M -268,700.15563 V 666.4259 c 0,-27.24324 3.88785,-31.13513 31.10302,-31.13513 h 33.79408 c 27.21507,0 31.1029,3.89189 31.1029,31.13513 v 33.72973 c 0,27.24325 -3.88783,31.13514 -31.1029,31.13514 h -33.79408 c -27.21517,0 -31.10302,-3.89189 -31.10302,-31.13514 z" | 233 | offset="1" |
4913 | 234 | style="display:inline;fill:#df382c;fill-opacity:1;stroke:none" /> | 234 | style="stop-color: rgb(6, 46, 5); stop-opacity: 0;" /> |
4914 | 235 | </linearGradient> | ||
4915 | 236 | <linearGradient | ||
4916 | 237 | id="linearGradient3238" | ||
4917 | 238 | inkscape:collect="always"> | ||
4918 | 239 | <stop | ||
4919 | 240 | id="stop3240" | ||
4920 | 241 | offset="0" | ||
4921 | 242 | style="stop-color: rgb(0, 0, 0); stop-opacity: 1;" /> | ||
4922 | 243 | <stop | ||
4923 | 244 | id="stop3242" | ||
4924 | 245 | offset="1" | ||
4925 | 246 | style="stop-color: rgb(0, 0, 0); stop-opacity: 0;" /> | ||
4926 | 247 | </linearGradient> | ||
4927 | 248 | <linearGradient | ||
4928 | 249 | id="linearGradient3230"> | ||
4929 | 250 | <stop | ||
4930 | 251 | id="stop3232" | ||
4931 | 252 | offset="0" | ||
4932 | 253 | style="stop-color:#ea009b;stop-opacity:1;" /> | ||
4933 | 254 | <stop | ||
4934 | 255 | id="stop3234" | ||
4935 | 256 | offset="1" | ||
4936 | 257 | style="stop-color: rgb(209, 249, 208); stop-opacity: 0;" /> | ||
4937 | 258 | </linearGradient> | ||
4938 | 259 | <linearGradient | ||
4939 | 260 | id="linearGradient3218" | ||
4940 | 261 | inkscape:collect="always"> | ||
4941 | 262 | <stop | ||
4942 | 263 | id="stop3220" | ||
4943 | 264 | offset="0" | ||
4944 | 265 | style="stop-color: rgb(136, 0, 170); stop-opacity: 1;" /> | ||
4945 | 266 | <stop | ||
4946 | 267 | id="stop3222" | ||
4947 | 268 | offset="1" | ||
4948 | 269 | style="stop-color: rgb(136, 0, 170); stop-opacity: 0;" /> | ||
4949 | 270 | </linearGradient> | ||
4950 | 271 | <linearGradient | ||
4951 | 272 | id="linearGradient3208" | ||
4952 | 273 | inkscape:collect="always"> | ||
4953 | 274 | <stop | ||
4954 | 275 | id="stop3210" | ||
4955 | 276 | offset="0" | ||
4956 | 277 | style="stop-color: rgb(36, 216, 31); stop-opacity: 1;" /> | ||
4957 | 278 | <stop | ||
4958 | 279 | id="stop3212" | ||
4959 | 280 | offset="1" | ||
4960 | 281 | style="stop-color: rgb(36, 216, 31); stop-opacity: 0;" /> | ||
4961 | 282 | </linearGradient> | ||
4962 | 283 | <linearGradient | ||
4963 | 284 | id="linearGradient3184"> | ||
4964 | 285 | <stop | ||
4965 | 286 | id="stop3186" | ||
4966 | 287 | offset="0" | ||
4967 | 288 | style="stop-color:#4200d2;stop-opacity:1;" /> | ||
4968 | 289 | <stop | ||
4969 | 290 | id="stop3188" | ||
4970 | 291 | offset="1" | ||
4971 | 292 | style="stop-color:#d42aff;stop-opacity:0;" /> | ||
4972 | 293 | </linearGradient> | ||
4973 | 294 | <linearGradient | ||
4974 | 295 | id="linearGradient3174" | ||
4975 | 296 | inkscape:collect="always"> | ||
4976 | 297 | <stop | ||
4977 | 298 | id="stop3176" | ||
4978 | 299 | offset="0" | ||
4979 | 300 | style="stop-color: rgb(0, 255, 0); stop-opacity: 1;" /> | ||
4980 | 301 | <stop | ||
4981 | 302 | id="stop3178" | ||
4982 | 303 | offset="1" | ||
4983 | 304 | style="stop-color: rgb(0, 255, 0); stop-opacity: 0;" /> | ||
4984 | 305 | </linearGradient> | ||
4985 | 306 | <linearGradient | ||
4986 | 307 | id="linearGradient3166" | ||
4987 | 308 | inkscape:collect="always"> | ||
4988 | 309 | <stop | ||
4989 | 310 | id="stop3168" | ||
4990 | 311 | offset="0" | ||
4991 | 312 | style="stop-color: rgb(0, 0, 0); stop-opacity: 1;" /> | ||
4992 | 313 | <stop | ||
4993 | 314 | id="stop3170" | ||
4994 | 315 | offset="1" | ||
4995 | 316 | style="stop-color: rgb(0, 0, 0); stop-opacity: 0;" /> | ||
4996 | 317 | </linearGradient> | ||
4997 | 318 | <inkscape:perspective | ||
4998 | 319 | id="perspective10" | ||
4999 | 320 | inkscape:persp3d-origin="372.04724 : 350.78739 : 1" | ||
5000 | 321 | inkscape:vp_z="744.09448 : 526.18109 : 1" |
The diff has been truncated for viewing.
LGTM, +1