Merge lp:~chad.smith/landscape-charm/add-apt-repository-retries into lp:~landscape/landscape-charm/trunk
- add-apt-repository-retries
- Merge into trunk
Status: | Merged | ||||
---|---|---|---|---|---|
Approved by: | Chad Smith | ||||
Approved revision: | 386 | ||||
Merged at revision: | 383 | ||||
Proposed branch: | lp:~chad.smith/landscape-charm/add-apt-repository-retries | ||||
Merge into: | lp:~landscape/landscape-charm/trunk | ||||
Diff against target: |
3080 lines (+1633/-771) 34 files modified
charm-helpers.yaml (+1/-0) charmhelpers/__init__.py (+11/-13) charmhelpers/contrib/__init__.py (+11/-13) charmhelpers/contrib/hahelpers/__init__.py (+11/-13) charmhelpers/contrib/hahelpers/apache.py (+30/-17) charmhelpers/contrib/hahelpers/cluster.py (+70/-23) charmhelpers/core/__init__.py (+11/-13) charmhelpers/core/decorators.py (+11/-13) charmhelpers/core/files.py (+11/-13) charmhelpers/core/fstab.py (+11/-13) charmhelpers/core/hookenv.py (+73/-14) charmhelpers/core/host.py (+327/-126) charmhelpers/core/host_factory/centos.py (+56/-0) charmhelpers/core/host_factory/ubuntu.py (+56/-0) charmhelpers/core/hugepage.py (+11/-13) charmhelpers/core/kernel.py (+34/-30) charmhelpers/core/kernel_factory/centos.py (+17/-0) charmhelpers/core/kernel_factory/ubuntu.py (+13/-0) charmhelpers/core/services/__init__.py (+11/-13) charmhelpers/core/services/base.py (+11/-13) charmhelpers/core/services/helpers.py (+11/-13) charmhelpers/core/strutils.py (+11/-13) charmhelpers/core/sysctl.py (+11/-13) charmhelpers/core/templating.py (+19/-16) charmhelpers/core/unitdata.py (+11/-14) charmhelpers/fetch/__init__.py (+42/-309) charmhelpers/fetch/archiveurl.py (+11/-13) charmhelpers/fetch/bzrurl.py (+31/-23) charmhelpers/fetch/centos.py (+171/-0) charmhelpers/fetch/giturl.py (+15/-16) charmhelpers/fetch/snap.py (+122/-0) charmhelpers/fetch/ubuntu.py (+364/-0) charmhelpers/osplatform.py (+25/-0) lib/apt.py (+2/-1) |
||||
To merge this branch: | bzr merge lp:~chad.smith/landscape-charm/add-apt-repository-retries | ||||
Related bugs: |
|
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
🤖 Landscape Builder | test results | Approve | |
Eric Snow (community) | Approve | ||
Review via email: mp+318962@code.launchpad.net |
Commit message
Sync of charmhelpers to get updated fetch.add_source which handles 30 retries upon exit 1
Add charmhelpers.
Description of the change
Sync of charmhelpers to get updated fetch.add_source which handles 30 retries upon exit 1. Generally this exit code is returned upon network failure like bug https:/
Retry fixes are in charmhelpers/
🤖 Landscape Builder (landscape-builder) : | # |
🤖 Landscape Builder (landscape-builder) wrote : | # |
Eric Snow (ericsnowcurrently) wrote : | # |
As this doesn't introduce any changes, LGTM. :)
- 384. By Chad Smith
-
add charmhelpers.
osplatform dependency for charmhelpers.host get_platform - 385. By Chad Smith
-
make sync pulling in osplatform
🤖 Landscape Builder (landscape-builder) : | # |
🤖 Landscape Builder (landscape-builder) wrote : | # |
Command: make ci-test
Result: Fail
Revno: 385
Branch: lp:~chad.smith/landscape-charm/add-apt-repository-retries
Jenkins: https:/
- 386. By Chad Smith
-
lint me
🤖 Landscape Builder (landscape-builder) : | # |
🤖 Landscape Builder (landscape-builder) wrote : | # |
Command: make ci-test
Result: Success
Revno: 386
Branch: lp:~chad.smith/landscape-charm/add-apt-repository-retries
Jenkins: https:/
Preview Diff
1 | === modified file 'charm-helpers.yaml' | |||
2 | --- charm-helpers.yaml 2015-05-07 10:26:45 +0000 | |||
3 | +++ charm-helpers.yaml 2017-03-04 02:50:20 +0000 | |||
4 | @@ -4,4 +4,5 @@ | |||
5 | 4 | - __init__ | 4 | - __init__ |
6 | 5 | - core | 5 | - core |
7 | 6 | - fetch | 6 | - fetch |
8 | 7 | - osplatform | ||
9 | 7 | - contrib.hahelpers | 8 | - contrib.hahelpers |
10 | 8 | 9 | ||
11 | === modified file 'charmhelpers/__init__.py' | |||
12 | --- charmhelpers/__init__.py 2015-01-28 08:59:02 +0000 | |||
13 | +++ charmhelpers/__init__.py 2017-03-04 02:50:20 +0000 | |||
14 | @@ -1,18 +1,16 @@ | |||
15 | 1 | # Copyright 2014-2015 Canonical Limited. | 1 | # Copyright 2014-2015 Canonical Limited. |
16 | 2 | # | 2 | # |
30 | 3 | # This file is part of charm-helpers. | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); |
31 | 4 | # | 4 | # you may not use this file except in compliance with the License. |
32 | 5 | # charm-helpers is free software: you can redistribute it and/or modify | 5 | # You may obtain a copy of the License at |
33 | 6 | # it under the terms of the GNU Lesser General Public License version 3 as | 6 | # |
34 | 7 | # published by the Free Software Foundation. | 7 | # http://www.apache.org/licenses/LICENSE-2.0 |
35 | 8 | # | 8 | # |
36 | 9 | # charm-helpers is distributed in the hope that it will be useful, | 9 | # Unless required by applicable law or agreed to in writing, software |
37 | 10 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 10 | # distributed under the License is distributed on an "AS IS" BASIS, |
38 | 11 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
39 | 12 | # GNU Lesser General Public License for more details. | 12 | # See the License for the specific language governing permissions and |
40 | 13 | # | 13 | # limitations under the License. |
28 | 14 | # You should have received a copy of the GNU Lesser General Public License | ||
29 | 15 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
41 | 16 | 14 | ||
42 | 17 | # Bootstrap charm-helpers, installing its dependencies if necessary using | 15 | # Bootstrap charm-helpers, installing its dependencies if necessary using |
43 | 18 | # only standard libraries. | 16 | # only standard libraries. |
44 | 19 | 17 | ||
45 | === modified file 'charmhelpers/contrib/__init__.py' | |||
46 | --- charmhelpers/contrib/__init__.py 2015-01-30 11:16:09 +0000 | |||
47 | +++ charmhelpers/contrib/__init__.py 2017-03-04 02:50:20 +0000 | |||
48 | @@ -1,15 +1,13 @@ | |||
49 | 1 | # Copyright 2014-2015 Canonical Limited. | 1 | # Copyright 2014-2015 Canonical Limited. |
50 | 2 | # | 2 | # |
64 | 3 | # This file is part of charm-helpers. | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); |
65 | 4 | # | 4 | # you may not use this file except in compliance with the License. |
66 | 5 | # charm-helpers is free software: you can redistribute it and/or modify | 5 | # You may obtain a copy of the License at |
67 | 6 | # it under the terms of the GNU Lesser General Public License version 3 as | 6 | # |
68 | 7 | # published by the Free Software Foundation. | 7 | # http://www.apache.org/licenses/LICENSE-2.0 |
69 | 8 | # | 8 | # |
70 | 9 | # charm-helpers is distributed in the hope that it will be useful, | 9 | # Unless required by applicable law or agreed to in writing, software |
71 | 10 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 10 | # distributed under the License is distributed on an "AS IS" BASIS, |
72 | 11 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
73 | 12 | # GNU Lesser General Public License for more details. | 12 | # See the License for the specific language governing permissions and |
74 | 13 | # | 13 | # limitations under the License. |
62 | 14 | # You should have received a copy of the GNU Lesser General Public License | ||
63 | 15 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
75 | 16 | 14 | ||
76 | === modified file 'charmhelpers/contrib/hahelpers/__init__.py' | |||
77 | --- charmhelpers/contrib/hahelpers/__init__.py 2015-01-30 11:16:09 +0000 | |||
78 | +++ charmhelpers/contrib/hahelpers/__init__.py 2017-03-04 02:50:20 +0000 | |||
79 | @@ -1,15 +1,13 @@ | |||
80 | 1 | # Copyright 2014-2015 Canonical Limited. | 1 | # Copyright 2014-2015 Canonical Limited. |
81 | 2 | # | 2 | # |
95 | 3 | # This file is part of charm-helpers. | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); |
96 | 4 | # | 4 | # you may not use this file except in compliance with the License. |
97 | 5 | # charm-helpers is free software: you can redistribute it and/or modify | 5 | # You may obtain a copy of the License at |
98 | 6 | # it under the terms of the GNU Lesser General Public License version 3 as | 6 | # |
99 | 7 | # published by the Free Software Foundation. | 7 | # http://www.apache.org/licenses/LICENSE-2.0 |
100 | 8 | # | 8 | # |
101 | 9 | # charm-helpers is distributed in the hope that it will be useful, | 9 | # Unless required by applicable law or agreed to in writing, software |
102 | 10 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 10 | # distributed under the License is distributed on an "AS IS" BASIS, |
103 | 11 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
104 | 12 | # GNU Lesser General Public License for more details. | 12 | # See the License for the specific language governing permissions and |
105 | 13 | # | 13 | # limitations under the License. |
93 | 14 | # You should have received a copy of the GNU Lesser General Public License | ||
94 | 15 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
106 | 16 | 14 | ||
107 | === modified file 'charmhelpers/contrib/hahelpers/apache.py' | |||
108 | --- charmhelpers/contrib/hahelpers/apache.py 2015-01-30 11:16:09 +0000 | |||
109 | +++ charmhelpers/contrib/hahelpers/apache.py 2017-03-04 02:50:20 +0000 | |||
110 | @@ -1,18 +1,16 @@ | |||
111 | 1 | # Copyright 2014-2015 Canonical Limited. | 1 | # Copyright 2014-2015 Canonical Limited. |
112 | 2 | # | 2 | # |
126 | 3 | # This file is part of charm-helpers. | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); |
127 | 4 | # | 4 | # you may not use this file except in compliance with the License. |
128 | 5 | # charm-helpers is free software: you can redistribute it and/or modify | 5 | # You may obtain a copy of the License at |
129 | 6 | # it under the terms of the GNU Lesser General Public License version 3 as | 6 | # |
130 | 7 | # published by the Free Software Foundation. | 7 | # http://www.apache.org/licenses/LICENSE-2.0 |
131 | 8 | # | 8 | # |
132 | 9 | # charm-helpers is distributed in the hope that it will be useful, | 9 | # Unless required by applicable law or agreed to in writing, software |
133 | 10 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 10 | # distributed under the License is distributed on an "AS IS" BASIS, |
134 | 11 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
135 | 12 | # GNU Lesser General Public License for more details. | 12 | # See the License for the specific language governing permissions and |
136 | 13 | # | 13 | # limitations under the License. |
124 | 14 | # You should have received a copy of the GNU Lesser General Public License | ||
125 | 15 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
137 | 16 | 14 | ||
138 | 17 | # | 15 | # |
139 | 18 | # Copyright 2012 Canonical Ltd. | 16 | # Copyright 2012 Canonical Ltd. |
140 | @@ -24,6 +22,7 @@ | |||
141 | 24 | # Adam Gandelman <adamg@ubuntu.com> | 22 | # Adam Gandelman <adamg@ubuntu.com> |
142 | 25 | # | 23 | # |
143 | 26 | 24 | ||
144 | 25 | import os | ||
145 | 27 | import subprocess | 26 | import subprocess |
146 | 28 | 27 | ||
147 | 29 | from charmhelpers.core.hookenv import ( | 28 | from charmhelpers.core.hookenv import ( |
148 | @@ -74,9 +73,23 @@ | |||
149 | 74 | return ca_cert | 73 | return ca_cert |
150 | 75 | 74 | ||
151 | 76 | 75 | ||
152 | 76 | def retrieve_ca_cert(cert_file): | ||
153 | 77 | cert = None | ||
154 | 78 | if os.path.isfile(cert_file): | ||
155 | 79 | with open(cert_file, 'r') as crt: | ||
156 | 80 | cert = crt.read() | ||
157 | 81 | return cert | ||
158 | 82 | |||
159 | 83 | |||
160 | 77 | def install_ca_cert(ca_cert): | 84 | def install_ca_cert(ca_cert): |
161 | 78 | if ca_cert: | 85 | if ca_cert: |
166 | 79 | with open('/usr/local/share/ca-certificates/keystone_juju_ca_cert.crt', | 86 | cert_file = ('/usr/local/share/ca-certificates/' |
167 | 80 | 'w') as crt: | 87 | 'keystone_juju_ca_cert.crt') |
168 | 81 | crt.write(ca_cert) | 88 | old_cert = retrieve_ca_cert(cert_file) |
169 | 82 | subprocess.check_call(['update-ca-certificates', '--fresh']) | 89 | if old_cert and old_cert == ca_cert: |
170 | 90 | log("CA cert is the same as installed version", level=INFO) | ||
171 | 91 | else: | ||
172 | 92 | log("Installing new CA cert", level=INFO) | ||
173 | 93 | with open(cert_file, 'w') as crt: | ||
174 | 94 | crt.write(ca_cert) | ||
175 | 95 | subprocess.check_call(['update-ca-certificates', '--fresh']) | ||
176 | 83 | 96 | ||
177 | === modified file 'charmhelpers/contrib/hahelpers/cluster.py' | |||
178 | --- charmhelpers/contrib/hahelpers/cluster.py 2015-07-03 09:13:26 +0000 | |||
179 | +++ charmhelpers/contrib/hahelpers/cluster.py 2017-03-04 02:50:20 +0000 | |||
180 | @@ -1,18 +1,16 @@ | |||
181 | 1 | # Copyright 2014-2015 Canonical Limited. | 1 | # Copyright 2014-2015 Canonical Limited. |
182 | 2 | # | 2 | # |
196 | 3 | # This file is part of charm-helpers. | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); |
197 | 4 | # | 4 | # you may not use this file except in compliance with the License. |
198 | 5 | # charm-helpers is free software: you can redistribute it and/or modify | 5 | # You may obtain a copy of the License at |
199 | 6 | # it under the terms of the GNU Lesser General Public License version 3 as | 6 | # |
200 | 7 | # published by the Free Software Foundation. | 7 | # http://www.apache.org/licenses/LICENSE-2.0 |
201 | 8 | # | 8 | # |
202 | 9 | # charm-helpers is distributed in the hope that it will be useful, | 9 | # Unless required by applicable law or agreed to in writing, software |
203 | 10 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 10 | # distributed under the License is distributed on an "AS IS" BASIS, |
204 | 11 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
205 | 12 | # GNU Lesser General Public License for more details. | 12 | # See the License for the specific language governing permissions and |
206 | 13 | # | 13 | # limitations under the License. |
194 | 14 | # You should have received a copy of the GNU Lesser General Public License | ||
195 | 15 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
207 | 16 | 14 | ||
208 | 17 | # | 15 | # |
209 | 18 | # Copyright 2012 Canonical Ltd. | 16 | # Copyright 2012 Canonical Ltd. |
210 | @@ -41,10 +39,11 @@ | |||
211 | 41 | relation_get, | 39 | relation_get, |
212 | 42 | config as config_get, | 40 | config as config_get, |
213 | 43 | INFO, | 41 | INFO, |
215 | 44 | ERROR, | 42 | DEBUG, |
216 | 45 | WARNING, | 43 | WARNING, |
217 | 46 | unit_get, | 44 | unit_get, |
219 | 47 | is_leader as juju_is_leader | 45 | is_leader as juju_is_leader, |
220 | 46 | status_set, | ||
221 | 48 | ) | 47 | ) |
222 | 49 | from charmhelpers.core.decorators import ( | 48 | from charmhelpers.core.decorators import ( |
223 | 50 | retry_on_exception, | 49 | retry_on_exception, |
224 | @@ -60,6 +59,10 @@ | |||
225 | 60 | pass | 59 | pass |
226 | 61 | 60 | ||
227 | 62 | 61 | ||
228 | 62 | class HAIncorrectConfig(Exception): | ||
229 | 63 | pass | ||
230 | 64 | |||
231 | 65 | |||
232 | 63 | class CRMResourceNotFound(Exception): | 66 | class CRMResourceNotFound(Exception): |
233 | 64 | pass | 67 | pass |
234 | 65 | 68 | ||
235 | @@ -274,27 +277,71 @@ | |||
236 | 274 | Obtains all relevant configuration from charm configuration required | 277 | Obtains all relevant configuration from charm configuration required |
237 | 275 | for initiating a relation to hacluster: | 278 | for initiating a relation to hacluster: |
238 | 276 | 279 | ||
240 | 277 | ha-bindiface, ha-mcastport, vip | 280 | ha-bindiface, ha-mcastport, vip, os-internal-hostname, |
241 | 281 | os-admin-hostname, os-public-hostname, os-access-hostname | ||
242 | 278 | 282 | ||
243 | 279 | param: exclude_keys: list of setting key(s) to be excluded. | 283 | param: exclude_keys: list of setting key(s) to be excluded. |
244 | 280 | returns: dict: A dict containing settings keyed by setting name. | 284 | returns: dict: A dict containing settings keyed by setting name. |
246 | 281 | raises: HAIncompleteConfig if settings are missing. | 285 | raises: HAIncompleteConfig if settings are missing or incorrect. |
247 | 282 | ''' | 286 | ''' |
249 | 283 | settings = ['ha-bindiface', 'ha-mcastport', 'vip'] | 287 | settings = ['ha-bindiface', 'ha-mcastport', 'vip', 'os-internal-hostname', |
250 | 288 | 'os-admin-hostname', 'os-public-hostname', 'os-access-hostname'] | ||
251 | 284 | conf = {} | 289 | conf = {} |
252 | 285 | for setting in settings: | 290 | for setting in settings: |
253 | 286 | if exclude_keys and setting in exclude_keys: | 291 | if exclude_keys and setting in exclude_keys: |
254 | 287 | continue | 292 | continue |
255 | 288 | 293 | ||
256 | 289 | conf[setting] = config_get(setting) | 294 | conf[setting] = config_get(setting) |
262 | 290 | missing = [] | 295 | |
263 | 291 | [missing.append(s) for s, v in six.iteritems(conf) if v is None] | 296 | if not valid_hacluster_config(): |
264 | 292 | if missing: | 297 | raise HAIncorrectConfig('Insufficient or incorrect config data to ' |
265 | 293 | log('Insufficient config data to configure hacluster.', level=ERROR) | 298 | 'configure hacluster.') |
261 | 294 | raise HAIncompleteConfig | ||
266 | 295 | return conf | 299 | return conf |
267 | 296 | 300 | ||
268 | 297 | 301 | ||
269 | 302 | def valid_hacluster_config(): | ||
270 | 303 | ''' | ||
271 | 304 | Check that either vip or dns-ha is set. If dns-ha then one of os-*-hostname | ||
272 | 305 | must be set. | ||
273 | 306 | |||
274 | 307 | Note: ha-bindiface and ha-macastport both have defaults and will always | ||
275 | 308 | be set. We only care that either vip or dns-ha is set. | ||
276 | 309 | |||
277 | 310 | :returns: boolean: valid config returns true. | ||
278 | 311 | raises: HAIncompatibileConfig if settings conflict. | ||
279 | 312 | raises: HAIncompleteConfig if settings are missing. | ||
280 | 313 | ''' | ||
281 | 314 | vip = config_get('vip') | ||
282 | 315 | dns = config_get('dns-ha') | ||
283 | 316 | if not(bool(vip) ^ bool(dns)): | ||
284 | 317 | msg = ('HA: Either vip or dns-ha must be set but not both in order to ' | ||
285 | 318 | 'use high availability') | ||
286 | 319 | status_set('blocked', msg) | ||
287 | 320 | raise HAIncorrectConfig(msg) | ||
288 | 321 | |||
289 | 322 | # If dns-ha then one of os-*-hostname must be set | ||
290 | 323 | if dns: | ||
291 | 324 | dns_settings = ['os-internal-hostname', 'os-admin-hostname', | ||
292 | 325 | 'os-public-hostname', 'os-access-hostname'] | ||
293 | 326 | # At this point it is unknown if one or all of the possible | ||
294 | 327 | # network spaces are in HA. Validate at least one is set which is | ||
295 | 328 | # the minimum required. | ||
296 | 329 | for setting in dns_settings: | ||
297 | 330 | if config_get(setting): | ||
298 | 331 | log('DNS HA: At least one hostname is set {}: {}' | ||
299 | 332 | ''.format(setting, config_get(setting)), | ||
300 | 333 | level=DEBUG) | ||
301 | 334 | return True | ||
302 | 335 | |||
303 | 336 | msg = ('DNS HA: At least one os-*-hostname(s) must be set to use ' | ||
304 | 337 | 'DNS HA') | ||
305 | 338 | status_set('blocked', msg) | ||
306 | 339 | raise HAIncompleteConfig(msg) | ||
307 | 340 | |||
308 | 341 | log('VIP HA: VIP is set {}'.format(vip), level=DEBUG) | ||
309 | 342 | return True | ||
310 | 343 | |||
311 | 344 | |||
312 | 298 | def canonical_url(configs, vip_setting='vip'): | 345 | def canonical_url(configs, vip_setting='vip'): |
313 | 299 | ''' | 346 | ''' |
314 | 300 | Returns the correct HTTP URL to this host given the state of HTTPS | 347 | Returns the correct HTTP URL to this host given the state of HTTPS |
315 | 301 | 348 | ||
316 | === modified file 'charmhelpers/core/__init__.py' | |||
317 | --- charmhelpers/core/__init__.py 2015-01-28 08:59:02 +0000 | |||
318 | +++ charmhelpers/core/__init__.py 2017-03-04 02:50:20 +0000 | |||
319 | @@ -1,15 +1,13 @@ | |||
320 | 1 | # Copyright 2014-2015 Canonical Limited. | 1 | # Copyright 2014-2015 Canonical Limited. |
321 | 2 | # | 2 | # |
335 | 3 | # This file is part of charm-helpers. | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); |
336 | 4 | # | 4 | # you may not use this file except in compliance with the License. |
337 | 5 | # charm-helpers is free software: you can redistribute it and/or modify | 5 | # You may obtain a copy of the License at |
338 | 6 | # it under the terms of the GNU Lesser General Public License version 3 as | 6 | # |
339 | 7 | # published by the Free Software Foundation. | 7 | # http://www.apache.org/licenses/LICENSE-2.0 |
340 | 8 | # | 8 | # |
341 | 9 | # charm-helpers is distributed in the hope that it will be useful, | 9 | # Unless required by applicable law or agreed to in writing, software |
342 | 10 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 10 | # distributed under the License is distributed on an "AS IS" BASIS, |
343 | 11 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
344 | 12 | # GNU Lesser General Public License for more details. | 12 | # See the License for the specific language governing permissions and |
345 | 13 | # | 13 | # limitations under the License. |
333 | 14 | # You should have received a copy of the GNU Lesser General Public License | ||
334 | 15 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
346 | 16 | 14 | ||
347 | === modified file 'charmhelpers/core/decorators.py' | |||
348 | --- charmhelpers/core/decorators.py 2015-01-28 08:59:02 +0000 | |||
349 | +++ charmhelpers/core/decorators.py 2017-03-04 02:50:20 +0000 | |||
350 | @@ -1,18 +1,16 @@ | |||
351 | 1 | # Copyright 2014-2015 Canonical Limited. | 1 | # Copyright 2014-2015 Canonical Limited. |
352 | 2 | # | 2 | # |
366 | 3 | # This file is part of charm-helpers. | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); |
367 | 4 | # | 4 | # you may not use this file except in compliance with the License. |
368 | 5 | # charm-helpers is free software: you can redistribute it and/or modify | 5 | # You may obtain a copy of the License at |
369 | 6 | # it under the terms of the GNU Lesser General Public License version 3 as | 6 | # |
370 | 7 | # published by the Free Software Foundation. | 7 | # http://www.apache.org/licenses/LICENSE-2.0 |
371 | 8 | # | 8 | # |
372 | 9 | # charm-helpers is distributed in the hope that it will be useful, | 9 | # Unless required by applicable law or agreed to in writing, software |
373 | 10 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 10 | # distributed under the License is distributed on an "AS IS" BASIS, |
374 | 11 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
375 | 12 | # GNU Lesser General Public License for more details. | 12 | # See the License for the specific language governing permissions and |
376 | 13 | # | 13 | # limitations under the License. |
364 | 14 | # You should have received a copy of the GNU Lesser General Public License | ||
365 | 15 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
377 | 16 | 14 | ||
378 | 17 | # | 15 | # |
379 | 18 | # Copyright 2014 Canonical Ltd. | 16 | # Copyright 2014 Canonical Ltd. |
380 | 19 | 17 | ||
381 | === modified file 'charmhelpers/core/files.py' | |||
382 | --- charmhelpers/core/files.py 2015-12-11 15:23:38 +0000 | |||
383 | +++ charmhelpers/core/files.py 2017-03-04 02:50:20 +0000 | |||
384 | @@ -3,19 +3,17 @@ | |||
385 | 3 | 3 | ||
386 | 4 | # Copyright 2014-2015 Canonical Limited. | 4 | # Copyright 2014-2015 Canonical Limited. |
387 | 5 | # | 5 | # |
401 | 6 | # This file is part of charm-helpers. | 6 | # Licensed under the Apache License, Version 2.0 (the "License"); |
402 | 7 | # | 7 | # you may not use this file except in compliance with the License. |
403 | 8 | # charm-helpers is free software: you can redistribute it and/or modify | 8 | # You may obtain a copy of the License at |
404 | 9 | # it under the terms of the GNU Lesser General Public License version 3 as | 9 | # |
405 | 10 | # published by the Free Software Foundation. | 10 | # http://www.apache.org/licenses/LICENSE-2.0 |
406 | 11 | # | 11 | # |
407 | 12 | # charm-helpers is distributed in the hope that it will be useful, | 12 | # Unless required by applicable law or agreed to in writing, software |
408 | 13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 13 | # distributed under the License is distributed on an "AS IS" BASIS, |
409 | 14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 14 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
410 | 15 | # GNU Lesser General Public License for more details. | 15 | # See the License for the specific language governing permissions and |
411 | 16 | # | 16 | # limitations under the License. |
399 | 17 | # You should have received a copy of the GNU Lesser General Public License | ||
400 | 18 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
412 | 19 | 17 | ||
413 | 20 | __author__ = 'Jorge Niedbalski <niedbalski@ubuntu.com>' | 18 | __author__ = 'Jorge Niedbalski <niedbalski@ubuntu.com>' |
414 | 21 | 19 | ||
415 | 22 | 20 | ||
416 | === modified file 'charmhelpers/core/fstab.py' | |||
417 | --- charmhelpers/core/fstab.py 2015-03-12 11:42:26 +0000 | |||
418 | +++ charmhelpers/core/fstab.py 2017-03-04 02:50:20 +0000 | |||
419 | @@ -3,19 +3,17 @@ | |||
420 | 3 | 3 | ||
421 | 4 | # Copyright 2014-2015 Canonical Limited. | 4 | # Copyright 2014-2015 Canonical Limited. |
422 | 5 | # | 5 | # |
436 | 6 | # This file is part of charm-helpers. | 6 | # Licensed under the Apache License, Version 2.0 (the "License"); |
437 | 7 | # | 7 | # you may not use this file except in compliance with the License. |
438 | 8 | # charm-helpers is free software: you can redistribute it and/or modify | 8 | # You may obtain a copy of the License at |
439 | 9 | # it under the terms of the GNU Lesser General Public License version 3 as | 9 | # |
440 | 10 | # published by the Free Software Foundation. | 10 | # http://www.apache.org/licenses/LICENSE-2.0 |
441 | 11 | # | 11 | # |
442 | 12 | # charm-helpers is distributed in the hope that it will be useful, | 12 | # Unless required by applicable law or agreed to in writing, software |
443 | 13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 13 | # distributed under the License is distributed on an "AS IS" BASIS, |
444 | 14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 14 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
445 | 15 | # GNU Lesser General Public License for more details. | 15 | # See the License for the specific language governing permissions and |
446 | 16 | # | 16 | # limitations under the License. |
434 | 17 | # You should have received a copy of the GNU Lesser General Public License | ||
435 | 18 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
447 | 19 | 17 | ||
448 | 20 | import io | 18 | import io |
449 | 21 | import os | 19 | import os |
450 | 22 | 20 | ||
451 | === modified file 'charmhelpers/core/hookenv.py' | |||
452 | --- charmhelpers/core/hookenv.py 2016-05-06 07:12:40 +0000 | |||
453 | +++ charmhelpers/core/hookenv.py 2017-03-04 02:50:20 +0000 | |||
454 | @@ -1,18 +1,16 @@ | |||
455 | 1 | # Copyright 2014-2015 Canonical Limited. | 1 | # Copyright 2014-2015 Canonical Limited. |
456 | 2 | # | 2 | # |
470 | 3 | # This file is part of charm-helpers. | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); |
471 | 4 | # | 4 | # you may not use this file except in compliance with the License. |
472 | 5 | # charm-helpers is free software: you can redistribute it and/or modify | 5 | # You may obtain a copy of the License at |
473 | 6 | # it under the terms of the GNU Lesser General Public License version 3 as | 6 | # |
474 | 7 | # published by the Free Software Foundation. | 7 | # http://www.apache.org/licenses/LICENSE-2.0 |
475 | 8 | # | 8 | # |
476 | 9 | # charm-helpers is distributed in the hope that it will be useful, | 9 | # Unless required by applicable law or agreed to in writing, software |
477 | 10 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 10 | # distributed under the License is distributed on an "AS IS" BASIS, |
478 | 11 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
479 | 12 | # GNU Lesser General Public License for more details. | 12 | # See the License for the specific language governing permissions and |
480 | 13 | # | 13 | # limitations under the License. |
468 | 14 | # You should have received a copy of the GNU Lesser General Public License | ||
469 | 15 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
481 | 16 | 14 | ||
482 | 17 | "Interactions with the Juju environment" | 15 | "Interactions with the Juju environment" |
483 | 18 | # Copyright 2013 Canonical Ltd. | 16 | # Copyright 2013 Canonical Ltd. |
484 | @@ -334,6 +332,8 @@ | |||
485 | 334 | config_cmd_line = ['config-get'] | 332 | config_cmd_line = ['config-get'] |
486 | 335 | if scope is not None: | 333 | if scope is not None: |
487 | 336 | config_cmd_line.append(scope) | 334 | config_cmd_line.append(scope) |
488 | 335 | else: | ||
489 | 336 | config_cmd_line.append('--all') | ||
490 | 337 | config_cmd_line.append('--format=json') | 337 | config_cmd_line.append('--format=json') |
491 | 338 | try: | 338 | try: |
492 | 339 | config_data = json.loads( | 339 | config_data = json.loads( |
493 | @@ -616,6 +616,20 @@ | |||
494 | 616 | subprocess.check_call(_args) | 616 | subprocess.check_call(_args) |
495 | 617 | 617 | ||
496 | 618 | 618 | ||
497 | 619 | def open_ports(start, end, protocol="TCP"): | ||
498 | 620 | """Opens a range of service network ports""" | ||
499 | 621 | _args = ['open-port'] | ||
500 | 622 | _args.append('{}-{}/{}'.format(start, end, protocol)) | ||
501 | 623 | subprocess.check_call(_args) | ||
502 | 624 | |||
503 | 625 | |||
504 | 626 | def close_ports(start, end, protocol="TCP"): | ||
505 | 627 | """Close a range of service network ports""" | ||
506 | 628 | _args = ['close-port'] | ||
507 | 629 | _args.append('{}-{}/{}'.format(start, end, protocol)) | ||
508 | 630 | subprocess.check_call(_args) | ||
509 | 631 | |||
510 | 632 | |||
511 | 619 | @cached | 633 | @cached |
512 | 620 | def unit_get(attribute): | 634 | def unit_get(attribute): |
513 | 621 | """Get the unit ID for the remote unit""" | 635 | """Get the unit ID for the remote unit""" |
514 | @@ -845,6 +859,20 @@ | |||
515 | 845 | return inner_translate_exc1 | 859 | return inner_translate_exc1 |
516 | 846 | 860 | ||
517 | 847 | 861 | ||
518 | 862 | def application_version_set(version): | ||
519 | 863 | """Charm authors may trigger this command from any hook to output what | ||
520 | 864 | version of the application is running. This could be a package version, | ||
521 | 865 | for instance postgres version 9.5. It could also be a build number or | ||
522 | 866 | version control revision identifier, for instance git sha 6fb7ba68. """ | ||
523 | 867 | |||
524 | 868 | cmd = ['application-version-set'] | ||
525 | 869 | cmd.append(version) | ||
526 | 870 | try: | ||
527 | 871 | subprocess.check_call(cmd) | ||
528 | 872 | except OSError: | ||
529 | 873 | log("Application Version: {}".format(version)) | ||
530 | 874 | |||
531 | 875 | |||
532 | 848 | @translate_exc(from_exc=OSError, to_exc=NotImplementedError) | 876 | @translate_exc(from_exc=OSError, to_exc=NotImplementedError) |
533 | 849 | def is_leader(): | 877 | def is_leader(): |
534 | 850 | """Does the current unit hold the juju leadership | 878 | """Does the current unit hold the juju leadership |
535 | @@ -1006,4 +1034,35 @@ | |||
536 | 1006 | :raise: NotImplementedError if run on Juju < 2.0 | 1034 | :raise: NotImplementedError if run on Juju < 2.0 |
537 | 1007 | ''' | 1035 | ''' |
538 | 1008 | cmd = ['network-get', '--primary-address', binding] | 1036 | cmd = ['network-get', '--primary-address', binding] |
540 | 1009 | return subprocess.check_output(cmd).strip() | 1037 | return subprocess.check_output(cmd).decode('UTF-8').strip() |
541 | 1038 | |||
542 | 1039 | |||
543 | 1040 | def add_metric(*args, **kwargs): | ||
544 | 1041 | """Add metric values. Values may be expressed with keyword arguments. For | ||
545 | 1042 | metric names containing dashes, these may be expressed as one or more | ||
546 | 1043 | 'key=value' positional arguments. May only be called from the collect-metrics | ||
547 | 1044 | hook.""" | ||
548 | 1045 | _args = ['add-metric'] | ||
549 | 1046 | _kvpairs = [] | ||
550 | 1047 | _kvpairs.extend(args) | ||
551 | 1048 | _kvpairs.extend(['{}={}'.format(k, v) for k, v in kwargs.items()]) | ||
552 | 1049 | _args.extend(sorted(_kvpairs)) | ||
553 | 1050 | try: | ||
554 | 1051 | subprocess.check_call(_args) | ||
555 | 1052 | return | ||
556 | 1053 | except EnvironmentError as e: | ||
557 | 1054 | if e.errno != errno.ENOENT: | ||
558 | 1055 | raise | ||
559 | 1056 | log_message = 'add-metric failed: {}'.format(' '.join(_kvpairs)) | ||
560 | 1057 | log(log_message, level='INFO') | ||
561 | 1058 | |||
562 | 1059 | |||
563 | 1060 | def meter_status(): | ||
564 | 1061 | """Get the meter status, if running in the meter-status-changed hook.""" | ||
565 | 1062 | return os.environ.get('JUJU_METER_STATUS') | ||
566 | 1063 | |||
567 | 1064 | |||
568 | 1065 | def meter_info(): | ||
569 | 1066 | """Get the meter status information, if running in the meter-status-changed | ||
570 | 1067 | hook.""" | ||
571 | 1068 | return os.environ.get('JUJU_METER_INFO') | ||
572 | 1010 | 1069 | ||
573 | === modified file 'charmhelpers/core/host.py' | |||
574 | --- charmhelpers/core/host.py 2016-05-06 07:12:40 +0000 | |||
575 | +++ charmhelpers/core/host.py 2017-03-04 02:50:20 +0000 | |||
576 | @@ -1,18 +1,16 @@ | |||
577 | 1 | # Copyright 2014-2015 Canonical Limited. | 1 | # Copyright 2014-2015 Canonical Limited. |
578 | 2 | # | 2 | # |
592 | 3 | # This file is part of charm-helpers. | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); |
593 | 4 | # | 4 | # you may not use this file except in compliance with the License. |
594 | 5 | # charm-helpers is free software: you can redistribute it and/or modify | 5 | # You may obtain a copy of the License at |
595 | 6 | # it under the terms of the GNU Lesser General Public License version 3 as | 6 | # |
596 | 7 | # published by the Free Software Foundation. | 7 | # http://www.apache.org/licenses/LICENSE-2.0 |
597 | 8 | # | 8 | # |
598 | 9 | # charm-helpers is distributed in the hope that it will be useful, | 9 | # Unless required by applicable law or agreed to in writing, software |
599 | 10 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 10 | # distributed under the License is distributed on an "AS IS" BASIS, |
600 | 11 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
601 | 12 | # GNU Lesser General Public License for more details. | 12 | # See the License for the specific language governing permissions and |
602 | 13 | # | 13 | # limitations under the License. |
590 | 14 | # You should have received a copy of the GNU Lesser General Public License | ||
591 | 15 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
603 | 16 | 14 | ||
604 | 17 | """Tools for working with the host system""" | 15 | """Tools for working with the host system""" |
605 | 18 | # Copyright 2012 Canonical Ltd. | 16 | # Copyright 2012 Canonical Ltd. |
606 | @@ -32,46 +30,162 @@ | |||
607 | 32 | import hashlib | 30 | import hashlib |
608 | 33 | import functools | 31 | import functools |
609 | 34 | import itertools | 32 | import itertools |
610 | 33 | import six | ||
611 | 34 | |||
612 | 35 | from contextlib import contextmanager | 35 | from contextlib import contextmanager |
613 | 36 | from collections import OrderedDict | 36 | from collections import OrderedDict |
614 | 37 | |||
615 | 38 | import six | ||
616 | 39 | |||
617 | 40 | from .hookenv import log | 37 | from .hookenv import log |
618 | 41 | from .fstab import Fstab | 38 | from .fstab import Fstab |
633 | 42 | 39 | from charmhelpers.osplatform import get_platform | |
634 | 43 | 40 | ||
635 | 44 | def service_start(service_name): | 41 | __platform__ = get_platform() |
636 | 45 | """Start a system service""" | 42 | if __platform__ == "ubuntu": |
637 | 46 | return service('start', service_name) | 43 | from charmhelpers.core.host_factory.ubuntu import ( |
638 | 47 | 44 | service_available, | |
639 | 48 | 45 | add_new_group, | |
640 | 49 | def service_stop(service_name): | 46 | lsb_release, |
641 | 50 | """Stop a system service""" | 47 | cmp_pkgrevno, |
642 | 51 | return service('stop', service_name) | 48 | ) # flake8: noqa -- ignore F401 for this import |
643 | 52 | 49 | elif __platform__ == "centos": | |
644 | 53 | 50 | from charmhelpers.core.host_factory.centos import ( | |
645 | 54 | def service_restart(service_name): | 51 | service_available, |
646 | 55 | """Restart a system service""" | 52 | add_new_group, |
647 | 53 | lsb_release, | ||
648 | 54 | cmp_pkgrevno, | ||
649 | 55 | ) # flake8: noqa -- ignore F401 for this import | ||
650 | 56 | |||
651 | 57 | UPDATEDB_PATH = '/etc/updatedb.conf' | ||
652 | 58 | |||
653 | 59 | def service_start(service_name, **kwargs): | ||
654 | 60 | """Start a system service. | ||
655 | 61 | |||
656 | 62 | The specified service name is managed via the system level init system. | ||
657 | 63 | Some init systems (e.g. upstart) require that additional arguments be | ||
658 | 64 | provided in order to directly control service instances whereas other init | ||
659 | 65 | systems allow for addressing instances of a service directly by name (e.g. | ||
660 | 66 | systemd). | ||
661 | 67 | |||
662 | 68 | The kwargs allow for the additional parameters to be passed to underlying | ||
663 | 69 | init systems for those systems which require/allow for them. For example, | ||
664 | 70 | the ceph-osd upstart script requires the id parameter to be passed along | ||
665 | 71 | in order to identify which running daemon should be reloaded. The follow- | ||
666 | 72 | ing example stops the ceph-osd service for instance id=4: | ||
667 | 73 | |||
668 | 74 | service_stop('ceph-osd', id=4) | ||
669 | 75 | |||
670 | 76 | :param service_name: the name of the service to stop | ||
671 | 77 | :param **kwargs: additional parameters to pass to the init system when | ||
672 | 78 | managing services. These will be passed as key=value | ||
673 | 79 | parameters to the init system's commandline. kwargs | ||
674 | 80 | are ignored for systemd enabled systems. | ||
675 | 81 | """ | ||
676 | 82 | return service('start', service_name, **kwargs) | ||
677 | 83 | |||
678 | 84 | |||
679 | 85 | def service_stop(service_name, **kwargs): | ||
680 | 86 | """Stop a system service. | ||
681 | 87 | |||
682 | 88 | The specified service name is managed via the system level init system. | ||
683 | 89 | Some init systems (e.g. upstart) require that additional arguments be | ||
684 | 90 | provided in order to directly control service instances whereas other init | ||
685 | 91 | systems allow for addressing instances of a service directly by name (e.g. | ||
686 | 92 | systemd). | ||
687 | 93 | |||
688 | 94 | The kwargs allow for the additional parameters to be passed to underlying | ||
689 | 95 | init systems for those systems which require/allow for them. For example, | ||
690 | 96 | the ceph-osd upstart script requires the id parameter to be passed along | ||
691 | 97 | in order to identify which running daemon should be reloaded. The follow- | ||
692 | 98 | ing example stops the ceph-osd service for instance id=4: | ||
693 | 99 | |||
694 | 100 | service_stop('ceph-osd', id=4) | ||
695 | 101 | |||
696 | 102 | :param service_name: the name of the service to stop | ||
697 | 103 | :param **kwargs: additional parameters to pass to the init system when | ||
698 | 104 | managing services. These will be passed as key=value | ||
699 | 105 | parameters to the init system's commandline. kwargs | ||
700 | 106 | are ignored for systemd enabled systems. | ||
701 | 107 | """ | ||
702 | 108 | return service('stop', service_name, **kwargs) | ||
703 | 109 | |||
704 | 110 | |||
705 | 111 | def service_restart(service_name, **kwargs): | ||
706 | 112 | """Restart a system service. | ||
707 | 113 | |||
708 | 114 | The specified service name is managed via the system level init system. | ||
709 | 115 | Some init systems (e.g. upstart) require that additional arguments be | ||
710 | 116 | provided in order to directly control service instances whereas other init | ||
711 | 117 | systems allow for addressing instances of a service directly by name (e.g. | ||
712 | 118 | systemd). | ||
713 | 119 | |||
714 | 120 | The kwargs allow for the additional parameters to be passed to underlying | ||
715 | 121 | init systems for those systems which require/allow for them. For example, | ||
716 | 122 | the ceph-osd upstart script requires the id parameter to be passed along | ||
717 | 123 | in order to identify which running daemon should be restarted. The follow- | ||
718 | 124 | ing example restarts the ceph-osd service for instance id=4: | ||
719 | 125 | |||
720 | 126 | service_restart('ceph-osd', id=4) | ||
721 | 127 | |||
722 | 128 | :param service_name: the name of the service to restart | ||
723 | 129 | :param **kwargs: additional parameters to pass to the init system when | ||
724 | 130 | managing services. These will be passed as key=value | ||
725 | 131 | parameters to the init system's commandline. kwargs | ||
726 | 132 | are ignored for init systems not allowing additional | ||
727 | 133 | parameters via the commandline (systemd). | ||
728 | 134 | """ | ||
729 | 56 | return service('restart', service_name) | 135 | return service('restart', service_name) |
730 | 57 | 136 | ||
731 | 58 | 137 | ||
733 | 59 | def service_reload(service_name, restart_on_failure=False): | 138 | def service_reload(service_name, restart_on_failure=False, **kwargs): |
734 | 60 | """Reload a system service, optionally falling back to restart if | 139 | """Reload a system service, optionally falling back to restart if |
737 | 61 | reload fails""" | 140 | reload fails. |
738 | 62 | service_result = service('reload', service_name) | 141 | |
739 | 142 | The specified service name is managed via the system level init system. | ||
740 | 143 | Some init systems (e.g. upstart) require that additional arguments be | ||
741 | 144 | provided in order to directly control service instances whereas other init | ||
742 | 145 | systems allow for addressing instances of a service directly by name (e.g. | ||
743 | 146 | systemd). | ||
744 | 147 | |||
745 | 148 | The kwargs allow for the additional parameters to be passed to underlying | ||
746 | 149 | init systems for those systems which require/allow for them. For example, | ||
747 | 150 | the ceph-osd upstart script requires the id parameter to be passed along | ||
748 | 151 | in order to identify which running daemon should be reloaded. The follow- | ||
749 | 152 | ing example restarts the ceph-osd service for instance id=4: | ||
750 | 153 | |||
751 | 154 | service_reload('ceph-osd', id=4) | ||
752 | 155 | |||
753 | 156 | :param service_name: the name of the service to reload | ||
754 | 157 | :param restart_on_failure: boolean indicating whether to fallback to a | ||
755 | 158 | restart if the reload fails. | ||
756 | 159 | :param **kwargs: additional parameters to pass to the init system when | ||
757 | 160 | managing services. These will be passed as key=value | ||
758 | 161 | parameters to the init system's commandline. kwargs | ||
759 | 162 | are ignored for init systems not allowing additional | ||
760 | 163 | parameters via the commandline (systemd). | ||
761 | 164 | """ | ||
762 | 165 | service_result = service('reload', service_name, **kwargs) | ||
763 | 63 | if not service_result and restart_on_failure: | 166 | if not service_result and restart_on_failure: |
765 | 64 | service_result = service('restart', service_name) | 167 | service_result = service('restart', service_name, **kwargs) |
766 | 65 | return service_result | 168 | return service_result |
767 | 66 | 169 | ||
768 | 67 | 170 | ||
770 | 68 | def service_pause(service_name, init_dir="/etc/init", initd_dir="/etc/init.d"): | 171 | def service_pause(service_name, init_dir="/etc/init", initd_dir="/etc/init.d", |
771 | 172 | **kwargs): | ||
772 | 69 | """Pause a system service. | 173 | """Pause a system service. |
773 | 70 | 174 | ||
775 | 71 | Stop it, and prevent it from starting again at boot.""" | 175 | Stop it, and prevent it from starting again at boot. |
776 | 176 | |||
777 | 177 | :param service_name: the name of the service to pause | ||
778 | 178 | :param init_dir: path to the upstart init directory | ||
779 | 179 | :param initd_dir: path to the sysv init directory | ||
780 | 180 | :param **kwargs: additional parameters to pass to the init system when | ||
781 | 181 | managing services. These will be passed as key=value | ||
782 | 182 | parameters to the init system's commandline. kwargs | ||
783 | 183 | are ignored for init systems which do not support | ||
784 | 184 | key=value arguments via the commandline. | ||
785 | 185 | """ | ||
786 | 72 | stopped = True | 186 | stopped = True |
789 | 73 | if service_running(service_name): | 187 | if service_running(service_name, **kwargs): |
790 | 74 | stopped = service_stop(service_name) | 188 | stopped = service_stop(service_name, **kwargs) |
791 | 75 | upstart_file = os.path.join(init_dir, "{}.conf".format(service_name)) | 189 | upstart_file = os.path.join(init_dir, "{}.conf".format(service_name)) |
792 | 76 | sysv_file = os.path.join(initd_dir, service_name) | 190 | sysv_file = os.path.join(initd_dir, service_name) |
793 | 77 | if init_is_systemd(): | 191 | if init_is_systemd(): |
794 | @@ -92,10 +206,19 @@ | |||
795 | 92 | 206 | ||
796 | 93 | 207 | ||
797 | 94 | def service_resume(service_name, init_dir="/etc/init", | 208 | def service_resume(service_name, init_dir="/etc/init", |
799 | 95 | initd_dir="/etc/init.d"): | 209 | initd_dir="/etc/init.d", **kwargs): |
800 | 96 | """Resume a system service. | 210 | """Resume a system service. |
801 | 97 | 211 | ||
803 | 98 | Reenable starting again at boot. Start the service""" | 212 | Reenable starting again at boot. Start the service. |
804 | 213 | |||
805 | 214 | :param service_name: the name of the service to resume | ||
806 | 215 | :param init_dir: the path to the init dir | ||
807 | 216 | :param initd dir: the path to the initd dir | ||
808 | 217 | :param **kwargs: additional parameters to pass to the init system when | ||
809 | 218 | managing services. These will be passed as key=value | ||
810 | 219 | parameters to the init system's commandline. kwargs | ||
811 | 220 | are ignored for systemd enabled systems. | ||
812 | 221 | """ | ||
813 | 99 | upstart_file = os.path.join(init_dir, "{}.conf".format(service_name)) | 222 | upstart_file = os.path.join(init_dir, "{}.conf".format(service_name)) |
814 | 100 | sysv_file = os.path.join(initd_dir, service_name) | 223 | sysv_file = os.path.join(initd_dir, service_name) |
815 | 101 | if init_is_systemd(): | 224 | if init_is_systemd(): |
816 | @@ -112,62 +235,70 @@ | |||
817 | 112 | "Unable to detect {0} as SystemD, Upstart {1} or" | 235 | "Unable to detect {0} as SystemD, Upstart {1} or" |
818 | 113 | " SysV {2}".format( | 236 | " SysV {2}".format( |
819 | 114 | service_name, upstart_file, sysv_file)) | 237 | service_name, upstart_file, sysv_file)) |
820 | 238 | started = service_running(service_name, **kwargs) | ||
821 | 115 | 239 | ||
822 | 116 | started = service_running(service_name) | ||
823 | 117 | if not started: | 240 | if not started: |
825 | 118 | started = service_start(service_name) | 241 | started = service_start(service_name, **kwargs) |
826 | 119 | return started | 242 | return started |
827 | 120 | 243 | ||
828 | 121 | 244 | ||
831 | 122 | def service(action, service_name): | 245 | def service(action, service_name, **kwargs): |
832 | 123 | """Control a system service""" | 246 | """Control a system service. |
833 | 247 | |||
834 | 248 | :param action: the action to take on the service | ||
835 | 249 | :param service_name: the name of the service to perform th action on | ||
836 | 250 | :param **kwargs: additional params to be passed to the service command in | ||
837 | 251 | the form of key=value. | ||
838 | 252 | """ | ||
839 | 124 | if init_is_systemd(): | 253 | if init_is_systemd(): |
840 | 125 | cmd = ['systemctl', action, service_name] | 254 | cmd = ['systemctl', action, service_name] |
841 | 126 | else: | 255 | else: |
842 | 127 | cmd = ['service', service_name, action] | 256 | cmd = ['service', service_name, action] |
843 | 257 | for key, value in six.iteritems(kwargs): | ||
844 | 258 | parameter = '%s=%s' % (key, value) | ||
845 | 259 | cmd.append(parameter) | ||
846 | 128 | return subprocess.call(cmd) == 0 | 260 | return subprocess.call(cmd) == 0 |
847 | 129 | 261 | ||
848 | 130 | 262 | ||
858 | 131 | def systemv_services_running(): | 263 | _UPSTART_CONF = "/etc/init/{}.conf" |
859 | 132 | output = subprocess.check_output( | 264 | _INIT_D_CONF = "/etc/init.d/{}" |
860 | 133 | ['service', '--status-all'], | 265 | |
861 | 134 | stderr=subprocess.STDOUT).decode('UTF-8') | 266 | |
862 | 135 | return [row.split()[-1] for row in output.split('\n') if '[ + ]' in row] | 267 | def service_running(service_name, **kwargs): |
863 | 136 | 268 | """Determine whether a system service is running. | |
864 | 137 | 269 | ||
865 | 138 | def service_running(service_name): | 270 | :param service_name: the name of the service |
866 | 139 | """Determine whether a system service is running""" | 271 | :param **kwargs: additional args to pass to the service command. This is |
867 | 272 | used to pass additional key=value arguments to the | ||
868 | 273 | service command line for managing specific instance | ||
869 | 274 | units (e.g. service ceph-osd status id=2). The kwargs | ||
870 | 275 | are ignored in systemd services. | ||
871 | 276 | """ | ||
872 | 140 | if init_is_systemd(): | 277 | if init_is_systemd(): |
873 | 141 | return service('is-active', service_name) | 278 | return service('is-active', service_name) |
874 | 142 | else: | 279 | else: |
887 | 143 | try: | 280 | if os.path.exists(_UPSTART_CONF.format(service_name)): |
888 | 144 | output = subprocess.check_output( | 281 | try: |
889 | 145 | ['service', service_name, 'status'], | 282 | cmd = ['status', service_name] |
890 | 146 | stderr=subprocess.STDOUT).decode('UTF-8') | 283 | for key, value in six.iteritems(kwargs): |
891 | 147 | except subprocess.CalledProcessError: | 284 | parameter = '%s=%s' % (key, value) |
892 | 148 | return False | 285 | cmd.append(parameter) |
893 | 149 | else: | 286 | output = subprocess.check_output(cmd, |
894 | 150 | # This works for upstart scripts where the 'service' command | 287 | stderr=subprocess.STDOUT).decode('UTF-8') |
895 | 151 | # returns a consistent string to represent running 'start/running' | 288 | except subprocess.CalledProcessError: |
896 | 152 | if ("start/running" in output or "is running" in output or | 289 | return False |
897 | 153 | "up and running" in output): | 290 | else: |
898 | 154 | return True | 291 | # This works for upstart scripts where the 'service' command |
899 | 292 | # returns a consistent string to represent running | ||
900 | 293 | # 'start/running' | ||
901 | 294 | if ("start/running" in output or | ||
902 | 295 | "is running" in output or | ||
903 | 296 | "up and running" in output): | ||
904 | 297 | return True | ||
905 | 298 | elif os.path.exists(_INIT_D_CONF.format(service_name)): | ||
906 | 155 | # Check System V scripts init script return codes | 299 | # Check System V scripts init script return codes |
922 | 156 | if service_name in systemv_services_running(): | 300 | return service('status', service_name) |
923 | 157 | return True | 301 | return False |
909 | 158 | return False | ||
910 | 159 | |||
911 | 160 | |||
912 | 161 | def service_available(service_name): | ||
913 | 162 | """Determine whether a system service is available""" | ||
914 | 163 | try: | ||
915 | 164 | subprocess.check_output( | ||
916 | 165 | ['service', service_name, 'status'], | ||
917 | 166 | stderr=subprocess.STDOUT).decode('UTF-8') | ||
918 | 167 | except subprocess.CalledProcessError as e: | ||
919 | 168 | return b'unrecognized service' not in e.output | ||
920 | 169 | else: | ||
921 | 170 | return True | ||
924 | 171 | 302 | ||
925 | 172 | 303 | ||
926 | 173 | SYSTEMD_SYSTEM = '/run/systemd/system' | 304 | SYSTEMD_SYSTEM = '/run/systemd/system' |
927 | @@ -178,8 +309,9 @@ | |||
928 | 178 | return os.path.isdir(SYSTEMD_SYSTEM) | 309 | return os.path.isdir(SYSTEMD_SYSTEM) |
929 | 179 | 310 | ||
930 | 180 | 311 | ||
933 | 181 | def adduser(username, password=None, shell='/bin/bash', system_user=False, | 312 | def adduser(username, password=None, shell='/bin/bash', |
934 | 182 | primary_group=None, secondary_groups=None): | 313 | system_user=False, primary_group=None, |
935 | 314 | secondary_groups=None, uid=None, home_dir=None): | ||
936 | 183 | """Add a user to the system. | 315 | """Add a user to the system. |
937 | 184 | 316 | ||
938 | 185 | Will log but otherwise succeed if the user already exists. | 317 | Will log but otherwise succeed if the user already exists. |
939 | @@ -190,15 +322,24 @@ | |||
940 | 190 | :param bool system_user: Whether to create a login or system user | 322 | :param bool system_user: Whether to create a login or system user |
941 | 191 | :param str primary_group: Primary group for user; defaults to username | 323 | :param str primary_group: Primary group for user; defaults to username |
942 | 192 | :param list secondary_groups: Optional list of additional groups | 324 | :param list secondary_groups: Optional list of additional groups |
943 | 325 | :param int uid: UID for user being created | ||
944 | 326 | :param str home_dir: Home directory for user | ||
945 | 193 | 327 | ||
946 | 194 | :returns: The password database entry struct, as returned by `pwd.getpwnam` | 328 | :returns: The password database entry struct, as returned by `pwd.getpwnam` |
947 | 195 | """ | 329 | """ |
948 | 196 | try: | 330 | try: |
949 | 197 | user_info = pwd.getpwnam(username) | 331 | user_info = pwd.getpwnam(username) |
950 | 198 | log('user {0} already exists!'.format(username)) | 332 | log('user {0} already exists!'.format(username)) |
951 | 333 | if uid: | ||
952 | 334 | user_info = pwd.getpwuid(int(uid)) | ||
953 | 335 | log('user with uid {0} already exists!'.format(uid)) | ||
954 | 199 | except KeyError: | 336 | except KeyError: |
955 | 200 | log('creating user {0}'.format(username)) | 337 | log('creating user {0}'.format(username)) |
956 | 201 | cmd = ['useradd'] | 338 | cmd = ['useradd'] |
957 | 339 | if uid: | ||
958 | 340 | cmd.extend(['--uid', str(uid)]) | ||
959 | 341 | if home_dir: | ||
960 | 342 | cmd.extend(['--home', str(home_dir)]) | ||
961 | 202 | if system_user or password is None: | 343 | if system_user or password is None: |
962 | 203 | cmd.append('--system') | 344 | cmd.append('--system') |
963 | 204 | else: | 345 | else: |
964 | @@ -233,22 +374,56 @@ | |||
965 | 233 | return user_exists | 374 | return user_exists |
966 | 234 | 375 | ||
967 | 235 | 376 | ||
970 | 236 | def add_group(group_name, system_group=False): | 377 | def uid_exists(uid): |
971 | 237 | """Add a group to the system""" | 378 | """Check if a uid exists""" |
972 | 379 | try: | ||
973 | 380 | pwd.getpwuid(uid) | ||
974 | 381 | uid_exists = True | ||
975 | 382 | except KeyError: | ||
976 | 383 | uid_exists = False | ||
977 | 384 | return uid_exists | ||
978 | 385 | |||
979 | 386 | |||
980 | 387 | def group_exists(groupname): | ||
981 | 388 | """Check if a group exists""" | ||
982 | 389 | try: | ||
983 | 390 | grp.getgrnam(groupname) | ||
984 | 391 | group_exists = True | ||
985 | 392 | except KeyError: | ||
986 | 393 | group_exists = False | ||
987 | 394 | return group_exists | ||
988 | 395 | |||
989 | 396 | |||
990 | 397 | def gid_exists(gid): | ||
991 | 398 | """Check if a gid exists""" | ||
992 | 399 | try: | ||
993 | 400 | grp.getgrgid(gid) | ||
994 | 401 | gid_exists = True | ||
995 | 402 | except KeyError: | ||
996 | 403 | gid_exists = False | ||
997 | 404 | return gid_exists | ||
998 | 405 | |||
999 | 406 | |||
1000 | 407 | def add_group(group_name, system_group=False, gid=None): | ||
1001 | 408 | """Add a group to the system | ||
1002 | 409 | |||
1003 | 410 | Will log but otherwise succeed if the group already exists. | ||
1004 | 411 | |||
1005 | 412 | :param str group_name: group to create | ||
1006 | 413 | :param bool system_group: Create system group | ||
1007 | 414 | :param int gid: GID for user being created | ||
1008 | 415 | |||
1009 | 416 | :returns: The password database entry struct, as returned by `grp.getgrnam` | ||
1010 | 417 | """ | ||
1011 | 238 | try: | 418 | try: |
1012 | 239 | group_info = grp.getgrnam(group_name) | 419 | group_info = grp.getgrnam(group_name) |
1013 | 240 | log('group {0} already exists!'.format(group_name)) | 420 | log('group {0} already exists!'.format(group_name)) |
1014 | 421 | if gid: | ||
1015 | 422 | group_info = grp.getgrgid(gid) | ||
1016 | 423 | log('group with gid {0} already exists!'.format(gid)) | ||
1017 | 241 | except KeyError: | 424 | except KeyError: |
1018 | 242 | log('creating group {0}'.format(group_name)) | 425 | log('creating group {0}'.format(group_name)) |
1028 | 243 | cmd = ['addgroup'] | 426 | add_new_group(group_name, system_group, gid) |
1020 | 244 | if system_group: | ||
1021 | 245 | cmd.append('--system') | ||
1022 | 246 | else: | ||
1023 | 247 | cmd.extend([ | ||
1024 | 248 | '--group', | ||
1025 | 249 | ]) | ||
1026 | 250 | cmd.append(group_name) | ||
1027 | 251 | subprocess.check_call(cmd) | ||
1029 | 252 | group_info = grp.getgrnam(group_name) | 427 | group_info = grp.getgrnam(group_name) |
1030 | 253 | return group_info | 428 | return group_info |
1031 | 254 | 429 | ||
1032 | @@ -260,15 +435,17 @@ | |||
1033 | 260 | subprocess.check_call(cmd) | 435 | subprocess.check_call(cmd) |
1034 | 261 | 436 | ||
1035 | 262 | 437 | ||
1037 | 263 | def rsync(from_path, to_path, flags='-r', options=None): | 438 | def rsync(from_path, to_path, flags='-r', options=None, timeout=None): |
1038 | 264 | """Replicate the contents of a path""" | 439 | """Replicate the contents of a path""" |
1039 | 265 | options = options or ['--delete', '--executability'] | 440 | options = options or ['--delete', '--executability'] |
1040 | 266 | cmd = ['/usr/bin/rsync', flags] | 441 | cmd = ['/usr/bin/rsync', flags] |
1041 | 442 | if timeout: | ||
1042 | 443 | cmd = ['timeout', str(timeout)] + cmd | ||
1043 | 267 | cmd.extend(options) | 444 | cmd.extend(options) |
1044 | 268 | cmd.append(from_path) | 445 | cmd.append(from_path) |
1045 | 269 | cmd.append(to_path) | 446 | cmd.append(to_path) |
1046 | 270 | log(" ".join(cmd)) | 447 | log(" ".join(cmd)) |
1048 | 271 | return subprocess.check_output(cmd).decode('UTF-8').strip() | 448 | return subprocess.check_output(cmd, stderr=subprocess.STDOUT).decode('UTF-8').strip() |
1049 | 272 | 449 | ||
1050 | 273 | 450 | ||
1051 | 274 | def symlink(source, destination): | 451 | def symlink(source, destination): |
1052 | @@ -493,16 +670,6 @@ | |||
1053 | 493 | return r | 670 | return r |
1054 | 494 | 671 | ||
1055 | 495 | 672 | ||
1056 | 496 | def lsb_release(): | ||
1057 | 497 | """Return /etc/lsb-release in a dict""" | ||
1058 | 498 | d = {} | ||
1059 | 499 | with open('/etc/lsb-release', 'r') as lsb: | ||
1060 | 500 | for l in lsb: | ||
1061 | 501 | k, v = l.split('=') | ||
1062 | 502 | d[k.strip()] = v.strip() | ||
1063 | 503 | return d | ||
1064 | 504 | |||
1065 | 505 | |||
1066 | 506 | def pwgen(length=None): | 673 | def pwgen(length=None): |
1067 | 507 | """Generate a random pasword.""" | 674 | """Generate a random pasword.""" |
1068 | 508 | if length is None: | 675 | if length is None: |
1069 | @@ -626,25 +793,6 @@ | |||
1070 | 626 | return hwaddr | 793 | return hwaddr |
1071 | 627 | 794 | ||
1072 | 628 | 795 | ||
1073 | 629 | def cmp_pkgrevno(package, revno, pkgcache=None): | ||
1074 | 630 | """Compare supplied revno with the revno of the installed package | ||
1075 | 631 | |||
1076 | 632 | * 1 => Installed revno is greater than supplied arg | ||
1077 | 633 | * 0 => Installed revno is the same as supplied arg | ||
1078 | 634 | * -1 => Installed revno is less than supplied arg | ||
1079 | 635 | |||
1080 | 636 | This function imports apt_cache function from charmhelpers.fetch if | ||
1081 | 637 | the pkgcache argument is None. Be sure to add charmhelpers.fetch if | ||
1082 | 638 | you call this function, or pass an apt_pkg.Cache() instance. | ||
1083 | 639 | """ | ||
1084 | 640 | import apt_pkg | ||
1085 | 641 | if not pkgcache: | ||
1086 | 642 | from charmhelpers.fetch import apt_cache | ||
1087 | 643 | pkgcache = apt_cache() | ||
1088 | 644 | pkg = pkgcache[package] | ||
1089 | 645 | return apt_pkg.version_compare(pkg.current_ver.ver_str, revno) | ||
1090 | 646 | |||
1091 | 647 | |||
1092 | 648 | @contextmanager | 796 | @contextmanager |
1093 | 649 | def chdir(directory): | 797 | def chdir(directory): |
1094 | 650 | """Change the current working directory to a different directory for a code | 798 | """Change the current working directory to a different directory for a code |
1095 | @@ -667,7 +815,7 @@ | |||
1096 | 667 | :param str path: The string path to start changing ownership. | 815 | :param str path: The string path to start changing ownership. |
1097 | 668 | :param str owner: The owner string to use when looking up the uid. | 816 | :param str owner: The owner string to use when looking up the uid. |
1098 | 669 | :param str group: The group string to use when looking up the gid. | 817 | :param str group: The group string to use when looking up the gid. |
1100 | 670 | :param bool follow_links: Also Chown links if True | 818 | :param bool follow_links: Also follow and chown links if True |
1101 | 671 | :param bool chowntopdir: Also chown path itself if True | 819 | :param bool chowntopdir: Also chown path itself if True |
1102 | 672 | """ | 820 | """ |
1103 | 673 | uid = pwd.getpwnam(owner).pw_uid | 821 | uid = pwd.getpwnam(owner).pw_uid |
1104 | @@ -681,7 +829,7 @@ | |||
1105 | 681 | broken_symlink = os.path.lexists(path) and not os.path.exists(path) | 829 | broken_symlink = os.path.lexists(path) and not os.path.exists(path) |
1106 | 682 | if not broken_symlink: | 830 | if not broken_symlink: |
1107 | 683 | chown(path, uid, gid) | 831 | chown(path, uid, gid) |
1109 | 684 | for root, dirs, files in os.walk(path): | 832 | for root, dirs, files in os.walk(path, followlinks=follow_links): |
1110 | 685 | for name in dirs + files: | 833 | for name in dirs + files: |
1111 | 686 | full = os.path.join(root, name) | 834 | full = os.path.join(root, name) |
1112 | 687 | broken_symlink = os.path.lexists(full) and not os.path.exists(full) | 835 | broken_symlink = os.path.lexists(full) and not os.path.exists(full) |
1113 | @@ -701,6 +849,20 @@ | |||
1114 | 701 | chownr(path, owner, group, follow_links=False) | 849 | chownr(path, owner, group, follow_links=False) |
1115 | 702 | 850 | ||
1116 | 703 | 851 | ||
1117 | 852 | def owner(path): | ||
1118 | 853 | """Returns a tuple containing the username & groupname owning the path. | ||
1119 | 854 | |||
1120 | 855 | :param str path: the string path to retrieve the ownership | ||
1121 | 856 | :return tuple(str, str): A (username, groupname) tuple containing the | ||
1122 | 857 | name of the user and group owning the path. | ||
1123 | 858 | :raises OSError: if the specified path does not exist | ||
1124 | 859 | """ | ||
1125 | 860 | stat = os.stat(path) | ||
1126 | 861 | username = pwd.getpwuid(stat.st_uid)[0] | ||
1127 | 862 | groupname = grp.getgrgid(stat.st_gid)[0] | ||
1128 | 863 | return username, groupname | ||
1129 | 864 | |||
1130 | 865 | |||
1131 | 704 | def get_total_ram(): | 866 | def get_total_ram(): |
1132 | 705 | """The total amount of system RAM in bytes. | 867 | """The total amount of system RAM in bytes. |
1133 | 706 | 868 | ||
1134 | @@ -715,3 +877,42 @@ | |||
1135 | 715 | assert unit == 'kB', 'Unknown unit' | 877 | assert unit == 'kB', 'Unknown unit' |
1136 | 716 | return int(value) * 1024 # Classic, not KiB. | 878 | return int(value) * 1024 # Classic, not KiB. |
1137 | 717 | raise NotImplementedError() | 879 | raise NotImplementedError() |
1138 | 880 | |||
1139 | 881 | |||
1140 | 882 | UPSTART_CONTAINER_TYPE = '/run/container_type' | ||
1141 | 883 | |||
1142 | 884 | |||
1143 | 885 | def is_container(): | ||
1144 | 886 | """Determine whether unit is running in a container | ||
1145 | 887 | |||
1146 | 888 | @return: boolean indicating if unit is in a container | ||
1147 | 889 | """ | ||
1148 | 890 | if init_is_systemd(): | ||
1149 | 891 | # Detect using systemd-detect-virt | ||
1150 | 892 | return subprocess.call(['systemd-detect-virt', | ||
1151 | 893 | '--container']) == 0 | ||
1152 | 894 | else: | ||
1153 | 895 | # Detect using upstart container file marker | ||
1154 | 896 | return os.path.exists(UPSTART_CONTAINER_TYPE) | ||
1155 | 897 | |||
1156 | 898 | |||
1157 | 899 | def add_to_updatedb_prunepath(path, updatedb_path=UPDATEDB_PATH): | ||
1158 | 900 | with open(updatedb_path, 'r+') as f_id: | ||
1159 | 901 | updatedb_text = f_id.read() | ||
1160 | 902 | output = updatedb(updatedb_text, path) | ||
1161 | 903 | f_id.seek(0) | ||
1162 | 904 | f_id.write(output) | ||
1163 | 905 | f_id.truncate() | ||
1164 | 906 | |||
1165 | 907 | |||
1166 | 908 | def updatedb(updatedb_text, new_path): | ||
1167 | 909 | lines = [line for line in updatedb_text.split("\n")] | ||
1168 | 910 | for i, line in enumerate(lines): | ||
1169 | 911 | if line.startswith("PRUNEPATHS="): | ||
1170 | 912 | paths_line = line.split("=")[1].replace('"', '') | ||
1171 | 913 | paths = paths_line.split(" ") | ||
1172 | 914 | if new_path not in paths: | ||
1173 | 915 | paths.append(new_path) | ||
1174 | 916 | lines[i] = 'PRUNEPATHS="{}"'.format(' '.join(paths)) | ||
1175 | 917 | output = "\n".join(lines) | ||
1176 | 918 | return output | ||
1177 | 718 | 919 | ||
1178 | === added directory 'charmhelpers/core/host_factory' | |||
1179 | === added file 'charmhelpers/core/host_factory/__init__.py' | |||
1180 | === added file 'charmhelpers/core/host_factory/centos.py' | |||
1181 | --- charmhelpers/core/host_factory/centos.py 1970-01-01 00:00:00 +0000 | |||
1182 | +++ charmhelpers/core/host_factory/centos.py 2017-03-04 02:50:20 +0000 | |||
1183 | @@ -0,0 +1,56 @@ | |||
1184 | 1 | import subprocess | ||
1185 | 2 | import yum | ||
1186 | 3 | import os | ||
1187 | 4 | |||
1188 | 5 | |||
1189 | 6 | def service_available(service_name): | ||
1190 | 7 | # """Determine whether a system service is available.""" | ||
1191 | 8 | if os.path.isdir('/run/systemd/system'): | ||
1192 | 9 | cmd = ['systemctl', 'is-enabled', service_name] | ||
1193 | 10 | else: | ||
1194 | 11 | cmd = ['service', service_name, 'is-enabled'] | ||
1195 | 12 | return subprocess.call(cmd) == 0 | ||
1196 | 13 | |||
1197 | 14 | |||
1198 | 15 | def add_new_group(group_name, system_group=False, gid=None): | ||
1199 | 16 | cmd = ['groupadd'] | ||
1200 | 17 | if gid: | ||
1201 | 18 | cmd.extend(['--gid', str(gid)]) | ||
1202 | 19 | if system_group: | ||
1203 | 20 | cmd.append('-r') | ||
1204 | 21 | cmd.append(group_name) | ||
1205 | 22 | subprocess.check_call(cmd) | ||
1206 | 23 | |||
1207 | 24 | |||
1208 | 25 | def lsb_release(): | ||
1209 | 26 | """Return /etc/os-release in a dict.""" | ||
1210 | 27 | d = {} | ||
1211 | 28 | with open('/etc/os-release', 'r') as lsb: | ||
1212 | 29 | for l in lsb: | ||
1213 | 30 | s = l.split('=') | ||
1214 | 31 | if len(s) != 2: | ||
1215 | 32 | continue | ||
1216 | 33 | d[s[0].strip()] = s[1].strip() | ||
1217 | 34 | return d | ||
1218 | 35 | |||
1219 | 36 | |||
1220 | 37 | def cmp_pkgrevno(package, revno, pkgcache=None): | ||
1221 | 38 | """Compare supplied revno with the revno of the installed package. | ||
1222 | 39 | |||
1223 | 40 | * 1 => Installed revno is greater than supplied arg | ||
1224 | 41 | * 0 => Installed revno is the same as supplied arg | ||
1225 | 42 | * -1 => Installed revno is less than supplied arg | ||
1226 | 43 | |||
1227 | 44 | This function imports YumBase function if the pkgcache argument | ||
1228 | 45 | is None. | ||
1229 | 46 | """ | ||
1230 | 47 | if not pkgcache: | ||
1231 | 48 | y = yum.YumBase() | ||
1232 | 49 | packages = y.doPackageLists() | ||
1233 | 50 | pkgcache = {i.Name: i.version for i in packages['installed']} | ||
1234 | 51 | pkg = pkgcache[package] | ||
1235 | 52 | if pkg > revno: | ||
1236 | 53 | return 1 | ||
1237 | 54 | if pkg < revno: | ||
1238 | 55 | return -1 | ||
1239 | 56 | return 0 | ||
1240 | 0 | 57 | ||
1241 | === added file 'charmhelpers/core/host_factory/ubuntu.py' | |||
1242 | --- charmhelpers/core/host_factory/ubuntu.py 1970-01-01 00:00:00 +0000 | |||
1243 | +++ charmhelpers/core/host_factory/ubuntu.py 2017-03-04 02:50:20 +0000 | |||
1244 | @@ -0,0 +1,56 @@ | |||
1245 | 1 | import subprocess | ||
1246 | 2 | |||
1247 | 3 | |||
1248 | 4 | def service_available(service_name): | ||
1249 | 5 | """Determine whether a system service is available""" | ||
1250 | 6 | try: | ||
1251 | 7 | subprocess.check_output( | ||
1252 | 8 | ['service', service_name, 'status'], | ||
1253 | 9 | stderr=subprocess.STDOUT).decode('UTF-8') | ||
1254 | 10 | except subprocess.CalledProcessError as e: | ||
1255 | 11 | return b'unrecognized service' not in e.output | ||
1256 | 12 | else: | ||
1257 | 13 | return True | ||
1258 | 14 | |||
1259 | 15 | |||
1260 | 16 | def add_new_group(group_name, system_group=False, gid=None): | ||
1261 | 17 | cmd = ['addgroup'] | ||
1262 | 18 | if gid: | ||
1263 | 19 | cmd.extend(['--gid', str(gid)]) | ||
1264 | 20 | if system_group: | ||
1265 | 21 | cmd.append('--system') | ||
1266 | 22 | else: | ||
1267 | 23 | cmd.extend([ | ||
1268 | 24 | '--group', | ||
1269 | 25 | ]) | ||
1270 | 26 | cmd.append(group_name) | ||
1271 | 27 | subprocess.check_call(cmd) | ||
1272 | 28 | |||
1273 | 29 | |||
1274 | 30 | def lsb_release(): | ||
1275 | 31 | """Return /etc/lsb-release in a dict""" | ||
1276 | 32 | d = {} | ||
1277 | 33 | with open('/etc/lsb-release', 'r') as lsb: | ||
1278 | 34 | for l in lsb: | ||
1279 | 35 | k, v = l.split('=') | ||
1280 | 36 | d[k.strip()] = v.strip() | ||
1281 | 37 | return d | ||
1282 | 38 | |||
1283 | 39 | |||
1284 | 40 | def cmp_pkgrevno(package, revno, pkgcache=None): | ||
1285 | 41 | """Compare supplied revno with the revno of the installed package. | ||
1286 | 42 | |||
1287 | 43 | * 1 => Installed revno is greater than supplied arg | ||
1288 | 44 | * 0 => Installed revno is the same as supplied arg | ||
1289 | 45 | * -1 => Installed revno is less than supplied arg | ||
1290 | 46 | |||
1291 | 47 | This function imports apt_cache function from charmhelpers.fetch if | ||
1292 | 48 | the pkgcache argument is None. Be sure to add charmhelpers.fetch if | ||
1293 | 49 | you call this function, or pass an apt_pkg.Cache() instance. | ||
1294 | 50 | """ | ||
1295 | 51 | import apt_pkg | ||
1296 | 52 | if not pkgcache: | ||
1297 | 53 | from charmhelpers.fetch import apt_cache | ||
1298 | 54 | pkgcache = apt_cache() | ||
1299 | 55 | pkg = pkgcache[package] | ||
1300 | 56 | return apt_pkg.version_compare(pkg.current_ver.ver_str, revno) | ||
1301 | 0 | 57 | ||
1302 | === modified file 'charmhelpers/core/hugepage.py' | |||
1303 | --- charmhelpers/core/hugepage.py 2015-12-11 15:23:38 +0000 | |||
1304 | +++ charmhelpers/core/hugepage.py 2017-03-04 02:50:20 +0000 | |||
1305 | @@ -2,19 +2,17 @@ | |||
1306 | 2 | 2 | ||
1307 | 3 | # Copyright 2014-2015 Canonical Limited. | 3 | # Copyright 2014-2015 Canonical Limited. |
1308 | 4 | # | 4 | # |
1322 | 5 | # This file is part of charm-helpers. | 5 | # Licensed under the Apache License, Version 2.0 (the "License"); |
1323 | 6 | # | 6 | # you may not use this file except in compliance with the License. |
1324 | 7 | # charm-helpers is free software: you can redistribute it and/or modify | 7 | # You may obtain a copy of the License at |
1325 | 8 | # it under the terms of the GNU Lesser General Public License version 3 as | 8 | # |
1326 | 9 | # published by the Free Software Foundation. | 9 | # http://www.apache.org/licenses/LICENSE-2.0 |
1327 | 10 | # | 10 | # |
1328 | 11 | # charm-helpers is distributed in the hope that it will be useful, | 11 | # Unless required by applicable law or agreed to in writing, software |
1329 | 12 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 12 | # distributed under the License is distributed on an "AS IS" BASIS, |
1330 | 13 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 13 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
1331 | 14 | # GNU Lesser General Public License for more details. | 14 | # See the License for the specific language governing permissions and |
1332 | 15 | # | 15 | # limitations under the License. |
1320 | 16 | # You should have received a copy of the GNU Lesser General Public License | ||
1321 | 17 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
1333 | 18 | 16 | ||
1334 | 19 | import yaml | 17 | import yaml |
1335 | 20 | from charmhelpers.core import fstab | 18 | from charmhelpers.core import fstab |
1336 | 21 | 19 | ||
1337 | === modified file 'charmhelpers/core/kernel.py' | |||
1338 | --- charmhelpers/core/kernel.py 2015-12-11 15:23:38 +0000 | |||
1339 | +++ charmhelpers/core/kernel.py 2017-03-04 02:50:20 +0000 | |||
1340 | @@ -3,29 +3,40 @@ | |||
1341 | 3 | 3 | ||
1342 | 4 | # Copyright 2014-2015 Canonical Limited. | 4 | # Copyright 2014-2015 Canonical Limited. |
1343 | 5 | # | 5 | # |
1360 | 6 | # This file is part of charm-helpers. | 6 | # Licensed under the Apache License, Version 2.0 (the "License"); |
1361 | 7 | # | 7 | # you may not use this file except in compliance with the License. |
1362 | 8 | # charm-helpers is free software: you can redistribute it and/or modify | 8 | # You may obtain a copy of the License at |
1363 | 9 | # it under the terms of the GNU Lesser General Public License version 3 as | 9 | # |
1364 | 10 | # published by the Free Software Foundation. | 10 | # http://www.apache.org/licenses/LICENSE-2.0 |
1365 | 11 | # | 11 | # |
1366 | 12 | # charm-helpers is distributed in the hope that it will be useful, | 12 | # Unless required by applicable law or agreed to in writing, software |
1367 | 13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 13 | # distributed under the License is distributed on an "AS IS" BASIS, |
1368 | 14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 14 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
1369 | 15 | # GNU Lesser General Public License for more details. | 15 | # See the License for the specific language governing permissions and |
1370 | 16 | # | 16 | # limitations under the License. |
1371 | 17 | # You should have received a copy of the GNU Lesser General Public License | 17 | |
1372 | 18 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | 18 | import re |
1373 | 19 | 19 | import subprocess | |
1374 | 20 | __author__ = "Jorge Niedbalski <jorge.niedbalski@canonical.com>" | 20 | |
1375 | 21 | 21 | from charmhelpers.osplatform import get_platform | |
1376 | 22 | from charmhelpers.core.hookenv import ( | 22 | from charmhelpers.core.hookenv import ( |
1377 | 23 | log, | 23 | log, |
1378 | 24 | INFO | 24 | INFO |
1379 | 25 | ) | 25 | ) |
1380 | 26 | 26 | ||
1383 | 27 | from subprocess import check_call, check_output | 27 | __platform__ = get_platform() |
1384 | 28 | import re | 28 | if __platform__ == "ubuntu": |
1385 | 29 | from charmhelpers.core.kernel_factory.ubuntu import ( | ||
1386 | 30 | persistent_modprobe, | ||
1387 | 31 | update_initramfs, | ||
1388 | 32 | ) # flake8: noqa -- ignore F401 for this import | ||
1389 | 33 | elif __platform__ == "centos": | ||
1390 | 34 | from charmhelpers.core.kernel_factory.centos import ( | ||
1391 | 35 | persistent_modprobe, | ||
1392 | 36 | update_initramfs, | ||
1393 | 37 | ) # flake8: noqa -- ignore F401 for this import | ||
1394 | 38 | |||
1395 | 39 | __author__ = "Jorge Niedbalski <jorge.niedbalski@canonical.com>" | ||
1396 | 29 | 40 | ||
1397 | 30 | 41 | ||
1398 | 31 | def modprobe(module, persist=True): | 42 | def modprobe(module, persist=True): |
1399 | @@ -34,11 +45,9 @@ | |||
1400 | 34 | 45 | ||
1401 | 35 | log('Loading kernel module %s' % module, level=INFO) | 46 | log('Loading kernel module %s' % module, level=INFO) |
1402 | 36 | 47 | ||
1404 | 37 | check_call(cmd) | 48 | subprocess.check_call(cmd) |
1405 | 38 | if persist: | 49 | if persist: |
1409 | 39 | with open('/etc/modules', 'r+') as modules: | 50 | persistent_modprobe(module) |
1407 | 40 | if module not in modules.read(): | ||
1408 | 41 | modules.write(module) | ||
1410 | 42 | 51 | ||
1411 | 43 | 52 | ||
1412 | 44 | def rmmod(module, force=False): | 53 | def rmmod(module, force=False): |
1413 | @@ -48,21 +57,16 @@ | |||
1414 | 48 | cmd.append('-f') | 57 | cmd.append('-f') |
1415 | 49 | cmd.append(module) | 58 | cmd.append(module) |
1416 | 50 | log('Removing kernel module %s' % module, level=INFO) | 59 | log('Removing kernel module %s' % module, level=INFO) |
1418 | 51 | return check_call(cmd) | 60 | return subprocess.check_call(cmd) |
1419 | 52 | 61 | ||
1420 | 53 | 62 | ||
1421 | 54 | def lsmod(): | 63 | def lsmod(): |
1422 | 55 | """Shows what kernel modules are currently loaded""" | 64 | """Shows what kernel modules are currently loaded""" |
1425 | 56 | return check_output(['lsmod'], | 65 | return subprocess.check_output(['lsmod'], |
1426 | 57 | universal_newlines=True) | 66 | universal_newlines=True) |
1427 | 58 | 67 | ||
1428 | 59 | 68 | ||
1429 | 60 | def is_module_loaded(module): | 69 | def is_module_loaded(module): |
1430 | 61 | """Checks if a kernel module is already loaded""" | 70 | """Checks if a kernel module is already loaded""" |
1431 | 62 | matches = re.findall('^%s[ ]+' % module, lsmod(), re.M) | 71 | matches = re.findall('^%s[ ]+' % module, lsmod(), re.M) |
1432 | 63 | return len(matches) > 0 | 72 | return len(matches) > 0 |
1433 | 64 | |||
1434 | 65 | |||
1435 | 66 | def update_initramfs(version='all'): | ||
1436 | 67 | """Updates an initramfs image""" | ||
1437 | 68 | return check_call(["update-initramfs", "-k", version, "-u"]) | ||
1438 | 69 | 73 | ||
1439 | === added directory 'charmhelpers/core/kernel_factory' | |||
1440 | === added file 'charmhelpers/core/kernel_factory/__init__.py' | |||
1441 | === added file 'charmhelpers/core/kernel_factory/centos.py' | |||
1442 | --- charmhelpers/core/kernel_factory/centos.py 1970-01-01 00:00:00 +0000 | |||
1443 | +++ charmhelpers/core/kernel_factory/centos.py 2017-03-04 02:50:20 +0000 | |||
1444 | @@ -0,0 +1,17 @@ | |||
1445 | 1 | import subprocess | ||
1446 | 2 | import os | ||
1447 | 3 | |||
1448 | 4 | |||
1449 | 5 | def persistent_modprobe(module): | ||
1450 | 6 | """Load a kernel module and configure for auto-load on reboot.""" | ||
1451 | 7 | if not os.path.exists('/etc/rc.modules'): | ||
1452 | 8 | open('/etc/rc.modules', 'a') | ||
1453 | 9 | os.chmod('/etc/rc.modules', 111) | ||
1454 | 10 | with open('/etc/rc.modules', 'r+') as modules: | ||
1455 | 11 | if module not in modules.read(): | ||
1456 | 12 | modules.write('modprobe %s\n' % module) | ||
1457 | 13 | |||
1458 | 14 | |||
1459 | 15 | def update_initramfs(version='all'): | ||
1460 | 16 | """Updates an initramfs image.""" | ||
1461 | 17 | return subprocess.check_call(["dracut", "-f", version]) | ||
1462 | 0 | 18 | ||
1463 | === added file 'charmhelpers/core/kernel_factory/ubuntu.py' | |||
1464 | --- charmhelpers/core/kernel_factory/ubuntu.py 1970-01-01 00:00:00 +0000 | |||
1465 | +++ charmhelpers/core/kernel_factory/ubuntu.py 2017-03-04 02:50:20 +0000 | |||
1466 | @@ -0,0 +1,13 @@ | |||
1467 | 1 | import subprocess | ||
1468 | 2 | |||
1469 | 3 | |||
1470 | 4 | def persistent_modprobe(module): | ||
1471 | 5 | """Load a kernel module and configure for auto-load on reboot.""" | ||
1472 | 6 | with open('/etc/modules', 'r+') as modules: | ||
1473 | 7 | if module not in modules.read(): | ||
1474 | 8 | modules.write(module + "\n") | ||
1475 | 9 | |||
1476 | 10 | |||
1477 | 11 | def update_initramfs(version='all'): | ||
1478 | 12 | """Updates an initramfs image.""" | ||
1479 | 13 | return subprocess.check_call(["update-initramfs", "-k", version, "-u"]) | ||
1480 | 0 | 14 | ||
1481 | === modified file 'charmhelpers/core/services/__init__.py' | |||
1482 | --- charmhelpers/core/services/__init__.py 2015-01-28 08:59:02 +0000 | |||
1483 | +++ charmhelpers/core/services/__init__.py 2017-03-04 02:50:20 +0000 | |||
1484 | @@ -1,18 +1,16 @@ | |||
1485 | 1 | # Copyright 2014-2015 Canonical Limited. | 1 | # Copyright 2014-2015 Canonical Limited. |
1486 | 2 | # | 2 | # |
1500 | 3 | # This file is part of charm-helpers. | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); |
1501 | 4 | # | 4 | # you may not use this file except in compliance with the License. |
1502 | 5 | # charm-helpers is free software: you can redistribute it and/or modify | 5 | # You may obtain a copy of the License at |
1503 | 6 | # it under the terms of the GNU Lesser General Public License version 3 as | 6 | # |
1504 | 7 | # published by the Free Software Foundation. | 7 | # http://www.apache.org/licenses/LICENSE-2.0 |
1505 | 8 | # | 8 | # |
1506 | 9 | # charm-helpers is distributed in the hope that it will be useful, | 9 | # Unless required by applicable law or agreed to in writing, software |
1507 | 10 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 10 | # distributed under the License is distributed on an "AS IS" BASIS, |
1508 | 11 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
1509 | 12 | # GNU Lesser General Public License for more details. | 12 | # See the License for the specific language governing permissions and |
1510 | 13 | # | 13 | # limitations under the License. |
1498 | 14 | # You should have received a copy of the GNU Lesser General Public License | ||
1499 | 15 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
1511 | 16 | 14 | ||
1512 | 17 | from .base import * # NOQA | 15 | from .base import * # NOQA |
1513 | 18 | from .helpers import * # NOQA | 16 | from .helpers import * # NOQA |
1514 | 19 | 17 | ||
1515 | === modified file 'charmhelpers/core/services/base.py' | |||
1516 | --- charmhelpers/core/services/base.py 2015-07-03 09:13:26 +0000 | |||
1517 | +++ charmhelpers/core/services/base.py 2017-03-04 02:50:20 +0000 | |||
1518 | @@ -1,18 +1,16 @@ | |||
1519 | 1 | # Copyright 2014-2015 Canonical Limited. | 1 | # Copyright 2014-2015 Canonical Limited. |
1520 | 2 | # | 2 | # |
1534 | 3 | # This file is part of charm-helpers. | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); |
1535 | 4 | # | 4 | # you may not use this file except in compliance with the License. |
1536 | 5 | # charm-helpers is free software: you can redistribute it and/or modify | 5 | # You may obtain a copy of the License at |
1537 | 6 | # it under the terms of the GNU Lesser General Public License version 3 as | 6 | # |
1538 | 7 | # published by the Free Software Foundation. | 7 | # http://www.apache.org/licenses/LICENSE-2.0 |
1539 | 8 | # | 8 | # |
1540 | 9 | # charm-helpers is distributed in the hope that it will be useful, | 9 | # Unless required by applicable law or agreed to in writing, software |
1541 | 10 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 10 | # distributed under the License is distributed on an "AS IS" BASIS, |
1542 | 11 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
1543 | 12 | # GNU Lesser General Public License for more details. | 12 | # See the License for the specific language governing permissions and |
1544 | 13 | # | 13 | # limitations under the License. |
1532 | 14 | # You should have received a copy of the GNU Lesser General Public License | ||
1533 | 15 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
1545 | 16 | 14 | ||
1546 | 17 | import os | 15 | import os |
1547 | 18 | import json | 16 | import json |
1548 | 19 | 17 | ||
1549 | === modified file 'charmhelpers/core/services/helpers.py' | |||
1550 | --- charmhelpers/core/services/helpers.py 2015-12-11 15:23:38 +0000 | |||
1551 | +++ charmhelpers/core/services/helpers.py 2017-03-04 02:50:20 +0000 | |||
1552 | @@ -1,18 +1,16 @@ | |||
1553 | 1 | # Copyright 2014-2015 Canonical Limited. | 1 | # Copyright 2014-2015 Canonical Limited. |
1554 | 2 | # | 2 | # |
1568 | 3 | # This file is part of charm-helpers. | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); |
1569 | 4 | # | 4 | # you may not use this file except in compliance with the License. |
1570 | 5 | # charm-helpers is free software: you can redistribute it and/or modify | 5 | # You may obtain a copy of the License at |
1571 | 6 | # it under the terms of the GNU Lesser General Public License version 3 as | 6 | # |
1572 | 7 | # published by the Free Software Foundation. | 7 | # http://www.apache.org/licenses/LICENSE-2.0 |
1573 | 8 | # | 8 | # |
1574 | 9 | # charm-helpers is distributed in the hope that it will be useful, | 9 | # Unless required by applicable law or agreed to in writing, software |
1575 | 10 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 10 | # distributed under the License is distributed on an "AS IS" BASIS, |
1576 | 11 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
1577 | 12 | # GNU Lesser General Public License for more details. | 12 | # See the License for the specific language governing permissions and |
1578 | 13 | # | 13 | # limitations under the License. |
1566 | 14 | # You should have received a copy of the GNU Lesser General Public License | ||
1567 | 15 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
1579 | 16 | 14 | ||
1580 | 17 | import os | 15 | import os |
1581 | 18 | import yaml | 16 | import yaml |
1582 | 19 | 17 | ||
1583 | === modified file 'charmhelpers/core/strutils.py' | |||
1584 | --- charmhelpers/core/strutils.py 2015-12-11 15:23:38 +0000 | |||
1585 | +++ charmhelpers/core/strutils.py 2017-03-04 02:50:20 +0000 | |||
1586 | @@ -3,19 +3,17 @@ | |||
1587 | 3 | 3 | ||
1588 | 4 | # Copyright 2014-2015 Canonical Limited. | 4 | # Copyright 2014-2015 Canonical Limited. |
1589 | 5 | # | 5 | # |
1603 | 6 | # This file is part of charm-helpers. | 6 | # Licensed under the Apache License, Version 2.0 (the "License"); |
1604 | 7 | # | 7 | # you may not use this file except in compliance with the License. |
1605 | 8 | # charm-helpers is free software: you can redistribute it and/or modify | 8 | # You may obtain a copy of the License at |
1606 | 9 | # it under the terms of the GNU Lesser General Public License version 3 as | 9 | # |
1607 | 10 | # published by the Free Software Foundation. | 10 | # http://www.apache.org/licenses/LICENSE-2.0 |
1608 | 11 | # | 11 | # |
1609 | 12 | # charm-helpers is distributed in the hope that it will be useful, | 12 | # Unless required by applicable law or agreed to in writing, software |
1610 | 13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 13 | # distributed under the License is distributed on an "AS IS" BASIS, |
1611 | 14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 14 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
1612 | 15 | # GNU Lesser General Public License for more details. | 15 | # See the License for the specific language governing permissions and |
1613 | 16 | # | 16 | # limitations under the License. |
1601 | 17 | # You should have received a copy of the GNU Lesser General Public License | ||
1602 | 18 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
1614 | 19 | 17 | ||
1615 | 20 | import six | 18 | import six |
1616 | 21 | import re | 19 | import re |
1617 | 22 | 20 | ||
1618 | === modified file 'charmhelpers/core/sysctl.py' | |||
1619 | --- charmhelpers/core/sysctl.py 2015-03-12 11:42:26 +0000 | |||
1620 | +++ charmhelpers/core/sysctl.py 2017-03-04 02:50:20 +0000 | |||
1621 | @@ -3,19 +3,17 @@ | |||
1622 | 3 | 3 | ||
1623 | 4 | # Copyright 2014-2015 Canonical Limited. | 4 | # Copyright 2014-2015 Canonical Limited. |
1624 | 5 | # | 5 | # |
1638 | 6 | # This file is part of charm-helpers. | 6 | # Licensed under the Apache License, Version 2.0 (the "License"); |
1639 | 7 | # | 7 | # you may not use this file except in compliance with the License. |
1640 | 8 | # charm-helpers is free software: you can redistribute it and/or modify | 8 | # You may obtain a copy of the License at |
1641 | 9 | # it under the terms of the GNU Lesser General Public License version 3 as | 9 | # |
1642 | 10 | # published by the Free Software Foundation. | 10 | # http://www.apache.org/licenses/LICENSE-2.0 |
1643 | 11 | # | 11 | # |
1644 | 12 | # charm-helpers is distributed in the hope that it will be useful, | 12 | # Unless required by applicable law or agreed to in writing, software |
1645 | 13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 13 | # distributed under the License is distributed on an "AS IS" BASIS, |
1646 | 14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 14 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
1647 | 15 | # GNU Lesser General Public License for more details. | 15 | # See the License for the specific language governing permissions and |
1648 | 16 | # | 16 | # limitations under the License. |
1636 | 17 | # You should have received a copy of the GNU Lesser General Public License | ||
1637 | 18 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
1649 | 19 | 17 | ||
1650 | 20 | import yaml | 18 | import yaml |
1651 | 21 | 19 | ||
1652 | 22 | 20 | ||
1653 | === modified file 'charmhelpers/core/templating.py' | |||
1654 | --- charmhelpers/core/templating.py 2015-12-11 15:23:38 +0000 | |||
1655 | +++ charmhelpers/core/templating.py 2017-03-04 02:50:20 +0000 | |||
1656 | @@ -1,20 +1,19 @@ | |||
1657 | 1 | # Copyright 2014-2015 Canonical Limited. | 1 | # Copyright 2014-2015 Canonical Limited. |
1658 | 2 | # | 2 | # |
1672 | 3 | # This file is part of charm-helpers. | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); |
1673 | 4 | # | 4 | # you may not use this file except in compliance with the License. |
1674 | 5 | # charm-helpers is free software: you can redistribute it and/or modify | 5 | # You may obtain a copy of the License at |
1675 | 6 | # it under the terms of the GNU Lesser General Public License version 3 as | 6 | # |
1676 | 7 | # published by the Free Software Foundation. | 7 | # http://www.apache.org/licenses/LICENSE-2.0 |
1677 | 8 | # | 8 | # |
1678 | 9 | # charm-helpers is distributed in the hope that it will be useful, | 9 | # Unless required by applicable law or agreed to in writing, software |
1679 | 10 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 10 | # distributed under the License is distributed on an "AS IS" BASIS, |
1680 | 11 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
1681 | 12 | # GNU Lesser General Public License for more details. | 12 | # See the License for the specific language governing permissions and |
1682 | 13 | # | 13 | # limitations under the License. |
1670 | 14 | # You should have received a copy of the GNU Lesser General Public License | ||
1671 | 15 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
1683 | 16 | 14 | ||
1684 | 17 | import os | 15 | import os |
1685 | 16 | import sys | ||
1686 | 18 | 17 | ||
1687 | 19 | from charmhelpers.core import host | 18 | from charmhelpers.core import host |
1688 | 20 | from charmhelpers.core import hookenv | 19 | from charmhelpers.core import hookenv |
1689 | @@ -40,8 +39,9 @@ | |||
1690 | 40 | The rendered template will be written to the file as well as being returned | 39 | The rendered template will be written to the file as well as being returned |
1691 | 41 | as a string. | 40 | as a string. |
1692 | 42 | 41 | ||
1695 | 43 | Note: Using this requires python-jinja2; if it is not installed, calling | 42 | Note: Using this requires python-jinja2 or python3-jinja2; if it is not |
1696 | 44 | this will attempt to use charmhelpers.fetch.apt_install to install it. | 43 | installed, calling this will attempt to use charmhelpers.fetch.apt_install |
1697 | 44 | to install it. | ||
1698 | 45 | """ | 45 | """ |
1699 | 46 | try: | 46 | try: |
1700 | 47 | from jinja2 import FileSystemLoader, Environment, exceptions | 47 | from jinja2 import FileSystemLoader, Environment, exceptions |
1701 | @@ -53,7 +53,10 @@ | |||
1702 | 53 | 'charmhelpers.fetch to install it', | 53 | 'charmhelpers.fetch to install it', |
1703 | 54 | level=hookenv.ERROR) | 54 | level=hookenv.ERROR) |
1704 | 55 | raise | 55 | raise |
1706 | 56 | apt_install('python-jinja2', fatal=True) | 56 | if sys.version_info.major == 2: |
1707 | 57 | apt_install('python-jinja2', fatal=True) | ||
1708 | 58 | else: | ||
1709 | 59 | apt_install('python3-jinja2', fatal=True) | ||
1710 | 57 | from jinja2 import FileSystemLoader, Environment, exceptions | 60 | from jinja2 import FileSystemLoader, Environment, exceptions |
1711 | 58 | 61 | ||
1712 | 59 | if template_loader: | 62 | if template_loader: |
1713 | 60 | 63 | ||
1714 | === modified file 'charmhelpers/core/unitdata.py' | |||
1715 | --- charmhelpers/core/unitdata.py 2015-12-11 15:23:38 +0000 | |||
1716 | +++ charmhelpers/core/unitdata.py 2017-03-04 02:50:20 +0000 | |||
1717 | @@ -3,20 +3,17 @@ | |||
1718 | 3 | # | 3 | # |
1719 | 4 | # Copyright 2014-2015 Canonical Limited. | 4 | # Copyright 2014-2015 Canonical Limited. |
1720 | 5 | # | 5 | # |
1735 | 6 | # This file is part of charm-helpers. | 6 | # Licensed under the Apache License, Version 2.0 (the "License"); |
1736 | 7 | # | 7 | # you may not use this file except in compliance with the License. |
1737 | 8 | # charm-helpers is free software: you can redistribute it and/or modify | 8 | # You may obtain a copy of the License at |
1738 | 9 | # it under the terms of the GNU Lesser General Public License version 3 as | 9 | # |
1739 | 10 | # published by the Free Software Foundation. | 10 | # http://www.apache.org/licenses/LICENSE-2.0 |
1740 | 11 | # | 11 | # |
1741 | 12 | # charm-helpers is distributed in the hope that it will be useful, | 12 | # Unless required by applicable law or agreed to in writing, software |
1742 | 13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 13 | # distributed under the License is distributed on an "AS IS" BASIS, |
1743 | 14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 14 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
1744 | 15 | # GNU Lesser General Public License for more details. | 15 | # See the License for the specific language governing permissions and |
1745 | 16 | # | 16 | # limitations under the License. |
1732 | 17 | # You should have received a copy of the GNU Lesser General Public License | ||
1733 | 18 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
1734 | 19 | # | ||
1746 | 20 | # | 17 | # |
1747 | 21 | # Authors: | 18 | # Authors: |
1748 | 22 | # Kapil Thangavelu <kapil.foss@gmail.com> | 19 | # Kapil Thangavelu <kapil.foss@gmail.com> |
1749 | 23 | 20 | ||
1750 | === modified file 'charmhelpers/fetch/__init__.py' | |||
1751 | --- charmhelpers/fetch/__init__.py 2016-05-06 07:12:40 +0000 | |||
1752 | +++ charmhelpers/fetch/__init__.py 2017-03-04 02:50:20 +0000 | |||
1753 | @@ -1,32 +1,24 @@ | |||
1754 | 1 | # Copyright 2014-2015 Canonical Limited. | 1 | # Copyright 2014-2015 Canonical Limited. |
1755 | 2 | # | 2 | # |
1769 | 3 | # This file is part of charm-helpers. | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); |
1770 | 4 | # | 4 | # you may not use this file except in compliance with the License. |
1771 | 5 | # charm-helpers is free software: you can redistribute it and/or modify | 5 | # You may obtain a copy of the License at |
1772 | 6 | # it under the terms of the GNU Lesser General Public License version 3 as | 6 | # |
1773 | 7 | # published by the Free Software Foundation. | 7 | # http://www.apache.org/licenses/LICENSE-2.0 |
1774 | 8 | # | 8 | # |
1775 | 9 | # charm-helpers is distributed in the hope that it will be useful, | 9 | # Unless required by applicable law or agreed to in writing, software |
1776 | 10 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 10 | # distributed under the License is distributed on an "AS IS" BASIS, |
1777 | 11 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
1778 | 12 | # GNU Lesser General Public License for more details. | 12 | # See the License for the specific language governing permissions and |
1779 | 13 | # | 13 | # limitations under the License. |
1767 | 14 | # You should have received a copy of the GNU Lesser General Public License | ||
1768 | 15 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
1780 | 16 | 14 | ||
1781 | 17 | import importlib | 15 | import importlib |
1784 | 18 | from tempfile import NamedTemporaryFile | 16 | from charmhelpers.osplatform import get_platform |
1783 | 19 | import time | ||
1785 | 20 | from yaml import safe_load | 17 | from yaml import safe_load |
1786 | 21 | from charmhelpers.core.host import ( | ||
1787 | 22 | lsb_release | ||
1788 | 23 | ) | ||
1789 | 24 | import subprocess | ||
1790 | 25 | from charmhelpers.core.hookenv import ( | 18 | from charmhelpers.core.hookenv import ( |
1791 | 26 | config, | 19 | config, |
1792 | 27 | log, | 20 | log, |
1793 | 28 | ) | 21 | ) |
1794 | 29 | import os | ||
1795 | 30 | 22 | ||
1796 | 31 | import six | 23 | import six |
1797 | 32 | if six.PY3: | 24 | if six.PY3: |
1798 | @@ -35,79 +27,6 @@ | |||
1799 | 35 | from urlparse import urlparse, urlunparse | 27 | from urlparse import urlparse, urlunparse |
1800 | 36 | 28 | ||
1801 | 37 | 29 | ||
1802 | 38 | CLOUD_ARCHIVE = """# Ubuntu Cloud Archive | ||
1803 | 39 | deb http://ubuntu-cloud.archive.canonical.com/ubuntu {} main | ||
1804 | 40 | """ | ||
1805 | 41 | PROPOSED_POCKET = """# Proposed | ||
1806 | 42 | deb http://archive.ubuntu.com/ubuntu {}-proposed main universe multiverse restricted | ||
1807 | 43 | """ | ||
1808 | 44 | CLOUD_ARCHIVE_POCKETS = { | ||
1809 | 45 | # Folsom | ||
1810 | 46 | 'folsom': 'precise-updates/folsom', | ||
1811 | 47 | 'precise-folsom': 'precise-updates/folsom', | ||
1812 | 48 | 'precise-folsom/updates': 'precise-updates/folsom', | ||
1813 | 49 | 'precise-updates/folsom': 'precise-updates/folsom', | ||
1814 | 50 | 'folsom/proposed': 'precise-proposed/folsom', | ||
1815 | 51 | 'precise-folsom/proposed': 'precise-proposed/folsom', | ||
1816 | 52 | 'precise-proposed/folsom': 'precise-proposed/folsom', | ||
1817 | 53 | # Grizzly | ||
1818 | 54 | 'grizzly': 'precise-updates/grizzly', | ||
1819 | 55 | 'precise-grizzly': 'precise-updates/grizzly', | ||
1820 | 56 | 'precise-grizzly/updates': 'precise-updates/grizzly', | ||
1821 | 57 | 'precise-updates/grizzly': 'precise-updates/grizzly', | ||
1822 | 58 | 'grizzly/proposed': 'precise-proposed/grizzly', | ||
1823 | 59 | 'precise-grizzly/proposed': 'precise-proposed/grizzly', | ||
1824 | 60 | 'precise-proposed/grizzly': 'precise-proposed/grizzly', | ||
1825 | 61 | # Havana | ||
1826 | 62 | 'havana': 'precise-updates/havana', | ||
1827 | 63 | 'precise-havana': 'precise-updates/havana', | ||
1828 | 64 | 'precise-havana/updates': 'precise-updates/havana', | ||
1829 | 65 | 'precise-updates/havana': 'precise-updates/havana', | ||
1830 | 66 | 'havana/proposed': 'precise-proposed/havana', | ||
1831 | 67 | 'precise-havana/proposed': 'precise-proposed/havana', | ||
1832 | 68 | 'precise-proposed/havana': 'precise-proposed/havana', | ||
1833 | 69 | # Icehouse | ||
1834 | 70 | 'icehouse': 'precise-updates/icehouse', | ||
1835 | 71 | 'precise-icehouse': 'precise-updates/icehouse', | ||
1836 | 72 | 'precise-icehouse/updates': 'precise-updates/icehouse', | ||
1837 | 73 | 'precise-updates/icehouse': 'precise-updates/icehouse', | ||
1838 | 74 | 'icehouse/proposed': 'precise-proposed/icehouse', | ||
1839 | 75 | 'precise-icehouse/proposed': 'precise-proposed/icehouse', | ||
1840 | 76 | 'precise-proposed/icehouse': 'precise-proposed/icehouse', | ||
1841 | 77 | # Juno | ||
1842 | 78 | 'juno': 'trusty-updates/juno', | ||
1843 | 79 | 'trusty-juno': 'trusty-updates/juno', | ||
1844 | 80 | 'trusty-juno/updates': 'trusty-updates/juno', | ||
1845 | 81 | 'trusty-updates/juno': 'trusty-updates/juno', | ||
1846 | 82 | 'juno/proposed': 'trusty-proposed/juno', | ||
1847 | 83 | 'trusty-juno/proposed': 'trusty-proposed/juno', | ||
1848 | 84 | 'trusty-proposed/juno': 'trusty-proposed/juno', | ||
1849 | 85 | # Kilo | ||
1850 | 86 | 'kilo': 'trusty-updates/kilo', | ||
1851 | 87 | 'trusty-kilo': 'trusty-updates/kilo', | ||
1852 | 88 | 'trusty-kilo/updates': 'trusty-updates/kilo', | ||
1853 | 89 | 'trusty-updates/kilo': 'trusty-updates/kilo', | ||
1854 | 90 | 'kilo/proposed': 'trusty-proposed/kilo', | ||
1855 | 91 | 'trusty-kilo/proposed': 'trusty-proposed/kilo', | ||
1856 | 92 | 'trusty-proposed/kilo': 'trusty-proposed/kilo', | ||
1857 | 93 | # Liberty | ||
1858 | 94 | 'liberty': 'trusty-updates/liberty', | ||
1859 | 95 | 'trusty-liberty': 'trusty-updates/liberty', | ||
1860 | 96 | 'trusty-liberty/updates': 'trusty-updates/liberty', | ||
1861 | 97 | 'trusty-updates/liberty': 'trusty-updates/liberty', | ||
1862 | 98 | 'liberty/proposed': 'trusty-proposed/liberty', | ||
1863 | 99 | 'trusty-liberty/proposed': 'trusty-proposed/liberty', | ||
1864 | 100 | 'trusty-proposed/liberty': 'trusty-proposed/liberty', | ||
1865 | 101 | # Mitaka | ||
1866 | 102 | 'mitaka': 'trusty-updates/mitaka', | ||
1867 | 103 | 'trusty-mitaka': 'trusty-updates/mitaka', | ||
1868 | 104 | 'trusty-mitaka/updates': 'trusty-updates/mitaka', | ||
1869 | 105 | 'trusty-updates/mitaka': 'trusty-updates/mitaka', | ||
1870 | 106 | 'mitaka/proposed': 'trusty-proposed/mitaka', | ||
1871 | 107 | 'trusty-mitaka/proposed': 'trusty-proposed/mitaka', | ||
1872 | 108 | 'trusty-proposed/mitaka': 'trusty-proposed/mitaka', | ||
1873 | 109 | } | ||
1874 | 110 | |||
1875 | 111 | # The order of this list is very important. Handlers should be listed in from | 30 | # The order of this list is very important. Handlers should be listed in from |
1876 | 112 | # least- to most-specific URL matching. | 31 | # least- to most-specific URL matching. |
1877 | 113 | FETCH_HANDLERS = ( | 32 | FETCH_HANDLERS = ( |
1878 | @@ -116,10 +35,6 @@ | |||
1879 | 116 | 'charmhelpers.fetch.giturl.GitUrlFetchHandler', | 35 | 'charmhelpers.fetch.giturl.GitUrlFetchHandler', |
1880 | 117 | ) | 36 | ) |
1881 | 118 | 37 | ||
1882 | 119 | APT_NO_LOCK = 100 # The return code for "couldn't acquire lock" in APT. | ||
1883 | 120 | APT_NO_LOCK_RETRY_DELAY = 10 # Wait 10 seconds between apt lock checks. | ||
1884 | 121 | APT_NO_LOCK_RETRY_COUNT = 30 # Retry to acquire the lock X times. | ||
1885 | 122 | |||
1886 | 123 | 38 | ||
1887 | 124 | class SourceConfigError(Exception): | 39 | class SourceConfigError(Exception): |
1888 | 125 | pass | 40 | pass |
1889 | @@ -157,180 +72,38 @@ | |||
1890 | 157 | return urlunparse(parts) | 72 | return urlunparse(parts) |
1891 | 158 | 73 | ||
1892 | 159 | 74 | ||
2057 | 160 | def filter_installed_packages(packages): | 75 | __platform__ = get_platform() |
2058 | 161 | """Returns a list of packages that require installation""" | 76 | module = "charmhelpers.fetch.%s" % __platform__ |
2059 | 162 | cache = apt_cache() | 77 | fetch = importlib.import_module(module) |
2060 | 163 | _pkgs = [] | 78 | |
2061 | 164 | for package in packages: | 79 | filter_installed_packages = fetch.filter_installed_packages |
2062 | 165 | try: | 80 | install = fetch.install |
2063 | 166 | p = cache[package] | 81 | upgrade = fetch.upgrade |
2064 | 167 | p.current_ver or _pkgs.append(package) | 82 | update = fetch.update |
2065 | 168 | except KeyError: | 83 | purge = fetch.purge |
2066 | 169 | log('Package {} has no installation candidate.'.format(package), | 84 | add_source = fetch.add_source |
2067 | 170 | level='WARNING') | 85 | |
2068 | 171 | _pkgs.append(package) | 86 | if __platform__ == "ubuntu": |
2069 | 172 | return _pkgs | 87 | apt_cache = fetch.apt_cache |
2070 | 173 | 88 | apt_install = fetch.install | |
2071 | 174 | 89 | apt_update = fetch.update | |
2072 | 175 | def apt_cache(in_memory=True): | 90 | apt_upgrade = fetch.upgrade |
2073 | 176 | """Build and return an apt cache""" | 91 | apt_purge = fetch.purge |
2074 | 177 | from apt import apt_pkg | 92 | apt_mark = fetch.apt_mark |
2075 | 178 | apt_pkg.init() | 93 | apt_hold = fetch.apt_hold |
2076 | 179 | if in_memory: | 94 | apt_unhold = fetch.apt_unhold |
2077 | 180 | apt_pkg.config.set("Dir::Cache::pkgcache", "") | 95 | get_upstream_version = fetch.get_upstream_version |
2078 | 181 | apt_pkg.config.set("Dir::Cache::srcpkgcache", "") | 96 | elif __platform__ == "centos": |
2079 | 182 | return apt_pkg.Cache() | 97 | yum_search = fetch.yum_search |
1916 | 183 | |||
1917 | 184 | |||
1918 | 185 | def apt_install(packages, options=None, fatal=False): | ||
1919 | 186 | """Install one or more packages""" | ||
1920 | 187 | if options is None: | ||
1921 | 188 | options = ['--option=Dpkg::Options::=--force-confold'] | ||
1922 | 189 | |||
1923 | 190 | cmd = ['apt-get', '--assume-yes'] | ||
1924 | 191 | cmd.extend(options) | ||
1925 | 192 | cmd.append('install') | ||
1926 | 193 | if isinstance(packages, six.string_types): | ||
1927 | 194 | cmd.append(packages) | ||
1928 | 195 | else: | ||
1929 | 196 | cmd.extend(packages) | ||
1930 | 197 | log("Installing {} with options: {}".format(packages, | ||
1931 | 198 | options)) | ||
1932 | 199 | _run_apt_command(cmd, fatal) | ||
1933 | 200 | |||
1934 | 201 | |||
1935 | 202 | def apt_upgrade(options=None, fatal=False, dist=False): | ||
1936 | 203 | """Upgrade all packages""" | ||
1937 | 204 | if options is None: | ||
1938 | 205 | options = ['--option=Dpkg::Options::=--force-confold'] | ||
1939 | 206 | |||
1940 | 207 | cmd = ['apt-get', '--assume-yes'] | ||
1941 | 208 | cmd.extend(options) | ||
1942 | 209 | if dist: | ||
1943 | 210 | cmd.append('dist-upgrade') | ||
1944 | 211 | else: | ||
1945 | 212 | cmd.append('upgrade') | ||
1946 | 213 | log("Upgrading with options: {}".format(options)) | ||
1947 | 214 | _run_apt_command(cmd, fatal) | ||
1948 | 215 | |||
1949 | 216 | |||
1950 | 217 | def apt_update(fatal=False): | ||
1951 | 218 | """Update local apt cache""" | ||
1952 | 219 | cmd = ['apt-get', 'update'] | ||
1953 | 220 | _run_apt_command(cmd, fatal) | ||
1954 | 221 | |||
1955 | 222 | |||
1956 | 223 | def apt_purge(packages, fatal=False): | ||
1957 | 224 | """Purge one or more packages""" | ||
1958 | 225 | cmd = ['apt-get', '--assume-yes', 'purge'] | ||
1959 | 226 | if isinstance(packages, six.string_types): | ||
1960 | 227 | cmd.append(packages) | ||
1961 | 228 | else: | ||
1962 | 229 | cmd.extend(packages) | ||
1963 | 230 | log("Purging {}".format(packages)) | ||
1964 | 231 | _run_apt_command(cmd, fatal) | ||
1965 | 232 | |||
1966 | 233 | |||
1967 | 234 | def apt_mark(packages, mark, fatal=False): | ||
1968 | 235 | """Flag one or more packages using apt-mark""" | ||
1969 | 236 | log("Marking {} as {}".format(packages, mark)) | ||
1970 | 237 | cmd = ['apt-mark', mark] | ||
1971 | 238 | if isinstance(packages, six.string_types): | ||
1972 | 239 | cmd.append(packages) | ||
1973 | 240 | else: | ||
1974 | 241 | cmd.extend(packages) | ||
1975 | 242 | |||
1976 | 243 | if fatal: | ||
1977 | 244 | subprocess.check_call(cmd, universal_newlines=True) | ||
1978 | 245 | else: | ||
1979 | 246 | subprocess.call(cmd, universal_newlines=True) | ||
1980 | 247 | |||
1981 | 248 | |||
1982 | 249 | def apt_hold(packages, fatal=False): | ||
1983 | 250 | return apt_mark(packages, 'hold', fatal=fatal) | ||
1984 | 251 | |||
1985 | 252 | |||
1986 | 253 | def apt_unhold(packages, fatal=False): | ||
1987 | 254 | return apt_mark(packages, 'unhold', fatal=fatal) | ||
1988 | 255 | |||
1989 | 256 | |||
1990 | 257 | def add_source(source, key=None): | ||
1991 | 258 | """Add a package source to this system. | ||
1992 | 259 | |||
1993 | 260 | @param source: a URL or sources.list entry, as supported by | ||
1994 | 261 | add-apt-repository(1). Examples:: | ||
1995 | 262 | |||
1996 | 263 | ppa:charmers/example | ||
1997 | 264 | deb https://stub:key@private.example.com/ubuntu trusty main | ||
1998 | 265 | |||
1999 | 266 | In addition: | ||
2000 | 267 | 'proposed:' may be used to enable the standard 'proposed' | ||
2001 | 268 | pocket for the release. | ||
2002 | 269 | 'cloud:' may be used to activate official cloud archive pockets, | ||
2003 | 270 | such as 'cloud:icehouse' | ||
2004 | 271 | 'distro' may be used as a noop | ||
2005 | 272 | |||
2006 | 273 | @param key: A key to be added to the system's APT keyring and used | ||
2007 | 274 | to verify the signatures on packages. Ideally, this should be an | ||
2008 | 275 | ASCII format GPG public key including the block headers. A GPG key | ||
2009 | 276 | id may also be used, but be aware that only insecure protocols are | ||
2010 | 277 | available to retrieve the actual public key from a public keyserver | ||
2011 | 278 | placing your Juju environment at risk. ppa and cloud archive keys | ||
2012 | 279 | are securely added automtically, so sould not be provided. | ||
2013 | 280 | """ | ||
2014 | 281 | if source is None: | ||
2015 | 282 | log('Source is not present. Skipping') | ||
2016 | 283 | return | ||
2017 | 284 | |||
2018 | 285 | if (source.startswith('ppa:') or | ||
2019 | 286 | source.startswith('http') or | ||
2020 | 287 | source.startswith('deb ') or | ||
2021 | 288 | source.startswith('cloud-archive:')): | ||
2022 | 289 | subprocess.check_call(['add-apt-repository', '--yes', source]) | ||
2023 | 290 | elif source.startswith('cloud:'): | ||
2024 | 291 | apt_install(filter_installed_packages(['ubuntu-cloud-keyring']), | ||
2025 | 292 | fatal=True) | ||
2026 | 293 | pocket = source.split(':')[-1] | ||
2027 | 294 | if pocket not in CLOUD_ARCHIVE_POCKETS: | ||
2028 | 295 | raise SourceConfigError( | ||
2029 | 296 | 'Unsupported cloud: source option %s' % | ||
2030 | 297 | pocket) | ||
2031 | 298 | actual_pocket = CLOUD_ARCHIVE_POCKETS[pocket] | ||
2032 | 299 | with open('/etc/apt/sources.list.d/cloud-archive.list', 'w') as apt: | ||
2033 | 300 | apt.write(CLOUD_ARCHIVE.format(actual_pocket)) | ||
2034 | 301 | elif source == 'proposed': | ||
2035 | 302 | release = lsb_release()['DISTRIB_CODENAME'] | ||
2036 | 303 | with open('/etc/apt/sources.list.d/proposed.list', 'w') as apt: | ||
2037 | 304 | apt.write(PROPOSED_POCKET.format(release)) | ||
2038 | 305 | elif source == 'distro': | ||
2039 | 306 | pass | ||
2040 | 307 | else: | ||
2041 | 308 | log("Unknown source: {!r}".format(source)) | ||
2042 | 309 | |||
2043 | 310 | if key: | ||
2044 | 311 | if '-----BEGIN PGP PUBLIC KEY BLOCK-----' in key: | ||
2045 | 312 | with NamedTemporaryFile('w+') as key_file: | ||
2046 | 313 | key_file.write(key) | ||
2047 | 314 | key_file.flush() | ||
2048 | 315 | key_file.seek(0) | ||
2049 | 316 | subprocess.check_call(['apt-key', 'add', '-'], stdin=key_file) | ||
2050 | 317 | else: | ||
2051 | 318 | # Note that hkp: is in no way a secure protocol. Using a | ||
2052 | 319 | # GPG key id is pointless from a security POV unless you | ||
2053 | 320 | # absolutely trust your network and DNS. | ||
2054 | 321 | subprocess.check_call(['apt-key', 'adv', '--keyserver', | ||
2055 | 322 | 'hkp://keyserver.ubuntu.com:80', '--recv', | ||
2056 | 323 | key]) | ||
2080 | 324 | 98 | ||
2081 | 325 | 99 | ||
2082 | 326 | def configure_sources(update=False, | 100 | def configure_sources(update=False, |
2083 | 327 | sources_var='install_sources', | 101 | sources_var='install_sources', |
2084 | 328 | keys_var='install_keys'): | 102 | keys_var='install_keys'): |
2087 | 329 | """ | 103 | """Configure multiple sources from charm configuration. |
2086 | 330 | Configure multiple sources from charm configuration. | ||
2088 | 331 | 104 | ||
2089 | 332 | The lists are encoded as yaml fragments in the configuration. | 105 | The lists are encoded as yaml fragments in the configuration. |
2091 | 333 | The frament needs to be included as a string. Sources and their | 106 | The fragment needs to be included as a string. Sources and their |
2092 | 334 | corresponding keys are of the types supported by add_source(). | 107 | corresponding keys are of the types supported by add_source(). |
2093 | 335 | 108 | ||
2094 | 336 | Example config: | 109 | Example config: |
2095 | @@ -362,12 +135,11 @@ | |||
2096 | 362 | for source, key in zip(sources, keys): | 135 | for source, key in zip(sources, keys): |
2097 | 363 | add_source(source, key) | 136 | add_source(source, key) |
2098 | 364 | if update: | 137 | if update: |
2100 | 365 | apt_update(fatal=True) | 138 | fetch.update(fatal=True) |
2101 | 366 | 139 | ||
2102 | 367 | 140 | ||
2103 | 368 | def install_remote(source, *args, **kwargs): | 141 | def install_remote(source, *args, **kwargs): |
2106 | 369 | """ | 142 | """Install a file tree from a remote source. |
2105 | 370 | Install a file tree from a remote source | ||
2107 | 371 | 143 | ||
2108 | 372 | The specified source should be a url of the form: | 144 | The specified source should be a url of the form: |
2109 | 373 | scheme://[host]/path[#[option=value][&...]] | 145 | scheme://[host]/path[#[option=value][&...]] |
2110 | @@ -390,19 +162,17 @@ | |||
2111 | 390 | # We ONLY check for True here because can_handle may return a string | 162 | # We ONLY check for True here because can_handle may return a string |
2112 | 391 | # explaining why it can't handle a given source. | 163 | # explaining why it can't handle a given source. |
2113 | 392 | handlers = [h for h in plugins() if h.can_handle(source) is True] | 164 | handlers = [h for h in plugins() if h.can_handle(source) is True] |
2114 | 393 | installed_to = None | ||
2115 | 394 | for handler in handlers: | 165 | for handler in handlers: |
2116 | 395 | try: | 166 | try: |
2118 | 396 | installed_to = handler.install(source, *args, **kwargs) | 167 | return handler.install(source, *args, **kwargs) |
2119 | 397 | except UnhandledSource as e: | 168 | except UnhandledSource as e: |
2120 | 398 | log('Install source attempt unsuccessful: {}'.format(e), | 169 | log('Install source attempt unsuccessful: {}'.format(e), |
2121 | 399 | level='WARNING') | 170 | level='WARNING') |
2125 | 400 | if not installed_to: | 171 | raise UnhandledSource("No handler found for source {}".format(source)) |
2123 | 401 | raise UnhandledSource("No handler found for source {}".format(source)) | ||
2124 | 402 | return installed_to | ||
2126 | 403 | 172 | ||
2127 | 404 | 173 | ||
2128 | 405 | def install_from_config(config_var_name): | 174 | def install_from_config(config_var_name): |
2129 | 175 | """Install a file from config.""" | ||
2130 | 406 | charm_config = config() | 176 | charm_config = config() |
2131 | 407 | source = charm_config[config_var_name] | 177 | source = charm_config[config_var_name] |
2132 | 408 | return install_remote(source) | 178 | return install_remote(source) |
2133 | @@ -425,40 +195,3 @@ | |||
2134 | 425 | log("FetchHandler {} not found, skipping plugin".format( | 195 | log("FetchHandler {} not found, skipping plugin".format( |
2135 | 426 | handler_name)) | 196 | handler_name)) |
2136 | 427 | return plugin_list | 197 | return plugin_list |
2137 | 428 | |||
2138 | 429 | |||
2139 | 430 | def _run_apt_command(cmd, fatal=False): | ||
2140 | 431 | """ | ||
2141 | 432 | Run an APT command, checking output and retrying if the fatal flag is set | ||
2142 | 433 | to True. | ||
2143 | 434 | |||
2144 | 435 | :param: cmd: str: The apt command to run. | ||
2145 | 436 | :param: fatal: bool: Whether the command's output should be checked and | ||
2146 | 437 | retried. | ||
2147 | 438 | """ | ||
2148 | 439 | env = os.environ.copy() | ||
2149 | 440 | |||
2150 | 441 | if 'DEBIAN_FRONTEND' not in env: | ||
2151 | 442 | env['DEBIAN_FRONTEND'] = 'noninteractive' | ||
2152 | 443 | |||
2153 | 444 | if fatal: | ||
2154 | 445 | retry_count = 0 | ||
2155 | 446 | result = None | ||
2156 | 447 | |||
2157 | 448 | # If the command is considered "fatal", we need to retry if the apt | ||
2158 | 449 | # lock was not acquired. | ||
2159 | 450 | |||
2160 | 451 | while result is None or result == APT_NO_LOCK: | ||
2161 | 452 | try: | ||
2162 | 453 | result = subprocess.check_call(cmd, env=env) | ||
2163 | 454 | except subprocess.CalledProcessError as e: | ||
2164 | 455 | retry_count = retry_count + 1 | ||
2165 | 456 | if retry_count > APT_NO_LOCK_RETRY_COUNT: | ||
2166 | 457 | raise | ||
2167 | 458 | result = e.returncode | ||
2168 | 459 | log("Couldn't acquire DPKG lock. Will retry in {} seconds." | ||
2169 | 460 | "".format(APT_NO_LOCK_RETRY_DELAY)) | ||
2170 | 461 | time.sleep(APT_NO_LOCK_RETRY_DELAY) | ||
2171 | 462 | |||
2172 | 463 | else: | ||
2173 | 464 | subprocess.call(cmd, env=env) | ||
2174 | 465 | 198 | ||
2175 | === modified file 'charmhelpers/fetch/archiveurl.py' | |||
2176 | --- charmhelpers/fetch/archiveurl.py 2015-12-11 15:23:38 +0000 | |||
2177 | +++ charmhelpers/fetch/archiveurl.py 2017-03-04 02:50:20 +0000 | |||
2178 | @@ -1,18 +1,16 @@ | |||
2179 | 1 | # Copyright 2014-2015 Canonical Limited. | 1 | # Copyright 2014-2015 Canonical Limited. |
2180 | 2 | # | 2 | # |
2194 | 3 | # This file is part of charm-helpers. | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); |
2195 | 4 | # | 4 | # you may not use this file except in compliance with the License. |
2196 | 5 | # charm-helpers is free software: you can redistribute it and/or modify | 5 | # You may obtain a copy of the License at |
2197 | 6 | # it under the terms of the GNU Lesser General Public License version 3 as | 6 | # |
2198 | 7 | # published by the Free Software Foundation. | 7 | # http://www.apache.org/licenses/LICENSE-2.0 |
2199 | 8 | # | 8 | # |
2200 | 9 | # charm-helpers is distributed in the hope that it will be useful, | 9 | # Unless required by applicable law or agreed to in writing, software |
2201 | 10 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 10 | # distributed under the License is distributed on an "AS IS" BASIS, |
2202 | 11 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
2203 | 12 | # GNU Lesser General Public License for more details. | 12 | # See the License for the specific language governing permissions and |
2204 | 13 | # | 13 | # limitations under the License. |
2192 | 14 | # You should have received a copy of the GNU Lesser General Public License | ||
2193 | 15 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
2205 | 16 | 14 | ||
2206 | 17 | import os | 15 | import os |
2207 | 18 | import hashlib | 16 | import hashlib |
2208 | 19 | 17 | ||
2209 | === modified file 'charmhelpers/fetch/bzrurl.py' | |||
2210 | --- charmhelpers/fetch/bzrurl.py 2016-05-06 07:12:40 +0000 | |||
2211 | +++ charmhelpers/fetch/bzrurl.py 2017-03-04 02:50:20 +0000 | |||
2212 | @@ -1,18 +1,16 @@ | |||
2213 | 1 | # Copyright 2014-2015 Canonical Limited. | 1 | # Copyright 2014-2015 Canonical Limited. |
2214 | 2 | # | 2 | # |
2228 | 3 | # This file is part of charm-helpers. | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); |
2229 | 4 | # | 4 | # you may not use this file except in compliance with the License. |
2230 | 5 | # charm-helpers is free software: you can redistribute it and/or modify | 5 | # You may obtain a copy of the License at |
2231 | 6 | # it under the terms of the GNU Lesser General Public License version 3 as | 6 | # |
2232 | 7 | # published by the Free Software Foundation. | 7 | # http://www.apache.org/licenses/LICENSE-2.0 |
2233 | 8 | # | 8 | # |
2234 | 9 | # charm-helpers is distributed in the hope that it will be useful, | 9 | # Unless required by applicable law or agreed to in writing, software |
2235 | 10 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 10 | # distributed under the License is distributed on an "AS IS" BASIS, |
2236 | 11 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
2237 | 12 | # GNU Lesser General Public License for more details. | 12 | # See the License for the specific language governing permissions and |
2238 | 13 | # | 13 | # limitations under the License. |
2226 | 14 | # You should have received a copy of the GNU Lesser General Public License | ||
2227 | 15 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
2239 | 16 | 14 | ||
2240 | 17 | import os | 15 | import os |
2241 | 18 | from subprocess import check_call | 16 | from subprocess import check_call |
2242 | @@ -20,19 +18,20 @@ | |||
2243 | 20 | BaseFetchHandler, | 18 | BaseFetchHandler, |
2244 | 21 | UnhandledSource, | 19 | UnhandledSource, |
2245 | 22 | filter_installed_packages, | 20 | filter_installed_packages, |
2247 | 23 | apt_install, | 21 | install, |
2248 | 24 | ) | 22 | ) |
2249 | 25 | from charmhelpers.core.host import mkdir | 23 | from charmhelpers.core.host import mkdir |
2250 | 26 | 24 | ||
2251 | 27 | 25 | ||
2252 | 28 | if filter_installed_packages(['bzr']) != []: | 26 | if filter_installed_packages(['bzr']) != []: |
2254 | 29 | apt_install(['bzr']) | 27 | install(['bzr']) |
2255 | 30 | if filter_installed_packages(['bzr']) != []: | 28 | if filter_installed_packages(['bzr']) != []: |
2256 | 31 | raise NotImplementedError('Unable to install bzr') | 29 | raise NotImplementedError('Unable to install bzr') |
2257 | 32 | 30 | ||
2258 | 33 | 31 | ||
2259 | 34 | class BzrUrlFetchHandler(BaseFetchHandler): | 32 | class BzrUrlFetchHandler(BaseFetchHandler): |
2261 | 35 | """Handler for bazaar branches via generic and lp URLs""" | 33 | """Handler for bazaar branches via generic and lp URLs.""" |
2262 | 34 | |||
2263 | 36 | def can_handle(self, source): | 35 | def can_handle(self, source): |
2264 | 37 | url_parts = self.parse_url(source) | 36 | url_parts = self.parse_url(source) |
2265 | 38 | if url_parts.scheme not in ('bzr+ssh', 'lp', ''): | 37 | if url_parts.scheme not in ('bzr+ssh', 'lp', ''): |
2266 | @@ -42,15 +41,23 @@ | |||
2267 | 42 | else: | 41 | else: |
2268 | 43 | return True | 42 | return True |
2269 | 44 | 43 | ||
2271 | 45 | def branch(self, source, dest): | 44 | def branch(self, source, dest, revno=None): |
2272 | 46 | if not self.can_handle(source): | 45 | if not self.can_handle(source): |
2273 | 47 | raise UnhandledSource("Cannot handle {}".format(source)) | 46 | raise UnhandledSource("Cannot handle {}".format(source)) |
2274 | 47 | cmd_opts = [] | ||
2275 | 48 | if revno: | ||
2276 | 49 | cmd_opts += ['-r', str(revno)] | ||
2277 | 48 | if os.path.exists(dest): | 50 | if os.path.exists(dest): |
2279 | 49 | check_call(['bzr', 'pull', '--overwrite', '-d', dest, source]) | 51 | cmd = ['bzr', 'pull'] |
2280 | 52 | cmd += cmd_opts | ||
2281 | 53 | cmd += ['--overwrite', '-d', dest, source] | ||
2282 | 50 | else: | 54 | else: |
2284 | 51 | check_call(['bzr', 'branch', source, dest]) | 55 | cmd = ['bzr', 'branch'] |
2285 | 56 | cmd += cmd_opts | ||
2286 | 57 | cmd += [source, dest] | ||
2287 | 58 | check_call(cmd) | ||
2288 | 52 | 59 | ||
2290 | 53 | def install(self, source, dest=None): | 60 | def install(self, source, dest=None, revno=None): |
2291 | 54 | url_parts = self.parse_url(source) | 61 | url_parts = self.parse_url(source) |
2292 | 55 | branch_name = url_parts.path.strip("/").split("/")[-1] | 62 | branch_name = url_parts.path.strip("/").split("/")[-1] |
2293 | 56 | if dest: | 63 | if dest: |
2294 | @@ -59,10 +66,11 @@ | |||
2295 | 59 | dest_dir = os.path.join(os.environ.get('CHARM_DIR'), "fetched", | 66 | dest_dir = os.path.join(os.environ.get('CHARM_DIR'), "fetched", |
2296 | 60 | branch_name) | 67 | branch_name) |
2297 | 61 | 68 | ||
2300 | 62 | if not os.path.exists(dest_dir): | 69 | if dest and not os.path.exists(dest): |
2301 | 63 | mkdir(dest_dir, perms=0o755) | 70 | mkdir(dest, perms=0o755) |
2302 | 71 | |||
2303 | 64 | try: | 72 | try: |
2305 | 65 | self.branch(source, dest_dir) | 73 | self.branch(source, dest_dir, revno) |
2306 | 66 | except OSError as e: | 74 | except OSError as e: |
2307 | 67 | raise UnhandledSource(e.strerror) | 75 | raise UnhandledSource(e.strerror) |
2308 | 68 | return dest_dir | 76 | return dest_dir |
2309 | 69 | 77 | ||
2310 | === added file 'charmhelpers/fetch/centos.py' | |||
2311 | --- charmhelpers/fetch/centos.py 1970-01-01 00:00:00 +0000 | |||
2312 | +++ charmhelpers/fetch/centos.py 2017-03-04 02:50:20 +0000 | |||
2313 | @@ -0,0 +1,171 @@ | |||
2314 | 1 | # Copyright 2014-2015 Canonical Limited. | ||
2315 | 2 | # | ||
2316 | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); | ||
2317 | 4 | # you may not use this file except in compliance with the License. | ||
2318 | 5 | # You may obtain a copy of the License at | ||
2319 | 6 | # | ||
2320 | 7 | # http://www.apache.org/licenses/LICENSE-2.0 | ||
2321 | 8 | # | ||
2322 | 9 | # Unless required by applicable law or agreed to in writing, software | ||
2323 | 10 | # distributed under the License is distributed on an "AS IS" BASIS, | ||
2324 | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
2325 | 12 | # See the License for the specific language governing permissions and | ||
2326 | 13 | # limitations under the License. | ||
2327 | 14 | |||
2328 | 15 | import subprocess | ||
2329 | 16 | import os | ||
2330 | 17 | import time | ||
2331 | 18 | import six | ||
2332 | 19 | import yum | ||
2333 | 20 | |||
2334 | 21 | from tempfile import NamedTemporaryFile | ||
2335 | 22 | from charmhelpers.core.hookenv import log | ||
2336 | 23 | |||
2337 | 24 | YUM_NO_LOCK = 1 # The return code for "couldn't acquire lock" in YUM. | ||
2338 | 25 | YUM_NO_LOCK_RETRY_DELAY = 10 # Wait 10 seconds between apt lock checks. | ||
2339 | 26 | YUM_NO_LOCK_RETRY_COUNT = 30 # Retry to acquire the lock X times. | ||
2340 | 27 | |||
2341 | 28 | |||
2342 | 29 | def filter_installed_packages(packages): | ||
2343 | 30 | """Return a list of packages that require installation.""" | ||
2344 | 31 | yb = yum.YumBase() | ||
2345 | 32 | package_list = yb.doPackageLists() | ||
2346 | 33 | temp_cache = {p.base_package_name: 1 for p in package_list['installed']} | ||
2347 | 34 | |||
2348 | 35 | _pkgs = [p for p in packages if not temp_cache.get(p, False)] | ||
2349 | 36 | return _pkgs | ||
2350 | 37 | |||
2351 | 38 | |||
2352 | 39 | def install(packages, options=None, fatal=False): | ||
2353 | 40 | """Install one or more packages.""" | ||
2354 | 41 | cmd = ['yum', '--assumeyes'] | ||
2355 | 42 | if options is not None: | ||
2356 | 43 | cmd.extend(options) | ||
2357 | 44 | cmd.append('install') | ||
2358 | 45 | if isinstance(packages, six.string_types): | ||
2359 | 46 | cmd.append(packages) | ||
2360 | 47 | else: | ||
2361 | 48 | cmd.extend(packages) | ||
2362 | 49 | log("Installing {} with options: {}".format(packages, | ||
2363 | 50 | options)) | ||
2364 | 51 | _run_yum_command(cmd, fatal) | ||
2365 | 52 | |||
2366 | 53 | |||
2367 | 54 | def upgrade(options=None, fatal=False, dist=False): | ||
2368 | 55 | """Upgrade all packages.""" | ||
2369 | 56 | cmd = ['yum', '--assumeyes'] | ||
2370 | 57 | if options is not None: | ||
2371 | 58 | cmd.extend(options) | ||
2372 | 59 | cmd.append('upgrade') | ||
2373 | 60 | log("Upgrading with options: {}".format(options)) | ||
2374 | 61 | _run_yum_command(cmd, fatal) | ||
2375 | 62 | |||
2376 | 63 | |||
2377 | 64 | def update(fatal=False): | ||
2378 | 65 | """Update local yum cache.""" | ||
2379 | 66 | cmd = ['yum', '--assumeyes', 'update'] | ||
2380 | 67 | log("Update with fatal: {}".format(fatal)) | ||
2381 | 68 | _run_yum_command(cmd, fatal) | ||
2382 | 69 | |||
2383 | 70 | |||
2384 | 71 | def purge(packages, fatal=False): | ||
2385 | 72 | """Purge one or more packages.""" | ||
2386 | 73 | cmd = ['yum', '--assumeyes', 'remove'] | ||
2387 | 74 | if isinstance(packages, six.string_types): | ||
2388 | 75 | cmd.append(packages) | ||
2389 | 76 | else: | ||
2390 | 77 | cmd.extend(packages) | ||
2391 | 78 | log("Purging {}".format(packages)) | ||
2392 | 79 | _run_yum_command(cmd, fatal) | ||
2393 | 80 | |||
2394 | 81 | |||
2395 | 82 | def yum_search(packages): | ||
2396 | 83 | """Search for a package.""" | ||
2397 | 84 | output = {} | ||
2398 | 85 | cmd = ['yum', 'search'] | ||
2399 | 86 | if isinstance(packages, six.string_types): | ||
2400 | 87 | cmd.append(packages) | ||
2401 | 88 | else: | ||
2402 | 89 | cmd.extend(packages) | ||
2403 | 90 | log("Searching for {}".format(packages)) | ||
2404 | 91 | result = subprocess.check_output(cmd) | ||
2405 | 92 | for package in list(packages): | ||
2406 | 93 | output[package] = package in result | ||
2407 | 94 | return output | ||
2408 | 95 | |||
2409 | 96 | |||
2410 | 97 | def add_source(source, key=None): | ||
2411 | 98 | """Add a package source to this system. | ||
2412 | 99 | |||
2413 | 100 | @param source: a URL with a rpm package | ||
2414 | 101 | |||
2415 | 102 | @param key: A key to be added to the system's keyring and used | ||
2416 | 103 | to verify the signatures on packages. Ideally, this should be an | ||
2417 | 104 | ASCII format GPG public key including the block headers. A GPG key | ||
2418 | 105 | id may also be used, but be aware that only insecure protocols are | ||
2419 | 106 | available to retrieve the actual public key from a public keyserver | ||
2420 | 107 | placing your Juju environment at risk. | ||
2421 | 108 | """ | ||
2422 | 109 | if source is None: | ||
2423 | 110 | log('Source is not present. Skipping') | ||
2424 | 111 | return | ||
2425 | 112 | |||
2426 | 113 | if source.startswith('http'): | ||
2427 | 114 | directory = '/etc/yum.repos.d/' | ||
2428 | 115 | for filename in os.listdir(directory): | ||
2429 | 116 | with open(directory + filename, 'r') as rpm_file: | ||
2430 | 117 | if source in rpm_file.read(): | ||
2431 | 118 | break | ||
2432 | 119 | else: | ||
2433 | 120 | log("Add source: {!r}".format(source)) | ||
2434 | 121 | # write in the charms.repo | ||
2435 | 122 | with open(directory + 'Charms.repo', 'a') as rpm_file: | ||
2436 | 123 | rpm_file.write('[%s]\n' % source[7:].replace('/', '_')) | ||
2437 | 124 | rpm_file.write('name=%s\n' % source[7:]) | ||
2438 | 125 | rpm_file.write('baseurl=%s\n\n' % source) | ||
2439 | 126 | else: | ||
2440 | 127 | log("Unknown source: {!r}".format(source)) | ||
2441 | 128 | |||
2442 | 129 | if key: | ||
2443 | 130 | if '-----BEGIN PGP PUBLIC KEY BLOCK-----' in key: | ||
2444 | 131 | with NamedTemporaryFile('w+') as key_file: | ||
2445 | 132 | key_file.write(key) | ||
2446 | 133 | key_file.flush() | ||
2447 | 134 | key_file.seek(0) | ||
2448 | 135 | subprocess.check_call(['rpm', '--import', key_file]) | ||
2449 | 136 | else: | ||
2450 | 137 | subprocess.check_call(['rpm', '--import', key]) | ||
2451 | 138 | |||
2452 | 139 | |||
2453 | 140 | def _run_yum_command(cmd, fatal=False): | ||
2454 | 141 | """Run an YUM command. | ||
2455 | 142 | |||
2456 | 143 | Checks the output and retry if the fatal flag is set to True. | ||
2457 | 144 | |||
2458 | 145 | :param: cmd: str: The yum command to run. | ||
2459 | 146 | :param: fatal: bool: Whether the command's output should be checked and | ||
2460 | 147 | retried. | ||
2461 | 148 | """ | ||
2462 | 149 | env = os.environ.copy() | ||
2463 | 150 | |||
2464 | 151 | if fatal: | ||
2465 | 152 | retry_count = 0 | ||
2466 | 153 | result = None | ||
2467 | 154 | |||
2468 | 155 | # If the command is considered "fatal", we need to retry if the yum | ||
2469 | 156 | # lock was not acquired. | ||
2470 | 157 | |||
2471 | 158 | while result is None or result == YUM_NO_LOCK: | ||
2472 | 159 | try: | ||
2473 | 160 | result = subprocess.check_call(cmd, env=env) | ||
2474 | 161 | except subprocess.CalledProcessError as e: | ||
2475 | 162 | retry_count = retry_count + 1 | ||
2476 | 163 | if retry_count > YUM_NO_LOCK_RETRY_COUNT: | ||
2477 | 164 | raise | ||
2478 | 165 | result = e.returncode | ||
2479 | 166 | log("Couldn't acquire YUM lock. Will retry in {} seconds." | ||
2480 | 167 | "".format(YUM_NO_LOCK_RETRY_DELAY)) | ||
2481 | 168 | time.sleep(YUM_NO_LOCK_RETRY_DELAY) | ||
2482 | 169 | |||
2483 | 170 | else: | ||
2484 | 171 | subprocess.call(cmd, env=env) | ||
2485 | 0 | 172 | ||
2486 | === modified file 'charmhelpers/fetch/giturl.py' | |||
2487 | --- charmhelpers/fetch/giturl.py 2016-05-06 07:12:40 +0000 | |||
2488 | +++ charmhelpers/fetch/giturl.py 2017-03-04 02:50:20 +0000 | |||
2489 | @@ -1,18 +1,16 @@ | |||
2490 | 1 | # Copyright 2014-2015 Canonical Limited. | 1 | # Copyright 2014-2015 Canonical Limited. |
2491 | 2 | # | 2 | # |
2505 | 3 | # This file is part of charm-helpers. | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); |
2506 | 4 | # | 4 | # you may not use this file except in compliance with the License. |
2507 | 5 | # charm-helpers is free software: you can redistribute it and/or modify | 5 | # You may obtain a copy of the License at |
2508 | 6 | # it under the terms of the GNU Lesser General Public License version 3 as | 6 | # |
2509 | 7 | # published by the Free Software Foundation. | 7 | # http://www.apache.org/licenses/LICENSE-2.0 |
2510 | 8 | # | 8 | # |
2511 | 9 | # charm-helpers is distributed in the hope that it will be useful, | 9 | # Unless required by applicable law or agreed to in writing, software |
2512 | 10 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 10 | # distributed under the License is distributed on an "AS IS" BASIS, |
2513 | 11 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
2514 | 12 | # GNU Lesser General Public License for more details. | 12 | # See the License for the specific language governing permissions and |
2515 | 13 | # | 13 | # limitations under the License. |
2503 | 14 | # You should have received a copy of the GNU Lesser General Public License | ||
2504 | 15 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
2516 | 16 | 14 | ||
2517 | 17 | import os | 15 | import os |
2518 | 18 | from subprocess import check_call, CalledProcessError | 16 | from subprocess import check_call, CalledProcessError |
2519 | @@ -20,17 +18,18 @@ | |||
2520 | 20 | BaseFetchHandler, | 18 | BaseFetchHandler, |
2521 | 21 | UnhandledSource, | 19 | UnhandledSource, |
2522 | 22 | filter_installed_packages, | 20 | filter_installed_packages, |
2524 | 23 | apt_install, | 21 | install, |
2525 | 24 | ) | 22 | ) |
2526 | 25 | 23 | ||
2527 | 26 | if filter_installed_packages(['git']) != []: | 24 | if filter_installed_packages(['git']) != []: |
2529 | 27 | apt_install(['git']) | 25 | install(['git']) |
2530 | 28 | if filter_installed_packages(['git']) != []: | 26 | if filter_installed_packages(['git']) != []: |
2531 | 29 | raise NotImplementedError('Unable to install git') | 27 | raise NotImplementedError('Unable to install git') |
2532 | 30 | 28 | ||
2533 | 31 | 29 | ||
2534 | 32 | class GitUrlFetchHandler(BaseFetchHandler): | 30 | class GitUrlFetchHandler(BaseFetchHandler): |
2536 | 33 | """Handler for git branches via generic and github URLs""" | 31 | """Handler for git branches via generic and github URLs.""" |
2537 | 32 | |||
2538 | 34 | def can_handle(self, source): | 33 | def can_handle(self, source): |
2539 | 35 | url_parts = self.parse_url(source) | 34 | url_parts = self.parse_url(source) |
2540 | 36 | # TODO (mattyw) no support for ssh git@ yet | 35 | # TODO (mattyw) no support for ssh git@ yet |
2541 | 37 | 36 | ||
2542 | === added file 'charmhelpers/fetch/snap.py' | |||
2543 | --- charmhelpers/fetch/snap.py 1970-01-01 00:00:00 +0000 | |||
2544 | +++ charmhelpers/fetch/snap.py 2017-03-04 02:50:20 +0000 | |||
2545 | @@ -0,0 +1,122 @@ | |||
2546 | 1 | # Copyright 2014-2017 Canonical Limited. | ||
2547 | 2 | # | ||
2548 | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); | ||
2549 | 4 | # you may not use this file except in compliance with the License. | ||
2550 | 5 | # You may obtain a copy of the License at | ||
2551 | 6 | # | ||
2552 | 7 | # http://www.apache.org/licenses/LICENSE-2.0 | ||
2553 | 8 | # | ||
2554 | 9 | # Unless required by applicable law or agreed to in writing, software | ||
2555 | 10 | # distributed under the License is distributed on an "AS IS" BASIS, | ||
2556 | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
2557 | 12 | # See the License for the specific language governing permissions and | ||
2558 | 13 | # limitations under the License. | ||
2559 | 14 | """ | ||
2560 | 15 | Charm helpers snap for classic charms. | ||
2561 | 16 | |||
2562 | 17 | If writing reactive charms, use the snap layer: | ||
2563 | 18 | https://lists.ubuntu.com/archives/snapcraft/2016-September/001114.html | ||
2564 | 19 | """ | ||
2565 | 20 | import subprocess | ||
2566 | 21 | from os import environ | ||
2567 | 22 | from time import sleep | ||
2568 | 23 | from charmhelpers.core.hookenv import log | ||
2569 | 24 | |||
2570 | 25 | __author__ = 'Joseph Borg <joseph.borg@canonical.com>' | ||
2571 | 26 | |||
2572 | 27 | SNAP_NO_LOCK = 1 # The return code for "couldn't acquire lock" in Snap (hopefully this will be improved). | ||
2573 | 28 | SNAP_NO_LOCK_RETRY_DELAY = 10 # Wait X seconds between Snap lock checks. | ||
2574 | 29 | SNAP_NO_LOCK_RETRY_COUNT = 30 # Retry to acquire the lock X times. | ||
2575 | 30 | |||
2576 | 31 | |||
2577 | 32 | class CouldNotAcquireLockException(Exception): | ||
2578 | 33 | pass | ||
2579 | 34 | |||
2580 | 35 | |||
2581 | 36 | def _snap_exec(commands): | ||
2582 | 37 | """ | ||
2583 | 38 | Execute snap commands. | ||
2584 | 39 | |||
2585 | 40 | :param commands: List commands | ||
2586 | 41 | :return: Integer exit code | ||
2587 | 42 | """ | ||
2588 | 43 | assert type(commands) == list | ||
2589 | 44 | |||
2590 | 45 | retry_count = 0 | ||
2591 | 46 | return_code = None | ||
2592 | 47 | |||
2593 | 48 | while return_code is None or return_code == SNAP_NO_LOCK: | ||
2594 | 49 | try: | ||
2595 | 50 | return_code = subprocess.check_call(['snap'] + commands, env=environ) | ||
2596 | 51 | except subprocess.CalledProcessError as e: | ||
2597 | 52 | retry_count += + 1 | ||
2598 | 53 | if retry_count > SNAP_NO_LOCK_RETRY_COUNT: | ||
2599 | 54 | raise CouldNotAcquireLockException('Could not aquire lock after %s attempts' % SNAP_NO_LOCK_RETRY_COUNT) | ||
2600 | 55 | return_code = e.returncode | ||
2601 | 56 | log('Snap failed to acquire lock, trying again in %s seconds.' % SNAP_NO_LOCK_RETRY_DELAY, level='WARN') | ||
2602 | 57 | sleep(SNAP_NO_LOCK_RETRY_DELAY) | ||
2603 | 58 | |||
2604 | 59 | return return_code | ||
2605 | 60 | |||
2606 | 61 | |||
2607 | 62 | def snap_install(packages, *flags): | ||
2608 | 63 | """ | ||
2609 | 64 | Install a snap package. | ||
2610 | 65 | |||
2611 | 66 | :param packages: String or List String package name | ||
2612 | 67 | :param flags: List String flags to pass to install command | ||
2613 | 68 | :return: Integer return code from snap | ||
2614 | 69 | """ | ||
2615 | 70 | if type(packages) is not list: | ||
2616 | 71 | packages = [packages] | ||
2617 | 72 | |||
2618 | 73 | flags = list(flags) | ||
2619 | 74 | |||
2620 | 75 | message = 'Installing snap(s) "%s"' % ', '.join(packages) | ||
2621 | 76 | if flags: | ||
2622 | 77 | message += ' with option(s) "%s"' % ', '.join(flags) | ||
2623 | 78 | |||
2624 | 79 | log(message, level='INFO') | ||
2625 | 80 | return _snap_exec(['install'] + flags + packages) | ||
2626 | 81 | |||
2627 | 82 | |||
2628 | 83 | def snap_remove(packages, *flags): | ||
2629 | 84 | """ | ||
2630 | 85 | Remove a snap package. | ||
2631 | 86 | |||
2632 | 87 | :param packages: String or List String package name | ||
2633 | 88 | :param flags: List String flags to pass to remove command | ||
2634 | 89 | :return: Integer return code from snap | ||
2635 | 90 | """ | ||
2636 | 91 | if type(packages) is not list: | ||
2637 | 92 | packages = [packages] | ||
2638 | 93 | |||
2639 | 94 | flags = list(flags) | ||
2640 | 95 | |||
2641 | 96 | message = 'Removing snap(s) "%s"' % ', '.join(packages) | ||
2642 | 97 | if flags: | ||
2643 | 98 | message += ' with options "%s"' % ', '.join(flags) | ||
2644 | 99 | |||
2645 | 100 | log(message, level='INFO') | ||
2646 | 101 | return _snap_exec(['remove'] + flags + packages) | ||
2647 | 102 | |||
2648 | 103 | |||
2649 | 104 | def snap_refresh(packages, *flags): | ||
2650 | 105 | """ | ||
2651 | 106 | Refresh / Update snap package. | ||
2652 | 107 | |||
2653 | 108 | :param packages: String or List String package name | ||
2654 | 109 | :param flags: List String flags to pass to refresh command | ||
2655 | 110 | :return: Integer return code from snap | ||
2656 | 111 | """ | ||
2657 | 112 | if type(packages) is not list: | ||
2658 | 113 | packages = [packages] | ||
2659 | 114 | |||
2660 | 115 | flags = list(flags) | ||
2661 | 116 | |||
2662 | 117 | message = 'Refreshing snap(s) "%s"' % ', '.join(packages) | ||
2663 | 118 | if flags: | ||
2664 | 119 | message += ' with options "%s"' % ', '.join(flags) | ||
2665 | 120 | |||
2666 | 121 | log(message, level='INFO') | ||
2667 | 122 | return _snap_exec(['refresh'] + flags + packages) | ||
2668 | 0 | 123 | ||
2669 | === added file 'charmhelpers/fetch/ubuntu.py' | |||
2670 | --- charmhelpers/fetch/ubuntu.py 1970-01-01 00:00:00 +0000 | |||
2671 | +++ charmhelpers/fetch/ubuntu.py 2017-03-04 02:50:20 +0000 | |||
2672 | @@ -0,0 +1,364 @@ | |||
2673 | 1 | # Copyright 2014-2015 Canonical Limited. | ||
2674 | 2 | # | ||
2675 | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); | ||
2676 | 4 | # you may not use this file except in compliance with the License. | ||
2677 | 5 | # You may obtain a copy of the License at | ||
2678 | 6 | # | ||
2679 | 7 | # http://www.apache.org/licenses/LICENSE-2.0 | ||
2680 | 8 | # | ||
2681 | 9 | # Unless required by applicable law or agreed to in writing, software | ||
2682 | 10 | # distributed under the License is distributed on an "AS IS" BASIS, | ||
2683 | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
2684 | 12 | # See the License for the specific language governing permissions and | ||
2685 | 13 | # limitations under the License. | ||
2686 | 14 | |||
2687 | 15 | import os | ||
2688 | 16 | import six | ||
2689 | 17 | import time | ||
2690 | 18 | import subprocess | ||
2691 | 19 | |||
2692 | 20 | from tempfile import NamedTemporaryFile | ||
2693 | 21 | from charmhelpers.core.host import ( | ||
2694 | 22 | lsb_release | ||
2695 | 23 | ) | ||
2696 | 24 | from charmhelpers.core.hookenv import log | ||
2697 | 25 | from charmhelpers.fetch import SourceConfigError | ||
2698 | 26 | |||
2699 | 27 | CLOUD_ARCHIVE = """# Ubuntu Cloud Archive | ||
2700 | 28 | deb http://ubuntu-cloud.archive.canonical.com/ubuntu {} main | ||
2701 | 29 | """ | ||
2702 | 30 | |||
2703 | 31 | PROPOSED_POCKET = """# Proposed | ||
2704 | 32 | deb http://archive.ubuntu.com/ubuntu {}-proposed main universe multiverse restricted | ||
2705 | 33 | """ | ||
2706 | 34 | |||
2707 | 35 | CLOUD_ARCHIVE_POCKETS = { | ||
2708 | 36 | # Folsom | ||
2709 | 37 | 'folsom': 'precise-updates/folsom', | ||
2710 | 38 | 'precise-folsom': 'precise-updates/folsom', | ||
2711 | 39 | 'precise-folsom/updates': 'precise-updates/folsom', | ||
2712 | 40 | 'precise-updates/folsom': 'precise-updates/folsom', | ||
2713 | 41 | 'folsom/proposed': 'precise-proposed/folsom', | ||
2714 | 42 | 'precise-folsom/proposed': 'precise-proposed/folsom', | ||
2715 | 43 | 'precise-proposed/folsom': 'precise-proposed/folsom', | ||
2716 | 44 | # Grizzly | ||
2717 | 45 | 'grizzly': 'precise-updates/grizzly', | ||
2718 | 46 | 'precise-grizzly': 'precise-updates/grizzly', | ||
2719 | 47 | 'precise-grizzly/updates': 'precise-updates/grizzly', | ||
2720 | 48 | 'precise-updates/grizzly': 'precise-updates/grizzly', | ||
2721 | 49 | 'grizzly/proposed': 'precise-proposed/grizzly', | ||
2722 | 50 | 'precise-grizzly/proposed': 'precise-proposed/grizzly', | ||
2723 | 51 | 'precise-proposed/grizzly': 'precise-proposed/grizzly', | ||
2724 | 52 | # Havana | ||
2725 | 53 | 'havana': 'precise-updates/havana', | ||
2726 | 54 | 'precise-havana': 'precise-updates/havana', | ||
2727 | 55 | 'precise-havana/updates': 'precise-updates/havana', | ||
2728 | 56 | 'precise-updates/havana': 'precise-updates/havana', | ||
2729 | 57 | 'havana/proposed': 'precise-proposed/havana', | ||
2730 | 58 | 'precise-havana/proposed': 'precise-proposed/havana', | ||
2731 | 59 | 'precise-proposed/havana': 'precise-proposed/havana', | ||
2732 | 60 | # Icehouse | ||
2733 | 61 | 'icehouse': 'precise-updates/icehouse', | ||
2734 | 62 | 'precise-icehouse': 'precise-updates/icehouse', | ||
2735 | 63 | 'precise-icehouse/updates': 'precise-updates/icehouse', | ||
2736 | 64 | 'precise-updates/icehouse': 'precise-updates/icehouse', | ||
2737 | 65 | 'icehouse/proposed': 'precise-proposed/icehouse', | ||
2738 | 66 | 'precise-icehouse/proposed': 'precise-proposed/icehouse', | ||
2739 | 67 | 'precise-proposed/icehouse': 'precise-proposed/icehouse', | ||
2740 | 68 | # Juno | ||
2741 | 69 | 'juno': 'trusty-updates/juno', | ||
2742 | 70 | 'trusty-juno': 'trusty-updates/juno', | ||
2743 | 71 | 'trusty-juno/updates': 'trusty-updates/juno', | ||
2744 | 72 | 'trusty-updates/juno': 'trusty-updates/juno', | ||
2745 | 73 | 'juno/proposed': 'trusty-proposed/juno', | ||
2746 | 74 | 'trusty-juno/proposed': 'trusty-proposed/juno', | ||
2747 | 75 | 'trusty-proposed/juno': 'trusty-proposed/juno', | ||
2748 | 76 | # Kilo | ||
2749 | 77 | 'kilo': 'trusty-updates/kilo', | ||
2750 | 78 | 'trusty-kilo': 'trusty-updates/kilo', | ||
2751 | 79 | 'trusty-kilo/updates': 'trusty-updates/kilo', | ||
2752 | 80 | 'trusty-updates/kilo': 'trusty-updates/kilo', | ||
2753 | 81 | 'kilo/proposed': 'trusty-proposed/kilo', | ||
2754 | 82 | 'trusty-kilo/proposed': 'trusty-proposed/kilo', | ||
2755 | 83 | 'trusty-proposed/kilo': 'trusty-proposed/kilo', | ||
2756 | 84 | # Liberty | ||
2757 | 85 | 'liberty': 'trusty-updates/liberty', | ||
2758 | 86 | 'trusty-liberty': 'trusty-updates/liberty', | ||
2759 | 87 | 'trusty-liberty/updates': 'trusty-updates/liberty', | ||
2760 | 88 | 'trusty-updates/liberty': 'trusty-updates/liberty', | ||
2761 | 89 | 'liberty/proposed': 'trusty-proposed/liberty', | ||
2762 | 90 | 'trusty-liberty/proposed': 'trusty-proposed/liberty', | ||
2763 | 91 | 'trusty-proposed/liberty': 'trusty-proposed/liberty', | ||
2764 | 92 | # Mitaka | ||
2765 | 93 | 'mitaka': 'trusty-updates/mitaka', | ||
2766 | 94 | 'trusty-mitaka': 'trusty-updates/mitaka', | ||
2767 | 95 | 'trusty-mitaka/updates': 'trusty-updates/mitaka', | ||
2768 | 96 | 'trusty-updates/mitaka': 'trusty-updates/mitaka', | ||
2769 | 97 | 'mitaka/proposed': 'trusty-proposed/mitaka', | ||
2770 | 98 | 'trusty-mitaka/proposed': 'trusty-proposed/mitaka', | ||
2771 | 99 | 'trusty-proposed/mitaka': 'trusty-proposed/mitaka', | ||
2772 | 100 | # Newton | ||
2773 | 101 | 'newton': 'xenial-updates/newton', | ||
2774 | 102 | 'xenial-newton': 'xenial-updates/newton', | ||
2775 | 103 | 'xenial-newton/updates': 'xenial-updates/newton', | ||
2776 | 104 | 'xenial-updates/newton': 'xenial-updates/newton', | ||
2777 | 105 | 'newton/proposed': 'xenial-proposed/newton', | ||
2778 | 106 | 'xenial-newton/proposed': 'xenial-proposed/newton', | ||
2779 | 107 | 'xenial-proposed/newton': 'xenial-proposed/newton', | ||
2780 | 108 | # Ocata | ||
2781 | 109 | 'ocata': 'xenial-updates/ocata', | ||
2782 | 110 | 'xenial-ocata': 'xenial-updates/ocata', | ||
2783 | 111 | 'xenial-ocata/updates': 'xenial-updates/ocata', | ||
2784 | 112 | 'xenial-updates/ocata': 'xenial-updates/ocata', | ||
2785 | 113 | 'ocata/proposed': 'xenial-proposed/ocata', | ||
2786 | 114 | 'xenial-ocata/proposed': 'xenial-proposed/ocata', | ||
2787 | 115 | 'xenial-ocata/newton': 'xenial-proposed/ocata', | ||
2788 | 116 | } | ||
2789 | 117 | |||
2790 | 118 | APT_NO_LOCK = 100 # The return code for "couldn't acquire lock" in APT. | ||
2791 | 119 | CMD_RETRY_DELAY = 10 # Wait 10 seconds between command retries. | ||
2792 | 120 | CMD_RETRY_COUNT = 30 # Retry a failing fatal command X times. | ||
2793 | 121 | |||
2794 | 122 | |||
2795 | 123 | def filter_installed_packages(packages): | ||
2796 | 124 | """Return a list of packages that require installation.""" | ||
2797 | 125 | cache = apt_cache() | ||
2798 | 126 | _pkgs = [] | ||
2799 | 127 | for package in packages: | ||
2800 | 128 | try: | ||
2801 | 129 | p = cache[package] | ||
2802 | 130 | p.current_ver or _pkgs.append(package) | ||
2803 | 131 | except KeyError: | ||
2804 | 132 | log('Package {} has no installation candidate.'.format(package), | ||
2805 | 133 | level='WARNING') | ||
2806 | 134 | _pkgs.append(package) | ||
2807 | 135 | return _pkgs | ||
2808 | 136 | |||
2809 | 137 | |||
2810 | 138 | def apt_cache(in_memory=True, progress=None): | ||
2811 | 139 | """Build and return an apt cache.""" | ||
2812 | 140 | from apt import apt_pkg | ||
2813 | 141 | apt_pkg.init() | ||
2814 | 142 | if in_memory: | ||
2815 | 143 | apt_pkg.config.set("Dir::Cache::pkgcache", "") | ||
2816 | 144 | apt_pkg.config.set("Dir::Cache::srcpkgcache", "") | ||
2817 | 145 | return apt_pkg.Cache(progress) | ||
2818 | 146 | |||
2819 | 147 | |||
2820 | 148 | def install(packages, options=None, fatal=False): | ||
2821 | 149 | """Install one or more packages.""" | ||
2822 | 150 | if options is None: | ||
2823 | 151 | options = ['--option=Dpkg::Options::=--force-confold'] | ||
2824 | 152 | |||
2825 | 153 | cmd = ['apt-get', '--assume-yes'] | ||
2826 | 154 | cmd.extend(options) | ||
2827 | 155 | cmd.append('install') | ||
2828 | 156 | if isinstance(packages, six.string_types): | ||
2829 | 157 | cmd.append(packages) | ||
2830 | 158 | else: | ||
2831 | 159 | cmd.extend(packages) | ||
2832 | 160 | log("Installing {} with options: {}".format(packages, | ||
2833 | 161 | options)) | ||
2834 | 162 | _run_apt_command(cmd, fatal) | ||
2835 | 163 | |||
2836 | 164 | |||
2837 | 165 | def upgrade(options=None, fatal=False, dist=False): | ||
2838 | 166 | """Upgrade all packages.""" | ||
2839 | 167 | if options is None: | ||
2840 | 168 | options = ['--option=Dpkg::Options::=--force-confold'] | ||
2841 | 169 | |||
2842 | 170 | cmd = ['apt-get', '--assume-yes'] | ||
2843 | 171 | cmd.extend(options) | ||
2844 | 172 | if dist: | ||
2845 | 173 | cmd.append('dist-upgrade') | ||
2846 | 174 | else: | ||
2847 | 175 | cmd.append('upgrade') | ||
2848 | 176 | log("Upgrading with options: {}".format(options)) | ||
2849 | 177 | _run_apt_command(cmd, fatal) | ||
2850 | 178 | |||
2851 | 179 | |||
2852 | 180 | def update(fatal=False): | ||
2853 | 181 | """Update local apt cache.""" | ||
2854 | 182 | cmd = ['apt-get', 'update'] | ||
2855 | 183 | _run_apt_command(cmd, fatal) | ||
2856 | 184 | |||
2857 | 185 | |||
2858 | 186 | def purge(packages, fatal=False): | ||
2859 | 187 | """Purge one or more packages.""" | ||
2860 | 188 | cmd = ['apt-get', '--assume-yes', 'purge'] | ||
2861 | 189 | if isinstance(packages, six.string_types): | ||
2862 | 190 | cmd.append(packages) | ||
2863 | 191 | else: | ||
2864 | 192 | cmd.extend(packages) | ||
2865 | 193 | log("Purging {}".format(packages)) | ||
2866 | 194 | _run_apt_command(cmd, fatal) | ||
2867 | 195 | |||
2868 | 196 | |||
2869 | 197 | def apt_mark(packages, mark, fatal=False): | ||
2870 | 198 | """Flag one or more packages using apt-mark.""" | ||
2871 | 199 | log("Marking {} as {}".format(packages, mark)) | ||
2872 | 200 | cmd = ['apt-mark', mark] | ||
2873 | 201 | if isinstance(packages, six.string_types): | ||
2874 | 202 | cmd.append(packages) | ||
2875 | 203 | else: | ||
2876 | 204 | cmd.extend(packages) | ||
2877 | 205 | |||
2878 | 206 | if fatal: | ||
2879 | 207 | subprocess.check_call(cmd, universal_newlines=True) | ||
2880 | 208 | else: | ||
2881 | 209 | subprocess.call(cmd, universal_newlines=True) | ||
2882 | 210 | |||
2883 | 211 | |||
2884 | 212 | def apt_hold(packages, fatal=False): | ||
2885 | 213 | return apt_mark(packages, 'hold', fatal=fatal) | ||
2886 | 214 | |||
2887 | 215 | |||
2888 | 216 | def apt_unhold(packages, fatal=False): | ||
2889 | 217 | return apt_mark(packages, 'unhold', fatal=fatal) | ||
2890 | 218 | |||
2891 | 219 | |||
2892 | 220 | def add_source(source, key=None): | ||
2893 | 221 | """Add a package source to this system. | ||
2894 | 222 | |||
2895 | 223 | @param source: a URL or sources.list entry, as supported by | ||
2896 | 224 | add-apt-repository(1). Examples:: | ||
2897 | 225 | |||
2898 | 226 | ppa:charmers/example | ||
2899 | 227 | deb https://stub:key@private.example.com/ubuntu trusty main | ||
2900 | 228 | |||
2901 | 229 | In addition: | ||
2902 | 230 | 'proposed:' may be used to enable the standard 'proposed' | ||
2903 | 231 | pocket for the release. | ||
2904 | 232 | 'cloud:' may be used to activate official cloud archive pockets, | ||
2905 | 233 | such as 'cloud:icehouse' | ||
2906 | 234 | 'distro' may be used as a noop | ||
2907 | 235 | |||
2908 | 236 | @param key: A key to be added to the system's APT keyring and used | ||
2909 | 237 | to verify the signatures on packages. Ideally, this should be an | ||
2910 | 238 | ASCII format GPG public key including the block headers. A GPG key | ||
2911 | 239 | id may also be used, but be aware that only insecure protocols are | ||
2912 | 240 | available to retrieve the actual public key from a public keyserver | ||
2913 | 241 | placing your Juju environment at risk. ppa and cloud archive keys | ||
2914 | 242 | are securely added automtically, so sould not be provided. | ||
2915 | 243 | """ | ||
2916 | 244 | if source is None: | ||
2917 | 245 | log('Source is not present. Skipping') | ||
2918 | 246 | return | ||
2919 | 247 | |||
2920 | 248 | if (source.startswith('ppa:') or | ||
2921 | 249 | source.startswith('http') or | ||
2922 | 250 | source.startswith('deb ') or | ||
2923 | 251 | source.startswith('cloud-archive:')): | ||
2924 | 252 | cmd = ['add-apt-repository', '--yes', source] | ||
2925 | 253 | _run_with_retries(cmd) | ||
2926 | 254 | elif source.startswith('cloud:'): | ||
2927 | 255 | install(filter_installed_packages(['ubuntu-cloud-keyring']), | ||
2928 | 256 | fatal=True) | ||
2929 | 257 | pocket = source.split(':')[-1] | ||
2930 | 258 | if pocket not in CLOUD_ARCHIVE_POCKETS: | ||
2931 | 259 | raise SourceConfigError( | ||
2932 | 260 | 'Unsupported cloud: source option %s' % | ||
2933 | 261 | pocket) | ||
2934 | 262 | actual_pocket = CLOUD_ARCHIVE_POCKETS[pocket] | ||
2935 | 263 | with open('/etc/apt/sources.list.d/cloud-archive.list', 'w') as apt: | ||
2936 | 264 | apt.write(CLOUD_ARCHIVE.format(actual_pocket)) | ||
2937 | 265 | elif source == 'proposed': | ||
2938 | 266 | release = lsb_release()['DISTRIB_CODENAME'] | ||
2939 | 267 | with open('/etc/apt/sources.list.d/proposed.list', 'w') as apt: | ||
2940 | 268 | apt.write(PROPOSED_POCKET.format(release)) | ||
2941 | 269 | elif source == 'distro': | ||
2942 | 270 | pass | ||
2943 | 271 | else: | ||
2944 | 272 | log("Unknown source: {!r}".format(source)) | ||
2945 | 273 | |||
2946 | 274 | if key: | ||
2947 | 275 | if '-----BEGIN PGP PUBLIC KEY BLOCK-----' in key: | ||
2948 | 276 | with NamedTemporaryFile('w+') as key_file: | ||
2949 | 277 | key_file.write(key) | ||
2950 | 278 | key_file.flush() | ||
2951 | 279 | key_file.seek(0) | ||
2952 | 280 | subprocess.check_call(['apt-key', 'add', '-'], stdin=key_file) | ||
2953 | 281 | else: | ||
2954 | 282 | # Note that hkp: is in no way a secure protocol. Using a | ||
2955 | 283 | # GPG key id is pointless from a security POV unless you | ||
2956 | 284 | # absolutely trust your network and DNS. | ||
2957 | 285 | subprocess.check_call(['apt-key', 'adv', '--keyserver', | ||
2958 | 286 | 'hkp://keyserver.ubuntu.com:80', '--recv', | ||
2959 | 287 | key]) | ||
2960 | 288 | |||
2961 | 289 | |||
2962 | 290 | def _run_with_retries(cmd, max_retries=CMD_RETRY_COUNT, retry_exitcodes=(1,), | ||
2963 | 291 | retry_message="", cmd_env=None): | ||
2964 | 292 | """Run a command and retry until success or max_retries is reached. | ||
2965 | 293 | |||
2966 | 294 | :param: cmd: str: The apt command to run. | ||
2967 | 295 | :param: max_retries: int: The number of retries to attempt on a fatal | ||
2968 | 296 | command. Defaults to CMD_RETRY_COUNT. | ||
2969 | 297 | :param: retry_exitcodes: tuple: Optional additional exit codes to retry. | ||
2970 | 298 | Defaults to retry on exit code 1. | ||
2971 | 299 | :param: retry_message: str: Optional log prefix emitted during retries. | ||
2972 | 300 | :param: cmd_env: dict: Environment variables to add to the command run. | ||
2973 | 301 | """ | ||
2974 | 302 | |||
2975 | 303 | env = os.environ.copy() | ||
2976 | 304 | if cmd_env: | ||
2977 | 305 | env.update(cmd_env) | ||
2978 | 306 | |||
2979 | 307 | if not retry_message: | ||
2980 | 308 | retry_message = "Failed executing '{}'".format(" ".join(cmd)) | ||
2981 | 309 | retry_message += ". Will retry in {} seconds".format(CMD_RETRY_DELAY) | ||
2982 | 310 | |||
2983 | 311 | retry_count = 0 | ||
2984 | 312 | result = None | ||
2985 | 313 | |||
2986 | 314 | retry_results = (None,) + retry_exitcodes | ||
2987 | 315 | while result in retry_results: | ||
2988 | 316 | try: | ||
2989 | 317 | result = subprocess.check_call(cmd, env=env) | ||
2990 | 318 | except subprocess.CalledProcessError as e: | ||
2991 | 319 | retry_count = retry_count + 1 | ||
2992 | 320 | if retry_count > max_retries: | ||
2993 | 321 | raise | ||
2994 | 322 | result = e.returncode | ||
2995 | 323 | log(retry_message) | ||
2996 | 324 | time.sleep(CMD_RETRY_DELAY) | ||
2997 | 325 | |||
2998 | 326 | |||
2999 | 327 | def _run_apt_command(cmd, fatal=False): | ||
3000 | 328 | """Run an apt command with optional retries. | ||
3001 | 329 | |||
3002 | 330 | :param: fatal: bool: Whether the command's output should be checked and | ||
3003 | 331 | retried. | ||
3004 | 332 | """ | ||
3005 | 333 | # Provide DEBIAN_FRONTEND=noninteractive if not present in the environment. | ||
3006 | 334 | cmd_env = { | ||
3007 | 335 | 'DEBIAN_FRONTEND': os.environ.get('DEBIAN_FRONTEND', 'noninteractive')} | ||
3008 | 336 | |||
3009 | 337 | if fatal: | ||
3010 | 338 | _run_with_retries( | ||
3011 | 339 | cmd, cmd_env=cmd_env, retry_exitcodes=(1, APT_NO_LOCK,), | ||
3012 | 340 | retry_message="Couldn't acquire DPKG lock") | ||
3013 | 341 | else: | ||
3014 | 342 | env = os.environ.copy() | ||
3015 | 343 | env.update(cmd_env) | ||
3016 | 344 | subprocess.call(cmd, env=env) | ||
3017 | 345 | |||
3018 | 346 | |||
3019 | 347 | def get_upstream_version(package): | ||
3020 | 348 | """Determine upstream version based on installed package | ||
3021 | 349 | |||
3022 | 350 | @returns None (if not installed) or the upstream version | ||
3023 | 351 | """ | ||
3024 | 352 | import apt_pkg | ||
3025 | 353 | cache = apt_cache() | ||
3026 | 354 | try: | ||
3027 | 355 | pkg = cache[package] | ||
3028 | 356 | except: | ||
3029 | 357 | # the package is unknown to the current apt cache. | ||
3030 | 358 | return None | ||
3031 | 359 | |||
3032 | 360 | if not pkg.current_ver: | ||
3033 | 361 | # package is known, but no version is currently installed. | ||
3034 | 362 | return None | ||
3035 | 363 | |||
3036 | 364 | return apt_pkg.upstream_version(pkg.current_ver.ver_str) | ||
3037 | 0 | 365 | ||
3038 | === added file 'charmhelpers/osplatform.py' | |||
3039 | --- charmhelpers/osplatform.py 1970-01-01 00:00:00 +0000 | |||
3040 | +++ charmhelpers/osplatform.py 2017-03-04 02:50:20 +0000 | |||
3041 | @@ -0,0 +1,25 @@ | |||
3042 | 1 | import platform | ||
3043 | 2 | |||
3044 | 3 | |||
3045 | 4 | def get_platform(): | ||
3046 | 5 | """Return the current OS platform. | ||
3047 | 6 | |||
3048 | 7 | For example: if current os platform is Ubuntu then a string "ubuntu" | ||
3049 | 8 | will be returned (which is the name of the module). | ||
3050 | 9 | This string is used to decide which platform module should be imported. | ||
3051 | 10 | """ | ||
3052 | 11 | # linux_distribution is deprecated and will be removed in Python 3.7 | ||
3053 | 12 | # Warings *not* disabled, as we certainly need to fix this. | ||
3054 | 13 | tuple_platform = platform.linux_distribution() | ||
3055 | 14 | current_platform = tuple_platform[0] | ||
3056 | 15 | if "Ubuntu" in current_platform: | ||
3057 | 16 | return "ubuntu" | ||
3058 | 17 | elif "CentOS" in current_platform: | ||
3059 | 18 | return "centos" | ||
3060 | 19 | elif "debian" in current_platform: | ||
3061 | 20 | # Stock Python does not detect Ubuntu and instead returns debian. | ||
3062 | 21 | # Or at least it does in some build environments like Travis CI | ||
3063 | 22 | return "ubuntu" | ||
3064 | 23 | else: | ||
3065 | 24 | raise RuntimeError("This module is not supported on {}." | ||
3066 | 25 | .format(current_platform)) | ||
3067 | 0 | 26 | ||
3068 | === modified file 'lib/apt.py' | |||
3069 | --- lib/apt.py 2017-03-01 15:05:23 +0000 | |||
3070 | +++ lib/apt.py 2017-03-04 02:50:20 +0000 | |||
3071 | @@ -14,7 +14,8 @@ | |||
3072 | 14 | 14 | ||
3073 | 15 | LANDSCAPE_PACKAGES = ("landscape-server", "landscape-hashids") | 15 | LANDSCAPE_PACKAGES = ("landscape-server", "landscape-hashids") |
3074 | 16 | INSTALL_PACKAGES = LANDSCAPE_PACKAGES + ("python-minimal", "python-psutil") | 16 | INSTALL_PACKAGES = LANDSCAPE_PACKAGES + ("python-minimal", "python-psutil") |
3076 | 17 | PACKAGES_DEV = ("dpkg-dev", "devscripts", "pbuilder", "aptitude", "build-essential") | 17 | PACKAGES_DEV = ( |
3077 | 18 | "dpkg-dev", "devscripts", "pbuilder", "aptitude", "build-essential") | ||
3078 | 18 | TARBALL = "landscape-server_*.tar.gz" | 19 | TARBALL = "landscape-server_*.tar.gz" |
3079 | 19 | 20 | ||
3080 | 20 | # XXX Eventually we'll want to use a dedicated PPA, populated by Jenkins. | 21 | # XXX Eventually we'll want to use a dedicated PPA, populated by Jenkins. |
Command: make ci-test /ci.lscape. net/job/ latch-test- xenial/ 1447/
Result: Fail
Revno: 383
Branch: lp:~chad.smith/landscape-charm/add-apt-repository-retries
Jenkins: https:/