Merge lp:~chad.smith/landscape-charm/add-apt-repository-retries into lp:~landscape/landscape-charm/trunk
- add-apt-repository-retries
- Merge into trunk
Status: | Merged | ||||
---|---|---|---|---|---|
Approved by: | Chad Smith | ||||
Approved revision: | 386 | ||||
Merged at revision: | 383 | ||||
Proposed branch: | lp:~chad.smith/landscape-charm/add-apt-repository-retries | ||||
Merge into: | lp:~landscape/landscape-charm/trunk | ||||
Diff against target: |
3080 lines (+1633/-771) 34 files modified
charm-helpers.yaml (+1/-0) charmhelpers/__init__.py (+11/-13) charmhelpers/contrib/__init__.py (+11/-13) charmhelpers/contrib/hahelpers/__init__.py (+11/-13) charmhelpers/contrib/hahelpers/apache.py (+30/-17) charmhelpers/contrib/hahelpers/cluster.py (+70/-23) charmhelpers/core/__init__.py (+11/-13) charmhelpers/core/decorators.py (+11/-13) charmhelpers/core/files.py (+11/-13) charmhelpers/core/fstab.py (+11/-13) charmhelpers/core/hookenv.py (+73/-14) charmhelpers/core/host.py (+327/-126) charmhelpers/core/host_factory/centos.py (+56/-0) charmhelpers/core/host_factory/ubuntu.py (+56/-0) charmhelpers/core/hugepage.py (+11/-13) charmhelpers/core/kernel.py (+34/-30) charmhelpers/core/kernel_factory/centos.py (+17/-0) charmhelpers/core/kernel_factory/ubuntu.py (+13/-0) charmhelpers/core/services/__init__.py (+11/-13) charmhelpers/core/services/base.py (+11/-13) charmhelpers/core/services/helpers.py (+11/-13) charmhelpers/core/strutils.py (+11/-13) charmhelpers/core/sysctl.py (+11/-13) charmhelpers/core/templating.py (+19/-16) charmhelpers/core/unitdata.py (+11/-14) charmhelpers/fetch/__init__.py (+42/-309) charmhelpers/fetch/archiveurl.py (+11/-13) charmhelpers/fetch/bzrurl.py (+31/-23) charmhelpers/fetch/centos.py (+171/-0) charmhelpers/fetch/giturl.py (+15/-16) charmhelpers/fetch/snap.py (+122/-0) charmhelpers/fetch/ubuntu.py (+364/-0) charmhelpers/osplatform.py (+25/-0) lib/apt.py (+2/-1) |
||||
To merge this branch: | bzr merge lp:~chad.smith/landscape-charm/add-apt-repository-retries | ||||
Related bugs: |
|
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
🤖 Landscape Builder | test results | Approve | |
Eric Snow (community) | Approve | ||
Review via email:
|
Commit message
Sync of charmhelpers to get updated fetch.add_source which handles 30 retries upon exit 1
Add charmhelpers.
Description of the change
Sync of charmhelpers to get updated fetch.add_source which handles 30 retries upon exit 1. Generally this exit code is returned upon network failure like bug https:/
Retry fixes are in charmhelpers/
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
🤖 Landscape Builder (landscape-builder) : | # |
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
🤖 Landscape Builder (landscape-builder) wrote : | # |
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Eric Snow (ericsnowcurrently) wrote : | # |
As this doesn't introduce any changes, LGTM. :)
- 384. By Chad Smith
-
add charmhelpers.
osplatform dependency for charmhelpers.host get_platform - 385. By Chad Smith
-
make sync pulling in osplatform
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
🤖 Landscape Builder (landscape-builder) : | # |
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
🤖 Landscape Builder (landscape-builder) wrote : | # |
Command: make ci-test
Result: Fail
Revno: 385
Branch: lp:~chad.smith/landscape-charm/add-apt-repository-retries
Jenkins: https:/
- 386. By Chad Smith
-
lint me
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
🤖 Landscape Builder (landscape-builder) : | # |
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
🤖 Landscape Builder (landscape-builder) wrote : | # |
Command: make ci-test
Result: Success
Revno: 386
Branch: lp:~chad.smith/landscape-charm/add-apt-repository-retries
Jenkins: https:/
Preview Diff
1 | === modified file 'charm-helpers.yaml' |
2 | --- charm-helpers.yaml 2015-05-07 10:26:45 +0000 |
3 | +++ charm-helpers.yaml 2017-03-04 02:50:20 +0000 |
4 | @@ -4,4 +4,5 @@ |
5 | - __init__ |
6 | - core |
7 | - fetch |
8 | + - osplatform |
9 | - contrib.hahelpers |
10 | |
11 | === modified file 'charmhelpers/__init__.py' |
12 | --- charmhelpers/__init__.py 2015-01-28 08:59:02 +0000 |
13 | +++ charmhelpers/__init__.py 2017-03-04 02:50:20 +0000 |
14 | @@ -1,18 +1,16 @@ |
15 | # Copyright 2014-2015 Canonical Limited. |
16 | # |
17 | -# This file is part of charm-helpers. |
18 | -# |
19 | -# charm-helpers is free software: you can redistribute it and/or modify |
20 | -# it under the terms of the GNU Lesser General Public License version 3 as |
21 | -# published by the Free Software Foundation. |
22 | -# |
23 | -# charm-helpers is distributed in the hope that it will be useful, |
24 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
25 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
26 | -# GNU Lesser General Public License for more details. |
27 | -# |
28 | -# You should have received a copy of the GNU Lesser General Public License |
29 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
30 | +# Licensed under the Apache License, Version 2.0 (the "License"); |
31 | +# you may not use this file except in compliance with the License. |
32 | +# You may obtain a copy of the License at |
33 | +# |
34 | +# http://www.apache.org/licenses/LICENSE-2.0 |
35 | +# |
36 | +# Unless required by applicable law or agreed to in writing, software |
37 | +# distributed under the License is distributed on an "AS IS" BASIS, |
38 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
39 | +# See the License for the specific language governing permissions and |
40 | +# limitations under the License. |
41 | |
42 | # Bootstrap charm-helpers, installing its dependencies if necessary using |
43 | # only standard libraries. |
44 | |
45 | === modified file 'charmhelpers/contrib/__init__.py' |
46 | --- charmhelpers/contrib/__init__.py 2015-01-30 11:16:09 +0000 |
47 | +++ charmhelpers/contrib/__init__.py 2017-03-04 02:50:20 +0000 |
48 | @@ -1,15 +1,13 @@ |
49 | # Copyright 2014-2015 Canonical Limited. |
50 | # |
51 | -# This file is part of charm-helpers. |
52 | -# |
53 | -# charm-helpers is free software: you can redistribute it and/or modify |
54 | -# it under the terms of the GNU Lesser General Public License version 3 as |
55 | -# published by the Free Software Foundation. |
56 | -# |
57 | -# charm-helpers is distributed in the hope that it will be useful, |
58 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
59 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
60 | -# GNU Lesser General Public License for more details. |
61 | -# |
62 | -# You should have received a copy of the GNU Lesser General Public License |
63 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
64 | +# Licensed under the Apache License, Version 2.0 (the "License"); |
65 | +# you may not use this file except in compliance with the License. |
66 | +# You may obtain a copy of the License at |
67 | +# |
68 | +# http://www.apache.org/licenses/LICENSE-2.0 |
69 | +# |
70 | +# Unless required by applicable law or agreed to in writing, software |
71 | +# distributed under the License is distributed on an "AS IS" BASIS, |
72 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
73 | +# See the License for the specific language governing permissions and |
74 | +# limitations under the License. |
75 | |
76 | === modified file 'charmhelpers/contrib/hahelpers/__init__.py' |
77 | --- charmhelpers/contrib/hahelpers/__init__.py 2015-01-30 11:16:09 +0000 |
78 | +++ charmhelpers/contrib/hahelpers/__init__.py 2017-03-04 02:50:20 +0000 |
79 | @@ -1,15 +1,13 @@ |
80 | # Copyright 2014-2015 Canonical Limited. |
81 | # |
82 | -# This file is part of charm-helpers. |
83 | -# |
84 | -# charm-helpers is free software: you can redistribute it and/or modify |
85 | -# it under the terms of the GNU Lesser General Public License version 3 as |
86 | -# published by the Free Software Foundation. |
87 | -# |
88 | -# charm-helpers is distributed in the hope that it will be useful, |
89 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
90 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
91 | -# GNU Lesser General Public License for more details. |
92 | -# |
93 | -# You should have received a copy of the GNU Lesser General Public License |
94 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
95 | +# Licensed under the Apache License, Version 2.0 (the "License"); |
96 | +# you may not use this file except in compliance with the License. |
97 | +# You may obtain a copy of the License at |
98 | +# |
99 | +# http://www.apache.org/licenses/LICENSE-2.0 |
100 | +# |
101 | +# Unless required by applicable law or agreed to in writing, software |
102 | +# distributed under the License is distributed on an "AS IS" BASIS, |
103 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
104 | +# See the License for the specific language governing permissions and |
105 | +# limitations under the License. |
106 | |
107 | === modified file 'charmhelpers/contrib/hahelpers/apache.py' |
108 | --- charmhelpers/contrib/hahelpers/apache.py 2015-01-30 11:16:09 +0000 |
109 | +++ charmhelpers/contrib/hahelpers/apache.py 2017-03-04 02:50:20 +0000 |
110 | @@ -1,18 +1,16 @@ |
111 | # Copyright 2014-2015 Canonical Limited. |
112 | # |
113 | -# This file is part of charm-helpers. |
114 | -# |
115 | -# charm-helpers is free software: you can redistribute it and/or modify |
116 | -# it under the terms of the GNU Lesser General Public License version 3 as |
117 | -# published by the Free Software Foundation. |
118 | -# |
119 | -# charm-helpers is distributed in the hope that it will be useful, |
120 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
121 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
122 | -# GNU Lesser General Public License for more details. |
123 | -# |
124 | -# You should have received a copy of the GNU Lesser General Public License |
125 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
126 | +# Licensed under the Apache License, Version 2.0 (the "License"); |
127 | +# you may not use this file except in compliance with the License. |
128 | +# You may obtain a copy of the License at |
129 | +# |
130 | +# http://www.apache.org/licenses/LICENSE-2.0 |
131 | +# |
132 | +# Unless required by applicable law or agreed to in writing, software |
133 | +# distributed under the License is distributed on an "AS IS" BASIS, |
134 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
135 | +# See the License for the specific language governing permissions and |
136 | +# limitations under the License. |
137 | |
138 | # |
139 | # Copyright 2012 Canonical Ltd. |
140 | @@ -24,6 +22,7 @@ |
141 | # Adam Gandelman <adamg@ubuntu.com> |
142 | # |
143 | |
144 | +import os |
145 | import subprocess |
146 | |
147 | from charmhelpers.core.hookenv import ( |
148 | @@ -74,9 +73,23 @@ |
149 | return ca_cert |
150 | |
151 | |
152 | +def retrieve_ca_cert(cert_file): |
153 | + cert = None |
154 | + if os.path.isfile(cert_file): |
155 | + with open(cert_file, 'r') as crt: |
156 | + cert = crt.read() |
157 | + return cert |
158 | + |
159 | + |
160 | def install_ca_cert(ca_cert): |
161 | if ca_cert: |
162 | - with open('/usr/local/share/ca-certificates/keystone_juju_ca_cert.crt', |
163 | - 'w') as crt: |
164 | - crt.write(ca_cert) |
165 | - subprocess.check_call(['update-ca-certificates', '--fresh']) |
166 | + cert_file = ('/usr/local/share/ca-certificates/' |
167 | + 'keystone_juju_ca_cert.crt') |
168 | + old_cert = retrieve_ca_cert(cert_file) |
169 | + if old_cert and old_cert == ca_cert: |
170 | + log("CA cert is the same as installed version", level=INFO) |
171 | + else: |
172 | + log("Installing new CA cert", level=INFO) |
173 | + with open(cert_file, 'w') as crt: |
174 | + crt.write(ca_cert) |
175 | + subprocess.check_call(['update-ca-certificates', '--fresh']) |
176 | |
177 | === modified file 'charmhelpers/contrib/hahelpers/cluster.py' |
178 | --- charmhelpers/contrib/hahelpers/cluster.py 2015-07-03 09:13:26 +0000 |
179 | +++ charmhelpers/contrib/hahelpers/cluster.py 2017-03-04 02:50:20 +0000 |
180 | @@ -1,18 +1,16 @@ |
181 | # Copyright 2014-2015 Canonical Limited. |
182 | # |
183 | -# This file is part of charm-helpers. |
184 | -# |
185 | -# charm-helpers is free software: you can redistribute it and/or modify |
186 | -# it under the terms of the GNU Lesser General Public License version 3 as |
187 | -# published by the Free Software Foundation. |
188 | -# |
189 | -# charm-helpers is distributed in the hope that it will be useful, |
190 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
191 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
192 | -# GNU Lesser General Public License for more details. |
193 | -# |
194 | -# You should have received a copy of the GNU Lesser General Public License |
195 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
196 | +# Licensed under the Apache License, Version 2.0 (the "License"); |
197 | +# you may not use this file except in compliance with the License. |
198 | +# You may obtain a copy of the License at |
199 | +# |
200 | +# http://www.apache.org/licenses/LICENSE-2.0 |
201 | +# |
202 | +# Unless required by applicable law or agreed to in writing, software |
203 | +# distributed under the License is distributed on an "AS IS" BASIS, |
204 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
205 | +# See the License for the specific language governing permissions and |
206 | +# limitations under the License. |
207 | |
208 | # |
209 | # Copyright 2012 Canonical Ltd. |
210 | @@ -41,10 +39,11 @@ |
211 | relation_get, |
212 | config as config_get, |
213 | INFO, |
214 | - ERROR, |
215 | + DEBUG, |
216 | WARNING, |
217 | unit_get, |
218 | - is_leader as juju_is_leader |
219 | + is_leader as juju_is_leader, |
220 | + status_set, |
221 | ) |
222 | from charmhelpers.core.decorators import ( |
223 | retry_on_exception, |
224 | @@ -60,6 +59,10 @@ |
225 | pass |
226 | |
227 | |
228 | +class HAIncorrectConfig(Exception): |
229 | + pass |
230 | + |
231 | + |
232 | class CRMResourceNotFound(Exception): |
233 | pass |
234 | |
235 | @@ -274,27 +277,71 @@ |
236 | Obtains all relevant configuration from charm configuration required |
237 | for initiating a relation to hacluster: |
238 | |
239 | - ha-bindiface, ha-mcastport, vip |
240 | + ha-bindiface, ha-mcastport, vip, os-internal-hostname, |
241 | + os-admin-hostname, os-public-hostname, os-access-hostname |
242 | |
243 | param: exclude_keys: list of setting key(s) to be excluded. |
244 | returns: dict: A dict containing settings keyed by setting name. |
245 | - raises: HAIncompleteConfig if settings are missing. |
246 | + raises: HAIncompleteConfig if settings are missing or incorrect. |
247 | ''' |
248 | - settings = ['ha-bindiface', 'ha-mcastport', 'vip'] |
249 | + settings = ['ha-bindiface', 'ha-mcastport', 'vip', 'os-internal-hostname', |
250 | + 'os-admin-hostname', 'os-public-hostname', 'os-access-hostname'] |
251 | conf = {} |
252 | for setting in settings: |
253 | if exclude_keys and setting in exclude_keys: |
254 | continue |
255 | |
256 | conf[setting] = config_get(setting) |
257 | - missing = [] |
258 | - [missing.append(s) for s, v in six.iteritems(conf) if v is None] |
259 | - if missing: |
260 | - log('Insufficient config data to configure hacluster.', level=ERROR) |
261 | - raise HAIncompleteConfig |
262 | + |
263 | + if not valid_hacluster_config(): |
264 | + raise HAIncorrectConfig('Insufficient or incorrect config data to ' |
265 | + 'configure hacluster.') |
266 | return conf |
267 | |
268 | |
269 | +def valid_hacluster_config(): |
270 | + ''' |
271 | + Check that either vip or dns-ha is set. If dns-ha then one of os-*-hostname |
272 | + must be set. |
273 | + |
274 | + Note: ha-bindiface and ha-macastport both have defaults and will always |
275 | + be set. We only care that either vip or dns-ha is set. |
276 | + |
277 | + :returns: boolean: valid config returns true. |
278 | + raises: HAIncompatibileConfig if settings conflict. |
279 | + raises: HAIncompleteConfig if settings are missing. |
280 | + ''' |
281 | + vip = config_get('vip') |
282 | + dns = config_get('dns-ha') |
283 | + if not(bool(vip) ^ bool(dns)): |
284 | + msg = ('HA: Either vip or dns-ha must be set but not both in order to ' |
285 | + 'use high availability') |
286 | + status_set('blocked', msg) |
287 | + raise HAIncorrectConfig(msg) |
288 | + |
289 | + # If dns-ha then one of os-*-hostname must be set |
290 | + if dns: |
291 | + dns_settings = ['os-internal-hostname', 'os-admin-hostname', |
292 | + 'os-public-hostname', 'os-access-hostname'] |
293 | + # At this point it is unknown if one or all of the possible |
294 | + # network spaces are in HA. Validate at least one is set which is |
295 | + # the minimum required. |
296 | + for setting in dns_settings: |
297 | + if config_get(setting): |
298 | + log('DNS HA: At least one hostname is set {}: {}' |
299 | + ''.format(setting, config_get(setting)), |
300 | + level=DEBUG) |
301 | + return True |
302 | + |
303 | + msg = ('DNS HA: At least one os-*-hostname(s) must be set to use ' |
304 | + 'DNS HA') |
305 | + status_set('blocked', msg) |
306 | + raise HAIncompleteConfig(msg) |
307 | + |
308 | + log('VIP HA: VIP is set {}'.format(vip), level=DEBUG) |
309 | + return True |
310 | + |
311 | + |
312 | def canonical_url(configs, vip_setting='vip'): |
313 | ''' |
314 | Returns the correct HTTP URL to this host given the state of HTTPS |
315 | |
316 | === modified file 'charmhelpers/core/__init__.py' |
317 | --- charmhelpers/core/__init__.py 2015-01-28 08:59:02 +0000 |
318 | +++ charmhelpers/core/__init__.py 2017-03-04 02:50:20 +0000 |
319 | @@ -1,15 +1,13 @@ |
320 | # Copyright 2014-2015 Canonical Limited. |
321 | # |
322 | -# This file is part of charm-helpers. |
323 | -# |
324 | -# charm-helpers is free software: you can redistribute it and/or modify |
325 | -# it under the terms of the GNU Lesser General Public License version 3 as |
326 | -# published by the Free Software Foundation. |
327 | -# |
328 | -# charm-helpers is distributed in the hope that it will be useful, |
329 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
330 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
331 | -# GNU Lesser General Public License for more details. |
332 | -# |
333 | -# You should have received a copy of the GNU Lesser General Public License |
334 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
335 | +# Licensed under the Apache License, Version 2.0 (the "License"); |
336 | +# you may not use this file except in compliance with the License. |
337 | +# You may obtain a copy of the License at |
338 | +# |
339 | +# http://www.apache.org/licenses/LICENSE-2.0 |
340 | +# |
341 | +# Unless required by applicable law or agreed to in writing, software |
342 | +# distributed under the License is distributed on an "AS IS" BASIS, |
343 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
344 | +# See the License for the specific language governing permissions and |
345 | +# limitations under the License. |
346 | |
347 | === modified file 'charmhelpers/core/decorators.py' |
348 | --- charmhelpers/core/decorators.py 2015-01-28 08:59:02 +0000 |
349 | +++ charmhelpers/core/decorators.py 2017-03-04 02:50:20 +0000 |
350 | @@ -1,18 +1,16 @@ |
351 | # Copyright 2014-2015 Canonical Limited. |
352 | # |
353 | -# This file is part of charm-helpers. |
354 | -# |
355 | -# charm-helpers is free software: you can redistribute it and/or modify |
356 | -# it under the terms of the GNU Lesser General Public License version 3 as |
357 | -# published by the Free Software Foundation. |
358 | -# |
359 | -# charm-helpers is distributed in the hope that it will be useful, |
360 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
361 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
362 | -# GNU Lesser General Public License for more details. |
363 | -# |
364 | -# You should have received a copy of the GNU Lesser General Public License |
365 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
366 | +# Licensed under the Apache License, Version 2.0 (the "License"); |
367 | +# you may not use this file except in compliance with the License. |
368 | +# You may obtain a copy of the License at |
369 | +# |
370 | +# http://www.apache.org/licenses/LICENSE-2.0 |
371 | +# |
372 | +# Unless required by applicable law or agreed to in writing, software |
373 | +# distributed under the License is distributed on an "AS IS" BASIS, |
374 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
375 | +# See the License for the specific language governing permissions and |
376 | +# limitations under the License. |
377 | |
378 | # |
379 | # Copyright 2014 Canonical Ltd. |
380 | |
381 | === modified file 'charmhelpers/core/files.py' |
382 | --- charmhelpers/core/files.py 2015-12-11 15:23:38 +0000 |
383 | +++ charmhelpers/core/files.py 2017-03-04 02:50:20 +0000 |
384 | @@ -3,19 +3,17 @@ |
385 | |
386 | # Copyright 2014-2015 Canonical Limited. |
387 | # |
388 | -# This file is part of charm-helpers. |
389 | -# |
390 | -# charm-helpers is free software: you can redistribute it and/or modify |
391 | -# it under the terms of the GNU Lesser General Public License version 3 as |
392 | -# published by the Free Software Foundation. |
393 | -# |
394 | -# charm-helpers is distributed in the hope that it will be useful, |
395 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
396 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
397 | -# GNU Lesser General Public License for more details. |
398 | -# |
399 | -# You should have received a copy of the GNU Lesser General Public License |
400 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
401 | +# Licensed under the Apache License, Version 2.0 (the "License"); |
402 | +# you may not use this file except in compliance with the License. |
403 | +# You may obtain a copy of the License at |
404 | +# |
405 | +# http://www.apache.org/licenses/LICENSE-2.0 |
406 | +# |
407 | +# Unless required by applicable law or agreed to in writing, software |
408 | +# distributed under the License is distributed on an "AS IS" BASIS, |
409 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
410 | +# See the License for the specific language governing permissions and |
411 | +# limitations under the License. |
412 | |
413 | __author__ = 'Jorge Niedbalski <niedbalski@ubuntu.com>' |
414 | |
415 | |
416 | === modified file 'charmhelpers/core/fstab.py' |
417 | --- charmhelpers/core/fstab.py 2015-03-12 11:42:26 +0000 |
418 | +++ charmhelpers/core/fstab.py 2017-03-04 02:50:20 +0000 |
419 | @@ -3,19 +3,17 @@ |
420 | |
421 | # Copyright 2014-2015 Canonical Limited. |
422 | # |
423 | -# This file is part of charm-helpers. |
424 | -# |
425 | -# charm-helpers is free software: you can redistribute it and/or modify |
426 | -# it under the terms of the GNU Lesser General Public License version 3 as |
427 | -# published by the Free Software Foundation. |
428 | -# |
429 | -# charm-helpers is distributed in the hope that it will be useful, |
430 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
431 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
432 | -# GNU Lesser General Public License for more details. |
433 | -# |
434 | -# You should have received a copy of the GNU Lesser General Public License |
435 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
436 | +# Licensed under the Apache License, Version 2.0 (the "License"); |
437 | +# you may not use this file except in compliance with the License. |
438 | +# You may obtain a copy of the License at |
439 | +# |
440 | +# http://www.apache.org/licenses/LICENSE-2.0 |
441 | +# |
442 | +# Unless required by applicable law or agreed to in writing, software |
443 | +# distributed under the License is distributed on an "AS IS" BASIS, |
444 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
445 | +# See the License for the specific language governing permissions and |
446 | +# limitations under the License. |
447 | |
448 | import io |
449 | import os |
450 | |
451 | === modified file 'charmhelpers/core/hookenv.py' |
452 | --- charmhelpers/core/hookenv.py 2016-05-06 07:12:40 +0000 |
453 | +++ charmhelpers/core/hookenv.py 2017-03-04 02:50:20 +0000 |
454 | @@ -1,18 +1,16 @@ |
455 | # Copyright 2014-2015 Canonical Limited. |
456 | # |
457 | -# This file is part of charm-helpers. |
458 | -# |
459 | -# charm-helpers is free software: you can redistribute it and/or modify |
460 | -# it under the terms of the GNU Lesser General Public License version 3 as |
461 | -# published by the Free Software Foundation. |
462 | -# |
463 | -# charm-helpers is distributed in the hope that it will be useful, |
464 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
465 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
466 | -# GNU Lesser General Public License for more details. |
467 | -# |
468 | -# You should have received a copy of the GNU Lesser General Public License |
469 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
470 | +# Licensed under the Apache License, Version 2.0 (the "License"); |
471 | +# you may not use this file except in compliance with the License. |
472 | +# You may obtain a copy of the License at |
473 | +# |
474 | +# http://www.apache.org/licenses/LICENSE-2.0 |
475 | +# |
476 | +# Unless required by applicable law or agreed to in writing, software |
477 | +# distributed under the License is distributed on an "AS IS" BASIS, |
478 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
479 | +# See the License for the specific language governing permissions and |
480 | +# limitations under the License. |
481 | |
482 | "Interactions with the Juju environment" |
483 | # Copyright 2013 Canonical Ltd. |
484 | @@ -334,6 +332,8 @@ |
485 | config_cmd_line = ['config-get'] |
486 | if scope is not None: |
487 | config_cmd_line.append(scope) |
488 | + else: |
489 | + config_cmd_line.append('--all') |
490 | config_cmd_line.append('--format=json') |
491 | try: |
492 | config_data = json.loads( |
493 | @@ -616,6 +616,20 @@ |
494 | subprocess.check_call(_args) |
495 | |
496 | |
497 | +def open_ports(start, end, protocol="TCP"): |
498 | + """Opens a range of service network ports""" |
499 | + _args = ['open-port'] |
500 | + _args.append('{}-{}/{}'.format(start, end, protocol)) |
501 | + subprocess.check_call(_args) |
502 | + |
503 | + |
504 | +def close_ports(start, end, protocol="TCP"): |
505 | + """Close a range of service network ports""" |
506 | + _args = ['close-port'] |
507 | + _args.append('{}-{}/{}'.format(start, end, protocol)) |
508 | + subprocess.check_call(_args) |
509 | + |
510 | + |
511 | @cached |
512 | def unit_get(attribute): |
513 | """Get the unit ID for the remote unit""" |
514 | @@ -845,6 +859,20 @@ |
515 | return inner_translate_exc1 |
516 | |
517 | |
518 | +def application_version_set(version): |
519 | + """Charm authors may trigger this command from any hook to output what |
520 | + version of the application is running. This could be a package version, |
521 | + for instance postgres version 9.5. It could also be a build number or |
522 | + version control revision identifier, for instance git sha 6fb7ba68. """ |
523 | + |
524 | + cmd = ['application-version-set'] |
525 | + cmd.append(version) |
526 | + try: |
527 | + subprocess.check_call(cmd) |
528 | + except OSError: |
529 | + log("Application Version: {}".format(version)) |
530 | + |
531 | + |
532 | @translate_exc(from_exc=OSError, to_exc=NotImplementedError) |
533 | def is_leader(): |
534 | """Does the current unit hold the juju leadership |
535 | @@ -1006,4 +1034,35 @@ |
536 | :raise: NotImplementedError if run on Juju < 2.0 |
537 | ''' |
538 | cmd = ['network-get', '--primary-address', binding] |
539 | - return subprocess.check_output(cmd).strip() |
540 | + return subprocess.check_output(cmd).decode('UTF-8').strip() |
541 | + |
542 | + |
543 | +def add_metric(*args, **kwargs): |
544 | + """Add metric values. Values may be expressed with keyword arguments. For |
545 | + metric names containing dashes, these may be expressed as one or more |
546 | + 'key=value' positional arguments. May only be called from the collect-metrics |
547 | + hook.""" |
548 | + _args = ['add-metric'] |
549 | + _kvpairs = [] |
550 | + _kvpairs.extend(args) |
551 | + _kvpairs.extend(['{}={}'.format(k, v) for k, v in kwargs.items()]) |
552 | + _args.extend(sorted(_kvpairs)) |
553 | + try: |
554 | + subprocess.check_call(_args) |
555 | + return |
556 | + except EnvironmentError as e: |
557 | + if e.errno != errno.ENOENT: |
558 | + raise |
559 | + log_message = 'add-metric failed: {}'.format(' '.join(_kvpairs)) |
560 | + log(log_message, level='INFO') |
561 | + |
562 | + |
563 | +def meter_status(): |
564 | + """Get the meter status, if running in the meter-status-changed hook.""" |
565 | + return os.environ.get('JUJU_METER_STATUS') |
566 | + |
567 | + |
568 | +def meter_info(): |
569 | + """Get the meter status information, if running in the meter-status-changed |
570 | + hook.""" |
571 | + return os.environ.get('JUJU_METER_INFO') |
572 | |
573 | === modified file 'charmhelpers/core/host.py' |
574 | --- charmhelpers/core/host.py 2016-05-06 07:12:40 +0000 |
575 | +++ charmhelpers/core/host.py 2017-03-04 02:50:20 +0000 |
576 | @@ -1,18 +1,16 @@ |
577 | # Copyright 2014-2015 Canonical Limited. |
578 | # |
579 | -# This file is part of charm-helpers. |
580 | -# |
581 | -# charm-helpers is free software: you can redistribute it and/or modify |
582 | -# it under the terms of the GNU Lesser General Public License version 3 as |
583 | -# published by the Free Software Foundation. |
584 | -# |
585 | -# charm-helpers is distributed in the hope that it will be useful, |
586 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
587 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
588 | -# GNU Lesser General Public License for more details. |
589 | -# |
590 | -# You should have received a copy of the GNU Lesser General Public License |
591 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
592 | +# Licensed under the Apache License, Version 2.0 (the "License"); |
593 | +# you may not use this file except in compliance with the License. |
594 | +# You may obtain a copy of the License at |
595 | +# |
596 | +# http://www.apache.org/licenses/LICENSE-2.0 |
597 | +# |
598 | +# Unless required by applicable law or agreed to in writing, software |
599 | +# distributed under the License is distributed on an "AS IS" BASIS, |
600 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
601 | +# See the License for the specific language governing permissions and |
602 | +# limitations under the License. |
603 | |
604 | """Tools for working with the host system""" |
605 | # Copyright 2012 Canonical Ltd. |
606 | @@ -32,46 +30,162 @@ |
607 | import hashlib |
608 | import functools |
609 | import itertools |
610 | +import six |
611 | + |
612 | from contextlib import contextmanager |
613 | from collections import OrderedDict |
614 | - |
615 | -import six |
616 | - |
617 | from .hookenv import log |
618 | from .fstab import Fstab |
619 | - |
620 | - |
621 | -def service_start(service_name): |
622 | - """Start a system service""" |
623 | - return service('start', service_name) |
624 | - |
625 | - |
626 | -def service_stop(service_name): |
627 | - """Stop a system service""" |
628 | - return service('stop', service_name) |
629 | - |
630 | - |
631 | -def service_restart(service_name): |
632 | - """Restart a system service""" |
633 | +from charmhelpers.osplatform import get_platform |
634 | + |
635 | +__platform__ = get_platform() |
636 | +if __platform__ == "ubuntu": |
637 | + from charmhelpers.core.host_factory.ubuntu import ( |
638 | + service_available, |
639 | + add_new_group, |
640 | + lsb_release, |
641 | + cmp_pkgrevno, |
642 | + ) # flake8: noqa -- ignore F401 for this import |
643 | +elif __platform__ == "centos": |
644 | + from charmhelpers.core.host_factory.centos import ( |
645 | + service_available, |
646 | + add_new_group, |
647 | + lsb_release, |
648 | + cmp_pkgrevno, |
649 | + ) # flake8: noqa -- ignore F401 for this import |
650 | + |
651 | +UPDATEDB_PATH = '/etc/updatedb.conf' |
652 | + |
653 | +def service_start(service_name, **kwargs): |
654 | + """Start a system service. |
655 | + |
656 | + The specified service name is managed via the system level init system. |
657 | + Some init systems (e.g. upstart) require that additional arguments be |
658 | + provided in order to directly control service instances whereas other init |
659 | + systems allow for addressing instances of a service directly by name (e.g. |
660 | + systemd). |
661 | + |
662 | + The kwargs allow for the additional parameters to be passed to underlying |
663 | + init systems for those systems which require/allow for them. For example, |
664 | + the ceph-osd upstart script requires the id parameter to be passed along |
665 | + in order to identify which running daemon should be reloaded. The follow- |
666 | + ing example stops the ceph-osd service for instance id=4: |
667 | + |
668 | + service_stop('ceph-osd', id=4) |
669 | + |
670 | + :param service_name: the name of the service to stop |
671 | + :param **kwargs: additional parameters to pass to the init system when |
672 | + managing services. These will be passed as key=value |
673 | + parameters to the init system's commandline. kwargs |
674 | + are ignored for systemd enabled systems. |
675 | + """ |
676 | + return service('start', service_name, **kwargs) |
677 | + |
678 | + |
679 | +def service_stop(service_name, **kwargs): |
680 | + """Stop a system service. |
681 | + |
682 | + The specified service name is managed via the system level init system. |
683 | + Some init systems (e.g. upstart) require that additional arguments be |
684 | + provided in order to directly control service instances whereas other init |
685 | + systems allow for addressing instances of a service directly by name (e.g. |
686 | + systemd). |
687 | + |
688 | + The kwargs allow for the additional parameters to be passed to underlying |
689 | + init systems for those systems which require/allow for them. For example, |
690 | + the ceph-osd upstart script requires the id parameter to be passed along |
691 | + in order to identify which running daemon should be reloaded. The follow- |
692 | + ing example stops the ceph-osd service for instance id=4: |
693 | + |
694 | + service_stop('ceph-osd', id=4) |
695 | + |
696 | + :param service_name: the name of the service to stop |
697 | + :param **kwargs: additional parameters to pass to the init system when |
698 | + managing services. These will be passed as key=value |
699 | + parameters to the init system's commandline. kwargs |
700 | + are ignored for systemd enabled systems. |
701 | + """ |
702 | + return service('stop', service_name, **kwargs) |
703 | + |
704 | + |
705 | +def service_restart(service_name, **kwargs): |
706 | + """Restart a system service. |
707 | + |
708 | + The specified service name is managed via the system level init system. |
709 | + Some init systems (e.g. upstart) require that additional arguments be |
710 | + provided in order to directly control service instances whereas other init |
711 | + systems allow for addressing instances of a service directly by name (e.g. |
712 | + systemd). |
713 | + |
714 | + The kwargs allow for the additional parameters to be passed to underlying |
715 | + init systems for those systems which require/allow for them. For example, |
716 | + the ceph-osd upstart script requires the id parameter to be passed along |
717 | + in order to identify which running daemon should be restarted. The follow- |
718 | + ing example restarts the ceph-osd service for instance id=4: |
719 | + |
720 | + service_restart('ceph-osd', id=4) |
721 | + |
722 | + :param service_name: the name of the service to restart |
723 | + :param **kwargs: additional parameters to pass to the init system when |
724 | + managing services. These will be passed as key=value |
725 | + parameters to the init system's commandline. kwargs |
726 | + are ignored for init systems not allowing additional |
727 | + parameters via the commandline (systemd). |
728 | + """ |
729 | return service('restart', service_name) |
730 | |
731 | |
732 | -def service_reload(service_name, restart_on_failure=False): |
733 | +def service_reload(service_name, restart_on_failure=False, **kwargs): |
734 | """Reload a system service, optionally falling back to restart if |
735 | - reload fails""" |
736 | - service_result = service('reload', service_name) |
737 | + reload fails. |
738 | + |
739 | + The specified service name is managed via the system level init system. |
740 | + Some init systems (e.g. upstart) require that additional arguments be |
741 | + provided in order to directly control service instances whereas other init |
742 | + systems allow for addressing instances of a service directly by name (e.g. |
743 | + systemd). |
744 | + |
745 | + The kwargs allow for the additional parameters to be passed to underlying |
746 | + init systems for those systems which require/allow for them. For example, |
747 | + the ceph-osd upstart script requires the id parameter to be passed along |
748 | + in order to identify which running daemon should be reloaded. The follow- |
749 | + ing example restarts the ceph-osd service for instance id=4: |
750 | + |
751 | + service_reload('ceph-osd', id=4) |
752 | + |
753 | + :param service_name: the name of the service to reload |
754 | + :param restart_on_failure: boolean indicating whether to fallback to a |
755 | + restart if the reload fails. |
756 | + :param **kwargs: additional parameters to pass to the init system when |
757 | + managing services. These will be passed as key=value |
758 | + parameters to the init system's commandline. kwargs |
759 | + are ignored for init systems not allowing additional |
760 | + parameters via the commandline (systemd). |
761 | + """ |
762 | + service_result = service('reload', service_name, **kwargs) |
763 | if not service_result and restart_on_failure: |
764 | - service_result = service('restart', service_name) |
765 | + service_result = service('restart', service_name, **kwargs) |
766 | return service_result |
767 | |
768 | |
769 | -def service_pause(service_name, init_dir="/etc/init", initd_dir="/etc/init.d"): |
770 | +def service_pause(service_name, init_dir="/etc/init", initd_dir="/etc/init.d", |
771 | + **kwargs): |
772 | """Pause a system service. |
773 | |
774 | - Stop it, and prevent it from starting again at boot.""" |
775 | + Stop it, and prevent it from starting again at boot. |
776 | + |
777 | + :param service_name: the name of the service to pause |
778 | + :param init_dir: path to the upstart init directory |
779 | + :param initd_dir: path to the sysv init directory |
780 | + :param **kwargs: additional parameters to pass to the init system when |
781 | + managing services. These will be passed as key=value |
782 | + parameters to the init system's commandline. kwargs |
783 | + are ignored for init systems which do not support |
784 | + key=value arguments via the commandline. |
785 | + """ |
786 | stopped = True |
787 | - if service_running(service_name): |
788 | - stopped = service_stop(service_name) |
789 | + if service_running(service_name, **kwargs): |
790 | + stopped = service_stop(service_name, **kwargs) |
791 | upstart_file = os.path.join(init_dir, "{}.conf".format(service_name)) |
792 | sysv_file = os.path.join(initd_dir, service_name) |
793 | if init_is_systemd(): |
794 | @@ -92,10 +206,19 @@ |
795 | |
796 | |
797 | def service_resume(service_name, init_dir="/etc/init", |
798 | - initd_dir="/etc/init.d"): |
799 | + initd_dir="/etc/init.d", **kwargs): |
800 | """Resume a system service. |
801 | |
802 | - Reenable starting again at boot. Start the service""" |
803 | + Reenable starting again at boot. Start the service. |
804 | + |
805 | + :param service_name: the name of the service to resume |
806 | + :param init_dir: the path to the init dir |
807 | + :param initd dir: the path to the initd dir |
808 | + :param **kwargs: additional parameters to pass to the init system when |
809 | + managing services. These will be passed as key=value |
810 | + parameters to the init system's commandline. kwargs |
811 | + are ignored for systemd enabled systems. |
812 | + """ |
813 | upstart_file = os.path.join(init_dir, "{}.conf".format(service_name)) |
814 | sysv_file = os.path.join(initd_dir, service_name) |
815 | if init_is_systemd(): |
816 | @@ -112,62 +235,70 @@ |
817 | "Unable to detect {0} as SystemD, Upstart {1} or" |
818 | " SysV {2}".format( |
819 | service_name, upstart_file, sysv_file)) |
820 | + started = service_running(service_name, **kwargs) |
821 | |
822 | - started = service_running(service_name) |
823 | if not started: |
824 | - started = service_start(service_name) |
825 | + started = service_start(service_name, **kwargs) |
826 | return started |
827 | |
828 | |
829 | -def service(action, service_name): |
830 | - """Control a system service""" |
831 | +def service(action, service_name, **kwargs): |
832 | + """Control a system service. |
833 | + |
834 | + :param action: the action to take on the service |
835 | + :param service_name: the name of the service to perform th action on |
836 | + :param **kwargs: additional params to be passed to the service command in |
837 | + the form of key=value. |
838 | + """ |
839 | if init_is_systemd(): |
840 | cmd = ['systemctl', action, service_name] |
841 | else: |
842 | cmd = ['service', service_name, action] |
843 | + for key, value in six.iteritems(kwargs): |
844 | + parameter = '%s=%s' % (key, value) |
845 | + cmd.append(parameter) |
846 | return subprocess.call(cmd) == 0 |
847 | |
848 | |
849 | -def systemv_services_running(): |
850 | - output = subprocess.check_output( |
851 | - ['service', '--status-all'], |
852 | - stderr=subprocess.STDOUT).decode('UTF-8') |
853 | - return [row.split()[-1] for row in output.split('\n') if '[ + ]' in row] |
854 | - |
855 | - |
856 | -def service_running(service_name): |
857 | - """Determine whether a system service is running""" |
858 | +_UPSTART_CONF = "/etc/init/{}.conf" |
859 | +_INIT_D_CONF = "/etc/init.d/{}" |
860 | + |
861 | + |
862 | +def service_running(service_name, **kwargs): |
863 | + """Determine whether a system service is running. |
864 | + |
865 | + :param service_name: the name of the service |
866 | + :param **kwargs: additional args to pass to the service command. This is |
867 | + used to pass additional key=value arguments to the |
868 | + service command line for managing specific instance |
869 | + units (e.g. service ceph-osd status id=2). The kwargs |
870 | + are ignored in systemd services. |
871 | + """ |
872 | if init_is_systemd(): |
873 | return service('is-active', service_name) |
874 | else: |
875 | - try: |
876 | - output = subprocess.check_output( |
877 | - ['service', service_name, 'status'], |
878 | - stderr=subprocess.STDOUT).decode('UTF-8') |
879 | - except subprocess.CalledProcessError: |
880 | - return False |
881 | - else: |
882 | - # This works for upstart scripts where the 'service' command |
883 | - # returns a consistent string to represent running 'start/running' |
884 | - if ("start/running" in output or "is running" in output or |
885 | - "up and running" in output): |
886 | - return True |
887 | + if os.path.exists(_UPSTART_CONF.format(service_name)): |
888 | + try: |
889 | + cmd = ['status', service_name] |
890 | + for key, value in six.iteritems(kwargs): |
891 | + parameter = '%s=%s' % (key, value) |
892 | + cmd.append(parameter) |
893 | + output = subprocess.check_output(cmd, |
894 | + stderr=subprocess.STDOUT).decode('UTF-8') |
895 | + except subprocess.CalledProcessError: |
896 | + return False |
897 | + else: |
898 | + # This works for upstart scripts where the 'service' command |
899 | + # returns a consistent string to represent running |
900 | + # 'start/running' |
901 | + if ("start/running" in output or |
902 | + "is running" in output or |
903 | + "up and running" in output): |
904 | + return True |
905 | + elif os.path.exists(_INIT_D_CONF.format(service_name)): |
906 | # Check System V scripts init script return codes |
907 | - if service_name in systemv_services_running(): |
908 | - return True |
909 | - return False |
910 | - |
911 | - |
912 | -def service_available(service_name): |
913 | - """Determine whether a system service is available""" |
914 | - try: |
915 | - subprocess.check_output( |
916 | - ['service', service_name, 'status'], |
917 | - stderr=subprocess.STDOUT).decode('UTF-8') |
918 | - except subprocess.CalledProcessError as e: |
919 | - return b'unrecognized service' not in e.output |
920 | - else: |
921 | - return True |
922 | + return service('status', service_name) |
923 | + return False |
924 | |
925 | |
926 | SYSTEMD_SYSTEM = '/run/systemd/system' |
927 | @@ -178,8 +309,9 @@ |
928 | return os.path.isdir(SYSTEMD_SYSTEM) |
929 | |
930 | |
931 | -def adduser(username, password=None, shell='/bin/bash', system_user=False, |
932 | - primary_group=None, secondary_groups=None): |
933 | +def adduser(username, password=None, shell='/bin/bash', |
934 | + system_user=False, primary_group=None, |
935 | + secondary_groups=None, uid=None, home_dir=None): |
936 | """Add a user to the system. |
937 | |
938 | Will log but otherwise succeed if the user already exists. |
939 | @@ -190,15 +322,24 @@ |
940 | :param bool system_user: Whether to create a login or system user |
941 | :param str primary_group: Primary group for user; defaults to username |
942 | :param list secondary_groups: Optional list of additional groups |
943 | + :param int uid: UID for user being created |
944 | + :param str home_dir: Home directory for user |
945 | |
946 | :returns: The password database entry struct, as returned by `pwd.getpwnam` |
947 | """ |
948 | try: |
949 | user_info = pwd.getpwnam(username) |
950 | log('user {0} already exists!'.format(username)) |
951 | + if uid: |
952 | + user_info = pwd.getpwuid(int(uid)) |
953 | + log('user with uid {0} already exists!'.format(uid)) |
954 | except KeyError: |
955 | log('creating user {0}'.format(username)) |
956 | cmd = ['useradd'] |
957 | + if uid: |
958 | + cmd.extend(['--uid', str(uid)]) |
959 | + if home_dir: |
960 | + cmd.extend(['--home', str(home_dir)]) |
961 | if system_user or password is None: |
962 | cmd.append('--system') |
963 | else: |
964 | @@ -233,22 +374,56 @@ |
965 | return user_exists |
966 | |
967 | |
968 | -def add_group(group_name, system_group=False): |
969 | - """Add a group to the system""" |
970 | +def uid_exists(uid): |
971 | + """Check if a uid exists""" |
972 | + try: |
973 | + pwd.getpwuid(uid) |
974 | + uid_exists = True |
975 | + except KeyError: |
976 | + uid_exists = False |
977 | + return uid_exists |
978 | + |
979 | + |
980 | +def group_exists(groupname): |
981 | + """Check if a group exists""" |
982 | + try: |
983 | + grp.getgrnam(groupname) |
984 | + group_exists = True |
985 | + except KeyError: |
986 | + group_exists = False |
987 | + return group_exists |
988 | + |
989 | + |
990 | +def gid_exists(gid): |
991 | + """Check if a gid exists""" |
992 | + try: |
993 | + grp.getgrgid(gid) |
994 | + gid_exists = True |
995 | + except KeyError: |
996 | + gid_exists = False |
997 | + return gid_exists |
998 | + |
999 | + |
1000 | +def add_group(group_name, system_group=False, gid=None): |
1001 | + """Add a group to the system |
1002 | + |
1003 | + Will log but otherwise succeed if the group already exists. |
1004 | + |
1005 | + :param str group_name: group to create |
1006 | + :param bool system_group: Create system group |
1007 | + :param int gid: GID for user being created |
1008 | + |
1009 | + :returns: The password database entry struct, as returned by `grp.getgrnam` |
1010 | + """ |
1011 | try: |
1012 | group_info = grp.getgrnam(group_name) |
1013 | log('group {0} already exists!'.format(group_name)) |
1014 | + if gid: |
1015 | + group_info = grp.getgrgid(gid) |
1016 | + log('group with gid {0} already exists!'.format(gid)) |
1017 | except KeyError: |
1018 | log('creating group {0}'.format(group_name)) |
1019 | - cmd = ['addgroup'] |
1020 | - if system_group: |
1021 | - cmd.append('--system') |
1022 | - else: |
1023 | - cmd.extend([ |
1024 | - '--group', |
1025 | - ]) |
1026 | - cmd.append(group_name) |
1027 | - subprocess.check_call(cmd) |
1028 | + add_new_group(group_name, system_group, gid) |
1029 | group_info = grp.getgrnam(group_name) |
1030 | return group_info |
1031 | |
1032 | @@ -260,15 +435,17 @@ |
1033 | subprocess.check_call(cmd) |
1034 | |
1035 | |
1036 | -def rsync(from_path, to_path, flags='-r', options=None): |
1037 | +def rsync(from_path, to_path, flags='-r', options=None, timeout=None): |
1038 | """Replicate the contents of a path""" |
1039 | options = options or ['--delete', '--executability'] |
1040 | cmd = ['/usr/bin/rsync', flags] |
1041 | + if timeout: |
1042 | + cmd = ['timeout', str(timeout)] + cmd |
1043 | cmd.extend(options) |
1044 | cmd.append(from_path) |
1045 | cmd.append(to_path) |
1046 | log(" ".join(cmd)) |
1047 | - return subprocess.check_output(cmd).decode('UTF-8').strip() |
1048 | + return subprocess.check_output(cmd, stderr=subprocess.STDOUT).decode('UTF-8').strip() |
1049 | |
1050 | |
1051 | def symlink(source, destination): |
1052 | @@ -493,16 +670,6 @@ |
1053 | return r |
1054 | |
1055 | |
1056 | -def lsb_release(): |
1057 | - """Return /etc/lsb-release in a dict""" |
1058 | - d = {} |
1059 | - with open('/etc/lsb-release', 'r') as lsb: |
1060 | - for l in lsb: |
1061 | - k, v = l.split('=') |
1062 | - d[k.strip()] = v.strip() |
1063 | - return d |
1064 | - |
1065 | - |
1066 | def pwgen(length=None): |
1067 | """Generate a random pasword.""" |
1068 | if length is None: |
1069 | @@ -626,25 +793,6 @@ |
1070 | return hwaddr |
1071 | |
1072 | |
1073 | -def cmp_pkgrevno(package, revno, pkgcache=None): |
1074 | - """Compare supplied revno with the revno of the installed package |
1075 | - |
1076 | - * 1 => Installed revno is greater than supplied arg |
1077 | - * 0 => Installed revno is the same as supplied arg |
1078 | - * -1 => Installed revno is less than supplied arg |
1079 | - |
1080 | - This function imports apt_cache function from charmhelpers.fetch if |
1081 | - the pkgcache argument is None. Be sure to add charmhelpers.fetch if |
1082 | - you call this function, or pass an apt_pkg.Cache() instance. |
1083 | - """ |
1084 | - import apt_pkg |
1085 | - if not pkgcache: |
1086 | - from charmhelpers.fetch import apt_cache |
1087 | - pkgcache = apt_cache() |
1088 | - pkg = pkgcache[package] |
1089 | - return apt_pkg.version_compare(pkg.current_ver.ver_str, revno) |
1090 | - |
1091 | - |
1092 | @contextmanager |
1093 | def chdir(directory): |
1094 | """Change the current working directory to a different directory for a code |
1095 | @@ -667,7 +815,7 @@ |
1096 | :param str path: The string path to start changing ownership. |
1097 | :param str owner: The owner string to use when looking up the uid. |
1098 | :param str group: The group string to use when looking up the gid. |
1099 | - :param bool follow_links: Also Chown links if True |
1100 | + :param bool follow_links: Also follow and chown links if True |
1101 | :param bool chowntopdir: Also chown path itself if True |
1102 | """ |
1103 | uid = pwd.getpwnam(owner).pw_uid |
1104 | @@ -681,7 +829,7 @@ |
1105 | broken_symlink = os.path.lexists(path) and not os.path.exists(path) |
1106 | if not broken_symlink: |
1107 | chown(path, uid, gid) |
1108 | - for root, dirs, files in os.walk(path): |
1109 | + for root, dirs, files in os.walk(path, followlinks=follow_links): |
1110 | for name in dirs + files: |
1111 | full = os.path.join(root, name) |
1112 | broken_symlink = os.path.lexists(full) and not os.path.exists(full) |
1113 | @@ -701,6 +849,20 @@ |
1114 | chownr(path, owner, group, follow_links=False) |
1115 | |
1116 | |
1117 | +def owner(path): |
1118 | + """Returns a tuple containing the username & groupname owning the path. |
1119 | + |
1120 | + :param str path: the string path to retrieve the ownership |
1121 | + :return tuple(str, str): A (username, groupname) tuple containing the |
1122 | + name of the user and group owning the path. |
1123 | + :raises OSError: if the specified path does not exist |
1124 | + """ |
1125 | + stat = os.stat(path) |
1126 | + username = pwd.getpwuid(stat.st_uid)[0] |
1127 | + groupname = grp.getgrgid(stat.st_gid)[0] |
1128 | + return username, groupname |
1129 | + |
1130 | + |
1131 | def get_total_ram(): |
1132 | """The total amount of system RAM in bytes. |
1133 | |
1134 | @@ -715,3 +877,42 @@ |
1135 | assert unit == 'kB', 'Unknown unit' |
1136 | return int(value) * 1024 # Classic, not KiB. |
1137 | raise NotImplementedError() |
1138 | + |
1139 | + |
1140 | +UPSTART_CONTAINER_TYPE = '/run/container_type' |
1141 | + |
1142 | + |
1143 | +def is_container(): |
1144 | + """Determine whether unit is running in a container |
1145 | + |
1146 | + @return: boolean indicating if unit is in a container |
1147 | + """ |
1148 | + if init_is_systemd(): |
1149 | + # Detect using systemd-detect-virt |
1150 | + return subprocess.call(['systemd-detect-virt', |
1151 | + '--container']) == 0 |
1152 | + else: |
1153 | + # Detect using upstart container file marker |
1154 | + return os.path.exists(UPSTART_CONTAINER_TYPE) |
1155 | + |
1156 | + |
1157 | +def add_to_updatedb_prunepath(path, updatedb_path=UPDATEDB_PATH): |
1158 | + with open(updatedb_path, 'r+') as f_id: |
1159 | + updatedb_text = f_id.read() |
1160 | + output = updatedb(updatedb_text, path) |
1161 | + f_id.seek(0) |
1162 | + f_id.write(output) |
1163 | + f_id.truncate() |
1164 | + |
1165 | + |
1166 | +def updatedb(updatedb_text, new_path): |
1167 | + lines = [line for line in updatedb_text.split("\n")] |
1168 | + for i, line in enumerate(lines): |
1169 | + if line.startswith("PRUNEPATHS="): |
1170 | + paths_line = line.split("=")[1].replace('"', '') |
1171 | + paths = paths_line.split(" ") |
1172 | + if new_path not in paths: |
1173 | + paths.append(new_path) |
1174 | + lines[i] = 'PRUNEPATHS="{}"'.format(' '.join(paths)) |
1175 | + output = "\n".join(lines) |
1176 | + return output |
1177 | |
1178 | === added directory 'charmhelpers/core/host_factory' |
1179 | === added file 'charmhelpers/core/host_factory/__init__.py' |
1180 | === added file 'charmhelpers/core/host_factory/centos.py' |
1181 | --- charmhelpers/core/host_factory/centos.py 1970-01-01 00:00:00 +0000 |
1182 | +++ charmhelpers/core/host_factory/centos.py 2017-03-04 02:50:20 +0000 |
1183 | @@ -0,0 +1,56 @@ |
1184 | +import subprocess |
1185 | +import yum |
1186 | +import os |
1187 | + |
1188 | + |
1189 | +def service_available(service_name): |
1190 | + # """Determine whether a system service is available.""" |
1191 | + if os.path.isdir('/run/systemd/system'): |
1192 | + cmd = ['systemctl', 'is-enabled', service_name] |
1193 | + else: |
1194 | + cmd = ['service', service_name, 'is-enabled'] |
1195 | + return subprocess.call(cmd) == 0 |
1196 | + |
1197 | + |
1198 | +def add_new_group(group_name, system_group=False, gid=None): |
1199 | + cmd = ['groupadd'] |
1200 | + if gid: |
1201 | + cmd.extend(['--gid', str(gid)]) |
1202 | + if system_group: |
1203 | + cmd.append('-r') |
1204 | + cmd.append(group_name) |
1205 | + subprocess.check_call(cmd) |
1206 | + |
1207 | + |
1208 | +def lsb_release(): |
1209 | + """Return /etc/os-release in a dict.""" |
1210 | + d = {} |
1211 | + with open('/etc/os-release', 'r') as lsb: |
1212 | + for l in lsb: |
1213 | + s = l.split('=') |
1214 | + if len(s) != 2: |
1215 | + continue |
1216 | + d[s[0].strip()] = s[1].strip() |
1217 | + return d |
1218 | + |
1219 | + |
1220 | +def cmp_pkgrevno(package, revno, pkgcache=None): |
1221 | + """Compare supplied revno with the revno of the installed package. |
1222 | + |
1223 | + * 1 => Installed revno is greater than supplied arg |
1224 | + * 0 => Installed revno is the same as supplied arg |
1225 | + * -1 => Installed revno is less than supplied arg |
1226 | + |
1227 | + This function imports YumBase function if the pkgcache argument |
1228 | + is None. |
1229 | + """ |
1230 | + if not pkgcache: |
1231 | + y = yum.YumBase() |
1232 | + packages = y.doPackageLists() |
1233 | + pkgcache = {i.Name: i.version for i in packages['installed']} |
1234 | + pkg = pkgcache[package] |
1235 | + if pkg > revno: |
1236 | + return 1 |
1237 | + if pkg < revno: |
1238 | + return -1 |
1239 | + return 0 |
1240 | |
1241 | === added file 'charmhelpers/core/host_factory/ubuntu.py' |
1242 | --- charmhelpers/core/host_factory/ubuntu.py 1970-01-01 00:00:00 +0000 |
1243 | +++ charmhelpers/core/host_factory/ubuntu.py 2017-03-04 02:50:20 +0000 |
1244 | @@ -0,0 +1,56 @@ |
1245 | +import subprocess |
1246 | + |
1247 | + |
1248 | +def service_available(service_name): |
1249 | + """Determine whether a system service is available""" |
1250 | + try: |
1251 | + subprocess.check_output( |
1252 | + ['service', service_name, 'status'], |
1253 | + stderr=subprocess.STDOUT).decode('UTF-8') |
1254 | + except subprocess.CalledProcessError as e: |
1255 | + return b'unrecognized service' not in e.output |
1256 | + else: |
1257 | + return True |
1258 | + |
1259 | + |
1260 | +def add_new_group(group_name, system_group=False, gid=None): |
1261 | + cmd = ['addgroup'] |
1262 | + if gid: |
1263 | + cmd.extend(['--gid', str(gid)]) |
1264 | + if system_group: |
1265 | + cmd.append('--system') |
1266 | + else: |
1267 | + cmd.extend([ |
1268 | + '--group', |
1269 | + ]) |
1270 | + cmd.append(group_name) |
1271 | + subprocess.check_call(cmd) |
1272 | + |
1273 | + |
1274 | +def lsb_release(): |
1275 | + """Return /etc/lsb-release in a dict""" |
1276 | + d = {} |
1277 | + with open('/etc/lsb-release', 'r') as lsb: |
1278 | + for l in lsb: |
1279 | + k, v = l.split('=') |
1280 | + d[k.strip()] = v.strip() |
1281 | + return d |
1282 | + |
1283 | + |
1284 | +def cmp_pkgrevno(package, revno, pkgcache=None): |
1285 | + """Compare supplied revno with the revno of the installed package. |
1286 | + |
1287 | + * 1 => Installed revno is greater than supplied arg |
1288 | + * 0 => Installed revno is the same as supplied arg |
1289 | + * -1 => Installed revno is less than supplied arg |
1290 | + |
1291 | + This function imports apt_cache function from charmhelpers.fetch if |
1292 | + the pkgcache argument is None. Be sure to add charmhelpers.fetch if |
1293 | + you call this function, or pass an apt_pkg.Cache() instance. |
1294 | + """ |
1295 | + import apt_pkg |
1296 | + if not pkgcache: |
1297 | + from charmhelpers.fetch import apt_cache |
1298 | + pkgcache = apt_cache() |
1299 | + pkg = pkgcache[package] |
1300 | + return apt_pkg.version_compare(pkg.current_ver.ver_str, revno) |
1301 | |
1302 | === modified file 'charmhelpers/core/hugepage.py' |
1303 | --- charmhelpers/core/hugepage.py 2015-12-11 15:23:38 +0000 |
1304 | +++ charmhelpers/core/hugepage.py 2017-03-04 02:50:20 +0000 |
1305 | @@ -2,19 +2,17 @@ |
1306 | |
1307 | # Copyright 2014-2015 Canonical Limited. |
1308 | # |
1309 | -# This file is part of charm-helpers. |
1310 | -# |
1311 | -# charm-helpers is free software: you can redistribute it and/or modify |
1312 | -# it under the terms of the GNU Lesser General Public License version 3 as |
1313 | -# published by the Free Software Foundation. |
1314 | -# |
1315 | -# charm-helpers is distributed in the hope that it will be useful, |
1316 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
1317 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
1318 | -# GNU Lesser General Public License for more details. |
1319 | -# |
1320 | -# You should have received a copy of the GNU Lesser General Public License |
1321 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
1322 | +# Licensed under the Apache License, Version 2.0 (the "License"); |
1323 | +# you may not use this file except in compliance with the License. |
1324 | +# You may obtain a copy of the License at |
1325 | +# |
1326 | +# http://www.apache.org/licenses/LICENSE-2.0 |
1327 | +# |
1328 | +# Unless required by applicable law or agreed to in writing, software |
1329 | +# distributed under the License is distributed on an "AS IS" BASIS, |
1330 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
1331 | +# See the License for the specific language governing permissions and |
1332 | +# limitations under the License. |
1333 | |
1334 | import yaml |
1335 | from charmhelpers.core import fstab |
1336 | |
1337 | === modified file 'charmhelpers/core/kernel.py' |
1338 | --- charmhelpers/core/kernel.py 2015-12-11 15:23:38 +0000 |
1339 | +++ charmhelpers/core/kernel.py 2017-03-04 02:50:20 +0000 |
1340 | @@ -3,29 +3,40 @@ |
1341 | |
1342 | # Copyright 2014-2015 Canonical Limited. |
1343 | # |
1344 | -# This file is part of charm-helpers. |
1345 | -# |
1346 | -# charm-helpers is free software: you can redistribute it and/or modify |
1347 | -# it under the terms of the GNU Lesser General Public License version 3 as |
1348 | -# published by the Free Software Foundation. |
1349 | -# |
1350 | -# charm-helpers is distributed in the hope that it will be useful, |
1351 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
1352 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
1353 | -# GNU Lesser General Public License for more details. |
1354 | -# |
1355 | -# You should have received a copy of the GNU Lesser General Public License |
1356 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
1357 | - |
1358 | -__author__ = "Jorge Niedbalski <jorge.niedbalski@canonical.com>" |
1359 | - |
1360 | +# Licensed under the Apache License, Version 2.0 (the "License"); |
1361 | +# you may not use this file except in compliance with the License. |
1362 | +# You may obtain a copy of the License at |
1363 | +# |
1364 | +# http://www.apache.org/licenses/LICENSE-2.0 |
1365 | +# |
1366 | +# Unless required by applicable law or agreed to in writing, software |
1367 | +# distributed under the License is distributed on an "AS IS" BASIS, |
1368 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
1369 | +# See the License for the specific language governing permissions and |
1370 | +# limitations under the License. |
1371 | + |
1372 | +import re |
1373 | +import subprocess |
1374 | + |
1375 | +from charmhelpers.osplatform import get_platform |
1376 | from charmhelpers.core.hookenv import ( |
1377 | log, |
1378 | INFO |
1379 | ) |
1380 | |
1381 | -from subprocess import check_call, check_output |
1382 | -import re |
1383 | +__platform__ = get_platform() |
1384 | +if __platform__ == "ubuntu": |
1385 | + from charmhelpers.core.kernel_factory.ubuntu import ( |
1386 | + persistent_modprobe, |
1387 | + update_initramfs, |
1388 | + ) # flake8: noqa -- ignore F401 for this import |
1389 | +elif __platform__ == "centos": |
1390 | + from charmhelpers.core.kernel_factory.centos import ( |
1391 | + persistent_modprobe, |
1392 | + update_initramfs, |
1393 | + ) # flake8: noqa -- ignore F401 for this import |
1394 | + |
1395 | +__author__ = "Jorge Niedbalski <jorge.niedbalski@canonical.com>" |
1396 | |
1397 | |
1398 | def modprobe(module, persist=True): |
1399 | @@ -34,11 +45,9 @@ |
1400 | |
1401 | log('Loading kernel module %s' % module, level=INFO) |
1402 | |
1403 | - check_call(cmd) |
1404 | + subprocess.check_call(cmd) |
1405 | if persist: |
1406 | - with open('/etc/modules', 'r+') as modules: |
1407 | - if module not in modules.read(): |
1408 | - modules.write(module) |
1409 | + persistent_modprobe(module) |
1410 | |
1411 | |
1412 | def rmmod(module, force=False): |
1413 | @@ -48,21 +57,16 @@ |
1414 | cmd.append('-f') |
1415 | cmd.append(module) |
1416 | log('Removing kernel module %s' % module, level=INFO) |
1417 | - return check_call(cmd) |
1418 | + return subprocess.check_call(cmd) |
1419 | |
1420 | |
1421 | def lsmod(): |
1422 | """Shows what kernel modules are currently loaded""" |
1423 | - return check_output(['lsmod'], |
1424 | - universal_newlines=True) |
1425 | + return subprocess.check_output(['lsmod'], |
1426 | + universal_newlines=True) |
1427 | |
1428 | |
1429 | def is_module_loaded(module): |
1430 | """Checks if a kernel module is already loaded""" |
1431 | matches = re.findall('^%s[ ]+' % module, lsmod(), re.M) |
1432 | return len(matches) > 0 |
1433 | - |
1434 | - |
1435 | -def update_initramfs(version='all'): |
1436 | - """Updates an initramfs image""" |
1437 | - return check_call(["update-initramfs", "-k", version, "-u"]) |
1438 | |
1439 | === added directory 'charmhelpers/core/kernel_factory' |
1440 | === added file 'charmhelpers/core/kernel_factory/__init__.py' |
1441 | === added file 'charmhelpers/core/kernel_factory/centos.py' |
1442 | --- charmhelpers/core/kernel_factory/centos.py 1970-01-01 00:00:00 +0000 |
1443 | +++ charmhelpers/core/kernel_factory/centos.py 2017-03-04 02:50:20 +0000 |
1444 | @@ -0,0 +1,17 @@ |
1445 | +import subprocess |
1446 | +import os |
1447 | + |
1448 | + |
1449 | +def persistent_modprobe(module): |
1450 | + """Load a kernel module and configure for auto-load on reboot.""" |
1451 | + if not os.path.exists('/etc/rc.modules'): |
1452 | + open('/etc/rc.modules', 'a') |
1453 | + os.chmod('/etc/rc.modules', 111) |
1454 | + with open('/etc/rc.modules', 'r+') as modules: |
1455 | + if module not in modules.read(): |
1456 | + modules.write('modprobe %s\n' % module) |
1457 | + |
1458 | + |
1459 | +def update_initramfs(version='all'): |
1460 | + """Updates an initramfs image.""" |
1461 | + return subprocess.check_call(["dracut", "-f", version]) |
1462 | |
1463 | === added file 'charmhelpers/core/kernel_factory/ubuntu.py' |
1464 | --- charmhelpers/core/kernel_factory/ubuntu.py 1970-01-01 00:00:00 +0000 |
1465 | +++ charmhelpers/core/kernel_factory/ubuntu.py 2017-03-04 02:50:20 +0000 |
1466 | @@ -0,0 +1,13 @@ |
1467 | +import subprocess |
1468 | + |
1469 | + |
1470 | +def persistent_modprobe(module): |
1471 | + """Load a kernel module and configure for auto-load on reboot.""" |
1472 | + with open('/etc/modules', 'r+') as modules: |
1473 | + if module not in modules.read(): |
1474 | + modules.write(module + "\n") |
1475 | + |
1476 | + |
1477 | +def update_initramfs(version='all'): |
1478 | + """Updates an initramfs image.""" |
1479 | + return subprocess.check_call(["update-initramfs", "-k", version, "-u"]) |
1480 | |
1481 | === modified file 'charmhelpers/core/services/__init__.py' |
1482 | --- charmhelpers/core/services/__init__.py 2015-01-28 08:59:02 +0000 |
1483 | +++ charmhelpers/core/services/__init__.py 2017-03-04 02:50:20 +0000 |
1484 | @@ -1,18 +1,16 @@ |
1485 | # Copyright 2014-2015 Canonical Limited. |
1486 | # |
1487 | -# This file is part of charm-helpers. |
1488 | -# |
1489 | -# charm-helpers is free software: you can redistribute it and/or modify |
1490 | -# it under the terms of the GNU Lesser General Public License version 3 as |
1491 | -# published by the Free Software Foundation. |
1492 | -# |
1493 | -# charm-helpers is distributed in the hope that it will be useful, |
1494 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
1495 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
1496 | -# GNU Lesser General Public License for more details. |
1497 | -# |
1498 | -# You should have received a copy of the GNU Lesser General Public License |
1499 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
1500 | +# Licensed under the Apache License, Version 2.0 (the "License"); |
1501 | +# you may not use this file except in compliance with the License. |
1502 | +# You may obtain a copy of the License at |
1503 | +# |
1504 | +# http://www.apache.org/licenses/LICENSE-2.0 |
1505 | +# |
1506 | +# Unless required by applicable law or agreed to in writing, software |
1507 | +# distributed under the License is distributed on an "AS IS" BASIS, |
1508 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
1509 | +# See the License for the specific language governing permissions and |
1510 | +# limitations under the License. |
1511 | |
1512 | from .base import * # NOQA |
1513 | from .helpers import * # NOQA |
1514 | |
1515 | === modified file 'charmhelpers/core/services/base.py' |
1516 | --- charmhelpers/core/services/base.py 2015-07-03 09:13:26 +0000 |
1517 | +++ charmhelpers/core/services/base.py 2017-03-04 02:50:20 +0000 |
1518 | @@ -1,18 +1,16 @@ |
1519 | # Copyright 2014-2015 Canonical Limited. |
1520 | # |
1521 | -# This file is part of charm-helpers. |
1522 | -# |
1523 | -# charm-helpers is free software: you can redistribute it and/or modify |
1524 | -# it under the terms of the GNU Lesser General Public License version 3 as |
1525 | -# published by the Free Software Foundation. |
1526 | -# |
1527 | -# charm-helpers is distributed in the hope that it will be useful, |
1528 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
1529 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
1530 | -# GNU Lesser General Public License for more details. |
1531 | -# |
1532 | -# You should have received a copy of the GNU Lesser General Public License |
1533 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
1534 | +# Licensed under the Apache License, Version 2.0 (the "License"); |
1535 | +# you may not use this file except in compliance with the License. |
1536 | +# You may obtain a copy of the License at |
1537 | +# |
1538 | +# http://www.apache.org/licenses/LICENSE-2.0 |
1539 | +# |
1540 | +# Unless required by applicable law or agreed to in writing, software |
1541 | +# distributed under the License is distributed on an "AS IS" BASIS, |
1542 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
1543 | +# See the License for the specific language governing permissions and |
1544 | +# limitations under the License. |
1545 | |
1546 | import os |
1547 | import json |
1548 | |
1549 | === modified file 'charmhelpers/core/services/helpers.py' |
1550 | --- charmhelpers/core/services/helpers.py 2015-12-11 15:23:38 +0000 |
1551 | +++ charmhelpers/core/services/helpers.py 2017-03-04 02:50:20 +0000 |
1552 | @@ -1,18 +1,16 @@ |
1553 | # Copyright 2014-2015 Canonical Limited. |
1554 | # |
1555 | -# This file is part of charm-helpers. |
1556 | -# |
1557 | -# charm-helpers is free software: you can redistribute it and/or modify |
1558 | -# it under the terms of the GNU Lesser General Public License version 3 as |
1559 | -# published by the Free Software Foundation. |
1560 | -# |
1561 | -# charm-helpers is distributed in the hope that it will be useful, |
1562 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
1563 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
1564 | -# GNU Lesser General Public License for more details. |
1565 | -# |
1566 | -# You should have received a copy of the GNU Lesser General Public License |
1567 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
1568 | +# Licensed under the Apache License, Version 2.0 (the "License"); |
1569 | +# you may not use this file except in compliance with the License. |
1570 | +# You may obtain a copy of the License at |
1571 | +# |
1572 | +# http://www.apache.org/licenses/LICENSE-2.0 |
1573 | +# |
1574 | +# Unless required by applicable law or agreed to in writing, software |
1575 | +# distributed under the License is distributed on an "AS IS" BASIS, |
1576 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
1577 | +# See the License for the specific language governing permissions and |
1578 | +# limitations under the License. |
1579 | |
1580 | import os |
1581 | import yaml |
1582 | |
1583 | === modified file 'charmhelpers/core/strutils.py' |
1584 | --- charmhelpers/core/strutils.py 2015-12-11 15:23:38 +0000 |
1585 | +++ charmhelpers/core/strutils.py 2017-03-04 02:50:20 +0000 |
1586 | @@ -3,19 +3,17 @@ |
1587 | |
1588 | # Copyright 2014-2015 Canonical Limited. |
1589 | # |
1590 | -# This file is part of charm-helpers. |
1591 | -# |
1592 | -# charm-helpers is free software: you can redistribute it and/or modify |
1593 | -# it under the terms of the GNU Lesser General Public License version 3 as |
1594 | -# published by the Free Software Foundation. |
1595 | -# |
1596 | -# charm-helpers is distributed in the hope that it will be useful, |
1597 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
1598 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
1599 | -# GNU Lesser General Public License for more details. |
1600 | -# |
1601 | -# You should have received a copy of the GNU Lesser General Public License |
1602 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
1603 | +# Licensed under the Apache License, Version 2.0 (the "License"); |
1604 | +# you may not use this file except in compliance with the License. |
1605 | +# You may obtain a copy of the License at |
1606 | +# |
1607 | +# http://www.apache.org/licenses/LICENSE-2.0 |
1608 | +# |
1609 | +# Unless required by applicable law or agreed to in writing, software |
1610 | +# distributed under the License is distributed on an "AS IS" BASIS, |
1611 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
1612 | +# See the License for the specific language governing permissions and |
1613 | +# limitations under the License. |
1614 | |
1615 | import six |
1616 | import re |
1617 | |
1618 | === modified file 'charmhelpers/core/sysctl.py' |
1619 | --- charmhelpers/core/sysctl.py 2015-03-12 11:42:26 +0000 |
1620 | +++ charmhelpers/core/sysctl.py 2017-03-04 02:50:20 +0000 |
1621 | @@ -3,19 +3,17 @@ |
1622 | |
1623 | # Copyright 2014-2015 Canonical Limited. |
1624 | # |
1625 | -# This file is part of charm-helpers. |
1626 | -# |
1627 | -# charm-helpers is free software: you can redistribute it and/or modify |
1628 | -# it under the terms of the GNU Lesser General Public License version 3 as |
1629 | -# published by the Free Software Foundation. |
1630 | -# |
1631 | -# charm-helpers is distributed in the hope that it will be useful, |
1632 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
1633 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
1634 | -# GNU Lesser General Public License for more details. |
1635 | -# |
1636 | -# You should have received a copy of the GNU Lesser General Public License |
1637 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
1638 | +# Licensed under the Apache License, Version 2.0 (the "License"); |
1639 | +# you may not use this file except in compliance with the License. |
1640 | +# You may obtain a copy of the License at |
1641 | +# |
1642 | +# http://www.apache.org/licenses/LICENSE-2.0 |
1643 | +# |
1644 | +# Unless required by applicable law or agreed to in writing, software |
1645 | +# distributed under the License is distributed on an "AS IS" BASIS, |
1646 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
1647 | +# See the License for the specific language governing permissions and |
1648 | +# limitations under the License. |
1649 | |
1650 | import yaml |
1651 | |
1652 | |
1653 | === modified file 'charmhelpers/core/templating.py' |
1654 | --- charmhelpers/core/templating.py 2015-12-11 15:23:38 +0000 |
1655 | +++ charmhelpers/core/templating.py 2017-03-04 02:50:20 +0000 |
1656 | @@ -1,20 +1,19 @@ |
1657 | # Copyright 2014-2015 Canonical Limited. |
1658 | # |
1659 | -# This file is part of charm-helpers. |
1660 | -# |
1661 | -# charm-helpers is free software: you can redistribute it and/or modify |
1662 | -# it under the terms of the GNU Lesser General Public License version 3 as |
1663 | -# published by the Free Software Foundation. |
1664 | -# |
1665 | -# charm-helpers is distributed in the hope that it will be useful, |
1666 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
1667 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
1668 | -# GNU Lesser General Public License for more details. |
1669 | -# |
1670 | -# You should have received a copy of the GNU Lesser General Public License |
1671 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
1672 | +# Licensed under the Apache License, Version 2.0 (the "License"); |
1673 | +# you may not use this file except in compliance with the License. |
1674 | +# You may obtain a copy of the License at |
1675 | +# |
1676 | +# http://www.apache.org/licenses/LICENSE-2.0 |
1677 | +# |
1678 | +# Unless required by applicable law or agreed to in writing, software |
1679 | +# distributed under the License is distributed on an "AS IS" BASIS, |
1680 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
1681 | +# See the License for the specific language governing permissions and |
1682 | +# limitations under the License. |
1683 | |
1684 | import os |
1685 | +import sys |
1686 | |
1687 | from charmhelpers.core import host |
1688 | from charmhelpers.core import hookenv |
1689 | @@ -40,8 +39,9 @@ |
1690 | The rendered template will be written to the file as well as being returned |
1691 | as a string. |
1692 | |
1693 | - Note: Using this requires python-jinja2; if it is not installed, calling |
1694 | - this will attempt to use charmhelpers.fetch.apt_install to install it. |
1695 | + Note: Using this requires python-jinja2 or python3-jinja2; if it is not |
1696 | + installed, calling this will attempt to use charmhelpers.fetch.apt_install |
1697 | + to install it. |
1698 | """ |
1699 | try: |
1700 | from jinja2 import FileSystemLoader, Environment, exceptions |
1701 | @@ -53,7 +53,10 @@ |
1702 | 'charmhelpers.fetch to install it', |
1703 | level=hookenv.ERROR) |
1704 | raise |
1705 | - apt_install('python-jinja2', fatal=True) |
1706 | + if sys.version_info.major == 2: |
1707 | + apt_install('python-jinja2', fatal=True) |
1708 | + else: |
1709 | + apt_install('python3-jinja2', fatal=True) |
1710 | from jinja2 import FileSystemLoader, Environment, exceptions |
1711 | |
1712 | if template_loader: |
1713 | |
1714 | === modified file 'charmhelpers/core/unitdata.py' |
1715 | --- charmhelpers/core/unitdata.py 2015-12-11 15:23:38 +0000 |
1716 | +++ charmhelpers/core/unitdata.py 2017-03-04 02:50:20 +0000 |
1717 | @@ -3,20 +3,17 @@ |
1718 | # |
1719 | # Copyright 2014-2015 Canonical Limited. |
1720 | # |
1721 | -# This file is part of charm-helpers. |
1722 | -# |
1723 | -# charm-helpers is free software: you can redistribute it and/or modify |
1724 | -# it under the terms of the GNU Lesser General Public License version 3 as |
1725 | -# published by the Free Software Foundation. |
1726 | -# |
1727 | -# charm-helpers is distributed in the hope that it will be useful, |
1728 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
1729 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
1730 | -# GNU Lesser General Public License for more details. |
1731 | -# |
1732 | -# You should have received a copy of the GNU Lesser General Public License |
1733 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
1734 | -# |
1735 | +# Licensed under the Apache License, Version 2.0 (the "License"); |
1736 | +# you may not use this file except in compliance with the License. |
1737 | +# You may obtain a copy of the License at |
1738 | +# |
1739 | +# http://www.apache.org/licenses/LICENSE-2.0 |
1740 | +# |
1741 | +# Unless required by applicable law or agreed to in writing, software |
1742 | +# distributed under the License is distributed on an "AS IS" BASIS, |
1743 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
1744 | +# See the License for the specific language governing permissions and |
1745 | +# limitations under the License. |
1746 | # |
1747 | # Authors: |
1748 | # Kapil Thangavelu <kapil.foss@gmail.com> |
1749 | |
1750 | === modified file 'charmhelpers/fetch/__init__.py' |
1751 | --- charmhelpers/fetch/__init__.py 2016-05-06 07:12:40 +0000 |
1752 | +++ charmhelpers/fetch/__init__.py 2017-03-04 02:50:20 +0000 |
1753 | @@ -1,32 +1,24 @@ |
1754 | # Copyright 2014-2015 Canonical Limited. |
1755 | # |
1756 | -# This file is part of charm-helpers. |
1757 | -# |
1758 | -# charm-helpers is free software: you can redistribute it and/or modify |
1759 | -# it under the terms of the GNU Lesser General Public License version 3 as |
1760 | -# published by the Free Software Foundation. |
1761 | -# |
1762 | -# charm-helpers is distributed in the hope that it will be useful, |
1763 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
1764 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
1765 | -# GNU Lesser General Public License for more details. |
1766 | -# |
1767 | -# You should have received a copy of the GNU Lesser General Public License |
1768 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
1769 | +# Licensed under the Apache License, Version 2.0 (the "License"); |
1770 | +# you may not use this file except in compliance with the License. |
1771 | +# You may obtain a copy of the License at |
1772 | +# |
1773 | +# http://www.apache.org/licenses/LICENSE-2.0 |
1774 | +# |
1775 | +# Unless required by applicable law or agreed to in writing, software |
1776 | +# distributed under the License is distributed on an "AS IS" BASIS, |
1777 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
1778 | +# See the License for the specific language governing permissions and |
1779 | +# limitations under the License. |
1780 | |
1781 | import importlib |
1782 | -from tempfile import NamedTemporaryFile |
1783 | -import time |
1784 | +from charmhelpers.osplatform import get_platform |
1785 | from yaml import safe_load |
1786 | -from charmhelpers.core.host import ( |
1787 | - lsb_release |
1788 | -) |
1789 | -import subprocess |
1790 | from charmhelpers.core.hookenv import ( |
1791 | config, |
1792 | log, |
1793 | ) |
1794 | -import os |
1795 | |
1796 | import six |
1797 | if six.PY3: |
1798 | @@ -35,79 +27,6 @@ |
1799 | from urlparse import urlparse, urlunparse |
1800 | |
1801 | |
1802 | -CLOUD_ARCHIVE = """# Ubuntu Cloud Archive |
1803 | -deb http://ubuntu-cloud.archive.canonical.com/ubuntu {} main |
1804 | -""" |
1805 | -PROPOSED_POCKET = """# Proposed |
1806 | -deb http://archive.ubuntu.com/ubuntu {}-proposed main universe multiverse restricted |
1807 | -""" |
1808 | -CLOUD_ARCHIVE_POCKETS = { |
1809 | - # Folsom |
1810 | - 'folsom': 'precise-updates/folsom', |
1811 | - 'precise-folsom': 'precise-updates/folsom', |
1812 | - 'precise-folsom/updates': 'precise-updates/folsom', |
1813 | - 'precise-updates/folsom': 'precise-updates/folsom', |
1814 | - 'folsom/proposed': 'precise-proposed/folsom', |
1815 | - 'precise-folsom/proposed': 'precise-proposed/folsom', |
1816 | - 'precise-proposed/folsom': 'precise-proposed/folsom', |
1817 | - # Grizzly |
1818 | - 'grizzly': 'precise-updates/grizzly', |
1819 | - 'precise-grizzly': 'precise-updates/grizzly', |
1820 | - 'precise-grizzly/updates': 'precise-updates/grizzly', |
1821 | - 'precise-updates/grizzly': 'precise-updates/grizzly', |
1822 | - 'grizzly/proposed': 'precise-proposed/grizzly', |
1823 | - 'precise-grizzly/proposed': 'precise-proposed/grizzly', |
1824 | - 'precise-proposed/grizzly': 'precise-proposed/grizzly', |
1825 | - # Havana |
1826 | - 'havana': 'precise-updates/havana', |
1827 | - 'precise-havana': 'precise-updates/havana', |
1828 | - 'precise-havana/updates': 'precise-updates/havana', |
1829 | - 'precise-updates/havana': 'precise-updates/havana', |
1830 | - 'havana/proposed': 'precise-proposed/havana', |
1831 | - 'precise-havana/proposed': 'precise-proposed/havana', |
1832 | - 'precise-proposed/havana': 'precise-proposed/havana', |
1833 | - # Icehouse |
1834 | - 'icehouse': 'precise-updates/icehouse', |
1835 | - 'precise-icehouse': 'precise-updates/icehouse', |
1836 | - 'precise-icehouse/updates': 'precise-updates/icehouse', |
1837 | - 'precise-updates/icehouse': 'precise-updates/icehouse', |
1838 | - 'icehouse/proposed': 'precise-proposed/icehouse', |
1839 | - 'precise-icehouse/proposed': 'precise-proposed/icehouse', |
1840 | - 'precise-proposed/icehouse': 'precise-proposed/icehouse', |
1841 | - # Juno |
1842 | - 'juno': 'trusty-updates/juno', |
1843 | - 'trusty-juno': 'trusty-updates/juno', |
1844 | - 'trusty-juno/updates': 'trusty-updates/juno', |
1845 | - 'trusty-updates/juno': 'trusty-updates/juno', |
1846 | - 'juno/proposed': 'trusty-proposed/juno', |
1847 | - 'trusty-juno/proposed': 'trusty-proposed/juno', |
1848 | - 'trusty-proposed/juno': 'trusty-proposed/juno', |
1849 | - # Kilo |
1850 | - 'kilo': 'trusty-updates/kilo', |
1851 | - 'trusty-kilo': 'trusty-updates/kilo', |
1852 | - 'trusty-kilo/updates': 'trusty-updates/kilo', |
1853 | - 'trusty-updates/kilo': 'trusty-updates/kilo', |
1854 | - 'kilo/proposed': 'trusty-proposed/kilo', |
1855 | - 'trusty-kilo/proposed': 'trusty-proposed/kilo', |
1856 | - 'trusty-proposed/kilo': 'trusty-proposed/kilo', |
1857 | - # Liberty |
1858 | - 'liberty': 'trusty-updates/liberty', |
1859 | - 'trusty-liberty': 'trusty-updates/liberty', |
1860 | - 'trusty-liberty/updates': 'trusty-updates/liberty', |
1861 | - 'trusty-updates/liberty': 'trusty-updates/liberty', |
1862 | - 'liberty/proposed': 'trusty-proposed/liberty', |
1863 | - 'trusty-liberty/proposed': 'trusty-proposed/liberty', |
1864 | - 'trusty-proposed/liberty': 'trusty-proposed/liberty', |
1865 | - # Mitaka |
1866 | - 'mitaka': 'trusty-updates/mitaka', |
1867 | - 'trusty-mitaka': 'trusty-updates/mitaka', |
1868 | - 'trusty-mitaka/updates': 'trusty-updates/mitaka', |
1869 | - 'trusty-updates/mitaka': 'trusty-updates/mitaka', |
1870 | - 'mitaka/proposed': 'trusty-proposed/mitaka', |
1871 | - 'trusty-mitaka/proposed': 'trusty-proposed/mitaka', |
1872 | - 'trusty-proposed/mitaka': 'trusty-proposed/mitaka', |
1873 | -} |
1874 | - |
1875 | # The order of this list is very important. Handlers should be listed in from |
1876 | # least- to most-specific URL matching. |
1877 | FETCH_HANDLERS = ( |
1878 | @@ -116,10 +35,6 @@ |
1879 | 'charmhelpers.fetch.giturl.GitUrlFetchHandler', |
1880 | ) |
1881 | |
1882 | -APT_NO_LOCK = 100 # The return code for "couldn't acquire lock" in APT. |
1883 | -APT_NO_LOCK_RETRY_DELAY = 10 # Wait 10 seconds between apt lock checks. |
1884 | -APT_NO_LOCK_RETRY_COUNT = 30 # Retry to acquire the lock X times. |
1885 | - |
1886 | |
1887 | class SourceConfigError(Exception): |
1888 | pass |
1889 | @@ -157,180 +72,38 @@ |
1890 | return urlunparse(parts) |
1891 | |
1892 | |
1893 | -def filter_installed_packages(packages): |
1894 | - """Returns a list of packages that require installation""" |
1895 | - cache = apt_cache() |
1896 | - _pkgs = [] |
1897 | - for package in packages: |
1898 | - try: |
1899 | - p = cache[package] |
1900 | - p.current_ver or _pkgs.append(package) |
1901 | - except KeyError: |
1902 | - log('Package {} has no installation candidate.'.format(package), |
1903 | - level='WARNING') |
1904 | - _pkgs.append(package) |
1905 | - return _pkgs |
1906 | - |
1907 | - |
1908 | -def apt_cache(in_memory=True): |
1909 | - """Build and return an apt cache""" |
1910 | - from apt import apt_pkg |
1911 | - apt_pkg.init() |
1912 | - if in_memory: |
1913 | - apt_pkg.config.set("Dir::Cache::pkgcache", "") |
1914 | - apt_pkg.config.set("Dir::Cache::srcpkgcache", "") |
1915 | - return apt_pkg.Cache() |
1916 | - |
1917 | - |
1918 | -def apt_install(packages, options=None, fatal=False): |
1919 | - """Install one or more packages""" |
1920 | - if options is None: |
1921 | - options = ['--option=Dpkg::Options::=--force-confold'] |
1922 | - |
1923 | - cmd = ['apt-get', '--assume-yes'] |
1924 | - cmd.extend(options) |
1925 | - cmd.append('install') |
1926 | - if isinstance(packages, six.string_types): |
1927 | - cmd.append(packages) |
1928 | - else: |
1929 | - cmd.extend(packages) |
1930 | - log("Installing {} with options: {}".format(packages, |
1931 | - options)) |
1932 | - _run_apt_command(cmd, fatal) |
1933 | - |
1934 | - |
1935 | -def apt_upgrade(options=None, fatal=False, dist=False): |
1936 | - """Upgrade all packages""" |
1937 | - if options is None: |
1938 | - options = ['--option=Dpkg::Options::=--force-confold'] |
1939 | - |
1940 | - cmd = ['apt-get', '--assume-yes'] |
1941 | - cmd.extend(options) |
1942 | - if dist: |
1943 | - cmd.append('dist-upgrade') |
1944 | - else: |
1945 | - cmd.append('upgrade') |
1946 | - log("Upgrading with options: {}".format(options)) |
1947 | - _run_apt_command(cmd, fatal) |
1948 | - |
1949 | - |
1950 | -def apt_update(fatal=False): |
1951 | - """Update local apt cache""" |
1952 | - cmd = ['apt-get', 'update'] |
1953 | - _run_apt_command(cmd, fatal) |
1954 | - |
1955 | - |
1956 | -def apt_purge(packages, fatal=False): |
1957 | - """Purge one or more packages""" |
1958 | - cmd = ['apt-get', '--assume-yes', 'purge'] |
1959 | - if isinstance(packages, six.string_types): |
1960 | - cmd.append(packages) |
1961 | - else: |
1962 | - cmd.extend(packages) |
1963 | - log("Purging {}".format(packages)) |
1964 | - _run_apt_command(cmd, fatal) |
1965 | - |
1966 | - |
1967 | -def apt_mark(packages, mark, fatal=False): |
1968 | - """Flag one or more packages using apt-mark""" |
1969 | - log("Marking {} as {}".format(packages, mark)) |
1970 | - cmd = ['apt-mark', mark] |
1971 | - if isinstance(packages, six.string_types): |
1972 | - cmd.append(packages) |
1973 | - else: |
1974 | - cmd.extend(packages) |
1975 | - |
1976 | - if fatal: |
1977 | - subprocess.check_call(cmd, universal_newlines=True) |
1978 | - else: |
1979 | - subprocess.call(cmd, universal_newlines=True) |
1980 | - |
1981 | - |
1982 | -def apt_hold(packages, fatal=False): |
1983 | - return apt_mark(packages, 'hold', fatal=fatal) |
1984 | - |
1985 | - |
1986 | -def apt_unhold(packages, fatal=False): |
1987 | - return apt_mark(packages, 'unhold', fatal=fatal) |
1988 | - |
1989 | - |
1990 | -def add_source(source, key=None): |
1991 | - """Add a package source to this system. |
1992 | - |
1993 | - @param source: a URL or sources.list entry, as supported by |
1994 | - add-apt-repository(1). Examples:: |
1995 | - |
1996 | - ppa:charmers/example |
1997 | - deb https://stub:key@private.example.com/ubuntu trusty main |
1998 | - |
1999 | - In addition: |
2000 | - 'proposed:' may be used to enable the standard 'proposed' |
2001 | - pocket for the release. |
2002 | - 'cloud:' may be used to activate official cloud archive pockets, |
2003 | - such as 'cloud:icehouse' |
2004 | - 'distro' may be used as a noop |
2005 | - |
2006 | - @param key: A key to be added to the system's APT keyring and used |
2007 | - to verify the signatures on packages. Ideally, this should be an |
2008 | - ASCII format GPG public key including the block headers. A GPG key |
2009 | - id may also be used, but be aware that only insecure protocols are |
2010 | - available to retrieve the actual public key from a public keyserver |
2011 | - placing your Juju environment at risk. ppa and cloud archive keys |
2012 | - are securely added automtically, so sould not be provided. |
2013 | - """ |
2014 | - if source is None: |
2015 | - log('Source is not present. Skipping') |
2016 | - return |
2017 | - |
2018 | - if (source.startswith('ppa:') or |
2019 | - source.startswith('http') or |
2020 | - source.startswith('deb ') or |
2021 | - source.startswith('cloud-archive:')): |
2022 | - subprocess.check_call(['add-apt-repository', '--yes', source]) |
2023 | - elif source.startswith('cloud:'): |
2024 | - apt_install(filter_installed_packages(['ubuntu-cloud-keyring']), |
2025 | - fatal=True) |
2026 | - pocket = source.split(':')[-1] |
2027 | - if pocket not in CLOUD_ARCHIVE_POCKETS: |
2028 | - raise SourceConfigError( |
2029 | - 'Unsupported cloud: source option %s' % |
2030 | - pocket) |
2031 | - actual_pocket = CLOUD_ARCHIVE_POCKETS[pocket] |
2032 | - with open('/etc/apt/sources.list.d/cloud-archive.list', 'w') as apt: |
2033 | - apt.write(CLOUD_ARCHIVE.format(actual_pocket)) |
2034 | - elif source == 'proposed': |
2035 | - release = lsb_release()['DISTRIB_CODENAME'] |
2036 | - with open('/etc/apt/sources.list.d/proposed.list', 'w') as apt: |
2037 | - apt.write(PROPOSED_POCKET.format(release)) |
2038 | - elif source == 'distro': |
2039 | - pass |
2040 | - else: |
2041 | - log("Unknown source: {!r}".format(source)) |
2042 | - |
2043 | - if key: |
2044 | - if '-----BEGIN PGP PUBLIC KEY BLOCK-----' in key: |
2045 | - with NamedTemporaryFile('w+') as key_file: |
2046 | - key_file.write(key) |
2047 | - key_file.flush() |
2048 | - key_file.seek(0) |
2049 | - subprocess.check_call(['apt-key', 'add', '-'], stdin=key_file) |
2050 | - else: |
2051 | - # Note that hkp: is in no way a secure protocol. Using a |
2052 | - # GPG key id is pointless from a security POV unless you |
2053 | - # absolutely trust your network and DNS. |
2054 | - subprocess.check_call(['apt-key', 'adv', '--keyserver', |
2055 | - 'hkp://keyserver.ubuntu.com:80', '--recv', |
2056 | - key]) |
2057 | +__platform__ = get_platform() |
2058 | +module = "charmhelpers.fetch.%s" % __platform__ |
2059 | +fetch = importlib.import_module(module) |
2060 | + |
2061 | +filter_installed_packages = fetch.filter_installed_packages |
2062 | +install = fetch.install |
2063 | +upgrade = fetch.upgrade |
2064 | +update = fetch.update |
2065 | +purge = fetch.purge |
2066 | +add_source = fetch.add_source |
2067 | + |
2068 | +if __platform__ == "ubuntu": |
2069 | + apt_cache = fetch.apt_cache |
2070 | + apt_install = fetch.install |
2071 | + apt_update = fetch.update |
2072 | + apt_upgrade = fetch.upgrade |
2073 | + apt_purge = fetch.purge |
2074 | + apt_mark = fetch.apt_mark |
2075 | + apt_hold = fetch.apt_hold |
2076 | + apt_unhold = fetch.apt_unhold |
2077 | + get_upstream_version = fetch.get_upstream_version |
2078 | +elif __platform__ == "centos": |
2079 | + yum_search = fetch.yum_search |
2080 | |
2081 | |
2082 | def configure_sources(update=False, |
2083 | sources_var='install_sources', |
2084 | keys_var='install_keys'): |
2085 | - """ |
2086 | - Configure multiple sources from charm configuration. |
2087 | + """Configure multiple sources from charm configuration. |
2088 | |
2089 | The lists are encoded as yaml fragments in the configuration. |
2090 | - The frament needs to be included as a string. Sources and their |
2091 | + The fragment needs to be included as a string. Sources and their |
2092 | corresponding keys are of the types supported by add_source(). |
2093 | |
2094 | Example config: |
2095 | @@ -362,12 +135,11 @@ |
2096 | for source, key in zip(sources, keys): |
2097 | add_source(source, key) |
2098 | if update: |
2099 | - apt_update(fatal=True) |
2100 | + fetch.update(fatal=True) |
2101 | |
2102 | |
2103 | def install_remote(source, *args, **kwargs): |
2104 | - """ |
2105 | - Install a file tree from a remote source |
2106 | + """Install a file tree from a remote source. |
2107 | |
2108 | The specified source should be a url of the form: |
2109 | scheme://[host]/path[#[option=value][&...]] |
2110 | @@ -390,19 +162,17 @@ |
2111 | # We ONLY check for True here because can_handle may return a string |
2112 | # explaining why it can't handle a given source. |
2113 | handlers = [h for h in plugins() if h.can_handle(source) is True] |
2114 | - installed_to = None |
2115 | for handler in handlers: |
2116 | try: |
2117 | - installed_to = handler.install(source, *args, **kwargs) |
2118 | + return handler.install(source, *args, **kwargs) |
2119 | except UnhandledSource as e: |
2120 | log('Install source attempt unsuccessful: {}'.format(e), |
2121 | level='WARNING') |
2122 | - if not installed_to: |
2123 | - raise UnhandledSource("No handler found for source {}".format(source)) |
2124 | - return installed_to |
2125 | + raise UnhandledSource("No handler found for source {}".format(source)) |
2126 | |
2127 | |
2128 | def install_from_config(config_var_name): |
2129 | + """Install a file from config.""" |
2130 | charm_config = config() |
2131 | source = charm_config[config_var_name] |
2132 | return install_remote(source) |
2133 | @@ -425,40 +195,3 @@ |
2134 | log("FetchHandler {} not found, skipping plugin".format( |
2135 | handler_name)) |
2136 | return plugin_list |
2137 | - |
2138 | - |
2139 | -def _run_apt_command(cmd, fatal=False): |
2140 | - """ |
2141 | - Run an APT command, checking output and retrying if the fatal flag is set |
2142 | - to True. |
2143 | - |
2144 | - :param: cmd: str: The apt command to run. |
2145 | - :param: fatal: bool: Whether the command's output should be checked and |
2146 | - retried. |
2147 | - """ |
2148 | - env = os.environ.copy() |
2149 | - |
2150 | - if 'DEBIAN_FRONTEND' not in env: |
2151 | - env['DEBIAN_FRONTEND'] = 'noninteractive' |
2152 | - |
2153 | - if fatal: |
2154 | - retry_count = 0 |
2155 | - result = None |
2156 | - |
2157 | - # If the command is considered "fatal", we need to retry if the apt |
2158 | - # lock was not acquired. |
2159 | - |
2160 | - while result is None or result == APT_NO_LOCK: |
2161 | - try: |
2162 | - result = subprocess.check_call(cmd, env=env) |
2163 | - except subprocess.CalledProcessError as e: |
2164 | - retry_count = retry_count + 1 |
2165 | - if retry_count > APT_NO_LOCK_RETRY_COUNT: |
2166 | - raise |
2167 | - result = e.returncode |
2168 | - log("Couldn't acquire DPKG lock. Will retry in {} seconds." |
2169 | - "".format(APT_NO_LOCK_RETRY_DELAY)) |
2170 | - time.sleep(APT_NO_LOCK_RETRY_DELAY) |
2171 | - |
2172 | - else: |
2173 | - subprocess.call(cmd, env=env) |
2174 | |
2175 | === modified file 'charmhelpers/fetch/archiveurl.py' |
2176 | --- charmhelpers/fetch/archiveurl.py 2015-12-11 15:23:38 +0000 |
2177 | +++ charmhelpers/fetch/archiveurl.py 2017-03-04 02:50:20 +0000 |
2178 | @@ -1,18 +1,16 @@ |
2179 | # Copyright 2014-2015 Canonical Limited. |
2180 | # |
2181 | -# This file is part of charm-helpers. |
2182 | -# |
2183 | -# charm-helpers is free software: you can redistribute it and/or modify |
2184 | -# it under the terms of the GNU Lesser General Public License version 3 as |
2185 | -# published by the Free Software Foundation. |
2186 | -# |
2187 | -# charm-helpers is distributed in the hope that it will be useful, |
2188 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
2189 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
2190 | -# GNU Lesser General Public License for more details. |
2191 | -# |
2192 | -# You should have received a copy of the GNU Lesser General Public License |
2193 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
2194 | +# Licensed under the Apache License, Version 2.0 (the "License"); |
2195 | +# you may not use this file except in compliance with the License. |
2196 | +# You may obtain a copy of the License at |
2197 | +# |
2198 | +# http://www.apache.org/licenses/LICENSE-2.0 |
2199 | +# |
2200 | +# Unless required by applicable law or agreed to in writing, software |
2201 | +# distributed under the License is distributed on an "AS IS" BASIS, |
2202 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
2203 | +# See the License for the specific language governing permissions and |
2204 | +# limitations under the License. |
2205 | |
2206 | import os |
2207 | import hashlib |
2208 | |
2209 | === modified file 'charmhelpers/fetch/bzrurl.py' |
2210 | --- charmhelpers/fetch/bzrurl.py 2016-05-06 07:12:40 +0000 |
2211 | +++ charmhelpers/fetch/bzrurl.py 2017-03-04 02:50:20 +0000 |
2212 | @@ -1,18 +1,16 @@ |
2213 | # Copyright 2014-2015 Canonical Limited. |
2214 | # |
2215 | -# This file is part of charm-helpers. |
2216 | -# |
2217 | -# charm-helpers is free software: you can redistribute it and/or modify |
2218 | -# it under the terms of the GNU Lesser General Public License version 3 as |
2219 | -# published by the Free Software Foundation. |
2220 | -# |
2221 | -# charm-helpers is distributed in the hope that it will be useful, |
2222 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
2223 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
2224 | -# GNU Lesser General Public License for more details. |
2225 | -# |
2226 | -# You should have received a copy of the GNU Lesser General Public License |
2227 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
2228 | +# Licensed under the Apache License, Version 2.0 (the "License"); |
2229 | +# you may not use this file except in compliance with the License. |
2230 | +# You may obtain a copy of the License at |
2231 | +# |
2232 | +# http://www.apache.org/licenses/LICENSE-2.0 |
2233 | +# |
2234 | +# Unless required by applicable law or agreed to in writing, software |
2235 | +# distributed under the License is distributed on an "AS IS" BASIS, |
2236 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
2237 | +# See the License for the specific language governing permissions and |
2238 | +# limitations under the License. |
2239 | |
2240 | import os |
2241 | from subprocess import check_call |
2242 | @@ -20,19 +18,20 @@ |
2243 | BaseFetchHandler, |
2244 | UnhandledSource, |
2245 | filter_installed_packages, |
2246 | - apt_install, |
2247 | + install, |
2248 | ) |
2249 | from charmhelpers.core.host import mkdir |
2250 | |
2251 | |
2252 | if filter_installed_packages(['bzr']) != []: |
2253 | - apt_install(['bzr']) |
2254 | + install(['bzr']) |
2255 | if filter_installed_packages(['bzr']) != []: |
2256 | raise NotImplementedError('Unable to install bzr') |
2257 | |
2258 | |
2259 | class BzrUrlFetchHandler(BaseFetchHandler): |
2260 | - """Handler for bazaar branches via generic and lp URLs""" |
2261 | + """Handler for bazaar branches via generic and lp URLs.""" |
2262 | + |
2263 | def can_handle(self, source): |
2264 | url_parts = self.parse_url(source) |
2265 | if url_parts.scheme not in ('bzr+ssh', 'lp', ''): |
2266 | @@ -42,15 +41,23 @@ |
2267 | else: |
2268 | return True |
2269 | |
2270 | - def branch(self, source, dest): |
2271 | + def branch(self, source, dest, revno=None): |
2272 | if not self.can_handle(source): |
2273 | raise UnhandledSource("Cannot handle {}".format(source)) |
2274 | + cmd_opts = [] |
2275 | + if revno: |
2276 | + cmd_opts += ['-r', str(revno)] |
2277 | if os.path.exists(dest): |
2278 | - check_call(['bzr', 'pull', '--overwrite', '-d', dest, source]) |
2279 | + cmd = ['bzr', 'pull'] |
2280 | + cmd += cmd_opts |
2281 | + cmd += ['--overwrite', '-d', dest, source] |
2282 | else: |
2283 | - check_call(['bzr', 'branch', source, dest]) |
2284 | + cmd = ['bzr', 'branch'] |
2285 | + cmd += cmd_opts |
2286 | + cmd += [source, dest] |
2287 | + check_call(cmd) |
2288 | |
2289 | - def install(self, source, dest=None): |
2290 | + def install(self, source, dest=None, revno=None): |
2291 | url_parts = self.parse_url(source) |
2292 | branch_name = url_parts.path.strip("/").split("/")[-1] |
2293 | if dest: |
2294 | @@ -59,10 +66,11 @@ |
2295 | dest_dir = os.path.join(os.environ.get('CHARM_DIR'), "fetched", |
2296 | branch_name) |
2297 | |
2298 | - if not os.path.exists(dest_dir): |
2299 | - mkdir(dest_dir, perms=0o755) |
2300 | + if dest and not os.path.exists(dest): |
2301 | + mkdir(dest, perms=0o755) |
2302 | + |
2303 | try: |
2304 | - self.branch(source, dest_dir) |
2305 | + self.branch(source, dest_dir, revno) |
2306 | except OSError as e: |
2307 | raise UnhandledSource(e.strerror) |
2308 | return dest_dir |
2309 | |
2310 | === added file 'charmhelpers/fetch/centos.py' |
2311 | --- charmhelpers/fetch/centos.py 1970-01-01 00:00:00 +0000 |
2312 | +++ charmhelpers/fetch/centos.py 2017-03-04 02:50:20 +0000 |
2313 | @@ -0,0 +1,171 @@ |
2314 | +# Copyright 2014-2015 Canonical Limited. |
2315 | +# |
2316 | +# Licensed under the Apache License, Version 2.0 (the "License"); |
2317 | +# you may not use this file except in compliance with the License. |
2318 | +# You may obtain a copy of the License at |
2319 | +# |
2320 | +# http://www.apache.org/licenses/LICENSE-2.0 |
2321 | +# |
2322 | +# Unless required by applicable law or agreed to in writing, software |
2323 | +# distributed under the License is distributed on an "AS IS" BASIS, |
2324 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
2325 | +# See the License for the specific language governing permissions and |
2326 | +# limitations under the License. |
2327 | + |
2328 | +import subprocess |
2329 | +import os |
2330 | +import time |
2331 | +import six |
2332 | +import yum |
2333 | + |
2334 | +from tempfile import NamedTemporaryFile |
2335 | +from charmhelpers.core.hookenv import log |
2336 | + |
2337 | +YUM_NO_LOCK = 1 # The return code for "couldn't acquire lock" in YUM. |
2338 | +YUM_NO_LOCK_RETRY_DELAY = 10 # Wait 10 seconds between apt lock checks. |
2339 | +YUM_NO_LOCK_RETRY_COUNT = 30 # Retry to acquire the lock X times. |
2340 | + |
2341 | + |
2342 | +def filter_installed_packages(packages): |
2343 | + """Return a list of packages that require installation.""" |
2344 | + yb = yum.YumBase() |
2345 | + package_list = yb.doPackageLists() |
2346 | + temp_cache = {p.base_package_name: 1 for p in package_list['installed']} |
2347 | + |
2348 | + _pkgs = [p for p in packages if not temp_cache.get(p, False)] |
2349 | + return _pkgs |
2350 | + |
2351 | + |
2352 | +def install(packages, options=None, fatal=False): |
2353 | + """Install one or more packages.""" |
2354 | + cmd = ['yum', '--assumeyes'] |
2355 | + if options is not None: |
2356 | + cmd.extend(options) |
2357 | + cmd.append('install') |
2358 | + if isinstance(packages, six.string_types): |
2359 | + cmd.append(packages) |
2360 | + else: |
2361 | + cmd.extend(packages) |
2362 | + log("Installing {} with options: {}".format(packages, |
2363 | + options)) |
2364 | + _run_yum_command(cmd, fatal) |
2365 | + |
2366 | + |
2367 | +def upgrade(options=None, fatal=False, dist=False): |
2368 | + """Upgrade all packages.""" |
2369 | + cmd = ['yum', '--assumeyes'] |
2370 | + if options is not None: |
2371 | + cmd.extend(options) |
2372 | + cmd.append('upgrade') |
2373 | + log("Upgrading with options: {}".format(options)) |
2374 | + _run_yum_command(cmd, fatal) |
2375 | + |
2376 | + |
2377 | +def update(fatal=False): |
2378 | + """Update local yum cache.""" |
2379 | + cmd = ['yum', '--assumeyes', 'update'] |
2380 | + log("Update with fatal: {}".format(fatal)) |
2381 | + _run_yum_command(cmd, fatal) |
2382 | + |
2383 | + |
2384 | +def purge(packages, fatal=False): |
2385 | + """Purge one or more packages.""" |
2386 | + cmd = ['yum', '--assumeyes', 'remove'] |
2387 | + if isinstance(packages, six.string_types): |
2388 | + cmd.append(packages) |
2389 | + else: |
2390 | + cmd.extend(packages) |
2391 | + log("Purging {}".format(packages)) |
2392 | + _run_yum_command(cmd, fatal) |
2393 | + |
2394 | + |
2395 | +def yum_search(packages): |
2396 | + """Search for a package.""" |
2397 | + output = {} |
2398 | + cmd = ['yum', 'search'] |
2399 | + if isinstance(packages, six.string_types): |
2400 | + cmd.append(packages) |
2401 | + else: |
2402 | + cmd.extend(packages) |
2403 | + log("Searching for {}".format(packages)) |
2404 | + result = subprocess.check_output(cmd) |
2405 | + for package in list(packages): |
2406 | + output[package] = package in result |
2407 | + return output |
2408 | + |
2409 | + |
2410 | +def add_source(source, key=None): |
2411 | + """Add a package source to this system. |
2412 | + |
2413 | + @param source: a URL with a rpm package |
2414 | + |
2415 | + @param key: A key to be added to the system's keyring and used |
2416 | + to verify the signatures on packages. Ideally, this should be an |
2417 | + ASCII format GPG public key including the block headers. A GPG key |
2418 | + id may also be used, but be aware that only insecure protocols are |
2419 | + available to retrieve the actual public key from a public keyserver |
2420 | + placing your Juju environment at risk. |
2421 | + """ |
2422 | + if source is None: |
2423 | + log('Source is not present. Skipping') |
2424 | + return |
2425 | + |
2426 | + if source.startswith('http'): |
2427 | + directory = '/etc/yum.repos.d/' |
2428 | + for filename in os.listdir(directory): |
2429 | + with open(directory + filename, 'r') as rpm_file: |
2430 | + if source in rpm_file.read(): |
2431 | + break |
2432 | + else: |
2433 | + log("Add source: {!r}".format(source)) |
2434 | + # write in the charms.repo |
2435 | + with open(directory + 'Charms.repo', 'a') as rpm_file: |
2436 | + rpm_file.write('[%s]\n' % source[7:].replace('/', '_')) |
2437 | + rpm_file.write('name=%s\n' % source[7:]) |
2438 | + rpm_file.write('baseurl=%s\n\n' % source) |
2439 | + else: |
2440 | + log("Unknown source: {!r}".format(source)) |
2441 | + |
2442 | + if key: |
2443 | + if '-----BEGIN PGP PUBLIC KEY BLOCK-----' in key: |
2444 | + with NamedTemporaryFile('w+') as key_file: |
2445 | + key_file.write(key) |
2446 | + key_file.flush() |
2447 | + key_file.seek(0) |
2448 | + subprocess.check_call(['rpm', '--import', key_file]) |
2449 | + else: |
2450 | + subprocess.check_call(['rpm', '--import', key]) |
2451 | + |
2452 | + |
2453 | +def _run_yum_command(cmd, fatal=False): |
2454 | + """Run an YUM command. |
2455 | + |
2456 | + Checks the output and retry if the fatal flag is set to True. |
2457 | + |
2458 | + :param: cmd: str: The yum command to run. |
2459 | + :param: fatal: bool: Whether the command's output should be checked and |
2460 | + retried. |
2461 | + """ |
2462 | + env = os.environ.copy() |
2463 | + |
2464 | + if fatal: |
2465 | + retry_count = 0 |
2466 | + result = None |
2467 | + |
2468 | + # If the command is considered "fatal", we need to retry if the yum |
2469 | + # lock was not acquired. |
2470 | + |
2471 | + while result is None or result == YUM_NO_LOCK: |
2472 | + try: |
2473 | + result = subprocess.check_call(cmd, env=env) |
2474 | + except subprocess.CalledProcessError as e: |
2475 | + retry_count = retry_count + 1 |
2476 | + if retry_count > YUM_NO_LOCK_RETRY_COUNT: |
2477 | + raise |
2478 | + result = e.returncode |
2479 | + log("Couldn't acquire YUM lock. Will retry in {} seconds." |
2480 | + "".format(YUM_NO_LOCK_RETRY_DELAY)) |
2481 | + time.sleep(YUM_NO_LOCK_RETRY_DELAY) |
2482 | + |
2483 | + else: |
2484 | + subprocess.call(cmd, env=env) |
2485 | |
2486 | === modified file 'charmhelpers/fetch/giturl.py' |
2487 | --- charmhelpers/fetch/giturl.py 2016-05-06 07:12:40 +0000 |
2488 | +++ charmhelpers/fetch/giturl.py 2017-03-04 02:50:20 +0000 |
2489 | @@ -1,18 +1,16 @@ |
2490 | # Copyright 2014-2015 Canonical Limited. |
2491 | # |
2492 | -# This file is part of charm-helpers. |
2493 | -# |
2494 | -# charm-helpers is free software: you can redistribute it and/or modify |
2495 | -# it under the terms of the GNU Lesser General Public License version 3 as |
2496 | -# published by the Free Software Foundation. |
2497 | -# |
2498 | -# charm-helpers is distributed in the hope that it will be useful, |
2499 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
2500 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
2501 | -# GNU Lesser General Public License for more details. |
2502 | -# |
2503 | -# You should have received a copy of the GNU Lesser General Public License |
2504 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
2505 | +# Licensed under the Apache License, Version 2.0 (the "License"); |
2506 | +# you may not use this file except in compliance with the License. |
2507 | +# You may obtain a copy of the License at |
2508 | +# |
2509 | +# http://www.apache.org/licenses/LICENSE-2.0 |
2510 | +# |
2511 | +# Unless required by applicable law or agreed to in writing, software |
2512 | +# distributed under the License is distributed on an "AS IS" BASIS, |
2513 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
2514 | +# See the License for the specific language governing permissions and |
2515 | +# limitations under the License. |
2516 | |
2517 | import os |
2518 | from subprocess import check_call, CalledProcessError |
2519 | @@ -20,17 +18,18 @@ |
2520 | BaseFetchHandler, |
2521 | UnhandledSource, |
2522 | filter_installed_packages, |
2523 | - apt_install, |
2524 | + install, |
2525 | ) |
2526 | |
2527 | if filter_installed_packages(['git']) != []: |
2528 | - apt_install(['git']) |
2529 | + install(['git']) |
2530 | if filter_installed_packages(['git']) != []: |
2531 | raise NotImplementedError('Unable to install git') |
2532 | |
2533 | |
2534 | class GitUrlFetchHandler(BaseFetchHandler): |
2535 | - """Handler for git branches via generic and github URLs""" |
2536 | + """Handler for git branches via generic and github URLs.""" |
2537 | + |
2538 | def can_handle(self, source): |
2539 | url_parts = self.parse_url(source) |
2540 | # TODO (mattyw) no support for ssh git@ yet |
2541 | |
2542 | === added file 'charmhelpers/fetch/snap.py' |
2543 | --- charmhelpers/fetch/snap.py 1970-01-01 00:00:00 +0000 |
2544 | +++ charmhelpers/fetch/snap.py 2017-03-04 02:50:20 +0000 |
2545 | @@ -0,0 +1,122 @@ |
2546 | +# Copyright 2014-2017 Canonical Limited. |
2547 | +# |
2548 | +# Licensed under the Apache License, Version 2.0 (the "License"); |
2549 | +# you may not use this file except in compliance with the License. |
2550 | +# You may obtain a copy of the License at |
2551 | +# |
2552 | +# http://www.apache.org/licenses/LICENSE-2.0 |
2553 | +# |
2554 | +# Unless required by applicable law or agreed to in writing, software |
2555 | +# distributed under the License is distributed on an "AS IS" BASIS, |
2556 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
2557 | +# See the License for the specific language governing permissions and |
2558 | +# limitations under the License. |
2559 | +""" |
2560 | +Charm helpers snap for classic charms. |
2561 | + |
2562 | +If writing reactive charms, use the snap layer: |
2563 | +https://lists.ubuntu.com/archives/snapcraft/2016-September/001114.html |
2564 | +""" |
2565 | +import subprocess |
2566 | +from os import environ |
2567 | +from time import sleep |
2568 | +from charmhelpers.core.hookenv import log |
2569 | + |
2570 | +__author__ = 'Joseph Borg <joseph.borg@canonical.com>' |
2571 | + |
2572 | +SNAP_NO_LOCK = 1 # The return code for "couldn't acquire lock" in Snap (hopefully this will be improved). |
2573 | +SNAP_NO_LOCK_RETRY_DELAY = 10 # Wait X seconds between Snap lock checks. |
2574 | +SNAP_NO_LOCK_RETRY_COUNT = 30 # Retry to acquire the lock X times. |
2575 | + |
2576 | + |
2577 | +class CouldNotAcquireLockException(Exception): |
2578 | + pass |
2579 | + |
2580 | + |
2581 | +def _snap_exec(commands): |
2582 | + """ |
2583 | + Execute snap commands. |
2584 | + |
2585 | + :param commands: List commands |
2586 | + :return: Integer exit code |
2587 | + """ |
2588 | + assert type(commands) == list |
2589 | + |
2590 | + retry_count = 0 |
2591 | + return_code = None |
2592 | + |
2593 | + while return_code is None or return_code == SNAP_NO_LOCK: |
2594 | + try: |
2595 | + return_code = subprocess.check_call(['snap'] + commands, env=environ) |
2596 | + except subprocess.CalledProcessError as e: |
2597 | + retry_count += + 1 |
2598 | + if retry_count > SNAP_NO_LOCK_RETRY_COUNT: |
2599 | + raise CouldNotAcquireLockException('Could not aquire lock after %s attempts' % SNAP_NO_LOCK_RETRY_COUNT) |
2600 | + return_code = e.returncode |
2601 | + log('Snap failed to acquire lock, trying again in %s seconds.' % SNAP_NO_LOCK_RETRY_DELAY, level='WARN') |
2602 | + sleep(SNAP_NO_LOCK_RETRY_DELAY) |
2603 | + |
2604 | + return return_code |
2605 | + |
2606 | + |
2607 | +def snap_install(packages, *flags): |
2608 | + """ |
2609 | + Install a snap package. |
2610 | + |
2611 | + :param packages: String or List String package name |
2612 | + :param flags: List String flags to pass to install command |
2613 | + :return: Integer return code from snap |
2614 | + """ |
2615 | + if type(packages) is not list: |
2616 | + packages = [packages] |
2617 | + |
2618 | + flags = list(flags) |
2619 | + |
2620 | + message = 'Installing snap(s) "%s"' % ', '.join(packages) |
2621 | + if flags: |
2622 | + message += ' with option(s) "%s"' % ', '.join(flags) |
2623 | + |
2624 | + log(message, level='INFO') |
2625 | + return _snap_exec(['install'] + flags + packages) |
2626 | + |
2627 | + |
2628 | +def snap_remove(packages, *flags): |
2629 | + """ |
2630 | + Remove a snap package. |
2631 | + |
2632 | + :param packages: String or List String package name |
2633 | + :param flags: List String flags to pass to remove command |
2634 | + :return: Integer return code from snap |
2635 | + """ |
2636 | + if type(packages) is not list: |
2637 | + packages = [packages] |
2638 | + |
2639 | + flags = list(flags) |
2640 | + |
2641 | + message = 'Removing snap(s) "%s"' % ', '.join(packages) |
2642 | + if flags: |
2643 | + message += ' with options "%s"' % ', '.join(flags) |
2644 | + |
2645 | + log(message, level='INFO') |
2646 | + return _snap_exec(['remove'] + flags + packages) |
2647 | + |
2648 | + |
2649 | +def snap_refresh(packages, *flags): |
2650 | + """ |
2651 | + Refresh / Update snap package. |
2652 | + |
2653 | + :param packages: String or List String package name |
2654 | + :param flags: List String flags to pass to refresh command |
2655 | + :return: Integer return code from snap |
2656 | + """ |
2657 | + if type(packages) is not list: |
2658 | + packages = [packages] |
2659 | + |
2660 | + flags = list(flags) |
2661 | + |
2662 | + message = 'Refreshing snap(s) "%s"' % ', '.join(packages) |
2663 | + if flags: |
2664 | + message += ' with options "%s"' % ', '.join(flags) |
2665 | + |
2666 | + log(message, level='INFO') |
2667 | + return _snap_exec(['refresh'] + flags + packages) |
2668 | |
2669 | === added file 'charmhelpers/fetch/ubuntu.py' |
2670 | --- charmhelpers/fetch/ubuntu.py 1970-01-01 00:00:00 +0000 |
2671 | +++ charmhelpers/fetch/ubuntu.py 2017-03-04 02:50:20 +0000 |
2672 | @@ -0,0 +1,364 @@ |
2673 | +# Copyright 2014-2015 Canonical Limited. |
2674 | +# |
2675 | +# Licensed under the Apache License, Version 2.0 (the "License"); |
2676 | +# you may not use this file except in compliance with the License. |
2677 | +# You may obtain a copy of the License at |
2678 | +# |
2679 | +# http://www.apache.org/licenses/LICENSE-2.0 |
2680 | +# |
2681 | +# Unless required by applicable law or agreed to in writing, software |
2682 | +# distributed under the License is distributed on an "AS IS" BASIS, |
2683 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
2684 | +# See the License for the specific language governing permissions and |
2685 | +# limitations under the License. |
2686 | + |
2687 | +import os |
2688 | +import six |
2689 | +import time |
2690 | +import subprocess |
2691 | + |
2692 | +from tempfile import NamedTemporaryFile |
2693 | +from charmhelpers.core.host import ( |
2694 | + lsb_release |
2695 | +) |
2696 | +from charmhelpers.core.hookenv import log |
2697 | +from charmhelpers.fetch import SourceConfigError |
2698 | + |
2699 | +CLOUD_ARCHIVE = """# Ubuntu Cloud Archive |
2700 | +deb http://ubuntu-cloud.archive.canonical.com/ubuntu {} main |
2701 | +""" |
2702 | + |
2703 | +PROPOSED_POCKET = """# Proposed |
2704 | +deb http://archive.ubuntu.com/ubuntu {}-proposed main universe multiverse restricted |
2705 | +""" |
2706 | + |
2707 | +CLOUD_ARCHIVE_POCKETS = { |
2708 | + # Folsom |
2709 | + 'folsom': 'precise-updates/folsom', |
2710 | + 'precise-folsom': 'precise-updates/folsom', |
2711 | + 'precise-folsom/updates': 'precise-updates/folsom', |
2712 | + 'precise-updates/folsom': 'precise-updates/folsom', |
2713 | + 'folsom/proposed': 'precise-proposed/folsom', |
2714 | + 'precise-folsom/proposed': 'precise-proposed/folsom', |
2715 | + 'precise-proposed/folsom': 'precise-proposed/folsom', |
2716 | + # Grizzly |
2717 | + 'grizzly': 'precise-updates/grizzly', |
2718 | + 'precise-grizzly': 'precise-updates/grizzly', |
2719 | + 'precise-grizzly/updates': 'precise-updates/grizzly', |
2720 | + 'precise-updates/grizzly': 'precise-updates/grizzly', |
2721 | + 'grizzly/proposed': 'precise-proposed/grizzly', |
2722 | + 'precise-grizzly/proposed': 'precise-proposed/grizzly', |
2723 | + 'precise-proposed/grizzly': 'precise-proposed/grizzly', |
2724 | + # Havana |
2725 | + 'havana': 'precise-updates/havana', |
2726 | + 'precise-havana': 'precise-updates/havana', |
2727 | + 'precise-havana/updates': 'precise-updates/havana', |
2728 | + 'precise-updates/havana': 'precise-updates/havana', |
2729 | + 'havana/proposed': 'precise-proposed/havana', |
2730 | + 'precise-havana/proposed': 'precise-proposed/havana', |
2731 | + 'precise-proposed/havana': 'precise-proposed/havana', |
2732 | + # Icehouse |
2733 | + 'icehouse': 'precise-updates/icehouse', |
2734 | + 'precise-icehouse': 'precise-updates/icehouse', |
2735 | + 'precise-icehouse/updates': 'precise-updates/icehouse', |
2736 | + 'precise-updates/icehouse': 'precise-updates/icehouse', |
2737 | + 'icehouse/proposed': 'precise-proposed/icehouse', |
2738 | + 'precise-icehouse/proposed': 'precise-proposed/icehouse', |
2739 | + 'precise-proposed/icehouse': 'precise-proposed/icehouse', |
2740 | + # Juno |
2741 | + 'juno': 'trusty-updates/juno', |
2742 | + 'trusty-juno': 'trusty-updates/juno', |
2743 | + 'trusty-juno/updates': 'trusty-updates/juno', |
2744 | + 'trusty-updates/juno': 'trusty-updates/juno', |
2745 | + 'juno/proposed': 'trusty-proposed/juno', |
2746 | + 'trusty-juno/proposed': 'trusty-proposed/juno', |
2747 | + 'trusty-proposed/juno': 'trusty-proposed/juno', |
2748 | + # Kilo |
2749 | + 'kilo': 'trusty-updates/kilo', |
2750 | + 'trusty-kilo': 'trusty-updates/kilo', |
2751 | + 'trusty-kilo/updates': 'trusty-updates/kilo', |
2752 | + 'trusty-updates/kilo': 'trusty-updates/kilo', |
2753 | + 'kilo/proposed': 'trusty-proposed/kilo', |
2754 | + 'trusty-kilo/proposed': 'trusty-proposed/kilo', |
2755 | + 'trusty-proposed/kilo': 'trusty-proposed/kilo', |
2756 | + # Liberty |
2757 | + 'liberty': 'trusty-updates/liberty', |
2758 | + 'trusty-liberty': 'trusty-updates/liberty', |
2759 | + 'trusty-liberty/updates': 'trusty-updates/liberty', |
2760 | + 'trusty-updates/liberty': 'trusty-updates/liberty', |
2761 | + 'liberty/proposed': 'trusty-proposed/liberty', |
2762 | + 'trusty-liberty/proposed': 'trusty-proposed/liberty', |
2763 | + 'trusty-proposed/liberty': 'trusty-proposed/liberty', |
2764 | + # Mitaka |
2765 | + 'mitaka': 'trusty-updates/mitaka', |
2766 | + 'trusty-mitaka': 'trusty-updates/mitaka', |
2767 | + 'trusty-mitaka/updates': 'trusty-updates/mitaka', |
2768 | + 'trusty-updates/mitaka': 'trusty-updates/mitaka', |
2769 | + 'mitaka/proposed': 'trusty-proposed/mitaka', |
2770 | + 'trusty-mitaka/proposed': 'trusty-proposed/mitaka', |
2771 | + 'trusty-proposed/mitaka': 'trusty-proposed/mitaka', |
2772 | + # Newton |
2773 | + 'newton': 'xenial-updates/newton', |
2774 | + 'xenial-newton': 'xenial-updates/newton', |
2775 | + 'xenial-newton/updates': 'xenial-updates/newton', |
2776 | + 'xenial-updates/newton': 'xenial-updates/newton', |
2777 | + 'newton/proposed': 'xenial-proposed/newton', |
2778 | + 'xenial-newton/proposed': 'xenial-proposed/newton', |
2779 | + 'xenial-proposed/newton': 'xenial-proposed/newton', |
2780 | + # Ocata |
2781 | + 'ocata': 'xenial-updates/ocata', |
2782 | + 'xenial-ocata': 'xenial-updates/ocata', |
2783 | + 'xenial-ocata/updates': 'xenial-updates/ocata', |
2784 | + 'xenial-updates/ocata': 'xenial-updates/ocata', |
2785 | + 'ocata/proposed': 'xenial-proposed/ocata', |
2786 | + 'xenial-ocata/proposed': 'xenial-proposed/ocata', |
2787 | + 'xenial-ocata/newton': 'xenial-proposed/ocata', |
2788 | +} |
2789 | + |
2790 | +APT_NO_LOCK = 100 # The return code for "couldn't acquire lock" in APT. |
2791 | +CMD_RETRY_DELAY = 10 # Wait 10 seconds between command retries. |
2792 | +CMD_RETRY_COUNT = 30 # Retry a failing fatal command X times. |
2793 | + |
2794 | + |
2795 | +def filter_installed_packages(packages): |
2796 | + """Return a list of packages that require installation.""" |
2797 | + cache = apt_cache() |
2798 | + _pkgs = [] |
2799 | + for package in packages: |
2800 | + try: |
2801 | + p = cache[package] |
2802 | + p.current_ver or _pkgs.append(package) |
2803 | + except KeyError: |
2804 | + log('Package {} has no installation candidate.'.format(package), |
2805 | + level='WARNING') |
2806 | + _pkgs.append(package) |
2807 | + return _pkgs |
2808 | + |
2809 | + |
2810 | +def apt_cache(in_memory=True, progress=None): |
2811 | + """Build and return an apt cache.""" |
2812 | + from apt import apt_pkg |
2813 | + apt_pkg.init() |
2814 | + if in_memory: |
2815 | + apt_pkg.config.set("Dir::Cache::pkgcache", "") |
2816 | + apt_pkg.config.set("Dir::Cache::srcpkgcache", "") |
2817 | + return apt_pkg.Cache(progress) |
2818 | + |
2819 | + |
2820 | +def install(packages, options=None, fatal=False): |
2821 | + """Install one or more packages.""" |
2822 | + if options is None: |
2823 | + options = ['--option=Dpkg::Options::=--force-confold'] |
2824 | + |
2825 | + cmd = ['apt-get', '--assume-yes'] |
2826 | + cmd.extend(options) |
2827 | + cmd.append('install') |
2828 | + if isinstance(packages, six.string_types): |
2829 | + cmd.append(packages) |
2830 | + else: |
2831 | + cmd.extend(packages) |
2832 | + log("Installing {} with options: {}".format(packages, |
2833 | + options)) |
2834 | + _run_apt_command(cmd, fatal) |
2835 | + |
2836 | + |
2837 | +def upgrade(options=None, fatal=False, dist=False): |
2838 | + """Upgrade all packages.""" |
2839 | + if options is None: |
2840 | + options = ['--option=Dpkg::Options::=--force-confold'] |
2841 | + |
2842 | + cmd = ['apt-get', '--assume-yes'] |
2843 | + cmd.extend(options) |
2844 | + if dist: |
2845 | + cmd.append('dist-upgrade') |
2846 | + else: |
2847 | + cmd.append('upgrade') |
2848 | + log("Upgrading with options: {}".format(options)) |
2849 | + _run_apt_command(cmd, fatal) |
2850 | + |
2851 | + |
2852 | +def update(fatal=False): |
2853 | + """Update local apt cache.""" |
2854 | + cmd = ['apt-get', 'update'] |
2855 | + _run_apt_command(cmd, fatal) |
2856 | + |
2857 | + |
2858 | +def purge(packages, fatal=False): |
2859 | + """Purge one or more packages.""" |
2860 | + cmd = ['apt-get', '--assume-yes', 'purge'] |
2861 | + if isinstance(packages, six.string_types): |
2862 | + cmd.append(packages) |
2863 | + else: |
2864 | + cmd.extend(packages) |
2865 | + log("Purging {}".format(packages)) |
2866 | + _run_apt_command(cmd, fatal) |
2867 | + |
2868 | + |
2869 | +def apt_mark(packages, mark, fatal=False): |
2870 | + """Flag one or more packages using apt-mark.""" |
2871 | + log("Marking {} as {}".format(packages, mark)) |
2872 | + cmd = ['apt-mark', mark] |
2873 | + if isinstance(packages, six.string_types): |
2874 | + cmd.append(packages) |
2875 | + else: |
2876 | + cmd.extend(packages) |
2877 | + |
2878 | + if fatal: |
2879 | + subprocess.check_call(cmd, universal_newlines=True) |
2880 | + else: |
2881 | + subprocess.call(cmd, universal_newlines=True) |
2882 | + |
2883 | + |
2884 | +def apt_hold(packages, fatal=False): |
2885 | + return apt_mark(packages, 'hold', fatal=fatal) |
2886 | + |
2887 | + |
2888 | +def apt_unhold(packages, fatal=False): |
2889 | + return apt_mark(packages, 'unhold', fatal=fatal) |
2890 | + |
2891 | + |
2892 | +def add_source(source, key=None): |
2893 | + """Add a package source to this system. |
2894 | + |
2895 | + @param source: a URL or sources.list entry, as supported by |
2896 | + add-apt-repository(1). Examples:: |
2897 | + |
2898 | + ppa:charmers/example |
2899 | + deb https://stub:key@private.example.com/ubuntu trusty main |
2900 | + |
2901 | + In addition: |
2902 | + 'proposed:' may be used to enable the standard 'proposed' |
2903 | + pocket for the release. |
2904 | + 'cloud:' may be used to activate official cloud archive pockets, |
2905 | + such as 'cloud:icehouse' |
2906 | + 'distro' may be used as a noop |
2907 | + |
2908 | + @param key: A key to be added to the system's APT keyring and used |
2909 | + to verify the signatures on packages. Ideally, this should be an |
2910 | + ASCII format GPG public key including the block headers. A GPG key |
2911 | + id may also be used, but be aware that only insecure protocols are |
2912 | + available to retrieve the actual public key from a public keyserver |
2913 | + placing your Juju environment at risk. ppa and cloud archive keys |
2914 | + are securely added automtically, so sould not be provided. |
2915 | + """ |
2916 | + if source is None: |
2917 | + log('Source is not present. Skipping') |
2918 | + return |
2919 | + |
2920 | + if (source.startswith('ppa:') or |
2921 | + source.startswith('http') or |
2922 | + source.startswith('deb ') or |
2923 | + source.startswith('cloud-archive:')): |
2924 | + cmd = ['add-apt-repository', '--yes', source] |
2925 | + _run_with_retries(cmd) |
2926 | + elif source.startswith('cloud:'): |
2927 | + install(filter_installed_packages(['ubuntu-cloud-keyring']), |
2928 | + fatal=True) |
2929 | + pocket = source.split(':')[-1] |
2930 | + if pocket not in CLOUD_ARCHIVE_POCKETS: |
2931 | + raise SourceConfigError( |
2932 | + 'Unsupported cloud: source option %s' % |
2933 | + pocket) |
2934 | + actual_pocket = CLOUD_ARCHIVE_POCKETS[pocket] |
2935 | + with open('/etc/apt/sources.list.d/cloud-archive.list', 'w') as apt: |
2936 | + apt.write(CLOUD_ARCHIVE.format(actual_pocket)) |
2937 | + elif source == 'proposed': |
2938 | + release = lsb_release()['DISTRIB_CODENAME'] |
2939 | + with open('/etc/apt/sources.list.d/proposed.list', 'w') as apt: |
2940 | + apt.write(PROPOSED_POCKET.format(release)) |
2941 | + elif source == 'distro': |
2942 | + pass |
2943 | + else: |
2944 | + log("Unknown source: {!r}".format(source)) |
2945 | + |
2946 | + if key: |
2947 | + if '-----BEGIN PGP PUBLIC KEY BLOCK-----' in key: |
2948 | + with NamedTemporaryFile('w+') as key_file: |
2949 | + key_file.write(key) |
2950 | + key_file.flush() |
2951 | + key_file.seek(0) |
2952 | + subprocess.check_call(['apt-key', 'add', '-'], stdin=key_file) |
2953 | + else: |
2954 | + # Note that hkp: is in no way a secure protocol. Using a |
2955 | + # GPG key id is pointless from a security POV unless you |
2956 | + # absolutely trust your network and DNS. |
2957 | + subprocess.check_call(['apt-key', 'adv', '--keyserver', |
2958 | + 'hkp://keyserver.ubuntu.com:80', '--recv', |
2959 | + key]) |
2960 | + |
2961 | + |
2962 | +def _run_with_retries(cmd, max_retries=CMD_RETRY_COUNT, retry_exitcodes=(1,), |
2963 | + retry_message="", cmd_env=None): |
2964 | + """Run a command and retry until success or max_retries is reached. |
2965 | + |
2966 | + :param: cmd: str: The apt command to run. |
2967 | + :param: max_retries: int: The number of retries to attempt on a fatal |
2968 | + command. Defaults to CMD_RETRY_COUNT. |
2969 | + :param: retry_exitcodes: tuple: Optional additional exit codes to retry. |
2970 | + Defaults to retry on exit code 1. |
2971 | + :param: retry_message: str: Optional log prefix emitted during retries. |
2972 | + :param: cmd_env: dict: Environment variables to add to the command run. |
2973 | + """ |
2974 | + |
2975 | + env = os.environ.copy() |
2976 | + if cmd_env: |
2977 | + env.update(cmd_env) |
2978 | + |
2979 | + if not retry_message: |
2980 | + retry_message = "Failed executing '{}'".format(" ".join(cmd)) |
2981 | + retry_message += ". Will retry in {} seconds".format(CMD_RETRY_DELAY) |
2982 | + |
2983 | + retry_count = 0 |
2984 | + result = None |
2985 | + |
2986 | + retry_results = (None,) + retry_exitcodes |
2987 | + while result in retry_results: |
2988 | + try: |
2989 | + result = subprocess.check_call(cmd, env=env) |
2990 | + except subprocess.CalledProcessError as e: |
2991 | + retry_count = retry_count + 1 |
2992 | + if retry_count > max_retries: |
2993 | + raise |
2994 | + result = e.returncode |
2995 | + log(retry_message) |
2996 | + time.sleep(CMD_RETRY_DELAY) |
2997 | + |
2998 | + |
2999 | +def _run_apt_command(cmd, fatal=False): |
3000 | + """Run an apt command with optional retries. |
3001 | + |
3002 | + :param: fatal: bool: Whether the command's output should be checked and |
3003 | + retried. |
3004 | + """ |
3005 | + # Provide DEBIAN_FRONTEND=noninteractive if not present in the environment. |
3006 | + cmd_env = { |
3007 | + 'DEBIAN_FRONTEND': os.environ.get('DEBIAN_FRONTEND', 'noninteractive')} |
3008 | + |
3009 | + if fatal: |
3010 | + _run_with_retries( |
3011 | + cmd, cmd_env=cmd_env, retry_exitcodes=(1, APT_NO_LOCK,), |
3012 | + retry_message="Couldn't acquire DPKG lock") |
3013 | + else: |
3014 | + env = os.environ.copy() |
3015 | + env.update(cmd_env) |
3016 | + subprocess.call(cmd, env=env) |
3017 | + |
3018 | + |
3019 | +def get_upstream_version(package): |
3020 | + """Determine upstream version based on installed package |
3021 | + |
3022 | + @returns None (if not installed) or the upstream version |
3023 | + """ |
3024 | + import apt_pkg |
3025 | + cache = apt_cache() |
3026 | + try: |
3027 | + pkg = cache[package] |
3028 | + except: |
3029 | + # the package is unknown to the current apt cache. |
3030 | + return None |
3031 | + |
3032 | + if not pkg.current_ver: |
3033 | + # package is known, but no version is currently installed. |
3034 | + return None |
3035 | + |
3036 | + return apt_pkg.upstream_version(pkg.current_ver.ver_str) |
3037 | |
3038 | === added file 'charmhelpers/osplatform.py' |
3039 | --- charmhelpers/osplatform.py 1970-01-01 00:00:00 +0000 |
3040 | +++ charmhelpers/osplatform.py 2017-03-04 02:50:20 +0000 |
3041 | @@ -0,0 +1,25 @@ |
3042 | +import platform |
3043 | + |
3044 | + |
3045 | +def get_platform(): |
3046 | + """Return the current OS platform. |
3047 | + |
3048 | + For example: if current os platform is Ubuntu then a string "ubuntu" |
3049 | + will be returned (which is the name of the module). |
3050 | + This string is used to decide which platform module should be imported. |
3051 | + """ |
3052 | + # linux_distribution is deprecated and will be removed in Python 3.7 |
3053 | + # Warings *not* disabled, as we certainly need to fix this. |
3054 | + tuple_platform = platform.linux_distribution() |
3055 | + current_platform = tuple_platform[0] |
3056 | + if "Ubuntu" in current_platform: |
3057 | + return "ubuntu" |
3058 | + elif "CentOS" in current_platform: |
3059 | + return "centos" |
3060 | + elif "debian" in current_platform: |
3061 | + # Stock Python does not detect Ubuntu and instead returns debian. |
3062 | + # Or at least it does in some build environments like Travis CI |
3063 | + return "ubuntu" |
3064 | + else: |
3065 | + raise RuntimeError("This module is not supported on {}." |
3066 | + .format(current_platform)) |
3067 | |
3068 | === modified file 'lib/apt.py' |
3069 | --- lib/apt.py 2017-03-01 15:05:23 +0000 |
3070 | +++ lib/apt.py 2017-03-04 02:50:20 +0000 |
3071 | @@ -14,7 +14,8 @@ |
3072 | |
3073 | LANDSCAPE_PACKAGES = ("landscape-server", "landscape-hashids") |
3074 | INSTALL_PACKAGES = LANDSCAPE_PACKAGES + ("python-minimal", "python-psutil") |
3075 | -PACKAGES_DEV = ("dpkg-dev", "devscripts", "pbuilder", "aptitude", "build-essential") |
3076 | +PACKAGES_DEV = ( |
3077 | + "dpkg-dev", "devscripts", "pbuilder", "aptitude", "build-essential") |
3078 | TARBALL = "landscape-server_*.tar.gz" |
3079 | |
3080 | # XXX Eventually we'll want to use a dedicated PPA, populated by Jenkins. |
Command: make ci-test /ci.lscape. net/job/ latch-test- xenial/ 1447/
Result: Fail
Revno: 383
Branch: lp:~chad.smith/landscape-charm/add-apt-repository-retries
Jenkins: https:/