Merge lp:~chad.smith/landscape-charm/add-apt-repository-retries into lp:landscape-charm

Proposed by Chad Smith
Status: Merged
Approved by: Chad Smith
Approved revision: 386
Merged at revision: 383
Proposed branch: lp:~chad.smith/landscape-charm/add-apt-repository-retries
Merge into: lp:landscape-charm
Diff against target: 3080 lines (+1633/-771)
34 files modified
charm-helpers.yaml (+1/-0)
charmhelpers/__init__.py (+11/-13)
charmhelpers/contrib/__init__.py (+11/-13)
charmhelpers/contrib/hahelpers/__init__.py (+11/-13)
charmhelpers/contrib/hahelpers/apache.py (+30/-17)
charmhelpers/contrib/hahelpers/cluster.py (+70/-23)
charmhelpers/core/__init__.py (+11/-13)
charmhelpers/core/decorators.py (+11/-13)
charmhelpers/core/files.py (+11/-13)
charmhelpers/core/fstab.py (+11/-13)
charmhelpers/core/hookenv.py (+73/-14)
charmhelpers/core/host.py (+327/-126)
charmhelpers/core/host_factory/centos.py (+56/-0)
charmhelpers/core/host_factory/ubuntu.py (+56/-0)
charmhelpers/core/hugepage.py (+11/-13)
charmhelpers/core/kernel.py (+34/-30)
charmhelpers/core/kernel_factory/centos.py (+17/-0)
charmhelpers/core/kernel_factory/ubuntu.py (+13/-0)
charmhelpers/core/services/__init__.py (+11/-13)
charmhelpers/core/services/base.py (+11/-13)
charmhelpers/core/services/helpers.py (+11/-13)
charmhelpers/core/strutils.py (+11/-13)
charmhelpers/core/sysctl.py (+11/-13)
charmhelpers/core/templating.py (+19/-16)
charmhelpers/core/unitdata.py (+11/-14)
charmhelpers/fetch/__init__.py (+42/-309)
charmhelpers/fetch/archiveurl.py (+11/-13)
charmhelpers/fetch/bzrurl.py (+31/-23)
charmhelpers/fetch/centos.py (+171/-0)
charmhelpers/fetch/giturl.py (+15/-16)
charmhelpers/fetch/snap.py (+122/-0)
charmhelpers/fetch/ubuntu.py (+364/-0)
charmhelpers/osplatform.py (+25/-0)
lib/apt.py (+2/-1)
To merge this branch: bzr merge lp:~chad.smith/landscape-charm/add-apt-repository-retries
Reviewer Review Type Date Requested Status
Landscape Builder test results Approve
Eric Snow (community) Approve
Review via email: mp+318962@code.launchpad.net

Commit message

Sync of charmhelpers to get updated fetch.add_source which handles 30 retries upon exit 1

Add charmhelpers.osplatform dependency to charm-helpers.yaml

Description of the change

Sync of charmhelpers to get updated fetch.add_source which handles 30 retries upon exit 1. Generally this exit code is returned upon network failure like bug https://bugs.launchpad.net/charm-helpers/+bug/1370933.

Retry fixes are in charmhelpers/fetch/ubuntu.py

To post a comment you must log in.
Revision history for this message
Landscape Builder (landscape-builder) :
review: Abstain (executing tests)
Revision history for this message
Landscape Builder (landscape-builder) wrote :
review: Needs Fixing (test results)
Revision history for this message
Eric Snow (ericsnowcurrently) wrote :

As this doesn't introduce any changes, LGTM. :)

review: Approve
384. By Chad Smith

add charmhelpers.osplatform dependency for charmhelpers.host get_platform

385. By Chad Smith

make sync pulling in osplatform

Revision history for this message
Landscape Builder (landscape-builder) :
review: Abstain (executing tests)
Revision history for this message
Landscape Builder (landscape-builder) wrote :
review: Needs Fixing (test results)
386. By Chad Smith

lint me

Revision history for this message
Landscape Builder (landscape-builder) :
review: Abstain (executing tests)
Revision history for this message
Landscape Builder (landscape-builder) wrote :
review: Approve (test results)

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'charm-helpers.yaml'
2--- charm-helpers.yaml 2015-05-07 10:26:45 +0000
3+++ charm-helpers.yaml 2017-03-04 02:50:20 +0000
4@@ -4,4 +4,5 @@
5 - __init__
6 - core
7 - fetch
8+ - osplatform
9 - contrib.hahelpers
10
11=== modified file 'charmhelpers/__init__.py'
12--- charmhelpers/__init__.py 2015-01-28 08:59:02 +0000
13+++ charmhelpers/__init__.py 2017-03-04 02:50:20 +0000
14@@ -1,18 +1,16 @@
15 # Copyright 2014-2015 Canonical Limited.
16 #
17-# This file is part of charm-helpers.
18-#
19-# charm-helpers is free software: you can redistribute it and/or modify
20-# it under the terms of the GNU Lesser General Public License version 3 as
21-# published by the Free Software Foundation.
22-#
23-# charm-helpers is distributed in the hope that it will be useful,
24-# but WITHOUT ANY WARRANTY; without even the implied warranty of
25-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
26-# GNU Lesser General Public License for more details.
27-#
28-# You should have received a copy of the GNU Lesser General Public License
29-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
30+# Licensed under the Apache License, Version 2.0 (the "License");
31+# you may not use this file except in compliance with the License.
32+# You may obtain a copy of the License at
33+#
34+# http://www.apache.org/licenses/LICENSE-2.0
35+#
36+# Unless required by applicable law or agreed to in writing, software
37+# distributed under the License is distributed on an "AS IS" BASIS,
38+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
39+# See the License for the specific language governing permissions and
40+# limitations under the License.
41
42 # Bootstrap charm-helpers, installing its dependencies if necessary using
43 # only standard libraries.
44
45=== modified file 'charmhelpers/contrib/__init__.py'
46--- charmhelpers/contrib/__init__.py 2015-01-30 11:16:09 +0000
47+++ charmhelpers/contrib/__init__.py 2017-03-04 02:50:20 +0000
48@@ -1,15 +1,13 @@
49 # Copyright 2014-2015 Canonical Limited.
50 #
51-# This file is part of charm-helpers.
52-#
53-# charm-helpers is free software: you can redistribute it and/or modify
54-# it under the terms of the GNU Lesser General Public License version 3 as
55-# published by the Free Software Foundation.
56-#
57-# charm-helpers is distributed in the hope that it will be useful,
58-# but WITHOUT ANY WARRANTY; without even the implied warranty of
59-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
60-# GNU Lesser General Public License for more details.
61-#
62-# You should have received a copy of the GNU Lesser General Public License
63-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
64+# Licensed under the Apache License, Version 2.0 (the "License");
65+# you may not use this file except in compliance with the License.
66+# You may obtain a copy of the License at
67+#
68+# http://www.apache.org/licenses/LICENSE-2.0
69+#
70+# Unless required by applicable law or agreed to in writing, software
71+# distributed under the License is distributed on an "AS IS" BASIS,
72+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
73+# See the License for the specific language governing permissions and
74+# limitations under the License.
75
76=== modified file 'charmhelpers/contrib/hahelpers/__init__.py'
77--- charmhelpers/contrib/hahelpers/__init__.py 2015-01-30 11:16:09 +0000
78+++ charmhelpers/contrib/hahelpers/__init__.py 2017-03-04 02:50:20 +0000
79@@ -1,15 +1,13 @@
80 # Copyright 2014-2015 Canonical Limited.
81 #
82-# This file is part of charm-helpers.
83-#
84-# charm-helpers is free software: you can redistribute it and/or modify
85-# it under the terms of the GNU Lesser General Public License version 3 as
86-# published by the Free Software Foundation.
87-#
88-# charm-helpers is distributed in the hope that it will be useful,
89-# but WITHOUT ANY WARRANTY; without even the implied warranty of
90-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
91-# GNU Lesser General Public License for more details.
92-#
93-# You should have received a copy of the GNU Lesser General Public License
94-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
95+# Licensed under the Apache License, Version 2.0 (the "License");
96+# you may not use this file except in compliance with the License.
97+# You may obtain a copy of the License at
98+#
99+# http://www.apache.org/licenses/LICENSE-2.0
100+#
101+# Unless required by applicable law or agreed to in writing, software
102+# distributed under the License is distributed on an "AS IS" BASIS,
103+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
104+# See the License for the specific language governing permissions and
105+# limitations under the License.
106
107=== modified file 'charmhelpers/contrib/hahelpers/apache.py'
108--- charmhelpers/contrib/hahelpers/apache.py 2015-01-30 11:16:09 +0000
109+++ charmhelpers/contrib/hahelpers/apache.py 2017-03-04 02:50:20 +0000
110@@ -1,18 +1,16 @@
111 # Copyright 2014-2015 Canonical Limited.
112 #
113-# This file is part of charm-helpers.
114-#
115-# charm-helpers is free software: you can redistribute it and/or modify
116-# it under the terms of the GNU Lesser General Public License version 3 as
117-# published by the Free Software Foundation.
118-#
119-# charm-helpers is distributed in the hope that it will be useful,
120-# but WITHOUT ANY WARRANTY; without even the implied warranty of
121-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
122-# GNU Lesser General Public License for more details.
123-#
124-# You should have received a copy of the GNU Lesser General Public License
125-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
126+# Licensed under the Apache License, Version 2.0 (the "License");
127+# you may not use this file except in compliance with the License.
128+# You may obtain a copy of the License at
129+#
130+# http://www.apache.org/licenses/LICENSE-2.0
131+#
132+# Unless required by applicable law or agreed to in writing, software
133+# distributed under the License is distributed on an "AS IS" BASIS,
134+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
135+# See the License for the specific language governing permissions and
136+# limitations under the License.
137
138 #
139 # Copyright 2012 Canonical Ltd.
140@@ -24,6 +22,7 @@
141 # Adam Gandelman <adamg@ubuntu.com>
142 #
143
144+import os
145 import subprocess
146
147 from charmhelpers.core.hookenv import (
148@@ -74,9 +73,23 @@
149 return ca_cert
150
151
152+def retrieve_ca_cert(cert_file):
153+ cert = None
154+ if os.path.isfile(cert_file):
155+ with open(cert_file, 'r') as crt:
156+ cert = crt.read()
157+ return cert
158+
159+
160 def install_ca_cert(ca_cert):
161 if ca_cert:
162- with open('/usr/local/share/ca-certificates/keystone_juju_ca_cert.crt',
163- 'w') as crt:
164- crt.write(ca_cert)
165- subprocess.check_call(['update-ca-certificates', '--fresh'])
166+ cert_file = ('/usr/local/share/ca-certificates/'
167+ 'keystone_juju_ca_cert.crt')
168+ old_cert = retrieve_ca_cert(cert_file)
169+ if old_cert and old_cert == ca_cert:
170+ log("CA cert is the same as installed version", level=INFO)
171+ else:
172+ log("Installing new CA cert", level=INFO)
173+ with open(cert_file, 'w') as crt:
174+ crt.write(ca_cert)
175+ subprocess.check_call(['update-ca-certificates', '--fresh'])
176
177=== modified file 'charmhelpers/contrib/hahelpers/cluster.py'
178--- charmhelpers/contrib/hahelpers/cluster.py 2015-07-03 09:13:26 +0000
179+++ charmhelpers/contrib/hahelpers/cluster.py 2017-03-04 02:50:20 +0000
180@@ -1,18 +1,16 @@
181 # Copyright 2014-2015 Canonical Limited.
182 #
183-# This file is part of charm-helpers.
184-#
185-# charm-helpers is free software: you can redistribute it and/or modify
186-# it under the terms of the GNU Lesser General Public License version 3 as
187-# published by the Free Software Foundation.
188-#
189-# charm-helpers is distributed in the hope that it will be useful,
190-# but WITHOUT ANY WARRANTY; without even the implied warranty of
191-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
192-# GNU Lesser General Public License for more details.
193-#
194-# You should have received a copy of the GNU Lesser General Public License
195-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
196+# Licensed under the Apache License, Version 2.0 (the "License");
197+# you may not use this file except in compliance with the License.
198+# You may obtain a copy of the License at
199+#
200+# http://www.apache.org/licenses/LICENSE-2.0
201+#
202+# Unless required by applicable law or agreed to in writing, software
203+# distributed under the License is distributed on an "AS IS" BASIS,
204+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
205+# See the License for the specific language governing permissions and
206+# limitations under the License.
207
208 #
209 # Copyright 2012 Canonical Ltd.
210@@ -41,10 +39,11 @@
211 relation_get,
212 config as config_get,
213 INFO,
214- ERROR,
215+ DEBUG,
216 WARNING,
217 unit_get,
218- is_leader as juju_is_leader
219+ is_leader as juju_is_leader,
220+ status_set,
221 )
222 from charmhelpers.core.decorators import (
223 retry_on_exception,
224@@ -60,6 +59,10 @@
225 pass
226
227
228+class HAIncorrectConfig(Exception):
229+ pass
230+
231+
232 class CRMResourceNotFound(Exception):
233 pass
234
235@@ -274,27 +277,71 @@
236 Obtains all relevant configuration from charm configuration required
237 for initiating a relation to hacluster:
238
239- ha-bindiface, ha-mcastport, vip
240+ ha-bindiface, ha-mcastport, vip, os-internal-hostname,
241+ os-admin-hostname, os-public-hostname, os-access-hostname
242
243 param: exclude_keys: list of setting key(s) to be excluded.
244 returns: dict: A dict containing settings keyed by setting name.
245- raises: HAIncompleteConfig if settings are missing.
246+ raises: HAIncompleteConfig if settings are missing or incorrect.
247 '''
248- settings = ['ha-bindiface', 'ha-mcastport', 'vip']
249+ settings = ['ha-bindiface', 'ha-mcastport', 'vip', 'os-internal-hostname',
250+ 'os-admin-hostname', 'os-public-hostname', 'os-access-hostname']
251 conf = {}
252 for setting in settings:
253 if exclude_keys and setting in exclude_keys:
254 continue
255
256 conf[setting] = config_get(setting)
257- missing = []
258- [missing.append(s) for s, v in six.iteritems(conf) if v is None]
259- if missing:
260- log('Insufficient config data to configure hacluster.', level=ERROR)
261- raise HAIncompleteConfig
262+
263+ if not valid_hacluster_config():
264+ raise HAIncorrectConfig('Insufficient or incorrect config data to '
265+ 'configure hacluster.')
266 return conf
267
268
269+def valid_hacluster_config():
270+ '''
271+ Check that either vip or dns-ha is set. If dns-ha then one of os-*-hostname
272+ must be set.
273+
274+ Note: ha-bindiface and ha-macastport both have defaults and will always
275+ be set. We only care that either vip or dns-ha is set.
276+
277+ :returns: boolean: valid config returns true.
278+ raises: HAIncompatibileConfig if settings conflict.
279+ raises: HAIncompleteConfig if settings are missing.
280+ '''
281+ vip = config_get('vip')
282+ dns = config_get('dns-ha')
283+ if not(bool(vip) ^ bool(dns)):
284+ msg = ('HA: Either vip or dns-ha must be set but not both in order to '
285+ 'use high availability')
286+ status_set('blocked', msg)
287+ raise HAIncorrectConfig(msg)
288+
289+ # If dns-ha then one of os-*-hostname must be set
290+ if dns:
291+ dns_settings = ['os-internal-hostname', 'os-admin-hostname',
292+ 'os-public-hostname', 'os-access-hostname']
293+ # At this point it is unknown if one or all of the possible
294+ # network spaces are in HA. Validate at least one is set which is
295+ # the minimum required.
296+ for setting in dns_settings:
297+ if config_get(setting):
298+ log('DNS HA: At least one hostname is set {}: {}'
299+ ''.format(setting, config_get(setting)),
300+ level=DEBUG)
301+ return True
302+
303+ msg = ('DNS HA: At least one os-*-hostname(s) must be set to use '
304+ 'DNS HA')
305+ status_set('blocked', msg)
306+ raise HAIncompleteConfig(msg)
307+
308+ log('VIP HA: VIP is set {}'.format(vip), level=DEBUG)
309+ return True
310+
311+
312 def canonical_url(configs, vip_setting='vip'):
313 '''
314 Returns the correct HTTP URL to this host given the state of HTTPS
315
316=== modified file 'charmhelpers/core/__init__.py'
317--- charmhelpers/core/__init__.py 2015-01-28 08:59:02 +0000
318+++ charmhelpers/core/__init__.py 2017-03-04 02:50:20 +0000
319@@ -1,15 +1,13 @@
320 # Copyright 2014-2015 Canonical Limited.
321 #
322-# This file is part of charm-helpers.
323-#
324-# charm-helpers is free software: you can redistribute it and/or modify
325-# it under the terms of the GNU Lesser General Public License version 3 as
326-# published by the Free Software Foundation.
327-#
328-# charm-helpers is distributed in the hope that it will be useful,
329-# but WITHOUT ANY WARRANTY; without even the implied warranty of
330-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
331-# GNU Lesser General Public License for more details.
332-#
333-# You should have received a copy of the GNU Lesser General Public License
334-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
335+# Licensed under the Apache License, Version 2.0 (the "License");
336+# you may not use this file except in compliance with the License.
337+# You may obtain a copy of the License at
338+#
339+# http://www.apache.org/licenses/LICENSE-2.0
340+#
341+# Unless required by applicable law or agreed to in writing, software
342+# distributed under the License is distributed on an "AS IS" BASIS,
343+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
344+# See the License for the specific language governing permissions and
345+# limitations under the License.
346
347=== modified file 'charmhelpers/core/decorators.py'
348--- charmhelpers/core/decorators.py 2015-01-28 08:59:02 +0000
349+++ charmhelpers/core/decorators.py 2017-03-04 02:50:20 +0000
350@@ -1,18 +1,16 @@
351 # Copyright 2014-2015 Canonical Limited.
352 #
353-# This file is part of charm-helpers.
354-#
355-# charm-helpers is free software: you can redistribute it and/or modify
356-# it under the terms of the GNU Lesser General Public License version 3 as
357-# published by the Free Software Foundation.
358-#
359-# charm-helpers is distributed in the hope that it will be useful,
360-# but WITHOUT ANY WARRANTY; without even the implied warranty of
361-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
362-# GNU Lesser General Public License for more details.
363-#
364-# You should have received a copy of the GNU Lesser General Public License
365-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
366+# Licensed under the Apache License, Version 2.0 (the "License");
367+# you may not use this file except in compliance with the License.
368+# You may obtain a copy of the License at
369+#
370+# http://www.apache.org/licenses/LICENSE-2.0
371+#
372+# Unless required by applicable law or agreed to in writing, software
373+# distributed under the License is distributed on an "AS IS" BASIS,
374+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
375+# See the License for the specific language governing permissions and
376+# limitations under the License.
377
378 #
379 # Copyright 2014 Canonical Ltd.
380
381=== modified file 'charmhelpers/core/files.py'
382--- charmhelpers/core/files.py 2015-12-11 15:23:38 +0000
383+++ charmhelpers/core/files.py 2017-03-04 02:50:20 +0000
384@@ -3,19 +3,17 @@
385
386 # Copyright 2014-2015 Canonical Limited.
387 #
388-# This file is part of charm-helpers.
389-#
390-# charm-helpers is free software: you can redistribute it and/or modify
391-# it under the terms of the GNU Lesser General Public License version 3 as
392-# published by the Free Software Foundation.
393-#
394-# charm-helpers is distributed in the hope that it will be useful,
395-# but WITHOUT ANY WARRANTY; without even the implied warranty of
396-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
397-# GNU Lesser General Public License for more details.
398-#
399-# You should have received a copy of the GNU Lesser General Public License
400-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
401+# Licensed under the Apache License, Version 2.0 (the "License");
402+# you may not use this file except in compliance with the License.
403+# You may obtain a copy of the License at
404+#
405+# http://www.apache.org/licenses/LICENSE-2.0
406+#
407+# Unless required by applicable law or agreed to in writing, software
408+# distributed under the License is distributed on an "AS IS" BASIS,
409+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
410+# See the License for the specific language governing permissions and
411+# limitations under the License.
412
413 __author__ = 'Jorge Niedbalski <niedbalski@ubuntu.com>'
414
415
416=== modified file 'charmhelpers/core/fstab.py'
417--- charmhelpers/core/fstab.py 2015-03-12 11:42:26 +0000
418+++ charmhelpers/core/fstab.py 2017-03-04 02:50:20 +0000
419@@ -3,19 +3,17 @@
420
421 # Copyright 2014-2015 Canonical Limited.
422 #
423-# This file is part of charm-helpers.
424-#
425-# charm-helpers is free software: you can redistribute it and/or modify
426-# it under the terms of the GNU Lesser General Public License version 3 as
427-# published by the Free Software Foundation.
428-#
429-# charm-helpers is distributed in the hope that it will be useful,
430-# but WITHOUT ANY WARRANTY; without even the implied warranty of
431-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
432-# GNU Lesser General Public License for more details.
433-#
434-# You should have received a copy of the GNU Lesser General Public License
435-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
436+# Licensed under the Apache License, Version 2.0 (the "License");
437+# you may not use this file except in compliance with the License.
438+# You may obtain a copy of the License at
439+#
440+# http://www.apache.org/licenses/LICENSE-2.0
441+#
442+# Unless required by applicable law or agreed to in writing, software
443+# distributed under the License is distributed on an "AS IS" BASIS,
444+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
445+# See the License for the specific language governing permissions and
446+# limitations under the License.
447
448 import io
449 import os
450
451=== modified file 'charmhelpers/core/hookenv.py'
452--- charmhelpers/core/hookenv.py 2016-05-06 07:12:40 +0000
453+++ charmhelpers/core/hookenv.py 2017-03-04 02:50:20 +0000
454@@ -1,18 +1,16 @@
455 # Copyright 2014-2015 Canonical Limited.
456 #
457-# This file is part of charm-helpers.
458-#
459-# charm-helpers is free software: you can redistribute it and/or modify
460-# it under the terms of the GNU Lesser General Public License version 3 as
461-# published by the Free Software Foundation.
462-#
463-# charm-helpers is distributed in the hope that it will be useful,
464-# but WITHOUT ANY WARRANTY; without even the implied warranty of
465-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
466-# GNU Lesser General Public License for more details.
467-#
468-# You should have received a copy of the GNU Lesser General Public License
469-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
470+# Licensed under the Apache License, Version 2.0 (the "License");
471+# you may not use this file except in compliance with the License.
472+# You may obtain a copy of the License at
473+#
474+# http://www.apache.org/licenses/LICENSE-2.0
475+#
476+# Unless required by applicable law or agreed to in writing, software
477+# distributed under the License is distributed on an "AS IS" BASIS,
478+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
479+# See the License for the specific language governing permissions and
480+# limitations under the License.
481
482 "Interactions with the Juju environment"
483 # Copyright 2013 Canonical Ltd.
484@@ -334,6 +332,8 @@
485 config_cmd_line = ['config-get']
486 if scope is not None:
487 config_cmd_line.append(scope)
488+ else:
489+ config_cmd_line.append('--all')
490 config_cmd_line.append('--format=json')
491 try:
492 config_data = json.loads(
493@@ -616,6 +616,20 @@
494 subprocess.check_call(_args)
495
496
497+def open_ports(start, end, protocol="TCP"):
498+ """Opens a range of service network ports"""
499+ _args = ['open-port']
500+ _args.append('{}-{}/{}'.format(start, end, protocol))
501+ subprocess.check_call(_args)
502+
503+
504+def close_ports(start, end, protocol="TCP"):
505+ """Close a range of service network ports"""
506+ _args = ['close-port']
507+ _args.append('{}-{}/{}'.format(start, end, protocol))
508+ subprocess.check_call(_args)
509+
510+
511 @cached
512 def unit_get(attribute):
513 """Get the unit ID for the remote unit"""
514@@ -845,6 +859,20 @@
515 return inner_translate_exc1
516
517
518+def application_version_set(version):
519+ """Charm authors may trigger this command from any hook to output what
520+ version of the application is running. This could be a package version,
521+ for instance postgres version 9.5. It could also be a build number or
522+ version control revision identifier, for instance git sha 6fb7ba68. """
523+
524+ cmd = ['application-version-set']
525+ cmd.append(version)
526+ try:
527+ subprocess.check_call(cmd)
528+ except OSError:
529+ log("Application Version: {}".format(version))
530+
531+
532 @translate_exc(from_exc=OSError, to_exc=NotImplementedError)
533 def is_leader():
534 """Does the current unit hold the juju leadership
535@@ -1006,4 +1034,35 @@
536 :raise: NotImplementedError if run on Juju < 2.0
537 '''
538 cmd = ['network-get', '--primary-address', binding]
539- return subprocess.check_output(cmd).strip()
540+ return subprocess.check_output(cmd).decode('UTF-8').strip()
541+
542+
543+def add_metric(*args, **kwargs):
544+ """Add metric values. Values may be expressed with keyword arguments. For
545+ metric names containing dashes, these may be expressed as one or more
546+ 'key=value' positional arguments. May only be called from the collect-metrics
547+ hook."""
548+ _args = ['add-metric']
549+ _kvpairs = []
550+ _kvpairs.extend(args)
551+ _kvpairs.extend(['{}={}'.format(k, v) for k, v in kwargs.items()])
552+ _args.extend(sorted(_kvpairs))
553+ try:
554+ subprocess.check_call(_args)
555+ return
556+ except EnvironmentError as e:
557+ if e.errno != errno.ENOENT:
558+ raise
559+ log_message = 'add-metric failed: {}'.format(' '.join(_kvpairs))
560+ log(log_message, level='INFO')
561+
562+
563+def meter_status():
564+ """Get the meter status, if running in the meter-status-changed hook."""
565+ return os.environ.get('JUJU_METER_STATUS')
566+
567+
568+def meter_info():
569+ """Get the meter status information, if running in the meter-status-changed
570+ hook."""
571+ return os.environ.get('JUJU_METER_INFO')
572
573=== modified file 'charmhelpers/core/host.py'
574--- charmhelpers/core/host.py 2016-05-06 07:12:40 +0000
575+++ charmhelpers/core/host.py 2017-03-04 02:50:20 +0000
576@@ -1,18 +1,16 @@
577 # Copyright 2014-2015 Canonical Limited.
578 #
579-# This file is part of charm-helpers.
580-#
581-# charm-helpers is free software: you can redistribute it and/or modify
582-# it under the terms of the GNU Lesser General Public License version 3 as
583-# published by the Free Software Foundation.
584-#
585-# charm-helpers is distributed in the hope that it will be useful,
586-# but WITHOUT ANY WARRANTY; without even the implied warranty of
587-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
588-# GNU Lesser General Public License for more details.
589-#
590-# You should have received a copy of the GNU Lesser General Public License
591-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
592+# Licensed under the Apache License, Version 2.0 (the "License");
593+# you may not use this file except in compliance with the License.
594+# You may obtain a copy of the License at
595+#
596+# http://www.apache.org/licenses/LICENSE-2.0
597+#
598+# Unless required by applicable law or agreed to in writing, software
599+# distributed under the License is distributed on an "AS IS" BASIS,
600+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
601+# See the License for the specific language governing permissions and
602+# limitations under the License.
603
604 """Tools for working with the host system"""
605 # Copyright 2012 Canonical Ltd.
606@@ -32,46 +30,162 @@
607 import hashlib
608 import functools
609 import itertools
610+import six
611+
612 from contextlib import contextmanager
613 from collections import OrderedDict
614-
615-import six
616-
617 from .hookenv import log
618 from .fstab import Fstab
619-
620-
621-def service_start(service_name):
622- """Start a system service"""
623- return service('start', service_name)
624-
625-
626-def service_stop(service_name):
627- """Stop a system service"""
628- return service('stop', service_name)
629-
630-
631-def service_restart(service_name):
632- """Restart a system service"""
633+from charmhelpers.osplatform import get_platform
634+
635+__platform__ = get_platform()
636+if __platform__ == "ubuntu":
637+ from charmhelpers.core.host_factory.ubuntu import (
638+ service_available,
639+ add_new_group,
640+ lsb_release,
641+ cmp_pkgrevno,
642+ ) # flake8: noqa -- ignore F401 for this import
643+elif __platform__ == "centos":
644+ from charmhelpers.core.host_factory.centos import (
645+ service_available,
646+ add_new_group,
647+ lsb_release,
648+ cmp_pkgrevno,
649+ ) # flake8: noqa -- ignore F401 for this import
650+
651+UPDATEDB_PATH = '/etc/updatedb.conf'
652+
653+def service_start(service_name, **kwargs):
654+ """Start a system service.
655+
656+ The specified service name is managed via the system level init system.
657+ Some init systems (e.g. upstart) require that additional arguments be
658+ provided in order to directly control service instances whereas other init
659+ systems allow for addressing instances of a service directly by name (e.g.
660+ systemd).
661+
662+ The kwargs allow for the additional parameters to be passed to underlying
663+ init systems for those systems which require/allow for them. For example,
664+ the ceph-osd upstart script requires the id parameter to be passed along
665+ in order to identify which running daemon should be reloaded. The follow-
666+ ing example stops the ceph-osd service for instance id=4:
667+
668+ service_stop('ceph-osd', id=4)
669+
670+ :param service_name: the name of the service to stop
671+ :param **kwargs: additional parameters to pass to the init system when
672+ managing services. These will be passed as key=value
673+ parameters to the init system's commandline. kwargs
674+ are ignored for systemd enabled systems.
675+ """
676+ return service('start', service_name, **kwargs)
677+
678+
679+def service_stop(service_name, **kwargs):
680+ """Stop a system service.
681+
682+ The specified service name is managed via the system level init system.
683+ Some init systems (e.g. upstart) require that additional arguments be
684+ provided in order to directly control service instances whereas other init
685+ systems allow for addressing instances of a service directly by name (e.g.
686+ systemd).
687+
688+ The kwargs allow for the additional parameters to be passed to underlying
689+ init systems for those systems which require/allow for them. For example,
690+ the ceph-osd upstart script requires the id parameter to be passed along
691+ in order to identify which running daemon should be reloaded. The follow-
692+ ing example stops the ceph-osd service for instance id=4:
693+
694+ service_stop('ceph-osd', id=4)
695+
696+ :param service_name: the name of the service to stop
697+ :param **kwargs: additional parameters to pass to the init system when
698+ managing services. These will be passed as key=value
699+ parameters to the init system's commandline. kwargs
700+ are ignored for systemd enabled systems.
701+ """
702+ return service('stop', service_name, **kwargs)
703+
704+
705+def service_restart(service_name, **kwargs):
706+ """Restart a system service.
707+
708+ The specified service name is managed via the system level init system.
709+ Some init systems (e.g. upstart) require that additional arguments be
710+ provided in order to directly control service instances whereas other init
711+ systems allow for addressing instances of a service directly by name (e.g.
712+ systemd).
713+
714+ The kwargs allow for the additional parameters to be passed to underlying
715+ init systems for those systems which require/allow for them. For example,
716+ the ceph-osd upstart script requires the id parameter to be passed along
717+ in order to identify which running daemon should be restarted. The follow-
718+ ing example restarts the ceph-osd service for instance id=4:
719+
720+ service_restart('ceph-osd', id=4)
721+
722+ :param service_name: the name of the service to restart
723+ :param **kwargs: additional parameters to pass to the init system when
724+ managing services. These will be passed as key=value
725+ parameters to the init system's commandline. kwargs
726+ are ignored for init systems not allowing additional
727+ parameters via the commandline (systemd).
728+ """
729 return service('restart', service_name)
730
731
732-def service_reload(service_name, restart_on_failure=False):
733+def service_reload(service_name, restart_on_failure=False, **kwargs):
734 """Reload a system service, optionally falling back to restart if
735- reload fails"""
736- service_result = service('reload', service_name)
737+ reload fails.
738+
739+ The specified service name is managed via the system level init system.
740+ Some init systems (e.g. upstart) require that additional arguments be
741+ provided in order to directly control service instances whereas other init
742+ systems allow for addressing instances of a service directly by name (e.g.
743+ systemd).
744+
745+ The kwargs allow for the additional parameters to be passed to underlying
746+ init systems for those systems which require/allow for them. For example,
747+ the ceph-osd upstart script requires the id parameter to be passed along
748+ in order to identify which running daemon should be reloaded. The follow-
749+ ing example restarts the ceph-osd service for instance id=4:
750+
751+ service_reload('ceph-osd', id=4)
752+
753+ :param service_name: the name of the service to reload
754+ :param restart_on_failure: boolean indicating whether to fallback to a
755+ restart if the reload fails.
756+ :param **kwargs: additional parameters to pass to the init system when
757+ managing services. These will be passed as key=value
758+ parameters to the init system's commandline. kwargs
759+ are ignored for init systems not allowing additional
760+ parameters via the commandline (systemd).
761+ """
762+ service_result = service('reload', service_name, **kwargs)
763 if not service_result and restart_on_failure:
764- service_result = service('restart', service_name)
765+ service_result = service('restart', service_name, **kwargs)
766 return service_result
767
768
769-def service_pause(service_name, init_dir="/etc/init", initd_dir="/etc/init.d"):
770+def service_pause(service_name, init_dir="/etc/init", initd_dir="/etc/init.d",
771+ **kwargs):
772 """Pause a system service.
773
774- Stop it, and prevent it from starting again at boot."""
775+ Stop it, and prevent it from starting again at boot.
776+
777+ :param service_name: the name of the service to pause
778+ :param init_dir: path to the upstart init directory
779+ :param initd_dir: path to the sysv init directory
780+ :param **kwargs: additional parameters to pass to the init system when
781+ managing services. These will be passed as key=value
782+ parameters to the init system's commandline. kwargs
783+ are ignored for init systems which do not support
784+ key=value arguments via the commandline.
785+ """
786 stopped = True
787- if service_running(service_name):
788- stopped = service_stop(service_name)
789+ if service_running(service_name, **kwargs):
790+ stopped = service_stop(service_name, **kwargs)
791 upstart_file = os.path.join(init_dir, "{}.conf".format(service_name))
792 sysv_file = os.path.join(initd_dir, service_name)
793 if init_is_systemd():
794@@ -92,10 +206,19 @@
795
796
797 def service_resume(service_name, init_dir="/etc/init",
798- initd_dir="/etc/init.d"):
799+ initd_dir="/etc/init.d", **kwargs):
800 """Resume a system service.
801
802- Reenable starting again at boot. Start the service"""
803+ Reenable starting again at boot. Start the service.
804+
805+ :param service_name: the name of the service to resume
806+ :param init_dir: the path to the init dir
807+ :param initd dir: the path to the initd dir
808+ :param **kwargs: additional parameters to pass to the init system when
809+ managing services. These will be passed as key=value
810+ parameters to the init system's commandline. kwargs
811+ are ignored for systemd enabled systems.
812+ """
813 upstart_file = os.path.join(init_dir, "{}.conf".format(service_name))
814 sysv_file = os.path.join(initd_dir, service_name)
815 if init_is_systemd():
816@@ -112,62 +235,70 @@
817 "Unable to detect {0} as SystemD, Upstart {1} or"
818 " SysV {2}".format(
819 service_name, upstart_file, sysv_file))
820+ started = service_running(service_name, **kwargs)
821
822- started = service_running(service_name)
823 if not started:
824- started = service_start(service_name)
825+ started = service_start(service_name, **kwargs)
826 return started
827
828
829-def service(action, service_name):
830- """Control a system service"""
831+def service(action, service_name, **kwargs):
832+ """Control a system service.
833+
834+ :param action: the action to take on the service
835+ :param service_name: the name of the service to perform th action on
836+ :param **kwargs: additional params to be passed to the service command in
837+ the form of key=value.
838+ """
839 if init_is_systemd():
840 cmd = ['systemctl', action, service_name]
841 else:
842 cmd = ['service', service_name, action]
843+ for key, value in six.iteritems(kwargs):
844+ parameter = '%s=%s' % (key, value)
845+ cmd.append(parameter)
846 return subprocess.call(cmd) == 0
847
848
849-def systemv_services_running():
850- output = subprocess.check_output(
851- ['service', '--status-all'],
852- stderr=subprocess.STDOUT).decode('UTF-8')
853- return [row.split()[-1] for row in output.split('\n') if '[ + ]' in row]
854-
855-
856-def service_running(service_name):
857- """Determine whether a system service is running"""
858+_UPSTART_CONF = "/etc/init/{}.conf"
859+_INIT_D_CONF = "/etc/init.d/{}"
860+
861+
862+def service_running(service_name, **kwargs):
863+ """Determine whether a system service is running.
864+
865+ :param service_name: the name of the service
866+ :param **kwargs: additional args to pass to the service command. This is
867+ used to pass additional key=value arguments to the
868+ service command line for managing specific instance
869+ units (e.g. service ceph-osd status id=2). The kwargs
870+ are ignored in systemd services.
871+ """
872 if init_is_systemd():
873 return service('is-active', service_name)
874 else:
875- try:
876- output = subprocess.check_output(
877- ['service', service_name, 'status'],
878- stderr=subprocess.STDOUT).decode('UTF-8')
879- except subprocess.CalledProcessError:
880- return False
881- else:
882- # This works for upstart scripts where the 'service' command
883- # returns a consistent string to represent running 'start/running'
884- if ("start/running" in output or "is running" in output or
885- "up and running" in output):
886- return True
887+ if os.path.exists(_UPSTART_CONF.format(service_name)):
888+ try:
889+ cmd = ['status', service_name]
890+ for key, value in six.iteritems(kwargs):
891+ parameter = '%s=%s' % (key, value)
892+ cmd.append(parameter)
893+ output = subprocess.check_output(cmd,
894+ stderr=subprocess.STDOUT).decode('UTF-8')
895+ except subprocess.CalledProcessError:
896+ return False
897+ else:
898+ # This works for upstart scripts where the 'service' command
899+ # returns a consistent string to represent running
900+ # 'start/running'
901+ if ("start/running" in output or
902+ "is running" in output or
903+ "up and running" in output):
904+ return True
905+ elif os.path.exists(_INIT_D_CONF.format(service_name)):
906 # Check System V scripts init script return codes
907- if service_name in systemv_services_running():
908- return True
909- return False
910-
911-
912-def service_available(service_name):
913- """Determine whether a system service is available"""
914- try:
915- subprocess.check_output(
916- ['service', service_name, 'status'],
917- stderr=subprocess.STDOUT).decode('UTF-8')
918- except subprocess.CalledProcessError as e:
919- return b'unrecognized service' not in e.output
920- else:
921- return True
922+ return service('status', service_name)
923+ return False
924
925
926 SYSTEMD_SYSTEM = '/run/systemd/system'
927@@ -178,8 +309,9 @@
928 return os.path.isdir(SYSTEMD_SYSTEM)
929
930
931-def adduser(username, password=None, shell='/bin/bash', system_user=False,
932- primary_group=None, secondary_groups=None):
933+def adduser(username, password=None, shell='/bin/bash',
934+ system_user=False, primary_group=None,
935+ secondary_groups=None, uid=None, home_dir=None):
936 """Add a user to the system.
937
938 Will log but otherwise succeed if the user already exists.
939@@ -190,15 +322,24 @@
940 :param bool system_user: Whether to create a login or system user
941 :param str primary_group: Primary group for user; defaults to username
942 :param list secondary_groups: Optional list of additional groups
943+ :param int uid: UID for user being created
944+ :param str home_dir: Home directory for user
945
946 :returns: The password database entry struct, as returned by `pwd.getpwnam`
947 """
948 try:
949 user_info = pwd.getpwnam(username)
950 log('user {0} already exists!'.format(username))
951+ if uid:
952+ user_info = pwd.getpwuid(int(uid))
953+ log('user with uid {0} already exists!'.format(uid))
954 except KeyError:
955 log('creating user {0}'.format(username))
956 cmd = ['useradd']
957+ if uid:
958+ cmd.extend(['--uid', str(uid)])
959+ if home_dir:
960+ cmd.extend(['--home', str(home_dir)])
961 if system_user or password is None:
962 cmd.append('--system')
963 else:
964@@ -233,22 +374,56 @@
965 return user_exists
966
967
968-def add_group(group_name, system_group=False):
969- """Add a group to the system"""
970+def uid_exists(uid):
971+ """Check if a uid exists"""
972+ try:
973+ pwd.getpwuid(uid)
974+ uid_exists = True
975+ except KeyError:
976+ uid_exists = False
977+ return uid_exists
978+
979+
980+def group_exists(groupname):
981+ """Check if a group exists"""
982+ try:
983+ grp.getgrnam(groupname)
984+ group_exists = True
985+ except KeyError:
986+ group_exists = False
987+ return group_exists
988+
989+
990+def gid_exists(gid):
991+ """Check if a gid exists"""
992+ try:
993+ grp.getgrgid(gid)
994+ gid_exists = True
995+ except KeyError:
996+ gid_exists = False
997+ return gid_exists
998+
999+
1000+def add_group(group_name, system_group=False, gid=None):
1001+ """Add a group to the system
1002+
1003+ Will log but otherwise succeed if the group already exists.
1004+
1005+ :param str group_name: group to create
1006+ :param bool system_group: Create system group
1007+ :param int gid: GID for user being created
1008+
1009+ :returns: The password database entry struct, as returned by `grp.getgrnam`
1010+ """
1011 try:
1012 group_info = grp.getgrnam(group_name)
1013 log('group {0} already exists!'.format(group_name))
1014+ if gid:
1015+ group_info = grp.getgrgid(gid)
1016+ log('group with gid {0} already exists!'.format(gid))
1017 except KeyError:
1018 log('creating group {0}'.format(group_name))
1019- cmd = ['addgroup']
1020- if system_group:
1021- cmd.append('--system')
1022- else:
1023- cmd.extend([
1024- '--group',
1025- ])
1026- cmd.append(group_name)
1027- subprocess.check_call(cmd)
1028+ add_new_group(group_name, system_group, gid)
1029 group_info = grp.getgrnam(group_name)
1030 return group_info
1031
1032@@ -260,15 +435,17 @@
1033 subprocess.check_call(cmd)
1034
1035
1036-def rsync(from_path, to_path, flags='-r', options=None):
1037+def rsync(from_path, to_path, flags='-r', options=None, timeout=None):
1038 """Replicate the contents of a path"""
1039 options = options or ['--delete', '--executability']
1040 cmd = ['/usr/bin/rsync', flags]
1041+ if timeout:
1042+ cmd = ['timeout', str(timeout)] + cmd
1043 cmd.extend(options)
1044 cmd.append(from_path)
1045 cmd.append(to_path)
1046 log(" ".join(cmd))
1047- return subprocess.check_output(cmd).decode('UTF-8').strip()
1048+ return subprocess.check_output(cmd, stderr=subprocess.STDOUT).decode('UTF-8').strip()
1049
1050
1051 def symlink(source, destination):
1052@@ -493,16 +670,6 @@
1053 return r
1054
1055
1056-def lsb_release():
1057- """Return /etc/lsb-release in a dict"""
1058- d = {}
1059- with open('/etc/lsb-release', 'r') as lsb:
1060- for l in lsb:
1061- k, v = l.split('=')
1062- d[k.strip()] = v.strip()
1063- return d
1064-
1065-
1066 def pwgen(length=None):
1067 """Generate a random pasword."""
1068 if length is None:
1069@@ -626,25 +793,6 @@
1070 return hwaddr
1071
1072
1073-def cmp_pkgrevno(package, revno, pkgcache=None):
1074- """Compare supplied revno with the revno of the installed package
1075-
1076- * 1 => Installed revno is greater than supplied arg
1077- * 0 => Installed revno is the same as supplied arg
1078- * -1 => Installed revno is less than supplied arg
1079-
1080- This function imports apt_cache function from charmhelpers.fetch if
1081- the pkgcache argument is None. Be sure to add charmhelpers.fetch if
1082- you call this function, or pass an apt_pkg.Cache() instance.
1083- """
1084- import apt_pkg
1085- if not pkgcache:
1086- from charmhelpers.fetch import apt_cache
1087- pkgcache = apt_cache()
1088- pkg = pkgcache[package]
1089- return apt_pkg.version_compare(pkg.current_ver.ver_str, revno)
1090-
1091-
1092 @contextmanager
1093 def chdir(directory):
1094 """Change the current working directory to a different directory for a code
1095@@ -667,7 +815,7 @@
1096 :param str path: The string path to start changing ownership.
1097 :param str owner: The owner string to use when looking up the uid.
1098 :param str group: The group string to use when looking up the gid.
1099- :param bool follow_links: Also Chown links if True
1100+ :param bool follow_links: Also follow and chown links if True
1101 :param bool chowntopdir: Also chown path itself if True
1102 """
1103 uid = pwd.getpwnam(owner).pw_uid
1104@@ -681,7 +829,7 @@
1105 broken_symlink = os.path.lexists(path) and not os.path.exists(path)
1106 if not broken_symlink:
1107 chown(path, uid, gid)
1108- for root, dirs, files in os.walk(path):
1109+ for root, dirs, files in os.walk(path, followlinks=follow_links):
1110 for name in dirs + files:
1111 full = os.path.join(root, name)
1112 broken_symlink = os.path.lexists(full) and not os.path.exists(full)
1113@@ -701,6 +849,20 @@
1114 chownr(path, owner, group, follow_links=False)
1115
1116
1117+def owner(path):
1118+ """Returns a tuple containing the username & groupname owning the path.
1119+
1120+ :param str path: the string path to retrieve the ownership
1121+ :return tuple(str, str): A (username, groupname) tuple containing the
1122+ name of the user and group owning the path.
1123+ :raises OSError: if the specified path does not exist
1124+ """
1125+ stat = os.stat(path)
1126+ username = pwd.getpwuid(stat.st_uid)[0]
1127+ groupname = grp.getgrgid(stat.st_gid)[0]
1128+ return username, groupname
1129+
1130+
1131 def get_total_ram():
1132 """The total amount of system RAM in bytes.
1133
1134@@ -715,3 +877,42 @@
1135 assert unit == 'kB', 'Unknown unit'
1136 return int(value) * 1024 # Classic, not KiB.
1137 raise NotImplementedError()
1138+
1139+
1140+UPSTART_CONTAINER_TYPE = '/run/container_type'
1141+
1142+
1143+def is_container():
1144+ """Determine whether unit is running in a container
1145+
1146+ @return: boolean indicating if unit is in a container
1147+ """
1148+ if init_is_systemd():
1149+ # Detect using systemd-detect-virt
1150+ return subprocess.call(['systemd-detect-virt',
1151+ '--container']) == 0
1152+ else:
1153+ # Detect using upstart container file marker
1154+ return os.path.exists(UPSTART_CONTAINER_TYPE)
1155+
1156+
1157+def add_to_updatedb_prunepath(path, updatedb_path=UPDATEDB_PATH):
1158+ with open(updatedb_path, 'r+') as f_id:
1159+ updatedb_text = f_id.read()
1160+ output = updatedb(updatedb_text, path)
1161+ f_id.seek(0)
1162+ f_id.write(output)
1163+ f_id.truncate()
1164+
1165+
1166+def updatedb(updatedb_text, new_path):
1167+ lines = [line for line in updatedb_text.split("\n")]
1168+ for i, line in enumerate(lines):
1169+ if line.startswith("PRUNEPATHS="):
1170+ paths_line = line.split("=")[1].replace('"', '')
1171+ paths = paths_line.split(" ")
1172+ if new_path not in paths:
1173+ paths.append(new_path)
1174+ lines[i] = 'PRUNEPATHS="{}"'.format(' '.join(paths))
1175+ output = "\n".join(lines)
1176+ return output
1177
1178=== added directory 'charmhelpers/core/host_factory'
1179=== added file 'charmhelpers/core/host_factory/__init__.py'
1180=== added file 'charmhelpers/core/host_factory/centos.py'
1181--- charmhelpers/core/host_factory/centos.py 1970-01-01 00:00:00 +0000
1182+++ charmhelpers/core/host_factory/centos.py 2017-03-04 02:50:20 +0000
1183@@ -0,0 +1,56 @@
1184+import subprocess
1185+import yum
1186+import os
1187+
1188+
1189+def service_available(service_name):
1190+ # """Determine whether a system service is available."""
1191+ if os.path.isdir('/run/systemd/system'):
1192+ cmd = ['systemctl', 'is-enabled', service_name]
1193+ else:
1194+ cmd = ['service', service_name, 'is-enabled']
1195+ return subprocess.call(cmd) == 0
1196+
1197+
1198+def add_new_group(group_name, system_group=False, gid=None):
1199+ cmd = ['groupadd']
1200+ if gid:
1201+ cmd.extend(['--gid', str(gid)])
1202+ if system_group:
1203+ cmd.append('-r')
1204+ cmd.append(group_name)
1205+ subprocess.check_call(cmd)
1206+
1207+
1208+def lsb_release():
1209+ """Return /etc/os-release in a dict."""
1210+ d = {}
1211+ with open('/etc/os-release', 'r') as lsb:
1212+ for l in lsb:
1213+ s = l.split('=')
1214+ if len(s) != 2:
1215+ continue
1216+ d[s[0].strip()] = s[1].strip()
1217+ return d
1218+
1219+
1220+def cmp_pkgrevno(package, revno, pkgcache=None):
1221+ """Compare supplied revno with the revno of the installed package.
1222+
1223+ * 1 => Installed revno is greater than supplied arg
1224+ * 0 => Installed revno is the same as supplied arg
1225+ * -1 => Installed revno is less than supplied arg
1226+
1227+ This function imports YumBase function if the pkgcache argument
1228+ is None.
1229+ """
1230+ if not pkgcache:
1231+ y = yum.YumBase()
1232+ packages = y.doPackageLists()
1233+ pkgcache = {i.Name: i.version for i in packages['installed']}
1234+ pkg = pkgcache[package]
1235+ if pkg > revno:
1236+ return 1
1237+ if pkg < revno:
1238+ return -1
1239+ return 0
1240
1241=== added file 'charmhelpers/core/host_factory/ubuntu.py'
1242--- charmhelpers/core/host_factory/ubuntu.py 1970-01-01 00:00:00 +0000
1243+++ charmhelpers/core/host_factory/ubuntu.py 2017-03-04 02:50:20 +0000
1244@@ -0,0 +1,56 @@
1245+import subprocess
1246+
1247+
1248+def service_available(service_name):
1249+ """Determine whether a system service is available"""
1250+ try:
1251+ subprocess.check_output(
1252+ ['service', service_name, 'status'],
1253+ stderr=subprocess.STDOUT).decode('UTF-8')
1254+ except subprocess.CalledProcessError as e:
1255+ return b'unrecognized service' not in e.output
1256+ else:
1257+ return True
1258+
1259+
1260+def add_new_group(group_name, system_group=False, gid=None):
1261+ cmd = ['addgroup']
1262+ if gid:
1263+ cmd.extend(['--gid', str(gid)])
1264+ if system_group:
1265+ cmd.append('--system')
1266+ else:
1267+ cmd.extend([
1268+ '--group',
1269+ ])
1270+ cmd.append(group_name)
1271+ subprocess.check_call(cmd)
1272+
1273+
1274+def lsb_release():
1275+ """Return /etc/lsb-release in a dict"""
1276+ d = {}
1277+ with open('/etc/lsb-release', 'r') as lsb:
1278+ for l in lsb:
1279+ k, v = l.split('=')
1280+ d[k.strip()] = v.strip()
1281+ return d
1282+
1283+
1284+def cmp_pkgrevno(package, revno, pkgcache=None):
1285+ """Compare supplied revno with the revno of the installed package.
1286+
1287+ * 1 => Installed revno is greater than supplied arg
1288+ * 0 => Installed revno is the same as supplied arg
1289+ * -1 => Installed revno is less than supplied arg
1290+
1291+ This function imports apt_cache function from charmhelpers.fetch if
1292+ the pkgcache argument is None. Be sure to add charmhelpers.fetch if
1293+ you call this function, or pass an apt_pkg.Cache() instance.
1294+ """
1295+ import apt_pkg
1296+ if not pkgcache:
1297+ from charmhelpers.fetch import apt_cache
1298+ pkgcache = apt_cache()
1299+ pkg = pkgcache[package]
1300+ return apt_pkg.version_compare(pkg.current_ver.ver_str, revno)
1301
1302=== modified file 'charmhelpers/core/hugepage.py'
1303--- charmhelpers/core/hugepage.py 2015-12-11 15:23:38 +0000
1304+++ charmhelpers/core/hugepage.py 2017-03-04 02:50:20 +0000
1305@@ -2,19 +2,17 @@
1306
1307 # Copyright 2014-2015 Canonical Limited.
1308 #
1309-# This file is part of charm-helpers.
1310-#
1311-# charm-helpers is free software: you can redistribute it and/or modify
1312-# it under the terms of the GNU Lesser General Public License version 3 as
1313-# published by the Free Software Foundation.
1314-#
1315-# charm-helpers is distributed in the hope that it will be useful,
1316-# but WITHOUT ANY WARRANTY; without even the implied warranty of
1317-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1318-# GNU Lesser General Public License for more details.
1319-#
1320-# You should have received a copy of the GNU Lesser General Public License
1321-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1322+# Licensed under the Apache License, Version 2.0 (the "License");
1323+# you may not use this file except in compliance with the License.
1324+# You may obtain a copy of the License at
1325+#
1326+# http://www.apache.org/licenses/LICENSE-2.0
1327+#
1328+# Unless required by applicable law or agreed to in writing, software
1329+# distributed under the License is distributed on an "AS IS" BASIS,
1330+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
1331+# See the License for the specific language governing permissions and
1332+# limitations under the License.
1333
1334 import yaml
1335 from charmhelpers.core import fstab
1336
1337=== modified file 'charmhelpers/core/kernel.py'
1338--- charmhelpers/core/kernel.py 2015-12-11 15:23:38 +0000
1339+++ charmhelpers/core/kernel.py 2017-03-04 02:50:20 +0000
1340@@ -3,29 +3,40 @@
1341
1342 # Copyright 2014-2015 Canonical Limited.
1343 #
1344-# This file is part of charm-helpers.
1345-#
1346-# charm-helpers is free software: you can redistribute it and/or modify
1347-# it under the terms of the GNU Lesser General Public License version 3 as
1348-# published by the Free Software Foundation.
1349-#
1350-# charm-helpers is distributed in the hope that it will be useful,
1351-# but WITHOUT ANY WARRANTY; without even the implied warranty of
1352-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1353-# GNU Lesser General Public License for more details.
1354-#
1355-# You should have received a copy of the GNU Lesser General Public License
1356-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1357-
1358-__author__ = "Jorge Niedbalski <jorge.niedbalski@canonical.com>"
1359-
1360+# Licensed under the Apache License, Version 2.0 (the "License");
1361+# you may not use this file except in compliance with the License.
1362+# You may obtain a copy of the License at
1363+#
1364+# http://www.apache.org/licenses/LICENSE-2.0
1365+#
1366+# Unless required by applicable law or agreed to in writing, software
1367+# distributed under the License is distributed on an "AS IS" BASIS,
1368+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
1369+# See the License for the specific language governing permissions and
1370+# limitations under the License.
1371+
1372+import re
1373+import subprocess
1374+
1375+from charmhelpers.osplatform import get_platform
1376 from charmhelpers.core.hookenv import (
1377 log,
1378 INFO
1379 )
1380
1381-from subprocess import check_call, check_output
1382-import re
1383+__platform__ = get_platform()
1384+if __platform__ == "ubuntu":
1385+ from charmhelpers.core.kernel_factory.ubuntu import (
1386+ persistent_modprobe,
1387+ update_initramfs,
1388+ ) # flake8: noqa -- ignore F401 for this import
1389+elif __platform__ == "centos":
1390+ from charmhelpers.core.kernel_factory.centos import (
1391+ persistent_modprobe,
1392+ update_initramfs,
1393+ ) # flake8: noqa -- ignore F401 for this import
1394+
1395+__author__ = "Jorge Niedbalski <jorge.niedbalski@canonical.com>"
1396
1397
1398 def modprobe(module, persist=True):
1399@@ -34,11 +45,9 @@
1400
1401 log('Loading kernel module %s' % module, level=INFO)
1402
1403- check_call(cmd)
1404+ subprocess.check_call(cmd)
1405 if persist:
1406- with open('/etc/modules', 'r+') as modules:
1407- if module not in modules.read():
1408- modules.write(module)
1409+ persistent_modprobe(module)
1410
1411
1412 def rmmod(module, force=False):
1413@@ -48,21 +57,16 @@
1414 cmd.append('-f')
1415 cmd.append(module)
1416 log('Removing kernel module %s' % module, level=INFO)
1417- return check_call(cmd)
1418+ return subprocess.check_call(cmd)
1419
1420
1421 def lsmod():
1422 """Shows what kernel modules are currently loaded"""
1423- return check_output(['lsmod'],
1424- universal_newlines=True)
1425+ return subprocess.check_output(['lsmod'],
1426+ universal_newlines=True)
1427
1428
1429 def is_module_loaded(module):
1430 """Checks if a kernel module is already loaded"""
1431 matches = re.findall('^%s[ ]+' % module, lsmod(), re.M)
1432 return len(matches) > 0
1433-
1434-
1435-def update_initramfs(version='all'):
1436- """Updates an initramfs image"""
1437- return check_call(["update-initramfs", "-k", version, "-u"])
1438
1439=== added directory 'charmhelpers/core/kernel_factory'
1440=== added file 'charmhelpers/core/kernel_factory/__init__.py'
1441=== added file 'charmhelpers/core/kernel_factory/centos.py'
1442--- charmhelpers/core/kernel_factory/centos.py 1970-01-01 00:00:00 +0000
1443+++ charmhelpers/core/kernel_factory/centos.py 2017-03-04 02:50:20 +0000
1444@@ -0,0 +1,17 @@
1445+import subprocess
1446+import os
1447+
1448+
1449+def persistent_modprobe(module):
1450+ """Load a kernel module and configure for auto-load on reboot."""
1451+ if not os.path.exists('/etc/rc.modules'):
1452+ open('/etc/rc.modules', 'a')
1453+ os.chmod('/etc/rc.modules', 111)
1454+ with open('/etc/rc.modules', 'r+') as modules:
1455+ if module not in modules.read():
1456+ modules.write('modprobe %s\n' % module)
1457+
1458+
1459+def update_initramfs(version='all'):
1460+ """Updates an initramfs image."""
1461+ return subprocess.check_call(["dracut", "-f", version])
1462
1463=== added file 'charmhelpers/core/kernel_factory/ubuntu.py'
1464--- charmhelpers/core/kernel_factory/ubuntu.py 1970-01-01 00:00:00 +0000
1465+++ charmhelpers/core/kernel_factory/ubuntu.py 2017-03-04 02:50:20 +0000
1466@@ -0,0 +1,13 @@
1467+import subprocess
1468+
1469+
1470+def persistent_modprobe(module):
1471+ """Load a kernel module and configure for auto-load on reboot."""
1472+ with open('/etc/modules', 'r+') as modules:
1473+ if module not in modules.read():
1474+ modules.write(module + "\n")
1475+
1476+
1477+def update_initramfs(version='all'):
1478+ """Updates an initramfs image."""
1479+ return subprocess.check_call(["update-initramfs", "-k", version, "-u"])
1480
1481=== modified file 'charmhelpers/core/services/__init__.py'
1482--- charmhelpers/core/services/__init__.py 2015-01-28 08:59:02 +0000
1483+++ charmhelpers/core/services/__init__.py 2017-03-04 02:50:20 +0000
1484@@ -1,18 +1,16 @@
1485 # Copyright 2014-2015 Canonical Limited.
1486 #
1487-# This file is part of charm-helpers.
1488-#
1489-# charm-helpers is free software: you can redistribute it and/or modify
1490-# it under the terms of the GNU Lesser General Public License version 3 as
1491-# published by the Free Software Foundation.
1492-#
1493-# charm-helpers is distributed in the hope that it will be useful,
1494-# but WITHOUT ANY WARRANTY; without even the implied warranty of
1495-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1496-# GNU Lesser General Public License for more details.
1497-#
1498-# You should have received a copy of the GNU Lesser General Public License
1499-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1500+# Licensed under the Apache License, Version 2.0 (the "License");
1501+# you may not use this file except in compliance with the License.
1502+# You may obtain a copy of the License at
1503+#
1504+# http://www.apache.org/licenses/LICENSE-2.0
1505+#
1506+# Unless required by applicable law or agreed to in writing, software
1507+# distributed under the License is distributed on an "AS IS" BASIS,
1508+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
1509+# See the License for the specific language governing permissions and
1510+# limitations under the License.
1511
1512 from .base import * # NOQA
1513 from .helpers import * # NOQA
1514
1515=== modified file 'charmhelpers/core/services/base.py'
1516--- charmhelpers/core/services/base.py 2015-07-03 09:13:26 +0000
1517+++ charmhelpers/core/services/base.py 2017-03-04 02:50:20 +0000
1518@@ -1,18 +1,16 @@
1519 # Copyright 2014-2015 Canonical Limited.
1520 #
1521-# This file is part of charm-helpers.
1522-#
1523-# charm-helpers is free software: you can redistribute it and/or modify
1524-# it under the terms of the GNU Lesser General Public License version 3 as
1525-# published by the Free Software Foundation.
1526-#
1527-# charm-helpers is distributed in the hope that it will be useful,
1528-# but WITHOUT ANY WARRANTY; without even the implied warranty of
1529-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1530-# GNU Lesser General Public License for more details.
1531-#
1532-# You should have received a copy of the GNU Lesser General Public License
1533-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1534+# Licensed under the Apache License, Version 2.0 (the "License");
1535+# you may not use this file except in compliance with the License.
1536+# You may obtain a copy of the License at
1537+#
1538+# http://www.apache.org/licenses/LICENSE-2.0
1539+#
1540+# Unless required by applicable law or agreed to in writing, software
1541+# distributed under the License is distributed on an "AS IS" BASIS,
1542+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
1543+# See the License for the specific language governing permissions and
1544+# limitations under the License.
1545
1546 import os
1547 import json
1548
1549=== modified file 'charmhelpers/core/services/helpers.py'
1550--- charmhelpers/core/services/helpers.py 2015-12-11 15:23:38 +0000
1551+++ charmhelpers/core/services/helpers.py 2017-03-04 02:50:20 +0000
1552@@ -1,18 +1,16 @@
1553 # Copyright 2014-2015 Canonical Limited.
1554 #
1555-# This file is part of charm-helpers.
1556-#
1557-# charm-helpers is free software: you can redistribute it and/or modify
1558-# it under the terms of the GNU Lesser General Public License version 3 as
1559-# published by the Free Software Foundation.
1560-#
1561-# charm-helpers is distributed in the hope that it will be useful,
1562-# but WITHOUT ANY WARRANTY; without even the implied warranty of
1563-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1564-# GNU Lesser General Public License for more details.
1565-#
1566-# You should have received a copy of the GNU Lesser General Public License
1567-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1568+# Licensed under the Apache License, Version 2.0 (the "License");
1569+# you may not use this file except in compliance with the License.
1570+# You may obtain a copy of the License at
1571+#
1572+# http://www.apache.org/licenses/LICENSE-2.0
1573+#
1574+# Unless required by applicable law or agreed to in writing, software
1575+# distributed under the License is distributed on an "AS IS" BASIS,
1576+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
1577+# See the License for the specific language governing permissions and
1578+# limitations under the License.
1579
1580 import os
1581 import yaml
1582
1583=== modified file 'charmhelpers/core/strutils.py'
1584--- charmhelpers/core/strutils.py 2015-12-11 15:23:38 +0000
1585+++ charmhelpers/core/strutils.py 2017-03-04 02:50:20 +0000
1586@@ -3,19 +3,17 @@
1587
1588 # Copyright 2014-2015 Canonical Limited.
1589 #
1590-# This file is part of charm-helpers.
1591-#
1592-# charm-helpers is free software: you can redistribute it and/or modify
1593-# it under the terms of the GNU Lesser General Public License version 3 as
1594-# published by the Free Software Foundation.
1595-#
1596-# charm-helpers is distributed in the hope that it will be useful,
1597-# but WITHOUT ANY WARRANTY; without even the implied warranty of
1598-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1599-# GNU Lesser General Public License for more details.
1600-#
1601-# You should have received a copy of the GNU Lesser General Public License
1602-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1603+# Licensed under the Apache License, Version 2.0 (the "License");
1604+# you may not use this file except in compliance with the License.
1605+# You may obtain a copy of the License at
1606+#
1607+# http://www.apache.org/licenses/LICENSE-2.0
1608+#
1609+# Unless required by applicable law or agreed to in writing, software
1610+# distributed under the License is distributed on an "AS IS" BASIS,
1611+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
1612+# See the License for the specific language governing permissions and
1613+# limitations under the License.
1614
1615 import six
1616 import re
1617
1618=== modified file 'charmhelpers/core/sysctl.py'
1619--- charmhelpers/core/sysctl.py 2015-03-12 11:42:26 +0000
1620+++ charmhelpers/core/sysctl.py 2017-03-04 02:50:20 +0000
1621@@ -3,19 +3,17 @@
1622
1623 # Copyright 2014-2015 Canonical Limited.
1624 #
1625-# This file is part of charm-helpers.
1626-#
1627-# charm-helpers is free software: you can redistribute it and/or modify
1628-# it under the terms of the GNU Lesser General Public License version 3 as
1629-# published by the Free Software Foundation.
1630-#
1631-# charm-helpers is distributed in the hope that it will be useful,
1632-# but WITHOUT ANY WARRANTY; without even the implied warranty of
1633-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1634-# GNU Lesser General Public License for more details.
1635-#
1636-# You should have received a copy of the GNU Lesser General Public License
1637-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1638+# Licensed under the Apache License, Version 2.0 (the "License");
1639+# you may not use this file except in compliance with the License.
1640+# You may obtain a copy of the License at
1641+#
1642+# http://www.apache.org/licenses/LICENSE-2.0
1643+#
1644+# Unless required by applicable law or agreed to in writing, software
1645+# distributed under the License is distributed on an "AS IS" BASIS,
1646+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
1647+# See the License for the specific language governing permissions and
1648+# limitations under the License.
1649
1650 import yaml
1651
1652
1653=== modified file 'charmhelpers/core/templating.py'
1654--- charmhelpers/core/templating.py 2015-12-11 15:23:38 +0000
1655+++ charmhelpers/core/templating.py 2017-03-04 02:50:20 +0000
1656@@ -1,20 +1,19 @@
1657 # Copyright 2014-2015 Canonical Limited.
1658 #
1659-# This file is part of charm-helpers.
1660-#
1661-# charm-helpers is free software: you can redistribute it and/or modify
1662-# it under the terms of the GNU Lesser General Public License version 3 as
1663-# published by the Free Software Foundation.
1664-#
1665-# charm-helpers is distributed in the hope that it will be useful,
1666-# but WITHOUT ANY WARRANTY; without even the implied warranty of
1667-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1668-# GNU Lesser General Public License for more details.
1669-#
1670-# You should have received a copy of the GNU Lesser General Public License
1671-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1672+# Licensed under the Apache License, Version 2.0 (the "License");
1673+# you may not use this file except in compliance with the License.
1674+# You may obtain a copy of the License at
1675+#
1676+# http://www.apache.org/licenses/LICENSE-2.0
1677+#
1678+# Unless required by applicable law or agreed to in writing, software
1679+# distributed under the License is distributed on an "AS IS" BASIS,
1680+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
1681+# See the License for the specific language governing permissions and
1682+# limitations under the License.
1683
1684 import os
1685+import sys
1686
1687 from charmhelpers.core import host
1688 from charmhelpers.core import hookenv
1689@@ -40,8 +39,9 @@
1690 The rendered template will be written to the file as well as being returned
1691 as a string.
1692
1693- Note: Using this requires python-jinja2; if it is not installed, calling
1694- this will attempt to use charmhelpers.fetch.apt_install to install it.
1695+ Note: Using this requires python-jinja2 or python3-jinja2; if it is not
1696+ installed, calling this will attempt to use charmhelpers.fetch.apt_install
1697+ to install it.
1698 """
1699 try:
1700 from jinja2 import FileSystemLoader, Environment, exceptions
1701@@ -53,7 +53,10 @@
1702 'charmhelpers.fetch to install it',
1703 level=hookenv.ERROR)
1704 raise
1705- apt_install('python-jinja2', fatal=True)
1706+ if sys.version_info.major == 2:
1707+ apt_install('python-jinja2', fatal=True)
1708+ else:
1709+ apt_install('python3-jinja2', fatal=True)
1710 from jinja2 import FileSystemLoader, Environment, exceptions
1711
1712 if template_loader:
1713
1714=== modified file 'charmhelpers/core/unitdata.py'
1715--- charmhelpers/core/unitdata.py 2015-12-11 15:23:38 +0000
1716+++ charmhelpers/core/unitdata.py 2017-03-04 02:50:20 +0000
1717@@ -3,20 +3,17 @@
1718 #
1719 # Copyright 2014-2015 Canonical Limited.
1720 #
1721-# This file is part of charm-helpers.
1722-#
1723-# charm-helpers is free software: you can redistribute it and/or modify
1724-# it under the terms of the GNU Lesser General Public License version 3 as
1725-# published by the Free Software Foundation.
1726-#
1727-# charm-helpers is distributed in the hope that it will be useful,
1728-# but WITHOUT ANY WARRANTY; without even the implied warranty of
1729-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1730-# GNU Lesser General Public License for more details.
1731-#
1732-# You should have received a copy of the GNU Lesser General Public License
1733-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1734-#
1735+# Licensed under the Apache License, Version 2.0 (the "License");
1736+# you may not use this file except in compliance with the License.
1737+# You may obtain a copy of the License at
1738+#
1739+# http://www.apache.org/licenses/LICENSE-2.0
1740+#
1741+# Unless required by applicable law or agreed to in writing, software
1742+# distributed under the License is distributed on an "AS IS" BASIS,
1743+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
1744+# See the License for the specific language governing permissions and
1745+# limitations under the License.
1746 #
1747 # Authors:
1748 # Kapil Thangavelu <kapil.foss@gmail.com>
1749
1750=== modified file 'charmhelpers/fetch/__init__.py'
1751--- charmhelpers/fetch/__init__.py 2016-05-06 07:12:40 +0000
1752+++ charmhelpers/fetch/__init__.py 2017-03-04 02:50:20 +0000
1753@@ -1,32 +1,24 @@
1754 # Copyright 2014-2015 Canonical Limited.
1755 #
1756-# This file is part of charm-helpers.
1757-#
1758-# charm-helpers is free software: you can redistribute it and/or modify
1759-# it under the terms of the GNU Lesser General Public License version 3 as
1760-# published by the Free Software Foundation.
1761-#
1762-# charm-helpers is distributed in the hope that it will be useful,
1763-# but WITHOUT ANY WARRANTY; without even the implied warranty of
1764-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1765-# GNU Lesser General Public License for more details.
1766-#
1767-# You should have received a copy of the GNU Lesser General Public License
1768-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1769+# Licensed under the Apache License, Version 2.0 (the "License");
1770+# you may not use this file except in compliance with the License.
1771+# You may obtain a copy of the License at
1772+#
1773+# http://www.apache.org/licenses/LICENSE-2.0
1774+#
1775+# Unless required by applicable law or agreed to in writing, software
1776+# distributed under the License is distributed on an "AS IS" BASIS,
1777+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
1778+# See the License for the specific language governing permissions and
1779+# limitations under the License.
1780
1781 import importlib
1782-from tempfile import NamedTemporaryFile
1783-import time
1784+from charmhelpers.osplatform import get_platform
1785 from yaml import safe_load
1786-from charmhelpers.core.host import (
1787- lsb_release
1788-)
1789-import subprocess
1790 from charmhelpers.core.hookenv import (
1791 config,
1792 log,
1793 )
1794-import os
1795
1796 import six
1797 if six.PY3:
1798@@ -35,79 +27,6 @@
1799 from urlparse import urlparse, urlunparse
1800
1801
1802-CLOUD_ARCHIVE = """# Ubuntu Cloud Archive
1803-deb http://ubuntu-cloud.archive.canonical.com/ubuntu {} main
1804-"""
1805-PROPOSED_POCKET = """# Proposed
1806-deb http://archive.ubuntu.com/ubuntu {}-proposed main universe multiverse restricted
1807-"""
1808-CLOUD_ARCHIVE_POCKETS = {
1809- # Folsom
1810- 'folsom': 'precise-updates/folsom',
1811- 'precise-folsom': 'precise-updates/folsom',
1812- 'precise-folsom/updates': 'precise-updates/folsom',
1813- 'precise-updates/folsom': 'precise-updates/folsom',
1814- 'folsom/proposed': 'precise-proposed/folsom',
1815- 'precise-folsom/proposed': 'precise-proposed/folsom',
1816- 'precise-proposed/folsom': 'precise-proposed/folsom',
1817- # Grizzly
1818- 'grizzly': 'precise-updates/grizzly',
1819- 'precise-grizzly': 'precise-updates/grizzly',
1820- 'precise-grizzly/updates': 'precise-updates/grizzly',
1821- 'precise-updates/grizzly': 'precise-updates/grizzly',
1822- 'grizzly/proposed': 'precise-proposed/grizzly',
1823- 'precise-grizzly/proposed': 'precise-proposed/grizzly',
1824- 'precise-proposed/grizzly': 'precise-proposed/grizzly',
1825- # Havana
1826- 'havana': 'precise-updates/havana',
1827- 'precise-havana': 'precise-updates/havana',
1828- 'precise-havana/updates': 'precise-updates/havana',
1829- 'precise-updates/havana': 'precise-updates/havana',
1830- 'havana/proposed': 'precise-proposed/havana',
1831- 'precise-havana/proposed': 'precise-proposed/havana',
1832- 'precise-proposed/havana': 'precise-proposed/havana',
1833- # Icehouse
1834- 'icehouse': 'precise-updates/icehouse',
1835- 'precise-icehouse': 'precise-updates/icehouse',
1836- 'precise-icehouse/updates': 'precise-updates/icehouse',
1837- 'precise-updates/icehouse': 'precise-updates/icehouse',
1838- 'icehouse/proposed': 'precise-proposed/icehouse',
1839- 'precise-icehouse/proposed': 'precise-proposed/icehouse',
1840- 'precise-proposed/icehouse': 'precise-proposed/icehouse',
1841- # Juno
1842- 'juno': 'trusty-updates/juno',
1843- 'trusty-juno': 'trusty-updates/juno',
1844- 'trusty-juno/updates': 'trusty-updates/juno',
1845- 'trusty-updates/juno': 'trusty-updates/juno',
1846- 'juno/proposed': 'trusty-proposed/juno',
1847- 'trusty-juno/proposed': 'trusty-proposed/juno',
1848- 'trusty-proposed/juno': 'trusty-proposed/juno',
1849- # Kilo
1850- 'kilo': 'trusty-updates/kilo',
1851- 'trusty-kilo': 'trusty-updates/kilo',
1852- 'trusty-kilo/updates': 'trusty-updates/kilo',
1853- 'trusty-updates/kilo': 'trusty-updates/kilo',
1854- 'kilo/proposed': 'trusty-proposed/kilo',
1855- 'trusty-kilo/proposed': 'trusty-proposed/kilo',
1856- 'trusty-proposed/kilo': 'trusty-proposed/kilo',
1857- # Liberty
1858- 'liberty': 'trusty-updates/liberty',
1859- 'trusty-liberty': 'trusty-updates/liberty',
1860- 'trusty-liberty/updates': 'trusty-updates/liberty',
1861- 'trusty-updates/liberty': 'trusty-updates/liberty',
1862- 'liberty/proposed': 'trusty-proposed/liberty',
1863- 'trusty-liberty/proposed': 'trusty-proposed/liberty',
1864- 'trusty-proposed/liberty': 'trusty-proposed/liberty',
1865- # Mitaka
1866- 'mitaka': 'trusty-updates/mitaka',
1867- 'trusty-mitaka': 'trusty-updates/mitaka',
1868- 'trusty-mitaka/updates': 'trusty-updates/mitaka',
1869- 'trusty-updates/mitaka': 'trusty-updates/mitaka',
1870- 'mitaka/proposed': 'trusty-proposed/mitaka',
1871- 'trusty-mitaka/proposed': 'trusty-proposed/mitaka',
1872- 'trusty-proposed/mitaka': 'trusty-proposed/mitaka',
1873-}
1874-
1875 # The order of this list is very important. Handlers should be listed in from
1876 # least- to most-specific URL matching.
1877 FETCH_HANDLERS = (
1878@@ -116,10 +35,6 @@
1879 'charmhelpers.fetch.giturl.GitUrlFetchHandler',
1880 )
1881
1882-APT_NO_LOCK = 100 # The return code for "couldn't acquire lock" in APT.
1883-APT_NO_LOCK_RETRY_DELAY = 10 # Wait 10 seconds between apt lock checks.
1884-APT_NO_LOCK_RETRY_COUNT = 30 # Retry to acquire the lock X times.
1885-
1886
1887 class SourceConfigError(Exception):
1888 pass
1889@@ -157,180 +72,38 @@
1890 return urlunparse(parts)
1891
1892
1893-def filter_installed_packages(packages):
1894- """Returns a list of packages that require installation"""
1895- cache = apt_cache()
1896- _pkgs = []
1897- for package in packages:
1898- try:
1899- p = cache[package]
1900- p.current_ver or _pkgs.append(package)
1901- except KeyError:
1902- log('Package {} has no installation candidate.'.format(package),
1903- level='WARNING')
1904- _pkgs.append(package)
1905- return _pkgs
1906-
1907-
1908-def apt_cache(in_memory=True):
1909- """Build and return an apt cache"""
1910- from apt import apt_pkg
1911- apt_pkg.init()
1912- if in_memory:
1913- apt_pkg.config.set("Dir::Cache::pkgcache", "")
1914- apt_pkg.config.set("Dir::Cache::srcpkgcache", "")
1915- return apt_pkg.Cache()
1916-
1917-
1918-def apt_install(packages, options=None, fatal=False):
1919- """Install one or more packages"""
1920- if options is None:
1921- options = ['--option=Dpkg::Options::=--force-confold']
1922-
1923- cmd = ['apt-get', '--assume-yes']
1924- cmd.extend(options)
1925- cmd.append('install')
1926- if isinstance(packages, six.string_types):
1927- cmd.append(packages)
1928- else:
1929- cmd.extend(packages)
1930- log("Installing {} with options: {}".format(packages,
1931- options))
1932- _run_apt_command(cmd, fatal)
1933-
1934-
1935-def apt_upgrade(options=None, fatal=False, dist=False):
1936- """Upgrade all packages"""
1937- if options is None:
1938- options = ['--option=Dpkg::Options::=--force-confold']
1939-
1940- cmd = ['apt-get', '--assume-yes']
1941- cmd.extend(options)
1942- if dist:
1943- cmd.append('dist-upgrade')
1944- else:
1945- cmd.append('upgrade')
1946- log("Upgrading with options: {}".format(options))
1947- _run_apt_command(cmd, fatal)
1948-
1949-
1950-def apt_update(fatal=False):
1951- """Update local apt cache"""
1952- cmd = ['apt-get', 'update']
1953- _run_apt_command(cmd, fatal)
1954-
1955-
1956-def apt_purge(packages, fatal=False):
1957- """Purge one or more packages"""
1958- cmd = ['apt-get', '--assume-yes', 'purge']
1959- if isinstance(packages, six.string_types):
1960- cmd.append(packages)
1961- else:
1962- cmd.extend(packages)
1963- log("Purging {}".format(packages))
1964- _run_apt_command(cmd, fatal)
1965-
1966-
1967-def apt_mark(packages, mark, fatal=False):
1968- """Flag one or more packages using apt-mark"""
1969- log("Marking {} as {}".format(packages, mark))
1970- cmd = ['apt-mark', mark]
1971- if isinstance(packages, six.string_types):
1972- cmd.append(packages)
1973- else:
1974- cmd.extend(packages)
1975-
1976- if fatal:
1977- subprocess.check_call(cmd, universal_newlines=True)
1978- else:
1979- subprocess.call(cmd, universal_newlines=True)
1980-
1981-
1982-def apt_hold(packages, fatal=False):
1983- return apt_mark(packages, 'hold', fatal=fatal)
1984-
1985-
1986-def apt_unhold(packages, fatal=False):
1987- return apt_mark(packages, 'unhold', fatal=fatal)
1988-
1989-
1990-def add_source(source, key=None):
1991- """Add a package source to this system.
1992-
1993- @param source: a URL or sources.list entry, as supported by
1994- add-apt-repository(1). Examples::
1995-
1996- ppa:charmers/example
1997- deb https://stub:key@private.example.com/ubuntu trusty main
1998-
1999- In addition:
2000- 'proposed:' may be used to enable the standard 'proposed'
2001- pocket for the release.
2002- 'cloud:' may be used to activate official cloud archive pockets,
2003- such as 'cloud:icehouse'
2004- 'distro' may be used as a noop
2005-
2006- @param key: A key to be added to the system's APT keyring and used
2007- to verify the signatures on packages. Ideally, this should be an
2008- ASCII format GPG public key including the block headers. A GPG key
2009- id may also be used, but be aware that only insecure protocols are
2010- available to retrieve the actual public key from a public keyserver
2011- placing your Juju environment at risk. ppa and cloud archive keys
2012- are securely added automtically, so sould not be provided.
2013- """
2014- if source is None:
2015- log('Source is not present. Skipping')
2016- return
2017-
2018- if (source.startswith('ppa:') or
2019- source.startswith('http') or
2020- source.startswith('deb ') or
2021- source.startswith('cloud-archive:')):
2022- subprocess.check_call(['add-apt-repository', '--yes', source])
2023- elif source.startswith('cloud:'):
2024- apt_install(filter_installed_packages(['ubuntu-cloud-keyring']),
2025- fatal=True)
2026- pocket = source.split(':')[-1]
2027- if pocket not in CLOUD_ARCHIVE_POCKETS:
2028- raise SourceConfigError(
2029- 'Unsupported cloud: source option %s' %
2030- pocket)
2031- actual_pocket = CLOUD_ARCHIVE_POCKETS[pocket]
2032- with open('/etc/apt/sources.list.d/cloud-archive.list', 'w') as apt:
2033- apt.write(CLOUD_ARCHIVE.format(actual_pocket))
2034- elif source == 'proposed':
2035- release = lsb_release()['DISTRIB_CODENAME']
2036- with open('/etc/apt/sources.list.d/proposed.list', 'w') as apt:
2037- apt.write(PROPOSED_POCKET.format(release))
2038- elif source == 'distro':
2039- pass
2040- else:
2041- log("Unknown source: {!r}".format(source))
2042-
2043- if key:
2044- if '-----BEGIN PGP PUBLIC KEY BLOCK-----' in key:
2045- with NamedTemporaryFile('w+') as key_file:
2046- key_file.write(key)
2047- key_file.flush()
2048- key_file.seek(0)
2049- subprocess.check_call(['apt-key', 'add', '-'], stdin=key_file)
2050- else:
2051- # Note that hkp: is in no way a secure protocol. Using a
2052- # GPG key id is pointless from a security POV unless you
2053- # absolutely trust your network and DNS.
2054- subprocess.check_call(['apt-key', 'adv', '--keyserver',
2055- 'hkp://keyserver.ubuntu.com:80', '--recv',
2056- key])
2057+__platform__ = get_platform()
2058+module = "charmhelpers.fetch.%s" % __platform__
2059+fetch = importlib.import_module(module)
2060+
2061+filter_installed_packages = fetch.filter_installed_packages
2062+install = fetch.install
2063+upgrade = fetch.upgrade
2064+update = fetch.update
2065+purge = fetch.purge
2066+add_source = fetch.add_source
2067+
2068+if __platform__ == "ubuntu":
2069+ apt_cache = fetch.apt_cache
2070+ apt_install = fetch.install
2071+ apt_update = fetch.update
2072+ apt_upgrade = fetch.upgrade
2073+ apt_purge = fetch.purge
2074+ apt_mark = fetch.apt_mark
2075+ apt_hold = fetch.apt_hold
2076+ apt_unhold = fetch.apt_unhold
2077+ get_upstream_version = fetch.get_upstream_version
2078+elif __platform__ == "centos":
2079+ yum_search = fetch.yum_search
2080
2081
2082 def configure_sources(update=False,
2083 sources_var='install_sources',
2084 keys_var='install_keys'):
2085- """
2086- Configure multiple sources from charm configuration.
2087+ """Configure multiple sources from charm configuration.
2088
2089 The lists are encoded as yaml fragments in the configuration.
2090- The frament needs to be included as a string. Sources and their
2091+ The fragment needs to be included as a string. Sources and their
2092 corresponding keys are of the types supported by add_source().
2093
2094 Example config:
2095@@ -362,12 +135,11 @@
2096 for source, key in zip(sources, keys):
2097 add_source(source, key)
2098 if update:
2099- apt_update(fatal=True)
2100+ fetch.update(fatal=True)
2101
2102
2103 def install_remote(source, *args, **kwargs):
2104- """
2105- Install a file tree from a remote source
2106+ """Install a file tree from a remote source.
2107
2108 The specified source should be a url of the form:
2109 scheme://[host]/path[#[option=value][&...]]
2110@@ -390,19 +162,17 @@
2111 # We ONLY check for True here because can_handle may return a string
2112 # explaining why it can't handle a given source.
2113 handlers = [h for h in plugins() if h.can_handle(source) is True]
2114- installed_to = None
2115 for handler in handlers:
2116 try:
2117- installed_to = handler.install(source, *args, **kwargs)
2118+ return handler.install(source, *args, **kwargs)
2119 except UnhandledSource as e:
2120 log('Install source attempt unsuccessful: {}'.format(e),
2121 level='WARNING')
2122- if not installed_to:
2123- raise UnhandledSource("No handler found for source {}".format(source))
2124- return installed_to
2125+ raise UnhandledSource("No handler found for source {}".format(source))
2126
2127
2128 def install_from_config(config_var_name):
2129+ """Install a file from config."""
2130 charm_config = config()
2131 source = charm_config[config_var_name]
2132 return install_remote(source)
2133@@ -425,40 +195,3 @@
2134 log("FetchHandler {} not found, skipping plugin".format(
2135 handler_name))
2136 return plugin_list
2137-
2138-
2139-def _run_apt_command(cmd, fatal=False):
2140- """
2141- Run an APT command, checking output and retrying if the fatal flag is set
2142- to True.
2143-
2144- :param: cmd: str: The apt command to run.
2145- :param: fatal: bool: Whether the command's output should be checked and
2146- retried.
2147- """
2148- env = os.environ.copy()
2149-
2150- if 'DEBIAN_FRONTEND' not in env:
2151- env['DEBIAN_FRONTEND'] = 'noninteractive'
2152-
2153- if fatal:
2154- retry_count = 0
2155- result = None
2156-
2157- # If the command is considered "fatal", we need to retry if the apt
2158- # lock was not acquired.
2159-
2160- while result is None or result == APT_NO_LOCK:
2161- try:
2162- result = subprocess.check_call(cmd, env=env)
2163- except subprocess.CalledProcessError as e:
2164- retry_count = retry_count + 1
2165- if retry_count > APT_NO_LOCK_RETRY_COUNT:
2166- raise
2167- result = e.returncode
2168- log("Couldn't acquire DPKG lock. Will retry in {} seconds."
2169- "".format(APT_NO_LOCK_RETRY_DELAY))
2170- time.sleep(APT_NO_LOCK_RETRY_DELAY)
2171-
2172- else:
2173- subprocess.call(cmd, env=env)
2174
2175=== modified file 'charmhelpers/fetch/archiveurl.py'
2176--- charmhelpers/fetch/archiveurl.py 2015-12-11 15:23:38 +0000
2177+++ charmhelpers/fetch/archiveurl.py 2017-03-04 02:50:20 +0000
2178@@ -1,18 +1,16 @@
2179 # Copyright 2014-2015 Canonical Limited.
2180 #
2181-# This file is part of charm-helpers.
2182-#
2183-# charm-helpers is free software: you can redistribute it and/or modify
2184-# it under the terms of the GNU Lesser General Public License version 3 as
2185-# published by the Free Software Foundation.
2186-#
2187-# charm-helpers is distributed in the hope that it will be useful,
2188-# but WITHOUT ANY WARRANTY; without even the implied warranty of
2189-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
2190-# GNU Lesser General Public License for more details.
2191-#
2192-# You should have received a copy of the GNU Lesser General Public License
2193-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
2194+# Licensed under the Apache License, Version 2.0 (the "License");
2195+# you may not use this file except in compliance with the License.
2196+# You may obtain a copy of the License at
2197+#
2198+# http://www.apache.org/licenses/LICENSE-2.0
2199+#
2200+# Unless required by applicable law or agreed to in writing, software
2201+# distributed under the License is distributed on an "AS IS" BASIS,
2202+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
2203+# See the License for the specific language governing permissions and
2204+# limitations under the License.
2205
2206 import os
2207 import hashlib
2208
2209=== modified file 'charmhelpers/fetch/bzrurl.py'
2210--- charmhelpers/fetch/bzrurl.py 2016-05-06 07:12:40 +0000
2211+++ charmhelpers/fetch/bzrurl.py 2017-03-04 02:50:20 +0000
2212@@ -1,18 +1,16 @@
2213 # Copyright 2014-2015 Canonical Limited.
2214 #
2215-# This file is part of charm-helpers.
2216-#
2217-# charm-helpers is free software: you can redistribute it and/or modify
2218-# it under the terms of the GNU Lesser General Public License version 3 as
2219-# published by the Free Software Foundation.
2220-#
2221-# charm-helpers is distributed in the hope that it will be useful,
2222-# but WITHOUT ANY WARRANTY; without even the implied warranty of
2223-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
2224-# GNU Lesser General Public License for more details.
2225-#
2226-# You should have received a copy of the GNU Lesser General Public License
2227-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
2228+# Licensed under the Apache License, Version 2.0 (the "License");
2229+# you may not use this file except in compliance with the License.
2230+# You may obtain a copy of the License at
2231+#
2232+# http://www.apache.org/licenses/LICENSE-2.0
2233+#
2234+# Unless required by applicable law or agreed to in writing, software
2235+# distributed under the License is distributed on an "AS IS" BASIS,
2236+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
2237+# See the License for the specific language governing permissions and
2238+# limitations under the License.
2239
2240 import os
2241 from subprocess import check_call
2242@@ -20,19 +18,20 @@
2243 BaseFetchHandler,
2244 UnhandledSource,
2245 filter_installed_packages,
2246- apt_install,
2247+ install,
2248 )
2249 from charmhelpers.core.host import mkdir
2250
2251
2252 if filter_installed_packages(['bzr']) != []:
2253- apt_install(['bzr'])
2254+ install(['bzr'])
2255 if filter_installed_packages(['bzr']) != []:
2256 raise NotImplementedError('Unable to install bzr')
2257
2258
2259 class BzrUrlFetchHandler(BaseFetchHandler):
2260- """Handler for bazaar branches via generic and lp URLs"""
2261+ """Handler for bazaar branches via generic and lp URLs."""
2262+
2263 def can_handle(self, source):
2264 url_parts = self.parse_url(source)
2265 if url_parts.scheme not in ('bzr+ssh', 'lp', ''):
2266@@ -42,15 +41,23 @@
2267 else:
2268 return True
2269
2270- def branch(self, source, dest):
2271+ def branch(self, source, dest, revno=None):
2272 if not self.can_handle(source):
2273 raise UnhandledSource("Cannot handle {}".format(source))
2274+ cmd_opts = []
2275+ if revno:
2276+ cmd_opts += ['-r', str(revno)]
2277 if os.path.exists(dest):
2278- check_call(['bzr', 'pull', '--overwrite', '-d', dest, source])
2279+ cmd = ['bzr', 'pull']
2280+ cmd += cmd_opts
2281+ cmd += ['--overwrite', '-d', dest, source]
2282 else:
2283- check_call(['bzr', 'branch', source, dest])
2284+ cmd = ['bzr', 'branch']
2285+ cmd += cmd_opts
2286+ cmd += [source, dest]
2287+ check_call(cmd)
2288
2289- def install(self, source, dest=None):
2290+ def install(self, source, dest=None, revno=None):
2291 url_parts = self.parse_url(source)
2292 branch_name = url_parts.path.strip("/").split("/")[-1]
2293 if dest:
2294@@ -59,10 +66,11 @@
2295 dest_dir = os.path.join(os.environ.get('CHARM_DIR'), "fetched",
2296 branch_name)
2297
2298- if not os.path.exists(dest_dir):
2299- mkdir(dest_dir, perms=0o755)
2300+ if dest and not os.path.exists(dest):
2301+ mkdir(dest, perms=0o755)
2302+
2303 try:
2304- self.branch(source, dest_dir)
2305+ self.branch(source, dest_dir, revno)
2306 except OSError as e:
2307 raise UnhandledSource(e.strerror)
2308 return dest_dir
2309
2310=== added file 'charmhelpers/fetch/centos.py'
2311--- charmhelpers/fetch/centos.py 1970-01-01 00:00:00 +0000
2312+++ charmhelpers/fetch/centos.py 2017-03-04 02:50:20 +0000
2313@@ -0,0 +1,171 @@
2314+# Copyright 2014-2015 Canonical Limited.
2315+#
2316+# Licensed under the Apache License, Version 2.0 (the "License");
2317+# you may not use this file except in compliance with the License.
2318+# You may obtain a copy of the License at
2319+#
2320+# http://www.apache.org/licenses/LICENSE-2.0
2321+#
2322+# Unless required by applicable law or agreed to in writing, software
2323+# distributed under the License is distributed on an "AS IS" BASIS,
2324+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
2325+# See the License for the specific language governing permissions and
2326+# limitations under the License.
2327+
2328+import subprocess
2329+import os
2330+import time
2331+import six
2332+import yum
2333+
2334+from tempfile import NamedTemporaryFile
2335+from charmhelpers.core.hookenv import log
2336+
2337+YUM_NO_LOCK = 1 # The return code for "couldn't acquire lock" in YUM.
2338+YUM_NO_LOCK_RETRY_DELAY = 10 # Wait 10 seconds between apt lock checks.
2339+YUM_NO_LOCK_RETRY_COUNT = 30 # Retry to acquire the lock X times.
2340+
2341+
2342+def filter_installed_packages(packages):
2343+ """Return a list of packages that require installation."""
2344+ yb = yum.YumBase()
2345+ package_list = yb.doPackageLists()
2346+ temp_cache = {p.base_package_name: 1 for p in package_list['installed']}
2347+
2348+ _pkgs = [p for p in packages if not temp_cache.get(p, False)]
2349+ return _pkgs
2350+
2351+
2352+def install(packages, options=None, fatal=False):
2353+ """Install one or more packages."""
2354+ cmd = ['yum', '--assumeyes']
2355+ if options is not None:
2356+ cmd.extend(options)
2357+ cmd.append('install')
2358+ if isinstance(packages, six.string_types):
2359+ cmd.append(packages)
2360+ else:
2361+ cmd.extend(packages)
2362+ log("Installing {} with options: {}".format(packages,
2363+ options))
2364+ _run_yum_command(cmd, fatal)
2365+
2366+
2367+def upgrade(options=None, fatal=False, dist=False):
2368+ """Upgrade all packages."""
2369+ cmd = ['yum', '--assumeyes']
2370+ if options is not None:
2371+ cmd.extend(options)
2372+ cmd.append('upgrade')
2373+ log("Upgrading with options: {}".format(options))
2374+ _run_yum_command(cmd, fatal)
2375+
2376+
2377+def update(fatal=False):
2378+ """Update local yum cache."""
2379+ cmd = ['yum', '--assumeyes', 'update']
2380+ log("Update with fatal: {}".format(fatal))
2381+ _run_yum_command(cmd, fatal)
2382+
2383+
2384+def purge(packages, fatal=False):
2385+ """Purge one or more packages."""
2386+ cmd = ['yum', '--assumeyes', 'remove']
2387+ if isinstance(packages, six.string_types):
2388+ cmd.append(packages)
2389+ else:
2390+ cmd.extend(packages)
2391+ log("Purging {}".format(packages))
2392+ _run_yum_command(cmd, fatal)
2393+
2394+
2395+def yum_search(packages):
2396+ """Search for a package."""
2397+ output = {}
2398+ cmd = ['yum', 'search']
2399+ if isinstance(packages, six.string_types):
2400+ cmd.append(packages)
2401+ else:
2402+ cmd.extend(packages)
2403+ log("Searching for {}".format(packages))
2404+ result = subprocess.check_output(cmd)
2405+ for package in list(packages):
2406+ output[package] = package in result
2407+ return output
2408+
2409+
2410+def add_source(source, key=None):
2411+ """Add a package source to this system.
2412+
2413+ @param source: a URL with a rpm package
2414+
2415+ @param key: A key to be added to the system's keyring and used
2416+ to verify the signatures on packages. Ideally, this should be an
2417+ ASCII format GPG public key including the block headers. A GPG key
2418+ id may also be used, but be aware that only insecure protocols are
2419+ available to retrieve the actual public key from a public keyserver
2420+ placing your Juju environment at risk.
2421+ """
2422+ if source is None:
2423+ log('Source is not present. Skipping')
2424+ return
2425+
2426+ if source.startswith('http'):
2427+ directory = '/etc/yum.repos.d/'
2428+ for filename in os.listdir(directory):
2429+ with open(directory + filename, 'r') as rpm_file:
2430+ if source in rpm_file.read():
2431+ break
2432+ else:
2433+ log("Add source: {!r}".format(source))
2434+ # write in the charms.repo
2435+ with open(directory + 'Charms.repo', 'a') as rpm_file:
2436+ rpm_file.write('[%s]\n' % source[7:].replace('/', '_'))
2437+ rpm_file.write('name=%s\n' % source[7:])
2438+ rpm_file.write('baseurl=%s\n\n' % source)
2439+ else:
2440+ log("Unknown source: {!r}".format(source))
2441+
2442+ if key:
2443+ if '-----BEGIN PGP PUBLIC KEY BLOCK-----' in key:
2444+ with NamedTemporaryFile('w+') as key_file:
2445+ key_file.write(key)
2446+ key_file.flush()
2447+ key_file.seek(0)
2448+ subprocess.check_call(['rpm', '--import', key_file])
2449+ else:
2450+ subprocess.check_call(['rpm', '--import', key])
2451+
2452+
2453+def _run_yum_command(cmd, fatal=False):
2454+ """Run an YUM command.
2455+
2456+ Checks the output and retry if the fatal flag is set to True.
2457+
2458+ :param: cmd: str: The yum command to run.
2459+ :param: fatal: bool: Whether the command's output should be checked and
2460+ retried.
2461+ """
2462+ env = os.environ.copy()
2463+
2464+ if fatal:
2465+ retry_count = 0
2466+ result = None
2467+
2468+ # If the command is considered "fatal", we need to retry if the yum
2469+ # lock was not acquired.
2470+
2471+ while result is None or result == YUM_NO_LOCK:
2472+ try:
2473+ result = subprocess.check_call(cmd, env=env)
2474+ except subprocess.CalledProcessError as e:
2475+ retry_count = retry_count + 1
2476+ if retry_count > YUM_NO_LOCK_RETRY_COUNT:
2477+ raise
2478+ result = e.returncode
2479+ log("Couldn't acquire YUM lock. Will retry in {} seconds."
2480+ "".format(YUM_NO_LOCK_RETRY_DELAY))
2481+ time.sleep(YUM_NO_LOCK_RETRY_DELAY)
2482+
2483+ else:
2484+ subprocess.call(cmd, env=env)
2485
2486=== modified file 'charmhelpers/fetch/giturl.py'
2487--- charmhelpers/fetch/giturl.py 2016-05-06 07:12:40 +0000
2488+++ charmhelpers/fetch/giturl.py 2017-03-04 02:50:20 +0000
2489@@ -1,18 +1,16 @@
2490 # Copyright 2014-2015 Canonical Limited.
2491 #
2492-# This file is part of charm-helpers.
2493-#
2494-# charm-helpers is free software: you can redistribute it and/or modify
2495-# it under the terms of the GNU Lesser General Public License version 3 as
2496-# published by the Free Software Foundation.
2497-#
2498-# charm-helpers is distributed in the hope that it will be useful,
2499-# but WITHOUT ANY WARRANTY; without even the implied warranty of
2500-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
2501-# GNU Lesser General Public License for more details.
2502-#
2503-# You should have received a copy of the GNU Lesser General Public License
2504-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
2505+# Licensed under the Apache License, Version 2.0 (the "License");
2506+# you may not use this file except in compliance with the License.
2507+# You may obtain a copy of the License at
2508+#
2509+# http://www.apache.org/licenses/LICENSE-2.0
2510+#
2511+# Unless required by applicable law or agreed to in writing, software
2512+# distributed under the License is distributed on an "AS IS" BASIS,
2513+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
2514+# See the License for the specific language governing permissions and
2515+# limitations under the License.
2516
2517 import os
2518 from subprocess import check_call, CalledProcessError
2519@@ -20,17 +18,18 @@
2520 BaseFetchHandler,
2521 UnhandledSource,
2522 filter_installed_packages,
2523- apt_install,
2524+ install,
2525 )
2526
2527 if filter_installed_packages(['git']) != []:
2528- apt_install(['git'])
2529+ install(['git'])
2530 if filter_installed_packages(['git']) != []:
2531 raise NotImplementedError('Unable to install git')
2532
2533
2534 class GitUrlFetchHandler(BaseFetchHandler):
2535- """Handler for git branches via generic and github URLs"""
2536+ """Handler for git branches via generic and github URLs."""
2537+
2538 def can_handle(self, source):
2539 url_parts = self.parse_url(source)
2540 # TODO (mattyw) no support for ssh git@ yet
2541
2542=== added file 'charmhelpers/fetch/snap.py'
2543--- charmhelpers/fetch/snap.py 1970-01-01 00:00:00 +0000
2544+++ charmhelpers/fetch/snap.py 2017-03-04 02:50:20 +0000
2545@@ -0,0 +1,122 @@
2546+# Copyright 2014-2017 Canonical Limited.
2547+#
2548+# Licensed under the Apache License, Version 2.0 (the "License");
2549+# you may not use this file except in compliance with the License.
2550+# You may obtain a copy of the License at
2551+#
2552+# http://www.apache.org/licenses/LICENSE-2.0
2553+#
2554+# Unless required by applicable law or agreed to in writing, software
2555+# distributed under the License is distributed on an "AS IS" BASIS,
2556+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
2557+# See the License for the specific language governing permissions and
2558+# limitations under the License.
2559+"""
2560+Charm helpers snap for classic charms.
2561+
2562+If writing reactive charms, use the snap layer:
2563+https://lists.ubuntu.com/archives/snapcraft/2016-September/001114.html
2564+"""
2565+import subprocess
2566+from os import environ
2567+from time import sleep
2568+from charmhelpers.core.hookenv import log
2569+
2570+__author__ = 'Joseph Borg <joseph.borg@canonical.com>'
2571+
2572+SNAP_NO_LOCK = 1 # The return code for "couldn't acquire lock" in Snap (hopefully this will be improved).
2573+SNAP_NO_LOCK_RETRY_DELAY = 10 # Wait X seconds between Snap lock checks.
2574+SNAP_NO_LOCK_RETRY_COUNT = 30 # Retry to acquire the lock X times.
2575+
2576+
2577+class CouldNotAcquireLockException(Exception):
2578+ pass
2579+
2580+
2581+def _snap_exec(commands):
2582+ """
2583+ Execute snap commands.
2584+
2585+ :param commands: List commands
2586+ :return: Integer exit code
2587+ """
2588+ assert type(commands) == list
2589+
2590+ retry_count = 0
2591+ return_code = None
2592+
2593+ while return_code is None or return_code == SNAP_NO_LOCK:
2594+ try:
2595+ return_code = subprocess.check_call(['snap'] + commands, env=environ)
2596+ except subprocess.CalledProcessError as e:
2597+ retry_count += + 1
2598+ if retry_count > SNAP_NO_LOCK_RETRY_COUNT:
2599+ raise CouldNotAcquireLockException('Could not aquire lock after %s attempts' % SNAP_NO_LOCK_RETRY_COUNT)
2600+ return_code = e.returncode
2601+ log('Snap failed to acquire lock, trying again in %s seconds.' % SNAP_NO_LOCK_RETRY_DELAY, level='WARN')
2602+ sleep(SNAP_NO_LOCK_RETRY_DELAY)
2603+
2604+ return return_code
2605+
2606+
2607+def snap_install(packages, *flags):
2608+ """
2609+ Install a snap package.
2610+
2611+ :param packages: String or List String package name
2612+ :param flags: List String flags to pass to install command
2613+ :return: Integer return code from snap
2614+ """
2615+ if type(packages) is not list:
2616+ packages = [packages]
2617+
2618+ flags = list(flags)
2619+
2620+ message = 'Installing snap(s) "%s"' % ', '.join(packages)
2621+ if flags:
2622+ message += ' with option(s) "%s"' % ', '.join(flags)
2623+
2624+ log(message, level='INFO')
2625+ return _snap_exec(['install'] + flags + packages)
2626+
2627+
2628+def snap_remove(packages, *flags):
2629+ """
2630+ Remove a snap package.
2631+
2632+ :param packages: String or List String package name
2633+ :param flags: List String flags to pass to remove command
2634+ :return: Integer return code from snap
2635+ """
2636+ if type(packages) is not list:
2637+ packages = [packages]
2638+
2639+ flags = list(flags)
2640+
2641+ message = 'Removing snap(s) "%s"' % ', '.join(packages)
2642+ if flags:
2643+ message += ' with options "%s"' % ', '.join(flags)
2644+
2645+ log(message, level='INFO')
2646+ return _snap_exec(['remove'] + flags + packages)
2647+
2648+
2649+def snap_refresh(packages, *flags):
2650+ """
2651+ Refresh / Update snap package.
2652+
2653+ :param packages: String or List String package name
2654+ :param flags: List String flags to pass to refresh command
2655+ :return: Integer return code from snap
2656+ """
2657+ if type(packages) is not list:
2658+ packages = [packages]
2659+
2660+ flags = list(flags)
2661+
2662+ message = 'Refreshing snap(s) "%s"' % ', '.join(packages)
2663+ if flags:
2664+ message += ' with options "%s"' % ', '.join(flags)
2665+
2666+ log(message, level='INFO')
2667+ return _snap_exec(['refresh'] + flags + packages)
2668
2669=== added file 'charmhelpers/fetch/ubuntu.py'
2670--- charmhelpers/fetch/ubuntu.py 1970-01-01 00:00:00 +0000
2671+++ charmhelpers/fetch/ubuntu.py 2017-03-04 02:50:20 +0000
2672@@ -0,0 +1,364 @@
2673+# Copyright 2014-2015 Canonical Limited.
2674+#
2675+# Licensed under the Apache License, Version 2.0 (the "License");
2676+# you may not use this file except in compliance with the License.
2677+# You may obtain a copy of the License at
2678+#
2679+# http://www.apache.org/licenses/LICENSE-2.0
2680+#
2681+# Unless required by applicable law or agreed to in writing, software
2682+# distributed under the License is distributed on an "AS IS" BASIS,
2683+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
2684+# See the License for the specific language governing permissions and
2685+# limitations under the License.
2686+
2687+import os
2688+import six
2689+import time
2690+import subprocess
2691+
2692+from tempfile import NamedTemporaryFile
2693+from charmhelpers.core.host import (
2694+ lsb_release
2695+)
2696+from charmhelpers.core.hookenv import log
2697+from charmhelpers.fetch import SourceConfigError
2698+
2699+CLOUD_ARCHIVE = """# Ubuntu Cloud Archive
2700+deb http://ubuntu-cloud.archive.canonical.com/ubuntu {} main
2701+"""
2702+
2703+PROPOSED_POCKET = """# Proposed
2704+deb http://archive.ubuntu.com/ubuntu {}-proposed main universe multiverse restricted
2705+"""
2706+
2707+CLOUD_ARCHIVE_POCKETS = {
2708+ # Folsom
2709+ 'folsom': 'precise-updates/folsom',
2710+ 'precise-folsom': 'precise-updates/folsom',
2711+ 'precise-folsom/updates': 'precise-updates/folsom',
2712+ 'precise-updates/folsom': 'precise-updates/folsom',
2713+ 'folsom/proposed': 'precise-proposed/folsom',
2714+ 'precise-folsom/proposed': 'precise-proposed/folsom',
2715+ 'precise-proposed/folsom': 'precise-proposed/folsom',
2716+ # Grizzly
2717+ 'grizzly': 'precise-updates/grizzly',
2718+ 'precise-grizzly': 'precise-updates/grizzly',
2719+ 'precise-grizzly/updates': 'precise-updates/grizzly',
2720+ 'precise-updates/grizzly': 'precise-updates/grizzly',
2721+ 'grizzly/proposed': 'precise-proposed/grizzly',
2722+ 'precise-grizzly/proposed': 'precise-proposed/grizzly',
2723+ 'precise-proposed/grizzly': 'precise-proposed/grizzly',
2724+ # Havana
2725+ 'havana': 'precise-updates/havana',
2726+ 'precise-havana': 'precise-updates/havana',
2727+ 'precise-havana/updates': 'precise-updates/havana',
2728+ 'precise-updates/havana': 'precise-updates/havana',
2729+ 'havana/proposed': 'precise-proposed/havana',
2730+ 'precise-havana/proposed': 'precise-proposed/havana',
2731+ 'precise-proposed/havana': 'precise-proposed/havana',
2732+ # Icehouse
2733+ 'icehouse': 'precise-updates/icehouse',
2734+ 'precise-icehouse': 'precise-updates/icehouse',
2735+ 'precise-icehouse/updates': 'precise-updates/icehouse',
2736+ 'precise-updates/icehouse': 'precise-updates/icehouse',
2737+ 'icehouse/proposed': 'precise-proposed/icehouse',
2738+ 'precise-icehouse/proposed': 'precise-proposed/icehouse',
2739+ 'precise-proposed/icehouse': 'precise-proposed/icehouse',
2740+ # Juno
2741+ 'juno': 'trusty-updates/juno',
2742+ 'trusty-juno': 'trusty-updates/juno',
2743+ 'trusty-juno/updates': 'trusty-updates/juno',
2744+ 'trusty-updates/juno': 'trusty-updates/juno',
2745+ 'juno/proposed': 'trusty-proposed/juno',
2746+ 'trusty-juno/proposed': 'trusty-proposed/juno',
2747+ 'trusty-proposed/juno': 'trusty-proposed/juno',
2748+ # Kilo
2749+ 'kilo': 'trusty-updates/kilo',
2750+ 'trusty-kilo': 'trusty-updates/kilo',
2751+ 'trusty-kilo/updates': 'trusty-updates/kilo',
2752+ 'trusty-updates/kilo': 'trusty-updates/kilo',
2753+ 'kilo/proposed': 'trusty-proposed/kilo',
2754+ 'trusty-kilo/proposed': 'trusty-proposed/kilo',
2755+ 'trusty-proposed/kilo': 'trusty-proposed/kilo',
2756+ # Liberty
2757+ 'liberty': 'trusty-updates/liberty',
2758+ 'trusty-liberty': 'trusty-updates/liberty',
2759+ 'trusty-liberty/updates': 'trusty-updates/liberty',
2760+ 'trusty-updates/liberty': 'trusty-updates/liberty',
2761+ 'liberty/proposed': 'trusty-proposed/liberty',
2762+ 'trusty-liberty/proposed': 'trusty-proposed/liberty',
2763+ 'trusty-proposed/liberty': 'trusty-proposed/liberty',
2764+ # Mitaka
2765+ 'mitaka': 'trusty-updates/mitaka',
2766+ 'trusty-mitaka': 'trusty-updates/mitaka',
2767+ 'trusty-mitaka/updates': 'trusty-updates/mitaka',
2768+ 'trusty-updates/mitaka': 'trusty-updates/mitaka',
2769+ 'mitaka/proposed': 'trusty-proposed/mitaka',
2770+ 'trusty-mitaka/proposed': 'trusty-proposed/mitaka',
2771+ 'trusty-proposed/mitaka': 'trusty-proposed/mitaka',
2772+ # Newton
2773+ 'newton': 'xenial-updates/newton',
2774+ 'xenial-newton': 'xenial-updates/newton',
2775+ 'xenial-newton/updates': 'xenial-updates/newton',
2776+ 'xenial-updates/newton': 'xenial-updates/newton',
2777+ 'newton/proposed': 'xenial-proposed/newton',
2778+ 'xenial-newton/proposed': 'xenial-proposed/newton',
2779+ 'xenial-proposed/newton': 'xenial-proposed/newton',
2780+ # Ocata
2781+ 'ocata': 'xenial-updates/ocata',
2782+ 'xenial-ocata': 'xenial-updates/ocata',
2783+ 'xenial-ocata/updates': 'xenial-updates/ocata',
2784+ 'xenial-updates/ocata': 'xenial-updates/ocata',
2785+ 'ocata/proposed': 'xenial-proposed/ocata',
2786+ 'xenial-ocata/proposed': 'xenial-proposed/ocata',
2787+ 'xenial-ocata/newton': 'xenial-proposed/ocata',
2788+}
2789+
2790+APT_NO_LOCK = 100 # The return code for "couldn't acquire lock" in APT.
2791+CMD_RETRY_DELAY = 10 # Wait 10 seconds between command retries.
2792+CMD_RETRY_COUNT = 30 # Retry a failing fatal command X times.
2793+
2794+
2795+def filter_installed_packages(packages):
2796+ """Return a list of packages that require installation."""
2797+ cache = apt_cache()
2798+ _pkgs = []
2799+ for package in packages:
2800+ try:
2801+ p = cache[package]
2802+ p.current_ver or _pkgs.append(package)
2803+ except KeyError:
2804+ log('Package {} has no installation candidate.'.format(package),
2805+ level='WARNING')
2806+ _pkgs.append(package)
2807+ return _pkgs
2808+
2809+
2810+def apt_cache(in_memory=True, progress=None):
2811+ """Build and return an apt cache."""
2812+ from apt import apt_pkg
2813+ apt_pkg.init()
2814+ if in_memory:
2815+ apt_pkg.config.set("Dir::Cache::pkgcache", "")
2816+ apt_pkg.config.set("Dir::Cache::srcpkgcache", "")
2817+ return apt_pkg.Cache(progress)
2818+
2819+
2820+def install(packages, options=None, fatal=False):
2821+ """Install one or more packages."""
2822+ if options is None:
2823+ options = ['--option=Dpkg::Options::=--force-confold']
2824+
2825+ cmd = ['apt-get', '--assume-yes']
2826+ cmd.extend(options)
2827+ cmd.append('install')
2828+ if isinstance(packages, six.string_types):
2829+ cmd.append(packages)
2830+ else:
2831+ cmd.extend(packages)
2832+ log("Installing {} with options: {}".format(packages,
2833+ options))
2834+ _run_apt_command(cmd, fatal)
2835+
2836+
2837+def upgrade(options=None, fatal=False, dist=False):
2838+ """Upgrade all packages."""
2839+ if options is None:
2840+ options = ['--option=Dpkg::Options::=--force-confold']
2841+
2842+ cmd = ['apt-get', '--assume-yes']
2843+ cmd.extend(options)
2844+ if dist:
2845+ cmd.append('dist-upgrade')
2846+ else:
2847+ cmd.append('upgrade')
2848+ log("Upgrading with options: {}".format(options))
2849+ _run_apt_command(cmd, fatal)
2850+
2851+
2852+def update(fatal=False):
2853+ """Update local apt cache."""
2854+ cmd = ['apt-get', 'update']
2855+ _run_apt_command(cmd, fatal)
2856+
2857+
2858+def purge(packages, fatal=False):
2859+ """Purge one or more packages."""
2860+ cmd = ['apt-get', '--assume-yes', 'purge']
2861+ if isinstance(packages, six.string_types):
2862+ cmd.append(packages)
2863+ else:
2864+ cmd.extend(packages)
2865+ log("Purging {}".format(packages))
2866+ _run_apt_command(cmd, fatal)
2867+
2868+
2869+def apt_mark(packages, mark, fatal=False):
2870+ """Flag one or more packages using apt-mark."""
2871+ log("Marking {} as {}".format(packages, mark))
2872+ cmd = ['apt-mark', mark]
2873+ if isinstance(packages, six.string_types):
2874+ cmd.append(packages)
2875+ else:
2876+ cmd.extend(packages)
2877+
2878+ if fatal:
2879+ subprocess.check_call(cmd, universal_newlines=True)
2880+ else:
2881+ subprocess.call(cmd, universal_newlines=True)
2882+
2883+
2884+def apt_hold(packages, fatal=False):
2885+ return apt_mark(packages, 'hold', fatal=fatal)
2886+
2887+
2888+def apt_unhold(packages, fatal=False):
2889+ return apt_mark(packages, 'unhold', fatal=fatal)
2890+
2891+
2892+def add_source(source, key=None):
2893+ """Add a package source to this system.
2894+
2895+ @param source: a URL or sources.list entry, as supported by
2896+ add-apt-repository(1). Examples::
2897+
2898+ ppa:charmers/example
2899+ deb https://stub:key@private.example.com/ubuntu trusty main
2900+
2901+ In addition:
2902+ 'proposed:' may be used to enable the standard 'proposed'
2903+ pocket for the release.
2904+ 'cloud:' may be used to activate official cloud archive pockets,
2905+ such as 'cloud:icehouse'
2906+ 'distro' may be used as a noop
2907+
2908+ @param key: A key to be added to the system's APT keyring and used
2909+ to verify the signatures on packages. Ideally, this should be an
2910+ ASCII format GPG public key including the block headers. A GPG key
2911+ id may also be used, but be aware that only insecure protocols are
2912+ available to retrieve the actual public key from a public keyserver
2913+ placing your Juju environment at risk. ppa and cloud archive keys
2914+ are securely added automtically, so sould not be provided.
2915+ """
2916+ if source is None:
2917+ log('Source is not present. Skipping')
2918+ return
2919+
2920+ if (source.startswith('ppa:') or
2921+ source.startswith('http') or
2922+ source.startswith('deb ') or
2923+ source.startswith('cloud-archive:')):
2924+ cmd = ['add-apt-repository', '--yes', source]
2925+ _run_with_retries(cmd)
2926+ elif source.startswith('cloud:'):
2927+ install(filter_installed_packages(['ubuntu-cloud-keyring']),
2928+ fatal=True)
2929+ pocket = source.split(':')[-1]
2930+ if pocket not in CLOUD_ARCHIVE_POCKETS:
2931+ raise SourceConfigError(
2932+ 'Unsupported cloud: source option %s' %
2933+ pocket)
2934+ actual_pocket = CLOUD_ARCHIVE_POCKETS[pocket]
2935+ with open('/etc/apt/sources.list.d/cloud-archive.list', 'w') as apt:
2936+ apt.write(CLOUD_ARCHIVE.format(actual_pocket))
2937+ elif source == 'proposed':
2938+ release = lsb_release()['DISTRIB_CODENAME']
2939+ with open('/etc/apt/sources.list.d/proposed.list', 'w') as apt:
2940+ apt.write(PROPOSED_POCKET.format(release))
2941+ elif source == 'distro':
2942+ pass
2943+ else:
2944+ log("Unknown source: {!r}".format(source))
2945+
2946+ if key:
2947+ if '-----BEGIN PGP PUBLIC KEY BLOCK-----' in key:
2948+ with NamedTemporaryFile('w+') as key_file:
2949+ key_file.write(key)
2950+ key_file.flush()
2951+ key_file.seek(0)
2952+ subprocess.check_call(['apt-key', 'add', '-'], stdin=key_file)
2953+ else:
2954+ # Note that hkp: is in no way a secure protocol. Using a
2955+ # GPG key id is pointless from a security POV unless you
2956+ # absolutely trust your network and DNS.
2957+ subprocess.check_call(['apt-key', 'adv', '--keyserver',
2958+ 'hkp://keyserver.ubuntu.com:80', '--recv',
2959+ key])
2960+
2961+
2962+def _run_with_retries(cmd, max_retries=CMD_RETRY_COUNT, retry_exitcodes=(1,),
2963+ retry_message="", cmd_env=None):
2964+ """Run a command and retry until success or max_retries is reached.
2965+
2966+ :param: cmd: str: The apt command to run.
2967+ :param: max_retries: int: The number of retries to attempt on a fatal
2968+ command. Defaults to CMD_RETRY_COUNT.
2969+ :param: retry_exitcodes: tuple: Optional additional exit codes to retry.
2970+ Defaults to retry on exit code 1.
2971+ :param: retry_message: str: Optional log prefix emitted during retries.
2972+ :param: cmd_env: dict: Environment variables to add to the command run.
2973+ """
2974+
2975+ env = os.environ.copy()
2976+ if cmd_env:
2977+ env.update(cmd_env)
2978+
2979+ if not retry_message:
2980+ retry_message = "Failed executing '{}'".format(" ".join(cmd))
2981+ retry_message += ". Will retry in {} seconds".format(CMD_RETRY_DELAY)
2982+
2983+ retry_count = 0
2984+ result = None
2985+
2986+ retry_results = (None,) + retry_exitcodes
2987+ while result in retry_results:
2988+ try:
2989+ result = subprocess.check_call(cmd, env=env)
2990+ except subprocess.CalledProcessError as e:
2991+ retry_count = retry_count + 1
2992+ if retry_count > max_retries:
2993+ raise
2994+ result = e.returncode
2995+ log(retry_message)
2996+ time.sleep(CMD_RETRY_DELAY)
2997+
2998+
2999+def _run_apt_command(cmd, fatal=False):
3000+ """Run an apt command with optional retries.
3001+
3002+ :param: fatal: bool: Whether the command's output should be checked and
3003+ retried.
3004+ """
3005+ # Provide DEBIAN_FRONTEND=noninteractive if not present in the environment.
3006+ cmd_env = {
3007+ 'DEBIAN_FRONTEND': os.environ.get('DEBIAN_FRONTEND', 'noninteractive')}
3008+
3009+ if fatal:
3010+ _run_with_retries(
3011+ cmd, cmd_env=cmd_env, retry_exitcodes=(1, APT_NO_LOCK,),
3012+ retry_message="Couldn't acquire DPKG lock")
3013+ else:
3014+ env = os.environ.copy()
3015+ env.update(cmd_env)
3016+ subprocess.call(cmd, env=env)
3017+
3018+
3019+def get_upstream_version(package):
3020+ """Determine upstream version based on installed package
3021+
3022+ @returns None (if not installed) or the upstream version
3023+ """
3024+ import apt_pkg
3025+ cache = apt_cache()
3026+ try:
3027+ pkg = cache[package]
3028+ except:
3029+ # the package is unknown to the current apt cache.
3030+ return None
3031+
3032+ if not pkg.current_ver:
3033+ # package is known, but no version is currently installed.
3034+ return None
3035+
3036+ return apt_pkg.upstream_version(pkg.current_ver.ver_str)
3037
3038=== added file 'charmhelpers/osplatform.py'
3039--- charmhelpers/osplatform.py 1970-01-01 00:00:00 +0000
3040+++ charmhelpers/osplatform.py 2017-03-04 02:50:20 +0000
3041@@ -0,0 +1,25 @@
3042+import platform
3043+
3044+
3045+def get_platform():
3046+ """Return the current OS platform.
3047+
3048+ For example: if current os platform is Ubuntu then a string "ubuntu"
3049+ will be returned (which is the name of the module).
3050+ This string is used to decide which platform module should be imported.
3051+ """
3052+ # linux_distribution is deprecated and will be removed in Python 3.7
3053+ # Warings *not* disabled, as we certainly need to fix this.
3054+ tuple_platform = platform.linux_distribution()
3055+ current_platform = tuple_platform[0]
3056+ if "Ubuntu" in current_platform:
3057+ return "ubuntu"
3058+ elif "CentOS" in current_platform:
3059+ return "centos"
3060+ elif "debian" in current_platform:
3061+ # Stock Python does not detect Ubuntu and instead returns debian.
3062+ # Or at least it does in some build environments like Travis CI
3063+ return "ubuntu"
3064+ else:
3065+ raise RuntimeError("This module is not supported on {}."
3066+ .format(current_platform))
3067
3068=== modified file 'lib/apt.py'
3069--- lib/apt.py 2017-03-01 15:05:23 +0000
3070+++ lib/apt.py 2017-03-04 02:50:20 +0000
3071@@ -14,7 +14,8 @@
3072
3073 LANDSCAPE_PACKAGES = ("landscape-server", "landscape-hashids")
3074 INSTALL_PACKAGES = LANDSCAPE_PACKAGES + ("python-minimal", "python-psutil")
3075-PACKAGES_DEV = ("dpkg-dev", "devscripts", "pbuilder", "aptitude", "build-essential")
3076+PACKAGES_DEV = (
3077+ "dpkg-dev", "devscripts", "pbuilder", "aptitude", "build-essential")
3078 TARBALL = "landscape-server_*.tar.gz"
3079
3080 # XXX Eventually we'll want to use a dedicated PPA, populated by Jenkins.

Subscribers

People subscribed via source and target branches