Merge lp:landscape-charm into lp:landscape-charm/stable

Proposed by mastier1
Status: Superseded
Proposed branch: lp:landscape-charm
Merge into: lp:landscape-charm/stable
Diff against target: 9376 lines (+5057/-1599) (has conflicts)
84 files modified
.bzrignore (+1/-2)
HACKING.md (+44/-13)
Makefile (+30/-21)
README.md (+46/-28)
actions.yaml (+5/-5)
charm-helpers.yaml (+1/-0)
charmhelpers/__init__.py (+76/-17)
charmhelpers/contrib/__init__.py (+11/-13)
charmhelpers/contrib/hahelpers/__init__.py (+11/-13)
charmhelpers/contrib/hahelpers/apache.py (+24/-20)
charmhelpers/contrib/hahelpers/cluster.py (+113/-23)
charmhelpers/core/__init__.py (+11/-13)
charmhelpers/core/decorators.py (+11/-13)
charmhelpers/core/files.py (+11/-13)
charmhelpers/core/fstab.py (+11/-13)
charmhelpers/core/hookenv.py (+548/-36)
charmhelpers/core/host.py (+602/-166)
charmhelpers/core/host_factory/centos.py (+72/-0)
charmhelpers/core/host_factory/ubuntu.py (+114/-0)
charmhelpers/core/hugepage.py (+11/-13)
charmhelpers/core/kernel.py (+34/-30)
charmhelpers/core/kernel_factory/centos.py (+17/-0)
charmhelpers/core/kernel_factory/ubuntu.py (+13/-0)
charmhelpers/core/services/__init__.py (+11/-13)
charmhelpers/core/services/base.py (+29/-20)
charmhelpers/core/services/helpers.py (+11/-13)
charmhelpers/core/strutils.py (+75/-18)
charmhelpers/core/sysctl.py (+32/-23)
charmhelpers/core/templating.py (+37/-25)
charmhelpers/core/unitdata.py (+19/-15)
charmhelpers/fetch/__init__.py (+53/-302)
charmhelpers/fetch/archiveurl.py (+12/-14)
charmhelpers/fetch/bzrurl.py (+42/-48)
charmhelpers/fetch/centos.py (+171/-0)
charmhelpers/fetch/giturl.py (+33/-33)
charmhelpers/fetch/python/__init__.py (+13/-0)
charmhelpers/fetch/python/debug.py (+54/-0)
charmhelpers/fetch/python/packages.py (+154/-0)
charmhelpers/fetch/python/rpdb.py (+56/-0)
charmhelpers/fetch/python/version.py (+32/-0)
charmhelpers/fetch/snap.py (+150/-0)
charmhelpers/fetch/ubuntu.py (+730/-0)
charmhelpers/osplatform.py (+25/-0)
config.yaml (+34/-0)
dev/charm_helpers_sync.py (+32/-24)
dev/deployer (+29/-30)
dev/ubuntu-deps (+13/-10)
hooks/install (+1/-1)
icon.svg (+1/-288)
lib/action.py (+1/-1)
lib/apt.py (+60/-26)
lib/bootstrap.py (+21/-15)
lib/callbacks/scripts.py (+48/-2)
lib/callbacks/tests/test_apt.py (+1/-1)
lib/callbacks/tests/test_scripts.py (+43/-2)
lib/paths.py (+1/-0)
lib/relations/haproxy.py (+57/-6)
lib/relations/hosted.py (+61/-2)
lib/relations/postgresql.py (+30/-35)
lib/relations/tests/test_haproxy.py (+175/-14)
lib/relations/tests/test_hosted.py (+148/-8)
lib/relations/tests/test_postgresql.py (+18/-17)
lib/services.py (+7/-3)
lib/tests/rootdir.py (+1/-1)
lib/tests/sample.py (+4/-3)
lib/tests/stubs.py (+19/-2)
lib/tests/test_apt.py (+69/-28)
lib/tests/test_bootstrap.py (+46/-13)
lib/tests/test_install.py (+6/-5)
lib/tests/test_services.py (+16/-4)
lib/tests/test_templates.py (+95/-0)
lib/tests/test_upgrade.py (+67/-4)
lib/tests/test_utils.py (+150/-1)
lib/utils.py (+71/-0)
metadata.yaml (+4/-0)
templates/landscape-server (+4/-0)
templates/service.conf (+21/-0)
tests/basic/test_actions.py (+1/-1)
tests/basic/test_ha.py (+21/-21)
tests/basic/test_leader.py (+2/-4)
tests/basic/test_service.py (+20/-3)
tests/helpers.py (+90/-38)
tests/layers.py (+11/-5)
tests/test_helpers.py (+2/-5)
Text conflict in Makefile
Text conflict in README.md
Text conflict in config.yaml
Text conflict in tests/basic/test_service.py
To merge this branch: bzr merge lp:landscape-charm
Reviewer Review Type Date Requested Status
Simon Poirier Needs Resubmitting
Review via email: mp+401061@code.launchpad.net

This proposal has been superseded by a proposal from 2021-04-14.

Commit message

add oidc-* options

Description of the change

Added changes to support OpenID-Connect options.

To post a comment you must log in.
Revision history for this message
Simon Poirier (simpoir) wrote :

This doesn't seem to point to the right branches. This MP doesn't seem to contain anything related to oidc config.
Please branch from an up-to-date lp:landscape-charm and submit an MP of your branch targeting lp:landscape-charm.

review: Needs Resubmitting
lp:landscape-charm updated
403. By Simon Poirier on 2021-04-23

Merge landscape-charm [f=1923863] [r=simpoir] [a=mastier1]
add oidc-* options

Unmerged revisions

403. By Simon Poirier on 2021-04-23

Merge landscape-charm [f=1923863] [r=simpoir] [a=mastier1]
add oidc-* options

402. By Simon Poirier on 2020-05-26

Merge pass_ping_through_https [f=1878265] [r=roadmr,landscape-builder] [a=Simon Poirier]
Add ping service to the https frontend in haproxy.

401. By Simon Poirier on 2020-02-27

Merge trivial_doc_updates [f=1846394,1864699] [r=landscape-builder,cjohnston,roadmr] [a=Simon Poirier]
Trivial documentation updates on config and actions.

400. By Simon Poirier on 2019-10-21

Update to 19.10 release PPA

399. By Guillermo Gonzalez on 2019-07-26

use postgresql-charm v2 protocol, manually parse the dsn as the available psycopg2 (2.6.x) doesn't provide parse_dsn (added in 2.7.x)

398. By Simon Poirier on 2019-05-30

Use API to bootstrap to enable passing registration_key.

397. By Simon Poirier on 2019-05-24

This branch updates charm helpers, and add the fix proposed as
https://github.com/juju/charm-helpers/pull/326

This should fix apt failures when specifying a deb source and key.

396. By Adam Collard on 2019-01-24

Switch to 19.01, update README

395. By Adam Collard on 2019-01-17

Add support for the harmonised configuration keys for setting sources.list: install_sources, install_keys.

394. By Adam Collard on 2019-01-17

Revert r393, r392

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file '.bzrignore'
2--- .bzrignore 2015-07-09 09:00:17 +0000
3+++ .bzrignore 2021-04-13 19:16:05 +0000
4@@ -1,7 +1,6 @@
5 _trial_temp
6 bundles
7-config/license-file
8-config/repo-file
9+config
10 hooks/_trial_temp
11 tags
12 secrets
13
14=== modified file 'HACKING.md'
15--- HACKING.md 2014-03-04 11:36:06 +0000
16+++ HACKING.md 2021-04-13 19:16:05 +0000
17@@ -9,20 +9,51 @@
18 Integration Testing
19 ===================
20
21-This charm comes with an integration test suite. You can run it as follows:
22+This charm comes with an integration test suite, which lives in the 'tests'
23+directory. You can run it as follows:
24
25 make integration-test
26
27-N.B., It will deploy into a real juju environment. It uses the juju-test
28-command to facilitate this, which takes care of bootstrapping for you. It
29-will use whatever 'juju env' reports as your current environment. It will use
30-a number of machines to do this test -- it should work on a local environment
31-(LXC), but could be quite resource intensive.
32-
33-Please also note that if you want to deploy the current charm branch you are
34-working on, you need to change the branch URL in the landscape-deployments.yaml
35-file to point to your branch and commit all changes. After you are done with
36-testing, remember to return the branch URL to what it was before
37+N.B., It will deploy into a the current juju model. It uses the bundletester
38+command to facilitate this. the `JUJU_MODEL` environment variable can be passed
39+to specify a different model. It will use a number of machines to do this test
40+-- it should work on a local controller (LXD), but could be quite resource
41+intensive.
42+
43+
44+Running parts of the integration tests
45+--------------------------------------
46+
47+Running all the integration tests may take quite a while, and sometimes
48+you want to just run the ones that you are working on. To do that, you
49+can bootstrap the Juju environment yourself, and then use the zope
50+testrunner directly. For example:
51+
52+ zope-testrunner3 -vv --path tests --tests-pattern basic --test some_test
53+
54+If the different services already are deployed, the command above is enough.
55+But if you run it against an empty environment, you have to remember to pass
56+along environment variables that affect the deployment, such as
57+LS_CHARM_SOURCE=lds-trunk-ppa as to generate the secrets and bundles:
58+
59+ make secrets bundles
60+
61+
62+Running integration tests on dense MAAS deployment
63+--------------------------------------------------
64+
65+It's possible to run the integration tests on a dense MAAS deployment,
66+where all the services run on the bootstrap node. But given that it's a
67+bit different from other deployments, you have to explicitly tell that
68+it's such a deployment using the DENSE_MAAS environment variable. For
69+example:
70+
71+ DENSE_MAAS=1 make integration-test-trunk
72+
73+Or, if you already have the environment bootstrapped:
74+
75+ DENSE_MAAS=1 LS_CHARM_SOURCE=lds-trunk-ppa zope-testrunner3 -vv \
76+ --path tests --tests-pattern basic --test some_test
77
78
79 Testing with Dependent Upstream Charms
80@@ -33,5 +64,5 @@
81
82 make update-charm-revision-numbers
83
84-After running this, you can use bzr diff to see what (if any) changes were
85-made to landscape-deployments.yaml.
86+After running this, you can use 'bzr diff bundles' to see what (if any)
87+changes were made to landscape-deployments.yaml.
88
89=== modified file 'Makefile'
90--- Makefile 2016-02-10 22:05:36 +0000
91+++ Makefile 2021-04-13 19:16:05 +0000
92@@ -2,35 +2,45 @@
93
94 test:
95 trial lib
96+ # For now only the install hook runs against python3
97+ trial3 lib/tests/test_apt.py lib/tests/test_install.py
98
99 ci-test:
100 ./dev/ubuntu-deps
101 $(MAKE) test lint
102
103-verify-juju-test:
104- @echo "Checking for ... "
105- @echo -n "juju-test: "
106- @if [ -z `which juju-test` ]; then \
107- echo -e "\nRun ./dev/ubuntu-deps to get the juju-test command installed"; \
108- exit 1;\
109- else \
110- echo "installed"; \
111- fi
112-
113 update-charm-revision-numbers: bundles
114 @dev/update-charm-revision-numbers \
115 $(EXTRA_UPDATE_ARGUMENTS) \
116 apache2 postgresql juju-gui haproxy rabbitmq-server nfs
117
118-test-depends: verify-juju-test bundles
119- @cd tests && python3 test_helpers.py
120+test-depends: bundles
121+ pip install --user bundletester juju-deployer
122+ pip3 install --user amulet
123+ cd tests && python3 test_helpers.py
124
125-bundles:
126+bundles-checkout:
127 @if [ -d bundles ]; then \
128 bzr up bundles; \
129 else \
130+<<<<<<< TREE
131 bzr co lp:~landscape/landscape-charm/bundles-stable bundles; \
132 fi
133+=======
134+ bzr co lp:landscape-bundles bundles; \
135+ fi; \
136+ make -C bundles deps
137+ make -C bundles clean
138+
139+bundles: bundles-checkout
140+ bundles/render-bundles
141+
142+bundles-local-branch: bundles-checkout
143+ bundles/render-bundles --landscape-branch $(CURDIR)
144+
145+bundles-local-charm: bundles-checkout
146+ bundles/render-bundles --landscape-charm $(CURDIR)
147+>>>>>>> MERGE-SOURCE
148
149 secrets:
150 @if [ -d secrets ]; then \
151@@ -40,29 +50,29 @@
152 fi
153
154 integration-test: test-depends
155- juju test --set-e -p LS_CHARM_SOURCE,JUJU_HOME,JUJU_ENV,PG_MANUAL_TUNING -v --timeout 3000s
156+ python2 ~/.local/bin/bundletester -v -l DEBUG --skip-implicit -t .
157
158 # Run integration tests using the LDS package from the lds-trunk PPA
159 integration-test-trunk: secrets
160 LS_CHARM_SOURCE=lds-trunk-ppa $(MAKE) $(subst -trunk,,$@)
161
162-deploy-dense-maas: bundles
163+deploy-dense-maas: bundles-local-branch config
164 ./dev/deployer dense-maas
165
166-deploy-dense-maas-dev: bundles
167+deploy-dense-maas-dev: bundles-local-branch config repo-file-trunk
168 ./dev/deployer dense-maas --flags juju-debug
169
170-deploy: bundles
171+deploy: bundles-local-branch
172 ./dev/deployer scalable
173
174-repo-file-trunk: secrets
175+repo-file-trunk: secrets config
176 grep -e "^source:" secrets/lds-trunk-ppa | cut -f 2- -d " " > config/repo-file
177
178 lint:
179 flake8 --filename='*' hooks
180 flake8 lib tests
181 pyflakes3 tests dev/update-charm-revision-numbers
182- find . -name *.py -not -path "./old/*" -not -path "*/charmhelpers/*" -print0 | xargs -0 flake8
183+ find . -name *.py -not -path "./old/*" -not -path "./build/*" -not -path "*/charmhelpers/*" -print0 | xargs -0 flake8
184 flake8 tests dev/update-charm-revision-numbers
185
186 clean:
187@@ -83,8 +93,7 @@
188
189 dev/charm_helpers_sync.py:
190 @mkdir -p dev
191- @bzr cat lp:charm-helpers/tools/charm_helpers_sync/charm_helpers_sync.py \
192- > dev/charm_helpers_sync.py
193+ @curl https://git.launchpad.net/charm-helpers/plain/tools/charm_helpers_sync/charm_helpers_sync.py > dev/charm_helpers_sync.py
194
195 sync: dev/charm_helpers_sync.py
196 $(PYTHON) dev/charm_helpers_sync.py -c charm-helpers.yaml
197
198=== modified file 'README.md'
199--- README.md 2015-10-14 14:56:41 +0000
200+++ README.md 2021-04-13 19:16:05 +0000
201@@ -1,43 +1,37 @@
202 Overview
203 ========
204-
205-The Landscape systems management tool helps you monitor, manage and update your
206-entire Ubuntu infrastructure from a single interface. Part of Canonical's
207-Ubuntu Advantage support service, Landscape brings you intuitive systems
208-management tools combined with world-class support.
209-
210-This charm will deploy Landscape Dedicated Server (LDS), and needs to be
211-connected to other charms to be fully functional. Example deployments are given
212-below.
213+The Landscape systems management tool helps you monitor, manage and update your entire Ubuntu infrastructure from a single interface. Part of Canonical's Ubuntu Advantage support service, Landscape brings you intuitive systems management tools combined with world-class support.
214+
215+This charm will deploy Landscape On Premises, and needs to be connected to other charms to be fully functional. Example deployments are given below.
216
217 For more information about Landscape, go to http://www.ubuntu.com/management
218
219 Standard Usage
220 ==============
221
222-The typical deployment of Landscape happens using a Juju bundle. This charm is
223-not useful without a deployed bundle of services.
224+The typical deployment of Landscape happens using a Juju bundle. This charm is not useful without a deployed bundle of services.
225
226 Please use one of the following bundle types depending on your needs:
227
228+<<<<<<< TREE
229 * [landscape-scalable](https://jujucharms.com/u/landscape/landscape-scalable/)
230 * [landscape-dense-maas](https://jujucharms.com/u/landscape/landscape-dense-maas/)
231 * [landscape-dense](https://jujucharms.com/u/landscape/landscape-dense/)
232+=======
233+* https://jujucharms.com/landscape-scalable/
234+* https://jujucharms.com/landscape-dense-maas/
235+* https://jujucharms.com/landscape-dense/
236+>>>>>>> MERGE-SOURCE
237
238 For the landscape-scalable case:
239
240- sudo apt-add-repository ppa:juju/stable
241- sudo apt-get update
242- juju quickstart u/landscape/landscape-scalable
243-
244-
245-Customized Deployments
246+ $ juju deploy landscape-scalable
247+
248+
249+Customised Deployments
250 ======================
251
252-The standard deployment of Landscape will give you the latest released code.
253-If you want a different version, different options, etc, you will need to
254-download one of the bundles, and add/change options in the file before
255-supplying it to juju quickstart.
256+The standard deployment of Landscape will give you the latest released code. If you want a different version, different options, etc, you will need to download one of the bundles, and add/change options in the file before supplying it to juju.
257
258 On the bundle page, download the `bundle.yaml` file.
259
260@@ -45,6 +39,7 @@
261 Configuration
262 =============
263
264+<<<<<<< TREE
265 Landscape is a commercial product which is bundled with a license
266 allowing management of up to 10 physical machines and 50 more LXC
267 containers, for a total of 60 seats.
268@@ -55,34 +50,44 @@
269 gather these details after purchasing seats for LDS. All information
270 is found by following a link on the left side of the page called
271 "access the Landscape Dedicated Server archive"
272+=======
273+Landscape is a commercial product and as such it needs configuration of a license and password protected repository before deployment. Please login to your "hosted account" (on landscape.canonical.com) to gather these details after purchasing seats for Landscape On Premises. All information is found by following a link on the left side of the page called "access the Landscape On Premises archive"
274+>>>>>>> MERGE-SOURCE
275
276 license-file
277 ------------
278
279+<<<<<<< TREE
280 You can set this as a Juju configuration option after deployment
281 on the landscape service like this:
282+=======
283+You can set this as a juju configuration option after deployment on each deployed landscape-server application like:
284+>>>>>>> MERGE-SOURCE
285
286+<<<<<<< TREE
287 $ juju set landscape-server "license-file=$(cat license-file)"
288+=======
289+ $ juju config landscape-server "license-file=$(cat license-file)"
290+>>>>>>> MERGE-SOURCE
291
292
293 SSL
294 ===
295
296-The pre-packaged bundles will ask the HAProxy charm to generate a self
297-signed certificate. While useful for testing, this must not be used for
298-production deployments.
299+The pre-packaged bundles will ask the HAProxy charm to generate a self signed certificate. While useful for testing, this must not be used for production deployments.
300
301-For production deployments, you should include a "real" SSL certificate key
302-pair that has been signed by a CA that your clients trust in the haproxy service
303-configuration (or in the landscape-server service configuration if you need to
304-use your haproxy service for other services too with different certificates).
305+For production deployments, you should include a "real" SSL certificate key pair that has been signed by a CA that your clients trust in the haproxy service configuration (or in the landscape-server service configuration if you need to use your haproxy service for other services too with different certificates).
306
307
308 Unit Testing
309 ============
310
311+<<<<<<< TREE
312 The Landscape charm is unit tested and new code changes should be
313 submitted with unit tests. You can run them like this:
314+=======
315+The Landscape charm is fairly well unit tested and new code changes should be submitted with unit tests. You can run them like this:
316+>>>>>>> MERGE-SOURCE
317
318 $ make test
319
320@@ -90,6 +95,7 @@
321 Integration Testing
322 ===================
323
324+<<<<<<< TREE
325 This charm makes use of
326 [juju-deployer](http://pythonhosted.org/juju-deployer/) and
327 [charm-tools](https://jujucharms.com/docs/1.20/tools-charm-tools) to
328@@ -105,3 +111,15 @@
329
330 The JUJU_ENV environment variable can be omitted if you want to use the
331 current juju environment (as set by "juju switch").
332+=======
333+This charm makes use of juju-deployer and the charm-tools package to enable end-to-end integration testing. This is how you proceed with running them:
334+
335+ $ juju bootstrap localhost
336+ $ make integration-test
337+
338+Or if you want to use the packages from the lds-trunk PPA:
339+
340+ $ JUJU_MODEL=<model> make integration-test-trunk
341+
342+The JUJU_MODEL environment variable can be omitted if you want to use the current model.
343+>>>>>>> MERGE-SOURCE
344
345=== modified file 'actions.yaml'
346--- actions.yaml 2015-06-25 16:01:22 +0000
347+++ actions.yaml 2021-04-13 19:16:05 +0000
348@@ -1,10 +1,7 @@
349 pause:
350- description: Pause the Landscape unit. This action will interrupt any
351- Landscape-related processing on the unit and prevent any further processing
352- from happening.
353+ description: Pause the Landscape service.
354 resume:
355- description: Resume the Landscape unit. This action will start all the
356- Landscape services on the unit.
357+ description: Resume the Landscape service.
358 upgrade:
359 description: Upgrade software on the Landscape unit. This action will update
360 APT package indexes and upgrade the landscape-server package.
361@@ -25,5 +22,8 @@
362 admin-password:
363 type: string
364 description: Password for the administrator to add.
365+ registration-key:
366+ type: string
367+ description: Registration key to set on the account.
368 required: [admin-name, admin-email, admin-password]
369 additionalProperties: false
370
371=== modified file 'charm-helpers.yaml'
372--- charm-helpers.yaml 2015-05-07 10:26:45 +0000
373+++ charm-helpers.yaml 2021-04-13 19:16:05 +0000
374@@ -4,4 +4,5 @@
375 - __init__
376 - core
377 - fetch
378+ - osplatform
379 - contrib.hahelpers
380
381=== modified file 'charmhelpers/__init__.py'
382--- charmhelpers/__init__.py 2015-01-28 08:59:02 +0000
383+++ charmhelpers/__init__.py 2021-04-13 19:16:05 +0000
384@@ -1,38 +1,97 @@
385 # Copyright 2014-2015 Canonical Limited.
386 #
387-# This file is part of charm-helpers.
388-#
389-# charm-helpers is free software: you can redistribute it and/or modify
390-# it under the terms of the GNU Lesser General Public License version 3 as
391-# published by the Free Software Foundation.
392-#
393-# charm-helpers is distributed in the hope that it will be useful,
394-# but WITHOUT ANY WARRANTY; without even the implied warranty of
395-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
396-# GNU Lesser General Public License for more details.
397-#
398-# You should have received a copy of the GNU Lesser General Public License
399-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
400+# Licensed under the Apache License, Version 2.0 (the "License");
401+# you may not use this file except in compliance with the License.
402+# You may obtain a copy of the License at
403+#
404+# http://www.apache.org/licenses/LICENSE-2.0
405+#
406+# Unless required by applicable law or agreed to in writing, software
407+# distributed under the License is distributed on an "AS IS" BASIS,
408+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
409+# See the License for the specific language governing permissions and
410+# limitations under the License.
411
412 # Bootstrap charm-helpers, installing its dependencies if necessary using
413 # only standard libraries.
414+from __future__ import print_function
415+from __future__ import absolute_import
416+
417+import functools
418+import inspect
419 import subprocess
420 import sys
421
422 try:
423- import six # flake8: noqa
424+ import six # NOQA:F401
425 except ImportError:
426 if sys.version_info.major == 2:
427 subprocess.check_call(['apt-get', 'install', '-y', 'python-six'])
428 else:
429 subprocess.check_call(['apt-get', 'install', '-y', 'python3-six'])
430- import six # flake8: noqa
431+ import six # NOQA:F401
432
433 try:
434- import yaml # flake8: noqa
435+ import yaml # NOQA:F401
436 except ImportError:
437 if sys.version_info.major == 2:
438 subprocess.check_call(['apt-get', 'install', '-y', 'python-yaml'])
439 else:
440 subprocess.check_call(['apt-get', 'install', '-y', 'python3-yaml'])
441- import yaml # flake8: noqa
442+ import yaml # NOQA:F401
443+
444+
445+# Holds a list of mapping of mangled function names that have been deprecated
446+# using the @deprecate decorator below. This is so that the warning is only
447+# printed once for each usage of the function.
448+__deprecated_functions = {}
449+
450+
451+def deprecate(warning, date=None, log=None):
452+ """Add a deprecation warning the first time the function is used.
453+ The date, which is a string in semi-ISO8660 format indicate the year-month
454+ that the function is officially going to be removed.
455+
456+ usage:
457+
458+ @deprecate('use core/fetch/add_source() instead', '2017-04')
459+ def contributed_add_source_thing(...):
460+ ...
461+
462+ And it then prints to the log ONCE that the function is deprecated.
463+ The reason for passing the logging function (log) is so that hookenv.log
464+ can be used for a charm if needed.
465+
466+ :param warning: String to indicat where it has moved ot.
467+ :param date: optional sting, in YYYY-MM format to indicate when the
468+ function will definitely (probably) be removed.
469+ :param log: The log function to call to log. If not, logs to stdout
470+ """
471+ def wrap(f):
472+
473+ @functools.wraps(f)
474+ def wrapped_f(*args, **kwargs):
475+ try:
476+ module = inspect.getmodule(f)
477+ file = inspect.getsourcefile(f)
478+ lines = inspect.getsourcelines(f)
479+ f_name = "{}-{}-{}..{}-{}".format(
480+ module.__name__, file, lines[0], lines[-1], f.__name__)
481+ except (IOError, TypeError):
482+ # assume it was local, so just use the name of the function
483+ f_name = f.__name__
484+ if f_name not in __deprecated_functions:
485+ __deprecated_functions[f_name] = True
486+ s = "DEPRECATION WARNING: Function {} is being removed".format(
487+ f.__name__)
488+ if date:
489+ s = "{} on/around {}".format(s, date)
490+ if warning:
491+ s = "{} : {}".format(s, warning)
492+ if log:
493+ log(s)
494+ else:
495+ print(s)
496+ return f(*args, **kwargs)
497+ return wrapped_f
498+ return wrap
499
500=== modified file 'charmhelpers/contrib/__init__.py'
501--- charmhelpers/contrib/__init__.py 2015-01-30 11:16:09 +0000
502+++ charmhelpers/contrib/__init__.py 2021-04-13 19:16:05 +0000
503@@ -1,15 +1,13 @@
504 # Copyright 2014-2015 Canonical Limited.
505 #
506-# This file is part of charm-helpers.
507-#
508-# charm-helpers is free software: you can redistribute it and/or modify
509-# it under the terms of the GNU Lesser General Public License version 3 as
510-# published by the Free Software Foundation.
511-#
512-# charm-helpers is distributed in the hope that it will be useful,
513-# but WITHOUT ANY WARRANTY; without even the implied warranty of
514-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
515-# GNU Lesser General Public License for more details.
516-#
517-# You should have received a copy of the GNU Lesser General Public License
518-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
519+# Licensed under the Apache License, Version 2.0 (the "License");
520+# you may not use this file except in compliance with the License.
521+# You may obtain a copy of the License at
522+#
523+# http://www.apache.org/licenses/LICENSE-2.0
524+#
525+# Unless required by applicable law or agreed to in writing, software
526+# distributed under the License is distributed on an "AS IS" BASIS,
527+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
528+# See the License for the specific language governing permissions and
529+# limitations under the License.
530
531=== modified file 'charmhelpers/contrib/hahelpers/__init__.py'
532--- charmhelpers/contrib/hahelpers/__init__.py 2015-01-30 11:16:09 +0000
533+++ charmhelpers/contrib/hahelpers/__init__.py 2021-04-13 19:16:05 +0000
534@@ -1,15 +1,13 @@
535 # Copyright 2014-2015 Canonical Limited.
536 #
537-# This file is part of charm-helpers.
538-#
539-# charm-helpers is free software: you can redistribute it and/or modify
540-# it under the terms of the GNU Lesser General Public License version 3 as
541-# published by the Free Software Foundation.
542-#
543-# charm-helpers is distributed in the hope that it will be useful,
544-# but WITHOUT ANY WARRANTY; without even the implied warranty of
545-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
546-# GNU Lesser General Public License for more details.
547-#
548-# You should have received a copy of the GNU Lesser General Public License
549-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
550+# Licensed under the Apache License, Version 2.0 (the "License");
551+# you may not use this file except in compliance with the License.
552+# You may obtain a copy of the License at
553+#
554+# http://www.apache.org/licenses/LICENSE-2.0
555+#
556+# Unless required by applicable law or agreed to in writing, software
557+# distributed under the License is distributed on an "AS IS" BASIS,
558+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
559+# See the License for the specific language governing permissions and
560+# limitations under the License.
561
562=== modified file 'charmhelpers/contrib/hahelpers/apache.py'
563--- charmhelpers/contrib/hahelpers/apache.py 2015-01-30 11:16:09 +0000
564+++ charmhelpers/contrib/hahelpers/apache.py 2021-04-13 19:16:05 +0000
565@@ -1,18 +1,16 @@
566 # Copyright 2014-2015 Canonical Limited.
567 #
568-# This file is part of charm-helpers.
569-#
570-# charm-helpers is free software: you can redistribute it and/or modify
571-# it under the terms of the GNU Lesser General Public License version 3 as
572-# published by the Free Software Foundation.
573-#
574-# charm-helpers is distributed in the hope that it will be useful,
575-# but WITHOUT ANY WARRANTY; without even the implied warranty of
576-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
577-# GNU Lesser General Public License for more details.
578-#
579-# You should have received a copy of the GNU Lesser General Public License
580-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
581+# Licensed under the Apache License, Version 2.0 (the "License");
582+# you may not use this file except in compliance with the License.
583+# You may obtain a copy of the License at
584+#
585+# http://www.apache.org/licenses/LICENSE-2.0
586+#
587+# Unless required by applicable law or agreed to in writing, software
588+# distributed under the License is distributed on an "AS IS" BASIS,
589+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
590+# See the License for the specific language governing permissions and
591+# limitations under the License.
592
593 #
594 # Copyright 2012 Canonical Ltd.
595@@ -24,8 +22,9 @@
596 # Adam Gandelman <adamg@ubuntu.com>
597 #
598
599-import subprocess
600+import os
601
602+from charmhelpers.core import host
603 from charmhelpers.core.hookenv import (
604 config as config_get,
605 relation_get,
606@@ -66,7 +65,8 @@
607 if ca_cert is None:
608 log("Inspecting identity-service relations for CA SSL certificate.",
609 level=INFO)
610- for r_id in relation_ids('identity-service'):
611+ for r_id in (relation_ids('identity-service') +
612+ relation_ids('identity-credentials')):
613 for unit in relation_list(r_id):
614 if ca_cert is None:
615 ca_cert = relation_get('ca_cert',
616@@ -74,9 +74,13 @@
617 return ca_cert
618
619
620+def retrieve_ca_cert(cert_file):
621+ cert = None
622+ if os.path.isfile(cert_file):
623+ with open(cert_file, 'rb') as crt:
624+ cert = crt.read()
625+ return cert
626+
627+
628 def install_ca_cert(ca_cert):
629- if ca_cert:
630- with open('/usr/local/share/ca-certificates/keystone_juju_ca_cert.crt',
631- 'w') as crt:
632- crt.write(ca_cert)
633- subprocess.check_call(['update-ca-certificates', '--fresh'])
634+ host.install_ca_cert(ca_cert, 'keystone_juju_ca_cert')
635
636=== modified file 'charmhelpers/contrib/hahelpers/cluster.py'
637--- charmhelpers/contrib/hahelpers/cluster.py 2015-07-03 09:13:26 +0000
638+++ charmhelpers/contrib/hahelpers/cluster.py 2021-04-13 19:16:05 +0000
639@@ -1,18 +1,16 @@
640 # Copyright 2014-2015 Canonical Limited.
641 #
642-# This file is part of charm-helpers.
643-#
644-# charm-helpers is free software: you can redistribute it and/or modify
645-# it under the terms of the GNU Lesser General Public License version 3 as
646-# published by the Free Software Foundation.
647-#
648-# charm-helpers is distributed in the hope that it will be useful,
649-# but WITHOUT ANY WARRANTY; without even the implied warranty of
650-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
651-# GNU Lesser General Public License for more details.
652-#
653-# You should have received a copy of the GNU Lesser General Public License
654-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
655+# Licensed under the Apache License, Version 2.0 (the "License");
656+# you may not use this file except in compliance with the License.
657+# You may obtain a copy of the License at
658+#
659+# http://www.apache.org/licenses/LICENSE-2.0
660+#
661+# Unless required by applicable law or agreed to in writing, software
662+# distributed under the License is distributed on an "AS IS" BASIS,
663+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
664+# See the License for the specific language governing permissions and
665+# limitations under the License.
666
667 #
668 # Copyright 2012 Canonical Ltd.
669@@ -29,6 +27,7 @@
670
671 import subprocess
672 import os
673+import time
674
675 from socket import gethostname as get_unit_hostname
676
677@@ -41,10 +40,14 @@
678 relation_get,
679 config as config_get,
680 INFO,
681- ERROR,
682+ DEBUG,
683 WARNING,
684 unit_get,
685- is_leader as juju_is_leader
686+ is_leader as juju_is_leader,
687+ status_set,
688+)
689+from charmhelpers.core.host import (
690+ modulo_distribution,
691 )
692 from charmhelpers.core.decorators import (
693 retry_on_exception,
694@@ -60,6 +63,10 @@
695 pass
696
697
698+class HAIncorrectConfig(Exception):
699+ pass
700+
701+
702 class CRMResourceNotFound(Exception):
703 pass
704
705@@ -216,6 +223,11 @@
706 return True
707 if config_get('ssl_cert') and config_get('ssl_key'):
708 return True
709+ for r_id in relation_ids('certificates'):
710+ for unit in relation_list(r_id):
711+ ca = relation_get('ca', rid=r_id, unit=unit)
712+ if ca:
713+ return True
714 for r_id in relation_ids('identity-service'):
715 for unit in relation_list(r_id):
716 # TODO - needs fixing for new helper as ssl_cert/key suffixes with CN
717@@ -274,27 +286,71 @@
718 Obtains all relevant configuration from charm configuration required
719 for initiating a relation to hacluster:
720
721- ha-bindiface, ha-mcastport, vip
722+ ha-bindiface, ha-mcastport, vip, os-internal-hostname,
723+ os-admin-hostname, os-public-hostname, os-access-hostname
724
725 param: exclude_keys: list of setting key(s) to be excluded.
726 returns: dict: A dict containing settings keyed by setting name.
727- raises: HAIncompleteConfig if settings are missing.
728+ raises: HAIncompleteConfig if settings are missing or incorrect.
729 '''
730- settings = ['ha-bindiface', 'ha-mcastport', 'vip']
731+ settings = ['ha-bindiface', 'ha-mcastport', 'vip', 'os-internal-hostname',
732+ 'os-admin-hostname', 'os-public-hostname', 'os-access-hostname']
733 conf = {}
734 for setting in settings:
735 if exclude_keys and setting in exclude_keys:
736 continue
737
738 conf[setting] = config_get(setting)
739- missing = []
740- [missing.append(s) for s, v in six.iteritems(conf) if v is None]
741- if missing:
742- log('Insufficient config data to configure hacluster.', level=ERROR)
743- raise HAIncompleteConfig
744+
745+ if not valid_hacluster_config():
746+ raise HAIncorrectConfig('Insufficient or incorrect config data to '
747+ 'configure hacluster.')
748 return conf
749
750
751+def valid_hacluster_config():
752+ '''
753+ Check that either vip or dns-ha is set. If dns-ha then one of os-*-hostname
754+ must be set.
755+
756+ Note: ha-bindiface and ha-macastport both have defaults and will always
757+ be set. We only care that either vip or dns-ha is set.
758+
759+ :returns: boolean: valid config returns true.
760+ raises: HAIncompatibileConfig if settings conflict.
761+ raises: HAIncompleteConfig if settings are missing.
762+ '''
763+ vip = config_get('vip')
764+ dns = config_get('dns-ha')
765+ if not(bool(vip) ^ bool(dns)):
766+ msg = ('HA: Either vip or dns-ha must be set but not both in order to '
767+ 'use high availability')
768+ status_set('blocked', msg)
769+ raise HAIncorrectConfig(msg)
770+
771+ # If dns-ha then one of os-*-hostname must be set
772+ if dns:
773+ dns_settings = ['os-internal-hostname', 'os-admin-hostname',
774+ 'os-public-hostname', 'os-access-hostname']
775+ # At this point it is unknown if one or all of the possible
776+ # network spaces are in HA. Validate at least one is set which is
777+ # the minimum required.
778+ for setting in dns_settings:
779+ if config_get(setting):
780+ log('DNS HA: At least one hostname is set {}: {}'
781+ ''.format(setting, config_get(setting)),
782+ level=DEBUG)
783+ return True
784+
785+ msg = ('DNS HA: At least one os-*-hostname(s) must be set to use '
786+ 'DNS HA')
787+ status_set('blocked', msg)
788+ raise HAIncompleteConfig(msg)
789+
790+ log('VIP HA: VIP is set {}'.format(vip), level=DEBUG)
791+ return True
792+
793+
794 def canonical_url(configs, vip_setting='vip'):
795 '''
796 Returns the correct HTTP URL to this host given the state of HTTPS
797@@ -314,3 +370,37 @@
798 else:
799 addr = unit_get('private-address')
800 return '%s://%s' % (scheme, addr)
801+
802+
803+def distributed_wait(modulo=None, wait=None, operation_name='operation'):
804+ ''' Distribute operations by waiting based on modulo_distribution
805+
806+ If modulo and or wait are not set, check config_get for those values.
807+ If config values are not set, default to modulo=3 and wait=30.
808+
809+ :param modulo: int The modulo number creates the group distribution
810+ :param wait: int The constant time wait value
811+ :param operation_name: string Operation name for status message
812+ i.e. 'restart'
813+ :side effect: Calls config_get()
814+ :side effect: Calls log()
815+ :side effect: Calls status_set()
816+ :side effect: Calls time.sleep()
817+ '''
818+ if modulo is None:
819+ modulo = config_get('modulo-nodes') or 3
820+ if wait is None:
821+ wait = config_get('known-wait') or 30
822+ if juju_is_leader():
823+ # The leader should never wait
824+ calculated_wait = 0
825+ else:
826+ # non_zero_wait=True guarantees the non-leader who gets modulo 0
827+ # will still wait
828+ calculated_wait = modulo_distribution(modulo=modulo, wait=wait,
829+ non_zero_wait=True)
830+ msg = "Waiting {} seconds for {} ...".format(calculated_wait,
831+ operation_name)
832+ log(msg, DEBUG)
833+ status_set('maintenance', msg)
834+ time.sleep(calculated_wait)
835
836=== modified file 'charmhelpers/core/__init__.py'
837--- charmhelpers/core/__init__.py 2015-01-28 08:59:02 +0000
838+++ charmhelpers/core/__init__.py 2021-04-13 19:16:05 +0000
839@@ -1,15 +1,13 @@
840 # Copyright 2014-2015 Canonical Limited.
841 #
842-# This file is part of charm-helpers.
843-#
844-# charm-helpers is free software: you can redistribute it and/or modify
845-# it under the terms of the GNU Lesser General Public License version 3 as
846-# published by the Free Software Foundation.
847-#
848-# charm-helpers is distributed in the hope that it will be useful,
849-# but WITHOUT ANY WARRANTY; without even the implied warranty of
850-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
851-# GNU Lesser General Public License for more details.
852-#
853-# You should have received a copy of the GNU Lesser General Public License
854-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
855+# Licensed under the Apache License, Version 2.0 (the "License");
856+# you may not use this file except in compliance with the License.
857+# You may obtain a copy of the License at
858+#
859+# http://www.apache.org/licenses/LICENSE-2.0
860+#
861+# Unless required by applicable law or agreed to in writing, software
862+# distributed under the License is distributed on an "AS IS" BASIS,
863+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
864+# See the License for the specific language governing permissions and
865+# limitations under the License.
866
867=== modified file 'charmhelpers/core/decorators.py'
868--- charmhelpers/core/decorators.py 2015-01-28 08:59:02 +0000
869+++ charmhelpers/core/decorators.py 2021-04-13 19:16:05 +0000
870@@ -1,18 +1,16 @@
871 # Copyright 2014-2015 Canonical Limited.
872 #
873-# This file is part of charm-helpers.
874-#
875-# charm-helpers is free software: you can redistribute it and/or modify
876-# it under the terms of the GNU Lesser General Public License version 3 as
877-# published by the Free Software Foundation.
878-#
879-# charm-helpers is distributed in the hope that it will be useful,
880-# but WITHOUT ANY WARRANTY; without even the implied warranty of
881-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
882-# GNU Lesser General Public License for more details.
883-#
884-# You should have received a copy of the GNU Lesser General Public License
885-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
886+# Licensed under the Apache License, Version 2.0 (the "License");
887+# you may not use this file except in compliance with the License.
888+# You may obtain a copy of the License at
889+#
890+# http://www.apache.org/licenses/LICENSE-2.0
891+#
892+# Unless required by applicable law or agreed to in writing, software
893+# distributed under the License is distributed on an "AS IS" BASIS,
894+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
895+# See the License for the specific language governing permissions and
896+# limitations under the License.
897
898 #
899 # Copyright 2014 Canonical Ltd.
900
901=== modified file 'charmhelpers/core/files.py'
902--- charmhelpers/core/files.py 2015-12-11 15:23:38 +0000
903+++ charmhelpers/core/files.py 2021-04-13 19:16:05 +0000
904@@ -3,19 +3,17 @@
905
906 # Copyright 2014-2015 Canonical Limited.
907 #
908-# This file is part of charm-helpers.
909-#
910-# charm-helpers is free software: you can redistribute it and/or modify
911-# it under the terms of the GNU Lesser General Public License version 3 as
912-# published by the Free Software Foundation.
913-#
914-# charm-helpers is distributed in the hope that it will be useful,
915-# but WITHOUT ANY WARRANTY; without even the implied warranty of
916-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
917-# GNU Lesser General Public License for more details.
918-#
919-# You should have received a copy of the GNU Lesser General Public License
920-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
921+# Licensed under the Apache License, Version 2.0 (the "License");
922+# you may not use this file except in compliance with the License.
923+# You may obtain a copy of the License at
924+#
925+# http://www.apache.org/licenses/LICENSE-2.0
926+#
927+# Unless required by applicable law or agreed to in writing, software
928+# distributed under the License is distributed on an "AS IS" BASIS,
929+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
930+# See the License for the specific language governing permissions and
931+# limitations under the License.
932
933 __author__ = 'Jorge Niedbalski <niedbalski@ubuntu.com>'
934
935
936=== modified file 'charmhelpers/core/fstab.py'
937--- charmhelpers/core/fstab.py 2015-03-12 11:42:26 +0000
938+++ charmhelpers/core/fstab.py 2021-04-13 19:16:05 +0000
939@@ -3,19 +3,17 @@
940
941 # Copyright 2014-2015 Canonical Limited.
942 #
943-# This file is part of charm-helpers.
944-#
945-# charm-helpers is free software: you can redistribute it and/or modify
946-# it under the terms of the GNU Lesser General Public License version 3 as
947-# published by the Free Software Foundation.
948-#
949-# charm-helpers is distributed in the hope that it will be useful,
950-# but WITHOUT ANY WARRANTY; without even the implied warranty of
951-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
952-# GNU Lesser General Public License for more details.
953-#
954-# You should have received a copy of the GNU Lesser General Public License
955-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
956+# Licensed under the Apache License, Version 2.0 (the "License");
957+# you may not use this file except in compliance with the License.
958+# You may obtain a copy of the License at
959+#
960+# http://www.apache.org/licenses/LICENSE-2.0
961+#
962+# Unless required by applicable law or agreed to in writing, software
963+# distributed under the License is distributed on an "AS IS" BASIS,
964+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
965+# See the License for the specific language governing permissions and
966+# limitations under the License.
967
968 import io
969 import os
970
971=== modified file 'charmhelpers/core/hookenv.py'
972--- charmhelpers/core/hookenv.py 2015-12-11 15:23:38 +0000
973+++ charmhelpers/core/hookenv.py 2021-04-13 19:16:05 +0000
974@@ -1,18 +1,16 @@
975 # Copyright 2014-2015 Canonical Limited.
976 #
977-# This file is part of charm-helpers.
978-#
979-# charm-helpers is free software: you can redistribute it and/or modify
980-# it under the terms of the GNU Lesser General Public License version 3 as
981-# published by the Free Software Foundation.
982-#
983-# charm-helpers is distributed in the hope that it will be useful,
984-# but WITHOUT ANY WARRANTY; without even the implied warranty of
985-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
986-# GNU Lesser General Public License for more details.
987-#
988-# You should have received a copy of the GNU Lesser General Public License
989-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
990+# Licensed under the Apache License, Version 2.0 (the "License");
991+# you may not use this file except in compliance with the License.
992+# You may obtain a copy of the License at
993+#
994+# http://www.apache.org/licenses/LICENSE-2.0
995+#
996+# Unless required by applicable law or agreed to in writing, software
997+# distributed under the License is distributed on an "AS IS" BASIS,
998+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
999+# See the License for the specific language governing permissions and
1000+# limitations under the License.
1001
1002 "Interactions with the Juju environment"
1003 # Copyright 2013 Canonical Ltd.
1004@@ -24,10 +22,12 @@
1005 import copy
1006 from distutils.version import LooseVersion
1007 from functools import wraps
1008+from collections import namedtuple
1009 import glob
1010 import os
1011 import json
1012 import yaml
1013+import re
1014 import subprocess
1015 import sys
1016 import errno
1017@@ -40,12 +40,20 @@
1018 else:
1019 from collections import UserDict
1020
1021+
1022 CRITICAL = "CRITICAL"
1023 ERROR = "ERROR"
1024 WARNING = "WARNING"
1025 INFO = "INFO"
1026 DEBUG = "DEBUG"
1027+TRACE = "TRACE"
1028 MARKER = object()
1029+SH_MAX_ARG = 131071
1030+
1031+
1032+RANGE_WARNING = ('Passing NO_PROXY string that includes a cidr. '
1033+ 'This may not be compatible with software you are '
1034+ 'running in your shell.')
1035
1036 cache = {}
1037
1038@@ -66,7 +74,7 @@
1039 @wraps(func)
1040 def wrapper(*args, **kwargs):
1041 global cache
1042- key = str((func, args, kwargs))
1043+ key = json.dumps((func, args, kwargs), sort_keys=True, default=str)
1044 try:
1045 return cache[key]
1046 except KeyError:
1047@@ -96,7 +104,7 @@
1048 command += ['-l', level]
1049 if not isinstance(message, six.string_types):
1050 message = repr(message)
1051- command += [message]
1052+ command += [message[:SH_MAX_ARG]]
1053 # Missing juju-log should not cause failures in unit tests
1054 # Send log output to stderr
1055 try:
1056@@ -199,9 +207,56 @@
1057 return os.environ.get('JUJU_REMOTE_UNIT', None)
1058
1059
1060+def application_name():
1061+ """
1062+ The name of the deployed application this unit belongs to.
1063+ """
1064+ return local_unit().split('/')[0]
1065+
1066+
1067 def service_name():
1068- """The name service group this unit belongs to"""
1069- return local_unit().split('/')[0]
1070+ """
1071+ .. deprecated:: 0.19.1
1072+ Alias for :func:`application_name`.
1073+ """
1074+ return application_name()
1075+
1076+
1077+def model_name():
1078+ """
1079+ Name of the model that this unit is deployed in.
1080+ """
1081+ return os.environ['JUJU_MODEL_NAME']
1082+
1083+
1084+def model_uuid():
1085+ """
1086+ UUID of the model that this unit is deployed in.
1087+ """
1088+ return os.environ['JUJU_MODEL_UUID']
1089+
1090+
1091+def principal_unit():
1092+ """Returns the principal unit of this unit, otherwise None"""
1093+ # Juju 2.2 and above provides JUJU_PRINCIPAL_UNIT
1094+ principal_unit = os.environ.get('JUJU_PRINCIPAL_UNIT', None)
1095+ # If it's empty, then this unit is the principal
1096+ if principal_unit == '':
1097+ return os.environ['JUJU_UNIT_NAME']
1098+ elif principal_unit is not None:
1099+ return principal_unit
1100+ # For Juju 2.1 and below, let's try work out the principle unit by
1101+ # the various charms' metadata.yaml.
1102+ for reltype in relation_types():
1103+ for rid in relation_ids(reltype):
1104+ for unit in related_units(rid):
1105+ md = _metadata_unit(unit)
1106+ if not md:
1107+ continue
1108+ subordinate = md.pop('subordinate', None)
1109+ if not subordinate:
1110+ return unit
1111+ return None
1112
1113
1114 @cached
1115@@ -265,7 +320,7 @@
1116 self.implicit_save = True
1117 self._prev_dict = None
1118 self.path = os.path.join(charm_dir(), Config.CONFIG_FILE_NAME)
1119- if os.path.exists(self.path):
1120+ if os.path.exists(self.path) and os.stat(self.path).st_size:
1121 self.load_previous()
1122 atexit(self._implicit_save)
1123
1124@@ -285,7 +340,11 @@
1125 """
1126 self.path = path or self.path
1127 with open(self.path) as f:
1128- self._prev_dict = json.load(f)
1129+ try:
1130+ self._prev_dict = json.load(f)
1131+ except ValueError as e:
1132+ log('Unable to parse previous config data - {}'.format(str(e)),
1133+ level=ERROR)
1134 for k, v in copy.deepcopy(self._prev_dict).items():
1135 if k not in self:
1136 self[k] = v
1137@@ -321,6 +380,7 @@
1138
1139 """
1140 with open(self.path, 'w') as f:
1141+ os.fchmod(f.fileno(), 0o600)
1142 json.dump(self, f)
1143
1144 def _implicit_save(self):
1145@@ -328,20 +388,40 @@
1146 self.save()
1147
1148
1149-@cached
1150+_cache_config = None
1151+
1152+
1153 def config(scope=None):
1154- """Juju charm configuration"""
1155- config_cmd_line = ['config-get']
1156- if scope is not None:
1157- config_cmd_line.append(scope)
1158- config_cmd_line.append('--format=json')
1159- try:
1160- config_data = json.loads(
1161- subprocess.check_output(config_cmd_line).decode('UTF-8'))
1162+ """
1163+ Get the juju charm configuration (scope==None) or individual key,
1164+ (scope=str). The returned value is a Python data structure loaded as
1165+ JSON from the Juju config command.
1166+
1167+ :param scope: If set, return the value for the specified key.
1168+ :type scope: Optional[str]
1169+ :returns: Either the whole config as a Config, or a key from it.
1170+ :rtype: Any
1171+ """
1172+ global _cache_config
1173+ config_cmd_line = ['config-get', '--all', '--format=json']
1174+ try:
1175+ # JSON Decode Exception for Python3.5+
1176+ exc_json = json.decoder.JSONDecodeError
1177+ except AttributeError:
1178+ # JSON Decode Exception for Python2.7 through Python3.4
1179+ exc_json = ValueError
1180+ try:
1181+ if _cache_config is None:
1182+ config_data = json.loads(
1183+ subprocess.check_output(config_cmd_line).decode('UTF-8'))
1184+ _cache_config = Config(config_data)
1185 if scope is not None:
1186- return config_data
1187- return Config(config_data)
1188- except ValueError:
1189+ return _cache_config.get(scope)
1190+ return _cache_config
1191+ except (exc_json, UnicodeDecodeError) as e:
1192+ log('Unable to parse output from config-get: config_cmd_line="{}" '
1193+ 'message="{}"'
1194+ .format(config_cmd_line, str(e)), level=ERROR)
1195 return None
1196
1197
1198@@ -435,6 +515,67 @@
1199 subprocess.check_output(units_cmd_line).decode('UTF-8')) or []
1200
1201
1202+def expected_peer_units():
1203+ """Get a generator for units we expect to join peer relation based on
1204+ goal-state.
1205+
1206+ The local unit is excluded from the result to make it easy to gauge
1207+ completion of all peers joining the relation with existing hook tools.
1208+
1209+ Example usage:
1210+ log('peer {} of {} joined peer relation'
1211+ .format(len(related_units()),
1212+ len(list(expected_peer_units()))))
1213+
1214+ This function will raise NotImplementedError if used with juju versions
1215+ without goal-state support.
1216+
1217+ :returns: iterator
1218+ :rtype: types.GeneratorType
1219+ :raises: NotImplementedError
1220+ """
1221+ if not has_juju_version("2.4.0"):
1222+ # goal-state first appeared in 2.4.0.
1223+ raise NotImplementedError("goal-state")
1224+ _goal_state = goal_state()
1225+ return (key for key in _goal_state['units']
1226+ if '/' in key and key != local_unit())
1227+
1228+
1229+def expected_related_units(reltype=None):
1230+ """Get a generator for units we expect to join relation based on
1231+ goal-state.
1232+
1233+ Note that you can not use this function for the peer relation, take a look
1234+ at expected_peer_units() for that.
1235+
1236+ This function will raise KeyError if you request information for a
1237+ relation type for which juju goal-state does not have information. It will
1238+ raise NotImplementedError if used with juju versions without goal-state
1239+ support.
1240+
1241+ Example usage:
1242+ log('participant {} of {} joined relation {}'
1243+ .format(len(related_units()),
1244+ len(list(expected_related_units())),
1245+ relation_type()))
1246+
1247+ :param reltype: Relation type to list data for, default is to list data for
1248+ the realtion type we are currently executing a hook for.
1249+ :type reltype: str
1250+ :returns: iterator
1251+ :rtype: types.GeneratorType
1252+ :raises: KeyError, NotImplementedError
1253+ """
1254+ if not has_juju_version("2.4.4"):
1255+ # goal-state existed in 2.4.0, but did not list individual units to
1256+ # join a relation in 2.4.1 through 2.4.3. (LP: #1794739)
1257+ raise NotImplementedError("goal-state relation unit count")
1258+ reltype = reltype or relation_type()
1259+ _goal_state = goal_state()
1260+ return (key for key in _goal_state['relations'][reltype] if '/' in key)
1261+
1262+
1263 @cached
1264 def relation_for_unit(unit=None, rid=None):
1265 """Get the json represenation of a unit's relation"""
1266@@ -478,6 +619,24 @@
1267 return yaml.safe_load(md)
1268
1269
1270+def _metadata_unit(unit):
1271+ """Given the name of a unit (e.g. apache2/0), get the unit charm's
1272+ metadata.yaml. Very similar to metadata() but allows us to inspect
1273+ other units. Unit needs to be co-located, such as a subordinate or
1274+ principal/primary.
1275+
1276+ :returns: metadata.yaml as a python object.
1277+
1278+ """
1279+ basedir = os.sep.join(charm_dir().split(os.sep)[:-2])
1280+ unitdir = 'unit-{}'.format(unit.replace(os.sep, '-'))
1281+ joineddir = os.path.join(basedir, unitdir, 'charm', 'metadata.yaml')
1282+ if not os.path.exists(joineddir):
1283+ return None
1284+ with open(joineddir) as md:
1285+ return yaml.safe_load(md)
1286+
1287+
1288 @cached
1289 def relation_types():
1290 """Get a list of relation types supported by this charm"""
1291@@ -602,20 +761,58 @@
1292 return False
1293
1294
1295+def _port_op(op_name, port, protocol="TCP"):
1296+ """Open or close a service network port"""
1297+ _args = [op_name]
1298+ icmp = protocol.upper() == "ICMP"
1299+ if icmp:
1300+ _args.append(protocol)
1301+ else:
1302+ _args.append('{}/{}'.format(port, protocol))
1303+ try:
1304+ subprocess.check_call(_args)
1305+ except subprocess.CalledProcessError:
1306+ # Older Juju pre 2.3 doesn't support ICMP
1307+ # so treat it as a no-op if it fails.
1308+ if not icmp:
1309+ raise
1310+
1311+
1312 def open_port(port, protocol="TCP"):
1313 """Open a service network port"""
1314- _args = ['open-port']
1315- _args.append('{}/{}'.format(port, protocol))
1316- subprocess.check_call(_args)
1317+ _port_op('open-port', port, protocol)
1318
1319
1320 def close_port(port, protocol="TCP"):
1321 """Close a service network port"""
1322+ _port_op('close-port', port, protocol)
1323+
1324+
1325+def open_ports(start, end, protocol="TCP"):
1326+ """Opens a range of service network ports"""
1327+ _args = ['open-port']
1328+ _args.append('{}-{}/{}'.format(start, end, protocol))
1329+ subprocess.check_call(_args)
1330+
1331+
1332+def close_ports(start, end, protocol="TCP"):
1333+ """Close a range of service network ports"""
1334 _args = ['close-port']
1335- _args.append('{}/{}'.format(port, protocol))
1336+ _args.append('{}-{}/{}'.format(start, end, protocol))
1337 subprocess.check_call(_args)
1338
1339
1340+def opened_ports():
1341+ """Get the opened ports
1342+
1343+ *Note that this will only show ports opened in a previous hook*
1344+
1345+ :returns: Opened ports as a list of strings: ``['8080/tcp', '8081-8083/tcp']``
1346+ """
1347+ _args = ['opened-ports', '--format=json']
1348+ return json.loads(subprocess.check_output(_args).decode('UTF-8'))
1349+
1350+
1351 @cached
1352 def unit_get(attribute):
1353 """Get the unit ID for the remote unit"""
1354@@ -737,8 +934,15 @@
1355 return wrapper
1356
1357
1358+class NoNetworkBinding(Exception):
1359+ pass
1360+
1361+
1362 def charm_dir():
1363 """Return the root directory of the current charm"""
1364+ d = os.environ.get('JUJU_CHARM_DIR')
1365+ if d is not None:
1366+ return d
1367 return os.environ.get('CHARM_DIR')
1368
1369
1370@@ -845,6 +1049,28 @@
1371 return inner_translate_exc1
1372
1373
1374+def application_version_set(version):
1375+ """Charm authors may trigger this command from any hook to output what
1376+ version of the application is running. This could be a package version,
1377+ for instance postgres version 9.5. It could also be a build number or
1378+ version control revision identifier, for instance git sha 6fb7ba68. """
1379+
1380+ cmd = ['application-version-set']
1381+ cmd.append(version)
1382+ try:
1383+ subprocess.check_call(cmd)
1384+ except OSError:
1385+ log("Application Version: {}".format(version))
1386+
1387+
1388+@translate_exc(from_exc=OSError, to_exc=NotImplementedError)
1389+@cached
1390+def goal_state():
1391+ """Juju goal state values"""
1392+ cmd = ['goal-state', '--format=json']
1393+ return json.loads(subprocess.check_output(cmd).decode('UTF-8'))
1394+
1395+
1396 @translate_exc(from_exc=OSError, to_exc=NotImplementedError)
1397 def is_leader():
1398 """Does the current unit hold the juju leadership
1399@@ -912,6 +1138,24 @@
1400 subprocess.check_call(cmd)
1401
1402
1403+@translate_exc(from_exc=OSError, to_exc=NotImplementedError)
1404+def resource_get(name):
1405+ """used to fetch the resource path of the given name.
1406+
1407+ <name> must match a name of defined resource in metadata.yaml
1408+
1409+ returns either a path or False if resource not available
1410+ """
1411+ if not name:
1412+ return False
1413+
1414+ cmd = ['resource-get', name]
1415+ try:
1416+ return subprocess.check_output(cmd).decode('UTF-8')
1417+ except subprocess.CalledProcessError:
1418+ return False
1419+
1420+
1421 @cached
1422 def juju_version():
1423 """Full version string (eg. '1.23.3.1-trusty-amd64')"""
1424@@ -921,7 +1165,6 @@
1425 universal_newlines=True).strip()
1426
1427
1428-@cached
1429 def has_juju_version(minimum_version):
1430 """Return True if the Juju version is at least the provided version"""
1431 return LooseVersion(juju_version()) >= LooseVersion(minimum_version)
1432@@ -976,3 +1219,272 @@
1433 for callback, args, kwargs in reversed(_atexit):
1434 callback(*args, **kwargs)
1435 del _atexit[:]
1436+
1437+
1438+@translate_exc(from_exc=OSError, to_exc=NotImplementedError)
1439+def network_get_primary_address(binding):
1440+ '''
1441+ Deprecated since Juju 2.3; use network_get()
1442+
1443+ Retrieve the primary network address for a named binding
1444+
1445+ :param binding: string. The name of a relation of extra-binding
1446+ :return: string. The primary IP address for the named binding
1447+ :raise: NotImplementedError if run on Juju < 2.0
1448+ '''
1449+ cmd = ['network-get', '--primary-address', binding]
1450+ try:
1451+ response = subprocess.check_output(
1452+ cmd,
1453+ stderr=subprocess.STDOUT).decode('UTF-8').strip()
1454+ except CalledProcessError as e:
1455+ if 'no network config found for binding' in e.output.decode('UTF-8'):
1456+ raise NoNetworkBinding("No network binding for {}"
1457+ .format(binding))
1458+ else:
1459+ raise
1460+ return response
1461+
1462+
1463+def network_get(endpoint, relation_id=None):
1464+ """
1465+ Retrieve the network details for a relation endpoint
1466+
1467+ :param endpoint: string. The name of a relation endpoint
1468+ :param relation_id: int. The ID of the relation for the current context.
1469+ :return: dict. The loaded YAML output of the network-get query.
1470+ :raise: NotImplementedError if request not supported by the Juju version.
1471+ """
1472+ if not has_juju_version('2.2'):
1473+ raise NotImplementedError(juju_version()) # earlier versions require --primary-address
1474+ if relation_id and not has_juju_version('2.3'):
1475+ raise NotImplementedError # 2.3 added the -r option
1476+
1477+ cmd = ['network-get', endpoint, '--format', 'yaml']
1478+ if relation_id:
1479+ cmd.append('-r')
1480+ cmd.append(relation_id)
1481+ response = subprocess.check_output(
1482+ cmd,
1483+ stderr=subprocess.STDOUT).decode('UTF-8').strip()
1484+ return yaml.safe_load(response)
1485+
1486+
1487+def add_metric(*args, **kwargs):
1488+ """Add metric values. Values may be expressed with keyword arguments. For
1489+ metric names containing dashes, these may be expressed as one or more
1490+ 'key=value' positional arguments. May only be called from the collect-metrics
1491+ hook."""
1492+ _args = ['add-metric']
1493+ _kvpairs = []
1494+ _kvpairs.extend(args)
1495+ _kvpairs.extend(['{}={}'.format(k, v) for k, v in kwargs.items()])
1496+ _args.extend(sorted(_kvpairs))
1497+ try:
1498+ subprocess.check_call(_args)
1499+ return
1500+ except EnvironmentError as e:
1501+ if e.errno != errno.ENOENT:
1502+ raise
1503+ log_message = 'add-metric failed: {}'.format(' '.join(_kvpairs))
1504+ log(log_message, level='INFO')
1505+
1506+
1507+def meter_status():
1508+ """Get the meter status, if running in the meter-status-changed hook."""
1509+ return os.environ.get('JUJU_METER_STATUS')
1510+
1511+
1512+def meter_info():
1513+ """Get the meter status information, if running in the meter-status-changed
1514+ hook."""
1515+ return os.environ.get('JUJU_METER_INFO')
1516+
1517+
1518+def iter_units_for_relation_name(relation_name):
1519+ """Iterate through all units in a relation
1520+
1521+ Generator that iterates through all the units in a relation and yields
1522+ a named tuple with rid and unit field names.
1523+
1524+ Usage:
1525+ data = [(u.rid, u.unit)
1526+ for u in iter_units_for_relation_name(relation_name)]
1527+
1528+ :param relation_name: string relation name
1529+ :yield: Named Tuple with rid and unit field names
1530+ """
1531+ RelatedUnit = namedtuple('RelatedUnit', 'rid, unit')
1532+ for rid in relation_ids(relation_name):
1533+ for unit in related_units(rid):
1534+ yield RelatedUnit(rid, unit)
1535+
1536+
1537+def ingress_address(rid=None, unit=None):
1538+ """
1539+ Retrieve the ingress-address from a relation when available.
1540+ Otherwise, return the private-address.
1541+
1542+ When used on the consuming side of the relation (unit is a remote
1543+ unit), the ingress-address is the IP address that this unit needs
1544+ to use to reach the provided service on the remote unit.
1545+
1546+ When used on the providing side of the relation (unit == local_unit()),
1547+ the ingress-address is the IP address that is advertised to remote
1548+ units on this relation. Remote units need to use this address to
1549+ reach the local provided service on this unit.
1550+
1551+ Note that charms may document some other method to use in
1552+ preference to the ingress_address(), such as an address provided
1553+ on a different relation attribute or a service discovery mechanism.
1554+ This allows charms to redirect inbound connections to their peers
1555+ or different applications such as load balancers.
1556+
1557+ Usage:
1558+ addresses = [ingress_address(rid=u.rid, unit=u.unit)
1559+ for u in iter_units_for_relation_name(relation_name)]
1560+
1561+ :param rid: string relation id
1562+ :param unit: string unit name
1563+ :side effect: calls relation_get
1564+ :return: string IP address
1565+ """
1566+ settings = relation_get(rid=rid, unit=unit)
1567+ return (settings.get('ingress-address') or
1568+ settings.get('private-address'))
1569+
1570+
1571+def egress_subnets(rid=None, unit=None):
1572+ """
1573+ Retrieve the egress-subnets from a relation.
1574+
1575+ This function is to be used on the providing side of the
1576+ relation, and provides the ranges of addresses that client
1577+ connections may come from. The result is uninteresting on
1578+ the consuming side of a relation (unit == local_unit()).
1579+
1580+ Returns a stable list of subnets in CIDR format.
1581+ eg. ['192.168.1.0/24', '2001::F00F/128']
1582+
1583+ If egress-subnets is not available, falls back to using the published
1584+ ingress-address, or finally private-address.
1585+
1586+ :param rid: string relation id
1587+ :param unit: string unit name
1588+ :side effect: calls relation_get
1589+ :return: list of subnets in CIDR format. eg. ['192.168.1.0/24', '2001::F00F/128']
1590+ """
1591+ def _to_range(addr):
1592+ if re.search(r'^(?:\d{1,3}\.){3}\d{1,3}$', addr) is not None:
1593+ addr += '/32'
1594+ elif ':' in addr and '/' not in addr: # IPv6
1595+ addr += '/128'
1596+ return addr
1597+
1598+ settings = relation_get(rid=rid, unit=unit)
1599+ if 'egress-subnets' in settings:
1600+ return [n.strip() for n in settings['egress-subnets'].split(',') if n.strip()]
1601+ if 'ingress-address' in settings:
1602+ return [_to_range(settings['ingress-address'])]
1603+ if 'private-address' in settings:
1604+ return [_to_range(settings['private-address'])]
1605+ return [] # Should never happen
1606+
1607+
1608+def unit_doomed(unit=None):
1609+ """Determines if the unit is being removed from the model
1610+
1611+ Requires Juju 2.4.1.
1612+
1613+ :param unit: string unit name, defaults to local_unit
1614+ :side effect: calls goal_state
1615+ :side effect: calls local_unit
1616+ :side effect: calls has_juju_version
1617+ :return: True if the unit is being removed, already gone, or never existed
1618+ """
1619+ if not has_juju_version("2.4.1"):
1620+ # We cannot risk blindly returning False for 'we don't know',
1621+ # because that could cause data loss; if call sites don't
1622+ # need an accurate answer, they likely don't need this helper
1623+ # at all.
1624+ # goal-state existed in 2.4.0, but did not handle removals
1625+ # correctly until 2.4.1.
1626+ raise NotImplementedError("is_doomed")
1627+ if unit is None:
1628+ unit = local_unit()
1629+ gs = goal_state()
1630+ units = gs.get('units', {})
1631+ if unit not in units:
1632+ return True
1633+ # I don't think 'dead' units ever show up in the goal-state, but
1634+ # check anyway in addition to 'dying'.
1635+ return units[unit]['status'] in ('dying', 'dead')
1636+
1637+
1638+def env_proxy_settings(selected_settings=None):
1639+ """Get proxy settings from process environment variables.
1640+
1641+ Get charm proxy settings from environment variables that correspond to
1642+ juju-http-proxy, juju-https-proxy and juju-no-proxy (available as of 2.4.2,
1643+ see lp:1782236) in a format suitable for passing to an application that
1644+ reacts to proxy settings passed as environment variables. Some applications
1645+ support lowercase or uppercase notation (e.g. curl), some support only
1646+ lowercase (e.g. wget), there are also subjectively rare cases of only
1647+ uppercase notation support. no_proxy CIDR and wildcard support also varies
1648+ between runtimes and applications as there is no enforced standard.
1649+
1650+ Some applications may connect to multiple destinations and expose config
1651+ options that would affect only proxy settings for a specific destination
1652+ these should be handled in charms in an application-specific manner.
1653+
1654+ :param selected_settings: format only a subset of possible settings
1655+ :type selected_settings: list
1656+ :rtype: Option(None, dict[str, str])
1657+ """
1658+ SUPPORTED_SETTINGS = {
1659+ 'http': 'HTTP_PROXY',
1660+ 'https': 'HTTPS_PROXY',
1661+ 'no_proxy': 'NO_PROXY',
1662+ 'ftp': 'FTP_PROXY'
1663+ }
1664+ if selected_settings is None:
1665+ selected_settings = SUPPORTED_SETTINGS
1666+
1667+ selected_vars = [v for k, v in SUPPORTED_SETTINGS.items()
1668+ if k in selected_settings]
1669+ proxy_settings = {}
1670+ for var in selected_vars:
1671+ var_val = os.getenv(var)
1672+ if var_val:
1673+ proxy_settings[var] = var_val
1674+ proxy_settings[var.lower()] = var_val
1675+ # Now handle juju-prefixed environment variables. The legacy vs new
1676+ # environment variable usage is mutually exclusive
1677+ charm_var_val = os.getenv('JUJU_CHARM_{}'.format(var))
1678+ if charm_var_val:
1679+ proxy_settings[var] = charm_var_val
1680+ proxy_settings[var.lower()] = charm_var_val
1681+ if 'no_proxy' in proxy_settings:
1682+ if _contains_range(proxy_settings['no_proxy']):
1683+ log(RANGE_WARNING, level=WARNING)
1684+ return proxy_settings if proxy_settings else None
1685+
1686+
1687+def _contains_range(addresses):
1688+ """Check for cidr or wildcard domain in a string.
1689+
1690+ Given a string comprising a comma seperated list of ip addresses
1691+ and domain names, determine whether the string contains IP ranges
1692+ or wildcard domains.
1693+
1694+ :param addresses: comma seperated list of domains and ip addresses.
1695+ :type addresses: str
1696+ """
1697+ return (
1698+ # Test for cidr (e.g. 10.20.20.0/24)
1699+ "/" in addresses or
1700+ # Test for wildcard domains (*.foo.com or .foo.com)
1701+ "*" in addresses or
1702+ addresses.startswith(".") or
1703+ ",." in addresses or
1704+ " ." in addresses)
1705
1706=== modified file 'charmhelpers/core/host.py'
1707--- charmhelpers/core/host.py 2015-12-11 15:23:38 +0000
1708+++ charmhelpers/core/host.py 2021-04-13 19:16:05 +0000
1709@@ -1,18 +1,16 @@
1710 # Copyright 2014-2015 Canonical Limited.
1711 #
1712-# This file is part of charm-helpers.
1713-#
1714-# charm-helpers is free software: you can redistribute it and/or modify
1715-# it under the terms of the GNU Lesser General Public License version 3 as
1716-# published by the Free Software Foundation.
1717-#
1718-# charm-helpers is distributed in the hope that it will be useful,
1719-# but WITHOUT ANY WARRANTY; without even the implied warranty of
1720-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1721-# GNU Lesser General Public License for more details.
1722-#
1723-# You should have received a copy of the GNU Lesser General Public License
1724-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1725+# Licensed under the Apache License, Version 2.0 (the "License");
1726+# you may not use this file except in compliance with the License.
1727+# You may obtain a copy of the License at
1728+#
1729+# http://www.apache.org/licenses/LICENSE-2.0
1730+#
1731+# Unless required by applicable law or agreed to in writing, software
1732+# distributed under the License is distributed on an "AS IS" BASIS,
1733+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
1734+# See the License for the specific language governing permissions and
1735+# limitations under the License.
1736
1737 """Tools for working with the host system"""
1738 # Copyright 2012 Canonical Ltd.
1739@@ -30,49 +28,175 @@
1740 import string
1741 import subprocess
1742 import hashlib
1743+import functools
1744+import itertools
1745+import six
1746+
1747 from contextlib import contextmanager
1748 from collections import OrderedDict
1749-
1750-import six
1751-
1752-from .hookenv import log
1753+from .hookenv import log, INFO, DEBUG, local_unit, charm_name
1754 from .fstab import Fstab
1755-
1756-
1757-def service_start(service_name):
1758- """Start a system service"""
1759- return service('start', service_name)
1760-
1761-
1762-def service_stop(service_name):
1763- """Stop a system service"""
1764- return service('stop', service_name)
1765-
1766-
1767-def service_restart(service_name):
1768- """Restart a system service"""
1769+from charmhelpers.osplatform import get_platform
1770+
1771+__platform__ = get_platform()
1772+if __platform__ == "ubuntu":
1773+ from charmhelpers.core.host_factory.ubuntu import ( # NOQA:F401
1774+ service_available,
1775+ add_new_group,
1776+ lsb_release,
1777+ cmp_pkgrevno,
1778+ CompareHostReleases,
1779+ get_distrib_codename,
1780+ arch
1781+ ) # flake8: noqa -- ignore F401 for this import
1782+elif __platform__ == "centos":
1783+ from charmhelpers.core.host_factory.centos import ( # NOQA:F401
1784+ service_available,
1785+ add_new_group,
1786+ lsb_release,
1787+ cmp_pkgrevno,
1788+ CompareHostReleases,
1789+ ) # flake8: noqa -- ignore F401 for this import
1790+
1791+UPDATEDB_PATH = '/etc/updatedb.conf'
1792+
1793+
1794+def service_start(service_name, **kwargs):
1795+ """Start a system service.
1796+
1797+ The specified service name is managed via the system level init system.
1798+ Some init systems (e.g. upstart) require that additional arguments be
1799+ provided in order to directly control service instances whereas other init
1800+ systems allow for addressing instances of a service directly by name (e.g.
1801+ systemd).
1802+
1803+ The kwargs allow for the additional parameters to be passed to underlying
1804+ init systems for those systems which require/allow for them. For example,
1805+ the ceph-osd upstart script requires the id parameter to be passed along
1806+ in order to identify which running daemon should be reloaded. The follow-
1807+ ing example stops the ceph-osd service for instance id=4:
1808+
1809+ service_stop('ceph-osd', id=4)
1810+
1811+ :param service_name: the name of the service to stop
1812+ :param **kwargs: additional parameters to pass to the init system when
1813+ managing services. These will be passed as key=value
1814+ parameters to the init system's commandline. kwargs
1815+ are ignored for systemd enabled systems.
1816+ """
1817+ return service('start', service_name, **kwargs)
1818+
1819+
1820+def service_stop(service_name, **kwargs):
1821+ """Stop a system service.
1822+
1823+ The specified service name is managed via the system level init system.
1824+ Some init systems (e.g. upstart) require that additional arguments be
1825+ provided in order to directly control service instances whereas other init
1826+ systems allow for addressing instances of a service directly by name (e.g.
1827+ systemd).
1828+
1829+ The kwargs allow for the additional parameters to be passed to underlying
1830+ init systems for those systems which require/allow for them. For example,
1831+ the ceph-osd upstart script requires the id parameter to be passed along
1832+ in order to identify which running daemon should be reloaded. The follow-
1833+ ing example stops the ceph-osd service for instance id=4:
1834+
1835+ service_stop('ceph-osd', id=4)
1836+
1837+ :param service_name: the name of the service to stop
1838+ :param **kwargs: additional parameters to pass to the init system when
1839+ managing services. These will be passed as key=value
1840+ parameters to the init system's commandline. kwargs
1841+ are ignored for systemd enabled systems.
1842+ """
1843+ return service('stop', service_name, **kwargs)
1844+
1845+
1846+def service_restart(service_name, **kwargs):
1847+ """Restart a system service.
1848+
1849+ The specified service name is managed via the system level init system.
1850+ Some init systems (e.g. upstart) require that additional arguments be
1851+ provided in order to directly control service instances whereas other init
1852+ systems allow for addressing instances of a service directly by name (e.g.
1853+ systemd).
1854+
1855+ The kwargs allow for the additional parameters to be passed to underlying
1856+ init systems for those systems which require/allow for them. For example,
1857+ the ceph-osd upstart script requires the id parameter to be passed along
1858+ in order to identify which running daemon should be restarted. The follow-
1859+ ing example restarts the ceph-osd service for instance id=4:
1860+
1861+ service_restart('ceph-osd', id=4)
1862+
1863+ :param service_name: the name of the service to restart
1864+ :param **kwargs: additional parameters to pass to the init system when
1865+ managing services. These will be passed as key=value
1866+ parameters to the init system's commandline. kwargs
1867+ are ignored for init systems not allowing additional
1868+ parameters via the commandline (systemd).
1869+ """
1870 return service('restart', service_name)
1871
1872
1873-def service_reload(service_name, restart_on_failure=False):
1874+def service_reload(service_name, restart_on_failure=False, **kwargs):
1875 """Reload a system service, optionally falling back to restart if
1876- reload fails"""
1877- service_result = service('reload', service_name)
1878+ reload fails.
1879+
1880+ The specified service name is managed via the system level init system.
1881+ Some init systems (e.g. upstart) require that additional arguments be
1882+ provided in order to directly control service instances whereas other init
1883+ systems allow for addressing instances of a service directly by name (e.g.
1884+ systemd).
1885+
1886+ The kwargs allow for the additional parameters to be passed to underlying
1887+ init systems for those systems which require/allow for them. For example,
1888+ the ceph-osd upstart script requires the id parameter to be passed along
1889+ in order to identify which running daemon should be reloaded. The follow-
1890+ ing example restarts the ceph-osd service for instance id=4:
1891+
1892+ service_reload('ceph-osd', id=4)
1893+
1894+ :param service_name: the name of the service to reload
1895+ :param restart_on_failure: boolean indicating whether to fallback to a
1896+ restart if the reload fails.
1897+ :param **kwargs: additional parameters to pass to the init system when
1898+ managing services. These will be passed as key=value
1899+ parameters to the init system's commandline. kwargs
1900+ are ignored for init systems not allowing additional
1901+ parameters via the commandline (systemd).
1902+ """
1903+ service_result = service('reload', service_name, **kwargs)
1904 if not service_result and restart_on_failure:
1905- service_result = service('restart', service_name)
1906+ service_result = service('restart', service_name, **kwargs)
1907 return service_result
1908
1909
1910-def service_pause(service_name, init_dir="/etc/init", initd_dir="/etc/init.d"):
1911+def service_pause(service_name, init_dir="/etc/init", initd_dir="/etc/init.d",
1912+ **kwargs):
1913 """Pause a system service.
1914
1915- Stop it, and prevent it from starting again at boot."""
1916+ Stop it, and prevent it from starting again at boot.
1917+
1918+ :param service_name: the name of the service to pause
1919+ :param init_dir: path to the upstart init directory
1920+ :param initd_dir: path to the sysv init directory
1921+ :param **kwargs: additional parameters to pass to the init system when
1922+ managing services. These will be passed as key=value
1923+ parameters to the init system's commandline. kwargs
1924+ are ignored for init systems which do not support
1925+ key=value arguments via the commandline.
1926+ """
1927 stopped = True
1928- if service_running(service_name):
1929- stopped = service_stop(service_name)
1930+ if service_running(service_name, **kwargs):
1931+ stopped = service_stop(service_name, **kwargs)
1932 upstart_file = os.path.join(init_dir, "{}.conf".format(service_name))
1933 sysv_file = os.path.join(initd_dir, service_name)
1934- if os.path.exists(upstart_file):
1935+ if init_is_systemd():
1936+ service('disable', service_name)
1937+ service('mask', service_name)
1938+ elif os.path.exists(upstart_file):
1939 override_path = os.path.join(
1940 init_dir, '{}.override'.format(service_name))
1941 with open(override_path, 'w') as fh:
1942@@ -80,21 +204,33 @@
1943 elif os.path.exists(sysv_file):
1944 subprocess.check_call(["update-rc.d", service_name, "disable"])
1945 else:
1946- # XXX: Support SystemD too
1947 raise ValueError(
1948- "Unable to detect {0} as either Upstart {1} or SysV {2}".format(
1949+ "Unable to detect {0} as SystemD, Upstart {1} or"
1950+ " SysV {2}".format(
1951 service_name, upstart_file, sysv_file))
1952 return stopped
1953
1954
1955 def service_resume(service_name, init_dir="/etc/init",
1956- initd_dir="/etc/init.d"):
1957+ initd_dir="/etc/init.d", **kwargs):
1958 """Resume a system service.
1959
1960- Reenable starting again at boot. Start the service"""
1961+ Reenable starting again at boot. Start the service.
1962+
1963+ :param service_name: the name of the service to resume
1964+ :param init_dir: the path to the init dir
1965+ :param initd dir: the path to the initd dir
1966+ :param **kwargs: additional parameters to pass to the init system when
1967+ managing services. These will be passed as key=value
1968+ parameters to the init system's commandline. kwargs
1969+ are ignored for systemd enabled systems.
1970+ """
1971 upstart_file = os.path.join(init_dir, "{}.conf".format(service_name))
1972 sysv_file = os.path.join(initd_dir, service_name)
1973- if os.path.exists(upstart_file):
1974+ if init_is_systemd():
1975+ service('unmask', service_name)
1976+ service('enable', service_name)
1977+ elif os.path.exists(upstart_file):
1978 override_path = os.path.join(
1979 init_dir, '{}.override'.format(service_name))
1980 if os.path.exists(override_path):
1981@@ -102,54 +238,90 @@
1982 elif os.path.exists(sysv_file):
1983 subprocess.check_call(["update-rc.d", service_name, "enable"])
1984 else:
1985- # XXX: Support SystemD too
1986 raise ValueError(
1987- "Unable to detect {0} as either Upstart {1} or SysV {2}".format(
1988+ "Unable to detect {0} as SystemD, Upstart {1} or"
1989+ " SysV {2}".format(
1990 service_name, upstart_file, sysv_file))
1991+ started = service_running(service_name, **kwargs)
1992
1993- started = service_running(service_name)
1994 if not started:
1995- started = service_start(service_name)
1996+ started = service_start(service_name, **kwargs)
1997 return started
1998
1999
2000-def service(action, service_name):
2001- """Control a system service"""
2002- cmd = ['service', service_name, action]
2003+def service(action, service_name, **kwargs):
2004+ """Control a system service.
2005+
2006+ :param action: the action to take on the service
2007+ :param service_name: the name of the service to perform th action on
2008+ :param **kwargs: additional params to be passed to the service command in
2009+ the form of key=value.
2010+ """
2011+ if init_is_systemd():
2012+ cmd = ['systemctl', action, service_name]
2013+ else:
2014+ cmd = ['service', service_name, action]
2015+ for key, value in six.iteritems(kwargs):
2016+ parameter = '%s=%s' % (key, value)
2017+ cmd.append(parameter)
2018 return subprocess.call(cmd) == 0
2019
2020
2021-def service_running(service):
2022- """Determine whether a system service is running"""
2023- try:
2024- output = subprocess.check_output(
2025- ['service', service, 'status'],
2026- stderr=subprocess.STDOUT).decode('UTF-8')
2027- except subprocess.CalledProcessError:
2028- return False
2029- else:
2030- if ("start/running" in output or "is running" in output):
2031- return True
2032- else:
2033- return False
2034-
2035-
2036-def service_available(service_name):
2037- """Determine whether a system service is available"""
2038- try:
2039- subprocess.check_output(
2040- ['service', service_name, 'status'],
2041- stderr=subprocess.STDOUT).decode('UTF-8')
2042- except subprocess.CalledProcessError as e:
2043- return b'unrecognized service' not in e.output
2044- else:
2045- return True
2046-
2047-
2048-def adduser(username, password=None, shell='/bin/bash', system_user=False,
2049- primary_group=None, secondary_groups=None):
2050+_UPSTART_CONF = "/etc/init/{}.conf"
2051+_INIT_D_CONF = "/etc/init.d/{}"
2052+
2053+
2054+def service_running(service_name, **kwargs):
2055+ """Determine whether a system service is running.
2056+
2057+ :param service_name: the name of the service
2058+ :param **kwargs: additional args to pass to the service command. This is
2059+ used to pass additional key=value arguments to the
2060+ service command line for managing specific instance
2061+ units (e.g. service ceph-osd status id=2). The kwargs
2062+ are ignored in systemd services.
2063 """
2064- Add a user to the system.
2065+ if init_is_systemd():
2066+ return service('is-active', service_name)
2067+ else:
2068+ if os.path.exists(_UPSTART_CONF.format(service_name)):
2069+ try:
2070+ cmd = ['status', service_name]
2071+ for key, value in six.iteritems(kwargs):
2072+ parameter = '%s=%s' % (key, value)
2073+ cmd.append(parameter)
2074+ output = subprocess.check_output(
2075+ cmd, stderr=subprocess.STDOUT).decode('UTF-8')
2076+ except subprocess.CalledProcessError:
2077+ return False
2078+ else:
2079+ # This works for upstart scripts where the 'service' command
2080+ # returns a consistent string to represent running
2081+ # 'start/running'
2082+ if ("start/running" in output or
2083+ "is running" in output or
2084+ "up and running" in output):
2085+ return True
2086+ elif os.path.exists(_INIT_D_CONF.format(service_name)):
2087+ # Check System V scripts init script return codes
2088+ return service('status', service_name)
2089+ return False
2090+
2091+
2092+SYSTEMD_SYSTEM = '/run/systemd/system'
2093+
2094+
2095+def init_is_systemd():
2096+ """Return True if the host system uses systemd, False otherwise."""
2097+ if lsb_release()['DISTRIB_CODENAME'] == 'trusty':
2098+ return False
2099+ return os.path.isdir(SYSTEMD_SYSTEM)
2100+
2101+
2102+def adduser(username, password=None, shell='/bin/bash',
2103+ system_user=False, primary_group=None,
2104+ secondary_groups=None, uid=None, home_dir=None):
2105+ """Add a user to the system.
2106
2107 Will log but otherwise succeed if the user already exists.
2108
2109@@ -157,17 +329,26 @@
2110 :param str password: Password for user; if ``None``, create a system user
2111 :param str shell: The default shell for the user
2112 :param bool system_user: Whether to create a login or system user
2113- :param str primary_group: Primary group for user; defaults to their username
2114+ :param str primary_group: Primary group for user; defaults to username
2115 :param list secondary_groups: Optional list of additional groups
2116+ :param int uid: UID for user being created
2117+ :param str home_dir: Home directory for user
2118
2119 :returns: The password database entry struct, as returned by `pwd.getpwnam`
2120 """
2121 try:
2122 user_info = pwd.getpwnam(username)
2123 log('user {0} already exists!'.format(username))
2124+ if uid:
2125+ user_info = pwd.getpwuid(int(uid))
2126+ log('user with uid {0} already exists!'.format(uid))
2127 except KeyError:
2128 log('creating user {0}'.format(username))
2129 cmd = ['useradd']
2130+ if uid:
2131+ cmd.extend(['--uid', str(uid)])
2132+ if home_dir:
2133+ cmd.extend(['--home', str(home_dir)])
2134 if system_user or password is None:
2135 cmd.append('--system')
2136 else:
2137@@ -202,22 +383,56 @@
2138 return user_exists
2139
2140
2141-def add_group(group_name, system_group=False):
2142- """Add a group to the system"""
2143+def uid_exists(uid):
2144+ """Check if a uid exists"""
2145+ try:
2146+ pwd.getpwuid(uid)
2147+ uid_exists = True
2148+ except KeyError:
2149+ uid_exists = False
2150+ return uid_exists
2151+
2152+
2153+def group_exists(groupname):
2154+ """Check if a group exists"""
2155+ try:
2156+ grp.getgrnam(groupname)
2157+ group_exists = True
2158+ except KeyError:
2159+ group_exists = False
2160+ return group_exists
2161+
2162+
2163+def gid_exists(gid):
2164+ """Check if a gid exists"""
2165+ try:
2166+ grp.getgrgid(gid)
2167+ gid_exists = True
2168+ except KeyError:
2169+ gid_exists = False
2170+ return gid_exists
2171+
2172+
2173+def add_group(group_name, system_group=False, gid=None):
2174+ """Add a group to the system
2175+
2176+ Will log but otherwise succeed if the group already exists.
2177+
2178+ :param str group_name: group to create
2179+ :param bool system_group: Create system group
2180+ :param int gid: GID for user being created
2181+
2182+ :returns: The password database entry struct, as returned by `grp.getgrnam`
2183+ """
2184 try:
2185 group_info = grp.getgrnam(group_name)
2186 log('group {0} already exists!'.format(group_name))
2187+ if gid:
2188+ group_info = grp.getgrgid(gid)
2189+ log('group with gid {0} already exists!'.format(gid))
2190 except KeyError:
2191 log('creating group {0}'.format(group_name))
2192- cmd = ['addgroup']
2193- if system_group:
2194- cmd.append('--system')
2195- else:
2196- cmd.extend([
2197- '--group',
2198- ])
2199- cmd.append(group_name)
2200- subprocess.check_call(cmd)
2201+ add_new_group(group_name, system_group, gid)
2202 group_info = grp.getgrnam(group_name)
2203 return group_info
2204
2205@@ -229,15 +444,62 @@
2206 subprocess.check_call(cmd)
2207
2208
2209-def rsync(from_path, to_path, flags='-r', options=None):
2210+def chage(username, lastday=None, expiredate=None, inactive=None,
2211+ mindays=None, maxdays=None, root=None, warndays=None):
2212+ """Change user password expiry information
2213+
2214+ :param str username: User to update
2215+ :param str lastday: Set when password was changed in YYYY-MM-DD format
2216+ :param str expiredate: Set when user's account will no longer be
2217+ accessible in YYYY-MM-DD format.
2218+ -1 will remove an account expiration date.
2219+ :param str inactive: Set the number of days of inactivity after a password
2220+ has expired before the account is locked.
2221+ -1 will remove an account's inactivity.
2222+ :param str mindays: Set the minimum number of days between password
2223+ changes to MIN_DAYS.
2224+ 0 indicates the password can be changed anytime.
2225+ :param str maxdays: Set the maximum number of days during which a
2226+ password is valid.
2227+ -1 as MAX_DAYS will remove checking maxdays
2228+ :param str root: Apply changes in the CHROOT_DIR directory
2229+ :param str warndays: Set the number of days of warning before a password
2230+ change is required
2231+ :raises subprocess.CalledProcessError: if call to chage fails
2232+ """
2233+ cmd = ['chage']
2234+ if root:
2235+ cmd.extend(['--root', root])
2236+ if lastday:
2237+ cmd.extend(['--lastday', lastday])
2238+ if expiredate:
2239+ cmd.extend(['--expiredate', expiredate])
2240+ if inactive:
2241+ cmd.extend(['--inactive', inactive])
2242+ if mindays:
2243+ cmd.extend(['--mindays', mindays])
2244+ if maxdays:
2245+ cmd.extend(['--maxdays', maxdays])
2246+ if warndays:
2247+ cmd.extend(['--warndays', warndays])
2248+ cmd.append(username)
2249+ subprocess.check_call(cmd)
2250+
2251+
2252+remove_password_expiry = functools.partial(chage, expiredate='-1', inactive='-1', mindays='0', maxdays='-1')
2253+
2254+
2255+def rsync(from_path, to_path, flags='-r', options=None, timeout=None):
2256 """Replicate the contents of a path"""
2257 options = options or ['--delete', '--executability']
2258 cmd = ['/usr/bin/rsync', flags]
2259+ if timeout:
2260+ cmd = ['timeout', str(timeout)] + cmd
2261 cmd.extend(options)
2262 cmd.append(from_path)
2263 cmd.append(to_path)
2264 log(" ".join(cmd))
2265- return subprocess.check_output(cmd).decode('UTF-8').strip()
2266+ return subprocess.check_output(cmd, stderr=subprocess.STDOUT).decode('UTF-8').strip()
2267
2268
2269 def symlink(source, destination):
2270@@ -273,24 +535,54 @@
2271
2272 def write_file(path, content, owner='root', group='root', perms=0o444):
2273 """Create or overwrite a file with the contents of a byte string."""
2274- log("Writing file {} {}:{} {:o}".format(path, owner, group, perms))
2275 uid = pwd.getpwnam(owner).pw_uid
2276 gid = grp.getgrnam(group).gr_gid
2277- with open(path, 'wb') as target:
2278- os.fchown(target.fileno(), uid, gid)
2279- os.fchmod(target.fileno(), perms)
2280- target.write(content)
2281+ # lets see if we can grab the file and compare the context, to avoid doing
2282+ # a write.
2283+ existing_content = None
2284+ existing_uid, existing_gid, existing_perms = None, None, None
2285+ try:
2286+ with open(path, 'rb') as target:
2287+ existing_content = target.read()
2288+ stat = os.stat(path)
2289+ existing_uid, existing_gid, existing_perms = (
2290+ stat.st_uid, stat.st_gid, stat.st_mode
2291+ )
2292+ except Exception:
2293+ pass
2294+ if content != existing_content:
2295+ log("Writing file {} {}:{} {:o}".format(path, owner, group, perms),
2296+ level=DEBUG)
2297+ with open(path, 'wb') as target:
2298+ os.fchown(target.fileno(), uid, gid)
2299+ os.fchmod(target.fileno(), perms)
2300+ if six.PY3 and isinstance(content, six.string_types):
2301+ content = content.encode('UTF-8')
2302+ target.write(content)
2303+ return
2304+ # the contents were the same, but we might still need to change the
2305+ # ownership or permissions.
2306+ if existing_uid != uid:
2307+ log("Changing uid on already existing content: {} -> {}"
2308+ .format(existing_uid, uid), level=DEBUG)
2309+ os.chown(path, uid, -1)
2310+ if existing_gid != gid:
2311+ log("Changing gid on already existing content: {} -> {}"
2312+ .format(existing_gid, gid), level=DEBUG)
2313+ os.chown(path, -1, gid)
2314+ if existing_perms != perms:
2315+ log("Changing permissions on existing content: {} -> {}"
2316+ .format(existing_perms, perms), level=DEBUG)
2317+ os.chmod(path, perms)
2318
2319
2320 def fstab_remove(mp):
2321- """Remove the given mountpoint entry from /etc/fstab
2322- """
2323+ """Remove the given mountpoint entry from /etc/fstab"""
2324 return Fstab.remove_by_mountpoint(mp)
2325
2326
2327 def fstab_add(dev, mp, fs, options=None):
2328- """Adds the given device entry to the /etc/fstab file
2329- """
2330+ """Adds the given device entry to the /etc/fstab file"""
2331 return Fstab.add(dev, mp, fs, options=options)
2332
2333
2334@@ -346,8 +638,7 @@
2335
2336
2337 def file_hash(path, hash_type='md5'):
2338- """
2339- Generate a hash checksum of the contents of 'path' or None if not found.
2340+ """Generate a hash checksum of the contents of 'path' or None if not found.
2341
2342 :param str hash_type: Any hash alrgorithm supported by :mod:`hashlib`,
2343 such as md5, sha1, sha256, sha512, etc.
2344@@ -362,10 +653,9 @@
2345
2346
2347 def path_hash(path):
2348- """
2349- Generate a hash checksum of all files matching 'path'. Standard wildcards
2350- like '*' and '?' are supported, see documentation for the 'glob' module for
2351- more information.
2352+ """Generate a hash checksum of all files matching 'path'. Standard
2353+ wildcards like '*' and '?' are supported, see documentation for the 'glob'
2354+ module for more information.
2355
2356 :return: dict: A { filename: hash } dictionary for all matched files.
2357 Empty if none found.
2358@@ -377,8 +667,7 @@
2359
2360
2361 def check_hash(path, checksum, hash_type='md5'):
2362- """
2363- Validate a file using a cryptographic checksum.
2364+ """Validate a file using a cryptographic checksum.
2365
2366 :param str checksum: Value of the checksum used to validate the file.
2367 :param str hash_type: Hash algorithm used to generate `checksum`.
2368@@ -393,10 +682,11 @@
2369
2370
2371 class ChecksumError(ValueError):
2372+ """A class derived from Value error to indicate the checksum failed."""
2373 pass
2374
2375
2376-def restart_on_change(restart_map, stopstart=False):
2377+def restart_on_change(restart_map, stopstart=False, restart_functions=None):
2378 """Restart services based on configuration files changing
2379
2380 This function is used a decorator, for example::
2381@@ -414,35 +704,56 @@
2382 restarted if any file matching the pattern got changed, created
2383 or removed. Standard wildcards are supported, see documentation
2384 for the 'glob' module for more information.
2385+
2386+ @param restart_map: {path_file_name: [service_name, ...]
2387+ @param stopstart: DEFAULT false; whether to stop, start OR restart
2388+ @param restart_functions: nonstandard functions to use to restart services
2389+ {svc: func, ...}
2390+ @returns result from decorated function
2391 """
2392 def wrap(f):
2393+ @functools.wraps(f)
2394 def wrapped_f(*args, **kwargs):
2395- checksums = {path: path_hash(path) for path in restart_map}
2396- f(*args, **kwargs)
2397- restarts = []
2398- for path in restart_map:
2399- if path_hash(path) != checksums[path]:
2400- restarts += restart_map[path]
2401- services_list = list(OrderedDict.fromkeys(restarts))
2402- if not stopstart:
2403- for service_name in services_list:
2404- service('restart', service_name)
2405- else:
2406- for action in ['stop', 'start']:
2407- for service_name in services_list:
2408- service(action, service_name)
2409+ return restart_on_change_helper(
2410+ (lambda: f(*args, **kwargs)), restart_map, stopstart,
2411+ restart_functions)
2412 return wrapped_f
2413 return wrap
2414
2415
2416-def lsb_release():
2417- """Return /etc/lsb-release in a dict"""
2418- d = {}
2419- with open('/etc/lsb-release', 'r') as lsb:
2420- for l in lsb:
2421- k, v = l.split('=')
2422- d[k.strip()] = v.strip()
2423- return d
2424+def restart_on_change_helper(lambda_f, restart_map, stopstart=False,
2425+ restart_functions=None):
2426+ """Helper function to perform the restart_on_change function.
2427+
2428+ This is provided for decorators to restart services if files described
2429+ in the restart_map have changed after an invocation of lambda_f().
2430+
2431+ @param lambda_f: function to call.
2432+ @param restart_map: {file: [service, ...]}
2433+ @param stopstart: whether to stop, start or restart a service
2434+ @param restart_functions: nonstandard functions to use to restart services
2435+ {svc: func, ...}
2436+ @returns result of lambda_f()
2437+ """
2438+ if restart_functions is None:
2439+ restart_functions = {}
2440+ checksums = {path: path_hash(path) for path in restart_map}
2441+ r = lambda_f()
2442+ # create a list of lists of the services to restart
2443+ restarts = [restart_map[path]
2444+ for path in restart_map
2445+ if path_hash(path) != checksums[path]]
2446+ # create a flat list of ordered services without duplicates from lists
2447+ services_list = list(OrderedDict.fromkeys(itertools.chain(*restarts)))
2448+ if services_list:
2449+ actions = ('stop', 'start') if stopstart else ('restart',)
2450+ for service_name in services_list:
2451+ if service_name in restart_functions:
2452+ restart_functions[service_name](service_name)
2453+ else:
2454+ for action in actions:
2455+ service(action, service_name)
2456+ return r
2457
2458
2459 def pwgen(length=None):
2460@@ -498,7 +809,7 @@
2461
2462
2463 def list_nics(nic_type=None):
2464- '''Return a list of nics of given type(s)'''
2465+ """Return a list of nics of given type(s)"""
2466 if isinstance(nic_type, six.string_types):
2467 int_types = [nic_type]
2468 else:
2469@@ -527,7 +838,7 @@
2470 ip_output = subprocess.check_output(cmd).decode('UTF-8').split('\n')
2471 ip_output = (line.strip() for line in ip_output if line)
2472
2473- key = re.compile('^[0-9]+:\s+(.+):')
2474+ key = re.compile(r'^[0-9]+:\s+(.+):')
2475 for line in ip_output:
2476 matched = re.search(key, line)
2477 if matched:
2478@@ -540,12 +851,13 @@
2479
2480
2481 def set_nic_mtu(nic, mtu):
2482- '''Set MTU on a network interface'''
2483+ """Set the Maximum Transmission Unit (MTU) on a network interface."""
2484 cmd = ['ip', 'link', 'set', nic, 'mtu', mtu]
2485 subprocess.check_call(cmd)
2486
2487
2488 def get_nic_mtu(nic):
2489+ """Return the Maximum Transmission Unit (MTU) for a network interface."""
2490 cmd = ['ip', 'addr', 'show', nic]
2491 ip_output = subprocess.check_output(cmd).decode('UTF-8').split('\n')
2492 mtu = ""
2493@@ -557,6 +869,7 @@
2494
2495
2496 def get_nic_hwaddr(nic):
2497+ """Return the Media Access Control (MAC) for a network interface."""
2498 cmd = ['ip', '-o', '-0', 'addr', 'show', nic]
2499 ip_output = subprocess.check_output(cmd).decode('UTF-8')
2500 hwaddr = ""
2501@@ -566,40 +879,29 @@
2502 return hwaddr
2503
2504
2505-def cmp_pkgrevno(package, revno, pkgcache=None):
2506- '''Compare supplied revno with the revno of the installed package
2507-
2508- * 1 => Installed revno is greater than supplied arg
2509- * 0 => Installed revno is the same as supplied arg
2510- * -1 => Installed revno is less than supplied arg
2511-
2512- This function imports apt_cache function from charmhelpers.fetch if
2513- the pkgcache argument is None. Be sure to add charmhelpers.fetch if
2514- you call this function, or pass an apt_pkg.Cache() instance.
2515- '''
2516- import apt_pkg
2517- if not pkgcache:
2518- from charmhelpers.fetch import apt_cache
2519- pkgcache = apt_cache()
2520- pkg = pkgcache[package]
2521- return apt_pkg.version_compare(pkg.current_ver.ver_str, revno)
2522-
2523-
2524 @contextmanager
2525-def chdir(d):
2526+def chdir(directory):
2527+ """Change the current working directory to a different directory for a code
2528+ block and return the previous directory after the block exits. Useful to
2529+ run commands from a specificed directory.
2530+
2531+ :param str directory: The directory path to change to for this context.
2532+ """
2533 cur = os.getcwd()
2534 try:
2535- yield os.chdir(d)
2536+ yield os.chdir(directory)
2537 finally:
2538 os.chdir(cur)
2539
2540
2541 def chownr(path, owner, group, follow_links=True, chowntopdir=False):
2542- """
2543- Recursively change user and group ownership of files and directories
2544+ """Recursively change user and group ownership of files and directories
2545 in given path. Doesn't chown path itself by default, only its children.
2546
2547- :param bool follow_links: Also Chown links if True
2548+ :param str path: The string path to start changing ownership.
2549+ :param str owner: The owner string to use when looking up the uid.
2550+ :param str group: The group string to use when looking up the gid.
2551+ :param bool follow_links: Also follow and chown links if True
2552 :param bool chowntopdir: Also chown path itself if True
2553 """
2554 uid = pwd.getpwnam(owner).pw_uid
2555@@ -613,7 +915,7 @@
2556 broken_symlink = os.path.lexists(path) and not os.path.exists(path)
2557 if not broken_symlink:
2558 chown(path, uid, gid)
2559- for root, dirs, files in os.walk(path):
2560+ for root, dirs, files in os.walk(path, followlinks=follow_links):
2561 for name in dirs + files:
2562 full = os.path.join(root, name)
2563 broken_symlink = os.path.lexists(full) and not os.path.exists(full)
2564@@ -622,15 +924,37 @@
2565
2566
2567 def lchownr(path, owner, group):
2568+ """Recursively change user and group ownership of files and directories
2569+ in a given path, not following symbolic links. See the documentation for
2570+ 'os.lchown' for more information.
2571+
2572+ :param str path: The string path to start changing ownership.
2573+ :param str owner: The owner string to use when looking up the uid.
2574+ :param str group: The group string to use when looking up the gid.
2575+ """
2576 chownr(path, owner, group, follow_links=False)
2577
2578
2579+def owner(path):
2580+ """Returns a tuple containing the username & groupname owning the path.
2581+
2582+ :param str path: the string path to retrieve the ownership
2583+ :return tuple(str, str): A (username, groupname) tuple containing the
2584+ name of the user and group owning the path.
2585+ :raises OSError: if the specified path does not exist
2586+ """
2587+ stat = os.stat(path)
2588+ username = pwd.getpwuid(stat.st_uid)[0]
2589+ groupname = grp.getgrgid(stat.st_gid)[0]
2590+ return username, groupname
2591+
2592+
2593 def get_total_ram():
2594- '''The total amount of system RAM in bytes.
2595+ """The total amount of system RAM in bytes.
2596
2597 This is what is reported by the OS, and may be overcommitted when
2598 there are multiple containers hosted on the same machine.
2599- '''
2600+ """
2601 with open('/proc/meminfo', 'r') as f:
2602 for line in f.readlines():
2603 if line:
2604@@ -639,3 +963,115 @@
2605 assert unit == 'kB', 'Unknown unit'
2606 return int(value) * 1024 # Classic, not KiB.
2607 raise NotImplementedError()
2608+
2609+
2610+UPSTART_CONTAINER_TYPE = '/run/container_type'
2611+
2612+
2613+def is_container():
2614+ """Determine whether unit is running in a container
2615+
2616+ @return: boolean indicating if unit is in a container
2617+ """
2618+ if init_is_systemd():
2619+ # Detect using systemd-detect-virt
2620+ return subprocess.call(['systemd-detect-virt',
2621+ '--container']) == 0
2622+ else:
2623+ # Detect using upstart container file marker
2624+ return os.path.exists(UPSTART_CONTAINER_TYPE)
2625+
2626+
2627+def add_to_updatedb_prunepath(path, updatedb_path=UPDATEDB_PATH):
2628+ """Adds the specified path to the mlocate's udpatedb.conf PRUNEPATH list.
2629+
2630+ This method has no effect if the path specified by updatedb_path does not
2631+ exist or is not a file.
2632+
2633+ @param path: string the path to add to the updatedb.conf PRUNEPATHS value
2634+ @param updatedb_path: the path the updatedb.conf file
2635+ """
2636+ if not os.path.exists(updatedb_path) or os.path.isdir(updatedb_path):
2637+ # If the updatedb.conf file doesn't exist then don't attempt to update
2638+ # the file as the package providing mlocate may not be installed on
2639+ # the local system
2640+ return
2641+
2642+ with open(updatedb_path, 'r+') as f_id:
2643+ updatedb_text = f_id.read()
2644+ output = updatedb(updatedb_text, path)
2645+ f_id.seek(0)
2646+ f_id.write(output)
2647+ f_id.truncate()
2648+
2649+
2650+def updatedb(updatedb_text, new_path):
2651+ lines = [line for line in updatedb_text.split("\n")]
2652+ for i, line in enumerate(lines):
2653+ if line.startswith("PRUNEPATHS="):
2654+ paths_line = line.split("=")[1].replace('"', '')
2655+ paths = paths_line.split(" ")
2656+ if new_path not in paths:
2657+ paths.append(new_path)
2658+ lines[i] = 'PRUNEPATHS="{}"'.format(' '.join(paths))
2659+ output = "\n".join(lines)
2660+ return output
2661+
2662+
2663+def modulo_distribution(modulo=3, wait=30, non_zero_wait=False):
2664+ """ Modulo distribution
2665+
2666+ This helper uses the unit number, a modulo value and a constant wait time
2667+ to produce a calculated wait time distribution. This is useful in large
2668+ scale deployments to distribute load during an expensive operation such as
2669+ service restarts.
2670+
2671+ If you have 1000 nodes that need to restart 100 at a time 1 minute at a
2672+ time:
2673+
2674+ time.wait(modulo_distribution(modulo=100, wait=60))
2675+ restart()
2676+
2677+ If you need restarts to happen serially set modulo to the exact number of
2678+ nodes and set a high constant wait time:
2679+
2680+ time.wait(modulo_distribution(modulo=10, wait=120))
2681+ restart()
2682+
2683+ @param modulo: int The modulo number creates the group distribution
2684+ @param wait: int The constant time wait value
2685+ @param non_zero_wait: boolean Override unit % modulo == 0,
2686+ return modulo * wait. Used to avoid collisions with
2687+ leader nodes which are often given priority.
2688+ @return: int Calculated time to wait for unit operation
2689+ """
2690+ unit_number = int(local_unit().split('/')[1])
2691+ calculated_wait_time = (unit_number % modulo) * wait
2692+ if non_zero_wait and calculated_wait_time == 0:
2693+ return modulo * wait
2694+ else:
2695+ return calculated_wait_time
2696+
2697+
2698+def install_ca_cert(ca_cert, name=None):
2699+ """
2700+ Install the given cert as a trusted CA.
2701+
2702+ The ``name`` is the stem of the filename where the cert is written, and if
2703+ not provided, it will default to ``juju-{charm_name}``.
2704+
2705+ If the cert is empty or None, or is unchanged, nothing is done.
2706+ """
2707+ if not ca_cert:
2708+ return
2709+ if not isinstance(ca_cert, bytes):
2710+ ca_cert = ca_cert.encode('utf8')
2711+ if not name:
2712+ name = 'juju-{}'.format(charm_name())
2713+ cert_file = '/usr/local/share/ca-certificates/{}.crt'.format(name)
2714+ new_hash = hashlib.md5(ca_cert).hexdigest()
2715+ if file_hash(cert_file) == new_hash:
2716+ return
2717+ log("Installing new CA cert at: {}".format(cert_file), level=INFO)
2718+ write_file(cert_file, ca_cert)
2719+ subprocess.check_call(['update-ca-certificates', '--fresh'])
2720
2721=== added directory 'charmhelpers/core/host_factory'
2722=== added file 'charmhelpers/core/host_factory/__init__.py'
2723=== added file 'charmhelpers/core/host_factory/centos.py'
2724--- charmhelpers/core/host_factory/centos.py 1970-01-01 00:00:00 +0000
2725+++ charmhelpers/core/host_factory/centos.py 2021-04-13 19:16:05 +0000
2726@@ -0,0 +1,72 @@
2727+import subprocess
2728+import yum
2729+import os
2730+
2731+from charmhelpers.core.strutils import BasicStringComparator
2732+
2733+
2734+class CompareHostReleases(BasicStringComparator):
2735+ """Provide comparisons of Host releases.
2736+
2737+ Use in the form of
2738+
2739+ if CompareHostReleases(release) > 'trusty':
2740+ # do something with mitaka
2741+ """
2742+
2743+ def __init__(self, item):
2744+ raise NotImplementedError(
2745+ "CompareHostReleases() is not implemented for CentOS")
2746+
2747+
2748+def service_available(service_name):
2749+ # """Determine whether a system service is available."""
2750+ if os.path.isdir('/run/systemd/system'):
2751+ cmd = ['systemctl', 'is-enabled', service_name]
2752+ else:
2753+ cmd = ['service', service_name, 'is-enabled']
2754+ return subprocess.call(cmd) == 0
2755+
2756+
2757+def add_new_group(group_name, system_group=False, gid=None):
2758+ cmd = ['groupadd']
2759+ if gid:
2760+ cmd.extend(['--gid', str(gid)])
2761+ if system_group:
2762+ cmd.append('-r')
2763+ cmd.append(group_name)
2764+ subprocess.check_call(cmd)
2765+
2766+
2767+def lsb_release():
2768+ """Return /etc/os-release in a dict."""
2769+ d = {}
2770+ with open('/etc/os-release', 'r') as lsb:
2771+ for l in lsb:
2772+ s = l.split('=')
2773+ if len(s) != 2:
2774+ continue
2775+ d[s[0].strip()] = s[1].strip()
2776+ return d
2777+
2778+
2779+def cmp_pkgrevno(package, revno, pkgcache=None):
2780+ """Compare supplied revno with the revno of the installed package.
2781+
2782+ * 1 => Installed revno is greater than supplied arg
2783+ * 0 => Installed revno is the same as supplied arg
2784+ * -1 => Installed revno is less than supplied arg
2785+
2786+ This function imports YumBase function if the pkgcache argument
2787+ is None.
2788+ """
2789+ if not pkgcache:
2790+ y = yum.YumBase()
2791+ packages = y.doPackageLists()
2792+ pkgcache = {i.Name: i.version for i in packages['installed']}
2793+ pkg = pkgcache[package]
2794+ if pkg > revno:
2795+ return 1
2796+ if pkg < revno:
2797+ return -1
2798+ return 0
2799
2800=== added file 'charmhelpers/core/host_factory/ubuntu.py'
2801--- charmhelpers/core/host_factory/ubuntu.py 1970-01-01 00:00:00 +0000
2802+++ charmhelpers/core/host_factory/ubuntu.py 2021-04-13 19:16:05 +0000
2803@@ -0,0 +1,114 @@
2804+import subprocess
2805+
2806+from charmhelpers.core.hookenv import cached
2807+from charmhelpers.core.strutils import BasicStringComparator
2808+
2809+
2810+UBUNTU_RELEASES = (
2811+ 'lucid',
2812+ 'maverick',
2813+ 'natty',
2814+ 'oneiric',
2815+ 'precise',
2816+ 'quantal',
2817+ 'raring',
2818+ 'saucy',
2819+ 'trusty',
2820+ 'utopic',
2821+ 'vivid',
2822+ 'wily',
2823+ 'xenial',
2824+ 'yakkety',
2825+ 'zesty',
2826+ 'artful',
2827+ 'bionic',
2828+ 'cosmic',
2829+ 'disco',
2830+)
2831+
2832+
2833+class CompareHostReleases(BasicStringComparator):
2834+ """Provide comparisons of Ubuntu releases.
2835+
2836+ Use in the form of
2837+
2838+ if CompareHostReleases(release) > 'trusty':
2839+ # do something with mitaka
2840+ """
2841+ _list = UBUNTU_RELEASES
2842+
2843+
2844+def service_available(service_name):
2845+ """Determine whether a system service is available"""
2846+ try:
2847+ subprocess.check_output(
2848+ ['service', service_name, 'status'],
2849+ stderr=subprocess.STDOUT).decode('UTF-8')
2850+ except subprocess.CalledProcessError as e:
2851+ return b'unrecognized service' not in e.output
2852+ else:
2853+ return True
2854+
2855+
2856+def add_new_group(group_name, system_group=False, gid=None):
2857+ cmd = ['addgroup']
2858+ if gid:
2859+ cmd.extend(['--gid', str(gid)])
2860+ if system_group:
2861+ cmd.append('--system')
2862+ else:
2863+ cmd.extend([
2864+ '--group',
2865+ ])
2866+ cmd.append(group_name)
2867+ subprocess.check_call(cmd)
2868+
2869+
2870+def lsb_release():
2871+ """Return /etc/lsb-release in a dict"""
2872+ d = {}
2873+ with open('/etc/lsb-release', 'r') as lsb:
2874+ for l in lsb:
2875+ k, v = l.split('=')
2876+ d[k.strip()] = v.strip()
2877+ return d
2878+
2879+
2880+def get_distrib_codename():
2881+ """Return the codename of the distribution
2882+ :returns: The codename
2883+ :rtype: str
2884+ """
2885+ return lsb_release()['DISTRIB_CODENAME'].lower()
2886+
2887+
2888+def cmp_pkgrevno(package, revno, pkgcache=None):
2889+ """Compare supplied revno with the revno of the installed package.
2890+
2891+ * 1 => Installed revno is greater than supplied arg
2892+ * 0 => Installed revno is the same as supplied arg
2893+ * -1 => Installed revno is less than supplied arg
2894+
2895+ This function imports apt_cache function from charmhelpers.fetch if
2896+ the pkgcache argument is None. Be sure to add charmhelpers.fetch if
2897+ you call this function, or pass an apt_pkg.Cache() instance.
2898+ """
2899+ import apt_pkg
2900+ if not pkgcache:
2901+ from charmhelpers.fetch import apt_cache
2902+ pkgcache = apt_cache()
2903+ pkg = pkgcache[package]
2904+ return apt_pkg.version_compare(pkg.current_ver.ver_str, revno)
2905+
2906+
2907+@cached
2908+def arch():
2909+ """Return the package architecture as a string.
2910+
2911+ :returns: the architecture
2912+ :rtype: str
2913+ :raises: subprocess.CalledProcessError if dpkg command fails
2914+ """
2915+ return subprocess.check_output(
2916+ ['dpkg', '--print-architecture']
2917+ ).rstrip().decode('UTF-8')
2918
2919=== modified file 'charmhelpers/core/hugepage.py'
2920--- charmhelpers/core/hugepage.py 2015-12-11 15:23:38 +0000
2921+++ charmhelpers/core/hugepage.py 2021-04-13 19:16:05 +0000
2922@@ -2,19 +2,17 @@
2923
2924 # Copyright 2014-2015 Canonical Limited.
2925 #
2926-# This file is part of charm-helpers.
2927-#
2928-# charm-helpers is free software: you can redistribute it and/or modify
2929-# it under the terms of the GNU Lesser General Public License version 3 as
2930-# published by the Free Software Foundation.
2931-#
2932-# charm-helpers is distributed in the hope that it will be useful,
2933-# but WITHOUT ANY WARRANTY; without even the implied warranty of
2934-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
2935-# GNU Lesser General Public License for more details.
2936-#
2937-# You should have received a copy of the GNU Lesser General Public License
2938-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
2939+# Licensed under the Apache License, Version 2.0 (the "License");
2940+# you may not use this file except in compliance with the License.
2941+# You may obtain a copy of the License at
2942+#
2943+# http://www.apache.org/licenses/LICENSE-2.0
2944+#
2945+# Unless required by applicable law or agreed to in writing, software
2946+# distributed under the License is distributed on an "AS IS" BASIS,
2947+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
2948+# See the License for the specific language governing permissions and
2949+# limitations under the License.
2950
2951 import yaml
2952 from charmhelpers.core import fstab
2953
2954=== modified file 'charmhelpers/core/kernel.py'
2955--- charmhelpers/core/kernel.py 2015-12-11 15:23:38 +0000
2956+++ charmhelpers/core/kernel.py 2021-04-13 19:16:05 +0000
2957@@ -3,29 +3,40 @@
2958
2959 # Copyright 2014-2015 Canonical Limited.
2960 #
2961-# This file is part of charm-helpers.
2962-#
2963-# charm-helpers is free software: you can redistribute it and/or modify
2964-# it under the terms of the GNU Lesser General Public License version 3 as
2965-# published by the Free Software Foundation.
2966-#
2967-# charm-helpers is distributed in the hope that it will be useful,
2968-# but WITHOUT ANY WARRANTY; without even the implied warranty of
2969-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
2970-# GNU Lesser General Public License for more details.
2971-#
2972-# You should have received a copy of the GNU Lesser General Public License
2973-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
2974-
2975-__author__ = "Jorge Niedbalski <jorge.niedbalski@canonical.com>"
2976-
2977+# Licensed under the Apache License, Version 2.0 (the "License");
2978+# you may not use this file except in compliance with the License.
2979+# You may obtain a copy of the License at
2980+#
2981+# http://www.apache.org/licenses/LICENSE-2.0
2982+#
2983+# Unless required by applicable law or agreed to in writing, software
2984+# distributed under the License is distributed on an "AS IS" BASIS,
2985+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
2986+# See the License for the specific language governing permissions and
2987+# limitations under the License.
2988+
2989+import re
2990+import subprocess
2991+
2992+from charmhelpers.osplatform import get_platform
2993 from charmhelpers.core.hookenv import (
2994 log,
2995 INFO
2996 )
2997
2998-from subprocess import check_call, check_output
2999-import re
3000+__platform__ = get_platform()
3001+if __platform__ == "ubuntu":
3002+ from charmhelpers.core.kernel_factory.ubuntu import ( # NOQA:F401
3003+ persistent_modprobe,
3004+ update_initramfs,
3005+ ) # flake8: noqa -- ignore F401 for this import
3006+elif __platform__ == "centos":
3007+ from charmhelpers.core.kernel_factory.centos import ( # NOQA:F401
3008+ persistent_modprobe,
3009+ update_initramfs,
3010+ ) # flake8: noqa -- ignore F401 for this import
3011+
3012+__author__ = "Jorge Niedbalski <jorge.niedbalski@canonical.com>"
3013
3014
3015 def modprobe(module, persist=True):
3016@@ -34,11 +45,9 @@
3017
3018 log('Loading kernel module %s' % module, level=INFO)
3019
3020- check_call(cmd)
3021+ subprocess.check_call(cmd)
3022 if persist:
3023- with open('/etc/modules', 'r+') as modules:
3024- if module not in modules.read():
3025- modules.write(module)
3026+ persistent_modprobe(module)
3027
3028
3029 def rmmod(module, force=False):
3030@@ -48,21 +57,16 @@
3031 cmd.append('-f')
3032 cmd.append(module)
3033 log('Removing kernel module %s' % module, level=INFO)
3034- return check_call(cmd)
3035+ return subprocess.check_call(cmd)
3036
3037
3038 def lsmod():
3039 """Shows what kernel modules are currently loaded"""
3040- return check_output(['lsmod'],
3041- universal_newlines=True)
3042+ return subprocess.check_output(['lsmod'],
3043+ universal_newlines=True)
3044
3045
3046 def is_module_loaded(module):
3047 """Checks if a kernel module is already loaded"""
3048 matches = re.findall('^%s[ ]+' % module, lsmod(), re.M)
3049 return len(matches) > 0
3050-
3051-
3052-def update_initramfs(version='all'):
3053- """Updates an initramfs image"""
3054- return check_call(["update-initramfs", "-k", version, "-u"])
3055
3056=== added directory 'charmhelpers/core/kernel_factory'
3057=== added file 'charmhelpers/core/kernel_factory/__init__.py'
3058=== added file 'charmhelpers/core/kernel_factory/centos.py'
3059--- charmhelpers/core/kernel_factory/centos.py 1970-01-01 00:00:00 +0000
3060+++ charmhelpers/core/kernel_factory/centos.py 2021-04-13 19:16:05 +0000
3061@@ -0,0 +1,17 @@
3062+import subprocess
3063+import os
3064+
3065+
3066+def persistent_modprobe(module):
3067+ """Load a kernel module and configure for auto-load on reboot."""
3068+ if not os.path.exists('/etc/rc.modules'):
3069+ open('/etc/rc.modules', 'a')
3070+ os.chmod('/etc/rc.modules', 111)
3071+ with open('/etc/rc.modules', 'r+') as modules:
3072+ if module not in modules.read():
3073+ modules.write('modprobe %s\n' % module)
3074+
3075+
3076+def update_initramfs(version='all'):
3077+ """Updates an initramfs image."""
3078+ return subprocess.check_call(["dracut", "-f", version])
3079
3080=== added file 'charmhelpers/core/kernel_factory/ubuntu.py'
3081--- charmhelpers/core/kernel_factory/ubuntu.py 1970-01-01 00:00:00 +0000
3082+++ charmhelpers/core/kernel_factory/ubuntu.py 2021-04-13 19:16:05 +0000
3083@@ -0,0 +1,13 @@
3084+import subprocess
3085+
3086+
3087+def persistent_modprobe(module):
3088+ """Load a kernel module and configure for auto-load on reboot."""
3089+ with open('/etc/modules', 'r+') as modules:
3090+ if module not in modules.read():
3091+ modules.write(module + "\n")
3092+
3093+
3094+def update_initramfs(version='all'):
3095+ """Updates an initramfs image."""
3096+ return subprocess.check_call(["update-initramfs", "-k", version, "-u"])
3097
3098=== modified file 'charmhelpers/core/services/__init__.py'
3099--- charmhelpers/core/services/__init__.py 2015-01-28 08:59:02 +0000
3100+++ charmhelpers/core/services/__init__.py 2021-04-13 19:16:05 +0000
3101@@ -1,18 +1,16 @@
3102 # Copyright 2014-2015 Canonical Limited.
3103 #
3104-# This file is part of charm-helpers.
3105-#
3106-# charm-helpers is free software: you can redistribute it and/or modify
3107-# it under the terms of the GNU Lesser General Public License version 3 as
3108-# published by the Free Software Foundation.
3109-#
3110-# charm-helpers is distributed in the hope that it will be useful,
3111-# but WITHOUT ANY WARRANTY; without even the implied warranty of
3112-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
3113-# GNU Lesser General Public License for more details.
3114-#
3115-# You should have received a copy of the GNU Lesser General Public License
3116-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
3117+# Licensed under the Apache License, Version 2.0 (the "License");
3118+# you may not use this file except in compliance with the License.
3119+# You may obtain a copy of the License at
3120+#
3121+# http://www.apache.org/licenses/LICENSE-2.0
3122+#
3123+# Unless required by applicable law or agreed to in writing, software
3124+# distributed under the License is distributed on an "AS IS" BASIS,
3125+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
3126+# See the License for the specific language governing permissions and
3127+# limitations under the License.
3128
3129 from .base import * # NOQA
3130 from .helpers import * # NOQA
3131
3132=== modified file 'charmhelpers/core/services/base.py'
3133--- charmhelpers/core/services/base.py 2015-07-03 09:13:26 +0000
3134+++ charmhelpers/core/services/base.py 2021-04-13 19:16:05 +0000
3135@@ -1,18 +1,16 @@
3136 # Copyright 2014-2015 Canonical Limited.
3137 #
3138-# This file is part of charm-helpers.
3139-#
3140-# charm-helpers is free software: you can redistribute it and/or modify
3141-# it under the terms of the GNU Lesser General Public License version 3 as
3142-# published by the Free Software Foundation.
3143-#
3144-# charm-helpers is distributed in the hope that it will be useful,
3145-# but WITHOUT ANY WARRANTY; without even the implied warranty of
3146-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
3147-# GNU Lesser General Public License for more details.
3148-#
3149-# You should have received a copy of the GNU Lesser General Public License
3150-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
3151+# Licensed under the Apache License, Version 2.0 (the "License");
3152+# you may not use this file except in compliance with the License.
3153+# You may obtain a copy of the License at
3154+#
3155+# http://www.apache.org/licenses/LICENSE-2.0
3156+#
3157+# Unless required by applicable law or agreed to in writing, software
3158+# distributed under the License is distributed on an "AS IS" BASIS,
3159+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
3160+# See the License for the specific language governing permissions and
3161+# limitations under the License.
3162
3163 import os
3164 import json
3165@@ -309,23 +307,34 @@
3166 """
3167 def __call__(self, manager, service_name, event_name):
3168 service = manager.get_service(service_name)
3169- new_ports = service.get('ports', [])
3170+ # turn this generator into a list,
3171+ # as we'll be going over it multiple times
3172+ new_ports = list(service.get('ports', []))
3173 port_file = os.path.join(hookenv.charm_dir(), '.{}.ports'.format(service_name))
3174 if os.path.exists(port_file):
3175 with open(port_file) as fp:
3176 old_ports = fp.read().split(',')
3177 for old_port in old_ports:
3178- if bool(old_port):
3179- old_port = int(old_port)
3180- if old_port not in new_ports:
3181- hookenv.close_port(old_port)
3182+ if bool(old_port) and not self.ports_contains(old_port, new_ports):
3183+ hookenv.close_port(old_port)
3184 with open(port_file, 'w') as fp:
3185 fp.write(','.join(str(port) for port in new_ports))
3186 for port in new_ports:
3187+ # A port is either a number or 'ICMP'
3188+ protocol = 'TCP'
3189+ if str(port).upper() == 'ICMP':
3190+ protocol = 'ICMP'
3191 if event_name == 'start':
3192- hookenv.open_port(port)
3193+ hookenv.open_port(port, protocol)
3194 elif event_name == 'stop':
3195- hookenv.close_port(port)
3196+ hookenv.close_port(port, protocol)
3197+
3198+ def ports_contains(self, port, ports):
3199+ if not bool(port):
3200+ return False
3201+ if str(port).upper() != 'ICMP':
3202+ port = int(port)
3203+ return port in ports
3204
3205
3206 def service_stop(service_name):
3207
3208=== modified file 'charmhelpers/core/services/helpers.py'
3209--- charmhelpers/core/services/helpers.py 2015-12-11 15:23:38 +0000
3210+++ charmhelpers/core/services/helpers.py 2021-04-13 19:16:05 +0000
3211@@ -1,18 +1,16 @@
3212 # Copyright 2014-2015 Canonical Limited.
3213 #
3214-# This file is part of charm-helpers.
3215-#
3216-# charm-helpers is free software: you can redistribute it and/or modify
3217-# it under the terms of the GNU Lesser General Public License version 3 as
3218-# published by the Free Software Foundation.
3219-#
3220-# charm-helpers is distributed in the hope that it will be useful,
3221-# but WITHOUT ANY WARRANTY; without even the implied warranty of
3222-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
3223-# GNU Lesser General Public License for more details.
3224-#
3225-# You should have received a copy of the GNU Lesser General Public License
3226-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
3227+# Licensed under the Apache License, Version 2.0 (the "License");
3228+# you may not use this file except in compliance with the License.
3229+# You may obtain a copy of the License at
3230+#
3231+# http://www.apache.org/licenses/LICENSE-2.0
3232+#
3233+# Unless required by applicable law or agreed to in writing, software
3234+# distributed under the License is distributed on an "AS IS" BASIS,
3235+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
3236+# See the License for the specific language governing permissions and
3237+# limitations under the License.
3238
3239 import os
3240 import yaml
3241
3242=== modified file 'charmhelpers/core/strutils.py'
3243--- charmhelpers/core/strutils.py 2015-12-11 15:23:38 +0000
3244+++ charmhelpers/core/strutils.py 2021-04-13 19:16:05 +0000
3245@@ -3,19 +3,17 @@
3246
3247 # Copyright 2014-2015 Canonical Limited.
3248 #
3249-# This file is part of charm-helpers.
3250-#
3251-# charm-helpers is free software: you can redistribute it and/or modify
3252-# it under the terms of the GNU Lesser General Public License version 3 as
3253-# published by the Free Software Foundation.
3254-#
3255-# charm-helpers is distributed in the hope that it will be useful,
3256-# but WITHOUT ANY WARRANTY; without even the implied warranty of
3257-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
3258-# GNU Lesser General Public License for more details.
3259-#
3260-# You should have received a copy of the GNU Lesser General Public License
3261-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
3262+# Licensed under the Apache License, Version 2.0 (the "License");
3263+# you may not use this file except in compliance with the License.
3264+# You may obtain a copy of the License at
3265+#
3266+# http://www.apache.org/licenses/LICENSE-2.0
3267+#
3268+# Unless required by applicable law or agreed to in writing, software
3269+# distributed under the License is distributed on an "AS IS" BASIS,
3270+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
3271+# See the License for the specific language governing permissions and
3272+# limitations under the License.
3273
3274 import six
3275 import re
3276@@ -63,10 +61,69 @@
3277 if isinstance(value, six.string_types):
3278 value = six.text_type(value)
3279 else:
3280- msg = "Unable to interpret non-string value '%s' as boolean" % (value)
3281+ msg = "Unable to interpret non-string value '%s' as bytes" % (value)
3282 raise ValueError(msg)
3283 matches = re.match("([0-9]+)([a-zA-Z]+)", value)
3284- if not matches:
3285- msg = "Unable to interpret string value '%s' as bytes" % (value)
3286- raise ValueError(msg)
3287- return int(matches.group(1)) * (1024 ** BYTE_POWER[matches.group(2)])
3288+ if matches:
3289+ size = int(matches.group(1)) * (1024 ** BYTE_POWER[matches.group(2)])
3290+ else:
3291+ # Assume that value passed in is bytes
3292+ try:
3293+ size = int(value)
3294+ except ValueError:
3295+ msg = "Unable to interpret string value '%s' as bytes" % (value)
3296+ raise ValueError(msg)
3297+ return size
3298+
3299+
3300+class BasicStringComparator(object):
3301+ """Provides a class that will compare strings from an iterator type object.
3302+ Used to provide > and < comparisons on strings that may not necessarily be
3303+ alphanumerically ordered. e.g. OpenStack or Ubuntu releases AFTER the
3304+ z-wrap.
3305+ """
3306+
3307+ _list = None
3308+
3309+ def __init__(self, item):
3310+ if self._list is None:
3311+ raise Exception("Must define the _list in the class definition!")
3312+ try:
3313+ self.index = self._list.index(item)
3314+ except Exception:
3315+ raise KeyError("Item '{}' is not in list '{}'"
3316+ .format(item, self._list))
3317+
3318+ def __eq__(self, other):
3319+ assert isinstance(other, str) or isinstance(other, self.__class__)
3320+ return self.index == self._list.index(other)
3321+
3322+ def __ne__(self, other):
3323+ return not self.__eq__(other)
3324+
3325+ def __lt__(self, other):
3326+ assert isinstance(other, str) or isinstance(other, self.__class__)
3327+ return self.index < self._list.index(other)
3328+
3329+ def __ge__(self, other):
3330+ return not self.__lt__(other)
3331+
3332+ def __gt__(self, other):
3333+ assert isinstance(other, str) or isinstance(other, self.__class__)
3334+ return self.index > self._list.index(other)
3335+
3336+ def __le__(self, other):
3337+ return not self.__gt__(other)
3338+
3339+ def __str__(self):
3340+ """Always give back the item at the index so it can be used in
3341+ comparisons like:
3342+
3343+ s_mitaka = CompareOpenStack('mitaka')
3344+ s_newton = CompareOpenstack('newton')
3345+
3346+ assert s_newton > s_mitaka
3347+
3348+ @returns: <string>
3349+ """
3350+ return self._list[self.index]
3351
3352=== modified file 'charmhelpers/core/sysctl.py'
3353--- charmhelpers/core/sysctl.py 2015-03-12 11:42:26 +0000
3354+++ charmhelpers/core/sysctl.py 2021-04-13 19:16:05 +0000
3355@@ -3,19 +3,17 @@
3356
3357 # Copyright 2014-2015 Canonical Limited.
3358 #
3359-# This file is part of charm-helpers.
3360-#
3361-# charm-helpers is free software: you can redistribute it and/or modify
3362-# it under the terms of the GNU Lesser General Public License version 3 as
3363-# published by the Free Software Foundation.
3364-#
3365-# charm-helpers is distributed in the hope that it will be useful,
3366-# but WITHOUT ANY WARRANTY; without even the implied warranty of
3367-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
3368-# GNU Lesser General Public License for more details.
3369-#
3370-# You should have received a copy of the GNU Lesser General Public License
3371-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
3372+# Licensed under the Apache License, Version 2.0 (the "License");
3373+# you may not use this file except in compliance with the License.
3374+# You may obtain a copy of the License at
3375+#
3376+# http://www.apache.org/licenses/LICENSE-2.0
3377+#
3378+# Unless required by applicable law or agreed to in writing, software
3379+# distributed under the License is distributed on an "AS IS" BASIS,
3380+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
3381+# See the License for the specific language governing permissions and
3382+# limitations under the License.
3383
3384 import yaml
3385
3386@@ -30,27 +28,38 @@
3387 __author__ = 'Jorge Niedbalski R. <jorge.niedbalski@canonical.com>'
3388
3389
3390-def create(sysctl_dict, sysctl_file):
3391+def create(sysctl_dict, sysctl_file, ignore=False):
3392 """Creates a sysctl.conf file from a YAML associative array
3393
3394- :param sysctl_dict: a YAML-formatted string of sysctl options eg "{ 'kernel.max_pid': 1337 }"
3395+ :param sysctl_dict: a dict or YAML-formatted string of sysctl
3396+ options eg "{ 'kernel.max_pid': 1337 }"
3397 :type sysctl_dict: str
3398 :param sysctl_file: path to the sysctl file to be saved
3399 :type sysctl_file: str or unicode
3400+ :param ignore: If True, ignore "unknown variable" errors.
3401+ :type ignore: bool
3402 :returns: None
3403 """
3404- try:
3405- sysctl_dict_parsed = yaml.safe_load(sysctl_dict)
3406- except yaml.YAMLError:
3407- log("Error parsing YAML sysctl_dict: {}".format(sysctl_dict),
3408- level=ERROR)
3409- return
3410+ if type(sysctl_dict) is not dict:
3411+ try:
3412+ sysctl_dict_parsed = yaml.safe_load(sysctl_dict)
3413+ except yaml.YAMLError:
3414+ log("Error parsing YAML sysctl_dict: {}".format(sysctl_dict),
3415+ level=ERROR)
3416+ return
3417+ else:
3418+ sysctl_dict_parsed = sysctl_dict
3419
3420 with open(sysctl_file, "w") as fd:
3421 for key, value in sysctl_dict_parsed.items():
3422 fd.write("{}={}\n".format(key, value))
3423
3424- log("Updating sysctl_file: %s values: %s" % (sysctl_file, sysctl_dict_parsed),
3425+ log("Updating sysctl_file: {} values: {}".format(sysctl_file,
3426+ sysctl_dict_parsed),
3427 level=DEBUG)
3428
3429- check_call(["sysctl", "-p", sysctl_file])
3430+ call = ["sysctl", "-p", sysctl_file]
3431+ if ignore:
3432+ call.append("-e")
3433+
3434+ check_call(call)
3435
3436=== modified file 'charmhelpers/core/templating.py'
3437--- charmhelpers/core/templating.py 2015-12-11 15:23:38 +0000
3438+++ charmhelpers/core/templating.py 2021-04-13 19:16:05 +0000
3439@@ -1,27 +1,27 @@
3440 # Copyright 2014-2015 Canonical Limited.
3441 #
3442-# This file is part of charm-helpers.
3443-#
3444-# charm-helpers is free software: you can redistribute it and/or modify
3445-# it under the terms of the GNU Lesser General Public License version 3 as
3446-# published by the Free Software Foundation.
3447-#
3448-# charm-helpers is distributed in the hope that it will be useful,
3449-# but WITHOUT ANY WARRANTY; without even the implied warranty of
3450-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
3451-# GNU Lesser General Public License for more details.
3452-#
3453-# You should have received a copy of the GNU Lesser General Public License
3454-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
3455+# Licensed under the Apache License, Version 2.0 (the "License");
3456+# you may not use this file except in compliance with the License.
3457+# You may obtain a copy of the License at
3458+#
3459+# http://www.apache.org/licenses/LICENSE-2.0
3460+#
3461+# Unless required by applicable law or agreed to in writing, software
3462+# distributed under the License is distributed on an "AS IS" BASIS,
3463+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
3464+# See the License for the specific language governing permissions and
3465+# limitations under the License.
3466
3467 import os
3468+import sys
3469
3470 from charmhelpers.core import host
3471 from charmhelpers.core import hookenv
3472
3473
3474 def render(source, target, context, owner='root', group='root',
3475- perms=0o444, templates_dir=None, encoding='UTF-8', template_loader=None):
3476+ perms=0o444, templates_dir=None, encoding='UTF-8',
3477+ template_loader=None, config_template=None):
3478 """
3479 Render a template.
3480
3481@@ -33,6 +33,9 @@
3482 The context should be a dict containing the values to be replaced in the
3483 template.
3484
3485+ config_template may be provided to render from a provided template instead
3486+ of loading from a file.
3487+
3488 The `owner`, `group`, and `perms` options will be passed to `write_file`.
3489
3490 If omitted, `templates_dir` defaults to the `templates` folder in the charm.
3491@@ -40,8 +43,9 @@
3492 The rendered template will be written to the file as well as being returned
3493 as a string.
3494
3495- Note: Using this requires python-jinja2; if it is not installed, calling
3496- this will attempt to use charmhelpers.fetch.apt_install to install it.
3497+ Note: Using this requires python-jinja2 or python3-jinja2; if it is not
3498+ installed, calling this will attempt to use charmhelpers.fetch.apt_install
3499+ to install it.
3500 """
3501 try:
3502 from jinja2 import FileSystemLoader, Environment, exceptions
3503@@ -53,7 +57,10 @@
3504 'charmhelpers.fetch to install it',
3505 level=hookenv.ERROR)
3506 raise
3507- apt_install('python-jinja2', fatal=True)
3508+ if sys.version_info.major == 2:
3509+ apt_install('python-jinja2', fatal=True)
3510+ else:
3511+ apt_install('python3-jinja2', fatal=True)
3512 from jinja2 import FileSystemLoader, Environment, exceptions
3513
3514 if template_loader:
3515@@ -62,14 +69,19 @@
3516 if templates_dir is None:
3517 templates_dir = os.path.join(hookenv.charm_dir(), 'templates')
3518 template_env = Environment(loader=FileSystemLoader(templates_dir))
3519- try:
3520- source = source
3521- template = template_env.get_template(source)
3522- except exceptions.TemplateNotFound as e:
3523- hookenv.log('Could not load template %s from %s.' %
3524- (source, templates_dir),
3525- level=hookenv.ERROR)
3526- raise e
3527+
3528+ # load from a string if provided explicitly
3529+ if config_template is not None:
3530+ template = template_env.from_string(config_template)
3531+ else:
3532+ try:
3533+ source = source
3534+ template = template_env.get_template(source)
3535+ except exceptions.TemplateNotFound as e:
3536+ hookenv.log('Could not load template %s from %s.' %
3537+ (source, templates_dir),
3538+ level=hookenv.ERROR)
3539+ raise e
3540 content = template.render(context)
3541 if target is not None:
3542 target_dir = os.path.dirname(target)
3543
3544=== modified file 'charmhelpers/core/unitdata.py'
3545--- charmhelpers/core/unitdata.py 2015-12-11 15:23:38 +0000
3546+++ charmhelpers/core/unitdata.py 2021-04-13 19:16:05 +0000
3547@@ -3,20 +3,17 @@
3548 #
3549 # Copyright 2014-2015 Canonical Limited.
3550 #
3551-# This file is part of charm-helpers.
3552-#
3553-# charm-helpers is free software: you can redistribute it and/or modify
3554-# it under the terms of the GNU Lesser General Public License version 3 as
3555-# published by the Free Software Foundation.
3556-#
3557-# charm-helpers is distributed in the hope that it will be useful,
3558-# but WITHOUT ANY WARRANTY; without even the implied warranty of
3559-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
3560-# GNU Lesser General Public License for more details.
3561-#
3562-# You should have received a copy of the GNU Lesser General Public License
3563-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
3564-#
3565+# Licensed under the Apache License, Version 2.0 (the "License");
3566+# you may not use this file except in compliance with the License.
3567+# You may obtain a copy of the License at
3568+#
3569+# http://www.apache.org/licenses/LICENSE-2.0
3570+#
3571+# Unless required by applicable law or agreed to in writing, software
3572+# distributed under the License is distributed on an "AS IS" BASIS,
3573+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
3574+# See the License for the specific language governing permissions and
3575+# limitations under the License.
3576 #
3577 # Authors:
3578 # Kapil Thangavelu <kapil.foss@gmail.com>
3579@@ -169,6 +166,10 @@
3580
3581 To support dicts, lists, integer, floats, and booleans values
3582 are automatically json encoded/decoded.
3583+
3584+ Note: to facilitate unit testing, ':memory:' can be passed as the
3585+ path parameter which causes sqlite3 to only build the db in memory.
3586+ This should only be used for testing purposes.
3587 """
3588 def __init__(self, path=None):
3589 self.db_path = path
3590@@ -178,6 +179,9 @@
3591 else:
3592 self.db_path = os.path.join(
3593 os.environ.get('CHARM_DIR', ''), '.unit-state.db')
3594+ if self.db_path != ':memory:':
3595+ with open(self.db_path, 'a') as f:
3596+ os.fchmod(f.fileno(), 0o600)
3597 self.conn = sqlite3.connect('%s' % self.db_path)
3598 self.cursor = self.conn.cursor()
3599 self.revision = None
3600@@ -361,7 +365,7 @@
3601 try:
3602 yield self.revision
3603 self.revision = None
3604- except:
3605+ except Exception:
3606 self.flush(False)
3607 self.revision = None
3608 raise
3609
3610=== modified file 'charmhelpers/fetch/__init__.py'
3611--- charmhelpers/fetch/__init__.py 2015-12-11 15:23:38 +0000
3612+++ charmhelpers/fetch/__init__.py 2021-04-13 19:16:05 +0000
3613@@ -1,32 +1,24 @@
3614 # Copyright 2014-2015 Canonical Limited.
3615 #
3616-# This file is part of charm-helpers.
3617-#
3618-# charm-helpers is free software: you can redistribute it and/or modify
3619-# it under the terms of the GNU Lesser General Public License version 3 as
3620-# published by the Free Software Foundation.
3621-#
3622-# charm-helpers is distributed in the hope that it will be useful,
3623-# but WITHOUT ANY WARRANTY; without even the implied warranty of
3624-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
3625-# GNU Lesser General Public License for more details.
3626-#
3627-# You should have received a copy of the GNU Lesser General Public License
3628-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
3629+# Licensed under the Apache License, Version 2.0 (the "License");
3630+# you may not use this file except in compliance with the License.
3631+# You may obtain a copy of the License at
3632+#
3633+# http://www.apache.org/licenses/LICENSE-2.0
3634+#
3635+# Unless required by applicable law or agreed to in writing, software
3636+# distributed under the License is distributed on an "AS IS" BASIS,
3637+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
3638+# See the License for the specific language governing permissions and
3639+# limitations under the License.
3640
3641 import importlib
3642-from tempfile import NamedTemporaryFile
3643-import time
3644+from charmhelpers.osplatform import get_platform
3645 from yaml import safe_load
3646-from charmhelpers.core.host import (
3647- lsb_release
3648-)
3649-import subprocess
3650 from charmhelpers.core.hookenv import (
3651 config,
3652 log,
3653 )
3654-import os
3655
3656 import six
3657 if six.PY3:
3658@@ -35,71 +27,6 @@
3659 from urlparse import urlparse, urlunparse
3660
3661
3662-CLOUD_ARCHIVE = """# Ubuntu Cloud Archive
3663-deb http://ubuntu-cloud.archive.canonical.com/ubuntu {} main
3664-"""
3665-PROPOSED_POCKET = """# Proposed
3666-deb http://archive.ubuntu.com/ubuntu {}-proposed main universe multiverse restricted
3667-"""
3668-CLOUD_ARCHIVE_POCKETS = {
3669- # Folsom
3670- 'folsom': 'precise-updates/folsom',
3671- 'precise-folsom': 'precise-updates/folsom',
3672- 'precise-folsom/updates': 'precise-updates/folsom',
3673- 'precise-updates/folsom': 'precise-updates/folsom',
3674- 'folsom/proposed': 'precise-proposed/folsom',
3675- 'precise-folsom/proposed': 'precise-proposed/folsom',
3676- 'precise-proposed/folsom': 'precise-proposed/folsom',
3677- # Grizzly
3678- 'grizzly': 'precise-updates/grizzly',
3679- 'precise-grizzly': 'precise-updates/grizzly',
3680- 'precise-grizzly/updates': 'precise-updates/grizzly',
3681- 'precise-updates/grizzly': 'precise-updates/grizzly',
3682- 'grizzly/proposed': 'precise-proposed/grizzly',
3683- 'precise-grizzly/proposed': 'precise-proposed/grizzly',
3684- 'precise-proposed/grizzly': 'precise-proposed/grizzly',
3685- # Havana
3686- 'havana': 'precise-updates/havana',
3687- 'precise-havana': 'precise-updates/havana',
3688- 'precise-havana/updates': 'precise-updates/havana',
3689- 'precise-updates/havana': 'precise-updates/havana',
3690- 'havana/proposed': 'precise-proposed/havana',
3691- 'precise-havana/proposed': 'precise-proposed/havana',
3692- 'precise-proposed/havana': 'precise-proposed/havana',
3693- # Icehouse
3694- 'icehouse': 'precise-updates/icehouse',
3695- 'precise-icehouse': 'precise-updates/icehouse',
3696- 'precise-icehouse/updates': 'precise-updates/icehouse',
3697- 'precise-updates/icehouse': 'precise-updates/icehouse',
3698- 'icehouse/proposed': 'precise-proposed/icehouse',
3699- 'precise-icehouse/proposed': 'precise-proposed/icehouse',
3700- 'precise-proposed/icehouse': 'precise-proposed/icehouse',
3701- # Juno
3702- 'juno': 'trusty-updates/juno',
3703- 'trusty-juno': 'trusty-updates/juno',
3704- 'trusty-juno/updates': 'trusty-updates/juno',
3705- 'trusty-updates/juno': 'trusty-updates/juno',
3706- 'juno/proposed': 'trusty-proposed/juno',
3707- 'trusty-juno/proposed': 'trusty-proposed/juno',
3708- 'trusty-proposed/juno': 'trusty-proposed/juno',
3709- # Kilo
3710- 'kilo': 'trusty-updates/kilo',
3711- 'trusty-kilo': 'trusty-updates/kilo',
3712- 'trusty-kilo/updates': 'trusty-updates/kilo',
3713- 'trusty-updates/kilo': 'trusty-updates/kilo',
3714- 'kilo/proposed': 'trusty-proposed/kilo',
3715- 'trusty-kilo/proposed': 'trusty-proposed/kilo',
3716- 'trusty-proposed/kilo': 'trusty-proposed/kilo',
3717- # Liberty
3718- 'liberty': 'trusty-updates/liberty',
3719- 'trusty-liberty': 'trusty-updates/liberty',
3720- 'trusty-liberty/updates': 'trusty-updates/liberty',
3721- 'trusty-updates/liberty': 'trusty-updates/liberty',
3722- 'liberty/proposed': 'trusty-proposed/liberty',
3723- 'trusty-liberty/proposed': 'trusty-proposed/liberty',
3724- 'trusty-proposed/liberty': 'trusty-proposed/liberty',
3725-}
3726-
3727 # The order of this list is very important. Handlers should be listed in from
3728 # least- to most-specific URL matching.
3729 FETCH_HANDLERS = (
3730@@ -108,10 +35,6 @@
3731 'charmhelpers.fetch.giturl.GitUrlFetchHandler',
3732 )
3733
3734-APT_NO_LOCK = 100 # The return code for "couldn't acquire lock" in APT.
3735-APT_NO_LOCK_RETRY_DELAY = 10 # Wait 10 seconds between apt lock checks.
3736-APT_NO_LOCK_RETRY_COUNT = 30 # Retry to acquire the lock X times.
3737-
3738
3739 class SourceConfigError(Exception):
3740 pass
3741@@ -125,6 +48,13 @@
3742 pass
3743
3744
3745+class GPGKeyError(Exception):
3746+ """Exception occurs when a GPG key cannot be fetched or used. The message
3747+ indicates what the problem is.
3748+ """
3749+ pass
3750+
3751+
3752 class BaseFetchHandler(object):
3753
3754 """Base class for FetchHandler implementations in fetch plugins"""
3755@@ -149,180 +79,41 @@
3756 return urlunparse(parts)
3757
3758
3759-def filter_installed_packages(packages):
3760- """Returns a list of packages that require installation"""
3761- cache = apt_cache()
3762- _pkgs = []
3763- for package in packages:
3764- try:
3765- p = cache[package]
3766- p.current_ver or _pkgs.append(package)
3767- except KeyError:
3768- log('Package {} has no installation candidate.'.format(package),
3769- level='WARNING')
3770- _pkgs.append(package)
3771- return _pkgs
3772-
3773-
3774-def apt_cache(in_memory=True):
3775- """Build and return an apt cache"""
3776- from apt import apt_pkg
3777- apt_pkg.init()
3778- if in_memory:
3779- apt_pkg.config.set("Dir::Cache::pkgcache", "")
3780- apt_pkg.config.set("Dir::Cache::srcpkgcache", "")
3781- return apt_pkg.Cache()
3782-
3783-
3784-def apt_install(packages, options=None, fatal=False):
3785- """Install one or more packages"""
3786- if options is None:
3787- options = ['--option=Dpkg::Options::=--force-confold']
3788-
3789- cmd = ['apt-get', '--assume-yes']
3790- cmd.extend(options)
3791- cmd.append('install')
3792- if isinstance(packages, six.string_types):
3793- cmd.append(packages)
3794- else:
3795- cmd.extend(packages)
3796- log("Installing {} with options: {}".format(packages,
3797- options))
3798- _run_apt_command(cmd, fatal)
3799-
3800-
3801-def apt_upgrade(options=None, fatal=False, dist=False):
3802- """Upgrade all packages"""
3803- if options is None:
3804- options = ['--option=Dpkg::Options::=--force-confold']
3805-
3806- cmd = ['apt-get', '--assume-yes']
3807- cmd.extend(options)
3808- if dist:
3809- cmd.append('dist-upgrade')
3810- else:
3811- cmd.append('upgrade')
3812- log("Upgrading with options: {}".format(options))
3813- _run_apt_command(cmd, fatal)
3814-
3815-
3816-def apt_update(fatal=False):
3817- """Update local apt cache"""
3818- cmd = ['apt-get', 'update']
3819- _run_apt_command(cmd, fatal)
3820-
3821-
3822-def apt_purge(packages, fatal=False):
3823- """Purge one or more packages"""
3824- cmd = ['apt-get', '--assume-yes', 'purge']
3825- if isinstance(packages, six.string_types):
3826- cmd.append(packages)
3827- else:
3828- cmd.extend(packages)
3829- log("Purging {}".format(packages))
3830- _run_apt_command(cmd, fatal)
3831-
3832-
3833-def apt_mark(packages, mark, fatal=False):
3834- """Flag one or more packages using apt-mark"""
3835- log("Marking {} as {}".format(packages, mark))
3836- cmd = ['apt-mark', mark]
3837- if isinstance(packages, six.string_types):
3838- cmd.append(packages)
3839- else:
3840- cmd.extend(packages)
3841-
3842- if fatal:
3843- subprocess.check_call(cmd, universal_newlines=True)
3844- else:
3845- subprocess.call(cmd, universal_newlines=True)
3846-
3847-
3848-def apt_hold(packages, fatal=False):
3849- return apt_mark(packages, 'hold', fatal=fatal)
3850-
3851-
3852-def apt_unhold(packages, fatal=False):
3853- return apt_mark(packages, 'unhold', fatal=fatal)
3854-
3855-
3856-def add_source(source, key=None):
3857- """Add a package source to this system.
3858-
3859- @param source: a URL or sources.list entry, as supported by
3860- add-apt-repository(1). Examples::
3861-
3862- ppa:charmers/example
3863- deb https://stub:key@private.example.com/ubuntu trusty main
3864-
3865- In addition:
3866- 'proposed:' may be used to enable the standard 'proposed'
3867- pocket for the release.
3868- 'cloud:' may be used to activate official cloud archive pockets,
3869- such as 'cloud:icehouse'
3870- 'distro' may be used as a noop
3871-
3872- @param key: A key to be added to the system's APT keyring and used
3873- to verify the signatures on packages. Ideally, this should be an
3874- ASCII format GPG public key including the block headers. A GPG key
3875- id may also be used, but be aware that only insecure protocols are
3876- available to retrieve the actual public key from a public keyserver
3877- placing your Juju environment at risk. ppa and cloud archive keys
3878- are securely added automtically, so sould not be provided.
3879- """
3880- if source is None:
3881- log('Source is not present. Skipping')
3882- return
3883-
3884- if (source.startswith('ppa:') or
3885- source.startswith('http') or
3886- source.startswith('deb ') or
3887- source.startswith('cloud-archive:')):
3888- subprocess.check_call(['add-apt-repository', '--yes', source])
3889- elif source.startswith('cloud:'):
3890- apt_install(filter_installed_packages(['ubuntu-cloud-keyring']),
3891- fatal=True)
3892- pocket = source.split(':')[-1]
3893- if pocket not in CLOUD_ARCHIVE_POCKETS:
3894- raise SourceConfigError(
3895- 'Unsupported cloud: source option %s' %
3896- pocket)
3897- actual_pocket = CLOUD_ARCHIVE_POCKETS[pocket]
3898- with open('/etc/apt/sources.list.d/cloud-archive.list', 'w') as apt:
3899- apt.write(CLOUD_ARCHIVE.format(actual_pocket))
3900- elif source == 'proposed':
3901- release = lsb_release()['DISTRIB_CODENAME']
3902- with open('/etc/apt/sources.list.d/proposed.list', 'w') as apt:
3903- apt.write(PROPOSED_POCKET.format(release))
3904- elif source == 'distro':
3905- pass
3906- else:
3907- log("Unknown source: {!r}".format(source))
3908-
3909- if key:
3910- if '-----BEGIN PGP PUBLIC KEY BLOCK-----' in key:
3911- with NamedTemporaryFile('w+') as key_file:
3912- key_file.write(key)
3913- key_file.flush()
3914- key_file.seek(0)
3915- subprocess.check_call(['apt-key', 'add', '-'], stdin=key_file)
3916- else:
3917- # Note that hkp: is in no way a secure protocol. Using a
3918- # GPG key id is pointless from a security POV unless you
3919- # absolutely trust your network and DNS.
3920- subprocess.check_call(['apt-key', 'adv', '--keyserver',
3921- 'hkp://keyserver.ubuntu.com:80', '--recv',
3922- key])
3923+__platform__ = get_platform()
3924+module = "charmhelpers.fetch.%s" % __platform__
3925+fetch = importlib.import_module(module)
3926+
3927+filter_installed_packages = fetch.filter_installed_packages
3928+filter_missing_packages = fetch.filter_missing_packages
3929+install = fetch.apt_install
3930+upgrade = fetch.apt_upgrade
3931+update = _fetch_update = fetch.apt_update
3932+purge = fetch.apt_purge
3933+add_source = fetch.add_source
3934+
3935+if __platform__ == "ubuntu":
3936+ apt_cache = fetch.apt_cache
3937+ apt_install = fetch.apt_install
3938+ apt_update = fetch.apt_update
3939+ apt_upgrade = fetch.apt_upgrade
3940+ apt_purge = fetch.apt_purge
3941+ apt_autoremove = fetch.apt_autoremove
3942+ apt_mark = fetch.apt_mark
3943+ apt_hold = fetch.apt_hold
3944+ apt_unhold = fetch.apt_unhold
3945+ import_key = fetch.import_key
3946+ get_upstream_version = fetch.get_upstream_version
3947+elif __platform__ == "centos":
3948+ yum_search = fetch.yum_search
3949
3950
3951 def configure_sources(update=False,
3952 sources_var='install_sources',
3953 keys_var='install_keys'):
3954- """
3955- Configure multiple sources from charm configuration.
3956+ """Configure multiple sources from charm configuration.
3957
3958 The lists are encoded as yaml fragments in the configuration.
3959- The frament needs to be included as a string. Sources and their
3960+ The fragment needs to be included as a string. Sources and their
3961 corresponding keys are of the types supported by add_source().
3962
3963 Example config:
3964@@ -354,12 +145,11 @@
3965 for source, key in zip(sources, keys):
3966 add_source(source, key)
3967 if update:
3968- apt_update(fatal=True)
3969+ _fetch_update(fatal=True)
3970
3971
3972 def install_remote(source, *args, **kwargs):
3973- """
3974- Install a file tree from a remote source
3975+ """Install a file tree from a remote source.
3976
3977 The specified source should be a url of the form:
3978 scheme://[host]/path[#[option=value][&...]]
3979@@ -382,19 +172,17 @@
3980 # We ONLY check for True here because can_handle may return a string
3981 # explaining why it can't handle a given source.
3982 handlers = [h for h in plugins() if h.can_handle(source) is True]
3983- installed_to = None
3984 for handler in handlers:
3985 try:
3986- installed_to = handler.install(source, *args, **kwargs)
3987+ return handler.install(source, *args, **kwargs)
3988 except UnhandledSource as e:
3989 log('Install source attempt unsuccessful: {}'.format(e),
3990 level='WARNING')
3991- if not installed_to:
3992- raise UnhandledSource("No handler found for source {}".format(source))
3993- return installed_to
3994+ raise UnhandledSource("No handler found for source {}".format(source))
3995
3996
3997 def install_from_config(config_var_name):
3998+ """Install a file from config."""
3999 charm_config = config()
4000 source = charm_config[config_var_name]
4001 return install_remote(source)
4002@@ -411,46 +199,9 @@
4003 importlib.import_module(package),
4004 classname)
4005 plugin_list.append(handler_class())
4006- except (ImportError, AttributeError):
4007+ except NotImplementedError:
4008 # Skip missing plugins so that they can be ommitted from
4009 # installation if desired
4010 log("FetchHandler {} not found, skipping plugin".format(
4011 handler_name))
4012 return plugin_list
4013-
4014-
4015-def _run_apt_command(cmd, fatal=False):
4016- """
4017- Run an APT command, checking output and retrying if the fatal flag is set
4018- to True.
4019-
4020- :param: cmd: str: The apt command to run.
4021- :param: fatal: bool: Whether the command's output should be checked and
4022- retried.
4023- """
4024- env = os.environ.copy()
4025-
4026- if 'DEBIAN_FRONTEND' not in env:
4027- env['DEBIAN_FRONTEND'] = 'noninteractive'
4028-
4029- if fatal:
4030- retry_count = 0
4031- result = None
4032-
4033- # If the command is considered "fatal", we need to retry if the apt
4034- # lock was not acquired.
4035-
4036- while result is None or result == APT_NO_LOCK:
4037- try:
4038- result = subprocess.check_call(cmd, env=env)
4039- except subprocess.CalledProcessError as e:
4040- retry_count = retry_count + 1
4041- if retry_count > APT_NO_LOCK_RETRY_COUNT:
4042- raise
4043- result = e.returncode
4044- log("Couldn't acquire DPKG lock. Will retry in {} seconds."
4045- "".format(APT_NO_LOCK_RETRY_DELAY))
4046- time.sleep(APT_NO_LOCK_RETRY_DELAY)
4047-
4048- else:
4049- subprocess.call(cmd, env=env)
4050
4051=== modified file 'charmhelpers/fetch/archiveurl.py'
4052--- charmhelpers/fetch/archiveurl.py 2015-12-11 15:23:38 +0000
4053+++ charmhelpers/fetch/archiveurl.py 2021-04-13 19:16:05 +0000
4054@@ -1,18 +1,16 @@
4055 # Copyright 2014-2015 Canonical Limited.
4056 #
4057-# This file is part of charm-helpers.
4058-#
4059-# charm-helpers is free software: you can redistribute it and/or modify
4060-# it under the terms of the GNU Lesser General Public License version 3 as
4061-# published by the Free Software Foundation.
4062-#
4063-# charm-helpers is distributed in the hope that it will be useful,
4064-# but WITHOUT ANY WARRANTY; without even the implied warranty of
4065-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
4066-# GNU Lesser General Public License for more details.
4067-#
4068-# You should have received a copy of the GNU Lesser General Public License
4069-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
4070+# Licensed under the Apache License, Version 2.0 (the "License");
4071+# you may not use this file except in compliance with the License.
4072+# You may obtain a copy of the License at
4073+#
4074+# http://www.apache.org/licenses/LICENSE-2.0
4075+#
4076+# Unless required by applicable law or agreed to in writing, software
4077+# distributed under the License is distributed on an "AS IS" BASIS,
4078+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
4079+# See the License for the specific language governing permissions and
4080+# limitations under the License.
4081
4082 import os
4083 import hashlib
4084@@ -91,7 +89,7 @@
4085 :param str source: URL pointing to an archive file.
4086 :param str dest: Local path location to download archive file to.
4087 """
4088- # propogate all exceptions
4089+ # propagate all exceptions
4090 # URLError, OSError, etc
4091 proto, netloc, path, params, query, fragment = urlparse(source)
4092 if proto in ('http', 'https'):
4093
4094=== modified file 'charmhelpers/fetch/bzrurl.py'
4095--- charmhelpers/fetch/bzrurl.py 2015-12-11 15:23:38 +0000
4096+++ charmhelpers/fetch/bzrurl.py 2021-04-13 19:16:05 +0000
4097@@ -1,70 +1,63 @@
4098 # Copyright 2014-2015 Canonical Limited.
4099 #
4100-# This file is part of charm-helpers.
4101-#
4102-# charm-helpers is free software: you can redistribute it and/or modify
4103-# it under the terms of the GNU Lesser General Public License version 3 as
4104-# published by the Free Software Foundation.
4105-#
4106-# charm-helpers is distributed in the hope that it will be useful,
4107-# but WITHOUT ANY WARRANTY; without even the implied warranty of
4108-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
4109-# GNU Lesser General Public License for more details.
4110-#
4111-# You should have received a copy of the GNU Lesser General Public License
4112-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
4113+# Licensed under the Apache License, Version 2.0 (the "License");
4114+# you may not use this file except in compliance with the License.
4115+# You may obtain a copy of the License at
4116+#
4117+# http://www.apache.org/licenses/LICENSE-2.0
4118+#
4119+# Unless required by applicable law or agreed to in writing, software
4120+# distributed under the License is distributed on an "AS IS" BASIS,
4121+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
4122+# See the License for the specific language governing permissions and
4123+# limitations under the License.
4124
4125 import os
4126+from subprocess import STDOUT, check_output
4127 from charmhelpers.fetch import (
4128 BaseFetchHandler,
4129- UnhandledSource
4130+ UnhandledSource,
4131+ filter_installed_packages,
4132+ install,
4133 )
4134 from charmhelpers.core.host import mkdir
4135
4136-import six
4137-if six.PY3:
4138- raise ImportError('bzrlib does not support Python3')
4139
4140-try:
4141- from bzrlib.branch import Branch
4142- from bzrlib import bzrdir, workingtree, errors
4143-except ImportError:
4144- from charmhelpers.fetch import apt_install
4145- apt_install("python-bzrlib")
4146- from bzrlib.branch import Branch
4147- from bzrlib import bzrdir, workingtree, errors
4148+if filter_installed_packages(['bzr']) != []:
4149+ install(['bzr'])
4150+ if filter_installed_packages(['bzr']) != []:
4151+ raise NotImplementedError('Unable to install bzr')
4152
4153
4154 class BzrUrlFetchHandler(BaseFetchHandler):
4155- """Handler for bazaar branches via generic and lp URLs"""
4156+ """Handler for bazaar branches via generic and lp URLs."""
4157+
4158 def can_handle(self, source):
4159 url_parts = self.parse_url(source)
4160- if url_parts.scheme not in ('bzr+ssh', 'lp'):
4161+ if url_parts.scheme not in ('bzr+ssh', 'lp', ''):
4162 return False
4163+ elif not url_parts.scheme:
4164+ return os.path.exists(os.path.join(source, '.bzr'))
4165 else:
4166 return True
4167
4168- def branch(self, source, dest):
4169- url_parts = self.parse_url(source)
4170- # If we use lp:branchname scheme we need to load plugins
4171+ def branch(self, source, dest, revno=None):
4172 if not self.can_handle(source):
4173 raise UnhandledSource("Cannot handle {}".format(source))
4174- if url_parts.scheme == "lp":
4175- from bzrlib.plugin import load_plugins
4176- load_plugins()
4177- try:
4178- local_branch = bzrdir.BzrDir.create_branch_convenience(dest)
4179- except errors.AlreadyControlDirError:
4180- local_branch = Branch.open(dest)
4181- try:
4182- remote_branch = Branch.open(source)
4183- remote_branch.push(local_branch)
4184- tree = workingtree.WorkingTree.open(dest)
4185- tree.update()
4186- except Exception as e:
4187- raise e
4188+ cmd_opts = []
4189+ if revno:
4190+ cmd_opts += ['-r', str(revno)]
4191+ if os.path.exists(dest):
4192+ cmd = ['bzr', 'pull']
4193+ cmd += cmd_opts
4194+ cmd += ['--overwrite', '-d', dest, source]
4195+ else:
4196+ cmd = ['bzr', 'branch']
4197+ cmd += cmd_opts
4198+ cmd += [source, dest]
4199+ check_output(cmd, stderr=STDOUT)
4200
4201- def install(self, source, dest=None):
4202+ def install(self, source, dest=None, revno=None):
4203 url_parts = self.parse_url(source)
4204 branch_name = url_parts.path.strip("/").split("/")[-1]
4205 if dest:
4206@@ -73,10 +66,11 @@
4207 dest_dir = os.path.join(os.environ.get('CHARM_DIR'), "fetched",
4208 branch_name)
4209
4210- if not os.path.exists(dest_dir):
4211- mkdir(dest_dir, perms=0o755)
4212+ if dest and not os.path.exists(dest):
4213+ mkdir(dest, perms=0o755)
4214+
4215 try:
4216- self.branch(source, dest_dir)
4217+ self.branch(source, dest_dir, revno)
4218 except OSError as e:
4219 raise UnhandledSource(e.strerror)
4220 return dest_dir
4221
4222=== added file 'charmhelpers/fetch/centos.py'
4223--- charmhelpers/fetch/centos.py 1970-01-01 00:00:00 +0000
4224+++ charmhelpers/fetch/centos.py 2021-04-13 19:16:05 +0000
4225@@ -0,0 +1,171 @@
4226+# Copyright 2014-2015 Canonical Limited.
4227+#
4228+# Licensed under the Apache License, Version 2.0 (the "License");
4229+# you may not use this file except in compliance with the License.
4230+# You may obtain a copy of the License at
4231+#
4232+# http://www.apache.org/licenses/LICENSE-2.0
4233+#
4234+# Unless required by applicable law or agreed to in writing, software
4235+# distributed under the License is distributed on an "AS IS" BASIS,
4236+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
4237+# See the License for the specific language governing permissions and
4238+# limitations under the License.
4239+
4240+import subprocess
4241+import os
4242+import time
4243+import six
4244+import yum
4245+
4246+from tempfile import NamedTemporaryFile
4247+from charmhelpers.core.hookenv import log
4248+
4249+YUM_NO_LOCK = 1 # The return code for "couldn't acquire lock" in YUM.
4250+YUM_NO_LOCK_RETRY_DELAY = 10 # Wait 10 seconds between apt lock checks.
4251+YUM_NO_LOCK_RETRY_COUNT = 30 # Retry to acquire the lock X times.
4252+
4253+
4254+def filter_installed_packages(packages):
4255+ """Return a list of packages that require installation."""
4256+ yb = yum.YumBase()
4257+ package_list = yb.doPackageLists()
4258+ temp_cache = {p.base_package_name: 1 for p in package_list['installed']}
4259+
4260+ _pkgs = [p for p in packages if not temp_cache.get(p, False)]
4261+ return _pkgs
4262+
4263+
4264+def install(packages, options=None, fatal=False):
4265+ """Install one or more packages."""
4266+ cmd = ['yum', '--assumeyes']
4267+ if options is not None:
4268+ cmd.extend(options)
4269+ cmd.append('install')
4270+ if isinstance(packages, six.string_types):
4271+ cmd.append(packages)
4272+ else:
4273+ cmd.extend(packages)
4274+ log("Installing {} with options: {}".format(packages,
4275+ options))
4276+ _run_yum_command(cmd, fatal)
4277+
4278+
4279+def upgrade(options=None, fatal=False, dist=False):
4280+ """Upgrade all packages."""
4281+ cmd = ['yum', '--assumeyes']
4282+ if options is not None:
4283+ cmd.extend(options)
4284+ cmd.append('upgrade')
4285+ log("Upgrading with options: {}".format(options))
4286+ _run_yum_command(cmd, fatal)
4287+
4288+
4289+def update(fatal=False):
4290+ """Update local yum cache."""
4291+ cmd = ['yum', '--assumeyes', 'update']
4292+ log("Update with fatal: {}".format(fatal))
4293+ _run_yum_command(cmd, fatal)
4294+
4295+
4296+def purge(packages, fatal=False):
4297+ """Purge one or more packages."""
4298+ cmd = ['yum', '--assumeyes', 'remove']
4299+ if isinstance(packages, six.string_types):
4300+ cmd.append(packages)
4301+ else:
4302+ cmd.extend(packages)
4303+ log("Purging {}".format(packages))
4304+ _run_yum_command(cmd, fatal)
4305+
4306+
4307+def yum_search(packages):
4308+ """Search for a package."""
4309+ output = {}
4310+ cmd = ['yum', 'search']
4311+ if isinstance(packages, six.string_types):
4312+ cmd.append(packages)
4313+ else:
4314+ cmd.extend(packages)
4315+ log("Searching for {}".format(packages))
4316+ result = subprocess.check_output(cmd)
4317+ for package in list(packages):
4318+ output[package] = package in result
4319+ return output
4320+
4321+
4322+def add_source(source, key=None):
4323+ """Add a package source to this system.
4324+
4325+ @param source: a URL with a rpm package
4326+
4327+ @param key: A key to be added to the system's keyring and used
4328+ to verify the signatures on packages. Ideally, this should be an
4329+ ASCII format GPG public key including the block headers. A GPG key
4330+ id may also be used, but be aware that only insecure protocols are
4331+ available to retrieve the actual public key from a public keyserver
4332+ placing your Juju environment at risk.
4333+ """
4334+ if source is None:
4335+ log('Source is not present. Skipping')
4336+ return
4337+
4338+ if source.startswith('http'):
4339+ directory = '/etc/yum.repos.d/'
4340+ for filename in os.listdir(directory):
4341+ with open(directory + filename, 'r') as rpm_file:
4342+ if source in rpm_file.read():
4343+ break
4344+ else:
4345+ log("Add source: {!r}".format(source))
4346+ # write in the charms.repo
4347+ with open(directory + 'Charms.repo', 'a') as rpm_file:
4348+ rpm_file.write('[%s]\n' % source[7:].replace('/', '_'))
4349+ rpm_file.write('name=%s\n' % source[7:])
4350+ rpm_file.write('baseurl=%s\n\n' % source)
4351+ else:
4352+ log("Unknown source: {!r}".format(source))
4353+
4354+ if key:
4355+ if '-----BEGIN PGP PUBLIC KEY BLOCK-----' in key:
4356+ with NamedTemporaryFile('w+') as key_file:
4357+ key_file.write(key)
4358+ key_file.flush()
4359+ key_file.seek(0)
4360+ subprocess.check_call(['rpm', '--import', key_file.name])
4361+ else:
4362+ subprocess.check_call(['rpm', '--import', key])
4363+
4364+
4365+def _run_yum_command(cmd, fatal=False):
4366+ """Run an YUM command.
4367+
4368+ Checks the output and retry if the fatal flag is set to True.
4369+
4370+ :param: cmd: str: The yum command to run.
4371+ :param: fatal: bool: Whether the command's output should be checked and
4372+ retried.
4373+ """
4374+ env = os.environ.copy()
4375+
4376+ if fatal:
4377+ retry_count = 0
4378+ result = None
4379+
4380+ # If the command is considered "fatal", we need to retry if the yum
4381+ # lock was not acquired.
4382+
4383+ while result is None or result == YUM_NO_LOCK:
4384+ try:
4385+ result = subprocess.check_call(cmd, env=env)
4386+ except subprocess.CalledProcessError as e:
4387+ retry_count = retry_count + 1
4388+ if retry_count > YUM_NO_LOCK_RETRY_COUNT:
4389+ raise
4390+ result = e.returncode
4391+ log("Couldn't acquire YUM lock. Will retry in {} seconds."
4392+ "".format(YUM_NO_LOCK_RETRY_DELAY))
4393+ time.sleep(YUM_NO_LOCK_RETRY_DELAY)
4394+
4395+ else:
4396+ subprocess.call(cmd, env=env)
4397
4398=== modified file 'charmhelpers/fetch/giturl.py'
4399--- charmhelpers/fetch/giturl.py 2015-12-11 15:23:38 +0000
4400+++ charmhelpers/fetch/giturl.py 2021-04-13 19:16:05 +0000
4401@@ -1,54 +1,56 @@
4402 # Copyright 2014-2015 Canonical Limited.
4403 #
4404-# This file is part of charm-helpers.
4405-#
4406-# charm-helpers is free software: you can redistribute it and/or modify
4407-# it under the terms of the GNU Lesser General Public License version 3 as
4408-# published by the Free Software Foundation.
4409-#
4410-# charm-helpers is distributed in the hope that it will be useful,
4411-# but WITHOUT ANY WARRANTY; without even the implied warranty of
4412-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
4413-# GNU Lesser General Public License for more details.
4414-#
4415-# You should have received a copy of the GNU Lesser General Public License
4416-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
4417+# Licensed under the Apache License, Version 2.0 (the "License");
4418+# you may not use this file except in compliance with the License.
4419+# You may obtain a copy of the License at
4420+#
4421+# http://www.apache.org/licenses/LICENSE-2.0
4422+#
4423+# Unless required by applicable law or agreed to in writing, software
4424+# distributed under the License is distributed on an "AS IS" BASIS,
4425+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
4426+# See the License for the specific language governing permissions and
4427+# limitations under the License.
4428
4429 import os
4430+from subprocess import check_output, CalledProcessError, STDOUT
4431 from charmhelpers.fetch import (
4432 BaseFetchHandler,
4433- UnhandledSource
4434+ UnhandledSource,
4435+ filter_installed_packages,
4436+ install,
4437 )
4438-from charmhelpers.core.host import mkdir
4439-
4440-try:
4441- from git import Repo
4442-except ImportError:
4443- from charmhelpers.fetch import apt_install
4444- apt_install("python-git")
4445- from git import Repo
4446-
4447-from git.exc import GitCommandError # noqa E402
4448+
4449+if filter_installed_packages(['git']) != []:
4450+ install(['git'])
4451+ if filter_installed_packages(['git']) != []:
4452+ raise NotImplementedError('Unable to install git')
4453
4454
4455 class GitUrlFetchHandler(BaseFetchHandler):
4456- """Handler for git branches via generic and github URLs"""
4457+ """Handler for git branches via generic and github URLs."""
4458+
4459 def can_handle(self, source):
4460 url_parts = self.parse_url(source)
4461 # TODO (mattyw) no support for ssh git@ yet
4462- if url_parts.scheme not in ('http', 'https', 'git'):
4463+ if url_parts.scheme not in ('http', 'https', 'git', ''):
4464 return False
4465+ elif not url_parts.scheme:
4466+ return os.path.exists(os.path.join(source, '.git'))
4467 else:
4468 return True
4469
4470- def clone(self, source, dest, branch, depth=None):
4471+ def clone(self, source, dest, branch="master", depth=None):
4472 if not self.can_handle(source):
4473 raise UnhandledSource("Cannot handle {}".format(source))
4474
4475- if depth:
4476- Repo.clone_from(source, dest, branch=branch, depth=depth)
4477+ if os.path.exists(dest):
4478+ cmd = ['git', '-C', dest, 'pull', source, branch]
4479 else:
4480- Repo.clone_from(source, dest, branch=branch)
4481+ cmd = ['git', 'clone', source, dest, '--branch', branch]
4482+ if depth:
4483+ cmd.extend(['--depth', depth])
4484+ check_output(cmd, stderr=STDOUT)
4485
4486 def install(self, source, branch="master", dest=None, depth=None):
4487 url_parts = self.parse_url(source)
4488@@ -58,11 +60,9 @@
4489 else:
4490 dest_dir = os.path.join(os.environ.get('CHARM_DIR'), "fetched",
4491 branch_name)
4492- if not os.path.exists(dest_dir):
4493- mkdir(dest_dir, perms=0o755)
4494 try:
4495 self.clone(source, dest_dir, branch, depth)
4496- except GitCommandError as e:
4497+ except CalledProcessError as e:
4498 raise UnhandledSource(e)
4499 except OSError as e:
4500 raise UnhandledSource(e.strerror)
4501
4502=== added directory 'charmhelpers/fetch/python'
4503=== added file 'charmhelpers/fetch/python/__init__.py'
4504--- charmhelpers/fetch/python/__init__.py 1970-01-01 00:00:00 +0000
4505+++ charmhelpers/fetch/python/__init__.py 2021-04-13 19:16:05 +0000
4506@@ -0,0 +1,13 @@
4507+# Copyright 2014-2019 Canonical Limited.
4508+#
4509+# Licensed under the Apache License, Version 2.0 (the "License");
4510+# you may not use this file except in compliance with the License.
4511+# You may obtain a copy of the License at
4512+#
4513+# http://www.apache.org/licenses/LICENSE-2.0
4514+#
4515+# Unless required by applicable law or agreed to in writing, software
4516+# distributed under the License is distributed on an "AS IS" BASIS,
4517+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
4518+# See the License for the specific language governing permissions and
4519+# limitations under the License.
4520
4521=== added file 'charmhelpers/fetch/python/debug.py'
4522--- charmhelpers/fetch/python/debug.py 1970-01-01 00:00:00 +0000
4523+++ charmhelpers/fetch/python/debug.py 2021-04-13 19:16:05 +0000
4524@@ -0,0 +1,54 @@
4525+#!/usr/bin/env python
4526+# coding: utf-8
4527+
4528+# Copyright 2014-2015 Canonical Limited.
4529+#
4530+# Licensed under the Apache License, Version 2.0 (the "License");
4531+# you may not use this file except in compliance with the License.
4532+# You may obtain a copy of the License at
4533+#
4534+# http://www.apache.org/licenses/LICENSE-2.0
4535+#
4536+# Unless required by applicable law or agreed to in writing, software
4537+# distributed under the License is distributed on an "AS IS" BASIS,
4538+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
4539+# See the License for the specific language governing permissions and
4540+# limitations under the License.
4541+
4542+from __future__ import print_function
4543+
4544+import atexit
4545+import sys
4546+
4547+from charmhelpers.fetch.python.rpdb import Rpdb
4548+from charmhelpers.core.hookenv import (
4549+ open_port,
4550+ close_port,
4551+ ERROR,
4552+ log
4553+)
4554+
4555+__author__ = "Jorge Niedbalski <jorge.niedbalski@canonical.com>"
4556+
4557+DEFAULT_ADDR = "0.0.0.0"
4558+DEFAULT_PORT = 4444
4559+
4560+
4561+def _error(message):
4562+ log(message, level=ERROR)
4563+
4564+
4565+def set_trace(addr=DEFAULT_ADDR, port=DEFAULT_PORT):
4566+ """
4567+ Set a trace point using the remote debugger
4568+ """
4569+ atexit.register(close_port, port)
4570+ try:
4571+ log("Starting a remote python debugger session on %s:%s" % (addr,
4572+ port))
4573+ open_port(port)
4574+ debugger = Rpdb(addr=addr, port=port)
4575+ debugger.set_trace(sys._getframe().f_back)
4576+ except Exception:
4577+ _error("Cannot start a remote debug session on %s:%s" % (addr,
4578+ port))
4579
4580=== added file 'charmhelpers/fetch/python/packages.py'
4581--- charmhelpers/fetch/python/packages.py 1970-01-01 00:00:00 +0000
4582+++ charmhelpers/fetch/python/packages.py 2021-04-13 19:16:05 +0000
4583@@ -0,0 +1,154 @@
4584+#!/usr/bin/env python
4585+# coding: utf-8
4586+
4587+# Copyright 2014-2015 Canonical Limited.
4588+#
4589+# Licensed under the Apache License, Version 2.0 (the "License");
4590+# you may not use this file except in compliance with the License.
4591+# You may obtain a copy of the License at
4592+#
4593+# http://www.apache.org/licenses/LICENSE-2.0
4594+#
4595+# Unless required by applicable law or agreed to in writing, software
4596+# distributed under the License is distributed on an "AS IS" BASIS,
4597+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
4598+# See the License for the specific language governing permissions and
4599+# limitations under the License.
4600+
4601+import os
4602+import six
4603+import subprocess
4604+import sys
4605+
4606+from charmhelpers.fetch import apt_install, apt_update
4607+from charmhelpers.core.hookenv import charm_dir, log
4608+
4609+__author__ = "Jorge Niedbalski <jorge.niedbalski@canonical.com>"
4610+
4611+
4612+def pip_execute(*args, **kwargs):
4613+ """Overriden pip_execute() to stop sys.path being changed.
4614+
4615+ The act of importing main from the pip module seems to cause add wheels
4616+ from the /usr/share/python-wheels which are installed by various tools.
4617+ This function ensures that sys.path remains the same after the call is
4618+ executed.
4619+ """
4620+ try:
4621+ _path = sys.path
4622+ try:
4623+ from pip import main as _pip_execute
4624+ except ImportError:
4625+ apt_update()
4626+ if six.PY2:
4627+ apt_install('python-pip')
4628+ else:
4629+ apt_install('python3-pip')
4630+ from pip import main as _pip_execute
4631+ _pip_execute(*args, **kwargs)
4632+ finally:
4633+ sys.path = _path
4634+
4635+
4636+def parse_options(given, available):
4637+ """Given a set of options, check if available"""
4638+ for key, value in sorted(given.items()):
4639+ if not value:
4640+ continue
4641+ if key in available:
4642+ yield "--{0}={1}".format(key, value)
4643+
4644+
4645+def pip_install_requirements(requirements, constraints=None, **options):
4646+ """Install a requirements file.
4647+
4648+ :param constraints: Path to pip constraints file.
4649+ http://pip.readthedocs.org/en/stable/user_guide/#constraints-files
4650+ """
4651+ command = ["install"]
4652+
4653+ available_options = ('proxy', 'src', 'log', )
4654+ for option in parse_options(options, available_options):
4655+ command.append(option)
4656+
4657+ command.append("-r {0}".format(requirements))
4658+ if constraints:
4659+ command.append("-c {0}".format(constraints))
4660+ log("Installing from file: {} with constraints {} "
4661+ "and options: {}".format(requirements, constraints, command))
4662+ else:
4663+ log("Installing from file: {} with options: {}".format(requirements,
4664+ command))
4665+ pip_execute(command)
4666+
4667+
4668+def pip_install(package, fatal=False, upgrade=False, venv=None,
4669+ constraints=None, **options):
4670+ """Install a python package"""
4671+ if venv:
4672+ venv_python = os.path.join(venv, 'bin/pip')
4673+ command = [venv_python, "install"]
4674+ else:
4675+ command = ["install"]
4676+
4677+ available_options = ('proxy', 'src', 'log', 'index-url', )
4678+ for option in parse_options(options, available_options):
4679+ command.append(option)
4680+
4681+ if upgrade:
4682+ command.append('--upgrade')
4683+
4684+ if constraints:
4685+ command.extend(['-c', constraints])
4686+
4687+ if isinstance(package, list):
4688+ command.extend(package)
4689+ else:
4690+ command.append(package)
4691+
4692+ log("Installing {} package with options: {}".format(package,
4693+ command))
4694+ if venv:
4695+ subprocess.check_call(command)
4696+ else:
4697+ pip_execute(command)
4698+
4699+
4700+def pip_uninstall(package, **options):
4701+ """Uninstall a python package"""
4702+ command = ["uninstall", "-q", "-y"]
4703+
4704+ available_options = ('proxy', 'log', )
4705+ for option in parse_options(options, available_options):
4706+ command.append(option)
4707+
4708+ if isinstance(package, list):
4709+ command.extend(package)
4710+ else:
4711+ command.append(package)
4712+
4713+ log("Uninstalling {} package with options: {}".format(package,
4714+ command))
4715+ pip_execute(command)
4716+
4717+
4718+def pip_list():
4719+ """Returns the list of current python installed packages
4720+ """
4721+ return pip_execute(["list"])
4722+
4723+
4724+def pip_create_virtualenv(path=None):
4725+ """Create an isolated Python environment."""
4726+ if six.PY2:
4727+ apt_install('python-virtualenv')
4728+ else:
4729+ apt_install('python3-virtualenv')
4730+
4731+ if path:
4732+ venv_path = path
4733+ else:
4734+ venv_path = os.path.join(charm_dir(), 'venv')
4735+
4736+ if not os.path.exists(venv_path):
4737+ subprocess.check_call(['virtualenv', venv_path])
4738
4739=== added file 'charmhelpers/fetch/python/rpdb.py'
4740--- charmhelpers/fetch/python/rpdb.py 1970-01-01 00:00:00 +0000
4741+++ charmhelpers/fetch/python/rpdb.py 2021-04-13 19:16:05 +0000
4742@@ -0,0 +1,56 @@
4743+# Copyright 2014-2015 Canonical Limited.
4744+#
4745+# Licensed under the Apache License, Version 2.0 (the "License");
4746+# you may not use this file except in compliance with the License.
4747+# You may obtain a copy of the License at
4748+#
4749+# http://www.apache.org/licenses/LICENSE-2.0
4750+#
4751+# Unless required by applicable law or agreed to in writing, software
4752+# distributed under the License is distributed on an "AS IS" BASIS,
4753+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
4754+# See the License for the specific language governing permissions and
4755+# limitations under the License.
4756+
4757+"""Remote Python Debugger (pdb wrapper)."""
4758+
4759+import pdb
4760+import socket
4761+import sys
4762+
4763+__author__ = "Bertrand Janin <b@janin.com>"
4764+__version__ = "0.1.3"
4765+
4766+
4767+class Rpdb(pdb.Pdb):
4768+
4769+ def __init__(self, addr="127.0.0.1", port=4444):
4770+ """Initialize the socket and initialize pdb."""
4771+
4772+ # Backup stdin and stdout before replacing them by the socket handle
4773+ self.old_stdout = sys.stdout
4774+ self.old_stdin = sys.stdin
4775+
4776+ # Open a 'reusable' socket to let the webapp reload on the same port
4777+ self.skt = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
4778+ self.skt.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, True)
4779+ self.skt.bind((addr, port))
4780+ self.skt.listen(1)
4781+ (clientsocket, address) = self.skt.accept()
4782+ handle = clientsocket.makefile('rw')
4783+ pdb.Pdb.__init__(self, completekey='tab', stdin=handle, stdout=handle)
4784+ sys.stdout = sys.stdin = handle
4785+
4786+ def shutdown(self):
4787+ """Revert stdin and stdout, close the socket."""
4788+ sys.stdout = self.old_stdout
4789+ sys.stdin = self.old_stdin
4790+ self.skt.close()
4791+ self.set_continue()
4792+
4793+ def do_continue(self, arg):
4794+ """Stop all operation on ``continue``."""
4795+ self.shutdown()
4796+ return 1
4797+
4798+ do_EOF = do_quit = do_exit = do_c = do_cont = do_continue
4799
4800=== added file 'charmhelpers/fetch/python/version.py'
4801--- charmhelpers/fetch/python/version.py 1970-01-01 00:00:00 +0000
4802+++ charmhelpers/fetch/python/version.py 2021-04-13 19:16:05 +0000
4803@@ -0,0 +1,32 @@
4804+#!/usr/bin/env python
4805+# coding: utf-8
4806+
4807+# Copyright 2014-2015 Canonical Limited.
4808+#
4809+# Licensed under the Apache License, Version 2.0 (the "License");
4810+# you may not use this file except in compliance with the License.
4811+# You may obtain a copy of the License at
4812+#
4813+# http://www.apache.org/licenses/LICENSE-2.0
4814+#
4815+# Unless required by applicable law or agreed to in writing, software
4816+# distributed under the License is distributed on an "AS IS" BASIS,
4817+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
4818+# See the License for the specific language governing permissions and
4819+# limitations under the License.
4820+
4821+import sys
4822+
4823+__author__ = "Jorge Niedbalski <jorge.niedbalski@canonical.com>"
4824+
4825+
4826+def current_version():
4827+ """Current system python version"""
4828+ return sys.version_info
4829+
4830+
4831+def current_version_string():
4832+ """Current system python version as string major.minor.micro"""
4833+ return "{0}.{1}.{2}".format(sys.version_info.major,
4834+ sys.version_info.minor,
4835+ sys.version_info.micro)
4836
4837=== added file 'charmhelpers/fetch/snap.py'
4838--- charmhelpers/fetch/snap.py 1970-01-01 00:00:00 +0000
4839+++ charmhelpers/fetch/snap.py 2021-04-13 19:16:05 +0000
4840@@ -0,0 +1,150 @@
4841+# Copyright 2014-2017 Canonical Limited.
4842+#
4843+# Licensed under the Apache License, Version 2.0 (the "License");
4844+# you may not use this file except in compliance with the License.
4845+# You may obtain a copy of the License at
4846+#
4847+# http://www.apache.org/licenses/LICENSE-2.0
4848+#
4849+# Unless required by applicable law or agreed to in writing, software
4850+# distributed under the License is distributed on an "AS IS" BASIS,
4851+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
4852+# See the License for the specific language governing permissions and
4853+# limitations under the License.
4854+"""
4855+Charm helpers snap for classic charms.
4856+
4857+If writing reactive charms, use the snap layer:
4858+https://lists.ubuntu.com/archives/snapcraft/2016-September/001114.html
4859+"""
4860+import subprocess
4861+import os
4862+from time import sleep
4863+from charmhelpers.core.hookenv import log
4864+
4865+__author__ = 'Joseph Borg <joseph.borg@canonical.com>'
4866+
4867+# The return code for "couldn't acquire lock" in Snap
4868+# (hopefully this will be improved).
4869+SNAP_NO_LOCK = 1
4870+SNAP_NO_LOCK_RETRY_DELAY = 10 # Wait X seconds between Snap lock checks.
4871+SNAP_NO_LOCK_RETRY_COUNT = 30 # Retry to acquire the lock X times.
4872+SNAP_CHANNELS = [
4873+ 'edge',
4874+ 'beta',
4875+ 'candidate',
4876+ 'stable',
4877+]
4878+
4879+
4880+class CouldNotAcquireLockException(Exception):
4881+ pass
4882+
4883+
4884+class InvalidSnapChannel(Exception):
4885+ pass
4886+
4887+
4888+def _snap_exec(commands):
4889+ """
4890+ Execute snap commands.
4891+
4892+ :param commands: List commands
4893+ :return: Integer exit code
4894+ """
4895+ assert type(commands) == list
4896+
4897+ retry_count = 0
4898+ return_code = None
4899+
4900+ while return_code is None or return_code == SNAP_NO_LOCK:
4901+ try:
4902+ return_code = subprocess.check_call(['snap'] + commands,
4903+ env=os.environ)
4904+ except subprocess.CalledProcessError as e:
4905+ retry_count += + 1
4906+ if retry_count > SNAP_NO_LOCK_RETRY_COUNT:
4907+ raise CouldNotAcquireLockException(
4908+ 'Could not aquire lock after {} attempts'
4909+ .format(SNAP_NO_LOCK_RETRY_COUNT))
4910+ return_code = e.returncode
4911+ log('Snap failed to acquire lock, trying again in {} seconds.'
4912+ .format(SNAP_NO_LOCK_RETRY_DELAY, level='WARN'))
4913+ sleep(SNAP_NO_LOCK_RETRY_DELAY)
4914+
4915+ return return_code
4916+
4917+
4918+def snap_install(packages, *flags):
4919+ """
4920+ Install a snap package.
4921+
4922+ :param packages: String or List String package name
4923+ :param flags: List String flags to pass to install command
4924+ :return: Integer return code from snap
4925+ """
4926+ if type(packages) is not list:
4927+ packages = [packages]
4928+
4929+ flags = list(flags)
4930+
4931+ message = 'Installing snap(s) "%s"' % ', '.join(packages)
4932+ if flags:
4933+ message += ' with option(s) "%s"' % ', '.join(flags)
4934+
4935+ log(message, level='INFO')
4936+ return _snap_exec(['install'] + flags + packages)
4937+
4938+
4939+def snap_remove(packages, *flags):
4940+ """
4941+ Remove a snap package.
4942+
4943+ :param packages: String or List String package name
4944+ :param flags: List String flags to pass to remove command
4945+ :return: Integer return code from snap
4946+ """
4947+ if type(packages) is not list:
4948+ packages = [packages]
4949+
4950+ flags = list(flags)
4951+
4952+ message = 'Removing snap(s) "%s"' % ', '.join(packages)
4953+ if flags:
4954+ message += ' with options "%s"' % ', '.join(flags)
4955+
4956+ log(message, level='INFO')
4957+ return _snap_exec(['remove'] + flags + packages)
4958+
4959+
4960+def snap_refresh(packages, *flags):
4961+ """
4962+ Refresh / Update snap package.
4963+
4964+ :param packages: String or List String package name
4965+ :param flags: List String flags to pass to refresh command
4966+ :return: Integer return code from snap
4967+ """
4968+ if type(packages) is not list:
4969+ packages = [packages]
4970+
4971+ flags = list(flags)
4972+
4973+ message = 'Refreshing snap(s) "%s"' % ', '.join(packages)
4974+ if flags:
4975+ message += ' with options "%s"' % ', '.join(flags)
4976+
4977+ log(message, level='INFO')
4978+ return _snap_exec(['refresh'] + flags + packages)
4979+
4980+
4981+def valid_snap_channel(channel):
4982+ """ Validate snap channel exists
4983+
4984+ :raises InvalidSnapChannel: When channel does not exist
4985+ :return: Boolean
4986+ """
4987+ if channel.lower() in SNAP_CHANNELS:
4988+ return True
4989+ else:
4990+ raise InvalidSnapChannel("Invalid Snap Channel: {}".format(channel))
4991
4992=== added file 'charmhelpers/fetch/ubuntu.py'
4993--- charmhelpers/fetch/ubuntu.py 1970-01-01 00:00:00 +0000
4994+++ charmhelpers/fetch/ubuntu.py 2021-04-13 19:16:05 +0000
4995@@ -0,0 +1,730 @@
4996+# Copyright 2014-2015 Canonical Limited.
4997+#
4998+# Licensed under the Apache License, Version 2.0 (the "License");
4999+# you may not use this file except in compliance with the License.
5000+# You may obtain a copy of the License at
The diff has been truncated for viewing.

Subscribers

People subscribed via source and target branches