Merge lp:~landscape/landscape-charm/trunk into lp:~landscape/landscape-charm/stable
- trunk
- Merge into stable
Status: | Superseded |
---|---|
Proposed branch: | lp:~landscape/landscape-charm/trunk |
Merge into: | lp:~landscape/landscape-charm/stable |
Diff against target: |
9376 lines (+5057/-1599) (has conflicts) 84 files modified
.bzrignore (+1/-2) HACKING.md (+44/-13) Makefile (+30/-21) README.md (+46/-28) actions.yaml (+5/-5) charm-helpers.yaml (+1/-0) charmhelpers/__init__.py (+76/-17) charmhelpers/contrib/__init__.py (+11/-13) charmhelpers/contrib/hahelpers/__init__.py (+11/-13) charmhelpers/contrib/hahelpers/apache.py (+24/-20) charmhelpers/contrib/hahelpers/cluster.py (+113/-23) charmhelpers/core/__init__.py (+11/-13) charmhelpers/core/decorators.py (+11/-13) charmhelpers/core/files.py (+11/-13) charmhelpers/core/fstab.py (+11/-13) charmhelpers/core/hookenv.py (+548/-36) charmhelpers/core/host.py (+602/-166) charmhelpers/core/host_factory/centos.py (+72/-0) charmhelpers/core/host_factory/ubuntu.py (+114/-0) charmhelpers/core/hugepage.py (+11/-13) charmhelpers/core/kernel.py (+34/-30) charmhelpers/core/kernel_factory/centos.py (+17/-0) charmhelpers/core/kernel_factory/ubuntu.py (+13/-0) charmhelpers/core/services/__init__.py (+11/-13) charmhelpers/core/services/base.py (+29/-20) charmhelpers/core/services/helpers.py (+11/-13) charmhelpers/core/strutils.py (+75/-18) charmhelpers/core/sysctl.py (+32/-23) charmhelpers/core/templating.py (+37/-25) charmhelpers/core/unitdata.py (+19/-15) charmhelpers/fetch/__init__.py (+53/-302) charmhelpers/fetch/archiveurl.py (+12/-14) charmhelpers/fetch/bzrurl.py (+42/-48) charmhelpers/fetch/centos.py (+171/-0) charmhelpers/fetch/giturl.py (+33/-33) charmhelpers/fetch/python/__init__.py (+13/-0) charmhelpers/fetch/python/debug.py (+54/-0) charmhelpers/fetch/python/packages.py (+154/-0) charmhelpers/fetch/python/rpdb.py (+56/-0) charmhelpers/fetch/python/version.py (+32/-0) charmhelpers/fetch/snap.py (+150/-0) charmhelpers/fetch/ubuntu.py (+730/-0) charmhelpers/osplatform.py (+25/-0) config.yaml (+34/-0) dev/charm_helpers_sync.py (+32/-24) dev/deployer (+29/-30) dev/ubuntu-deps (+13/-10) hooks/install (+1/-1) icon.svg (+1/-288) lib/action.py (+1/-1) lib/apt.py (+60/-26) lib/bootstrap.py (+21/-15) lib/callbacks/scripts.py (+48/-2) lib/callbacks/tests/test_apt.py (+1/-1) lib/callbacks/tests/test_scripts.py (+43/-2) lib/paths.py (+1/-0) lib/relations/haproxy.py (+57/-6) lib/relations/hosted.py (+61/-2) lib/relations/postgresql.py (+30/-35) lib/relations/tests/test_haproxy.py (+175/-14) lib/relations/tests/test_hosted.py (+148/-8) lib/relations/tests/test_postgresql.py (+18/-17) lib/services.py (+7/-3) lib/tests/rootdir.py (+1/-1) lib/tests/sample.py (+4/-3) lib/tests/stubs.py (+19/-2) lib/tests/test_apt.py (+69/-28) lib/tests/test_bootstrap.py (+46/-13) lib/tests/test_install.py (+6/-5) lib/tests/test_services.py (+16/-4) lib/tests/test_templates.py (+95/-0) lib/tests/test_upgrade.py (+67/-4) lib/tests/test_utils.py (+150/-1) lib/utils.py (+71/-0) metadata.yaml (+4/-0) templates/landscape-server (+4/-0) templates/service.conf (+21/-0) tests/basic/test_actions.py (+1/-1) tests/basic/test_ha.py (+21/-21) tests/basic/test_leader.py (+2/-4) tests/basic/test_service.py (+20/-3) tests/helpers.py (+90/-38) tests/layers.py (+11/-5) tests/test_helpers.py (+2/-5) Text conflict in Makefile Text conflict in README.md Text conflict in config.yaml Text conflict in tests/basic/test_service.py |
To merge this branch: | bzr merge lp:~landscape/landscape-charm/trunk |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Simon Poirier (community) | Needs Resubmitting | ||
Review via email: mp+401061@code.launchpad.net |
This proposal has been superseded by a proposal from 2021-04-14.
Commit message
add oidc-* options
Description of the change
Added changes to support OpenID-Connect options.
- 403. By Simon Poirier
-
Merge landscape-charm [f=1923863] [r=simpoir] [a=mastier1]
add oidc-* options - 404. By Simon Poirier
-
Merge 1501803_ha_amqp [f=1501803] [r=maxiberta] [a=Simon Poirier]
Add all units to broker section with HA rabbitmq. - 405. By Simon Poirier
-
Merge juju-deploy-err [f=] [r=maxiberta] [a=Simon Poirier]
Replace juju-deployer by plain "juju deploy" in integration tests to allow testing on bionic. - 406. By Linda Guo <email address hidden>
-
Merge registration-
relation [f=1945732] [r=landscape- builder, simpoir] [a=Linda Guo]
Add application-dashboard relation By adding application-
dashboard relation in landscape charm,
we can register landscape in Homer dashboard after
creating relation between landscape charm and Homer charm - 407. By Linda Guo <email address hidden>
-
Merge bug-1934816 [f=1934816] [r=landscape-
builder, simpoir, elmo] [a=Linda Guo]
Sync charm-helpers
Add relation: nrpe-external-master
Add nrpe checks check_systemd for landscape services - 408. By Simon Poirier
-
Merge 1682105_
revisit_ config [f=1499686,1682105] [r=landscape- builder, silverdrake11] [a=Simon Poirier]
Make proxy, system-email and root-url configurations updatable through the charm config.
Unmerged revisions
- 408. By Simon Poirier
-
Merge 1682105_
revisit_ config [f=1499686,1682105] [r=landscape- builder, silverdrake11] [a=Simon Poirier]
Make proxy, system-email and root-url configurations updatable through the charm config. - 407. By Linda Guo <email address hidden>
-
Merge bug-1934816 [f=1934816] [r=landscape-
builder, simpoir, elmo] [a=Linda Guo]
Sync charm-helpers
Add relation: nrpe-external-master
Add nrpe checks check_systemd for landscape services - 406. By Linda Guo <email address hidden>
-
Merge registration-
relation [f=1945732] [r=landscape- builder, simpoir] [a=Linda Guo]
Add application-dashboard relation By adding application-
dashboard relation in landscape charm,
we can register landscape in Homer dashboard after
creating relation between landscape charm and Homer charm - 405. By Simon Poirier
-
Merge juju-deploy-err [f=] [r=maxiberta] [a=Simon Poirier]
Replace juju-deployer by plain "juju deploy" in integration tests to allow testing on bionic. - 404. By Simon Poirier
-
Merge 1501803_ha_amqp [f=1501803] [r=maxiberta] [a=Simon Poirier]
Add all units to broker section with HA rabbitmq. - 403. By Simon Poirier
-
Merge landscape-charm [f=1923863] [r=simpoir] [a=mastier1]
add oidc-* options - 402. By Simon Poirier
-
Merge pass_ping_
through_ https [f=1878265] [r=roadmr, landscape- builder] [a=Simon Poirier]
Add ping service to the https frontend in haproxy. - 401. By Simon Poirier
-
Merge trivial_doc_updates [f=1846394,1864699] [r=landscape-
builder, cjohnston, roadmr] [a=Simon Poirier]
Trivial documentation updates on config and actions. - 400. By Simon Poirier
-
Update to 19.10 release PPA
- 399. By Guillermo Gonzalez
-
use postgresql-charm v2 protocol, manually parse the dsn as the available psycopg2 (2.6.x) doesn't provide parse_dsn (added in 2.7.x)
Preview Diff
1 | === modified file '.bzrignore' | |||
2 | --- .bzrignore 2015-07-09 09:00:17 +0000 | |||
3 | +++ .bzrignore 2021-04-13 19:16:05 +0000 | |||
4 | @@ -1,7 +1,6 @@ | |||
5 | 1 | _trial_temp | 1 | _trial_temp |
6 | 2 | bundles | 2 | bundles |
9 | 3 | config/license-file | 3 | config |
8 | 4 | config/repo-file | ||
10 | 5 | hooks/_trial_temp | 4 | hooks/_trial_temp |
11 | 6 | tags | 5 | tags |
12 | 7 | secrets | 6 | secrets |
13 | 8 | 7 | ||
14 | === modified file 'HACKING.md' | |||
15 | --- HACKING.md 2014-03-04 11:36:06 +0000 | |||
16 | +++ HACKING.md 2021-04-13 19:16:05 +0000 | |||
17 | @@ -9,20 +9,51 @@ | |||
18 | 9 | Integration Testing | 9 | Integration Testing |
19 | 10 | =================== | 10 | =================== |
20 | 11 | 11 | ||
22 | 12 | This charm comes with an integration test suite. You can run it as follows: | 12 | This charm comes with an integration test suite, which lives in the 'tests' |
23 | 13 | directory. You can run it as follows: | ||
24 | 13 | 14 | ||
25 | 14 | make integration-test | 15 | make integration-test |
26 | 15 | 16 | ||
37 | 16 | N.B., It will deploy into a real juju environment. It uses the juju-test | 17 | N.B., It will deploy into a the current juju model. It uses the bundletester |
38 | 17 | command to facilitate this, which takes care of bootstrapping for you. It | 18 | command to facilitate this. the `JUJU_MODEL` environment variable can be passed |
39 | 18 | will use whatever 'juju env' reports as your current environment. It will use | 19 | to specify a different model. It will use a number of machines to do this test |
40 | 19 | a number of machines to do this test -- it should work on a local environment | 20 | -- it should work on a local controller (LXD), but could be quite resource |
41 | 20 | (LXC), but could be quite resource intensive. | 21 | intensive. |
42 | 21 | 22 | ||
43 | 22 | Please also note that if you want to deploy the current charm branch you are | 23 | |
44 | 23 | working on, you need to change the branch URL in the landscape-deployments.yaml | 24 | Running parts of the integration tests |
45 | 24 | file to point to your branch and commit all changes. After you are done with | 25 | -------------------------------------- |
46 | 25 | testing, remember to return the branch URL to what it was before | 26 | |
47 | 27 | Running all the integration tests may take quite a while, and sometimes | ||
48 | 28 | you want to just run the ones that you are working on. To do that, you | ||
49 | 29 | can bootstrap the Juju environment yourself, and then use the zope | ||
50 | 30 | testrunner directly. For example: | ||
51 | 31 | |||
52 | 32 | zope-testrunner3 -vv --path tests --tests-pattern basic --test some_test | ||
53 | 33 | |||
54 | 34 | If the different services already are deployed, the command above is enough. | ||
55 | 35 | But if you run it against an empty environment, you have to remember to pass | ||
56 | 36 | along environment variables that affect the deployment, such as | ||
57 | 37 | LS_CHARM_SOURCE=lds-trunk-ppa as to generate the secrets and bundles: | ||
58 | 38 | |||
59 | 39 | make secrets bundles | ||
60 | 40 | |||
61 | 41 | |||
62 | 42 | Running integration tests on dense MAAS deployment | ||
63 | 43 | -------------------------------------------------- | ||
64 | 44 | |||
65 | 45 | It's possible to run the integration tests on a dense MAAS deployment, | ||
66 | 46 | where all the services run on the bootstrap node. But given that it's a | ||
67 | 47 | bit different from other deployments, you have to explicitly tell that | ||
68 | 48 | it's such a deployment using the DENSE_MAAS environment variable. For | ||
69 | 49 | example: | ||
70 | 50 | |||
71 | 51 | DENSE_MAAS=1 make integration-test-trunk | ||
72 | 52 | |||
73 | 53 | Or, if you already have the environment bootstrapped: | ||
74 | 54 | |||
75 | 55 | DENSE_MAAS=1 LS_CHARM_SOURCE=lds-trunk-ppa zope-testrunner3 -vv \ | ||
76 | 56 | --path tests --tests-pattern basic --test some_test | ||
77 | 26 | 57 | ||
78 | 27 | 58 | ||
79 | 28 | Testing with Dependent Upstream Charms | 59 | Testing with Dependent Upstream Charms |
80 | @@ -33,5 +64,5 @@ | |||
81 | 33 | 64 | ||
82 | 34 | make update-charm-revision-numbers | 65 | make update-charm-revision-numbers |
83 | 35 | 66 | ||
86 | 36 | After running this, you can use bzr diff to see what (if any) changes were | 67 | After running this, you can use 'bzr diff bundles' to see what (if any) |
87 | 37 | made to landscape-deployments.yaml. | 68 | changes were made to landscape-deployments.yaml. |
88 | 38 | 69 | ||
89 | === modified file 'Makefile' | |||
90 | --- Makefile 2016-02-10 22:05:36 +0000 | |||
91 | +++ Makefile 2021-04-13 19:16:05 +0000 | |||
92 | @@ -2,35 +2,45 @@ | |||
93 | 2 | 2 | ||
94 | 3 | test: | 3 | test: |
95 | 4 | trial lib | 4 | trial lib |
96 | 5 | # For now only the install hook runs against python3 | ||
97 | 6 | trial3 lib/tests/test_apt.py lib/tests/test_install.py | ||
98 | 5 | 7 | ||
99 | 6 | ci-test: | 8 | ci-test: |
100 | 7 | ./dev/ubuntu-deps | 9 | ./dev/ubuntu-deps |
101 | 8 | $(MAKE) test lint | 10 | $(MAKE) test lint |
102 | 9 | 11 | ||
103 | 10 | verify-juju-test: | ||
104 | 11 | @echo "Checking for ... " | ||
105 | 12 | @echo -n "juju-test: " | ||
106 | 13 | @if [ -z `which juju-test` ]; then \ | ||
107 | 14 | echo -e "\nRun ./dev/ubuntu-deps to get the juju-test command installed"; \ | ||
108 | 15 | exit 1;\ | ||
109 | 16 | else \ | ||
110 | 17 | echo "installed"; \ | ||
111 | 18 | fi | ||
112 | 19 | |||
113 | 20 | update-charm-revision-numbers: bundles | 12 | update-charm-revision-numbers: bundles |
114 | 21 | @dev/update-charm-revision-numbers \ | 13 | @dev/update-charm-revision-numbers \ |
115 | 22 | $(EXTRA_UPDATE_ARGUMENTS) \ | 14 | $(EXTRA_UPDATE_ARGUMENTS) \ |
116 | 23 | apache2 postgresql juju-gui haproxy rabbitmq-server nfs | 15 | apache2 postgresql juju-gui haproxy rabbitmq-server nfs |
117 | 24 | 16 | ||
120 | 25 | test-depends: verify-juju-test bundles | 17 | test-depends: bundles |
121 | 26 | @cd tests && python3 test_helpers.py | 18 | pip install --user bundletester juju-deployer |
122 | 19 | pip3 install --user amulet | ||
123 | 20 | cd tests && python3 test_helpers.py | ||
124 | 27 | 21 | ||
126 | 28 | bundles: | 22 | bundles-checkout: |
127 | 29 | @if [ -d bundles ]; then \ | 23 | @if [ -d bundles ]; then \ |
128 | 30 | bzr up bundles; \ | 24 | bzr up bundles; \ |
129 | 31 | else \ | 25 | else \ |
130 | 26 | <<<<<<< TREE | ||
131 | 32 | bzr co lp:~landscape/landscape-charm/bundles-stable bundles; \ | 27 | bzr co lp:~landscape/landscape-charm/bundles-stable bundles; \ |
132 | 33 | fi | 28 | fi |
133 | 29 | ======= | ||
134 | 30 | bzr co lp:landscape-bundles bundles; \ | ||
135 | 31 | fi; \ | ||
136 | 32 | make -C bundles deps | ||
137 | 33 | make -C bundles clean | ||
138 | 34 | |||
139 | 35 | bundles: bundles-checkout | ||
140 | 36 | bundles/render-bundles | ||
141 | 37 | |||
142 | 38 | bundles-local-branch: bundles-checkout | ||
143 | 39 | bundles/render-bundles --landscape-branch $(CURDIR) | ||
144 | 40 | |||
145 | 41 | bundles-local-charm: bundles-checkout | ||
146 | 42 | bundles/render-bundles --landscape-charm $(CURDIR) | ||
147 | 43 | >>>>>>> MERGE-SOURCE | ||
148 | 34 | 44 | ||
149 | 35 | secrets: | 45 | secrets: |
150 | 36 | @if [ -d secrets ]; then \ | 46 | @if [ -d secrets ]; then \ |
151 | @@ -40,29 +50,29 @@ | |||
152 | 40 | fi | 50 | fi |
153 | 41 | 51 | ||
154 | 42 | integration-test: test-depends | 52 | integration-test: test-depends |
156 | 43 | juju test --set-e -p LS_CHARM_SOURCE,JUJU_HOME,JUJU_ENV,PG_MANUAL_TUNING -v --timeout 3000s | 53 | python2 ~/.local/bin/bundletester -v -l DEBUG --skip-implicit -t . |
157 | 44 | 54 | ||
158 | 45 | # Run integration tests using the LDS package from the lds-trunk PPA | 55 | # Run integration tests using the LDS package from the lds-trunk PPA |
159 | 46 | integration-test-trunk: secrets | 56 | integration-test-trunk: secrets |
160 | 47 | LS_CHARM_SOURCE=lds-trunk-ppa $(MAKE) $(subst -trunk,,$@) | 57 | LS_CHARM_SOURCE=lds-trunk-ppa $(MAKE) $(subst -trunk,,$@) |
161 | 48 | 58 | ||
163 | 49 | deploy-dense-maas: bundles | 59 | deploy-dense-maas: bundles-local-branch config |
164 | 50 | ./dev/deployer dense-maas | 60 | ./dev/deployer dense-maas |
165 | 51 | 61 | ||
167 | 52 | deploy-dense-maas-dev: bundles | 62 | deploy-dense-maas-dev: bundles-local-branch config repo-file-trunk |
168 | 53 | ./dev/deployer dense-maas --flags juju-debug | 63 | ./dev/deployer dense-maas --flags juju-debug |
169 | 54 | 64 | ||
171 | 55 | deploy: bundles | 65 | deploy: bundles-local-branch |
172 | 56 | ./dev/deployer scalable | 66 | ./dev/deployer scalable |
173 | 57 | 67 | ||
175 | 58 | repo-file-trunk: secrets | 68 | repo-file-trunk: secrets config |
176 | 59 | grep -e "^source:" secrets/lds-trunk-ppa | cut -f 2- -d " " > config/repo-file | 69 | grep -e "^source:" secrets/lds-trunk-ppa | cut -f 2- -d " " > config/repo-file |
177 | 60 | 70 | ||
178 | 61 | lint: | 71 | lint: |
179 | 62 | flake8 --filename='*' hooks | 72 | flake8 --filename='*' hooks |
180 | 63 | flake8 lib tests | 73 | flake8 lib tests |
181 | 64 | pyflakes3 tests dev/update-charm-revision-numbers | 74 | pyflakes3 tests dev/update-charm-revision-numbers |
183 | 65 | find . -name *.py -not -path "./old/*" -not -path "*/charmhelpers/*" -print0 | xargs -0 flake8 | 75 | find . -name *.py -not -path "./old/*" -not -path "./build/*" -not -path "*/charmhelpers/*" -print0 | xargs -0 flake8 |
184 | 66 | flake8 tests dev/update-charm-revision-numbers | 76 | flake8 tests dev/update-charm-revision-numbers |
185 | 67 | 77 | ||
186 | 68 | clean: | 78 | clean: |
187 | @@ -83,8 +93,7 @@ | |||
188 | 83 | 93 | ||
189 | 84 | dev/charm_helpers_sync.py: | 94 | dev/charm_helpers_sync.py: |
190 | 85 | @mkdir -p dev | 95 | @mkdir -p dev |
193 | 86 | @bzr cat lp:charm-helpers/tools/charm_helpers_sync/charm_helpers_sync.py \ | 96 | @curl https://git.launchpad.net/charm-helpers/plain/tools/charm_helpers_sync/charm_helpers_sync.py > dev/charm_helpers_sync.py |
192 | 87 | > dev/charm_helpers_sync.py | ||
194 | 88 | 97 | ||
195 | 89 | sync: dev/charm_helpers_sync.py | 98 | sync: dev/charm_helpers_sync.py |
196 | 90 | $(PYTHON) dev/charm_helpers_sync.py -c charm-helpers.yaml | 99 | $(PYTHON) dev/charm_helpers_sync.py -c charm-helpers.yaml |
197 | 91 | 100 | ||
198 | === modified file 'README.md' | |||
199 | --- README.md 2015-10-14 14:56:41 +0000 | |||
200 | +++ README.md 2021-04-13 19:16:05 +0000 | |||
201 | @@ -1,43 +1,37 @@ | |||
202 | 1 | Overview | 1 | Overview |
203 | 2 | ======== | 2 | ======== |
213 | 3 | 3 | The Landscape systems management tool helps you monitor, manage and update your entire Ubuntu infrastructure from a single interface. Part of Canonical's Ubuntu Advantage support service, Landscape brings you intuitive systems management tools combined with world-class support. | |
214 | 4 | The Landscape systems management tool helps you monitor, manage and update your | 4 | |
215 | 5 | entire Ubuntu infrastructure from a single interface. Part of Canonical's | 5 | This charm will deploy Landscape On Premises, and needs to be connected to other charms to be fully functional. Example deployments are given below. |
207 | 6 | Ubuntu Advantage support service, Landscape brings you intuitive systems | ||
208 | 7 | management tools combined with world-class support. | ||
209 | 8 | |||
210 | 9 | This charm will deploy Landscape Dedicated Server (LDS), and needs to be | ||
211 | 10 | connected to other charms to be fully functional. Example deployments are given | ||
212 | 11 | below. | ||
216 | 12 | 6 | ||
217 | 13 | For more information about Landscape, go to http://www.ubuntu.com/management | 7 | For more information about Landscape, go to http://www.ubuntu.com/management |
218 | 14 | 8 | ||
219 | 15 | Standard Usage | 9 | Standard Usage |
220 | 16 | ============== | 10 | ============== |
221 | 17 | 11 | ||
224 | 18 | The typical deployment of Landscape happens using a Juju bundle. This charm is | 12 | The typical deployment of Landscape happens using a Juju bundle. This charm is not useful without a deployed bundle of services. |
223 | 19 | not useful without a deployed bundle of services. | ||
225 | 20 | 13 | ||
226 | 21 | Please use one of the following bundle types depending on your needs: | 14 | Please use one of the following bundle types depending on your needs: |
227 | 22 | 15 | ||
228 | 16 | <<<<<<< TREE | ||
229 | 23 | * [landscape-scalable](https://jujucharms.com/u/landscape/landscape-scalable/) | 17 | * [landscape-scalable](https://jujucharms.com/u/landscape/landscape-scalable/) |
230 | 24 | * [landscape-dense-maas](https://jujucharms.com/u/landscape/landscape-dense-maas/) | 18 | * [landscape-dense-maas](https://jujucharms.com/u/landscape/landscape-dense-maas/) |
231 | 25 | * [landscape-dense](https://jujucharms.com/u/landscape/landscape-dense/) | 19 | * [landscape-dense](https://jujucharms.com/u/landscape/landscape-dense/) |
232 | 20 | ======= | ||
233 | 21 | * https://jujucharms.com/landscape-scalable/ | ||
234 | 22 | * https://jujucharms.com/landscape-dense-maas/ | ||
235 | 23 | * https://jujucharms.com/landscape-dense/ | ||
236 | 24 | >>>>>>> MERGE-SOURCE | ||
237 | 26 | 25 | ||
238 | 27 | For the landscape-scalable case: | 26 | For the landscape-scalable case: |
239 | 28 | 27 | ||
246 | 29 | sudo apt-add-repository ppa:juju/stable | 28 | $ juju deploy landscape-scalable |
247 | 30 | sudo apt-get update | 29 | |
248 | 31 | juju quickstart u/landscape/landscape-scalable | 30 | |
249 | 32 | 31 | Customised Deployments | |
244 | 33 | |||
245 | 34 | Customized Deployments | ||
250 | 35 | ====================== | 32 | ====================== |
251 | 36 | 33 | ||
256 | 37 | The standard deployment of Landscape will give you the latest released code. | 34 | The standard deployment of Landscape will give you the latest released code. If you want a different version, different options, etc, you will need to download one of the bundles, and add/change options in the file before supplying it to juju. |
253 | 38 | If you want a different version, different options, etc, you will need to | ||
254 | 39 | download one of the bundles, and add/change options in the file before | ||
255 | 40 | supplying it to juju quickstart. | ||
257 | 41 | 35 | ||
258 | 42 | On the bundle page, download the `bundle.yaml` file. | 36 | On the bundle page, download the `bundle.yaml` file. |
259 | 43 | 37 | ||
260 | @@ -45,6 +39,7 @@ | |||
261 | 45 | Configuration | 39 | Configuration |
262 | 46 | ============= | 40 | ============= |
263 | 47 | 41 | ||
264 | 42 | <<<<<<< TREE | ||
265 | 48 | Landscape is a commercial product which is bundled with a license | 43 | Landscape is a commercial product which is bundled with a license |
266 | 49 | allowing management of up to 10 physical machines and 50 more LXC | 44 | allowing management of up to 10 physical machines and 50 more LXC |
267 | 50 | containers, for a total of 60 seats. | 45 | containers, for a total of 60 seats. |
268 | @@ -55,34 +50,44 @@ | |||
269 | 55 | gather these details after purchasing seats for LDS. All information | 50 | gather these details after purchasing seats for LDS. All information |
270 | 56 | is found by following a link on the left side of the page called | 51 | is found by following a link on the left side of the page called |
271 | 57 | "access the Landscape Dedicated Server archive" | 52 | "access the Landscape Dedicated Server archive" |
272 | 53 | ======= | ||
273 | 54 | Landscape is a commercial product and as such it needs configuration of a license and password protected repository before deployment. Please login to your "hosted account" (on landscape.canonical.com) to gather these details after purchasing seats for Landscape On Premises. All information is found by following a link on the left side of the page called "access the Landscape On Premises archive" | ||
274 | 55 | >>>>>>> MERGE-SOURCE | ||
275 | 58 | 56 | ||
276 | 59 | license-file | 57 | license-file |
277 | 60 | ------------ | 58 | ------------ |
278 | 61 | 59 | ||
279 | 60 | <<<<<<< TREE | ||
280 | 62 | You can set this as a Juju configuration option after deployment | 61 | You can set this as a Juju configuration option after deployment |
281 | 63 | on the landscape service like this: | 62 | on the landscape service like this: |
282 | 63 | ======= | ||
283 | 64 | You can set this as a juju configuration option after deployment on each deployed landscape-server application like: | ||
284 | 65 | >>>>>>> MERGE-SOURCE | ||
285 | 64 | 66 | ||
286 | 67 | <<<<<<< TREE | ||
287 | 65 | $ juju set landscape-server "license-file=$(cat license-file)" | 68 | $ juju set landscape-server "license-file=$(cat license-file)" |
288 | 69 | ======= | ||
289 | 70 | $ juju config landscape-server "license-file=$(cat license-file)" | ||
290 | 71 | >>>>>>> MERGE-SOURCE | ||
291 | 66 | 72 | ||
292 | 67 | 73 | ||
293 | 68 | SSL | 74 | SSL |
294 | 69 | === | 75 | === |
295 | 70 | 76 | ||
299 | 71 | The pre-packaged bundles will ask the HAProxy charm to generate a self | 77 | The pre-packaged bundles will ask the HAProxy charm to generate a self signed certificate. While useful for testing, this must not be used for production deployments. |
297 | 72 | signed certificate. While useful for testing, this must not be used for | ||
298 | 73 | production deployments. | ||
300 | 74 | 78 | ||
305 | 75 | For production deployments, you should include a "real" SSL certificate key | 79 | For production deployments, you should include a "real" SSL certificate key pair that has been signed by a CA that your clients trust in the haproxy service configuration (or in the landscape-server service configuration if you need to use your haproxy service for other services too with different certificates). |
302 | 76 | pair that has been signed by a CA that your clients trust in the haproxy service | ||
303 | 77 | configuration (or in the landscape-server service configuration if you need to | ||
304 | 78 | use your haproxy service for other services too with different certificates). | ||
306 | 79 | 80 | ||
307 | 80 | 81 | ||
308 | 81 | Unit Testing | 82 | Unit Testing |
309 | 82 | ============ | 83 | ============ |
310 | 83 | 84 | ||
311 | 85 | <<<<<<< TREE | ||
312 | 84 | The Landscape charm is unit tested and new code changes should be | 86 | The Landscape charm is unit tested and new code changes should be |
313 | 85 | submitted with unit tests. You can run them like this: | 87 | submitted with unit tests. You can run them like this: |
314 | 88 | ======= | ||
315 | 89 | The Landscape charm is fairly well unit tested and new code changes should be submitted with unit tests. You can run them like this: | ||
316 | 90 | >>>>>>> MERGE-SOURCE | ||
317 | 86 | 91 | ||
318 | 87 | $ make test | 92 | $ make test |
319 | 88 | 93 | ||
320 | @@ -90,6 +95,7 @@ | |||
321 | 90 | Integration Testing | 95 | Integration Testing |
322 | 91 | =================== | 96 | =================== |
323 | 92 | 97 | ||
324 | 98 | <<<<<<< TREE | ||
325 | 93 | This charm makes use of | 99 | This charm makes use of |
326 | 94 | [juju-deployer](http://pythonhosted.org/juju-deployer/) and | 100 | [juju-deployer](http://pythonhosted.org/juju-deployer/) and |
327 | 95 | [charm-tools](https://jujucharms.com/docs/1.20/tools-charm-tools) to | 101 | [charm-tools](https://jujucharms.com/docs/1.20/tools-charm-tools) to |
328 | @@ -105,3 +111,15 @@ | |||
329 | 105 | 111 | ||
330 | 106 | The JUJU_ENV environment variable can be omitted if you want to use the | 112 | The JUJU_ENV environment variable can be omitted if you want to use the |
331 | 107 | current juju environment (as set by "juju switch"). | 113 | current juju environment (as set by "juju switch"). |
332 | 114 | ======= | ||
333 | 115 | This charm makes use of juju-deployer and the charm-tools package to enable end-to-end integration testing. This is how you proceed with running them: | ||
334 | 116 | |||
335 | 117 | $ juju bootstrap localhost | ||
336 | 118 | $ make integration-test | ||
337 | 119 | |||
338 | 120 | Or if you want to use the packages from the lds-trunk PPA: | ||
339 | 121 | |||
340 | 122 | $ JUJU_MODEL=<model> make integration-test-trunk | ||
341 | 123 | |||
342 | 124 | The JUJU_MODEL environment variable can be omitted if you want to use the current model. | ||
343 | 125 | >>>>>>> MERGE-SOURCE | ||
344 | 108 | 126 | ||
345 | === modified file 'actions.yaml' | |||
346 | --- actions.yaml 2015-06-25 16:01:22 +0000 | |||
347 | +++ actions.yaml 2021-04-13 19:16:05 +0000 | |||
348 | @@ -1,10 +1,7 @@ | |||
349 | 1 | pause: | 1 | pause: |
353 | 2 | description: Pause the Landscape unit. This action will interrupt any | 2 | description: Pause the Landscape service. |
351 | 3 | Landscape-related processing on the unit and prevent any further processing | ||
352 | 4 | from happening. | ||
354 | 5 | resume: | 3 | resume: |
357 | 6 | description: Resume the Landscape unit. This action will start all the | 4 | description: Resume the Landscape service. |
356 | 7 | Landscape services on the unit. | ||
358 | 8 | upgrade: | 5 | upgrade: |
359 | 9 | description: Upgrade software on the Landscape unit. This action will update | 6 | description: Upgrade software on the Landscape unit. This action will update |
360 | 10 | APT package indexes and upgrade the landscape-server package. | 7 | APT package indexes and upgrade the landscape-server package. |
361 | @@ -25,5 +22,8 @@ | |||
362 | 25 | admin-password: | 22 | admin-password: |
363 | 26 | type: string | 23 | type: string |
364 | 27 | description: Password for the administrator to add. | 24 | description: Password for the administrator to add. |
365 | 25 | registration-key: | ||
366 | 26 | type: string | ||
367 | 27 | description: Registration key to set on the account. | ||
368 | 28 | required: [admin-name, admin-email, admin-password] | 28 | required: [admin-name, admin-email, admin-password] |
369 | 29 | additionalProperties: false | 29 | additionalProperties: false |
370 | 30 | 30 | ||
371 | === modified file 'charm-helpers.yaml' | |||
372 | --- charm-helpers.yaml 2015-05-07 10:26:45 +0000 | |||
373 | +++ charm-helpers.yaml 2021-04-13 19:16:05 +0000 | |||
374 | @@ -4,4 +4,5 @@ | |||
375 | 4 | - __init__ | 4 | - __init__ |
376 | 5 | - core | 5 | - core |
377 | 6 | - fetch | 6 | - fetch |
378 | 7 | - osplatform | ||
379 | 7 | - contrib.hahelpers | 8 | - contrib.hahelpers |
380 | 8 | 9 | ||
381 | === modified file 'charmhelpers/__init__.py' | |||
382 | --- charmhelpers/__init__.py 2015-01-28 08:59:02 +0000 | |||
383 | +++ charmhelpers/__init__.py 2021-04-13 19:16:05 +0000 | |||
384 | @@ -1,38 +1,97 @@ | |||
385 | 1 | # Copyright 2014-2015 Canonical Limited. | 1 | # Copyright 2014-2015 Canonical Limited. |
386 | 2 | # | 2 | # |
400 | 3 | # This file is part of charm-helpers. | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); |
401 | 4 | # | 4 | # you may not use this file except in compliance with the License. |
402 | 5 | # charm-helpers is free software: you can redistribute it and/or modify | 5 | # You may obtain a copy of the License at |
403 | 6 | # it under the terms of the GNU Lesser General Public License version 3 as | 6 | # |
404 | 7 | # published by the Free Software Foundation. | 7 | # http://www.apache.org/licenses/LICENSE-2.0 |
405 | 8 | # | 8 | # |
406 | 9 | # charm-helpers is distributed in the hope that it will be useful, | 9 | # Unless required by applicable law or agreed to in writing, software |
407 | 10 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 10 | # distributed under the License is distributed on an "AS IS" BASIS, |
408 | 11 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
409 | 12 | # GNU Lesser General Public License for more details. | 12 | # See the License for the specific language governing permissions and |
410 | 13 | # | 13 | # limitations under the License. |
398 | 14 | # You should have received a copy of the GNU Lesser General Public License | ||
399 | 15 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
411 | 16 | 14 | ||
412 | 17 | # Bootstrap charm-helpers, installing its dependencies if necessary using | 15 | # Bootstrap charm-helpers, installing its dependencies if necessary using |
413 | 18 | # only standard libraries. | 16 | # only standard libraries. |
414 | 17 | from __future__ import print_function | ||
415 | 18 | from __future__ import absolute_import | ||
416 | 19 | |||
417 | 20 | import functools | ||
418 | 21 | import inspect | ||
419 | 19 | import subprocess | 22 | import subprocess |
420 | 20 | import sys | 23 | import sys |
421 | 21 | 24 | ||
422 | 22 | try: | 25 | try: |
424 | 23 | import six # flake8: noqa | 26 | import six # NOQA:F401 |
425 | 24 | except ImportError: | 27 | except ImportError: |
426 | 25 | if sys.version_info.major == 2: | 28 | if sys.version_info.major == 2: |
427 | 26 | subprocess.check_call(['apt-get', 'install', '-y', 'python-six']) | 29 | subprocess.check_call(['apt-get', 'install', '-y', 'python-six']) |
428 | 27 | else: | 30 | else: |
429 | 28 | subprocess.check_call(['apt-get', 'install', '-y', 'python3-six']) | 31 | subprocess.check_call(['apt-get', 'install', '-y', 'python3-six']) |
431 | 29 | import six # flake8: noqa | 32 | import six # NOQA:F401 |
432 | 30 | 33 | ||
433 | 31 | try: | 34 | try: |
435 | 32 | import yaml # flake8: noqa | 35 | import yaml # NOQA:F401 |
436 | 33 | except ImportError: | 36 | except ImportError: |
437 | 34 | if sys.version_info.major == 2: | 37 | if sys.version_info.major == 2: |
438 | 35 | subprocess.check_call(['apt-get', 'install', '-y', 'python-yaml']) | 38 | subprocess.check_call(['apt-get', 'install', '-y', 'python-yaml']) |
439 | 36 | else: | 39 | else: |
440 | 37 | subprocess.check_call(['apt-get', 'install', '-y', 'python3-yaml']) | 40 | subprocess.check_call(['apt-get', 'install', '-y', 'python3-yaml']) |
442 | 38 | import yaml # flake8: noqa | 41 | import yaml # NOQA:F401 |
443 | 42 | |||
444 | 43 | |||
445 | 44 | # Holds a list of mapping of mangled function names that have been deprecated | ||
446 | 45 | # using the @deprecate decorator below. This is so that the warning is only | ||
447 | 46 | # printed once for each usage of the function. | ||
448 | 47 | __deprecated_functions = {} | ||
449 | 48 | |||
450 | 49 | |||
451 | 50 | def deprecate(warning, date=None, log=None): | ||
452 | 51 | """Add a deprecation warning the first time the function is used. | ||
453 | 52 | The date, which is a string in semi-ISO8660 format indicate the year-month | ||
454 | 53 | that the function is officially going to be removed. | ||
455 | 54 | |||
456 | 55 | usage: | ||
457 | 56 | |||
458 | 57 | @deprecate('use core/fetch/add_source() instead', '2017-04') | ||
459 | 58 | def contributed_add_source_thing(...): | ||
460 | 59 | ... | ||
461 | 60 | |||
462 | 61 | And it then prints to the log ONCE that the function is deprecated. | ||
463 | 62 | The reason for passing the logging function (log) is so that hookenv.log | ||
464 | 63 | can be used for a charm if needed. | ||
465 | 64 | |||
466 | 65 | :param warning: String to indicat where it has moved ot. | ||
467 | 66 | :param date: optional sting, in YYYY-MM format to indicate when the | ||
468 | 67 | function will definitely (probably) be removed. | ||
469 | 68 | :param log: The log function to call to log. If not, logs to stdout | ||
470 | 69 | """ | ||
471 | 70 | def wrap(f): | ||
472 | 71 | |||
473 | 72 | @functools.wraps(f) | ||
474 | 73 | def wrapped_f(*args, **kwargs): | ||
475 | 74 | try: | ||
476 | 75 | module = inspect.getmodule(f) | ||
477 | 76 | file = inspect.getsourcefile(f) | ||
478 | 77 | lines = inspect.getsourcelines(f) | ||
479 | 78 | f_name = "{}-{}-{}..{}-{}".format( | ||
480 | 79 | module.__name__, file, lines[0], lines[-1], f.__name__) | ||
481 | 80 | except (IOError, TypeError): | ||
482 | 81 | # assume it was local, so just use the name of the function | ||
483 | 82 | f_name = f.__name__ | ||
484 | 83 | if f_name not in __deprecated_functions: | ||
485 | 84 | __deprecated_functions[f_name] = True | ||
486 | 85 | s = "DEPRECATION WARNING: Function {} is being removed".format( | ||
487 | 86 | f.__name__) | ||
488 | 87 | if date: | ||
489 | 88 | s = "{} on/around {}".format(s, date) | ||
490 | 89 | if warning: | ||
491 | 90 | s = "{} : {}".format(s, warning) | ||
492 | 91 | if log: | ||
493 | 92 | log(s) | ||
494 | 93 | else: | ||
495 | 94 | print(s) | ||
496 | 95 | return f(*args, **kwargs) | ||
497 | 96 | return wrapped_f | ||
498 | 97 | return wrap | ||
499 | 39 | 98 | ||
500 | === modified file 'charmhelpers/contrib/__init__.py' | |||
501 | --- charmhelpers/contrib/__init__.py 2015-01-30 11:16:09 +0000 | |||
502 | +++ charmhelpers/contrib/__init__.py 2021-04-13 19:16:05 +0000 | |||
503 | @@ -1,15 +1,13 @@ | |||
504 | 1 | # Copyright 2014-2015 Canonical Limited. | 1 | # Copyright 2014-2015 Canonical Limited. |
505 | 2 | # | 2 | # |
519 | 3 | # This file is part of charm-helpers. | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); |
520 | 4 | # | 4 | # you may not use this file except in compliance with the License. |
521 | 5 | # charm-helpers is free software: you can redistribute it and/or modify | 5 | # You may obtain a copy of the License at |
522 | 6 | # it under the terms of the GNU Lesser General Public License version 3 as | 6 | # |
523 | 7 | # published by the Free Software Foundation. | 7 | # http://www.apache.org/licenses/LICENSE-2.0 |
524 | 8 | # | 8 | # |
525 | 9 | # charm-helpers is distributed in the hope that it will be useful, | 9 | # Unless required by applicable law or agreed to in writing, software |
526 | 10 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 10 | # distributed under the License is distributed on an "AS IS" BASIS, |
527 | 11 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
528 | 12 | # GNU Lesser General Public License for more details. | 12 | # See the License for the specific language governing permissions and |
529 | 13 | # | 13 | # limitations under the License. |
517 | 14 | # You should have received a copy of the GNU Lesser General Public License | ||
518 | 15 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
530 | 16 | 14 | ||
531 | === modified file 'charmhelpers/contrib/hahelpers/__init__.py' | |||
532 | --- charmhelpers/contrib/hahelpers/__init__.py 2015-01-30 11:16:09 +0000 | |||
533 | +++ charmhelpers/contrib/hahelpers/__init__.py 2021-04-13 19:16:05 +0000 | |||
534 | @@ -1,15 +1,13 @@ | |||
535 | 1 | # Copyright 2014-2015 Canonical Limited. | 1 | # Copyright 2014-2015 Canonical Limited. |
536 | 2 | # | 2 | # |
550 | 3 | # This file is part of charm-helpers. | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); |
551 | 4 | # | 4 | # you may not use this file except in compliance with the License. |
552 | 5 | # charm-helpers is free software: you can redistribute it and/or modify | 5 | # You may obtain a copy of the License at |
553 | 6 | # it under the terms of the GNU Lesser General Public License version 3 as | 6 | # |
554 | 7 | # published by the Free Software Foundation. | 7 | # http://www.apache.org/licenses/LICENSE-2.0 |
555 | 8 | # | 8 | # |
556 | 9 | # charm-helpers is distributed in the hope that it will be useful, | 9 | # Unless required by applicable law or agreed to in writing, software |
557 | 10 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 10 | # distributed under the License is distributed on an "AS IS" BASIS, |
558 | 11 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
559 | 12 | # GNU Lesser General Public License for more details. | 12 | # See the License for the specific language governing permissions and |
560 | 13 | # | 13 | # limitations under the License. |
548 | 14 | # You should have received a copy of the GNU Lesser General Public License | ||
549 | 15 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
561 | 16 | 14 | ||
562 | === modified file 'charmhelpers/contrib/hahelpers/apache.py' | |||
563 | --- charmhelpers/contrib/hahelpers/apache.py 2015-01-30 11:16:09 +0000 | |||
564 | +++ charmhelpers/contrib/hahelpers/apache.py 2021-04-13 19:16:05 +0000 | |||
565 | @@ -1,18 +1,16 @@ | |||
566 | 1 | # Copyright 2014-2015 Canonical Limited. | 1 | # Copyright 2014-2015 Canonical Limited. |
567 | 2 | # | 2 | # |
581 | 3 | # This file is part of charm-helpers. | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); |
582 | 4 | # | 4 | # you may not use this file except in compliance with the License. |
583 | 5 | # charm-helpers is free software: you can redistribute it and/or modify | 5 | # You may obtain a copy of the License at |
584 | 6 | # it under the terms of the GNU Lesser General Public License version 3 as | 6 | # |
585 | 7 | # published by the Free Software Foundation. | 7 | # http://www.apache.org/licenses/LICENSE-2.0 |
586 | 8 | # | 8 | # |
587 | 9 | # charm-helpers is distributed in the hope that it will be useful, | 9 | # Unless required by applicable law or agreed to in writing, software |
588 | 10 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 10 | # distributed under the License is distributed on an "AS IS" BASIS, |
589 | 11 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
590 | 12 | # GNU Lesser General Public License for more details. | 12 | # See the License for the specific language governing permissions and |
591 | 13 | # | 13 | # limitations under the License. |
579 | 14 | # You should have received a copy of the GNU Lesser General Public License | ||
580 | 15 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
592 | 16 | 14 | ||
593 | 17 | # | 15 | # |
594 | 18 | # Copyright 2012 Canonical Ltd. | 16 | # Copyright 2012 Canonical Ltd. |
595 | @@ -24,8 +22,9 @@ | |||
596 | 24 | # Adam Gandelman <adamg@ubuntu.com> | 22 | # Adam Gandelman <adamg@ubuntu.com> |
597 | 25 | # | 23 | # |
598 | 26 | 24 | ||
600 | 27 | import subprocess | 25 | import os |
601 | 28 | 26 | ||
602 | 27 | from charmhelpers.core import host | ||
603 | 29 | from charmhelpers.core.hookenv import ( | 28 | from charmhelpers.core.hookenv import ( |
604 | 30 | config as config_get, | 29 | config as config_get, |
605 | 31 | relation_get, | 30 | relation_get, |
606 | @@ -66,7 +65,8 @@ | |||
607 | 66 | if ca_cert is None: | 65 | if ca_cert is None: |
608 | 67 | log("Inspecting identity-service relations for CA SSL certificate.", | 66 | log("Inspecting identity-service relations for CA SSL certificate.", |
609 | 68 | level=INFO) | 67 | level=INFO) |
611 | 69 | for r_id in relation_ids('identity-service'): | 68 | for r_id in (relation_ids('identity-service') + |
612 | 69 | relation_ids('identity-credentials')): | ||
613 | 70 | for unit in relation_list(r_id): | 70 | for unit in relation_list(r_id): |
614 | 71 | if ca_cert is None: | 71 | if ca_cert is None: |
615 | 72 | ca_cert = relation_get('ca_cert', | 72 | ca_cert = relation_get('ca_cert', |
616 | @@ -74,9 +74,13 @@ | |||
617 | 74 | return ca_cert | 74 | return ca_cert |
618 | 75 | 75 | ||
619 | 76 | 76 | ||
620 | 77 | def retrieve_ca_cert(cert_file): | ||
621 | 78 | cert = None | ||
622 | 79 | if os.path.isfile(cert_file): | ||
623 | 80 | with open(cert_file, 'rb') as crt: | ||
624 | 81 | cert = crt.read() | ||
625 | 82 | return cert | ||
626 | 83 | |||
627 | 84 | |||
628 | 77 | def install_ca_cert(ca_cert): | 85 | def install_ca_cert(ca_cert): |
634 | 78 | if ca_cert: | 86 | host.install_ca_cert(ca_cert, 'keystone_juju_ca_cert') |
630 | 79 | with open('/usr/local/share/ca-certificates/keystone_juju_ca_cert.crt', | ||
631 | 80 | 'w') as crt: | ||
632 | 81 | crt.write(ca_cert) | ||
633 | 82 | subprocess.check_call(['update-ca-certificates', '--fresh']) | ||
635 | 83 | 87 | ||
636 | === modified file 'charmhelpers/contrib/hahelpers/cluster.py' | |||
637 | --- charmhelpers/contrib/hahelpers/cluster.py 2015-07-03 09:13:26 +0000 | |||
638 | +++ charmhelpers/contrib/hahelpers/cluster.py 2021-04-13 19:16:05 +0000 | |||
639 | @@ -1,18 +1,16 @@ | |||
640 | 1 | # Copyright 2014-2015 Canonical Limited. | 1 | # Copyright 2014-2015 Canonical Limited. |
641 | 2 | # | 2 | # |
655 | 3 | # This file is part of charm-helpers. | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); |
656 | 4 | # | 4 | # you may not use this file except in compliance with the License. |
657 | 5 | # charm-helpers is free software: you can redistribute it and/or modify | 5 | # You may obtain a copy of the License at |
658 | 6 | # it under the terms of the GNU Lesser General Public License version 3 as | 6 | # |
659 | 7 | # published by the Free Software Foundation. | 7 | # http://www.apache.org/licenses/LICENSE-2.0 |
660 | 8 | # | 8 | # |
661 | 9 | # charm-helpers is distributed in the hope that it will be useful, | 9 | # Unless required by applicable law or agreed to in writing, software |
662 | 10 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 10 | # distributed under the License is distributed on an "AS IS" BASIS, |
663 | 11 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
664 | 12 | # GNU Lesser General Public License for more details. | 12 | # See the License for the specific language governing permissions and |
665 | 13 | # | 13 | # limitations under the License. |
653 | 14 | # You should have received a copy of the GNU Lesser General Public License | ||
654 | 15 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
666 | 16 | 14 | ||
667 | 17 | # | 15 | # |
668 | 18 | # Copyright 2012 Canonical Ltd. | 16 | # Copyright 2012 Canonical Ltd. |
669 | @@ -29,6 +27,7 @@ | |||
670 | 29 | 27 | ||
671 | 30 | import subprocess | 28 | import subprocess |
672 | 31 | import os | 29 | import os |
673 | 30 | import time | ||
674 | 32 | 31 | ||
675 | 33 | from socket import gethostname as get_unit_hostname | 32 | from socket import gethostname as get_unit_hostname |
676 | 34 | 33 | ||
677 | @@ -41,10 +40,14 @@ | |||
678 | 41 | relation_get, | 40 | relation_get, |
679 | 42 | config as config_get, | 41 | config as config_get, |
680 | 43 | INFO, | 42 | INFO, |
682 | 44 | ERROR, | 43 | DEBUG, |
683 | 45 | WARNING, | 44 | WARNING, |
684 | 46 | unit_get, | 45 | unit_get, |
686 | 47 | is_leader as juju_is_leader | 46 | is_leader as juju_is_leader, |
687 | 47 | status_set, | ||
688 | 48 | ) | ||
689 | 49 | from charmhelpers.core.host import ( | ||
690 | 50 | modulo_distribution, | ||
691 | 48 | ) | 51 | ) |
692 | 49 | from charmhelpers.core.decorators import ( | 52 | from charmhelpers.core.decorators import ( |
693 | 50 | retry_on_exception, | 53 | retry_on_exception, |
694 | @@ -60,6 +63,10 @@ | |||
695 | 60 | pass | 63 | pass |
696 | 61 | 64 | ||
697 | 62 | 65 | ||
698 | 66 | class HAIncorrectConfig(Exception): | ||
699 | 67 | pass | ||
700 | 68 | |||
701 | 69 | |||
702 | 63 | class CRMResourceNotFound(Exception): | 70 | class CRMResourceNotFound(Exception): |
703 | 64 | pass | 71 | pass |
704 | 65 | 72 | ||
705 | @@ -216,6 +223,11 @@ | |||
706 | 216 | return True | 223 | return True |
707 | 217 | if config_get('ssl_cert') and config_get('ssl_key'): | 224 | if config_get('ssl_cert') and config_get('ssl_key'): |
708 | 218 | return True | 225 | return True |
709 | 226 | for r_id in relation_ids('certificates'): | ||
710 | 227 | for unit in relation_list(r_id): | ||
711 | 228 | ca = relation_get('ca', rid=r_id, unit=unit) | ||
712 | 229 | if ca: | ||
713 | 230 | return True | ||
714 | 219 | for r_id in relation_ids('identity-service'): | 231 | for r_id in relation_ids('identity-service'): |
715 | 220 | for unit in relation_list(r_id): | 232 | for unit in relation_list(r_id): |
716 | 221 | # TODO - needs fixing for new helper as ssl_cert/key suffixes with CN | 233 | # TODO - needs fixing for new helper as ssl_cert/key suffixes with CN |
717 | @@ -274,27 +286,71 @@ | |||
718 | 274 | Obtains all relevant configuration from charm configuration required | 286 | Obtains all relevant configuration from charm configuration required |
719 | 275 | for initiating a relation to hacluster: | 287 | for initiating a relation to hacluster: |
720 | 276 | 288 | ||
722 | 277 | ha-bindiface, ha-mcastport, vip | 289 | ha-bindiface, ha-mcastport, vip, os-internal-hostname, |
723 | 290 | os-admin-hostname, os-public-hostname, os-access-hostname | ||
724 | 278 | 291 | ||
725 | 279 | param: exclude_keys: list of setting key(s) to be excluded. | 292 | param: exclude_keys: list of setting key(s) to be excluded. |
726 | 280 | returns: dict: A dict containing settings keyed by setting name. | 293 | returns: dict: A dict containing settings keyed by setting name. |
728 | 281 | raises: HAIncompleteConfig if settings are missing. | 294 | raises: HAIncompleteConfig if settings are missing or incorrect. |
729 | 282 | ''' | 295 | ''' |
731 | 283 | settings = ['ha-bindiface', 'ha-mcastport', 'vip'] | 296 | settings = ['ha-bindiface', 'ha-mcastport', 'vip', 'os-internal-hostname', |
732 | 297 | 'os-admin-hostname', 'os-public-hostname', 'os-access-hostname'] | ||
733 | 284 | conf = {} | 298 | conf = {} |
734 | 285 | for setting in settings: | 299 | for setting in settings: |
735 | 286 | if exclude_keys and setting in exclude_keys: | 300 | if exclude_keys and setting in exclude_keys: |
736 | 287 | continue | 301 | continue |
737 | 288 | 302 | ||
738 | 289 | conf[setting] = config_get(setting) | 303 | conf[setting] = config_get(setting) |
744 | 290 | missing = [] | 304 | |
745 | 291 | [missing.append(s) for s, v in six.iteritems(conf) if v is None] | 305 | if not valid_hacluster_config(): |
746 | 292 | if missing: | 306 | raise HAIncorrectConfig('Insufficient or incorrect config data to ' |
747 | 293 | log('Insufficient config data to configure hacluster.', level=ERROR) | 307 | 'configure hacluster.') |
743 | 294 | raise HAIncompleteConfig | ||
748 | 295 | return conf | 308 | return conf |
749 | 296 | 309 | ||
750 | 297 | 310 | ||
751 | 311 | def valid_hacluster_config(): | ||
752 | 312 | ''' | ||
753 | 313 | Check that either vip or dns-ha is set. If dns-ha then one of os-*-hostname | ||
754 | 314 | must be set. | ||
755 | 315 | |||
756 | 316 | Note: ha-bindiface and ha-macastport both have defaults and will always | ||
757 | 317 | be set. We only care that either vip or dns-ha is set. | ||
758 | 318 | |||
759 | 319 | :returns: boolean: valid config returns true. | ||
760 | 320 | raises: HAIncompatibileConfig if settings conflict. | ||
761 | 321 | raises: HAIncompleteConfig if settings are missing. | ||
762 | 322 | ''' | ||
763 | 323 | vip = config_get('vip') | ||
764 | 324 | dns = config_get('dns-ha') | ||
765 | 325 | if not(bool(vip) ^ bool(dns)): | ||
766 | 326 | msg = ('HA: Either vip or dns-ha must be set but not both in order to ' | ||
767 | 327 | 'use high availability') | ||
768 | 328 | status_set('blocked', msg) | ||
769 | 329 | raise HAIncorrectConfig(msg) | ||
770 | 330 | |||
771 | 331 | # If dns-ha then one of os-*-hostname must be set | ||
772 | 332 | if dns: | ||
773 | 333 | dns_settings = ['os-internal-hostname', 'os-admin-hostname', | ||
774 | 334 | 'os-public-hostname', 'os-access-hostname'] | ||
775 | 335 | # At this point it is unknown if one or all of the possible | ||
776 | 336 | # network spaces are in HA. Validate at least one is set which is | ||
777 | 337 | # the minimum required. | ||
778 | 338 | for setting in dns_settings: | ||
779 | 339 | if config_get(setting): | ||
780 | 340 | log('DNS HA: At least one hostname is set {}: {}' | ||
781 | 341 | ''.format(setting, config_get(setting)), | ||
782 | 342 | level=DEBUG) | ||
783 | 343 | return True | ||
784 | 344 | |||
785 | 345 | msg = ('DNS HA: At least one os-*-hostname(s) must be set to use ' | ||
786 | 346 | 'DNS HA') | ||
787 | 347 | status_set('blocked', msg) | ||
788 | 348 | raise HAIncompleteConfig(msg) | ||
789 | 349 | |||
790 | 350 | log('VIP HA: VIP is set {}'.format(vip), level=DEBUG) | ||
791 | 351 | return True | ||
792 | 352 | |||
793 | 353 | |||
794 | 298 | def canonical_url(configs, vip_setting='vip'): | 354 | def canonical_url(configs, vip_setting='vip'): |
795 | 299 | ''' | 355 | ''' |
796 | 300 | Returns the correct HTTP URL to this host given the state of HTTPS | 356 | Returns the correct HTTP URL to this host given the state of HTTPS |
797 | @@ -314,3 +370,37 @@ | |||
798 | 314 | else: | 370 | else: |
799 | 315 | addr = unit_get('private-address') | 371 | addr = unit_get('private-address') |
800 | 316 | return '%s://%s' % (scheme, addr) | 372 | return '%s://%s' % (scheme, addr) |
801 | 373 | |||
802 | 374 | |||
803 | 375 | def distributed_wait(modulo=None, wait=None, operation_name='operation'): | ||
804 | 376 | ''' Distribute operations by waiting based on modulo_distribution | ||
805 | 377 | |||
806 | 378 | If modulo and or wait are not set, check config_get for those values. | ||
807 | 379 | If config values are not set, default to modulo=3 and wait=30. | ||
808 | 380 | |||
809 | 381 | :param modulo: int The modulo number creates the group distribution | ||
810 | 382 | :param wait: int The constant time wait value | ||
811 | 383 | :param operation_name: string Operation name for status message | ||
812 | 384 | i.e. 'restart' | ||
813 | 385 | :side effect: Calls config_get() | ||
814 | 386 | :side effect: Calls log() | ||
815 | 387 | :side effect: Calls status_set() | ||
816 | 388 | :side effect: Calls time.sleep() | ||
817 | 389 | ''' | ||
818 | 390 | if modulo is None: | ||
819 | 391 | modulo = config_get('modulo-nodes') or 3 | ||
820 | 392 | if wait is None: | ||
821 | 393 | wait = config_get('known-wait') or 30 | ||
822 | 394 | if juju_is_leader(): | ||
823 | 395 | # The leader should never wait | ||
824 | 396 | calculated_wait = 0 | ||
825 | 397 | else: | ||
826 | 398 | # non_zero_wait=True guarantees the non-leader who gets modulo 0 | ||
827 | 399 | # will still wait | ||
828 | 400 | calculated_wait = modulo_distribution(modulo=modulo, wait=wait, | ||
829 | 401 | non_zero_wait=True) | ||
830 | 402 | msg = "Waiting {} seconds for {} ...".format(calculated_wait, | ||
831 | 403 | operation_name) | ||
832 | 404 | log(msg, DEBUG) | ||
833 | 405 | status_set('maintenance', msg) | ||
834 | 406 | time.sleep(calculated_wait) | ||
835 | 317 | 407 | ||
836 | === modified file 'charmhelpers/core/__init__.py' | |||
837 | --- charmhelpers/core/__init__.py 2015-01-28 08:59:02 +0000 | |||
838 | +++ charmhelpers/core/__init__.py 2021-04-13 19:16:05 +0000 | |||
839 | @@ -1,15 +1,13 @@ | |||
840 | 1 | # Copyright 2014-2015 Canonical Limited. | 1 | # Copyright 2014-2015 Canonical Limited. |
841 | 2 | # | 2 | # |
855 | 3 | # This file is part of charm-helpers. | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); |
856 | 4 | # | 4 | # you may not use this file except in compliance with the License. |
857 | 5 | # charm-helpers is free software: you can redistribute it and/or modify | 5 | # You may obtain a copy of the License at |
858 | 6 | # it under the terms of the GNU Lesser General Public License version 3 as | 6 | # |
859 | 7 | # published by the Free Software Foundation. | 7 | # http://www.apache.org/licenses/LICENSE-2.0 |
860 | 8 | # | 8 | # |
861 | 9 | # charm-helpers is distributed in the hope that it will be useful, | 9 | # Unless required by applicable law or agreed to in writing, software |
862 | 10 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 10 | # distributed under the License is distributed on an "AS IS" BASIS, |
863 | 11 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
864 | 12 | # GNU Lesser General Public License for more details. | 12 | # See the License for the specific language governing permissions and |
865 | 13 | # | 13 | # limitations under the License. |
853 | 14 | # You should have received a copy of the GNU Lesser General Public License | ||
854 | 15 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
866 | 16 | 14 | ||
867 | === modified file 'charmhelpers/core/decorators.py' | |||
868 | --- charmhelpers/core/decorators.py 2015-01-28 08:59:02 +0000 | |||
869 | +++ charmhelpers/core/decorators.py 2021-04-13 19:16:05 +0000 | |||
870 | @@ -1,18 +1,16 @@ | |||
871 | 1 | # Copyright 2014-2015 Canonical Limited. | 1 | # Copyright 2014-2015 Canonical Limited. |
872 | 2 | # | 2 | # |
886 | 3 | # This file is part of charm-helpers. | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); |
887 | 4 | # | 4 | # you may not use this file except in compliance with the License. |
888 | 5 | # charm-helpers is free software: you can redistribute it and/or modify | 5 | # You may obtain a copy of the License at |
889 | 6 | # it under the terms of the GNU Lesser General Public License version 3 as | 6 | # |
890 | 7 | # published by the Free Software Foundation. | 7 | # http://www.apache.org/licenses/LICENSE-2.0 |
891 | 8 | # | 8 | # |
892 | 9 | # charm-helpers is distributed in the hope that it will be useful, | 9 | # Unless required by applicable law or agreed to in writing, software |
893 | 10 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 10 | # distributed under the License is distributed on an "AS IS" BASIS, |
894 | 11 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
895 | 12 | # GNU Lesser General Public License for more details. | 12 | # See the License for the specific language governing permissions and |
896 | 13 | # | 13 | # limitations under the License. |
884 | 14 | # You should have received a copy of the GNU Lesser General Public License | ||
885 | 15 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
897 | 16 | 14 | ||
898 | 17 | # | 15 | # |
899 | 18 | # Copyright 2014 Canonical Ltd. | 16 | # Copyright 2014 Canonical Ltd. |
900 | 19 | 17 | ||
901 | === modified file 'charmhelpers/core/files.py' | |||
902 | --- charmhelpers/core/files.py 2015-12-11 15:23:38 +0000 | |||
903 | +++ charmhelpers/core/files.py 2021-04-13 19:16:05 +0000 | |||
904 | @@ -3,19 +3,17 @@ | |||
905 | 3 | 3 | ||
906 | 4 | # Copyright 2014-2015 Canonical Limited. | 4 | # Copyright 2014-2015 Canonical Limited. |
907 | 5 | # | 5 | # |
921 | 6 | # This file is part of charm-helpers. | 6 | # Licensed under the Apache License, Version 2.0 (the "License"); |
922 | 7 | # | 7 | # you may not use this file except in compliance with the License. |
923 | 8 | # charm-helpers is free software: you can redistribute it and/or modify | 8 | # You may obtain a copy of the License at |
924 | 9 | # it under the terms of the GNU Lesser General Public License version 3 as | 9 | # |
925 | 10 | # published by the Free Software Foundation. | 10 | # http://www.apache.org/licenses/LICENSE-2.0 |
926 | 11 | # | 11 | # |
927 | 12 | # charm-helpers is distributed in the hope that it will be useful, | 12 | # Unless required by applicable law or agreed to in writing, software |
928 | 13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 13 | # distributed under the License is distributed on an "AS IS" BASIS, |
929 | 14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 14 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
930 | 15 | # GNU Lesser General Public License for more details. | 15 | # See the License for the specific language governing permissions and |
931 | 16 | # | 16 | # limitations under the License. |
919 | 17 | # You should have received a copy of the GNU Lesser General Public License | ||
920 | 18 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
932 | 19 | 17 | ||
933 | 20 | __author__ = 'Jorge Niedbalski <niedbalski@ubuntu.com>' | 18 | __author__ = 'Jorge Niedbalski <niedbalski@ubuntu.com>' |
934 | 21 | 19 | ||
935 | 22 | 20 | ||
936 | === modified file 'charmhelpers/core/fstab.py' | |||
937 | --- charmhelpers/core/fstab.py 2015-03-12 11:42:26 +0000 | |||
938 | +++ charmhelpers/core/fstab.py 2021-04-13 19:16:05 +0000 | |||
939 | @@ -3,19 +3,17 @@ | |||
940 | 3 | 3 | ||
941 | 4 | # Copyright 2014-2015 Canonical Limited. | 4 | # Copyright 2014-2015 Canonical Limited. |
942 | 5 | # | 5 | # |
956 | 6 | # This file is part of charm-helpers. | 6 | # Licensed under the Apache License, Version 2.0 (the "License"); |
957 | 7 | # | 7 | # you may not use this file except in compliance with the License. |
958 | 8 | # charm-helpers is free software: you can redistribute it and/or modify | 8 | # You may obtain a copy of the License at |
959 | 9 | # it under the terms of the GNU Lesser General Public License version 3 as | 9 | # |
960 | 10 | # published by the Free Software Foundation. | 10 | # http://www.apache.org/licenses/LICENSE-2.0 |
961 | 11 | # | 11 | # |
962 | 12 | # charm-helpers is distributed in the hope that it will be useful, | 12 | # Unless required by applicable law or agreed to in writing, software |
963 | 13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 13 | # distributed under the License is distributed on an "AS IS" BASIS, |
964 | 14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 14 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
965 | 15 | # GNU Lesser General Public License for more details. | 15 | # See the License for the specific language governing permissions and |
966 | 16 | # | 16 | # limitations under the License. |
954 | 17 | # You should have received a copy of the GNU Lesser General Public License | ||
955 | 18 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
967 | 19 | 17 | ||
968 | 20 | import io | 18 | import io |
969 | 21 | import os | 19 | import os |
970 | 22 | 20 | ||
971 | === modified file 'charmhelpers/core/hookenv.py' | |||
972 | --- charmhelpers/core/hookenv.py 2015-12-11 15:23:38 +0000 | |||
973 | +++ charmhelpers/core/hookenv.py 2021-04-13 19:16:05 +0000 | |||
974 | @@ -1,18 +1,16 @@ | |||
975 | 1 | # Copyright 2014-2015 Canonical Limited. | 1 | # Copyright 2014-2015 Canonical Limited. |
976 | 2 | # | 2 | # |
990 | 3 | # This file is part of charm-helpers. | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); |
991 | 4 | # | 4 | # you may not use this file except in compliance with the License. |
992 | 5 | # charm-helpers is free software: you can redistribute it and/or modify | 5 | # You may obtain a copy of the License at |
993 | 6 | # it under the terms of the GNU Lesser General Public License version 3 as | 6 | # |
994 | 7 | # published by the Free Software Foundation. | 7 | # http://www.apache.org/licenses/LICENSE-2.0 |
995 | 8 | # | 8 | # |
996 | 9 | # charm-helpers is distributed in the hope that it will be useful, | 9 | # Unless required by applicable law or agreed to in writing, software |
997 | 10 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 10 | # distributed under the License is distributed on an "AS IS" BASIS, |
998 | 11 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
999 | 12 | # GNU Lesser General Public License for more details. | 12 | # See the License for the specific language governing permissions and |
1000 | 13 | # | 13 | # limitations under the License. |
988 | 14 | # You should have received a copy of the GNU Lesser General Public License | ||
989 | 15 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
1001 | 16 | 14 | ||
1002 | 17 | "Interactions with the Juju environment" | 15 | "Interactions with the Juju environment" |
1003 | 18 | # Copyright 2013 Canonical Ltd. | 16 | # Copyright 2013 Canonical Ltd. |
1004 | @@ -24,10 +22,12 @@ | |||
1005 | 24 | import copy | 22 | import copy |
1006 | 25 | from distutils.version import LooseVersion | 23 | from distutils.version import LooseVersion |
1007 | 26 | from functools import wraps | 24 | from functools import wraps |
1008 | 25 | from collections import namedtuple | ||
1009 | 27 | import glob | 26 | import glob |
1010 | 28 | import os | 27 | import os |
1011 | 29 | import json | 28 | import json |
1012 | 30 | import yaml | 29 | import yaml |
1013 | 30 | import re | ||
1014 | 31 | import subprocess | 31 | import subprocess |
1015 | 32 | import sys | 32 | import sys |
1016 | 33 | import errno | 33 | import errno |
1017 | @@ -40,12 +40,20 @@ | |||
1018 | 40 | else: | 40 | else: |
1019 | 41 | from collections import UserDict | 41 | from collections import UserDict |
1020 | 42 | 42 | ||
1021 | 43 | |||
1022 | 43 | CRITICAL = "CRITICAL" | 44 | CRITICAL = "CRITICAL" |
1023 | 44 | ERROR = "ERROR" | 45 | ERROR = "ERROR" |
1024 | 45 | WARNING = "WARNING" | 46 | WARNING = "WARNING" |
1025 | 46 | INFO = "INFO" | 47 | INFO = "INFO" |
1026 | 47 | DEBUG = "DEBUG" | 48 | DEBUG = "DEBUG" |
1027 | 49 | TRACE = "TRACE" | ||
1028 | 48 | MARKER = object() | 50 | MARKER = object() |
1029 | 51 | SH_MAX_ARG = 131071 | ||
1030 | 52 | |||
1031 | 53 | |||
1032 | 54 | RANGE_WARNING = ('Passing NO_PROXY string that includes a cidr. ' | ||
1033 | 55 | 'This may not be compatible with software you are ' | ||
1034 | 56 | 'running in your shell.') | ||
1035 | 49 | 57 | ||
1036 | 50 | cache = {} | 58 | cache = {} |
1037 | 51 | 59 | ||
1038 | @@ -66,7 +74,7 @@ | |||
1039 | 66 | @wraps(func) | 74 | @wraps(func) |
1040 | 67 | def wrapper(*args, **kwargs): | 75 | def wrapper(*args, **kwargs): |
1041 | 68 | global cache | 76 | global cache |
1043 | 69 | key = str((func, args, kwargs)) | 77 | key = json.dumps((func, args, kwargs), sort_keys=True, default=str) |
1044 | 70 | try: | 78 | try: |
1045 | 71 | return cache[key] | 79 | return cache[key] |
1046 | 72 | except KeyError: | 80 | except KeyError: |
1047 | @@ -96,7 +104,7 @@ | |||
1048 | 96 | command += ['-l', level] | 104 | command += ['-l', level] |
1049 | 97 | if not isinstance(message, six.string_types): | 105 | if not isinstance(message, six.string_types): |
1050 | 98 | message = repr(message) | 106 | message = repr(message) |
1052 | 99 | command += [message] | 107 | command += [message[:SH_MAX_ARG]] |
1053 | 100 | # Missing juju-log should not cause failures in unit tests | 108 | # Missing juju-log should not cause failures in unit tests |
1054 | 101 | # Send log output to stderr | 109 | # Send log output to stderr |
1055 | 102 | try: | 110 | try: |
1056 | @@ -199,9 +207,56 @@ | |||
1057 | 199 | return os.environ.get('JUJU_REMOTE_UNIT', None) | 207 | return os.environ.get('JUJU_REMOTE_UNIT', None) |
1058 | 200 | 208 | ||
1059 | 201 | 209 | ||
1060 | 210 | def application_name(): | ||
1061 | 211 | """ | ||
1062 | 212 | The name of the deployed application this unit belongs to. | ||
1063 | 213 | """ | ||
1064 | 214 | return local_unit().split('/')[0] | ||
1065 | 215 | |||
1066 | 216 | |||
1067 | 202 | def service_name(): | 217 | def service_name(): |
1070 | 203 | """The name service group this unit belongs to""" | 218 | """ |
1071 | 204 | return local_unit().split('/')[0] | 219 | .. deprecated:: 0.19.1 |
1072 | 220 | Alias for :func:`application_name`. | ||
1073 | 221 | """ | ||
1074 | 222 | return application_name() | ||
1075 | 223 | |||
1076 | 224 | |||
1077 | 225 | def model_name(): | ||
1078 | 226 | """ | ||
1079 | 227 | Name of the model that this unit is deployed in. | ||
1080 | 228 | """ | ||
1081 | 229 | return os.environ['JUJU_MODEL_NAME'] | ||
1082 | 230 | |||
1083 | 231 | |||
1084 | 232 | def model_uuid(): | ||
1085 | 233 | """ | ||
1086 | 234 | UUID of the model that this unit is deployed in. | ||
1087 | 235 | """ | ||
1088 | 236 | return os.environ['JUJU_MODEL_UUID'] | ||
1089 | 237 | |||
1090 | 238 | |||
1091 | 239 | def principal_unit(): | ||
1092 | 240 | """Returns the principal unit of this unit, otherwise None""" | ||
1093 | 241 | # Juju 2.2 and above provides JUJU_PRINCIPAL_UNIT | ||
1094 | 242 | principal_unit = os.environ.get('JUJU_PRINCIPAL_UNIT', None) | ||
1095 | 243 | # If it's empty, then this unit is the principal | ||
1096 | 244 | if principal_unit == '': | ||
1097 | 245 | return os.environ['JUJU_UNIT_NAME'] | ||
1098 | 246 | elif principal_unit is not None: | ||
1099 | 247 | return principal_unit | ||
1100 | 248 | # For Juju 2.1 and below, let's try work out the principle unit by | ||
1101 | 249 | # the various charms' metadata.yaml. | ||
1102 | 250 | for reltype in relation_types(): | ||
1103 | 251 | for rid in relation_ids(reltype): | ||
1104 | 252 | for unit in related_units(rid): | ||
1105 | 253 | md = _metadata_unit(unit) | ||
1106 | 254 | if not md: | ||
1107 | 255 | continue | ||
1108 | 256 | subordinate = md.pop('subordinate', None) | ||
1109 | 257 | if not subordinate: | ||
1110 | 258 | return unit | ||
1111 | 259 | return None | ||
1112 | 205 | 260 | ||
1113 | 206 | 261 | ||
1114 | 207 | @cached | 262 | @cached |
1115 | @@ -265,7 +320,7 @@ | |||
1116 | 265 | self.implicit_save = True | 320 | self.implicit_save = True |
1117 | 266 | self._prev_dict = None | 321 | self._prev_dict = None |
1118 | 267 | self.path = os.path.join(charm_dir(), Config.CONFIG_FILE_NAME) | 322 | self.path = os.path.join(charm_dir(), Config.CONFIG_FILE_NAME) |
1120 | 268 | if os.path.exists(self.path): | 323 | if os.path.exists(self.path) and os.stat(self.path).st_size: |
1121 | 269 | self.load_previous() | 324 | self.load_previous() |
1122 | 270 | atexit(self._implicit_save) | 325 | atexit(self._implicit_save) |
1123 | 271 | 326 | ||
1124 | @@ -285,7 +340,11 @@ | |||
1125 | 285 | """ | 340 | """ |
1126 | 286 | self.path = path or self.path | 341 | self.path = path or self.path |
1127 | 287 | with open(self.path) as f: | 342 | with open(self.path) as f: |
1129 | 288 | self._prev_dict = json.load(f) | 343 | try: |
1130 | 344 | self._prev_dict = json.load(f) | ||
1131 | 345 | except ValueError as e: | ||
1132 | 346 | log('Unable to parse previous config data - {}'.format(str(e)), | ||
1133 | 347 | level=ERROR) | ||
1134 | 289 | for k, v in copy.deepcopy(self._prev_dict).items(): | 348 | for k, v in copy.deepcopy(self._prev_dict).items(): |
1135 | 290 | if k not in self: | 349 | if k not in self: |
1136 | 291 | self[k] = v | 350 | self[k] = v |
1137 | @@ -321,6 +380,7 @@ | |||
1138 | 321 | 380 | ||
1139 | 322 | """ | 381 | """ |
1140 | 323 | with open(self.path, 'w') as f: | 382 | with open(self.path, 'w') as f: |
1141 | 383 | os.fchmod(f.fileno(), 0o600) | ||
1142 | 324 | json.dump(self, f) | 384 | json.dump(self, f) |
1143 | 325 | 385 | ||
1144 | 326 | def _implicit_save(self): | 386 | def _implicit_save(self): |
1145 | @@ -328,20 +388,40 @@ | |||
1146 | 328 | self.save() | 388 | self.save() |
1147 | 329 | 389 | ||
1148 | 330 | 390 | ||
1150 | 331 | @cached | 391 | _cache_config = None |
1151 | 392 | |||
1152 | 393 | |||
1153 | 332 | def config(scope=None): | 394 | def config(scope=None): |
1162 | 333 | """Juju charm configuration""" | 395 | """ |
1163 | 334 | config_cmd_line = ['config-get'] | 396 | Get the juju charm configuration (scope==None) or individual key, |
1164 | 335 | if scope is not None: | 397 | (scope=str). The returned value is a Python data structure loaded as |
1165 | 336 | config_cmd_line.append(scope) | 398 | JSON from the Juju config command. |
1166 | 337 | config_cmd_line.append('--format=json') | 399 | |
1167 | 338 | try: | 400 | :param scope: If set, return the value for the specified key. |
1168 | 339 | config_data = json.loads( | 401 | :type scope: Optional[str] |
1169 | 340 | subprocess.check_output(config_cmd_line).decode('UTF-8')) | 402 | :returns: Either the whole config as a Config, or a key from it. |
1170 | 403 | :rtype: Any | ||
1171 | 404 | """ | ||
1172 | 405 | global _cache_config | ||
1173 | 406 | config_cmd_line = ['config-get', '--all', '--format=json'] | ||
1174 | 407 | try: | ||
1175 | 408 | # JSON Decode Exception for Python3.5+ | ||
1176 | 409 | exc_json = json.decoder.JSONDecodeError | ||
1177 | 410 | except AttributeError: | ||
1178 | 411 | # JSON Decode Exception for Python2.7 through Python3.4 | ||
1179 | 412 | exc_json = ValueError | ||
1180 | 413 | try: | ||
1181 | 414 | if _cache_config is None: | ||
1182 | 415 | config_data = json.loads( | ||
1183 | 416 | subprocess.check_output(config_cmd_line).decode('UTF-8')) | ||
1184 | 417 | _cache_config = Config(config_data) | ||
1185 | 341 | if scope is not None: | 418 | if scope is not None: |
1189 | 342 | return config_data | 419 | return _cache_config.get(scope) |
1190 | 343 | return Config(config_data) | 420 | return _cache_config |
1191 | 344 | except ValueError: | 421 | except (exc_json, UnicodeDecodeError) as e: |
1192 | 422 | log('Unable to parse output from config-get: config_cmd_line="{}" ' | ||
1193 | 423 | 'message="{}"' | ||
1194 | 424 | .format(config_cmd_line, str(e)), level=ERROR) | ||
1195 | 345 | return None | 425 | return None |
1196 | 346 | 426 | ||
1197 | 347 | 427 | ||
1198 | @@ -435,6 +515,67 @@ | |||
1199 | 435 | subprocess.check_output(units_cmd_line).decode('UTF-8')) or [] | 515 | subprocess.check_output(units_cmd_line).decode('UTF-8')) or [] |
1200 | 436 | 516 | ||
1201 | 437 | 517 | ||
1202 | 518 | def expected_peer_units(): | ||
1203 | 519 | """Get a generator for units we expect to join peer relation based on | ||
1204 | 520 | goal-state. | ||
1205 | 521 | |||
1206 | 522 | The local unit is excluded from the result to make it easy to gauge | ||
1207 | 523 | completion of all peers joining the relation with existing hook tools. | ||
1208 | 524 | |||
1209 | 525 | Example usage: | ||
1210 | 526 | log('peer {} of {} joined peer relation' | ||
1211 | 527 | .format(len(related_units()), | ||
1212 | 528 | len(list(expected_peer_units())))) | ||
1213 | 529 | |||
1214 | 530 | This function will raise NotImplementedError if used with juju versions | ||
1215 | 531 | without goal-state support. | ||
1216 | 532 | |||
1217 | 533 | :returns: iterator | ||
1218 | 534 | :rtype: types.GeneratorType | ||
1219 | 535 | :raises: NotImplementedError | ||
1220 | 536 | """ | ||
1221 | 537 | if not has_juju_version("2.4.0"): | ||
1222 | 538 | # goal-state first appeared in 2.4.0. | ||
1223 | 539 | raise NotImplementedError("goal-state") | ||
1224 | 540 | _goal_state = goal_state() | ||
1225 | 541 | return (key for key in _goal_state['units'] | ||
1226 | 542 | if '/' in key and key != local_unit()) | ||
1227 | 543 | |||
1228 | 544 | |||
1229 | 545 | def expected_related_units(reltype=None): | ||
1230 | 546 | """Get a generator for units we expect to join relation based on | ||
1231 | 547 | goal-state. | ||
1232 | 548 | |||
1233 | 549 | Note that you can not use this function for the peer relation, take a look | ||
1234 | 550 | at expected_peer_units() for that. | ||
1235 | 551 | |||
1236 | 552 | This function will raise KeyError if you request information for a | ||
1237 | 553 | relation type for which juju goal-state does not have information. It will | ||
1238 | 554 | raise NotImplementedError if used with juju versions without goal-state | ||
1239 | 555 | support. | ||
1240 | 556 | |||
1241 | 557 | Example usage: | ||
1242 | 558 | log('participant {} of {} joined relation {}' | ||
1243 | 559 | .format(len(related_units()), | ||
1244 | 560 | len(list(expected_related_units())), | ||
1245 | 561 | relation_type())) | ||
1246 | 562 | |||
1247 | 563 | :param reltype: Relation type to list data for, default is to list data for | ||
1248 | 564 | the realtion type we are currently executing a hook for. | ||
1249 | 565 | :type reltype: str | ||
1250 | 566 | :returns: iterator | ||
1251 | 567 | :rtype: types.GeneratorType | ||
1252 | 568 | :raises: KeyError, NotImplementedError | ||
1253 | 569 | """ | ||
1254 | 570 | if not has_juju_version("2.4.4"): | ||
1255 | 571 | # goal-state existed in 2.4.0, but did not list individual units to | ||
1256 | 572 | # join a relation in 2.4.1 through 2.4.3. (LP: #1794739) | ||
1257 | 573 | raise NotImplementedError("goal-state relation unit count") | ||
1258 | 574 | reltype = reltype or relation_type() | ||
1259 | 575 | _goal_state = goal_state() | ||
1260 | 576 | return (key for key in _goal_state['relations'][reltype] if '/' in key) | ||
1261 | 577 | |||
1262 | 578 | |||
1263 | 438 | @cached | 579 | @cached |
1264 | 439 | def relation_for_unit(unit=None, rid=None): | 580 | def relation_for_unit(unit=None, rid=None): |
1265 | 440 | """Get the json represenation of a unit's relation""" | 581 | """Get the json represenation of a unit's relation""" |
1266 | @@ -478,6 +619,24 @@ | |||
1267 | 478 | return yaml.safe_load(md) | 619 | return yaml.safe_load(md) |
1268 | 479 | 620 | ||
1269 | 480 | 621 | ||
1270 | 622 | def _metadata_unit(unit): | ||
1271 | 623 | """Given the name of a unit (e.g. apache2/0), get the unit charm's | ||
1272 | 624 | metadata.yaml. Very similar to metadata() but allows us to inspect | ||
1273 | 625 | other units. Unit needs to be co-located, such as a subordinate or | ||
1274 | 626 | principal/primary. | ||
1275 | 627 | |||
1276 | 628 | :returns: metadata.yaml as a python object. | ||
1277 | 629 | |||
1278 | 630 | """ | ||
1279 | 631 | basedir = os.sep.join(charm_dir().split(os.sep)[:-2]) | ||
1280 | 632 | unitdir = 'unit-{}'.format(unit.replace(os.sep, '-')) | ||
1281 | 633 | joineddir = os.path.join(basedir, unitdir, 'charm', 'metadata.yaml') | ||
1282 | 634 | if not os.path.exists(joineddir): | ||
1283 | 635 | return None | ||
1284 | 636 | with open(joineddir) as md: | ||
1285 | 637 | return yaml.safe_load(md) | ||
1286 | 638 | |||
1287 | 639 | |||
1288 | 481 | @cached | 640 | @cached |
1289 | 482 | def relation_types(): | 641 | def relation_types(): |
1290 | 483 | """Get a list of relation types supported by this charm""" | 642 | """Get a list of relation types supported by this charm""" |
1291 | @@ -602,20 +761,58 @@ | |||
1292 | 602 | return False | 761 | return False |
1293 | 603 | 762 | ||
1294 | 604 | 763 | ||
1295 | 764 | def _port_op(op_name, port, protocol="TCP"): | ||
1296 | 765 | """Open or close a service network port""" | ||
1297 | 766 | _args = [op_name] | ||
1298 | 767 | icmp = protocol.upper() == "ICMP" | ||
1299 | 768 | if icmp: | ||
1300 | 769 | _args.append(protocol) | ||
1301 | 770 | else: | ||
1302 | 771 | _args.append('{}/{}'.format(port, protocol)) | ||
1303 | 772 | try: | ||
1304 | 773 | subprocess.check_call(_args) | ||
1305 | 774 | except subprocess.CalledProcessError: | ||
1306 | 775 | # Older Juju pre 2.3 doesn't support ICMP | ||
1307 | 776 | # so treat it as a no-op if it fails. | ||
1308 | 777 | if not icmp: | ||
1309 | 778 | raise | ||
1310 | 779 | |||
1311 | 780 | |||
1312 | 605 | def open_port(port, protocol="TCP"): | 781 | def open_port(port, protocol="TCP"): |
1313 | 606 | """Open a service network port""" | 782 | """Open a service network port""" |
1317 | 607 | _args = ['open-port'] | 783 | _port_op('open-port', port, protocol) |
1315 | 608 | _args.append('{}/{}'.format(port, protocol)) | ||
1316 | 609 | subprocess.check_call(_args) | ||
1318 | 610 | 784 | ||
1319 | 611 | 785 | ||
1320 | 612 | def close_port(port, protocol="TCP"): | 786 | def close_port(port, protocol="TCP"): |
1321 | 613 | """Close a service network port""" | 787 | """Close a service network port""" |
1322 | 788 | _port_op('close-port', port, protocol) | ||
1323 | 789 | |||
1324 | 790 | |||
1325 | 791 | def open_ports(start, end, protocol="TCP"): | ||
1326 | 792 | """Opens a range of service network ports""" | ||
1327 | 793 | _args = ['open-port'] | ||
1328 | 794 | _args.append('{}-{}/{}'.format(start, end, protocol)) | ||
1329 | 795 | subprocess.check_call(_args) | ||
1330 | 796 | |||
1331 | 797 | |||
1332 | 798 | def close_ports(start, end, protocol="TCP"): | ||
1333 | 799 | """Close a range of service network ports""" | ||
1334 | 614 | _args = ['close-port'] | 800 | _args = ['close-port'] |
1336 | 615 | _args.append('{}/{}'.format(port, protocol)) | 801 | _args.append('{}-{}/{}'.format(start, end, protocol)) |
1337 | 616 | subprocess.check_call(_args) | 802 | subprocess.check_call(_args) |
1338 | 617 | 803 | ||
1339 | 618 | 804 | ||
1340 | 805 | def opened_ports(): | ||
1341 | 806 | """Get the opened ports | ||
1342 | 807 | |||
1343 | 808 | *Note that this will only show ports opened in a previous hook* | ||
1344 | 809 | |||
1345 | 810 | :returns: Opened ports as a list of strings: ``['8080/tcp', '8081-8083/tcp']`` | ||
1346 | 811 | """ | ||
1347 | 812 | _args = ['opened-ports', '--format=json'] | ||
1348 | 813 | return json.loads(subprocess.check_output(_args).decode('UTF-8')) | ||
1349 | 814 | |||
1350 | 815 | |||
1351 | 619 | @cached | 816 | @cached |
1352 | 620 | def unit_get(attribute): | 817 | def unit_get(attribute): |
1353 | 621 | """Get the unit ID for the remote unit""" | 818 | """Get the unit ID for the remote unit""" |
1354 | @@ -737,8 +934,15 @@ | |||
1355 | 737 | return wrapper | 934 | return wrapper |
1356 | 738 | 935 | ||
1357 | 739 | 936 | ||
1358 | 937 | class NoNetworkBinding(Exception): | ||
1359 | 938 | pass | ||
1360 | 939 | |||
1361 | 940 | |||
1362 | 740 | def charm_dir(): | 941 | def charm_dir(): |
1363 | 741 | """Return the root directory of the current charm""" | 942 | """Return the root directory of the current charm""" |
1364 | 943 | d = os.environ.get('JUJU_CHARM_DIR') | ||
1365 | 944 | if d is not None: | ||
1366 | 945 | return d | ||
1367 | 742 | return os.environ.get('CHARM_DIR') | 946 | return os.environ.get('CHARM_DIR') |
1368 | 743 | 947 | ||
1369 | 744 | 948 | ||
1370 | @@ -845,6 +1049,28 @@ | |||
1371 | 845 | return inner_translate_exc1 | 1049 | return inner_translate_exc1 |
1372 | 846 | 1050 | ||
1373 | 847 | 1051 | ||
1374 | 1052 | def application_version_set(version): | ||
1375 | 1053 | """Charm authors may trigger this command from any hook to output what | ||
1376 | 1054 | version of the application is running. This could be a package version, | ||
1377 | 1055 | for instance postgres version 9.5. It could also be a build number or | ||
1378 | 1056 | version control revision identifier, for instance git sha 6fb7ba68. """ | ||
1379 | 1057 | |||
1380 | 1058 | cmd = ['application-version-set'] | ||
1381 | 1059 | cmd.append(version) | ||
1382 | 1060 | try: | ||
1383 | 1061 | subprocess.check_call(cmd) | ||
1384 | 1062 | except OSError: | ||
1385 | 1063 | log("Application Version: {}".format(version)) | ||
1386 | 1064 | |||
1387 | 1065 | |||
1388 | 1066 | @translate_exc(from_exc=OSError, to_exc=NotImplementedError) | ||
1389 | 1067 | @cached | ||
1390 | 1068 | def goal_state(): | ||
1391 | 1069 | """Juju goal state values""" | ||
1392 | 1070 | cmd = ['goal-state', '--format=json'] | ||
1393 | 1071 | return json.loads(subprocess.check_output(cmd).decode('UTF-8')) | ||
1394 | 1072 | |||
1395 | 1073 | |||
1396 | 848 | @translate_exc(from_exc=OSError, to_exc=NotImplementedError) | 1074 | @translate_exc(from_exc=OSError, to_exc=NotImplementedError) |
1397 | 849 | def is_leader(): | 1075 | def is_leader(): |
1398 | 850 | """Does the current unit hold the juju leadership | 1076 | """Does the current unit hold the juju leadership |
1399 | @@ -912,6 +1138,24 @@ | |||
1400 | 912 | subprocess.check_call(cmd) | 1138 | subprocess.check_call(cmd) |
1401 | 913 | 1139 | ||
1402 | 914 | 1140 | ||
1403 | 1141 | @translate_exc(from_exc=OSError, to_exc=NotImplementedError) | ||
1404 | 1142 | def resource_get(name): | ||
1405 | 1143 | """used to fetch the resource path of the given name. | ||
1406 | 1144 | |||
1407 | 1145 | <name> must match a name of defined resource in metadata.yaml | ||
1408 | 1146 | |||
1409 | 1147 | returns either a path or False if resource not available | ||
1410 | 1148 | """ | ||
1411 | 1149 | if not name: | ||
1412 | 1150 | return False | ||
1413 | 1151 | |||
1414 | 1152 | cmd = ['resource-get', name] | ||
1415 | 1153 | try: | ||
1416 | 1154 | return subprocess.check_output(cmd).decode('UTF-8') | ||
1417 | 1155 | except subprocess.CalledProcessError: | ||
1418 | 1156 | return False | ||
1419 | 1157 | |||
1420 | 1158 | |||
1421 | 915 | @cached | 1159 | @cached |
1422 | 916 | def juju_version(): | 1160 | def juju_version(): |
1423 | 917 | """Full version string (eg. '1.23.3.1-trusty-amd64')""" | 1161 | """Full version string (eg. '1.23.3.1-trusty-amd64')""" |
1424 | @@ -921,7 +1165,6 @@ | |||
1425 | 921 | universal_newlines=True).strip() | 1165 | universal_newlines=True).strip() |
1426 | 922 | 1166 | ||
1427 | 923 | 1167 | ||
1428 | 924 | @cached | ||
1429 | 925 | def has_juju_version(minimum_version): | 1168 | def has_juju_version(minimum_version): |
1430 | 926 | """Return True if the Juju version is at least the provided version""" | 1169 | """Return True if the Juju version is at least the provided version""" |
1431 | 927 | return LooseVersion(juju_version()) >= LooseVersion(minimum_version) | 1170 | return LooseVersion(juju_version()) >= LooseVersion(minimum_version) |
1432 | @@ -976,3 +1219,272 @@ | |||
1433 | 976 | for callback, args, kwargs in reversed(_atexit): | 1219 | for callback, args, kwargs in reversed(_atexit): |
1434 | 977 | callback(*args, **kwargs) | 1220 | callback(*args, **kwargs) |
1435 | 978 | del _atexit[:] | 1221 | del _atexit[:] |
1436 | 1222 | |||
1437 | 1223 | |||
1438 | 1224 | @translate_exc(from_exc=OSError, to_exc=NotImplementedError) | ||
1439 | 1225 | def network_get_primary_address(binding): | ||
1440 | 1226 | ''' | ||
1441 | 1227 | Deprecated since Juju 2.3; use network_get() | ||
1442 | 1228 | |||
1443 | 1229 | Retrieve the primary network address for a named binding | ||
1444 | 1230 | |||
1445 | 1231 | :param binding: string. The name of a relation of extra-binding | ||
1446 | 1232 | :return: string. The primary IP address for the named binding | ||
1447 | 1233 | :raise: NotImplementedError if run on Juju < 2.0 | ||
1448 | 1234 | ''' | ||
1449 | 1235 | cmd = ['network-get', '--primary-address', binding] | ||
1450 | 1236 | try: | ||
1451 | 1237 | response = subprocess.check_output( | ||
1452 | 1238 | cmd, | ||
1453 | 1239 | stderr=subprocess.STDOUT).decode('UTF-8').strip() | ||
1454 | 1240 | except CalledProcessError as e: | ||
1455 | 1241 | if 'no network config found for binding' in e.output.decode('UTF-8'): | ||
1456 | 1242 | raise NoNetworkBinding("No network binding for {}" | ||
1457 | 1243 | .format(binding)) | ||
1458 | 1244 | else: | ||
1459 | 1245 | raise | ||
1460 | 1246 | return response | ||
1461 | 1247 | |||
1462 | 1248 | |||
1463 | 1249 | def network_get(endpoint, relation_id=None): | ||
1464 | 1250 | """ | ||
1465 | 1251 | Retrieve the network details for a relation endpoint | ||
1466 | 1252 | |||
1467 | 1253 | :param endpoint: string. The name of a relation endpoint | ||
1468 | 1254 | :param relation_id: int. The ID of the relation for the current context. | ||
1469 | 1255 | :return: dict. The loaded YAML output of the network-get query. | ||
1470 | 1256 | :raise: NotImplementedError if request not supported by the Juju version. | ||
1471 | 1257 | """ | ||
1472 | 1258 | if not has_juju_version('2.2'): | ||
1473 | 1259 | raise NotImplementedError(juju_version()) # earlier versions require --primary-address | ||
1474 | 1260 | if relation_id and not has_juju_version('2.3'): | ||
1475 | 1261 | raise NotImplementedError # 2.3 added the -r option | ||
1476 | 1262 | |||
1477 | 1263 | cmd = ['network-get', endpoint, '--format', 'yaml'] | ||
1478 | 1264 | if relation_id: | ||
1479 | 1265 | cmd.append('-r') | ||
1480 | 1266 | cmd.append(relation_id) | ||
1481 | 1267 | response = subprocess.check_output( | ||
1482 | 1268 | cmd, | ||
1483 | 1269 | stderr=subprocess.STDOUT).decode('UTF-8').strip() | ||
1484 | 1270 | return yaml.safe_load(response) | ||
1485 | 1271 | |||
1486 | 1272 | |||
1487 | 1273 | def add_metric(*args, **kwargs): | ||
1488 | 1274 | """Add metric values. Values may be expressed with keyword arguments. For | ||
1489 | 1275 | metric names containing dashes, these may be expressed as one or more | ||
1490 | 1276 | 'key=value' positional arguments. May only be called from the collect-metrics | ||
1491 | 1277 | hook.""" | ||
1492 | 1278 | _args = ['add-metric'] | ||
1493 | 1279 | _kvpairs = [] | ||
1494 | 1280 | _kvpairs.extend(args) | ||
1495 | 1281 | _kvpairs.extend(['{}={}'.format(k, v) for k, v in kwargs.items()]) | ||
1496 | 1282 | _args.extend(sorted(_kvpairs)) | ||
1497 | 1283 | try: | ||
1498 | 1284 | subprocess.check_call(_args) | ||
1499 | 1285 | return | ||
1500 | 1286 | except EnvironmentError as e: | ||
1501 | 1287 | if e.errno != errno.ENOENT: | ||
1502 | 1288 | raise | ||
1503 | 1289 | log_message = 'add-metric failed: {}'.format(' '.join(_kvpairs)) | ||
1504 | 1290 | log(log_message, level='INFO') | ||
1505 | 1291 | |||
1506 | 1292 | |||
1507 | 1293 | def meter_status(): | ||
1508 | 1294 | """Get the meter status, if running in the meter-status-changed hook.""" | ||
1509 | 1295 | return os.environ.get('JUJU_METER_STATUS') | ||
1510 | 1296 | |||
1511 | 1297 | |||
1512 | 1298 | def meter_info(): | ||
1513 | 1299 | """Get the meter status information, if running in the meter-status-changed | ||
1514 | 1300 | hook.""" | ||
1515 | 1301 | return os.environ.get('JUJU_METER_INFO') | ||
1516 | 1302 | |||
1517 | 1303 | |||
1518 | 1304 | def iter_units_for_relation_name(relation_name): | ||
1519 | 1305 | """Iterate through all units in a relation | ||
1520 | 1306 | |||
1521 | 1307 | Generator that iterates through all the units in a relation and yields | ||
1522 | 1308 | a named tuple with rid and unit field names. | ||
1523 | 1309 | |||
1524 | 1310 | Usage: | ||
1525 | 1311 | data = [(u.rid, u.unit) | ||
1526 | 1312 | for u in iter_units_for_relation_name(relation_name)] | ||
1527 | 1313 | |||
1528 | 1314 | :param relation_name: string relation name | ||
1529 | 1315 | :yield: Named Tuple with rid and unit field names | ||
1530 | 1316 | """ | ||
1531 | 1317 | RelatedUnit = namedtuple('RelatedUnit', 'rid, unit') | ||
1532 | 1318 | for rid in relation_ids(relation_name): | ||
1533 | 1319 | for unit in related_units(rid): | ||
1534 | 1320 | yield RelatedUnit(rid, unit) | ||
1535 | 1321 | |||
1536 | 1322 | |||
1537 | 1323 | def ingress_address(rid=None, unit=None): | ||
1538 | 1324 | """ | ||
1539 | 1325 | Retrieve the ingress-address from a relation when available. | ||
1540 | 1326 | Otherwise, return the private-address. | ||
1541 | 1327 | |||
1542 | 1328 | When used on the consuming side of the relation (unit is a remote | ||
1543 | 1329 | unit), the ingress-address is the IP address that this unit needs | ||
1544 | 1330 | to use to reach the provided service on the remote unit. | ||
1545 | 1331 | |||
1546 | 1332 | When used on the providing side of the relation (unit == local_unit()), | ||
1547 | 1333 | the ingress-address is the IP address that is advertised to remote | ||
1548 | 1334 | units on this relation. Remote units need to use this address to | ||
1549 | 1335 | reach the local provided service on this unit. | ||
1550 | 1336 | |||
1551 | 1337 | Note that charms may document some other method to use in | ||
1552 | 1338 | preference to the ingress_address(), such as an address provided | ||
1553 | 1339 | on a different relation attribute or a service discovery mechanism. | ||
1554 | 1340 | This allows charms to redirect inbound connections to their peers | ||
1555 | 1341 | or different applications such as load balancers. | ||
1556 | 1342 | |||
1557 | 1343 | Usage: | ||
1558 | 1344 | addresses = [ingress_address(rid=u.rid, unit=u.unit) | ||
1559 | 1345 | for u in iter_units_for_relation_name(relation_name)] | ||
1560 | 1346 | |||
1561 | 1347 | :param rid: string relation id | ||
1562 | 1348 | :param unit: string unit name | ||
1563 | 1349 | :side effect: calls relation_get | ||
1564 | 1350 | :return: string IP address | ||
1565 | 1351 | """ | ||
1566 | 1352 | settings = relation_get(rid=rid, unit=unit) | ||
1567 | 1353 | return (settings.get('ingress-address') or | ||
1568 | 1354 | settings.get('private-address')) | ||
1569 | 1355 | |||
1570 | 1356 | |||
1571 | 1357 | def egress_subnets(rid=None, unit=None): | ||
1572 | 1358 | """ | ||
1573 | 1359 | Retrieve the egress-subnets from a relation. | ||
1574 | 1360 | |||
1575 | 1361 | This function is to be used on the providing side of the | ||
1576 | 1362 | relation, and provides the ranges of addresses that client | ||
1577 | 1363 | connections may come from. The result is uninteresting on | ||
1578 | 1364 | the consuming side of a relation (unit == local_unit()). | ||
1579 | 1365 | |||
1580 | 1366 | Returns a stable list of subnets in CIDR format. | ||
1581 | 1367 | eg. ['192.168.1.0/24', '2001::F00F/128'] | ||
1582 | 1368 | |||
1583 | 1369 | If egress-subnets is not available, falls back to using the published | ||
1584 | 1370 | ingress-address, or finally private-address. | ||
1585 | 1371 | |||
1586 | 1372 | :param rid: string relation id | ||
1587 | 1373 | :param unit: string unit name | ||
1588 | 1374 | :side effect: calls relation_get | ||
1589 | 1375 | :return: list of subnets in CIDR format. eg. ['192.168.1.0/24', '2001::F00F/128'] | ||
1590 | 1376 | """ | ||
1591 | 1377 | def _to_range(addr): | ||
1592 | 1378 | if re.search(r'^(?:\d{1,3}\.){3}\d{1,3}$', addr) is not None: | ||
1593 | 1379 | addr += '/32' | ||
1594 | 1380 | elif ':' in addr and '/' not in addr: # IPv6 | ||
1595 | 1381 | addr += '/128' | ||
1596 | 1382 | return addr | ||
1597 | 1383 | |||
1598 | 1384 | settings = relation_get(rid=rid, unit=unit) | ||
1599 | 1385 | if 'egress-subnets' in settings: | ||
1600 | 1386 | return [n.strip() for n in settings['egress-subnets'].split(',') if n.strip()] | ||
1601 | 1387 | if 'ingress-address' in settings: | ||
1602 | 1388 | return [_to_range(settings['ingress-address'])] | ||
1603 | 1389 | if 'private-address' in settings: | ||
1604 | 1390 | return [_to_range(settings['private-address'])] | ||
1605 | 1391 | return [] # Should never happen | ||
1606 | 1392 | |||
1607 | 1393 | |||
1608 | 1394 | def unit_doomed(unit=None): | ||
1609 | 1395 | """Determines if the unit is being removed from the model | ||
1610 | 1396 | |||
1611 | 1397 | Requires Juju 2.4.1. | ||
1612 | 1398 | |||
1613 | 1399 | :param unit: string unit name, defaults to local_unit | ||
1614 | 1400 | :side effect: calls goal_state | ||
1615 | 1401 | :side effect: calls local_unit | ||
1616 | 1402 | :side effect: calls has_juju_version | ||
1617 | 1403 | :return: True if the unit is being removed, already gone, or never existed | ||
1618 | 1404 | """ | ||
1619 | 1405 | if not has_juju_version("2.4.1"): | ||
1620 | 1406 | # We cannot risk blindly returning False for 'we don't know', | ||
1621 | 1407 | # because that could cause data loss; if call sites don't | ||
1622 | 1408 | # need an accurate answer, they likely don't need this helper | ||
1623 | 1409 | # at all. | ||
1624 | 1410 | # goal-state existed in 2.4.0, but did not handle removals | ||
1625 | 1411 | # correctly until 2.4.1. | ||
1626 | 1412 | raise NotImplementedError("is_doomed") | ||
1627 | 1413 | if unit is None: | ||
1628 | 1414 | unit = local_unit() | ||
1629 | 1415 | gs = goal_state() | ||
1630 | 1416 | units = gs.get('units', {}) | ||
1631 | 1417 | if unit not in units: | ||
1632 | 1418 | return True | ||
1633 | 1419 | # I don't think 'dead' units ever show up in the goal-state, but | ||
1634 | 1420 | # check anyway in addition to 'dying'. | ||
1635 | 1421 | return units[unit]['status'] in ('dying', 'dead') | ||
1636 | 1422 | |||
1637 | 1423 | |||
1638 | 1424 | def env_proxy_settings(selected_settings=None): | ||
1639 | 1425 | """Get proxy settings from process environment variables. | ||
1640 | 1426 | |||
1641 | 1427 | Get charm proxy settings from environment variables that correspond to | ||
1642 | 1428 | juju-http-proxy, juju-https-proxy and juju-no-proxy (available as of 2.4.2, | ||
1643 | 1429 | see lp:1782236) in a format suitable for passing to an application that | ||
1644 | 1430 | reacts to proxy settings passed as environment variables. Some applications | ||
1645 | 1431 | support lowercase or uppercase notation (e.g. curl), some support only | ||
1646 | 1432 | lowercase (e.g. wget), there are also subjectively rare cases of only | ||
1647 | 1433 | uppercase notation support. no_proxy CIDR and wildcard support also varies | ||
1648 | 1434 | between runtimes and applications as there is no enforced standard. | ||
1649 | 1435 | |||
1650 | 1436 | Some applications may connect to multiple destinations and expose config | ||
1651 | 1437 | options that would affect only proxy settings for a specific destination | ||
1652 | 1438 | these should be handled in charms in an application-specific manner. | ||
1653 | 1439 | |||
1654 | 1440 | :param selected_settings: format only a subset of possible settings | ||
1655 | 1441 | :type selected_settings: list | ||
1656 | 1442 | :rtype: Option(None, dict[str, str]) | ||
1657 | 1443 | """ | ||
1658 | 1444 | SUPPORTED_SETTINGS = { | ||
1659 | 1445 | 'http': 'HTTP_PROXY', | ||
1660 | 1446 | 'https': 'HTTPS_PROXY', | ||
1661 | 1447 | 'no_proxy': 'NO_PROXY', | ||
1662 | 1448 | 'ftp': 'FTP_PROXY' | ||
1663 | 1449 | } | ||
1664 | 1450 | if selected_settings is None: | ||
1665 | 1451 | selected_settings = SUPPORTED_SETTINGS | ||
1666 | 1452 | |||
1667 | 1453 | selected_vars = [v for k, v in SUPPORTED_SETTINGS.items() | ||
1668 | 1454 | if k in selected_settings] | ||
1669 | 1455 | proxy_settings = {} | ||
1670 | 1456 | for var in selected_vars: | ||
1671 | 1457 | var_val = os.getenv(var) | ||
1672 | 1458 | if var_val: | ||
1673 | 1459 | proxy_settings[var] = var_val | ||
1674 | 1460 | proxy_settings[var.lower()] = var_val | ||
1675 | 1461 | # Now handle juju-prefixed environment variables. The legacy vs new | ||
1676 | 1462 | # environment variable usage is mutually exclusive | ||
1677 | 1463 | charm_var_val = os.getenv('JUJU_CHARM_{}'.format(var)) | ||
1678 | 1464 | if charm_var_val: | ||
1679 | 1465 | proxy_settings[var] = charm_var_val | ||
1680 | 1466 | proxy_settings[var.lower()] = charm_var_val | ||
1681 | 1467 | if 'no_proxy' in proxy_settings: | ||
1682 | 1468 | if _contains_range(proxy_settings['no_proxy']): | ||
1683 | 1469 | log(RANGE_WARNING, level=WARNING) | ||
1684 | 1470 | return proxy_settings if proxy_settings else None | ||
1685 | 1471 | |||
1686 | 1472 | |||
1687 | 1473 | def _contains_range(addresses): | ||
1688 | 1474 | """Check for cidr or wildcard domain in a string. | ||
1689 | 1475 | |||
1690 | 1476 | Given a string comprising a comma seperated list of ip addresses | ||
1691 | 1477 | and domain names, determine whether the string contains IP ranges | ||
1692 | 1478 | or wildcard domains. | ||
1693 | 1479 | |||
1694 | 1480 | :param addresses: comma seperated list of domains and ip addresses. | ||
1695 | 1481 | :type addresses: str | ||
1696 | 1482 | """ | ||
1697 | 1483 | return ( | ||
1698 | 1484 | # Test for cidr (e.g. 10.20.20.0/24) | ||
1699 | 1485 | "/" in addresses or | ||
1700 | 1486 | # Test for wildcard domains (*.foo.com or .foo.com) | ||
1701 | 1487 | "*" in addresses or | ||
1702 | 1488 | addresses.startswith(".") or | ||
1703 | 1489 | ",." in addresses or | ||
1704 | 1490 | " ." in addresses) | ||
1705 | 979 | 1491 | ||
1706 | === modified file 'charmhelpers/core/host.py' | |||
1707 | --- charmhelpers/core/host.py 2015-12-11 15:23:38 +0000 | |||
1708 | +++ charmhelpers/core/host.py 2021-04-13 19:16:05 +0000 | |||
1709 | @@ -1,18 +1,16 @@ | |||
1710 | 1 | # Copyright 2014-2015 Canonical Limited. | 1 | # Copyright 2014-2015 Canonical Limited. |
1711 | 2 | # | 2 | # |
1725 | 3 | # This file is part of charm-helpers. | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); |
1726 | 4 | # | 4 | # you may not use this file except in compliance with the License. |
1727 | 5 | # charm-helpers is free software: you can redistribute it and/or modify | 5 | # You may obtain a copy of the License at |
1728 | 6 | # it under the terms of the GNU Lesser General Public License version 3 as | 6 | # |
1729 | 7 | # published by the Free Software Foundation. | 7 | # http://www.apache.org/licenses/LICENSE-2.0 |
1730 | 8 | # | 8 | # |
1731 | 9 | # charm-helpers is distributed in the hope that it will be useful, | 9 | # Unless required by applicable law or agreed to in writing, software |
1732 | 10 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 10 | # distributed under the License is distributed on an "AS IS" BASIS, |
1733 | 11 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
1734 | 12 | # GNU Lesser General Public License for more details. | 12 | # See the License for the specific language governing permissions and |
1735 | 13 | # | 13 | # limitations under the License. |
1723 | 14 | # You should have received a copy of the GNU Lesser General Public License | ||
1724 | 15 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
1736 | 16 | 14 | ||
1737 | 17 | """Tools for working with the host system""" | 15 | """Tools for working with the host system""" |
1738 | 18 | # Copyright 2012 Canonical Ltd. | 16 | # Copyright 2012 Canonical Ltd. |
1739 | @@ -30,49 +28,175 @@ | |||
1740 | 30 | import string | 28 | import string |
1741 | 31 | import subprocess | 29 | import subprocess |
1742 | 32 | import hashlib | 30 | import hashlib |
1743 | 31 | import functools | ||
1744 | 32 | import itertools | ||
1745 | 33 | import six | ||
1746 | 34 | |||
1747 | 33 | from contextlib import contextmanager | 35 | from contextlib import contextmanager |
1748 | 34 | from collections import OrderedDict | 36 | from collections import OrderedDict |
1753 | 35 | 37 | from .hookenv import log, INFO, DEBUG, local_unit, charm_name | |
1750 | 36 | import six | ||
1751 | 37 | |||
1752 | 38 | from .hookenv import log | ||
1754 | 39 | from .fstab import Fstab | 38 | from .fstab import Fstab |
1769 | 40 | 39 | from charmhelpers.osplatform import get_platform | |
1770 | 41 | 40 | ||
1771 | 42 | def service_start(service_name): | 41 | __platform__ = get_platform() |
1772 | 43 | """Start a system service""" | 42 | if __platform__ == "ubuntu": |
1773 | 44 | return service('start', service_name) | 43 | from charmhelpers.core.host_factory.ubuntu import ( # NOQA:F401 |
1774 | 45 | 44 | service_available, | |
1775 | 46 | 45 | add_new_group, | |
1776 | 47 | def service_stop(service_name): | 46 | lsb_release, |
1777 | 48 | """Stop a system service""" | 47 | cmp_pkgrevno, |
1778 | 49 | return service('stop', service_name) | 48 | CompareHostReleases, |
1779 | 50 | 49 | get_distrib_codename, | |
1780 | 51 | 50 | arch | |
1781 | 52 | def service_restart(service_name): | 51 | ) # flake8: noqa -- ignore F401 for this import |
1782 | 53 | """Restart a system service""" | 52 | elif __platform__ == "centos": |
1783 | 53 | from charmhelpers.core.host_factory.centos import ( # NOQA:F401 | ||
1784 | 54 | service_available, | ||
1785 | 55 | add_new_group, | ||
1786 | 56 | lsb_release, | ||
1787 | 57 | cmp_pkgrevno, | ||
1788 | 58 | CompareHostReleases, | ||
1789 | 59 | ) # flake8: noqa -- ignore F401 for this import | ||
1790 | 60 | |||
1791 | 61 | UPDATEDB_PATH = '/etc/updatedb.conf' | ||
1792 | 62 | |||
1793 | 63 | |||
1794 | 64 | def service_start(service_name, **kwargs): | ||
1795 | 65 | """Start a system service. | ||
1796 | 66 | |||
1797 | 67 | The specified service name is managed via the system level init system. | ||
1798 | 68 | Some init systems (e.g. upstart) require that additional arguments be | ||
1799 | 69 | provided in order to directly control service instances whereas other init | ||
1800 | 70 | systems allow for addressing instances of a service directly by name (e.g. | ||
1801 | 71 | systemd). | ||
1802 | 72 | |||
1803 | 73 | The kwargs allow for the additional parameters to be passed to underlying | ||
1804 | 74 | init systems for those systems which require/allow for them. For example, | ||
1805 | 75 | the ceph-osd upstart script requires the id parameter to be passed along | ||
1806 | 76 | in order to identify which running daemon should be reloaded. The follow- | ||
1807 | 77 | ing example stops the ceph-osd service for instance id=4: | ||
1808 | 78 | |||
1809 | 79 | service_stop('ceph-osd', id=4) | ||
1810 | 80 | |||
1811 | 81 | :param service_name: the name of the service to stop | ||
1812 | 82 | :param **kwargs: additional parameters to pass to the init system when | ||
1813 | 83 | managing services. These will be passed as key=value | ||
1814 | 84 | parameters to the init system's commandline. kwargs | ||
1815 | 85 | are ignored for systemd enabled systems. | ||
1816 | 86 | """ | ||
1817 | 87 | return service('start', service_name, **kwargs) | ||
1818 | 88 | |||
1819 | 89 | |||
1820 | 90 | def service_stop(service_name, **kwargs): | ||
1821 | 91 | """Stop a system service. | ||
1822 | 92 | |||
1823 | 93 | The specified service name is managed via the system level init system. | ||
1824 | 94 | Some init systems (e.g. upstart) require that additional arguments be | ||
1825 | 95 | provided in order to directly control service instances whereas other init | ||
1826 | 96 | systems allow for addressing instances of a service directly by name (e.g. | ||
1827 | 97 | systemd). | ||
1828 | 98 | |||
1829 | 99 | The kwargs allow for the additional parameters to be passed to underlying | ||
1830 | 100 | init systems for those systems which require/allow for them. For example, | ||
1831 | 101 | the ceph-osd upstart script requires the id parameter to be passed along | ||
1832 | 102 | in order to identify which running daemon should be reloaded. The follow- | ||
1833 | 103 | ing example stops the ceph-osd service for instance id=4: | ||
1834 | 104 | |||
1835 | 105 | service_stop('ceph-osd', id=4) | ||
1836 | 106 | |||
1837 | 107 | :param service_name: the name of the service to stop | ||
1838 | 108 | :param **kwargs: additional parameters to pass to the init system when | ||
1839 | 109 | managing services. These will be passed as key=value | ||
1840 | 110 | parameters to the init system's commandline. kwargs | ||
1841 | 111 | are ignored for systemd enabled systems. | ||
1842 | 112 | """ | ||
1843 | 113 | return service('stop', service_name, **kwargs) | ||
1844 | 114 | |||
1845 | 115 | |||
1846 | 116 | def service_restart(service_name, **kwargs): | ||
1847 | 117 | """Restart a system service. | ||
1848 | 118 | |||
1849 | 119 | The specified service name is managed via the system level init system. | ||
1850 | 120 | Some init systems (e.g. upstart) require that additional arguments be | ||
1851 | 121 | provided in order to directly control service instances whereas other init | ||
1852 | 122 | systems allow for addressing instances of a service directly by name (e.g. | ||
1853 | 123 | systemd). | ||
1854 | 124 | |||
1855 | 125 | The kwargs allow for the additional parameters to be passed to underlying | ||
1856 | 126 | init systems for those systems which require/allow for them. For example, | ||
1857 | 127 | the ceph-osd upstart script requires the id parameter to be passed along | ||
1858 | 128 | in order to identify which running daemon should be restarted. The follow- | ||
1859 | 129 | ing example restarts the ceph-osd service for instance id=4: | ||
1860 | 130 | |||
1861 | 131 | service_restart('ceph-osd', id=4) | ||
1862 | 132 | |||
1863 | 133 | :param service_name: the name of the service to restart | ||
1864 | 134 | :param **kwargs: additional parameters to pass to the init system when | ||
1865 | 135 | managing services. These will be passed as key=value | ||
1866 | 136 | parameters to the init system's commandline. kwargs | ||
1867 | 137 | are ignored for init systems not allowing additional | ||
1868 | 138 | parameters via the commandline (systemd). | ||
1869 | 139 | """ | ||
1870 | 54 | return service('restart', service_name) | 140 | return service('restart', service_name) |
1871 | 55 | 141 | ||
1872 | 56 | 142 | ||
1874 | 57 | def service_reload(service_name, restart_on_failure=False): | 143 | def service_reload(service_name, restart_on_failure=False, **kwargs): |
1875 | 58 | """Reload a system service, optionally falling back to restart if | 144 | """Reload a system service, optionally falling back to restart if |
1878 | 59 | reload fails""" | 145 | reload fails. |
1879 | 60 | service_result = service('reload', service_name) | 146 | |
1880 | 147 | The specified service name is managed via the system level init system. | ||
1881 | 148 | Some init systems (e.g. upstart) require that additional arguments be | ||
1882 | 149 | provided in order to directly control service instances whereas other init | ||
1883 | 150 | systems allow for addressing instances of a service directly by name (e.g. | ||
1884 | 151 | systemd). | ||
1885 | 152 | |||
1886 | 153 | The kwargs allow for the additional parameters to be passed to underlying | ||
1887 | 154 | init systems for those systems which require/allow for them. For example, | ||
1888 | 155 | the ceph-osd upstart script requires the id parameter to be passed along | ||
1889 | 156 | in order to identify which running daemon should be reloaded. The follow- | ||
1890 | 157 | ing example restarts the ceph-osd service for instance id=4: | ||
1891 | 158 | |||
1892 | 159 | service_reload('ceph-osd', id=4) | ||
1893 | 160 | |||
1894 | 161 | :param service_name: the name of the service to reload | ||
1895 | 162 | :param restart_on_failure: boolean indicating whether to fallback to a | ||
1896 | 163 | restart if the reload fails. | ||
1897 | 164 | :param **kwargs: additional parameters to pass to the init system when | ||
1898 | 165 | managing services. These will be passed as key=value | ||
1899 | 166 | parameters to the init system's commandline. kwargs | ||
1900 | 167 | are ignored for init systems not allowing additional | ||
1901 | 168 | parameters via the commandline (systemd). | ||
1902 | 169 | """ | ||
1903 | 170 | service_result = service('reload', service_name, **kwargs) | ||
1904 | 61 | if not service_result and restart_on_failure: | 171 | if not service_result and restart_on_failure: |
1906 | 62 | service_result = service('restart', service_name) | 172 | service_result = service('restart', service_name, **kwargs) |
1907 | 63 | return service_result | 173 | return service_result |
1908 | 64 | 174 | ||
1909 | 65 | 175 | ||
1911 | 66 | def service_pause(service_name, init_dir="/etc/init", initd_dir="/etc/init.d"): | 176 | def service_pause(service_name, init_dir="/etc/init", initd_dir="/etc/init.d", |
1912 | 177 | **kwargs): | ||
1913 | 67 | """Pause a system service. | 178 | """Pause a system service. |
1914 | 68 | 179 | ||
1916 | 69 | Stop it, and prevent it from starting again at boot.""" | 180 | Stop it, and prevent it from starting again at boot. |
1917 | 181 | |||
1918 | 182 | :param service_name: the name of the service to pause | ||
1919 | 183 | :param init_dir: path to the upstart init directory | ||
1920 | 184 | :param initd_dir: path to the sysv init directory | ||
1921 | 185 | :param **kwargs: additional parameters to pass to the init system when | ||
1922 | 186 | managing services. These will be passed as key=value | ||
1923 | 187 | parameters to the init system's commandline. kwargs | ||
1924 | 188 | are ignored for init systems which do not support | ||
1925 | 189 | key=value arguments via the commandline. | ||
1926 | 190 | """ | ||
1927 | 70 | stopped = True | 191 | stopped = True |
1930 | 71 | if service_running(service_name): | 192 | if service_running(service_name, **kwargs): |
1931 | 72 | stopped = service_stop(service_name) | 193 | stopped = service_stop(service_name, **kwargs) |
1932 | 73 | upstart_file = os.path.join(init_dir, "{}.conf".format(service_name)) | 194 | upstart_file = os.path.join(init_dir, "{}.conf".format(service_name)) |
1933 | 74 | sysv_file = os.path.join(initd_dir, service_name) | 195 | sysv_file = os.path.join(initd_dir, service_name) |
1935 | 75 | if os.path.exists(upstart_file): | 196 | if init_is_systemd(): |
1936 | 197 | service('disable', service_name) | ||
1937 | 198 | service('mask', service_name) | ||
1938 | 199 | elif os.path.exists(upstart_file): | ||
1939 | 76 | override_path = os.path.join( | 200 | override_path = os.path.join( |
1940 | 77 | init_dir, '{}.override'.format(service_name)) | 201 | init_dir, '{}.override'.format(service_name)) |
1941 | 78 | with open(override_path, 'w') as fh: | 202 | with open(override_path, 'w') as fh: |
1942 | @@ -80,21 +204,33 @@ | |||
1943 | 80 | elif os.path.exists(sysv_file): | 204 | elif os.path.exists(sysv_file): |
1944 | 81 | subprocess.check_call(["update-rc.d", service_name, "disable"]) | 205 | subprocess.check_call(["update-rc.d", service_name, "disable"]) |
1945 | 82 | else: | 206 | else: |
1946 | 83 | # XXX: Support SystemD too | ||
1947 | 84 | raise ValueError( | 207 | raise ValueError( |
1949 | 85 | "Unable to detect {0} as either Upstart {1} or SysV {2}".format( | 208 | "Unable to detect {0} as SystemD, Upstart {1} or" |
1950 | 209 | " SysV {2}".format( | ||
1951 | 86 | service_name, upstart_file, sysv_file)) | 210 | service_name, upstart_file, sysv_file)) |
1952 | 87 | return stopped | 211 | return stopped |
1953 | 88 | 212 | ||
1954 | 89 | 213 | ||
1955 | 90 | def service_resume(service_name, init_dir="/etc/init", | 214 | def service_resume(service_name, init_dir="/etc/init", |
1957 | 91 | initd_dir="/etc/init.d"): | 215 | initd_dir="/etc/init.d", **kwargs): |
1958 | 92 | """Resume a system service. | 216 | """Resume a system service. |
1959 | 93 | 217 | ||
1961 | 94 | Reenable starting again at boot. Start the service""" | 218 | Reenable starting again at boot. Start the service. |
1962 | 219 | |||
1963 | 220 | :param service_name: the name of the service to resume | ||
1964 | 221 | :param init_dir: the path to the init dir | ||
1965 | 222 | :param initd dir: the path to the initd dir | ||
1966 | 223 | :param **kwargs: additional parameters to pass to the init system when | ||
1967 | 224 | managing services. These will be passed as key=value | ||
1968 | 225 | parameters to the init system's commandline. kwargs | ||
1969 | 226 | are ignored for systemd enabled systems. | ||
1970 | 227 | """ | ||
1971 | 95 | upstart_file = os.path.join(init_dir, "{}.conf".format(service_name)) | 228 | upstart_file = os.path.join(init_dir, "{}.conf".format(service_name)) |
1972 | 96 | sysv_file = os.path.join(initd_dir, service_name) | 229 | sysv_file = os.path.join(initd_dir, service_name) |
1974 | 97 | if os.path.exists(upstart_file): | 230 | if init_is_systemd(): |
1975 | 231 | service('unmask', service_name) | ||
1976 | 232 | service('enable', service_name) | ||
1977 | 233 | elif os.path.exists(upstart_file): | ||
1978 | 98 | override_path = os.path.join( | 234 | override_path = os.path.join( |
1979 | 99 | init_dir, '{}.override'.format(service_name)) | 235 | init_dir, '{}.override'.format(service_name)) |
1980 | 100 | if os.path.exists(override_path): | 236 | if os.path.exists(override_path): |
1981 | @@ -102,54 +238,90 @@ | |||
1982 | 102 | elif os.path.exists(sysv_file): | 238 | elif os.path.exists(sysv_file): |
1983 | 103 | subprocess.check_call(["update-rc.d", service_name, "enable"]) | 239 | subprocess.check_call(["update-rc.d", service_name, "enable"]) |
1984 | 104 | else: | 240 | else: |
1985 | 105 | # XXX: Support SystemD too | ||
1986 | 106 | raise ValueError( | 241 | raise ValueError( |
1988 | 107 | "Unable to detect {0} as either Upstart {1} or SysV {2}".format( | 242 | "Unable to detect {0} as SystemD, Upstart {1} or" |
1989 | 243 | " SysV {2}".format( | ||
1990 | 108 | service_name, upstart_file, sysv_file)) | 244 | service_name, upstart_file, sysv_file)) |
1991 | 245 | started = service_running(service_name, **kwargs) | ||
1992 | 109 | 246 | ||
1993 | 110 | started = service_running(service_name) | ||
1994 | 111 | if not started: | 247 | if not started: |
1996 | 112 | started = service_start(service_name) | 248 | started = service_start(service_name, **kwargs) |
1997 | 113 | return started | 249 | return started |
1998 | 114 | 250 | ||
1999 | 115 | 251 | ||
2003 | 116 | def service(action, service_name): | 252 | def service(action, service_name, **kwargs): |
2004 | 117 | """Control a system service""" | 253 | """Control a system service. |
2005 | 118 | cmd = ['service', service_name, action] | 254 | |
2006 | 255 | :param action: the action to take on the service | ||
2007 | 256 | :param service_name: the name of the service to perform th action on | ||
2008 | 257 | :param **kwargs: additional params to be passed to the service command in | ||
2009 | 258 | the form of key=value. | ||
2010 | 259 | """ | ||
2011 | 260 | if init_is_systemd(): | ||
2012 | 261 | cmd = ['systemctl', action, service_name] | ||
2013 | 262 | else: | ||
2014 | 263 | cmd = ['service', service_name, action] | ||
2015 | 264 | for key, value in six.iteritems(kwargs): | ||
2016 | 265 | parameter = '%s=%s' % (key, value) | ||
2017 | 266 | cmd.append(parameter) | ||
2018 | 119 | return subprocess.call(cmd) == 0 | 267 | return subprocess.call(cmd) == 0 |
2019 | 120 | 268 | ||
2020 | 121 | 269 | ||
2050 | 122 | def service_running(service): | 270 | _UPSTART_CONF = "/etc/init/{}.conf" |
2051 | 123 | """Determine whether a system service is running""" | 271 | _INIT_D_CONF = "/etc/init.d/{}" |
2052 | 124 | try: | 272 | |
2053 | 125 | output = subprocess.check_output( | 273 | |
2054 | 126 | ['service', service, 'status'], | 274 | def service_running(service_name, **kwargs): |
2055 | 127 | stderr=subprocess.STDOUT).decode('UTF-8') | 275 | """Determine whether a system service is running. |
2056 | 128 | except subprocess.CalledProcessError: | 276 | |
2057 | 129 | return False | 277 | :param service_name: the name of the service |
2058 | 130 | else: | 278 | :param **kwargs: additional args to pass to the service command. This is |
2059 | 131 | if ("start/running" in output or "is running" in output): | 279 | used to pass additional key=value arguments to the |
2060 | 132 | return True | 280 | service command line for managing specific instance |
2061 | 133 | else: | 281 | units (e.g. service ceph-osd status id=2). The kwargs |
2062 | 134 | return False | 282 | are ignored in systemd services. |
2034 | 135 | |||
2035 | 136 | |||
2036 | 137 | def service_available(service_name): | ||
2037 | 138 | """Determine whether a system service is available""" | ||
2038 | 139 | try: | ||
2039 | 140 | subprocess.check_output( | ||
2040 | 141 | ['service', service_name, 'status'], | ||
2041 | 142 | stderr=subprocess.STDOUT).decode('UTF-8') | ||
2042 | 143 | except subprocess.CalledProcessError as e: | ||
2043 | 144 | return b'unrecognized service' not in e.output | ||
2044 | 145 | else: | ||
2045 | 146 | return True | ||
2046 | 147 | |||
2047 | 148 | |||
2048 | 149 | def adduser(username, password=None, shell='/bin/bash', system_user=False, | ||
2049 | 150 | primary_group=None, secondary_groups=None): | ||
2063 | 151 | """ | 283 | """ |
2065 | 152 | Add a user to the system. | 284 | if init_is_systemd(): |
2066 | 285 | return service('is-active', service_name) | ||
2067 | 286 | else: | ||
2068 | 287 | if os.path.exists(_UPSTART_CONF.format(service_name)): | ||
2069 | 288 | try: | ||
2070 | 289 | cmd = ['status', service_name] | ||
2071 | 290 | for key, value in six.iteritems(kwargs): | ||
2072 | 291 | parameter = '%s=%s' % (key, value) | ||
2073 | 292 | cmd.append(parameter) | ||
2074 | 293 | output = subprocess.check_output( | ||
2075 | 294 | cmd, stderr=subprocess.STDOUT).decode('UTF-8') | ||
2076 | 295 | except subprocess.CalledProcessError: | ||
2077 | 296 | return False | ||
2078 | 297 | else: | ||
2079 | 298 | # This works for upstart scripts where the 'service' command | ||
2080 | 299 | # returns a consistent string to represent running | ||
2081 | 300 | # 'start/running' | ||
2082 | 301 | if ("start/running" in output or | ||
2083 | 302 | "is running" in output or | ||
2084 | 303 | "up and running" in output): | ||
2085 | 304 | return True | ||
2086 | 305 | elif os.path.exists(_INIT_D_CONF.format(service_name)): | ||
2087 | 306 | # Check System V scripts init script return codes | ||
2088 | 307 | return service('status', service_name) | ||
2089 | 308 | return False | ||
2090 | 309 | |||
2091 | 310 | |||
2092 | 311 | SYSTEMD_SYSTEM = '/run/systemd/system' | ||
2093 | 312 | |||
2094 | 313 | |||
2095 | 314 | def init_is_systemd(): | ||
2096 | 315 | """Return True if the host system uses systemd, False otherwise.""" | ||
2097 | 316 | if lsb_release()['DISTRIB_CODENAME'] == 'trusty': | ||
2098 | 317 | return False | ||
2099 | 318 | return os.path.isdir(SYSTEMD_SYSTEM) | ||
2100 | 319 | |||
2101 | 320 | |||
2102 | 321 | def adduser(username, password=None, shell='/bin/bash', | ||
2103 | 322 | system_user=False, primary_group=None, | ||
2104 | 323 | secondary_groups=None, uid=None, home_dir=None): | ||
2105 | 324 | """Add a user to the system. | ||
2106 | 153 | 325 | ||
2107 | 154 | Will log but otherwise succeed if the user already exists. | 326 | Will log but otherwise succeed if the user already exists. |
2108 | 155 | 327 | ||
2109 | @@ -157,17 +329,26 @@ | |||
2110 | 157 | :param str password: Password for user; if ``None``, create a system user | 329 | :param str password: Password for user; if ``None``, create a system user |
2111 | 158 | :param str shell: The default shell for the user | 330 | :param str shell: The default shell for the user |
2112 | 159 | :param bool system_user: Whether to create a login or system user | 331 | :param bool system_user: Whether to create a login or system user |
2114 | 160 | :param str primary_group: Primary group for user; defaults to their username | 332 | :param str primary_group: Primary group for user; defaults to username |
2115 | 161 | :param list secondary_groups: Optional list of additional groups | 333 | :param list secondary_groups: Optional list of additional groups |
2116 | 334 | :param int uid: UID for user being created | ||
2117 | 335 | :param str home_dir: Home directory for user | ||
2118 | 162 | 336 | ||
2119 | 163 | :returns: The password database entry struct, as returned by `pwd.getpwnam` | 337 | :returns: The password database entry struct, as returned by `pwd.getpwnam` |
2120 | 164 | """ | 338 | """ |
2121 | 165 | try: | 339 | try: |
2122 | 166 | user_info = pwd.getpwnam(username) | 340 | user_info = pwd.getpwnam(username) |
2123 | 167 | log('user {0} already exists!'.format(username)) | 341 | log('user {0} already exists!'.format(username)) |
2124 | 342 | if uid: | ||
2125 | 343 | user_info = pwd.getpwuid(int(uid)) | ||
2126 | 344 | log('user with uid {0} already exists!'.format(uid)) | ||
2127 | 168 | except KeyError: | 345 | except KeyError: |
2128 | 169 | log('creating user {0}'.format(username)) | 346 | log('creating user {0}'.format(username)) |
2129 | 170 | cmd = ['useradd'] | 347 | cmd = ['useradd'] |
2130 | 348 | if uid: | ||
2131 | 349 | cmd.extend(['--uid', str(uid)]) | ||
2132 | 350 | if home_dir: | ||
2133 | 351 | cmd.extend(['--home', str(home_dir)]) | ||
2134 | 171 | if system_user or password is None: | 352 | if system_user or password is None: |
2135 | 172 | cmd.append('--system') | 353 | cmd.append('--system') |
2136 | 173 | else: | 354 | else: |
2137 | @@ -202,22 +383,56 @@ | |||
2138 | 202 | return user_exists | 383 | return user_exists |
2139 | 203 | 384 | ||
2140 | 204 | 385 | ||
2143 | 205 | def add_group(group_name, system_group=False): | 386 | def uid_exists(uid): |
2144 | 206 | """Add a group to the system""" | 387 | """Check if a uid exists""" |
2145 | 388 | try: | ||
2146 | 389 | pwd.getpwuid(uid) | ||
2147 | 390 | uid_exists = True | ||
2148 | 391 | except KeyError: | ||
2149 | 392 | uid_exists = False | ||
2150 | 393 | return uid_exists | ||
2151 | 394 | |||
2152 | 395 | |||
2153 | 396 | def group_exists(groupname): | ||
2154 | 397 | """Check if a group exists""" | ||
2155 | 398 | try: | ||
2156 | 399 | grp.getgrnam(groupname) | ||
2157 | 400 | group_exists = True | ||
2158 | 401 | except KeyError: | ||
2159 | 402 | group_exists = False | ||
2160 | 403 | return group_exists | ||
2161 | 404 | |||
2162 | 405 | |||
2163 | 406 | def gid_exists(gid): | ||
2164 | 407 | """Check if a gid exists""" | ||
2165 | 408 | try: | ||
2166 | 409 | grp.getgrgid(gid) | ||
2167 | 410 | gid_exists = True | ||
2168 | 411 | except KeyError: | ||
2169 | 412 | gid_exists = False | ||
2170 | 413 | return gid_exists | ||
2171 | 414 | |||
2172 | 415 | |||
2173 | 416 | def add_group(group_name, system_group=False, gid=None): | ||
2174 | 417 | """Add a group to the system | ||
2175 | 418 | |||
2176 | 419 | Will log but otherwise succeed if the group already exists. | ||
2177 | 420 | |||
2178 | 421 | :param str group_name: group to create | ||
2179 | 422 | :param bool system_group: Create system group | ||
2180 | 423 | :param int gid: GID for user being created | ||
2181 | 424 | |||
2182 | 425 | :returns: The password database entry struct, as returned by `grp.getgrnam` | ||
2183 | 426 | """ | ||
2184 | 207 | try: | 427 | try: |
2185 | 208 | group_info = grp.getgrnam(group_name) | 428 | group_info = grp.getgrnam(group_name) |
2186 | 209 | log('group {0} already exists!'.format(group_name)) | 429 | log('group {0} already exists!'.format(group_name)) |
2187 | 430 | if gid: | ||
2188 | 431 | group_info = grp.getgrgid(gid) | ||
2189 | 432 | log('group with gid {0} already exists!'.format(gid)) | ||
2190 | 210 | except KeyError: | 433 | except KeyError: |
2191 | 211 | log('creating group {0}'.format(group_name)) | 434 | log('creating group {0}'.format(group_name)) |
2201 | 212 | cmd = ['addgroup'] | 435 | add_new_group(group_name, system_group, gid) |
2193 | 213 | if system_group: | ||
2194 | 214 | cmd.append('--system') | ||
2195 | 215 | else: | ||
2196 | 216 | cmd.extend([ | ||
2197 | 217 | '--group', | ||
2198 | 218 | ]) | ||
2199 | 219 | cmd.append(group_name) | ||
2200 | 220 | subprocess.check_call(cmd) | ||
2202 | 221 | group_info = grp.getgrnam(group_name) | 436 | group_info = grp.getgrnam(group_name) |
2203 | 222 | return group_info | 437 | return group_info |
2204 | 223 | 438 | ||
2205 | @@ -229,15 +444,62 @@ | |||
2206 | 229 | subprocess.check_call(cmd) | 444 | subprocess.check_call(cmd) |
2207 | 230 | 445 | ||
2208 | 231 | 446 | ||
2210 | 232 | def rsync(from_path, to_path, flags='-r', options=None): | 447 | def chage(username, lastday=None, expiredate=None, inactive=None, |
2211 | 448 | mindays=None, maxdays=None, root=None, warndays=None): | ||
2212 | 449 | """Change user password expiry information | ||
2213 | 450 | |||
2214 | 451 | :param str username: User to update | ||
2215 | 452 | :param str lastday: Set when password was changed in YYYY-MM-DD format | ||
2216 | 453 | :param str expiredate: Set when user's account will no longer be | ||
2217 | 454 | accessible in YYYY-MM-DD format. | ||
2218 | 455 | -1 will remove an account expiration date. | ||
2219 | 456 | :param str inactive: Set the number of days of inactivity after a password | ||
2220 | 457 | has expired before the account is locked. | ||
2221 | 458 | -1 will remove an account's inactivity. | ||
2222 | 459 | :param str mindays: Set the minimum number of days between password | ||
2223 | 460 | changes to MIN_DAYS. | ||
2224 | 461 | 0 indicates the password can be changed anytime. | ||
2225 | 462 | :param str maxdays: Set the maximum number of days during which a | ||
2226 | 463 | password is valid. | ||
2227 | 464 | -1 as MAX_DAYS will remove checking maxdays | ||
2228 | 465 | :param str root: Apply changes in the CHROOT_DIR directory | ||
2229 | 466 | :param str warndays: Set the number of days of warning before a password | ||
2230 | 467 | change is required | ||
2231 | 468 | :raises subprocess.CalledProcessError: if call to chage fails | ||
2232 | 469 | """ | ||
2233 | 470 | cmd = ['chage'] | ||
2234 | 471 | if root: | ||
2235 | 472 | cmd.extend(['--root', root]) | ||
2236 | 473 | if lastday: | ||
2237 | 474 | cmd.extend(['--lastday', lastday]) | ||
2238 | 475 | if expiredate: | ||
2239 | 476 | cmd.extend(['--expiredate', expiredate]) | ||
2240 | 477 | if inactive: | ||
2241 | 478 | cmd.extend(['--inactive', inactive]) | ||
2242 | 479 | if mindays: | ||
2243 | 480 | cmd.extend(['--mindays', mindays]) | ||
2244 | 481 | if maxdays: | ||
2245 | 482 | cmd.extend(['--maxdays', maxdays]) | ||
2246 | 483 | if warndays: | ||
2247 | 484 | cmd.extend(['--warndays', warndays]) | ||
2248 | 485 | cmd.append(username) | ||
2249 | 486 | subprocess.check_call(cmd) | ||
2250 | 487 | |||
2251 | 488 | |||
2252 | 489 | remove_password_expiry = functools.partial(chage, expiredate='-1', inactive='-1', mindays='0', maxdays='-1') | ||
2253 | 490 | |||
2254 | 491 | |||
2255 | 492 | def rsync(from_path, to_path, flags='-r', options=None, timeout=None): | ||
2256 | 233 | """Replicate the contents of a path""" | 493 | """Replicate the contents of a path""" |
2257 | 234 | options = options or ['--delete', '--executability'] | 494 | options = options or ['--delete', '--executability'] |
2258 | 235 | cmd = ['/usr/bin/rsync', flags] | 495 | cmd = ['/usr/bin/rsync', flags] |
2259 | 496 | if timeout: | ||
2260 | 497 | cmd = ['timeout', str(timeout)] + cmd | ||
2261 | 236 | cmd.extend(options) | 498 | cmd.extend(options) |
2262 | 237 | cmd.append(from_path) | 499 | cmd.append(from_path) |
2263 | 238 | cmd.append(to_path) | 500 | cmd.append(to_path) |
2264 | 239 | log(" ".join(cmd)) | 501 | log(" ".join(cmd)) |
2266 | 240 | return subprocess.check_output(cmd).decode('UTF-8').strip() | 502 | return subprocess.check_output(cmd, stderr=subprocess.STDOUT).decode('UTF-8').strip() |
2267 | 241 | 503 | ||
2268 | 242 | 504 | ||
2269 | 243 | def symlink(source, destination): | 505 | def symlink(source, destination): |
2270 | @@ -273,24 +535,54 @@ | |||
2271 | 273 | 535 | ||
2272 | 274 | def write_file(path, content, owner='root', group='root', perms=0o444): | 536 | def write_file(path, content, owner='root', group='root', perms=0o444): |
2273 | 275 | """Create or overwrite a file with the contents of a byte string.""" | 537 | """Create or overwrite a file with the contents of a byte string.""" |
2274 | 276 | log("Writing file {} {}:{} {:o}".format(path, owner, group, perms)) | ||
2275 | 277 | uid = pwd.getpwnam(owner).pw_uid | 538 | uid = pwd.getpwnam(owner).pw_uid |
2276 | 278 | gid = grp.getgrnam(group).gr_gid | 539 | gid = grp.getgrnam(group).gr_gid |
2281 | 279 | with open(path, 'wb') as target: | 540 | # lets see if we can grab the file and compare the context, to avoid doing |
2282 | 280 | os.fchown(target.fileno(), uid, gid) | 541 | # a write. |
2283 | 281 | os.fchmod(target.fileno(), perms) | 542 | existing_content = None |
2284 | 282 | target.write(content) | 543 | existing_uid, existing_gid, existing_perms = None, None, None |
2285 | 544 | try: | ||
2286 | 545 | with open(path, 'rb') as target: | ||
2287 | 546 | existing_content = target.read() | ||
2288 | 547 | stat = os.stat(path) | ||
2289 | 548 | existing_uid, existing_gid, existing_perms = ( | ||
2290 | 549 | stat.st_uid, stat.st_gid, stat.st_mode | ||
2291 | 550 | ) | ||
2292 | 551 | except Exception: | ||
2293 | 552 | pass | ||
2294 | 553 | if content != existing_content: | ||
2295 | 554 | log("Writing file {} {}:{} {:o}".format(path, owner, group, perms), | ||
2296 | 555 | level=DEBUG) | ||
2297 | 556 | with open(path, 'wb') as target: | ||
2298 | 557 | os.fchown(target.fileno(), uid, gid) | ||
2299 | 558 | os.fchmod(target.fileno(), perms) | ||
2300 | 559 | if six.PY3 and isinstance(content, six.string_types): | ||
2301 | 560 | content = content.encode('UTF-8') | ||
2302 | 561 | target.write(content) | ||
2303 | 562 | return | ||
2304 | 563 | # the contents were the same, but we might still need to change the | ||
2305 | 564 | # ownership or permissions. | ||
2306 | 565 | if existing_uid != uid: | ||
2307 | 566 | log("Changing uid on already existing content: {} -> {}" | ||
2308 | 567 | .format(existing_uid, uid), level=DEBUG) | ||
2309 | 568 | os.chown(path, uid, -1) | ||
2310 | 569 | if existing_gid != gid: | ||
2311 | 570 | log("Changing gid on already existing content: {} -> {}" | ||
2312 | 571 | .format(existing_gid, gid), level=DEBUG) | ||
2313 | 572 | os.chown(path, -1, gid) | ||
2314 | 573 | if existing_perms != perms: | ||
2315 | 574 | log("Changing permissions on existing content: {} -> {}" | ||
2316 | 575 | .format(existing_perms, perms), level=DEBUG) | ||
2317 | 576 | os.chmod(path, perms) | ||
2318 | 283 | 577 | ||
2319 | 284 | 578 | ||
2320 | 285 | def fstab_remove(mp): | 579 | def fstab_remove(mp): |
2323 | 286 | """Remove the given mountpoint entry from /etc/fstab | 580 | """Remove the given mountpoint entry from /etc/fstab""" |
2322 | 287 | """ | ||
2324 | 288 | return Fstab.remove_by_mountpoint(mp) | 581 | return Fstab.remove_by_mountpoint(mp) |
2325 | 289 | 582 | ||
2326 | 290 | 583 | ||
2327 | 291 | def fstab_add(dev, mp, fs, options=None): | 584 | def fstab_add(dev, mp, fs, options=None): |
2330 | 292 | """Adds the given device entry to the /etc/fstab file | 585 | """Adds the given device entry to the /etc/fstab file""" |
2329 | 293 | """ | ||
2331 | 294 | return Fstab.add(dev, mp, fs, options=options) | 586 | return Fstab.add(dev, mp, fs, options=options) |
2332 | 295 | 587 | ||
2333 | 296 | 588 | ||
2334 | @@ -346,8 +638,7 @@ | |||
2335 | 346 | 638 | ||
2336 | 347 | 639 | ||
2337 | 348 | def file_hash(path, hash_type='md5'): | 640 | def file_hash(path, hash_type='md5'): |
2340 | 349 | """ | 641 | """Generate a hash checksum of the contents of 'path' or None if not found. |
2339 | 350 | Generate a hash checksum of the contents of 'path' or None if not found. | ||
2341 | 351 | 642 | ||
2342 | 352 | :param str hash_type: Any hash alrgorithm supported by :mod:`hashlib`, | 643 | :param str hash_type: Any hash alrgorithm supported by :mod:`hashlib`, |
2343 | 353 | such as md5, sha1, sha256, sha512, etc. | 644 | such as md5, sha1, sha256, sha512, etc. |
2344 | @@ -362,10 +653,9 @@ | |||
2345 | 362 | 653 | ||
2346 | 363 | 654 | ||
2347 | 364 | def path_hash(path): | 655 | def path_hash(path): |
2352 | 365 | """ | 656 | """Generate a hash checksum of all files matching 'path'. Standard |
2353 | 366 | Generate a hash checksum of all files matching 'path'. Standard wildcards | 657 | wildcards like '*' and '?' are supported, see documentation for the 'glob' |
2354 | 367 | like '*' and '?' are supported, see documentation for the 'glob' module for | 658 | module for more information. |
2351 | 368 | more information. | ||
2355 | 369 | 659 | ||
2356 | 370 | :return: dict: A { filename: hash } dictionary for all matched files. | 660 | :return: dict: A { filename: hash } dictionary for all matched files. |
2357 | 371 | Empty if none found. | 661 | Empty if none found. |
2358 | @@ -377,8 +667,7 @@ | |||
2359 | 377 | 667 | ||
2360 | 378 | 668 | ||
2361 | 379 | def check_hash(path, checksum, hash_type='md5'): | 669 | def check_hash(path, checksum, hash_type='md5'): |
2364 | 380 | """ | 670 | """Validate a file using a cryptographic checksum. |
2363 | 381 | Validate a file using a cryptographic checksum. | ||
2365 | 382 | 671 | ||
2366 | 383 | :param str checksum: Value of the checksum used to validate the file. | 672 | :param str checksum: Value of the checksum used to validate the file. |
2367 | 384 | :param str hash_type: Hash algorithm used to generate `checksum`. | 673 | :param str hash_type: Hash algorithm used to generate `checksum`. |
2368 | @@ -393,10 +682,11 @@ | |||
2369 | 393 | 682 | ||
2370 | 394 | 683 | ||
2371 | 395 | class ChecksumError(ValueError): | 684 | class ChecksumError(ValueError): |
2372 | 685 | """A class derived from Value error to indicate the checksum failed.""" | ||
2373 | 396 | pass | 686 | pass |
2374 | 397 | 687 | ||
2375 | 398 | 688 | ||
2377 | 399 | def restart_on_change(restart_map, stopstart=False): | 689 | def restart_on_change(restart_map, stopstart=False, restart_functions=None): |
2378 | 400 | """Restart services based on configuration files changing | 690 | """Restart services based on configuration files changing |
2379 | 401 | 691 | ||
2380 | 402 | This function is used a decorator, for example:: | 692 | This function is used a decorator, for example:: |
2381 | @@ -414,35 +704,56 @@ | |||
2382 | 414 | restarted if any file matching the pattern got changed, created | 704 | restarted if any file matching the pattern got changed, created |
2383 | 415 | or removed. Standard wildcards are supported, see documentation | 705 | or removed. Standard wildcards are supported, see documentation |
2384 | 416 | for the 'glob' module for more information. | 706 | for the 'glob' module for more information. |
2385 | 707 | |||
2386 | 708 | @param restart_map: {path_file_name: [service_name, ...] | ||
2387 | 709 | @param stopstart: DEFAULT false; whether to stop, start OR restart | ||
2388 | 710 | @param restart_functions: nonstandard functions to use to restart services | ||
2389 | 711 | {svc: func, ...} | ||
2390 | 712 | @returns result from decorated function | ||
2391 | 417 | """ | 713 | """ |
2392 | 418 | def wrap(f): | 714 | def wrap(f): |
2393 | 715 | @functools.wraps(f) | ||
2394 | 419 | def wrapped_f(*args, **kwargs): | 716 | def wrapped_f(*args, **kwargs): |
2409 | 420 | checksums = {path: path_hash(path) for path in restart_map} | 717 | return restart_on_change_helper( |
2410 | 421 | f(*args, **kwargs) | 718 | (lambda: f(*args, **kwargs)), restart_map, stopstart, |
2411 | 422 | restarts = [] | 719 | restart_functions) |
2398 | 423 | for path in restart_map: | ||
2399 | 424 | if path_hash(path) != checksums[path]: | ||
2400 | 425 | restarts += restart_map[path] | ||
2401 | 426 | services_list = list(OrderedDict.fromkeys(restarts)) | ||
2402 | 427 | if not stopstart: | ||
2403 | 428 | for service_name in services_list: | ||
2404 | 429 | service('restart', service_name) | ||
2405 | 430 | else: | ||
2406 | 431 | for action in ['stop', 'start']: | ||
2407 | 432 | for service_name in services_list: | ||
2408 | 433 | service(action, service_name) | ||
2412 | 434 | return wrapped_f | 720 | return wrapped_f |
2413 | 435 | return wrap | 721 | return wrap |
2414 | 436 | 722 | ||
2415 | 437 | 723 | ||
2424 | 438 | def lsb_release(): | 724 | def restart_on_change_helper(lambda_f, restart_map, stopstart=False, |
2425 | 439 | """Return /etc/lsb-release in a dict""" | 725 | restart_functions=None): |
2426 | 440 | d = {} | 726 | """Helper function to perform the restart_on_change function. |
2427 | 441 | with open('/etc/lsb-release', 'r') as lsb: | 727 | |
2428 | 442 | for l in lsb: | 728 | This is provided for decorators to restart services if files described |
2429 | 443 | k, v = l.split('=') | 729 | in the restart_map have changed after an invocation of lambda_f(). |
2430 | 444 | d[k.strip()] = v.strip() | 730 | |
2431 | 445 | return d | 731 | @param lambda_f: function to call. |
2432 | 732 | @param restart_map: {file: [service, ...]} | ||
2433 | 733 | @param stopstart: whether to stop, start or restart a service | ||
2434 | 734 | @param restart_functions: nonstandard functions to use to restart services | ||
2435 | 735 | {svc: func, ...} | ||
2436 | 736 | @returns result of lambda_f() | ||
2437 | 737 | """ | ||
2438 | 738 | if restart_functions is None: | ||
2439 | 739 | restart_functions = {} | ||
2440 | 740 | checksums = {path: path_hash(path) for path in restart_map} | ||
2441 | 741 | r = lambda_f() | ||
2442 | 742 | # create a list of lists of the services to restart | ||
2443 | 743 | restarts = [restart_map[path] | ||
2444 | 744 | for path in restart_map | ||
2445 | 745 | if path_hash(path) != checksums[path]] | ||
2446 | 746 | # create a flat list of ordered services without duplicates from lists | ||
2447 | 747 | services_list = list(OrderedDict.fromkeys(itertools.chain(*restarts))) | ||
2448 | 748 | if services_list: | ||
2449 | 749 | actions = ('stop', 'start') if stopstart else ('restart',) | ||
2450 | 750 | for service_name in services_list: | ||
2451 | 751 | if service_name in restart_functions: | ||
2452 | 752 | restart_functions[service_name](service_name) | ||
2453 | 753 | else: | ||
2454 | 754 | for action in actions: | ||
2455 | 755 | service(action, service_name) | ||
2456 | 756 | return r | ||
2457 | 446 | 757 | ||
2458 | 447 | 758 | ||
2459 | 448 | def pwgen(length=None): | 759 | def pwgen(length=None): |
2460 | @@ -498,7 +809,7 @@ | |||
2461 | 498 | 809 | ||
2462 | 499 | 810 | ||
2463 | 500 | def list_nics(nic_type=None): | 811 | def list_nics(nic_type=None): |
2465 | 501 | '''Return a list of nics of given type(s)''' | 812 | """Return a list of nics of given type(s)""" |
2466 | 502 | if isinstance(nic_type, six.string_types): | 813 | if isinstance(nic_type, six.string_types): |
2467 | 503 | int_types = [nic_type] | 814 | int_types = [nic_type] |
2468 | 504 | else: | 815 | else: |
2469 | @@ -527,7 +838,7 @@ | |||
2470 | 527 | ip_output = subprocess.check_output(cmd).decode('UTF-8').split('\n') | 838 | ip_output = subprocess.check_output(cmd).decode('UTF-8').split('\n') |
2471 | 528 | ip_output = (line.strip() for line in ip_output if line) | 839 | ip_output = (line.strip() for line in ip_output if line) |
2472 | 529 | 840 | ||
2474 | 530 | key = re.compile('^[0-9]+:\s+(.+):') | 841 | key = re.compile(r'^[0-9]+:\s+(.+):') |
2475 | 531 | for line in ip_output: | 842 | for line in ip_output: |
2476 | 532 | matched = re.search(key, line) | 843 | matched = re.search(key, line) |
2477 | 533 | if matched: | 844 | if matched: |
2478 | @@ -540,12 +851,13 @@ | |||
2479 | 540 | 851 | ||
2480 | 541 | 852 | ||
2481 | 542 | def set_nic_mtu(nic, mtu): | 853 | def set_nic_mtu(nic, mtu): |
2483 | 543 | '''Set MTU on a network interface''' | 854 | """Set the Maximum Transmission Unit (MTU) on a network interface.""" |
2484 | 544 | cmd = ['ip', 'link', 'set', nic, 'mtu', mtu] | 855 | cmd = ['ip', 'link', 'set', nic, 'mtu', mtu] |
2485 | 545 | subprocess.check_call(cmd) | 856 | subprocess.check_call(cmd) |
2486 | 546 | 857 | ||
2487 | 547 | 858 | ||
2488 | 548 | def get_nic_mtu(nic): | 859 | def get_nic_mtu(nic): |
2489 | 860 | """Return the Maximum Transmission Unit (MTU) for a network interface.""" | ||
2490 | 549 | cmd = ['ip', 'addr', 'show', nic] | 861 | cmd = ['ip', 'addr', 'show', nic] |
2491 | 550 | ip_output = subprocess.check_output(cmd).decode('UTF-8').split('\n') | 862 | ip_output = subprocess.check_output(cmd).decode('UTF-8').split('\n') |
2492 | 551 | mtu = "" | 863 | mtu = "" |
2493 | @@ -557,6 +869,7 @@ | |||
2494 | 557 | 869 | ||
2495 | 558 | 870 | ||
2496 | 559 | def get_nic_hwaddr(nic): | 871 | def get_nic_hwaddr(nic): |
2497 | 872 | """Return the Media Access Control (MAC) for a network interface.""" | ||
2498 | 560 | cmd = ['ip', '-o', '-0', 'addr', 'show', nic] | 873 | cmd = ['ip', '-o', '-0', 'addr', 'show', nic] |
2499 | 561 | ip_output = subprocess.check_output(cmd).decode('UTF-8') | 874 | ip_output = subprocess.check_output(cmd).decode('UTF-8') |
2500 | 562 | hwaddr = "" | 875 | hwaddr = "" |
2501 | @@ -566,40 +879,29 @@ | |||
2502 | 566 | return hwaddr | 879 | return hwaddr |
2503 | 567 | 880 | ||
2504 | 568 | 881 | ||
2505 | 569 | def cmp_pkgrevno(package, revno, pkgcache=None): | ||
2506 | 570 | '''Compare supplied revno with the revno of the installed package | ||
2507 | 571 | |||
2508 | 572 | * 1 => Installed revno is greater than supplied arg | ||
2509 | 573 | * 0 => Installed revno is the same as supplied arg | ||
2510 | 574 | * -1 => Installed revno is less than supplied arg | ||
2511 | 575 | |||
2512 | 576 | This function imports apt_cache function from charmhelpers.fetch if | ||
2513 | 577 | the pkgcache argument is None. Be sure to add charmhelpers.fetch if | ||
2514 | 578 | you call this function, or pass an apt_pkg.Cache() instance. | ||
2515 | 579 | ''' | ||
2516 | 580 | import apt_pkg | ||
2517 | 581 | if not pkgcache: | ||
2518 | 582 | from charmhelpers.fetch import apt_cache | ||
2519 | 583 | pkgcache = apt_cache() | ||
2520 | 584 | pkg = pkgcache[package] | ||
2521 | 585 | return apt_pkg.version_compare(pkg.current_ver.ver_str, revno) | ||
2522 | 586 | |||
2523 | 587 | |||
2524 | 588 | @contextmanager | 882 | @contextmanager |
2526 | 589 | def chdir(d): | 883 | def chdir(directory): |
2527 | 884 | """Change the current working directory to a different directory for a code | ||
2528 | 885 | block and return the previous directory after the block exits. Useful to | ||
2529 | 886 | run commands from a specificed directory. | ||
2530 | 887 | |||
2531 | 888 | :param str directory: The directory path to change to for this context. | ||
2532 | 889 | """ | ||
2533 | 590 | cur = os.getcwd() | 890 | cur = os.getcwd() |
2534 | 591 | try: | 891 | try: |
2536 | 592 | yield os.chdir(d) | 892 | yield os.chdir(directory) |
2537 | 593 | finally: | 893 | finally: |
2538 | 594 | os.chdir(cur) | 894 | os.chdir(cur) |
2539 | 595 | 895 | ||
2540 | 596 | 896 | ||
2541 | 597 | def chownr(path, owner, group, follow_links=True, chowntopdir=False): | 897 | def chownr(path, owner, group, follow_links=True, chowntopdir=False): |
2544 | 598 | """ | 898 | """Recursively change user and group ownership of files and directories |
2543 | 599 | Recursively change user and group ownership of files and directories | ||
2545 | 600 | in given path. Doesn't chown path itself by default, only its children. | 899 | in given path. Doesn't chown path itself by default, only its children. |
2546 | 601 | 900 | ||
2548 | 602 | :param bool follow_links: Also Chown links if True | 901 | :param str path: The string path to start changing ownership. |
2549 | 902 | :param str owner: The owner string to use when looking up the uid. | ||
2550 | 903 | :param str group: The group string to use when looking up the gid. | ||
2551 | 904 | :param bool follow_links: Also follow and chown links if True | ||
2552 | 603 | :param bool chowntopdir: Also chown path itself if True | 905 | :param bool chowntopdir: Also chown path itself if True |
2553 | 604 | """ | 906 | """ |
2554 | 605 | uid = pwd.getpwnam(owner).pw_uid | 907 | uid = pwd.getpwnam(owner).pw_uid |
2555 | @@ -613,7 +915,7 @@ | |||
2556 | 613 | broken_symlink = os.path.lexists(path) and not os.path.exists(path) | 915 | broken_symlink = os.path.lexists(path) and not os.path.exists(path) |
2557 | 614 | if not broken_symlink: | 916 | if not broken_symlink: |
2558 | 615 | chown(path, uid, gid) | 917 | chown(path, uid, gid) |
2560 | 616 | for root, dirs, files in os.walk(path): | 918 | for root, dirs, files in os.walk(path, followlinks=follow_links): |
2561 | 617 | for name in dirs + files: | 919 | for name in dirs + files: |
2562 | 618 | full = os.path.join(root, name) | 920 | full = os.path.join(root, name) |
2563 | 619 | broken_symlink = os.path.lexists(full) and not os.path.exists(full) | 921 | broken_symlink = os.path.lexists(full) and not os.path.exists(full) |
2564 | @@ -622,15 +924,37 @@ | |||
2565 | 622 | 924 | ||
2566 | 623 | 925 | ||
2567 | 624 | def lchownr(path, owner, group): | 926 | def lchownr(path, owner, group): |
2568 | 927 | """Recursively change user and group ownership of files and directories | ||
2569 | 928 | in a given path, not following symbolic links. See the documentation for | ||
2570 | 929 | 'os.lchown' for more information. | ||
2571 | 930 | |||
2572 | 931 | :param str path: The string path to start changing ownership. | ||
2573 | 932 | :param str owner: The owner string to use when looking up the uid. | ||
2574 | 933 | :param str group: The group string to use when looking up the gid. | ||
2575 | 934 | """ | ||
2576 | 625 | chownr(path, owner, group, follow_links=False) | 935 | chownr(path, owner, group, follow_links=False) |
2577 | 626 | 936 | ||
2578 | 627 | 937 | ||
2579 | 938 | def owner(path): | ||
2580 | 939 | """Returns a tuple containing the username & groupname owning the path. | ||
2581 | 940 | |||
2582 | 941 | :param str path: the string path to retrieve the ownership | ||
2583 | 942 | :return tuple(str, str): A (username, groupname) tuple containing the | ||
2584 | 943 | name of the user and group owning the path. | ||
2585 | 944 | :raises OSError: if the specified path does not exist | ||
2586 | 945 | """ | ||
2587 | 946 | stat = os.stat(path) | ||
2588 | 947 | username = pwd.getpwuid(stat.st_uid)[0] | ||
2589 | 948 | groupname = grp.getgrgid(stat.st_gid)[0] | ||
2590 | 949 | return username, groupname | ||
2591 | 950 | |||
2592 | 951 | |||
2593 | 628 | def get_total_ram(): | 952 | def get_total_ram(): |
2595 | 629 | '''The total amount of system RAM in bytes. | 953 | """The total amount of system RAM in bytes. |
2596 | 630 | 954 | ||
2597 | 631 | This is what is reported by the OS, and may be overcommitted when | 955 | This is what is reported by the OS, and may be overcommitted when |
2598 | 632 | there are multiple containers hosted on the same machine. | 956 | there are multiple containers hosted on the same machine. |
2600 | 633 | ''' | 957 | """ |
2601 | 634 | with open('/proc/meminfo', 'r') as f: | 958 | with open('/proc/meminfo', 'r') as f: |
2602 | 635 | for line in f.readlines(): | 959 | for line in f.readlines(): |
2603 | 636 | if line: | 960 | if line: |
2604 | @@ -639,3 +963,115 @@ | |||
2605 | 639 | assert unit == 'kB', 'Unknown unit' | 963 | assert unit == 'kB', 'Unknown unit' |
2606 | 640 | return int(value) * 1024 # Classic, not KiB. | 964 | return int(value) * 1024 # Classic, not KiB. |
2607 | 641 | raise NotImplementedError() | 965 | raise NotImplementedError() |
2608 | 966 | |||
2609 | 967 | |||
2610 | 968 | UPSTART_CONTAINER_TYPE = '/run/container_type' | ||
2611 | 969 | |||
2612 | 970 | |||
2613 | 971 | def is_container(): | ||
2614 | 972 | """Determine whether unit is running in a container | ||
2615 | 973 | |||
2616 | 974 | @return: boolean indicating if unit is in a container | ||
2617 | 975 | """ | ||
2618 | 976 | if init_is_systemd(): | ||
2619 | 977 | # Detect using systemd-detect-virt | ||
2620 | 978 | return subprocess.call(['systemd-detect-virt', | ||
2621 | 979 | '--container']) == 0 | ||
2622 | 980 | else: | ||
2623 | 981 | # Detect using upstart container file marker | ||
2624 | 982 | return os.path.exists(UPSTART_CONTAINER_TYPE) | ||
2625 | 983 | |||
2626 | 984 | |||
2627 | 985 | def add_to_updatedb_prunepath(path, updatedb_path=UPDATEDB_PATH): | ||
2628 | 986 | """Adds the specified path to the mlocate's udpatedb.conf PRUNEPATH list. | ||
2629 | 987 | |||
2630 | 988 | This method has no effect if the path specified by updatedb_path does not | ||
2631 | 989 | exist or is not a file. | ||
2632 | 990 | |||
2633 | 991 | @param path: string the path to add to the updatedb.conf PRUNEPATHS value | ||
2634 | 992 | @param updatedb_path: the path the updatedb.conf file | ||
2635 | 993 | """ | ||
2636 | 994 | if not os.path.exists(updatedb_path) or os.path.isdir(updatedb_path): | ||
2637 | 995 | # If the updatedb.conf file doesn't exist then don't attempt to update | ||
2638 | 996 | # the file as the package providing mlocate may not be installed on | ||
2639 | 997 | # the local system | ||
2640 | 998 | return | ||
2641 | 999 | |||
2642 | 1000 | with open(updatedb_path, 'r+') as f_id: | ||
2643 | 1001 | updatedb_text = f_id.read() | ||
2644 | 1002 | output = updatedb(updatedb_text, path) | ||
2645 | 1003 | f_id.seek(0) | ||
2646 | 1004 | f_id.write(output) | ||
2647 | 1005 | f_id.truncate() | ||
2648 | 1006 | |||
2649 | 1007 | |||
2650 | 1008 | def updatedb(updatedb_text, new_path): | ||
2651 | 1009 | lines = [line for line in updatedb_text.split("\n")] | ||
2652 | 1010 | for i, line in enumerate(lines): | ||
2653 | 1011 | if line.startswith("PRUNEPATHS="): | ||
2654 | 1012 | paths_line = line.split("=")[1].replace('"', '') | ||
2655 | 1013 | paths = paths_line.split(" ") | ||
2656 | 1014 | if new_path not in paths: | ||
2657 | 1015 | paths.append(new_path) | ||
2658 | 1016 | lines[i] = 'PRUNEPATHS="{}"'.format(' '.join(paths)) | ||
2659 | 1017 | output = "\n".join(lines) | ||
2660 | 1018 | return output | ||
2661 | 1019 | |||
2662 | 1020 | |||
2663 | 1021 | def modulo_distribution(modulo=3, wait=30, non_zero_wait=False): | ||
2664 | 1022 | """ Modulo distribution | ||
2665 | 1023 | |||
2666 | 1024 | This helper uses the unit number, a modulo value and a constant wait time | ||
2667 | 1025 | to produce a calculated wait time distribution. This is useful in large | ||
2668 | 1026 | scale deployments to distribute load during an expensive operation such as | ||
2669 | 1027 | service restarts. | ||
2670 | 1028 | |||
2671 | 1029 | If you have 1000 nodes that need to restart 100 at a time 1 minute at a | ||
2672 | 1030 | time: | ||
2673 | 1031 | |||
2674 | 1032 | time.wait(modulo_distribution(modulo=100, wait=60)) | ||
2675 | 1033 | restart() | ||
2676 | 1034 | |||
2677 | 1035 | If you need restarts to happen serially set modulo to the exact number of | ||
2678 | 1036 | nodes and set a high constant wait time: | ||
2679 | 1037 | |||
2680 | 1038 | time.wait(modulo_distribution(modulo=10, wait=120)) | ||
2681 | 1039 | restart() | ||
2682 | 1040 | |||
2683 | 1041 | @param modulo: int The modulo number creates the group distribution | ||
2684 | 1042 | @param wait: int The constant time wait value | ||
2685 | 1043 | @param non_zero_wait: boolean Override unit % modulo == 0, | ||
2686 | 1044 | return modulo * wait. Used to avoid collisions with | ||
2687 | 1045 | leader nodes which are often given priority. | ||
2688 | 1046 | @return: int Calculated time to wait for unit operation | ||
2689 | 1047 | """ | ||
2690 | 1048 | unit_number = int(local_unit().split('/')[1]) | ||
2691 | 1049 | calculated_wait_time = (unit_number % modulo) * wait | ||
2692 | 1050 | if non_zero_wait and calculated_wait_time == 0: | ||
2693 | 1051 | return modulo * wait | ||
2694 | 1052 | else: | ||
2695 | 1053 | return calculated_wait_time | ||
2696 | 1054 | |||
2697 | 1055 | |||
2698 | 1056 | def install_ca_cert(ca_cert, name=None): | ||
2699 | 1057 | """ | ||
2700 | 1058 | Install the given cert as a trusted CA. | ||
2701 | 1059 | |||
2702 | 1060 | The ``name`` is the stem of the filename where the cert is written, and if | ||
2703 | 1061 | not provided, it will default to ``juju-{charm_name}``. | ||
2704 | 1062 | |||
2705 | 1063 | If the cert is empty or None, or is unchanged, nothing is done. | ||
2706 | 1064 | """ | ||
2707 | 1065 | if not ca_cert: | ||
2708 | 1066 | return | ||
2709 | 1067 | if not isinstance(ca_cert, bytes): | ||
2710 | 1068 | ca_cert = ca_cert.encode('utf8') | ||
2711 | 1069 | if not name: | ||
2712 | 1070 | name = 'juju-{}'.format(charm_name()) | ||
2713 | 1071 | cert_file = '/usr/local/share/ca-certificates/{}.crt'.format(name) | ||
2714 | 1072 | new_hash = hashlib.md5(ca_cert).hexdigest() | ||
2715 | 1073 | if file_hash(cert_file) == new_hash: | ||
2716 | 1074 | return | ||
2717 | 1075 | log("Installing new CA cert at: {}".format(cert_file), level=INFO) | ||
2718 | 1076 | write_file(cert_file, ca_cert) | ||
2719 | 1077 | subprocess.check_call(['update-ca-certificates', '--fresh']) | ||
2720 | 642 | 1078 | ||
2721 | === added directory 'charmhelpers/core/host_factory' | |||
2722 | === added file 'charmhelpers/core/host_factory/__init__.py' | |||
2723 | === added file 'charmhelpers/core/host_factory/centos.py' | |||
2724 | --- charmhelpers/core/host_factory/centos.py 1970-01-01 00:00:00 +0000 | |||
2725 | +++ charmhelpers/core/host_factory/centos.py 2021-04-13 19:16:05 +0000 | |||
2726 | @@ -0,0 +1,72 @@ | |||
2727 | 1 | import subprocess | ||
2728 | 2 | import yum | ||
2729 | 3 | import os | ||
2730 | 4 | |||
2731 | 5 | from charmhelpers.core.strutils import BasicStringComparator | ||
2732 | 6 | |||
2733 | 7 | |||
2734 | 8 | class CompareHostReleases(BasicStringComparator): | ||
2735 | 9 | """Provide comparisons of Host releases. | ||
2736 | 10 | |||
2737 | 11 | Use in the form of | ||
2738 | 12 | |||
2739 | 13 | if CompareHostReleases(release) > 'trusty': | ||
2740 | 14 | # do something with mitaka | ||
2741 | 15 | """ | ||
2742 | 16 | |||
2743 | 17 | def __init__(self, item): | ||
2744 | 18 | raise NotImplementedError( | ||
2745 | 19 | "CompareHostReleases() is not implemented for CentOS") | ||
2746 | 20 | |||
2747 | 21 | |||
2748 | 22 | def service_available(service_name): | ||
2749 | 23 | # """Determine whether a system service is available.""" | ||
2750 | 24 | if os.path.isdir('/run/systemd/system'): | ||
2751 | 25 | cmd = ['systemctl', 'is-enabled', service_name] | ||
2752 | 26 | else: | ||
2753 | 27 | cmd = ['service', service_name, 'is-enabled'] | ||
2754 | 28 | return subprocess.call(cmd) == 0 | ||
2755 | 29 | |||
2756 | 30 | |||
2757 | 31 | def add_new_group(group_name, system_group=False, gid=None): | ||
2758 | 32 | cmd = ['groupadd'] | ||
2759 | 33 | if gid: | ||
2760 | 34 | cmd.extend(['--gid', str(gid)]) | ||
2761 | 35 | if system_group: | ||
2762 | 36 | cmd.append('-r') | ||
2763 | 37 | cmd.append(group_name) | ||
2764 | 38 | subprocess.check_call(cmd) | ||
2765 | 39 | |||
2766 | 40 | |||
2767 | 41 | def lsb_release(): | ||
2768 | 42 | """Return /etc/os-release in a dict.""" | ||
2769 | 43 | d = {} | ||
2770 | 44 | with open('/etc/os-release', 'r') as lsb: | ||
2771 | 45 | for l in lsb: | ||
2772 | 46 | s = l.split('=') | ||
2773 | 47 | if len(s) != 2: | ||
2774 | 48 | continue | ||
2775 | 49 | d[s[0].strip()] = s[1].strip() | ||
2776 | 50 | return d | ||
2777 | 51 | |||
2778 | 52 | |||
2779 | 53 | def cmp_pkgrevno(package, revno, pkgcache=None): | ||
2780 | 54 | """Compare supplied revno with the revno of the installed package. | ||
2781 | 55 | |||
2782 | 56 | * 1 => Installed revno is greater than supplied arg | ||
2783 | 57 | * 0 => Installed revno is the same as supplied arg | ||
2784 | 58 | * -1 => Installed revno is less than supplied arg | ||
2785 | 59 | |||
2786 | 60 | This function imports YumBase function if the pkgcache argument | ||
2787 | 61 | is None. | ||
2788 | 62 | """ | ||
2789 | 63 | if not pkgcache: | ||
2790 | 64 | y = yum.YumBase() | ||
2791 | 65 | packages = y.doPackageLists() | ||
2792 | 66 | pkgcache = {i.Name: i.version for i in packages['installed']} | ||
2793 | 67 | pkg = pkgcache[package] | ||
2794 | 68 | if pkg > revno: | ||
2795 | 69 | return 1 | ||
2796 | 70 | if pkg < revno: | ||
2797 | 71 | return -1 | ||
2798 | 72 | return 0 | ||
2799 | 0 | 73 | ||
2800 | === added file 'charmhelpers/core/host_factory/ubuntu.py' | |||
2801 | --- charmhelpers/core/host_factory/ubuntu.py 1970-01-01 00:00:00 +0000 | |||
2802 | +++ charmhelpers/core/host_factory/ubuntu.py 2021-04-13 19:16:05 +0000 | |||
2803 | @@ -0,0 +1,114 @@ | |||
2804 | 1 | import subprocess | ||
2805 | 2 | |||
2806 | 3 | from charmhelpers.core.hookenv import cached | ||
2807 | 4 | from charmhelpers.core.strutils import BasicStringComparator | ||
2808 | 5 | |||
2809 | 6 | |||
2810 | 7 | UBUNTU_RELEASES = ( | ||
2811 | 8 | 'lucid', | ||
2812 | 9 | 'maverick', | ||
2813 | 10 | 'natty', | ||
2814 | 11 | 'oneiric', | ||
2815 | 12 | 'precise', | ||
2816 | 13 | 'quantal', | ||
2817 | 14 | 'raring', | ||
2818 | 15 | 'saucy', | ||
2819 | 16 | 'trusty', | ||
2820 | 17 | 'utopic', | ||
2821 | 18 | 'vivid', | ||
2822 | 19 | 'wily', | ||
2823 | 20 | 'xenial', | ||
2824 | 21 | 'yakkety', | ||
2825 | 22 | 'zesty', | ||
2826 | 23 | 'artful', | ||
2827 | 24 | 'bionic', | ||
2828 | 25 | 'cosmic', | ||
2829 | 26 | 'disco', | ||
2830 | 27 | ) | ||
2831 | 28 | |||
2832 | 29 | |||
2833 | 30 | class CompareHostReleases(BasicStringComparator): | ||
2834 | 31 | """Provide comparisons of Ubuntu releases. | ||
2835 | 32 | |||
2836 | 33 | Use in the form of | ||
2837 | 34 | |||
2838 | 35 | if CompareHostReleases(release) > 'trusty': | ||
2839 | 36 | # do something with mitaka | ||
2840 | 37 | """ | ||
2841 | 38 | _list = UBUNTU_RELEASES | ||
2842 | 39 | |||
2843 | 40 | |||
2844 | 41 | def service_available(service_name): | ||
2845 | 42 | """Determine whether a system service is available""" | ||
2846 | 43 | try: | ||
2847 | 44 | subprocess.check_output( | ||
2848 | 45 | ['service', service_name, 'status'], | ||
2849 | 46 | stderr=subprocess.STDOUT).decode('UTF-8') | ||
2850 | 47 | except subprocess.CalledProcessError as e: | ||
2851 | 48 | return b'unrecognized service' not in e.output | ||
2852 | 49 | else: | ||
2853 | 50 | return True | ||
2854 | 51 | |||
2855 | 52 | |||
2856 | 53 | def add_new_group(group_name, system_group=False, gid=None): | ||
2857 | 54 | cmd = ['addgroup'] | ||
2858 | 55 | if gid: | ||
2859 | 56 | cmd.extend(['--gid', str(gid)]) | ||
2860 | 57 | if system_group: | ||
2861 | 58 | cmd.append('--system') | ||
2862 | 59 | else: | ||
2863 | 60 | cmd.extend([ | ||
2864 | 61 | '--group', | ||
2865 | 62 | ]) | ||
2866 | 63 | cmd.append(group_name) | ||
2867 | 64 | subprocess.check_call(cmd) | ||
2868 | 65 | |||
2869 | 66 | |||
2870 | 67 | def lsb_release(): | ||
2871 | 68 | """Return /etc/lsb-release in a dict""" | ||
2872 | 69 | d = {} | ||
2873 | 70 | with open('/etc/lsb-release', 'r') as lsb: | ||
2874 | 71 | for l in lsb: | ||
2875 | 72 | k, v = l.split('=') | ||
2876 | 73 | d[k.strip()] = v.strip() | ||
2877 | 74 | return d | ||
2878 | 75 | |||
2879 | 76 | |||
2880 | 77 | def get_distrib_codename(): | ||
2881 | 78 | """Return the codename of the distribution | ||
2882 | 79 | :returns: The codename | ||
2883 | 80 | :rtype: str | ||
2884 | 81 | """ | ||
2885 | 82 | return lsb_release()['DISTRIB_CODENAME'].lower() | ||
2886 | 83 | |||
2887 | 84 | |||
2888 | 85 | def cmp_pkgrevno(package, revno, pkgcache=None): | ||
2889 | 86 | """Compare supplied revno with the revno of the installed package. | ||
2890 | 87 | |||
2891 | 88 | * 1 => Installed revno is greater than supplied arg | ||
2892 | 89 | * 0 => Installed revno is the same as supplied arg | ||
2893 | 90 | * -1 => Installed revno is less than supplied arg | ||
2894 | 91 | |||
2895 | 92 | This function imports apt_cache function from charmhelpers.fetch if | ||
2896 | 93 | the pkgcache argument is None. Be sure to add charmhelpers.fetch if | ||
2897 | 94 | you call this function, or pass an apt_pkg.Cache() instance. | ||
2898 | 95 | """ | ||
2899 | 96 | import apt_pkg | ||
2900 | 97 | if not pkgcache: | ||
2901 | 98 | from charmhelpers.fetch import apt_cache | ||
2902 | 99 | pkgcache = apt_cache() | ||
2903 | 100 | pkg = pkgcache[package] | ||
2904 | 101 | return apt_pkg.version_compare(pkg.current_ver.ver_str, revno) | ||
2905 | 102 | |||
2906 | 103 | |||
2907 | 104 | @cached | ||
2908 | 105 | def arch(): | ||
2909 | 106 | """Return the package architecture as a string. | ||
2910 | 107 | |||
2911 | 108 | :returns: the architecture | ||
2912 | 109 | :rtype: str | ||
2913 | 110 | :raises: subprocess.CalledProcessError if dpkg command fails | ||
2914 | 111 | """ | ||
2915 | 112 | return subprocess.check_output( | ||
2916 | 113 | ['dpkg', '--print-architecture'] | ||
2917 | 114 | ).rstrip().decode('UTF-8') | ||
2918 | 0 | 115 | ||
2919 | === modified file 'charmhelpers/core/hugepage.py' | |||
2920 | --- charmhelpers/core/hugepage.py 2015-12-11 15:23:38 +0000 | |||
2921 | +++ charmhelpers/core/hugepage.py 2021-04-13 19:16:05 +0000 | |||
2922 | @@ -2,19 +2,17 @@ | |||
2923 | 2 | 2 | ||
2924 | 3 | # Copyright 2014-2015 Canonical Limited. | 3 | # Copyright 2014-2015 Canonical Limited. |
2925 | 4 | # | 4 | # |
2939 | 5 | # This file is part of charm-helpers. | 5 | # Licensed under the Apache License, Version 2.0 (the "License"); |
2940 | 6 | # | 6 | # you may not use this file except in compliance with the License. |
2941 | 7 | # charm-helpers is free software: you can redistribute it and/or modify | 7 | # You may obtain a copy of the License at |
2942 | 8 | # it under the terms of the GNU Lesser General Public License version 3 as | 8 | # |
2943 | 9 | # published by the Free Software Foundation. | 9 | # http://www.apache.org/licenses/LICENSE-2.0 |
2944 | 10 | # | 10 | # |
2945 | 11 | # charm-helpers is distributed in the hope that it will be useful, | 11 | # Unless required by applicable law or agreed to in writing, software |
2946 | 12 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 12 | # distributed under the License is distributed on an "AS IS" BASIS, |
2947 | 13 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 13 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
2948 | 14 | # GNU Lesser General Public License for more details. | 14 | # See the License for the specific language governing permissions and |
2949 | 15 | # | 15 | # limitations under the License. |
2937 | 16 | # You should have received a copy of the GNU Lesser General Public License | ||
2938 | 17 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
2950 | 18 | 16 | ||
2951 | 19 | import yaml | 17 | import yaml |
2952 | 20 | from charmhelpers.core import fstab | 18 | from charmhelpers.core import fstab |
2953 | 21 | 19 | ||
2954 | === modified file 'charmhelpers/core/kernel.py' | |||
2955 | --- charmhelpers/core/kernel.py 2015-12-11 15:23:38 +0000 | |||
2956 | +++ charmhelpers/core/kernel.py 2021-04-13 19:16:05 +0000 | |||
2957 | @@ -3,29 +3,40 @@ | |||
2958 | 3 | 3 | ||
2959 | 4 | # Copyright 2014-2015 Canonical Limited. | 4 | # Copyright 2014-2015 Canonical Limited. |
2960 | 5 | # | 5 | # |
2977 | 6 | # This file is part of charm-helpers. | 6 | # Licensed under the Apache License, Version 2.0 (the "License"); |
2978 | 7 | # | 7 | # you may not use this file except in compliance with the License. |
2979 | 8 | # charm-helpers is free software: you can redistribute it and/or modify | 8 | # You may obtain a copy of the License at |
2980 | 9 | # it under the terms of the GNU Lesser General Public License version 3 as | 9 | # |
2981 | 10 | # published by the Free Software Foundation. | 10 | # http://www.apache.org/licenses/LICENSE-2.0 |
2982 | 11 | # | 11 | # |
2983 | 12 | # charm-helpers is distributed in the hope that it will be useful, | 12 | # Unless required by applicable law or agreed to in writing, software |
2984 | 13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 13 | # distributed under the License is distributed on an "AS IS" BASIS, |
2985 | 14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 14 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
2986 | 15 | # GNU Lesser General Public License for more details. | 15 | # See the License for the specific language governing permissions and |
2987 | 16 | # | 16 | # limitations under the License. |
2988 | 17 | # You should have received a copy of the GNU Lesser General Public License | 17 | |
2989 | 18 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | 18 | import re |
2990 | 19 | 19 | import subprocess | |
2991 | 20 | __author__ = "Jorge Niedbalski <jorge.niedbalski@canonical.com>" | 20 | |
2992 | 21 | 21 | from charmhelpers.osplatform import get_platform | |
2993 | 22 | from charmhelpers.core.hookenv import ( | 22 | from charmhelpers.core.hookenv import ( |
2994 | 23 | log, | 23 | log, |
2995 | 24 | INFO | 24 | INFO |
2996 | 25 | ) | 25 | ) |
2997 | 26 | 26 | ||
3000 | 27 | from subprocess import check_call, check_output | 27 | __platform__ = get_platform() |
3001 | 28 | import re | 28 | if __platform__ == "ubuntu": |
3002 | 29 | from charmhelpers.core.kernel_factory.ubuntu import ( # NOQA:F401 | ||
3003 | 30 | persistent_modprobe, | ||
3004 | 31 | update_initramfs, | ||
3005 | 32 | ) # flake8: noqa -- ignore F401 for this import | ||
3006 | 33 | elif __platform__ == "centos": | ||
3007 | 34 | from charmhelpers.core.kernel_factory.centos import ( # NOQA:F401 | ||
3008 | 35 | persistent_modprobe, | ||
3009 | 36 | update_initramfs, | ||
3010 | 37 | ) # flake8: noqa -- ignore F401 for this import | ||
3011 | 38 | |||
3012 | 39 | __author__ = "Jorge Niedbalski <jorge.niedbalski@canonical.com>" | ||
3013 | 29 | 40 | ||
3014 | 30 | 41 | ||
3015 | 31 | def modprobe(module, persist=True): | 42 | def modprobe(module, persist=True): |
3016 | @@ -34,11 +45,9 @@ | |||
3017 | 34 | 45 | ||
3018 | 35 | log('Loading kernel module %s' % module, level=INFO) | 46 | log('Loading kernel module %s' % module, level=INFO) |
3019 | 36 | 47 | ||
3021 | 37 | check_call(cmd) | 48 | subprocess.check_call(cmd) |
3022 | 38 | if persist: | 49 | if persist: |
3026 | 39 | with open('/etc/modules', 'r+') as modules: | 50 | persistent_modprobe(module) |
3024 | 40 | if module not in modules.read(): | ||
3025 | 41 | modules.write(module) | ||
3027 | 42 | 51 | ||
3028 | 43 | 52 | ||
3029 | 44 | def rmmod(module, force=False): | 53 | def rmmod(module, force=False): |
3030 | @@ -48,21 +57,16 @@ | |||
3031 | 48 | cmd.append('-f') | 57 | cmd.append('-f') |
3032 | 49 | cmd.append(module) | 58 | cmd.append(module) |
3033 | 50 | log('Removing kernel module %s' % module, level=INFO) | 59 | log('Removing kernel module %s' % module, level=INFO) |
3035 | 51 | return check_call(cmd) | 60 | return subprocess.check_call(cmd) |
3036 | 52 | 61 | ||
3037 | 53 | 62 | ||
3038 | 54 | def lsmod(): | 63 | def lsmod(): |
3039 | 55 | """Shows what kernel modules are currently loaded""" | 64 | """Shows what kernel modules are currently loaded""" |
3042 | 56 | return check_output(['lsmod'], | 65 | return subprocess.check_output(['lsmod'], |
3043 | 57 | universal_newlines=True) | 66 | universal_newlines=True) |
3044 | 58 | 67 | ||
3045 | 59 | 68 | ||
3046 | 60 | def is_module_loaded(module): | 69 | def is_module_loaded(module): |
3047 | 61 | """Checks if a kernel module is already loaded""" | 70 | """Checks if a kernel module is already loaded""" |
3048 | 62 | matches = re.findall('^%s[ ]+' % module, lsmod(), re.M) | 71 | matches = re.findall('^%s[ ]+' % module, lsmod(), re.M) |
3049 | 63 | return len(matches) > 0 | 72 | return len(matches) > 0 |
3050 | 64 | |||
3051 | 65 | |||
3052 | 66 | def update_initramfs(version='all'): | ||
3053 | 67 | """Updates an initramfs image""" | ||
3054 | 68 | return check_call(["update-initramfs", "-k", version, "-u"]) | ||
3055 | 69 | 73 | ||
3056 | === added directory 'charmhelpers/core/kernel_factory' | |||
3057 | === added file 'charmhelpers/core/kernel_factory/__init__.py' | |||
3058 | === added file 'charmhelpers/core/kernel_factory/centos.py' | |||
3059 | --- charmhelpers/core/kernel_factory/centos.py 1970-01-01 00:00:00 +0000 | |||
3060 | +++ charmhelpers/core/kernel_factory/centos.py 2021-04-13 19:16:05 +0000 | |||
3061 | @@ -0,0 +1,17 @@ | |||
3062 | 1 | import subprocess | ||
3063 | 2 | import os | ||
3064 | 3 | |||
3065 | 4 | |||
3066 | 5 | def persistent_modprobe(module): | ||
3067 | 6 | """Load a kernel module and configure for auto-load on reboot.""" | ||
3068 | 7 | if not os.path.exists('/etc/rc.modules'): | ||
3069 | 8 | open('/etc/rc.modules', 'a') | ||
3070 | 9 | os.chmod('/etc/rc.modules', 111) | ||
3071 | 10 | with open('/etc/rc.modules', 'r+') as modules: | ||
3072 | 11 | if module not in modules.read(): | ||
3073 | 12 | modules.write('modprobe %s\n' % module) | ||
3074 | 13 | |||
3075 | 14 | |||
3076 | 15 | def update_initramfs(version='all'): | ||
3077 | 16 | """Updates an initramfs image.""" | ||
3078 | 17 | return subprocess.check_call(["dracut", "-f", version]) | ||
3079 | 0 | 18 | ||
3080 | === added file 'charmhelpers/core/kernel_factory/ubuntu.py' | |||
3081 | --- charmhelpers/core/kernel_factory/ubuntu.py 1970-01-01 00:00:00 +0000 | |||
3082 | +++ charmhelpers/core/kernel_factory/ubuntu.py 2021-04-13 19:16:05 +0000 | |||
3083 | @@ -0,0 +1,13 @@ | |||
3084 | 1 | import subprocess | ||
3085 | 2 | |||
3086 | 3 | |||
3087 | 4 | def persistent_modprobe(module): | ||
3088 | 5 | """Load a kernel module and configure for auto-load on reboot.""" | ||
3089 | 6 | with open('/etc/modules', 'r+') as modules: | ||
3090 | 7 | if module not in modules.read(): | ||
3091 | 8 | modules.write(module + "\n") | ||
3092 | 9 | |||
3093 | 10 | |||
3094 | 11 | def update_initramfs(version='all'): | ||
3095 | 12 | """Updates an initramfs image.""" | ||
3096 | 13 | return subprocess.check_call(["update-initramfs", "-k", version, "-u"]) | ||
3097 | 0 | 14 | ||
3098 | === modified file 'charmhelpers/core/services/__init__.py' | |||
3099 | --- charmhelpers/core/services/__init__.py 2015-01-28 08:59:02 +0000 | |||
3100 | +++ charmhelpers/core/services/__init__.py 2021-04-13 19:16:05 +0000 | |||
3101 | @@ -1,18 +1,16 @@ | |||
3102 | 1 | # Copyright 2014-2015 Canonical Limited. | 1 | # Copyright 2014-2015 Canonical Limited. |
3103 | 2 | # | 2 | # |
3117 | 3 | # This file is part of charm-helpers. | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); |
3118 | 4 | # | 4 | # you may not use this file except in compliance with the License. |
3119 | 5 | # charm-helpers is free software: you can redistribute it and/or modify | 5 | # You may obtain a copy of the License at |
3120 | 6 | # it under the terms of the GNU Lesser General Public License version 3 as | 6 | # |
3121 | 7 | # published by the Free Software Foundation. | 7 | # http://www.apache.org/licenses/LICENSE-2.0 |
3122 | 8 | # | 8 | # |
3123 | 9 | # charm-helpers is distributed in the hope that it will be useful, | 9 | # Unless required by applicable law or agreed to in writing, software |
3124 | 10 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 10 | # distributed under the License is distributed on an "AS IS" BASIS, |
3125 | 11 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
3126 | 12 | # GNU Lesser General Public License for more details. | 12 | # See the License for the specific language governing permissions and |
3127 | 13 | # | 13 | # limitations under the License. |
3115 | 14 | # You should have received a copy of the GNU Lesser General Public License | ||
3116 | 15 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
3128 | 16 | 14 | ||
3129 | 17 | from .base import * # NOQA | 15 | from .base import * # NOQA |
3130 | 18 | from .helpers import * # NOQA | 16 | from .helpers import * # NOQA |
3131 | 19 | 17 | ||
3132 | === modified file 'charmhelpers/core/services/base.py' | |||
3133 | --- charmhelpers/core/services/base.py 2015-07-03 09:13:26 +0000 | |||
3134 | +++ charmhelpers/core/services/base.py 2021-04-13 19:16:05 +0000 | |||
3135 | @@ -1,18 +1,16 @@ | |||
3136 | 1 | # Copyright 2014-2015 Canonical Limited. | 1 | # Copyright 2014-2015 Canonical Limited. |
3137 | 2 | # | 2 | # |
3151 | 3 | # This file is part of charm-helpers. | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); |
3152 | 4 | # | 4 | # you may not use this file except in compliance with the License. |
3153 | 5 | # charm-helpers is free software: you can redistribute it and/or modify | 5 | # You may obtain a copy of the License at |
3154 | 6 | # it under the terms of the GNU Lesser General Public License version 3 as | 6 | # |
3155 | 7 | # published by the Free Software Foundation. | 7 | # http://www.apache.org/licenses/LICENSE-2.0 |
3156 | 8 | # | 8 | # |
3157 | 9 | # charm-helpers is distributed in the hope that it will be useful, | 9 | # Unless required by applicable law or agreed to in writing, software |
3158 | 10 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 10 | # distributed under the License is distributed on an "AS IS" BASIS, |
3159 | 11 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
3160 | 12 | # GNU Lesser General Public License for more details. | 12 | # See the License for the specific language governing permissions and |
3161 | 13 | # | 13 | # limitations under the License. |
3149 | 14 | # You should have received a copy of the GNU Lesser General Public License | ||
3150 | 15 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
3162 | 16 | 14 | ||
3163 | 17 | import os | 15 | import os |
3164 | 18 | import json | 16 | import json |
3165 | @@ -309,23 +307,34 @@ | |||
3166 | 309 | """ | 307 | """ |
3167 | 310 | def __call__(self, manager, service_name, event_name): | 308 | def __call__(self, manager, service_name, event_name): |
3168 | 311 | service = manager.get_service(service_name) | 309 | service = manager.get_service(service_name) |
3170 | 312 | new_ports = service.get('ports', []) | 310 | # turn this generator into a list, |
3171 | 311 | # as we'll be going over it multiple times | ||
3172 | 312 | new_ports = list(service.get('ports', [])) | ||
3173 | 313 | port_file = os.path.join(hookenv.charm_dir(), '.{}.ports'.format(service_name)) | 313 | port_file = os.path.join(hookenv.charm_dir(), '.{}.ports'.format(service_name)) |
3174 | 314 | if os.path.exists(port_file): | 314 | if os.path.exists(port_file): |
3175 | 315 | with open(port_file) as fp: | 315 | with open(port_file) as fp: |
3176 | 316 | old_ports = fp.read().split(',') | 316 | old_ports = fp.read().split(',') |
3177 | 317 | for old_port in old_ports: | 317 | for old_port in old_ports: |
3182 | 318 | if bool(old_port): | 318 | if bool(old_port) and not self.ports_contains(old_port, new_ports): |
3183 | 319 | old_port = int(old_port) | 319 | hookenv.close_port(old_port) |
3180 | 320 | if old_port not in new_ports: | ||
3181 | 321 | hookenv.close_port(old_port) | ||
3184 | 322 | with open(port_file, 'w') as fp: | 320 | with open(port_file, 'w') as fp: |
3185 | 323 | fp.write(','.join(str(port) for port in new_ports)) | 321 | fp.write(','.join(str(port) for port in new_ports)) |
3186 | 324 | for port in new_ports: | 322 | for port in new_ports: |
3187 | 323 | # A port is either a number or 'ICMP' | ||
3188 | 324 | protocol = 'TCP' | ||
3189 | 325 | if str(port).upper() == 'ICMP': | ||
3190 | 326 | protocol = 'ICMP' | ||
3191 | 325 | if event_name == 'start': | 327 | if event_name == 'start': |
3193 | 326 | hookenv.open_port(port) | 328 | hookenv.open_port(port, protocol) |
3194 | 327 | elif event_name == 'stop': | 329 | elif event_name == 'stop': |
3196 | 328 | hookenv.close_port(port) | 330 | hookenv.close_port(port, protocol) |
3197 | 331 | |||
3198 | 332 | def ports_contains(self, port, ports): | ||
3199 | 333 | if not bool(port): | ||
3200 | 334 | return False | ||
3201 | 335 | if str(port).upper() != 'ICMP': | ||
3202 | 336 | port = int(port) | ||
3203 | 337 | return port in ports | ||
3204 | 329 | 338 | ||
3205 | 330 | 339 | ||
3206 | 331 | def service_stop(service_name): | 340 | def service_stop(service_name): |
3207 | 332 | 341 | ||
3208 | === modified file 'charmhelpers/core/services/helpers.py' | |||
3209 | --- charmhelpers/core/services/helpers.py 2015-12-11 15:23:38 +0000 | |||
3210 | +++ charmhelpers/core/services/helpers.py 2021-04-13 19:16:05 +0000 | |||
3211 | @@ -1,18 +1,16 @@ | |||
3212 | 1 | # Copyright 2014-2015 Canonical Limited. | 1 | # Copyright 2014-2015 Canonical Limited. |
3213 | 2 | # | 2 | # |
3227 | 3 | # This file is part of charm-helpers. | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); |
3228 | 4 | # | 4 | # you may not use this file except in compliance with the License. |
3229 | 5 | # charm-helpers is free software: you can redistribute it and/or modify | 5 | # You may obtain a copy of the License at |
3230 | 6 | # it under the terms of the GNU Lesser General Public License version 3 as | 6 | # |
3231 | 7 | # published by the Free Software Foundation. | 7 | # http://www.apache.org/licenses/LICENSE-2.0 |
3232 | 8 | # | 8 | # |
3233 | 9 | # charm-helpers is distributed in the hope that it will be useful, | 9 | # Unless required by applicable law or agreed to in writing, software |
3234 | 10 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 10 | # distributed under the License is distributed on an "AS IS" BASIS, |
3235 | 11 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
3236 | 12 | # GNU Lesser General Public License for more details. | 12 | # See the License for the specific language governing permissions and |
3237 | 13 | # | 13 | # limitations under the License. |
3225 | 14 | # You should have received a copy of the GNU Lesser General Public License | ||
3226 | 15 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
3238 | 16 | 14 | ||
3239 | 17 | import os | 15 | import os |
3240 | 18 | import yaml | 16 | import yaml |
3241 | 19 | 17 | ||
3242 | === modified file 'charmhelpers/core/strutils.py' | |||
3243 | --- charmhelpers/core/strutils.py 2015-12-11 15:23:38 +0000 | |||
3244 | +++ charmhelpers/core/strutils.py 2021-04-13 19:16:05 +0000 | |||
3245 | @@ -3,19 +3,17 @@ | |||
3246 | 3 | 3 | ||
3247 | 4 | # Copyright 2014-2015 Canonical Limited. | 4 | # Copyright 2014-2015 Canonical Limited. |
3248 | 5 | # | 5 | # |
3262 | 6 | # This file is part of charm-helpers. | 6 | # Licensed under the Apache License, Version 2.0 (the "License"); |
3263 | 7 | # | 7 | # you may not use this file except in compliance with the License. |
3264 | 8 | # charm-helpers is free software: you can redistribute it and/or modify | 8 | # You may obtain a copy of the License at |
3265 | 9 | # it under the terms of the GNU Lesser General Public License version 3 as | 9 | # |
3266 | 10 | # published by the Free Software Foundation. | 10 | # http://www.apache.org/licenses/LICENSE-2.0 |
3267 | 11 | # | 11 | # |
3268 | 12 | # charm-helpers is distributed in the hope that it will be useful, | 12 | # Unless required by applicable law or agreed to in writing, software |
3269 | 13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 13 | # distributed under the License is distributed on an "AS IS" BASIS, |
3270 | 14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 14 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
3271 | 15 | # GNU Lesser General Public License for more details. | 15 | # See the License for the specific language governing permissions and |
3272 | 16 | # | 16 | # limitations under the License. |
3260 | 17 | # You should have received a copy of the GNU Lesser General Public License | ||
3261 | 18 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
3273 | 19 | 17 | ||
3274 | 20 | import six | 18 | import six |
3275 | 21 | import re | 19 | import re |
3276 | @@ -63,10 +61,69 @@ | |||
3277 | 63 | if isinstance(value, six.string_types): | 61 | if isinstance(value, six.string_types): |
3278 | 64 | value = six.text_type(value) | 62 | value = six.text_type(value) |
3279 | 65 | else: | 63 | else: |
3281 | 66 | msg = "Unable to interpret non-string value '%s' as boolean" % (value) | 64 | msg = "Unable to interpret non-string value '%s' as bytes" % (value) |
3282 | 67 | raise ValueError(msg) | 65 | raise ValueError(msg) |
3283 | 68 | matches = re.match("([0-9]+)([a-zA-Z]+)", value) | 66 | matches = re.match("([0-9]+)([a-zA-Z]+)", value) |
3288 | 69 | if not matches: | 67 | if matches: |
3289 | 70 | msg = "Unable to interpret string value '%s' as bytes" % (value) | 68 | size = int(matches.group(1)) * (1024 ** BYTE_POWER[matches.group(2)]) |
3290 | 71 | raise ValueError(msg) | 69 | else: |
3291 | 72 | return int(matches.group(1)) * (1024 ** BYTE_POWER[matches.group(2)]) | 70 | # Assume that value passed in is bytes |
3292 | 71 | try: | ||
3293 | 72 | size = int(value) | ||
3294 | 73 | except ValueError: | ||
3295 | 74 | msg = "Unable to interpret string value '%s' as bytes" % (value) | ||
3296 | 75 | raise ValueError(msg) | ||
3297 | 76 | return size | ||
3298 | 77 | |||
3299 | 78 | |||
3300 | 79 | class BasicStringComparator(object): | ||
3301 | 80 | """Provides a class that will compare strings from an iterator type object. | ||
3302 | 81 | Used to provide > and < comparisons on strings that may not necessarily be | ||
3303 | 82 | alphanumerically ordered. e.g. OpenStack or Ubuntu releases AFTER the | ||
3304 | 83 | z-wrap. | ||
3305 | 84 | """ | ||
3306 | 85 | |||
3307 | 86 | _list = None | ||
3308 | 87 | |||
3309 | 88 | def __init__(self, item): | ||
3310 | 89 | if self._list is None: | ||
3311 | 90 | raise Exception("Must define the _list in the class definition!") | ||
3312 | 91 | try: | ||
3313 | 92 | self.index = self._list.index(item) | ||
3314 | 93 | except Exception: | ||
3315 | 94 | raise KeyError("Item '{}' is not in list '{}'" | ||
3316 | 95 | .format(item, self._list)) | ||
3317 | 96 | |||
3318 | 97 | def __eq__(self, other): | ||
3319 | 98 | assert isinstance(other, str) or isinstance(other, self.__class__) | ||
3320 | 99 | return self.index == self._list.index(other) | ||
3321 | 100 | |||
3322 | 101 | def __ne__(self, other): | ||
3323 | 102 | return not self.__eq__(other) | ||
3324 | 103 | |||
3325 | 104 | def __lt__(self, other): | ||
3326 | 105 | assert isinstance(other, str) or isinstance(other, self.__class__) | ||
3327 | 106 | return self.index < self._list.index(other) | ||
3328 | 107 | |||
3329 | 108 | def __ge__(self, other): | ||
3330 | 109 | return not self.__lt__(other) | ||
3331 | 110 | |||
3332 | 111 | def __gt__(self, other): | ||
3333 | 112 | assert isinstance(other, str) or isinstance(other, self.__class__) | ||
3334 | 113 | return self.index > self._list.index(other) | ||
3335 | 114 | |||
3336 | 115 | def __le__(self, other): | ||
3337 | 116 | return not self.__gt__(other) | ||
3338 | 117 | |||
3339 | 118 | def __str__(self): | ||
3340 | 119 | """Always give back the item at the index so it can be used in | ||
3341 | 120 | comparisons like: | ||
3342 | 121 | |||
3343 | 122 | s_mitaka = CompareOpenStack('mitaka') | ||
3344 | 123 | s_newton = CompareOpenstack('newton') | ||
3345 | 124 | |||
3346 | 125 | assert s_newton > s_mitaka | ||
3347 | 126 | |||
3348 | 127 | @returns: <string> | ||
3349 | 128 | """ | ||
3350 | 129 | return self._list[self.index] | ||
3351 | 73 | 130 | ||
3352 | === modified file 'charmhelpers/core/sysctl.py' | |||
3353 | --- charmhelpers/core/sysctl.py 2015-03-12 11:42:26 +0000 | |||
3354 | +++ charmhelpers/core/sysctl.py 2021-04-13 19:16:05 +0000 | |||
3355 | @@ -3,19 +3,17 @@ | |||
3356 | 3 | 3 | ||
3357 | 4 | # Copyright 2014-2015 Canonical Limited. | 4 | # Copyright 2014-2015 Canonical Limited. |
3358 | 5 | # | 5 | # |
3372 | 6 | # This file is part of charm-helpers. | 6 | # Licensed under the Apache License, Version 2.0 (the "License"); |
3373 | 7 | # | 7 | # you may not use this file except in compliance with the License. |
3374 | 8 | # charm-helpers is free software: you can redistribute it and/or modify | 8 | # You may obtain a copy of the License at |
3375 | 9 | # it under the terms of the GNU Lesser General Public License version 3 as | 9 | # |
3376 | 10 | # published by the Free Software Foundation. | 10 | # http://www.apache.org/licenses/LICENSE-2.0 |
3377 | 11 | # | 11 | # |
3378 | 12 | # charm-helpers is distributed in the hope that it will be useful, | 12 | # Unless required by applicable law or agreed to in writing, software |
3379 | 13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 13 | # distributed under the License is distributed on an "AS IS" BASIS, |
3380 | 14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 14 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
3381 | 15 | # GNU Lesser General Public License for more details. | 15 | # See the License for the specific language governing permissions and |
3382 | 16 | # | 16 | # limitations under the License. |
3370 | 17 | # You should have received a copy of the GNU Lesser General Public License | ||
3371 | 18 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
3383 | 19 | 17 | ||
3384 | 20 | import yaml | 18 | import yaml |
3385 | 21 | 19 | ||
3386 | @@ -30,27 +28,38 @@ | |||
3387 | 30 | __author__ = 'Jorge Niedbalski R. <jorge.niedbalski@canonical.com>' | 28 | __author__ = 'Jorge Niedbalski R. <jorge.niedbalski@canonical.com>' |
3388 | 31 | 29 | ||
3389 | 32 | 30 | ||
3391 | 33 | def create(sysctl_dict, sysctl_file): | 31 | def create(sysctl_dict, sysctl_file, ignore=False): |
3392 | 34 | """Creates a sysctl.conf file from a YAML associative array | 32 | """Creates a sysctl.conf file from a YAML associative array |
3393 | 35 | 33 | ||
3395 | 36 | :param sysctl_dict: a YAML-formatted string of sysctl options eg "{ 'kernel.max_pid': 1337 }" | 34 | :param sysctl_dict: a dict or YAML-formatted string of sysctl |
3396 | 35 | options eg "{ 'kernel.max_pid': 1337 }" | ||
3397 | 37 | :type sysctl_dict: str | 36 | :type sysctl_dict: str |
3398 | 38 | :param sysctl_file: path to the sysctl file to be saved | 37 | :param sysctl_file: path to the sysctl file to be saved |
3399 | 39 | :type sysctl_file: str or unicode | 38 | :type sysctl_file: str or unicode |
3400 | 39 | :param ignore: If True, ignore "unknown variable" errors. | ||
3401 | 40 | :type ignore: bool | ||
3402 | 40 | :returns: None | 41 | :returns: None |
3403 | 41 | """ | 42 | """ |
3410 | 42 | try: | 43 | if type(sysctl_dict) is not dict: |
3411 | 43 | sysctl_dict_parsed = yaml.safe_load(sysctl_dict) | 44 | try: |
3412 | 44 | except yaml.YAMLError: | 45 | sysctl_dict_parsed = yaml.safe_load(sysctl_dict) |
3413 | 45 | log("Error parsing YAML sysctl_dict: {}".format(sysctl_dict), | 46 | except yaml.YAMLError: |
3414 | 46 | level=ERROR) | 47 | log("Error parsing YAML sysctl_dict: {}".format(sysctl_dict), |
3415 | 47 | return | 48 | level=ERROR) |
3416 | 49 | return | ||
3417 | 50 | else: | ||
3418 | 51 | sysctl_dict_parsed = sysctl_dict | ||
3419 | 48 | 52 | ||
3420 | 49 | with open(sysctl_file, "w") as fd: | 53 | with open(sysctl_file, "w") as fd: |
3421 | 50 | for key, value in sysctl_dict_parsed.items(): | 54 | for key, value in sysctl_dict_parsed.items(): |
3422 | 51 | fd.write("{}={}\n".format(key, value)) | 55 | fd.write("{}={}\n".format(key, value)) |
3423 | 52 | 56 | ||
3425 | 53 | log("Updating sysctl_file: %s values: %s" % (sysctl_file, sysctl_dict_parsed), | 57 | log("Updating sysctl_file: {} values: {}".format(sysctl_file, |
3426 | 58 | sysctl_dict_parsed), | ||
3427 | 54 | level=DEBUG) | 59 | level=DEBUG) |
3428 | 55 | 60 | ||
3430 | 56 | check_call(["sysctl", "-p", sysctl_file]) | 61 | call = ["sysctl", "-p", sysctl_file] |
3431 | 62 | if ignore: | ||
3432 | 63 | call.append("-e") | ||
3433 | 64 | |||
3434 | 65 | check_call(call) | ||
3435 | 57 | 66 | ||
3436 | === modified file 'charmhelpers/core/templating.py' | |||
3437 | --- charmhelpers/core/templating.py 2015-12-11 15:23:38 +0000 | |||
3438 | +++ charmhelpers/core/templating.py 2021-04-13 19:16:05 +0000 | |||
3439 | @@ -1,27 +1,27 @@ | |||
3440 | 1 | # Copyright 2014-2015 Canonical Limited. | 1 | # Copyright 2014-2015 Canonical Limited. |
3441 | 2 | # | 2 | # |
3455 | 3 | # This file is part of charm-helpers. | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); |
3456 | 4 | # | 4 | # you may not use this file except in compliance with the License. |
3457 | 5 | # charm-helpers is free software: you can redistribute it and/or modify | 5 | # You may obtain a copy of the License at |
3458 | 6 | # it under the terms of the GNU Lesser General Public License version 3 as | 6 | # |
3459 | 7 | # published by the Free Software Foundation. | 7 | # http://www.apache.org/licenses/LICENSE-2.0 |
3460 | 8 | # | 8 | # |
3461 | 9 | # charm-helpers is distributed in the hope that it will be useful, | 9 | # Unless required by applicable law or agreed to in writing, software |
3462 | 10 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 10 | # distributed under the License is distributed on an "AS IS" BASIS, |
3463 | 11 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
3464 | 12 | # GNU Lesser General Public License for more details. | 12 | # See the License for the specific language governing permissions and |
3465 | 13 | # | 13 | # limitations under the License. |
3453 | 14 | # You should have received a copy of the GNU Lesser General Public License | ||
3454 | 15 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
3466 | 16 | 14 | ||
3467 | 17 | import os | 15 | import os |
3468 | 16 | import sys | ||
3469 | 18 | 17 | ||
3470 | 19 | from charmhelpers.core import host | 18 | from charmhelpers.core import host |
3471 | 20 | from charmhelpers.core import hookenv | 19 | from charmhelpers.core import hookenv |
3472 | 21 | 20 | ||
3473 | 22 | 21 | ||
3474 | 23 | def render(source, target, context, owner='root', group='root', | 22 | def render(source, target, context, owner='root', group='root', |
3476 | 24 | perms=0o444, templates_dir=None, encoding='UTF-8', template_loader=None): | 23 | perms=0o444, templates_dir=None, encoding='UTF-8', |
3477 | 24 | template_loader=None, config_template=None): | ||
3478 | 25 | """ | 25 | """ |
3479 | 26 | Render a template. | 26 | Render a template. |
3480 | 27 | 27 | ||
3481 | @@ -33,6 +33,9 @@ | |||
3482 | 33 | The context should be a dict containing the values to be replaced in the | 33 | The context should be a dict containing the values to be replaced in the |
3483 | 34 | template. | 34 | template. |
3484 | 35 | 35 | ||
3485 | 36 | config_template may be provided to render from a provided template instead | ||
3486 | 37 | of loading from a file. | ||
3487 | 38 | |||
3488 | 36 | The `owner`, `group`, and `perms` options will be passed to `write_file`. | 39 | The `owner`, `group`, and `perms` options will be passed to `write_file`. |
3489 | 37 | 40 | ||
3490 | 38 | If omitted, `templates_dir` defaults to the `templates` folder in the charm. | 41 | If omitted, `templates_dir` defaults to the `templates` folder in the charm. |
3491 | @@ -40,8 +43,9 @@ | |||
3492 | 40 | The rendered template will be written to the file as well as being returned | 43 | The rendered template will be written to the file as well as being returned |
3493 | 41 | as a string. | 44 | as a string. |
3494 | 42 | 45 | ||
3497 | 43 | Note: Using this requires python-jinja2; if it is not installed, calling | 46 | Note: Using this requires python-jinja2 or python3-jinja2; if it is not |
3498 | 44 | this will attempt to use charmhelpers.fetch.apt_install to install it. | 47 | installed, calling this will attempt to use charmhelpers.fetch.apt_install |
3499 | 48 | to install it. | ||
3500 | 45 | """ | 49 | """ |
3501 | 46 | try: | 50 | try: |
3502 | 47 | from jinja2 import FileSystemLoader, Environment, exceptions | 51 | from jinja2 import FileSystemLoader, Environment, exceptions |
3503 | @@ -53,7 +57,10 @@ | |||
3504 | 53 | 'charmhelpers.fetch to install it', | 57 | 'charmhelpers.fetch to install it', |
3505 | 54 | level=hookenv.ERROR) | 58 | level=hookenv.ERROR) |
3506 | 55 | raise | 59 | raise |
3508 | 56 | apt_install('python-jinja2', fatal=True) | 60 | if sys.version_info.major == 2: |
3509 | 61 | apt_install('python-jinja2', fatal=True) | ||
3510 | 62 | else: | ||
3511 | 63 | apt_install('python3-jinja2', fatal=True) | ||
3512 | 57 | from jinja2 import FileSystemLoader, Environment, exceptions | 64 | from jinja2 import FileSystemLoader, Environment, exceptions |
3513 | 58 | 65 | ||
3514 | 59 | if template_loader: | 66 | if template_loader: |
3515 | @@ -62,14 +69,19 @@ | |||
3516 | 62 | if templates_dir is None: | 69 | if templates_dir is None: |
3517 | 63 | templates_dir = os.path.join(hookenv.charm_dir(), 'templates') | 70 | templates_dir = os.path.join(hookenv.charm_dir(), 'templates') |
3518 | 64 | template_env = Environment(loader=FileSystemLoader(templates_dir)) | 71 | template_env = Environment(loader=FileSystemLoader(templates_dir)) |
3527 | 65 | try: | 72 | |
3528 | 66 | source = source | 73 | # load from a string if provided explicitly |
3529 | 67 | template = template_env.get_template(source) | 74 | if config_template is not None: |
3530 | 68 | except exceptions.TemplateNotFound as e: | 75 | template = template_env.from_string(config_template) |
3531 | 69 | hookenv.log('Could not load template %s from %s.' % | 76 | else: |
3532 | 70 | (source, templates_dir), | 77 | try: |
3533 | 71 | level=hookenv.ERROR) | 78 | source = source |
3534 | 72 | raise e | 79 | template = template_env.get_template(source) |
3535 | 80 | except exceptions.TemplateNotFound as e: | ||
3536 | 81 | hookenv.log('Could not load template %s from %s.' % | ||
3537 | 82 | (source, templates_dir), | ||
3538 | 83 | level=hookenv.ERROR) | ||
3539 | 84 | raise e | ||
3540 | 73 | content = template.render(context) | 85 | content = template.render(context) |
3541 | 74 | if target is not None: | 86 | if target is not None: |
3542 | 75 | target_dir = os.path.dirname(target) | 87 | target_dir = os.path.dirname(target) |
3543 | 76 | 88 | ||
3544 | === modified file 'charmhelpers/core/unitdata.py' | |||
3545 | --- charmhelpers/core/unitdata.py 2015-12-11 15:23:38 +0000 | |||
3546 | +++ charmhelpers/core/unitdata.py 2021-04-13 19:16:05 +0000 | |||
3547 | @@ -3,20 +3,17 @@ | |||
3548 | 3 | # | 3 | # |
3549 | 4 | # Copyright 2014-2015 Canonical Limited. | 4 | # Copyright 2014-2015 Canonical Limited. |
3550 | 5 | # | 5 | # |
3565 | 6 | # This file is part of charm-helpers. | 6 | # Licensed under the Apache License, Version 2.0 (the "License"); |
3566 | 7 | # | 7 | # you may not use this file except in compliance with the License. |
3567 | 8 | # charm-helpers is free software: you can redistribute it and/or modify | 8 | # You may obtain a copy of the License at |
3568 | 9 | # it under the terms of the GNU Lesser General Public License version 3 as | 9 | # |
3569 | 10 | # published by the Free Software Foundation. | 10 | # http://www.apache.org/licenses/LICENSE-2.0 |
3570 | 11 | # | 11 | # |
3571 | 12 | # charm-helpers is distributed in the hope that it will be useful, | 12 | # Unless required by applicable law or agreed to in writing, software |
3572 | 13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 13 | # distributed under the License is distributed on an "AS IS" BASIS, |
3573 | 14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 14 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
3574 | 15 | # GNU Lesser General Public License for more details. | 15 | # See the License for the specific language governing permissions and |
3575 | 16 | # | 16 | # limitations under the License. |
3562 | 17 | # You should have received a copy of the GNU Lesser General Public License | ||
3563 | 18 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
3564 | 19 | # | ||
3576 | 20 | # | 17 | # |
3577 | 21 | # Authors: | 18 | # Authors: |
3578 | 22 | # Kapil Thangavelu <kapil.foss@gmail.com> | 19 | # Kapil Thangavelu <kapil.foss@gmail.com> |
3579 | @@ -169,6 +166,10 @@ | |||
3580 | 169 | 166 | ||
3581 | 170 | To support dicts, lists, integer, floats, and booleans values | 167 | To support dicts, lists, integer, floats, and booleans values |
3582 | 171 | are automatically json encoded/decoded. | 168 | are automatically json encoded/decoded. |
3583 | 169 | |||
3584 | 170 | Note: to facilitate unit testing, ':memory:' can be passed as the | ||
3585 | 171 | path parameter which causes sqlite3 to only build the db in memory. | ||
3586 | 172 | This should only be used for testing purposes. | ||
3587 | 172 | """ | 173 | """ |
3588 | 173 | def __init__(self, path=None): | 174 | def __init__(self, path=None): |
3589 | 174 | self.db_path = path | 175 | self.db_path = path |
3590 | @@ -178,6 +179,9 @@ | |||
3591 | 178 | else: | 179 | else: |
3592 | 179 | self.db_path = os.path.join( | 180 | self.db_path = os.path.join( |
3593 | 180 | os.environ.get('CHARM_DIR', ''), '.unit-state.db') | 181 | os.environ.get('CHARM_DIR', ''), '.unit-state.db') |
3594 | 182 | if self.db_path != ':memory:': | ||
3595 | 183 | with open(self.db_path, 'a') as f: | ||
3596 | 184 | os.fchmod(f.fileno(), 0o600) | ||
3597 | 181 | self.conn = sqlite3.connect('%s' % self.db_path) | 185 | self.conn = sqlite3.connect('%s' % self.db_path) |
3598 | 182 | self.cursor = self.conn.cursor() | 186 | self.cursor = self.conn.cursor() |
3599 | 183 | self.revision = None | 187 | self.revision = None |
3600 | @@ -361,7 +365,7 @@ | |||
3601 | 361 | try: | 365 | try: |
3602 | 362 | yield self.revision | 366 | yield self.revision |
3603 | 363 | self.revision = None | 367 | self.revision = None |
3605 | 364 | except: | 368 | except Exception: |
3606 | 365 | self.flush(False) | 369 | self.flush(False) |
3607 | 366 | self.revision = None | 370 | self.revision = None |
3608 | 367 | raise | 371 | raise |
3609 | 368 | 372 | ||
3610 | === modified file 'charmhelpers/fetch/__init__.py' | |||
3611 | --- charmhelpers/fetch/__init__.py 2015-12-11 15:23:38 +0000 | |||
3612 | +++ charmhelpers/fetch/__init__.py 2021-04-13 19:16:05 +0000 | |||
3613 | @@ -1,32 +1,24 @@ | |||
3614 | 1 | # Copyright 2014-2015 Canonical Limited. | 1 | # Copyright 2014-2015 Canonical Limited. |
3615 | 2 | # | 2 | # |
3629 | 3 | # This file is part of charm-helpers. | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); |
3630 | 4 | # | 4 | # you may not use this file except in compliance with the License. |
3631 | 5 | # charm-helpers is free software: you can redistribute it and/or modify | 5 | # You may obtain a copy of the License at |
3632 | 6 | # it under the terms of the GNU Lesser General Public License version 3 as | 6 | # |
3633 | 7 | # published by the Free Software Foundation. | 7 | # http://www.apache.org/licenses/LICENSE-2.0 |
3634 | 8 | # | 8 | # |
3635 | 9 | # charm-helpers is distributed in the hope that it will be useful, | 9 | # Unless required by applicable law or agreed to in writing, software |
3636 | 10 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 10 | # distributed under the License is distributed on an "AS IS" BASIS, |
3637 | 11 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
3638 | 12 | # GNU Lesser General Public License for more details. | 12 | # See the License for the specific language governing permissions and |
3639 | 13 | # | 13 | # limitations under the License. |
3627 | 14 | # You should have received a copy of the GNU Lesser General Public License | ||
3628 | 15 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
3640 | 16 | 14 | ||
3641 | 17 | import importlib | 15 | import importlib |
3644 | 18 | from tempfile import NamedTemporaryFile | 16 | from charmhelpers.osplatform import get_platform |
3643 | 19 | import time | ||
3645 | 20 | from yaml import safe_load | 17 | from yaml import safe_load |
3646 | 21 | from charmhelpers.core.host import ( | ||
3647 | 22 | lsb_release | ||
3648 | 23 | ) | ||
3649 | 24 | import subprocess | ||
3650 | 25 | from charmhelpers.core.hookenv import ( | 18 | from charmhelpers.core.hookenv import ( |
3651 | 26 | config, | 19 | config, |
3652 | 27 | log, | 20 | log, |
3653 | 28 | ) | 21 | ) |
3654 | 29 | import os | ||
3655 | 30 | 22 | ||
3656 | 31 | import six | 23 | import six |
3657 | 32 | if six.PY3: | 24 | if six.PY3: |
3658 | @@ -35,71 +27,6 @@ | |||
3659 | 35 | from urlparse import urlparse, urlunparse | 27 | from urlparse import urlparse, urlunparse |
3660 | 36 | 28 | ||
3661 | 37 | 29 | ||
3662 | 38 | CLOUD_ARCHIVE = """# Ubuntu Cloud Archive | ||
3663 | 39 | deb http://ubuntu-cloud.archive.canonical.com/ubuntu {} main | ||
3664 | 40 | """ | ||
3665 | 41 | PROPOSED_POCKET = """# Proposed | ||
3666 | 42 | deb http://archive.ubuntu.com/ubuntu {}-proposed main universe multiverse restricted | ||
3667 | 43 | """ | ||
3668 | 44 | CLOUD_ARCHIVE_POCKETS = { | ||
3669 | 45 | # Folsom | ||
3670 | 46 | 'folsom': 'precise-updates/folsom', | ||
3671 | 47 | 'precise-folsom': 'precise-updates/folsom', | ||
3672 | 48 | 'precise-folsom/updates': 'precise-updates/folsom', | ||
3673 | 49 | 'precise-updates/folsom': 'precise-updates/folsom', | ||
3674 | 50 | 'folsom/proposed': 'precise-proposed/folsom', | ||
3675 | 51 | 'precise-folsom/proposed': 'precise-proposed/folsom', | ||
3676 | 52 | 'precise-proposed/folsom': 'precise-proposed/folsom', | ||
3677 | 53 | # Grizzly | ||
3678 | 54 | 'grizzly': 'precise-updates/grizzly', | ||
3679 | 55 | 'precise-grizzly': 'precise-updates/grizzly', | ||
3680 | 56 | 'precise-grizzly/updates': 'precise-updates/grizzly', | ||
3681 | 57 | 'precise-updates/grizzly': 'precise-updates/grizzly', | ||
3682 | 58 | 'grizzly/proposed': 'precise-proposed/grizzly', | ||
3683 | 59 | 'precise-grizzly/proposed': 'precise-proposed/grizzly', | ||
3684 | 60 | 'precise-proposed/grizzly': 'precise-proposed/grizzly', | ||
3685 | 61 | # Havana | ||
3686 | 62 | 'havana': 'precise-updates/havana', | ||
3687 | 63 | 'precise-havana': 'precise-updates/havana', | ||
3688 | 64 | 'precise-havana/updates': 'precise-updates/havana', | ||
3689 | 65 | 'precise-updates/havana': 'precise-updates/havana', | ||
3690 | 66 | 'havana/proposed': 'precise-proposed/havana', | ||
3691 | 67 | 'precise-havana/proposed': 'precise-proposed/havana', | ||
3692 | 68 | 'precise-proposed/havana': 'precise-proposed/havana', | ||
3693 | 69 | # Icehouse | ||
3694 | 70 | 'icehouse': 'precise-updates/icehouse', | ||
3695 | 71 | 'precise-icehouse': 'precise-updates/icehouse', | ||
3696 | 72 | 'precise-icehouse/updates': 'precise-updates/icehouse', | ||
3697 | 73 | 'precise-updates/icehouse': 'precise-updates/icehouse', | ||
3698 | 74 | 'icehouse/proposed': 'precise-proposed/icehouse', | ||
3699 | 75 | 'precise-icehouse/proposed': 'precise-proposed/icehouse', | ||
3700 | 76 | 'precise-proposed/icehouse': 'precise-proposed/icehouse', | ||
3701 | 77 | # Juno | ||
3702 | 78 | 'juno': 'trusty-updates/juno', | ||
3703 | 79 | 'trusty-juno': 'trusty-updates/juno', | ||
3704 | 80 | 'trusty-juno/updates': 'trusty-updates/juno', | ||
3705 | 81 | 'trusty-updates/juno': 'trusty-updates/juno', | ||
3706 | 82 | 'juno/proposed': 'trusty-proposed/juno', | ||
3707 | 83 | 'trusty-juno/proposed': 'trusty-proposed/juno', | ||
3708 | 84 | 'trusty-proposed/juno': 'trusty-proposed/juno', | ||
3709 | 85 | # Kilo | ||
3710 | 86 | 'kilo': 'trusty-updates/kilo', | ||
3711 | 87 | 'trusty-kilo': 'trusty-updates/kilo', | ||
3712 | 88 | 'trusty-kilo/updates': 'trusty-updates/kilo', | ||
3713 | 89 | 'trusty-updates/kilo': 'trusty-updates/kilo', | ||
3714 | 90 | 'kilo/proposed': 'trusty-proposed/kilo', | ||
3715 | 91 | 'trusty-kilo/proposed': 'trusty-proposed/kilo', | ||
3716 | 92 | 'trusty-proposed/kilo': 'trusty-proposed/kilo', | ||
3717 | 93 | # Liberty | ||
3718 | 94 | 'liberty': 'trusty-updates/liberty', | ||
3719 | 95 | 'trusty-liberty': 'trusty-updates/liberty', | ||
3720 | 96 | 'trusty-liberty/updates': 'trusty-updates/liberty', | ||
3721 | 97 | 'trusty-updates/liberty': 'trusty-updates/liberty', | ||
3722 | 98 | 'liberty/proposed': 'trusty-proposed/liberty', | ||
3723 | 99 | 'trusty-liberty/proposed': 'trusty-proposed/liberty', | ||
3724 | 100 | 'trusty-proposed/liberty': 'trusty-proposed/liberty', | ||
3725 | 101 | } | ||
3726 | 102 | |||
3727 | 103 | # The order of this list is very important. Handlers should be listed in from | 30 | # The order of this list is very important. Handlers should be listed in from |
3728 | 104 | # least- to most-specific URL matching. | 31 | # least- to most-specific URL matching. |
3729 | 105 | FETCH_HANDLERS = ( | 32 | FETCH_HANDLERS = ( |
3730 | @@ -108,10 +35,6 @@ | |||
3731 | 108 | 'charmhelpers.fetch.giturl.GitUrlFetchHandler', | 35 | 'charmhelpers.fetch.giturl.GitUrlFetchHandler', |
3732 | 109 | ) | 36 | ) |
3733 | 110 | 37 | ||
3734 | 111 | APT_NO_LOCK = 100 # The return code for "couldn't acquire lock" in APT. | ||
3735 | 112 | APT_NO_LOCK_RETRY_DELAY = 10 # Wait 10 seconds between apt lock checks. | ||
3736 | 113 | APT_NO_LOCK_RETRY_COUNT = 30 # Retry to acquire the lock X times. | ||
3737 | 114 | |||
3738 | 115 | 38 | ||
3739 | 116 | class SourceConfigError(Exception): | 39 | class SourceConfigError(Exception): |
3740 | 117 | pass | 40 | pass |
3741 | @@ -125,6 +48,13 @@ | |||
3742 | 125 | pass | 48 | pass |
3743 | 126 | 49 | ||
3744 | 127 | 50 | ||
3745 | 51 | class GPGKeyError(Exception): | ||
3746 | 52 | """Exception occurs when a GPG key cannot be fetched or used. The message | ||
3747 | 53 | indicates what the problem is. | ||
3748 | 54 | """ | ||
3749 | 55 | pass | ||
3750 | 56 | |||
3751 | 57 | |||
3752 | 128 | class BaseFetchHandler(object): | 58 | class BaseFetchHandler(object): |
3753 | 129 | 59 | ||
3754 | 130 | """Base class for FetchHandler implementations in fetch plugins""" | 60 | """Base class for FetchHandler implementations in fetch plugins""" |
3755 | @@ -149,180 +79,41 @@ | |||
3756 | 149 | return urlunparse(parts) | 79 | return urlunparse(parts) |
3757 | 150 | 80 | ||
3758 | 151 | 81 | ||
3923 | 152 | def filter_installed_packages(packages): | 82 | __platform__ = get_platform() |
3924 | 153 | """Returns a list of packages that require installation""" | 83 | module = "charmhelpers.fetch.%s" % __platform__ |
3925 | 154 | cache = apt_cache() | 84 | fetch = importlib.import_module(module) |
3926 | 155 | _pkgs = [] | 85 | |
3927 | 156 | for package in packages: | 86 | filter_installed_packages = fetch.filter_installed_packages |
3928 | 157 | try: | 87 | filter_missing_packages = fetch.filter_missing_packages |
3929 | 158 | p = cache[package] | 88 | install = fetch.apt_install |
3930 | 159 | p.current_ver or _pkgs.append(package) | 89 | upgrade = fetch.apt_upgrade |
3931 | 160 | except KeyError: | 90 | update = _fetch_update = fetch.apt_update |
3932 | 161 | log('Package {} has no installation candidate.'.format(package), | 91 | purge = fetch.apt_purge |
3933 | 162 | level='WARNING') | 92 | add_source = fetch.add_source |
3934 | 163 | _pkgs.append(package) | 93 | |
3935 | 164 | return _pkgs | 94 | if __platform__ == "ubuntu": |
3936 | 165 | 95 | apt_cache = fetch.apt_cache | |
3937 | 166 | 96 | apt_install = fetch.apt_install | |
3938 | 167 | def apt_cache(in_memory=True): | 97 | apt_update = fetch.apt_update |
3939 | 168 | """Build and return an apt cache""" | 98 | apt_upgrade = fetch.apt_upgrade |
3940 | 169 | from apt import apt_pkg | 99 | apt_purge = fetch.apt_purge |
3941 | 170 | apt_pkg.init() | 100 | apt_autoremove = fetch.apt_autoremove |
3942 | 171 | if in_memory: | 101 | apt_mark = fetch.apt_mark |
3943 | 172 | apt_pkg.config.set("Dir::Cache::pkgcache", "") | 102 | apt_hold = fetch.apt_hold |
3944 | 173 | apt_pkg.config.set("Dir::Cache::srcpkgcache", "") | 103 | apt_unhold = fetch.apt_unhold |
3945 | 174 | return apt_pkg.Cache() | 104 | import_key = fetch.import_key |
3946 | 175 | 105 | get_upstream_version = fetch.get_upstream_version | |
3947 | 176 | 106 | elif __platform__ == "centos": | |
3948 | 177 | def apt_install(packages, options=None, fatal=False): | 107 | yum_search = fetch.yum_search |
3785 | 178 | """Install one or more packages""" | ||
3786 | 179 | if options is None: | ||
3787 | 180 | options = ['--option=Dpkg::Options::=--force-confold'] | ||
3788 | 181 | |||
3789 | 182 | cmd = ['apt-get', '--assume-yes'] | ||
3790 | 183 | cmd.extend(options) | ||
3791 | 184 | cmd.append('install') | ||
3792 | 185 | if isinstance(packages, six.string_types): | ||
3793 | 186 | cmd.append(packages) | ||
3794 | 187 | else: | ||
3795 | 188 | cmd.extend(packages) | ||
3796 | 189 | log("Installing {} with options: {}".format(packages, | ||
3797 | 190 | options)) | ||
3798 | 191 | _run_apt_command(cmd, fatal) | ||
3799 | 192 | |||
3800 | 193 | |||
3801 | 194 | def apt_upgrade(options=None, fatal=False, dist=False): | ||
3802 | 195 | """Upgrade all packages""" | ||
3803 | 196 | if options is None: | ||
3804 | 197 | options = ['--option=Dpkg::Options::=--force-confold'] | ||
3805 | 198 | |||
3806 | 199 | cmd = ['apt-get', '--assume-yes'] | ||
3807 | 200 | cmd.extend(options) | ||
3808 | 201 | if dist: | ||
3809 | 202 | cmd.append('dist-upgrade') | ||
3810 | 203 | else: | ||
3811 | 204 | cmd.append('upgrade') | ||
3812 | 205 | log("Upgrading with options: {}".format(options)) | ||
3813 | 206 | _run_apt_command(cmd, fatal) | ||
3814 | 207 | |||
3815 | 208 | |||
3816 | 209 | def apt_update(fatal=False): | ||
3817 | 210 | """Update local apt cache""" | ||
3818 | 211 | cmd = ['apt-get', 'update'] | ||
3819 | 212 | _run_apt_command(cmd, fatal) | ||
3820 | 213 | |||
3821 | 214 | |||
3822 | 215 | def apt_purge(packages, fatal=False): | ||
3823 | 216 | """Purge one or more packages""" | ||
3824 | 217 | cmd = ['apt-get', '--assume-yes', 'purge'] | ||
3825 | 218 | if isinstance(packages, six.string_types): | ||
3826 | 219 | cmd.append(packages) | ||
3827 | 220 | else: | ||
3828 | 221 | cmd.extend(packages) | ||
3829 | 222 | log("Purging {}".format(packages)) | ||
3830 | 223 | _run_apt_command(cmd, fatal) | ||
3831 | 224 | |||
3832 | 225 | |||
3833 | 226 | def apt_mark(packages, mark, fatal=False): | ||
3834 | 227 | """Flag one or more packages using apt-mark""" | ||
3835 | 228 | log("Marking {} as {}".format(packages, mark)) | ||
3836 | 229 | cmd = ['apt-mark', mark] | ||
3837 | 230 | if isinstance(packages, six.string_types): | ||
3838 | 231 | cmd.append(packages) | ||
3839 | 232 | else: | ||
3840 | 233 | cmd.extend(packages) | ||
3841 | 234 | |||
3842 | 235 | if fatal: | ||
3843 | 236 | subprocess.check_call(cmd, universal_newlines=True) | ||
3844 | 237 | else: | ||
3845 | 238 | subprocess.call(cmd, universal_newlines=True) | ||
3846 | 239 | |||
3847 | 240 | |||
3848 | 241 | def apt_hold(packages, fatal=False): | ||
3849 | 242 | return apt_mark(packages, 'hold', fatal=fatal) | ||
3850 | 243 | |||
3851 | 244 | |||
3852 | 245 | def apt_unhold(packages, fatal=False): | ||
3853 | 246 | return apt_mark(packages, 'unhold', fatal=fatal) | ||
3854 | 247 | |||
3855 | 248 | |||
3856 | 249 | def add_source(source, key=None): | ||
3857 | 250 | """Add a package source to this system. | ||
3858 | 251 | |||
3859 | 252 | @param source: a URL or sources.list entry, as supported by | ||
3860 | 253 | add-apt-repository(1). Examples:: | ||
3861 | 254 | |||
3862 | 255 | ppa:charmers/example | ||
3863 | 256 | deb https://stub:key@private.example.com/ubuntu trusty main | ||
3864 | 257 | |||
3865 | 258 | In addition: | ||
3866 | 259 | 'proposed:' may be used to enable the standard 'proposed' | ||
3867 | 260 | pocket for the release. | ||
3868 | 261 | 'cloud:' may be used to activate official cloud archive pockets, | ||
3869 | 262 | such as 'cloud:icehouse' | ||
3870 | 263 | 'distro' may be used as a noop | ||
3871 | 264 | |||
3872 | 265 | @param key: A key to be added to the system's APT keyring and used | ||
3873 | 266 | to verify the signatures on packages. Ideally, this should be an | ||
3874 | 267 | ASCII format GPG public key including the block headers. A GPG key | ||
3875 | 268 | id may also be used, but be aware that only insecure protocols are | ||
3876 | 269 | available to retrieve the actual public key from a public keyserver | ||
3877 | 270 | placing your Juju environment at risk. ppa and cloud archive keys | ||
3878 | 271 | are securely added automtically, so sould not be provided. | ||
3879 | 272 | """ | ||
3880 | 273 | if source is None: | ||
3881 | 274 | log('Source is not present. Skipping') | ||
3882 | 275 | return | ||
3883 | 276 | |||
3884 | 277 | if (source.startswith('ppa:') or | ||
3885 | 278 | source.startswith('http') or | ||
3886 | 279 | source.startswith('deb ') or | ||
3887 | 280 | source.startswith('cloud-archive:')): | ||
3888 | 281 | subprocess.check_call(['add-apt-repository', '--yes', source]) | ||
3889 | 282 | elif source.startswith('cloud:'): | ||
3890 | 283 | apt_install(filter_installed_packages(['ubuntu-cloud-keyring']), | ||
3891 | 284 | fatal=True) | ||
3892 | 285 | pocket = source.split(':')[-1] | ||
3893 | 286 | if pocket not in CLOUD_ARCHIVE_POCKETS: | ||
3894 | 287 | raise SourceConfigError( | ||
3895 | 288 | 'Unsupported cloud: source option %s' % | ||
3896 | 289 | pocket) | ||
3897 | 290 | actual_pocket = CLOUD_ARCHIVE_POCKETS[pocket] | ||
3898 | 291 | with open('/etc/apt/sources.list.d/cloud-archive.list', 'w') as apt: | ||
3899 | 292 | apt.write(CLOUD_ARCHIVE.format(actual_pocket)) | ||
3900 | 293 | elif source == 'proposed': | ||
3901 | 294 | release = lsb_release()['DISTRIB_CODENAME'] | ||
3902 | 295 | with open('/etc/apt/sources.list.d/proposed.list', 'w') as apt: | ||
3903 | 296 | apt.write(PROPOSED_POCKET.format(release)) | ||
3904 | 297 | elif source == 'distro': | ||
3905 | 298 | pass | ||
3906 | 299 | else: | ||
3907 | 300 | log("Unknown source: {!r}".format(source)) | ||
3908 | 301 | |||
3909 | 302 | if key: | ||
3910 | 303 | if '-----BEGIN PGP PUBLIC KEY BLOCK-----' in key: | ||
3911 | 304 | with NamedTemporaryFile('w+') as key_file: | ||
3912 | 305 | key_file.write(key) | ||
3913 | 306 | key_file.flush() | ||
3914 | 307 | key_file.seek(0) | ||
3915 | 308 | subprocess.check_call(['apt-key', 'add', '-'], stdin=key_file) | ||
3916 | 309 | else: | ||
3917 | 310 | # Note that hkp: is in no way a secure protocol. Using a | ||
3918 | 311 | # GPG key id is pointless from a security POV unless you | ||
3919 | 312 | # absolutely trust your network and DNS. | ||
3920 | 313 | subprocess.check_call(['apt-key', 'adv', '--keyserver', | ||
3921 | 314 | 'hkp://keyserver.ubuntu.com:80', '--recv', | ||
3922 | 315 | key]) | ||
3949 | 316 | 108 | ||
3950 | 317 | 109 | ||
3951 | 318 | def configure_sources(update=False, | 110 | def configure_sources(update=False, |
3952 | 319 | sources_var='install_sources', | 111 | sources_var='install_sources', |
3953 | 320 | keys_var='install_keys'): | 112 | keys_var='install_keys'): |
3956 | 321 | """ | 113 | """Configure multiple sources from charm configuration. |
3955 | 322 | Configure multiple sources from charm configuration. | ||
3957 | 323 | 114 | ||
3958 | 324 | The lists are encoded as yaml fragments in the configuration. | 115 | The lists are encoded as yaml fragments in the configuration. |
3960 | 325 | The frament needs to be included as a string. Sources and their | 116 | The fragment needs to be included as a string. Sources and their |
3961 | 326 | corresponding keys are of the types supported by add_source(). | 117 | corresponding keys are of the types supported by add_source(). |
3962 | 327 | 118 | ||
3963 | 328 | Example config: | 119 | Example config: |
3964 | @@ -354,12 +145,11 @@ | |||
3965 | 354 | for source, key in zip(sources, keys): | 145 | for source, key in zip(sources, keys): |
3966 | 355 | add_source(source, key) | 146 | add_source(source, key) |
3967 | 356 | if update: | 147 | if update: |
3969 | 357 | apt_update(fatal=True) | 148 | _fetch_update(fatal=True) |
3970 | 358 | 149 | ||
3971 | 359 | 150 | ||
3972 | 360 | def install_remote(source, *args, **kwargs): | 151 | def install_remote(source, *args, **kwargs): |
3975 | 361 | """ | 152 | """Install a file tree from a remote source. |
3974 | 362 | Install a file tree from a remote source | ||
3976 | 363 | 153 | ||
3977 | 364 | The specified source should be a url of the form: | 154 | The specified source should be a url of the form: |
3978 | 365 | scheme://[host]/path[#[option=value][&...]] | 155 | scheme://[host]/path[#[option=value][&...]] |
3979 | @@ -382,19 +172,17 @@ | |||
3980 | 382 | # We ONLY check for True here because can_handle may return a string | 172 | # We ONLY check for True here because can_handle may return a string |
3981 | 383 | # explaining why it can't handle a given source. | 173 | # explaining why it can't handle a given source. |
3982 | 384 | handlers = [h for h in plugins() if h.can_handle(source) is True] | 174 | handlers = [h for h in plugins() if h.can_handle(source) is True] |
3983 | 385 | installed_to = None | ||
3984 | 386 | for handler in handlers: | 175 | for handler in handlers: |
3985 | 387 | try: | 176 | try: |
3987 | 388 | installed_to = handler.install(source, *args, **kwargs) | 177 | return handler.install(source, *args, **kwargs) |
3988 | 389 | except UnhandledSource as e: | 178 | except UnhandledSource as e: |
3989 | 390 | log('Install source attempt unsuccessful: {}'.format(e), | 179 | log('Install source attempt unsuccessful: {}'.format(e), |
3990 | 391 | level='WARNING') | 180 | level='WARNING') |
3994 | 392 | if not installed_to: | 181 | raise UnhandledSource("No handler found for source {}".format(source)) |
3992 | 393 | raise UnhandledSource("No handler found for source {}".format(source)) | ||
3993 | 394 | return installed_to | ||
3995 | 395 | 182 | ||
3996 | 396 | 183 | ||
3997 | 397 | def install_from_config(config_var_name): | 184 | def install_from_config(config_var_name): |
3998 | 185 | """Install a file from config.""" | ||
3999 | 398 | charm_config = config() | 186 | charm_config = config() |
4000 | 399 | source = charm_config[config_var_name] | 187 | source = charm_config[config_var_name] |
4001 | 400 | return install_remote(source) | 188 | return install_remote(source) |
4002 | @@ -411,46 +199,9 @@ | |||
4003 | 411 | importlib.import_module(package), | 199 | importlib.import_module(package), |
4004 | 412 | classname) | 200 | classname) |
4005 | 413 | plugin_list.append(handler_class()) | 201 | plugin_list.append(handler_class()) |
4007 | 414 | except (ImportError, AttributeError): | 202 | except NotImplementedError: |
4008 | 415 | # Skip missing plugins so that they can be ommitted from | 203 | # Skip missing plugins so that they can be ommitted from |
4009 | 416 | # installation if desired | 204 | # installation if desired |
4010 | 417 | log("FetchHandler {} not found, skipping plugin".format( | 205 | log("FetchHandler {} not found, skipping plugin".format( |
4011 | 418 | handler_name)) | 206 | handler_name)) |
4012 | 419 | return plugin_list | 207 | return plugin_list |
4013 | 420 | |||
4014 | 421 | |||
4015 | 422 | def _run_apt_command(cmd, fatal=False): | ||
4016 | 423 | """ | ||
4017 | 424 | Run an APT command, checking output and retrying if the fatal flag is set | ||
4018 | 425 | to True. | ||
4019 | 426 | |||
4020 | 427 | :param: cmd: str: The apt command to run. | ||
4021 | 428 | :param: fatal: bool: Whether the command's output should be checked and | ||
4022 | 429 | retried. | ||
4023 | 430 | """ | ||
4024 | 431 | env = os.environ.copy() | ||
4025 | 432 | |||
4026 | 433 | if 'DEBIAN_FRONTEND' not in env: | ||
4027 | 434 | env['DEBIAN_FRONTEND'] = 'noninteractive' | ||
4028 | 435 | |||
4029 | 436 | if fatal: | ||
4030 | 437 | retry_count = 0 | ||
4031 | 438 | result = None | ||
4032 | 439 | |||
4033 | 440 | # If the command is considered "fatal", we need to retry if the apt | ||
4034 | 441 | # lock was not acquired. | ||
4035 | 442 | |||
4036 | 443 | while result is None or result == APT_NO_LOCK: | ||
4037 | 444 | try: | ||
4038 | 445 | result = subprocess.check_call(cmd, env=env) | ||
4039 | 446 | except subprocess.CalledProcessError as e: | ||
4040 | 447 | retry_count = retry_count + 1 | ||
4041 | 448 | if retry_count > APT_NO_LOCK_RETRY_COUNT: | ||
4042 | 449 | raise | ||
4043 | 450 | result = e.returncode | ||
4044 | 451 | log("Couldn't acquire DPKG lock. Will retry in {} seconds." | ||
4045 | 452 | "".format(APT_NO_LOCK_RETRY_DELAY)) | ||
4046 | 453 | time.sleep(APT_NO_LOCK_RETRY_DELAY) | ||
4047 | 454 | |||
4048 | 455 | else: | ||
4049 | 456 | subprocess.call(cmd, env=env) | ||
4050 | 457 | 208 | ||
4051 | === modified file 'charmhelpers/fetch/archiveurl.py' | |||
4052 | --- charmhelpers/fetch/archiveurl.py 2015-12-11 15:23:38 +0000 | |||
4053 | +++ charmhelpers/fetch/archiveurl.py 2021-04-13 19:16:05 +0000 | |||
4054 | @@ -1,18 +1,16 @@ | |||
4055 | 1 | # Copyright 2014-2015 Canonical Limited. | 1 | # Copyright 2014-2015 Canonical Limited. |
4056 | 2 | # | 2 | # |
4070 | 3 | # This file is part of charm-helpers. | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); |
4071 | 4 | # | 4 | # you may not use this file except in compliance with the License. |
4072 | 5 | # charm-helpers is free software: you can redistribute it and/or modify | 5 | # You may obtain a copy of the License at |
4073 | 6 | # it under the terms of the GNU Lesser General Public License version 3 as | 6 | # |
4074 | 7 | # published by the Free Software Foundation. | 7 | # http://www.apache.org/licenses/LICENSE-2.0 |
4075 | 8 | # | 8 | # |
4076 | 9 | # charm-helpers is distributed in the hope that it will be useful, | 9 | # Unless required by applicable law or agreed to in writing, software |
4077 | 10 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 10 | # distributed under the License is distributed on an "AS IS" BASIS, |
4078 | 11 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
4079 | 12 | # GNU Lesser General Public License for more details. | 12 | # See the License for the specific language governing permissions and |
4080 | 13 | # | 13 | # limitations under the License. |
4068 | 14 | # You should have received a copy of the GNU Lesser General Public License | ||
4069 | 15 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
4081 | 16 | 14 | ||
4082 | 17 | import os | 15 | import os |
4083 | 18 | import hashlib | 16 | import hashlib |
4084 | @@ -91,7 +89,7 @@ | |||
4085 | 91 | :param str source: URL pointing to an archive file. | 89 | :param str source: URL pointing to an archive file. |
4086 | 92 | :param str dest: Local path location to download archive file to. | 90 | :param str dest: Local path location to download archive file to. |
4087 | 93 | """ | 91 | """ |
4089 | 94 | # propogate all exceptions | 92 | # propagate all exceptions |
4090 | 95 | # URLError, OSError, etc | 93 | # URLError, OSError, etc |
4091 | 96 | proto, netloc, path, params, query, fragment = urlparse(source) | 94 | proto, netloc, path, params, query, fragment = urlparse(source) |
4092 | 97 | if proto in ('http', 'https'): | 95 | if proto in ('http', 'https'): |
4093 | 98 | 96 | ||
4094 | === modified file 'charmhelpers/fetch/bzrurl.py' | |||
4095 | --- charmhelpers/fetch/bzrurl.py 2015-12-11 15:23:38 +0000 | |||
4096 | +++ charmhelpers/fetch/bzrurl.py 2021-04-13 19:16:05 +0000 | |||
4097 | @@ -1,70 +1,63 @@ | |||
4098 | 1 | # Copyright 2014-2015 Canonical Limited. | 1 | # Copyright 2014-2015 Canonical Limited. |
4099 | 2 | # | 2 | # |
4113 | 3 | # This file is part of charm-helpers. | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); |
4114 | 4 | # | 4 | # you may not use this file except in compliance with the License. |
4115 | 5 | # charm-helpers is free software: you can redistribute it and/or modify | 5 | # You may obtain a copy of the License at |
4116 | 6 | # it under the terms of the GNU Lesser General Public License version 3 as | 6 | # |
4117 | 7 | # published by the Free Software Foundation. | 7 | # http://www.apache.org/licenses/LICENSE-2.0 |
4118 | 8 | # | 8 | # |
4119 | 9 | # charm-helpers is distributed in the hope that it will be useful, | 9 | # Unless required by applicable law or agreed to in writing, software |
4120 | 10 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 10 | # distributed under the License is distributed on an "AS IS" BASIS, |
4121 | 11 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
4122 | 12 | # GNU Lesser General Public License for more details. | 12 | # See the License for the specific language governing permissions and |
4123 | 13 | # | 13 | # limitations under the License. |
4111 | 14 | # You should have received a copy of the GNU Lesser General Public License | ||
4112 | 15 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
4124 | 16 | 14 | ||
4125 | 17 | import os | 15 | import os |
4126 | 16 | from subprocess import STDOUT, check_output | ||
4127 | 18 | from charmhelpers.fetch import ( | 17 | from charmhelpers.fetch import ( |
4128 | 19 | BaseFetchHandler, | 18 | BaseFetchHandler, |
4130 | 20 | UnhandledSource | 19 | UnhandledSource, |
4131 | 20 | filter_installed_packages, | ||
4132 | 21 | install, | ||
4133 | 21 | ) | 22 | ) |
4134 | 22 | from charmhelpers.core.host import mkdir | 23 | from charmhelpers.core.host import mkdir |
4135 | 23 | 24 | ||
4136 | 24 | import six | ||
4137 | 25 | if six.PY3: | ||
4138 | 26 | raise ImportError('bzrlib does not support Python3') | ||
4139 | 27 | 25 | ||
4148 | 28 | try: | 26 | if filter_installed_packages(['bzr']) != []: |
4149 | 29 | from bzrlib.branch import Branch | 27 | install(['bzr']) |
4150 | 30 | from bzrlib import bzrdir, workingtree, errors | 28 | if filter_installed_packages(['bzr']) != []: |
4151 | 31 | except ImportError: | 29 | raise NotImplementedError('Unable to install bzr') |
4144 | 32 | from charmhelpers.fetch import apt_install | ||
4145 | 33 | apt_install("python-bzrlib") | ||
4146 | 34 | from bzrlib.branch import Branch | ||
4147 | 35 | from bzrlib import bzrdir, workingtree, errors | ||
4152 | 36 | 30 | ||
4153 | 37 | 31 | ||
4154 | 38 | class BzrUrlFetchHandler(BaseFetchHandler): | 32 | class BzrUrlFetchHandler(BaseFetchHandler): |
4156 | 39 | """Handler for bazaar branches via generic and lp URLs""" | 33 | """Handler for bazaar branches via generic and lp URLs.""" |
4157 | 34 | |||
4158 | 40 | def can_handle(self, source): | 35 | def can_handle(self, source): |
4159 | 41 | url_parts = self.parse_url(source) | 36 | url_parts = self.parse_url(source) |
4161 | 42 | if url_parts.scheme not in ('bzr+ssh', 'lp'): | 37 | if url_parts.scheme not in ('bzr+ssh', 'lp', ''): |
4162 | 43 | return False | 38 | return False |
4163 | 39 | elif not url_parts.scheme: | ||
4164 | 40 | return os.path.exists(os.path.join(source, '.bzr')) | ||
4165 | 44 | else: | 41 | else: |
4166 | 45 | return True | 42 | return True |
4167 | 46 | 43 | ||
4171 | 47 | def branch(self, source, dest): | 44 | def branch(self, source, dest, revno=None): |
4169 | 48 | url_parts = self.parse_url(source) | ||
4170 | 49 | # If we use lp:branchname scheme we need to load plugins | ||
4172 | 50 | if not self.can_handle(source): | 45 | if not self.can_handle(source): |
4173 | 51 | raise UnhandledSource("Cannot handle {}".format(source)) | 46 | raise UnhandledSource("Cannot handle {}".format(source)) |
4188 | 52 | if url_parts.scheme == "lp": | 47 | cmd_opts = [] |
4189 | 53 | from bzrlib.plugin import load_plugins | 48 | if revno: |
4190 | 54 | load_plugins() | 49 | cmd_opts += ['-r', str(revno)] |
4191 | 55 | try: | 50 | if os.path.exists(dest): |
4192 | 56 | local_branch = bzrdir.BzrDir.create_branch_convenience(dest) | 51 | cmd = ['bzr', 'pull'] |
4193 | 57 | except errors.AlreadyControlDirError: | 52 | cmd += cmd_opts |
4194 | 58 | local_branch = Branch.open(dest) | 53 | cmd += ['--overwrite', '-d', dest, source] |
4195 | 59 | try: | 54 | else: |
4196 | 60 | remote_branch = Branch.open(source) | 55 | cmd = ['bzr', 'branch'] |
4197 | 61 | remote_branch.push(local_branch) | 56 | cmd += cmd_opts |
4198 | 62 | tree = workingtree.WorkingTree.open(dest) | 57 | cmd += [source, dest] |
4199 | 63 | tree.update() | 58 | check_output(cmd, stderr=STDOUT) |
4186 | 64 | except Exception as e: | ||
4187 | 65 | raise e | ||
4200 | 66 | 59 | ||
4202 | 67 | def install(self, source, dest=None): | 60 | def install(self, source, dest=None, revno=None): |
4203 | 68 | url_parts = self.parse_url(source) | 61 | url_parts = self.parse_url(source) |
4204 | 69 | branch_name = url_parts.path.strip("/").split("/")[-1] | 62 | branch_name = url_parts.path.strip("/").split("/")[-1] |
4205 | 70 | if dest: | 63 | if dest: |
4206 | @@ -73,10 +66,11 @@ | |||
4207 | 73 | dest_dir = os.path.join(os.environ.get('CHARM_DIR'), "fetched", | 66 | dest_dir = os.path.join(os.environ.get('CHARM_DIR'), "fetched", |
4208 | 74 | branch_name) | 67 | branch_name) |
4209 | 75 | 68 | ||
4212 | 76 | if not os.path.exists(dest_dir): | 69 | if dest and not os.path.exists(dest): |
4213 | 77 | mkdir(dest_dir, perms=0o755) | 70 | mkdir(dest, perms=0o755) |
4214 | 71 | |||
4215 | 78 | try: | 72 | try: |
4217 | 79 | self.branch(source, dest_dir) | 73 | self.branch(source, dest_dir, revno) |
4218 | 80 | except OSError as e: | 74 | except OSError as e: |
4219 | 81 | raise UnhandledSource(e.strerror) | 75 | raise UnhandledSource(e.strerror) |
4220 | 82 | return dest_dir | 76 | return dest_dir |
4221 | 83 | 77 | ||
4222 | === added file 'charmhelpers/fetch/centos.py' | |||
4223 | --- charmhelpers/fetch/centos.py 1970-01-01 00:00:00 +0000 | |||
4224 | +++ charmhelpers/fetch/centos.py 2021-04-13 19:16:05 +0000 | |||
4225 | @@ -0,0 +1,171 @@ | |||
4226 | 1 | # Copyright 2014-2015 Canonical Limited. | ||
4227 | 2 | # | ||
4228 | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); | ||
4229 | 4 | # you may not use this file except in compliance with the License. | ||
4230 | 5 | # You may obtain a copy of the License at | ||
4231 | 6 | # | ||
4232 | 7 | # http://www.apache.org/licenses/LICENSE-2.0 | ||
4233 | 8 | # | ||
4234 | 9 | # Unless required by applicable law or agreed to in writing, software | ||
4235 | 10 | # distributed under the License is distributed on an "AS IS" BASIS, | ||
4236 | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
4237 | 12 | # See the License for the specific language governing permissions and | ||
4238 | 13 | # limitations under the License. | ||
4239 | 14 | |||
4240 | 15 | import subprocess | ||
4241 | 16 | import os | ||
4242 | 17 | import time | ||
4243 | 18 | import six | ||
4244 | 19 | import yum | ||
4245 | 20 | |||
4246 | 21 | from tempfile import NamedTemporaryFile | ||
4247 | 22 | from charmhelpers.core.hookenv import log | ||
4248 | 23 | |||
4249 | 24 | YUM_NO_LOCK = 1 # The return code for "couldn't acquire lock" in YUM. | ||
4250 | 25 | YUM_NO_LOCK_RETRY_DELAY = 10 # Wait 10 seconds between apt lock checks. | ||
4251 | 26 | YUM_NO_LOCK_RETRY_COUNT = 30 # Retry to acquire the lock X times. | ||
4252 | 27 | |||
4253 | 28 | |||
4254 | 29 | def filter_installed_packages(packages): | ||
4255 | 30 | """Return a list of packages that require installation.""" | ||
4256 | 31 | yb = yum.YumBase() | ||
4257 | 32 | package_list = yb.doPackageLists() | ||
4258 | 33 | temp_cache = {p.base_package_name: 1 for p in package_list['installed']} | ||
4259 | 34 | |||
4260 | 35 | _pkgs = [p for p in packages if not temp_cache.get(p, False)] | ||
4261 | 36 | return _pkgs | ||
4262 | 37 | |||
4263 | 38 | |||
4264 | 39 | def install(packages, options=None, fatal=False): | ||
4265 | 40 | """Install one or more packages.""" | ||
4266 | 41 | cmd = ['yum', '--assumeyes'] | ||
4267 | 42 | if options is not None: | ||
4268 | 43 | cmd.extend(options) | ||
4269 | 44 | cmd.append('install') | ||
4270 | 45 | if isinstance(packages, six.string_types): | ||
4271 | 46 | cmd.append(packages) | ||
4272 | 47 | else: | ||
4273 | 48 | cmd.extend(packages) | ||
4274 | 49 | log("Installing {} with options: {}".format(packages, | ||
4275 | 50 | options)) | ||
4276 | 51 | _run_yum_command(cmd, fatal) | ||
4277 | 52 | |||
4278 | 53 | |||
4279 | 54 | def upgrade(options=None, fatal=False, dist=False): | ||
4280 | 55 | """Upgrade all packages.""" | ||
4281 | 56 | cmd = ['yum', '--assumeyes'] | ||
4282 | 57 | if options is not None: | ||
4283 | 58 | cmd.extend(options) | ||
4284 | 59 | cmd.append('upgrade') | ||
4285 | 60 | log("Upgrading with options: {}".format(options)) | ||
4286 | 61 | _run_yum_command(cmd, fatal) | ||
4287 | 62 | |||
4288 | 63 | |||
4289 | 64 | def update(fatal=False): | ||
4290 | 65 | """Update local yum cache.""" | ||
4291 | 66 | cmd = ['yum', '--assumeyes', 'update'] | ||
4292 | 67 | log("Update with fatal: {}".format(fatal)) | ||
4293 | 68 | _run_yum_command(cmd, fatal) | ||
4294 | 69 | |||
4295 | 70 | |||
4296 | 71 | def purge(packages, fatal=False): | ||
4297 | 72 | """Purge one or more packages.""" | ||
4298 | 73 | cmd = ['yum', '--assumeyes', 'remove'] | ||
4299 | 74 | if isinstance(packages, six.string_types): | ||
4300 | 75 | cmd.append(packages) | ||
4301 | 76 | else: | ||
4302 | 77 | cmd.extend(packages) | ||
4303 | 78 | log("Purging {}".format(packages)) | ||
4304 | 79 | _run_yum_command(cmd, fatal) | ||
4305 | 80 | |||
4306 | 81 | |||
4307 | 82 | def yum_search(packages): | ||
4308 | 83 | """Search for a package.""" | ||
4309 | 84 | output = {} | ||
4310 | 85 | cmd = ['yum', 'search'] | ||
4311 | 86 | if isinstance(packages, six.string_types): | ||
4312 | 87 | cmd.append(packages) | ||
4313 | 88 | else: | ||
4314 | 89 | cmd.extend(packages) | ||
4315 | 90 | log("Searching for {}".format(packages)) | ||
4316 | 91 | result = subprocess.check_output(cmd) | ||
4317 | 92 | for package in list(packages): | ||
4318 | 93 | output[package] = package in result | ||
4319 | 94 | return output | ||
4320 | 95 | |||
4321 | 96 | |||
4322 | 97 | def add_source(source, key=None): | ||
4323 | 98 | """Add a package source to this system. | ||
4324 | 99 | |||
4325 | 100 | @param source: a URL with a rpm package | ||
4326 | 101 | |||
4327 | 102 | @param key: A key to be added to the system's keyring and used | ||
4328 | 103 | to verify the signatures on packages. Ideally, this should be an | ||
4329 | 104 | ASCII format GPG public key including the block headers. A GPG key | ||
4330 | 105 | id may also be used, but be aware that only insecure protocols are | ||
4331 | 106 | available to retrieve the actual public key from a public keyserver | ||
4332 | 107 | placing your Juju environment at risk. | ||
4333 | 108 | """ | ||
4334 | 109 | if source is None: | ||
4335 | 110 | log('Source is not present. Skipping') | ||
4336 | 111 | return | ||
4337 | 112 | |||
4338 | 113 | if source.startswith('http'): | ||
4339 | 114 | directory = '/etc/yum.repos.d/' | ||
4340 | 115 | for filename in os.listdir(directory): | ||
4341 | 116 | with open(directory + filename, 'r') as rpm_file: | ||
4342 | 117 | if source in rpm_file.read(): | ||
4343 | 118 | break | ||
4344 | 119 | else: | ||
4345 | 120 | log("Add source: {!r}".format(source)) | ||
4346 | 121 | # write in the charms.repo | ||
4347 | 122 | with open(directory + 'Charms.repo', 'a') as rpm_file: | ||
4348 | 123 | rpm_file.write('[%s]\n' % source[7:].replace('/', '_')) | ||
4349 | 124 | rpm_file.write('name=%s\n' % source[7:]) | ||
4350 | 125 | rpm_file.write('baseurl=%s\n\n' % source) | ||
4351 | 126 | else: | ||
4352 | 127 | log("Unknown source: {!r}".format(source)) | ||
4353 | 128 | |||
4354 | 129 | if key: | ||
4355 | 130 | if '-----BEGIN PGP PUBLIC KEY BLOCK-----' in key: | ||
4356 | 131 | with NamedTemporaryFile('w+') as key_file: | ||
4357 | 132 | key_file.write(key) | ||
4358 | 133 | key_file.flush() | ||
4359 | 134 | key_file.seek(0) | ||
4360 | 135 | subprocess.check_call(['rpm', '--import', key_file.name]) | ||
4361 | 136 | else: | ||
4362 | 137 | subprocess.check_call(['rpm', '--import', key]) | ||
4363 | 138 | |||
4364 | 139 | |||
4365 | 140 | def _run_yum_command(cmd, fatal=False): | ||
4366 | 141 | """Run an YUM command. | ||
4367 | 142 | |||
4368 | 143 | Checks the output and retry if the fatal flag is set to True. | ||
4369 | 144 | |||
4370 | 145 | :param: cmd: str: The yum command to run. | ||
4371 | 146 | :param: fatal: bool: Whether the command's output should be checked and | ||
4372 | 147 | retried. | ||
4373 | 148 | """ | ||
4374 | 149 | env = os.environ.copy() | ||
4375 | 150 | |||
4376 | 151 | if fatal: | ||
4377 | 152 | retry_count = 0 | ||
4378 | 153 | result = None | ||
4379 | 154 | |||
4380 | 155 | # If the command is considered "fatal", we need to retry if the yum | ||
4381 | 156 | # lock was not acquired. | ||
4382 | 157 | |||
4383 | 158 | while result is None or result == YUM_NO_LOCK: | ||
4384 | 159 | try: | ||
4385 | 160 | result = subprocess.check_call(cmd, env=env) | ||
4386 | 161 | except subprocess.CalledProcessError as e: | ||
4387 | 162 | retry_count = retry_count + 1 | ||
4388 | 163 | if retry_count > YUM_NO_LOCK_RETRY_COUNT: | ||
4389 | 164 | raise | ||
4390 | 165 | result = e.returncode | ||
4391 | 166 | log("Couldn't acquire YUM lock. Will retry in {} seconds." | ||
4392 | 167 | "".format(YUM_NO_LOCK_RETRY_DELAY)) | ||
4393 | 168 | time.sleep(YUM_NO_LOCK_RETRY_DELAY) | ||
4394 | 169 | |||
4395 | 170 | else: | ||
4396 | 171 | subprocess.call(cmd, env=env) | ||
4397 | 0 | 172 | ||
4398 | === modified file 'charmhelpers/fetch/giturl.py' | |||
4399 | --- charmhelpers/fetch/giturl.py 2015-12-11 15:23:38 +0000 | |||
4400 | +++ charmhelpers/fetch/giturl.py 2021-04-13 19:16:05 +0000 | |||
4401 | @@ -1,54 +1,56 @@ | |||
4402 | 1 | # Copyright 2014-2015 Canonical Limited. | 1 | # Copyright 2014-2015 Canonical Limited. |
4403 | 2 | # | 2 | # |
4417 | 3 | # This file is part of charm-helpers. | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); |
4418 | 4 | # | 4 | # you may not use this file except in compliance with the License. |
4419 | 5 | # charm-helpers is free software: you can redistribute it and/or modify | 5 | # You may obtain a copy of the License at |
4420 | 6 | # it under the terms of the GNU Lesser General Public License version 3 as | 6 | # |
4421 | 7 | # published by the Free Software Foundation. | 7 | # http://www.apache.org/licenses/LICENSE-2.0 |
4422 | 8 | # | 8 | # |
4423 | 9 | # charm-helpers is distributed in the hope that it will be useful, | 9 | # Unless required by applicable law or agreed to in writing, software |
4424 | 10 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | 10 | # distributed under the License is distributed on an "AS IS" BASIS, |
4425 | 11 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
4426 | 12 | # GNU Lesser General Public License for more details. | 12 | # See the License for the specific language governing permissions and |
4427 | 13 | # | 13 | # limitations under the License. |
4415 | 14 | # You should have received a copy of the GNU Lesser General Public License | ||
4416 | 15 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
4428 | 16 | 14 | ||
4429 | 17 | import os | 15 | import os |
4430 | 16 | from subprocess import check_output, CalledProcessError, STDOUT | ||
4431 | 18 | from charmhelpers.fetch import ( | 17 | from charmhelpers.fetch import ( |
4432 | 19 | BaseFetchHandler, | 18 | BaseFetchHandler, |
4434 | 20 | UnhandledSource | 19 | UnhandledSource, |
4435 | 20 | filter_installed_packages, | ||
4436 | 21 | install, | ||
4437 | 21 | ) | 22 | ) |
4448 | 22 | from charmhelpers.core.host import mkdir | 23 | |
4449 | 23 | 24 | if filter_installed_packages(['git']) != []: | |
4450 | 24 | try: | 25 | install(['git']) |
4451 | 25 | from git import Repo | 26 | if filter_installed_packages(['git']) != []: |
4452 | 26 | except ImportError: | 27 | raise NotImplementedError('Unable to install git') |
4443 | 27 | from charmhelpers.fetch import apt_install | ||
4444 | 28 | apt_install("python-git") | ||
4445 | 29 | from git import Repo | ||
4446 | 30 | |||
4447 | 31 | from git.exc import GitCommandError # noqa E402 | ||
4453 | 32 | 28 | ||
4454 | 33 | 29 | ||
4455 | 34 | class GitUrlFetchHandler(BaseFetchHandler): | 30 | class GitUrlFetchHandler(BaseFetchHandler): |
4457 | 35 | """Handler for git branches via generic and github URLs""" | 31 | """Handler for git branches via generic and github URLs.""" |
4458 | 32 | |||
4459 | 36 | def can_handle(self, source): | 33 | def can_handle(self, source): |
4460 | 37 | url_parts = self.parse_url(source) | 34 | url_parts = self.parse_url(source) |
4461 | 38 | # TODO (mattyw) no support for ssh git@ yet | 35 | # TODO (mattyw) no support for ssh git@ yet |
4463 | 39 | if url_parts.scheme not in ('http', 'https', 'git'): | 36 | if url_parts.scheme not in ('http', 'https', 'git', ''): |
4464 | 40 | return False | 37 | return False |
4465 | 38 | elif not url_parts.scheme: | ||
4466 | 39 | return os.path.exists(os.path.join(source, '.git')) | ||
4467 | 41 | else: | 40 | else: |
4468 | 42 | return True | 41 | return True |
4469 | 43 | 42 | ||
4471 | 44 | def clone(self, source, dest, branch, depth=None): | 43 | def clone(self, source, dest, branch="master", depth=None): |
4472 | 45 | if not self.can_handle(source): | 44 | if not self.can_handle(source): |
4473 | 46 | raise UnhandledSource("Cannot handle {}".format(source)) | 45 | raise UnhandledSource("Cannot handle {}".format(source)) |
4474 | 47 | 46 | ||
4477 | 48 | if depth: | 47 | if os.path.exists(dest): |
4478 | 49 | Repo.clone_from(source, dest, branch=branch, depth=depth) | 48 | cmd = ['git', '-C', dest, 'pull', source, branch] |
4479 | 50 | else: | 49 | else: |
4481 | 51 | Repo.clone_from(source, dest, branch=branch) | 50 | cmd = ['git', 'clone', source, dest, '--branch', branch] |
4482 | 51 | if depth: | ||
4483 | 52 | cmd.extend(['--depth', depth]) | ||
4484 | 53 | check_output(cmd, stderr=STDOUT) | ||
4485 | 52 | 54 | ||
4486 | 53 | def install(self, source, branch="master", dest=None, depth=None): | 55 | def install(self, source, branch="master", dest=None, depth=None): |
4487 | 54 | url_parts = self.parse_url(source) | 56 | url_parts = self.parse_url(source) |
4488 | @@ -58,11 +60,9 @@ | |||
4489 | 58 | else: | 60 | else: |
4490 | 59 | dest_dir = os.path.join(os.environ.get('CHARM_DIR'), "fetched", | 61 | dest_dir = os.path.join(os.environ.get('CHARM_DIR'), "fetched", |
4491 | 60 | branch_name) | 62 | branch_name) |
4492 | 61 | if not os.path.exists(dest_dir): | ||
4493 | 62 | mkdir(dest_dir, perms=0o755) | ||
4494 | 63 | try: | 63 | try: |
4495 | 64 | self.clone(source, dest_dir, branch, depth) | 64 | self.clone(source, dest_dir, branch, depth) |
4497 | 65 | except GitCommandError as e: | 65 | except CalledProcessError as e: |
4498 | 66 | raise UnhandledSource(e) | 66 | raise UnhandledSource(e) |
4499 | 67 | except OSError as e: | 67 | except OSError as e: |
4500 | 68 | raise UnhandledSource(e.strerror) | 68 | raise UnhandledSource(e.strerror) |
4501 | 69 | 69 | ||
4502 | === added directory 'charmhelpers/fetch/python' | |||
4503 | === added file 'charmhelpers/fetch/python/__init__.py' | |||
4504 | --- charmhelpers/fetch/python/__init__.py 1970-01-01 00:00:00 +0000 | |||
4505 | +++ charmhelpers/fetch/python/__init__.py 2021-04-13 19:16:05 +0000 | |||
4506 | @@ -0,0 +1,13 @@ | |||
4507 | 1 | # Copyright 2014-2019 Canonical Limited. | ||
4508 | 2 | # | ||
4509 | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); | ||
4510 | 4 | # you may not use this file except in compliance with the License. | ||
4511 | 5 | # You may obtain a copy of the License at | ||
4512 | 6 | # | ||
4513 | 7 | # http://www.apache.org/licenses/LICENSE-2.0 | ||
4514 | 8 | # | ||
4515 | 9 | # Unless required by applicable law or agreed to in writing, software | ||
4516 | 10 | # distributed under the License is distributed on an "AS IS" BASIS, | ||
4517 | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
4518 | 12 | # See the License for the specific language governing permissions and | ||
4519 | 13 | # limitations under the License. | ||
4520 | 0 | 14 | ||
4521 | === added file 'charmhelpers/fetch/python/debug.py' | |||
4522 | --- charmhelpers/fetch/python/debug.py 1970-01-01 00:00:00 +0000 | |||
4523 | +++ charmhelpers/fetch/python/debug.py 2021-04-13 19:16:05 +0000 | |||
4524 | @@ -0,0 +1,54 @@ | |||
4525 | 1 | #!/usr/bin/env python | ||
4526 | 2 | # coding: utf-8 | ||
4527 | 3 | |||
4528 | 4 | # Copyright 2014-2015 Canonical Limited. | ||
4529 | 5 | # | ||
4530 | 6 | # Licensed under the Apache License, Version 2.0 (the "License"); | ||
4531 | 7 | # you may not use this file except in compliance with the License. | ||
4532 | 8 | # You may obtain a copy of the License at | ||
4533 | 9 | # | ||
4534 | 10 | # http://www.apache.org/licenses/LICENSE-2.0 | ||
4535 | 11 | # | ||
4536 | 12 | # Unless required by applicable law or agreed to in writing, software | ||
4537 | 13 | # distributed under the License is distributed on an "AS IS" BASIS, | ||
4538 | 14 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
4539 | 15 | # See the License for the specific language governing permissions and | ||
4540 | 16 | # limitations under the License. | ||
4541 | 17 | |||
4542 | 18 | from __future__ import print_function | ||
4543 | 19 | |||
4544 | 20 | import atexit | ||
4545 | 21 | import sys | ||
4546 | 22 | |||
4547 | 23 | from charmhelpers.fetch.python.rpdb import Rpdb | ||
4548 | 24 | from charmhelpers.core.hookenv import ( | ||
4549 | 25 | open_port, | ||
4550 | 26 | close_port, | ||
4551 | 27 | ERROR, | ||
4552 | 28 | log | ||
4553 | 29 | ) | ||
4554 | 30 | |||
4555 | 31 | __author__ = "Jorge Niedbalski <jorge.niedbalski@canonical.com>" | ||
4556 | 32 | |||
4557 | 33 | DEFAULT_ADDR = "0.0.0.0" | ||
4558 | 34 | DEFAULT_PORT = 4444 | ||
4559 | 35 | |||
4560 | 36 | |||
4561 | 37 | def _error(message): | ||
4562 | 38 | log(message, level=ERROR) | ||
4563 | 39 | |||
4564 | 40 | |||
4565 | 41 | def set_trace(addr=DEFAULT_ADDR, port=DEFAULT_PORT): | ||
4566 | 42 | """ | ||
4567 | 43 | Set a trace point using the remote debugger | ||
4568 | 44 | """ | ||
4569 | 45 | atexit.register(close_port, port) | ||
4570 | 46 | try: | ||
4571 | 47 | log("Starting a remote python debugger session on %s:%s" % (addr, | ||
4572 | 48 | port)) | ||
4573 | 49 | open_port(port) | ||
4574 | 50 | debugger = Rpdb(addr=addr, port=port) | ||
4575 | 51 | debugger.set_trace(sys._getframe().f_back) | ||
4576 | 52 | except Exception: | ||
4577 | 53 | _error("Cannot start a remote debug session on %s:%s" % (addr, | ||
4578 | 54 | port)) | ||
4579 | 0 | 55 | ||
4580 | === added file 'charmhelpers/fetch/python/packages.py' | |||
4581 | --- charmhelpers/fetch/python/packages.py 1970-01-01 00:00:00 +0000 | |||
4582 | +++ charmhelpers/fetch/python/packages.py 2021-04-13 19:16:05 +0000 | |||
4583 | @@ -0,0 +1,154 @@ | |||
4584 | 1 | #!/usr/bin/env python | ||
4585 | 2 | # coding: utf-8 | ||
4586 | 3 | |||
4587 | 4 | # Copyright 2014-2015 Canonical Limited. | ||
4588 | 5 | # | ||
4589 | 6 | # Licensed under the Apache License, Version 2.0 (the "License"); | ||
4590 | 7 | # you may not use this file except in compliance with the License. | ||
4591 | 8 | # You may obtain a copy of the License at | ||
4592 | 9 | # | ||
4593 | 10 | # http://www.apache.org/licenses/LICENSE-2.0 | ||
4594 | 11 | # | ||
4595 | 12 | # Unless required by applicable law or agreed to in writing, software | ||
4596 | 13 | # distributed under the License is distributed on an "AS IS" BASIS, | ||
4597 | 14 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
4598 | 15 | # See the License for the specific language governing permissions and | ||
4599 | 16 | # limitations under the License. | ||
4600 | 17 | |||
4601 | 18 | import os | ||
4602 | 19 | import six | ||
4603 | 20 | import subprocess | ||
4604 | 21 | import sys | ||
4605 | 22 | |||
4606 | 23 | from charmhelpers.fetch import apt_install, apt_update | ||
4607 | 24 | from charmhelpers.core.hookenv import charm_dir, log | ||
4608 | 25 | |||
4609 | 26 | __author__ = "Jorge Niedbalski <jorge.niedbalski@canonical.com>" | ||
4610 | 27 | |||
4611 | 28 | |||
4612 | 29 | def pip_execute(*args, **kwargs): | ||
4613 | 30 | """Overriden pip_execute() to stop sys.path being changed. | ||
4614 | 31 | |||
4615 | 32 | The act of importing main from the pip module seems to cause add wheels | ||
4616 | 33 | from the /usr/share/python-wheels which are installed by various tools. | ||
4617 | 34 | This function ensures that sys.path remains the same after the call is | ||
4618 | 35 | executed. | ||
4619 | 36 | """ | ||
4620 | 37 | try: | ||
4621 | 38 | _path = sys.path | ||
4622 | 39 | try: | ||
4623 | 40 | from pip import main as _pip_execute | ||
4624 | 41 | except ImportError: | ||
4625 | 42 | apt_update() | ||
4626 | 43 | if six.PY2: | ||
4627 | 44 | apt_install('python-pip') | ||
4628 | 45 | else: | ||
4629 | 46 | apt_install('python3-pip') | ||
4630 | 47 | from pip import main as _pip_execute | ||
4631 | 48 | _pip_execute(*args, **kwargs) | ||
4632 | 49 | finally: | ||
4633 | 50 | sys.path = _path | ||
4634 | 51 | |||
4635 | 52 | |||
4636 | 53 | def parse_options(given, available): | ||
4637 | 54 | """Given a set of options, check if available""" | ||
4638 | 55 | for key, value in sorted(given.items()): | ||
4639 | 56 | if not value: | ||
4640 | 57 | continue | ||
4641 | 58 | if key in available: | ||
4642 | 59 | yield "--{0}={1}".format(key, value) | ||
4643 | 60 | |||
4644 | 61 | |||
4645 | 62 | def pip_install_requirements(requirements, constraints=None, **options): | ||
4646 | 63 | """Install a requirements file. | ||
4647 | 64 | |||
4648 | 65 | :param constraints: Path to pip constraints file. | ||
4649 | 66 | http://pip.readthedocs.org/en/stable/user_guide/#constraints-files | ||
4650 | 67 | """ | ||
4651 | 68 | command = ["install"] | ||
4652 | 69 | |||
4653 | 70 | available_options = ('proxy', 'src', 'log', ) | ||
4654 | 71 | for option in parse_options(options, available_options): | ||
4655 | 72 | command.append(option) | ||
4656 | 73 | |||
4657 | 74 | command.append("-r {0}".format(requirements)) | ||
4658 | 75 | if constraints: | ||
4659 | 76 | command.append("-c {0}".format(constraints)) | ||
4660 | 77 | log("Installing from file: {} with constraints {} " | ||
4661 | 78 | "and options: {}".format(requirements, constraints, command)) | ||
4662 | 79 | else: | ||
4663 | 80 | log("Installing from file: {} with options: {}".format(requirements, | ||
4664 | 81 | command)) | ||
4665 | 82 | pip_execute(command) | ||
4666 | 83 | |||
4667 | 84 | |||
4668 | 85 | def pip_install(package, fatal=False, upgrade=False, venv=None, | ||
4669 | 86 | constraints=None, **options): | ||
4670 | 87 | """Install a python package""" | ||
4671 | 88 | if venv: | ||
4672 | 89 | venv_python = os.path.join(venv, 'bin/pip') | ||
4673 | 90 | command = [venv_python, "install"] | ||
4674 | 91 | else: | ||
4675 | 92 | command = ["install"] | ||
4676 | 93 | |||
4677 | 94 | available_options = ('proxy', 'src', 'log', 'index-url', ) | ||
4678 | 95 | for option in parse_options(options, available_options): | ||
4679 | 96 | command.append(option) | ||
4680 | 97 | |||
4681 | 98 | if upgrade: | ||
4682 | 99 | command.append('--upgrade') | ||
4683 | 100 | |||
4684 | 101 | if constraints: | ||
4685 | 102 | command.extend(['-c', constraints]) | ||
4686 | 103 | |||
4687 | 104 | if isinstance(package, list): | ||
4688 | 105 | command.extend(package) | ||
4689 | 106 | else: | ||
4690 | 107 | command.append(package) | ||
4691 | 108 | |||
4692 | 109 | log("Installing {} package with options: {}".format(package, | ||
4693 | 110 | command)) | ||
4694 | 111 | if venv: | ||
4695 | 112 | subprocess.check_call(command) | ||
4696 | 113 | else: | ||
4697 | 114 | pip_execute(command) | ||
4698 | 115 | |||
4699 | 116 | |||
4700 | 117 | def pip_uninstall(package, **options): | ||
4701 | 118 | """Uninstall a python package""" | ||
4702 | 119 | command = ["uninstall", "-q", "-y"] | ||
4703 | 120 | |||
4704 | 121 | available_options = ('proxy', 'log', ) | ||
4705 | 122 | for option in parse_options(options, available_options): | ||
4706 | 123 | command.append(option) | ||
4707 | 124 | |||
4708 | 125 | if isinstance(package, list): | ||
4709 | 126 | command.extend(package) | ||
4710 | 127 | else: | ||
4711 | 128 | command.append(package) | ||
4712 | 129 | |||
4713 | 130 | log("Uninstalling {} package with options: {}".format(package, | ||
4714 | 131 | command)) | ||
4715 | 132 | pip_execute(command) | ||
4716 | 133 | |||
4717 | 134 | |||
4718 | 135 | def pip_list(): | ||
4719 | 136 | """Returns the list of current python installed packages | ||
4720 | 137 | """ | ||
4721 | 138 | return pip_execute(["list"]) | ||
4722 | 139 | |||
4723 | 140 | |||
4724 | 141 | def pip_create_virtualenv(path=None): | ||
4725 | 142 | """Create an isolated Python environment.""" | ||
4726 | 143 | if six.PY2: | ||
4727 | 144 | apt_install('python-virtualenv') | ||
4728 | 145 | else: | ||
4729 | 146 | apt_install('python3-virtualenv') | ||
4730 | 147 | |||
4731 | 148 | if path: | ||
4732 | 149 | venv_path = path | ||
4733 | 150 | else: | ||
4734 | 151 | venv_path = os.path.join(charm_dir(), 'venv') | ||
4735 | 152 | |||
4736 | 153 | if not os.path.exists(venv_path): | ||
4737 | 154 | subprocess.check_call(['virtualenv', venv_path]) | ||
4738 | 0 | 155 | ||
4739 | === added file 'charmhelpers/fetch/python/rpdb.py' | |||
4740 | --- charmhelpers/fetch/python/rpdb.py 1970-01-01 00:00:00 +0000 | |||
4741 | +++ charmhelpers/fetch/python/rpdb.py 2021-04-13 19:16:05 +0000 | |||
4742 | @@ -0,0 +1,56 @@ | |||
4743 | 1 | # Copyright 2014-2015 Canonical Limited. | ||
4744 | 2 | # | ||
4745 | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); | ||
4746 | 4 | # you may not use this file except in compliance with the License. | ||
4747 | 5 | # You may obtain a copy of the License at | ||
4748 | 6 | # | ||
4749 | 7 | # http://www.apache.org/licenses/LICENSE-2.0 | ||
4750 | 8 | # | ||
4751 | 9 | # Unless required by applicable law or agreed to in writing, software | ||
4752 | 10 | # distributed under the License is distributed on an "AS IS" BASIS, | ||
4753 | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
4754 | 12 | # See the License for the specific language governing permissions and | ||
4755 | 13 | # limitations under the License. | ||
4756 | 14 | |||
4757 | 15 | """Remote Python Debugger (pdb wrapper).""" | ||
4758 | 16 | |||
4759 | 17 | import pdb | ||
4760 | 18 | import socket | ||
4761 | 19 | import sys | ||
4762 | 20 | |||
4763 | 21 | __author__ = "Bertrand Janin <b@janin.com>" | ||
4764 | 22 | __version__ = "0.1.3" | ||
4765 | 23 | |||
4766 | 24 | |||
4767 | 25 | class Rpdb(pdb.Pdb): | ||
4768 | 26 | |||
4769 | 27 | def __init__(self, addr="127.0.0.1", port=4444): | ||
4770 | 28 | """Initialize the socket and initialize pdb.""" | ||
4771 | 29 | |||
4772 | 30 | # Backup stdin and stdout before replacing them by the socket handle | ||
4773 | 31 | self.old_stdout = sys.stdout | ||
4774 | 32 | self.old_stdin = sys.stdin | ||
4775 | 33 | |||
4776 | 34 | # Open a 'reusable' socket to let the webapp reload on the same port | ||
4777 | 35 | self.skt = socket.socket(socket.AF_INET, socket.SOCK_STREAM) | ||
4778 | 36 | self.skt.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, True) | ||
4779 | 37 | self.skt.bind((addr, port)) | ||
4780 | 38 | self.skt.listen(1) | ||
4781 | 39 | (clientsocket, address) = self.skt.accept() | ||
4782 | 40 | handle = clientsocket.makefile('rw') | ||
4783 | 41 | pdb.Pdb.__init__(self, completekey='tab', stdin=handle, stdout=handle) | ||
4784 | 42 | sys.stdout = sys.stdin = handle | ||
4785 | 43 | |||
4786 | 44 | def shutdown(self): | ||
4787 | 45 | """Revert stdin and stdout, close the socket.""" | ||
4788 | 46 | sys.stdout = self.old_stdout | ||
4789 | 47 | sys.stdin = self.old_stdin | ||
4790 | 48 | self.skt.close() | ||
4791 | 49 | self.set_continue() | ||
4792 | 50 | |||
4793 | 51 | def do_continue(self, arg): | ||
4794 | 52 | """Stop all operation on ``continue``.""" | ||
4795 | 53 | self.shutdown() | ||
4796 | 54 | return 1 | ||
4797 | 55 | |||
4798 | 56 | do_EOF = do_quit = do_exit = do_c = do_cont = do_continue | ||
4799 | 0 | 57 | ||
4800 | === added file 'charmhelpers/fetch/python/version.py' | |||
4801 | --- charmhelpers/fetch/python/version.py 1970-01-01 00:00:00 +0000 | |||
4802 | +++ charmhelpers/fetch/python/version.py 2021-04-13 19:16:05 +0000 | |||
4803 | @@ -0,0 +1,32 @@ | |||
4804 | 1 | #!/usr/bin/env python | ||
4805 | 2 | # coding: utf-8 | ||
4806 | 3 | |||
4807 | 4 | # Copyright 2014-2015 Canonical Limited. | ||
4808 | 5 | # | ||
4809 | 6 | # Licensed under the Apache License, Version 2.0 (the "License"); | ||
4810 | 7 | # you may not use this file except in compliance with the License. | ||
4811 | 8 | # You may obtain a copy of the License at | ||
4812 | 9 | # | ||
4813 | 10 | # http://www.apache.org/licenses/LICENSE-2.0 | ||
4814 | 11 | # | ||
4815 | 12 | # Unless required by applicable law or agreed to in writing, software | ||
4816 | 13 | # distributed under the License is distributed on an "AS IS" BASIS, | ||
4817 | 14 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
4818 | 15 | # See the License for the specific language governing permissions and | ||
4819 | 16 | # limitations under the License. | ||
4820 | 17 | |||
4821 | 18 | import sys | ||
4822 | 19 | |||
4823 | 20 | __author__ = "Jorge Niedbalski <jorge.niedbalski@canonical.com>" | ||
4824 | 21 | |||
4825 | 22 | |||
4826 | 23 | def current_version(): | ||
4827 | 24 | """Current system python version""" | ||
4828 | 25 | return sys.version_info | ||
4829 | 26 | |||
4830 | 27 | |||
4831 | 28 | def current_version_string(): | ||
4832 | 29 | """Current system python version as string major.minor.micro""" | ||
4833 | 30 | return "{0}.{1}.{2}".format(sys.version_info.major, | ||
4834 | 31 | sys.version_info.minor, | ||
4835 | 32 | sys.version_info.micro) | ||
4836 | 0 | 33 | ||
4837 | === added file 'charmhelpers/fetch/snap.py' | |||
4838 | --- charmhelpers/fetch/snap.py 1970-01-01 00:00:00 +0000 | |||
4839 | +++ charmhelpers/fetch/snap.py 2021-04-13 19:16:05 +0000 | |||
4840 | @@ -0,0 +1,150 @@ | |||
4841 | 1 | # Copyright 2014-2017 Canonical Limited. | ||
4842 | 2 | # | ||
4843 | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); | ||
4844 | 4 | # you may not use this file except in compliance with the License. | ||
4845 | 5 | # You may obtain a copy of the License at | ||
4846 | 6 | # | ||
4847 | 7 | # http://www.apache.org/licenses/LICENSE-2.0 | ||
4848 | 8 | # | ||
4849 | 9 | # Unless required by applicable law or agreed to in writing, software | ||
4850 | 10 | # distributed under the License is distributed on an "AS IS" BASIS, | ||
4851 | 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
4852 | 12 | # See the License for the specific language governing permissions and | ||
4853 | 13 | # limitations under the License. | ||
4854 | 14 | """ | ||
4855 | 15 | Charm helpers snap for classic charms. | ||
4856 | 16 | |||
4857 | 17 | If writing reactive charms, use the snap layer: | ||
4858 | 18 | https://lists.ubuntu.com/archives/snapcraft/2016-September/001114.html | ||
4859 | 19 | """ | ||
4860 | 20 | import subprocess | ||
4861 | 21 | import os | ||
4862 | 22 | from time import sleep | ||
4863 | 23 | from charmhelpers.core.hookenv import log | ||
4864 | 24 | |||
4865 | 25 | __author__ = 'Joseph Borg <joseph.borg@canonical.com>' | ||
4866 | 26 | |||
4867 | 27 | # The return code for "couldn't acquire lock" in Snap | ||
4868 | 28 | # (hopefully this will be improved). | ||
4869 | 29 | SNAP_NO_LOCK = 1 | ||
4870 | 30 | SNAP_NO_LOCK_RETRY_DELAY = 10 # Wait X seconds between Snap lock checks. | ||
4871 | 31 | SNAP_NO_LOCK_RETRY_COUNT = 30 # Retry to acquire the lock X times. | ||
4872 | 32 | SNAP_CHANNELS = [ | ||
4873 | 33 | 'edge', | ||
4874 | 34 | 'beta', | ||
4875 | 35 | 'candidate', | ||
4876 | 36 | 'stable', | ||
4877 | 37 | ] | ||
4878 | 38 | |||
4879 | 39 | |||
4880 | 40 | class CouldNotAcquireLockException(Exception): | ||
4881 | 41 | pass | ||
4882 | 42 | |||
4883 | 43 | |||
4884 | 44 | class InvalidSnapChannel(Exception): | ||
4885 | 45 | pass | ||
4886 | 46 | |||
4887 | 47 | |||
4888 | 48 | def _snap_exec(commands): | ||
4889 | 49 | """ | ||
4890 | 50 | Execute snap commands. | ||
4891 | 51 | |||
4892 | 52 | :param commands: List commands | ||
4893 | 53 | :return: Integer exit code | ||
4894 | 54 | """ | ||
4895 | 55 | assert type(commands) == list | ||
4896 | 56 | |||
4897 | 57 | retry_count = 0 | ||
4898 | 58 | return_code = None | ||
4899 | 59 | |||
4900 | 60 | while return_code is None or return_code == SNAP_NO_LOCK: | ||
4901 | 61 | try: | ||
4902 | 62 | return_code = subprocess.check_call(['snap'] + commands, | ||
4903 | 63 | env=os.environ) | ||
4904 | 64 | except subprocess.CalledProcessError as e: | ||
4905 | 65 | retry_count += + 1 | ||
4906 | 66 | if retry_count > SNAP_NO_LOCK_RETRY_COUNT: | ||
4907 | 67 | raise CouldNotAcquireLockException( | ||
4908 | 68 | 'Could not aquire lock after {} attempts' | ||
4909 | 69 | .format(SNAP_NO_LOCK_RETRY_COUNT)) | ||
4910 | 70 | return_code = e.returncode | ||
4911 | 71 | log('Snap failed to acquire lock, trying again in {} seconds.' | ||
4912 | 72 | .format(SNAP_NO_LOCK_RETRY_DELAY, level='WARN')) | ||
4913 | 73 | sleep(SNAP_NO_LOCK_RETRY_DELAY) | ||
4914 | 74 | |||
4915 | 75 | return return_code | ||
4916 | 76 | |||
4917 | 77 | |||
4918 | 78 | def snap_install(packages, *flags): | ||
4919 | 79 | """ | ||
4920 | 80 | Install a snap package. | ||
4921 | 81 | |||
4922 | 82 | :param packages: String or List String package name | ||
4923 | 83 | :param flags: List String flags to pass to install command | ||
4924 | 84 | :return: Integer return code from snap | ||
4925 | 85 | """ | ||
4926 | 86 | if type(packages) is not list: | ||
4927 | 87 | packages = [packages] | ||
4928 | 88 | |||
4929 | 89 | flags = list(flags) | ||
4930 | 90 | |||
4931 | 91 | message = 'Installing snap(s) "%s"' % ', '.join(packages) | ||
4932 | 92 | if flags: | ||
4933 | 93 | message += ' with option(s) "%s"' % ', '.join(flags) | ||
4934 | 94 | |||
4935 | 95 | log(message, level='INFO') | ||
4936 | 96 | return _snap_exec(['install'] + flags + packages) | ||
4937 | 97 | |||
4938 | 98 | |||
4939 | 99 | def snap_remove(packages, *flags): | ||
4940 | 100 | """ | ||
4941 | 101 | Remove a snap package. | ||
4942 | 102 | |||
4943 | 103 | :param packages: String or List String package name | ||
4944 | 104 | :param flags: List String flags to pass to remove command | ||
4945 | 105 | :return: Integer return code from snap | ||
4946 | 106 | """ | ||
4947 | 107 | if type(packages) is not list: | ||
4948 | 108 | packages = [packages] | ||
4949 | 109 | |||
4950 | 110 | flags = list(flags) | ||
4951 | 111 | |||
4952 | 112 | message = 'Removing snap(s) "%s"' % ', '.join(packages) | ||
4953 | 113 | if flags: | ||
4954 | 114 | message += ' with options "%s"' % ', '.join(flags) | ||
4955 | 115 | |||
4956 | 116 | log(message, level='INFO') | ||
4957 | 117 | return _snap_exec(['remove'] + flags + packages) | ||
4958 | 118 | |||
4959 | 119 | |||
4960 | 120 | def snap_refresh(packages, *flags): | ||
4961 | 121 | """ | ||
4962 | 122 | Refresh / Update snap package. | ||
4963 | 123 | |||
4964 | 124 | :param packages: String or List String package name | ||
4965 | 125 | :param flags: List String flags to pass to refresh command | ||
4966 | 126 | :return: Integer return code from snap | ||
4967 | 127 | """ | ||
4968 | 128 | if type(packages) is not list: | ||
4969 | 129 | packages = [packages] | ||
4970 | 130 | |||
4971 | 131 | flags = list(flags) | ||
4972 | 132 | |||
4973 | 133 | message = 'Refreshing snap(s) "%s"' % ', '.join(packages) | ||
4974 | 134 | if flags: | ||
4975 | 135 | message += ' with options "%s"' % ', '.join(flags) | ||
4976 | 136 | |||
4977 | 137 | log(message, level='INFO') | ||
4978 | 138 | return _snap_exec(['refresh'] + flags + packages) | ||
4979 | 139 | |||
4980 | 140 | |||
4981 | 141 | def valid_snap_channel(channel): | ||
4982 | 142 | """ Validate snap channel exists | ||
4983 | 143 | |||
4984 | 144 | :raises InvalidSnapChannel: When channel does not exist | ||
4985 | 145 | :return: Boolean | ||
4986 | 146 | """ | ||
4987 | 147 | if channel.lower() in SNAP_CHANNELS: | ||
4988 | 148 | return True | ||
4989 | 149 | else: | ||
4990 | 150 | raise InvalidSnapChannel("Invalid Snap Channel: {}".format(channel)) | ||
4991 | 0 | 151 | ||
4992 | === added file 'charmhelpers/fetch/ubuntu.py' | |||
4993 | --- charmhelpers/fetch/ubuntu.py 1970-01-01 00:00:00 +0000 | |||
4994 | +++ charmhelpers/fetch/ubuntu.py 2021-04-13 19:16:05 +0000 | |||
4995 | @@ -0,0 +1,730 @@ | |||
4996 | 1 | # Copyright 2014-2015 Canonical Limited. | ||
4997 | 2 | # | ||
4998 | 3 | # Licensed under the Apache License, Version 2.0 (the "License"); | ||
4999 | 4 | # you may not use this file except in compliance with the License. | ||
5000 | 5 | # You may obtain a copy of the License at |
This doesn't seem to point to the right branches. This MP doesn't seem to contain anything related to oidc config.
Please branch from an up-to-date lp:landscape-charm and submit an MP of your branch targeting lp:landscape-charm.