Merge lp:~sajoupa/charms/trusty/wordpress-services/wordpress-services-merge-with-is-charms into lp:~canonical-sysadmins/charms/trusty/wordpress-services/trunk
- Trusty Tahr (14.04)
- wordpress-services-merge-with-is-charms
- Merge into trunk
Proposed by
Laurent Sesquès
Status: | Merged |
---|---|
Merged at revision: | 109 |
Proposed branch: | lp:~sajoupa/charms/trusty/wordpress-services/wordpress-services-merge-with-is-charms |
Merge into: | lp:~canonical-sysadmins/charms/trusty/wordpress-services/trunk |
Diff against target: |
3469 lines (+2091/-370) 29 files modified
Makefile (+15/-0) config.yaml (+37/-0) files/wp-upgrade-check.php (+29/-0) hooks/actions.py (+36/-6) hooks/charmhelpers/contrib/charmsupport/nrpe.py (+111/-23) hooks/charmhelpers/core/files.py (+45/-0) hooks/charmhelpers/core/fstab.py (+4/-4) hooks/charmhelpers/core/hookenv.py (+484/-43) hooks/charmhelpers/core/host.py (+316/-67) hooks/charmhelpers/core/hugepage.py (+71/-0) hooks/charmhelpers/core/kernel.py (+68/-0) hooks/charmhelpers/core/services/base.py (+43/-19) hooks/charmhelpers/core/services/helpers.py (+43/-10) hooks/charmhelpers/core/strutils.py (+72/-0) hooks/charmhelpers/core/sysctl.py (+13/-7) hooks/charmhelpers/core/templating.py (+21/-8) hooks/charmhelpers/core/unitdata.py (+521/-0) hooks/charmhelpers/fetch/__init__.py (+41/-16) hooks/charmhelpers/fetch/archiveurl.py (+18/-12) hooks/charmhelpers/fetch/bzrurl.py (+27/-33) hooks/charmhelpers/fetch/giturl.py (+24/-25) hooks/install (+7/-0) hooks/services.py (+7/-27) hooks/wp_helpers.py (+22/-4) metadata.yaml (+2/-1) templates/wp-apparmor.j2 (+1/-0) templates/wp-info.php.j2 (+13/-0) templates/wp-nagios.j2 (+0/-48) templates/wp-nrpe.j2 (+0/-17) |
To merge this branch: | bzr merge lp:~sajoupa/charms/trusty/wordpress-services/wordpress-services-merge-with-is-charms |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Thomas Cuthbert | Pending | ||
Review via email: mp+291000@code.launchpad.net |
Commit message
Description of the change
To post a comment you must log in.
Revision history for this message
Laurent Sesquès (sajoupa) wrote : | # |
Revision history for this message
Thomas Cuthbert (tcuthbert) wrote : | # |
This looks fine to me, I ran it in mojo-ci with no issues. Are you able to provide a summary of what is changing and I'll approve it.
Preview Diff
[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1 | === added file 'Makefile' | |||
2 | --- Makefile 1970-01-01 00:00:00 +0000 | |||
3 | +++ Makefile 2016-04-05 14:45:53 +0000 | |||
4 | @@ -0,0 +1,15 @@ | |||
5 | 1 | #!/usr/bin/make | ||
6 | 2 | HOOKS_DIR := $(PWD)/hooks | ||
7 | 3 | TEST_PREFIX := PYTHONPATH=$(HOOKS_DIR) | ||
8 | 4 | |||
9 | 5 | lint: | ||
10 | 6 | flake8 --exclude hooks/charmhelpers --ignore=E501 hooks tests | ||
11 | 7 | @charm proof | ||
12 | 8 | |||
13 | 9 | bin/charm_helpers_sync.py: | ||
14 | 10 | @mkdir -p bin | ||
15 | 11 | @bzr cat lp:charm-helpers/tools/charm_helpers_sync/charm_helpers_sync.py \ | ||
16 | 12 | > bin/charm_helpers_sync.py | ||
17 | 13 | |||
18 | 14 | sync: bin/charm_helpers_sync.py | ||
19 | 15 | @python bin/charm_helpers_sync.py -c charm-helpers.yaml | ||
20 | 0 | 16 | ||
21 | === modified file 'config.yaml' | |||
22 | --- config.yaml 2015-02-25 17:05:27 +0000 | |||
23 | +++ config.yaml 2016-04-05 14:45:53 +0000 | |||
24 | @@ -46,6 +46,12 @@ | |||
25 | 46 | 46 | ||
26 | 47 | If you're running multiple environments with the same services in them | 47 | If you're running multiple environments with the same services in them |
27 | 48 | this allows you to differentiate between them. | 48 | this allows you to differentiate between them. |
28 | 49 | nagios_servicegroups: | ||
29 | 50 | default: "" | ||
30 | 51 | type: string | ||
31 | 52 | description: | | ||
32 | 53 | A comma-separated list of nagios servicegroups. | ||
33 | 54 | If left empty, the nagios_context will be used as the servicegroup | ||
34 | 49 | nagios_check_string: | 55 | nagios_check_string: |
35 | 50 | default: "Proudly powered by WordPress" | 56 | default: "Proudly powered by WordPress" |
36 | 51 | type: string | 57 | type: string |
37 | @@ -77,3 +83,34 @@ | |||
38 | 77 | 83 | ||
39 | 78 | If admin_password is not provided it will be automatically generated | 84 | If admin_password is not provided it will be automatically generated |
40 | 79 | and stored on wordpress unit in the charm directory | 85 | and stored on wordpress unit in the charm directory |
41 | 86 | outbound_http_proxy: | ||
42 | 87 | default: "" | ||
43 | 88 | type: string | ||
44 | 89 | description: > | ||
45 | 90 | Optional URL specifying a "forward" proxy to allow wordpress and its | ||
46 | 91 | plugins access to the Web. As an example: | ||
47 | 92 | |||
48 | 93 | outbound_http_proxy: http://user:pass@squid.example.com:3128/ | ||
49 | 94 | redirects: | ||
50 | 95 | default: "" | ||
51 | 96 | type: string | ||
52 | 97 | description: > | ||
53 | 98 | Optional YAML formatted list of redirects that will be added to | ||
54 | 99 | apache vhost. For example setting this optino to: | ||
55 | 100 | |||
56 | 101 | [{"match": "(.*)\.gif$", "target": "http://example.com$1.jpg"}, | ||
57 | 102 | {"match": "/old", "target": "http://example.com/new/", "type": "permanent"}] | ||
58 | 103 | |||
59 | 104 | Will result in the following configuration stanzas if apache2-subordinate is used: | ||
60 | 105 | RedirectMatch (.*)\.gif$ http://example.com$1.jpg | ||
61 | 106 | RedirectMatch permanent /old http://example.com/new | ||
62 | 107 | vhost_options: | ||
63 | 108 | default: "" | ||
64 | 109 | type: string | ||
65 | 110 | description: > | ||
66 | 111 | Optional YAML formatted list of additional virtual host config directives. | ||
67 | 112 | For example: | ||
68 | 113 | |||
69 | 114 | [{"Header": "append Cache-Control \"proxy-revalidate\""}, | ||
70 | 115 | {"Header": "unset ETag"}, | ||
71 | 116 | {"ExpiresDefault": "\"access plus 1 days\""}] | ||
72 | 80 | 117 | ||
73 | === added file 'files/wp-upgrade-check.php' | |||
74 | --- files/wp-upgrade-check.php 1970-01-01 00:00:00 +0000 | |||
75 | +++ files/wp-upgrade-check.php 2016-04-05 14:45:53 +0000 | |||
76 | @@ -0,0 +1,29 @@ | |||
77 | 1 | <?php | ||
78 | 2 | |||
79 | 3 | require_once('wp-load.php'); | ||
80 | 4 | |||
81 | 5 | global $wp_version; | ||
82 | 6 | $core_updates = 0; | ||
83 | 7 | |||
84 | 8 | wp_version_check(); | ||
85 | 9 | |||
86 | 10 | $core = get_site_transient('update_core'); | ||
87 | 11 | |||
88 | 12 | foreach($core->updates as $update) { | ||
89 | 13 | if($update->current != $wp_version) { | ||
90 | 14 | $core_updates++; | ||
91 | 15 | } | ||
92 | 16 | } | ||
93 | 17 | |||
94 | 18 | if($core_updates) { | ||
95 | 19 | print("CRITICAL : $core_updates core updates available !\n"); | ||
96 | 20 | exit(2); | ||
97 | 21 | } else { | ||
98 | 22 | print("OK : no core update available\n"); | ||
99 | 23 | exit(0); | ||
100 | 24 | } | ||
101 | 25 | |||
102 | 26 | print("CRITICAL : Error in " . __FILE__ . "\n"); | ||
103 | 27 | exit(2); | ||
104 | 28 | |||
105 | 29 | ?> | ||
106 | 0 | 30 | ||
107 | === modified file 'hooks/actions.py' | |||
108 | --- hooks/actions.py 2015-02-25 17:05:27 +0000 | |||
109 | +++ hooks/actions.py 2016-04-05 14:45:53 +0000 | |||
110 | @@ -30,6 +30,8 @@ | |||
111 | 30 | host.rsync(upstream_code + '/', | 30 | host.rsync(upstream_code + '/', |
112 | 31 | config['install_path'], | 31 | config['install_path'], |
113 | 32 | options=['--executability']) # Because we don't want --delete | 32 | options=['--executability']) # Because we don't want --delete |
114 | 33 | host.rsync('files/wp-upgrade-check.php', | ||
115 | 34 | os.path.join(config['install_path'], 'wp-upgrade-check.php')) | ||
116 | 33 | host.mkdir('{}/wp-content/uploads'.format(config['install_path']), | 35 | host.mkdir('{}/wp-content/uploads'.format(config['install_path']), |
117 | 34 | owner='www-data', perms=0755) | 36 | owner='www-data', perms=0755) |
118 | 35 | if wp_helpers.wordpress_configured(): | 37 | if wp_helpers.wordpress_configured(): |
119 | @@ -38,7 +40,10 @@ | |||
120 | 38 | 40 | ||
121 | 39 | 41 | ||
122 | 40 | def install_packages(service_name): | 42 | def install_packages(service_name): |
124 | 41 | packages = ['php5-cli', 'php5-mysql', 'php-symfony-yaml'] | 43 | if host.lsb_release()['DISTRIB_CODENAME'] == 'trusty': |
125 | 44 | packages = ['php5-cli', 'php5-mysql', 'php-symfony-yaml', 'php5-curl'] | ||
126 | 45 | else: | ||
127 | 46 | packages = ['php-cli', 'php-mysql', 'php-symfony-yaml', 'php-curl', 'libapache2-mod-php'] | ||
128 | 42 | fetch.apt_update() | 47 | fetch.apt_update() |
129 | 43 | fetch.apt_install(packages) | 48 | fetch.apt_install(packages) |
130 | 44 | 49 | ||
131 | @@ -55,7 +60,7 @@ | |||
132 | 55 | stdin=subprocess.PIPE, | 60 | stdin=subprocess.PIPE, |
133 | 56 | stdout=subprocess.PIPE, | 61 | stdout=subprocess.PIPE, |
134 | 57 | stderr=subprocess.STDOUT, | 62 | stderr=subprocess.STDOUT, |
136 | 58 | ) | 63 | ) |
137 | 59 | return process.communicate(stdin)[0] # spit back stdout+stderr combined | 64 | return process.communicate(stdin)[0] # spit back stdout+stderr combined |
138 | 60 | 65 | ||
139 | 61 | 66 | ||
140 | @@ -122,7 +127,7 @@ | |||
141 | 122 | """Perform initial configuratin of wordpress if needed.""" | 127 | """Perform initial configuratin of wordpress if needed.""" |
142 | 123 | config = hookenv.config() | 128 | config = hookenv.config() |
143 | 124 | if wp_helpers.wordpress_configured() or not config['initial_settings']: | 129 | if wp_helpers.wordpress_configured() or not config['initial_settings']: |
145 | 125 | hookenv.log('No initial_setting provided or wordprass already ' | 130 | hookenv.log('No initial_setting provided or wordpress already ' |
146 | 126 | 'configured. Skipping first install.') | 131 | 'configured. Skipping first install.') |
147 | 127 | return | 132 | return |
148 | 128 | hookenv.log('Starting wordpress initial configuration') | 133 | hookenv.log('Starting wordpress initial configuration') |
149 | @@ -183,7 +188,7 @@ | |||
150 | 183 | def write_nrpe_checks(service_name): | 188 | def write_nrpe_checks(service_name): |
151 | 184 | config = hookenv.config() | 189 | config = hookenv.config() |
152 | 185 | relation = wp_helpers.NEMRelation()['nrpe-external-master'][0] | 190 | relation = wp_helpers.NEMRelation()['nrpe-external-master'][0] |
154 | 186 | nrpe = NRPE(hostname=relation['nagios_hostname']) | 191 | nrpe = NRPE(hostname=relation['nagios_hostname'], primary=True) |
155 | 187 | 192 | ||
156 | 188 | nrpe.add_check( | 193 | nrpe.add_check( |
157 | 189 | shortname='wordpress_http', | 194 | shortname='wordpress_http', |
158 | @@ -198,11 +203,36 @@ | |||
159 | 198 | check_cmd='check_http -I localhost -H {} -p {} -S'.format( | 203 | check_cmd='check_http -I localhost -H {} -p {} -S'.format( |
160 | 199 | config['blog_hostname'], config['ssl_port_number']) | 204 | config['blog_hostname'], config['ssl_port_number']) |
161 | 200 | ) | 205 | ) |
162 | 206 | nrpe.add_check( | ||
163 | 207 | shortname='wordpress_upgrades', | ||
164 | 208 | description='Check Wordpress core upgrades', | ||
165 | 209 | check_cmd='/usr/bin/php {}/wp-upgrade-check.php'.format( | ||
166 | 210 | config['install_path']) | ||
167 | 211 | ) | ||
168 | 212 | |||
169 | 213 | plugin_relations = wp_helpers.PluginRelation()['wordpress-plugin'] | ||
170 | 214 | for plugin in plugin_relations: | ||
171 | 215 | plugin_name = plugin.get('plugin_name') | ||
172 | 216 | if plugin_name: | ||
173 | 217 | nrpe.add_check( | ||
174 | 218 | shortname='wordpress_plugin_{}'.format(plugin_name), | ||
175 | 219 | description='Check Wordpress Plugin {}'.format(plugin_name), | ||
176 | 220 | check_cmd='check_file_age -w 31536000 -c 33696000 -f ' | ||
177 | 221 | '{}/wp-content/plugins/{}' | ||
178 | 222 | ''.format(config['install_path'], plugin_name) | ||
179 | 223 | |||
180 | 224 | ) | ||
181 | 225 | |||
182 | 201 | nrpe.write() | 226 | nrpe.write() |
183 | 202 | 227 | ||
184 | 203 | 228 | ||
185 | 204 | def wipe_nrpe_checks(service_name): | 229 | def wipe_nrpe_checks(service_name): |
187 | 205 | os.unlink('/var/lib/nagios/export/{}.cfg'.format(hookenv.local_unit())) | 230 | for f in glob.glob('/var/lib/nagios/export/service__*wordpress_http?.cfg'): |
188 | 231 | if os.path.isfile(f): | ||
189 | 232 | os.unlink(f) | ||
190 | 233 | for f in glob.glob('/var/lib/nagios/export/service__*wordpress_plugin*.cfg'): | ||
191 | 234 | if os.path.isfile(f): | ||
192 | 235 | os.unlink(f) | ||
193 | 206 | 236 | ||
194 | 207 | 237 | ||
195 | 208 | def apparmor_dirs(service_name): | 238 | def apparmor_dirs(service_name): |
196 | @@ -234,7 +264,7 @@ | |||
197 | 234 | options were changed | 264 | options were changed |
198 | 235 | """ | 265 | """ |
199 | 236 | config = hookenv.config() | 266 | config = hookenv.config() |
201 | 237 | options = ['port_number', 'ssl_enabled', 'ssl_port_number', | 267 | options = ['port_number', 'ssl_enabled', 'ssl_port_number', 'redirects', |
202 | 238 | 'install_path', 'blog_hostname', 'additional_hostnames'] | 268 | 'install_path', 'blog_hostname', 'additional_hostnames'] |
203 | 239 | if not (wp_helpers.object_storage() | 269 | if not (wp_helpers.object_storage() |
204 | 240 | or any(config.changed(option) for option in options)): | 270 | or any(config.changed(option) for option in options)): |
205 | 241 | 271 | ||
206 | === modified file 'hooks/charmhelpers/contrib/charmsupport/nrpe.py' | |||
207 | --- hooks/charmhelpers/contrib/charmsupport/nrpe.py 2015-01-27 14:54:02 +0000 | |||
208 | +++ hooks/charmhelpers/contrib/charmsupport/nrpe.py 2016-04-05 14:45:53 +0000 | |||
209 | @@ -24,6 +24,8 @@ | |||
210 | 24 | import pwd | 24 | import pwd |
211 | 25 | import grp | 25 | import grp |
212 | 26 | import os | 26 | import os |
213 | 27 | import glob | ||
214 | 28 | import shutil | ||
215 | 27 | import re | 29 | import re |
216 | 28 | import shlex | 30 | import shlex |
217 | 29 | import yaml | 31 | import yaml |
218 | @@ -108,6 +110,13 @@ | |||
219 | 108 | # def local_monitors_relation_changed(): | 110 | # def local_monitors_relation_changed(): |
220 | 109 | # update_nrpe_config() | 111 | # update_nrpe_config() |
221 | 110 | # | 112 | # |
222 | 113 | # 4.a If your charm is a subordinate charm set primary=False | ||
223 | 114 | # | ||
224 | 115 | # from charmsupport.nrpe import NRPE | ||
225 | 116 | # (...) | ||
226 | 117 | # def update_nrpe_config(): | ||
227 | 118 | # nrpe_compat = NRPE(primary=False) | ||
228 | 119 | # | ||
229 | 111 | # 5. ln -s hooks.py nrpe-external-master-relation-changed | 120 | # 5. ln -s hooks.py nrpe-external-master-relation-changed |
230 | 112 | # ln -s hooks.py local-monitors-relation-changed | 121 | # ln -s hooks.py local-monitors-relation-changed |
231 | 113 | 122 | ||
232 | @@ -146,6 +155,13 @@ | |||
233 | 146 | self.description = description | 155 | self.description = description |
234 | 147 | self.check_cmd = self._locate_cmd(check_cmd) | 156 | self.check_cmd = self._locate_cmd(check_cmd) |
235 | 148 | 157 | ||
236 | 158 | def _get_check_filename(self): | ||
237 | 159 | return os.path.join(NRPE.nrpe_confdir, '{}.cfg'.format(self.command)) | ||
238 | 160 | |||
239 | 161 | def _get_service_filename(self, hostname): | ||
240 | 162 | return os.path.join(NRPE.nagios_exportdir, | ||
241 | 163 | 'service__{}_{}.cfg'.format(hostname, self.command)) | ||
242 | 164 | |||
243 | 149 | def _locate_cmd(self, check_cmd): | 165 | def _locate_cmd(self, check_cmd): |
244 | 150 | search_path = ( | 166 | search_path = ( |
245 | 151 | '/usr/lib/nagios/plugins', | 167 | '/usr/lib/nagios/plugins', |
246 | @@ -161,9 +177,21 @@ | |||
247 | 161 | log('Check command not found: {}'.format(parts[0])) | 177 | log('Check command not found: {}'.format(parts[0])) |
248 | 162 | return '' | 178 | return '' |
249 | 163 | 179 | ||
253 | 164 | def write(self, nagios_context, hostname, nagios_servicegroups=None): | 180 | def _remove_service_files(self): |
254 | 165 | nrpe_check_file = '/etc/nagios/nrpe.d/{}.cfg'.format( | 181 | if not os.path.exists(NRPE.nagios_exportdir): |
255 | 166 | self.command) | 182 | return |
256 | 183 | for f in os.listdir(NRPE.nagios_exportdir): | ||
257 | 184 | if f.endswith('_{}.cfg'.format(self.command)): | ||
258 | 185 | os.remove(os.path.join(NRPE.nagios_exportdir, f)) | ||
259 | 186 | |||
260 | 187 | def remove(self, hostname): | ||
261 | 188 | nrpe_check_file = self._get_check_filename() | ||
262 | 189 | if os.path.exists(nrpe_check_file): | ||
263 | 190 | os.remove(nrpe_check_file) | ||
264 | 191 | self._remove_service_files() | ||
265 | 192 | |||
266 | 193 | def write(self, nagios_context, hostname, nagios_servicegroups): | ||
267 | 194 | nrpe_check_file = self._get_check_filename() | ||
268 | 167 | with open(nrpe_check_file, 'w') as nrpe_check_config: | 195 | with open(nrpe_check_file, 'w') as nrpe_check_config: |
269 | 168 | nrpe_check_config.write("# check {}\n".format(self.shortname)) | 196 | nrpe_check_config.write("# check {}\n".format(self.shortname)) |
270 | 169 | nrpe_check_config.write("command[{}]={}\n".format( | 197 | nrpe_check_config.write("command[{}]={}\n".format( |
271 | @@ -177,13 +205,8 @@ | |||
272 | 177 | nagios_servicegroups) | 205 | nagios_servicegroups) |
273 | 178 | 206 | ||
274 | 179 | def write_service_config(self, nagios_context, hostname, | 207 | def write_service_config(self, nagios_context, hostname, |
282 | 180 | nagios_servicegroups=None): | 208 | nagios_servicegroups): |
283 | 181 | for f in os.listdir(NRPE.nagios_exportdir): | 209 | self._remove_service_files() |
277 | 182 | if re.search('.*{}.cfg'.format(self.command), f): | ||
278 | 183 | os.remove(os.path.join(NRPE.nagios_exportdir, f)) | ||
279 | 184 | |||
280 | 185 | if not nagios_servicegroups: | ||
281 | 186 | nagios_servicegroups = nagios_context | ||
284 | 187 | 210 | ||
285 | 188 | templ_vars = { | 211 | templ_vars = { |
286 | 189 | 'nagios_hostname': hostname, | 212 | 'nagios_hostname': hostname, |
287 | @@ -193,8 +216,7 @@ | |||
288 | 193 | 'command': self.command, | 216 | 'command': self.command, |
289 | 194 | } | 217 | } |
290 | 195 | nrpe_service_text = Check.service_template.format(**templ_vars) | 218 | nrpe_service_text = Check.service_template.format(**templ_vars) |
293 | 196 | nrpe_service_file = '{}/service__{}_{}.cfg'.format( | 219 | nrpe_service_file = self._get_service_filename(hostname) |
292 | 197 | NRPE.nagios_exportdir, hostname, self.command) | ||
294 | 198 | with open(nrpe_service_file, 'w') as nrpe_service_config: | 220 | with open(nrpe_service_file, 'w') as nrpe_service_config: |
295 | 199 | nrpe_service_config.write(str(nrpe_service_text)) | 221 | nrpe_service_config.write(str(nrpe_service_text)) |
296 | 200 | 222 | ||
297 | @@ -207,24 +229,51 @@ | |||
298 | 207 | nagios_exportdir = '/var/lib/nagios/export' | 229 | nagios_exportdir = '/var/lib/nagios/export' |
299 | 208 | nrpe_confdir = '/etc/nagios/nrpe.d' | 230 | nrpe_confdir = '/etc/nagios/nrpe.d' |
300 | 209 | 231 | ||
302 | 210 | def __init__(self, hostname=None): | 232 | def __init__(self, hostname=None, primary=True): |
303 | 211 | super(NRPE, self).__init__() | 233 | super(NRPE, self).__init__() |
304 | 212 | self.config = config() | 234 | self.config = config() |
305 | 235 | self.primary = primary | ||
306 | 213 | self.nagios_context = self.config['nagios_context'] | 236 | self.nagios_context = self.config['nagios_context'] |
308 | 214 | if 'nagios_servicegroups' in self.config: | 237 | if 'nagios_servicegroups' in self.config and self.config['nagios_servicegroups']: |
309 | 215 | self.nagios_servicegroups = self.config['nagios_servicegroups'] | 238 | self.nagios_servicegroups = self.config['nagios_servicegroups'] |
310 | 216 | else: | 239 | else: |
312 | 217 | self.nagios_servicegroups = 'juju' | 240 | self.nagios_servicegroups = self.nagios_context |
313 | 218 | self.unit_name = local_unit().replace('/', '-') | 241 | self.unit_name = local_unit().replace('/', '-') |
314 | 219 | if hostname: | 242 | if hostname: |
315 | 220 | self.hostname = hostname | 243 | self.hostname = hostname |
316 | 221 | else: | 244 | else: |
318 | 222 | self.hostname = "{}-{}".format(self.nagios_context, self.unit_name) | 245 | nagios_hostname = get_nagios_hostname() |
319 | 246 | if nagios_hostname: | ||
320 | 247 | self.hostname = nagios_hostname | ||
321 | 248 | else: | ||
322 | 249 | self.hostname = "{}-{}".format(self.nagios_context, self.unit_name) | ||
323 | 223 | self.checks = [] | 250 | self.checks = [] |
324 | 251 | # Iff in an nrpe-external-master relation hook set primary status | ||
325 | 252 | relation = relation_ids() | ||
326 | 253 | if relation: | ||
327 | 254 | if relation[0].startswith('nrpe-external-master'): | ||
328 | 255 | log("Setting charm primary status {}".format(primary)) | ||
329 | 256 | relation_set(relation_settings={'primary': self.primary}) | ||
330 | 224 | 257 | ||
331 | 225 | def add_check(self, *args, **kwargs): | 258 | def add_check(self, *args, **kwargs): |
332 | 226 | self.checks.append(Check(*args, **kwargs)) | 259 | self.checks.append(Check(*args, **kwargs)) |
333 | 227 | 260 | ||
334 | 261 | def remove_check(self, *args, **kwargs): | ||
335 | 262 | if kwargs.get('shortname') is None: | ||
336 | 263 | raise ValueError('shortname of check must be specified') | ||
337 | 264 | |||
338 | 265 | # Use sensible defaults if they're not specified - these are not | ||
339 | 266 | # actually used during removal, but they're required for constructing | ||
340 | 267 | # the Check object; check_disk is chosen because it's part of the | ||
341 | 268 | # nagios-plugins-basic package. | ||
342 | 269 | if kwargs.get('check_cmd') is None: | ||
343 | 270 | kwargs['check_cmd'] = 'check_disk' | ||
344 | 271 | if kwargs.get('description') is None: | ||
345 | 272 | kwargs['description'] = '' | ||
346 | 273 | |||
347 | 274 | check = Check(*args, **kwargs) | ||
348 | 275 | check.remove(self.hostname) | ||
349 | 276 | |||
350 | 228 | def write(self): | 277 | def write(self): |
351 | 229 | try: | 278 | try: |
352 | 230 | nagios_uid = pwd.getpwnam('nagios').pw_uid | 279 | nagios_uid = pwd.getpwnam('nagios').pw_uid |
353 | @@ -248,7 +297,9 @@ | |||
354 | 248 | 297 | ||
355 | 249 | service('restart', 'nagios-nrpe-server') | 298 | service('restart', 'nagios-nrpe-server') |
356 | 250 | 299 | ||
358 | 251 | for rid in relation_ids("local-monitors"): | 300 | monitor_ids = relation_ids("local-monitors") + \ |
359 | 301 | relation_ids("nrpe-external-master") | ||
360 | 302 | for rid in monitor_ids: | ||
361 | 252 | relation_set(relation_id=rid, monitors=yaml.dump(monitors)) | 303 | relation_set(relation_id=rid, monitors=yaml.dump(monitors)) |
362 | 253 | 304 | ||
363 | 254 | 305 | ||
364 | @@ -259,7 +310,7 @@ | |||
365 | 259 | :param str relation_name: Name of relation nrpe sub joined to | 310 | :param str relation_name: Name of relation nrpe sub joined to |
366 | 260 | """ | 311 | """ |
367 | 261 | for rel in relations_of_type(relation_name): | 312 | for rel in relations_of_type(relation_name): |
369 | 262 | if 'nagios_hostname' in rel: | 313 | if 'nagios_host_context' in rel: |
370 | 263 | return rel['nagios_host_context'] | 314 | return rel['nagios_host_context'] |
371 | 264 | 315 | ||
372 | 265 | 316 | ||
373 | @@ -300,11 +351,13 @@ | |||
374 | 300 | upstart_init = '/etc/init/%s.conf' % svc | 351 | upstart_init = '/etc/init/%s.conf' % svc |
375 | 301 | sysv_init = '/etc/init.d/%s' % svc | 352 | sysv_init = '/etc/init.d/%s' % svc |
376 | 302 | if os.path.exists(upstart_init): | 353 | if os.path.exists(upstart_init): |
382 | 303 | nrpe.add_check( | 354 | # Don't add a check for these services from neutron-gateway |
383 | 304 | shortname=svc, | 355 | if svc not in ['ext-port', 'os-charm-phy-nic-mtu']: |
384 | 305 | description='process check {%s}' % unit_name, | 356 | nrpe.add_check( |
385 | 306 | check_cmd='check_upstart_job %s' % svc | 357 | shortname=svc, |
386 | 307 | ) | 358 | description='process check {%s}' % unit_name, |
387 | 359 | check_cmd='check_upstart_job %s' % svc | ||
388 | 360 | ) | ||
389 | 308 | elif os.path.exists(sysv_init): | 361 | elif os.path.exists(sysv_init): |
390 | 309 | cronpath = '/etc/cron.d/nagios-service-check-%s' % svc | 362 | cronpath = '/etc/cron.d/nagios-service-check-%s' % svc |
391 | 310 | cron_file = ('*/5 * * * * root ' | 363 | cron_file = ('*/5 * * * * root ' |
392 | @@ -322,3 +375,38 @@ | |||
393 | 322 | check_cmd='check_status_file.py -f ' | 375 | check_cmd='check_status_file.py -f ' |
394 | 323 | '/var/lib/nagios/service-check-%s.txt' % svc, | 376 | '/var/lib/nagios/service-check-%s.txt' % svc, |
395 | 324 | ) | 377 | ) |
396 | 378 | |||
397 | 379 | |||
398 | 380 | def copy_nrpe_checks(): | ||
399 | 381 | """ | ||
400 | 382 | Copy the nrpe checks into place | ||
401 | 383 | |||
402 | 384 | """ | ||
403 | 385 | NAGIOS_PLUGINS = '/usr/local/lib/nagios/plugins' | ||
404 | 386 | nrpe_files_dir = os.path.join(os.getenv('CHARM_DIR'), 'hooks', | ||
405 | 387 | 'charmhelpers', 'contrib', 'openstack', | ||
406 | 388 | 'files') | ||
407 | 389 | |||
408 | 390 | if not os.path.exists(NAGIOS_PLUGINS): | ||
409 | 391 | os.makedirs(NAGIOS_PLUGINS) | ||
410 | 392 | for fname in glob.glob(os.path.join(nrpe_files_dir, "check_*")): | ||
411 | 393 | if os.path.isfile(fname): | ||
412 | 394 | shutil.copy2(fname, | ||
413 | 395 | os.path.join(NAGIOS_PLUGINS, os.path.basename(fname))) | ||
414 | 396 | |||
415 | 397 | |||
416 | 398 | def add_haproxy_checks(nrpe, unit_name): | ||
417 | 399 | """ | ||
418 | 400 | Add checks for each service in list | ||
419 | 401 | |||
420 | 402 | :param NRPE nrpe: NRPE object to add check to | ||
421 | 403 | :param str unit_name: Unit name to use in check description | ||
422 | 404 | """ | ||
423 | 405 | nrpe.add_check( | ||
424 | 406 | shortname='haproxy_servers', | ||
425 | 407 | description='Check HAProxy {%s}' % unit_name, | ||
426 | 408 | check_cmd='check_haproxy.sh') | ||
427 | 409 | nrpe.add_check( | ||
428 | 410 | shortname='haproxy_queue', | ||
429 | 411 | description='Check HAProxy queue depth {%s}' % unit_name, | ||
430 | 412 | check_cmd='check_haproxy_queue_depth.sh') | ||
431 | 325 | 413 | ||
432 | === added file 'hooks/charmhelpers/core/files.py' | |||
433 | --- hooks/charmhelpers/core/files.py 1970-01-01 00:00:00 +0000 | |||
434 | +++ hooks/charmhelpers/core/files.py 2016-04-05 14:45:53 +0000 | |||
435 | @@ -0,0 +1,45 @@ | |||
436 | 1 | #!/usr/bin/env python | ||
437 | 2 | # -*- coding: utf-8 -*- | ||
438 | 3 | |||
439 | 4 | # Copyright 2014-2015 Canonical Limited. | ||
440 | 5 | # | ||
441 | 6 | # This file is part of charm-helpers. | ||
442 | 7 | # | ||
443 | 8 | # charm-helpers is free software: you can redistribute it and/or modify | ||
444 | 9 | # it under the terms of the GNU Lesser General Public License version 3 as | ||
445 | 10 | # published by the Free Software Foundation. | ||
446 | 11 | # | ||
447 | 12 | # charm-helpers is distributed in the hope that it will be useful, | ||
448 | 13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | ||
449 | 14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | ||
450 | 15 | # GNU Lesser General Public License for more details. | ||
451 | 16 | # | ||
452 | 17 | # You should have received a copy of the GNU Lesser General Public License | ||
453 | 18 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
454 | 19 | |||
455 | 20 | __author__ = 'Jorge Niedbalski <niedbalski@ubuntu.com>' | ||
456 | 21 | |||
457 | 22 | import os | ||
458 | 23 | import subprocess | ||
459 | 24 | |||
460 | 25 | |||
461 | 26 | def sed(filename, before, after, flags='g'): | ||
462 | 27 | """ | ||
463 | 28 | Search and replaces the given pattern on filename. | ||
464 | 29 | |||
465 | 30 | :param filename: relative or absolute file path. | ||
466 | 31 | :param before: expression to be replaced (see 'man sed') | ||
467 | 32 | :param after: expression to replace with (see 'man sed') | ||
468 | 33 | :param flags: sed-compatible regex flags in example, to make | ||
469 | 34 | the search and replace case insensitive, specify ``flags="i"``. | ||
470 | 35 | The ``g`` flag is always specified regardless, so you do not | ||
471 | 36 | need to remember to include it when overriding this parameter. | ||
472 | 37 | :returns: If the sed command exit code was zero then return, | ||
473 | 38 | otherwise raise CalledProcessError. | ||
474 | 39 | """ | ||
475 | 40 | expression = r's/{0}/{1}/{2}'.format(before, | ||
476 | 41 | after, flags) | ||
477 | 42 | |||
478 | 43 | return subprocess.check_call(["sed", "-i", "-r", "-e", | ||
479 | 44 | expression, | ||
480 | 45 | os.path.expanduser(filename)]) | ||
481 | 0 | 46 | ||
482 | === modified file 'hooks/charmhelpers/core/fstab.py' | |||
483 | --- hooks/charmhelpers/core/fstab.py 2015-01-27 14:54:02 +0000 | |||
484 | +++ hooks/charmhelpers/core/fstab.py 2016-04-05 14:45:53 +0000 | |||
485 | @@ -17,11 +17,11 @@ | |||
486 | 17 | # You should have received a copy of the GNU Lesser General Public License | 17 | # You should have received a copy of the GNU Lesser General Public License |
487 | 18 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | 18 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
488 | 19 | 19 | ||
489 | 20 | __author__ = 'Jorge Niedbalski R. <jorge.niedbalski@canonical.com>' | ||
490 | 21 | |||
491 | 22 | import io | 20 | import io |
492 | 23 | import os | 21 | import os |
493 | 24 | 22 | ||
494 | 23 | __author__ = 'Jorge Niedbalski R. <jorge.niedbalski@canonical.com>' | ||
495 | 24 | |||
496 | 25 | 25 | ||
497 | 26 | class Fstab(io.FileIO): | 26 | class Fstab(io.FileIO): |
498 | 27 | """This class extends file in order to implement a file reader/writer | 27 | """This class extends file in order to implement a file reader/writer |
499 | @@ -77,7 +77,7 @@ | |||
500 | 77 | for line in self.readlines(): | 77 | for line in self.readlines(): |
501 | 78 | line = line.decode('us-ascii') | 78 | line = line.decode('us-ascii') |
502 | 79 | try: | 79 | try: |
504 | 80 | if line.strip() and not line.startswith("#"): | 80 | if line.strip() and not line.strip().startswith("#"): |
505 | 81 | yield self._hydrate_entry(line) | 81 | yield self._hydrate_entry(line) |
506 | 82 | except ValueError: | 82 | except ValueError: |
507 | 83 | pass | 83 | pass |
508 | @@ -104,7 +104,7 @@ | |||
509 | 104 | 104 | ||
510 | 105 | found = False | 105 | found = False |
511 | 106 | for index, line in enumerate(lines): | 106 | for index, line in enumerate(lines): |
513 | 107 | if not line.startswith("#"): | 107 | if line.strip() and not line.strip().startswith("#"): |
514 | 108 | if self._hydrate_entry(line) == entry: | 108 | if self._hydrate_entry(line) == entry: |
515 | 109 | found = True | 109 | found = True |
516 | 110 | break | 110 | break |
517 | 111 | 111 | ||
518 | === modified file 'hooks/charmhelpers/core/hookenv.py' | |||
519 | --- hooks/charmhelpers/core/hookenv.py 2015-01-27 14:54:02 +0000 | |||
520 | +++ hooks/charmhelpers/core/hookenv.py 2016-04-05 14:45:53 +0000 | |||
521 | @@ -20,11 +20,18 @@ | |||
522 | 20 | # Authors: | 20 | # Authors: |
523 | 21 | # Charm Helpers Developers <juju@lists.ubuntu.com> | 21 | # Charm Helpers Developers <juju@lists.ubuntu.com> |
524 | 22 | 22 | ||
525 | 23 | from __future__ import print_function | ||
526 | 24 | import copy | ||
527 | 25 | from distutils.version import LooseVersion | ||
528 | 26 | from functools import wraps | ||
529 | 27 | import glob | ||
530 | 23 | import os | 28 | import os |
531 | 24 | import json | 29 | import json |
532 | 25 | import yaml | 30 | import yaml |
533 | 26 | import subprocess | 31 | import subprocess |
534 | 27 | import sys | 32 | import sys |
535 | 33 | import errno | ||
536 | 34 | import tempfile | ||
537 | 28 | from subprocess import CalledProcessError | 35 | from subprocess import CalledProcessError |
538 | 29 | 36 | ||
539 | 30 | import six | 37 | import six |
540 | @@ -56,15 +63,18 @@ | |||
541 | 56 | 63 | ||
542 | 57 | will cache the result of unit_get + 'test' for future calls. | 64 | will cache the result of unit_get + 'test' for future calls. |
543 | 58 | """ | 65 | """ |
544 | 66 | @wraps(func) | ||
545 | 59 | def wrapper(*args, **kwargs): | 67 | def wrapper(*args, **kwargs): |
546 | 60 | global cache | 68 | global cache |
547 | 61 | key = str((func, args, kwargs)) | 69 | key = str((func, args, kwargs)) |
548 | 62 | try: | 70 | try: |
549 | 63 | return cache[key] | 71 | return cache[key] |
550 | 64 | except KeyError: | 72 | except KeyError: |
554 | 65 | res = func(*args, **kwargs) | 73 | pass # Drop out of the exception handler scope. |
555 | 66 | cache[key] = res | 74 | res = func(*args, **kwargs) |
556 | 67 | return res | 75 | cache[key] = res |
557 | 76 | return res | ||
558 | 77 | wrapper._wrapped = func | ||
559 | 68 | return wrapper | 78 | return wrapper |
560 | 69 | 79 | ||
561 | 70 | 80 | ||
562 | @@ -87,7 +97,18 @@ | |||
563 | 87 | if not isinstance(message, six.string_types): | 97 | if not isinstance(message, six.string_types): |
564 | 88 | message = repr(message) | 98 | message = repr(message) |
565 | 89 | command += [message] | 99 | command += [message] |
567 | 90 | subprocess.call(command) | 100 | # Missing juju-log should not cause failures in unit tests |
568 | 101 | # Send log output to stderr | ||
569 | 102 | try: | ||
570 | 103 | subprocess.call(command) | ||
571 | 104 | except OSError as e: | ||
572 | 105 | if e.errno == errno.ENOENT: | ||
573 | 106 | if level: | ||
574 | 107 | message = "{}: {}".format(level, message) | ||
575 | 108 | message = "juju-log: {}".format(message) | ||
576 | 109 | print(message, file=sys.stderr) | ||
577 | 110 | else: | ||
578 | 111 | raise | ||
579 | 91 | 112 | ||
580 | 92 | 113 | ||
581 | 93 | class Serializable(UserDict): | 114 | class Serializable(UserDict): |
582 | @@ -153,9 +174,19 @@ | |||
583 | 153 | return os.environ.get('JUJU_RELATION', None) | 174 | return os.environ.get('JUJU_RELATION', None) |
584 | 154 | 175 | ||
585 | 155 | 176 | ||
589 | 156 | def relation_id(): | 177 | @cached |
590 | 157 | """The relation ID for the current relation hook""" | 178 | def relation_id(relation_name=None, service_or_unit=None): |
591 | 158 | return os.environ.get('JUJU_RELATION_ID', None) | 179 | """The relation ID for the current or a specified relation""" |
592 | 180 | if not relation_name and not service_or_unit: | ||
593 | 181 | return os.environ.get('JUJU_RELATION_ID', None) | ||
594 | 182 | elif relation_name and service_or_unit: | ||
595 | 183 | service_name = service_or_unit.split('/')[0] | ||
596 | 184 | for relid in relation_ids(relation_name): | ||
597 | 185 | remote_service = remote_service_name(relid) | ||
598 | 186 | if remote_service == service_name: | ||
599 | 187 | return relid | ||
600 | 188 | else: | ||
601 | 189 | raise ValueError('Must specify neither or both of relation_name and service_or_unit') | ||
602 | 159 | 190 | ||
603 | 160 | 191 | ||
604 | 161 | def local_unit(): | 192 | def local_unit(): |
605 | @@ -165,7 +196,7 @@ | |||
606 | 165 | 196 | ||
607 | 166 | def remote_unit(): | 197 | def remote_unit(): |
608 | 167 | """The remote unit for the current relation hook""" | 198 | """The remote unit for the current relation hook""" |
610 | 168 | return os.environ['JUJU_REMOTE_UNIT'] | 199 | return os.environ.get('JUJU_REMOTE_UNIT', None) |
611 | 169 | 200 | ||
612 | 170 | 201 | ||
613 | 171 | def service_name(): | 202 | def service_name(): |
614 | @@ -173,9 +204,20 @@ | |||
615 | 173 | return local_unit().split('/')[0] | 204 | return local_unit().split('/')[0] |
616 | 174 | 205 | ||
617 | 175 | 206 | ||
618 | 207 | @cached | ||
619 | 208 | def remote_service_name(relid=None): | ||
620 | 209 | """The remote service name for a given relation-id (or the current relation)""" | ||
621 | 210 | if relid is None: | ||
622 | 211 | unit = remote_unit() | ||
623 | 212 | else: | ||
624 | 213 | units = related_units(relid) | ||
625 | 214 | unit = units[0] if units else None | ||
626 | 215 | return unit.split('/')[0] if unit else None | ||
627 | 216 | |||
628 | 217 | |||
629 | 176 | def hook_name(): | 218 | def hook_name(): |
630 | 177 | """The name of the currently executing hook""" | 219 | """The name of the currently executing hook""" |
632 | 178 | return os.path.basename(sys.argv[0]) | 220 | return os.environ.get('JUJU_HOOK_NAME', os.path.basename(sys.argv[0])) |
633 | 179 | 221 | ||
634 | 180 | 222 | ||
635 | 181 | class Config(dict): | 223 | class Config(dict): |
636 | @@ -225,23 +267,7 @@ | |||
637 | 225 | self.path = os.path.join(charm_dir(), Config.CONFIG_FILE_NAME) | 267 | self.path = os.path.join(charm_dir(), Config.CONFIG_FILE_NAME) |
638 | 226 | if os.path.exists(self.path): | 268 | if os.path.exists(self.path): |
639 | 227 | self.load_previous() | 269 | self.load_previous() |
657 | 228 | 270 | atexit(self._implicit_save) | |
641 | 229 | def __getitem__(self, key): | ||
642 | 230 | """For regular dict lookups, check the current juju config first, | ||
643 | 231 | then the previous (saved) copy. This ensures that user-saved values | ||
644 | 232 | will be returned by a dict lookup. | ||
645 | 233 | |||
646 | 234 | """ | ||
647 | 235 | try: | ||
648 | 236 | return dict.__getitem__(self, key) | ||
649 | 237 | except KeyError: | ||
650 | 238 | return (self._prev_dict or {})[key] | ||
651 | 239 | |||
652 | 240 | def keys(self): | ||
653 | 241 | prev_keys = [] | ||
654 | 242 | if self._prev_dict is not None: | ||
655 | 243 | prev_keys = self._prev_dict.keys() | ||
656 | 244 | return list(set(prev_keys + list(dict.keys(self)))) | ||
658 | 245 | 271 | ||
659 | 246 | def load_previous(self, path=None): | 272 | def load_previous(self, path=None): |
660 | 247 | """Load previous copy of config from disk. | 273 | """Load previous copy of config from disk. |
661 | @@ -260,6 +286,9 @@ | |||
662 | 260 | self.path = path or self.path | 286 | self.path = path or self.path |
663 | 261 | with open(self.path) as f: | 287 | with open(self.path) as f: |
664 | 262 | self._prev_dict = json.load(f) | 288 | self._prev_dict = json.load(f) |
665 | 289 | for k, v in copy.deepcopy(self._prev_dict).items(): | ||
666 | 290 | if k not in self: | ||
667 | 291 | self[k] = v | ||
668 | 263 | 292 | ||
669 | 264 | def changed(self, key): | 293 | def changed(self, key): |
670 | 265 | """Return True if the current value for this key is different from | 294 | """Return True if the current value for this key is different from |
671 | @@ -291,13 +320,13 @@ | |||
672 | 291 | instance. | 320 | instance. |
673 | 292 | 321 | ||
674 | 293 | """ | 322 | """ |
675 | 294 | if self._prev_dict: | ||
676 | 295 | for k, v in six.iteritems(self._prev_dict): | ||
677 | 296 | if k not in self: | ||
678 | 297 | self[k] = v | ||
679 | 298 | with open(self.path, 'w') as f: | 323 | with open(self.path, 'w') as f: |
680 | 299 | json.dump(self, f) | 324 | json.dump(self, f) |
681 | 300 | 325 | ||
682 | 326 | def _implicit_save(self): | ||
683 | 327 | if self.implicit_save: | ||
684 | 328 | self.save() | ||
685 | 329 | |||
686 | 301 | 330 | ||
687 | 302 | @cached | 331 | @cached |
688 | 303 | def config(scope=None): | 332 | def config(scope=None): |
689 | @@ -340,18 +369,49 @@ | |||
690 | 340 | """Set relation information for the current unit""" | 369 | """Set relation information for the current unit""" |
691 | 341 | relation_settings = relation_settings if relation_settings else {} | 370 | relation_settings = relation_settings if relation_settings else {} |
692 | 342 | relation_cmd_line = ['relation-set'] | 371 | relation_cmd_line = ['relation-set'] |
693 | 372 | accepts_file = "--file" in subprocess.check_output( | ||
694 | 373 | relation_cmd_line + ["--help"], universal_newlines=True) | ||
695 | 343 | if relation_id is not None: | 374 | if relation_id is not None: |
696 | 344 | relation_cmd_line.extend(('-r', relation_id)) | 375 | relation_cmd_line.extend(('-r', relation_id)) |
703 | 345 | for k, v in (list(relation_settings.items()) + list(kwargs.items())): | 376 | settings = relation_settings.copy() |
704 | 346 | if v is None: | 377 | settings.update(kwargs) |
705 | 347 | relation_cmd_line.append('{}='.format(k)) | 378 | for key, value in settings.items(): |
706 | 348 | else: | 379 | # Force value to be a string: it always should, but some call |
707 | 349 | relation_cmd_line.append('{}={}'.format(k, v)) | 380 | # sites pass in things like dicts or numbers. |
708 | 350 | subprocess.check_call(relation_cmd_line) | 381 | if value is not None: |
709 | 382 | settings[key] = "{}".format(value) | ||
710 | 383 | if accepts_file: | ||
711 | 384 | # --file was introduced in Juju 1.23.2. Use it by default if | ||
712 | 385 | # available, since otherwise we'll break if the relation data is | ||
713 | 386 | # too big. Ideally we should tell relation-set to read the data from | ||
714 | 387 | # stdin, but that feature is broken in 1.23.2: Bug #1454678. | ||
715 | 388 | with tempfile.NamedTemporaryFile(delete=False) as settings_file: | ||
716 | 389 | settings_file.write(yaml.safe_dump(settings).encode("utf-8")) | ||
717 | 390 | subprocess.check_call( | ||
718 | 391 | relation_cmd_line + ["--file", settings_file.name]) | ||
719 | 392 | os.remove(settings_file.name) | ||
720 | 393 | else: | ||
721 | 394 | for key, value in settings.items(): | ||
722 | 395 | if value is None: | ||
723 | 396 | relation_cmd_line.append('{}='.format(key)) | ||
724 | 397 | else: | ||
725 | 398 | relation_cmd_line.append('{}={}'.format(key, value)) | ||
726 | 399 | subprocess.check_call(relation_cmd_line) | ||
727 | 351 | # Flush cache of any relation-gets for local unit | 400 | # Flush cache of any relation-gets for local unit |
728 | 352 | flush(local_unit()) | 401 | flush(local_unit()) |
729 | 353 | 402 | ||
730 | 354 | 403 | ||
731 | 404 | def relation_clear(r_id=None): | ||
732 | 405 | ''' Clears any relation data already set on relation r_id ''' | ||
733 | 406 | settings = relation_get(rid=r_id, | ||
734 | 407 | unit=local_unit()) | ||
735 | 408 | for setting in settings: | ||
736 | 409 | if setting not in ['public-address', 'private-address']: | ||
737 | 410 | settings[setting] = None | ||
738 | 411 | relation_set(relation_id=r_id, | ||
739 | 412 | **settings) | ||
740 | 413 | |||
741 | 414 | |||
742 | 355 | @cached | 415 | @cached |
743 | 356 | def relation_ids(reltype=None): | 416 | def relation_ids(reltype=None): |
744 | 357 | """A list of relation_ids""" | 417 | """A list of relation_ids""" |
745 | @@ -431,6 +491,76 @@ | |||
746 | 431 | 491 | ||
747 | 432 | 492 | ||
748 | 433 | @cached | 493 | @cached |
749 | 494 | def peer_relation_id(): | ||
750 | 495 | '''Get the peers relation id if a peers relation has been joined, else None.''' | ||
751 | 496 | md = metadata() | ||
752 | 497 | section = md.get('peers') | ||
753 | 498 | if section: | ||
754 | 499 | for key in section: | ||
755 | 500 | relids = relation_ids(key) | ||
756 | 501 | if relids: | ||
757 | 502 | return relids[0] | ||
758 | 503 | return None | ||
759 | 504 | |||
760 | 505 | |||
761 | 506 | @cached | ||
762 | 507 | def relation_to_interface(relation_name): | ||
763 | 508 | """ | ||
764 | 509 | Given the name of a relation, return the interface that relation uses. | ||
765 | 510 | |||
766 | 511 | :returns: The interface name, or ``None``. | ||
767 | 512 | """ | ||
768 | 513 | return relation_to_role_and_interface(relation_name)[1] | ||
769 | 514 | |||
770 | 515 | |||
771 | 516 | @cached | ||
772 | 517 | def relation_to_role_and_interface(relation_name): | ||
773 | 518 | """ | ||
774 | 519 | Given the name of a relation, return the role and the name of the interface | ||
775 | 520 | that relation uses (where role is one of ``provides``, ``requires``, or ``peers``). | ||
776 | 521 | |||
777 | 522 | :returns: A tuple containing ``(role, interface)``, or ``(None, None)``. | ||
778 | 523 | """ | ||
779 | 524 | _metadata = metadata() | ||
780 | 525 | for role in ('provides', 'requires', 'peers'): | ||
781 | 526 | interface = _metadata.get(role, {}).get(relation_name, {}).get('interface') | ||
782 | 527 | if interface: | ||
783 | 528 | return role, interface | ||
784 | 529 | return None, None | ||
785 | 530 | |||
786 | 531 | |||
787 | 532 | @cached | ||
788 | 533 | def role_and_interface_to_relations(role, interface_name): | ||
789 | 534 | """ | ||
790 | 535 | Given a role and interface name, return a list of relation names for the | ||
791 | 536 | current charm that use that interface under that role (where role is one | ||
792 | 537 | of ``provides``, ``requires``, or ``peers``). | ||
793 | 538 | |||
794 | 539 | :returns: A list of relation names. | ||
795 | 540 | """ | ||
796 | 541 | _metadata = metadata() | ||
797 | 542 | results = [] | ||
798 | 543 | for relation_name, relation in _metadata.get(role, {}).items(): | ||
799 | 544 | if relation['interface'] == interface_name: | ||
800 | 545 | results.append(relation_name) | ||
801 | 546 | return results | ||
802 | 547 | |||
803 | 548 | |||
804 | 549 | @cached | ||
805 | 550 | def interface_to_relations(interface_name): | ||
806 | 551 | """ | ||
807 | 552 | Given an interface, return a list of relation names for the current | ||
808 | 553 | charm that use that interface. | ||
809 | 554 | |||
810 | 555 | :returns: A list of relation names. | ||
811 | 556 | """ | ||
812 | 557 | results = [] | ||
813 | 558 | for role in ('provides', 'requires', 'peers'): | ||
814 | 559 | results.extend(role_and_interface_to_relations(role, interface_name)) | ||
815 | 560 | return results | ||
816 | 561 | |||
817 | 562 | |||
818 | 563 | @cached | ||
819 | 434 | def charm_name(): | 564 | def charm_name(): |
820 | 435 | """Get the name of the current charm as is specified on metadata.yaml""" | 565 | """Get the name of the current charm as is specified on metadata.yaml""" |
821 | 436 | return metadata().get('name') | 566 | return metadata().get('name') |
822 | @@ -496,11 +626,48 @@ | |||
823 | 496 | return None | 626 | return None |
824 | 497 | 627 | ||
825 | 498 | 628 | ||
826 | 629 | def unit_public_ip(): | ||
827 | 630 | """Get this unit's public IP address""" | ||
828 | 631 | return unit_get('public-address') | ||
829 | 632 | |||
830 | 633 | |||
831 | 499 | def unit_private_ip(): | 634 | def unit_private_ip(): |
832 | 500 | """Get this unit's private IP address""" | 635 | """Get this unit's private IP address""" |
833 | 501 | return unit_get('private-address') | 636 | return unit_get('private-address') |
834 | 502 | 637 | ||
835 | 503 | 638 | ||
836 | 639 | @cached | ||
837 | 640 | def storage_get(attribute=None, storage_id=None): | ||
838 | 641 | """Get storage attributes""" | ||
839 | 642 | _args = ['storage-get', '--format=json'] | ||
840 | 643 | if storage_id: | ||
841 | 644 | _args.extend(('-s', storage_id)) | ||
842 | 645 | if attribute: | ||
843 | 646 | _args.append(attribute) | ||
844 | 647 | try: | ||
845 | 648 | return json.loads(subprocess.check_output(_args).decode('UTF-8')) | ||
846 | 649 | except ValueError: | ||
847 | 650 | return None | ||
848 | 651 | |||
849 | 652 | |||
850 | 653 | @cached | ||
851 | 654 | def storage_list(storage_name=None): | ||
852 | 655 | """List the storage IDs for the unit""" | ||
853 | 656 | _args = ['storage-list', '--format=json'] | ||
854 | 657 | if storage_name: | ||
855 | 658 | _args.append(storage_name) | ||
856 | 659 | try: | ||
857 | 660 | return json.loads(subprocess.check_output(_args).decode('UTF-8')) | ||
858 | 661 | except ValueError: | ||
859 | 662 | return None | ||
860 | 663 | except OSError as e: | ||
861 | 664 | import errno | ||
862 | 665 | if e.errno == errno.ENOENT: | ||
863 | 666 | # storage-list does not exist | ||
864 | 667 | return [] | ||
865 | 668 | raise | ||
866 | 669 | |||
867 | 670 | |||
868 | 504 | class UnregisteredHookError(Exception): | 671 | class UnregisteredHookError(Exception): |
869 | 505 | """Raised when an undefined hook is called""" | 672 | """Raised when an undefined hook is called""" |
870 | 506 | pass | 673 | pass |
871 | @@ -528,10 +695,14 @@ | |||
872 | 528 | hooks.execute(sys.argv) | 695 | hooks.execute(sys.argv) |
873 | 529 | """ | 696 | """ |
874 | 530 | 697 | ||
876 | 531 | def __init__(self, config_save=True): | 698 | def __init__(self, config_save=None): |
877 | 532 | super(Hooks, self).__init__() | 699 | super(Hooks, self).__init__() |
878 | 533 | self._hooks = {} | 700 | self._hooks = {} |
880 | 534 | self._config_save = config_save | 701 | |
881 | 702 | # For unknown reasons, we allow the Hooks constructor to override | ||
882 | 703 | # config().implicit_save. | ||
883 | 704 | if config_save is not None: | ||
884 | 705 | config().implicit_save = config_save | ||
885 | 535 | 706 | ||
886 | 536 | def register(self, name, function): | 707 | def register(self, name, function): |
887 | 537 | """Register a hook""" | 708 | """Register a hook""" |
888 | @@ -539,13 +710,16 @@ | |||
889 | 539 | 710 | ||
890 | 540 | def execute(self, args): | 711 | def execute(self, args): |
891 | 541 | """Execute a registered hook based on args[0]""" | 712 | """Execute a registered hook based on args[0]""" |
892 | 713 | _run_atstart() | ||
893 | 542 | hook_name = os.path.basename(args[0]) | 714 | hook_name = os.path.basename(args[0]) |
894 | 543 | if hook_name in self._hooks: | 715 | if hook_name in self._hooks: |
900 | 544 | self._hooks[hook_name]() | 716 | try: |
901 | 545 | if self._config_save: | 717 | self._hooks[hook_name]() |
902 | 546 | cfg = config() | 718 | except SystemExit as x: |
903 | 547 | if cfg.implicit_save: | 719 | if x.code is None or x.code == 0: |
904 | 548 | cfg.save() | 720 | _run_atexit() |
905 | 721 | raise | ||
906 | 722 | _run_atexit() | ||
907 | 549 | else: | 723 | else: |
908 | 550 | raise UnregisteredHookError(hook_name) | 724 | raise UnregisteredHookError(hook_name) |
909 | 551 | 725 | ||
910 | @@ -566,3 +740,270 @@ | |||
911 | 566 | def charm_dir(): | 740 | def charm_dir(): |
912 | 567 | """Return the root directory of the current charm""" | 741 | """Return the root directory of the current charm""" |
913 | 568 | return os.environ.get('CHARM_DIR') | 742 | return os.environ.get('CHARM_DIR') |
914 | 743 | |||
915 | 744 | |||
916 | 745 | @cached | ||
917 | 746 | def action_get(key=None): | ||
918 | 747 | """Gets the value of an action parameter, or all key/value param pairs""" | ||
919 | 748 | cmd = ['action-get'] | ||
920 | 749 | if key is not None: | ||
921 | 750 | cmd.append(key) | ||
922 | 751 | cmd.append('--format=json') | ||
923 | 752 | action_data = json.loads(subprocess.check_output(cmd).decode('UTF-8')) | ||
924 | 753 | return action_data | ||
925 | 754 | |||
926 | 755 | |||
927 | 756 | def action_set(values): | ||
928 | 757 | """Sets the values to be returned after the action finishes""" | ||
929 | 758 | cmd = ['action-set'] | ||
930 | 759 | for k, v in list(values.items()): | ||
931 | 760 | cmd.append('{}={}'.format(k, v)) | ||
932 | 761 | subprocess.check_call(cmd) | ||
933 | 762 | |||
934 | 763 | |||
935 | 764 | def action_fail(message): | ||
936 | 765 | """Sets the action status to failed and sets the error message. | ||
937 | 766 | |||
938 | 767 | The results set by action_set are preserved.""" | ||
939 | 768 | subprocess.check_call(['action-fail', message]) | ||
940 | 769 | |||
941 | 770 | |||
942 | 771 | def action_name(): | ||
943 | 772 | """Get the name of the currently executing action.""" | ||
944 | 773 | return os.environ.get('JUJU_ACTION_NAME') | ||
945 | 774 | |||
946 | 775 | |||
947 | 776 | def action_uuid(): | ||
948 | 777 | """Get the UUID of the currently executing action.""" | ||
949 | 778 | return os.environ.get('JUJU_ACTION_UUID') | ||
950 | 779 | |||
951 | 780 | |||
952 | 781 | def action_tag(): | ||
953 | 782 | """Get the tag for the currently executing action.""" | ||
954 | 783 | return os.environ.get('JUJU_ACTION_TAG') | ||
955 | 784 | |||
956 | 785 | |||
957 | 786 | def status_set(workload_state, message): | ||
958 | 787 | """Set the workload state with a message | ||
959 | 788 | |||
960 | 789 | Use status-set to set the workload state with a message which is visible | ||
961 | 790 | to the user via juju status. If the status-set command is not found then | ||
962 | 791 | assume this is juju < 1.23 and juju-log the message unstead. | ||
963 | 792 | |||
964 | 793 | workload_state -- valid juju workload state. | ||
965 | 794 | message -- status update message | ||
966 | 795 | """ | ||
967 | 796 | valid_states = ['maintenance', 'blocked', 'waiting', 'active'] | ||
968 | 797 | if workload_state not in valid_states: | ||
969 | 798 | raise ValueError( | ||
970 | 799 | '{!r} is not a valid workload state'.format(workload_state) | ||
971 | 800 | ) | ||
972 | 801 | cmd = ['status-set', workload_state, message] | ||
973 | 802 | try: | ||
974 | 803 | ret = subprocess.call(cmd) | ||
975 | 804 | if ret == 0: | ||
976 | 805 | return | ||
977 | 806 | except OSError as e: | ||
978 | 807 | if e.errno != errno.ENOENT: | ||
979 | 808 | raise | ||
980 | 809 | log_message = 'status-set failed: {} {}'.format(workload_state, | ||
981 | 810 | message) | ||
982 | 811 | log(log_message, level='INFO') | ||
983 | 812 | |||
984 | 813 | |||
985 | 814 | def status_get(): | ||
986 | 815 | """Retrieve the previously set juju workload state and message | ||
987 | 816 | |||
988 | 817 | If the status-get command is not found then assume this is juju < 1.23 and | ||
989 | 818 | return 'unknown', "" | ||
990 | 819 | |||
991 | 820 | """ | ||
992 | 821 | cmd = ['status-get', "--format=json", "--include-data"] | ||
993 | 822 | try: | ||
994 | 823 | raw_status = subprocess.check_output(cmd) | ||
995 | 824 | except OSError as e: | ||
996 | 825 | if e.errno == errno.ENOENT: | ||
997 | 826 | return ('unknown', "") | ||
998 | 827 | else: | ||
999 | 828 | raise | ||
1000 | 829 | else: | ||
1001 | 830 | status = json.loads(raw_status.decode("UTF-8")) | ||
1002 | 831 | return (status["status"], status["message"]) | ||
1003 | 832 | |||
1004 | 833 | |||
1005 | 834 | def translate_exc(from_exc, to_exc): | ||
1006 | 835 | def inner_translate_exc1(f): | ||
1007 | 836 | @wraps(f) | ||
1008 | 837 | def inner_translate_exc2(*args, **kwargs): | ||
1009 | 838 | try: | ||
1010 | 839 | return f(*args, **kwargs) | ||
1011 | 840 | except from_exc: | ||
1012 | 841 | raise to_exc | ||
1013 | 842 | |||
1014 | 843 | return inner_translate_exc2 | ||
1015 | 844 | |||
1016 | 845 | return inner_translate_exc1 | ||
1017 | 846 | |||
1018 | 847 | |||
1019 | 848 | @translate_exc(from_exc=OSError, to_exc=NotImplementedError) | ||
1020 | 849 | def is_leader(): | ||
1021 | 850 | """Does the current unit hold the juju leadership | ||
1022 | 851 | |||
1023 | 852 | Uses juju to determine whether the current unit is the leader of its peers | ||
1024 | 853 | """ | ||
1025 | 854 | cmd = ['is-leader', '--format=json'] | ||
1026 | 855 | return json.loads(subprocess.check_output(cmd).decode('UTF-8')) | ||
1027 | 856 | |||
1028 | 857 | |||
1029 | 858 | @translate_exc(from_exc=OSError, to_exc=NotImplementedError) | ||
1030 | 859 | def leader_get(attribute=None): | ||
1031 | 860 | """Juju leader get value(s)""" | ||
1032 | 861 | cmd = ['leader-get', '--format=json'] + [attribute or '-'] | ||
1033 | 862 | return json.loads(subprocess.check_output(cmd).decode('UTF-8')) | ||
1034 | 863 | |||
1035 | 864 | |||
1036 | 865 | @translate_exc(from_exc=OSError, to_exc=NotImplementedError) | ||
1037 | 866 | def leader_set(settings=None, **kwargs): | ||
1038 | 867 | """Juju leader set value(s)""" | ||
1039 | 868 | # Don't log secrets. | ||
1040 | 869 | # log("Juju leader-set '%s'" % (settings), level=DEBUG) | ||
1041 | 870 | cmd = ['leader-set'] | ||
1042 | 871 | settings = settings or {} | ||
1043 | 872 | settings.update(kwargs) | ||
1044 | 873 | for k, v in settings.items(): | ||
1045 | 874 | if v is None: | ||
1046 | 875 | cmd.append('{}='.format(k)) | ||
1047 | 876 | else: | ||
1048 | 877 | cmd.append('{}={}'.format(k, v)) | ||
1049 | 878 | subprocess.check_call(cmd) | ||
1050 | 879 | |||
1051 | 880 | |||
1052 | 881 | @translate_exc(from_exc=OSError, to_exc=NotImplementedError) | ||
1053 | 882 | def payload_register(ptype, klass, pid): | ||
1054 | 883 | """ is used while a hook is running to let Juju know that a | ||
1055 | 884 | payload has been started.""" | ||
1056 | 885 | cmd = ['payload-register'] | ||
1057 | 886 | for x in [ptype, klass, pid]: | ||
1058 | 887 | cmd.append(x) | ||
1059 | 888 | subprocess.check_call(cmd) | ||
1060 | 889 | |||
1061 | 890 | |||
1062 | 891 | @translate_exc(from_exc=OSError, to_exc=NotImplementedError) | ||
1063 | 892 | def payload_unregister(klass, pid): | ||
1064 | 893 | """ is used while a hook is running to let Juju know | ||
1065 | 894 | that a payload has been manually stopped. The <class> and <id> provided | ||
1066 | 895 | must match a payload that has been previously registered with juju using | ||
1067 | 896 | payload-register.""" | ||
1068 | 897 | cmd = ['payload-unregister'] | ||
1069 | 898 | for x in [klass, pid]: | ||
1070 | 899 | cmd.append(x) | ||
1071 | 900 | subprocess.check_call(cmd) | ||
1072 | 901 | |||
1073 | 902 | |||
1074 | 903 | @translate_exc(from_exc=OSError, to_exc=NotImplementedError) | ||
1075 | 904 | def payload_status_set(klass, pid, status): | ||
1076 | 905 | """is used to update the current status of a registered payload. | ||
1077 | 906 | The <class> and <id> provided must match a payload that has been previously | ||
1078 | 907 | registered with juju using payload-register. The <status> must be one of the | ||
1079 | 908 | follow: starting, started, stopping, stopped""" | ||
1080 | 909 | cmd = ['payload-status-set'] | ||
1081 | 910 | for x in [klass, pid, status]: | ||
1082 | 911 | cmd.append(x) | ||
1083 | 912 | subprocess.check_call(cmd) | ||
1084 | 913 | |||
1085 | 914 | |||
1086 | 915 | @translate_exc(from_exc=OSError, to_exc=NotImplementedError) | ||
1087 | 916 | def resource_get(name): | ||
1088 | 917 | """used to fetch the resource path of the given name. | ||
1089 | 918 | |||
1090 | 919 | <name> must match a name of defined resource in metadata.yaml | ||
1091 | 920 | |||
1092 | 921 | returns either a path or False if resource not available | ||
1093 | 922 | """ | ||
1094 | 923 | if not name: | ||
1095 | 924 | return False | ||
1096 | 925 | |||
1097 | 926 | cmd = ['resource-get', name] | ||
1098 | 927 | try: | ||
1099 | 928 | return subprocess.check_output(cmd).decode('UTF-8') | ||
1100 | 929 | except subprocess.CalledProcessError: | ||
1101 | 930 | return False | ||
1102 | 931 | |||
1103 | 932 | |||
1104 | 933 | @cached | ||
1105 | 934 | def juju_version(): | ||
1106 | 935 | """Full version string (eg. '1.23.3.1-trusty-amd64')""" | ||
1107 | 936 | # Per https://bugs.launchpad.net/juju-core/+bug/1455368/comments/1 | ||
1108 | 937 | jujud = glob.glob('/var/lib/juju/tools/machine-*/jujud')[0] | ||
1109 | 938 | return subprocess.check_output([jujud, 'version'], | ||
1110 | 939 | universal_newlines=True).strip() | ||
1111 | 940 | |||
1112 | 941 | |||
1113 | 942 | @cached | ||
1114 | 943 | def has_juju_version(minimum_version): | ||
1115 | 944 | """Return True if the Juju version is at least the provided version""" | ||
1116 | 945 | return LooseVersion(juju_version()) >= LooseVersion(minimum_version) | ||
1117 | 946 | |||
1118 | 947 | |||
1119 | 948 | _atexit = [] | ||
1120 | 949 | _atstart = [] | ||
1121 | 950 | |||
1122 | 951 | |||
1123 | 952 | def atstart(callback, *args, **kwargs): | ||
1124 | 953 | '''Schedule a callback to run before the main hook. | ||
1125 | 954 | |||
1126 | 955 | Callbacks are run in the order they were added. | ||
1127 | 956 | |||
1128 | 957 | This is useful for modules and classes to perform initialization | ||
1129 | 958 | and inject behavior. In particular: | ||
1130 | 959 | |||
1131 | 960 | - Run common code before all of your hooks, such as logging | ||
1132 | 961 | the hook name or interesting relation data. | ||
1133 | 962 | - Defer object or module initialization that requires a hook | ||
1134 | 963 | context until we know there actually is a hook context, | ||
1135 | 964 | making testing easier. | ||
1136 | 965 | - Rather than requiring charm authors to include boilerplate to | ||
1137 | 966 | invoke your helper's behavior, have it run automatically if | ||
1138 | 967 | your object is instantiated or module imported. | ||
1139 | 968 | |||
1140 | 969 | This is not at all useful after your hook framework as been launched. | ||
1141 | 970 | ''' | ||
1142 | 971 | global _atstart | ||
1143 | 972 | _atstart.append((callback, args, kwargs)) | ||
1144 | 973 | |||
1145 | 974 | |||
1146 | 975 | def atexit(callback, *args, **kwargs): | ||
1147 | 976 | '''Schedule a callback to run on successful hook completion. | ||
1148 | 977 | |||
1149 | 978 | Callbacks are run in the reverse order that they were added.''' | ||
1150 | 979 | _atexit.append((callback, args, kwargs)) | ||
1151 | 980 | |||
1152 | 981 | |||
1153 | 982 | def _run_atstart(): | ||
1154 | 983 | '''Hook frameworks must invoke this before running the main hook body.''' | ||
1155 | 984 | global _atstart | ||
1156 | 985 | for callback, args, kwargs in _atstart: | ||
1157 | 986 | callback(*args, **kwargs) | ||
1158 | 987 | del _atstart[:] | ||
1159 | 988 | |||
1160 | 989 | |||
1161 | 990 | def _run_atexit(): | ||
1162 | 991 | '''Hook frameworks must invoke this after the main hook body has | ||
1163 | 992 | successfully completed. Do not invoke it if the hook fails.''' | ||
1164 | 993 | global _atexit | ||
1165 | 994 | for callback, args, kwargs in reversed(_atexit): | ||
1166 | 995 | callback(*args, **kwargs) | ||
1167 | 996 | del _atexit[:] | ||
1168 | 997 | |||
1169 | 998 | |||
1170 | 999 | @translate_exc(from_exc=OSError, to_exc=NotImplementedError) | ||
1171 | 1000 | def network_get_primary_address(binding): | ||
1172 | 1001 | ''' | ||
1173 | 1002 | Retrieve the primary network address for a named binding | ||
1174 | 1003 | |||
1175 | 1004 | :param binding: string. The name of a relation of extra-binding | ||
1176 | 1005 | :return: string. The primary IP address for the named binding | ||
1177 | 1006 | :raise: NotImplementedError if run on Juju < 2.0 | ||
1178 | 1007 | ''' | ||
1179 | 1008 | cmd = ['network-get', '--primary-address', binding] | ||
1180 | 1009 | return subprocess.check_output(cmd).strip() | ||
1181 | 569 | 1010 | ||
1182 | === modified file 'hooks/charmhelpers/core/host.py' | |||
1183 | --- hooks/charmhelpers/core/host.py 2015-01-27 14:54:02 +0000 | |||
1184 | +++ hooks/charmhelpers/core/host.py 2016-04-05 14:45:53 +0000 | |||
1185 | @@ -24,11 +24,14 @@ | |||
1186 | 24 | import os | 24 | import os |
1187 | 25 | import re | 25 | import re |
1188 | 26 | import pwd | 26 | import pwd |
1189 | 27 | import glob | ||
1190 | 27 | import grp | 28 | import grp |
1191 | 28 | import random | 29 | import random |
1192 | 29 | import string | 30 | import string |
1193 | 30 | import subprocess | 31 | import subprocess |
1194 | 31 | import hashlib | 32 | import hashlib |
1195 | 33 | import functools | ||
1196 | 34 | import itertools | ||
1197 | 32 | from contextlib import contextmanager | 35 | from contextlib import contextmanager |
1198 | 33 | from collections import OrderedDict | 36 | from collections import OrderedDict |
1199 | 34 | 37 | ||
1200 | @@ -62,25 +65,86 @@ | |||
1201 | 62 | return service_result | 65 | return service_result |
1202 | 63 | 66 | ||
1203 | 64 | 67 | ||
1204 | 68 | def service_pause(service_name, init_dir="/etc/init", initd_dir="/etc/init.d"): | ||
1205 | 69 | """Pause a system service. | ||
1206 | 70 | |||
1207 | 71 | Stop it, and prevent it from starting again at boot.""" | ||
1208 | 72 | stopped = True | ||
1209 | 73 | if service_running(service_name): | ||
1210 | 74 | stopped = service_stop(service_name) | ||
1211 | 75 | upstart_file = os.path.join(init_dir, "{}.conf".format(service_name)) | ||
1212 | 76 | sysv_file = os.path.join(initd_dir, service_name) | ||
1213 | 77 | if init_is_systemd(): | ||
1214 | 78 | service('disable', service_name) | ||
1215 | 79 | elif os.path.exists(upstart_file): | ||
1216 | 80 | override_path = os.path.join( | ||
1217 | 81 | init_dir, '{}.override'.format(service_name)) | ||
1218 | 82 | with open(override_path, 'w') as fh: | ||
1219 | 83 | fh.write("manual\n") | ||
1220 | 84 | elif os.path.exists(sysv_file): | ||
1221 | 85 | subprocess.check_call(["update-rc.d", service_name, "disable"]) | ||
1222 | 86 | else: | ||
1223 | 87 | raise ValueError( | ||
1224 | 88 | "Unable to detect {0} as SystemD, Upstart {1} or" | ||
1225 | 89 | " SysV {2}".format( | ||
1226 | 90 | service_name, upstart_file, sysv_file)) | ||
1227 | 91 | return stopped | ||
1228 | 92 | |||
1229 | 93 | |||
1230 | 94 | def service_resume(service_name, init_dir="/etc/init", | ||
1231 | 95 | initd_dir="/etc/init.d"): | ||
1232 | 96 | """Resume a system service. | ||
1233 | 97 | |||
1234 | 98 | Reenable starting again at boot. Start the service""" | ||
1235 | 99 | upstart_file = os.path.join(init_dir, "{}.conf".format(service_name)) | ||
1236 | 100 | sysv_file = os.path.join(initd_dir, service_name) | ||
1237 | 101 | if init_is_systemd(): | ||
1238 | 102 | service('enable', service_name) | ||
1239 | 103 | elif os.path.exists(upstart_file): | ||
1240 | 104 | override_path = os.path.join( | ||
1241 | 105 | init_dir, '{}.override'.format(service_name)) | ||
1242 | 106 | if os.path.exists(override_path): | ||
1243 | 107 | os.unlink(override_path) | ||
1244 | 108 | elif os.path.exists(sysv_file): | ||
1245 | 109 | subprocess.check_call(["update-rc.d", service_name, "enable"]) | ||
1246 | 110 | else: | ||
1247 | 111 | raise ValueError( | ||
1248 | 112 | "Unable to detect {0} as SystemD, Upstart {1} or" | ||
1249 | 113 | " SysV {2}".format( | ||
1250 | 114 | service_name, upstart_file, sysv_file)) | ||
1251 | 115 | |||
1252 | 116 | started = service_running(service_name) | ||
1253 | 117 | if not started: | ||
1254 | 118 | started = service_start(service_name) | ||
1255 | 119 | return started | ||
1256 | 120 | |||
1257 | 121 | |||
1258 | 65 | def service(action, service_name): | 122 | def service(action, service_name): |
1259 | 66 | """Control a system service""" | 123 | """Control a system service""" |
1261 | 67 | cmd = ['service', service_name, action] | 124 | if init_is_systemd(): |
1262 | 125 | cmd = ['systemctl', action, service_name] | ||
1263 | 126 | else: | ||
1264 | 127 | cmd = ['service', service_name, action] | ||
1265 | 68 | return subprocess.call(cmd) == 0 | 128 | return subprocess.call(cmd) == 0 |
1266 | 69 | 129 | ||
1267 | 70 | 130 | ||
1269 | 71 | def service_running(service): | 131 | def service_running(service_name): |
1270 | 72 | """Determine whether a system service is running""" | 132 | """Determine whether a system service is running""" |
1277 | 73 | try: | 133 | if init_is_systemd(): |
1278 | 74 | output = subprocess.check_output( | 134 | return service('is-active', service_name) |
1273 | 75 | ['service', service, 'status'], | ||
1274 | 76 | stderr=subprocess.STDOUT).decode('UTF-8') | ||
1275 | 77 | except subprocess.CalledProcessError: | ||
1276 | 78 | return False | ||
1279 | 79 | else: | 135 | else: |
1283 | 80 | if ("start/running" in output or "is running" in output): | 136 | try: |
1284 | 81 | return True | 137 | output = subprocess.check_output( |
1285 | 82 | else: | 138 | ['service', service_name, 'status'], |
1286 | 139 | stderr=subprocess.STDOUT).decode('UTF-8') | ||
1287 | 140 | except subprocess.CalledProcessError: | ||
1288 | 83 | return False | 141 | return False |
1289 | 142 | else: | ||
1290 | 143 | if ("start/running" in output or "is running" in output or | ||
1291 | 144 | "up and running" in output): | ||
1292 | 145 | return True | ||
1293 | 146 | else: | ||
1294 | 147 | return False | ||
1295 | 84 | 148 | ||
1296 | 85 | 149 | ||
1297 | 86 | def service_available(service_name): | 150 | def service_available(service_name): |
1298 | @@ -90,13 +154,34 @@ | |||
1299 | 90 | ['service', service_name, 'status'], | 154 | ['service', service_name, 'status'], |
1300 | 91 | stderr=subprocess.STDOUT).decode('UTF-8') | 155 | stderr=subprocess.STDOUT).decode('UTF-8') |
1301 | 92 | except subprocess.CalledProcessError as e: | 156 | except subprocess.CalledProcessError as e: |
1303 | 93 | return 'unrecognized service' not in e.output | 157 | return b'unrecognized service' not in e.output |
1304 | 94 | else: | 158 | else: |
1305 | 95 | return True | 159 | return True |
1306 | 96 | 160 | ||
1307 | 97 | 161 | ||
1310 | 98 | def adduser(username, password=None, shell='/bin/bash', system_user=False): | 162 | SYSTEMD_SYSTEM = '/run/systemd/system' |
1311 | 99 | """Add a user to the system""" | 163 | |
1312 | 164 | |||
1313 | 165 | def init_is_systemd(): | ||
1314 | 166 | """Return True if the host system uses systemd, False otherwise.""" | ||
1315 | 167 | return os.path.isdir(SYSTEMD_SYSTEM) | ||
1316 | 168 | |||
1317 | 169 | |||
1318 | 170 | def adduser(username, password=None, shell='/bin/bash', system_user=False, | ||
1319 | 171 | primary_group=None, secondary_groups=None): | ||
1320 | 172 | """Add a user to the system. | ||
1321 | 173 | |||
1322 | 174 | Will log but otherwise succeed if the user already exists. | ||
1323 | 175 | |||
1324 | 176 | :param str username: Username to create | ||
1325 | 177 | :param str password: Password for user; if ``None``, create a system user | ||
1326 | 178 | :param str shell: The default shell for the user | ||
1327 | 179 | :param bool system_user: Whether to create a login or system user | ||
1328 | 180 | :param str primary_group: Primary group for user; defaults to username | ||
1329 | 181 | :param list secondary_groups: Optional list of additional groups | ||
1330 | 182 | |||
1331 | 183 | :returns: The password database entry struct, as returned by `pwd.getpwnam` | ||
1332 | 184 | """ | ||
1333 | 100 | try: | 185 | try: |
1334 | 101 | user_info = pwd.getpwnam(username) | 186 | user_info = pwd.getpwnam(username) |
1335 | 102 | log('user {0} already exists!'.format(username)) | 187 | log('user {0} already exists!'.format(username)) |
1336 | @@ -111,12 +196,32 @@ | |||
1337 | 111 | '--shell', shell, | 196 | '--shell', shell, |
1338 | 112 | '--password', password, | 197 | '--password', password, |
1339 | 113 | ]) | 198 | ]) |
1340 | 199 | if not primary_group: | ||
1341 | 200 | try: | ||
1342 | 201 | grp.getgrnam(username) | ||
1343 | 202 | primary_group = username # avoid "group exists" error | ||
1344 | 203 | except KeyError: | ||
1345 | 204 | pass | ||
1346 | 205 | if primary_group: | ||
1347 | 206 | cmd.extend(['-g', primary_group]) | ||
1348 | 207 | if secondary_groups: | ||
1349 | 208 | cmd.extend(['-G', ','.join(secondary_groups)]) | ||
1350 | 114 | cmd.append(username) | 209 | cmd.append(username) |
1351 | 115 | subprocess.check_call(cmd) | 210 | subprocess.check_call(cmd) |
1352 | 116 | user_info = pwd.getpwnam(username) | 211 | user_info = pwd.getpwnam(username) |
1353 | 117 | return user_info | 212 | return user_info |
1354 | 118 | 213 | ||
1355 | 119 | 214 | ||
1356 | 215 | def user_exists(username): | ||
1357 | 216 | """Check if a user exists""" | ||
1358 | 217 | try: | ||
1359 | 218 | pwd.getpwnam(username) | ||
1360 | 219 | user_exists = True | ||
1361 | 220 | except KeyError: | ||
1362 | 221 | user_exists = False | ||
1363 | 222 | return user_exists | ||
1364 | 223 | |||
1365 | 224 | |||
1366 | 120 | def add_group(group_name, system_group=False): | 225 | def add_group(group_name, system_group=False): |
1367 | 121 | """Add a group to the system""" | 226 | """Add a group to the system""" |
1368 | 122 | try: | 227 | try: |
1369 | @@ -139,11 +244,7 @@ | |||
1370 | 139 | 244 | ||
1371 | 140 | def add_user_to_group(username, group): | 245 | def add_user_to_group(username, group): |
1372 | 141 | """Add a user to a group""" | 246 | """Add a user to a group""" |
1378 | 142 | cmd = [ | 247 | cmd = ['gpasswd', '-a', username, group] |
1374 | 143 | 'gpasswd', '-a', | ||
1375 | 144 | username, | ||
1376 | 145 | group | ||
1377 | 146 | ] | ||
1379 | 147 | log("Adding user {} to group {}".format(username, group)) | 248 | log("Adding user {} to group {}".format(username, group)) |
1380 | 148 | subprocess.check_call(cmd) | 249 | subprocess.check_call(cmd) |
1381 | 149 | 250 | ||
1382 | @@ -191,25 +292,23 @@ | |||
1383 | 191 | 292 | ||
1384 | 192 | 293 | ||
1385 | 193 | def write_file(path, content, owner='root', group='root', perms=0o444): | 294 | def write_file(path, content, owner='root', group='root', perms=0o444): |
1387 | 194 | """Create or overwrite a file with the contents of a string""" | 295 | """Create or overwrite a file with the contents of a byte string.""" |
1388 | 195 | log("Writing file {} {}:{} {:o}".format(path, owner, group, perms)) | 296 | log("Writing file {} {}:{} {:o}".format(path, owner, group, perms)) |
1389 | 196 | uid = pwd.getpwnam(owner).pw_uid | 297 | uid = pwd.getpwnam(owner).pw_uid |
1390 | 197 | gid = grp.getgrnam(group).gr_gid | 298 | gid = grp.getgrnam(group).gr_gid |
1392 | 198 | with open(path, 'w') as target: | 299 | with open(path, 'wb') as target: |
1393 | 199 | os.fchown(target.fileno(), uid, gid) | 300 | os.fchown(target.fileno(), uid, gid) |
1394 | 200 | os.fchmod(target.fileno(), perms) | 301 | os.fchmod(target.fileno(), perms) |
1395 | 201 | target.write(content) | 302 | target.write(content) |
1396 | 202 | 303 | ||
1397 | 203 | 304 | ||
1398 | 204 | def fstab_remove(mp): | 305 | def fstab_remove(mp): |
1401 | 205 | """Remove the given mountpoint entry from /etc/fstab | 306 | """Remove the given mountpoint entry from /etc/fstab""" |
1400 | 206 | """ | ||
1402 | 207 | return Fstab.remove_by_mountpoint(mp) | 307 | return Fstab.remove_by_mountpoint(mp) |
1403 | 208 | 308 | ||
1404 | 209 | 309 | ||
1405 | 210 | def fstab_add(dev, mp, fs, options=None): | 310 | def fstab_add(dev, mp, fs, options=None): |
1408 | 211 | """Adds the given device entry to the /etc/fstab file | 311 | """Adds the given device entry to the /etc/fstab file""" |
1407 | 212 | """ | ||
1409 | 213 | return Fstab.add(dev, mp, fs, options=options) | 312 | return Fstab.add(dev, mp, fs, options=options) |
1410 | 214 | 313 | ||
1411 | 215 | 314 | ||
1412 | @@ -253,9 +352,19 @@ | |||
1413 | 253 | return system_mounts | 352 | return system_mounts |
1414 | 254 | 353 | ||
1415 | 255 | 354 | ||
1416 | 355 | def fstab_mount(mountpoint): | ||
1417 | 356 | """Mount filesystem using fstab""" | ||
1418 | 357 | cmd_args = ['mount', mountpoint] | ||
1419 | 358 | try: | ||
1420 | 359 | subprocess.check_output(cmd_args) | ||
1421 | 360 | except subprocess.CalledProcessError as e: | ||
1422 | 361 | log('Error unmounting {}\n{}'.format(mountpoint, e.output)) | ||
1423 | 362 | return False | ||
1424 | 363 | return True | ||
1425 | 364 | |||
1426 | 365 | |||
1427 | 256 | def file_hash(path, hash_type='md5'): | 366 | def file_hash(path, hash_type='md5'): |
1430 | 257 | """ | 367 | """Generate a hash checksum of the contents of 'path' or None if not found. |
1429 | 258 | Generate a hash checksum of the contents of 'path' or None if not found. | ||
1431 | 259 | 368 | ||
1432 | 260 | :param str hash_type: Any hash alrgorithm supported by :mod:`hashlib`, | 369 | :param str hash_type: Any hash alrgorithm supported by :mod:`hashlib`, |
1433 | 261 | such as md5, sha1, sha256, sha512, etc. | 370 | such as md5, sha1, sha256, sha512, etc. |
1434 | @@ -269,9 +378,22 @@ | |||
1435 | 269 | return None | 378 | return None |
1436 | 270 | 379 | ||
1437 | 271 | 380 | ||
1438 | 381 | def path_hash(path): | ||
1439 | 382 | """Generate a hash checksum of all files matching 'path'. Standard | ||
1440 | 383 | wildcards like '*' and '?' are supported, see documentation for the 'glob' | ||
1441 | 384 | module for more information. | ||
1442 | 385 | |||
1443 | 386 | :return: dict: A { filename: hash } dictionary for all matched files. | ||
1444 | 387 | Empty if none found. | ||
1445 | 388 | """ | ||
1446 | 389 | return { | ||
1447 | 390 | filename: file_hash(filename) | ||
1448 | 391 | for filename in glob.iglob(path) | ||
1449 | 392 | } | ||
1450 | 393 | |||
1451 | 394 | |||
1452 | 272 | def check_hash(path, checksum, hash_type='md5'): | 395 | def check_hash(path, checksum, hash_type='md5'): |
1455 | 273 | """ | 396 | """Validate a file using a cryptographic checksum. |
1454 | 274 | Validate a file using a cryptographic checksum. | ||
1456 | 275 | 397 | ||
1457 | 276 | :param str checksum: Value of the checksum used to validate the file. | 398 | :param str checksum: Value of the checksum used to validate the file. |
1458 | 277 | :param str hash_type: Hash algorithm used to generate `checksum`. | 399 | :param str hash_type: Hash algorithm used to generate `checksum`. |
1459 | @@ -286,6 +408,7 @@ | |||
1460 | 286 | 408 | ||
1461 | 287 | 409 | ||
1462 | 288 | class ChecksumError(ValueError): | 410 | class ChecksumError(ValueError): |
1463 | 411 | """A class derived from Value error to indicate the checksum failed.""" | ||
1464 | 289 | pass | 412 | pass |
1465 | 290 | 413 | ||
1466 | 291 | 414 | ||
1467 | @@ -296,36 +419,58 @@ | |||
1468 | 296 | 419 | ||
1469 | 297 | @restart_on_change({ | 420 | @restart_on_change({ |
1470 | 298 | '/etc/ceph/ceph.conf': [ 'cinder-api', 'cinder-volume' ] | 421 | '/etc/ceph/ceph.conf': [ 'cinder-api', 'cinder-volume' ] |
1471 | 422 | '/etc/apache/sites-enabled/*': [ 'apache2' ] | ||
1472 | 299 | }) | 423 | }) |
1474 | 300 | def ceph_client_changed(): | 424 | def config_changed(): |
1475 | 301 | pass # your code here | 425 | pass # your code here |
1476 | 302 | 426 | ||
1477 | 303 | In this example, the cinder-api and cinder-volume services | 427 | In this example, the cinder-api and cinder-volume services |
1478 | 304 | would be restarted if /etc/ceph/ceph.conf is changed by the | 428 | would be restarted if /etc/ceph/ceph.conf is changed by the |
1480 | 305 | ceph_client_changed function. | 429 | ceph_client_changed function. The apache2 service would be |
1481 | 430 | restarted if any file matching the pattern got changed, created | ||
1482 | 431 | or removed. Standard wildcards are supported, see documentation | ||
1483 | 432 | for the 'glob' module for more information. | ||
1484 | 433 | |||
1485 | 434 | @param restart_map: {path_file_name: [service_name, ...] | ||
1486 | 435 | @param stopstart: DEFAULT false; whether to stop, start OR restart | ||
1487 | 436 | @returns result from decorated function | ||
1488 | 306 | """ | 437 | """ |
1489 | 307 | def wrap(f): | 438 | def wrap(f): |
1507 | 308 | def wrapped_f(*args): | 439 | @functools.wraps(f) |
1508 | 309 | checksums = {} | 440 | def wrapped_f(*args, **kwargs): |
1509 | 310 | for path in restart_map: | 441 | return restart_on_change_helper( |
1510 | 311 | checksums[path] = file_hash(path) | 442 | (lambda: f(*args, **kwargs)), restart_map, stopstart) |
1494 | 312 | f(*args) | ||
1495 | 313 | restarts = [] | ||
1496 | 314 | for path in restart_map: | ||
1497 | 315 | if checksums[path] != file_hash(path): | ||
1498 | 316 | restarts += restart_map[path] | ||
1499 | 317 | services_list = list(OrderedDict.fromkeys(restarts)) | ||
1500 | 318 | if not stopstart: | ||
1501 | 319 | for service_name in services_list: | ||
1502 | 320 | service('restart', service_name) | ||
1503 | 321 | else: | ||
1504 | 322 | for action in ['stop', 'start']: | ||
1505 | 323 | for service_name in services_list: | ||
1506 | 324 | service(action, service_name) | ||
1511 | 325 | return wrapped_f | 443 | return wrapped_f |
1512 | 326 | return wrap | 444 | return wrap |
1513 | 327 | 445 | ||
1514 | 328 | 446 | ||
1515 | 447 | def restart_on_change_helper(lambda_f, restart_map, stopstart=False): | ||
1516 | 448 | """Helper function to perform the restart_on_change function. | ||
1517 | 449 | |||
1518 | 450 | This is provided for decorators to restart services if files described | ||
1519 | 451 | in the restart_map have changed after an invocation of lambda_f(). | ||
1520 | 452 | |||
1521 | 453 | @param lambda_f: function to call. | ||
1522 | 454 | @param restart_map: {file: [service, ...]} | ||
1523 | 455 | @param stopstart: whether to stop, start or restart a service | ||
1524 | 456 | @returns result of lambda_f() | ||
1525 | 457 | """ | ||
1526 | 458 | checksums = {path: path_hash(path) for path in restart_map} | ||
1527 | 459 | r = lambda_f() | ||
1528 | 460 | # create a list of lists of the services to restart | ||
1529 | 461 | restarts = [restart_map[path] | ||
1530 | 462 | for path in restart_map | ||
1531 | 463 | if path_hash(path) != checksums[path]] | ||
1532 | 464 | # create a flat list of ordered services without duplicates from lists | ||
1533 | 465 | services_list = list(OrderedDict.fromkeys(itertools.chain(*restarts))) | ||
1534 | 466 | if services_list: | ||
1535 | 467 | actions = ('stop', 'start') if stopstart else ('restart',) | ||
1536 | 468 | for action in actions: | ||
1537 | 469 | for service_name in services_list: | ||
1538 | 470 | service(action, service_name) | ||
1539 | 471 | return r | ||
1540 | 472 | |||
1541 | 473 | |||
1542 | 329 | def lsb_release(): | 474 | def lsb_release(): |
1543 | 330 | """Return /etc/lsb-release in a dict""" | 475 | """Return /etc/lsb-release in a dict""" |
1544 | 331 | d = {} | 476 | d = {} |
1545 | @@ -339,45 +484,105 @@ | |||
1546 | 339 | def pwgen(length=None): | 484 | def pwgen(length=None): |
1547 | 340 | """Generate a random pasword.""" | 485 | """Generate a random pasword.""" |
1548 | 341 | if length is None: | 486 | if length is None: |
1549 | 487 | # A random length is ok to use a weak PRNG | ||
1550 | 342 | length = random.choice(range(35, 45)) | 488 | length = random.choice(range(35, 45)) |
1551 | 343 | alphanumeric_chars = [ | 489 | alphanumeric_chars = [ |
1552 | 344 | l for l in (string.ascii_letters + string.digits) | 490 | l for l in (string.ascii_letters + string.digits) |
1553 | 345 | if l not in 'l0QD1vAEIOUaeiou'] | 491 | if l not in 'l0QD1vAEIOUaeiou'] |
1554 | 492 | # Use a crypto-friendly PRNG (e.g. /dev/urandom) for making the | ||
1555 | 493 | # actual password | ||
1556 | 494 | random_generator = random.SystemRandom() | ||
1557 | 346 | random_chars = [ | 495 | random_chars = [ |
1559 | 347 | random.choice(alphanumeric_chars) for _ in range(length)] | 496 | random_generator.choice(alphanumeric_chars) for _ in range(length)] |
1560 | 348 | return(''.join(random_chars)) | 497 | return(''.join(random_chars)) |
1561 | 349 | 498 | ||
1562 | 350 | 499 | ||
1565 | 351 | def list_nics(nic_type): | 500 | def is_phy_iface(interface): |
1566 | 352 | '''Return a list of nics of given type(s)''' | 501 | """Returns True if interface is not virtual, otherwise False.""" |
1567 | 502 | if interface: | ||
1568 | 503 | sys_net = '/sys/class/net' | ||
1569 | 504 | if os.path.isdir(sys_net): | ||
1570 | 505 | for iface in glob.glob(os.path.join(sys_net, '*')): | ||
1571 | 506 | if '/virtual/' in os.path.realpath(iface): | ||
1572 | 507 | continue | ||
1573 | 508 | |||
1574 | 509 | if interface == os.path.basename(iface): | ||
1575 | 510 | return True | ||
1576 | 511 | |||
1577 | 512 | return False | ||
1578 | 513 | |||
1579 | 514 | |||
1580 | 515 | def get_bond_master(interface): | ||
1581 | 516 | """Returns bond master if interface is bond slave otherwise None. | ||
1582 | 517 | |||
1583 | 518 | NOTE: the provided interface is expected to be physical | ||
1584 | 519 | """ | ||
1585 | 520 | if interface: | ||
1586 | 521 | iface_path = '/sys/class/net/%s' % (interface) | ||
1587 | 522 | if os.path.exists(iface_path): | ||
1588 | 523 | if '/virtual/' in os.path.realpath(iface_path): | ||
1589 | 524 | return None | ||
1590 | 525 | |||
1591 | 526 | master = os.path.join(iface_path, 'master') | ||
1592 | 527 | if os.path.exists(master): | ||
1593 | 528 | master = os.path.realpath(master) | ||
1594 | 529 | # make sure it is a bond master | ||
1595 | 530 | if os.path.exists(os.path.join(master, 'bonding')): | ||
1596 | 531 | return os.path.basename(master) | ||
1597 | 532 | |||
1598 | 533 | return None | ||
1599 | 534 | |||
1600 | 535 | |||
1601 | 536 | def list_nics(nic_type=None): | ||
1602 | 537 | """Return a list of nics of given type(s)""" | ||
1603 | 353 | if isinstance(nic_type, six.string_types): | 538 | if isinstance(nic_type, six.string_types): |
1604 | 354 | int_types = [nic_type] | 539 | int_types = [nic_type] |
1605 | 355 | else: | 540 | else: |
1606 | 356 | int_types = nic_type | 541 | int_types = nic_type |
1607 | 542 | |||
1608 | 357 | interfaces = [] | 543 | interfaces = [] |
1611 | 358 | for int_type in int_types: | 544 | if nic_type: |
1612 | 359 | cmd = ['ip', 'addr', 'show', 'label', int_type + '*'] | 545 | for int_type in int_types: |
1613 | 546 | cmd = ['ip', 'addr', 'show', 'label', int_type + '*'] | ||
1614 | 547 | ip_output = subprocess.check_output(cmd).decode('UTF-8') | ||
1615 | 548 | ip_output = ip_output.split('\n') | ||
1616 | 549 | ip_output = (line for line in ip_output if line) | ||
1617 | 550 | for line in ip_output: | ||
1618 | 551 | if line.split()[1].startswith(int_type): | ||
1619 | 552 | matched = re.search('.*: (' + int_type + | ||
1620 | 553 | r'[0-9]+\.[0-9]+)@.*', line) | ||
1621 | 554 | if matched: | ||
1622 | 555 | iface = matched.groups()[0] | ||
1623 | 556 | else: | ||
1624 | 557 | iface = line.split()[1].replace(":", "") | ||
1625 | 558 | |||
1626 | 559 | if iface not in interfaces: | ||
1627 | 560 | interfaces.append(iface) | ||
1628 | 561 | else: | ||
1629 | 562 | cmd = ['ip', 'a'] | ||
1630 | 360 | ip_output = subprocess.check_output(cmd).decode('UTF-8').split('\n') | 563 | ip_output = subprocess.check_output(cmd).decode('UTF-8').split('\n') |
1632 | 361 | ip_output = (line for line in ip_output if line) | 564 | ip_output = (line.strip() for line in ip_output if line) |
1633 | 565 | |||
1634 | 566 | key = re.compile('^[0-9]+:\s+(.+):') | ||
1635 | 362 | for line in ip_output: | 567 | for line in ip_output: |
1643 | 363 | if line.split()[1].startswith(int_type): | 568 | matched = re.search(key, line) |
1644 | 364 | matched = re.search('.*: (bond[0-9]+\.[0-9]+)@.*', line) | 569 | if matched: |
1645 | 365 | if matched: | 570 | iface = matched.group(1) |
1646 | 366 | interface = matched.groups()[0] | 571 | iface = iface.partition("@")[0] |
1647 | 367 | else: | 572 | if iface not in interfaces: |
1648 | 368 | interface = line.split()[1].replace(":", "") | 573 | interfaces.append(iface) |
1642 | 369 | interfaces.append(interface) | ||
1649 | 370 | 574 | ||
1650 | 371 | return interfaces | 575 | return interfaces |
1651 | 372 | 576 | ||
1652 | 373 | 577 | ||
1653 | 374 | def set_nic_mtu(nic, mtu): | 578 | def set_nic_mtu(nic, mtu): |
1655 | 375 | '''Set MTU on a network interface''' | 579 | """Set the Maximum Transmission Unit (MTU) on a network interface.""" |
1656 | 376 | cmd = ['ip', 'link', 'set', nic, 'mtu', mtu] | 580 | cmd = ['ip', 'link', 'set', nic, 'mtu', mtu] |
1657 | 377 | subprocess.check_call(cmd) | 581 | subprocess.check_call(cmd) |
1658 | 378 | 582 | ||
1659 | 379 | 583 | ||
1660 | 380 | def get_nic_mtu(nic): | 584 | def get_nic_mtu(nic): |
1661 | 585 | """Return the Maximum Transmission Unit (MTU) for a network interface.""" | ||
1662 | 381 | cmd = ['ip', 'addr', 'show', nic] | 586 | cmd = ['ip', 'addr', 'show', nic] |
1663 | 382 | ip_output = subprocess.check_output(cmd).decode('UTF-8').split('\n') | 587 | ip_output = subprocess.check_output(cmd).decode('UTF-8').split('\n') |
1664 | 383 | mtu = "" | 588 | mtu = "" |
1665 | @@ -389,6 +594,7 @@ | |||
1666 | 389 | 594 | ||
1667 | 390 | 595 | ||
1668 | 391 | def get_nic_hwaddr(nic): | 596 | def get_nic_hwaddr(nic): |
1669 | 597 | """Return the Media Access Control (MAC) for a network interface.""" | ||
1670 | 392 | cmd = ['ip', '-o', '-0', 'addr', 'show', nic] | 598 | cmd = ['ip', '-o', '-0', 'addr', 'show', nic] |
1671 | 393 | ip_output = subprocess.check_output(cmd).decode('UTF-8') | 599 | ip_output = subprocess.check_output(cmd).decode('UTF-8') |
1672 | 394 | hwaddr = "" | 600 | hwaddr = "" |
1673 | @@ -399,7 +605,7 @@ | |||
1674 | 399 | 605 | ||
1675 | 400 | 606 | ||
1676 | 401 | def cmp_pkgrevno(package, revno, pkgcache=None): | 607 | def cmp_pkgrevno(package, revno, pkgcache=None): |
1678 | 402 | '''Compare supplied revno with the revno of the installed package | 608 | """Compare supplied revno with the revno of the installed package |
1679 | 403 | 609 | ||
1680 | 404 | * 1 => Installed revno is greater than supplied arg | 610 | * 1 => Installed revno is greater than supplied arg |
1681 | 405 | * 0 => Installed revno is the same as supplied arg | 611 | * 0 => Installed revno is the same as supplied arg |
1682 | @@ -408,7 +614,7 @@ | |||
1683 | 408 | This function imports apt_cache function from charmhelpers.fetch if | 614 | This function imports apt_cache function from charmhelpers.fetch if |
1684 | 409 | the pkgcache argument is None. Be sure to add charmhelpers.fetch if | 615 | the pkgcache argument is None. Be sure to add charmhelpers.fetch if |
1685 | 410 | you call this function, or pass an apt_pkg.Cache() instance. | 616 | you call this function, or pass an apt_pkg.Cache() instance. |
1687 | 411 | ''' | 617 | """ |
1688 | 412 | import apt_pkg | 618 | import apt_pkg |
1689 | 413 | if not pkgcache: | 619 | if not pkgcache: |
1690 | 414 | from charmhelpers.fetch import apt_cache | 620 | from charmhelpers.fetch import apt_cache |
1691 | @@ -418,15 +624,30 @@ | |||
1692 | 418 | 624 | ||
1693 | 419 | 625 | ||
1694 | 420 | @contextmanager | 626 | @contextmanager |
1696 | 421 | def chdir(d): | 627 | def chdir(directory): |
1697 | 628 | """Change the current working directory to a different directory for a code | ||
1698 | 629 | block and return the previous directory after the block exits. Useful to | ||
1699 | 630 | run commands from a specificed directory. | ||
1700 | 631 | |||
1701 | 632 | :param str directory: The directory path to change to for this context. | ||
1702 | 633 | """ | ||
1703 | 422 | cur = os.getcwd() | 634 | cur = os.getcwd() |
1704 | 423 | try: | 635 | try: |
1706 | 424 | yield os.chdir(d) | 636 | yield os.chdir(directory) |
1707 | 425 | finally: | 637 | finally: |
1708 | 426 | os.chdir(cur) | 638 | os.chdir(cur) |
1709 | 427 | 639 | ||
1710 | 428 | 640 | ||
1712 | 429 | def chownr(path, owner, group, follow_links=True): | 641 | def chownr(path, owner, group, follow_links=True, chowntopdir=False): |
1713 | 642 | """Recursively change user and group ownership of files and directories | ||
1714 | 643 | in given path. Doesn't chown path itself by default, only its children. | ||
1715 | 644 | |||
1716 | 645 | :param str path: The string path to start changing ownership. | ||
1717 | 646 | :param str owner: The owner string to use when looking up the uid. | ||
1718 | 647 | :param str group: The group string to use when looking up the gid. | ||
1719 | 648 | :param bool follow_links: Also Chown links if True | ||
1720 | 649 | :param bool chowntopdir: Also chown path itself if True | ||
1721 | 650 | """ | ||
1722 | 430 | uid = pwd.getpwnam(owner).pw_uid | 651 | uid = pwd.getpwnam(owner).pw_uid |
1723 | 431 | gid = grp.getgrnam(group).gr_gid | 652 | gid = grp.getgrnam(group).gr_gid |
1724 | 432 | if follow_links: | 653 | if follow_links: |
1725 | @@ -434,6 +655,10 @@ | |||
1726 | 434 | else: | 655 | else: |
1727 | 435 | chown = os.lchown | 656 | chown = os.lchown |
1728 | 436 | 657 | ||
1729 | 658 | if chowntopdir: | ||
1730 | 659 | broken_symlink = os.path.lexists(path) and not os.path.exists(path) | ||
1731 | 660 | if not broken_symlink: | ||
1732 | 661 | chown(path, uid, gid) | ||
1733 | 437 | for root, dirs, files in os.walk(path): | 662 | for root, dirs, files in os.walk(path): |
1734 | 438 | for name in dirs + files: | 663 | for name in dirs + files: |
1735 | 439 | full = os.path.join(root, name) | 664 | full = os.path.join(root, name) |
1736 | @@ -443,4 +668,28 @@ | |||
1737 | 443 | 668 | ||
1738 | 444 | 669 | ||
1739 | 445 | def lchownr(path, owner, group): | 670 | def lchownr(path, owner, group): |
1740 | 671 | """Recursively change user and group ownership of files and directories | ||
1741 | 672 | in a given path, not following symbolic links. See the documentation for | ||
1742 | 673 | 'os.lchown' for more information. | ||
1743 | 674 | |||
1744 | 675 | :param str path: The string path to start changing ownership. | ||
1745 | 676 | :param str owner: The owner string to use when looking up the uid. | ||
1746 | 677 | :param str group: The group string to use when looking up the gid. | ||
1747 | 678 | """ | ||
1748 | 446 | chownr(path, owner, group, follow_links=False) | 679 | chownr(path, owner, group, follow_links=False) |
1749 | 680 | |||
1750 | 681 | |||
1751 | 682 | def get_total_ram(): | ||
1752 | 683 | """The total amount of system RAM in bytes. | ||
1753 | 684 | |||
1754 | 685 | This is what is reported by the OS, and may be overcommitted when | ||
1755 | 686 | there are multiple containers hosted on the same machine. | ||
1756 | 687 | """ | ||
1757 | 688 | with open('/proc/meminfo', 'r') as f: | ||
1758 | 689 | for line in f.readlines(): | ||
1759 | 690 | if line: | ||
1760 | 691 | key, value, unit = line.split() | ||
1761 | 692 | if key == 'MemTotal:': | ||
1762 | 693 | assert unit == 'kB', 'Unknown unit' | ||
1763 | 694 | return int(value) * 1024 # Classic, not KiB. | ||
1764 | 695 | raise NotImplementedError() | ||
1765 | 447 | 696 | ||
1766 | === added file 'hooks/charmhelpers/core/hugepage.py' | |||
1767 | --- hooks/charmhelpers/core/hugepage.py 1970-01-01 00:00:00 +0000 | |||
1768 | +++ hooks/charmhelpers/core/hugepage.py 2016-04-05 14:45:53 +0000 | |||
1769 | @@ -0,0 +1,71 @@ | |||
1770 | 1 | # -*- coding: utf-8 -*- | ||
1771 | 2 | |||
1772 | 3 | # Copyright 2014-2015 Canonical Limited. | ||
1773 | 4 | # | ||
1774 | 5 | # This file is part of charm-helpers. | ||
1775 | 6 | # | ||
1776 | 7 | # charm-helpers is free software: you can redistribute it and/or modify | ||
1777 | 8 | # it under the terms of the GNU Lesser General Public License version 3 as | ||
1778 | 9 | # published by the Free Software Foundation. | ||
1779 | 10 | # | ||
1780 | 11 | # charm-helpers is distributed in the hope that it will be useful, | ||
1781 | 12 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | ||
1782 | 13 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | ||
1783 | 14 | # GNU Lesser General Public License for more details. | ||
1784 | 15 | # | ||
1785 | 16 | # You should have received a copy of the GNU Lesser General Public License | ||
1786 | 17 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
1787 | 18 | |||
1788 | 19 | import yaml | ||
1789 | 20 | from charmhelpers.core import fstab | ||
1790 | 21 | from charmhelpers.core import sysctl | ||
1791 | 22 | from charmhelpers.core.host import ( | ||
1792 | 23 | add_group, | ||
1793 | 24 | add_user_to_group, | ||
1794 | 25 | fstab_mount, | ||
1795 | 26 | mkdir, | ||
1796 | 27 | ) | ||
1797 | 28 | from charmhelpers.core.strutils import bytes_from_string | ||
1798 | 29 | from subprocess import check_output | ||
1799 | 30 | |||
1800 | 31 | |||
1801 | 32 | def hugepage_support(user, group='hugetlb', nr_hugepages=256, | ||
1802 | 33 | max_map_count=65536, mnt_point='/run/hugepages/kvm', | ||
1803 | 34 | pagesize='2MB', mount=True, set_shmmax=False): | ||
1804 | 35 | """Enable hugepages on system. | ||
1805 | 36 | |||
1806 | 37 | Args: | ||
1807 | 38 | user (str) -- Username to allow access to hugepages to | ||
1808 | 39 | group (str) -- Group name to own hugepages | ||
1809 | 40 | nr_hugepages (int) -- Number of pages to reserve | ||
1810 | 41 | max_map_count (int) -- Number of Virtual Memory Areas a process can own | ||
1811 | 42 | mnt_point (str) -- Directory to mount hugepages on | ||
1812 | 43 | pagesize (str) -- Size of hugepages | ||
1813 | 44 | mount (bool) -- Whether to Mount hugepages | ||
1814 | 45 | """ | ||
1815 | 46 | group_info = add_group(group) | ||
1816 | 47 | gid = group_info.gr_gid | ||
1817 | 48 | add_user_to_group(user, group) | ||
1818 | 49 | if max_map_count < 2 * nr_hugepages: | ||
1819 | 50 | max_map_count = 2 * nr_hugepages | ||
1820 | 51 | sysctl_settings = { | ||
1821 | 52 | 'vm.nr_hugepages': nr_hugepages, | ||
1822 | 53 | 'vm.max_map_count': max_map_count, | ||
1823 | 54 | 'vm.hugetlb_shm_group': gid, | ||
1824 | 55 | } | ||
1825 | 56 | if set_shmmax: | ||
1826 | 57 | shmmax_current = int(check_output(['sysctl', '-n', 'kernel.shmmax'])) | ||
1827 | 58 | shmmax_minsize = bytes_from_string(pagesize) * nr_hugepages | ||
1828 | 59 | if shmmax_minsize > shmmax_current: | ||
1829 | 60 | sysctl_settings['kernel.shmmax'] = shmmax_minsize | ||
1830 | 61 | sysctl.create(yaml.dump(sysctl_settings), '/etc/sysctl.d/10-hugepage.conf') | ||
1831 | 62 | mkdir(mnt_point, owner='root', group='root', perms=0o755, force=False) | ||
1832 | 63 | lfstab = fstab.Fstab() | ||
1833 | 64 | fstab_entry = lfstab.get_entry_by_attr('mountpoint', mnt_point) | ||
1834 | 65 | if fstab_entry: | ||
1835 | 66 | lfstab.remove_entry(fstab_entry) | ||
1836 | 67 | entry = lfstab.Entry('nodev', mnt_point, 'hugetlbfs', | ||
1837 | 68 | 'mode=1770,gid={},pagesize={}'.format(gid, pagesize), 0, 0) | ||
1838 | 69 | lfstab.add_entry(entry) | ||
1839 | 70 | if mount: | ||
1840 | 71 | fstab_mount(mnt_point) | ||
1841 | 0 | 72 | ||
1842 | === added file 'hooks/charmhelpers/core/kernel.py' | |||
1843 | --- hooks/charmhelpers/core/kernel.py 1970-01-01 00:00:00 +0000 | |||
1844 | +++ hooks/charmhelpers/core/kernel.py 2016-04-05 14:45:53 +0000 | |||
1845 | @@ -0,0 +1,68 @@ | |||
1846 | 1 | #!/usr/bin/env python | ||
1847 | 2 | # -*- coding: utf-8 -*- | ||
1848 | 3 | |||
1849 | 4 | # Copyright 2014-2015 Canonical Limited. | ||
1850 | 5 | # | ||
1851 | 6 | # This file is part of charm-helpers. | ||
1852 | 7 | # | ||
1853 | 8 | # charm-helpers is free software: you can redistribute it and/or modify | ||
1854 | 9 | # it under the terms of the GNU Lesser General Public License version 3 as | ||
1855 | 10 | # published by the Free Software Foundation. | ||
1856 | 11 | # | ||
1857 | 12 | # charm-helpers is distributed in the hope that it will be useful, | ||
1858 | 13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | ||
1859 | 14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | ||
1860 | 15 | # GNU Lesser General Public License for more details. | ||
1861 | 16 | # | ||
1862 | 17 | # You should have received a copy of the GNU Lesser General Public License | ||
1863 | 18 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
1864 | 19 | |||
1865 | 20 | __author__ = "Jorge Niedbalski <jorge.niedbalski@canonical.com>" | ||
1866 | 21 | |||
1867 | 22 | from charmhelpers.core.hookenv import ( | ||
1868 | 23 | log, | ||
1869 | 24 | INFO | ||
1870 | 25 | ) | ||
1871 | 26 | |||
1872 | 27 | from subprocess import check_call, check_output | ||
1873 | 28 | import re | ||
1874 | 29 | |||
1875 | 30 | |||
1876 | 31 | def modprobe(module, persist=True): | ||
1877 | 32 | """Load a kernel module and configure for auto-load on reboot.""" | ||
1878 | 33 | cmd = ['modprobe', module] | ||
1879 | 34 | |||
1880 | 35 | log('Loading kernel module %s' % module, level=INFO) | ||
1881 | 36 | |||
1882 | 37 | check_call(cmd) | ||
1883 | 38 | if persist: | ||
1884 | 39 | with open('/etc/modules', 'r+') as modules: | ||
1885 | 40 | if module not in modules.read(): | ||
1886 | 41 | modules.write(module) | ||
1887 | 42 | |||
1888 | 43 | |||
1889 | 44 | def rmmod(module, force=False): | ||
1890 | 45 | """Remove a module from the linux kernel""" | ||
1891 | 46 | cmd = ['rmmod'] | ||
1892 | 47 | if force: | ||
1893 | 48 | cmd.append('-f') | ||
1894 | 49 | cmd.append(module) | ||
1895 | 50 | log('Removing kernel module %s' % module, level=INFO) | ||
1896 | 51 | return check_call(cmd) | ||
1897 | 52 | |||
1898 | 53 | |||
1899 | 54 | def lsmod(): | ||
1900 | 55 | """Shows what kernel modules are currently loaded""" | ||
1901 | 56 | return check_output(['lsmod'], | ||
1902 | 57 | universal_newlines=True) | ||
1903 | 58 | |||
1904 | 59 | |||
1905 | 60 | def is_module_loaded(module): | ||
1906 | 61 | """Checks if a kernel module is already loaded""" | ||
1907 | 62 | matches = re.findall('^%s[ ]+' % module, lsmod(), re.M) | ||
1908 | 63 | return len(matches) > 0 | ||
1909 | 64 | |||
1910 | 65 | |||
1911 | 66 | def update_initramfs(version='all'): | ||
1912 | 67 | """Updates an initramfs image""" | ||
1913 | 68 | return check_call(["update-initramfs", "-k", version, "-u"]) | ||
1914 | 0 | 69 | ||
1915 | === modified file 'hooks/charmhelpers/core/services/base.py' | |||
1916 | --- hooks/charmhelpers/core/services/base.py 2015-01-27 14:54:02 +0000 | |||
1917 | +++ hooks/charmhelpers/core/services/base.py 2016-04-05 14:45:53 +0000 | |||
1918 | @@ -15,9 +15,9 @@ | |||
1919 | 15 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | 15 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
1920 | 16 | 16 | ||
1921 | 17 | import os | 17 | import os |
1922 | 18 | import re | ||
1923 | 19 | import json | 18 | import json |
1925 | 20 | from collections import Iterable | 19 | from inspect import getargspec |
1926 | 20 | from collections import Iterable, OrderedDict | ||
1927 | 21 | 21 | ||
1928 | 22 | from charmhelpers.core import host | 22 | from charmhelpers.core import host |
1929 | 23 | from charmhelpers.core import hookenv | 23 | from charmhelpers.core import hookenv |
1930 | @@ -119,7 +119,7 @@ | |||
1931 | 119 | """ | 119 | """ |
1932 | 120 | self._ready_file = os.path.join(hookenv.charm_dir(), 'READY-SERVICES.json') | 120 | self._ready_file = os.path.join(hookenv.charm_dir(), 'READY-SERVICES.json') |
1933 | 121 | self._ready = None | 121 | self._ready = None |
1935 | 122 | self.services = {} | 122 | self.services = OrderedDict() |
1936 | 123 | for service in services or []: | 123 | for service in services or []: |
1937 | 124 | service_name = service['service'] | 124 | service_name = service['service'] |
1938 | 125 | self.services[service_name] = service | 125 | self.services[service_name] = service |
1939 | @@ -128,15 +128,18 @@ | |||
1940 | 128 | """ | 128 | """ |
1941 | 129 | Handle the current hook by doing The Right Thing with the registered services. | 129 | Handle the current hook by doing The Right Thing with the registered services. |
1942 | 130 | """ | 130 | """ |
1952 | 131 | hook_name = hookenv.hook_name() | 131 | hookenv._run_atstart() |
1953 | 132 | if hook_name == 'stop': | 132 | try: |
1954 | 133 | self.stop_services() | 133 | hook_name = hookenv.hook_name() |
1955 | 134 | else: | 134 | if hook_name == 'stop': |
1956 | 135 | self.provide_data() | 135 | self.stop_services() |
1957 | 136 | self.reconfigure_services() | 136 | else: |
1958 | 137 | cfg = hookenv.config() | 137 | self.reconfigure_services() |
1959 | 138 | if cfg.implicit_save: | 138 | self.provide_data() |
1960 | 139 | cfg.save() | 139 | except SystemExit as x: |
1961 | 140 | if x.code is None or x.code == 0: | ||
1962 | 141 | hookenv._run_atexit() | ||
1963 | 142 | hookenv._run_atexit() | ||
1964 | 140 | 143 | ||
1965 | 141 | def provide_data(self): | 144 | def provide_data(self): |
1966 | 142 | """ | 145 | """ |
1967 | @@ -145,15 +148,36 @@ | |||
1968 | 145 | A provider must have a `name` attribute, which indicates which relation | 148 | A provider must have a `name` attribute, which indicates which relation |
1969 | 146 | to set data on, and a `provide_data()` method, which returns a dict of | 149 | to set data on, and a `provide_data()` method, which returns a dict of |
1970 | 147 | data to set. | 150 | data to set. |
1971 | 151 | |||
1972 | 152 | The `provide_data()` method can optionally accept two parameters: | ||
1973 | 153 | |||
1974 | 154 | * ``remote_service`` The name of the remote service that the data will | ||
1975 | 155 | be provided to. The `provide_data()` method will be called once | ||
1976 | 156 | for each connected service (not unit). This allows the method to | ||
1977 | 157 | tailor its data to the given service. | ||
1978 | 158 | * ``service_ready`` Whether or not the service definition had all of | ||
1979 | 159 | its requirements met, and thus the ``data_ready`` callbacks run. | ||
1980 | 160 | |||
1981 | 161 | Note that the ``provided_data`` methods are now called **after** the | ||
1982 | 162 | ``data_ready`` callbacks are run. This gives the ``data_ready`` callbacks | ||
1983 | 163 | a chance to generate any data necessary for the providing to the remote | ||
1984 | 164 | services. | ||
1985 | 148 | """ | 165 | """ |
1988 | 149 | hook_name = hookenv.hook_name() | 166 | for service_name, service in self.services.items(): |
1989 | 150 | for service in self.services.values(): | 167 | service_ready = self.is_ready(service_name) |
1990 | 151 | for provider in service.get('provided_data', []): | 168 | for provider in service.get('provided_data', []): |
1996 | 152 | if re.match(r'{}-relation-(joined|changed)'.format(provider.name), hook_name): | 169 | for relid in hookenv.relation_ids(provider.name): |
1997 | 153 | data = provider.provide_data() | 170 | units = hookenv.related_units(relid) |
1998 | 154 | _ready = provider._is_ready(data) if hasattr(provider, '_is_ready') else data | 171 | if not units: |
1999 | 155 | if _ready: | 172 | continue |
2000 | 156 | hookenv.relation_set(None, data) | 173 | remote_service = units[0].split('/')[0] |
2001 | 174 | argspec = getargspec(provider.provide_data) | ||
2002 | 175 | if len(argspec.args) > 2: | ||
2003 | 176 | data = provider.provide_data(remote_service, service_ready) | ||
2004 | 177 | else: | ||
2005 | 178 | data = provider.provide_data() | ||
2006 | 179 | if data: | ||
2007 | 180 | hookenv.relation_set(relid, data) | ||
2008 | 157 | 181 | ||
2009 | 158 | def reconfigure_services(self, *service_names): | 182 | def reconfigure_services(self, *service_names): |
2010 | 159 | """ | 183 | """ |
2011 | 160 | 184 | ||
2012 | === modified file 'hooks/charmhelpers/core/services/helpers.py' | |||
2013 | --- hooks/charmhelpers/core/services/helpers.py 2015-01-27 14:54:02 +0000 | |||
2014 | +++ hooks/charmhelpers/core/services/helpers.py 2016-04-05 14:45:53 +0000 | |||
2015 | @@ -16,7 +16,9 @@ | |||
2016 | 16 | 16 | ||
2017 | 17 | import os | 17 | import os |
2018 | 18 | import yaml | 18 | import yaml |
2019 | 19 | |||
2020 | 19 | from charmhelpers.core import hookenv | 20 | from charmhelpers.core import hookenv |
2021 | 21 | from charmhelpers.core import host | ||
2022 | 20 | from charmhelpers.core import templating | 22 | from charmhelpers.core import templating |
2023 | 21 | 23 | ||
2024 | 22 | from charmhelpers.core.services.base import ManagerCallback | 24 | from charmhelpers.core.services.base import ManagerCallback |
2025 | @@ -45,12 +47,14 @@ | |||
2026 | 45 | """ | 47 | """ |
2027 | 46 | name = None | 48 | name = None |
2028 | 47 | interface = None | 49 | interface = None |
2029 | 48 | required_keys = [] | ||
2030 | 49 | 50 | ||
2031 | 50 | def __init__(self, name=None, additional_required_keys=None): | 51 | def __init__(self, name=None, additional_required_keys=None): |
2032 | 52 | if not hasattr(self, 'required_keys'): | ||
2033 | 53 | self.required_keys = [] | ||
2034 | 54 | |||
2035 | 51 | if name is not None: | 55 | if name is not None: |
2036 | 52 | self.name = name | 56 | self.name = name |
2038 | 53 | if additional_required_keys is not None: | 57 | if additional_required_keys: |
2039 | 54 | self.required_keys.extend(additional_required_keys) | 58 | self.required_keys.extend(additional_required_keys) |
2040 | 55 | self.get_data() | 59 | self.get_data() |
2041 | 56 | 60 | ||
2042 | @@ -134,7 +138,10 @@ | |||
2043 | 134 | """ | 138 | """ |
2044 | 135 | name = 'db' | 139 | name = 'db' |
2045 | 136 | interface = 'mysql' | 140 | interface = 'mysql' |
2047 | 137 | required_keys = ['host', 'user', 'password', 'database'] | 141 | |
2048 | 142 | def __init__(self, *args, **kwargs): | ||
2049 | 143 | self.required_keys = ['host', 'user', 'password', 'database'] | ||
2050 | 144 | RelationContext.__init__(self, *args, **kwargs) | ||
2051 | 138 | 145 | ||
2052 | 139 | 146 | ||
2053 | 140 | class HttpRelation(RelationContext): | 147 | class HttpRelation(RelationContext): |
2054 | @@ -146,7 +153,10 @@ | |||
2055 | 146 | """ | 153 | """ |
2056 | 147 | name = 'website' | 154 | name = 'website' |
2057 | 148 | interface = 'http' | 155 | interface = 'http' |
2059 | 149 | required_keys = ['host', 'port'] | 156 | |
2060 | 157 | def __init__(self, *args, **kwargs): | ||
2061 | 158 | self.required_keys = ['host', 'port'] | ||
2062 | 159 | RelationContext.__init__(self, *args, **kwargs) | ||
2063 | 150 | 160 | ||
2064 | 151 | def provide_data(self): | 161 | def provide_data(self): |
2065 | 152 | return { | 162 | return { |
2066 | @@ -231,28 +241,51 @@ | |||
2067 | 231 | action. | 241 | action. |
2068 | 232 | 242 | ||
2069 | 233 | :param str source: The template source file, relative to | 243 | :param str source: The template source file, relative to |
2071 | 234 | `$CHARM_DIR/templates` | 244 | `$CHARM_DIR/templates` |
2072 | 235 | 245 | ||
2074 | 236 | :param str target: The target to write the rendered template to | 246 | :param str target: The target to write the rendered template to (or None) |
2075 | 237 | :param str owner: The owner of the rendered file | 247 | :param str owner: The owner of the rendered file |
2076 | 238 | :param str group: The group of the rendered file | 248 | :param str group: The group of the rendered file |
2077 | 239 | :param int perms: The permissions of the rendered file | 249 | :param int perms: The permissions of the rendered file |
2078 | 250 | :param partial on_change_action: functools partial to be executed when | ||
2079 | 251 | rendered file changes | ||
2080 | 252 | :param jinja2 loader template_loader: A jinja2 template loader | ||
2081 | 253 | |||
2082 | 254 | :return str: The rendered template | ||
2083 | 240 | """ | 255 | """ |
2084 | 241 | def __init__(self, source, target, | 256 | def __init__(self, source, target, |
2086 | 242 | owner='root', group='root', perms=0o444): | 257 | owner='root', group='root', perms=0o444, |
2087 | 258 | on_change_action=None, template_loader=None): | ||
2088 | 243 | self.source = source | 259 | self.source = source |
2089 | 244 | self.target = target | 260 | self.target = target |
2090 | 245 | self.owner = owner | 261 | self.owner = owner |
2091 | 246 | self.group = group | 262 | self.group = group |
2092 | 247 | self.perms = perms | 263 | self.perms = perms |
2093 | 264 | self.on_change_action = on_change_action | ||
2094 | 265 | self.template_loader = template_loader | ||
2095 | 248 | 266 | ||
2096 | 249 | def __call__(self, manager, service_name, event_name): | 267 | def __call__(self, manager, service_name, event_name): |
2097 | 268 | pre_checksum = '' | ||
2098 | 269 | if self.on_change_action and os.path.isfile(self.target): | ||
2099 | 270 | pre_checksum = host.file_hash(self.target) | ||
2100 | 250 | service = manager.get_service(service_name) | 271 | service = manager.get_service(service_name) |
2102 | 251 | context = {} | 272 | context = {'ctx': {}} |
2103 | 252 | for ctx in service.get('required_data', []): | 273 | for ctx in service.get('required_data', []): |
2104 | 253 | context.update(ctx) | 274 | context.update(ctx) |
2107 | 254 | templating.render(self.source, self.target, context, | 275 | context['ctx'].update(ctx) |
2108 | 255 | self.owner, self.group, self.perms) | 276 | |
2109 | 277 | result = templating.render(self.source, self.target, context, | ||
2110 | 278 | self.owner, self.group, self.perms, | ||
2111 | 279 | template_loader=self.template_loader) | ||
2112 | 280 | if self.on_change_action: | ||
2113 | 281 | if pre_checksum == host.file_hash(self.target): | ||
2114 | 282 | hookenv.log( | ||
2115 | 283 | 'No change detected: {}'.format(self.target), | ||
2116 | 284 | hookenv.DEBUG) | ||
2117 | 285 | else: | ||
2118 | 286 | self.on_change_action() | ||
2119 | 287 | |||
2120 | 288 | return result | ||
2121 | 256 | 289 | ||
2122 | 257 | 290 | ||
2123 | 258 | # Convenience aliases for templates | 291 | # Convenience aliases for templates |
2124 | 259 | 292 | ||
2125 | === added file 'hooks/charmhelpers/core/strutils.py' | |||
2126 | --- hooks/charmhelpers/core/strutils.py 1970-01-01 00:00:00 +0000 | |||
2127 | +++ hooks/charmhelpers/core/strutils.py 2016-04-05 14:45:53 +0000 | |||
2128 | @@ -0,0 +1,72 @@ | |||
2129 | 1 | #!/usr/bin/env python | ||
2130 | 2 | # -*- coding: utf-8 -*- | ||
2131 | 3 | |||
2132 | 4 | # Copyright 2014-2015 Canonical Limited. | ||
2133 | 5 | # | ||
2134 | 6 | # This file is part of charm-helpers. | ||
2135 | 7 | # | ||
2136 | 8 | # charm-helpers is free software: you can redistribute it and/or modify | ||
2137 | 9 | # it under the terms of the GNU Lesser General Public License version 3 as | ||
2138 | 10 | # published by the Free Software Foundation. | ||
2139 | 11 | # | ||
2140 | 12 | # charm-helpers is distributed in the hope that it will be useful, | ||
2141 | 13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | ||
2142 | 14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | ||
2143 | 15 | # GNU Lesser General Public License for more details. | ||
2144 | 16 | # | ||
2145 | 17 | # You should have received a copy of the GNU Lesser General Public License | ||
2146 | 18 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
2147 | 19 | |||
2148 | 20 | import six | ||
2149 | 21 | import re | ||
2150 | 22 | |||
2151 | 23 | |||
2152 | 24 | def bool_from_string(value): | ||
2153 | 25 | """Interpret string value as boolean. | ||
2154 | 26 | |||
2155 | 27 | Returns True if value translates to True otherwise False. | ||
2156 | 28 | """ | ||
2157 | 29 | if isinstance(value, six.string_types): | ||
2158 | 30 | value = six.text_type(value) | ||
2159 | 31 | else: | ||
2160 | 32 | msg = "Unable to interpret non-string value '%s' as boolean" % (value) | ||
2161 | 33 | raise ValueError(msg) | ||
2162 | 34 | |||
2163 | 35 | value = value.strip().lower() | ||
2164 | 36 | |||
2165 | 37 | if value in ['y', 'yes', 'true', 't', 'on']: | ||
2166 | 38 | return True | ||
2167 | 39 | elif value in ['n', 'no', 'false', 'f', 'off']: | ||
2168 | 40 | return False | ||
2169 | 41 | |||
2170 | 42 | msg = "Unable to interpret string value '%s' as boolean" % (value) | ||
2171 | 43 | raise ValueError(msg) | ||
2172 | 44 | |||
2173 | 45 | |||
2174 | 46 | def bytes_from_string(value): | ||
2175 | 47 | """Interpret human readable string value as bytes. | ||
2176 | 48 | |||
2177 | 49 | Returns int | ||
2178 | 50 | """ | ||
2179 | 51 | BYTE_POWER = { | ||
2180 | 52 | 'K': 1, | ||
2181 | 53 | 'KB': 1, | ||
2182 | 54 | 'M': 2, | ||
2183 | 55 | 'MB': 2, | ||
2184 | 56 | 'G': 3, | ||
2185 | 57 | 'GB': 3, | ||
2186 | 58 | 'T': 4, | ||
2187 | 59 | 'TB': 4, | ||
2188 | 60 | 'P': 5, | ||
2189 | 61 | 'PB': 5, | ||
2190 | 62 | } | ||
2191 | 63 | if isinstance(value, six.string_types): | ||
2192 | 64 | value = six.text_type(value) | ||
2193 | 65 | else: | ||
2194 | 66 | msg = "Unable to interpret non-string value '%s' as boolean" % (value) | ||
2195 | 67 | raise ValueError(msg) | ||
2196 | 68 | matches = re.match("([0-9]+)([a-zA-Z]+)", value) | ||
2197 | 69 | if not matches: | ||
2198 | 70 | msg = "Unable to interpret string value '%s' as bytes" % (value) | ||
2199 | 71 | raise ValueError(msg) | ||
2200 | 72 | return int(matches.group(1)) * (1024 ** BYTE_POWER[matches.group(2)]) | ||
2201 | 0 | 73 | ||
2202 | === modified file 'hooks/charmhelpers/core/sysctl.py' | |||
2203 | --- hooks/charmhelpers/core/sysctl.py 2015-01-27 14:54:02 +0000 | |||
2204 | +++ hooks/charmhelpers/core/sysctl.py 2016-04-05 14:45:53 +0000 | |||
2205 | @@ -17,8 +17,6 @@ | |||
2206 | 17 | # You should have received a copy of the GNU Lesser General Public License | 17 | # You should have received a copy of the GNU Lesser General Public License |
2207 | 18 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | 18 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
2208 | 19 | 19 | ||
2209 | 20 | __author__ = 'Jorge Niedbalski R. <jorge.niedbalski@canonical.com>' | ||
2210 | 21 | |||
2211 | 22 | import yaml | 20 | import yaml |
2212 | 23 | 21 | ||
2213 | 24 | from subprocess import check_call | 22 | from subprocess import check_call |
2214 | @@ -26,25 +24,33 @@ | |||
2215 | 26 | from charmhelpers.core.hookenv import ( | 24 | from charmhelpers.core.hookenv import ( |
2216 | 27 | log, | 25 | log, |
2217 | 28 | DEBUG, | 26 | DEBUG, |
2218 | 27 | ERROR, | ||
2219 | 29 | ) | 28 | ) |
2220 | 30 | 29 | ||
2221 | 30 | __author__ = 'Jorge Niedbalski R. <jorge.niedbalski@canonical.com>' | ||
2222 | 31 | |||
2223 | 31 | 32 | ||
2224 | 32 | def create(sysctl_dict, sysctl_file): | 33 | def create(sysctl_dict, sysctl_file): |
2225 | 33 | """Creates a sysctl.conf file from a YAML associative array | 34 | """Creates a sysctl.conf file from a YAML associative array |
2226 | 34 | 35 | ||
2229 | 35 | :param sysctl_dict: a dict of sysctl options eg { 'kernel.max_pid': 1337 } | 36 | :param sysctl_dict: a YAML-formatted string of sysctl options eg "{ 'kernel.max_pid': 1337 }" |
2230 | 36 | :type sysctl_dict: dict | 37 | :type sysctl_dict: str |
2231 | 37 | :param sysctl_file: path to the sysctl file to be saved | 38 | :param sysctl_file: path to the sysctl file to be saved |
2232 | 38 | :type sysctl_file: str or unicode | 39 | :type sysctl_file: str or unicode |
2233 | 39 | :returns: None | 40 | :returns: None |
2234 | 40 | """ | 41 | """ |
2236 | 41 | sysctl_dict = yaml.load(sysctl_dict) | 42 | try: |
2237 | 43 | sysctl_dict_parsed = yaml.safe_load(sysctl_dict) | ||
2238 | 44 | except yaml.YAMLError: | ||
2239 | 45 | log("Error parsing YAML sysctl_dict: {}".format(sysctl_dict), | ||
2240 | 46 | level=ERROR) | ||
2241 | 47 | return | ||
2242 | 42 | 48 | ||
2243 | 43 | with open(sysctl_file, "w") as fd: | 49 | with open(sysctl_file, "w") as fd: |
2245 | 44 | for key, value in sysctl_dict.items(): | 50 | for key, value in sysctl_dict_parsed.items(): |
2246 | 45 | fd.write("{}={}\n".format(key, value)) | 51 | fd.write("{}={}\n".format(key, value)) |
2247 | 46 | 52 | ||
2249 | 47 | log("Updating sysctl_file: %s values: %s" % (sysctl_file, sysctl_dict), | 53 | log("Updating sysctl_file: %s values: %s" % (sysctl_file, sysctl_dict_parsed), |
2250 | 48 | level=DEBUG) | 54 | level=DEBUG) |
2251 | 49 | 55 | ||
2252 | 50 | check_call(["sysctl", "-p", sysctl_file]) | 56 | check_call(["sysctl", "-p", sysctl_file]) |
2253 | 51 | 57 | ||
2254 | === modified file 'hooks/charmhelpers/core/templating.py' | |||
2255 | --- hooks/charmhelpers/core/templating.py 2015-01-27 14:54:02 +0000 | |||
2256 | +++ hooks/charmhelpers/core/templating.py 2016-04-05 14:45:53 +0000 | |||
2257 | @@ -21,13 +21,14 @@ | |||
2258 | 21 | 21 | ||
2259 | 22 | 22 | ||
2260 | 23 | def render(source, target, context, owner='root', group='root', | 23 | def render(source, target, context, owner='root', group='root', |
2262 | 24 | perms=0o444, templates_dir=None): | 24 | perms=0o444, templates_dir=None, encoding='UTF-8', template_loader=None): |
2263 | 25 | """ | 25 | """ |
2264 | 26 | Render a template. | 26 | Render a template. |
2265 | 27 | 27 | ||
2266 | 28 | The `source` path, if not absolute, is relative to the `templates_dir`. | 28 | The `source` path, if not absolute, is relative to the `templates_dir`. |
2267 | 29 | 29 | ||
2269 | 30 | The `target` path should be absolute. | 30 | The `target` path should be absolute. It can also be `None`, in which |
2270 | 31 | case no file will be written. | ||
2271 | 31 | 32 | ||
2272 | 32 | The context should be a dict containing the values to be replaced in the | 33 | The context should be a dict containing the values to be replaced in the |
2273 | 33 | template. | 34 | template. |
2274 | @@ -36,6 +37,9 @@ | |||
2275 | 36 | 37 | ||
2276 | 37 | If omitted, `templates_dir` defaults to the `templates` folder in the charm. | 38 | If omitted, `templates_dir` defaults to the `templates` folder in the charm. |
2277 | 38 | 39 | ||
2278 | 40 | The rendered template will be written to the file as well as being returned | ||
2279 | 41 | as a string. | ||
2280 | 42 | |||
2281 | 39 | Note: Using this requires python-jinja2; if it is not installed, calling | 43 | Note: Using this requires python-jinja2; if it is not installed, calling |
2282 | 40 | this will attempt to use charmhelpers.fetch.apt_install to install it. | 44 | this will attempt to use charmhelpers.fetch.apt_install to install it. |
2283 | 41 | """ | 45 | """ |
2284 | @@ -52,17 +56,26 @@ | |||
2285 | 52 | apt_install('python-jinja2', fatal=True) | 56 | apt_install('python-jinja2', fatal=True) |
2286 | 53 | from jinja2 import FileSystemLoader, Environment, exceptions | 57 | from jinja2 import FileSystemLoader, Environment, exceptions |
2287 | 54 | 58 | ||
2291 | 55 | if templates_dir is None: | 59 | if template_loader: |
2292 | 56 | templates_dir = os.path.join(hookenv.charm_dir(), 'templates') | 60 | template_env = Environment(loader=template_loader) |
2293 | 57 | loader = Environment(loader=FileSystemLoader(templates_dir)) | 61 | else: |
2294 | 62 | if templates_dir is None: | ||
2295 | 63 | templates_dir = os.path.join(hookenv.charm_dir(), 'templates') | ||
2296 | 64 | template_env = Environment(loader=FileSystemLoader(templates_dir)) | ||
2297 | 58 | try: | 65 | try: |
2298 | 59 | source = source | 66 | source = source |
2300 | 60 | template = loader.get_template(source) | 67 | template = template_env.get_template(source) |
2301 | 61 | except exceptions.TemplateNotFound as e: | 68 | except exceptions.TemplateNotFound as e: |
2302 | 62 | hookenv.log('Could not load template %s from %s.' % | 69 | hookenv.log('Could not load template %s from %s.' % |
2303 | 63 | (source, templates_dir), | 70 | (source, templates_dir), |
2304 | 64 | level=hookenv.ERROR) | 71 | level=hookenv.ERROR) |
2305 | 65 | raise e | 72 | raise e |
2306 | 66 | content = template.render(context) | 73 | content = template.render(context) |
2309 | 67 | host.mkdir(os.path.dirname(target), owner, group) | 74 | if target is not None: |
2310 | 68 | host.write_file(target, content, owner, group, perms) | 75 | target_dir = os.path.dirname(target) |
2311 | 76 | if not os.path.exists(target_dir): | ||
2312 | 77 | # This is a terrible default directory permission, as the file | ||
2313 | 78 | # or its siblings will often contain secrets. | ||
2314 | 79 | host.mkdir(os.path.dirname(target), owner, group, perms=0o755) | ||
2315 | 80 | host.write_file(target, content.encode(encoding), owner, group, perms) | ||
2316 | 81 | return content | ||
2317 | 69 | 82 | ||
2318 | === added file 'hooks/charmhelpers/core/unitdata.py' | |||
2319 | --- hooks/charmhelpers/core/unitdata.py 1970-01-01 00:00:00 +0000 | |||
2320 | +++ hooks/charmhelpers/core/unitdata.py 2016-04-05 14:45:53 +0000 | |||
2321 | @@ -0,0 +1,521 @@ | |||
2322 | 1 | #!/usr/bin/env python | ||
2323 | 2 | # -*- coding: utf-8 -*- | ||
2324 | 3 | # | ||
2325 | 4 | # Copyright 2014-2015 Canonical Limited. | ||
2326 | 5 | # | ||
2327 | 6 | # This file is part of charm-helpers. | ||
2328 | 7 | # | ||
2329 | 8 | # charm-helpers is free software: you can redistribute it and/or modify | ||
2330 | 9 | # it under the terms of the GNU Lesser General Public License version 3 as | ||
2331 | 10 | # published by the Free Software Foundation. | ||
2332 | 11 | # | ||
2333 | 12 | # charm-helpers is distributed in the hope that it will be useful, | ||
2334 | 13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | ||
2335 | 14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | ||
2336 | 15 | # GNU Lesser General Public License for more details. | ||
2337 | 16 | # | ||
2338 | 17 | # You should have received a copy of the GNU Lesser General Public License | ||
2339 | 18 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | ||
2340 | 19 | # | ||
2341 | 20 | # | ||
2342 | 21 | # Authors: | ||
2343 | 22 | # Kapil Thangavelu <kapil.foss@gmail.com> | ||
2344 | 23 | # | ||
2345 | 24 | """ | ||
2346 | 25 | Intro | ||
2347 | 26 | ----- | ||
2348 | 27 | |||
2349 | 28 | A simple way to store state in units. This provides a key value | ||
2350 | 29 | storage with support for versioned, transactional operation, | ||
2351 | 30 | and can calculate deltas from previous values to simplify unit logic | ||
2352 | 31 | when processing changes. | ||
2353 | 32 | |||
2354 | 33 | |||
2355 | 34 | Hook Integration | ||
2356 | 35 | ---------------- | ||
2357 | 36 | |||
2358 | 37 | There are several extant frameworks for hook execution, including | ||
2359 | 38 | |||
2360 | 39 | - charmhelpers.core.hookenv.Hooks | ||
2361 | 40 | - charmhelpers.core.services.ServiceManager | ||
2362 | 41 | |||
2363 | 42 | The storage classes are framework agnostic, one simple integration is | ||
2364 | 43 | via the HookData contextmanager. It will record the current hook | ||
2365 | 44 | execution environment (including relation data, config data, etc.), | ||
2366 | 45 | setup a transaction and allow easy access to the changes from | ||
2367 | 46 | previously seen values. One consequence of the integration is the | ||
2368 | 47 | reservation of particular keys ('rels', 'unit', 'env', 'config', | ||
2369 | 48 | 'charm_revisions') for their respective values. | ||
2370 | 49 | |||
2371 | 50 | Here's a fully worked integration example using hookenv.Hooks:: | ||
2372 | 51 | |||
2373 | 52 | from charmhelper.core import hookenv, unitdata | ||
2374 | 53 | |||
2375 | 54 | hook_data = unitdata.HookData() | ||
2376 | 55 | db = unitdata.kv() | ||
2377 | 56 | hooks = hookenv.Hooks() | ||
2378 | 57 | |||
2379 | 58 | @hooks.hook | ||
2380 | 59 | def config_changed(): | ||
2381 | 60 | # Print all changes to configuration from previously seen | ||
2382 | 61 | # values. | ||
2383 | 62 | for changed, (prev, cur) in hook_data.conf.items(): | ||
2384 | 63 | print('config changed', changed, | ||
2385 | 64 | 'previous value', prev, | ||
2386 | 65 | 'current value', cur) | ||
2387 | 66 | |||
2388 | 67 | # Get some unit specific bookeeping | ||
2389 | 68 | if not db.get('pkg_key'): | ||
2390 | 69 | key = urllib.urlopen('https://example.com/pkg_key').read() | ||
2391 | 70 | db.set('pkg_key', key) | ||
2392 | 71 | |||
2393 | 72 | # Directly access all charm config as a mapping. | ||
2394 | 73 | conf = db.getrange('config', True) | ||
2395 | 74 | |||
2396 | 75 | # Directly access all relation data as a mapping | ||
2397 | 76 | rels = db.getrange('rels', True) | ||
2398 | 77 | |||
2399 | 78 | if __name__ == '__main__': | ||
2400 | 79 | with hook_data(): | ||
2401 | 80 | hook.execute() | ||
2402 | 81 | |||
2403 | 82 | |||
2404 | 83 | A more basic integration is via the hook_scope context manager which simply | ||
2405 | 84 | manages transaction scope (and records hook name, and timestamp):: | ||
2406 | 85 | |||
2407 | 86 | >>> from unitdata import kv | ||
2408 | 87 | >>> db = kv() | ||
2409 | 88 | >>> with db.hook_scope('install'): | ||
2410 | 89 | ... # do work, in transactional scope. | ||
2411 | 90 | ... db.set('x', 1) | ||
2412 | 91 | >>> db.get('x') | ||
2413 | 92 | 1 | ||
2414 | 93 | |||
2415 | 94 | |||
2416 | 95 | Usage | ||
2417 | 96 | ----- | ||
2418 | 97 | |||
2419 | 98 | Values are automatically json de/serialized to preserve basic typing | ||
2420 | 99 | and complex data struct capabilities (dicts, lists, ints, booleans, etc). | ||
2421 | 100 | |||
2422 | 101 | Individual values can be manipulated via get/set:: | ||
2423 | 102 | |||
2424 | 103 | >>> kv.set('y', True) | ||
2425 | 104 | >>> kv.get('y') | ||
2426 | 105 | True | ||
2427 | 106 | |||
2428 | 107 | # We can set complex values (dicts, lists) as a single key. | ||
2429 | 108 | >>> kv.set('config', {'a': 1, 'b': True'}) | ||
2430 | 109 | |||
2431 | 110 | # Also supports returning dictionaries as a record which | ||
2432 | 111 | # provides attribute access. | ||
2433 | 112 | >>> config = kv.get('config', record=True) | ||
2434 | 113 | >>> config.b | ||
2435 | 114 | True | ||
2436 | 115 | |||
2437 | 116 | |||
2438 | 117 | Groups of keys can be manipulated with update/getrange:: | ||
2439 | 118 | |||
2440 | 119 | >>> kv.update({'z': 1, 'y': 2}, prefix="gui.") | ||
2441 | 120 | >>> kv.getrange('gui.', strip=True) | ||
2442 | 121 | {'z': 1, 'y': 2} | ||
2443 | 122 | |||
2444 | 123 | When updating values, its very helpful to understand which values | ||
2445 | 124 | have actually changed and how have they changed. The storage | ||
2446 | 125 | provides a delta method to provide for this:: | ||
2447 | 126 | |||
2448 | 127 | >>> data = {'debug': True, 'option': 2} | ||
2449 | 128 | >>> delta = kv.delta(data, 'config.') | ||
2450 | 129 | >>> delta.debug.previous | ||
2451 | 130 | None | ||
2452 | 131 | >>> delta.debug.current | ||
2453 | 132 | True | ||
2454 | 133 | >>> delta | ||
2455 | 134 | {'debug': (None, True), 'option': (None, 2)} | ||
2456 | 135 | |||
2457 | 136 | Note the delta method does not persist the actual change, it needs to | ||
2458 | 137 | be explicitly saved via 'update' method:: | ||
2459 | 138 | |||
2460 | 139 | >>> kv.update(data, 'config.') | ||
2461 | 140 | |||
2462 | 141 | Values modified in the context of a hook scope retain historical values | ||
2463 | 142 | associated to the hookname. | ||
2464 | 143 | |||
2465 | 144 | >>> with db.hook_scope('config-changed'): | ||
2466 | 145 | ... db.set('x', 42) | ||
2467 | 146 | >>> db.gethistory('x') | ||
2468 | 147 | [(1, u'x', 1, u'install', u'2015-01-21T16:49:30.038372'), | ||
2469 | 148 | (2, u'x', 42, u'config-changed', u'2015-01-21T16:49:30.038786')] | ||
2470 | 149 | |||
2471 | 150 | """ | ||
2472 | 151 | |||
2473 | 152 | import collections | ||
2474 | 153 | import contextlib | ||
2475 | 154 | import datetime | ||
2476 | 155 | import itertools | ||
2477 | 156 | import json | ||
2478 | 157 | import os | ||
2479 | 158 | import pprint | ||
2480 | 159 | import sqlite3 | ||
2481 | 160 | import sys | ||
2482 | 161 | |||
2483 | 162 | __author__ = 'Kapil Thangavelu <kapil.foss@gmail.com>' | ||
2484 | 163 | |||
2485 | 164 | |||
2486 | 165 | class Storage(object): | ||
2487 | 166 | """Simple key value database for local unit state within charms. | ||
2488 | 167 | |||
2489 | 168 | Modifications are not persisted unless :meth:`flush` is called. | ||
2490 | 169 | |||
2491 | 170 | To support dicts, lists, integer, floats, and booleans values | ||
2492 | 171 | are automatically json encoded/decoded. | ||
2493 | 172 | """ | ||
2494 | 173 | def __init__(self, path=None): | ||
2495 | 174 | self.db_path = path | ||
2496 | 175 | if path is None: | ||
2497 | 176 | if 'UNIT_STATE_DB' in os.environ: | ||
2498 | 177 | self.db_path = os.environ['UNIT_STATE_DB'] | ||
2499 | 178 | else: | ||
2500 | 179 | self.db_path = os.path.join( | ||
2501 | 180 | os.environ.get('CHARM_DIR', ''), '.unit-state.db') | ||
2502 | 181 | self.conn = sqlite3.connect('%s' % self.db_path) | ||
2503 | 182 | self.cursor = self.conn.cursor() | ||
2504 | 183 | self.revision = None | ||
2505 | 184 | self._closed = False | ||
2506 | 185 | self._init() | ||
2507 | 186 | |||
2508 | 187 | def close(self): | ||
2509 | 188 | if self._closed: | ||
2510 | 189 | return | ||
2511 | 190 | self.flush(False) | ||
2512 | 191 | self.cursor.close() | ||
2513 | 192 | self.conn.close() | ||
2514 | 193 | self._closed = True | ||
2515 | 194 | |||
2516 | 195 | def get(self, key, default=None, record=False): | ||
2517 | 196 | self.cursor.execute('select data from kv where key=?', [key]) | ||
2518 | 197 | result = self.cursor.fetchone() | ||
2519 | 198 | if not result: | ||
2520 | 199 | return default | ||
2521 | 200 | if record: | ||
2522 | 201 | return Record(json.loads(result[0])) | ||
2523 | 202 | return json.loads(result[0]) | ||
2524 | 203 | |||
2525 | 204 | def getrange(self, key_prefix, strip=False): | ||
2526 | 205 | """ | ||
2527 | 206 | Get a range of keys starting with a common prefix as a mapping of | ||
2528 | 207 | keys to values. | ||
2529 | 208 | |||
2530 | 209 | :param str key_prefix: Common prefix among all keys | ||
2531 | 210 | :param bool strip: Optionally strip the common prefix from the key | ||
2532 | 211 | names in the returned dict | ||
2533 | 212 | :return dict: A (possibly empty) dict of key-value mappings | ||
2534 | 213 | """ | ||
2535 | 214 | self.cursor.execute("select key, data from kv where key like ?", | ||
2536 | 215 | ['%s%%' % key_prefix]) | ||
2537 | 216 | result = self.cursor.fetchall() | ||
2538 | 217 | |||
2539 | 218 | if not result: | ||
2540 | 219 | return {} | ||
2541 | 220 | if not strip: | ||
2542 | 221 | key_prefix = '' | ||
2543 | 222 | return dict([ | ||
2544 | 223 | (k[len(key_prefix):], json.loads(v)) for k, v in result]) | ||
2545 | 224 | |||
2546 | 225 | def update(self, mapping, prefix=""): | ||
2547 | 226 | """ | ||
2548 | 227 | Set the values of multiple keys at once. | ||
2549 | 228 | |||
2550 | 229 | :param dict mapping: Mapping of keys to values | ||
2551 | 230 | :param str prefix: Optional prefix to apply to all keys in `mapping` | ||
2552 | 231 | before setting | ||
2553 | 232 | """ | ||
2554 | 233 | for k, v in mapping.items(): | ||
2555 | 234 | self.set("%s%s" % (prefix, k), v) | ||
2556 | 235 | |||
2557 | 236 | def unset(self, key): | ||
2558 | 237 | """ | ||
2559 | 238 | Remove a key from the database entirely. | ||
2560 | 239 | """ | ||
2561 | 240 | self.cursor.execute('delete from kv where key=?', [key]) | ||
2562 | 241 | if self.revision and self.cursor.rowcount: | ||
2563 | 242 | self.cursor.execute( | ||
2564 | 243 | 'insert into kv_revisions values (?, ?, ?)', | ||
2565 | 244 | [key, self.revision, json.dumps('DELETED')]) | ||
2566 | 245 | |||
2567 | 246 | def unsetrange(self, keys=None, prefix=""): | ||
2568 | 247 | """ | ||
2569 | 248 | Remove a range of keys starting with a common prefix, from the database | ||
2570 | 249 | entirely. | ||
2571 | 250 | |||
2572 | 251 | :param list keys: List of keys to remove. | ||
2573 | 252 | :param str prefix: Optional prefix to apply to all keys in ``keys`` | ||
2574 | 253 | before removing. | ||
2575 | 254 | """ | ||
2576 | 255 | if keys is not None: | ||
2577 | 256 | keys = ['%s%s' % (prefix, key) for key in keys] | ||
2578 | 257 | self.cursor.execute('delete from kv where key in (%s)' % ','.join(['?'] * len(keys)), keys) | ||
2579 | 258 | if self.revision and self.cursor.rowcount: | ||
2580 | 259 | self.cursor.execute( | ||
2581 | 260 | 'insert into kv_revisions values %s' % ','.join(['(?, ?, ?)'] * len(keys)), | ||
2582 | 261 | list(itertools.chain.from_iterable((key, self.revision, json.dumps('DELETED')) for key in keys))) | ||
2583 | 262 | else: | ||
2584 | 263 | self.cursor.execute('delete from kv where key like ?', | ||
2585 | 264 | ['%s%%' % prefix]) | ||
2586 | 265 | if self.revision and self.cursor.rowcount: | ||
2587 | 266 | self.cursor.execute( | ||
2588 | 267 | 'insert into kv_revisions values (?, ?, ?)', | ||
2589 | 268 | ['%s%%' % prefix, self.revision, json.dumps('DELETED')]) | ||
2590 | 269 | |||
2591 | 270 | def set(self, key, value): | ||
2592 | 271 | """ | ||
2593 | 272 | Set a value in the database. | ||
2594 | 273 | |||
2595 | 274 | :param str key: Key to set the value for | ||
2596 | 275 | :param value: Any JSON-serializable value to be set | ||
2597 | 276 | """ | ||
2598 | 277 | serialized = json.dumps(value) | ||
2599 | 278 | |||
2600 | 279 | self.cursor.execute('select data from kv where key=?', [key]) | ||
2601 | 280 | exists = self.cursor.fetchone() | ||
2602 | 281 | |||
2603 | 282 | # Skip mutations to the same value | ||
2604 | 283 | if exists: | ||
2605 | 284 | if exists[0] == serialized: | ||
2606 | 285 | return value | ||
2607 | 286 | |||
2608 | 287 | if not exists: | ||
2609 | 288 | self.cursor.execute( | ||
2610 | 289 | 'insert into kv (key, data) values (?, ?)', | ||
2611 | 290 | (key, serialized)) | ||
2612 | 291 | else: | ||
2613 | 292 | self.cursor.execute(''' | ||
2614 | 293 | update kv | ||
2615 | 294 | set data = ? | ||
2616 | 295 | where key = ?''', [serialized, key]) | ||
2617 | 296 | |||
2618 | 297 | # Save | ||
2619 | 298 | if not self.revision: | ||
2620 | 299 | return value | ||
2621 | 300 | |||
2622 | 301 | self.cursor.execute( | ||
2623 | 302 | 'select 1 from kv_revisions where key=? and revision=?', | ||
2624 | 303 | [key, self.revision]) | ||
2625 | 304 | exists = self.cursor.fetchone() | ||
2626 | 305 | |||
2627 | 306 | if not exists: | ||
2628 | 307 | self.cursor.execute( | ||
2629 | 308 | '''insert into kv_revisions ( | ||
2630 | 309 | revision, key, data) values (?, ?, ?)''', | ||
2631 | 310 | (self.revision, key, serialized)) | ||
2632 | 311 | else: | ||
2633 | 312 | self.cursor.execute( | ||
2634 | 313 | ''' | ||
2635 | 314 | update kv_revisions | ||
2636 | 315 | set data = ? | ||
2637 | 316 | where key = ? | ||
2638 | 317 | and revision = ?''', | ||
2639 | 318 | [serialized, key, self.revision]) | ||
2640 | 319 | |||
2641 | 320 | return value | ||
2642 | 321 | |||
2643 | 322 | def delta(self, mapping, prefix): | ||
2644 | 323 | """ | ||
2645 | 324 | return a delta containing values that have changed. | ||
2646 | 325 | """ | ||
2647 | 326 | previous = self.getrange(prefix, strip=True) | ||
2648 | 327 | if not previous: | ||
2649 | 328 | pk = set() | ||
2650 | 329 | else: | ||
2651 | 330 | pk = set(previous.keys()) | ||
2652 | 331 | ck = set(mapping.keys()) | ||
2653 | 332 | delta = DeltaSet() | ||
2654 | 333 | |||
2655 | 334 | # added | ||
2656 | 335 | for k in ck.difference(pk): | ||
2657 | 336 | delta[k] = Delta(None, mapping[k]) | ||
2658 | 337 | |||
2659 | 338 | # removed | ||
2660 | 339 | for k in pk.difference(ck): | ||
2661 | 340 | delta[k] = Delta(previous[k], None) | ||
2662 | 341 | |||
2663 | 342 | # changed | ||
2664 | 343 | for k in pk.intersection(ck): | ||
2665 | 344 | c = mapping[k] | ||
2666 | 345 | p = previous[k] | ||
2667 | 346 | if c != p: | ||
2668 | 347 | delta[k] = Delta(p, c) | ||
2669 | 348 | |||
2670 | 349 | return delta | ||
2671 | 350 | |||
2672 | 351 | @contextlib.contextmanager | ||
2673 | 352 | def hook_scope(self, name=""): | ||
2674 | 353 | """Scope all future interactions to the current hook execution | ||
2675 | 354 | revision.""" | ||
2676 | 355 | assert not self.revision | ||
2677 | 356 | self.cursor.execute( | ||
2678 | 357 | 'insert into hooks (hook, date) values (?, ?)', | ||
2679 | 358 | (name or sys.argv[0], | ||
2680 | 359 | datetime.datetime.utcnow().isoformat())) | ||
2681 | 360 | self.revision = self.cursor.lastrowid | ||
2682 | 361 | try: | ||
2683 | 362 | yield self.revision | ||
2684 | 363 | self.revision = None | ||
2685 | 364 | except: | ||
2686 | 365 | self.flush(False) | ||
2687 | 366 | self.revision = None | ||
2688 | 367 | raise | ||
2689 | 368 | else: | ||
2690 | 369 | self.flush() | ||
2691 | 370 | |||
2692 | 371 | def flush(self, save=True): | ||
2693 | 372 | if save: | ||
2694 | 373 | self.conn.commit() | ||
2695 | 374 | elif self._closed: | ||
2696 | 375 | return | ||
2697 | 376 | else: | ||
2698 | 377 | self.conn.rollback() | ||
2699 | 378 | |||
2700 | 379 | def _init(self): | ||
2701 | 380 | self.cursor.execute(''' | ||
2702 | 381 | create table if not exists kv ( | ||
2703 | 382 | key text, | ||
2704 | 383 | data text, | ||
2705 | 384 | primary key (key) | ||
2706 | 385 | )''') | ||
2707 | 386 | self.cursor.execute(''' | ||
2708 | 387 | create table if not exists kv_revisions ( | ||
2709 | 388 | key text, | ||
2710 | 389 | revision integer, | ||
2711 | 390 | data text, | ||
2712 | 391 | primary key (key, revision) | ||
2713 | 392 | )''') | ||
2714 | 393 | self.cursor.execute(''' | ||
2715 | 394 | create table if not exists hooks ( | ||
2716 | 395 | version integer primary key autoincrement, | ||
2717 | 396 | hook text, | ||
2718 | 397 | date text | ||
2719 | 398 | )''') | ||
2720 | 399 | self.conn.commit() | ||
2721 | 400 | |||
2722 | 401 | def gethistory(self, key, deserialize=False): | ||
2723 | 402 | self.cursor.execute( | ||
2724 | 403 | ''' | ||
2725 | 404 | select kv.revision, kv.key, kv.data, h.hook, h.date | ||
2726 | 405 | from kv_revisions kv, | ||
2727 | 406 | hooks h | ||
2728 | 407 | where kv.key=? | ||
2729 | 408 | and kv.revision = h.version | ||
2730 | 409 | ''', [key]) | ||
2731 | 410 | if deserialize is False: | ||
2732 | 411 | return self.cursor.fetchall() | ||
2733 | 412 | return map(_parse_history, self.cursor.fetchall()) | ||
2734 | 413 | |||
2735 | 414 | def debug(self, fh=sys.stderr): | ||
2736 | 415 | self.cursor.execute('select * from kv') | ||
2737 | 416 | pprint.pprint(self.cursor.fetchall(), stream=fh) | ||
2738 | 417 | self.cursor.execute('select * from kv_revisions') | ||
2739 | 418 | pprint.pprint(self.cursor.fetchall(), stream=fh) | ||
2740 | 419 | |||
2741 | 420 | |||
2742 | 421 | def _parse_history(d): | ||
2743 | 422 | return (d[0], d[1], json.loads(d[2]), d[3], | ||
2744 | 423 | datetime.datetime.strptime(d[-1], "%Y-%m-%dT%H:%M:%S.%f")) | ||
2745 | 424 | |||
2746 | 425 | |||
2747 | 426 | class HookData(object): | ||
2748 | 427 | """Simple integration for existing hook exec frameworks. | ||
2749 | 428 | |||
2750 | 429 | Records all unit information, and stores deltas for processing | ||
2751 | 430 | by the hook. | ||
2752 | 431 | |||
2753 | 432 | Sample:: | ||
2754 | 433 | |||
2755 | 434 | from charmhelper.core import hookenv, unitdata | ||
2756 | 435 | |||
2757 | 436 | changes = unitdata.HookData() | ||
2758 | 437 | db = unitdata.kv() | ||
2759 | 438 | hooks = hookenv.Hooks() | ||
2760 | 439 | |||
2761 | 440 | @hooks.hook | ||
2762 | 441 | def config_changed(): | ||
2763 | 442 | # View all changes to configuration | ||
2764 | 443 | for changed, (prev, cur) in changes.conf.items(): | ||
2765 | 444 | print('config changed', changed, | ||
2766 | 445 | 'previous value', prev, | ||
2767 | 446 | 'current value', cur) | ||
2768 | 447 | |||
2769 | 448 | # Get some unit specific bookeeping | ||
2770 | 449 | if not db.get('pkg_key'): | ||
2771 | 450 | key = urllib.urlopen('https://example.com/pkg_key').read() | ||
2772 | 451 | db.set('pkg_key', key) | ||
2773 | 452 | |||
2774 | 453 | if __name__ == '__main__': | ||
2775 | 454 | with changes(): | ||
2776 | 455 | hook.execute() | ||
2777 | 456 | |||
2778 | 457 | """ | ||
2779 | 458 | def __init__(self): | ||
2780 | 459 | self.kv = kv() | ||
2781 | 460 | self.conf = None | ||
2782 | 461 | self.rels = None | ||
2783 | 462 | |||
2784 | 463 | @contextlib.contextmanager | ||
2785 | 464 | def __call__(self): | ||
2786 | 465 | from charmhelpers.core import hookenv | ||
2787 | 466 | hook_name = hookenv.hook_name() | ||
2788 | 467 | |||
2789 | 468 | with self.kv.hook_scope(hook_name): | ||
2790 | 469 | self._record_charm_version(hookenv.charm_dir()) | ||
2791 | 470 | delta_config, delta_relation = self._record_hook(hookenv) | ||
2792 | 471 | yield self.kv, delta_config, delta_relation | ||
2793 | 472 | |||
2794 | 473 | def _record_charm_version(self, charm_dir): | ||
2795 | 474 | # Record revisions.. charm revisions are meaningless | ||
2796 | 475 | # to charm authors as they don't control the revision. | ||
2797 | 476 | # so logic dependnent on revision is not particularly | ||
2798 | 477 | # useful, however it is useful for debugging analysis. | ||
2799 | 478 | charm_rev = open( | ||
2800 | 479 | os.path.join(charm_dir, 'revision')).read().strip() | ||
2801 | 480 | charm_rev = charm_rev or '0' | ||
2802 | 481 | revs = self.kv.get('charm_revisions', []) | ||
2803 | 482 | if charm_rev not in revs: | ||
2804 | 483 | revs.append(charm_rev.strip() or '0') | ||
2805 | 484 | self.kv.set('charm_revisions', revs) | ||
2806 | 485 | |||
2807 | 486 | def _record_hook(self, hookenv): | ||
2808 | 487 | data = hookenv.execution_environment() | ||
2809 | 488 | self.conf = conf_delta = self.kv.delta(data['conf'], 'config') | ||
2810 | 489 | self.rels = rels_delta = self.kv.delta(data['rels'], 'rels') | ||
2811 | 490 | self.kv.set('env', dict(data['env'])) | ||
2812 | 491 | self.kv.set('unit', data['unit']) | ||
2813 | 492 | self.kv.set('relid', data.get('relid')) | ||
2814 | 493 | return conf_delta, rels_delta | ||
2815 | 494 | |||
2816 | 495 | |||
2817 | 496 | class Record(dict): | ||
2818 | 497 | |||
2819 | 498 | __slots__ = () | ||
2820 | 499 | |||
2821 | 500 | def __getattr__(self, k): | ||
2822 | 501 | if k in self: | ||
2823 | 502 | return self[k] | ||
2824 | 503 | raise AttributeError(k) | ||
2825 | 504 | |||
2826 | 505 | |||
2827 | 506 | class DeltaSet(Record): | ||
2828 | 507 | |||
2829 | 508 | __slots__ = () | ||
2830 | 509 | |||
2831 | 510 | |||
2832 | 511 | Delta = collections.namedtuple('Delta', ['previous', 'current']) | ||
2833 | 512 | |||
2834 | 513 | |||
2835 | 514 | _KV = None | ||
2836 | 515 | |||
2837 | 516 | |||
2838 | 517 | def kv(): | ||
2839 | 518 | global _KV | ||
2840 | 519 | if _KV is None: | ||
2841 | 520 | _KV = Storage() | ||
2842 | 521 | return _KV | ||
2843 | 0 | 522 | ||
2844 | === modified file 'hooks/charmhelpers/fetch/__init__.py' | |||
2845 | --- hooks/charmhelpers/fetch/__init__.py 2015-01-27 14:54:02 +0000 | |||
2846 | +++ hooks/charmhelpers/fetch/__init__.py 2016-04-05 14:45:53 +0000 | |||
2847 | @@ -90,6 +90,22 @@ | |||
2848 | 90 | 'kilo/proposed': 'trusty-proposed/kilo', | 90 | 'kilo/proposed': 'trusty-proposed/kilo', |
2849 | 91 | 'trusty-kilo/proposed': 'trusty-proposed/kilo', | 91 | 'trusty-kilo/proposed': 'trusty-proposed/kilo', |
2850 | 92 | 'trusty-proposed/kilo': 'trusty-proposed/kilo', | 92 | 'trusty-proposed/kilo': 'trusty-proposed/kilo', |
2851 | 93 | # Liberty | ||
2852 | 94 | 'liberty': 'trusty-updates/liberty', | ||
2853 | 95 | 'trusty-liberty': 'trusty-updates/liberty', | ||
2854 | 96 | 'trusty-liberty/updates': 'trusty-updates/liberty', | ||
2855 | 97 | 'trusty-updates/liberty': 'trusty-updates/liberty', | ||
2856 | 98 | 'liberty/proposed': 'trusty-proposed/liberty', | ||
2857 | 99 | 'trusty-liberty/proposed': 'trusty-proposed/liberty', | ||
2858 | 100 | 'trusty-proposed/liberty': 'trusty-proposed/liberty', | ||
2859 | 101 | # Mitaka | ||
2860 | 102 | 'mitaka': 'trusty-updates/mitaka', | ||
2861 | 103 | 'trusty-mitaka': 'trusty-updates/mitaka', | ||
2862 | 104 | 'trusty-mitaka/updates': 'trusty-updates/mitaka', | ||
2863 | 105 | 'trusty-updates/mitaka': 'trusty-updates/mitaka', | ||
2864 | 106 | 'mitaka/proposed': 'trusty-proposed/mitaka', | ||
2865 | 107 | 'trusty-mitaka/proposed': 'trusty-proposed/mitaka', | ||
2866 | 108 | 'trusty-proposed/mitaka': 'trusty-proposed/mitaka', | ||
2867 | 93 | } | 109 | } |
2868 | 94 | 110 | ||
2869 | 95 | # The order of this list is very important. Handlers should be listed in from | 111 | # The order of this list is very important. Handlers should be listed in from |
2870 | @@ -158,7 +174,7 @@ | |||
2871 | 158 | 174 | ||
2872 | 159 | def apt_cache(in_memory=True): | 175 | def apt_cache(in_memory=True): |
2873 | 160 | """Build and return an apt cache""" | 176 | """Build and return an apt cache""" |
2875 | 161 | import apt_pkg | 177 | from apt import apt_pkg |
2876 | 162 | apt_pkg.init() | 178 | apt_pkg.init() |
2877 | 163 | if in_memory: | 179 | if in_memory: |
2878 | 164 | apt_pkg.config.set("Dir::Cache::pkgcache", "") | 180 | apt_pkg.config.set("Dir::Cache::pkgcache", "") |
2879 | @@ -215,19 +231,27 @@ | |||
2880 | 215 | _run_apt_command(cmd, fatal) | 231 | _run_apt_command(cmd, fatal) |
2881 | 216 | 232 | ||
2882 | 217 | 233 | ||
2883 | 234 | def apt_mark(packages, mark, fatal=False): | ||
2884 | 235 | """Flag one or more packages using apt-mark""" | ||
2885 | 236 | log("Marking {} as {}".format(packages, mark)) | ||
2886 | 237 | cmd = ['apt-mark', mark] | ||
2887 | 238 | if isinstance(packages, six.string_types): | ||
2888 | 239 | cmd.append(packages) | ||
2889 | 240 | else: | ||
2890 | 241 | cmd.extend(packages) | ||
2891 | 242 | |||
2892 | 243 | if fatal: | ||
2893 | 244 | subprocess.check_call(cmd, universal_newlines=True) | ||
2894 | 245 | else: | ||
2895 | 246 | subprocess.call(cmd, universal_newlines=True) | ||
2896 | 247 | |||
2897 | 248 | |||
2898 | 218 | def apt_hold(packages, fatal=False): | 249 | def apt_hold(packages, fatal=False): |
2911 | 219 | """Hold one or more packages""" | 250 | return apt_mark(packages, 'hold', fatal=fatal) |
2912 | 220 | cmd = ['apt-mark', 'hold'] | 251 | |
2913 | 221 | if isinstance(packages, six.string_types): | 252 | |
2914 | 222 | cmd.append(packages) | 253 | def apt_unhold(packages, fatal=False): |
2915 | 223 | else: | 254 | return apt_mark(packages, 'unhold', fatal=fatal) |
2904 | 224 | cmd.extend(packages) | ||
2905 | 225 | log("Holding {}".format(packages)) | ||
2906 | 226 | |||
2907 | 227 | if fatal: | ||
2908 | 228 | subprocess.check_call(cmd) | ||
2909 | 229 | else: | ||
2910 | 230 | subprocess.call(cmd) | ||
2916 | 231 | 255 | ||
2917 | 232 | 256 | ||
2918 | 233 | def add_source(source, key=None): | 257 | def add_source(source, key=None): |
2919 | @@ -370,8 +394,9 @@ | |||
2920 | 370 | for handler in handlers: | 394 | for handler in handlers: |
2921 | 371 | try: | 395 | try: |
2922 | 372 | installed_to = handler.install(source, *args, **kwargs) | 396 | installed_to = handler.install(source, *args, **kwargs) |
2925 | 373 | except UnhandledSource: | 397 | except UnhandledSource as e: |
2926 | 374 | pass | 398 | log('Install source attempt unsuccessful: {}'.format(e), |
2927 | 399 | level='WARNING') | ||
2928 | 375 | if not installed_to: | 400 | if not installed_to: |
2929 | 376 | raise UnhandledSource("No handler found for source {}".format(source)) | 401 | raise UnhandledSource("No handler found for source {}".format(source)) |
2930 | 377 | return installed_to | 402 | return installed_to |
2931 | @@ -394,7 +419,7 @@ | |||
2932 | 394 | importlib.import_module(package), | 419 | importlib.import_module(package), |
2933 | 395 | classname) | 420 | classname) |
2934 | 396 | plugin_list.append(handler_class()) | 421 | plugin_list.append(handler_class()) |
2936 | 397 | except (ImportError, AttributeError): | 422 | except NotImplementedError: |
2937 | 398 | # Skip missing plugins so that they can be ommitted from | 423 | # Skip missing plugins so that they can be ommitted from |
2938 | 399 | # installation if desired | 424 | # installation if desired |
2939 | 400 | log("FetchHandler {} not found, skipping plugin".format( | 425 | log("FetchHandler {} not found, skipping plugin".format( |
2940 | 401 | 426 | ||
2941 | === modified file 'hooks/charmhelpers/fetch/archiveurl.py' | |||
2942 | --- hooks/charmhelpers/fetch/archiveurl.py 2015-01-27 14:54:02 +0000 | |||
2943 | +++ hooks/charmhelpers/fetch/archiveurl.py 2016-04-05 14:45:53 +0000 | |||
2944 | @@ -18,6 +18,16 @@ | |||
2945 | 18 | import hashlib | 18 | import hashlib |
2946 | 19 | import re | 19 | import re |
2947 | 20 | 20 | ||
2948 | 21 | from charmhelpers.fetch import ( | ||
2949 | 22 | BaseFetchHandler, | ||
2950 | 23 | UnhandledSource | ||
2951 | 24 | ) | ||
2952 | 25 | from charmhelpers.payload.archive import ( | ||
2953 | 26 | get_archive_handler, | ||
2954 | 27 | extract, | ||
2955 | 28 | ) | ||
2956 | 29 | from charmhelpers.core.host import mkdir, check_hash | ||
2957 | 30 | |||
2958 | 21 | import six | 31 | import six |
2959 | 22 | if six.PY3: | 32 | if six.PY3: |
2960 | 23 | from urllib.request import ( | 33 | from urllib.request import ( |
2961 | @@ -35,16 +45,6 @@ | |||
2962 | 35 | ) | 45 | ) |
2963 | 36 | from urlparse import urlparse, urlunparse, parse_qs | 46 | from urlparse import urlparse, urlunparse, parse_qs |
2964 | 37 | 47 | ||
2965 | 38 | from charmhelpers.fetch import ( | ||
2966 | 39 | BaseFetchHandler, | ||
2967 | 40 | UnhandledSource | ||
2968 | 41 | ) | ||
2969 | 42 | from charmhelpers.payload.archive import ( | ||
2970 | 43 | get_archive_handler, | ||
2971 | 44 | extract, | ||
2972 | 45 | ) | ||
2973 | 46 | from charmhelpers.core.host import mkdir, check_hash | ||
2974 | 47 | |||
2975 | 48 | 48 | ||
2976 | 49 | def splituser(host): | 49 | def splituser(host): |
2977 | 50 | '''urllib.splituser(), but six's support of this seems broken''' | 50 | '''urllib.splituser(), but six's support of this seems broken''' |
2978 | @@ -77,6 +77,8 @@ | |||
2979 | 77 | def can_handle(self, source): | 77 | def can_handle(self, source): |
2980 | 78 | url_parts = self.parse_url(source) | 78 | url_parts = self.parse_url(source) |
2981 | 79 | if url_parts.scheme not in ('http', 'https', 'ftp', 'file'): | 79 | if url_parts.scheme not in ('http', 'https', 'ftp', 'file'): |
2982 | 80 | # XXX: Why is this returning a boolean and a string? It's | ||
2983 | 81 | # doomed to fail since "bool(can_handle('foo://'))" will be True. | ||
2984 | 80 | return "Wrong source type" | 82 | return "Wrong source type" |
2985 | 81 | if get_archive_handler(self.base_url(source)): | 83 | if get_archive_handler(self.base_url(source)): |
2986 | 82 | return True | 84 | return True |
2987 | @@ -106,7 +108,7 @@ | |||
2988 | 106 | install_opener(opener) | 108 | install_opener(opener) |
2989 | 107 | response = urlopen(source) | 109 | response = urlopen(source) |
2990 | 108 | try: | 110 | try: |
2992 | 109 | with open(dest, 'w') as dest_file: | 111 | with open(dest, 'wb') as dest_file: |
2993 | 110 | dest_file.write(response.read()) | 112 | dest_file.write(response.read()) |
2994 | 111 | except Exception as e: | 113 | except Exception as e: |
2995 | 112 | if os.path.isfile(dest): | 114 | if os.path.isfile(dest): |
2996 | @@ -155,7 +157,11 @@ | |||
2997 | 155 | else: | 157 | else: |
2998 | 156 | algorithms = hashlib.algorithms_available | 158 | algorithms = hashlib.algorithms_available |
2999 | 157 | if key in algorithms: | 159 | if key in algorithms: |
3001 | 158 | check_hash(dld_file, value, key) | 160 | if len(value) != 1: |
3002 | 161 | raise TypeError( | ||
3003 | 162 | "Expected 1 hash value, not %d" % len(value)) | ||
3004 | 163 | expected = value[0] | ||
3005 | 164 | check_hash(dld_file, expected, key) | ||
3006 | 159 | if checksum: | 165 | if checksum: |
3007 | 160 | check_hash(dld_file, checksum, hash_type) | 166 | check_hash(dld_file, checksum, hash_type) |
3008 | 161 | return extract(dld_file, dest) | 167 | return extract(dld_file, dest) |
3009 | 162 | 168 | ||
3010 | === modified file 'hooks/charmhelpers/fetch/bzrurl.py' | |||
3011 | --- hooks/charmhelpers/fetch/bzrurl.py 2015-01-27 14:54:02 +0000 | |||
3012 | +++ hooks/charmhelpers/fetch/bzrurl.py 2016-04-05 14:45:53 +0000 | |||
3013 | @@ -15,60 +15,54 @@ | |||
3014 | 15 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | 15 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
3015 | 16 | 16 | ||
3016 | 17 | import os | 17 | import os |
3017 | 18 | from subprocess import check_call, CalledProcessError | ||
3018 | 18 | from charmhelpers.fetch import ( | 19 | from charmhelpers.fetch import ( |
3019 | 19 | BaseFetchHandler, | 20 | BaseFetchHandler, |
3021 | 20 | UnhandledSource | 21 | UnhandledSource, |
3022 | 22 | filter_installed_packages, | ||
3023 | 23 | apt_install, | ||
3024 | 21 | ) | 24 | ) |
3025 | 22 | from charmhelpers.core.host import mkdir | 25 | from charmhelpers.core.host import mkdir |
3026 | 23 | 26 | ||
3027 | 24 | import six | ||
3028 | 25 | if six.PY3: | ||
3029 | 26 | raise ImportError('bzrlib does not support Python3') | ||
3030 | 27 | 27 | ||
3039 | 28 | try: | 28 | if filter_installed_packages(['bzr']) != []: |
3040 | 29 | from bzrlib.branch import Branch | 29 | apt_install(['bzr']) |
3041 | 30 | from bzrlib import bzrdir, workingtree, errors | 30 | if filter_installed_packages(['bzr']) != []: |
3042 | 31 | except ImportError: | 31 | raise NotImplementedError('Unable to install bzr') |
3035 | 32 | from charmhelpers.fetch import apt_install | ||
3036 | 33 | apt_install("python-bzrlib") | ||
3037 | 34 | from bzrlib.branch import Branch | ||
3038 | 35 | from bzrlib import bzrdir, workingtree, errors | ||
3043 | 36 | 32 | ||
3044 | 37 | 33 | ||
3045 | 38 | class BzrUrlFetchHandler(BaseFetchHandler): | 34 | class BzrUrlFetchHandler(BaseFetchHandler): |
3046 | 39 | """Handler for bazaar branches via generic and lp URLs""" | 35 | """Handler for bazaar branches via generic and lp URLs""" |
3047 | 40 | def can_handle(self, source): | 36 | def can_handle(self, source): |
3048 | 41 | url_parts = self.parse_url(source) | 37 | url_parts = self.parse_url(source) |
3050 | 42 | if url_parts.scheme not in ('bzr+ssh', 'lp'): | 38 | if url_parts.scheme not in ('bzr+ssh', 'lp', ''): |
3051 | 43 | return False | 39 | return False |
3052 | 40 | elif not url_parts.scheme: | ||
3053 | 41 | return os.path.exists(os.path.join(source, '.bzr')) | ||
3054 | 44 | else: | 42 | else: |
3055 | 45 | return True | 43 | return True |
3056 | 46 | 44 | ||
3057 | 47 | def branch(self, source, dest): | 45 | def branch(self, source, dest): |
3058 | 48 | url_parts = self.parse_url(source) | ||
3059 | 49 | # If we use lp:branchname scheme we need to load plugins | ||
3060 | 50 | if not self.can_handle(source): | 46 | if not self.can_handle(source): |
3061 | 51 | raise UnhandledSource("Cannot handle {}".format(source)) | 47 | raise UnhandledSource("Cannot handle {}".format(source)) |
3078 | 52 | if url_parts.scheme == "lp": | 48 | |
3079 | 53 | from bzrlib.plugin import load_plugins | 49 | # if the target is a bzr repo, pull, else branch |
3080 | 54 | load_plugins() | 50 | try: |
3081 | 55 | try: | 51 | check_call(['bzr', 'revno', dest]) |
3082 | 56 | local_branch = bzrdir.BzrDir.create_branch_convenience(dest) | 52 | except CalledProcessError: |
3083 | 57 | except errors.AlreadyControlDirError: | 53 | check_call(['bzr', 'branch', '--use-existing-dir', source, dest]) |
3084 | 58 | local_branch = Branch.open(dest) | 54 | else: |
3085 | 59 | try: | 55 | check_call(['bzr', 'pull', '--overwrite', '-d', dest, source]) |
3086 | 60 | remote_branch = Branch.open(source) | 56 | |
3087 | 61 | remote_branch.push(local_branch) | 57 | def install(self, source, dest=None): |
3072 | 62 | tree = workingtree.WorkingTree.open(dest) | ||
3073 | 63 | tree.update() | ||
3074 | 64 | except Exception as e: | ||
3075 | 65 | raise e | ||
3076 | 66 | |||
3077 | 67 | def install(self, source): | ||
3088 | 68 | url_parts = self.parse_url(source) | 58 | url_parts = self.parse_url(source) |
3089 | 69 | branch_name = url_parts.path.strip("/").split("/")[-1] | 59 | branch_name = url_parts.path.strip("/").split("/")[-1] |
3092 | 70 | dest_dir = os.path.join(os.environ.get('CHARM_DIR'), "fetched", | 60 | if dest: |
3093 | 71 | branch_name) | 61 | dest_dir = os.path.join(dest, branch_name) |
3094 | 62 | else: | ||
3095 | 63 | dest_dir = os.path.join(os.environ.get('CHARM_DIR'), "fetched", | ||
3096 | 64 | branch_name) | ||
3097 | 65 | |||
3098 | 72 | if not os.path.exists(dest_dir): | 66 | if not os.path.exists(dest_dir): |
3099 | 73 | mkdir(dest_dir, perms=0o755) | 67 | mkdir(dest_dir, perms=0o755) |
3100 | 74 | try: | 68 | try: |
3101 | 75 | 69 | ||
3102 | === modified file 'hooks/charmhelpers/fetch/giturl.py' | |||
3103 | --- hooks/charmhelpers/fetch/giturl.py 2015-01-27 14:54:02 +0000 | |||
3104 | +++ hooks/charmhelpers/fetch/giturl.py 2016-04-05 14:45:53 +0000 | |||
3105 | @@ -15,24 +15,18 @@ | |||
3106 | 15 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. | 15 | # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
3107 | 16 | 16 | ||
3108 | 17 | import os | 17 | import os |
3109 | 18 | from subprocess import check_call, CalledProcessError | ||
3110 | 18 | from charmhelpers.fetch import ( | 19 | from charmhelpers.fetch import ( |
3111 | 19 | BaseFetchHandler, | 20 | BaseFetchHandler, |
3113 | 20 | UnhandledSource | 21 | UnhandledSource, |
3114 | 22 | filter_installed_packages, | ||
3115 | 23 | apt_install, | ||
3116 | 21 | ) | 24 | ) |
3131 | 22 | from charmhelpers.core.host import mkdir | 25 | |
3132 | 23 | 26 | if filter_installed_packages(['git']) != []: | |
3133 | 24 | import six | 27 | apt_install(['git']) |
3134 | 25 | if six.PY3: | 28 | if filter_installed_packages(['git']) != []: |
3135 | 26 | raise ImportError('GitPython does not support Python 3') | 29 | raise NotImplementedError('Unable to install git') |
3122 | 27 | |||
3123 | 28 | try: | ||
3124 | 29 | from git import Repo | ||
3125 | 30 | except ImportError: | ||
3126 | 31 | from charmhelpers.fetch import apt_install | ||
3127 | 32 | apt_install("python-git") | ||
3128 | 33 | from git import Repo | ||
3129 | 34 | |||
3130 | 35 | from git.exc import GitCommandError | ||
3136 | 36 | 30 | ||
3137 | 37 | 31 | ||
3138 | 38 | class GitUrlFetchHandler(BaseFetchHandler): | 32 | class GitUrlFetchHandler(BaseFetchHandler): |
3139 | @@ -40,19 +34,26 @@ | |||
3140 | 40 | def can_handle(self, source): | 34 | def can_handle(self, source): |
3141 | 41 | url_parts = self.parse_url(source) | 35 | url_parts = self.parse_url(source) |
3142 | 42 | # TODO (mattyw) no support for ssh git@ yet | 36 | # TODO (mattyw) no support for ssh git@ yet |
3144 | 43 | if url_parts.scheme not in ('http', 'https', 'git'): | 37 | if url_parts.scheme not in ('http', 'https', 'git', ''): |
3145 | 44 | return False | 38 | return False |
3146 | 39 | elif not url_parts.scheme: | ||
3147 | 40 | return os.path.exists(os.path.join(source, '.git')) | ||
3148 | 45 | else: | 41 | else: |
3149 | 46 | return True | 42 | return True |
3150 | 47 | 43 | ||
3152 | 48 | def clone(self, source, dest, branch): | 44 | def clone(self, source, dest, branch="master", depth=None): |
3153 | 49 | if not self.can_handle(source): | 45 | if not self.can_handle(source): |
3154 | 50 | raise UnhandledSource("Cannot handle {}".format(source)) | 46 | raise UnhandledSource("Cannot handle {}".format(source)) |
3155 | 51 | 47 | ||
3158 | 52 | repo = Repo.clone_from(source, dest) | 48 | if os.path.exists(dest): |
3159 | 53 | repo.git.checkout(branch) | 49 | cmd = ['git', '-C', dest, 'pull', source, branch] |
3160 | 50 | else: | ||
3161 | 51 | cmd = ['git', 'clone', source, dest, '--branch', branch] | ||
3162 | 52 | if depth: | ||
3163 | 53 | cmd.extend(['--depth', depth]) | ||
3164 | 54 | check_call(cmd) | ||
3165 | 54 | 55 | ||
3167 | 55 | def install(self, source, branch="master", dest=None): | 56 | def install(self, source, branch="master", dest=None, depth=None): |
3168 | 56 | url_parts = self.parse_url(source) | 57 | url_parts = self.parse_url(source) |
3169 | 57 | branch_name = url_parts.path.strip("/").split("/")[-1] | 58 | branch_name = url_parts.path.strip("/").split("/")[-1] |
3170 | 58 | if dest: | 59 | if dest: |
3171 | @@ -60,12 +61,10 @@ | |||
3172 | 60 | else: | 61 | else: |
3173 | 61 | dest_dir = os.path.join(os.environ.get('CHARM_DIR'), "fetched", | 62 | dest_dir = os.path.join(os.environ.get('CHARM_DIR'), "fetched", |
3174 | 62 | branch_name) | 63 | branch_name) |
3175 | 63 | if not os.path.exists(dest_dir): | ||
3176 | 64 | mkdir(dest_dir, perms=0o755) | ||
3177 | 65 | try: | 64 | try: |
3181 | 66 | self.clone(source, dest_dir, branch) | 65 | self.clone(source, dest_dir, branch, depth) |
3182 | 67 | except GitCommandError as e: | 66 | except CalledProcessError as e: |
3183 | 68 | raise UnhandledSource(e.message) | 67 | raise UnhandledSource(e) |
3184 | 69 | except OSError as e: | 68 | except OSError as e: |
3185 | 70 | raise UnhandledSource(e.strerror) | 69 | raise UnhandledSource(e.strerror) |
3186 | 71 | return dest_dir | 70 | return dest_dir |
3187 | 72 | 71 | ||
3188 | === added directory 'hooks/diverted' | |||
3189 | === renamed symlink 'hooks/install' => 'hooks/diverted/install' | |||
3190 | === target changed u'hooks.py' => u'../hooks.py' | |||
3191 | === added file 'hooks/install' | |||
3192 | --- hooks/install 1970-01-01 00:00:00 +0000 | |||
3193 | +++ hooks/install 2016-04-05 14:45:53 +0000 | |||
3194 | @@ -0,0 +1,7 @@ | |||
3195 | 1 | #!/bin/bash | ||
3196 | 2 | # Wrapper to deal with newer Ubuntu versions that don't have py2 installed | ||
3197 | 3 | # by default. | ||
3198 | 4 | |||
3199 | 5 | apt-get install -y python python-apt python-requests python-six python-yaml | ||
3200 | 6 | |||
3201 | 7 | exec ./hooks/diverted/install | ||
3202 | 0 | 8 | ||
3203 | === modified file 'hooks/services.py' | |||
3204 | --- hooks/services.py 2015-01-30 10:22:34 +0000 | |||
3205 | +++ hooks/services.py 2016-04-05 14:45:53 +0000 | |||
3206 | @@ -1,4 +1,5 @@ | |||
3207 | 1 | import os | 1 | import os |
3208 | 2 | from urlparse import urlparse | ||
3209 | 2 | 3 | ||
3210 | 3 | from charmhelpers.core import hookenv | 4 | from charmhelpers.core import hookenv |
3211 | 4 | from charmhelpers.core.services.base import ServiceManager | 5 | from charmhelpers.core.services.base import ServiceManager |
3212 | @@ -15,6 +16,9 @@ | |||
3213 | 15 | 'service': 'wp-db', | 16 | 'service': 'wp-db', |
3214 | 16 | 'required_data': [ | 17 | 'required_data': [ |
3215 | 17 | helpers.MysqlRelation(), | 18 | helpers.MysqlRelation(), |
3216 | 19 | { | ||
3217 | 20 | 'outbound_http_proxy': urlparse(config['outbound_http_proxy']) | ||
3218 | 21 | }, | ||
3219 | 18 | helpers.StoredContext('wp-secrets.json', | 22 | helpers.StoredContext('wp-secrets.json', |
3220 | 19 | actions.generate_secrets()), | 23 | actions.generate_secrets()), |
3221 | 20 | ], | 24 | ], |
3222 | @@ -61,7 +65,7 @@ | |||
3223 | 61 | helpers.MysqlRelation(), | 65 | helpers.MysqlRelation(), |
3224 | 62 | wp_helpers.wordpress_configured(), | 66 | wp_helpers.wordpress_configured(), |
3225 | 63 | helpers.RequiredConfig('akismet_key'), | 67 | helpers.RequiredConfig('akismet_key'), |
3227 | 64 | ], | 68 | ], |
3228 | 65 | 'data_ready': [actions.enable_akismet], | 69 | 'data_ready': [actions.enable_akismet], |
3229 | 66 | }, | 70 | }, |
3230 | 67 | { | 71 | { |
3231 | @@ -102,34 +106,10 @@ | |||
3232 | 102 | helpers.RequiredConfig(), | 106 | helpers.RequiredConfig(), |
3233 | 103 | ], | 107 | ], |
3234 | 104 | 'data_ready': [ | 108 | 'data_ready': [ |
3248 | 105 | helpers.render_template( | 109 | actions.write_nrpe_checks |
3236 | 106 | source='wp-nrpe.j2', | ||
3237 | 107 | target='/etc/nagios/nrpe.d/check_{}.cfg'.format( | ||
3238 | 108 | hookenv.local_unit().replace('/', '-'), | ||
3239 | 109 | ) | ||
3240 | 110 | ), | ||
3241 | 111 | helpers.render_template( | ||
3242 | 112 | source='wp-nagios.j2', | ||
3243 | 113 | target='/var/lib/nagios/export/service__{}-{}.cfg'.format( | ||
3244 | 114 | config['nagios_context'], | ||
3245 | 115 | hookenv.local_unit().replace('/', '-'), | ||
3246 | 116 | ) | ||
3247 | 117 | ), | ||
3249 | 118 | ], | 110 | ], |
3250 | 119 | 'data_lost': [ | 111 | 'data_lost': [ |
3264 | 120 | helpers.render_template( | 112 | actions.wipe_nrpe_checks |
3252 | 121 | source='wp-nrpe.j2', | ||
3253 | 122 | target='/etc/nagios/nrpe.d/check_{}.cfg'.format( | ||
3254 | 123 | hookenv.local_unit().replace('/', '-'), | ||
3255 | 124 | ) | ||
3256 | 125 | ), | ||
3257 | 126 | helpers.render_template( | ||
3258 | 127 | source='wp-nagios.j2', | ||
3259 | 128 | target='/var/lib/nagios/export/service__{}-{}.cfg'.format( | ||
3260 | 129 | config['nagios_context'], | ||
3261 | 130 | hookenv.local_unit().replace('/', '-'), | ||
3262 | 131 | ) | ||
3263 | 132 | ), | ||
3265 | 133 | ], | 113 | ], |
3266 | 134 | }, | 114 | }, |
3267 | 135 | ]) | 115 | ]) |
3268 | 136 | 116 | ||
3269 | === modified file 'hooks/wp_helpers.py' | |||
3270 | --- hooks/wp_helpers.py 2015-02-25 17:05:27 +0000 | |||
3271 | +++ hooks/wp_helpers.py 2016-04-05 14:45:53 +0000 | |||
3272 | @@ -1,5 +1,6 @@ | |||
3273 | 1 | import os | 1 | import os |
3274 | 2 | import re | 2 | import re |
3275 | 3 | import yaml | ||
3276 | 3 | import requests | 4 | import requests |
3277 | 4 | import urlparse | 5 | import urlparse |
3278 | 5 | 6 | ||
3279 | @@ -36,6 +37,14 @@ | |||
3280 | 36 | return [] | 37 | return [] |
3281 | 37 | 38 | ||
3282 | 38 | 39 | ||
3283 | 40 | def get_vhost_options(): | ||
3284 | 41 | config = hookenv.config() | ||
3285 | 42 | if config.get('vhost_options', False): | ||
3286 | 43 | vhost_options = yaml.safe_load(config['vhost_options']) | ||
3287 | 44 | return vhost_options | ||
3288 | 45 | return None | ||
3289 | 46 | |||
3290 | 47 | |||
3291 | 39 | class PluginRelation(helpers.RelationContext): | 48 | class PluginRelation(helpers.RelationContext): |
3292 | 40 | name = 'wordpress-plugin' | 49 | name = 'wordpress-plugin' |
3293 | 41 | interface = 'wordpress-plugin' | 50 | interface = 'wordpress-plugin' |
3294 | @@ -85,7 +94,9 @@ | |||
3295 | 85 | urls = [] | 94 | urls = [] |
3296 | 86 | hostnames = [config["blog_hostname"]] | 95 | hostnames = [config["blog_hostname"]] |
3297 | 87 | if config["additional_hostnames"]: | 96 | if config["additional_hostnames"]: |
3299 | 88 | hostnames.extend([h.strip() for h in config["additional_hostnames"].split(",")]) | 97 | hostnames.extend( |
3300 | 98 | [h.strip() for h in | ||
3301 | 99 | config["additional_hostnames"].split(",")]) | ||
3302 | 89 | for hostname in hostnames: | 100 | for hostname in hostnames: |
3303 | 90 | urls.append(urlparse.ParseResult( | 101 | urls.append(urlparse.ParseResult( |
3304 | 91 | 'http', | 102 | 'http', |
3305 | @@ -102,11 +113,12 @@ | |||
3306 | 102 | config = hookenv.config() | 113 | config = hookenv.config() |
3307 | 103 | services = [] | 114 | services = [] |
3308 | 104 | redirects = get_redirects() | 115 | redirects = get_redirects() |
3309 | 116 | extra_vhost_options = get_vhost_options() | ||
3310 | 105 | proxy = self.get_proxy() | 117 | proxy = self.get_proxy() |
3311 | 106 | base_vhost = {"type": "php", | 118 | base_vhost = {"type": "php", |
3312 | 107 | "document_root": str(config["install_path"]), | 119 | "document_root": str(config["install_path"]), |
3313 | 108 | "webserver_options": ["mod_rewrite", "mod_headers"], | 120 | "webserver_options": ["mod_rewrite", "mod_headers"], |
3315 | 109 | "vhost_options": {'Header': 'append Vary "Cookie"'}, | 121 | "vhost_options": [{'Header': 'append Vary "Cookie"'}], |
3316 | 110 | } | 122 | } |
3317 | 111 | for url in self.get_urls(): | 123 | for url in self.get_urls(): |
3318 | 112 | vhost = base_vhost.copy() | 124 | vhost = base_vhost.copy() |
3319 | @@ -115,6 +127,10 @@ | |||
3320 | 115 | vhost["proxy"] = proxy | 127 | vhost["proxy"] = proxy |
3321 | 116 | if url.scheme == "http" and redirects: | 128 | if url.scheme == "http" and redirects: |
3322 | 117 | vhost["redirects"] = redirects | 129 | vhost["redirects"] = redirects |
3323 | 130 | if "redirects" in config and config["redirects"]: | ||
3324 | 131 | vhost["redirect_match"] = yaml.safe_load(config["redirects"]) | ||
3325 | 132 | if extra_vhost_options: | ||
3326 | 133 | vhost["vhost_options"] = vhost["vhost_options"] + extra_vhost_options | ||
3327 | 118 | services.append(vhost) | 134 | services.append(vhost) |
3328 | 119 | 135 | ||
3329 | 120 | return {'services': services} | 136 | return {'services': services} |
3330 | @@ -177,8 +193,8 @@ | |||
3331 | 177 | 'ignore-must-revalidate', | 193 | 'ignore-must-revalidate', |
3332 | 178 | 'ignore-private', | 194 | 'ignore-private', |
3333 | 179 | 'ignore-auth' | 195 | 'ignore-auth' |
3336 | 180 | ] | 196 | ] |
3337 | 181 | }] | 197 | }] |
3338 | 182 | 198 | ||
3339 | 183 | new_services = [] | 199 | new_services = [] |
3340 | 184 | for service in services: | 200 | for service in services: |
3341 | @@ -196,6 +212,8 @@ | |||
3342 | 196 | del(service['document_root']) | 212 | del(service['document_root']) |
3343 | 197 | if 'webserver_options' in service: | 213 | if 'webserver_options' in service: |
3344 | 198 | del(service['webserver_options']) | 214 | del(service['webserver_options']) |
3345 | 215 | if 'redirect_match' in service: | ||
3346 | 216 | del(service['redirect_match']) | ||
3347 | 199 | new_services.append(service) | 217 | new_services.append(service) |
3348 | 200 | return {'services': new_services} | 218 | return {'services': new_services} |
3349 | 201 | 219 | ||
3350 | 202 | 220 | ||
3351 | === modified file 'metadata.yaml' | |||
3352 | --- metadata.yaml 2015-02-24 17:05:07 +0000 | |||
3353 | +++ metadata.yaml 2016-04-05 14:45:53 +0000 | |||
3354 | @@ -1,5 +1,6 @@ | |||
3355 | 1 | name: wordpress | 1 | name: wordpress |
3357 | 2 | maintainer: Nick Moffit <nick.moffit@canonical.com> Jacek Nykis <jacek.nykis@canonical.com> | 2 | maintainer: Nick Moffit <nick.moffit@canonical.com> |
3358 | 3 | maintainer: Jacek Nykis <jacek.nykis@canonical.com> | ||
3359 | 3 | summary: "Wordpress" | 4 | summary: "Wordpress" |
3360 | 4 | description: "Wordpress blog" | 5 | description: "Wordpress blog" |
3361 | 5 | categories: ["applications"] | 6 | categories: ["applications"] |
3362 | 6 | 7 | ||
3363 | === modified file 'templates/wp-apparmor.j2' | |||
3364 | --- templates/wp-apparmor.j2 2014-12-11 18:10:02 +0000 | |||
3365 | +++ templates/wp-apparmor.j2 2016-04-05 14:45:53 +0000 | |||
3366 | @@ -15,6 +15,7 @@ | |||
3367 | 15 | # Webserver subordinate will drop relevant rules here | 15 | # Webserver subordinate will drop relevant rules here |
3368 | 16 | #include <webserver.d> | 16 | #include <webserver.d> |
3369 | 17 | 17 | ||
3370 | 18 | @{PROC}/@{pid}/** r, | ||
3371 | 18 | {{install_path}}/ r, | 19 | {{install_path}}/ r, |
3372 | 19 | {{install_path}}/** r, | 20 | {{install_path}}/** r, |
3373 | 20 | {{install_path}}/wp-content/uploads/ rw, | 21 | {{install_path}}/wp-content/uploads/ rw, |
3374 | 21 | 22 | ||
3375 | === modified file 'templates/wp-info.php.j2' | |||
3376 | --- templates/wp-info.php.j2 2015-01-29 15:48:37 +0000 | |||
3377 | +++ templates/wp-info.php.j2 2016-04-05 14:45:53 +0000 | |||
3378 | @@ -51,3 +51,16 @@ | |||
3379 | 51 | 51 | ||
3380 | 52 | $table_prefix = 'wp_'; | 52 | $table_prefix = 'wp_'; |
3381 | 53 | 53 | ||
3382 | 54 | {% if outbound_http_proxy.hostname and outbound_http_proxy.port %} | ||
3383 | 55 | define('WP_PROXY_HOST', '{{outbound_http_proxy.hostname}}'); | ||
3384 | 56 | define('WP_PROXY_PORT', '{{outbound_http_proxy.port}}'); | ||
3385 | 57 | define('WP_PROXY_BYPASS_HOSTS', 'localhost'); | ||
3386 | 58 | {% endif %} | ||
3387 | 59 | |||
3388 | 60 | {% if outbound_http_proxy.username %} | ||
3389 | 61 | define('WP_PROXY_USERNAME', '{{outbound_http_proxy.username}}'); | ||
3390 | 62 | {% endif %} | ||
3391 | 63 | |||
3392 | 64 | {% if outbound_http_proxy.username and outbound_http_proxy.password %} | ||
3393 | 65 | define('WP_PROXY_PASSWORD', '{{outbound_http_proxy.password}}'); | ||
3394 | 66 | {% endif %} | ||
3395 | 54 | 67 | ||
3396 | === removed file 'templates/wp-nagios.j2' | |||
3397 | --- templates/wp-nagios.j2 2014-12-17 11:21:05 +0000 | |||
3398 | +++ templates/wp-nagios.j2 1970-01-01 00:00:00 +0000 | |||
3399 | @@ -1,48 +0,0 @@ | |||
3400 | 1 | # | ||
3401 | 2 | # " " | ||
3402 | 3 | # mmm m m mmm m m | ||
3403 | 4 | # # # # # # # | ||
3404 | 5 | # # # # # # # | ||
3405 | 6 | # # "mm"# # "mm"# | ||
3406 | 7 | # # # | ||
3407 | 8 | # "" "" | ||
3408 | 9 | # This file is managed by Juju. Do not make local changes. | ||
3409 | 10 | |||
3410 | 11 | {% macro servicegroup(default='juju') -%} | ||
3411 | 12 | {%- if config['nagios_servicegroups'] -%} | ||
3412 | 13 | {{ config['nagios_servicegroups'] }} | ||
3413 | 14 | {%- elif config['nagios_context'] -%} | ||
3414 | 15 | {{ config['nagios_context'] }} | ||
3415 | 16 | {%- else -%} | ||
3416 | 17 | {{ default }} | ||
3417 | 18 | {%- endif -%} | ||
3418 | 19 | {%- endmacro %} | ||
3419 | 20 | |||
3420 | 21 | {% for rel in nrpe_external_master -%} | ||
3421 | 22 | define service { | ||
3422 | 23 | use active-service | ||
3423 | 24 | host_name {{rel.nagios_hostname}} | ||
3424 | 25 | service_description {{rel.nagios_hostname}}[wordpress_http] Check Wordpress HTTP | ||
3425 | 26 | check_command check_nrpe!check_wordpress_http | ||
3426 | 27 | servicegroups {{ servicegroup() }} | ||
3427 | 28 | } | ||
3428 | 29 | |||
3429 | 30 | {% if config['ssl_enabled'] -%} | ||
3430 | 31 | define service { | ||
3431 | 32 | use active-service | ||
3432 | 33 | host_name {{rel.nagios_hostname}} | ||
3433 | 34 | service_description {{rel.nagios_hostname}}[wordpress_https] Check Wordpress HTTPS | ||
3434 | 35 | check_command check_nrpe!check_wordpress_https | ||
3435 | 36 | servicegroups {{ servicegroup() }} | ||
3436 | 37 | } | ||
3437 | 38 | {%- endif %} | ||
3438 | 39 | {% for plugin in wordpress_plugin %} | ||
3439 | 40 | define service { | ||
3440 | 41 | use active-service | ||
3441 | 42 | host_name {{rel.nagios_hostname}} | ||
3442 | 43 | service_description {{rel.nagios_hostname}}[{{plugin['plugin_name']|replace('-','_')}}] Check {{plugin['plugin_name']}} Wordpress Plug-In | ||
3443 | 44 | check_command check_nrpe!check_{{plugin['plugin_name']|replace('-','_')}} | ||
3444 | 45 | servicegroups {{ servicegroup() }} | ||
3445 | 46 | } | ||
3446 | 47 | {%- endfor %} | ||
3447 | 48 | {%- endfor %} | ||
3448 | 49 | 0 | ||
3449 | === removed file 'templates/wp-nrpe.j2' | |||
3450 | --- templates/wp-nrpe.j2 2014-12-22 10:40:03 +0000 | |||
3451 | +++ templates/wp-nrpe.j2 1970-01-01 00:00:00 +0000 | |||
3452 | @@ -1,17 +0,0 @@ | |||
3453 | 1 | # | ||
3454 | 2 | # " " | ||
3455 | 3 | # mmm m m mmm m m | ||
3456 | 4 | # # # # # # # | ||
3457 | 5 | # # # # # # # | ||
3458 | 6 | # # "mm"# # "mm"# | ||
3459 | 7 | # # # | ||
3460 | 8 | # "" "" | ||
3461 | 9 | # This file is managed by Juju. Do not make local changes. | ||
3462 | 10 | |||
3463 | 11 | command[check_wordpress_http]=/usr/lib/nagios/plugins/check_http -I localhost -H {{config['blog_hostname']}} -p {{config['port_number']}} --onredirect=critical {% if config['nagios_check_string'] %} -s '{{config['nagios_check_string']}}'{% endif %} | ||
3464 | 12 | {% if config['ssl_enabled'] -%} | ||
3465 | 13 | command[check_wordpress_https]=/usr/lib/nagios/plugins/check_http -I localhost -H {{config['blog_hostname']}} -p {{config['ssl_port_number']}} -S --onredirect=critical {% if config['nagios_check_string'] %} -s '{{config['nagios_check_string']}}'{% endif %} | ||
3466 | 14 | {%- endif %} | ||
3467 | 15 | {%- for rel in wordpress_plugin %} | ||
3468 | 16 | command[check_{{rel['plugin_name']|replace('-','_')}}]=/usr/lib/nagios/plugins/check_file_age -w 31536000 -c 33696000 -f {{config['install_path']}}/wp-content/plugins/{{rel['plugin_name']}} | ||
3469 | 17 | {% endfor -%} |
Also, includes my fix for "INFO install E: Unable to locate package php-cli" (rev 108.2.15 - see https:/ /code.launchpad .net/~sajoupa/ canonical- is-charms/ wordpress- services- fix-lsb- check/+ merge/290981)