Merge lp:~junaidali/charms/trusty/plumgrid-gateway/pg-restart into lp:~plumgrid-team/charms/trusty/plumgrid-gateway/trunk
- Trusty Tahr (14.04)
- pg-restart
- Merge into trunk
Proposed by
Junaid Ali
Status: | Merged |
---|---|
Merged at revision: | 28 |
Proposed branch: | lp:~junaidali/charms/trusty/plumgrid-gateway/pg-restart |
Merge into: | lp:~plumgrid-team/charms/trusty/plumgrid-gateway/trunk |
Diff against target: |
6049 lines (+11/-5761) 48 files modified
charm-helpers-sync.yaml (+6/-1) hooks/charmhelpers/contrib/ansible/__init__.py (+0/-254) hooks/charmhelpers/contrib/benchmark/__init__.py (+0/-126) hooks/charmhelpers/contrib/charmhelpers/__init__.py (+0/-208) hooks/charmhelpers/contrib/charmsupport/__init__.py (+0/-15) hooks/charmhelpers/contrib/charmsupport/nrpe.py (+0/-398) hooks/charmhelpers/contrib/charmsupport/volumes.py (+0/-175) hooks/charmhelpers/contrib/database/mysql.py (+0/-412) hooks/charmhelpers/contrib/hardening/__init__.py (+0/-15) hooks/charmhelpers/contrib/hardening/apache/__init__.py (+0/-19) hooks/charmhelpers/contrib/hardening/apache/checks/__init__.py (+0/-31) hooks/charmhelpers/contrib/hardening/apache/checks/config.py (+0/-100) hooks/charmhelpers/contrib/hardening/audits/__init__.py (+0/-63) hooks/charmhelpers/contrib/hardening/audits/apache.py (+0/-100) hooks/charmhelpers/contrib/hardening/audits/apt.py (+0/-105) hooks/charmhelpers/contrib/hardening/audits/file.py (+0/-552) hooks/charmhelpers/contrib/hardening/harden.py (+0/-84) hooks/charmhelpers/contrib/hardening/host/__init__.py (+0/-19) hooks/charmhelpers/contrib/hardening/host/checks/__init__.py (+0/-50) hooks/charmhelpers/contrib/hardening/host/checks/apt.py (+0/-39) hooks/charmhelpers/contrib/hardening/host/checks/limits.py (+0/-55) hooks/charmhelpers/contrib/hardening/host/checks/login.py (+0/-67) hooks/charmhelpers/contrib/hardening/host/checks/minimize_access.py (+0/-52) hooks/charmhelpers/contrib/hardening/host/checks/pam.py (+0/-134) hooks/charmhelpers/contrib/hardening/host/checks/profile.py (+0/-45) hooks/charmhelpers/contrib/hardening/host/checks/securetty.py (+0/-39) hooks/charmhelpers/contrib/hardening/host/checks/suid_sgid.py (+0/-131) hooks/charmhelpers/contrib/hardening/host/checks/sysctl.py (+0/-211) hooks/charmhelpers/contrib/hardening/mysql/__init__.py (+0/-19) hooks/charmhelpers/contrib/hardening/mysql/checks/__init__.py (+0/-31) hooks/charmhelpers/contrib/hardening/mysql/checks/config.py (+0/-89) hooks/charmhelpers/contrib/hardening/ssh/__init__.py (+0/-19) hooks/charmhelpers/contrib/hardening/ssh/checks/__init__.py (+0/-31) hooks/charmhelpers/contrib/hardening/ssh/checks/config.py (+0/-394) hooks/charmhelpers/contrib/hardening/templating.py (+0/-71) hooks/charmhelpers/contrib/hardening/utils.py (+0/-157) hooks/charmhelpers/contrib/mellanox/infiniband.py (+0/-151) hooks/charmhelpers/contrib/peerstorage/__init__.py (+0/-269) hooks/charmhelpers/contrib/saltstack/__init__.py (+0/-118) hooks/charmhelpers/contrib/ssl/__init__.py (+0/-94) hooks/charmhelpers/contrib/ssl/service.py (+0/-279) hooks/charmhelpers/contrib/templating/__init__.py (+0/-15) hooks/charmhelpers/contrib/templating/contexts.py (+0/-139) hooks/charmhelpers/contrib/templating/jinja.py (+0/-40) hooks/charmhelpers/contrib/templating/pyformat.py (+0/-29) hooks/charmhelpers/contrib/unison/__init__.py (+0/-313) hooks/pg_gw_utils.py (+3/-2) unit_tests/test_pg_gw_hooks.py (+2/-1) |
To merge this branch: | bzr merge lp:~junaidali/charms/trusty/plumgrid-gateway/pg-restart |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Bilal Baqar | Pending | ||
Review via email: mp+293103@code.launchpad.net |
Commit message
Description of the change
To post a comment you must log in.
- 29. By Junaid Ali
-
Updated restart_pg function
- 30. By Junaid Ali
-
Updated sleep time after starting libvirt-bin
- 31. By Junaid Ali
-
update sleep time in restart_pg, changes for make sync
Preview Diff
[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1 | === modified file 'charm-helpers-sync.yaml' |
2 | --- charm-helpers-sync.yaml 2015-07-29 18:23:55 +0000 |
3 | +++ charm-helpers-sync.yaml 2016-05-01 02:23:39 +0000 |
4 | @@ -3,5 +3,10 @@ |
5 | include: |
6 | - core |
7 | - fetch |
8 | - - contrib |
9 | + - contrib.amulet |
10 | + - contrib.hahelpers |
11 | + - contrib.network |
12 | + - contrib.openstack |
13 | + - contrib.python |
14 | + - contrib.storage |
15 | - payload |
16 | |
17 | === removed directory 'hooks/charmhelpers/contrib/ansible' |
18 | === removed file 'hooks/charmhelpers/contrib/ansible/__init__.py' |
19 | --- hooks/charmhelpers/contrib/ansible/__init__.py 2015-07-29 18:23:55 +0000 |
20 | +++ hooks/charmhelpers/contrib/ansible/__init__.py 1970-01-01 00:00:00 +0000 |
21 | @@ -1,254 +0,0 @@ |
22 | -# Copyright 2014-2015 Canonical Limited. |
23 | -# |
24 | -# This file is part of charm-helpers. |
25 | -# |
26 | -# charm-helpers is free software: you can redistribute it and/or modify |
27 | -# it under the terms of the GNU Lesser General Public License version 3 as |
28 | -# published by the Free Software Foundation. |
29 | -# |
30 | -# charm-helpers is distributed in the hope that it will be useful, |
31 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
32 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
33 | -# GNU Lesser General Public License for more details. |
34 | -# |
35 | -# You should have received a copy of the GNU Lesser General Public License |
36 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
37 | - |
38 | -# Copyright 2013 Canonical Ltd. |
39 | -# |
40 | -# Authors: |
41 | -# Charm Helpers Developers <juju@lists.ubuntu.com> |
42 | -"""Charm Helpers ansible - declare the state of your machines. |
43 | - |
44 | -This helper enables you to declare your machine state, rather than |
45 | -program it procedurally (and have to test each change to your procedures). |
46 | -Your install hook can be as simple as:: |
47 | - |
48 | - {{{ |
49 | - import charmhelpers.contrib.ansible |
50 | - |
51 | - |
52 | - def install(): |
53 | - charmhelpers.contrib.ansible.install_ansible_support() |
54 | - charmhelpers.contrib.ansible.apply_playbook('playbooks/install.yaml') |
55 | - }}} |
56 | - |
57 | -and won't need to change (nor will its tests) when you change the machine |
58 | -state. |
59 | - |
60 | -All of your juju config and relation-data are available as template |
61 | -variables within your playbooks and templates. An install playbook looks |
62 | -something like:: |
63 | - |
64 | - {{{ |
65 | - --- |
66 | - - hosts: localhost |
67 | - user: root |
68 | - |
69 | - tasks: |
70 | - - name: Add private repositories. |
71 | - template: |
72 | - src: ../templates/private-repositories.list.jinja2 |
73 | - dest: /etc/apt/sources.list.d/private.list |
74 | - |
75 | - - name: Update the cache. |
76 | - apt: update_cache=yes |
77 | - |
78 | - - name: Install dependencies. |
79 | - apt: pkg={{ item }} |
80 | - with_items: |
81 | - - python-mimeparse |
82 | - - python-webob |
83 | - - sunburnt |
84 | - |
85 | - - name: Setup groups. |
86 | - group: name={{ item.name }} gid={{ item.gid }} |
87 | - with_items: |
88 | - - { name: 'deploy_user', gid: 1800 } |
89 | - - { name: 'service_user', gid: 1500 } |
90 | - |
91 | - ... |
92 | - }}} |
93 | - |
94 | -Read more online about `playbooks`_ and standard ansible `modules`_. |
95 | - |
96 | -.. _playbooks: http://www.ansibleworks.com/docs/playbooks.html |
97 | -.. _modules: http://www.ansibleworks.com/docs/modules.html |
98 | - |
99 | -A further feature os the ansible hooks is to provide a light weight "action" |
100 | -scripting tool. This is a decorator that you apply to a function, and that |
101 | -function can now receive cli args, and can pass extra args to the playbook. |
102 | - |
103 | -e.g. |
104 | - |
105 | - |
106 | -@hooks.action() |
107 | -def some_action(amount, force="False"): |
108 | - "Usage: some-action AMOUNT [force=True]" # <-- shown on error |
109 | - # process the arguments |
110 | - # do some calls |
111 | - # return extra-vars to be passed to ansible-playbook |
112 | - return { |
113 | - 'amount': int(amount), |
114 | - 'type': force, |
115 | - } |
116 | - |
117 | -You can now create a symlink to hooks.py that can be invoked like a hook, but |
118 | -with cli params: |
119 | - |
120 | -# link actions/some-action to hooks/hooks.py |
121 | - |
122 | -actions/some-action amount=10 force=true |
123 | - |
124 | -""" |
125 | -import os |
126 | -import stat |
127 | -import subprocess |
128 | -import functools |
129 | - |
130 | -import charmhelpers.contrib.templating.contexts |
131 | -import charmhelpers.core.host |
132 | -import charmhelpers.core.hookenv |
133 | -import charmhelpers.fetch |
134 | - |
135 | - |
136 | -charm_dir = os.environ.get('CHARM_DIR', '') |
137 | -ansible_hosts_path = '/etc/ansible/hosts' |
138 | -# Ansible will automatically include any vars in the following |
139 | -# file in its inventory when run locally. |
140 | -ansible_vars_path = '/etc/ansible/host_vars/localhost' |
141 | - |
142 | - |
143 | -def install_ansible_support(from_ppa=True, ppa_location='ppa:rquillo/ansible'): |
144 | - """Installs the ansible package. |
145 | - |
146 | - By default it is installed from the `PPA`_ linked from |
147 | - the ansible `website`_ or from a ppa specified by a charm config.. |
148 | - |
149 | - .. _PPA: https://launchpad.net/~rquillo/+archive/ansible |
150 | - .. _website: http://docs.ansible.com/intro_installation.html#latest-releases-via-apt-ubuntu |
151 | - |
152 | - If from_ppa is empty, you must ensure that the package is available |
153 | - from a configured repository. |
154 | - """ |
155 | - if from_ppa: |
156 | - charmhelpers.fetch.add_source(ppa_location) |
157 | - charmhelpers.fetch.apt_update(fatal=True) |
158 | - charmhelpers.fetch.apt_install('ansible') |
159 | - with open(ansible_hosts_path, 'w+') as hosts_file: |
160 | - hosts_file.write('localhost ansible_connection=local') |
161 | - |
162 | - |
163 | -def apply_playbook(playbook, tags=None, extra_vars=None): |
164 | - tags = tags or [] |
165 | - tags = ",".join(tags) |
166 | - charmhelpers.contrib.templating.contexts.juju_state_to_yaml( |
167 | - ansible_vars_path, namespace_separator='__', |
168 | - allow_hyphens_in_keys=False, mode=(stat.S_IRUSR | stat.S_IWUSR)) |
169 | - |
170 | - # we want ansible's log output to be unbuffered |
171 | - env = os.environ.copy() |
172 | - env['PYTHONUNBUFFERED'] = "1" |
173 | - call = [ |
174 | - 'ansible-playbook', |
175 | - '-c', |
176 | - 'local', |
177 | - playbook, |
178 | - ] |
179 | - if tags: |
180 | - call.extend(['--tags', '{}'.format(tags)]) |
181 | - if extra_vars: |
182 | - extra = ["%s=%s" % (k, v) for k, v in extra_vars.items()] |
183 | - call.extend(['--extra-vars', " ".join(extra)]) |
184 | - subprocess.check_call(call, env=env) |
185 | - |
186 | - |
187 | -class AnsibleHooks(charmhelpers.core.hookenv.Hooks): |
188 | - """Run a playbook with the hook-name as the tag. |
189 | - |
190 | - This helper builds on the standard hookenv.Hooks helper, |
191 | - but additionally runs the playbook with the hook-name specified |
192 | - using --tags (ie. running all the tasks tagged with the hook-name). |
193 | - |
194 | - Example:: |
195 | - |
196 | - hooks = AnsibleHooks(playbook_path='playbooks/my_machine_state.yaml') |
197 | - |
198 | - # All the tasks within my_machine_state.yaml tagged with 'install' |
199 | - # will be run automatically after do_custom_work() |
200 | - @hooks.hook() |
201 | - def install(): |
202 | - do_custom_work() |
203 | - |
204 | - # For most of your hooks, you won't need to do anything other |
205 | - # than run the tagged tasks for the hook: |
206 | - @hooks.hook('config-changed', 'start', 'stop') |
207 | - def just_use_playbook(): |
208 | - pass |
209 | - |
210 | - # As a convenience, you can avoid the above noop function by specifying |
211 | - # the hooks which are handled by ansible-only and they'll be registered |
212 | - # for you: |
213 | - # hooks = AnsibleHooks( |
214 | - # 'playbooks/my_machine_state.yaml', |
215 | - # default_hooks=['config-changed', 'start', 'stop']) |
216 | - |
217 | - if __name__ == "__main__": |
218 | - # execute a hook based on the name the program is called by |
219 | - hooks.execute(sys.argv) |
220 | - |
221 | - """ |
222 | - |
223 | - def __init__(self, playbook_path, default_hooks=None): |
224 | - """Register any hooks handled by ansible.""" |
225 | - super(AnsibleHooks, self).__init__() |
226 | - |
227 | - self._actions = {} |
228 | - self.playbook_path = playbook_path |
229 | - |
230 | - default_hooks = default_hooks or [] |
231 | - |
232 | - def noop(*args, **kwargs): |
233 | - pass |
234 | - |
235 | - for hook in default_hooks: |
236 | - self.register(hook, noop) |
237 | - |
238 | - def register_action(self, name, function): |
239 | - """Register a hook""" |
240 | - self._actions[name] = function |
241 | - |
242 | - def execute(self, args): |
243 | - """Execute the hook followed by the playbook using the hook as tag.""" |
244 | - hook_name = os.path.basename(args[0]) |
245 | - extra_vars = None |
246 | - if hook_name in self._actions: |
247 | - extra_vars = self._actions[hook_name](args[1:]) |
248 | - else: |
249 | - super(AnsibleHooks, self).execute(args) |
250 | - |
251 | - charmhelpers.contrib.ansible.apply_playbook( |
252 | - self.playbook_path, tags=[hook_name], extra_vars=extra_vars) |
253 | - |
254 | - def action(self, *action_names): |
255 | - """Decorator, registering them as actions""" |
256 | - def action_wrapper(decorated): |
257 | - |
258 | - @functools.wraps(decorated) |
259 | - def wrapper(argv): |
260 | - kwargs = dict(arg.split('=') for arg in argv) |
261 | - try: |
262 | - return decorated(**kwargs) |
263 | - except TypeError as e: |
264 | - if decorated.__doc__: |
265 | - e.args += (decorated.__doc__,) |
266 | - raise |
267 | - |
268 | - self.register_action(decorated.__name__, wrapper) |
269 | - if '_' in decorated.__name__: |
270 | - self.register_action( |
271 | - decorated.__name__.replace('_', '-'), wrapper) |
272 | - |
273 | - return wrapper |
274 | - |
275 | - return action_wrapper |
276 | |
277 | === removed directory 'hooks/charmhelpers/contrib/benchmark' |
278 | === removed file 'hooks/charmhelpers/contrib/benchmark/__init__.py' |
279 | --- hooks/charmhelpers/contrib/benchmark/__init__.py 2015-07-29 18:23:55 +0000 |
280 | +++ hooks/charmhelpers/contrib/benchmark/__init__.py 1970-01-01 00:00:00 +0000 |
281 | @@ -1,126 +0,0 @@ |
282 | -# Copyright 2014-2015 Canonical Limited. |
283 | -# |
284 | -# This file is part of charm-helpers. |
285 | -# |
286 | -# charm-helpers is free software: you can redistribute it and/or modify |
287 | -# it under the terms of the GNU Lesser General Public License version 3 as |
288 | -# published by the Free Software Foundation. |
289 | -# |
290 | -# charm-helpers is distributed in the hope that it will be useful, |
291 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
292 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
293 | -# GNU Lesser General Public License for more details. |
294 | -# |
295 | -# You should have received a copy of the GNU Lesser General Public License |
296 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
297 | - |
298 | -import subprocess |
299 | -import time |
300 | -import os |
301 | -from distutils.spawn import find_executable |
302 | - |
303 | -from charmhelpers.core.hookenv import ( |
304 | - in_relation_hook, |
305 | - relation_ids, |
306 | - relation_set, |
307 | - relation_get, |
308 | -) |
309 | - |
310 | - |
311 | -def action_set(key, val): |
312 | - if find_executable('action-set'): |
313 | - action_cmd = ['action-set'] |
314 | - |
315 | - if isinstance(val, dict): |
316 | - for k, v in iter(val.items()): |
317 | - action_set('%s.%s' % (key, k), v) |
318 | - return True |
319 | - |
320 | - action_cmd.append('%s=%s' % (key, val)) |
321 | - subprocess.check_call(action_cmd) |
322 | - return True |
323 | - return False |
324 | - |
325 | - |
326 | -class Benchmark(): |
327 | - """ |
328 | - Helper class for the `benchmark` interface. |
329 | - |
330 | - :param list actions: Define the actions that are also benchmarks |
331 | - |
332 | - From inside the benchmark-relation-changed hook, you would |
333 | - Benchmark(['memory', 'cpu', 'disk', 'smoke', 'custom']) |
334 | - |
335 | - Examples: |
336 | - |
337 | - siege = Benchmark(['siege']) |
338 | - siege.start() |
339 | - [... run siege ...] |
340 | - # The higher the score, the better the benchmark |
341 | - siege.set_composite_score(16.70, 'trans/sec', 'desc') |
342 | - siege.finish() |
343 | - |
344 | - |
345 | - """ |
346 | - |
347 | - BENCHMARK_CONF = '/etc/benchmark.conf' # Replaced in testing |
348 | - |
349 | - required_keys = [ |
350 | - 'hostname', |
351 | - 'port', |
352 | - 'graphite_port', |
353 | - 'graphite_endpoint', |
354 | - 'api_port' |
355 | - ] |
356 | - |
357 | - def __init__(self, benchmarks=None): |
358 | - if in_relation_hook(): |
359 | - if benchmarks is not None: |
360 | - for rid in sorted(relation_ids('benchmark')): |
361 | - relation_set(relation_id=rid, relation_settings={ |
362 | - 'benchmarks': ",".join(benchmarks) |
363 | - }) |
364 | - |
365 | - # Check the relation data |
366 | - config = {} |
367 | - for key in self.required_keys: |
368 | - val = relation_get(key) |
369 | - if val is not None: |
370 | - config[key] = val |
371 | - else: |
372 | - # We don't have all of the required keys |
373 | - config = {} |
374 | - break |
375 | - |
376 | - if len(config): |
377 | - with open(self.BENCHMARK_CONF, 'w') as f: |
378 | - for key, val in iter(config.items()): |
379 | - f.write("%s=%s\n" % (key, val)) |
380 | - |
381 | - @staticmethod |
382 | - def start(): |
383 | - action_set('meta.start', time.strftime('%Y-%m-%dT%H:%M:%SZ')) |
384 | - |
385 | - """ |
386 | - If the collectd charm is also installed, tell it to send a snapshot |
387 | - of the current profile data. |
388 | - """ |
389 | - COLLECT_PROFILE_DATA = '/usr/local/bin/collect-profile-data' |
390 | - if os.path.exists(COLLECT_PROFILE_DATA): |
391 | - subprocess.check_output([COLLECT_PROFILE_DATA]) |
392 | - |
393 | - @staticmethod |
394 | - def finish(): |
395 | - action_set('meta.stop', time.strftime('%Y-%m-%dT%H:%M:%SZ')) |
396 | - |
397 | - @staticmethod |
398 | - def set_composite_score(value, units, direction='asc'): |
399 | - """ |
400 | - Set the composite score for a benchmark run. This is a single number |
401 | - representative of the benchmark results. This could be the most |
402 | - important metric, or an amalgamation of metric scores. |
403 | - """ |
404 | - return action_set( |
405 | - "meta.composite", |
406 | - {'value': value, 'units': units, 'direction': direction} |
407 | - ) |
408 | |
409 | === removed directory 'hooks/charmhelpers/contrib/charmhelpers' |
410 | === removed file 'hooks/charmhelpers/contrib/charmhelpers/__init__.py' |
411 | --- hooks/charmhelpers/contrib/charmhelpers/__init__.py 2015-07-29 18:23:55 +0000 |
412 | +++ hooks/charmhelpers/contrib/charmhelpers/__init__.py 1970-01-01 00:00:00 +0000 |
413 | @@ -1,208 +0,0 @@ |
414 | -# Copyright 2014-2015 Canonical Limited. |
415 | -# |
416 | -# This file is part of charm-helpers. |
417 | -# |
418 | -# charm-helpers is free software: you can redistribute it and/or modify |
419 | -# it under the terms of the GNU Lesser General Public License version 3 as |
420 | -# published by the Free Software Foundation. |
421 | -# |
422 | -# charm-helpers is distributed in the hope that it will be useful, |
423 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
424 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
425 | -# GNU Lesser General Public License for more details. |
426 | -# |
427 | -# You should have received a copy of the GNU Lesser General Public License |
428 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
429 | - |
430 | -# Copyright 2012 Canonical Ltd. This software is licensed under the |
431 | -# GNU Affero General Public License version 3 (see the file LICENSE). |
432 | - |
433 | -import warnings |
434 | -warnings.warn("contrib.charmhelpers is deprecated", DeprecationWarning) # noqa |
435 | - |
436 | -import operator |
437 | -import tempfile |
438 | -import time |
439 | -import yaml |
440 | -import subprocess |
441 | - |
442 | -import six |
443 | -if six.PY3: |
444 | - from urllib.request import urlopen |
445 | - from urllib.error import (HTTPError, URLError) |
446 | -else: |
447 | - from urllib2 import (urlopen, HTTPError, URLError) |
448 | - |
449 | -"""Helper functions for writing Juju charms in Python.""" |
450 | - |
451 | -__metaclass__ = type |
452 | -__all__ = [ |
453 | - # 'get_config', # core.hookenv.config() |
454 | - # 'log', # core.hookenv.log() |
455 | - # 'log_entry', # core.hookenv.log() |
456 | - # 'log_exit', # core.hookenv.log() |
457 | - # 'relation_get', # core.hookenv.relation_get() |
458 | - # 'relation_set', # core.hookenv.relation_set() |
459 | - # 'relation_ids', # core.hookenv.relation_ids() |
460 | - # 'relation_list', # core.hookenv.relation_units() |
461 | - # 'config_get', # core.hookenv.config() |
462 | - # 'unit_get', # core.hookenv.unit_get() |
463 | - # 'open_port', # core.hookenv.open_port() |
464 | - # 'close_port', # core.hookenv.close_port() |
465 | - # 'service_control', # core.host.service() |
466 | - 'unit_info', # client-side, NOT IMPLEMENTED |
467 | - 'wait_for_machine', # client-side, NOT IMPLEMENTED |
468 | - 'wait_for_page_contents', # client-side, NOT IMPLEMENTED |
469 | - 'wait_for_relation', # client-side, NOT IMPLEMENTED |
470 | - 'wait_for_unit', # client-side, NOT IMPLEMENTED |
471 | -] |
472 | - |
473 | - |
474 | -SLEEP_AMOUNT = 0.1 |
475 | - |
476 | - |
477 | -# We create a juju_status Command here because it makes testing much, |
478 | -# much easier. |
479 | -def juju_status(): |
480 | - subprocess.check_call(['juju', 'status']) |
481 | - |
482 | -# re-implemented as charmhelpers.fetch.configure_sources() |
483 | -# def configure_source(update=False): |
484 | -# source = config_get('source') |
485 | -# if ((source.startswith('ppa:') or |
486 | -# source.startswith('cloud:') or |
487 | -# source.startswith('http:'))): |
488 | -# run('add-apt-repository', source) |
489 | -# if source.startswith("http:"): |
490 | -# run('apt-key', 'import', config_get('key')) |
491 | -# if update: |
492 | -# run('apt-get', 'update') |
493 | - |
494 | - |
495 | -# DEPRECATED: client-side only |
496 | -def make_charm_config_file(charm_config): |
497 | - charm_config_file = tempfile.NamedTemporaryFile(mode='w+') |
498 | - charm_config_file.write(yaml.dump(charm_config)) |
499 | - charm_config_file.flush() |
500 | - # The NamedTemporaryFile instance is returned instead of just the name |
501 | - # because we want to take advantage of garbage collection-triggered |
502 | - # deletion of the temp file when it goes out of scope in the caller. |
503 | - return charm_config_file |
504 | - |
505 | - |
506 | -# DEPRECATED: client-side only |
507 | -def unit_info(service_name, item_name, data=None, unit=None): |
508 | - if data is None: |
509 | - data = yaml.safe_load(juju_status()) |
510 | - service = data['services'].get(service_name) |
511 | - if service is None: |
512 | - # XXX 2012-02-08 gmb: |
513 | - # This allows us to cope with the race condition that we |
514 | - # have between deploying a service and having it come up in |
515 | - # `juju status`. We could probably do with cleaning it up so |
516 | - # that it fails a bit more noisily after a while. |
517 | - return '' |
518 | - units = service['units'] |
519 | - if unit is not None: |
520 | - item = units[unit][item_name] |
521 | - else: |
522 | - # It might seem odd to sort the units here, but we do it to |
523 | - # ensure that when no unit is specified, the first unit for the |
524 | - # service (or at least the one with the lowest number) is the |
525 | - # one whose data gets returned. |
526 | - sorted_unit_names = sorted(units.keys()) |
527 | - item = units[sorted_unit_names[0]][item_name] |
528 | - return item |
529 | - |
530 | - |
531 | -# DEPRECATED: client-side only |
532 | -def get_machine_data(): |
533 | - return yaml.safe_load(juju_status())['machines'] |
534 | - |
535 | - |
536 | -# DEPRECATED: client-side only |
537 | -def wait_for_machine(num_machines=1, timeout=300): |
538 | - """Wait `timeout` seconds for `num_machines` machines to come up. |
539 | - |
540 | - This wait_for... function can be called by other wait_for functions |
541 | - whose timeouts might be too short in situations where only a bare |
542 | - Juju setup has been bootstrapped. |
543 | - |
544 | - :return: A tuple of (num_machines, time_taken). This is used for |
545 | - testing. |
546 | - """ |
547 | - # You may think this is a hack, and you'd be right. The easiest way |
548 | - # to tell what environment we're working in (LXC vs EC2) is to check |
549 | - # the dns-name of the first machine. If it's localhost we're in LXC |
550 | - # and we can just return here. |
551 | - if get_machine_data()[0]['dns-name'] == 'localhost': |
552 | - return 1, 0 |
553 | - start_time = time.time() |
554 | - while True: |
555 | - # Drop the first machine, since it's the Zookeeper and that's |
556 | - # not a machine that we need to wait for. This will only work |
557 | - # for EC2 environments, which is why we return early above if |
558 | - # we're in LXC. |
559 | - machine_data = get_machine_data() |
560 | - non_zookeeper_machines = [ |
561 | - machine_data[key] for key in list(machine_data.keys())[1:]] |
562 | - if len(non_zookeeper_machines) >= num_machines: |
563 | - all_machines_running = True |
564 | - for machine in non_zookeeper_machines: |
565 | - if machine.get('instance-state') != 'running': |
566 | - all_machines_running = False |
567 | - break |
568 | - if all_machines_running: |
569 | - break |
570 | - if time.time() - start_time >= timeout: |
571 | - raise RuntimeError('timeout waiting for service to start') |
572 | - time.sleep(SLEEP_AMOUNT) |
573 | - return num_machines, time.time() - start_time |
574 | - |
575 | - |
576 | -# DEPRECATED: client-side only |
577 | -def wait_for_unit(service_name, timeout=480): |
578 | - """Wait `timeout` seconds for a given service name to come up.""" |
579 | - wait_for_machine(num_machines=1) |
580 | - start_time = time.time() |
581 | - while True: |
582 | - state = unit_info(service_name, 'agent-state') |
583 | - if 'error' in state or state == 'started': |
584 | - break |
585 | - if time.time() - start_time >= timeout: |
586 | - raise RuntimeError('timeout waiting for service to start') |
587 | - time.sleep(SLEEP_AMOUNT) |
588 | - if state != 'started': |
589 | - raise RuntimeError('unit did not start, agent-state: ' + state) |
590 | - |
591 | - |
592 | -# DEPRECATED: client-side only |
593 | -def wait_for_relation(service_name, relation_name, timeout=120): |
594 | - """Wait `timeout` seconds for a given relation to come up.""" |
595 | - start_time = time.time() |
596 | - while True: |
597 | - relation = unit_info(service_name, 'relations').get(relation_name) |
598 | - if relation is not None and relation['state'] == 'up': |
599 | - break |
600 | - if time.time() - start_time >= timeout: |
601 | - raise RuntimeError('timeout waiting for relation to be up') |
602 | - time.sleep(SLEEP_AMOUNT) |
603 | - |
604 | - |
605 | -# DEPRECATED: client-side only |
606 | -def wait_for_page_contents(url, contents, timeout=120, validate=None): |
607 | - if validate is None: |
608 | - validate = operator.contains |
609 | - start_time = time.time() |
610 | - while True: |
611 | - try: |
612 | - stream = urlopen(url) |
613 | - except (HTTPError, URLError): |
614 | - pass |
615 | - else: |
616 | - page = stream.read() |
617 | - if validate(page, contents): |
618 | - return page |
619 | - if time.time() - start_time >= timeout: |
620 | - raise RuntimeError('timeout waiting for contents of ' + url) |
621 | - time.sleep(SLEEP_AMOUNT) |
622 | |
623 | === removed directory 'hooks/charmhelpers/contrib/charmsupport' |
624 | === removed file 'hooks/charmhelpers/contrib/charmsupport/__init__.py' |
625 | --- hooks/charmhelpers/contrib/charmsupport/__init__.py 2015-07-29 18:23:55 +0000 |
626 | +++ hooks/charmhelpers/contrib/charmsupport/__init__.py 1970-01-01 00:00:00 +0000 |
627 | @@ -1,15 +0,0 @@ |
628 | -# Copyright 2014-2015 Canonical Limited. |
629 | -# |
630 | -# This file is part of charm-helpers. |
631 | -# |
632 | -# charm-helpers is free software: you can redistribute it and/or modify |
633 | -# it under the terms of the GNU Lesser General Public License version 3 as |
634 | -# published by the Free Software Foundation. |
635 | -# |
636 | -# charm-helpers is distributed in the hope that it will be useful, |
637 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
638 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
639 | -# GNU Lesser General Public License for more details. |
640 | -# |
641 | -# You should have received a copy of the GNU Lesser General Public License |
642 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
643 | |
644 | === removed file 'hooks/charmhelpers/contrib/charmsupport/nrpe.py' |
645 | --- hooks/charmhelpers/contrib/charmsupport/nrpe.py 2016-01-30 22:40:29 +0000 |
646 | +++ hooks/charmhelpers/contrib/charmsupport/nrpe.py 1970-01-01 00:00:00 +0000 |
647 | @@ -1,398 +0,0 @@ |
648 | -# Copyright 2014-2015 Canonical Limited. |
649 | -# |
650 | -# This file is part of charm-helpers. |
651 | -# |
652 | -# charm-helpers is free software: you can redistribute it and/or modify |
653 | -# it under the terms of the GNU Lesser General Public License version 3 as |
654 | -# published by the Free Software Foundation. |
655 | -# |
656 | -# charm-helpers is distributed in the hope that it will be useful, |
657 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
658 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
659 | -# GNU Lesser General Public License for more details. |
660 | -# |
661 | -# You should have received a copy of the GNU Lesser General Public License |
662 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
663 | - |
664 | -"""Compatibility with the nrpe-external-master charm""" |
665 | -# Copyright 2012 Canonical Ltd. |
666 | -# |
667 | -# Authors: |
668 | -# Matthew Wedgwood <matthew.wedgwood@canonical.com> |
669 | - |
670 | -import subprocess |
671 | -import pwd |
672 | -import grp |
673 | -import os |
674 | -import glob |
675 | -import shutil |
676 | -import re |
677 | -import shlex |
678 | -import yaml |
679 | - |
680 | -from charmhelpers.core.hookenv import ( |
681 | - config, |
682 | - local_unit, |
683 | - log, |
684 | - relation_ids, |
685 | - relation_set, |
686 | - relations_of_type, |
687 | -) |
688 | - |
689 | -from charmhelpers.core.host import service |
690 | - |
691 | -# This module adds compatibility with the nrpe-external-master and plain nrpe |
692 | -# subordinate charms. To use it in your charm: |
693 | -# |
694 | -# 1. Update metadata.yaml |
695 | -# |
696 | -# provides: |
697 | -# (...) |
698 | -# nrpe-external-master: |
699 | -# interface: nrpe-external-master |
700 | -# scope: container |
701 | -# |
702 | -# and/or |
703 | -# |
704 | -# provides: |
705 | -# (...) |
706 | -# local-monitors: |
707 | -# interface: local-monitors |
708 | -# scope: container |
709 | - |
710 | -# |
711 | -# 2. Add the following to config.yaml |
712 | -# |
713 | -# nagios_context: |
714 | -# default: "juju" |
715 | -# type: string |
716 | -# description: | |
717 | -# Used by the nrpe subordinate charms. |
718 | -# A string that will be prepended to instance name to set the host name |
719 | -# in nagios. So for instance the hostname would be something like: |
720 | -# juju-myservice-0 |
721 | -# If you're running multiple environments with the same services in them |
722 | -# this allows you to differentiate between them. |
723 | -# nagios_servicegroups: |
724 | -# default: "" |
725 | -# type: string |
726 | -# description: | |
727 | -# A comma-separated list of nagios servicegroups. |
728 | -# If left empty, the nagios_context will be used as the servicegroup |
729 | -# |
730 | -# 3. Add custom checks (Nagios plugins) to files/nrpe-external-master |
731 | -# |
732 | -# 4. Update your hooks.py with something like this: |
733 | -# |
734 | -# from charmsupport.nrpe import NRPE |
735 | -# (...) |
736 | -# def update_nrpe_config(): |
737 | -# nrpe_compat = NRPE() |
738 | -# nrpe_compat.add_check( |
739 | -# shortname = "myservice", |
740 | -# description = "Check MyService", |
741 | -# check_cmd = "check_http -w 2 -c 10 http://localhost" |
742 | -# ) |
743 | -# nrpe_compat.add_check( |
744 | -# "myservice_other", |
745 | -# "Check for widget failures", |
746 | -# check_cmd = "/srv/myapp/scripts/widget_check" |
747 | -# ) |
748 | -# nrpe_compat.write() |
749 | -# |
750 | -# def config_changed(): |
751 | -# (...) |
752 | -# update_nrpe_config() |
753 | -# |
754 | -# def nrpe_external_master_relation_changed(): |
755 | -# update_nrpe_config() |
756 | -# |
757 | -# def local_monitors_relation_changed(): |
758 | -# update_nrpe_config() |
759 | -# |
760 | -# 5. ln -s hooks.py nrpe-external-master-relation-changed |
761 | -# ln -s hooks.py local-monitors-relation-changed |
762 | - |
763 | - |
764 | -class CheckException(Exception): |
765 | - pass |
766 | - |
767 | - |
768 | -class Check(object): |
769 | - shortname_re = '[A-Za-z0-9-_]+$' |
770 | - service_template = (""" |
771 | -#--------------------------------------------------- |
772 | -# This file is Juju managed |
773 | -#--------------------------------------------------- |
774 | -define service {{ |
775 | - use active-service |
776 | - host_name {nagios_hostname} |
777 | - service_description {nagios_hostname}[{shortname}] """ |
778 | - """{description} |
779 | - check_command check_nrpe!{command} |
780 | - servicegroups {nagios_servicegroup} |
781 | -}} |
782 | -""") |
783 | - |
784 | - def __init__(self, shortname, description, check_cmd): |
785 | - super(Check, self).__init__() |
786 | - # XXX: could be better to calculate this from the service name |
787 | - if not re.match(self.shortname_re, shortname): |
788 | - raise CheckException("shortname must match {}".format( |
789 | - Check.shortname_re)) |
790 | - self.shortname = shortname |
791 | - self.command = "check_{}".format(shortname) |
792 | - # Note: a set of invalid characters is defined by the |
793 | - # Nagios server config |
794 | - # The default is: illegal_object_name_chars=`~!$%^&*"|'<>?,()= |
795 | - self.description = description |
796 | - self.check_cmd = self._locate_cmd(check_cmd) |
797 | - |
798 | - def _get_check_filename(self): |
799 | - return os.path.join(NRPE.nrpe_confdir, '{}.cfg'.format(self.command)) |
800 | - |
801 | - def _get_service_filename(self, hostname): |
802 | - return os.path.join(NRPE.nagios_exportdir, |
803 | - 'service__{}_{}.cfg'.format(hostname, self.command)) |
804 | - |
805 | - def _locate_cmd(self, check_cmd): |
806 | - search_path = ( |
807 | - '/usr/lib/nagios/plugins', |
808 | - '/usr/local/lib/nagios/plugins', |
809 | - ) |
810 | - parts = shlex.split(check_cmd) |
811 | - for path in search_path: |
812 | - if os.path.exists(os.path.join(path, parts[0])): |
813 | - command = os.path.join(path, parts[0]) |
814 | - if len(parts) > 1: |
815 | - command += " " + " ".join(parts[1:]) |
816 | - return command |
817 | - log('Check command not found: {}'.format(parts[0])) |
818 | - return '' |
819 | - |
820 | - def _remove_service_files(self): |
821 | - if not os.path.exists(NRPE.nagios_exportdir): |
822 | - return |
823 | - for f in os.listdir(NRPE.nagios_exportdir): |
824 | - if f.endswith('_{}.cfg'.format(self.command)): |
825 | - os.remove(os.path.join(NRPE.nagios_exportdir, f)) |
826 | - |
827 | - def remove(self, hostname): |
828 | - nrpe_check_file = self._get_check_filename() |
829 | - if os.path.exists(nrpe_check_file): |
830 | - os.remove(nrpe_check_file) |
831 | - self._remove_service_files() |
832 | - |
833 | - def write(self, nagios_context, hostname, nagios_servicegroups): |
834 | - nrpe_check_file = self._get_check_filename() |
835 | - with open(nrpe_check_file, 'w') as nrpe_check_config: |
836 | - nrpe_check_config.write("# check {}\n".format(self.shortname)) |
837 | - nrpe_check_config.write("command[{}]={}\n".format( |
838 | - self.command, self.check_cmd)) |
839 | - |
840 | - if not os.path.exists(NRPE.nagios_exportdir): |
841 | - log('Not writing service config as {} is not accessible'.format( |
842 | - NRPE.nagios_exportdir)) |
843 | - else: |
844 | - self.write_service_config(nagios_context, hostname, |
845 | - nagios_servicegroups) |
846 | - |
847 | - def write_service_config(self, nagios_context, hostname, |
848 | - nagios_servicegroups): |
849 | - self._remove_service_files() |
850 | - |
851 | - templ_vars = { |
852 | - 'nagios_hostname': hostname, |
853 | - 'nagios_servicegroup': nagios_servicegroups, |
854 | - 'description': self.description, |
855 | - 'shortname': self.shortname, |
856 | - 'command': self.command, |
857 | - } |
858 | - nrpe_service_text = Check.service_template.format(**templ_vars) |
859 | - nrpe_service_file = self._get_service_filename(hostname) |
860 | - with open(nrpe_service_file, 'w') as nrpe_service_config: |
861 | - nrpe_service_config.write(str(nrpe_service_text)) |
862 | - |
863 | - def run(self): |
864 | - subprocess.call(self.check_cmd) |
865 | - |
866 | - |
867 | -class NRPE(object): |
868 | - nagios_logdir = '/var/log/nagios' |
869 | - nagios_exportdir = '/var/lib/nagios/export' |
870 | - nrpe_confdir = '/etc/nagios/nrpe.d' |
871 | - |
872 | - def __init__(self, hostname=None): |
873 | - super(NRPE, self).__init__() |
874 | - self.config = config() |
875 | - self.nagios_context = self.config['nagios_context'] |
876 | - if 'nagios_servicegroups' in self.config and self.config['nagios_servicegroups']: |
877 | - self.nagios_servicegroups = self.config['nagios_servicegroups'] |
878 | - else: |
879 | - self.nagios_servicegroups = self.nagios_context |
880 | - self.unit_name = local_unit().replace('/', '-') |
881 | - if hostname: |
882 | - self.hostname = hostname |
883 | - else: |
884 | - nagios_hostname = get_nagios_hostname() |
885 | - if nagios_hostname: |
886 | - self.hostname = nagios_hostname |
887 | - else: |
888 | - self.hostname = "{}-{}".format(self.nagios_context, self.unit_name) |
889 | - self.checks = [] |
890 | - |
891 | - def add_check(self, *args, **kwargs): |
892 | - self.checks.append(Check(*args, **kwargs)) |
893 | - |
894 | - def remove_check(self, *args, **kwargs): |
895 | - if kwargs.get('shortname') is None: |
896 | - raise ValueError('shortname of check must be specified') |
897 | - |
898 | - # Use sensible defaults if they're not specified - these are not |
899 | - # actually used during removal, but they're required for constructing |
900 | - # the Check object; check_disk is chosen because it's part of the |
901 | - # nagios-plugins-basic package. |
902 | - if kwargs.get('check_cmd') is None: |
903 | - kwargs['check_cmd'] = 'check_disk' |
904 | - if kwargs.get('description') is None: |
905 | - kwargs['description'] = '' |
906 | - |
907 | - check = Check(*args, **kwargs) |
908 | - check.remove(self.hostname) |
909 | - |
910 | - def write(self): |
911 | - try: |
912 | - nagios_uid = pwd.getpwnam('nagios').pw_uid |
913 | - nagios_gid = grp.getgrnam('nagios').gr_gid |
914 | - except: |
915 | - log("Nagios user not set up, nrpe checks not updated") |
916 | - return |
917 | - |
918 | - if not os.path.exists(NRPE.nagios_logdir): |
919 | - os.mkdir(NRPE.nagios_logdir) |
920 | - os.chown(NRPE.nagios_logdir, nagios_uid, nagios_gid) |
921 | - |
922 | - nrpe_monitors = {} |
923 | - monitors = {"monitors": {"remote": {"nrpe": nrpe_monitors}}} |
924 | - for nrpecheck in self.checks: |
925 | - nrpecheck.write(self.nagios_context, self.hostname, |
926 | - self.nagios_servicegroups) |
927 | - nrpe_monitors[nrpecheck.shortname] = { |
928 | - "command": nrpecheck.command, |
929 | - } |
930 | - |
931 | - service('restart', 'nagios-nrpe-server') |
932 | - |
933 | - monitor_ids = relation_ids("local-monitors") + \ |
934 | - relation_ids("nrpe-external-master") |
935 | - for rid in monitor_ids: |
936 | - relation_set(relation_id=rid, monitors=yaml.dump(monitors)) |
937 | - |
938 | - |
939 | -def get_nagios_hostcontext(relation_name='nrpe-external-master'): |
940 | - """ |
941 | - Query relation with nrpe subordinate, return the nagios_host_context |
942 | - |
943 | - :param str relation_name: Name of relation nrpe sub joined to |
944 | - """ |
945 | - for rel in relations_of_type(relation_name): |
946 | - if 'nagios_host_context' in rel: |
947 | - return rel['nagios_host_context'] |
948 | - |
949 | - |
950 | -def get_nagios_hostname(relation_name='nrpe-external-master'): |
951 | - """ |
952 | - Query relation with nrpe subordinate, return the nagios_hostname |
953 | - |
954 | - :param str relation_name: Name of relation nrpe sub joined to |
955 | - """ |
956 | - for rel in relations_of_type(relation_name): |
957 | - if 'nagios_hostname' in rel: |
958 | - return rel['nagios_hostname'] |
959 | - |
960 | - |
961 | -def get_nagios_unit_name(relation_name='nrpe-external-master'): |
962 | - """ |
963 | - Return the nagios unit name prepended with host_context if needed |
964 | - |
965 | - :param str relation_name: Name of relation nrpe sub joined to |
966 | - """ |
967 | - host_context = get_nagios_hostcontext(relation_name) |
968 | - if host_context: |
969 | - unit = "%s:%s" % (host_context, local_unit()) |
970 | - else: |
971 | - unit = local_unit() |
972 | - return unit |
973 | - |
974 | - |
975 | -def add_init_service_checks(nrpe, services, unit_name): |
976 | - """ |
977 | - Add checks for each service in list |
978 | - |
979 | - :param NRPE nrpe: NRPE object to add check to |
980 | - :param list services: List of services to check |
981 | - :param str unit_name: Unit name to use in check description |
982 | - """ |
983 | - for svc in services: |
984 | - upstart_init = '/etc/init/%s.conf' % svc |
985 | - sysv_init = '/etc/init.d/%s' % svc |
986 | - if os.path.exists(upstart_init): |
987 | - # Don't add a check for these services from neutron-gateway |
988 | - if svc not in ['ext-port', 'os-charm-phy-nic-mtu']: |
989 | - nrpe.add_check( |
990 | - shortname=svc, |
991 | - description='process check {%s}' % unit_name, |
992 | - check_cmd='check_upstart_job %s' % svc |
993 | - ) |
994 | - elif os.path.exists(sysv_init): |
995 | - cronpath = '/etc/cron.d/nagios-service-check-%s' % svc |
996 | - cron_file = ('*/5 * * * * root ' |
997 | - '/usr/local/lib/nagios/plugins/check_exit_status.pl ' |
998 | - '-s /etc/init.d/%s status > ' |
999 | - '/var/lib/nagios/service-check-%s.txt\n' % (svc, |
1000 | - svc) |
1001 | - ) |
1002 | - f = open(cronpath, 'w') |
1003 | - f.write(cron_file) |
1004 | - f.close() |
1005 | - nrpe.add_check( |
1006 | - shortname=svc, |
1007 | - description='process check {%s}' % unit_name, |
1008 | - check_cmd='check_status_file.py -f ' |
1009 | - '/var/lib/nagios/service-check-%s.txt' % svc, |
1010 | - ) |
1011 | - |
1012 | - |
1013 | -def copy_nrpe_checks(): |
1014 | - """ |
1015 | - Copy the nrpe checks into place |
1016 | - |
1017 | - """ |
1018 | - NAGIOS_PLUGINS = '/usr/local/lib/nagios/plugins' |
1019 | - nrpe_files_dir = os.path.join(os.getenv('CHARM_DIR'), 'hooks', |
1020 | - 'charmhelpers', 'contrib', 'openstack', |
1021 | - 'files') |
1022 | - |
1023 | - if not os.path.exists(NAGIOS_PLUGINS): |
1024 | - os.makedirs(NAGIOS_PLUGINS) |
1025 | - for fname in glob.glob(os.path.join(nrpe_files_dir, "check_*")): |
1026 | - if os.path.isfile(fname): |
1027 | - shutil.copy2(fname, |
1028 | - os.path.join(NAGIOS_PLUGINS, os.path.basename(fname))) |
1029 | - |
1030 | - |
1031 | -def add_haproxy_checks(nrpe, unit_name): |
1032 | - """ |
1033 | - Add checks for each service in list |
1034 | - |
1035 | - :param NRPE nrpe: NRPE object to add check to |
1036 | - :param str unit_name: Unit name to use in check description |
1037 | - """ |
1038 | - nrpe.add_check( |
1039 | - shortname='haproxy_servers', |
1040 | - description='Check HAProxy {%s}' % unit_name, |
1041 | - check_cmd='check_haproxy.sh') |
1042 | - nrpe.add_check( |
1043 | - shortname='haproxy_queue', |
1044 | - description='Check HAProxy queue depth {%s}' % unit_name, |
1045 | - check_cmd='check_haproxy_queue_depth.sh') |
1046 | |
1047 | === removed file 'hooks/charmhelpers/contrib/charmsupport/volumes.py' |
1048 | --- hooks/charmhelpers/contrib/charmsupport/volumes.py 2015-07-29 18:23:55 +0000 |
1049 | +++ hooks/charmhelpers/contrib/charmsupport/volumes.py 1970-01-01 00:00:00 +0000 |
1050 | @@ -1,175 +0,0 @@ |
1051 | -# Copyright 2014-2015 Canonical Limited. |
1052 | -# |
1053 | -# This file is part of charm-helpers. |
1054 | -# |
1055 | -# charm-helpers is free software: you can redistribute it and/or modify |
1056 | -# it under the terms of the GNU Lesser General Public License version 3 as |
1057 | -# published by the Free Software Foundation. |
1058 | -# |
1059 | -# charm-helpers is distributed in the hope that it will be useful, |
1060 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
1061 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
1062 | -# GNU Lesser General Public License for more details. |
1063 | -# |
1064 | -# You should have received a copy of the GNU Lesser General Public License |
1065 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
1066 | - |
1067 | -''' |
1068 | -Functions for managing volumes in juju units. One volume is supported per unit. |
1069 | -Subordinates may have their own storage, provided it is on its own partition. |
1070 | - |
1071 | -Configuration stanzas:: |
1072 | - |
1073 | - volume-ephemeral: |
1074 | - type: boolean |
1075 | - default: true |
1076 | - description: > |
1077 | - If false, a volume is mounted as sepecified in "volume-map" |
1078 | - If true, ephemeral storage will be used, meaning that log data |
1079 | - will only exist as long as the machine. YOU HAVE BEEN WARNED. |
1080 | - volume-map: |
1081 | - type: string |
1082 | - default: {} |
1083 | - description: > |
1084 | - YAML map of units to device names, e.g: |
1085 | - "{ rsyslog/0: /dev/vdb, rsyslog/1: /dev/vdb }" |
1086 | - Service units will raise a configure-error if volume-ephemeral |
1087 | - is 'true' and no volume-map value is set. Use 'juju set' to set a |
1088 | - value and 'juju resolved' to complete configuration. |
1089 | - |
1090 | -Usage:: |
1091 | - |
1092 | - from charmsupport.volumes import configure_volume, VolumeConfigurationError |
1093 | - from charmsupport.hookenv import log, ERROR |
1094 | - def post_mount_hook(): |
1095 | - stop_service('myservice') |
1096 | - def post_mount_hook(): |
1097 | - start_service('myservice') |
1098 | - |
1099 | - if __name__ == '__main__': |
1100 | - try: |
1101 | - configure_volume(before_change=pre_mount_hook, |
1102 | - after_change=post_mount_hook) |
1103 | - except VolumeConfigurationError: |
1104 | - log('Storage could not be configured', ERROR) |
1105 | - |
1106 | -''' |
1107 | - |
1108 | -# XXX: Known limitations |
1109 | -# - fstab is neither consulted nor updated |
1110 | - |
1111 | -import os |
1112 | -from charmhelpers.core import hookenv |
1113 | -from charmhelpers.core import host |
1114 | -import yaml |
1115 | - |
1116 | - |
1117 | -MOUNT_BASE = '/srv/juju/volumes' |
1118 | - |
1119 | - |
1120 | -class VolumeConfigurationError(Exception): |
1121 | - '''Volume configuration data is missing or invalid''' |
1122 | - pass |
1123 | - |
1124 | - |
1125 | -def get_config(): |
1126 | - '''Gather and sanity-check volume configuration data''' |
1127 | - volume_config = {} |
1128 | - config = hookenv.config() |
1129 | - |
1130 | - errors = False |
1131 | - |
1132 | - if config.get('volume-ephemeral') in (True, 'True', 'true', 'Yes', 'yes'): |
1133 | - volume_config['ephemeral'] = True |
1134 | - else: |
1135 | - volume_config['ephemeral'] = False |
1136 | - |
1137 | - try: |
1138 | - volume_map = yaml.safe_load(config.get('volume-map', '{}')) |
1139 | - except yaml.YAMLError as e: |
1140 | - hookenv.log("Error parsing YAML volume-map: {}".format(e), |
1141 | - hookenv.ERROR) |
1142 | - errors = True |
1143 | - if volume_map is None: |
1144 | - # probably an empty string |
1145 | - volume_map = {} |
1146 | - elif not isinstance(volume_map, dict): |
1147 | - hookenv.log("Volume-map should be a dictionary, not {}".format( |
1148 | - type(volume_map))) |
1149 | - errors = True |
1150 | - |
1151 | - volume_config['device'] = volume_map.get(os.environ['JUJU_UNIT_NAME']) |
1152 | - if volume_config['device'] and volume_config['ephemeral']: |
1153 | - # asked for ephemeral storage but also defined a volume ID |
1154 | - hookenv.log('A volume is defined for this unit, but ephemeral ' |
1155 | - 'storage was requested', hookenv.ERROR) |
1156 | - errors = True |
1157 | - elif not volume_config['device'] and not volume_config['ephemeral']: |
1158 | - # asked for permanent storage but did not define volume ID |
1159 | - hookenv.log('Ephemeral storage was requested, but there is no volume ' |
1160 | - 'defined for this unit.', hookenv.ERROR) |
1161 | - errors = True |
1162 | - |
1163 | - unit_mount_name = hookenv.local_unit().replace('/', '-') |
1164 | - volume_config['mountpoint'] = os.path.join(MOUNT_BASE, unit_mount_name) |
1165 | - |
1166 | - if errors: |
1167 | - return None |
1168 | - return volume_config |
1169 | - |
1170 | - |
1171 | -def mount_volume(config): |
1172 | - if os.path.exists(config['mountpoint']): |
1173 | - if not os.path.isdir(config['mountpoint']): |
1174 | - hookenv.log('Not a directory: {}'.format(config['mountpoint'])) |
1175 | - raise VolumeConfigurationError() |
1176 | - else: |
1177 | - host.mkdir(config['mountpoint']) |
1178 | - if os.path.ismount(config['mountpoint']): |
1179 | - unmount_volume(config) |
1180 | - if not host.mount(config['device'], config['mountpoint'], persist=True): |
1181 | - raise VolumeConfigurationError() |
1182 | - |
1183 | - |
1184 | -def unmount_volume(config): |
1185 | - if os.path.ismount(config['mountpoint']): |
1186 | - if not host.umount(config['mountpoint'], persist=True): |
1187 | - raise VolumeConfigurationError() |
1188 | - |
1189 | - |
1190 | -def managed_mounts(): |
1191 | - '''List of all mounted managed volumes''' |
1192 | - return filter(lambda mount: mount[0].startswith(MOUNT_BASE), host.mounts()) |
1193 | - |
1194 | - |
1195 | -def configure_volume(before_change=lambda: None, after_change=lambda: None): |
1196 | - '''Set up storage (or don't) according to the charm's volume configuration. |
1197 | - Returns the mount point or "ephemeral". before_change and after_change |
1198 | - are optional functions to be called if the volume configuration changes. |
1199 | - ''' |
1200 | - |
1201 | - config = get_config() |
1202 | - if not config: |
1203 | - hookenv.log('Failed to read volume configuration', hookenv.CRITICAL) |
1204 | - raise VolumeConfigurationError() |
1205 | - |
1206 | - if config['ephemeral']: |
1207 | - if os.path.ismount(config['mountpoint']): |
1208 | - before_change() |
1209 | - unmount_volume(config) |
1210 | - after_change() |
1211 | - return 'ephemeral' |
1212 | - else: |
1213 | - # persistent storage |
1214 | - if os.path.ismount(config['mountpoint']): |
1215 | - mounts = dict(managed_mounts()) |
1216 | - if mounts.get(config['mountpoint']) != config['device']: |
1217 | - before_change() |
1218 | - unmount_volume(config) |
1219 | - mount_volume(config) |
1220 | - after_change() |
1221 | - else: |
1222 | - before_change() |
1223 | - mount_volume(config) |
1224 | - after_change() |
1225 | - return config['mountpoint'] |
1226 | |
1227 | === removed directory 'hooks/charmhelpers/contrib/database' |
1228 | === removed file 'hooks/charmhelpers/contrib/database/__init__.py' |
1229 | === removed file 'hooks/charmhelpers/contrib/database/mysql.py' |
1230 | --- hooks/charmhelpers/contrib/database/mysql.py 2015-07-29 18:23:55 +0000 |
1231 | +++ hooks/charmhelpers/contrib/database/mysql.py 1970-01-01 00:00:00 +0000 |
1232 | @@ -1,412 +0,0 @@ |
1233 | -"""Helper for working with a MySQL database""" |
1234 | -import json |
1235 | -import re |
1236 | -import sys |
1237 | -import platform |
1238 | -import os |
1239 | -import glob |
1240 | - |
1241 | -# from string import upper |
1242 | - |
1243 | -from charmhelpers.core.host import ( |
1244 | - mkdir, |
1245 | - pwgen, |
1246 | - write_file |
1247 | -) |
1248 | -from charmhelpers.core.hookenv import ( |
1249 | - config as config_get, |
1250 | - relation_get, |
1251 | - related_units, |
1252 | - unit_get, |
1253 | - log, |
1254 | - DEBUG, |
1255 | - INFO, |
1256 | - WARNING, |
1257 | -) |
1258 | -from charmhelpers.fetch import ( |
1259 | - apt_install, |
1260 | - apt_update, |
1261 | - filter_installed_packages, |
1262 | -) |
1263 | -from charmhelpers.contrib.peerstorage import ( |
1264 | - peer_store, |
1265 | - peer_retrieve, |
1266 | -) |
1267 | -from charmhelpers.contrib.network.ip import get_host_ip |
1268 | - |
1269 | -try: |
1270 | - import MySQLdb |
1271 | -except ImportError: |
1272 | - apt_update(fatal=True) |
1273 | - apt_install(filter_installed_packages(['python-mysqldb']), fatal=True) |
1274 | - import MySQLdb |
1275 | - |
1276 | - |
1277 | -class MySQLHelper(object): |
1278 | - |
1279 | - def __init__(self, rpasswdf_template, upasswdf_template, host='localhost', |
1280 | - migrate_passwd_to_peer_relation=True, |
1281 | - delete_ondisk_passwd_file=True): |
1282 | - self.host = host |
1283 | - # Password file path templates |
1284 | - self.root_passwd_file_template = rpasswdf_template |
1285 | - self.user_passwd_file_template = upasswdf_template |
1286 | - |
1287 | - self.migrate_passwd_to_peer_relation = migrate_passwd_to_peer_relation |
1288 | - # If we migrate we have the option to delete local copy of root passwd |
1289 | - self.delete_ondisk_passwd_file = delete_ondisk_passwd_file |
1290 | - |
1291 | - def connect(self, user='root', password=None): |
1292 | - log("Opening db connection for %s@%s" % (user, self.host), level=DEBUG) |
1293 | - self.connection = MySQLdb.connect(user=user, host=self.host, |
1294 | - passwd=password) |
1295 | - |
1296 | - def database_exists(self, db_name): |
1297 | - cursor = self.connection.cursor() |
1298 | - try: |
1299 | - cursor.execute("SHOW DATABASES") |
1300 | - databases = [i[0] for i in cursor.fetchall()] |
1301 | - finally: |
1302 | - cursor.close() |
1303 | - |
1304 | - return db_name in databases |
1305 | - |
1306 | - def create_database(self, db_name): |
1307 | - cursor = self.connection.cursor() |
1308 | - try: |
1309 | - cursor.execute("CREATE DATABASE {} CHARACTER SET UTF8" |
1310 | - .format(db_name)) |
1311 | - finally: |
1312 | - cursor.close() |
1313 | - |
1314 | - def grant_exists(self, db_name, db_user, remote_ip): |
1315 | - cursor = self.connection.cursor() |
1316 | - priv_string = "GRANT ALL PRIVILEGES ON `{}`.* " \ |
1317 | - "TO '{}'@'{}'".format(db_name, db_user, remote_ip) |
1318 | - try: |
1319 | - cursor.execute("SHOW GRANTS for '{}'@'{}'".format(db_user, |
1320 | - remote_ip)) |
1321 | - grants = [i[0] for i in cursor.fetchall()] |
1322 | - except MySQLdb.OperationalError: |
1323 | - return False |
1324 | - finally: |
1325 | - cursor.close() |
1326 | - |
1327 | - # TODO: review for different grants |
1328 | - return priv_string in grants |
1329 | - |
1330 | - def create_grant(self, db_name, db_user, remote_ip, password): |
1331 | - cursor = self.connection.cursor() |
1332 | - try: |
1333 | - # TODO: review for different grants |
1334 | - cursor.execute("GRANT ALL PRIVILEGES ON {}.* TO '{}'@'{}' " |
1335 | - "IDENTIFIED BY '{}'".format(db_name, |
1336 | - db_user, |
1337 | - remote_ip, |
1338 | - password)) |
1339 | - finally: |
1340 | - cursor.close() |
1341 | - |
1342 | - def create_admin_grant(self, db_user, remote_ip, password): |
1343 | - cursor = self.connection.cursor() |
1344 | - try: |
1345 | - cursor.execute("GRANT ALL PRIVILEGES ON *.* TO '{}'@'{}' " |
1346 | - "IDENTIFIED BY '{}'".format(db_user, |
1347 | - remote_ip, |
1348 | - password)) |
1349 | - finally: |
1350 | - cursor.close() |
1351 | - |
1352 | - def cleanup_grant(self, db_user, remote_ip): |
1353 | - cursor = self.connection.cursor() |
1354 | - try: |
1355 | - cursor.execute("DROP FROM mysql.user WHERE user='{}' " |
1356 | - "AND HOST='{}'".format(db_user, |
1357 | - remote_ip)) |
1358 | - finally: |
1359 | - cursor.close() |
1360 | - |
1361 | - def execute(self, sql): |
1362 | - """Execute arbitary SQL against the database.""" |
1363 | - cursor = self.connection.cursor() |
1364 | - try: |
1365 | - cursor.execute(sql) |
1366 | - finally: |
1367 | - cursor.close() |
1368 | - |
1369 | - def migrate_passwords_to_peer_relation(self, excludes=None): |
1370 | - """Migrate any passwords storage on disk to cluster peer relation.""" |
1371 | - dirname = os.path.dirname(self.root_passwd_file_template) |
1372 | - path = os.path.join(dirname, '*.passwd') |
1373 | - for f in glob.glob(path): |
1374 | - if excludes and f in excludes: |
1375 | - log("Excluding %s from peer migration" % (f), level=DEBUG) |
1376 | - continue |
1377 | - |
1378 | - key = os.path.basename(f) |
1379 | - with open(f, 'r') as passwd: |
1380 | - _value = passwd.read().strip() |
1381 | - |
1382 | - try: |
1383 | - peer_store(key, _value) |
1384 | - |
1385 | - if self.delete_ondisk_passwd_file: |
1386 | - os.unlink(f) |
1387 | - except ValueError: |
1388 | - # NOTE cluster relation not yet ready - skip for now |
1389 | - pass |
1390 | - |
1391 | - def get_mysql_password_on_disk(self, username=None, password=None): |
1392 | - """Retrieve, generate or store a mysql password for the provided |
1393 | - username on disk.""" |
1394 | - if username: |
1395 | - template = self.user_passwd_file_template |
1396 | - passwd_file = template.format(username) |
1397 | - else: |
1398 | - passwd_file = self.root_passwd_file_template |
1399 | - |
1400 | - _password = None |
1401 | - if os.path.exists(passwd_file): |
1402 | - log("Using existing password file '%s'" % passwd_file, level=DEBUG) |
1403 | - with open(passwd_file, 'r') as passwd: |
1404 | - _password = passwd.read().strip() |
1405 | - else: |
1406 | - log("Generating new password file '%s'" % passwd_file, level=DEBUG) |
1407 | - if not os.path.isdir(os.path.dirname(passwd_file)): |
1408 | - # NOTE: need to ensure this is not mysql root dir (which needs |
1409 | - # to be mysql readable) |
1410 | - mkdir(os.path.dirname(passwd_file), owner='root', group='root', |
1411 | - perms=0o770) |
1412 | - # Force permissions - for some reason the chmod in makedirs |
1413 | - # fails |
1414 | - os.chmod(os.path.dirname(passwd_file), 0o770) |
1415 | - |
1416 | - _password = password or pwgen(length=32) |
1417 | - write_file(passwd_file, _password, owner='root', group='root', |
1418 | - perms=0o660) |
1419 | - |
1420 | - return _password |
1421 | - |
1422 | - def passwd_keys(self, username): |
1423 | - """Generator to return keys used to store passwords in peer store. |
1424 | - |
1425 | - NOTE: we support both legacy and new format to support mysql |
1426 | - charm prior to refactor. This is necessary to avoid LP 1451890. |
1427 | - """ |
1428 | - keys = [] |
1429 | - if username == 'mysql': |
1430 | - log("Bad username '%s'" % (username), level=WARNING) |
1431 | - |
1432 | - if username: |
1433 | - # IMPORTANT: *newer* format must be returned first |
1434 | - keys.append('mysql-%s.passwd' % (username)) |
1435 | - keys.append('%s.passwd' % (username)) |
1436 | - else: |
1437 | - keys.append('mysql.passwd') |
1438 | - |
1439 | - for key in keys: |
1440 | - yield key |
1441 | - |
1442 | - def get_mysql_password(self, username=None, password=None): |
1443 | - """Retrieve, generate or store a mysql password for the provided |
1444 | - username using peer relation cluster.""" |
1445 | - excludes = [] |
1446 | - |
1447 | - # First check peer relation. |
1448 | - try: |
1449 | - for key in self.passwd_keys(username): |
1450 | - _password = peer_retrieve(key) |
1451 | - if _password: |
1452 | - break |
1453 | - |
1454 | - # If root password available don't update peer relation from local |
1455 | - if _password and not username: |
1456 | - excludes.append(self.root_passwd_file_template) |
1457 | - |
1458 | - except ValueError: |
1459 | - # cluster relation is not yet started; use on-disk |
1460 | - _password = None |
1461 | - |
1462 | - # If none available, generate new one |
1463 | - if not _password: |
1464 | - _password = self.get_mysql_password_on_disk(username, password) |
1465 | - |
1466 | - # Put on wire if required |
1467 | - if self.migrate_passwd_to_peer_relation: |
1468 | - self.migrate_passwords_to_peer_relation(excludes=excludes) |
1469 | - |
1470 | - return _password |
1471 | - |
1472 | - def get_mysql_root_password(self, password=None): |
1473 | - """Retrieve or generate mysql root password for service units.""" |
1474 | - return self.get_mysql_password(username=None, password=password) |
1475 | - |
1476 | - def normalize_address(self, hostname): |
1477 | - """Ensure that address returned is an IP address (i.e. not fqdn)""" |
1478 | - if config_get('prefer-ipv6'): |
1479 | - # TODO: add support for ipv6 dns |
1480 | - return hostname |
1481 | - |
1482 | - if hostname != unit_get('private-address'): |
1483 | - return get_host_ip(hostname, fallback=hostname) |
1484 | - |
1485 | - # Otherwise assume localhost |
1486 | - return '127.0.0.1' |
1487 | - |
1488 | - def get_allowed_units(self, database, username, relation_id=None): |
1489 | - """Get list of units with access grants for database with username. |
1490 | - |
1491 | - This is typically used to provide shared-db relations with a list of |
1492 | - which units have been granted access to the given database. |
1493 | - """ |
1494 | - self.connect(password=self.get_mysql_root_password()) |
1495 | - allowed_units = set() |
1496 | - for unit in related_units(relation_id): |
1497 | - settings = relation_get(rid=relation_id, unit=unit) |
1498 | - # First check for setting with prefix, then without |
1499 | - for attr in ["%s_hostname" % (database), 'hostname']: |
1500 | - hosts = settings.get(attr, None) |
1501 | - if hosts: |
1502 | - break |
1503 | - |
1504 | - if hosts: |
1505 | - # hostname can be json-encoded list of hostnames |
1506 | - try: |
1507 | - hosts = json.loads(hosts) |
1508 | - except ValueError: |
1509 | - hosts = [hosts] |
1510 | - else: |
1511 | - hosts = [settings['private-address']] |
1512 | - |
1513 | - if hosts: |
1514 | - for host in hosts: |
1515 | - host = self.normalize_address(host) |
1516 | - if self.grant_exists(database, username, host): |
1517 | - log("Grant exists for host '%s' on db '%s'" % |
1518 | - (host, database), level=DEBUG) |
1519 | - if unit not in allowed_units: |
1520 | - allowed_units.add(unit) |
1521 | - else: |
1522 | - log("Grant does NOT exist for host '%s' on db '%s'" % |
1523 | - (host, database), level=DEBUG) |
1524 | - else: |
1525 | - log("No hosts found for grant check", level=INFO) |
1526 | - |
1527 | - return allowed_units |
1528 | - |
1529 | - def configure_db(self, hostname, database, username, admin=False): |
1530 | - """Configure access to database for username from hostname.""" |
1531 | - self.connect(password=self.get_mysql_root_password()) |
1532 | - if not self.database_exists(database): |
1533 | - self.create_database(database) |
1534 | - |
1535 | - remote_ip = self.normalize_address(hostname) |
1536 | - password = self.get_mysql_password(username) |
1537 | - if not self.grant_exists(database, username, remote_ip): |
1538 | - if not admin: |
1539 | - self.create_grant(database, username, remote_ip, password) |
1540 | - else: |
1541 | - self.create_admin_grant(username, remote_ip, password) |
1542 | - |
1543 | - return password |
1544 | - |
1545 | - |
1546 | -class PerconaClusterHelper(object): |
1547 | - |
1548 | - # Going for the biggest page size to avoid wasted bytes. |
1549 | - # InnoDB page size is 16MB |
1550 | - |
1551 | - DEFAULT_PAGE_SIZE = 16 * 1024 * 1024 |
1552 | - DEFAULT_INNODB_BUFFER_FACTOR = 0.50 |
1553 | - |
1554 | - def human_to_bytes(self, human): |
1555 | - """Convert human readable configuration options to bytes.""" |
1556 | - num_re = re.compile('^[0-9]+$') |
1557 | - if num_re.match(human): |
1558 | - return human |
1559 | - |
1560 | - factors = { |
1561 | - 'K': 1024, |
1562 | - 'M': 1048576, |
1563 | - 'G': 1073741824, |
1564 | - 'T': 1099511627776 |
1565 | - } |
1566 | - modifier = human[-1] |
1567 | - if modifier in factors: |
1568 | - return int(human[:-1]) * factors[modifier] |
1569 | - |
1570 | - if modifier == '%': |
1571 | - total_ram = self.human_to_bytes(self.get_mem_total()) |
1572 | - if self.is_32bit_system() and total_ram > self.sys_mem_limit(): |
1573 | - total_ram = self.sys_mem_limit() |
1574 | - factor = int(human[:-1]) * 0.01 |
1575 | - pctram = total_ram * factor |
1576 | - return int(pctram - (pctram % self.DEFAULT_PAGE_SIZE)) |
1577 | - |
1578 | - raise ValueError("Can only convert K,M,G, or T") |
1579 | - |
1580 | - def is_32bit_system(self): |
1581 | - """Determine whether system is 32 or 64 bit.""" |
1582 | - try: |
1583 | - return sys.maxsize < 2 ** 32 |
1584 | - except OverflowError: |
1585 | - return False |
1586 | - |
1587 | - def sys_mem_limit(self): |
1588 | - """Determine the default memory limit for the current service unit.""" |
1589 | - if platform.machine() in ['armv7l']: |
1590 | - _mem_limit = self.human_to_bytes('2700M') # experimentally determined |
1591 | - else: |
1592 | - # Limit for x86 based 32bit systems |
1593 | - _mem_limit = self.human_to_bytes('4G') |
1594 | - |
1595 | - return _mem_limit |
1596 | - |
1597 | - def get_mem_total(self): |
1598 | - """Calculate the total memory in the current service unit.""" |
1599 | - with open('/proc/meminfo') as meminfo_file: |
1600 | - for line in meminfo_file: |
1601 | - key, mem = line.split(':', 2) |
1602 | - if key == 'MemTotal': |
1603 | - mtot, modifier = mem.strip().split(' ') |
1604 | - return '%s%s' % (mtot, modifier[0].upper()) |
1605 | - |
1606 | - def parse_config(self): |
1607 | - """Parse charm configuration and calculate values for config files.""" |
1608 | - config = config_get() |
1609 | - mysql_config = {} |
1610 | - if 'max-connections' in config: |
1611 | - mysql_config['max_connections'] = config['max-connections'] |
1612 | - |
1613 | - if 'wait-timeout' in config: |
1614 | - mysql_config['wait_timeout'] = config['wait-timeout'] |
1615 | - |
1616 | - if 'innodb-flush-log-at-trx-commit' in config: |
1617 | - mysql_config['innodb_flush_log_at_trx_commit'] = config['innodb-flush-log-at-trx-commit'] |
1618 | - |
1619 | - # Set a sane default key_buffer size |
1620 | - mysql_config['key_buffer'] = self.human_to_bytes('32M') |
1621 | - total_memory = self.human_to_bytes(self.get_mem_total()) |
1622 | - |
1623 | - dataset_bytes = config.get('dataset-size', None) |
1624 | - innodb_buffer_pool_size = config.get('innodb-buffer-pool-size', None) |
1625 | - |
1626 | - if innodb_buffer_pool_size: |
1627 | - innodb_buffer_pool_size = self.human_to_bytes( |
1628 | - innodb_buffer_pool_size) |
1629 | - elif dataset_bytes: |
1630 | - log("Option 'dataset-size' has been deprecated, please use" |
1631 | - "innodb_buffer_pool_size option instead", level="WARN") |
1632 | - innodb_buffer_pool_size = self.human_to_bytes( |
1633 | - dataset_bytes) |
1634 | - else: |
1635 | - innodb_buffer_pool_size = int( |
1636 | - total_memory * self.DEFAULT_INNODB_BUFFER_FACTOR) |
1637 | - |
1638 | - if innodb_buffer_pool_size > total_memory: |
1639 | - log("innodb_buffer_pool_size; {} is greater than system available memory:{}".format( |
1640 | - innodb_buffer_pool_size, |
1641 | - total_memory), level='WARN') |
1642 | - |
1643 | - mysql_config['innodb_buffer_pool_size'] = innodb_buffer_pool_size |
1644 | - return mysql_config |
1645 | |
1646 | === removed directory 'hooks/charmhelpers/contrib/hardening' |
1647 | === removed file 'hooks/charmhelpers/contrib/hardening/__init__.py' |
1648 | --- hooks/charmhelpers/contrib/hardening/__init__.py 2016-04-22 08:15:33 +0000 |
1649 | +++ hooks/charmhelpers/contrib/hardening/__init__.py 1970-01-01 00:00:00 +0000 |
1650 | @@ -1,15 +0,0 @@ |
1651 | -# Copyright 2016 Canonical Limited. |
1652 | -# |
1653 | -# This file is part of charm-helpers. |
1654 | -# |
1655 | -# charm-helpers is free software: you can redistribute it and/or modify |
1656 | -# it under the terms of the GNU Lesser General Public License version 3 as |
1657 | -# published by the Free Software Foundation. |
1658 | -# |
1659 | -# charm-helpers is distributed in the hope that it will be useful, |
1660 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
1661 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
1662 | -# GNU Lesser General Public License for more details. |
1663 | -# |
1664 | -# You should have received a copy of the GNU Lesser General Public License |
1665 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
1666 | |
1667 | === removed directory 'hooks/charmhelpers/contrib/hardening/apache' |
1668 | === removed file 'hooks/charmhelpers/contrib/hardening/apache/__init__.py' |
1669 | --- hooks/charmhelpers/contrib/hardening/apache/__init__.py 2016-04-22 08:15:33 +0000 |
1670 | +++ hooks/charmhelpers/contrib/hardening/apache/__init__.py 1970-01-01 00:00:00 +0000 |
1671 | @@ -1,19 +0,0 @@ |
1672 | -# Copyright 2016 Canonical Limited. |
1673 | -# |
1674 | -# This file is part of charm-helpers. |
1675 | -# |
1676 | -# charm-helpers is free software: you can redistribute it and/or modify |
1677 | -# it under the terms of the GNU Lesser General Public License version 3 as |
1678 | -# published by the Free Software Foundation. |
1679 | -# |
1680 | -# charm-helpers is distributed in the hope that it will be useful, |
1681 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
1682 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
1683 | -# GNU Lesser General Public License for more details. |
1684 | -# |
1685 | -# You should have received a copy of the GNU Lesser General Public License |
1686 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
1687 | - |
1688 | -from os import path |
1689 | - |
1690 | -TEMPLATES_DIR = path.join(path.dirname(__file__), 'templates') |
1691 | |
1692 | === removed directory 'hooks/charmhelpers/contrib/hardening/apache/checks' |
1693 | === removed file 'hooks/charmhelpers/contrib/hardening/apache/checks/__init__.py' |
1694 | --- hooks/charmhelpers/contrib/hardening/apache/checks/__init__.py 2016-04-22 08:15:33 +0000 |
1695 | +++ hooks/charmhelpers/contrib/hardening/apache/checks/__init__.py 1970-01-01 00:00:00 +0000 |
1696 | @@ -1,31 +0,0 @@ |
1697 | -# Copyright 2016 Canonical Limited. |
1698 | -# |
1699 | -# This file is part of charm-helpers. |
1700 | -# |
1701 | -# charm-helpers is free software: you can redistribute it and/or modify |
1702 | -# it under the terms of the GNU Lesser General Public License version 3 as |
1703 | -# published by the Free Software Foundation. |
1704 | -# |
1705 | -# charm-helpers is distributed in the hope that it will be useful, |
1706 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
1707 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
1708 | -# GNU Lesser General Public License for more details. |
1709 | -# |
1710 | -# You should have received a copy of the GNU Lesser General Public License |
1711 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
1712 | - |
1713 | -from charmhelpers.core.hookenv import ( |
1714 | - log, |
1715 | - DEBUG, |
1716 | -) |
1717 | -from charmhelpers.contrib.hardening.apache.checks import config |
1718 | - |
1719 | - |
1720 | -def run_apache_checks(): |
1721 | - log("Starting Apache hardening checks.", level=DEBUG) |
1722 | - checks = config.get_audits() |
1723 | - for check in checks: |
1724 | - log("Running '%s' check" % (check.__class__.__name__), level=DEBUG) |
1725 | - check.ensure_compliance() |
1726 | - |
1727 | - log("Apache hardening checks complete.", level=DEBUG) |
1728 | |
1729 | === removed file 'hooks/charmhelpers/contrib/hardening/apache/checks/config.py' |
1730 | --- hooks/charmhelpers/contrib/hardening/apache/checks/config.py 2016-04-22 08:15:33 +0000 |
1731 | +++ hooks/charmhelpers/contrib/hardening/apache/checks/config.py 1970-01-01 00:00:00 +0000 |
1732 | @@ -1,100 +0,0 @@ |
1733 | -# Copyright 2016 Canonical Limited. |
1734 | -# |
1735 | -# This file is part of charm-helpers. |
1736 | -# |
1737 | -# charm-helpers is free software: you can redistribute it and/or modify |
1738 | -# it under the terms of the GNU Lesser General Public License version 3 as |
1739 | -# published by the Free Software Foundation. |
1740 | -# |
1741 | -# charm-helpers is distributed in the hope that it will be useful, |
1742 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
1743 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
1744 | -# GNU Lesser General Public License for more details. |
1745 | -# |
1746 | -# You should have received a copy of the GNU Lesser General Public License |
1747 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
1748 | - |
1749 | -import os |
1750 | -import re |
1751 | -import subprocess |
1752 | - |
1753 | - |
1754 | -from charmhelpers.core.hookenv import ( |
1755 | - log, |
1756 | - INFO, |
1757 | -) |
1758 | -from charmhelpers.contrib.hardening.audits.file import ( |
1759 | - FilePermissionAudit, |
1760 | - DirectoryPermissionAudit, |
1761 | - NoReadWriteForOther, |
1762 | - TemplatedFile, |
1763 | -) |
1764 | -from charmhelpers.contrib.hardening.audits.apache import DisabledModuleAudit |
1765 | -from charmhelpers.contrib.hardening.apache import TEMPLATES_DIR |
1766 | -from charmhelpers.contrib.hardening import utils |
1767 | - |
1768 | - |
1769 | -def get_audits(): |
1770 | - """Get Apache hardening config audits. |
1771 | - |
1772 | - :returns: dictionary of audits |
1773 | - """ |
1774 | - if subprocess.call(['which', 'apache2'], stdout=subprocess.PIPE) != 0: |
1775 | - log("Apache server does not appear to be installed on this node - " |
1776 | - "skipping apache hardening", level=INFO) |
1777 | - return [] |
1778 | - |
1779 | - context = ApacheConfContext() |
1780 | - settings = utils.get_settings('apache') |
1781 | - audits = [ |
1782 | - FilePermissionAudit(paths='/etc/apache2/apache2.conf', user='root', |
1783 | - group='root', mode=0o0640), |
1784 | - |
1785 | - TemplatedFile(os.path.join(settings['common']['apache_dir'], |
1786 | - 'mods-available/alias.conf'), |
1787 | - context, |
1788 | - TEMPLATES_DIR, |
1789 | - mode=0o0755, |
1790 | - user='root', |
1791 | - service_actions=[{'service': 'apache2', |
1792 | - 'actions': ['restart']}]), |
1793 | - |
1794 | - TemplatedFile(os.path.join(settings['common']['apache_dir'], |
1795 | - 'conf-enabled/hardening.conf'), |
1796 | - context, |
1797 | - TEMPLATES_DIR, |
1798 | - mode=0o0640, |
1799 | - user='root', |
1800 | - service_actions=[{'service': 'apache2', |
1801 | - 'actions': ['restart']}]), |
1802 | - |
1803 | - DirectoryPermissionAudit(settings['common']['apache_dir'], |
1804 | - user='root', |
1805 | - group='root', |
1806 | - mode=0o640), |
1807 | - |
1808 | - DisabledModuleAudit(settings['hardening']['modules_to_disable']), |
1809 | - |
1810 | - NoReadWriteForOther(settings['common']['apache_dir']), |
1811 | - ] |
1812 | - |
1813 | - return audits |
1814 | - |
1815 | - |
1816 | -class ApacheConfContext(object): |
1817 | - """Defines the set of key/value pairs to set in a apache config file. |
1818 | - |
1819 | - This context, when called, will return a dictionary containing the |
1820 | - key/value pairs of setting to specify in the |
1821 | - /etc/apache/conf-enabled/hardening.conf file. |
1822 | - """ |
1823 | - def __call__(self): |
1824 | - settings = utils.get_settings('apache') |
1825 | - ctxt = settings['hardening'] |
1826 | - |
1827 | - out = subprocess.check_output(['apache2', '-v']) |
1828 | - ctxt['apache_version'] = re.search(r'.+version: Apache/(.+?)\s.+', |
1829 | - out).group(1) |
1830 | - ctxt['apache_icondir'] = '/usr/share/apache2/icons/' |
1831 | - ctxt['traceenable'] = settings['hardening']['traceenable'] |
1832 | - return ctxt |
1833 | |
1834 | === removed directory 'hooks/charmhelpers/contrib/hardening/audits' |
1835 | === removed file 'hooks/charmhelpers/contrib/hardening/audits/__init__.py' |
1836 | --- hooks/charmhelpers/contrib/hardening/audits/__init__.py 2016-04-22 08:15:33 +0000 |
1837 | +++ hooks/charmhelpers/contrib/hardening/audits/__init__.py 1970-01-01 00:00:00 +0000 |
1838 | @@ -1,63 +0,0 @@ |
1839 | -# Copyright 2016 Canonical Limited. |
1840 | -# |
1841 | -# This file is part of charm-helpers. |
1842 | -# |
1843 | -# charm-helpers is free software: you can redistribute it and/or modify |
1844 | -# it under the terms of the GNU Lesser General Public License version 3 as |
1845 | -# published by the Free Software Foundation. |
1846 | -# |
1847 | -# charm-helpers is distributed in the hope that it will be useful, |
1848 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
1849 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
1850 | -# GNU Lesser General Public License for more details. |
1851 | -# |
1852 | -# You should have received a copy of the GNU Lesser General Public License |
1853 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
1854 | - |
1855 | - |
1856 | -class BaseAudit(object): # NO-QA |
1857 | - """Base class for hardening checks. |
1858 | - |
1859 | - The lifecycle of a hardening check is to first check to see if the system |
1860 | - is in compliance for the specified check. If it is not in compliance, the |
1861 | - check method will return a value which will be supplied to the. |
1862 | - """ |
1863 | - def __init__(self, *args, **kwargs): |
1864 | - self.unless = kwargs.get('unless', None) |
1865 | - super(BaseAudit, self).__init__() |
1866 | - |
1867 | - def ensure_compliance(self): |
1868 | - """Checks to see if the current hardening check is in compliance or |
1869 | - not. |
1870 | - |
1871 | - If the check that is performed is not in compliance, then an exception |
1872 | - should be raised. |
1873 | - """ |
1874 | - pass |
1875 | - |
1876 | - def _take_action(self): |
1877 | - """Determines whether to perform the action or not. |
1878 | - |
1879 | - Checks whether or not an action should be taken. This is determined by |
1880 | - the truthy value for the unless parameter. If unless is a callback |
1881 | - method, it will be invoked with no parameters in order to determine |
1882 | - whether or not the action should be taken. Otherwise, the truthy value |
1883 | - of the unless attribute will determine if the action should be |
1884 | - performed. |
1885 | - """ |
1886 | - # Do the action if there isn't an unless override. |
1887 | - if self.unless is None: |
1888 | - return True |
1889 | - |
1890 | - # Invoke the callback if there is one. |
1891 | - if hasattr(self.unless, '__call__'): |
1892 | - results = self.unless() |
1893 | - if results: |
1894 | - return False |
1895 | - else: |
1896 | - return True |
1897 | - |
1898 | - if self.unless: |
1899 | - return False |
1900 | - else: |
1901 | - return True |
1902 | |
1903 | === removed file 'hooks/charmhelpers/contrib/hardening/audits/apache.py' |
1904 | --- hooks/charmhelpers/contrib/hardening/audits/apache.py 2016-04-22 08:15:33 +0000 |
1905 | +++ hooks/charmhelpers/contrib/hardening/audits/apache.py 1970-01-01 00:00:00 +0000 |
1906 | @@ -1,100 +0,0 @@ |
1907 | -# Copyright 2016 Canonical Limited. |
1908 | -# |
1909 | -# This file is part of charm-helpers. |
1910 | -# |
1911 | -# charm-helpers is free software: you can redistribute it and/or modify |
1912 | -# it under the terms of the GNU Lesser General Public License version 3 as |
1913 | -# published by the Free Software Foundation. |
1914 | -# |
1915 | -# charm-helpers is distributed in the hope that it will be useful, |
1916 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
1917 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
1918 | -# GNU Lesser General Public License for more details. |
1919 | -# |
1920 | -# You should have received a copy of the GNU Lesser General Public License |
1921 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
1922 | - |
1923 | -import re |
1924 | -import subprocess |
1925 | - |
1926 | -from six import string_types |
1927 | - |
1928 | -from charmhelpers.core.hookenv import ( |
1929 | - log, |
1930 | - INFO, |
1931 | - ERROR, |
1932 | -) |
1933 | - |
1934 | -from charmhelpers.contrib.hardening.audits import BaseAudit |
1935 | - |
1936 | - |
1937 | -class DisabledModuleAudit(BaseAudit): |
1938 | - """Audits Apache2 modules. |
1939 | - |
1940 | - Determines if the apache2 modules are enabled. If the modules are enabled |
1941 | - then they are removed in the ensure_compliance. |
1942 | - """ |
1943 | - def __init__(self, modules): |
1944 | - if modules is None: |
1945 | - self.modules = [] |
1946 | - elif isinstance(modules, string_types): |
1947 | - self.modules = [modules] |
1948 | - else: |
1949 | - self.modules = modules |
1950 | - |
1951 | - def ensure_compliance(self): |
1952 | - """Ensures that the modules are not loaded.""" |
1953 | - if not self.modules: |
1954 | - return |
1955 | - |
1956 | - try: |
1957 | - loaded_modules = self._get_loaded_modules() |
1958 | - non_compliant_modules = [] |
1959 | - for module in self.modules: |
1960 | - if module in loaded_modules: |
1961 | - log("Module '%s' is enabled but should not be." % |
1962 | - (module), level=INFO) |
1963 | - non_compliant_modules.append(module) |
1964 | - |
1965 | - if len(non_compliant_modules) == 0: |
1966 | - return |
1967 | - |
1968 | - for module in non_compliant_modules: |
1969 | - self._disable_module(module) |
1970 | - self._restart_apache() |
1971 | - except subprocess.CalledProcessError as e: |
1972 | - log('Error occurred auditing apache module compliance. ' |
1973 | - 'This may have been already reported. ' |
1974 | - 'Output is: %s' % e.output, level=ERROR) |
1975 | - |
1976 | - @staticmethod |
1977 | - def _get_loaded_modules(): |
1978 | - """Returns the modules which are enabled in Apache.""" |
1979 | - output = subprocess.check_output(['apache2ctl', '-M']) |
1980 | - modules = [] |
1981 | - for line in output.strip().split(): |
1982 | - # Each line of the enabled module output looks like: |
1983 | - # module_name (static|shared) |
1984 | - # Plus a header line at the top of the output which is stripped |
1985 | - # out by the regex. |
1986 | - matcher = re.search(r'^ (\S*)', line) |
1987 | - if matcher: |
1988 | - modules.append(matcher.group(1)) |
1989 | - return modules |
1990 | - |
1991 | - @staticmethod |
1992 | - def _disable_module(module): |
1993 | - """Disables the specified module in Apache.""" |
1994 | - try: |
1995 | - subprocess.check_call(['a2dismod', module]) |
1996 | - except subprocess.CalledProcessError as e: |
1997 | - # Note: catch error here to allow the attempt of disabling |
1998 | - # multiple modules in one go rather than failing after the |
1999 | - # first module fails. |
2000 | - log('Error occurred disabling module %s. ' |
2001 | - 'Output is: %s' % (module, e.output), level=ERROR) |
2002 | - |
2003 | - @staticmethod |
2004 | - def _restart_apache(): |
2005 | - """Restarts the apache process""" |
2006 | - subprocess.check_output(['service', 'apache2', 'restart']) |
2007 | |
2008 | === removed file 'hooks/charmhelpers/contrib/hardening/audits/apt.py' |
2009 | --- hooks/charmhelpers/contrib/hardening/audits/apt.py 2016-04-22 08:15:33 +0000 |
2010 | +++ hooks/charmhelpers/contrib/hardening/audits/apt.py 1970-01-01 00:00:00 +0000 |
2011 | @@ -1,105 +0,0 @@ |
2012 | -# Copyright 2016 Canonical Limited. |
2013 | -# |
2014 | -# This file is part of charm-helpers. |
2015 | -# |
2016 | -# charm-helpers is free software: you can redistribute it and/or modify |
2017 | -# it under the terms of the GNU Lesser General Public License version 3 as |
2018 | -# published by the Free Software Foundation. |
2019 | -# |
2020 | -# charm-helpers is distributed in the hope that it will be useful, |
2021 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
2022 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
2023 | -# GNU Lesser General Public License for more details. |
2024 | -# |
2025 | -# You should have received a copy of the GNU Lesser General Public License |
2026 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
2027 | - |
2028 | -from __future__ import absolute_import # required for external apt import |
2029 | -from apt import apt_pkg |
2030 | -from six import string_types |
2031 | - |
2032 | -from charmhelpers.fetch import ( |
2033 | - apt_cache, |
2034 | - apt_purge |
2035 | -) |
2036 | -from charmhelpers.core.hookenv import ( |
2037 | - log, |
2038 | - DEBUG, |
2039 | - WARNING, |
2040 | -) |
2041 | -from charmhelpers.contrib.hardening.audits import BaseAudit |
2042 | - |
2043 | - |
2044 | -class AptConfig(BaseAudit): |
2045 | - |
2046 | - def __init__(self, config, **kwargs): |
2047 | - self.config = config |
2048 | - |
2049 | - def verify_config(self): |
2050 | - apt_pkg.init() |
2051 | - for cfg in self.config: |
2052 | - value = apt_pkg.config.get(cfg['key'], cfg.get('default', '')) |
2053 | - if value and value != cfg['expected']: |
2054 | - log("APT config '%s' has unexpected value '%s' " |
2055 | - "(expected='%s')" % |
2056 | - (cfg['key'], value, cfg['expected']), level=WARNING) |
2057 | - |
2058 | - def ensure_compliance(self): |
2059 | - self.verify_config() |
2060 | - |
2061 | - |
2062 | -class RestrictedPackages(BaseAudit): |
2063 | - """Class used to audit restricted packages on the system.""" |
2064 | - |
2065 | - def __init__(self, pkgs, **kwargs): |
2066 | - super(RestrictedPackages, self).__init__(**kwargs) |
2067 | - if isinstance(pkgs, string_types) or not hasattr(pkgs, '__iter__'): |
2068 | - self.pkgs = [pkgs] |
2069 | - else: |
2070 | - self.pkgs = pkgs |
2071 | - |
2072 | - def ensure_compliance(self): |
2073 | - cache = apt_cache() |
2074 | - |
2075 | - for p in self.pkgs: |
2076 | - if p not in cache: |
2077 | - continue |
2078 | - |
2079 | - pkg = cache[p] |
2080 | - if not self.is_virtual_package(pkg): |
2081 | - if not pkg.current_ver: |
2082 | - log("Package '%s' is not installed." % pkg.name, |
2083 | - level=DEBUG) |
2084 | - continue |
2085 | - else: |
2086 | - log("Restricted package '%s' is installed" % pkg.name, |
2087 | - level=WARNING) |
2088 | - self.delete_package(cache, pkg) |
2089 | - else: |
2090 | - log("Checking restricted virtual package '%s' provides" % |
2091 | - pkg.name, level=DEBUG) |
2092 | - self.delete_package(cache, pkg) |
2093 | - |
2094 | - def delete_package(self, cache, pkg): |
2095 | - """Deletes the package from the system. |
2096 | - |
2097 | - Deletes the package form the system, properly handling virtual |
2098 | - packages. |
2099 | - |
2100 | - :param cache: the apt cache |
2101 | - :param pkg: the package to remove |
2102 | - """ |
2103 | - if self.is_virtual_package(pkg): |
2104 | - log("Package '%s' appears to be virtual - purging provides" % |
2105 | - pkg.name, level=DEBUG) |
2106 | - for _p in pkg.provides_list: |
2107 | - self.delete_package(cache, _p[2].parent_pkg) |
2108 | - elif not pkg.current_ver: |
2109 | - log("Package '%s' not installed" % pkg.name, level=DEBUG) |
2110 | - return |
2111 | - else: |
2112 | - log("Purging package '%s'" % pkg.name, level=DEBUG) |
2113 | - apt_purge(pkg.name) |
2114 | - |
2115 | - def is_virtual_package(self, pkg): |
2116 | - return pkg.has_provides and not pkg.has_versions |
2117 | |
2118 | === removed file 'hooks/charmhelpers/contrib/hardening/audits/file.py' |
2119 | --- hooks/charmhelpers/contrib/hardening/audits/file.py 2016-04-22 08:15:33 +0000 |
2120 | +++ hooks/charmhelpers/contrib/hardening/audits/file.py 1970-01-01 00:00:00 +0000 |
2121 | @@ -1,552 +0,0 @@ |
2122 | -# Copyright 2016 Canonical Limited. |
2123 | -# |
2124 | -# This file is part of charm-helpers. |
2125 | -# |
2126 | -# charm-helpers is free software: you can redistribute it and/or modify |
2127 | -# it under the terms of the GNU Lesser General Public License version 3 as |
2128 | -# published by the Free Software Foundation. |
2129 | -# |
2130 | -# charm-helpers is distributed in the hope that it will be useful, |
2131 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
2132 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
2133 | -# GNU Lesser General Public License for more details. |
2134 | -# |
2135 | -# You should have received a copy of the GNU Lesser General Public License |
2136 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
2137 | - |
2138 | -import grp |
2139 | -import os |
2140 | -import pwd |
2141 | -import re |
2142 | - |
2143 | -from subprocess import ( |
2144 | - CalledProcessError, |
2145 | - check_output, |
2146 | - check_call, |
2147 | -) |
2148 | -from traceback import format_exc |
2149 | -from six import string_types |
2150 | -from stat import ( |
2151 | - S_ISGID, |
2152 | - S_ISUID |
2153 | -) |
2154 | - |
2155 | -from charmhelpers.core.hookenv import ( |
2156 | - log, |
2157 | - DEBUG, |
2158 | - INFO, |
2159 | - WARNING, |
2160 | - ERROR, |
2161 | -) |
2162 | -from charmhelpers.core import unitdata |
2163 | -from charmhelpers.core.host import file_hash |
2164 | -from charmhelpers.contrib.hardening.audits import BaseAudit |
2165 | -from charmhelpers.contrib.hardening.templating import ( |
2166 | - get_template_path, |
2167 | - render_and_write, |
2168 | -) |
2169 | -from charmhelpers.contrib.hardening import utils |
2170 | - |
2171 | - |
2172 | -class BaseFileAudit(BaseAudit): |
2173 | - """Base class for file audits. |
2174 | - |
2175 | - Provides api stubs for compliance check flow that must be used by any class |
2176 | - that implemented this one. |
2177 | - """ |
2178 | - |
2179 | - def __init__(self, paths, always_comply=False, *args, **kwargs): |
2180 | - """ |
2181 | - :param paths: string path of list of paths of files we want to apply |
2182 | - compliance checks are criteria to. |
2183 | - :param always_comply: if true compliance criteria is always applied |
2184 | - else compliance is skipped for non-existent |
2185 | - paths. |
2186 | - """ |
2187 | - super(BaseFileAudit, self).__init__(*args, **kwargs) |
2188 | - self.always_comply = always_comply |
2189 | - if isinstance(paths, string_types) or not hasattr(paths, '__iter__'): |
2190 | - self.paths = [paths] |
2191 | - else: |
2192 | - self.paths = paths |
2193 | - |
2194 | - def ensure_compliance(self): |
2195 | - """Ensure that the all registered files comply to registered criteria. |
2196 | - """ |
2197 | - for p in self.paths: |
2198 | - if os.path.exists(p): |
2199 | - if self.is_compliant(p): |
2200 | - continue |
2201 | - |
2202 | - log('File %s is not in compliance.' % p, level=INFO) |
2203 | - else: |
2204 | - if not self.always_comply: |
2205 | - log("Non-existent path '%s' - skipping compliance check" |
2206 | - % (p), level=INFO) |
2207 | - continue |
2208 | - |
2209 | - if self._take_action(): |
2210 | - log("Applying compliance criteria to '%s'" % (p), level=INFO) |
2211 | - self.comply(p) |
2212 | - |
2213 | - def is_compliant(self, path): |
2214 | - """Audits the path to see if it is compliance. |
2215 | - |
2216 | - :param path: the path to the file that should be checked. |
2217 | - """ |
2218 | - raise NotImplementedError |
2219 | - |
2220 | - def comply(self, path): |
2221 | - """Enforces the compliance of a path. |
2222 | - |
2223 | - :param path: the path to the file that should be enforced. |
2224 | - """ |
2225 | - raise NotImplementedError |
2226 | - |
2227 | - @classmethod |
2228 | - def _get_stat(cls, path): |
2229 | - """Returns the Posix st_stat information for the specified file path. |
2230 | - |
2231 | - :param path: the path to get the st_stat information for. |
2232 | - :returns: an st_stat object for the path or None if the path doesn't |
2233 | - exist. |
2234 | - """ |
2235 | - return os.stat(path) |
2236 | - |
2237 | - |
2238 | -class FilePermissionAudit(BaseFileAudit): |
2239 | - """Implements an audit for file permissions and ownership for a user. |
2240 | - |
2241 | - This class implements functionality that ensures that a specific user/group |
2242 | - will own the file(s) specified and that the permissions specified are |
2243 | - applied properly to the file. |
2244 | - """ |
2245 | - def __init__(self, paths, user, group=None, mode=0o600, **kwargs): |
2246 | - self.user = user |
2247 | - self.group = group |
2248 | - self.mode = mode |
2249 | - super(FilePermissionAudit, self).__init__(paths, user, group, mode, |
2250 | - **kwargs) |
2251 | - |
2252 | - @property |
2253 | - def user(self): |
2254 | - return self._user |
2255 | - |
2256 | - @user.setter |
2257 | - def user(self, name): |
2258 | - try: |
2259 | - user = pwd.getpwnam(name) |
2260 | - except KeyError: |
2261 | - log('Unknown user %s' % name, level=ERROR) |
2262 | - user = None |
2263 | - self._user = user |
2264 | - |
2265 | - @property |
2266 | - def group(self): |
2267 | - return self._group |
2268 | - |
2269 | - @group.setter |
2270 | - def group(self, name): |
2271 | - try: |
2272 | - group = None |
2273 | - if name: |
2274 | - group = grp.getgrnam(name) |
2275 | - else: |
2276 | - group = grp.getgrgid(self.user.pw_gid) |
2277 | - except KeyError: |
2278 | - log('Unknown group %s' % name, level=ERROR) |
2279 | - self._group = group |
2280 | - |
2281 | - def is_compliant(self, path): |
2282 | - """Checks if the path is in compliance. |
2283 | - |
2284 | - Used to determine if the path specified meets the necessary |
2285 | - requirements to be in compliance with the check itself. |
2286 | - |
2287 | - :param path: the file path to check |
2288 | - :returns: True if the path is compliant, False otherwise. |
2289 | - """ |
2290 | - stat = self._get_stat(path) |
2291 | - user = self.user |
2292 | - group = self.group |
2293 | - |
2294 | - compliant = True |
2295 | - if stat.st_uid != user.pw_uid or stat.st_gid != group.gr_gid: |
2296 | - log('File %s is not owned by %s:%s.' % (path, user.pw_name, |
2297 | - group.gr_name), |
2298 | - level=INFO) |
2299 | - compliant = False |
2300 | - |
2301 | - # POSIX refers to the st_mode bits as corresponding to both the |
2302 | - # file type and file permission bits, where the least significant 12 |
2303 | - # bits (o7777) are the suid (11), sgid (10), sticky bits (9), and the |
2304 | - # file permission bits (8-0) |
2305 | - perms = stat.st_mode & 0o7777 |
2306 | - if perms != self.mode: |
2307 | - log('File %s has incorrect permissions, currently set to %s' % |
2308 | - (path, oct(stat.st_mode & 0o7777)), level=INFO) |
2309 | - compliant = False |
2310 | - |
2311 | - return compliant |
2312 | - |
2313 | - def comply(self, path): |
2314 | - """Issues a chown and chmod to the file paths specified.""" |
2315 | - utils.ensure_permissions(path, self.user.pw_name, self.group.gr_name, |
2316 | - self.mode) |
2317 | - |
2318 | - |
2319 | -class DirectoryPermissionAudit(FilePermissionAudit): |
2320 | - """Performs a permission check for the specified directory path.""" |
2321 | - |
2322 | - def __init__(self, paths, user, group=None, mode=0o600, |
2323 | - recursive=True, **kwargs): |
2324 | - super(DirectoryPermissionAudit, self).__init__(paths, user, group, |
2325 | - mode, **kwargs) |
2326 | - self.recursive = recursive |
2327 | - |
2328 | - def is_compliant(self, path): |
2329 | - """Checks if the directory is compliant. |
2330 | - |
2331 | - Used to determine if the path specified and all of its children |
2332 | - directories are in compliance with the check itself. |
2333 | - |
2334 | - :param path: the directory path to check |
2335 | - :returns: True if the directory tree is compliant, otherwise False. |
2336 | - """ |
2337 | - if not os.path.isdir(path): |
2338 | - log('Path specified %s is not a directory.' % path, level=ERROR) |
2339 | - raise ValueError("%s is not a directory." % path) |
2340 | - |
2341 | - if not self.recursive: |
2342 | - return super(DirectoryPermissionAudit, self).is_compliant(path) |
2343 | - |
2344 | - compliant = True |
2345 | - for root, dirs, _ in os.walk(path): |
2346 | - if len(dirs) > 0: |
2347 | - continue |
2348 | - |
2349 | - if not super(DirectoryPermissionAudit, self).is_compliant(root): |
2350 | - compliant = False |
2351 | - continue |
2352 | - |
2353 | - return compliant |
2354 | - |
2355 | - def comply(self, path): |
2356 | - for root, dirs, _ in os.walk(path): |
2357 | - if len(dirs) > 0: |
2358 | - super(DirectoryPermissionAudit, self).comply(root) |
2359 | - |
2360 | - |
2361 | -class ReadOnly(BaseFileAudit): |
2362 | - """Audits that files and folders are read only.""" |
2363 | - def __init__(self, paths, *args, **kwargs): |
2364 | - super(ReadOnly, self).__init__(paths=paths, *args, **kwargs) |
2365 | - |
2366 | - def is_compliant(self, path): |
2367 | - try: |
2368 | - output = check_output(['find', path, '-perm', '-go+w', |
2369 | - '-type', 'f']).strip() |
2370 | - |
2371 | - # The find above will find any files which have permission sets |
2372 | - # which allow too broad of write access. As such, the path is |
2373 | - # compliant if there is no output. |
2374 | - if output: |
2375 | - return False |
2376 | - |
2377 | - return True |
2378 | - except CalledProcessError as e: |
2379 | - log('Error occurred checking finding writable files for %s. ' |
2380 | - 'Error information is: command %s failed with returncode ' |
2381 | - '%d and output %s.\n%s' % (path, e.cmd, e.returncode, e.output, |
2382 | - format_exc(e)), level=ERROR) |
2383 | - return False |
2384 | - |
2385 | - def comply(self, path): |
2386 | - try: |
2387 | - check_output(['chmod', 'go-w', '-R', path]) |
2388 | - except CalledProcessError as e: |
2389 | - log('Error occurred removing writeable permissions for %s. ' |
2390 | - 'Error information is: command %s failed with returncode ' |
2391 | - '%d and output %s.\n%s' % (path, e.cmd, e.returncode, e.output, |
2392 | - format_exc(e)), level=ERROR) |
2393 | - |
2394 | - |
2395 | -class NoReadWriteForOther(BaseFileAudit): |
2396 | - """Ensures that the files found under the base path are readable or |
2397 | - writable by anyone other than the owner or the group. |
2398 | - """ |
2399 | - def __init__(self, paths): |
2400 | - super(NoReadWriteForOther, self).__init__(paths) |
2401 | - |
2402 | - def is_compliant(self, path): |
2403 | - try: |
2404 | - cmd = ['find', path, '-perm', '-o+r', '-type', 'f', '-o', |
2405 | - '-perm', '-o+w', '-type', 'f'] |
2406 | - output = check_output(cmd).strip() |
2407 | - |
2408 | - # The find above here will find any files which have read or |
2409 | - # write permissions for other, meaning there is too broad of access |
2410 | - # to read/write the file. As such, the path is compliant if there's |
2411 | - # no output. |
2412 | - if output: |
2413 | - return False |
2414 | - |
2415 | - return True |
2416 | - except CalledProcessError as e: |
2417 | - log('Error occurred while finding files which are readable or ' |
2418 | - 'writable to the world in %s. ' |
2419 | - 'Command output is: %s.' % (path, e.output), level=ERROR) |
2420 | - |
2421 | - def comply(self, path): |
2422 | - try: |
2423 | - check_output(['chmod', '-R', 'o-rw', path]) |
2424 | - except CalledProcessError as e: |
2425 | - log('Error occurred attempting to change modes of files under ' |
2426 | - 'path %s. Output of command is: %s' % (path, e.output)) |
2427 | - |
2428 | - |
2429 | -class NoSUIDSGIDAudit(BaseFileAudit): |
2430 | - """Audits that specified files do not have SUID/SGID bits set.""" |
2431 | - def __init__(self, paths, *args, **kwargs): |
2432 | - super(NoSUIDSGIDAudit, self).__init__(paths=paths, *args, **kwargs) |
2433 | - |
2434 | - def is_compliant(self, path): |
2435 | - stat = self._get_stat(path) |
2436 | - if (stat.st_mode & (S_ISGID | S_ISUID)) != 0: |
2437 | - return False |
2438 | - |
2439 | - return True |
2440 | - |
2441 | - def comply(self, path): |
2442 | - try: |
2443 | - log('Removing suid/sgid from %s.' % path, level=DEBUG) |
2444 | - check_output(['chmod', '-s', path]) |
2445 | - except CalledProcessError as e: |
2446 | - log('Error occurred removing suid/sgid from %s.' |
2447 | - 'Error information is: command %s failed with returncode ' |
2448 | - '%d and output %s.\n%s' % (path, e.cmd, e.returncode, e.output, |
2449 | - format_exc(e)), level=ERROR) |
2450 | - |
2451 | - |
2452 | -class TemplatedFile(BaseFileAudit): |
2453 | - """The TemplatedFileAudit audits the contents of a templated file. |
2454 | - |
2455 | - This audit renders a file from a template, sets the appropriate file |
2456 | - permissions, then generates a hashsum with which to check the content |
2457 | - changed. |
2458 | - """ |
2459 | - def __init__(self, path, context, template_dir, mode, user='root', |
2460 | - group='root', service_actions=None, **kwargs): |
2461 | - self.context = context |
2462 | - self.user = user |
2463 | - self.group = group |
2464 | - self.mode = mode |
2465 | - self.template_dir = template_dir |
2466 | - self.service_actions = service_actions |
2467 | - super(TemplatedFile, self).__init__(paths=path, always_comply=True, |
2468 | - **kwargs) |
2469 | - |
2470 | - def is_compliant(self, path): |
2471 | - """Determines if the templated file is compliant. |
2472 | - |
2473 | - A templated file is only compliant if it has not changed (as |
2474 | - determined by its sha256 hashsum) AND its file permissions are set |
2475 | - appropriately. |
2476 | - |
2477 | - :param path: the path to check compliance. |
2478 | - """ |
2479 | - same_templates = self.templates_match(path) |
2480 | - same_content = self.contents_match(path) |
2481 | - same_permissions = self.permissions_match(path) |
2482 | - |
2483 | - if same_content and same_permissions and same_templates: |
2484 | - return True |
2485 | - |
2486 | - return False |
2487 | - |
2488 | - def run_service_actions(self): |
2489 | - """Run any actions on services requested.""" |
2490 | - if not self.service_actions: |
2491 | - return |
2492 | - |
2493 | - for svc_action in self.service_actions: |
2494 | - name = svc_action['service'] |
2495 | - actions = svc_action['actions'] |
2496 | - log("Running service '%s' actions '%s'" % (name, actions), |
2497 | - level=DEBUG) |
2498 | - for action in actions: |
2499 | - cmd = ['service', name, action] |
2500 | - try: |
2501 | - check_call(cmd) |
2502 | - except CalledProcessError as exc: |
2503 | - log("Service name='%s' action='%s' failed - %s" % |
2504 | - (name, action, exc), level=WARNING) |
2505 | - |
2506 | - def comply(self, path): |
2507 | - """Ensures the contents and the permissions of the file. |
2508 | - |
2509 | - :param path: the path to correct |
2510 | - """ |
2511 | - dirname = os.path.dirname(path) |
2512 | - if not os.path.exists(dirname): |
2513 | - os.makedirs(dirname) |
2514 | - |
2515 | - self.pre_write() |
2516 | - render_and_write(self.template_dir, path, self.context()) |
2517 | - utils.ensure_permissions(path, self.user, self.group, self.mode) |
2518 | - self.run_service_actions() |
2519 | - self.save_checksum(path) |
2520 | - self.post_write() |
2521 | - |
2522 | - def pre_write(self): |
2523 | - """Invoked prior to writing the template.""" |
2524 | - pass |
2525 | - |
2526 | - def post_write(self): |
2527 | - """Invoked after writing the template.""" |
2528 | - pass |
2529 | - |
2530 | - def templates_match(self, path): |
2531 | - """Determines if the template files are the same. |
2532 | - |
2533 | - The template file equality is determined by the hashsum of the |
2534 | - template files themselves. If there is no hashsum, then the content |
2535 | - cannot be sure to be the same so treat it as if they changed. |
2536 | - Otherwise, return whether or not the hashsums are the same. |
2537 | - |
2538 | - :param path: the path to check |
2539 | - :returns: boolean |
2540 | - """ |
2541 | - template_path = get_template_path(self.template_dir, path) |
2542 | - key = 'hardening:template:%s' % template_path |
2543 | - template_checksum = file_hash(template_path) |
2544 | - kv = unitdata.kv() |
2545 | - stored_tmplt_checksum = kv.get(key) |
2546 | - if not stored_tmplt_checksum: |
2547 | - kv.set(key, template_checksum) |
2548 | - kv.flush() |
2549 | - log('Saved template checksum for %s.' % template_path, |
2550 | - level=DEBUG) |
2551 | - # Since we don't have a template checksum, then assume it doesn't |
2552 | - # match and return that the template is different. |
2553 | - return False |
2554 | - elif stored_tmplt_checksum != template_checksum: |
2555 | - kv.set(key, template_checksum) |
2556 | - kv.flush() |
2557 | - log('Updated template checksum for %s.' % template_path, |
2558 | - level=DEBUG) |
2559 | - return False |
2560 | - |
2561 | - # Here the template hasn't changed based upon the calculated |
2562 | - # checksum of the template and what was previously stored. |
2563 | - return True |
2564 | - |
2565 | - def contents_match(self, path): |
2566 | - """Determines if the file content is the same. |
2567 | - |
2568 | - This is determined by comparing hashsum of the file contents and |
2569 | - the saved hashsum. If there is no hashsum, then the content cannot |
2570 | - be sure to be the same so treat them as if they are not the same. |
2571 | - Otherwise, return True if the hashsums are the same, False if they |
2572 | - are not the same. |
2573 | - |
2574 | - :param path: the file to check. |
2575 | - """ |
2576 | - checksum = file_hash(path) |
2577 | - |
2578 | - kv = unitdata.kv() |
2579 | - stored_checksum = kv.get('hardening:%s' % path) |
2580 | - if not stored_checksum: |
2581 | - # If the checksum hasn't been generated, return False to ensure |
2582 | - # the file is written and the checksum stored. |
2583 | - log('Checksum for %s has not been calculated.' % path, level=DEBUG) |
2584 | - return False |
2585 | - elif stored_checksum != checksum: |
2586 | - log('Checksum mismatch for %s.' % path, level=DEBUG) |
2587 | - return False |
2588 | - |
2589 | - return True |
2590 | - |
2591 | - def permissions_match(self, path): |
2592 | - """Determines if the file owner and permissions match. |
2593 | - |
2594 | - :param path: the path to check. |
2595 | - """ |
2596 | - audit = FilePermissionAudit(path, self.user, self.group, self.mode) |
2597 | - return audit.is_compliant(path) |
2598 | - |
2599 | - def save_checksum(self, path): |
2600 | - """Calculates and saves the checksum for the path specified. |
2601 | - |
2602 | - :param path: the path of the file to save the checksum. |
2603 | - """ |
2604 | - checksum = file_hash(path) |
2605 | - kv = unitdata.kv() |
2606 | - kv.set('hardening:%s' % path, checksum) |
2607 | - kv.flush() |
2608 | - |
2609 | - |
2610 | -class DeletedFile(BaseFileAudit): |
2611 | - """Audit to ensure that a file is deleted.""" |
2612 | - def __init__(self, paths): |
2613 | - super(DeletedFile, self).__init__(paths) |
2614 | - |
2615 | - def is_compliant(self, path): |
2616 | - return not os.path.exists(path) |
2617 | - |
2618 | - def comply(self, path): |
2619 | - os.remove(path) |
2620 | - |
2621 | - |
2622 | -class FileContentAudit(BaseFileAudit): |
2623 | - """Audit the contents of a file.""" |
2624 | - def __init__(self, paths, cases, **kwargs): |
2625 | - # Cases we expect to pass |
2626 | - self.pass_cases = cases.get('pass', []) |
2627 | - # Cases we expect to fail |
2628 | - self.fail_cases = cases.get('fail', []) |
2629 | - super(FileContentAudit, self).__init__(paths, **kwargs) |
2630 | - |
2631 | - def is_compliant(self, path): |
2632 | - """ |
2633 | - Given a set of content matching cases i.e. tuple(regex, bool) where |
2634 | - bool value denotes whether or not regex is expected to match, check that |
2635 | - all cases match as expected with the contents of the file. Cases can be |
2636 | - expected to pass of fail. |
2637 | - |
2638 | - :param path: Path of file to check. |
2639 | - :returns: Boolean value representing whether or not all cases are |
2640 | - found to be compliant. |
2641 | - """ |
2642 | - log("Auditing contents of file '%s'" % (path), level=DEBUG) |
2643 | - with open(path, 'r') as fd: |
2644 | - contents = fd.read() |
2645 | - |
2646 | - matches = 0 |
2647 | - for pattern in self.pass_cases: |
2648 | - key = re.compile(pattern, flags=re.MULTILINE) |
2649 | - results = re.search(key, contents) |
2650 | - if results: |
2651 | - matches += 1 |
2652 | - else: |
2653 | - log("Pattern '%s' was expected to pass but instead it failed" |
2654 | - % (pattern), level=WARNING) |
2655 | - |
2656 | - for pattern in self.fail_cases: |
2657 | - key = re.compile(pattern, flags=re.MULTILINE) |
2658 | - results = re.search(key, contents) |
2659 | - if not results: |
2660 | - matches += 1 |
2661 | - else: |
2662 | - log("Pattern '%s' was expected to fail but instead it passed" |
2663 | - % (pattern), level=WARNING) |
2664 | - |
2665 | - total = len(self.pass_cases) + len(self.fail_cases) |
2666 | - log("Checked %s cases and %s passed" % (total, matches), level=DEBUG) |
2667 | - return matches == total |
2668 | - |
2669 | - def comply(self, *args, **kwargs): |
2670 | - """NOOP since we just issue warnings. This is to avoid the |
2671 | - NotImplememtedError. |
2672 | - """ |
2673 | - log("Not applying any compliance criteria, only checks.", level=INFO) |
2674 | |
2675 | === removed file 'hooks/charmhelpers/contrib/hardening/harden.py' |
2676 | --- hooks/charmhelpers/contrib/hardening/harden.py 2016-04-22 08:15:33 +0000 |
2677 | +++ hooks/charmhelpers/contrib/hardening/harden.py 1970-01-01 00:00:00 +0000 |
2678 | @@ -1,84 +0,0 @@ |
2679 | -# Copyright 2016 Canonical Limited. |
2680 | -# |
2681 | -# This file is part of charm-helpers. |
2682 | -# |
2683 | -# charm-helpers is free software: you can redistribute it and/or modify |
2684 | -# it under the terms of the GNU Lesser General Public License version 3 as |
2685 | -# published by the Free Software Foundation. |
2686 | -# |
2687 | -# charm-helpers is distributed in the hope that it will be useful, |
2688 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
2689 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
2690 | -# GNU Lesser General Public License for more details. |
2691 | -# |
2692 | -# You should have received a copy of the GNU Lesser General Public License |
2693 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
2694 | - |
2695 | -import six |
2696 | - |
2697 | -from collections import OrderedDict |
2698 | - |
2699 | -from charmhelpers.core.hookenv import ( |
2700 | - config, |
2701 | - log, |
2702 | - DEBUG, |
2703 | - WARNING, |
2704 | -) |
2705 | -from charmhelpers.contrib.hardening.host.checks import run_os_checks |
2706 | -from charmhelpers.contrib.hardening.ssh.checks import run_ssh_checks |
2707 | -from charmhelpers.contrib.hardening.mysql.checks import run_mysql_checks |
2708 | -from charmhelpers.contrib.hardening.apache.checks import run_apache_checks |
2709 | - |
2710 | - |
2711 | -def harden(overrides=None): |
2712 | - """Hardening decorator. |
2713 | - |
2714 | - This is the main entry point for running the hardening stack. In order to |
2715 | - run modules of the stack you must add this decorator to charm hook(s) and |
2716 | - ensure that your charm config.yaml contains the 'harden' option set to |
2717 | - one or more of the supported modules. Setting these will cause the |
2718 | - corresponding hardening code to be run when the hook fires. |
2719 | - |
2720 | - This decorator can and should be applied to more than one hook or function |
2721 | - such that hardening modules are called multiple times. This is because |
2722 | - subsequent calls will perform auditing checks that will report any changes |
2723 | - to resources hardened by the first run (and possibly perform compliance |
2724 | - actions as a result of any detected infractions). |
2725 | - |
2726 | - :param overrides: Optional list of stack modules used to override those |
2727 | - provided with 'harden' config. |
2728 | - :returns: Returns value returned by decorated function once executed. |
2729 | - """ |
2730 | - def _harden_inner1(f): |
2731 | - log("Hardening function '%s'" % (f.__name__), level=DEBUG) |
2732 | - |
2733 | - def _harden_inner2(*args, **kwargs): |
2734 | - RUN_CATALOG = OrderedDict([('os', run_os_checks), |
2735 | - ('ssh', run_ssh_checks), |
2736 | - ('mysql', run_mysql_checks), |
2737 | - ('apache', run_apache_checks)]) |
2738 | - |
2739 | - enabled = overrides or (config("harden") or "").split() |
2740 | - if enabled: |
2741 | - modules_to_run = [] |
2742 | - # modules will always be performed in the following order |
2743 | - for module, func in six.iteritems(RUN_CATALOG): |
2744 | - if module in enabled: |
2745 | - enabled.remove(module) |
2746 | - modules_to_run.append(func) |
2747 | - |
2748 | - if enabled: |
2749 | - log("Unknown hardening modules '%s' - ignoring" % |
2750 | - (', '.join(enabled)), level=WARNING) |
2751 | - |
2752 | - for hardener in modules_to_run: |
2753 | - log("Executing hardening module '%s'" % |
2754 | - (hardener.__name__), level=DEBUG) |
2755 | - hardener() |
2756 | - else: |
2757 | - log("No hardening applied to '%s'" % (f.__name__), level=DEBUG) |
2758 | - |
2759 | - return f(*args, **kwargs) |
2760 | - return _harden_inner2 |
2761 | - |
2762 | - return _harden_inner1 |
2763 | |
2764 | === removed directory 'hooks/charmhelpers/contrib/hardening/host' |
2765 | === removed file 'hooks/charmhelpers/contrib/hardening/host/__init__.py' |
2766 | --- hooks/charmhelpers/contrib/hardening/host/__init__.py 2016-04-22 08:15:33 +0000 |
2767 | +++ hooks/charmhelpers/contrib/hardening/host/__init__.py 1970-01-01 00:00:00 +0000 |
2768 | @@ -1,19 +0,0 @@ |
2769 | -# Copyright 2016 Canonical Limited. |
2770 | -# |
2771 | -# This file is part of charm-helpers. |
2772 | -# |
2773 | -# charm-helpers is free software: you can redistribute it and/or modify |
2774 | -# it under the terms of the GNU Lesser General Public License version 3 as |
2775 | -# published by the Free Software Foundation. |
2776 | -# |
2777 | -# charm-helpers is distributed in the hope that it will be useful, |
2778 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
2779 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
2780 | -# GNU Lesser General Public License for more details. |
2781 | -# |
2782 | -# You should have received a copy of the GNU Lesser General Public License |
2783 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
2784 | - |
2785 | -from os import path |
2786 | - |
2787 | -TEMPLATES_DIR = path.join(path.dirname(__file__), 'templates') |
2788 | |
2789 | === removed directory 'hooks/charmhelpers/contrib/hardening/host/checks' |
2790 | === removed file 'hooks/charmhelpers/contrib/hardening/host/checks/__init__.py' |
2791 | --- hooks/charmhelpers/contrib/hardening/host/checks/__init__.py 2016-04-22 08:15:33 +0000 |
2792 | +++ hooks/charmhelpers/contrib/hardening/host/checks/__init__.py 1970-01-01 00:00:00 +0000 |
2793 | @@ -1,50 +0,0 @@ |
2794 | -# Copyright 2016 Canonical Limited. |
2795 | -# |
2796 | -# This file is part of charm-helpers. |
2797 | -# |
2798 | -# charm-helpers is free software: you can redistribute it and/or modify |
2799 | -# it under the terms of the GNU Lesser General Public License version 3 as |
2800 | -# published by the Free Software Foundation. |
2801 | -# |
2802 | -# charm-helpers is distributed in the hope that it will be useful, |
2803 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
2804 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
2805 | -# GNU Lesser General Public License for more details. |
2806 | -# |
2807 | -# You should have received a copy of the GNU Lesser General Public License |
2808 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
2809 | - |
2810 | -from charmhelpers.core.hookenv import ( |
2811 | - log, |
2812 | - DEBUG, |
2813 | -) |
2814 | -from charmhelpers.contrib.hardening.host.checks import ( |
2815 | - apt, |
2816 | - limits, |
2817 | - login, |
2818 | - minimize_access, |
2819 | - pam, |
2820 | - profile, |
2821 | - securetty, |
2822 | - suid_sgid, |
2823 | - sysctl |
2824 | -) |
2825 | - |
2826 | - |
2827 | -def run_os_checks(): |
2828 | - log("Starting OS hardening checks.", level=DEBUG) |
2829 | - checks = apt.get_audits() |
2830 | - checks.extend(limits.get_audits()) |
2831 | - checks.extend(login.get_audits()) |
2832 | - checks.extend(minimize_access.get_audits()) |
2833 | - checks.extend(pam.get_audits()) |
2834 | - checks.extend(profile.get_audits()) |
2835 | - checks.extend(securetty.get_audits()) |
2836 | - checks.extend(suid_sgid.get_audits()) |
2837 | - checks.extend(sysctl.get_audits()) |
2838 | - |
2839 | - for check in checks: |
2840 | - log("Running '%s' check" % (check.__class__.__name__), level=DEBUG) |
2841 | - check.ensure_compliance() |
2842 | - |
2843 | - log("OS hardening checks complete.", level=DEBUG) |
2844 | |
2845 | === removed file 'hooks/charmhelpers/contrib/hardening/host/checks/apt.py' |
2846 | --- hooks/charmhelpers/contrib/hardening/host/checks/apt.py 2016-04-22 08:15:33 +0000 |
2847 | +++ hooks/charmhelpers/contrib/hardening/host/checks/apt.py 1970-01-01 00:00:00 +0000 |
2848 | @@ -1,39 +0,0 @@ |
2849 | -# Copyright 2016 Canonical Limited. |
2850 | -# |
2851 | -# This file is part of charm-helpers. |
2852 | -# |
2853 | -# charm-helpers is free software: you can redistribute it and/or modify |
2854 | -# it under the terms of the GNU Lesser General Public License version 3 as |
2855 | -# published by the Free Software Foundation. |
2856 | -# |
2857 | -# charm-helpers is distributed in the hope that it will be useful, |
2858 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
2859 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
2860 | -# GNU Lesser General Public License for more details. |
2861 | -# |
2862 | -# You should have received a copy of the GNU Lesser General Public License |
2863 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
2864 | - |
2865 | -from charmhelpers.contrib.hardening.utils import get_settings |
2866 | -from charmhelpers.contrib.hardening.audits.apt import ( |
2867 | - AptConfig, |
2868 | - RestrictedPackages, |
2869 | -) |
2870 | - |
2871 | - |
2872 | -def get_audits(): |
2873 | - """Get OS hardening apt audits. |
2874 | - |
2875 | - :returns: dictionary of audits |
2876 | - """ |
2877 | - audits = [AptConfig([{'key': 'APT::Get::AllowUnauthenticated', |
2878 | - 'expected': 'false'}])] |
2879 | - |
2880 | - settings = get_settings('os') |
2881 | - clean_packages = settings['security']['packages_clean'] |
2882 | - if clean_packages: |
2883 | - security_packages = settings['security']['packages_list'] |
2884 | - if security_packages: |
2885 | - audits.append(RestrictedPackages(security_packages)) |
2886 | - |
2887 | - return audits |
2888 | |
2889 | === removed file 'hooks/charmhelpers/contrib/hardening/host/checks/limits.py' |
2890 | --- hooks/charmhelpers/contrib/hardening/host/checks/limits.py 2016-04-22 08:15:33 +0000 |
2891 | +++ hooks/charmhelpers/contrib/hardening/host/checks/limits.py 1970-01-01 00:00:00 +0000 |
2892 | @@ -1,55 +0,0 @@ |
2893 | -# Copyright 2016 Canonical Limited. |
2894 | -# |
2895 | -# This file is part of charm-helpers. |
2896 | -# |
2897 | -# charm-helpers is free software: you can redistribute it and/or modify |
2898 | -# it under the terms of the GNU Lesser General Public License version 3 as |
2899 | -# published by the Free Software Foundation. |
2900 | -# |
2901 | -# charm-helpers is distributed in the hope that it will be useful, |
2902 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
2903 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
2904 | -# GNU Lesser General Public License for more details. |
2905 | -# |
2906 | -# You should have received a copy of the GNU Lesser General Public License |
2907 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
2908 | - |
2909 | -from charmhelpers.contrib.hardening.audits.file import ( |
2910 | - DirectoryPermissionAudit, |
2911 | - TemplatedFile, |
2912 | -) |
2913 | -from charmhelpers.contrib.hardening.host import TEMPLATES_DIR |
2914 | -from charmhelpers.contrib.hardening import utils |
2915 | - |
2916 | - |
2917 | -def get_audits(): |
2918 | - """Get OS hardening security limits audits. |
2919 | - |
2920 | - :returns: dictionary of audits |
2921 | - """ |
2922 | - audits = [] |
2923 | - settings = utils.get_settings('os') |
2924 | - |
2925 | - # Ensure that the /etc/security/limits.d directory is only writable |
2926 | - # by the root user, but others can execute and read. |
2927 | - audits.append(DirectoryPermissionAudit('/etc/security/limits.d', |
2928 | - user='root', group='root', |
2929 | - mode=0o755)) |
2930 | - |
2931 | - # If core dumps are not enabled, then don't allow core dumps to be |
2932 | - # created as they may contain sensitive information. |
2933 | - if not settings['security']['kernel_enable_core_dump']: |
2934 | - audits.append(TemplatedFile('/etc/security/limits.d/10.hardcore.conf', |
2935 | - SecurityLimitsContext(), |
2936 | - template_dir=TEMPLATES_DIR, |
2937 | - user='root', group='root', mode=0o0440)) |
2938 | - return audits |
2939 | - |
2940 | - |
2941 | -class SecurityLimitsContext(object): |
2942 | - |
2943 | - def __call__(self): |
2944 | - settings = utils.get_settings('os') |
2945 | - ctxt = {'disable_core_dump': |
2946 | - not settings['security']['kernel_enable_core_dump']} |
2947 | - return ctxt |
2948 | |
2949 | === removed file 'hooks/charmhelpers/contrib/hardening/host/checks/login.py' |
2950 | --- hooks/charmhelpers/contrib/hardening/host/checks/login.py 2016-04-22 08:15:33 +0000 |
2951 | +++ hooks/charmhelpers/contrib/hardening/host/checks/login.py 1970-01-01 00:00:00 +0000 |
2952 | @@ -1,67 +0,0 @@ |
2953 | -# Copyright 2016 Canonical Limited. |
2954 | -# |
2955 | -# This file is part of charm-helpers. |
2956 | -# |
2957 | -# charm-helpers is free software: you can redistribute it and/or modify |
2958 | -# it under the terms of the GNU Lesser General Public License version 3 as |
2959 | -# published by the Free Software Foundation. |
2960 | -# |
2961 | -# charm-helpers is distributed in the hope that it will be useful, |
2962 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
2963 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
2964 | -# GNU Lesser General Public License for more details. |
2965 | -# |
2966 | -# You should have received a copy of the GNU Lesser General Public License |
2967 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
2968 | - |
2969 | -from six import string_types |
2970 | - |
2971 | -from charmhelpers.contrib.hardening.audits.file import TemplatedFile |
2972 | -from charmhelpers.contrib.hardening.host import TEMPLATES_DIR |
2973 | -from charmhelpers.contrib.hardening import utils |
2974 | - |
2975 | - |
2976 | -def get_audits(): |
2977 | - """Get OS hardening login.defs audits. |
2978 | - |
2979 | - :returns: dictionary of audits |
2980 | - """ |
2981 | - audits = [TemplatedFile('/etc/login.defs', LoginContext(), |
2982 | - template_dir=TEMPLATES_DIR, |
2983 | - user='root', group='root', mode=0o0444)] |
2984 | - return audits |
2985 | - |
2986 | - |
2987 | -class LoginContext(object): |
2988 | - |
2989 | - def __call__(self): |
2990 | - settings = utils.get_settings('os') |
2991 | - |
2992 | - # Octal numbers in yaml end up being turned into decimal, |
2993 | - # so check if the umask is entered as a string (e.g. '027') |
2994 | - # or as an octal umask as we know it (e.g. 002). If its not |
2995 | - # a string assume it to be octal and turn it into an octal |
2996 | - # string. |
2997 | - umask = settings['environment']['umask'] |
2998 | - if not isinstance(umask, string_types): |
2999 | - umask = '%s' % oct(umask) |
3000 | - |
3001 | - ctxt = { |
3002 | - 'additional_user_paths': |
3003 | - settings['environment']['extra_user_paths'], |
3004 | - 'umask': umask, |
3005 | - 'pwd_max_age': settings['auth']['pw_max_age'], |
3006 | - 'pwd_min_age': settings['auth']['pw_min_age'], |
3007 | - 'uid_min': settings['auth']['uid_min'], |
3008 | - 'sys_uid_min': settings['auth']['sys_uid_min'], |
3009 | - 'sys_uid_max': settings['auth']['sys_uid_max'], |
3010 | - 'gid_min': settings['auth']['gid_min'], |
3011 | - 'sys_gid_min': settings['auth']['sys_gid_min'], |
3012 | - 'sys_gid_max': settings['auth']['sys_gid_max'], |
3013 | - 'login_retries': settings['auth']['retries'], |
3014 | - 'login_timeout': settings['auth']['timeout'], |
3015 | - 'chfn_restrict': settings['auth']['chfn_restrict'], |
3016 | - 'allow_login_without_home': settings['auth']['allow_homeless'] |
3017 | - } |
3018 | - |
3019 | - return ctxt |
3020 | |
3021 | === removed file 'hooks/charmhelpers/contrib/hardening/host/checks/minimize_access.py' |
3022 | --- hooks/charmhelpers/contrib/hardening/host/checks/minimize_access.py 2016-04-22 08:15:33 +0000 |
3023 | +++ hooks/charmhelpers/contrib/hardening/host/checks/minimize_access.py 1970-01-01 00:00:00 +0000 |
3024 | @@ -1,52 +0,0 @@ |
3025 | -# Copyright 2016 Canonical Limited. |
3026 | -# |
3027 | -# This file is part of charm-helpers. |
3028 | -# |
3029 | -# charm-helpers is free software: you can redistribute it and/or modify |
3030 | -# it under the terms of the GNU Lesser General Public License version 3 as |
3031 | -# published by the Free Software Foundation. |
3032 | -# |
3033 | -# charm-helpers is distributed in the hope that it will be useful, |
3034 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
3035 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
3036 | -# GNU Lesser General Public License for more details. |
3037 | -# |
3038 | -# You should have received a copy of the GNU Lesser General Public License |
3039 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
3040 | - |
3041 | -from charmhelpers.contrib.hardening.audits.file import ( |
3042 | - FilePermissionAudit, |
3043 | - ReadOnly, |
3044 | -) |
3045 | -from charmhelpers.contrib.hardening import utils |
3046 | - |
3047 | - |
3048 | -def get_audits(): |
3049 | - """Get OS hardening access audits. |
3050 | - |
3051 | - :returns: dictionary of audits |
3052 | - """ |
3053 | - audits = [] |
3054 | - settings = utils.get_settings('os') |
3055 | - |
3056 | - # Remove write permissions from $PATH folders for all regular users. |
3057 | - # This prevents changing system-wide commands from normal users. |
3058 | - path_folders = {'/usr/local/sbin', |
3059 | - '/usr/local/bin', |
3060 | - '/usr/sbin', |
3061 | - '/usr/bin', |
3062 | - '/bin'} |
3063 | - extra_user_paths = settings['environment']['extra_user_paths'] |
3064 | - path_folders.update(extra_user_paths) |
3065 | - audits.append(ReadOnly(path_folders)) |
3066 | - |
3067 | - # Only allow the root user to have access to the shadow file. |
3068 | - audits.append(FilePermissionAudit('/etc/shadow', 'root', 'root', 0o0600)) |
3069 | - |
3070 | - if 'change_user' not in settings['security']['users_allow']: |
3071 | - # su should only be accessible to user and group root, unless it is |
3072 | - # expressly defined to allow users to change to root via the |
3073 | - # security_users_allow config option. |
3074 | - audits.append(FilePermissionAudit('/bin/su', 'root', 'root', 0o750)) |
3075 | - |
3076 | - return audits |
3077 | |
3078 | === removed file 'hooks/charmhelpers/contrib/hardening/host/checks/pam.py' |
3079 | --- hooks/charmhelpers/contrib/hardening/host/checks/pam.py 2016-04-22 08:15:33 +0000 |
3080 | +++ hooks/charmhelpers/contrib/hardening/host/checks/pam.py 1970-01-01 00:00:00 +0000 |
3081 | @@ -1,134 +0,0 @@ |
3082 | -# Copyright 2016 Canonical Limited. |
3083 | -# |
3084 | -# This file is part of charm-helpers. |
3085 | -# |
3086 | -# charm-helpers is free software: you can redistribute it and/or modify |
3087 | -# it under the terms of the GNU Lesser General Public License version 3 as |
3088 | -# published by the Free Software Foundation. |
3089 | -# |
3090 | -# charm-helpers is distributed in the hope that it will be useful, |
3091 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
3092 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
3093 | -# GNU Lesser General Public License for more details. |
3094 | -# |
3095 | -# You should have received a copy of the GNU Lesser General Public License |
3096 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
3097 | - |
3098 | -from subprocess import ( |
3099 | - check_output, |
3100 | - CalledProcessError, |
3101 | -) |
3102 | - |
3103 | -from charmhelpers.core.hookenv import ( |
3104 | - log, |
3105 | - DEBUG, |
3106 | - ERROR, |
3107 | -) |
3108 | -from charmhelpers.fetch import ( |
3109 | - apt_install, |
3110 | - apt_purge, |
3111 | - apt_update, |
3112 | -) |
3113 | -from charmhelpers.contrib.hardening.audits.file import ( |
3114 | - TemplatedFile, |
3115 | - DeletedFile, |
3116 | -) |
3117 | -from charmhelpers.contrib.hardening import utils |
3118 | -from charmhelpers.contrib.hardening.host import TEMPLATES_DIR |
3119 | - |
3120 | - |
3121 | -def get_audits(): |
3122 | - """Get OS hardening PAM authentication audits. |
3123 | - |
3124 | - :returns: dictionary of audits |
3125 | - """ |
3126 | - audits = [] |
3127 | - |
3128 | - settings = utils.get_settings('os') |
3129 | - |
3130 | - if settings['auth']['pam_passwdqc_enable']: |
3131 | - audits.append(PasswdqcPAM('/etc/passwdqc.conf')) |
3132 | - |
3133 | - if settings['auth']['retries']: |
3134 | - audits.append(Tally2PAM('/usr/share/pam-configs/tally2')) |
3135 | - else: |
3136 | - audits.append(DeletedFile('/usr/share/pam-configs/tally2')) |
3137 | - |
3138 | - return audits |
3139 | - |
3140 | - |
3141 | -class PasswdqcPAMContext(object): |
3142 | - |
3143 | - def __call__(self): |
3144 | - ctxt = {} |
3145 | - settings = utils.get_settings('os') |
3146 | - |
3147 | - ctxt['auth_pam_passwdqc_options'] = \ |
3148 | - settings['auth']['pam_passwdqc_options'] |
3149 | - |
3150 | - return ctxt |
3151 | - |
3152 | - |
3153 | -class PasswdqcPAM(TemplatedFile): |
3154 | - """The PAM Audit verifies the linux PAM settings.""" |
3155 | - def __init__(self, path): |
3156 | - super(PasswdqcPAM, self).__init__(path=path, |
3157 | - template_dir=TEMPLATES_DIR, |
3158 | - context=PasswdqcPAMContext(), |
3159 | - user='root', |
3160 | - group='root', |
3161 | - mode=0o0640) |
3162 | - |
3163 | - def pre_write(self): |
3164 | - # Always remove? |
3165 | - for pkg in ['libpam-ccreds', 'libpam-cracklib']: |
3166 | - log("Purging package '%s'" % pkg, level=DEBUG), |
3167 | - apt_purge(pkg) |
3168 | - |
3169 | - apt_update(fatal=True) |
3170 | - for pkg in ['libpam-passwdqc']: |
3171 | - log("Installing package '%s'" % pkg, level=DEBUG), |
3172 | - apt_install(pkg) |
3173 | - |
3174 | - def post_write(self): |
3175 | - """Updates the PAM configuration after the file has been written""" |
3176 | - try: |
3177 | - check_output(['pam-auth-update', '--package']) |
3178 | - except CalledProcessError as e: |
3179 | - log('Error calling pam-auth-update: %s' % e, level=ERROR) |
3180 | - |
3181 | - |
3182 | -class Tally2PAMContext(object): |
3183 | - |
3184 | - def __call__(self): |
3185 | - ctxt = {} |
3186 | - settings = utils.get_settings('os') |
3187 | - |
3188 | - ctxt['auth_lockout_time'] = settings['auth']['lockout_time'] |
3189 | - ctxt['auth_retries'] = settings['auth']['retries'] |
3190 | - |
3191 | - return ctxt |
3192 | - |
3193 | - |
3194 | -class Tally2PAM(TemplatedFile): |
3195 | - """The PAM Audit verifies the linux PAM settings.""" |
3196 | - def __init__(self, path): |
3197 | - super(Tally2PAM, self).__init__(path=path, |
3198 | - template_dir=TEMPLATES_DIR, |
3199 | - context=Tally2PAMContext(), |
3200 | - user='root', |
3201 | - group='root', |
3202 | - mode=0o0640) |
3203 | - |
3204 | - def pre_write(self): |
3205 | - # Always remove? |
3206 | - apt_purge('libpam-ccreds') |
3207 | - apt_update(fatal=True) |
3208 | - apt_install('libpam-modules') |
3209 | - |
3210 | - def post_write(self): |
3211 | - """Updates the PAM configuration after the file has been written""" |
3212 | - try: |
3213 | - check_output(['pam-auth-update', '--package']) |
3214 | - except CalledProcessError as e: |
3215 | - log('Error calling pam-auth-update: %s' % e, level=ERROR) |
3216 | |
3217 | === removed file 'hooks/charmhelpers/contrib/hardening/host/checks/profile.py' |
3218 | --- hooks/charmhelpers/contrib/hardening/host/checks/profile.py 2016-04-22 08:15:33 +0000 |
3219 | +++ hooks/charmhelpers/contrib/hardening/host/checks/profile.py 1970-01-01 00:00:00 +0000 |
3220 | @@ -1,45 +0,0 @@ |
3221 | -# Copyright 2016 Canonical Limited. |
3222 | -# |
3223 | -# This file is part of charm-helpers. |
3224 | -# |
3225 | -# charm-helpers is free software: you can redistribute it and/or modify |
3226 | -# it under the terms of the GNU Lesser General Public License version 3 as |
3227 | -# published by the Free Software Foundation. |
3228 | -# |
3229 | -# charm-helpers is distributed in the hope that it will be useful, |
3230 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
3231 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
3232 | -# GNU Lesser General Public License for more details. |
3233 | -# |
3234 | -# You should have received a copy of the GNU Lesser General Public License |
3235 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
3236 | - |
3237 | -from charmhelpers.contrib.hardening.audits.file import TemplatedFile |
3238 | -from charmhelpers.contrib.hardening.host import TEMPLATES_DIR |
3239 | -from charmhelpers.contrib.hardening import utils |
3240 | - |
3241 | - |
3242 | -def get_audits(): |
3243 | - """Get OS hardening profile audits. |
3244 | - |
3245 | - :returns: dictionary of audits |
3246 | - """ |
3247 | - audits = [] |
3248 | - |
3249 | - settings = utils.get_settings('os') |
3250 | - |
3251 | - # If core dumps are not enabled, then don't allow core dumps to be |
3252 | - # created as they may contain sensitive information. |
3253 | - if not settings['security']['kernel_enable_core_dump']: |
3254 | - audits.append(TemplatedFile('/etc/profile.d/pinerolo_profile.sh', |
3255 | - ProfileContext(), |
3256 | - template_dir=TEMPLATES_DIR, |
3257 | - mode=0o0755, user='root', group='root')) |
3258 | - return audits |
3259 | - |
3260 | - |
3261 | -class ProfileContext(object): |
3262 | - |
3263 | - def __call__(self): |
3264 | - ctxt = {} |
3265 | - return ctxt |
3266 | |
3267 | === removed file 'hooks/charmhelpers/contrib/hardening/host/checks/securetty.py' |
3268 | --- hooks/charmhelpers/contrib/hardening/host/checks/securetty.py 2016-04-22 08:15:33 +0000 |
3269 | +++ hooks/charmhelpers/contrib/hardening/host/checks/securetty.py 1970-01-01 00:00:00 +0000 |
3270 | @@ -1,39 +0,0 @@ |
3271 | -# Copyright 2016 Canonical Limited. |
3272 | -# |
3273 | -# This file is part of charm-helpers. |
3274 | -# |
3275 | -# charm-helpers is free software: you can redistribute it and/or modify |
3276 | -# it under the terms of the GNU Lesser General Public License version 3 as |
3277 | -# published by the Free Software Foundation. |
3278 | -# |
3279 | -# charm-helpers is distributed in the hope that it will be useful, |
3280 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
3281 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
3282 | -# GNU Lesser General Public License for more details. |
3283 | -# |
3284 | -# You should have received a copy of the GNU Lesser General Public License |
3285 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
3286 | - |
3287 | -from charmhelpers.contrib.hardening.audits.file import TemplatedFile |
3288 | -from charmhelpers.contrib.hardening.host import TEMPLATES_DIR |
3289 | -from charmhelpers.contrib.hardening import utils |
3290 | - |
3291 | - |
3292 | -def get_audits(): |
3293 | - """Get OS hardening Secure TTY audits. |
3294 | - |
3295 | - :returns: dictionary of audits |
3296 | - """ |
3297 | - audits = [] |
3298 | - audits.append(TemplatedFile('/etc/securetty', SecureTTYContext(), |
3299 | - template_dir=TEMPLATES_DIR, |
3300 | - mode=0o0400, user='root', group='root')) |
3301 | - return audits |
3302 | - |
3303 | - |
3304 | -class SecureTTYContext(object): |
3305 | - |
3306 | - def __call__(self): |
3307 | - settings = utils.get_settings('os') |
3308 | - ctxt = {'ttys': settings['auth']['root_ttys']} |
3309 | - return ctxt |
3310 | |
3311 | === removed file 'hooks/charmhelpers/contrib/hardening/host/checks/suid_sgid.py' |
3312 | --- hooks/charmhelpers/contrib/hardening/host/checks/suid_sgid.py 2016-04-22 08:15:33 +0000 |
3313 | +++ hooks/charmhelpers/contrib/hardening/host/checks/suid_sgid.py 1970-01-01 00:00:00 +0000 |
3314 | @@ -1,131 +0,0 @@ |
3315 | -# Copyright 2016 Canonical Limited. |
3316 | -# |
3317 | -# This file is part of charm-helpers. |
3318 | -# |
3319 | -# charm-helpers is free software: you can redistribute it and/or modify |
3320 | -# it under the terms of the GNU Lesser General Public License version 3 as |
3321 | -# published by the Free Software Foundation. |
3322 | -# |
3323 | -# charm-helpers is distributed in the hope that it will be useful, |
3324 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
3325 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
3326 | -# GNU Lesser General Public License for more details. |
3327 | -# |
3328 | -# You should have received a copy of the GNU Lesser General Public License |
3329 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
3330 | - |
3331 | -import subprocess |
3332 | - |
3333 | -from charmhelpers.core.hookenv import ( |
3334 | - log, |
3335 | - INFO, |
3336 | -) |
3337 | -from charmhelpers.contrib.hardening.audits.file import NoSUIDSGIDAudit |
3338 | -from charmhelpers.contrib.hardening import utils |
3339 | - |
3340 | - |
3341 | -BLACKLIST = ['/usr/bin/rcp', '/usr/bin/rlogin', '/usr/bin/rsh', |
3342 | - '/usr/libexec/openssh/ssh-keysign', |
3343 | - '/usr/lib/openssh/ssh-keysign', |
3344 | - '/sbin/netreport', |
3345 | - '/usr/sbin/usernetctl', |
3346 | - '/usr/sbin/userisdnctl', |
3347 | - '/usr/sbin/pppd', |
3348 | - '/usr/bin/lockfile', |
3349 | - '/usr/bin/mail-lock', |
3350 | - '/usr/bin/mail-unlock', |
3351 | - '/usr/bin/mail-touchlock', |
3352 | - '/usr/bin/dotlockfile', |
3353 | - '/usr/bin/arping', |
3354 | - '/usr/sbin/uuidd', |
3355 | - '/usr/bin/mtr', |
3356 | - '/usr/lib/evolution/camel-lock-helper-1.2', |
3357 | - '/usr/lib/pt_chown', |
3358 | - '/usr/lib/eject/dmcrypt-get-device', |
3359 | - '/usr/lib/mc/cons.saver'] |
3360 | - |
3361 | -WHITELIST = ['/bin/mount', '/bin/ping', '/bin/su', '/bin/umount', |
3362 | - '/sbin/pam_timestamp_check', '/sbin/unix_chkpwd', '/usr/bin/at', |
3363 | - '/usr/bin/gpasswd', '/usr/bin/locate', '/usr/bin/newgrp', |
3364 | - '/usr/bin/passwd', '/usr/bin/ssh-agent', |
3365 | - '/usr/libexec/utempter/utempter', '/usr/sbin/lockdev', |
3366 | - '/usr/sbin/sendmail.sendmail', '/usr/bin/expiry', |
3367 | - '/bin/ping6', '/usr/bin/traceroute6.iputils', |
3368 | - '/sbin/mount.nfs', '/sbin/umount.nfs', |
3369 | - '/sbin/mount.nfs4', '/sbin/umount.nfs4', |
3370 | - '/usr/bin/crontab', |
3371 | - '/usr/bin/wall', '/usr/bin/write', |
3372 | - '/usr/bin/screen', |
3373 | - '/usr/bin/mlocate', |
3374 | - '/usr/bin/chage', '/usr/bin/chfn', '/usr/bin/chsh', |
3375 | - '/bin/fusermount', |
3376 | - '/usr/bin/pkexec', |
3377 | - '/usr/bin/sudo', '/usr/bin/sudoedit', |
3378 | - '/usr/sbin/postdrop', '/usr/sbin/postqueue', |
3379 | - '/usr/sbin/suexec', |
3380 | - '/usr/lib/squid/ncsa_auth', '/usr/lib/squid/pam_auth', |
3381 | - '/usr/kerberos/bin/ksu', |
3382 | - '/usr/sbin/ccreds_validate', |
3383 | - '/usr/bin/Xorg', |
3384 | - '/usr/bin/X', |
3385 | - '/usr/lib/dbus-1.0/dbus-daemon-launch-helper', |
3386 | - '/usr/lib/vte/gnome-pty-helper', |
3387 | - '/usr/lib/libvte9/gnome-pty-helper', |
3388 | - '/usr/lib/libvte-2.90-9/gnome-pty-helper'] |
3389 | - |
3390 | - |
3391 | -def get_audits(): |
3392 | - """Get OS hardening suid/sgid audits. |
3393 | - |
3394 | - :returns: dictionary of audits |
3395 | - """ |
3396 | - checks = [] |
3397 | - settings = utils.get_settings('os') |
3398 | - if not settings['security']['suid_sgid_enforce']: |
3399 | - log("Skipping suid/sgid hardening", level=INFO) |
3400 | - return checks |
3401 | - |
3402 | - # Build the blacklist and whitelist of files for suid/sgid checks. |
3403 | - # There are a total of 4 lists: |
3404 | - # 1. the system blacklist |
3405 | - # 2. the system whitelist |
3406 | - # 3. the user blacklist |
3407 | - # 4. the user whitelist |
3408 | - # |
3409 | - # The blacklist is the set of paths which should NOT have the suid/sgid bit |
3410 | - # set and the whitelist is the set of paths which MAY have the suid/sgid |
3411 | - # bit setl. The user whitelist/blacklist effectively override the system |
3412 | - # whitelist/blacklist. |
3413 | - u_b = settings['security']['suid_sgid_blacklist'] |
3414 | - u_w = settings['security']['suid_sgid_whitelist'] |
3415 | - |
3416 | - blacklist = set(BLACKLIST) - set(u_w + u_b) |
3417 | - whitelist = set(WHITELIST) - set(u_b + u_w) |
3418 | - |
3419 | - checks.append(NoSUIDSGIDAudit(blacklist)) |
3420 | - |
3421 | - dry_run = settings['security']['suid_sgid_dry_run_on_unknown'] |
3422 | - |
3423 | - if settings['security']['suid_sgid_remove_from_unknown'] or dry_run: |
3424 | - # If the policy is a dry_run (e.g. complain only) or remove unknown |
3425 | - # suid/sgid bits then find all of the paths which have the suid/sgid |
3426 | - # bit set and then remove the whitelisted paths. |
3427 | - root_path = settings['environment']['root_path'] |
3428 | - unknown_paths = find_paths_with_suid_sgid(root_path) - set(whitelist) |
3429 | - checks.append(NoSUIDSGIDAudit(unknown_paths, unless=dry_run)) |
3430 | - |
3431 | - return checks |
3432 | - |
3433 | - |
3434 | -def find_paths_with_suid_sgid(root_path): |
3435 | - """Finds all paths/files which have an suid/sgid bit enabled. |
3436 | - |
3437 | - Starting with the root_path, this will recursively find all paths which |
3438 | - have an suid or sgid bit set. |
3439 | - """ |
3440 | - cmd = ['find', root_path, '-perm', '-4000', '-o', '-perm', '-2000', |
3441 | - '-type', 'f', '!', '-path', '/proc/*', '-print'] |
3442 | - |
3443 | - p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE) |
3444 | - out, _ = p.communicate() |
3445 | - return set(out.split('\n')) |
3446 | |
3447 | === removed file 'hooks/charmhelpers/contrib/hardening/host/checks/sysctl.py' |
3448 | --- hooks/charmhelpers/contrib/hardening/host/checks/sysctl.py 2016-04-22 08:15:33 +0000 |
3449 | +++ hooks/charmhelpers/contrib/hardening/host/checks/sysctl.py 1970-01-01 00:00:00 +0000 |
3450 | @@ -1,211 +0,0 @@ |
3451 | -# Copyright 2016 Canonical Limited. |
3452 | -# |
3453 | -# This file is part of charm-helpers. |
3454 | -# |
3455 | -# charm-helpers is free software: you can redistribute it and/or modify |
3456 | -# it under the terms of the GNU Lesser General Public License version 3 as |
3457 | -# published by the Free Software Foundation. |
3458 | -# |
3459 | -# charm-helpers is distributed in the hope that it will be useful, |
3460 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
3461 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
3462 | -# GNU Lesser General Public License for more details. |
3463 | -# |
3464 | -# You should have received a copy of the GNU Lesser General Public License |
3465 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
3466 | - |
3467 | -import os |
3468 | -import platform |
3469 | -import re |
3470 | -import six |
3471 | -import subprocess |
3472 | - |
3473 | -from charmhelpers.core.hookenv import ( |
3474 | - log, |
3475 | - INFO, |
3476 | - WARNING, |
3477 | -) |
3478 | -from charmhelpers.contrib.hardening import utils |
3479 | -from charmhelpers.contrib.hardening.audits.file import ( |
3480 | - FilePermissionAudit, |
3481 | - TemplatedFile, |
3482 | -) |
3483 | -from charmhelpers.contrib.hardening.host import TEMPLATES_DIR |
3484 | - |
3485 | - |
3486 | -SYSCTL_DEFAULTS = """net.ipv4.ip_forward=%(net_ipv4_ip_forward)s |
3487 | -net.ipv6.conf.all.forwarding=%(net_ipv6_conf_all_forwarding)s |
3488 | -net.ipv4.conf.all.rp_filter=1 |
3489 | -net.ipv4.conf.default.rp_filter=1 |
3490 | -net.ipv4.icmp_echo_ignore_broadcasts=1 |
3491 | -net.ipv4.icmp_ignore_bogus_error_responses=1 |
3492 | -net.ipv4.icmp_ratelimit=100 |
3493 | -net.ipv4.icmp_ratemask=88089 |
3494 | -net.ipv6.conf.all.disable_ipv6=%(net_ipv6_conf_all_disable_ipv6)s |
3495 | -net.ipv4.tcp_timestamps=%(net_ipv4_tcp_timestamps)s |
3496 | -net.ipv4.conf.all.arp_ignore=%(net_ipv4_conf_all_arp_ignore)s |
3497 | -net.ipv4.conf.all.arp_announce=%(net_ipv4_conf_all_arp_announce)s |
3498 | -net.ipv4.tcp_rfc1337=1 |
3499 | -net.ipv4.tcp_syncookies=1 |
3500 | -net.ipv4.conf.all.shared_media=1 |
3501 | -net.ipv4.conf.default.shared_media=1 |
3502 | -net.ipv4.conf.all.accept_source_route=0 |
3503 | -net.ipv4.conf.default.accept_source_route=0 |
3504 | -net.ipv4.conf.all.accept_redirects=0 |
3505 | -net.ipv4.conf.default.accept_redirects=0 |
3506 | -net.ipv6.conf.all.accept_redirects=0 |
3507 | -net.ipv6.conf.default.accept_redirects=0 |
3508 | -net.ipv4.conf.all.secure_redirects=0 |
3509 | -net.ipv4.conf.default.secure_redirects=0 |
3510 | -net.ipv4.conf.all.send_redirects=0 |
3511 | -net.ipv4.conf.default.send_redirects=0 |
3512 | -net.ipv4.conf.all.log_martians=0 |
3513 | -net.ipv6.conf.default.router_solicitations=0 |
3514 | -net.ipv6.conf.default.accept_ra_rtr_pref=0 |
3515 | -net.ipv6.conf.default.accept_ra_pinfo=0 |
3516 | -net.ipv6.conf.default.accept_ra_defrtr=0 |
3517 | -net.ipv6.conf.default.autoconf=0 |
3518 | -net.ipv6.conf.default.dad_transmits=0 |
3519 | -net.ipv6.conf.default.max_addresses=1 |
3520 | -net.ipv6.conf.all.accept_ra=0 |
3521 | -net.ipv6.conf.default.accept_ra=0 |
3522 | -kernel.modules_disabled=%(kernel_modules_disabled)s |
3523 | -kernel.sysrq=%(kernel_sysrq)s |
3524 | -fs.suid_dumpable=%(fs_suid_dumpable)s |
3525 | -kernel.randomize_va_space=2 |
3526 | -""" |
3527 | - |
3528 | - |
3529 | -def get_audits(): |
3530 | - """Get OS hardening sysctl audits. |
3531 | - |
3532 | - :returns: dictionary of audits |
3533 | - """ |
3534 | - audits = [] |
3535 | - settings = utils.get_settings('os') |
3536 | - |
3537 | - # Apply the sysctl settings which are configured to be applied. |
3538 | - audits.append(SysctlConf()) |
3539 | - # Make sure that only root has access to the sysctl.conf file, and |
3540 | - # that it is read-only. |
3541 | - audits.append(FilePermissionAudit('/etc/sysctl.conf', |
3542 | - user='root', |
3543 | - group='root', mode=0o0440)) |
3544 | - # If module loading is not enabled, then ensure that the modules |
3545 | - # file has the appropriate permissions and rebuild the initramfs |
3546 | - if not settings['security']['kernel_enable_module_loading']: |
3547 | - audits.append(ModulesTemplate()) |
3548 | - |
3549 | - return audits |
3550 | - |
3551 | - |
3552 | -class ModulesContext(object): |
3553 | - |
3554 | - def __call__(self): |
3555 | - settings = utils.get_settings('os') |
3556 | - with open('/proc/cpuinfo', 'r') as fd: |
3557 | - cpuinfo = fd.readlines() |
3558 | - |
3559 | - for line in cpuinfo: |
3560 | - match = re.search(r"^vendor_id\s+:\s+(.+)", line) |
3561 | - if match: |
3562 | - vendor = match.group(1) |
3563 | - |
3564 | - if vendor == "GenuineIntel": |
3565 | - vendor = "intel" |
3566 | - elif vendor == "AuthenticAMD": |
3567 | - vendor = "amd" |
3568 | - |
3569 | - ctxt = {'arch': platform.processor(), |
3570 | - 'cpuVendor': vendor, |
3571 | - 'desktop_enable': settings['general']['desktop_enable']} |
3572 | - |
3573 | - return ctxt |
3574 | - |
3575 | - |
3576 | -class ModulesTemplate(object): |
3577 | - |
3578 | - def __init__(self): |
3579 | - super(ModulesTemplate, self).__init__('/etc/initramfs-tools/modules', |
3580 | - ModulesContext(), |
3581 | - templates_dir=TEMPLATES_DIR, |
3582 | - user='root', group='root', |
3583 | - mode=0o0440) |
3584 | - |
3585 | - def post_write(self): |
3586 | - subprocess.check_call(['update-initramfs', '-u']) |
3587 | - |
3588 | - |
3589 | -class SysCtlHardeningContext(object): |
3590 | - def __call__(self): |
3591 | - settings = utils.get_settings('os') |
3592 | - ctxt = {'sysctl': {}} |
3593 | - |
3594 | - log("Applying sysctl settings", level=INFO) |
3595 | - extras = {'net_ipv4_ip_forward': 0, |
3596 | - 'net_ipv6_conf_all_forwarding': 0, |
3597 | - 'net_ipv6_conf_all_disable_ipv6': 1, |
3598 | - 'net_ipv4_tcp_timestamps': 0, |
3599 | - 'net_ipv4_conf_all_arp_ignore': 0, |
3600 | - 'net_ipv4_conf_all_arp_announce': 0, |
3601 | - 'kernel_sysrq': 0, |
3602 | - 'fs_suid_dumpable': 0, |
3603 | - 'kernel_modules_disabled': 1} |
3604 | - |
3605 | - if settings['sysctl']['ipv6_enable']: |
3606 | - extras['net_ipv6_conf_all_disable_ipv6'] = 0 |
3607 | - |
3608 | - if settings['sysctl']['forwarding']: |
3609 | - extras['net_ipv4_ip_forward'] = 1 |
3610 | - extras['net_ipv6_conf_all_forwarding'] = 1 |
3611 | - |
3612 | - if settings['sysctl']['arp_restricted']: |
3613 | - extras['net_ipv4_conf_all_arp_ignore'] = 1 |
3614 | - extras['net_ipv4_conf_all_arp_announce'] = 2 |
3615 | - |
3616 | - if settings['security']['kernel_enable_module_loading']: |
3617 | - extras['kernel_modules_disabled'] = 0 |
3618 | - |
3619 | - if settings['sysctl']['kernel_enable_sysrq']: |
3620 | - sysrq_val = settings['sysctl']['kernel_secure_sysrq'] |
3621 | - extras['kernel_sysrq'] = sysrq_val |
3622 | - |
3623 | - if settings['security']['kernel_enable_core_dump']: |
3624 | - extras['fs_suid_dumpable'] = 1 |
3625 | - |
3626 | - settings.update(extras) |
3627 | - for d in (SYSCTL_DEFAULTS % settings).split(): |
3628 | - d = d.strip().partition('=') |
3629 | - key = d[0].strip() |
3630 | - path = os.path.join('/proc/sys', key.replace('.', '/')) |
3631 | - if not os.path.exists(path): |
3632 | - log("Skipping '%s' since '%s' does not exist" % (key, path), |
3633 | - level=WARNING) |
3634 | - continue |
3635 | - |
3636 | - ctxt['sysctl'][key] = d[2] or None |
3637 | - |
3638 | - # Translate for python3 |
3639 | - return {'sysctl_settings': |
3640 | - [(k, v) for k, v in six.iteritems(ctxt['sysctl'])]} |
3641 | - |
3642 | - |
3643 | -class SysctlConf(TemplatedFile): |
3644 | - """An audit check for sysctl settings.""" |
3645 | - def __init__(self): |
3646 | - self.conffile = '/etc/sysctl.d/99-juju-hardening.conf' |
3647 | - super(SysctlConf, self).__init__(self.conffile, |
3648 | - SysCtlHardeningContext(), |
3649 | - template_dir=TEMPLATES_DIR, |
3650 | - user='root', group='root', |
3651 | - mode=0o0440) |
3652 | - |
3653 | - def post_write(self): |
3654 | - try: |
3655 | - subprocess.check_call(['sysctl', '-p', self.conffile]) |
3656 | - except subprocess.CalledProcessError as e: |
3657 | - # NOTE: on some systems if sysctl cannot apply all settings it |
3658 | - # will return non-zero as well. |
3659 | - log("sysctl command returned an error (maybe some " |
3660 | - "keys could not be set) - %s" % (e), |
3661 | - level=WARNING) |
3662 | |
3663 | === removed directory 'hooks/charmhelpers/contrib/hardening/mysql' |
3664 | === removed file 'hooks/charmhelpers/contrib/hardening/mysql/__init__.py' |
3665 | --- hooks/charmhelpers/contrib/hardening/mysql/__init__.py 2016-04-22 08:15:33 +0000 |
3666 | +++ hooks/charmhelpers/contrib/hardening/mysql/__init__.py 1970-01-01 00:00:00 +0000 |
3667 | @@ -1,19 +0,0 @@ |
3668 | -# Copyright 2016 Canonical Limited. |
3669 | -# |
3670 | -# This file is part of charm-helpers. |
3671 | -# |
3672 | -# charm-helpers is free software: you can redistribute it and/or modify |
3673 | -# it under the terms of the GNU Lesser General Public License version 3 as |
3674 | -# published by the Free Software Foundation. |
3675 | -# |
3676 | -# charm-helpers is distributed in the hope that it will be useful, |
3677 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
3678 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
3679 | -# GNU Lesser General Public License for more details. |
3680 | -# |
3681 | -# You should have received a copy of the GNU Lesser General Public License |
3682 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
3683 | - |
3684 | -from os import path |
3685 | - |
3686 | -TEMPLATES_DIR = path.join(path.dirname(__file__), 'templates') |
3687 | |
3688 | === removed directory 'hooks/charmhelpers/contrib/hardening/mysql/checks' |
3689 | === removed file 'hooks/charmhelpers/contrib/hardening/mysql/checks/__init__.py' |
3690 | --- hooks/charmhelpers/contrib/hardening/mysql/checks/__init__.py 2016-04-22 08:15:33 +0000 |
3691 | +++ hooks/charmhelpers/contrib/hardening/mysql/checks/__init__.py 1970-01-01 00:00:00 +0000 |
3692 | @@ -1,31 +0,0 @@ |
3693 | -# Copyright 2016 Canonical Limited. |
3694 | -# |
3695 | -# This file is part of charm-helpers. |
3696 | -# |
3697 | -# charm-helpers is free software: you can redistribute it and/or modify |
3698 | -# it under the terms of the GNU Lesser General Public License version 3 as |
3699 | -# published by the Free Software Foundation. |
3700 | -# |
3701 | -# charm-helpers is distributed in the hope that it will be useful, |
3702 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
3703 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
3704 | -# GNU Lesser General Public License for more details. |
3705 | -# |
3706 | -# You should have received a copy of the GNU Lesser General Public License |
3707 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
3708 | - |
3709 | -from charmhelpers.core.hookenv import ( |
3710 | - log, |
3711 | - DEBUG, |
3712 | -) |
3713 | -from charmhelpers.contrib.hardening.mysql.checks import config |
3714 | - |
3715 | - |
3716 | -def run_mysql_checks(): |
3717 | - log("Starting MySQL hardening checks.", level=DEBUG) |
3718 | - checks = config.get_audits() |
3719 | - for check in checks: |
3720 | - log("Running '%s' check" % (check.__class__.__name__), level=DEBUG) |
3721 | - check.ensure_compliance() |
3722 | - |
3723 | - log("MySQL hardening checks complete.", level=DEBUG) |
3724 | |
3725 | === removed file 'hooks/charmhelpers/contrib/hardening/mysql/checks/config.py' |
3726 | --- hooks/charmhelpers/contrib/hardening/mysql/checks/config.py 2016-04-22 08:15:33 +0000 |
3727 | +++ hooks/charmhelpers/contrib/hardening/mysql/checks/config.py 1970-01-01 00:00:00 +0000 |
3728 | @@ -1,89 +0,0 @@ |
3729 | -# Copyright 2016 Canonical Limited. |
3730 | -# |
3731 | -# This file is part of charm-helpers. |
3732 | -# |
3733 | -# charm-helpers is free software: you can redistribute it and/or modify |
3734 | -# it under the terms of the GNU Lesser General Public License version 3 as |
3735 | -# published by the Free Software Foundation. |
3736 | -# |
3737 | -# charm-helpers is distributed in the hope that it will be useful, |
3738 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
3739 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
3740 | -# GNU Lesser General Public License for more details. |
3741 | -# |
3742 | -# You should have received a copy of the GNU Lesser General Public License |
3743 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
3744 | - |
3745 | -import six |
3746 | -import subprocess |
3747 | - |
3748 | -from charmhelpers.core.hookenv import ( |
3749 | - log, |
3750 | - WARNING, |
3751 | -) |
3752 | -from charmhelpers.contrib.hardening.audits.file import ( |
3753 | - FilePermissionAudit, |
3754 | - DirectoryPermissionAudit, |
3755 | - TemplatedFile, |
3756 | -) |
3757 | -from charmhelpers.contrib.hardening.mysql import TEMPLATES_DIR |
3758 | -from charmhelpers.contrib.hardening import utils |
3759 | - |
3760 | - |
3761 | -def get_audits(): |
3762 | - """Get MySQL hardening config audits. |
3763 | - |
3764 | - :returns: dictionary of audits |
3765 | - """ |
3766 | - if subprocess.call(['which', 'mysql'], stdout=subprocess.PIPE) != 0: |
3767 | - log("MySQL does not appear to be installed on this node - " |
3768 | - "skipping mysql hardening", level=WARNING) |
3769 | - return [] |
3770 | - |
3771 | - settings = utils.get_settings('mysql') |
3772 | - hardening_settings = settings['hardening'] |
3773 | - my_cnf = hardening_settings['mysql-conf'] |
3774 | - |
3775 | - audits = [ |
3776 | - FilePermissionAudit(paths=[my_cnf], user='root', |
3777 | - group='root', mode=0o0600), |
3778 | - |
3779 | - TemplatedFile(hardening_settings['hardening-conf'], |
3780 | - MySQLConfContext(), |
3781 | - TEMPLATES_DIR, |
3782 | - mode=0o0750, |
3783 | - user='mysql', |
3784 | - group='root', |
3785 | - service_actions=[{'service': 'mysql', |
3786 | - 'actions': ['restart']}]), |
3787 | - |
3788 | - # MySQL and Percona charms do not allow configuration of the |
3789 | - # data directory, so use the default. |
3790 | - DirectoryPermissionAudit('/var/lib/mysql', |
3791 | - user='mysql', |
3792 | - group='mysql', |
3793 | - recursive=False, |
3794 | - mode=0o755), |
3795 | - |
3796 | - DirectoryPermissionAudit('/etc/mysql', |
3797 | - user='root', |
3798 | - group='root', |
3799 | - recursive=False, |
3800 | - mode=0o700), |
3801 | - ] |
3802 | - |
3803 | - return audits |
3804 | - |
3805 | - |
3806 | -class MySQLConfContext(object): |
3807 | - """Defines the set of key/value pairs to set in a mysql config file. |
3808 | - |
3809 | - This context, when called, will return a dictionary containing the |
3810 | - key/value pairs of setting to specify in the |
3811 | - /etc/mysql/conf.d/hardening.cnf file. |
3812 | - """ |
3813 | - def __call__(self): |
3814 | - settings = utils.get_settings('mysql') |
3815 | - # Translate for python3 |
3816 | - return {'mysql_settings': |
3817 | - [(k, v) for k, v in six.iteritems(settings['security'])]} |
3818 | |
3819 | === removed directory 'hooks/charmhelpers/contrib/hardening/ssh' |
3820 | === removed file 'hooks/charmhelpers/contrib/hardening/ssh/__init__.py' |
3821 | --- hooks/charmhelpers/contrib/hardening/ssh/__init__.py 2016-04-22 08:15:33 +0000 |
3822 | +++ hooks/charmhelpers/contrib/hardening/ssh/__init__.py 1970-01-01 00:00:00 +0000 |
3823 | @@ -1,19 +0,0 @@ |
3824 | -# Copyright 2016 Canonical Limited. |
3825 | -# |
3826 | -# This file is part of charm-helpers. |
3827 | -# |
3828 | -# charm-helpers is free software: you can redistribute it and/or modify |
3829 | -# it under the terms of the GNU Lesser General Public License version 3 as |
3830 | -# published by the Free Software Foundation. |
3831 | -# |
3832 | -# charm-helpers is distributed in the hope that it will be useful, |
3833 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
3834 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
3835 | -# GNU Lesser General Public License for more details. |
3836 | -# |
3837 | -# You should have received a copy of the GNU Lesser General Public License |
3838 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
3839 | - |
3840 | -from os import path |
3841 | - |
3842 | -TEMPLATES_DIR = path.join(path.dirname(__file__), 'templates') |
3843 | |
3844 | === removed directory 'hooks/charmhelpers/contrib/hardening/ssh/checks' |
3845 | === removed file 'hooks/charmhelpers/contrib/hardening/ssh/checks/__init__.py' |
3846 | --- hooks/charmhelpers/contrib/hardening/ssh/checks/__init__.py 2016-04-22 08:15:33 +0000 |
3847 | +++ hooks/charmhelpers/contrib/hardening/ssh/checks/__init__.py 1970-01-01 00:00:00 +0000 |
3848 | @@ -1,31 +0,0 @@ |
3849 | -# Copyright 2016 Canonical Limited. |
3850 | -# |
3851 | -# This file is part of charm-helpers. |
3852 | -# |
3853 | -# charm-helpers is free software: you can redistribute it and/or modify |
3854 | -# it under the terms of the GNU Lesser General Public License version 3 as |
3855 | -# published by the Free Software Foundation. |
3856 | -# |
3857 | -# charm-helpers is distributed in the hope that it will be useful, |
3858 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
3859 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
3860 | -# GNU Lesser General Public License for more details. |
3861 | -# |
3862 | -# You should have received a copy of the GNU Lesser General Public License |
3863 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
3864 | - |
3865 | -from charmhelpers.core.hookenv import ( |
3866 | - log, |
3867 | - DEBUG, |
3868 | -) |
3869 | -from charmhelpers.contrib.hardening.ssh.checks import config |
3870 | - |
3871 | - |
3872 | -def run_ssh_checks(): |
3873 | - log("Starting SSH hardening checks.", level=DEBUG) |
3874 | - checks = config.get_audits() |
3875 | - for check in checks: |
3876 | - log("Running '%s' check" % (check.__class__.__name__), level=DEBUG) |
3877 | - check.ensure_compliance() |
3878 | - |
3879 | - log("SSH hardening checks complete.", level=DEBUG) |
3880 | |
3881 | === removed file 'hooks/charmhelpers/contrib/hardening/ssh/checks/config.py' |
3882 | --- hooks/charmhelpers/contrib/hardening/ssh/checks/config.py 2016-04-22 08:15:33 +0000 |
3883 | +++ hooks/charmhelpers/contrib/hardening/ssh/checks/config.py 1970-01-01 00:00:00 +0000 |
3884 | @@ -1,394 +0,0 @@ |
3885 | -# Copyright 2016 Canonical Limited. |
3886 | -# |
3887 | -# This file is part of charm-helpers. |
3888 | -# |
3889 | -# charm-helpers is free software: you can redistribute it and/or modify |
3890 | -# it under the terms of the GNU Lesser General Public License version 3 as |
3891 | -# published by the Free Software Foundation. |
3892 | -# |
3893 | -# charm-helpers is distributed in the hope that it will be useful, |
3894 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
3895 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
3896 | -# GNU Lesser General Public License for more details. |
3897 | -# |
3898 | -# You should have received a copy of the GNU Lesser General Public License |
3899 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
3900 | - |
3901 | -import os |
3902 | - |
3903 | -from charmhelpers.core.hookenv import ( |
3904 | - log, |
3905 | - DEBUG, |
3906 | -) |
3907 | -from charmhelpers.fetch import ( |
3908 | - apt_install, |
3909 | - apt_update, |
3910 | -) |
3911 | -from charmhelpers.core.host import lsb_release |
3912 | -from charmhelpers.contrib.hardening.audits.file import ( |
3913 | - TemplatedFile, |
3914 | - FileContentAudit, |
3915 | -) |
3916 | -from charmhelpers.contrib.hardening.ssh import TEMPLATES_DIR |
3917 | -from charmhelpers.contrib.hardening import utils |
3918 | - |
3919 | - |
3920 | -def get_audits(): |
3921 | - """Get SSH hardening config audits. |
3922 | - |
3923 | - :returns: dictionary of audits |
3924 | - """ |
3925 | - audits = [SSHConfig(), SSHDConfig(), SSHConfigFileContentAudit(), |
3926 | - SSHDConfigFileContentAudit()] |
3927 | - return audits |
3928 | - |
3929 | - |
3930 | -class SSHConfigContext(object): |
3931 | - |
3932 | - type = 'client' |
3933 | - |
3934 | - def get_macs(self, allow_weak_mac): |
3935 | - if allow_weak_mac: |
3936 | - weak_macs = 'weak' |
3937 | - else: |
3938 | - weak_macs = 'default' |
3939 | - |
3940 | - default = 'hmac-sha2-512,hmac-sha2-256,hmac-ripemd160' |
3941 | - macs = {'default': default, |
3942 | - 'weak': default + ',hmac-sha1'} |
3943 | - |
3944 | - default = ('hmac-sha2-512-etm@openssh.com,' |
3945 | - 'hmac-sha2-256-etm@openssh.com,' |
3946 | - 'hmac-ripemd160-etm@openssh.com,umac-128-etm@openssh.com,' |
3947 | - 'hmac-sha2-512,hmac-sha2-256,hmac-ripemd160') |
3948 | - macs_66 = {'default': default, |
3949 | - 'weak': default + ',hmac-sha1'} |
3950 | - |
3951 | - # Use newer ciphers on Ubuntu Trusty and above |
3952 | - if lsb_release()['DISTRIB_CODENAME'].lower() >= 'trusty': |
3953 | - log("Detected Ubuntu 14.04 or newer, using new macs", level=DEBUG) |
3954 | - macs = macs_66 |
3955 | - |
3956 | - return macs[weak_macs] |
3957 | - |
3958 | - def get_kexs(self, allow_weak_kex): |
3959 | - if allow_weak_kex: |
3960 | - weak_kex = 'weak' |
3961 | - else: |
3962 | - weak_kex = 'default' |
3963 | - |
3964 | - default = 'diffie-hellman-group-exchange-sha256' |
3965 | - weak = (default + ',diffie-hellman-group14-sha1,' |
3966 | - 'diffie-hellman-group-exchange-sha1,' |
3967 | - 'diffie-hellman-group1-sha1') |
3968 | - kex = {'default': default, |
3969 | - 'weak': weak} |
3970 | - |
3971 | - default = ('curve25519-sha256@libssh.org,' |
3972 | - 'diffie-hellman-group-exchange-sha256') |
3973 | - weak = (default + ',diffie-hellman-group14-sha1,' |
3974 | - 'diffie-hellman-group-exchange-sha1,' |
3975 | - 'diffie-hellman-group1-sha1') |
3976 | - kex_66 = {'default': default, |
3977 | - 'weak': weak} |
3978 | - |
3979 | - # Use newer kex on Ubuntu Trusty and above |
3980 | - if lsb_release()['DISTRIB_CODENAME'].lower() >= 'trusty': |
3981 | - log('Detected Ubuntu 14.04 or newer, using new key exchange ' |
3982 | - 'algorithms', level=DEBUG) |
3983 | - kex = kex_66 |
3984 | - |
3985 | - return kex[weak_kex] |
3986 | - |
3987 | - def get_ciphers(self, cbc_required): |
3988 | - if cbc_required: |
3989 | - weak_ciphers = 'weak' |
3990 | - else: |
3991 | - weak_ciphers = 'default' |
3992 | - |
3993 | - default = 'aes256-ctr,aes192-ctr,aes128-ctr' |
3994 | - cipher = {'default': default, |
3995 | - 'weak': default + 'aes256-cbc,aes192-cbc,aes128-cbc'} |
3996 | - |
3997 | - default = ('chacha20-poly1305@openssh.com,aes256-gcm@openssh.com,' |
3998 | - 'aes128-gcm@openssh.com,aes256-ctr,aes192-ctr,aes128-ctr') |
3999 | - ciphers_66 = {'default': default, |
4000 | - 'weak': default + ',aes256-cbc,aes192-cbc,aes128-cbc'} |
4001 | - |
4002 | - # Use newer ciphers on ubuntu Trusty and above |
4003 | - if lsb_release()['DISTRIB_CODENAME'].lower() >= 'trusty': |
4004 | - log('Detected Ubuntu 14.04 or newer, using new ciphers', |
4005 | - level=DEBUG) |
4006 | - cipher = ciphers_66 |
4007 | - |
4008 | - return cipher[weak_ciphers] |
4009 | - |
4010 | - def __call__(self): |
4011 | - settings = utils.get_settings('ssh') |
4012 | - if settings['common']['network_ipv6_enable']: |
4013 | - addr_family = 'any' |
4014 | - else: |
4015 | - addr_family = 'inet' |
4016 | - |
4017 | - ctxt = { |
4018 | - 'addr_family': addr_family, |
4019 | - 'remote_hosts': settings['common']['remote_hosts'], |
4020 | - 'password_auth_allowed': |
4021 | - settings['client']['password_authentication'], |
4022 | - 'ports': settings['common']['ports'], |
4023 | - 'ciphers': self.get_ciphers(settings['client']['cbc_required']), |
4024 | - 'macs': self.get_macs(settings['client']['weak_hmac']), |
4025 | - 'kexs': self.get_kexs(settings['client']['weak_kex']), |
4026 | - 'roaming': settings['client']['roaming'], |
4027 | - } |
4028 | - return ctxt |
4029 | - |
4030 | - |
4031 | -class SSHConfig(TemplatedFile): |
4032 | - def __init__(self): |
4033 | - path = '/etc/ssh/ssh_config' |
4034 | - super(SSHConfig, self).__init__(path=path, |
4035 | - template_dir=TEMPLATES_DIR, |
4036 | - context=SSHConfigContext(), |
4037 | - user='root', |
4038 | - group='root', |
4039 | - mode=0o0644) |
4040 | - |
4041 | - def pre_write(self): |
4042 | - settings = utils.get_settings('ssh') |
4043 | - apt_update(fatal=True) |
4044 | - apt_install(settings['client']['package']) |
4045 | - if not os.path.exists('/etc/ssh'): |
4046 | - os.makedir('/etc/ssh') |
4047 | - # NOTE: don't recurse |
4048 | - utils.ensure_permissions('/etc/ssh', 'root', 'root', 0o0755, |
4049 | - maxdepth=0) |
4050 | - |
4051 | - def post_write(self): |
4052 | - # NOTE: don't recurse |
4053 | - utils.ensure_permissions('/etc/ssh', 'root', 'root', 0o0755, |
4054 | - maxdepth=0) |
4055 | - |
4056 | - |
4057 | -class SSHDConfigContext(SSHConfigContext): |
4058 | - |
4059 | - type = 'server' |
4060 | - |
4061 | - def __call__(self): |
4062 | - settings = utils.get_settings('ssh') |
4063 | - if settings['common']['network_ipv6_enable']: |
4064 | - addr_family = 'any' |
4065 | - else: |
4066 | - addr_family = 'inet' |
4067 | - |
4068 | - ctxt = { |
4069 | - 'ssh_ip': settings['server']['listen_to'], |
4070 | - 'password_auth_allowed': |
4071 | - settings['server']['password_authentication'], |
4072 | - 'ports': settings['common']['ports'], |
4073 | - 'addr_family': addr_family, |
4074 | - 'ciphers': self.get_ciphers(settings['server']['cbc_required']), |
4075 | - 'macs': self.get_macs(settings['server']['weak_hmac']), |
4076 | - 'kexs': self.get_kexs(settings['server']['weak_kex']), |
4077 | - 'host_key_files': settings['server']['host_key_files'], |
4078 | - 'allow_root_with_key': settings['server']['allow_root_with_key'], |
4079 | - 'password_authentication': |
4080 | - settings['server']['password_authentication'], |
4081 | - 'use_priv_sep': settings['server']['use_privilege_separation'], |
4082 | - 'use_pam': settings['server']['use_pam'], |
4083 | - 'allow_x11_forwarding': settings['server']['allow_x11_forwarding'], |
4084 | - 'print_motd': settings['server']['print_motd'], |
4085 | - 'print_last_log': settings['server']['print_last_log'], |
4086 | - 'client_alive_interval': |
4087 | - settings['server']['alive_interval'], |
4088 | - 'client_alive_count': settings['server']['alive_count'], |
4089 | - 'allow_tcp_forwarding': settings['server']['allow_tcp_forwarding'], |
4090 | - 'allow_agent_forwarding': |
4091 | - settings['server']['allow_agent_forwarding'], |
4092 | - 'deny_users': settings['server']['deny_users'], |
4093 | - 'allow_users': settings['server']['allow_users'], |
4094 | - 'deny_groups': settings['server']['deny_groups'], |
4095 | - 'allow_groups': settings['server']['allow_groups'], |
4096 | - 'use_dns': settings['server']['use_dns'], |
4097 | - 'sftp_enable': settings['server']['sftp_enable'], |
4098 | - 'sftp_group': settings['server']['sftp_group'], |
4099 | - 'sftp_chroot': settings['server']['sftp_chroot'], |
4100 | - 'max_auth_tries': settings['server']['max_auth_tries'], |
4101 | - 'max_sessions': settings['server']['max_sessions'], |
4102 | - } |
4103 | - return ctxt |
4104 | - |
4105 | - |
4106 | -class SSHDConfig(TemplatedFile): |
4107 | - def __init__(self): |
4108 | - path = '/etc/ssh/sshd_config' |
4109 | - super(SSHDConfig, self).__init__(path=path, |
4110 | - template_dir=TEMPLATES_DIR, |
4111 | - context=SSHDConfigContext(), |
4112 | - user='root', |
4113 | - group='root', |
4114 | - mode=0o0600, |
4115 | - service_actions=[{'service': 'ssh', |
4116 | - 'actions': |
4117 | - ['restart']}]) |
4118 | - |
4119 | - def pre_write(self): |
4120 | - settings = utils.get_settings('ssh') |
4121 | - apt_update(fatal=True) |
4122 | - apt_install(settings['server']['package']) |
4123 | - if not os.path.exists('/etc/ssh'): |
4124 | - os.makedir('/etc/ssh') |
4125 | - # NOTE: don't recurse |
4126 | - utils.ensure_permissions('/etc/ssh', 'root', 'root', 0o0755, |
4127 | - maxdepth=0) |
4128 | - |
4129 | - def post_write(self): |
4130 | - # NOTE: don't recurse |
4131 | - utils.ensure_permissions('/etc/ssh', 'root', 'root', 0o0755, |
4132 | - maxdepth=0) |
4133 | - |
4134 | - |
4135 | -class SSHConfigFileContentAudit(FileContentAudit): |
4136 | - def __init__(self): |
4137 | - self.path = '/etc/ssh/ssh_config' |
4138 | - super(SSHConfigFileContentAudit, self).__init__(self.path, {}) |
4139 | - |
4140 | - def is_compliant(self, *args, **kwargs): |
4141 | - self.pass_cases = [] |
4142 | - self.fail_cases = [] |
4143 | - settings = utils.get_settings('ssh') |
4144 | - |
4145 | - if lsb_release()['DISTRIB_CODENAME'].lower() >= 'trusty': |
4146 | - if not settings['server']['weak_hmac']: |
4147 | - self.pass_cases.append(r'^MACs.+,hmac-ripemd160$') |
4148 | - else: |
4149 | - self.pass_cases.append(r'^MACs.+,hmac-sha1$') |
4150 | - |
4151 | - if settings['server']['weak_kex']: |
4152 | - self.fail_cases.append(r'^KexAlgorithms\sdiffie-hellman-group-exchange-sha256[,\s]?') # noqa |
4153 | - self.pass_cases.append(r'^KexAlgorithms\sdiffie-hellman-group14-sha1[,\s]?') # noqa |
4154 | - self.pass_cases.append(r'^KexAlgorithms\sdiffie-hellman-group-exchange-sha1[,\s]?') # noqa |
4155 | - self.pass_cases.append(r'^KexAlgorithms\sdiffie-hellman-group1-sha1[,\s]?') # noqa |
4156 | - else: |
4157 | - self.pass_cases.append(r'^KexAlgorithms.+,diffie-hellman-group-exchange-sha256$') # noqa |
4158 | - self.fail_cases.append(r'^KexAlgorithms.*diffie-hellman-group14-sha1[,\s]?') # noqa |
4159 | - |
4160 | - if settings['server']['cbc_required']: |
4161 | - self.pass_cases.append(r'^Ciphers\s.*-cbc[,\s]?') |
4162 | - self.fail_cases.append(r'^Ciphers\s.*aes128-ctr[,\s]?') |
4163 | - self.fail_cases.append(r'^Ciphers\s.*aes192-ctr[,\s]?') |
4164 | - self.fail_cases.append(r'^Ciphers\s.*aes256-ctr[,\s]?') |
4165 | - else: |
4166 | - self.fail_cases.append(r'^Ciphers\s.*-cbc[,\s]?') |
4167 | - self.pass_cases.append(r'^Ciphers\schacha20-poly1305@openssh.com,.+') # noqa |
4168 | - self.pass_cases.append(r'^Ciphers\s.*aes128-ctr$') |
4169 | - self.pass_cases.append(r'^Ciphers\s.*aes192-ctr[,\s]?') |
4170 | - self.pass_cases.append(r'^Ciphers\s.*aes256-ctr[,\s]?') |
4171 | - else: |
4172 | - if not settings['client']['weak_hmac']: |
4173 | - self.fail_cases.append(r'^MACs.+,hmac-sha1$') |
4174 | - else: |
4175 | - self.pass_cases.append(r'^MACs.+,hmac-sha1$') |
4176 | - |
4177 | - if settings['client']['weak_kex']: |
4178 | - self.fail_cases.append(r'^KexAlgorithms\sdiffie-hellman-group-exchange-sha256[,\s]?') # noqa |
4179 | - self.pass_cases.append(r'^KexAlgorithms\sdiffie-hellman-group14-sha1[,\s]?') # noqa |
4180 | - self.pass_cases.append(r'^KexAlgorithms\sdiffie-hellman-group-exchange-sha1[,\s]?') # noqa |
4181 | - self.pass_cases.append(r'^KexAlgorithms\sdiffie-hellman-group1-sha1[,\s]?') # noqa |
4182 | - else: |
4183 | - self.pass_cases.append(r'^KexAlgorithms\sdiffie-hellman-group-exchange-sha256$') # noqa |
4184 | - self.fail_cases.append(r'^KexAlgorithms\sdiffie-hellman-group14-sha1[,\s]?') # noqa |
4185 | - self.fail_cases.append(r'^KexAlgorithms\sdiffie-hellman-group-exchange-sha1[,\s]?') # noqa |
4186 | - self.fail_cases.append(r'^KexAlgorithms\sdiffie-hellman-group1-sha1[,\s]?') # noqa |
4187 | - |
4188 | - if settings['client']['cbc_required']: |
4189 | - self.pass_cases.append(r'^Ciphers\s.*-cbc[,\s]?') |
4190 | - self.fail_cases.append(r'^Ciphers\s.*aes128-ctr[,\s]?') |
4191 | - self.fail_cases.append(r'^Ciphers\s.*aes192-ctr[,\s]?') |
4192 | - self.fail_cases.append(r'^Ciphers\s.*aes256-ctr[,\s]?') |
4193 | - else: |
4194 | - self.fail_cases.append(r'^Ciphers\s.*-cbc[,\s]?') |
4195 | - self.pass_cases.append(r'^Ciphers\s.*aes128-ctr[,\s]?') |
4196 | - self.pass_cases.append(r'^Ciphers\s.*aes192-ctr[,\s]?') |
4197 | - self.pass_cases.append(r'^Ciphers\s.*aes256-ctr[,\s]?') |
4198 | - |
4199 | - if settings['client']['roaming']: |
4200 | - self.pass_cases.append(r'^UseRoaming yes$') |
4201 | - else: |
4202 | - self.fail_cases.append(r'^UseRoaming yes$') |
4203 | - |
4204 | - return super(SSHConfigFileContentAudit, self).is_compliant(*args, |
4205 | - **kwargs) |
4206 | - |
4207 | - |
4208 | -class SSHDConfigFileContentAudit(FileContentAudit): |
4209 | - def __init__(self): |
4210 | - self.path = '/etc/ssh/sshd_config' |
4211 | - super(SSHDConfigFileContentAudit, self).__init__(self.path, {}) |
4212 | - |
4213 | - def is_compliant(self, *args, **kwargs): |
4214 | - self.pass_cases = [] |
4215 | - self.fail_cases = [] |
4216 | - settings = utils.get_settings('ssh') |
4217 | - |
4218 | - if lsb_release()['DISTRIB_CODENAME'].lower() >= 'trusty': |
4219 | - if not settings['server']['weak_hmac']: |
4220 | - self.pass_cases.append(r'^MACs.+,hmac-ripemd160$') |
4221 | - else: |
4222 | - self.pass_cases.append(r'^MACs.+,hmac-sha1$') |
4223 | - |
4224 | - if settings['server']['weak_kex']: |
4225 | - self.fail_cases.append(r'^KexAlgorithms\sdiffie-hellman-group-exchange-sha256[,\s]?') # noqa |
4226 | - self.pass_cases.append(r'^KexAlgorithms\sdiffie-hellman-group14-sha1[,\s]?') # noqa |
4227 | - self.pass_cases.append(r'^KexAlgorithms\sdiffie-hellman-group-exchange-sha1[,\s]?') # noqa |
4228 | - self.pass_cases.append(r'^KexAlgorithms\sdiffie-hellman-group1-sha1[,\s]?') # noqa |
4229 | - else: |
4230 | - self.pass_cases.append(r'^KexAlgorithms.+,diffie-hellman-group-exchange-sha256$') # noqa |
4231 | - self.fail_cases.append(r'^KexAlgorithms.*diffie-hellman-group14-sha1[,\s]?') # noqa |
4232 | - |
4233 | - if settings['server']['cbc_required']: |
4234 | - self.pass_cases.append(r'^Ciphers\s.*-cbc[,\s]?') |
4235 | - self.fail_cases.append(r'^Ciphers\s.*aes128-ctr[,\s]?') |
4236 | - self.fail_cases.append(r'^Ciphers\s.*aes192-ctr[,\s]?') |
4237 | - self.fail_cases.append(r'^Ciphers\s.*aes256-ctr[,\s]?') |
4238 | - else: |
4239 | - self.fail_cases.append(r'^Ciphers\s.*-cbc[,\s]?') |
4240 | - self.pass_cases.append(r'^Ciphers\schacha20-poly1305@openssh.com,.+') # noqa |
4241 | - self.pass_cases.append(r'^Ciphers\s.*aes128-ctr$') |
4242 | - self.pass_cases.append(r'^Ciphers\s.*aes192-ctr[,\s]?') |
4243 | - self.pass_cases.append(r'^Ciphers\s.*aes256-ctr[,\s]?') |
4244 | - else: |
4245 | - if not settings['server']['weak_hmac']: |
4246 | - self.pass_cases.append(r'^MACs.+,hmac-ripemd160$') |
4247 | - else: |
4248 | - self.pass_cases.append(r'^MACs.+,hmac-sha1$') |
4249 | - |
4250 | - if settings['server']['weak_kex']: |
4251 | - self.fail_cases.append(r'^KexAlgorithms\sdiffie-hellman-group-exchange-sha256[,\s]?') # noqa |
4252 | - self.pass_cases.append(r'^KexAlgorithms\sdiffie-hellman-group14-sha1[,\s]?') # noqa |
4253 | - self.pass_cases.append(r'^KexAlgorithms\sdiffie-hellman-group-exchange-sha1[,\s]?') # noqa |
4254 | - self.pass_cases.append(r'^KexAlgorithms\sdiffie-hellman-group1-sha1[,\s]?') # noqa |
4255 | - else: |
4256 | - self.pass_cases.append(r'^KexAlgorithms\sdiffie-hellman-group-exchange-sha256$') # noqa |
4257 | - self.fail_cases.append(r'^KexAlgorithms\sdiffie-hellman-group14-sha1[,\s]?') # noqa |
4258 | - self.fail_cases.append(r'^KexAlgorithms\sdiffie-hellman-group-exchange-sha1[,\s]?') # noqa |
4259 | - self.fail_cases.append(r'^KexAlgorithms\sdiffie-hellman-group1-sha1[,\s]?') # noqa |
4260 | - |
4261 | - if settings['server']['cbc_required']: |
4262 | - self.pass_cases.append(r'^Ciphers\s.*-cbc[,\s]?') |
4263 | - self.fail_cases.append(r'^Ciphers\s.*aes128-ctr[,\s]?') |
4264 | - self.fail_cases.append(r'^Ciphers\s.*aes192-ctr[,\s]?') |
4265 | - self.fail_cases.append(r'^Ciphers\s.*aes256-ctr[,\s]?') |
4266 | - else: |
4267 | - self.fail_cases.append(r'^Ciphers\s.*-cbc[,\s]?') |
4268 | - self.pass_cases.append(r'^Ciphers\s.*aes128-ctr[,\s]?') |
4269 | - self.pass_cases.append(r'^Ciphers\s.*aes192-ctr[,\s]?') |
4270 | - self.pass_cases.append(r'^Ciphers\s.*aes256-ctr[,\s]?') |
4271 | - |
4272 | - if settings['server']['sftp_enable']: |
4273 | - self.pass_cases.append(r'^Subsystem\ssftp') |
4274 | - else: |
4275 | - self.fail_cases.append(r'^Subsystem\ssftp') |
4276 | - |
4277 | - return super(SSHDConfigFileContentAudit, self).is_compliant(*args, |
4278 | - **kwargs) |
4279 | |
4280 | === removed file 'hooks/charmhelpers/contrib/hardening/templating.py' |
4281 | --- hooks/charmhelpers/contrib/hardening/templating.py 2016-04-22 08:15:33 +0000 |
4282 | +++ hooks/charmhelpers/contrib/hardening/templating.py 1970-01-01 00:00:00 +0000 |
4283 | @@ -1,71 +0,0 @@ |
4284 | -# Copyright 2016 Canonical Limited. |
4285 | -# |
4286 | -# This file is part of charm-helpers. |
4287 | -# |
4288 | -# charm-helpers is free software: you can redistribute it and/or modify |
4289 | -# it under the terms of the GNU Lesser General Public License version 3 as |
4290 | -# published by the Free Software Foundation. |
4291 | -# |
4292 | -# charm-helpers is distributed in the hope that it will be useful, |
4293 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
4294 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
4295 | -# GNU Lesser General Public License for more details. |
4296 | -# |
4297 | -# You should have received a copy of the GNU Lesser General Public License |
4298 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
4299 | - |
4300 | -import os |
4301 | - |
4302 | -from charmhelpers.core.hookenv import ( |
4303 | - log, |
4304 | - DEBUG, |
4305 | - WARNING, |
4306 | -) |
4307 | - |
4308 | -try: |
4309 | - from jinja2 import FileSystemLoader, Environment |
4310 | -except ImportError: |
4311 | - from charmhelpers.fetch import apt_install |
4312 | - from charmhelpers.fetch import apt_update |
4313 | - apt_update(fatal=True) |
4314 | - apt_install('python-jinja2', fatal=True) |
4315 | - from jinja2 import FileSystemLoader, Environment |
4316 | - |
4317 | - |
4318 | -# NOTE: function separated from main rendering code to facilitate easier |
4319 | -# mocking in unit tests. |
4320 | -def write(path, data): |
4321 | - with open(path, 'wb') as out: |
4322 | - out.write(data) |
4323 | - |
4324 | - |
4325 | -def get_template_path(template_dir, path): |
4326 | - """Returns the template file which would be used to render the path. |
4327 | - |
4328 | - The path to the template file is returned. |
4329 | - :param template_dir: the directory the templates are located in |
4330 | - :param path: the file path to be written to. |
4331 | - :returns: path to the template file |
4332 | - """ |
4333 | - return os.path.join(template_dir, os.path.basename(path)) |
4334 | - |
4335 | - |
4336 | -def render_and_write(template_dir, path, context): |
4337 | - """Renders the specified template into the file. |
4338 | - |
4339 | - :param template_dir: the directory to load the template from |
4340 | - :param path: the path to write the templated contents to |
4341 | - :param context: the parameters to pass to the rendering engine |
4342 | - """ |
4343 | - env = Environment(loader=FileSystemLoader(template_dir)) |
4344 | - template_file = os.path.basename(path) |
4345 | - template = env.get_template(template_file) |
4346 | - log('Rendering from template: %s' % template.name, level=DEBUG) |
4347 | - rendered_content = template.render(context) |
4348 | - if not rendered_content: |
4349 | - log("Render returned None - skipping '%s'" % path, |
4350 | - level=WARNING) |
4351 | - return |
4352 | - |
4353 | - write(path, rendered_content.encode('utf-8').strip()) |
4354 | - log('Wrote template %s' % path, level=DEBUG) |
4355 | |
4356 | === removed file 'hooks/charmhelpers/contrib/hardening/utils.py' |
4357 | --- hooks/charmhelpers/contrib/hardening/utils.py 2016-04-22 08:15:33 +0000 |
4358 | +++ hooks/charmhelpers/contrib/hardening/utils.py 1970-01-01 00:00:00 +0000 |
4359 | @@ -1,157 +0,0 @@ |
4360 | -# Copyright 2016 Canonical Limited. |
4361 | -# |
4362 | -# This file is part of charm-helpers. |
4363 | -# |
4364 | -# charm-helpers is free software: you can redistribute it and/or modify |
4365 | -# it under the terms of the GNU Lesser General Public License version 3 as |
4366 | -# published by the Free Software Foundation. |
4367 | -# |
4368 | -# charm-helpers is distributed in the hope that it will be useful, |
4369 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
4370 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
4371 | -# GNU Lesser General Public License for more details. |
4372 | -# |
4373 | -# You should have received a copy of the GNU Lesser General Public License |
4374 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
4375 | - |
4376 | -import glob |
4377 | -import grp |
4378 | -import os |
4379 | -import pwd |
4380 | -import six |
4381 | -import yaml |
4382 | - |
4383 | -from charmhelpers.core.hookenv import ( |
4384 | - log, |
4385 | - DEBUG, |
4386 | - INFO, |
4387 | - WARNING, |
4388 | - ERROR, |
4389 | -) |
4390 | - |
4391 | - |
4392 | -# Global settings cache. Since each hook fire entails a fresh module import it |
4393 | -# is safe to hold this in memory and not risk missing config changes (since |
4394 | -# they will result in a new hook fire and thus re-import). |
4395 | -__SETTINGS__ = {} |
4396 | - |
4397 | - |
4398 | -def _get_defaults(modules): |
4399 | - """Load the default config for the provided modules. |
4400 | - |
4401 | - :param modules: stack modules config defaults to lookup. |
4402 | - :returns: modules default config dictionary. |
4403 | - """ |
4404 | - default = os.path.join(os.path.dirname(__file__), |
4405 | - 'defaults/%s.yaml' % (modules)) |
4406 | - return yaml.safe_load(open(default)) |
4407 | - |
4408 | - |
4409 | -def _get_schema(modules): |
4410 | - """Load the config schema for the provided modules. |
4411 | - |
4412 | - NOTE: this schema is intended to have 1-1 relationship with they keys in |
4413 | - the default config and is used a means to verify valid overrides provided |
4414 | - by the user. |
4415 | - |
4416 | - :param modules: stack modules config schema to lookup. |
4417 | - :returns: modules default schema dictionary. |
4418 | - """ |
4419 | - schema = os.path.join(os.path.dirname(__file__), |
4420 | - 'defaults/%s.yaml.schema' % (modules)) |
4421 | - return yaml.safe_load(open(schema)) |
4422 | - |
4423 | - |
4424 | -def _get_user_provided_overrides(modules): |
4425 | - """Load user-provided config overrides. |
4426 | - |
4427 | - :param modules: stack modules to lookup in user overrides yaml file. |
4428 | - :returns: overrides dictionary. |
4429 | - """ |
4430 | - overrides = os.path.join(os.environ['JUJU_CHARM_DIR'], |
4431 | - 'hardening.yaml') |
4432 | - if os.path.exists(overrides): |
4433 | - log("Found user-provided config overrides file '%s'" % |
4434 | - (overrides), level=DEBUG) |
4435 | - settings = yaml.safe_load(open(overrides)) |
4436 | - if settings and settings.get(modules): |
4437 | - log("Applying '%s' overrides" % (modules), level=DEBUG) |
4438 | - return settings.get(modules) |
4439 | - |
4440 | - log("No overrides found for '%s'" % (modules), level=DEBUG) |
4441 | - else: |
4442 | - log("No hardening config overrides file '%s' found in charm " |
4443 | - "root dir" % (overrides), level=DEBUG) |
4444 | - |
4445 | - return {} |
4446 | - |
4447 | - |
4448 | -def _apply_overrides(settings, overrides, schema): |
4449 | - """Get overrides config overlayed onto modules defaults. |
4450 | - |
4451 | - :param modules: require stack modules config. |
4452 | - :returns: dictionary of modules config with user overrides applied. |
4453 | - """ |
4454 | - if overrides: |
4455 | - for k, v in six.iteritems(overrides): |
4456 | - if k in schema: |
4457 | - if schema[k] is None: |
4458 | - settings[k] = v |
4459 | - elif type(schema[k]) is dict: |
4460 | - settings[k] = _apply_overrides(settings[k], overrides[k], |
4461 | - schema[k]) |
4462 | - else: |
4463 | - raise Exception("Unexpected type found in schema '%s'" % |
4464 | - type(schema[k]), level=ERROR) |
4465 | - else: |
4466 | - log("Unknown override key '%s' - ignoring" % (k), level=INFO) |
4467 | - |
4468 | - return settings |
4469 | - |
4470 | - |
4471 | -def get_settings(modules): |
4472 | - global __SETTINGS__ |
4473 | - if modules in __SETTINGS__: |
4474 | - return __SETTINGS__[modules] |
4475 | - |
4476 | - schema = _get_schema(modules) |
4477 | - settings = _get_defaults(modules) |
4478 | - overrides = _get_user_provided_overrides(modules) |
4479 | - __SETTINGS__[modules] = _apply_overrides(settings, overrides, schema) |
4480 | - return __SETTINGS__[modules] |
4481 | - |
4482 | - |
4483 | -def ensure_permissions(path, user, group, permissions, maxdepth=-1): |
4484 | - """Ensure permissions for path. |
4485 | - |
4486 | - If path is a file, apply to file and return. If path is a directory, |
4487 | - apply recursively (if required) to directory contents and return. |
4488 | - |
4489 | - :param user: user name |
4490 | - :param group: group name |
4491 | - :param permissions: octal permissions |
4492 | - :param maxdepth: maximum recursion depth. A negative maxdepth allows |
4493 | - infinite recursion and maxdepth=0 means no recursion. |
4494 | - :returns: None |
4495 | - """ |
4496 | - if not os.path.exists(path): |
4497 | - log("File '%s' does not exist - cannot set permissions" % (path), |
4498 | - level=WARNING) |
4499 | - return |
4500 | - |
4501 | - _user = pwd.getpwnam(user) |
4502 | - os.chown(path, _user.pw_uid, grp.getgrnam(group).gr_gid) |
4503 | - os.chmod(path, permissions) |
4504 | - |
4505 | - if maxdepth == 0: |
4506 | - log("Max recursion depth reached - skipping further recursion", |
4507 | - level=DEBUG) |
4508 | - return |
4509 | - elif maxdepth > 0: |
4510 | - maxdepth -= 1 |
4511 | - |
4512 | - if os.path.isdir(path): |
4513 | - contents = glob.glob("%s/*" % (path)) |
4514 | - for c in contents: |
4515 | - ensure_permissions(c, user=user, group=group, |
4516 | - permissions=permissions, maxdepth=maxdepth) |
4517 | |
4518 | === removed directory 'hooks/charmhelpers/contrib/mellanox' |
4519 | === removed file 'hooks/charmhelpers/contrib/mellanox/__init__.py' |
4520 | === removed file 'hooks/charmhelpers/contrib/mellanox/infiniband.py' |
4521 | --- hooks/charmhelpers/contrib/mellanox/infiniband.py 2016-01-30 22:44:06 +0000 |
4522 | +++ hooks/charmhelpers/contrib/mellanox/infiniband.py 1970-01-01 00:00:00 +0000 |
4523 | @@ -1,151 +0,0 @@ |
4524 | -#!/usr/bin/env python |
4525 | -# -*- coding: utf-8 -*- |
4526 | - |
4527 | -# Copyright 2014-2015 Canonical Limited. |
4528 | -# |
4529 | -# This file is part of charm-helpers. |
4530 | -# |
4531 | -# charm-helpers is free software: you can redistribute it and/or modify |
4532 | -# it under the terms of the GNU Lesser General Public License version 3 as |
4533 | -# published by the Free Software Foundation. |
4534 | -# |
4535 | -# charm-helpers is distributed in the hope that it will be useful, |
4536 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
4537 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
4538 | -# GNU Lesser General Public License for more details. |
4539 | -# |
4540 | -# You should have received a copy of the GNU Lesser General Public License |
4541 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
4542 | - |
4543 | - |
4544 | -__author__ = "Jorge Niedbalski <jorge.niedbalski@canonical.com>" |
4545 | - |
4546 | -from charmhelpers.fetch import ( |
4547 | - apt_install, |
4548 | - apt_update, |
4549 | -) |
4550 | - |
4551 | -from charmhelpers.core.hookenv import ( |
4552 | - log, |
4553 | - INFO, |
4554 | -) |
4555 | - |
4556 | -try: |
4557 | - from netifaces import interfaces as network_interfaces |
4558 | -except ImportError: |
4559 | - apt_install('python-netifaces') |
4560 | - from netifaces import interfaces as network_interfaces |
4561 | - |
4562 | -import os |
4563 | -import re |
4564 | -import subprocess |
4565 | - |
4566 | -from charmhelpers.core.kernel import modprobe |
4567 | - |
4568 | -REQUIRED_MODULES = ( |
4569 | - "mlx4_ib", |
4570 | - "mlx4_en", |
4571 | - "mlx4_core", |
4572 | - "ib_ipath", |
4573 | - "ib_mthca", |
4574 | - "ib_srpt", |
4575 | - "ib_srp", |
4576 | - "ib_ucm", |
4577 | - "ib_isert", |
4578 | - "ib_iser", |
4579 | - "ib_ipoib", |
4580 | - "ib_cm", |
4581 | - "ib_uverbs" |
4582 | - "ib_umad", |
4583 | - "ib_sa", |
4584 | - "ib_mad", |
4585 | - "ib_core", |
4586 | - "ib_addr", |
4587 | - "rdma_ucm", |
4588 | -) |
4589 | - |
4590 | -REQUIRED_PACKAGES = ( |
4591 | - "ibutils", |
4592 | - "infiniband-diags", |
4593 | - "ibverbs-utils", |
4594 | -) |
4595 | - |
4596 | -IPOIB_DRIVERS = ( |
4597 | - "ib_ipoib", |
4598 | -) |
4599 | - |
4600 | -ABI_VERSION_FILE = "/sys/class/infiniband_mad/abi_version" |
4601 | - |
4602 | - |
4603 | -class DeviceInfo(object): |
4604 | - pass |
4605 | - |
4606 | - |
4607 | -def install_packages(): |
4608 | - apt_update() |
4609 | - apt_install(REQUIRED_PACKAGES, fatal=True) |
4610 | - |
4611 | - |
4612 | -def load_modules(): |
4613 | - for module in REQUIRED_MODULES: |
4614 | - modprobe(module, persist=True) |
4615 | - |
4616 | - |
4617 | -def is_enabled(): |
4618 | - """Check if infiniband is loaded on the system""" |
4619 | - return os.path.exists(ABI_VERSION_FILE) |
4620 | - |
4621 | - |
4622 | -def stat(): |
4623 | - """Return full output of ibstat""" |
4624 | - return subprocess.check_output(["ibstat"]) |
4625 | - |
4626 | - |
4627 | -def devices(): |
4628 | - """Returns a list of IB enabled devices""" |
4629 | - return subprocess.check_output(['ibstat', '-l']).splitlines() |
4630 | - |
4631 | - |
4632 | -def device_info(device): |
4633 | - """Returns a DeviceInfo object with the current device settings""" |
4634 | - |
4635 | - status = subprocess.check_output([ |
4636 | - 'ibstat', device, '-s']).splitlines() |
4637 | - |
4638 | - regexes = { |
4639 | - "CA type: (.*)": "device_type", |
4640 | - "Number of ports: (.*)": "num_ports", |
4641 | - "Firmware version: (.*)": "fw_ver", |
4642 | - "Hardware version: (.*)": "hw_ver", |
4643 | - "Node GUID: (.*)": "node_guid", |
4644 | - "System image GUID: (.*)": "sys_guid", |
4645 | - } |
4646 | - |
4647 | - device = DeviceInfo() |
4648 | - |
4649 | - for line in status: |
4650 | - for expression, key in regexes.items(): |
4651 | - matches = re.search(expression, line) |
4652 | - if matches: |
4653 | - setattr(device, key, matches.group(1)) |
4654 | - |
4655 | - return device |
4656 | - |
4657 | - |
4658 | -def ipoib_interfaces(): |
4659 | - """Return a list of IPOIB capable ethernet interfaces""" |
4660 | - interfaces = [] |
4661 | - |
4662 | - for interface in network_interfaces(): |
4663 | - try: |
4664 | - driver = re.search('^driver: (.+)$', subprocess.check_output([ |
4665 | - 'ethtool', '-i', |
4666 | - interface]), re.M).group(1) |
4667 | - |
4668 | - if driver in IPOIB_DRIVERS: |
4669 | - interfaces.append(interface) |
4670 | - except: |
4671 | - log("Skipping interface %s" % interface, level=INFO) |
4672 | - continue |
4673 | - |
4674 | - return interfaces |
4675 | |
4676 | === removed directory 'hooks/charmhelpers/contrib/peerstorage' |
4677 | === removed file 'hooks/charmhelpers/contrib/peerstorage/__init__.py' |
4678 | --- hooks/charmhelpers/contrib/peerstorage/__init__.py 2016-01-30 22:40:29 +0000 |
4679 | +++ hooks/charmhelpers/contrib/peerstorage/__init__.py 1970-01-01 00:00:00 +0000 |
4680 | @@ -1,269 +0,0 @@ |
4681 | -# Copyright 2014-2015 Canonical Limited. |
4682 | -# |
4683 | -# This file is part of charm-helpers. |
4684 | -# |
4685 | -# charm-helpers is free software: you can redistribute it and/or modify |
4686 | -# it under the terms of the GNU Lesser General Public License version 3 as |
4687 | -# published by the Free Software Foundation. |
4688 | -# |
4689 | -# charm-helpers is distributed in the hope that it will be useful, |
4690 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
4691 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
4692 | -# GNU Lesser General Public License for more details. |
4693 | -# |
4694 | -# You should have received a copy of the GNU Lesser General Public License |
4695 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
4696 | - |
4697 | -import json |
4698 | -import six |
4699 | - |
4700 | -from charmhelpers.core.hookenv import relation_id as current_relation_id |
4701 | -from charmhelpers.core.hookenv import ( |
4702 | - is_relation_made, |
4703 | - relation_ids, |
4704 | - relation_get as _relation_get, |
4705 | - local_unit, |
4706 | - relation_set as _relation_set, |
4707 | - leader_get as _leader_get, |
4708 | - leader_set, |
4709 | - is_leader, |
4710 | -) |
4711 | - |
4712 | - |
4713 | -""" |
4714 | -This helper provides functions to support use of a peer relation |
4715 | -for basic key/value storage, with the added benefit that all storage |
4716 | -can be replicated across peer units. |
4717 | - |
4718 | -Requirement to use: |
4719 | - |
4720 | -To use this, the "peer_echo()" method has to be called form the peer |
4721 | -relation's relation-changed hook: |
4722 | - |
4723 | -@hooks.hook("cluster-relation-changed") # Adapt the to your peer relation name |
4724 | -def cluster_relation_changed(): |
4725 | - peer_echo() |
4726 | - |
4727 | -Once this is done, you can use peer storage from anywhere: |
4728 | - |
4729 | -@hooks.hook("some-hook") |
4730 | -def some_hook(): |
4731 | - # You can store and retrieve key/values this way: |
4732 | - if is_relation_made("cluster"): # from charmhelpers.core.hookenv |
4733 | - # There are peers available so we can work with peer storage |
4734 | - peer_store("mykey", "myvalue") |
4735 | - value = peer_retrieve("mykey") |
4736 | - print value |
4737 | - else: |
4738 | - print "No peers joind the relation, cannot share key/values :(" |
4739 | -""" |
4740 | - |
4741 | - |
4742 | -def leader_get(attribute=None, rid=None): |
4743 | - """Wrapper to ensure that settings are migrated from the peer relation. |
4744 | - |
4745 | - This is to support upgrading an environment that does not support |
4746 | - Juju leadership election to one that does. |
4747 | - |
4748 | - If a setting is not extant in the leader-get but is on the relation-get |
4749 | - peer rel, it is migrated and marked as such so that it is not re-migrated. |
4750 | - """ |
4751 | - migration_key = '__leader_get_migrated_settings__' |
4752 | - if not is_leader(): |
4753 | - return _leader_get(attribute=attribute) |
4754 | - |
4755 | - settings_migrated = False |
4756 | - leader_settings = _leader_get(attribute=attribute) |
4757 | - previously_migrated = _leader_get(attribute=migration_key) |
4758 | - |
4759 | - if previously_migrated: |
4760 | - migrated = set(json.loads(previously_migrated)) |
4761 | - else: |
4762 | - migrated = set([]) |
4763 | - |
4764 | - try: |
4765 | - if migration_key in leader_settings: |
4766 | - del leader_settings[migration_key] |
4767 | - except TypeError: |
4768 | - pass |
4769 | - |
4770 | - if attribute: |
4771 | - if attribute in migrated: |
4772 | - return leader_settings |
4773 | - |
4774 | - # If attribute not present in leader db, check if this unit has set |
4775 | - # the attribute in the peer relation |
4776 | - if not leader_settings: |
4777 | - peer_setting = _relation_get(attribute=attribute, unit=local_unit(), |
4778 | - rid=rid) |
4779 | - if peer_setting: |
4780 | - leader_set(settings={attribute: peer_setting}) |
4781 | - leader_settings = peer_setting |
4782 | - |
4783 | - if leader_settings: |
4784 | - settings_migrated = True |
4785 | - migrated.add(attribute) |
4786 | - else: |
4787 | - r_settings = _relation_get(unit=local_unit(), rid=rid) |
4788 | - if r_settings: |
4789 | - for key in set(r_settings.keys()).difference(migrated): |
4790 | - # Leader setting wins |
4791 | - if not leader_settings.get(key): |
4792 | - leader_settings[key] = r_settings[key] |
4793 | - |
4794 | - settings_migrated = True |
4795 | - migrated.add(key) |
4796 | - |
4797 | - if settings_migrated: |
4798 | - leader_set(**leader_settings) |
4799 | - |
4800 | - if migrated and settings_migrated: |
4801 | - migrated = json.dumps(list(migrated)) |
4802 | - leader_set(settings={migration_key: migrated}) |
4803 | - |
4804 | - return leader_settings |
4805 | - |
4806 | - |
4807 | -def relation_set(relation_id=None, relation_settings=None, **kwargs): |
4808 | - """Attempt to use leader-set if supported in the current version of Juju, |
4809 | - otherwise falls back on relation-set. |
4810 | - |
4811 | - Note that we only attempt to use leader-set if the provided relation_id is |
4812 | - a peer relation id or no relation id is provided (in which case we assume |
4813 | - we are within the peer relation context). |
4814 | - """ |
4815 | - try: |
4816 | - if relation_id in relation_ids('cluster'): |
4817 | - return leader_set(settings=relation_settings, **kwargs) |
4818 | - else: |
4819 | - raise NotImplementedError |
4820 | - except NotImplementedError: |
4821 | - return _relation_set(relation_id=relation_id, |
4822 | - relation_settings=relation_settings, **kwargs) |
4823 | - |
4824 | - |
4825 | -def relation_get(attribute=None, unit=None, rid=None): |
4826 | - """Attempt to use leader-get if supported in the current version of Juju, |
4827 | - otherwise falls back on relation-get. |
4828 | - |
4829 | - Note that we only attempt to use leader-get if the provided rid is a peer |
4830 | - relation id or no relation id is provided (in which case we assume we are |
4831 | - within the peer relation context). |
4832 | - """ |
4833 | - try: |
4834 | - if rid in relation_ids('cluster'): |
4835 | - return leader_get(attribute, rid) |
4836 | - else: |
4837 | - raise NotImplementedError |
4838 | - except NotImplementedError: |
4839 | - return _relation_get(attribute=attribute, rid=rid, unit=unit) |
4840 | - |
4841 | - |
4842 | -def peer_retrieve(key, relation_name='cluster'): |
4843 | - """Retrieve a named key from peer relation `relation_name`.""" |
4844 | - cluster_rels = relation_ids(relation_name) |
4845 | - if len(cluster_rels) > 0: |
4846 | - cluster_rid = cluster_rels[0] |
4847 | - return relation_get(attribute=key, rid=cluster_rid, |
4848 | - unit=local_unit()) |
4849 | - else: |
4850 | - raise ValueError('Unable to detect' |
4851 | - 'peer relation {}'.format(relation_name)) |
4852 | - |
4853 | - |
4854 | -def peer_retrieve_by_prefix(prefix, relation_name='cluster', delimiter='_', |
4855 | - inc_list=None, exc_list=None): |
4856 | - """ Retrieve k/v pairs given a prefix and filter using {inc,exc}_list """ |
4857 | - inc_list = inc_list if inc_list else [] |
4858 | - exc_list = exc_list if exc_list else [] |
4859 | - peerdb_settings = peer_retrieve('-', relation_name=relation_name) |
4860 | - matched = {} |
4861 | - if peerdb_settings is None: |
4862 | - return matched |
4863 | - for k, v in peerdb_settings.items(): |
4864 | - full_prefix = prefix + delimiter |
4865 | - if k.startswith(full_prefix): |
4866 | - new_key = k.replace(full_prefix, '') |
4867 | - if new_key in exc_list: |
4868 | - continue |
4869 | - if new_key in inc_list or len(inc_list) == 0: |
4870 | - matched[new_key] = v |
4871 | - return matched |
4872 | - |
4873 | - |
4874 | -def peer_store(key, value, relation_name='cluster'): |
4875 | - """Store the key/value pair on the named peer relation `relation_name`.""" |
4876 | - cluster_rels = relation_ids(relation_name) |
4877 | - if len(cluster_rels) > 0: |
4878 | - cluster_rid = cluster_rels[0] |
4879 | - relation_set(relation_id=cluster_rid, |
4880 | - relation_settings={key: value}) |
4881 | - else: |
4882 | - raise ValueError('Unable to detect ' |
4883 | - 'peer relation {}'.format(relation_name)) |
4884 | - |
4885 | - |
4886 | -def peer_echo(includes=None, force=False): |
4887 | - """Echo filtered attributes back onto the same relation for storage. |
4888 | - |
4889 | - This is a requirement to use the peerstorage module - it needs to be called |
4890 | - from the peer relation's changed hook. |
4891 | - |
4892 | - If Juju leader support exists this will be a noop unless force is True. |
4893 | - """ |
4894 | - try: |
4895 | - is_leader() |
4896 | - except NotImplementedError: |
4897 | - pass |
4898 | - else: |
4899 | - if not force: |
4900 | - return # NOOP if leader-election is supported |
4901 | - |
4902 | - # Use original non-leader calls |
4903 | - relation_get = _relation_get |
4904 | - relation_set = _relation_set |
4905 | - |
4906 | - rdata = relation_get() |
4907 | - echo_data = {} |
4908 | - if includes is None: |
4909 | - echo_data = rdata.copy() |
4910 | - for ex in ['private-address', 'public-address']: |
4911 | - if ex in echo_data: |
4912 | - echo_data.pop(ex) |
4913 | - else: |
4914 | - for attribute, value in six.iteritems(rdata): |
4915 | - for include in includes: |
4916 | - if include in attribute: |
4917 | - echo_data[attribute] = value |
4918 | - if len(echo_data) > 0: |
4919 | - relation_set(relation_settings=echo_data) |
4920 | - |
4921 | - |
4922 | -def peer_store_and_set(relation_id=None, peer_relation_name='cluster', |
4923 | - peer_store_fatal=False, relation_settings=None, |
4924 | - delimiter='_', **kwargs): |
4925 | - """Store passed-in arguments both in argument relation and in peer storage. |
4926 | - |
4927 | - It functions like doing relation_set() and peer_store() at the same time, |
4928 | - with the same data. |
4929 | - |
4930 | - @param relation_id: the id of the relation to store the data on. Defaults |
4931 | - to the current relation. |
4932 | - @param peer_store_fatal: Set to True, the function will raise an exception |
4933 | - should the peer sotrage not be avialable.""" |
4934 | - |
4935 | - relation_settings = relation_settings if relation_settings else {} |
4936 | - relation_set(relation_id=relation_id, |
4937 | - relation_settings=relation_settings, |
4938 | - **kwargs) |
4939 | - if is_relation_made(peer_relation_name): |
4940 | - for key, value in six.iteritems(dict(list(kwargs.items()) + |
4941 | - list(relation_settings.items()))): |
4942 | - key_prefix = relation_id or current_relation_id() |
4943 | - peer_store(key_prefix + delimiter + key, |
4944 | - value, |
4945 | - relation_name=peer_relation_name) |
4946 | - else: |
4947 | - if peer_store_fatal: |
4948 | - raise ValueError('Unable to detect ' |
4949 | - 'peer relation {}'.format(peer_relation_name)) |
4950 | |
4951 | === removed directory 'hooks/charmhelpers/contrib/saltstack' |
4952 | === removed file 'hooks/charmhelpers/contrib/saltstack/__init__.py' |
4953 | --- hooks/charmhelpers/contrib/saltstack/__init__.py 2015-07-29 18:23:55 +0000 |
4954 | +++ hooks/charmhelpers/contrib/saltstack/__init__.py 1970-01-01 00:00:00 +0000 |
4955 | @@ -1,118 +0,0 @@ |
4956 | -# Copyright 2014-2015 Canonical Limited. |
4957 | -# |
4958 | -# This file is part of charm-helpers. |
4959 | -# |
4960 | -# charm-helpers is free software: you can redistribute it and/or modify |
4961 | -# it under the terms of the GNU Lesser General Public License version 3 as |
4962 | -# published by the Free Software Foundation. |
4963 | -# |
4964 | -# charm-helpers is distributed in the hope that it will be useful, |
4965 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
4966 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
4967 | -# GNU Lesser General Public License for more details. |
4968 | -# |
4969 | -# You should have received a copy of the GNU Lesser General Public License |
4970 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
4971 | - |
4972 | -"""Charm Helpers saltstack - declare the state of your machines. |
4973 | - |
4974 | -This helper enables you to declare your machine state, rather than |
4975 | -program it procedurally (and have to test each change to your procedures). |
4976 | -Your install hook can be as simple as:: |
4977 | - |
4978 | - {{{ |
4979 | - from charmhelpers.contrib.saltstack import ( |
4980 | - install_salt_support, |
4981 | - update_machine_state, |
4982 | - ) |
4983 | - |
4984 | - |
4985 | - def install(): |
4986 | - install_salt_support() |
4987 | - update_machine_state('machine_states/dependencies.yaml') |
4988 | - update_machine_state('machine_states/installed.yaml') |
4989 | - }}} |
4990 | - |
4991 | -and won't need to change (nor will its tests) when you change the machine |
4992 | -state. |
4993 | - |
4994 | -It's using a python package called salt-minion which allows various formats for |
4995 | -specifying resources, such as:: |
4996 | - |
4997 | - {{{ |
4998 | - /srv/{{ basedir }}: |
4999 | - file.directory: |
5000 | - - group: ubunet |
The diff has been truncated for viewing.