Merge lp:~junaidali/charms/trusty/plumgrid-gateway/trunk into lp:~bbaqar/charms/trusty/plumgrid-gateway/ext-interface-static
- Trusty Tahr (14.04)
- trunk
- Merge into ext-interface-static
Proposed by
Junaid Ali
Status: | Needs review |
---|---|
Proposed branch: | lp:~junaidali/charms/trusty/plumgrid-gateway/trunk |
Merge into: | lp:~bbaqar/charms/trusty/plumgrid-gateway/ext-interface-static |
Diff against target: |
6218 lines (+60/-5789) 53 files modified
charm-helpers-sync.yaml (+6/-1) config.yaml (+1/-11) hooks/charmhelpers/contrib/ansible/__init__.py (+0/-254) hooks/charmhelpers/contrib/benchmark/__init__.py (+0/-126) hooks/charmhelpers/contrib/charmhelpers/__init__.py (+0/-208) hooks/charmhelpers/contrib/charmsupport/__init__.py (+0/-15) hooks/charmhelpers/contrib/charmsupport/nrpe.py (+0/-398) hooks/charmhelpers/contrib/charmsupport/volumes.py (+0/-175) hooks/charmhelpers/contrib/database/mysql.py (+0/-412) hooks/charmhelpers/contrib/hardening/__init__.py (+0/-15) hooks/charmhelpers/contrib/hardening/apache/__init__.py (+0/-19) hooks/charmhelpers/contrib/hardening/apache/checks/__init__.py (+0/-31) hooks/charmhelpers/contrib/hardening/apache/checks/config.py (+0/-100) hooks/charmhelpers/contrib/hardening/audits/__init__.py (+0/-63) hooks/charmhelpers/contrib/hardening/audits/apache.py (+0/-100) hooks/charmhelpers/contrib/hardening/audits/apt.py (+0/-105) hooks/charmhelpers/contrib/hardening/audits/file.py (+0/-552) hooks/charmhelpers/contrib/hardening/harden.py (+0/-84) hooks/charmhelpers/contrib/hardening/host/__init__.py (+0/-19) hooks/charmhelpers/contrib/hardening/host/checks/__init__.py (+0/-50) hooks/charmhelpers/contrib/hardening/host/checks/apt.py (+0/-39) hooks/charmhelpers/contrib/hardening/host/checks/limits.py (+0/-55) hooks/charmhelpers/contrib/hardening/host/checks/login.py (+0/-67) hooks/charmhelpers/contrib/hardening/host/checks/minimize_access.py (+0/-52) hooks/charmhelpers/contrib/hardening/host/checks/pam.py (+0/-134) hooks/charmhelpers/contrib/hardening/host/checks/profile.py (+0/-45) hooks/charmhelpers/contrib/hardening/host/checks/securetty.py (+0/-39) hooks/charmhelpers/contrib/hardening/host/checks/suid_sgid.py (+0/-131) hooks/charmhelpers/contrib/hardening/host/checks/sysctl.py (+0/-211) hooks/charmhelpers/contrib/hardening/mysql/__init__.py (+0/-19) hooks/charmhelpers/contrib/hardening/mysql/checks/__init__.py (+0/-31) hooks/charmhelpers/contrib/hardening/mysql/checks/config.py (+0/-89) hooks/charmhelpers/contrib/hardening/ssh/__init__.py (+0/-19) hooks/charmhelpers/contrib/hardening/ssh/checks/__init__.py (+0/-31) hooks/charmhelpers/contrib/hardening/ssh/checks/config.py (+0/-394) hooks/charmhelpers/contrib/hardening/templating.py (+0/-71) hooks/charmhelpers/contrib/hardening/utils.py (+0/-157) hooks/charmhelpers/contrib/mellanox/infiniband.py (+0/-151) hooks/charmhelpers/contrib/peerstorage/__init__.py (+0/-269) hooks/charmhelpers/contrib/saltstack/__init__.py (+0/-118) hooks/charmhelpers/contrib/ssl/__init__.py (+0/-94) hooks/charmhelpers/contrib/ssl/service.py (+0/-279) hooks/charmhelpers/contrib/templating/__init__.py (+0/-15) hooks/charmhelpers/contrib/templating/contexts.py (+0/-139) hooks/charmhelpers/contrib/templating/jinja.py (+0/-40) hooks/charmhelpers/contrib/templating/pyformat.py (+0/-29) hooks/charmhelpers/contrib/unison/__init__.py (+0/-313) hooks/pg_gw_hooks.py (+4/-1) hooks/pg_gw_utils.py (+37/-15) templates/kilo/00-pg.conf (+2/-1) templates/kilo/ifcs.conf (+5/-2) templates/kilo/plumgrid.conf (+3/-0) unit_tests/test_pg_gw_hooks.py (+2/-1) |
To merge this branch: | bzr merge lp:~junaidali/charms/trusty/plumgrid-gateway/trunk |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Bilal Baqar | Pending | ||
Review via email: mp+296971@code.launchpad.net |
Commit message
Description of the change
To post a comment you must log in.
- 30. By Junaid Ali
-
L3 fabric changes
- 31. By Junaid Ali
-
Changes:
Updated fabric-interfaces config description
Removed unused imports
Unmerged revisions
- 31. By Junaid Ali
-
Changes:
Updated fabric-interfaces config description
Removed unused imports - 30. By Junaid Ali
-
L3 fabric changes
- 29. By Bilal Baqar
-
5.1 changes
- configure-pg-sources added
- updated templates - 28. By Bilal Baqar
-
Merge - Charmhelpers sync and improved pg-restart
Preview Diff
[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1 | === modified file 'charm-helpers-sync.yaml' |
2 | --- charm-helpers-sync.yaml 2015-07-29 18:23:55 +0000 |
3 | +++ charm-helpers-sync.yaml 2016-06-19 18:55:17 +0000 |
4 | @@ -3,5 +3,10 @@ |
5 | include: |
6 | - core |
7 | - fetch |
8 | - - contrib |
9 | + - contrib.amulet |
10 | + - contrib.hahelpers |
11 | + - contrib.network |
12 | + - contrib.openstack |
13 | + - contrib.python |
14 | + - contrib.storage |
15 | - payload |
16 | |
17 | === modified file 'config.yaml' |
18 | --- config.yaml 2015-11-22 03:26:23 +0000 |
19 | +++ config.yaml 2016-06-19 18:55:17 +0000 |
20 | @@ -13,22 +13,12 @@ |
21 | type: string |
22 | default: 'juju-br0' |
23 | description: The interface connected to PLUMgrid Managment network. |
24 | - os-data-network: |
25 | - type: string |
26 | - default: |
27 | - description: | |
28 | - The IP address and netmask of the OpenStack Data network (e.g., |
29 | - 192.168.0.0/24) |
30 | - . |
31 | - This network will be used for tenant network traffic in overlay |
32 | - networks. |
33 | fabric-interfaces: |
34 | default: 'MANAGEMENT' |
35 | type: string |
36 | description: | |
37 | Interfaces that will provide fabric connectivity on the gateway nodes. |
38 | - Provided in form of json in a string. These interfaces have to be connected |
39 | - to the os-data-network specified in the config. Default value is MANAGEMENT which |
40 | + Provided in form of json in a string. Default value is MANAGEMENT which |
41 | will configure the management interface as the fabric interface on each |
42 | director. |
43 | network-device-mtu: |
44 | |
45 | === removed directory 'hooks/charmhelpers/contrib/ansible' |
46 | === removed file 'hooks/charmhelpers/contrib/ansible/__init__.py' |
47 | --- hooks/charmhelpers/contrib/ansible/__init__.py 2015-07-29 18:23:55 +0000 |
48 | +++ hooks/charmhelpers/contrib/ansible/__init__.py 1970-01-01 00:00:00 +0000 |
49 | @@ -1,254 +0,0 @@ |
50 | -# Copyright 2014-2015 Canonical Limited. |
51 | -# |
52 | -# This file is part of charm-helpers. |
53 | -# |
54 | -# charm-helpers is free software: you can redistribute it and/or modify |
55 | -# it under the terms of the GNU Lesser General Public License version 3 as |
56 | -# published by the Free Software Foundation. |
57 | -# |
58 | -# charm-helpers is distributed in the hope that it will be useful, |
59 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
60 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
61 | -# GNU Lesser General Public License for more details. |
62 | -# |
63 | -# You should have received a copy of the GNU Lesser General Public License |
64 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
65 | - |
66 | -# Copyright 2013 Canonical Ltd. |
67 | -# |
68 | -# Authors: |
69 | -# Charm Helpers Developers <juju@lists.ubuntu.com> |
70 | -"""Charm Helpers ansible - declare the state of your machines. |
71 | - |
72 | -This helper enables you to declare your machine state, rather than |
73 | -program it procedurally (and have to test each change to your procedures). |
74 | -Your install hook can be as simple as:: |
75 | - |
76 | - {{{ |
77 | - import charmhelpers.contrib.ansible |
78 | - |
79 | - |
80 | - def install(): |
81 | - charmhelpers.contrib.ansible.install_ansible_support() |
82 | - charmhelpers.contrib.ansible.apply_playbook('playbooks/install.yaml') |
83 | - }}} |
84 | - |
85 | -and won't need to change (nor will its tests) when you change the machine |
86 | -state. |
87 | - |
88 | -All of your juju config and relation-data are available as template |
89 | -variables within your playbooks and templates. An install playbook looks |
90 | -something like:: |
91 | - |
92 | - {{{ |
93 | - --- |
94 | - - hosts: localhost |
95 | - user: root |
96 | - |
97 | - tasks: |
98 | - - name: Add private repositories. |
99 | - template: |
100 | - src: ../templates/private-repositories.list.jinja2 |
101 | - dest: /etc/apt/sources.list.d/private.list |
102 | - |
103 | - - name: Update the cache. |
104 | - apt: update_cache=yes |
105 | - |
106 | - - name: Install dependencies. |
107 | - apt: pkg={{ item }} |
108 | - with_items: |
109 | - - python-mimeparse |
110 | - - python-webob |
111 | - - sunburnt |
112 | - |
113 | - - name: Setup groups. |
114 | - group: name={{ item.name }} gid={{ item.gid }} |
115 | - with_items: |
116 | - - { name: 'deploy_user', gid: 1800 } |
117 | - - { name: 'service_user', gid: 1500 } |
118 | - |
119 | - ... |
120 | - }}} |
121 | - |
122 | -Read more online about `playbooks`_ and standard ansible `modules`_. |
123 | - |
124 | -.. _playbooks: http://www.ansibleworks.com/docs/playbooks.html |
125 | -.. _modules: http://www.ansibleworks.com/docs/modules.html |
126 | - |
127 | -A further feature os the ansible hooks is to provide a light weight "action" |
128 | -scripting tool. This is a decorator that you apply to a function, and that |
129 | -function can now receive cli args, and can pass extra args to the playbook. |
130 | - |
131 | -e.g. |
132 | - |
133 | - |
134 | -@hooks.action() |
135 | -def some_action(amount, force="False"): |
136 | - "Usage: some-action AMOUNT [force=True]" # <-- shown on error |
137 | - # process the arguments |
138 | - # do some calls |
139 | - # return extra-vars to be passed to ansible-playbook |
140 | - return { |
141 | - 'amount': int(amount), |
142 | - 'type': force, |
143 | - } |
144 | - |
145 | -You can now create a symlink to hooks.py that can be invoked like a hook, but |
146 | -with cli params: |
147 | - |
148 | -# link actions/some-action to hooks/hooks.py |
149 | - |
150 | -actions/some-action amount=10 force=true |
151 | - |
152 | -""" |
153 | -import os |
154 | -import stat |
155 | -import subprocess |
156 | -import functools |
157 | - |
158 | -import charmhelpers.contrib.templating.contexts |
159 | -import charmhelpers.core.host |
160 | -import charmhelpers.core.hookenv |
161 | -import charmhelpers.fetch |
162 | - |
163 | - |
164 | -charm_dir = os.environ.get('CHARM_DIR', '') |
165 | -ansible_hosts_path = '/etc/ansible/hosts' |
166 | -# Ansible will automatically include any vars in the following |
167 | -# file in its inventory when run locally. |
168 | -ansible_vars_path = '/etc/ansible/host_vars/localhost' |
169 | - |
170 | - |
171 | -def install_ansible_support(from_ppa=True, ppa_location='ppa:rquillo/ansible'): |
172 | - """Installs the ansible package. |
173 | - |
174 | - By default it is installed from the `PPA`_ linked from |
175 | - the ansible `website`_ or from a ppa specified by a charm config.. |
176 | - |
177 | - .. _PPA: https://launchpad.net/~rquillo/+archive/ansible |
178 | - .. _website: http://docs.ansible.com/intro_installation.html#latest-releases-via-apt-ubuntu |
179 | - |
180 | - If from_ppa is empty, you must ensure that the package is available |
181 | - from a configured repository. |
182 | - """ |
183 | - if from_ppa: |
184 | - charmhelpers.fetch.add_source(ppa_location) |
185 | - charmhelpers.fetch.apt_update(fatal=True) |
186 | - charmhelpers.fetch.apt_install('ansible') |
187 | - with open(ansible_hosts_path, 'w+') as hosts_file: |
188 | - hosts_file.write('localhost ansible_connection=local') |
189 | - |
190 | - |
191 | -def apply_playbook(playbook, tags=None, extra_vars=None): |
192 | - tags = tags or [] |
193 | - tags = ",".join(tags) |
194 | - charmhelpers.contrib.templating.contexts.juju_state_to_yaml( |
195 | - ansible_vars_path, namespace_separator='__', |
196 | - allow_hyphens_in_keys=False, mode=(stat.S_IRUSR | stat.S_IWUSR)) |
197 | - |
198 | - # we want ansible's log output to be unbuffered |
199 | - env = os.environ.copy() |
200 | - env['PYTHONUNBUFFERED'] = "1" |
201 | - call = [ |
202 | - 'ansible-playbook', |
203 | - '-c', |
204 | - 'local', |
205 | - playbook, |
206 | - ] |
207 | - if tags: |
208 | - call.extend(['--tags', '{}'.format(tags)]) |
209 | - if extra_vars: |
210 | - extra = ["%s=%s" % (k, v) for k, v in extra_vars.items()] |
211 | - call.extend(['--extra-vars', " ".join(extra)]) |
212 | - subprocess.check_call(call, env=env) |
213 | - |
214 | - |
215 | -class AnsibleHooks(charmhelpers.core.hookenv.Hooks): |
216 | - """Run a playbook with the hook-name as the tag. |
217 | - |
218 | - This helper builds on the standard hookenv.Hooks helper, |
219 | - but additionally runs the playbook with the hook-name specified |
220 | - using --tags (ie. running all the tasks tagged with the hook-name). |
221 | - |
222 | - Example:: |
223 | - |
224 | - hooks = AnsibleHooks(playbook_path='playbooks/my_machine_state.yaml') |
225 | - |
226 | - # All the tasks within my_machine_state.yaml tagged with 'install' |
227 | - # will be run automatically after do_custom_work() |
228 | - @hooks.hook() |
229 | - def install(): |
230 | - do_custom_work() |
231 | - |
232 | - # For most of your hooks, you won't need to do anything other |
233 | - # than run the tagged tasks for the hook: |
234 | - @hooks.hook('config-changed', 'start', 'stop') |
235 | - def just_use_playbook(): |
236 | - pass |
237 | - |
238 | - # As a convenience, you can avoid the above noop function by specifying |
239 | - # the hooks which are handled by ansible-only and they'll be registered |
240 | - # for you: |
241 | - # hooks = AnsibleHooks( |
242 | - # 'playbooks/my_machine_state.yaml', |
243 | - # default_hooks=['config-changed', 'start', 'stop']) |
244 | - |
245 | - if __name__ == "__main__": |
246 | - # execute a hook based on the name the program is called by |
247 | - hooks.execute(sys.argv) |
248 | - |
249 | - """ |
250 | - |
251 | - def __init__(self, playbook_path, default_hooks=None): |
252 | - """Register any hooks handled by ansible.""" |
253 | - super(AnsibleHooks, self).__init__() |
254 | - |
255 | - self._actions = {} |
256 | - self.playbook_path = playbook_path |
257 | - |
258 | - default_hooks = default_hooks or [] |
259 | - |
260 | - def noop(*args, **kwargs): |
261 | - pass |
262 | - |
263 | - for hook in default_hooks: |
264 | - self.register(hook, noop) |
265 | - |
266 | - def register_action(self, name, function): |
267 | - """Register a hook""" |
268 | - self._actions[name] = function |
269 | - |
270 | - def execute(self, args): |
271 | - """Execute the hook followed by the playbook using the hook as tag.""" |
272 | - hook_name = os.path.basename(args[0]) |
273 | - extra_vars = None |
274 | - if hook_name in self._actions: |
275 | - extra_vars = self._actions[hook_name](args[1:]) |
276 | - else: |
277 | - super(AnsibleHooks, self).execute(args) |
278 | - |
279 | - charmhelpers.contrib.ansible.apply_playbook( |
280 | - self.playbook_path, tags=[hook_name], extra_vars=extra_vars) |
281 | - |
282 | - def action(self, *action_names): |
283 | - """Decorator, registering them as actions""" |
284 | - def action_wrapper(decorated): |
285 | - |
286 | - @functools.wraps(decorated) |
287 | - def wrapper(argv): |
288 | - kwargs = dict(arg.split('=') for arg in argv) |
289 | - try: |
290 | - return decorated(**kwargs) |
291 | - except TypeError as e: |
292 | - if decorated.__doc__: |
293 | - e.args += (decorated.__doc__,) |
294 | - raise |
295 | - |
296 | - self.register_action(decorated.__name__, wrapper) |
297 | - if '_' in decorated.__name__: |
298 | - self.register_action( |
299 | - decorated.__name__.replace('_', '-'), wrapper) |
300 | - |
301 | - return wrapper |
302 | - |
303 | - return action_wrapper |
304 | |
305 | === removed directory 'hooks/charmhelpers/contrib/benchmark' |
306 | === removed file 'hooks/charmhelpers/contrib/benchmark/__init__.py' |
307 | --- hooks/charmhelpers/contrib/benchmark/__init__.py 2015-07-29 18:23:55 +0000 |
308 | +++ hooks/charmhelpers/contrib/benchmark/__init__.py 1970-01-01 00:00:00 +0000 |
309 | @@ -1,126 +0,0 @@ |
310 | -# Copyright 2014-2015 Canonical Limited. |
311 | -# |
312 | -# This file is part of charm-helpers. |
313 | -# |
314 | -# charm-helpers is free software: you can redistribute it and/or modify |
315 | -# it under the terms of the GNU Lesser General Public License version 3 as |
316 | -# published by the Free Software Foundation. |
317 | -# |
318 | -# charm-helpers is distributed in the hope that it will be useful, |
319 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
320 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
321 | -# GNU Lesser General Public License for more details. |
322 | -# |
323 | -# You should have received a copy of the GNU Lesser General Public License |
324 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
325 | - |
326 | -import subprocess |
327 | -import time |
328 | -import os |
329 | -from distutils.spawn import find_executable |
330 | - |
331 | -from charmhelpers.core.hookenv import ( |
332 | - in_relation_hook, |
333 | - relation_ids, |
334 | - relation_set, |
335 | - relation_get, |
336 | -) |
337 | - |
338 | - |
339 | -def action_set(key, val): |
340 | - if find_executable('action-set'): |
341 | - action_cmd = ['action-set'] |
342 | - |
343 | - if isinstance(val, dict): |
344 | - for k, v in iter(val.items()): |
345 | - action_set('%s.%s' % (key, k), v) |
346 | - return True |
347 | - |
348 | - action_cmd.append('%s=%s' % (key, val)) |
349 | - subprocess.check_call(action_cmd) |
350 | - return True |
351 | - return False |
352 | - |
353 | - |
354 | -class Benchmark(): |
355 | - """ |
356 | - Helper class for the `benchmark` interface. |
357 | - |
358 | - :param list actions: Define the actions that are also benchmarks |
359 | - |
360 | - From inside the benchmark-relation-changed hook, you would |
361 | - Benchmark(['memory', 'cpu', 'disk', 'smoke', 'custom']) |
362 | - |
363 | - Examples: |
364 | - |
365 | - siege = Benchmark(['siege']) |
366 | - siege.start() |
367 | - [... run siege ...] |
368 | - # The higher the score, the better the benchmark |
369 | - siege.set_composite_score(16.70, 'trans/sec', 'desc') |
370 | - siege.finish() |
371 | - |
372 | - |
373 | - """ |
374 | - |
375 | - BENCHMARK_CONF = '/etc/benchmark.conf' # Replaced in testing |
376 | - |
377 | - required_keys = [ |
378 | - 'hostname', |
379 | - 'port', |
380 | - 'graphite_port', |
381 | - 'graphite_endpoint', |
382 | - 'api_port' |
383 | - ] |
384 | - |
385 | - def __init__(self, benchmarks=None): |
386 | - if in_relation_hook(): |
387 | - if benchmarks is not None: |
388 | - for rid in sorted(relation_ids('benchmark')): |
389 | - relation_set(relation_id=rid, relation_settings={ |
390 | - 'benchmarks': ",".join(benchmarks) |
391 | - }) |
392 | - |
393 | - # Check the relation data |
394 | - config = {} |
395 | - for key in self.required_keys: |
396 | - val = relation_get(key) |
397 | - if val is not None: |
398 | - config[key] = val |
399 | - else: |
400 | - # We don't have all of the required keys |
401 | - config = {} |
402 | - break |
403 | - |
404 | - if len(config): |
405 | - with open(self.BENCHMARK_CONF, 'w') as f: |
406 | - for key, val in iter(config.items()): |
407 | - f.write("%s=%s\n" % (key, val)) |
408 | - |
409 | - @staticmethod |
410 | - def start(): |
411 | - action_set('meta.start', time.strftime('%Y-%m-%dT%H:%M:%SZ')) |
412 | - |
413 | - """ |
414 | - If the collectd charm is also installed, tell it to send a snapshot |
415 | - of the current profile data. |
416 | - """ |
417 | - COLLECT_PROFILE_DATA = '/usr/local/bin/collect-profile-data' |
418 | - if os.path.exists(COLLECT_PROFILE_DATA): |
419 | - subprocess.check_output([COLLECT_PROFILE_DATA]) |
420 | - |
421 | - @staticmethod |
422 | - def finish(): |
423 | - action_set('meta.stop', time.strftime('%Y-%m-%dT%H:%M:%SZ')) |
424 | - |
425 | - @staticmethod |
426 | - def set_composite_score(value, units, direction='asc'): |
427 | - """ |
428 | - Set the composite score for a benchmark run. This is a single number |
429 | - representative of the benchmark results. This could be the most |
430 | - important metric, or an amalgamation of metric scores. |
431 | - """ |
432 | - return action_set( |
433 | - "meta.composite", |
434 | - {'value': value, 'units': units, 'direction': direction} |
435 | - ) |
436 | |
437 | === removed directory 'hooks/charmhelpers/contrib/charmhelpers' |
438 | === removed file 'hooks/charmhelpers/contrib/charmhelpers/__init__.py' |
439 | --- hooks/charmhelpers/contrib/charmhelpers/__init__.py 2015-07-29 18:23:55 +0000 |
440 | +++ hooks/charmhelpers/contrib/charmhelpers/__init__.py 1970-01-01 00:00:00 +0000 |
441 | @@ -1,208 +0,0 @@ |
442 | -# Copyright 2014-2015 Canonical Limited. |
443 | -# |
444 | -# This file is part of charm-helpers. |
445 | -# |
446 | -# charm-helpers is free software: you can redistribute it and/or modify |
447 | -# it under the terms of the GNU Lesser General Public License version 3 as |
448 | -# published by the Free Software Foundation. |
449 | -# |
450 | -# charm-helpers is distributed in the hope that it will be useful, |
451 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
452 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
453 | -# GNU Lesser General Public License for more details. |
454 | -# |
455 | -# You should have received a copy of the GNU Lesser General Public License |
456 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
457 | - |
458 | -# Copyright 2012 Canonical Ltd. This software is licensed under the |
459 | -# GNU Affero General Public License version 3 (see the file LICENSE). |
460 | - |
461 | -import warnings |
462 | -warnings.warn("contrib.charmhelpers is deprecated", DeprecationWarning) # noqa |
463 | - |
464 | -import operator |
465 | -import tempfile |
466 | -import time |
467 | -import yaml |
468 | -import subprocess |
469 | - |
470 | -import six |
471 | -if six.PY3: |
472 | - from urllib.request import urlopen |
473 | - from urllib.error import (HTTPError, URLError) |
474 | -else: |
475 | - from urllib2 import (urlopen, HTTPError, URLError) |
476 | - |
477 | -"""Helper functions for writing Juju charms in Python.""" |
478 | - |
479 | -__metaclass__ = type |
480 | -__all__ = [ |
481 | - # 'get_config', # core.hookenv.config() |
482 | - # 'log', # core.hookenv.log() |
483 | - # 'log_entry', # core.hookenv.log() |
484 | - # 'log_exit', # core.hookenv.log() |
485 | - # 'relation_get', # core.hookenv.relation_get() |
486 | - # 'relation_set', # core.hookenv.relation_set() |
487 | - # 'relation_ids', # core.hookenv.relation_ids() |
488 | - # 'relation_list', # core.hookenv.relation_units() |
489 | - # 'config_get', # core.hookenv.config() |
490 | - # 'unit_get', # core.hookenv.unit_get() |
491 | - # 'open_port', # core.hookenv.open_port() |
492 | - # 'close_port', # core.hookenv.close_port() |
493 | - # 'service_control', # core.host.service() |
494 | - 'unit_info', # client-side, NOT IMPLEMENTED |
495 | - 'wait_for_machine', # client-side, NOT IMPLEMENTED |
496 | - 'wait_for_page_contents', # client-side, NOT IMPLEMENTED |
497 | - 'wait_for_relation', # client-side, NOT IMPLEMENTED |
498 | - 'wait_for_unit', # client-side, NOT IMPLEMENTED |
499 | -] |
500 | - |
501 | - |
502 | -SLEEP_AMOUNT = 0.1 |
503 | - |
504 | - |
505 | -# We create a juju_status Command here because it makes testing much, |
506 | -# much easier. |
507 | -def juju_status(): |
508 | - subprocess.check_call(['juju', 'status']) |
509 | - |
510 | -# re-implemented as charmhelpers.fetch.configure_sources() |
511 | -# def configure_source(update=False): |
512 | -# source = config_get('source') |
513 | -# if ((source.startswith('ppa:') or |
514 | -# source.startswith('cloud:') or |
515 | -# source.startswith('http:'))): |
516 | -# run('add-apt-repository', source) |
517 | -# if source.startswith("http:"): |
518 | -# run('apt-key', 'import', config_get('key')) |
519 | -# if update: |
520 | -# run('apt-get', 'update') |
521 | - |
522 | - |
523 | -# DEPRECATED: client-side only |
524 | -def make_charm_config_file(charm_config): |
525 | - charm_config_file = tempfile.NamedTemporaryFile(mode='w+') |
526 | - charm_config_file.write(yaml.dump(charm_config)) |
527 | - charm_config_file.flush() |
528 | - # The NamedTemporaryFile instance is returned instead of just the name |
529 | - # because we want to take advantage of garbage collection-triggered |
530 | - # deletion of the temp file when it goes out of scope in the caller. |
531 | - return charm_config_file |
532 | - |
533 | - |
534 | -# DEPRECATED: client-side only |
535 | -def unit_info(service_name, item_name, data=None, unit=None): |
536 | - if data is None: |
537 | - data = yaml.safe_load(juju_status()) |
538 | - service = data['services'].get(service_name) |
539 | - if service is None: |
540 | - # XXX 2012-02-08 gmb: |
541 | - # This allows us to cope with the race condition that we |
542 | - # have between deploying a service and having it come up in |
543 | - # `juju status`. We could probably do with cleaning it up so |
544 | - # that it fails a bit more noisily after a while. |
545 | - return '' |
546 | - units = service['units'] |
547 | - if unit is not None: |
548 | - item = units[unit][item_name] |
549 | - else: |
550 | - # It might seem odd to sort the units here, but we do it to |
551 | - # ensure that when no unit is specified, the first unit for the |
552 | - # service (or at least the one with the lowest number) is the |
553 | - # one whose data gets returned. |
554 | - sorted_unit_names = sorted(units.keys()) |
555 | - item = units[sorted_unit_names[0]][item_name] |
556 | - return item |
557 | - |
558 | - |
559 | -# DEPRECATED: client-side only |
560 | -def get_machine_data(): |
561 | - return yaml.safe_load(juju_status())['machines'] |
562 | - |
563 | - |
564 | -# DEPRECATED: client-side only |
565 | -def wait_for_machine(num_machines=1, timeout=300): |
566 | - """Wait `timeout` seconds for `num_machines` machines to come up. |
567 | - |
568 | - This wait_for... function can be called by other wait_for functions |
569 | - whose timeouts might be too short in situations where only a bare |
570 | - Juju setup has been bootstrapped. |
571 | - |
572 | - :return: A tuple of (num_machines, time_taken). This is used for |
573 | - testing. |
574 | - """ |
575 | - # You may think this is a hack, and you'd be right. The easiest way |
576 | - # to tell what environment we're working in (LXC vs EC2) is to check |
577 | - # the dns-name of the first machine. If it's localhost we're in LXC |
578 | - # and we can just return here. |
579 | - if get_machine_data()[0]['dns-name'] == 'localhost': |
580 | - return 1, 0 |
581 | - start_time = time.time() |
582 | - while True: |
583 | - # Drop the first machine, since it's the Zookeeper and that's |
584 | - # not a machine that we need to wait for. This will only work |
585 | - # for EC2 environments, which is why we return early above if |
586 | - # we're in LXC. |
587 | - machine_data = get_machine_data() |
588 | - non_zookeeper_machines = [ |
589 | - machine_data[key] for key in list(machine_data.keys())[1:]] |
590 | - if len(non_zookeeper_machines) >= num_machines: |
591 | - all_machines_running = True |
592 | - for machine in non_zookeeper_machines: |
593 | - if machine.get('instance-state') != 'running': |
594 | - all_machines_running = False |
595 | - break |
596 | - if all_machines_running: |
597 | - break |
598 | - if time.time() - start_time >= timeout: |
599 | - raise RuntimeError('timeout waiting for service to start') |
600 | - time.sleep(SLEEP_AMOUNT) |
601 | - return num_machines, time.time() - start_time |
602 | - |
603 | - |
604 | -# DEPRECATED: client-side only |
605 | -def wait_for_unit(service_name, timeout=480): |
606 | - """Wait `timeout` seconds for a given service name to come up.""" |
607 | - wait_for_machine(num_machines=1) |
608 | - start_time = time.time() |
609 | - while True: |
610 | - state = unit_info(service_name, 'agent-state') |
611 | - if 'error' in state or state == 'started': |
612 | - break |
613 | - if time.time() - start_time >= timeout: |
614 | - raise RuntimeError('timeout waiting for service to start') |
615 | - time.sleep(SLEEP_AMOUNT) |
616 | - if state != 'started': |
617 | - raise RuntimeError('unit did not start, agent-state: ' + state) |
618 | - |
619 | - |
620 | -# DEPRECATED: client-side only |
621 | -def wait_for_relation(service_name, relation_name, timeout=120): |
622 | - """Wait `timeout` seconds for a given relation to come up.""" |
623 | - start_time = time.time() |
624 | - while True: |
625 | - relation = unit_info(service_name, 'relations').get(relation_name) |
626 | - if relation is not None and relation['state'] == 'up': |
627 | - break |
628 | - if time.time() - start_time >= timeout: |
629 | - raise RuntimeError('timeout waiting for relation to be up') |
630 | - time.sleep(SLEEP_AMOUNT) |
631 | - |
632 | - |
633 | -# DEPRECATED: client-side only |
634 | -def wait_for_page_contents(url, contents, timeout=120, validate=None): |
635 | - if validate is None: |
636 | - validate = operator.contains |
637 | - start_time = time.time() |
638 | - while True: |
639 | - try: |
640 | - stream = urlopen(url) |
641 | - except (HTTPError, URLError): |
642 | - pass |
643 | - else: |
644 | - page = stream.read() |
645 | - if validate(page, contents): |
646 | - return page |
647 | - if time.time() - start_time >= timeout: |
648 | - raise RuntimeError('timeout waiting for contents of ' + url) |
649 | - time.sleep(SLEEP_AMOUNT) |
650 | |
651 | === removed directory 'hooks/charmhelpers/contrib/charmsupport' |
652 | === removed file 'hooks/charmhelpers/contrib/charmsupport/__init__.py' |
653 | --- hooks/charmhelpers/contrib/charmsupport/__init__.py 2015-07-29 18:23:55 +0000 |
654 | +++ hooks/charmhelpers/contrib/charmsupport/__init__.py 1970-01-01 00:00:00 +0000 |
655 | @@ -1,15 +0,0 @@ |
656 | -# Copyright 2014-2015 Canonical Limited. |
657 | -# |
658 | -# This file is part of charm-helpers. |
659 | -# |
660 | -# charm-helpers is free software: you can redistribute it and/or modify |
661 | -# it under the terms of the GNU Lesser General Public License version 3 as |
662 | -# published by the Free Software Foundation. |
663 | -# |
664 | -# charm-helpers is distributed in the hope that it will be useful, |
665 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
666 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
667 | -# GNU Lesser General Public License for more details. |
668 | -# |
669 | -# You should have received a copy of the GNU Lesser General Public License |
670 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
671 | |
672 | === removed file 'hooks/charmhelpers/contrib/charmsupport/nrpe.py' |
673 | --- hooks/charmhelpers/contrib/charmsupport/nrpe.py 2016-01-30 22:40:29 +0000 |
674 | +++ hooks/charmhelpers/contrib/charmsupport/nrpe.py 1970-01-01 00:00:00 +0000 |
675 | @@ -1,398 +0,0 @@ |
676 | -# Copyright 2014-2015 Canonical Limited. |
677 | -# |
678 | -# This file is part of charm-helpers. |
679 | -# |
680 | -# charm-helpers is free software: you can redistribute it and/or modify |
681 | -# it under the terms of the GNU Lesser General Public License version 3 as |
682 | -# published by the Free Software Foundation. |
683 | -# |
684 | -# charm-helpers is distributed in the hope that it will be useful, |
685 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
686 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
687 | -# GNU Lesser General Public License for more details. |
688 | -# |
689 | -# You should have received a copy of the GNU Lesser General Public License |
690 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
691 | - |
692 | -"""Compatibility with the nrpe-external-master charm""" |
693 | -# Copyright 2012 Canonical Ltd. |
694 | -# |
695 | -# Authors: |
696 | -# Matthew Wedgwood <matthew.wedgwood@canonical.com> |
697 | - |
698 | -import subprocess |
699 | -import pwd |
700 | -import grp |
701 | -import os |
702 | -import glob |
703 | -import shutil |
704 | -import re |
705 | -import shlex |
706 | -import yaml |
707 | - |
708 | -from charmhelpers.core.hookenv import ( |
709 | - config, |
710 | - local_unit, |
711 | - log, |
712 | - relation_ids, |
713 | - relation_set, |
714 | - relations_of_type, |
715 | -) |
716 | - |
717 | -from charmhelpers.core.host import service |
718 | - |
719 | -# This module adds compatibility with the nrpe-external-master and plain nrpe |
720 | -# subordinate charms. To use it in your charm: |
721 | -# |
722 | -# 1. Update metadata.yaml |
723 | -# |
724 | -# provides: |
725 | -# (...) |
726 | -# nrpe-external-master: |
727 | -# interface: nrpe-external-master |
728 | -# scope: container |
729 | -# |
730 | -# and/or |
731 | -# |
732 | -# provides: |
733 | -# (...) |
734 | -# local-monitors: |
735 | -# interface: local-monitors |
736 | -# scope: container |
737 | - |
738 | -# |
739 | -# 2. Add the following to config.yaml |
740 | -# |
741 | -# nagios_context: |
742 | -# default: "juju" |
743 | -# type: string |
744 | -# description: | |
745 | -# Used by the nrpe subordinate charms. |
746 | -# A string that will be prepended to instance name to set the host name |
747 | -# in nagios. So for instance the hostname would be something like: |
748 | -# juju-myservice-0 |
749 | -# If you're running multiple environments with the same services in them |
750 | -# this allows you to differentiate between them. |
751 | -# nagios_servicegroups: |
752 | -# default: "" |
753 | -# type: string |
754 | -# description: | |
755 | -# A comma-separated list of nagios servicegroups. |
756 | -# If left empty, the nagios_context will be used as the servicegroup |
757 | -# |
758 | -# 3. Add custom checks (Nagios plugins) to files/nrpe-external-master |
759 | -# |
760 | -# 4. Update your hooks.py with something like this: |
761 | -# |
762 | -# from charmsupport.nrpe import NRPE |
763 | -# (...) |
764 | -# def update_nrpe_config(): |
765 | -# nrpe_compat = NRPE() |
766 | -# nrpe_compat.add_check( |
767 | -# shortname = "myservice", |
768 | -# description = "Check MyService", |
769 | -# check_cmd = "check_http -w 2 -c 10 http://localhost" |
770 | -# ) |
771 | -# nrpe_compat.add_check( |
772 | -# "myservice_other", |
773 | -# "Check for widget failures", |
774 | -# check_cmd = "/srv/myapp/scripts/widget_check" |
775 | -# ) |
776 | -# nrpe_compat.write() |
777 | -# |
778 | -# def config_changed(): |
779 | -# (...) |
780 | -# update_nrpe_config() |
781 | -# |
782 | -# def nrpe_external_master_relation_changed(): |
783 | -# update_nrpe_config() |
784 | -# |
785 | -# def local_monitors_relation_changed(): |
786 | -# update_nrpe_config() |
787 | -# |
788 | -# 5. ln -s hooks.py nrpe-external-master-relation-changed |
789 | -# ln -s hooks.py local-monitors-relation-changed |
790 | - |
791 | - |
792 | -class CheckException(Exception): |
793 | - pass |
794 | - |
795 | - |
796 | -class Check(object): |
797 | - shortname_re = '[A-Za-z0-9-_]+$' |
798 | - service_template = (""" |
799 | -#--------------------------------------------------- |
800 | -# This file is Juju managed |
801 | -#--------------------------------------------------- |
802 | -define service {{ |
803 | - use active-service |
804 | - host_name {nagios_hostname} |
805 | - service_description {nagios_hostname}[{shortname}] """ |
806 | - """{description} |
807 | - check_command check_nrpe!{command} |
808 | - servicegroups {nagios_servicegroup} |
809 | -}} |
810 | -""") |
811 | - |
812 | - def __init__(self, shortname, description, check_cmd): |
813 | - super(Check, self).__init__() |
814 | - # XXX: could be better to calculate this from the service name |
815 | - if not re.match(self.shortname_re, shortname): |
816 | - raise CheckException("shortname must match {}".format( |
817 | - Check.shortname_re)) |
818 | - self.shortname = shortname |
819 | - self.command = "check_{}".format(shortname) |
820 | - # Note: a set of invalid characters is defined by the |
821 | - # Nagios server config |
822 | - # The default is: illegal_object_name_chars=`~!$%^&*"|'<>?,()= |
823 | - self.description = description |
824 | - self.check_cmd = self._locate_cmd(check_cmd) |
825 | - |
826 | - def _get_check_filename(self): |
827 | - return os.path.join(NRPE.nrpe_confdir, '{}.cfg'.format(self.command)) |
828 | - |
829 | - def _get_service_filename(self, hostname): |
830 | - return os.path.join(NRPE.nagios_exportdir, |
831 | - 'service__{}_{}.cfg'.format(hostname, self.command)) |
832 | - |
833 | - def _locate_cmd(self, check_cmd): |
834 | - search_path = ( |
835 | - '/usr/lib/nagios/plugins', |
836 | - '/usr/local/lib/nagios/plugins', |
837 | - ) |
838 | - parts = shlex.split(check_cmd) |
839 | - for path in search_path: |
840 | - if os.path.exists(os.path.join(path, parts[0])): |
841 | - command = os.path.join(path, parts[0]) |
842 | - if len(parts) > 1: |
843 | - command += " " + " ".join(parts[1:]) |
844 | - return command |
845 | - log('Check command not found: {}'.format(parts[0])) |
846 | - return '' |
847 | - |
848 | - def _remove_service_files(self): |
849 | - if not os.path.exists(NRPE.nagios_exportdir): |
850 | - return |
851 | - for f in os.listdir(NRPE.nagios_exportdir): |
852 | - if f.endswith('_{}.cfg'.format(self.command)): |
853 | - os.remove(os.path.join(NRPE.nagios_exportdir, f)) |
854 | - |
855 | - def remove(self, hostname): |
856 | - nrpe_check_file = self._get_check_filename() |
857 | - if os.path.exists(nrpe_check_file): |
858 | - os.remove(nrpe_check_file) |
859 | - self._remove_service_files() |
860 | - |
861 | - def write(self, nagios_context, hostname, nagios_servicegroups): |
862 | - nrpe_check_file = self._get_check_filename() |
863 | - with open(nrpe_check_file, 'w') as nrpe_check_config: |
864 | - nrpe_check_config.write("# check {}\n".format(self.shortname)) |
865 | - nrpe_check_config.write("command[{}]={}\n".format( |
866 | - self.command, self.check_cmd)) |
867 | - |
868 | - if not os.path.exists(NRPE.nagios_exportdir): |
869 | - log('Not writing service config as {} is not accessible'.format( |
870 | - NRPE.nagios_exportdir)) |
871 | - else: |
872 | - self.write_service_config(nagios_context, hostname, |
873 | - nagios_servicegroups) |
874 | - |
875 | - def write_service_config(self, nagios_context, hostname, |
876 | - nagios_servicegroups): |
877 | - self._remove_service_files() |
878 | - |
879 | - templ_vars = { |
880 | - 'nagios_hostname': hostname, |
881 | - 'nagios_servicegroup': nagios_servicegroups, |
882 | - 'description': self.description, |
883 | - 'shortname': self.shortname, |
884 | - 'command': self.command, |
885 | - } |
886 | - nrpe_service_text = Check.service_template.format(**templ_vars) |
887 | - nrpe_service_file = self._get_service_filename(hostname) |
888 | - with open(nrpe_service_file, 'w') as nrpe_service_config: |
889 | - nrpe_service_config.write(str(nrpe_service_text)) |
890 | - |
891 | - def run(self): |
892 | - subprocess.call(self.check_cmd) |
893 | - |
894 | - |
895 | -class NRPE(object): |
896 | - nagios_logdir = '/var/log/nagios' |
897 | - nagios_exportdir = '/var/lib/nagios/export' |
898 | - nrpe_confdir = '/etc/nagios/nrpe.d' |
899 | - |
900 | - def __init__(self, hostname=None): |
901 | - super(NRPE, self).__init__() |
902 | - self.config = config() |
903 | - self.nagios_context = self.config['nagios_context'] |
904 | - if 'nagios_servicegroups' in self.config and self.config['nagios_servicegroups']: |
905 | - self.nagios_servicegroups = self.config['nagios_servicegroups'] |
906 | - else: |
907 | - self.nagios_servicegroups = self.nagios_context |
908 | - self.unit_name = local_unit().replace('/', '-') |
909 | - if hostname: |
910 | - self.hostname = hostname |
911 | - else: |
912 | - nagios_hostname = get_nagios_hostname() |
913 | - if nagios_hostname: |
914 | - self.hostname = nagios_hostname |
915 | - else: |
916 | - self.hostname = "{}-{}".format(self.nagios_context, self.unit_name) |
917 | - self.checks = [] |
918 | - |
919 | - def add_check(self, *args, **kwargs): |
920 | - self.checks.append(Check(*args, **kwargs)) |
921 | - |
922 | - def remove_check(self, *args, **kwargs): |
923 | - if kwargs.get('shortname') is None: |
924 | - raise ValueError('shortname of check must be specified') |
925 | - |
926 | - # Use sensible defaults if they're not specified - these are not |
927 | - # actually used during removal, but they're required for constructing |
928 | - # the Check object; check_disk is chosen because it's part of the |
929 | - # nagios-plugins-basic package. |
930 | - if kwargs.get('check_cmd') is None: |
931 | - kwargs['check_cmd'] = 'check_disk' |
932 | - if kwargs.get('description') is None: |
933 | - kwargs['description'] = '' |
934 | - |
935 | - check = Check(*args, **kwargs) |
936 | - check.remove(self.hostname) |
937 | - |
938 | - def write(self): |
939 | - try: |
940 | - nagios_uid = pwd.getpwnam('nagios').pw_uid |
941 | - nagios_gid = grp.getgrnam('nagios').gr_gid |
942 | - except: |
943 | - log("Nagios user not set up, nrpe checks not updated") |
944 | - return |
945 | - |
946 | - if not os.path.exists(NRPE.nagios_logdir): |
947 | - os.mkdir(NRPE.nagios_logdir) |
948 | - os.chown(NRPE.nagios_logdir, nagios_uid, nagios_gid) |
949 | - |
950 | - nrpe_monitors = {} |
951 | - monitors = {"monitors": {"remote": {"nrpe": nrpe_monitors}}} |
952 | - for nrpecheck in self.checks: |
953 | - nrpecheck.write(self.nagios_context, self.hostname, |
954 | - self.nagios_servicegroups) |
955 | - nrpe_monitors[nrpecheck.shortname] = { |
956 | - "command": nrpecheck.command, |
957 | - } |
958 | - |
959 | - service('restart', 'nagios-nrpe-server') |
960 | - |
961 | - monitor_ids = relation_ids("local-monitors") + \ |
962 | - relation_ids("nrpe-external-master") |
963 | - for rid in monitor_ids: |
964 | - relation_set(relation_id=rid, monitors=yaml.dump(monitors)) |
965 | - |
966 | - |
967 | -def get_nagios_hostcontext(relation_name='nrpe-external-master'): |
968 | - """ |
969 | - Query relation with nrpe subordinate, return the nagios_host_context |
970 | - |
971 | - :param str relation_name: Name of relation nrpe sub joined to |
972 | - """ |
973 | - for rel in relations_of_type(relation_name): |
974 | - if 'nagios_host_context' in rel: |
975 | - return rel['nagios_host_context'] |
976 | - |
977 | - |
978 | -def get_nagios_hostname(relation_name='nrpe-external-master'): |
979 | - """ |
980 | - Query relation with nrpe subordinate, return the nagios_hostname |
981 | - |
982 | - :param str relation_name: Name of relation nrpe sub joined to |
983 | - """ |
984 | - for rel in relations_of_type(relation_name): |
985 | - if 'nagios_hostname' in rel: |
986 | - return rel['nagios_hostname'] |
987 | - |
988 | - |
989 | -def get_nagios_unit_name(relation_name='nrpe-external-master'): |
990 | - """ |
991 | - Return the nagios unit name prepended with host_context if needed |
992 | - |
993 | - :param str relation_name: Name of relation nrpe sub joined to |
994 | - """ |
995 | - host_context = get_nagios_hostcontext(relation_name) |
996 | - if host_context: |
997 | - unit = "%s:%s" % (host_context, local_unit()) |
998 | - else: |
999 | - unit = local_unit() |
1000 | - return unit |
1001 | - |
1002 | - |
1003 | -def add_init_service_checks(nrpe, services, unit_name): |
1004 | - """ |
1005 | - Add checks for each service in list |
1006 | - |
1007 | - :param NRPE nrpe: NRPE object to add check to |
1008 | - :param list services: List of services to check |
1009 | - :param str unit_name: Unit name to use in check description |
1010 | - """ |
1011 | - for svc in services: |
1012 | - upstart_init = '/etc/init/%s.conf' % svc |
1013 | - sysv_init = '/etc/init.d/%s' % svc |
1014 | - if os.path.exists(upstart_init): |
1015 | - # Don't add a check for these services from neutron-gateway |
1016 | - if svc not in ['ext-port', 'os-charm-phy-nic-mtu']: |
1017 | - nrpe.add_check( |
1018 | - shortname=svc, |
1019 | - description='process check {%s}' % unit_name, |
1020 | - check_cmd='check_upstart_job %s' % svc |
1021 | - ) |
1022 | - elif os.path.exists(sysv_init): |
1023 | - cronpath = '/etc/cron.d/nagios-service-check-%s' % svc |
1024 | - cron_file = ('*/5 * * * * root ' |
1025 | - '/usr/local/lib/nagios/plugins/check_exit_status.pl ' |
1026 | - '-s /etc/init.d/%s status > ' |
1027 | - '/var/lib/nagios/service-check-%s.txt\n' % (svc, |
1028 | - svc) |
1029 | - ) |
1030 | - f = open(cronpath, 'w') |
1031 | - f.write(cron_file) |
1032 | - f.close() |
1033 | - nrpe.add_check( |
1034 | - shortname=svc, |
1035 | - description='process check {%s}' % unit_name, |
1036 | - check_cmd='check_status_file.py -f ' |
1037 | - '/var/lib/nagios/service-check-%s.txt' % svc, |
1038 | - ) |
1039 | - |
1040 | - |
1041 | -def copy_nrpe_checks(): |
1042 | - """ |
1043 | - Copy the nrpe checks into place |
1044 | - |
1045 | - """ |
1046 | - NAGIOS_PLUGINS = '/usr/local/lib/nagios/plugins' |
1047 | - nrpe_files_dir = os.path.join(os.getenv('CHARM_DIR'), 'hooks', |
1048 | - 'charmhelpers', 'contrib', 'openstack', |
1049 | - 'files') |
1050 | - |
1051 | - if not os.path.exists(NAGIOS_PLUGINS): |
1052 | - os.makedirs(NAGIOS_PLUGINS) |
1053 | - for fname in glob.glob(os.path.join(nrpe_files_dir, "check_*")): |
1054 | - if os.path.isfile(fname): |
1055 | - shutil.copy2(fname, |
1056 | - os.path.join(NAGIOS_PLUGINS, os.path.basename(fname))) |
1057 | - |
1058 | - |
1059 | -def add_haproxy_checks(nrpe, unit_name): |
1060 | - """ |
1061 | - Add checks for each service in list |
1062 | - |
1063 | - :param NRPE nrpe: NRPE object to add check to |
1064 | - :param str unit_name: Unit name to use in check description |
1065 | - """ |
1066 | - nrpe.add_check( |
1067 | - shortname='haproxy_servers', |
1068 | - description='Check HAProxy {%s}' % unit_name, |
1069 | - check_cmd='check_haproxy.sh') |
1070 | - nrpe.add_check( |
1071 | - shortname='haproxy_queue', |
1072 | - description='Check HAProxy queue depth {%s}' % unit_name, |
1073 | - check_cmd='check_haproxy_queue_depth.sh') |
1074 | |
1075 | === removed file 'hooks/charmhelpers/contrib/charmsupport/volumes.py' |
1076 | --- hooks/charmhelpers/contrib/charmsupport/volumes.py 2015-07-29 18:23:55 +0000 |
1077 | +++ hooks/charmhelpers/contrib/charmsupport/volumes.py 1970-01-01 00:00:00 +0000 |
1078 | @@ -1,175 +0,0 @@ |
1079 | -# Copyright 2014-2015 Canonical Limited. |
1080 | -# |
1081 | -# This file is part of charm-helpers. |
1082 | -# |
1083 | -# charm-helpers is free software: you can redistribute it and/or modify |
1084 | -# it under the terms of the GNU Lesser General Public License version 3 as |
1085 | -# published by the Free Software Foundation. |
1086 | -# |
1087 | -# charm-helpers is distributed in the hope that it will be useful, |
1088 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
1089 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
1090 | -# GNU Lesser General Public License for more details. |
1091 | -# |
1092 | -# You should have received a copy of the GNU Lesser General Public License |
1093 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
1094 | - |
1095 | -''' |
1096 | -Functions for managing volumes in juju units. One volume is supported per unit. |
1097 | -Subordinates may have their own storage, provided it is on its own partition. |
1098 | - |
1099 | -Configuration stanzas:: |
1100 | - |
1101 | - volume-ephemeral: |
1102 | - type: boolean |
1103 | - default: true |
1104 | - description: > |
1105 | - If false, a volume is mounted as sepecified in "volume-map" |
1106 | - If true, ephemeral storage will be used, meaning that log data |
1107 | - will only exist as long as the machine. YOU HAVE BEEN WARNED. |
1108 | - volume-map: |
1109 | - type: string |
1110 | - default: {} |
1111 | - description: > |
1112 | - YAML map of units to device names, e.g: |
1113 | - "{ rsyslog/0: /dev/vdb, rsyslog/1: /dev/vdb }" |
1114 | - Service units will raise a configure-error if volume-ephemeral |
1115 | - is 'true' and no volume-map value is set. Use 'juju set' to set a |
1116 | - value and 'juju resolved' to complete configuration. |
1117 | - |
1118 | -Usage:: |
1119 | - |
1120 | - from charmsupport.volumes import configure_volume, VolumeConfigurationError |
1121 | - from charmsupport.hookenv import log, ERROR |
1122 | - def post_mount_hook(): |
1123 | - stop_service('myservice') |
1124 | - def post_mount_hook(): |
1125 | - start_service('myservice') |
1126 | - |
1127 | - if __name__ == '__main__': |
1128 | - try: |
1129 | - configure_volume(before_change=pre_mount_hook, |
1130 | - after_change=post_mount_hook) |
1131 | - except VolumeConfigurationError: |
1132 | - log('Storage could not be configured', ERROR) |
1133 | - |
1134 | -''' |
1135 | - |
1136 | -# XXX: Known limitations |
1137 | -# - fstab is neither consulted nor updated |
1138 | - |
1139 | -import os |
1140 | -from charmhelpers.core import hookenv |
1141 | -from charmhelpers.core import host |
1142 | -import yaml |
1143 | - |
1144 | - |
1145 | -MOUNT_BASE = '/srv/juju/volumes' |
1146 | - |
1147 | - |
1148 | -class VolumeConfigurationError(Exception): |
1149 | - '''Volume configuration data is missing or invalid''' |
1150 | - pass |
1151 | - |
1152 | - |
1153 | -def get_config(): |
1154 | - '''Gather and sanity-check volume configuration data''' |
1155 | - volume_config = {} |
1156 | - config = hookenv.config() |
1157 | - |
1158 | - errors = False |
1159 | - |
1160 | - if config.get('volume-ephemeral') in (True, 'True', 'true', 'Yes', 'yes'): |
1161 | - volume_config['ephemeral'] = True |
1162 | - else: |
1163 | - volume_config['ephemeral'] = False |
1164 | - |
1165 | - try: |
1166 | - volume_map = yaml.safe_load(config.get('volume-map', '{}')) |
1167 | - except yaml.YAMLError as e: |
1168 | - hookenv.log("Error parsing YAML volume-map: {}".format(e), |
1169 | - hookenv.ERROR) |
1170 | - errors = True |
1171 | - if volume_map is None: |
1172 | - # probably an empty string |
1173 | - volume_map = {} |
1174 | - elif not isinstance(volume_map, dict): |
1175 | - hookenv.log("Volume-map should be a dictionary, not {}".format( |
1176 | - type(volume_map))) |
1177 | - errors = True |
1178 | - |
1179 | - volume_config['device'] = volume_map.get(os.environ['JUJU_UNIT_NAME']) |
1180 | - if volume_config['device'] and volume_config['ephemeral']: |
1181 | - # asked for ephemeral storage but also defined a volume ID |
1182 | - hookenv.log('A volume is defined for this unit, but ephemeral ' |
1183 | - 'storage was requested', hookenv.ERROR) |
1184 | - errors = True |
1185 | - elif not volume_config['device'] and not volume_config['ephemeral']: |
1186 | - # asked for permanent storage but did not define volume ID |
1187 | - hookenv.log('Ephemeral storage was requested, but there is no volume ' |
1188 | - 'defined for this unit.', hookenv.ERROR) |
1189 | - errors = True |
1190 | - |
1191 | - unit_mount_name = hookenv.local_unit().replace('/', '-') |
1192 | - volume_config['mountpoint'] = os.path.join(MOUNT_BASE, unit_mount_name) |
1193 | - |
1194 | - if errors: |
1195 | - return None |
1196 | - return volume_config |
1197 | - |
1198 | - |
1199 | -def mount_volume(config): |
1200 | - if os.path.exists(config['mountpoint']): |
1201 | - if not os.path.isdir(config['mountpoint']): |
1202 | - hookenv.log('Not a directory: {}'.format(config['mountpoint'])) |
1203 | - raise VolumeConfigurationError() |
1204 | - else: |
1205 | - host.mkdir(config['mountpoint']) |
1206 | - if os.path.ismount(config['mountpoint']): |
1207 | - unmount_volume(config) |
1208 | - if not host.mount(config['device'], config['mountpoint'], persist=True): |
1209 | - raise VolumeConfigurationError() |
1210 | - |
1211 | - |
1212 | -def unmount_volume(config): |
1213 | - if os.path.ismount(config['mountpoint']): |
1214 | - if not host.umount(config['mountpoint'], persist=True): |
1215 | - raise VolumeConfigurationError() |
1216 | - |
1217 | - |
1218 | -def managed_mounts(): |
1219 | - '''List of all mounted managed volumes''' |
1220 | - return filter(lambda mount: mount[0].startswith(MOUNT_BASE), host.mounts()) |
1221 | - |
1222 | - |
1223 | -def configure_volume(before_change=lambda: None, after_change=lambda: None): |
1224 | - '''Set up storage (or don't) according to the charm's volume configuration. |
1225 | - Returns the mount point or "ephemeral". before_change and after_change |
1226 | - are optional functions to be called if the volume configuration changes. |
1227 | - ''' |
1228 | - |
1229 | - config = get_config() |
1230 | - if not config: |
1231 | - hookenv.log('Failed to read volume configuration', hookenv.CRITICAL) |
1232 | - raise VolumeConfigurationError() |
1233 | - |
1234 | - if config['ephemeral']: |
1235 | - if os.path.ismount(config['mountpoint']): |
1236 | - before_change() |
1237 | - unmount_volume(config) |
1238 | - after_change() |
1239 | - return 'ephemeral' |
1240 | - else: |
1241 | - # persistent storage |
1242 | - if os.path.ismount(config['mountpoint']): |
1243 | - mounts = dict(managed_mounts()) |
1244 | - if mounts.get(config['mountpoint']) != config['device']: |
1245 | - before_change() |
1246 | - unmount_volume(config) |
1247 | - mount_volume(config) |
1248 | - after_change() |
1249 | - else: |
1250 | - before_change() |
1251 | - mount_volume(config) |
1252 | - after_change() |
1253 | - return config['mountpoint'] |
1254 | |
1255 | === removed directory 'hooks/charmhelpers/contrib/database' |
1256 | === removed file 'hooks/charmhelpers/contrib/database/__init__.py' |
1257 | === removed file 'hooks/charmhelpers/contrib/database/mysql.py' |
1258 | --- hooks/charmhelpers/contrib/database/mysql.py 2015-07-29 18:23:55 +0000 |
1259 | +++ hooks/charmhelpers/contrib/database/mysql.py 1970-01-01 00:00:00 +0000 |
1260 | @@ -1,412 +0,0 @@ |
1261 | -"""Helper for working with a MySQL database""" |
1262 | -import json |
1263 | -import re |
1264 | -import sys |
1265 | -import platform |
1266 | -import os |
1267 | -import glob |
1268 | - |
1269 | -# from string import upper |
1270 | - |
1271 | -from charmhelpers.core.host import ( |
1272 | - mkdir, |
1273 | - pwgen, |
1274 | - write_file |
1275 | -) |
1276 | -from charmhelpers.core.hookenv import ( |
1277 | - config as config_get, |
1278 | - relation_get, |
1279 | - related_units, |
1280 | - unit_get, |
1281 | - log, |
1282 | - DEBUG, |
1283 | - INFO, |
1284 | - WARNING, |
1285 | -) |
1286 | -from charmhelpers.fetch import ( |
1287 | - apt_install, |
1288 | - apt_update, |
1289 | - filter_installed_packages, |
1290 | -) |
1291 | -from charmhelpers.contrib.peerstorage import ( |
1292 | - peer_store, |
1293 | - peer_retrieve, |
1294 | -) |
1295 | -from charmhelpers.contrib.network.ip import get_host_ip |
1296 | - |
1297 | -try: |
1298 | - import MySQLdb |
1299 | -except ImportError: |
1300 | - apt_update(fatal=True) |
1301 | - apt_install(filter_installed_packages(['python-mysqldb']), fatal=True) |
1302 | - import MySQLdb |
1303 | - |
1304 | - |
1305 | -class MySQLHelper(object): |
1306 | - |
1307 | - def __init__(self, rpasswdf_template, upasswdf_template, host='localhost', |
1308 | - migrate_passwd_to_peer_relation=True, |
1309 | - delete_ondisk_passwd_file=True): |
1310 | - self.host = host |
1311 | - # Password file path templates |
1312 | - self.root_passwd_file_template = rpasswdf_template |
1313 | - self.user_passwd_file_template = upasswdf_template |
1314 | - |
1315 | - self.migrate_passwd_to_peer_relation = migrate_passwd_to_peer_relation |
1316 | - # If we migrate we have the option to delete local copy of root passwd |
1317 | - self.delete_ondisk_passwd_file = delete_ondisk_passwd_file |
1318 | - |
1319 | - def connect(self, user='root', password=None): |
1320 | - log("Opening db connection for %s@%s" % (user, self.host), level=DEBUG) |
1321 | - self.connection = MySQLdb.connect(user=user, host=self.host, |
1322 | - passwd=password) |
1323 | - |
1324 | - def database_exists(self, db_name): |
1325 | - cursor = self.connection.cursor() |
1326 | - try: |
1327 | - cursor.execute("SHOW DATABASES") |
1328 | - databases = [i[0] for i in cursor.fetchall()] |
1329 | - finally: |
1330 | - cursor.close() |
1331 | - |
1332 | - return db_name in databases |
1333 | - |
1334 | - def create_database(self, db_name): |
1335 | - cursor = self.connection.cursor() |
1336 | - try: |
1337 | - cursor.execute("CREATE DATABASE {} CHARACTER SET UTF8" |
1338 | - .format(db_name)) |
1339 | - finally: |
1340 | - cursor.close() |
1341 | - |
1342 | - def grant_exists(self, db_name, db_user, remote_ip): |
1343 | - cursor = self.connection.cursor() |
1344 | - priv_string = "GRANT ALL PRIVILEGES ON `{}`.* " \ |
1345 | - "TO '{}'@'{}'".format(db_name, db_user, remote_ip) |
1346 | - try: |
1347 | - cursor.execute("SHOW GRANTS for '{}'@'{}'".format(db_user, |
1348 | - remote_ip)) |
1349 | - grants = [i[0] for i in cursor.fetchall()] |
1350 | - except MySQLdb.OperationalError: |
1351 | - return False |
1352 | - finally: |
1353 | - cursor.close() |
1354 | - |
1355 | - # TODO: review for different grants |
1356 | - return priv_string in grants |
1357 | - |
1358 | - def create_grant(self, db_name, db_user, remote_ip, password): |
1359 | - cursor = self.connection.cursor() |
1360 | - try: |
1361 | - # TODO: review for different grants |
1362 | - cursor.execute("GRANT ALL PRIVILEGES ON {}.* TO '{}'@'{}' " |
1363 | - "IDENTIFIED BY '{}'".format(db_name, |
1364 | - db_user, |
1365 | - remote_ip, |
1366 | - password)) |
1367 | - finally: |
1368 | - cursor.close() |
1369 | - |
1370 | - def create_admin_grant(self, db_user, remote_ip, password): |
1371 | - cursor = self.connection.cursor() |
1372 | - try: |
1373 | - cursor.execute("GRANT ALL PRIVILEGES ON *.* TO '{}'@'{}' " |
1374 | - "IDENTIFIED BY '{}'".format(db_user, |
1375 | - remote_ip, |
1376 | - password)) |
1377 | - finally: |
1378 | - cursor.close() |
1379 | - |
1380 | - def cleanup_grant(self, db_user, remote_ip): |
1381 | - cursor = self.connection.cursor() |
1382 | - try: |
1383 | - cursor.execute("DROP FROM mysql.user WHERE user='{}' " |
1384 | - "AND HOST='{}'".format(db_user, |
1385 | - remote_ip)) |
1386 | - finally: |
1387 | - cursor.close() |
1388 | - |
1389 | - def execute(self, sql): |
1390 | - """Execute arbitary SQL against the database.""" |
1391 | - cursor = self.connection.cursor() |
1392 | - try: |
1393 | - cursor.execute(sql) |
1394 | - finally: |
1395 | - cursor.close() |
1396 | - |
1397 | - def migrate_passwords_to_peer_relation(self, excludes=None): |
1398 | - """Migrate any passwords storage on disk to cluster peer relation.""" |
1399 | - dirname = os.path.dirname(self.root_passwd_file_template) |
1400 | - path = os.path.join(dirname, '*.passwd') |
1401 | - for f in glob.glob(path): |
1402 | - if excludes and f in excludes: |
1403 | - log("Excluding %s from peer migration" % (f), level=DEBUG) |
1404 | - continue |
1405 | - |
1406 | - key = os.path.basename(f) |
1407 | - with open(f, 'r') as passwd: |
1408 | - _value = passwd.read().strip() |
1409 | - |
1410 | - try: |
1411 | - peer_store(key, _value) |
1412 | - |
1413 | - if self.delete_ondisk_passwd_file: |
1414 | - os.unlink(f) |
1415 | - except ValueError: |
1416 | - # NOTE cluster relation not yet ready - skip for now |
1417 | - pass |
1418 | - |
1419 | - def get_mysql_password_on_disk(self, username=None, password=None): |
1420 | - """Retrieve, generate or store a mysql password for the provided |
1421 | - username on disk.""" |
1422 | - if username: |
1423 | - template = self.user_passwd_file_template |
1424 | - passwd_file = template.format(username) |
1425 | - else: |
1426 | - passwd_file = self.root_passwd_file_template |
1427 | - |
1428 | - _password = None |
1429 | - if os.path.exists(passwd_file): |
1430 | - log("Using existing password file '%s'" % passwd_file, level=DEBUG) |
1431 | - with open(passwd_file, 'r') as passwd: |
1432 | - _password = passwd.read().strip() |
1433 | - else: |
1434 | - log("Generating new password file '%s'" % passwd_file, level=DEBUG) |
1435 | - if not os.path.isdir(os.path.dirname(passwd_file)): |
1436 | - # NOTE: need to ensure this is not mysql root dir (which needs |
1437 | - # to be mysql readable) |
1438 | - mkdir(os.path.dirname(passwd_file), owner='root', group='root', |
1439 | - perms=0o770) |
1440 | - # Force permissions - for some reason the chmod in makedirs |
1441 | - # fails |
1442 | - os.chmod(os.path.dirname(passwd_file), 0o770) |
1443 | - |
1444 | - _password = password or pwgen(length=32) |
1445 | - write_file(passwd_file, _password, owner='root', group='root', |
1446 | - perms=0o660) |
1447 | - |
1448 | - return _password |
1449 | - |
1450 | - def passwd_keys(self, username): |
1451 | - """Generator to return keys used to store passwords in peer store. |
1452 | - |
1453 | - NOTE: we support both legacy and new format to support mysql |
1454 | - charm prior to refactor. This is necessary to avoid LP 1451890. |
1455 | - """ |
1456 | - keys = [] |
1457 | - if username == 'mysql': |
1458 | - log("Bad username '%s'" % (username), level=WARNING) |
1459 | - |
1460 | - if username: |
1461 | - # IMPORTANT: *newer* format must be returned first |
1462 | - keys.append('mysql-%s.passwd' % (username)) |
1463 | - keys.append('%s.passwd' % (username)) |
1464 | - else: |
1465 | - keys.append('mysql.passwd') |
1466 | - |
1467 | - for key in keys: |
1468 | - yield key |
1469 | - |
1470 | - def get_mysql_password(self, username=None, password=None): |
1471 | - """Retrieve, generate or store a mysql password for the provided |
1472 | - username using peer relation cluster.""" |
1473 | - excludes = [] |
1474 | - |
1475 | - # First check peer relation. |
1476 | - try: |
1477 | - for key in self.passwd_keys(username): |
1478 | - _password = peer_retrieve(key) |
1479 | - if _password: |
1480 | - break |
1481 | - |
1482 | - # If root password available don't update peer relation from local |
1483 | - if _password and not username: |
1484 | - excludes.append(self.root_passwd_file_template) |
1485 | - |
1486 | - except ValueError: |
1487 | - # cluster relation is not yet started; use on-disk |
1488 | - _password = None |
1489 | - |
1490 | - # If none available, generate new one |
1491 | - if not _password: |
1492 | - _password = self.get_mysql_password_on_disk(username, password) |
1493 | - |
1494 | - # Put on wire if required |
1495 | - if self.migrate_passwd_to_peer_relation: |
1496 | - self.migrate_passwords_to_peer_relation(excludes=excludes) |
1497 | - |
1498 | - return _password |
1499 | - |
1500 | - def get_mysql_root_password(self, password=None): |
1501 | - """Retrieve or generate mysql root password for service units.""" |
1502 | - return self.get_mysql_password(username=None, password=password) |
1503 | - |
1504 | - def normalize_address(self, hostname): |
1505 | - """Ensure that address returned is an IP address (i.e. not fqdn)""" |
1506 | - if config_get('prefer-ipv6'): |
1507 | - # TODO: add support for ipv6 dns |
1508 | - return hostname |
1509 | - |
1510 | - if hostname != unit_get('private-address'): |
1511 | - return get_host_ip(hostname, fallback=hostname) |
1512 | - |
1513 | - # Otherwise assume localhost |
1514 | - return '127.0.0.1' |
1515 | - |
1516 | - def get_allowed_units(self, database, username, relation_id=None): |
1517 | - """Get list of units with access grants for database with username. |
1518 | - |
1519 | - This is typically used to provide shared-db relations with a list of |
1520 | - which units have been granted access to the given database. |
1521 | - """ |
1522 | - self.connect(password=self.get_mysql_root_password()) |
1523 | - allowed_units = set() |
1524 | - for unit in related_units(relation_id): |
1525 | - settings = relation_get(rid=relation_id, unit=unit) |
1526 | - # First check for setting with prefix, then without |
1527 | - for attr in ["%s_hostname" % (database), 'hostname']: |
1528 | - hosts = settings.get(attr, None) |
1529 | - if hosts: |
1530 | - break |
1531 | - |
1532 | - if hosts: |
1533 | - # hostname can be json-encoded list of hostnames |
1534 | - try: |
1535 | - hosts = json.loads(hosts) |
1536 | - except ValueError: |
1537 | - hosts = [hosts] |
1538 | - else: |
1539 | - hosts = [settings['private-address']] |
1540 | - |
1541 | - if hosts: |
1542 | - for host in hosts: |
1543 | - host = self.normalize_address(host) |
1544 | - if self.grant_exists(database, username, host): |
1545 | - log("Grant exists for host '%s' on db '%s'" % |
1546 | - (host, database), level=DEBUG) |
1547 | - if unit not in allowed_units: |
1548 | - allowed_units.add(unit) |
1549 | - else: |
1550 | - log("Grant does NOT exist for host '%s' on db '%s'" % |
1551 | - (host, database), level=DEBUG) |
1552 | - else: |
1553 | - log("No hosts found for grant check", level=INFO) |
1554 | - |
1555 | - return allowed_units |
1556 | - |
1557 | - def configure_db(self, hostname, database, username, admin=False): |
1558 | - """Configure access to database for username from hostname.""" |
1559 | - self.connect(password=self.get_mysql_root_password()) |
1560 | - if not self.database_exists(database): |
1561 | - self.create_database(database) |
1562 | - |
1563 | - remote_ip = self.normalize_address(hostname) |
1564 | - password = self.get_mysql_password(username) |
1565 | - if not self.grant_exists(database, username, remote_ip): |
1566 | - if not admin: |
1567 | - self.create_grant(database, username, remote_ip, password) |
1568 | - else: |
1569 | - self.create_admin_grant(username, remote_ip, password) |
1570 | - |
1571 | - return password |
1572 | - |
1573 | - |
1574 | -class PerconaClusterHelper(object): |
1575 | - |
1576 | - # Going for the biggest page size to avoid wasted bytes. |
1577 | - # InnoDB page size is 16MB |
1578 | - |
1579 | - DEFAULT_PAGE_SIZE = 16 * 1024 * 1024 |
1580 | - DEFAULT_INNODB_BUFFER_FACTOR = 0.50 |
1581 | - |
1582 | - def human_to_bytes(self, human): |
1583 | - """Convert human readable configuration options to bytes.""" |
1584 | - num_re = re.compile('^[0-9]+$') |
1585 | - if num_re.match(human): |
1586 | - return human |
1587 | - |
1588 | - factors = { |
1589 | - 'K': 1024, |
1590 | - 'M': 1048576, |
1591 | - 'G': 1073741824, |
1592 | - 'T': 1099511627776 |
1593 | - } |
1594 | - modifier = human[-1] |
1595 | - if modifier in factors: |
1596 | - return int(human[:-1]) * factors[modifier] |
1597 | - |
1598 | - if modifier == '%': |
1599 | - total_ram = self.human_to_bytes(self.get_mem_total()) |
1600 | - if self.is_32bit_system() and total_ram > self.sys_mem_limit(): |
1601 | - total_ram = self.sys_mem_limit() |
1602 | - factor = int(human[:-1]) * 0.01 |
1603 | - pctram = total_ram * factor |
1604 | - return int(pctram - (pctram % self.DEFAULT_PAGE_SIZE)) |
1605 | - |
1606 | - raise ValueError("Can only convert K,M,G, or T") |
1607 | - |
1608 | - def is_32bit_system(self): |
1609 | - """Determine whether system is 32 or 64 bit.""" |
1610 | - try: |
1611 | - return sys.maxsize < 2 ** 32 |
1612 | - except OverflowError: |
1613 | - return False |
1614 | - |
1615 | - def sys_mem_limit(self): |
1616 | - """Determine the default memory limit for the current service unit.""" |
1617 | - if platform.machine() in ['armv7l']: |
1618 | - _mem_limit = self.human_to_bytes('2700M') # experimentally determined |
1619 | - else: |
1620 | - # Limit for x86 based 32bit systems |
1621 | - _mem_limit = self.human_to_bytes('4G') |
1622 | - |
1623 | - return _mem_limit |
1624 | - |
1625 | - def get_mem_total(self): |
1626 | - """Calculate the total memory in the current service unit.""" |
1627 | - with open('/proc/meminfo') as meminfo_file: |
1628 | - for line in meminfo_file: |
1629 | - key, mem = line.split(':', 2) |
1630 | - if key == 'MemTotal': |
1631 | - mtot, modifier = mem.strip().split(' ') |
1632 | - return '%s%s' % (mtot, modifier[0].upper()) |
1633 | - |
1634 | - def parse_config(self): |
1635 | - """Parse charm configuration and calculate values for config files.""" |
1636 | - config = config_get() |
1637 | - mysql_config = {} |
1638 | - if 'max-connections' in config: |
1639 | - mysql_config['max_connections'] = config['max-connections'] |
1640 | - |
1641 | - if 'wait-timeout' in config: |
1642 | - mysql_config['wait_timeout'] = config['wait-timeout'] |
1643 | - |
1644 | - if 'innodb-flush-log-at-trx-commit' in config: |
1645 | - mysql_config['innodb_flush_log_at_trx_commit'] = config['innodb-flush-log-at-trx-commit'] |
1646 | - |
1647 | - # Set a sane default key_buffer size |
1648 | - mysql_config['key_buffer'] = self.human_to_bytes('32M') |
1649 | - total_memory = self.human_to_bytes(self.get_mem_total()) |
1650 | - |
1651 | - dataset_bytes = config.get('dataset-size', None) |
1652 | - innodb_buffer_pool_size = config.get('innodb-buffer-pool-size', None) |
1653 | - |
1654 | - if innodb_buffer_pool_size: |
1655 | - innodb_buffer_pool_size = self.human_to_bytes( |
1656 | - innodb_buffer_pool_size) |
1657 | - elif dataset_bytes: |
1658 | - log("Option 'dataset-size' has been deprecated, please use" |
1659 | - "innodb_buffer_pool_size option instead", level="WARN") |
1660 | - innodb_buffer_pool_size = self.human_to_bytes( |
1661 | - dataset_bytes) |
1662 | - else: |
1663 | - innodb_buffer_pool_size = int( |
1664 | - total_memory * self.DEFAULT_INNODB_BUFFER_FACTOR) |
1665 | - |
1666 | - if innodb_buffer_pool_size > total_memory: |
1667 | - log("innodb_buffer_pool_size; {} is greater than system available memory:{}".format( |
1668 | - innodb_buffer_pool_size, |
1669 | - total_memory), level='WARN') |
1670 | - |
1671 | - mysql_config['innodb_buffer_pool_size'] = innodb_buffer_pool_size |
1672 | - return mysql_config |
1673 | |
1674 | === removed directory 'hooks/charmhelpers/contrib/hardening' |
1675 | === removed file 'hooks/charmhelpers/contrib/hardening/__init__.py' |
1676 | --- hooks/charmhelpers/contrib/hardening/__init__.py 2016-04-22 08:15:33 +0000 |
1677 | +++ hooks/charmhelpers/contrib/hardening/__init__.py 1970-01-01 00:00:00 +0000 |
1678 | @@ -1,15 +0,0 @@ |
1679 | -# Copyright 2016 Canonical Limited. |
1680 | -# |
1681 | -# This file is part of charm-helpers. |
1682 | -# |
1683 | -# charm-helpers is free software: you can redistribute it and/or modify |
1684 | -# it under the terms of the GNU Lesser General Public License version 3 as |
1685 | -# published by the Free Software Foundation. |
1686 | -# |
1687 | -# charm-helpers is distributed in the hope that it will be useful, |
1688 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
1689 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
1690 | -# GNU Lesser General Public License for more details. |
1691 | -# |
1692 | -# You should have received a copy of the GNU Lesser General Public License |
1693 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
1694 | |
1695 | === removed directory 'hooks/charmhelpers/contrib/hardening/apache' |
1696 | === removed file 'hooks/charmhelpers/contrib/hardening/apache/__init__.py' |
1697 | --- hooks/charmhelpers/contrib/hardening/apache/__init__.py 2016-04-22 08:15:33 +0000 |
1698 | +++ hooks/charmhelpers/contrib/hardening/apache/__init__.py 1970-01-01 00:00:00 +0000 |
1699 | @@ -1,19 +0,0 @@ |
1700 | -# Copyright 2016 Canonical Limited. |
1701 | -# |
1702 | -# This file is part of charm-helpers. |
1703 | -# |
1704 | -# charm-helpers is free software: you can redistribute it and/or modify |
1705 | -# it under the terms of the GNU Lesser General Public License version 3 as |
1706 | -# published by the Free Software Foundation. |
1707 | -# |
1708 | -# charm-helpers is distributed in the hope that it will be useful, |
1709 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
1710 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
1711 | -# GNU Lesser General Public License for more details. |
1712 | -# |
1713 | -# You should have received a copy of the GNU Lesser General Public License |
1714 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
1715 | - |
1716 | -from os import path |
1717 | - |
1718 | -TEMPLATES_DIR = path.join(path.dirname(__file__), 'templates') |
1719 | |
1720 | === removed directory 'hooks/charmhelpers/contrib/hardening/apache/checks' |
1721 | === removed file 'hooks/charmhelpers/contrib/hardening/apache/checks/__init__.py' |
1722 | --- hooks/charmhelpers/contrib/hardening/apache/checks/__init__.py 2016-04-22 08:15:33 +0000 |
1723 | +++ hooks/charmhelpers/contrib/hardening/apache/checks/__init__.py 1970-01-01 00:00:00 +0000 |
1724 | @@ -1,31 +0,0 @@ |
1725 | -# Copyright 2016 Canonical Limited. |
1726 | -# |
1727 | -# This file is part of charm-helpers. |
1728 | -# |
1729 | -# charm-helpers is free software: you can redistribute it and/or modify |
1730 | -# it under the terms of the GNU Lesser General Public License version 3 as |
1731 | -# published by the Free Software Foundation. |
1732 | -# |
1733 | -# charm-helpers is distributed in the hope that it will be useful, |
1734 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
1735 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
1736 | -# GNU Lesser General Public License for more details. |
1737 | -# |
1738 | -# You should have received a copy of the GNU Lesser General Public License |
1739 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
1740 | - |
1741 | -from charmhelpers.core.hookenv import ( |
1742 | - log, |
1743 | - DEBUG, |
1744 | -) |
1745 | -from charmhelpers.contrib.hardening.apache.checks import config |
1746 | - |
1747 | - |
1748 | -def run_apache_checks(): |
1749 | - log("Starting Apache hardening checks.", level=DEBUG) |
1750 | - checks = config.get_audits() |
1751 | - for check in checks: |
1752 | - log("Running '%s' check" % (check.__class__.__name__), level=DEBUG) |
1753 | - check.ensure_compliance() |
1754 | - |
1755 | - log("Apache hardening checks complete.", level=DEBUG) |
1756 | |
1757 | === removed file 'hooks/charmhelpers/contrib/hardening/apache/checks/config.py' |
1758 | --- hooks/charmhelpers/contrib/hardening/apache/checks/config.py 2016-04-22 08:15:33 +0000 |
1759 | +++ hooks/charmhelpers/contrib/hardening/apache/checks/config.py 1970-01-01 00:00:00 +0000 |
1760 | @@ -1,100 +0,0 @@ |
1761 | -# Copyright 2016 Canonical Limited. |
1762 | -# |
1763 | -# This file is part of charm-helpers. |
1764 | -# |
1765 | -# charm-helpers is free software: you can redistribute it and/or modify |
1766 | -# it under the terms of the GNU Lesser General Public License version 3 as |
1767 | -# published by the Free Software Foundation. |
1768 | -# |
1769 | -# charm-helpers is distributed in the hope that it will be useful, |
1770 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
1771 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
1772 | -# GNU Lesser General Public License for more details. |
1773 | -# |
1774 | -# You should have received a copy of the GNU Lesser General Public License |
1775 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
1776 | - |
1777 | -import os |
1778 | -import re |
1779 | -import subprocess |
1780 | - |
1781 | - |
1782 | -from charmhelpers.core.hookenv import ( |
1783 | - log, |
1784 | - INFO, |
1785 | -) |
1786 | -from charmhelpers.contrib.hardening.audits.file import ( |
1787 | - FilePermissionAudit, |
1788 | - DirectoryPermissionAudit, |
1789 | - NoReadWriteForOther, |
1790 | - TemplatedFile, |
1791 | -) |
1792 | -from charmhelpers.contrib.hardening.audits.apache import DisabledModuleAudit |
1793 | -from charmhelpers.contrib.hardening.apache import TEMPLATES_DIR |
1794 | -from charmhelpers.contrib.hardening import utils |
1795 | - |
1796 | - |
1797 | -def get_audits(): |
1798 | - """Get Apache hardening config audits. |
1799 | - |
1800 | - :returns: dictionary of audits |
1801 | - """ |
1802 | - if subprocess.call(['which', 'apache2'], stdout=subprocess.PIPE) != 0: |
1803 | - log("Apache server does not appear to be installed on this node - " |
1804 | - "skipping apache hardening", level=INFO) |
1805 | - return [] |
1806 | - |
1807 | - context = ApacheConfContext() |
1808 | - settings = utils.get_settings('apache') |
1809 | - audits = [ |
1810 | - FilePermissionAudit(paths='/etc/apache2/apache2.conf', user='root', |
1811 | - group='root', mode=0o0640), |
1812 | - |
1813 | - TemplatedFile(os.path.join(settings['common']['apache_dir'], |
1814 | - 'mods-available/alias.conf'), |
1815 | - context, |
1816 | - TEMPLATES_DIR, |
1817 | - mode=0o0755, |
1818 | - user='root', |
1819 | - service_actions=[{'service': 'apache2', |
1820 | - 'actions': ['restart']}]), |
1821 | - |
1822 | - TemplatedFile(os.path.join(settings['common']['apache_dir'], |
1823 | - 'conf-enabled/hardening.conf'), |
1824 | - context, |
1825 | - TEMPLATES_DIR, |
1826 | - mode=0o0640, |
1827 | - user='root', |
1828 | - service_actions=[{'service': 'apache2', |
1829 | - 'actions': ['restart']}]), |
1830 | - |
1831 | - DirectoryPermissionAudit(settings['common']['apache_dir'], |
1832 | - user='root', |
1833 | - group='root', |
1834 | - mode=0o640), |
1835 | - |
1836 | - DisabledModuleAudit(settings['hardening']['modules_to_disable']), |
1837 | - |
1838 | - NoReadWriteForOther(settings['common']['apache_dir']), |
1839 | - ] |
1840 | - |
1841 | - return audits |
1842 | - |
1843 | - |
1844 | -class ApacheConfContext(object): |
1845 | - """Defines the set of key/value pairs to set in a apache config file. |
1846 | - |
1847 | - This context, when called, will return a dictionary containing the |
1848 | - key/value pairs of setting to specify in the |
1849 | - /etc/apache/conf-enabled/hardening.conf file. |
1850 | - """ |
1851 | - def __call__(self): |
1852 | - settings = utils.get_settings('apache') |
1853 | - ctxt = settings['hardening'] |
1854 | - |
1855 | - out = subprocess.check_output(['apache2', '-v']) |
1856 | - ctxt['apache_version'] = re.search(r'.+version: Apache/(.+?)\s.+', |
1857 | - out).group(1) |
1858 | - ctxt['apache_icondir'] = '/usr/share/apache2/icons/' |
1859 | - ctxt['traceenable'] = settings['hardening']['traceenable'] |
1860 | - return ctxt |
1861 | |
1862 | === removed directory 'hooks/charmhelpers/contrib/hardening/audits' |
1863 | === removed file 'hooks/charmhelpers/contrib/hardening/audits/__init__.py' |
1864 | --- hooks/charmhelpers/contrib/hardening/audits/__init__.py 2016-04-22 08:15:33 +0000 |
1865 | +++ hooks/charmhelpers/contrib/hardening/audits/__init__.py 1970-01-01 00:00:00 +0000 |
1866 | @@ -1,63 +0,0 @@ |
1867 | -# Copyright 2016 Canonical Limited. |
1868 | -# |
1869 | -# This file is part of charm-helpers. |
1870 | -# |
1871 | -# charm-helpers is free software: you can redistribute it and/or modify |
1872 | -# it under the terms of the GNU Lesser General Public License version 3 as |
1873 | -# published by the Free Software Foundation. |
1874 | -# |
1875 | -# charm-helpers is distributed in the hope that it will be useful, |
1876 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
1877 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
1878 | -# GNU Lesser General Public License for more details. |
1879 | -# |
1880 | -# You should have received a copy of the GNU Lesser General Public License |
1881 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
1882 | - |
1883 | - |
1884 | -class BaseAudit(object): # NO-QA |
1885 | - """Base class for hardening checks. |
1886 | - |
1887 | - The lifecycle of a hardening check is to first check to see if the system |
1888 | - is in compliance for the specified check. If it is not in compliance, the |
1889 | - check method will return a value which will be supplied to the. |
1890 | - """ |
1891 | - def __init__(self, *args, **kwargs): |
1892 | - self.unless = kwargs.get('unless', None) |
1893 | - super(BaseAudit, self).__init__() |
1894 | - |
1895 | - def ensure_compliance(self): |
1896 | - """Checks to see if the current hardening check is in compliance or |
1897 | - not. |
1898 | - |
1899 | - If the check that is performed is not in compliance, then an exception |
1900 | - should be raised. |
1901 | - """ |
1902 | - pass |
1903 | - |
1904 | - def _take_action(self): |
1905 | - """Determines whether to perform the action or not. |
1906 | - |
1907 | - Checks whether or not an action should be taken. This is determined by |
1908 | - the truthy value for the unless parameter. If unless is a callback |
1909 | - method, it will be invoked with no parameters in order to determine |
1910 | - whether or not the action should be taken. Otherwise, the truthy value |
1911 | - of the unless attribute will determine if the action should be |
1912 | - performed. |
1913 | - """ |
1914 | - # Do the action if there isn't an unless override. |
1915 | - if self.unless is None: |
1916 | - return True |
1917 | - |
1918 | - # Invoke the callback if there is one. |
1919 | - if hasattr(self.unless, '__call__'): |
1920 | - results = self.unless() |
1921 | - if results: |
1922 | - return False |
1923 | - else: |
1924 | - return True |
1925 | - |
1926 | - if self.unless: |
1927 | - return False |
1928 | - else: |
1929 | - return True |
1930 | |
1931 | === removed file 'hooks/charmhelpers/contrib/hardening/audits/apache.py' |
1932 | --- hooks/charmhelpers/contrib/hardening/audits/apache.py 2016-04-22 08:15:33 +0000 |
1933 | +++ hooks/charmhelpers/contrib/hardening/audits/apache.py 1970-01-01 00:00:00 +0000 |
1934 | @@ -1,100 +0,0 @@ |
1935 | -# Copyright 2016 Canonical Limited. |
1936 | -# |
1937 | -# This file is part of charm-helpers. |
1938 | -# |
1939 | -# charm-helpers is free software: you can redistribute it and/or modify |
1940 | -# it under the terms of the GNU Lesser General Public License version 3 as |
1941 | -# published by the Free Software Foundation. |
1942 | -# |
1943 | -# charm-helpers is distributed in the hope that it will be useful, |
1944 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
1945 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
1946 | -# GNU Lesser General Public License for more details. |
1947 | -# |
1948 | -# You should have received a copy of the GNU Lesser General Public License |
1949 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
1950 | - |
1951 | -import re |
1952 | -import subprocess |
1953 | - |
1954 | -from six import string_types |
1955 | - |
1956 | -from charmhelpers.core.hookenv import ( |
1957 | - log, |
1958 | - INFO, |
1959 | - ERROR, |
1960 | -) |
1961 | - |
1962 | -from charmhelpers.contrib.hardening.audits import BaseAudit |
1963 | - |
1964 | - |
1965 | -class DisabledModuleAudit(BaseAudit): |
1966 | - """Audits Apache2 modules. |
1967 | - |
1968 | - Determines if the apache2 modules are enabled. If the modules are enabled |
1969 | - then they are removed in the ensure_compliance. |
1970 | - """ |
1971 | - def __init__(self, modules): |
1972 | - if modules is None: |
1973 | - self.modules = [] |
1974 | - elif isinstance(modules, string_types): |
1975 | - self.modules = [modules] |
1976 | - else: |
1977 | - self.modules = modules |
1978 | - |
1979 | - def ensure_compliance(self): |
1980 | - """Ensures that the modules are not loaded.""" |
1981 | - if not self.modules: |
1982 | - return |
1983 | - |
1984 | - try: |
1985 | - loaded_modules = self._get_loaded_modules() |
1986 | - non_compliant_modules = [] |
1987 | - for module in self.modules: |
1988 | - if module in loaded_modules: |
1989 | - log("Module '%s' is enabled but should not be." % |
1990 | - (module), level=INFO) |
1991 | - non_compliant_modules.append(module) |
1992 | - |
1993 | - if len(non_compliant_modules) == 0: |
1994 | - return |
1995 | - |
1996 | - for module in non_compliant_modules: |
1997 | - self._disable_module(module) |
1998 | - self._restart_apache() |
1999 | - except subprocess.CalledProcessError as e: |
2000 | - log('Error occurred auditing apache module compliance. ' |
2001 | - 'This may have been already reported. ' |
2002 | - 'Output is: %s' % e.output, level=ERROR) |
2003 | - |
2004 | - @staticmethod |
2005 | - def _get_loaded_modules(): |
2006 | - """Returns the modules which are enabled in Apache.""" |
2007 | - output = subprocess.check_output(['apache2ctl', '-M']) |
2008 | - modules = [] |
2009 | - for line in output.strip().split(): |
2010 | - # Each line of the enabled module output looks like: |
2011 | - # module_name (static|shared) |
2012 | - # Plus a header line at the top of the output which is stripped |
2013 | - # out by the regex. |
2014 | - matcher = re.search(r'^ (\S*)', line) |
2015 | - if matcher: |
2016 | - modules.append(matcher.group(1)) |
2017 | - return modules |
2018 | - |
2019 | - @staticmethod |
2020 | - def _disable_module(module): |
2021 | - """Disables the specified module in Apache.""" |
2022 | - try: |
2023 | - subprocess.check_call(['a2dismod', module]) |
2024 | - except subprocess.CalledProcessError as e: |
2025 | - # Note: catch error here to allow the attempt of disabling |
2026 | - # multiple modules in one go rather than failing after the |
2027 | - # first module fails. |
2028 | - log('Error occurred disabling module %s. ' |
2029 | - 'Output is: %s' % (module, e.output), level=ERROR) |
2030 | - |
2031 | - @staticmethod |
2032 | - def _restart_apache(): |
2033 | - """Restarts the apache process""" |
2034 | - subprocess.check_output(['service', 'apache2', 'restart']) |
2035 | |
2036 | === removed file 'hooks/charmhelpers/contrib/hardening/audits/apt.py' |
2037 | --- hooks/charmhelpers/contrib/hardening/audits/apt.py 2016-04-22 08:15:33 +0000 |
2038 | +++ hooks/charmhelpers/contrib/hardening/audits/apt.py 1970-01-01 00:00:00 +0000 |
2039 | @@ -1,105 +0,0 @@ |
2040 | -# Copyright 2016 Canonical Limited. |
2041 | -# |
2042 | -# This file is part of charm-helpers. |
2043 | -# |
2044 | -# charm-helpers is free software: you can redistribute it and/or modify |
2045 | -# it under the terms of the GNU Lesser General Public License version 3 as |
2046 | -# published by the Free Software Foundation. |
2047 | -# |
2048 | -# charm-helpers is distributed in the hope that it will be useful, |
2049 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
2050 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
2051 | -# GNU Lesser General Public License for more details. |
2052 | -# |
2053 | -# You should have received a copy of the GNU Lesser General Public License |
2054 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
2055 | - |
2056 | -from __future__ import absolute_import # required for external apt import |
2057 | -from apt import apt_pkg |
2058 | -from six import string_types |
2059 | - |
2060 | -from charmhelpers.fetch import ( |
2061 | - apt_cache, |
2062 | - apt_purge |
2063 | -) |
2064 | -from charmhelpers.core.hookenv import ( |
2065 | - log, |
2066 | - DEBUG, |
2067 | - WARNING, |
2068 | -) |
2069 | -from charmhelpers.contrib.hardening.audits import BaseAudit |
2070 | - |
2071 | - |
2072 | -class AptConfig(BaseAudit): |
2073 | - |
2074 | - def __init__(self, config, **kwargs): |
2075 | - self.config = config |
2076 | - |
2077 | - def verify_config(self): |
2078 | - apt_pkg.init() |
2079 | - for cfg in self.config: |
2080 | - value = apt_pkg.config.get(cfg['key'], cfg.get('default', '')) |
2081 | - if value and value != cfg['expected']: |
2082 | - log("APT config '%s' has unexpected value '%s' " |
2083 | - "(expected='%s')" % |
2084 | - (cfg['key'], value, cfg['expected']), level=WARNING) |
2085 | - |
2086 | - def ensure_compliance(self): |
2087 | - self.verify_config() |
2088 | - |
2089 | - |
2090 | -class RestrictedPackages(BaseAudit): |
2091 | - """Class used to audit restricted packages on the system.""" |
2092 | - |
2093 | - def __init__(self, pkgs, **kwargs): |
2094 | - super(RestrictedPackages, self).__init__(**kwargs) |
2095 | - if isinstance(pkgs, string_types) or not hasattr(pkgs, '__iter__'): |
2096 | - self.pkgs = [pkgs] |
2097 | - else: |
2098 | - self.pkgs = pkgs |
2099 | - |
2100 | - def ensure_compliance(self): |
2101 | - cache = apt_cache() |
2102 | - |
2103 | - for p in self.pkgs: |
2104 | - if p not in cache: |
2105 | - continue |
2106 | - |
2107 | - pkg = cache[p] |
2108 | - if not self.is_virtual_package(pkg): |
2109 | - if not pkg.current_ver: |
2110 | - log("Package '%s' is not installed." % pkg.name, |
2111 | - level=DEBUG) |
2112 | - continue |
2113 | - else: |
2114 | - log("Restricted package '%s' is installed" % pkg.name, |
2115 | - level=WARNING) |
2116 | - self.delete_package(cache, pkg) |
2117 | - else: |
2118 | - log("Checking restricted virtual package '%s' provides" % |
2119 | - pkg.name, level=DEBUG) |
2120 | - self.delete_package(cache, pkg) |
2121 | - |
2122 | - def delete_package(self, cache, pkg): |
2123 | - """Deletes the package from the system. |
2124 | - |
2125 | - Deletes the package form the system, properly handling virtual |
2126 | - packages. |
2127 | - |
2128 | - :param cache: the apt cache |
2129 | - :param pkg: the package to remove |
2130 | - """ |
2131 | - if self.is_virtual_package(pkg): |
2132 | - log("Package '%s' appears to be virtual - purging provides" % |
2133 | - pkg.name, level=DEBUG) |
2134 | - for _p in pkg.provides_list: |
2135 | - self.delete_package(cache, _p[2].parent_pkg) |
2136 | - elif not pkg.current_ver: |
2137 | - log("Package '%s' not installed" % pkg.name, level=DEBUG) |
2138 | - return |
2139 | - else: |
2140 | - log("Purging package '%s'" % pkg.name, level=DEBUG) |
2141 | - apt_purge(pkg.name) |
2142 | - |
2143 | - def is_virtual_package(self, pkg): |
2144 | - return pkg.has_provides and not pkg.has_versions |
2145 | |
2146 | === removed file 'hooks/charmhelpers/contrib/hardening/audits/file.py' |
2147 | --- hooks/charmhelpers/contrib/hardening/audits/file.py 2016-04-22 08:15:33 +0000 |
2148 | +++ hooks/charmhelpers/contrib/hardening/audits/file.py 1970-01-01 00:00:00 +0000 |
2149 | @@ -1,552 +0,0 @@ |
2150 | -# Copyright 2016 Canonical Limited. |
2151 | -# |
2152 | -# This file is part of charm-helpers. |
2153 | -# |
2154 | -# charm-helpers is free software: you can redistribute it and/or modify |
2155 | -# it under the terms of the GNU Lesser General Public License version 3 as |
2156 | -# published by the Free Software Foundation. |
2157 | -# |
2158 | -# charm-helpers is distributed in the hope that it will be useful, |
2159 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
2160 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
2161 | -# GNU Lesser General Public License for more details. |
2162 | -# |
2163 | -# You should have received a copy of the GNU Lesser General Public License |
2164 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
2165 | - |
2166 | -import grp |
2167 | -import os |
2168 | -import pwd |
2169 | -import re |
2170 | - |
2171 | -from subprocess import ( |
2172 | - CalledProcessError, |
2173 | - check_output, |
2174 | - check_call, |
2175 | -) |
2176 | -from traceback import format_exc |
2177 | -from six import string_types |
2178 | -from stat import ( |
2179 | - S_ISGID, |
2180 | - S_ISUID |
2181 | -) |
2182 | - |
2183 | -from charmhelpers.core.hookenv import ( |
2184 | - log, |
2185 | - DEBUG, |
2186 | - INFO, |
2187 | - WARNING, |
2188 | - ERROR, |
2189 | -) |
2190 | -from charmhelpers.core import unitdata |
2191 | -from charmhelpers.core.host import file_hash |
2192 | -from charmhelpers.contrib.hardening.audits import BaseAudit |
2193 | -from charmhelpers.contrib.hardening.templating import ( |
2194 | - get_template_path, |
2195 | - render_and_write, |
2196 | -) |
2197 | -from charmhelpers.contrib.hardening import utils |
2198 | - |
2199 | - |
2200 | -class BaseFileAudit(BaseAudit): |
2201 | - """Base class for file audits. |
2202 | - |
2203 | - Provides api stubs for compliance check flow that must be used by any class |
2204 | - that implemented this one. |
2205 | - """ |
2206 | - |
2207 | - def __init__(self, paths, always_comply=False, *args, **kwargs): |
2208 | - """ |
2209 | - :param paths: string path of list of paths of files we want to apply |
2210 | - compliance checks are criteria to. |
2211 | - :param always_comply: if true compliance criteria is always applied |
2212 | - else compliance is skipped for non-existent |
2213 | - paths. |
2214 | - """ |
2215 | - super(BaseFileAudit, self).__init__(*args, **kwargs) |
2216 | - self.always_comply = always_comply |
2217 | - if isinstance(paths, string_types) or not hasattr(paths, '__iter__'): |
2218 | - self.paths = [paths] |
2219 | - else: |
2220 | - self.paths = paths |
2221 | - |
2222 | - def ensure_compliance(self): |
2223 | - """Ensure that the all registered files comply to registered criteria. |
2224 | - """ |
2225 | - for p in self.paths: |
2226 | - if os.path.exists(p): |
2227 | - if self.is_compliant(p): |
2228 | - continue |
2229 | - |
2230 | - log('File %s is not in compliance.' % p, level=INFO) |
2231 | - else: |
2232 | - if not self.always_comply: |
2233 | - log("Non-existent path '%s' - skipping compliance check" |
2234 | - % (p), level=INFO) |
2235 | - continue |
2236 | - |
2237 | - if self._take_action(): |
2238 | - log("Applying compliance criteria to '%s'" % (p), level=INFO) |
2239 | - self.comply(p) |
2240 | - |
2241 | - def is_compliant(self, path): |
2242 | - """Audits the path to see if it is compliance. |
2243 | - |
2244 | - :param path: the path to the file that should be checked. |
2245 | - """ |
2246 | - raise NotImplementedError |
2247 | - |
2248 | - def comply(self, path): |
2249 | - """Enforces the compliance of a path. |
2250 | - |
2251 | - :param path: the path to the file that should be enforced. |
2252 | - """ |
2253 | - raise NotImplementedError |
2254 | - |
2255 | - @classmethod |
2256 | - def _get_stat(cls, path): |
2257 | - """Returns the Posix st_stat information for the specified file path. |
2258 | - |
2259 | - :param path: the path to get the st_stat information for. |
2260 | - :returns: an st_stat object for the path or None if the path doesn't |
2261 | - exist. |
2262 | - """ |
2263 | - return os.stat(path) |
2264 | - |
2265 | - |
2266 | -class FilePermissionAudit(BaseFileAudit): |
2267 | - """Implements an audit for file permissions and ownership for a user. |
2268 | - |
2269 | - This class implements functionality that ensures that a specific user/group |
2270 | - will own the file(s) specified and that the permissions specified are |
2271 | - applied properly to the file. |
2272 | - """ |
2273 | - def __init__(self, paths, user, group=None, mode=0o600, **kwargs): |
2274 | - self.user = user |
2275 | - self.group = group |
2276 | - self.mode = mode |
2277 | - super(FilePermissionAudit, self).__init__(paths, user, group, mode, |
2278 | - **kwargs) |
2279 | - |
2280 | - @property |
2281 | - def user(self): |
2282 | - return self._user |
2283 | - |
2284 | - @user.setter |
2285 | - def user(self, name): |
2286 | - try: |
2287 | - user = pwd.getpwnam(name) |
2288 | - except KeyError: |
2289 | - log('Unknown user %s' % name, level=ERROR) |
2290 | - user = None |
2291 | - self._user = user |
2292 | - |
2293 | - @property |
2294 | - def group(self): |
2295 | - return self._group |
2296 | - |
2297 | - @group.setter |
2298 | - def group(self, name): |
2299 | - try: |
2300 | - group = None |
2301 | - if name: |
2302 | - group = grp.getgrnam(name) |
2303 | - else: |
2304 | - group = grp.getgrgid(self.user.pw_gid) |
2305 | - except KeyError: |
2306 | - log('Unknown group %s' % name, level=ERROR) |
2307 | - self._group = group |
2308 | - |
2309 | - def is_compliant(self, path): |
2310 | - """Checks if the path is in compliance. |
2311 | - |
2312 | - Used to determine if the path specified meets the necessary |
2313 | - requirements to be in compliance with the check itself. |
2314 | - |
2315 | - :param path: the file path to check |
2316 | - :returns: True if the path is compliant, False otherwise. |
2317 | - """ |
2318 | - stat = self._get_stat(path) |
2319 | - user = self.user |
2320 | - group = self.group |
2321 | - |
2322 | - compliant = True |
2323 | - if stat.st_uid != user.pw_uid or stat.st_gid != group.gr_gid: |
2324 | - log('File %s is not owned by %s:%s.' % (path, user.pw_name, |
2325 | - group.gr_name), |
2326 | - level=INFO) |
2327 | - compliant = False |
2328 | - |
2329 | - # POSIX refers to the st_mode bits as corresponding to both the |
2330 | - # file type and file permission bits, where the least significant 12 |
2331 | - # bits (o7777) are the suid (11), sgid (10), sticky bits (9), and the |
2332 | - # file permission bits (8-0) |
2333 | - perms = stat.st_mode & 0o7777 |
2334 | - if perms != self.mode: |
2335 | - log('File %s has incorrect permissions, currently set to %s' % |
2336 | - (path, oct(stat.st_mode & 0o7777)), level=INFO) |
2337 | - compliant = False |
2338 | - |
2339 | - return compliant |
2340 | - |
2341 | - def comply(self, path): |
2342 | - """Issues a chown and chmod to the file paths specified.""" |
2343 | - utils.ensure_permissions(path, self.user.pw_name, self.group.gr_name, |
2344 | - self.mode) |
2345 | - |
2346 | - |
2347 | -class DirectoryPermissionAudit(FilePermissionAudit): |
2348 | - """Performs a permission check for the specified directory path.""" |
2349 | - |
2350 | - def __init__(self, paths, user, group=None, mode=0o600, |
2351 | - recursive=True, **kwargs): |
2352 | - super(DirectoryPermissionAudit, self).__init__(paths, user, group, |
2353 | - mode, **kwargs) |
2354 | - self.recursive = recursive |
2355 | - |
2356 | - def is_compliant(self, path): |
2357 | - """Checks if the directory is compliant. |
2358 | - |
2359 | - Used to determine if the path specified and all of its children |
2360 | - directories are in compliance with the check itself. |
2361 | - |
2362 | - :param path: the directory path to check |
2363 | - :returns: True if the directory tree is compliant, otherwise False. |
2364 | - """ |
2365 | - if not os.path.isdir(path): |
2366 | - log('Path specified %s is not a directory.' % path, level=ERROR) |
2367 | - raise ValueError("%s is not a directory." % path) |
2368 | - |
2369 | - if not self.recursive: |
2370 | - return super(DirectoryPermissionAudit, self).is_compliant(path) |
2371 | - |
2372 | - compliant = True |
2373 | - for root, dirs, _ in os.walk(path): |
2374 | - if len(dirs) > 0: |
2375 | - continue |
2376 | - |
2377 | - if not super(DirectoryPermissionAudit, self).is_compliant(root): |
2378 | - compliant = False |
2379 | - continue |
2380 | - |
2381 | - return compliant |
2382 | - |
2383 | - def comply(self, path): |
2384 | - for root, dirs, _ in os.walk(path): |
2385 | - if len(dirs) > 0: |
2386 | - super(DirectoryPermissionAudit, self).comply(root) |
2387 | - |
2388 | - |
2389 | -class ReadOnly(BaseFileAudit): |
2390 | - """Audits that files and folders are read only.""" |
2391 | - def __init__(self, paths, *args, **kwargs): |
2392 | - super(ReadOnly, self).__init__(paths=paths, *args, **kwargs) |
2393 | - |
2394 | - def is_compliant(self, path): |
2395 | - try: |
2396 | - output = check_output(['find', path, '-perm', '-go+w', |
2397 | - '-type', 'f']).strip() |
2398 | - |
2399 | - # The find above will find any files which have permission sets |
2400 | - # which allow too broad of write access. As such, the path is |
2401 | - # compliant if there is no output. |
2402 | - if output: |
2403 | - return False |
2404 | - |
2405 | - return True |
2406 | - except CalledProcessError as e: |
2407 | - log('Error occurred checking finding writable files for %s. ' |
2408 | - 'Error information is: command %s failed with returncode ' |
2409 | - '%d and output %s.\n%s' % (path, e.cmd, e.returncode, e.output, |
2410 | - format_exc(e)), level=ERROR) |
2411 | - return False |
2412 | - |
2413 | - def comply(self, path): |
2414 | - try: |
2415 | - check_output(['chmod', 'go-w', '-R', path]) |
2416 | - except CalledProcessError as e: |
2417 | - log('Error occurred removing writeable permissions for %s. ' |
2418 | - 'Error information is: command %s failed with returncode ' |
2419 | - '%d and output %s.\n%s' % (path, e.cmd, e.returncode, e.output, |
2420 | - format_exc(e)), level=ERROR) |
2421 | - |
2422 | - |
2423 | -class NoReadWriteForOther(BaseFileAudit): |
2424 | - """Ensures that the files found under the base path are readable or |
2425 | - writable by anyone other than the owner or the group. |
2426 | - """ |
2427 | - def __init__(self, paths): |
2428 | - super(NoReadWriteForOther, self).__init__(paths) |
2429 | - |
2430 | - def is_compliant(self, path): |
2431 | - try: |
2432 | - cmd = ['find', path, '-perm', '-o+r', '-type', 'f', '-o', |
2433 | - '-perm', '-o+w', '-type', 'f'] |
2434 | - output = check_output(cmd).strip() |
2435 | - |
2436 | - # The find above here will find any files which have read or |
2437 | - # write permissions for other, meaning there is too broad of access |
2438 | - # to read/write the file. As such, the path is compliant if there's |
2439 | - # no output. |
2440 | - if output: |
2441 | - return False |
2442 | - |
2443 | - return True |
2444 | - except CalledProcessError as e: |
2445 | - log('Error occurred while finding files which are readable or ' |
2446 | - 'writable to the world in %s. ' |
2447 | - 'Command output is: %s.' % (path, e.output), level=ERROR) |
2448 | - |
2449 | - def comply(self, path): |
2450 | - try: |
2451 | - check_output(['chmod', '-R', 'o-rw', path]) |
2452 | - except CalledProcessError as e: |
2453 | - log('Error occurred attempting to change modes of files under ' |
2454 | - 'path %s. Output of command is: %s' % (path, e.output)) |
2455 | - |
2456 | - |
2457 | -class NoSUIDSGIDAudit(BaseFileAudit): |
2458 | - """Audits that specified files do not have SUID/SGID bits set.""" |
2459 | - def __init__(self, paths, *args, **kwargs): |
2460 | - super(NoSUIDSGIDAudit, self).__init__(paths=paths, *args, **kwargs) |
2461 | - |
2462 | - def is_compliant(self, path): |
2463 | - stat = self._get_stat(path) |
2464 | - if (stat.st_mode & (S_ISGID | S_ISUID)) != 0: |
2465 | - return False |
2466 | - |
2467 | - return True |
2468 | - |
2469 | - def comply(self, path): |
2470 | - try: |
2471 | - log('Removing suid/sgid from %s.' % path, level=DEBUG) |
2472 | - check_output(['chmod', '-s', path]) |
2473 | - except CalledProcessError as e: |
2474 | - log('Error occurred removing suid/sgid from %s.' |
2475 | - 'Error information is: command %s failed with returncode ' |
2476 | - '%d and output %s.\n%s' % (path, e.cmd, e.returncode, e.output, |
2477 | - format_exc(e)), level=ERROR) |
2478 | - |
2479 | - |
2480 | -class TemplatedFile(BaseFileAudit): |
2481 | - """The TemplatedFileAudit audits the contents of a templated file. |
2482 | - |
2483 | - This audit renders a file from a template, sets the appropriate file |
2484 | - permissions, then generates a hashsum with which to check the content |
2485 | - changed. |
2486 | - """ |
2487 | - def __init__(self, path, context, template_dir, mode, user='root', |
2488 | - group='root', service_actions=None, **kwargs): |
2489 | - self.context = context |
2490 | - self.user = user |
2491 | - self.group = group |
2492 | - self.mode = mode |
2493 | - self.template_dir = template_dir |
2494 | - self.service_actions = service_actions |
2495 | - super(TemplatedFile, self).__init__(paths=path, always_comply=True, |
2496 | - **kwargs) |
2497 | - |
2498 | - def is_compliant(self, path): |
2499 | - """Determines if the templated file is compliant. |
2500 | - |
2501 | - A templated file is only compliant if it has not changed (as |
2502 | - determined by its sha256 hashsum) AND its file permissions are set |
2503 | - appropriately. |
2504 | - |
2505 | - :param path: the path to check compliance. |
2506 | - """ |
2507 | - same_templates = self.templates_match(path) |
2508 | - same_content = self.contents_match(path) |
2509 | - same_permissions = self.permissions_match(path) |
2510 | - |
2511 | - if same_content and same_permissions and same_templates: |
2512 | - return True |
2513 | - |
2514 | - return False |
2515 | - |
2516 | - def run_service_actions(self): |
2517 | - """Run any actions on services requested.""" |
2518 | - if not self.service_actions: |
2519 | - return |
2520 | - |
2521 | - for svc_action in self.service_actions: |
2522 | - name = svc_action['service'] |
2523 | - actions = svc_action['actions'] |
2524 | - log("Running service '%s' actions '%s'" % (name, actions), |
2525 | - level=DEBUG) |
2526 | - for action in actions: |
2527 | - cmd = ['service', name, action] |
2528 | - try: |
2529 | - check_call(cmd) |
2530 | - except CalledProcessError as exc: |
2531 | - log("Service name='%s' action='%s' failed - %s" % |
2532 | - (name, action, exc), level=WARNING) |
2533 | - |
2534 | - def comply(self, path): |
2535 | - """Ensures the contents and the permissions of the file. |
2536 | - |
2537 | - :param path: the path to correct |
2538 | - """ |
2539 | - dirname = os.path.dirname(path) |
2540 | - if not os.path.exists(dirname): |
2541 | - os.makedirs(dirname) |
2542 | - |
2543 | - self.pre_write() |
2544 | - render_and_write(self.template_dir, path, self.context()) |
2545 | - utils.ensure_permissions(path, self.user, self.group, self.mode) |
2546 | - self.run_service_actions() |
2547 | - self.save_checksum(path) |
2548 | - self.post_write() |
2549 | - |
2550 | - def pre_write(self): |
2551 | - """Invoked prior to writing the template.""" |
2552 | - pass |
2553 | - |
2554 | - def post_write(self): |
2555 | - """Invoked after writing the template.""" |
2556 | - pass |
2557 | - |
2558 | - def templates_match(self, path): |
2559 | - """Determines if the template files are the same. |
2560 | - |
2561 | - The template file equality is determined by the hashsum of the |
2562 | - template files themselves. If there is no hashsum, then the content |
2563 | - cannot be sure to be the same so treat it as if they changed. |
2564 | - Otherwise, return whether or not the hashsums are the same. |
2565 | - |
2566 | - :param path: the path to check |
2567 | - :returns: boolean |
2568 | - """ |
2569 | - template_path = get_template_path(self.template_dir, path) |
2570 | - key = 'hardening:template:%s' % template_path |
2571 | - template_checksum = file_hash(template_path) |
2572 | - kv = unitdata.kv() |
2573 | - stored_tmplt_checksum = kv.get(key) |
2574 | - if not stored_tmplt_checksum: |
2575 | - kv.set(key, template_checksum) |
2576 | - kv.flush() |
2577 | - log('Saved template checksum for %s.' % template_path, |
2578 | - level=DEBUG) |
2579 | - # Since we don't have a template checksum, then assume it doesn't |
2580 | - # match and return that the template is different. |
2581 | - return False |
2582 | - elif stored_tmplt_checksum != template_checksum: |
2583 | - kv.set(key, template_checksum) |
2584 | - kv.flush() |
2585 | - log('Updated template checksum for %s.' % template_path, |
2586 | - level=DEBUG) |
2587 | - return False |
2588 | - |
2589 | - # Here the template hasn't changed based upon the calculated |
2590 | - # checksum of the template and what was previously stored. |
2591 | - return True |
2592 | - |
2593 | - def contents_match(self, path): |
2594 | - """Determines if the file content is the same. |
2595 | - |
2596 | - This is determined by comparing hashsum of the file contents and |
2597 | - the saved hashsum. If there is no hashsum, then the content cannot |
2598 | - be sure to be the same so treat them as if they are not the same. |
2599 | - Otherwise, return True if the hashsums are the same, False if they |
2600 | - are not the same. |
2601 | - |
2602 | - :param path: the file to check. |
2603 | - """ |
2604 | - checksum = file_hash(path) |
2605 | - |
2606 | - kv = unitdata.kv() |
2607 | - stored_checksum = kv.get('hardening:%s' % path) |
2608 | - if not stored_checksum: |
2609 | - # If the checksum hasn't been generated, return False to ensure |
2610 | - # the file is written and the checksum stored. |
2611 | - log('Checksum for %s has not been calculated.' % path, level=DEBUG) |
2612 | - return False |
2613 | - elif stored_checksum != checksum: |
2614 | - log('Checksum mismatch for %s.' % path, level=DEBUG) |
2615 | - return False |
2616 | - |
2617 | - return True |
2618 | - |
2619 | - def permissions_match(self, path): |
2620 | - """Determines if the file owner and permissions match. |
2621 | - |
2622 | - :param path: the path to check. |
2623 | - """ |
2624 | - audit = FilePermissionAudit(path, self.user, self.group, self.mode) |
2625 | - return audit.is_compliant(path) |
2626 | - |
2627 | - def save_checksum(self, path): |
2628 | - """Calculates and saves the checksum for the path specified. |
2629 | - |
2630 | - :param path: the path of the file to save the checksum. |
2631 | - """ |
2632 | - checksum = file_hash(path) |
2633 | - kv = unitdata.kv() |
2634 | - kv.set('hardening:%s' % path, checksum) |
2635 | - kv.flush() |
2636 | - |
2637 | - |
2638 | -class DeletedFile(BaseFileAudit): |
2639 | - """Audit to ensure that a file is deleted.""" |
2640 | - def __init__(self, paths): |
2641 | - super(DeletedFile, self).__init__(paths) |
2642 | - |
2643 | - def is_compliant(self, path): |
2644 | - return not os.path.exists(path) |
2645 | - |
2646 | - def comply(self, path): |
2647 | - os.remove(path) |
2648 | - |
2649 | - |
2650 | -class FileContentAudit(BaseFileAudit): |
2651 | - """Audit the contents of a file.""" |
2652 | - def __init__(self, paths, cases, **kwargs): |
2653 | - # Cases we expect to pass |
2654 | - self.pass_cases = cases.get('pass', []) |
2655 | - # Cases we expect to fail |
2656 | - self.fail_cases = cases.get('fail', []) |
2657 | - super(FileContentAudit, self).__init__(paths, **kwargs) |
2658 | - |
2659 | - def is_compliant(self, path): |
2660 | - """ |
2661 | - Given a set of content matching cases i.e. tuple(regex, bool) where |
2662 | - bool value denotes whether or not regex is expected to match, check that |
2663 | - all cases match as expected with the contents of the file. Cases can be |
2664 | - expected to pass of fail. |
2665 | - |
2666 | - :param path: Path of file to check. |
2667 | - :returns: Boolean value representing whether or not all cases are |
2668 | - found to be compliant. |
2669 | - """ |
2670 | - log("Auditing contents of file '%s'" % (path), level=DEBUG) |
2671 | - with open(path, 'r') as fd: |
2672 | - contents = fd.read() |
2673 | - |
2674 | - matches = 0 |
2675 | - for pattern in self.pass_cases: |
2676 | - key = re.compile(pattern, flags=re.MULTILINE) |
2677 | - results = re.search(key, contents) |
2678 | - if results: |
2679 | - matches += 1 |
2680 | - else: |
2681 | - log("Pattern '%s' was expected to pass but instead it failed" |
2682 | - % (pattern), level=WARNING) |
2683 | - |
2684 | - for pattern in self.fail_cases: |
2685 | - key = re.compile(pattern, flags=re.MULTILINE) |
2686 | - results = re.search(key, contents) |
2687 | - if not results: |
2688 | - matches += 1 |
2689 | - else: |
2690 | - log("Pattern '%s' was expected to fail but instead it passed" |
2691 | - % (pattern), level=WARNING) |
2692 | - |
2693 | - total = len(self.pass_cases) + len(self.fail_cases) |
2694 | - log("Checked %s cases and %s passed" % (total, matches), level=DEBUG) |
2695 | - return matches == total |
2696 | - |
2697 | - def comply(self, *args, **kwargs): |
2698 | - """NOOP since we just issue warnings. This is to avoid the |
2699 | - NotImplememtedError. |
2700 | - """ |
2701 | - log("Not applying any compliance criteria, only checks.", level=INFO) |
2702 | |
2703 | === removed file 'hooks/charmhelpers/contrib/hardening/harden.py' |
2704 | --- hooks/charmhelpers/contrib/hardening/harden.py 2016-04-22 08:15:33 +0000 |
2705 | +++ hooks/charmhelpers/contrib/hardening/harden.py 1970-01-01 00:00:00 +0000 |
2706 | @@ -1,84 +0,0 @@ |
2707 | -# Copyright 2016 Canonical Limited. |
2708 | -# |
2709 | -# This file is part of charm-helpers. |
2710 | -# |
2711 | -# charm-helpers is free software: you can redistribute it and/or modify |
2712 | -# it under the terms of the GNU Lesser General Public License version 3 as |
2713 | -# published by the Free Software Foundation. |
2714 | -# |
2715 | -# charm-helpers is distributed in the hope that it will be useful, |
2716 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
2717 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
2718 | -# GNU Lesser General Public License for more details. |
2719 | -# |
2720 | -# You should have received a copy of the GNU Lesser General Public License |
2721 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
2722 | - |
2723 | -import six |
2724 | - |
2725 | -from collections import OrderedDict |
2726 | - |
2727 | -from charmhelpers.core.hookenv import ( |
2728 | - config, |
2729 | - log, |
2730 | - DEBUG, |
2731 | - WARNING, |
2732 | -) |
2733 | -from charmhelpers.contrib.hardening.host.checks import run_os_checks |
2734 | -from charmhelpers.contrib.hardening.ssh.checks import run_ssh_checks |
2735 | -from charmhelpers.contrib.hardening.mysql.checks import run_mysql_checks |
2736 | -from charmhelpers.contrib.hardening.apache.checks import run_apache_checks |
2737 | - |
2738 | - |
2739 | -def harden(overrides=None): |
2740 | - """Hardening decorator. |
2741 | - |
2742 | - This is the main entry point for running the hardening stack. In order to |
2743 | - run modules of the stack you must add this decorator to charm hook(s) and |
2744 | - ensure that your charm config.yaml contains the 'harden' option set to |
2745 | - one or more of the supported modules. Setting these will cause the |
2746 | - corresponding hardening code to be run when the hook fires. |
2747 | - |
2748 | - This decorator can and should be applied to more than one hook or function |
2749 | - such that hardening modules are called multiple times. This is because |
2750 | - subsequent calls will perform auditing checks that will report any changes |
2751 | - to resources hardened by the first run (and possibly perform compliance |
2752 | - actions as a result of any detected infractions). |
2753 | - |
2754 | - :param overrides: Optional list of stack modules used to override those |
2755 | - provided with 'harden' config. |
2756 | - :returns: Returns value returned by decorated function once executed. |
2757 | - """ |
2758 | - def _harden_inner1(f): |
2759 | - log("Hardening function '%s'" % (f.__name__), level=DEBUG) |
2760 | - |
2761 | - def _harden_inner2(*args, **kwargs): |
2762 | - RUN_CATALOG = OrderedDict([('os', run_os_checks), |
2763 | - ('ssh', run_ssh_checks), |
2764 | - ('mysql', run_mysql_checks), |
2765 | - ('apache', run_apache_checks)]) |
2766 | - |
2767 | - enabled = overrides or (config("harden") or "").split() |
2768 | - if enabled: |
2769 | - modules_to_run = [] |
2770 | - # modules will always be performed in the following order |
2771 | - for module, func in six.iteritems(RUN_CATALOG): |
2772 | - if module in enabled: |
2773 | - enabled.remove(module) |
2774 | - modules_to_run.append(func) |
2775 | - |
2776 | - if enabled: |
2777 | - log("Unknown hardening modules '%s' - ignoring" % |
2778 | - (', '.join(enabled)), level=WARNING) |
2779 | - |
2780 | - for hardener in modules_to_run: |
2781 | - log("Executing hardening module '%s'" % |
2782 | - (hardener.__name__), level=DEBUG) |
2783 | - hardener() |
2784 | - else: |
2785 | - log("No hardening applied to '%s'" % (f.__name__), level=DEBUG) |
2786 | - |
2787 | - return f(*args, **kwargs) |
2788 | - return _harden_inner2 |
2789 | - |
2790 | - return _harden_inner1 |
2791 | |
2792 | === removed directory 'hooks/charmhelpers/contrib/hardening/host' |
2793 | === removed file 'hooks/charmhelpers/contrib/hardening/host/__init__.py' |
2794 | --- hooks/charmhelpers/contrib/hardening/host/__init__.py 2016-04-22 08:15:33 +0000 |
2795 | +++ hooks/charmhelpers/contrib/hardening/host/__init__.py 1970-01-01 00:00:00 +0000 |
2796 | @@ -1,19 +0,0 @@ |
2797 | -# Copyright 2016 Canonical Limited. |
2798 | -# |
2799 | -# This file is part of charm-helpers. |
2800 | -# |
2801 | -# charm-helpers is free software: you can redistribute it and/or modify |
2802 | -# it under the terms of the GNU Lesser General Public License version 3 as |
2803 | -# published by the Free Software Foundation. |
2804 | -# |
2805 | -# charm-helpers is distributed in the hope that it will be useful, |
2806 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
2807 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
2808 | -# GNU Lesser General Public License for more details. |
2809 | -# |
2810 | -# You should have received a copy of the GNU Lesser General Public License |
2811 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
2812 | - |
2813 | -from os import path |
2814 | - |
2815 | -TEMPLATES_DIR = path.join(path.dirname(__file__), 'templates') |
2816 | |
2817 | === removed directory 'hooks/charmhelpers/contrib/hardening/host/checks' |
2818 | === removed file 'hooks/charmhelpers/contrib/hardening/host/checks/__init__.py' |
2819 | --- hooks/charmhelpers/contrib/hardening/host/checks/__init__.py 2016-04-22 08:15:33 +0000 |
2820 | +++ hooks/charmhelpers/contrib/hardening/host/checks/__init__.py 1970-01-01 00:00:00 +0000 |
2821 | @@ -1,50 +0,0 @@ |
2822 | -# Copyright 2016 Canonical Limited. |
2823 | -# |
2824 | -# This file is part of charm-helpers. |
2825 | -# |
2826 | -# charm-helpers is free software: you can redistribute it and/or modify |
2827 | -# it under the terms of the GNU Lesser General Public License version 3 as |
2828 | -# published by the Free Software Foundation. |
2829 | -# |
2830 | -# charm-helpers is distributed in the hope that it will be useful, |
2831 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
2832 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
2833 | -# GNU Lesser General Public License for more details. |
2834 | -# |
2835 | -# You should have received a copy of the GNU Lesser General Public License |
2836 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
2837 | - |
2838 | -from charmhelpers.core.hookenv import ( |
2839 | - log, |
2840 | - DEBUG, |
2841 | -) |
2842 | -from charmhelpers.contrib.hardening.host.checks import ( |
2843 | - apt, |
2844 | - limits, |
2845 | - login, |
2846 | - minimize_access, |
2847 | - pam, |
2848 | - profile, |
2849 | - securetty, |
2850 | - suid_sgid, |
2851 | - sysctl |
2852 | -) |
2853 | - |
2854 | - |
2855 | -def run_os_checks(): |
2856 | - log("Starting OS hardening checks.", level=DEBUG) |
2857 | - checks = apt.get_audits() |
2858 | - checks.extend(limits.get_audits()) |
2859 | - checks.extend(login.get_audits()) |
2860 | - checks.extend(minimize_access.get_audits()) |
2861 | - checks.extend(pam.get_audits()) |
2862 | - checks.extend(profile.get_audits()) |
2863 | - checks.extend(securetty.get_audits()) |
2864 | - checks.extend(suid_sgid.get_audits()) |
2865 | - checks.extend(sysctl.get_audits()) |
2866 | - |
2867 | - for check in checks: |
2868 | - log("Running '%s' check" % (check.__class__.__name__), level=DEBUG) |
2869 | - check.ensure_compliance() |
2870 | - |
2871 | - log("OS hardening checks complete.", level=DEBUG) |
2872 | |
2873 | === removed file 'hooks/charmhelpers/contrib/hardening/host/checks/apt.py' |
2874 | --- hooks/charmhelpers/contrib/hardening/host/checks/apt.py 2016-04-22 08:15:33 +0000 |
2875 | +++ hooks/charmhelpers/contrib/hardening/host/checks/apt.py 1970-01-01 00:00:00 +0000 |
2876 | @@ -1,39 +0,0 @@ |
2877 | -# Copyright 2016 Canonical Limited. |
2878 | -# |
2879 | -# This file is part of charm-helpers. |
2880 | -# |
2881 | -# charm-helpers is free software: you can redistribute it and/or modify |
2882 | -# it under the terms of the GNU Lesser General Public License version 3 as |
2883 | -# published by the Free Software Foundation. |
2884 | -# |
2885 | -# charm-helpers is distributed in the hope that it will be useful, |
2886 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
2887 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
2888 | -# GNU Lesser General Public License for more details. |
2889 | -# |
2890 | -# You should have received a copy of the GNU Lesser General Public License |
2891 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
2892 | - |
2893 | -from charmhelpers.contrib.hardening.utils import get_settings |
2894 | -from charmhelpers.contrib.hardening.audits.apt import ( |
2895 | - AptConfig, |
2896 | - RestrictedPackages, |
2897 | -) |
2898 | - |
2899 | - |
2900 | -def get_audits(): |
2901 | - """Get OS hardening apt audits. |
2902 | - |
2903 | - :returns: dictionary of audits |
2904 | - """ |
2905 | - audits = [AptConfig([{'key': 'APT::Get::AllowUnauthenticated', |
2906 | - 'expected': 'false'}])] |
2907 | - |
2908 | - settings = get_settings('os') |
2909 | - clean_packages = settings['security']['packages_clean'] |
2910 | - if clean_packages: |
2911 | - security_packages = settings['security']['packages_list'] |
2912 | - if security_packages: |
2913 | - audits.append(RestrictedPackages(security_packages)) |
2914 | - |
2915 | - return audits |
2916 | |
2917 | === removed file 'hooks/charmhelpers/contrib/hardening/host/checks/limits.py' |
2918 | --- hooks/charmhelpers/contrib/hardening/host/checks/limits.py 2016-04-22 08:15:33 +0000 |
2919 | +++ hooks/charmhelpers/contrib/hardening/host/checks/limits.py 1970-01-01 00:00:00 +0000 |
2920 | @@ -1,55 +0,0 @@ |
2921 | -# Copyright 2016 Canonical Limited. |
2922 | -# |
2923 | -# This file is part of charm-helpers. |
2924 | -# |
2925 | -# charm-helpers is free software: you can redistribute it and/or modify |
2926 | -# it under the terms of the GNU Lesser General Public License version 3 as |
2927 | -# published by the Free Software Foundation. |
2928 | -# |
2929 | -# charm-helpers is distributed in the hope that it will be useful, |
2930 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
2931 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
2932 | -# GNU Lesser General Public License for more details. |
2933 | -# |
2934 | -# You should have received a copy of the GNU Lesser General Public License |
2935 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
2936 | - |
2937 | -from charmhelpers.contrib.hardening.audits.file import ( |
2938 | - DirectoryPermissionAudit, |
2939 | - TemplatedFile, |
2940 | -) |
2941 | -from charmhelpers.contrib.hardening.host import TEMPLATES_DIR |
2942 | -from charmhelpers.contrib.hardening import utils |
2943 | - |
2944 | - |
2945 | -def get_audits(): |
2946 | - """Get OS hardening security limits audits. |
2947 | - |
2948 | - :returns: dictionary of audits |
2949 | - """ |
2950 | - audits = [] |
2951 | - settings = utils.get_settings('os') |
2952 | - |
2953 | - # Ensure that the /etc/security/limits.d directory is only writable |
2954 | - # by the root user, but others can execute and read. |
2955 | - audits.append(DirectoryPermissionAudit('/etc/security/limits.d', |
2956 | - user='root', group='root', |
2957 | - mode=0o755)) |
2958 | - |
2959 | - # If core dumps are not enabled, then don't allow core dumps to be |
2960 | - # created as they may contain sensitive information. |
2961 | - if not settings['security']['kernel_enable_core_dump']: |
2962 | - audits.append(TemplatedFile('/etc/security/limits.d/10.hardcore.conf', |
2963 | - SecurityLimitsContext(), |
2964 | - template_dir=TEMPLATES_DIR, |
2965 | - user='root', group='root', mode=0o0440)) |
2966 | - return audits |
2967 | - |
2968 | - |
2969 | -class SecurityLimitsContext(object): |
2970 | - |
2971 | - def __call__(self): |
2972 | - settings = utils.get_settings('os') |
2973 | - ctxt = {'disable_core_dump': |
2974 | - not settings['security']['kernel_enable_core_dump']} |
2975 | - return ctxt |
2976 | |
2977 | === removed file 'hooks/charmhelpers/contrib/hardening/host/checks/login.py' |
2978 | --- hooks/charmhelpers/contrib/hardening/host/checks/login.py 2016-04-22 08:15:33 +0000 |
2979 | +++ hooks/charmhelpers/contrib/hardening/host/checks/login.py 1970-01-01 00:00:00 +0000 |
2980 | @@ -1,67 +0,0 @@ |
2981 | -# Copyright 2016 Canonical Limited. |
2982 | -# |
2983 | -# This file is part of charm-helpers. |
2984 | -# |
2985 | -# charm-helpers is free software: you can redistribute it and/or modify |
2986 | -# it under the terms of the GNU Lesser General Public License version 3 as |
2987 | -# published by the Free Software Foundation. |
2988 | -# |
2989 | -# charm-helpers is distributed in the hope that it will be useful, |
2990 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
2991 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
2992 | -# GNU Lesser General Public License for more details. |
2993 | -# |
2994 | -# You should have received a copy of the GNU Lesser General Public License |
2995 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
2996 | - |
2997 | -from six import string_types |
2998 | - |
2999 | -from charmhelpers.contrib.hardening.audits.file import TemplatedFile |
3000 | -from charmhelpers.contrib.hardening.host import TEMPLATES_DIR |
3001 | -from charmhelpers.contrib.hardening import utils |
3002 | - |
3003 | - |
3004 | -def get_audits(): |
3005 | - """Get OS hardening login.defs audits. |
3006 | - |
3007 | - :returns: dictionary of audits |
3008 | - """ |
3009 | - audits = [TemplatedFile('/etc/login.defs', LoginContext(), |
3010 | - template_dir=TEMPLATES_DIR, |
3011 | - user='root', group='root', mode=0o0444)] |
3012 | - return audits |
3013 | - |
3014 | - |
3015 | -class LoginContext(object): |
3016 | - |
3017 | - def __call__(self): |
3018 | - settings = utils.get_settings('os') |
3019 | - |
3020 | - # Octal numbers in yaml end up being turned into decimal, |
3021 | - # so check if the umask is entered as a string (e.g. '027') |
3022 | - # or as an octal umask as we know it (e.g. 002). If its not |
3023 | - # a string assume it to be octal and turn it into an octal |
3024 | - # string. |
3025 | - umask = settings['environment']['umask'] |
3026 | - if not isinstance(umask, string_types): |
3027 | - umask = '%s' % oct(umask) |
3028 | - |
3029 | - ctxt = { |
3030 | - 'additional_user_paths': |
3031 | - settings['environment']['extra_user_paths'], |
3032 | - 'umask': umask, |
3033 | - 'pwd_max_age': settings['auth']['pw_max_age'], |
3034 | - 'pwd_min_age': settings['auth']['pw_min_age'], |
3035 | - 'uid_min': settings['auth']['uid_min'], |
3036 | - 'sys_uid_min': settings['auth']['sys_uid_min'], |
3037 | - 'sys_uid_max': settings['auth']['sys_uid_max'], |
3038 | - 'gid_min': settings['auth']['gid_min'], |
3039 | - 'sys_gid_min': settings['auth']['sys_gid_min'], |
3040 | - 'sys_gid_max': settings['auth']['sys_gid_max'], |
3041 | - 'login_retries': settings['auth']['retries'], |
3042 | - 'login_timeout': settings['auth']['timeout'], |
3043 | - 'chfn_restrict': settings['auth']['chfn_restrict'], |
3044 | - 'allow_login_without_home': settings['auth']['allow_homeless'] |
3045 | - } |
3046 | - |
3047 | - return ctxt |
3048 | |
3049 | === removed file 'hooks/charmhelpers/contrib/hardening/host/checks/minimize_access.py' |
3050 | --- hooks/charmhelpers/contrib/hardening/host/checks/minimize_access.py 2016-04-22 08:15:33 +0000 |
3051 | +++ hooks/charmhelpers/contrib/hardening/host/checks/minimize_access.py 1970-01-01 00:00:00 +0000 |
3052 | @@ -1,52 +0,0 @@ |
3053 | -# Copyright 2016 Canonical Limited. |
3054 | -# |
3055 | -# This file is part of charm-helpers. |
3056 | -# |
3057 | -# charm-helpers is free software: you can redistribute it and/or modify |
3058 | -# it under the terms of the GNU Lesser General Public License version 3 as |
3059 | -# published by the Free Software Foundation. |
3060 | -# |
3061 | -# charm-helpers is distributed in the hope that it will be useful, |
3062 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
3063 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
3064 | -# GNU Lesser General Public License for more details. |
3065 | -# |
3066 | -# You should have received a copy of the GNU Lesser General Public License |
3067 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
3068 | - |
3069 | -from charmhelpers.contrib.hardening.audits.file import ( |
3070 | - FilePermissionAudit, |
3071 | - ReadOnly, |
3072 | -) |
3073 | -from charmhelpers.contrib.hardening import utils |
3074 | - |
3075 | - |
3076 | -def get_audits(): |
3077 | - """Get OS hardening access audits. |
3078 | - |
3079 | - :returns: dictionary of audits |
3080 | - """ |
3081 | - audits = [] |
3082 | - settings = utils.get_settings('os') |
3083 | - |
3084 | - # Remove write permissions from $PATH folders for all regular users. |
3085 | - # This prevents changing system-wide commands from normal users. |
3086 | - path_folders = {'/usr/local/sbin', |
3087 | - '/usr/local/bin', |
3088 | - '/usr/sbin', |
3089 | - '/usr/bin', |
3090 | - '/bin'} |
3091 | - extra_user_paths = settings['environment']['extra_user_paths'] |
3092 | - path_folders.update(extra_user_paths) |
3093 | - audits.append(ReadOnly(path_folders)) |
3094 | - |
3095 | - # Only allow the root user to have access to the shadow file. |
3096 | - audits.append(FilePermissionAudit('/etc/shadow', 'root', 'root', 0o0600)) |
3097 | - |
3098 | - if 'change_user' not in settings['security']['users_allow']: |
3099 | - # su should only be accessible to user and group root, unless it is |
3100 | - # expressly defined to allow users to change to root via the |
3101 | - # security_users_allow config option. |
3102 | - audits.append(FilePermissionAudit('/bin/su', 'root', 'root', 0o750)) |
3103 | - |
3104 | - return audits |
3105 | |
3106 | === removed file 'hooks/charmhelpers/contrib/hardening/host/checks/pam.py' |
3107 | --- hooks/charmhelpers/contrib/hardening/host/checks/pam.py 2016-04-22 08:15:33 +0000 |
3108 | +++ hooks/charmhelpers/contrib/hardening/host/checks/pam.py 1970-01-01 00:00:00 +0000 |
3109 | @@ -1,134 +0,0 @@ |
3110 | -# Copyright 2016 Canonical Limited. |
3111 | -# |
3112 | -# This file is part of charm-helpers. |
3113 | -# |
3114 | -# charm-helpers is free software: you can redistribute it and/or modify |
3115 | -# it under the terms of the GNU Lesser General Public License version 3 as |
3116 | -# published by the Free Software Foundation. |
3117 | -# |
3118 | -# charm-helpers is distributed in the hope that it will be useful, |
3119 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
3120 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
3121 | -# GNU Lesser General Public License for more details. |
3122 | -# |
3123 | -# You should have received a copy of the GNU Lesser General Public License |
3124 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
3125 | - |
3126 | -from subprocess import ( |
3127 | - check_output, |
3128 | - CalledProcessError, |
3129 | -) |
3130 | - |
3131 | -from charmhelpers.core.hookenv import ( |
3132 | - log, |
3133 | - DEBUG, |
3134 | - ERROR, |
3135 | -) |
3136 | -from charmhelpers.fetch import ( |
3137 | - apt_install, |
3138 | - apt_purge, |
3139 | - apt_update, |
3140 | -) |
3141 | -from charmhelpers.contrib.hardening.audits.file import ( |
3142 | - TemplatedFile, |
3143 | - DeletedFile, |
3144 | -) |
3145 | -from charmhelpers.contrib.hardening import utils |
3146 | -from charmhelpers.contrib.hardening.host import TEMPLATES_DIR |
3147 | - |
3148 | - |
3149 | -def get_audits(): |
3150 | - """Get OS hardening PAM authentication audits. |
3151 | - |
3152 | - :returns: dictionary of audits |
3153 | - """ |
3154 | - audits = [] |
3155 | - |
3156 | - settings = utils.get_settings('os') |
3157 | - |
3158 | - if settings['auth']['pam_passwdqc_enable']: |
3159 | - audits.append(PasswdqcPAM('/etc/passwdqc.conf')) |
3160 | - |
3161 | - if settings['auth']['retries']: |
3162 | - audits.append(Tally2PAM('/usr/share/pam-configs/tally2')) |
3163 | - else: |
3164 | - audits.append(DeletedFile('/usr/share/pam-configs/tally2')) |
3165 | - |
3166 | - return audits |
3167 | - |
3168 | - |
3169 | -class PasswdqcPAMContext(object): |
3170 | - |
3171 | - def __call__(self): |
3172 | - ctxt = {} |
3173 | - settings = utils.get_settings('os') |
3174 | - |
3175 | - ctxt['auth_pam_passwdqc_options'] = \ |
3176 | - settings['auth']['pam_passwdqc_options'] |
3177 | - |
3178 | - return ctxt |
3179 | - |
3180 | - |
3181 | -class PasswdqcPAM(TemplatedFile): |
3182 | - """The PAM Audit verifies the linux PAM settings.""" |
3183 | - def __init__(self, path): |
3184 | - super(PasswdqcPAM, self).__init__(path=path, |
3185 | - template_dir=TEMPLATES_DIR, |
3186 | - context=PasswdqcPAMContext(), |
3187 | - user='root', |
3188 | - group='root', |
3189 | - mode=0o0640) |
3190 | - |
3191 | - def pre_write(self): |
3192 | - # Always remove? |
3193 | - for pkg in ['libpam-ccreds', 'libpam-cracklib']: |
3194 | - log("Purging package '%s'" % pkg, level=DEBUG), |
3195 | - apt_purge(pkg) |
3196 | - |
3197 | - apt_update(fatal=True) |
3198 | - for pkg in ['libpam-passwdqc']: |
3199 | - log("Installing package '%s'" % pkg, level=DEBUG), |
3200 | - apt_install(pkg) |
3201 | - |
3202 | - def post_write(self): |
3203 | - """Updates the PAM configuration after the file has been written""" |
3204 | - try: |
3205 | - check_output(['pam-auth-update', '--package']) |
3206 | - except CalledProcessError as e: |
3207 | - log('Error calling pam-auth-update: %s' % e, level=ERROR) |
3208 | - |
3209 | - |
3210 | -class Tally2PAMContext(object): |
3211 | - |
3212 | - def __call__(self): |
3213 | - ctxt = {} |
3214 | - settings = utils.get_settings('os') |
3215 | - |
3216 | - ctxt['auth_lockout_time'] = settings['auth']['lockout_time'] |
3217 | - ctxt['auth_retries'] = settings['auth']['retries'] |
3218 | - |
3219 | - return ctxt |
3220 | - |
3221 | - |
3222 | -class Tally2PAM(TemplatedFile): |
3223 | - """The PAM Audit verifies the linux PAM settings.""" |
3224 | - def __init__(self, path): |
3225 | - super(Tally2PAM, self).__init__(path=path, |
3226 | - template_dir=TEMPLATES_DIR, |
3227 | - context=Tally2PAMContext(), |
3228 | - user='root', |
3229 | - group='root', |
3230 | - mode=0o0640) |
3231 | - |
3232 | - def pre_write(self): |
3233 | - # Always remove? |
3234 | - apt_purge('libpam-ccreds') |
3235 | - apt_update(fatal=True) |
3236 | - apt_install('libpam-modules') |
3237 | - |
3238 | - def post_write(self): |
3239 | - """Updates the PAM configuration after the file has been written""" |
3240 | - try: |
3241 | - check_output(['pam-auth-update', '--package']) |
3242 | - except CalledProcessError as e: |
3243 | - log('Error calling pam-auth-update: %s' % e, level=ERROR) |
3244 | |
3245 | === removed file 'hooks/charmhelpers/contrib/hardening/host/checks/profile.py' |
3246 | --- hooks/charmhelpers/contrib/hardening/host/checks/profile.py 2016-04-22 08:15:33 +0000 |
3247 | +++ hooks/charmhelpers/contrib/hardening/host/checks/profile.py 1970-01-01 00:00:00 +0000 |
3248 | @@ -1,45 +0,0 @@ |
3249 | -# Copyright 2016 Canonical Limited. |
3250 | -# |
3251 | -# This file is part of charm-helpers. |
3252 | -# |
3253 | -# charm-helpers is free software: you can redistribute it and/or modify |
3254 | -# it under the terms of the GNU Lesser General Public License version 3 as |
3255 | -# published by the Free Software Foundation. |
3256 | -# |
3257 | -# charm-helpers is distributed in the hope that it will be useful, |
3258 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
3259 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
3260 | -# GNU Lesser General Public License for more details. |
3261 | -# |
3262 | -# You should have received a copy of the GNU Lesser General Public License |
3263 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
3264 | - |
3265 | -from charmhelpers.contrib.hardening.audits.file import TemplatedFile |
3266 | -from charmhelpers.contrib.hardening.host import TEMPLATES_DIR |
3267 | -from charmhelpers.contrib.hardening import utils |
3268 | - |
3269 | - |
3270 | -def get_audits(): |
3271 | - """Get OS hardening profile audits. |
3272 | - |
3273 | - :returns: dictionary of audits |
3274 | - """ |
3275 | - audits = [] |
3276 | - |
3277 | - settings = utils.get_settings('os') |
3278 | - |
3279 | - # If core dumps are not enabled, then don't allow core dumps to be |
3280 | - # created as they may contain sensitive information. |
3281 | - if not settings['security']['kernel_enable_core_dump']: |
3282 | - audits.append(TemplatedFile('/etc/profile.d/pinerolo_profile.sh', |
3283 | - ProfileContext(), |
3284 | - template_dir=TEMPLATES_DIR, |
3285 | - mode=0o0755, user='root', group='root')) |
3286 | - return audits |
3287 | - |
3288 | - |
3289 | -class ProfileContext(object): |
3290 | - |
3291 | - def __call__(self): |
3292 | - ctxt = {} |
3293 | - return ctxt |
3294 | |
3295 | === removed file 'hooks/charmhelpers/contrib/hardening/host/checks/securetty.py' |
3296 | --- hooks/charmhelpers/contrib/hardening/host/checks/securetty.py 2016-04-22 08:15:33 +0000 |
3297 | +++ hooks/charmhelpers/contrib/hardening/host/checks/securetty.py 1970-01-01 00:00:00 +0000 |
3298 | @@ -1,39 +0,0 @@ |
3299 | -# Copyright 2016 Canonical Limited. |
3300 | -# |
3301 | -# This file is part of charm-helpers. |
3302 | -# |
3303 | -# charm-helpers is free software: you can redistribute it and/or modify |
3304 | -# it under the terms of the GNU Lesser General Public License version 3 as |
3305 | -# published by the Free Software Foundation. |
3306 | -# |
3307 | -# charm-helpers is distributed in the hope that it will be useful, |
3308 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
3309 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
3310 | -# GNU Lesser General Public License for more details. |
3311 | -# |
3312 | -# You should have received a copy of the GNU Lesser General Public License |
3313 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
3314 | - |
3315 | -from charmhelpers.contrib.hardening.audits.file import TemplatedFile |
3316 | -from charmhelpers.contrib.hardening.host import TEMPLATES_DIR |
3317 | -from charmhelpers.contrib.hardening import utils |
3318 | - |
3319 | - |
3320 | -def get_audits(): |
3321 | - """Get OS hardening Secure TTY audits. |
3322 | - |
3323 | - :returns: dictionary of audits |
3324 | - """ |
3325 | - audits = [] |
3326 | - audits.append(TemplatedFile('/etc/securetty', SecureTTYContext(), |
3327 | - template_dir=TEMPLATES_DIR, |
3328 | - mode=0o0400, user='root', group='root')) |
3329 | - return audits |
3330 | - |
3331 | - |
3332 | -class SecureTTYContext(object): |
3333 | - |
3334 | - def __call__(self): |
3335 | - settings = utils.get_settings('os') |
3336 | - ctxt = {'ttys': settings['auth']['root_ttys']} |
3337 | - return ctxt |
3338 | |
3339 | === removed file 'hooks/charmhelpers/contrib/hardening/host/checks/suid_sgid.py' |
3340 | --- hooks/charmhelpers/contrib/hardening/host/checks/suid_sgid.py 2016-04-22 08:15:33 +0000 |
3341 | +++ hooks/charmhelpers/contrib/hardening/host/checks/suid_sgid.py 1970-01-01 00:00:00 +0000 |
3342 | @@ -1,131 +0,0 @@ |
3343 | -# Copyright 2016 Canonical Limited. |
3344 | -# |
3345 | -# This file is part of charm-helpers. |
3346 | -# |
3347 | -# charm-helpers is free software: you can redistribute it and/or modify |
3348 | -# it under the terms of the GNU Lesser General Public License version 3 as |
3349 | -# published by the Free Software Foundation. |
3350 | -# |
3351 | -# charm-helpers is distributed in the hope that it will be useful, |
3352 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
3353 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
3354 | -# GNU Lesser General Public License for more details. |
3355 | -# |
3356 | -# You should have received a copy of the GNU Lesser General Public License |
3357 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
3358 | - |
3359 | -import subprocess |
3360 | - |
3361 | -from charmhelpers.core.hookenv import ( |
3362 | - log, |
3363 | - INFO, |
3364 | -) |
3365 | -from charmhelpers.contrib.hardening.audits.file import NoSUIDSGIDAudit |
3366 | -from charmhelpers.contrib.hardening import utils |
3367 | - |
3368 | - |
3369 | -BLACKLIST = ['/usr/bin/rcp', '/usr/bin/rlogin', '/usr/bin/rsh', |
3370 | - '/usr/libexec/openssh/ssh-keysign', |
3371 | - '/usr/lib/openssh/ssh-keysign', |
3372 | - '/sbin/netreport', |
3373 | - '/usr/sbin/usernetctl', |
3374 | - '/usr/sbin/userisdnctl', |
3375 | - '/usr/sbin/pppd', |
3376 | - '/usr/bin/lockfile', |
3377 | - '/usr/bin/mail-lock', |
3378 | - '/usr/bin/mail-unlock', |
3379 | - '/usr/bin/mail-touchlock', |
3380 | - '/usr/bin/dotlockfile', |
3381 | - '/usr/bin/arping', |
3382 | - '/usr/sbin/uuidd', |
3383 | - '/usr/bin/mtr', |
3384 | - '/usr/lib/evolution/camel-lock-helper-1.2', |
3385 | - '/usr/lib/pt_chown', |
3386 | - '/usr/lib/eject/dmcrypt-get-device', |
3387 | - '/usr/lib/mc/cons.saver'] |
3388 | - |
3389 | -WHITELIST = ['/bin/mount', '/bin/ping', '/bin/su', '/bin/umount', |
3390 | - '/sbin/pam_timestamp_check', '/sbin/unix_chkpwd', '/usr/bin/at', |
3391 | - '/usr/bin/gpasswd', '/usr/bin/locate', '/usr/bin/newgrp', |
3392 | - '/usr/bin/passwd', '/usr/bin/ssh-agent', |
3393 | - '/usr/libexec/utempter/utempter', '/usr/sbin/lockdev', |
3394 | - '/usr/sbin/sendmail.sendmail', '/usr/bin/expiry', |
3395 | - '/bin/ping6', '/usr/bin/traceroute6.iputils', |
3396 | - '/sbin/mount.nfs', '/sbin/umount.nfs', |
3397 | - '/sbin/mount.nfs4', '/sbin/umount.nfs4', |
3398 | - '/usr/bin/crontab', |
3399 | - '/usr/bin/wall', '/usr/bin/write', |
3400 | - '/usr/bin/screen', |
3401 | - '/usr/bin/mlocate', |
3402 | - '/usr/bin/chage', '/usr/bin/chfn', '/usr/bin/chsh', |
3403 | - '/bin/fusermount', |
3404 | - '/usr/bin/pkexec', |
3405 | - '/usr/bin/sudo', '/usr/bin/sudoedit', |
3406 | - '/usr/sbin/postdrop', '/usr/sbin/postqueue', |
3407 | - '/usr/sbin/suexec', |
3408 | - '/usr/lib/squid/ncsa_auth', '/usr/lib/squid/pam_auth', |
3409 | - '/usr/kerberos/bin/ksu', |
3410 | - '/usr/sbin/ccreds_validate', |
3411 | - '/usr/bin/Xorg', |
3412 | - '/usr/bin/X', |
3413 | - '/usr/lib/dbus-1.0/dbus-daemon-launch-helper', |
3414 | - '/usr/lib/vte/gnome-pty-helper', |
3415 | - '/usr/lib/libvte9/gnome-pty-helper', |
3416 | - '/usr/lib/libvte-2.90-9/gnome-pty-helper'] |
3417 | - |
3418 | - |
3419 | -def get_audits(): |
3420 | - """Get OS hardening suid/sgid audits. |
3421 | - |
3422 | - :returns: dictionary of audits |
3423 | - """ |
3424 | - checks = [] |
3425 | - settings = utils.get_settings('os') |
3426 | - if not settings['security']['suid_sgid_enforce']: |
3427 | - log("Skipping suid/sgid hardening", level=INFO) |
3428 | - return checks |
3429 | - |
3430 | - # Build the blacklist and whitelist of files for suid/sgid checks. |
3431 | - # There are a total of 4 lists: |
3432 | - # 1. the system blacklist |
3433 | - # 2. the system whitelist |
3434 | - # 3. the user blacklist |
3435 | - # 4. the user whitelist |
3436 | - # |
3437 | - # The blacklist is the set of paths which should NOT have the suid/sgid bit |
3438 | - # set and the whitelist is the set of paths which MAY have the suid/sgid |
3439 | - # bit setl. The user whitelist/blacklist effectively override the system |
3440 | - # whitelist/blacklist. |
3441 | - u_b = settings['security']['suid_sgid_blacklist'] |
3442 | - u_w = settings['security']['suid_sgid_whitelist'] |
3443 | - |
3444 | - blacklist = set(BLACKLIST) - set(u_w + u_b) |
3445 | - whitelist = set(WHITELIST) - set(u_b + u_w) |
3446 | - |
3447 | - checks.append(NoSUIDSGIDAudit(blacklist)) |
3448 | - |
3449 | - dry_run = settings['security']['suid_sgid_dry_run_on_unknown'] |
3450 | - |
3451 | - if settings['security']['suid_sgid_remove_from_unknown'] or dry_run: |
3452 | - # If the policy is a dry_run (e.g. complain only) or remove unknown |
3453 | - # suid/sgid bits then find all of the paths which have the suid/sgid |
3454 | - # bit set and then remove the whitelisted paths. |
3455 | - root_path = settings['environment']['root_path'] |
3456 | - unknown_paths = find_paths_with_suid_sgid(root_path) - set(whitelist) |
3457 | - checks.append(NoSUIDSGIDAudit(unknown_paths, unless=dry_run)) |
3458 | - |
3459 | - return checks |
3460 | - |
3461 | - |
3462 | -def find_paths_with_suid_sgid(root_path): |
3463 | - """Finds all paths/files which have an suid/sgid bit enabled. |
3464 | - |
3465 | - Starting with the root_path, this will recursively find all paths which |
3466 | - have an suid or sgid bit set. |
3467 | - """ |
3468 | - cmd = ['find', root_path, '-perm', '-4000', '-o', '-perm', '-2000', |
3469 | - '-type', 'f', '!', '-path', '/proc/*', '-print'] |
3470 | - |
3471 | - p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE) |
3472 | - out, _ = p.communicate() |
3473 | - return set(out.split('\n')) |
3474 | |
3475 | === removed file 'hooks/charmhelpers/contrib/hardening/host/checks/sysctl.py' |
3476 | --- hooks/charmhelpers/contrib/hardening/host/checks/sysctl.py 2016-04-22 08:15:33 +0000 |
3477 | +++ hooks/charmhelpers/contrib/hardening/host/checks/sysctl.py 1970-01-01 00:00:00 +0000 |
3478 | @@ -1,211 +0,0 @@ |
3479 | -# Copyright 2016 Canonical Limited. |
3480 | -# |
3481 | -# This file is part of charm-helpers. |
3482 | -# |
3483 | -# charm-helpers is free software: you can redistribute it and/or modify |
3484 | -# it under the terms of the GNU Lesser General Public License version 3 as |
3485 | -# published by the Free Software Foundation. |
3486 | -# |
3487 | -# charm-helpers is distributed in the hope that it will be useful, |
3488 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
3489 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
3490 | -# GNU Lesser General Public License for more details. |
3491 | -# |
3492 | -# You should have received a copy of the GNU Lesser General Public License |
3493 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
3494 | - |
3495 | -import os |
3496 | -import platform |
3497 | -import re |
3498 | -import six |
3499 | -import subprocess |
3500 | - |
3501 | -from charmhelpers.core.hookenv import ( |
3502 | - log, |
3503 | - INFO, |
3504 | - WARNING, |
3505 | -) |
3506 | -from charmhelpers.contrib.hardening import utils |
3507 | -from charmhelpers.contrib.hardening.audits.file import ( |
3508 | - FilePermissionAudit, |
3509 | - TemplatedFile, |
3510 | -) |
3511 | -from charmhelpers.contrib.hardening.host import TEMPLATES_DIR |
3512 | - |
3513 | - |
3514 | -SYSCTL_DEFAULTS = """net.ipv4.ip_forward=%(net_ipv4_ip_forward)s |
3515 | -net.ipv6.conf.all.forwarding=%(net_ipv6_conf_all_forwarding)s |
3516 | -net.ipv4.conf.all.rp_filter=1 |
3517 | -net.ipv4.conf.default.rp_filter=1 |
3518 | -net.ipv4.icmp_echo_ignore_broadcasts=1 |
3519 | -net.ipv4.icmp_ignore_bogus_error_responses=1 |
3520 | -net.ipv4.icmp_ratelimit=100 |
3521 | -net.ipv4.icmp_ratemask=88089 |
3522 | -net.ipv6.conf.all.disable_ipv6=%(net_ipv6_conf_all_disable_ipv6)s |
3523 | -net.ipv4.tcp_timestamps=%(net_ipv4_tcp_timestamps)s |
3524 | -net.ipv4.conf.all.arp_ignore=%(net_ipv4_conf_all_arp_ignore)s |
3525 | -net.ipv4.conf.all.arp_announce=%(net_ipv4_conf_all_arp_announce)s |
3526 | -net.ipv4.tcp_rfc1337=1 |
3527 | -net.ipv4.tcp_syncookies=1 |
3528 | -net.ipv4.conf.all.shared_media=1 |
3529 | -net.ipv4.conf.default.shared_media=1 |
3530 | -net.ipv4.conf.all.accept_source_route=0 |
3531 | -net.ipv4.conf.default.accept_source_route=0 |
3532 | -net.ipv4.conf.all.accept_redirects=0 |
3533 | -net.ipv4.conf.default.accept_redirects=0 |
3534 | -net.ipv6.conf.all.accept_redirects=0 |
3535 | -net.ipv6.conf.default.accept_redirects=0 |
3536 | -net.ipv4.conf.all.secure_redirects=0 |
3537 | -net.ipv4.conf.default.secure_redirects=0 |
3538 | -net.ipv4.conf.all.send_redirects=0 |
3539 | -net.ipv4.conf.default.send_redirects=0 |
3540 | -net.ipv4.conf.all.log_martians=0 |
3541 | -net.ipv6.conf.default.router_solicitations=0 |
3542 | -net.ipv6.conf.default.accept_ra_rtr_pref=0 |
3543 | -net.ipv6.conf.default.accept_ra_pinfo=0 |
3544 | -net.ipv6.conf.default.accept_ra_defrtr=0 |
3545 | -net.ipv6.conf.default.autoconf=0 |
3546 | -net.ipv6.conf.default.dad_transmits=0 |
3547 | -net.ipv6.conf.default.max_addresses=1 |
3548 | -net.ipv6.conf.all.accept_ra=0 |
3549 | -net.ipv6.conf.default.accept_ra=0 |
3550 | -kernel.modules_disabled=%(kernel_modules_disabled)s |
3551 | -kernel.sysrq=%(kernel_sysrq)s |
3552 | -fs.suid_dumpable=%(fs_suid_dumpable)s |
3553 | -kernel.randomize_va_space=2 |
3554 | -""" |
3555 | - |
3556 | - |
3557 | -def get_audits(): |
3558 | - """Get OS hardening sysctl audits. |
3559 | - |
3560 | - :returns: dictionary of audits |
3561 | - """ |
3562 | - audits = [] |
3563 | - settings = utils.get_settings('os') |
3564 | - |
3565 | - # Apply the sysctl settings which are configured to be applied. |
3566 | - audits.append(SysctlConf()) |
3567 | - # Make sure that only root has access to the sysctl.conf file, and |
3568 | - # that it is read-only. |
3569 | - audits.append(FilePermissionAudit('/etc/sysctl.conf', |
3570 | - user='root', |
3571 | - group='root', mode=0o0440)) |
3572 | - # If module loading is not enabled, then ensure that the modules |
3573 | - # file has the appropriate permissions and rebuild the initramfs |
3574 | - if not settings['security']['kernel_enable_module_loading']: |
3575 | - audits.append(ModulesTemplate()) |
3576 | - |
3577 | - return audits |
3578 | - |
3579 | - |
3580 | -class ModulesContext(object): |
3581 | - |
3582 | - def __call__(self): |
3583 | - settings = utils.get_settings('os') |
3584 | - with open('/proc/cpuinfo', 'r') as fd: |
3585 | - cpuinfo = fd.readlines() |
3586 | - |
3587 | - for line in cpuinfo: |
3588 | - match = re.search(r"^vendor_id\s+:\s+(.+)", line) |
3589 | - if match: |
3590 | - vendor = match.group(1) |
3591 | - |
3592 | - if vendor == "GenuineIntel": |
3593 | - vendor = "intel" |
3594 | - elif vendor == "AuthenticAMD": |
3595 | - vendor = "amd" |
3596 | - |
3597 | - ctxt = {'arch': platform.processor(), |
3598 | - 'cpuVendor': vendor, |
3599 | - 'desktop_enable': settings['general']['desktop_enable']} |
3600 | - |
3601 | - return ctxt |
3602 | - |
3603 | - |
3604 | -class ModulesTemplate(object): |
3605 | - |
3606 | - def __init__(self): |
3607 | - super(ModulesTemplate, self).__init__('/etc/initramfs-tools/modules', |
3608 | - ModulesContext(), |
3609 | - templates_dir=TEMPLATES_DIR, |
3610 | - user='root', group='root', |
3611 | - mode=0o0440) |
3612 | - |
3613 | - def post_write(self): |
3614 | - subprocess.check_call(['update-initramfs', '-u']) |
3615 | - |
3616 | - |
3617 | -class SysCtlHardeningContext(object): |
3618 | - def __call__(self): |
3619 | - settings = utils.get_settings('os') |
3620 | - ctxt = {'sysctl': {}} |
3621 | - |
3622 | - log("Applying sysctl settings", level=INFO) |
3623 | - extras = {'net_ipv4_ip_forward': 0, |
3624 | - 'net_ipv6_conf_all_forwarding': 0, |
3625 | - 'net_ipv6_conf_all_disable_ipv6': 1, |
3626 | - 'net_ipv4_tcp_timestamps': 0, |
3627 | - 'net_ipv4_conf_all_arp_ignore': 0, |
3628 | - 'net_ipv4_conf_all_arp_announce': 0, |
3629 | - 'kernel_sysrq': 0, |
3630 | - 'fs_suid_dumpable': 0, |
3631 | - 'kernel_modules_disabled': 1} |
3632 | - |
3633 | - if settings['sysctl']['ipv6_enable']: |
3634 | - extras['net_ipv6_conf_all_disable_ipv6'] = 0 |
3635 | - |
3636 | - if settings['sysctl']['forwarding']: |
3637 | - extras['net_ipv4_ip_forward'] = 1 |
3638 | - extras['net_ipv6_conf_all_forwarding'] = 1 |
3639 | - |
3640 | - if settings['sysctl']['arp_restricted']: |
3641 | - extras['net_ipv4_conf_all_arp_ignore'] = 1 |
3642 | - extras['net_ipv4_conf_all_arp_announce'] = 2 |
3643 | - |
3644 | - if settings['security']['kernel_enable_module_loading']: |
3645 | - extras['kernel_modules_disabled'] = 0 |
3646 | - |
3647 | - if settings['sysctl']['kernel_enable_sysrq']: |
3648 | - sysrq_val = settings['sysctl']['kernel_secure_sysrq'] |
3649 | - extras['kernel_sysrq'] = sysrq_val |
3650 | - |
3651 | - if settings['security']['kernel_enable_core_dump']: |
3652 | - extras['fs_suid_dumpable'] = 1 |
3653 | - |
3654 | - settings.update(extras) |
3655 | - for d in (SYSCTL_DEFAULTS % settings).split(): |
3656 | - d = d.strip().partition('=') |
3657 | - key = d[0].strip() |
3658 | - path = os.path.join('/proc/sys', key.replace('.', '/')) |
3659 | - if not os.path.exists(path): |
3660 | - log("Skipping '%s' since '%s' does not exist" % (key, path), |
3661 | - level=WARNING) |
3662 | - continue |
3663 | - |
3664 | - ctxt['sysctl'][key] = d[2] or None |
3665 | - |
3666 | - # Translate for python3 |
3667 | - return {'sysctl_settings': |
3668 | - [(k, v) for k, v in six.iteritems(ctxt['sysctl'])]} |
3669 | - |
3670 | - |
3671 | -class SysctlConf(TemplatedFile): |
3672 | - """An audit check for sysctl settings.""" |
3673 | - def __init__(self): |
3674 | - self.conffile = '/etc/sysctl.d/99-juju-hardening.conf' |
3675 | - super(SysctlConf, self).__init__(self.conffile, |
3676 | - SysCtlHardeningContext(), |
3677 | - template_dir=TEMPLATES_DIR, |
3678 | - user='root', group='root', |
3679 | - mode=0o0440) |
3680 | - |
3681 | - def post_write(self): |
3682 | - try: |
3683 | - subprocess.check_call(['sysctl', '-p', self.conffile]) |
3684 | - except subprocess.CalledProcessError as e: |
3685 | - # NOTE: on some systems if sysctl cannot apply all settings it |
3686 | - # will return non-zero as well. |
3687 | - log("sysctl command returned an error (maybe some " |
3688 | - "keys could not be set) - %s" % (e), |
3689 | - level=WARNING) |
3690 | |
3691 | === removed directory 'hooks/charmhelpers/contrib/hardening/mysql' |
3692 | === removed file 'hooks/charmhelpers/contrib/hardening/mysql/__init__.py' |
3693 | --- hooks/charmhelpers/contrib/hardening/mysql/__init__.py 2016-04-22 08:15:33 +0000 |
3694 | +++ hooks/charmhelpers/contrib/hardening/mysql/__init__.py 1970-01-01 00:00:00 +0000 |
3695 | @@ -1,19 +0,0 @@ |
3696 | -# Copyright 2016 Canonical Limited. |
3697 | -# |
3698 | -# This file is part of charm-helpers. |
3699 | -# |
3700 | -# charm-helpers is free software: you can redistribute it and/or modify |
3701 | -# it under the terms of the GNU Lesser General Public License version 3 as |
3702 | -# published by the Free Software Foundation. |
3703 | -# |
3704 | -# charm-helpers is distributed in the hope that it will be useful, |
3705 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
3706 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
3707 | -# GNU Lesser General Public License for more details. |
3708 | -# |
3709 | -# You should have received a copy of the GNU Lesser General Public License |
3710 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
3711 | - |
3712 | -from os import path |
3713 | - |
3714 | -TEMPLATES_DIR = path.join(path.dirname(__file__), 'templates') |
3715 | |
3716 | === removed directory 'hooks/charmhelpers/contrib/hardening/mysql/checks' |
3717 | === removed file 'hooks/charmhelpers/contrib/hardening/mysql/checks/__init__.py' |
3718 | --- hooks/charmhelpers/contrib/hardening/mysql/checks/__init__.py 2016-04-22 08:15:33 +0000 |
3719 | +++ hooks/charmhelpers/contrib/hardening/mysql/checks/__init__.py 1970-01-01 00:00:00 +0000 |
3720 | @@ -1,31 +0,0 @@ |
3721 | -# Copyright 2016 Canonical Limited. |
3722 | -# |
3723 | -# This file is part of charm-helpers. |
3724 | -# |
3725 | -# charm-helpers is free software: you can redistribute it and/or modify |
3726 | -# it under the terms of the GNU Lesser General Public License version 3 as |
3727 | -# published by the Free Software Foundation. |
3728 | -# |
3729 | -# charm-helpers is distributed in the hope that it will be useful, |
3730 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
3731 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
3732 | -# GNU Lesser General Public License for more details. |
3733 | -# |
3734 | -# You should have received a copy of the GNU Lesser General Public License |
3735 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
3736 | - |
3737 | -from charmhelpers.core.hookenv import ( |
3738 | - log, |
3739 | - DEBUG, |
3740 | -) |
3741 | -from charmhelpers.contrib.hardening.mysql.checks import config |
3742 | - |
3743 | - |
3744 | -def run_mysql_checks(): |
3745 | - log("Starting MySQL hardening checks.", level=DEBUG) |
3746 | - checks = config.get_audits() |
3747 | - for check in checks: |
3748 | - log("Running '%s' check" % (check.__class__.__name__), level=DEBUG) |
3749 | - check.ensure_compliance() |
3750 | - |
3751 | - log("MySQL hardening checks complete.", level=DEBUG) |
3752 | |
3753 | === removed file 'hooks/charmhelpers/contrib/hardening/mysql/checks/config.py' |
3754 | --- hooks/charmhelpers/contrib/hardening/mysql/checks/config.py 2016-04-22 08:15:33 +0000 |
3755 | +++ hooks/charmhelpers/contrib/hardening/mysql/checks/config.py 1970-01-01 00:00:00 +0000 |
3756 | @@ -1,89 +0,0 @@ |
3757 | -# Copyright 2016 Canonical Limited. |
3758 | -# |
3759 | -# This file is part of charm-helpers. |
3760 | -# |
3761 | -# charm-helpers is free software: you can redistribute it and/or modify |
3762 | -# it under the terms of the GNU Lesser General Public License version 3 as |
3763 | -# published by the Free Software Foundation. |
3764 | -# |
3765 | -# charm-helpers is distributed in the hope that it will be useful, |
3766 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
3767 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
3768 | -# GNU Lesser General Public License for more details. |
3769 | -# |
3770 | -# You should have received a copy of the GNU Lesser General Public License |
3771 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
3772 | - |
3773 | -import six |
3774 | -import subprocess |
3775 | - |
3776 | -from charmhelpers.core.hookenv import ( |
3777 | - log, |
3778 | - WARNING, |
3779 | -) |
3780 | -from charmhelpers.contrib.hardening.audits.file import ( |
3781 | - FilePermissionAudit, |
3782 | - DirectoryPermissionAudit, |
3783 | - TemplatedFile, |
3784 | -) |
3785 | -from charmhelpers.contrib.hardening.mysql import TEMPLATES_DIR |
3786 | -from charmhelpers.contrib.hardening import utils |
3787 | - |
3788 | - |
3789 | -def get_audits(): |
3790 | - """Get MySQL hardening config audits. |
3791 | - |
3792 | - :returns: dictionary of audits |
3793 | - """ |
3794 | - if subprocess.call(['which', 'mysql'], stdout=subprocess.PIPE) != 0: |
3795 | - log("MySQL does not appear to be installed on this node - " |
3796 | - "skipping mysql hardening", level=WARNING) |
3797 | - return [] |
3798 | - |
3799 | - settings = utils.get_settings('mysql') |
3800 | - hardening_settings = settings['hardening'] |
3801 | - my_cnf = hardening_settings['mysql-conf'] |
3802 | - |
3803 | - audits = [ |
3804 | - FilePermissionAudit(paths=[my_cnf], user='root', |
3805 | - group='root', mode=0o0600), |
3806 | - |
3807 | - TemplatedFile(hardening_settings['hardening-conf'], |
3808 | - MySQLConfContext(), |
3809 | - TEMPLATES_DIR, |
3810 | - mode=0o0750, |
3811 | - user='mysql', |
3812 | - group='root', |
3813 | - service_actions=[{'service': 'mysql', |
3814 | - 'actions': ['restart']}]), |
3815 | - |
3816 | - # MySQL and Percona charms do not allow configuration of the |
3817 | - # data directory, so use the default. |
3818 | - DirectoryPermissionAudit('/var/lib/mysql', |
3819 | - user='mysql', |
3820 | - group='mysql', |
3821 | - recursive=False, |
3822 | - mode=0o755), |
3823 | - |
3824 | - DirectoryPermissionAudit('/etc/mysql', |
3825 | - user='root', |
3826 | - group='root', |
3827 | - recursive=False, |
3828 | - mode=0o700), |
3829 | - ] |
3830 | - |
3831 | - return audits |
3832 | - |
3833 | - |
3834 | -class MySQLConfContext(object): |
3835 | - """Defines the set of key/value pairs to set in a mysql config file. |
3836 | - |
3837 | - This context, when called, will return a dictionary containing the |
3838 | - key/value pairs of setting to specify in the |
3839 | - /etc/mysql/conf.d/hardening.cnf file. |
3840 | - """ |
3841 | - def __call__(self): |
3842 | - settings = utils.get_settings('mysql') |
3843 | - # Translate for python3 |
3844 | - return {'mysql_settings': |
3845 | - [(k, v) for k, v in six.iteritems(settings['security'])]} |
3846 | |
3847 | === removed directory 'hooks/charmhelpers/contrib/hardening/ssh' |
3848 | === removed file 'hooks/charmhelpers/contrib/hardening/ssh/__init__.py' |
3849 | --- hooks/charmhelpers/contrib/hardening/ssh/__init__.py 2016-04-22 08:15:33 +0000 |
3850 | +++ hooks/charmhelpers/contrib/hardening/ssh/__init__.py 1970-01-01 00:00:00 +0000 |
3851 | @@ -1,19 +0,0 @@ |
3852 | -# Copyright 2016 Canonical Limited. |
3853 | -# |
3854 | -# This file is part of charm-helpers. |
3855 | -# |
3856 | -# charm-helpers is free software: you can redistribute it and/or modify |
3857 | -# it under the terms of the GNU Lesser General Public License version 3 as |
3858 | -# published by the Free Software Foundation. |
3859 | -# |
3860 | -# charm-helpers is distributed in the hope that it will be useful, |
3861 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
3862 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
3863 | -# GNU Lesser General Public License for more details. |
3864 | -# |
3865 | -# You should have received a copy of the GNU Lesser General Public License |
3866 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
3867 | - |
3868 | -from os import path |
3869 | - |
3870 | -TEMPLATES_DIR = path.join(path.dirname(__file__), 'templates') |
3871 | |
3872 | === removed directory 'hooks/charmhelpers/contrib/hardening/ssh/checks' |
3873 | === removed file 'hooks/charmhelpers/contrib/hardening/ssh/checks/__init__.py' |
3874 | --- hooks/charmhelpers/contrib/hardening/ssh/checks/__init__.py 2016-04-22 08:15:33 +0000 |
3875 | +++ hooks/charmhelpers/contrib/hardening/ssh/checks/__init__.py 1970-01-01 00:00:00 +0000 |
3876 | @@ -1,31 +0,0 @@ |
3877 | -# Copyright 2016 Canonical Limited. |
3878 | -# |
3879 | -# This file is part of charm-helpers. |
3880 | -# |
3881 | -# charm-helpers is free software: you can redistribute it and/or modify |
3882 | -# it under the terms of the GNU Lesser General Public License version 3 as |
3883 | -# published by the Free Software Foundation. |
3884 | -# |
3885 | -# charm-helpers is distributed in the hope that it will be useful, |
3886 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
3887 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
3888 | -# GNU Lesser General Public License for more details. |
3889 | -# |
3890 | -# You should have received a copy of the GNU Lesser General Public License |
3891 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
3892 | - |
3893 | -from charmhelpers.core.hookenv import ( |
3894 | - log, |
3895 | - DEBUG, |
3896 | -) |
3897 | -from charmhelpers.contrib.hardening.ssh.checks import config |
3898 | - |
3899 | - |
3900 | -def run_ssh_checks(): |
3901 | - log("Starting SSH hardening checks.", level=DEBUG) |
3902 | - checks = config.get_audits() |
3903 | - for check in checks: |
3904 | - log("Running '%s' check" % (check.__class__.__name__), level=DEBUG) |
3905 | - check.ensure_compliance() |
3906 | - |
3907 | - log("SSH hardening checks complete.", level=DEBUG) |
3908 | |
3909 | === removed file 'hooks/charmhelpers/contrib/hardening/ssh/checks/config.py' |
3910 | --- hooks/charmhelpers/contrib/hardening/ssh/checks/config.py 2016-04-22 08:15:33 +0000 |
3911 | +++ hooks/charmhelpers/contrib/hardening/ssh/checks/config.py 1970-01-01 00:00:00 +0000 |
3912 | @@ -1,394 +0,0 @@ |
3913 | -# Copyright 2016 Canonical Limited. |
3914 | -# |
3915 | -# This file is part of charm-helpers. |
3916 | -# |
3917 | -# charm-helpers is free software: you can redistribute it and/or modify |
3918 | -# it under the terms of the GNU Lesser General Public License version 3 as |
3919 | -# published by the Free Software Foundation. |
3920 | -# |
3921 | -# charm-helpers is distributed in the hope that it will be useful, |
3922 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
3923 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
3924 | -# GNU Lesser General Public License for more details. |
3925 | -# |
3926 | -# You should have received a copy of the GNU Lesser General Public License |
3927 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
3928 | - |
3929 | -import os |
3930 | - |
3931 | -from charmhelpers.core.hookenv import ( |
3932 | - log, |
3933 | - DEBUG, |
3934 | -) |
3935 | -from charmhelpers.fetch import ( |
3936 | - apt_install, |
3937 | - apt_update, |
3938 | -) |
3939 | -from charmhelpers.core.host import lsb_release |
3940 | -from charmhelpers.contrib.hardening.audits.file import ( |
3941 | - TemplatedFile, |
3942 | - FileContentAudit, |
3943 | -) |
3944 | -from charmhelpers.contrib.hardening.ssh import TEMPLATES_DIR |
3945 | -from charmhelpers.contrib.hardening import utils |
3946 | - |
3947 | - |
3948 | -def get_audits(): |
3949 | - """Get SSH hardening config audits. |
3950 | - |
3951 | - :returns: dictionary of audits |
3952 | - """ |
3953 | - audits = [SSHConfig(), SSHDConfig(), SSHConfigFileContentAudit(), |
3954 | - SSHDConfigFileContentAudit()] |
3955 | - return audits |
3956 | - |
3957 | - |
3958 | -class SSHConfigContext(object): |
3959 | - |
3960 | - type = 'client' |
3961 | - |
3962 | - def get_macs(self, allow_weak_mac): |
3963 | - if allow_weak_mac: |
3964 | - weak_macs = 'weak' |
3965 | - else: |
3966 | - weak_macs = 'default' |
3967 | - |
3968 | - default = 'hmac-sha2-512,hmac-sha2-256,hmac-ripemd160' |
3969 | - macs = {'default': default, |
3970 | - 'weak': default + ',hmac-sha1'} |
3971 | - |
3972 | - default = ('hmac-sha2-512-etm@openssh.com,' |
3973 | - 'hmac-sha2-256-etm@openssh.com,' |
3974 | - 'hmac-ripemd160-etm@openssh.com,umac-128-etm@openssh.com,' |
3975 | - 'hmac-sha2-512,hmac-sha2-256,hmac-ripemd160') |
3976 | - macs_66 = {'default': default, |
3977 | - 'weak': default + ',hmac-sha1'} |
3978 | - |
3979 | - # Use newer ciphers on Ubuntu Trusty and above |
3980 | - if lsb_release()['DISTRIB_CODENAME'].lower() >= 'trusty': |
3981 | - log("Detected Ubuntu 14.04 or newer, using new macs", level=DEBUG) |
3982 | - macs = macs_66 |
3983 | - |
3984 | - return macs[weak_macs] |
3985 | - |
3986 | - def get_kexs(self, allow_weak_kex): |
3987 | - if allow_weak_kex: |
3988 | - weak_kex = 'weak' |
3989 | - else: |
3990 | - weak_kex = 'default' |
3991 | - |
3992 | - default = 'diffie-hellman-group-exchange-sha256' |
3993 | - weak = (default + ',diffie-hellman-group14-sha1,' |
3994 | - 'diffie-hellman-group-exchange-sha1,' |
3995 | - 'diffie-hellman-group1-sha1') |
3996 | - kex = {'default': default, |
3997 | - 'weak': weak} |
3998 | - |
3999 | - default = ('curve25519-sha256@libssh.org,' |
4000 | - 'diffie-hellman-group-exchange-sha256') |
4001 | - weak = (default + ',diffie-hellman-group14-sha1,' |
4002 | - 'diffie-hellman-group-exchange-sha1,' |
4003 | - 'diffie-hellman-group1-sha1') |
4004 | - kex_66 = {'default': default, |
4005 | - 'weak': weak} |
4006 | - |
4007 | - # Use newer kex on Ubuntu Trusty and above |
4008 | - if lsb_release()['DISTRIB_CODENAME'].lower() >= 'trusty': |
4009 | - log('Detected Ubuntu 14.04 or newer, using new key exchange ' |
4010 | - 'algorithms', level=DEBUG) |
4011 | - kex = kex_66 |
4012 | - |
4013 | - return kex[weak_kex] |
4014 | - |
4015 | - def get_ciphers(self, cbc_required): |
4016 | - if cbc_required: |
4017 | - weak_ciphers = 'weak' |
4018 | - else: |
4019 | - weak_ciphers = 'default' |
4020 | - |
4021 | - default = 'aes256-ctr,aes192-ctr,aes128-ctr' |
4022 | - cipher = {'default': default, |
4023 | - 'weak': default + 'aes256-cbc,aes192-cbc,aes128-cbc'} |
4024 | - |
4025 | - default = ('chacha20-poly1305@openssh.com,aes256-gcm@openssh.com,' |
4026 | - 'aes128-gcm@openssh.com,aes256-ctr,aes192-ctr,aes128-ctr') |
4027 | - ciphers_66 = {'default': default, |
4028 | - 'weak': default + ',aes256-cbc,aes192-cbc,aes128-cbc'} |
4029 | - |
4030 | - # Use newer ciphers on ubuntu Trusty and above |
4031 | - if lsb_release()['DISTRIB_CODENAME'].lower() >= 'trusty': |
4032 | - log('Detected Ubuntu 14.04 or newer, using new ciphers', |
4033 | - level=DEBUG) |
4034 | - cipher = ciphers_66 |
4035 | - |
4036 | - return cipher[weak_ciphers] |
4037 | - |
4038 | - def __call__(self): |
4039 | - settings = utils.get_settings('ssh') |
4040 | - if settings['common']['network_ipv6_enable']: |
4041 | - addr_family = 'any' |
4042 | - else: |
4043 | - addr_family = 'inet' |
4044 | - |
4045 | - ctxt = { |
4046 | - 'addr_family': addr_family, |
4047 | - 'remote_hosts': settings['common']['remote_hosts'], |
4048 | - 'password_auth_allowed': |
4049 | - settings['client']['password_authentication'], |
4050 | - 'ports': settings['common']['ports'], |
4051 | - 'ciphers': self.get_ciphers(settings['client']['cbc_required']), |
4052 | - 'macs': self.get_macs(settings['client']['weak_hmac']), |
4053 | - 'kexs': self.get_kexs(settings['client']['weak_kex']), |
4054 | - 'roaming': settings['client']['roaming'], |
4055 | - } |
4056 | - return ctxt |
4057 | - |
4058 | - |
4059 | -class SSHConfig(TemplatedFile): |
4060 | - def __init__(self): |
4061 | - path = '/etc/ssh/ssh_config' |
4062 | - super(SSHConfig, self).__init__(path=path, |
4063 | - template_dir=TEMPLATES_DIR, |
4064 | - context=SSHConfigContext(), |
4065 | - user='root', |
4066 | - group='root', |
4067 | - mode=0o0644) |
4068 | - |
4069 | - def pre_write(self): |
4070 | - settings = utils.get_settings('ssh') |
4071 | - apt_update(fatal=True) |
4072 | - apt_install(settings['client']['package']) |
4073 | - if not os.path.exists('/etc/ssh'): |
4074 | - os.makedir('/etc/ssh') |
4075 | - # NOTE: don't recurse |
4076 | - utils.ensure_permissions('/etc/ssh', 'root', 'root', 0o0755, |
4077 | - maxdepth=0) |
4078 | - |
4079 | - def post_write(self): |
4080 | - # NOTE: don't recurse |
4081 | - utils.ensure_permissions('/etc/ssh', 'root', 'root', 0o0755, |
4082 | - maxdepth=0) |
4083 | - |
4084 | - |
4085 | -class SSHDConfigContext(SSHConfigContext): |
4086 | - |
4087 | - type = 'server' |
4088 | - |
4089 | - def __call__(self): |
4090 | - settings = utils.get_settings('ssh') |
4091 | - if settings['common']['network_ipv6_enable']: |
4092 | - addr_family = 'any' |
4093 | - else: |
4094 | - addr_family = 'inet' |
4095 | - |
4096 | - ctxt = { |
4097 | - 'ssh_ip': settings['server']['listen_to'], |
4098 | - 'password_auth_allowed': |
4099 | - settings['server']['password_authentication'], |
4100 | - 'ports': settings['common']['ports'], |
4101 | - 'addr_family': addr_family, |
4102 | - 'ciphers': self.get_ciphers(settings['server']['cbc_required']), |
4103 | - 'macs': self.get_macs(settings['server']['weak_hmac']), |
4104 | - 'kexs': self.get_kexs(settings['server']['weak_kex']), |
4105 | - 'host_key_files': settings['server']['host_key_files'], |
4106 | - 'allow_root_with_key': settings['server']['allow_root_with_key'], |
4107 | - 'password_authentication': |
4108 | - settings['server']['password_authentication'], |
4109 | - 'use_priv_sep': settings['server']['use_privilege_separation'], |
4110 | - 'use_pam': settings['server']['use_pam'], |
4111 | - 'allow_x11_forwarding': settings['server']['allow_x11_forwarding'], |
4112 | - 'print_motd': settings['server']['print_motd'], |
4113 | - 'print_last_log': settings['server']['print_last_log'], |
4114 | - 'client_alive_interval': |
4115 | - settings['server']['alive_interval'], |
4116 | - 'client_alive_count': settings['server']['alive_count'], |
4117 | - 'allow_tcp_forwarding': settings['server']['allow_tcp_forwarding'], |
4118 | - 'allow_agent_forwarding': |
4119 | - settings['server']['allow_agent_forwarding'], |
4120 | - 'deny_users': settings['server']['deny_users'], |
4121 | - 'allow_users': settings['server']['allow_users'], |
4122 | - 'deny_groups': settings['server']['deny_groups'], |
4123 | - 'allow_groups': settings['server']['allow_groups'], |
4124 | - 'use_dns': settings['server']['use_dns'], |
4125 | - 'sftp_enable': settings['server']['sftp_enable'], |
4126 | - 'sftp_group': settings['server']['sftp_group'], |
4127 | - 'sftp_chroot': settings['server']['sftp_chroot'], |
4128 | - 'max_auth_tries': settings['server']['max_auth_tries'], |
4129 | - 'max_sessions': settings['server']['max_sessions'], |
4130 | - } |
4131 | - return ctxt |
4132 | - |
4133 | - |
4134 | -class SSHDConfig(TemplatedFile): |
4135 | - def __init__(self): |
4136 | - path = '/etc/ssh/sshd_config' |
4137 | - super(SSHDConfig, self).__init__(path=path, |
4138 | - template_dir=TEMPLATES_DIR, |
4139 | - context=SSHDConfigContext(), |
4140 | - user='root', |
4141 | - group='root', |
4142 | - mode=0o0600, |
4143 | - service_actions=[{'service': 'ssh', |
4144 | - 'actions': |
4145 | - ['restart']}]) |
4146 | - |
4147 | - def pre_write(self): |
4148 | - settings = utils.get_settings('ssh') |
4149 | - apt_update(fatal=True) |
4150 | - apt_install(settings['server']['package']) |
4151 | - if not os.path.exists('/etc/ssh'): |
4152 | - os.makedir('/etc/ssh') |
4153 | - # NOTE: don't recurse |
4154 | - utils.ensure_permissions('/etc/ssh', 'root', 'root', 0o0755, |
4155 | - maxdepth=0) |
4156 | - |
4157 | - def post_write(self): |
4158 | - # NOTE: don't recurse |
4159 | - utils.ensure_permissions('/etc/ssh', 'root', 'root', 0o0755, |
4160 | - maxdepth=0) |
4161 | - |
4162 | - |
4163 | -class SSHConfigFileContentAudit(FileContentAudit): |
4164 | - def __init__(self): |
4165 | - self.path = '/etc/ssh/ssh_config' |
4166 | - super(SSHConfigFileContentAudit, self).__init__(self.path, {}) |
4167 | - |
4168 | - def is_compliant(self, *args, **kwargs): |
4169 | - self.pass_cases = [] |
4170 | - self.fail_cases = [] |
4171 | - settings = utils.get_settings('ssh') |
4172 | - |
4173 | - if lsb_release()['DISTRIB_CODENAME'].lower() >= 'trusty': |
4174 | - if not settings['server']['weak_hmac']: |
4175 | - self.pass_cases.append(r'^MACs.+,hmac-ripemd160$') |
4176 | - else: |
4177 | - self.pass_cases.append(r'^MACs.+,hmac-sha1$') |
4178 | - |
4179 | - if settings['server']['weak_kex']: |
4180 | - self.fail_cases.append(r'^KexAlgorithms\sdiffie-hellman-group-exchange-sha256[,\s]?') # noqa |
4181 | - self.pass_cases.append(r'^KexAlgorithms\sdiffie-hellman-group14-sha1[,\s]?') # noqa |
4182 | - self.pass_cases.append(r'^KexAlgorithms\sdiffie-hellman-group-exchange-sha1[,\s]?') # noqa |
4183 | - self.pass_cases.append(r'^KexAlgorithms\sdiffie-hellman-group1-sha1[,\s]?') # noqa |
4184 | - else: |
4185 | - self.pass_cases.append(r'^KexAlgorithms.+,diffie-hellman-group-exchange-sha256$') # noqa |
4186 | - self.fail_cases.append(r'^KexAlgorithms.*diffie-hellman-group14-sha1[,\s]?') # noqa |
4187 | - |
4188 | - if settings['server']['cbc_required']: |
4189 | - self.pass_cases.append(r'^Ciphers\s.*-cbc[,\s]?') |
4190 | - self.fail_cases.append(r'^Ciphers\s.*aes128-ctr[,\s]?') |
4191 | - self.fail_cases.append(r'^Ciphers\s.*aes192-ctr[,\s]?') |
4192 | - self.fail_cases.append(r'^Ciphers\s.*aes256-ctr[,\s]?') |
4193 | - else: |
4194 | - self.fail_cases.append(r'^Ciphers\s.*-cbc[,\s]?') |
4195 | - self.pass_cases.append(r'^Ciphers\schacha20-poly1305@openssh.com,.+') # noqa |
4196 | - self.pass_cases.append(r'^Ciphers\s.*aes128-ctr$') |
4197 | - self.pass_cases.append(r'^Ciphers\s.*aes192-ctr[,\s]?') |
4198 | - self.pass_cases.append(r'^Ciphers\s.*aes256-ctr[,\s]?') |
4199 | - else: |
4200 | - if not settings['client']['weak_hmac']: |
4201 | - self.fail_cases.append(r'^MACs.+,hmac-sha1$') |
4202 | - else: |
4203 | - self.pass_cases.append(r'^MACs.+,hmac-sha1$') |
4204 | - |
4205 | - if settings['client']['weak_kex']: |
4206 | - self.fail_cases.append(r'^KexAlgorithms\sdiffie-hellman-group-exchange-sha256[,\s]?') # noqa |
4207 | - self.pass_cases.append(r'^KexAlgorithms\sdiffie-hellman-group14-sha1[,\s]?') # noqa |
4208 | - self.pass_cases.append(r'^KexAlgorithms\sdiffie-hellman-group-exchange-sha1[,\s]?') # noqa |
4209 | - self.pass_cases.append(r'^KexAlgorithms\sdiffie-hellman-group1-sha1[,\s]?') # noqa |
4210 | - else: |
4211 | - self.pass_cases.append(r'^KexAlgorithms\sdiffie-hellman-group-exchange-sha256$') # noqa |
4212 | - self.fail_cases.append(r'^KexAlgorithms\sdiffie-hellman-group14-sha1[,\s]?') # noqa |
4213 | - self.fail_cases.append(r'^KexAlgorithms\sdiffie-hellman-group-exchange-sha1[,\s]?') # noqa |
4214 | - self.fail_cases.append(r'^KexAlgorithms\sdiffie-hellman-group1-sha1[,\s]?') # noqa |
4215 | - |
4216 | - if settings['client']['cbc_required']: |
4217 | - self.pass_cases.append(r'^Ciphers\s.*-cbc[,\s]?') |
4218 | - self.fail_cases.append(r'^Ciphers\s.*aes128-ctr[,\s]?') |
4219 | - self.fail_cases.append(r'^Ciphers\s.*aes192-ctr[,\s]?') |
4220 | - self.fail_cases.append(r'^Ciphers\s.*aes256-ctr[,\s]?') |
4221 | - else: |
4222 | - self.fail_cases.append(r'^Ciphers\s.*-cbc[,\s]?') |
4223 | - self.pass_cases.append(r'^Ciphers\s.*aes128-ctr[,\s]?') |
4224 | - self.pass_cases.append(r'^Ciphers\s.*aes192-ctr[,\s]?') |
4225 | - self.pass_cases.append(r'^Ciphers\s.*aes256-ctr[,\s]?') |
4226 | - |
4227 | - if settings['client']['roaming']: |
4228 | - self.pass_cases.append(r'^UseRoaming yes$') |
4229 | - else: |
4230 | - self.fail_cases.append(r'^UseRoaming yes$') |
4231 | - |
4232 | - return super(SSHConfigFileContentAudit, self).is_compliant(*args, |
4233 | - **kwargs) |
4234 | - |
4235 | - |
4236 | -class SSHDConfigFileContentAudit(FileContentAudit): |
4237 | - def __init__(self): |
4238 | - self.path = '/etc/ssh/sshd_config' |
4239 | - super(SSHDConfigFileContentAudit, self).__init__(self.path, {}) |
4240 | - |
4241 | - def is_compliant(self, *args, **kwargs): |
4242 | - self.pass_cases = [] |
4243 | - self.fail_cases = [] |
4244 | - settings = utils.get_settings('ssh') |
4245 | - |
4246 | - if lsb_release()['DISTRIB_CODENAME'].lower() >= 'trusty': |
4247 | - if not settings['server']['weak_hmac']: |
4248 | - self.pass_cases.append(r'^MACs.+,hmac-ripemd160$') |
4249 | - else: |
4250 | - self.pass_cases.append(r'^MACs.+,hmac-sha1$') |
4251 | - |
4252 | - if settings['server']['weak_kex']: |
4253 | - self.fail_cases.append(r'^KexAlgorithms\sdiffie-hellman-group-exchange-sha256[,\s]?') # noqa |
4254 | - self.pass_cases.append(r'^KexAlgorithms\sdiffie-hellman-group14-sha1[,\s]?') # noqa |
4255 | - self.pass_cases.append(r'^KexAlgorithms\sdiffie-hellman-group-exchange-sha1[,\s]?') # noqa |
4256 | - self.pass_cases.append(r'^KexAlgorithms\sdiffie-hellman-group1-sha1[,\s]?') # noqa |
4257 | - else: |
4258 | - self.pass_cases.append(r'^KexAlgorithms.+,diffie-hellman-group-exchange-sha256$') # noqa |
4259 | - self.fail_cases.append(r'^KexAlgorithms.*diffie-hellman-group14-sha1[,\s]?') # noqa |
4260 | - |
4261 | - if settings['server']['cbc_required']: |
4262 | - self.pass_cases.append(r'^Ciphers\s.*-cbc[,\s]?') |
4263 | - self.fail_cases.append(r'^Ciphers\s.*aes128-ctr[,\s]?') |
4264 | - self.fail_cases.append(r'^Ciphers\s.*aes192-ctr[,\s]?') |
4265 | - self.fail_cases.append(r'^Ciphers\s.*aes256-ctr[,\s]?') |
4266 | - else: |
4267 | - self.fail_cases.append(r'^Ciphers\s.*-cbc[,\s]?') |
4268 | - self.pass_cases.append(r'^Ciphers\schacha20-poly1305@openssh.com,.+') # noqa |
4269 | - self.pass_cases.append(r'^Ciphers\s.*aes128-ctr$') |
4270 | - self.pass_cases.append(r'^Ciphers\s.*aes192-ctr[,\s]?') |
4271 | - self.pass_cases.append(r'^Ciphers\s.*aes256-ctr[,\s]?') |
4272 | - else: |
4273 | - if not settings['server']['weak_hmac']: |
4274 | - self.pass_cases.append(r'^MACs.+,hmac-ripemd160$') |
4275 | - else: |
4276 | - self.pass_cases.append(r'^MACs.+,hmac-sha1$') |
4277 | - |
4278 | - if settings['server']['weak_kex']: |
4279 | - self.fail_cases.append(r'^KexAlgorithms\sdiffie-hellman-group-exchange-sha256[,\s]?') # noqa |
4280 | - self.pass_cases.append(r'^KexAlgorithms\sdiffie-hellman-group14-sha1[,\s]?') # noqa |
4281 | - self.pass_cases.append(r'^KexAlgorithms\sdiffie-hellman-group-exchange-sha1[,\s]?') # noqa |
4282 | - self.pass_cases.append(r'^KexAlgorithms\sdiffie-hellman-group1-sha1[,\s]?') # noqa |
4283 | - else: |
4284 | - self.pass_cases.append(r'^KexAlgorithms\sdiffie-hellman-group-exchange-sha256$') # noqa |
4285 | - self.fail_cases.append(r'^KexAlgorithms\sdiffie-hellman-group14-sha1[,\s]?') # noqa |
4286 | - self.fail_cases.append(r'^KexAlgorithms\sdiffie-hellman-group-exchange-sha1[,\s]?') # noqa |
4287 | - self.fail_cases.append(r'^KexAlgorithms\sdiffie-hellman-group1-sha1[,\s]?') # noqa |
4288 | - |
4289 | - if settings['server']['cbc_required']: |
4290 | - self.pass_cases.append(r'^Ciphers\s.*-cbc[,\s]?') |
4291 | - self.fail_cases.append(r'^Ciphers\s.*aes128-ctr[,\s]?') |
4292 | - self.fail_cases.append(r'^Ciphers\s.*aes192-ctr[,\s]?') |
4293 | - self.fail_cases.append(r'^Ciphers\s.*aes256-ctr[,\s]?') |
4294 | - else: |
4295 | - self.fail_cases.append(r'^Ciphers\s.*-cbc[,\s]?') |
4296 | - self.pass_cases.append(r'^Ciphers\s.*aes128-ctr[,\s]?') |
4297 | - self.pass_cases.append(r'^Ciphers\s.*aes192-ctr[,\s]?') |
4298 | - self.pass_cases.append(r'^Ciphers\s.*aes256-ctr[,\s]?') |
4299 | - |
4300 | - if settings['server']['sftp_enable']: |
4301 | - self.pass_cases.append(r'^Subsystem\ssftp') |
4302 | - else: |
4303 | - self.fail_cases.append(r'^Subsystem\ssftp') |
4304 | - |
4305 | - return super(SSHDConfigFileContentAudit, self).is_compliant(*args, |
4306 | - **kwargs) |
4307 | |
4308 | === removed file 'hooks/charmhelpers/contrib/hardening/templating.py' |
4309 | --- hooks/charmhelpers/contrib/hardening/templating.py 2016-04-22 08:15:33 +0000 |
4310 | +++ hooks/charmhelpers/contrib/hardening/templating.py 1970-01-01 00:00:00 +0000 |
4311 | @@ -1,71 +0,0 @@ |
4312 | -# Copyright 2016 Canonical Limited. |
4313 | -# |
4314 | -# This file is part of charm-helpers. |
4315 | -# |
4316 | -# charm-helpers is free software: you can redistribute it and/or modify |
4317 | -# it under the terms of the GNU Lesser General Public License version 3 as |
4318 | -# published by the Free Software Foundation. |
4319 | -# |
4320 | -# charm-helpers is distributed in the hope that it will be useful, |
4321 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
4322 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
4323 | -# GNU Lesser General Public License for more details. |
4324 | -# |
4325 | -# You should have received a copy of the GNU Lesser General Public License |
4326 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
4327 | - |
4328 | -import os |
4329 | - |
4330 | -from charmhelpers.core.hookenv import ( |
4331 | - log, |
4332 | - DEBUG, |
4333 | - WARNING, |
4334 | -) |
4335 | - |
4336 | -try: |
4337 | - from jinja2 import FileSystemLoader, Environment |
4338 | -except ImportError: |
4339 | - from charmhelpers.fetch import apt_install |
4340 | - from charmhelpers.fetch import apt_update |
4341 | - apt_update(fatal=True) |
4342 | - apt_install('python-jinja2', fatal=True) |
4343 | - from jinja2 import FileSystemLoader, Environment |
4344 | - |
4345 | - |
4346 | -# NOTE: function separated from main rendering code to facilitate easier |
4347 | -# mocking in unit tests. |
4348 | -def write(path, data): |
4349 | - with open(path, 'wb') as out: |
4350 | - out.write(data) |
4351 | - |
4352 | - |
4353 | -def get_template_path(template_dir, path): |
4354 | - """Returns the template file which would be used to render the path. |
4355 | - |
4356 | - The path to the template file is returned. |
4357 | - :param template_dir: the directory the templates are located in |
4358 | - :param path: the file path to be written to. |
4359 | - :returns: path to the template file |
4360 | - """ |
4361 | - return os.path.join(template_dir, os.path.basename(path)) |
4362 | - |
4363 | - |
4364 | -def render_and_write(template_dir, path, context): |
4365 | - """Renders the specified template into the file. |
4366 | - |
4367 | - :param template_dir: the directory to load the template from |
4368 | - :param path: the path to write the templated contents to |
4369 | - :param context: the parameters to pass to the rendering engine |
4370 | - """ |
4371 | - env = Environment(loader=FileSystemLoader(template_dir)) |
4372 | - template_file = os.path.basename(path) |
4373 | - template = env.get_template(template_file) |
4374 | - log('Rendering from template: %s' % template.name, level=DEBUG) |
4375 | - rendered_content = template.render(context) |
4376 | - if not rendered_content: |
4377 | - log("Render returned None - skipping '%s'" % path, |
4378 | - level=WARNING) |
4379 | - return |
4380 | - |
4381 | - write(path, rendered_content.encode('utf-8').strip()) |
4382 | - log('Wrote template %s' % path, level=DEBUG) |
4383 | |
4384 | === removed file 'hooks/charmhelpers/contrib/hardening/utils.py' |
4385 | --- hooks/charmhelpers/contrib/hardening/utils.py 2016-04-22 08:15:33 +0000 |
4386 | +++ hooks/charmhelpers/contrib/hardening/utils.py 1970-01-01 00:00:00 +0000 |
4387 | @@ -1,157 +0,0 @@ |
4388 | -# Copyright 2016 Canonical Limited. |
4389 | -# |
4390 | -# This file is part of charm-helpers. |
4391 | -# |
4392 | -# charm-helpers is free software: you can redistribute it and/or modify |
4393 | -# it under the terms of the GNU Lesser General Public License version 3 as |
4394 | -# published by the Free Software Foundation. |
4395 | -# |
4396 | -# charm-helpers is distributed in the hope that it will be useful, |
4397 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
4398 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
4399 | -# GNU Lesser General Public License for more details. |
4400 | -# |
4401 | -# You should have received a copy of the GNU Lesser General Public License |
4402 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
4403 | - |
4404 | -import glob |
4405 | -import grp |
4406 | -import os |
4407 | -import pwd |
4408 | -import six |
4409 | -import yaml |
4410 | - |
4411 | -from charmhelpers.core.hookenv import ( |
4412 | - log, |
4413 | - DEBUG, |
4414 | - INFO, |
4415 | - WARNING, |
4416 | - ERROR, |
4417 | -) |
4418 | - |
4419 | - |
4420 | -# Global settings cache. Since each hook fire entails a fresh module import it |
4421 | -# is safe to hold this in memory and not risk missing config changes (since |
4422 | -# they will result in a new hook fire and thus re-import). |
4423 | -__SETTINGS__ = {} |
4424 | - |
4425 | - |
4426 | -def _get_defaults(modules): |
4427 | - """Load the default config for the provided modules. |
4428 | - |
4429 | - :param modules: stack modules config defaults to lookup. |
4430 | - :returns: modules default config dictionary. |
4431 | - """ |
4432 | - default = os.path.join(os.path.dirname(__file__), |
4433 | - 'defaults/%s.yaml' % (modules)) |
4434 | - return yaml.safe_load(open(default)) |
4435 | - |
4436 | - |
4437 | -def _get_schema(modules): |
4438 | - """Load the config schema for the provided modules. |
4439 | - |
4440 | - NOTE: this schema is intended to have 1-1 relationship with they keys in |
4441 | - the default config and is used a means to verify valid overrides provided |
4442 | - by the user. |
4443 | - |
4444 | - :param modules: stack modules config schema to lookup. |
4445 | - :returns: modules default schema dictionary. |
4446 | - """ |
4447 | - schema = os.path.join(os.path.dirname(__file__), |
4448 | - 'defaults/%s.yaml.schema' % (modules)) |
4449 | - return yaml.safe_load(open(schema)) |
4450 | - |
4451 | - |
4452 | -def _get_user_provided_overrides(modules): |
4453 | - """Load user-provided config overrides. |
4454 | - |
4455 | - :param modules: stack modules to lookup in user overrides yaml file. |
4456 | - :returns: overrides dictionary. |
4457 | - """ |
4458 | - overrides = os.path.join(os.environ['JUJU_CHARM_DIR'], |
4459 | - 'hardening.yaml') |
4460 | - if os.path.exists(overrides): |
4461 | - log("Found user-provided config overrides file '%s'" % |
4462 | - (overrides), level=DEBUG) |
4463 | - settings = yaml.safe_load(open(overrides)) |
4464 | - if settings and settings.get(modules): |
4465 | - log("Applying '%s' overrides" % (modules), level=DEBUG) |
4466 | - return settings.get(modules) |
4467 | - |
4468 | - log("No overrides found for '%s'" % (modules), level=DEBUG) |
4469 | - else: |
4470 | - log("No hardening config overrides file '%s' found in charm " |
4471 | - "root dir" % (overrides), level=DEBUG) |
4472 | - |
4473 | - return {} |
4474 | - |
4475 | - |
4476 | -def _apply_overrides(settings, overrides, schema): |
4477 | - """Get overrides config overlayed onto modules defaults. |
4478 | - |
4479 | - :param modules: require stack modules config. |
4480 | - :returns: dictionary of modules config with user overrides applied. |
4481 | - """ |
4482 | - if overrides: |
4483 | - for k, v in six.iteritems(overrides): |
4484 | - if k in schema: |
4485 | - if schema[k] is None: |
4486 | - settings[k] = v |
4487 | - elif type(schema[k]) is dict: |
4488 | - settings[k] = _apply_overrides(settings[k], overrides[k], |
4489 | - schema[k]) |
4490 | - else: |
4491 | - raise Exception("Unexpected type found in schema '%s'" % |
4492 | - type(schema[k]), level=ERROR) |
4493 | - else: |
4494 | - log("Unknown override key '%s' - ignoring" % (k), level=INFO) |
4495 | - |
4496 | - return settings |
4497 | - |
4498 | - |
4499 | -def get_settings(modules): |
4500 | - global __SETTINGS__ |
4501 | - if modules in __SETTINGS__: |
4502 | - return __SETTINGS__[modules] |
4503 | - |
4504 | - schema = _get_schema(modules) |
4505 | - settings = _get_defaults(modules) |
4506 | - overrides = _get_user_provided_overrides(modules) |
4507 | - __SETTINGS__[modules] = _apply_overrides(settings, overrides, schema) |
4508 | - return __SETTINGS__[modules] |
4509 | - |
4510 | - |
4511 | -def ensure_permissions(path, user, group, permissions, maxdepth=-1): |
4512 | - """Ensure permissions for path. |
4513 | - |
4514 | - If path is a file, apply to file and return. If path is a directory, |
4515 | - apply recursively (if required) to directory contents and return. |
4516 | - |
4517 | - :param user: user name |
4518 | - :param group: group name |
4519 | - :param permissions: octal permissions |
4520 | - :param maxdepth: maximum recursion depth. A negative maxdepth allows |
4521 | - infinite recursion and maxdepth=0 means no recursion. |
4522 | - :returns: None |
4523 | - """ |
4524 | - if not os.path.exists(path): |
4525 | - log("File '%s' does not exist - cannot set permissions" % (path), |
4526 | - level=WARNING) |
4527 | - return |
4528 | - |
4529 | - _user = pwd.getpwnam(user) |
4530 | - os.chown(path, _user.pw_uid, grp.getgrnam(group).gr_gid) |
4531 | - os.chmod(path, permissions) |
4532 | - |
4533 | - if maxdepth == 0: |
4534 | - log("Max recursion depth reached - skipping further recursion", |
4535 | - level=DEBUG) |
4536 | - return |
4537 | - elif maxdepth > 0: |
4538 | - maxdepth -= 1 |
4539 | - |
4540 | - if os.path.isdir(path): |
4541 | - contents = glob.glob("%s/*" % (path)) |
4542 | - for c in contents: |
4543 | - ensure_permissions(c, user=user, group=group, |
4544 | - permissions=permissions, maxdepth=maxdepth) |
4545 | |
4546 | === removed directory 'hooks/charmhelpers/contrib/mellanox' |
4547 | === removed file 'hooks/charmhelpers/contrib/mellanox/__init__.py' |
4548 | === removed file 'hooks/charmhelpers/contrib/mellanox/infiniband.py' |
4549 | --- hooks/charmhelpers/contrib/mellanox/infiniband.py 2016-01-30 22:44:06 +0000 |
4550 | +++ hooks/charmhelpers/contrib/mellanox/infiniband.py 1970-01-01 00:00:00 +0000 |
4551 | @@ -1,151 +0,0 @@ |
4552 | -#!/usr/bin/env python |
4553 | -# -*- coding: utf-8 -*- |
4554 | - |
4555 | -# Copyright 2014-2015 Canonical Limited. |
4556 | -# |
4557 | -# This file is part of charm-helpers. |
4558 | -# |
4559 | -# charm-helpers is free software: you can redistribute it and/or modify |
4560 | -# it under the terms of the GNU Lesser General Public License version 3 as |
4561 | -# published by the Free Software Foundation. |
4562 | -# |
4563 | -# charm-helpers is distributed in the hope that it will be useful, |
4564 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
4565 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
4566 | -# GNU Lesser General Public License for more details. |
4567 | -# |
4568 | -# You should have received a copy of the GNU Lesser General Public License |
4569 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
4570 | - |
4571 | - |
4572 | -__author__ = "Jorge Niedbalski <jorge.niedbalski@canonical.com>" |
4573 | - |
4574 | -from charmhelpers.fetch import ( |
4575 | - apt_install, |
4576 | - apt_update, |
4577 | -) |
4578 | - |
4579 | -from charmhelpers.core.hookenv import ( |
4580 | - log, |
4581 | - INFO, |
4582 | -) |
4583 | - |
4584 | -try: |
4585 | - from netifaces import interfaces as network_interfaces |
4586 | -except ImportError: |
4587 | - apt_install('python-netifaces') |
4588 | - from netifaces import interfaces as network_interfaces |
4589 | - |
4590 | -import os |
4591 | -import re |
4592 | -import subprocess |
4593 | - |
4594 | -from charmhelpers.core.kernel import modprobe |
4595 | - |
4596 | -REQUIRED_MODULES = ( |
4597 | - "mlx4_ib", |
4598 | - "mlx4_en", |
4599 | - "mlx4_core", |
4600 | - "ib_ipath", |
4601 | - "ib_mthca", |
4602 | - "ib_srpt", |
4603 | - "ib_srp", |
4604 | - "ib_ucm", |
4605 | - "ib_isert", |
4606 | - "ib_iser", |
4607 | - "ib_ipoib", |
4608 | - "ib_cm", |
4609 | - "ib_uverbs" |
4610 | - "ib_umad", |
4611 | - "ib_sa", |
4612 | - "ib_mad", |
4613 | - "ib_core", |
4614 | - "ib_addr", |
4615 | - "rdma_ucm", |
4616 | -) |
4617 | - |
4618 | -REQUIRED_PACKAGES = ( |
4619 | - "ibutils", |
4620 | - "infiniband-diags", |
4621 | - "ibverbs-utils", |
4622 | -) |
4623 | - |
4624 | -IPOIB_DRIVERS = ( |
4625 | - "ib_ipoib", |
4626 | -) |
4627 | - |
4628 | -ABI_VERSION_FILE = "/sys/class/infiniband_mad/abi_version" |
4629 | - |
4630 | - |
4631 | -class DeviceInfo(object): |
4632 | - pass |
4633 | - |
4634 | - |
4635 | -def install_packages(): |
4636 | - apt_update() |
4637 | - apt_install(REQUIRED_PACKAGES, fatal=True) |
4638 | - |
4639 | - |
4640 | -def load_modules(): |
4641 | - for module in REQUIRED_MODULES: |
4642 | - modprobe(module, persist=True) |
4643 | - |
4644 | - |
4645 | -def is_enabled(): |
4646 | - """Check if infiniband is loaded on the system""" |
4647 | - return os.path.exists(ABI_VERSION_FILE) |
4648 | - |
4649 | - |
4650 | -def stat(): |
4651 | - """Return full output of ibstat""" |
4652 | - return subprocess.check_output(["ibstat"]) |
4653 | - |
4654 | - |
4655 | -def devices(): |
4656 | - """Returns a list of IB enabled devices""" |
4657 | - return subprocess.check_output(['ibstat', '-l']).splitlines() |
4658 | - |
4659 | - |
4660 | -def device_info(device): |
4661 | - """Returns a DeviceInfo object with the current device settings""" |
4662 | - |
4663 | - status = subprocess.check_output([ |
4664 | - 'ibstat', device, '-s']).splitlines() |
4665 | - |
4666 | - regexes = { |
4667 | - "CA type: (.*)": "device_type", |
4668 | - "Number of ports: (.*)": "num_ports", |
4669 | - "Firmware version: (.*)": "fw_ver", |
4670 | - "Hardware version: (.*)": "hw_ver", |
4671 | - "Node GUID: (.*)": "node_guid", |
4672 | - "System image GUID: (.*)": "sys_guid", |
4673 | - } |
4674 | - |
4675 | - device = DeviceInfo() |
4676 | - |
4677 | - for line in status: |
4678 | - for expression, key in regexes.items(): |
4679 | - matches = re.search(expression, line) |
4680 | - if matches: |
4681 | - setattr(device, key, matches.group(1)) |
4682 | - |
4683 | - return device |
4684 | - |
4685 | - |
4686 | -def ipoib_interfaces(): |
4687 | - """Return a list of IPOIB capable ethernet interfaces""" |
4688 | - interfaces = [] |
4689 | - |
4690 | - for interface in network_interfaces(): |
4691 | - try: |
4692 | - driver = re.search('^driver: (.+)$', subprocess.check_output([ |
4693 | - 'ethtool', '-i', |
4694 | - interface]), re.M).group(1) |
4695 | - |
4696 | - if driver in IPOIB_DRIVERS: |
4697 | - interfaces.append(interface) |
4698 | - except: |
4699 | - log("Skipping interface %s" % interface, level=INFO) |
4700 | - continue |
4701 | - |
4702 | - return interfaces |
4703 | |
4704 | === removed directory 'hooks/charmhelpers/contrib/peerstorage' |
4705 | === removed file 'hooks/charmhelpers/contrib/peerstorage/__init__.py' |
4706 | --- hooks/charmhelpers/contrib/peerstorage/__init__.py 2016-01-30 22:40:29 +0000 |
4707 | +++ hooks/charmhelpers/contrib/peerstorage/__init__.py 1970-01-01 00:00:00 +0000 |
4708 | @@ -1,269 +0,0 @@ |
4709 | -# Copyright 2014-2015 Canonical Limited. |
4710 | -# |
4711 | -# This file is part of charm-helpers. |
4712 | -# |
4713 | -# charm-helpers is free software: you can redistribute it and/or modify |
4714 | -# it under the terms of the GNU Lesser General Public License version 3 as |
4715 | -# published by the Free Software Foundation. |
4716 | -# |
4717 | -# charm-helpers is distributed in the hope that it will be useful, |
4718 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
4719 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
4720 | -# GNU Lesser General Public License for more details. |
4721 | -# |
4722 | -# You should have received a copy of the GNU Lesser General Public License |
4723 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
4724 | - |
4725 | -import json |
4726 | -import six |
4727 | - |
4728 | -from charmhelpers.core.hookenv import relation_id as current_relation_id |
4729 | -from charmhelpers.core.hookenv import ( |
4730 | - is_relation_made, |
4731 | - relation_ids, |
4732 | - relation_get as _relation_get, |
4733 | - local_unit, |
4734 | - relation_set as _relation_set, |
4735 | - leader_get as _leader_get, |
4736 | - leader_set, |
4737 | - is_leader, |
4738 | -) |
4739 | - |
4740 | - |
4741 | -""" |
4742 | -This helper provides functions to support use of a peer relation |
4743 | -for basic key/value storage, with the added benefit that all storage |
4744 | -can be replicated across peer units. |
4745 | - |
4746 | -Requirement to use: |
4747 | - |
4748 | -To use this, the "peer_echo()" method has to be called form the peer |
4749 | -relation's relation-changed hook: |
4750 | - |
4751 | -@hooks.hook("cluster-relation-changed") # Adapt the to your peer relation name |
4752 | -def cluster_relation_changed(): |
4753 | - peer_echo() |
4754 | - |
4755 | -Once this is done, you can use peer storage from anywhere: |
4756 | - |
4757 | -@hooks.hook("some-hook") |
4758 | -def some_hook(): |
4759 | - # You can store and retrieve key/values this way: |
4760 | - if is_relation_made("cluster"): # from charmhelpers.core.hookenv |
4761 | - # There are peers available so we can work with peer storage |
4762 | - peer_store("mykey", "myvalue") |
4763 | - value = peer_retrieve("mykey") |
4764 | - print value |
4765 | - else: |
4766 | - print "No peers joind the relation, cannot share key/values :(" |
4767 | -""" |
4768 | - |
4769 | - |
4770 | -def leader_get(attribute=None, rid=None): |
4771 | - """Wrapper to ensure that settings are migrated from the peer relation. |
4772 | - |
4773 | - This is to support upgrading an environment that does not support |
4774 | - Juju leadership election to one that does. |
4775 | - |
4776 | - If a setting is not extant in the leader-get but is on the relation-get |
4777 | - peer rel, it is migrated and marked as such so that it is not re-migrated. |
4778 | - """ |
4779 | - migration_key = '__leader_get_migrated_settings__' |
4780 | - if not is_leader(): |
4781 | - return _leader_get(attribute=attribute) |
4782 | - |
4783 | - settings_migrated = False |
4784 | - leader_settings = _leader_get(attribute=attribute) |
4785 | - previously_migrated = _leader_get(attribute=migration_key) |
4786 | - |
4787 | - if previously_migrated: |
4788 | - migrated = set(json.loads(previously_migrated)) |
4789 | - else: |
4790 | - migrated = set([]) |
4791 | - |
4792 | - try: |
4793 | - if migration_key in leader_settings: |
4794 | - del leader_settings[migration_key] |
4795 | - except TypeError: |
4796 | - pass |
4797 | - |
4798 | - if attribute: |
4799 | - if attribute in migrated: |
4800 | - return leader_settings |
4801 | - |
4802 | - # If attribute not present in leader db, check if this unit has set |
4803 | - # the attribute in the peer relation |
4804 | - if not leader_settings: |
4805 | - peer_setting = _relation_get(attribute=attribute, unit=local_unit(), |
4806 | - rid=rid) |
4807 | - if peer_setting: |
4808 | - leader_set(settings={attribute: peer_setting}) |
4809 | - leader_settings = peer_setting |
4810 | - |
4811 | - if leader_settings: |
4812 | - settings_migrated = True |
4813 | - migrated.add(attribute) |
4814 | - else: |
4815 | - r_settings = _relation_get(unit=local_unit(), rid=rid) |
4816 | - if r_settings: |
4817 | - for key in set(r_settings.keys()).difference(migrated): |
4818 | - # Leader setting wins |
4819 | - if not leader_settings.get(key): |
4820 | - leader_settings[key] = r_settings[key] |
4821 | - |
4822 | - settings_migrated = True |
4823 | - migrated.add(key) |
4824 | - |
4825 | - if settings_migrated: |
4826 | - leader_set(**leader_settings) |
4827 | - |
4828 | - if migrated and settings_migrated: |
4829 | - migrated = json.dumps(list(migrated)) |
4830 | - leader_set(settings={migration_key: migrated}) |
4831 | - |
4832 | - return leader_settings |
4833 | - |
4834 | - |
4835 | -def relation_set(relation_id=None, relation_settings=None, **kwargs): |
4836 | - """Attempt to use leader-set if supported in the current version of Juju, |
4837 | - otherwise falls back on relation-set. |
4838 | - |
4839 | - Note that we only attempt to use leader-set if the provided relation_id is |
4840 | - a peer relation id or no relation id is provided (in which case we assume |
4841 | - we are within the peer relation context). |
4842 | - """ |
4843 | - try: |
4844 | - if relation_id in relation_ids('cluster'): |
4845 | - return leader_set(settings=relation_settings, **kwargs) |
4846 | - else: |
4847 | - raise NotImplementedError |
4848 | - except NotImplementedError: |
4849 | - return _relation_set(relation_id=relation_id, |
4850 | - relation_settings=relation_settings, **kwargs) |
4851 | - |
4852 | - |
4853 | -def relation_get(attribute=None, unit=None, rid=None): |
4854 | - """Attempt to use leader-get if supported in the current version of Juju, |
4855 | - otherwise falls back on relation-get. |
4856 | - |
4857 | - Note that we only attempt to use leader-get if the provided rid is a peer |
4858 | - relation id or no relation id is provided (in which case we assume we are |
4859 | - within the peer relation context). |
4860 | - """ |
4861 | - try: |
4862 | - if rid in relation_ids('cluster'): |
4863 | - return leader_get(attribute, rid) |
4864 | - else: |
4865 | - raise NotImplementedError |
4866 | - except NotImplementedError: |
4867 | - return _relation_get(attribute=attribute, rid=rid, unit=unit) |
4868 | - |
4869 | - |
4870 | -def peer_retrieve(key, relation_name='cluster'): |
4871 | - """Retrieve a named key from peer relation `relation_name`.""" |
4872 | - cluster_rels = relation_ids(relation_name) |
4873 | - if len(cluster_rels) > 0: |
4874 | - cluster_rid = cluster_rels[0] |
4875 | - return relation_get(attribute=key, rid=cluster_rid, |
4876 | - unit=local_unit()) |
4877 | - else: |
4878 | - raise ValueError('Unable to detect' |
4879 | - 'peer relation {}'.format(relation_name)) |
4880 | - |
4881 | - |
4882 | -def peer_retrieve_by_prefix(prefix, relation_name='cluster', delimiter='_', |
4883 | - inc_list=None, exc_list=None): |
4884 | - """ Retrieve k/v pairs given a prefix and filter using {inc,exc}_list """ |
4885 | - inc_list = inc_list if inc_list else [] |
4886 | - exc_list = exc_list if exc_list else [] |
4887 | - peerdb_settings = peer_retrieve('-', relation_name=relation_name) |
4888 | - matched = {} |
4889 | - if peerdb_settings is None: |
4890 | - return matched |
4891 | - for k, v in peerdb_settings.items(): |
4892 | - full_prefix = prefix + delimiter |
4893 | - if k.startswith(full_prefix): |
4894 | - new_key = k.replace(full_prefix, '') |
4895 | - if new_key in exc_list: |
4896 | - continue |
4897 | - if new_key in inc_list or len(inc_list) == 0: |
4898 | - matched[new_key] = v |
4899 | - return matched |
4900 | - |
4901 | - |
4902 | -def peer_store(key, value, relation_name='cluster'): |
4903 | - """Store the key/value pair on the named peer relation `relation_name`.""" |
4904 | - cluster_rels = relation_ids(relation_name) |
4905 | - if len(cluster_rels) > 0: |
4906 | - cluster_rid = cluster_rels[0] |
4907 | - relation_set(relation_id=cluster_rid, |
4908 | - relation_settings={key: value}) |
4909 | - else: |
4910 | - raise ValueError('Unable to detect ' |
4911 | - 'peer relation {}'.format(relation_name)) |
4912 | - |
4913 | - |
4914 | -def peer_echo(includes=None, force=False): |
4915 | - """Echo filtered attributes back onto the same relation for storage. |
4916 | - |
4917 | - This is a requirement to use the peerstorage module - it needs to be called |
4918 | - from the peer relation's changed hook. |
4919 | - |
4920 | - If Juju leader support exists this will be a noop unless force is True. |
4921 | - """ |
4922 | - try: |
4923 | - is_leader() |
4924 | - except NotImplementedError: |
4925 | - pass |
4926 | - else: |
4927 | - if not force: |
4928 | - return # NOOP if leader-election is supported |
4929 | - |
4930 | - # Use original non-leader calls |
4931 | - relation_get = _relation_get |
4932 | - relation_set = _relation_set |
4933 | - |
4934 | - rdata = relation_get() |
4935 | - echo_data = {} |
4936 | - if includes is None: |
4937 | - echo_data = rdata.copy() |
4938 | - for ex in ['private-address', 'public-address']: |
4939 | - if ex in echo_data: |
4940 | - echo_data.pop(ex) |
4941 | - else: |
4942 | - for attribute, value in six.iteritems(rdata): |
4943 | - for include in includes: |
4944 | - if include in attribute: |
4945 | - echo_data[attribute] = value |
4946 | - if len(echo_data) > 0: |
4947 | - relation_set(relation_settings=echo_data) |
4948 | - |
4949 | - |
4950 | -def peer_store_and_set(relation_id=None, peer_relation_name='cluster', |
4951 | - peer_store_fatal=False, relation_settings=None, |
4952 | - delimiter='_', **kwargs): |
4953 | - """Store passed-in arguments both in argument relation and in peer storage. |
4954 | - |
4955 | - It functions like doing relation_set() and peer_store() at the same time, |
4956 | - with the same data. |
4957 | - |
4958 | - @param relation_id: the id of the relation to store the data on. Defaults |
4959 | - to the current relation. |
4960 | - @param peer_store_fatal: Set to True, the function will raise an exception |
4961 | - should the peer sotrage not be avialable.""" |
4962 | - |
4963 | - relation_settings = relation_settings if relation_settings else {} |
4964 | - relation_set(relation_id=relation_id, |
4965 | - relation_settings=relation_settings, |
4966 | - **kwargs) |
4967 | - if is_relation_made(peer_relation_name): |
4968 | - for key, value in six.iteritems(dict(list(kwargs.items()) + |
4969 | - list(relation_settings.items()))): |
4970 | - key_prefix = relation_id or current_relation_id() |
4971 | - peer_store(key_prefix + delimiter + key, |
4972 | - value, |
4973 | - relation_name=peer_relation_name) |
4974 | - else: |
4975 | - if peer_store_fatal: |
4976 | - raise ValueError('Unable to detect ' |
4977 | - 'peer relation {}'.format(peer_relation_name)) |
4978 | |
4979 | === removed directory 'hooks/charmhelpers/contrib/saltstack' |
4980 | === removed file 'hooks/charmhelpers/contrib/saltstack/__init__.py' |
4981 | --- hooks/charmhelpers/contrib/saltstack/__init__.py 2015-07-29 18:23:55 +0000 |
4982 | +++ hooks/charmhelpers/contrib/saltstack/__init__.py 1970-01-01 00:00:00 +0000 |
4983 | @@ -1,118 +0,0 @@ |
4984 | -# Copyright 2014-2015 Canonical Limited. |
4985 | -# |
4986 | -# This file is part of charm-helpers. |
4987 | -# |
4988 | -# charm-helpers is free software: you can redistribute it and/or modify |
4989 | -# it under the terms of the GNU Lesser General Public License version 3 as |
4990 | -# published by the Free Software Foundation. |
4991 | -# |
4992 | -# charm-helpers is distributed in the hope that it will be useful, |
4993 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
4994 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
4995 | -# GNU Lesser General Public License for more details. |
4996 | -# |
4997 | -# You should have received a copy of the GNU Lesser General Public License |
4998 | -# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
4999 | - |
5000 | -"""Charm Helpers saltstack - declare the state of your machines. |
The diff has been truncated for viewing.