Merge lp:~junaidali/charms/trusty/plumgrid-gateway/pg-restart into lp:~plumgrid-team/charms/trusty/plumgrid-gateway/trunk

Proposed by Junaid Ali on 2016-04-27
Status: Merged
Merged at revision: 28
Proposed branch: lp:~junaidali/charms/trusty/plumgrid-gateway/pg-restart
Merge into: lp:~plumgrid-team/charms/trusty/plumgrid-gateway/trunk
Diff against target: 6049 lines (+11/-5761)
48 files modified
charm-helpers-sync.yaml (+6/-1)
hooks/charmhelpers/contrib/ansible/__init__.py (+0/-254)
hooks/charmhelpers/contrib/benchmark/__init__.py (+0/-126)
hooks/charmhelpers/contrib/charmhelpers/__init__.py (+0/-208)
hooks/charmhelpers/contrib/charmsupport/__init__.py (+0/-15)
hooks/charmhelpers/contrib/charmsupport/nrpe.py (+0/-398)
hooks/charmhelpers/contrib/charmsupport/volumes.py (+0/-175)
hooks/charmhelpers/contrib/database/mysql.py (+0/-412)
hooks/charmhelpers/contrib/hardening/__init__.py (+0/-15)
hooks/charmhelpers/contrib/hardening/apache/__init__.py (+0/-19)
hooks/charmhelpers/contrib/hardening/apache/checks/__init__.py (+0/-31)
hooks/charmhelpers/contrib/hardening/apache/checks/config.py (+0/-100)
hooks/charmhelpers/contrib/hardening/audits/__init__.py (+0/-63)
hooks/charmhelpers/contrib/hardening/audits/apache.py (+0/-100)
hooks/charmhelpers/contrib/hardening/audits/apt.py (+0/-105)
hooks/charmhelpers/contrib/hardening/audits/file.py (+0/-552)
hooks/charmhelpers/contrib/hardening/harden.py (+0/-84)
hooks/charmhelpers/contrib/hardening/host/__init__.py (+0/-19)
hooks/charmhelpers/contrib/hardening/host/checks/__init__.py (+0/-50)
hooks/charmhelpers/contrib/hardening/host/checks/apt.py (+0/-39)
hooks/charmhelpers/contrib/hardening/host/checks/limits.py (+0/-55)
hooks/charmhelpers/contrib/hardening/host/checks/login.py (+0/-67)
hooks/charmhelpers/contrib/hardening/host/checks/minimize_access.py (+0/-52)
hooks/charmhelpers/contrib/hardening/host/checks/pam.py (+0/-134)
hooks/charmhelpers/contrib/hardening/host/checks/profile.py (+0/-45)
hooks/charmhelpers/contrib/hardening/host/checks/securetty.py (+0/-39)
hooks/charmhelpers/contrib/hardening/host/checks/suid_sgid.py (+0/-131)
hooks/charmhelpers/contrib/hardening/host/checks/sysctl.py (+0/-211)
hooks/charmhelpers/contrib/hardening/mysql/__init__.py (+0/-19)
hooks/charmhelpers/contrib/hardening/mysql/checks/__init__.py (+0/-31)
hooks/charmhelpers/contrib/hardening/mysql/checks/config.py (+0/-89)
hooks/charmhelpers/contrib/hardening/ssh/__init__.py (+0/-19)
hooks/charmhelpers/contrib/hardening/ssh/checks/__init__.py (+0/-31)
hooks/charmhelpers/contrib/hardening/ssh/checks/config.py (+0/-394)
hooks/charmhelpers/contrib/hardening/templating.py (+0/-71)
hooks/charmhelpers/contrib/hardening/utils.py (+0/-157)
hooks/charmhelpers/contrib/mellanox/infiniband.py (+0/-151)
hooks/charmhelpers/contrib/peerstorage/__init__.py (+0/-269)
hooks/charmhelpers/contrib/saltstack/__init__.py (+0/-118)
hooks/charmhelpers/contrib/ssl/__init__.py (+0/-94)
hooks/charmhelpers/contrib/ssl/service.py (+0/-279)
hooks/charmhelpers/contrib/templating/__init__.py (+0/-15)
hooks/charmhelpers/contrib/templating/contexts.py (+0/-139)
hooks/charmhelpers/contrib/templating/jinja.py (+0/-40)
hooks/charmhelpers/contrib/templating/pyformat.py (+0/-29)
hooks/charmhelpers/contrib/unison/__init__.py (+0/-313)
hooks/pg_gw_utils.py (+3/-2)
unit_tests/test_pg_gw_hooks.py (+2/-1)
To merge this branch: bzr merge lp:~junaidali/charms/trusty/plumgrid-gateway/pg-restart
Reviewer Review Type Date Requested Status
Bilal Baqar 2016-04-27 Pending
Review via email: mp+293103@code.launchpad.net
To post a comment you must log in.
29. By Junaid Ali on 2016-04-28

Updated restart_pg function

30. By Junaid Ali on 2016-04-28

Updated sleep time after starting libvirt-bin

31. By Junaid Ali on 2016-05-01

update sleep time in restart_pg, changes for make sync

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'charm-helpers-sync.yaml'
2--- charm-helpers-sync.yaml 2015-07-29 18:23:55 +0000
3+++ charm-helpers-sync.yaml 2016-05-01 02:23:39 +0000
4@@ -3,5 +3,10 @@
5 include:
6 - core
7 - fetch
8- - contrib
9+ - contrib.amulet
10+ - contrib.hahelpers
11+ - contrib.network
12+ - contrib.openstack
13+ - contrib.python
14+ - contrib.storage
15 - payload
16
17=== removed directory 'hooks/charmhelpers/contrib/ansible'
18=== removed file 'hooks/charmhelpers/contrib/ansible/__init__.py'
19--- hooks/charmhelpers/contrib/ansible/__init__.py 2015-07-29 18:23:55 +0000
20+++ hooks/charmhelpers/contrib/ansible/__init__.py 1970-01-01 00:00:00 +0000
21@@ -1,254 +0,0 @@
22-# Copyright 2014-2015 Canonical Limited.
23-#
24-# This file is part of charm-helpers.
25-#
26-# charm-helpers is free software: you can redistribute it and/or modify
27-# it under the terms of the GNU Lesser General Public License version 3 as
28-# published by the Free Software Foundation.
29-#
30-# charm-helpers is distributed in the hope that it will be useful,
31-# but WITHOUT ANY WARRANTY; without even the implied warranty of
32-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
33-# GNU Lesser General Public License for more details.
34-#
35-# You should have received a copy of the GNU Lesser General Public License
36-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
37-
38-# Copyright 2013 Canonical Ltd.
39-#
40-# Authors:
41-# Charm Helpers Developers <juju@lists.ubuntu.com>
42-"""Charm Helpers ansible - declare the state of your machines.
43-
44-This helper enables you to declare your machine state, rather than
45-program it procedurally (and have to test each change to your procedures).
46-Your install hook can be as simple as::
47-
48- {{{
49- import charmhelpers.contrib.ansible
50-
51-
52- def install():
53- charmhelpers.contrib.ansible.install_ansible_support()
54- charmhelpers.contrib.ansible.apply_playbook('playbooks/install.yaml')
55- }}}
56-
57-and won't need to change (nor will its tests) when you change the machine
58-state.
59-
60-All of your juju config and relation-data are available as template
61-variables within your playbooks and templates. An install playbook looks
62-something like::
63-
64- {{{
65- ---
66- - hosts: localhost
67- user: root
68-
69- tasks:
70- - name: Add private repositories.
71- template:
72- src: ../templates/private-repositories.list.jinja2
73- dest: /etc/apt/sources.list.d/private.list
74-
75- - name: Update the cache.
76- apt: update_cache=yes
77-
78- - name: Install dependencies.
79- apt: pkg={{ item }}
80- with_items:
81- - python-mimeparse
82- - python-webob
83- - sunburnt
84-
85- - name: Setup groups.
86- group: name={{ item.name }} gid={{ item.gid }}
87- with_items:
88- - { name: 'deploy_user', gid: 1800 }
89- - { name: 'service_user', gid: 1500 }
90-
91- ...
92- }}}
93-
94-Read more online about `playbooks`_ and standard ansible `modules`_.
95-
96-.. _playbooks: http://www.ansibleworks.com/docs/playbooks.html
97-.. _modules: http://www.ansibleworks.com/docs/modules.html
98-
99-A further feature os the ansible hooks is to provide a light weight "action"
100-scripting tool. This is a decorator that you apply to a function, and that
101-function can now receive cli args, and can pass extra args to the playbook.
102-
103-e.g.
104-
105-
106-@hooks.action()
107-def some_action(amount, force="False"):
108- "Usage: some-action AMOUNT [force=True]" # <-- shown on error
109- # process the arguments
110- # do some calls
111- # return extra-vars to be passed to ansible-playbook
112- return {
113- 'amount': int(amount),
114- 'type': force,
115- }
116-
117-You can now create a symlink to hooks.py that can be invoked like a hook, but
118-with cli params:
119-
120-# link actions/some-action to hooks/hooks.py
121-
122-actions/some-action amount=10 force=true
123-
124-"""
125-import os
126-import stat
127-import subprocess
128-import functools
129-
130-import charmhelpers.contrib.templating.contexts
131-import charmhelpers.core.host
132-import charmhelpers.core.hookenv
133-import charmhelpers.fetch
134-
135-
136-charm_dir = os.environ.get('CHARM_DIR', '')
137-ansible_hosts_path = '/etc/ansible/hosts'
138-# Ansible will automatically include any vars in the following
139-# file in its inventory when run locally.
140-ansible_vars_path = '/etc/ansible/host_vars/localhost'
141-
142-
143-def install_ansible_support(from_ppa=True, ppa_location='ppa:rquillo/ansible'):
144- """Installs the ansible package.
145-
146- By default it is installed from the `PPA`_ linked from
147- the ansible `website`_ or from a ppa specified by a charm config..
148-
149- .. _PPA: https://launchpad.net/~rquillo/+archive/ansible
150- .. _website: http://docs.ansible.com/intro_installation.html#latest-releases-via-apt-ubuntu
151-
152- If from_ppa is empty, you must ensure that the package is available
153- from a configured repository.
154- """
155- if from_ppa:
156- charmhelpers.fetch.add_source(ppa_location)
157- charmhelpers.fetch.apt_update(fatal=True)
158- charmhelpers.fetch.apt_install('ansible')
159- with open(ansible_hosts_path, 'w+') as hosts_file:
160- hosts_file.write('localhost ansible_connection=local')
161-
162-
163-def apply_playbook(playbook, tags=None, extra_vars=None):
164- tags = tags or []
165- tags = ",".join(tags)
166- charmhelpers.contrib.templating.contexts.juju_state_to_yaml(
167- ansible_vars_path, namespace_separator='__',
168- allow_hyphens_in_keys=False, mode=(stat.S_IRUSR | stat.S_IWUSR))
169-
170- # we want ansible's log output to be unbuffered
171- env = os.environ.copy()
172- env['PYTHONUNBUFFERED'] = "1"
173- call = [
174- 'ansible-playbook',
175- '-c',
176- 'local',
177- playbook,
178- ]
179- if tags:
180- call.extend(['--tags', '{}'.format(tags)])
181- if extra_vars:
182- extra = ["%s=%s" % (k, v) for k, v in extra_vars.items()]
183- call.extend(['--extra-vars', " ".join(extra)])
184- subprocess.check_call(call, env=env)
185-
186-
187-class AnsibleHooks(charmhelpers.core.hookenv.Hooks):
188- """Run a playbook with the hook-name as the tag.
189-
190- This helper builds on the standard hookenv.Hooks helper,
191- but additionally runs the playbook with the hook-name specified
192- using --tags (ie. running all the tasks tagged with the hook-name).
193-
194- Example::
195-
196- hooks = AnsibleHooks(playbook_path='playbooks/my_machine_state.yaml')
197-
198- # All the tasks within my_machine_state.yaml tagged with 'install'
199- # will be run automatically after do_custom_work()
200- @hooks.hook()
201- def install():
202- do_custom_work()
203-
204- # For most of your hooks, you won't need to do anything other
205- # than run the tagged tasks for the hook:
206- @hooks.hook('config-changed', 'start', 'stop')
207- def just_use_playbook():
208- pass
209-
210- # As a convenience, you can avoid the above noop function by specifying
211- # the hooks which are handled by ansible-only and they'll be registered
212- # for you:
213- # hooks = AnsibleHooks(
214- # 'playbooks/my_machine_state.yaml',
215- # default_hooks=['config-changed', 'start', 'stop'])
216-
217- if __name__ == "__main__":
218- # execute a hook based on the name the program is called by
219- hooks.execute(sys.argv)
220-
221- """
222-
223- def __init__(self, playbook_path, default_hooks=None):
224- """Register any hooks handled by ansible."""
225- super(AnsibleHooks, self).__init__()
226-
227- self._actions = {}
228- self.playbook_path = playbook_path
229-
230- default_hooks = default_hooks or []
231-
232- def noop(*args, **kwargs):
233- pass
234-
235- for hook in default_hooks:
236- self.register(hook, noop)
237-
238- def register_action(self, name, function):
239- """Register a hook"""
240- self._actions[name] = function
241-
242- def execute(self, args):
243- """Execute the hook followed by the playbook using the hook as tag."""
244- hook_name = os.path.basename(args[0])
245- extra_vars = None
246- if hook_name in self._actions:
247- extra_vars = self._actions[hook_name](args[1:])
248- else:
249- super(AnsibleHooks, self).execute(args)
250-
251- charmhelpers.contrib.ansible.apply_playbook(
252- self.playbook_path, tags=[hook_name], extra_vars=extra_vars)
253-
254- def action(self, *action_names):
255- """Decorator, registering them as actions"""
256- def action_wrapper(decorated):
257-
258- @functools.wraps(decorated)
259- def wrapper(argv):
260- kwargs = dict(arg.split('=') for arg in argv)
261- try:
262- return decorated(**kwargs)
263- except TypeError as e:
264- if decorated.__doc__:
265- e.args += (decorated.__doc__,)
266- raise
267-
268- self.register_action(decorated.__name__, wrapper)
269- if '_' in decorated.__name__:
270- self.register_action(
271- decorated.__name__.replace('_', '-'), wrapper)
272-
273- return wrapper
274-
275- return action_wrapper
276
277=== removed directory 'hooks/charmhelpers/contrib/benchmark'
278=== removed file 'hooks/charmhelpers/contrib/benchmark/__init__.py'
279--- hooks/charmhelpers/contrib/benchmark/__init__.py 2015-07-29 18:23:55 +0000
280+++ hooks/charmhelpers/contrib/benchmark/__init__.py 1970-01-01 00:00:00 +0000
281@@ -1,126 +0,0 @@
282-# Copyright 2014-2015 Canonical Limited.
283-#
284-# This file is part of charm-helpers.
285-#
286-# charm-helpers is free software: you can redistribute it and/or modify
287-# it under the terms of the GNU Lesser General Public License version 3 as
288-# published by the Free Software Foundation.
289-#
290-# charm-helpers is distributed in the hope that it will be useful,
291-# but WITHOUT ANY WARRANTY; without even the implied warranty of
292-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
293-# GNU Lesser General Public License for more details.
294-#
295-# You should have received a copy of the GNU Lesser General Public License
296-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
297-
298-import subprocess
299-import time
300-import os
301-from distutils.spawn import find_executable
302-
303-from charmhelpers.core.hookenv import (
304- in_relation_hook,
305- relation_ids,
306- relation_set,
307- relation_get,
308-)
309-
310-
311-def action_set(key, val):
312- if find_executable('action-set'):
313- action_cmd = ['action-set']
314-
315- if isinstance(val, dict):
316- for k, v in iter(val.items()):
317- action_set('%s.%s' % (key, k), v)
318- return True
319-
320- action_cmd.append('%s=%s' % (key, val))
321- subprocess.check_call(action_cmd)
322- return True
323- return False
324-
325-
326-class Benchmark():
327- """
328- Helper class for the `benchmark` interface.
329-
330- :param list actions: Define the actions that are also benchmarks
331-
332- From inside the benchmark-relation-changed hook, you would
333- Benchmark(['memory', 'cpu', 'disk', 'smoke', 'custom'])
334-
335- Examples:
336-
337- siege = Benchmark(['siege'])
338- siege.start()
339- [... run siege ...]
340- # The higher the score, the better the benchmark
341- siege.set_composite_score(16.70, 'trans/sec', 'desc')
342- siege.finish()
343-
344-
345- """
346-
347- BENCHMARK_CONF = '/etc/benchmark.conf' # Replaced in testing
348-
349- required_keys = [
350- 'hostname',
351- 'port',
352- 'graphite_port',
353- 'graphite_endpoint',
354- 'api_port'
355- ]
356-
357- def __init__(self, benchmarks=None):
358- if in_relation_hook():
359- if benchmarks is not None:
360- for rid in sorted(relation_ids('benchmark')):
361- relation_set(relation_id=rid, relation_settings={
362- 'benchmarks': ",".join(benchmarks)
363- })
364-
365- # Check the relation data
366- config = {}
367- for key in self.required_keys:
368- val = relation_get(key)
369- if val is not None:
370- config[key] = val
371- else:
372- # We don't have all of the required keys
373- config = {}
374- break
375-
376- if len(config):
377- with open(self.BENCHMARK_CONF, 'w') as f:
378- for key, val in iter(config.items()):
379- f.write("%s=%s\n" % (key, val))
380-
381- @staticmethod
382- def start():
383- action_set('meta.start', time.strftime('%Y-%m-%dT%H:%M:%SZ'))
384-
385- """
386- If the collectd charm is also installed, tell it to send a snapshot
387- of the current profile data.
388- """
389- COLLECT_PROFILE_DATA = '/usr/local/bin/collect-profile-data'
390- if os.path.exists(COLLECT_PROFILE_DATA):
391- subprocess.check_output([COLLECT_PROFILE_DATA])
392-
393- @staticmethod
394- def finish():
395- action_set('meta.stop', time.strftime('%Y-%m-%dT%H:%M:%SZ'))
396-
397- @staticmethod
398- def set_composite_score(value, units, direction='asc'):
399- """
400- Set the composite score for a benchmark run. This is a single number
401- representative of the benchmark results. This could be the most
402- important metric, or an amalgamation of metric scores.
403- """
404- return action_set(
405- "meta.composite",
406- {'value': value, 'units': units, 'direction': direction}
407- )
408
409=== removed directory 'hooks/charmhelpers/contrib/charmhelpers'
410=== removed file 'hooks/charmhelpers/contrib/charmhelpers/__init__.py'
411--- hooks/charmhelpers/contrib/charmhelpers/__init__.py 2015-07-29 18:23:55 +0000
412+++ hooks/charmhelpers/contrib/charmhelpers/__init__.py 1970-01-01 00:00:00 +0000
413@@ -1,208 +0,0 @@
414-# Copyright 2014-2015 Canonical Limited.
415-#
416-# This file is part of charm-helpers.
417-#
418-# charm-helpers is free software: you can redistribute it and/or modify
419-# it under the terms of the GNU Lesser General Public License version 3 as
420-# published by the Free Software Foundation.
421-#
422-# charm-helpers is distributed in the hope that it will be useful,
423-# but WITHOUT ANY WARRANTY; without even the implied warranty of
424-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
425-# GNU Lesser General Public License for more details.
426-#
427-# You should have received a copy of the GNU Lesser General Public License
428-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
429-
430-# Copyright 2012 Canonical Ltd. This software is licensed under the
431-# GNU Affero General Public License version 3 (see the file LICENSE).
432-
433-import warnings
434-warnings.warn("contrib.charmhelpers is deprecated", DeprecationWarning) # noqa
435-
436-import operator
437-import tempfile
438-import time
439-import yaml
440-import subprocess
441-
442-import six
443-if six.PY3:
444- from urllib.request import urlopen
445- from urllib.error import (HTTPError, URLError)
446-else:
447- from urllib2 import (urlopen, HTTPError, URLError)
448-
449-"""Helper functions for writing Juju charms in Python."""
450-
451-__metaclass__ = type
452-__all__ = [
453- # 'get_config', # core.hookenv.config()
454- # 'log', # core.hookenv.log()
455- # 'log_entry', # core.hookenv.log()
456- # 'log_exit', # core.hookenv.log()
457- # 'relation_get', # core.hookenv.relation_get()
458- # 'relation_set', # core.hookenv.relation_set()
459- # 'relation_ids', # core.hookenv.relation_ids()
460- # 'relation_list', # core.hookenv.relation_units()
461- # 'config_get', # core.hookenv.config()
462- # 'unit_get', # core.hookenv.unit_get()
463- # 'open_port', # core.hookenv.open_port()
464- # 'close_port', # core.hookenv.close_port()
465- # 'service_control', # core.host.service()
466- 'unit_info', # client-side, NOT IMPLEMENTED
467- 'wait_for_machine', # client-side, NOT IMPLEMENTED
468- 'wait_for_page_contents', # client-side, NOT IMPLEMENTED
469- 'wait_for_relation', # client-side, NOT IMPLEMENTED
470- 'wait_for_unit', # client-side, NOT IMPLEMENTED
471-]
472-
473-
474-SLEEP_AMOUNT = 0.1
475-
476-
477-# We create a juju_status Command here because it makes testing much,
478-# much easier.
479-def juju_status():
480- subprocess.check_call(['juju', 'status'])
481-
482-# re-implemented as charmhelpers.fetch.configure_sources()
483-# def configure_source(update=False):
484-# source = config_get('source')
485-# if ((source.startswith('ppa:') or
486-# source.startswith('cloud:') or
487-# source.startswith('http:'))):
488-# run('add-apt-repository', source)
489-# if source.startswith("http:"):
490-# run('apt-key', 'import', config_get('key'))
491-# if update:
492-# run('apt-get', 'update')
493-
494-
495-# DEPRECATED: client-side only
496-def make_charm_config_file(charm_config):
497- charm_config_file = tempfile.NamedTemporaryFile(mode='w+')
498- charm_config_file.write(yaml.dump(charm_config))
499- charm_config_file.flush()
500- # The NamedTemporaryFile instance is returned instead of just the name
501- # because we want to take advantage of garbage collection-triggered
502- # deletion of the temp file when it goes out of scope in the caller.
503- return charm_config_file
504-
505-
506-# DEPRECATED: client-side only
507-def unit_info(service_name, item_name, data=None, unit=None):
508- if data is None:
509- data = yaml.safe_load(juju_status())
510- service = data['services'].get(service_name)
511- if service is None:
512- # XXX 2012-02-08 gmb:
513- # This allows us to cope with the race condition that we
514- # have between deploying a service and having it come up in
515- # `juju status`. We could probably do with cleaning it up so
516- # that it fails a bit more noisily after a while.
517- return ''
518- units = service['units']
519- if unit is not None:
520- item = units[unit][item_name]
521- else:
522- # It might seem odd to sort the units here, but we do it to
523- # ensure that when no unit is specified, the first unit for the
524- # service (or at least the one with the lowest number) is the
525- # one whose data gets returned.
526- sorted_unit_names = sorted(units.keys())
527- item = units[sorted_unit_names[0]][item_name]
528- return item
529-
530-
531-# DEPRECATED: client-side only
532-def get_machine_data():
533- return yaml.safe_load(juju_status())['machines']
534-
535-
536-# DEPRECATED: client-side only
537-def wait_for_machine(num_machines=1, timeout=300):
538- """Wait `timeout` seconds for `num_machines` machines to come up.
539-
540- This wait_for... function can be called by other wait_for functions
541- whose timeouts might be too short in situations where only a bare
542- Juju setup has been bootstrapped.
543-
544- :return: A tuple of (num_machines, time_taken). This is used for
545- testing.
546- """
547- # You may think this is a hack, and you'd be right. The easiest way
548- # to tell what environment we're working in (LXC vs EC2) is to check
549- # the dns-name of the first machine. If it's localhost we're in LXC
550- # and we can just return here.
551- if get_machine_data()[0]['dns-name'] == 'localhost':
552- return 1, 0
553- start_time = time.time()
554- while True:
555- # Drop the first machine, since it's the Zookeeper and that's
556- # not a machine that we need to wait for. This will only work
557- # for EC2 environments, which is why we return early above if
558- # we're in LXC.
559- machine_data = get_machine_data()
560- non_zookeeper_machines = [
561- machine_data[key] for key in list(machine_data.keys())[1:]]
562- if len(non_zookeeper_machines) >= num_machines:
563- all_machines_running = True
564- for machine in non_zookeeper_machines:
565- if machine.get('instance-state') != 'running':
566- all_machines_running = False
567- break
568- if all_machines_running:
569- break
570- if time.time() - start_time >= timeout:
571- raise RuntimeError('timeout waiting for service to start')
572- time.sleep(SLEEP_AMOUNT)
573- return num_machines, time.time() - start_time
574-
575-
576-# DEPRECATED: client-side only
577-def wait_for_unit(service_name, timeout=480):
578- """Wait `timeout` seconds for a given service name to come up."""
579- wait_for_machine(num_machines=1)
580- start_time = time.time()
581- while True:
582- state = unit_info(service_name, 'agent-state')
583- if 'error' in state or state == 'started':
584- break
585- if time.time() - start_time >= timeout:
586- raise RuntimeError('timeout waiting for service to start')
587- time.sleep(SLEEP_AMOUNT)
588- if state != 'started':
589- raise RuntimeError('unit did not start, agent-state: ' + state)
590-
591-
592-# DEPRECATED: client-side only
593-def wait_for_relation(service_name, relation_name, timeout=120):
594- """Wait `timeout` seconds for a given relation to come up."""
595- start_time = time.time()
596- while True:
597- relation = unit_info(service_name, 'relations').get(relation_name)
598- if relation is not None and relation['state'] == 'up':
599- break
600- if time.time() - start_time >= timeout:
601- raise RuntimeError('timeout waiting for relation to be up')
602- time.sleep(SLEEP_AMOUNT)
603-
604-
605-# DEPRECATED: client-side only
606-def wait_for_page_contents(url, contents, timeout=120, validate=None):
607- if validate is None:
608- validate = operator.contains
609- start_time = time.time()
610- while True:
611- try:
612- stream = urlopen(url)
613- except (HTTPError, URLError):
614- pass
615- else:
616- page = stream.read()
617- if validate(page, contents):
618- return page
619- if time.time() - start_time >= timeout:
620- raise RuntimeError('timeout waiting for contents of ' + url)
621- time.sleep(SLEEP_AMOUNT)
622
623=== removed directory 'hooks/charmhelpers/contrib/charmsupport'
624=== removed file 'hooks/charmhelpers/contrib/charmsupport/__init__.py'
625--- hooks/charmhelpers/contrib/charmsupport/__init__.py 2015-07-29 18:23:55 +0000
626+++ hooks/charmhelpers/contrib/charmsupport/__init__.py 1970-01-01 00:00:00 +0000
627@@ -1,15 +0,0 @@
628-# Copyright 2014-2015 Canonical Limited.
629-#
630-# This file is part of charm-helpers.
631-#
632-# charm-helpers is free software: you can redistribute it and/or modify
633-# it under the terms of the GNU Lesser General Public License version 3 as
634-# published by the Free Software Foundation.
635-#
636-# charm-helpers is distributed in the hope that it will be useful,
637-# but WITHOUT ANY WARRANTY; without even the implied warranty of
638-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
639-# GNU Lesser General Public License for more details.
640-#
641-# You should have received a copy of the GNU Lesser General Public License
642-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
643
644=== removed file 'hooks/charmhelpers/contrib/charmsupport/nrpe.py'
645--- hooks/charmhelpers/contrib/charmsupport/nrpe.py 2016-01-30 22:40:29 +0000
646+++ hooks/charmhelpers/contrib/charmsupport/nrpe.py 1970-01-01 00:00:00 +0000
647@@ -1,398 +0,0 @@
648-# Copyright 2014-2015 Canonical Limited.
649-#
650-# This file is part of charm-helpers.
651-#
652-# charm-helpers is free software: you can redistribute it and/or modify
653-# it under the terms of the GNU Lesser General Public License version 3 as
654-# published by the Free Software Foundation.
655-#
656-# charm-helpers is distributed in the hope that it will be useful,
657-# but WITHOUT ANY WARRANTY; without even the implied warranty of
658-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
659-# GNU Lesser General Public License for more details.
660-#
661-# You should have received a copy of the GNU Lesser General Public License
662-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
663-
664-"""Compatibility with the nrpe-external-master charm"""
665-# Copyright 2012 Canonical Ltd.
666-#
667-# Authors:
668-# Matthew Wedgwood <matthew.wedgwood@canonical.com>
669-
670-import subprocess
671-import pwd
672-import grp
673-import os
674-import glob
675-import shutil
676-import re
677-import shlex
678-import yaml
679-
680-from charmhelpers.core.hookenv import (
681- config,
682- local_unit,
683- log,
684- relation_ids,
685- relation_set,
686- relations_of_type,
687-)
688-
689-from charmhelpers.core.host import service
690-
691-# This module adds compatibility with the nrpe-external-master and plain nrpe
692-# subordinate charms. To use it in your charm:
693-#
694-# 1. Update metadata.yaml
695-#
696-# provides:
697-# (...)
698-# nrpe-external-master:
699-# interface: nrpe-external-master
700-# scope: container
701-#
702-# and/or
703-#
704-# provides:
705-# (...)
706-# local-monitors:
707-# interface: local-monitors
708-# scope: container
709-
710-#
711-# 2. Add the following to config.yaml
712-#
713-# nagios_context:
714-# default: "juju"
715-# type: string
716-# description: |
717-# Used by the nrpe subordinate charms.
718-# A string that will be prepended to instance name to set the host name
719-# in nagios. So for instance the hostname would be something like:
720-# juju-myservice-0
721-# If you're running multiple environments with the same services in them
722-# this allows you to differentiate between them.
723-# nagios_servicegroups:
724-# default: ""
725-# type: string
726-# description: |
727-# A comma-separated list of nagios servicegroups.
728-# If left empty, the nagios_context will be used as the servicegroup
729-#
730-# 3. Add custom checks (Nagios plugins) to files/nrpe-external-master
731-#
732-# 4. Update your hooks.py with something like this:
733-#
734-# from charmsupport.nrpe import NRPE
735-# (...)
736-# def update_nrpe_config():
737-# nrpe_compat = NRPE()
738-# nrpe_compat.add_check(
739-# shortname = "myservice",
740-# description = "Check MyService",
741-# check_cmd = "check_http -w 2 -c 10 http://localhost"
742-# )
743-# nrpe_compat.add_check(
744-# "myservice_other",
745-# "Check for widget failures",
746-# check_cmd = "/srv/myapp/scripts/widget_check"
747-# )
748-# nrpe_compat.write()
749-#
750-# def config_changed():
751-# (...)
752-# update_nrpe_config()
753-#
754-# def nrpe_external_master_relation_changed():
755-# update_nrpe_config()
756-#
757-# def local_monitors_relation_changed():
758-# update_nrpe_config()
759-#
760-# 5. ln -s hooks.py nrpe-external-master-relation-changed
761-# ln -s hooks.py local-monitors-relation-changed
762-
763-
764-class CheckException(Exception):
765- pass
766-
767-
768-class Check(object):
769- shortname_re = '[A-Za-z0-9-_]+$'
770- service_template = ("""
771-#---------------------------------------------------
772-# This file is Juju managed
773-#---------------------------------------------------
774-define service {{
775- use active-service
776- host_name {nagios_hostname}
777- service_description {nagios_hostname}[{shortname}] """
778- """{description}
779- check_command check_nrpe!{command}
780- servicegroups {nagios_servicegroup}
781-}}
782-""")
783-
784- def __init__(self, shortname, description, check_cmd):
785- super(Check, self).__init__()
786- # XXX: could be better to calculate this from the service name
787- if not re.match(self.shortname_re, shortname):
788- raise CheckException("shortname must match {}".format(
789- Check.shortname_re))
790- self.shortname = shortname
791- self.command = "check_{}".format(shortname)
792- # Note: a set of invalid characters is defined by the
793- # Nagios server config
794- # The default is: illegal_object_name_chars=`~!$%^&*"|'<>?,()=
795- self.description = description
796- self.check_cmd = self._locate_cmd(check_cmd)
797-
798- def _get_check_filename(self):
799- return os.path.join(NRPE.nrpe_confdir, '{}.cfg'.format(self.command))
800-
801- def _get_service_filename(self, hostname):
802- return os.path.join(NRPE.nagios_exportdir,
803- 'service__{}_{}.cfg'.format(hostname, self.command))
804-
805- def _locate_cmd(self, check_cmd):
806- search_path = (
807- '/usr/lib/nagios/plugins',
808- '/usr/local/lib/nagios/plugins',
809- )
810- parts = shlex.split(check_cmd)
811- for path in search_path:
812- if os.path.exists(os.path.join(path, parts[0])):
813- command = os.path.join(path, parts[0])
814- if len(parts) > 1:
815- command += " " + " ".join(parts[1:])
816- return command
817- log('Check command not found: {}'.format(parts[0]))
818- return ''
819-
820- def _remove_service_files(self):
821- if not os.path.exists(NRPE.nagios_exportdir):
822- return
823- for f in os.listdir(NRPE.nagios_exportdir):
824- if f.endswith('_{}.cfg'.format(self.command)):
825- os.remove(os.path.join(NRPE.nagios_exportdir, f))
826-
827- def remove(self, hostname):
828- nrpe_check_file = self._get_check_filename()
829- if os.path.exists(nrpe_check_file):
830- os.remove(nrpe_check_file)
831- self._remove_service_files()
832-
833- def write(self, nagios_context, hostname, nagios_servicegroups):
834- nrpe_check_file = self._get_check_filename()
835- with open(nrpe_check_file, 'w') as nrpe_check_config:
836- nrpe_check_config.write("# check {}\n".format(self.shortname))
837- nrpe_check_config.write("command[{}]={}\n".format(
838- self.command, self.check_cmd))
839-
840- if not os.path.exists(NRPE.nagios_exportdir):
841- log('Not writing service config as {} is not accessible'.format(
842- NRPE.nagios_exportdir))
843- else:
844- self.write_service_config(nagios_context, hostname,
845- nagios_servicegroups)
846-
847- def write_service_config(self, nagios_context, hostname,
848- nagios_servicegroups):
849- self._remove_service_files()
850-
851- templ_vars = {
852- 'nagios_hostname': hostname,
853- 'nagios_servicegroup': nagios_servicegroups,
854- 'description': self.description,
855- 'shortname': self.shortname,
856- 'command': self.command,
857- }
858- nrpe_service_text = Check.service_template.format(**templ_vars)
859- nrpe_service_file = self._get_service_filename(hostname)
860- with open(nrpe_service_file, 'w') as nrpe_service_config:
861- nrpe_service_config.write(str(nrpe_service_text))
862-
863- def run(self):
864- subprocess.call(self.check_cmd)
865-
866-
867-class NRPE(object):
868- nagios_logdir = '/var/log/nagios'
869- nagios_exportdir = '/var/lib/nagios/export'
870- nrpe_confdir = '/etc/nagios/nrpe.d'
871-
872- def __init__(self, hostname=None):
873- super(NRPE, self).__init__()
874- self.config = config()
875- self.nagios_context = self.config['nagios_context']
876- if 'nagios_servicegroups' in self.config and self.config['nagios_servicegroups']:
877- self.nagios_servicegroups = self.config['nagios_servicegroups']
878- else:
879- self.nagios_servicegroups = self.nagios_context
880- self.unit_name = local_unit().replace('/', '-')
881- if hostname:
882- self.hostname = hostname
883- else:
884- nagios_hostname = get_nagios_hostname()
885- if nagios_hostname:
886- self.hostname = nagios_hostname
887- else:
888- self.hostname = "{}-{}".format(self.nagios_context, self.unit_name)
889- self.checks = []
890-
891- def add_check(self, *args, **kwargs):
892- self.checks.append(Check(*args, **kwargs))
893-
894- def remove_check(self, *args, **kwargs):
895- if kwargs.get('shortname') is None:
896- raise ValueError('shortname of check must be specified')
897-
898- # Use sensible defaults if they're not specified - these are not
899- # actually used during removal, but they're required for constructing
900- # the Check object; check_disk is chosen because it's part of the
901- # nagios-plugins-basic package.
902- if kwargs.get('check_cmd') is None:
903- kwargs['check_cmd'] = 'check_disk'
904- if kwargs.get('description') is None:
905- kwargs['description'] = ''
906-
907- check = Check(*args, **kwargs)
908- check.remove(self.hostname)
909-
910- def write(self):
911- try:
912- nagios_uid = pwd.getpwnam('nagios').pw_uid
913- nagios_gid = grp.getgrnam('nagios').gr_gid
914- except:
915- log("Nagios user not set up, nrpe checks not updated")
916- return
917-
918- if not os.path.exists(NRPE.nagios_logdir):
919- os.mkdir(NRPE.nagios_logdir)
920- os.chown(NRPE.nagios_logdir, nagios_uid, nagios_gid)
921-
922- nrpe_monitors = {}
923- monitors = {"monitors": {"remote": {"nrpe": nrpe_monitors}}}
924- for nrpecheck in self.checks:
925- nrpecheck.write(self.nagios_context, self.hostname,
926- self.nagios_servicegroups)
927- nrpe_monitors[nrpecheck.shortname] = {
928- "command": nrpecheck.command,
929- }
930-
931- service('restart', 'nagios-nrpe-server')
932-
933- monitor_ids = relation_ids("local-monitors") + \
934- relation_ids("nrpe-external-master")
935- for rid in monitor_ids:
936- relation_set(relation_id=rid, monitors=yaml.dump(monitors))
937-
938-
939-def get_nagios_hostcontext(relation_name='nrpe-external-master'):
940- """
941- Query relation with nrpe subordinate, return the nagios_host_context
942-
943- :param str relation_name: Name of relation nrpe sub joined to
944- """
945- for rel in relations_of_type(relation_name):
946- if 'nagios_host_context' in rel:
947- return rel['nagios_host_context']
948-
949-
950-def get_nagios_hostname(relation_name='nrpe-external-master'):
951- """
952- Query relation with nrpe subordinate, return the nagios_hostname
953-
954- :param str relation_name: Name of relation nrpe sub joined to
955- """
956- for rel in relations_of_type(relation_name):
957- if 'nagios_hostname' in rel:
958- return rel['nagios_hostname']
959-
960-
961-def get_nagios_unit_name(relation_name='nrpe-external-master'):
962- """
963- Return the nagios unit name prepended with host_context if needed
964-
965- :param str relation_name: Name of relation nrpe sub joined to
966- """
967- host_context = get_nagios_hostcontext(relation_name)
968- if host_context:
969- unit = "%s:%s" % (host_context, local_unit())
970- else:
971- unit = local_unit()
972- return unit
973-
974-
975-def add_init_service_checks(nrpe, services, unit_name):
976- """
977- Add checks for each service in list
978-
979- :param NRPE nrpe: NRPE object to add check to
980- :param list services: List of services to check
981- :param str unit_name: Unit name to use in check description
982- """
983- for svc in services:
984- upstart_init = '/etc/init/%s.conf' % svc
985- sysv_init = '/etc/init.d/%s' % svc
986- if os.path.exists(upstart_init):
987- # Don't add a check for these services from neutron-gateway
988- if svc not in ['ext-port', 'os-charm-phy-nic-mtu']:
989- nrpe.add_check(
990- shortname=svc,
991- description='process check {%s}' % unit_name,
992- check_cmd='check_upstart_job %s' % svc
993- )
994- elif os.path.exists(sysv_init):
995- cronpath = '/etc/cron.d/nagios-service-check-%s' % svc
996- cron_file = ('*/5 * * * * root '
997- '/usr/local/lib/nagios/plugins/check_exit_status.pl '
998- '-s /etc/init.d/%s status > '
999- '/var/lib/nagios/service-check-%s.txt\n' % (svc,
1000- svc)
1001- )
1002- f = open(cronpath, 'w')
1003- f.write(cron_file)
1004- f.close()
1005- nrpe.add_check(
1006- shortname=svc,
1007- description='process check {%s}' % unit_name,
1008- check_cmd='check_status_file.py -f '
1009- '/var/lib/nagios/service-check-%s.txt' % svc,
1010- )
1011-
1012-
1013-def copy_nrpe_checks():
1014- """
1015- Copy the nrpe checks into place
1016-
1017- """
1018- NAGIOS_PLUGINS = '/usr/local/lib/nagios/plugins'
1019- nrpe_files_dir = os.path.join(os.getenv('CHARM_DIR'), 'hooks',
1020- 'charmhelpers', 'contrib', 'openstack',
1021- 'files')
1022-
1023- if not os.path.exists(NAGIOS_PLUGINS):
1024- os.makedirs(NAGIOS_PLUGINS)
1025- for fname in glob.glob(os.path.join(nrpe_files_dir, "check_*")):
1026- if os.path.isfile(fname):
1027- shutil.copy2(fname,
1028- os.path.join(NAGIOS_PLUGINS, os.path.basename(fname)))
1029-
1030-
1031-def add_haproxy_checks(nrpe, unit_name):
1032- """
1033- Add checks for each service in list
1034-
1035- :param NRPE nrpe: NRPE object to add check to
1036- :param str unit_name: Unit name to use in check description
1037- """
1038- nrpe.add_check(
1039- shortname='haproxy_servers',
1040- description='Check HAProxy {%s}' % unit_name,
1041- check_cmd='check_haproxy.sh')
1042- nrpe.add_check(
1043- shortname='haproxy_queue',
1044- description='Check HAProxy queue depth {%s}' % unit_name,
1045- check_cmd='check_haproxy_queue_depth.sh')
1046
1047=== removed file 'hooks/charmhelpers/contrib/charmsupport/volumes.py'
1048--- hooks/charmhelpers/contrib/charmsupport/volumes.py 2015-07-29 18:23:55 +0000
1049+++ hooks/charmhelpers/contrib/charmsupport/volumes.py 1970-01-01 00:00:00 +0000
1050@@ -1,175 +0,0 @@
1051-# Copyright 2014-2015 Canonical Limited.
1052-#
1053-# This file is part of charm-helpers.
1054-#
1055-# charm-helpers is free software: you can redistribute it and/or modify
1056-# it under the terms of the GNU Lesser General Public License version 3 as
1057-# published by the Free Software Foundation.
1058-#
1059-# charm-helpers is distributed in the hope that it will be useful,
1060-# but WITHOUT ANY WARRANTY; without even the implied warranty of
1061-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1062-# GNU Lesser General Public License for more details.
1063-#
1064-# You should have received a copy of the GNU Lesser General Public License
1065-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1066-
1067-'''
1068-Functions for managing volumes in juju units. One volume is supported per unit.
1069-Subordinates may have their own storage, provided it is on its own partition.
1070-
1071-Configuration stanzas::
1072-
1073- volume-ephemeral:
1074- type: boolean
1075- default: true
1076- description: >
1077- If false, a volume is mounted as sepecified in "volume-map"
1078- If true, ephemeral storage will be used, meaning that log data
1079- will only exist as long as the machine. YOU HAVE BEEN WARNED.
1080- volume-map:
1081- type: string
1082- default: {}
1083- description: >
1084- YAML map of units to device names, e.g:
1085- "{ rsyslog/0: /dev/vdb, rsyslog/1: /dev/vdb }"
1086- Service units will raise a configure-error if volume-ephemeral
1087- is 'true' and no volume-map value is set. Use 'juju set' to set a
1088- value and 'juju resolved' to complete configuration.
1089-
1090-Usage::
1091-
1092- from charmsupport.volumes import configure_volume, VolumeConfigurationError
1093- from charmsupport.hookenv import log, ERROR
1094- def post_mount_hook():
1095- stop_service('myservice')
1096- def post_mount_hook():
1097- start_service('myservice')
1098-
1099- if __name__ == '__main__':
1100- try:
1101- configure_volume(before_change=pre_mount_hook,
1102- after_change=post_mount_hook)
1103- except VolumeConfigurationError:
1104- log('Storage could not be configured', ERROR)
1105-
1106-'''
1107-
1108-# XXX: Known limitations
1109-# - fstab is neither consulted nor updated
1110-
1111-import os
1112-from charmhelpers.core import hookenv
1113-from charmhelpers.core import host
1114-import yaml
1115-
1116-
1117-MOUNT_BASE = '/srv/juju/volumes'
1118-
1119-
1120-class VolumeConfigurationError(Exception):
1121- '''Volume configuration data is missing or invalid'''
1122- pass
1123-
1124-
1125-def get_config():
1126- '''Gather and sanity-check volume configuration data'''
1127- volume_config = {}
1128- config = hookenv.config()
1129-
1130- errors = False
1131-
1132- if config.get('volume-ephemeral') in (True, 'True', 'true', 'Yes', 'yes'):
1133- volume_config['ephemeral'] = True
1134- else:
1135- volume_config['ephemeral'] = False
1136-
1137- try:
1138- volume_map = yaml.safe_load(config.get('volume-map', '{}'))
1139- except yaml.YAMLError as e:
1140- hookenv.log("Error parsing YAML volume-map: {}".format(e),
1141- hookenv.ERROR)
1142- errors = True
1143- if volume_map is None:
1144- # probably an empty string
1145- volume_map = {}
1146- elif not isinstance(volume_map, dict):
1147- hookenv.log("Volume-map should be a dictionary, not {}".format(
1148- type(volume_map)))
1149- errors = True
1150-
1151- volume_config['device'] = volume_map.get(os.environ['JUJU_UNIT_NAME'])
1152- if volume_config['device'] and volume_config['ephemeral']:
1153- # asked for ephemeral storage but also defined a volume ID
1154- hookenv.log('A volume is defined for this unit, but ephemeral '
1155- 'storage was requested', hookenv.ERROR)
1156- errors = True
1157- elif not volume_config['device'] and not volume_config['ephemeral']:
1158- # asked for permanent storage but did not define volume ID
1159- hookenv.log('Ephemeral storage was requested, but there is no volume '
1160- 'defined for this unit.', hookenv.ERROR)
1161- errors = True
1162-
1163- unit_mount_name = hookenv.local_unit().replace('/', '-')
1164- volume_config['mountpoint'] = os.path.join(MOUNT_BASE, unit_mount_name)
1165-
1166- if errors:
1167- return None
1168- return volume_config
1169-
1170-
1171-def mount_volume(config):
1172- if os.path.exists(config['mountpoint']):
1173- if not os.path.isdir(config['mountpoint']):
1174- hookenv.log('Not a directory: {}'.format(config['mountpoint']))
1175- raise VolumeConfigurationError()
1176- else:
1177- host.mkdir(config['mountpoint'])
1178- if os.path.ismount(config['mountpoint']):
1179- unmount_volume(config)
1180- if not host.mount(config['device'], config['mountpoint'], persist=True):
1181- raise VolumeConfigurationError()
1182-
1183-
1184-def unmount_volume(config):
1185- if os.path.ismount(config['mountpoint']):
1186- if not host.umount(config['mountpoint'], persist=True):
1187- raise VolumeConfigurationError()
1188-
1189-
1190-def managed_mounts():
1191- '''List of all mounted managed volumes'''
1192- return filter(lambda mount: mount[0].startswith(MOUNT_BASE), host.mounts())
1193-
1194-
1195-def configure_volume(before_change=lambda: None, after_change=lambda: None):
1196- '''Set up storage (or don't) according to the charm's volume configuration.
1197- Returns the mount point or "ephemeral". before_change and after_change
1198- are optional functions to be called if the volume configuration changes.
1199- '''
1200-
1201- config = get_config()
1202- if not config:
1203- hookenv.log('Failed to read volume configuration', hookenv.CRITICAL)
1204- raise VolumeConfigurationError()
1205-
1206- if config['ephemeral']:
1207- if os.path.ismount(config['mountpoint']):
1208- before_change()
1209- unmount_volume(config)
1210- after_change()
1211- return 'ephemeral'
1212- else:
1213- # persistent storage
1214- if os.path.ismount(config['mountpoint']):
1215- mounts = dict(managed_mounts())
1216- if mounts.get(config['mountpoint']) != config['device']:
1217- before_change()
1218- unmount_volume(config)
1219- mount_volume(config)
1220- after_change()
1221- else:
1222- before_change()
1223- mount_volume(config)
1224- after_change()
1225- return config['mountpoint']
1226
1227=== removed directory 'hooks/charmhelpers/contrib/database'
1228=== removed file 'hooks/charmhelpers/contrib/database/__init__.py'
1229=== removed file 'hooks/charmhelpers/contrib/database/mysql.py'
1230--- hooks/charmhelpers/contrib/database/mysql.py 2015-07-29 18:23:55 +0000
1231+++ hooks/charmhelpers/contrib/database/mysql.py 1970-01-01 00:00:00 +0000
1232@@ -1,412 +0,0 @@
1233-"""Helper for working with a MySQL database"""
1234-import json
1235-import re
1236-import sys
1237-import platform
1238-import os
1239-import glob
1240-
1241-# from string import upper
1242-
1243-from charmhelpers.core.host import (
1244- mkdir,
1245- pwgen,
1246- write_file
1247-)
1248-from charmhelpers.core.hookenv import (
1249- config as config_get,
1250- relation_get,
1251- related_units,
1252- unit_get,
1253- log,
1254- DEBUG,
1255- INFO,
1256- WARNING,
1257-)
1258-from charmhelpers.fetch import (
1259- apt_install,
1260- apt_update,
1261- filter_installed_packages,
1262-)
1263-from charmhelpers.contrib.peerstorage import (
1264- peer_store,
1265- peer_retrieve,
1266-)
1267-from charmhelpers.contrib.network.ip import get_host_ip
1268-
1269-try:
1270- import MySQLdb
1271-except ImportError:
1272- apt_update(fatal=True)
1273- apt_install(filter_installed_packages(['python-mysqldb']), fatal=True)
1274- import MySQLdb
1275-
1276-
1277-class MySQLHelper(object):
1278-
1279- def __init__(self, rpasswdf_template, upasswdf_template, host='localhost',
1280- migrate_passwd_to_peer_relation=True,
1281- delete_ondisk_passwd_file=True):
1282- self.host = host
1283- # Password file path templates
1284- self.root_passwd_file_template = rpasswdf_template
1285- self.user_passwd_file_template = upasswdf_template
1286-
1287- self.migrate_passwd_to_peer_relation = migrate_passwd_to_peer_relation
1288- # If we migrate we have the option to delete local copy of root passwd
1289- self.delete_ondisk_passwd_file = delete_ondisk_passwd_file
1290-
1291- def connect(self, user='root', password=None):
1292- log("Opening db connection for %s@%s" % (user, self.host), level=DEBUG)
1293- self.connection = MySQLdb.connect(user=user, host=self.host,
1294- passwd=password)
1295-
1296- def database_exists(self, db_name):
1297- cursor = self.connection.cursor()
1298- try:
1299- cursor.execute("SHOW DATABASES")
1300- databases = [i[0] for i in cursor.fetchall()]
1301- finally:
1302- cursor.close()
1303-
1304- return db_name in databases
1305-
1306- def create_database(self, db_name):
1307- cursor = self.connection.cursor()
1308- try:
1309- cursor.execute("CREATE DATABASE {} CHARACTER SET UTF8"
1310- .format(db_name))
1311- finally:
1312- cursor.close()
1313-
1314- def grant_exists(self, db_name, db_user, remote_ip):
1315- cursor = self.connection.cursor()
1316- priv_string = "GRANT ALL PRIVILEGES ON `{}`.* " \
1317- "TO '{}'@'{}'".format(db_name, db_user, remote_ip)
1318- try:
1319- cursor.execute("SHOW GRANTS for '{}'@'{}'".format(db_user,
1320- remote_ip))
1321- grants = [i[0] for i in cursor.fetchall()]
1322- except MySQLdb.OperationalError:
1323- return False
1324- finally:
1325- cursor.close()
1326-
1327- # TODO: review for different grants
1328- return priv_string in grants
1329-
1330- def create_grant(self, db_name, db_user, remote_ip, password):
1331- cursor = self.connection.cursor()
1332- try:
1333- # TODO: review for different grants
1334- cursor.execute("GRANT ALL PRIVILEGES ON {}.* TO '{}'@'{}' "
1335- "IDENTIFIED BY '{}'".format(db_name,
1336- db_user,
1337- remote_ip,
1338- password))
1339- finally:
1340- cursor.close()
1341-
1342- def create_admin_grant(self, db_user, remote_ip, password):
1343- cursor = self.connection.cursor()
1344- try:
1345- cursor.execute("GRANT ALL PRIVILEGES ON *.* TO '{}'@'{}' "
1346- "IDENTIFIED BY '{}'".format(db_user,
1347- remote_ip,
1348- password))
1349- finally:
1350- cursor.close()
1351-
1352- def cleanup_grant(self, db_user, remote_ip):
1353- cursor = self.connection.cursor()
1354- try:
1355- cursor.execute("DROP FROM mysql.user WHERE user='{}' "
1356- "AND HOST='{}'".format(db_user,
1357- remote_ip))
1358- finally:
1359- cursor.close()
1360-
1361- def execute(self, sql):
1362- """Execute arbitary SQL against the database."""
1363- cursor = self.connection.cursor()
1364- try:
1365- cursor.execute(sql)
1366- finally:
1367- cursor.close()
1368-
1369- def migrate_passwords_to_peer_relation(self, excludes=None):
1370- """Migrate any passwords storage on disk to cluster peer relation."""
1371- dirname = os.path.dirname(self.root_passwd_file_template)
1372- path = os.path.join(dirname, '*.passwd')
1373- for f in glob.glob(path):
1374- if excludes and f in excludes:
1375- log("Excluding %s from peer migration" % (f), level=DEBUG)
1376- continue
1377-
1378- key = os.path.basename(f)
1379- with open(f, 'r') as passwd:
1380- _value = passwd.read().strip()
1381-
1382- try:
1383- peer_store(key, _value)
1384-
1385- if self.delete_ondisk_passwd_file:
1386- os.unlink(f)
1387- except ValueError:
1388- # NOTE cluster relation not yet ready - skip for now
1389- pass
1390-
1391- def get_mysql_password_on_disk(self, username=None, password=None):
1392- """Retrieve, generate or store a mysql password for the provided
1393- username on disk."""
1394- if username:
1395- template = self.user_passwd_file_template
1396- passwd_file = template.format(username)
1397- else:
1398- passwd_file = self.root_passwd_file_template
1399-
1400- _password = None
1401- if os.path.exists(passwd_file):
1402- log("Using existing password file '%s'" % passwd_file, level=DEBUG)
1403- with open(passwd_file, 'r') as passwd:
1404- _password = passwd.read().strip()
1405- else:
1406- log("Generating new password file '%s'" % passwd_file, level=DEBUG)
1407- if not os.path.isdir(os.path.dirname(passwd_file)):
1408- # NOTE: need to ensure this is not mysql root dir (which needs
1409- # to be mysql readable)
1410- mkdir(os.path.dirname(passwd_file), owner='root', group='root',
1411- perms=0o770)
1412- # Force permissions - for some reason the chmod in makedirs
1413- # fails
1414- os.chmod(os.path.dirname(passwd_file), 0o770)
1415-
1416- _password = password or pwgen(length=32)
1417- write_file(passwd_file, _password, owner='root', group='root',
1418- perms=0o660)
1419-
1420- return _password
1421-
1422- def passwd_keys(self, username):
1423- """Generator to return keys used to store passwords in peer store.
1424-
1425- NOTE: we support both legacy and new format to support mysql
1426- charm prior to refactor. This is necessary to avoid LP 1451890.
1427- """
1428- keys = []
1429- if username == 'mysql':
1430- log("Bad username '%s'" % (username), level=WARNING)
1431-
1432- if username:
1433- # IMPORTANT: *newer* format must be returned first
1434- keys.append('mysql-%s.passwd' % (username))
1435- keys.append('%s.passwd' % (username))
1436- else:
1437- keys.append('mysql.passwd')
1438-
1439- for key in keys:
1440- yield key
1441-
1442- def get_mysql_password(self, username=None, password=None):
1443- """Retrieve, generate or store a mysql password for the provided
1444- username using peer relation cluster."""
1445- excludes = []
1446-
1447- # First check peer relation.
1448- try:
1449- for key in self.passwd_keys(username):
1450- _password = peer_retrieve(key)
1451- if _password:
1452- break
1453-
1454- # If root password available don't update peer relation from local
1455- if _password and not username:
1456- excludes.append(self.root_passwd_file_template)
1457-
1458- except ValueError:
1459- # cluster relation is not yet started; use on-disk
1460- _password = None
1461-
1462- # If none available, generate new one
1463- if not _password:
1464- _password = self.get_mysql_password_on_disk(username, password)
1465-
1466- # Put on wire if required
1467- if self.migrate_passwd_to_peer_relation:
1468- self.migrate_passwords_to_peer_relation(excludes=excludes)
1469-
1470- return _password
1471-
1472- def get_mysql_root_password(self, password=None):
1473- """Retrieve or generate mysql root password for service units."""
1474- return self.get_mysql_password(username=None, password=password)
1475-
1476- def normalize_address(self, hostname):
1477- """Ensure that address returned is an IP address (i.e. not fqdn)"""
1478- if config_get('prefer-ipv6'):
1479- # TODO: add support for ipv6 dns
1480- return hostname
1481-
1482- if hostname != unit_get('private-address'):
1483- return get_host_ip(hostname, fallback=hostname)
1484-
1485- # Otherwise assume localhost
1486- return '127.0.0.1'
1487-
1488- def get_allowed_units(self, database, username, relation_id=None):
1489- """Get list of units with access grants for database with username.
1490-
1491- This is typically used to provide shared-db relations with a list of
1492- which units have been granted access to the given database.
1493- """
1494- self.connect(password=self.get_mysql_root_password())
1495- allowed_units = set()
1496- for unit in related_units(relation_id):
1497- settings = relation_get(rid=relation_id, unit=unit)
1498- # First check for setting with prefix, then without
1499- for attr in ["%s_hostname" % (database), 'hostname']:
1500- hosts = settings.get(attr, None)
1501- if hosts:
1502- break
1503-
1504- if hosts:
1505- # hostname can be json-encoded list of hostnames
1506- try:
1507- hosts = json.loads(hosts)
1508- except ValueError:
1509- hosts = [hosts]
1510- else:
1511- hosts = [settings['private-address']]
1512-
1513- if hosts:
1514- for host in hosts:
1515- host = self.normalize_address(host)
1516- if self.grant_exists(database, username, host):
1517- log("Grant exists for host '%s' on db '%s'" %
1518- (host, database), level=DEBUG)
1519- if unit not in allowed_units:
1520- allowed_units.add(unit)
1521- else:
1522- log("Grant does NOT exist for host '%s' on db '%s'" %
1523- (host, database), level=DEBUG)
1524- else:
1525- log("No hosts found for grant check", level=INFO)
1526-
1527- return allowed_units
1528-
1529- def configure_db(self, hostname, database, username, admin=False):
1530- """Configure access to database for username from hostname."""
1531- self.connect(password=self.get_mysql_root_password())
1532- if not self.database_exists(database):
1533- self.create_database(database)
1534-
1535- remote_ip = self.normalize_address(hostname)
1536- password = self.get_mysql_password(username)
1537- if not self.grant_exists(database, username, remote_ip):
1538- if not admin:
1539- self.create_grant(database, username, remote_ip, password)
1540- else:
1541- self.create_admin_grant(username, remote_ip, password)
1542-
1543- return password
1544-
1545-
1546-class PerconaClusterHelper(object):
1547-
1548- # Going for the biggest page size to avoid wasted bytes.
1549- # InnoDB page size is 16MB
1550-
1551- DEFAULT_PAGE_SIZE = 16 * 1024 * 1024
1552- DEFAULT_INNODB_BUFFER_FACTOR = 0.50
1553-
1554- def human_to_bytes(self, human):
1555- """Convert human readable configuration options to bytes."""
1556- num_re = re.compile('^[0-9]+$')
1557- if num_re.match(human):
1558- return human
1559-
1560- factors = {
1561- 'K': 1024,
1562- 'M': 1048576,
1563- 'G': 1073741824,
1564- 'T': 1099511627776
1565- }
1566- modifier = human[-1]
1567- if modifier in factors:
1568- return int(human[:-1]) * factors[modifier]
1569-
1570- if modifier == '%':
1571- total_ram = self.human_to_bytes(self.get_mem_total())
1572- if self.is_32bit_system() and total_ram > self.sys_mem_limit():
1573- total_ram = self.sys_mem_limit()
1574- factor = int(human[:-1]) * 0.01
1575- pctram = total_ram * factor
1576- return int(pctram - (pctram % self.DEFAULT_PAGE_SIZE))
1577-
1578- raise ValueError("Can only convert K,M,G, or T")
1579-
1580- def is_32bit_system(self):
1581- """Determine whether system is 32 or 64 bit."""
1582- try:
1583- return sys.maxsize < 2 ** 32
1584- except OverflowError:
1585- return False
1586-
1587- def sys_mem_limit(self):
1588- """Determine the default memory limit for the current service unit."""
1589- if platform.machine() in ['armv7l']:
1590- _mem_limit = self.human_to_bytes('2700M') # experimentally determined
1591- else:
1592- # Limit for x86 based 32bit systems
1593- _mem_limit = self.human_to_bytes('4G')
1594-
1595- return _mem_limit
1596-
1597- def get_mem_total(self):
1598- """Calculate the total memory in the current service unit."""
1599- with open('/proc/meminfo') as meminfo_file:
1600- for line in meminfo_file:
1601- key, mem = line.split(':', 2)
1602- if key == 'MemTotal':
1603- mtot, modifier = mem.strip().split(' ')
1604- return '%s%s' % (mtot, modifier[0].upper())
1605-
1606- def parse_config(self):
1607- """Parse charm configuration and calculate values for config files."""
1608- config = config_get()
1609- mysql_config = {}
1610- if 'max-connections' in config:
1611- mysql_config['max_connections'] = config['max-connections']
1612-
1613- if 'wait-timeout' in config:
1614- mysql_config['wait_timeout'] = config['wait-timeout']
1615-
1616- if 'innodb-flush-log-at-trx-commit' in config:
1617- mysql_config['innodb_flush_log_at_trx_commit'] = config['innodb-flush-log-at-trx-commit']
1618-
1619- # Set a sane default key_buffer size
1620- mysql_config['key_buffer'] = self.human_to_bytes('32M')
1621- total_memory = self.human_to_bytes(self.get_mem_total())
1622-
1623- dataset_bytes = config.get('dataset-size', None)
1624- innodb_buffer_pool_size = config.get('innodb-buffer-pool-size', None)
1625-
1626- if innodb_buffer_pool_size:
1627- innodb_buffer_pool_size = self.human_to_bytes(
1628- innodb_buffer_pool_size)
1629- elif dataset_bytes:
1630- log("Option 'dataset-size' has been deprecated, please use"
1631- "innodb_buffer_pool_size option instead", level="WARN")
1632- innodb_buffer_pool_size = self.human_to_bytes(
1633- dataset_bytes)
1634- else:
1635- innodb_buffer_pool_size = int(
1636- total_memory * self.DEFAULT_INNODB_BUFFER_FACTOR)
1637-
1638- if innodb_buffer_pool_size > total_memory:
1639- log("innodb_buffer_pool_size; {} is greater than system available memory:{}".format(
1640- innodb_buffer_pool_size,
1641- total_memory), level='WARN')
1642-
1643- mysql_config['innodb_buffer_pool_size'] = innodb_buffer_pool_size
1644- return mysql_config
1645
1646=== removed directory 'hooks/charmhelpers/contrib/hardening'
1647=== removed file 'hooks/charmhelpers/contrib/hardening/__init__.py'
1648--- hooks/charmhelpers/contrib/hardening/__init__.py 2016-04-22 08:15:33 +0000
1649+++ hooks/charmhelpers/contrib/hardening/__init__.py 1970-01-01 00:00:00 +0000
1650@@ -1,15 +0,0 @@
1651-# Copyright 2016 Canonical Limited.
1652-#
1653-# This file is part of charm-helpers.
1654-#
1655-# charm-helpers is free software: you can redistribute it and/or modify
1656-# it under the terms of the GNU Lesser General Public License version 3 as
1657-# published by the Free Software Foundation.
1658-#
1659-# charm-helpers is distributed in the hope that it will be useful,
1660-# but WITHOUT ANY WARRANTY; without even the implied warranty of
1661-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1662-# GNU Lesser General Public License for more details.
1663-#
1664-# You should have received a copy of the GNU Lesser General Public License
1665-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1666
1667=== removed directory 'hooks/charmhelpers/contrib/hardening/apache'
1668=== removed file 'hooks/charmhelpers/contrib/hardening/apache/__init__.py'
1669--- hooks/charmhelpers/contrib/hardening/apache/__init__.py 2016-04-22 08:15:33 +0000
1670+++ hooks/charmhelpers/contrib/hardening/apache/__init__.py 1970-01-01 00:00:00 +0000
1671@@ -1,19 +0,0 @@
1672-# Copyright 2016 Canonical Limited.
1673-#
1674-# This file is part of charm-helpers.
1675-#
1676-# charm-helpers is free software: you can redistribute it and/or modify
1677-# it under the terms of the GNU Lesser General Public License version 3 as
1678-# published by the Free Software Foundation.
1679-#
1680-# charm-helpers is distributed in the hope that it will be useful,
1681-# but WITHOUT ANY WARRANTY; without even the implied warranty of
1682-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1683-# GNU Lesser General Public License for more details.
1684-#
1685-# You should have received a copy of the GNU Lesser General Public License
1686-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1687-
1688-from os import path
1689-
1690-TEMPLATES_DIR = path.join(path.dirname(__file__), 'templates')
1691
1692=== removed directory 'hooks/charmhelpers/contrib/hardening/apache/checks'
1693=== removed file 'hooks/charmhelpers/contrib/hardening/apache/checks/__init__.py'
1694--- hooks/charmhelpers/contrib/hardening/apache/checks/__init__.py 2016-04-22 08:15:33 +0000
1695+++ hooks/charmhelpers/contrib/hardening/apache/checks/__init__.py 1970-01-01 00:00:00 +0000
1696@@ -1,31 +0,0 @@
1697-# Copyright 2016 Canonical Limited.
1698-#
1699-# This file is part of charm-helpers.
1700-#
1701-# charm-helpers is free software: you can redistribute it and/or modify
1702-# it under the terms of the GNU Lesser General Public License version 3 as
1703-# published by the Free Software Foundation.
1704-#
1705-# charm-helpers is distributed in the hope that it will be useful,
1706-# but WITHOUT ANY WARRANTY; without even the implied warranty of
1707-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1708-# GNU Lesser General Public License for more details.
1709-#
1710-# You should have received a copy of the GNU Lesser General Public License
1711-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1712-
1713-from charmhelpers.core.hookenv import (
1714- log,
1715- DEBUG,
1716-)
1717-from charmhelpers.contrib.hardening.apache.checks import config
1718-
1719-
1720-def run_apache_checks():
1721- log("Starting Apache hardening checks.", level=DEBUG)
1722- checks = config.get_audits()
1723- for check in checks:
1724- log("Running '%s' check" % (check.__class__.__name__), level=DEBUG)
1725- check.ensure_compliance()
1726-
1727- log("Apache hardening checks complete.", level=DEBUG)
1728
1729=== removed file 'hooks/charmhelpers/contrib/hardening/apache/checks/config.py'
1730--- hooks/charmhelpers/contrib/hardening/apache/checks/config.py 2016-04-22 08:15:33 +0000
1731+++ hooks/charmhelpers/contrib/hardening/apache/checks/config.py 1970-01-01 00:00:00 +0000
1732@@ -1,100 +0,0 @@
1733-# Copyright 2016 Canonical Limited.
1734-#
1735-# This file is part of charm-helpers.
1736-#
1737-# charm-helpers is free software: you can redistribute it and/or modify
1738-# it under the terms of the GNU Lesser General Public License version 3 as
1739-# published by the Free Software Foundation.
1740-#
1741-# charm-helpers is distributed in the hope that it will be useful,
1742-# but WITHOUT ANY WARRANTY; without even the implied warranty of
1743-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1744-# GNU Lesser General Public License for more details.
1745-#
1746-# You should have received a copy of the GNU Lesser General Public License
1747-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1748-
1749-import os
1750-import re
1751-import subprocess
1752-
1753-
1754-from charmhelpers.core.hookenv import (
1755- log,
1756- INFO,
1757-)
1758-from charmhelpers.contrib.hardening.audits.file import (
1759- FilePermissionAudit,
1760- DirectoryPermissionAudit,
1761- NoReadWriteForOther,
1762- TemplatedFile,
1763-)
1764-from charmhelpers.contrib.hardening.audits.apache import DisabledModuleAudit
1765-from charmhelpers.contrib.hardening.apache import TEMPLATES_DIR
1766-from charmhelpers.contrib.hardening import utils
1767-
1768-
1769-def get_audits():
1770- """Get Apache hardening config audits.
1771-
1772- :returns: dictionary of audits
1773- """
1774- if subprocess.call(['which', 'apache2'], stdout=subprocess.PIPE) != 0:
1775- log("Apache server does not appear to be installed on this node - "
1776- "skipping apache hardening", level=INFO)
1777- return []
1778-
1779- context = ApacheConfContext()
1780- settings = utils.get_settings('apache')
1781- audits = [
1782- FilePermissionAudit(paths='/etc/apache2/apache2.conf', user='root',
1783- group='root', mode=0o0640),
1784-
1785- TemplatedFile(os.path.join(settings['common']['apache_dir'],
1786- 'mods-available/alias.conf'),
1787- context,
1788- TEMPLATES_DIR,
1789- mode=0o0755,
1790- user='root',
1791- service_actions=[{'service': 'apache2',
1792- 'actions': ['restart']}]),
1793-
1794- TemplatedFile(os.path.join(settings['common']['apache_dir'],
1795- 'conf-enabled/hardening.conf'),
1796- context,
1797- TEMPLATES_DIR,
1798- mode=0o0640,
1799- user='root',
1800- service_actions=[{'service': 'apache2',
1801- 'actions': ['restart']}]),
1802-
1803- DirectoryPermissionAudit(settings['common']['apache_dir'],
1804- user='root',
1805- group='root',
1806- mode=0o640),
1807-
1808- DisabledModuleAudit(settings['hardening']['modules_to_disable']),
1809-
1810- NoReadWriteForOther(settings['common']['apache_dir']),
1811- ]
1812-
1813- return audits
1814-
1815-
1816-class ApacheConfContext(object):
1817- """Defines the set of key/value pairs to set in a apache config file.
1818-
1819- This context, when called, will return a dictionary containing the
1820- key/value pairs of setting to specify in the
1821- /etc/apache/conf-enabled/hardening.conf file.
1822- """
1823- def __call__(self):
1824- settings = utils.get_settings('apache')
1825- ctxt = settings['hardening']
1826-
1827- out = subprocess.check_output(['apache2', '-v'])
1828- ctxt['apache_version'] = re.search(r'.+version: Apache/(.+?)\s.+',
1829- out).group(1)
1830- ctxt['apache_icondir'] = '/usr/share/apache2/icons/'
1831- ctxt['traceenable'] = settings['hardening']['traceenable']
1832- return ctxt
1833
1834=== removed directory 'hooks/charmhelpers/contrib/hardening/audits'
1835=== removed file 'hooks/charmhelpers/contrib/hardening/audits/__init__.py'
1836--- hooks/charmhelpers/contrib/hardening/audits/__init__.py 2016-04-22 08:15:33 +0000
1837+++ hooks/charmhelpers/contrib/hardening/audits/__init__.py 1970-01-01 00:00:00 +0000
1838@@ -1,63 +0,0 @@
1839-# Copyright 2016 Canonical Limited.
1840-#
1841-# This file is part of charm-helpers.
1842-#
1843-# charm-helpers is free software: you can redistribute it and/or modify
1844-# it under the terms of the GNU Lesser General Public License version 3 as
1845-# published by the Free Software Foundation.
1846-#
1847-# charm-helpers is distributed in the hope that it will be useful,
1848-# but WITHOUT ANY WARRANTY; without even the implied warranty of
1849-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1850-# GNU Lesser General Public License for more details.
1851-#
1852-# You should have received a copy of the GNU Lesser General Public License
1853-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1854-
1855-
1856-class BaseAudit(object): # NO-QA
1857- """Base class for hardening checks.
1858-
1859- The lifecycle of a hardening check is to first check to see if the system
1860- is in compliance for the specified check. If it is not in compliance, the
1861- check method will return a value which will be supplied to the.
1862- """
1863- def __init__(self, *args, **kwargs):
1864- self.unless = kwargs.get('unless', None)
1865- super(BaseAudit, self).__init__()
1866-
1867- def ensure_compliance(self):
1868- """Checks to see if the current hardening check is in compliance or
1869- not.
1870-
1871- If the check that is performed is not in compliance, then an exception
1872- should be raised.
1873- """
1874- pass
1875-
1876- def _take_action(self):
1877- """Determines whether to perform the action or not.
1878-
1879- Checks whether or not an action should be taken. This is determined by
1880- the truthy value for the unless parameter. If unless is a callback
1881- method, it will be invoked with no parameters in order to determine
1882- whether or not the action should be taken. Otherwise, the truthy value
1883- of the unless attribute will determine if the action should be
1884- performed.
1885- """
1886- # Do the action if there isn't an unless override.
1887- if self.unless is None:
1888- return True
1889-
1890- # Invoke the callback if there is one.
1891- if hasattr(self.unless, '__call__'):
1892- results = self.unless()
1893- if results:
1894- return False
1895- else:
1896- return True
1897-
1898- if self.unless:
1899- return False
1900- else:
1901- return True
1902
1903=== removed file 'hooks/charmhelpers/contrib/hardening/audits/apache.py'
1904--- hooks/charmhelpers/contrib/hardening/audits/apache.py 2016-04-22 08:15:33 +0000
1905+++ hooks/charmhelpers/contrib/hardening/audits/apache.py 1970-01-01 00:00:00 +0000
1906@@ -1,100 +0,0 @@
1907-# Copyright 2016 Canonical Limited.
1908-#
1909-# This file is part of charm-helpers.
1910-#
1911-# charm-helpers is free software: you can redistribute it and/or modify
1912-# it under the terms of the GNU Lesser General Public License version 3 as
1913-# published by the Free Software Foundation.
1914-#
1915-# charm-helpers is distributed in the hope that it will be useful,
1916-# but WITHOUT ANY WARRANTY; without even the implied warranty of
1917-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1918-# GNU Lesser General Public License for more details.
1919-#
1920-# You should have received a copy of the GNU Lesser General Public License
1921-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1922-
1923-import re
1924-import subprocess
1925-
1926-from six import string_types
1927-
1928-from charmhelpers.core.hookenv import (
1929- log,
1930- INFO,
1931- ERROR,
1932-)
1933-
1934-from charmhelpers.contrib.hardening.audits import BaseAudit
1935-
1936-
1937-class DisabledModuleAudit(BaseAudit):
1938- """Audits Apache2 modules.
1939-
1940- Determines if the apache2 modules are enabled. If the modules are enabled
1941- then they are removed in the ensure_compliance.
1942- """
1943- def __init__(self, modules):
1944- if modules is None:
1945- self.modules = []
1946- elif isinstance(modules, string_types):
1947- self.modules = [modules]
1948- else:
1949- self.modules = modules
1950-
1951- def ensure_compliance(self):
1952- """Ensures that the modules are not loaded."""
1953- if not self.modules:
1954- return
1955-
1956- try:
1957- loaded_modules = self._get_loaded_modules()
1958- non_compliant_modules = []
1959- for module in self.modules:
1960- if module in loaded_modules:
1961- log("Module '%s' is enabled but should not be." %
1962- (module), level=INFO)
1963- non_compliant_modules.append(module)
1964-
1965- if len(non_compliant_modules) == 0:
1966- return
1967-
1968- for module in non_compliant_modules:
1969- self._disable_module(module)
1970- self._restart_apache()
1971- except subprocess.CalledProcessError as e:
1972- log('Error occurred auditing apache module compliance. '
1973- 'This may have been already reported. '
1974- 'Output is: %s' % e.output, level=ERROR)
1975-
1976- @staticmethod
1977- def _get_loaded_modules():
1978- """Returns the modules which are enabled in Apache."""
1979- output = subprocess.check_output(['apache2ctl', '-M'])
1980- modules = []
1981- for line in output.strip().split():
1982- # Each line of the enabled module output looks like:
1983- # module_name (static|shared)
1984- # Plus a header line at the top of the output which is stripped
1985- # out by the regex.
1986- matcher = re.search(r'^ (\S*)', line)
1987- if matcher:
1988- modules.append(matcher.group(1))
1989- return modules
1990-
1991- @staticmethod
1992- def _disable_module(module):
1993- """Disables the specified module in Apache."""
1994- try:
1995- subprocess.check_call(['a2dismod', module])
1996- except subprocess.CalledProcessError as e:
1997- # Note: catch error here to allow the attempt of disabling
1998- # multiple modules in one go rather than failing after the
1999- # first module fails.
2000- log('Error occurred disabling module %s. '
2001- 'Output is: %s' % (module, e.output), level=ERROR)
2002-
2003- @staticmethod
2004- def _restart_apache():
2005- """Restarts the apache process"""
2006- subprocess.check_output(['service', 'apache2', 'restart'])
2007
2008=== removed file 'hooks/charmhelpers/contrib/hardening/audits/apt.py'
2009--- hooks/charmhelpers/contrib/hardening/audits/apt.py 2016-04-22 08:15:33 +0000
2010+++ hooks/charmhelpers/contrib/hardening/audits/apt.py 1970-01-01 00:00:00 +0000
2011@@ -1,105 +0,0 @@
2012-# Copyright 2016 Canonical Limited.
2013-#
2014-# This file is part of charm-helpers.
2015-#
2016-# charm-helpers is free software: you can redistribute it and/or modify
2017-# it under the terms of the GNU Lesser General Public License version 3 as
2018-# published by the Free Software Foundation.
2019-#
2020-# charm-helpers is distributed in the hope that it will be useful,
2021-# but WITHOUT ANY WARRANTY; without even the implied warranty of
2022-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
2023-# GNU Lesser General Public License for more details.
2024-#
2025-# You should have received a copy of the GNU Lesser General Public License
2026-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
2027-
2028-from __future__ import absolute_import # required for external apt import
2029-from apt import apt_pkg
2030-from six import string_types
2031-
2032-from charmhelpers.fetch import (
2033- apt_cache,
2034- apt_purge
2035-)
2036-from charmhelpers.core.hookenv import (
2037- log,
2038- DEBUG,
2039- WARNING,
2040-)
2041-from charmhelpers.contrib.hardening.audits import BaseAudit
2042-
2043-
2044-class AptConfig(BaseAudit):
2045-
2046- def __init__(self, config, **kwargs):
2047- self.config = config
2048-
2049- def verify_config(self):
2050- apt_pkg.init()
2051- for cfg in self.config:
2052- value = apt_pkg.config.get(cfg['key'], cfg.get('default', ''))
2053- if value and value != cfg['expected']:
2054- log("APT config '%s' has unexpected value '%s' "
2055- "(expected='%s')" %
2056- (cfg['key'], value, cfg['expected']), level=WARNING)
2057-
2058- def ensure_compliance(self):
2059- self.verify_config()
2060-
2061-
2062-class RestrictedPackages(BaseAudit):
2063- """Class used to audit restricted packages on the system."""
2064-
2065- def __init__(self, pkgs, **kwargs):
2066- super(RestrictedPackages, self).__init__(**kwargs)
2067- if isinstance(pkgs, string_types) or not hasattr(pkgs, '__iter__'):
2068- self.pkgs = [pkgs]
2069- else:
2070- self.pkgs = pkgs
2071-
2072- def ensure_compliance(self):
2073- cache = apt_cache()
2074-
2075- for p in self.pkgs:
2076- if p not in cache:
2077- continue
2078-
2079- pkg = cache[p]
2080- if not self.is_virtual_package(pkg):
2081- if not pkg.current_ver:
2082- log("Package '%s' is not installed." % pkg.name,
2083- level=DEBUG)
2084- continue
2085- else:
2086- log("Restricted package '%s' is installed" % pkg.name,
2087- level=WARNING)
2088- self.delete_package(cache, pkg)
2089- else:
2090- log("Checking restricted virtual package '%s' provides" %
2091- pkg.name, level=DEBUG)
2092- self.delete_package(cache, pkg)
2093-
2094- def delete_package(self, cache, pkg):
2095- """Deletes the package from the system.
2096-
2097- Deletes the package form the system, properly handling virtual
2098- packages.
2099-
2100- :param cache: the apt cache
2101- :param pkg: the package to remove
2102- """
2103- if self.is_virtual_package(pkg):
2104- log("Package '%s' appears to be virtual - purging provides" %
2105- pkg.name, level=DEBUG)
2106- for _p in pkg.provides_list:
2107- self.delete_package(cache, _p[2].parent_pkg)
2108- elif not pkg.current_ver:
2109- log("Package '%s' not installed" % pkg.name, level=DEBUG)
2110- return
2111- else:
2112- log("Purging package '%s'" % pkg.name, level=DEBUG)
2113- apt_purge(pkg.name)
2114-
2115- def is_virtual_package(self, pkg):
2116- return pkg.has_provides and not pkg.has_versions
2117
2118=== removed file 'hooks/charmhelpers/contrib/hardening/audits/file.py'
2119--- hooks/charmhelpers/contrib/hardening/audits/file.py 2016-04-22 08:15:33 +0000
2120+++ hooks/charmhelpers/contrib/hardening/audits/file.py 1970-01-01 00:00:00 +0000
2121@@ -1,552 +0,0 @@
2122-# Copyright 2016 Canonical Limited.
2123-#
2124-# This file is part of charm-helpers.
2125-#
2126-# charm-helpers is free software: you can redistribute it and/or modify
2127-# it under the terms of the GNU Lesser General Public License version 3 as
2128-# published by the Free Software Foundation.
2129-#
2130-# charm-helpers is distributed in the hope that it will be useful,
2131-# but WITHOUT ANY WARRANTY; without even the implied warranty of
2132-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
2133-# GNU Lesser General Public License for more details.
2134-#
2135-# You should have received a copy of the GNU Lesser General Public License
2136-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
2137-
2138-import grp
2139-import os
2140-import pwd
2141-import re
2142-
2143-from subprocess import (
2144- CalledProcessError,
2145- check_output,
2146- check_call,
2147-)
2148-from traceback import format_exc
2149-from six import string_types
2150-from stat import (
2151- S_ISGID,
2152- S_ISUID
2153-)
2154-
2155-from charmhelpers.core.hookenv import (
2156- log,
2157- DEBUG,
2158- INFO,
2159- WARNING,
2160- ERROR,
2161-)
2162-from charmhelpers.core import unitdata
2163-from charmhelpers.core.host import file_hash
2164-from charmhelpers.contrib.hardening.audits import BaseAudit
2165-from charmhelpers.contrib.hardening.templating import (
2166- get_template_path,
2167- render_and_write,
2168-)
2169-from charmhelpers.contrib.hardening import utils
2170-
2171-
2172-class BaseFileAudit(BaseAudit):
2173- """Base class for file audits.
2174-
2175- Provides api stubs for compliance check flow that must be used by any class
2176- that implemented this one.
2177- """
2178-
2179- def __init__(self, paths, always_comply=False, *args, **kwargs):
2180- """
2181- :param paths: string path of list of paths of files we want to apply
2182- compliance checks are criteria to.
2183- :param always_comply: if true compliance criteria is always applied
2184- else compliance is skipped for non-existent
2185- paths.
2186- """
2187- super(BaseFileAudit, self).__init__(*args, **kwargs)
2188- self.always_comply = always_comply
2189- if isinstance(paths, string_types) or not hasattr(paths, '__iter__'):
2190- self.paths = [paths]
2191- else:
2192- self.paths = paths
2193-
2194- def ensure_compliance(self):
2195- """Ensure that the all registered files comply to registered criteria.
2196- """
2197- for p in self.paths:
2198- if os.path.exists(p):
2199- if self.is_compliant(p):
2200- continue
2201-
2202- log('File %s is not in compliance.' % p, level=INFO)
2203- else:
2204- if not self.always_comply:
2205- log("Non-existent path '%s' - skipping compliance check"
2206- % (p), level=INFO)
2207- continue
2208-
2209- if self._take_action():
2210- log("Applying compliance criteria to '%s'" % (p), level=INFO)
2211- self.comply(p)
2212-
2213- def is_compliant(self, path):
2214- """Audits the path to see if it is compliance.
2215-
2216- :param path: the path to the file that should be checked.
2217- """
2218- raise NotImplementedError
2219-
2220- def comply(self, path):
2221- """Enforces the compliance of a path.
2222-
2223- :param path: the path to the file that should be enforced.
2224- """
2225- raise NotImplementedError
2226-
2227- @classmethod
2228- def _get_stat(cls, path):
2229- """Returns the Posix st_stat information for the specified file path.
2230-
2231- :param path: the path to get the st_stat information for.
2232- :returns: an st_stat object for the path or None if the path doesn't
2233- exist.
2234- """
2235- return os.stat(path)
2236-
2237-
2238-class FilePermissionAudit(BaseFileAudit):
2239- """Implements an audit for file permissions and ownership for a user.
2240-
2241- This class implements functionality that ensures that a specific user/group
2242- will own the file(s) specified and that the permissions specified are
2243- applied properly to the file.
2244- """
2245- def __init__(self, paths, user, group=None, mode=0o600, **kwargs):
2246- self.user = user
2247- self.group = group
2248- self.mode = mode
2249- super(FilePermissionAudit, self).__init__(paths, user, group, mode,
2250- **kwargs)
2251-
2252- @property
2253- def user(self):
2254- return self._user
2255-
2256- @user.setter
2257- def user(self, name):
2258- try:
2259- user = pwd.getpwnam(name)
2260- except KeyError:
2261- log('Unknown user %s' % name, level=ERROR)
2262- user = None
2263- self._user = user
2264-
2265- @property
2266- def group(self):
2267- return self._group
2268-
2269- @group.setter
2270- def group(self, name):
2271- try:
2272- group = None
2273- if name:
2274- group = grp.getgrnam(name)
2275- else:
2276- group = grp.getgrgid(self.user.pw_gid)
2277- except KeyError:
2278- log('Unknown group %s' % name, level=ERROR)
2279- self._group = group
2280-
2281- def is_compliant(self, path):
2282- """Checks if the path is in compliance.
2283-
2284- Used to determine if the path specified meets the necessary
2285- requirements to be in compliance with the check itself.
2286-
2287- :param path: the file path to check
2288- :returns: True if the path is compliant, False otherwise.
2289- """
2290- stat = self._get_stat(path)
2291- user = self.user
2292- group = self.group
2293-
2294- compliant = True
2295- if stat.st_uid != user.pw_uid or stat.st_gid != group.gr_gid:
2296- log('File %s is not owned by %s:%s.' % (path, user.pw_name,
2297- group.gr_name),
2298- level=INFO)
2299- compliant = False
2300-
2301- # POSIX refers to the st_mode bits as corresponding to both the
2302- # file type and file permission bits, where the least significant 12
2303- # bits (o7777) are the suid (11), sgid (10), sticky bits (9), and the
2304- # file permission bits (8-0)
2305- perms = stat.st_mode & 0o7777
2306- if perms != self.mode:
2307- log('File %s has incorrect permissions, currently set to %s' %
2308- (path, oct(stat.st_mode & 0o7777)), level=INFO)
2309- compliant = False
2310-
2311- return compliant
2312-
2313- def comply(self, path):
2314- """Issues a chown and chmod to the file paths specified."""
2315- utils.ensure_permissions(path, self.user.pw_name, self.group.gr_name,
2316- self.mode)
2317-
2318-
2319-class DirectoryPermissionAudit(FilePermissionAudit):
2320- """Performs a permission check for the specified directory path."""
2321-
2322- def __init__(self, paths, user, group=None, mode=0o600,
2323- recursive=True, **kwargs):
2324- super(DirectoryPermissionAudit, self).__init__(paths, user, group,
2325- mode, **kwargs)
2326- self.recursive = recursive
2327-
2328- def is_compliant(self, path):
2329- """Checks if the directory is compliant.
2330-
2331- Used to determine if the path specified and all of its children
2332- directories are in compliance with the check itself.
2333-
2334- :param path: the directory path to check
2335- :returns: True if the directory tree is compliant, otherwise False.
2336- """
2337- if not os.path.isdir(path):
2338- log('Path specified %s is not a directory.' % path, level=ERROR)
2339- raise ValueError("%s is not a directory." % path)
2340-
2341- if not self.recursive:
2342- return super(DirectoryPermissionAudit, self).is_compliant(path)
2343-
2344- compliant = True
2345- for root, dirs, _ in os.walk(path):
2346- if len(dirs) > 0:
2347- continue
2348-
2349- if not super(DirectoryPermissionAudit, self).is_compliant(root):
2350- compliant = False
2351- continue
2352-
2353- return compliant
2354-
2355- def comply(self, path):
2356- for root, dirs, _ in os.walk(path):
2357- if len(dirs) > 0:
2358- super(DirectoryPermissionAudit, self).comply(root)
2359-
2360-
2361-class ReadOnly(BaseFileAudit):
2362- """Audits that files and folders are read only."""
2363- def __init__(self, paths, *args, **kwargs):
2364- super(ReadOnly, self).__init__(paths=paths, *args, **kwargs)
2365-
2366- def is_compliant(self, path):
2367- try:
2368- output = check_output(['find', path, '-perm', '-go+w',
2369- '-type', 'f']).strip()
2370-
2371- # The find above will find any files which have permission sets
2372- # which allow too broad of write access. As such, the path is
2373- # compliant if there is no output.
2374- if output:
2375- return False
2376-
2377- return True
2378- except CalledProcessError as e:
2379- log('Error occurred checking finding writable files for %s. '
2380- 'Error information is: command %s failed with returncode '
2381- '%d and output %s.\n%s' % (path, e.cmd, e.returncode, e.output,
2382- format_exc(e)), level=ERROR)
2383- return False
2384-
2385- def comply(self, path):
2386- try:
2387- check_output(['chmod', 'go-w', '-R', path])
2388- except CalledProcessError as e:
2389- log('Error occurred removing writeable permissions for %s. '
2390- 'Error information is: command %s failed with returncode '
2391- '%d and output %s.\n%s' % (path, e.cmd, e.returncode, e.output,
2392- format_exc(e)), level=ERROR)
2393-
2394-
2395-class NoReadWriteForOther(BaseFileAudit):
2396- """Ensures that the files found under the base path are readable or
2397- writable by anyone other than the owner or the group.
2398- """
2399- def __init__(self, paths):
2400- super(NoReadWriteForOther, self).__init__(paths)
2401-
2402- def is_compliant(self, path):
2403- try:
2404- cmd = ['find', path, '-perm', '-o+r', '-type', 'f', '-o',
2405- '-perm', '-o+w', '-type', 'f']
2406- output = check_output(cmd).strip()
2407-
2408- # The find above here will find any files which have read or
2409- # write permissions for other, meaning there is too broad of access
2410- # to read/write the file. As such, the path is compliant if there's
2411- # no output.
2412- if output:
2413- return False
2414-
2415- return True
2416- except CalledProcessError as e:
2417- log('Error occurred while finding files which are readable or '
2418- 'writable to the world in %s. '
2419- 'Command output is: %s.' % (path, e.output), level=ERROR)
2420-
2421- def comply(self, path):
2422- try:
2423- check_output(['chmod', '-R', 'o-rw', path])
2424- except CalledProcessError as e:
2425- log('Error occurred attempting to change modes of files under '
2426- 'path %s. Output of command is: %s' % (path, e.output))
2427-
2428-
2429-class NoSUIDSGIDAudit(BaseFileAudit):
2430- """Audits that specified files do not have SUID/SGID bits set."""
2431- def __init__(self, paths, *args, **kwargs):
2432- super(NoSUIDSGIDAudit, self).__init__(paths=paths, *args, **kwargs)
2433-
2434- def is_compliant(self, path):
2435- stat = self._get_stat(path)
2436- if (stat.st_mode & (S_ISGID | S_ISUID)) != 0:
2437- return False
2438-
2439- return True
2440-
2441- def comply(self, path):
2442- try:
2443- log('Removing suid/sgid from %s.' % path, level=DEBUG)
2444- check_output(['chmod', '-s', path])
2445- except CalledProcessError as e:
2446- log('Error occurred removing suid/sgid from %s.'
2447- 'Error information is: command %s failed with returncode '
2448- '%d and output %s.\n%s' % (path, e.cmd, e.returncode, e.output,
2449- format_exc(e)), level=ERROR)
2450-
2451-
2452-class TemplatedFile(BaseFileAudit):
2453- """The TemplatedFileAudit audits the contents of a templated file.
2454-
2455- This audit renders a file from a template, sets the appropriate file
2456- permissions, then generates a hashsum with which to check the content
2457- changed.
2458- """
2459- def __init__(self, path, context, template_dir, mode, user='root',
2460- group='root', service_actions=None, **kwargs):
2461- self.context = context
2462- self.user = user
2463- self.group = group
2464- self.mode = mode
2465- self.template_dir = template_dir
2466- self.service_actions = service_actions
2467- super(TemplatedFile, self).__init__(paths=path, always_comply=True,
2468- **kwargs)
2469-
2470- def is_compliant(self, path):
2471- """Determines if the templated file is compliant.
2472-
2473- A templated file is only compliant if it has not changed (as
2474- determined by its sha256 hashsum) AND its file permissions are set
2475- appropriately.
2476-
2477- :param path: the path to check compliance.
2478- """
2479- same_templates = self.templates_match(path)
2480- same_content = self.contents_match(path)
2481- same_permissions = self.permissions_match(path)
2482-
2483- if same_content and same_permissions and same_templates:
2484- return True
2485-
2486- return False
2487-
2488- def run_service_actions(self):
2489- """Run any actions on services requested."""
2490- if not self.service_actions:
2491- return
2492-
2493- for svc_action in self.service_actions:
2494- name = svc_action['service']
2495- actions = svc_action['actions']
2496- log("Running service '%s' actions '%s'" % (name, actions),
2497- level=DEBUG)
2498- for action in actions:
2499- cmd = ['service', name, action]
2500- try:
2501- check_call(cmd)
2502- except CalledProcessError as exc:
2503- log("Service name='%s' action='%s' failed - %s" %
2504- (name, action, exc), level=WARNING)
2505-
2506- def comply(self, path):
2507- """Ensures the contents and the permissions of the file.
2508-
2509- :param path: the path to correct
2510- """
2511- dirname = os.path.dirname(path)
2512- if not os.path.exists(dirname):
2513- os.makedirs(dirname)
2514-
2515- self.pre_write()
2516- render_and_write(self.template_dir, path, self.context())
2517- utils.ensure_permissions(path, self.user, self.group, self.mode)
2518- self.run_service_actions()
2519- self.save_checksum(path)
2520- self.post_write()
2521-
2522- def pre_write(self):
2523- """Invoked prior to writing the template."""
2524- pass
2525-
2526- def post_write(self):
2527- """Invoked after writing the template."""
2528- pass
2529-
2530- def templates_match(self, path):
2531- """Determines if the template files are the same.
2532-
2533- The template file equality is determined by the hashsum of the
2534- template files themselves. If there is no hashsum, then the content
2535- cannot be sure to be the same so treat it as if they changed.
2536- Otherwise, return whether or not the hashsums are the same.
2537-
2538- :param path: the path to check
2539- :returns: boolean
2540- """
2541- template_path = get_template_path(self.template_dir, path)
2542- key = 'hardening:template:%s' % template_path
2543- template_checksum = file_hash(template_path)
2544- kv = unitdata.kv()
2545- stored_tmplt_checksum = kv.get(key)
2546- if not stored_tmplt_checksum:
2547- kv.set(key, template_checksum)
2548- kv.flush()
2549- log('Saved template checksum for %s.' % template_path,
2550- level=DEBUG)
2551- # Since we don't have a template checksum, then assume it doesn't
2552- # match and return that the template is different.
2553- return False
2554- elif stored_tmplt_checksum != template_checksum:
2555- kv.set(key, template_checksum)
2556- kv.flush()
2557- log('Updated template checksum for %s.' % template_path,
2558- level=DEBUG)
2559- return False
2560-
2561- # Here the template hasn't changed based upon the calculated
2562- # checksum of the template and what was previously stored.
2563- return True
2564-
2565- def contents_match(self, path):
2566- """Determines if the file content is the same.
2567-
2568- This is determined by comparing hashsum of the file contents and
2569- the saved hashsum. If there is no hashsum, then the content cannot
2570- be sure to be the same so treat them as if they are not the same.
2571- Otherwise, return True if the hashsums are the same, False if they
2572- are not the same.
2573-
2574- :param path: the file to check.
2575- """
2576- checksum = file_hash(path)
2577-
2578- kv = unitdata.kv()
2579- stored_checksum = kv.get('hardening:%s' % path)
2580- if not stored_checksum:
2581- # If the checksum hasn't been generated, return False to ensure
2582- # the file is written and the checksum stored.
2583- log('Checksum for %s has not been calculated.' % path, level=DEBUG)
2584- return False
2585- elif stored_checksum != checksum:
2586- log('Checksum mismatch for %s.' % path, level=DEBUG)
2587- return False
2588-
2589- return True
2590-
2591- def permissions_match(self, path):
2592- """Determines if the file owner and permissions match.
2593-
2594- :param path: the path to check.
2595- """
2596- audit = FilePermissionAudit(path, self.user, self.group, self.mode)
2597- return audit.is_compliant(path)
2598-
2599- def save_checksum(self, path):
2600- """Calculates and saves the checksum for the path specified.
2601-
2602- :param path: the path of the file to save the checksum.
2603- """
2604- checksum = file_hash(path)
2605- kv = unitdata.kv()
2606- kv.set('hardening:%s' % path, checksum)
2607- kv.flush()
2608-
2609-
2610-class DeletedFile(BaseFileAudit):
2611- """Audit to ensure that a file is deleted."""
2612- def __init__(self, paths):
2613- super(DeletedFile, self).__init__(paths)
2614-
2615- def is_compliant(self, path):
2616- return not os.path.exists(path)
2617-
2618- def comply(self, path):
2619- os.remove(path)
2620-
2621-
2622-class FileContentAudit(BaseFileAudit):
2623- """Audit the contents of a file."""
2624- def __init__(self, paths, cases, **kwargs):
2625- # Cases we expect to pass
2626- self.pass_cases = cases.get('pass', [])
2627- # Cases we expect to fail
2628- self.fail_cases = cases.get('fail', [])
2629- super(FileContentAudit, self).__init__(paths, **kwargs)
2630-
2631- def is_compliant(self, path):
2632- """
2633- Given a set of content matching cases i.e. tuple(regex, bool) where
2634- bool value denotes whether or not regex is expected to match, check that
2635- all cases match as expected with the contents of the file. Cases can be
2636- expected to pass of fail.
2637-
2638- :param path: Path of file to check.
2639- :returns: Boolean value representing whether or not all cases are
2640- found to be compliant.
2641- """
2642- log("Auditing contents of file '%s'" % (path), level=DEBUG)
2643- with open(path, 'r') as fd:
2644- contents = fd.read()
2645-
2646- matches = 0
2647- for pattern in self.pass_cases:
2648- key = re.compile(pattern, flags=re.MULTILINE)
2649- results = re.search(key, contents)
2650- if results:
2651- matches += 1
2652- else:
2653- log("Pattern '%s' was expected to pass but instead it failed"
2654- % (pattern), level=WARNING)
2655-
2656- for pattern in self.fail_cases:
2657- key = re.compile(pattern, flags=re.MULTILINE)
2658- results = re.search(key, contents)
2659- if not results:
2660- matches += 1
2661- else:
2662- log("Pattern '%s' was expected to fail but instead it passed"
2663- % (pattern), level=WARNING)
2664-
2665- total = len(self.pass_cases) + len(self.fail_cases)
2666- log("Checked %s cases and %s passed" % (total, matches), level=DEBUG)
2667- return matches == total
2668-
2669- def comply(self, *args, **kwargs):
2670- """NOOP since we just issue warnings. This is to avoid the
2671- NotImplememtedError.
2672- """
2673- log("Not applying any compliance criteria, only checks.", level=INFO)
2674
2675=== removed file 'hooks/charmhelpers/contrib/hardening/harden.py'
2676--- hooks/charmhelpers/contrib/hardening/harden.py 2016-04-22 08:15:33 +0000
2677+++ hooks/charmhelpers/contrib/hardening/harden.py 1970-01-01 00:00:00 +0000
2678@@ -1,84 +0,0 @@
2679-# Copyright 2016 Canonical Limited.
2680-#
2681-# This file is part of charm-helpers.
2682-#
2683-# charm-helpers is free software: you can redistribute it and/or modify
2684-# it under the terms of the GNU Lesser General Public License version 3 as
2685-# published by the Free Software Foundation.
2686-#
2687-# charm-helpers is distributed in the hope that it will be useful,
2688-# but WITHOUT ANY WARRANTY; without even the implied warranty of
2689-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
2690-# GNU Lesser General Public License for more details.
2691-#
2692-# You should have received a copy of the GNU Lesser General Public License
2693-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
2694-
2695-import six
2696-
2697-from collections import OrderedDict
2698-
2699-from charmhelpers.core.hookenv import (
2700- config,
2701- log,
2702- DEBUG,
2703- WARNING,
2704-)
2705-from charmhelpers.contrib.hardening.host.checks import run_os_checks
2706-from charmhelpers.contrib.hardening.ssh.checks import run_ssh_checks
2707-from charmhelpers.contrib.hardening.mysql.checks import run_mysql_checks
2708-from charmhelpers.contrib.hardening.apache.checks import run_apache_checks
2709-
2710-
2711-def harden(overrides=None):
2712- """Hardening decorator.
2713-
2714- This is the main entry point for running the hardening stack. In order to
2715- run modules of the stack you must add this decorator to charm hook(s) and
2716- ensure that your charm config.yaml contains the 'harden' option set to
2717- one or more of the supported modules. Setting these will cause the
2718- corresponding hardening code to be run when the hook fires.
2719-
2720- This decorator can and should be applied to more than one hook or function
2721- such that hardening modules are called multiple times. This is because
2722- subsequent calls will perform auditing checks that will report any changes
2723- to resources hardened by the first run (and possibly perform compliance
2724- actions as a result of any detected infractions).
2725-
2726- :param overrides: Optional list of stack modules used to override those
2727- provided with 'harden' config.
2728- :returns: Returns value returned by decorated function once executed.
2729- """
2730- def _harden_inner1(f):
2731- log("Hardening function '%s'" % (f.__name__), level=DEBUG)
2732-
2733- def _harden_inner2(*args, **kwargs):
2734- RUN_CATALOG = OrderedDict([('os', run_os_checks),
2735- ('ssh', run_ssh_checks),
2736- ('mysql', run_mysql_checks),
2737- ('apache', run_apache_checks)])
2738-
2739- enabled = overrides or (config("harden") or "").split()
2740- if enabled:
2741- modules_to_run = []
2742- # modules will always be performed in the following order
2743- for module, func in six.iteritems(RUN_CATALOG):
2744- if module in enabled:
2745- enabled.remove(module)
2746- modules_to_run.append(func)
2747-
2748- if enabled:
2749- log("Unknown hardening modules '%s' - ignoring" %
2750- (', '.join(enabled)), level=WARNING)
2751-
2752- for hardener in modules_to_run:
2753- log("Executing hardening module '%s'" %
2754- (hardener.__name__), level=DEBUG)
2755- hardener()
2756- else:
2757- log("No hardening applied to '%s'" % (f.__name__), level=DEBUG)
2758-
2759- return f(*args, **kwargs)
2760- return _harden_inner2
2761-
2762- return _harden_inner1
2763
2764=== removed directory 'hooks/charmhelpers/contrib/hardening/host'
2765=== removed file 'hooks/charmhelpers/contrib/hardening/host/__init__.py'
2766--- hooks/charmhelpers/contrib/hardening/host/__init__.py 2016-04-22 08:15:33 +0000
2767+++ hooks/charmhelpers/contrib/hardening/host/__init__.py 1970-01-01 00:00:00 +0000
2768@@ -1,19 +0,0 @@
2769-# Copyright 2016 Canonical Limited.
2770-#
2771-# This file is part of charm-helpers.
2772-#
2773-# charm-helpers is free software: you can redistribute it and/or modify
2774-# it under the terms of the GNU Lesser General Public License version 3 as
2775-# published by the Free Software Foundation.
2776-#
2777-# charm-helpers is distributed in the hope that it will be useful,
2778-# but WITHOUT ANY WARRANTY; without even the implied warranty of
2779-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
2780-# GNU Lesser General Public License for more details.
2781-#
2782-# You should have received a copy of the GNU Lesser General Public License
2783-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
2784-
2785-from os import path
2786-
2787-TEMPLATES_DIR = path.join(path.dirname(__file__), 'templates')
2788
2789=== removed directory 'hooks/charmhelpers/contrib/hardening/host/checks'
2790=== removed file 'hooks/charmhelpers/contrib/hardening/host/checks/__init__.py'
2791--- hooks/charmhelpers/contrib/hardening/host/checks/__init__.py 2016-04-22 08:15:33 +0000
2792+++ hooks/charmhelpers/contrib/hardening/host/checks/__init__.py 1970-01-01 00:00:00 +0000
2793@@ -1,50 +0,0 @@
2794-# Copyright 2016 Canonical Limited.
2795-#
2796-# This file is part of charm-helpers.
2797-#
2798-# charm-helpers is free software: you can redistribute it and/or modify
2799-# it under the terms of the GNU Lesser General Public License version 3 as
2800-# published by the Free Software Foundation.
2801-#
2802-# charm-helpers is distributed in the hope that it will be useful,
2803-# but WITHOUT ANY WARRANTY; without even the implied warranty of
2804-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
2805-# GNU Lesser General Public License for more details.
2806-#
2807-# You should have received a copy of the GNU Lesser General Public License
2808-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
2809-
2810-from charmhelpers.core.hookenv import (
2811- log,
2812- DEBUG,
2813-)
2814-from charmhelpers.contrib.hardening.host.checks import (
2815- apt,
2816- limits,
2817- login,
2818- minimize_access,
2819- pam,
2820- profile,
2821- securetty,
2822- suid_sgid,
2823- sysctl
2824-)
2825-
2826-
2827-def run_os_checks():
2828- log("Starting OS hardening checks.", level=DEBUG)
2829- checks = apt.get_audits()
2830- checks.extend(limits.get_audits())
2831- checks.extend(login.get_audits())
2832- checks.extend(minimize_access.get_audits())
2833- checks.extend(pam.get_audits())
2834- checks.extend(profile.get_audits())
2835- checks.extend(securetty.get_audits())
2836- checks.extend(suid_sgid.get_audits())
2837- checks.extend(sysctl.get_audits())
2838-
2839- for check in checks:
2840- log("Running '%s' check" % (check.__class__.__name__), level=DEBUG)
2841- check.ensure_compliance()
2842-
2843- log("OS hardening checks complete.", level=DEBUG)
2844
2845=== removed file 'hooks/charmhelpers/contrib/hardening/host/checks/apt.py'
2846--- hooks/charmhelpers/contrib/hardening/host/checks/apt.py 2016-04-22 08:15:33 +0000
2847+++ hooks/charmhelpers/contrib/hardening/host/checks/apt.py 1970-01-01 00:00:00 +0000
2848@@ -1,39 +0,0 @@
2849-# Copyright 2016 Canonical Limited.
2850-#
2851-# This file is part of charm-helpers.
2852-#
2853-# charm-helpers is free software: you can redistribute it and/or modify
2854-# it under the terms of the GNU Lesser General Public License version 3 as
2855-# published by the Free Software Foundation.
2856-#
2857-# charm-helpers is distributed in the hope that it will be useful,
2858-# but WITHOUT ANY WARRANTY; without even the implied warranty of
2859-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
2860-# GNU Lesser General Public License for more details.
2861-#
2862-# You should have received a copy of the GNU Lesser General Public License
2863-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
2864-
2865-from charmhelpers.contrib.hardening.utils import get_settings
2866-from charmhelpers.contrib.hardening.audits.apt import (
2867- AptConfig,
2868- RestrictedPackages,
2869-)
2870-
2871-
2872-def get_audits():
2873- """Get OS hardening apt audits.
2874-
2875- :returns: dictionary of audits
2876- """
2877- audits = [AptConfig([{'key': 'APT::Get::AllowUnauthenticated',
2878- 'expected': 'false'}])]
2879-
2880- settings = get_settings('os')
2881- clean_packages = settings['security']['packages_clean']
2882- if clean_packages:
2883- security_packages = settings['security']['packages_list']
2884- if security_packages:
2885- audits.append(RestrictedPackages(security_packages))
2886-
2887- return audits
2888
2889=== removed file 'hooks/charmhelpers/contrib/hardening/host/checks/limits.py'
2890--- hooks/charmhelpers/contrib/hardening/host/checks/limits.py 2016-04-22 08:15:33 +0000
2891+++ hooks/charmhelpers/contrib/hardening/host/checks/limits.py 1970-01-01 00:00:00 +0000
2892@@ -1,55 +0,0 @@
2893-# Copyright 2016 Canonical Limited.
2894-#
2895-# This file is part of charm-helpers.
2896-#
2897-# charm-helpers is free software: you can redistribute it and/or modify
2898-# it under the terms of the GNU Lesser General Public License version 3 as
2899-# published by the Free Software Foundation.
2900-#
2901-# charm-helpers is distributed in the hope that it will be useful,
2902-# but WITHOUT ANY WARRANTY; without even the implied warranty of
2903-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
2904-# GNU Lesser General Public License for more details.
2905-#
2906-# You should have received a copy of the GNU Lesser General Public License
2907-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
2908-
2909-from charmhelpers.contrib.hardening.audits.file import (
2910- DirectoryPermissionAudit,
2911- TemplatedFile,
2912-)
2913-from charmhelpers.contrib.hardening.host import TEMPLATES_DIR
2914-from charmhelpers.contrib.hardening import utils
2915-
2916-
2917-def get_audits():
2918- """Get OS hardening security limits audits.
2919-
2920- :returns: dictionary of audits
2921- """
2922- audits = []
2923- settings = utils.get_settings('os')
2924-
2925- # Ensure that the /etc/security/limits.d directory is only writable
2926- # by the root user, but others can execute and read.
2927- audits.append(DirectoryPermissionAudit('/etc/security/limits.d',
2928- user='root', group='root',
2929- mode=0o755))
2930-
2931- # If core dumps are not enabled, then don't allow core dumps to be
2932- # created as they may contain sensitive information.
2933- if not settings['security']['kernel_enable_core_dump']:
2934- audits.append(TemplatedFile('/etc/security/limits.d/10.hardcore.conf',
2935- SecurityLimitsContext(),
2936- template_dir=TEMPLATES_DIR,
2937- user='root', group='root', mode=0o0440))
2938- return audits
2939-
2940-
2941-class SecurityLimitsContext(object):
2942-
2943- def __call__(self):
2944- settings = utils.get_settings('os')
2945- ctxt = {'disable_core_dump':
2946- not settings['security']['kernel_enable_core_dump']}
2947- return ctxt
2948
2949=== removed file 'hooks/charmhelpers/contrib/hardening/host/checks/login.py'
2950--- hooks/charmhelpers/contrib/hardening/host/checks/login.py 2016-04-22 08:15:33 +0000
2951+++ hooks/charmhelpers/contrib/hardening/host/checks/login.py 1970-01-01 00:00:00 +0000
2952@@ -1,67 +0,0 @@
2953-# Copyright 2016 Canonical Limited.
2954-#
2955-# This file is part of charm-helpers.
2956-#
2957-# charm-helpers is free software: you can redistribute it and/or modify
2958-# it under the terms of the GNU Lesser General Public License version 3 as
2959-# published by the Free Software Foundation.
2960-#
2961-# charm-helpers is distributed in the hope that it will be useful,
2962-# but WITHOUT ANY WARRANTY; without even the implied warranty of
2963-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
2964-# GNU Lesser General Public License for more details.
2965-#
2966-# You should have received a copy of the GNU Lesser General Public License
2967-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
2968-
2969-from six import string_types
2970-
2971-from charmhelpers.contrib.hardening.audits.file import TemplatedFile
2972-from charmhelpers.contrib.hardening.host import TEMPLATES_DIR
2973-from charmhelpers.contrib.hardening import utils
2974-
2975-
2976-def get_audits():
2977- """Get OS hardening login.defs audits.
2978-
2979- :returns: dictionary of audits
2980- """
2981- audits = [TemplatedFile('/etc/login.defs', LoginContext(),
2982- template_dir=TEMPLATES_DIR,
2983- user='root', group='root', mode=0o0444)]
2984- return audits
2985-
2986-
2987-class LoginContext(object):
2988-
2989- def __call__(self):
2990- settings = utils.get_settings('os')
2991-
2992- # Octal numbers in yaml end up being turned into decimal,
2993- # so check if the umask is entered as a string (e.g. '027')
2994- # or as an octal umask as we know it (e.g. 002). If its not
2995- # a string assume it to be octal and turn it into an octal
2996- # string.
2997- umask = settings['environment']['umask']
2998- if not isinstance(umask, string_types):
2999- umask = '%s' % oct(umask)
3000-
3001- ctxt = {
3002- 'additional_user_paths':
3003- settings['environment']['extra_user_paths'],
3004- 'umask': umask,
3005- 'pwd_max_age': settings['auth']['pw_max_age'],
3006- 'pwd_min_age': settings['auth']['pw_min_age'],
3007- 'uid_min': settings['auth']['uid_min'],
3008- 'sys_uid_min': settings['auth']['sys_uid_min'],
3009- 'sys_uid_max': settings['auth']['sys_uid_max'],
3010- 'gid_min': settings['auth']['gid_min'],
3011- 'sys_gid_min': settings['auth']['sys_gid_min'],
3012- 'sys_gid_max': settings['auth']['sys_gid_max'],
3013- 'login_retries': settings['auth']['retries'],
3014- 'login_timeout': settings['auth']['timeout'],
3015- 'chfn_restrict': settings['auth']['chfn_restrict'],
3016- 'allow_login_without_home': settings['auth']['allow_homeless']
3017- }
3018-
3019- return ctxt
3020
3021=== removed file 'hooks/charmhelpers/contrib/hardening/host/checks/minimize_access.py'
3022--- hooks/charmhelpers/contrib/hardening/host/checks/minimize_access.py 2016-04-22 08:15:33 +0000
3023+++ hooks/charmhelpers/contrib/hardening/host/checks/minimize_access.py 1970-01-01 00:00:00 +0000
3024@@ -1,52 +0,0 @@
3025-# Copyright 2016 Canonical Limited.
3026-#
3027-# This file is part of charm-helpers.
3028-#
3029-# charm-helpers is free software: you can redistribute it and/or modify
3030-# it under the terms of the GNU Lesser General Public License version 3 as
3031-# published by the Free Software Foundation.
3032-#
3033-# charm-helpers is distributed in the hope that it will be useful,
3034-# but WITHOUT ANY WARRANTY; without even the implied warranty of
3035-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
3036-# GNU Lesser General Public License for more details.
3037-#
3038-# You should have received a copy of the GNU Lesser General Public License
3039-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
3040-
3041-from charmhelpers.contrib.hardening.audits.file import (
3042- FilePermissionAudit,
3043- ReadOnly,
3044-)
3045-from charmhelpers.contrib.hardening import utils
3046-
3047-
3048-def get_audits():
3049- """Get OS hardening access audits.
3050-
3051- :returns: dictionary of audits
3052- """
3053- audits = []
3054- settings = utils.get_settings('os')
3055-
3056- # Remove write permissions from $PATH folders for all regular users.
3057- # This prevents changing system-wide commands from normal users.
3058- path_folders = {'/usr/local/sbin',
3059- '/usr/local/bin',
3060- '/usr/sbin',
3061- '/usr/bin',
3062- '/bin'}
3063- extra_user_paths = settings['environment']['extra_user_paths']
3064- path_folders.update(extra_user_paths)
3065- audits.append(ReadOnly(path_folders))
3066-
3067- # Only allow the root user to have access to the shadow file.
3068- audits.append(FilePermissionAudit('/etc/shadow', 'root', 'root', 0o0600))
3069-
3070- if 'change_user' not in settings['security']['users_allow']:
3071- # su should only be accessible to user and group root, unless it is
3072- # expressly defined to allow users to change to root via the
3073- # security_users_allow config option.
3074- audits.append(FilePermissionAudit('/bin/su', 'root', 'root', 0o750))
3075-
3076- return audits
3077
3078=== removed file 'hooks/charmhelpers/contrib/hardening/host/checks/pam.py'
3079--- hooks/charmhelpers/contrib/hardening/host/checks/pam.py 2016-04-22 08:15:33 +0000
3080+++ hooks/charmhelpers/contrib/hardening/host/checks/pam.py 1970-01-01 00:00:00 +0000
3081@@ -1,134 +0,0 @@
3082-# Copyright 2016 Canonical Limited.
3083-#
3084-# This file is part of charm-helpers.
3085-#
3086-# charm-helpers is free software: you can redistribute it and/or modify
3087-# it under the terms of the GNU Lesser General Public License version 3 as
3088-# published by the Free Software Foundation.
3089-#
3090-# charm-helpers is distributed in the hope that it will be useful,
3091-# but WITHOUT ANY WARRANTY; without even the implied warranty of
3092-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
3093-# GNU Lesser General Public License for more details.
3094-#
3095-# You should have received a copy of the GNU Lesser General Public License
3096-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
3097-
3098-from subprocess import (
3099- check_output,
3100- CalledProcessError,
3101-)
3102-
3103-from charmhelpers.core.hookenv import (
3104- log,
3105- DEBUG,
3106- ERROR,
3107-)
3108-from charmhelpers.fetch import (
3109- apt_install,
3110- apt_purge,
3111- apt_update,
3112-)
3113-from charmhelpers.contrib.hardening.audits.file import (
3114- TemplatedFile,
3115- DeletedFile,
3116-)
3117-from charmhelpers.contrib.hardening import utils
3118-from charmhelpers.contrib.hardening.host import TEMPLATES_DIR
3119-
3120-
3121-def get_audits():
3122- """Get OS hardening PAM authentication audits.
3123-
3124- :returns: dictionary of audits
3125- """
3126- audits = []
3127-
3128- settings = utils.get_settings('os')
3129-
3130- if settings['auth']['pam_passwdqc_enable']:
3131- audits.append(PasswdqcPAM('/etc/passwdqc.conf'))
3132-
3133- if settings['auth']['retries']:
3134- audits.append(Tally2PAM('/usr/share/pam-configs/tally2'))
3135- else:
3136- audits.append(DeletedFile('/usr/share/pam-configs/tally2'))
3137-
3138- return audits
3139-
3140-
3141-class PasswdqcPAMContext(object):
3142-
3143- def __call__(self):
3144- ctxt = {}
3145- settings = utils.get_settings('os')
3146-
3147- ctxt['auth_pam_passwdqc_options'] = \
3148- settings['auth']['pam_passwdqc_options']
3149-
3150- return ctxt
3151-
3152-
3153-class PasswdqcPAM(TemplatedFile):
3154- """The PAM Audit verifies the linux PAM settings."""
3155- def __init__(self, path):
3156- super(PasswdqcPAM, self).__init__(path=path,
3157- template_dir=TEMPLATES_DIR,
3158- context=PasswdqcPAMContext(),
3159- user='root',
3160- group='root',
3161- mode=0o0640)
3162-
3163- def pre_write(self):
3164- # Always remove?
3165- for pkg in ['libpam-ccreds', 'libpam-cracklib']:
3166- log("Purging package '%s'" % pkg, level=DEBUG),
3167- apt_purge(pkg)
3168-
3169- apt_update(fatal=True)
3170- for pkg in ['libpam-passwdqc']:
3171- log("Installing package '%s'" % pkg, level=DEBUG),
3172- apt_install(pkg)
3173-
3174- def post_write(self):
3175- """Updates the PAM configuration after the file has been written"""
3176- try:
3177- check_output(['pam-auth-update', '--package'])
3178- except CalledProcessError as e:
3179- log('Error calling pam-auth-update: %s' % e, level=ERROR)
3180-
3181-
3182-class Tally2PAMContext(object):
3183-
3184- def __call__(self):
3185- ctxt = {}
3186- settings = utils.get_settings('os')
3187-
3188- ctxt['auth_lockout_time'] = settings['auth']['lockout_time']
3189- ctxt['auth_retries'] = settings['auth']['retries']
3190-
3191- return ctxt
3192-
3193-
3194-class Tally2PAM(TemplatedFile):
3195- """The PAM Audit verifies the linux PAM settings."""
3196- def __init__(self, path):
3197- super(Tally2PAM, self).__init__(path=path,
3198- template_dir=TEMPLATES_DIR,
3199- context=Tally2PAMContext(),
3200- user='root',
3201- group='root',
3202- mode=0o0640)
3203-
3204- def pre_write(self):
3205- # Always remove?
3206- apt_purge('libpam-ccreds')
3207- apt_update(fatal=True)
3208- apt_install('libpam-modules')
3209-
3210- def post_write(self):
3211- """Updates the PAM configuration after the file has been written"""
3212- try:
3213- check_output(['pam-auth-update', '--package'])
3214- except CalledProcessError as e:
3215- log('Error calling pam-auth-update: %s' % e, level=ERROR)
3216
3217=== removed file 'hooks/charmhelpers/contrib/hardening/host/checks/profile.py'
3218--- hooks/charmhelpers/contrib/hardening/host/checks/profile.py 2016-04-22 08:15:33 +0000
3219+++ hooks/charmhelpers/contrib/hardening/host/checks/profile.py 1970-01-01 00:00:00 +0000
3220@@ -1,45 +0,0 @@
3221-# Copyright 2016 Canonical Limited.
3222-#
3223-# This file is part of charm-helpers.
3224-#
3225-# charm-helpers is free software: you can redistribute it and/or modify
3226-# it under the terms of the GNU Lesser General Public License version 3 as
3227-# published by the Free Software Foundation.
3228-#
3229-# charm-helpers is distributed in the hope that it will be useful,
3230-# but WITHOUT ANY WARRANTY; without even the implied warranty of
3231-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
3232-# GNU Lesser General Public License for more details.
3233-#
3234-# You should have received a copy of the GNU Lesser General Public License
3235-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
3236-
3237-from charmhelpers.contrib.hardening.audits.file import TemplatedFile
3238-from charmhelpers.contrib.hardening.host import TEMPLATES_DIR
3239-from charmhelpers.contrib.hardening import utils
3240-
3241-
3242-def get_audits():
3243- """Get OS hardening profile audits.
3244-
3245- :returns: dictionary of audits
3246- """
3247- audits = []
3248-
3249- settings = utils.get_settings('os')
3250-
3251- # If core dumps are not enabled, then don't allow core dumps to be
3252- # created as they may contain sensitive information.
3253- if not settings['security']['kernel_enable_core_dump']:
3254- audits.append(TemplatedFile('/etc/profile.d/pinerolo_profile.sh',
3255- ProfileContext(),
3256- template_dir=TEMPLATES_DIR,
3257- mode=0o0755, user='root', group='root'))
3258- return audits
3259-
3260-
3261-class ProfileContext(object):
3262-
3263- def __call__(self):
3264- ctxt = {}
3265- return ctxt
3266
3267=== removed file 'hooks/charmhelpers/contrib/hardening/host/checks/securetty.py'
3268--- hooks/charmhelpers/contrib/hardening/host/checks/securetty.py 2016-04-22 08:15:33 +0000
3269+++ hooks/charmhelpers/contrib/hardening/host/checks/securetty.py 1970-01-01 00:00:00 +0000
3270@@ -1,39 +0,0 @@
3271-# Copyright 2016 Canonical Limited.
3272-#
3273-# This file is part of charm-helpers.
3274-#
3275-# charm-helpers is free software: you can redistribute it and/or modify
3276-# it under the terms of the GNU Lesser General Public License version 3 as
3277-# published by the Free Software Foundation.
3278-#
3279-# charm-helpers is distributed in the hope that it will be useful,
3280-# but WITHOUT ANY WARRANTY; without even the implied warranty of
3281-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
3282-# GNU Lesser General Public License for more details.
3283-#
3284-# You should have received a copy of the GNU Lesser General Public License
3285-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
3286-
3287-from charmhelpers.contrib.hardening.audits.file import TemplatedFile
3288-from charmhelpers.contrib.hardening.host import TEMPLATES_DIR
3289-from charmhelpers.contrib.hardening import utils
3290-
3291-
3292-def get_audits():
3293- """Get OS hardening Secure TTY audits.
3294-
3295- :returns: dictionary of audits
3296- """
3297- audits = []
3298- audits.append(TemplatedFile('/etc/securetty', SecureTTYContext(),
3299- template_dir=TEMPLATES_DIR,
3300- mode=0o0400, user='root', group='root'))
3301- return audits
3302-
3303-
3304-class SecureTTYContext(object):
3305-
3306- def __call__(self):
3307- settings = utils.get_settings('os')
3308- ctxt = {'ttys': settings['auth']['root_ttys']}
3309- return ctxt
3310
3311=== removed file 'hooks/charmhelpers/contrib/hardening/host/checks/suid_sgid.py'
3312--- hooks/charmhelpers/contrib/hardening/host/checks/suid_sgid.py 2016-04-22 08:15:33 +0000
3313+++ hooks/charmhelpers/contrib/hardening/host/checks/suid_sgid.py 1970-01-01 00:00:00 +0000
3314@@ -1,131 +0,0 @@
3315-# Copyright 2016 Canonical Limited.
3316-#
3317-# This file is part of charm-helpers.
3318-#
3319-# charm-helpers is free software: you can redistribute it and/or modify
3320-# it under the terms of the GNU Lesser General Public License version 3 as
3321-# published by the Free Software Foundation.
3322-#
3323-# charm-helpers is distributed in the hope that it will be useful,
3324-# but WITHOUT ANY WARRANTY; without even the implied warranty of
3325-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
3326-# GNU Lesser General Public License for more details.
3327-#
3328-# You should have received a copy of the GNU Lesser General Public License
3329-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
3330-
3331-import subprocess
3332-
3333-from charmhelpers.core.hookenv import (
3334- log,
3335- INFO,
3336-)
3337-from charmhelpers.contrib.hardening.audits.file import NoSUIDSGIDAudit
3338-from charmhelpers.contrib.hardening import utils
3339-
3340-
3341-BLACKLIST = ['/usr/bin/rcp', '/usr/bin/rlogin', '/usr/bin/rsh',
3342- '/usr/libexec/openssh/ssh-keysign',
3343- '/usr/lib/openssh/ssh-keysign',
3344- '/sbin/netreport',
3345- '/usr/sbin/usernetctl',
3346- '/usr/sbin/userisdnctl',
3347- '/usr/sbin/pppd',
3348- '/usr/bin/lockfile',
3349- '/usr/bin/mail-lock',
3350- '/usr/bin/mail-unlock',
3351- '/usr/bin/mail-touchlock',
3352- '/usr/bin/dotlockfile',
3353- '/usr/bin/arping',
3354- '/usr/sbin/uuidd',
3355- '/usr/bin/mtr',
3356- '/usr/lib/evolution/camel-lock-helper-1.2',
3357- '/usr/lib/pt_chown',
3358- '/usr/lib/eject/dmcrypt-get-device',
3359- '/usr/lib/mc/cons.saver']
3360-
3361-WHITELIST = ['/bin/mount', '/bin/ping', '/bin/su', '/bin/umount',
3362- '/sbin/pam_timestamp_check', '/sbin/unix_chkpwd', '/usr/bin/at',
3363- '/usr/bin/gpasswd', '/usr/bin/locate', '/usr/bin/newgrp',
3364- '/usr/bin/passwd', '/usr/bin/ssh-agent',
3365- '/usr/libexec/utempter/utempter', '/usr/sbin/lockdev',
3366- '/usr/sbin/sendmail.sendmail', '/usr/bin/expiry',
3367- '/bin/ping6', '/usr/bin/traceroute6.iputils',
3368- '/sbin/mount.nfs', '/sbin/umount.nfs',
3369- '/sbin/mount.nfs4', '/sbin/umount.nfs4',
3370- '/usr/bin/crontab',
3371- '/usr/bin/wall', '/usr/bin/write',
3372- '/usr/bin/screen',
3373- '/usr/bin/mlocate',
3374- '/usr/bin/chage', '/usr/bin/chfn', '/usr/bin/chsh',
3375- '/bin/fusermount',
3376- '/usr/bin/pkexec',
3377- '/usr/bin/sudo', '/usr/bin/sudoedit',
3378- '/usr/sbin/postdrop', '/usr/sbin/postqueue',
3379- '/usr/sbin/suexec',
3380- '/usr/lib/squid/ncsa_auth', '/usr/lib/squid/pam_auth',
3381- '/usr/kerberos/bin/ksu',
3382- '/usr/sbin/ccreds_validate',
3383- '/usr/bin/Xorg',
3384- '/usr/bin/X',
3385- '/usr/lib/dbus-1.0/dbus-daemon-launch-helper',
3386- '/usr/lib/vte/gnome-pty-helper',
3387- '/usr/lib/libvte9/gnome-pty-helper',
3388- '/usr/lib/libvte-2.90-9/gnome-pty-helper']
3389-
3390-
3391-def get_audits():
3392- """Get OS hardening suid/sgid audits.
3393-
3394- :returns: dictionary of audits
3395- """
3396- checks = []
3397- settings = utils.get_settings('os')
3398- if not settings['security']['suid_sgid_enforce']:
3399- log("Skipping suid/sgid hardening", level=INFO)
3400- return checks
3401-
3402- # Build the blacklist and whitelist of files for suid/sgid checks.
3403- # There are a total of 4 lists:
3404- # 1. the system blacklist
3405- # 2. the system whitelist
3406- # 3. the user blacklist
3407- # 4. the user whitelist
3408- #
3409- # The blacklist is the set of paths which should NOT have the suid/sgid bit
3410- # set and the whitelist is the set of paths which MAY have the suid/sgid
3411- # bit setl. The user whitelist/blacklist effectively override the system
3412- # whitelist/blacklist.
3413- u_b = settings['security']['suid_sgid_blacklist']
3414- u_w = settings['security']['suid_sgid_whitelist']
3415-
3416- blacklist = set(BLACKLIST) - set(u_w + u_b)
3417- whitelist = set(WHITELIST) - set(u_b + u_w)
3418-
3419- checks.append(NoSUIDSGIDAudit(blacklist))
3420-
3421- dry_run = settings['security']['suid_sgid_dry_run_on_unknown']
3422-
3423- if settings['security']['suid_sgid_remove_from_unknown'] or dry_run:
3424- # If the policy is a dry_run (e.g. complain only) or remove unknown
3425- # suid/sgid bits then find all of the paths which have the suid/sgid
3426- # bit set and then remove the whitelisted paths.
3427- root_path = settings['environment']['root_path']
3428- unknown_paths = find_paths_with_suid_sgid(root_path) - set(whitelist)
3429- checks.append(NoSUIDSGIDAudit(unknown_paths, unless=dry_run))
3430-
3431- return checks
3432-
3433-
3434-def find_paths_with_suid_sgid(root_path):
3435- """Finds all paths/files which have an suid/sgid bit enabled.
3436-
3437- Starting with the root_path, this will recursively find all paths which
3438- have an suid or sgid bit set.
3439- """
3440- cmd = ['find', root_path, '-perm', '-4000', '-o', '-perm', '-2000',
3441- '-type', 'f', '!', '-path', '/proc/*', '-print']
3442-
3443- p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
3444- out, _ = p.communicate()
3445- return set(out.split('\n'))
3446
3447=== removed file 'hooks/charmhelpers/contrib/hardening/host/checks/sysctl.py'
3448--- hooks/charmhelpers/contrib/hardening/host/checks/sysctl.py 2016-04-22 08:15:33 +0000
3449+++ hooks/charmhelpers/contrib/hardening/host/checks/sysctl.py 1970-01-01 00:00:00 +0000
3450@@ -1,211 +0,0 @@
3451-# Copyright 2016 Canonical Limited.
3452-#
3453-# This file is part of charm-helpers.
3454-#
3455-# charm-helpers is free software: you can redistribute it and/or modify
3456-# it under the terms of the GNU Lesser General Public License version 3 as
3457-# published by the Free Software Foundation.
3458-#
3459-# charm-helpers is distributed in the hope that it will be useful,
3460-# but WITHOUT ANY WARRANTY; without even the implied warranty of
3461-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
3462-# GNU Lesser General Public License for more details.
3463-#
3464-# You should have received a copy of the GNU Lesser General Public License
3465-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
3466-
3467-import os
3468-import platform
3469-import re
3470-import six
3471-import subprocess
3472-
3473-from charmhelpers.core.hookenv import (
3474- log,
3475- INFO,
3476- WARNING,
3477-)
3478-from charmhelpers.contrib.hardening import utils
3479-from charmhelpers.contrib.hardening.audits.file import (
3480- FilePermissionAudit,
3481- TemplatedFile,
3482-)
3483-from charmhelpers.contrib.hardening.host import TEMPLATES_DIR
3484-
3485-
3486-SYSCTL_DEFAULTS = """net.ipv4.ip_forward=%(net_ipv4_ip_forward)s
3487-net.ipv6.conf.all.forwarding=%(net_ipv6_conf_all_forwarding)s
3488-net.ipv4.conf.all.rp_filter=1
3489-net.ipv4.conf.default.rp_filter=1
3490-net.ipv4.icmp_echo_ignore_broadcasts=1
3491-net.ipv4.icmp_ignore_bogus_error_responses=1
3492-net.ipv4.icmp_ratelimit=100
3493-net.ipv4.icmp_ratemask=88089
3494-net.ipv6.conf.all.disable_ipv6=%(net_ipv6_conf_all_disable_ipv6)s
3495-net.ipv4.tcp_timestamps=%(net_ipv4_tcp_timestamps)s
3496-net.ipv4.conf.all.arp_ignore=%(net_ipv4_conf_all_arp_ignore)s
3497-net.ipv4.conf.all.arp_announce=%(net_ipv4_conf_all_arp_announce)s
3498-net.ipv4.tcp_rfc1337=1
3499-net.ipv4.tcp_syncookies=1
3500-net.ipv4.conf.all.shared_media=1
3501-net.ipv4.conf.default.shared_media=1
3502-net.ipv4.conf.all.accept_source_route=0
3503-net.ipv4.conf.default.accept_source_route=0
3504-net.ipv4.conf.all.accept_redirects=0
3505-net.ipv4.conf.default.accept_redirects=0
3506-net.ipv6.conf.all.accept_redirects=0
3507-net.ipv6.conf.default.accept_redirects=0
3508-net.ipv4.conf.all.secure_redirects=0
3509-net.ipv4.conf.default.secure_redirects=0
3510-net.ipv4.conf.all.send_redirects=0
3511-net.ipv4.conf.default.send_redirects=0
3512-net.ipv4.conf.all.log_martians=0
3513-net.ipv6.conf.default.router_solicitations=0
3514-net.ipv6.conf.default.accept_ra_rtr_pref=0
3515-net.ipv6.conf.default.accept_ra_pinfo=0
3516-net.ipv6.conf.default.accept_ra_defrtr=0
3517-net.ipv6.conf.default.autoconf=0
3518-net.ipv6.conf.default.dad_transmits=0
3519-net.ipv6.conf.default.max_addresses=1
3520-net.ipv6.conf.all.accept_ra=0
3521-net.ipv6.conf.default.accept_ra=0
3522-kernel.modules_disabled=%(kernel_modules_disabled)s
3523-kernel.sysrq=%(kernel_sysrq)s
3524-fs.suid_dumpable=%(fs_suid_dumpable)s
3525-kernel.randomize_va_space=2
3526-"""
3527-
3528-
3529-def get_audits():
3530- """Get OS hardening sysctl audits.
3531-
3532- :returns: dictionary of audits
3533- """
3534- audits = []
3535- settings = utils.get_settings('os')
3536-
3537- # Apply the sysctl settings which are configured to be applied.
3538- audits.append(SysctlConf())
3539- # Make sure that only root has access to the sysctl.conf file, and
3540- # that it is read-only.
3541- audits.append(FilePermissionAudit('/etc/sysctl.conf',
3542- user='root',
3543- group='root', mode=0o0440))
3544- # If module loading is not enabled, then ensure that the modules
3545- # file has the appropriate permissions and rebuild the initramfs
3546- if not settings['security']['kernel_enable_module_loading']:
3547- audits.append(ModulesTemplate())
3548-
3549- return audits
3550-
3551-
3552-class ModulesContext(object):
3553-
3554- def __call__(self):
3555- settings = utils.get_settings('os')
3556- with open('/proc/cpuinfo', 'r') as fd:
3557- cpuinfo = fd.readlines()
3558-
3559- for line in cpuinfo:
3560- match = re.search(r"^vendor_id\s+:\s+(.+)", line)
3561- if match:
3562- vendor = match.group(1)
3563-
3564- if vendor == "GenuineIntel":
3565- vendor = "intel"
3566- elif vendor == "AuthenticAMD":
3567- vendor = "amd"
3568-
3569- ctxt = {'arch': platform.processor(),
3570- 'cpuVendor': vendor,
3571- 'desktop_enable': settings['general']['desktop_enable']}
3572-
3573- return ctxt
3574-
3575-
3576-class ModulesTemplate(object):
3577-
3578- def __init__(self):
3579- super(ModulesTemplate, self).__init__('/etc/initramfs-tools/modules',
3580- ModulesContext(),
3581- templates_dir=TEMPLATES_DIR,
3582- user='root', group='root',
3583- mode=0o0440)
3584-
3585- def post_write(self):
3586- subprocess.check_call(['update-initramfs', '-u'])
3587-
3588-
3589-class SysCtlHardeningContext(object):
3590- def __call__(self):
3591- settings = utils.get_settings('os')
3592- ctxt = {'sysctl': {}}
3593-
3594- log("Applying sysctl settings", level=INFO)
3595- extras = {'net_ipv4_ip_forward': 0,
3596- 'net_ipv6_conf_all_forwarding': 0,
3597- 'net_ipv6_conf_all_disable_ipv6': 1,
3598- 'net_ipv4_tcp_timestamps': 0,
3599- 'net_ipv4_conf_all_arp_ignore': 0,
3600- 'net_ipv4_conf_all_arp_announce': 0,
3601- 'kernel_sysrq': 0,
3602- 'fs_suid_dumpable': 0,
3603- 'kernel_modules_disabled': 1}
3604-
3605- if settings['sysctl']['ipv6_enable']:
3606- extras['net_ipv6_conf_all_disable_ipv6'] = 0
3607-
3608- if settings['sysctl']['forwarding']:
3609- extras['net_ipv4_ip_forward'] = 1
3610- extras['net_ipv6_conf_all_forwarding'] = 1
3611-
3612- if settings['sysctl']['arp_restricted']:
3613- extras['net_ipv4_conf_all_arp_ignore'] = 1
3614- extras['net_ipv4_conf_all_arp_announce'] = 2
3615-
3616- if settings['security']['kernel_enable_module_loading']:
3617- extras['kernel_modules_disabled'] = 0
3618-
3619- if settings['sysctl']['kernel_enable_sysrq']:
3620- sysrq_val = settings['sysctl']['kernel_secure_sysrq']
3621- extras['kernel_sysrq'] = sysrq_val
3622-
3623- if settings['security']['kernel_enable_core_dump']:
3624- extras['fs_suid_dumpable'] = 1
3625-
3626- settings.update(extras)
3627- for d in (SYSCTL_DEFAULTS % settings).split():
3628- d = d.strip().partition('=')
3629- key = d[0].strip()
3630- path = os.path.join('/proc/sys', key.replace('.', '/'))
3631- if not os.path.exists(path):
3632- log("Skipping '%s' since '%s' does not exist" % (key, path),
3633- level=WARNING)
3634- continue
3635-
3636- ctxt['sysctl'][key] = d[2] or None
3637-
3638- # Translate for python3
3639- return {'sysctl_settings':
3640- [(k, v) for k, v in six.iteritems(ctxt['sysctl'])]}
3641-
3642-
3643-class SysctlConf(TemplatedFile):
3644- """An audit check for sysctl settings."""
3645- def __init__(self):
3646- self.conffile = '/etc/sysctl.d/99-juju-hardening.conf'
3647- super(SysctlConf, self).__init__(self.conffile,
3648- SysCtlHardeningContext(),
3649- template_dir=TEMPLATES_DIR,
3650- user='root', group='root',
3651- mode=0o0440)
3652-
3653- def post_write(self):
3654- try:
3655- subprocess.check_call(['sysctl', '-p', self.conffile])
3656- except subprocess.CalledProcessError as e:
3657- # NOTE: on some systems if sysctl cannot apply all settings it
3658- # will return non-zero as well.
3659- log("sysctl command returned an error (maybe some "
3660- "keys could not be set) - %s" % (e),
3661- level=WARNING)
3662
3663=== removed directory 'hooks/charmhelpers/contrib/hardening/mysql'
3664=== removed file 'hooks/charmhelpers/contrib/hardening/mysql/__init__.py'
3665--- hooks/charmhelpers/contrib/hardening/mysql/__init__.py 2016-04-22 08:15:33 +0000
3666+++ hooks/charmhelpers/contrib/hardening/mysql/__init__.py 1970-01-01 00:00:00 +0000
3667@@ -1,19 +0,0 @@
3668-# Copyright 2016 Canonical Limited.
3669-#
3670-# This file is part of charm-helpers.
3671-#
3672-# charm-helpers is free software: you can redistribute it and/or modify
3673-# it under the terms of the GNU Lesser General Public License version 3 as
3674-# published by the Free Software Foundation.
3675-#
3676-# charm-helpers is distributed in the hope that it will be useful,
3677-# but WITHOUT ANY WARRANTY; without even the implied warranty of
3678-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
3679-# GNU Lesser General Public License for more details.
3680-#
3681-# You should have received a copy of the GNU Lesser General Public License
3682-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
3683-
3684-from os import path
3685-
3686-TEMPLATES_DIR = path.join(path.dirname(__file__), 'templates')
3687
3688=== removed directory 'hooks/charmhelpers/contrib/hardening/mysql/checks'
3689=== removed file 'hooks/charmhelpers/contrib/hardening/mysql/checks/__init__.py'
3690--- hooks/charmhelpers/contrib/hardening/mysql/checks/__init__.py 2016-04-22 08:15:33 +0000
3691+++ hooks/charmhelpers/contrib/hardening/mysql/checks/__init__.py 1970-01-01 00:00:00 +0000
3692@@ -1,31 +0,0 @@
3693-# Copyright 2016 Canonical Limited.
3694-#
3695-# This file is part of charm-helpers.
3696-#
3697-# charm-helpers is free software: you can redistribute it and/or modify
3698-# it under the terms of the GNU Lesser General Public License version 3 as
3699-# published by the Free Software Foundation.
3700-#
3701-# charm-helpers is distributed in the hope that it will be useful,
3702-# but WITHOUT ANY WARRANTY; without even the implied warranty of
3703-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
3704-# GNU Lesser General Public License for more details.
3705-#
3706-# You should have received a copy of the GNU Lesser General Public License
3707-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
3708-
3709-from charmhelpers.core.hookenv import (
3710- log,
3711- DEBUG,
3712-)
3713-from charmhelpers.contrib.hardening.mysql.checks import config
3714-
3715-
3716-def run_mysql_checks():
3717- log("Starting MySQL hardening checks.", level=DEBUG)
3718- checks = config.get_audits()
3719- for check in checks:
3720- log("Running '%s' check" % (check.__class__.__name__), level=DEBUG)
3721- check.ensure_compliance()
3722-
3723- log("MySQL hardening checks complete.", level=DEBUG)
3724
3725=== removed file 'hooks/charmhelpers/contrib/hardening/mysql/checks/config.py'
3726--- hooks/charmhelpers/contrib/hardening/mysql/checks/config.py 2016-04-22 08:15:33 +0000
3727+++ hooks/charmhelpers/contrib/hardening/mysql/checks/config.py 1970-01-01 00:00:00 +0000
3728@@ -1,89 +0,0 @@
3729-# Copyright 2016 Canonical Limited.
3730-#
3731-# This file is part of charm-helpers.
3732-#
3733-# charm-helpers is free software: you can redistribute it and/or modify
3734-# it under the terms of the GNU Lesser General Public License version 3 as
3735-# published by the Free Software Foundation.
3736-#
3737-# charm-helpers is distributed in the hope that it will be useful,
3738-# but WITHOUT ANY WARRANTY; without even the implied warranty of
3739-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
3740-# GNU Lesser General Public License for more details.
3741-#
3742-# You should have received a copy of the GNU Lesser General Public License
3743-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
3744-
3745-import six
3746-import subprocess
3747-
3748-from charmhelpers.core.hookenv import (
3749- log,
3750- WARNING,
3751-)
3752-from charmhelpers.contrib.hardening.audits.file import (
3753- FilePermissionAudit,
3754- DirectoryPermissionAudit,
3755- TemplatedFile,
3756-)
3757-from charmhelpers.contrib.hardening.mysql import TEMPLATES_DIR
3758-from charmhelpers.contrib.hardening import utils
3759-
3760-
3761-def get_audits():
3762- """Get MySQL hardening config audits.
3763-
3764- :returns: dictionary of audits
3765- """
3766- if subprocess.call(['which', 'mysql'], stdout=subprocess.PIPE) != 0:
3767- log("MySQL does not appear to be installed on this node - "
3768- "skipping mysql hardening", level=WARNING)
3769- return []
3770-
3771- settings = utils.get_settings('mysql')
3772- hardening_settings = settings['hardening']
3773- my_cnf = hardening_settings['mysql-conf']
3774-
3775- audits = [
3776- FilePermissionAudit(paths=[my_cnf], user='root',
3777- group='root', mode=0o0600),
3778-
3779- TemplatedFile(hardening_settings['hardening-conf'],
3780- MySQLConfContext(),
3781- TEMPLATES_DIR,
3782- mode=0o0750,
3783- user='mysql',
3784- group='root',
3785- service_actions=[{'service': 'mysql',
3786- 'actions': ['restart']}]),
3787-
3788- # MySQL and Percona charms do not allow configuration of the
3789- # data directory, so use the default.
3790- DirectoryPermissionAudit('/var/lib/mysql',
3791- user='mysql',
3792- group='mysql',
3793- recursive=False,
3794- mode=0o755),
3795-
3796- DirectoryPermissionAudit('/etc/mysql',
3797- user='root',
3798- group='root',
3799- recursive=False,
3800- mode=0o700),
3801- ]
3802-
3803- return audits
3804-
3805-
3806-class MySQLConfContext(object):
3807- """Defines the set of key/value pairs to set in a mysql config file.
3808-
3809- This context, when called, will return a dictionary containing the
3810- key/value pairs of setting to specify in the
3811- /etc/mysql/conf.d/hardening.cnf file.
3812- """
3813- def __call__(self):
3814- settings = utils.get_settings('mysql')
3815- # Translate for python3
3816- return {'mysql_settings':
3817- [(k, v) for k, v in six.iteritems(settings['security'])]}
3818
3819=== removed directory 'hooks/charmhelpers/contrib/hardening/ssh'
3820=== removed file 'hooks/charmhelpers/contrib/hardening/ssh/__init__.py'
3821--- hooks/charmhelpers/contrib/hardening/ssh/__init__.py 2016-04-22 08:15:33 +0000
3822+++ hooks/charmhelpers/contrib/hardening/ssh/__init__.py 1970-01-01 00:00:00 +0000
3823@@ -1,19 +0,0 @@
3824-# Copyright 2016 Canonical Limited.
3825-#
3826-# This file is part of charm-helpers.
3827-#
3828-# charm-helpers is free software: you can redistribute it and/or modify
3829-# it under the terms of the GNU Lesser General Public License version 3 as
3830-# published by the Free Software Foundation.
3831-#
3832-# charm-helpers is distributed in the hope that it will be useful,
3833-# but WITHOUT ANY WARRANTY; without even the implied warranty of
3834-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
3835-# GNU Lesser General Public License for more details.
3836-#
3837-# You should have received a copy of the GNU Lesser General Public License
3838-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
3839-
3840-from os import path
3841-
3842-TEMPLATES_DIR = path.join(path.dirname(__file__), 'templates')
3843
3844=== removed directory 'hooks/charmhelpers/contrib/hardening/ssh/checks'
3845=== removed file 'hooks/charmhelpers/contrib/hardening/ssh/checks/__init__.py'
3846--- hooks/charmhelpers/contrib/hardening/ssh/checks/__init__.py 2016-04-22 08:15:33 +0000
3847+++ hooks/charmhelpers/contrib/hardening/ssh/checks/__init__.py 1970-01-01 00:00:00 +0000
3848@@ -1,31 +0,0 @@
3849-# Copyright 2016 Canonical Limited.
3850-#
3851-# This file is part of charm-helpers.
3852-#
3853-# charm-helpers is free software: you can redistribute it and/or modify
3854-# it under the terms of the GNU Lesser General Public License version 3 as
3855-# published by the Free Software Foundation.
3856-#
3857-# charm-helpers is distributed in the hope that it will be useful,
3858-# but WITHOUT ANY WARRANTY; without even the implied warranty of
3859-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
3860-# GNU Lesser General Public License for more details.
3861-#
3862-# You should have received a copy of the GNU Lesser General Public License
3863-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
3864-
3865-from charmhelpers.core.hookenv import (
3866- log,
3867- DEBUG,
3868-)
3869-from charmhelpers.contrib.hardening.ssh.checks import config
3870-
3871-
3872-def run_ssh_checks():
3873- log("Starting SSH hardening checks.", level=DEBUG)
3874- checks = config.get_audits()
3875- for check in checks:
3876- log("Running '%s' check" % (check.__class__.__name__), level=DEBUG)
3877- check.ensure_compliance()
3878-
3879- log("SSH hardening checks complete.", level=DEBUG)
3880
3881=== removed file 'hooks/charmhelpers/contrib/hardening/ssh/checks/config.py'
3882--- hooks/charmhelpers/contrib/hardening/ssh/checks/config.py 2016-04-22 08:15:33 +0000
3883+++ hooks/charmhelpers/contrib/hardening/ssh/checks/config.py 1970-01-01 00:00:00 +0000
3884@@ -1,394 +0,0 @@
3885-# Copyright 2016 Canonical Limited.
3886-#
3887-# This file is part of charm-helpers.
3888-#
3889-# charm-helpers is free software: you can redistribute it and/or modify
3890-# it under the terms of the GNU Lesser General Public License version 3 as
3891-# published by the Free Software Foundation.
3892-#
3893-# charm-helpers is distributed in the hope that it will be useful,
3894-# but WITHOUT ANY WARRANTY; without even the implied warranty of
3895-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
3896-# GNU Lesser General Public License for more details.
3897-#
3898-# You should have received a copy of the GNU Lesser General Public License
3899-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
3900-
3901-import os
3902-
3903-from charmhelpers.core.hookenv import (
3904- log,
3905- DEBUG,
3906-)
3907-from charmhelpers.fetch import (
3908- apt_install,
3909- apt_update,
3910-)
3911-from charmhelpers.core.host import lsb_release
3912-from charmhelpers.contrib.hardening.audits.file import (
3913- TemplatedFile,
3914- FileContentAudit,
3915-)
3916-from charmhelpers.contrib.hardening.ssh import TEMPLATES_DIR
3917-from charmhelpers.contrib.hardening import utils
3918-
3919-
3920-def get_audits():
3921- """Get SSH hardening config audits.
3922-
3923- :returns: dictionary of audits
3924- """
3925- audits = [SSHConfig(), SSHDConfig(), SSHConfigFileContentAudit(),
3926- SSHDConfigFileContentAudit()]
3927- return audits
3928-
3929-
3930-class SSHConfigContext(object):
3931-
3932- type = 'client'
3933-
3934- def get_macs(self, allow_weak_mac):
3935- if allow_weak_mac:
3936- weak_macs = 'weak'
3937- else:
3938- weak_macs = 'default'
3939-
3940- default = 'hmac-sha2-512,hmac-sha2-256,hmac-ripemd160'
3941- macs = {'default': default,
3942- 'weak': default + ',hmac-sha1'}
3943-
3944- default = ('hmac-sha2-512-etm@openssh.com,'
3945- 'hmac-sha2-256-etm@openssh.com,'
3946- 'hmac-ripemd160-etm@openssh.com,umac-128-etm@openssh.com,'
3947- 'hmac-sha2-512,hmac-sha2-256,hmac-ripemd160')
3948- macs_66 = {'default': default,
3949- 'weak': default + ',hmac-sha1'}
3950-
3951- # Use newer ciphers on Ubuntu Trusty and above
3952- if lsb_release()['DISTRIB_CODENAME'].lower() >= 'trusty':
3953- log("Detected Ubuntu 14.04 or newer, using new macs", level=DEBUG)
3954- macs = macs_66
3955-
3956- return macs[weak_macs]
3957-
3958- def get_kexs(self, allow_weak_kex):
3959- if allow_weak_kex:
3960- weak_kex = 'weak'
3961- else:
3962- weak_kex = 'default'
3963-
3964- default = 'diffie-hellman-group-exchange-sha256'
3965- weak = (default + ',diffie-hellman-group14-sha1,'
3966- 'diffie-hellman-group-exchange-sha1,'
3967- 'diffie-hellman-group1-sha1')
3968- kex = {'default': default,
3969- 'weak': weak}
3970-
3971- default = ('curve25519-sha256@libssh.org,'
3972- 'diffie-hellman-group-exchange-sha256')
3973- weak = (default + ',diffie-hellman-group14-sha1,'
3974- 'diffie-hellman-group-exchange-sha1,'
3975- 'diffie-hellman-group1-sha1')
3976- kex_66 = {'default': default,
3977- 'weak': weak}
3978-
3979- # Use newer kex on Ubuntu Trusty and above
3980- if lsb_release()['DISTRIB_CODENAME'].lower() >= 'trusty':
3981- log('Detected Ubuntu 14.04 or newer, using new key exchange '
3982- 'algorithms', level=DEBUG)
3983- kex = kex_66
3984-
3985- return kex[weak_kex]
3986-
3987- def get_ciphers(self, cbc_required):
3988- if cbc_required:
3989- weak_ciphers = 'weak'
3990- else:
3991- weak_ciphers = 'default'
3992-
3993- default = 'aes256-ctr,aes192-ctr,aes128-ctr'
3994- cipher = {'default': default,
3995- 'weak': default + 'aes256-cbc,aes192-cbc,aes128-cbc'}
3996-
3997- default = ('chacha20-poly1305@openssh.com,aes256-gcm@openssh.com,'
3998- 'aes128-gcm@openssh.com,aes256-ctr,aes192-ctr,aes128-ctr')
3999- ciphers_66 = {'default': default,
4000- 'weak': default + ',aes256-cbc,aes192-cbc,aes128-cbc'}
4001-
4002- # Use newer ciphers on ubuntu Trusty and above
4003- if lsb_release()['DISTRIB_CODENAME'].lower() >= 'trusty':
4004- log('Detected Ubuntu 14.04 or newer, using new ciphers',
4005- level=DEBUG)
4006- cipher = ciphers_66
4007-
4008- return cipher[weak_ciphers]
4009-
4010- def __call__(self):
4011- settings = utils.get_settings('ssh')
4012- if settings['common']['network_ipv6_enable']:
4013- addr_family = 'any'
4014- else:
4015- addr_family = 'inet'
4016-
4017- ctxt = {
4018- 'addr_family': addr_family,
4019- 'remote_hosts': settings['common']['remote_hosts'],
4020- 'password_auth_allowed':
4021- settings['client']['password_authentication'],
4022- 'ports': settings['common']['ports'],
4023- 'ciphers': self.get_ciphers(settings['client']['cbc_required']),
4024- 'macs': self.get_macs(settings['client']['weak_hmac']),
4025- 'kexs': self.get_kexs(settings['client']['weak_kex']),
4026- 'roaming': settings['client']['roaming'],
4027- }
4028- return ctxt
4029-
4030-
4031-class SSHConfig(TemplatedFile):
4032- def __init__(self):
4033- path = '/etc/ssh/ssh_config'
4034- super(SSHConfig, self).__init__(path=path,
4035- template_dir=TEMPLATES_DIR,
4036- context=SSHConfigContext(),
4037- user='root',
4038- group='root',
4039- mode=0o0644)
4040-
4041- def pre_write(self):
4042- settings = utils.get_settings('ssh')
4043- apt_update(fatal=True)
4044- apt_install(settings['client']['package'])
4045- if not os.path.exists('/etc/ssh'):
4046- os.makedir('/etc/ssh')
4047- # NOTE: don't recurse
4048- utils.ensure_permissions('/etc/ssh', 'root', 'root', 0o0755,
4049- maxdepth=0)
4050-
4051- def post_write(self):
4052- # NOTE: don't recurse
4053- utils.ensure_permissions('/etc/ssh', 'root', 'root', 0o0755,
4054- maxdepth=0)
4055-
4056-
4057-class SSHDConfigContext(SSHConfigContext):
4058-
4059- type = 'server'
4060-
4061- def __call__(self):
4062- settings = utils.get_settings('ssh')
4063- if settings['common']['network_ipv6_enable']:
4064- addr_family = 'any'
4065- else:
4066- addr_family = 'inet'
4067-
4068- ctxt = {
4069- 'ssh_ip': settings['server']['listen_to'],
4070- 'password_auth_allowed':
4071- settings['server']['password_authentication'],
4072- 'ports': settings['common']['ports'],
4073- 'addr_family': addr_family,
4074- 'ciphers': self.get_ciphers(settings['server']['cbc_required']),
4075- 'macs': self.get_macs(settings['server']['weak_hmac']),
4076- 'kexs': self.get_kexs(settings['server']['weak_kex']),
4077- 'host_key_files': settings['server']['host_key_files'],
4078- 'allow_root_with_key': settings['server']['allow_root_with_key'],
4079- 'password_authentication':
4080- settings['server']['password_authentication'],
4081- 'use_priv_sep': settings['server']['use_privilege_separation'],
4082- 'use_pam': settings['server']['use_pam'],
4083- 'allow_x11_forwarding': settings['server']['allow_x11_forwarding'],
4084- 'print_motd': settings['server']['print_motd'],
4085- 'print_last_log': settings['server']['print_last_log'],
4086- 'client_alive_interval':
4087- settings['server']['alive_interval'],
4088- 'client_alive_count': settings['server']['alive_count'],
4089- 'allow_tcp_forwarding': settings['server']['allow_tcp_forwarding'],
4090- 'allow_agent_forwarding':
4091- settings['server']['allow_agent_forwarding'],
4092- 'deny_users': settings['server']['deny_users'],
4093- 'allow_users': settings['server']['allow_users'],
4094- 'deny_groups': settings['server']['deny_groups'],
4095- 'allow_groups': settings['server']['allow_groups'],
4096- 'use_dns': settings['server']['use_dns'],
4097- 'sftp_enable': settings['server']['sftp_enable'],
4098- 'sftp_group': settings['server']['sftp_group'],
4099- 'sftp_chroot': settings['server']['sftp_chroot'],
4100- 'max_auth_tries': settings['server']['max_auth_tries'],
4101- 'max_sessions': settings['server']['max_sessions'],
4102- }
4103- return ctxt
4104-
4105-
4106-class SSHDConfig(TemplatedFile):
4107- def __init__(self):
4108- path = '/etc/ssh/sshd_config'
4109- super(SSHDConfig, self).__init__(path=path,
4110- template_dir=TEMPLATES_DIR,
4111- context=SSHDConfigContext(),
4112- user='root',
4113- group='root',
4114- mode=0o0600,
4115- service_actions=[{'service': 'ssh',
4116- 'actions':
4117- ['restart']}])
4118-
4119- def pre_write(self):
4120- settings = utils.get_settings('ssh')
4121- apt_update(fatal=True)
4122- apt_install(settings['server']['package'])
4123- if not os.path.exists('/etc/ssh'):
4124- os.makedir('/etc/ssh')
4125- # NOTE: don't recurse
4126- utils.ensure_permissions('/etc/ssh', 'root', 'root', 0o0755,
4127- maxdepth=0)
4128-
4129- def post_write(self):
4130- # NOTE: don't recurse
4131- utils.ensure_permissions('/etc/ssh', 'root', 'root', 0o0755,
4132- maxdepth=0)
4133-
4134-
4135-class SSHConfigFileContentAudit(FileContentAudit):
4136- def __init__(self):
4137- self.path = '/etc/ssh/ssh_config'
4138- super(SSHConfigFileContentAudit, self).__init__(self.path, {})
4139-
4140- def is_compliant(self, *args, **kwargs):
4141- self.pass_cases = []
4142- self.fail_cases = []
4143- settings = utils.get_settings('ssh')
4144-
4145- if lsb_release()['DISTRIB_CODENAME'].lower() >= 'trusty':
4146- if not settings['server']['weak_hmac']:
4147- self.pass_cases.append(r'^MACs.+,hmac-ripemd160$')
4148- else:
4149- self.pass_cases.append(r'^MACs.+,hmac-sha1$')
4150-
4151- if settings['server']['weak_kex']:
4152- self.fail_cases.append(r'^KexAlgorithms\sdiffie-hellman-group-exchange-sha256[,\s]?') # noqa
4153- self.pass_cases.append(r'^KexAlgorithms\sdiffie-hellman-group14-sha1[,\s]?') # noqa
4154- self.pass_cases.append(r'^KexAlgorithms\sdiffie-hellman-group-exchange-sha1[,\s]?') # noqa
4155- self.pass_cases.append(r'^KexAlgorithms\sdiffie-hellman-group1-sha1[,\s]?') # noqa
4156- else:
4157- self.pass_cases.append(r'^KexAlgorithms.+,diffie-hellman-group-exchange-sha256$') # noqa
4158- self.fail_cases.append(r'^KexAlgorithms.*diffie-hellman-group14-sha1[,\s]?') # noqa
4159-
4160- if settings['server']['cbc_required']:
4161- self.pass_cases.append(r'^Ciphers\s.*-cbc[,\s]?')
4162- self.fail_cases.append(r'^Ciphers\s.*aes128-ctr[,\s]?')
4163- self.fail_cases.append(r'^Ciphers\s.*aes192-ctr[,\s]?')
4164- self.fail_cases.append(r'^Ciphers\s.*aes256-ctr[,\s]?')
4165- else:
4166- self.fail_cases.append(r'^Ciphers\s.*-cbc[,\s]?')
4167- self.pass_cases.append(r'^Ciphers\schacha20-poly1305@openssh.com,.+') # noqa
4168- self.pass_cases.append(r'^Ciphers\s.*aes128-ctr$')
4169- self.pass_cases.append(r'^Ciphers\s.*aes192-ctr[,\s]?')
4170- self.pass_cases.append(r'^Ciphers\s.*aes256-ctr[,\s]?')
4171- else:
4172- if not settings['client']['weak_hmac']:
4173- self.fail_cases.append(r'^MACs.+,hmac-sha1$')
4174- else:
4175- self.pass_cases.append(r'^MACs.+,hmac-sha1$')
4176-
4177- if settings['client']['weak_kex']:
4178- self.fail_cases.append(r'^KexAlgorithms\sdiffie-hellman-group-exchange-sha256[,\s]?') # noqa
4179- self.pass_cases.append(r'^KexAlgorithms\sdiffie-hellman-group14-sha1[,\s]?') # noqa
4180- self.pass_cases.append(r'^KexAlgorithms\sdiffie-hellman-group-exchange-sha1[,\s]?') # noqa
4181- self.pass_cases.append(r'^KexAlgorithms\sdiffie-hellman-group1-sha1[,\s]?') # noqa
4182- else:
4183- self.pass_cases.append(r'^KexAlgorithms\sdiffie-hellman-group-exchange-sha256$') # noqa
4184- self.fail_cases.append(r'^KexAlgorithms\sdiffie-hellman-group14-sha1[,\s]?') # noqa
4185- self.fail_cases.append(r'^KexAlgorithms\sdiffie-hellman-group-exchange-sha1[,\s]?') # noqa
4186- self.fail_cases.append(r'^KexAlgorithms\sdiffie-hellman-group1-sha1[,\s]?') # noqa
4187-
4188- if settings['client']['cbc_required']:
4189- self.pass_cases.append(r'^Ciphers\s.*-cbc[,\s]?')
4190- self.fail_cases.append(r'^Ciphers\s.*aes128-ctr[,\s]?')
4191- self.fail_cases.append(r'^Ciphers\s.*aes192-ctr[,\s]?')
4192- self.fail_cases.append(r'^Ciphers\s.*aes256-ctr[,\s]?')
4193- else:
4194- self.fail_cases.append(r'^Ciphers\s.*-cbc[,\s]?')
4195- self.pass_cases.append(r'^Ciphers\s.*aes128-ctr[,\s]?')
4196- self.pass_cases.append(r'^Ciphers\s.*aes192-ctr[,\s]?')
4197- self.pass_cases.append(r'^Ciphers\s.*aes256-ctr[,\s]?')
4198-
4199- if settings['client']['roaming']:
4200- self.pass_cases.append(r'^UseRoaming yes$')
4201- else:
4202- self.fail_cases.append(r'^UseRoaming yes$')
4203-
4204- return super(SSHConfigFileContentAudit, self).is_compliant(*args,
4205- **kwargs)
4206-
4207-
4208-class SSHDConfigFileContentAudit(FileContentAudit):
4209- def __init__(self):
4210- self.path = '/etc/ssh/sshd_config'
4211- super(SSHDConfigFileContentAudit, self).__init__(self.path, {})
4212-
4213- def is_compliant(self, *args, **kwargs):
4214- self.pass_cases = []
4215- self.fail_cases = []
4216- settings = utils.get_settings('ssh')
4217-
4218- if lsb_release()['DISTRIB_CODENAME'].lower() >= 'trusty':
4219- if not settings['server']['weak_hmac']:
4220- self.pass_cases.append(r'^MACs.+,hmac-ripemd160$')
4221- else:
4222- self.pass_cases.append(r'^MACs.+,hmac-sha1$')
4223-
4224- if settings['server']['weak_kex']:
4225- self.fail_cases.append(r'^KexAlgorithms\sdiffie-hellman-group-exchange-sha256[,\s]?') # noqa
4226- self.pass_cases.append(r'^KexAlgorithms\sdiffie-hellman-group14-sha1[,\s]?') # noqa
4227- self.pass_cases.append(r'^KexAlgorithms\sdiffie-hellman-group-exchange-sha1[,\s]?') # noqa
4228- self.pass_cases.append(r'^KexAlgorithms\sdiffie-hellman-group1-sha1[,\s]?') # noqa
4229- else:
4230- self.pass_cases.append(r'^KexAlgorithms.+,diffie-hellman-group-exchange-sha256$') # noqa
4231- self.fail_cases.append(r'^KexAlgorithms.*diffie-hellman-group14-sha1[,\s]?') # noqa
4232-
4233- if settings['server']['cbc_required']:
4234- self.pass_cases.append(r'^Ciphers\s.*-cbc[,\s]?')
4235- self.fail_cases.append(r'^Ciphers\s.*aes128-ctr[,\s]?')
4236- self.fail_cases.append(r'^Ciphers\s.*aes192-ctr[,\s]?')
4237- self.fail_cases.append(r'^Ciphers\s.*aes256-ctr[,\s]?')
4238- else:
4239- self.fail_cases.append(r'^Ciphers\s.*-cbc[,\s]?')
4240- self.pass_cases.append(r'^Ciphers\schacha20-poly1305@openssh.com,.+') # noqa
4241- self.pass_cases.append(r'^Ciphers\s.*aes128-ctr$')
4242- self.pass_cases.append(r'^Ciphers\s.*aes192-ctr[,\s]?')
4243- self.pass_cases.append(r'^Ciphers\s.*aes256-ctr[,\s]?')
4244- else:
4245- if not settings['server']['weak_hmac']:
4246- self.pass_cases.append(r'^MACs.+,hmac-ripemd160$')
4247- else:
4248- self.pass_cases.append(r'^MACs.+,hmac-sha1$')
4249-
4250- if settings['server']['weak_kex']:
4251- self.fail_cases.append(r'^KexAlgorithms\sdiffie-hellman-group-exchange-sha256[,\s]?') # noqa
4252- self.pass_cases.append(r'^KexAlgorithms\sdiffie-hellman-group14-sha1[,\s]?') # noqa
4253- self.pass_cases.append(r'^KexAlgorithms\sdiffie-hellman-group-exchange-sha1[,\s]?') # noqa
4254- self.pass_cases.append(r'^KexAlgorithms\sdiffie-hellman-group1-sha1[,\s]?') # noqa
4255- else:
4256- self.pass_cases.append(r'^KexAlgorithms\sdiffie-hellman-group-exchange-sha256$') # noqa
4257- self.fail_cases.append(r'^KexAlgorithms\sdiffie-hellman-group14-sha1[,\s]?') # noqa
4258- self.fail_cases.append(r'^KexAlgorithms\sdiffie-hellman-group-exchange-sha1[,\s]?') # noqa
4259- self.fail_cases.append(r'^KexAlgorithms\sdiffie-hellman-group1-sha1[,\s]?') # noqa
4260-
4261- if settings['server']['cbc_required']:
4262- self.pass_cases.append(r'^Ciphers\s.*-cbc[,\s]?')
4263- self.fail_cases.append(r'^Ciphers\s.*aes128-ctr[,\s]?')
4264- self.fail_cases.append(r'^Ciphers\s.*aes192-ctr[,\s]?')
4265- self.fail_cases.append(r'^Ciphers\s.*aes256-ctr[,\s]?')
4266- else:
4267- self.fail_cases.append(r'^Ciphers\s.*-cbc[,\s]?')
4268- self.pass_cases.append(r'^Ciphers\s.*aes128-ctr[,\s]?')
4269- self.pass_cases.append(r'^Ciphers\s.*aes192-ctr[,\s]?')
4270- self.pass_cases.append(r'^Ciphers\s.*aes256-ctr[,\s]?')
4271-
4272- if settings['server']['sftp_enable']:
4273- self.pass_cases.append(r'^Subsystem\ssftp')
4274- else:
4275- self.fail_cases.append(r'^Subsystem\ssftp')
4276-
4277- return super(SSHDConfigFileContentAudit, self).is_compliant(*args,
4278- **kwargs)
4279
4280=== removed file 'hooks/charmhelpers/contrib/hardening/templating.py'
4281--- hooks/charmhelpers/contrib/hardening/templating.py 2016-04-22 08:15:33 +0000
4282+++ hooks/charmhelpers/contrib/hardening/templating.py 1970-01-01 00:00:00 +0000
4283@@ -1,71 +0,0 @@
4284-# Copyright 2016 Canonical Limited.
4285-#
4286-# This file is part of charm-helpers.
4287-#
4288-# charm-helpers is free software: you can redistribute it and/or modify
4289-# it under the terms of the GNU Lesser General Public License version 3 as
4290-# published by the Free Software Foundation.
4291-#
4292-# charm-helpers is distributed in the hope that it will be useful,
4293-# but WITHOUT ANY WARRANTY; without even the implied warranty of
4294-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
4295-# GNU Lesser General Public License for more details.
4296-#
4297-# You should have received a copy of the GNU Lesser General Public License
4298-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
4299-
4300-import os
4301-
4302-from charmhelpers.core.hookenv import (
4303- log,
4304- DEBUG,
4305- WARNING,
4306-)
4307-
4308-try:
4309- from jinja2 import FileSystemLoader, Environment
4310-except ImportError:
4311- from charmhelpers.fetch import apt_install
4312- from charmhelpers.fetch import apt_update
4313- apt_update(fatal=True)
4314- apt_install('python-jinja2', fatal=True)
4315- from jinja2 import FileSystemLoader, Environment
4316-
4317-
4318-# NOTE: function separated from main rendering code to facilitate easier
4319-# mocking in unit tests.
4320-def write(path, data):
4321- with open(path, 'wb') as out:
4322- out.write(data)
4323-
4324-
4325-def get_template_path(template_dir, path):
4326- """Returns the template file which would be used to render the path.
4327-
4328- The path to the template file is returned.
4329- :param template_dir: the directory the templates are located in
4330- :param path: the file path to be written to.
4331- :returns: path to the template file
4332- """
4333- return os.path.join(template_dir, os.path.basename(path))
4334-
4335-
4336-def render_and_write(template_dir, path, context):
4337- """Renders the specified template into the file.
4338-
4339- :param template_dir: the directory to load the template from
4340- :param path: the path to write the templated contents to
4341- :param context: the parameters to pass to the rendering engine
4342- """
4343- env = Environment(loader=FileSystemLoader(template_dir))
4344- template_file = os.path.basename(path)
4345- template = env.get_template(template_file)
4346- log('Rendering from template: %s' % template.name, level=DEBUG)
4347- rendered_content = template.render(context)
4348- if not rendered_content:
4349- log("Render returned None - skipping '%s'" % path,
4350- level=WARNING)
4351- return
4352-
4353- write(path, rendered_content.encode('utf-8').strip())
4354- log('Wrote template %s' % path, level=DEBUG)
4355
4356=== removed file 'hooks/charmhelpers/contrib/hardening/utils.py'
4357--- hooks/charmhelpers/contrib/hardening/utils.py 2016-04-22 08:15:33 +0000
4358+++ hooks/charmhelpers/contrib/hardening/utils.py 1970-01-01 00:00:00 +0000
4359@@ -1,157 +0,0 @@
4360-# Copyright 2016 Canonical Limited.
4361-#
4362-# This file is part of charm-helpers.
4363-#
4364-# charm-helpers is free software: you can redistribute it and/or modify
4365-# it under the terms of the GNU Lesser General Public License version 3 as
4366-# published by the Free Software Foundation.
4367-#
4368-# charm-helpers is distributed in the hope that it will be useful,
4369-# but WITHOUT ANY WARRANTY; without even the implied warranty of
4370-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
4371-# GNU Lesser General Public License for more details.
4372-#
4373-# You should have received a copy of the GNU Lesser General Public License
4374-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
4375-
4376-import glob
4377-import grp
4378-import os
4379-import pwd
4380-import six
4381-import yaml
4382-
4383-from charmhelpers.core.hookenv import (
4384- log,
4385- DEBUG,
4386- INFO,
4387- WARNING,
4388- ERROR,
4389-)
4390-
4391-
4392-# Global settings cache. Since each hook fire entails a fresh module import it
4393-# is safe to hold this in memory and not risk missing config changes (since
4394-# they will result in a new hook fire and thus re-import).
4395-__SETTINGS__ = {}
4396-
4397-
4398-def _get_defaults(modules):
4399- """Load the default config for the provided modules.
4400-
4401- :param modules: stack modules config defaults to lookup.
4402- :returns: modules default config dictionary.
4403- """
4404- default = os.path.join(os.path.dirname(__file__),
4405- 'defaults/%s.yaml' % (modules))
4406- return yaml.safe_load(open(default))
4407-
4408-
4409-def _get_schema(modules):
4410- """Load the config schema for the provided modules.
4411-
4412- NOTE: this schema is intended to have 1-1 relationship with they keys in
4413- the default config and is used a means to verify valid overrides provided
4414- by the user.
4415-
4416- :param modules: stack modules config schema to lookup.
4417- :returns: modules default schema dictionary.
4418- """
4419- schema = os.path.join(os.path.dirname(__file__),
4420- 'defaults/%s.yaml.schema' % (modules))
4421- return yaml.safe_load(open(schema))
4422-
4423-
4424-def _get_user_provided_overrides(modules):
4425- """Load user-provided config overrides.
4426-
4427- :param modules: stack modules to lookup in user overrides yaml file.
4428- :returns: overrides dictionary.
4429- """
4430- overrides = os.path.join(os.environ['JUJU_CHARM_DIR'],
4431- 'hardening.yaml')
4432- if os.path.exists(overrides):
4433- log("Found user-provided config overrides file '%s'" %
4434- (overrides), level=DEBUG)
4435- settings = yaml.safe_load(open(overrides))
4436- if settings and settings.get(modules):
4437- log("Applying '%s' overrides" % (modules), level=DEBUG)
4438- return settings.get(modules)
4439-
4440- log("No overrides found for '%s'" % (modules), level=DEBUG)
4441- else:
4442- log("No hardening config overrides file '%s' found in charm "
4443- "root dir" % (overrides), level=DEBUG)
4444-
4445- return {}
4446-
4447-
4448-def _apply_overrides(settings, overrides, schema):
4449- """Get overrides config overlayed onto modules defaults.
4450-
4451- :param modules: require stack modules config.
4452- :returns: dictionary of modules config with user overrides applied.
4453- """
4454- if overrides:
4455- for k, v in six.iteritems(overrides):
4456- if k in schema:
4457- if schema[k] is None:
4458- settings[k] = v
4459- elif type(schema[k]) is dict:
4460- settings[k] = _apply_overrides(settings[k], overrides[k],
4461- schema[k])
4462- else:
4463- raise Exception("Unexpected type found in schema '%s'" %
4464- type(schema[k]), level=ERROR)
4465- else:
4466- log("Unknown override key '%s' - ignoring" % (k), level=INFO)
4467-
4468- return settings
4469-
4470-
4471-def get_settings(modules):
4472- global __SETTINGS__
4473- if modules in __SETTINGS__:
4474- return __SETTINGS__[modules]
4475-
4476- schema = _get_schema(modules)
4477- settings = _get_defaults(modules)
4478- overrides = _get_user_provided_overrides(modules)
4479- __SETTINGS__[modules] = _apply_overrides(settings, overrides, schema)
4480- return __SETTINGS__[modules]
4481-
4482-
4483-def ensure_permissions(path, user, group, permissions, maxdepth=-1):
4484- """Ensure permissions for path.
4485-
4486- If path is a file, apply to file and return. If path is a directory,
4487- apply recursively (if required) to directory contents and return.
4488-
4489- :param user: user name
4490- :param group: group name
4491- :param permissions: octal permissions
4492- :param maxdepth: maximum recursion depth. A negative maxdepth allows
4493- infinite recursion and maxdepth=0 means no recursion.
4494- :returns: None
4495- """
4496- if not os.path.exists(path):
4497- log("File '%s' does not exist - cannot set permissions" % (path),
4498- level=WARNING)
4499- return
4500-
4501- _user = pwd.getpwnam(user)
4502- os.chown(path, _user.pw_uid, grp.getgrnam(group).gr_gid)
4503- os.chmod(path, permissions)
4504-
4505- if maxdepth == 0:
4506- log("Max recursion depth reached - skipping further recursion",
4507- level=DEBUG)
4508- return
4509- elif maxdepth > 0:
4510- maxdepth -= 1
4511-
4512- if os.path.isdir(path):
4513- contents = glob.glob("%s/*" % (path))
4514- for c in contents:
4515- ensure_permissions(c, user=user, group=group,
4516- permissions=permissions, maxdepth=maxdepth)
4517
4518=== removed directory 'hooks/charmhelpers/contrib/mellanox'
4519=== removed file 'hooks/charmhelpers/contrib/mellanox/__init__.py'
4520=== removed file 'hooks/charmhelpers/contrib/mellanox/infiniband.py'
4521--- hooks/charmhelpers/contrib/mellanox/infiniband.py 2016-01-30 22:44:06 +0000
4522+++ hooks/charmhelpers/contrib/mellanox/infiniband.py 1970-01-01 00:00:00 +0000
4523@@ -1,151 +0,0 @@
4524-#!/usr/bin/env python
4525-# -*- coding: utf-8 -*-
4526-
4527-# Copyright 2014-2015 Canonical Limited.
4528-#
4529-# This file is part of charm-helpers.
4530-#
4531-# charm-helpers is free software: you can redistribute it and/or modify
4532-# it under the terms of the GNU Lesser General Public License version 3 as
4533-# published by the Free Software Foundation.
4534-#
4535-# charm-helpers is distributed in the hope that it will be useful,
4536-# but WITHOUT ANY WARRANTY; without even the implied warranty of
4537-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
4538-# GNU Lesser General Public License for more details.
4539-#
4540-# You should have received a copy of the GNU Lesser General Public License
4541-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
4542-
4543-
4544-__author__ = "Jorge Niedbalski <jorge.niedbalski@canonical.com>"
4545-
4546-from charmhelpers.fetch import (
4547- apt_install,
4548- apt_update,
4549-)
4550-
4551-from charmhelpers.core.hookenv import (
4552- log,
4553- INFO,
4554-)
4555-
4556-try:
4557- from netifaces import interfaces as network_interfaces
4558-except ImportError:
4559- apt_install('python-netifaces')
4560- from netifaces import interfaces as network_interfaces
4561-
4562-import os
4563-import re
4564-import subprocess
4565-
4566-from charmhelpers.core.kernel import modprobe
4567-
4568-REQUIRED_MODULES = (
4569- "mlx4_ib",
4570- "mlx4_en",
4571- "mlx4_core",
4572- "ib_ipath",
4573- "ib_mthca",
4574- "ib_srpt",
4575- "ib_srp",
4576- "ib_ucm",
4577- "ib_isert",
4578- "ib_iser",
4579- "ib_ipoib",
4580- "ib_cm",
4581- "ib_uverbs"
4582- "ib_umad",
4583- "ib_sa",
4584- "ib_mad",
4585- "ib_core",
4586- "ib_addr",
4587- "rdma_ucm",
4588-)
4589-
4590-REQUIRED_PACKAGES = (
4591- "ibutils",
4592- "infiniband-diags",
4593- "ibverbs-utils",
4594-)
4595-
4596-IPOIB_DRIVERS = (
4597- "ib_ipoib",
4598-)
4599-
4600-ABI_VERSION_FILE = "/sys/class/infiniband_mad/abi_version"
4601-
4602-
4603-class DeviceInfo(object):
4604- pass
4605-
4606-
4607-def install_packages():
4608- apt_update()
4609- apt_install(REQUIRED_PACKAGES, fatal=True)
4610-
4611-
4612-def load_modules():
4613- for module in REQUIRED_MODULES:
4614- modprobe(module, persist=True)
4615-
4616-
4617-def is_enabled():
4618- """Check if infiniband is loaded on the system"""
4619- return os.path.exists(ABI_VERSION_FILE)
4620-
4621-
4622-def stat():
4623- """Return full output of ibstat"""
4624- return subprocess.check_output(["ibstat"])
4625-
4626-
4627-def devices():
4628- """Returns a list of IB enabled devices"""
4629- return subprocess.check_output(['ibstat', '-l']).splitlines()
4630-
4631-
4632-def device_info(device):
4633- """Returns a DeviceInfo object with the current device settings"""
4634-
4635- status = subprocess.check_output([
4636- 'ibstat', device, '-s']).splitlines()
4637-
4638- regexes = {
4639- "CA type: (.*)": "device_type",
4640- "Number of ports: (.*)": "num_ports",
4641- "Firmware version: (.*)": "fw_ver",
4642- "Hardware version: (.*)": "hw_ver",
4643- "Node GUID: (.*)": "node_guid",
4644- "System image GUID: (.*)": "sys_guid",
4645- }
4646-
4647- device = DeviceInfo()
4648-
4649- for line in status:
4650- for expression, key in regexes.items():
4651- matches = re.search(expression, line)
4652- if matches:
4653- setattr(device, key, matches.group(1))
4654-
4655- return device
4656-
4657-
4658-def ipoib_interfaces():
4659- """Return a list of IPOIB capable ethernet interfaces"""
4660- interfaces = []
4661-
4662- for interface in network_interfaces():
4663- try:
4664- driver = re.search('^driver: (.+)$', subprocess.check_output([
4665- 'ethtool', '-i',
4666- interface]), re.M).group(1)
4667-
4668- if driver in IPOIB_DRIVERS:
4669- interfaces.append(interface)
4670- except:
4671- log("Skipping interface %s" % interface, level=INFO)
4672- continue
4673-
4674- return interfaces
4675
4676=== removed directory 'hooks/charmhelpers/contrib/peerstorage'
4677=== removed file 'hooks/charmhelpers/contrib/peerstorage/__init__.py'
4678--- hooks/charmhelpers/contrib/peerstorage/__init__.py 2016-01-30 22:40:29 +0000
4679+++ hooks/charmhelpers/contrib/peerstorage/__init__.py 1970-01-01 00:00:00 +0000
4680@@ -1,269 +0,0 @@
4681-# Copyright 2014-2015 Canonical Limited.
4682-#
4683-# This file is part of charm-helpers.
4684-#
4685-# charm-helpers is free software: you can redistribute it and/or modify
4686-# it under the terms of the GNU Lesser General Public License version 3 as
4687-# published by the Free Software Foundation.
4688-#
4689-# charm-helpers is distributed in the hope that it will be useful,
4690-# but WITHOUT ANY WARRANTY; without even the implied warranty of
4691-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
4692-# GNU Lesser General Public License for more details.
4693-#
4694-# You should have received a copy of the GNU Lesser General Public License
4695-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
4696-
4697-import json
4698-import six
4699-
4700-from charmhelpers.core.hookenv import relation_id as current_relation_id
4701-from charmhelpers.core.hookenv import (
4702- is_relation_made,
4703- relation_ids,
4704- relation_get as _relation_get,
4705- local_unit,
4706- relation_set as _relation_set,
4707- leader_get as _leader_get,
4708- leader_set,
4709- is_leader,
4710-)
4711-
4712-
4713-"""
4714-This helper provides functions to support use of a peer relation
4715-for basic key/value storage, with the added benefit that all storage
4716-can be replicated across peer units.
4717-
4718-Requirement to use:
4719-
4720-To use this, the "peer_echo()" method has to be called form the peer
4721-relation's relation-changed hook:
4722-
4723-@hooks.hook("cluster-relation-changed") # Adapt the to your peer relation name
4724-def cluster_relation_changed():
4725- peer_echo()
4726-
4727-Once this is done, you can use peer storage from anywhere:
4728-
4729-@hooks.hook("some-hook")
4730-def some_hook():
4731- # You can store and retrieve key/values this way:
4732- if is_relation_made("cluster"): # from charmhelpers.core.hookenv
4733- # There are peers available so we can work with peer storage
4734- peer_store("mykey", "myvalue")
4735- value = peer_retrieve("mykey")
4736- print value
4737- else:
4738- print "No peers joind the relation, cannot share key/values :("
4739-"""
4740-
4741-
4742-def leader_get(attribute=None, rid=None):
4743- """Wrapper to ensure that settings are migrated from the peer relation.
4744-
4745- This is to support upgrading an environment that does not support
4746- Juju leadership election to one that does.
4747-
4748- If a setting is not extant in the leader-get but is on the relation-get
4749- peer rel, it is migrated and marked as such so that it is not re-migrated.
4750- """
4751- migration_key = '__leader_get_migrated_settings__'
4752- if not is_leader():
4753- return _leader_get(attribute=attribute)
4754-
4755- settings_migrated = False
4756- leader_settings = _leader_get(attribute=attribute)
4757- previously_migrated = _leader_get(attribute=migration_key)
4758-
4759- if previously_migrated:
4760- migrated = set(json.loads(previously_migrated))
4761- else:
4762- migrated = set([])
4763-
4764- try:
4765- if migration_key in leader_settings:
4766- del leader_settings[migration_key]
4767- except TypeError:
4768- pass
4769-
4770- if attribute:
4771- if attribute in migrated:
4772- return leader_settings
4773-
4774- # If attribute not present in leader db, check if this unit has set
4775- # the attribute in the peer relation
4776- if not leader_settings:
4777- peer_setting = _relation_get(attribute=attribute, unit=local_unit(),
4778- rid=rid)
4779- if peer_setting:
4780- leader_set(settings={attribute: peer_setting})
4781- leader_settings = peer_setting
4782-
4783- if leader_settings:
4784- settings_migrated = True
4785- migrated.add(attribute)
4786- else:
4787- r_settings = _relation_get(unit=local_unit(), rid=rid)
4788- if r_settings:
4789- for key in set(r_settings.keys()).difference(migrated):
4790- # Leader setting wins
4791- if not leader_settings.get(key):
4792- leader_settings[key] = r_settings[key]
4793-
4794- settings_migrated = True
4795- migrated.add(key)
4796-
4797- if settings_migrated:
4798- leader_set(**leader_settings)
4799-
4800- if migrated and settings_migrated:
4801- migrated = json.dumps(list(migrated))
4802- leader_set(settings={migration_key: migrated})
4803-
4804- return leader_settings
4805-
4806-
4807-def relation_set(relation_id=None, relation_settings=None, **kwargs):
4808- """Attempt to use leader-set if supported in the current version of Juju,
4809- otherwise falls back on relation-set.
4810-
4811- Note that we only attempt to use leader-set if the provided relation_id is
4812- a peer relation id or no relation id is provided (in which case we assume
4813- we are within the peer relation context).
4814- """
4815- try:
4816- if relation_id in relation_ids('cluster'):
4817- return leader_set(settings=relation_settings, **kwargs)
4818- else:
4819- raise NotImplementedError
4820- except NotImplementedError:
4821- return _relation_set(relation_id=relation_id,
4822- relation_settings=relation_settings, **kwargs)
4823-
4824-
4825-def relation_get(attribute=None, unit=None, rid=None):
4826- """Attempt to use leader-get if supported in the current version of Juju,
4827- otherwise falls back on relation-get.
4828-
4829- Note that we only attempt to use leader-get if the provided rid is a peer
4830- relation id or no relation id is provided (in which case we assume we are
4831- within the peer relation context).
4832- """
4833- try:
4834- if rid in relation_ids('cluster'):
4835- return leader_get(attribute, rid)
4836- else:
4837- raise NotImplementedError
4838- except NotImplementedError:
4839- return _relation_get(attribute=attribute, rid=rid, unit=unit)
4840-
4841-
4842-def peer_retrieve(key, relation_name='cluster'):
4843- """Retrieve a named key from peer relation `relation_name`."""
4844- cluster_rels = relation_ids(relation_name)
4845- if len(cluster_rels) > 0:
4846- cluster_rid = cluster_rels[0]
4847- return relation_get(attribute=key, rid=cluster_rid,
4848- unit=local_unit())
4849- else:
4850- raise ValueError('Unable to detect'
4851- 'peer relation {}'.format(relation_name))
4852-
4853-
4854-def peer_retrieve_by_prefix(prefix, relation_name='cluster', delimiter='_',
4855- inc_list=None, exc_list=None):
4856- """ Retrieve k/v pairs given a prefix and filter using {inc,exc}_list """
4857- inc_list = inc_list if inc_list else []
4858- exc_list = exc_list if exc_list else []
4859- peerdb_settings = peer_retrieve('-', relation_name=relation_name)
4860- matched = {}
4861- if peerdb_settings is None:
4862- return matched
4863- for k, v in peerdb_settings.items():
4864- full_prefix = prefix + delimiter
4865- if k.startswith(full_prefix):
4866- new_key = k.replace(full_prefix, '')
4867- if new_key in exc_list:
4868- continue
4869- if new_key in inc_list or len(inc_list) == 0:
4870- matched[new_key] = v
4871- return matched
4872-
4873-
4874-def peer_store(key, value, relation_name='cluster'):
4875- """Store the key/value pair on the named peer relation `relation_name`."""
4876- cluster_rels = relation_ids(relation_name)
4877- if len(cluster_rels) > 0:
4878- cluster_rid = cluster_rels[0]
4879- relation_set(relation_id=cluster_rid,
4880- relation_settings={key: value})
4881- else:
4882- raise ValueError('Unable to detect '
4883- 'peer relation {}'.format(relation_name))
4884-
4885-
4886-def peer_echo(includes=None, force=False):
4887- """Echo filtered attributes back onto the same relation for storage.
4888-
4889- This is a requirement to use the peerstorage module - it needs to be called
4890- from the peer relation's changed hook.
4891-
4892- If Juju leader support exists this will be a noop unless force is True.
4893- """
4894- try:
4895- is_leader()
4896- except NotImplementedError:
4897- pass
4898- else:
4899- if not force:
4900- return # NOOP if leader-election is supported
4901-
4902- # Use original non-leader calls
4903- relation_get = _relation_get
4904- relation_set = _relation_set
4905-
4906- rdata = relation_get()
4907- echo_data = {}
4908- if includes is None:
4909- echo_data = rdata.copy()
4910- for ex in ['private-address', 'public-address']:
4911- if ex in echo_data:
4912- echo_data.pop(ex)
4913- else:
4914- for attribute, value in six.iteritems(rdata):
4915- for include in includes:
4916- if include in attribute:
4917- echo_data[attribute] = value
4918- if len(echo_data) > 0:
4919- relation_set(relation_settings=echo_data)
4920-
4921-
4922-def peer_store_and_set(relation_id=None, peer_relation_name='cluster',
4923- peer_store_fatal=False, relation_settings=None,
4924- delimiter='_', **kwargs):
4925- """Store passed-in arguments both in argument relation and in peer storage.
4926-
4927- It functions like doing relation_set() and peer_store() at the same time,
4928- with the same data.
4929-
4930- @param relation_id: the id of the relation to store the data on. Defaults
4931- to the current relation.
4932- @param peer_store_fatal: Set to True, the function will raise an exception
4933- should the peer sotrage not be avialable."""
4934-
4935- relation_settings = relation_settings if relation_settings else {}
4936- relation_set(relation_id=relation_id,
4937- relation_settings=relation_settings,
4938- **kwargs)
4939- if is_relation_made(peer_relation_name):
4940- for key, value in six.iteritems(dict(list(kwargs.items()) +
4941- list(relation_settings.items()))):
4942- key_prefix = relation_id or current_relation_id()
4943- peer_store(key_prefix + delimiter + key,
4944- value,
4945- relation_name=peer_relation_name)
4946- else:
4947- if peer_store_fatal:
4948- raise ValueError('Unable to detect '
4949- 'peer relation {}'.format(peer_relation_name))
4950
4951=== removed directory 'hooks/charmhelpers/contrib/saltstack'
4952=== removed file 'hooks/charmhelpers/contrib/saltstack/__init__.py'
4953--- hooks/charmhelpers/contrib/saltstack/__init__.py 2015-07-29 18:23:55 +0000
4954+++ hooks/charmhelpers/contrib/saltstack/__init__.py 1970-01-01 00:00:00 +0000
4955@@ -1,118 +0,0 @@
4956-# Copyright 2014-2015 Canonical Limited.
4957-#
4958-# This file is part of charm-helpers.
4959-#
4960-# charm-helpers is free software: you can redistribute it and/or modify
4961-# it under the terms of the GNU Lesser General Public License version 3 as
4962-# published by the Free Software Foundation.
4963-#
4964-# charm-helpers is distributed in the hope that it will be useful,
4965-# but WITHOUT ANY WARRANTY; without even the implied warranty of
4966-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
4967-# GNU Lesser General Public License for more details.
4968-#
4969-# You should have received a copy of the GNU Lesser General Public License
4970-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
4971-
4972-"""Charm Helpers saltstack - declare the state of your machines.
4973-
4974-This helper enables you to declare your machine state, rather than
4975-program it procedurally (and have to test each change to your procedures).
4976-Your install hook can be as simple as::
4977-
4978- {{{
4979- from charmhelpers.contrib.saltstack import (
4980- install_salt_support,
4981- update_machine_state,
4982- )
4983-
4984-
4985- def install():
4986- install_salt_support()
4987- update_machine_state('machine_states/dependencies.yaml')
4988- update_machine_state('machine_states/installed.yaml')
4989- }}}
4990-
4991-and won't need to change (nor will its tests) when you change the machine
4992-state.
4993-
4994-It's using a python package called salt-minion which allows various formats for
4995-specifying resources, such as::
4996-
4997- {{{
4998- /srv/{{ basedir }}:
4999- file.directory:
5000- - group: ubunet
The diff has been truncated for viewing.

Subscribers

People subscribed via source and target branches