Merge lp:~junaidali/charms/trusty/plumgrid-director/pg-restart into lp:~plumgrid-team/charms/trusty/plumgrid-director/trunk

[Junaid Ali]

[lp:~junaidali/charms/trusty/plumgrid-director/pg-restart]

33. By Junaid Ali on 2016-04-28

Updated restart_pg function

34. By Junaid Ali on 2016-04-28

Updated sleep time after starting libvirt-bin

35. By Junaid Ali on 2016-05-01

update sleep time in restart_pg, changes for make sync

36. By Junaid Ali on 2016-05-03

nginx changes

37. By Junaid Ali on 2016-05-04

director provides a new relations for pg-vip etc

38. By Junaid Ali on 2016-05-04

added symlink

39. By Junaid Ali on 2016-05-04

leader check for plumgrid-configs relation

=== modified file 'charm-helpers-sync.yaml'
--- charm-helpers-sync.yaml	2015-07-29 18:07:31 +0000
+++ charm-helpers-sync.yaml	2016-05-04 13:25:41 +0000
@@ -3,5 +3,10 @@
 include:
     - core
     - fetch
-    - contrib
+    - contrib.amulet
+    - contrib.hahelpers
+    - contrib.network
+    - contrib.openstack
+    - contrib.python
+    - contrib.storage
     - payload

=== modified file 'config.yaml'
--- config.yaml	2016-03-24 12:33:25 +0000
+++ config.yaml	2016-05-04 13:25:41 +0000
@@ -3,6 +3,14 @@
     default: 192.168.100.250
     type: string
     description: IP address of the Director's Management interface. Same IP can be used to access PG Console.
+  plumgrid-username:
+    default: plumgrid
+    type: string
+    description: Username to access PLUMgrid Director
+  plumgrid-password:
+    default: plumgrid
+    type: string
+    description: Password to access PLUMgrid Director
   lcm-ssh-key:
     default: 'null'
     type: string

=== removed directory 'hooks/charmhelpers/contrib/ansible'
=== removed file 'hooks/charmhelpers/contrib/ansible/__init__.py'
--- hooks/charmhelpers/contrib/ansible/__init__.py	2015-07-29 18:07:31 +0000
+++ hooks/charmhelpers/contrib/ansible/__init__.py	1970-01-01 00:00:00 +0000
@@ -1,254 +0,0 @@
-# Copyright 2014-2015 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-# Copyright 2013 Canonical Ltd.
-#
-# Authors:
-#  Charm Helpers Developers <[e-mail]>
-"""Charm Helpers ansible - declare the state of your machines.
-
-This helper enables you to declare your machine state, rather than
-program it procedurally (and have to test each change to your procedures).
-Your install hook can be as simple as::
-
-    {{{
-    import charmhelpers.contrib.ansible
-
-
-    def install():
-        charmhelpers.contrib.ansible.install_ansible_support()
-        charmhelpers.contrib.ansible.apply_playbook('playbooks/install.yaml')
-    }}}
-
-and won't need to change (nor will its tests) when you change the machine
-state.
-
-All of your juju config and relation-data are available as template
-variables within your playbooks and templates. An install playbook looks
-something like::
-
-    {{{
-    ---
-    - hosts: localhost
-      user: root
-
-      tasks:
-        - name: Add private repositories.
-          template:
-            src: ../templates/private-repositories.list.jinja2
-            dest: /etc/apt/sources.list.d/private.list
-
-        - name: Update the cache.
-          apt: update_cache=yes
-
-        - name: Install dependencies.
-          apt: pkg={{ item }}
-          with_items:
-            - python-mimeparse
-            - python-webob
-            - sunburnt
-
-        - name: Setup groups.
-          group: name={{ item.name }} gid={{ item.gid }}
-          with_items:
-            - { name: 'deploy_user', gid: 1800 }
-            - { name: 'service_user', gid: 1500 }
-
-      ...
-    }}}
-
-Read more online about `playbooks`_ and standard ansible `modules`_.
-
-.. _playbooks: http://www.ansibleworks.com/docs/playbooks.html
-.. _modules: http://www.ansibleworks.com/docs/modules.html
-
-A further feature os the ansible hooks is to provide a light weight "action"
-scripting tool. This is a decorator that you apply to a function, and that
-function can now receive cli args, and can pass extra args to the playbook.
-
-e.g.
-
-
[e-mail]()
-def some_action(amount, force="False"):
-    "Usage: some-action AMOUNT [force=True]"  # <-- shown on error
-    # process the arguments
-    # do some calls
-    # return extra-vars to be passed to ansible-playbook
-    return {
-        'amount': int(amount),
-        'type': force,
-    }
-
-You can now create a symlink to hooks.py that can be invoked like a hook, but
-with cli params:
-
-# link actions/some-action to hooks/hooks.py
-
-actions/some-action amount=10 force=true
-
-"""
-import os
-import stat
-import subprocess
-import functools
-
-import charmhelpers.contrib.templating.contexts
-import charmhelpers.core.host
-import charmhelpers.core.hookenv
-import charmhelpers.fetch
-
-
-charm_dir = os.environ.get('CHARM_DIR', '')
-ansible_hosts_path = '/etc/ansible/hosts'
-# Ansible will automatically include any vars in the following
-# file in its inventory when run locally.
-ansible_vars_path = '/etc/ansible/host_vars/localhost'
-
-
-def install_ansible_support(from_ppa=True, ppa_location='ppa:rquillo/ansible'):
-    """Installs the ansible package.
-
-    By default it is installed from the `PPA`_ linked from
-    the ansible `website`_ or from a ppa specified by a charm config..
-
-    .. _PPA: https://launchpad.net/~rquillo/+archive/ansible
-    .. _website: http://docs.ansible.com/intro_installation.html#latest-releases-via-apt-ubuntu
-
-    If from_ppa is empty, you must ensure that the package is available
-    from a configured repository.
-    """
-    if from_ppa:
-        charmhelpers.fetch.add_source(ppa_location)
-        charmhelpers.fetch.apt_update(fatal=True)
-    charmhelpers.fetch.apt_install('ansible')
-    with open(ansible_hosts_path, 'w+') as hosts_file:
-        hosts_file.write('localhost ansible_connection=local')
-
-
-def apply_playbook(playbook, tags=None, extra_vars=None):
-    tags = tags or []
-    tags = ",".join(tags)
-    charmhelpers.contrib.templating.contexts.juju_state_to_yaml(
-        ansible_vars_path, namespace_separator='__',
-        allow_hyphens_in_keys=False, mode=(stat.S_IRUSR | stat.S_IWUSR))
-
-    # we want ansible's log output to be unbuffered
-    env = os.environ.copy()
-    env['PYTHONUNBUFFERED'] = "1"
-    call = [
-        'ansible-playbook',
-        '-c',
-        'local',
-        playbook,
-    ]
-    if tags:
-        call.extend(['--tags', '{}'.format(tags)])
-    if extra_vars:
-        extra = ["%s=%s" % (k, v) for k, v in extra_vars.items()]
-        call.extend(['--extra-vars', " ".join(extra)])
-    subprocess.check_call(call, env=env)
-
-
-class AnsibleHooks(charmhelpers.core.hookenv.Hooks):
-    """Run a playbook with the hook-name as the tag.
-
-    This helper builds on the standard hookenv.Hooks helper,
-    but additionally runs the playbook with the hook-name specified
-    using --tags (ie. running all the tasks tagged with the hook-name).
-
-    Example::
-
-        hooks = AnsibleHooks(playbook_path='playbooks/my_machine_state.yaml')
-
-        # All the tasks within my_machine_state.yaml tagged with 'install'
-        # will be run automatically after do_custom_work()
-        @hooks.hook()
-        def install():
-            do_custom_work()
-
-        # For most of your hooks, you won't need to do anything other
-        # than run the tagged tasks for the hook:
-        @hooks.hook('config-changed', 'start', 'stop')
-        def just_use_playbook():
-            pass
-
-        # As a convenience, you can avoid the above noop function by specifying
-        # the hooks which are handled by ansible-only and they'll be registered
-        # for you:
-        # hooks = AnsibleHooks(
-        #     'playbooks/my_machine_state.yaml',
-        #     default_hooks=['config-changed', 'start', 'stop'])
-
-        if __name__ == "__main__":
-            # execute a hook based on the name the program is called by
-            hooks.execute(sys.argv)
-
-    """
-
-    def __init__(self, playbook_path, default_hooks=None):
-        """Register any hooks handled by ansible."""
-        super(AnsibleHooks, self).__init__()
-
-        self._actions = {}
-        self.playbook_path = playbook_path
-
-        default_hooks = default_hooks or []
-
-        def noop(*args, **kwargs):
-            pass
-
-        for hook in default_hooks:
-            self.register(hook, noop)
-
-    def register_action(self, name, function):
-        """Register a hook"""
-        self._actions[name] = function
-
-    def execute(self, args):
-        """Execute the hook followed by the playbook using the hook as tag."""
-        hook_name = os.path.basename(args[0])
-        extra_vars = None
-        if hook_name in self._actions:
-            extra_vars = self._actions[hook_name](args[1:])
-        else:
-            super(AnsibleHooks, self).execute(args)
-
-        charmhelpers.contrib.ansible.apply_playbook(
-            self.playbook_path, tags=[hook_name], extra_vars=extra_vars)
-
-    def action(self, *action_names):
-        """Decorator, registering them as actions"""
-        def action_wrapper(decorated):
-
-            @functools.wraps(decorated)
-            def wrapper(argv):
-                kwargs = dict(arg.split('=') for arg in argv)
-                try:
-                    return decorated(**kwargs)
-                except TypeError as e:
-                    if decorated.__doc__:
-                        e.args += (decorated.__doc__,)
-                    raise
-
-            self.register_action(decorated.__name__, wrapper)
-            if '_' in decorated.__name__:
-                self.register_action(
-                    decorated.__name__.replace('_', '-'), wrapper)
-
-            return wrapper
-
-        return action_wrapper

=== removed directory 'hooks/charmhelpers/contrib/benchmark'
=== removed file 'hooks/charmhelpers/contrib/benchmark/__init__.py'
--- hooks/charmhelpers/contrib/benchmark/__init__.py	2015-07-29 18:07:31 +0000
+++ hooks/charmhelpers/contrib/benchmark/__init__.py	1970-01-01 00:00:00 +0000
@@ -1,126 +0,0 @@
-# Copyright 2014-2015 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-import subprocess
-import time
-import os
-from distutils.spawn import find_executable
-
-from charmhelpers.core.hookenv import (
-    in_relation_hook,
-    relation_ids,
-    relation_set,
-    relation_get,
-)
-
-
-def action_set(key, val):
-    if find_executable('action-set'):
-        action_cmd = ['action-set']
-
-        if isinstance(val, dict):
-            for k, v in iter(val.items()):
-                action_set('%s.%s' % (key, k), v)
-            return True
-
-        action_cmd.append('%s=%s' % (key, val))
-        subprocess.check_call(action_cmd)
-        return True
-    return False
-
-
-class Benchmark():
-    """
-    Helper class for the `benchmark` interface.
-
-    :param list actions: Define the actions that are also benchmarks
-
-    From inside the benchmark-relation-changed hook, you would
-    Benchmark(['memory', 'cpu', 'disk', 'smoke', 'custom'])
-
-    Examples:
-
-        siege = Benchmark(['siege'])
-        siege.start()
-        [... run siege ...]
-        # The higher the score, the better the benchmark
-        siege.set_composite_score(16.70, 'trans/sec', 'desc')
-        siege.finish()
-
-
-    """
-
-    BENCHMARK_CONF = '/etc/benchmark.conf'  # Replaced in testing
-
-    required_keys = [
-        'hostname',
-        'port',
-        'graphite_port',
-        'graphite_endpoint',
-        'api_port'
-    ]
-
-    def __init__(self, benchmarks=None):
-        if in_relation_hook():
-            if benchmarks is not None:
-                for rid in sorted(relation_ids('benchmark')):
-                    relation_set(relation_id=rid, relation_settings={
-                        'benchmarks': ",".join(benchmarks)
-                    })
-
-            # Check the relation data
-            config = {}
-            for key in self.required_keys:
-                val = relation_get(key)
-                if val is not None:
-                    config[key] = val
-                else:
-                    # We don't have all of the required keys
-                    config = {}
-                    break
-
-            if len(config):
-                with open(self.BENCHMARK_CONF, 'w') as f:
-                    for key, val in iter(config.items()):
-                        f.write("%s=%s\n" % (key, val))
-
-    @staticmethod
-    def start():
-        action_set('meta.start', time.strftime('%Y-%m-%dT%H:%M:%SZ'))
-
-        """
-        If the collectd charm is also installed, tell it to send a snapshot
-        of the current profile data.
-        """
-        COLLECT_PROFILE_DATA = '/usr/local/bin/collect-profile-data'
-        if os.path.exists(COLLECT_PROFILE_DATA):
-            subprocess.check_output([COLLECT_PROFILE_DATA])
-
-    @staticmethod
-    def finish():
-        action_set('meta.stop', time.strftime('%Y-%m-%dT%H:%M:%SZ'))
-
-    @staticmethod
-    def set_composite_score(value, units, direction='asc'):
-        """
-        Set the composite score for a benchmark run. This is a single number
-        representative of the benchmark results. This could be the most
-        important metric, or an amalgamation of metric scores.
-        """
-        return action_set(
-            "meta.composite",
-            {'value': value, 'units': units, 'direction': direction}
-        )

=== removed directory 'hooks/charmhelpers/contrib/charmhelpers'
=== removed file 'hooks/charmhelpers/contrib/charmhelpers/__init__.py'
--- hooks/charmhelpers/contrib/charmhelpers/__init__.py	2015-07-29 18:07:31 +0000
+++ hooks/charmhelpers/contrib/charmhelpers/__init__.py	1970-01-01 00:00:00 +0000
@@ -1,208 +0,0 @@
-# Copyright 2014-2015 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-# Copyright 2012 Canonical Ltd.  This software is licensed under the
-# GNU Affero General Public License version 3 (see the file LICENSE).
-
-import warnings
-warnings.warn("contrib.charmhelpers is deprecated", DeprecationWarning)  # noqa
-
-import operator
-import tempfile
-import time
-import yaml
-import subprocess
-
-import six
-if six.PY3:
-    from urllib.request import urlopen
-    from urllib.error import (HTTPError, URLError)
-else:
-    from urllib2 import (urlopen, HTTPError, URLError)
-
-"""Helper functions for writing Juju charms in Python."""
-
-__metaclass__ = type
-__all__ = [
-    # 'get_config',             # core.hookenv.config()
-    # 'log',                    # core.hookenv.log()
-    # 'log_entry',              # core.hookenv.log()
-    # 'log_exit',               # core.hookenv.log()
-    # 'relation_get',           # core.hookenv.relation_get()
-    # 'relation_set',           # core.hookenv.relation_set()
-    # 'relation_ids',           # core.hookenv.relation_ids()
-    # 'relation_list',          # core.hookenv.relation_units()
-    # 'config_get',             # core.hookenv.config()
-    # 'unit_get',               # core.hookenv.unit_get()
-    # 'open_port',              # core.hookenv.open_port()
-    # 'close_port',             # core.hookenv.close_port()
-    # 'service_control',        # core.host.service()
-    'unit_info',              # client-side, NOT IMPLEMENTED
-    'wait_for_machine',       # client-side, NOT IMPLEMENTED
-    'wait_for_page_contents',  # client-side, NOT IMPLEMENTED
-    'wait_for_relation',      # client-side, NOT IMPLEMENTED
-    'wait_for_unit',          # client-side, NOT IMPLEMENTED
-]
-
-
-SLEEP_AMOUNT = 0.1
-
-
-# We create a juju_status Command here because it makes testing much,
-# much easier.
-def juju_status():
-    subprocess.check_call(['juju', 'status'])
-
-# re-implemented as charmhelpers.fetch.configure_sources()
-# def configure_source(update=False):
-#    source = config_get('source')
-#    if ((source.startswith('ppa:') or
-#         source.startswith('cloud:') or
-#         source.startswith('http:'))):
-#        run('add-apt-repository', source)
-#    if source.startswith("http:"):
-#        run('apt-key', 'import', config_get('key'))
-#    if update:
-#        run('apt-get', 'update')
-
-
-# DEPRECATED: client-side only
-def make_charm_config_file(charm_config):
-    charm_config_file = tempfile.NamedTemporaryFile(mode='w+')
-    charm_config_file.write(yaml.dump(charm_config))
-    charm_config_file.flush()
-    # The NamedTemporaryFile instance is returned instead of just the name
-    # because we want to take advantage of garbage collection-triggered
-    # deletion of the temp file when it goes out of scope in the caller.
-    return charm_config_file
-
-
-# DEPRECATED: client-side only
-def unit_info(service_name, item_name, data=None, unit=None):
-    if data is None:
-        data = yaml.safe_load(juju_status())
-    service = data['services'].get(service_name)
-    if service is None:
-        # XXX 2012-02-08 gmb:
-        #     This allows us to cope with the race condition that we
-        #     have between deploying a service and having it come up in
-        #     `juju status`. We could probably do with cleaning it up so
-        #     that it fails a bit more noisily after a while.
-        return ''
-    units = service['units']
-    if unit is not None:
-        item = units[unit][item_name]
-    else:
-        # It might seem odd to sort the units here, but we do it to
-        # ensure that when no unit is specified, the first unit for the
-        # service (or at least the one with the lowest number) is the
-        # one whose data gets returned.
-        sorted_unit_names = sorted(units.keys())
-        item = units[sorted_unit_names[0]][item_name]
-    return item
-
-
-# DEPRECATED: client-side only
-def get_machine_data():
-    return yaml.safe_load(juju_status())['machines']
-
-
-# DEPRECATED: client-side only
-def wait_for_machine(num_machines=1, timeout=300):
-    """Wait `timeout` seconds for `num_machines` machines to come up.
-
-    This wait_for... function can be called by other wait_for functions
-    whose timeouts might be too short in situations where only a bare
-    Juju setup has been bootstrapped.
-
-    :return: A tuple of (num_machines, time_taken). This is used for
-             testing.
-    """
-    # You may think this is a hack, and you'd be right. The easiest way
-    # to tell what environment we're working in (LXC vs EC2) is to check
-    # the dns-name of the first machine. If it's localhost we're in LXC
-    # and we can just return here.
-    if get_machine_data()[0]['dns-name'] == 'localhost':
-        return 1, 0
-    start_time = time.time()
-    while True:
-        # Drop the first machine, since it's the Zookeeper and that's
-        # not a machine that we need to wait for. This will only work
-        # for EC2 environments, which is why we return early above if
-        # we're in LXC.
-        machine_data = get_machine_data()
-        non_zookeeper_machines = [
-            machine_data[key] for key in list(machine_data.keys())[1:]]
-        if len(non_zookeeper_machines) >= num_machines:
-            all_machines_running = True
-            for machine in non_zookeeper_machines:
-                if machine.get('instance-state') != 'running':
-                    all_machines_running = False
-                    break
-            if all_machines_running:
-                break
-        if time.time() - start_time >= timeout:
-            raise RuntimeError('timeout waiting for service to start')
-        time.sleep(SLEEP_AMOUNT)
-    return num_machines, time.time() - start_time
-
-
-# DEPRECATED: client-side only
-def wait_for_unit(service_name, timeout=480):
-    """Wait `timeout` seconds for a given service name to come up."""
-    wait_for_machine(num_machines=1)
-    start_time = time.time()
-    while True:
-        state = unit_info(service_name, 'agent-state')
-        if 'error' in state or state == 'started':
-            break
-        if time.time() - start_time >= timeout:
-            raise RuntimeError('timeout waiting for service to start')
-        time.sleep(SLEEP_AMOUNT)
-    if state != 'started':
-        raise RuntimeError('unit did not start, agent-state: ' + state)
-
-
-# DEPRECATED: client-side only
-def wait_for_relation(service_name, relation_name, timeout=120):
-    """Wait `timeout` seconds for a given relation to come up."""
-    start_time = time.time()
-    while True:
-        relation = unit_info(service_name, 'relations').get(relation_name)
-        if relation is not None and relation['state'] == 'up':
-            break
-        if time.time() - start_time >= timeout:
-            raise RuntimeError('timeout waiting for relation to be up')
-        time.sleep(SLEEP_AMOUNT)
-
-
-# DEPRECATED: client-side only
-def wait_for_page_contents(url, contents, timeout=120, validate=None):
-    if validate is None:
-        validate = operator.contains
-    start_time = time.time()
-    while True:
-        try:
-            stream = urlopen(url)
-        except (HTTPError, URLError):
-            pass
-        else:
-            page = stream.read()
-            if validate(page, contents):
-                return page
-        if time.time() - start_time >= timeout:
-            raise RuntimeError('timeout waiting for contents of ' + url)
-        time.sleep(SLEEP_AMOUNT)

=== removed directory 'hooks/charmhelpers/contrib/charmsupport'
=== removed file 'hooks/charmhelpers/contrib/charmsupport/__init__.py'
--- hooks/charmhelpers/contrib/charmsupport/__init__.py	2015-07-29 18:07:31 +0000
+++ hooks/charmhelpers/contrib/charmsupport/__init__.py	1970-01-01 00:00:00 +0000
@@ -1,15 +0,0 @@
-# Copyright 2014-2015 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.

=== removed file 'hooks/charmhelpers/contrib/charmsupport/nrpe.py'
--- hooks/charmhelpers/contrib/charmsupport/nrpe.py	2016-01-30 22:40:26 +0000
+++ hooks/charmhelpers/contrib/charmsupport/nrpe.py	1970-01-01 00:00:00 +0000
@@ -1,398 +0,0 @@
-# Copyright 2014-2015 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-"""Compatibility with the nrpe-external-master charm"""
-# Copyright 2012 Canonical Ltd.
-#
-# Authors:
-#  Matthew Wedgwood <[e-mail]>
-
-import subprocess
-import pwd
-import grp
-import os
-import glob
-import shutil
-import re
-import shlex
-import yaml
-
-from charmhelpers.core.hookenv import (
-    config,
-    local_unit,
-    log,
-    relation_ids,
-    relation_set,
-    relations_of_type,
-)
-
-from charmhelpers.core.host import service
-
-# This module adds compatibility with the nrpe-external-master and plain nrpe
-# subordinate charms. To use it in your charm:
-#
-# 1. Update metadata.yaml
-#
-#   provides:
-#     (...)
-#     nrpe-external-master:
-#       interface: nrpe-external-master
-#       scope: container
-#
-#   and/or
-#
-#   provides:
-#     (...)
-#     local-monitors:
-#       interface: local-monitors
-#       scope: container
-
-#
-# 2. Add the following to config.yaml
-#
-#    nagios_context:
-#      default: "juju"
-#      type: string
-#      description: |
-#        Used by the nrpe subordinate charms.
-#        A string that will be prepended to instance name to set the host name
-#        in nagios. So for instance the hostname would be something like:
-#            juju-myservice-0
-#        If you're running multiple environments with the same services in them
-#        this allows you to differentiate between them.
-#    nagios_servicegroups:
-#      default: ""
-#      type: string
-#      description: |
-#        A comma-separated list of nagios servicegroups.
-#        If left empty, the nagios_context will be used as the servicegroup
-#
-# 3. Add custom checks (Nagios plugins) to files/nrpe-external-master
-#
-# 4. Update your hooks.py with something like this:
-#
-#    from charmsupport.nrpe import NRPE
-#    (...)
-#    def update_nrpe_config():
-#        nrpe_compat = NRPE()
-#        nrpe_compat.add_check(
-#            shortname = "myservice",
-#            description = "Check MyService",
-#            check_cmd = "check_http -w 2 -c 10 http://localhost"
-#            )
-#        nrpe_compat.add_check(
-#            "myservice_other",
-#            "Check for widget failures",
-#            check_cmd = "/srv/myapp/scripts/widget_check"
-#            )
-#        nrpe_compat.write()
-#
-#    def config_changed():
-#        (...)
-#        update_nrpe_config()
-#
-#    def nrpe_external_master_relation_changed():
-#        update_nrpe_config()
-#
-#    def local_monitors_relation_changed():
-#        update_nrpe_config()
-#
-# 5. ln -s hooks.py nrpe-external-master-relation-changed
-#    ln -s hooks.py local-monitors-relation-changed
-
-
-class CheckException(Exception):
-    pass
-
-
-class Check(object):
-    shortname_re = '[A-Za-z0-9-_]+$'
-    service_template = ("""
-#---------------------------------------------------
-# This file is Juju managed
-#---------------------------------------------------
-define service {{
-    use                             active-service
-    host_name                       {nagios_hostname}
-    service_description             {nagios_hostname}[{shortname}] """
-                        """{description}
-    check_command                   check_nrpe!{command}
-    servicegroups                   {nagios_servicegroup}
-}}
-""")
-
-    def __init__(self, shortname, description, check_cmd):
-        super(Check, self).__init__()
-        # XXX: could be better to calculate this from the service name
-        if not re.match(self.shortname_re, shortname):
-            raise CheckException("shortname must match {}".format(
-                Check.shortname_re))
-        self.shortname = shortname
-        self.command = "check_{}".format(shortname)
-        # Note: a set of invalid characters is defined by the
-        # Nagios server config
-        # The default is: illegal_object_name_chars=`~!$%^&*"|'<>?,()=
-        self.description = description
-        self.check_cmd = self._locate_cmd(check_cmd)
-
-    def _get_check_filename(self):
-        return os.path.join(NRPE.nrpe_confdir, '{}.cfg'.format(self.command))
-
-    def _get_service_filename(self, hostname):
-        return os.path.join(NRPE.nagios_exportdir,
-                            'service__{}_{}.cfg'.format(hostname, self.command))
-
-    def _locate_cmd(self, check_cmd):
-        search_path = (
-            '/usr/lib/nagios/plugins',
-            '/usr/local/lib/nagios/plugins',
-        )
-        parts = shlex.split(check_cmd)
-        for path in search_path:
-            if os.path.exists(os.path.join(path, parts[0])):
-                command = os.path.join(path, parts[0])
-                if len(parts) > 1:
-                    command += " " + " ".join(parts[1:])
-                return command
-        log('Check command not found: {}'.format(parts[0]))
-        return ''
-
-    def _remove_service_files(self):
-        if not os.path.exists(NRPE.nagios_exportdir):
-            return
-        for f in os.listdir(NRPE.nagios_exportdir):
-            if f.endswith('_{}.cfg'.format(self.command)):
-                os.remove(os.path.join(NRPE.nagios_exportdir, f))
-
-    def remove(self, hostname):
-        nrpe_check_file = self._get_check_filename()
-        if os.path.exists(nrpe_check_file):
-            os.remove(nrpe_check_file)
-        self._remove_service_files()
-
-    def write(self, nagios_context, hostname, nagios_servicegroups):
-        nrpe_check_file = self._get_check_filename()
-        with open(nrpe_check_file, 'w') as nrpe_check_config:
-            nrpe_check_config.write("# check {}\n".format(self.shortname))
-            nrpe_check_config.write("command[{}]={}\n".format(
-                self.command, self.check_cmd))
-
-        if not os.path.exists(NRPE.nagios_exportdir):
-            log('Not writing service config as {} is not accessible'.format(
-                NRPE.nagios_exportdir))
-        else:
-            self.write_service_config(nagios_context, hostname,
-                                      nagios_servicegroups)
-
-    def write_service_config(self, nagios_context, hostname,
-                             nagios_servicegroups):
-        self._remove_service_files()
-
-        templ_vars = {
-            'nagios_hostname': hostname,
-            'nagios_servicegroup': nagios_servicegroups,
-            'description': self.description,
-            'shortname': self.shortname,
-            'command': self.command,
-        }
-        nrpe_service_text = Check.service_template.format(**templ_vars)
-        nrpe_service_file = self._get_service_filename(hostname)
-        with open(nrpe_service_file, 'w') as nrpe_service_config:
-            nrpe_service_config.write(str(nrpe_service_text))
-
-    def run(self):
-        subprocess.call(self.check_cmd)
-
-
-class NRPE(object):
-    nagios_logdir = '/var/log/nagios'
-    nagios_exportdir = '/var/lib/nagios/export'
-    nrpe_confdir = '/etc/nagios/nrpe.d'
-
-    def __init__(self, hostname=None):
-        super(NRPE, self).__init__()
-        self.config = config()
-        self.nagios_context = self.config['nagios_context']
-        if 'nagios_servicegroups' in self.config and self.config['nagios_servicegroups']:
-            self.nagios_servicegroups = self.config['nagios_servicegroups']
-        else:
-            self.nagios_servicegroups = self.nagios_context
-        self.unit_name = local_unit().replace('/', '-')
-        if hostname:
-            self.hostname = hostname
-        else:
-            nagios_hostname = get_nagios_hostname()
-            if nagios_hostname:
-                self.hostname = nagios_hostname
-            else:
-                self.hostname = "{}-{}".format(self.nagios_context, self.unit_name)
-        self.checks = []
-
-    def add_check(self, *args, **kwargs):
-        self.checks.append(Check(*args, **kwargs))
-
-    def remove_check(self, *args, **kwargs):
-        if kwargs.get('shortname') is None:
-            raise ValueError('shortname of check must be specified')
-
-        # Use sensible defaults if they're not specified - these are not
-        # actually used during removal, but they're required for constructing
-        # the Check object; check_disk is chosen because it's part of the
-        # nagios-plugins-basic package.
-        if kwargs.get('check_cmd') is None:
-            kwargs['check_cmd'] = 'check_disk'
-        if kwargs.get('description') is None:
-            kwargs['description'] = ''
-
-        check = Check(*args, **kwargs)
-        check.remove(self.hostname)
-
-    def write(self):
-        try:
-            nagios_uid = pwd.getpwnam('nagios').pw_uid
-            nagios_gid = grp.getgrnam('nagios').gr_gid
-        except:
-            log("Nagios user not set up, nrpe checks not updated")
-            return
-
-        if not os.path.exists(NRPE.nagios_logdir):
-            os.mkdir(NRPE.nagios_logdir)
-            os.chown(NRPE.nagios_logdir, nagios_uid, nagios_gid)
-
-        nrpe_monitors = {}
-        monitors = {"monitors": {"remote": {"nrpe": nrpe_monitors}}}
-        for nrpecheck in self.checks:
-            nrpecheck.write(self.nagios_context, self.hostname,
-                            self.nagios_servicegroups)
-            nrpe_monitors[nrpecheck.shortname] = {
-                "command": nrpecheck.command,
-            }
-
-        service('restart', 'nagios-nrpe-server')
-
-        monitor_ids = relation_ids("local-monitors") + \
-            relation_ids("nrpe-external-master")
-        for rid in monitor_ids:
-            relation_set(relation_id=rid, monitors=yaml.dump(monitors))
-
-
-def get_nagios_hostcontext(relation_name='nrpe-external-master'):
-    """
-    Query relation with nrpe subordinate, return the nagios_host_context
-
-    :param str relation_name: Name of relation nrpe sub joined to
-    """
-    for rel in relations_of_type(relation_name):
-        if 'nagios_host_context' in rel:
-            return rel['nagios_host_context']
-
-
-def get_nagios_hostname(relation_name='nrpe-external-master'):
-    """
-    Query relation with nrpe subordinate, return the nagios_hostname
-
-    :param str relation_name: Name of relation nrpe sub joined to
-    """
-    for rel in relations_of_type(relation_name):
-        if 'nagios_hostname' in rel:
-            return rel['nagios_hostname']
-
-
-def get_nagios_unit_name(relation_name='nrpe-external-master'):
-    """
-    Return the nagios unit name prepended with host_context if needed
-
-    :param str relation_name: Name of relation nrpe sub joined to
-    """
-    host_context = get_nagios_hostcontext(relation_name)
-    if host_context:
-        unit = "%s:%s" % (host_context, local_unit())
-    else:
-        unit = local_unit()
-    return unit
-
-
-def add_init_service_checks(nrpe, services, unit_name):
-    """
-    Add checks for each service in list
-
-    :param NRPE nrpe: NRPE object to add check to
-    :param list services: List of services to check
-    :param str unit_name: Unit name to use in check description
-    """
-    for svc in services:
-        upstart_init = '/etc/init/%s.conf' % svc
-        sysv_init = '/etc/init.d/%s' % svc
-        if os.path.exists(upstart_init):
-            # Don't add a check for these services from neutron-gateway
-            if svc not in ['ext-port', 'os-charm-phy-nic-mtu']:
-                nrpe.add_check(
-                    shortname=svc,
-                    description='process check {%s}' % unit_name,
-                    check_cmd='check_upstart_job %s' % svc
-                )
-        elif os.path.exists(sysv_init):
-            cronpath = '/etc/cron.d/nagios-service-check-%s' % svc
-            cron_file = ('*/5 * * * * root '
-                         '/usr/local/lib/nagios/plugins/check_exit_status.pl '
-                         '-s /etc/init.d/%s status > '
-                         '/var/lib/nagios/service-check-%s.txt\n' % (svc,
-                                                                     svc)
-                         )
-            f = open(cronpath, 'w')
-            f.write(cron_file)
-            f.close()
-            nrpe.add_check(
-                shortname=svc,
-                description='process check {%s}' % unit_name,
-                check_cmd='check_status_file.py -f '
-                          '/var/lib/nagios/service-check-%s.txt' % svc,
-            )
-
-
-def copy_nrpe_checks():
-    """
-    Copy the nrpe checks into place
-
-    """
-    NAGIOS_PLUGINS = '/usr/local/lib/nagios/plugins'
-    nrpe_files_dir = os.path.join(os.getenv('CHARM_DIR'), 'hooks',
-                                  'charmhelpers', 'contrib', 'openstack',
-                                  'files')
-
-    if not os.path.exists(NAGIOS_PLUGINS):
-        os.makedirs(NAGIOS_PLUGINS)
-    for fname in glob.glob(os.path.join(nrpe_files_dir, "check_*")):
-        if os.path.isfile(fname):
-            shutil.copy2(fname,
-                         os.path.join(NAGIOS_PLUGINS, os.path.basename(fname)))
-
-
-def add_haproxy_checks(nrpe, unit_name):
-    """
-    Add checks for each service in list
-
-    :param NRPE nrpe: NRPE object to add check to
-    :param str unit_name: Unit name to use in check description
-    """
-    nrpe.add_check(
-        shortname='haproxy_servers',
-        description='Check HAProxy {%s}' % unit_name,
-        check_cmd='check_haproxy.sh')
-    nrpe.add_check(
-        shortname='haproxy_queue',
-        description='Check HAProxy queue depth {%s}' % unit_name,
-        check_cmd='check_haproxy_queue_depth.sh')

=== removed file 'hooks/charmhelpers/contrib/charmsupport/volumes.py'
--- hooks/charmhelpers/contrib/charmsupport/volumes.py	2015-07-29 18:07:31 +0000
+++ hooks/charmhelpers/contrib/charmsupport/volumes.py	1970-01-01 00:00:00 +0000
@@ -1,175 +0,0 @@
-# Copyright 2014-2015 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-'''
-Functions for managing volumes in juju units. One volume is supported per unit.
-Subordinates may have their own storage, provided it is on its own partition.
-
-Configuration stanzas::
-
-  volume-ephemeral:
-    type: boolean
-    default: true
-    description: >
-      If false, a volume is mounted as sepecified in "volume-map"
-      If true, ephemeral storage will be used, meaning that log data
-         will only exist as long as the machine. YOU HAVE BEEN WARNED.
-  volume-map:
-    type: string
-    default: {}
-    description: >
-      YAML map of units to device names, e.g:
-        "{ rsyslog/0: /dev/vdb, rsyslog/1: /dev/vdb }"
-      Service units will raise a configure-error if volume-ephemeral
-      is 'true' and no volume-map value is set. Use 'juju set' to set a
-      value and 'juju resolved' to complete configuration.
-
-Usage::
-
-    from charmsupport.volumes import configure_volume, VolumeConfigurationError
-    from charmsupport.hookenv import log, ERROR
-    def post_mount_hook():
-        stop_service('myservice')
-    def post_mount_hook():
-        start_service('myservice')
-
-    if __name__ == '__main__':
-        try:
-            configure_volume(before_change=pre_mount_hook,
-                             after_change=post_mount_hook)
-        except VolumeConfigurationError:
-            log('Storage could not be configured', ERROR)
-
-'''
-
-# XXX: Known limitations
-# - fstab is neither consulted nor updated
-
-import os
-from charmhelpers.core import hookenv
-from charmhelpers.core import host
-import yaml
-
-
-MOUNT_BASE = '/srv/juju/volumes'
-
-
-class VolumeConfigurationError(Exception):
-    '''Volume configuration data is missing or invalid'''
-    pass
-
-
-def get_config():
-    '''Gather and sanity-check volume configuration data'''
-    volume_config = {}
-    config = hookenv.config()
-
-    errors = False
-
-    if config.get('volume-ephemeral') in (True, 'True', 'true', 'Yes', 'yes'):
-        volume_config['ephemeral'] = True
-    else:
-        volume_config['ephemeral'] = False
-
-    try:
-        volume_map = yaml.safe_load(config.get('volume-map', '{}'))
-    except yaml.YAMLError as e:
-        hookenv.log("Error parsing YAML volume-map: {}".format(e),
-                    hookenv.ERROR)
-        errors = True
-    if volume_map is None:
-        # probably an empty string
-        volume_map = {}
-    elif not isinstance(volume_map, dict):
-        hookenv.log("Volume-map should be a dictionary, not {}".format(
-            type(volume_map)))
-        errors = True
-
-    volume_config['device'] = volume_map.get(os.environ['JUJU_UNIT_NAME'])
-    if volume_config['device'] and volume_config['ephemeral']:
-        # asked for ephemeral storage but also defined a volume ID
-        hookenv.log('A volume is defined for this unit, but ephemeral '
-                    'storage was requested', hookenv.ERROR)
-        errors = True
-    elif not volume_config['device'] and not volume_config['ephemeral']:
-        # asked for permanent storage but did not define volume ID
-        hookenv.log('Ephemeral storage was requested, but there is no volume '
-                    'defined for this unit.', hookenv.ERROR)
-        errors = True
-
-    unit_mount_name = hookenv.local_unit().replace('/', '-')
-    volume_config['mountpoint'] = os.path.join(MOUNT_BASE, unit_mount_name)
-
-    if errors:
-        return None
-    return volume_config
-
-
-def mount_volume(config):
-    if os.path.exists(config['mountpoint']):
-        if not os.path.isdir(config['mountpoint']):
-            hookenv.log('Not a directory: {}'.format(config['mountpoint']))
-            raise VolumeConfigurationError()
-    else:
-        host.mkdir(config['mountpoint'])
-    if os.path.ismount(config['mountpoint']):
-        unmount_volume(config)
-    if not host.mount(config['device'], config['mountpoint'], persist=True):
-        raise VolumeConfigurationError()
-
-
-def unmount_volume(config):
-    if os.path.ismount(config['mountpoint']):
-        if not host.umount(config['mountpoint'], persist=True):
-            raise VolumeConfigurationError()
-
-
-def managed_mounts():
-    '''List of all mounted managed volumes'''
-    return filter(lambda mount: mount[0].startswith(MOUNT_BASE), host.mounts())
-
-
-def configure_volume(before_change=lambda: None, after_change=lambda: None):
-    '''Set up storage (or don't) according to the charm's volume configuration.
-       Returns the mount point or "ephemeral". before_change and after_change
-       are optional functions to be called if the volume configuration changes.
-    '''
-
-    config = get_config()
-    if not config:
-        hookenv.log('Failed to read volume configuration', hookenv.CRITICAL)
-        raise VolumeConfigurationError()
-
-    if config['ephemeral']:
-        if os.path.ismount(config['mountpoint']):
-            before_change()
-            unmount_volume(config)
-            after_change()
-        return 'ephemeral'
-    else:
-        # persistent storage
-        if os.path.ismount(config['mountpoint']):
-            mounts = dict(managed_mounts())
-            if mounts.get(config['mountpoint']) != config['device']:
-                before_change()
-                unmount_volume(config)
-                mount_volume(config)
-                after_change()
-        else:
-            before_change()
-            mount_volume(config)
-            after_change()
-        return config['mountpoint']

=== removed directory 'hooks/charmhelpers/contrib/database'
=== removed file 'hooks/charmhelpers/contrib/database/__init__.py'
=== removed file 'hooks/charmhelpers/contrib/database/mysql.py'
--- hooks/charmhelpers/contrib/database/mysql.py	2015-07-29 18:07:31 +0000
+++ hooks/charmhelpers/contrib/database/mysql.py	1970-01-01 00:00:00 +0000
@@ -1,412 +0,0 @@
-"""Helper for working with a MySQL database"""
-import json
-import re
-import sys
-import platform
-import os
-import glob
-
-# from string import upper
-
-from charmhelpers.core.host import (
-    mkdir,
-    pwgen,
-    write_file
-)
-from charmhelpers.core.hookenv import (
-    config as config_get,
-    relation_get,
-    related_units,
-    unit_get,
-    log,
-    DEBUG,
-    INFO,
-    WARNING,
-)
-from charmhelpers.fetch import (
-    apt_install,
-    apt_update,
-    filter_installed_packages,
-)
-from charmhelpers.contrib.peerstorage import (
-    peer_store,
-    peer_retrieve,
-)
-from charmhelpers.contrib.network.ip import get_host_ip
-
-try:
-    import MySQLdb
-except ImportError:
-    apt_update(fatal=True)
-    apt_install(filter_installed_packages(['python-mysqldb']), fatal=True)
-    import MySQLdb
-
-
-class MySQLHelper(object):
-
-    def __init__(self, rpasswdf_template, upasswdf_template, host='localhost',
-                 migrate_passwd_to_peer_relation=True,
-                 delete_ondisk_passwd_file=True):
-        self.host = host
-        # Password file path templates
-        self.root_passwd_file_template = rpasswdf_template
-        self.user_passwd_file_template = upasswdf_template
-
-        self.migrate_passwd_to_peer_relation = migrate_passwd_to_peer_relation
-        # If we migrate we have the option to delete local copy of root passwd
-        self.delete_ondisk_passwd_file = delete_ondisk_passwd_file
-
-    def connect(self, user='root', password=None):
-        log("Opening db connection for %s@%s" % (user, self.host), level=DEBUG)
-        self.connection = MySQLdb.connect(user=user, host=self.host,
-                                          passwd=password)
-
-    def database_exists(self, db_name):
-        cursor = self.connection.cursor()
-        try:
-            cursor.execute("SHOW DATABASES")
-            databases = [i[0] for i in cursor.fetchall()]
-        finally:
-            cursor.close()
-
-        return db_name in databases
-
-    def create_database(self, db_name):
-        cursor = self.connection.cursor()
-        try:
-            cursor.execute("CREATE DATABASE {} CHARACTER SET UTF8"
-                           .format(db_name))
-        finally:
-            cursor.close()
-
-    def grant_exists(self, db_name, db_user, remote_ip):
-        cursor = self.connection.cursor()
-        priv_string = "GRANT ALL PRIVILEGES ON `{}`.* " \
-                      "TO '{}'@'{}'".format(db_name, db_user, remote_ip)
-        try:
-            cursor.execute("SHOW GRANTS for '{}'@'{}'".format(db_user,
-                                                              remote_ip))
-            grants = [i[0] for i in cursor.fetchall()]
-        except MySQLdb.OperationalError:
-            return False
-        finally:
-            cursor.close()
-
-        # TODO: review for different grants
-        return priv_string in grants
-
-    def create_grant(self, db_name, db_user, remote_ip, password):
-        cursor = self.connection.cursor()
-        try:
-            # TODO: review for different grants
-            cursor.execute("GRANT ALL PRIVILEGES ON {}.* TO '{}'@'{}' "
-                           "IDENTIFIED BY '{}'".format(db_name,
-                                                       db_user,
-                                                       remote_ip,
-                                                       password))
-        finally:
-            cursor.close()
-
-    def create_admin_grant(self, db_user, remote_ip, password):
-        cursor = self.connection.cursor()
-        try:
-            cursor.execute("GRANT ALL PRIVILEGES ON *.* TO '{}'@'{}' "
-                           "IDENTIFIED BY '{}'".format(db_user,
-                                                       remote_ip,
-                                                       password))
-        finally:
-            cursor.close()
-
-    def cleanup_grant(self, db_user, remote_ip):
-        cursor = self.connection.cursor()
-        try:
-            cursor.execute("DROP FROM mysql.user WHERE user='{}' "
-                           "AND HOST='{}'".format(db_user,
-                                                  remote_ip))
-        finally:
-            cursor.close()
-
-    def execute(self, sql):
-        """Execute arbitary SQL against the database."""
-        cursor = self.connection.cursor()
-        try:
-            cursor.execute(sql)
-        finally:
-            cursor.close()
-
-    def migrate_passwords_to_peer_relation(self, excludes=None):
-        """Migrate any passwords storage on disk to cluster peer relation."""
-        dirname = os.path.dirname(self.root_passwd_file_template)
-        path = os.path.join(dirname, '*.passwd')
-        for f in glob.glob(path):
-            if excludes and f in excludes:
-                log("Excluding %s from peer migration" % (f), level=DEBUG)
-                continue
-
-            key = os.path.basename(f)
-            with open(f, 'r') as passwd:
-                _value = passwd.read().strip()
-
-            try:
-                peer_store(key, _value)
-
-                if self.delete_ondisk_passwd_file:
-                    os.unlink(f)
-            except ValueError:
-                # NOTE cluster relation not yet ready - skip for now
-                pass
-
-    def get_mysql_password_on_disk(self, username=None, password=None):
-        """Retrieve, generate or store a mysql password for the provided
-        username on disk."""
-        if username:
-            template = self.user_passwd_file_template
-            passwd_file = template.format(username)
-        else:
-            passwd_file = self.root_passwd_file_template
-
-        _password = None
-        if os.path.exists(passwd_file):
-            log("Using existing password file '%s'" % passwd_file, level=DEBUG)
-            with open(passwd_file, 'r') as passwd:
-                _password = passwd.read().strip()
-        else:
-            log("Generating new password file '%s'" % passwd_file, level=DEBUG)
-            if not os.path.isdir(os.path.dirname(passwd_file)):
-                # NOTE: need to ensure this is not mysql root dir (which needs
-                # to be mysql readable)
-                mkdir(os.path.dirname(passwd_file), owner='root', group='root',
-                      perms=0o770)
-                # Force permissions - for some reason the chmod in makedirs
-                # fails
-                os.chmod(os.path.dirname(passwd_file), 0o770)
-
-            _password = password or pwgen(length=32)
-            write_file(passwd_file, _password, owner='root', group='root',
-                       perms=0o660)
-
-        return _password
-
-    def passwd_keys(self, username):
-        """Generator to return keys used to store passwords in peer store.
-
-        NOTE: we support both legacy and new format to support mysql
-        charm prior to refactor. This is necessary to avoid LP 1451890.
-        """
-        keys = []
-        if username == 'mysql':
-            log("Bad username '%s'" % (username), level=WARNING)
-
-        if username:
-            # IMPORTANT: *newer* format must be returned first
-            keys.append('mysql-%s.passwd' % (username))
-            keys.append('%s.passwd' % (username))
-        else:
-            keys.append('mysql.passwd')
-
-        for key in keys:
-            yield key
-
-    def get_mysql_password(self, username=None, password=None):
-        """Retrieve, generate or store a mysql password for the provided
-        username using peer relation cluster."""
-        excludes = []
-
-        # First check peer relation.
-        try:
-            for key in self.passwd_keys(username):
-                _password = peer_retrieve(key)
-                if _password:
-                    break
-
-            # If root password available don't update peer relation from local
-            if _password and not username:
-                excludes.append(self.root_passwd_file_template)
-
-        except ValueError:
-            # cluster relation is not yet started; use on-disk
-            _password = None
-
-        # If none available, generate new one
-        if not _password:
-            _password = self.get_mysql_password_on_disk(username, password)
-
-        # Put on wire if required
-        if self.migrate_passwd_to_peer_relation:
-            self.migrate_passwords_to_peer_relation(excludes=excludes)
-
-        return _password
-
-    def get_mysql_root_password(self, password=None):
-        """Retrieve or generate mysql root password for service units."""
-        return self.get_mysql_password(username=None, password=password)
-
-    def normalize_address(self, hostname):
-        """Ensure that address returned is an IP address (i.e. not fqdn)"""
-        if config_get('prefer-ipv6'):
-            # TODO: add support for ipv6 dns
-            return hostname
-
-        if hostname != unit_get('private-address'):
-            return get_host_ip(hostname, fallback=hostname)
-
-        # Otherwise assume localhost
-        return '127.0.0.1'
-
-    def get_allowed_units(self, database, username, relation_id=None):
-        """Get list of units with access grants for database with username.
-
-        This is typically used to provide shared-db relations with a list of
-        which units have been granted access to the given database.
-        """
-        self.connect(password=self.get_mysql_root_password())
-        allowed_units = set()
-        for unit in related_units(relation_id):
-            settings = relation_get(rid=relation_id, unit=unit)
-            # First check for setting with prefix, then without
-            for attr in ["%s_hostname" % (database), 'hostname']:
-                hosts = settings.get(attr, None)
-                if hosts:
-                    break
-
-            if hosts:
-                # hostname can be json-encoded list of hostnames
-                try:
-                    hosts = json.loads(hosts)
-                except ValueError:
-                    hosts = [hosts]
-            else:
-                hosts = [settings['private-address']]
-
-            if hosts:
-                for host in hosts:
-                    host = self.normalize_address(host)
-                    if self.grant_exists(database, username, host):
-                        log("Grant exists for host '%s' on db '%s'" %
-                            (host, database), level=DEBUG)
-                        if unit not in allowed_units:
-                            allowed_units.add(unit)
-                    else:
-                        log("Grant does NOT exist for host '%s' on db '%s'" %
-                            (host, database), level=DEBUG)
-            else:
-                log("No hosts found for grant check", level=INFO)
-
-        return allowed_units
-
-    def configure_db(self, hostname, database, username, admin=False):
-        """Configure access to database for username from hostname."""
-        self.connect(password=self.get_mysql_root_password())
-        if not self.database_exists(database):
-            self.create_database(database)
-
-        remote_ip = self.normalize_address(hostname)
-        password = self.get_mysql_password(username)
-        if not self.grant_exists(database, username, remote_ip):
-            if not admin:
-                self.create_grant(database, username, remote_ip, password)
-            else:
-                self.create_admin_grant(username, remote_ip, password)
-
-        return password
-
-
-class PerconaClusterHelper(object):
-
-    # Going for the biggest page size to avoid wasted bytes.
-    # InnoDB page size is 16MB
-
-    DEFAULT_PAGE_SIZE = 16 * 1024 * 1024
-    DEFAULT_INNODB_BUFFER_FACTOR = 0.50
-
-    def human_to_bytes(self, human):
-        """Convert human readable configuration options to bytes."""
-        num_re = re.compile('^[0-9]+$')
-        if num_re.match(human):
-            return human
-
-        factors = {
-            'K': 1024,
-            'M': 1048576,
-            'G': 1073741824,
-            'T': 1099511627776
-        }
-        modifier = human[-1]
-        if modifier in factors:
-            return int(human[:-1]) * factors[modifier]
-
-        if modifier == '%':
-            total_ram = self.human_to_bytes(self.get_mem_total())
-            if self.is_32bit_system() and total_ram > self.sys_mem_limit():
-                total_ram = self.sys_mem_limit()
-            factor = int(human[:-1]) * 0.01
-            pctram = total_ram * factor
-            return int(pctram - (pctram % self.DEFAULT_PAGE_SIZE))
-
-        raise ValueError("Can only convert K,M,G, or T")
-
-    def is_32bit_system(self):
-        """Determine whether system is 32 or 64 bit."""
-        try:
-            return sys.maxsize < 2 ** 32
-        except OverflowError:
-            return False
-
-    def sys_mem_limit(self):
-        """Determine the default memory limit for the current service unit."""
-        if platform.machine() in ['armv7l']:
-            _mem_limit = self.human_to_bytes('2700M')  # experimentally determined
-        else:
-            # Limit for x86 based 32bit systems
-            _mem_limit = self.human_to_bytes('4G')
-
-        return _mem_limit
-
-    def get_mem_total(self):
-        """Calculate the total memory in the current service unit."""
-        with open('/proc/meminfo') as meminfo_file:
-            for line in meminfo_file:
-                key, mem = line.split(':', 2)
-                if key == 'MemTotal':
-                    mtot, modifier = mem.strip().split(' ')
-                    return '%s%s' % (mtot, modifier[0].upper())
-
-    def parse_config(self):
-        """Parse charm configuration and calculate values for config files."""
-        config = config_get()
-        mysql_config = {}
-        if 'max-connections' in config:
-            mysql_config['max_connections'] = config['max-connections']
-
-        if 'wait-timeout' in config:
-            mysql_config['wait_timeout'] = config['wait-timeout']
-
-        if 'innodb-flush-log-at-trx-commit' in config:
-            mysql_config['innodb_flush_log_at_trx_commit'] = config['innodb-flush-log-at-trx-commit']
-
-        # Set a sane default key_buffer size
-        mysql_config['key_buffer'] = self.human_to_bytes('32M')
-        total_memory = self.human_to_bytes(self.get_mem_total())
-
-        dataset_bytes = config.get('dataset-size', None)
-        innodb_buffer_pool_size = config.get('innodb-buffer-pool-size', None)
-
-        if innodb_buffer_pool_size:
-            innodb_buffer_pool_size = self.human_to_bytes(
-                innodb_buffer_pool_size)
-        elif dataset_bytes:
-            log("Option 'dataset-size' has been deprecated, please use"
-                "innodb_buffer_pool_size option instead", level="WARN")
-            innodb_buffer_pool_size = self.human_to_bytes(
-                dataset_bytes)
-        else:
-            innodb_buffer_pool_size = int(
-                total_memory * self.DEFAULT_INNODB_BUFFER_FACTOR)
-
-        if innodb_buffer_pool_size > total_memory:
-            log("innodb_buffer_pool_size; {} is greater than system available memory:{}".format(
-                innodb_buffer_pool_size,
-                total_memory), level='WARN')
-
-        mysql_config['innodb_buffer_pool_size'] = innodb_buffer_pool_size
-        return mysql_config

=== removed directory 'hooks/charmhelpers/contrib/hardening'
=== removed file 'hooks/charmhelpers/contrib/hardening/__init__.py'
--- hooks/charmhelpers/contrib/hardening/__init__.py	2016-04-22 04:53:43 +0000
+++ hooks/charmhelpers/contrib/hardening/__init__.py	1970-01-01 00:00:00 +0000
@@ -1,15 +0,0 @@
-# Copyright 2016 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.

=== removed directory 'hooks/charmhelpers/contrib/hardening/apache'
=== removed file 'hooks/charmhelpers/contrib/hardening/apache/__init__.py'
--- hooks/charmhelpers/contrib/hardening/apache/__init__.py	2016-04-22 04:53:43 +0000
+++ hooks/charmhelpers/contrib/hardening/apache/__init__.py	1970-01-01 00:00:00 +0000
@@ -1,19 +0,0 @@
-# Copyright 2016 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-from os import path
-
-TEMPLATES_DIR = path.join(path.dirname(__file__), 'templates')

=== removed directory 'hooks/charmhelpers/contrib/hardening/apache/checks'
=== removed file 'hooks/charmhelpers/contrib/hardening/apache/checks/__init__.py'
--- hooks/charmhelpers/contrib/hardening/apache/checks/__init__.py	2016-04-22 04:53:43 +0000
+++ hooks/charmhelpers/contrib/hardening/apache/checks/__init__.py	1970-01-01 00:00:00 +0000
@@ -1,31 +0,0 @@
-# Copyright 2016 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-from charmhelpers.core.hookenv import (
-    log,
-    DEBUG,
-)
-from charmhelpers.contrib.hardening.apache.checks import config
-
-
-def run_apache_checks():
-    log("Starting Apache hardening checks.", level=DEBUG)
-    checks = config.get_audits()
-    for check in checks:
-        log("Running '%s' check" % (check.__class__.__name__), level=DEBUG)
-        check.ensure_compliance()
-
-    log("Apache hardening checks complete.", level=DEBUG)

=== removed file 'hooks/charmhelpers/contrib/hardening/apache/checks/config.py'
--- hooks/charmhelpers/contrib/hardening/apache/checks/config.py	2016-04-22 04:53:43 +0000
+++ hooks/charmhelpers/contrib/hardening/apache/checks/config.py	1970-01-01 00:00:00 +0000
@@ -1,100 +0,0 @@
-# Copyright 2016 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-import os
-import re
-import subprocess
-
-
-from charmhelpers.core.hookenv import (
-    log,
-    INFO,
-)
-from charmhelpers.contrib.hardening.audits.file import (
-    FilePermissionAudit,
-    DirectoryPermissionAudit,
-    NoReadWriteForOther,
-    TemplatedFile,
-)
-from charmhelpers.contrib.hardening.audits.apache import DisabledModuleAudit
-from charmhelpers.contrib.hardening.apache import TEMPLATES_DIR
-from charmhelpers.contrib.hardening import utils
-
-
-def get_audits():
-    """Get Apache hardening config audits.
-
-    :returns:  dictionary of audits
-    """
-    if subprocess.call(['which', 'apache2'], stdout=subprocess.PIPE) != 0:
-        log("Apache server does not appear to be installed on this node - "
-            "skipping apache hardening", level=INFO)
-        return []
-
-    context = ApacheConfContext()
-    settings = utils.get_settings('apache')
-    audits = [
-        FilePermissionAudit(paths='/etc/apache2/apache2.conf', user='root',
-                            group='root', mode=0o0640),
-
-        TemplatedFile(os.path.join(settings['common']['apache_dir'],
-                                   'mods-available/alias.conf'),
-                      context,
-                      TEMPLATES_DIR,
-                      mode=0o0755,
-                      user='root',
-                      service_actions=[{'service': 'apache2',
-                                        'actions': ['restart']}]),
-
-        TemplatedFile(os.path.join(settings['common']['apache_dir'],
-                                   'conf-enabled/hardening.conf'),
-                      context,
-                      TEMPLATES_DIR,
-                      mode=0o0640,
-                      user='root',
-                      service_actions=[{'service': 'apache2',
-                                        'actions': ['restart']}]),
-
-        DirectoryPermissionAudit(settings['common']['apache_dir'],
-                                 user='root',
-                                 group='root',
-                                 mode=0o640),
-
-        DisabledModuleAudit(settings['hardening']['modules_to_disable']),
-
-        NoReadWriteForOther(settings['common']['apache_dir']),
-    ]
-
-    return audits
-
-
-class ApacheConfContext(object):
-    """Defines the set of key/value pairs to set in a apache config file.
-
-    This context, when called, will return a dictionary containing the
-    key/value pairs of setting to specify in the
-    /etc/apache/conf-enabled/hardening.conf file.
-    """
-    def __call__(self):
-        settings = utils.get_settings('apache')
-        ctxt = settings['hardening']
-
-        out = subprocess.check_output(['apache2', '-v'])
-        ctxt['apache_version'] = re.search(r'.+version: Apache/(.+?)\s.+',
-                                           out).group(1)
-        ctxt['apache_icondir'] = '/usr/share/apache2/icons/'
-        ctxt['traceenable'] = settings['hardening']['traceenable']
-        return ctxt

=== removed directory 'hooks/charmhelpers/contrib/hardening/audits'
=== removed file 'hooks/charmhelpers/contrib/hardening/audits/__init__.py'
--- hooks/charmhelpers/contrib/hardening/audits/__init__.py	2016-04-22 04:53:43 +0000
+++ hooks/charmhelpers/contrib/hardening/audits/__init__.py	1970-01-01 00:00:00 +0000
@@ -1,63 +0,0 @@
-# Copyright 2016 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-
-class BaseAudit(object):  # NO-QA
-    """Base class for hardening checks.
-
-    The lifecycle of a hardening check is to first check to see if the system
-    is in compliance for the specified check. If it is not in compliance, the
-    check method will return a value which will be supplied to the.
-    """
-    def __init__(self, *args, **kwargs):
-        self.unless = kwargs.get('unless', None)
-        super(BaseAudit, self).__init__()
-
-    def ensure_compliance(self):
-        """Checks to see if the current hardening check is in compliance or
-        not.
-
-        If the check that is performed is not in compliance, then an exception
-        should be raised.
-        """
-        pass
-
-    def _take_action(self):
-        """Determines whether to perform the action or not.
-
-        Checks whether or not an action should be taken. This is determined by
-        the truthy value for the unless parameter. If unless is a callback
-        method, it will be invoked with no parameters in order to determine
-        whether or not the action should be taken. Otherwise, the truthy value
-        of the unless attribute will determine if the action should be
-        performed.
-        """
-        # Do the action if there isn't an unless override.
-        if self.unless is None:
-            return True
-
-        # Invoke the callback if there is one.
-        if hasattr(self.unless, '__call__'):
-            results = self.unless()
-            if results:
-                return False
-            else:
-                return True
-
-        if self.unless:
-            return False
-        else:
-            return True

=== removed file 'hooks/charmhelpers/contrib/hardening/audits/apache.py'
--- hooks/charmhelpers/contrib/hardening/audits/apache.py	2016-04-22 04:53:43 +0000
+++ hooks/charmhelpers/contrib/hardening/audits/apache.py	1970-01-01 00:00:00 +0000
@@ -1,100 +0,0 @@
-# Copyright 2016 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-import re
-import subprocess
-
-from six import string_types
-
-from charmhelpers.core.hookenv import (
-    log,
-    INFO,
-    ERROR,
-)
-
-from charmhelpers.contrib.hardening.audits import BaseAudit
-
-
-class DisabledModuleAudit(BaseAudit):
-    """Audits Apache2 modules.
-
-    Determines if the apache2 modules are enabled. If the modules are enabled
-    then they are removed in the ensure_compliance.
-    """
-    def __init__(self, modules):
-        if modules is None:
-            self.modules = []
-        elif isinstance(modules, string_types):
-            self.modules = [modules]
-        else:
-            self.modules = modules
-
-    def ensure_compliance(self):
-        """Ensures that the modules are not loaded."""
-        if not self.modules:
-            return
-
-        try:
-            loaded_modules = self._get_loaded_modules()
-            non_compliant_modules = []
-            for module in self.modules:
-                if module in loaded_modules:
-                    log("Module '%s' is enabled but should not be." %
-                        (module), level=INFO)
-                    non_compliant_modules.append(module)
-
-            if len(non_compliant_modules) == 0:
-                return
-
-            for module in non_compliant_modules:
-                self._disable_module(module)
-            self._restart_apache()
-        except subprocess.CalledProcessError as e:
-            log('Error occurred auditing apache module compliance. '
-                'This may have been already reported. '
-                'Output is: %s' % e.output, level=ERROR)
-
-    @staticmethod
-    def _get_loaded_modules():
-        """Returns the modules which are enabled in Apache."""
-        output = subprocess.check_output(['apache2ctl', '-M'])
-        modules = []
-        for line in output.strip().split():
-            # Each line of the enabled module output looks like:
-            #  module_name (static|shared)
-            # Plus a header line at the top of the output which is stripped
-            # out by the regex.
-            matcher = re.search(r'^ (\S*)', line)
-            if matcher:
-                modules.append(matcher.group(1))
-        return modules
-
-    @staticmethod
-    def _disable_module(module):
-        """Disables the specified module in Apache."""
-        try:
-            subprocess.check_call(['a2dismod', module])
-        except subprocess.CalledProcessError as e:
-            # Note: catch error here to allow the attempt of disabling
-            # multiple modules in one go rather than failing after the
-            # first module fails.
-            log('Error occurred disabling module %s. '
-                'Output is: %s' % (module, e.output), level=ERROR)
-
-    @staticmethod
-    def _restart_apache():
-        """Restarts the apache process"""
-        subprocess.check_output(['service', 'apache2', 'restart'])

=== removed file 'hooks/charmhelpers/contrib/hardening/audits/apt.py'
--- hooks/charmhelpers/contrib/hardening/audits/apt.py	2016-04-22 04:53:43 +0000
+++ hooks/charmhelpers/contrib/hardening/audits/apt.py	1970-01-01 00:00:00 +0000
@@ -1,105 +0,0 @@
-# Copyright 2016 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-from __future__ import absolute_import  # required for external apt import
-from apt import apt_pkg
-from six import string_types
-
-from charmhelpers.fetch import (
-    apt_cache,
-    apt_purge
-)
-from charmhelpers.core.hookenv import (
-    log,
-    DEBUG,
-    WARNING,
-)
-from charmhelpers.contrib.hardening.audits import BaseAudit
-
-
-class AptConfig(BaseAudit):
-
-    def __init__(self, config, **kwargs):
-        self.config = config
-
-    def verify_config(self):
-        apt_pkg.init()
-        for cfg in self.config:
-            value = apt_pkg.config.get(cfg['key'], cfg.get('default', ''))
-            if value and value != cfg['expected']:
-                log("APT config '%s' has unexpected value '%s' "
-                    "(expected='%s')" %
-                    (cfg['key'], value, cfg['expected']), level=WARNING)
-
-    def ensure_compliance(self):
-        self.verify_config()
-
-
-class RestrictedPackages(BaseAudit):
-    """Class used to audit restricted packages on the system."""
-
-    def __init__(self, pkgs, **kwargs):
-        super(RestrictedPackages, self).__init__(**kwargs)
-        if isinstance(pkgs, string_types) or not hasattr(pkgs, '__iter__'):
-            self.pkgs = [pkgs]
-        else:
-            self.pkgs = pkgs
-
-    def ensure_compliance(self):
-        cache = apt_cache()
-
-        for p in self.pkgs:
-            if p not in cache:
-                continue
-
-            pkg = cache[p]
-            if not self.is_virtual_package(pkg):
-                if not pkg.current_ver:
-                    log("Package '%s' is not installed." % pkg.name,
-                        level=DEBUG)
-                    continue
-                else:
-                    log("Restricted package '%s' is installed" % pkg.name,
-                        level=WARNING)
-                    self.delete_package(cache, pkg)
-            else:
-                log("Checking restricted virtual package '%s' provides" %
-                    pkg.name, level=DEBUG)
-                self.delete_package(cache, pkg)
-
-    def delete_package(self, cache, pkg):
-        """Deletes the package from the system.
-
-        Deletes the package form the system, properly handling virtual
-        packages.
-
-        :param cache: the apt cache
-        :param pkg: the package to remove
-        """
-        if self.is_virtual_package(pkg):
-            log("Package '%s' appears to be virtual - purging provides" %
-                pkg.name, level=DEBUG)
-            for _p in pkg.provides_list:
-                self.delete_package(cache, _p[2].parent_pkg)
-        elif not pkg.current_ver:
-            log("Package '%s' not installed" % pkg.name, level=DEBUG)
-            return
-        else:
-            log("Purging package '%s'" % pkg.name, level=DEBUG)
-            apt_purge(pkg.name)
-
-    def is_virtual_package(self, pkg):
-        return pkg.has_provides and not pkg.has_versions

=== removed file 'hooks/charmhelpers/contrib/hardening/audits/file.py'
--- hooks/charmhelpers/contrib/hardening/audits/file.py	2016-04-22 04:53:43 +0000
+++ hooks/charmhelpers/contrib/hardening/audits/file.py	1970-01-01 00:00:00 +0000
@@ -1,552 +0,0 @@
-# Copyright 2016 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-import grp
-import os
-import pwd
-import re
-
-from subprocess import (
-    CalledProcessError,
-    check_output,
-    check_call,
-)
-from traceback import format_exc
-from six import string_types
-from stat import (
-    S_ISGID,
-    S_ISUID
-)
-
-from charmhelpers.core.hookenv import (
-    log,
-    DEBUG,
-    INFO,
-    WARNING,
-    ERROR,
-)
-from charmhelpers.core import unitdata
-from charmhelpers.core.host import file_hash
-from charmhelpers.contrib.hardening.audits import BaseAudit
-from charmhelpers.contrib.hardening.templating import (
-    get_template_path,
-    render_and_write,
-)
-from charmhelpers.contrib.hardening import utils
-
-
-class BaseFileAudit(BaseAudit):
-    """Base class for file audits.
-
-    Provides api stubs for compliance check flow that must be used by any class
-    that implemented this one.
-    """
-
-    def __init__(self, paths, always_comply=False, *args, **kwargs):
-        """
-        :param paths: string path of list of paths of files we want to apply
-                      compliance checks are criteria to.
-        :param always_comply: if true compliance criteria is always applied
-                              else compliance is skipped for non-existent
-                              paths.
-        """
-        super(BaseFileAudit, self).__init__(*args, **kwargs)
-        self.always_comply = always_comply
-        if isinstance(paths, string_types) or not hasattr(paths, '__iter__'):
-            self.paths = [paths]
-        else:
-            self.paths = paths
-
-    def ensure_compliance(self):
-        """Ensure that the all registered files comply to registered criteria.
-        """
-        for p in self.paths:
-            if os.path.exists(p):
-                if self.is_compliant(p):
-                    continue
-
-                log('File %s is not in compliance.' % p, level=INFO)
-            else:
-                if not self.always_comply:
-                    log("Non-existent path '%s' - skipping compliance check"
-                        % (p), level=INFO)
-                    continue
-
-            if self._take_action():
-                log("Applying compliance criteria to '%s'" % (p), level=INFO)
-                self.comply(p)
-
-    def is_compliant(self, path):
-        """Audits the path to see if it is compliance.
-
-        :param path: the path to the file that should be checked.
-        """
-        raise NotImplementedError
-
-    def comply(self, path):
-        """Enforces the compliance of a path.
-
-        :param path: the path to the file that should be enforced.
-        """
-        raise NotImplementedError
-
-    @classmethod
-    def _get_stat(cls, path):
-        """Returns the Posix st_stat information for the specified file path.
-
-        :param path: the path to get the st_stat information for.
-        :returns: an st_stat object for the path or None if the path doesn't
-                  exist.
-        """
-        return os.stat(path)
-
-
-class FilePermissionAudit(BaseFileAudit):
-    """Implements an audit for file permissions and ownership for a user.
-
-    This class implements functionality that ensures that a specific user/group
-    will own the file(s) specified and that the permissions specified are
-    applied properly to the file.
-    """
-    def __init__(self, paths, user, group=None, mode=0o600, **kwargs):
-        self.user = user
-        self.group = group
-        self.mode = mode
-        super(FilePermissionAudit, self).__init__(paths, user, group, mode,
-                                                  **kwargs)
-
-    @property
-    def user(self):
-        return self._user
-
-    @user.setter
-    def user(self, name):
-        try:
-            user = pwd.getpwnam(name)
-        except KeyError:
-            log('Unknown user %s' % name, level=ERROR)
-            user = None
-        self._user = user
-
-    @property
-    def group(self):
-        return self._group
-
-    @group.setter
-    def group(self, name):
-        try:
-            group = None
-            if name:
-                group = grp.getgrnam(name)
-            else:
-                group = grp.getgrgid(self.user.pw_gid)
-        except KeyError:
-            log('Unknown group %s' % name, level=ERROR)
-        self._group = group
-
-    def is_compliant(self, path):
-        """Checks if the path is in compliance.
-
-        Used to determine if the path specified meets the necessary
-        requirements to be in compliance with the check itself.
-
-        :param path: the file path to check
-        :returns: True if the path is compliant, False otherwise.
-        """
-        stat = self._get_stat(path)
-        user = self.user
-        group = self.group
-
-        compliant = True
-        if stat.st_uid != user.pw_uid or stat.st_gid != group.gr_gid:
-            log('File %s is not owned by %s:%s.' % (path, user.pw_name,
-                                                    group.gr_name),
-                level=INFO)
-            compliant = False
-
-        # POSIX refers to the st_mode bits as corresponding to both the
-        # file type and file permission bits, where the least significant 12
-        # bits (o7777) are the suid (11), sgid (10), sticky bits (9), and the
-        # file permission bits (8-0)
-        perms = stat.st_mode & 0o7777
-        if perms != self.mode:
-            log('File %s has incorrect permissions, currently set to %s' %
-                (path, oct(stat.st_mode & 0o7777)), level=INFO)
-            compliant = False
-
-        return compliant
-
-    def comply(self, path):
-        """Issues a chown and chmod to the file paths specified."""
-        utils.ensure_permissions(path, self.user.pw_name, self.group.gr_name,
-                                 self.mode)
-
-
-class DirectoryPermissionAudit(FilePermissionAudit):
-    """Performs a permission check for the  specified directory path."""
-
-    def __init__(self, paths, user, group=None, mode=0o600,
-                 recursive=True, **kwargs):
-        super(DirectoryPermissionAudit, self).__init__(paths, user, group,
-                                                       mode, **kwargs)
-        self.recursive = recursive
-
-    def is_compliant(self, path):
-        """Checks if the directory is compliant.
-
-        Used to determine if the path specified and all of its children
-        directories are in compliance with the check itself.
-
-        :param path: the directory path to check
-        :returns: True if the directory tree is compliant, otherwise False.
-        """
-        if not os.path.isdir(path):
-            log('Path specified %s is not a directory.' % path, level=ERROR)
-            raise ValueError("%s is not a directory." % path)
-
-        if not self.recursive:
-            return super(DirectoryPermissionAudit, self).is_compliant(path)
-
-        compliant = True
-        for root, dirs, _ in os.walk(path):
-            if len(dirs) > 0:
-                continue
-
-            if not super(DirectoryPermissionAudit, self).is_compliant(root):
-                compliant = False
-                continue
-
-        return compliant
-
-    def comply(self, path):
-        for root, dirs, _ in os.walk(path):
-            if len(dirs) > 0:
-                super(DirectoryPermissionAudit, self).comply(root)
-
-
-class ReadOnly(BaseFileAudit):
-    """Audits that files and folders are read only."""
-    def __init__(self, paths, *args, **kwargs):
-        super(ReadOnly, self).__init__(paths=paths, *args, **kwargs)
-
-    def is_compliant(self, path):
-        try:
-            output = check_output(['find', path, '-perm', '-go+w',
-                                   '-type', 'f']).strip()
-
-            # The find above will find any files which have permission sets
-            # which allow too broad of write access. As such, the path is
-            # compliant if there is no output.
-            if output:
-                return False
-
-            return True
-        except CalledProcessError as e:
-            log('Error occurred checking finding writable files for %s. '
-                'Error information is: command %s failed with returncode '
-                '%d and output %s.\n%s' % (path, e.cmd, e.returncode, e.output,
-                                           format_exc(e)), level=ERROR)
-            return False
-
-    def comply(self, path):
-        try:
-            check_output(['chmod', 'go-w', '-R', path])
-        except CalledProcessError as e:
-            log('Error occurred removing writeable permissions for %s. '
-                'Error information is: command %s failed with returncode '
-                '%d and output %s.\n%s' % (path, e.cmd, e.returncode, e.output,
-                                           format_exc(e)), level=ERROR)
-
-
-class NoReadWriteForOther(BaseFileAudit):
-    """Ensures that the files found under the base path are readable or
-    writable by anyone other than the owner or the group.
-    """
-    def __init__(self, paths):
-        super(NoReadWriteForOther, self).__init__(paths)
-
-    def is_compliant(self, path):
-        try:
-            cmd = ['find', path, '-perm', '-o+r', '-type', 'f', '-o',
-                   '-perm', '-o+w', '-type', 'f']
-            output = check_output(cmd).strip()
-
-            # The find above here will find any files which have read or
-            # write permissions for other, meaning there is too broad of access
-            # to read/write the file. As such, the path is compliant if there's
-            # no output.
-            if output:
-                return False
-
-            return True
-        except CalledProcessError as e:
-            log('Error occurred while finding files which are readable or '
-                'writable to the world in %s. '
-                'Command output is: %s.' % (path, e.output), level=ERROR)
-
-    def comply(self, path):
-        try:
-            check_output(['chmod', '-R', 'o-rw', path])
-        except CalledProcessError as e:
-            log('Error occurred attempting to change modes of files under '
-                'path %s. Output of command is: %s' % (path, e.output))
-
-
-class NoSUIDSGIDAudit(BaseFileAudit):
-    """Audits that specified files do not have SUID/SGID bits set."""
-    def __init__(self, paths, *args, **kwargs):
-        super(NoSUIDSGIDAudit, self).__init__(paths=paths, *args, **kwargs)
-
-    def is_compliant(self, path):
-        stat = self._get_stat(path)
-        if (stat.st_mode & (S_ISGID | S_ISUID)) != 0:
-            return False
-
-        return True
-
-    def comply(self, path):
-        try:
-            log('Removing suid/sgid from %s.' % path, level=DEBUG)
-            check_output(['chmod', '-s', path])
-        except CalledProcessError as e:
-            log('Error occurred removing suid/sgid from %s.'
-                'Error information is: command %s failed with returncode '
-                '%d and output %s.\n%s' % (path, e.cmd, e.returncode, e.output,
-                                           format_exc(e)), level=ERROR)
-
-
-class TemplatedFile(BaseFileAudit):
-    """The TemplatedFileAudit audits the contents of a templated file.
-
-    This audit renders a file from a template, sets the appropriate file
-    permissions, then generates a hashsum with which to check the content
-    changed.
-    """
-    def __init__(self, path, context, template_dir, mode, user='root',
-                 group='root', service_actions=None, **kwargs):
-        self.context = context
-        self.user = user
-        self.group = group
-        self.mode = mode
-        self.template_dir = template_dir
-        self.service_actions = service_actions
-        super(TemplatedFile, self).__init__(paths=path, always_comply=True,
-                                            **kwargs)
-
-    def is_compliant(self, path):
-        """Determines if the templated file is compliant.
-
-        A templated file is only compliant if it has not changed (as
-        determined by its sha256 hashsum) AND its file permissions are set
-        appropriately.
-
-        :param path: the path to check compliance.
-        """
-        same_templates = self.templates_match(path)
-        same_content = self.contents_match(path)
-        same_permissions = self.permissions_match(path)
-
-        if same_content and same_permissions and same_templates:
-            return True
-
-        return False
-
-    def run_service_actions(self):
-        """Run any actions on services requested."""
-        if not self.service_actions:
-            return
-
-        for svc_action in self.service_actions:
-            name = svc_action['service']
-            actions = svc_action['actions']
-            log("Running service '%s' actions '%s'" % (name, actions),
-                level=DEBUG)
-            for action in actions:
-                cmd = ['service', name, action]
-                try:
-                    check_call(cmd)
-                except CalledProcessError as exc:
-                    log("Service name='%s' action='%s' failed - %s" %
-                        (name, action, exc), level=WARNING)
-
-    def comply(self, path):
-        """Ensures the contents and the permissions of the file.
-
-        :param path: the path to correct
-        """
-        dirname = os.path.dirname(path)
-        if not os.path.exists(dirname):
-            os.makedirs(dirname)
-
-        self.pre_write()
-        render_and_write(self.template_dir, path, self.context())
-        utils.ensure_permissions(path, self.user, self.group, self.mode)
-        self.run_service_actions()
-        self.save_checksum(path)
-        self.post_write()
-
-    def pre_write(self):
-        """Invoked prior to writing the template."""
-        pass
-
-    def post_write(self):
-        """Invoked after writing the template."""
-        pass
-
-    def templates_match(self, path):
-        """Determines if the template files are the same.
-
-        The template file equality is determined by the hashsum of the
-        template files themselves. If there is no hashsum, then the content
-        cannot be sure to be the same so treat it as if they changed.
-        Otherwise, return whether or not the hashsums are the same.
-
-        :param path: the path to check
-        :returns: boolean
-        """
-        template_path = get_template_path(self.template_dir, path)
-        key = 'hardening:template:%s' % template_path
-        template_checksum = file_hash(template_path)
-        kv = unitdata.kv()
-        stored_tmplt_checksum = kv.get(key)
-        if not stored_tmplt_checksum:
-            kv.set(key, template_checksum)
-            kv.flush()
-            log('Saved template checksum for %s.' % template_path,
-                level=DEBUG)
-            # Since we don't have a template checksum, then assume it doesn't
-            # match and return that the template is different.
-            return False
-        elif stored_tmplt_checksum != template_checksum:
-            kv.set(key, template_checksum)
-            kv.flush()
-            log('Updated template checksum for %s.' % template_path,
-                level=DEBUG)
-            return False
-
-        # Here the template hasn't changed based upon the calculated
-        # checksum of the template and what was previously stored.
-        return True
-
-    def contents_match(self, path):
-        """Determines if the file content is the same.
-
-        This is determined by comparing hashsum of the file contents and
-        the saved hashsum. If there is no hashsum, then the content cannot
-        be sure to be the same so treat them as if they are not the same.
-        Otherwise, return True if the hashsums are the same, False if they
-        are not the same.
-
-        :param path: the file to check.
-        """
-        checksum = file_hash(path)
-
-        kv = unitdata.kv()
-        stored_checksum = kv.get('hardening:%s' % path)
-        if not stored_checksum:
-            # If the checksum hasn't been generated, return False to ensure
-            # the file is written and the checksum stored.
-            log('Checksum for %s has not been calculated.' % path, level=DEBUG)
-            return False
-        elif stored_checksum != checksum:
-            log('Checksum mismatch for %s.' % path, level=DEBUG)
-            return False
-
-        return True
-
-    def permissions_match(self, path):
-        """Determines if the file owner and permissions match.
-
-        :param path: the path to check.
-        """
-        audit = FilePermissionAudit(path, self.user, self.group, self.mode)
-        return audit.is_compliant(path)
-
-    def save_checksum(self, path):
-        """Calculates and saves the checksum for the path specified.
-
-        :param path: the path of the file to save the checksum.
-        """
-        checksum = file_hash(path)
-        kv = unitdata.kv()
-        kv.set('hardening:%s' % path, checksum)
-        kv.flush()
-
-
-class DeletedFile(BaseFileAudit):
-    """Audit to ensure that a file is deleted."""
-    def __init__(self, paths):
-        super(DeletedFile, self).__init__(paths)
-
-    def is_compliant(self, path):
-        return not os.path.exists(path)
-
-    def comply(self, path):
-        os.remove(path)
-
-
-class FileContentAudit(BaseFileAudit):
-    """Audit the contents of a file."""
-    def __init__(self, paths, cases, **kwargs):
-        # Cases we expect to pass
-        self.pass_cases = cases.get('pass', [])
-        # Cases we expect to fail
-        self.fail_cases = cases.get('fail', [])
-        super(FileContentAudit, self).__init__(paths, **kwargs)
-
-    def is_compliant(self, path):
-        """
-        Given a set of content matching cases i.e. tuple(regex, bool) where
-        bool value denotes whether or not regex is expected to match, check that
-        all cases match as expected with the contents of the file. Cases can be
-        expected to pass of fail.
-
-        :param path: Path of file to check.
-        :returns: Boolean value representing whether or not all cases are
-                  found to be compliant.
-        """
-        log("Auditing contents of file '%s'" % (path), level=DEBUG)
-        with open(path, 'r') as fd:
-            contents = fd.read()
-
-        matches = 0
-        for pattern in self.pass_cases:
-            key = re.compile(pattern, flags=re.MULTILINE)
-            results = re.search(key, contents)
-            if results:
-                matches += 1
-            else:
-                log("Pattern '%s' was expected to pass but instead it failed"
-                    % (pattern), level=WARNING)
-
-        for pattern in self.fail_cases:
-            key = re.compile(pattern, flags=re.MULTILINE)
-            results = re.search(key, contents)
-            if not results:
-                matches += 1
-            else:
-                log("Pattern '%s' was expected to fail but instead it passed"
-                    % (pattern), level=WARNING)
-
-        total = len(self.pass_cases) + len(self.fail_cases)
-        log("Checked %s cases and %s passed" % (total, matches), level=DEBUG)
-        return matches == total
-
-    def comply(self, *args, **kwargs):
-        """NOOP since we just issue warnings. This is to avoid the
-        NotImplememtedError.
-        """
-        log("Not applying any compliance criteria, only checks.", level=INFO)

=== removed file 'hooks/charmhelpers/contrib/hardening/harden.py'
--- hooks/charmhelpers/contrib/hardening/harden.py	2016-04-22 04:53:43 +0000
+++ hooks/charmhelpers/contrib/hardening/harden.py	1970-01-01 00:00:00 +0000
@@ -1,84 +0,0 @@
-# Copyright 2016 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-import six
-
-from collections import OrderedDict
-
-from charmhelpers.core.hookenv import (
-    config,
-    log,
-    DEBUG,
-    WARNING,
-)
-from charmhelpers.contrib.hardening.host.checks import run_os_checks
-from charmhelpers.contrib.hardening.ssh.checks import run_ssh_checks
-from charmhelpers.contrib.hardening.mysql.checks import run_mysql_checks
-from charmhelpers.contrib.hardening.apache.checks import run_apache_checks
-
-
-def harden(overrides=None):
-    """Hardening decorator.
-
-    This is the main entry point for running the hardening stack. In order to
-    run modules of the stack you must add this decorator to charm hook(s) and
-    ensure that your charm config.yaml contains the 'harden' option set to
-    one or more of the supported modules. Setting these will cause the
-    corresponding hardening code to be run when the hook fires.
-
-    This decorator can and should be applied to more than one hook or function
-    such that hardening modules are called multiple times. This is because
-    subsequent calls will perform auditing checks that will report any changes
-    to resources hardened by the first run (and possibly perform compliance
-    actions as a result of any detected infractions).
-
-    :param overrides: Optional list of stack modules used to override those
-                      provided with 'harden' config.
-    :returns: Returns value returned by decorated function once executed.
-    """
-    def _harden_inner1(f):
-        log("Hardening function '%s'" % (f.__name__), level=DEBUG)
-
-        def _harden_inner2(*args, **kwargs):
-            RUN_CATALOG = OrderedDict([('os', run_os_checks),
-                                       ('ssh', run_ssh_checks),
-                                       ('mysql', run_mysql_checks),
-                                       ('apache', run_apache_checks)])
-
-            enabled = overrides or (config("harden") or "").split()
-            if enabled:
-                modules_to_run = []
-                # modules will always be performed in the following order
-                for module, func in six.iteritems(RUN_CATALOG):
-                    if module in enabled:
-                        enabled.remove(module)
-                        modules_to_run.append(func)
-
-                if enabled:
-                    log("Unknown hardening modules '%s' - ignoring" %
-                        (', '.join(enabled)), level=WARNING)
-
-                for hardener in modules_to_run:
-                    log("Executing hardening module '%s'" %
-                        (hardener.__name__), level=DEBUG)
-                    hardener()
-            else:
-                log("No hardening applied to '%s'" % (f.__name__), level=DEBUG)
-
-            return f(*args, **kwargs)
-        return _harden_inner2
-
-    return _harden_inner1

=== removed directory 'hooks/charmhelpers/contrib/hardening/host'
=== removed file 'hooks/charmhelpers/contrib/hardening/host/__init__.py'
--- hooks/charmhelpers/contrib/hardening/host/__init__.py	2016-04-22 04:53:43 +0000
+++ hooks/charmhelpers/contrib/hardening/host/__init__.py	1970-01-01 00:00:00 +0000
@@ -1,19 +0,0 @@
-# Copyright 2016 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-from os import path
-
-TEMPLATES_DIR = path.join(path.dirname(__file__), 'templates')

=== removed directory 'hooks/charmhelpers/contrib/hardening/host/checks'
=== removed file 'hooks/charmhelpers/contrib/hardening/host/checks/__init__.py'
--- hooks/charmhelpers/contrib/hardening/host/checks/__init__.py	2016-04-22 04:53:43 +0000
+++ hooks/charmhelpers/contrib/hardening/host/checks/__init__.py	1970-01-01 00:00:00 +0000
@@ -1,50 +0,0 @@
-# Copyright 2016 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-from charmhelpers.core.hookenv import (
-    log,
-    DEBUG,
-)
-from charmhelpers.contrib.hardening.host.checks import (
-    apt,
-    limits,
-    login,
-    minimize_access,
-    pam,
-    profile,
-    securetty,
-    suid_sgid,
-    sysctl
-)
-
-
-def run_os_checks():
-    log("Starting OS hardening checks.", level=DEBUG)
-    checks = apt.get_audits()
-    checks.extend(limits.get_audits())
-    checks.extend(login.get_audits())
-    checks.extend(minimize_access.get_audits())
-    checks.extend(pam.get_audits())
-    checks.extend(profile.get_audits())
-    checks.extend(securetty.get_audits())
-    checks.extend(suid_sgid.get_audits())
-    checks.extend(sysctl.get_audits())
-
-    for check in checks:
-        log("Running '%s' check" % (check.__class__.__name__), level=DEBUG)
-        check.ensure_compliance()
-
-    log("OS hardening checks complete.", level=DEBUG)

=== removed file 'hooks/charmhelpers/contrib/hardening/host/checks/apt.py'
--- hooks/charmhelpers/contrib/hardening/host/checks/apt.py	2016-04-22 04:53:43 +0000
+++ hooks/charmhelpers/contrib/hardening/host/checks/apt.py	1970-01-01 00:00:00 +0000
@@ -1,39 +0,0 @@
-# Copyright 2016 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-from charmhelpers.contrib.hardening.utils import get_settings
-from charmhelpers.contrib.hardening.audits.apt import (
-    AptConfig,
-    RestrictedPackages,
-)
-
-
-def get_audits():
-    """Get OS hardening apt audits.
-
-    :returns:  dictionary of audits
-    """
-    audits = [AptConfig([{'key': 'APT::Get::AllowUnauthenticated',
-                          'expected': 'false'}])]
-
-    settings = get_settings('os')
-    clean_packages = settings['security']['packages_clean']
-    if clean_packages:
-        security_packages = settings['security']['packages_list']
-        if security_packages:
-            audits.append(RestrictedPackages(security_packages))
-
-    return audits

=== removed file 'hooks/charmhelpers/contrib/hardening/host/checks/limits.py'
--- hooks/charmhelpers/contrib/hardening/host/checks/limits.py	2016-04-22 04:53:43 +0000
+++ hooks/charmhelpers/contrib/hardening/host/checks/limits.py	1970-01-01 00:00:00 +0000
@@ -1,55 +0,0 @@
-# Copyright 2016 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-from charmhelpers.contrib.hardening.audits.file import (
-    DirectoryPermissionAudit,
-    TemplatedFile,
-)
-from charmhelpers.contrib.hardening.host import TEMPLATES_DIR
-from charmhelpers.contrib.hardening import utils
-
-
-def get_audits():
-    """Get OS hardening security limits audits.
-
-    :returns:  dictionary of audits
-    """
-    audits = []
-    settings = utils.get_settings('os')
-
-    # Ensure that the /etc/security/limits.d directory is only writable
-    # by the root user, but others can execute and read.
-    audits.append(DirectoryPermissionAudit('/etc/security/limits.d',
-                                           user='root', group='root',
-                                           mode=0o755))
-
-    # If core dumps are not enabled, then don't allow core dumps to be
-    # created as they may contain sensitive information.
-    if not settings['security']['kernel_enable_core_dump']:
-        audits.append(TemplatedFile('/etc/security/limits.d/10.hardcore.conf',
-                                    SecurityLimitsContext(),
-                                    template_dir=TEMPLATES_DIR,
-                                    user='root', group='root', mode=0o0440))
-    return audits
-
-
-class SecurityLimitsContext(object):
-
-    def __call__(self):
-        settings = utils.get_settings('os')
-        ctxt = {'disable_core_dump':
-                not settings['security']['kernel_enable_core_dump']}
-        return ctxt

=== removed file 'hooks/charmhelpers/contrib/hardening/host/checks/login.py'
--- hooks/charmhelpers/contrib/hardening/host/checks/login.py	2016-04-22 04:53:43 +0000
+++ hooks/charmhelpers/contrib/hardening/host/checks/login.py	1970-01-01 00:00:00 +0000
@@ -1,67 +0,0 @@
-# Copyright 2016 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-from six import string_types
-
-from charmhelpers.contrib.hardening.audits.file import TemplatedFile
-from charmhelpers.contrib.hardening.host import TEMPLATES_DIR
-from charmhelpers.contrib.hardening import utils
-
-
-def get_audits():
-    """Get OS hardening login.defs audits.
-
-    :returns:  dictionary of audits
-    """
-    audits = [TemplatedFile('/etc/login.defs', LoginContext(),
-                            template_dir=TEMPLATES_DIR,
-                            user='root', group='root', mode=0o0444)]
-    return audits
-
-
-class LoginContext(object):
-
-    def __call__(self):
-        settings = utils.get_settings('os')
-
-        # Octal numbers in yaml end up being turned into decimal,
-        # so check if the umask is entered as a string (e.g. '027')
-        # or as an octal umask as we know it (e.g. 002). If its not
-        # a string assume it to be octal and turn it into an octal
-        # string.
-        umask = settings['environment']['umask']
-        if not isinstance(umask, string_types):
-            umask = '%s' % oct(umask)
-
-        ctxt = {
-            'additional_user_paths':
-            settings['environment']['extra_user_paths'],
-            'umask': umask,
-            'pwd_max_age': settings['auth']['pw_max_age'],
-            'pwd_min_age': settings['auth']['pw_min_age'],
-            'uid_min': settings['auth']['uid_min'],
-            'sys_uid_min': settings['auth']['sys_uid_min'],
-            'sys_uid_max': settings['auth']['sys_uid_max'],
-            'gid_min': settings['auth']['gid_min'],
-            'sys_gid_min': settings['auth']['sys_gid_min'],
-            'sys_gid_max': settings['auth']['sys_gid_max'],
-            'login_retries': settings['auth']['retries'],
-            'login_timeout': settings['auth']['timeout'],
-            'chfn_restrict': settings['auth']['chfn_restrict'],
-            'allow_login_without_home': settings['auth']['allow_homeless']
-        }
-
-        return ctxt

=== removed file 'hooks/charmhelpers/contrib/hardening/host/checks/minimize_access.py'
--- hooks/charmhelpers/contrib/hardening/host/checks/minimize_access.py	2016-04-22 04:53:43 +0000
+++ hooks/charmhelpers/contrib/hardening/host/checks/minimize_access.py	1970-01-01 00:00:00 +0000
@@ -1,52 +0,0 @@
-# Copyright 2016 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-from charmhelpers.contrib.hardening.audits.file import (
-    FilePermissionAudit,
-    ReadOnly,
-)
-from charmhelpers.contrib.hardening import utils
-
-
-def get_audits():
-    """Get OS hardening access audits.
-
-    :returns:  dictionary of audits
-    """
-    audits = []
-    settings = utils.get_settings('os')
-
-    # Remove write permissions from $PATH folders for all regular users.
-    # This prevents changing system-wide commands from normal users.
-    path_folders = {'/usr/local/sbin',
-                    '/usr/local/bin',
-                    '/usr/sbin',
-                    '/usr/bin',
-                    '/bin'}
-    extra_user_paths = settings['environment']['extra_user_paths']
-    path_folders.update(extra_user_paths)
-    audits.append(ReadOnly(path_folders))
-
-    # Only allow the root user to have access to the shadow file.
-    audits.append(FilePermissionAudit('/etc/shadow', 'root', 'root', 0o0600))
-
-    if 'change_user' not in settings['security']['users_allow']:
-        # su should only be accessible to user and group root, unless it is
-        # expressly defined to allow users to change to root via the
-        # security_users_allow config option.
-        audits.append(FilePermissionAudit('/bin/su', 'root', 'root', 0o750))
-
-    return audits

=== removed file 'hooks/charmhelpers/contrib/hardening/host/checks/pam.py'
--- hooks/charmhelpers/contrib/hardening/host/checks/pam.py	2016-04-22 04:53:43 +0000
+++ hooks/charmhelpers/contrib/hardening/host/checks/pam.py	1970-01-01 00:00:00 +0000
@@ -1,134 +0,0 @@
-# Copyright 2016 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-from subprocess import (
-    check_output,
-    CalledProcessError,
-)
-
-from charmhelpers.core.hookenv import (
-    log,
-    DEBUG,
-    ERROR,
-)
-from charmhelpers.fetch import (
-    apt_install,
-    apt_purge,
-    apt_update,
-)
-from charmhelpers.contrib.hardening.audits.file import (
-    TemplatedFile,
-    DeletedFile,
-)
-from charmhelpers.contrib.hardening import utils
-from charmhelpers.contrib.hardening.host import TEMPLATES_DIR
-
-
-def get_audits():
-    """Get OS hardening PAM authentication audits.
-
-    :returns:  dictionary of audits
-    """
-    audits = []
-
-    settings = utils.get_settings('os')
-
-    if settings['auth']['pam_passwdqc_enable']:
-        audits.append(PasswdqcPAM('/etc/passwdqc.conf'))
-
-    if settings['auth']['retries']:
-        audits.append(Tally2PAM('/usr/share/pam-configs/tally2'))
-    else:
-        audits.append(DeletedFile('/usr/share/pam-configs/tally2'))
-
-    return audits
-
-
-class PasswdqcPAMContext(object):
-
-    def __call__(self):
-        ctxt = {}
-        settings = utils.get_settings('os')
-
-        ctxt['auth_pam_passwdqc_options'] = \
-            settings['auth']['pam_passwdqc_options']
-
-        return ctxt
-
-
-class PasswdqcPAM(TemplatedFile):
-    """The PAM Audit verifies the linux PAM settings."""
-    def __init__(self, path):
-        super(PasswdqcPAM, self).__init__(path=path,
-                                          template_dir=TEMPLATES_DIR,
-                                          context=PasswdqcPAMContext(),
-                                          user='root',
-                                          group='root',
-                                          mode=0o0640)
-
-    def pre_write(self):
-        # Always remove?
-        for pkg in ['libpam-ccreds', 'libpam-cracklib']:
-            log("Purging package '%s'" % pkg, level=DEBUG),
-            apt_purge(pkg)
-
-        apt_update(fatal=True)
-        for pkg in ['libpam-passwdqc']:
-            log("Installing package '%s'" % pkg, level=DEBUG),
-            apt_install(pkg)
-
-    def post_write(self):
-        """Updates the PAM configuration after the file has been written"""
-        try:
-            check_output(['pam-auth-update', '--package'])
-        except CalledProcessError as e:
-            log('Error calling pam-auth-update: %s' % e, level=ERROR)
-
-
-class Tally2PAMContext(object):
-
-    def __call__(self):
-        ctxt = {}
-        settings = utils.get_settings('os')
-
-        ctxt['auth_lockout_time'] = settings['auth']['lockout_time']
-        ctxt['auth_retries'] = settings['auth']['retries']
-
-        return ctxt
-
-
-class Tally2PAM(TemplatedFile):
-    """The PAM Audit verifies the linux PAM settings."""
-    def __init__(self, path):
-        super(Tally2PAM, self).__init__(path=path,
-                                        template_dir=TEMPLATES_DIR,
-                                        context=Tally2PAMContext(),
-                                        user='root',
-                                        group='root',
-                                        mode=0o0640)
-
-    def pre_write(self):
-        # Always remove?
-        apt_purge('libpam-ccreds')
-        apt_update(fatal=True)
-        apt_install('libpam-modules')
-
-    def post_write(self):
-        """Updates the PAM configuration after the file has been written"""
-        try:
-            check_output(['pam-auth-update', '--package'])
-        except CalledProcessError as e:
-            log('Error calling pam-auth-update: %s' % e, level=ERROR)

=== removed file 'hooks/charmhelpers/contrib/hardening/host/checks/profile.py'
--- hooks/charmhelpers/contrib/hardening/host/checks/profile.py	2016-04-22 04:53:43 +0000
+++ hooks/charmhelpers/contrib/hardening/host/checks/profile.py	1970-01-01 00:00:00 +0000
@@ -1,45 +0,0 @@
-# Copyright 2016 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-from charmhelpers.contrib.hardening.audits.file import TemplatedFile
-from charmhelpers.contrib.hardening.host import TEMPLATES_DIR
-from charmhelpers.contrib.hardening import utils
-
-
-def get_audits():
-    """Get OS hardening profile audits.
-
-    :returns:  dictionary of audits
-    """
-    audits = []
-
-    settings = utils.get_settings('os')
-
-    # If core dumps are not enabled, then don't allow core dumps to be
-    # created as they may contain sensitive information.
-    if not settings['security']['kernel_enable_core_dump']:
-        audits.append(TemplatedFile('/etc/profile.d/pinerolo_profile.sh',
-                                    ProfileContext(),
-                                    template_dir=TEMPLATES_DIR,
-                                    mode=0o0755, user='root', group='root'))
-    return audits
-
-
-class ProfileContext(object):
-
-    def __call__(self):
-        ctxt = {}
-        return ctxt

=== removed file 'hooks/charmhelpers/contrib/hardening/host/checks/securetty.py'
--- hooks/charmhelpers/contrib/hardening/host/checks/securetty.py	2016-04-22 04:53:43 +0000
+++ hooks/charmhelpers/contrib/hardening/host/checks/securetty.py	1970-01-01 00:00:00 +0000
@@ -1,39 +0,0 @@
-# Copyright 2016 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-from charmhelpers.contrib.hardening.audits.file import TemplatedFile
-from charmhelpers.contrib.hardening.host import TEMPLATES_DIR
-from charmhelpers.contrib.hardening import utils
-
-
-def get_audits():
-    """Get OS hardening Secure TTY audits.
-
-    :returns:  dictionary of audits
-    """
-    audits = []
-    audits.append(TemplatedFile('/etc/securetty', SecureTTYContext(),
-                                template_dir=TEMPLATES_DIR,
-                                mode=0o0400, user='root', group='root'))
-    return audits
-
-
-class SecureTTYContext(object):
-
-    def __call__(self):
-        settings = utils.get_settings('os')
-        ctxt = {'ttys': settings['auth']['root_ttys']}
-        return ctxt

=== removed file 'hooks/charmhelpers/contrib/hardening/host/checks/suid_sgid.py'
--- hooks/charmhelpers/contrib/hardening/host/checks/suid_sgid.py	2016-04-22 04:53:43 +0000
+++ hooks/charmhelpers/contrib/hardening/host/checks/suid_sgid.py	1970-01-01 00:00:00 +0000
@@ -1,131 +0,0 @@
-# Copyright 2016 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-import subprocess
-
-from charmhelpers.core.hookenv import (
-    log,
-    INFO,
-)
-from charmhelpers.contrib.hardening.audits.file import NoSUIDSGIDAudit
-from charmhelpers.contrib.hardening import utils
-
-
-BLACKLIST = ['/usr/bin/rcp', '/usr/bin/rlogin', '/usr/bin/rsh',
-             '/usr/libexec/openssh/ssh-keysign',
-             '/usr/lib/openssh/ssh-keysign',
-             '/sbin/netreport',
-             '/usr/sbin/usernetctl',
-             '/usr/sbin/userisdnctl',
-             '/usr/sbin/pppd',
-             '/usr/bin/lockfile',
-             '/usr/bin/mail-lock',
-             '/usr/bin/mail-unlock',
-             '/usr/bin/mail-touchlock',
-             '/usr/bin/dotlockfile',
-             '/usr/bin/arping',
-             '/usr/sbin/uuidd',
-             '/usr/bin/mtr',
-             '/usr/lib/evolution/camel-lock-helper-1.2',
-             '/usr/lib/pt_chown',
-             '/usr/lib/eject/dmcrypt-get-device',
-             '/usr/lib/mc/cons.saver']
-
-WHITELIST = ['/bin/mount', '/bin/ping', '/bin/su', '/bin/umount',
-             '/sbin/pam_timestamp_check', '/sbin/unix_chkpwd', '/usr/bin/at',
-             '/usr/bin/gpasswd', '/usr/bin/locate', '/usr/bin/newgrp',
-             '/usr/bin/passwd', '/usr/bin/ssh-agent',
-             '/usr/libexec/utempter/utempter', '/usr/sbin/lockdev',
-             '/usr/sbin/sendmail.sendmail', '/usr/bin/expiry',
-             '/bin/ping6', '/usr/bin/traceroute6.iputils',
-             '/sbin/mount.nfs', '/sbin/umount.nfs',
-             '/sbin/mount.nfs4', '/sbin/umount.nfs4',
-             '/usr/bin/crontab',
-             '/usr/bin/wall', '/usr/bin/write',
-             '/usr/bin/screen',
-             '/usr/bin/mlocate',
-             '/usr/bin/chage', '/usr/bin/chfn', '/usr/bin/chsh',
-             '/bin/fusermount',
-             '/usr/bin/pkexec',
-             '/usr/bin/sudo', '/usr/bin/sudoedit',
-             '/usr/sbin/postdrop', '/usr/sbin/postqueue',
-             '/usr/sbin/suexec',
-             '/usr/lib/squid/ncsa_auth', '/usr/lib/squid/pam_auth',
-             '/usr/kerberos/bin/ksu',
-             '/usr/sbin/ccreds_validate',
-             '/usr/bin/Xorg',
-             '/usr/bin/X',
-             '/usr/lib/dbus-1.0/dbus-daemon-launch-helper',
-             '/usr/lib/vte/gnome-pty-helper',
-             '/usr/lib/libvte9/gnome-pty-helper',
-             '/usr/lib/libvte-2.90-9/gnome-pty-helper']
-
-
-def get_audits():
-    """Get OS hardening suid/sgid audits.
-
-    :returns:  dictionary of audits
-    """
-    checks = []
-    settings = utils.get_settings('os')
-    if not settings['security']['suid_sgid_enforce']:
-        log("Skipping suid/sgid hardening", level=INFO)
-        return checks
-
-    # Build the blacklist and whitelist of files for suid/sgid checks.
-    # There are a total of 4 lists:
-    #   1. the system blacklist
-    #   2. the system whitelist
-    #   3. the user blacklist
-    #   4. the user whitelist
-    #
-    # The blacklist is the set of paths which should NOT have the suid/sgid bit
-    # set and the whitelist is the set of paths which MAY have the suid/sgid
-    # bit setl. The user whitelist/blacklist effectively override the system
-    # whitelist/blacklist.
-    u_b = settings['security']['suid_sgid_blacklist']
-    u_w = settings['security']['suid_sgid_whitelist']
-
-    blacklist = set(BLACKLIST) - set(u_w + u_b)
-    whitelist = set(WHITELIST) - set(u_b + u_w)
-
-    checks.append(NoSUIDSGIDAudit(blacklist))
-
-    dry_run = settings['security']['suid_sgid_dry_run_on_unknown']
-
-    if settings['security']['suid_sgid_remove_from_unknown'] or dry_run:
-        # If the policy is a dry_run (e.g. complain only) or remove unknown
-        # suid/sgid bits then find all of the paths which have the suid/sgid
-        # bit set and then remove the whitelisted paths.
-        root_path = settings['environment']['root_path']
-        unknown_paths = find_paths_with_suid_sgid(root_path) - set(whitelist)
-        checks.append(NoSUIDSGIDAudit(unknown_paths, unless=dry_run))
-
-    return checks
-
-
-def find_paths_with_suid_sgid(root_path):
-    """Finds all paths/files which have an suid/sgid bit enabled.
-
-    Starting with the root_path, this will recursively find all paths which
-    have an suid or sgid bit set.
-    """
-    cmd = ['find', root_path, '-perm', '-4000', '-o', '-perm', '-2000',
-           '-type', 'f', '!', '-path', '/proc/*', '-print']
-
-    p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
-    out, _ = p.communicate()
-    return set(out.split('\n'))

=== removed file 'hooks/charmhelpers/contrib/hardening/host/checks/sysctl.py'
--- hooks/charmhelpers/contrib/hardening/host/checks/sysctl.py	2016-04-22 04:53:43 +0000
+++ hooks/charmhelpers/contrib/hardening/host/checks/sysctl.py	1970-01-01 00:00:00 +0000
@@ -1,211 +0,0 @@
-# Copyright 2016 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-import os
-import platform
-import re
-import six
-import subprocess
-
-from charmhelpers.core.hookenv import (
-    log,
-    INFO,
-    WARNING,
-)
-from charmhelpers.contrib.hardening import utils
-from charmhelpers.contrib.hardening.audits.file import (
-    FilePermissionAudit,
-    TemplatedFile,
-)
-from charmhelpers.contrib.hardening.host import TEMPLATES_DIR
-
-
-SYSCTL_DEFAULTS = """net.ipv4.ip_forward=%(net_ipv4_ip_forward)s
-net.ipv6.conf.all.forwarding=%(net_ipv6_conf_all_forwarding)s
-net.ipv4.conf.all.rp_filter=1
-net.ipv4.conf.default.rp_filter=1
-net.ipv4.icmp_echo_ignore_broadcasts=1
-net.ipv4.icmp_ignore_bogus_error_responses=1
-net.ipv4.icmp_ratelimit=100
-net.ipv4.icmp_ratemask=88089
-net.ipv6.conf.all.disable_ipv6=%(net_ipv6_conf_all_disable_ipv6)s
-net.ipv4.tcp_timestamps=%(net_ipv4_tcp_timestamps)s
-net.ipv4.conf.all.arp_ignore=%(net_ipv4_conf_all_arp_ignore)s
-net.ipv4.conf.all.arp_announce=%(net_ipv4_conf_all_arp_announce)s
-net.ipv4.tcp_rfc1337=1
-net.ipv4.tcp_syncookies=1
-net.ipv4.conf.all.shared_media=1
-net.ipv4.conf.default.shared_media=1
-net.ipv4.conf.all.accept_source_route=0
-net.ipv4.conf.default.accept_source_route=0
-net.ipv4.conf.all.accept_redirects=0
-net.ipv4.conf.default.accept_redirects=0
-net.ipv6.conf.all.accept_redirects=0
-net.ipv6.conf.default.accept_redirects=0
-net.ipv4.conf.all.secure_redirects=0
-net.ipv4.conf.default.secure_redirects=0
-net.ipv4.conf.all.send_redirects=0
-net.ipv4.conf.default.send_redirects=0
-net.ipv4.conf.all.log_martians=0
-net.ipv6.conf.default.router_solicitations=0
-net.ipv6.conf.default.accept_ra_rtr_pref=0
-net.ipv6.conf.default.accept_ra_pinfo=0
-net.ipv6.conf.default.accept_ra_defrtr=0
-net.ipv6.conf.default.autoconf=0
-net.ipv6.conf.default.dad_transmits=0
-net.ipv6.conf.default.max_addresses=1
-net.ipv6.conf.all.accept_ra=0
-net.ipv6.conf.default.accept_ra=0
-kernel.modules_disabled=%(kernel_modules_disabled)s
-kernel.sysrq=%(kernel_sysrq)s
-fs.suid_dumpable=%(fs_suid_dumpable)s
-kernel.randomize_va_space=2
-"""
-
-
-def get_audits():
-    """Get OS hardening sysctl audits.
-
-    :returns:  dictionary of audits
-    """
-    audits = []
-    settings = utils.get_settings('os')
-
-    # Apply the sysctl settings which are configured to be applied.
-    audits.append(SysctlConf())
-    # Make sure that only root has access to the sysctl.conf file, and
-    # that it is read-only.
-    audits.append(FilePermissionAudit('/etc/sysctl.conf',
-                                      user='root',
-                                      group='root', mode=0o0440))
-    # If module loading is not enabled, then ensure that the modules
-    # file has the appropriate permissions and rebuild the initramfs
-    if not settings['security']['kernel_enable_module_loading']:
-        audits.append(ModulesTemplate())
-
-    return audits
-
-
-class ModulesContext(object):
-
-    def __call__(self):
-        settings = utils.get_settings('os')
-        with open('/proc/cpuinfo', 'r') as fd:
-            cpuinfo = fd.readlines()
-
-        for line in cpuinfo:
-            match = re.search(r"^vendor_id\s+:\s+(.+)", line)
-            if match:
-                vendor = match.group(1)
-
-        if vendor == "GenuineIntel":
-            vendor = "intel"
-        elif vendor == "AuthenticAMD":
-            vendor = "amd"
-
-        ctxt = {'arch': platform.processor(),
-                'cpuVendor': vendor,
-                'desktop_enable': settings['general']['desktop_enable']}
-
-        return ctxt
-
-
-class ModulesTemplate(object):
-
-    def __init__(self):
-        super(ModulesTemplate, self).__init__('/etc/initramfs-tools/modules',
-                                              ModulesContext(),
-                                              templates_dir=TEMPLATES_DIR,
-                                              user='root', group='root',
-                                              mode=0o0440)
-
-    def post_write(self):
-        subprocess.check_call(['update-initramfs', '-u'])
-
-
-class SysCtlHardeningContext(object):
-    def __call__(self):
-        settings = utils.get_settings('os')
-        ctxt = {'sysctl': {}}
-
-        log("Applying sysctl settings", level=INFO)
-        extras = {'net_ipv4_ip_forward': 0,
-                  'net_ipv6_conf_all_forwarding': 0,
-                  'net_ipv6_conf_all_disable_ipv6': 1,
-                  'net_ipv4_tcp_timestamps': 0,
-                  'net_ipv4_conf_all_arp_ignore': 0,
-                  'net_ipv4_conf_all_arp_announce': 0,
-                  'kernel_sysrq': 0,
-                  'fs_suid_dumpable': 0,
-                  'kernel_modules_disabled': 1}
-
-        if settings['sysctl']['ipv6_enable']:
-            extras['net_ipv6_conf_all_disable_ipv6'] = 0
-
-        if settings['sysctl']['forwarding']:
-            extras['net_ipv4_ip_forward'] = 1
-            extras['net_ipv6_conf_all_forwarding'] = 1
-
-        if settings['sysctl']['arp_restricted']:
-            extras['net_ipv4_conf_all_arp_ignore'] = 1
-            extras['net_ipv4_conf_all_arp_announce'] = 2
-
-        if settings['security']['kernel_enable_module_loading']:
-            extras['kernel_modules_disabled'] = 0
-
-        if settings['sysctl']['kernel_enable_sysrq']:
-            sysrq_val = settings['sysctl']['kernel_secure_sysrq']
-            extras['kernel_sysrq'] = sysrq_val
-
-        if settings['security']['kernel_enable_core_dump']:
-            extras['fs_suid_dumpable'] = 1
-
-        settings.update(extras)
-        for d in (SYSCTL_DEFAULTS % settings).split():
-            d = d.strip().partition('=')
-            key = d[0].strip()
-            path = os.path.join('/proc/sys', key.replace('.', '/'))
-            if not os.path.exists(path):
-                log("Skipping '%s' since '%s' does not exist" % (key, path),
-                    level=WARNING)
-                continue
-
-            ctxt['sysctl'][key] = d[2] or None
-
-        # Translate for python3
-        return {'sysctl_settings':
-                [(k, v) for k, v in six.iteritems(ctxt['sysctl'])]}
-
-
-class SysctlConf(TemplatedFile):
-    """An audit check for sysctl settings."""
-    def __init__(self):
-        self.conffile = '/etc/sysctl.d/99-juju-hardening.conf'
-        super(SysctlConf, self).__init__(self.conffile,
-                                         SysCtlHardeningContext(),
-                                         template_dir=TEMPLATES_DIR,
-                                         user='root', group='root',
-                                         mode=0o0440)
-
-    def post_write(self):
-        try:
-            subprocess.check_call(['sysctl', '-p', self.conffile])
-        except subprocess.CalledProcessError as e:
-            # NOTE: on some systems if sysctl cannot apply all settings it
-            #       will return non-zero as well.
-            log("sysctl command returned an error (maybe some "
-                "keys could not be set) - %s" % (e),
-                level=WARNING)

=== removed directory 'hooks/charmhelpers/contrib/hardening/mysql'
=== removed file 'hooks/charmhelpers/contrib/hardening/mysql/__init__.py'
--- hooks/charmhelpers/contrib/hardening/mysql/__init__.py	2016-04-22 04:53:43 +0000
+++ hooks/charmhelpers/contrib/hardening/mysql/__init__.py	1970-01-01 00:00:00 +0000
@@ -1,19 +0,0 @@
-# Copyright 2016 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-from os import path
-
-TEMPLATES_DIR = path.join(path.dirname(__file__), 'templates')

=== removed directory 'hooks/charmhelpers/contrib/hardening/mysql/checks'
=== removed file 'hooks/charmhelpers/contrib/hardening/mysql/checks/__init__.py'
--- hooks/charmhelpers/contrib/hardening/mysql/checks/__init__.py	2016-04-22 04:53:43 +0000
+++ hooks/charmhelpers/contrib/hardening/mysql/checks/__init__.py	1970-01-01 00:00:00 +0000
@@ -1,31 +0,0 @@
-# Copyright 2016 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-from charmhelpers.core.hookenv import (
-    log,
-    DEBUG,
-)
-from charmhelpers.contrib.hardening.mysql.checks import config
-
-
-def run_mysql_checks():
-    log("Starting MySQL hardening checks.", level=DEBUG)
-    checks = config.get_audits()
-    for check in checks:
-        log("Running '%s' check" % (check.__class__.__name__), level=DEBUG)
-        check.ensure_compliance()
-
-    log("MySQL hardening checks complete.", level=DEBUG)

=== removed file 'hooks/charmhelpers/contrib/hardening/mysql/checks/config.py'
--- hooks/charmhelpers/contrib/hardening/mysql/checks/config.py	2016-04-22 04:53:43 +0000
+++ hooks/charmhelpers/contrib/hardening/mysql/checks/config.py	1970-01-01 00:00:00 +0000
@@ -1,89 +0,0 @@
-# Copyright 2016 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-import six
-import subprocess
-
-from charmhelpers.core.hookenv import (
-    log,
-    WARNING,
-)
-from charmhelpers.contrib.hardening.audits.file import (
-    FilePermissionAudit,
-    DirectoryPermissionAudit,
-    TemplatedFile,
-)
-from charmhelpers.contrib.hardening.mysql import TEMPLATES_DIR
-from charmhelpers.contrib.hardening import utils
-
-
-def get_audits():
-    """Get MySQL hardening config audits.
-
-    :returns:  dictionary of audits
-    """
-    if subprocess.call(['which', 'mysql'], stdout=subprocess.PIPE) != 0:
-        log("MySQL does not appear to be installed on this node - "
-            "skipping mysql hardening", level=WARNING)
-        return []
-
-    settings = utils.get_settings('mysql')
-    hardening_settings = settings['hardening']
-    my_cnf = hardening_settings['mysql-conf']
-
-    audits = [
-        FilePermissionAudit(paths=[my_cnf], user='root',
-                            group='root', mode=0o0600),
-
-        TemplatedFile(hardening_settings['hardening-conf'],
-                      MySQLConfContext(),
-                      TEMPLATES_DIR,
-                      mode=0o0750,
-                      user='mysql',
-                      group='root',
-                      service_actions=[{'service': 'mysql',
-                                        'actions': ['restart']}]),
-
-        # MySQL and Percona charms do not allow configuration of the
-        # data directory, so use the default.
-        DirectoryPermissionAudit('/var/lib/mysql',
-                                 user='mysql',
-                                 group='mysql',
-                                 recursive=False,
-                                 mode=0o755),
-
-        DirectoryPermissionAudit('/etc/mysql',
-                                 user='root',
-                                 group='root',
-                                 recursive=False,
-                                 mode=0o700),
-    ]
-
-    return audits
-
-
-class MySQLConfContext(object):
-    """Defines the set of key/value pairs to set in a mysql config file.
-
-    This context, when called, will return a dictionary containing the
-    key/value pairs of setting to specify in the
-    /etc/mysql/conf.d/hardening.cnf file.
-    """
-    def __call__(self):
-        settings = utils.get_settings('mysql')
-        # Translate for python3
-        return {'mysql_settings':
-                [(k, v) for k, v in six.iteritems(settings['security'])]}

=== removed directory 'hooks/charmhelpers/contrib/hardening/ssh'
=== removed file 'hooks/charmhelpers/contrib/hardening/ssh/__init__.py'
--- hooks/charmhelpers/contrib/hardening/ssh/__init__.py	2016-04-22 04:53:43 +0000
+++ hooks/charmhelpers/contrib/hardening/ssh/__init__.py	1970-01-01 00:00:00 +0000
@@ -1,19 +0,0 @@
-# Copyright 2016 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-from os import path
-
-TEMPLATES_DIR = path.join(path.dirname(__file__), 'templates')

=== removed directory 'hooks/charmhelpers/contrib/hardening/ssh/checks'
=== removed file 'hooks/charmhelpers/contrib/hardening/ssh/checks/__init__.py'
--- hooks/charmhelpers/contrib/hardening/ssh/checks/__init__.py	2016-04-22 04:53:43 +0000
+++ hooks/charmhelpers/contrib/hardening/ssh/checks/__init__.py	1970-01-01 00:00:00 +0000
@@ -1,31 +0,0 @@
-# Copyright 2016 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-from charmhelpers.core.hookenv import (
-    log,
-    DEBUG,
-)
-from charmhelpers.contrib.hardening.ssh.checks import config
-
-
-def run_ssh_checks():
-    log("Starting SSH hardening checks.", level=DEBUG)
-    checks = config.get_audits()
-    for check in checks:
-        log("Running '%s' check" % (check.__class__.__name__), level=DEBUG)
-        check.ensure_compliance()
-
-    log("SSH hardening checks complete.", level=DEBUG)

=== removed file 'hooks/charmhelpers/contrib/hardening/ssh/checks/config.py'
--- hooks/charmhelpers/contrib/hardening/ssh/checks/config.py	2016-04-22 04:53:43 +0000
+++ hooks/charmhelpers/contrib/hardening/ssh/checks/config.py	1970-01-01 00:00:00 +0000
@@ -1,394 +0,0 @@
-# Copyright 2016 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-import os
-
-from charmhelpers.core.hookenv import (
-    log,
-    DEBUG,
-)
-from charmhelpers.fetch import (
-    apt_install,
-    apt_update,
-)
-from charmhelpers.core.host import lsb_release
-from charmhelpers.contrib.hardening.audits.file import (
-    TemplatedFile,
-    FileContentAudit,
-)
-from charmhelpers.contrib.hardening.ssh import TEMPLATES_DIR
-from charmhelpers.contrib.hardening import utils
-
-
-def get_audits():
-    """Get SSH hardening config audits.
-
-    :returns:  dictionary of audits
-    """
-    audits = [SSHConfig(), SSHDConfig(), SSHConfigFileContentAudit(),
-              SSHDConfigFileContentAudit()]
-    return audits
-
-
-class SSHConfigContext(object):
-
-    type = 'client'
-
-    def get_macs(self, allow_weak_mac):
-        if allow_weak_mac:
-            weak_macs = 'weak'
-        else:
-            weak_macs = 'default'
-
-        default = 'hmac-sha2-512,hmac-sha2-256,hmac-ripemd160'
-        macs = {'default': default,
-                'weak': default + ',hmac-sha1'}
-
-        default = ('[e-mail],'
-                   '[e-mail],'
-                   '[e-mail],[e-mail],'
-                   'hmac-sha2-512,hmac-sha2-256,hmac-ripemd160')
-        macs_66 = {'default': default,
-                   'weak': default + ',hmac-sha1'}
-
-        # Use newer ciphers on Ubuntu Trusty and above
-        if lsb_release()['DISTRIB_CODENAME'].lower() >= 'trusty':
-            log("Detected Ubuntu 14.04 or newer, using new macs", level=DEBUG)
-            macs = macs_66
-
-        return macs[weak_macs]
-
-    def get_kexs(self, allow_weak_kex):
-        if allow_weak_kex:
-            weak_kex = 'weak'
-        else:
-            weak_kex = 'default'
-
-        default = 'diffie-hellman-group-exchange-sha256'
-        weak = (default + ',diffie-hellman-group14-sha1,'
-                'diffie-hellman-group-exchange-sha1,'
-                'diffie-hellman-group1-sha1')
-        kex = {'default': default,
-               'weak': weak}
-
-        default = ('[e-mail],'
-                   'diffie-hellman-group-exchange-sha256')
-        weak = (default + ',diffie-hellman-group14-sha1,'
-                'diffie-hellman-group-exchange-sha1,'
-                'diffie-hellman-group1-sha1')
-        kex_66 = {'default': default,
-                  'weak': weak}
-
-        # Use newer kex on Ubuntu Trusty and above
-        if lsb_release()['DISTRIB_CODENAME'].lower() >= 'trusty':
-            log('Detected Ubuntu 14.04 or newer, using new key exchange '
-                'algorithms', level=DEBUG)
-            kex = kex_66
-
-        return kex[weak_kex]
-
-    def get_ciphers(self, cbc_required):
-        if cbc_required:
-            weak_ciphers = 'weak'
-        else:
-            weak_ciphers = 'default'
-
-        default = 'aes256-ctr,aes192-ctr,aes128-ctr'
-        cipher = {'default': default,
-                  'weak': default + 'aes256-cbc,aes192-cbc,aes128-cbc'}
-
-        default = ('[e-mail],[e-mail],'
-                   '[e-mail],aes256-ctr,aes192-ctr,aes128-ctr')
-        ciphers_66 = {'default': default,
-                      'weak': default + ',aes256-cbc,aes192-cbc,aes128-cbc'}
-
-        # Use newer ciphers on ubuntu Trusty and above
-        if lsb_release()['DISTRIB_CODENAME'].lower() >= 'trusty':
-            log('Detected Ubuntu 14.04 or newer, using new ciphers',
-                level=DEBUG)
-            cipher = ciphers_66
-
-        return cipher[weak_ciphers]
-
-    def __call__(self):
-        settings = utils.get_settings('ssh')
-        if settings['common']['network_ipv6_enable']:
-            addr_family = 'any'
-        else:
-            addr_family = 'inet'
-
-        ctxt = {
-            'addr_family': addr_family,
-            'remote_hosts': settings['common']['remote_hosts'],
-            'password_auth_allowed':
-            settings['client']['password_authentication'],
-            'ports': settings['common']['ports'],
-            'ciphers': self.get_ciphers(settings['client']['cbc_required']),
-            'macs': self.get_macs(settings['client']['weak_hmac']),
-            'kexs': self.get_kexs(settings['client']['weak_kex']),
-            'roaming': settings['client']['roaming'],
-        }
-        return ctxt
-
-
-class SSHConfig(TemplatedFile):
-    def __init__(self):
-        path = '/etc/ssh/ssh_config'
-        super(SSHConfig, self).__init__(path=path,
-                                        template_dir=TEMPLATES_DIR,
-                                        context=SSHConfigContext(),
-                                        user='root',
-                                        group='root',
-                                        mode=0o0644)
-
-    def pre_write(self):
-        settings = utils.get_settings('ssh')
-        apt_update(fatal=True)
-        apt_install(settings['client']['package'])
-        if not os.path.exists('/etc/ssh'):
-            os.makedir('/etc/ssh')
-            # NOTE: don't recurse
-            utils.ensure_permissions('/etc/ssh', 'root', 'root', 0o0755,
-                                     maxdepth=0)
-
-    def post_write(self):
-        # NOTE: don't recurse
-        utils.ensure_permissions('/etc/ssh', 'root', 'root', 0o0755,
-                                 maxdepth=0)
-
-
-class SSHDConfigContext(SSHConfigContext):
-
-    type = 'server'
-
-    def __call__(self):
-        settings = utils.get_settings('ssh')
-        if settings['common']['network_ipv6_enable']:
-            addr_family = 'any'
-        else:
-            addr_family = 'inet'
-
-        ctxt = {
-            'ssh_ip': settings['server']['listen_to'],
-            'password_auth_allowed':
-            settings['server']['password_authentication'],
-            'ports': settings['common']['ports'],
-            'addr_family': addr_family,
-            'ciphers': self.get_ciphers(settings['server']['cbc_required']),
-            'macs': self.get_macs(settings['server']['weak_hmac']),
-            'kexs': self.get_kexs(settings['server']['weak_kex']),
-            'host_key_files': settings['server']['host_key_files'],
-            'allow_root_with_key': settings['server']['allow_root_with_key'],
-            'password_authentication':
-            settings['server']['password_authentication'],
-            'use_priv_sep': settings['server']['use_privilege_separation'],
-            'use_pam': settings['server']['use_pam'],
-            'allow_x11_forwarding': settings['server']['allow_x11_forwarding'],
-            'print_motd': settings['server']['print_motd'],
-            'print_last_log': settings['server']['print_last_log'],
-            'client_alive_interval':
-            settings['server']['alive_interval'],
-            'client_alive_count': settings['server']['alive_count'],
-            'allow_tcp_forwarding': settings['server']['allow_tcp_forwarding'],
-            'allow_agent_forwarding':
-            settings['server']['allow_agent_forwarding'],
-            'deny_users': settings['server']['deny_users'],
-            'allow_users': settings['server']['allow_users'],
-            'deny_groups': settings['server']['deny_groups'],
-            'allow_groups': settings['server']['allow_groups'],
-            'use_dns': settings['server']['use_dns'],
-            'sftp_enable': settings['server']['sftp_enable'],
-            'sftp_group': settings['server']['sftp_group'],
-            'sftp_chroot': settings['server']['sftp_chroot'],
-            'max_auth_tries': settings['server']['max_auth_tries'],
-            'max_sessions': settings['server']['max_sessions'],
-        }
-        return ctxt
-
-
-class SSHDConfig(TemplatedFile):
-    def __init__(self):
-        path = '/etc/ssh/sshd_config'
-        super(SSHDConfig, self).__init__(path=path,
-                                         template_dir=TEMPLATES_DIR,
-                                         context=SSHDConfigContext(),
-                                         user='root',
-                                         group='root',
-                                         mode=0o0600,
-                                         service_actions=[{'service': 'ssh',
-                                                           'actions':
-                                                           ['restart']}])
-
-    def pre_write(self):
-        settings = utils.get_settings('ssh')
-        apt_update(fatal=True)
-        apt_install(settings['server']['package'])
-        if not os.path.exists('/etc/ssh'):
-            os.makedir('/etc/ssh')
-            # NOTE: don't recurse
-            utils.ensure_permissions('/etc/ssh', 'root', 'root', 0o0755,
-                                     maxdepth=0)
-
-    def post_write(self):
-        # NOTE: don't recurse
-        utils.ensure_permissions('/etc/ssh', 'root', 'root', 0o0755,
-                                 maxdepth=0)
-
-
-class SSHConfigFileContentAudit(FileContentAudit):
-    def __init__(self):
-        self.path = '/etc/ssh/ssh_config'
-        super(SSHConfigFileContentAudit, self).__init__(self.path, {})
-
-    def is_compliant(self, *args, **kwargs):
-        self.pass_cases = []
-        self.fail_cases = []
-        settings = utils.get_settings('ssh')
-
-        if lsb_release()['DISTRIB_CODENAME'].lower() >= 'trusty':
-            if not settings['server']['weak_hmac']:
-                self.pass_cases.append(r'^MACs.+,hmac-ripemd160$')
-            else:
-                self.pass_cases.append(r'^MACs.+,hmac-sha1$')
-
-            if settings['server']['weak_kex']:
-                self.fail_cases.append(r'^KexAlgorithms\sdiffie-hellman-group-exchange-sha256[,\s]?')  # noqa
-                self.pass_cases.append(r'^KexAlgorithms\sdiffie-hellman-group14-sha1[,\s]?')  # noqa
-                self.pass_cases.append(r'^KexAlgorithms\sdiffie-hellman-group-exchange-sha1[,\s]?')  # noqa
-                self.pass_cases.append(r'^KexAlgorithms\sdiffie-hellman-group1-sha1[,\s]?')  # noqa
-            else:
-                self.pass_cases.append(r'^KexAlgorithms.+,diffie-hellman-group-exchange-sha256$')  # noqa
-                self.fail_cases.append(r'^KexAlgorithms.*diffie-hellman-group14-sha1[,\s]?')  # noqa
-
-            if settings['server']['cbc_required']:
-                self.pass_cases.append(r'^Ciphers\s.*-cbc[,\s]?')
-                self.fail_cases.append(r'^Ciphers\s.*aes128-ctr[,\s]?')
-                self.fail_cases.append(r'^Ciphers\s.*aes192-ctr[,\s]?')
-                self.fail_cases.append(r'^Ciphers\s.*aes256-ctr[,\s]?')
-            else:
-                self.fail_cases.append(r'^Ciphers\s.*-cbc[,\s]?')
-                self.pass_cases.append(r'^Ciphers\[e-mail],.+')  # noqa
-                self.pass_cases.append(r'^Ciphers\s.*aes128-ctr$')
-                self.pass_cases.append(r'^Ciphers\s.*aes192-ctr[,\s]?')
-                self.pass_cases.append(r'^Ciphers\s.*aes256-ctr[,\s]?')
-        else:
-            if not settings['client']['weak_hmac']:
-                self.fail_cases.append(r'^MACs.+,hmac-sha1$')
-            else:
-                self.pass_cases.append(r'^MACs.+,hmac-sha1$')
-
-            if settings['client']['weak_kex']:
-                self.fail_cases.append(r'^KexAlgorithms\sdiffie-hellman-group-exchange-sha256[,\s]?')  # noqa
-                self.pass_cases.append(r'^KexAlgorithms\sdiffie-hellman-group14-sha1[,\s]?')  # noqa
-                self.pass_cases.append(r'^KexAlgorithms\sdiffie-hellman-group-exchange-sha1[,\s]?')  # noqa
-                self.pass_cases.append(r'^KexAlgorithms\sdiffie-hellman-group1-sha1[,\s]?')  # noqa
-            else:
-                self.pass_cases.append(r'^KexAlgorithms\sdiffie-hellman-group-exchange-sha256$')  # noqa
-                self.fail_cases.append(r'^KexAlgorithms\sdiffie-hellman-group14-sha1[,\s]?')  # noqa
-                self.fail_cases.append(r'^KexAlgorithms\sdiffie-hellman-group-exchange-sha1[,\s]?')  # noqa
-                self.fail_cases.append(r'^KexAlgorithms\sdiffie-hellman-group1-sha1[,\s]?')  # noqa
-
-            if settings['client']['cbc_required']:
-                self.pass_cases.append(r'^Ciphers\s.*-cbc[,\s]?')
-                self.fail_cases.append(r'^Ciphers\s.*aes128-ctr[,\s]?')
-                self.fail_cases.append(r'^Ciphers\s.*aes192-ctr[,\s]?')
-                self.fail_cases.append(r'^Ciphers\s.*aes256-ctr[,\s]?')
-            else:
-                self.fail_cases.append(r'^Ciphers\s.*-cbc[,\s]?')
-                self.pass_cases.append(r'^Ciphers\s.*aes128-ctr[,\s]?')
-                self.pass_cases.append(r'^Ciphers\s.*aes192-ctr[,\s]?')
-                self.pass_cases.append(r'^Ciphers\s.*aes256-ctr[,\s]?')
-
-        if settings['client']['roaming']:
-            self.pass_cases.append(r'^UseRoaming yes$')
-        else:
-            self.fail_cases.append(r'^UseRoaming yes$')
-
-        return super(SSHConfigFileContentAudit, self).is_compliant(*args,
-                                                                   **kwargs)
-
-
-class SSHDConfigFileContentAudit(FileContentAudit):
-    def __init__(self):
-        self.path = '/etc/ssh/sshd_config'
-        super(SSHDConfigFileContentAudit, self).__init__(self.path, {})
-
-    def is_compliant(self, *args, **kwargs):
-        self.pass_cases = []
-        self.fail_cases = []
-        settings = utils.get_settings('ssh')
-
-        if lsb_release()['DISTRIB_CODENAME'].lower() >= 'trusty':
-            if not settings['server']['weak_hmac']:
-                self.pass_cases.append(r'^MACs.+,hmac-ripemd160$')
-            else:
-                self.pass_cases.append(r'^MACs.+,hmac-sha1$')
-
-            if settings['server']['weak_kex']:
-                self.fail_cases.append(r'^KexAlgorithms\sdiffie-hellman-group-exchange-sha256[,\s]?')  # noqa
-                self.pass_cases.append(r'^KexAlgorithms\sdiffie-hellman-group14-sha1[,\s]?')  # noqa
-                self.pass_cases.append(r'^KexAlgorithms\sdiffie-hellman-group-exchange-sha1[,\s]?')  # noqa
-                self.pass_cases.append(r'^KexAlgorithms\sdiffie-hellman-group1-sha1[,\s]?')  # noqa
-            else:
-                self.pass_cases.append(r'^KexAlgorithms.+,diffie-hellman-group-exchange-sha256$')  # noqa
-                self.fail_cases.append(r'^KexAlgorithms.*diffie-hellman-group14-sha1[,\s]?')  # noqa
-
-            if settings['server']['cbc_required']:
-                self.pass_cases.append(r'^Ciphers\s.*-cbc[,\s]?')
-                self.fail_cases.append(r'^Ciphers\s.*aes128-ctr[,\s]?')
-                self.fail_cases.append(r'^Ciphers\s.*aes192-ctr[,\s]?')
-                self.fail_cases.append(r'^Ciphers\s.*aes256-ctr[,\s]?')
-            else:
-                self.fail_cases.append(r'^Ciphers\s.*-cbc[,\s]?')
-                self.pass_cases.append(r'^Ciphers\[e-mail],.+')  # noqa
-                self.pass_cases.append(r'^Ciphers\s.*aes128-ctr$')
-                self.pass_cases.append(r'^Ciphers\s.*aes192-ctr[,\s]?')
-                self.pass_cases.append(r'^Ciphers\s.*aes256-ctr[,\s]?')
-        else:
-            if not settings['server']['weak_hmac']:
-                self.pass_cases.append(r'^MACs.+,hmac-ripemd160$')
-            else:
-                self.pass_cases.append(r'^MACs.+,hmac-sha1$')
-
-            if settings['server']['weak_kex']:
-                self.fail_cases.append(r'^KexAlgorithms\sdiffie-hellman-group-exchange-sha256[,\s]?')  # noqa
-                self.pass_cases.append(r'^KexAlgorithms\sdiffie-hellman-group14-sha1[,\s]?')  # noqa
-                self.pass_cases.append(r'^KexAlgorithms\sdiffie-hellman-group-exchange-sha1[,\s]?')  # noqa
-                self.pass_cases.append(r'^KexAlgorithms\sdiffie-hellman-group1-sha1[,\s]?')  # noqa
-            else:
-                self.pass_cases.append(r'^KexAlgorithms\sdiffie-hellman-group-exchange-sha256$')  # noqa
-                self.fail_cases.append(r'^KexAlgorithms\sdiffie-hellman-group14-sha1[,\s]?')  # noqa
-                self.fail_cases.append(r'^KexAlgorithms\sdiffie-hellman-group-exchange-sha1[,\s]?')  # noqa
-                self.fail_cases.append(r'^KexAlgorithms\sdiffie-hellman-group1-sha1[,\s]?')  # noqa
-
-            if settings['server']['cbc_required']:
-                self.pass_cases.append(r'^Ciphers\s.*-cbc[,\s]?')
-                self.fail_cases.append(r'^Ciphers\s.*aes128-ctr[,\s]?')
-                self.fail_cases.append(r'^Ciphers\s.*aes192-ctr[,\s]?')
-                self.fail_cases.append(r'^Ciphers\s.*aes256-ctr[,\s]?')
-            else:
-                self.fail_cases.append(r'^Ciphers\s.*-cbc[,\s]?')
-                self.pass_cases.append(r'^Ciphers\s.*aes128-ctr[,\s]?')
-                self.pass_cases.append(r'^Ciphers\s.*aes192-ctr[,\s]?')
-                self.pass_cases.append(r'^Ciphers\s.*aes256-ctr[,\s]?')
-
-        if settings['server']['sftp_enable']:
-            self.pass_cases.append(r'^Subsystem\ssftp')
-        else:
-            self.fail_cases.append(r'^Subsystem\ssftp')
-
-        return super(SSHDConfigFileContentAudit, self).is_compliant(*args,
-                                                                    **kwargs)

=== removed file 'hooks/charmhelpers/contrib/hardening/templating.py'
--- hooks/charmhelpers/contrib/hardening/templating.py	2016-04-22 04:53:43 +0000
+++ hooks/charmhelpers/contrib/hardening/templating.py	1970-01-01 00:00:00 +0000
@@ -1,71 +0,0 @@
-# Copyright 2016 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-import os
-
-from charmhelpers.core.hookenv import (
-    log,
-    DEBUG,
-    WARNING,
-)
-
-try:
-    from jinja2 import FileSystemLoader, Environment
-except ImportError:
-    from charmhelpers.fetch import apt_install
-    from charmhelpers.fetch import apt_update
-    apt_update(fatal=True)
-    apt_install('python-jinja2', fatal=True)
-    from jinja2 import FileSystemLoader, Environment
-
-
-# NOTE: function separated from main rendering code to facilitate easier
-#       mocking in unit tests.
-def write(path, data):
-    with open(path, 'wb') as out:
-        out.write(data)
-
-
-def get_template_path(template_dir, path):
-    """Returns the template file which would be used to render the path.
-
-    The path to the template file is returned.
-    :param template_dir: the directory the templates are located in
-    :param path: the file path to be written to.
-    :returns: path to the template file
-    """
-    return os.path.join(template_dir, os.path.basename(path))
-
-
-def render_and_write(template_dir, path, context):
-    """Renders the specified template into the file.
-
-    :param template_dir: the directory to load the template from
-    :param path: the path to write the templated contents to
-    :param context: the parameters to pass to the rendering engine
-    """
-    env = Environment(loader=FileSystemLoader(template_dir))
-    template_file = os.path.basename(path)
-    template = env.get_template(template_file)
-    log('Rendering from template: %s' % template.name, level=DEBUG)
-    rendered_content = template.render(context)
-    if not rendered_content:
-        log("Render returned None - skipping '%s'" % path,
-            level=WARNING)
-        return
-
-    write(path, rendered_content.encode('utf-8').strip())
-    log('Wrote template %s' % path, level=DEBUG)

=== removed file 'hooks/charmhelpers/contrib/hardening/utils.py'
--- hooks/charmhelpers/contrib/hardening/utils.py	2016-04-22 04:53:43 +0000
+++ hooks/charmhelpers/contrib/hardening/utils.py	1970-01-01 00:00:00 +0000
@@ -1,157 +0,0 @@
-# Copyright 2016 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-import glob
-import grp
-import os
-import pwd
-import six
-import yaml
-
-from charmhelpers.core.hookenv import (
-    log,
-    DEBUG,
-    INFO,
-    WARNING,
-    ERROR,
-)
-
-
-# Global settings cache. Since each hook fire entails a fresh module import it
-# is safe to hold this in memory and not risk missing config changes (since
-# they will result in a new hook fire and thus re-import).
-__SETTINGS__ = {}
-
-
-def _get_defaults(modules):
-    """Load the default config for the provided modules.
-
-    :param modules: stack modules config defaults to lookup.
-    :returns: modules default config dictionary.
-    """
-    default = os.path.join(os.path.dirname(__file__),
-                           'defaults/%s.yaml' % (modules))
-    return yaml.safe_load(open(default))
-
-
-def _get_schema(modules):
-    """Load the config schema for the provided modules.
-
-    NOTE: this schema is intended to have 1-1 relationship with they keys in
-    the default config and is used a means to verify valid overrides provided
-    by the user.
-
-    :param modules: stack modules config schema to lookup.
-    :returns: modules default schema dictionary.
-    """
-    schema = os.path.join(os.path.dirname(__file__),
-                          'defaults/%s.yaml.schema' % (modules))
-    return yaml.safe_load(open(schema))
-
-
-def _get_user_provided_overrides(modules):
-    """Load user-provided config overrides.
-
-    :param modules: stack modules to lookup in user overrides yaml file.
-    :returns: overrides dictionary.
-    """
-    overrides = os.path.join(os.environ['JUJU_CHARM_DIR'],
-                             'hardening.yaml')
-    if os.path.exists(overrides):
-        log("Found user-provided config overrides file '%s'" %
-            (overrides), level=DEBUG)
-        settings = yaml.safe_load(open(overrides))
-        if settings and settings.get(modules):
-            log("Applying '%s' overrides" % (modules), level=DEBUG)
-            return settings.get(modules)
-
-        log("No overrides found for '%s'" % (modules), level=DEBUG)
-    else:
-        log("No hardening config overrides file '%s' found in charm "
-            "root dir" % (overrides), level=DEBUG)
-
-    return {}
-
-
-def _apply_overrides(settings, overrides, schema):
-    """Get overrides config overlayed onto modules defaults.
-
-    :param modules: require stack modules config.
-    :returns: dictionary of modules config with user overrides applied.
-    """
-    if overrides:
-        for k, v in six.iteritems(overrides):
-            if k in schema:
-                if schema[k] is None:
-                    settings[k] = v
-                elif type(schema[k]) is dict:
-                    settings[k] = _apply_overrides(settings[k], overrides[k],
-                                                   schema[k])
-                else:
-                    raise Exception("Unexpected type found in schema '%s'" %
-                                    type(schema[k]), level=ERROR)
-            else:
-                log("Unknown override key '%s' - ignoring" % (k), level=INFO)
-
-    return settings
-
-
-def get_settings(modules):
-    global __SETTINGS__
-    if modules in __SETTINGS__:
-        return __SETTINGS__[modules]
-
-    schema = _get_schema(modules)
-    settings = _get_defaults(modules)
-    overrides = _get_user_provided_overrides(modules)
-    __SETTINGS__[modules] = _apply_overrides(settings, overrides, schema)
-    return __SETTINGS__[modules]
-
-
-def ensure_permissions(path, user, group, permissions, maxdepth=-1):
-    """Ensure permissions for path.
-
-    If path is a file, apply to file and return. If path is a directory,
-    apply recursively (if required) to directory contents and return.
-
-    :param user: user name
-    :param group: group name
-    :param permissions: octal permissions
-    :param maxdepth: maximum recursion depth. A negative maxdepth allows
-                     infinite recursion and maxdepth=0 means no recursion.
-    :returns: None
-    """
-    if not os.path.exists(path):
-        log("File '%s' does not exist - cannot set permissions" % (path),
-            level=WARNING)
-        return
-
-    _user = pwd.getpwnam(user)
-    os.chown(path, _user.pw_uid, grp.getgrnam(group).gr_gid)
-    os.chmod(path, permissions)
-
-    if maxdepth == 0:
-        log("Max recursion depth reached - skipping further recursion",
-            level=DEBUG)
-        return
-    elif maxdepth > 0:
-        maxdepth -= 1
-
-    if os.path.isdir(path):
-        contents = glob.glob("%s/*" % (path))
-        for c in contents:
-            ensure_permissions(c, user=user, group=group,
-                               permissions=permissions, maxdepth=maxdepth)

=== removed directory 'hooks/charmhelpers/contrib/mellanox'
=== removed file 'hooks/charmhelpers/contrib/mellanox/__init__.py'
=== removed file 'hooks/charmhelpers/contrib/mellanox/infiniband.py'
--- hooks/charmhelpers/contrib/mellanox/infiniband.py	2016-01-30 22:41:50 +0000
+++ hooks/charmhelpers/contrib/mellanox/infiniband.py	1970-01-01 00:00:00 +0000
@@ -1,151 +0,0 @@
-#!/usr/bin/env python
-# -*- coding: utf-8 -*-
-
-# Copyright 2014-2015 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-
-__author__ = "Jorge Niedbalski <[e-mail]>"
-
-from charmhelpers.fetch import (
-    apt_install,
-    apt_update,
-)
-
-from charmhelpers.core.hookenv import (
-    log,
-    INFO,
-)
-
-try:
-    from netifaces import interfaces as network_interfaces
-except ImportError:
-    apt_install('python-netifaces')
-    from netifaces import interfaces as network_interfaces
-
-import os
-import re
-import subprocess
-
-from charmhelpers.core.kernel import modprobe
-
-REQUIRED_MODULES = (
-    "mlx4_ib",
-    "mlx4_en",
-    "mlx4_core",
-    "ib_ipath",
-    "ib_mthca",
-    "ib_srpt",
-    "ib_srp",
-    "ib_ucm",
-    "ib_isert",
-    "ib_iser",
-    "ib_ipoib",
-    "ib_cm",
-    "ib_uverbs"
-    "ib_umad",
-    "ib_sa",
-    "ib_mad",
-    "ib_core",
-    "ib_addr",
-    "rdma_ucm",
-)
-
-REQUIRED_PACKAGES = (
-    "ibutils",
-    "infiniband-diags",
-    "ibverbs-utils",
-)
-
-IPOIB_DRIVERS = (
-    "ib_ipoib",
-)
-
-ABI_VERSION_FILE = "/sys/class/infiniband_mad/abi_version"
-
-
-class DeviceInfo(object):
-    pass
-
-
-def install_packages():
-    apt_update()
-    apt_install(REQUIRED_PACKAGES, fatal=True)
-
-
-def load_modules():
-    for module in REQUIRED_MODULES:
-        modprobe(module, persist=True)
-
-
-def is_enabled():
-    """Check if infiniband is loaded on the system"""
-    return os.path.exists(ABI_VERSION_FILE)
-
-
-def stat():
-    """Return full output of ibstat"""
-    return subprocess.check_output(["ibstat"])
-
-
-def devices():
-    """Returns a list of IB enabled devices"""
-    return subprocess.check_output(['ibstat', '-l']).splitlines()
-
-
-def device_info(device):
-    """Returns a DeviceInfo object with the current device settings"""
-
-    status = subprocess.check_output([
-        'ibstat', device, '-s']).splitlines()
-
-    regexes = {
-        "CA type: (.*)": "device_type",
-        "Number of ports: (.*)": "num_ports",
-        "Firmware version: (.*)": "fw_ver",
-        "Hardware version: (.*)": "hw_ver",
-        "Node GUID: (.*)": "node_guid",
-        "System image GUID: (.*)": "sys_guid",
-    }
-
-    device = DeviceInfo()
-
-    for line in status:
-        for expression, key in regexes.items():
-            matches = re.search(expression, line)
-            if matches:
-                setattr(device, key, matches.group(1))
-
-    return device
-
-
-def ipoib_interfaces():
-    """Return a list of IPOIB capable ethernet interfaces"""
-    interfaces = []
-
-    for interface in network_interfaces():
-        try:
-            driver = re.search('^driver: (.+)$', subprocess.check_output([
-                'ethtool', '-i',
-                interface]), re.M).group(1)
-
-            if driver in IPOIB_DRIVERS:
-                interfaces.append(interface)
-        except:
-            log("Skipping interface %s" % interface, level=INFO)
-            continue
-
-    return interfaces

=== removed directory 'hooks/charmhelpers/contrib/peerstorage'
=== removed file 'hooks/charmhelpers/contrib/peerstorage/__init__.py'
--- hooks/charmhelpers/contrib/peerstorage/__init__.py	2016-01-30 22:40:26 +0000
+++ hooks/charmhelpers/contrib/peerstorage/__init__.py	1970-01-01 00:00:00 +0000
@@ -1,269 +0,0 @@
-# Copyright 2014-2015 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-import json
-import six
-
-from charmhelpers.core.hookenv import relation_id as current_relation_id
-from charmhelpers.core.hookenv import (
-    is_relation_made,
-    relation_ids,
-    relation_get as _relation_get,
-    local_unit,
-    relation_set as _relation_set,
-    leader_get as _leader_get,
-    leader_set,
-    is_leader,
-)
-
-
-"""
-This helper provides functions to support use of a peer relation
-for basic key/value storage, with the added benefit that all storage
-can be replicated across peer units.
-
-Requirement to use:
-
-To use this, the "peer_echo()" method has to be called form the peer
-relation's relation-changed hook:
-
[e-mail]("cluster-relation-changed") # Adapt the to your peer relation name
-def cluster_relation_changed():
-    peer_echo()
-
-Once this is done, you can use peer storage from anywhere:
-
[e-mail]("some-hook")
-def some_hook():
-    # You can store and retrieve key/values this way:
-    if is_relation_made("cluster"):  # from charmhelpers.core.hookenv
-        # There are peers available so we can work with peer storage
-        peer_store("mykey", "myvalue")
-        value = peer_retrieve("mykey")
-        print value
-    else:
-        print "No peers joind the relation, cannot share key/values :("
-"""
-
-
-def leader_get(attribute=None, rid=None):
-    """Wrapper to ensure that settings are migrated from the peer relation.
-
-    This is to support upgrading an environment that does not support
-    Juju leadership election to one that does.
-
-    If a setting is not extant in the leader-get but is on the relation-get
-    peer rel, it is migrated and marked as such so that it is not re-migrated.
-    """
-    migration_key = '__leader_get_migrated_settings__'
-    if not is_leader():
-        return _leader_get(attribute=attribute)
-
-    settings_migrated = False
-    leader_settings = _leader_get(attribute=attribute)
-    previously_migrated = _leader_get(attribute=migration_key)
-
-    if previously_migrated:
-        migrated = set(json.loads(previously_migrated))
-    else:
-        migrated = set([])
-
-    try:
-        if migration_key in leader_settings:
-            del leader_settings[migration_key]
-    except TypeError:
-        pass
-
-    if attribute:
-        if attribute in migrated:
-            return leader_settings
-
-        # If attribute not present in leader db, check if this unit has set
-        # the attribute in the peer relation
-        if not leader_settings:
-            peer_setting = _relation_get(attribute=attribute, unit=local_unit(),
-                                         rid=rid)
-            if peer_setting:
-                leader_set(settings={attribute: peer_setting})
-                leader_settings = peer_setting
-
-        if leader_settings:
-            settings_migrated = True
-            migrated.add(attribute)
-    else:
-        r_settings = _relation_get(unit=local_unit(), rid=rid)
-        if r_settings:
-            for key in set(r_settings.keys()).difference(migrated):
-                # Leader setting wins
-                if not leader_settings.get(key):
-                    leader_settings[key] = r_settings[key]
-
-                settings_migrated = True
-                migrated.add(key)
-
-            if settings_migrated:
-                leader_set(**leader_settings)
-
-    if migrated and settings_migrated:
-        migrated = json.dumps(list(migrated))
-        leader_set(settings={migration_key: migrated})
-
-    return leader_settings
-
-
-def relation_set(relation_id=None, relation_settings=None, **kwargs):
-    """Attempt to use leader-set if supported in the current version of Juju,
-    otherwise falls back on relation-set.
-
-    Note that we only attempt to use leader-set if the provided relation_id is
-    a peer relation id or no relation id is provided (in which case we assume
-    we are within the peer relation context).
-    """
-    try:
-        if relation_id in relation_ids('cluster'):
-            return leader_set(settings=relation_settings, **kwargs)
-        else:
-            raise NotImplementedError
-    except NotImplementedError:
-        return _relation_set(relation_id=relation_id,
-                             relation_settings=relation_settings, **kwargs)
-
-
-def relation_get(attribute=None, unit=None, rid=None):
-    """Attempt to use leader-get if supported in the current version of Juju,
-    otherwise falls back on relation-get.
-
-    Note that we only attempt to use leader-get if the provided rid is a peer
-    relation id or no relation id is provided (in which case we assume we are
-    within the peer relation context).
-    """
-    try:
-        if rid in relation_ids('cluster'):
-            return leader_get(attribute, rid)
-        else:
-            raise NotImplementedError
-    except NotImplementedError:
-        return _relation_get(attribute=attribute, rid=rid, unit=unit)
-
-
-def peer_retrieve(key, relation_name='cluster'):
-    """Retrieve a named key from peer relation `relation_name`."""
-    cluster_rels = relation_ids(relation_name)
-    if len(cluster_rels) > 0:
-        cluster_rid = cluster_rels[0]
-        return relation_get(attribute=key, rid=cluster_rid,
-                            unit=local_unit())
-    else:
-        raise ValueError('Unable to detect'
-                         'peer relation {}'.format(relation_name))
-
-
-def peer_retrieve_by_prefix(prefix, relation_name='cluster', delimiter='_',
-                            inc_list=None, exc_list=None):
-    """ Retrieve k/v pairs given a prefix and filter using {inc,exc}_list """
-    inc_list = inc_list if inc_list else []
-    exc_list = exc_list if exc_list else []
-    peerdb_settings = peer_retrieve('-', relation_name=relation_name)
-    matched = {}
-    if peerdb_settings is None:
-        return matched
-    for k, v in peerdb_settings.items():
-        full_prefix = prefix + delimiter
-        if k.startswith(full_prefix):
-            new_key = k.replace(full_prefix, '')
-            if new_key in exc_list:
-                continue
-            if new_key in inc_list or len(inc_list) == 0:
-                matched[new_key] = v
-    return matched
-
-
-def peer_store(key, value, relation_name='cluster'):
-    """Store the key/value pair on the named peer relation `relation_name`."""
-    cluster_rels = relation_ids(relation_name)
-    if len(cluster_rels) > 0:
-        cluster_rid = cluster_rels[0]
-        relation_set(relation_id=cluster_rid,
-                     relation_settings={key: value})
-    else:
-        raise ValueError('Unable to detect '
-                         'peer relation {}'.format(relation_name))
-
-
-def peer_echo(includes=None, force=False):
-    """Echo filtered attributes back onto the same relation for storage.
-
-    This is a requirement to use the peerstorage module - it needs to be called
-    from the peer relation's changed hook.
-
-    If Juju leader support exists this will be a noop unless force is True.
-    """
-    try:
-        is_leader()
-    except NotImplementedError:
-        pass
-    else:
-        if not force:
-            return  # NOOP if leader-election is supported
-
-    # Use original non-leader calls
-    relation_get = _relation_get
-    relation_set = _relation_set
-
-    rdata = relation_get()
-    echo_data = {}
-    if includes is None:
-        echo_data = rdata.copy()
-        for ex in ['private-address', 'public-address']:
-            if ex in echo_data:
-                echo_data.pop(ex)
-    else:
-        for attribute, value in six.iteritems(rdata):
-            for include in includes:
-                if include in attribute:
-                    echo_data[attribute] = value
-    if len(echo_data) > 0:
-        relation_set(relation_settings=echo_data)
-
-
-def peer_store_and_set(relation_id=None, peer_relation_name='cluster',
-                       peer_store_fatal=False, relation_settings=None,
-                       delimiter='_', **kwargs):
-    """Store passed-in arguments both in argument relation and in peer storage.
-
-    It functions like doing relation_set() and peer_store() at the same time,
-    with the same data.
-
-    @param relation_id: the id of the relation to store the data on. Defaults
-                        to the current relation.
-    @param peer_store_fatal: Set to True, the function will raise an exception
-                             should the peer sotrage not be avialable."""
-
-    relation_settings = relation_settings if relation_settings else {}
-    relation_set(relation_id=relation_id,
-                 relation_settings=relation_settings,
-                 **kwargs)
-    if is_relation_made(peer_relation_name):
-        for key, value in six.iteritems(dict(list(kwargs.items()) +
-                                             list(relation_settings.items()))):
-            key_prefix = relation_id or current_relation_id()
-            peer_store(key_prefix + delimiter + key,
-                       value,
-                       relation_name=peer_relation_name)
-    else:
-        if peer_store_fatal:
-            raise ValueError('Unable to detect '
-                             'peer relation {}'.format(peer_relation_name))

=== removed directory 'hooks/charmhelpers/contrib/saltstack'
=== removed file 'hooks/charmhelpers/contrib/saltstack/__init__.py'
--- hooks/charmhelpers/contrib/saltstack/__init__.py	2015-07-29 18:07:31 +0000
+++ hooks/charmhelpers/contrib/saltstack/__init__.py	1970-01-01 00:00:00 +0000
@@ -1,118 +0,0 @@
-# Copyright 2014-2015 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-"""Charm Helpers saltstack - declare the state of your machines.
-
-This helper enables you to declare your machine state, rather than
-program it procedurally (and have to test each change to your procedures).
-Your install hook can be as simple as::
-
-    {{{
-    from charmhelpers.contrib.saltstack import (
-        install_salt_support,
-        update_machine_state,