Merge lp:~mbruzek/charms/trusty/fail2ban/trunk into lp:~lazypower/charms/trusty/fail2ban/trunk

[Matt Bruzek]

I removed some things from the Makefile, wrote proper tests and updated the README.

[Charles Butler (lazypower)]

Thanks for the cleanup Matt! These additions look good to me.

review: Approve

[lp:~mbruzek/charms/trusty/fail2ban/trunk]

4. By Charles Butler on 2014-11-21

   [r=lazypower]Matt Bruzek 2014-11-21 Updates to Makefile, README.md, config, and metadata, and now with better ...
    Matt Bruzek 2014-11-20 Removing redundant sync file and tests.

=== modified file 'Makefile'
--- Makefile	2014-10-22 04:18:26 +0000
+++ Makefile	2014-11-21 16:00:44 +0000
@@ -1,37 +1,24 @@
 PYTHON := /usr/bin/env python
 
-build: clean test
-
 clean:
 	find . -name \*.pyc -delete
 	find . -name '*.bak' -delete
-	rm -f .coverage
-
-clean_all: clean
-	rm -rf .venv
 
 lint:
 	@flake8 --exclude hooks/charmhelpers --ignore=E125 hooks
-	@charm proof
-
-test: .venv
-	@echo Starting unit tests...
-	@PYTHONPATH=./hooks $(PYTHON) .venv/bin/nosetests --nologcapture unit_tests
-
-.venv:
-	sudo apt-get install python-virtualenv
-	virtualenv .venv
-	.venv/bin/pip install nose mock pyyaml
+	charm proof
 
 deploy:
-	@echo Deploying local elasticsearch charm
-	@juju deploy --num-units=2 --repository=../.. local:trusty/elasticsearch
-	@ juju deploy --repository=../.. local:trusty/logstash
+	@echo Deploying ubuntu charm
+	juju deploy cs:trusty/ubuntu
+	@echo Deploying local fail2ban charm
+	juju deploy --repository=../.. local:trusty/fail2ban
+	juju add-relation ubuntu:juju-info fail2ban:juju-info
 
 # The following targets are used for charm maintenance only.
 bin/charm_helpers_sync.py:
 	@mkdir -p bin
-	@bzr cat lp:charm-helpers/tools/charm_helpers_sync/charm_helpers_sync.py \
+	bzr cat lp:charm-helpers/tools/charm_helpers_sync/charm_helpers_sync.py \
 		> bin/charm_helpers_sync.py
 
 sync: bin/charm_helpers_sync.py

=== modified file 'README.md'
--- README.md	2014-10-22 04:18:26 +0000
+++ README.md	2014-11-21 16:00:44 +0000
@@ -1,40 +1,47 @@
 # Fail2Ban
 
-Deploys fail2ban monitoring and DDOS prevention service, with exposed
-configuration for SSH DDOS
+Deploys fail2ban monitoring and denial-of-service (DoS) prevention service,
+with exposed configuration to help prevent SSH DoS attacks.
+
+The fail2ban service scans log files and bans IPs that have too many password
+failures.  The number of failures, and ban time are configurable.
 
 # Deployment
 
-Step by step instructions on deploying the charm:
+The fail2ban charm is a subordinate charm a container to deploy.
+The fail2ban charm uses the implicit juju-info relationship so it can be
+related to any container charm.  Here are the steps to deploy the charm:
 
+    juju deploy ubuntu
     juju deploy fail2ban
-    juju add-relation fail2ban service
+    juju add-relation fail2ban:juju-info ubuntu:juju-info
 
-This will install, and configure fail2ban to monitor SSH by default with a 1
-hour delay on incorrect password attempts, after 3 failed attempts.
+These steps will install, and configure fail2ban to monitor SSH by default
+with a 1 hour delay on incorrect password attempts, after 3 failed attempts.
 
 ## Known Limitations and Issues
 
-Does not configure any of the other services fail2ban can monitor, such as
-http, ftp, etc.
+This charm does not configure any of the other services fail2ban can monitor,
+such as http, ftp, etc.  If you wish to configure these services you can find
+the configuration file at `/etc/fail2ban/jail.local`.
 
 # Configuration
 
-- **maxretry**: "number of attempts before ban"
-- **ignoreip**: "Additional IP's (space separated) to add to the ignore ruleset.
-Supports IP and CIDR"
-- **bantime**: "Ban time in seconds (defaults to 1 hour)"
-- **destemail**: "Address to send mail to on abuse"
+- **maxretry**: number of attempts before banning the IP adddress.
+- **ignoreip**: Additional IP's (space separated) to add to the ignore ruleset.
+Supports IP and CIDR.
+- **bantime**: Ban time in seconds (defaults to 1 hour).
+- **destemail**: Email address to send mail to on abuse.
 
 Example configuration
 
-    juju set fail2ban bantime=300 maxretry=5 ignoreip=192.262.3.0/24
+    juju set fail2ban bantime=3000 maxretry=5 ignoreip=192.262.3.0/24
 
 ## Maintainer
 
 - [Charles Butler <[e-mail]>](mailto:[e-mail])
 
-## Upstream Project Name
+## Fail2ban upstream project
 
-- [Upstream Website](http://www.fail2ban.org/wiki/index.php/Main_Page)
-- [Upstream bug tracker](https://github.com/fail2ban/fail2ban/issues)
+- [Fail2ban Website](http://www.fail2ban.org/wiki/index.php/Main_Page)
+- [Fail2ban issues](https://github.com/fail2ban/fail2ban/issues)

=== modified file 'config.yaml'
--- config.yaml	2014-10-22 04:18:26 +0000
+++ config.yaml	2014-11-21 16:00:44 +0000
@@ -2,7 +2,7 @@
   maxretry:
     type: int
     default: 3
-    description: "number of attempts before ban"
+    description: "number of attempts before banning the IP address"
   ignoreip:
     type: string
     default:

=== modified file 'lib/charmhelpers/fetch/__init__.py'
--- lib/charmhelpers/fetch/__init__.py	2014-10-22 04:18:26 +0000
+++ lib/charmhelpers/fetch/__init__.py	2014-11-21 16:00:44 +0000
@@ -256,7 +256,7 @@
     elif source == 'distro':
         pass
     else:
-        raise SourceConfigError("Unknown source: {!r}".format(source))
+        log("Unknown source: {!r}".format(source))
 
     if key:
         if '-----BEGIN PGP PUBLIC KEY BLOCK-----' in key:

=== modified file 'metadata.yaml'
--- metadata.yaml	2014-10-22 04:18:26 +0000
+++ metadata.yaml	2014-11-21 16:00:44 +0000
@@ -1,6 +1,6 @@
 name: fail2ban
-summary: ban hosts that cause multiple authentication errors
-maintainer: charles <charles@desktop>
+summary: Bans IP addresses that have too many authentication failures.
+maintainer: Charles Butler <[e-mail]>
 description: |
   Fail2ban monitors log files (e.g. /var/log/auth.log,
   /var/log/apache/access.log) and temporarily or persistently bans

=== removed directory 'scripts'
=== removed file 'scripts/charm_helpers_sync.py'
--- scripts/charm_helpers_sync.py	2014-10-22 04:18:26 +0000
+++ scripts/charm_helpers_sync.py	1970-01-01 00:00:00 +0000
@@ -1,223 +0,0 @@
-#!/usr/bin/env python
-# Copyright 2013 Canonical Ltd.
-
-# Authors:
-#   Adam Gandelman <[e-mail]>
-
-import logging
-import optparse
-import os
-import subprocess
-import shutil
-import sys
-import tempfile
-import yaml
-
-from fnmatch import fnmatch
-
-CHARM_HELPERS_BRANCH = 'lp:charm-helpers'
-
-
-def parse_config(conf_file):
-    if not os.path.isfile(conf_file):
-        logging.error('Invalid config file: %s.' % conf_file)
-        return False
-    return yaml.load(open(conf_file).read())
-
-
-def clone_helpers(work_dir, branch):
-    dest = os.path.join(work_dir, 'charm-helpers')
-    logging.info('Checking out %s to %s.' % (branch, dest))
-    cmd = ['bzr', 'branch', branch, dest]
-    subprocess.check_call(cmd)
-    return dest
-
-
-def _module_path(module):
-    return os.path.join(*module.split('.'))
-
-
-def _src_path(src, module):
-    return os.path.join(src, 'charmhelpers', _module_path(module))
-
-
-def _dest_path(dest, module):
-    return os.path.join(dest, _module_path(module))
-
-
-def _is_pyfile(path):
-    return os.path.isfile(path + '.py')
-
-
-def ensure_init(path):
-    '''
-    ensure directories leading up to path are importable, omitting
-    parent directory, eg path='/hooks/helpers/foo'/:
-        hooks/
-        hooks/helpers/__init__.py
-        hooks/helpers/foo/__init__.py
-    '''
-    for d, dirs, files in os.walk(os.path.join(*path.split('/')[:2])):
-        _i = os.path.join(d, '__init__.py')
-        if not os.path.exists(_i):
-            logging.info('Adding missing __init__.py: %s' % _i)
-            open(_i, 'wb').close()
-
-
-def sync_pyfile(src, dest):
-    src = src + '.py'
-    src_dir = os.path.dirname(src)
-    logging.info('Syncing pyfile: %s -> %s.' % (src, dest))
-    if not os.path.exists(dest):
-        os.makedirs(dest)
-    shutil.copy(src, dest)
-    if os.path.isfile(os.path.join(src_dir, '__init__.py')):
-        shutil.copy(os.path.join(src_dir, '__init__.py'),
-                    dest)
-    ensure_init(dest)
-
-
-def get_filter(opts=None):
-    opts = opts or []
-    if 'inc=*' in opts:
-        # do not filter any files, include everything
-        return None
-
-    def _filter(dir, ls):
-        incs = [opt.split('=').pop() for opt in opts if 'inc=' in opt]
-        _filter = []
-        for f in ls:
-            _f = os.path.join(dir, f)
-
-            if not os.path.isdir(_f) and not _f.endswith('.py') and incs:
-                if True not in [fnmatch(_f, inc) for inc in incs]:
-                    logging.debug('Not syncing %s, does not match include '
-                                  'filters (%s)' % (_f, incs))
-                    _filter.append(f)
-                else:
-                    logging.debug('Including file, which matches include '
-                                  'filters (%s): %s' % (incs, _f))
-            elif (os.path.isfile(_f) and not _f.endswith('.py')):
-                logging.debug('Not syncing file: %s' % f)
-                _filter.append(f)
-            elif (os.path.isdir(_f) and not
-                  os.path.isfile(os.path.join(_f, '__init__.py'))):
-                logging.debug('Not syncing directory: %s' % f)
-                _filter.append(f)
-        return _filter
-    return _filter
-
-
-def sync_directory(src, dest, opts=None):
-    if os.path.exists(dest):
-        logging.debug('Removing existing directory: %s' % dest)
-        shutil.rmtree(dest)
-    logging.info('Syncing directory: %s -> %s.' % (src, dest))
-
-    shutil.copytree(src, dest, ignore=get_filter(opts))
-    ensure_init(dest)
-
-
-def sync(src, dest, module, opts=None):
-    if os.path.isdir(_src_path(src, module)):
-        sync_directory(_src_path(src, module), _dest_path(dest, module), opts)
-    elif _is_pyfile(_src_path(src, module)):
-        sync_pyfile(_src_path(src, module),
-                    os.path.dirname(_dest_path(dest, module)))
-    else:
-        logging.warn('Could not sync: %s. Neither a pyfile or directory, '
-                     'does it even exist?' % module)
-
-
-def parse_sync_options(options):
-    if not options:
-        return []
-    return options.split(',')
-
-
-def extract_options(inc, global_options=None):
-    global_options = global_options or []
-    if global_options and isinstance(global_options, basestring):
-        global_options = [global_options]
-    if '|' not in inc:
-        return (inc, global_options)
-    inc, opts = inc.split('|')
-    return (inc, parse_sync_options(opts) + global_options)
-
-
-def sync_helpers(include, src, dest, options=None):
-    if not os.path.isdir(dest):
-        os.mkdir(dest)
-
-    global_options = parse_sync_options(options)
-
-    for inc in include:
-        if isinstance(inc, str):
-            inc, opts = extract_options(inc, global_options)
-            sync(src, dest, inc, opts)
-        elif isinstance(inc, dict):
-            # could also do nested dicts here.
-            for k, v in inc.iteritems():
-                if isinstance(v, list):
-                    for m in v:
-                        inc, opts = extract_options(m, global_options)
-                        sync(src, dest, '%s.%s' % (k, inc), opts)
-
-if __name__ == '__main__':
-    parser = optparse.OptionParser()
-    parser.add_option('-c', '--config', action='store', dest='config',
-                      default=None, help='helper config file')
-    parser.add_option('-D', '--debug', action='store_true', dest='debug',
-                      default=False, help='debug')
-    parser.add_option('-b', '--branch', action='store', dest='branch',
-                      help='charm-helpers bzr branch (overrides config)')
-    parser.add_option('-d', '--destination', action='store', dest='dest_dir',
-                      help='sync destination dir (overrides config)')
-    (opts, args) = parser.parse_args()
-
-    if opts.debug:
-        logging.basicConfig(level=logging.DEBUG)
-    else:
-        logging.basicConfig(level=logging.INFO)
-
-    if opts.config:
-        logging.info('Loading charm helper config from %s.' % opts.config)
-        config = parse_config(opts.config)
-        if not config:
-            logging.error('Could not parse config from %s.' % opts.config)
-            sys.exit(1)
-    else:
-        config = {}
-
-    if 'branch' not in config:
-        config['branch'] = CHARM_HELPERS_BRANCH
-    if opts.branch:
-        config['branch'] = opts.branch
-    if opts.dest_dir:
-        config['destination'] = opts.dest_dir
-
-    if 'destination' not in config:
-        logging.error('No destination dir. specified as option or config.')
-        sys.exit(1)
-
-    if 'include' not in config:
-        if not args:
-            logging.error('No modules to sync specified as option or config.')
-            sys.exit(1)
-        config['include'] = []
-        [config['include'].append(a) for a in args]
-
-    sync_options = None
-    if 'options' in config:
-        sync_options = config['options']
-    tmpd = tempfile.mkdtemp()
-    try:
-        checkout = clone_helpers(tmpd, config['branch'])
-        sync_helpers(config['include'], checkout, config['destination'],
-                     options=sync_options)
-    except Exception, e:
-        logging.error("Could not sync: %s" % e)
-        raise e
-    finally:
-        logging.debug('Cleaning up %s' % tmpd)
-        shutil.rmtree(tmpd)

=== modified file 'tests/00-setup'
--- tests/00-setup	2014-10-22 04:18:26 +0000
+++ tests/00-setup	2014-11-21 16:00:44 +0000
@@ -1,5 +1,5 @@
 #!/bin/bash
 
 sudo add-apt-repository ppa:juju/stable -y
-sudo apt-get update
-sudo apt-get install amulet python-requests -y
+sudo apt-get update -qq
+sudo apt-get install amulet python3-requests -y

=== modified file 'tests/10-deploy'
--- tests/10-deploy	2014-10-22 04:18:26 +0000
+++ tests/10-deploy	2014-11-21 16:00:44 +0000
@@ -1,35 +1,76 @@
-#!/usr/bin/python3
+#!/usr/bin/env python3
+
+# The amulet code to test the fail2ban charm.
 
 import amulet
-import requests
-
-d = amulet.Deployment()
-
-d.add('fail2ban')
-d.expose('fail2ban')
-
-try:
-    d.setup(timeout=900)
-    d.sentry.wait()
-except amulet.helpers.TimeoutError:
-    amulet.raise_status(amulet.SKIP, msg="Environment wasn't stood up in time")
-except:
-    raise
-
-unit = d.sentry.unit['fail2ban/0']
-
-# test we can access over http
-page = requests.get('http://{}'.format(unit.info['public-address']))
-page.raise_for_status()
-
-
-# Now you can use d.sentry.unit[UNIT] to address each of the units and perform
-# more in-depth steps. There are three test statuses: amulet.PASS, amulet.FAIL,
-# and amulet.SKIP - these can be triggered with amulet.raise_status(). Each
-# d.sentry.unit[] has the following methods:
-# - .info - An array of the information of that unit from Juju
-# - .file(PATH) - Get the details of a file on that unit
-# - .file_contents(PATH) - Get plain text output of PATH file from that unit
-# - .directory(PATH) - Get details of directory
-# - .directory_contents(PATH) - List files and folders in PATH on that unit
-# - .relation(relation, service:rel) - Get relation data from return service
+import unittest
+
+
+# The number of seconds to wait for Juju to set up the environment.
+seconds = 900
+
+# Configure fail2ban
+configuration = {
+    'maxretry': '6',
+    'bantime': '3660',
+    'destemail': '[e-mail]'
+}
+
+
+class TestDeployment(unittest.TestCase):
+    ''' A subclass of TestCase to perform a deployment and run the tests. '''
+    @classmethod
+    def setUpClass(self):
+        ''' The setup class gets run one time before tests. '''
+        self.deployment = amulet.Deployment(series='trusty')
+        self.deployment.add('ubuntu')
+        self.deployment.add('fail2ban')
+        self.deployment.configure('fail2ban', configuration)
+        # Relate the two charms.
+        self.deployment.relate('ubuntu:juju-info', 'fail2ban:juju-info')
+
+        try:
+            self.deployment.setup(timeout=seconds)
+            self.deployment.sentry.wait()
+        except amulet.helpers.TimeoutError:
+            message = 'The environment did not set up in %d seconds!' % seconds
+            amulet.raise_status(amulet.SKIP, msg=message)
+        except:
+            raise
+        # Get a reference to the Ubuntu unit.
+        self.unit = self.deployment.sentry.unit['ubuntu/0']
+
+    def test_status(self):
+        ''' Test the status of the fail2ban process. '''
+        # Verify the fail2ban service is installed and runing on ubuntu.
+        command = 'sudo service fail2ban status'
+        print(command)
+        # Run the command to see if fail2ban is running.
+        output, code = self.unit.run(command)
+        print(output)
+        if code != 0:
+            message = 'The fail2ban process is not running!'
+            amulet.raise_status(amulet.FAIL, msg=message)
+
+    def test_config(self):
+        ''' Test the configuration was set on the local configuration file. '''
+        config_string = self.unit.file_contents('/etc/fail2ban/jail.local')
+        print(config_string)
+        if configuration['destemail'] not in config_string:
+            message = 'The destemail value was not in the configuration file!'
+            amulet.raise_status(amulet.FAIL, msg=message)
+
+    def test_iptables(self):
+        ''' Test to set if 'fail2ban' is in the iptables firewall rules. '''
+        command = 'sudo iptables -S'
+        print(command)
+        # Run the command to see if fail2ban is in the iptables rules.
+        output, code = self.unit.run(command)
+        print(output)
+        if 'fail2ban' not in output:
+            message = 'There were no firewall rules including fail2ban!'
+            amulet.raise_status(amulet.FAIL, msg=message)
+
+
+if __name__ == '__main__':
+    unittest.main()