Merge ~jfguedez/charm-nrpe:cis-hardening-check into charm-nrpe:master

[Jose Guedez]

Add cis-audit check and cron job

This change allows the monitoring of cis-hardened systems by running a cis-audit via cron job and verifying the results of the audit via check_cis_audit.py.

Depends on usg-cisbenchmark to be installed (Ubuntu Advantage), will alert if it is not installed.

supersedes https://code.launchpad.net/~stephanpampel/charm-nrpe/+git/charm-nrpe/+merge/414231

[🤖 Canonical IS Merge Bot (canonical-is-mergebot)]

This merge proposal is being monitored by mergebot. Change the status to Approved to merge.

[🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack)]

A CI job is currently in progress. A follow up comment will be added when it completes.

[🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack)]

PASSED: Continuous integration, rev:1b3c96df945cbc3cf7bd857a6c5ebbc4874638e3
https://jenkins.canonical.com/bootstack/job/lp-charm-nrpe-ci/84/
Executed test runs:
    SUCCESS: https://jenkins.canonical.com/bootstack/job/lp-charm-test-functest/526/
    None: https://jenkins.canonical.com/bootstack/job/lp-update-mp/988/

Click here to trigger a rebuild:
https://jenkins.canonical.com/bootstack/job/lp-charm-nrpe-ci/84//rebuild

review: Approve (continuous-integration)

[James Troup (elmo)]

Thanks for picking this work up - see comments inline.

[Jose Guedez (jfguedez)]

Thanks for the review. I have addressed the issues and pushed the changes. Replies inline.

[🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack)]

A CI job is currently in progress. A follow up comment will be added when it completes.

[🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack)]

PASSED: Continuous integration, rev:a9752abce7122f4d70b217ce7c0bc324b7cb93d9
https://jenkins.canonical.com/bootstack/job/lp-charm-nrpe-ci/85/
Executed test runs:
    SUCCESS: https://jenkins.canonical.com/bootstack/job/lp-charm-test-functest/528/
    None: https://jenkins.canonical.com/bootstack/job/lp-update-mp/990/

Click here to trigger a rebuild:
https://jenkins.canonical.com/bootstack/job/lp-charm-nrpe-ci/85//rebuild

review: Approve (continuous-integration)

[Jose Guedez (jfguedez)]

A different MP was merged overnight, causing a conflict in tox.ini. Rebased to master and pushed again

[🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack)]

A CI job is currently in progress. A follow up comment will be added when it completes.

[🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack)]

PASSED: Continuous integration, rev:628764521cece73b165203c99ce0f5d7a30fb439
https://jenkins.canonical.com/bootstack/job/lp-charm-nrpe-ci/86/
Executed test runs:
    SUCCESS: https://jenkins.canonical.com/bootstack/job/lp-charm-test-functest/529/
    None: https://jenkins.canonical.com/bootstack/job/lp-update-mp/991/

Click here to trigger a rebuild:
https://jenkins.canonical.com/bootstack/job/lp-charm-nrpe-ci/86//rebuild

review: Approve (continuous-integration)

[🤖 Canonical IS Merge Bot (canonical-is-mergebot)]

Change successfully merged at revision 270a31d7643f914c47bb32007368615a715567b3

diff --git a/.coveragerc b/.coveragerc
index 3d6d8f3..7ead038 100644
--- a/.coveragerc
+++ b/.coveragerc
@@ -4,3 +4,4 @@ exclude_lines =
     if __name__ == .__main__.:
 include=
     hooks/nrpe_*
+    files/plugins/*
diff --git a/config.yaml b/config.yaml
index a7cb687..4494993 100644
--- a/config.yaml
+++ b/config.yaml
@@ -31,7 +31,7 @@ options:
         Determines whether a server is identified by its unit name or
         host name.  If you're in a virtual environment, "unit" is
         probably best.  If you're using MaaS, you may prefer "host".
-        Use "auto" to have nrpe automatically distinguish between 
+        Use "auto" to have nrpe automatically distinguish between
         metal and non-metal hosts.
   dont_blame_nrpe:
     default: False
@@ -229,6 +229,30 @@ options:
       'ondemand', 'performance', 'powersave'. Unset value means the check will be disabled.
       There is a relation key called requested_cpu_governor='string', but the charm config value
       will take precedence over the relation data.
+  cis_audit_enabled:
+    default: False
+    type: boolean
+    description: |
+      Enabled cis-audit cron job which periodically runs cis-audit and enables check_cis_audit.py
+      which verifies that cis-audit was run recently with an acceptable score (see cis_audit_score)
+      and optionally a specific profile (cis_audit_profile).
+  cis_audit_score:
+    default: "-w 85 -c 80"
+    type: string
+    description: |
+      CIS audit score threshold for alerts. Per default it only checks if hardening was run and if
+      usg-cisbenchmark is installed and reports the score. To enable alerts base on the score this
+      config option can be set.
+      example: -w 85 -c 80
+  cis_audit_profile:
+    default: ""
+    type: string
+    description: |
+      Verify that a specific cis audit profile was used for the audit. If not specified the
+      profile will be extracted from '/var/log/cloud-init-output.log', fallback is 'level1_server'.
+      Options are: '' (disable profile check), 'level1_server', 'level2_server',
+                   'level1_workstation' or 'level2_workstation'
+      See also https://ubuntu.com/security/certifications/docs/cis-audit
   reboot:
     default: True
     type: boolean
diff --git a/files/nagios_plugin.py b/files/nagios_plugin.py
deleted file mode 100644
index d5a1ff2..0000000
--- a/files/nagios_plugin.py
+++ /dev/null
@@ -1,84 +0,0 @@
-#!/usr/bin/env python
-"""Nagios plugin for python2.7."""
-# Copyright (C) 2005, 2006, 2007, 2012  James Troup <[e-mail]>
-
-import os
-import stat
-import time
-import traceback
-import sys
-
-
-################################################################################
-
-
-class CriticalError(Exception):
-    """This indicates a critical error."""
-
-    pass
-
-
-class WarnError(Exception):
-    """This indicates a warning condition."""
-
-    pass
-
-
-class UnknownError(Exception):
-    """This indicates a unknown error was encountered."""
-
-    pass
-
-
-def try_check(function, *args, **kwargs):
-    """Perform a check with error/warn/unknown handling."""
-    try:
-        function(*args, **kwargs)
-    except UnknownError, msg:  # noqa: E999
-        print msg
-        sys.exit(3)
-    except CriticalError, msg:  # noqa: E999
-        print msg
-        sys.exit(2)
-    except WarnError, msg:  # noqa: E999
-        print msg
-        sys.exit(1)
-    except:  # noqa: E722
-        print "%s raised unknown exception '%s'" % (function, sys.exc_info()[0])
-        print "=" * 60
-        traceback.print_exc(file=sys.stdout)
-        print "=" * 60
-        sys.exit(3)
-
-
-################################################################################
-
-
-def check_file_freshness(filename, newer_than=600):
-    """Check a file.
-
-    It check that file exists, is readable and is newer than <n> seconds (where
-    <n> defaults to 600).
-    """
-    # First check the file exists and is readable
-    if not os.path.exists(filename):
-        raise CriticalError("%s: does not exist." % (filename))
-    if os.access(filename, os.R_OK) == 0:
-        raise CriticalError("%s: is not readable." % (filename))
-
-    # Then ensure the file is up-to-date enough
-    mtime = os.stat(filename)[stat.ST_MTIME]
-    last_modified = time.time() - mtime
-    if last_modified > newer_than:
-        raise CriticalError(
-            "%s: was last modified on %s and is too old (> %s seconds)."
-            % (filename, time.ctime(mtime), newer_than)
-        )
-    if last_modified < 0:
-        raise CriticalError(
-            "%s: was last modified on %s which is in the future."
-            % (filename, time.ctime(mtime))
-        )
-
-
-################################################################################
diff --git a/files/plugins/check_cis_audit.py b/files/plugins/check_cis_audit.py
new file mode 100755
index 0000000..47c57d2
--- /dev/null
+++ b/files/plugins/check_cis_audit.py
@@ -0,0 +1,167 @@
+#!/usr/bin/env python3
+"""
+Check CIS audit score and verify the age of the last report.
+
+This check relies on a cron job that runs 'cis-audit' periodically.
+'cis-audit' is part of the 'usg-cisbenchmark' package that is available to
+Ubuntu Advantage customers.
+
+Example: check_cis_audit.py -p level2_server -a 170 -w 85 -c 80
+"""
+
+
+import argparse
+import glob
+import os
+import sys
+import time
+import xml.etree.ElementTree as ElementTree
+
+
+from nagios_plugin3 import (
+    CriticalError,
+    WarnError,
+    try_check,
+)
+
+
+AUDIT_FOLDER = "/usr/share/ubuntu-scap-security-guides"
+AUDIT_RESULT_GLOB = AUDIT_FOLDER + "/cis-*-results.xml"
+
+
+def get_audit_result_filepath():
+    """Get the path of the newest audit results file."""
+    audit_files = glob.glob(AUDIT_RESULT_GLOB)
+    if not audit_files:
+        msg = (
+            "CRITICAL: Could not find audit results file '{}', "
+            "make sure package usg-cisbenchmark is installed and cis-audit "
+            "cron job is running"
+        ).format(AUDIT_RESULT_GLOB)
+        raise CriticalError(msg)
+    # get newest results file if there are multiple (e.g. after upgrade)
+    return sorted(audit_files, key=os.path.getmtime).pop()
+
+
+def check_file_max_age(max_age, results_filepath):
+    """Verify the age of the file against the max_age parameter."""
+    age_hours = (time.time() - os.path.getmtime(results_filepath)) / 3600
+    if age_hours > max_age:
+        msg = (
+            "CRITICAL: The audit result file age {:.2f}h is older than threshold {}h "
+            "for '{}', make sure the cis-audit cronjob is working"
+        ).format(age_hours, max_age, results_filepath)
+        raise CriticalError(msg)
+
+
+def parse_profile_idref(profile_idref):
+    """Parse the profile idref and return cis-audit level."""
+    profiles = {  # name: match_string
+        "level1_server": "cis_profile_Level_1_Server",
+        "level2_server": "cis_profile_Level_2_Server",
+        "level1_workstation": "cis_profile_Level_1_Workstation",
+        "level2_workstation": "cis_profile_Level_2_Workstation",
+    }
+    for profile in profiles:
+        if profile_idref.endswith(profiles[profile]):
+            return profile
+
+    msg = "CRITICAL: could not determine profile from idref '{}'"
+    raise CriticalError(msg.format(profile_idref))
+
+
+def get_audit_score_and_profile(results_filepath):
+    """Extract audit score and profile level from results xml file."""
+    try:
+        root = ElementTree.parse(results_filepath).getroot()
+        namespace = root.tag.split("Benchmark")[0]
+        score = root.find(namespace + "TestResult/" + namespace + "score").text
+        profile_xml = root.find(namespace + "TestResult/" + namespace + "profile")
+        profile = parse_profile_idref(profile_xml.attrib["idref"])
+    except ElementTree.ParseError as parse_error:
+        msg = "CRITICAL: Could not parse audit results file '{}': '{}'"
+        raise CriticalError(msg.format(results_filepath, parse_error))
+    except PermissionError as permission_error:
+        msg = "CRITICAL: Could not read audit results file '{}': {}"
+        raise CriticalError(msg.format(results_filepath, permission_error))
+    return float(score), profile
+
+
+def check_cis_audit(target_profile, max_age, warning, critical):
+    """Check if recent audit report exists and score and level are as specified."""
+    results_filepath = get_audit_result_filepath()
+    check_file_max_age(max_age, results_filepath)
+    score, profile = get_audit_score_and_profile(results_filepath)
+
+    msg = "{}: cis-audit score is {:.2f} of 100; threshold -c {} -w {} ({}; {})"
+    if score < critical:
+        raise CriticalError(
+            msg.format("CRITICAL", score, critical, warning, profile, results_filepath)
+        )
+    if score < warning:
+        raise WarnError(
+            msg.format("WARNING", score, critical, warning, profile, results_filepath)
+        )
+
+    if target_profile != "" and target_profile != profile:
+        msg = (
+            "CRITICAL: requested audit profile '{}' does not match "
+            "report profile '{}' from '{}'"
+        ).format(target_profile, profile, results_filepath)
+        raise CriticalError(msg)
+
+    print("OK: cis-audit score is {:.2f} of 100 (profile: {})".format(score, profile))
+
+
+def parse_args(args):
+    """Parse command-line options."""
+    parser = argparse.ArgumentParser(
+        prog=__file__,
+        description="Check CIS audit score",
+    )
+    parser.add_argument(
+        "--max-age",
+        "-a",
+        type=int,
+        help="maximum age (h) of audit result file before alerting (default 170)",
+        default=170,
+    )
+    parser.add_argument(
+        "--cis-profile",
+        "-p",
+        choices=[
+            "",
+            "level1_server",
+            "level2_server",
+            "level1_workstation",
+            "level2_workstation",
+        ],
+        help="cis-audit level parameter (verifies if audit report matches)",
+        default="",
+    )
+    parser.add_argument(
+        "--warn",
+        "-w",
+        type=int,
+        help="a score below this number results in status WARNING (default: -1)",
+        default=-1,
+    )
+    parser.add_argument(
+        "--crit",
+        "-c",
+        type=int,
+        help="a score below this number results in status CRITICAL (default: -1)",
+        default=-1,
+    )
+    args = parser.parse_args(args)
+    return args
+
+
+def main():
+    """Parse args and check the audit report."""
+    args = parse_args(sys.argv[1:])
+    try_check(check_cis_audit, args.cis_profile, args.max_age, args.warn, args.crit)
+
+
+if __name__ == "__main__":
+    main()
diff --git a/files/plugins/cron_cis_audit.py b/files/plugins/cron_cis_audit.py
new file mode 100755
index 0000000..706337b
--- /dev/null
+++ b/files/plugins/cron_cis_audit.py
@@ -0,0 +1,138 @@
+#!/usr/bin/python3
+"""Run cis-audit if latest results are outdated."""
+
+import argparse
+import glob
+import grp
+import os
+import random
+import re
+import subprocess
+import sys
+import time
+
+
+AUDIT_FOLDER = "/usr/share/ubuntu-scap-security-guides"
+AUDIT_RESULT_GLOB = AUDIT_FOLDER + "/cis-*-results.xml"
+CLOUD_INIT_LOG = "/var/log/cloud-init-output.log"
+DEFAULT_PROFILE = "level1_server"
+PROFILES = [
+    "level1_server",
+    "level2_server",
+    "level1_workstation",
+    "level2_workstation",
+]
+MAX_SLEEP = 600
+PID_FILENAME = "/tmp/cron_cis_audit.pid"
+
+
+def _get_cis_hardening_profile(profile):
+    """Try to read the cis profile from cloud init log or default to level1_server."""
+    if profile in PROFILES:
+        return profile
+
+    if not os.path.exists(CLOUD_INIT_LOG) or not os.access(CLOUD_INIT_LOG, os.R_OK):
+        print(
+            "{} not existing/accessible, default to profile '{}'".format(
+                CLOUD_INIT_LOG, DEFAULT_PROFILE
+            )
+        )
+        return DEFAULT_PROFILE
+    pattern = re.compile(r"Applying Level-(1|2) scored (server|workstation)")
+    for _, line in enumerate(open(CLOUD_INIT_LOG)):
+        for match in re.finditer(pattern, line):
+            level, machine_type = match.groups()
+            return "level{}_{}".format(level, machine_type)
+    return DEFAULT_PROFILE
+
+
+def _get_cis_result_age():
+    """Get the age of the newest audit results file."""
+    audit_files = glob.glob(AUDIT_RESULT_GLOB)
+    if not audit_files:
+        return False
+    if len(audit_files) >= 1:
+        audit_file = sorted(audit_files, key=os.path.getmtime).pop()
+        return (time.time() - os.path.getmtime(audit_file)) / 3600
+
+
+def run_audit(profile):
+    """Execute the cis-audit as subprocess and allow nagios group to read result."""
+    cmd_run_audit = ["/usr/sbin/cis-audit", profile]
+    sleep_time = random.randint(0, MAX_SLEEP)
+    print("Sleeping for {}s to randomize the cis-audit start time".format(sleep_time))
+    time.sleep(sleep_time)
+    try:
+        print("Run cis-audit: {}".format(cmd_run_audit), flush=True)
+        subprocess.run(
+            cmd_run_audit, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL
+        )
+        print("Done cis-audit, change group of result file to nagios", flush=True)
+        for file in glob.glob(AUDIT_RESULT_GLOB):
+            os.chown(file, 0, grp.getgrnam("nagios").gr_gid)
+    except subprocess.CalledProcessError as e:
+        sys.exit(
+            "Failed running command '{}' Return Code {}: {}".format(
+                cmd_run_audit, e.returncode, e.output
+            )
+        )
+
+
+def parse_args(args):
+    """Parse command-line options."""
+    parser = argparse.ArgumentParser(
+        prog=__file__,
+        description="Run cis-audit if report is outdated",
+    )
+    parser.add_argument(
+        "--max-age",
+        "-a",
+        type=int,
+        help="maximum age (h) of audit result file before alerting (default 170)",
+        default=170,
+    )
+    profile_options = PROFILES + [""]
+    parser.add_argument(
+        "--cis-profile",
+        "-p",
+        choices=profile_options,
+        default="",
+        type=str,
+        help="cis-audit level parameter (verifies if audit report matches)",
+    )
+    return parser.parse_args(args)
+
+
+def main():
+    """Run cis-audit if audit results are outdated."""
+    args = parse_args(sys.argv[1:])
+
+    # folder does not exist - usg-cisbenchmark likely not installed
+    if not os.path.exists(AUDIT_FOLDER):
+        raise FileNotFoundError(
+            "Folder {} does not exist, is usg-cisbenchmark installed?".format(
+                AUDIT_FOLDER
+            )
+        )
+
+    # Ensure a single instance via a simple pidfile
+    pid = str(os.getpid())
+
+    if os.path.isfile(PID_FILENAME):
+        sys.exit("{} already exists, exiting".format(PID_FILENAME))
+
+    with open(PID_FILENAME, "w") as f:
+        f.write(pid)
+
+    try:
+        # audit result file does not exist or is outdated
+        audit_file_age_hours = _get_cis_result_age()
+        if audit_file_age_hours is False or audit_file_age_hours > args.max_age:
+            profile = _get_cis_hardening_profile(args.cis_profile)
+            run_audit(profile)
+    finally:
+        os.unlink(PID_FILENAME)
+
+
+if __name__ == "__main__":
+    main()
diff --git a/files/plugins/nagios_plugin3.py b/files/plugins/nagios_plugin3.py
new file mode 120000
index 0000000..045dad3
--- /dev/null
+++ b/files/plugins/nagios_plugin3.py
@@ -0,0 +1 @@
+../nagios_plugin3.py
\ No newline at end of file
diff --git a/hooks/nrpe_helpers.py b/hooks/nrpe_helpers.py
index b5631b6..6887069 100644
--- a/hooks/nrpe_helpers.py
+++ b/hooks/nrpe_helpers.py
@@ -443,7 +443,7 @@ class NRPECheckCtxt(dict):
 class SubordinateCheckDefinitions(dict):
     """Return dict of checks the charm configures."""
 
-    def __init__(self):
+    def __init__(self):  # noqa: C901
         """Set dict values."""
         self.procs = self.proc_count()
         load_thresholds = self._get_load_thresholds()
@@ -483,6 +483,19 @@ class SubordinateCheckDefinitions(dict):
             },
         ]
 
+        if hookenv.config("cis_audit_enabled"):
+            cmd_params = "-p '{}' {}".format(
+                hookenv.config("cis_audit_profile"),
+                hookenv.config("cis_audit_score"),
+            )
+            cis_audit_check = {
+                "description": "Check CIS audit",
+                "cmd_name": "check_cis_audit",
+                "cmd_exec": local_plugin_dir + "check_cis_audit.py",
+                "cmd_params": cmd_params,
+            }
+            checks.append(cis_audit_check)
+
         if not is_container():
             checks.extend(
                 [
diff --git a/hooks/nrpe_utils.py b/hooks/nrpe_utils.py
index bc342b0..14ce194 100644
--- a/hooks/nrpe_utils.py
+++ b/hooks/nrpe_utils.py
@@ -91,12 +91,9 @@ def install_charm_files(service_name):
     charm_plugin_dir = os.path.join(charm_file_dir, "plugins")
     pkg_plugin_dir = "/usr/lib/nagios/plugins/"
     local_plugin_dir = "/usr/local/lib/nagios/plugins/"
+    nagios_plugin = "nagios_plugin3.py"
 
     shutil.copy2(
-        os.path.join(charm_file_dir, "nagios_plugin.py"),
-        pkg_plugin_dir + "/nagios_plugin.py",
-    )
-    shutil.copy2(
         os.path.join(charm_file_dir, "nagios_plugin3.py"),
         pkg_plugin_dir + "/nagios_plugin3.py",
     )
@@ -104,9 +101,12 @@ def install_charm_files(service_name):
     shutil.copy2(os.path.join(charm_file_dir, "rsyncd.conf"), "/etc/rsyncd.conf")
     host.mkdir("/etc/rsync-juju.d", perms=0o755)
     host.rsync(charm_plugin_dir, "/usr/local/lib/nagios/", options=["--executability"])
-    for nagios_plugin in ("nagios_plugin.py", "nagios_plugin3.py"):
-        if not os.path.exists(local_plugin_dir + nagios_plugin):
-            os.symlink(pkg_plugin_dir + nagios_plugin, local_plugin_dir + nagios_plugin)
+
+    if not os.path.exists(local_plugin_dir + nagios_plugin):
+        os.symlink(
+            os.path.join(pkg_plugin_dir, nagios_plugin),
+            os.path.join(local_plugin_dir, nagios_plugin),
+        )
 
 
 def render_nrpe_check_config(checkctxt):
@@ -202,6 +202,24 @@ def has_consumer():
     )
 
 
+def update_cis_audit_cronjob(service_name):
+    """Install/Remove the cis-audit cron job."""
+    crond_file = "/etc/cron.d/cis-audit"
+
+    if not hookenv.config("cis_audit_enabled"):
+        if os.path.exists(crond_file):
+            os.remove(crond_file)
+            hookenv.log("Cronjob removed at {}".format(crond_file), hookenv.DEBUG)
+        return
+
+    file = "/usr/local/lib/nagios/plugins/cron_cis_audit.py"
+    profile = hookenv.config("cis_audit_profile")
+    cronjob = "*/10 * * * * root ({} -p '{}') 2>&1 | logger -t {}\n"
+    with open(crond_file, "w") as crond_fd:
+        crond_fd.write(cronjob.format(file, profile, "cron_cis_audit"))
+        hookenv.log("Cronjob configured at {}".format(crond_file), hookenv.DEBUG)
+
+
 class TolerantPortManagerCallback(PortManagerCallback):
     """Manage unit ports.
 
diff --git a/hooks/services.py b/hooks/services.py
index df360bd..34b9887 100644
--- a/hooks/services.py
+++ b/hooks/services.py
@@ -37,6 +37,7 @@ manager = ServiceManager(
                 nrpe_utils.update_monitor_relation,
                 nrpe_utils.create_host_export_fragment,
                 nrpe_utils.render_nrped_files,
+                nrpe_utils.update_cis_audit_cronjob,
                 helpers.render_template(
                     source="nrpe.tmpl", target="/etc/nagios/nrpe.cfg"
                 ),
diff --git a/mod/charmhelpers b/mod/charmhelpers
index b53f741..446cbfd 160000
--- a/mod/charmhelpers
+++ b/mod/charmhelpers
@@ -1 +1 @@
-Subproject commit b53f741d1c6f34f26f889d79afaad838dc14fdfa
+Subproject commit 446cbfdad83e15b5cfd20f862d3c3b5b1956b998
diff --git a/pytest.ini b/pytest.ini
new file mode 100644
index 0000000..d347a54
--- /dev/null
+++ b/pytest.ini
@@ -0,0 +1,4 @@
+[pytest]
+# Ignore: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working
+filterwarnings =
+    ignore::DeprecationWarning
diff --git a/tests/functional/tests/nrpe_tests.py b/tests/functional/tests/nrpe_tests.py
index 1b626a6..53fb6b9 100644
--- a/tests/functional/tests/nrpe_tests.py
+++ b/tests/functional/tests/nrpe_tests.py
@@ -281,6 +281,29 @@ class TestNrpe(TestBase):
             self._get_set_comparison(expected_host_only_checks, container_checks),
         )
 
+    def test_07_cronjob_checks(self):
+        """Check that cron job is installed and check enabled."""
+        model.set_application_config(
+            self.application_name,
+            {
+                "cis_audit_enabled": "True",
+            },
+        )
+        model.block_until_all_units_idle()
+        host_checks = self._get_unit_check_files("rabbitmq-server/0")
+        expected_shared_checks = set(["check_cis_audit.cfg"])
+        self.assertTrue(
+            expected_shared_checks.issubset(host_checks),
+            self._get_set_comparison(expected_shared_checks, host_checks),
+        )
+
+        cronjobs = self._get_cronjob_files("rabbitmq-server/0")
+        expected_cronjobs = set(["cis-audit"])
+        self.assertTrue(
+            expected_cronjobs.issubset(cronjobs),
+            self._get_set_comparison(expected_cronjobs, cronjobs),
+        )
+
     def _get_unit_check_files(self, unit):
         cmdline = "ls /etc/nagios/nrpe.d/"
         result = model.run_on_unit(unit, cmdline)
@@ -295,3 +318,9 @@ class TestNrpe(TestBase):
                 "Actual:": actual_checks,
             }
         )
+
+    def _get_cronjob_files(self, unit):
+        cmdline = "ls /etc/cron.d/"
+        result = model.run_on_unit(unit, cmdline)
+        self.assertEqual(result["Code"], "0")
+        return set(result["Stdout"].splitlines())
diff --git a/tests/unit/requirements.txt b/tests/unit/requirements.txt
index 9d85006..9ddb28d 100644
--- a/tests/unit/requirements.txt
+++ b/tests/unit/requirements.txt
@@ -2,3 +2,5 @@ coverage
 six
 PyYAML
 netifaces
+pytest
+pytest-cov
diff --git a/tests/unit/test_plugins_cis_audit.py b/tests/unit/test_plugins_cis_audit.py
new file mode 100644
index 0000000..e9a9319
--- /dev/null
+++ b/tests/unit/test_plugins_cis_audit.py
@@ -0,0 +1,273 @@
+"""Unit tests for files/plugins/(cron_cis_audit.py|check_cis_audit.py) module."""
+
+import argparse
+import os
+import tempfile
+from io import StringIO
+from time import sleep
+from unittest import TestCase, mock
+
+from files.plugins import check_cis_audit, cron_cis_audit
+
+from nagios_plugin3 import CriticalError, WarnError
+
+DUMMY_LOGLINES = """
+Processing triggers for man-db (2.8.3-2ubuntu0.1) ...
+Processing triggers for libc-bin (2.27-3ubuntu1.4) ...
+***Applying Level-2 scored server remediation for failures on a fresh Ubuntu 18.04 install***
+"""  # noqa: E501
+
+DUMMY_AUDIT_RESULT = """<?xml version="1.0" encoding="UTF-8"?>
+<Benchmark xmlns="http://checklists.nist.gov/xccdf/1.2" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" id="xccdf_com.ubuntu.bionic.cis_benchmark_CIS" resolved="1" xml:lang="en" style="SCAP_1.2">
+  <TestResult id="xccdf_org.open-scap_testresult_xccdf_com.ubuntu.bionic.cis_profile_Level_1_Server" start-time="2022-01-13T10:40:20" end-time="2022-01-13T10:40:45" version="2.0.1" test-system="cpe:/a:redhat:openscap:1.2.15">
+    <profile idref="xccdf_com.ubuntu.bionic.cis_profile_Level_1_Server"/>
+    <score system="urn:xccdf:scoring:default" maximum="100.000000">89.444443</score>
+  </TestResult>
+</Benchmark>
+"""  # noqa: E501
+
+
+class TestCronCisAudit(TestCase):
+    """Test the cis-audit cron job functions."""
+
+    cloud_init_logfile = os.path.join(tempfile.gettempdir(), "cloud-init-output.log")
+
+    @classmethod
+    def setUpClass(cls):
+        """Create dummy log file."""
+        with open(cls.cloud_init_logfile, "w") as f:
+            f.write(DUMMY_LOGLINES)
+
+    @classmethod
+    def tearDownClass(cls):
+        """Delete dummy log file."""
+        if os.path.exists(cls.cloud_init_logfile):
+            os.remove(cls.cloud_init_logfile)
+
+    def test_get_cis_hardening_profile_default(self):
+        """Test hardening profile passing defaults."""
+        # default profile should be return if profile passed is invalid
+        profile = cron_cis_audit._get_cis_hardening_profile("")
+        self.assertEqual(
+            profile,
+            cron_cis_audit.DEFAULT_PROFILE,
+            "Default profile should have been returned",
+        )
+        # parameter should be returned if parameter contains a valid profile
+        expected_profile = cron_cis_audit.PROFILES[3]
+        profile = cron_cis_audit._get_cis_hardening_profile(expected_profile)
+        self.assertEqual(
+            profile,
+            expected_profile,
+            "The profile in the parameter should have been returned",
+        )
+
+    @mock.patch("files.plugins.cron_cis_audit.CLOUD_INIT_LOG", cloud_init_logfile)
+    def test_get_cis_hardening_profile_cloudinit(self):
+        """Test the detection of the hardening profile from cloudinit.log."""
+        expected_profile = "level2_server"
+        profile = cron_cis_audit._get_cis_hardening_profile("")
+        self.assertEqual(
+            profile,
+            expected_profile,
+            "Profile from Dummy file should be 'level2_server'",
+        )
+
+    def test_get_cis_result_age(self):
+        """Test file age function."""
+        # file does not exist, returns false
+        self.assertFalse(cron_cis_audit._get_cis_result_age())
+
+        # file was created when test initiated, should return 0
+        with mock.patch(
+            "files.plugins.cron_cis_audit.AUDIT_RESULT_GLOB", self.cloud_init_logfile
+        ):
+            age_in_hours = cron_cis_audit._get_cis_result_age()
+            self.assertLess(
+                age_in_hours,
+                0.1,
+                "File age should be small because the file was just created",
+            )
+
+    @mock.patch("sys.stderr", new_callable=StringIO)
+    def test_parse_args(self, mock_stderr):
+        """Test the default parsing behavior of the argument parser."""
+        # test empty parameters
+        args = cron_cis_audit.parse_args([])
+        self.assertEqual(args, argparse.Namespace(cis_profile="", max_age=170))
+
+        # test setting parameters
+        args = cron_cis_audit.parse_args(["-a 1", "-p=level2_workstation"])
+        self.assertEqual(
+            args, argparse.Namespace(cis_profile="level2_workstation", max_age=1)
+        )
+
+        # test setting invalid parameter
+        with self.assertRaises(SystemExit):
+            cron_cis_audit.parse_args(["-p=invalid-parameter-test"])
+        self.assertRegex(
+            mock_stderr.getvalue(), r"invalid choice: 'invalid-parameter-test'"
+        )
+
+    @mock.patch("sys.argv", [])
+    def test_main_raise_exception(self):
+        """Test if main() raises FileNotFoundError if AUDIT_FOLDER does not exist."""
+        with self.assertRaises(FileNotFoundError):
+            cron_cis_audit.main()
+
+    @mock.patch("files.plugins.cron_cis_audit.MAX_SLEEP", 1)
+    @mock.patch("files.plugins.cron_cis_audit.AUDIT_FOLDER", "/tmp")
+    @mock.patch("sys.argv", [])
+    def test_main_run_audit(self):
+        """Test if main() calles cis-audit is called with correct arguments."""
+        with mock.patch("subprocess.run") as mock_subprocess_run:
+            process_mock = mock.Mock()
+            attrs = {"communicate.return_value": ("output", "error")}
+            process_mock.configure_mock(**attrs)
+            mock_subprocess_run.return_value = process_mock
+            cron_cis_audit.main()
+            self.assertTrue(mock_subprocess_run.called)
+            self.assertEqual(
+                str(mock_subprocess_run.call_args),
+                "call(['/usr/sbin/cis-audit', 'level1_server'], stdout=-3, stderr=-3)",
+            )
+
+
+class TestCheckCisAudit(TestCase):
+    """Test the cis-audit cron job functions."""
+
+    audit_result_folder = os.path.join(tempfile.gettempdir(), "test-audit-result")
+    audit_results_glob = audit_result_folder + "/cis-*-results.xml"
+    testfile1 = os.path.join(audit_result_folder, "cis-testfile1-results.xml")
+    testfile2 = os.path.join(audit_result_folder, "cis-testfile2-results.xml")
+
+    @classmethod
+    def setUpClass(cls):
+        """Create dummy audit folder and files."""
+        if not os.path.exists(cls.audit_result_folder):
+            os.mkdir(cls.audit_result_folder)
+            with open(cls.testfile1, mode="a"):
+                pass  # create empty file
+            sleep(0.1)
+            with open(cls.testfile2, mode="w") as f:
+                f.write(DUMMY_AUDIT_RESULT)
+
+    @classmethod
+    def tearDownClass(cls):
+        """Delete dummy log file."""
+        if os.path.exists(cls.audit_result_folder):
+            for file in os.listdir(cls.audit_result_folder):
+                os.remove(os.path.join(cls.audit_result_folder, file))
+            os.rmdir(cls.audit_result_folder)
+
+    def test_get_audit_result_filepath_not_found(self):
+        """Test that the audit results file can be found."""
+        with self.assertRaises(CriticalError):
+            check_cis_audit.get_audit_result_filepath()
+
+    @mock.patch("files.plugins.check_cis_audit.AUDIT_RESULT_GLOB", audit_results_glob)
+    def test_get_audit_result_filepath_found(self):
+        """Test that the newest audit file is returned."""
+        audit_result_filepath = check_cis_audit.get_audit_result_filepath()
+        expected = os.path.join(self.audit_result_folder, "cis-testfile2-results.xml")
+        self.assertEqual(audit_result_filepath, expected)
+
+    def test_check_file_max_age(self):
+        """Test that an exception is raised if the file is too old."""
+        with self.assertRaises(CriticalError):
+            check_cis_audit.check_file_max_age(0, self.testfile1)
+
+    def test_parse_profile_idref(self):
+        """Test that profile parsing works correctly."""
+        with self.assertRaises(CriticalError):
+            check_cis_audit.parse_profile_idref("unknown_profile")
+
+        profile_id = "xccdf_com.ubuntu.bionic.cis_profile_Level_2_Workstation"
+        self.assertEqual(
+            "level2_workstation", check_cis_audit.parse_profile_idref(profile_id)
+        )
+
+    def test_get_audit_score_and_profile(self):
+        """Test the parsing of the audit results file."""
+        # empty file raises CriticalError
+        with self.assertRaises(CriticalError):
+            check_cis_audit.get_audit_score_and_profile(self.testfile1)
+
+        # score and profile correctly read from xml
+        score, profile = check_cis_audit.get_audit_score_and_profile(self.testfile2)
+        self.assertEqual(score, 89.444443)
+        self.assertEqual(profile, "level1_server")
+
+    @mock.patch("sys.argv", [])
+    def test_parse_args(self):
+        """Test the argument parsing."""
+        # test default arguments
+        arguments = check_cis_audit.parse_args([])
+        self.assertEqual(
+            arguments,
+            argparse.Namespace(
+                cis_profile="",
+                crit=-1,
+                max_age=170,
+                warn=-1,
+            ),
+        )
+
+        # test setting arguments
+        arguments = check_cis_audit.parse_args(
+            ["-a", "1", "-c", "99", "-w", "90", "-p", "level2_server"]
+        )
+        self.assertEqual(
+            arguments,
+            argparse.Namespace(
+                cis_profile="level2_server",
+                crit=99,
+                max_age=1,
+                warn=90,
+            ),
+        )
+
+    @mock.patch("files.plugins.check_cis_audit.AUDIT_RESULT_GLOB", audit_results_glob)
+    def test_check_cis_audit(self):
+        """Test the check function with different parameters."""
+        # all ok
+        check_cis_audit.check_cis_audit("", 1, 80, 85)
+
+        # too old
+        with self.assertRaises(CriticalError) as error:
+            check_cis_audit.check_cis_audit("", 0, 80, 85)
+        self.assertRegex(
+            str(error.exception),
+            "CRITICAL: The audit result file age 0.00h is older than threshold.*",
+        )
+
+        # score below warning
+        with self.assertRaises(WarnError) as error:
+            check_cis_audit.check_cis_audit("", 1, 90, 80)
+        self.assertRegex(
+            str(error.exception),
+            "WARNING: cis-audit score is 89.44 of 100; threshold -c 80 -w 90",
+        )
+
+        # score below critical
+        with self.assertRaises(CriticalError) as error:
+            check_cis_audit.check_cis_audit("", 1, 95, 90)
+        self.assertRegex(
+            str(error.exception),
+            "CRITICAL: cis-audit score is 89.44 of 100; threshold -c 90 -w 95",
+        )
+
+        # profile does not match
+        with self.assertRaises(CriticalError) as error:
+            check_cis_audit.check_cis_audit("level2_workstation", 1, 85, 80)
+        self.assertRegex(
+            str(error.exception),
+            "CRITICAL: requested audit profile 'level2_workstation' does not match",
+        )
+
+    @mock.patch("files.plugins.check_cis_audit.AUDIT_RESULT_GLOB", audit_results_glob)
+    def test_main(self):
+        """Test the main function."""
+        namespace = argparse.Namespace(cis_profile="", max_age=1, crit=80, warn=70)
+        with mock.patch("argparse.ArgumentParser.parse_args", return_value=namespace):
+            check_cis_audit.main()
diff --git a/tox.ini b/tox.ini
index 096b228..21b6e4b 100644
--- a/tox.ini
+++ b/tox.ini
@@ -64,9 +64,8 @@ deps =
 
 [testenv:unit]
 commands =
-    coverage run -m unittest discover -s {toxinidir}/tests/unit -v
-    coverage report --omit tests/*,mod/*,.tox/*
-    coverage html --omit tests/*,mod/*,.tox/*
+    pytest {posargs:-v --cov --cov-config={toxinidir}/.coveragerc --cov-report=term-missing --cov-report html --cov-branch --ignore={toxinidir}/tests/charmhelpers} \
+        {toxinidir}/tests/unit
 deps = -r{toxinidir}/tests/unit/requirements.txt
 
 [testenv:func]