Merge ~aluria/charm-openstack-service-checks/+git/charm-openstack-service-checks:bug/1835433-part2 into ~canonical-bootstack/charm-openstack-service-checks:master

[Alvaro Uria]

Improve support to rally tests

[🤖 Canonical IS Merge Bot (canonical-is-mergebot)]

This merge proposal is being monitored by mergebot. Change the status to Approved to merge.

[Alvaro Uria (aluria)]

Regarding bug 1837234 (TLS verification failure), it is still happening after this MP. I need to review it. Rally is now unconfined and update-ca-certificates was run, so there is no reason for verifications to fail. Nevertheless, configuring https_cacert with the same value as OS_CACERT doesn't fix the problem. It may be a problem on the version supported in bionic (v0.9.1).

OTOH, Xenial rally apt package is v0.1.1, which doesn't support the same commands as in v0.9.1. So the rally check is only supported in Bionic and on. Furthermore, latest versions of rally use "rally db recreate" instead of "rally-manage db recreate", which may break the charm code in the future.

Other tests around proxying, choosing custom flavor and image IDs, and cleaning the fcbtest snap and created nagiososc user are tested and work.

All tests pass except the functional test for "enable_rally", because "rally verify" uses "install" (v0.1.1) isntead of "create" (v0.9.1).

[Peter Sabaini (peter-sabaini)]

Hey Alvaro,

some comments inline.

I'm also getting a linter error:

./files/plugins/check_rally.py:19:1: C901 'main' is too complex (11)

Thanks,
peter.

review: Needs Fixing

[Alvaro Uria (aluria)]

I've tried to address all the suggestions, and have rewritten the charm code to support the new snap that we use (bsrally). Rally behind a proxy now works.

However, Rally TLS support is not there yet because of a snap bug (the charm does support its configuration):
https://forum.snapcraft.io/t/using-the-system-certificate-authorities/10732/10

Nevertheless, I think this code should be merged as the "upgrade-charm" hook will remove the "fcbtest" snap and the new one won't be installed except if "check-rally=true" (which shouldn't be done if we're using a custom CA).

[Peter Sabaini (peter-sabaini)]

Hey, some minor nits / comments inline, but all in all lgtm

review: Approve

[🤖 Canonical IS Merge Bot (canonical-is-mergebot)]

Failed to merge change (unable to merge source repository due to conflicts), setting status to needs review.

[James Troup (elmo)]

Setting this back to Work In Progress. Please rebase?

Unmerged commits

e5c15b7... by Alvaro Uria on 2019-10-16

Fix TLS support on rally check

d49965f... by Alvaro Uria on 2019-10-16

Fix bugs, Add the create-flavor-rally action.

 * Fix bug on run_rally.py. Confined snaps don't use /tmp but
 /tmp/snap.<snapname>. Temporary files need to prefix this path.
 * Create rally flavor unless it exists, or booting test instances will
 fail (Cirros image can be created by rally, though). This step is
 called when check-rally=true
 * Alternatively, an action has been added to verify if a flavor already
 exists or create it (1 vcpu, 64MB RAM).

8a1e0d1... by Alvaro Uria on 2019-10-15

Rewrite rally check to use bsrally snap

19fd7fc... by Alvaro Uria on 2019-09-13

Improve support to rally tests

A new nrpe check has been added, which parses the results of running a
subset of rally tests against an environment. Now rally uses the APT
package and imports a tempest repository from a maintainers fork.
 * 'check-rally' is set to false by default
 * certain OpenStack components tests can be skipped. The parameter has
 been renamed from 'skip-rally' to 'rally-skip-services'
 rally-skip-services="cinder,glance,nova,neutron" would skip all tests
 except Keystone's token issue
 * cronjob can be scheduled via 'rally-cron-schedule' config parameter
 * 'rally-flavor-id': Specify flavor for test VMs. It is important that
 no disk nor ephemeral options are set on the flavor, since tests expect
 volumes to be attached as vdb
 * 'rally-image-id': ID of the image to be used. Otherwise, rally will
 try to discover an image under a name containing "cirros" or "testvm".
 * upgrade-charm supports the transition from the fcbtest snap to the
 rally apt package (check-rally needs to be set to false).

Closes-Bug: 1835433
Closes-Bug: 1841092
Closes-Bug: 1838174
Closes-Bug: 1837234

diff --git a/actions.yaml b/actions.yaml
new file mode 100644
index 0000000..078ad56
--- /dev/null
+++ b/actions.yaml
@@ -0,0 +1,2 @@
+create-flavor-rally:
+  description: Creates the default flavor used by rally to boot test instances
diff --git a/actions/create-flavor-rally b/actions/create-flavor-rally
new file mode 100755
index 0000000..d3ef3f8
--- /dev/null
+++ b/actions/create-flavor-rally
@@ -0,0 +1,15 @@
+#!/usr/local/sbin/charm-env python3
+# Create the default flavor used by rally to boot test instances
+
+from charmhelpers.core.hookenv import (
+    action_fail,
+    action_set,
+)
+from lib_openstack_service_checks import OSCHelper
+
+helper = OSCHelper()
+
+if helper.create_flavor_rally():
+    action_set({'status': 'Flavor available. check-rally can be enabled.'})
+else:
+    action_fail('Unable to create a flavor for rally.')
diff --git a/config.yaml b/config.yaml
index 86ade7d..08aa4f1 100644
--- a/config.yaml
+++ b/config.yaml
@@ -9,19 +9,32 @@ options:
     type: boolean
     description: |
       Switch to turn on or off rally checks via the fcbtest snap. By default, rally nrpe check is disabled.
-  skip-rally:
-    default: ""
-    type: string
-    description: |
-      Comma separated list of OpenStack components to not monitor. An empty string means all components will be
-      monitored (up to the number of currently supported components: Cinder, Glance, Nova, Neutron).
-      Sample: skip-rally=cinder
   rally-cron-schedule:
     default: "*/15 * * * *"
     type: string
     description: |
       Cron schedule used to run the rally tests. Default value is every 15 minutes.
       Furthermore, the cronjob is scheduled to time out after 13 minutes (SIGTERM) or 14 minutes (SIGKILL).
+  rally-flavor-id:
+    default: ""
+    type: string
+    description: |
+      ID of the flavor to be used on the rally verifier tests (tempestverifier). By default, flavor discovery
+      under the following specs (1 vcpu, 64MB RAM, 0 extra disk, 10G ephemeral disk) will happen.
+      The alternative flavor (128MB RAM) will be used when needed (ie. resize tests).
+  rally-image-id:
+    default: ""
+    type: string
+    description: |
+      ID of the image to be used on the rally verifier tests (tempestverifier). By default, a cirros image
+      will be created under a name that contains "cirros" or "testvm" on it.
+  rally-skip-services:
+    default: ""
+    type: string
+    description: |
+      Comma separated list of OpenStack components to not monitor. An empty string means all components will be
+      monitored (up to the number of currently supported components: Cinder, Glance, Nova, Neutron).
+      Sample: rally-skip-services=cinder
   os-credentials:
     default: ""
     type: string
diff --git a/files/plugins/check_rally.py b/files/plugins/check_rally.py
index 0bf2dae..5c6b6f8 100755
--- a/files/plugins/check_rally.py
+++ b/files/plugins/check_rally.py
@@ -1,70 +1,30 @@
 #!/usr/bin/env python3
-import collections
 import os
-import re
 import sys
 
 import json
 
 # ie. {0} tempest.test.test1 ... success
 TEMPEST_TEST_RE = r'{\d+} [.\w]+ ... (\w+)'
-INPUT_FILE = '/home/nagiososc/rally.status'
-
-
-def print_results(results):
-    status_message = collections.defaultdict(lambda: 'UNKNOWN')  # 3
-    status_message.update({
-        'success': 'OK',  # 0
-        'fail': 'CRITICAL',  # 2
-        'skip': 'WARNING',  # 1
-    })
-    return_codes = {msg: code
-                    for code, msg in enumerate(['OK', 'WARNING', 'CRITICAL', 'UNKNOWN'])}
-    rc = return_codes['OK']
-    summary = collections.defaultdict(lambda: 0)
-
-    output = []
-    for result in sorted(results, key=lambda result: result['message']):
-        if result.get('message', '').startswith('CRITICAL: '):
-            # Exception caused by run_rally.py, without running 'rally verify'
-            output.append(result['message'])
-            continue
-        elif result.get('message', '').startswith('{'):
-            # only parse json lines - rest, ignore
-            test_re = re.match(TEMPEST_TEST_RE, result.get('message', ''))
-            if not test_re:
-                continue
-
-            test_status = test_re.groups()[0]
-            output.append('{}: {}'.format(status_message[test_status],
-                                          result['message']))
-            summary[test_status] += 1
-
-    # make the first line carry the worst event out of all the parsed ones
-    # ie. all ok except one critical event will return the first line (and return code)
-    # as critical
-    nagios_status = 'OK'
-    for status_msg in ['CRITICAL', 'WARNING', 'UNKNOWN', 'OK']:
-        status = [msg for msg in status_message.keys()
-                  if status_message[msg] == status_msg]
-        if not status or status[0] not in summary:
-            continue
-
-        status = status[0]
-        if summary[status] > 0:
-            rc = return_codes[status_msg]
-            nagios_status = status_msg
-            break
-
-    if len(summary) > 0:
-        print('{}: '.format(nagios_status)
-              + ', '.join(['{}[{}]'.format(status_msg, summary[status_msg])
-                           for status_msg in sorted(summary,
-                                                    key=lambda status: summary[status],
-                                                    reverse=True)
-                           ]))
-    print('\n'.join(output))
-    return rc
+INPUT_FILE = '/home/nagiososc/rally.status.json'
+
+STATUS_CODES = {
+    'OK': 0,
+    'WARNING': 1,
+    'CRITICAL': 2,
+    'UNKNOWN': 3,
+}
+
+
+def calculate_criticality(status):
+    if status.get('failures', 0) > 0:
+        return 'CRITICAL'
+    elif status.get('skipped', 0) > 0:
+        return 'WARNING'
+    elif status.get('success', 0) > 0:
+        return 'OK'
+    else:
+        return 'UNKNOWN'
 
 
 def main(results_filename):
@@ -73,26 +33,40 @@ def main(results_filename):
         return 3
 
     results = []
-    with open(results_filename, 'r') as fd:
-        for line in fd.readlines():
-            line = line.strip()
-            if not line:
-                continue
-            elif len(line) > 5 and line[-5:] == '\x1b[00m':
-                line = line[:-5]
-
-            if not line.startswith('{'):
-                results.append({'message': line})
-                continue
-
-            try:
-                results.append(json.loads(line))
-            except json.decoder.JSONDecodeError as error:
-                print('UNKNOWN: line[{}], error[{}]'.format(line, str(error)))
-                return 3
-
-    rc = print_results(results)
-    return rc
+    try:
+        with open(results_filename, 'r') as fd:
+            results = json.load(fd)
+    except json.decoder.JSONDecodeError as error:
+        print('UNKNOWN: {}'.format(str(error)))
+        return STATUS_CODES['UNKNOWN']
+
+    # Internal cronjob errors (not returned by rally)
+    for status in ['CRITICAL', 'UNKNOWN']:
+        if status in results:
+            print('{}: {}'.format(status, results[status]))
+            return STATUS_CODES[status]
+
+    if 'verifications' not in results:
+        print('UNKNOWN: {} is not parsable, missing "verifications" key'.format(results_filename))
+        return STATUS_CODES['UNKNOWN']
+
+    verif_id = list(results['verifications'])[0]
+    verif_summary = results['verifications'].get(verif_id, {})
+
+    status = {key: value for key, value in verif_summary.items()
+              if key in ['skipped', 'success', 'failures', 'tests_duration']}
+    status['NAGIOS_MSG'] = calculate_criticality(status)
+
+    testresults = []
+    for testname in results['tests'].keys():
+        verif_id = list(results['tests'][testname]['by_verification'].keys())[0]
+        verif_summary = results['tests'][testname]['by_verification'].get(verif_id, {'status': 'unknown'})
+        testresults.append('{}... {}'.format(testname, verif_summary['status']))
+
+    print('{NAGIOS_MSG}: success[{success}], failures[{failures}], skipped[{skipped}]...'
+          ' tests_duration[{tests_duration}]'.format(**status))
+    print('\n'.join(testresults))
+    return STATUS_CODES[status['NAGIOS_MSG']]
 
 
 if __name__ == '__main__':
diff --git a/files/run_rally.py b/files/run_rally.py
index c0e5655..5acbf55 100755
--- a/files/run_rally.py
+++ b/files/run_rally.py
@@ -1,29 +1,18 @@
-#!/usr/bin/env python3
+#!/usr/local/sbin/charm-env python3
 import datetime
 import json
 import os
 import subprocess
-import sys
 
+from pid import PidFile, PidFileAlreadyLockedError
 import shutil
 import tempfile
 
-OUTPUT_FILE = '/home/nagiososc/rally.status'
-HISTORY_FOLDER = '/home/nagiososc/rallystatuses'
-
-
-def get_backup_output_filename():
-    if not os.path.isdir(HISTORY_FOLDER):
-        os.mkdir(HISTORY_FOLDER, mode=0o755)
-
-    weekday = datetime.datetime.today().weekday()
-    i = 0
-    statusfile = os.path.join(HISTORY_FOLDER, 'rally.status.{}.{}'.format(weekday, i))
-    while os.path.exists(statusfile):
-        i += 1
-        statusfile = os.path.join(HISTORY_FOLDER, 'rally.status.{}.{}'.format(weekday, i))
-
-    return statusfile
+HOME_DIR = '/home/nagiososc'
+RALLY_CONFIGFILE = os.path.join(HOME_DIR, 'snap', 'bsrally', 'common', 'rally.conf')
+TESTS_FILE = os.path.join(HOME_DIR, 'ostests.txt')
+OUTPUT_FILE = os.path.join(HOME_DIR, 'rally.status.json')
+HISTORY_FOLDER = os.path.join(HOME_DIR, 'rallystatuses')
 
 
 def _load_envvars(novarc='/var/lib/nagios/nagios.novarc'):
@@ -41,41 +30,87 @@ def _load_envvars(novarc='/var/lib/nagios/nagios.novarc'):
         i += 1
 
     os.environ['SHELL'] = '/bin/bash'
-    os.environ['HOME'] = '/home/nagiososc'
+    os.environ['HOME'] = HOME_DIR
     os.environ['PATH'] = '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin'
 
     return i >= 3
 
 
-def main(testfile='/home/nagiososc/ostests.txt'):
-    if not _load_envvars():
-        print('UNKNOWN: could not load OS_ envvars')
-        sys.exit(3)
+def get_backup_output_filename():
+    if not os.path.isdir(HISTORY_FOLDER):
+        os.mkdir(HISTORY_FOLDER, mode=0o755)
+
+    weekday = datetime.datetime.today().weekday()
+    i = 0
+    statusfile = os.path.join(HISTORY_FOLDER, 'rally.status.{}.{}'.format(weekday, i))
+    while os.path.exists(statusfile):
+        i += 1
+        statusfile = os.path.join(HISTORY_FOLDER, 'rally.status.{}.{}'.format(weekday, i))
+
+    return statusfile
+
+
+def log_and_backup(filename, message=None):
+    if message:
+        with open(filename, 'w') as fd:
+            fd.write(message)
+
+    # Snaps generate "/tmp/filename" in a cage (see prefix) that we check
+    # If the rally command is replaced by an APT pkg, no prefix will exist
+    tmp_filename = None
+    for prefix in ['', '/tmp/snap.bsrally']:
+        searched_file = '{}{}'.format(prefix, filename)
+        if os.path.exists(searched_file):
+            tmp_filename = searched_file
+            break
 
+    # Unlikely, but no action would be taken if tempfile does not exist
+    if tmp_filename is None:
+        return
+
+    shutil.copy2(tmp_filename, OUTPUT_FILE)
+    shutil.copy2(OUTPUT_FILE, get_backup_output_filename())
+    os.unlink(tmp_filename)
+
+
+def run_rally(testfile=TESTS_FILE):
     tempoutputfile = tempfile.mktemp()
-    cmd1 = ['fcbtest.rally', 'deployment', 'use', 'snap_generated']
-    cmd2 = ['fcbtest.rally', '--use-json', 'verify', 'start', '--load-list', testfile, '--detailed']
+
+    if not _load_envvars():
+        msg = {'UNKNOWN': 'could not load OS_ envvars'}
+        log_and_backup(tempoutputfile, json.dumps(msg))
+        return
+
+    rally_cmd = ['bsrally.rally', '--config-file', RALLY_CONFIGFILE]
+    cmd1 = rally_cmd + ['deployment', 'use', 'charm_generated']
+    cmd2 = rally_cmd + ['verify', 'start', '--load-list', testfile]
+    cmd3 = rally_cmd + ['verify', 'report', '--to', tempoutputfile]
     try:
         subprocess.check_output(cmd1, stderr=subprocess.STDOUT)
-        output = subprocess.check_output(cmd2, stderr=subprocess.STDOUT)
-        with open(tempoutputfile, 'w') as fd:
-            fd.write(output.decode('utf-8'))
-        shutil.copy2(tempoutputfile, OUTPUT_FILE)
-        shutil.copy2(OUTPUT_FILE, get_backup_output_filename())
-        os.unlink(tempoutputfile)
+        subprocess.check_output(cmd2, stderr=subprocess.STDOUT)
+        subprocess.check_output(cmd3, stderr=subprocess.STDOUT)
+        # File is moved from the temporary location - no further msg is recorded
+        log_and_backup(tempoutputfile)
     except subprocess.CalledProcessError as error:
         msg = {
-            'message': 'CRITICAL: fcbtest.rally command failed. {} - {}'.format(str(error), str(error.stdout)),
+            'CRITICAL': 'rally command failed. {} - {}'.format(str(error),
+                                                               str(error.stdout)),
         }
-        with open(OUTPUT_FILE, 'w') as fd:
-            fd.write('{}\n'.format(json.dumps(msg)))
+        log_and_backup(tempoutputfile, json.dumps(msg))
 
     except IOError as error:
-        msg = {
-            'message': 'CRITICAL: IOError. {}'.format(str(error)),
-        }
-        with open(OUTPUT_FILE, 'a') as fd:
-            fd.write('{}\n'.format(json.dumps(msg)))
+        msg = {'CRITICAL': 'IOError. {}'.format(str(error))}
+        log_and_backup(tempoutputfile, json.dumps(msg))
+
+
+def main():
+    try:
+        with PidFile(pidname='rally', piddir='/var/lock'):
+            run_rally()
+    except PidFileAlreadyLockedError as error:
+        tempoutputfile = tempfile.mktemp()
+        msg = {'CRITICAL': 'Lock error: {}'.format(str(error))}
+        log_and_backup(tempoutputfile, json.dumps(msg))
 
 
 if __name__ == '__main__':
diff --git a/layer.yaml b/layer.yaml
index d8a972a..3741f72 100644
--- a/layer.yaml
+++ b/layer.yaml
@@ -19,10 +19,3 @@ options:
       - python3-keystoneclient
       - python3-openstackclient
       - python-openstackclient
-  snap:
-    fcbtest:
-      channel: stable
-      devmode: false
-      jailmode: false
-      dangerous: false
-      classic: false
diff --git a/lib/lib_openstack_service_checks.py b/lib/lib_openstack_service_checks.py
index fd12f51..f05a191 100644
--- a/lib/lib_openstack_service_checks.py
+++ b/lib/lib_openstack_service_checks.py
@@ -1,20 +1,18 @@
-import collections
-import glob
 import os
 import pwd
 import re
 import subprocess
 from urllib.parse import urlparse
 
-import configparser
-
 from charmhelpers.core.templating import render
 from charmhelpers.contrib.openstack.utils import config_flags_parser
 from charmhelpers.core import hookenv, host, unitdata
 from charmhelpers.contrib.charmsupport.nrpe import NRPE
-from charms.reactive import any_file_changed
+from charms.layer import snap
+
 import keystoneauth1
 from keystoneclient import session
+import os_client_config
 
 
 class OSCCredentialsError(Exception):
@@ -49,6 +47,30 @@ class OSCHelper():
         return '/usr/local/bin/'
 
     @property
+    def rallyuser(self):
+        return 'nagiososc'
+
+    @property
+    def rally_home(self):
+        return os.path.join('/home', self.rallyuser)
+
+    @property
+    def rally_snap_common(self):
+        return os.path.join(self.rally_home, 'snap', 'bsrally', 'common')
+
+    @property
+    def rallydb(self):
+        return os.path.join(self.rally_snap_common, 'rally.db')
+
+    @property
+    def rally_extra_configfile(self):
+        return os.path.join(self.rally_snap_common, 'extra_config.conf')
+
+    @property
+    def rally_configfile(self):
+        return os.path.join(self.rally_snap_common, 'rally.conf')
+
+    @property
     def rally_cron_file(self):
         return '/etc/cron.d/osc_rally'
 
@@ -57,16 +79,9 @@ class OSCHelper():
         return self.charm_config['check-rally']
 
     @property
-    def skipped_rally_checks(self):
-        skipped_os_components = self.charm_config['skip-rally'].strip()
-        if not skipped_os_components:
-            return []
-
-        # filter skip-rally input to match available (or supported) components that
-        # should be disabled
-        available_os_components = 'cinder glance nova neutron'.split()
-        return [comp.strip().lower() for comp in skipped_os_components.split(',')
-                if comp.strip().lower() in available_os_components]
+    def is_rally_installed(self):
+        kv = unitdata.kv()
+        return kv.get('rallyinstalled', False)
 
     @property
     def rally_cron_schedule(self):
@@ -76,6 +91,31 @@ class OSCHelper():
         else:
             return schedule.strip()
 
+    @property
+    def rally_image_id(self):
+        return self.charm_config['rally-image-id']
+
+    @property
+    def rally_flavor_id(self):
+        return self.charm_config['rally-flavor-id']
+
+    @property
+    def rally_available_os_components(self):
+        return ['cinder', 'glance', 'nova', 'neutron']
+
+    @property
+    def rally_skipped_checks(self):
+        # Expected comma-separated value
+        skipped_os_components = self.charm_config['rally-skip-services'].strip()
+        if not skipped_os_components:
+            return set()
+
+        # Filter rally-skip-services input to match available (and supported) components
+        # that should be disabled
+        return set([component.strip().lower()
+                    for component in skipped_os_components.split(',')
+                    if component.strip().lower() in self.rally_available_os_components])
+
     def get_os_credentials(self):
         ident_creds = config_flags_parser(self.charm_config['os-credentials'])
         if not ident_creds.get('auth_url'):
@@ -361,6 +401,10 @@ class OSCHelper():
             os.environ[key] = value
             i += 1
 
+        # Helps clone repo from github (tempest verifier)
+        proxy_settings = hookenv.env_proxy_settings()
+        os.environ.update({key: value for key, value in proxy_settings.items()})
+
         return i >= 3
 
     def _run_as(self, user, user_cmd):
@@ -383,107 +427,136 @@ class OSCHelper():
             return True
 
         except KeyError as error:
-            hookenv.log('_run_as - user does not exist => {}'.format(str(error)))
+            hookenv.log('_run_as - user does not exist => {}'.format(str(error)),
+                        hookenv.ERROR)
             return False
         except subprocess.CalledProcessError as error:
-            hookenv.log('_run_as - cmd failed => {}'.format(str(error)))
+            hookenv.log('_run_as - cmd failed => {}'.format(str(error)),
+                        hookenv.ERROR)
             if error.stderr:
-                hookenv.log('_run_as stderr => {}'.format(error.stderr))
+                hookenv.log('_run_as stderr => {}'.format(error.stderr),
+                            hookenv.ERROR)
             if error.stdout:
-                hookenv.log('_run_as stderr => {}'.format(error.stdout))
+                hookenv.log('_run_as stderr => {}'.format(error.stdout),
+                            hookenv.ERROR)
             return False
 
-    @property
-    def _rallyuser(self):
-        return 'nagiososc'
+    def add_nagios_to_rally(self, user, group):
+        host.add_user_to_group(user, group)
+        host.service_restart('nagios-nrpe-server')
+
+    def regenerate_extra_config(self):
+        context = {
+            'image_ref': self.rally_image_id,
+            'flavor_ref': self.rally_flavor_id,
+        }
+        render(source='extra_config.conf.j2', target=self.rally_extra_configfile,
+               context=context, perms=0o640, owner=self.rallyuser, group=self.rallyuser)
+
+    def regenerate_rally_config(self):
+        # Render rally config file
+        render(source='rally.conf.j2', target=self.rally_configfile,
+               context={}, perms=0o640, owner=self.rallyuser, group=self.rallyuser)
+
+    def regenerate_tempest_conf(self):
+        cmd = [
+            'bsrally.rally', '--config-file', self.rally_configfile,
+            'verify', 'configure-verifier',
+            '--extend', self.rally_extra_configfile,
+            '--reconfig',
+        ]
+        self.regenerate_extra_config()
+        installed = self._run_as(self.rallyuser, cmd)
+        return installed
+
+    def propagate_rally_cacert(self, cacert_filename):
+        """Copies trusted_ssl_ca to rallyuser home dir (snap common folder)
+        """
+        if cacert_filename and os.path.exists(cacert_filename):
+            host.rsync(cacert_filename, self.rally_snap_common)
+            new_cacert_filename = os.path.join(
+                self.rally_snap_common, os.path.basename(cacert_filename))
+            os.chmod(new_cacert_filename, 0o644)
+            os.environ['OS_CACERT'] = new_cacert_filename
 
     def install_rally(self):
+        # Persistent value to avoid running this process more than needed
         kv = unitdata.kv()
         if kv.get('rallyinstalled', False):
             return True
 
+        # Load OpenStack env vars (OS_*)
         if not self._load_envvars:
             hookenv.log('install_rally - could not load nagios.novarc')
             return False
 
-        user = self._rallyuser
-        host.adduser(user)
-        host.mkdir(os.path.join('/home', user), owner=user, group=user, perms=0o755, force=False)
-
-        for tool in ['rally', 'tempest']:
-            toolname = 'fcbtest.{}init'.format(tool)
-            installed = self._run_as(user, [toolname])
+        # Create user that will run rally
+        user = self.rallyuser
+        user_pwd = host.adduser(user)
+        host.mkdir(self.rally_home, owner=user, group=user, perms=0o755, force=False)
+
+        # rally.conf and tempest extra configs when the new verifier is created
+        self.regenerate_rally_config()
+        self.regenerate_extra_config()
+
+        # touch /path/to/rally.db so we can recreate it
+        # (otherwise, rally db recreate will fail)
+        if not os.path.isfile(self.rallydb):
+            open(self.rallydb, 'a').close()
+            os.chown(self.rallydb, user_pwd.pw_uid, user_pwd.pw_gid)
+
+        # Make 'snap' directory owned by the new user
+        dir = self.rally_home
+        for subdir in ['snap', 'bsrally']:
+            dir = os.path.join(dir, subdir)
+            os.chown(dir, user_pwd.pw_uid, user_pwd.pw_gid)
+
+        # OS_CACERT needs to point to the user home dir (snap is confined)
+        os_cacert = os.getenv('OS_CACERT', False)
+        self.propagate_rally_cacert(os_cacert)
+
+        # Initialize Rally
+        rally_cmd = 'bsrally.rally --config-file {}'.format(self.rally_configfile)
+        cmds = [
+            '{} db recreate'.format(rally_cmd),
+            '{} deployment create --fromenv --name charm_generated'.format(rally_cmd),
+            ('{} verify create-verifier --type tempest'
+             ' --source /snap/bsrally/current/tempest'
+             ' --system-wide --name tempestverifier'.format(rally_cmd)
+             ),
+            '{} verify configure-verifier --extend {} --reconfig'.format(
+                rally_cmd, self.rally_extra_configfile),
+        ]
+
+        for cmd in cmds:
+            installed = self._run_as(user, cmd.split())
             if not installed:
-                hookenv.log('install_rally - could not initialize {}'.format(tool))
+                hookenv.log('install_rally: could not initialize... {}'.format(cmd))
                 return False
 
+        # By default, snap's rally tempdir is uid=0, chmod=700,
+        # but cronjobs run as rallyuser
+        tmpdir = '/tmp/snap.bsrally'
+        if os.path.isdir(tmpdir):
+            os.chmod(tmpdir, 0o755)
+
         kv.set('rallyinstalled', True)
         return True
 
-    def _regenerate_tempest_conf(self, tempestfile):
-        config = configparser.ConfigParser()
-        config.read(tempestfile)
-        for section in config.keys():
-            for key, value in config[section].items():
-                try:
-                    if section != 'DEFAULT' and key in config['DEFAULT'].keys():
-                        # avoid copying the DEFAULT config options to the rest of sections
-                        continue
-                except KeyError:
-                    # DEFAULT section does not exist
-                    pass
-
-                # Enable Cinder, which is a default OpenStack service
-                if section == 'service_available' and key == 'cinder':
-                    config[section][key] = 'True'
-
-        with open(tempestfile, 'w') as fd:
-            config.write(fd)
-
-    def reconfigure_tempest(self):
-        """Expects an external network already configured, and enables cinder tests
+    def _get_rally_checks_context(self):
+        """Returns dict with enabled/disabled OpenStack services
 
-        Sample:
-        RALLY_VERIFIER=7b9d06ef-e651-4da3-a56b-ecac67c595c5
-        RALLY_VERIFICATION=4a730963-083f-4e1e-8c55-f2b4b9c9c0ac
-        RALLY_DEPLOYMENT=a75657c6-9eea-4f00-9117-2580fe056a80
-        RALLY_ENV=a75657c6-9eea-4f00-9117-2580fe056a80
+        Used to render ostests.txt (list of tempest tests to be run)
         """
-        RALLY_CONF = ['/home', self._rallyuser, 'snap', 'fcbtest', 'current', '.rally']
-        rally_globalconfig = os.path.join(*RALLY_CONF, 'globals')
-        if not os.path.isfile(rally_globalconfig):
-            return False
-
-        uuids = collections.defaultdict(lambda: '*')
-        with open(rally_globalconfig, 'r') as fd:
-            for line in fd.readlines():
-                key, value = line.strip().split('=')
-                if key in ['RALLY_VERIFIER', 'RALLY_DEPLOYMENT']:
-                    uuids[key] = value
-
-        tempest_path = os.path.join(*RALLY_CONF, 'verification',
-                                    'verifier-{RALLY_VERIFIER}'.format(**uuids),
-                                    'for-deployment-{RALLY_DEPLOYMENT}'.format(**uuids),
-                                    'tempest.conf')
-        tempestfile = glob.glob(tempest_path)
-        if len(tempestfile) == 0:
-            # No tempest.conf file generated, yet
-            return False
-
-        if not any_file_changed([tempestfile[0]]):
-            return False
-
-        self._regenerate_tempest_conf(tempestfile[0])
-        return True
-
-    def _get_rally_checks_context(self):
-        os_components_skip_list = self.skipped_rally_checks
+        os_components_skip_list = self.rally_skipped_checks
         ctxt = {}
-        for comp in 'cinder glance nova neutron'.split():
-            ctxt.update({comp: comp not in os_components_skip_list})
+        ctxt = {component: component not in os_components_skip_list
+                for component in self.rally_available_os_components}
         return ctxt
 
     def update_rally_checkfiles(self):
+        """Called on upgrade-charm or on any config change
+        """
         if not self.is_rally_enabled:
             return
 
@@ -491,28 +564,39 @@ class OSCHelper():
         rally_script = os.path.join(hookenv.charm_dir(), 'files', 'run_rally.py')
         host.rsync(rally_script, self.scripts_dir, options=['--executability'])
 
-        ostestsfile = os.path.join('/home', self._rallyuser, 'ostests.txt')
+        ostestsfile = os.path.join(self.rally_home, 'ostests.txt')
         render(source='ostests.txt.j2', target=ostestsfile,
                context=self._get_rally_checks_context(),
-               owner=self._rallyuser, group=self._rallyuser)
+               owner=self.rallyuser, group=self.rallyuser)
+
+        cron_script = os.path.join(self.scripts_dir, 'run_rally.sh')
+        cmd = os.path.join(self.scripts_dir, 'run_rally.py')
 
+        # Bash script has ENVVARS + python script call
         proxy_settings = hookenv.env_proxy_settings()
+        content = ''
         if proxy_settings:
-            content = '\n'.join(['{}={}'.format(proxy_var, proxy_var_val)
+            content = '\n'.join(['export {}={}'.format(proxy_var, proxy_var_val)
                                  for proxy_var, proxy_var_val in proxy_settings.items()])
-        else:
-            content = ''
 
+        content += '\n{}'.format(cmd)
+        with open(cron_script, 'w') as fd:
+            fd.write('#!/bin/bash\n\n{}\n'.format(content))
+        os.chmod(cron_script, mode=0o755)
+
+        # Cronjob runs the Bash script
         context = {
             'schedule': self.rally_cron_schedule,
-            'user': self._rallyuser,
-            'cmd': os.path.join(self.scripts_dir, 'run_rally.py'),
+            'user': self.rallyuser,
+            'cmd': cron_script,
         }
-        content += '\n#\n{schedule} {user} timeout -k 840s -s SIGTERM 780s {cmd}'.format(**context)
+        cron_content = '{schedule} {user} timeout -k 840s -s SIGTERM 780s {cmd}'.format(**context)
         with open(self.rally_cron_file, 'w') as fd:
-            fd.write('# Juju generated - DO NOT EDIT\n{}\n\n'.format(content))
+            fd.write('# Juju generated - DO NOT EDIT\n{}\n\n'.format(cron_content))
 
     def configure_rally_check(self):
+        """Add the nrpe check if check-rally=true
+        """
         kv = unitdata.kv()
         if kv.get('rallyconfigured', False):
             return
@@ -528,9 +612,10 @@ class OSCHelper():
         kv.set('rallyconfigured', True)
 
     def remove_rally_check(self):
-        filename = self.rally_cron_file
-        if os.path.exists(filename):
-            os.unlink(filename)
+        """Called from deploy_rally ir check-rally=false
+        """
+        if os.path.exists(self.rally_cron_file):
+            os.unlink(self.rally_cron_file)
 
         if os.path.exists('/etc/nagios/nrpe.d/check_rally.cfg'):
             nrpe = NRPE()
@@ -538,12 +623,44 @@ class OSCHelper():
             nrpe.write()
 
     def deploy_rally(self):
+        """Installs rally or removes the nrpe check
+        """
         if self.is_rally_enabled:
+            snap.install('bsrally')
             installed = self.install_rally()
             if not installed:
                 return False
             self.configure_rally_check()
+            self.create_flavor_rally()
         else:
             self.remove_rally_check()
             unitdata.kv().set('rallyconfigured', False)
         return True
+
+    def create_flavor_rally(self):
+        """Searches for the default rally flavor (1 vcpu, 64MB RAM) or creates it
+        """
+        if not self._load_envvars:
+            return False
+        nova = os_client_config.make_client('compute', cloud='envvars')
+        ref = {'OS-FLV-EXT-DATA:ephemeral': 0,
+               'disk': 0, 'ram': 64, 'vcpus': 1,
+               'os-flavor-access:is_public': True,
+               }
+        flavor_exists = False
+        for flavor in nova.flavors.list():
+            flavor_exists = all(getattr(flavor, attr) == ref[attr] for attr in ref.keys())
+            if flavor_exists:
+                break
+
+        if flavor_exists:
+            hookenv.log('default rally flavor already exists.'
+                        ' No flavor needs to be created.')
+        else:
+            ref.update({'name': 'rally.tiny', 'ephemeral': 0, 'is_public': True})
+            del ref['OS-FLV-EXT-DATA:ephemeral']
+            del ref['os-flavor-access:is_public']
+            nova.flavors.create(**ref)
+            hookenv.log('flavor created to boot test instances via rally')
+
+        return True
diff --git a/reactive/openstack_service_checks.py b/reactive/openstack_service_checks.py
index b96269d..2ac4baa 100644
--- a/reactive/openstack_service_checks.py
+++ b/reactive/openstack_service_checks.py
@@ -19,7 +19,8 @@ import base64
 import subprocess
 
 from charmhelpers.core import hookenv, host, unitdata
-from charms.reactive import any_flags_set, clear_flag, is_flag_set, set_flag, when, when_not
+from charms.reactive import any_flags_set, clear_flag, set_flag, when, when_not
+from charms.layer import snap
 
 from lib_openstack_service_checks import (
     OSCHelper,
@@ -31,8 +32,21 @@ CERT_FILE = '/usr/local/share/ca-certificates/openstack-service-checks.crt'
 helper = OSCHelper()
 
 
+@when('identity-credentials.available')
+@when('nrpe-external-master.available')
 @when('config.changed')
 def config_changed():
+    # Update Rally scripts and tempest config, if applicable
+    if helper.is_rally_enabled and helper.is_rally_installed:
+        if any_flags_set('config.changed.rally-skip-services',
+                         'config.changed.rally-cron-schedule'):
+            helper.update_rally_checkfiles()
+
+        if any_flags_set('config.changed.rally-flavor-id',
+                         'config.changed.rally-image-id'):
+            helper.regenerate_tempest_conf()
+
+    # Update Keystone related config
     clear_flag('openstack-service-checks.configured')
 
 
@@ -162,6 +176,9 @@ def render_config():
             with open(CERT_FILE, 'w') as fd:
                 fd.write(cert_content)
             subprocess.call(['/usr/sbin/update-ca-certificates'])
+            creds['cacert'] = CERT_FILE
+            if helper.is_rally_enabled and helper.is_rally_installed:
+                helper.propagate_rally_cacert(CERT_FILE)
 
         except subprocess.CalledProcessError as error:
             block_tls_failure(error)
@@ -220,27 +237,6 @@ def do_restart():
     set_flag('openstack-service-checks.started')
 
 
-@when('nrpe-external-master.available')
-def do_reconfigure_nrpe():
-    os_credentials_flag = 'config.changed.os-credentials'
-    flags = ['config.changed.check_{}_urls'.format(interface) for interface in ['admin', 'internal', 'public']]
-    flags.extend(os_credentials_flag)
-
-    if is_flag_set('config.changed'):
-        clear_flag('openstack-service-checks.configured')
-
-    if any_flags_set(*flags):
-        if is_flag_set(os_credentials_flag):
-            clear_flag('openstack-service-checks.configured')
-        clear_flag('openstack-service-checks.endpoints.configured')
-
-    if helper.is_rally_enabled:
-        helper.reconfigure_tempest()
-
-        if is_flag_set('config.changed.skip-rally'):
-            helper.update_rally_checkfiles()
-
-
 @when_not('nrpe-external-master.available')
 def missing_nrpe():
     """Avoid a user action to be missed or overwritten by another hook
@@ -274,9 +270,12 @@ def parse_hooks():
         if old_creds:
             kv.unset('keystone-relation-creds')
 
+        # Remove the rally snap (deprecated)
+        snapname = 'fcbtest'
+        if snap.is_installed(snapname):
+            snap.remove(snapname)
+            helper.deploy_rally()
+
         # update rally check files and plugins, which may have changed
         helper.update_plugins()
         helper.update_rally_checkfiles()
-
-        # render configs again
-        do_reconfigure_nrpe()
diff --git a/requirements.txt b/requirements.txt
index 28ecaca..29c95c2 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -1,2 +1,3 @@
 flake8
+os-client-config
 pytest
diff --git a/templates/extra_config.conf.j2 b/templates/extra_config.conf.j2
new file mode 100644
index 0000000..9f15bc1
--- /dev/null
+++ b/templates/extra_config.conf.j2
@@ -0,0 +1,12 @@
+{% if image_ref or flavor_ref -%}
+[compute]
+{%- endif %}
+{%- if image_ref %}
+image_ref = {{image_ref}}
+{%- endif %}
+{%- if flavor_ref %}
+flavor_ref = {{flavor_ref}}
+{%- endif %}
+
+[volume]
+catalog_type = volumev3
diff --git a/templates/rally.conf.j2 b/templates/rally.conf.j2
new file mode 100644
index 0000000..aa8b07a
--- /dev/null
+++ b/templates/rally.conf.j2
@@ -0,0 +1,756 @@
+[DEFAULT]
+
+#
+# From oslo.log
+#
+
+# If set to true, the logging level will be set to DEBUG instead of
+# the default INFO level. (boolean value)
+# Note: This option can be changed without restarting.
+#debug = false
+
+# DEPRECATED: If set to false, the logging level will be set to
+# WARNING instead of the default INFO level. (boolean value)
+# This option is deprecated for removal.
+# Its value may be silently ignored in the future.
+#verbose = true
+
+# The name of a logging configuration file. This file is appended to
+# any existing logging configuration files. For details about logging
+# configuration files, see the Python logging module documentation.
+# Note that when logging configuration files are used then all logging
+# configuration is set in the configuration file and other logging
+# configuration options are ignored (for example,
+# logging_context_format_string). (string value)
+# Note: This option can be changed without restarting.
+# Deprecated group/name - [DEFAULT]/log_config
+#log_config_append = <None>
+
+# Defines the format string for %%(asctime)s in log records. Default:
+# %(default)s . This option is ignored if log_config_append is set.
+# (string value)
+#log_date_format = %Y-%m-%d %H:%M:%S
+
+# (Optional) Name of log file to send logging output to. If no default
+# is set, logging will go to stderr as defined by use_stderr. This
+# option is ignored if log_config_append is set. (string value)
+# Deprecated group/name - [DEFAULT]/logfile
+#log_file = <None>
+log_file = rally.log
+
+# (Optional) The base directory used for relative log_file  paths.
+# This option is ignored if log_config_append is set. (string value)
+# Deprecated group/name - [DEFAULT]/logdir
+#log_dir = <None>
+log_dir = /home/nagiososc
+
+# Uses logging handler designed to watch file system. When log file is
+# moved or removed this handler will open a new log file with
+# specified path instantaneously. It makes sense only if log_file
+# option is specified and Linux platform is used. This option is
+# ignored if log_config_append is set. (boolean value)
+#watch_log_file = false
+
+# Use syslog for logging. Existing syslog format is DEPRECATED and
+# will be changed later to honor RFC5424. This option is ignored if
+# log_config_append is set. (boolean value)
+#use_syslog = false
+
+# Syslog facility to receive log lines. This option is ignored if
+# log_config_append is set. (string value)
+#syslog_log_facility = LOG_USER
+
+# Log output to standard error. This option is ignored if
+# log_config_append is set. (boolean value)
+#use_stderr = true
+
+# Format string to use for log messages with context. (string value)
+#logging_context_format_string = %(asctime)s.%(msecs)03d %(process)d %(levelname)s %(name)s [%(request_id)s %(user_identity)s] %(instance)s%(message)s
+
+# Format string to use for log messages when context is undefined.
+# (string value)
+#logging_default_format_string = %(asctime)s.%(msecs)03d %(process)d %(levelname)s %(name)s [-] %(instance)s%(message)s
+
+# Additional data to append to log message when logging level for the
+# message is DEBUG. (string value)
+#logging_debug_format_suffix = %(funcName)s %(pathname)s:%(lineno)d
+
+# Prefix each line of exception output with this format. (string
+# value)
+#logging_exception_prefix = %(asctime)s.%(msecs)03d %(process)d ERROR %(name)s %(instance)s
+
+# Defines the format string for %(user_identity)s that is used in
+# logging_context_format_string. (string value)
+#logging_user_identity_format = %(user)s %(tenant)s %(domain)s %(user_domain)s %(project_domain)s
+
+# List of package logging levels in logger=LEVEL pairs. This option is
+# ignored if log_config_append is set. (list value)
+#default_log_levels = amqp=WARN,amqplib=WARN,boto=WARN,qpid=WARN,sqlalchemy=WARN,suds=INFO,oslo.messaging=INFO,iso8601=WARN,requests.packages.urllib3.connectionpool=WARN,urllib3.connectionpool=WARN,websocket=WARN,requests.packages.urllib3.util.retry=WARN,urllib3.util.retry=WARN,keystonemiddleware=WARN,routes.middleware=WARN,stevedore=WARN,taskflow=WARN,keystoneauth=WARN,oslo.cache=INFO,dogpile.core.dogpile=INFO
+
+# Enables or disables publication of error events. (boolean value)
+#publish_errors = false
+
+# The format for an instance that is passed with the log message.
+# (string value)
+#instance_format = "[instance: %(uuid)s] "
+
+# The format for an instance UUID that is passed with the log message.
+# (string value)
+#instance_uuid_format = "[instance: %(uuid)s] "
+
+# Enables or disables fatal status of deprecations. (boolean value)
+#fatal_deprecations = false
+
+#
+# From rally
+#
+
+# Print debugging output only for Rally. Off-site components stay
+# quiet. (boolean value)
+#rally_debug = false
+
+# HTTP timeout for any of OpenStack service in seconds (floating point
+# value)
+#openstack_client_http_timeout = 180.0
+
+# Size of raw result chunk in iterations (integer value)
+# Minimum value: 1
+#raw_result_chunk_size = 1000
+
+
+[benchmark]
+
+#
+# From rally
+#
+
+# Time to sleep after creating a resource before polling for it status
+# (floating point value)
+#cinder_volume_create_prepoll_delay = 2.0
+
+# Time to wait for cinder volume to be created. (floating point value)
+#cinder_volume_create_timeout = 600.0
+
+# Interval between checks when waiting for volume creation. (floating
+# point value)
+#cinder_volume_create_poll_interval = 2.0
+
+# Time to wait for cinder volume to be deleted. (floating point value)
+#cinder_volume_delete_timeout = 600.0
+
+# Interval between checks when waiting for volume deletion. (floating
+# point value)
+#cinder_volume_delete_poll_interval = 2.0
+
+# Time to wait for cinder backup to be restored. (floating point
+# value)
+#cinder_backup_restore_timeout = 600.0
+
+# Interval between checks when waiting for backup restoring. (floating
+# point value)
+#cinder_backup_restore_poll_interval = 2.0
+
+# Time to sleep after boot before polling for status (floating point
+# value)
+#ec2_server_boot_prepoll_delay = 1.0
+
+# Server boot timeout (floating point value)
+#ec2_server_boot_timeout = 300.0
+
+# Server boot poll interval (floating point value)
+#ec2_server_boot_poll_interval = 1.0
+
+# Time to sleep after creating a resource before polling for it status
+# (floating point value)
+#glance_image_create_prepoll_delay = 2.0
+
+# Time to wait for glance image to be created. (floating point value)
+#glance_image_create_timeout = 120.0
+
+# Interval between checks when waiting for image creation. (floating
+# point value)
+#glance_image_create_poll_interval = 1.0
+
+# Time(in sec) to sleep after creating a resource before polling for
+# it status. (floating point value)
+#heat_stack_create_prepoll_delay = 2.0
+
+# Time(in sec) to wait for heat stack to be created. (floating point
+# value)
+#heat_stack_create_timeout = 3600.0
+
+# Time interval(in sec) between checks when waiting for stack
+# creation. (floating point value)
+#heat_stack_create_poll_interval = 1.0
+
+# Time(in sec) to wait for heat stack to be deleted. (floating point
+# value)
+#heat_stack_delete_timeout = 3600.0
+
+# Time interval(in sec) between checks when waiting for stack
+# deletion. (floating point value)
+#heat_stack_delete_poll_interval = 1.0
+
+# Time(in sec) to wait for stack to be checked. (floating point value)
+#heat_stack_check_timeout = 3600.0
+
+# Time interval(in sec) between checks when waiting for stack
+# checking. (floating point value)
+#heat_stack_check_poll_interval = 1.0
+
+# Time(in sec) to sleep after updating a resource before polling for
+# it status. (floating point value)
+#heat_stack_update_prepoll_delay = 2.0
+
+# Time(in sec) to wait for stack to be updated. (floating point value)
+#heat_stack_update_timeout = 3600.0
+
+# Time interval(in sec) between checks when waiting for stack update.
+# (floating point value)
+#heat_stack_update_poll_interval = 1.0
+
+# Time(in sec) to wait for stack to be suspended. (floating point
+# value)
+#heat_stack_suspend_timeout = 3600.0
+
+# Time interval(in sec) between checks when waiting for stack suspend.
+# (floating point value)
+#heat_stack_suspend_poll_interval = 1.0
+
+# Time(in sec) to wait for stack to be resumed. (floating point value)
+#heat_stack_resume_timeout = 3600.0
+
+# Time interval(in sec) between checks when waiting for stack resume.
+# (floating point value)
+#heat_stack_resume_poll_interval = 1.0
+
+# Time(in sec) to wait for stack snapshot to be created. (floating
+# point value)
+#heat_stack_snapshot_timeout = 3600.0
+
+# Time interval(in sec) between checks when waiting for stack snapshot
+# to be created. (floating point value)
+#heat_stack_snapshot_poll_interval = 1.0
+
+# Time(in sec) to wait for stack to be restored from snapshot.
+# (floating point value)
+#heat_stack_restore_timeout = 3600.0
+
+# Time interval(in sec) between checks when waiting for stack to be
+# restored. (floating point value)
+#heat_stack_restore_poll_interval = 1.0
+
+# Time (in sec) to wait for stack to scale up or down. (floating point
+# value)
+#heat_stack_scale_timeout = 3600.0
+
+# Time interval (in sec) between checks when waiting for a stack to
+# scale up or down. (floating point value)
+#heat_stack_scale_poll_interval = 1.0
+
+# Interval(in sec) between checks when waiting for node creation.
+# (floating point value)
+#ironic_node_create_poll_interval = 1.0
+
+# Time(in sec) to sleep after creating a resource before polling for
+# the status. (floating point value)
+#magnum_cluster_create_prepoll_delay = 5.0
+
+# Time(in sec) to wait for magnum cluster to be created. (floating
+# point value)
+#magnum_cluster_create_timeout = 1200.0
+
+# Time interval(in sec) between checks when waiting for cluster
+# creation. (floating point value)
+#magnum_cluster_create_poll_interval = 1.0
+
+# Delay between creating Manila share and polling for its status.
+# (floating point value)
+#manila_share_create_prepoll_delay = 2.0
+
+# Timeout for Manila share creation. (floating point value)
+#manila_share_create_timeout = 300.0
+
+# Interval between checks when waiting for Manila share creation.
+# (floating point value)
+#manila_share_create_poll_interval = 3.0
+
+# Timeout for Manila share deletion. (floating point value)
+#manila_share_delete_timeout = 180.0
+
+# Interval between checks when waiting for Manila share deletion.
+# (floating point value)
+#manila_share_delete_poll_interval = 2.0
+
+# mistral execution timeout (integer value)
+#mistral_execution_timeout = 200
+
+# Delay between creating Monasca metrics and polling for its elements.
+# (floating point value)
+#monasca_metric_create_prepoll_delay = 15.0
+
+# A timeout in seconds for an environment deploy (integer value)
+# Deprecated group/name - [benchmark]/deploy_environment_timeout
+#murano_deploy_environment_timeout = 1200
+
+# Deploy environment check interval in seconds (integer value)
+# Deprecated group/name - [benchmark]/deploy_environment_check_interval
+#murano_deploy_environment_check_interval = 5
+
+# Time to sleep after start before polling for status (floating point
+# value)
+#nova_server_start_prepoll_delay = 0.0
+
+# Server start timeout (floating point value)
+#nova_server_start_timeout = 300.0
+
+# Server start poll interval (floating point value)
+#nova_server_start_poll_interval = 1.0
+
+# Time to sleep after stop before polling for status (floating point
+# value)
+#nova_server_stop_prepoll_delay = 0.0
+
+# Server stop timeout (floating point value)
+#nova_server_stop_timeout = 300.0
+
+# Server stop poll interval (floating point value)
+#nova_server_stop_poll_interval = 2.0
+
+# Time to sleep after boot before polling for status (floating point
+# value)
+#nova_server_boot_prepoll_delay = 1.0
+
+# Server boot timeout (floating point value)
+#nova_server_boot_timeout = 300.0
+
+# Server boot poll interval (floating point value)
+#nova_server_boot_poll_interval = 1.0
+
+# Time to sleep after delete before polling for status (floating point
+# value)
+#nova_server_delete_prepoll_delay = 2.0
+
+# Server delete timeout (floating point value)
+#nova_server_delete_timeout = 300.0
+
+# Server delete poll interval (floating point value)
+#nova_server_delete_poll_interval = 2.0
+
+# Time to sleep after reboot before polling for status (floating point
+# value)
+#nova_server_reboot_prepoll_delay = 2.0
+
+# Server reboot timeout (floating point value)
+#nova_server_reboot_timeout = 300.0
+
+# Server reboot poll interval (floating point value)
+#nova_server_reboot_poll_interval = 2.0
+
+# Time to sleep after rebuild before polling for status (floating
+# point value)
+#nova_server_rebuild_prepoll_delay = 1.0
+
+# Server rebuild timeout (floating point value)
+#nova_server_rebuild_timeout = 300.0
+
+# Server rebuild poll interval (floating point value)
+#nova_server_rebuild_poll_interval = 1.0
+
+# Time to sleep after rescue before polling for status (floating point
+# value)
+#nova_server_rescue_prepoll_delay = 2.0
+
+# Server rescue timeout (floating point value)
+#nova_server_rescue_timeout = 300.0
+
+# Server rescue poll interval (floating point value)
+#nova_server_rescue_poll_interval = 2.0
+
+# Time to sleep after unrescue before polling for status (floating
+# point value)
+#nova_server_unrescue_prepoll_delay = 2.0
+
+# Server unrescue timeout (floating point value)
+#nova_server_unrescue_timeout = 300.0
+
+# Server unrescue poll interval (floating point value)
+#nova_server_unrescue_poll_interval = 2.0
+
+# Time to sleep after suspend before polling for status (floating
+# point value)
+#nova_server_suspend_prepoll_delay = 2.0
+
+# Server suspend timeout (floating point value)
+#nova_server_suspend_timeout = 300.0
+
+# Server suspend poll interval (floating point value)
+#nova_server_suspend_poll_interval = 2.0
+
+# Time to sleep after resume before polling for status (floating point
+# value)
+#nova_server_resume_prepoll_delay = 2.0
+
+# Server resume timeout (floating point value)
+#nova_server_resume_timeout = 300.0
+
+# Server resume poll interval (floating point value)
+#nova_server_resume_poll_interval = 2.0
+
+# Time to sleep after pause before polling for status (floating point
+# value)
+#nova_server_pause_prepoll_delay = 2.0
+
+# Server pause timeout (floating point value)
+#nova_server_pause_timeout = 300.0
+
+# Server pause poll interval (floating point value)
+#nova_server_pause_poll_interval = 2.0
+
+# Time to sleep after unpause before polling for status (floating
+# point value)
+#nova_server_unpause_prepoll_delay = 2.0
+
+# Server unpause timeout (floating point value)
+#nova_server_unpause_timeout = 300.0
+
+# Server unpause poll interval (floating point value)
+#nova_server_unpause_poll_interval = 2.0
+
+# Time to sleep after shelve before polling for status (floating point
+# value)
+#nova_server_shelve_prepoll_delay = 2.0
+
+# Server shelve timeout (floating point value)
+#nova_server_shelve_timeout = 300.0
+
+# Server shelve poll interval (floating point value)
+#nova_server_shelve_poll_interval = 2.0
+
+# Time to sleep after unshelve before polling for status (floating
+# point value)
+#nova_server_unshelve_prepoll_delay = 2.0
+
+# Server unshelve timeout (floating point value)
+#nova_server_unshelve_timeout = 300.0
+
+# Server unshelve poll interval (floating point value)
+#nova_server_unshelve_poll_interval = 2.0
+
+# Time to sleep after image_create before polling for status (floating
+# point value)
+#nova_server_image_create_prepoll_delay = 0.0
+
+# Server image_create timeout (floating point value)
+#nova_server_image_create_timeout = 300.0
+
+# Server image_create poll interval (floating point value)
+#nova_server_image_create_poll_interval = 2.0
+
+# Time to sleep after image_delete before polling for status (floating
+# point value)
+#nova_server_image_delete_prepoll_delay = 0.0
+
+# Server image_delete timeout (floating point value)
+#nova_server_image_delete_timeout = 300.0
+
+# Server image_delete poll interval (floating point value)
+#nova_server_image_delete_poll_interval = 2.0
+
+# Time to sleep after resize before polling for status (floating point
+# value)
+#nova_server_resize_prepoll_delay = 2.0
+
+# Server resize timeout (floating point value)
+#nova_server_resize_timeout = 400.0
+
+# Server resize poll interval (floating point value)
+#nova_server_resize_poll_interval = 5.0
+
+# Time to sleep after resize_confirm before polling for status
+# (floating point value)
+#nova_server_resize_confirm_prepoll_delay = 0.0
+
+# Server resize_confirm timeout (floating point value)
+#nova_server_resize_confirm_timeout = 200.0
+
+# Server resize_confirm poll interval (floating point value)
+#nova_server_resize_confirm_poll_interval = 2.0
+
+# Time to sleep after resize_revert before polling for status
+# (floating point value)
+#nova_server_resize_revert_prepoll_delay = 0.0
+
+# Server resize_revert timeout (floating point value)
+#nova_server_resize_revert_timeout = 200.0
+
+# Server resize_revert poll interval (floating point value)
+#nova_server_resize_revert_poll_interval = 2.0
+
+# Time to sleep after live_migrate before polling for status (floating
+# point value)
+#nova_server_live_migrate_prepoll_delay = 1.0
+
+# Server live_migrate timeout (floating point value)
+#nova_server_live_migrate_timeout = 400.0
+
+# Server live_migrate poll interval (floating point value)
+#nova_server_live_migrate_poll_interval = 2.0
+
+# Time to sleep after migrate before polling for status (floating
+# point value)
+#nova_server_migrate_prepoll_delay = 1.0
+
+# Server migrate timeout (floating point value)
+#nova_server_migrate_timeout = 400.0
+
+# Server migrate poll interval (floating point value)
+#nova_server_migrate_poll_interval = 2.0
+
+# Nova volume detach timeout (floating point value)
+#nova_detach_volume_timeout = 200.0
+
+# Nova volume detach poll interval (floating point value)
+#nova_detach_volume_poll_interval = 2.0
+
+# A timeout in seconds for a cluster create operation (integer value)
+# Deprecated group/name - [benchmark]/cluster_create_timeout
+#sahara_cluster_create_timeout = 1800
+
+# A timeout in seconds for a cluster delete operation (integer value)
+# Deprecated group/name - [benchmark]/cluster_delete_timeout
+#sahara_cluster_delete_timeout = 900
+
+# Cluster status polling interval in seconds (integer value)
+# Deprecated group/name - [benchmark]/cluster_check_interval
+#sahara_cluster_check_interval = 5
+
+# A timeout in seconds for a Job Execution to complete (integer value)
+# Deprecated group/name - [benchmark]/job_execution_timeout
+#sahara_job_execution_timeout = 600
+
+# Job Execution status polling interval in seconds (integer value)
+# Deprecated group/name - [benchmark]/job_check_interval
+#sahara_job_check_interval = 5
+
+# Amount of workers one proxy should serve to. (integer value)
+#sahara_workers_per_proxy = 20
+
+# Interval between checks when waiting for a VM to become pingable
+# (floating point value)
+#vm_ping_poll_interval = 1.0
+
+# Time to wait for a VM to become pingable (floating point value)
+#vm_ping_timeout = 120.0
+
+# Watcher audit launch interval (floating point value)
+#watcher_audit_launch_poll_interval = 2.0
+
+# Watcher audit launch timeout (integer value)
+#watcher_audit_launch_timeout = 300
+
+
+[cleanup]
+
+#
+# From rally
+#
+
+# A timeout in seconds for deleting resources (integer value)
+#resource_deletion_timeout = 600
+
+# Number of cleanup threads to run (integer value)
+#cleanup_threads = 20
+
+
+[database]
+
+#
+# From oslo.db
+#
+
+# DEPRECATED: The file name to use with SQLite. (string value)
+# Deprecated group/name - [DEFAULT]/sqlite_db
+# This option is deprecated for removal.
+# Its value may be silently ignored in the future.
+# Reason: Should use config option connection or slave_connection to
+# connect the database.
+#sqlite_db = oslo.sqlite
+
+# If True, SQLite uses synchronous mode. (boolean value)
+# Deprecated group/name - [DEFAULT]/sqlite_synchronous
+#sqlite_synchronous = true
+
+# The back end to use for the database. (string value)
+# Deprecated group/name - [DEFAULT]/db_backend
+#backend = sqlalchemy
+
+# The SQLAlchemy connection string to use to connect to the database.
+# (string value)
+# Deprecated group/name - [DEFAULT]/sql_connection
+# Deprecated group/name - [DATABASE]/sql_connection
+# Deprecated group/name - [sql]/connection
+connection=sqlite:////home/nagiososc/snap/bsrally/common/rally.db
+
+# The SQLAlchemy connection string to use to connect to the slave
+# database. (string value)
+#slave_connection = <None>
+
+# The SQL mode to be used for MySQL sessions. This option, including
+# the default, overrides any server-set SQL mode. To use whatever SQL
+# mode is set by the server configuration, set this to no value.
+# Example: mysql_sql_mode= (string value)
+#mysql_sql_mode = TRADITIONAL
+
+# Timeout before idle SQL connections are reaped. (integer value)
+# Deprecated group/name - [DEFAULT]/sql_idle_timeout
+# Deprecated group/name - [DATABASE]/sql_idle_timeout
+# Deprecated group/name - [sql]/idle_timeout
+#idle_timeout = 3600
+
+# Minimum number of SQL connections to keep open in a pool. (integer
+# value)
+# Deprecated group/name - [DEFAULT]/sql_min_pool_size
+# Deprecated group/name - [DATABASE]/sql_min_pool_size
+#min_pool_size = 1
+
+# Maximum number of SQL connections to keep open in a pool. Setting a
+# value of 0 indicates no limit. (integer value)
+# Deprecated group/name - [DEFAULT]/sql_max_pool_size
+# Deprecated group/name - [DATABASE]/sql_max_pool_size
+#max_pool_size = 5
+
+# Maximum number of database connection retries during startup. Set to
+# -1 to specify an infinite retry count. (integer value)
+# Deprecated group/name - [DEFAULT]/sql_max_retries
+# Deprecated group/name - [DATABASE]/sql_max_retries
+#max_retries = 10
+
+# Interval between retries of opening a SQL connection. (integer
+# value)
+# Deprecated group/name - [DEFAULT]/sql_retry_interval
+# Deprecated group/name - [DATABASE]/reconnect_interval
+#retry_interval = 10
+
+# If set, use this value for max_overflow with SQLAlchemy. (integer
+# value)
+# Deprecated group/name - [DEFAULT]/sql_max_overflow
+# Deprecated group/name - [DATABASE]/sqlalchemy_max_overflow
+#max_overflow = 50
+
+# Verbosity of SQL debugging information: 0=None, 100=Everything.
+# (integer value)
+# Minimum value: 0
+# Maximum value: 100
+# Deprecated group/name - [DEFAULT]/sql_connection_debug
+#connection_debug = 0
+
+# Add Python stack traces to SQL as comment strings. (boolean value)
+# Deprecated group/name - [DEFAULT]/sql_connection_trace
+#connection_trace = false
+
+# If set, use this value for pool_timeout with SQLAlchemy. (integer
+# value)
+# Deprecated group/name - [DATABASE]/sqlalchemy_pool_timeout
+#pool_timeout = <None>
+
+# Enable the experimental use of database reconnect on connection
+# lost. (boolean value)
+#use_db_reconnect = false
+
+# Seconds between retries of a database transaction. (integer value)
+#db_retry_interval = 1
+
+# If True, increases the interval between retries of a database
+# operation up to db_max_retry_interval. (boolean value)
+#db_inc_retry_interval = true
+
+# If db_inc_retry_interval is set, the maximum seconds between retries
+# of a database operation. (integer value)
+#db_max_retry_interval = 10
+
+# Maximum retries in case of connection error or deadlock error before
+# error is raised. Set to -1 to specify an infinite retry count.
+# (integer value)
+#db_max_retries = 20
+
+
+[roles_context]
+
+#
+# From rally
+#
+
+# How many concurrent threads to use for serving roles context
+# (integer value)
+#resource_management_workers = 30
+
+
+[tempest]
+
+#
+# From rally
+#
+
+# image URL (string value)
+#img_url = http://download.cirros-cloud.net/0.3.5/cirros-0.3.5-x86_64-disk.img
+
+# Image disk format to use when creating the image (string value)
+#img_disk_format = qcow2
+
+# Image container format to use when creating the image (string value)
+#img_container_format = bare
+
+# Regular expression for name of a public image to discover it in the
+# cloud and use it for the tests. Note that when Rally is searching
+# for the image, case insensitive matching is performed. Specify
+# nothing ('img_name_regex =') if you want to disable discovering. In
+# this case Rally will create needed resources by itself if the values
+# for the corresponding config options are not specified in the
+# Tempest config file (string value)
+#img_name_regex = ^.*(cirros|testvm).*$
+
+# Role required for users to be able to create Swift containers
+# (string value)
+#swift_operator_role = Member
+
+# User role that has reseller admin (string value)
+#swift_reseller_admin_role = ResellerAdmin
+
+# Role required for users to be able to manage Heat stacks (string
+# value)
+#heat_stack_owner_role = heat_stack_owner
+
+# Role for Heat template-defined users (string value)
+#heat_stack_user_role = heat_stack_user
+
+# Primary flavor RAM size used by most of the test cases (integer
+# value)
+#flavor_ref_ram = 64
+
+# Alternate reference flavor RAM size used by test that need two
+# flavors, like those that resize an instance (integer value)
+#flavor_ref_alt_ram = 128
+
+# RAM size flavor used for orchestration test cases (integer value)
+#heat_instance_type_ram = 64
+
+
+[users_context]
+
+#
+# From rally
+#
+
+# How many concurrent threads use for serving users context (integer
+# value)
+#resource_management_workers = 20
+
+# ID of domain in which projects will be created. (string value)
+#project_domain = default
+
+# ID of domain in which users will be created. (string value)
+#user_domain = default
+
+# The default role name of the keystone. (string value)
+#keystone_default_role = member
diff --git a/tests/unit/conftest.py b/tests/unit/conftest.py
index 32a4af9..ce38155 100644
--- a/tests/unit/conftest.py
+++ b/tests/unit/conftest.py
@@ -14,13 +14,10 @@ def mock_layers(monkeypatch):
 
     def options(layer):
         # mock options for layers here
-        if layer == 'example-layer':
-            options = {'port': 9999}
-            return options
-        else:
+        if layer == 'install':
             return None
 
-    monkeypatch.setattr('lib_openstack_service_checks.layer.options', options)
+    monkeypatch.setattr('lib_openstack_service_checks.snap', options)
 
 
 @pytest.fixture
@@ -66,7 +63,7 @@ def mock_unitdata_keystonecreds(monkeypatch):
 
 
 @pytest.fixture
-def openstackservicechecks(tmpdir, mock_hookenv_config, mock_charm_dir, monkeypatch):
+def openstackservicechecks(tmpdir, mock_layers, mock_hookenv_config, mock_charm_dir, monkeypatch):
     from lib_openstack_service_checks import OSCHelper
     helper = OSCHelper()
 
diff --git a/tests/unit/test_lib.py b/tests/unit/test_lib.py
index fba093f..6f5a328 100644
--- a/tests/unit/test_lib.py
+++ b/tests/unit/test_lib.py
@@ -11,7 +11,9 @@ def test_openstackservicechecks_common_properties(openstackservicechecks):
     assert openstackservicechecks.skip_disabled == ''
     assert openstackservicechecks.check_dns == ''
     assert not openstackservicechecks.is_rally_enabled
-    assert not openstackservicechecks.skipped_rally_checks
+    assert not openstackservicechecks.rally_skipped_checks
+    assert not openstackservicechecks.rally_image_id
+    assert not openstackservicechecks.rally_flavor_id
     assert openstackservicechecks.rally_cron_schedule == '*/15 * * * *'
 
 
@@ -47,15 +49,15 @@ def test_openstackservicechecks_get_keystone_credentials_oscredentials(
     assert openstackservicechecks.get_os_credentials() == expected
 
 
[e-mail]('skip_rally,result', [
[e-mail]('rally_skip_services,result', [
     ('nova,neutron', [True, True, False, False]),
     ('cinder,neutron', [False, True, True, False]),
     ('glance', [True, False, True, True]),
     ('nova neutron', [True, True, True, True]),  # needs to be comma-separated
     ('', [True, True, True, True]),
 ])
-def test_get_rally_checks_context(skip_rally, result, openstackservicechecks):
-    openstackservicechecks.charm_config['skip-rally'] = skip_rally
+def test_get_rally_checks_context(rally_skip_services, result, openstackservicechecks):
+    openstackservicechecks.charm_config['rally-skip-services'] = rally_skip_services
     expected = {comp: result[num]
                 for num, comp in enumerate('cinder glance nova neutron'.split())}
     assert openstackservicechecks._get_rally_checks_context() == expected
diff --git a/wheelhouse.txt b/wheelhouse.txt
new file mode 100644
index 0000000..7fe681d
--- /dev/null
+++ b/wheelhouse.txt
@@ -0,0 +1 @@
+pid