Merge ~canonical-is-bootstack/charm-grafana:aluria/add-tests-lint into charm-grafana:master

[Alvaro Uria]

Functional tests support on multiple versions

* OpenStack: Xenial-Queens, Bionic-Queens and Bionic-Stein
* Ubuntu series in Grafana: Xenial and Bionic
* Install methods: apt, snap and random wrong method
* Checks: Juju status messages, and imported dashboards exist

[🤖 Canonical IS Merge Bot (canonical-is-mergebot)]

This merge proposal is being monitored by mergebot. Change the status to Approved to merge.

[🤖 Canonical IS Merge Bot (canonical-is-mergebot)]

Unable to determine commit message from repository - please click "Set commit message" and enter the commit message manually.

[Alvaro Uria (aluria)]

* pytest fixture scopes
  - module scope used on "model" and "deploy_openstack"
  - function scope used on "deploy_app"

  With the above, I can create a model per Ubuntu OpenStack version we
  want to test on, and deploy openstack (setting openstack-origin or
  source by rendering a bundle.yaml.j2).

  Then, on the "deploy_app" function-scoped fixture, I can deploy each
  permutation of series and install method supported by the charm,
  relating it to prometheus via an rendered overlay.yaml.j2 config file.

  For now, we check that "juju status" is the expected one per install
  method, and we verify that automatically imported dashboards (the ones
  that start with "juju-") exist.

* There are no unit tests because the charm is coded in a single
reactive script. There is a bug to rewrite the charm following
template-python-pytest that includes a 1y old rewrite (with unit tests,
and these same functional tests with no support for multiple OpenStack
versions testing)

Rest of changes happened after I rebased from master and were not intended. I have also checked the previous MP to fix [1] and it doesn't work. There will be another MP after this one is merged.

1. https://bugs.launchpad.net/charm-grafana/+bug/1858490

[Alvaro Uria (aluria)]

BTW, my func tests run:
* https://pastebin.canonical.com/p/8NmqJK7jVG/
* https://pastebin.canonical.com/p/FbfvMmCtGX/

[Paul Goins (vultaire)]

This looks good. And yes, I read everything which wasn't from template-python-pytest.

My only real concern is regarding the use of JUJU_REPOSITORY in the Makefile, and the lack of a default. I'd like to see either a default added or a mention that JUJU_REPOSITORY must be defined. (Of course, ideal would be to replace JUJU_REPOSITORY with CHARM_BUILD_DIR.)

This said, I don't think this needs to be fixed "now". Other than that, it basically looks OK. Thus, I'm marking this as approved.

review: Approve

[Alvaro Uria (aluria)]

Thank you for the review Paul. I agree that in the long term, we want to use the new charm-tools env vars (CHARM_BUILD_DIR is one of them). I used the old env var because template-python-pytest still uses it and wanted to try and keep all charms as uniform as possible.

I've also removed the unused custom fixtures "app" and "units". Thanks again.

[🤖 Canonical IS Merge Bot (canonical-is-mergebot)]

Change successfully merged at revision 93a72a956e90762f95c9f76be915126286e4afe7

diff --git a/.gitignore b/.gitignore
index 88f01bf..32e2995 100644
--- a/.gitignore
+++ b/.gitignore
@@ -1,13 +1,22 @@
-*.swp
-*~
-.idea
-.tox
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# Log files
+*.log
+
+.tox/
 .coverage
-.vscode
-/builds/
-/deb_dist
-/dist
-/repo-info
-__pycache__
-/report/
 
+# vi
+.*.swp
+
+# pycharm
+.idea/
+
+# version data
+repo-info
+
+# reports
+report/*
diff --git a/Makefile b/Makefile
new file mode 100644
index 0000000..0066378
--- /dev/null
+++ b/Makefile
@@ -0,0 +1,44 @@
+help:
+	@echo "This project supports the following targets"
+	@echo ""
+	@echo " make help - show this text"
+	@echo " make lint - run flake8"
+	@echo " make test - run the lint and functional tests"
+	@echo " make functional - run the tests defined in the functional subdirectory"
+	@echo " make release - build the charm"
+	@echo " make clean - remove unneeded files"
+	@echo ""
+
+submodules:
+	@echo "Cloning submodules"
+	@git submodule update --init --recursive
+
+lint:
+	@echo "Running flake8"
+	@tox -e lint
+
+test: lint functional
+
+functional: build
+	@PYTEST_KEEP_MODEL=$(PYTEST_KEEP_MODEL) \
+	    PYTEST_CLOUD_NAME=$(PYTEST_CLOUD_NAME) \
+	    PYTEST_CLOUD_REGION=$(PYTEST_CLOUD_REGION) \
+	    tox -e functional
+
+build:
+	@echo "Building charm to base directory $(JUJU_REPOSITORY)"
+	@-git describe --tags > ./repo-info
+	@CHARM_LAYERS_DIR=./layers CHARM_INTERFACES_DIR=./interfaces TERM=linux \
+		JUJU_REPOSITORY=$(JUJU_REPOSITORY) charm build . --force
+
+release: clean build
+	@echo "Charm is built at $(JUJU_REPOSITORY)/builds"
+
+clean:
+	@echo "Cleaning files"
+	@if [ -d .tox ] ; then rm -r .tox ; fi
+	@if [ -d .pytest_cache ] ; then rm -r .pytest_cache ; fi
+	@find . -iname __pycache__ -exec rm -r {} +
+
+# The targets below don't depend on a file
+.PHONY: lint test unittest functional build release clean help
diff --git a/lib/charms/layer/grafana.py b/lib/charms/layer/grafana.py
index b482203..67b53ea 100644
--- a/lib/charms/layer/grafana.py
+++ b/lib/charms/layer/grafana.py
@@ -2,13 +2,12 @@
 
 import json
 import requests
+from charmhelpers.core import unitdata
 from charmhelpers.core.hookenv import (
     config,
     log,
 )
 
-from charmhelpers.core import unitdata
-
 
 def get_admin_password():
     kv = unitdata.kv()
@@ -30,7 +29,7 @@ def import_dashboard(dashboard, name=None):
         name = dashboard['dashboard'].get('title') or 'Untitled'
     headers = {'Content-Type': 'application/json'}
     import_url = 'http://localhost:{}/api/dashboards/db'.format(
-                  config('port'))
+                 config('port'))
     passwd = get_admin_password()
     if passwd is None:
         return (False, 'Unable to retrieve grafana password.')
diff --git a/reactive/grafana.py b/reactive/grafana.py
index 713aa8f..a34f41b 100644
--- a/reactive/grafana.py
+++ b/reactive/grafana.py
@@ -4,13 +4,11 @@ import glob
 import json
 import os
 import re
-import requests
 import shutil
-import six
 import subprocess
 import time
-from jsondiff import diff
 
+from charmhelpers import fetch
 from charmhelpers.contrib.charmsupport import nrpe
 from charmhelpers.core import (
     hookenv,
@@ -18,11 +16,9 @@ from charmhelpers.core import (
     unitdata,
 )
 from charmhelpers.core.templating import render
-from charmhelpers import fetch
-from charms.reactive.helpers import (
-    any_file_changed,
-    is_state,
-)
+
+from charms.layer import snap
+from charms.layer.grafana import import_dashboard
 from charms.reactive import (
     hook,
     remove_state,
@@ -30,11 +26,20 @@ from charms.reactive import (
     when,
     when_not,
 )
+from charms.reactive.helpers import (
+    any_file_changed,
+    is_state,
+)
 
-from charms.layer import snap
-from charms.layer.grafana import import_dashboard
 from jinja2 import Environment, FileSystemLoader, exceptions
 
+from jsondiff import diff
+
+import requests
+
+import six
+
+
 SVCNAME = {'snap': 'snap.grafana.grafana',
            'apt': 'grafana-server'}
 SNAP_NAME = 'grafana'
@@ -120,8 +125,7 @@ def install_packages():
         set_state('grafana.installed')
         hookenv.status_set('active', 'Completed installing grafana')
     elif source == 'snap' and \
-            (host.lsb_release()['DISTRIB_CODENAME'] >= 'xenial' or
-             host.lsb_release()['DISTRIB_CODENAME'] < 'p'):
+            (host.lsb_release()['DISTRIB_CODENAME'] >= 'xenial' or host.lsb_release()['DISTRIB_CODENAME'] < 'p'):
         # NOTE(aluria): precise is the last supported Ubuntu release, so
         # anything below 'p' is actually newer than xenial (systemd support)
         snap.install(SNAP_NAME, channel=channel, force_dangerous=False)
@@ -433,8 +437,9 @@ def configure_website(website):
 
 
 def validate_datasources():
-    """TODO: make sure datasources option is merged with
-    relation data
+    """Verify that datasources configuration is valid, if existing.
+
+    TODO: make sure datasources option is merged with relation data
     TODO: make sure datasources are validated
     """
     config = hookenv.config()
@@ -448,7 +453,8 @@ def validate_datasources():
 
 
 def check_datasource(ds):
-    """
+    """Check for and add datasources not currently in grafana DB.
+
     CREATE TABLE `data_source` (
     `id` INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL
     , `org_id` INTEGER NOT NULL
@@ -470,7 +476,6 @@ def check_datasource(ds):
     , `with_credentials` INTEGER NOT NULL DEFAULT 0);
     INSERT INTO "data_source" VALUES(1,1,0,'prometheus','BootStack Prometheus','proxy','http://localhost:9090','','','',0,'','',1,'{}','2016-01-22 12:11:06','2016-01-22 12:11:11',0);
     """  # noqa E501
-
     # ds will be similar to:
     # {'service_name': 'prometheus',
     #  'url': 'http://10.0.3.216:9090',
@@ -505,9 +510,8 @@ def check_datasource(ds):
 
 # This isn't exposed in charmhelpers: https://github.com/juju/charm-helpers/issues/367
 def render_custom(source, context, **parameters):
-    """
-    Renders a template from the template folder with custom environment
-    parameters.
+    """Render a template from the template folder with custom environment parameters.
+
     source: template file name to render from
     context: template context variables
     parameters: initialization parameters for the jinja Environment
@@ -750,7 +754,8 @@ def generate_query(ds, is_default, id=None):
 @when('grafana.started')
 @when_not('grafana.admin_password.set')
 def check_adminuser():
-    """
+    """Create Adminuser if not existing.
+
     CREATE TABLE `user` (
     `id` INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL
     , `version` INTEGER NOT NULL
@@ -770,7 +775,6 @@ def check_adminuser():
     );
     INSERT INTO "user" VALUES(1,0,'admin','[e-mail]','BootStack Team','309bc4e78bc60d02dc0371d9e9fa6bf9a809d5dc25c745b9e3f85c3ed49c6feccd4ffc96d1db922f4297663a209e93f7f2b6','LZeJ3nSdrC','hseJcLcnPN','',1,1,0,'light','2016-01-22 12:00:08','2016-01-22 12:02:13');
     """  # noqa E501
-
     # XXX: If you add any dependencies on config items here,
     #      be sure to update config_changed() accordingly!
 
@@ -806,9 +810,7 @@ def check_adminuser():
         query = cur.execute('SELECT id, login, salt FROM user')
         for row in query.fetchall():
             if row[1] == 'admin':
-                nagios_context = config.get('nagios_context', False)
-                if not nagios_context:
-                    nagios_context = 'UNKNOWN'
+                nagios_context = config.get('nagios_context', 'UNKNOWN')
                 email = '[e-mail]' % nagios_context
                 hpasswd = hpwgen(passwd, row[2])
                 if hpasswd:
diff --git a/requirements.txt b/requirements.txt
new file mode 100644
index 0000000..8462291
--- /dev/null
+++ b/requirements.txt
@@ -0,0 +1 @@
+# Include python requirements here
diff --git a/tests/functional/bundle.yaml.j2 b/tests/functional/bundle.yaml.j2
new file mode 100644
index 0000000..20f0672
--- /dev/null
+++ b/tests/functional/bundle.yaml.j2
@@ -0,0 +1,162 @@
+series: {{ series }}
+applications:
+  ceph-mon:
+    charm: cs:ceph-mon
+    num_units: 3
+    options:
+      expected-osd-count: 3
+      source: {{ openstack_origin }}
+  ceph-osd:
+    charm: cs:ceph-osd
+    num_units: 3
+    options:
+      osd-devices: /srv/osd
+      use-direct-io: False
+      bluestore: False
+      source: {{ openstack_origin }}
+  cinder:
+    charm: cs:cinder
+    num_units: 1
+    options:
+      worker-multiplier: 0.1
+      block-device: None
+      glance-api-version: 2
+      openstack-origin: {{ openstack_origin }}
+  cinder-ceph:
+    charm: cs:cinder-ceph
+    num_units: 0
+  glance:
+    charm: cs:glance
+    num_units: 1
+    options:
+      worker-multiplier: 0.1
+      openstack-origin: {{ openstack_origin }}
+  keystone:
+    charm: cs:keystone
+    num_units: 1
+    options:
+      worker-multiplier: 0.1
+      openstack-origin: {{ openstack_origin }}
+  mysql:
+    charm: cs:percona-cluster
+    num_units: 1
+    options:
+      max-connections: 1000
+      innodb-buffer-pool-size: 256M
+      tuning-level: fast
+      source: {{ openstack_origin }}
+  neutron-api:
+    charm: cs:neutron-api
+    num_units: 1
+    options:
+      worker-multiplier: 0.1
+      neutron-security-groups: true
+      overlay-network-type: "gre vxlan"
+      flat-network-providers: physnet1
+      openstack-origin: {{ openstack_origin }}
+  neutron-openvswitch:
+    charm: cs:neutron-openvswitch
+    num_units: 0
+  nova-cloud-controller:
+    charm: cs:nova-cloud-controller
+    num_units: 1
+    options:
+      worker-multiplier: 0.1
+      network-manager: Neutron
+      ram-allocation-ratio: '64'
+      cpu-allocation-ratio: '64'
+      openstack-origin: {{ openstack_origin }}
+  nova-compute:
+    charm: cs:nova-compute
+    num_units: 1
+    options:
+      enable-live-migration: False
+      enable-resize: False
+      migration-auth-type: ssh
+      force-raw-images: False
+      openstack-origin: {{ openstack_origin }}
+  rabbitmq-server:
+    charm: cs:rabbitmq-server
+    num_units: 1
+
+  prometheus:
+    charm: cs:prometheus2
+    num_units: 1
+    options:
+      label-juju-units: true
+  telegraf:
+    charm: cs:telegraf
+    options:
+      hostname: '{unit}'
+      prometheus_output_port: default
+  prometheus-ceph-exporter:
+    charm: cs:prometheus-ceph-exporter
+    num_units: 1
+
+relations:
+- - nova-compute:ceph-access
+  - cinder-ceph:ceph-access
+- - nova-compute:amqp
+  - rabbitmq-server:amqp
+- - keystone:shared-db
+  - mysql:shared-db
+- - nova-cloud-controller:identity-service
+  - keystone:identity-service
+- - glance:identity-service
+  - keystone:identity-service
+- - neutron-api:identity-service
+  - keystone:identity-service
+- - neutron-openvswitch:neutron-plugin-api
+  - neutron-api:neutron-plugin-api
+- - neutron-api:shared-db
+  - mysql:shared-db
+- - neutron-api:amqp
+  - rabbitmq-server:amqp
+- - glance:shared-db
+  - mysql:shared-db
+- - glance:amqp
+  - rabbitmq-server:amqp
+- - nova-cloud-controller:image-service
+  - glance:image-service
+- - nova-compute:image-service
+  - glance:image-service
+- - nova-cloud-controller:cloud-compute
+  - nova-compute:cloud-compute
+- - nova-cloud-controller:amqp
+  - rabbitmq-server:amqp
+- - nova-compute:neutron-plugin
+  - neutron-openvswitch:neutron-plugin
+- - neutron-openvswitch:amqp
+  - rabbitmq-server:amqp
+- - nova-cloud-controller:shared-db
+  - mysql:shared-db
+- - nova-cloud-controller:neutron-api
+  - neutron-api:neutron-api
+- - cinder:image-service
+  - glance:image-service
+- - cinder:amqp
+  - rabbitmq-server:amqp
+- - cinder:identity-service
+  - keystone:identity-service
+- - cinder:cinder-volume-service
+  - nova-cloud-controller:cinder-volume-service
+- - cinder-ceph:storage-backend
+  - cinder:storage-backend
+- - ceph-mon:client
+  - nova-compute:ceph
+- - cinder:shared-db
+  - mysql:shared-db
+- - ceph-mon:client
+  - cinder-ceph:ceph
+- - ceph-mon:client
+  - glance:ceph
+- - ceph-osd:mon
+  - ceph-mon:osd
+- - prometheus-ceph-exporter
+  - ceph-mon:client
+- - prometheus:target
+  - telegraf:prometheus-client
+- - prometheus:target
+  - prometheus-ceph-exporter
+- - telegraf
+  - ceph-mon
diff --git a/tests/functional/conftest.py b/tests/functional/conftest.py
new file mode 100644
index 0000000..1d2d6e9
--- /dev/null
+++ b/tests/functional/conftest.py
@@ -0,0 +1,67 @@
+#!/usr/bin/python3
+"""
+Reusable pytest fixtures for functional testing
+
+Environment variables
+---------------------
+
+PYTEST_CLOUD_REGION, PYTEST_CLOUD_NAME: cloud name and region to use for juju model creation
+
+PYTEST_KEEP_MODEL: if set, the testing model won't be torn down at the end of the testing session
+
+"""
+
+import asyncio
+import os
+import uuid
+import pytest
+import subprocess
+
+from juju.controller import Controller
+from juju_tools import JujuTools
+
+
[e-mail](scope='module')
+def event_loop():
+    """Override the default pytest event loop to allow for fixtures using a broader scope"""
+    loop = asyncio.get_event_loop_policy().new_event_loop()
+    asyncio.set_event_loop(loop)
+    loop.set_debug(True)
+    yield loop
+    loop.close()
+    asyncio.set_event_loop(None)
+
+
[e-mail](scope='module')
+async def controller():
+    """Connect to the current controller"""
+    _controller = Controller()
+    await _controller.connect_current()
+    yield _controller
+    await _controller.disconnect()
+
+
[e-mail](scope='module')
+async def model(controller):
+    """This model lives only for the duration of the test"""
+    model_name = "functest-{}".format(str(uuid.uuid4())[-12:])
+    _model = await controller.add_model(model_name,
+                                        cloud_name=os.getenv('PYTEST_CLOUD_NAME'),
+                                        region=os.getenv('PYTEST_CLOUD_REGION'),
+                                        )
+    # https://github.com/juju/python-libjuju/issues/267
+    subprocess.check_call(['juju', 'models'])
+    while model_name not in await controller.list_models():
+        await asyncio.sleep(1)
+    yield _model
+    await _model.disconnect()
+    if not os.getenv('PYTEST_KEEP_MODEL'):
+        await controller.destroy_model(model_name)
+        while model_name in await controller.list_models():
+            await asyncio.sleep(1)
+
+
[e-mail](scope='module')
+async def jujutools(controller, model):
+    tools = JujuTools(controller, model)
+    return tools
diff --git a/tests/functional/juju_tools.py b/tests/functional/juju_tools.py
new file mode 100644
index 0000000..4b4884f
--- /dev/null
+++ b/tests/functional/juju_tools.py
@@ -0,0 +1,68 @@
+import pickle
+import juju
+import base64
+
+# from juju.errors import JujuError
+
+
+class JujuTools:
+    def __init__(self, controller, model):
+        self.controller = controller
+        self.model = model
+
+    async def run_command(self, cmd, target):
+        """
+        Runs a command on a unit.
+
+        :param cmd: Command to be run
+        :param unit: Unit object or unit name string
+        """
+        unit = (
+            target
+            if isinstance(target, juju.unit.Unit)
+            else await self.get_unit(target)
+        )
+        action = await unit.run(cmd)
+        return action.results
+
+    async def remote_object(self, imports, remote_cmd, target):
+        """
+        Runs command on target machine and returns a python object of the result
+
+        :param imports: Imports needed for the command to run
+        :param remote_cmd: The python command to execute
+        :param target: Unit object or unit name string
+        """
+        python3 = "python3 -c '{}'"
+        python_cmd = ('import pickle;'
+                      'import base64;'
+                      '{}'
+                      'print(base64.b64encode(pickle.dumps({})), end="")'
+                      .format(imports, remote_cmd))
+        cmd = python3.format(python_cmd)
+        results = await self.run_command(cmd, target)
+        return pickle.loads(base64.b64decode(bytes(results['Stdout'][2:-1], 'utf8')))
+
+    async def file_stat(self, path, target):
+        """
+        Runs stat on a file
+
+        :param path: File path
+        :param target: Unit object or unit name string
+        """
+        imports = 'import os;'
+        python_cmd = ('os.stat("{}")'
+                      .format(path))
+        print("Calling remote cmd: " + python_cmd)
+        return await self.remote_object(imports, python_cmd, target)
+
+    async def file_contents(self, path, target):
+        """
+        Returns the contents of a file
+
+        :param path: File path
+        :param target: Unit object or unit name string
+        """
+        cmd = 'cat {}'.format(path)
+        result = await self.run_command(cmd, target)
+        return result['Stdout']
diff --git a/tests/functional/overlay.yaml.j2 b/tests/functional/overlay.yaml.j2
new file mode 100644
index 0000000..cab782a
--- /dev/null
+++ b/tests/functional/overlay.yaml.j2
@@ -0,0 +1,11 @@
+applications:
+  {{ appname }}:
+    series: {{ series }}
+    charm: {{ charm_path }}
+    num_units: 1
+    options:
+      install_method: {{ install_method }}
+
+relations:
+- - prometheus:grafana-source
+  - {{ appname }}:grafana-source
diff --git a/tests/functional/requirements.txt b/tests/functional/requirements.txt
new file mode 100644
index 0000000..b2e6ce6
--- /dev/null
+++ b/tests/functional/requirements.txt
@@ -0,0 +1,7 @@
+flake8
+Jinja2
+juju
+mock
+pytest
+pytest-asyncio
+requests
diff --git a/tests/functional/test_deploy.py b/tests/functional/test_deploy.py
new file mode 100644
index 0000000..45dbc16
--- /dev/null
+++ b/tests/functional/test_deploy.py
@@ -0,0 +1,240 @@
+import asyncio
+import json
+import os
+import subprocess
+
+import pytest
+import requests
+import yaml
+import jinja2
+
+# Treat all tests as coroutines
+pytestmark = pytest.mark.asyncio
+
+OPENSTACK_TIMEOUT = 3600
+APP_TIMEOUT = 900
+
+CHARM_BUILD_DIR = os.getenv('JUJU_REPOSITORY', '/tmp/charm-builds/grafana').rstrip('/')
+BUNDLE_PATH = os.path.join(CHARM_BUILD_DIR, "..", "bundle.yaml")
+OVERLAY_PATH = os.path.join(CHARM_BUILD_DIR, "..", "overlay.yaml")
+
+APP_SERIES = [
+    "xenial",
+    "bionic",
+    # "trusty",  # not supported in LXD containers
+]
+FAKE_INSTALL_METHOD = 'wrong-method'
+INSTALL_METHOD = {
+    'apt': {"message": "Started grafana-server"},
+    'snap': {"message": "Started snap.grafana.grafana"},
+    'wrong-method': None,
+}
+OPENSTACK_VERSIONS = {
+    "xenial-queens": "cloud:xenial-queens",
+    "bionic-queens": "distro",
+    "bionic-stein": "cloud:bionic-stein",
+}
+
+APP_SERIES_INSTALL_METHOD_MAP = [
+    [series, install_method]
+    for series in APP_SERIES
+    for install_method in INSTALL_METHOD
+    if [series, install_method] != ["trusty", "snap"]  # snaps not supported in trusty
+]
+
+
+################
+# Helper funcs #
+################
+
+def render(templates_dir, template_name, context):
+    templates = jinja2.Environment(
+        loader=jinja2.FileSystemLoader(templates_dir))
+    template = templates.get_template(template_name)
+    return template.render(context)
+
+
+###################
+# Custom fixtures #
+###################
+
[e-mail](scope='module', params=OPENSTACK_VERSIONS)
+async def model(request, controller):
+    ubuntu_os_version = request.param
+    model_name = 'functest-grafana-{}'.format(ubuntu_os_version)
+    if model_name not in await controller.list_models():
+        _model = await controller.add_model(model_name,
+                                            cloud_name=os.getenv('PYTEST_CLOUD_NAME'),
+                                            region=os.getenv('PYTEST_CLOUD_REGION'),
+                                            )
+    else:
+        _model = await controller.get_model(model_name)
+
+    # https://github.com/juju/python-libjuju/issues/267
+    subprocess.check_call(['juju', 'models'])
+    while model_name not in await controller.list_models():
+        await asyncio.sleep(1)
+    yield _model
+    await _model.disconnect()
+    if not os.getenv('PYTEST_KEEP_MODEL'):
+        await controller.destroy_model(model_name)
+        while model_name in await controller.list_models():
+            await asyncio.sleep(1)
+
+
[e-mail](scope='module')
+async def deploy_openstack(model):
+    dir_path = os.path.dirname(os.path.realpath(__file__))
+
+    # functest-grafana-SERIES-OSVERSION
+    _, _, series, os_version = model.info.name.split("-")
+
+    context = {
+        'series': series,
+        'openstack_origin': OPENSTACK_VERSIONS["{}-{}".format(series, os_version)],
+    }
+
+    # render overlay config file
+    rendered = render(dir_path, 'bundle.yaml.j2', context)
+    with open(BUNDLE_PATH, 'w') as fd:
+        fd.write(rendered)
+
+    if os.path.exists(BUNDLE_PATH):
+        # Note(aluria): using await model.deploy(BUNDLE_PATH) will raise an error
+        # because ceph-mon application already exists and cannot be added
+        # OTOH, 'juju deploy bundle.yaml' will try to finalize a previous run
+        subprocess.check_call([
+            'juju',
+            'deploy',
+            '-m',
+            model.info.name,
+            BUNDLE_PATH,
+        ])
+        assert True
+    else:
+        assert False
+
+
[e-mail](
+    scope='function',
+    params=APP_SERIES_INSTALL_METHOD_MAP,
+    ids=["{}-{}".format(series, install_method)
+         for series, install_method in APP_SERIES_INSTALL_METHOD_MAP],
+)
+async def deploy_app(request, model):
+    series, install_method = request.param
+    app_name = 'grafana-{series}-{imethod}'.format(
+        series=series,
+        imethod=install_method,
+    )
+    # functest-grafana-SERIES-OSVERSION
+    ubuntu_os_version = model.info.name[len("functest-grafana-"):]
+
+    dir_path = os.path.dirname(os.path.realpath(__file__))
+    context = {
+        'appname': app_name,
+        'series': series,
+        'install_method': install_method,
+        'charm_path': CHARM_BUILD_DIR,
+        'openstack_origin': OPENSTACK_VERSIONS[ubuntu_os_version],
+    }
+    # render bundle config file
+    rendered = render(dir_path, 'bundle.yaml.j2', context)
+    with open(BUNDLE_PATH, 'w') as fd:
+        fd.write(rendered)
+
+    # render overlay config file
+    rendered = render(dir_path, 'overlay.yaml.j2', context)
+    with open(OVERLAY_PATH, 'w') as fd:
+        fd.write(rendered)
+
+    subprocess.check_call([
+        'juju',
+        'deploy',
+        '-m',
+        model.info.name,
+        BUNDLE_PATH,
+        '--overlay',
+        OVERLAY_PATH,
+    ])
+    while True:
+        try:
+            grafana_app = model.applications[app_name]
+            break
+        except KeyError:
+            await asyncio.sleep(5)
+    yield grafana_app
+
+
+#########
+# Tests #
+#########
+
+async def test_openstack_deploy(deploy_openstack, model):
+    with open(BUNDLE_PATH, 'r') as fd:
+        num_apps = len(yaml.safe_load(fd)["applications"].keys())
+
+    await model.block_until(lambda: (len(model.applications) >= num_apps
+                                     and all(app.status == 'active'
+                                             for name, app in model.applications.items()
+                                             if name.find(FAKE_INSTALL_METHOD) == -1)),
+                            timeout=OPENSTACK_TIMEOUT)
+    assert True
+
+
+async def test_grafana_status(deploy_app, model):
+    config = await deploy_app.get_config()
+    install_method = config['install_method']['value']
+    if install_method == FAKE_INSTALL_METHOD:
+        status = 'blocked'
+        message = 'Unsupported install_method'
+    else:
+        status = 'active'
+        message = INSTALL_METHOD[install_method].get("message")
+
+    await model.block_until(lambda: (deploy_app.status == status
+                                     and all(unit.workload_status_message == message
+                                             for unit in deploy_app.units)),
+                            timeout=APP_TIMEOUT)
+    assert True
+
+
+async def test_grafana_imported_dashboards(deploy_app, model):
+    config = await deploy_app.get_config()
+    install_method = config['install_method']['value']
+    if install_method == FAKE_INSTALL_METHOD:
+        status = 'blocked'
+        message = 'Unsupported install_method'
+    else:
+        status = 'active'
+        message = INSTALL_METHOD[install_method].get("message")
+
+    port = config['port']['value']
+    units = deploy_app.units
+    for grafana_unit in units:
+        await model.block_until(lambda: (grafana_unit.workload_status == status
+                                         and grafana_unit.agent_status == 'idle'
+                                         and grafana_unit.workload_status_message == message),
+                                timeout=APP_TIMEOUT)
+        if install_method == FAKE_INSTALL_METHOD:
+            # no further checks
+            continue
+
+        action = await grafana_unit.run_action('get-admin-password')
+        action = await action.wait()
+        passwd = action.results['password']
+        host = grafana_unit.public_address
+        req = requests.get('http://{host}:{port}'
+                           '/api/search?dashboardIds'.format(host=host, port=port),
+                           auth=('admin', passwd))
+        if req.status_code == 200:
+            dashboards = json.loads(req.text)
+            if len(dashboards) >= 3:
+                assert all(dash.get('type', '') == 'dash-db'
+                           and dash.get('uri', '').startswith('db/juju-')
+                           for dash in dashboards)
+                continue
+            assert len(dashboards) >= 3
+        assert req.status_code == 200
+
+    assert len(units) > 0
diff --git a/tox.ini b/tox.ini
new file mode 100644
index 0000000..288fd77
--- /dev/null
+++ b/tox.ini
@@ -0,0 +1,71 @@
+[tox]
+skipsdist=True
+envlist = unit, functional
+skip_missing_interpreters = True
+
+[testenv]
+basepython = python3
+setenv =
+  PYTHONPATH = .
+
+[testenv:unit]
+commands = pytest -v --ignore {toxinidir}/tests/functional \
+	   --cov=lib \
+	   --cov=reactive \
+	   --cov=actions \
+	   --cov-report=term \
+	   --cov-report=annotate:report/annotated \
+	   --cov-report=html:report/html
+deps = -r{toxinidir}/tests/unit/requirements.txt
+       -r{toxinidir}/requirements.txt
+setenv = PYTHONPATH={toxinidir}/lib
+
+[testenv:functional]
+passenv =
+  HOME
+  JUJU_REPOSITORY
+  PATH
+  PYTEST_KEEP_MODEL
+  PYTEST_CLOUD_NAME
+  PYTEST_CLOUD_REGION
+commands = pytest -v --ignore {toxinidir}/tests/unit
+deps = -r{toxinidir}/tests/functional/requirements.txt
+       -r{toxinidir}/requirements.txt
+
+[testenv:lint]
+commands = flake8
+deps =
+    flake8
+    flake8-docstrings
+    flake8-import-order
+    pep8-naming
+    flake8-colors
+
+[flake8]
+exclude =
+    .git,
+    __pycache__,
+    .tox,
+# H405: Multi line docstrings should start with a one line summary followed by
+#       an empty line.
+# D100: Missing docstring in public module
+# D101: Missing docstring in public class
+# D102: Missing docstring in public method
+# D103: Missing docstring in public function
+# D104: Missing docstring in public package
+# D105: Missing docstring in magic method
+# D107: Missing docstring in __init__
+# D200: One-line docstring should fit on one line with quotes
+# D202: No blank lines allowed after function docstring
+# D203: 1 blank required before class docstring
+# D204: 1 blank line required after class docstring
+# D205: 1 blank line required between summary line and description
+# D208: Docstring is over-indented
+# D400: First line should end with a period
+# D401: First line should be in imperative mood
+# I201: Missing newline between import groups
+# I100: Import statements are in the wrong order
+
+ignore = H405,D100,D101,D102,D103,D104,D105,D107,D200,D202,D203,D204,D205,D208,D400,D401,I100,I201,W503
+max-line-length = 120
+max-complexity = 10