Merge ~sylvain-pineau/checkbox/+git/metabox:configs into ~checkbox-dev/checkbox/+git/metabox:main

[Sylvain Pineau]

First commit (configs now define an URI when they are referring to a file/dir path or a ppa isntead of abusing the origin field)

[Maciej Kisielewski (kissiel)]

2.5kLOC dump is difficult to review. This is why I started with just configs/cli opts. Do you have a version with a better split? Like a more granular git history or something.

Also one quick comment inline.

review: Needs Information

[Sylvain Pineau (sylvain-pineau)]

I can drop comments in the config.py indeed or even prepare some production files, one for ppa, one for checkbox-core snaps

Here's the unsquashed version of the history: https://code.launchpad.net/~sylvain-pineau/checkbox/+git/metabox/+merge/400341 (not super clean though)

[Maciej Kisielewski (kissiel)]

Blocking problems:
I'd like to ask for a cleanup of physical structure of scenarios.
As in reboot is not basic, neither is audio/opengl stuff.
The configs cleanup.

./setup test fails, no dependency on yaml declared.

Non-blocking issues (Don't have to be resolved at this time):

It'd be nicer if yaml files had a '.yaml' suffix.

In the scenarios I'd not match against fancy unicode (checkmark). Does remote even print that glyph?

In the steps:
- Send is a misnomer. Send signal? Send file? It's for typing keys, so TBH, it should just be PushKeys, Type, or soemthing that's close to reality.
For the keys themselves, I think having to use keys.KEY_SPACE for ' ' is bad. The spaces from the string should get auto translated into that. Same goes for enter (\n or \r).

I still think that no audio or rendering should happen. It's just against test design principles. If checkbox is responsible for forwarding some environment info that's needed for audio tests then we should test for those variables, not variables _and_ actual hardware. This frees up dependencies, time, etc.

I still think that assertions are not actions. Assertions should be checked against completed scenario. Like assertion on output or a report. The action is expect, where indeed there is a sequence of things that happen.

The setup commands run in the provider are worse than what we originally had. Less readable, with more escaping and bashisms.

Encoding re patterns to bytes and running searches on bytes is backwards. The output that's being searched should be decoded using stdout's encoding. Instead of hardcoding the pattern encoding.

review: Needs Fixing

[Sylvain Pineau (sylvain-pineau)]

Fixed blocking issues, scn structure is not perfect though.

review: Needs Resubmitting

[Maciej Kisielewski (kissiel)]

We'll improve as we go.

review: Approve

diff --git a/.gitignore b/.gitignore
 new file mode 100644
index 0000000..8d4988b
--- /dev/null
+++ b/.gitignore
@@ -0,0 +1,6 @@
+__pycache__/
+metabox.egg-info/
+venv
+build
+dist
+.tox
diff --git a/MANIFEST.in b/MANIFEST.in
 new file mode 100644
index 0000000..4511d8e
--- /dev/null
+++ b/MANIFEST.in
@@ -0,0 +1,2 @@
+recursive-include metabox/metabox-provider/ *
+recursive-include metabox/lxd_profiles/ *
diff --git a/configs/checkbox-core-snap-classic-beta-config.py b/configs/checkbox-core-snap-classic-beta-config.py
 new file mode 100644
index 0000000..5ffb5da
--- /dev/null
+++ b/configs/checkbox-core-snap-classic-beta-config.py
@@ -0,0 +1,20 @@
+configuration = {
+    'local': {
+        'origin': 'classic_snap',
+        'checkbox_core_snap': {'risk': 'beta'},
+        'checkbox_snap': {'risk': 'stable'},
+        'releases': ['bionic', 'focal'],
+    },
+    'remote': {
+        'origin': 'classic_snap',
+        'checkbox_core_snap': {'risk': 'beta'},
+        'checkbox_snap': {'risk': 'stable'},
+        'releases': ['focal'],
+    },
+    'service': {
+        'origin': 'classic_snap',
+        'checkbox_core_snap': {'risk': 'beta'},
+        'checkbox_snap': {'risk': 'stable'},
+        'releases': ['bionic', 'focal'],
+    },
+}
diff --git a/configs/dev-ppa-config.py b/configs/dev-ppa-config.py
 new file mode 100644
index 0000000..a49d96a
--- /dev/null
+++ b/configs/dev-ppa-config.py
@@ -0,0 +1,17 @@
+configuration = {
+    'local': {
+        'origin': 'ppa',
+        'uri': 'ppa:checkbox-dev/ppa',
+        'releases': ['bionic', 'focal'],
+    },
+    'remote': {
+        'origin': 'ppa',
+        'uri': 'ppa:checkbox-dev/ppa',
+        'releases': ['focal'],
+    },
+    'service': {
+        'origin': 'ppa',
+        'uri': 'ppa:checkbox-dev/ppa',
+        'releases': ['bionic', 'focal'],
+    },
+}
diff --git a/configs/testing-ppa-config.py b/configs/testing-ppa-config.py
 new file mode 100644
index 0000000..2550937
--- /dev/null
+++ b/configs/testing-ppa-config.py
@@ -0,0 +1,17 @@
+configuration = {
+    'local': {
+        'origin': 'ppa',
+        'uri': 'ppa:checkbox-dev/testing',
+        'releases': ['bionic', 'focal'],
+    },
+    'remote': {
+        'origin': 'ppa',
+        'uri': 'ppa:checkbox-dev/testing',
+        'releases': ['focal'],
+    },
+    'service': {
+        'origin': 'ppa',
+        'uri': 'ppa:checkbox-dev/testing',
+        'releases': ['bionic', 'focal'],
+    },
+}
diff --git a/metabox/__init__.py b/metabox/__init__.py
 new file mode 100644
index 0000000..e69de29
--- /dev/null
+++ b/metabox/__init__.py
diff --git a/metabox/core/__init__.py b/metabox/core/__init__.py
 new file mode 100644
index 0000000..e69de29
--- /dev/null
+++ b/metabox/core/__init__.py
diff --git a/metabox/core/actions.py b/metabox/core/actions.py
 new file mode 100644
index 0000000..c4f57b1
--- /dev/null
+++ b/metabox/core/actions.py
@@ -0,0 +1,97 @@
+# This file is part of Checkbox.
+#
+# Copyright 2021 Canonical Ltd.
+# Written by:
+#   Maciej Kisielewski <[e-mail]>
+#   Sylvain Pineau <[e-mail]>
+#
+# Checkbox is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 3,
+# as published by the Free Software Foundation.
+#
+# Checkbox is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with Checkbox.  If not, see <http://www.gnu.org/licenses/>.
+"""
+This module defines the Actions classes.
+
+"""
+
+__all__ = [
+    "Start", "Expect", "Send", "SelectTestPlan",
+    "AssertPrinted", "AssertNotPrinted", "AssertRetCode",
+    "AssertServiceActive", "Sleep", "RunCmd", "Signal", "Reboot",
+    "NetUp", "NetDown",
+]
+
+
+class ActionBase:
+    handler = None
+
+    def __init__(self, *args, **kwargs):
+        self.args = args
+        self.kwargs = kwargs
+
+    def __call__(self, scn):
+        assert(self.handler is not None)
+        getattr(scn, self.handler)(*self.args, **self.kwargs)
+
+
+class Start(ActionBase):
+    handler = 'start'
+
+
+class Expect(ActionBase):
+    handler = 'expect'
+
+
+class Send(ActionBase):
+    handler = 'send'
+
+
+class SelectTestPlan(ActionBase):
+    handler = 'select_test_plan'
+
+
+class AssertPrinted(ActionBase):
+    handler = 'assert_printed'
+
+
+class AssertNotPrinted(ActionBase):
+    handler = 'assert_not_printed'
+
+
+class AssertRetCode(ActionBase):
+    handler = 'assert_ret_code'
+
+
+class AssertServiceActive(ActionBase):
+    handler = 'is_service_active'
+
+
+class Sleep(ActionBase):
+    handler = 'sleep'
+
+
+class RunCmd(ActionBase):
+    handler = 'run_cmd'
+
+
+class Signal(ActionBase):
+    handler = 'signal'
+
+
+class Reboot(ActionBase):
+    handler = 'reboot'
+
+
+class NetUp(ActionBase):
+    handler = 'switch_on_networking'
+
+
+class NetDown(ActionBase):
+    handler = 'switch_off_networking'
diff --git a/metabox/core/aggregator.py b/metabox/core/aggregator.py
 new file mode 100644
index 0000000..c8886d1
--- /dev/null
+++ b/metabox/core/aggregator.py
@@ -0,0 +1,50 @@
+# This file is part of Checkbox.
+#
+# Copyright 2021 Canonical Ltd.
+# Written by:
+#   Maciej Kisielewski <[e-mail]>
+#   Sylvain Pineau <[e-mail]>
+#
+# Checkbox is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 3,
+# as published by the Free Software Foundation.
+#
+# Checkbox is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with Checkbox.  If not, see <http://www.gnu.org/licenses/>.
+"""
+This module implements a mechanism to auto-register metabox scenarios.
+"""
+
+import pkgutil
+
+import metabox.scenarios
+from loguru import logger
+
+
+class _ScenarioAggregator:
+    def __init__(self):
+        self._scenarios = []
+
+    def add_scenario(self, scenario_cls):
+        """Add a scenario to the collection."""
+        logger.debug("Registering a scenario: {}", scenario_cls.name)
+        self._scenarios.append(scenario_cls)
+
+    @staticmethod
+    def load_all():
+        """Import all modules so the scenarios can be auto-loaded."""
+        path = metabox.scenarios.__path__
+        for loader, name, _ in pkgutil.walk_packages(path):
+            loader.find_module(name).load_module(name)
+
+    def all_scenarios(self):
+        """Return all available scenarios."""
+        return self._scenarios
+
+
+aggregator = _ScenarioAggregator()
diff --git a/metabox/core/configuration.py b/metabox/core/configuration.py
 new file mode 100644
index 0000000..1ba8a55
--- /dev/null
+++ b/metabox/core/configuration.py
@@ -0,0 +1,144 @@
+# This file is part of Checkbox.
+#
+# Copyright 2021 Canonical Ltd.
+# Written by:
+#   Maciej Kisielewski <[e-mail]>
+#   Sylvain Pineau <[e-mail]>
+#
+# Checkbox is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 3,
+# as published by the Free Software Foundation.
+#
+# Checkbox is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with Checkbox.  If not, see <http://www.gnu.org/licenses/>.
+"""
+This module implements functions necessary to load metabox configs.
+"""
+import importlib.util
+import subprocess
+from pathlib import Path
+
+from loguru import logger
+
+
+def read_config(filename):
+    """
+    Parse a config file from the path `filename` and yield
+    a configuration object or raise SystemExit on problems.
+    """
+
+    try:
+        mod_spec = importlib.util.spec_from_file_location(
+            'config_file', filename)
+        module = importlib.util.module_from_spec(mod_spec)
+        mod_spec.loader.exec_module(module)
+        config = module.configuration
+        return config
+    except (AttributeError, SyntaxError, FileNotFoundError) as exc:
+        logger.critical(exc)
+        raise SystemExit()
+
+
+def validate_config(config):
+    """
+    Run a sanity check validation of the config.
+    Raises SystemExit when a problem is found.
+    """
+    if not _has_local_or_remote_declaration(config):
+        logger.critical(
+            "Configuration has to define at least one way of running checkbox."
+            "Define 'local' or 'service' and 'remote'.")
+        raise SystemExit()
+    for kind in config:
+        if kind not in ('local', 'service', 'remote'):
+            logger.critical(
+                "Configuration has to define at least one way "
+                "of running checkbox."
+                "Define 'local' or 'service' and 'remote'.")
+            raise SystemExit()
+        for decl in config[kind]:
+            if not _decl_has_a_valid_origin(config[kind]):
+                logger.critical(
+                    "Missing or invalid origin for the {} "
+                    "declaration in config!", kind)
+                raise SystemExit()
+
+
+def _has_local_or_remote_declaration(config):
+    """
+    >>> config = {'local': 'something'}
+    >>> _has_local_or_remote_declaration(config)
+    True
+    >>> config = {'local': ['something_else']}
+    >>> _has_local_or_remote_declaration(config)
+    True
+    >>> config = {'local': []}
+    >>> _has_local_or_remote_declaration(config)
+    False
+    >>> config = {'service': 'something'}
+    >>> _has_local_or_remote_declaration(config)
+    False
+    >>> config = {'remote': 'something'}
+    >>> _has_local_or_remote_declaration(config)
+    False
+    >>> config = {'remote': 'something', 'service': 'somethig_else'}
+    >>> _has_local_or_remote_declaration(config)
+    True
+    """
+
+    return bool(config.get('local') or (
+        config.get('service') and config.get('remote')))
+
+
+def _decl_has_a_valid_origin(decl):
+    """
+    >>> decl = {'origin': 'ppa'}
+    >>> _decl_has_a_valid_origin(decl)
+    True
+    >>> decl = {'origin': 'source'}
+    >>> _decl_has_a_valid_origin(decl)
+    True
+    >>> decl = {'origin': 'snap'}
+    >>> _decl_has_a_valid_origin(decl)
+    True
+    >>> decl = {'origin': 'flatpak'}
+    >>> _decl_has_a_valid_origin(decl)
+    False
+    >>> decl = {}
+    >>> _decl_has_a_valid_origin(decl)
+    False
+    """
+    if 'origin' not in decl:
+        return False
+    if decl['origin'] == 'snap':
+        return True
+    elif decl['origin'] == 'classic_snap':
+        return True
+    elif decl['origin'] == 'ppa':
+        return True
+    elif decl['origin'] == 'source':
+        source = Path(decl['uri']).expanduser()
+        if not source.is_dir():
+            logger.error("{} doesn't look like a directory", source)
+            return False
+        setup_file = source / 'setup.py'
+        if not setup_file.exists():
+            logger.error("{} not found", setup_file)
+            return False
+        try:
+            package_name = subprocess.check_output(
+                [setup_file, '--name'],
+                stderr=subprocess.DEVNULL).decode('utf-8').rstrip()
+        except subprocess.CalledProcessError:
+            logger.error("{} --name failed", setup_file)
+            return False
+        if not package_name == 'checkbox-ng':
+            logger.error("{} must be a lp:checkbox-ng fork", source)
+            return False
+        return True
+    return False
diff --git a/metabox/core/keys.py b/metabox/core/keys.py
 new file mode 100644
index 0000000..e2cbc6f
--- /dev/null
+++ b/metabox/core/keys.py
@@ -0,0 +1,36 @@
+# This file is part of Checkbox.
+#
+# Copyright 2021 Canonical Ltd.
+# Written by:
+#   Maciej Kisielewski <[e-mail]>
+#   Sylvain Pineau <[e-mail]>
+#
+# Checkbox is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 3,
+# as published by the Free Software Foundation.
+#
+# Checkbox is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with Checkbox.  If not, see <http://www.gnu.org/licenses/>.
+"""Key constants and utilities for Scenarios."""
+
+import signal
+
+KEY_ENTER = "\n"
+KEY_UP = "\x1b[A"
+KEY_DOWN = "\x1b[B"
+KEY_RIGHT = "\x1b[C"
+KEY_LEFT = "\x1b[D"
+KEY_PAGEUP = "\x1b[5~"
+KEY_PAGEDOWN = "\x1b[6~"
+KEY_HOME = "\x1b[H"
+KEY_END = "\x1b[F"
+KEY_SPACE = "\x20"
+KEY_ESCAPE = "\x1b"
+
+SIGINT = signal.SIGINT.value
+SIGKILL = signal.SIGKILL.value
diff --git a/metabox/core/lxd_execute.py b/metabox/core/lxd_execute.py
 new file mode 100644
index 0000000..1dbb0d7
--- /dev/null
+++ b/metabox/core/lxd_execute.py
@@ -0,0 +1,236 @@
+# This file is part of Checkbox.
+#
+# Copyright 2021 Canonical Ltd.
+# Written by:
+#   Maciej Kisielewski <[e-mail]>
+#   Sylvain Pineau <[e-mail]>
+#
+# Checkbox is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 3,
+# as published by the Free Software Foundation.
+#
+# Checkbox is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with Checkbox.  If not, see <http://www.gnu.org/licenses/>.
+import json
+import re
+import shlex
+import threading
+import time
+
+from loguru import logger
+import metabox.core.keys as keys
+from metabox.core.utils import ExecuteResult
+from ws4py.client.threadedclient import WebSocketClient
+
+
+base_env = {
+    'PYTHONUNBUFFERED': '1',
+    'DISABLE_URWID_ESCAPE_CODES': '1',
+    'XDG_RUNTIME_DIR': '/run/user/1000'
+}
+login_shell = ['sudo', '--user', 'ubuntu', '--login']
+
+
+class InteractiveWebsocket(WebSocketClient):
+
+    # https://stackoverflow.com/a/14693789/1154487
+    ansi_escape = re.compile(r'\x1B(?:[@-Z\\-_]|\[[0-?]*[ -/]*[@-~])')
+
+    def __init__(self, *args, **kwargs):
+        super().__init__(*args, **kwargs)
+        self.stdout_data = bytearray()
+        self.stdout_data_full = bytearray()
+        self.stdout_lock = threading.Lock()
+        self._new_data = False
+        self._lookup_by_id = False
+
+    def received_message(self, message):
+        if len(message.data) == 0:
+            self.close()
+        if self.verbose:
+            raw_msg = self.ansi_escape.sub(
+                '', message.data.decode('utf-8', errors='ignore'))
+            logger.trace(raw_msg.rstrip())
+        with self.stdout_lock:
+            self.stdout_data += message.data
+            self.stdout_data_full += message.data
+            self._new_data = True
+
+    def expect(self, data, timeout=0):
+        not_found = True
+        start_time = time.time()
+        while not_found:
+            time.sleep(0.1)
+            if type(data) != str:
+                check = data.search(self.stdout_data)
+            else:
+                check = data.encode('utf-8') in self.stdout_data
+            if check:
+                with self.stdout_lock:
+                    self.stdout_data = bytearray()
+                not_found = False
+            if timeout and time.time() > start_time + timeout:
+                logger.warning(
+                    "{} not found! Timeout is reached (set to {})",
+                    data, timeout)
+                raise TimeoutError
+        return not_found is False
+
+    def select_test_plan(self, data, timeout=0):
+        if not self._lookup_by_id:
+            self.send(('i' + keys.KEY_HOME).encode('utf-8'), binary=True)
+            self._lookup_by_id = True
+        else:
+            self.send(keys.KEY_HOME.encode('utf-8'), binary=True)
+        not_found = True
+        max_attemps = 10
+        attempt = 0
+        still_on_first_screen = True
+        old_stdout_data = b''
+        if len(data) > 67:
+            data = data[:67] + '   │\r\n│        ' + data[67:]
+        while attempt < max_attemps:
+            if self._new_data and self.stdout_data:
+                if old_stdout_data == self.stdout_data:
+                    break
+                check = data.encode('utf-8') in self.stdout_data
+                if not check:
+                    self._new_data = False
+                    with self.stdout_lock:
+                        old_stdout_data = self.stdout_data
+                        self.stdout_data = bytearray()
+                    stdin_payload = keys.KEY_PAGEDOWN + keys.KEY_SPACE
+                    self.send(stdin_payload.encode('utf-8'), binary=True)
+                    still_on_first_screen = False
+                    attempt = 0
+                else:
+                    not_found = False
+                    break
+            else:
+                time.sleep(0.1)
+                attempt += 1
+        if not_found:
+            return False
+        data = '(X) ' + data
+        attempt = 0
+        if still_on_first_screen:
+            self.send(keys.KEY_PAGEDOWN.encode('utf-8'), binary=True)
+        while attempt < max_attemps:
+            if self._new_data and self.stdout_data:
+                check = data.encode('utf-8') in self.stdout_data
+                if not check:
+                    self._new_data = False
+                    with self.stdout_lock:
+                        self.stdout_data = bytearray()
+                    stdin_payload = keys.KEY_UP + keys.KEY_SPACE
+                    self.send(stdin_payload.encode('utf-8'), binary=True)
+                    attempt = 0
+                else:
+                    not_found = False
+                    with self.stdout_lock:
+                        self.stdout_data = bytearray()
+                    break
+            else:
+                time.sleep(0.1)
+                attempt += 1
+        return not_found is False
+
+    def send_signal(self, signal):
+        self.ctl.send(json.dumps({'command': 'signal', 'signal': signal}))
+
+    @property
+    def container(self):
+        return self._container
+
+    @container.setter
+    def container(self, container):
+        self._container = container
+
+    @property
+    def ctl(self):
+        return self._ctl
+
+    @ctl.setter
+    def ctl(self, ctl):
+        self._ctl = ctl
+
+    @property
+    def verbose(self):
+        return self._verbose
+
+    @verbose.setter
+    def verbose(self, verbose):
+        self._verbose = verbose
+
+
+def env_wrapper(env):
+    env_cmd = ['env']
+    env.update(base_env)
+    env_cmd += [
+        "{key}={value}".format(key=key, value=value)
+        for key, value in sorted(env.items())]
+    return env_cmd
+
+
+def timeout_wrapper(timeout):
+    if timeout:
+        return ['timeout', '--signal=KILL', str(timeout)]
+    else:
+        return []
+
+
+def interactive_execute(container, cmd, env={}, verbose=False, timeout=0):
+    if verbose:
+        logger.trace(cmd)
+    ws_urls = container.raw_interactive_execute(
+        login_shell + env_wrapper(env) + shlex.split(cmd))
+    base_websocket_url = container.client.websocket_url
+    ctl = WebSocketClient(base_websocket_url)
+    ctl.resource = ws_urls['control']
+    ctl.connect()
+    pts = InteractiveWebsocket(base_websocket_url)
+    pts.resource = ws_urls['ws']
+    pts.verbose = verbose
+    pts.container = container
+    pts.ctl = ctl
+    pts.connect()
+    return pts
+
+
+def run_or_raise(container, cmd, env={}, verbose=False, timeout=0):
+    stdout_data = ''
+    stderr_data = ''
+
+    def on_stdout(msg):
+        nonlocal stdout_data
+        stdout_data += msg
+        logger.trace(msg.rstrip())
+
+    def on_stderr(msg):
+        nonlocal stderr_data
+        stderr_data += msg
+        logger.trace(msg.rstrip())
+
+    if verbose:
+        logger.trace(cmd)
+    res = container.execute(
+        login_shell + env_wrapper(env) + timeout_wrapper(timeout)
+        + shlex.split(cmd),  # noqa 503
+        stdout_handler=on_stdout if verbose else None,
+        stderr_handler=on_stderr if verbose else None,
+        stdin_payload=open(__file__))
+    if timeout and res.exit_code == 137:
+        logger.warning("{} Timeout is reached (set to {})", cmd, timeout)
+        raise TimeoutError
+    elif res.exit_code:
+        msg = "Failed to run command in the container! Command: \n"
+        msg += cmd + ' ' + res.stderr
+        # raise SystemExit(msg)
+    if verbose:
+        return (ExecuteResult(res.exit_code, stdout_data, stderr_data))
+    return res
diff --git a/metabox/core/lxd_provider.py b/metabox/core/lxd_provider.py
 new file mode 100644
index 0000000..df70b8a
--- /dev/null
+++ b/metabox/core/lxd_provider.py
@@ -0,0 +1,229 @@
+# This file is part of Checkbox.
+#
+# Copyright 2021 Canonical Ltd.
+# Written by:
+#   Maciej Kisielewski <[e-mail]>
+#   Sylvain Pineau <[e-mail]>
+#
+# Checkbox is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 3,
+# as published by the Free Software Foundation.
+#
+# Checkbox is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with Checkbox.  If not, see <http://www.gnu.org/licenses/>.
+
+"""
+lxd_provider
+============
+
+This module implements the LXD Machine and LXD Machine Provider.
+LXD machines are containers that can run metabox scenarios in them.
+"""
+import json
+import os
+import time
+import yaml
+from pathlib import Path
+
+import pkg_resources
+import pylxd
+from loguru import logger
+from pylxd.exceptions import ClientConnectionFailed, LXDAPIException
+
+from metabox.core.machine import MachineConfig
+from metabox.core.machine import machine_selector
+from metabox.core.lxd_execute import run_or_raise
+
+
+class LxdMachineProvider():
+    """Machine provider that uses container managed by LXD as targets."""
+
+    LXD_CREATE_TIMEOUT = 120
+    LXD_POLL_INTERVAL = 5
+    LXD_INTERNAL_CONFIG_PATH = '/var/tmp/machine_config.json'
+
+    def __init__(self, session_config, effective_machine_config,
+                 debug_machine_setup=False, dispose=False):
+        self._session_config = session_config
+        self._machine_config = effective_machine_config
+        self._debug_machine_setup = debug_machine_setup
+        self._owned_containers = []
+        self._dispose = dispose
+
+        # TODO: maybe add handlers for more complicated client connections
+        #       like a remote LXD host and/or authenticated access
+        try:
+            # TODO: Find a suitable timeout value here
+            self.client = pylxd.Client(timeout=None)
+        except ClientConnectionFailed as exc:
+            raise SystemExit from exc
+
+    def setup(self):
+        self._create_profiles()
+        self._get_existing_machines()
+        for config in self._machine_config:
+            if config in [oc.config for oc in self._owned_containers]:
+                continue
+            self._create_machine(config)
+
+    def _get_existing_machines(self):
+        for container in self.client.containers.all():
+            try:
+                content = container.files.get(self.LXD_INTERNAL_CONFIG_PATH)
+                config_dict = json.loads(content)
+                config = MachineConfig(config_dict['role'], config_dict)
+            except (KeyError, LXDAPIException):
+                continue
+            if config in self._machine_config:
+                self._owned_containers.append(
+                    machine_selector(config, container))
+
+    def _create_profiles(self):
+        profiles_path = pkg_resources.resource_filename(
+            'metabox', 'lxd_profiles')
+        for profile_file in os.listdir(profiles_path):
+            profile_name = Path(profile_file).stem
+            with open(os.path.join(profiles_path, profile_file)) as f:
+                profile_dict = yaml.load(f, Loader=yaml.FullLoader)
+            if self.client.profiles.exists(profile_name):
+                profile = self.client.profiles.get(profile_name)
+                if 'config' in profile_dict:
+                    profile.config = profile_dict['config']
+                if 'devices' in profile_dict:
+                    profile.devices = profile_dict['devices']
+                profile.save()
+                logger.debug(
+                    '{} LXD profile updated successfully', profile_name)
+            else:
+                profile = self.client.profiles.create(
+                    profile_name,
+                    config=profile_dict.get('config', None),
+                    devices=profile_dict.get('devices', None))
+                logger.debug(
+                    '{} LXD profile created successfully', profile_name)
+
+    def _create_machine(self, config):
+        name = 'metabox-{}'.format(config)
+        base_profiles = ["default", "checkbox"]
+        if config.origin == 'snap':
+            base_profiles.append('snap')
+        lxd_config = {
+            "name": name,
+            "profiles": base_profiles + config.profiles,
+            "source": {
+                "type": "image",
+                "alias": config.alias,
+                'protocol': 'simplestreams',
+                'server': 'https://cloud-images.ubuntu.com/releases'
+            }
+        }
+        try:
+            logger.opt(colors=True).debug("[<y>creating</y>    ] {}", name)
+            container = self.client.containers.create(lxd_config, wait=True)
+            machine = machine_selector(config, container)
+            container.start(wait=True)
+            attempt = 0
+            max_attempt = self.LXD_CREATE_TIMEOUT / self.LXD_POLL_INTERVAL
+            while attempt < max_attempt:
+                time.sleep(self.LXD_POLL_INTERVAL)
+                (ret, out, err) = container.execute(
+                    ['cloud-init', 'status', '--long'])
+                if 'status: done' in out:
+                    break
+                elif ret != 0:
+                    logger.error(out)
+                    raise SystemExit(out)
+                attempt += 1
+            else:
+                raise SystemExit("Timeout reached (still running cloud-init)")
+            container.stop(wait=True)
+            container.snapshots.create('base', stateful=False, wait=True)
+            container.start(wait=True)
+            logger.opt(colors=True).debug(
+                "[<y>created</y>     ] {}", container.name)
+            logger.opt(colors=True).debug(
+                "[<y>provisioning</y>] {}", container.name)
+            self._run_transfer_commands(machine)
+            time.sleep(5)  # FIXME: is it still needed?
+            self._run_setup_commands(machine)
+            self._store_config(machine)
+            container.stop(wait=True)
+            container.snapshots.create(
+                'provisioned', stateful=False, wait=True)
+            container.start(wait=True)
+            self._owned_containers.append(machine)
+            logger.opt(colors=True).debug(
+                "[<y>provisioned</y> ] {}", container.name)
+
+        except LXDAPIException as exc:
+            error = self._api_exc_to_human(exc)
+            raise SystemExit(error) from exc
+
+    def _run_transfer_commands(self, machine):
+        provider_path = pkg_resources.resource_filename(
+            'metabox', 'metabox-provider')
+        metabox_dir_transfers = machine.get_early_dir_transfer() + [
+            (provider_path, '/var/tmp/checkbox-providers/metabox-provider')]
+        for src, dest in metabox_dir_transfers + machine.config.transfer:
+            run_or_raise(
+                machine._container, 'mkdir -p {}'.format(dest),
+                verbose=self._debug_machine_setup)
+            machine._container.files.recursive_put(
+                os.path.expanduser(src), dest)
+            run_or_raise(
+                machine._container,
+                'sudo chown -R ubuntu:ubuntu {}'.format(dest),
+                verbose=self._debug_machine_setup)
+            # FIXME: preserve the +x bit
+        for src, dest in machine.get_file_transfer():
+            with open(src, "rb") as f:
+                machine._container.files.put(dest, f.read())
+                # FIXME: preserve the +x bit
+
+    def _run_setup_commands(self, machine):
+        pre_cmds = machine.get_early_setup()
+        post_cmds = machine.get_late_setup()
+        for cmd in pre_cmds + machine.config.setup + post_cmds:
+            res = run_or_raise(
+                machine._container, cmd,
+                verbose=self._debug_machine_setup)
+            if res.exit_code:
+                msg = "Failed to run command in the container! Command: \n"
+                msg += cmd + '\n' + res.stderr
+                logger.critical(msg)
+                raise SystemExit()
+
+    def _store_config(self, machine):
+        machine._container.files.put(
+            self.LXD_INTERNAL_CONFIG_PATH, json.dumps(machine.config.__dict__))
+
+    def get_machine_by_config(self, config):
+        for machine in self._owned_containers:
+            if config == machine.config:
+                return machine
+
+    def cleanup(self, dispose=False):
+        """Stop and delete (on request) all the containers."""
+        for machine in self._owned_containers:
+            container = machine._container
+            if container.status == "Running":
+                container.stop(wait=True)
+                logger.opt(colors=True).debug(
+                    "[<y>stopped</y>     ] {}", container.name)
+            if dispose:
+                container.delete(wait=True)
+                logger.opt(colors=True).debug(
+                    "[<y>deleted</y>     ] {}", container.name)
+
+    def _api_exc_to_human(self, exc):
+        response = json.loads(exc.response.text)
+        # TODO: wrap in try/except and on wrong fields dump all info we have
+        return response.get('error') or response['metadata']['err']
+
+    def __del__(self):
+        self.cleanup(self._dispose)
diff --git a/metabox/core/machine.py b/metabox/core/machine.py
 new file mode 100644
index 0000000..2004097
--- /dev/null
+++ b/metabox/core/machine.py
@@ -0,0 +1,433 @@
+# This file is part of Checkbox.
+#
+# Copyright 2021 Canonical Ltd.
+# Written by:
+#   Maciej Kisielewski <[e-mail]>
+#   Sylvain Pineau <[e-mail]>
+#
+# Checkbox is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 3,
+# as published by the Free Software Foundation.
+#
+# Checkbox is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with Checkbox.  If not, see <http://www.gnu.org/licenses/>.
+import itertools
+import signal
+import time
+from pathlib import Path
+
+from loguru import logger
+from metabox.core.lxd_execute import interactive_execute
+from metabox.core.lxd_execute import run_or_raise
+
+
+class MachineConfig:
+
+    def __init__(self, role, config):
+        self.role = role
+        self.alias = config['alias']
+        self.origin = config['origin']
+        self.uri = config.get("uri", "")
+        self.profiles = config.get("profiles", [])
+        self.transfer = config.get("transfer", [])
+        self.setup = config.get("setup", [])
+        self.checkbox_core_snap = config.get("checkbox_core_snap", {})
+        self.checkbox_snap = config.get("checkbox_snap", {})
+        self.snap_name = config.get("name", "")
+        if not self.snap_name:
+            if self.origin == 'snap':
+                self.snap_name = 'checkbox-snappy'
+            elif self.origin == 'classic_snap':
+                self.snap_name = 'checkbox-snappy-classic'
+
+    def __members(self):
+        return (self.role, self.alias, self.origin, self.snap_name, self.uri,
+                ' '.join(self.profiles),
+                ' '.join(itertools.chain(*self.transfer)),
+                ' '.join(self.setup))
+
+    def __repr__(self):
+        return "<{} alias:{!r} origin:{!r}>".format(
+            self.role, self.alias, self.origin)
+
+    def __str__(self):
+        return "{}-{}".format(self.role, self.alias)
+
+    def __hash__(self):
+        return hash(self.__members())
+
+    def __eq__(self, other):
+        if type(other) is type(self):
+            return self.__members() == other.__members()
+        else:
+            return False
+
+
+class ContainerBaseMachine():
+    """Base implementation of Machine using LXD container as the backend."""
+
+    CHECKBOX = 'checkbox-cli '  # mind the trailing space !
+
+    def __init__(self, config, container):
+        self.config = config
+        self._container = container
+        self._checkbox_wrapper = self.CHECKBOX
+
+    def execute(self, cmd, env={}, verbose=False, timeout=0):
+        return run_or_raise(
+            self._container, self._checkbox_wrapper + cmd, env, verbose,
+            timeout)
+
+    def interactive_execute(self, cmd, env={}, verbose=False, timeout=0):
+        return interactive_execute(
+            self._container, self._checkbox_wrapper + cmd, env, verbose,
+            timeout)
+
+    def rollback_to(self, savepoint):
+        if self._container.status != 'Stopped':
+            self._container.stop(wait=True)
+        self._container.restore_snapshot(savepoint, wait=True)
+        self._container.start(wait=True)
+        logger.opt(colors=True).debug(
+            "[<y>restored</y>    ] {}", self._container.name)
+        attempt = 0
+        # FIXME: in case containers are restarted after reboot, sometime we hit
+        # the degraded state.
+        # So be sure to wait long enough to claim it started.
+        # https://discuss.linuxcontainers.org/t/snap-lxd-activate-service-hanging-after-system-reboot/8374/16
+        max_attempt = 60
+        old_out = ''
+        while attempt < max_attempt:
+            time.sleep(1)
+            (ret, out, err) = self._container.execute(
+                ['systemctl', 'is-system-running'])
+            if out != old_out:
+                logger.opt(colors=True).debug(
+                    "[<y>{: <12}</y>] {}", out.rstrip(), self._container.name)
+                old_out = out
+            if 'running' in out:
+                break
+            elif 'degraded' in out:
+                break
+            attempt += 1
+        else:
+            raise SystemExit(
+                "Rollback to {} failed (systemd not in running state)".format(
+                    savepoint))
+
+    def put(self, filepath, data, mode=None, uid=1000, gid=1000):
+        self._container.files.put(filepath, data, mode, uid, gid)
+
+    def get_connecting_cmd(self):
+        return "lxc exec {} -- sudo --user ubuntu --login".format(
+            self._container.name)
+
+    @property
+    def address(self):
+        addresses = self._container.state().network['eth0']['addresses']
+        return addresses[0]['address']
+
+    def get_early_dir_transfer(self):
+        """
+        .. note:: You should override this method in your subclass.
+        """
+        return []
+
+    def get_file_transfer(self):
+        """
+        .. note:: You should override this method in your subclass.
+        """
+        return []
+
+    def get_early_setup(self):
+        """
+        .. note:: You should override this method in your subclass.
+        """
+        return []
+
+    def get_late_setup(self):
+        """
+        .. note:: You should override this method in your subclass.
+        """
+        return []
+
+    def start_remote(self, host, launcher, interactive=False, timeout=0):
+        assert(self.config.role == 'remote')
+
+        if interactive:
+            # Return a PTS object to interact with
+            return self.interactive_execute(
+                'remote {} {}'.format(host, launcher), verbose=True,
+                timeout=timeout)
+        else:
+            # Return an ExecuteResult named tuple
+            return self.execute(
+                'remote {} {}'.format(host, launcher), verbose=True,
+                timeout=timeout)
+
+    def start(self, cmd=None, env={}, interactive=False, timeout=0):
+        assert(self.config.role == 'local')
+        if interactive:
+            # Return a PTS object to interact with
+            return self.interactive_execute(
+                cmd, env=env, verbose=True, timeout=timeout)
+        else:
+            # Return an ExecuteResult named tuple
+            return self.execute(
+                cmd, env=env, verbose=True, timeout=timeout)
+
+    def run_cmd(self, cmd, env={}, interactive=False, timeout=0):
+        verbose = True
+        if interactive:
+            # Return a PTS object to interact with
+            return interactive_execute(
+                self._container, cmd, env, verbose, timeout)
+        else:
+            # Return an ExecuteResult named tuple
+            return run_or_raise(
+                self._container, cmd, env, verbose, timeout)
+
+    def start_service(self, force=False):
+        """
+        .. note:: You should override this method in your subclass.
+        """
+        pass
+
+    def stop_service(self):
+        """
+        .. note:: You should override this method in your subclass.
+        """
+        pass
+
+    def reboot(self, timeout=0):
+        verbose = True
+        return run_or_raise(
+            self._container, "sudo reboot", verbose, timeout)
+
+    def is_service_active(self):
+        """
+        .. note:: You should override this method in your subclass.
+        """
+        pass
+
+    def start_user_session(self):
+        assert(self.config.role in ('service', 'local'))
+        # Start a set of ubuntu-user-owned processes to fake an active GDM user
+        # session (A virtual framebuffer and a pulseaudio server with a dummy
+        # output)
+        interactive_execute(
+            self._container, "/usr/bin/Xvfb -screen 0 1280x1024x24")
+        # Note: running the following commands as part of standard setup does
+        # not make them persistent as after restoring snapshots user/1000
+        # is gone from /run
+        pulseaudio_setup_cmds = [
+            'sudo mkdir -v -p /run/user/1000/pulse',
+            'sudo chown -R ubuntu:ubuntu /run/user/1000/',
+            "pulseaudio --start --exit-idle-time=-1 --disallow-module-loading",
+        ]
+        env = {'XDG_RUNTIME_DIR': '/run/user/1000'}
+        for cmd in pulseaudio_setup_cmds:
+            run_or_raise(self._container, cmd, env)
+
+    def switch_off_networking(self):
+        return run_or_raise(self._container, "sudo ip link set eth0 down")
+
+    def switch_on_networking(self):
+        return run_or_raise(self._container, "sudo ip link set eth0 up")
+
+
+class ContainerVenvMachine(ContainerBaseMachine):
+    """
+    Machine using LXD container as the backend and running checkbox
+    in a virtualenv.
+    """
+
+    def __init__(self, config, container):
+        super().__init__(config, container)
+        if self.config.role == 'service':
+            self._checkbox_wrapper = 'sudo /home/ubuntu/run.sh {}'.format(
+                self.CHECKBOX)
+        else:
+            self._checkbox_wrapper = '/home/ubuntu/run.sh {}'.format(
+                self.CHECKBOX)
+        self._pts = None  # Keep a pointer to started pts for easy kill
+
+    def get_early_dir_transfer(self):
+        return [(self.config.uri, '/home/ubuntu/checkbox-ng')]
+
+    def get_early_setup(self):
+        """Virtualenv creation."""
+        return [
+            '''bash -c "printf '#!/bin/bash\\n. '''
+            '''/home/ubuntu/checkbox-ng/venv/bin/activate\\n$@' > '''
+            '''/home/ubuntu/run.sh"''',
+            'chmod +x /home/ubuntu/run.sh',
+            'chmod +x /home/ubuntu/checkbox-ng/setup.py',
+            'chmod +x /home/ubuntu/checkbox-ng/mk-venv',
+            "bash -c 'pushd /home/ubuntu/checkbox-ng ; ./mk-venv venv'",
+        ]
+
+    def start_service(self, force=False):
+        assert(self.config.role == 'service')
+        self._pts = self.interactive_execute('service', verbose=True)
+        return self._pts
+
+    def stop_service(self):
+        assert(self.config.role == 'service')
+        return self._pts.send_signal(signal.SIGINT.value)
+
+    def reboot_service(self):
+        """
+        Venv Service is not a systemd service.
+        It won't show up after a reboot.
+        """
+        raise RuntimeError
+
+    def is_service_active(self):
+        assert(self.config.role == 'service')
+        return run_or_raise(
+            self._container, 'pgrep -f "python3.*checkbox-cli service"')
+
+
+class ContainerPPAMachine(ContainerBaseMachine):
+    """
+    Machine using LXD container as the backend and running checkbox
+    from PPA.
+    """
+
+    def __init__(self, config, container):
+        super().__init__(config, container)
+
+    def get_early_setup(self):
+        if self.config.setup:
+            return []
+        if self.config.role == 'remote':
+            deb = 'checkbox-ng'
+        else:
+            deb = 'canonical-certification-client'
+        return [
+            'sudo add-apt-repository {}'.format(self.config.uri),
+            'sudo apt-get update',
+            'sudo apt-get install -y --no-install-recommends {}'.format(deb),
+        ]
+
+    def start_service(self, force=False):
+        assert(self.config.role == 'service')
+        if force:
+            return run_or_raise(
+                self._container, "sudo systemctl start checkbox-ng.service")
+
+    def stop_service(self):
+        assert(self.config.role == 'service')
+        return run_or_raise(
+            self._container, "sudo systemctl stop checkbox-ng.service")
+
+    def is_service_active(self):
+        assert(self.config.role == 'service')
+        return run_or_raise(
+            self._container,
+            "systemctl is-active checkbox-ng.service").stdout == 'active'
+
+
+class ContainerSnapMachine(ContainerBaseMachine):
+    """
+    Machine using LXD container as the backend and running checkbox
+    from a snap.
+    """
+
+    CHECKBOX_CORE_SNAP_MAP = {
+        'xenial': 'checkbox',
+        'bionic': 'checkbox18',
+        'focal':  'checkbox20',
+    }
+    CHECKBOX_SNAP_TRACK_MAP = {
+        'xenial': '16',
+        'bionic': '18',
+        'focal':  '20',
+    }
+
+    def __init__(self, config, container):
+        super().__init__(config, container)
+        self._snap_name = self.config.snap_name
+        self._checkbox_wrapper = '{}.{}'.format(self._snap_name, self.CHECKBOX)
+
+    def get_file_transfer(self):
+        file_tranfer_list = []
+        if self.config.checkbox_core_snap.get('uri'):
+            core_filename = Path(
+                self.config.checkbox_core_snap.get('uri')).expanduser()
+            self.core_dest = Path('/home', 'ubuntu', core_filename.name)
+            file_tranfer_list.append((core_filename, self.core_dest))
+        if self.config.checkbox_snap.get('uri'):
+            filename = Path(
+                self.config.checkbox_snap.get('uri')).expanduser()
+            self.dest = Path('/home', 'ubuntu', filename.name)
+            file_tranfer_list.append((filename, self.dest))
+        return file_tranfer_list
+
+    def get_early_setup(self):
+        cmds = []
+        # First install the checkbox core snap if the related section exists
+        if self.config.checkbox_core_snap:
+            if self.config.checkbox_core_snap.get('uri'):
+                cmds.append(f'sudo snap install {self.core_dest} --dangerous')
+            else:
+                core_snap = self.CHECKBOX_CORE_SNAP_MAP[self.config.alias]
+                channel = f"latest/{self.config.checkbox_core_snap['risk']}"
+                cmds.append(
+                    f'sudo snap install {core_snap} --channel={channel}')
+        # Then install the checkbox snap
+        confinement = 'devmode'
+        if self.config.origin == 'classic_snap':
+            confinement = 'classic'
+        if self.config.checkbox_snap.get('uri'):
+            cmds.append(
+                f'sudo snap install {self.dest} --{confinement} --dangerous')
+        else:
+            try:
+                track_map = self.config.checkbox_snap['track_map']
+            except KeyError:
+                track_map = self.CHECKBOX_SNAP_TRACK_MAP
+            channel = "{}/{}".format(
+                track_map[self.config.alias],
+                self.config.checkbox_snap['risk'])
+            cmds.append('sudo snap install {} --channel={} --{}'.format(
+                self._snap_name, channel, confinement))
+        return cmds
+
+    def start_service(self, force=False):
+        assert(self.config.role == 'service')
+        if force:
+            return run_or_raise(
+                self._container,
+                "sudo systemctl start snap.{}.service.service".format(
+                    self._snap_name))
+
+    def stop_service(self):
+        assert(self.config.role == 'service')
+        return run_or_raise(
+            self._container,
+            "sudo systemctl stop snap.{}.service.service".format(
+                self._snap_name))
+
+    def is_service_active(self):
+        assert(self.config.role == 'service')
+        return run_or_raise(
+            self._container,
+            "systemctl is-active snap.{}.service.service".format(
+                self._snap_name)
+        ).stdout == 'active'
+
+
+def machine_selector(config, container):
+    if config.origin in ('snap', 'classic_snap'):
+        return(ContainerSnapMachine(config, container))
+    elif config.origin == 'ppa':
+        return(ContainerPPAMachine(config, container))
+    elif config.origin == 'source':
+        return(ContainerVenvMachine(config, container))
diff --git a/metabox/core/runner.py b/metabox/core/runner.py
 new file mode 100644
index 0000000..57c8530
--- /dev/null
+++ b/metabox/core/runner.py
@@ -0,0 +1,221 @@
+# This file is part of Checkbox.
+#
+# Copyright 2021 Canonical Ltd.
+# Written by:
+#   Maciej Kisielewski <[e-mail]>
+#   Sylvain Pineau <[e-mail]>
+#
+# Checkbox is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 3,
+# as published by the Free Software Foundation.
+#
+# Checkbox is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with Checkbox.  If not, see <http://www.gnu.org/licenses/>.
+import sys
+import time
+
+from loguru import logger
+from metabox.core.aggregator import aggregator
+from metabox.core.configuration import read_config
+from metabox.core.configuration import validate_config
+from metabox.core.lxd_provider import LxdMachineProvider
+from metabox.core.machine import MachineConfig
+
+logger = logger.opt(colors=True)
+
+
+class Runner:
+    """Metabox scenario discovery and runner."""
+
+    def __init__(self, args):
+        self.args = args
+        # logging
+        logger.remove()
+        logger.add(
+            sys.stdout,
+            format=self._formatter,
+            level=args.log_level)
+        logger.level("TRACE", color="<w><dim>")
+        logger.level("DEBUG", color="<w><dim>")
+        # session config
+        if not args.config.exists():
+            raise SystemExit('Config file not found!')
+        else:
+            self.config = read_config(args.config)
+            validate_config(self.config)
+        # effective set of machine configs required by scenarios
+        self.combo = set()
+        self.machine_provider = None
+        self.failed = False
+        self.tags = set(self.args.tags or [])
+        self.exclude_tags = set(self.args.exclude_tags or [])
+        self.hold_on_fail = self.args.hold_on_fail
+        self.debug_machine_setup = self.args.debug_machine_setup
+        self.dispose = not self.args.do_not_dispose
+        aggregator.load_all()
+
+    def _formatter(self, record):
+        if record["level"].no < 10:
+            return "<level>{message}</level>\n"
+        else:
+            return (
+                "{time:HH:mm:ss} | <level>{level: <8}</level> "
+                "<level>{message}</level>\n"
+            )
+
+    def _gather_all_machine_spec(self):
+        for v in self.scn_variants:
+            if v.mode == 'remote':
+                remote_config = self.config['remote'].copy()
+                service_config = self.config['service'].copy()
+                remote_release, service_release = v.releases
+                remote_config['alias'] = remote_release
+                service_config['alias'] = service_release
+                self.combo.add(MachineConfig('remote', remote_config))
+                self.combo.add(MachineConfig('service', service_config))
+            elif v.mode == 'local':
+                local_config = self.config['local'].copy()
+                local_config['alias'] = v.releases[0]
+                self.combo.add(MachineConfig('local', local_config))
+
+    def _filter_scn_by_tags(self):
+        filtered_suite = []
+        for scn in self.scn_variants:
+            # Add scenario name,file,dir,mode and releases as implicit tags
+            scn_tags = set(scn.name.split('.'))
+            scn_tags.add(scn.mode)
+            scn_tags.update(scn.releases)
+            scn_tags.update(getattr(scn, 'tags', set()))
+            matched_tags = scn_tags.intersection(self.tags)
+            if (
+                (matched_tags or not self.tags) and not
+                scn_tags.intersection(self.exclude_tags)
+            ):
+                filtered_suite.append(scn)
+        return filtered_suite
+
+    def setup(self):
+        self.scenarios = aggregator.all_scenarios()
+        self.scn_variants = []
+        # Generate all scenario variants
+        for scenario_cls in self.scenarios:
+            for mode in scenario_cls.mode:
+                if mode not in self.config:
+                    logger.debug(
+                        "Skipping a scenario: [{}] {}",
+                        mode, scenario_cls.name)
+                    continue
+                scn_config = scenario_cls.config_override
+                if mode == 'remote':
+                    try:
+                        remote_releases = scn_config['remote']['releases']
+                    except KeyError:
+                        remote_releases = self.config['remote']['releases']
+                    try:
+                        service_releases = scn_config['service']['releases']
+                    except KeyError:
+                        service_releases = self.config['service']['releases']
+                    for r_alias in self.config['remote']['releases']:
+                        if r_alias not in remote_releases:
+                            continue
+                        for s_alias in self.config['service']['releases']:
+                            if s_alias not in service_releases:
+                                continue
+                            self.scn_variants.append(
+                                scenario_cls(mode, r_alias, s_alias))
+                elif mode == 'local':
+                    try:
+                        local_releases = scn_config[mode]['releases']
+                    except KeyError:
+                        local_releases = self.config[mode]['releases']
+                    for alias in self.config[mode]['releases']:
+                        if alias not in local_releases:
+                            continue
+                        self.scn_variants.append(scenario_cls(mode, alias))
+        if self.args.tags or self.args.exclude_tags:
+            if self.args.tags:
+                logger.info('Including scenario tag(s): %s' % ', '.join(
+                    sorted(self.args.tags)))
+            if self.args.exclude_tags:
+                logger.info('Excluding scenario tag(s): %s' % ', '.join(
+                    sorted(self.args.exclude_tags)))
+            self.scn_variants = self._filter_scn_by_tags()
+            if not self.scn_variants:
+                logger.warning('No match found!')
+                raise SystemExit(1)
+        self._gather_all_machine_spec()
+        self.machine_provider = LxdMachineProvider(
+            self.config, self.combo,
+            self.debug_machine_setup, self.dispose)
+        self.machine_provider.setup()
+
+    def _load(self, mode, release_alias):
+        config = self.config[mode].copy()
+        config['alias'] = release_alias
+        config['role'] = mode
+        return self.machine_provider.get_machine_by_config(
+                        MachineConfig(mode, config))
+
+    def run(self):
+        startTime = time.perf_counter()
+        for scn in self.scn_variants:
+            if scn.mode == "remote":
+                scn.remote_machine = self._load("remote", scn.releases[0])
+                scn.service_machine = self._load("service", scn.releases[1])
+                scn.remote_machine.rollback_to('provisioned')
+                scn.service_machine.rollback_to('provisioned')
+                if scn.launcher:
+                    scn.remote_machine.put(scn.LAUNCHER_PATH, scn.launcher)
+                scn.service_machine.start_user_session()
+            elif scn.mode == "local":
+                scn.local_machine = self._load("local", scn.releases[0])
+                scn.local_machine.rollback_to('provisioned')
+                if scn.launcher:
+                    scn.local_machine.put(scn.LAUNCHER_PATH, scn.launcher)
+                scn.local_machine.start_user_session()
+            logger.info("Starting scenario: {}".format(scn.name))
+            scn.run()
+            if not scn.has_passed():
+                self.failed = True
+                logger.error("{} scenario has failed.".format(scn.name))
+                if self.hold_on_fail:
+                    if scn.mode == "remote":
+                        msg = (
+                            "You may hop onto the target machines by issuing "
+                            "the following commands:\n{}\n{}\n"
+                            "Press enter to continue testing").format(
+                                scn.remote_machine.get_connecting_cmd(),
+                                scn.service_machine.get_connecting_cmd())
+                    elif scn.mode == "local":
+                        msg = (
+                            "You may hop onto the target machine by issuing "
+                            "the following command:\n{}\n"
+                            "Press enter to continue testing").format(
+                                scn.local_machine.get_connecting_cmd())
+                    print(msg)
+                    input()
+            else:
+                logger.success("{} scenario has passed.".format(scn.name))
+            self.machine_provider.cleanup()
+        del self.machine_provider
+        stopTime = time.perf_counter()
+        timeTaken = stopTime - startTime
+        print('-' * 80)
+        total = len(self.scn_variants)
+        status = "Ran {} scenario{} in {:.3f}s".format(
+            total, total != 1 and "s" or "", timeTaken)
+        if self.wasSuccessful():
+            logger.success(status)
+        else:
+            logger.error(status)
+
+    def _run_single_scn(self, scenario_cls, mode, *releases):
+        pass
+
+    def wasSuccessful(self):
+        return self.failed is False
diff --git a/metabox/core/scenario.py b/metabox/core/scenario.py
 new file mode 100644
index 0000000..bbb7673
--- /dev/null
+++ b/metabox/core/scenario.py
@@ -0,0 +1,244 @@
+# This file is part of Checkbox.
+#
+# Copyright 2021 Canonical Ltd.
+# Written by:
+#   Maciej Kisielewski <[e-mail]>
+#   Sylvain Pineau <[e-mail]>
+#
+# Checkbox is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 3,
+# as published by the Free Software Foundation.
+#
+# Checkbox is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with Checkbox.  If not, see <http://www.gnu.org/licenses/>.
+"""
+This module defines the Scenario class.
+
+See Scenario class properties and the assert_* functions, as they serve as
+the interface to a Scenario.
+"""
+import re
+import time
+
+from metabox.core.actions import Start, Expect, Send, SelectTestPlan
+from metabox.core.aggregator import aggregator
+
+
+class Scenario:
+    """Definition of how to run a Checkbox session."""
+    config_override = {}
+    environment = {}
+    launcher = None
+    LAUNCHER_PATH = '/home/ubuntu/launcher.checkbox'
+
+    def __init_subclass__(cls, **kwargs):
+        super().__init_subclass__(**kwargs)
+        cls.name = '{}.{}'.format(cls.__module__, cls.__name__)
+        aggregator.add_scenario(cls)
+
+    def __init__(self, mode, *releases):
+        self.mode = mode
+        self.releases = releases
+        self._checks = []
+        self._ret_code = None
+        self._stdout = ''
+        self._stderr = ''
+        self._pts = None
+
+    def has_passed(self):
+        """Check whether all the assertions passed."""
+        return all(self._checks)
+
+    def run(self):
+        # Simple scenarios don't need to specify a START step
+        if not any([isinstance(s, Start) for s in self.steps]):
+            self.steps.insert(0, Start())
+        for i, step in enumerate(self.steps):
+            # Check how to start checkbox, interactively or not
+            if isinstance(step, Start):
+                interactive = False
+                # CHECK if any EXPECT/SEND command follows
+                # w/o a new call to START before it
+                for next_step in self.steps[i + 1:]:
+                    if isinstance(next_step, Start):
+                        break
+                    if isinstance(next_step, (Expect, Send, SelectTestPlan)):
+                        interactive = True
+                        break
+                step.kwargs['interactive'] = interactive
+            try:
+                step(self)
+            except TimeoutError:
+                self._checks.append(False)
+                break
+        if self._pts:
+            self._stdout = self._pts.stdout_data_full
+            # Mute the PTS since we're about to stop the machine to avoid
+            # getting empty log trace events
+            self._pts.verbose = False
+
+    def _assign_outcome(self, ret_code, stdout, stderr):
+        """Store remnants of a machine that run the scenario."""
+        self._ret_code = ret_code
+        self._stdout = stdout
+        self._stderr = stderr
+
+    # TODO: add storing of what actually failed in the assert methods
+    def assert_printed(self, pattern):
+        """
+        Check if during Checkbox execution a line produced that matches the
+        pattern.
+        :param patter: regular expresion to check against the lines.
+        """
+        regex = re.compile(pattern)
+        self._checks.append(bool(regex.search(self._stdout)))
+
+    def assert_not_printed(self, pattern):
+        """
+        Check if during Checkbox execution a line did not produced that matches
+        the pattern.
+        :param patter: regular expresion to check against the lines.
+        """
+        regex = re.compile(pattern)
+        if self._pts:
+            self._checks.append(
+                bool(not regex.search(self._pts.stdout_data_full.decode(
+                    'utf-8', errors='ignore'))))
+        else:
+            self._checks.append(bool(not regex.search(self._stdout)))
+
+    def assert_ret_code(self, code):
+        """Check if Checkbox returned given code."""
+        self._checks.append(code == self._ret_code)
+
+    def assertIn(self, member, container):
+        self._checks.append(member in container)
+
+    def assertEqual(self, first, second):
+        self._checks.append(first == second)
+
+    def assertNotEqual(self, first, second):
+        self._checks.append(first != second)
+
+    def start(self, cmd='', interactive=False, timeout=0):
+        if self.mode == 'remote':
+            outcome = self.start_all(interactive=interactive, timeout=timeout)
+            if interactive:
+                self._pts = outcome
+            else:
+                self._assign_outcome(*outcome)
+        else:
+            if self.launcher:
+                cmd = self.LAUNCHER_PATH
+            outcome = self.local_machine.start(
+                cmd=cmd, env=self.environment,
+                interactive=interactive, timeout=timeout)
+            if interactive:
+                self._pts = outcome
+            else:
+                self._assign_outcome(*outcome)
+
+    def start_all(self, interactive=False, timeout=0):
+        self.start_service()
+        outcome = self.start_remote(interactive, timeout)
+        if interactive:
+            self._pts = outcome
+        else:
+            self._assign_outcome(*outcome)
+        return outcome
+
+    def start_remote(self, interactive=False, timeout=0):
+        outcome = self.remote_machine.start_remote(
+            self.service_machine.address, self.LAUNCHER_PATH, interactive,
+            timeout=timeout)
+        if interactive:
+            self._pts = outcome
+        else:
+            self._assign_outcome(*outcome)
+        return outcome
+
+    def start_service(self, force=False):
+        return self.service_machine.start_service(force)
+
+    def expect(self, data, timeout=60):
+        assert(self._pts is not None)
+        outcome = self._pts.expect(data, timeout)
+        self._checks.append(outcome)
+
+    def send(self, data):
+        assert(self._pts is not None)
+        self._pts.send(data.encode('utf-8'), binary=True)
+
+    def sleep(self, secs):
+        time.sleep(secs)
+
+    def signal(self, signal):
+        assert(self._pts is not None)
+        self._pts.send_signal(signal)
+
+    def select_test_plan(self, testplan_id, timeout=60):
+        assert(self._pts is not None)
+        outcome = self._pts.select_test_plan(testplan_id, timeout)
+        self._checks.append(outcome)
+
+    def run_cmd(self, cmd, env={}, interactive=False, timeout=0, target='all'):
+        if self.mode == 'remote':
+            if target == 'remote':
+                self.remote_machine.run_cmd(cmd, env, interactive, timeout)
+            elif target == 'service':
+                self.service_machine.run_cmd(cmd, env, interactive, timeout)
+            else:
+                self.remote_machine.run_cmd(cmd, env, interactive, timeout)
+                self.service_machine.run_cmd(cmd, env, interactive, timeout)
+        else:
+            self.local_machine.run_cmd(cmd, env, interactive, timeout)
+
+    def reboot(self, timeout=0, target='all'):
+        if self.mode == 'remote':
+            if target == 'remote':
+                self.remote_machine.reboot(timeout)
+            elif target == 'service':
+                self.service_machine.reboot(timeout)
+            else:
+                self.remote_machine.reboot(timeout)
+                self.service_machine.reboot(timeout)
+        else:
+            self.local_machine.reboot(timeout)
+
+    def switch_on_networking(self, target='all'):
+        if self.mode == 'remote':
+            if target == 'remote':
+                self.remote_machine.switch_on_networking()
+            elif target == 'service':
+                self.service_machine.switch_on_networking()
+            else:
+                self.remote_machine.switch_on_networking()
+                self.service_machine.switch_on_networking()
+        else:
+            self.local_machine.switch_on_networking()
+
+    def switch_off_networking(self, target='all'):
+        if self.mode == 'remote':
+            if target == 'remote':
+                self.remote_machine.switch_off_networking()
+            elif target == 'service':
+                self.service_machine.switch_off_networking()
+            else:
+                self.remote_machine.switch_off_networking()
+                self.service_machine.switch_off_networking()
+        else:
+            self.local_machine.switch_off_networking()
+
+    def stop_service(self):
+        return self.service_machine.stop_service()
+
+    def reboot_service(self):
+        return self.service_machine.reboot_service()
+
+    def is_service_active(self):
+        return self.service_machine.is_service_active()
diff --git a/metabox/core/utils.py b/metabox/core/utils.py
 new file mode 100644
index 0000000..ed8c1b0
--- /dev/null
+++ b/metabox/core/utils.py
@@ -0,0 +1,49 @@
+# This file is part of Checkbox.
+#
+# Copyright 2021 Canonical Ltd.
+# Written by:
+#   Maciej Kisielewski <[e-mail]>
+#   Sylvain Pineau <[e-mail]>
+#
+# Checkbox is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 3,
+# as published by the Free Software Foundation.
+#
+# Checkbox is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with Checkbox.  If not, see <http://www.gnu.org/licenses/>.
+import re
+from typing import IO, NamedTuple
+
+__all__ = ('tag', 'ExecuteResult')
+
+
+def tag(*tags):
+    """Decorator to add tags to a scenario class."""
+    def decorator(obj):
+        setattr(obj, 'tags', set(tags))
+        return obj
+    return decorator
+
+
+class ExecuteResult(NamedTuple):
+
+    exit_code: int
+    stdout: IO
+    stderr: IO
+
+
+class _re:
+    def __init__(self, pattern, flags=0):
+        self._raw_pattern = pattern
+        self._pattern = re.compile(pattern.encode('utf-8'), flags)
+
+    def __repr__(self):
+        return f"Regex {self._raw_pattern}"
+
+    def search(self, data):
+        return bool(self._pattern.search(data))
diff --git a/metabox/lxd_profiles/checkbox.profile b/metabox/lxd_profiles/checkbox.profile
 new file mode 100644
index 0000000..673590a
--- /dev/null
+++ b/metabox/lxd_profiles/checkbox.profile
@@ -0,0 +1,35 @@
+config:
+  raw.idmap: "both 1000 1000"
+  environment.DISPLAY: :0
+  user.user-data: |
+    #cloud-config
+    runcmd:
+      - 'sed -i "s/; enable-shm = yes/enable-shm = no/g" /etc/pulse/client.conf'
+      - "perl -i -p0e 's/(Unit.*?)\n\n/$1\nConditionVirtualization=!container\n\n/s' /lib/systemd/system/systemd-remount-fs.service"
+      - 'echo export XAUTHORITY=/run/user/1000/gdm/Xauthority | tee --append /home/ubuntu/.profile'
+    apt:
+      sources:
+        stable_ppa:
+          source: "ppa:hardware-certification/public"
+    packages:
+      - alsa-base
+      - gir1.2-cheese-3.0
+      - gir1.2-gst-plugins-base-1.0
+      - gir1.2-gstreamer-1.0
+      - gstreamer1.0-plugins-good
+      - gstreamer1.0-pulseaudio
+      - libgstreamer1.0-0
+      - mesa-utils
+      - pulseaudio
+      - python3-jinja2
+      - python3-markupsafe
+      - python3-padme
+      - python3-pip
+      - python3-psutil
+      - python3-requests-oauthlib
+      - python3-tqdm
+      - python3-urwid
+      - python3-xlsxwriter
+      - virtualenv
+      - x11-apps
+      - xvfb
diff --git a/metabox/lxd_profiles/lowmem.profile b/metabox/lxd_profiles/lowmem.profile
 new file mode 100644
index 0000000..299f2db
--- /dev/null
+++ b/metabox/lxd_profiles/lowmem.profile
@@ -0,0 +1,3 @@
+config:
+  limits.cpu: "1"
+  limits.memory: 512MB
diff --git a/metabox/lxd_profiles/snap.profile b/metabox/lxd_profiles/snap.profile
 new file mode 100644
index 0000000..a26370e
--- /dev/null
+++ b/metabox/lxd_profiles/snap.profile
@@ -0,0 +1,24 @@
+config:
+  raw.idmap: "both 1000 1000"
+  environment.DISPLAY: :0
+  user.user-data: |
+    #cloud-config
+    runcmd:
+      - 'sed -i "s/; enable-shm = yes/enable-shm = no/g" /etc/pulse/client.conf'
+      - "perl -i -p0e 's/(Unit.*?)\n\n/$1\nConditionVirtualization=!container\n\n/s' /lib/systemd/system/systemd-remount-fs.service"
+      - 'echo export XAUTHORITY=/run/user/1000/gdm/Xauthority | tee --append /home/ubuntu/.profile'
+    packages:
+      - alsa-base
+      - gir1.2-cheese-3.0
+      - gir1.2-gst-plugins-base-1.0
+      - gir1.2-gstreamer-1.0
+      - gstreamer1.0-plugins-good
+      - gstreamer1.0-pulseaudio
+      - libgstreamer1.0-0
+      - mesa-utils
+      - pulseaudio
+      - x11-apps
+      - xvfb
+    snap:
+      commands:
+        - systemctl start snapd.service
diff --git a/metabox/main.py b/metabox/main.py
index 7f6de93..44daf47 100644
--- a/metabox/main.py
+++ b/metabox/main.py
@@ -1,10 +1,74 @@
+#!/usr/bin/env python3
+# This file is part of Checkbox.
+#
 # Copyright 2021 Canonical Ltd.
 # Written by:
 #   Maciej Kisielewski <[e-mail]>
 #   Sylvain Pineau <[e-mail]>
+#
+# Checkbox is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 3,
+# as published by the Free Software Foundation.
+#
+# Checkbox is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with Checkbox.  If not, see <http://www.gnu.org/licenses/>.
 """
 Entry point to the Metabox program.
 """
+import argparse
+import warnings
+from pathlib import Path
+
+from loguru._logger import Core
+from metabox.core.runner import Runner
+
 
 def main():
-    pass
+    """Entry point to Metabox."""
+
+    parser = argparse.ArgumentParser()
+    parser.add_argument(
+        'config', metavar='CONFIG', type=Path,
+        help='Metabox configuration file'
+    )
+    parser.add_argument(
+        '--tag', action='append', dest='tags',
+        help='Run only scenario with the specified tag. '
+             'Can be used multiple times.',
+    )
+    parser.add_argument(
+        '--exclude-tag', action='append', dest='exclude_tags',
+        help='Do not run scenario with the specified tag. '
+             'Can be used multiple times.',
+    )
+    parser.add_argument(
+        "--log", dest="log_level", choices=Core().levels.keys(),
+        default='SUCCESS',
+        help="Set the logging level",
+    )
+    parser.add_argument(
+        '--do-not-dispose', action='store_true',
+        help="Do not delete LXD containers after the run")
+    parser.add_argument(
+        '--hold-on-fail', action='store_true',
+        help="Pause testing when a scenario fails")
+    parser.add_argument(
+        '--debug-machine-setup', action='store_true',
+        help="Turn on verbosity during machine setup. "
+             "Only works with --log TRACE")
+    # Ignore warnings issued by pylxd/models/operation.py
+    with warnings.catch_warnings():
+        warnings.simplefilter("ignore")
+        runner = Runner(parser.parse_args())
+        runner.setup()
+        runner.run()
+        raise SystemExit(not runner.wasSuccessful())
+
+
+if __name__ == '__main__':
+    main()
diff --git a/metabox/metabox-provider/manage.py b/metabox/metabox-provider/manage.py
 new file mode 100755
index 0000000..7dc5255
--- /dev/null
+++ b/metabox/metabox-provider/manage.py
@@ -0,0 +1,9 @@
+#!/usr/bin/env python3
+from plainbox.provider_manager import setup, N_
+
+setup(
+    name='2021.com.canonical.certification:metabox',
+    version="1.0",
+    description=N_("The 2021.com.canonical.certification:metabox provider"),
+    gettext_domain="2021_com_canonical_certification_metabox",
+)
diff --git a/metabox/metabox-provider/units/basic-jobs.pxu b/metabox/metabox-provider/units/basic-jobs.pxu
 new file mode 100644
index 0000000..098288a
--- /dev/null
+++ b/metabox/metabox-provider/units/basic-jobs.pxu
@@ -0,0 +1,224 @@
+id: basic-shell-passing
+flags: simple
+command: true
+
+id: basic-shell-failing
+flags: simple
+command: false
+
+id: basic-environ-printing
+flags: simple
+command: echo foo: $foo
+environ: foo
+
+id: stub/manual
+_summary: A simple manual job
+_description:
+ PURPOSE:
+     This test checks that the manual plugin works fine
+ STEPS:
+     1. Add a comment
+     2. Set the result as passed
+ VERIFICATION:
+     Check that in the report the result is passed and the comment is displayed
+plugin: manual
+estimated_duration: 30
+
+id: stub/split-fields/manual
+_summary: A simple manual job using finer description fields
+_purpose:
+    This test checks that the manual plugin works fine
+_steps:
+    1. Add a comment
+    2. Set the result as passed
+_verification:
+    Check that in the report the result is passed and the comment is displayed
+plugin: manual
+estimated_duration: 30
+
+id: stub/user-interact
+_summary: A simple user interaction job
+_description:
+ PURPOSE:
+     This test checks that the user-interact plugin works fine
+ STEPS:
+     1. Read this description
+     2. Press the test button
+ VERIFICATION:
+     Check that in the report the result is passed
+plugin: user-interact
+flags: preserve-locale
+command: true
+estimated_duration: 30
+
+id: stub/split-fields/user-interact
+_summary: User-interact job using finer description fields
+_purpose:
+    This is a purpose part of test description
+_steps:
+    1. First step in the user-iteract job
+    2. Second step in the user-iteract job
+_verification:
+    Verification part of test description
+plugin: user-interact
+flags: preserve-locale
+command: true
+estimated_duration: 30
+
+id: stub/user-verify
+_summary: A simple user verification job
+_description:
+ PURPOSE:
+     This test checks that the user-verify plugin works fine
+ STEPS:
+     1. Read this description
+     2. Ensure that the command has been started automatically
+     3. Do not press the test button
+     4. Look at the output and determine the outcome of the test
+ VERIFICATION:
+     The command should have printed "Please select 'pass'"
+plugin: user-verify
+flags: preserve-locale
+command: echo "Please select 'pass'"
+estimated_duration: 30
+
+id: stub/split-fields/user-verify
+_summary: User-verify job using finer description fields
+_purpose:
+     This test checks that the user-verify plugin works fine and that
+     description field is split properly
+_steps:
+     1. Read this description
+     2. Ensure that the command has been started automatically
+     3. Do not press the test button
+     4. Look at the output and determine the outcome of the test
+_verification:
+     The command should have printed "Please select 'pass'"
+plugin: user-verify
+flags: preserve-locale
+command: echo "Please select 'pass'"
+estimated_duration: 30
+
+id: stub/user-interact-verify
+_summary: A simple user interaction and verification job
+_description:
+ PURPOSE:
+     This test checks that the user-interact-verify plugin works fine
+ STEPS:
+     1. Read this description
+     2. Ensure that the command has not been started yet
+     3. Press the test button
+     4. Look at the output and determine the outcome of the test
+ VERIFICATION:
+     The command should have printed "Please select 'pass'"
+plugin: user-interact-verify
+flags: preserve-locale
+command: echo "Please select 'pass'"
+estimated_duration: 25
+
+id: stub/split-fields/user-interact-verify
+_summary: A simple user interaction and verification job using finer
+    description fields
+_purpose:
+     This test checks that the user-interact-verify plugin works fine
+_steps:
+     1. Read this description
+     2. Ensure that the command has not been started yet
+     3. Press the test button
+     4. Look at the output and determine the outcome of the test
+_verification:
+     The command should have printed "Please select 'pass'"
+plugin: user-interact-verify
+flags: preserve-locale
+command: echo "Please select 'pass'"
+estimated_duration: 25
+
+id: stub/user-interact-verify-passing
+_summary: A suggested-passing user-verification-interaction job
+_description:
+ PURPOSE:
+     This test checks that the application user interface auto-suggests 'pass'
+     as the outcome of a test for user-interact-verify jobs that have a command
+     which completes successfully.
+ STEPS:
+     1. Read this description
+     2. Ensure that the command has not been started yet
+     3. Press the test button
+     4. Confirm the auto-suggested value
+ VERIFICATION:
+     The auto suggested value should have been 'pass'
+plugin: user-interact-verify
+flags: preserve-locale
+command: true
+estimated_duration: 25
+
+id: stub/split-fields/user-interact-verify-passing
+_summary: A suggested-passing user-verification-interaction job using finer
+    description fields
+_purpose:
+     This test checks that the application user interface auto-suggests 'pass'
+     as the outcome of a test for user-interact-verify jobs that have a command
+     which completes successfully.
+_steps:
+     1. Read this description
+     2. Ensure that the command has not been started yet
+     3. Press the test button
+     4. Confirm the auto-suggested value
+_verification:
+     The auto suggested value should have been 'pass'
+plugin: user-interact-verify
+flags: preserve-locale
+command: true
+estimated_duration: 25
+
+id: stub/user-interact-verify-failing
+_summary: A suggested-failing user-verification-interaction job
+_description:
+ PURPOSE:
+     This test checks that the application user interface auto-suggests 'fail'
+     as the outcome of a test for user-interact-verify jobs that have a command
+     which completes unsuccessfully.
+ STEPS:
+     1. Read this description
+     2. Ensure that the command has not been started yet
+     3. Press the test button
+     4. Confirm the auto-suggested value
+ VERIFICATION:
+     The auto suggested value should have been 'fail'
+plugin: user-interact-verify
+flags: preserve-locale
+command: false
+estimated_duration: 25
+
+id: stub/split-fields/user-interact-verify-failing
+_summary: A suggested-failing user-verification-interaction job using finer
+    description fields
+_purpose:
+     This test checks that the application user interface auto-suggests 'fail'
+     as the outcome of a test for user-interact-verify jobs that have a command
+     which completes unsuccessfully.
+_steps:
+     1. Read this description
+     2. Ensure that the command has not been started yet
+     3. Press the test button
+     4. Confirm the auto-suggested value
+_verification:
+     The auto suggested value should have been 'fail'
+plugin: user-interact-verify
+flags: preserve-locale
+command: false
+estimated_duration: 25
+
+plugin: user-interact-verify
+id: graphics/glxgears
+command: glxgears; true
+_summary: Test that glxgears works
+_description:
+ PURPOSE:
+     This test tests the basic 3D capabilities of your video card
+ STEPS:
+     1. Click "Test" to execute an OpenGL demo. Press ESC at any time to close.
+     2. Verify that the animation is not jerky or slow.
+ VERIFICATION:
+     1. Did the 3d animation appear?
+     2. Was the animation free from slowness/jerkiness?
diff --git a/metabox/metabox-provider/units/basic-tps.pxu b/metabox/metabox-provider/units/basic-tps.pxu
 new file mode 100644
index 0000000..6e2ef73
--- /dev/null
+++ b/metabox/metabox-provider/units/basic-tps.pxu
@@ -0,0 +1,39 @@
+unit: test plan
+id: basic-automated
+_name: Basic Automated
+include:
+ basic-shell-passing
+ basic-shell-failing
+ basic-environ-printing
+
+unit: test plan
+id: basic-automated-passing
+_name: Basic Automated Passing
+include:
+ basic-shell-passing
+
+unit: test plan
+id: basic-automated-failing
+_name: Basic Automated Failing
+include:
+ basic-shell-failing
+
+unit: test plan
+id: basic-manual
+_name: Basic Manual
+include:
+ stub/user-interact
+ stub/user-verify
+ stub/user-interact-verify
+
+unit: test plan
+id: display-manual
+_name: Display Manual
+include:
+  graphics/glxgears
+
+unit: test plan
+id: audio-manual
+_name: Audio Manual
+include:
+  com.canonical.certification::audio/playback_auto
diff --git a/metabox/scenarios/__init__.py b/metabox/scenarios/__init__.py
 new file mode 100644
index 0000000..e69de29
--- /dev/null
+++ b/metabox/scenarios/__init__.py
diff --git a/metabox/scenarios/basic/__init__.py b/metabox/scenarios/basic/__init__.py
 new file mode 100644
index 0000000..e69de29
--- /dev/null
+++ b/metabox/scenarios/basic/__init__.py
diff --git a/metabox/scenarios/basic/run-invocation.py b/metabox/scenarios/basic/run-invocation.py
 new file mode 100644
index 0000000..fc2d198
--- /dev/null
+++ b/metabox/scenarios/basic/run-invocation.py
@@ -0,0 +1,86 @@
+# This file is part of Checkbox.
+#
+# Copyright 2021 Canonical Ltd.
+# Written by:
+#   Maciej Kisielewski <[e-mail]>
+#   Sylvain Pineau <[e-mail]>
+#
+# Checkbox is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 3,
+# as published by the Free Software Foundation.
+
+#
+# Checkbox is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with Checkbox.  If not, see <http://www.gnu.org/licenses/>.
+import metabox.core.keys as keys
+from metabox.core.actions import AssertPrinted
+from metabox.core.actions import AssertRetCode
+from metabox.core.actions import Expect
+from metabox.core.actions import Send
+from metabox.core.actions import Start
+from metabox.core.scenario import Scenario
+
+
+class RunTestplan(Scenario):
+
+    mode = ['local']
+    config_override = {'local': {'releases': ['bionic']}}
+    steps = [
+        Start('run 2021.com.canonical.certification::'
+              'basic-automated-passing', timeout=30),
+        AssertRetCode(0)
+    ]
+
+
+class RunFailingTestplan(Scenario):
+
+    mode = ['local']
+    steps = [
+        Start('run 2021.com.canonical.certification::'
+              'basic-automated-failing', timeout=30),
+        AssertRetCode(0)
+    ]
+
+
+class RunTestplanWithEnvvar(Scenario):
+
+    mode = ['local']
+    environment = {'foo': 42}
+    steps = [
+        Start('run 2021.com.canonical.certification::basic-automated',
+              timeout=30),
+        AssertPrinted("foo: 42"),
+    ]
+
+
+class RunTestplanWithTimeout(Scenario):
+
+    mode = ['local']
+    steps = [
+        Start('run 2021.com.canonical.certification::'
+              'basic-automated-passing', timeout='0.1s'),
+        AssertRetCode(137)
+    ]
+
+
+class RunManualplan(Scenario):
+
+    mode = ['local']
+    steps = [
+        Start('run 2021.com.canonical.certification::basic-manual'),
+        Expect('Pick an action'),
+        Send(keys.KEY_ENTER),
+        Expect('Pick an action', timeout=30),
+        Send('p' + keys.KEY_ENTER),
+        Expect('Pick an action'),
+        Send(keys.KEY_ENTER),
+        Expect('Pick an action'),
+        Send('p' + keys.KEY_ENTER),
+        Expect(' ☑ : '
+               'A simple user interaction and verification job'),
+    ]
diff --git a/metabox/scenarios/desktop_env/launcher.py b/metabox/scenarios/desktop_env/launcher.py
 new file mode 100644
index 0000000..8711e74
--- /dev/null
+++ b/metabox/scenarios/desktop_env/launcher.py
@@ -0,0 +1,76 @@
+# This file is part of Checkbox.
+#
+# Copyright 2021 Canonical Ltd.
+# Written by:
+#   Maciej Kisielewski <[e-mail]>
+#   Sylvain Pineau <[e-mail]>
+#
+# Checkbox is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 3,
+# as published by the Free Software Foundation.
+
+#
+# Checkbox is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with Checkbox.  If not, see <http://www.gnu.org/licenses/>.
+import textwrap
+
+import metabox.core.keys as keys
+from metabox.core.actions import AssertNotPrinted
+from metabox.core.actions import Expect
+from metabox.core.actions import RunCmd
+from metabox.core.actions import Send
+from metabox.core.scenario import Scenario
+from metabox.core.utils import _re
+
+
+class GlxGears(Scenario):
+
+    mode = ['local', 'remote']
+    launcher = textwrap.dedent("""
+    [launcher]
+    launcher_version = 1
+    stock_reports = text
+    [test plan]
+    unit = 2021.com.canonical.certification::display-manual
+    forced = yes
+    [test selection]
+    forced = yes
+    """)
+    steps = [
+        Expect('Pick an action'),
+        Send(keys.KEY_ENTER),
+        Expect('FPS'),
+        RunCmd('pkill -2 glxgears'),
+        Expect('Pick an action'),
+        Send('p' + keys.KEY_ENTER),
+        Expect(_re('(☑|job passed).*Test that glxgears works')),
+    ]
+
+
+class AudioPlayback(Scenario):
+
+    mode = ['local', 'remote']
+    launcher = textwrap.dedent("""
+    [launcher]
+    launcher_version = 1
+    stock_reports = text
+    [test plan]
+    unit = 2021.com.canonical.certification::audio-manual
+    forced = yes
+    [test selection]
+    forced = yes
+    """)
+    steps = [
+        Expect('Pick an action'),
+        Send(keys.KEY_ENTER),
+        Expect('Pipeline initialized, now starting playback.'),
+        Expect('Pick an action'),
+        Send('p' + keys.KEY_ENTER),
+        AssertNotPrinted('Connection failure: Connection refused'),
+        Expect(_re('(☑|job passed).*audio/playback_auto'), timeout=10),
+    ]
diff --git a/metabox/scenarios/restart/launcher.py b/metabox/scenarios/restart/launcher.py
 new file mode 100644
index 0000000..57d13d1
--- /dev/null
+++ b/metabox/scenarios/restart/launcher.py
@@ -0,0 +1,46 @@
+# This file is part of Checkbox.
+#
+# Copyright 2021 Canonical Ltd.
+# Written by:
+#   Maciej Kisielewski <[e-mail]>
+#   Sylvain Pineau <[e-mail]>
+#
+# Checkbox is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 3,
+# as published by the Free Software Foundation.
+
+#
+# Checkbox is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with Checkbox.  If not, see <http://www.gnu.org/licenses/>.
+import textwrap
+
+from metabox.core.actions import AssertPrinted
+from metabox.core.scenario import Scenario
+
+
+class Reboot(Scenario):
+
+    mode = ['remote']
+    launcher = textwrap.dedent("""
+    [launcher]
+    launcher_version = 1
+    stock_reports = text
+    [test plan]
+    unit = com.canonical.certification::power-automated
+    forced = yes
+    [test selection]
+    forced = yes
+    exclude = .*cold.*
+    [ui]
+    type = silent
+    """)
+    steps = [
+        AssertPrinted('Connection lost!'),
+        AssertPrinted('Reconnecting...'),
+        AssertPrinted('job passed   : Warm reboot'),
+    ]
diff --git a/metabox/scenarios/urwid/run-invocation.py b/metabox/scenarios/urwid/run-invocation.py
 new file mode 100644
index 0000000..e8af2ce
--- /dev/null
+++ b/metabox/scenarios/urwid/run-invocation.py
@@ -0,0 +1,111 @@
+# This file is part of Checkbox.
+#
+# Copyright 2021 Canonical Ltd.
+# Written by:
+#   Maciej Kisielewski <[e-mail]>
+#   Sylvain Pineau <[e-mail]>
+#
+# Checkbox is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 3,
+# as published by the Free Software Foundation.
+
+#
+# Checkbox is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with Checkbox.  If not, see <http://www.gnu.org/licenses/>.
+import metabox.core.keys as keys
+from metabox.core.actions import AssertPrinted
+from metabox.core.actions import AssertRetCode
+from metabox.core.actions import Expect
+from metabox.core.actions import Send
+from metabox.core.actions import SelectTestPlan
+from metabox.core.actions import Start
+from metabox.core.scenario import Scenario
+
+
+class RunTestplan(Scenario):
+
+    mode = ['local']
+    config_override = {'local': {'releases': ['bionic']}}
+    steps = [
+        Start('run 2021.com.canonical.certification::'
+              'basic-automated-passing', timeout=30),
+        AssertRetCode(0)
+    ]
+
+
+class RunFailingTestplan(Scenario):
+
+    mode = ['local']
+    steps = [
+        Start('run 2021.com.canonical.certification::'
+              'basic-automated-failing', timeout=30),
+        AssertRetCode(0)
+    ]
+
+
+class RunTestplanWithEnvvar(Scenario):
+
+    mode = ['local']
+    environment = {'foo': 42}
+    steps = [
+        Start('run 2021.com.canonical.certification::basic-automated',
+              timeout=30),
+        AssertPrinted("foo: 42"),
+    ]
+
+
+class RunTestplanWithTimeout(Scenario):
+
+    mode = ['local']
+    steps = [
+        Start('run 2021.com.canonical.certification::'
+              'basic-automated-passing', timeout='0.1s'),
+        AssertRetCode(137)
+    ]
+
+
+class RunManualplan(Scenario):
+
+    mode = ['local']
+    steps = [
+        Start('run 2021.com.canonical.certification::basic-manual'),
+        Expect('Pick an action'),
+        Send(keys.KEY_ENTER),
+        Expect('Pick an action', timeout=30),
+        Send('p' + keys.KEY_ENTER),
+        Expect('Pick an action'),
+        Send(keys.KEY_ENTER),
+        Expect('Pick an action'),
+        Send('p' + keys.KEY_ENTER),
+        Expect(' ☑ : '
+               'A simple user interaction and verification job'),
+    ]
+
+
+class UrwidTestPlanSelection(Scenario):
+
+    mode = ['local']
+    steps = [
+        Expect('Select test plan'),
+        SelectTestPlan('com.canonical.certification::stress-pm-graph'),
+        SelectTestPlan(
+            'com.canonical.certification::'
+            'after-suspend-graphics-discrete-gpu-cert-automated'),
+        SelectTestPlan('com.canonical.certification::client-cert-18-04'),
+        Send(keys.KEY_ENTER),
+        Expect('Choose tests to run on your system:'),
+        Send('d' + keys.KEY_ENTER),
+        Expect('Choose tests to run on your system:'),
+        Send(keys.KEY_DOWN * 18 + keys.KEY_SPACE + 't'),
+        Expect('System Manifest:'),
+        Send('y' * 9 + 't'),
+        Expect('Pick an action'),
+        Send('s' + keys.KEY_ENTER),
+        Expect('Finish'),
+        Send('f' + keys.KEY_ENTER),
+    ]
diff --git a/setup.py b/setup.py
index 95250eb..bac43ad 100755
--- a/setup.py
+++ b/setup.py
@@ -1,22 +1,37 @@
 #!/usr/bin/env python3
+# This file is part of Checkbox.
+#
 # Copyright 2021 Canonical Ltd.
 # Written by:
 #   Maciej Kisielewski <[e-mail]>
+#   Sylvain Pineau <[e-mail]>
+#
+# Checkbox is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 3,
+# as published by the Free Software Foundation.
+#
+# Checkbox is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with Checkbox.  If not, see <http://www.gnu.org/licenses/>.
 
 from setuptools import find_packages, setup
 
 setup(
     name='metabox',
-    version='0.1',
+    version='0.3',
     packages=find_packages(),
-    install_requires=['pylxd'],
+    install_requires=['loguru', 'pylxd', 'pyyaml'],
     entry_points={
         'console_scripts': [
             'metabox = metabox.main:main',
         ]
     },
-    include_package_data = True,
-    package_data = {
+    include_package_data=True,
+    package_data={
         '': ['metabox/metabox-provider/*'],
     }
 )