Merge lp:~sylvain-pineau/checkbox/checkbox-ng-sa into lp:checkbox

[Sylvain Pineau]

checkbox-cli rewritten using plainbox SessionAssistant as a standalone tool (called checkbox-cli2 to avoid conflicts)

[Zygmunt Krynicki (zyga)]

Hey. Thanks for sending this. I made several small comments below. I like the direction this is going but I'd like to overall streamline the code. I realize we cherry picked functions from various locations but now we have a chance to simplify and unify this so that it is easier to understand.

I would also like to know exactly which features we aim to support here (in other words, this is not a launcher interpreter). It would be good if we could write this down in the docstring of the module.

[Sylvain Pineau (sylvain-pineau)]

Ready for review, I put the app in the launcher folder to keep it in checkbox-ng for packaging needs

review: Needs Resubmitting

[Maciej Kisielewski (kissiel)]

My only -1 is on the `._export_results`. I really believe that talking to plainbox internals behind SessionAssistant could be avoided. (See inline comment).

Building tree (create_simple_tree) smells bad, the nested coprehension in the loop.
But after understanding how the code works I guess that's the cleanest way (reading-wise) of doing it, and the list of categories in the root node *should* be small.

Super minor thing: rev 4121 ‘aa’ typo in commit msg

review: Needs Fixing

[lp:~sylvain-pineau/checkbox/checkbox-ng-sa]

4121. By Sylvain Pineau on 2015-12-02

plainbox:session:assistant: Reset running_job_name when resuming with autorestart

When a session is resumed in autorestart mode, the running_job_name meta
data has to be reset and local self._metadata returned instead of the initial
self._resume_candidates[session_id].metadata.

4122. By Sylvain Pineau on 2015-12-02

plainbox:session:assistant: Create a checkpoint when calling use_job_result()

4123. By Sylvain Pineau on 2015-12-02

plainbox:session:restart: Append $SHELL to xdg desktop file command

To avoid closing the terminal when the resume command is over.

4124. By Sylvain Pineau on 2015-12-02

plainbox:session:assistant: Set FLAG_SUBMITTED after calling transport.send()

SessionAssistant export_to_transport() should set the flag FLAG_SUBMITTED if
the call to transport.send() succeeded.

4125. By Sylvain Pineau on 2015-12-02

checkbox-ng:checkbox-cli2: New CLI application based on Session Assistant

Not a true launcher but a standalone application aimed to replace checkbox-cli.
It relies on plainbox new Session Assistant to perform all low level operations.

The main differences between this version and current checkbox-cli are:
 - support for restart API
 - jobs are grouped by categories units (as opposed to Job.via links) in test
   selection screens.

[Sylvain Pineau (sylvain-pineau)]

_export_results fixed to use sa export to stream. and rev 4121 typo

review: Needs Resubmitting

[Sylvain Pineau (sylvain-pineau)]

Time to fix ppa packaging now

=== modified file 'checkbox-ng/checkbox_ng/commands/sru.py'
--- checkbox-ng/checkbox_ng/commands/sru.py	2015-09-07 13:40:15 +0000
+++ checkbox-ng/checkbox_ng/commands/sru.py	2015-12-02 09:12:28 +0000
@@ -128,7 +128,7 @@
 
     def _collect_info(self, rc, sa):
         sa.select_providers('*')
-        sa.start_new_session(_("Hardware Collection Session"))
+        sa.start_new_session(_("SRU Session"))
         sa.select_test_plan(self.config.test_plan)
         sa.bootstrap()
         for job_id in sa.get_static_todo_list():

=== modified file 'checkbox-ng/checkbox_ng/misc.py'
--- checkbox-ng/checkbox_ng/misc.py	2015-10-13 13:41:17 +0000
+++ checkbox-ng/checkbox_ng/misc.py	2015-12-02 09:12:28 +0000
@@ -145,6 +145,28 @@
             builder.auto_add_job(job)
         return builder.root_node
 
+    @classmethod
+    def create_simple_tree(cls, sa, job_list):
+        """
+        Build a rooted JobTreeNode from a job list.
+
+        :argument sa:
+            A session assistant object
+        :argument job_list:
+            List of jobs to consider for building the tree.
+        """
+        root_node = cls()
+        for job in job_list:
+            cat_name = sa.get_category(job.category_id).tr_name()
+            matches = [n for n in root_node.categories if n.name == cat_name]
+            if not matches:
+                node = cls(cat_name)
+                root_node.add_category(node)
+            else:
+                node = matches[0]
+            node.add_job(job)
+        return root_node
+
 
 class TreeBuilder:
 

=== modified file 'checkbox-ng/checkbox_ng/ui.py'
--- checkbox-ng/checkbox_ng/ui.py	2015-06-10 15:04:50 +0000
+++ checkbox-ng/checkbox_ng/ui.py	2015-12-02 09:12:28 +0000
@@ -77,12 +77,13 @@
     """
     Display the appropriate menu and return the selected options
     """
-    def __init__(self, title, menu, selection=[0]):
+    def __init__(self, title, menu, selection=[0], multiple_allowed=True):
         self.image = TextImage(Size(0, 0))
         self.title = title
         self.menu = menu
         self.option_count = len(menu)
         self.position = 0  # Zero-based index of the selected menu option
+        self.multiple_allowed = multiple_allowed
         if self.option_count:
             self.selection = selection
         else:
@@ -110,6 +111,8 @@
                     self.selection.remove(self.position)
                 elif self.position < self.option_count:
                     self.selection.append(self.position)
+                    if not self.multiple_allowed:
+                        self.selection = [self.position]
         self.repaint(event)
         return self.image
 

=== added file 'checkbox-ng/launchers/checkbox-cli2'
--- checkbox-ng/launchers/checkbox-cli2	1970-01-01 00:00:00 +0000
+++ checkbox-ng/launchers/checkbox-cli2	2015-12-02 09:12:28 +0000
@@ -0,0 +1,596 @@
+#!/usr/bin/env python3
+# This file is part of Checkbox.
+#
+# Copyright 2015 Canonical Ltd.
+# Written by:
+#   Sylvain Pineau <[e-mail]>
+#
+# Checkbox is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 3,
+# as published by the Free Software Foundation.
+#
+# Checkbox is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with Checkbox.  If not, see <http://www.gnu.org/licenses/>.
+
+"""
+Checkbox CLI Application.
+
+WARNING: this is not a launcher interpreter.
+"""
+
+from argparse import SUPPRESS
+from shutil import copyfileobj
+import gettext
+import io
+import json
+import logging
+import os
+import sys
+
+from guacamole import Command
+from guacamole.core import Ingredient
+from guacamole.ingredients import ansi
+from guacamole.ingredients import argparse
+from guacamole.ingredients import cmdtree
+from guacamole.recipes.cmd import CommandRecipe
+
+# TODO: use public APIs here
+from plainbox.abc import IJobResult
+from plainbox.i18n import ngettext
+from plainbox.i18n import pgettext as C_
+from plainbox.impl.commands.inv_run import seconds_to_human_duration
+from plainbox.impl.commands.inv_run import Action
+from plainbox.impl.commands.inv_run import ActionUI
+from plainbox.impl.commands.inv_run import NormalUI
+from plainbox.impl.commands.inv_run import ReRunJob
+from plainbox.impl.color import Colorizer
+from plainbox.impl.exporter import ByteStringStreamTranslator
+from plainbox.impl.ingredients import CanonicalCrashIngredient
+from plainbox.impl.ingredients import RenderingContextIngredient
+from plainbox.impl.ingredients import SessionAssistantIngredient
+from plainbox.impl.result import tr_outcome
+from plainbox.impl.result import JobResultBuilder
+from plainbox.impl.result import MemoryJobResult
+from plainbox.impl.session.assistant import SA_RESTARTABLE
+from plainbox.impl.session.jobs import InhibitionCause
+from plainbox.vendor.textland import get_display
+
+from checkbox_ng.misc import SelectableJobTreeNode
+from checkbox_ng.ui import ScrollableTreeNode
+from checkbox_ng.ui import ShowMenu
+from checkbox_ng.ui import ShowRerun
+
+
+_ = gettext.gettext
+
+_logger = logging.getLogger("checkbox-cli")
+
+
+class DisplayIngredient(Ingredient):
+
+    """Ingredient that adds a Textland display to guacamole."""
+
+    def late_init(self, context):
+        """Add a DisplayIngredient as ``display`` to the guacamole context."""
+        context.display = get_display()
+
+
+class CheckboxCommandRecipe(CommandRecipe):
+
+    """A recipe for using Checkbox-enhanced commands."""
+
+    def get_ingredients(self):
+        """Get a list of ingredients for guacamole."""
+        return [
+            cmdtree.CommandTreeBuilder(self.command),
+            cmdtree.CommandTreeDispatcher(),
+            argparse.ParserIngredient(),
+            CanonicalCrashIngredient(),
+            ansi.ANSIIngredient(),
+            SessionAssistantIngredient(),
+            RenderingContextIngredient(),
+            DisplayIngredient()
+        ]
+
+
+class CheckboxUI(NormalUI):
+
+    def considering_job(self, job, job_state):
+        pass
+
+
+class CheckboxCommand(Command):
+
+    """
+    A command with Checkbox-enhanced ingredients.
+
+    This command has two additional items in the guacamole execution context,
+    the :class:`DisplayIngredient` object ``display`` and the
+    :class:`SessionAssistant` object ``sa``.
+    """
+
+    bug_report_url = "https://bugs.launchpad.net/checkbox-ng/+filebug"
+
+    def main(self, argv=None, exit=True):
+        """
+        Shortcut for running a command.
+
+        See :meth:`guacamole.recipes.Recipe.main()` for details.
+        """
+        return CheckboxCommandRecipe(self).main(argv, exit)
+
+
+class checkbox_cli(CheckboxCommand):
+
+    """Tool to run Checkbox jobs interactively from the command line."""
+
+    app_id = 'checkbox-cli'
+
+    def get_sa_api_version(self):
+        return '0.99'
+
+    def get_sa_api_flags(self):
+        return (SA_RESTARTABLE,)
+
+    def register_arguments(self, parser):
+        """Method called to register command line arguments."""
+        parser.add_argument(
+            '-t', '--test-plan', action="store", metavar=_("TEST-PLAN-ID"),
+            default=None,
+            # TRANSLATORS: this is in imperative form
+            help=_("load the specified test plan"))
+        parser.add_argument(
+            '--secure_id', metavar="SECURE_ID",
+            help=_("Canonical hardware identifier (optional)"))
+        parser.add_argument(
+            '--non-interactive', action='store_true',
+            help=_("skip tests that require interactivity"))
+        parser.add_argument(
+            '--dont-suppress-output', action="store_true", default=False,
+            help=_("don't suppress the output of certain job plugin types"))
+        parser.add_argument(
+            '--staging', action='store_true', default=False,
+            # Hide staging from help message (See pad.lv/1350005)
+            help=SUPPRESS)
+        parser.add_argument(
+            '--resume', dest='session_id', metavar="SESSION_ID",
+            help=SUPPRESS)
+
+    def invoked(self, ctx):
+        """Method called when the command is invoked."""
+        self.ctx = ctx
+        self.transport = self._create_transport()
+        self.C = Colorizer()
+        try:
+            self._do_normal_sequence()
+            self._export_results()
+            if ctx.args.secure_id:
+                self._send_results()
+            ctx.sa.finalize_session()
+        except KeyboardInterrupt:
+            return 1
+
+    def _export_results(self):
+        if self.is_interactive:
+            print(self.C.header(_("Results")))
+            # This requires a bit more finesse, as exporters output bytes
+            # and stdout needs a string.
+            translating_stream = ByteStringStreamTranslator(
+                sys.stdout, "utf-8")
+            self.ctx.sa.export_to_stream(
+                '2013.com.canonical.plainbox::text', (), translating_stream)
+        base_dir = os.path.join(
+            os.getenv(
+                'XDG_DATA_HOME', os.path.expanduser("~/.local/share/")),
+            "checkbox-ng")
+        if not os.path.exists(base_dir):
+            os.makedirs(base_dir)
+        exp_options = ['with-sys-info', 'with-summary', 'with-job-description',
+                       'with-text-attachments', 'with-certification-status',
+                       'with-job-defs', 'with-io-log', 'with-comments']
+        exporters = [
+            '2013.com.canonical.plainbox::hexr',
+            '2013.com.canonical.plainbox::html',
+            '2013.com.canonical.plainbox::xlsx',
+            '2013.com.canonical.plainbox::json',
+        ]
+        print()
+        for unit_name in exporters:
+            results_path = self.ctx.sa.export_to_file(unit_name, exp_options,
+                                                      base_dir)
+            print("file://{}".format(results_path))
+
+    def _send_results(self):
+        print()
+        print(_("Sending hardware report to Canonical Certification"))
+        print(_("Server URL is: {0}").format(self.transport.url))
+        result = self.ctx.sa.export_to_transport(
+            "2013.com.canonical.plainbox::hexr", self.transport)
+        if 'url' in result:
+            print(result['url'])
+
+    def _create_transport(self):
+        if self.ctx.args.secure_id:
+            return self.ctx.sa.get_canonical_certification_transport(
+                self.ctx.args.secure_id, staging=self.ctx.args.staging)
+
+    def _get_interactively_picked_testplans(self):
+        test_plan_ids = self.ctx.sa.get_test_plans()
+        test_plan_names = [self.ctx.sa.get_test_plan(tp_id).name for tp_id in
+                           test_plan_ids]
+        try:
+            selected_index = self.ctx.display.run(
+                ShowMenu(_("Select test plan"),
+                         test_plan_names, [],
+                         multiple_allowed=False))[0]
+        except IndexError:
+            return None
+        return test_plan_ids[selected_index]
+
+    def _interactively_pick_jobs_to_run(self):
+        job_list = [self.ctx.sa.get_job(job_id) for job_id in
+                    self.ctx.sa.get_static_todo_list()]
+        tree = SelectableJobTreeNode.create_simple_tree(self.ctx.sa, job_list)
+        title = _('Choose tests to run on your system:')
+        self.ctx.display.run(ScrollableTreeNode(tree, title))
+        # NOTE: tree.selection is correct but ordered badly. To retain
+        # the original ordering we should just treat it as a mask and
+        # use it to filter jobs from get_static_todo_list.
+        wanted_set = frozenset([job.id for job in tree.selection])
+        job_id_list = [job_id for job_id in self.ctx.sa.get_static_todo_list()
+                       if job_id in wanted_set]
+        self.ctx.sa.use_alternate_selection(job_id_list)
+
+    @property
+    def is_interactive(self):
+        """
+        Flag indicating that this is an interactive invocation.
+
+        We can then interact with the user when we encounter OUTCOME_UNDECIDED.
+        """
+        return (sys.stdin.isatty() and sys.stdout.isatty() and not
+                self.ctx.args.non_interactive)
+
+    def _get_ui_for_job(self, job):
+        if self.ctx.args.dont_suppress_output is False and job.plugin in (
+                'local', 'resource', 'attachment'):
+            return CheckboxUI(self.C.c, show_cmd_output=False)
+        else:
+            return CheckboxUI(self.C.c, show_cmd_output=True)
+
+    def _run_single_job_with_ui_loop(self, job, ui):
+        print(self.C.header(job.tr_summary(), fill='-'))
+        print(_("ID: {0}").format(job.id))
+        print(_("Category: {0}").format(
+            self.ctx.sa.get_job_state(job.id).effective_category_id))
+        comments = ""
+        while True:
+            if job.plugin in ('user-interact', 'user-interact-verify',
+                              'user-verify', 'manual'):
+                ui.notify_about_purpose(job)
+                if (self.is_interactive and
+                        job.plugin in ('user-interact',
+                                       'user-interact-verify',
+                                       'manual')):
+                    ui.notify_about_steps(job)
+                    if job.plugin == 'manual':
+                        cmd = 'run'
+                    else:
+                        cmd = ui.wait_for_interaction_prompt(job)
+                    if cmd == 'run' or cmd is None:
+                        result_builder = self.ctx.sa.run_job(job.id, ui, False)
+                    elif cmd == 'comment':
+                        new_comment = input(self.C.BLUE(
+                            _('Please enter your comments:') + '\n'))
+                        if new_comment:
+                            comments += new_comment + '\n'
+                        continue
+                    elif cmd == 'skip':
+                        result_builder = JobResultBuilder(
+                            outcome=IJobResult.OUTCOME_SKIP,
+                            comments=_("Explicitly skipped before"
+                                       " execution"))
+                        if comments != "":
+                            result_builder.comments = comments
+                        break
+                    elif cmd == 'quit':
+                        raise SystemExit()
+                else:
+                    result_builder = self.ctx.sa.run_job(job.id, ui, False)
+            else:
+                if 'noreturn' in job.get_flag_set():
+                    ui.noreturn_job()
+                result_builder = self.ctx.sa.run_job(job.id, ui, False)
+            if (self.is_interactive and
+                    result_builder.outcome == IJobResult.OUTCOME_UNDECIDED):
+                try:
+                    if comments != "":
+                        result_builder.comments = comments
+                    ui.notify_about_verification(job)
+                    self._interaction_callback(job, result_builder)
+                except ReRunJob:
+                    self.ctx.sa.use_job_result(job.id,
+                                               result_builder.get_result())
+                    continue
+            break
+        return result_builder
+
+    def _pick_action_cmd(self, action_list, prompt=None):
+        return ActionUI(action_list, prompt).run()
+
+    def _interaction_callback(self, job, result_builder,
+                              prompt=None, allowed_outcome=None):
+        result = result_builder.get_result()
+        if prompt is None:
+            prompt = _("Select an outcome or an action: ")
+        if allowed_outcome is None:
+            allowed_outcome = [IJobResult.OUTCOME_PASS,
+                               IJobResult.OUTCOME_FAIL,
+                               IJobResult.OUTCOME_SKIP]
+        allowed_actions = [
+            Action('c', _('add a comment'), 'set-comments')
+        ]
+        if IJobResult.OUTCOME_PASS in allowed_outcome:
+            allowed_actions.append(
+                Action('p', _('set outcome to {0}').format(
+                    self.C.GREEN(C_('set outcome to <pass>', 'pass'))),
+                    'set-pass'))
+        if IJobResult.OUTCOME_FAIL in allowed_outcome:
+            allowed_actions.append(
+                Action('f', _('set outcome to {0}').format(
+                    self.C.RED(C_('set outcome to <fail>', 'fail'))),
+                    'set-fail'))
+        if IJobResult.OUTCOME_SKIP in allowed_outcome:
+            allowed_actions.append(
+                Action('s', _('set outcome to {0}').format(
+                    self.C.YELLOW(C_('set outcome to <skip>', 'skip'))),
+                    'set-skip'))
+        if job.command is not None:
+            allowed_actions.append(
+                Action('r', _('re-run this job'), 're-run'))
+        if result.return_code is not None:
+            if result.return_code == 0:
+                suggested_outcome = IJobResult.OUTCOME_PASS
+            else:
+                suggested_outcome = IJobResult.OUTCOME_FAIL
+            allowed_actions.append(
+                Action('', _('set suggested outcome [{0}]').format(
+                    tr_outcome(suggested_outcome)), 'set-suggested'))
+        while result.outcome not in allowed_outcome:
+            print(_("Please decide what to do next:"))
+            print("  " + _("outcome") + ": {0}".format(
+                self.C.result(result)))
+            if result.comments is None:
+                print("  " + _("comments") + ": {0}".format(
+                    C_("none comment", "none")))
+            else:
+                print("  " + _("comments") + ": {0}".format(
+                    self.C.CYAN(result.comments, bright=False)))
+            cmd = self._pick_action_cmd(allowed_actions)
+            if cmd == 'set-pass':
+                result_builder.outcome = IJobResult.OUTCOME_PASS
+            elif cmd == 'set-fail':
+                result_builder.outcome = IJobResult.OUTCOME_FAIL
+            elif cmd == 'set-skip' or cmd is None:
+                result_builder.outcome = IJobResult.OUTCOME_SKIP
+            elif cmd == 'set-suggested':
+                result_builder.outcome = suggested_outcome
+            elif cmd == 'set-comments':
+                new_comment = input(self.C.BLUE(
+                    _('Please enter your comments:') + '\n'))
+                if new_comment:
+                    result_builder.add_comment(new_comment)
+            elif cmd == 're-run':
+                raise ReRunJob
+            result = result_builder.get_result()
+
+    def _run_jobs(self, jobs_to_run):
+        estimated_time = 0
+        for job_id in jobs_to_run:
+            job = self.ctx.sa.get_job(job_id)
+            if (job.estimated_duration is not None
+                    and estimated_time is not None):
+                estimated_time += job.estimated_duration
+            else:
+                estimated_time = None
+        for job_no, job_id in enumerate(jobs_to_run, start=1):
+            print(self.C.header(
+                _('Running job {} / {}. Estimated time left: {}').format(
+                    job_no, len(jobs_to_run),
+                    seconds_to_human_duration(max(0, estimated_time))
+                    if estimated_time is not None else _("unknown")),
+                fill='-'))
+            job = self.ctx.sa.get_job(job_id)
+            builder = self._run_single_job_with_ui_loop(
+                job, self._get_ui_for_job(job))
+            result = builder.get_result()
+            self.ctx.sa.use_job_result(job_id, result)
+            if (job.estimated_duration is not None
+                    and estimated_time is not None):
+                estimated_time -= job.estimated_duration
+
+    def _get_rerun_candidates(self):
+        """Get all the tests that might be selected for rerunning."""
+        def rerun_predicate(job_state):
+            return job_state.result.outcome in (
+                IJobResult.OUTCOME_FAIL, IJobResult.OUTCOME_CRASH,
+                IJobResult.OUTCOME_NOT_SUPPORTED, IJobResult.OUTCOME_SKIP)
+        rerun_candidates = []
+        todo_list = self.ctx.sa.get_static_todo_list()
+        job_states = {job_id: self.ctx.sa.get_job_state(job_id) for job_id
+                      in todo_list}
+        for job_id, job_state in job_states.items():
+            if rerun_predicate(job_state):
+                rerun_candidates.append(self.ctx.sa.get_job(job_id))
+        return rerun_candidates
+
+    def _maybe_rerun_jobs(self):
+        # create a list of jobs that qualify for rerunning
+        rerun_candidates = self._get_rerun_candidates()
+        # bail-out early if no job qualifies for rerunning
+        if not rerun_candidates:
+            return False
+        tree = SelectableJobTreeNode.create_simple_tree(self.ctx.sa,
+                                                        rerun_candidates)
+        # nothing to select in root node and categories - bailing out
+        if not tree.jobs and not tree._categories:
+            return False
+        # deselect all by default
+        tree.set_descendants_state(False)
+        self.ctx.display.run(ShowRerun(tree, _("Select jobs to re-run")))
+        wanted_set = frozenset(tree.selection)
+        if not wanted_set:
+            # nothing selected - nothing to run
+            return False
+        rerun_candidates = []
+        # include resource jobs that selected jobs depend on
+        resources_to_rerun = []
+        for job in wanted_set:
+            job_state = self.ctx.sa.get_job_state(job.id)
+            for inhibitor in job_state.readiness_inhibitor_list:
+                if inhibitor.cause == InhibitionCause.FAILED_DEP:
+                    resources_to_rerun.append(inhibitor.related_job)
+        # reset outcome of jobs that are selected for re-running
+        for job in list(wanted_set) + resources_to_rerun:
+            self.ctx.sa.get_job_state(job.id).result = MemoryJobResult({})
+            rerun_candidates.append(job.id)
+        self._run_jobs(rerun_candidates)
+        return True
+
+    def _do_normal_sequence(self):
+        self.ctx.sa.select_providers("*")
+        self.ctx.sa.configure_application_restart(
+            lambda session_id: [
+                'sh', '-c', ' '.join([
+                    os.path.abspath(__file__),
+                    "--resume", session_id])
+            ])
+        resumed = self._maybe_resume_session()
+        if not resumed:
+            print(_("Preparing..."))
+            self.ctx.sa.start_new_session(_("Checkbox CLI Session"))
+            testplan_id = None
+            if self.ctx.args.test_plan:
+                if self.ctx.args.test_plan in self.ctx.sa.get_test_plans():
+                    testplan_id = self.ctx.args.test_plan
+            elif self.is_interactive:
+                testplan_id = self._get_interactively_picked_testplans()
+            if not testplan_id:
+                self.ctx.rc.reset()
+                self.ctx.rc.bg = 'red'
+                self.ctx.rc.fg = 'bright_white'
+                self.ctx.rc.bold = 1
+                self.ctx.rc.para(_("Test plan not found!"))
+                raise SystemExit(1)
+            self.ctx.sa.select_test_plan(testplan_id)
+            self.ctx.sa.update_app_blob(json.dumps(
+                {'testplan_id': testplan_id, }).encode("UTF-8"))
+            self.ctx.sa.bootstrap()
+            if self.is_interactive:
+                self._interactively_pick_jobs_to_run()
+            self._run_jobs(self.ctx.sa.get_dynamic_todo_list())
+        if self.is_interactive:
+            while True:
+                if self._maybe_rerun_jobs():
+                    continue
+                else:
+                    break
+
+    def _handle_last_job_after_resume(self, metadata):
+        last_job = metadata.running_job_name
+        if last_job is None:
+            return
+        print(_("Previous session run tried to execute job: {}").format(
+            last_job))
+        cmd = self._pick_action_cmd([
+            Action('s', _("skip that job"), 'skip'),
+            Action('p', _("mark it as passed and continue"), 'pass'),
+            Action('f', _("mark it as failed and continue"), 'fail'),
+            Action('r', _("run it again"), 'run'),
+        ], _("What do you want to do with that job?"))
+        if cmd == 'skip' or cmd is None:
+            result = MemoryJobResult({
+                'outcome': IJobResult.OUTCOME_SKIP,
+                'comments': _("Skipped after resuming execution")
+            })
+        elif cmd == 'pass':
+            result = MemoryJobResult({
+                'outcome': IJobResult.OUTCOME_PASS,
+                'comments': _("Passed after resuming execution")
+            })
+        elif cmd == 'fail':
+            result = MemoryJobResult({
+                'outcome': IJobResult.OUTCOME_FAIL,
+                'comments': _("Failed after resuming execution")
+            })
+        elif cmd == 'run':
+            result = None
+        if result:
+            self.ctx.sa.use_job_result(last_job, result)
+
+    def _maybe_resume_session(self):
+        # Try to use the first session that can be resumed if the user agrees
+        resume_candidates = list(self.ctx.sa.get_resumable_sessions())
+        resumed = False
+        if resume_candidates:
+            if self.ctx.args.session_id:
+                for candidate in resume_candidates:
+                    if candidate.id == self.args.session_id:
+                        resume_candidates = (candidate, )
+                        break
+                else:
+                    raise RuntimeError("Requested session is not resumable!")
+            elif self.is_interactive:
+                print(self.C.header(_("Resume Incomplete Session")))
+                print(ngettext(
+                    "There is {0} incomplete session that might be resumed",
+                    "There are {0} incomplete sessions that might be resumed",
+                    len(resume_candidates)
+                ).format(len(resume_candidates)))
+            else:
+                return
+            for candidate in resume_candidates:
+                if self.ctx.args.session_id:
+                    cmd = 'resume'
+                else:
+                    # Skip sessions that the user doesn't want to resume
+                    cmd = self._pick_action_cmd([
+                        Action('r', _("resume this session"), 'resume'),
+                        Action('n', _("next session"), 'next'),
+                        Action('c', _("create new session"), 'create')
+                    ], _("Do you want to resume session {0!a}?").format(
+                        candidate.id))
+                if cmd == 'next':
+                    continue
+                elif cmd == 'create' or cmd is None:
+                    break
+                elif cmd == 'resume':
+                    metadata = self.ctx.sa.resume_session(candidate.id)
+                    app_blob = json.loads(metadata.app_blob.decode("UTF-8"))
+                    test_plan_id = app_blob['testplan_id']
+                    # FIXME selecting again the testplan on resume resets both
+                    # the static and dynamic todo lists.
+                    # We're then saving the selection from the saved run_list
+                    # by accessing the session private context object.
+                    selected_id_list = [job.id for job in
+                                        self.ctx.sa._context.state.run_list]
+                    self.ctx.sa.select_test_plan(test_plan_id)
+                    self.ctx.sa.bootstrap()
+                    self.ctx.sa.use_alternate_selection(selected_id_list)
+                    # If we resumed maybe not rerun the same, probably broken
+                    # job
+                    self._handle_last_job_after_resume(metadata)
+                    self._run_jobs(self.ctx.sa.get_dynamic_todo_list())
+                    resumed = True
+                # Finally ignore other sessions that can be resumed
+                break
+        return resumed
+
+
+if __name__ == '__main__':
+    checkbox_cli().main()

=== modified file 'plainbox/plainbox/impl/session/assistant.py'
--- plainbox/plainbox/impl/session/assistant.py	2015-11-04 14:25:10 +0000
+++ plainbox/plainbox/impl/session/assistant.py	2015-12-02 09:12:28 +0000
@@ -551,6 +551,8 @@
                     io_log_filename=self._runner.get_record_path_for_job(job),
                 ).get_result()
                 self._context.state.update_job_result(job, result)
+                self._metadata.running_job_name = None
+                self._manager.checkpoint()
         if self._restart_strategy is not None:
             self._restart_strategy.diffuse_application_restart(self._app_id)
         self.session_available(self._manager.storage.id)
@@ -558,7 +560,7 @@
         UsageExpectation.of(self).allowed_calls = {
             self.select_test_plan: "to save test plan selection",
         }
-        return self._resume_candidates[session_id].metadata
+        return self._metadata
 
     @raises(UnexpectedMethodCall)
     def get_resumable_sessions(self) -> 'Tuple[str, SessionMetaData]':
@@ -1187,6 +1189,7 @@
         UsageExpectation.of(self).enforce()
         job = self._context.get_unit(job_id, 'job')
         self._context.state.update_job_result(job, result)
+        self._manager.checkpoint()
         # Set up expectations so that run_job() and use_job_result() must be
         # called in pairs and applications cannot just forget and call
         # run_job() all the time.
@@ -1284,7 +1287,11 @@
         exported_stream = io.BytesIO()
         exporter.dump_from_session_manager(self._manager, exported_stream)
         exported_stream.seek(0)
-        return transport.send(exported_stream)
+        result = transport.send(exported_stream)
+        if SessionMetaData.FLAG_SUBMITTED not in self._metadata.flags:
+            self._metadata.flags.add(SessionMetaData.FLAG_SUBMITTED)
+            self._manager.checkpoint()
+        return result
 
     @raises(KeyError, OSError)
     def export_to_file(
@@ -1312,7 +1319,8 @@
             When there is a problem when writing the output.
         """
         UsageExpectation.of(self).enforce()
-        exporter = self._manager.create_exporter(exporter_id, option_list)
+        exporter = self._manager.create_exporter(exporter_id, option_list,
+                                                 strict=False)
         timestamp = datetime.datetime.utcnow().isoformat()
         path = os.path.join(dir_path, ''.join(
             ['submission_', timestamp, '.', exporter.unit.file_extension]))

=== modified file 'plainbox/plainbox/impl/session/restart.py'
--- plainbox/plainbox/impl/session/restart.py	2015-11-04 14:25:10 +0000
+++ plainbox/plainbox/impl/session/restart.py	2015-12-02 09:12:28 +0000
@@ -75,6 +75,7 @@
             The command callback
         """
         self.config = config = PlainBoxConfigParser()
+        self.app_terminal = app_terminal
         section = 'Desktop Entry'
         config.add_section(section)
         config.set(section, 'Type', 'Application')
@@ -99,6 +100,8 @@
 
     def prime_application_restart(self, app_id: str, cmd: str) -> None:
         filename = self.get_desktop_filename(app_id)
+        if self.app_terminal:
+            cmd += ';$SHELL'
         self.config.set('Desktop Entry', 'Exec', cmd)
         os.makedirs(os.path.dirname(filename), exist_ok=True)
         with open(filename, 'wt') as stream: