Merge lp:~zyga/checkbox/readonly-results into lp:checkbox

[Zygmunt Krynicki]

This is a work in progress.

I want it out so that I can see the the diff and history and iterate on it,.

[lp:~zyga/checkbox/readonly-results]

3777. By Zygmunt Krynicki on 2015-05-14

support: run dist-upgrade after installing deps

Signed-off-by: Zygmunt Krynicki <email address hidden>

3778. By Zygmunt Krynicki on 2015-05-14

plainbox:commands:check_config: produce more useful output

Signed-off-by: Zygmunt Krynicki <email address hidden>

3779. By Zygmunt Krynicki on 2015-05-14

plainbox:pod: fix several PEP257 issues

Signed-off-by: Zygmunt Krynicki <email address hidden>

3780. By Zygmunt Krynicki on 2015-05-14

plainbox:pod: add unset variants for type assignment filters

Signed-off-by: Zygmunt Krynicki <email address hidden>

3781. By Zygmunt Krynicki on 2015-05-14

plainbox:pod: filter-out UNSET from as_dict()

Signed-off-by: Zygmunt Krynicki <email address hidden>

3782. By Zygmunt Krynicki on 2015-05-14

plainbox:pod: fix PEP257 issues

Some docstrings were rewritten entirely.

Signed-off-by: Zygmunt Krynicki <email address hidden>

3783. By Zygmunt Krynicki on 2015-05-14

plainbox:pod: add Field.change_notifier

This patch adds a new decorator that can be used similarly to
@foo.setter (when foo is a @property-decorated method).

This can simplify all fields that need to have a custom notification
method. All such methods can be implemented _after_ all of the field
definitions now.

Signed-off-by: Zygmunt Krynicki <email address hidden>

3784. By Zygmunt Krynicki on 2015-05-14

plainbox:result: fix many PEP257 and some PEP8 issues

Signed-off-by: Zygmunt Krynicki <email address hidden>

3785. By Zygmunt Krynicki on 2015-05-14

plainbox:result: warn about modification of existing results

Signed-off-by: Zygmunt Krynicki <email address hidden>

3786. By Zygmunt Krynicki on 2015-05-14

plainbox:result: add JobResultBuilder

This patch adds a new class that assists in building appropriate
JobResult objects.

Signed-off-by: Zygmunt Krynicki <email address hidden>

3787. By Zygmunt Krynicki on 2015-05-14

plainbox:result: add _JobReturnBase.unseal()

Signed-off-by: Zygmunt Krynicki <email address hidden>

3788. By Zygmunt Krynicki on 2015-05-14

plainbox:runner: fix many PEP257 issues

Signed-off-by: Zygmunt Krynicki <email address hidden>

3789. By Zygmunt Krynicki on 2015-05-14

plainbox:runner: use JobResultBuilder everywhere

Signed-off-by: Zygmunt Krynicki <email address hidden>

3790. By Zygmunt Krynicki on 2015-05-14

plainbox:commands:run: use JobResultBuilder to handle mutable results

Signed-off-by: Zygmunt Krynicki <email address hidden>

3791. By Zygmunt Krynicki on 2015-05-14

checkbox-ng:service: use JobResultBuilder

Signed-off-by: Zygmunt Krynicki <email address hidden>

3792. By Zygmunt Krynicki on 2015-05-14

plainbox:resource: make it possible to iterate over resources

Iterating over resources behaves just like iterating over dictionaries
(keys are returned). This is useful to access resource elements in Jinja
templates. In the past we just accessed ._data directly but this is a
hack and we should get rid of it.

Signed-off-by: Zygmunt Krynicki <email address hidden>

3793. By Zygmunt Krynicki on 2015-05-14

plainbox:stubbox: add set of representative jobs

This patch adds a number of jobs that validate and don't do anything.
Each job has an identifier that looks like
"representative/plugin/{plugin}".

The goal of this collection is to simplify unit tests that traditionally
re-created everything using python APIs (which is both fragile and more
verbose).

Signed-off-by: Zygmunt Krynicki <email address hidden>

3794. By Zygmunt Krynicki on 2015-05-14

plainbox:providers:special: allow passing keyword args to get_stubbox()

The provider constructor seems to call the old validate API. Since this
is pretty annoying (all the incorrect things the old validation says) I
wanted to have a way to disable that by passing validate=False,
check=True. This is a quick stop-gap measure. In reality we want to
purge all the old validation code.

Signed-off-by: Zygmunt Krynicki <email address hidden>

3795. By Zygmunt Krynicki on 2015-05-14

plainbox:session:jobs: fix PEP257 issues

Signed-off-by: Zygmunt Krynicki <email address hidden>

3796. By Zygmunt Krynicki on 2015-05-14

plainbox:session:jobs: keep track of result history

Signed-off-by: Zygmunt Krynicki <email address hidden>

3797. By Zygmunt Krynicki on 2015-05-14

plainbox:session:suspend: fix PEP257 issue

Signed-off-by: Zygmunt Krynicki <email address hidden>

3798. By Zygmunt Krynicki on 2015-05-14

plainbox:session:suspend: store full result history

Signed-off-by: Zygmunt Krynicki <email address hidden>

3799. By Zygmunt Krynicki on 2015-05-14

plainbox:session:result: replay the full result history

Signed-off-by: Zygmunt Krynicki <email address hidden>

3800. By Zygmunt Krynicki on 2015-05-14

plainbox:commands:run: print session identifier on startup

Signed-off-by: Zygmunt Krynicki <email address hidden>

3801. By Zygmunt Krynicki on 2015-05-14

checkbox-ng:commands:newcli: allow re-run to be done indefinitely

Signed-off-by: Zygmunt Krynicki <email address hidden>

3802. By Zygmunt Krynicki on 2015-05-14

checkbox-ng:commands:newcli: don't reset last outcome when re-running jobs

Signed-off-by: Zygmunt Krynicki <email address hidden>

3803. By Zygmunt Krynicki on 2015-05-14

remove shit from smoke whitelist

3804. By Zygmunt Krynicki on 2015-05-14

plainbox:exporter:jinja: add support for customizing Environment

The environment is how you control all of jinja's behavior. Currently
the environment is hidden in the initializer of the base jinja exporter
class. By having a method where we can customize the environment
_before_ a template is loaded we can add anything (additional tests and
filters). After a template is initialized, those changes don't affect
the template.

Signed-off-by: Zygmunt Krynicki <email address hidden>

3805. By Zygmunt Krynicki on 2015-05-14

plainbox:exporter:hexr: add HEXR exporter

This patch adds a new exporter (HEXR) that is intended to replace our
current "xml" exporter. The schema is specific to the Canonical HEXR
application so the name reflects that.

Note that due to the syntax used, this exporter requires Jinja2 >= 2.7.
The dependency wasn't there before so I've added it to install_requires.

Signed-off-by: Zygmunt Krynicki <email address hidden>

3806. By Zygmunt Krynicki on 2015-05-14

plainbox:exporter:text: fix PEP257 issue

Signed-off-by: Zygmunt Krynicki <email address hidden>

3807. By Zygmunt Krynicki on 2015-05-14

plainbox:exporter:text: display result history

Signed-off-by: Zygmunt Krynicki <email address hidden>

3808. By Zygmunt Krynicki on 2015-05-14

plainbox:exporter:html: fix broken substitution on tr_outcome

Signed-off-by: Zygmunt Krynicki <email address hidden>

3809. By Zygmunt Krynicki on 2015-05-14

plainbox:exporter:html: remove useless OOM object

Signed-off-by: Zygmunt Krynicki <email address hidden>

3810. By Zygmunt Krynicki on 2015-05-14

plainbox:exporter:html: remove useless newlines

Signed-off-by: Zygmunt Krynicki <email address hidden>

3811. By Zygmunt Krynicki on 2015-05-14

plainbox:exporter:html: don't spell the long namespace over and over

Signed-off-by: Zygmunt Krynicki <email address hidden>

3812. By Zygmunt Krynicki on 2015-05-14

plainbox:exporter:html: use a variable for resource_map

Signed-off-by: Zygmunt Krynicki <email address hidden>

3813. By Zygmunt Krynicki on 2015-05-14

plainbox:exporter:html: use a variable for job_state_map

Signed-off-by: Zygmunt Krynicki <email address hidden>

3814. By Zygmunt Krynicki on 2015-05-14

plainbox:exporter:html: inline checkbox.css

Signed-off-by: Zygmunt Krynicki <email address hidden>

3815. By Zygmunt Krynicki on 2015-05-14

plainbox:exporter:html: inline checkbox.js

Signed-off-by: Zygmunt Krynicki <email address hidden>

3816. By Zygmunt Krynicki on 2015-05-14

plainbox:exporter:html: remove unused css rules

Signed-off-by: Zygmunt Krynicki <email address hidden>

3817. By Zygmunt Krynicki on 2015-05-14

plainbox:exporter:html: fix some inconsistent css syntax

Signed-off-by: Zygmunt Krynicki <email address hidden>

3818. By Zygmunt Krynicki on 2015-05-14

plainbox:exporter:html: don't use deprecated -vendor properties

Signed-off-by: Zygmunt Krynicki <email address hidden>

3819. By Zygmunt Krynicki on 2015-05-14

plainbox:exporter:html: remove commented-out rule

Signed-off-by: Zygmunt Krynicki <email address hidden>

3820. By Zygmunt Krynicki on 2015-05-14

plainbox:report:html: simplify html structure + result history [WIP]

Signed-off-by: Zygmunt Krynicki <email address hidden>

Unmerged revisions

3820. By Zygmunt Krynicki on 2015-05-14

plainbox:report:html: simplify html structure + result history [WIP]

Signed-off-by: Zygmunt Krynicki <email address hidden>

3819. By Zygmunt Krynicki on 2015-05-14

plainbox:exporter:html: remove commented-out rule

Signed-off-by: Zygmunt Krynicki <email address hidden>

3818. By Zygmunt Krynicki on 2015-05-14

plainbox:exporter:html: don't use deprecated -vendor properties

Signed-off-by: Zygmunt Krynicki <email address hidden>

3817. By Zygmunt Krynicki on 2015-05-14

plainbox:exporter:html: fix some inconsistent css syntax

Signed-off-by: Zygmunt Krynicki <email address hidden>

3816. By Zygmunt Krynicki on 2015-05-14

plainbox:exporter:html: remove unused css rules

Signed-off-by: Zygmunt Krynicki <email address hidden>

3815. By Zygmunt Krynicki on 2015-05-14

plainbox:exporter:html: inline checkbox.js

Signed-off-by: Zygmunt Krynicki <email address hidden>

3814. By Zygmunt Krynicki on 2015-05-14

plainbox:exporter:html: inline checkbox.css

Signed-off-by: Zygmunt Krynicki <email address hidden>

3813. By Zygmunt Krynicki on 2015-05-14

plainbox:exporter:html: use a variable for job_state_map

Signed-off-by: Zygmunt Krynicki <email address hidden>

3812. By Zygmunt Krynicki on 2015-05-14

plainbox:exporter:html: use a variable for resource_map

Signed-off-by: Zygmunt Krynicki <email address hidden>

3811. By Zygmunt Krynicki on 2015-05-14

plainbox:exporter:html: don't spell the long namespace over and over

Signed-off-by: Zygmunt Krynicki <email address hidden>

=== modified file 'checkbox-ng/checkbox_ng/commands/newcli.py'
--- checkbox-ng/checkbox_ng/commands/newcli.py	2015-04-17 10:43:20 +0000
+++ checkbox-ng/checkbox_ng/commands/newcli.py	2015-05-14 11:57:31 +0000
@@ -207,7 +207,11 @@
         self.print_estimated_duration()
         self.run_all_selected_jobs()
         if self.is_interactive:
-            self.maybe_rerun_jobs()
+            while True:
+                if self.maybe_rerun_jobs():
+                    continue
+                else:
+                    break
         self.export_and_send_results()
         if SessionMetaData.FLAG_INCOMPLETE in self.metadata.flags:
             print(self.C.header("Session Complete!", "GREEN"))
@@ -517,7 +521,7 @@
                 rerun_candidates.append(job)
         # bail-out early if no job qualifies for rerunning
         if not rerun_candidates:
-            return
+            return False
         tree = SelectableJobTreeNode.create_tree(
             self.manager.state, rerun_candidates)
         # deselect all by default
@@ -526,10 +530,12 @@
         wanted_set = frozenset(tree.selection)
         if not wanted_set:
             # nothing selected - nothing to run
-            return
+            return False
         rerun_job_list = [job for job in self.manager.state.run_list
                           if job in wanted_set]
         # reset outcome of jobs that are selected for re-running
         for job in wanted_set:
-            self.manager.state.job_state_map[job.id].result.outcome = None
+            from plainbox.impl.result import MemoryJobResult
+            self.manager.state.job_state_map[job.id].result = MemoryJobResult({})
         self.run_all_selected_jobs()
+        return True

=== modified file 'checkbox-ng/checkbox_ng/service.py'
--- checkbox-ng/checkbox_ng/service.py	2015-05-06 10:18:36 +0000
+++ checkbox-ng/checkbox_ng/service.py	2015-05-14 11:57:31 +0000
@@ -37,7 +37,7 @@
         raise SystemExit(_("DBus parts require 'funcsigs' from pypi."))
 from plainbox.abc import IJobResult
 from plainbox.impl.job import JobDefinition
-from plainbox.impl.result import MemoryJobResult
+from plainbox.impl.result import JobResultBuilder
 from plainbox.impl.secure.qualifiers import select_jobs
 from plainbox.impl.session import JobState
 from plainbox.vendor import extcmd
@@ -1372,6 +1372,7 @@
         # A result object we got from running the command OR the result this
         # job used to have before. It should be always published on the bus.
         self._result = None
+        self._result_builder = None
         # A future for the result each time we're waiting for the command to
         # finish. Gets reset to None after the command is done executing.
         self._result_future = None
@@ -1447,16 +1448,20 @@
                     _("But the job is not manual, it is %s"),
                     self.native.job.plugin)
             # Create a new result object
-            self._result = MemoryJobResult({
-                'outcome': outcome,
-                'comments': comments
-            })
+            self._result_builder = JobResultBuilder()
+            self._result_builder.outcome = outcome
+            self._result_builder.comments = comments
+            self._result = self._result_builder.seal()
             # Add the new result object to the bus
             self._session_wrapper.add_result(self._result)
         else:
+            assert self._result_builder is not None
             # Set the values as requested
-            self._result.outcome = outcome
-            self._result.comments = comments
+            self._result_builder.outcome = outcome
+            self._result_builder.comments = comments
+            self._result = self._result_builder.seal()
+            # Add the new result object to the bus
+            self._session_wrapper.add_result(self._result)
         # Notify the application that the result is ready. This has to be
         # done unconditionally each time this method called.
         self.JobResultAvailable(
@@ -1548,6 +1553,7 @@
             self._session_wrapper.remove_result(self._result)
         # Unpack the result from the future
         self._result = result_future.result()
+        self._result_builder = self._result.unseal()
         # Add the new result object to the session wrapper (and to the bus)
         self._session_wrapper.add_result(self._result)
         # Reset the future so that RunCommand() can run the job again

=== removed file 'plainbox/plainbox/data/report/checkbox.css'
--- plainbox/plainbox/data/report/checkbox.css	2015-04-09 16:57:49 +0000
+++ plainbox/plainbox/data/report/checkbox.css	1970-01-01 00:00:00 +0000
@@ -1,261 +0,0 @@
-body {
-    font-family: "Ubuntu Beta", "Bitstream Vera Sans", DejaVu Sans, Tahoma, sans-serif;
-    color: #333;
-    background: white;
-    font-size: 12px;
-    line-height: 14px;
-    margin: 0px;
-    padding: 0px;
-}
-#container {
-    background: #f7f6f5;
-    margin: 0px auto 20px;
-    padding: 0px;
-    width: 976px;
-}
-#container-inner {
-    background-color: #dfdcd9;
-}
-#header, #container-inner {
-    -moz-border-radius: 0px 0px 5px 5px;
-    -webkit-border-bottom-left-radius: 5px;
-    -webkit-border-bottom-right-radius: 5px;
-    -moz-box-shadow: #bbb 0px 0px 5px;
-    -webkit-box-shadow: #bbb 0px 0px 5px;
-}
-#header {
-    background: #dd4814 top left repeat-x;
-    height: 64px;
-    margin: 0px;
-    padding: 0px;
-    position: relative;
-}
-
-#menu-search {
-    height: 40px;
-    margin: 0 16px;
-}
-
-#title {
-    padding: 28px 24px;
-}
-
-#content {
-    /*padding: 32px 80px 32px 80px;*/
-    padding: 32px 240px 32px 60px;
-    margin: 0 16px 16px;
-    width: 644px;
-    background-color: #fff;
-    -moz-border-radius: 4px;
-    -webkit-border-radius: 4px;
-}
-#end-content {
-    clear: both;
-}
-
-#content-panel {
-    width: 446px;
-    margin: 0px 0px 0px 0px;
-    padding: 8px 8px 32px 8px;
-    background-color: #fff;
-    -moz-border-radius: 4px;
-    -webkit-border-radius: 4px;
-}
-
-#copyright {
-    background-position: 803px 40px;
-    background-repeat: no-repeat;
-    text-align: center;
-    margin: 0 16px;
-    padding: 40px 0 0 0;
-    height: 32px;
-}
-#copyright p {
-    color: #aea79f;
-    font-size: 10px;
-    line-height: 14px;
-    margin: 2px 0;
-}
-
-#footer {
-    padding-top: 16px;
-}
-#footer * {
-    font-size: 10px;
-    line-height: 14px;
-}
-#footer p {
-    margin: 0;
-    padding-bottom: 3px;
-    border-bottom: 1px dotted #aea79f;
-}
-#footer p.footer-title {
-    font-weight: bold;
-}
-#footer .footer-div {
-    width: 144px;
-    float: left;
-    margin-left: 16px;
-}
-#footer .last-div {
-    margin-right: 16px;
-}
-#footer ul {
-    list-style: none;
-    margin: 0;
-    padding: 0;
-}
-#footer li {
-    margin: 0;
-    padding: 3px 0;
-    border-bottom: 1px dotted #aea79f;
-}
-
-h1, h2, h3, h4, h5 {
-    padding: 0;
-    margin: 0;
-    font-weight: normal;
-}
-h1 {
-    font-size: 36px;
-    line-height: 40px;
-    color: #dd4814;
-}
-h2 {
-    font-size: 24px;
-    line-height: 28px;
-    margin-bottom: 8px;
-}
-h3 {
-    font-size: 16px;
-    line-height: 20px;
-    margin-bottom: 8px;
-}
-h3.link-other {
-    color: #333;
-}
-h3.link-services {
-    color: #fff;
-}
-h4 {
-    font-size: 12px;
-    line-height: 14px;
-}
-h4.partners {
-    color: #333;
-    font-size: 16px;
-    line-height: 20px;
-}
-h5 {
-    color: #333;
-    font-size: 10px;
-    line-height: 14px;
-}
-h1 span.grey, h2 span.grey, h1 span, h2 span{
-    color: #aea79f;
-}
-p {
-    font-size: 12px;
-    line-height: 14px;
-    margin-bottom: 8px;
-}
-strong {
-    font-weight: bold;
-}
-
-a {
-    color: #333;
-    text-decoration: none;
-}
-a:hover {
-    color: #dd4814;
-    text-decoration: underline;
-}
-div.footer-div:hover a, div#content:hover a {
-    color: #dd4814;
-    text-decoration: none;
-}
-div.footer-div:hover a:hover, div#content:hover a:hover {
-    color: #dd4814;
-    text-decoration: underline;
-}
-
-ul {
-    margin-bottom: 16px;
-}
-ul li {
-    margin-bottom: 8px;
-    line-height: 14px;
-}
-ul li:last-child {
-    margin-bottom: 0px;
-}
-
-p.call-to-action {
-    color: #333;
-}
-p.case-study {
-    color: #333;
-}
-p.highlight {
-    font-size: 16px;
-    line-height: 20px;
-}
-p.introduction {
-    color: #333;
-    font-size: 16px;
-    line-height: 20px;
-}
-p.services {
-    color: #fff;
-}
-p.small-text {
-    color: #333;
-    font-size: 10px;
-}
-
-/* Clearing floats without extra markup
-Based on How To Clear Floats Without Structural Markup by PiE
-[http://www.positioniseverything.net/easyclearing.html] */
-.clearfix:after {
-    content: ".";
-    display: block;
-    height: 0;
-    clear: both;
-    visibility: hidden;
-}
-.clearfix {
-    -moz-border-radius: 5px 5px 5px 5px;
-    -webkit-border-bottom-top-radius: 5px;
-    -webkit-border-bottom-left-radius: 5px;
-    -webkit-border-bottom-bottom-radius: 5px;
-    -webkit-border-bottom-right-radius: 5px;
-    -moz-box-shadow: #bbb 0px 0px 5px;
-    -webkit-box-shadow: #bbb 0px 0px 5px;
-    display: inline-block;
-}  /* for IE/Mac */
-th
-{
-    text-align: left;
-}
-td
-{
-    margin: 0;
-    padding-bottom: 3px;
-    border-bottom: 1px dotted #aea79f;
-    font-size: 10px;
-    line-height: 14px;
-}
-.resultimg
-{
-    height: 12px;
-}
-.disclosureimg
-{
-    height:     .75em;
-    vertical-align: middle;
-}
-.data
-{
-    display:    none;
-}

=== modified file 'plainbox/plainbox/data/report/checkbox.html'
--- plainbox/plainbox/data/report/checkbox.html	2015-05-04 08:08:53 +0000
+++ plainbox/plainbox/data/report/checkbox.html	2015-05-14 11:57:31 +0000
@@ -1,153 +1,274 @@
+{%- set ns = '2013.com.canonical.certification::' -%}
+{%- set state = manager.default_device_context.state -%}
+{%- set resource_map = state.resource_map -%}
+{%- set job_state_map = state.job_state_map -%}
 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
 <html>
 <head>
     <title>System Testing Report</title>
     <style type="text/css">
-        {% include "checkbox.css" %}
+      html {
+        font-family: "Ubuntu Beta", "Bitstream Vera Sans", DejaVu Sans, Tahoma, sans-serif;
+        color: #333;
+        background: white;
+        font-size: 12px;
+        line-height: 14px;
+        margin: 0px;
+        padding: 0px;
+      }
+      body {
+        background: #f7f6f5;
+        margin: 0px auto 20px;
+        padding: 0px;
+        width: 976px;
+        background-color: #dfdcd9;
+        border-radius: 0px 0px 5px 5px;
+        box-shadow: #bbb 0px 0px 5px;
+      }
+      #title {
+        padding: 28px 24px;
+      }
+      #content {
+        padding: 32px 240px 32px 60px;
+        margin: 0 16px 16px;
+        width: 644px;
+        background-color: #fff;
+        border-radius: 4px;
+      }
+      h1, h2, h3, h4, h5 {
+        padding: 0;
+        margin: 0;
+        font-weight: normal;
+      }
+      h1 {
+        font-size: 36px;
+        line-height: 40px;
+        color: #dd4814;
+      }
+      h2 {
+        font-size: 24px;
+        line-height: 28px;
+        margin-bottom: 8px;
+      }
+      h3 {
+        font-size: 16px;
+        line-height: 20px;
+        margin-bottom: 8px;
+      }
+      h4 {
+        font-size: 12px;
+        line-height: 14px;
+      }
+      h5 {
+        color: #333;
+        font-size: 10px;
+        line-height: 14px;
+      }
+      h1 span.grey, h2 span.grey, h1 span, h2 span{
+        color: #aea79f;
+      }
+      p {
+        font-size: 12px;
+        line-height: 14px;
+        margin-bottom: 8px;
+      }
+      pre {
+          color: gray;
+      }
+      a {
+        color: #333;
+        text-decoration: none;
+      }
+      a:hover {
+        color: #dd4814;
+        text-decoration: underline;
+      }
+      div#content:hover a {
+        color: #dd4814;
+        text-decoration: none;
+      }
+      div#content:hover a:hover {
+        color: #dd4814;
+        text-decoration: underline;
+      }
+      ul {
+        margin-bottom: 16px;
+      }
+      ul li {
+        margin-bottom: 8px;
+        line-height: 14px;
+      }
+      ul li:last-child {
+        margin-bottom: 0px;
+      }
+      /* Clearing floats without extra markup
+      Based on How To Clear Floats Without Structural Markup by PiE
+      [http://www.positioniseverything.net/easyclearing.html] */
+      .clearfix:after {
+        content: ".";
+        display: block;
+        height: 0;
+        clear: both;
+        visibility: hidden;
+      }
+      .clearfix {
+        border-radius: 5px 5px 5px 5px;
+        box-shadow: #bbb 0px 0px 5px;
+        display: inline-block;
+      }  /* for IE/Mac */
+      th {
+        text-align: left;
+      }
+      td {
+        margin: 0;
+        padding-bottom: 3px;
+        border-bottom: 1px dotted #aea79f;
+        font-size: 10px;
+        line-height: 14px;
+        vertical-align: top;
+      }
+      .data {
+        display:    none;
+      }
     </style>
     <script type="text/javascript">
-        {% include "checkbox.js" %}
+      function showHide(what) {
+        var heading = document.getElementById(what);
+        var contents = document.getElementById(what + "-contents");
+        var headingcontents = heading.innerHTML;
+        var newcontents;
+
+        if (contents.style.display != "block") {
+          newcontents = headingcontents.replace(/[^\x00-\x80]/g, "&#9660;");
+          contents.style.display = "block";
+        } else {
+          newcontents = headingcontents.replace(/[^\x00-\x80]/g, "&#9654;");
+          contents.style.display = "none";
+        }
+
+        heading.innerHTML = newcontents;
+      }
     </script>
 </head>
 <body>
-    <div id="container">
-        <div id="container-inner">
-
-            <div id="title">
-                <h1>System Testing<span class="grey"> Report</span></h1>
-            </div>
-            <div id="content" class="clearfix">
-                This report was created using {{ client_name }} {{ client_version }} on {{ timestamp }}
-            </div>
-            {% if "2013.com.canonical.certification::lsb" in manager.state.resource_map %}
-            <div id="content" class="clearfix">
-                <h2>Software Information</h2>
-                <p>{{ manager.state.resource_map["2013.com.canonical.certification::lsb"][0]["description"] }}</p>
-            </div>
-            {% endif %}
-
-            {% if manager.state.get_certification_status_map() %}
-            <div id="content" class="clearfix">
-            <h2 id="questions">Certification Status - Blockers</h2>
-                <table style="width: 700px">
-                <thead>
-                    <tr>
-                        <th>Test ID</th>
-                        <th>Result</th>
-                    </tr>
-                </thead>
-                <tbody>
-                {% for job_id, job_state in manager.state.get_certification_status_map()|dictsort %}
-                    <tr>
-                        <td>{{ job_state.job.tr_summary() }}</td>
-                        <td style='font-weight: bold; color: {{ OMM[job_state.result.outcome].color_hex }}'>{{ OMM[job_state.result.outcome].tr_outcome|replace("job ", "") }}</td>
-                    </tr>
-                {% endfor %}
-                </tbody>
-            </table>
-            </div>
-            {% endif %}
-
-            {% if manager.state.get_certification_status_map(certification_status_filter=('non-blocker',)) %}
-            <div id="content" class="clearfix">
-            <h2 id="questions">Certification Status - Non Blockers</h2>
-                <table style="width: 700px">
-                <thead>
-                    <tr>
-                        <th>Test ID</th>
-                        <th>Result</th>
-                    </tr>
-                </thead>
-                <tbody>
-                {% for job_id, job_state in manager.state.get_certification_status_map(certification_status_filter=('non-blocker',))|dictsort %}
-                    <tr>
-                        <td>{{ job_state.job.tr_summary() }}</td>
-                        <td style='font-weight: bold; color: {{ OMM[job_state.result.outcome].color_hex }}'>{{ OMM[job_state.result.outcome].tr_outcome|replace("job ", "") }}</td>
-                    </tr>
-                {% endfor %}
-                </tbody>
-            </table>
-            </div>
-            {% endif %}
-
-            <div id="content" class="clearfix">
-            <h2 id="questions">Tests Performed</h2>
-                <table style="width: 824px">
-                <thead>
-                    <tr>
-                        <th>Test ID</th>
-                        <th>Result</th>
-                        <th>Certification status</th>
-                        <th>Comment</th>
-                    </tr>
-                </thead>
-                <tbody>
-                {% for job_id, job_state in manager.state.job_state_map|dictsort if job_state.result.outcome != None and job_state.job.plugin not in ("resource", "local", "attachment") %}
-                    <tr>
-                        <td>{{ job_state.job.tr_summary() }}</td>
-                        <td style='font-weight: bold; color: {{ OMM[job_state.result.outcome].color_hex }}'>{{ OMM[job_state.result.outcome].tr_outcome|replace("job ", "") }}</td>
-                        <td>{{ job_state.effective_certification_status }}</td>
-                        {% if job_state.result.comments != None %}
-                        <td>{{ job_state.result.comments }}</td>
-                        {% else %}
-                        {% if job_state.result.io_log_as_flat_text != "" %}
-                        <td><div style="vertical-align: middle; width: 420px; overflow: auto;">
-                                <pre>{{ job_state.result.io_log_as_flat_text }}</pre>
-                            </div>
-                        </td>
-                        {% else %}
-                        <td>&nbsp;</td>
-                        {% endif %}
-                        {% endif %}
-                    </tr>
-                {% endfor %}
-                </tbody>
-            </table>
-            </div>
-            {% if "2013.com.canonical.certification::requirements" in manager.state.resource_map %}
-            <div id="content" class="clearfix">
-                <h2>Requirements documents</h2>
-                <ul>
-                {% for doc in manager.state.resource_map["2013.com.canonical.certification::requirements"] %}
-                    <li><a href="{{ doc["link"] }}">{{ doc["name"] }}</a></li>
-                {% endfor %}
-                </ul>
-            </div>
-            {% endif %}
-            <div id="content" class="clearfix">
-                <h2>Log Files and Environment Information</h2>
-                <div id="packages-contents">
-                    {% if "2013.com.canonical.certification::package" in manager.state.resource_map %}
-                    <span onClick="showHide('package');"><h3><span id="package" style="color: #dd4814;">&#9654;</span>
-                        packages installed</h3></span>
-                    <div class="data" id="package-contents" style="overflow: auto;">
-                        <table>
-                            <thead>
-                                <tr>
-                                    <th>Name</th>
-                                    <th>Version</th>
-                                </tr>
-                            </thead>
-                            <tbody>
-                            {% for package in manager.state.resource_map["2013.com.canonical.certification::package"] %}
-                                <tr>
-                                    <td>{{ package["name"] }}</td>
-                                    <td>{{ package["version"] }}</td>
-                                </tr>
-                            {% endfor %}
-                            </tbody>
-                        </table>
-                    </div>
-                    {% endif %}
-                    {% for job_id, job_state in manager.state.job_state_map|dictsort if job_state.result.outcome != None and job_state.job.plugin == "attachment" %}
-                    <span onClick="showHide('att{{ loop.index }}');"><h3><span id="att{{ loop.index }}" style="color: #dd4814;">&#9654;</span>
-                        {{ job_state.job.partial_id|replace("_attachment", "") }}</h3></span>
-                    <div class="data" id="att{{ loop.index }}-contents" style="overflow: auto;">
-                        <pre>{{ job_state.result.io_log_as_text_attachment }}</pre>
-                    </div>
+    <div id="title">
+        <h1>System Testing<span class="grey"> Report</span></h1>
+    </div>
+    <div id="content" class="clearfix">
+        This report was created using {{ client_name }} {{ client_version }} on {{ timestamp }}
+    </div>
+    {% if ns ~ "lsb" in resource_map %}
+    <div id="content" class="clearfix">
+        <h2>Software Information</h2>
+        <p>{{ resource_map[ns ~ "lsb"][0]["description"] }}</p>
+    </div>
+    {% endif %}
+    {% if manager.state.failed_blockers_map %}
+    <div id="content" class="clearfix">
+    <h2 id="questions">Certification Status - Blockers</h2>
+        <table style="width: 700px">
+        <thead>
+            <tr>
+                <th>Test ID</th>
+                <th>Result</th>
+            </tr>
+        </thead>
+        <tbody>
+        {% for job_id, job_state in manager.state.get_certification_status_map()|dictsort %}
+            <tr>
+                <td>{{ job_state.job.tr_summary() }}</td>
+                <td style='font-weight: bold; color: {{ job_state.result.outcome_meta().color_hex }}'>{{ job_state.result.outcome_meta().tr_label }}</td>
+            </tr>
+        {% endfor %}
+        </tbody>
+    </table>
+    </div>
+    {% endif %}
+    {% if manager.state.failed_non_blockers_map %}
+    <div id="content" class="clearfix">
+    <h2 id="questions">Certification Status - Non Blockers</h2>
+        <table style="width: 700px">
+        <thead>
+            <tr>
+                <th>Test ID</th>
+                <th>Result</th>
+            </tr>
+        </thead>
+        <tbody>
+        {% for job_id, job_state in manager.state.get_certification_status_map(certification_status_filter=('non-blocker',))|dictsort %}
+            <tr>
+                <td>{{ job_state.job.tr_summary() }}</td>
+                <td style='font-weight: bold; color: {{ job_state.result.outcome_meta().color_hex }}'>{{ job_state.result.outcome_meta().tr_label }}</td>
+            </tr>
+        {% endfor %}
+        </tbody>
+    </table>
+    </div>
+    {% endif %}
+    <div id="content" class="clearfix">
+    <h2 id="questions">Tests Performed</h2>
+        <table style="width: 824px">
+        <thead>
+            <tr>
+                <th>Test ID</th>
+                <th>Result</th>
+                <th>Certification status</th>
+                <th>Comments</th>
+            </tr>
+        </thead>
+        <tbody>
+        {%- for job_id, job_state in job_state_map|dictsort if job_state.result.outcome != None and job_state.job.plugin not in ("resource", "local", "attachment") %}
+        {%- for result in job_state.result_history %}
+            <tr>
+                <td>{{ job_state.job.tr_summary() }}{% if loop.index > 1 %} (run {{ loop.index }}){% endif %}</td>
+                <td style='font-weight: bold; color: {{ result.outcome_meta().color_hex }}'>{{ result.outcome_meta().tr_label }}</td>
+                {%- if loop.index0 == 0 %}
+                <td rowspan="{{ 2 * job_state.result_history|length }}">{{ job_state.effective_certification_status }}</td>
+                {%- endif %}
+                <td>{{ result.comments|default("", true) }}</td>
+            </tr>
+            <tr>
+                <td colspan="4">
+                    <pre style="overflow: auto; width: 80em;">{{ result.io_log_as_flat_text }}</pre>
+                </td>
+            </tr>
+        {%- endfor %}
+        {%- endfor %}
+        </tbody>
+    </table>
+    </div>
+    <div id="content" class="clearfix">
+        <h2>Log Files and Environment Information</h2>
+        <div id="packages-contents">
+            {% if ns ~ "package" in resource_map %}
+            <span onClick="showHide('package');"><h3><span id="package" style="color: #dd4814;">&#9654;</span>
+                packages installed</h3></span>
+            <div class="data" id="package-contents" style="overflow: auto;">
+                <table>
+                    <thead>
+                        <tr>
+                            <th>Name</th>
+                            <th>Version</th>
+                        </tr>
+                    </thead>
+                    <tbody>
+                    {% for package in resource_map[ns ~ "package"] %}
+                        <tr>
+                            <td>{{ package["name"] }}</td>
+                            <td>{{ package["version"] }}</td>
+                        </tr>
                     {% endfor %}
-                </div>
-            </div>
+                    </tbody>
+                </table>
+            </div>
+            {% endif %}
+            {% for job_id, job_state in job_state_map|dictsort if job_state.result.outcome != None and job_state.job.plugin == "attachment" %}
+            <span onClick="showHide('att{{ loop.index }}');"><h3><span id="att{{ loop.index }}" style="color: #dd4814;">&#9654;</span>
+                {{ job_state.job.partial_id|replace("_attachment", "") }}</h3></span>
+            <div class="data" id="att{{ loop.index }}-contents" style="overflow: auto;">
+                <pre>{{ job_state.result.io_log_as_text_attachment }}</pre>
+            </div>
+            {% endfor %}
         </div>
     </div>
 </body>

=== removed file 'plainbox/plainbox/data/report/checkbox.js'
--- plainbox/plainbox/data/report/checkbox.js	2015-04-09 16:57:49 +0000
+++ plainbox/plainbox/data/report/checkbox.js	1970-01-01 00:00:00 +0000
@@ -1,16 +0,0 @@
-function showHide(what) {
-    var heading = document.getElementById(what);
-    var contents = document.getElementById(what + "-contents");
-    var headingcontents = heading.innerHTML;
-    var newcontents;
-
-    if (contents.style.display != "block") {
-        newcontents = headingcontents.replace(/[^\x00-\x80]/g, "&#9660;");
-        contents.style.display = "block";
-    } else {
-        newcontents = headingcontents.replace(/[^\x00-\x80]/g, "&#9654;");
-        contents.style.display = "none";
-    }
-
-    heading.innerHTML = newcontents;
-}

=== added file 'plainbox/plainbox/data/report/hexr.xml'
--- plainbox/plainbox/data/report/hexr.xml	1970-01-01 00:00:00 +0000
+++ plainbox/plainbox/data/report/hexr.xml	2015-05-14 11:57:31 +0000
@@ -0,0 +1,149 @@
+{%- set ns = '2013.com.canonical.certification::' -%}
+{%- set state = manager.default_device_context.state -%}
+{%- set job_state_map = state.job_state_map -%}
+{%- set hexr_outcome_list = OUTCOME_METADATA_MAP.values()|selectattr('hexr_xml_allowed')|sort(attribute='hexr_xml_order') -%}
+<?xml version="1.0"?>
+<system version="1.0">
+  <context>
+    {%- for job_id in job_state_map|sort %}
+    {%- set job_state = job_state_map[job_id] %}
+    {%- if job_state.job.id|strip_ns not in ("dmi_attachment", "sysfs_attachment", "udev_attachment") and job_state.result.outcome %}
+    <info command="{{ job_state.job.id|strip_ns }}">{{ job_state.result.io_log_as_text_attachment }}</info>
+    {%- endif %}
+    {%- endfor %}
+  </context>
+  <hardware>
+    {%- if ns ~ 'dmi_attachment' in state.job_state_map %}
+    {%- set dmi_attachment = state.job_state_map[ns ~ 'dmi_attachment'].result.io_log_as_text_attachment %}
+    <dmi>{{ dmi_attachment }}</dmi>
+    {%- else %}
+    <!-- the dmi_attachment job is not available, not producing the <dmi> section -->
+    {%- endif %}
+    {%- if ns ~ 'sysfs_attachment' in state.job_state_map %}
+    {%- set sysfs_attachment = state.job_state_map[ns ~ 'sysfs_attachment'].result.io_log_as_text_attachment %}
+    <sysfs-attributes>{{ sysfs_attachment }}</sysfs-attributes>
+    {%- else %}
+    <!-- the sysfs_attachment job is not available, not producing the <sysfs-attributes> tag -->
+    {%- endif %}
+    {%- if ns ~ 'udev_attachment' in state.job_state_map %}
+    {%- set udev_attachment = state.job_state_map[ns ~ 'udev_attachment'].result.io_log_as_text_attachment %}
+    <udev>{{ udev_attachment }}</udev>
+    {%- else %}
+    <!-- the udev_attachment job is not available, not producing the <udev> tag -->
+    {%- endif %}
+    {%- if ns ~ 'cpuinfo' in state.resource_map %}
+    {%- set processor_resource = state.resource_map[ns ~ 'cpuinfo'][0] %}
+    {#- FIXME: The <processors> section is quite broken by design. #}
+    {#- Yes, it does copy the data for the 0th CPU $count times. #}
+    <processors>
+      {%- for dummy_index in range(processor_resource.count|int) %}
+      <processor id="{{ loop.index0 }}" name="{{ loop.index0 }}">
+        {%- for key in processor_resource|sort %}
+        <property name="{{ key }}" type="str">{{ processor_resource[key] }}</property>
+        {%- endfor %}
+      </processor>
+      {%- endfor %}
+    </processors>
+    {%- else %}
+    <!-- cpuinfo resource is not available, not producing the <processors> section -->
+    {%- endif %}
+  </hardware>
+  <questions>
+    {%- for job_id in job_state_map|sort %}
+    {%- set job_state = job_state_map[job_id] %}
+    {%- set job = job_state.job %}
+    {%- set result = job_state.result %}
+    {%- if job.plugin not in ("resource", "local", "attachment") and job_state.result.outcome %}
+    <question{{ {
+        'name': job.id|strip_ns,
+        'certification_status': job_state.certification_status,
+      }|sorted_xmlattr }}>
+      <answer type="multiple_choice">{{ result.outcome_meta().hexr_xml_mapping }}</answer>
+      {#- FIXME: Yes the <answer_choices> section is useless #}
+      <answer_choices>
+      {%- for outcome_info in hexr_outcome_list %}
+        <value type="str">{{ outcome_info.hexr_xml_mapping }}</value>
+      {%- endfor %}
+      </answer_choices>
+      {#- FIXME: yes this does munges comments and I/O log #}
+      <comment>
+        {%- if result.comments -%}
+        {{ result.comments }}
+        {%- else -%}
+        {{ result.io_log_as_flat_text }}
+        {%- endif -%}
+      </comment>
+    </question>
+    {%- endif %}
+    {%- endfor %}
+  </questions>
+  <software>
+    {%- if ns ~ 'lsb' in state.resource_map %}
+    {%- set lsb_resource = state.resource_map[ns ~ 'lsb'][0] %}
+    <lsbrelease>
+      {%- for key in lsb_resource|sort %}
+      <property name="{{ key }}" type="str">{{ lsb_resource[key] }}</property>
+      {%- endfor %}
+    </lsbrelease>
+    {%- else %}
+    <!-- lsb resource is not available, not producing the <lsbrelease> tag -->
+    {%- endif %}
+    {%- if ns ~ 'package' in state.resource_map %}
+    {%- set package_resource_list = state.resource_map[ns ~ 'package'] %}
+    <packages>
+      {%- for package_resource in package_resource_list %}
+      <package id="{{ loop.index0 }}" name="{{ package_resource.name }}">
+        {%- for key in package_resource|reject('is_name')|sort %}
+        <property name="{{ key }}" type="str">{{ package_resource[key] }}</property>
+        {%- endfor %}
+      </package>
+      {%- endfor %}
+    </packages>
+    {%- else %}
+    <!-- package resource is not available, not producing the <packages> tag -->
+    {%- endif %}
+    {%- if ns ~ 'requirements' in state.resource_map %}
+    {%- set requirement_resource_list = state.resource_map[ns ~ 'requirements'] %}
+    <requirements>
+      {%- for requirement_resource in requirement_resource_list %}
+      <requirement id=" {{ loop.index0 }}" name="{{ requirement_resource.name }}">
+        {%- for key in requirement_resource|reject('is_name')|sort %}
+        <property name="{{ key }}" type="str">{{ requirement_resource[key] }}</property>
+        {%- endfor %}
+      </requirement>
+      {%- endfor %}
+    </requirements>
+    {%- else %}
+    <!-- requirements resource is not available, not producing the <requirements> tag -->
+    {%- endif %}
+  </software>
+  <summary>
+    {#- FIXME: with all the <property> tags those should not be custom #}
+    <client name="{{ client_name }}" version="{{ client_version }}"/>
+    <date_created value="{{ timestamp }}"/>
+    {%- if ns ~ 'dpkg' in state.resource_map %}
+    {%- set dpkg_resource = state.resource_map[ns ~ 'dpkg'][0] %}
+    <architecture value="{{ dpkg_resource.architecture }}"/>
+    {%- else %}
+    <!-- dpkg resource is not available, not producing the <architecture> tag -->
+    {%- endif %}
+    {%- if ns ~ 'lsb' in state.resource_map %}
+    {#- NOTE: lsb_resource is computed earlier, if the condition above holds #}
+    <distribution value="{{ lsb_resource.distributor_id }}"/>
+    <distroseries value="{{ lsb_resource.release }}"/>
+    {%- else %}
+    <!-- lsb resource is not available, not producing <distribution> and <distroseries> tags -->
+    {%- endif %}
+    {%- if ns ~ 'uname' in state.resource_map %}
+    {%- set uname_resource = state.resource_map[ns ~ 'uname'][0] %}
+    <kernel-release value="{{ uname_resource.release }}"/>
+    {%- else %}
+    <!-- uname resource is not available, not producing the <kernel-release> tag -->
+    {%- endif %}
+    {#- FIXME: yes, this is just hard-coded non-sense #}
+    <private value="False"/>
+    <contactable value="False"/>
+    <live_cd value="False"/>
+    <system_id value="{{ system_id }}"/>
+  </summary>
+</system>

=== modified file 'plainbox/plainbox/impl/commands/inv_check_config.py'
--- plainbox/plainbox/impl/commands/inv_check_config.py	2014-11-18 17:20:58 +0000
+++ plainbox/plainbox/impl/commands/inv_check_config.py	2015-05-14 11:57:31 +0000
@@ -43,14 +43,17 @@
                 print(_(" - {0} (not present)").format(filename))
         print(_("Variables:"))
         for variable in self.config.Meta.variable_list:
+            print("   [{0}]".format(variable.section))
             print("   {0}={1}".format(
                 variable.name,
                 variable.__get__(self.config, self.config.__class__)))
         print(_("Sections:"))
         for section in self.config.Meta.section_list:
-            print("   {0}={1}".format(
-                section.name,
-                section.__get__(self.config, self.config.__class__)))
+            print("   [{0}]".format(section.name))
+            section_value = section.__get__(self.config, self.config.__class__)
+            if section_value:
+                for key, value in sorted(section_value.items()):
+                    print("   {0}={1}".format(key, value))
         if self.config.problem_list:
             print(_("Problems:"))
             for problem in self.config.problem_list:

=== modified file 'plainbox/plainbox/impl/commands/inv_run.py'
--- plainbox/plainbox/impl/commands/inv_run.py	2015-05-12 16:09:40 +0000
+++ plainbox/plainbox/impl/commands/inv_run.py	2015-05-14 11:57:31 +0000
@@ -44,6 +44,7 @@
 from plainbox.impl.depmgr import DependencyDuplicateError
 from plainbox.impl.exporter import ByteStringStreamTranslator
 from plainbox.impl.exporter import get_all_exporters
+from plainbox.impl.result import JobResultBuilder
 from plainbox.impl.result import MemoryJobResult
 from plainbox.impl.result import tr_outcome
 from plainbox.impl.runner import JobRunner
@@ -626,6 +627,11 @@
             # Connect the on_job_added signal. We use it to mark the test loop
             # for re-execution and to update the list of desired jobs.
             self.state.on_job_added.connect(self.on_job_added)
+        finally:
+            print(_("Session ID: {0}").format(
+                self.C.YELLOW(self._manager.storage.id)
+            ))
+
 
     def create_runner(self):
         """
@@ -741,10 +747,7 @@
                     estimated_time -= job.estimated_duration
 
     def run_single_job(self, job):
-        job_start_time = time.time()
         self.run_single_job_with_ui(job, self.get_ui_for_job(job))
-        job_state = self.state.job_state_map[job.id]
-        job_state.result.execution_duration = time.time() - job_start_time
 
     def get_ui_for_job(self, job):
         if self.ns.dont_suppress_output is False and job.plugin in (
@@ -754,6 +757,7 @@
             return NormalUI(self.C.c, show_cmd_output=True)
 
     def run_single_job_with_ui(self, job, ui):
+        job_start_time = time.time()
         job_state = self.state.job_state_map[job.id]
         ui.considering_job(job, job_state)
         if job_state.can_start():
@@ -761,18 +765,21 @@
             self.metadata.running_job_name = job.id
             self.manager.checkpoint()
             ui.started_running(job, job_state)
-            job_result = self._run_single_job_with_ui_loop(job, job_state, ui)
+            result_builder = self._run_single_job_with_ui_loop(job, job_state, ui)
+            assert result_builder is not None
+            result_builder.execution_duration = time.time() - job_start_time
+            job_result = result_builder.seal()
             self.metadata.running_job_name = None
             self.manager.checkpoint()
             ui.finished_running(job, job_state, job_result)
         else:
-            job_result = MemoryJobResult({
-                'outcome': IJobResult.OUTCOME_NOT_SUPPORTED,
-                'comments': job_state.get_readiness_description()
-            })
+            result_builder = JobResultBuilder(
+                outcome=IJobResult.OUTCOME_NOT_SUPPORTED,
+                comments=job_state.get_readiness_description(),
+                execution_duration = time.time() - job_start_time)
+            job_result = result_builder.seal()
             ui.job_cannot_start(job, job_state, job_result)
-        if job_result is not None:
-            self.state.update_job_result(job, job_result)
+        self.state.update_job_result(job, job_result)
         ui.finished(job, job_state, job_result)
 
     def _run_single_job_with_ui_loop(self, job, job_state, ui):
@@ -788,8 +795,9 @@
                     cmd = ui.wait_for_interaction_prompt(job)
                     if cmd == 'run' or cmd is None:
                         ui.notify_about_steps(job)
-                        job_result = self.runner.run_job(
-                            job, job_state, self.config, ui)
+                        result_builder = self.runner.run_job(
+                            job, job_state, self.config, ui
+                        ).unseal()
                     elif cmd == 'comment':
                         new_comment = input(self.C.BLUE(
                             _('Please enter your comments:') + '\n'))
@@ -797,36 +805,37 @@
                             comments += new_comment + '\n'
                         continue
                     elif cmd == 'skip':
-                        job_result = MemoryJobResult({
-                            'outcome': IJobResult.OUTCOME_SKIP,
-                            'comments': _("Explicitly skipped before"
-                                          " execution")
-                        })
+                        result_builder = JobResultBuilder(
+                            outcome=IJobResult.OUTCOME_SKIP,
+                            comments=_("Explicitly skipped before"
+                                       " execution"))
                         if comments != "":
-                            job_result.comments = comments
+                            result_builder.comments = comments
                         break
                     elif cmd == 'quit':
                         raise SystemExit()
                 else:
-                    job_result = self.runner.run_job(
-                        job, job_state, self.config, ui)
+                    result_builder = self.runner.run_job(
+                        job, job_state, self.config, ui
+                    ).unseal()
             else:
                 if 'noreturn' in job.get_flag_set():
                     ui.noreturn_job()
-                job_result = self.runner.run_job(
-                    job, job_state, self.config, ui)
+                result_builder = self.runner.run_job(
+                    job, job_state, self.config, ui
+                ).unseal()
             if (self.is_interactive and
-                    job_result.outcome == IJobResult.OUTCOME_UNDECIDED):
+                    result_builder.outcome == IJobResult.OUTCOME_UNDECIDED):
                 try:
                     if comments != "":
-                        job_result.comments = comments
+                        result_builder.comments = comments
                     ui.notify_about_verification(job)
-                    job_result = self._interaction_callback(
-                        self.runner, job, job_result, self.config)
+                    self._interaction_callback(
+                        self.runner, job, result_builder, self.config)
                 except ReRunJob:
                     continue
             break
-        return job_result
+        return result_builder
 
     def export_and_send_results(self):
         # Get a stream with exported session data.
@@ -861,7 +870,7 @@
     def _pick_action_cmd(self, action_list, prompt=None):
         return ActionUI(action_list, prompt, self._color).run()
 
-    def _interaction_callback(self, runner, job, result, config,
+    def _interaction_callback(self, runner, job, result_builder, config,
                               prompt=None, allowed_outcome=None):
         if prompt is None:
             prompt = _("Select an outcome or an action: ")
@@ -890,42 +899,40 @@
         if job.command is not None:
             allowed_actions.append(
                 Action('r', _('re-run this job'), 're-run'))
-        if result.return_code is not None:
-            if result.return_code == 0:
+        if result_builder.return_code is not None:
+            if result_builder.return_code == 0:
                 suggested_outcome = IJobResult.OUTCOME_PASS
             else:
                 suggested_outcome = IJobResult.OUTCOME_FAIL
             allowed_actions.append(
                 Action('', _('set suggested outcome [{0}]').format(
                     tr_outcome(suggested_outcome)), 'set-suggested'))
-        while result.outcome not in allowed_outcome:
+        while result_builder.outcome not in allowed_outcome:
             print(_("Please decide what to do next:"))
-            print("  " + _("outcome") + ": {0}".format(self.C.result(result)))
-            if result.comments is None:
+            print("  " + _("outcome") + ": {0}".format(
+                self.C.result(result_builder.seal())))
+            if result_builder.comments is None:
                 print("  " + _("comments") + ": {0}".format(
                     C_("none comment", "none")))
             else:
                 print("  " + _("comments") + ": {0}".format(
-                    self.C.CYAN(result.comments, bright=False)))
+                    self.C.CYAN(result_builder.comments, bright=False)))
             cmd = self._pick_action_cmd(allowed_actions)
             if cmd == 'set-pass':
-                result.outcome = IJobResult.OUTCOME_PASS
+                result_builder.outcome = IJobResult.OUTCOME_PASS
             elif cmd == 'set-fail':
-                result.outcome = IJobResult.OUTCOME_FAIL
+                result_builder.outcome = IJobResult.OUTCOME_FAIL
             elif cmd == 'set-skip' or cmd is None:
-                result.outcome = IJobResult.OUTCOME_SKIP
+                result_builder.outcome = IJobResult.OUTCOME_SKIP
             elif cmd == 'set-suggested':
-                result.outcome = suggested_outcome
+                result_builder.outcome = suggested_outcome
             elif cmd == 'set-comments':
-                if result.comments is None:
-                    result.comments = ""
                 new_comment = input(self.C.BLUE(
                     _('Please enter your comments:') + '\n'))
                 if new_comment:
-                    result.comments += new_comment + '\n'
+                    result_builder.add_comment(new_comment)
             elif cmd == 're-run':
                 raise ReRunJob
-        return result
 
     def _update_desired_job_list(self, desired_job_list):
         problem_list = self.state.update_desired_job_list(desired_job_list)

=== added file 'plainbox/plainbox/impl/exporter/hexr.py'
--- plainbox/plainbox/impl/exporter/hexr.py	1970-01-01 00:00:00 +0000
+++ plainbox/plainbox/impl/exporter/hexr.py	2015-05-14 11:57:31 +0000
@@ -0,0 +1,138 @@
+# This file is part of Checkbox.
+#
+# Copyright 2015 Canonical Ltd.
+# Written by:
+#   Zygmunt Krynicki <[e-mail]>
+#
+# Checkbox is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 3,
+# as published by the Free Software Foundation.
+#
+# Checkbox is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with Checkbox.  If not, see <http://www.gnu.org/licenses/>.
+
+"""Exporter for creating the XML structure expected by HEXR."""
+
+from datetime import datetime
+
+from jinja2 import Markup
+from jinja2 import Undefined
+from jinja2 import environmentfilter
+from jinja2 import escape
+
+from plainbox import __version__ as version
+from plainbox.impl.exporter.jinja2 import Jinja2SessionStateExporter
+from plainbox.impl.result import OUTCOME_METADATA_MAP
+
+
+#: Name-space prefix for Canonical Certification
+CERTIFICATION_NS = '2013.com.canonical.certification::'
+
+
+@environmentfilter
+def do_sorted_xmlattr(_environment, d, autospace=True):
+    """A version of xmlattr filter that sorts attributes."""
+    rv = ' '.join(
+        '%s="%s"' % (escape(key), escape(value))
+        for key, value in sorted(d.items())
+        if value is not None and not isinstance(value, Undefined)
+    )
+    if autospace and rv:
+        rv = ' ' + rv
+    if _environment.autoescape:
+        rv = Markup(rv)
+    return rv
+
+
+@environmentfilter
+def do_strip_ns(_environment, unit_id, ns=CERTIFICATION_NS):
+    """Remove the namespace part of the identifier."""
+    if unit_id.startswith(ns):
+        rv = unit_id[len(ns):]
+    else:
+        rv = unit_id
+    rv = escape(rv)
+    if _environment.autoescape:
+        rv = Markup(rv)
+    return rv
+
+
+def do_is_name(text):
+    """A filter for checking if something is equal to "name"."""
+    return text == 'name'
+
+
+class HEXRExporter(Jinja2SessionStateExporter):
+
+    """
+    Exporter for creating HEXR XML documents.
+
+    This exporter takes the whole session and creates a XML document
+    containing a subset of the data. It is applicable for submission to HEXR
+    and ``C3`` (the certification website). It's also really bad.
+
+    The format is hardwired to require certain jobs. They are:
+
+    - 2013.com.canonical.certification::package          (Optional)
+    - 2013.com.canonical.certification::uname            (Optional)
+    - 2013.com.canonical.certification::lsb              (Mandatory)
+    - 2013.com.canonical.certification::cpuinfo          (Mandatory)
+    - 2013.com.canonical.certification::dpkg             (Mandatory)
+    - 2013.com.canonical.certification::dmi_attachment   (Mandatory)
+    - 2013.com.canonical.certification::sysfs_attachment (Mandatory)
+    - 2013.com.canonical.certification::udev_attachment  (Mandatory)
+
+    .. note::
+        The exporter won't misbehave if those are not available but the server
+        side component will most likely crash or reject the resulting document.
+    """
+
+    def __init__(self, option_list=None, system_id="", timestamp=None,
+                 client_version=None, client_name='plainbox'):
+        """Initialize the exporter with stuff that exporters need."""
+        super().__init__("hexr.xml", option_list)
+        self._system_id = system_id
+        # Generate a time-stamp if needed
+        if timestamp is None:
+            timestamp = datetime.utcnow().strftime("%Y-%m-%dT%H:%M:%S")
+        self._timestamp = timestamp
+        # Use current version unless told otherwise
+        if client_version is None:
+            client_version = "{}.{}.{}".format(*version[:3])
+        self._client_version = client_version
+        # Remember client name
+        self._client_name = client_name
+
+    def customize_environment(self, env):
+        """Register filters and tests custom to the HEXR exporter."""
+        env.autoescape = True
+        env.filters['sorted_xmlattr'] = do_sorted_xmlattr
+        env.filters['strip_ns'] = do_strip_ns
+        env.tests['is_name'] = do_is_name
+
+    def dump_from_session_manager(self, manager, stream):
+        """
+        Extract data from session_manager and dump it into the stream.
+
+        :param session_manager:
+            SessionManager instance that manages session to be exported by
+            this exporter
+        :param stream:
+            Byte stream to write to.
+
+        """
+        data = {
+            'OUTCOME_METADATA_MAP': OUTCOME_METADATA_MAP,
+            'client_name': self._client_name,
+            'client_version': self._client_version,
+            'manager': manager,
+            'options': self.option_list,
+            'system_id': self._system_id,
+            'timestamp': self._timestamp,
+        }
+        self.dump(data, stream)

=== modified file 'plainbox/plainbox/impl/exporter/html.py'
--- plainbox/plainbox/impl/exporter/html.py	2015-05-10 18:38:53 +0000
+++ plainbox/plainbox/impl/exporter/html.py	2015-05-14 11:57:31 +0000
@@ -33,7 +33,6 @@
 
 from plainbox import __version__ as version
 from plainbox.impl.exporter.jinja2 import Jinja2SessionStateExporter
-from plainbox.impl.result import OUTCOME_METADATA_MAP as OMM
 
 
 class HTMLSessionStateExporter(Jinja2SessionStateExporter):
@@ -75,7 +74,6 @@
         data = {
             'manager': session_manager,
             'options': self.option_list,
-            'OMM': OMM,
             'timestamp': self._timestamp,
             'client_version': self._client_version,
             'client_name': self._client_name

=== modified file 'plainbox/plainbox/impl/exporter/jinja2.py'
--- plainbox/plainbox/impl/exporter/jinja2.py	2015-04-09 16:57:49 +0000
+++ plainbox/plainbox/impl/exporter/jinja2.py	2015-05-14 11:57:31 +0000
@@ -55,6 +55,7 @@
         paths.extend(extra_paths)
         loader = FileSystemLoader(paths)
         env = Environment(loader=loader)
+        self.customize_environment(env)
 
         def include_file(name):
             # This helper function insert static files literally into Jinja
@@ -64,6 +65,14 @@
         self.template = env.get_template(jinja2_template)
         env.globals['include_file'] = include_file
 
+    def customize_environment(self, env):
+        """
+        Customize the jinja2 Environment object.
+
+        By default this method does nothing. Override it in your subclass to
+        define any extra filters, tests or other things that you may require.
+        """
+
     def dump(self, data, stream):
         """
         Render report using jinja2 and dump it to stream.

=== added file 'plainbox/plainbox/impl/exporter/test_hexr.py'
--- plainbox/plainbox/impl/exporter/test_hexr.py	1970-01-01 00:00:00 +0000
+++ plainbox/plainbox/impl/exporter/test_hexr.py	2015-05-14 11:57:31 +0000
@@ -0,0 +1,535 @@
+# This file is part of Checkbox.
+#
+# Copyright 2015 Canonical Ltd.
+# Written by:
+#   Zygmunt Krynicki <[e-mail]>
+#
+# Checkbox is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 3,
+# as published by the Free Software Foundation.
+#
+# Checkbox is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with Checkbox.  If not, see <http://www.gnu.org/licenses/>.
+
+"""Tests for the hexr exporter."""
+
+from io import BytesIO
+from unittest import TestCase
+from xml.etree import ElementTree
+
+from plainbox.impl.exporter.hexr import CERTIFICATION_NS
+from plainbox.impl.exporter.hexr import HEXRExporter
+from plainbox.impl.providers.special import get_stubbox
+from plainbox.impl.resource import Resource
+from plainbox.impl.result import MemoryJobResult
+from plainbox.impl.session import SessionManager
+from plainbox.impl.unit.job import JobDefinition
+
+
+class HexrExporterTests(TestCase):
+
+    """Tests for HEXRExporter."""
+
+    maxDiff = None
+
+    def setUp(self):
+        """Common initialization."""
+        self.exporter = HEXRExporter(
+            system_id='SYSTEM_ID', timestamp='TIMESTAMP',
+            client_version='CLIENT_VERSION', client_name='CLIENT_NAME')
+        self.manager = SessionManager.create()
+        self.manager.add_local_device_context()
+
+    def _populate_session(self):
+        self._make_representative_jobs()
+        self._make_cert_resources()
+        self._make_cert_attachments()
+
+    def _make_representative_jobs(self):
+        # Add all of the jobs from representative.pxu so that we don't have to
+        # create verbose fakes. Each job gets a simple passing result.
+        state = self.manager.default_device_context.state
+        stubbox = get_stubbox(validate=False, check=True)
+        for job in stubbox.job_list:
+            if not job.partial_id.startswith('representative/plugin/'):
+                continue
+            state.add_unit(job)
+            result = self._make_result_for(job)
+            state.update_job_result(job, result)
+        # Add a comment to one job (the last one)
+        result.comments = 'COMMENTS'
+
+    def _make_result_for(self, job):
+        data = {
+            'outcome': 'pass',
+            'comments': None
+        }
+        if job.plugin == 'local':
+            pass
+        elif job.plugin == 'resource':
+            pass
+        else:
+            data['io_log'] = (
+                (0, 'stdout', b'IO-LOG-STDOUT\n'),
+                (1, 'stderr', b'IO-LOG-STDERR\n')
+            )
+        return MemoryJobResult(data)
+
+    def _make_cert_resources(self):
+        # Create some specific resources that this exporter relies on. The
+        # corresponding jobs are _not_ loaded but this is irrelevant.
+        state = self.manager.default_device_context.state
+        ns = CERTIFICATION_NS
+        state.set_resource_list(ns + 'cpuinfo', [Resource({
+            'PROP-1': 'VALUE-1',
+            'PROP-2': 'VALUE-2',
+            'count': '2',  # NOTE: this has to be a number :/
+        })])
+        state.set_resource_list(ns + 'dpkg', [Resource({
+            'architecture': 'dpkg.ARCHITECTURE',
+        })])
+        state.set_resource_list(ns + 'lsb', [Resource({
+            'codename': 'lsb.CODENAME',
+            'description': 'lsb.DESCRIPTION',
+            'release': 'lsb.RELEASE',
+            'distributor_id': 'lsb.DISTRIBUTOR_ID',
+        })])
+        state.set_resource_list(ns + 'uname', [Resource({
+            'release': 'uname.RELEASE',
+        })])
+        state.set_resource_list(ns + 'package', [Resource({
+            'name': 'package.0.NAME',
+            'version': 'package.0.VERSION',
+        }), Resource({
+            'name': 'package.1.NAME',
+            'version': 'package.1.VERSION',
+        })])
+        state.set_resource_list(ns + 'requirements', [Resource({
+            'name': 'requirement.0.NAME',
+            'link': 'requirement.0.LINK',
+        }), Resource({
+            'name': 'requirement.1.NAME',
+            'link': 'requirement.1.LINK',
+        })])
+
+    def _make_cert_attachments(self):
+        state = self.manager.default_device_context.state
+        partial_id_list = ['dmi_attachment', 'sysfs_attachment',
+                           'udev_attachment']
+        for partial_id in partial_id_list:
+            job = JobDefinition({
+                'id': CERTIFICATION_NS + partial_id,
+                'plugin': 'attachment'
+            })
+            result = MemoryJobResult({
+                'io_log': (
+                    (0, 'stdout', 'STDOUT-{}\n'.format(
+                        partial_id).encode('utf-8')),
+                    (1, 'stderr', 'STDERR-{}\n'.format(
+                        partial_id).encode('utf-8'))),
+            })
+            state.add_unit(job)
+            state.update_job_result(job, result)
+
+    def _inject_evil_input(self):
+        evil = '"\'<&>'
+        self.exporter._system_id = evil
+        self.exporter._timestamp = evil
+        self.exporter._client_name = evil
+        self.exporter._client_version = evil
+        state = self.manager.default_device_context.state
+        for resource_id in state.resource_map:
+            resource_list = state.resource_map[resource_id]
+            for resource in resource_list:
+                for key in resource:
+                    if resource_id.endswith('cpuinfo') and key == 'count':
+                        # don't change resources for the <hardware> section
+                        continue
+                    resource[key] = evil
+        new_job_state_map = {}
+        for index, job_id in enumerate(sorted(state.job_state_map)):
+            job_state = state.job_state_map[job_id]
+            if (job_state.job.partial_id.endswith('_attachment')
+                    or job_state.job.partial_id == 'cpuinfo'):
+                # don't change attachments for the <hardware> section
+                evil_id = job_id
+            else:
+                evil_id = '{}-{}-{}'.format(evil, index, job_state.job.plugin)
+            # NOTE: using private API
+            job_state.job._data['id'] = evil_id
+            job_state.result.comments = evil
+            # NOTE: using private API
+            job_state.result._data['_io_log'] = (
+                (0, 'stdout', evil.encode("UTF-8")),
+            )
+            new_job_state_map[evil_id] = job_state
+        # NOTE: using private API
+        state._job_state_map = new_job_state_map
+
+    def tearDown(self):
+        """Common teardown."""
+        self.manager.destroy()
+
+    def test_smoke(self):
+        """The XML document has the right data in the right spot."""
+        self._populate_session()
+        stream = BytesIO()
+        self.exporter.dump_from_session_manager(self.manager, stream)
+        smoke_actual = stream.getvalue().decode("utf-8")
+        self.assertMultiLineEqual(_smoke_expected, smoke_actual)
+
+    def test_without_any_data(self):
+        """The XML document can be produced without any data in the session."""
+        stream = BytesIO()
+        self.exporter.dump_from_session_manager(self.manager, stream)
+        empty_actual = stream.getvalue().decode("utf-8")
+        self.assertMultiLineEqual(_empty_expected, empty_actual)
+
+    def test_escaping(self):
+        """Evil input doesn't break the correctness of the XML document."""
+        self._populate_session()
+        self._inject_evil_input()
+        stream = BytesIO()
+        self.exporter.dump_from_session_manager(self.manager, stream)
+        evil_actual = stream.getvalue().decode("utf-8")
+        self.assertMultiLineEqual(_evil_expected, evil_actual)
+
+    def test_xml_parsability(self):
+        """Each produced output can be parsed with an XML parser."""
+        stream1 = BytesIO(_smoke_expected.encode("utf-8"))
+        ElementTree.parse(stream1)
+        stream2 = BytesIO(_empty_expected.encode("utf-8"))
+        ElementTree.parse(stream2)
+        stream3 = BytesIO(_evil_expected.encode("utf-8"))
+        ElementTree.parse(stream3)
+
+
+_smoke_expected = """\
+<?xml version="1.0"?>
+<system version="1.0">
+  <context>
+    <info command="2013.com.canonical.plainbox::representative/plugin/attachment">IO-LOG-STDOUT
+</info>
+    <info command="2013.com.canonical.plainbox::representative/plugin/local"></info>
+    <info command="2013.com.canonical.plainbox::representative/plugin/manual">IO-LOG-STDOUT
+</info>
+    <info command="2013.com.canonical.plainbox::representative/plugin/qml">IO-LOG-STDOUT
+</info>
+    <info command="2013.com.canonical.plainbox::representative/plugin/resource"></info>
+    <info command="2013.com.canonical.plainbox::representative/plugin/shell">IO-LOG-STDOUT
+</info>
+    <info command="2013.com.canonical.plainbox::representative/plugin/user-interact">IO-LOG-STDOUT
+</info>
+    <info command="2013.com.canonical.plainbox::representative/plugin/user-interact-verify">IO-LOG-STDOUT
+</info>
+    <info command="2013.com.canonical.plainbox::representative/plugin/user-verify">IO-LOG-STDOUT
+</info>
+  </context>
+  <hardware>
+    <dmi>STDOUT-dmi_attachment
+</dmi>
+    <sysfs-attributes>STDOUT-sysfs_attachment
+</sysfs-attributes>
+    <udev>STDOUT-udev_attachment
+</udev>
+    <processors>
+      <processor id="0" name="0">
+        <property name="count" type="str">2</property>
+        <property name="PROP-1" type="str">VALUE-1</property>
+        <property name="PROP-2" type="str">VALUE-2</property>
+      </processor>
+      <processor id="1" name="1">
+        <property name="count" type="str">2</property>
+        <property name="PROP-1" type="str">VALUE-1</property>
+        <property name="PROP-2" type="str">VALUE-2</property>
+      </processor>
+    </processors>
+  </hardware>
+  <questions>
+    <question name="2013.com.canonical.plainbox::representative/plugin/manual">
+      <answer type="multiple_choice">pass</answer>
+      <answer_choices>
+        <value type="str">none</value>
+        <value type="str">pass</value>
+        <value type="str">fail</value>
+        <value type="str">skip</value>
+      </answer_choices>
+      <comment>IO-LOG-STDOUT
+IO-LOG-STDERR
+</comment>
+    </question>
+    <question name="2013.com.canonical.plainbox::representative/plugin/qml">
+      <answer type="multiple_choice">pass</answer>
+      <answer_choices>
+        <value type="str">none</value>
+        <value type="str">pass</value>
+        <value type="str">fail</value>
+        <value type="str">skip</value>
+      </answer_choices>
+      <comment>IO-LOG-STDOUT
+IO-LOG-STDERR
+</comment>
+    </question>
+    <question name="2013.com.canonical.plainbox::representative/plugin/shell">
+      <answer type="multiple_choice">pass</answer>
+      <answer_choices>
+        <value type="str">none</value>
+        <value type="str">pass</value>
+        <value type="str">fail</value>
+        <value type="str">skip</value>
+      </answer_choices>
+      <comment>IO-LOG-STDOUT
+IO-LOG-STDERR
+</comment>
+    </question>
+    <question name="2013.com.canonical.plainbox::representative/plugin/user-interact">
+      <answer type="multiple_choice">pass</answer>
+      <answer_choices>
+        <value type="str">none</value>
+        <value type="str">pass</value>
+        <value type="str">fail</value>
+        <value type="str">skip</value>
+      </answer_choices>
+      <comment>IO-LOG-STDOUT
+IO-LOG-STDERR
+</comment>
+    </question>
+    <question name="2013.com.canonical.plainbox::representative/plugin/user-interact-verify">
+      <answer type="multiple_choice">pass</answer>
+      <answer_choices>
+        <value type="str">none</value>
+        <value type="str">pass</value>
+        <value type="str">fail</value>
+        <value type="str">skip</value>
+      </answer_choices>
+      <comment>IO-LOG-STDOUT
+IO-LOG-STDERR
+</comment>
+    </question>
+    <question name="2013.com.canonical.plainbox::representative/plugin/user-verify">
+      <answer type="multiple_choice">pass</answer>
+      <answer_choices>
+        <value type="str">none</value>
+        <value type="str">pass</value>
+        <value type="str">fail</value>
+        <value type="str">skip</value>
+      </answer_choices>
+      <comment>COMMENTS</comment>
+    </question>
+  </questions>
+  <software>
+    <lsbrelease>
+      <property name="codename" type="str">lsb.CODENAME</property>
+      <property name="description" type="str">lsb.DESCRIPTION</property>
+      <property name="distributor_id" type="str">lsb.DISTRIBUTOR_ID</property>
+      <property name="release" type="str">lsb.RELEASE</property>
+    </lsbrelease>
+    <packages>
+      <package id="0" name="package.0.NAME">
+        <property name="version" type="str">package.0.VERSION</property>
+      </package>
+      <package id="1" name="package.1.NAME">
+        <property name="version" type="str">package.1.VERSION</property>
+      </package>
+    </packages>
+    <requirements>
+      <requirement id=" 0" name="requirement.0.NAME">
+        <property name="link" type="str">requirement.0.LINK</property>
+      </requirement>
+      <requirement id=" 1" name="requirement.1.NAME">
+        <property name="link" type="str">requirement.1.LINK</property>
+      </requirement>
+    </requirements>
+  </software>
+  <summary>
+    <client name="CLIENT_NAME" version="CLIENT_VERSION"/>
+    <date_created value="TIMESTAMP"/>
+    <architecture value="dpkg.ARCHITECTURE"/>
+    <distribution value="lsb.DISTRIBUTOR_ID"/>
+    <distroseries value="lsb.RELEASE"/>
+    <kernel-release value="uname.RELEASE"/>
+    <private value="False"/>
+    <contactable value="False"/>
+    <live_cd value="False"/>
+    <system_id value="SYSTEM_ID"/>
+  </summary>
+</system>"""
+
+
+_empty_expected = """\
+<?xml version="1.0"?>
+<system version="1.0">
+  <context>
+  </context>
+  <hardware>
+    <!-- the dmi_attachment job is not available, not producing the <dmi> section -->
+    <!-- the sysfs_attachment job is not available, not producing the <sysfs-attributes> tag -->
+    <!-- the udev_attachment job is not available, not producing the <udev> tag -->
+    <!-- cpuinfo resource is not available, not producing the <processors> section -->
+  </hardware>
+  <questions>
+  </questions>
+  <software>
+    <!-- lsb resource is not available, not producing the <lsbrelease> tag -->
+    <!-- package resource is not available, not producing the <packages> tag -->
+    <!-- requirements resource is not available, not producing the <requirements> tag -->
+  </software>
+  <summary>
+    <client name="CLIENT_NAME" version="CLIENT_VERSION"/>
+    <date_created value="TIMESTAMP"/>
+    <!-- dpkg resource is not available, not producing the <architecture> tag -->
+    <!-- lsb resource is not available, not producing <distribution> and <distroseries> tags -->
+    <!-- uname resource is not available, not producing the <kernel-release> tag -->
+    <private value="False"/>
+    <contactable value="False"/>
+    <live_cd value="False"/>
+    <system_id value="SYSTEM_ID"/>
+  </summary>
+</system>"""
+
+_escaped_evil_text = '&#34;&#39;&lt;&amp;&gt;'
+_evil_expected = """\
+<?xml version="1.0"?>
+<system version="1.0">
+  <context>
+    <info command="2013.com.canonical.plainbox::&#34;&#39;&lt;&amp;&gt;-10-user-interact-verify">IO-LOG-STDOUT
+</info>
+    <info command="2013.com.canonical.plainbox::&#34;&#39;&lt;&amp;&gt;-11-user-verify">IO-LOG-STDOUT
+</info>
+    <info command="2013.com.canonical.plainbox::&#34;&#39;&lt;&amp;&gt;-3-attachment">IO-LOG-STDOUT
+</info>
+    <info command="2013.com.canonical.plainbox::&#34;&#39;&lt;&amp;&gt;-4-local"></info>
+    <info command="2013.com.canonical.plainbox::&#34;&#39;&lt;&amp;&gt;-5-manual">IO-LOG-STDOUT
+</info>
+    <info command="2013.com.canonical.plainbox::&#34;&#39;&lt;&amp;&gt;-6-qml">IO-LOG-STDOUT
+</info>
+    <info command="2013.com.canonical.plainbox::&#34;&#39;&lt;&amp;&gt;-7-resource"></info>
+    <info command="2013.com.canonical.plainbox::&#34;&#39;&lt;&amp;&gt;-8-shell">IO-LOG-STDOUT
+</info>
+    <info command="2013.com.canonical.plainbox::&#34;&#39;&lt;&amp;&gt;-9-user-interact">IO-LOG-STDOUT
+</info>
+  </context>
+  <hardware>
+    <dmi>STDOUT-dmi_attachment
+</dmi>
+    <sysfs-attributes>STDOUT-sysfs_attachment
+</sysfs-attributes>
+    <udev>STDOUT-udev_attachment
+</udev>
+    <processors>
+      <processor id="0" name="0">
+        <property name="count" type="str">2</property>
+        <property name="PROP-1" type="str">{evil}</property>
+        <property name="PROP-2" type="str">{evil}</property>
+      </processor>
+      <processor id="1" name="1">
+        <property name="count" type="str">2</property>
+        <property name="PROP-1" type="str">{evil}</property>
+        <property name="PROP-2" type="str">{evil}</property>
+      </processor>
+    </processors>
+  </hardware>
+  <questions>
+    <question name="2013.com.canonical.plainbox::&#34;&#39;&lt;&amp;&gt;-10-user-interact-verify">
+      <answer type="multiple_choice">pass</answer>
+      <answer_choices>
+        <value type="str">none</value>
+        <value type="str">pass</value>
+        <value type="str">fail</value>
+        <value type="str">skip</value>
+      </answer_choices>
+      <comment>&#34;&#39;&lt;&amp;&gt;</comment>
+    </question>
+    <question name="2013.com.canonical.plainbox::&#34;&#39;&lt;&amp;&gt;-11-user-verify">
+      <answer type="multiple_choice">pass</answer>
+      <answer_choices>
+        <value type="str">none</value>
+        <value type="str">pass</value>
+        <value type="str">fail</value>
+        <value type="str">skip</value>
+      </answer_choices>
+      <comment>&#34;&#39;&lt;&amp;&gt;</comment>
+    </question>
+    <question name="2013.com.canonical.plainbox::&#34;&#39;&lt;&amp;&gt;-5-manual">
+      <answer type="multiple_choice">pass</answer>
+      <answer_choices>
+        <value type="str">none</value>
+        <value type="str">pass</value>
+        <value type="str">fail</value>
+        <value type="str">skip</value>
+      </answer_choices>
+      <comment>&#34;&#39;&lt;&amp;&gt;</comment>
+    </question>
+    <question name="2013.com.canonical.plainbox::&#34;&#39;&lt;&amp;&gt;-6-qml">
+      <answer type="multiple_choice">pass</answer>
+      <answer_choices>
+        <value type="str">none</value>
+        <value type="str">pass</value>
+        <value type="str">fail</value>
+        <value type="str">skip</value>
+      </answer_choices>
+      <comment>&#34;&#39;&lt;&amp;&gt;</comment>
+    </question>
+    <question name="2013.com.canonical.plainbox::&#34;&#39;&lt;&amp;&gt;-8-shell">
+      <answer type="multiple_choice">pass</answer>
+      <answer_choices>
+        <value type="str">none</value>
+        <value type="str">pass</value>
+        <value type="str">fail</value>
+        <value type="str">skip</value>
+      </answer_choices>
+      <comment>&#34;&#39;&lt;&amp;&gt;</comment>
+    </question>
+    <question name="2013.com.canonical.plainbox::&#34;&#39;&lt;&amp;&gt;-9-user-interact">
+      <answer type="multiple_choice">pass</answer>
+      <answer_choices>
+        <value type="str">none</value>
+        <value type="str">pass</value>
+        <value type="str">fail</value>
+        <value type="str">skip</value>
+      </answer_choices>
+      <comment>&#34;&#39;&lt;&amp;&gt;</comment>
+    </question>
+  </questions>
+  <software>
+    <lsbrelease>
+      <property name="codename" type="str">{evil}</property>
+      <property name="description" type="str">{evil}</property>
+      <property name="distributor_id" type="str">{evil}</property>
+      <property name="release" type="str">{evil}</property>
+    </lsbrelease>
+    <packages>
+      <package id="0" name="{evil}">
+        <property name="version" type="str">{evil}</property>
+      </package>
+      <package id="1" name="{evil}">
+        <property name="version" type="str">{evil}</property>
+      </package>
+    </packages>
+    <requirements>
+      <requirement id=" 0" name="{evil}">
+        <property name="link" type="str">{evil}</property>
+      </requirement>
+      <requirement id=" 1" name="{evil}">
+        <property name="link" type="str">{evil}</property>
+      </requirement>
+    </requirements>
+  </software>
+  <summary>
+    <client name="{evil}" version="{evil}"/>
+    <date_created value="{evil}"/>
+    <architecture value="{evil}"/>
+    <distribution value="{evil}"/>
+    <distroseries value="{evil}"/>
+    <kernel-release value="{evil}"/>
+    <private value="False"/>
+    <contactable value="False"/>
+    <live_cd value="False"/>
+    <system_id value="{evil}"/>
+  </summary>
+</system>""".format(evil=_escaped_evil_text)

=== modified file 'plainbox/plainbox/impl/exporter/text.py'
--- plainbox/plainbox/impl/exporter/text.py	2015-03-31 08:38:35 +0000
+++ plainbox/plainbox/impl/exporter/text.py	2015-05-14 11:57:31 +0000
@@ -25,15 +25,15 @@
 
     THIS MODULE DOES NOT HAVE STABLE PUBLIC API
 """
+from plainbox.i18n import gettext as _
 from plainbox.impl.commands.inv_run import Colorizer
 from plainbox.impl.exporter import SessionStateExporterBase
 from plainbox.impl.result import outcome_meta
 
 
 class TextSessionStateExporter(SessionStateExporterBase):
-    """
-    Human-readable session state exporter.
-    """
+
+    """Human-readable session state exporter."""
 
     def __init__(self, option_list=None, color=None):
         super().__init__(option_list)
@@ -55,9 +55,23 @@
                             outcome_meta(state.result.outcome).color_ansi
                         ), state.job.tr_summary(),
                     ).encode("UTF-8"))
+                if len(state.result_history) > 1:
+                    stream.write(_("  history: {0}\n").format(
+                        ', '.join(
+                            self.C.custom(
+                                result.outcome_meta().tr_outcome,
+                                result.outcome_meta().color_ansi)
+                            for result in state.result_history)
+                    ).encode("UTF-8"))
             else:
                 stream.write(
                     "{:^15}: {}\n".format(
                         state.result.tr_outcome(),
                         state.job.tr_summary(),
                     ).encode("UTF-8"))
+                if state.result_history:
+                    print(_("History:"), ', '.join(
+                        self.C.custom(
+                            result.outcome_meta().unicode_sigil,
+                            result.outcome_meta().color_ansi)
+                        for result in state.result_history))

=== modified file 'plainbox/plainbox/impl/pod.py'
--- plainbox/plainbox/impl/pod.py	2015-04-16 20:49:41 +0000
+++ plainbox/plainbox/impl/pod.py	2015-05-14 11:57:31 +0000
@@ -1,3 +1,4 @@
+# encoding: utf-8
 # This file is part of Checkbox.
 #
 # Copyright 2012-2015 Canonical Ltd.
@@ -75,7 +76,7 @@
 
 class _Singleton:
 
-    """ A simple object()-like singleton that has a more useful repr(). """
+    """A simple object()-like singleton that has a more useful repr()."""
 
     def __repr__(self):
         return self.__class__.__name__
@@ -202,7 +203,7 @@
 
     def __init__(self, doc=None, type=None, initial=None, initial_fn=None,
                  notify=False, notify_fn=None, assign_filter_list=None):
-        """ Initialize (define) a new POD field. """
+        """Initialize (define) a new POD field."""
         self.__doc__ = dedent(doc) if doc is not None else None
         self.type = type
         self.initial = initial
@@ -224,13 +225,47 @@
         self.counter = self.__class__._counter
         self.__class__._counter += 1
 
+    @property
+    def change_notifier(self):
+        """
+        Decorator for changing the change notification function.
+
+        This decorator can be used to define all the fields in one block and
+        all the notification function in another block. It helps to make the
+        code easier to read.
+
+        Example::
+
+            >>> class Person(POD):
+            ...     name = Field()
+            ...
+            ...     @name.change_notifier
+            ...     def _name_changed(self, old, new):
+            ...         print("changed from {!r} to {!r}".format(old, new))
+            >>> person = Person()
+            changed from UNSET to None
+            >>> person.name = "bob"
+            changed from None to 'bob'
+
+        .. note::
+            Keep in mind that the decorated function is converted to a signal
+            automatically. The name of the function is also irrelevant, the POD
+            core automatically creates signals that have consistent names of
+            ``on_{field}_changed()``.
+        """
+        def decorator(fn):
+            self.notify = True
+            self.notify_fn = fn
+            return fn
+        return decorator
+
     def __repr__(self):
-        """ Get a debugging representation of a field. """
+        """Get a debugging representation of a field."""
         return "<{} name:{!r}>".format(self.__class__.__name__, self.name)
 
     @property
     def is_mandatory(self) -> bool:
-        """ Flag indicating if the field needs a mandatory initializer. """
+        """Flag indicating if the field needs a mandatory initializer."""
         return self.initial is MANDATORY
 
     def gain_name(self, name: str) -> None:
@@ -328,7 +363,7 @@
 @total_ordering
 class PODBase:
 
-    """ Base class for POD-like classes. """
+    """Base class for POD-like classes."""
 
     field_list = []
     namedtuple_cls = namedtuple('PODBase', '')
@@ -386,7 +421,7 @@
             setattr(self, field.name, field_value)
 
     def __repr__(self):
-        """ Get a debugging representation of a POD object. """
+        """Get a debugging representation of a POD object."""
         return "{}({})".format(
             self.__class__.__name__,
             ', '.join([
@@ -428,10 +463,16 @@
         ])
 
     def as_dict(self) -> dict:
-        """ Return the data in this POD as a dictionary. """
+        """
+        Return the data in this POD as a dictionary.
+
+        .. note::
+            UNSET values are not added to the dictionary.
+        """
         return {
             field.name: getattr(self, field.name)
             for field in self.__class__.field_list
+            if getattr(self, field.name) is not UNSET
         }
 
 
@@ -457,7 +498,7 @@
         self.field_origin_map = {}  # field name -> defining class name
 
     def inspect_cls_for_decorator(self, cls: type) -> None:
-        """ Analyze a bare POD class. """
+        """Analyze a bare POD class."""
         self.inspect_base_classes(cls.__bases__)
         self.inspect_namespace(cls.__dict__, cls.__name__)
 
@@ -727,6 +768,39 @@
 typed = type_check_assign_filter
 
 
+@modify_field_docstring(
+    "unset or type-checked (value must be of type {field.type.__name__})")
+def unset_or_type_check_assign_filter(
+        instance: POD, field: Field, old: "Any", new: "Any") -> "Any":
+    """
+    An assign filter that type-checks the value according to the field type.
+
+    .. note::
+        This filter allows (passes through) the special ``UNSET`` value as-is.
+
+    The field must have a valid python type object stored in the .type field.
+
+    :param instance:
+        A subclass of :class:`POD` that contains ``field``
+    :param field:
+        The :class:`Field` being assigned to
+    :param old:
+        The current value of the field
+    :param new:
+        The proposed value of the field
+    :returns:
+        ``new``, as-is
+    :raises TypeError:
+        if ``new`` is not an instance of ``field.type``
+    """
+    if new is UNSET:
+        return new
+    return type_check_assign_filter(instance, field, old, new)
+
+
+unset_or_typed = unset_or_type_check_assign_filter
+
+
 class sequence_type_check_assign_filter:
 
     """
@@ -781,6 +855,55 @@
 typed.sequence = sequence_type_check_assign_filter
 
 
+class unset_or_sequence_type_check_assign_filter(typed.sequence):
+
+    """
+    Assign filter for typed sequences.
+
+    .. note::
+        This filter allows (passes through) the special ``UNSET`` value as-is.
+
+    An assign filter for typed sequences (lists or tuples) that must contain an
+    object of the given type.
+    """
+
+    @property
+    def field_docstring_ext(self) -> str:
+        return (
+            "unset or type-checked sequence (items must be of type {})"
+        ).format(self.item_type.__name__)
+
+    def __call__(
+            self, instance: POD, field: Field, old: "Any", new: "Any"
+    ) -> "Any":
+        """
+        An assign filter that type-checks the value of all sequence elements.
+
+        .. note::
+            This filter allows (passes through) the special ``UNSET`` value
+            as-is.
+
+        :param instance:
+            A subclass of :class:`POD` that contains ``field``
+        :param field:
+            The :class:`Field` being assigned to
+        :param old:
+            The current value of the field
+        :param new:
+            The proposed value of the field
+        :returns:
+            ``new``, as-is
+        :raises TypeError:
+            if ``new`` is not an instance of ``field.type``
+        """
+        if new is UNSET:
+            return new
+        return super().__call__(instance, field, old, new)
+
+
+unset_or_typed.sequence = unset_or_sequence_type_check_assign_filter
+
+
 @modify_field_docstring("unique elements (sequence elements cannot repeat)")
 def unique_elements_assign_filter(
         instance: POD, field: Field, old: "Any", new: "Any") -> "Any":

=== modified file 'plainbox/plainbox/impl/providers/special.py'
--- plainbox/plainbox/impl/providers/special.py	2015-03-30 17:36:35 +0000
+++ plainbox/plainbox/impl/providers/special.py	2015-05-14 11:57:31 +0000
@@ -52,8 +52,8 @@
     return stubbox_def
 
 
-def get_stubbox():
-    return Provider1.from_definition(get_stubbox_def(), secure=False)
+def get_stubbox(**kwargs):
+    return Provider1.from_definition(get_stubbox_def(), secure=False, **kwargs)
 
 
 def get_categories_def():

=== added file 'plainbox/plainbox/impl/providers/stubbox/units/jobs/representative.pxu'
--- plainbox/plainbox/impl/providers/stubbox/units/jobs/representative.pxu	1970-01-01 00:00:00 +0000
+++ plainbox/plainbox/impl/providers/stubbox/units/jobs/representative.pxu	2015-05-14 11:57:31 +0000
@@ -0,0 +1,98 @@
+# Definitions of jobs that are useful for testing. Whenever you need a scenario
+# for testing that involves realistic jobs and you don't want to painstakingly
+# define them manually just load stubbox and get all jobs matching the pattern
+# 'representative/plugin/.*'
+
+# NOTE:: all of the jobs below can be simplified to a template once static
+# resources are available.
+
+id: representative/plugin/shell
+_summary: Job with plugin=shell
+_description: Job with plugin=shell
+plugin: shell
+flags: preserve-locale
+command: true;
+estimated_duration: 0.1
+category_id: plugin-representative
+
+id: representative/plugin/resource
+_summary: Job with plugin=resource
+_description: Job with plugin=resource
+plugin: resource
+flags: preserve-locale
+command:
+ echo "key_a: value-a-1"
+ echo "key_b: value-b-1"
+ echo
+ echo "key_a: value-a-2"
+ echo "key_b: value-b-2"
+estimated_duration: 0.1
+category_id: plugin-representative
+
+id: representative/plugin/local
+_summary: Job with plugin=local
+_description: Job with plugin=local
+plugin: local
+flags: preserve-locale
+command: :
+estimated_duration: 0.1
+category_id: plugin-representative
+
+id: representative/plugin/attachment
+_summary: Job with plugin=attachment
+_description: Job with plugin=attachment
+plugin: attachment
+flags: preserve-locale
+command:
+ echo "Line 1"
+ echo "Line 2"
+ echo "Line 3 (last)"
+estimated_duration: 0.1
+category_id: plugin-representative
+
+id: representative/plugin/user-interact
+_summary: Job with plugin=user-interact
+_description: Job with plugin=user-interact
+plugin: user-interact
+flags: preserve-locale
+command:
+ echo "(interaction)"
+estimated_duration: 30
+category_id: plugin-representative
+
+id: representative/plugin/user-verify
+_summary: Job with plugin=user-verify
+_description: Job with plugin=user-verify
+plugin: user-verify
+flags: preserve-locale
+command:
+ echo "(verification)"
+estimated_duration: 30
+category_id: plugin-representative
+
+id: representative/plugin/user-interact-verify
+_summary: Job with plugin=user-interact-verify
+_description: Job with plugin=user-interact-verify
+plugin: user-interact-verify
+flags: preserve-locale
+command:
+ echo "(interaction)"
+ echo "(verification)"
+estimated_duration: 30
+category_id: plugin-representative
+
+id: representative/plugin/manual
+_summary: Job with plugin=manual
+_description: Job with plugin=manual
+plugin: manual
+estimated_duration: 1
+category_id: plugin-representative
+
+id: representative/plugin/qml
+_summary: Job with plugin=qml
+_description: Job with plugin=qml
+plugin: qml
+qml_file: qml-simple.qml
+flags: preserve-locale
+estimated_duration: 10
+category_id: plugin-representative

=== modified file 'plainbox/plainbox/impl/resource.py'
--- plainbox/plainbox/impl/resource.py	2014-09-13 11:37:17 +0000
+++ plainbox/plainbox/impl/resource.py	2015-05-14 11:57:31 +0000
@@ -88,6 +88,10 @@
             data = {}
         object.__setattr__(self, '_data', data)
 
+    def __iter__(self):
+        data = object.__getattribute__(self, '_data')
+        return iter(data)
+
     def __setattr__(self, attr, value):
         if attr.startswith("_"):
             raise AttributeError(attr)

=== modified file 'plainbox/plainbox/impl/result.py'
--- plainbox/plainbox/impl/result.py	2015-04-16 20:49:41 +0000
+++ plainbox/plainbox/impl/result.py	2015-05-14 11:57:31 +0000
@@ -1,13 +1,13 @@
+# encoding: utf-8
 # This file is part of Checkbox.
 #
-# Copyright 2012 Canonical Ltd.
+# Copyright 2012-2015 Canonical Ltd.
 # Written by:
 #   Zygmunt Krynicki <[e-mail]>
 #
 # Checkbox is free software: you can redistribute it and/or modify
 # it under the terms of the GNU General Public License version 3,
 # as published by the Free Software Foundation.
-
 #
 # Checkbox is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of
@@ -18,6 +18,8 @@
 # along with Checkbox.  If not, see <http://www.gnu.org/licenses/>.
 
 """
+Implementation of job result (test result) classes.
+
 :mod:`plainbox.impl.result` -- job result
 =========================================
 
@@ -25,19 +27,21 @@
 :class:`MemoryJobResult` and :class:`DiskJobResult`.
 """
 
-from collections import namedtuple
 import base64
 import codecs
 import gzip
+import inspect
 import io
 import json
 import logging
-import inspect
 import re
+from collections import namedtuple
 
 from plainbox.abc import IJobResult
 from plainbox.i18n import gettext as _
 from plainbox.i18n import pgettext as C_
+from plainbox.impl import pod
+from plainbox.impl.decorators import raises
 from plainbox.vendor import morris
 
 logger = logging.getLogger("plainbox.result")
@@ -209,34 +213,82 @@
 
 
 def tr_outcome(outcome):
-    """
-    Get the translated value of ``OUTCOME_`` constant
-    """
+    """Get the translated value of ``OUTCOME_`` constant."""
     return OUTCOME_METADATA_MAP[outcome].tr_outcome
 
 
 def outcome_color_hex(outcome):
-    """
-    Get the hexadecimal "#RRGGBB" color that represents this outcome
-    """
+    """Get the hexadecimal "#RRGGBB" color that represents this outcome."""
     return OUTCOME_METADATA_MAP[outcome].color_hex
 
 
 def outcome_color_ansi(outcome):
-    """
-    Get an ANSI escape sequence that represents this outcome
-    """
+    """Get an ANSI escape sequence that represents this outcome."""
     return OUTCOME_METADATA_MAP[outcome].color_ansi
 
 
 def outcome_meta(outcome):
-    """
-    Get the OutcomeMetadata object associated with this outcome.
-    """
+    """Get the OutcomeMetadata object associated with this outcome."""
     return OUTCOME_METADATA_MAP[outcome]
 
 
+class JobResultBuilder(pod.POD):
+
+    """A builder for job result objects."""
+
+    outcome = pod.Field(
+        'outcome of a test',
+        str, pod.UNSET, assign_filter_list=[pod.unset_or_typed])
+    execution_duration = pod.Field(
+        'time of test execution',
+        float, pod.UNSET, assign_filter_list=[pod.unset_or_typed])
+    comments = pod.Field(
+        'comments from the test operator',
+        str, pod.UNSET, assign_filter_list=[pod.unset_or_typed])
+    return_code = pod.Field(
+        'return code from the (optional) test process',
+        int, pod.UNSET, assign_filter_list=[pod.unset_or_typed])
+    io_log = pod.Field(
+        'history of the I/O log of the (optional) test process',
+        list, pod.UNSET, assign_filter_list=[
+            pod.unset_or_typed, pod.unset_or_typed.sequence(tuple)])
+    io_log_filename = pod.Field(
+        'path to a structured I/O log file of the (optional) test process',
+        str, pod.UNSET, assign_filter_list=[pod.unset_or_typed])
+
+    def add_comment(self, comment):
+        """
+        Add a new comment.
+
+        The comment is safely combined with any prior comments.
+        """
+        if self.comments is pod.UNSET:
+            self.comments = comments
+        else:
+            self.comments += '\n' + comment
+
+    @raises(ValueError)
+    def seal(self):
+        """
+        Seal the current state of the builder and create a new result.
+
+        :returns:
+            A new MemoryJobResult or DiskJobResult with all the data
+        :raises ValueError:
+            If both io_log and io_log_filename were used.
+        """
+        if not (self.io_log_filename is pod.UNSET or self.io_log is pod.UNSET):
+            raise ValueError(
+                "you can use only io_log or io_log_filename at a time")
+        if self.io_log_filename is not pod.UNSET:
+            cls = DiskJobResult
+        else:
+            cls = MemoryJobResult
+        return cls(self.as_dict())
+
+
 class _JobResultBase(IJobResult):
+
     """
     Base class for :`IJobResult` implementations.
 
@@ -245,7 +297,7 @@
 
     def __init__(self, data):
         """
-        Initialize a new result with the specified data
+        Initialize a new result with the specified data.
 
         Data is a dictionary that can hold arbitrary values. At least some
         values are explicitly used, such as 'outcome', 'comments' and
@@ -259,6 +311,10 @@
             key: value for key, value in data.items()
             if value is not None and value != []}
 
+    def unseal(self):
+        """Create a new job result builder from the data in this result."""
+        return JobResultBuilder(**self._data)
+
     def __eq__(self, other):
         if not isinstance(other, _JobResultBase):
             return NotImplemented
@@ -275,9 +331,7 @@
 
     @morris.signal
     def on_outcome_changed(self, old, new):
-        """
-        Signal sent when ``outcome`` property value is changed
-        """
+        """Signal sent when ``outcome`` property value is changed."""
 
     @property
     def outcome(self):
@@ -292,55 +346,46 @@
 
     @outcome.setter
     def outcome(self, new):
+        logger.warning("writing to result.outcome is deprecated!")
         old = self.outcome
         if old != new:
             self._data['outcome'] = new
             self.on_outcome_changed(old, new)
 
     def tr_outcome(self):
-        """
-        Get the translated value of the outcome
-        """
+        """Get the translated value of the outcome."""
         return tr_outcome(self.outcome)
 
     def outcome_color_hex(self):
-        """
-        Get the hexadecimal "#RRGGBB" color that represents this outcome
-        """
+        """Get the hexadecimal "#RRGGBB" color that represents this outcome."""
         return outcome_color_hex(self.outcome)
 
     def outcome_color_ansi(self):
-        """
-        Get an ANSI escape sequence that represents this outcome
-        """
+        """Get an ANSI escape sequence that represents this outcome."""
         return outcome_color_ansi(self.outcome)
 
     def outcome_meta(self):
-        """
-        Get the OutcomeMetadata object associated with this outcome.
-        """
+        """Get the OutcomeMetadata object associated with this outcome."""
         return outcome_meta(self.outcome)
 
     @property
     def execution_duration(self):
-        """
-        The amount of time in seconds it took to run this job.
-        """
+        """The amount of time in seconds it took to run this job."""
         return self._data.get('execution_duration')
 
     @execution_duration.setter
     def execution_duration(self, elapsed):
+        logger.warning("writing to result.execution_duration is deprecated!")
         self._data['execution_duration'] = elapsed
 
     @property
     def comments(self):
-        """
-        comments of the test operator
-        """
+        """Get the comments of the test operator."""
         return self._data.get('comments')
 
     @comments.setter
     def comments(self, new):
+        logger.warning("writing to result.comments is deprecated!")
         old = self.comments
         if old != new:
             self._data['comments'] = new
@@ -348,7 +393,7 @@
 
     def append_comments(self, comments):
         """
-        Append new comments to the result
+        Append new comments to the result.
 
         :param comments:
             The comments to append
@@ -365,15 +410,11 @@
 
     @morris.signal
     def on_comments_changed(self, old, new):
-        """
-        Signal sent when ``comments`` property value is changed
-        """
+        """Signal sent when ``comments`` property value is changed."""
 
     @property
     def return_code(self):
-        """
-        return code of the command associated with the job, if any
-        """
+        """return code of the command associated with the job, if any."""
         return self._data.get('return_code')
 
     @property
@@ -383,6 +424,8 @@
     @property
     def io_log_as_flat_text(self):
         """
+        Perform a lossly conversion from the binary I/O log to text.
+
         Convert the I/O log to a text string, replacing non unicode characters
         with U+FFFD, the REPLACEMENT CHARACTER.
 
@@ -413,6 +456,8 @@
     @property
     def io_log_as_text_attachment(self):
         """
+        Perform a conversion of the binary I/O log to text, if possible.
+
         Convert the I/O log to text attachment, if possible, otherwise return
         an empty string.
 
@@ -437,7 +482,7 @@
     @property
     def is_hollow(self):
         """
-        flag that indicates if the result is hollow
+        flag that indicates if the result is hollow.
 
         Hollow results may have been created but hold no data at all.
         Hollow results are also tentatively deprecated, once we have some
@@ -451,6 +496,7 @@
 
 
 class MemoryJobResult(_JobResultBase):
+
     """
     A :class:`IJobResult` that keeps IO logs in memory.
 
@@ -472,8 +518,9 @@
 
 
 class GzipFile(gzip.GzipFile):
+
     """
-    Subclass of GzipFile that works around missing read1() on python3.2
+    Subclass of GzipFile that works around missing read1() on python3.2.
 
     See: http://bugs.python.org/issue10791
     """
@@ -483,6 +530,7 @@
 
 
 class DiskJobResult(_JobResultBase):
+
     """
     A :class:`IJobResult` that keeps IO logs on disk.
 
@@ -495,9 +543,7 @@
 
     @property
     def io_log_filename(self):
-        """
-        pathname of the file containing serialized IO log records
-        """
+        """pathname of the file containing serialized IO log records."""
         return self._data.get("io_log_filename")
 
     def get_io_log(self):
@@ -519,9 +565,8 @@
 
 
 class IOLogRecordWriter:
-    """
-    Class for writing :class:`IOLogRecord` instances to a text stream
-    """
+
+    """Class for writing :class:`IOLogRecord` instances to a text stream."""
 
     def __init__(self, stream):
         self.stream = stream
@@ -530,9 +575,7 @@
         self.stream.close()
 
     def write_record(self, record):
-        """
-        Write an :class:`IOLogRecord` to the stream.
-        """
+        """Write an :class:`IOLogRecord` to the stream."""
         text = json.dumps([
             record[0], record[1],
             base64.standard_b64encode(record[2]).decode("ASCII")],
@@ -545,9 +588,8 @@
 
 
 class IOLogRecordReader:
-    """
-    Class for streaming :class`IOLogRecord` instances from a text stream
-    """
+
+    """Class for streaming :class`IOLogRecord` instances from a text stream."""
 
     def __init__(self, stream):
         self.stream = stream
@@ -572,8 +614,9 @@
 
     def __iter__(self):
         """
-        Iterate over the entire stream generating subsequent
-        :class:`IOLogRecord` entries.
+        Iterate over the entire stream generating subsequent records.
+
+        This method generates subsequent :class:`IOLogRecord` entries.
         """
         while True:
             record = self.read_record()

=== modified file 'plainbox/plainbox/impl/runner.py'
--- plainbox/plainbox/impl/runner.py	2015-04-16 20:49:41 +0000
+++ plainbox/plainbox/impl/runner.py	2015-05-14 11:57:31 +0000
@@ -1,13 +1,13 @@
+# encoding: utf-8
 # This file is part of Checkbox.
 #
-# Copyright 2012, 2013 Canonical Ltd.
+# Copyright 2012-2015 Canonical Ltd.
 # Written by:
 #   Zygmunt Krynicki <[e-mail]>
 #
 # Checkbox is free software: you can redistribute it and/or modify
 # it under the terms of the GNU General Public License version 3,
 # as published by the Free Software Foundation.
-
 #
 # Checkbox is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of
@@ -18,6 +18,8 @@
 # along with Checkbox.  If not, see <http://www.gnu.org/licenses/>.
 
 """
+Definition of JobRunner class.
+
 :mod:`plainbox.impl.runner` -- job runner
 =========================================
 
@@ -36,14 +38,12 @@
 import sys
 import time
 
-from plainbox.vendor import extcmd
-
-from plainbox.abc import IJobRunner, IJobResult
+from plainbox.abc import IJobResult, IJobRunner
 from plainbox.i18n import gettext as _
-from plainbox.impl.result import DiskJobResult
 from plainbox.impl.result import IOLogRecord
 from plainbox.impl.result import IOLogRecordWriter
-from plainbox.impl.result import MemoryJobResult
+from plainbox.impl.result import JobResultBuilder
+from plainbox.vendor import extcmd
 from plainbox.vendor import morris
 
 
@@ -51,23 +51,19 @@
 
 
 def slugify(_string):
-    """
-    Slugify - like Django does for URL - transform a random string to a valid
-    slug that can be later used in filenames
-    """
+    """Transform any string to onet that can be used in filenames."""
     valid_chars = frozenset(
         "-_.{}{}".format(string.ascii_letters, string.digits))
     return ''.join(c if c in valid_chars else '_' for c in _string)
 
 
 class IOLogRecordGenerator(extcmd.DelegateBase):
-    """
-    Delegate for extcmd that generates io_log entries.
-    """
+
+    """Delegate for extcmd that generates io_log entries."""
 
     def on_begin(self, args, kwargs):
         """
-        Internal method of extcmd.DelegateBase
+        Internal method of extcmd.DelegateBase.
 
         Called when a command is being invoked.
 
@@ -77,7 +73,7 @@
 
     def on_line(self, stream_name, line):
         """
-        Internal method of extcmd.DelegateBase
+        Internal method of extcmd.DelegateBase.
 
         Creates a new IOLogRecord and passes it to :meth:`on_new_record()`.
         Maintains a timestamp of the last message so that approximate delay
@@ -92,7 +88,7 @@
     @morris.signal
     def on_new_record(self, record):
         """
-        Internal signal method of :class:`IOLogRecordGenerator`
+        Internal signal method of :class:`IOLogRecordGenerator`.
 
         Called when a new record is generated and needs to be processed.
         """
@@ -101,6 +97,7 @@
 
 
 class CommandOutputWriter(extcmd.DelegateBase):
+
     """
     Delegate for extcmd that writes output to a file on disk.
 
@@ -120,7 +117,7 @@
 
     def on_begin(self, args, kwargs):
         """
-        Internal method of extcmd.DelegateBase
+        Internal method of extcmd.DelegateBase.
 
         Called when a command is being invoked
         """
@@ -129,7 +126,7 @@
 
     def on_end(self, returncode):
         """
-        Internal method of extcmd.DelegateBase
+        Internal method of extcmd.DelegateBase.
 
         Called when a command finishes running
         """
@@ -138,7 +135,7 @@
 
     def on_abnormal_end(self, signal_num):
         """
-        Internal method of extcmd.DelegateBase
+        Internal method of extcmd.DelegateBase.
 
         Called when a command abnormally finishes running
         """
@@ -147,7 +144,7 @@
 
     def on_line(self, stream_name, line):
         """
-        Internal method of extcmd.DelegateBase
+        Internal method of extcmd.DelegateBase.
 
         Called for each line of output.
         """
@@ -158,6 +155,7 @@
 
 
 class FallbackCommandOutputPrinter(extcmd.DelegateBase):
+
     """
     Delegate for extcmd that prints all output to stdout.
 
@@ -166,11 +164,19 @@
     """
 
     def __init__(self, prompt):
+        """Initialize a new fallback command output printer."""
         self._prompt = prompt
         self._lineno = collections.defaultdict(int)
         self._abort = False
 
     def on_line(self, stream_name, line):
+        """
+        Internal method of extcmd.DelegateBase.
+
+        Called for each line of output. Normally each line is just printed
+        (assuming UTF-8 encoding) If decoding fails for any reason that and all
+        subsequent lines are ignored.
+        """
         if self._abort:
             return
         self._lineno[stream_name] += 1
@@ -183,8 +189,9 @@
 
 
 class JobRunnerUIDelegate(extcmd.DelegateBase):
+
     """
-    Delegate for extcmd that delegates extcmd events to IJobRunnerUI
+    Delegate for extcmd that delegates extcmd events to IJobRunnerUI.
 
     The file itself is only opened once on_begin() gets called by extcmd. This
     makes it safe to instantiate this without worrying about dangling
@@ -197,7 +204,7 @@
 
     def __init__(self, ui=None):
         """
-        Initialize the JobRunnerUIDelegate
+        Initialize the JobRunnerUIDelegate.
 
         :param ui:
             (optional) an instnace of IJobRunnerUI to delegate events to
@@ -206,7 +213,7 @@
 
     def on_begin(self, args, kwargs):
         """
-        Internal method of extcmd.DelegateBase
+        Internal method of extcmd.DelegateBase.
 
         Called when a command is being invoked
         """
@@ -215,7 +222,7 @@
 
     def on_end(self, returncode):
         """
-        Internal method of extcmd.DelegateBase
+        Internal method of extcmd.DelegateBase.
 
         Called when a command finishes running
         """
@@ -224,7 +231,7 @@
 
     def on_abnormal_end(self, signal_num):
         """
-        Internal method of extcmd.DelegateBase
+        Internal method of extcmd.DelegateBase.
 
         Called when a command abnormally finishes running
 
@@ -236,7 +243,7 @@
 
     def on_line(self, stream_name, line):
         """
-        Internal method of extcmd.DelegateBase
+        Internal method of extcmd.DelegateBase.
 
         Called for each line of output.
         """
@@ -245,7 +252,7 @@
 
     def on_chunk(self, stream_name, chunk):
         """
-        Internal method of extcmd.DelegateBase
+        Internal method of extcmd.DelegateBase.
 
         Called for each chunk of output.
         """
@@ -254,8 +261,9 @@
 
 
 class JobRunner(IJobRunner):
+
     """
-    Runner for jobs - executes jobs and produces results
+    Runner for jobs - executes jobs and produces results.
 
     The runner is somewhat de-coupled from jobs and session. It still carries
     all checkbox-specific logic about the various types of plugins.
@@ -333,7 +341,7 @@
     @property
     def log_leftovers(self):
         """
-        flag controlling if leftover files should be logged
+        flag controlling if leftover files should be logged.
 
         If you wish to connect a custom handler to :meth:`on_leftover_files()`
         then it is advisable to set this property to False so that leftover
@@ -345,6 +353,7 @@
 
     @log_leftovers.setter
     def log_leftovers(self, value):
+        """setter for log_leftovers property."""
         self._log_leftovers = value
 
     def get_warm_up_sequence(self, job_list):
@@ -375,7 +384,7 @@
 
     def run_job(self, job, job_state, config=None, ui=None):
         """
-        Run the specified job an return the result
+        Run the specified job an return the result.
 
         :param job:
             A JobDefinition to run
@@ -412,10 +421,10 @@
         try:
             runner = getattr(self, func_name)
         except AttributeError:
-            return MemoryJobResult({
-                'outcome': IJobResult.OUTCOME_NOT_IMPLEMENTED,
-                'comment': _('This type of job is not supported'),
-            })
+            return JobResultBuilder(
+                outcome=IJobResult.OUTCOME_NOT_IMPLEMENTED,
+                comments=_('This type of job is not supported')
+            ).seal()
         else:
             if self._dry_run and job.plugin not in self._DRY_RUN_PLUGINS:
                 return self._get_dry_run_result(job)
@@ -428,7 +437,7 @@
 
     def run_shell_job(self, job, job_state, config):
         """
-        Method called to run a job with plugin field equal to 'shell'
+        Method called to run a job with plugin field equal to 'shell'.
 
         The 'shell' job implements the following scenario:
 
@@ -448,11 +457,11 @@
         if job.plugin != "shell":
             # TRANSLATORS: please keep 'plugin' untranslated
             raise ValueError(_("bad job plugin value"))
-        return self._just_run_command(job, job_state, config)
+        return self._just_run_command(job, job_state, config).seal()
 
     def run_attachment_job(self, job, job_state, config):
         """
-        Method called to run a job with plugin field equal to 'attachment'
+        Method called to run a job with plugin field equal to 'attachment'.
 
         The 'attachment' job implements the following scenario:
 
@@ -473,11 +482,11 @@
         if job.plugin != "attachment":
             # TRANSLATORS: please keep 'plugin' untranslated
             raise ValueError(_("bad job plugin value"))
-        return self._just_run_command(job, job_state, config)
+        return self._just_run_command(job, job_state, config).seal()
 
     def run_resource_job(self, job, job_state, config):
         """
-        Method called to run a job with plugin field equal to 'resource'
+        Method called to run a job with plugin field equal to 'resource'.
 
         The 'resource' job implements the following scenario:
 
@@ -499,11 +508,11 @@
         if job.plugin != "resource":
             # TRANSLATORS: please keep 'plugin' untranslated
             raise ValueError(_("bad job plugin value"))
-        return self._just_run_command(job, job_state, config)
+        return self._just_run_command(job, job_state, config).seal()
 
     def run_local_job(self, job, job_state, config):
         """
-        Method called to run a job with plugin field equal to 'local'
+        Method called to run a job with plugin field equal to 'local'.
 
         The 'local' job implements the following scenario:
 
@@ -525,11 +534,11 @@
         if job.plugin != "local":
             # TRANSLATORS: please keep 'plugin' untranslated
             raise ValueError(_("bad job plugin value"))
-        return self._just_run_command(job, job_state, config)
+        return self._just_run_command(job, job_state, config).seal()
 
     def run_manual_job(self, job, job_state, config):
         """
-        Method called to run a job with plugin field equal to 'manual'
+        Method called to run a job with plugin field equal to 'manual'.
 
         The 'manual' job implements the following scenario:
 
@@ -551,11 +560,11 @@
         if job.plugin != "manual":
             # TRANSLATORS: please keep 'plugin' untranslated
             raise ValueError(_("bad job plugin value"))
-        return MemoryJobResult({'outcome': IJobResult.OUTCOME_UNDECIDED})
+        return JobResultBuilder(outcome=IJobResult.OUTCOME_UNDECIDED).seal()
 
     def run_user_interact_job(self, job, job_state, config):
         """
-        Method called to run a job with plugin field equal to 'user-interact'
+        Method called to run a job with plugin field equal to 'user-interact'.
 
         The 'user-interact' job implements the following scenario:
 
@@ -592,11 +601,11 @@
         if job.plugin != "user-interact":
             # TRANSLATORS: please keep 'plugin' untranslated
             raise ValueError(_("bad job plugin value"))
-        return self._just_run_command(job, job_state, config)
+        return self._just_run_command(job, job_state, config).seal()
 
     def run_user_verify_job(self, job, job_state, config):
         """
-        Method called to run a job with plugin field equal to 'user-verify'
+        Method called to run a job with plugin field equal to 'user-verify'.
 
         The 'user-verify' job implements the following scenario:
 
@@ -637,15 +646,14 @@
             # TRANSLATORS: please keep 'plugin' untranslated
             raise ValueError(_("bad job plugin value"))
         # Run the command
-        result_cmd = self._just_run_command(job, job_state, config)
+        result_builder = self._just_run_command(job, job_state, config)
         # Maybe ask the user
-        result_cmd.outcome = IJobResult.OUTCOME_UNDECIDED
-        return result_cmd
+        result_builder.outcome = IJobResult.OUTCOME_UNDECIDED
+        return result_builder.seal()
 
     def run_user_interact_verify_job(self, job, job_state, config):
         """
-        Method called to run a job with plugin field equal to
-        'user-interact-verify'
+        Method for running jobs with plugin equal to 'user-interact-verify'.
 
         The 'user-interact-verify' job implements the following scenario:
 
@@ -686,14 +694,14 @@
             # TRANSLATORS: please keep 'plugin' untranslated
             raise ValueError(_("bad job plugin value"))
         # Run the command
-        result_cmd = self._just_run_command(job, job_state, config)
+        result_builder = self._just_run_command(job, job_state, config)
         # Maybe ask the user
-        result_cmd.outcome = IJobResult.OUTCOME_UNDECIDED
-        return result_cmd
+        result_builder.outcome = IJobResult.OUTCOME_UNDECIDED
+        return result_builder.seal()
 
     def run_qml_job(self, job, job_state, config):
         """
-        Method called to run a job with plugin field equal to 'qml'
+        Method called to run a job with plugin field equal to 'qml'.
 
         The 'qml' job implements the following scenario:
 
@@ -713,10 +721,10 @@
         try:
             ctrl = self._get_ctrl_for_job(job)
         except LookupError:
-            return MemoryJobResult({
-                'outcome': IJobResult.OUTCOME_NOT_SUPPORTED,
-                'comment': _('No suitable execution controller is available)'),
-            })
+            return JobResultBuilder(
+                outcome=IJobResult.OUTCOME_NOT_SUPPORTED,
+                comments=_('No suitable execution controller is available)')
+            ).seal()
         # Run the embedded command
         start_time = time.time()
         delegate, io_log_gen = self._prepare_io_handling(job, config)
@@ -755,12 +763,12 @@
         else:
             outcome = IJobResult.OUTCOME_FAIL
         # Create a result object and return it
-        return DiskJobResult({
-            'outcome': outcome,
-            'return_code': return_code,
-            'io_log_filename': record_path,
-            'execution_duration': execution_duration
-        })
+        return JobResultBuilder(
+            outcome=outcome,
+            return_code=return_code,
+            io_log_filename=record_path,
+            execution_duration=execution_duration
+        ).seal()
 
     def _get_dry_run_result(self, job):
         """
@@ -769,26 +777,24 @@
         Returns a result that is used when running in dry-run mode (where we
         don't really test anything)
         """
-        return MemoryJobResult({
-            'outcome': IJobResult.OUTCOME_SKIP,
-            'comments': _("Job skipped in dry-run mode")
-        })
+        return JobResultBuilder(
+            outcome=IJobResult.OUTCOME_SKIP,
+            comments=_("Job skipped in dry-run mode")
+        ).seal()
 
     def _just_run_command(self, job, job_state, config):
         """
         Internal method of JobRunner.
 
-        Runs the command embedded in the job and returns the DiskJobResult that
-        describes the result. If the command cannot be executed it returns
-        a MemoryJobResult instead.
+        Runs the command embedded in the job and returns a JobResultBuilder
+        that describes the result.
         """
         try:
             ctrl = self._get_ctrl_for_job(job)
         except LookupError:
-            return MemoryJobResult({
-                'outcome': IJobResult.OUTCOME_NOT_SUPPORTED,
-                'comment': _('No suitable execution controller is available)'),
-            })
+            return JobResultBuilder(
+                outcome=IJobResult.OUTCOME_NOT_SUPPORTED,
+                comments=_('No suitable execution controller is available)'))
         # Run the embedded command
         start_time = time.time()
         return_code, record_path = self._run_command(
@@ -802,12 +808,11 @@
         else:
             outcome = IJobResult.OUTCOME_FAIL
         # Create a result object and return it
-        return DiskJobResult({
-            'outcome': outcome,
-            'return_code': return_code,
-            'io_log_filename': record_path,
-            'execution_duration': execution_duration
-        })
+        return JobResultBuilder(
+            outcome=outcome,
+            return_code=return_code,
+            io_log_filename=record_path,
+            execution_duration=execution_duration)
 
     def _prepare_io_handling(self, job, config):
         ui_io_delegate = self._command_io_delegate
@@ -912,7 +917,7 @@
 
     def _get_ctrl_for_job(self, job):
         """
-        Get the execution controller most applicable to run this job
+        Get the execution controller most applicable to run this job.
 
         :param job:
             A job definition to run

=== modified file 'plainbox/plainbox/impl/session/jobs.py'
--- plainbox/plainbox/impl/session/jobs.py	2015-04-09 13:20:17 +0000
+++ plainbox/plainbox/impl/session/jobs.py	2015-05-14 11:57:31 +0000
@@ -16,6 +16,8 @@
 # You should have received a copy of the GNU General Public License
 # along with Checkbox.  If not, see <http://www.gnu.org/licenses/>.
 """
+Job State.
+
 :mod:`plainbox.impl.session.jobs` -- jobs state handling
 ========================================================
 
@@ -39,8 +41,9 @@
 
 
 class InhibitionCause(IntEnum):
+
     """
-    There are four possible not-ready causes:
+    There are four possible not-ready causes.
 
         UNDESIRED:
             This job was not selected to run in this session
@@ -58,6 +61,7 @@
         FAILED_RESOURCE:
             This job has a resource requirement that evaluated to a false value
     """
+
     UNDESIRED = 0
     PENDING_DEP = 1
     FAILED_DEP = 2
@@ -68,6 +72,8 @@
 def cause_convert_assign_filter(
         instance: pod.POD, field: pod.Field, old: "Any", new: "Any") -> "Any":
     """
+    Assign filter for for JobReadinessInhibitor.cause.
+
     Custom assign filter for the JobReadinessInhibitor.cause field that
     produces a very specific error message.
     """
@@ -78,6 +84,7 @@
 
 
 class JobReadinessInhibitor(pod.POD):
+
     """
     Class representing the cause of a job not being ready to execute.
 
@@ -122,6 +129,7 @@
             Provides additional context for the problem caused by a failing
             resource expression.
     """
+
     # XXX: PENDING_RESOURCE is not strict, there are multiple states that are
     # clumped here which is something I don't like. A resource may be still
     # "pending" as in PENDING_DEP (it has not ran yet) or it could have ran but
@@ -176,11 +184,13 @@
             ).format(self.cause.name))
 
     def __repr__(self):
+        """Get a custom debugging representation of an inhibitor."""
         return "<{} cause:{} related_job:{!r} related_expression:{!r}>".format(
             self.__class__.__name__, self.cause.name, self.related_job,
             self.related_expression)
 
     def __str__(self):
+        """Get a human-readable text representation of an inhibitor."""
         if self.cause == InhibitionCause.UNDESIRED:
             # TRANSLATORS: as in undesired job
             return _("undesired")
@@ -211,7 +221,10 @@
 
 
 class OverridableJobField(pod.Field):
+
     """
+    A custom Field for modeling job values that can be overridden.
+
     A readable-writable field that has a special initial value ``JOB_VALUE``
     which is interpreted as "load this value from the corresponding job
     definition".
@@ -222,11 +235,13 @@
 
     def __init__(self, job_field, doc=None, type=None, notify=False,
                  assign_filter_list=None):
+        """Initialize a new overridable job field."""
         super().__init__(
             doc, type, JOB_VALUE, None, notify, assign_filter_list)
         self.job_field = job_field
 
     def __get__(self, instance, owner):
+        """Get an overriden (if any) value of an overridable job field."""
         value = super().__get__(instance, owner)
         if value is JOB_VALUE:
             return getattr(instance.job, self.job_field)
@@ -235,22 +250,28 @@
 
 
 def job_assign_filter(instance, field, old_value, new_value):
-    # FIXME: This setter should not exist. job attribute should be
-    # read-only. This is a temporary kludge to get session restoring
-    # over DBus working. Once a solution that doesn't involve setting
-    # a JobState's job attribute is implemented, please remove this
-    # awful method.
+    """
+    A custom setter for the JobState.job.
+
+    .. warning::
+        This setter should not exist. job attribute should be read-only. This
+        is a temporary kludge to get session restoring over DBus working. Once
+        a solution that doesn't involve setting a JobState's job attribute is
+        implemented, please remove this awful method.
+    """
     return new_value
 
 
 def job_via_assign_filter(instance, field, old_value, new_value):
+    """A custom setter for JobState.via_job."""
     if (old_value is not pod.UNSET and not isinstance(new_value, JobDefinition)
-        and not new_value is None):
+            and new_value is not None):
         raise TypeError("via_job must be the actual job, not the checksum")
     return new_value
 
 
 class JobState(pod.POD):
+
     """
     Class representing the state of a job in a session.
 
@@ -284,6 +305,11 @@
         initial_fn=lambda: MemoryJobResult({}),
         notify=True)
 
+    result_history = pod.Field(
+        doc="a tuple of result_history of the associated job",
+        type=tuple, initial=(), notify=True,
+        assign_filter_list=[pod.typed, pod.typed.sequence(IJobResult)])
+
     via_job = pod.Field(
         doc="the parent job definition",
         type=JobDefinition,
@@ -299,16 +325,36 @@
         doc="the effective certification status of this job",
         type=str)
 
+    # NOTE: the `result` property just exposes the last result from the
+    # `result_history` tuple above. The API is used everywhere so it should not
+    # be broken in any way but the way forward is the sequence stored in
+    # `result_history`.
+    #
+    # The one particularly annoying part of this implementation is that each
+    # job state always has at least one result. Even if there was no testing
+    # done yet. This OUTCOME_NONE result needs to be filtered out at various
+    # times. I think it would be better if we could not have it in the
+    # sequence-based API anymore. Otherwise each test will have two
+    # result_history (more if you count things like resuming a session).
+
+    @result.change_notifier
+    def _result_changed(self, old, new):
+        # Don't track the initial assignment over UNSET
+        if old is pod.UNSET:
+            return
+        assert new != old
+        assert isinstance(new, IJobResult)
+        if new.is_hollow:
+            return
+        logger.debug("Appending result %r to history: %r", new, self.result_history)
+        self.result_history += (new,)
+
     def can_start(self):
-        """
-        Quickly check if the associated job can run right now.
-        """
+        """Quickly check if the associated job can run right now."""
         return len(self.readiness_inhibitor_list) == 0
 
     def get_readiness_description(self):
-        """
-        Get a human readable description of the current readiness state
-        """
+        """Get a human readable description of the current readiness state."""
         if self.readiness_inhibitor_list:
             return _("job cannot be started: {}").format(
                 ", ".join((str(inhibitor)

=== modified file 'plainbox/plainbox/impl/session/resume.py'
--- plainbox/plainbox/impl/session/resume.py	2015-04-13 18:20:44 +0000
+++ plainbox/plainbox/impl/session/resume.py	2015-05-14 11:57:31 +0000
@@ -707,13 +707,10 @@
             result = self._build_JobResult(
                 result_repr, self.flags, self.location)
             result_list.append(result)
-        # Show the _LAST_ result to the session. Currently we only store one
-        # result but showing the most recent (last) result should be good
-        # in general.
-        if len(result_list) > 0:
-            logger.debug(
-                _("calling update_job_result(%r, %r)"), job, result_list[-1])
-            session.update_job_result(job, result_list[-1])
+        # Replay each result, one by one
+        for result in result_list:
+            logger.debug(_("calling update_job_result(%r, %r)"), job, result)
+            session.update_job_result(job, result)
 
     @classmethod
     def _restore_SessionState_desired_job_list(cls, session, session_repr):

=== modified file 'plainbox/plainbox/impl/session/suspend.py'
--- plainbox/plainbox/impl/session/suspend.py	2015-04-13 13:58:00 +0000
+++ plainbox/plainbox/impl/session/suspend.py	2015-05-14 11:57:31 +0000
@@ -238,7 +238,7 @@
         }
 
     def _repr_JobResult(self, obj, session_dir):
-        """ Compute the representation of one of IJobResult subclasses.  """
+        """Compute the representation of one of IJobResult subclasses."""
         if isinstance(obj, DiskJobResult):
             return self._repr_DiskJobResult(obj, session_dir)
         elif isinstance(obj, MemoryJobResult):
@@ -510,11 +510,10 @@
                 if not state.result.is_hollow or state.job.id in id_run_list
             },
             "results": {
-                # Currently we store only one result but we may store
-                # more than that in a later version.
-                state.job.id: [self._repr_JobResult(state.result, session_dir)]
+                state.job.id: [self._repr_JobResult(result, session_dir)
+                               for result in state.result_history]
                 for state in obj.job_state_map.values()
-                if not state.result.is_hollow
+                if len(state.result_history) > 0
             },
             "desired_job_list": [
                 job.id for job in obj.desired_job_list

=== modified file 'plainbox/plainbox/impl/session/test_jobs.py'
--- plainbox/plainbox/impl/session/test_jobs.py	2015-04-09 13:20:17 +0000
+++ plainbox/plainbox/impl/session/test_jobs.py	2015-05-14 11:57:31 +0000
@@ -140,6 +140,7 @@
     def test_smoke(self):
         self.assertIsNotNone(self.job_state.result)
         self.assertIs(self.job_state.result.outcome, IJobResult.OUTCOME_NONE)
+        self.assertEqual(self.job_state.result_history, ())
         self.assertEqual(self.job_state.readiness_inhibitor_list, [
             UndesiredJobReadinessInhibitor])
         self.assertEqual(self.job_state.effective_category_id,
@@ -164,6 +165,16 @@
         self.job_state.result = result
         self.assertIs(self.job_state.result, result)
 
+    def test_result_history_keeps_track_of_result_changes(self):
+        # XXX: this example will fail if subsequent results are identical
+        self.assertEqual(self.job_state.result_history, ())
+        result1 = make_job_result(outcome='fail')
+        self.job_state.result = result1
+        self.assertEqual(self.job_state.result_history, (result1,))
+        result2 = make_job_result(outcome='pass')
+        self.job_state.result = result2
+        self.assertEqual(self.job_state.result_history, (result1, result2))
+
     def test_setting_result_fires_signal(self):
         """
         verify that assigning state.result fires the on_result_changed signal

=== modified file 'plainbox/plainbox/impl/test_pod.py'
--- plainbox/plainbox/impl/test_pod.py	2015-01-19 12:33:13 +0000
+++ plainbox/plainbox/impl/test_pod.py	2015-05-14 11:57:31 +0000
@@ -15,6 +15,9 @@
 #
 # You should have received a copy of the GNU General Public License
 # along with Checkbox.  If not, see <http://www.gnu.org/licenses/>.
+
+"""Tests for the plainbox.impl.pod module."""
+
 from doctest import DocTestSuite
 from unittest import TestCase
 
@@ -30,24 +33,36 @@
 
 
 def load_tests(loader, tests, ignore):
+    """
+    Protocol for loading unit tests.
+
+    This function ensures that doctests are executed as well.
+    """
     tests.addTests(DocTestSuite('plainbox.impl.pod'))
     return tests
 
 
 class SingletonTests(TestCase):
 
+    """Tests for several singleton objects."""
+
     def test_MANDATORY_repr(self):
+        """MANDATORY.repr() returns "MANDATORY"."""
         self.assertEqual(repr(MANDATORY), "MANDATORY")
 
     def test_UNSET_repr(self):
+        """UNSET.repr() returns "UNSET"."""
         self.assertEqual(repr(UNSET), "UNSET")
 
 
 class FieldTests(TestCase):
 
+    """Tests for the Field class."""
+
     FIELD_CLS = Field
 
     def setUp(self):
+        """Common set-up code."""
         self.doc = "doc"  # not a mock because it gets set to __doc__
         self.type = mock.Mock(name='type')
         self.initial = mock.Mock(name='initial')
@@ -58,18 +73,14 @@
         self.owner = mock.Mock(name='owner')
 
     def test_initializer(self):
-        """
-        Field initializer properly stored all attributes
-        """
+        """.__init__() stored data correctly."""
         self.assertEqual(self.field.__doc__, self.doc)
         self.assertEqual(self.field.type, self.type)
         self.assertEqual(self.field.initial, self.initial)
         self.assertEqual(self.field.initial_fn, self.initial_fn)
 
     def test_gain_name(self):
-        """
-        Using gain_name() sets three extra attributes
-        """
+        """.gain_name() sets three extra attributes."""
         self.assertIsNone(self.field.name)
         self.assertIsNone(self.field.instance_attr)
         self.assertIsNone(self.field.signal_name)
@@ -79,31 +90,23 @@
         self.assertEqual(self.field.signal_name, "on_abcd_changed")
 
     def test_repr(self):
-        """
-        Field has a working repr() method
-        """
+        """.repr() works as expected."""
         self.field.gain_name("field")
         self.assertEqual(repr(self.field), "<Field name:'field'>")
 
     def test_is_mandatory(self):
-        """
-        Fields with the initial value of MANDATORY are mandatory
-        """
+        """.is_mandatory looks for initial value of MANDATORY."""
         self.field.initial = None
         self.assertFalse(self.field.is_mandatory)
         self.field.initial = MANDATORY
         self.assertTrue(self.field.is_mandatory)
 
     def test_cls_reads(self):
-        """
-        Accessing fields on via the class exposes the field object itself
-        """
+        """.__get__() returns the field if accessed via a class."""
         self.assertIs(self.field.__get__(None, self.owner), self.field)
 
     def test_obj_reads(self):
-        """
-        Accessing fields via an object reads data from the object
-        """
+        """.__get__() reads POD data if accessed via an object."""
         # Reading the field requires the field to know its name
         self.field.gain_name("field")
         self.assertEqual(
@@ -111,18 +114,14 @@
             self.instance._field)
 
     def test_obj_writes(self):
-        """
-        Writing fields via an object writes data to the object
-        """
+        """.__set__() writes POD data."""
         # Writing the field requires the field to know its name
         self.field.gain_name("field")
         self.field.__set__(self.instance, "data")
         self.assertEqual(self.instance._field, "data")
 
     def test_obj_writes_fires_notification(self):
-        """
-        Writing fields via an object triggers notification, if enabled
-        """
+        """.__set__() fires change notifications."""
         # Let's enable notification and set the name so that the field knows
         # what to do when it gets set. Let's set the instance data to "old" to
         # track the actual change.
@@ -135,9 +134,7 @@
         self.instance.on_field_changed.assert_called_with("old", "new")
 
     def test_obj_writes_uses_assign_chain(self):
-        """
-        Writing fields via an object uses the assign filter list
-        """
+        """.__set__() uses the assign filter list."""
         # Let's enable the assign filter composed out of two functions
         # and set some data using the field.
         fn1 = mock.Mock()
@@ -153,9 +150,7 @@
         self.assertEqual(self.instance._field, fn2())
 
     def test_alter_cls_without_notification(self):
-        """
-        Using alter_cls() when notification is disabled does nothing
-        """
+        """.alter_cls() doesn't do anything if notify is False."""
         cls = mock.Mock(name='cls')
         del cls.on_field_changed
         self.field.notify = False
@@ -164,9 +159,7 @@
         self.assertFalse(hasattr(cls, "on_field_changed"))
 
     def test_alter_cls_with_notification(self):
-        """
-        Using alter_cls() when notification is enabled creates a signal
-        """
+        """.alter_cls() adds a change signal if notify is True."""
         cls = mock.Mock(name='cls')
         del cls.on_field_changed
         cls.__name__ = "Klass"
@@ -180,7 +173,10 @@
 
 class FieldCollectionTests(TestCase):
 
+    """Tests for the _FieldCollection class."""
+
     def setUp(self):
+        """Common set-up code."""
         self.foo = Field()
         self.bar = Field()
         self.ns = {
@@ -192,13 +188,12 @@
         self.fc = _FieldCollection()
 
     def set_field_names(self):
+        """Set names of the foo and bar fields."""
         self.foo.gain_name('foo')
         self.bar.gain_name('bar')
 
     def test_add_field_builds_field_list(self):
-        """
-        .add_field() appends new fields to field_list
-        """
+        """.add_field() appends new fields to field_list."""
         # because we're not calling inspect_namespace() which does that
         self.set_field_names()
         self.fc.add_field(self.foo, 'cls')
@@ -207,9 +202,7 @@
         self.assertEqual(self.fc.field_list, [self.foo, self.bar])
 
     def test_add_field_builds_field_origin_map(self):
-        """
-        .add_field() builds and maintains field_origin_map
-        """
+        """.add_field() builds and maintains field_origin_map."""
         # because we're not calling inspect_namespace() which does that
         self.set_field_names()
         self.fc.add_field(self.foo, 'cls')
@@ -219,9 +212,7 @@
             self.fc.field_origin_map, {'foo': 'cls', 'bar': 'cls'})
 
     def test_add_field_detects_clashes(self):
-        """
-        .add_Field() detects field clashes and raises TypeError
-        """
+        """.add_Field() detects field clashes and raises TypeError."""
         foo_clash = Field()
         foo_clash.name = 'foo'
         # because we're not calling inspect_namespace() which does that
@@ -232,10 +223,7 @@
             self.fc.add_field(foo_clash, 'other_cls')
 
     def test_inspect_base_classes_calls_add_field(self):
-        """
-        .inspect_base_classes() calls add_field() on each Field found
-        """
-
+        """.inspect_base_classes() calls add_field() on each Field found."""
         class Base1(POD):
             foo = Field()
             bar = Field()
@@ -255,18 +243,14 @@
             ])
 
     def test_inspect_namespace_calls_add_field(self):
-        """
-        .inspect_namespace() calls add_field() on each Field
-        """
+        """.inspect_namespace() calls add_field() on each Field."""
         with mock.patch.object(self.fc, 'add_field') as mock_add_field:
             self.fc.inspect_namespace(self.ns, 'cls')
         mock_add_field.assert_has_call(self.foo, 'cls')
         mock_add_field.assert_has_call(self.bar, 'cls')
 
     def test_inspect_namespace_sets_field_name(self):
-        """
-        .inspect_namespace() sets the .name attribute of each field.
-        """
+        """.inspect_namespace() sets .name of each field."""
         self.assertIsNone(self.foo.name)
         self.assertIsNone(self.bar.name)
         fc = _FieldCollection()
@@ -275,9 +259,7 @@
         self.assertEqual(self.bar.name, 'bar')
 
     def test_inspect_namespace_sets_field_instance_attr(self):
-        """
-        .inspect_namespace() sets the .instance_attr attribute of each field.
-        """
+        """.inspect_namespace() sets .instance_attr of each field."""
         self.assertIsNone(self.foo.instance_attr)
         self.assertIsNone(self.bar.instance_attr)
         fc = _FieldCollection()
@@ -285,13 +267,21 @@
         self.assertEqual(self.foo.instance_attr, '_foo')
         self.assertEqual(self.bar.instance_attr, '_bar')
 
+    def test_notifier(self):
+        """@field.change_notifier changes the notify function."""
+        @self.foo.change_notifier
+        def on_foo_changed(pod, old, new):
+            pass
+        self.assertTrue(self.foo.notify)
+        self.assertEqual(self.foo.notify_fn, on_foo_changed)
+
 
 class PODTests(TestCase):
 
+    """Tests for the POD class."""
+
     def test_field_list(self):
-        """
-        Test that PODMeta correctly set up the field_list attribute
-        """
+        """.field_list is set by PODMeta."""
         m = mock.Mock()
 
         class T(POD):
@@ -302,9 +292,7 @@
         self.assertEqual(T.field_list, [T.f1, T.f2, T.f3])
 
     def test_namedtuple_cls(self):
-        """
-        Test that PODMeta correctly set up the namedtuple_cls attribute
-        """
+        """Check that .namedtuple_cls is set up by PODMeta."""
         m = mock.Mock()
 
         class T(POD):
@@ -318,9 +306,7 @@
         self.assertIsInstance(T.namedtuple_cls.f3, property)
 
     def test_initializer_positional_arguments(self):
-        """
-        Test initializer operation with positional arguments
-        """
+        """.__init__() works correctly with positional arguments."""
         m = mock.Mock()
 
         class T(POD):
@@ -339,9 +325,7 @@
         self.assertEqual(T(1, 2, 3).f3, 3)
 
     def test_initializer_keyword_arguments(self):
-        """
-        Test initializer operation with positional arguments
-        """
+        """.__init__() works correctly with keyword arguments."""
         m = mock.Mock()
 
         class T(POD):
@@ -363,9 +347,7 @@
         self.assertEqual(T(f1=1, f2=2, f3=3).f3, 3)
 
     def test_initializer_mandatory_arguments(self):
-        """
-        Test initializer's response to mishandling of MANDATORY fields
-        """
+        """.__init__() understands MANDATORY fields."""
         class T(POD):
             m1 = Field(initial=MANDATORY)
             m2 = Field(initial=MANDATORY)
@@ -384,9 +366,7 @@
             T(m1=1)
 
     def test_initializer_default_arguments(self):
-        """
-        Test initializer's response to default values
-        """
+        """.__init__() understands initial (default) field values."""
         class T(POD):
             f = Field(initial=42)
         self.assertEqual(T().f, 42)
@@ -394,9 +374,7 @@
         self.assertEqual(T(f=1).f, 1)
 
     def test_initializer_duplicate_field_value(self):
-        """
-        Test that double initialization is not permitted
-        """
+        """.__init__() prevents double-initialization."""
         class T(POD):
             f = Field()
         with self.assertRaisesRegex(
@@ -404,9 +382,7 @@
             T(1, f=2)
 
     def test_initializer_unknown_field(self):
-        """
-        Test that initializing unknown fields is not permitted
-        """
+        """.__init__() prevents initializing unknown fields."""
         class T(POD):
             pass
         with self.assertRaisesRegex(TypeError, "too many arguments"):
@@ -415,9 +391,7 @@
             T(f=1)
 
     def test_smoke(self):
-        """
-        Test that a simple Person POD can be used to demonstrate basic features
-        """
+        """Check that basic POD behavior works okay."""
         class Person(POD):
             name = Field()
             age = Field()
@@ -452,10 +426,16 @@
         self.assertEqual(
             repr(joe), "Employee(name='Joe', age=42, salary=1000)")
 
+    def test_as_dict_filters_out_UNSET(self):
+        class P(POD):
+            f = Field()
+
+        p = P()
+        p.f = UNSET
+        self.assertEqual(p.as_dict(), {})
+
     def test_notifications(self):
-        """
-        Test that change notifications get sent
-        """
+        """.on_{field}_changed() gets fired by field modification."""
         class T(POD):
             f = Field(notify=True)
 
@@ -471,7 +451,7 @@
         field_callback.assert_called_with(None, 1)
 
     def test_pod_inheritance(self):
-
+        """Check that PODs can be subclassed and new fields can be added."""
         class B(POD):
             f1 = Field(notify=True)
 
@@ -485,14 +465,11 @@
         self.assertEqual(D.field_list, [B.f1, D.f2])
 
     def test_pod_ordering(self):
-        """
-        POD comparison doesn't care about the field names
-        """
-
+        """Check that comparison among single POD class works okay."""
         class A(POD):
             a = Field()
 
-        B = A  # easier to understand subsequent testds
+        B = A  # easier to understand subsequent tests
         self.assertTrue(A(1) == B(1))
         self.assertTrue(A(1) != B(0))
         self.assertTrue(A(0) < B(1))
@@ -501,10 +478,7 @@
         self.assertTrue(A(1) <= B(1))
 
     def test_pod_ordering_tricky1(self):
-        """
-        POD comparison doesn't care about actual classes
-        """
-
+        """Check that comparison among different POD classes works okay."""
         class A(POD):
             f = Field()
 
@@ -519,10 +493,7 @@
         self.assertTrue(A(1) <= B(1))
 
     def test_pod_ordering_tricky2(self):
-        """
-        POD comparison doesn't care about the field names
-        """
-
+        """Check that comparison doesn't care about field names."""
         class A(POD):
             a = Field()
 
@@ -537,10 +508,7 @@
         self.assertTrue(A(1) <= B(1))
 
     def test_pod_ordering_other_types(self):
-        """
-        POD comparison understands other types and is not equal to them
-        """
-
+        """Check that comparison between POD and not-POD types doesn't work."""
         class A(POD):
             f = Field()
 
@@ -551,10 +519,10 @@
 
 class AssignFilterTests(TestCase):
 
+    """Tests for assignment filters."""
+
     def test_read_only_assign_filter(self):
-        """
-        The read_only_assign_filter works as designed
-        """
+        """The read_only_assign_filter works as designed."""
         instance = mock.Mock(name='instance')
         instance.__class__.__name__ = 'cls'
         field = mock.Mock(name='field')
@@ -569,9 +537,7 @@
             read_only_assign_filter(instance, field, old, new)
 
     def test_type_convert_assign_filter(self):
-        """
-        The type_convert_assign_filter works as designed
-        """
+        """The type_convert_assign_filter works as designed."""
         instance = mock.Mock(name='instance')
         old = mock.Mock(name='old')
         field = mock.Mock(name='field')
@@ -585,9 +551,7 @@
             type_convert_assign_filter(instance, field, old, 'hello?')
 
     def test_type_check_assign_filter(self):
-        """
-        The type_convert_assign_filter works as designed
-        """
+        """The type_convert_assign_filter works as designed."""
         instance = mock.Mock(name='instance')
         instance.__class__.__name__ = 'cls'
         old = mock.Mock(name='old')

=== modified file 'plainbox/plainbox/impl/test_result.py'
--- plainbox/plainbox/impl/test_result.py	2015-03-30 19:25:56 +0000
+++ plainbox/plainbox/impl/test_result.py	2015-05-14 11:57:31 +0000
@@ -33,6 +33,7 @@
 from plainbox.impl.result import IOLogRecord
 from plainbox.impl.result import IOLogRecordReader
 from plainbox.impl.result import IOLogRecordWriter
+from plainbox.impl.result import JobResultBuilder
 from plainbox.impl.result import MemoryJobResult
 from plainbox.impl.testing_utils import make_io_log
 
@@ -150,6 +151,7 @@
         })
         self.assertEqual(result.io_log_as_text_attachment, 'foo')
 
+
 class IOLogRecordWriterTests(TestCase):
 
     _RECORD = IOLogRecord(0.123, 'stdout', b'some\ndata')
@@ -180,3 +182,45 @@
         reader = IOLogRecordReader(stream)
         record_list = list(reader)
         self.assertEqual(record_list, [self._RECORD])
+
+
+class JobResultBuildeTests(TestCase):
+
+    def test_smoke_hollow(self):
+        self.assertTrue(JobResultBuilder().seal().is_hollow)
+
+    def test_smoke_memory(self):
+        builder = JobResultBuilder()
+        builder.comments = 'it works'
+        builder.execution_duration = 0.1
+        builder.io_log = [(0, 'stdout', b'ok\n')]
+        builder.outcome = 'pass'
+        builder.return_code = 0
+        result = builder.seal()
+        self.assertEqual(result.comments, "it works")
+        self.assertEqual(result.execution_duration, 0.1)
+        self.assertEqual(result.io_log, (
+            IOLogRecord(delay=0, stream_name='stdout', data=b'ok\n'),))
+        self.assertEqual(result.outcome, "pass")
+        self.assertEqual(result.return_code, 0)
+
+    def test_smoke_disk(self):
+        builder = JobResultBuilder()
+        builder.comments = 'it works'
+        builder.execution_duration = 0.1
+        builder.io_log_filename = 'log'
+        builder.outcome = 'pass'
+        builder.return_code = 0
+        result = builder.seal()
+        self.assertEqual(result.comments, "it works")
+        self.assertEqual(result.execution_duration, 0.1)
+        self.assertEqual(result.io_log_filename, 'log')
+        self.assertEqual(result.outcome, "pass")
+        self.assertEqual(result.return_code, 0)
+
+    def test_io_log_clash(self):
+        builder = JobResultBuilder()
+        builder.io_log = [(0, 'stout', b'hi')]
+        builder.io_log_filename = 'log'
+        with self.assertRaises(ValueError):
+            builder.seal()

=== modified file 'plainbox/setup.py'
--- plainbox/setup.py	2015-05-10 18:38:53 +0000
+++ plainbox/setup.py	2015-05-14 11:57:31 +0000
@@ -70,6 +70,9 @@
         'Topic :: System :: Benchmark',
         'Topic :: Utilities',
     ],
+    install_requires=[
+        'Jinja2 >= 2.7',
+    ],
     extras_require={
         'XLSX': 'XlsxWriter >= 0.3',
         'XML': 'lxml >= 2.3',
@@ -89,6 +92,7 @@
             'xlsx=plainbox.impl.exporter.xlsx:XLSXSessionStateExporter [XLSX]',
             'xml=plainbox.impl.exporter.xml:XMLSessionStateExporter [XML]',
             'html=plainbox.impl.exporter.html:HTMLSessionStateExporter [XML]',
+            'hexr=plainbox.impl.exporter.hexr:HEXRExporter',
         ],
         'plainbox.buildsystem': [
             'make=plainbox.impl.buildsystems:MakefileBuildSystem',

=== modified file 'providers/plainbox-provider-checkbox/whitelists/smoke.whitelist'
--- providers/plainbox-provider-checkbox/whitelists/smoke.whitelist	2014-03-20 14:11:44 +0000
+++ providers/plainbox-provider-checkbox/whitelists/smoke.whitelist	2015-05-14 11:57:31 +0000
@@ -1,27 +1,1 @@
-## This is an example whitelist to start from.
-## To use, copy this file and add the jobs you want to run to the copy. Delete
-## these comments. DO NOT delete the first 9 jobs in tis file. They are resource
-## gathering jobs and are necessary to do any testing at all.
-# Resource Jobs (listed in jobs/resource.txt)
-cpuinfo
-cdimage
-dmi
-dpkg
-efi
-environment
-gconf
-lsb
-meminfo
-module
-package
-device
-uname
-# Smoke test cases
-__smoke__
-smoke/true
-smoke/false
-smoke/dependency/good
-smoke/dependency/bad
-smoke/requirement/good
-smoke/requirement/bad
 smoke/manual

=== modified file 'support/install-deb-dependencies'
--- support/install-deb-dependencies	2014-04-02 13:19:06 +0000
+++ support/install-deb-dependencies	2015-05-14 11:57:31 +0000
@@ -54,3 +54,6 @@
         apt-get install --quiet --quiet --yes $dpkgs_to_install_list
     fi
 fi
+
+# Update to latest packages, this is a bit annoying but this is what it takes
+sudo apt-get dist-upgrade --yes