Merge lp:~kissiel/checkbox/fix-1368001-auto-skipped-rerun into lp:checkbox

[Maciej Kisielewski]

This MR solves two issues.
1) being stuck on rerun screen with checkbox-ng silently trying to rerun failed resource job on and on.
2) being unable to rerun job that got skipped because the job that it depends on failed.

78d5274 checkbox-ng:misc: SelectionTrees no longer smuggle in resource jobs
3c2c0e7 checkbox-ng:commands:newcli: fix empty rerun screen
891044b checkbox-ng:commands:newcli: remove unused code
07f90bb checkbox-ng:commands:newcli: fix rerunning of auto-skipped jobs
5fd3391 checkbox-ng:commands:newcli: sort imports
6fa9c98 checkbox-touch: let auto-skipped jobs be rerun

[Sylvain Pineau (sylvain-pineau)]

Nice fix, thanks for moving the resource selection from the 'misc' to 'newcli' module.

+1

review: Approve

[Sylvain Pineau (sylvain-pineau)]

The attempt to merge lp:~kissiel/checkbox/fix-1368001-auto-skipped-rerun into lp:checkbox failed. Below is the output from the failed tests.

[precise] starting container
[precise] (timing) 0.09user 0.07system 0:04.32elapsed 3%CPU (0avgtext+0avgdata 10204maxresident)k
[precise] (timing) 0inputs+32outputs (0major+8240minor)pagefaults 0swaps
[precise] provisioning container
[precise] (timing) 39.22user 11.92system 1:00.86elapsed 84%CPU (0avgtext+0avgdata 94872maxresident)k
[precise] (timing) 0inputs+19064outputs (0major+1573821minor)pagefaults 0swaps
[precise-testing] Starting tests...
Found a test script: ./checkbox-ng/requirements/container-tests-checkbox-ng-unit
[precise-testing] container-tests-checkbox-ng-unit: FAIL
[precise-testing] stdout: http://paste.ubuntu.com/12772954/
[precise-testing] stderr: http://paste.ubuntu.com/12772955/
[precise-testing] (timing) Command exited with non-zero status 1
[precise-testing] (timing) 0.90user 0.21system 0:01.21elapsed 92%CPU (0avgtext+0avgdata 50484maxresident)k
[precise-testing] (timing) 0inputs+1976outputs (0major+27789minor)pagefaults 0swaps
Found a test script: ./checkbox-support/requirements/container-tests-checkbox-support
[precise-testing] container-tests-checkbox-support: PASS
[precise-testing] (timing) 24.57user 0.29system 0:24.92elapsed 99%CPU (0avgtext+0avgdata 150460maxresident)k
[precise-testing] (timing) 0inputs+1320outputs (0major+40418minor)pagefaults 0swaps
Found a test script: ./checkbox-touch/requirements/container-tests-touch-unit-tests
[precise-testing] container-tests-touch-unit-tests: PASS
[precise-testing] (timing) 0.00user 0.01system 0:00.02elapsed 73%CPU (0avgtext+0avgdata 2184maxresident)k
[precise-testing] (timing) 0inputs+8outputs (0major+2484minor)pagefaults 0swaps
Found a test script: ./plainbox/plainbox/impl/providers/categories/requirements/container-tests-provider-categories
[precise-testing] container-tests-provider-categories: PASS
[precise-testing] (timing) 0.97user 0.13system 0:01.12elapsed 98%CPU (0avgtext+0avgdata 45820maxresident)k
[precise-testing] (timing) 0inputs+64outputs (0major+14255minor)pagefaults 0swaps
Found a test script: ./plainbox/requirements/001-container-tests-plainbox-egg-info
[precise-testing] 001-container-tests-plainbox-egg-info: PASS
[precise-testing] (timing) 0.24user 0.09system 0:00.35elapsed 92%CPU (0avgtext+0avgdata 14508maxresident)k
[precise-testing] (timing) 0inputs+96outputs (0major+6170minor)pagefaults 0swaps
Found a test script: ./plainbox/requirements/container-tests-plainbox
[precise-testing] container-tests-plainbox: PASS
[precise-testing] (timing) 54.79user 1.33system 0:56.31elapsed 99%CPU (0avgtext+0avgdata 196588maxresident)k
[precise-testing] (timing) 0inputs+3256outputs (0major+200515minor)pagefaults 0swaps
Found a test script: ./plainbox/requirements/container-tests-plainbox-documentation
[precise-testing] container-tests-plainbox-documentation: PASS
[precise-testing] (timing) 147.22user 0.57system 2:28.07elapsed 99%CPU (0avgtext+0avgdata 176916maxresident)k
[precise-testing] (timing) 0inputs+39864outputs (0major+53693minor...

[lp:~kissiel/checkbox/fix-1368001-auto-skipped-rerun]

4050. By Maciej Kisielewski on 2015-10-13

checkbox-ng:misc: SelectionTrees no longer smuggle in resource jobs

This patch shifts responsibility of attaching resource jobs to the selection,
from the selection tree to the code using it. This way, when building rerun
screen, resource jobs are no longer smuggled in by default, and the calling
code might decide which resource jobs to include.

Fixes: https://bugs.launchpad.net/checkbox-ng/+bug/1505198

Signed-off-by: Maciej Kisielewski <email address hidden>

4051. By Maciej Kisielewski on 2015-10-13

checkbox-ng:commands:newcli: fix empty rerun screen

Previously, failed resources qualified for rerunning, yet they were not shown
on the rerun selection screen, hence the screen was rendered empty. This patch
fixes that.

Signed-off-by: Maciej Kisielewski <email address hidden>

4052. By Maciej Kisielewski on 2015-10-13

checkbox-ng:commands:newcli: remove unused code

Signed-off-by: Maciej Kisielewski <email address hidden>

4053. By Maciej Kisielewski on 2015-10-13

checkbox-ng:commands:newcli: fix rerunning of auto-skipped jobs

This patch makes jobs autmatically skipped (because of the failed dependencies)
to appear on rerun screen. If selected, they will force reruninng of the
resources they depend on that previously failed.

Fixes: https://bugs.launchpad.net/checkbox-converged/+bug/1368001

Signed-off-by: Maciej Kisielewski <email address hidden>

4054. By Maciej Kisielewski on 2015-10-13

checkbox-ng:commands:newcli: sort imports

Signed-off-by: Maciej Kisielewski <email address hidden>

4055. By Maciej Kisielewski on 2015-10-13

checkbox-touch: let auto-skipped jobs be rerun

Signed-off-by: Maciej Kisielewski <email address hidden>

=== modified file 'checkbox-ng/checkbox_ng/commands/newcli.py'
--- checkbox-ng/checkbox_ng/commands/newcli.py	2015-09-08 13:11:18 +0000
+++ checkbox-ng/checkbox_ng/commands/newcli.py	2015-10-13 13:41:31 +0000
@@ -46,8 +46,9 @@
 from plainbox.impl.secure.qualifiers import RegExpJobQualifier
 from plainbox.impl.secure.qualifiers import select_jobs
 from plainbox.impl.session import SessionMetaData
+from plainbox.impl.session.jobs import InhibitionCause
+from plainbox.impl.transport import TransportError
 from plainbox.impl.transport import get_all_transports
-from plainbox.impl.transport import TransportError
 from plainbox.vendor.textland import get_display
 
 from checkbox_ng.misc import SelectableJobTreeNode
@@ -318,7 +319,7 @@
         # NOTE: tree.selection is correct but ordered badly.  To retain
         # the original ordering we should just treat it as a mask and
         # use it to filter jobs from desired_job_list.
-        wanted_set = frozenset(tree.selection)
+        wanted_set = frozenset(tree.selection + tree.resource_jobs)
         job_list = [job for job in self.manager.state.run_list
                     if job in wanted_set]
         self._update_desired_job_list(job_list)
@@ -491,19 +492,23 @@
                 print(str(exc))
 
     def maybe_rerun_jobs(self):
-        def rerun_predicate(job_state):
-            return job_state.result.outcome in (
-                IJobResult.OUTCOME_FAIL, IJobResult.OUTCOME_CRASH)
         # create a list of jobs that qualify for rerunning
         rerun_candidates = []
         for job in self.manager.state.run_list:
-            if rerun_predicate(self.manager.state.job_state_map[job.id]):
+            job_state = self.manager.state.job_state_map[job.id]
+            if job_state.result.outcome in (
+                IJobResult.OUTCOME_FAIL, IJobResult.OUTCOME_CRASH,
+                    IJobResult.OUTCOME_NOT_SUPPORTED):
                 rerun_candidates.append(job)
+
         # bail-out early if no job qualifies for rerunning
         if not rerun_candidates:
             return False
         tree = SelectableJobTreeNode.create_tree(
             self.manager.state, rerun_candidates)
+        # nothing to select - bailing out
+        if not tree.jobs:
+            return False
         # deselect all by default
         tree.set_descendants_state(False)
         self.display.run(ShowRerun(tree, _("Select jobs to re-run")))
@@ -511,10 +516,15 @@
         if not wanted_set:
             # nothing selected - nothing to run
             return False
-        rerun_job_list = [job for job in self.manager.state.run_list
-                          if job in wanted_set]
+        # include resource jobs that selected jobs depend on
+        resources_to_rerun = []
+        for job in wanted_set:
+            job_state = self.manager.state.job_state_map[job.id]
+            for inhibitor in job_state.readiness_inhibitor_list:
+                if inhibitor.cause == InhibitionCause.FAILED_DEP:
+                    resources_to_rerun.append(inhibitor.related_job)
         # reset outcome of jobs that are selected for re-running
-        for job in wanted_set:
+        for job in list(wanted_set) + resources_to_rerun:
             from plainbox.impl.result import MemoryJobResult
             self.manager.state.job_state_map[job.id].result = \
                 MemoryJobResult({})

=== modified file 'checkbox-ng/checkbox_ng/misc.py'
--- checkbox-ng/checkbox_ng/misc.py	2015-09-17 13:32:38 +0000
+++ checkbox-ng/checkbox_ng/misc.py	2015-10-13 13:41:31 +0000
@@ -349,11 +349,13 @@
         for job in self.job_selection:
             if self.job_selection[job]:
                 self._selection_list.append(job)
-        # Don't forget to append the collected resource jobs to the final
-        # selection
-        self._selection_list.extend(self._resource_jobs)
         return self._selection_list
 
+    @property
+    def resource_jobs(self):
+        """Return all the resource jobs."""
+        return self._resource_jobs
+
     def set_ancestors_state(self, new_state):
         """
         Set the selection state of all ancestors consistently

=== modified file 'checkbox-ng/checkbox_ng/test_misc.py'
--- checkbox-ng/checkbox_ng/test_misc.py	2015-09-17 13:32:38 +0000
+++ checkbox-ng/checkbox_ng/test_misc.py	2015-10-13 13:41:31 +0000
@@ -180,4 +180,4 @@
         # Note that in addition to the selected (D) test, we need the
         # tree selection to contain the resource (F), even though the
         # user never saw it in the previous tests for visual presentation.
-        self.assertEqual(self.tree.selection, [self.D, self.F])
+        self.assertEqual(self.tree.selection, [self.D])

=== modified file 'checkbox-touch/py/checkbox_touch.py'
--- checkbox-touch/py/checkbox_touch.py	2015-10-05 18:39:26 +0000
+++ checkbox-touch/py/checkbox_touch.py	2015-10-13 13:41:31 +0000
@@ -303,7 +303,8 @@
         """Get all the tests that might be selected for rerunning."""
         def rerun_predicate(job_state):
             return job_state.result.outcome in (
-                IJobResult.OUTCOME_FAIL, IJobResult.OUTCOME_CRASH)
+                IJobResult.OUTCOME_FAIL, IJobResult.OUTCOME_CRASH,
+                IJobResult.OUTCOME_NOT_SUPPORTED)
         rerun_candidates = []
         todo_list = self.assistant.get_static_todo_list()
         job_units = {job_id: self.assistant.get_job(job_id) for job_id