Merge ~ltrager/maas:health_status into maas:master

[Lee Trager]

LP: #1721824, #1721823 - Add health status

The UI now shows the health status of the node when the spinner isn't being
shown. The health status is the overall status of all commissioning and
testing scripts. Like the health status icons for CPU, memory, and storage
the health status icon will only display if there is a problem.

The commissioning, testing, CPU, memory, storage, and overall health are now
reported on the API. Like the websocket ScriptResults are cached so they are
only loaded once.

Screenshot: https://screenshots.firefox.com/2AkiOet6vkZQt31m/10.0.0.2
A tooltip is also set but isn't being shown due to LP: #1718776

[Andres Rodriguez (andreserl)]

Let's minimize this branch to *only* include hardware test stuff.

You can put a different branch for commissioning, but thinking about it, we wont really need provided that commissioning will always have to finish successfully for a machine to be marked ready. e.g. there are no instances where a failed commissioning script would allow a machine to transition to ready state.

review: Needs Fixing

[Lee Trager (ltrager)]

I've split including commissioning results in the overall health status into its own review.

https://code.launchpad.net/~ltrager/maas/+git/maas/+merge/332883

[Andres Rodriguez (andreserl)]

Still needs fixing. This branch still includes commissioning status related stuff.

review: Needs Fixing

[Andres Rodriguez (andreserl)]

more comments

[MAAS Lander (maas-lander)]

UNIT TESTS
-b health_status lp:~ltrager/maas into -b master lp:~maas-committers/maas

STATUS: FAILED
LOG: http://maas-ci-jenkins.internal:8080/job/maas/job/branch-tester/551/console
COMMIT: c2a9049d9bedfd56b4f90cad7a81b329ca949eba

review: Needs Fixing

[Lee Trager (ltrager)]

As discussed this branch no longer calculates the overall health status by looking at the testing and commissioning results. Instead the health status is the testing status.

To produce the testing_status, cpu_status, memory_status and storage_status in the API I need to cache and iterate through all ScriptResults on the node. As I'm doing that I capture the overall commissioning status and output everything in the API. This gives the API and UI feature parity with regards to status.

In the websocket I noticed we were not calculating the testing_status or commissioning_status from cache. I modified the code to calculate both from cache the values are still the same.

[Andres Rodriguez (andreserl)]

review: Approve

[MAAS Lander (maas-lander)]

UNIT TESTS
-b health_status lp:~ltrager/maas into -b master lp:~maas-committers/maas

STATUS: FAILED
LOG: http://maas-ci-jenkins.internal:8080/job/maas/job/branch-tester/560/console
COMMIT: 6a8ddb9741edc1813f81a51f7745c32faeac7475

review: Needs Fixing

[MAAS Lander (maas-lander)]

LANDING
-b health_status lp:~ltrager/maas into -b master lp:~maas-committers/maas

STATUS: FAILED BUILD
LOG: http://maas-ci-jenkins.internal:8080/job/maas/job/branch-tester/561/consoleText

1f3b9c2... by Lee Trager on 2017-10-27

Fix lint

diff --git a/src/maasserver/api/machines.py b/src/maasserver/api/machines.py
index d68687a..4218bdc 100644
--- a/src/maasserver/api/machines.py
+++ b/src/maasserver/api/machines.py
@@ -146,6 +146,16 @@ DISPLAYED_MACHINE_FIELDS = (
     'current_commissioning_result_id',
     'current_testing_result_id',
     'current_installation_result_id',
+    'commissioning_status',
+    'commissioning_status_name',
+    'testing_status',
+    'testing_status_name',
+    'cpu_test_status',
+    'cpu_test_status_name',
+    'memory_test_status',
+    'memory_test_status_name',
+    'storage_test_status',
+    'storage_test_status_name',
 )
 
 # Limited set of machine fields exposed on the anonymous API.
diff --git a/src/maasserver/api/nodes.py b/src/maasserver/api/nodes.py
index 8743375..13d904b 100644
--- a/src/maasserver/api/nodes.py
+++ b/src/maasserver/api/nodes.py
@@ -1,4 +1,4 @@
-# Copyright 2012-2016 Canonical Ltd.  This software is licensed under the
+# Copyright 2012-2017 Canonical Ltd.  This software is licensed under the
 # GNU Affero General Public License version 3 (see the file LICENSE).
 
 __all__ = [
@@ -56,6 +56,13 @@ from maasserver.models import (
 )
 from maasserver.models.nodeprobeddetails import get_single_probed_details
 from maasserver.utils.orm import prefetch_queryset
+from metadataserver.enum import (
+    HARDWARE_TYPE,
+    RESULT_TYPE,
+    SCRIPT_STATUS,
+    SCRIPT_STATUS_CHOICES,
+)
+from metadataserver.models.scriptset import get_status_from_qs
 from piston3.utils import rc
 from provisioningserver.drivers.power import UNKNOWN_POWER_TYPE
 
@@ -246,6 +253,38 @@ def is_registered(request):
     return interfaces.exists()
 
 
+def get_cached_script_results(node):
+    """Load script results into cache and return the cached list."""
+    if not hasattr(node, '_cached_script_results'):
+        node._cached_script_results = list(node.get_latest_script_results)
+        node._cached_commissioning_script_results = []
+        node._cached_testing_script_results = []
+        for script_result in node._cached_script_results:
+            if (script_result.script_set.result_type ==
+                    RESULT_TYPE.INSTALLATION):
+                # Don't include installation results in the health
+                # status.
+                continue
+            elif script_result.status == SCRIPT_STATUS.ABORTED:
+                # LP: #1724235 - Ignore aborted scripts.
+                continue
+            elif (script_result.script_set.result_type ==
+                    RESULT_TYPE.COMMISSIONING):
+                node._cached_commissioning_script_results.append(script_result)
+            elif (script_result.script_set.result_type ==
+                    RESULT_TYPE.TESTING):
+                node._cached_testing_script_results.append(script_result)
+
+    return node._cached_script_results
+
+
+def get_script_status_name(script_status):
+    for id, name in SCRIPT_STATUS_CHOICES:
+        if id == script_status:
+            return name
+    return 'Unknown'
+
+
 class NodeHandler(OperationsHandler):
     """Manage an individual Node.
 
@@ -281,6 +320,60 @@ class NodeHandler(OperationsHandler):
     def current_installation_result_id(handler, node):
         return node.current_installation_script_set_id
 
+    @classmethod
+    def commissioning_status(handler, node):
+        get_cached_script_results(node)
+        return get_status_from_qs(node._cached_commissioning_script_results)
+
+    @classmethod
+    def commissioning_status_name(handler, node):
+        return get_script_status_name(handler.commissioning_status(node))
+
+    @classmethod
+    def testing_status(handler, node):
+        get_cached_script_results(node)
+        return get_status_from_qs(node._cached_testing_script_results)
+
+    @classmethod
+    def testing_status_name(handler, node):
+        return get_script_status_name(handler.testing_status(node))
+
+    @classmethod
+    def cpu_test_status(handler, node):
+        get_cached_script_results(node)
+        return get_status_from_qs([
+            script_result for script_result
+            in node._cached_testing_script_results
+            if script_result.script.hardware_type == HARDWARE_TYPE.CPU])
+
+    @classmethod
+    def cpu_test_status_name(handler, node):
+        return get_script_status_name(handler.cpu_test_status(node))
+
+    @classmethod
+    def memory_test_status(handler, node):
+        get_cached_script_results(node)
+        return get_status_from_qs([
+            script_result for script_result
+            in node._cached_testing_script_results
+            if script_result.script.hardware_type == HARDWARE_TYPE.MEMORY])
+
+    @classmethod
+    def memory_test_status_name(handler, node):
+        return get_script_status_name(handler.memory_test_status(node))
+
+    @classmethod
+    def storage_test_status(handler, node):
+        get_cached_script_results(node)
+        return get_status_from_qs([
+            script_result for script_result
+            in node._cached_testing_script_results
+            if script_result.script.hardware_type == HARDWARE_TYPE.STORAGE])
+
+    @classmethod
+    def storage_test_status_name(handler, node):
+        return get_script_status_name(handler.storage_test_status(node))
+
     def read(self, request, system_id):
         """Read a specific Node.
 
diff --git a/src/maasserver/api/rackcontrollers.py b/src/maasserver/api/rackcontrollers.py
index 5799f3a..d643f04 100644
--- a/src/maasserver/api/rackcontrollers.py
+++ b/src/maasserver/api/rackcontrollers.py
@@ -58,6 +58,16 @@ DISPLAYED_RACK_CONTROLLER_FIELDS = (
     'current_testing_result_id',
     'current_installation_result_id',
     'version',
+    'commissioning_status',
+    'commissioning_status_name',
+    'testing_status',
+    'testing_status_name',
+    'cpu_test_status',
+    'cpu_test_status_name',
+    'memory_test_status',
+    'memory_test_status_name',
+    'storage_test_status',
+    'storage_test_status_name',
 )
 
 
diff --git a/src/maasserver/api/regioncontrollers.py b/src/maasserver/api/regioncontrollers.py
index 9a2d244..69c7b08 100644
--- a/src/maasserver/api/regioncontrollers.py
+++ b/src/maasserver/api/regioncontrollers.py
@@ -41,6 +41,16 @@ DISPLAYED_REGION_CONTROLLER_FIELDS = (
     'current_testing_result_id',
     'current_installation_result_id',
     'version',
+    'commissioning_status',
+    'commissioning_status_name',
+    'testing_status',
+    'testing_status_name',
+    'cpu_test_status',
+    'cpu_test_status_name',
+    'memory_test_status',
+    'memory_test_status_name',
+    'storage_test_status',
+    'storage_test_status_name',
 )
 
 
diff --git a/src/maasserver/api/tests/test_enlistment.py b/src/maasserver/api/tests/test_enlistment.py
index 471eeb7..16bec8d 100644
--- a/src/maasserver/api/tests/test_enlistment.py
+++ b/src/maasserver/api/tests/test_enlistment.py
@@ -559,6 +559,16 @@ class SimpleUserLoggedInEnlistmentAPITest(APITestCase.ForUser):
                 'current_commissioning_result_id',
                 'current_testing_result_id',
                 'current_installation_result_id',
+                'commissioning_status',
+                'commissioning_status_name',
+                'testing_status',
+                'testing_status_name',
+                'cpu_test_status',
+                'cpu_test_status_name',
+                'memory_test_status',
+                'memory_test_status_name',
+                'storage_test_status',
+                'storage_test_status_name',
             ],
             list(parsed_result))
 
@@ -736,6 +746,16 @@ class AdminLoggedInEnlistmentAPITest(APITestCase.ForAdmin):
                 'current_commissioning_result_id',
                 'current_testing_result_id',
                 'current_installation_result_id',
+                'commissioning_status',
+                'commissioning_status_name',
+                'testing_status',
+                'testing_status_name',
+                'cpu_test_status',
+                'cpu_test_status_name',
+                'memory_test_status',
+                'memory_test_status_name',
+                'storage_test_status',
+                'storage_test_status_name',
             ],
             list(parsed_result))
 
diff --git a/src/maasserver/api/tests/test_machines.py b/src/maasserver/api/tests/test_machines.py
index c8199cd..0ebb9e1 100644
--- a/src/maasserver/api/tests/test_machines.py
+++ b/src/maasserver/api/tests/test_machines.py
@@ -342,12 +342,12 @@ class TestMachinesAPI(APITestCase.ForUser):
                 len(extract_system_ids(parsed_result_2)),
             ])
 
-        # Because of fields `status_action`, `status_message`, and
-        # `default_gateways`. The number of queries is not the same but it is
-        # proportional to the number of machines.
+        # Because of fields `status_action`, `status_message`,
+        # `default_gateways`, and `health_status` the number of queries is not
+        # the same but it is proportional to the number of machines.
         DEFAULT_NUM = 57
-        self.assertEqual(DEFAULT_NUM + (10 * 3), num_queries1)
-        self.assertEqual(DEFAULT_NUM + (20 * 3), num_queries2)
+        self.assertEqual(DEFAULT_NUM + (10 * 4), num_queries1)
+        self.assertEqual(DEFAULT_NUM + (20 * 4), num_queries2)
 
     def test_GET_without_machines_returns_empty_list(self):
         # If there are no machines to list, the "read" op still works but
diff --git a/src/maasserver/api/tests/test_node.py b/src/maasserver/api/tests/test_node.py
index 706a3a0..4fecb15 100644
--- a/src/maasserver/api/tests/test_node.py
+++ b/src/maasserver/api/tests/test_node.py
@@ -1,10 +1,11 @@
-# Copyright 2013-2016 Canonical Ltd.  This software is licensed under the
+# Copyright 2013-2017 Canonical Ltd.  This software is licensed under the
 # GNU Affero General Public License version 3 (see the file LICENSE).
 
 """Tests for the Node API."""
 
 __all__ = []
 
+from functools import partial
 import http.client
 import random
 from unittest.mock import (
@@ -40,11 +41,14 @@ from maastesting.matchers import (
     MockNotCalled,
 )
 from metadataserver.enum import (
+    HARDWARE_TYPE,
     RESULT_TYPE,
     SCRIPT_STATUS,
+    SCRIPT_STATUS_CHOICES,
     SCRIPT_TYPE,
 )
 from metadataserver.models import NodeKey
+from metadataserver.models.scriptset import get_status_from_qs
 from metadataserver.nodeinituser import get_node_init_user
 from provisioningserver.refresh.node_info_scripts import (
     LLDP_OUTPUT_NAME,
@@ -184,6 +188,70 @@ class TestNodeAPI(APITestCase.ForUser):
                 args=[parsed_result['system_id']]),
             parsed_result['resource_uri'])
 
+    def test_health_status(self):
+        self.become_admin()
+        machine = factory.make_Machine(owner=self.user)
+        commissioning_script_set = factory.make_ScriptSet(
+            result_type=RESULT_TYPE.COMMISSIONING, node=machine)
+        testing_script_set = factory.make_ScriptSet(
+            result_type=RESULT_TYPE.TESTING, node=machine)
+        make_script_result = partial(
+            factory.make_ScriptResult, script_set=testing_script_set,
+            status=factory.pick_choice(
+                SCRIPT_STATUS_CHOICES, but_not=[SCRIPT_STATUS.ABORTED]))
+        commissioning_script_result = make_script_result(
+            script_set=commissioning_script_set, script=factory.make_Script(
+                script_type=SCRIPT_TYPE.COMMISSIONING))
+        cpu_script_result = make_script_result(
+            script=factory.make_Script(
+                script_type=SCRIPT_TYPE.TESTING,
+                hardware_type=HARDWARE_TYPE.CPU))
+        memory_script_result = make_script_result(
+            script=factory.make_Script(
+                script_type=SCRIPT_TYPE.TESTING,
+                hardware_type=HARDWARE_TYPE.MEMORY))
+        storage_script_result = make_script_result(
+            script=factory.make_Script(
+                script_type=SCRIPT_TYPE.TESTING,
+                hardware_type=HARDWARE_TYPE.STORAGE))
+        testing_script_results = (
+            machine.get_latest_testing_script_results.exclude(
+                status=SCRIPT_STATUS.ABORTED))
+        testing_status = get_status_from_qs(testing_script_results)
+
+        response = self.client.get(self.get_node_uri(machine))
+        parsed_result = json_load_bytes(response.content)
+
+        status = lambda s: get_status_from_qs([s])
+        status_name = lambda s: SCRIPT_STATUS_CHOICES[status(s)][1]
+        self.assertThat(response, HasStatusCode(http.client.OK))
+        self.assertEquals(
+            status(commissioning_script_result),
+            parsed_result['commissioning_status'])
+        self.assertEquals(
+            status_name(commissioning_script_result),
+            parsed_result['commissioning_status_name'])
+        self.assertEquals(testing_status, parsed_result['testing_status'])
+        self.assertEquals(
+            SCRIPT_STATUS_CHOICES[testing_status][1],
+            parsed_result['testing_status_name'])
+        self.assertEquals(
+            status(cpu_script_result), parsed_result['cpu_test_status'])
+        self.assertEquals(
+            status_name(cpu_script_result),
+            parsed_result['cpu_test_status_name'])
+        self.assertEquals(
+            status(memory_script_result), parsed_result['memory_test_status'])
+        self.assertEquals(
+            status_name(memory_script_result),
+            parsed_result['memory_test_status_name'])
+        self.assertEquals(
+            status(storage_script_result),
+            parsed_result['storage_test_status'])
+        self.assertEquals(
+            status_name(storage_script_result),
+            parsed_result['storage_test_status_name'])
+
     def test_DELETE_deletes_node(self):
         # The api allows to delete a Node.
         self.become_admin()
diff --git a/src/maasserver/api/tests/test_rackcontroller.py b/src/maasserver/api/tests/test_rackcontroller.py
index a631d6c..5105c6d 100644
--- a/src/maasserver/api/tests/test_rackcontroller.py
+++ b/src/maasserver/api/tests/test_rackcontroller.py
@@ -135,6 +135,16 @@ class TestRackControllersAPI(APITestCase.ForUser):
                 'current_testing_result_id',
                 'current_installation_result_id',
                 'version',
+                'commissioning_status',
+                'commissioning_status_name',
+                'testing_status',
+                'testing_status_name',
+                'cpu_test_status',
+                'cpu_test_status_name',
+                'memory_test_status',
+                'memory_test_status_name',
+                'storage_test_status',
+                'storage_test_status_name',
             ],
             list(parsed_result[0]))
 
diff --git a/src/maasserver/api/tests/test_regioncontroller.py b/src/maasserver/api/tests/test_regioncontroller.py
index fd02644..3854e6f 100644
--- a/src/maasserver/api/tests/test_regioncontroller.py
+++ b/src/maasserver/api/tests/test_regioncontroller.py
@@ -83,5 +83,15 @@ class TestRegionControllersAPI(APITestCase.ForUser):
                 'current_testing_result_id',
                 'current_installation_result_id',
                 'version',
+                'commissioning_status',
+                'commissioning_status_name',
+                'testing_status',
+                'testing_status_name',
+                'cpu_test_status',
+                'cpu_test_status_name',
+                'memory_test_status',
+                'memory_test_status_name',
+                'storage_test_status',
+                'storage_test_status_name',
             ],
             list(parsed_result[0]))
diff --git a/src/maasserver/api/tests/test_tag.py b/src/maasserver/api/tests/test_tag.py
index ef0d546..4397dd3 100644
--- a/src/maasserver/api/tests/test_tag.py
+++ b/src/maasserver/api/tests/test_tag.py
@@ -197,10 +197,10 @@ class TestTagAPI(APITestCase.ForUser):
                 len(extract_system_ids(parsed_result_2)),
             ])
 
-        # Because of fields `status_action`, `status_message`, and
-        # `default_gateways`. The number of queries is not the same but it is
-        # proportional to the number of machines.
-        self.assertEquals(num_queries1, num_queries2 - (3 * 3))
+        # Because of fields `status_action`, `status_message`,
+        # `default_gateways`, and `health_status` the number of queries is not
+        # the same but it is proportional to the number of machines.
+        self.assertEquals(num_queries1, num_queries2 - (3 * 4))
 
     def test_GET_machines_returns_machines(self):
         tag = factory.make_Tag()
@@ -257,10 +257,10 @@ class TestTagAPI(APITestCase.ForUser):
                 len(extract_system_ids(parsed_result_2)),
             ])
 
-        # Because of fields `status_action`, `status_message`, and
-        # `default_gateways`. The number of queries is not the same but it is
-        # proportional to the number of machines.
-        self.assertEquals(num_queries1, num_queries2 - (3 * 3))
+        # Because of fields `status_action`, `status_message`,
+        # `default_gateways`, and `health_status` the number of queries is not
+        # the same but it is proportional to the number of machines.
+        self.assertEquals(num_queries1, num_queries2 - (3 * 4))
 
     def test_GET_devices_returns_devices(self):
         tag = factory.make_Tag()
@@ -375,7 +375,7 @@ class TestTagAPI(APITestCase.ForUser):
                 len(extract_system_ids(parsed_result_1)),
                 len(extract_system_ids(parsed_result_2)),
             ])
-        self.assertEquals(num_queries1, num_queries2 - (3 * 2))
+        self.assertEquals(num_queries1, num_queries2 - (3 * 3))
 
     def test_GET_rack_controllers_returns_no_rack_controllers_nonadmin(self):
         tag = factory.make_Tag()
@@ -456,7 +456,7 @@ class TestTagAPI(APITestCase.ForUser):
                 len(extract_system_ids(parsed_result_1)),
                 len(extract_system_ids(parsed_result_2)),
             ])
-        self.assertEquals(num_queries1, num_queries2 - 3)
+        self.assertEquals(num_queries1, num_queries2 - 6)
 
     def test_GET_region_controllers_returns_no_controllers_nonadmin(self):
         tag = factory.make_Tag()
diff --git a/src/maasserver/static/js/angular/controllers/node_details.js b/src/maasserver/static/js/angular/controllers/node_details.js
index 503d30a..ac96431 100644
--- a/src/maasserver/static/js/angular/controllers/node_details.js
+++ b/src/maasserver/static/js/angular/controllers/node_details.js
@@ -1000,7 +1000,7 @@ angular.module('MAAS').controller('NodeDetailsController', [
             var results = $scope.node.installation_results;
             if(!angular.isArray(results) ||
                     results.length === 0 || results[0].output === "") {
-                switch($scope.node.installation_script_set_status) {
+                switch($scope.node.installation_status) {
                     case 0:
                         return "System is booting...";
                     case 1:
diff --git a/src/maasserver/static/js/angular/controllers/tests/test_node_details.js b/src/maasserver/static/js/angular/controllers/tests/test_node_details.js
index ceeaef8..f670338 100644
--- a/src/maasserver/static/js/angular/controllers/tests/test_node_details.js
+++ b/src/maasserver/static/js/angular/controllers/tests/test_node_details.js
@@ -2197,7 +2197,7 @@ describe("NodeDetailsController", function() {
         it("returns status message when no output and status", function() {
             var controller = makeController();
             $scope.node = makeNode();
-            $scope.node.installation_script_set_status = makeInteger(0, 5);
+            $scope.node.installation_status = makeInteger(0, 5);
             expect($scope.getInstallationData()).not.toBe("");
         });
     });
diff --git a/src/maasserver/static/partials/machines-table.html b/src/maasserver/static/partials/machines-table.html
index e47b10d..24fb55a 100644
--- a/src/maasserver/static/partials/machines-table.html
+++ b/src/maasserver/static/partials/machines-table.html
@@ -57,8 +57,14 @@
             <td class="powerstate u-upper-case--first" aria-label="Power state">
                 <span title="{$ node.power_state $}" data-ng-if="node.power_state != 'unknown'" class="icon icon--power-{$ node.power_state $} u-margin--right-tiny"></span> {$ node.power_state $}
             </td>
-            <td class="table-col--3 u-display--desktop"><i data-ng-if="showSpinner(node)" class="icon icon--loading u-animation--spin u-display--desktop"></i></td>
-            <td class="status u-text--truncate" aria-label="{$ node.status_tooltip $}">{$ getStatusText(node) $} <span class="u-display--mobile"><i data-ng-if="showSpinner(node)" class="icon icon--loading u-animation--spin u-margin--left-tiny"></i></span></td>
+            <td class="table-col--3 u-display--desktop">
+                <i data-ng-if="showSpinner(node)" class="icon icon--loading u-animation--spin u-display--desktop"></i>
+                <i data-ng-if="!showSpinner(node) && node.testing_status !== 2" data-maas-script-status="script-status" data-script-status="node.testing_status" aria-label="{$ node.testing_status_tooltip $}"></i>
+	    </td>
+            <td class="status u-text--truncate" aria-label="{$ node.status_tooltip $}">{$ getStatusText(node) $} <span class="u-display--mobile">
+		<i data-ng-if="showSpinner(node)" class="icon icon--loading u-animation--spin u-margin--left-tiny"></i>
+		<i data-ng-if="!showSpinner(node) && node.testing_status !== 2" data-maas-script-status="script-status" data-script-status="node.testing_status" aria-label="{$ node.testing_status_tooltip $}"></i>
+	    </span></td>
             <td class="table-col--10" aria-label="Owner">{$ node.owner $}</td>
             <td class="u-align--right u-display--desktop" aria-label="CPU">
                 <span class="tooltip" data-maas-script-status="script-status" data-script-status="node.cpu_test_status" aria-label="{$ node.cpu_test_status_tooltip $}" data-ng-if="node.cpu_test_status !== 2"></span>
diff --git a/src/maasserver/static/partials/node-details.html b/src/maasserver/static/partials/node-details.html
index f8d8087..324f3d1 100755
--- a/src/maasserver/static/partials/node-details.html
+++ b/src/maasserver/static/partials/node-details.html
@@ -206,13 +206,13 @@
                     data-ng-click="section.area = 'storage'">Storage</button>
                 <button role="tab" class="page-navigation__link" data-ng-if="node.commissioning_script_count > 0"
                     data-ng-class="{ 'is-active': section.area === 'commissioning'}"
-                    data-ng-click="section.area = 'commissioning'"><span data-maas-script-status="script-status" data-script-status="node.commissioning_script_set_status"></span> Commissioning</button>
+                    data-ng-click="section.area = 'commissioning'"><span data-maas-script-status="script-status" data-script-status="node.commissioning_status"></span> Commissioning</button>
                 <button role="tab" class="page-navigation__link" data-ng-if="node.testing_script_count > 0"
                     data-ng-class="{ 'is-active': section.area === 'testing'}"
-                    data-ng-click="section.area = 'testing'"><span data-maas-script-status="script-status" data-script-status="node.testing_script_set_status"></span> Hardware tests</button>
+                    data-ng-click="section.area = 'testing'"><span data-maas-script-status="script-status" data-script-status="node.testing_status"></span> Hardware tests</button>
                 <button role="tab" class="page-navigation__link" data-ng-if="machine_output.viewable"
                     data-ng-class="{ 'is-active': section.area === 'logs'}"
-                    data-ng-click="section.area = 'logs'"><span data-maas-script-status="script-status" data-script-status="node.installation_script_set_status" data-ng-if="node.installation_results.length > 0"></span> Logs</button>
+                    data-ng-click="section.area = 'logs'"><span data-maas-script-status="script-status" data-script-status="node.installation_status" data-ng-if="node.installation_results.length > 0"></span> Logs</button>
                 <button role="tab" class="page-navigation__link" data-ng-if="!isDevice"
                     data-ng-class="{ 'is-active': section.area === 'events'}"
                     data-ng-click="section.area = 'events'">Events</button>
diff --git a/src/maasserver/websockets/handlers/machine.py b/src/maasserver/websockets/handlers/machine.py
index ac101fc..da51686 100644
--- a/src/maasserver/websockets/handlers/machine.py
+++ b/src/maasserver/websockets/handlers/machine.py
@@ -79,11 +79,7 @@ from maasserver.websockets.handlers.node import (
     node_prefetch,
     NodeHandler,
 )
-from metadataserver.enum import (
-    HARDWARE_TYPE,
-    SCRIPT_STATUS,
-    SCRIPT_STATUS_CHOICES,
-)
+from metadataserver.enum import HARDWARE_TYPE
 from metadataserver.models import ScriptResult
 from metadataserver.models.scriptset import get_status_from_qs
 from provisioningserver.logger import LegacyLogger
@@ -179,44 +175,6 @@ class MachineHandler(NodeHandler):
         return Machine.objects.get_nodes(
             self.user, NODE_PERMISSION.VIEW, from_nodes=self._meta.queryset)
 
-    def dehydrate_hardware_status_tooltip(self, script_results):
-        script_statuses = {}
-        for script_result in script_results:
-            if script_result.status in script_statuses:
-                script_statuses[script_result.status].add(script_result.name)
-            else:
-                script_statuses[script_result.status] = {script_result.name}
-
-        tooltip = ''
-        for status, scripts in script_statuses.items():
-            len_scripts = len(scripts)
-            if status in {
-                    SCRIPT_STATUS.PENDING, SCRIPT_STATUS.RUNNING,
-                    SCRIPT_STATUS.INSTALLING}:
-                verb = 'is' if len_scripts == 1 else 'are'
-            elif status in {
-                    SCRIPT_STATUS.PASSED, SCRIPT_STATUS.FAILED,
-                    SCRIPT_STATUS.TIMEDOUT, SCRIPT_STATUS.FAILED_INSTALLING}:
-                verb = 'has' if len_scripts == 1 else 'have'
-            else:
-                # Covers SCRIPT_STATUS.ABORTED, an else is used incase new
-                # statuses are ever added.
-                verb = 'was' if len_scripts == 1 else 'were'
-
-            if tooltip != '':
-                tooltip += ' '
-            if len_scripts == 1:
-                tooltip += '1 test '
-            else:
-                tooltip += '%s tests ' % len_scripts
-            tooltip += '%s %s.' % (
-                verb, SCRIPT_STATUS_CHOICES[status][1].lower())
-
-        if tooltip == '':
-            tooltip = 'No tests have been run.'
-
-        return tooltip
-
     def list(self, params):
         """List objects.
 
diff --git a/src/maasserver/websockets/handlers/node.py b/src/maasserver/websockets/handlers/node.py
index 8266b2a..81910f0 100644
--- a/src/maasserver/websockets/handlers/node.py
+++ b/src/maasserver/websockets/handlers/node.py
@@ -45,8 +45,10 @@ from maasserver.websockets.handlers.timestampedmodel import (
     TimestampedModelHandler,
 )
 from metadataserver.enum import (
+    HARDWARE_TYPE,
     RESULT_TYPE,
     SCRIPT_STATUS,
+    SCRIPT_STATUS_CHOICES,
 )
 from metadataserver.models.scriptset import get_status_from_qs
 from provisioningserver.tags import merge_details_cleanly
@@ -118,6 +120,44 @@ class NodeHandler(TimestampedModelHandler):
         """Return power_parameters None if empty."""
         return None if power_parameters == '' else power_parameters
 
+    def dehydrate_hardware_status_tooltip(self, script_results):
+        script_statuses = {}
+        for script_result in script_results:
+            if script_result.status in script_statuses:
+                script_statuses[script_result.status].add(script_result.name)
+            else:
+                script_statuses[script_result.status] = {script_result.name}
+
+        tooltip = ''
+        for status, scripts in script_statuses.items():
+            len_scripts = len(scripts)
+            if status in {
+                    SCRIPT_STATUS.PENDING, SCRIPT_STATUS.RUNNING,
+                    SCRIPT_STATUS.INSTALLING}:
+                verb = 'is' if len_scripts == 1 else 'are'
+            elif status in {
+                    SCRIPT_STATUS.PASSED, SCRIPT_STATUS.FAILED,
+                    SCRIPT_STATUS.TIMEDOUT, SCRIPT_STATUS.FAILED_INSTALLING}:
+                verb = 'has' if len_scripts == 1 else 'have'
+            else:
+                # Covers SCRIPT_STATUS.ABORTED, an else is used incase new
+                # statuses are ever added.
+                verb = 'was' if len_scripts == 1 else 'were'
+
+            if tooltip != '':
+                tooltip += ' '
+            if len_scripts == 1:
+                tooltip += '1 test '
+            else:
+                tooltip += '%s tests ' % len_scripts
+            tooltip += '%s %s.' % (
+                verb, SCRIPT_STATUS_CHOICES[status][1].lower())
+
+        if tooltip == '':
+            tooltip = 'No tests have been run.'
+
+        return tooltip
+
     def dehydrate(self, obj, data, for_list=False):
         """Add extra fields to `data`."""
         data["fqdn"] = obj.fqdn
@@ -174,6 +214,40 @@ class NodeHandler(TimestampedModelHandler):
             data["distro_series"] = obj.get_distro_series(
                 default=self.default_distro_series)
             data["dhcp_on"] = self.get_providing_dhcp(obj)
+
+        if obj.node_type != NODE_TYPE.DEVICE:
+            commissioning_script_results = []
+            testing_script_results = []
+            for hw_type in self._script_results.get(obj.id, {}).values():
+                for script_result in hw_type:
+                    if (script_result.script_set.result_type ==
+                            RESULT_TYPE.INSTALLATION):
+                        # Don't include installation results in the health
+                        # status.
+                        continue
+                    elif script_result.status == SCRIPT_STATUS.ABORTED:
+                        # LP: #1724235 - Ignore aborted scripts.
+                        continue
+                    elif (script_result.script_set.result_type ==
+                            RESULT_TYPE.COMMISSIONING):
+                        commissioning_script_results.append(script_result)
+                    elif (script_result.script_set.result_type ==
+                            RESULT_TYPE.TESTING):
+                        testing_script_results.append(script_result)
+
+            data["commissioning_script_count"] = len(
+                commissioning_script_results)
+            data["commissioning_status"] = get_status_from_qs(
+                commissioning_script_results)
+            data["commissioning_status_tooltip"] = (
+                self.dehydrate_hardware_status_tooltip(
+                    commissioning_script_results).replace(
+                        'test', 'commissioning script'))
+            data["testing_script_count"] = len(testing_script_results)
+            data["testing_status"] = get_status_from_qs(testing_script_results)
+            data["testing_status_tooltip"] = (
+                self.dehydrate_hardware_status_tooltip(testing_script_results))
+
         if not for_list:
             data["on_network"] = obj.on_network()
             if obj.node_type != NODE_TYPE.DEVICE:
@@ -216,23 +290,12 @@ class NodeHandler(TimestampedModelHandler):
                 # Events
                 data["events"] = self.dehydrate_events(obj)
 
-                # Machine output
+                # Machine logs
                 data = self.dehydrate_summary_output(obj, data)
-                data["commissioning_script_count"] = (
-                    obj.get_latest_commissioning_script_results.count())
-                data["commissioning_script_set_status"] = get_status_from_qs(
-                    obj.get_latest_commissioning_script_results.exclude(
-                        status=SCRIPT_STATUS.ABORTED))
-                data["testing_script_count"] = (
-                    obj.get_latest_testing_script_results.count())
-                data["testing_script_set_status"] = get_status_from_qs(
-                    obj.get_latest_testing_script_results.exclude(
-                        status=SCRIPT_STATUS.ABORTED))
+                data["installation_status"] = self.dehydrate_script_set_status(
+                    obj.current_installation_script_set)
                 data["installation_results"] = self.dehydrate_script_set(
                     obj.current_installation_script_set)
-                data["installation_script_set_status"] = (
-                    self.dehydrate_script_set_status(
-                        obj.current_installation_script_set))
 
                 # Third party drivers
                 if Config.objects.get_config('enable_third_party_drivers'):
@@ -255,11 +318,11 @@ class NodeHandler(TimestampedModelHandler):
         qs = qs.exclude(status=SCRIPT_STATUS.ABORTED)
         cleared_node_ids = []
         for script_result in qs:
-            # Builtin commissioning scripts are not stored in the database.
-            if script_result.script is None:
-                continue
             node_id = script_result.script_set.node_id
-            hardware_type = script_result.script.hardware_type
+            if script_result.script is not None:
+                hardware_type = script_result.script.hardware_type
+            else:
+                hardware_type = HARDWARE_TYPE.NODE
 
             if node_id not in cleared_node_ids:
                 self._script_results[node_id] = {}
diff --git a/src/maasserver/websockets/handlers/tests/test_device.py b/src/maasserver/websockets/handlers/tests/test_device.py
index b6a879b..b0b4490 100644
--- a/src/maasserver/websockets/handlers/tests/test_device.py
+++ b/src/maasserver/websockets/handlers/tests/test_device.py
@@ -186,18 +186,19 @@ class TestDeviceHandler(MAASTransactionServerTestCase):
         if for_list:
             allowed_fields = DeviceHandler.Meta.list_fields + [
                 "actions",
-                "fqdn",
                 "extra_macs",
-                "metadata",
-                "tags",
-                "primary_mac",
+                "fabrics",
+                "fqdn",
+                "installation_status",
                 "ip_address",
                 "ip_assignment",
-                "node_type_display",
                 "link_type",
-                "subnets",
+                "metadata",
+                "node_type_display",
+                "primary_mac",
                 "spaces",
-                "fabrics",
+                "subnets",
+                "tags",
                 ]
             for key in list(data):
                 if key not in allowed_fields:
diff --git a/src/maasserver/websockets/handlers/tests/test_machine.py b/src/maasserver/websockets/handlers/tests/test_machine.py
index 0dd01f2..15ab758 100644
--- a/src/maasserver/websockets/handlers/tests/test_machine.py
+++ b/src/maasserver/websockets/handlers/tests/test_machine.py
@@ -166,30 +166,36 @@ class TestMachineHandler(MAASServerTestCase):
         ]
         disks = sorted(disks, key=itemgetter("name"))
         subnets = handler.get_all_subnets(node)
+        commissioning_scripts = node.get_latest_commissioning_script_results
+        commissioning_scripts = commissioning_scripts.exclude(
+            status=SCRIPT_STATUS.ABORTED)
+        testing_scripts = node.get_latest_testing_script_results
+        testing_scripts = testing_scripts.exclude(
+            status=SCRIPT_STATUS.ABORTED)
         data = {
             "actions": list(compile_node_actions(node, handler.user).keys()),
             "architecture": node.architecture,
             "bmc": node.bmc_id,
             "boot_disk": node.boot_disk,
             "bios_boot_method": node.bios_boot_method,
-            "commissioning_script_count": (
-                node.get_latest_commissioning_script_results.count()),
-            "commissioning_script_set_status": get_status_from_qs(
-                node.get_latest_commissioning_script_results.exclude(
-                    status=SCRIPT_STATUS.ABORTED)),
+            "commissioning_script_count": commissioning_scripts.count(),
+            "commissioning_status": get_status_from_qs(commissioning_scripts),
+            "commissioning_status_tooltip": (
+                handler.dehydrate_hardware_status_tooltip(
+                    commissioning_scripts).replace(
+                        'test', 'commissioning script')),
             "current_commissioning_script_set": (
                 node.current_commissioning_script_set_id),
-            "testing_script_count": (
-                node.get_latest_testing_script_results.count()),
-            "testing_script_set_status": get_status_from_qs(
-                node.get_latest_testing_script_results.exclude(
-                    status=SCRIPT_STATUS.ABORTED)),
+            "testing_script_count": testing_scripts.count(),
+            "testing_status": get_status_from_qs(testing_scripts),
+            "testing_status_tooltip": (
+                handler.dehydrate_hardware_status_tooltip(testing_scripts)),
             "current_testing_script_set": node.current_testing_script_set_id,
             "installation_results": handler.dehydrate_script_set(
                 node.current_installation_script_set),
             "current_installation_script_set": (
                 node.current_installation_script_set_id),
-            "installation_script_set_status": (
+            "installation_status": (
                 handler.dehydrate_script_set_status(
                     node.current_installation_script_set)),
             "cpu_count": node.cpu_count,
@@ -286,26 +292,32 @@ class TestMachineHandler(MAASServerTestCase):
             allowed_fields = MachineHandler.Meta.list_fields + [
                 "actions",
                 "architecture",
+                "commissioning_script_count",
+                "commissioning_status",
+                "commissioning_status_tooltip",
+                "dhcp_on",
+                "distro_series",
+                "extra_macs",
+                "fabrics",
                 "fqdn",
+                "link_type",
                 "metadata",
-                "status",
-                "status_code",
+                "node_type_display",
+                "osystem",
+                "physical_disk_count",
+                "pod",
                 "pxe_mac",
                 "pxe_mac_vendor",
-                "extra_macs",
-                "tags",
-                "subnets",
-                "fabrics",
                 "spaces",
-                "physical_disk_count",
+                "status",
+                "status_code",
                 "storage",
                 "storage_tags",
-                "node_type_display",
-                "osystem",
-                "distro_series",
-                "dhcp_on",
-                "pod",
-                "link_type",
+                "subnets",
+                "tags",
+                "testing_script_count",
+                "testing_status",
+                "testing_status_tooltip",
             ]
             for key in list(data):
                 if key not in allowed_fields: