Merge lp:~newell-jensen/maas/fix-1567177 into lp:maas/trunk

[Newell Jensen]

Fixes miss catching of Node's power_query PowerProblem that is raised when there is "No BMC defined.". Also, fixes it so that Rack Controllers are no longer power queried.

[Gavin Panella (allenap)]

> fixes it so that Rack Controllers are no longer power queried.

Why? Seems like this is quite useful information, if there's BMC information available.

Some other questions too, but otherwise this looks good.

review: Needs Information

[Newell Jensen (newell-jensen)]

> > fixes it so that Rack Controllers are no longer power queried.
>
> Why? Seems like this is quite useful information, if there's BMC information
> available.
>
> Some other questions too, but otherwise this looks good.

This is ready for another look. This has changed to allow Rack Controllers to be power queried but not started or stopped (no power on/off).

[Gavin Panella (allenap)]

Looks good!

review: Approve

[MAAS Lander (maas-lander)]

The attempt to merge lp:~newell-jensen/maas/fix-1567177 into lp:maas failed. Below is the output from the failed tests.

Hit:1 http://security.ubuntu.com/ubuntu xenial-security InRelease
Get:2 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial InRelease [247 kB]
Hit:3 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial-updates InRelease
Hit:4 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial-backports InRelease
Get:5 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial/main amd64 Packages [1,197 kB]
Fetched 1,444 kB in 0s (2,314 kB/s)
Reading package lists...
sudo DEBIAN_FRONTEND=noninteractive apt-get -y \
    --no-install-recommends install apache2 archdetect-deb authbind bash bind9 bind9utils build-essential bzr-builddeb chromium-browser chromium-chromedriver curl daemontools debhelper dh-apport dh-systemd distro-info dnsutils firefox freeipmi-tools git gjs ipython isc-dhcp-common libjs-angularjs libjs-jquery libjs-jquery-hotkeys libjs-yui3-full libjs-yui3-min libpq-dev make nodejs-legacy npm postgresql pxelinux python3-all python3-apt python3-bson python3-convoy python3-coverage python3-crochet python3-cssselect python3-curtin python3-dev python3-distro-info python3-django python3-django-nose python3-django-piston3 python3-dnspython python3-docutils python3-formencode python3-hivex python3-httplib2 python3-jinja2 python3-jsonschema python3-lxml python3-mock python3-netaddr python3-netifaces python3-oauth python3-oauthlib python3-openssl python3-paramiko python3-petname python3-pexpect python3-psycopg2 python3-pyinotify python3-pyparsing python3-pyvmomi python3-requests python3-seamicroclient python3-setuptools python3-simplestreams python3-sphinx python3-tempita python3-twisted python3-txtftp python3-tz python3-yaml python3-zope.interface python-bson python-crochet python-django python-django-piston python-djorm-ext-pgarray python-formencode python-lxml python-netaddr python-netifaces python-pocket-lint python-psycopg2 python-tempita python-twisted python-yaml socat syslinux-common tgt ubuntu-cloudimage-keyring wget xvfb
Reading package lists...
Building dependency tree...
Reading state information...
apache2 is already the newest version (2.4.18-2ubuntu2).
archdetect-deb is already the newest version (1.117ubuntu1).
authbind is already the newest version (2.1.1+nmu1).
bash is already the newest version (4.3-14ubuntu1).
bind9 is already the newest version (1:9.10.3.dfsg.P4-5).
bind9utils is already the newest version (1:9.10.3.dfsg.P4-5).
build-essential is already the newest version (12.1ubuntu2).
curl is already the newest version (7.47.0-1ubuntu2).
debhelper is already the newest version (9.20160115ubuntu3).
distro-info is already the newest version (0.14build1).
dnsutils is already the newest version (1:9.10.3.dfsg.P4-5).
firefox is already the newest version (45.0.1+build1-0ubuntu1).
freeipmi-tools is already the newest version (1.4.11-1ubuntu1).
git is already the newest version (1:2.7.4-0ubuntu1).
isc-dhcp-common is already the newest version (4.3.3-5ubuntu11).
libjs-angularjs is already the newest version (1.2.28-1ubuntu2).
libjs-jquery is already the newest version (1.11.3+dfsg-4)...

[Gavin Panella (allenap)]

The failure is below. I saw this earlier today when trying to land a branch. I'll look into why this is happening tomorrow morning (for me). In the meantime give this another go at landing; the failure is not related to this branch.

FAIL: maasserver.rpc.tests.test_regionservice.TestRegionService.test_start_up_logs_failure_if_all_endpoint_options_fail
----------------------------------------------------------------------
testtools.testresult.real._StringException: Traceback (most recent call last):
  File "/tmp/tarmac/branch.TZVTGQ/src/maastesting/runtest.py", line 59, in _run_user
    result = function(*args, **kwargs)
  File "/home/ubuntu/.buildout/eggs/testtools-1.8.1-py3.5.egg/testtools/testcase.py", line 654, in _run_test_method
    return self._get_test_method()()
  File "/usr/lib/python3/dist-packages/crochet/_eventloop.py", line 461, in wrapper
    return eventual_result.wait(timeout)
  File "/usr/lib/python3/dist-packages/crochet/_eventloop.py", line 231, in wait
    result.raiseException()
  File "/usr/lib/python3/dist-packages/twisted/python/failure.py", line 368, in raiseException
    raise self.value.with_traceback(self.tb)
testtools.matchers._impl.MismatchError: Expected:
                RegionServer endpoint failed to listen.
                Traceback (most recent call last):
                ...
                maastesting.factory.TestException#35:

Got:
    Unhandled error in Deferred:
    ---
    Unhandled Error
    Traceback (most recent call last):
    Failure: twisted.internet.defer.CancelledError:
    <BLANKLINE>
    ---
    Unhandled error in Deferred:
    ---
    Unhandled Error
    Traceback (most recent call last):
      File "/tmp/tarmac/branch.TZVTGQ/src/maasserver/rpc/regionservice.py", line 753, in startService
        for endpoint_options in self.endpoints))
      File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 824, in __init__
        self._deferredList = list(deferredList)
      File "/tmp/tarmac/branch.TZVTGQ/src/maasserver/rpc/regionservice.py", line 753, in <genexpr>
        for endpoint_options in self.endpoints))
      File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1274, in unwindGenerator
        return _inlineCallbacks(None, gen, Deferred())
    --- <exception caught here> ---
      File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1126, in _inlineCallbacks
        result = result.throwExceptionIntoGenerator(g)
      File "/usr/lib/python3/dist-packages/twisted/python/failure.py", line 389, in throwExceptionIntoGenerator
        return g.throw(self.type, self.value, self.tb)
      File "/tmp/tarmac/branch.TZVTGQ/src/maasserver/rpc/regionservice.py", line 740, in _bindFirst
        port = yield endpoint.listen(factory)
    builtins.ValueError: This is not the messiah.
    <BLANKLINE>
    ---
    RegionServer endpoint failed to listen.
    Traceback (most recent call last):
      File "/tmp/tarmac/branch.TZVTGQ/src/maasserver/rpc/regionservice.py", line 753, in startService
        for endpoint_options in self.endpoints))
      File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 824, in __init__
        self._deferredList = ...

[MAAS Lander (maas-lander)]

The attempt to merge lp:~newell-jensen/maas/fix-1567177 into lp:maas failed. Below is the output from the failed tests.

Hit:1 http://security.ubuntu.com/ubuntu xenial-security InRelease
Get:2 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial InRelease [247 kB]
Hit:3 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial-updates InRelease
Hit:4 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial-backports InRelease
Get:5 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial/main Sources [875 kB]
Get:6 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial/universe Sources [7,751 kB]
Get:7 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial/multiverse Sources [182 kB]
Get:8 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial/main amd64 Packages [1,197 kB]
Get:9 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial/main Translation-en [567 kB]
Get:10 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial/universe amd64 Packages [7,530 kB]
Get:11 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial/universe Translation-en [4,358 kB]
Get:12 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial/multiverse amd64 Packages [146 kB]
Fetched 22.9 MB in 3s (6,300 kB/s)
Reading package lists...
sudo DEBIAN_FRONTEND=noninteractive apt-get -y \
    --no-install-recommends install apache2 archdetect-deb authbind bash bind9 bind9utils build-essential bzr-builddeb chromium-browser chromium-chromedriver curl daemontools debhelper dh-apport dh-systemd distro-info dnsutils firefox freeipmi-tools git gjs ipython isc-dhcp-common libjs-angularjs libjs-jquery libjs-jquery-hotkeys libjs-yui3-full libjs-yui3-min libpq-dev make nodejs-legacy npm postgresql pxelinux python3-all python3-apt python3-bson python3-convoy python3-coverage python3-crochet python3-cssselect python3-curtin python3-dev python3-distro-info python3-django python3-django-nose python3-django-piston3 python3-dnspython python3-docutils python3-formencode python3-hivex python3-httplib2 python3-jinja2 python3-jsonschema python3-lxml python3-mock python3-netaddr python3-netifaces python3-oauth python3-oauthlib python3-openssl python3-paramiko python3-petname python3-pexpect python3-psycopg2 python3-pyinotify python3-pyparsing python3-pyvmomi python3-requests python3-seamicroclient python3-setuptools python3-simplestreams python3-sphinx python3-tempita python3-twisted python3-txtftp python3-tz python3-yaml python3-zope.interface python-bson python-crochet python-django python-django-piston python-djorm-ext-pgarray python-formencode python-lxml python-netaddr python-netifaces python-pocket-lint python-psycopg2 python-tempita python-twisted python-yaml socat syslinux-common tgt ubuntu-cloudimage-keyring wget xvfb
Reading package lists...
Building dependency tree...
Reading state information...
apache2 is already the newest version (2.4.18-2ubuntu2).
archdetect-deb is already the newest version (1.117ubuntu1).
authbind is already the newest version (2.1.1+nmu1).
bash is already the newest version (4.3-14ubuntu1).
bind9 is already the newest version (1:9.10.3.dfsg.P4-5).
bind9utils is already ...

[MAAS Lander (maas-lander)]

The attempt to merge lp:~newell-jensen/maas/fix-1567177 into lp:maas failed. Below is the output from the failed tests.

Hit:1 http://security.ubuntu.com/ubuntu xenial-security InRelease
Get:2 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial InRelease [247 kB]
Hit:3 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial-updates InRelease
Hit:4 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial-backports InRelease
Get:5 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial/main Sources [875 kB]
Get:6 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial/universe Sources [7,751 kB]
Get:7 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial/main amd64 Packages [1,197 kB]
Get:8 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial/universe amd64 Packages [7,530 kB]
Fetched 17.6 MB in 3s (5,582 kB/s)
Reading package lists...
sudo DEBIAN_FRONTEND=noninteractive apt-get -y \
    --no-install-recommends install apache2 archdetect-deb authbind bash bind9 bind9utils build-essential bzr-builddeb chromium-browser chromium-chromedriver curl daemontools debhelper dh-apport dh-systemd distro-info dnsutils firefox freeipmi-tools git gjs ipython isc-dhcp-common libjs-angularjs libjs-jquery libjs-jquery-hotkeys libjs-yui3-full libjs-yui3-min libpq-dev make nodejs-legacy npm postgresql pxelinux python3-all python3-apt python3-bson python3-convoy python3-coverage python3-crochet python3-cssselect python3-curtin python3-dev python3-distro-info python3-django python3-django-nose python3-django-piston3 python3-dnspython python3-docutils python3-formencode python3-hivex python3-httplib2 python3-jinja2 python3-jsonschema python3-lxml python3-mock python3-netaddr python3-netifaces python3-oauth python3-oauthlib python3-openssl python3-paramiko python3-petname python3-pexpect python3-psycopg2 python3-pyinotify python3-pyparsing python3-pyvmomi python3-requests python3-seamicroclient python3-setuptools python3-simplestreams python3-sphinx python3-tempita python3-twisted python3-txtftp python3-tz python3-yaml python3-zope.interface python-bson python-crochet python-django python-django-piston python-djorm-ext-pgarray python-formencode python-lxml python-netaddr python-netifaces python-pocket-lint python-psycopg2 python-tempita python-twisted python-yaml socat syslinux-common tgt ubuntu-cloudimage-keyring wget xvfb
Reading package lists...
Building dependency tree...
Reading state information...
apache2 is already the newest version (2.4.18-2ubuntu2).
archdetect-deb is already the newest version (1.117ubuntu1).
authbind is already the newest version (2.1.1+nmu1).
bash is already the newest version (4.3-14ubuntu1).
bind9 is already the newest version (1:9.10.3.dfsg.P4-5).
bind9utils is already the newest version (1:9.10.3.dfsg.P4-5).
build-essential is already the newest version (12.1ubuntu2).
curl is already the newest version (7.47.0-1ubuntu2).
debhelper is already the newest version (9.20160115ubuntu3).
distro-info is already the newest version (0.14build1).
dnsutils is already the newest version (1:9.10.3.dfsg.P4-5).
firefox is already the newest version (45.0.1+build1-0ubuntu1).
freeipmi-too...

=== modified file 'src/maasserver/models/node.py'
--- src/maasserver/models/node.py	2016-04-11 07:25:18 +0000
+++ src/maasserver/models/node.py	2016-04-12 21:01:16 +0000
@@ -1844,7 +1844,9 @@
             maaslog.warning("%s: Unrecognised power type.", self.hostname)
             return PowerInfo(False, False, False, None, None)
         else:
-            if power_type == 'manual':
+            if power_type == 'manual' or self.node_type in (
+                    NODE_TYPE.RACK_CONTROLLER,
+                    NODE_TYPE.REGION_AND_RACK_CONTROLLER):
                 can_be_started = False
                 can_be_stopped = False
             else:

=== modified file 'src/maasserver/models/signals/power.py'
--- src/maasserver/models/signals/power.py	2016-04-01 20:04:41 +0000
+++ src/maasserver/models/signals/power.py	2016-04-12 21:01:16 +0000
@@ -44,11 +44,13 @@
         enum, or `None` which denotes that the status could not be queried or
         updated for any of a number of reasons; check the log.
     """
+    def eb_error(failure):
+        failure.trap(
+            Node.DoesNotExist, UnknownPowerType, PowerProblem)
+
     d = deferToDatabase(transactional(Node.objects.get), system_id=system_id)
     d.addCallback(lambda node: node.power_query())
-    d.addErrback(
-        lambda failure: failure.trap(
-            (Node.DoesNotExist, UnknownPowerType, PowerProblem)))
+    d.addErrback(eb_error)
     d.addErrback(
         log.err,
         "Failed to update power state of machine after state transition.")

=== modified file 'src/maasserver/models/signals/tests/test_power.py'
--- src/maasserver/models/signals/tests/test_power.py	2016-04-01 20:04:41 +0000
+++ src/maasserver/models/signals/tests/test_power.py	2016-04-12 21:01:16 +0000
@@ -102,8 +102,8 @@
     @defer.inlineCallbacks
     def test__traps_failure_for_UnknownPowerType(self):
         node = yield deferToDatabase(transactional(factory.make_Node))
-        mock_node_objects_get = self.patch(Node.objects, "get")
-        mock_node_objects_get.side_effect = UnknownPowerType()
+        mock_power_query = self.patch(Node, "power_query")
+        mock_power_query.side_effect = UnknownPowerType()
         power_state = yield power.update_power_state_of_node(node.system_id)
         self.assertIsNone(power_state)
 
@@ -111,8 +111,8 @@
     @defer.inlineCallbacks
     def test__traps_failure_for_PowerProblem(self):
         node = yield deferToDatabase(transactional(factory.make_Node))
-        mock_node_objects_get = self.patch(Node.objects, "get")
-        mock_node_objects_get.side_effect = PowerProblem()
+        mock_power_query = self.patch(Node, "power_query")
+        mock_power_query.side_effect = PowerProblem()
         power_state = yield power.update_power_state_of_node(node.system_id)
         self.assertIsNone(power_state)
 
@@ -120,8 +120,8 @@
     @defer.inlineCallbacks
     def test__logs_other_errors(self):
         node = yield deferToDatabase(transactional(factory.make_Node))
-        mock_node_objects_get = self.patch(Node.objects, "get")
-        mock_node_objects_get.side_effect = factory.make_exception('Error')
+        mock_power_query = self.patch(Node, "power_query")
+        mock_power_query.side_effect = factory.make_exception('Error')
         mock_log_err = self.patch(power.log, "err")
         yield power.update_power_state_of_node(node.system_id)
         self.assertThat(

=== modified file 'src/maasserver/models/tests/test_node.py'
--- src/maasserver/models/tests/test_node.py	2016-04-11 07:25:18 +0000
+++ src/maasserver/models/tests/test_node.py	2016-04-12 21:01:16 +0000
@@ -856,6 +856,17 @@
             (False, False, False, "manual", {}),
             node.get_effective_power_info())
 
+    def test_get_effective_power_info_can_be_False_for_rack_controller(self):
+        for node_type in (NODE_TYPE.REGION_AND_RACK_CONTROLLER,
+                          NODE_TYPE.RACK_CONTROLLER):
+            node = factory.make_Node(node_type=node_type)
+            gepp = self.patch(node, "get_effective_power_parameters")
+            # For manual the power can never be turned off or on.
+            gepp.return_value = sentinel.power_parameters
+            self.assertEqual(
+                (False, False, True, node.power_type,
+                 sentinel.power_parameters), node.get_effective_power_info())
+
     def test_get_effective_power_info_cant_be_queried(self):
         all_power_types = {
             power_type_details['name']