[2.0 RC1] Deploying node not powered on - Unable to change power state to 'on' for node krastin: another action is already in progress for that node.
| Affects | Status | Importance | Assigned to | Milestone | |
|---|---|---|---|---|---|
| MAAS |
Fix Released
|
High
|
Newell Jensen | ||
Bug Description
So, the scenario is as following. I am attempting to deploy a node. It's switching to deploying but I am expecting node to be powered on and it's never powered on. Looking at the event log, I see a node powered off where there should have been a Powering node on/Node powered on sequence events. I then retried and things went as expected.
This is different from bug 1427573 where system actually PXE'ed before being Powered Off.
Event Time
Queried node's BMC - Power state queried: on Wed, 13 Jul. 2016 01:41:31
PXE Request - installation Wed, 13 Jul. 2016 01:39:11
PXE Request - installation Wed, 13 Jul. 2016 01:39:11
Queried node's BMC - Power state queried: on Wed, 13 Jul. 2016 01:37:14
Node powered on Wed, 13 Jul. 2016 01:37:12
Powering node on Wed, 13 Jul. 2016 01:37:07
User starting deployment - (larry) Wed, 13 Jul. 2016 01:37:06
User acquiring node - (larry) Wed, 13 Jul. 2016 01:37:06
Queried node's BMC - Power state queried: off Wed, 13 Jul. 2016 01:37:01
User releasing node - (larry) Wed, 13 Jul. 2016 01:36:57
Queried node's BMC - Power state queried: off Wed, 13 Jul. 2016 01:36:04
Node powered off Wed, 13 Jul. 2016 01:32:57
User starting deployment - (larry) Wed, 13 Jul. 2016 01:32:47
User acquiring node - (larry) Wed, 13 Jul. 2016 01:32:47
Queried node's BMC - Power state queried: off Wed, 13 Jul. 2016 01:32:38
Powering node off Wed, 13 Jul. 2016 01:32:23
User releasing node - (larry) Wed, 13 Jul. 2016 01:32:23
Queried node's BMC - Power state queried: on Wed, 13 Jul. 2016 01:31:36
Queried node's BMC - Power state queried: on Wed, 13 Jul. 2016 01:26:02
Queried node's BMC - Power state queried: on Wed, 13 Jul. 2016 01:20:57
Queried node's BMC - Power state queried: on Wed, 13 Jul. 2016 01:15:43
Queried node's BMC - Power state queried: on Wed, 13 Jul. 2016 01:10:36
Queried node's BMC - Power state queried: on Wed, 13 Jul. 2016 01:05:12
Queried node's BMC - Power state queried: on Wed, 13 Jul. 2016 01:00:13
Queried node's BMC - Power state queried: on Wed, 13 Jul. 2016 00:54:58
Queried node's BMC - Power state queried: on Wed, 13 Jul. 2016 00:49:42
Queried node's BMC - Power state queried: on Wed, 13 Jul. 2016 00:44:27
Queried node's BMC - Power state queried: on Wed, 13 Jul. 2016 00:39:12
Queried node's BMC - Power state queried: on Wed, 13 Jul. 2016 00:33:57
Queried node's BMC - Power state queried: on Wed, 13 Jul. 2016 00:29:23
I have collected the logs and I am attaching to this bug.
Going back to recreate, I saw this message: "Node failed to be deployed, because of the following error: Unable to change power state to 'on' for node krastin: another action is already in progress for that node."
But that error message was incorrect, because node was still in the Deploying state. This is something I had previously observed recently where a system was in deploying state and powered off. I was able to do a manual poweron and it deployed OK. But, having been able to recreate it, it's looking more like an issue than a transient thing.
The method to recreate seems to be to release the node and there's a window after system switches back to the ready state where deploying will fail to power on the system because it's still being powered off.
ubuntu@
Desired=
| Status=
|/ Err?=(none)
||/ Name Version Architecture Description
+++-===
ii maas 2.0.0~rc1+
ii maas-cli 2.0.0~rc1+
un maas-cluster-
ii maas-common 2.0.0~rc1+
ii maas-dhcp 2.0.0~rc1+
ii maas-dns 2.0.0~rc1+
ii maas-proxy 2.0.0~rc1+
ii maas-rack-
ii maas-region-api 2.0.0~rc1+
ii maas-region-
un maas-region-
un python-django-maas <none> <none> (no description available)
un python-maas-client <none> <none> (no description available)
un python-
ii python3-django-maas 2.0.0~rc1+
ii python3-maas-client 2.0.0~rc1+
ii python3-
Related branches
- Gavin Panella (community): Approve
-
Diff: 372 lines (+62/-67)3 files modifiedsrc/maasserver/models/node.py (+11/-8)
src/maasserver/models/tests/test_node.py (+45/-56)
src/maasserver/tests/test_node_action.py (+6/-3)
| Larry Michel (lmic) wrote | #1 |
| summary: |
- [2.0 RC1] Node powered off soon after transitioning to deploying instead - of being Powered on + [2.0 RC1] Deploying node not powered on - Unable to change power state + to 'on' for node krastin: another action is already in progress for that + node. |
| Larry Michel (lmic) wrote | #3 |
| Andres Rodriguez (andreserl) wrote | #4 |
| Changed in maas: | |
| importance: | Undecided → High |
| status: | New → In Progress |
| assignee: | nobody → Newell Jensen (newell-jensen) |
| Changed in maas: | |
| status: | In Progress → Fix Committed |
| Changed in maas: | |
| status: | Fix Committed → Fix Released |
Hi Larry,
Can you confirm you see:
"Unable to change power state to 'on' for node krastin: another action is
already in progress for that node."
In the UI, but the node still switches to "deploying"?
On Tuesday, July 12, 2016, Larry Michel <email address hidden> wrote:
> Public bug reported:
>
> So, the scenario is as following. I am attempting to deploy a node. It's
> switching to deploying but I am expecting node to be powered on and it's
> never powered on. Looking at the event log, I see a node powered off
> where there should have been a Powering node on/Node powered on sequence
> events. I then retried and things went as expected.
>
>
> This is different from bug 1427573 where system actually PXE'ed before
> being Powered Off.
>
>
> Event Time
> Queried node's BMC - Power state queried: on Wed, 13 Jul. 2016 01:41:31
> PXE Request - installation Wed, 13 Jul. 2016 01:39:11
> PXE Request - installation Wed, 13 Jul. 2016 01:39:11
> Queried node's BMC - Power state queried: on Wed, 13 Jul. 2016 01:37:14
> Node powered on Wed, 13 Jul. 2016 01:37:12
> Powering node on Wed, 13 Jul. 2016 01:37:07
> User starting deployment - (larry) Wed, 13 Jul. 2016 01:37:06
> User acquiring node - (larry) Wed, 13 Jul. 2016 01:37:06
> Queried node's BMC - Power state queried: off Wed, 13 Jul. 2016 01:37:01
> User releasing node - (larry) Wed, 13 Jul. 2016 01:36:57
> Queried node's BMC - Power state queried: off Wed, 13 Jul. 2016 01:36:04
> Node powered off Wed, 13 Jul. 2016 01:32:57
> User starting deployment - (larry) Wed, 13 Jul. 2016 01:32:47
> User acquiring node - (larry) Wed, 13 Jul. 2016 01:32:47
> Queried node's BMC - Power state queried: off Wed, 13 Jul. 2016 01:32:38
> Powering node off Wed, 13 Jul. 2016 01:32:23
> User releasing node - (larry) Wed, 13 Jul. 2016 01:32:23
> Queried node's BMC - Power state queried: on Wed, 13 Jul. 2016 01:31:36
> Queried node's BMC - Power state queried: on Wed, 13 Jul. 2016 01:26:02
> Queried node's BMC - Power state queried: on Wed, 13 Jul. 2016 01:20:57
> Queried node's BMC - Power state queried: on Wed, 13 Jul. 2016 01:15:43
> Queried node's BMC - Power state queried: on Wed, 13 Jul. 2016 01:10:36
> Queried node's BMC - Power state queried: on Wed, 13 Jul. 2016 01:05:12
> Queried node's BMC - Power state queried: on Wed, 13 Jul. 2016 01:00:13
> Queried node's BMC - Power state queried: on Wed, 13 Jul. 2016 00:54:58
> Queried node's BMC - Power state queried: on Wed, 13 Jul. 2016 00:49:42
> Queried node's BMC - Power state queried: on Wed, 13 Jul. 2016 00:44:27
> Queried node's BMC - Power state queried: on Wed, 13 Jul. 2016 00:39:12
> Queried node's BMC - Power state queried: on Wed, 13 Jul. 2016 00:33:57
> Queried node's BMC - Power state queried: on Wed, 13 Jul. 2016 00:29:23
>
> I have collected the logs and I am attaching to this bug.
>
> Going back to recreate, I saw this message: "Node failed to be deployed,
> because of the following error: Unable to change power state to 'on' for
> node krastin: another action is already in progress for that node."
>
> But that error message was incorrect, because...