Merge ~travisghansen/maas:master into maas:master

Proposed by Travis Glenn Hansen
Status: Merged
Merge reported by: Lee Trager
Merged at revision: 13a6cadde0c989e676cc98f6d25264d1885b6509
Proposed branch: ~travisghansen/maas:master
Merge into: maas:master
Diff against target: 147 lines (+123/-0)
2 files modified
src/provisioningserver/drivers/power/registry.py (+2/-0)
src/provisioningserver/drivers/power/webhook.py (+121/-0)
Reviewer Review Type Date Requested Status
Lee Trager (community) Approve
MAAS Lander Needs Fixing
Review via email: mp+395743@code.launchpad.net

Commit message

Webhook Power Driver

Description of the change

This is a simple power driver that POSTs to an http(s) endpoint of the user's choosing. The endpoint receives a json payload which looks like:

{"action":"power_on","system_id":"<id>"}
OR
{"action":"power_off","system_id":"<id>"}
OR
{"action":"power_query","system_id":"<id>"}

The "power_query" POST expects a response of the following:
{"power_state": "on|off|unknown"}

SSL validation can be disabled to work with self-signed certs.

Authentication can be done by either setting a username/password for basic auth OR bearer auth by setting a token. Preference if given to the token if both token and username/password are set.

Of note, the URL can include query params.

I've tested this out by hacking the files into a snap and prototyping. In my simple use-case I'm configuring the endpoint to point to a Node-RED instance I have and then I wire everything up to control some crude/cheap iot power plugs (gosund devices with tasmota firmware flashed).

I'm completely new to maas (and python is not my strength) so I could be missing something entirely here so any tips are appreciated :)

To post a comment you must log in.
Revision history for this message
MAAS Lander (maas-lander) wrote :

UNIT TESTS
-b master lp:~travisghansen/maas/+git/maas into -b master lp:~maas-committers/maas

STATUS: FAILED
LOG: http://maas-ci.internal:8080/job/maas/job/branch-tester/8968/console
COMMIT: 64e77f06124644a038f2705826989b41be66580d

review: Needs Fixing
Revision history for this message
MAAS Lander (maas-lander) wrote :

UNIT TESTS
-b master lp:~travisghansen/maas/+git/maas into -b master lp:~maas-committers/maas

STATUS: FAILED
LOG: http://maas-ci.internal:8080/job/maas/job/branch-tester/8969/console
COMMIT: 23f40223494313dc3a298987b622604ef0387e98

review: Needs Fixing
Revision history for this message
Adam Collard (adam-collard) wrote :

Thanks for the contribution!

I haven't looked at the code per se but wanted to share output from the CI job:

ERROR: /run/build/maas/src/provisioningserver/drivers/power/webhook.py Imports are incorrectly sorted.
 --- /run/build/maas/src/provisioningserver/drivers/power/webhook.py:before 2021-01-04 19:19:49.596325
 +++ /run/build/maas/src/provisioningserver/drivers/power/webhook.py:after 2021-01-04 19:22:47.812771
 @@ -11,13 +11,13 @@
  import urllib.request

  from provisioningserver.drivers import make_ip_extractor, make_setting_field
 -from provisioningserver.logger import get_maas_logger
  from provisioningserver.drivers.power import (
      PowerActionError,
      PowerConnError,
      PowerDriver,
      PowerFatalError,
  )
 +from provisioningserver.logger import get_maas_logger

  maaslog = get_maas_logger("drivers.power.webhook")

 make: *** [Makefile:213: lint-py] Error 1

Revision history for this message
Travis Glenn Hansen (travisghansen) wrote :

Fun. Do you have a standard linter/code formatter I can use to sort it out (pun intended).

Revision history for this message
Adam Collard (adam-collard) wrote :

We do indeed!

`make format-py` is the easiest way to get it, else `bin/isort src/provisioningserver/drivers/power/webhook.py` for a more targetted fix.

In terms of formatting, we use https://github.com/psf/black

Revision history for this message
Travis Glenn Hansen (travisghansen) wrote :

I tried the make format-py and it updated basically everything, so I went with the isort command :)

Revision history for this message
MAAS Lander (maas-lander) wrote :

UNIT TESTS
-b master lp:~travisghansen/maas/+git/maas into -b master lp:~maas-committers/maas

STATUS: FAILED
LOG: http://maas-ci.internal:8080/job/maas/job/branch-tester/8992/console
COMMIT: a90321b475d79e7812d7a516e142bb3b7ddad855

review: Needs Fixing
Revision history for this message
Lee Trager (ltrager) wrote :

Thanks for the patch! Couple of things

1. I think this should have an optional "id" field. This will allow one webhook service to handle multiple machines. For example if you create a wake on LAN webhook the id would be the MAC address.
2. We require unit tests for all branches to land.
3. MAAS uses Twisted[1] for HTTP calls. This allows MAAS to perform other tasks while waiting for the response.
4. See other comments below

If Twisted looks to complicated you can skip 2 and 3 and I'll Twistify it and add unit tests in a follow up branch.

[1] https://twistedmatrix.com/documents/current/web/howto/client.html

review: Needs Fixing
Revision history for this message
MAAS Lander (maas-lander) wrote :

UNIT TESTS
-b master lp:~travisghansen/maas/+git/maas into -b master lp:~maas-committers/maas

STATUS: FAILED
LOG: http://maas-ci.internal:8080/job/maas/job/branch-tester/8993/console
COMMIT: 13a6cadde0c989e676cc98f6d25264d1885b6509

review: Needs Fixing
Revision history for this message
Travis Glenn Hansen (travisghansen) wrote :

Thanks for the review!

1. I considered this but figured it could be managed via the GET/query params within the URL itself.
2. I'll see what I can put together for that. Trying to setup a development environment has been...challenging. Even worse is trying to inject the code into a snap :( so testing for me generally has been quite challenging.
3. I based it on the code from recs.py which is using urllib

It would be great if you could twistify it as I'm sure I'll mess something up with the async/await stuff especially given how hard it is for me to *really* test changes. Is there a doc somewhere with instructions for running locally and what services are needed etc? Preferably something that is OS independent since I don't run ubuntu on my workstation.

Revision history for this message
Travis Glenn Hansen (travisghansen) wrote :

Any further guidance on this? I'd hate to see bitrot :)

Revision history for this message
Lee Trager (ltrager) wrote :

I created a follow on branch[1] which uses Twisted for the web calls and makes this a bit more extendable so it will work with multiple services. If that works I can land both.

[1] https://code.launchpad.net/~ltrager/maas/+git/maas/+merge/396049

Revision history for this message
Travis Glenn Hansen (travisghansen) wrote :

Awesome! I've commented over there to continue the conversation!

Revision history for this message
Lee Trager (ltrager) :
review: Approve
Revision history for this message
MAAS Lander (maas-lander) wrote :

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
diff --git a/src/provisioningserver/drivers/power/registry.py b/src/provisioningserver/drivers/power/registry.py
index 93db204..91baa68 100644
--- a/src/provisioningserver/drivers/power/registry.py
+++ b/src/provisioningserver/drivers/power/registry.py
@@ -25,6 +25,7 @@ from provisioningserver.drivers.power.redfish import RedfishPowerDriver
25from provisioningserver.drivers.power.seamicro import SeaMicroPowerDriver25from provisioningserver.drivers.power.seamicro import SeaMicroPowerDriver
26from provisioningserver.drivers.power.ucsm import UCSMPowerDriver26from provisioningserver.drivers.power.ucsm import UCSMPowerDriver
27from provisioningserver.drivers.power.vmware import VMwarePowerDriver27from provisioningserver.drivers.power.vmware import VMwarePowerDriver
28from provisioningserver.drivers.power.webhook import WebhookPowerDriver
28from provisioningserver.drivers.power.wedge import WedgePowerDriver29from provisioningserver.drivers.power.wedge import WedgePowerDriver
29from provisioningserver.utils.registry import Registry30from provisioningserver.utils.registry import Registry
3031
@@ -65,6 +66,7 @@ power_drivers = [
65 SeaMicroPowerDriver(),66 SeaMicroPowerDriver(),
66 UCSMPowerDriver(),67 UCSMPowerDriver(),
67 VMwarePowerDriver(),68 VMwarePowerDriver(),
69 WebhookPowerDriver(),
68 WedgePowerDriver(),70 WedgePowerDriver(),
69]71]
70for driver in power_drivers:72for driver in power_drivers:
diff --git a/src/provisioningserver/drivers/power/webhook.py b/src/provisioningserver/drivers/power/webhook.py
71new file mode 10064473new file mode 100644
index 0000000..ad7e5b6
--- /dev/null
+++ b/src/provisioningserver/drivers/power/webhook.py
@@ -0,0 +1,121 @@
1# Copyright 2016 Canonical Ltd. This software is licensed under the
2# GNU Affero General Public License version 3 (see the file LICENSE).
3
4"""Generic Webhook Power Driver."""
5
6import base64
7import json
8import ssl
9import urllib.error
10import urllib.parse
11import urllib.request
12
13from provisioningserver.drivers import make_ip_extractor, make_setting_field
14from provisioningserver.drivers.power import (
15 PowerActionError,
16 PowerConnError,
17 PowerDriver,
18 PowerFatalError,
19)
20
21SSL_INSECURE_YES="y"
22SSL_INSECURE_NO="n"
23
24SSL_INSECURE_CHOICES = [
25 [SSL_INSECURE_NO, "No"],
26 [SSL_INSECURE_YES, "Yes"]]
27
28
29class WebhookPowerDriver(PowerDriver):
30
31 name = "webhook"
32 chassis = False
33 can_probe = False
34 description = "Webhook"
35 settings = [
36 make_setting_field("power_url", "URL", required=True),
37 make_setting_field("power_insecure_ssl", "Allow insecure SSL connections", field_type='choice', required=True,
38 choices=SSL_INSECURE_CHOICES, default=SSL_INSECURE_NO),
39 make_setting_field("power_user", "Power user"),
40 make_setting_field(
41 "power_pass", "Power password", field_type="password"
42 ),
43 make_setting_field(
44 "power_token", "Power token", field_type="password"
45 ),
46 ]
47
48 #ip_extractor = make_ip_extractor("power_address")
49 ip_extractor = None
50
51 def post(self, context, params={}):
52 """Dispatch a POST request to webhook."""
53 url = context.get("power_url")
54 data = json.dumps(params).encode('utf-8')
55 req = urllib.request.Request(url, data, method="POST")
56
57 if context.get("power_token") and len(context.get("power_token")) > 0:
58 authString = "{token}".format(token = context.get("power_token")).encode("utf-8")
59 authString = str(authString, "utf-8")
60 req.add_header("Authorization", "Bearer %s" % authString)
61 elif context.get("power_user") and len(context.get("power_user")) > 0 and context.get("power_pass") and len(context.get("power_pass")) > 0:
62 authString = base64.urlsafe_b64encode("{username}:{password}".format(username = context.get("power_user"), password = context.get("power_pass")).encode("utf-8"))
63 authString = str(authString, "utf-8")
64 req.add_header("Authorization", "Basic %s" % authString)
65
66 req.add_header('content-type', 'application/json')
67
68 myssl = None
69 if context.get("power_insecure_ssl") and context.get("power_insecure_ssl") == SSL_INSECURE_YES:
70 myssl = ssl.create_default_context();
71 myssl.check_hostname=False
72 myssl.verify_mode=ssl.CERT_NONE
73 try:
74 res = urllib.request.urlopen(req, context=myssl)
75 except urllib.error.HTTPError as e:
76 raise PowerConnError(
77 "Could not make proper connection to Webhook."
78 " HTTP error code: %s" % e.code
79 )
80 except urllib.error.URLError as e:
81 raise PowerConnError(
82 "Could not make proper connection to Webhook."
83 " Server could not be reached: %s" % e.reason
84 )
85 else:
86 body = res.read();
87 if not body or len(body) == 0:
88 return dict()
89 return json.loads(body)
90
91 def detect_missing_packages(self):
92 # uses pure-python http client - nothing to look for!
93 return []
94
95 def power_on(self, system_id, context):
96 """Power on webhook."""
97 try:
98 self.post(context, {"action": "power_on", "system_id": system_id})
99 except PowerConnError:
100 raise PowerActionError("Webhook Driver unable to power on")
101
102 def power_off(self, system_id, context):
103 """Power off webhook."""
104 try:
105 self.post(context, {"action": "power_off", "system_id": system_id})
106 except PowerConnError:
107 raise PowerActionError("Webhook Driver unable to power off")
108
109 def power_query(self, system_id, context):
110 """Power query webhook."""
111 try:
112 res = self.post(context, {"action": "power_query", "system_id": system_id})
113 except PowerConnError:
114 raise PowerActionError("Webhook Driver unable to power query")
115 else:
116 if res["power_state"] == "off":
117 return "off"
118 elif res["power_state"] == "on":
119 return "on"
120 else:
121 return "unknown"

Subscribers

People subscribed via source and target branches