Merge lp:~jason-hobbs/maas/ucs-xml-api into lp:~maas-committers/maas/trunk
- ucs-xml-api
- Merge into trunk
Status: | Merged | ||||
---|---|---|---|---|---|
Approved by: | Jason Hobbs | ||||
Approved revision: | no longer in the source branch. | ||||
Merged at revision: | 2303 | ||||
Proposed branch: | lp:~jason-hobbs/maas/ucs-xml-api | ||||
Merge into: | lp:~maas-committers/maas/trunk | ||||
Diff against target: |
1349 lines (+1215/-0) 12 files modified
etc/maas/templates/power/ucsm.template (+15/-0) src/maasserver/api.py (+23/-0) src/maasserver/models/nodegroup.py (+11/-0) src/maasserver/tests/test_api_nodegroup.py (+27/-0) src/provisioningserver/custom_hardware/tests/test_ucsm.py (+638/-0) src/provisioningserver/custom_hardware/ucsm.py (+435/-0) src/provisioningserver/power/tests/test_poweraction.py (+11/-0) src/provisioningserver/power_schema.py (+10/-0) src/provisioningserver/tasks.py (+8/-0) src/provisioningserver/tests/test_tasks.py (+12/-0) src/provisioningserver/utils/__init__.py (+5/-0) src/provisioningserver/utils/tests/test_utils.py (+20/-0) |
||||
To merge this branch: | bzr merge lp:~jason-hobbs/maas/ucs-xml-api | ||||
Related bugs: |
|
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Gavin Panella (community) | Abstain | ||
Jeroen T. Vermeulen (community) | Approve | ||
Review via email: mp+216643@code.launchpad.net |
Commit message
Add support for managing nodes via Cisco UCS Manager's HTTP-XML API.
This adds a new probe and enlist API method and a new power control template for UCS managed nodes.
Description of the change
This adds probe/enlist and power control support for Cisco's UCS HTTP-XML API. I'm still in the final stages of testing it, but it's ready for review now. It's big commit; it wasn't clear to me how to make it smaller but still make it useful.
The power template here is kind of pointless, since the power control is pure python, but it wasn't trivial to work around it. I think we need to consider moving away from always using templates for power control, and make it an option to use a python method directly.
I also think there is room for some common code in handling probe and enlist. The basic pattern is build a list of new nodes and then add them to MAAS. How we deal with existing nodes could be common across all types of enlistments, and having a common interface for the methods could maybe make it easier to build a web UI interface at some point too.
Julian Edwards (julian-edwards) wrote : | # |
Gavin Panella (allenap) wrote : | # |
This is half a review. I have to go out, and I'll finish it later.
As for splitting it up, for one you could have added an empty add_ucsm()
function to NodeGroup, building the API calls around it, and writing
tests in which you mock the behaviour of add_ucsm().
[1]
Hope-quoting, as Julian has mentioned.
[2]
+ def add_ucsm(self, url, username, password):
This isn't a great name for this method. However, I see that it's
consistent with what's gone before.
[3]
+ children = map(Element, children_tags)
+ request_data = make_request_
Bear in mind that map() is lazy in Python 3. Will make_request_data()
deal with that? If not, make it explicit with a list comprehension.
[4]
+ self.assertEqua
The list-comprehension is not needed:
Not sure the slice is either?
Having said that, assertItemsEqual() is useful here:
[5]
+ self.assertRais
assertRaises() allows arguments, so you can write this as:
[6]
+ def make_test_api(self, url='http://
+ cookie='foo', mock_call=True):
+
+ api = make_test_api(url, user, password, cookie)
+
+ if mock_call:
+ mock = self.patch(api, '_call')
+ else:
+ mock = None
+
+ return api, mock
Boolean arguments are a bit smelly, as is returning irrelevant
things. Can you split this into two methods?
def make_test_api(
self, url='http://
return make_test_api(url, user, password, cookie)
def make_test_api(
self, url='http://
api = make_test_api(url, user, password, cookie)
return api, self.patch(api, '_call')
To be continued...
Jeroen T. Vermeulen (jtv) wrote : | # |
There's lots of good stuff here, well-factored, and with a very nice breakdown into coherent, low-complexity functions. But there's just so much of it that I can't complete this review all in one go!
I'll give you the notes I have now. I reviewed up to UCSM_XML_API, moving from the top of the diff downwards, except I skipped its tests for now.
.
The docstring for probe_and_
.
Typo: Cicso.
.
The test case for make_request_data is called TestGetRequestData — uncompleted change of name?
.
In TestUCSMXMLAPIE
(Also, “raise” is a keyword, not a function — no parentheses needed when using it.)
.
In TestParseRespon
No need for the lambda. You can just pass additional arguments for the callable (as well as keyword arguments) to assertRaises:
.
Unless you're in the process of adding more methods to APITestCase, I'd just drop it as a base class. Using inheritance to add features seems like a small thing, but consider how many test-case classes are suddenly tied together in a vague relationship! And once in place, these relationships tend to grow.
What to do with APITestCase.
Actually, I'd probably just leave out the option to suppress the patching, and accept a slightly special case in the one call site where you use that option. This kind of boolean argument is almost always a constant, which means you've really got two similar-
.
UCSM_XML_API is quite a test for naming conventions! The underscores are unusual in a class name for us, but these are unusual circumstances so you won't hear me complain. :)
I do wish I had a clearer understanding of the “children” that are being mentioned throughout. I understand that it's a domain term, so I'd probably need to read up on the UCSM API in order to understand. A very brief note might make it easier to deal with though. Setting the right expectations often helps prevents disasters.
Jeroen T. Vermeulen (jtv) wrote : | # |
Looks like you're getting multiple reviews. One of the risks of large branches! Reviewers' ability to spot problems drops off sharply after the first few hundred lines, so it's probably for the best. (The “sweet spot” for reviews seems to lie around 200 lines).
Some more notes:
.
Could you document what RO_KEYS and strip_ro_keys mean? I see how they fit together, but I lack context about the wider problem it solves.
.
In get_power_command, there's an if/elif without a final ‘else.’ So an unknown power state will just result in a None, which could produce failures that are hard to diagnose. You could avoid that hazard by adding an “else: raise AssertionError(
.
Some identifiers have abbreviations whose meanings are a complete mystery to me, such as dn and bp. Long, verbose names can be unpleasant, but names with unknown meanings can be a positive menace!
.
In power_control_ucsm, you do:
servers = get_servers(api, uuid)
server = servers[0]
Why server[0]? Are you taking an arbitrary one? The latest? Something else? Would be worth a comment.
Or, if you are absolutely 100% totally completely certainly positive that there is exactly one item in that list, pattern-matching lets you write the assignment as:
[server] = get_servers(api, uuid)
.
You document power_control_ucsm as “Power control template handler.” I think I sort of understand all of that, except the “template.” How does that word fit in? It may help to summarise what the function does in the imperative.
.
Similarly, probe_and_
.
For the addition to src/provisionin
.
In TestAddUCSM, a disadvantage of passing highly predictable fixed strings like “username” and “password” is that the test doesn't rule out silly bugs like “always passes ‘username’ instead of the actual user name.” Not saying that that is terribly likely in any given instance, but if we build the right habits, we won't need to make the mistake at all. These mistakes can happen in TDD when you try to write the simplest stub that will satisfy a test.
Jeroen T. Vermeulen (jtv) wrote : | # |
By the way, we deviate from PEP-8 in formatting. When linebreaking (except in def/class definitions) we only indent by 4 columns. We don't align to the opening brace/bracket/
So, where you wrote
...we might write:
mock, MockCalledOnceW
When we need to line-break an arguments list, we do make it a rule to start the first argument on a fresh line.
.
In further bikeshedding, method names “login” and “logout” look a bit strange for me — we generally prefer our function names to start with verbs, but these are the nouns.
.
At this point I'm a bit stuck. I'd like to review the tests in more detail, but it might not be much use because of my limited understanding of the API code's higher-level meaning. It would be nice to have that little bit more explanation!
Gavin Panella (allenap) wrote : | # |
Neat, nice stuff. My comments are almost all about style, not approach.
I think my only real problems with this branch are [1] and the lack of
commentary about how things work. If it's biting me now, it'll bite you
in 3 months when you've long moved on from this.
[7]
+ self.assertEqua
What's the [:] slice for?
[8]
+ def test_parameters
...
+ in_configs = mock.call_
+ self.assertEqua
I don't know what this is demonstrating. Can you add a comment, or
perhaps make it more intrinsically clear. There's a mock.ANY object
which you can use when you don't care what an argument is (e.g. so you
can use one of the Mock* matchers but only test one argument of
several), which may or may not be useful here.
[9]
+ self.assertThat
+ MockCalledOnceW
Fwiw, and it's a minor thing, the MAAS core team (former Red Squad) long
ago chose to not indent after braces like this. For one, it gets silly
when the function name is long and all the arguments get smashed against
the right margin. Prefer starting a new line:
For functions, it might be worth indenting twice:
def this_is_a_function(
or using a docstring or comment to delineate arguments from body:
def this_is_a_function(
I'm definitely not going to block on this though.
[10]
+ def test_uses_
+ uuid = factory.
+ api = make_test_api()
+ mock = self.patch(api, 'config_
+ get_servers(api, uuid)
+ filters = mock.call_
+ attrib = {'class': 'computeItem', 'property': 'uuid', 'value': uuid}
+ self.assertEqua
There could have been multiple calls to config_
using MockCalledOnceW
up the test, and make the test clearer and more concise.
[11]
+ def login(self):
...
+ UCS Manager allows a limited number of active cookies at any
+ point in time, so it's important to free the cookie up when
+ finished by logging out via the ``logout`` method.
Consider making UCSM_XML_API a context manager, where __enter__ logs-in,
and __exit__ logs-out.
[12]
+ def config_
+ """Issue a configResolveClass request."""
What does configResolveClass do? Either comment here, or provide a
(stable, preferably) link to the UCS API documentation. Extra beer for
both. The same goes for most of the other methods, if not all.
[13]
+def get_servers(api, uuid=None):
+ """Retrieve a list of servers from the UCS Manager."""
+ if uuid:
+ attrs = {'class': 'computeItem', 'property': 'uuid', 'value': uuid}
+ fil...
Jason Hobbs (jason-hobbs) wrote : | # |
I've addressed what I think are the critical feedback items, along with some others. There is time sensitivity in getting this in, so I want to repost for review again before all of the feedback items are addressed. I will post a summary of what is left to address, after I get my laptop on a charger!
Jason Hobbs (jason-hobbs) wrote : | # |
Here's the review comments I haven't addressed yet.
From Jeroen:
In TestUCSMXMLAPIE
In TestAddUCSM, a disadvantage of passing highly predictable fixed strings like “username” and “password” is that the test doesn't rule out silly bugs like “always passes ‘username’ instead of the actual user name.” Not saying that that is terribly likely in any given instance, but if we build the right habits, we won't need to make the mistake at all. These mistakes can happen in TDD when you try to write the simplest stub that will satisfy a test.
By the way, we deviate from PEP-8 in formatting. When linebreaking (except in def/class definitions) we only indent by 4 columns. We don't align to the opening brace/bracket/
In further bikeshedding, method names “login” and “logout” look a bit strange for me — we generally prefer our function names to start with verbs, but these are the nouns.
From Gavin:
[8], [9], [10], [11], [13], [14]
Jeroen T. Vermeulen (jtv) wrote : | # |
Great, thanks. I see no reason why this shouldn't be allowed in at this point. I think the remaining things are pretty much all non-essential cleanups that will just happen organically if they bother us again.
MAAS Lander (maas-lander) wrote : | # |
The attempt to merge lp:~jason-hobbs/maas/ucs-xml-api into lp:maas failed. Below is the output from the failed tests.
Ign http://
Get:1 http://
Get:2 http://
Ign http://
Ign http://
Hit http://
Get:3 http://
Hit http://
Get:4 http://
Get:5 http://
Hit http://
Get:6 http://
Hit http://
Get:7 http://
Hit http://
Hit http://
Hit http://
Get:8 http://
Hit http://
Get:9 http://
Get:10 http://
Ign http://
Ign http://
Ign http://
Ign http://
Get:11 http://
Get:12 http://
Get:13 http://
Get:14 http://
Get:15 http://
Get:16 http://
Ign http://
Ign http://
Fetched 348 kB in 1s (285 kB/s)
Reading package lists...
sudo DEBIAN_
--
Gavin Panella (allenap) wrote : | # |
I haven't gone through the changes but Jeroen has, and I doubt I can add much.
Preview Diff
1 | === added file 'etc/maas/templates/power/ucsm.template' |
2 | --- etc/maas/templates/power/ucsm.template 1970-01-01 00:00:00 +0000 |
3 | +++ etc/maas/templates/power/ucsm.template 2014-05-06 13:49:30 +0000 |
4 | @@ -0,0 +1,15 @@ |
5 | +# -*- mode: shell-script -*- |
6 | +# |
7 | +# Control a system via Cisco UCS Manager XML API. |
8 | + |
9 | +{{py: from provisioningserver.utils import escape_py_literal}} |
10 | +python - << END |
11 | +from provisioningserver.custom_hardware.ucsm import power_control_ucsm |
12 | +power_control_ucsm( |
13 | + {{escape_py_literal(power_address) | safe}}, |
14 | + {{escape_py_literal(power_user) | safe}}, |
15 | + {{escape_py_literal(power_pass) | safe}}, |
16 | + {{escape_py_literal(uuid) | safe}}, |
17 | + {{escape_py_literal(power_change) | safe}}, |
18 | +) |
19 | +END |
20 | |
21 | === modified file 'src/maasserver/api.py' |
22 | --- src/maasserver/api.py 2014-05-01 05:38:03 +0000 |
23 | +++ src/maasserver/api.py 2014-05-06 13:49:30 +0000 |
24 | @@ -1728,6 +1728,29 @@ |
25 | |
26 | return HttpResponse(status=httplib.OK) |
27 | |
28 | + @admin_method |
29 | + @operation(idempotent=False) |
30 | + def probe_and_enlist_ucsm(self, request, uuid): |
31 | + """Add the nodes from a Cisco UCS Manager. |
32 | + |
33 | + :param : The URL of the UCS Manager API. |
34 | + :type url: unicode |
35 | + :param username: The username for the API. |
36 | + :type username: unicode |
37 | + :param password: The password for the API. |
38 | + :type password: unicode |
39 | + |
40 | + """ |
41 | + nodegroup = get_object_or_404(NodeGroup, uuid=uuid) |
42 | + |
43 | + url = get_mandatory_param(request.data, 'url') |
44 | + username = get_mandatory_param(request.data, 'username') |
45 | + password = get_mandatory_param(request.data, 'password') |
46 | + |
47 | + nodegroup.enlist_nodes_from_ucsm(url, username, password) |
48 | + |
49 | + return HttpResponse(status=httplib.OK) |
50 | + |
51 | DISPLAYED_NODEGROUPINTERFACE_FIELDS = ( |
52 | 'ip', 'management', 'interface', 'subnet_mask', |
53 | 'broadcast_ip', 'ip_range_low', 'ip_range_high') |
54 | |
55 | === modified file 'src/maasserver/models/nodegroup.py' |
56 | --- src/maasserver/models/nodegroup.py 2014-04-01 06:50:01 +0000 |
57 | +++ src/maasserver/models/nodegroup.py 2014-05-06 13:49:30 +0000 |
58 | @@ -41,6 +41,7 @@ |
59 | from provisioningserver.tasks import ( |
60 | add_new_dhcp_host_map, |
61 | add_seamicro15k, |
62 | + enlist_nodes_from_ucsm, |
63 | import_boot_images, |
64 | report_boot_images, |
65 | ) |
66 | @@ -286,6 +287,16 @@ |
67 | args = (mac, username, password, power_control) |
68 | add_seamicro15k.apply_async(queue=self.uuid, args=args) |
69 | |
70 | + def enlist_nodes_from_ucsm(self, url, username, password): |
71 | + """ Add the servers from a Cicso UCS Manager. |
72 | + |
73 | + :param URL: URL of the Cisco UCS Manager HTTP-XML API. |
74 | + :param username: username for UCS Manager. |
75 | + :param password: password for UCS Manager. |
76 | + """ |
77 | + args = (url, username, password) |
78 | + enlist_nodes_from_ucsm.apply_async(queue=self.uuid, args=args) |
79 | + |
80 | def add_dhcp_host_maps(self, new_leases): |
81 | if len(new_leases) > 0 and len(self.get_managed_interfaces()) > 0: |
82 | # XXX JeroenVermeulen 2012-08-21, bug=1039362: the DHCP |
83 | |
84 | === modified file 'src/maasserver/tests/test_api_nodegroup.py' |
85 | --- src/maasserver/tests/test_api_nodegroup.py 2014-04-22 15:10:32 +0000 |
86 | +++ src/maasserver/tests/test_api_nodegroup.py 2014-05-06 13:49:30 +0000 |
87 | @@ -52,6 +52,7 @@ |
88 | from maasserver.testing.testcase import MAASServerTestCase |
89 | from maastesting.celery import CeleryFixture |
90 | from maastesting.fakemethod import FakeMethod |
91 | +from maastesting.matchers import MockCalledOnceWith |
92 | from metadataserver.fields import Bin |
93 | from metadataserver.models import ( |
94 | commissioningscript, |
95 | @@ -431,6 +432,32 @@ |
96 | httplib.BAD_REQUEST, response.status_code, |
97 | explain_unexpected_response(httplib.BAD_REQUEST, response)) |
98 | |
99 | + def test_probe_and_enlist_ucsm_adds_ucsm(self): |
100 | + nodegroup = factory.make_node_group() |
101 | + url = 'http://url' |
102 | + username = factory.make_name('user') |
103 | + password = factory.make_name('password') |
104 | + self.become_admin() |
105 | + |
106 | + mock = self.patch(nodegroup_module, 'enlist_nodes_from_ucsm') |
107 | + |
108 | + response = self.client.post( |
109 | + reverse('nodegroup_handler', args=[nodegroup.uuid]), |
110 | + { |
111 | + 'op': 'probe_and_enlist_ucsm', |
112 | + 'url': url, |
113 | + 'username': username, |
114 | + 'password': password, |
115 | + }) |
116 | + |
117 | + self.assertEqual( |
118 | + httplib.OK, response.status_code, |
119 | + explain_unexpected_response(httplib.OK, response)) |
120 | + |
121 | + args = (url, username, password) |
122 | + matcher = MockCalledOnceWith(queue=nodegroup.uuid, args=args) |
123 | + self.assertThat(mock.apply_async, matcher) |
124 | + |
125 | |
126 | class TestNodeGroupAPIAuth(MAASServerTestCase): |
127 | """Authorization tests for nodegroup API.""" |
128 | |
129 | === added file 'src/provisioningserver/custom_hardware/tests/test_ucsm.py' |
130 | --- src/provisioningserver/custom_hardware/tests/test_ucsm.py 1970-01-01 00:00:00 +0000 |
131 | +++ src/provisioningserver/custom_hardware/tests/test_ucsm.py 2014-05-06 13:49:30 +0000 |
132 | @@ -0,0 +1,638 @@ |
133 | +# Copyright 2014 Canonical Ltd. This software is licensed under the |
134 | +# GNU Affero General Public License version 3 (see the file LICENSE). |
135 | + |
136 | +"""Tests for ``provisioningserver.custom_hardware.ucsm``.""" |
137 | + |
138 | +from __future__ import ( |
139 | + absolute_import, |
140 | + print_function, |
141 | + unicode_literals, |
142 | + ) |
143 | + |
144 | +str = None |
145 | + |
146 | +__metaclass__ = type |
147 | +__all__ = [] |
148 | + |
149 | +from itertools import permutations |
150 | +import random |
151 | +from StringIO import StringIO |
152 | +import urllib2 |
153 | + |
154 | +from lxml.etree import ( |
155 | + Element, |
156 | + SubElement, |
157 | + XML, |
158 | + ) |
159 | +from maastesting.factory import factory |
160 | +from maastesting.matchers import ( |
161 | + MockCalledOnceWith, |
162 | + MockCallsMatch, |
163 | + MockNotCalled, |
164 | + ) |
165 | +from maastesting.testcase import MAASTestCase |
166 | +from mock import ( |
167 | + ANY, |
168 | + call, |
169 | + Mock, |
170 | + ) |
171 | +from provisioningserver.custom_hardware import ( |
172 | + ucsm, |
173 | + utils, |
174 | + ) |
175 | +from provisioningserver.custom_hardware.ucsm import ( |
176 | + get_children, |
177 | + get_first_booter, |
178 | + get_macs, |
179 | + get_power_command, |
180 | + get_server_power_control, |
181 | + get_servers, |
182 | + get_service_profile, |
183 | + logged_in, |
184 | + make_policy_change, |
185 | + make_request_data, |
186 | + parse_response, |
187 | + power_control_ucsm, |
188 | + probe_and_enlist_ucsm, |
189 | + probe_servers, |
190 | + RO_KEYS, |
191 | + set_lan_boot_default, |
192 | + set_server_power_control, |
193 | + strip_ro_keys, |
194 | + UCSM_XML_API, |
195 | + UCSM_XML_API_Error, |
196 | + ) |
197 | + |
198 | + |
199 | +def make_api(url='http://url', user='u', password='p', |
200 | + cookie='foo', mock_call=True): |
201 | + api = UCSM_XML_API(url, user, password) |
202 | + api.cookie = cookie |
203 | + return api |
204 | + |
205 | + |
206 | +def make_api_patch_call(testcase, *args, **kwargs): |
207 | + api = make_api(*args, **kwargs) |
208 | + mock = testcase.patch(api, '_call') |
209 | + return api, mock |
210 | + |
211 | + |
212 | +def make_fake_result(root_class, child_tag, container='outConfigs'): |
213 | + fake_result = Element(root_class) |
214 | + outConfigs = SubElement(fake_result, container) |
215 | + outConfigs.append(Element(child_tag)) |
216 | + return outConfigs |
217 | + |
218 | + |
219 | +def make_class(): |
220 | + return factory.make_name('class') |
221 | + |
222 | + |
223 | +def make_dn(): |
224 | + return factory.make_name('dn') |
225 | + |
226 | + |
227 | +def make_server(): |
228 | + return factory.make_name('server') |
229 | + |
230 | + |
231 | +class TestUCSMXMLAPIError(MAASTestCase): |
232 | + """Tests for ``UCSM_XML_API_Error``.""" |
233 | + |
234 | + def test_includes_code_and_msg(self): |
235 | + def raise_error(): |
236 | + raise UCSM_XML_API_Error('bad', 4224) |
237 | + |
238 | + error = self.assertRaises(UCSM_XML_API_Error, raise_error) |
239 | + |
240 | + self.assertEqual('bad', error.args[0]) |
241 | + self.assertEqual(4224, error.code) |
242 | + |
243 | + |
244 | +class TestMakeRequestData(MAASTestCase): |
245 | + """Tests for ``make_request_data``.""" |
246 | + |
247 | + def test_no_children(self): |
248 | + fields = {'hello': 'there'} |
249 | + request_data = make_request_data('foo', fields) |
250 | + root = XML(request_data) |
251 | + self.assertEqual('foo', root.tag) |
252 | + self.assertEqual('there', root.get('hello')) |
253 | + |
254 | + def test_with_children(self): |
255 | + fields = {'hello': 'there'} |
256 | + children_tags = ['bar', 'baz'] |
257 | + children = [Element(child_tag) for child_tag in children_tags] |
258 | + request_data = make_request_data('foo', fields, children) |
259 | + root = XML(request_data) |
260 | + self.assertEqual('foo', root.tag) |
261 | + self.assertItemsEqual(children_tags, (e.tag for e in root)) |
262 | + |
263 | + def test_no_fields(self): |
264 | + request_data = make_request_data('foo') |
265 | + root = XML(request_data) |
266 | + self.assertEqual('foo', root.tag) |
267 | + |
268 | + |
269 | +class TestParseResonse(MAASTestCase): |
270 | + """Tests for ``parse_response``.""" |
271 | + |
272 | + def test_no_error(self): |
273 | + xml = '<foo/>' |
274 | + response = parse_response(xml) |
275 | + self.assertEqual('foo', response.tag) |
276 | + |
277 | + def test_error(self): |
278 | + xml = '<foo errorCode="123" errorDescr="mayday!"/>' |
279 | + self.assertRaises(UCSM_XML_API_Error, parse_response, xml) |
280 | + |
281 | + |
282 | +class TestLogin(MAASTestCase): |
283 | + """"Tests for ``UCSM_XML_API.login``.""" |
284 | + |
285 | + def test_login_assigns_cookie(self): |
286 | + cookie = 'chocolate chip' |
287 | + api, mock = make_api_patch_call(self) |
288 | + mock.return_value = Element('aaaLogin', {'outCookie': cookie}) |
289 | + api.login() |
290 | + self.assertEqual(cookie, api.cookie) |
291 | + |
292 | + def test_login_call_parameters(self): |
293 | + user = 'user' |
294 | + password = 'pass' |
295 | + api, mock = make_api_patch_call(self, user=user, password=password) |
296 | + api.login() |
297 | + fields = {'inName': user, 'inPassword': password} |
298 | + self.assertThat(mock, MockCalledOnceWith('aaaLogin', fields)) |
299 | + |
300 | + |
301 | +class TestLogout(MAASTestCase): |
302 | + """"Tests for ``UCSM_XML_API.logout``.""" |
303 | + |
304 | + def test_logout_clears_cookie(self): |
305 | + api = make_api() |
306 | + self.patch(api, '_call') |
307 | + api.logout() |
308 | + self.assertIsNone(api.cookie) |
309 | + |
310 | + def test_logout_uses_cookie(self): |
311 | + api, mock = make_api_patch_call(self) |
312 | + cookie = api.cookie |
313 | + api.logout() |
314 | + fields = {'inCookie': cookie} |
315 | + self.assertThat(mock, MockCalledOnceWith('aaaLogout', fields)) |
316 | + |
317 | + |
318 | +class TestConfigResolveClass(MAASTestCase): |
319 | + """"Tests for ``UCSM_XML_API.config_resolve_class``.""" |
320 | + |
321 | + def test_no_filters(self): |
322 | + class_id = make_class() |
323 | + api, mock = make_api_patch_call(self) |
324 | + api.config_resolve_class(class_id) |
325 | + fields = {'cookie': api.cookie, 'classId': class_id} |
326 | + self.assertThat(mock, MockCalledOnceWith('configResolveClass', fields, |
327 | + ANY)) |
328 | + |
329 | + def test_with_filters(self): |
330 | + class_id = make_class() |
331 | + filter_element = Element('hi') |
332 | + api, mock = make_api_patch_call(self) |
333 | + api.config_resolve_class(class_id, [filter_element]) |
334 | + in_filters = mock.call_args[0][2] |
335 | + self.assertEqual([filter_element], in_filters[0][:]) |
336 | + |
337 | + def test_return_response(self): |
338 | + api, mock = make_api_patch_call(self) |
339 | + mock.return_value = Element('test') |
340 | + result = api.config_resolve_class('c') |
341 | + self.assertEqual(mock.return_value, result) |
342 | + |
343 | + |
344 | +class TestConfigResolveChildren(MAASTestCase): |
345 | + """"Tests for ``UCSM_XML_API.config_resolve_children``.""" |
346 | + |
347 | + def test_parameters(self): |
348 | + dn = make_dn() |
349 | + class_id = make_class() |
350 | + api, mock = make_api_patch_call(self) |
351 | + api.config_resolve_children(dn, class_id) |
352 | + fields = {'inDn': dn, 'classId': class_id, 'cookie': api.cookie} |
353 | + self.assertThat(mock, |
354 | + MockCalledOnceWith('configResolveChildren', fields)) |
355 | + |
356 | + def test_no_class_id(self): |
357 | + dn = make_dn() |
358 | + api, mock = make_api_patch_call(self) |
359 | + api.config_resolve_children(dn) |
360 | + fields = {'inDn': dn, 'cookie': api.cookie} |
361 | + self.assertThat(mock, |
362 | + MockCalledOnceWith('configResolveChildren', fields)) |
363 | + |
364 | + def test_return_response(self): |
365 | + api, mock = make_api_patch_call(self) |
366 | + mock.return_value = Element('test') |
367 | + result = api.config_resolve_children('d', 'c') |
368 | + self.assertEqual(mock.return_value, result) |
369 | + |
370 | + |
371 | +class TestConfigConfMo(MAASTestCase): |
372 | + """"Tests for ``UCSM_XML_API.config_conf_mo``.""" |
373 | + |
374 | + def test_parameters(self): |
375 | + dn = make_dn() |
376 | + config_items = [Element('hi')] |
377 | + api, mock = make_api_patch_call(self) |
378 | + api.config_conf_mo(dn, config_items) |
379 | + fields = {'dn': dn, 'cookie': api.cookie} |
380 | + self.assertThat(mock, MockCalledOnceWith('configConfMo', fields, ANY)) |
381 | + in_configs = mock.call_args[0][2] |
382 | + self.assertEqual(config_items, in_configs[0][:]) |
383 | + |
384 | + |
385 | +class TestCall(MAASTestCase): |
386 | + """"Tests for ``UCSM_XML_API._call``.""" |
387 | + |
388 | + def test_call(self): |
389 | + name = 'method' |
390 | + fields = {1: 2} |
391 | + children = [3, 4] |
392 | + request = '<yes/>' |
393 | + response = Element('good') |
394 | + api = make_api() |
395 | + |
396 | + mock_make_request_data = self.patch(ucsm, 'make_request_data') |
397 | + mock_make_request_data.return_value = request |
398 | + |
399 | + mock_send_request = self.patch(api, '_send_request') |
400 | + mock_send_request.return_value = response |
401 | + |
402 | + api._call(name, fields, children) |
403 | + self.assertThat(mock_make_request_data, |
404 | + MockCalledOnceWith(name, fields, children)) |
405 | + self.assertThat(mock_send_request, MockCalledOnceWith(request)) |
406 | + |
407 | + |
408 | +class TestSendRequest(MAASTestCase): |
409 | + """"Tests for ``UCSM_XML_API._send_request``.""" |
410 | + |
411 | + def test_send_request(self): |
412 | + request_data = 'foo' |
413 | + api = make_api() |
414 | + self.patch(api, '_call') |
415 | + stream = StringIO('<hi/>') |
416 | + mock = self.patch(urllib2, 'urlopen') |
417 | + mock.return_value = stream |
418 | + response = api._send_request(request_data) |
419 | + self.assertEqual('hi', response.tag) |
420 | + urllib_request = mock.call_args[0][0] |
421 | + self.assertEqual(request_data, urllib_request.data) |
422 | + |
423 | + |
424 | +class TestConfigResolveDn(MAASTestCase): |
425 | + """Tests for ``UCSM_XML_API.config_resolve_dn``.""" |
426 | + |
427 | + def test_parameters(self): |
428 | + api, mock = make_api_patch_call(self) |
429 | + test_dn = make_dn() |
430 | + fields = {'cookie': api.cookie, 'dn': test_dn} |
431 | + api.config_resolve_dn(test_dn) |
432 | + self.assertThat(mock, |
433 | + MockCalledOnceWith('configResolveDn', fields)) |
434 | + |
435 | + |
436 | +class TestGetServers(MAASTestCase): |
437 | + """Tests for ``get_servers``.""" |
438 | + |
439 | + def test_uses_uuid(self): |
440 | + uuid = factory.getRandomUUID() |
441 | + api = make_api() |
442 | + mock = self.patch(api, 'config_resolve_class') |
443 | + get_servers(api, uuid) |
444 | + filters = mock.call_args[0][1] |
445 | + attrib = {'class': 'computeItem', 'property': 'uuid', 'value': uuid} |
446 | + self.assertEqual(attrib, filters[0].attrib) |
447 | + |
448 | + def test_returns_result(self): |
449 | + uuid = factory.getRandomUUID() |
450 | + api = make_api() |
451 | + fake_result = make_fake_result('configResolveClass', 'found') |
452 | + self.patch(api, 'config_resolve_class').return_value = fake_result |
453 | + result = get_servers(api, uuid) |
454 | + self.assertEqual('found', result[0].tag) |
455 | + |
456 | + def test_class_id(self): |
457 | + uuid = factory.getRandomUUID() |
458 | + api = make_api() |
459 | + mock = self.patch(api, 'config_resolve_class') |
460 | + get_servers(api, uuid) |
461 | + self.assertThat(mock, MockCalledOnceWith('computeItem', ANY)) |
462 | + |
463 | + |
464 | +class TestGetChildren(MAASTestCase): |
465 | + """Tests for ``get_children``.""" |
466 | + |
467 | + def test_returns_result(self): |
468 | + search_class = make_class() |
469 | + api = make_api() |
470 | + fake_result = make_fake_result('configResolveChildren', search_class) |
471 | + self.patch(api, 'config_resolve_children').return_value = fake_result |
472 | + in_element = Element('test', {'dn': make_dn()}) |
473 | + class_id = search_class |
474 | + result = get_children(api, in_element, class_id) |
475 | + self.assertEqual(search_class, result[0].tag) |
476 | + |
477 | + def test_parameters(self): |
478 | + search_class = make_class() |
479 | + parent_dn = make_dn() |
480 | + api = make_api() |
481 | + mock = self.patch(api, 'config_resolve_children') |
482 | + in_element = Element('test', {'dn': parent_dn}) |
483 | + class_id = search_class |
484 | + get_children(api, in_element, class_id) |
485 | + self.assertThat(mock, MockCalledOnceWith(parent_dn, search_class)) |
486 | + |
487 | + |
488 | +class TestGetMacs(MAASTestCase): |
489 | + """Tests for ``get_macs``.""" |
490 | + |
491 | + def test_gets_adaptors(self): |
492 | + adaptor = 'adaptor' |
493 | + server = make_server() |
494 | + mac = 'xx' |
495 | + api = make_api() |
496 | + mock = self.patch(ucsm, 'get_children') |
497 | + |
498 | + def fake_get_children(api, element, class_id): |
499 | + if class_id == 'adaptorUnit': |
500 | + return [adaptor] |
501 | + elif class_id == 'adaptorHostEthIf': |
502 | + return [Element('ethif', {'mac': mac})] |
503 | + |
504 | + mock.side_effect = fake_get_children |
505 | + macs = get_macs(api, server) |
506 | + self.assertThat(mock, MockCallsMatch( |
507 | + call(api, server, 'adaptorUnit'), |
508 | + call(api, adaptor, 'adaptorHostEthIf'))) |
509 | + self.assertEqual([mac], macs) |
510 | + |
511 | + |
512 | +class TestProbeServers(MAASTestCase): |
513 | + """Tests for ``probe_servers``.""" |
514 | + |
515 | + def test_uses_api(self): |
516 | + api = make_api() |
517 | + mock = self.patch(ucsm, 'get_servers') |
518 | + probe_servers(api) |
519 | + self.assertThat(mock, MockCalledOnceWith(api)) |
520 | + |
521 | + def test_returns_results(self): |
522 | + servers = [{'uuid': factory.getRandomUUID()}] |
523 | + mac = 'mac' |
524 | + api = make_api() |
525 | + self.patch(ucsm, 'get_servers').return_value = servers |
526 | + self.patch(ucsm, 'get_macs').return_value = [mac] |
527 | + server_list = probe_servers(api) |
528 | + self.assertEqual([(servers[0], [mac])], server_list) |
529 | + |
530 | + |
531 | +class TestGetServerPowerControl(MAASTestCase): |
532 | + """Tests for ``get_server_power_control``.""" |
533 | + |
534 | + def test_get_server_power_control(self): |
535 | + api = make_api() |
536 | + mock = self.patch(api, 'config_resolve_children') |
537 | + fake_result = make_fake_result('configResolveChildren', 'lsPower') |
538 | + mock.return_value = fake_result |
539 | + dn = make_dn() |
540 | + server = Element('computeItem', {'assignedToDn': dn}) |
541 | + power_control = get_server_power_control(api, server) |
542 | + self.assertThat(mock, MockCalledOnceWith(dn, 'lsPower')) |
543 | + self.assertEqual('lsPower', power_control.tag) |
544 | + |
545 | + |
546 | +class TestSetServerPowerControl(MAASTestCase): |
547 | + """Tests for ``set_server_power_control``.""" |
548 | + |
549 | + def test_set_server_power_control(self): |
550 | + api = make_api() |
551 | + power_dn = make_dn() |
552 | + power_control = Element('lsPower', {'dn': power_dn}) |
553 | + config_conf_mo_mock = self.patch(api, 'config_conf_mo') |
554 | + state = 'state' |
555 | + set_server_power_control(api, power_control, state) |
556 | + self.assertThat(config_conf_mo_mock, MockCalledOnceWith(power_dn, ANY)) |
557 | + power_change = config_conf_mo_mock.call_args[0][1][0] |
558 | + self.assertEqual(power_change.tag, 'lsPower') |
559 | + self.assertEqual({'state': state, 'dn': power_dn}, power_change.attrib) |
560 | + |
561 | + |
562 | +class TestLoggedIn(MAASTestCase): |
563 | + """Tests for ``logged_in``.""" |
564 | + |
565 | + def test_logged_in(self): |
566 | + mock = self.patch(ucsm, 'UCSM_XML_API') |
567 | + url = 'url' |
568 | + username = 'username' |
569 | + password = 'password' |
570 | + mock.return_value = Mock() |
571 | + |
572 | + with logged_in(url, username, password) as api: |
573 | + self.assertEqual(mock.return_value, api) |
574 | + self.assertThat(api.login, MockCalledOnceWith()) |
575 | + |
576 | + self.assertThat(mock.return_value.logout, MockCalledOnceWith()) |
577 | + |
578 | + |
579 | +class TestValidGetPowerCommand(MAASTestCase): |
580 | + scenarios = [ |
581 | + ('Power On', dict( |
582 | + power_mode='on', current_state='down', command='admin-up')), |
583 | + ('Power On', dict( |
584 | + power_mode='on', current_state='up', command='cycle-immediate')), |
585 | + ('Power Off', dict( |
586 | + power_mode='off', current_state='up', command='admin-down')), |
587 | + ] |
588 | + |
589 | + def test_get_power_command(self): |
590 | + command = get_power_command(self.power_mode, self.current_state) |
591 | + self.assertEqual(self.command, command) |
592 | + |
593 | + |
594 | +class TestInvalidGetPowerCommand(MAASTestCase): |
595 | + |
596 | + def test_get_power_command_raises_assertion_error_on_bad_power_mode(self): |
597 | + bad_power_mode = factory.make_name('unlikely') |
598 | + error = self.assertRaises(AssertionError, get_power_command, |
599 | + bad_power_mode, None) |
600 | + self.assertIn(bad_power_mode, error.args[0]) |
601 | + |
602 | + |
603 | +class TestPowerControlUCSM(MAASTestCase): |
604 | + """Tests for ``power_control_ucsm``.""" |
605 | + |
606 | + def test_power_control_ucsm(self): |
607 | + uuid = factory.getRandomUUID() |
608 | + api = Mock() |
609 | + self.patch(ucsm, 'UCSM_XML_API').return_value = api |
610 | + get_servers_mock = self.patch(ucsm, 'get_servers') |
611 | + server = make_server() |
612 | + state = 'admin-down' |
613 | + power_control = Element('lsPower', {'state': state}) |
614 | + get_servers_mock.return_value = [server] |
615 | + get_server_power_control_mock = self.patch(ucsm, |
616 | + 'get_server_power_control') |
617 | + get_server_power_control_mock.return_value = power_control |
618 | + set_server_power_control_mock = self.patch(ucsm, |
619 | + 'set_server_power_control') |
620 | + power_control_ucsm('url', 'username', 'password', uuid, |
621 | + 'off') |
622 | + self.assertThat(get_servers_mock, MockCalledOnceWith(api, uuid)) |
623 | + self.assertThat(set_server_power_control_mock, |
624 | + MockCalledOnceWith(api, power_control, state)) |
625 | + |
626 | + |
627 | +class TestProbeAndEnlistUCSM(MAASTestCase): |
628 | + """Tests for ``probe_and_enlist_ucsm``.""" |
629 | + |
630 | + def test_probe_and_enlist(self): |
631 | + url = 'url' |
632 | + username = 'username' |
633 | + password = 'password' |
634 | + api = Mock() |
635 | + self.patch(ucsm, 'UCSM_XML_API').return_value = api |
636 | + server_element = {'uuid': 'uuid'} |
637 | + server = (server_element, ['mac'],) |
638 | + probe_servers_mock = self.patch(ucsm, 'probe_servers') |
639 | + probe_servers_mock.return_value = [server] |
640 | + set_lan_boot_default_mock = self.patch(ucsm, 'set_lan_boot_default') |
641 | + create_node_mock = self.patch(utils, 'create_node') |
642 | + probe_and_enlist_ucsm(url, username, password) |
643 | + self.assertThat(set_lan_boot_default_mock, |
644 | + MockCalledOnceWith(api, server_element)) |
645 | + self.assertThat(probe_servers_mock, MockCalledOnceWith(api)) |
646 | + params = { |
647 | + 'power_address': url, |
648 | + 'power_user': username, |
649 | + 'power_pass': password, |
650 | + 'uuid': server[0]['uuid'] |
651 | + } |
652 | + self.assertThat(create_node_mock, |
653 | + MockCalledOnceWith(server[1], 'amd64', 'ucsm', params)) |
654 | + |
655 | + |
656 | +class TestGetServiceProfile(MAASTestCase): |
657 | + """Tests for ``get_service_profile.``""" |
658 | + |
659 | + def test_get_service_profile(self): |
660 | + test_dn = make_dn() |
661 | + server = Element('computeBlade', {'assignedToDn': test_dn}) |
662 | + api = make_api() |
663 | + mock = self.patch(api, 'config_resolve_dn') |
664 | + mock.return_value = make_fake_result('configResolveDn', 'lsServer', |
665 | + 'outConfig') |
666 | + service_profile = get_service_profile(api, server) |
667 | + self.assertThat(mock, MockCalledOnceWith(test_dn)) |
668 | + self.assertEqual(mock.return_value[0], service_profile) |
669 | + |
670 | + |
671 | +def make_boot_order_scenarios(size): |
672 | + minimum = random.randint(0, 500) |
673 | + ordinals = xrange(minimum, minimum + size) |
674 | + |
675 | + elements = [ |
676 | + Element('Entry%d' % i, {'order': '%d' % i}) |
677 | + for i in ordinals |
678 | + ] |
679 | + |
680 | + orders = permutations(elements) |
681 | + orders = [{'order': order} for order in orders] |
682 | + |
683 | + scenarios = [('%d' % i, order) for i, order in enumerate(orders)] |
684 | + return scenarios, minimum |
685 | + |
686 | + |
687 | +class TestGetFirstBooter(MAASTestCase): |
688 | + """Tests for ``get_first_booter.``""" |
689 | + |
690 | + scenarios, minimum = make_boot_order_scenarios(3) |
691 | + |
692 | + def test_first_booter(self): |
693 | + root = Element('outConfigs') |
694 | + root.extend(self.order) |
695 | + picked = get_first_booter(root) |
696 | + self.assertEqual(picked.tag, 'Entry%d' % self.minimum) |
697 | + |
698 | + |
699 | +class TestsForStripRoKeys(MAASTestCase): |
700 | + """Tests for ``strip_ro_keys.``""" |
701 | + |
702 | + def test_strip_ro_keys(self): |
703 | + attributes = {key: 'DC' for key in RO_KEYS} |
704 | + |
705 | + elements = [ |
706 | + Element('Element%d' % i, attributes) |
707 | + for i in xrange(random.randint(0, 10)) |
708 | + ] |
709 | + |
710 | + strip_ro_keys(elements) |
711 | + |
712 | + for key in RO_KEYS: |
713 | + values = [element.get(key) for element in elements] |
714 | + for value in values: |
715 | + self.assertIsNone(value) |
716 | + |
717 | + |
718 | +class TestMakePolicyChange(MAASTestCase): |
719 | + """Tests for ``make_policy_change``.""" |
720 | + |
721 | + def test_lan_already_top_priority(self): |
722 | + boot_profile_response = make_fake_result('configResolveChildren', |
723 | + 'lsbootLan') |
724 | + mock = self.patch(ucsm, 'get_first_booter') |
725 | + mock.return_value = boot_profile_response[0] |
726 | + change = make_policy_change(boot_profile_response) |
727 | + self.assertIsNone(change) |
728 | + self.assertThat(mock, MockCalledOnceWith(boot_profile_response)) |
729 | + |
730 | + def test_change_lan_to_top_priority(self): |
731 | + boot_profile_response = Element('outConfigs') |
732 | + lan_boot = Element('lsbootLan', {'order': 'second'}) |
733 | + storage_boot = Element('lsbootStorage', {'order': 'first'}) |
734 | + boot_profile_response.extend([lan_boot, storage_boot]) |
735 | + self.patch(ucsm, 'get_first_booter').return_value = storage_boot |
736 | + self.patch(ucsm, 'strip_ro_keys') |
737 | + change = make_policy_change(boot_profile_response) |
738 | + lan_boot_order = change.xpath('//lsbootPolicy/lsbootLan/@order') |
739 | + storage_boot_order = \ |
740 | + change.xpath('//lsbootPolicy/lsbootStorage/@order') |
741 | + self.assertEqual(['first'], lan_boot_order) |
742 | + self.assertEqual(['second'], storage_boot_order) |
743 | + |
744 | + |
745 | +class TestSetLanBootDefault(MAASTestCase): |
746 | + """Tets for ``set_lan_boot_default.``""" |
747 | + |
748 | + def test_no_change(self): |
749 | + api = make_api() |
750 | + server = make_server() |
751 | + self.patch(ucsm, 'get_service_profile') |
752 | + self.patch(api, 'config_resolve_children') |
753 | + self.patch(ucsm, 'make_policy_change').return_value = None |
754 | + config_conf_mo = self.patch(api, 'config_conf_mo') |
755 | + set_lan_boot_default(api, server) |
756 | + self.assertThat(config_conf_mo, MockNotCalled()) |
757 | + |
758 | + def test_with_change(self): |
759 | + api = make_api() |
760 | + server = make_server() |
761 | + test_dn = make_dn() |
762 | + test_change = 'change' |
763 | + service_profile = Element('test', {'operBootPolicyName': test_dn}) |
764 | + self.patch(ucsm, 'get_service_profile').return_value = service_profile |
765 | + self.patch(api, 'config_resolve_children') |
766 | + self.patch(ucsm, 'make_policy_change').return_value = test_change |
767 | + config_conf_mo = self.patch(api, 'config_conf_mo') |
768 | + set_lan_boot_default(api, server) |
769 | + self.assertThat(config_conf_mo, |
770 | + MockCalledOnceWith(test_dn, [test_change])) |
771 | |
772 | === added file 'src/provisioningserver/custom_hardware/ucsm.py' |
773 | --- src/provisioningserver/custom_hardware/ucsm.py 1970-01-01 00:00:00 +0000 |
774 | +++ src/provisioningserver/custom_hardware/ucsm.py 2014-05-06 13:49:30 +0000 |
775 | @@ -0,0 +1,435 @@ |
776 | +# Copyright 2014 Canonical Ltd. This software is licensed under the |
777 | +# GNU Affero General Public License version 3 (see the file LICENSE). |
778 | + |
779 | +"""Support for managing nodes via Cisco UCS Manager's HTTP-XML API. |
780 | + |
781 | +It's useful to have a cursory understanding of how UCS Manager XML API |
782 | +works. Cisco has a proprietary document that describes all of this in |
783 | +more detail, and I would suggest you get a copy of that if you want more |
784 | +information than is provided here. |
785 | + |
786 | +The Cisco DevNet website for UCS Manager has a link to the document, |
787 | +which is behind a login wall, and links to example UCS queries: |
788 | + |
789 | +https://developer.cisco.com/web/unifiedcomputing/home |
790 | + |
791 | +UCS Manager is a tool for managing servers. It provides an XML API for |
792 | +external applications to use to interact with UCS Manager to manage |
793 | +servers. The API is available via HTTP, and requests and responses are |
794 | +made of XML strings. MAAS's code for interacting with a UCS Manager is |
795 | +concerned with building these requests, sending them to UCS Manager, and |
796 | +processing the responses. |
797 | + |
798 | +UCS Manager stores information in a hierarchical structure known as the |
799 | +management information tree. This structure is exposed via the XML API, |
800 | +where we can manipulate objects in the tree by finding them, reading |
801 | +them, and writing them. |
802 | + |
803 | +Some definitions for terms that are used in this code: |
804 | + |
805 | +Boot Policy - Controls the boot order for a server. Each service profile |
806 | +is associated with a boot policy. |
807 | + |
808 | +Distinguished Name (DN) - Each object in UCS has a unique DN, which |
809 | +describes its position in the tree. This is like a fully qualified path, |
810 | +and provides a way for objects to reference other objects at other |
811 | +places in the tree, or for API users to look up specific objects in the |
812 | +tree. |
813 | + |
814 | +Class - Classes define the properties and states of objects. An object's |
815 | +class is given in its tag name. |
816 | + |
817 | +Managed Object (MO) - An object in the management information tree. |
818 | +Objects are recursive, and may have children of multiple types. With the |
819 | +exception of the root object, all objects have parents. In the XML API, |
820 | +objects are represented as XML elements. |
821 | + |
822 | +Method - Actions performed by the API on managed objects. These can |
823 | +change state, or read the current state, or both. |
824 | + |
825 | +Server - A physical server managed by UCS Manager. Servers must be |
826 | +associated with service profiles in order to be used. |
827 | + |
828 | +Service Profile - A set of configuration options for a server. Service |
829 | +profiles define the server's personality, and can be migrated from |
830 | +server to server. Service profiles describe boot policy, MAC addresses, |
831 | +network connectivity, IPMI configuration, and more. MAAS requires |
832 | +servers to be associated with service profiles. |
833 | + |
834 | +UUID - The UUID for a server. MAAS persists the UUID of each UCS managed |
835 | +server it enlists, and uses it as a key for looking the server up later. |
836 | +""" |
837 | + |
838 | +from __future__ import ( |
839 | + absolute_import, |
840 | + print_function, |
841 | + unicode_literals, |
842 | + ) |
843 | + |
844 | +import contextlib |
845 | +import urllib2 |
846 | +import urlparse |
847 | + |
848 | +from lxml.etree import ( |
849 | + Element, |
850 | + tostring, |
851 | + XML, |
852 | + ) |
853 | +import provisioningserver.custom_hardware.utils as utils |
854 | + |
855 | + |
856 | +str = None |
857 | + |
858 | +__metaclass__ = type |
859 | +__all__ = [ |
860 | + 'power_control_ucsm', |
861 | + 'probe_and_enlist_ucsm', |
862 | +] |
863 | + |
864 | + |
865 | +class UCSM_XML_API_Error(Exception): |
866 | + """Failure talking to a Cisco UCS Manager.""" |
867 | + |
868 | + def __init__(self, msg, code): |
869 | + super(UCSM_XML_API_Error, self).__init__(msg) |
870 | + self.code = code |
871 | + |
872 | + |
873 | +def make_request_data(name, fields=None, children=None): |
874 | + """Build a request string for an API method.""" |
875 | + root = Element(name, fields) |
876 | + if children is not None: |
877 | + root.extend(children) |
878 | + return tostring(root) |
879 | + |
880 | + |
881 | +def parse_response(response_string): |
882 | + """Parse the response from an API method.""" |
883 | + doc = XML(response_string) |
884 | + |
885 | + error_code = doc.get('errorCode') |
886 | + if error_code is not None: |
887 | + raise UCSM_XML_API_Error(doc.get('errorDescr'), error_code) |
888 | + |
889 | + return doc |
890 | + |
891 | + |
892 | +class UCSM_XML_API(object): |
893 | + """Provides access to a Cisco UCS Manager's XML API. Public methods |
894 | + on this class correspond to UCS Manager XML API methods. |
895 | + |
896 | + Each request uses a new connection. The server supports keep-alive, |
897 | + so this client could be optimized to use it too. |
898 | + """ |
899 | + |
900 | + def __init__(self, url, username, password): |
901 | + self.url = url |
902 | + self.api_url = urlparse.urljoin(self.url, 'nuova') |
903 | + self.username = username |
904 | + self.password = password |
905 | + self.cookie = None |
906 | + |
907 | + def _send_request(self, request_data): |
908 | + """Issue a request via HTTP and parse the response.""" |
909 | + request = urllib2.Request(self.api_url, request_data) |
910 | + response = urllib2.urlopen(request) |
911 | + response_text = response.read() |
912 | + response_doc = parse_response(response_text) |
913 | + return response_doc |
914 | + |
915 | + def _call(self, name, fields=None, children=None): |
916 | + request_data = make_request_data(name, fields, children) |
917 | + response = self._send_request(request_data) |
918 | + return response |
919 | + |
920 | + def login(self): |
921 | + """Login to the API and get a cookie. |
922 | + |
923 | + Logging into the API gives a new cookie in response. The cookie |
924 | + will become inactive after it has been inactive for some amount |
925 | + of time (10 minutes is the default.) |
926 | + |
927 | + UCS Manager allows a limited number of active cookies at any |
928 | + point in time, so it's important to free the cookie up when |
929 | + finished by logging out via the ``logout`` method. |
930 | + """ |
931 | + fields = {'inName': self.username, 'inPassword': self.password} |
932 | + response = self._call('aaaLogin', fields) |
933 | + self.cookie = response.get('outCookie') |
934 | + |
935 | + def logout(self): |
936 | + """Logout from the API and free the cookie.""" |
937 | + fields = {'inCookie': self.cookie} |
938 | + self._call('aaaLogout', fields) |
939 | + self.cookie = None |
940 | + |
941 | + def config_resolve_class(self, class_id, filters=None): |
942 | + """Issue a configResolveClass request. |
943 | + |
944 | + This returns all of the objects of class ``class_id`` from the |
945 | + UCS Manager. |
946 | + |
947 | + Filters provide a way of limiting the classes returned according |
948 | + to their attributes. There are a number of filters available - |
949 | + Cisco's XML API documentation has a full chapter on filters. |
950 | + All we care about here is that filters are described with XML |
951 | + elements. |
952 | + """ |
953 | + fields = {'cookie': self.cookie, 'classId': class_id} |
954 | + |
955 | + in_filters = Element('inFilter') |
956 | + if filters: |
957 | + in_filters.extend(filters) |
958 | + |
959 | + return self._call('configResolveClass', fields, [in_filters]) |
960 | + |
961 | + def config_resolve_children(self, dn, class_id=None): |
962 | + """Issue a configResolveChildren request. |
963 | + |
964 | + This returns all of the children of the object named by ``dn``, |
965 | + or if ``class_id`` is not None, all of the children of type |
966 | + ``class_id``. |
967 | + """ |
968 | + fields = {'cookie': self.cookie, 'inDn': dn} |
969 | + if class_id is not None: |
970 | + fields['classId'] = class_id |
971 | + return self._call('configResolveChildren', fields) |
972 | + |
973 | + def config_resolve_dn(self, dn): |
974 | + """Retrieve a single object by name. |
975 | + |
976 | + This returns the object named by ``dn``, but not its children. |
977 | + """ |
978 | + fields = {'cookie': self.cookie, 'dn': dn} |
979 | + return self._call('configResolveDn', fields) |
980 | + |
981 | + def config_conf_mo(self, dn, config_items): |
982 | + """Issue a configConfMo request. |
983 | + |
984 | + This makes a configuration change on an object (MO). |
985 | + """ |
986 | + fields = {'cookie': self.cookie, 'dn': dn} |
987 | + |
988 | + in_configs = Element('inConfig') |
989 | + in_configs.extend(config_items) |
990 | + |
991 | + self._call('configConfMo', fields, [in_configs]) |
992 | + |
993 | + |
994 | +def get_servers(api, uuid=None): |
995 | + """Retrieve a list of servers from the UCS Manager.""" |
996 | + if uuid: |
997 | + attrs = {'class': 'computeItem', 'property': 'uuid', 'value': uuid} |
998 | + filters = [Element('eq', attrs)] |
999 | + else: |
1000 | + filters = None |
1001 | + |
1002 | + resolved = api.config_resolve_class('computeItem', filters) |
1003 | + return resolved.xpath('//outConfigs/*') |
1004 | + |
1005 | + |
1006 | +def get_children(api, element, class_id): |
1007 | + """Retrieve a list of child elements from the UCS Manager.""" |
1008 | + resolved = api.config_resolve_children(element.get('dn'), class_id) |
1009 | + return resolved.xpath('//outConfigs/%s' % class_id) |
1010 | + |
1011 | + |
1012 | +def get_macs(api, server): |
1013 | + """Retrieve the list of MAC addresses assigned to a server. |
1014 | + |
1015 | + Network interfaces are represented by 'adaptorUnit' objects, and |
1016 | + are stored as children of servers. |
1017 | + """ |
1018 | + adaptors = get_children(api, server, 'adaptorUnit') |
1019 | + |
1020 | + macs = [] |
1021 | + for adaptor in adaptors: |
1022 | + host_eth_ifs = get_children(api, adaptor, 'adaptorHostEthIf') |
1023 | + macs.extend([h.get('mac') for h in host_eth_ifs]) |
1024 | + |
1025 | + return macs |
1026 | + |
1027 | + |
1028 | +def probe_servers(api): |
1029 | + """Retrieve the UUID and MAC addresses for servers from the UCS Manager.""" |
1030 | + servers = get_servers(api) |
1031 | + server_list = [(s, get_macs(api, s)) for s in servers] |
1032 | + return server_list |
1033 | + |
1034 | + |
1035 | +def get_server_power_control(api, server): |
1036 | + """Retrieve the power control object for a server.""" |
1037 | + service_profile_dn = server.get('assignedToDn') |
1038 | + resolved = api.config_resolve_children(service_profile_dn, 'lsPower') |
1039 | + power_controls = resolved.xpath('//outConfigs/lsPower') |
1040 | + return power_controls[0] |
1041 | + |
1042 | + |
1043 | +def set_server_power_control(api, power_control, command): |
1044 | + """Issue a power command to a server's power control.""" |
1045 | + attrs = {'state': command, 'dn': power_control.get('dn')} |
1046 | + power_change = Element('lsPower', attrs) |
1047 | + api.config_conf_mo(power_control.get('dn'), [power_change]) |
1048 | + |
1049 | + |
1050 | +def get_service_profile(api, server): |
1051 | + """Get the server's assigned service profile.""" |
1052 | + service_profile_dn = server.get('assignedToDn') |
1053 | + result = api.config_resolve_dn(service_profile_dn) |
1054 | + service_profile = result.xpath('//outConfig/lsServer')[0] |
1055 | + return service_profile |
1056 | + |
1057 | + |
1058 | +def get_first_booter(boot_profile_response): |
1059 | + """Find the device currently set to boot by default.""" |
1060 | + ordinals = boot_profile_response.xpath('//outConfigs/*/@order') |
1061 | + top_boot_order = min(ordinals) |
1062 | + first_query = '//outConfigs/*[@order=%s]' % top_boot_order |
1063 | + current_first = boot_profile_response.xpath(first_query)[0] |
1064 | + return current_first |
1065 | + |
1066 | + |
1067 | +RO_KEYS = ['access', 'type'] |
1068 | + |
1069 | + |
1070 | +def strip_ro_keys(elements): |
1071 | + """Remove read-only keys from configuration elements. |
1072 | + |
1073 | + These are keys for attributes that aren't allowed to be changed via |
1074 | + configConfMo request. They are included in MO's that we read from the |
1075 | + API; stripping these attributes lets us reuse the elements for those |
1076 | + MO's rather than building new ones from scratch. |
1077 | + """ |
1078 | + for ro_key in RO_KEYS: |
1079 | + for element in elements: |
1080 | + del(element.attrib[ro_key]) |
1081 | + |
1082 | + |
1083 | +def make_policy_change(boot_profile_response): |
1084 | + """Build the policy change tree required to make LAN boot first |
1085 | + priority. |
1086 | + |
1087 | + The original top priority will be swapped with LAN boot's original |
1088 | + priority. |
1089 | + """ |
1090 | + current_first = get_first_booter(boot_profile_response) |
1091 | + lan_boot = boot_profile_response.xpath('//outConfigs/lsbootLan')[0] |
1092 | + |
1093 | + if current_first == lan_boot: |
1094 | + return |
1095 | + |
1096 | + top_boot_order = current_first.get('order') |
1097 | + current_first.set('order', lan_boot.get('order')) |
1098 | + lan_boot.set('order', top_boot_order) |
1099 | + |
1100 | + elements = [current_first, lan_boot] |
1101 | + strip_ro_keys(elements) |
1102 | + policy_change = Element('lsbootPolicy') |
1103 | + policy_change.extend(elements) |
1104 | + return policy_change |
1105 | + |
1106 | + |
1107 | +def set_lan_boot_default(api, server): |
1108 | + """Set a server to boot via LAN by default. |
1109 | + |
1110 | + If LAN boot is already the top priority, no change will |
1111 | + be made. |
1112 | + |
1113 | + This command changes the server's boot profile, which will affect |
1114 | + any other servers also using that boot profile. This is ok, because |
1115 | + probe and enlist enlists all the servers in the chassis. |
1116 | + """ |
1117 | + service_profile = get_service_profile(api, server) |
1118 | + boot_profile_dn = service_profile.get('operBootPolicyName') |
1119 | + response = api.config_resolve_children(boot_profile_dn) |
1120 | + policy_change = make_policy_change(response) |
1121 | + if policy_change is None: |
1122 | + return |
1123 | + api.config_conf_mo(boot_profile_dn, [policy_change]) |
1124 | + |
1125 | + |
1126 | +@contextlib.contextmanager |
1127 | +def logged_in(url, username, password): |
1128 | + """Context manager that ensures the logout from the API occurs.""" |
1129 | + api = UCSM_XML_API(url, username, password) |
1130 | + api.login() |
1131 | + try: |
1132 | + yield api |
1133 | + finally: |
1134 | + api.logout() |
1135 | + |
1136 | + |
1137 | +def get_power_command(maas_power_mode, current_state): |
1138 | + """Translate a MAAS on/off state into a UCSM power command. |
1139 | + |
1140 | + If the node is up already and receives a request to power on, power |
1141 | + cycle the node. |
1142 | + """ |
1143 | + if maas_power_mode == 'on': |
1144 | + if current_state == 'up': |
1145 | + return 'cycle-immediate' |
1146 | + return 'admin-up' |
1147 | + elif maas_power_mode == 'off': |
1148 | + return 'admin-down' |
1149 | + else: |
1150 | + message = 'Unexpected maas power mode: %s' % (maas_power_mode) |
1151 | + raise AssertionError(message) |
1152 | + |
1153 | + |
1154 | +def power_control_ucsm(url, username, password, uuid, maas_power_mode): |
1155 | + """Handle calls from the power template for nodes with a power type |
1156 | + of 'ucsm'. |
1157 | + """ |
1158 | + with logged_in(url, username, password) as api: |
1159 | + # UUIDs are unique per server, so we get either one or zero |
1160 | + # servers for a given UUID. |
1161 | + [server] = get_servers(api, uuid) |
1162 | + power_control = get_server_power_control(api, server) |
1163 | + command = get_power_command(maas_power_mode, |
1164 | + power_control.get('state')) |
1165 | + set_server_power_control(api, power_control, command) |
1166 | + |
1167 | + |
1168 | +def probe_and_enlist_ucsm(url, username, password): |
1169 | + """Probe a UCS Manager and enlist all its servers. |
1170 | + |
1171 | + Here's what happens here: 1. Get a list of servers from the UCS |
1172 | + Manager, along with their MAC addresses. |
1173 | + |
1174 | + 2. Configure each server to boot from LAN first. |
1175 | + |
1176 | + 3. Add each server to MAAS as a new node, with a power control |
1177 | + method of 'ucsm'. The URL and credentials supplied are persisted |
1178 | + with each node so MAAS knows how to access UCSM to manage the node |
1179 | + in the future. |
1180 | + |
1181 | + This code expects each server in the system to have already been |
1182 | + associated with a service profile. The servers must have networking |
1183 | + configured, and their boot profiles must include a boot from LAN |
1184 | + option. During enlistment, the boot profile for each service profile |
1185 | + used by a server will be modified to move LAN boot to the highest |
1186 | + priority boot option. |
1187 | + |
1188 | + Also, if any node fails to enlist, this enlistment process will |
1189 | + stop and won't attempt to enlist any additional nodes. If a node is |
1190 | + already known to MAAS, it will fail to enlist, so all nodes must be |
1191 | + added at once. |
1192 | + |
1193 | + There is also room for optimization during enlistment. While our |
1194 | + client deals with a single server at a time, the API is capable |
1195 | + of reading/writing the settings of multiple servers in the same |
1196 | + request. |
1197 | + """ |
1198 | + with logged_in(url, username, password) as api: |
1199 | + servers = probe_servers(api) |
1200 | + for server, _ in servers: |
1201 | + set_lan_boot_default(api, server) |
1202 | + |
1203 | + for server, macs in servers: |
1204 | + params = { |
1205 | + 'power_address': url, |
1206 | + 'power_user': username, |
1207 | + 'power_pass': password, |
1208 | + 'uuid': server.get('uuid'), |
1209 | + } |
1210 | + utils.create_node(macs, 'amd64', 'ucsm', params) |
1211 | |
1212 | === modified file 'src/provisioningserver/power/tests/test_poweraction.py' |
1213 | --- src/provisioningserver/power/tests/test_poweraction.py 2014-02-27 02:33:54 +0000 |
1214 | +++ src/provisioningserver/power/tests/test_poweraction.py 2014-05-06 13:49:30 +0000 |
1215 | @@ -223,3 +223,14 @@ |
1216 | power_pass='me', power_hwaddress='me', ipmitool='echo') |
1217 | output = action.run_shell(script) |
1218 | self.assertIn("Got unknown power state from ipmipower", output) |
1219 | + |
1220 | + def test_ucsm_renders_template(self): |
1221 | + # I'd like to assert that escape_py_literal is being used here, |
1222 | + # but it's not obvious how to mock things in the template |
1223 | + # rendering namespace so I passed on that. |
1224 | + action = PowerAction('ucsm') |
1225 | + script = action.render_template( |
1226 | + action.get_template(), power_address='foo', |
1227 | + power_user='bar', power_pass='baz', |
1228 | + uuid=factory.getRandomUUID(), power_change='on') |
1229 | + self.assertIn('power_control_ucsm', script) |
1230 | |
1231 | === modified file 'src/provisioningserver/power_schema.py' |
1232 | --- src/provisioningserver/power_schema.py 2014-04-23 14:21:03 +0000 |
1233 | +++ src/provisioningserver/power_schema.py 2014-05-06 13:49:30 +0000 |
1234 | @@ -242,4 +242,14 @@ |
1235 | make_json_field('power_pass', "Power password"), |
1236 | ], |
1237 | }, |
1238 | + { |
1239 | + 'name': 'ucsm', |
1240 | + 'description': "Cisco UCS Manager", |
1241 | + 'fields': [ |
1242 | + make_json_field('uuid', "Server UUID"), |
1243 | + make_json_field('power_address', "URL for XML API"), |
1244 | + make_json_field('power_user', "API user"), |
1245 | + make_json_field('power_pass', "API password"), |
1246 | + ], |
1247 | + }, |
1248 | ] |
1249 | |
1250 | === modified file 'src/provisioningserver/tasks.py' |
1251 | --- src/provisioningserver/tasks.py 2014-04-22 15:10:32 +0000 |
1252 | +++ src/provisioningserver/tasks.py 2014-05-06 13:49:30 +0000 |
1253 | @@ -45,6 +45,7 @@ |
1254 | from provisioningserver.custom_hardware.seamicro import ( |
1255 | probe_seamicro15k_and_enlist, |
1256 | ) |
1257 | +from provisioningserver.custom_hardware.ucsm import probe_and_enlist_ucsm |
1258 | from provisioningserver.dhcp import ( |
1259 | config, |
1260 | detect, |
1261 | @@ -477,3 +478,10 @@ |
1262 | power_control=power_control) |
1263 | else: |
1264 | logger.warning("Couldn't find IP address for MAC %s" % mac) |
1265 | + |
1266 | + |
1267 | +@task |
1268 | +@log_exception_text |
1269 | +def enlist_nodes_from_ucsm(url, username, password): |
1270 | + """ See `maasserver.api.NodeGroupsHandler.enlist_nodes_from_ucsm`. """ |
1271 | + probe_and_enlist_ucsm(url, username, password) |
1272 | |
1273 | === modified file 'src/provisioningserver/tests/test_tasks.py' |
1274 | --- src/provisioningserver/tests/test_tasks.py 2014-04-22 15:10:32 +0000 |
1275 | +++ src/provisioningserver/tests/test_tasks.py 2014-05-06 13:49:30 +0000 |
1276 | @@ -72,6 +72,7 @@ |
1277 | from provisioningserver.tags import MissingCredentials |
1278 | from provisioningserver.tasks import ( |
1279 | add_new_dhcp_host_map, |
1280 | + enlist_nodes_from_ucsm, |
1281 | import_boot_images, |
1282 | Omshell, |
1283 | power_off, |
1284 | @@ -676,3 +677,14 @@ |
1285 | mock_callback = Mock() |
1286 | import_boot_images(callback=mock_callback) |
1287 | self.assertThat(mock_callback.delay, MockCalledOnceWith()) |
1288 | + |
1289 | + |
1290 | +class TestAddUCSM(PservTestCase): |
1291 | + |
1292 | + def test_enlist_nodes_from_ucsm(self): |
1293 | + url = 'url' |
1294 | + username = 'username' |
1295 | + password = 'password' |
1296 | + mock = self.patch(tasks, 'probe_and_enlist_ucsm') |
1297 | + enlist_nodes_from_ucsm(url, username, password) |
1298 | + self.assertThat(mock, MockCalledOnceWith(url, username, password)) |
1299 | |
1300 | === modified file 'src/provisioningserver/utils/__init__.py' |
1301 | --- src/provisioningserver/utils/__init__.py 2014-04-21 23:20:49 +0000 |
1302 | +++ src/provisioningserver/utils/__init__.py 2014-05-06 13:49:30 +0000 |
1303 | @@ -512,6 +512,11 @@ |
1304 | self.__class__.__name__, self.value) |
1305 | |
1306 | |
1307 | +def escape_py_literal(string): |
1308 | + """Escape and quote a string for use as a python literal.""" |
1309 | + return repr(string).decode('ascii') |
1310 | + |
1311 | + |
1312 | class ShellTemplate(tempita.Template): |
1313 | """A Tempita template specialised for writing shell scripts. |
1314 | |
1315 | |
1316 | === modified file 'src/provisioningserver/utils/tests/test_utils.py' |
1317 | --- src/provisioningserver/utils/tests/test_utils.py 2014-04-24 08:51:41 +0000 |
1318 | +++ src/provisioningserver/utils/tests/test_utils.py 2014-05-06 13:49:30 +0000 |
1319 | @@ -82,6 +82,7 @@ |
1320 | MainScript, |
1321 | parse_key_value_file, |
1322 | pick_new_mtime, |
1323 | + escape_py_literal, |
1324 | read_text_file, |
1325 | Safe, |
1326 | ShellTemplate, |
1327 | @@ -1393,3 +1394,22 @@ |
1328 | # modification. The arguments passed back match those passed in |
1329 | # from do_stuff_in_thread(). |
1330 | self.assertEqual(((3, 4), {"five": 5}), result) |
1331 | + |
1332 | + |
1333 | +class TestQuotePyLiteral(MAASTestCase): |
1334 | + def test_uses_repr(self): |
1335 | + string = factory.make_name('string') |
1336 | + repr_mock = self.patch(provisioningserver.utils, 'repr') |
1337 | + escape_py_literal(string) |
1338 | + self.assertThat(repr_mock, MockCalledOnceWith(string)) |
1339 | + |
1340 | + def test_decodes_ascii(self): |
1341 | + string = factory.make_name('string') |
1342 | + output = factory.make_name('output') |
1343 | + repr_mock = self.patch(provisioningserver.utils, 'repr') |
1344 | + ascii_value = Mock() |
1345 | + ascii_value.decode = Mock(return_value=output) |
1346 | + repr_mock.return_value = ascii_value |
1347 | + value = escape_py_literal(string) |
1348 | + self.assertThat(ascii_value.decode, MockCalledOnceWith('ascii')) |
1349 | + self.assertEqual(value, output) |
1 === added file 'etc/maas/ templates/ power/ucsm. template' templates/ power/ucsm. template 1970-01-01 00:00:00 +0000 templates/ power/ucsm. template 2014-04-29 13:55:35 +0000 ver.custom_ hardware. ucsm import power_control_ucsm control_ ucsm('{ {power_ address} }', '{{power_user}}', '{{power_pass}}', '{{uuid}}', '{{power_change}}')
2 --- etc/maas/
3 +++ etc/maas/
4 @@ -0,0 +1,8 @@
5 +# -*- mode: shell-script -*-
6 +#
7 +# Control a system via Cisco UCS Manager XML API.
8 +
9 +python - << END
10 +from provisioningser
11 +power_
12 +END
This is "hope quoting" (as Gavin calls it) and ripe for abuse with unchecked inputs. See the conversation in https:/ /code.launchpad .net/~blake- rouse/maas/ virsh-probe- and-enlist/ +merge/ 216632