Merge lp:~jameinel/maas/cpu_mem_tag_constraints into lp:~maas-committers/maas/trunk
- cpu_mem_tag_constraints
- Merge into trunk
Status: | Merged |
---|---|
Merged at revision: | 1102 |
Proposed branch: | lp:~jameinel/maas/cpu_mem_tag_constraints |
Merge into: | lp:~maas-committers/maas/trunk |
Diff against target: |
492 lines (+330/-38) 8 files modified
src/maasserver/api.py (+17/-4) src/maasserver/exceptions.py (+14/-0) src/maasserver/models/node.py (+4/-15) src/maasserver/models/node_constraint_filter.py (+77/-0) src/maasserver/tests/test_api.py (+80/-0) src/maasserver/tests/test_exceptions.py (+19/-0) src/maasserver/tests/test_node.py (+17/-19) src/maasserver/tests/test_node_constraint_filter.py (+102/-0) |
To merge this branch: | bzr merge lp:~jameinel/maas/cpu_mem_tag_constraints |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Jeroen T. Vermeulen (community) | Approve | ||
Review via email: mp+126863@code.launchpad.net |
Commit message
Add CPU count, Memory and Tags as exposed logic for get_available_
Description of the change
This is my version of gz`s branch: https:/
Things were getting big enough that I evolved a helper class, which I think cleans things up and makes it easier to add more constraints in the future.
It also makes it more obvious to directly test that class.
And I changed the tag lookup logic, so it now gives InvalidConstraint rather than 404.
I *think* this helper class will even let us do the same constraints for a search interface, though we'll probably want to name it differently.
Jeroen T. Vermeulen (jtv) wrote : | # |
Jeroen T. Vermeulen (jtv) wrote : | # |
OH THANK YOU VERY MUCH QUANTAL for submitting my notes before I was done entering them.
Jeroen T. Vermeulen (jtv) wrote : | # |
I see that the API acceptance tests cover a lot of the same behaviours as the unit tests (and then there is some redundancy between unit tests as well). My guess is that the acceptance tests were written first, then the lower-level helpers were created to satisfy the tests, and those helpers were then unit-tested. I do that a lot as well, but I find it's usually best to strip down the acceptance tests to exercise integration paths only.
John A Meinel (jameinel) wrote : | # |
A lot of the 'test_api' level tests are there to test the actual HTTP response codes wrt different errors. (So have 1 success level and 1 failure level test for each parameter based on different error codes.)
So I think the test coverage is reasonable as it stands, but we did switch things to using a module.
Jeroen T. Vermeulen (jtv) wrote : | # |
Thanks for making the changes. I know I'm difficult.
MAAS Lander (maas-lander) wrote : | # |
The Jenkins job https:/
Not merging it.
Martin Packman (gz) wrote : | # |
Fixed the test failure back in the branch under my namespace and landed it.
Jeroen T. Vermeulen (jtv) wrote : | # |
Please, *always* run "make lint" and the full test suite before putting a
Jeroen T. Vermeulen (jtv) wrote : | # |
branch up for review!
(Thank you Quantal for moving my mouse pointer to the Save Comment button and then clicking it while I was typing)
John A Meinel (jameinel) wrote : | # |
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On 9/29/2012 8:07 AM, Jeroen T. Vermeulen wrote:
> branch up for review!
>
> (Thank you Quantal for moving my mouse pointer to the Save Comment
> button and then clicking it while I was typing)
>
Unfortunately, it looks like upgrading to Quantal broke my 'make lint':
ImportError: ..python2.
_PyNode_Sizeof.
That seems to be from 'compiler' which means my py2.7 install is now
broken?
I have tried to make sure to run it. However, if we are going to be
pedantic about it, shouldn't we just have tarmac run it?
John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (Cygwin)
Comment: Using GnuPG with Mozilla - http://
iEYEARECAAYFAlB
OcsAnRWwmcSWGVs
=2dua
-----END PGP SIGNATURE-----
Jeroen T. Vermeulen (jtv) wrote : | # |
That's been known to happen, but "make distclean" seems to fix it.
Having tarmac run it would be a good idea IMO.
Preview Diff
1 | === modified file 'src/maasserver/api.py' |
2 | --- src/maasserver/api.py 2012-09-28 09:45:18 +0000 |
3 | +++ src/maasserver/api.py 2012-09-28 11:10:27 +0000 |
4 | @@ -643,6 +643,17 @@ |
5 | return ('nodes_handler', []) |
6 | |
7 | |
8 | +# Map the constraint as passed in the request to the name of the constraint in |
9 | +# the database. Many of them have the same name |
10 | +_constraints_map = { |
11 | + 'name': 'hostname', |
12 | + 'tags': 'tags', |
13 | + 'arch': 'architecture', |
14 | + 'cpu_count': 'cpu_count', |
15 | + 'mem': 'memory', |
16 | + } |
17 | + |
18 | + |
19 | def extract_constraints(request_params): |
20 | """Extract a dict of node allocation constraints from http parameters. |
21 | |
22 | @@ -651,10 +662,12 @@ |
23 | :return: A mapping of applicable constraint names to their values. |
24 | :rtype: :class:`dict` |
25 | """ |
26 | - supported_constraints = ('name', 'arch') |
27 | - return {constraint: request_params[constraint] |
28 | - for constraint in supported_constraints |
29 | - if constraint in request_params} |
30 | + constraints = {} |
31 | + for request_name in _constraints_map: |
32 | + if request_name in request_params: |
33 | + db_name = _constraints_map[request_name] |
34 | + constraints[db_name] = request_params[request_name] |
35 | + return constraints |
36 | |
37 | |
38 | @api_operations |
39 | |
40 | === modified file 'src/maasserver/exceptions.py' |
41 | --- src/maasserver/exceptions.py 2012-08-21 20:14:12 +0000 |
42 | +++ src/maasserver/exceptions.py 2012-09-28 11:10:27 +0000 |
43 | @@ -84,6 +84,20 @@ |
44 | api_error = httplib.CONFLICT |
45 | |
46 | |
47 | +class InvalidConstraint(MAASAPIBadRequest): |
48 | + """Node allocation constraint given cannot be interpreted.""" |
49 | + |
50 | + def __init__(self, constraint, value, err=None): |
51 | + super(InvalidConstraint, self).__init__(constraint, value) |
52 | + self.err = err |
53 | + |
54 | + def __str__(self): |
55 | + s = "Invalid '%s' constraint '%s'" % self.args |
56 | + if self.err: |
57 | + return "%s: %s" % (s, str(self.err)) |
58 | + return s |
59 | + |
60 | + |
61 | class Redirect(MAASAPIException): |
62 | """Redirect. The exception message is the target URL.""" |
63 | api_error = httplib.FOUND |
64 | |
65 | === modified file 'src/maasserver/models/node.py' |
66 | --- src/maasserver/models/node.py 2012-09-26 08:54:29 +0000 |
67 | +++ src/maasserver/models/node.py 2012-09-28 11:10:27 +0000 |
68 | @@ -250,21 +250,10 @@ |
69 | :type constraints: :class:`dict` |
70 | :return: A matching `Node`, or None if none are available. |
71 | """ |
72 | - if constraints is None: |
73 | - constraints = {} |
74 | - available_nodes = ( |
75 | - self.get_nodes(for_user, NODE_PERMISSION.VIEW) |
76 | - .filter(status=NODE_STATUS.READY)) |
77 | - |
78 | - if constraints.get('name'): |
79 | - available_nodes = available_nodes.filter( |
80 | - hostname=constraints['name']) |
81 | - if constraints.get('arch'): |
82 | - # GZ 2012-09-11: This only supports an exact match on arch type, |
83 | - # using an i386 image on amd64 hardware will wait. |
84 | - available_nodes = available_nodes.filter( |
85 | - architecture=constraints['arch']) |
86 | - |
87 | + from maasserver.models.node_constraint_filter import constrain_nodes |
88 | + available_nodes = self.get_nodes(for_user, NODE_PERMISSION.VIEW) |
89 | + available_nodes = available_nodes.filter(status=NODE_STATUS.READY) |
90 | + available_nodes = constrain_nodes(available_nodes, constraints) |
91 | return get_first(available_nodes) |
92 | |
93 | def stop_nodes(self, ids, by_user): |
94 | |
95 | === added file 'src/maasserver/models/node_constraint_filter.py' |
96 | --- src/maasserver/models/node_constraint_filter.py 1970-01-01 00:00:00 +0000 |
97 | +++ src/maasserver/models/node_constraint_filter.py 2012-09-28 11:10:27 +0000 |
98 | @@ -0,0 +1,77 @@ |
99 | +# Copyright 2012 Canonical Ltd. This software is licensed under the |
100 | +# GNU Affero General Public License version 3 (see the file LICENSE). |
101 | + |
102 | +from __future__ import ( |
103 | + absolute_import, |
104 | + print_function, |
105 | + unicode_literals, |
106 | + ) |
107 | + |
108 | +__metaclass__ = type |
109 | +__all__ = [ |
110 | + 'constrain_nodes', |
111 | + ] |
112 | + |
113 | +from maasserver.exceptions import ( |
114 | + InvalidConstraint, |
115 | + ) |
116 | +from maasserver.models import Tag |
117 | +from maasserver.utils.orm import get_one |
118 | + |
119 | + |
120 | +def constrain_identical(nodes, key, value): |
121 | + """Match the field 'key' to exactly match 'value'""" |
122 | + return nodes.filter(**{key: value}) |
123 | + |
124 | + |
125 | +def constrain_int_greater_or_equal(nodes, key, str_value): |
126 | + """Filter nodes that have value >= supplied value. |
127 | + |
128 | + :param str_value: Will be cast to an integer, if this fails, it is treated |
129 | + as an invalid constraint. |
130 | + """ |
131 | + try: |
132 | + int_value = int(str_value) |
133 | + except ValueError as e: |
134 | + raise InvalidConstraint(key, str_value, e) |
135 | + return nodes.filter(**{'%s__gte' % (key,): int_value}) |
136 | + |
137 | + |
138 | +def constrain_tags(nodes, key, tag_expression): |
139 | + """Tags match: restrict to nodes that have all tags.""" |
140 | + # We use ',' separated or space ' ' separated values. |
141 | + tag_names = tag_expression.replace(",", " ").strip().split() |
142 | + for tag_name in tag_names: |
143 | + tag = get_one(Tag.objects.filter(name=tag_name)) |
144 | + if tag is None: |
145 | + raise InvalidConstraint('tags', tag_name, 'No such tag') |
146 | + nodes = nodes.filter(tags=tag) |
147 | + return nodes |
148 | + |
149 | + |
150 | +# this is the mapping of constraint names to how to apply the constraint |
151 | +constraint_filters = { |
152 | + # Currently architecture only supports exact matching. Eventually, we will |
153 | + # probably want more logic to note that eg, amd64 can be used for an i386 |
154 | + # request |
155 | + 'architecture': constrain_identical, |
156 | + 'hostname': constrain_identical, |
157 | + 'cpu_count': constrain_int_greater_or_equal, |
158 | + 'memory': constrain_int_greater_or_equal, |
159 | + 'tags': constrain_tags, |
160 | +} |
161 | + |
162 | + |
163 | +def constrain_nodes(nodes, constraints): |
164 | + """Apply the dict of constraints as filters against nodes. |
165 | + |
166 | + :param nodes: A QuerySet against nodes |
167 | + :param constraints: A dict of {'constraint': 'value'} |
168 | + """ |
169 | + if not constraints: |
170 | + return nodes |
171 | + for constraint_name, value in constraints.items(): |
172 | + filter = constraint_filters.get(constraint_name) |
173 | + if filter is not None: |
174 | + nodes = filter(nodes, constraint_name, value) |
175 | + return nodes |
176 | |
177 | === modified file 'src/maasserver/tests/test_api.py' |
178 | --- src/maasserver/tests/test_api.py 2012-09-28 07:31:54 +0000 |
179 | +++ src/maasserver/tests/test_api.py 2012-09-28 11:10:27 +0000 |
180 | @@ -829,6 +829,11 @@ |
181 | self.logged_in_user.is_superuser = True |
182 | self.logged_in_user.save() |
183 | |
184 | + def assertResponseCode(self, expected_code, response): |
185 | + if response.status_code != expected_code: |
186 | + self.fail("Expected %s response, got %s:\n%s" % ( |
187 | + expected_code, response.status_code, response.content)) |
188 | + |
189 | |
190 | def extract_system_ids(parsed_result): |
191 | """List the system_ids of the nodes in `parsed_result`.""" |
192 | @@ -1713,6 +1718,81 @@ |
193 | }) |
194 | self.assertEqual(httplib.CONFLICT, response.status_code) |
195 | |
196 | + def test_POST_acquire_allocates_node_by_cpu(self): |
197 | + # Asking for enough cpu acquires a node with at least that. |
198 | + node = factory.make_node(status=NODE_STATUS.READY, cpu_count=3) |
199 | + response = self.client.post(self.get_uri('nodes/'), { |
200 | + 'op': 'acquire', |
201 | + 'cpu_count': 2, |
202 | + }) |
203 | + self.assertResponseCode(httplib.OK, response) |
204 | + response_json = json.loads(response.content) |
205 | + self.assertEqual(node.system_id, response_json['system_id']) |
206 | + |
207 | + def test_POST_acquire_fails_with_invalid_cpu(self): |
208 | + # Asking for an invalid amount of cpu returns a bad request. |
209 | + factory.make_node(status=NODE_STATUS.READY) |
210 | + response = self.client.post(self.get_uri('nodes/'), { |
211 | + 'op': 'acquire', |
212 | + 'cpu_count': 'plenty', |
213 | + }) |
214 | + self.assertResponseCode(httplib.BAD_REQUEST, response) |
215 | + |
216 | + def test_POST_acquire_allocates_node_by_mem(self): |
217 | + # Asking for enough memory acquires a node with at least that. |
218 | + node = factory.make_node(status=NODE_STATUS.READY, memory=1024) |
219 | + response = self.client.post(self.get_uri('nodes/'), { |
220 | + 'op': 'acquire', |
221 | + 'mem': 1024, |
222 | + }) |
223 | + self.assertResponseCode(httplib.OK, response) |
224 | + response_json = json.loads(response.content) |
225 | + self.assertEqual(node.system_id, response_json['system_id']) |
226 | + |
227 | + def test_POST_acquire_fails_with_invalid_mem(self): |
228 | + # Asking for an invalid amount of cpu returns a bad request. |
229 | + factory.make_node(status=NODE_STATUS.READY) |
230 | + response = self.client.post(self.get_uri('nodes/'), { |
231 | + 'op': 'acquire', |
232 | + 'mem': 'bags', |
233 | + }) |
234 | + self.assertResponseCode(httplib.BAD_REQUEST, response) |
235 | + |
236 | + def test_POST_acquire_allocates_node_by_tags(self): |
237 | + # Asking for particular tags acquires a node with those tags. |
238 | + node = factory.make_node(status=NODE_STATUS.READY) |
239 | + node_tag_names = ["fast", "stable", "cute"] |
240 | + node.tags = [factory.make_tag(t) for t in node_tag_names] |
241 | + response = self.client.post(self.get_uri('nodes/'), { |
242 | + 'op': 'acquire', |
243 | + 'tags': 'fast, stable', |
244 | + }) |
245 | + self.assertResponseCode(httplib.OK, response) |
246 | + response_json = json.loads(response.content) |
247 | + self.assertEqual(node_tag_names, response_json['tag_names']) |
248 | + |
249 | + def test_POST_acquire_fails_without_all_tags(self): |
250 | + # Asking for particular tags does not acquire if no node has all tags. |
251 | + node1 = factory.make_node(status=NODE_STATUS.READY) |
252 | + node1.tags = [factory.make_tag(t) for t in ("fast", "stable", "cute")] |
253 | + node2 = factory.make_node(status=NODE_STATUS.READY) |
254 | + node2.tags = [factory.make_tag("cheap")] |
255 | + response = self.client.post(self.get_uri('nodes/'), { |
256 | + 'op': 'acquire', |
257 | + 'tags': 'fast, cheap', |
258 | + }) |
259 | + self.assertResponseCode(httplib.CONFLICT, response) |
260 | + |
261 | + def test_POST_acquire_fails_with_unknown_tags(self): |
262 | + # Asking for a tag that does not exist gives a specific error. |
263 | + node = factory.make_node(status=NODE_STATUS.READY) |
264 | + node.tags = [factory.make_tag("fast")] |
265 | + response = self.client.post(self.get_uri('nodes/'), { |
266 | + 'op': 'acquire', |
267 | + 'tags': 'fast, hairy', |
268 | + }) |
269 | + self.assertResponseCode(httplib.BAD_REQUEST, response) |
270 | + |
271 | def test_POST_acquire_sets_a_token(self): |
272 | # "acquire" should set the Token being used in the request on |
273 | # the Node that is allocated. |
274 | |
275 | === modified file 'src/maasserver/tests/test_exceptions.py' |
276 | --- src/maasserver/tests/test_exceptions.py 2012-04-27 12:38:18 +0000 |
277 | +++ src/maasserver/tests/test_exceptions.py 2012-09-28 11:10:27 +0000 |
278 | @@ -15,6 +15,7 @@ |
279 | import httplib |
280 | |
281 | from maasserver.exceptions import ( |
282 | + InvalidConstraint, |
283 | MAASAPIBadRequest, |
284 | Redirect, |
285 | ) |
286 | @@ -38,3 +39,21 @@ |
287 | exception = Redirect(target) |
288 | response = exception.make_http_response() |
289 | self.assertEqual(target, extract_redirect(response)) |
290 | + |
291 | + def test_InvalidConstraint_is_bad_request(self): |
292 | + err = InvalidConstraint("height", "-1", ValueError("Not positive")) |
293 | + response = err.make_http_response() |
294 | + self.assertEqual(httplib.BAD_REQUEST, response.status_code) |
295 | + |
296 | + def test_InvalidConstraint_str_without_context(self): |
297 | + err = InvalidConstraint("hue", "shiny") |
298 | + self.assertEqual(str(err), "Invalid 'hue' constraint 'shiny'") |
299 | + |
300 | + def test_InvalidConstraint_str_with_context(self): |
301 | + try: |
302 | + int("hundreds") |
303 | + except ValueError as e: |
304 | + context = e |
305 | + err = InvalidConstraint("limbs", "hundreds", context) |
306 | + self.assertEqual(str(err), |
307 | + "Invalid 'limbs' constraint 'hundreds': " + str(context)) |
308 | |
309 | === modified file 'src/maasserver/tests/test_node.py' |
310 | --- src/maasserver/tests/test_node.py 2012-09-26 08:54:29 +0000 |
311 | +++ src/maasserver/tests/test_node.py 2012-09-28 11:10:27 +0000 |
312 | @@ -19,6 +19,7 @@ |
313 | PermissionDenied, |
314 | ValidationError, |
315 | ) |
316 | +from django.http import Http404 |
317 | from maasserver.enum import ( |
318 | ARCHITECTURE, |
319 | DISTRO_SERIES, |
320 | @@ -27,7 +28,10 @@ |
321 | NODE_STATUS_CHOICES, |
322 | NODE_STATUS_CHOICES_DICT, |
323 | ) |
324 | -from maasserver.exceptions import NodeStateViolation |
325 | +from maasserver.exceptions import ( |
326 | + InvalidConstraint, |
327 | + NodeStateViolation, |
328 | + ) |
329 | from maasserver.models import ( |
330 | Config, |
331 | MACAddress, |
332 | @@ -685,7 +689,7 @@ |
333 | self.assertEqual( |
334 | None, |
335 | Node.objects.get_available_node_for_acquisition( |
336 | - user, {'name': node.system_id})) |
337 | + user, {'hostname': node.system_id})) |
338 | |
339 | def test_get_available_node_with_name(self): |
340 | """A single available node can be selected using its hostname""" |
341 | @@ -694,33 +698,27 @@ |
342 | self.assertEqual( |
343 | nodes[1], |
344 | Node.objects.get_available_node_for_acquisition( |
345 | - user, {'name': nodes[1].hostname})) |
346 | - |
347 | - def test_get_available_node_with_unknown_name(self): |
348 | - """None is returned if there is no node with a given name""" |
349 | - user = factory.make_user() |
350 | - self.assertEqual( |
351 | - None, |
352 | - Node.objects.get_available_node_for_acquisition( |
353 | - user, {'name': factory.getRandomString()})) |
354 | + user, {'hostname': nodes[1].hostname})) |
355 | |
356 | def test_get_available_node_with_arch(self): |
357 | - """An available node can be selected of a given architecture""" |
358 | + """An available node can be selected off a given architecture""" |
359 | user = factory.make_user() |
360 | nodes = [self.make_node(architecture=s) |
361 | for s in (ARCHITECTURE.amd64, ARCHITECTURE.i386)] |
362 | available_node = Node.objects.get_available_node_for_acquisition( |
363 | - user, {'arch': "i386"}) |
364 | + user, {'architecture': "i386"}) |
365 | self.assertEqual(ARCHITECTURE.i386, available_node.architecture) |
366 | self.assertEqual(nodes[1], available_node) |
367 | |
368 | - def test_get_available_node_with_unknown_arch(self): |
369 | - """None is returned if an arch not used by MaaS is given""" |
370 | + def test_get_available_node_with_tag(self): |
371 | + """An available node can be selected off a given tag""" |
372 | + nodes = [self.make_node() for i in range(2)] |
373 | + tag = factory.make_tag('strong') |
374 | user = factory.make_user() |
375 | - self.assertEqual( |
376 | - None, |
377 | - Node.objects.get_available_node_for_acquisition( |
378 | - user, {'arch': "sparc"})) |
379 | + nodes[1].tags.add(tag) |
380 | + available_node = Node.objects.get_available_node_for_acquisition( |
381 | + user, {'tags': "strong"}) |
382 | + self.assertEqual(nodes[1], available_node) |
383 | |
384 | def test_stop_nodes_stops_nodes(self): |
385 | # We don't actually want to fire off power events, but we'll go |
386 | |
387 | === added file 'src/maasserver/tests/test_node_constraint_filter.py' |
388 | --- src/maasserver/tests/test_node_constraint_filter.py 1970-01-01 00:00:00 +0000 |
389 | +++ src/maasserver/tests/test_node_constraint_filter.py 2012-09-28 11:10:27 +0000 |
390 | @@ -0,0 +1,102 @@ |
391 | +# Copyright 2012 Canonical Ltd. This software is licensed under the |
392 | +# GNU Affero General Public License version 3 (see the file LICENSE). |
393 | + |
394 | +"""Test node filtering on specific constraints.""" |
395 | + |
396 | +from __future__ import ( |
397 | + absolute_import, |
398 | + print_function, |
399 | + unicode_literals, |
400 | + ) |
401 | + |
402 | +__metaclass__ = type |
403 | +__all__ = [] |
404 | + |
405 | +from maasserver.enum import ARCHITECTURE |
406 | + |
407 | +from maasserver.exceptions import InvalidConstraint |
408 | +from maasserver.models import Node |
409 | +from maasserver.models.node_constraint_filter import constrain_nodes |
410 | +from maasserver.testing.factory import factory |
411 | +from maasserver.testing.testcase import TestCase |
412 | + |
413 | + |
414 | +class TestConstrainNodes(TestCase): |
415 | + |
416 | + def assertConstrainedNodes(self, expected_nodes, constraints): |
417 | + nodes = constrain_nodes(Node.objects.all(), constraints) |
418 | + self.assertItemsEqual(expected_nodes, nodes) |
419 | + |
420 | + def test_no_constraints(self): |
421 | + node1 = factory.make_node() |
422 | + node2 = factory.make_node() |
423 | + self.assertConstrainedNodes([node1, node2], None) |
424 | + self.assertConstrainedNodes([node1, node2], {}) |
425 | + |
426 | + def test_hostname(self): |
427 | + node1 = factory.make_node(set_hostname=True) |
428 | + node2 = factory.make_node(set_hostname=True) |
429 | + self.assertConstrainedNodes([node1], {'hostname': node1.hostname}) |
430 | + self.assertConstrainedNodes([node2], {'hostname': node2.hostname}) |
431 | + self.assertConstrainedNodes([], {'hostname': 'unknown-name'}) |
432 | + |
433 | + def test_architecture(self): |
434 | + node1 = factory.make_node(architecture=ARCHITECTURE.i386) |
435 | + node2 = factory.make_node(architecture=ARCHITECTURE.armhf) |
436 | + self.assertConstrainedNodes([node1], {'architecture': 'i386'}) |
437 | + self.assertConstrainedNodes([node2], {'architecture': 'armhf'}) |
438 | + self.assertConstrainedNodes([], {'architecture': 'sparc'}) |
439 | + |
440 | + def test_cpu_count(self): |
441 | + node1 = factory.make_node(cpu_count=1) |
442 | + node2 = factory.make_node(cpu_count=2) |
443 | + self.assertConstrainedNodes([node1, node2], {'cpu_count': '0'}) |
444 | + self.assertConstrainedNodes([node1, node2], {'cpu_count': '1'}) |
445 | + self.assertConstrainedNodes([node2], {'cpu_count': '2'}) |
446 | + self.assertConstrainedNodes([], {'cpu_count': '4'}) |
447 | + self.assertRaises(InvalidConstraint, |
448 | + self.assertConstrainedNodes, [], {'cpu_count': 'notint'}) |
449 | + |
450 | + def test_memory(self): |
451 | + node1 = factory.make_node(memory=1024) |
452 | + node2 = factory.make_node(memory=4096) |
453 | + self.assertConstrainedNodes([node1, node2], {'memory': '512'}) |
454 | + self.assertConstrainedNodes([node1, node2], {'memory': '1024'}) |
455 | + self.assertConstrainedNodes([node2], {'memory': '2048'}) |
456 | + self.assertConstrainedNodes([node2], {'memory': '4096'}) |
457 | + self.assertConstrainedNodes([], {'memory': '8192'}) |
458 | + self.assertRaises(InvalidConstraint, |
459 | + self.assertConstrainedNodes, [], {'memory': 'notint'}) |
460 | + |
461 | + def test_tags(self): |
462 | + tag_big = factory.make_tag(name='big') |
463 | + tag_burly = factory.make_tag(name='burly') |
464 | + node_big = factory.make_node() |
465 | + node_big.tags.add(tag_big) |
466 | + node_burly = factory.make_node() |
467 | + node_burly.tags.add(tag_burly) |
468 | + node_bignburly = factory.make_node() |
469 | + node_bignburly.tags.add(tag_big) |
470 | + node_bignburly.tags.add(tag_burly) |
471 | + self.assertConstrainedNodes([node_big, node_bignburly], |
472 | + {'tags': 'big'}) |
473 | + self.assertConstrainedNodes([node_burly, node_bignburly], |
474 | + {'tags': 'burly'}) |
475 | + self.assertConstrainedNodes([node_bignburly], |
476 | + {'tags': 'big,burly'}) |
477 | + self.assertConstrainedNodes([node_bignburly], |
478 | + {'tags': 'big burly'}) |
479 | + self.assertRaises(InvalidConstraint, |
480 | + self.assertConstrainedNodes, [], {'tags': 'big unknown'}) |
481 | + |
482 | + def test_combined_constraints(self): |
483 | + tag_big = factory.make_tag(name='big') |
484 | + node_big = factory.make_node(architecture=ARCHITECTURE.i386) |
485 | + node_big.tags.add(tag_big) |
486 | + node_small = factory.make_node(architecture=ARCHITECTURE.i386) |
487 | + node_big_arm = factory.make_node(architecture=ARCHITECTURE.armhf) |
488 | + node_big_arm.tags.add(tag_big) |
489 | + self.assertConstrainedNodes([node_big, node_big_arm], |
490 | + {'tags': 'big'}) |
491 | + self.assertConstrainedNodes([node_big], |
492 | + {'architecture': 'i386', 'tags': 'big'}) |
As mentioned on IRC, the error message in InvalidConstraint seems misleading. As far as I can see, "Invalid 'X' constraint 'Y'" can mean that Y is not an acceptable value for constraint X. So far, so good. But it can also mean that X is not a known constraint in the first place! I would suggest either splitting these two meanings into separate exception classes, or differentiating the error message so that it will contain the constraint value only if relevant. (Presumably it wouldn't need to be a separate attribute on the exception class).
The AcquisitionCons trainer looks like it wants to be a module, not a class. Its self.nodes looks like it wants to be a parameter and return value to the various filter functions, rather than implicit mutable state. And looking up a computed method name can be a handy shorthand, but if you're duplicating the list of accepted constraints here anyway (you also had it in the constraints map), then you might as well make the lookup in an explicit dict:
def constrain_ identical( nodes, key, value): key=value)
"""Generic constraint: filter nodes by exact value match."""
return nodes.filter(
def constrain_ int_gte( nodes, key, value): nt(key, value, e) **{"%s_ _gte" % key: value})
"""Integer constraint: greater than or equal to `value`."""
try:
int_value = int(value)
except ValueError as e:
raise InvalidConstrai
return nodes.filter(
def constrain_ tags(nodes, key, tag_expression): replace( ",", " ").strip().split() Tag.objects. filter( name=tag_ name)) nt('tags' , tag_name, "No such tag.") tags=tag)
"""Tags match: restrict to nodes that have all of given tags list."""
# We support both comma-separated and space-separated tag lists.
tag_names = tag_expression.
for tag_name in tag_names:
tag = get_one(
if tag is None:
raise InvalidConstrai
nodes = nodes.filter(
return nodes
# How to filter each individual constraint. identical, identical,
constraint_filters = {
# This only supports an exact match on arch type. Using an i386 image
# on amd64 hardware will wait.
'architecture': constrain_
'name': constrain_
'cpu_count': constrain_int_gte,
'memory': constrain_int_gte,
'tags': constrain_tags,
}
def constrain_ nodes(nodes, constraints): items() : filters. get(constraint)
for constraint, value in constraints.
filter = constraint_
if filter is not None:
nodes = filter(nodes, constraint, value)
return nodes
I think that's easier to follow, and despite having more documentation it's fewer lines as well.
(By the way, does an equality comparison on a tag value really do an "is in" comparison!? The tests verify it, but )