Merge lp:~markwash/nova/lp727225 into lp:~hudson-openstack/nova/trunk
- lp727225
- Merge into trunk
Status: | Merged |
---|---|
Approved by: | Jay Pipes |
Approved revision: | 799 |
Merged at revision: | 814 |
Proposed branch: | lp:~markwash/nova/lp727225 |
Merge into: | lp:~hudson-openstack/nova/trunk |
Diff against target: |
1111 lines (+816/-55) 11 files modified
Authors (+1/-0) nova/api/openstack/servers.py (+152/-13) nova/compute/api.py (+23/-2) nova/compute/manager.py (+3/-9) nova/db/sqlalchemy/models.py (+1/-1) nova/quota.py (+22/-0) nova/tests/api/openstack/fakes.py (+0/-2) nova/tests/api/openstack/test_servers.py (+527/-0) nova/tests/test_quota.py (+70/-0) nova/utils.py (+0/-12) nova/virt/xenapi/vmops.py (+17/-16) |
To merge this branch: | bzr merge lp:~markwash/nova/lp727225 |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Jay Pipes (community) | Approve | ||
Brian Lamar (community) | Approve | ||
Eric Day (community) | Approve | ||
Review via email: mp+52754@code.launchpad.net |
Commit message
Fix lp727225 by adding support for personality files to the openstack api.
Description:
This merge adds support for personality files to the openstack api. It leverages previous work which added this functionality to the compute api, compute manager, and xen api.
Description of the change
Fix lp727225 by adding support for personality files to the openstack api.
Mark Washenberger (markwash) wrote : | # |
Christopher MacGown (0x44) wrote : | # |
Needs a docstring in _check_
Mark Washenberger (markwash) wrote : | # |
Added a missing docstring to my branch
Eric Day (eday) wrote : | # |
There is some inconsistency in naming with "personality" and "onset". The nova/api/* can be whatever the external API uses, but within nova/compute/* and nova/db/* the name should be the same. For example, nova/compute/api.py uses personality and nova/compute/
Mark Washenberger (markwash) wrote : | # |
> There is some inconsistency in naming with "personality" and "onset". The
> nova/api/* can be whatever the external API uses, but within nova/compute/*
> and nova/db/* the name should be the same. For example, nova/compute/api.py
> uses personality and nova/compute/
Thanks for having a look Eric.
I'm definitely a bit nervous about the naming change. Let me put forward my reasoning, and then hopefully I can beg a little more guidance about how to address this.
The reason why I changed the naming down into the compute layer is because the text of the QuotaErrors associated with onset/personality files is what is actually used in the http error message. So, if I keep the name onset_files and make it consistent throughout the compute service layer, then when a user submits an invalid number of personality files, his error message will complain about too many onset files.
I would also suggest that the only place where onset_files remains in the compute manager is in instance_ref, which is actually coming out of the data layer. Therefore, the compute layer could be seen as a sort of adapter on top of the data layer for this particular naming issue.
Here are a couple of ways I could solve this differently:
1) make the quota error text say "personality" but change the variable names back to onset_files. This would leave a different kind of inconsistency in the compute service layer.
2) create new quotaerror exceptions that are specific to personality files, and catch them in the os api. The only drawback I see here is that I don't like to get trigger happy with exception types.
3) figure out some fancy way of using internationaliz
Anyone got any suggestions? I want to do the right thing here.
Eric Day (eday) wrote : | # |
We should probably be raising an exception with data limits but no text from nova/compute/* and let the API handle creating a message. This allows different APIs to do whatever they need with the exception and data, especially since not all APIs will want to even report a text message in the future. This allows us to keep names consistent within API boundaries.
As far as if we use "personality" or "onset", I don't really care (although both are not immediately obvious that they are injected files), I just think it's important to have consistency. :)
Mark Washenberger (markwash) wrote : | # |
Updated to return to consistent use of onset_files. Handles http error codes in the os api by checking the quota error code and raising a new http exception if it is a quota error where the message won't make sense to an os api client.
Eric Day (eday) wrote : | # |
141: Any way to make this generic and plug into the common deserializer in some way? Possibly passing options to trigger specific actions for certain elements?
243: any reason you assign the result of quota check (or why quota check returns anything)? this should always be a RO operation, no?
Other than that, looks good to me!
Mark Washenberger (markwash) wrote : | # |
> 141: Any way to make this generic and plug into the common deserializer in
> some way? Possibly passing options to trigger specific actions for certain
> elements?
I think it would require replacing the common serializer in order to make this work. In fact, I'd like to see serialization become more of an api- or controller-specific activity. However, it seems out of scope for this particular merge.
> 243: any reason you assign the result of quota check (or why quota check
> returns anything)? this should always be a RO operation, no?
I thought I was being consistent with other places in that function--but looking again I don't think so anymore. So I've refactored it in the latest commit.
Thanks!
- 791. By Mark Washenberger
-
merge lp:nova
- 792. By Mark Washenberger
-
refactor onset_files quota checking
Jay Pipes (jaypipes) wrote : | # |
Hi!
Good stuff :)
As mentioned on IRC, I'd suggest a global search and replace:
s/onset_
104 + expl = 'Bad personality format: missing %s' % key
107 + raise exc.HTTPBadRequ
111 + msg = 'Personality content for %s cannot be decoded' % path
121 + expl = "Personality file limit exceeded"
124 + expl = "Personality file path too long"
127 + expl = "Personality file content too long"
Please i18n all those! If you're unclear on how to do that, let us know.
309 + return int(FLAGS.
314 + return int(FLAGS.
319 + return int(FLAGS.
You used flags.DEFINE_
Throughout the test cases in TestServerCreat
364 + def test_minimal_
365 + serial_request = """
366 +<server xmlns="http://
367 + name="new-
368 + request = self.deserializ
369 + expected = {"server": {
370 + "name": "new-server-test",
371 + "imageId": "1",
372 + "flavorId": "1",
373 + }}
374 + self.assertEqua
Should be:
def test_minimal_
serial_request = """
<server xmlns="http://
name="new-
request = self.deserializ
expected = {"server": {
"name": "new-server-test",
"imageId": "1",
"flavorId": "1"}}
self.
This is important, because currently, you are testing an invalid XML string (having a \ between attributes and in other places, having a \ between element tags) and the serializer tests are apparently passing.
507 + def test_request_
508 + serial_request = """
509 +<server xmlns="http://
510 + name="new-
511 +<metadata>
512 + request = self.deserializ
513 + expected = {"": "beta"}
514 + self.assertEqua
The test case above technically passes. However, try adding in an additional <meta>gamma</meta> and see if the test passes ;)
Generally, when testing the parsing/
1) >1 metadata element with no key
2) >1 injected file in the personality
3) >1 personality in the server
I think you may find some interesting results :)
The tests in TestServerInsta
Cheers!
jay
- 793. By Mark Washenberger
-
s/onset_
files/injected_ files/g - 794. By Mark Washenberger
-
internationaliz
ation - 795. By Mark Washenberger
-
pep8
- 796. By Mark Washenberger
-
ignore differently-named nodes in personality and metadata parsing
- 797. By Mark Washenberger
-
merge lp:nova
- 798. By Mark Washenberger
-
pep8
Mark Washenberger (markwash) wrote : | # |
Thanks for the suggestions Jay.
I went ahead and renamed onset_files everywhere to injected_files. I also added internationaliz
I pulled the int cast off of the flags, but I decided to go ahead and keep the quota module functions. The don't do much at this time, but it would be better for other parts of the code not to directly access the flags.
I removed most of the backslashes in the xml tests and was pleased to find that my parser did just fine. However, inside personality and metadata extraction, it needed a little "ignore unknown" love, so I added that. I preserved the backslashes in one test, because as far as I know there cannot be newlines in the base64 contents.
I added a few more tests to the deserialization, according to your requests. Everything worked out fine.
Thanks again.
Jay Pipes (jaypipes) wrote : | # |
Excellent work, Mark. Thanks for those additional tests and cleanups.
Eric Day (eday) wrote : | # |
I'm good if folks are comfortable with accepting the one-off deserialization for now. We should address the serialization issue in another patch soon (possibly after proposal to rework plus a ML or summit discussion).
Brian Lamar (blamar) wrote : | # |
Looks good to us....Mark.
- 799. By Mark Washenberger
-
merge lp:nova and resolve conflicts
Jay Pipes (jaypipes) wrote : | # |
approved again after merge/resolve.
Preview Diff
1 | === modified file 'Authors' |
2 | --- Authors 2011-03-11 08:54:08 +0000 |
3 | +++ Authors 2011-03-16 19:19:09 +0000 |
4 | @@ -40,6 +40,7 @@ |
5 | Kevin L. Mitchell <kevin.mitchell@rackspace.com> |
6 | Koji Iida <iida.koji@lab.ntt.co.jp> |
7 | Lorin Hochstein <lorin@isi.edu> |
8 | +Mark Washenberger <mark.washenberger@rackspace.com> |
9 | Masanori Itoh <itoumsn@nttdata.co.jp> |
10 | Matt Dietz <matt.dietz@rackspace.com> |
11 | Michael Gundlach <michael.gundlach@rackspace.com> |
12 | |
13 | === modified file 'nova/api/openstack/servers.py' |
14 | --- nova/api/openstack/servers.py 2011-03-15 02:57:30 +0000 |
15 | +++ nova/api/openstack/servers.py 2011-03-16 19:19:09 +0000 |
16 | @@ -13,9 +13,11 @@ |
17 | # License for the specific language governing permissions and limitations |
18 | # under the License. |
19 | |
20 | +import base64 |
21 | import hashlib |
22 | import json |
23 | import traceback |
24 | +from xml.dom import minidom |
25 | |
26 | from webob import exc |
27 | |
28 | @@ -141,7 +143,7 @@ |
29 | |
30 | def create(self, req): |
31 | """ Creates a new server for a given user """ |
32 | - env = self._deserialize(req.body, req.get_content_type()) |
33 | + env = self._deserialize_create(req) |
34 | if not env: |
35 | return faults.Fault(exc.HTTPUnprocessableEntity()) |
36 | |
37 | @@ -170,18 +172,24 @@ |
38 | for k, v in env['server']['metadata'].items(): |
39 | metadata.append({'key': k, 'value': v}) |
40 | |
41 | - instances = self.compute_api.create( |
42 | - context, |
43 | - instance_types.get_by_flavor_id(env['server']['flavorId']), |
44 | - image_id, |
45 | - kernel_id=kernel_id, |
46 | - ramdisk_id=ramdisk_id, |
47 | - display_name=env['server']['name'], |
48 | - display_description=env['server']['name'], |
49 | - key_name=key_name, |
50 | - key_data=key_data, |
51 | - metadata=metadata, |
52 | - onset_files=env.get('onset_files', [])) |
53 | + personality = env['server'].get('personality', []) |
54 | + injected_files = self._get_injected_files(personality) |
55 | + |
56 | + try: |
57 | + instances = self.compute_api.create( |
58 | + context, |
59 | + instance_types.get_by_flavor_id(env['server']['flavorId']), |
60 | + image_id, |
61 | + kernel_id=kernel_id, |
62 | + ramdisk_id=ramdisk_id, |
63 | + display_name=env['server']['name'], |
64 | + display_description=env['server']['name'], |
65 | + key_name=key_name, |
66 | + key_data=key_data, |
67 | + metadata=metadata, |
68 | + injected_files=injected_files) |
69 | + except QuotaError as error: |
70 | + self._handle_quota_error(error) |
71 | |
72 | server = _translate_keys(instances[0]) |
73 | password = "%s%s" % (server['server']['name'][:4], |
74 | @@ -191,6 +199,61 @@ |
75 | password) |
76 | return server |
77 | |
78 | + def _deserialize_create(self, request): |
79 | + """ |
80 | + Deserialize a create request |
81 | + |
82 | + Overrides normal behavior in the case of xml content |
83 | + """ |
84 | + if request.content_type == "application/xml": |
85 | + deserializer = ServerCreateRequestXMLDeserializer() |
86 | + return deserializer.deserialize(request.body) |
87 | + else: |
88 | + return self._deserialize(request.body, request.get_content_type()) |
89 | + |
90 | + def _get_injected_files(self, personality): |
91 | + """ |
92 | + Create a list of injected files from the personality attribute |
93 | + |
94 | + At this time, injected_files must be formatted as a list of |
95 | + (file_path, file_content) pairs for compatibility with the |
96 | + underlying compute service. |
97 | + """ |
98 | + injected_files = [] |
99 | + for item in personality: |
100 | + try: |
101 | + path = item['path'] |
102 | + contents = item['contents'] |
103 | + except KeyError as key: |
104 | + expl = _('Bad personality format: missing %s') % key |
105 | + raise exc.HTTPBadRequest(explanation=expl) |
106 | + except TypeError: |
107 | + expl = _('Bad personality format') |
108 | + raise exc.HTTPBadRequest(explanation=expl) |
109 | + try: |
110 | + contents = base64.b64decode(contents) |
111 | + except TypeError: |
112 | + expl = _('Personality content for %s cannot be decoded') % path |
113 | + raise exc.HTTPBadRequest(explanation=expl) |
114 | + injected_files.append((path, contents)) |
115 | + return injected_files |
116 | + |
117 | + def _handle_quota_errors(self, error): |
118 | + """ |
119 | + Reraise quota errors as api-specific http exceptions |
120 | + """ |
121 | + if error.code == "OnsetFileLimitExceeded": |
122 | + expl = _("Personality file limit exceeded") |
123 | + raise exc.HTTPBadRequest(explanation=expl) |
124 | + if error.code == "OnsetFilePathLimitExceeded": |
125 | + expl = _("Personality file path too long") |
126 | + raise exc.HTTPBadRequest(explanation=expl) |
127 | + if error.code == "OnsetFileContentLimitExceeded": |
128 | + expl = _("Personality file content too long") |
129 | + raise exc.HTTPBadRequest(explanation=expl) |
130 | + # if the original error is okay, just reraise it |
131 | + raise error |
132 | + |
133 | def update(self, req, id): |
134 | """ Updates the server name or password """ |
135 | if len(req.body) == 0: |
136 | @@ -480,3 +543,79 @@ |
137 | _("Ramdisk not found for image %(image_id)s") % locals()) |
138 | |
139 | return kernel_id, ramdisk_id |
140 | + |
141 | + |
142 | +class ServerCreateRequestXMLDeserializer(object): |
143 | + """ |
144 | + Deserializer to handle xml-formatted server create requests. |
145 | + |
146 | + Handles standard server attributes as well as optional metadata |
147 | + and personality attributes |
148 | + """ |
149 | + |
150 | + def deserialize(self, string): |
151 | + """Deserialize an xml-formatted server create request""" |
152 | + dom = minidom.parseString(string) |
153 | + server = self._extract_server(dom) |
154 | + return {'server': server} |
155 | + |
156 | + def _extract_server(self, node): |
157 | + """Marshal the server attribute of a parsed request""" |
158 | + server = {} |
159 | + server_node = self._find_first_child_named(node, 'server') |
160 | + for attr in ["name", "imageId", "flavorId"]: |
161 | + server[attr] = server_node.getAttribute(attr) |
162 | + metadata = self._extract_metadata(server_node) |
163 | + if metadata is not None: |
164 | + server["metadata"] = metadata |
165 | + personality = self._extract_personality(server_node) |
166 | + if personality is not None: |
167 | + server["personality"] = personality |
168 | + return server |
169 | + |
170 | + def _extract_metadata(self, server_node): |
171 | + """Marshal the metadata attribute of a parsed request""" |
172 | + metadata_node = self._find_first_child_named(server_node, "metadata") |
173 | + if metadata_node is None: |
174 | + return None |
175 | + metadata = {} |
176 | + for meta_node in self._find_children_named(metadata_node, "meta"): |
177 | + key = meta_node.getAttribute("key") |
178 | + metadata[key] = self._extract_text(meta_node) |
179 | + return metadata |
180 | + |
181 | + def _extract_personality(self, server_node): |
182 | + """Marshal the personality attribute of a parsed request""" |
183 | + personality_node = \ |
184 | + self._find_first_child_named(server_node, "personality") |
185 | + if personality_node is None: |
186 | + return None |
187 | + personality = [] |
188 | + for file_node in self._find_children_named(personality_node, "file"): |
189 | + item = {} |
190 | + if file_node.hasAttribute("path"): |
191 | + item["path"] = file_node.getAttribute("path") |
192 | + item["contents"] = self._extract_text(file_node) |
193 | + personality.append(item) |
194 | + return personality |
195 | + |
196 | + def _find_first_child_named(self, parent, name): |
197 | + """Search a nodes children for the first child with a given name""" |
198 | + for node in parent.childNodes: |
199 | + if node.nodeName == name: |
200 | + return node |
201 | + return None |
202 | + |
203 | + def _find_children_named(self, parent, name): |
204 | + """Return all of a nodes children who have the given name""" |
205 | + for node in parent.childNodes: |
206 | + if node.nodeName == name: |
207 | + yield node |
208 | + |
209 | + def _extract_text(self, node): |
210 | + """Get the text field contained by the given node""" |
211 | + if len(node.childNodes) == 1: |
212 | + child = node.childNodes[0] |
213 | + if child.nodeType == child.TEXT_NODE: |
214 | + return child.nodeValue |
215 | + return "" |
216 | |
217 | === modified file 'nova/compute/api.py' |
218 | --- nova/compute/api.py 2011-03-14 17:59:41 +0000 |
219 | +++ nova/compute/api.py 2011-03-16 19:19:09 +0000 |
220 | @@ -80,13 +80,32 @@ |
221 | topic, |
222 | {"method": "get_network_topic", "args": {'fake': 1}}) |
223 | |
224 | + def _check_injected_file_quota(self, context, injected_files): |
225 | + """ |
226 | + Enforce quota limits on injected files |
227 | + |
228 | + Raises a QuotaError if any limit is exceeded |
229 | + """ |
230 | + if injected_files is None: |
231 | + return |
232 | + limit = quota.allowed_injected_files(context) |
233 | + if len(injected_files) > limit: |
234 | + raise quota.QuotaError(code="OnsetFileLimitExceeded") |
235 | + path_limit = quota.allowed_injected_file_path_bytes(context) |
236 | + content_limit = quota.allowed_injected_file_content_bytes(context) |
237 | + for path, content in injected_files: |
238 | + if len(path) > path_limit: |
239 | + raise quota.QuotaError(code="OnsetFilePathLimitExceeded") |
240 | + if len(content) > content_limit: |
241 | + raise quota.QuotaError(code="OnsetFileContentLimitExceeded") |
242 | + |
243 | def create(self, context, instance_type, |
244 | image_id, kernel_id=None, ramdisk_id=None, |
245 | min_count=1, max_count=1, |
246 | display_name='', display_description='', |
247 | key_name=None, key_data=None, security_group='default', |
248 | availability_zone=None, user_data=None, metadata=[], |
249 | - onset_files=None): |
250 | + injected_files=None): |
251 | """Create the number of instances requested if quota and |
252 | other arguments check out ok.""" |
253 | |
254 | @@ -124,6 +143,8 @@ |
255 | LOG.warn(msg) |
256 | raise quota.QuotaError(msg, "MetadataLimitExceeded") |
257 | |
258 | + self._check_injected_file_quota(context, injected_files) |
259 | + |
260 | image = self.image_service.show(context, image_id) |
261 | |
262 | os_type = None |
263 | @@ -225,7 +246,7 @@ |
264 | "args": {"topic": FLAGS.compute_topic, |
265 | "instance_id": instance_id, |
266 | "availability_zone": availability_zone, |
267 | - "onset_files": onset_files}}) |
268 | + "injected_files": injected_files}}) |
269 | |
270 | for group_id in security_groups: |
271 | self.trigger_security_group_members_refresh(elevated, group_id) |
272 | |
273 | === modified file 'nova/compute/manager.py' |
274 | --- nova/compute/manager.py 2011-03-10 04:30:52 +0000 |
275 | +++ nova/compute/manager.py 2011-03-16 19:19:09 +0000 |
276 | @@ -34,7 +34,6 @@ |
277 | :func:`nova.utils.import_object` |
278 | """ |
279 | |
280 | -import base64 |
281 | import datetime |
282 | import os |
283 | import random |
284 | @@ -180,7 +179,7 @@ |
285 | """Launch a new instance with specified options.""" |
286 | context = context.elevated() |
287 | instance_ref = self.db.instance_get(context, instance_id) |
288 | - instance_ref.onset_files = kwargs.get('onset_files', []) |
289 | + instance_ref.injected_files = kwargs.get('injected_files', []) |
290 | if instance_ref['name'] in self.driver.list_instances(): |
291 | raise exception.Error(_("Instance has already been created")) |
292 | LOG.audit(_("instance %s: starting..."), instance_id, |
293 | @@ -359,15 +358,10 @@ |
294 | LOG.warn(_('trying to inject a file into a non-running ' |
295 | 'instance: %(instance_id)s (state: %(instance_state)s ' |
296 | 'expected: %(expected_state)s)') % locals()) |
297 | - # Files/paths *should* be base64-encoded at this point, but |
298 | - # double-check to make sure. |
299 | - b64_path = utils.ensure_b64_encoding(path) |
300 | - b64_contents = utils.ensure_b64_encoding(file_contents) |
301 | - plain_path = base64.b64decode(b64_path) |
302 | nm = instance_ref['name'] |
303 | - msg = _('instance %(nm)s: injecting file to %(plain_path)s') % locals() |
304 | + msg = _('instance %(nm)s: injecting file to %(path)s') % locals() |
305 | LOG.audit(msg) |
306 | - self.driver.inject_file(instance_ref, b64_path, b64_contents) |
307 | + self.driver.inject_file(instance_ref, path, file_contents) |
308 | |
309 | @exception.wrap_exception |
310 | @checks_instance_lock |
311 | |
312 | === modified file 'nova/db/sqlalchemy/models.py' |
313 | --- nova/db/sqlalchemy/models.py 2011-03-14 17:59:41 +0000 |
314 | +++ nova/db/sqlalchemy/models.py 2011-03-16 19:19:09 +0000 |
315 | @@ -161,7 +161,7 @@ |
316 | class Instance(BASE, NovaBase): |
317 | """Represents a guest vm.""" |
318 | __tablename__ = 'instances' |
319 | - onset_files = [] |
320 | + injected_files = [] |
321 | |
322 | id = Column(Integer, primary_key=True, autoincrement=True) |
323 | |
324 | |
325 | === modified file 'nova/quota.py' |
326 | --- nova/quota.py 2011-02-17 23:00:18 +0000 |
327 | +++ nova/quota.py 2011-03-16 19:19:09 +0000 |
328 | @@ -37,6 +37,12 @@ |
329 | 'number of floating ips allowed per project') |
330 | flags.DEFINE_integer('quota_metadata_items', 128, |
331 | 'number of metadata items allowed per instance') |
332 | +flags.DEFINE_integer('quota_max_injected_files', 5, |
333 | + 'number of injected files allowed') |
334 | +flags.DEFINE_integer('quota_max_injected_file_content_bytes', 10 * 1024, |
335 | + 'number of bytes allowed per injected file') |
336 | +flags.DEFINE_integer('quota_max_injected_file_path_bytes', 255, |
337 | + 'number of bytes allowed per injected file path') |
338 | |
339 | |
340 | def get_quota(context, project_id): |
341 | @@ -46,6 +52,7 @@ |
342 | 'gigabytes': FLAGS.quota_gigabytes, |
343 | 'floating_ips': FLAGS.quota_floating_ips, |
344 | 'metadata_items': FLAGS.quota_metadata_items} |
345 | + |
346 | try: |
347 | quota = db.quota_get(context, project_id) |
348 | for key in rval.keys(): |
349 | @@ -106,6 +113,21 @@ |
350 | return min(num_metadata_items, num_allowed_metadata_items) |
351 | |
352 | |
353 | +def allowed_injected_files(context): |
354 | + """Return the number of injected files allowed""" |
355 | + return FLAGS.quota_max_injected_files |
356 | + |
357 | + |
358 | +def allowed_injected_file_content_bytes(context): |
359 | + """Return the number of bytes allowed per injected file content""" |
360 | + return FLAGS.quota_max_injected_file_content_bytes |
361 | + |
362 | + |
363 | +def allowed_injected_file_path_bytes(context): |
364 | + """Return the number of bytes allowed in an injected file path""" |
365 | + return FLAGS.quota_max_injected_file_path_bytes |
366 | + |
367 | + |
368 | class QuotaError(exception.ApiError): |
369 | """Quota Exceeeded""" |
370 | pass |
371 | |
372 | === modified file 'nova/tests/api/openstack/fakes.py' |
373 | --- nova/tests/api/openstack/fakes.py 2011-03-16 19:02:17 +0000 |
374 | +++ nova/tests/api/openstack/fakes.py 2011-03-16 19:19:09 +0000 |
375 | @@ -69,8 +69,6 @@ |
376 | @webob.dec.wsgify |
377 | def fake_wsgi(self, req): |
378 | req.environ['nova.context'] = context.RequestContext(1, 1) |
379 | - if req.body: |
380 | - req.environ['inst_dict'] = json.loads(req.body) |
381 | return self.application |
382 | |
383 | |
384 | |
385 | === modified file 'nova/tests/api/openstack/test_servers.py' |
386 | --- nova/tests/api/openstack/test_servers.py 2011-03-16 18:49:25 +0000 |
387 | +++ nova/tests/api/openstack/test_servers.py 2011-03-16 19:19:09 +0000 |
388 | @@ -15,8 +15,11 @@ |
389 | # License for the specific language governing permissions and limitations |
390 | # under the License. |
391 | |
392 | +import base64 |
393 | import datetime |
394 | import json |
395 | +import unittest |
396 | +from xml.dom import minidom |
397 | |
398 | import stubout |
399 | import webob |
400 | @@ -607,5 +610,529 @@ |
401 | res = req.get_response(fakes.wsgi_app()) |
402 | self.assertEqual(res.status_int, 400) |
403 | |
404 | + |
405 | +class TestServerCreateRequestXMLDeserializer(unittest.TestCase): |
406 | + |
407 | + def setUp(self): |
408 | + self.deserializer = servers.ServerCreateRequestXMLDeserializer() |
409 | + |
410 | + def test_minimal_request(self): |
411 | + serial_request = """ |
412 | +<server xmlns="http://docs.rackspacecloud.com/servers/api/v1.0" |
413 | + name="new-server-test" imageId="1" flavorId="1"/>""" |
414 | + request = self.deserializer.deserialize(serial_request) |
415 | + expected = {"server": { |
416 | + "name": "new-server-test", |
417 | + "imageId": "1", |
418 | + "flavorId": "1", |
419 | + }} |
420 | + self.assertEquals(request, expected) |
421 | + |
422 | + def test_request_with_empty_metadata(self): |
423 | + serial_request = """ |
424 | +<server xmlns="http://docs.rackspacecloud.com/servers/api/v1.0" |
425 | + name="new-server-test" imageId="1" flavorId="1"> |
426 | + <metadata/> |
427 | +</server>""" |
428 | + request = self.deserializer.deserialize(serial_request) |
429 | + expected = {"server": { |
430 | + "name": "new-server-test", |
431 | + "imageId": "1", |
432 | + "flavorId": "1", |
433 | + "metadata": {}, |
434 | + }} |
435 | + self.assertEquals(request, expected) |
436 | + |
437 | + def test_request_with_empty_personality(self): |
438 | + serial_request = """ |
439 | +<server xmlns="http://docs.rackspacecloud.com/servers/api/v1.0" |
440 | + name="new-server-test" imageId="1" flavorId="1"> |
441 | + <personality/> |
442 | +</server>""" |
443 | + request = self.deserializer.deserialize(serial_request) |
444 | + expected = {"server": { |
445 | + "name": "new-server-test", |
446 | + "imageId": "1", |
447 | + "flavorId": "1", |
448 | + "personality": [], |
449 | + }} |
450 | + self.assertEquals(request, expected) |
451 | + |
452 | + def test_request_with_empty_metadata_and_personality(self): |
453 | + serial_request = """ |
454 | +<server xmlns="http://docs.rackspacecloud.com/servers/api/v1.0" |
455 | + name="new-server-test" imageId="1" flavorId="1"> |
456 | + <metadata/> |
457 | + <personality/> |
458 | +</server>""" |
459 | + request = self.deserializer.deserialize(serial_request) |
460 | + expected = {"server": { |
461 | + "name": "new-server-test", |
462 | + "imageId": "1", |
463 | + "flavorId": "1", |
464 | + "metadata": {}, |
465 | + "personality": [], |
466 | + }} |
467 | + self.assertEquals(request, expected) |
468 | + |
469 | + def test_request_with_empty_metadata_and_personality_reversed(self): |
470 | + serial_request = """ |
471 | +<server xmlns="http://docs.rackspacecloud.com/servers/api/v1.0" |
472 | + name="new-server-test" imageId="1" flavorId="1"> |
473 | + <personality/> |
474 | + <metadata/> |
475 | +</server>""" |
476 | + request = self.deserializer.deserialize(serial_request) |
477 | + expected = {"server": { |
478 | + "name": "new-server-test", |
479 | + "imageId": "1", |
480 | + "flavorId": "1", |
481 | + "metadata": {}, |
482 | + "personality": [], |
483 | + }} |
484 | + self.assertEquals(request, expected) |
485 | + |
486 | + def test_request_with_one_personality(self): |
487 | + serial_request = """ |
488 | +<server xmlns="http://docs.rackspacecloud.com/servers/api/v1.0" |
489 | + name="new-server-test" imageId="1" flavorId="1"> |
490 | + <personality> |
491 | + <file path="/etc/conf">aabbccdd</file> |
492 | + </personality> |
493 | +</server>""" |
494 | + request = self.deserializer.deserialize(serial_request) |
495 | + expected = [{"path": "/etc/conf", "contents": "aabbccdd"}] |
496 | + self.assertEquals(request["server"]["personality"], expected) |
497 | + |
498 | + def test_request_with_two_personalities(self): |
499 | + serial_request = """ |
500 | +<server xmlns="http://docs.rackspacecloud.com/servers/api/v1.0" |
501 | + name="new-server-test" imageId="1" flavorId="1"> |
502 | +<personality><file path="/etc/conf">aabbccdd</file> |
503 | +<file path="/etc/sudoers">abcd</file></personality></server>""" |
504 | + request = self.deserializer.deserialize(serial_request) |
505 | + expected = [{"path": "/etc/conf", "contents": "aabbccdd"}, |
506 | + {"path": "/etc/sudoers", "contents": "abcd"}] |
507 | + self.assertEquals(request["server"]["personality"], expected) |
508 | + |
509 | + def test_request_second_personality_node_ignored(self): |
510 | + serial_request = """ |
511 | +<server xmlns="http://docs.rackspacecloud.com/servers/api/v1.0" |
512 | + name="new-server-test" imageId="1" flavorId="1"> |
513 | + <personality> |
514 | + <file path="/etc/conf">aabbccdd</file> |
515 | + </personality> |
516 | + <personality> |
517 | + <file path="/etc/ignoreme">anything</file> |
518 | + </personality> |
519 | +</server>""" |
520 | + request = self.deserializer.deserialize(serial_request) |
521 | + expected = [{"path": "/etc/conf", "contents": "aabbccdd"}] |
522 | + self.assertEquals(request["server"]["personality"], expected) |
523 | + |
524 | + def test_request_with_one_personality_missing_path(self): |
525 | + serial_request = """ |
526 | +<server xmlns="http://docs.rackspacecloud.com/servers/api/v1.0" |
527 | + name="new-server-test" imageId="1" flavorId="1"> |
528 | +<personality><file>aabbccdd</file></personality></server>""" |
529 | + request = self.deserializer.deserialize(serial_request) |
530 | + expected = [{"contents": "aabbccdd"}] |
531 | + self.assertEquals(request["server"]["personality"], expected) |
532 | + |
533 | + def test_request_with_one_personality_empty_contents(self): |
534 | + serial_request = """ |
535 | +<server xmlns="http://docs.rackspacecloud.com/servers/api/v1.0" |
536 | + name="new-server-test" imageId="1" flavorId="1"> |
537 | +<personality><file path="/etc/conf"></file></personality></server>""" |
538 | + request = self.deserializer.deserialize(serial_request) |
539 | + expected = [{"path": "/etc/conf", "contents": ""}] |
540 | + self.assertEquals(request["server"]["personality"], expected) |
541 | + |
542 | + def test_request_with_one_personality_empty_contents_variation(self): |
543 | + serial_request = """ |
544 | +<server xmlns="http://docs.rackspacecloud.com/servers/api/v1.0" |
545 | + name="new-server-test" imageId="1" flavorId="1"> |
546 | +<personality><file path="/etc/conf"/></personality></server>""" |
547 | + request = self.deserializer.deserialize(serial_request) |
548 | + expected = [{"path": "/etc/conf", "contents": ""}] |
549 | + self.assertEquals(request["server"]["personality"], expected) |
550 | + |
551 | + def test_request_with_one_metadata(self): |
552 | + serial_request = """ |
553 | +<server xmlns="http://docs.rackspacecloud.com/servers/api/v1.0" |
554 | + name="new-server-test" imageId="1" flavorId="1"> |
555 | + <metadata> |
556 | + <meta key="alpha">beta</meta> |
557 | + </metadata> |
558 | +</server>""" |
559 | + request = self.deserializer.deserialize(serial_request) |
560 | + expected = {"alpha": "beta"} |
561 | + self.assertEquals(request["server"]["metadata"], expected) |
562 | + |
563 | + def test_request_with_two_metadata(self): |
564 | + serial_request = """ |
565 | +<server xmlns="http://docs.rackspacecloud.com/servers/api/v1.0" |
566 | + name="new-server-test" imageId="1" flavorId="1"> |
567 | + <metadata> |
568 | + <meta key="alpha">beta</meta> |
569 | + <meta key="foo">bar</meta> |
570 | + </metadata> |
571 | +</server>""" |
572 | + request = self.deserializer.deserialize(serial_request) |
573 | + expected = {"alpha": "beta", "foo": "bar"} |
574 | + self.assertEquals(request["server"]["metadata"], expected) |
575 | + |
576 | + def test_request_with_metadata_missing_value(self): |
577 | + serial_request = """ |
578 | +<server xmlns="http://docs.rackspacecloud.com/servers/api/v1.0" |
579 | + name="new-server-test" imageId="1" flavorId="1"> |
580 | + <metadata> |
581 | + <meta key="alpha"></meta> |
582 | + </metadata> |
583 | +</server>""" |
584 | + request = self.deserializer.deserialize(serial_request) |
585 | + expected = {"alpha": ""} |
586 | + self.assertEquals(request["server"]["metadata"], expected) |
587 | + |
588 | + def test_request_with_two_metadata_missing_value(self): |
589 | + serial_request = """ |
590 | +<server xmlns="http://docs.rackspacecloud.com/servers/api/v1.0" |
591 | + name="new-server-test" imageId="1" flavorId="1"> |
592 | + <metadata> |
593 | + <meta key="alpha"/> |
594 | + <meta key="delta"/> |
595 | + </metadata> |
596 | +</server>""" |
597 | + request = self.deserializer.deserialize(serial_request) |
598 | + expected = {"alpha": "", "delta": ""} |
599 | + self.assertEquals(request["server"]["metadata"], expected) |
600 | + |
601 | + def test_request_with_metadata_missing_key(self): |
602 | + serial_request = """ |
603 | +<server xmlns="http://docs.rackspacecloud.com/servers/api/v1.0" |
604 | + name="new-server-test" imageId="1" flavorId="1"> |
605 | + <metadata> |
606 | + <meta>beta</meta> |
607 | + </metadata> |
608 | +</server>""" |
609 | + request = self.deserializer.deserialize(serial_request) |
610 | + expected = {"": "beta"} |
611 | + self.assertEquals(request["server"]["metadata"], expected) |
612 | + |
613 | + def test_request_with_two_metadata_missing_key(self): |
614 | + serial_request = """ |
615 | +<server xmlns="http://docs.rackspacecloud.com/servers/api/v1.0" |
616 | + name="new-server-test" imageId="1" flavorId="1"> |
617 | + <metadata> |
618 | + <meta>beta</meta> |
619 | + <meta>gamma</meta> |
620 | + </metadata> |
621 | +</server>""" |
622 | + request = self.deserializer.deserialize(serial_request) |
623 | + expected = {"": "gamma"} |
624 | + self.assertEquals(request["server"]["metadata"], expected) |
625 | + |
626 | + def test_request_with_metadata_duplicate_key(self): |
627 | + serial_request = """ |
628 | +<server xmlns="http://docs.rackspacecloud.com/servers/api/v1.0" |
629 | + name="new-server-test" imageId="1" flavorId="1"> |
630 | + <metadata> |
631 | + <meta key="foo">bar</meta> |
632 | + <meta key="foo">baz</meta> |
633 | + </metadata> |
634 | +</server>""" |
635 | + request = self.deserializer.deserialize(serial_request) |
636 | + expected = {"foo": "baz"} |
637 | + self.assertEquals(request["server"]["metadata"], expected) |
638 | + |
639 | + def test_canonical_request_from_docs(self): |
640 | + serial_request = """ |
641 | +<server xmlns="http://docs.rackspacecloud.com/servers/api/v1.0" |
642 | + name="new-server-test" imageId="1" flavorId="1"> |
643 | + <metadata> |
644 | + <meta key="My Server Name">Apache1</meta> |
645 | + </metadata> |
646 | + <personality> |
647 | + <file path="/etc/banner.txt">\ |
648 | +ICAgICAgDQoiQSBjbG91ZCBkb2VzIG5vdCBrbm93IHdoeSBp\ |
649 | +dCBtb3ZlcyBpbiBqdXN0IHN1Y2ggYSBkaXJlY3Rpb24gYW5k\ |
650 | +IGF0IHN1Y2ggYSBzcGVlZC4uLkl0IGZlZWxzIGFuIGltcHVs\ |
651 | +c2lvbi4uLnRoaXMgaXMgdGhlIHBsYWNlIHRvIGdvIG5vdy4g\ |
652 | +QnV0IHRoZSBza3kga25vd3MgdGhlIHJlYXNvbnMgYW5kIHRo\ |
653 | +ZSBwYXR0ZXJucyBiZWhpbmQgYWxsIGNsb3VkcywgYW5kIHlv\ |
654 | +dSB3aWxsIGtub3csIHRvbywgd2hlbiB5b3UgbGlmdCB5b3Vy\ |
655 | +c2VsZiBoaWdoIGVub3VnaCB0byBzZWUgYmV5b25kIGhvcml6\ |
656 | +b25zLiINCg0KLVJpY2hhcmQgQmFjaA==</file> |
657 | + </personality> |
658 | +</server>""" |
659 | + expected = {"server": { |
660 | + "name": "new-server-test", |
661 | + "imageId": "1", |
662 | + "flavorId": "1", |
663 | + "metadata": { |
664 | + "My Server Name": "Apache1", |
665 | + }, |
666 | + "personality": [ |
667 | + { |
668 | + "path": "/etc/banner.txt", |
669 | + "contents": """\ |
670 | +ICAgICAgDQoiQSBjbG91ZCBkb2VzIG5vdCBrbm93IHdoeSBp\ |
671 | +dCBtb3ZlcyBpbiBqdXN0IHN1Y2ggYSBkaXJlY3Rpb24gYW5k\ |
672 | +IGF0IHN1Y2ggYSBzcGVlZC4uLkl0IGZlZWxzIGFuIGltcHVs\ |
673 | +c2lvbi4uLnRoaXMgaXMgdGhlIHBsYWNlIHRvIGdvIG5vdy4g\ |
674 | +QnV0IHRoZSBza3kga25vd3MgdGhlIHJlYXNvbnMgYW5kIHRo\ |
675 | +ZSBwYXR0ZXJucyBiZWhpbmQgYWxsIGNsb3VkcywgYW5kIHlv\ |
676 | +dSB3aWxsIGtub3csIHRvbywgd2hlbiB5b3UgbGlmdCB5b3Vy\ |
677 | +c2VsZiBoaWdoIGVub3VnaCB0byBzZWUgYmV5b25kIGhvcml6\ |
678 | +b25zLiINCg0KLVJpY2hhcmQgQmFjaA==""", |
679 | + }, |
680 | + ], |
681 | + }} |
682 | + request = self.deserializer.deserialize(serial_request) |
683 | + self.assertEqual(request, expected) |
684 | + |
685 | + |
686 | +class TestServerInstanceCreation(test.TestCase): |
687 | + |
688 | + def setUp(self): |
689 | + super(TestServerInstanceCreation, self).setUp() |
690 | + self.stubs = stubout.StubOutForTesting() |
691 | + fakes.FakeAuthManager.auth_data = {} |
692 | + fakes.FakeAuthDatabase.data = {} |
693 | + fakes.stub_out_auth(self.stubs) |
694 | + fakes.stub_out_key_pair_funcs(self.stubs) |
695 | + self.allow_admin = FLAGS.allow_admin_api |
696 | + |
697 | + def tearDown(self): |
698 | + self.stubs.UnsetAll() |
699 | + FLAGS.allow_admin_api = self.allow_admin |
700 | + super(TestServerInstanceCreation, self).tearDown() |
701 | + |
702 | + def _setup_mock_compute_api_for_personality(self): |
703 | + |
704 | + class MockComputeAPI(object): |
705 | + |
706 | + def __init__(self): |
707 | + self.injected_files = None |
708 | + |
709 | + def create(self, *args, **kwargs): |
710 | + if 'injected_files' in kwargs: |
711 | + self.injected_files = kwargs['injected_files'] |
712 | + else: |
713 | + self.injected_files = None |
714 | + return [{'id': '1234', 'display_name': 'fakeinstance'}] |
715 | + |
716 | + def set_admin_password(self, *args, **kwargs): |
717 | + pass |
718 | + |
719 | + def make_stub_method(canned_return): |
720 | + def stub_method(*args, **kwargs): |
721 | + return canned_return |
722 | + return stub_method |
723 | + |
724 | + compute_api = MockComputeAPI() |
725 | + self.stubs.Set(nova.compute, 'API', make_stub_method(compute_api)) |
726 | + self.stubs.Set(nova.api.openstack.servers.Controller, |
727 | + '_get_kernel_ramdisk_from_image', make_stub_method((1, 1))) |
728 | + self.stubs.Set(nova.api.openstack.common, |
729 | + 'get_image_id_from_image_hash', make_stub_method(2)) |
730 | + return compute_api |
731 | + |
732 | + def _create_personality_request_dict(self, personality_files): |
733 | + server = {} |
734 | + server['name'] = 'new-server-test' |
735 | + server['imageId'] = 1 |
736 | + server['flavorId'] = 1 |
737 | + if personality_files is not None: |
738 | + personalities = [] |
739 | + for path, contents in personality_files: |
740 | + personalities.append({'path': path, 'contents': contents}) |
741 | + server['personality'] = personalities |
742 | + return {'server': server} |
743 | + |
744 | + def _get_create_request_json(self, body_dict): |
745 | + req = webob.Request.blank('/v1.0/servers') |
746 | + req.content_type = 'application/json' |
747 | + req.method = 'POST' |
748 | + req.body = json.dumps(body_dict) |
749 | + return req |
750 | + |
751 | + def _run_create_instance_with_mock_compute_api(self, request): |
752 | + compute_api = self._setup_mock_compute_api_for_personality() |
753 | + response = request.get_response(fakes.wsgi_app()) |
754 | + return compute_api, response |
755 | + |
756 | + def _format_xml_request_body(self, body_dict): |
757 | + server = body_dict['server'] |
758 | + body_parts = [] |
759 | + body_parts.extend([ |
760 | + '<?xml version="1.0" encoding="UTF-8"?>', |
761 | + '<server xmlns="http://docs.rackspacecloud.com/servers/api/v1.0"', |
762 | + ' name="%s" imageId="%s" flavorId="%s">' % ( |
763 | + server['name'], server['imageId'], server['flavorId'])]) |
764 | + if 'metadata' in server: |
765 | + metadata = server['metadata'] |
766 | + body_parts.append('<metadata>') |
767 | + for item in metadata.iteritems(): |
768 | + body_parts.append('<meta key="%s">%s</meta>' % item) |
769 | + body_parts.append('</metadata>') |
770 | + if 'personality' in server: |
771 | + personalities = server['personality'] |
772 | + body_parts.append('<personality>') |
773 | + for file in personalities: |
774 | + item = (file['path'], file['contents']) |
775 | + body_parts.append('<file path="%s">%s</file>' % item) |
776 | + body_parts.append('</personality>') |
777 | + body_parts.append('</server>') |
778 | + return ''.join(body_parts) |
779 | + |
780 | + def _get_create_request_xml(self, body_dict): |
781 | + req = webob.Request.blank('/v1.0/servers') |
782 | + req.content_type = 'application/xml' |
783 | + req.accept = 'application/xml' |
784 | + req.method = 'POST' |
785 | + req.body = self._format_xml_request_body(body_dict) |
786 | + return req |
787 | + |
788 | + def _create_instance_with_personality_json(self, personality): |
789 | + body_dict = self._create_personality_request_dict(personality) |
790 | + request = self._get_create_request_json(body_dict) |
791 | + compute_api, response = \ |
792 | + self._run_create_instance_with_mock_compute_api(request) |
793 | + return request, response, compute_api.injected_files |
794 | + |
795 | + def _create_instance_with_personality_xml(self, personality): |
796 | + body_dict = self._create_personality_request_dict(personality) |
797 | + request = self._get_create_request_xml(body_dict) |
798 | + compute_api, response = \ |
799 | + self._run_create_instance_with_mock_compute_api(request) |
800 | + return request, response, compute_api.injected_files |
801 | + |
802 | + def test_create_instance_with_no_personality(self): |
803 | + request, response, injected_files = \ |
804 | + self._create_instance_with_personality_json(personality=None) |
805 | + self.assertEquals(response.status_int, 200) |
806 | + self.assertEquals(injected_files, []) |
807 | + |
808 | + def test_create_instance_with_no_personality_xml(self): |
809 | + request, response, injected_files = \ |
810 | + self._create_instance_with_personality_xml(personality=None) |
811 | + self.assertEquals(response.status_int, 200) |
812 | + self.assertEquals(injected_files, []) |
813 | + |
814 | + def test_create_instance_with_personality(self): |
815 | + path = '/my/file/path' |
816 | + contents = '#!/bin/bash\necho "Hello, World!"\n' |
817 | + b64contents = base64.b64encode(contents) |
818 | + personality = [(path, b64contents)] |
819 | + request, response, injected_files = \ |
820 | + self._create_instance_with_personality_json(personality) |
821 | + self.assertEquals(response.status_int, 200) |
822 | + self.assertEquals(injected_files, [(path, contents)]) |
823 | + |
824 | + def test_create_instance_with_personality_xml(self): |
825 | + path = '/my/file/path' |
826 | + contents = '#!/bin/bash\necho "Hello, World!"\n' |
827 | + b64contents = base64.b64encode(contents) |
828 | + personality = [(path, b64contents)] |
829 | + request, response, injected_files = \ |
830 | + self._create_instance_with_personality_xml(personality) |
831 | + self.assertEquals(response.status_int, 200) |
832 | + self.assertEquals(injected_files, [(path, contents)]) |
833 | + |
834 | + def test_create_instance_with_personality_no_path(self): |
835 | + personality = [('/remove/this/path', |
836 | + base64.b64encode('my\n\file\ncontents'))] |
837 | + body_dict = self._create_personality_request_dict(personality) |
838 | + del body_dict['server']['personality'][0]['path'] |
839 | + request = self._get_create_request_json(body_dict) |
840 | + compute_api, response = \ |
841 | + self._run_create_instance_with_mock_compute_api(request) |
842 | + self.assertEquals(response.status_int, 400) |
843 | + self.assertEquals(compute_api.injected_files, None) |
844 | + |
845 | + def _test_create_instance_with_personality_no_path_xml(self): |
846 | + personality = [('/remove/this/path', |
847 | + base64.b64encode('my\n\file\ncontents'))] |
848 | + body_dict = self._create_personality_request_dict(personality) |
849 | + request = self._get_create_request_xml(body_dict) |
850 | + request.body = request.body.replace(' path="/remove/this/path"', '') |
851 | + compute_api, response = \ |
852 | + self._run_create_instance_with_mock_compute_api(request) |
853 | + self.assertEquals(response.status_int, 400) |
854 | + self.assertEquals(compute_api.injected_files, None) |
855 | + |
856 | + def test_create_instance_with_personality_no_contents(self): |
857 | + personality = [('/test/path', |
858 | + base64.b64encode('remove\nthese\ncontents'))] |
859 | + body_dict = self._create_personality_request_dict(personality) |
860 | + del body_dict['server']['personality'][0]['contents'] |
861 | + request = self._get_create_request_json(body_dict) |
862 | + compute_api, response = \ |
863 | + self._run_create_instance_with_mock_compute_api(request) |
864 | + self.assertEquals(response.status_int, 400) |
865 | + self.assertEquals(compute_api.injected_files, None) |
866 | + |
867 | + def test_create_instance_with_personality_not_a_list(self): |
868 | + personality = [('/test/path', base64.b64encode('test\ncontents\n'))] |
869 | + body_dict = self._create_personality_request_dict(personality) |
870 | + body_dict['server']['personality'] = \ |
871 | + body_dict['server']['personality'][0] |
872 | + request = self._get_create_request_json(body_dict) |
873 | + compute_api, response = \ |
874 | + self._run_create_instance_with_mock_compute_api(request) |
875 | + self.assertEquals(response.status_int, 400) |
876 | + self.assertEquals(compute_api.injected_files, None) |
877 | + |
878 | + def test_create_instance_with_personality_with_non_b64_content(self): |
879 | + path = '/my/file/path' |
880 | + contents = '#!/bin/bash\necho "Oh no!"\n' |
881 | + personality = [(path, contents)] |
882 | + request, response, injected_files = \ |
883 | + self._create_instance_with_personality_json(personality) |
884 | + self.assertEquals(response.status_int, 400) |
885 | + self.assertEquals(injected_files, None) |
886 | + |
887 | + def test_create_instance_with_three_personalities(self): |
888 | + files = [ |
889 | + ('/etc/sudoers', 'ALL ALL=NOPASSWD: ALL\n'), |
890 | + ('/etc/motd', 'Enjoy your root access!\n'), |
891 | + ('/etc/dovecot.conf', 'dovecot\nconfig\nstuff\n'), |
892 | + ] |
893 | + personality = [] |
894 | + for path, content in files: |
895 | + personality.append((path, base64.b64encode(content))) |
896 | + request, response, injected_files = \ |
897 | + self._create_instance_with_personality_json(personality) |
898 | + self.assertEquals(response.status_int, 200) |
899 | + self.assertEquals(injected_files, files) |
900 | + |
901 | + def test_create_instance_personality_empty_content(self): |
902 | + path = '/my/file/path' |
903 | + contents = '' |
904 | + personality = [(path, contents)] |
905 | + request, response, injected_files = \ |
906 | + self._create_instance_with_personality_json(personality) |
907 | + self.assertEquals(response.status_int, 200) |
908 | + self.assertEquals(injected_files, [(path, contents)]) |
909 | + |
910 | + def test_create_instance_admin_pass_json(self): |
911 | + request, response, dummy = \ |
912 | + self._create_instance_with_personality_json(None) |
913 | + self.assertEquals(response.status_int, 200) |
914 | + response = json.loads(response.body) |
915 | + self.assertTrue('adminPass' in response['server']) |
916 | + self.assertTrue(response['server']['adminPass'].startswith('fake')) |
917 | + |
918 | + def test_create_instance_admin_pass_xml(self): |
919 | + request, response, dummy = \ |
920 | + self._create_instance_with_personality_xml(None) |
921 | + self.assertEquals(response.status_int, 200) |
922 | + dom = minidom.parseString(response.body) |
923 | + server = dom.childNodes[0] |
924 | + self.assertEquals(server.nodeName, 'server') |
925 | + self.assertTrue(server.getAttribute('adminPass').startswith('fake')) |
926 | + |
927 | + |
928 | if __name__ == "__main__": |
929 | unittest.main() |
930 | |
931 | === modified file 'nova/tests/test_quota.py' |
932 | --- nova/tests/test_quota.py 2011-03-07 01:25:01 +0000 |
933 | +++ nova/tests/test_quota.py 2011-03-16 19:19:09 +0000 |
934 | @@ -33,6 +33,12 @@ |
935 | |
936 | |
937 | class QuotaTestCase(test.TestCase): |
938 | + |
939 | + class StubImageService(object): |
940 | + |
941 | + def show(self, *args, **kwargs): |
942 | + return {"properties": {}} |
943 | + |
944 | def setUp(self): |
945 | super(QuotaTestCase, self).setUp() |
946 | self.flags(connection_type='fake', |
947 | @@ -193,3 +199,67 @@ |
948 | instance_type='m1.small', |
949 | image_id='fake', |
950 | metadata=metadata) |
951 | + |
952 | + def test_allowed_injected_files(self): |
953 | + self.assertEqual( |
954 | + quota.allowed_injected_files(self.context), |
955 | + FLAGS.quota_max_injected_files) |
956 | + |
957 | + def _create_with_injected_files(self, files): |
958 | + api = compute.API(image_service=self.StubImageService()) |
959 | + api.create(self.context, min_count=1, max_count=1, |
960 | + instance_type='m1.small', image_id='fake', |
961 | + injected_files=files) |
962 | + |
963 | + def test_no_injected_files(self): |
964 | + api = compute.API(image_service=self.StubImageService()) |
965 | + api.create(self.context, instance_type='m1.small', image_id='fake') |
966 | + |
967 | + def test_max_injected_files(self): |
968 | + files = [] |
969 | + for i in xrange(FLAGS.quota_max_injected_files): |
970 | + files.append(('/my/path%d' % i, 'config = test\n')) |
971 | + self._create_with_injected_files(files) # no QuotaError |
972 | + |
973 | + def test_too_many_injected_files(self): |
974 | + files = [] |
975 | + for i in xrange(FLAGS.quota_max_injected_files + 1): |
976 | + files.append(('/my/path%d' % i, 'my\ncontent%d\n' % i)) |
977 | + self.assertRaises(quota.QuotaError, |
978 | + self._create_with_injected_files, files) |
979 | + |
980 | + def test_allowed_injected_file_content_bytes(self): |
981 | + self.assertEqual( |
982 | + quota.allowed_injected_file_content_bytes(self.context), |
983 | + FLAGS.quota_max_injected_file_content_bytes) |
984 | + |
985 | + def test_max_injected_file_content_bytes(self): |
986 | + max = FLAGS.quota_max_injected_file_content_bytes |
987 | + content = ''.join(['a' for i in xrange(max)]) |
988 | + files = [('/test/path', content)] |
989 | + self._create_with_injected_files(files) # no QuotaError |
990 | + |
991 | + def test_too_many_injected_file_content_bytes(self): |
992 | + max = FLAGS.quota_max_injected_file_content_bytes |
993 | + content = ''.join(['a' for i in xrange(max + 1)]) |
994 | + files = [('/test/path', content)] |
995 | + self.assertRaises(quota.QuotaError, |
996 | + self._create_with_injected_files, files) |
997 | + |
998 | + def test_allowed_injected_file_path_bytes(self): |
999 | + self.assertEqual( |
1000 | + quota.allowed_injected_file_path_bytes(self.context), |
1001 | + FLAGS.quota_max_injected_file_path_bytes) |
1002 | + |
1003 | + def test_max_injected_file_path_bytes(self): |
1004 | + max = FLAGS.quota_max_injected_file_path_bytes |
1005 | + path = ''.join(['a' for i in xrange(max)]) |
1006 | + files = [(path, 'config = quotatest')] |
1007 | + self._create_with_injected_files(files) # no QuotaError |
1008 | + |
1009 | + def test_too_many_injected_file_path_bytes(self): |
1010 | + max = FLAGS.quota_max_injected_file_path_bytes |
1011 | + path = ''.join(['a' for i in xrange(max + 1)]) |
1012 | + files = [(path, 'config = quotatest')] |
1013 | + self.assertRaises(quota.QuotaError, |
1014 | + self._create_with_injected_files, files) |
1015 | |
1016 | === modified file 'nova/utils.py' |
1017 | --- nova/utils.py 2011-03-15 18:24:07 +0000 |
1018 | +++ nova/utils.py 2011-03-16 19:19:09 +0000 |
1019 | @@ -538,18 +538,6 @@ |
1020 | return wrap |
1021 | |
1022 | |
1023 | -def ensure_b64_encoding(val): |
1024 | - """Safety method to ensure that values expected to be base64-encoded |
1025 | - actually are. If they are, the value is returned unchanged. Otherwise, |
1026 | - the encoded value is returned. |
1027 | - """ |
1028 | - try: |
1029 | - dummy = base64.decode(val) |
1030 | - return val |
1031 | - except TypeError: |
1032 | - return base64.b64encode(val) |
1033 | - |
1034 | - |
1035 | def get_from_path(items, path): |
1036 | """ Returns a list of items matching the specified path. Takes an |
1037 | XPath-like expression e.g. prop1/prop2/prop3, and for each item in items, |
1038 | |
1039 | === modified file 'nova/virt/xenapi/vmops.py' |
1040 | --- nova/virt/xenapi/vmops.py 2011-03-12 00:00:34 +0000 |
1041 | +++ nova/virt/xenapi/vmops.py 2011-03-16 19:19:09 +0000 |
1042 | @@ -19,6 +19,7 @@ |
1043 | Management class for VM-related functions (spawn, reboot, etc). |
1044 | """ |
1045 | |
1046 | +import base64 |
1047 | import json |
1048 | import M2Crypto |
1049 | import os |
1050 | @@ -136,19 +137,20 @@ |
1051 | LOG.info(_('Spawning VM %(instance_name)s created %(vm_ref)s.') |
1052 | % locals()) |
1053 | |
1054 | - def _inject_onset_files(): |
1055 | - onset_files = instance.onset_files |
1056 | - if onset_files: |
1057 | + def _inject_files(): |
1058 | + injected_files = instance.injected_files |
1059 | + if injected_files: |
1060 | # Check if this is a JSON-encoded string and convert if needed. |
1061 | - if isinstance(onset_files, basestring): |
1062 | + if isinstance(injected_files, basestring): |
1063 | try: |
1064 | - onset_files = json.loads(onset_files) |
1065 | + injected_files = json.loads(injected_files) |
1066 | except ValueError: |
1067 | - LOG.exception(_("Invalid value for onset_files: '%s'") |
1068 | - % onset_files) |
1069 | - onset_files = [] |
1070 | + LOG.exception( |
1071 | + _("Invalid value for injected_files: '%s'") |
1072 | + % injected_files) |
1073 | + injected_files = [] |
1074 | # Inject any files, if specified |
1075 | - for path, contents in instance.onset_files: |
1076 | + for path, contents in instance.injected_files: |
1077 | LOG.debug(_("Injecting file path: '%s'") % path) |
1078 | self.inject_file(instance, path, contents) |
1079 | # NOTE(armando): Do we really need to do this in virt? |
1080 | @@ -164,7 +166,7 @@ |
1081 | if state == power_state.RUNNING: |
1082 | LOG.debug(_('Instance %s: booted'), instance_name) |
1083 | timer.stop() |
1084 | - _inject_onset_files() |
1085 | + _inject_files() |
1086 | return True |
1087 | except Exception, exc: |
1088 | LOG.warn(exc) |
1089 | @@ -408,17 +410,16 @@ |
1090 | raise RuntimeError(resp_dict['message']) |
1091 | return resp_dict['message'] |
1092 | |
1093 | - def inject_file(self, instance, b64_path, b64_contents): |
1094 | + def inject_file(self, instance, path, contents): |
1095 | """Write a file to the VM instance. The path to which it is to be |
1096 | - written and the contents of the file need to be supplied; both should |
1097 | + written and the contents of the file need to be supplied; both will |
1098 | be base64-encoded to prevent errors with non-ASCII characters being |
1099 | transmitted. If the agent does not support file injection, or the user |
1100 | has disabled it, a NotImplementedError will be raised. |
1101 | """ |
1102 | - # Files/paths *should* be base64-encoded at this point, but |
1103 | - # double-check to make sure. |
1104 | - b64_path = utils.ensure_b64_encoding(b64_path) |
1105 | - b64_contents = utils.ensure_b64_encoding(b64_contents) |
1106 | + # Files/paths must be base64-encoded for transmission to agent |
1107 | + b64_path = base64.b64encode(path) |
1108 | + b64_contents = base64.b64encode(contents) |
1109 | |
1110 | # Need to uniquely identify this request. |
1111 | transaction_id = str(uuid.uuid4()) |
Sorry, maybe some of this should have gone in the original description. . .
Description:
This merge adds support for personality files to the openstack api. It leverages previous work which added this functionality to the compute api, compute manager, and xen api.
Stuff you should know up front:
- deserialization
I had to add a custom deserializer for server create requests. This deserializer is only called if the content type is application/xml. Adding this was the least intrusive way for me to get around the complete lack of support for slightly more complicated xml handling in the default serializer.
- variable name change
I changed onset_files to personality_files in the compute api at a few layers. I did this because the compute_api handles the quota enforcement, but unfortunately right now quota error messages appear to bubble up directly into the os api http error responses. It would be confusing for an os api user to give an invalid personality and get an error about onset_files.
- changes to base64 handling
Previously there was a safety function that was designed to ensure that a string is base64 encoded, and encode it if it is not. In fact this function is quite deceptive. As an example, for '/etc/passwd' it would return 'L2V0Yy9wYXNzd2Q=', but for '/etc/sudoers' it would return '/etc/sudoers', because the latter is already a valid base64 string. To remedy this problem, I removed the function and replaced its usage with stronger function contracts.
- additional flags
To support the quota on personality files, file paths, and file contents, I added a few new flags to nova/quota.py.