Merge lp:~markwash/nova/lp727225 into lp:~hudson-openstack/nova/trunk

Proposed by Mark Washenberger
Status: Merged
Approved by: Jay Pipes
Approved revision: 799
Merged at revision: 814
Proposed branch: lp:~markwash/nova/lp727225
Merge into: lp:~hudson-openstack/nova/trunk
Diff against target: 1111 lines (+816/-55)
11 files modified
Authors (+1/-0)
nova/api/openstack/servers.py (+152/-13)
nova/compute/api.py (+23/-2)
nova/compute/manager.py (+3/-9)
nova/db/sqlalchemy/models.py (+1/-1)
nova/quota.py (+22/-0)
nova/tests/api/openstack/fakes.py (+0/-2)
nova/tests/api/openstack/test_servers.py (+527/-0)
nova/tests/test_quota.py (+70/-0)
nova/utils.py (+0/-12)
nova/virt/xenapi/vmops.py (+17/-16)
To merge this branch: bzr merge lp:~markwash/nova/lp727225
Reviewer Review Type Date Requested Status
Jay Pipes (community) Approve
Brian Lamar (community) Approve
Eric Day (community) Approve
Review via email: mp+52754@code.launchpad.net

Commit message

Fix lp727225 by adding support for personality files to the openstack api.

Description:

This merge adds support for personality files to the openstack api. It leverages previous work which added this functionality to the compute api, compute manager, and xen api.

Description of the change

Fix lp727225 by adding support for personality files to the openstack api.

To post a comment you must log in.
Revision history for this message
Mark Washenberger (markwash) wrote :

Sorry, maybe some of this should have gone in the original description. . .

Description:

This merge adds support for personality files to the openstack api. It leverages previous work which added this functionality to the compute api, compute manager, and xen api.

Stuff you should know up front:

- deserialization
  I had to add a custom deserializer for server create requests. This deserializer is only called if the content type is application/xml. Adding this was the least intrusive way for me to get around the complete lack of support for slightly more complicated xml handling in the default serializer.
- variable name change
  I changed onset_files to personality_files in the compute api at a few layers. I did this because the compute_api handles the quota enforcement, but unfortunately right now quota error messages appear to bubble up directly into the os api http error responses. It would be confusing for an os api user to give an invalid personality and get an error about onset_files.
- changes to base64 handling
  Previously there was a safety function that was designed to ensure that a string is base64 encoded, and encode it if it is not. In fact this function is quite deceptive. As an example, for '/etc/passwd' it would return 'L2V0Yy9wYXNzd2Q=', but for '/etc/sudoers' it would return '/etc/sudoers', because the latter is already a valid base64 string. To remedy this problem, I removed the function and replaced its usage with stronger function contracts.
- additional flags
  To support the quota on personality files, file paths, and file contents, I added a few new flags to nova/quota.py.

Revision history for this message
Christopher MacGown (0x44) wrote :

Needs a docstring in _check_personality_file_quota otherwise will approve

Revision history for this message
Mark Washenberger (markwash) wrote :

Added a missing docstring to my branch

Revision history for this message
Eric Day (eday) wrote :

There is some inconsistency in naming with "personality" and "onset". The nova/api/* can be whatever the external API uses, but within nova/compute/* and nova/db/* the name should be the same. For example, nova/compute/api.py uses personality and nova/compute/manager.py uses onset.

review: Needs Fixing
Revision history for this message
Mark Washenberger (markwash) wrote :

> There is some inconsistency in naming with "personality" and "onset". The
> nova/api/* can be whatever the external API uses, but within nova/compute/*
> and nova/db/* the name should be the same. For example, nova/compute/api.py
> uses personality and nova/compute/manager.py uses onset.

Thanks for having a look Eric.

I'm definitely a bit nervous about the naming change. Let me put forward my reasoning, and then hopefully I can beg a little more guidance about how to address this.

The reason why I changed the naming down into the compute layer is because the text of the QuotaErrors associated with onset/personality files is what is actually used in the http error message. So, if I keep the name onset_files and make it consistent throughout the compute service layer, then when a user submits an invalid number of personality files, his error message will complain about too many onset files.

I would also suggest that the only place where onset_files remains in the compute manager is in instance_ref, which is actually coming out of the data layer. Therefore, the compute layer could be seen as a sort of adapter on top of the data layer for this particular naming issue.

Here are a couple of ways I could solve this differently:

1) make the quota error text say "personality" but change the variable names back to onset_files. This would leave a different kind of inconsistency in the compute service layer.
2) create new quotaerror exceptions that are specific to personality files, and catch them in the os api. The only drawback I see here is that I don't like to get trigger happy with exception types.
3) figure out some fancy way of using internationalization?

Anyone got any suggestions? I want to do the right thing here.

Revision history for this message
Eric Day (eday) wrote :

We should probably be raising an exception with data limits but no text from nova/compute/* and let the API handle creating a message. This allows different APIs to do whatever they need with the exception and data, especially since not all APIs will want to even report a text message in the future. This allows us to keep names consistent within API boundaries.

As far as if we use "personality" or "onset", I don't really care (although both are not immediately obvious that they are injected files), I just think it's important to have consistency. :)

Revision history for this message
Mark Washenberger (markwash) wrote :

Updated to return to consistent use of onset_files. Handles http error codes in the os api by checking the quota error code and raising a new http exception if it is a quota error where the message won't make sense to an os api client.

Revision history for this message
Eric Day (eday) wrote :

141: Any way to make this generic and plug into the common deserializer in some way? Possibly passing options to trigger specific actions for certain elements?

243: any reason you assign the result of quota check (or why quota check returns anything)? this should always be a RO operation, no?

Other than that, looks good to me!

review: Needs Information
Revision history for this message
Mark Washenberger (markwash) wrote :

> 141: Any way to make this generic and plug into the common deserializer in
> some way? Possibly passing options to trigger specific actions for certain
> elements?

I think it would require replacing the common serializer in order to make this work. In fact, I'd like to see serialization become more of an api- or controller-specific activity. However, it seems out of scope for this particular merge.

> 243: any reason you assign the result of quota check (or why quota check
> returns anything)? this should always be a RO operation, no?

I thought I was being consistent with other places in that function--but looking again I don't think so anymore. So I've refactored it in the latest commit.

Thanks!

lp:~markwash/nova/lp727225 updated
791. By Mark Washenberger

merge lp:nova

792. By Mark Washenberger

refactor onset_files quota checking

Revision history for this message
Jay Pipes (jaypipes) wrote :

Hi!

Good stuff :)

As mentioned on IRC, I'd suggest a global search and replace:

s/onset_files/injected_files/g

104 + expl = 'Bad personality format: missing %s' % key
107 + raise exc.HTTPBadRequest(explanation='Bad personality format')
111 + msg = 'Personality content for %s cannot be decoded' % path
121 + expl = "Personality file limit exceeded"
124 + expl = "Personality file path too long"
127 + expl = "Personality file content too long"

Please i18n all those! If you're unclear on how to do that, let us know.

309 + return int(FLAGS.quota_max_onset_files)
314 + return int(FLAGS.quota_max_onset_file_content_bytes)
319 + return int(FLAGS.quota_max_onset_file_path_bytes)

You used flags.DEFINE_integer, so the int()'s above are redundant. Not sure any of those functions are really necessary?

Throughout the test cases in TestServerCreateRequestXMLDeserializer, you put backslashes on the end of text lines, when the text is encloses in """'s. This is incorrect and these need to be removed. As an example:

364 + def test_minimal_request(self):
365 + serial_request = """
366 +<server xmlns="http://docs.rackspacecloud.com/servers/api/v1.0"\
367 + name="new-server-test" imageId="1" flavorId="1"/>"""
368 + request = self.deserializer.deserialize(serial_request)
369 + expected = {"server": {
370 + "name": "new-server-test",
371 + "imageId": "1",
372 + "flavorId": "1",
373 + }}
374 + self.assertEquals(request, expected)

Should be:

def test_minimal_request(self):
     serial_request = """
<server xmlns="http://docs.rackspacecloud.com/servers/api/v1.0"
name="new-server-test" imageId="1" flavorId="1"/>"""
    request = self.deserializer.deserialize(serial_request)
    expected = {"server": {
        "name": "new-server-test",
        "imageId": "1",
        "flavorId": "1"}}
    self.assertEquals(request, expected)

This is important, because currently, you are testing an invalid XML string (having a \ between attributes and in other places, having a \ between element tags) and the serializer tests are apparently passing.

507 + def test_request_with_metadata_missing_key(self):
508 + serial_request = """
509 +<server xmlns="http://docs.rackspacecloud.com/servers/api/v1.0"\
510 + name="new-server-test" imageId="1" flavorId="1">\
511 +<metadata><meta>beta</meta></metadata></server>"""
512 + request = self.deserializer.deserialize(serial_request)
513 + expected = {"": "beta"}
514 + self.assertEquals(request["server"]["metadata"], expected)

The test case above technically passes. However, try adding in an additional <meta>gamma</meta> and see if the test passes ;)

Generally, when testing the parsing/deserialization of lists/dicts, you will want to add at least one test that has >1 element in the list being tested, and check that tests with one element work the same as tests with >1 element. Please feel free to add a test case for each of these cases:

1) >1 metadata element with no key
2) >1 injected file in the personality
3) >1 personality in the server

I think you may find some interesting results :)

The tests in TestServerInstanceCreation look excellent. Nice work on those!

Cheers!
jay

review: Needs Fixing
lp:~markwash/nova/lp727225 updated
793. By Mark Washenberger

s/onset_files/injected_files/g

794. By Mark Washenberger

internationalization

795. By Mark Washenberger

pep8

796. By Mark Washenberger

ignore differently-named nodes in personality and metadata parsing

797. By Mark Washenberger

merge lp:nova

798. By Mark Washenberger

pep8

Revision history for this message
Mark Washenberger (markwash) wrote :

Thanks for the suggestions Jay.

I went ahead and renamed onset_files everywhere to injected_files. I also added internationalization to the strings you identified, and couldn't find any more that needed it.

I pulled the int cast off of the flags, but I decided to go ahead and keep the quota module functions. The don't do much at this time, but it would be better for other parts of the code not to directly access the flags.

I removed most of the backslashes in the xml tests and was pleased to find that my parser did just fine. However, inside personality and metadata extraction, it needed a little "ignore unknown" love, so I added that. I preserved the backslashes in one test, because as far as I know there cannot be newlines in the base64 contents.

I added a few more tests to the deserialization, according to your requests. Everything worked out fine.

Thanks again.

Revision history for this message
Jay Pipes (jaypipes) wrote :

Excellent work, Mark. Thanks for those additional tests and cleanups.

review: Approve
Revision history for this message
Eric Day (eday) wrote :

I'm good if folks are comfortable with accepting the one-off deserialization for now. We should address the serialization issue in another patch soon (possibly after proposal to rework plus a ML or summit discussion).

review: Approve
Revision history for this message
Brian Lamar (blamar) wrote :

Looks good to us....Mark.

review: Approve
lp:~markwash/nova/lp727225 updated
799. By Mark Washenberger

merge lp:nova and resolve conflicts

Revision history for this message
Jay Pipes (jaypipes) wrote :

approved again after merge/resolve.

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'Authors'
2--- Authors 2011-03-11 08:54:08 +0000
3+++ Authors 2011-03-16 19:19:09 +0000
4@@ -40,6 +40,7 @@
5 Kevin L. Mitchell <kevin.mitchell@rackspace.com>
6 Koji Iida <iida.koji@lab.ntt.co.jp>
7 Lorin Hochstein <lorin@isi.edu>
8+Mark Washenberger <mark.washenberger@rackspace.com>
9 Masanori Itoh <itoumsn@nttdata.co.jp>
10 Matt Dietz <matt.dietz@rackspace.com>
11 Michael Gundlach <michael.gundlach@rackspace.com>
12
13=== modified file 'nova/api/openstack/servers.py'
14--- nova/api/openstack/servers.py 2011-03-15 02:57:30 +0000
15+++ nova/api/openstack/servers.py 2011-03-16 19:19:09 +0000
16@@ -13,9 +13,11 @@
17 # License for the specific language governing permissions and limitations
18 # under the License.
19
20+import base64
21 import hashlib
22 import json
23 import traceback
24+from xml.dom import minidom
25
26 from webob import exc
27
28@@ -141,7 +143,7 @@
29
30 def create(self, req):
31 """ Creates a new server for a given user """
32- env = self._deserialize(req.body, req.get_content_type())
33+ env = self._deserialize_create(req)
34 if not env:
35 return faults.Fault(exc.HTTPUnprocessableEntity())
36
37@@ -170,18 +172,24 @@
38 for k, v in env['server']['metadata'].items():
39 metadata.append({'key': k, 'value': v})
40
41- instances = self.compute_api.create(
42- context,
43- instance_types.get_by_flavor_id(env['server']['flavorId']),
44- image_id,
45- kernel_id=kernel_id,
46- ramdisk_id=ramdisk_id,
47- display_name=env['server']['name'],
48- display_description=env['server']['name'],
49- key_name=key_name,
50- key_data=key_data,
51- metadata=metadata,
52- onset_files=env.get('onset_files', []))
53+ personality = env['server'].get('personality', [])
54+ injected_files = self._get_injected_files(personality)
55+
56+ try:
57+ instances = self.compute_api.create(
58+ context,
59+ instance_types.get_by_flavor_id(env['server']['flavorId']),
60+ image_id,
61+ kernel_id=kernel_id,
62+ ramdisk_id=ramdisk_id,
63+ display_name=env['server']['name'],
64+ display_description=env['server']['name'],
65+ key_name=key_name,
66+ key_data=key_data,
67+ metadata=metadata,
68+ injected_files=injected_files)
69+ except QuotaError as error:
70+ self._handle_quota_error(error)
71
72 server = _translate_keys(instances[0])
73 password = "%s%s" % (server['server']['name'][:4],
74@@ -191,6 +199,61 @@
75 password)
76 return server
77
78+ def _deserialize_create(self, request):
79+ """
80+ Deserialize a create request
81+
82+ Overrides normal behavior in the case of xml content
83+ """
84+ if request.content_type == "application/xml":
85+ deserializer = ServerCreateRequestXMLDeserializer()
86+ return deserializer.deserialize(request.body)
87+ else:
88+ return self._deserialize(request.body, request.get_content_type())
89+
90+ def _get_injected_files(self, personality):
91+ """
92+ Create a list of injected files from the personality attribute
93+
94+ At this time, injected_files must be formatted as a list of
95+ (file_path, file_content) pairs for compatibility with the
96+ underlying compute service.
97+ """
98+ injected_files = []
99+ for item in personality:
100+ try:
101+ path = item['path']
102+ contents = item['contents']
103+ except KeyError as key:
104+ expl = _('Bad personality format: missing %s') % key
105+ raise exc.HTTPBadRequest(explanation=expl)
106+ except TypeError:
107+ expl = _('Bad personality format')
108+ raise exc.HTTPBadRequest(explanation=expl)
109+ try:
110+ contents = base64.b64decode(contents)
111+ except TypeError:
112+ expl = _('Personality content for %s cannot be decoded') % path
113+ raise exc.HTTPBadRequest(explanation=expl)
114+ injected_files.append((path, contents))
115+ return injected_files
116+
117+ def _handle_quota_errors(self, error):
118+ """
119+ Reraise quota errors as api-specific http exceptions
120+ """
121+ if error.code == "OnsetFileLimitExceeded":
122+ expl = _("Personality file limit exceeded")
123+ raise exc.HTTPBadRequest(explanation=expl)
124+ if error.code == "OnsetFilePathLimitExceeded":
125+ expl = _("Personality file path too long")
126+ raise exc.HTTPBadRequest(explanation=expl)
127+ if error.code == "OnsetFileContentLimitExceeded":
128+ expl = _("Personality file content too long")
129+ raise exc.HTTPBadRequest(explanation=expl)
130+ # if the original error is okay, just reraise it
131+ raise error
132+
133 def update(self, req, id):
134 """ Updates the server name or password """
135 if len(req.body) == 0:
136@@ -480,3 +543,79 @@
137 _("Ramdisk not found for image %(image_id)s") % locals())
138
139 return kernel_id, ramdisk_id
140+
141+
142+class ServerCreateRequestXMLDeserializer(object):
143+ """
144+ Deserializer to handle xml-formatted server create requests.
145+
146+ Handles standard server attributes as well as optional metadata
147+ and personality attributes
148+ """
149+
150+ def deserialize(self, string):
151+ """Deserialize an xml-formatted server create request"""
152+ dom = minidom.parseString(string)
153+ server = self._extract_server(dom)
154+ return {'server': server}
155+
156+ def _extract_server(self, node):
157+ """Marshal the server attribute of a parsed request"""
158+ server = {}
159+ server_node = self._find_first_child_named(node, 'server')
160+ for attr in ["name", "imageId", "flavorId"]:
161+ server[attr] = server_node.getAttribute(attr)
162+ metadata = self._extract_metadata(server_node)
163+ if metadata is not None:
164+ server["metadata"] = metadata
165+ personality = self._extract_personality(server_node)
166+ if personality is not None:
167+ server["personality"] = personality
168+ return server
169+
170+ def _extract_metadata(self, server_node):
171+ """Marshal the metadata attribute of a parsed request"""
172+ metadata_node = self._find_first_child_named(server_node, "metadata")
173+ if metadata_node is None:
174+ return None
175+ metadata = {}
176+ for meta_node in self._find_children_named(metadata_node, "meta"):
177+ key = meta_node.getAttribute("key")
178+ metadata[key] = self._extract_text(meta_node)
179+ return metadata
180+
181+ def _extract_personality(self, server_node):
182+ """Marshal the personality attribute of a parsed request"""
183+ personality_node = \
184+ self._find_first_child_named(server_node, "personality")
185+ if personality_node is None:
186+ return None
187+ personality = []
188+ for file_node in self._find_children_named(personality_node, "file"):
189+ item = {}
190+ if file_node.hasAttribute("path"):
191+ item["path"] = file_node.getAttribute("path")
192+ item["contents"] = self._extract_text(file_node)
193+ personality.append(item)
194+ return personality
195+
196+ def _find_first_child_named(self, parent, name):
197+ """Search a nodes children for the first child with a given name"""
198+ for node in parent.childNodes:
199+ if node.nodeName == name:
200+ return node
201+ return None
202+
203+ def _find_children_named(self, parent, name):
204+ """Return all of a nodes children who have the given name"""
205+ for node in parent.childNodes:
206+ if node.nodeName == name:
207+ yield node
208+
209+ def _extract_text(self, node):
210+ """Get the text field contained by the given node"""
211+ if len(node.childNodes) == 1:
212+ child = node.childNodes[0]
213+ if child.nodeType == child.TEXT_NODE:
214+ return child.nodeValue
215+ return ""
216
217=== modified file 'nova/compute/api.py'
218--- nova/compute/api.py 2011-03-14 17:59:41 +0000
219+++ nova/compute/api.py 2011-03-16 19:19:09 +0000
220@@ -80,13 +80,32 @@
221 topic,
222 {"method": "get_network_topic", "args": {'fake': 1}})
223
224+ def _check_injected_file_quota(self, context, injected_files):
225+ """
226+ Enforce quota limits on injected files
227+
228+ Raises a QuotaError if any limit is exceeded
229+ """
230+ if injected_files is None:
231+ return
232+ limit = quota.allowed_injected_files(context)
233+ if len(injected_files) > limit:
234+ raise quota.QuotaError(code="OnsetFileLimitExceeded")
235+ path_limit = quota.allowed_injected_file_path_bytes(context)
236+ content_limit = quota.allowed_injected_file_content_bytes(context)
237+ for path, content in injected_files:
238+ if len(path) > path_limit:
239+ raise quota.QuotaError(code="OnsetFilePathLimitExceeded")
240+ if len(content) > content_limit:
241+ raise quota.QuotaError(code="OnsetFileContentLimitExceeded")
242+
243 def create(self, context, instance_type,
244 image_id, kernel_id=None, ramdisk_id=None,
245 min_count=1, max_count=1,
246 display_name='', display_description='',
247 key_name=None, key_data=None, security_group='default',
248 availability_zone=None, user_data=None, metadata=[],
249- onset_files=None):
250+ injected_files=None):
251 """Create the number of instances requested if quota and
252 other arguments check out ok."""
253
254@@ -124,6 +143,8 @@
255 LOG.warn(msg)
256 raise quota.QuotaError(msg, "MetadataLimitExceeded")
257
258+ self._check_injected_file_quota(context, injected_files)
259+
260 image = self.image_service.show(context, image_id)
261
262 os_type = None
263@@ -225,7 +246,7 @@
264 "args": {"topic": FLAGS.compute_topic,
265 "instance_id": instance_id,
266 "availability_zone": availability_zone,
267- "onset_files": onset_files}})
268+ "injected_files": injected_files}})
269
270 for group_id in security_groups:
271 self.trigger_security_group_members_refresh(elevated, group_id)
272
273=== modified file 'nova/compute/manager.py'
274--- nova/compute/manager.py 2011-03-10 04:30:52 +0000
275+++ nova/compute/manager.py 2011-03-16 19:19:09 +0000
276@@ -34,7 +34,6 @@
277 :func:`nova.utils.import_object`
278 """
279
280-import base64
281 import datetime
282 import os
283 import random
284@@ -180,7 +179,7 @@
285 """Launch a new instance with specified options."""
286 context = context.elevated()
287 instance_ref = self.db.instance_get(context, instance_id)
288- instance_ref.onset_files = kwargs.get('onset_files', [])
289+ instance_ref.injected_files = kwargs.get('injected_files', [])
290 if instance_ref['name'] in self.driver.list_instances():
291 raise exception.Error(_("Instance has already been created"))
292 LOG.audit(_("instance %s: starting..."), instance_id,
293@@ -359,15 +358,10 @@
294 LOG.warn(_('trying to inject a file into a non-running '
295 'instance: %(instance_id)s (state: %(instance_state)s '
296 'expected: %(expected_state)s)') % locals())
297- # Files/paths *should* be base64-encoded at this point, but
298- # double-check to make sure.
299- b64_path = utils.ensure_b64_encoding(path)
300- b64_contents = utils.ensure_b64_encoding(file_contents)
301- plain_path = base64.b64decode(b64_path)
302 nm = instance_ref['name']
303- msg = _('instance %(nm)s: injecting file to %(plain_path)s') % locals()
304+ msg = _('instance %(nm)s: injecting file to %(path)s') % locals()
305 LOG.audit(msg)
306- self.driver.inject_file(instance_ref, b64_path, b64_contents)
307+ self.driver.inject_file(instance_ref, path, file_contents)
308
309 @exception.wrap_exception
310 @checks_instance_lock
311
312=== modified file 'nova/db/sqlalchemy/models.py'
313--- nova/db/sqlalchemy/models.py 2011-03-14 17:59:41 +0000
314+++ nova/db/sqlalchemy/models.py 2011-03-16 19:19:09 +0000
315@@ -161,7 +161,7 @@
316 class Instance(BASE, NovaBase):
317 """Represents a guest vm."""
318 __tablename__ = 'instances'
319- onset_files = []
320+ injected_files = []
321
322 id = Column(Integer, primary_key=True, autoincrement=True)
323
324
325=== modified file 'nova/quota.py'
326--- nova/quota.py 2011-02-17 23:00:18 +0000
327+++ nova/quota.py 2011-03-16 19:19:09 +0000
328@@ -37,6 +37,12 @@
329 'number of floating ips allowed per project')
330 flags.DEFINE_integer('quota_metadata_items', 128,
331 'number of metadata items allowed per instance')
332+flags.DEFINE_integer('quota_max_injected_files', 5,
333+ 'number of injected files allowed')
334+flags.DEFINE_integer('quota_max_injected_file_content_bytes', 10 * 1024,
335+ 'number of bytes allowed per injected file')
336+flags.DEFINE_integer('quota_max_injected_file_path_bytes', 255,
337+ 'number of bytes allowed per injected file path')
338
339
340 def get_quota(context, project_id):
341@@ -46,6 +52,7 @@
342 'gigabytes': FLAGS.quota_gigabytes,
343 'floating_ips': FLAGS.quota_floating_ips,
344 'metadata_items': FLAGS.quota_metadata_items}
345+
346 try:
347 quota = db.quota_get(context, project_id)
348 for key in rval.keys():
349@@ -106,6 +113,21 @@
350 return min(num_metadata_items, num_allowed_metadata_items)
351
352
353+def allowed_injected_files(context):
354+ """Return the number of injected files allowed"""
355+ return FLAGS.quota_max_injected_files
356+
357+
358+def allowed_injected_file_content_bytes(context):
359+ """Return the number of bytes allowed per injected file content"""
360+ return FLAGS.quota_max_injected_file_content_bytes
361+
362+
363+def allowed_injected_file_path_bytes(context):
364+ """Return the number of bytes allowed in an injected file path"""
365+ return FLAGS.quota_max_injected_file_path_bytes
366+
367+
368 class QuotaError(exception.ApiError):
369 """Quota Exceeeded"""
370 pass
371
372=== modified file 'nova/tests/api/openstack/fakes.py'
373--- nova/tests/api/openstack/fakes.py 2011-03-16 19:02:17 +0000
374+++ nova/tests/api/openstack/fakes.py 2011-03-16 19:19:09 +0000
375@@ -69,8 +69,6 @@
376 @webob.dec.wsgify
377 def fake_wsgi(self, req):
378 req.environ['nova.context'] = context.RequestContext(1, 1)
379- if req.body:
380- req.environ['inst_dict'] = json.loads(req.body)
381 return self.application
382
383
384
385=== modified file 'nova/tests/api/openstack/test_servers.py'
386--- nova/tests/api/openstack/test_servers.py 2011-03-16 18:49:25 +0000
387+++ nova/tests/api/openstack/test_servers.py 2011-03-16 19:19:09 +0000
388@@ -15,8 +15,11 @@
389 # License for the specific language governing permissions and limitations
390 # under the License.
391
392+import base64
393 import datetime
394 import json
395+import unittest
396+from xml.dom import minidom
397
398 import stubout
399 import webob
400@@ -607,5 +610,529 @@
401 res = req.get_response(fakes.wsgi_app())
402 self.assertEqual(res.status_int, 400)
403
404+
405+class TestServerCreateRequestXMLDeserializer(unittest.TestCase):
406+
407+ def setUp(self):
408+ self.deserializer = servers.ServerCreateRequestXMLDeserializer()
409+
410+ def test_minimal_request(self):
411+ serial_request = """
412+<server xmlns="http://docs.rackspacecloud.com/servers/api/v1.0"
413+ name="new-server-test" imageId="1" flavorId="1"/>"""
414+ request = self.deserializer.deserialize(serial_request)
415+ expected = {"server": {
416+ "name": "new-server-test",
417+ "imageId": "1",
418+ "flavorId": "1",
419+ }}
420+ self.assertEquals(request, expected)
421+
422+ def test_request_with_empty_metadata(self):
423+ serial_request = """
424+<server xmlns="http://docs.rackspacecloud.com/servers/api/v1.0"
425+ name="new-server-test" imageId="1" flavorId="1">
426+ <metadata/>
427+</server>"""
428+ request = self.deserializer.deserialize(serial_request)
429+ expected = {"server": {
430+ "name": "new-server-test",
431+ "imageId": "1",
432+ "flavorId": "1",
433+ "metadata": {},
434+ }}
435+ self.assertEquals(request, expected)
436+
437+ def test_request_with_empty_personality(self):
438+ serial_request = """
439+<server xmlns="http://docs.rackspacecloud.com/servers/api/v1.0"
440+ name="new-server-test" imageId="1" flavorId="1">
441+ <personality/>
442+</server>"""
443+ request = self.deserializer.deserialize(serial_request)
444+ expected = {"server": {
445+ "name": "new-server-test",
446+ "imageId": "1",
447+ "flavorId": "1",
448+ "personality": [],
449+ }}
450+ self.assertEquals(request, expected)
451+
452+ def test_request_with_empty_metadata_and_personality(self):
453+ serial_request = """
454+<server xmlns="http://docs.rackspacecloud.com/servers/api/v1.0"
455+ name="new-server-test" imageId="1" flavorId="1">
456+ <metadata/>
457+ <personality/>
458+</server>"""
459+ request = self.deserializer.deserialize(serial_request)
460+ expected = {"server": {
461+ "name": "new-server-test",
462+ "imageId": "1",
463+ "flavorId": "1",
464+ "metadata": {},
465+ "personality": [],
466+ }}
467+ self.assertEquals(request, expected)
468+
469+ def test_request_with_empty_metadata_and_personality_reversed(self):
470+ serial_request = """
471+<server xmlns="http://docs.rackspacecloud.com/servers/api/v1.0"
472+ name="new-server-test" imageId="1" flavorId="1">
473+ <personality/>
474+ <metadata/>
475+</server>"""
476+ request = self.deserializer.deserialize(serial_request)
477+ expected = {"server": {
478+ "name": "new-server-test",
479+ "imageId": "1",
480+ "flavorId": "1",
481+ "metadata": {},
482+ "personality": [],
483+ }}
484+ self.assertEquals(request, expected)
485+
486+ def test_request_with_one_personality(self):
487+ serial_request = """
488+<server xmlns="http://docs.rackspacecloud.com/servers/api/v1.0"
489+ name="new-server-test" imageId="1" flavorId="1">
490+ <personality>
491+ <file path="/etc/conf">aabbccdd</file>
492+ </personality>
493+</server>"""
494+ request = self.deserializer.deserialize(serial_request)
495+ expected = [{"path": "/etc/conf", "contents": "aabbccdd"}]
496+ self.assertEquals(request["server"]["personality"], expected)
497+
498+ def test_request_with_two_personalities(self):
499+ serial_request = """
500+<server xmlns="http://docs.rackspacecloud.com/servers/api/v1.0"
501+ name="new-server-test" imageId="1" flavorId="1">
502+<personality><file path="/etc/conf">aabbccdd</file>
503+<file path="/etc/sudoers">abcd</file></personality></server>"""
504+ request = self.deserializer.deserialize(serial_request)
505+ expected = [{"path": "/etc/conf", "contents": "aabbccdd"},
506+ {"path": "/etc/sudoers", "contents": "abcd"}]
507+ self.assertEquals(request["server"]["personality"], expected)
508+
509+ def test_request_second_personality_node_ignored(self):
510+ serial_request = """
511+<server xmlns="http://docs.rackspacecloud.com/servers/api/v1.0"
512+ name="new-server-test" imageId="1" flavorId="1">
513+ <personality>
514+ <file path="/etc/conf">aabbccdd</file>
515+ </personality>
516+ <personality>
517+ <file path="/etc/ignoreme">anything</file>
518+ </personality>
519+</server>"""
520+ request = self.deserializer.deserialize(serial_request)
521+ expected = [{"path": "/etc/conf", "contents": "aabbccdd"}]
522+ self.assertEquals(request["server"]["personality"], expected)
523+
524+ def test_request_with_one_personality_missing_path(self):
525+ serial_request = """
526+<server xmlns="http://docs.rackspacecloud.com/servers/api/v1.0"
527+ name="new-server-test" imageId="1" flavorId="1">
528+<personality><file>aabbccdd</file></personality></server>"""
529+ request = self.deserializer.deserialize(serial_request)
530+ expected = [{"contents": "aabbccdd"}]
531+ self.assertEquals(request["server"]["personality"], expected)
532+
533+ def test_request_with_one_personality_empty_contents(self):
534+ serial_request = """
535+<server xmlns="http://docs.rackspacecloud.com/servers/api/v1.0"
536+ name="new-server-test" imageId="1" flavorId="1">
537+<personality><file path="/etc/conf"></file></personality></server>"""
538+ request = self.deserializer.deserialize(serial_request)
539+ expected = [{"path": "/etc/conf", "contents": ""}]
540+ self.assertEquals(request["server"]["personality"], expected)
541+
542+ def test_request_with_one_personality_empty_contents_variation(self):
543+ serial_request = """
544+<server xmlns="http://docs.rackspacecloud.com/servers/api/v1.0"
545+ name="new-server-test" imageId="1" flavorId="1">
546+<personality><file path="/etc/conf"/></personality></server>"""
547+ request = self.deserializer.deserialize(serial_request)
548+ expected = [{"path": "/etc/conf", "contents": ""}]
549+ self.assertEquals(request["server"]["personality"], expected)
550+
551+ def test_request_with_one_metadata(self):
552+ serial_request = """
553+<server xmlns="http://docs.rackspacecloud.com/servers/api/v1.0"
554+ name="new-server-test" imageId="1" flavorId="1">
555+ <metadata>
556+ <meta key="alpha">beta</meta>
557+ </metadata>
558+</server>"""
559+ request = self.deserializer.deserialize(serial_request)
560+ expected = {"alpha": "beta"}
561+ self.assertEquals(request["server"]["metadata"], expected)
562+
563+ def test_request_with_two_metadata(self):
564+ serial_request = """
565+<server xmlns="http://docs.rackspacecloud.com/servers/api/v1.0"
566+ name="new-server-test" imageId="1" flavorId="1">
567+ <metadata>
568+ <meta key="alpha">beta</meta>
569+ <meta key="foo">bar</meta>
570+ </metadata>
571+</server>"""
572+ request = self.deserializer.deserialize(serial_request)
573+ expected = {"alpha": "beta", "foo": "bar"}
574+ self.assertEquals(request["server"]["metadata"], expected)
575+
576+ def test_request_with_metadata_missing_value(self):
577+ serial_request = """
578+<server xmlns="http://docs.rackspacecloud.com/servers/api/v1.0"
579+ name="new-server-test" imageId="1" flavorId="1">
580+ <metadata>
581+ <meta key="alpha"></meta>
582+ </metadata>
583+</server>"""
584+ request = self.deserializer.deserialize(serial_request)
585+ expected = {"alpha": ""}
586+ self.assertEquals(request["server"]["metadata"], expected)
587+
588+ def test_request_with_two_metadata_missing_value(self):
589+ serial_request = """
590+<server xmlns="http://docs.rackspacecloud.com/servers/api/v1.0"
591+ name="new-server-test" imageId="1" flavorId="1">
592+ <metadata>
593+ <meta key="alpha"/>
594+ <meta key="delta"/>
595+ </metadata>
596+</server>"""
597+ request = self.deserializer.deserialize(serial_request)
598+ expected = {"alpha": "", "delta": ""}
599+ self.assertEquals(request["server"]["metadata"], expected)
600+
601+ def test_request_with_metadata_missing_key(self):
602+ serial_request = """
603+<server xmlns="http://docs.rackspacecloud.com/servers/api/v1.0"
604+ name="new-server-test" imageId="1" flavorId="1">
605+ <metadata>
606+ <meta>beta</meta>
607+ </metadata>
608+</server>"""
609+ request = self.deserializer.deserialize(serial_request)
610+ expected = {"": "beta"}
611+ self.assertEquals(request["server"]["metadata"], expected)
612+
613+ def test_request_with_two_metadata_missing_key(self):
614+ serial_request = """
615+<server xmlns="http://docs.rackspacecloud.com/servers/api/v1.0"
616+ name="new-server-test" imageId="1" flavorId="1">
617+ <metadata>
618+ <meta>beta</meta>
619+ <meta>gamma</meta>
620+ </metadata>
621+</server>"""
622+ request = self.deserializer.deserialize(serial_request)
623+ expected = {"": "gamma"}
624+ self.assertEquals(request["server"]["metadata"], expected)
625+
626+ def test_request_with_metadata_duplicate_key(self):
627+ serial_request = """
628+<server xmlns="http://docs.rackspacecloud.com/servers/api/v1.0"
629+ name="new-server-test" imageId="1" flavorId="1">
630+ <metadata>
631+ <meta key="foo">bar</meta>
632+ <meta key="foo">baz</meta>
633+ </metadata>
634+</server>"""
635+ request = self.deserializer.deserialize(serial_request)
636+ expected = {"foo": "baz"}
637+ self.assertEquals(request["server"]["metadata"], expected)
638+
639+ def test_canonical_request_from_docs(self):
640+ serial_request = """
641+<server xmlns="http://docs.rackspacecloud.com/servers/api/v1.0"
642+ name="new-server-test" imageId="1" flavorId="1">
643+ <metadata>
644+ <meta key="My Server Name">Apache1</meta>
645+ </metadata>
646+ <personality>
647+ <file path="/etc/banner.txt">\
648+ICAgICAgDQoiQSBjbG91ZCBkb2VzIG5vdCBrbm93IHdoeSBp\
649+dCBtb3ZlcyBpbiBqdXN0IHN1Y2ggYSBkaXJlY3Rpb24gYW5k\
650+IGF0IHN1Y2ggYSBzcGVlZC4uLkl0IGZlZWxzIGFuIGltcHVs\
651+c2lvbi4uLnRoaXMgaXMgdGhlIHBsYWNlIHRvIGdvIG5vdy4g\
652+QnV0IHRoZSBza3kga25vd3MgdGhlIHJlYXNvbnMgYW5kIHRo\
653+ZSBwYXR0ZXJucyBiZWhpbmQgYWxsIGNsb3VkcywgYW5kIHlv\
654+dSB3aWxsIGtub3csIHRvbywgd2hlbiB5b3UgbGlmdCB5b3Vy\
655+c2VsZiBoaWdoIGVub3VnaCB0byBzZWUgYmV5b25kIGhvcml6\
656+b25zLiINCg0KLVJpY2hhcmQgQmFjaA==</file>
657+ </personality>
658+</server>"""
659+ expected = {"server": {
660+ "name": "new-server-test",
661+ "imageId": "1",
662+ "flavorId": "1",
663+ "metadata": {
664+ "My Server Name": "Apache1",
665+ },
666+ "personality": [
667+ {
668+ "path": "/etc/banner.txt",
669+ "contents": """\
670+ICAgICAgDQoiQSBjbG91ZCBkb2VzIG5vdCBrbm93IHdoeSBp\
671+dCBtb3ZlcyBpbiBqdXN0IHN1Y2ggYSBkaXJlY3Rpb24gYW5k\
672+IGF0IHN1Y2ggYSBzcGVlZC4uLkl0IGZlZWxzIGFuIGltcHVs\
673+c2lvbi4uLnRoaXMgaXMgdGhlIHBsYWNlIHRvIGdvIG5vdy4g\
674+QnV0IHRoZSBza3kga25vd3MgdGhlIHJlYXNvbnMgYW5kIHRo\
675+ZSBwYXR0ZXJucyBiZWhpbmQgYWxsIGNsb3VkcywgYW5kIHlv\
676+dSB3aWxsIGtub3csIHRvbywgd2hlbiB5b3UgbGlmdCB5b3Vy\
677+c2VsZiBoaWdoIGVub3VnaCB0byBzZWUgYmV5b25kIGhvcml6\
678+b25zLiINCg0KLVJpY2hhcmQgQmFjaA==""",
679+ },
680+ ],
681+ }}
682+ request = self.deserializer.deserialize(serial_request)
683+ self.assertEqual(request, expected)
684+
685+
686+class TestServerInstanceCreation(test.TestCase):
687+
688+ def setUp(self):
689+ super(TestServerInstanceCreation, self).setUp()
690+ self.stubs = stubout.StubOutForTesting()
691+ fakes.FakeAuthManager.auth_data = {}
692+ fakes.FakeAuthDatabase.data = {}
693+ fakes.stub_out_auth(self.stubs)
694+ fakes.stub_out_key_pair_funcs(self.stubs)
695+ self.allow_admin = FLAGS.allow_admin_api
696+
697+ def tearDown(self):
698+ self.stubs.UnsetAll()
699+ FLAGS.allow_admin_api = self.allow_admin
700+ super(TestServerInstanceCreation, self).tearDown()
701+
702+ def _setup_mock_compute_api_for_personality(self):
703+
704+ class MockComputeAPI(object):
705+
706+ def __init__(self):
707+ self.injected_files = None
708+
709+ def create(self, *args, **kwargs):
710+ if 'injected_files' in kwargs:
711+ self.injected_files = kwargs['injected_files']
712+ else:
713+ self.injected_files = None
714+ return [{'id': '1234', 'display_name': 'fakeinstance'}]
715+
716+ def set_admin_password(self, *args, **kwargs):
717+ pass
718+
719+ def make_stub_method(canned_return):
720+ def stub_method(*args, **kwargs):
721+ return canned_return
722+ return stub_method
723+
724+ compute_api = MockComputeAPI()
725+ self.stubs.Set(nova.compute, 'API', make_stub_method(compute_api))
726+ self.stubs.Set(nova.api.openstack.servers.Controller,
727+ '_get_kernel_ramdisk_from_image', make_stub_method((1, 1)))
728+ self.stubs.Set(nova.api.openstack.common,
729+ 'get_image_id_from_image_hash', make_stub_method(2))
730+ return compute_api
731+
732+ def _create_personality_request_dict(self, personality_files):
733+ server = {}
734+ server['name'] = 'new-server-test'
735+ server['imageId'] = 1
736+ server['flavorId'] = 1
737+ if personality_files is not None:
738+ personalities = []
739+ for path, contents in personality_files:
740+ personalities.append({'path': path, 'contents': contents})
741+ server['personality'] = personalities
742+ return {'server': server}
743+
744+ def _get_create_request_json(self, body_dict):
745+ req = webob.Request.blank('/v1.0/servers')
746+ req.content_type = 'application/json'
747+ req.method = 'POST'
748+ req.body = json.dumps(body_dict)
749+ return req
750+
751+ def _run_create_instance_with_mock_compute_api(self, request):
752+ compute_api = self._setup_mock_compute_api_for_personality()
753+ response = request.get_response(fakes.wsgi_app())
754+ return compute_api, response
755+
756+ def _format_xml_request_body(self, body_dict):
757+ server = body_dict['server']
758+ body_parts = []
759+ body_parts.extend([
760+ '<?xml version="1.0" encoding="UTF-8"?>',
761+ '<server xmlns="http://docs.rackspacecloud.com/servers/api/v1.0"',
762+ ' name="%s" imageId="%s" flavorId="%s">' % (
763+ server['name'], server['imageId'], server['flavorId'])])
764+ if 'metadata' in server:
765+ metadata = server['metadata']
766+ body_parts.append('<metadata>')
767+ for item in metadata.iteritems():
768+ body_parts.append('<meta key="%s">%s</meta>' % item)
769+ body_parts.append('</metadata>')
770+ if 'personality' in server:
771+ personalities = server['personality']
772+ body_parts.append('<personality>')
773+ for file in personalities:
774+ item = (file['path'], file['contents'])
775+ body_parts.append('<file path="%s">%s</file>' % item)
776+ body_parts.append('</personality>')
777+ body_parts.append('</server>')
778+ return ''.join(body_parts)
779+
780+ def _get_create_request_xml(self, body_dict):
781+ req = webob.Request.blank('/v1.0/servers')
782+ req.content_type = 'application/xml'
783+ req.accept = 'application/xml'
784+ req.method = 'POST'
785+ req.body = self._format_xml_request_body(body_dict)
786+ return req
787+
788+ def _create_instance_with_personality_json(self, personality):
789+ body_dict = self._create_personality_request_dict(personality)
790+ request = self._get_create_request_json(body_dict)
791+ compute_api, response = \
792+ self._run_create_instance_with_mock_compute_api(request)
793+ return request, response, compute_api.injected_files
794+
795+ def _create_instance_with_personality_xml(self, personality):
796+ body_dict = self._create_personality_request_dict(personality)
797+ request = self._get_create_request_xml(body_dict)
798+ compute_api, response = \
799+ self._run_create_instance_with_mock_compute_api(request)
800+ return request, response, compute_api.injected_files
801+
802+ def test_create_instance_with_no_personality(self):
803+ request, response, injected_files = \
804+ self._create_instance_with_personality_json(personality=None)
805+ self.assertEquals(response.status_int, 200)
806+ self.assertEquals(injected_files, [])
807+
808+ def test_create_instance_with_no_personality_xml(self):
809+ request, response, injected_files = \
810+ self._create_instance_with_personality_xml(personality=None)
811+ self.assertEquals(response.status_int, 200)
812+ self.assertEquals(injected_files, [])
813+
814+ def test_create_instance_with_personality(self):
815+ path = '/my/file/path'
816+ contents = '#!/bin/bash\necho "Hello, World!"\n'
817+ b64contents = base64.b64encode(contents)
818+ personality = [(path, b64contents)]
819+ request, response, injected_files = \
820+ self._create_instance_with_personality_json(personality)
821+ self.assertEquals(response.status_int, 200)
822+ self.assertEquals(injected_files, [(path, contents)])
823+
824+ def test_create_instance_with_personality_xml(self):
825+ path = '/my/file/path'
826+ contents = '#!/bin/bash\necho "Hello, World!"\n'
827+ b64contents = base64.b64encode(contents)
828+ personality = [(path, b64contents)]
829+ request, response, injected_files = \
830+ self._create_instance_with_personality_xml(personality)
831+ self.assertEquals(response.status_int, 200)
832+ self.assertEquals(injected_files, [(path, contents)])
833+
834+ def test_create_instance_with_personality_no_path(self):
835+ personality = [('/remove/this/path',
836+ base64.b64encode('my\n\file\ncontents'))]
837+ body_dict = self._create_personality_request_dict(personality)
838+ del body_dict['server']['personality'][0]['path']
839+ request = self._get_create_request_json(body_dict)
840+ compute_api, response = \
841+ self._run_create_instance_with_mock_compute_api(request)
842+ self.assertEquals(response.status_int, 400)
843+ self.assertEquals(compute_api.injected_files, None)
844+
845+ def _test_create_instance_with_personality_no_path_xml(self):
846+ personality = [('/remove/this/path',
847+ base64.b64encode('my\n\file\ncontents'))]
848+ body_dict = self._create_personality_request_dict(personality)
849+ request = self._get_create_request_xml(body_dict)
850+ request.body = request.body.replace(' path="/remove/this/path"', '')
851+ compute_api, response = \
852+ self._run_create_instance_with_mock_compute_api(request)
853+ self.assertEquals(response.status_int, 400)
854+ self.assertEquals(compute_api.injected_files, None)
855+
856+ def test_create_instance_with_personality_no_contents(self):
857+ personality = [('/test/path',
858+ base64.b64encode('remove\nthese\ncontents'))]
859+ body_dict = self._create_personality_request_dict(personality)
860+ del body_dict['server']['personality'][0]['contents']
861+ request = self._get_create_request_json(body_dict)
862+ compute_api, response = \
863+ self._run_create_instance_with_mock_compute_api(request)
864+ self.assertEquals(response.status_int, 400)
865+ self.assertEquals(compute_api.injected_files, None)
866+
867+ def test_create_instance_with_personality_not_a_list(self):
868+ personality = [('/test/path', base64.b64encode('test\ncontents\n'))]
869+ body_dict = self._create_personality_request_dict(personality)
870+ body_dict['server']['personality'] = \
871+ body_dict['server']['personality'][0]
872+ request = self._get_create_request_json(body_dict)
873+ compute_api, response = \
874+ self._run_create_instance_with_mock_compute_api(request)
875+ self.assertEquals(response.status_int, 400)
876+ self.assertEquals(compute_api.injected_files, None)
877+
878+ def test_create_instance_with_personality_with_non_b64_content(self):
879+ path = '/my/file/path'
880+ contents = '#!/bin/bash\necho "Oh no!"\n'
881+ personality = [(path, contents)]
882+ request, response, injected_files = \
883+ self._create_instance_with_personality_json(personality)
884+ self.assertEquals(response.status_int, 400)
885+ self.assertEquals(injected_files, None)
886+
887+ def test_create_instance_with_three_personalities(self):
888+ files = [
889+ ('/etc/sudoers', 'ALL ALL=NOPASSWD: ALL\n'),
890+ ('/etc/motd', 'Enjoy your root access!\n'),
891+ ('/etc/dovecot.conf', 'dovecot\nconfig\nstuff\n'),
892+ ]
893+ personality = []
894+ for path, content in files:
895+ personality.append((path, base64.b64encode(content)))
896+ request, response, injected_files = \
897+ self._create_instance_with_personality_json(personality)
898+ self.assertEquals(response.status_int, 200)
899+ self.assertEquals(injected_files, files)
900+
901+ def test_create_instance_personality_empty_content(self):
902+ path = '/my/file/path'
903+ contents = ''
904+ personality = [(path, contents)]
905+ request, response, injected_files = \
906+ self._create_instance_with_personality_json(personality)
907+ self.assertEquals(response.status_int, 200)
908+ self.assertEquals(injected_files, [(path, contents)])
909+
910+ def test_create_instance_admin_pass_json(self):
911+ request, response, dummy = \
912+ self._create_instance_with_personality_json(None)
913+ self.assertEquals(response.status_int, 200)
914+ response = json.loads(response.body)
915+ self.assertTrue('adminPass' in response['server'])
916+ self.assertTrue(response['server']['adminPass'].startswith('fake'))
917+
918+ def test_create_instance_admin_pass_xml(self):
919+ request, response, dummy = \
920+ self._create_instance_with_personality_xml(None)
921+ self.assertEquals(response.status_int, 200)
922+ dom = minidom.parseString(response.body)
923+ server = dom.childNodes[0]
924+ self.assertEquals(server.nodeName, 'server')
925+ self.assertTrue(server.getAttribute('adminPass').startswith('fake'))
926+
927+
928 if __name__ == "__main__":
929 unittest.main()
930
931=== modified file 'nova/tests/test_quota.py'
932--- nova/tests/test_quota.py 2011-03-07 01:25:01 +0000
933+++ nova/tests/test_quota.py 2011-03-16 19:19:09 +0000
934@@ -33,6 +33,12 @@
935
936
937 class QuotaTestCase(test.TestCase):
938+
939+ class StubImageService(object):
940+
941+ def show(self, *args, **kwargs):
942+ return {"properties": {}}
943+
944 def setUp(self):
945 super(QuotaTestCase, self).setUp()
946 self.flags(connection_type='fake',
947@@ -193,3 +199,67 @@
948 instance_type='m1.small',
949 image_id='fake',
950 metadata=metadata)
951+
952+ def test_allowed_injected_files(self):
953+ self.assertEqual(
954+ quota.allowed_injected_files(self.context),
955+ FLAGS.quota_max_injected_files)
956+
957+ def _create_with_injected_files(self, files):
958+ api = compute.API(image_service=self.StubImageService())
959+ api.create(self.context, min_count=1, max_count=1,
960+ instance_type='m1.small', image_id='fake',
961+ injected_files=files)
962+
963+ def test_no_injected_files(self):
964+ api = compute.API(image_service=self.StubImageService())
965+ api.create(self.context, instance_type='m1.small', image_id='fake')
966+
967+ def test_max_injected_files(self):
968+ files = []
969+ for i in xrange(FLAGS.quota_max_injected_files):
970+ files.append(('/my/path%d' % i, 'config = test\n'))
971+ self._create_with_injected_files(files) # no QuotaError
972+
973+ def test_too_many_injected_files(self):
974+ files = []
975+ for i in xrange(FLAGS.quota_max_injected_files + 1):
976+ files.append(('/my/path%d' % i, 'my\ncontent%d\n' % i))
977+ self.assertRaises(quota.QuotaError,
978+ self._create_with_injected_files, files)
979+
980+ def test_allowed_injected_file_content_bytes(self):
981+ self.assertEqual(
982+ quota.allowed_injected_file_content_bytes(self.context),
983+ FLAGS.quota_max_injected_file_content_bytes)
984+
985+ def test_max_injected_file_content_bytes(self):
986+ max = FLAGS.quota_max_injected_file_content_bytes
987+ content = ''.join(['a' for i in xrange(max)])
988+ files = [('/test/path', content)]
989+ self._create_with_injected_files(files) # no QuotaError
990+
991+ def test_too_many_injected_file_content_bytes(self):
992+ max = FLAGS.quota_max_injected_file_content_bytes
993+ content = ''.join(['a' for i in xrange(max + 1)])
994+ files = [('/test/path', content)]
995+ self.assertRaises(quota.QuotaError,
996+ self._create_with_injected_files, files)
997+
998+ def test_allowed_injected_file_path_bytes(self):
999+ self.assertEqual(
1000+ quota.allowed_injected_file_path_bytes(self.context),
1001+ FLAGS.quota_max_injected_file_path_bytes)
1002+
1003+ def test_max_injected_file_path_bytes(self):
1004+ max = FLAGS.quota_max_injected_file_path_bytes
1005+ path = ''.join(['a' for i in xrange(max)])
1006+ files = [(path, 'config = quotatest')]
1007+ self._create_with_injected_files(files) # no QuotaError
1008+
1009+ def test_too_many_injected_file_path_bytes(self):
1010+ max = FLAGS.quota_max_injected_file_path_bytes
1011+ path = ''.join(['a' for i in xrange(max + 1)])
1012+ files = [(path, 'config = quotatest')]
1013+ self.assertRaises(quota.QuotaError,
1014+ self._create_with_injected_files, files)
1015
1016=== modified file 'nova/utils.py'
1017--- nova/utils.py 2011-03-15 18:24:07 +0000
1018+++ nova/utils.py 2011-03-16 19:19:09 +0000
1019@@ -538,18 +538,6 @@
1020 return wrap
1021
1022
1023-def ensure_b64_encoding(val):
1024- """Safety method to ensure that values expected to be base64-encoded
1025- actually are. If they are, the value is returned unchanged. Otherwise,
1026- the encoded value is returned.
1027- """
1028- try:
1029- dummy = base64.decode(val)
1030- return val
1031- except TypeError:
1032- return base64.b64encode(val)
1033-
1034-
1035 def get_from_path(items, path):
1036 """ Returns a list of items matching the specified path. Takes an
1037 XPath-like expression e.g. prop1/prop2/prop3, and for each item in items,
1038
1039=== modified file 'nova/virt/xenapi/vmops.py'
1040--- nova/virt/xenapi/vmops.py 2011-03-12 00:00:34 +0000
1041+++ nova/virt/xenapi/vmops.py 2011-03-16 19:19:09 +0000
1042@@ -19,6 +19,7 @@
1043 Management class for VM-related functions (spawn, reboot, etc).
1044 """
1045
1046+import base64
1047 import json
1048 import M2Crypto
1049 import os
1050@@ -136,19 +137,20 @@
1051 LOG.info(_('Spawning VM %(instance_name)s created %(vm_ref)s.')
1052 % locals())
1053
1054- def _inject_onset_files():
1055- onset_files = instance.onset_files
1056- if onset_files:
1057+ def _inject_files():
1058+ injected_files = instance.injected_files
1059+ if injected_files:
1060 # Check if this is a JSON-encoded string and convert if needed.
1061- if isinstance(onset_files, basestring):
1062+ if isinstance(injected_files, basestring):
1063 try:
1064- onset_files = json.loads(onset_files)
1065+ injected_files = json.loads(injected_files)
1066 except ValueError:
1067- LOG.exception(_("Invalid value for onset_files: '%s'")
1068- % onset_files)
1069- onset_files = []
1070+ LOG.exception(
1071+ _("Invalid value for injected_files: '%s'")
1072+ % injected_files)
1073+ injected_files = []
1074 # Inject any files, if specified
1075- for path, contents in instance.onset_files:
1076+ for path, contents in instance.injected_files:
1077 LOG.debug(_("Injecting file path: '%s'") % path)
1078 self.inject_file(instance, path, contents)
1079 # NOTE(armando): Do we really need to do this in virt?
1080@@ -164,7 +166,7 @@
1081 if state == power_state.RUNNING:
1082 LOG.debug(_('Instance %s: booted'), instance_name)
1083 timer.stop()
1084- _inject_onset_files()
1085+ _inject_files()
1086 return True
1087 except Exception, exc:
1088 LOG.warn(exc)
1089@@ -408,17 +410,16 @@
1090 raise RuntimeError(resp_dict['message'])
1091 return resp_dict['message']
1092
1093- def inject_file(self, instance, b64_path, b64_contents):
1094+ def inject_file(self, instance, path, contents):
1095 """Write a file to the VM instance. The path to which it is to be
1096- written and the contents of the file need to be supplied; both should
1097+ written and the contents of the file need to be supplied; both will
1098 be base64-encoded to prevent errors with non-ASCII characters being
1099 transmitted. If the agent does not support file injection, or the user
1100 has disabled it, a NotImplementedError will be raised.
1101 """
1102- # Files/paths *should* be base64-encoded at this point, but
1103- # double-check to make sure.
1104- b64_path = utils.ensure_b64_encoding(b64_path)
1105- b64_contents = utils.ensure_b64_encoding(b64_contents)
1106+ # Files/paths must be base64-encoded for transmission to agent
1107+ b64_path = base64.b64encode(path)
1108+ b64_contents = base64.b64encode(contents)
1109
1110 # Need to uniquely identify this request.
1111 transaction_id = str(uuid.uuid4())