Merge lp:~justin-fathomdb/nova/justinsb-openstack-api-keys into lp:~hudson-openstack/nova/trunk

[justinsb]

Support user SSH keys in OpenStack API (they're already supported in EC2, needed for integration tests). Disabled by FLAG option until approved by community, but needed for testing and hence stability.

[Jay Pipes (jaypipes)]

A good enhancement to Nova, Justin! :) Some smallish notes:

1)

14 +# Copyright 2011 Justin Santa Barbara
115 +# All Rights Reserved.

Much of the code in the newly-added file nova/api/keys.py was written by the Anso/NASA guys that was moved from the nova/api/ec2/cloud.py file. It would be best, in my opinion, to put this above your copyright line:

# Copyright 2010 United States Government as represented by the
# Administrator of the National Aeronautics and Space Administration.

In other words, it's good faith to put both your own copyright line as well as the NASA one above it to acknowledge that some significant portion of the code in the file can be attributed to Anso. Hope that makes sense. No worries on the copyright header stuff in all the other files (like nova/api/openstack/keys.py since you wrote all of that code yourself.

2)

I'd prefer to get rid of the Keys class in nova/api/keys.py. All of the methods of Keys should be simple module-level functions since there is no object-level or class-level data to the Keys class. This would also mean you could remove this line in nova/api/ec2/cloud.py:

55 + self.keys_api = Keys()

and change the import in that same file from this:

7 +from nova.api.keys import Keys

to simply this:

from nova.api import keys

Then, in the Controller methods, instead of something like this:

64 + key_pairs = self.keys_api.get_all_keys(context)

just do this:

key_pairs = keys.get_all_keys(context)

The above solves two problems:

a) from nova.api.keys import Keys breaks the rule in the HACKING file that you should only import modules, not objects (and yes, I know that rule is broken regularly in various parts of the source code, but still, it's a documented coding rule...)
b) Removes the unnecessary Keys class that does nothing more than provide a class-level namespace instead of the more appropriate module-level namespacing.

3)

218 + mapper.resource("key", "keys", controller=keys.Controller(),
219 + collection={'detail': 'GET'})

While I think your addition of the keys resource to the OpenStack REST API is very beneficial, it's up to the mailing list and community to decide on changes to the API. Please propose this change/enhancement to the mailing list for approval. We can't approve this patch until this community approval is gained, regardless of how we personally feel the API would be improved. Hope you understand.

4)

Please add some test cases for the OpenStack API /keys stuff, too :)

Cheers!
jay

review: Needs Fixing

[justinsb (justin-fathomdb)]

1) Quite right Jay & very sorry NASA/Anso guys! I started with a clean file and then didn't update the copyright as I refactored the code into there.

2) The idea of the Keys class is to make it look like volumes, network, images & compute. Each of the Web services instantiates a service.API class, and then makes calls on it. Although it's in-process at the moment, and thus could be a module, the intention is that this is an implementation detail. I renamed it to API to show this and added a comment. I did fix up the imports as you suggested.

3) This is necessary to support testing, which is needed for our goal of stability. However, until it has been approved by the community, it is disabled unless specifically enabled for testing, just like FLAGS.allow_admin_api. I called this flag FLAGS.allow_testing_api

4) Done! (Taking advantage of #3!)

Unmerged revisions

710. By justinsb on 2011-02-21

Eventlet seems to screw up httplib2; switched back to httplib as used elsewhere in nova

709. By justinsb on 2011-02-21

Merged with head - resolved conflict with moving of context.user.id => context.user_id

708. By justinsb on 2011-02-21

NASA copyright was in wrong file & a few formatting issues

707. By justinsb on 2011-02-21

Clean up Keys to meet style guidelines, and add explanatory note.
The idea of the API class is to make keys 'feel' like the other services (volumes, compute, images, network). Currently it's implemented in-process, but this way the REST interfaces all look the same, irrespective of this implementation detail

706. By justinsb on 2011-02-21

Don't allow keys to be part of the OpenStack API until it is approved, but support stability by allowing it to be tested using the same approach as allow_admin_api

705. By justinsb on 2011-02-21

Added NASA copyright into file into which I pasted their copy. Sorry guys!!

704. By justinsb on 2011-02-21

Added unit tests for keys

703. By justinsb on 2011-02-19

Fix broken unit tests

702. By justinsb on 2011-02-19

pep8 fixes

701. By justinsb on 2011-02-19

Support keys in OpenStack API (needed for integration tests)

=== modified file 'nova/api/ec2/cloud.py'
--- nova/api/ec2/cloud.py	2011-02-19 07:15:42 +0000
+++ nova/api/ec2/cloud.py	2011-02-21 22:19:33 +0000
@@ -31,7 +31,6 @@
 from nova import compute
 from nova import context
 
-from nova import crypto
 from nova import db
 from nova import exception
 from nova import flags
@@ -40,7 +39,7 @@
 from nova import utils
 from nova import volume
 from nova.compute import instance_types
-
+from nova.api import keys
 
 FLAGS = flags.FLAGS
 flags.DECLARE('service_down_time', 'nova.scheduler.driver')
@@ -50,29 +49,6 @@
 InvalidInputException = exception.InvalidInputException
 
 
-def _gen_key(context, user_id, key_name):
-    """Generate a key
-
-    This is a module level method because it is slow and we need to defer
-    it into a process pool."""
-    # NOTE(vish): generating key pair is slow so check for legal
-    #             creation before creating key_pair
-    try:
-        db.key_pair_get(context, user_id, key_name)
-        raise exception.Duplicate(_("The key_pair %s already exists")
-                                  % key_name)
-    except exception.NotFound:
-        pass
-    private_key, public_key, fingerprint = crypto.generate_key_pair()
-    key = {}
-    key['user_id'] = user_id
-    key['name'] = key_name
-    key['public_key'] = public_key
-    key['fingerprint'] = fingerprint
-    db.key_pair_create(context, key)
-    return {'private_key': private_key, 'fingerprint': fingerprint}
-
-
 def ec2_id_to_id(ec2_id):
     """Convert an ec2 ID (i-[base 16 number]) to an instance id (int)"""
     return int(ec2_id.split('-')[-1], 16)
@@ -97,6 +73,7 @@
                 image_service=self.image_service,
                 volume_api=self.volume_api,
                 hostname_factory=id_to_ec2_id)
+        self.keys_api = keys.API()
         self.setup()
 
     def __str__(self):
@@ -282,35 +259,31 @@
                                  'description': 'fixme'}]}
 
     def describe_key_pairs(self, context, key_name=None, **kwargs):
-        key_pairs = db.key_pair_get_all_by_user(context, context.user_id)
+        key_pairs = self.keys_api.get_all_keys(context)
         if not key_name is None:
             key_pairs = [x for x in key_pairs if x['name'] in key_name]
 
         result = []
         for key_pair in key_pairs:
-            # filter out the vpn keys
-            suffix = FLAGS.vpn_key_suffix
-            if context.is_admin or \
-               not key_pair['name'].endswith(suffix):
-                result.append({
-                    'keyName': key_pair['name'],
-                    'keyFingerprint': key_pair['fingerprint'],
-                })
+            result.append({
+                'keyName': key_pair['name'],
+                'keyFingerprint': key_pair['fingerprint'],
+            })
 
         return {'keypairsSet': result}
 
     def create_key_pair(self, context, key_name, **kwargs):
-        LOG.audit(_("Create key pair %s"), key_name, context=context)
-        data = _gen_key(context, context.user_id, key_name)
+        data = self.keys_api.create_key_pair(context, key_name)
+        private_key = data['private_key']
+        key = data['key']
         return {'keyName': key_name,
-                'keyFingerprint': data['fingerprint'],
-                'keyMaterial': data['private_key']}
-        # TODO(vish): when context is no longer an object, pass it here
+                'keyFingerprint': key['fingerprint'],
+                'keyMaterial': private_key}
 
     def delete_key_pair(self, context, key_name, **kwargs):
-        LOG.audit(_("Delete key pair %s"), key_name, context=context)
         try:
-            db.key_pair_destroy(context, context.user_id, key_name)
+            key = self.keys_api.get_key_by_name(context, key_name)
+            self.keys_api.delete_key_pair(context, key)
         except exception.NotFound:
             # aws returns true even if the key doesn't exist
             pass

=== added file 'nova/api/keys.py'
--- nova/api/keys.py	1970-01-01 00:00:00 +0000
+++ nova/api/keys.py	2011-02-21 22:19:33 +0000
@@ -0,0 +1,93 @@
+# vim: tabstop=4 shiftwidth=4 softtabstop=4
+
+# Copyright 2010 United States Government as represented by the
+# Administrator of the National Aeronautics and Space Administration.
+# Copyright 2011 Justin Santa Barbara
+# All Rights Reserved.
+#
+#    Licensed under the Apache License, Version 2.0 (the "License"); you may
+#    not use this file except in compliance with the License. You may obtain
+#    a copy of the License at
+#
+#         http://www.apache.org/licenses/LICENSE-2.0
+#
+#    Unless required by applicable law or agreed to in writing, software
+#    distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+#    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+#    License for the specific language governing permissions and limitations
+#    under the License.
+
+
+"""Function dealing with (crypto) keys"""
+
+from nova import crypto
+from nova import db
+from nova import exception
+from nova import flags
+from nova import log as logging
+
+
+FLAGS = flags.FLAGS
+
+LOG = logging.getLogger("nova.keys")
+
+
+def _gen_key(context, user_id, key_name):
+    """Generate a key
+
+    This is a module level method because it is slow and we need to defer
+    it into a process pool."""
+    # NOTE(vish): generating key pair is slow so check for legal
+    #             creation before creating key_pair
+    try:
+        db.key_pair_get(context, user_id, name=key_name)
+        raise exception.Duplicate(_("The key_pair %s already exists")
+                                  % key_name)
+    except exception.NotFound:
+        pass
+    private_key, public_key, fingerprint = crypto.generate_key_pair()
+    key = {}
+    key['user_id'] = user_id
+    key['name'] = key_name
+    key['public_key'] = public_key
+    key['fingerprint'] = fingerprint
+    created = db.key_pair_create(context, key)
+    return {'private_key': private_key, 'key': created}
+
+
+class API(object):
+    """Although keys is currently an in-process function, this abstraction
+    supports the idea that it may not be in future, so that all the APIs
+    look the same (volumes, compute, keys etc)."""
+    def get_all_keys(self, context):
+        key_pairs = db.key_pair_get_all_by_user(context, context.user_id)
+
+        result = []
+        for key_pair in key_pairs:
+            # filter out the vpn keys
+            suffix = FLAGS.vpn_key_suffix
+            if context.is_admin or not key_pair['name'].endswith(suffix):
+                result.append(key_pair)
+
+        return result
+
+    def get_key_by_name(self, context, key_name):
+        return db.key_pair_get(context, context.user_id, name=key_name)
+
+    def get_key_by_id(self, context, key_id):
+        return db.key_pair_get(context, context.user_id, id=key_id)
+
+    def create_key_pair(self, context, key_name):
+        LOG.audit(_("Create key pair %s"), key_name, context=context)
+        return _gen_key(context, context.user_id, key_name)
+        # TODO(vish): when context is no longer an object, pass it here
+
+    def delete_key_pair(self, context, key):
+        key_name = key['name']
+        LOG.audit(_("Delete key pair %s"), key_name, context=context)
+        try:
+            db.key_pair_destroy(context, context.user_id, name=key_name)
+        except exception.NotFound:
+            # aws returns true even if the key doesn't exist
+            pass
+        return True

=== modified file 'nova/api/openstack/__init__.py'
--- nova/api/openstack/__init__.py	2011-02-17 21:39:03 +0000
+++ nova/api/openstack/__init__.py	2011-02-21 22:19:33 +0000
@@ -32,6 +32,7 @@
 from nova.api.openstack import consoles
 from nova.api.openstack import flavors
 from nova.api.openstack import images
+from nova.api.openstack import keys
 from nova.api.openstack import servers
 from nova.api.openstack import shared_ip_groups
 from nova.api.openstack import zones
@@ -42,6 +43,10 @@
 flags.DEFINE_bool('allow_admin_api',
     False,
     'When True, this API service will accept admin operations.')
+flags.DEFINE_bool('allow_testing_api',
+    False,
+    'When True, this API service will support unofficial API extensions that '
+    'are intended solely for testing as part of the stability drive.')
 
 
 class FaultWrapper(wsgi.Middleware):
@@ -85,6 +90,11 @@
             mapper.resource("zone", "zones", controller=zones.Controller(),
                         collection={'detail': 'GET'})
 
+        if FLAGS.allow_testing_api:
+            LOG.debug(_("Including testing operations in API."))
+            mapper.resource("key", "keys", controller=keys.Controller(),
+                            collection={'detail': 'GET'})
+
         mapper.resource("server", "servers", controller=servers.Controller(),
                         collection={'detail': 'GET'},
                         member=server_members)

=== added file 'nova/api/openstack/keys.py'
--- nova/api/openstack/keys.py	1970-01-01 00:00:00 +0000
+++ nova/api/openstack/keys.py	2011-02-21 22:19:33 +0000
@@ -0,0 +1,122 @@
+# Copyright 2011 Justin Santa Barbara
+# All Rights Reserved.
+#
+#    Licensed under the Apache License, Version 2.0 (the "License"); you may
+#    not use this file except in compliance with the License. You may obtain
+#    a copy of the License at
+#
+#         http://www.apache.org/licenses/LICENSE-2.0
+#
+#    Unless required by applicable law or agreed to in writing, software
+#    distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+#    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+#    License for the specific language governing permissions and limitations
+#    under the License.
+
+from webob import exc
+
+from nova import exception
+from nova import flags
+from nova import log as logging
+from nova import wsgi
+from nova.api.openstack import common
+from nova.api.openstack import faults
+from nova.api import keys
+
+
+LOG = logging.getLogger("nova.api.keys")
+
+FLAGS = flags.FLAGS
+
+
+def _translate_detail_keys(inst):
+    """ Maps keys for details view"""
+
+    inst_dict = _translate_summary_keys(inst)
+
+    # No additional data / lookups at the moment
+
+    return inst_dict
+
+
+def _translate_summary_keys(inst):
+    """ Maps keys for summary view"""
+    inst_dict = {}
+    inst_dict['id'] = inst['id']
+    inst_dict['name'] = inst['name']
+    inst_dict['fingerprint'] = inst['fingerprint']
+
+    return inst_dict
+
+
+class Controller(wsgi.Controller):
+    """ The (crypto) Key API controller for the OpenStack API """
+
+    _serialization_metadata = {
+        'application/xml': {
+            "attributes": {
+                "key": ["id", "keyName", "keyFingerprint", "privateKey"]}}}
+
+    def __init__(self):
+        self.keys_api = keys.API()
+        super(Controller, self).__init__()
+
+    def show(self, req, id):
+        """Return data about the given key"""
+        context = req.environ['nova.context']
+
+        try:
+            key = self.keys_api.get_key_by_id(context, id)
+        except exception.NotFound:
+            return faults.Fault(exc.HTTPNotFound())
+
+        return {'key': _translate_detail_keys(key)}
+
+    def delete(self, req, id):
+        """ Delete a key """
+        context = req.environ['nova.context']
+        try:
+            key = self.keys_api.get_key_by_id(context, id)
+            self.keys_api.delete_key_pair(context, key)
+        except exception.NotFound:
+            return faults.Fault(exc.HTTPNotFound())
+        return exc.HTTPAccepted()
+
+    def index(self, req):
+        """ Returns a summary list of keys for a given user """
+        return self._items(req, entity_maker=_translate_summary_keys)
+
+    def detail(self, req):
+        """ Returns a detailed list of keys for a given user """
+        return self._items(req, entity_maker=_translate_detail_keys)
+
+    def _items(self, req, entity_maker):
+        """Returns a list of keys for a given user.
+
+        entity_maker - applied to each item to map to return format
+        """
+        context = req.environ['nova.context']
+
+        key_pairs = self.keys_api.get_all_keys(context)
+        limited_list = common.limited(key_pairs, req)
+        res = [entity_maker(inst) for inst in limited_list]
+        return {'keys': res}
+
+    def create(self, req):
+        """ Creates a new server for a given user """
+        context = req.environ['nova.context']
+
+        env = self._deserialize(req.body, req)
+        if not env:
+            return faults.Fault(exc.HTTPUnprocessableEntity())
+
+        key_name = env['key']['name']
+
+        data = self.keys_api.create_key_pair(context, key_name)
+        private_key = data['private_key']
+        key = data['key']
+
+        retval = _translate_detail_keys(key)
+        retval['privateKey'] = private_key
+
+        return {'key': retval}

=== modified file 'nova/cloudpipe/pipelib.py'
--- nova/cloudpipe/pipelib.py	2011-01-12 22:35:09 +0000
+++ nova/cloudpipe/pipelib.py	2011-02-21 22:19:33 +0000
@@ -37,7 +37,7 @@
 from nova.auth import manager
 # TODO(eday): Eventually changes these to something not ec2-specific
 from nova.api.ec2 import cloud
-
+from nova.api import keys
 
 FLAGS = flags.FLAGS
 flags.DEFINE_string('boot_script_template',
@@ -138,7 +138,7 @@
     def setup_key_pair(self, context):
         key_name = '%s%s' % (context.project.id, FLAGS.vpn_key_suffix)
         try:
-            result = cloud._gen_key(context, context.user.id, key_name)
+            result = keys._gen_key(context, context.user.id, key_name)
             private_key = result['private_key']
             try:
                 key_dir = os.path.join(FLAGS.keys_path, context.user.id)

=== modified file 'nova/compute/api.py'
--- nova/compute/api.py	2011-02-19 05:00:58 +0000
+++ nova/compute/api.py	2011-02-21 22:19:33 +0000
@@ -132,7 +132,8 @@
             security_groups.append(group['id'])
 
         if key_data is None and key_name:
-            key_pair = db.key_pair_get(context, context.user_id, key_name)
+            key_pair = db.key_pair_get(context, context.user_id,
+                                       name=key_name)
             key_data = key_pair['public_key']
 
         base_options = {

=== modified file 'nova/db/api.py'
--- nova/db/api.py	2011-02-17 22:14:07 +0000
+++ nova/db/api.py	2011-02-21 22:19:33 +0000
@@ -442,9 +442,9 @@
     return IMPL.key_pair_destroy_all_by_user(context, user_id)
 
 
-def key_pair_get(context, user_id, name):
+def key_pair_get(context, user_id, name=None, id=None):
     """Get a key_pair or raise if it does not exist."""
-    return IMPL.key_pair_get(context, user_id, name)
+    return IMPL.key_pair_get(context, user_id, name=name, id=id)
 
 
 def key_pair_get_all_by_user(context, user_id):

=== modified file 'nova/db/sqlalchemy/api.py'
--- nova/db/sqlalchemy/api.py	2011-02-17 21:39:03 +0000
+++ nova/db/sqlalchemy/api.py	2011-02-21 22:19:33 +0000
@@ -931,7 +931,8 @@
     authorize_user_context(context, user_id)
     session = get_session()
     with session.begin():
-        key_pair_ref = key_pair_get(context, user_id, name, session=session)
+        key_pair_ref = key_pair_get(context, user_id, name=name,
+                                    session=session)
         key_pair_ref.delete(session=session)
 
 
@@ -946,17 +947,27 @@
 
 
 @require_context
-def key_pair_get(context, user_id, name, session=None):
+def key_pair_get(context, user_id, name=None, id=None, session=None):
     authorize_user_context(context, user_id)
 
     if not session:
         session = get_session()
 
-    result = session.query(models.KeyPair).\
+    query = session.query(models.KeyPair).\
                      filter_by(user_id=user_id).\
-                     filter_by(name=name).\
-                     filter_by(deleted=can_read_deleted(context)).\
-                     first()
+                     filter_by(deleted=can_read_deleted(context))
+
+    if not name and not id:
+        raise exception.Error('Name or Id is required')
+
+    if name:
+        query = query.filter_by(name=name)
+
+    if id:
+        query = query.filter_by(id=id)
+
+    result = query.first()
+
     if not result:
         raise exception.NotFound(_('no keypair for user %(user_id)s,'
                 ' name %(name)s') % locals())

=== added directory 'nova/tests/integrated'
=== added file 'nova/tests/integrated/__init__.py'
--- nova/tests/integrated/__init__.py	1970-01-01 00:00:00 +0000
+++ nova/tests/integrated/__init__.py	2011-02-21 22:19:33 +0000
@@ -0,0 +1,20 @@
+# vim: tabstop=4 shiftwidth=4 softtabstop=4
+
+#    Copyright (c) 2011 Justin Santa Barbara
+#
+#    Licensed under the Apache License, Version 2.0 (the "License"); you may
+#    not use this file except in compliance with the License. You may obtain
+#    a copy of the License at
+#
+#         http://www.apache.org/licenses/LICENSE-2.0
+#
+#    Unless required by applicable law or agreed to in writing, software
+#    distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+#    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+#    License for the specific language governing permissions and limitations
+#    under the License.
+
+"""
+:mod:`integrated` -- Tests whole systems, using mock services where needed
+=================================
+"""

=== added directory 'nova/tests/integrated/api'
=== added file 'nova/tests/integrated/api/__init__.py'
--- nova/tests/integrated/api/__init__.py	1970-01-01 00:00:00 +0000
+++ nova/tests/integrated/api/__init__.py	2011-02-21 22:19:33 +0000
@@ -0,0 +1,20 @@
+# vim: tabstop=4 shiftwidth=4 softtabstop=4
+
+#    Copyright (c) 2011 Justin Santa Barbara
+#
+#    Licensed under the Apache License, Version 2.0 (the "License"); you may
+#    not use this file except in compliance with the License. You may obtain
+#    a copy of the License at
+#
+#         http://www.apache.org/licenses/LICENSE-2.0
+#
+#    Unless required by applicable law or agreed to in writing, software
+#    distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+#    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+#    License for the specific language governing permissions and limitations
+#    under the License.
+
+"""
+:mod:`api` -- OpenStack API client, for testing rather than production
+=================================
+"""

=== added file 'nova/tests/integrated/api/client.py'
--- nova/tests/integrated/api/client.py	1970-01-01 00:00:00 +0000
+++ nova/tests/integrated/api/client.py	2011-02-21 22:19:33 +0000
@@ -0,0 +1,130 @@
+# vim: tabstop=4 shiftwidth=4 softtabstop=4
+
+#    Copyright (c) 2011 Justin Santa Barbara
+#
+#    Licensed under the Apache License, Version 2.0 (the "License"); you may
+#    not use this file except in compliance with the License. You may obtain
+#    a copy of the License at
+#
+#         http://www.apache.org/licenses/LICENSE-2.0
+#
+#    Unless required by applicable law or agreed to in writing, software
+#    distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+#    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+#    License for the specific language governing permissions and limitations
+#    under the License.
+
+from nova import exception
+import json
+import httplib
+from urlparse import urlparse
+
+
+class TestOpenStackClient(object):
+    """ A really basic OpenStack API client that is under our control,
+    so we can make changes / insert hooks for testing"""
+
+    def __init__(self, auth_user, auth_key, auth_uri):
+        super(TestOpenStackClient, self).__init__()
+        self.auth_result = None
+        self.auth_user = auth_user
+        self.auth_key = auth_key
+        self.auth_uri = auth_uri
+
+    def request(self, url, method='GET', body=None, headers={}):
+        parsed_url = urlparse(url)
+        port = parsed_url.port
+        hostname = parsed_url.hostname
+        scheme = parsed_url.scheme
+
+        if scheme == 'http':
+            conn = httplib.HTTPConnection(hostname,
+                                          port=port)
+        elif scheme == 'https':
+            conn = httplib.HTTPSConnection(hostname,
+                                           port=port)
+        else:
+            raise exception.Error("Unknown scheme: %s" % url)
+
+        relative_url = parsed_url.path
+        if parsed_url.query:
+            relative_url = relative_url + parsed_url.query
+        conn.request(method, relative_url, body, headers)
+        response = conn.getresponse()
+        return response
+
+    def _authenticate(self):
+        if self.auth_result:
+            return self.auth_result
+
+        headers = {'X-Auth-User': self.auth_user,
+                   'X-Auth-Key': self.auth_key}
+        response = self.request(self.auth_uri,
+                                         headers=headers)
+        if not response.status in [204]:
+            raise exception.Error("Authentication failure: %s\n%s" %
+                                  (response.status, response.read()))
+
+        auth_headers = {}
+        for k, v in response.getheaders():
+            auth_headers[k] = v
+
+        self.auth_result = auth_headers
+        return self.auth_result
+
+    def api_request(self, relative_uri, check_response_status=None, **kwargs):
+        auth_result = self._authenticate()
+
+        #TODO(justinsb): Why is this lower case??
+        base_uri = auth_result['X-Server-Management-Url'.lower()]
+        full_uri = base_uri + relative_uri
+
+        headers = kwargs.setdefault('headers', {})
+        #TODO(justinsb): Why is this lower case??
+        headers['X-Auth-Token'] = auth_result['X-Auth-Token'.lower()]
+
+        response = self.request(full_uri, **kwargs)
+
+        print "%s => code %s" % (relative_uri, response.status)
+
+        if check_response_status:
+            if not response.status in check_response_status:
+                raise exception.Error("Unexpected status: %s\n%s" %
+                                      (response.status, response.read()))
+
+        return response
+
+    def _decode_json(self, response):
+        body = response.read()
+        print body
+        return json.loads(body)
+
+    def api_get(self, relative_uri, **kwargs):
+        kwargs.setdefault('check_response_status', [200])
+        response = self.api_request(relative_uri, **kwargs)
+        return self._decode_json(response)
+
+    def api_post(self, relative_uri, body, **kwargs):
+        kwargs['method'] = 'POST'
+        if body:
+            headers = kwargs.setdefault('headers', {})
+            headers['Content-Type'] = 'application/json'
+            kwargs['body'] = json.dumps(body)
+
+        kwargs.setdefault('check_response_status', [200])
+        response = self.api_request(relative_uri, **kwargs)
+        return self._decode_json(response)
+
+    def api_delete(self, relative_uri, **kwargs):
+        kwargs['method'] = 'DELETE'
+        kwargs.setdefault('check_response_status', [200, 202])
+        return self.api_request(relative_uri, **kwargs)
+
+    def get_keys_detail(self):
+        return self.api_get('/keys/detail')['keys']
+
+    def post_key(self, key):
+        return self.api_post('/keys', key)['key']
+
+    def delete_key(self, key_id):
+        return self.api_delete('/keys/%s' % key_id)

=== added file 'nova/tests/integrated/integrated_helpers.py'
--- nova/tests/integrated/integrated_helpers.py	1970-01-01 00:00:00 +0000
+++ nova/tests/integrated/integrated_helpers.py	2011-02-21 22:19:33 +0000
@@ -0,0 +1,164 @@
+# vim: tabstop=4 shiftwidth=4 softtabstop=4
+
+# Copyright 2011 Justin Santa Barbara
+# Copyright 2010 United States Government as represented by the
+# Administrator of the National Aeronautics and Space Administration.
+# All Rights Reserved.
+#
+#    Licensed under the Apache License, Version 2.0 (the "License"); you may
+#    not use this file except in compliance with the License. You may obtain
+#    a copy of the License at
+#
+#         http://www.apache.org/licenses/LICENSE-2.0
+#
+#    Unless required by applicable law or agreed to in writing, software
+#    distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+#    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+#    License for the specific language governing permissions and limitations
+#    under the License.
+
+"""
+Provides common functionality for integrated unit tests
+"""
+
+from nova.tests.integrated.api.client import TestOpenStackClient
+from nova import wsgi
+from nova import flags
+from nova.auth import manager
+import random
+import string
+from nova.exception import Error
+from nova.log import logging
+from nova.api import openstack  # For FLAGS.allow_testing_api
+
+FLAGS = flags.FLAGS
+
+
+def generate_random_alphanumeric(length):
+    """Creates a random string of specified length"""
+    return ''.join(random.choice(string.ascii_uppercase + string.digits)
+                   for _x in range(length))
+
+
+def generate_new_element(items, prefix):
+    """Creates a random string with prefix, that is not in 'items' list"""
+    while True:
+        candidate = prefix + generate_random_alphanumeric(8)
+        if not candidate in items:
+            return candidate
+        print "Random collision on %s" % candidate
+
+
+class IntegratedUnitTestContext(object):
+    __INSTANCE = None
+
+    def __init__(self):
+        self.old_allow_testing_api = FLAGS.allow_testing_api
+        FLAGS.allow_testing_api = True
+
+        self.auth_manager = manager.AuthManager()
+
+        self.auth_user, self.auth_key = self._create_unittest_user()
+
+        # No way to currently pass this through the OpenStack API
+        self.project_name = 'openstack'
+        self._configure_project(self.project_name)
+
+        self.auth_url = self._start_nova_api()
+
+        self.openstack_api = TestOpenStackClient(self.auth_user,
+                                                 self.auth_key,
+                                                 self.auth_url)
+
+    def _create_unittest_user(self):
+        users = self.auth_manager.get_users()
+        user_names = [user.name for user in users]
+        auth_name = generate_new_element(user_names, 'unittest_user_')
+        auth_key = generate_random_alphanumeric(16)
+
+        # Right now there's a bug where auth_name and auth_key are reversed
+        auth_key = auth_name
+
+        self.auth_manager.create_user(auth_name, auth_name, auth_key, False)
+        return auth_name, auth_key
+
+    def _configure_project(self, project_name):
+        projects = self.auth_manager.get_projects()
+        project_names = [project.name for project in projects]
+        if not project_name in project_names:
+            project = self.auth_manager.create_project(project_name,
+                                                       self.auth_user,
+                                                       description=None,
+                                                       member_users=None)
+        else:
+            self.auth_manager.add_to_project(self.auth_user, project_name)
+
+    def _start_nova_api(self):
+        # Copied and pasted from bin/nova-api... yuk!!
+        paste_config_file = wsgi.paste_config_file('nova-api.conf')
+
+        API_ENDPOINTS = ['ec2', 'osapi']
+
+        LOG = logging.getLogger('nova.api')
+        LOG.setLevel(logging.DEBUG)
+
+        LOG.debug(_("Using paste.deploy config at: %s"), paste_config_file)
+        apps = []
+        for api in API_ENDPOINTS:
+            config = wsgi.load_paste_configuration(paste_config_file, api)
+            if config is None:
+                #LOG.debug(_("No paste configuration for app: %s"), api)
+                continue
+            LOG.debug(_("App Config: %(api)s\n%(config)r") % locals())
+            wsgi.paste_config_to_flags(config, {
+                "verbose": FLAGS.verbose,
+                "%s_host" % api: config.get('host', '0.0.0.0'),
+                "%s_port" % api: getattr(FLAGS, "%s_port" % api)})
+            LOG.info(_("Running %s API"), api)
+            app = wsgi.load_paste_app(paste_config_file, api)
+            apps.append((app, getattr(FLAGS, "%s_port" % api),
+                         getattr(FLAGS, "%s_host" % api)))
+        if len(apps) == 0:
+            LOG.error(_("No known API applications configured in %s."),
+                      paste_config_file)
+            return
+
+        # NOTE(todd): redo logging config, verbose could be set in paste config
+        #logging.basicConfig()
+        wsgi_server = wsgi.Server()
+        for app in apps:
+            wsgi_server.start(*app)
+        #server.wait()
+        self.wsgi_server = wsgi_server
+        self.wsgi_apps = apps
+        return 'http://localhost:8774/v1.0'
+
+    def get_openstack_api(self):
+        return self.openstack_api
+
+    @staticmethod
+    def get():
+        if not IntegratedUnitTestContext.__INSTANCE:
+            IntegratedUnitTestContext.startup()
+            #raise Error("Must call IntegratedUnitTestContext::startup")
+        return IntegratedUnitTestContext.__INSTANCE
+
+    @staticmethod
+    def startup():
+        if IntegratedUnitTestContext.__INSTANCE:
+            raise Error("Multiple calls to IntegratedUnitTestContext.startup")
+        IntegratedUnitTestContext.__INSTANCE = IntegratedUnitTestContext()
+
+    def cleanup(self):
+        FLAGS.allow_testing_api = self.old_allow_testing_api
+        # TODO(justinsb): Shutdown WSGI & anything else we startup
+        # self.wsgi_server.terminate()
+        # self.wsgi_server = None
+        pass
+
+    @staticmethod
+    def shutdown():
+        if not IntegratedUnitTestContext.__INSTANCE:
+            raise Error("Must call IntegratedUnitTestContext::startup")
+        IntegratedUnitTestContext.__INSTANCE.cleanup()
+        IntegratedUnitTestContext.__INSTANCE = None

=== added file 'nova/tests/integrated/test_keys.py'
--- nova/tests/integrated/test_keys.py	1970-01-01 00:00:00 +0000
+++ nova/tests/integrated/test_keys.py	2011-02-21 22:19:33 +0000
@@ -0,0 +1,78 @@
+# vim: tabstop=4 shiftwidth=4 softtabstop=4
+
+# Copyright 2011 Justin Santa Barbara
+# All Rights Reserved.
+#
+#    Licensed under the Apache License, Version 2.0 (the "License"); you may
+#    not use this file except in compliance with the License. You may obtain
+#    a copy of the License at
+#
+#         http://www.apache.org/licenses/LICENSE-2.0
+#
+#    Unless required by applicable law or agreed to in writing, software
+#    distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+#    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+#    License for the specific language governing permissions and limitations
+#    under the License.
+
+import unittest
+from nova import flags
+from nova.tests.integrated.integrated_helpers import IntegratedUnitTestContext
+from nova.tests.integrated.integrated_helpers import generate_new_element
+
+FLAGS = flags.FLAGS
+FLAGS.verbose = True
+
+
+class KeysTest(unittest.TestCase):
+    def setUp(self):
+        context = IntegratedUnitTestContext.get()
+        self.api = context.get_openstack_api()
+
+    def tearDown(self):
+        pass
+
+    def test_get_keys(self):
+        """Simple check that listing keys works"""
+        keys = self.api.get_keys_detail()
+        for key in keys:
+            print("key: %s" % key)
+
+    def _find_key(self, key_name):
+        """Return key with specified name or None if not found"""
+        keys = self.api.get_keys_detail()
+        for key in keys:
+            if key['name'] == key_name:
+                return key
+        return None
+
+    def test_create_and_delete_key(self):
+        """Creates and deletes a key"""
+
+        # Find unused key name
+        keys = self.api.get_keys_detail()
+        key_names = [key.name for key in keys]
+        key_name = generate_new_element(key_names, 'unittest_')
+
+        # Create key with name
+        created_key = self.api.post_key({'key': {'name': key_name}})
+        print "created_key: %s" % created_key
+        private_key = created_key['privateKey']
+        print "private_key: %s" % private_key
+        self.assertTrue(private_key)
+        self.assertTrue('BEGIN RSA PRIVATE KEY' in private_key)
+
+        # Check it's there
+        found_key = self._find_key(key_name)
+        self.assertEqual(key_name, found_key['name'])
+
+        # Delete the key
+        self.api.delete_key(created_key['id'])
+
+        # Check it's not there any more
+        found_key = self._find_key(key_name)
+        self.assertFalse(found_key)
+
+
+if __name__ == "__main__":
+    unittest.main()

=== modified file 'nova/tests/test_api.py'
--- nova/tests/test_api.py	2011-02-15 12:18:27 +0000
+++ nova/tests/test_api.py	2011-02-21 22:19:33 +0000
@@ -31,6 +31,7 @@
 from nova.api.ec2 import cloud
 from nova.api.ec2 import apirequest
 from nova.auth import manager
+from nova.api import keys
 
 
 class FakeHttplibSocket(object):
@@ -164,7 +165,7 @@
         user = self.manager.create_user('fake', 'fake', 'fake')
         project = self.manager.create_project('fake', 'fake', 'fake')
         # NOTE(vish): create depends on pool, so call helper directly
-        cloud._gen_key(context.get_admin_context(), user.id, keyname)
+        keys._gen_key(context.get_admin_context(), user.id, keyname)
 
         rv = self.ec2.get_all_key_pairs()
         results = [k for k in rv if k.name == keyname]

=== modified file 'nova/tests/test_cloud.py'
--- nova/tests/test_cloud.py	2011-01-23 01:46:05 +0000
+++ nova/tests/test_cloud.py	2011-02-21 22:19:33 +0000
@@ -39,7 +39,7 @@
 from nova.compute import power_state
 from nova.api.ec2 import cloud
 from nova.objectstore import image
-
+from nova.api import keys
 
 FLAGS = flags.FLAGS
 LOG = logging.getLogger('nova.tests.cloud')
@@ -85,7 +85,7 @@
 
     def _create_key(self, name):
         # NOTE(vish): create depends on pool, so just call helper directly
-        return cloud._gen_key(self.context, self.context.user.id, name)
+        return keys._gen_key(self.context, self.context.user.id, name)
 
     def test_describe_regions(self):
         """Makes sure describe regions runs without raising an exception"""
@@ -232,7 +232,7 @@
         bio = BIO.MemoryBuffer()
         public_key = db.key_pair_get(self.context,
                                     self.context.user.id,
-                                    'test')['public_key']
+                                    name='test')['public_key']
         key.save_pub_key_bio(bio)
         converted = crypto.ssl_pub_to_ssh_pub(bio.read())
         # assert key fields are equal