Merge lp:~soren/nova/uml into lp:~hudson-openstack/nova/trunk

[Soren Hansen]

Add new libvirt_type option "uml" for user-mode-linux.. This switches the libvirt URI to uml:///system and uses a different template for the libvirt xml.

Yo dawg, I heard you like clouds, so I put a cloud in your cloud...

This patchset let us use UML guests. This makes it possible to run Nova on e.g. Rackspace Cloud Servers.

Mind you, you will need a user-mode-linux package other than the one from Lucid to make this work, since that one is broken. You can find a working version in the nova-core ppa (or in probably a week's time in Lucid-updates).

[lp:~soren/nova/uml]

229. By Ewan Mellor on 2010-08-14

Bug #617776: DescribeImagesResponse contains type element, when it should be called imageType

Make the objectstore respond with the field 'imageType' as well as 'type'.
The former is the correct one, according to the EC2 API specification for
the DescribeImages response. The latter is for compatibility with euca2ools
and other clients.

230. By Vish Ishaya on 2010-08-14

Improves pep8 compliance and pylint score in network code.

231. By Devin Carlen on 2010-08-14

Implemented admin client / admin api for fetching user roles.

232. By Vish Ishaya on 2010-08-14

Sets a hostname for instances that properly resolves and cleans up network classes

233. By Vish Ishaya on 2010-08-14

Fixes bug lp:616312 by reversing the order of args in nova-manage when it calls AuthManager.get_credentials.

234. By Vish Ishaya on 2010-08-14

Removes the workaround for syslog-ng of removing newlines.

235. By Joel Moore <email address hidden> on 2010-08-14

Fixed path to keys directory.

236. By Joel Moore <email address hidden> on 2010-08-14

Removes requirement of internet connectivity to run api server.

[Vish Ishaya (vishvananda)]

coolness. lgtm. I suppose the debian packaging will need to be updated to copy the xml files from the new location.

review: Approve

[lp:~soren/nova/uml]

237. By Vish Ishaya on 2010-08-14

Catches and logs exceptions for rpc calls and raises a RemoteError exception on the caller side.

[Eric Day (eday)]

lgtm

review: Approve

[lp:~soren/nova/uml]

238. By Soren Hansen on 2010-08-16

Add new libvirt_type option "uml" for user-mode-linux.. This switches the libvirt URI to uml:///system and uses a different template for the libvirt xml.

=== added file 'nova/tests/virt_unittest.py'
--- nova/tests/virt_unittest.py	1970-01-01 00:00:00 +0000
+++ nova/tests/virt_unittest.py	2010-08-14 20:03:43 +0000
@@ -0,0 +1,69 @@
+# vim: tabstop=4 shiftwidth=4 softtabstop=4
+#
+#    Copyright 2010 OpenStack LLC
+#
+#    Licensed under the Apache License, Version 2.0 (the "License"); you may
+#    not use this file except in compliance with the License. You may obtain
+#    a copy of the License at
+#
+#         http://www.apache.org/licenses/LICENSE-2.0
+#
+#    Unless required by applicable law or agreed to in writing, software
+#    distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+#    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+#    License for the specific language governing permissions and limitations
+#    under the License.
+
+from nova import flags
+from nova import test
+from nova.virt import libvirt_conn
+
+FLAGS = flags.FLAGS
+
+
+class LibvirtConnTestCase(test.TrialTestCase):
+    def test_get_uri_and_template(self):
+        class MockDataModel(object):
+            def __init__(self):
+                self.datamodel = { 'name' : 'i-cafebabe',
+                                   'memory_kb' : '1024000',
+                                   'basepath' : '/some/path',
+                                   'bridge_name' : 'br100',
+                                   'mac_address' : '02:12:34:46:56:67',
+                                   'vcpus' : 2 }
+
+        type_uri_map = { 'qemu' : ('qemu:///system',
+                                [lambda s: '<domain type=\'qemu\'>' in s,
+                                 lambda s: 'type>hvm</type' in s,
+                                 lambda s: 'emulator>/usr/bin/kvm' not in s]),
+                         'kvm' : ('qemu:///system',
+                                [lambda s: '<domain type=\'kvm\'>' in s,
+                                 lambda s: 'type>hvm</type' in s,
+                                 lambda s: 'emulator>/usr/bin/qemu<' not in s]),
+                         'uml' : ('uml:///system',
+                                [lambda s: '<domain type=\'uml\'>' in s,
+                                 lambda s: 'type>uml</type' in s]),
+                          }
+
+        for (libvirt_type,(expected_uri, checks)) in type_uri_map.iteritems():
+            FLAGS.libvirt_type = libvirt_type
+            conn = libvirt_conn.LibvirtConnection(True)
+
+            uri, template = conn.get_uri_and_template()
+            self.assertEquals(uri, expected_uri)
+
+            for i, check in enumerate(checks):
+                xml = conn.toXml(MockDataModel())
+                self.assertTrue(check(xml), '%s failed check %d' % (xml, i))
+
+        # Deliberately not just assigning this string to FLAGS.libvirt_uri and
+        # checking against that later on. This way we make sure the
+        # implementation doesn't fiddle around with the FLAGS.
+        testuri = 'something completely different'
+        FLAGS.libvirt_uri = testuri
+        for (libvirt_type,(expected_uri, checks)) in type_uri_map.iteritems():
+            FLAGS.libvirt_type = libvirt_type
+            conn = libvirt_conn.LibvirtConnection(True)
+            uri, template = conn.get_uri_and_template()
+            self.assertEquals(uri, testuri)
+

=== renamed file 'nova/compute/interfaces.template' => 'nova/virt/interfaces.template'
=== renamed file 'nova/compute/libvirt.xml.template' => 'nova/virt/libvirt.qemu.xml.template'
=== added file 'nova/virt/libvirt.uml.xml.template'
--- nova/virt/libvirt.uml.xml.template	1970-01-01 00:00:00 +0000
+++ nova/virt/libvirt.uml.xml.template	2010-08-14 20:03:43 +0000
@@ -0,0 +1,25 @@
+<domain type='%(type)s'>
+    <name>%(name)s</name>
+    <memory>%(memory_kb)s</memory>
+    <os>
+        <type>%(type)s</type>
+        <kernel>/usr/bin/linux</kernel>
+        <root>/dev/ubda1</root>
+    </os>
+    <devices>
+        <disk type='file'>
+            <source file='%(basepath)s/disk'/>
+            <target dev='ubd0' bus='uml'/>
+        </disk>
+        <interface type='bridge'>
+            <source bridge='%(bridge_name)s'/>
+            <mac address='%(mac_address)s'/>
+        </interface>
+        <console type="pty" />
+        <serial type="file">
+            <source path='%(basepath)s/console.log'/>
+            <target port='1'/>
+        </serial>
+    </devices>
+    <nova>%(nova)s</nova>
+</domain>

=== modified file 'nova/virt/libvirt_conn.py'
--- nova/virt/libvirt_conn.py	2010-08-10 15:37:07 +0000
+++ nova/virt/libvirt_conn.py	2010-08-14 20:03:43 +0000
@@ -44,15 +44,20 @@
 
 FLAGS = flags.FLAGS
 flags.DEFINE_string('libvirt_xml_template',
-                    utils.abspath('compute/libvirt.xml.template'),
-                    'Libvirt XML Template')
+                    utils.abspath('virt/libvirt.qemu.xml.template'),
+                    'Libvirt XML Template for QEmu/KVM')
+flags.DEFINE_string('libvirt_uml_xml_template',
+                    utils.abspath('virt/libvirt.uml.xml.template'),
+                    'Libvirt XML Template for user-mode-linux')
 flags.DEFINE_string('injected_network_template',
-                    utils.abspath('compute/interfaces.template'),
+                    utils.abspath('virt/interfaces.template'),
                     'Template file for injected network')
-
 flags.DEFINE_string('libvirt_type',
                     'kvm',
-                    'Libvirt domain type (kvm, qemu, etc)')
+                    'Libvirt domain type (valid options are: kvm, qemu, uml)')
+flags.DEFINE_string('libvirt_uri',
+                    '',
+                    'Override the default libvirt URI (which is dependent on libvirt_type)')
 
 def get_connection(read_only):
     # These are loaded late so that there's no need to install these
@@ -65,16 +70,42 @@
         libxml2 = __import__('libxml2')
     return LibvirtConnection(read_only)
 
-
 class LibvirtConnection(object):
     def __init__(self, read_only):
+        self.libvirt_uri, template_file = self.get_uri_and_template()
+
+        self.libvirt_xml = open(template_file).read()
+        self._wrapped_conn = None
+        self.read_only = read_only
+
+
+    @property
+    def _conn(self):
+        if not self._wrapped_conn:
+            self._wrapped_conn = self._connect(self.libvirt_uri, self.read_only)
+        return self._wrapped_conn
+
+
+    def get_uri_and_template(self):
+        if FLAGS.libvirt_type == 'uml':
+            uri = FLAGS.libvirt_uri or 'uml:///system'
+            template_file = FLAGS.libvirt_uml_xml_template
+        else:
+            uri = FLAGS.libvirt_uri or 'qemu:///system'
+            template_file = FLAGS.libvirt_xml_template
+        return uri, template_file
+
+
+    def _connect(self, uri, read_only):
         auth = [[libvirt.VIR_CRED_AUTHNAME, libvirt.VIR_CRED_NOECHOPROMPT],
                 'root',
                 None]
+
         if read_only:
-            self._conn = libvirt.openReadOnly('qemu:///system')
+            return libvirt.openReadOnly(uri)
         else:
-            self._conn = libvirt.openAuth('qemu:///system', auth, 0)
+            return libvirt.openAuth(uri, auth, 0)
+
 
 
     def list_instances(self):
@@ -237,14 +268,13 @@
     def toXml(self, instance):
         # TODO(termie): cache?
         logging.debug("Starting the toXML method")
-        libvirt_xml = open(FLAGS.libvirt_xml_template).read()
         xml_info = instance.datamodel.copy()
         # TODO(joshua): Make this xml express the attached disks as well
 
         # TODO(termie): lazy lazy hack because xml is annoying
         xml_info['nova'] = json.dumps(instance.datamodel.copy())
         xml_info['type'] = FLAGS.libvirt_type
-        libvirt_xml = libvirt_xml % xml_info
+        libvirt_xml = self.libvirt_xml % xml_info
         logging.debug("Finished the toXML method")
 
         return libvirt_xml