Merge lp:~terceiro/lava-dispatcher/kvm into lp:lava-dispatcher

[Antonio Terceiro]

This branch makes the QEMU device type backend more flexible and with that flexibility adds a KVM (aka qemu-system-x86_64) device type on top.

  * Extract QEMU ARM-specific options to configuration options
  * Allow qemu binary to actually be overriden for each device type
  * Add a very simple kvm.conf

The provided kvm.conf requires QEMU 1.5, although it should be easy enough to create other that supports older qemu/kvm if needed.

I was able to boot test a custom-made Debian image using this kvm device type. I will post instructions on how to create a LAVA-ready Debian image in the near future.

[lp:~terceiro/lava-dispatcher/kvm]

624. By Antonio Terceiro on 2013-06-06

Remove need for sudo in test images

One of the requirement for test images is auto login as root, so there is no
point in channel those calls through sudo.

625. By Antonio Terceiro on 2013-06-06

Force termination of QEMU process

For some reason it would fail with qemu-system-x86_64 (aka KVM) after a
lava-test-shell run.

[Antonio Terceiro (terceiro)]

I had some issues when running lava-test-shell, so I made another 2
changes:

- removed sudo from package installation calls in lava-test-shell
  debian-based and fedora image support (LAVA-compliant images are
  supposed to boot to a root prompt anyway). This relieves test images
  from having sudo installed.

- forcing termination of the QEMU process with proc.kill(9) - for some
  reason I was hitting an exception in pexpect.close saying (close()
  could not terminate the child using terminate())

So now I have tested and working both a simple boot test and lava-test-shell.

=== modified file 'lava_dispatcher/config.py'
--- lava_dispatcher/config.py	2013-06-04 16:00:29 +0000
+++ lava_dispatcher/config.py	2013-06-06 15:16:23 +0000
@@ -57,6 +57,8 @@
     lmc_dev_arg = schema.StringOption()
     master_str = schema.StringOption(default="root@master")
     pre_connect_command = schema.StringOption()
+    qemu_binary = schema.StringOption(default="qemu-system-arm")
+    qemu_options = schema.StringOption()
     qemu_drive_interface = schema.StringOption()
     qemu_machine_type = schema.StringOption()
     power_on_cmd = schema.StringOption()  # for sdmux
@@ -136,7 +138,6 @@
 
 
 class DispatcherSchema(schema.Schema):
-    default_qemu_binary = schema.StringOption(default="qemu")
     lava_cachedir = schema.StringOption()
     lava_cookies = schema.StringOption()
     lava_image_tmpdir = schema.StringOption()

=== added file 'lava_dispatcher/default-config/lava-dispatcher/device-types/kvm.conf'
--- lava_dispatcher/default-config/lava-dispatcher/device-types/kvm.conf	1970-01-01 00:00:00 +0000
+++ lava_dispatcher/default-config/lava-dispatcher/device-types/kvm.conf	2013-06-06 15:16:23 +0000
@@ -0,0 +1,3 @@
+client_type = qemu
+qemu_binary = qemu-system-x86_64
+qemu_options = -machine accel=kvm:tcg -hda {DISK_IMAGE} -nographic

=== modified file 'lava_dispatcher/default-config/lava-dispatcher/device-types/qemu.conf'
--- lava_dispatcher/default-config/lava-dispatcher/device-types/qemu.conf	2012-06-13 20:47:32 +0000
+++ lava_dispatcher/default-config/lava-dispatcher/device-types/qemu.conf	2013-06-06 15:16:23 +0000
@@ -1,6 +1,7 @@
-# NOTE: you also need to set something like:
-#  DEFAULT_QEMU_BINARY = qemu-system-arm
-# in your lava-dispatcher.conf file
+client_type=qemu
+
+qemu_binary = qemu-system-arm
+qemu_options = -M %(qemu_machine_type)s -drive if=%(qemu_drive_interface)s,cache=writeback,file={DISK_IMAGE} -clock unix -device usb-kbd -device usb-mouse -usb -device usb-net,netdev=mynet -netdev user,id=mynet -net nic -net user -nographic
+
 qemu_machine_type = beaglexm
 qemu_drive_interface = sd
-client_type=qemu

=== modified file 'lava_dispatcher/default-config/lava-dispatcher/lava-dispatcher.conf'
--- lava_dispatcher/default-config/lava-dispatcher/lava-dispatcher.conf	2012-09-19 18:35:42 +0000
+++ lava_dispatcher/default-config/lava-dispatcher/lava-dispatcher.conf	2013-06-06 15:16:23 +0000
@@ -36,7 +36,3 @@
 # Use this variable if you want to use apt-get instead of pip to install lava-test
 # LAVA_TEST_DEB = lava-test
 LAVA_TEST_DEB = 
-
-# The qemu command to use.  Called 'default_qemu_binary' because we
-# want to allow testing custom qemu binaries soon.
-default_qemu_binary = qemu-system-arm

=== modified file 'lava_dispatcher/device/qemu.py'
--- lava_dispatcher/device/qemu.py	2013-02-18 03:06:06 +0000
+++ lava_dispatcher/device/qemu.py	2013-06-06 15:16:23 +0000
@@ -69,22 +69,22 @@
             extract_targz(tb, '%s/%s' % (mntdir, directory))
 
     def power_on(self):
-        qemu_cmd = ('%s -M %s -drive if=%s,cache=writeback,file=%s '
-                    '-clock unix -device usb-kbd -device usb-mouse -usb '
-                    '-device usb-net,netdev=mynet -netdev user,id=mynet '
-                    '-net nic -net user -nographic') % (
-            self.context.config.default_qemu_binary,
-            self.config.qemu_machine_type,
-            self.config.qemu_drive_interface,
-            self._sd_image)
+        qemu_options = self.config.qemu_options.format(
+            DISK_IMAGE=self._sd_image)
+        qemu_cmd = '%s %s' % (self.config.qemu_binary, qemu_options)
         logging.info('launching qemu with command %r' % qemu_cmd)
         proc = self.context.spawn(qemu_cmd, timeout=1200)
         return proc
 
+    def power_off(self, proc):
+        if proc:
+            proc.kill(9)
+            proc.close()
+
     def get_device_version(self):
         try:
             output = subprocess.check_output(
-                [self.context.config.default_qemu_binary, '--version'])
+                [self.config.qemu_binary, '--version'])
             matches = re.findall('[0-9]+\.[0-9a-z.+\-:~]+', output)
             return matches[-1]
         except subprocess.CalledProcessError:

=== modified file 'lava_test_shell/distro/fedora/lava-install-packages'
--- lava_test_shell/distro/fedora/lava-install-packages	2013-04-16 02:59:05 +0000
+++ lava_test_shell/distro/fedora/lava-install-packages	2013-06-06 15:16:23 +0000
@@ -1,3 +1,3 @@
 #!/bin/sh
 
-sudo yum -e 0 -y -q install "$@"
+yum -e 0 -y -q install "$@"

=== modified file 'lava_test_shell/distro/ubuntu/lava-install-packages'
--- lava_test_shell/distro/ubuntu/lava-install-packages	2013-04-15 19:16:44 +0000
+++ lava_test_shell/distro/ubuntu/lava-install-packages	2013-06-06 15:16:23 +0000
@@ -1,4 +1,4 @@
 #!/bin/sh
 
-sudo DEBIAN_FRONTEND=noninteractive apt-get update
-sudo DEBIAN_FRONTEND=noninteractive apt-get install -y -q "$@"
+DEBIAN_FRONTEND=noninteractive apt-get update
+DEBIAN_FRONTEND=noninteractive apt-get install -y -q "$@"