Merge lp:~david-schwarz/lava-dispatcher/ssh_qemu_clients_v3 into lp:lava-dispatcher

[David Schwarz]

This is a cleaner version of the ssh_qemu_clients feature branch--the older version was uploaded by mistake.

[Paul Larson (pwlars)]

As discussed offline, I'd like to see about merging this with what Yongqin has been working on (https://code.launchpad.net/~liuyq0307/lava-dispatcher/test-client/+merge/69757) and make the config read from a file for the boards and specific machines. One general comment here... You seem to want the job to specify the type of client to use, but I'm betting that each test machine is only going to use pretty much one type of client. Can you envision a case where you might have a conmux client that you want to suddenly use as an ssh client? I don't think it typically makes sense, but if someone has a good reason why this might happen, let me know.

I think what we really want, is a directory containing config details for specific board types:
 - bootloader commands to send for booting to the test image
 - partition offsets
 - possibly even client type specific sections

and a config file listing the boards you have in your environment, and which board_type config to use, along with any specific information, or overrides for that individual system.

[David Schwarz (david-schwarz)]

I agree that the machine config rather than job should specify the
client type--particularly because it's entirely possible in our testing
that we may have a mix of different client types in a single
multi-target test job.

I was thinking of splitting the board and machine configs into separate
groups of files to help cut down duplication of common settings.
Machine config (currently in config/hosts in this branch) would include
any machine-specific configuration, including its board type:

panda01.conf:

[machine]
client_type = ssh
hostname = panda01
# References a board-specific config file
board_type = PandaBoard
ip_addr = 192.168.1.1
username = test1
password = test1
has_feature_x = yes

...then the board configuration file could include the information
common to all machines of a particular type, currently being stored in
the config.py Board classes:

PandaBoard.conf

[board]
uboot_cmds = yadda yadda
boot_part = 1
test_part = 2

David

[David Schwarz (david-schwarz)]

   Rereading, I believe we're proposing the same thing.

David

On 08/09/2011 04:46 PM, Paul Larson wrote:
> As discussed offline, I'd like to see about merging this with what Yongqin has been working on (https://code.launchpad.net/~liuyq0307/lava-dispatcher/test-client/+merge/69757) and make the config read from a file for the boards and specific machines. One general comment here... You seem to want the job to specify the type of client to use, but I'm betting that each test machine is only going to use pretty much one type of client. Can you envision a case where you might have a conmux client that you want to suddenly use as an ssh client? I don't think it typically makes sense, but if someone has a good reason why this might happen, let me know.
>
> I think what we really want, is a directory containing config details for specific board types:
> - bootloader commands to send for booting to the test image
> - partition offsets
> - possibly even client type specific sections
>
> and a config file listing the boards you have in your environment, and which board_type config to use, along with any specific information, or overrides for that individual system.

[Paul Larson (pwlars)]

Yep, exactly :)

=== added directory 'config'
=== added directory 'config/hosts'
=== added file 'config/hosts/localhost.json'
--- config/hosts/localhost.json	1970-01-01 00:00:00 +0000
+++ config/hosts/localhost.json	2011-08-09 14:40:36 +0000
@@ -0,0 +1,5 @@
+{
+    "board_name": "localhost",
+    "board_type": "BeagleBoard",
+    "hostname": "localhost"
+}

=== added file 'config/hosts/qemu1.json'
--- config/hosts/qemu1.json	1970-01-01 00:00:00 +0000
+++ config/hosts/qemu1.json	2011-08-09 14:40:36 +0000
@@ -0,0 +1,11 @@
+{
+    "board_name": "qemu1",
+    "board_type": "BeagleBoard",
+    "hostname": "qemu1",
+    "machine_type": "beagle",
+    "qemu_macaddr": "52:54:00:12:34:56",
+    "qemu_disk": "qemu/linaroDiskMaster",
+    "vlan_number": 0,
+    "boot_image": "qemu/startup.bin",
+    "additional_options": "-nographic -no-reboot "
+}

=== added file 'config/hosts/qemu2.json'
--- config/hosts/qemu2.json	1970-01-01 00:00:00 +0000
+++ config/hosts/qemu2.json	2011-08-09 14:40:36 +0000
@@ -0,0 +1,11 @@
+{
+    "board_name": "qemu2",
+    "board_type": "BeagleBoard",
+    "hostname": "qemu2",
+    "machine_type": "beagle",
+    "qemu_macaddr": "12:34:56:78:90:12",
+    "qemu_disk": "qemu/linaroDiskMaster",
+    "vlan_number": 0,
+    "boot_image": "qemu/startup.bin",
+    "additional_options": "-nographic -no-reboot "
+}

=== added file 'config/hosts/vs3.json'
--- config/hosts/vs3.json	1970-01-01 00:00:00 +0000
+++ config/hosts/vs3.json	2011-08-09 14:40:36 +0000
@@ -0,0 +1,6 @@
+{
+    "board_name": "vs3",
+    "hostname": "vs3",
+    "username": "test",
+    "password": "test"
+}

=== added file 'config/hosts/vs8.json'
--- config/hosts/vs8.json	1970-01-01 00:00:00 +0000
+++ config/hosts/vs8.json	2011-08-09 14:40:36 +0000
@@ -0,0 +1,6 @@
+{
+    "board_name": "vs8",
+    "hostname": "vs8",
+    "username": "test",
+    "password": "test"
+}

=== added file 'doc/qemu-ltp-job.json'
--- doc/qemu-ltp-job.json	1970-01-01 00:00:00 +0000
+++ doc/qemu-ltp-job.json	2011-08-09 14:40:36 +0000
@@ -0,0 +1,41 @@
+{
+  "job_name": "qemu_ltp",
+  "target": "qemu1",
+  "target_type": "qemu",
+  "timeout": 18000,
+  "actions": [
+    {
+      "command": "deploy_linaro_image",
+      "parameters":
+        {
+          "rootfs": "http://snapshots.linaro.org/11.05-daily/linaro-developer/20110208/0/images/tar/linaro-n-developer-tar-20110208-0.tar.gz",
+          "hwpack": "http://snapshots.linaro.org/11.05-daily/linaro-hwpacks/panda/20110208/0/images/hwpack/hwpack_linaro-panda_20110208-0_armel_supported.tar.gz"
+        }
+    },
+    {
+      "command": "lava_test_install",
+      "parameters":
+        {
+            "tests": ["ltp"]
+        }
+    },
+    {
+      "command": "boot_linaro_image"
+    },
+    {
+      "command": "lava_test_run",
+      "parameters":
+        {
+          "test_name": "ltp"
+        }
+    },
+    {
+      "command": "submit_results",
+      "parameters":
+        {
+          "server": "http://validation.linaro.org/launch-control",
+          "stream": "/anonymous/panda01-ltp/"
+        }
+    }
+  ]
+}

=== added file 'doc/ssh-ltp-job.json'
--- doc/ssh-ltp-job.json	1970-01-01 00:00:00 +0000
+++ doc/ssh-ltp-job.json	2011-08-09 14:40:36 +0000
@@ -0,0 +1,23 @@
+{
+  "job_name": "ssh_ltp",
+  "target": "vs3",
+  "target_type": "ssh",
+  "timeout": 18000,
+  "actions": [
+    {
+      "command": "lava_test_run",
+      "parameters":
+        {
+          "test_name": "ltp"
+        }
+    },
+    {
+      "command": "submit_results",
+      "parameters":
+        {
+          "server": "http://validation.linaro.org/launch-control",
+          "stream": "/anonymous/panda01-ltp/"
+        }
+    }
+  ]
+}

=== modified file 'lava_dispatcher/__init__.py'
--- lava_dispatcher/__init__.py	2011-08-04 07:19:53 +0000
+++ lava_dispatcher/__init__.py	2011-08-09 14:40:36 +0000
@@ -29,6 +29,8 @@
 from lava_dispatcher.actions import get_all_cmds
 from lava_dispatcher.client import LavaClient, CriticalError, GeneralError
 from lava_dispatcher.android_client import LavaAndroidClient
+from lava_dispatcher.qemu_client import LavaQEMUClient
+from lava_dispatcher.ssh_client import LavaSSHClient
 
 __version__ = "0.1.0"
 
@@ -36,19 +38,22 @@
     def __init__(self, job_json):
         self.job_status = 'pass'
         self.load_job_data(job_json)
-        self.context = LavaContext(self.target, self.image_type)
+        self.context = LavaContext(self.target, self.image_type, self.target_type)
 
     def load_job_data(self, job_json):
         self.job_data = json.loads(job_json)
 
     @property
     def target(self):
-        return self.job_data['target']
+        return self.job_data.get('target')
 
     @property
     def image_type(self):
-        if self.job_data.has_key('image_type'):
-            return self.job_data['image_type']
+        return self.job_data.get('image_type')
+
+    @property
+    def target_type(self):
+        return self.job_data.get('target_type')
 
     def run(self):
         lava_commands = get_all_cmds()
@@ -65,19 +70,26 @@
                 metadata['target.hostname'] = self.target
                 self.context.test_data.add_metadata(metadata)
                 action = lava_commands[cmd['command']](self.context)
+                except_str = ""
                 try:
                     status = 'fail'
                     action.run(**params)
-                except CriticalError, err:
-                    raise err
-                except (pexpect.TIMEOUT, GeneralError), err:
+                except CriticalError as err:
+                    except_str = str(err)
+                    raise
+                except (pexpect.TIMEOUT, GeneralError) as err:
+                    except_str = str(err)
                     pass
-                except Exception, err:
+                except Exception as err:
+                    except_str = str(err)
                     raise
                 else:
                     status = 'pass'
                 finally:
+                    err_msg = ""
+                    command = cmd['command']
                     if status == 'fail':
+<<<<<<< TREE
                         err_msg = "Lava failed at action " + cmd['command'] \
                             + " with error: " + str(err) + "\n"
                         if cmd['command'] == 'lava_test_run':
@@ -91,6 +103,18 @@
                         err_msg = ""
                     self.context.test_data.add_result(cmd['command'], 
                         status, err_msg)
+=======
+                        err_msg = "Lava failed at action %s with error: %s\n" %\
+                                  (command, except_str)
+                        if command == 'lava_test_run':
+                            err_msg += "Lava failed on test: %s" %\
+                                       params.get('test_name')
+                        exc_type, exc_value, exc_traceback = sys.exc_info()
+                        err_msg += repr(traceback.format_tb(exc_traceback))
+                        print >> sys.stderr, err_msg
+                    self.context.test_data.add_result(command, status, err_msg)
+
+>>>>>>> MERGE-SOURCE
         except:
             #Capture all user-defined and non-user-defined critical errors
             self.context.test_data.job_status='fail'
@@ -104,11 +128,16 @@
 
 
 class LavaContext(object):
-    def __init__(self, target, image_type):
-        if image_type != "android":
+    def __init__(self, target, image_type, target_type):
+        if image_type == "android":
+            self._client = LavaAndroidClient(target)
+        elif target_type == "qemu":
+            self._client = LavaQEMUClient(target)
+        elif target_type == "ssh":
+            self._client = LavaSSHClient(target)            
+        else:
+            # conmux / serial
             self._client = LavaClient(target)
-        else:
-            self._client = LavaAndroidClient(target)
         self.test_data = LavaTestData()
 
     @property
@@ -143,8 +172,8 @@
         self._job_status = status
 
     def add_result(self, test_case_id, result, message=""):
-        result_data = { 'test_case_id': test_case_id, 'result': result \
-                    , 'message': message}
+        result_data = {'test_case_id': test_case_id, 'result': result, \
+               'message': message}
         self._test_run['test_results'].append(result_data)
 
     def add_attachment(self, attachment):

=== modified file 'lava_dispatcher/actions/android_basic.py'
--- lava_dispatcher/actions/android_basic.py	2011-06-27 04:55:08 +0000
+++ lava_dispatcher/actions/android_basic.py	2011-08-09 14:40:36 +0000
@@ -21,7 +21,6 @@
 
 from lava_dispatcher.actions import BaseAndroidAction
 from lava_dispatcher.client import OperationFailed
-from lava_dispatcher.android_config import TESTER_STR
 import time
 import pexpect
 import sys

=== modified file 'lava_dispatcher/actions/android_deploy.py'
--- lava_dispatcher/actions/android_deploy.py	2011-08-04 07:19:53 +0000
+++ lava_dispatcher/actions/android_deploy.py	2011-08-09 14:40:36 +0000
@@ -20,7 +20,7 @@
 # along with this program; if not, see <http://www.gnu.org/licenses>.
 
 from lava_dispatcher.actions import BaseAction
-from lava_dispatcher.config import LAVA_IMAGE_TMPDIR, LAVA_IMAGE_URL, MASTER_STR
+from lava_dispatcher.config import LAVA_IMAGE_TMPDIR, LAVA_IMAGE_URL
 import os
 import sys
 import shutil
@@ -101,156 +101,88 @@
 
     def deploy_linaro_android_testboot(self, boottbz2):
         client = self.client
-        client.run_shell_command(
-            'mkfs.vfat /dev/disk/by-label/testboot -n testboot',
-            response = MASTER_STR)
-        client.run_shell_command(
-            'udevadm trigger',
-            response = MASTER_STR)
-        client.run_shell_command(
-            'mkdir -p /mnt/lava/boot',
-            response = MASTER_STR)
-        client.run_shell_command(
-            'mount /dev/disk/by-label/testboot /mnt/lava/boot',
-            response = MASTER_STR)
-        client.run_shell_command(
-            'wget -qO- %s |tar --numeric-owner -C /mnt/lava -xjf -' % boottbz2,
-            response = MASTER_STR)
+        client.run_cmd_master('mkfs.vfat /dev/disk/by-label/testboot '
+                              '-n testboot')
+        client.run_cmd_master('udevadm trigger')
+        client.run_cmd_master('mkdir -p /mnt/lava/boot')
+        client.run_cmd_master('mount /dev/disk/by-label/testboot '
+                              '/mnt/lava/boot')
+        client.run_cmd_master('wget -qO- %s |tar --numeric-owner -C /mnt/lava -xjf -' % boottbz2)
 
         self.recreate_uInitrd()
 
-        client.run_shell_command(
-            'umount /mnt/lava/boot',
-            response = MASTER_STR)
+        client.run_cmd_master('umount /mnt/lava/boot')
 
     def recreate_uInitrd(self):
         client = self.client
-        client.run_shell_command(
-            'mkdir -p ~/tmp/',
-            response = MASTER_STR)
-        client.run_shell_command(
-            'mv /mnt/lava/boot/uInitrd ~/tmp',
-            response = MASTER_STR)
-        client.run_shell_command(
-            'cd ~/tmp/',
-            response = MASTER_STR)
-
-        client.run_shell_command(
-            'dd if=uInitrd of=uInitrd.data ibs=64 skip=1',
-            response = MASTER_STR)
-        client.run_shell_command(
-            'mv uInitrd.data ramdisk.cpio.gz',
-            response = MASTER_STR)
-        client.run_shell_command(
-            'gzip -d ramdisk.cpio.gz; cpio -i -F ramdisk.cpio',
-            response = MASTER_STR)
-        client.run_shell_command(
-            'sed -i "/mount ext4 \/dev\/block\/mmcblk0p3/d" init.rc',
-            response = MASTER_STR)
-        client.run_shell_command(
-            'sed -i "/mount ext4 \/dev\/block\/mmcblk0p5/d" init.rc',
-            response = MASTER_STR)
-        client.run_shell_command(
-            'sed -i "s/mmcblk0p2/mmcblk0p5/g" init.rc',
-            response = MASTER_STR)
-        client.run_shell_command(
-            'sed -i "/export PATH/a \ \ \ \ export PS1 root@linaro: " init.rc',
-            response = MASTER_STR)
-
-        client.run_shell_command(
+        client.run_cmd_master('mkdir -p ~/tmp/')
+        client.run_cmd_master('mv /mnt/lava/boot/uInitrd ~/tmp')
+        client.run_cmd_master('cd ~/tmp/')
+
+        client.run_cmd_master('mv uInitrd.data ramdisk.cpio.gz')
+        client.run_cmd_master(
+            'gzip -d ramdisk.cpio.gz; cpio -i -F ramdisk.cpio')
+        client.run_cmd_master(
+            'sed -i "/mount ext4 \/dev\/block\/mmcblk0p3/d" init.rc')
+        client.run_cmd_master(
+            'sed -i "/mount ext4 \/dev\/block\/mmcblk0p5/d" init.rc')
+        client.run_cmd_master('sed -i "s/mmcblk0p2/mmcblk0p5/g" init.rc')
+        client.run_cmd_master(
+            'sed -i "/export PATH/a \ \ \ \ export PS1 root@linaro: " init.rc')
+
+        client.run_cmd_master(
             'cpio -i -t -F ramdisk.cpio | cpio -o -H newc | \
-                gzip > ramdisk_new.cpio.gz',
-            response = MASTER_STR)
+                gzip > ramdisk_new.cpio.gz')
 
-        client.run_shell_command(
+        client.run_cmd_master(
             'mkimage -A arm -O linux -T ramdisk -n "Android Ramdisk Image" \
-                -d ramdisk_new.cpio.gz uInitrd',
-            response = MASTER_STR)
+                -d ramdisk_new.cpio.gz uInitrd')
 
-        client.run_shell_command(
-            'cd -',
-            response = MASTER_STR)
-        client.run_shell_command(
-            'mv ~/tmp/uInitrd /mnt/lava/boot/uInitrd',
-            response = MASTER_STR)
-        client.run_shell_command(
-            'rm -rf ~/tmp',
-            response = MASTER_STR)
+        client.run_cmd_master('cd -')
+        client.run_cmd_master('mv ~/tmp/uInitrd /mnt/lava/boot/uInitrd')
+        client.run_cmd_master('rm -rf ~/tmp')
 
     def deploy_linaro_android_testrootfs(self, systemtbz2):
         client = self.client
-        client.run_shell_command(
-            'mkfs.ext4 -q /dev/disk/by-label/testrootfs -L testrootfs',
-            response = MASTER_STR)
-        client.run_shell_command(
-            'udevadm trigger',
-            response = MASTER_STR)
-        client.run_shell_command(
-            'mkdir -p /mnt/lava/system',
-            response = MASTER_STR)
-        client.run_shell_command(
-            'mount /dev/disk/by-label/testrootfs /mnt/lava/system',
-            response = MASTER_STR)
+        client.run_cmd_master(
+            'mkfs.ext4 -q /dev/disk/by-label/testrootfs -L testrootfs')
+        client.run_cmd_master('udevadm trigger')
+        client.run_cmd_master('mkdir -p /mnt/lava/system')
+        client.run_cmd_master(
+            'mount /dev/disk/by-label/testrootfs /mnt/lava/system')
         client.run_shell_command(
             'wget -qO- %s |tar --numeric-owner -C /mnt/lava -xjf -' % systemtbz2,
-            response = MASTER_STR, timeout = 600)
+            client.master_str, 600)
 
         sed_cmd = "/dev_mount sdcard \/mnt\/sdcard/c dev_mount sdcard /mnt/sdcard 6 " \
             "/devices/platform/omap/omap_hsmmc.0/mmc_host/mmc0"
-        client.run_shell_command(
-            'sed -i "%s" /mnt/lava/system/etc/vold.fstab' % sed_cmd,
-            response = MASTER_STR)
-        client.run_shell_command(
-            'umount /mnt/lava/system',
-            response = MASTER_STR)
+        client.run_cmd_master(
+            'sed -i "%s" /mnt/lava/system/etc/vold.fstab' % sed_cmd)
+        client.run_cmd_master('umount /mnt/lava/system')
 
     def purge_linaro_android_sdcard(self):
         client = self.client
-        client.run_shell_command(
-            'mkfs.vfat /dev/disk/by-label/sdcard -n sdcard',
-            response = MASTER_STR)
-        client.run_shell_command(
-            'udevadm trigger',
-            response = MASTER_STR)
+        client.run_cmd_master('mkfs.vfat /dev/disk/by-label/sdcard -n sdcard')
+        client.run_cmd_master('udevadm trigger')
 
     def deploy_linaro_android_system(self, systemtbz2):
         client = self.client
-        client.run_shell_command(
-            'mkfs.ext4 -q /dev/disk/by-label/system -L system',
-            response = MASTER_STR)
-        client.run_shell_command(
-            'udevadm trigger',
-            response = MASTER_STR)
-        client.run_shell_command(
-            'mkdir -p /mnt/lava/system',
-            response = MASTER_STR)
-        client.run_shell_command(
-            'mount /dev/disk/by-label/system /mnt/lava/system',
-            response = MASTER_STR)
+        client.run_cmd_master('mkfs.ext4 -q /dev/disk/by-label/system -L system')
+        client.run_cmd_master('udevadm trigger')
+        client.run_cmd_master('mkdir -p /mnt/lava/system')
+        client.run_cmd_master('mount /dev/disk/by-label/system /mnt/lava/system')
         client.run_shell_command(
             'wget -qO- %s |tar --numeric-owner -C /mnt/lava -xjf -' % systemtbz2,
-            response = MASTER_STR, timeout = 600)
-        client.run_shell_command(
-            'umount /mnt/lava/system',
-            response = MASTER_STR)
+            client.master_str, 600)
+        client.run_cmd_master('umount /mnt/lava/system')
 
     def deploy_linaro_android_data(self, datatbz2):
         client = self.client
-        client.run_shell_command(
-            'mkfs.ext4 -q /dev/disk/by-label/data -L data',
-            response = MASTER_STR)
-        client.run_shell_command(
-            'udevadm trigger',
-            response = MASTER_STR)
-        client.run_shell_command(
-            'mkdir -p /mnt/lava/data',
-            response = MASTER_STR)
-        client.run_shell_command(
-            'mount /dev/disk/by-label/data /mnt/lava/data',
-            response = MASTER_STR)
+        client.run_cmd_master('mkfs.ext4 -q /dev/disk/by-label/data -L data')
+        client.run_cmd_master('udevadm trigger')
+        client.run_cmd_master('mkdir -p /mnt/lava/data')
+        client.run_cmd_master('mount /dev/disk/by-label/data /mnt/lava/data')
         client.run_shell_command(
             'wget -qO- %s |tar --numeric-owner -C /mnt/lava -xjf -' % datatbz2,
-            response = MASTER_STR, timeout = 600)
-        client.run_shell_command(
-            'umount /mnt/lava/data',
-            response = MASTER_STR)
+            client.master_str, 600)
+        client.run_cmd_master('umount /mnt/lava/data')

=== modified file 'lava_dispatcher/actions/deploy.py'
--- lava_dispatcher/actions/deploy.py	2011-08-04 07:19:53 +0000
+++ lava_dispatcher/actions/deploy.py	2011-08-09 14:40:36 +0000
@@ -27,7 +27,7 @@
 from tempfile import mkdtemp
 
 from lava_dispatcher.actions import BaseAction
-from lava_dispatcher.config import LAVA_IMAGE_TMPDIR, LAVA_IMAGE_URL, MASTER_STR
+from lava_dispatcher.config import LAVA_IMAGE_TMPDIR, LAVA_IMAGE_URL
 from lava_dispatcher.utils import download, download_with_cache
 from lava_dispatcher.client import CriticalError
 
@@ -126,9 +126,9 @@
 
         image_file = os.path.join(tarball_dir, "lava.img")
         board = client.board
-        cmd = ("sudo linaro-media-create --hwpack-force-yes --dev %s "
-               "--image_file %s --binary %s --hwpack %s --image_size 3G" % (
-                board.type, image_file, rootfs_path, hwpack_path))
+        cmd = ("sudo linaro-media-create --hwpack-force-yes -dev %s "
+               "--image_file %s --binary %s --hwpack %s --image_size 3G" %
+               (board.type, image_file, rootfs_path, hwpack_path))
         rc, output = getstatusoutput(cmd)
         if rc:
             shutil.rmtree(tarball_dir)
@@ -152,46 +152,25 @@
     def deploy_linaro_rootfs(self, rootfs):
         client = self.client
         print "Deploying linaro image"
-        client.run_shell_command(
-            'mkfs.ext3 -q /dev/disk/by-label/testrootfs -L testrootfs',
-            response = MASTER_STR)
-        client.run_shell_command(
-            'udevadm trigger',
-            response = MASTER_STR)
-        client.run_shell_command(
-            'mkdir -p /mnt/root',
-            response = MASTER_STR)
-        client.run_shell_command(
-            'mount /dev/disk/by-label/testrootfs /mnt/root',
-            response = MASTER_STR)
+        client.run_cmd_master(
+            'mkfs.ext3 -q /dev/disk/by-label/testrootfs -L testrootfs')
+        client.run_cmd_master('udevadm trigger')
+        client.run_cmd_master('mkdir -p /mnt/root')
+        client.run_cmd_master('mount /dev/disk/by-label/testrootfs /mnt/root')
         client.run_shell_command(
             'wget -qO- %s |tar --numeric-owner -C /mnt/root -xzf -' % rootfs,
-            response = MASTER_STR, timeout = 3600)
-        client.run_shell_command(
-            'echo linaro > /mnt/root/etc/hostname',
-            response = MASTER_STR)
-        client.run_shell_command(
-            'umount /mnt/root',
-            response = MASTER_STR)
+            client.master_str, 3600)
+        client.run_cmd_master('echo linaro > /mnt/root/etc/hostname')
+        client.run_cmd_master('umount /mnt/root')
 
     def deploy_linaro_bootfs(self, bootfs):
         client = self.client
-        client.run_shell_command(
-            'mkfs.vfat /dev/disk/by-label/testboot -n testboot',
-            response = MASTER_STR)
-        client.run_shell_command(
-            'udevadm trigger',
-            response = MASTER_STR)
-        client.run_shell_command(
-            'mkdir -p /mnt/boot',
-            response = MASTER_STR)
-        client.run_shell_command(
-            'mount /dev/disk/by-label/testboot /mnt/boot',
-            response = MASTER_STR)
-        client.run_shell_command(
-            'wget -qO- %s |tar --numeric-owner -C /mnt/boot -xzf -' % bootfs,
-            response = MASTER_STR)
-        client.run_shell_command(
-            'umount /mnt/boot',
-            response = MASTER_STR)
+        client.run_cmd_master(
+            'mkfs.vfat /dev/disk/by-label/testboot -n testboot')
+        client.run_cmd_master('udevadm trigger')
+        client.run_cmd_master('mkdir -p /mnt/boot')
+        client.run_cmd_master('mount /dev/disk/by-label/testboot /mnt/boot')
+        client.run_cmd_master(
+            'wget -qO- %s |tar --numeric-owner -C /mnt/boot -xzf -' % bootfs)
+        client.run_cmd_master('umount /mnt/boot')
 

=== modified file 'lava_dispatcher/actions/launch_control.py'
--- lava_dispatcher/actions/launch_control.py	2011-06-27 04:55:08 +0000
+++ lava_dispatcher/actions/launch_control.py	2011-08-09 14:40:36 +0000
@@ -22,7 +22,7 @@
 
 import json
 from lava_dispatcher.actions import BaseAction
-from lava_dispatcher.config import LAVA_RESULT_DIR, MASTER_STR, LAVA_SERVER_IP
+from lava_dispatcher.config import LAVA_RESULT_DIR, LAVA_SERVER_IP
 import socket
 from threading import Thread
 import time
@@ -84,32 +84,26 @@
         except:
             client.boot_master_image()
 
-        client.run_shell_command(
-            'mkdir -p /mnt/root', response = MASTER_STR)
-        client.run_shell_command(
-            'mount /dev/disk/by-label/%s /mnt/root' % result_disk,
-            response = MASTER_STR)
-        client.run_shell_command(
-            'mkdir -p /tmp/%s' % LAVA_RESULT_DIR, response = MASTER_STR)
-        client.run_shell_command(
+        client.run_cmd_master('mkdir -p /mnt/root')
+        client.run_cmd_master(
+            'mount /dev/disk/by-label/%s /mnt/root' % result_disk)
+        client.run_cmd_master('mkdir -p /tmp/%s' % LAVA_RESULT_DIR)
+        client.run_cmd_master(
             'cp /mnt/root/%s/*.bundle /tmp/%s' % (LAVA_RESULT_DIR,
-                LAVA_RESULT_DIR), response = MASTER_STR)
-        client.run_shell_command('umount /mnt/root', response = MASTER_STR)
+                LAVA_RESULT_DIR))
+        client.run_cmd_master('umount /mnt/root')
 
         #Upload bundle list-bundle.lst
-        client.run_shell_command('cd /tmp/%s' % LAVA_RESULT_DIR,
-            response = MASTER_STR)
-        client.run_shell_command('ls *.bundle > bundle.lst',
-            response = MASTER_STR)
+        client.run_cmd_master('cd /tmp/%s' % LAVA_RESULT_DIR)
+        client.run_cmd_master('ls *.bundle > bundle.lst')
 
         t = ResultUploader()
         t.start()
         #XXX: Odd problem where we sometimes get stuck here.  This is just
         #     a hacky workaround to see if it's a race
         time.sleep(60)
-        client.run_shell_command(
-            'cat bundle.lst |nc %s %d' % (LAVA_SERVER_IP, t.get_port()),
-            response = MASTER_STR)
+        client.run_cmd_master('cat bundle.lst |nc %s %d' %
+                              (LAVA_SERVER_IP, t.get_port()))
         t.join()
 
         bundle_list = t.get_data().strip().splitlines()
@@ -124,10 +118,9 @@
             #XXX: Odd problem where we sometimes get stuck here.  This is just
             #     a hacky workaround to see if it's a race
             time.sleep(60)
-            client.run_shell_command(
+            client.run_cmd_master(
                 'cat /tmp/%s/%s | nc %s %s' % (LAVA_RESULT_DIR, bundle,
-                    LAVA_SERVER_IP, t.get_port()),
-                response = MASTER_STR)
+                    LAVA_SERVER_IP, t.get_port()))
             t.join()
             content = t.get_data()
 

=== modified file 'lava_dispatcher/actions/lava-test.py'
--- lava_dispatcher/actions/lava-test.py	2011-08-04 07:19:53 +0000
+++ lava_dispatcher/actions/lava-test.py	2011-08-09 14:40:36 +0000
@@ -25,7 +25,7 @@
 import traceback
 from lava_dispatcher.actions import BaseAction
 from lava_dispatcher.client import OperationFailed
-from lava_dispatcher.config import LAVA_RESULT_DIR, MASTER_STR, TESTER_STR
+from lava_dispatcher.config import LAVA_RESULT_DIR
 
 
 class cmd_lava_test_run(BaseAction):
@@ -33,14 +33,13 @@
         #Make sure in test image now
         client = self.client
         client.in_test_shell()
-        client.run_shell_command('mkdir -p %s' % LAVA_RESULT_DIR,
-            response=TESTER_STR)
+        client.run_cmd_tester('mkdir -p %s' % LAVA_RESULT_DIR)
         client.export_display()
         bundle_name = test_name + "-" + datetime.now().strftime("%H%M%S")
         client.run_shell_command(
             'lava-test run %s -o %s/%s.bundle' % (
                 test_name, LAVA_RESULT_DIR, bundle_name),
-            response=TESTER_STR, timeout=timeout)
+            client.tester_str, timeout)
 
 
 class cmd_lava_test_install(BaseAction):
@@ -57,65 +56,42 @@
             client.boot_master_image()
 
         #install bazaar in tester image
-        client.run_shell_command(
-            'mkdir -p /mnt/root',
-            response=MASTER_STR)
-        client.run_shell_command(
-            'mount /dev/disk/by-label/testrootfs /mnt/root',
-            response=MASTER_STR)
-        client.run_shell_command(
-            'cp -f /mnt/root/etc/resolv.conf /mnt/root/etc/resolv.conf.bak',
-            response=MASTER_STR)
-        client.run_shell_command(
-            'cp -L /etc/resolv.conf /mnt/root/etc',
-            response=MASTER_STR)
+        client.run_cmd_master('mkdir -p /mnt/root')
+        client.run_cmd_master('mount /dev/disk/by-label/testrootfs /mnt/root')
+        client.run_cmd_master('cp -f /mnt/root/etc/resolv.conf '
+            '/mnt/root/etc/resolv.conf.bak')
+        client.run_cmd_master('cp -L /etc/resolv.conf /mnt/root/etc')
         #eliminate warning: Can not write log, openpty() failed
         #                   (/dev/pts not mounted?), does not work
-        client.run_shell_command(
-            'mount --rbind /dev /mnt/root/dev',
-            response=MASTER_STR)
-        client.run_shell_command(
-            'chroot /mnt/root apt-get update',
-            response=MASTER_STR)
+        client.run_cmd_master('mount --rbind /dev /mnt/root/dev')
+        client.run_cmd_master('chroot /mnt/root apt-get update')
         #Install necessary packages for build lava-test
         cmd = ('chroot /mnt/root apt-get -y install bzr usbutils python-apt '
             'python-setuptools python-simplejson lsb-release')
         client.run_shell_command(
             cmd,
-            response=MASTER_STR, timeout=2400)
-        client.run_shell_command(
-            'chroot /mnt/root bzr branch lp:lava-test',
-            response=MASTER_STR)
-        client.run_shell_command(
-            'chroot /mnt/root sh -c "cd lava-test && python setup.py install"',
-            response=MASTER_STR)
+            self.client.master_str, 2400)
+        client.run_cmd_master('chroot /mnt/root bzr branch lp:lava-test')
+        client.run_cmd_master('chroot /mnt/root sh -c '
+            '"cd lava-test && python setup.py install"')
 
         #Test if lava-test installed
         try:
             client.run_shell_command(
                 'chroot /mnt/root lava-test help',
-                response="list-tests", timeout=10)
+                "list-tests", 10)
         except:
             tb = traceback.format_exc()
             client.sio.write(tb)
             raise OperationFailed("lava-test deployment failed")
 
         for test in tests:
-            client.run_shell_command(
-                'chroot /mnt/root lava-test install %s' % test,
-                response=MASTER_STR)
+            client.run_cmd_master('chroot /mnt/root lava-test install %s' % test)
         #clean up
-        client.run_shell_command(
-            'cp -f /mnt/root/etc/resolv.conf.bak /mnt/root/etc/resolv.conf',
-            response=MASTER_STR)
-        client.run_shell_command(
-            'rm -rf /mnt/root/lava-test',
-            response=MASTER_STR)
+        client.run_cmd_master('cp -f /mnt/root/etc/resolv.conf.bak ' 
+                '/mnt/root/etc/resolv.conf')
+        client.run_cmd_master('rm -rf /mnt/root/lava-test')
         cmd = ('cat /proc/mounts | awk \'{print $2}\' | grep "^/mnt/root/dev"'
             '| sort -r | xargs umount')
-        client.run_shell_command(
-            cmd,
-            response=MASTER_STR)
-        client.run_shell_command(
-            'umount /mnt/root',
-            response=MASTER_STR)
+        client.run_cmd_master(cmd)
+        client.run_cmd_master('umount /mnt/root')

=== modified file 'lava_dispatcher/android_client.py'
--- lava_dispatcher/android_client.py	2011-07-20 04:45:23 +0000
+++ lava_dispatcher/android_client.py	2011-08-09 14:40:36 +0000
@@ -42,7 +42,7 @@
         """ Check that we are in a shell on the test image
         """
         self.proc.sendline("")
-        id = self.proc.expect([TESTER_STR , pexpect.TIMEOUT])
+        id = self.proc.expect([self.tester_str , pexpect.TIMEOUT])
         if id == 1:
             raise OperationFailed
 
@@ -109,7 +109,7 @@
         network_interface = self.board.network_interface 
         try:
             self.run_shell_command('netcfg %s dhcp' % \
-                network_interface, response = TESTER_STR, timeout = 60)
+                network_interface, self.tester_str, 60)
         except:
             print "netcfg %s dhcp exception" % network_interface
             return False
@@ -120,7 +120,7 @@
         self.proc.sendline('')
         self.proc.sendline(cmd)
         try:
-            id = self.proc.expect([ip_pattern, pexpect.EOF], timeout = 60)
+            id = self.proc.expect([ip_pattern, pexpect.EOF], timeout=60)
         except:
             print "ifconfig can not match ip pattern"
             return False

=== modified file 'lava_dispatcher/client.py'
--- lava_dispatcher/client.py	2011-07-21 16:47:16 +0000
+++ lava_dispatcher/client.py	2011-08-09 14:40:36 +0000
@@ -33,6 +33,8 @@
 
 class LavaClient(object):
     def __init__(self, hostname):
+        self._master_str = MASTER_STR
+        self._tester_str = TESTER_STR
         cmd = "conmux-console %s" % hostname
         self.sio = SerialIO(sys.stdout)
         self.proc = pexpect.spawn(cmd, timeout=3600, logfile=self.sio)
@@ -42,11 +44,19 @@
         # will eventually come from the database
         self.board = BOARDS[hostname]
 
+    @property
+    def master_str(self): 
+        return self._master_str
+    
+    @property
+    def tester_str(self): 
+        return self._tester_str
+    
     def in_master_shell(self):
         """ Check that we are in a shell on the master image
         """
         self.proc.sendline("")
-        id = self.proc.expect([MASTER_STR, pexpect.TIMEOUT])
+        id = self.proc.expect([self.master_str, pexpect.TIMEOUT])
         if id == 1:
             raise OperationFailed
 
@@ -54,7 +64,7 @@
         """ Check that we are in a shell on the test image
         """
         self.proc.sendline("")
-        id = self.proc.expect([TESTER_STR, pexpect.TIMEOUT])
+        id = self.proc.expect([self.tester_str, pexpect.TIMEOUT])
         if id == 1:
             raise OperationFailed
 
@@ -109,11 +119,17 @@
         if response:
             self.proc.expect(response, timeout=timeout)
 
+    def run_cmd_master(self, cmd):
+        self.run_shell_command(cmd, self.master_str)
+        
+    def run_cmd_tester(self, cmd):
+        self.run_shell_command(cmd, self.tester_str)
+
     def check_network_up(self):
         self.proc.sendline("LC_ALL=C ping -W4 -c1 %s" % LAVA_SERVER_IP)
         id = self.proc.expect(["1 received", "0 received",
             "Network is unreachable"], timeout=5)
-        self.proc.expect(MASTER_STR)
+        self.proc.expect(self.master_str)
         if id == 0:
             return True
         else:
@@ -128,14 +144,12 @@
 
     def export_display(self):
         #export the display, ignore errors on non-graphical images
-        self.run_shell_command("su - linaro -c 'DISPLAY=:0 xhost local:'",
-            response=TESTER_STR)
-        self.run_shell_command("export DISPLAY=:0", response=TESTER_STR)
+        self.run_cmd_tester("su - linaro -c 'DISPLAY=:0 xhost local:'")
+        self.run_cmd_tester("export DISPLAY=:0")
 
     def get_seriallog(self):
         return self.sio.getvalue()
 
-
 class SerialIO(file):
     def __init__(self, logfile):
         self.serialio = StringIO()

=== modified file 'lava_dispatcher/config.py'
--- lava_dispatcher/config.py	2011-07-21 16:47:16 +0000
+++ lava_dispatcher/config.py	2011-08-09 14:40:36 +0000
@@ -18,6 +18,16 @@
 # along
 # with this program; if not, see <http://www.gnu.org/licenses>.
 
+import json
+import os
+
+def get_host(hostname): 
+    cfg_path = 'config/hosts/'
+    cfg_file = cfg_path + hostname + '.json'
+    host_file = os.path.join(os.curdir, cfg_file)
+    with open(host_file, 'r') as fp: 
+        return json.load(fp)
+
 """
 This is an ugly hack, the uboot commands for a given board type and the board
 type of a test machine need to come from the device registry.  This is an

=== added file 'lava_dispatcher/qemu_client.py'
--- lava_dispatcher/qemu_client.py	1970-01-01 00:00:00 +0000
+++ lava_dispatcher/qemu_client.py	2011-08-09 14:40:36 +0000
@@ -0,0 +1,135 @@
+# Copyright (C) 2011 Calxeda, Inc.
+#
+# This file is part of LAVA Dispatcher.
+#
+# LAVA Dispatcher is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# LAVA Dispatcher is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, see <http://www.gnu.org/licenses>.
+
+import pexpect
+import sys
+from lava_dispatcher.config import get_host
+from lava_dispatcher.client import LavaClient, SerialIO
+from lava_dispatcher.qemu_config import QEMU_PATH, MASTER_STR, TESTER_STR
+
+
+class LavaQEMUClient(LavaClient):
+
+    def __init__(self, hostname):
+        self.sio = SerialIO(sys.stdout)
+        self.hostname = hostname
+        self.host_config = get_host(hostname)
+        
+        self.start_qemu()
+
+        # Should U-Boot automatically to master image
+        self.proc.expect("Starting kernel")
+        
+        # will eventually come from the database
+        self.board = self.board_type
+        self._master_str = MASTER_STR
+        self._tester_str = TESTER_STR
+        self.in_master_shell()
+
+    @property
+    def macaddr(self):
+        return self.host_config.get('qemu_macaddr')
+
+    @property
+    def disk(self):
+        return self.host_config.get('qemu_disk')
+
+    @property
+    def board_type(self):
+        return self.host_config.get('board_type')
+
+    @property
+    def vlan_number(self):
+        return self.host_config.get('vlan_number')
+   
+    @property
+    def boot_image(self):
+        return self.host_config.get('boot_image')
+    
+    @property
+    def machine_type(self):
+        return self.host_config.get('machine_type')
+    
+    @property
+    def additional_options(self):
+        return self.host_config.get('additional_options')
+    
+    @property
+    def network_arguments(self):
+        if(self.vlan_number is not None and
+           self.macaddr is not None):
+            return ("-net nic,vlan=%d,macaddr=%s -net vde,vlan=%d " %
+                    (self.vlan_number, self.macaddr, self.vlan_number))
+        else:
+            return ""
+        
+    @property
+    def disk_arguments(self):
+        if(self.disk is not None):
+            return ('-drive id=disk,if=ide,file=%s '
+                    '-device ide-drive,drive=disk,bus=ide.0' % self.disk)
+        else:
+            return ""
+
+    def start_qemu(self):
+        cmd = ("%sqemu-system-arm %s -M %s -kernel %s " %
+               (QEMU_PATH, self.additional_options, self.machine_type, 
+                self.boot_image))
+        cmd += self.network_arguments
+        cmd += self.disk_arguments
+            
+        print "Starting QEMU:\n\t%s\n\n" % cmd   
+        self.proc = pexpect.spawn(cmd, timeout=7200, logfile=self.sio)
+        self.proc.delaybeforesend=1
+        
+    def boot_master_image(self):
+        """ reboot the system, and check that we are in a master shell
+        """
+        self.hard_reboot()
+        try:
+            # Necessary to get past U-Boot
+            self.proc.expect("Starting kernel")
+            self.in_master_shell()
+        except:
+            raise
+        
+    def boot_linaro_image(self):
+        """ Reboot the system to the test image
+        """
+        self.hard_reboot()
+        self.enter_uboot()
+        uboot_cmds = self.board.uboot_cmds
+        self.proc.sendline(uboot_cmds[0])
+        for cmd in uboot_cmd[1:]:
+            if self.board.type in ["mx51evk", "mx53loco"]:
+                self.proc.expect(">")
+            else:
+                self.proc.expect("#")
+            self.proc.sendline(cmd)
+        self.in_test_shell()
+
+    def soft_reboot(self):
+        # Assume soft reboot will not work
+        pass
+
+    def hard_reboot(self):
+        self.proc.sendline("reboot")
+        self.proc.expect([pexpect.TIMEOUT], timeout=60)
+        self.proc.sendcontrol('a')
+        self.proc.send('x')
+        self.start_qemu()
+

=== added file 'lava_dispatcher/qemu_config.py'
--- lava_dispatcher/qemu_config.py	1970-01-01 00:00:00 +0000
+++ lava_dispatcher/qemu_config.py	2011-08-09 14:40:36 +0000
@@ -0,0 +1,20 @@
+# Copyright (C) 2011 Calxeda, Inc.
+#
+# This file is part of LAVA Dispatcher.
+#
+# LAVA Dispatcher is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# LAVA Dispatcher is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, see <http://www.gnu.org/licenses>.
+
+from lava_dispatcher.config import MASTER_STR, TESTER_STR
+
+QEMU_PATH = ""

=== added file 'lava_dispatcher/ssh_client.py'
--- lava_dispatcher/ssh_client.py	1970-01-01 00:00:00 +0000
+++ lava_dispatcher/ssh_client.py	2011-08-09 14:40:36 +0000
@@ -0,0 +1,75 @@
+# Copyright (C) 2011 Calxeda, Inc.
+#
+# This file is part of LAVA Dispatcher.
+#
+# LAVA Dispatcher is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# LAVA Dispatcher is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, see <http://www.gnu.org/licenses>.
+
+import pexpect
+import sys
+from lava_dispatcher.client import LavaClient, SerialIO
+from lava_dispatcher.config import get_host
+import pxssh
+
+
+class LavaSSHClient(LavaClient):
+
+    def __init__(self, hostname):
+        self.sio = SerialIO(sys.stdout)
+        self.host_config = get_host(hostname)
+        try:
+            self.proc = pxssh.pxssh(logfile=self.sio)
+            self.proc.login(hostname, self.username, self.password)
+            self.proc.prompt()
+        except pxssh.ExceptionPxssh:
+            raise OperationFailed()
+
+    @property
+    def username(self):
+        return self.host_config['username']
+
+    @property
+    def password(self):
+        return self.host_config['password']
+
+    @property
+    def master_str(self):
+        return self.proc.PROMPT
+
+    @property
+    def tester_str(self):
+        return self.proc.PROMPT
+
+    def soft_reboot(self):
+        self.run_shell_command('reboot', sudo=True)
+
+    def hard_reboot(self):
+        """ No way to hard reboot through ssh """
+        msg = "hard_reboot is not supported by LavaSSHClient"
+        raise OperationFailed(msg)
+
+    def simple_command(self, cmd, timeout=5):
+        self.run_shell_command(cmd, timeout=timeout)
+
+    def run_shell_command(self, cmd, response=None, timeout=-1, sudo=False):
+        expectations = [pexpect.TIMEOUT if not response else response]
+        if sudo:
+            cmd = 'sudo ' + cmd
+            pw_req = "password for %s:" % self.get_username()
+            expectations.append(pw_req)
+
+        self.proc.sendline(cmd)
+        i = self.proc.expect(expectations, timeout=timeout)
+        if i != 0: # pexpect matched pw_req
+            self.proc.sendline(self.get_password())
+            self.proc.expect(expecting[:-1], timeout=timeout)

=== added file 'lava_dispatcher/ssh_config.py'
--- lava_dispatcher/ssh_config.py	1970-01-01 00:00:00 +0000
+++ lava_dispatcher/ssh_config.py	2011-08-09 14:40:36 +0000
@@ -0,0 +1,18 @@
+# Copyright (C) 2011 Calxeda, Inc.
+#
+# This file is part of LAVA Dispatcher.
+#
+# LAVA Dispatcher is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# LAVA Dispatcher is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, see <http://www.gnu.org/licenses>.
+
+CFG_PATH = 'config/hosts/'