Merge lp:~nuclearbob/utah/reorg into lp:utah

[Max Brustkern]

This branch reorganizes the provisioning code.

SSHMixin now lives in ssh.py

TinySQLiteInventory was moved into vm.py since it's only useful for VMs.

vmtools support was removed. We can add uvt support later if we need it.

Everything from libvirtvm.py was moved into vm.py since we don't support any other VM types.

ManualBaremetalSQLiteInventory was moved into an inventory.py file under baremetal.

The baremetal directory is now several packages: utah-baremetal, utah-cobbler, and utah-bamboofeeder. utah-cobbler and utah-bamboofeeder depend on utah-baremetal, and utah-all installs every utah package (including the parser.)

I've built and installed the packages and tested that VM provisioning still works as expected. I'm still working out a way to test a new package on physical installs without disrupting the ongoing tests in magners, but we can test those packages separately later if we need to. The code in them hasn't changed for the most part, it's just moved around a bit.

[Javier Collado (javier.collado)]

Looks good. I've checked that the packages and the documentation build
correctly and that the pass run list succeeds.

review: Approve

=== modified file 'conf/config'
--- conf/config	2012-07-11 13:56:31 +0000
+++ conf/config	2012-12-13 00:38:21 +0000
@@ -1,1 +1,1 @@
-IdentityFile %d/.ssh/%u
+IdentityFile %d/.ssh/utah
 
=== removed file 'conf/utah/uqt-vm-tools.conf'
--- conf/utah/uqt-vm-tools.conf	2012-11-07 19:52:52 +0000
+++ conf/utah/uqt-vm-tools.conf	1970-01-01 00:00:00 +0000
@@ -1,60 +0,0 @@
-# list of all active releases (included devel)
-vm_release_list="hardy lucid oneiric precise quantal raring"
-
-# used by vm-repo (ie 'umt repo' puts stuff in /var/www/debs/testing/..., so
-# vm_repo_url should be the URL to those files. The IP of the host is by
-# default 192.168.122.1, and guests are 192.168.122.2-254.
-vm_repo_url="http://192.168.122.1/debs/testing"
-
-# vm-tools specific settings
-vm_path="/var/lib/utah/vm"		# where to store the VM images
-vm_aptproxy=""                  # set if you want to use a local proxy (like, say, apt-cacher-ng)
-vm_mirror="http://us.archive.ubuntu.com/ubuntu"
-vm_security_mirror=""		# set if want to use a local mirror for security
-vm_mirror_host="us.archive.ubuntu.com"  # Used with the mini iso
-vm_mirror_dir="/ubuntu"         # Used with the mini iso
-vm_dir_iso="/var/cache/utah/iso"             # set to directory containing .iso images
-vm_dir_iso_cache="/var/cache/utah/iso/cache" # set to directory for preseeded iso cache
-vm_image_size="8"               # size in GB of vm-new disk images
-vm_memory="512"                 # 384 is needed for desktops, 256 for servers
-vm_ssh_key=~/.ssh/utah.pub                   # defaults to $HOME/.ssh/id_rsa.pub
-vm_connect="qemu:///system"
-vm_flavor=""                    # blank for default, set to override (eg 'rt')
-vm_archs="i386 amd64"           # architectures to use when using '-p PREFIX'
-                                # with some commands
-vm_extra_packages="python-yaml bzr git"      # list of packages to also
-                                # install via postinstall.sh
-vm_username="utah"      # defaults to your username (`whoami`)
-vm_password="ubuntu"           # defaults to "ubuntu"
-vm_latecmd=""                   # allows specifying an additional late command
-
-vm_root_size="4096"             # Used by deprecated vm-new-vmbuilder tool
-vm_swap_size="1024"             # Used by deprecated vm-new-vmbuilder tool
-
-# vm-iso specific settings (also uses vm-tools settings)
-vm_iso_ndisks="1"		# number of disks
-vm_iso_vcpus="1"		# number of virtual CPUs
-vm_iso_ostype="linux"
-vm_iso_osvariant="ubuntuLucid"
-# see 'man virt-install' for details on 'sparse', 'format' and 'cache'
-vm_iso_fully_allocate="yes"	# fully allocate the disk image (usu. faster)
-vm_iso_disk_format="qcow2"	# raw, qcow2, vmdk, etc
-vm_iso_disk_cache="none"	# none, writethrough, writeback
-
-# vm-new locale
-vm_locale="en_US.UTF-8"
-
-# vm-new keyboard layout
-vm_setkeyboard="false"  # set to "true" to enable the custom settings below
-vm_xkbmodel="pc105"
-vm_xkblayout="ca"
-vm_xkbvariant=""
-vm_xkboptions="lv3:ralt_switch"
-
-# Use an alternate viewer such as gvncviewer or xtightvncviewer. Defaults to
-# virt-viewer if unset. You can specify arguments to the viewer here.
-#vm_viewer="xvnc4viewer"
-#vm_viewer_args=""
-
-# Set to 'no' to disable '.local' mDNS (avahi) lookups for VMs
-vm_host_use_avahi="yes"
 
=== modified file 'debian/control'
--- debian/control	2012-12-11 14:28:43 +0000
+++ debian/control	2012-12-13 00:38:21 +0000
@@ -18,10 +18,34 @@
          python-netifaces, python-paramiko, python-psutil,
          utah-client (=${binary:Version})
 Recommends: dl-ubuntu-test-iso, kvm
-Suggests: cobbler, u-boot-tools, vm-tools
 Description: Ubuntu Test Automation Harness
  Automation framework for testing in Ubuntu
 
+Package: utah-all
+Architecture: all
+Depends: ${misc:Depends}, ${python:Depends}, utah-bamboofeeder, utah-cobbler,
+         utah-parser         
+Description: Ubuntu Test Automation Harness Complete Package
+ Automation framework for testing in Ubuntu, all sections
+
+Package: utah-bamboofeeder
+Architecture: all
+Depends: ${misc:Depends}, ${python:Depends}, u-boot-tools, utah-baremetal
+Description: Ubuntu Test Automation Harness Bamboo Feeder Support
+ Automation framework for testing in Ubuntu, bamboo feeder portion
+
+Package: utah-baremetal
+Architecture: all
+Depends: ${misc:Depends}, ${python:Depends}, utah
+Description: Ubuntu Test Automation Harness Bare Metal Support
+ Automation framework for testing in Ubuntu, bare metal portion
+
+Package: utah-cobbler
+Architecture: all
+Depends: ${misc:Depends}, ${python:Depends}, cobbler, utah-baremetal
+Description: Ubuntu Test Automation Harness Cobbler Support
+ Automation framework for testing in Ubuntu, cobbler portion
+
 Package: utah-client
 Architecture: all
 Depends: ${misc:Depends}, ${python:Depends},
 
=== modified file 'debian/rules'
--- debian/rules	2012-11-30 14:03:47 +0000
+++ debian/rules	2012-12-13 00:38:21 +0000
@@ -22,6 +22,7 @@
 	# utah should only install the provisioning subdirectory, and utah-client should install everything else
 	# except parser.py, which is now its own package
 	# and __init__.py, which is now in the common package
+	# The baremetal subdirectory in the provisioning directory is now 3 packages
 	# If we just use dh_auto_install, all packages will get everything
 	# We start by building the whole thing
 	set -e && for pyvers in $(PYVERS); do python$$pyvers setup.py install --install-layout=deb --root=$(CURDIR); done
@@ -37,6 +38,8 @@
 	rm -r build/*/utah/*
 	# And we put provisioning back
 	mv provisioning build/*/utah
+	# But remove baremetal
+	mv build/*/utah/provisioning/baremetal .
 	# Now we install just the provisioning directory, again using --skip-build into the utah package tree
 	set -e && for pyvers in $(PYVERS); do python$$pyvers setup.py install --skip-build --install-layout=deb --root=$(CURDIR)/debian/utah; done
 	# And now we put back the parser and install that
@@ -47,24 +50,48 @@
 	rm -r build/*/utah/*
 	mv __init__.py build/*/utah
 	set -e && for pyvers in $(PYVERS); do python$$pyvers setup.py install --skip-build --install-layout=deb --root=$(CURDIR)/debian/utah-common; done
+	# Now we install just baremetal
+	rm -r build/*/utah/*
+	mkdir provisioning
+	mv baremetal provisioning
+	mv provisioning build/*/utah
+	# But without cobbler
+	mv build/*/utah/provisioning/baremetal/cobbler.py .
+	# And without bamboofeeder
+	mv build/*/utah/provisioning/baremetal/bamboofeeder.py .
+	set -e && for pyvers in $(PYVERS); do python$$pyvers setup.py install --skip-build --install-layout=deb --root=$(CURDIR)/debian/utah-baremetal; done
+	# Now just cobbler
+	rm -r build/*/utah/provisioning/baremetal/*
+	mv cobbler.py build/*/utah/provisioning/baremetal
+	set -e && for pyvers in $(PYVERS); do python$$pyvers setup.py install --skip-build --install-layout=deb --root=$(CURDIR)/debian/utah-cobbler; done
+	# Now just bamboofeeder
+	rm -r build/*/utah/provisioning/baremetal/*
+	mv bamboofeeder.py build/*/utah/provisioning/baremetal
+	set -e && for pyvers in $(PYVERS); do python$$pyvers setup.py install --skip-build --install-layout=deb --root=$(CURDIR)/debian/utah-bamboofeeder; done
 	# Since the client changes names from client.py to utah, we can't use utah-client.install for that
 	# We also need to make our directory since we do this before dh_auto_install
 	mkdir -p $(CURDIR)/debian/utah-client/usr/bin
 	cp -aL client.py $(CURDIR)/debian/utah-client/usr/bin/utah
 	# phoenix.py needs to be in here to lose the .py
 	cp -aL utah/client/phoenix.py $(CURDIR)/debian/utah-client/usr/bin/phoenix
-	# Since utah and utah-client both got installed using python distutils, they both get an egg info directory
-	# This will conflict if we leave it in both of them, so we remove it from the client
-	rm -r $(CURDIR)/debian/utah-client/usr/lib/python*/dist-packages/utah-*.egg-info
-	# Let's remove it from the parser as well
-	rm -r $(CURDIR)/debian/utah-parser/usr/lib/python*/dist-packages/utah-*.egg-info
-	# We'll remove it from the server and leave it in the client package
-	rm -r $(CURDIR)/debian/utah/usr/lib/python*/dist-packages/utah-*.egg-info
+	# Since all packages get installed using python distutils, they all get an egg info directory
+	# This will conflict if we leave it in all of them, so we remove it from everything but common
+	for egg in $$(ls -d $(CURDIR)/debian/utah*/usr/lib/python*/dist-packages/utah-*.egg-info | grep -v "utah-common");\
+	    do rm -r $$egg;\
+	done
 	# We want to symlink the utah example scripts into /usr/bin, and we need a directory for that
 	mkdir -p $(CURDIR)/debian/utah/usr/bin
+	mkdir -p $(CURDIR)/debian/utah-cobbler/usr/bin
+	mkdir -p $(CURDIR)/debian/utah-bamboofeeder/usr/bin
 	for script in $$(find examples -type f -name "*.py" -printf "%f\n"); do\
 		if [ ! -h "$(CURDIR)/debian/utah/usr/bin/$$script" ]; then\
-			ln -s ../share/utah/examples/$$script $(CURDIR)/debian/utah/usr/bin/$$script;\
+		    if [ "$$script" = "run_test_cobbler.py" ]; then\
+			    ln -s ../share/utah/examples/$$script $(CURDIR)/debian/utah-cobbler/usr/bin/$$script;\
+			elif [ "$$script" = "run_test_bamboo_feeder.py" ]; then\
+			    ln -s ../share/utah/examples/$$script $(CURDIR)/debian/utah-bamboofeeder/usr/bin/$$script;\
+			else\
+			    ln -s ../share/utah/examples/$$script $(CURDIR)/debian/utah/usr/bin/$$script;\
+			fi;\
 		fi;\
 	done
 	dh_auto_install
 
=== modified file 'debian/utah.postinst'
--- debian/utah.postinst	2012-10-31 15:57:11 +0000
+++ debian/utah.postinst	2012-12-13 00:38:21 +0000
@@ -77,7 +77,6 @@
 
     usersetup
 
-    ln -sf /etc/utah/uqt-vm-tools.conf ~utah/.uqt-vm-tools.conf
     ln -sf /etc/utah/shell-profile ~utah/.profile
 
     if ! ([ -f ~utah/.ssh/utah ] && [ -f ~utah/.ssh/utah.pub ])
 
=== modified file 'docs/source/reference.rst'
--- docs/source/reference.rst	2012-11-30 09:36:56 +0000
+++ docs/source/reference.rst	2012-12-13 00:38:21 +0000
@@ -88,6 +88,9 @@
 .. automodule:: utah.provisioning.provisioning
    :members:
 
+.. automodule:: utah.provisioning.ssh
+   :members:
+
 .. automodule:: utah.provisioning.exceptions
    :members:
 
@@ -96,9 +99,15 @@
 
 .. automodule:: utah.provisioning.baremetal
 
+.. automodule:: utah.provisioning.baremetal.inventory
+   :members:
+
 .. automodule:: utah.provisioning.baremetal.cobbler
    :members:
 
+.. automodule:: utah.provisioning.baremetal.bamboofeeder
+   :members:
+
 .. automodule:: utah.provisioning.baremetal.exceptions
    :members:
 
@@ -124,9 +133,6 @@
 .. automodule:: utah.provisioning.vm.exceptions
    :members:
 
-.. automodule:: utah.provisioning.vm.libvirtvm
-   :members:
-
 .. automodule:: utah.provisioning.vm.vm
    :members:
 
 
=== modified file 'examples/run_install_test.py'
--- examples/run_install_test.py	2012-12-08 02:10:12 +0000
+++ examples/run_install_test.py	2012-12-13 00:38:21 +0000
@@ -22,8 +22,7 @@
 from utah.exceptions import UTAHException
 from utah.url import url_argument
 from utah.group import check_user_group, print_group_error_message
-from utah.provisioning.inventory.sqlite import TinySQLiteInventory
-from utah.provisioning.vm.libvirtvm import CustomVM
+from utah.provisioning.vm.vm import CustomVM, TinySQLiteInventory
 from utah.run import run_tests
 
 
 
=== modified file 'examples/run_test_bamboo_feeder.py'
--- examples/run_test_bamboo_feeder.py	2012-12-11 08:41:24 +0000
+++ examples/run_test_bamboo_feeder.py	2012-12-13 00:38:21 +0000
@@ -23,7 +23,8 @@
 from utah.exceptions import UTAHException
 from utah.group import check_user_group, print_group_error_message
 from utah.provisioning.baremetal.bamboofeeder import BambooFeederMachine
-from utah.provisioning.inventory.sqlite import ManualBaremetalSQLiteInventory
+from utah.provisioning.baremetal.inventory import \
+    ManualBaremetalSQLiteInventory
 from utah.run import run_tests
 from utah.url import url_argument
 
 
=== modified file 'examples/run_test_cobbler.py'
--- examples/run_test_cobbler.py	2012-12-11 18:56:17 +0000
+++ examples/run_test_cobbler.py	2012-12-13 00:38:21 +0000
@@ -21,7 +21,9 @@
 from utah import config
 from utah.exceptions import UTAHException
 from utah.group import check_user_group, print_group_error_message
-from utah.provisioning.inventory.sqlite import ManualBaremetalSQLiteInventory
+from utah.provisioning.baremetal.cobbler import CobblerMachine
+from utah.provisioning.baremetal.inventory import \
+    ManualBaremetalSQLiteInventory
 from utah.run import run_tests
 from utah.url import url_argument
 
@@ -105,7 +107,8 @@
                 kw[arg] = value
         if getattr(args, 'type') is not None:
             kw['installtype'] = args.type
-        machine = inventory.request(clean=(not args.no_destroy),
+        machine = inventory.request(CobblerMachine,
+                                    clean=(not args.no_destroy),
                                     debug=args.debug, dlpercentincrement=10,
                                     name=args.name, new=True, **kw)
         exitstatus, locallogs = run_tests(args, machine)
 
=== modified file 'examples/run_test_vm.py'
--- examples/run_test_vm.py	2012-12-10 19:18:18 +0000
+++ examples/run_test_vm.py	2012-12-13 00:38:21 +0000
@@ -21,7 +21,7 @@
 from utah import config
 from utah.exceptions import UTAHException
 from utah.group import check_user_group, print_group_error_message
-from utah.provisioning.inventory.sqlite import TinySQLiteInventory
+from utah.provisioning.vm.vm import TinySQLiteInventory
 from utah.run import run_tests
 
 
 
=== modified file 'examples/utah-user-setup.sh'
--- examples/utah-user-setup.sh	2012-07-19 20:00:02 +0000
+++ examples/utah-user-setup.sh	2012-12-13 00:38:21 +0000
@@ -63,13 +63,6 @@
 
 set +e
 
-CMD=/usr/share/utah/examples/vmtools-user-setup.sh
-if [ -f "$CMD" ]
-then
-  echo "Setting up vm-tools config"
-  keepgoing $CMD $AUTO
-fi
-
 if [ "$LOGOUT" ]
 then
   echo "User groups were updated"
 
=== removed file 'examples/vmtools-user-setup.sh'
--- examples/vmtools-user-setup.sh	2012-07-19 20:00:02 +0000
+++ examples/vmtools-user-setup.sh	1970-01-01 00:00:00 +0000
@@ -1,59 +0,0 @@
-#!/bin/bash
-
-set -e
-
-function keepgoing
-{
-  echo "Going to run:"
-  echo "$@"
-  if [ "$AUTO" ]
-  then
-    $@
-  else
-    echo "Hit enter to continue, enter any text to stop"
-    read ENTRY
-    if [ -n "$ENTRY" ]
-    then
-      exit 1
-    else
-      echo $@ | bash
-#      echo "$@"
-#      $@
-    fi
-  fi
-}
-
-if (echo "$@" | grep -iq "a")
-then
-  export AUTO="auto"
-else
-  export AUTO=""
-fi
-
-if ! [ -f ~/.uqt-vm-tools.conf ]
-then
-  echo "Running vm-new to create ~/.uqv-vm-tools.conf"
-  if [ "$AUTO" ]
-  then
-    CMD="echo \"y\" | vm-new"
-  else
-    CMD=vm-new
-  fi
-  keepgoing $CMD
-fi
-
-SSHKEY=$(echo -e "import utah.config\nprint(utah.config.sshpublickey)" | python)
-if ! (grep -q "^vm_ssh_key=$SSHKEY" ~/.uqt-vm-tools.conf)
-then
-  CMD="sed 's@^vm_ssh_key=[^#]*#@vm_ssh_key=$SSHKEY #@' -i ~/.uqt-vm-tools.conf"
-  echo "Adding SSH key to config file"
-  keepgoing $CMD
-fi
-
-if ! (grep "^vm_extra_packages=" ~/.uqt-vm-tools.conf | grep "python-yaml" | grep "bzr" | grep -q "git")
-then
-  CMD="sed 's/^vm_extra_packages=\"/vm_extra_packages=\"python-yaml bzr git /' -i ~/.uqt-vm-tools.conf"
-  echo "Adding client dependencies to config file"
-  keepgoing $CMD
-fi
-
 
=== modified file 'utah/provisioning/baremetal/bamboofeeder.py'
--- utah/provisioning/baremetal/bamboofeeder.py	2012-12-11 09:37:12 +0000
+++ utah/provisioning/baremetal/bamboofeeder.py	2012-12-13 00:38:21 +0000
@@ -25,13 +25,13 @@
 import tempfile
 
 from utah import config
+from utah.provisioning.baremetal.exceptions import UTAHBMProvisioningException
+from utah.provisioning.baremetal.power import PowerMixin
 from utah.provisioning.provisioning import (
     CustomInstallMixin,
     Machine,
-    SSHMixin,
 )
-from utah.provisioning.baremetal.power import PowerMixin
-from utah.provisioning.baremetal.exceptions import UTAHBMProvisioningException
+from utah.provisioning.ssh import SSHMixin
 from utah.retry import retry
 
 
 
=== modified file 'utah/provisioning/baremetal/cobbler.py'
--- utah/provisioning/baremetal/cobbler.py	2012-12-11 18:38:12 +0000
+++ utah/provisioning/baremetal/cobbler.py	2012-12-13 00:38:21 +0000
@@ -26,13 +26,13 @@
 import time
 
 from utah import config
+from utah.provisioning.baremetal.exceptions import UTAHBMProvisioningException
+from utah.provisioning.baremetal.power import PowerMixin
 from utah.provisioning.provisioning import (
     CustomInstallMixin,
     Machine,
-    SSHMixin,
 )
-from utah.provisioning.baremetal.exceptions import UTAHBMProvisioningException
-from utah.provisioning.baremetal.power import PowerMixin
+from utah.provisioning.ssh import SSHMixin
 from utah.retry import retry
 
 
 
=== added file 'utah/provisioning/baremetal/inventory.py'
--- utah/provisioning/baremetal/inventory.py	1970-01-01 00:00:00 +0000
+++ utah/provisioning/baremetal/inventory.py	2012-12-13 00:38:21 +0000
@@ -0,0 +1,153 @@
+# Ubuntu Testing Automation Harness
+# Copyright 2012 Canonical Ltd.
+
+# This program is free software: you can redistribute it and/or modify it
+# under the terms of the GNU General Public License version 3, as published
+# by the Free Software Foundation.
+
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranties of
+# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
+# PURPOSE.  See the GNU General Public License for more details.
+
+# You should have received a copy of the GNU General Public License along
+# with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+"""
+Provide inventory functions specific to bare metal deployments.
+"""
+
+import os
+import psutil
+from utah.provisioning.inventory.exceptions import \
+    UTAHProvisioningInventoryException
+from utah.provisioning.inventory.sqlite import SQLiteInventory
+
+
+class ManualBaremetalSQLiteInventory(SQLiteInventory):
+    """
+    Keep an inventory of manually entered machines.
+    All columns other than machineid, name, and state are assumed to be
+    arguments for system creation (i.e., with cobbler).
+    """
+    # TODO: rename this
+    def __init__(self, db='~/.utah-baremetal-inventory',
+                 lockfile='~/.utah-baremetal-lock', *args, **kw):
+        db = os.path.expanduser(db)
+        lockfile = os.path.expanduser(lockfile)
+        if not os.path.isfile(db):
+            raise UTAHProvisioningInventoryException(
+                'No machine database found at ' + db)
+        super(ManualBaremetalSQLiteInventory, self).__init__(*args, db=db,
+                                                             lockfile=lockfile,
+                                                             **kw)
+        machines_count = (self.connection
+                          .execute('SELECT COUNT(*) FROM machines')
+                          .fetchall()[0][0])
+        if machines_count == 0:
+            raise UTAHProvisioningInventoryException('No machines in database')
+        self.machines = []
+
+    def request(self, machinetype, name=None, *args, **kw):
+        query = 'SELECT * FROM machines'
+        queryvars = []
+        if name is not None:
+            query += ' WHERE name=?'
+            queryvars.append(name)
+        result = self.connection.execute(query, queryvars).fetchall()
+        if result is None:
+            raise UTAHProvisioningInventoryException(
+                'No machines meet criteria')
+        else:
+            for minfo in result:
+                machineinfo = dict(minfo)
+                if machineinfo['state'] == 'available':
+                    return self._take(machineinfo, machinetype, *args, **kw)
+            for minfo in result:
+                machineinfo = dict(minfo)
+                pid = machineinfo['pid']
+                try:
+                    if not (psutil.pid_exists(pid) and ('utah' in
+                            ' '.join(psutil.Process(pid).cmdline)
+                            or 'run_test' in
+                            ' '.join(psutil.Process(pid).cmdline))):
+                        return self._take(machineinfo, machinetype,
+                                          *args, **kw)
+                except ValueError:
+                    continue
+
+        raise UTAHProvisioningInventoryException(
+            'All machines meeting criteria are currently unavailable')
+
+    def _take(self, machineinfo, machinetype, *args, **kw):
+        machineid = machineinfo.pop('machineid')
+        name = machineinfo.pop('name')
+        state = machineinfo.pop('state')
+        machineinfo.pop('pid')
+        update = self.connection.execute(
+            "UPDATE machines SET pid=?, state='provisioned' WHERE machineid=?"
+            "AND state=?",
+            [os.getpid(), machineid, state]).rowcount
+        if update == 1:
+            machine = machinetype(*args, inventory=self,
+                                     machineinfo=machineinfo, name=name, **kw)
+            self.machines.append(machine)
+            return machine
+        elif update == 0:
+            raise UTAHProvisioningInventoryException(
+                'Machine was requested by another process '
+                'before we could request it')
+        elif update > 1:
+            raise UTAHProvisioningInventoryException(
+                'Multiple machines exist '
+                'matching those criteria; '
+                'database ' + self.db + ' may be corrupt')
+        else:
+            raise UTAHProvisioningInventoryException(
+                'Negative rowcount returned '
+                'when attempting to request machine')
+
+    def release(self, machine=None, name=None):
+        if machine is not None:
+            name = machine.name
+        if name is None:
+            raise UTAHProvisioningInventoryException(
+                'name required to release a machine')
+        query = "UPDATE machines SET state='available' WHERE name=?"
+        queryvars = [name]
+        update = self.connection.execute(query, queryvars).rowcount
+        if update == 1:
+            if machine is not None:
+                if machine in self.machines:
+                    self.machines.remove(machine)
+        elif update == 0:
+            raise UTAHProvisioningInventoryException(
+                'SERIOUS ERROR: Another process released this machine '
+                'before we could, which means two processes provisioned '
+                'the same machine simultaneously')
+        elif update > 1:
+            raise UTAHProvisioningInventoryException(
+                'Multiple machines exist matching those criteria; '
+                'database ' + self.db + ' may be corrupt')
+        else:
+            raise UTAHProvisioningInventoryException(
+                'Negative rowcount returned '
+                'when attempting to release machine')
+
+    # Here is how I currently create the database:
+    # CREATE TABLE machines (machineid INTEGER PRIMARY KEY,
+    #                        name TEXT NOT NULL UNIQUE,
+    #                        state TEXT default 'available',
+    #                        pid INT,
+    #                        [mac-address] TEXT NOT NULL UNIQUE,
+    #                        [power-address] TEXT DEFAULT '10.97.0.13',
+    #                        [power-id] TEXT,
+    #                        [power-user] TEXT DEFAULT 'ubuntu',
+    #                        [power-pass] TEXT DEFAULT 'ubuntu',
+    #                        [power-type] TEXT DEFAULT 'sentryswitch_cdu');
+
+    # Here is how I currently populate the database:
+    # INSERT INTO machines (name, [mac-address], [power-id])
+    # VALUES ('acer-veriton-01-Pete', 'd0:27:88:9f:73:ce', 'Veriton_1');
+    # INSERT INTO machines (name, [mac-address], [power-id])
+    # VALUES ('acer-veriton-02-Pete', 'd0:27:88:9b:84:5b', 'Veriton_2');
 
=== modified file 'utah/provisioning/inventory/sqlite.py'
--- utah/provisioning/inventory/sqlite.py	2012-12-11 18:45:40 +0000
+++ utah/provisioning/inventory/sqlite.py	2012-12-13 00:38:21 +0000
@@ -17,12 +17,7 @@
 
 import sqlite3
 import os
-import psutil
-from utah.provisioning.inventory.exceptions import \
-    UTAHProvisioningInventoryException
 from utah.provisioning.inventory.inventory import Inventory
-from utah.provisioning.vm.libvirtvm import CustomVM
-from utah.provisioning.baremetal.cobbler import CobblerMachine
 
 
 class SQLiteInventory(Inventory):
@@ -41,181 +36,3 @@
     def delete(self):
         os.unlink(self.db)
         super(SQLiteInventory, self).delete()
-
-
-class TinySQLiteInventory(SQLiteInventory):
-    """
-    Tiny SQLite inventory that implements request, release, and destroy.
-
-    No authentication or conflict checking currently exists.
-    """
-    def __init__(self, *args, **kw):
-        """
-        Initialize simple database.
-        """
-        super(TinySQLiteInventory, self).__init__(*args, **kw)
-        self.connection.execute(
-            'CREATE TABLE IF NOT EXISTS '
-            'machines(machineid INTEGER PRIMARY KEY, state TEXT)')
-
-    def request(self, machinetype=CustomVM, *args, **kw):
-        """
-        Takes a Machine class as machinetype, and passes the newly generated
-        machineid along with all other arguments to that class's constructor,
-        returning the resulting object.
-        """
-        cursor = self.connection.cursor()
-        cursor.execute("INSERT INTO machines (state) VALUES ('provisioned')")
-        machineid = cursor.lastrowid
-        return machinetype(machineid=machineid, *args, **kw)
-
-    def release(self, machineid):
-        """
-        Updates the database to indicate the machine is available.
-        """
-        if self.connection.execute(
-                "UPDATE machines SET state='available' WHERE machineid=?",
-                [machineid]):
-            return True
-        else:
-            return False
-
-    def destroy(self, machineid):
-        """
-        Updates the database to indicate the machine is destroyed, but does not
-        destroy the machine.
-        """
-        if self.connection.execute(
-                "UPDATE machines SET state='destroyed' ""WHERE machineid=?",
-                [machineid]):
-            return True
-        else:
-            return False
-
-
-class ManualBaremetalSQLiteInventory(SQLiteInventory):
-    """
-    Keep an inventory of manually entered machines.
-    All columns other than machineid, name, and state are assumed to be
-    arguments for system creation (i.e., with cobbler).
-    """
-    # TODO: rename this
-    def __init__(self, db='~/.utah-baremetal-inventory',
-                 lockfile='~/.utah-baremetal-lock', *args, **kw):
-        db = os.path.expanduser(db)
-        lockfile = os.path.expanduser(lockfile)
-        if not os.path.isfile(db):
-            raise UTAHProvisioningInventoryException(
-                'No machine database found at ' + db)
-        super(ManualBaremetalSQLiteInventory, self).__init__(*args, db=db,
-                                                             lockfile=lockfile,
-                                                             **kw)
-        machines_count = (self.connection
-                          .execute('SELECT COUNT(*) FROM machines')
-                          .fetchall()[0][0])
-        if machines_count == 0:
-            raise UTAHProvisioningInventoryException('No machines in database')
-        self.machines = []
-
-    def request(self, machinetype=CobblerMachine, name=None, *args, **kw):
-        query = 'SELECT * FROM machines'
-        queryvars = []
-        if name is not None:
-            query += ' WHERE name=?'
-            queryvars.append(name)
-        result = self.connection.execute(query, queryvars).fetchall()
-        if result is None:
-            raise UTAHProvisioningInventoryException(
-                'No machines meet criteria')
-        else:
-            for minfo in result:
-                machineinfo = dict(minfo)
-                if machineinfo['state'] == 'available':
-                    return self._take(machineinfo, *args, **kw)
-            for minfo in result:
-                machineinfo = dict(minfo)
-                pid = machineinfo['pid']
-                try:
-                    if not (psutil.pid_exists(pid) and ('utah' in
-                            ' '.join(psutil.Process(pid).cmdline)
-                            or 'run_test' in
-                            ' '.join(psutil.Process(pid).cmdline))):
-                        return self._take(machineinfo, *args, **kw)
-                except ValueError:
-                    continue
-
-        raise UTAHProvisioningInventoryException(
-            'All machines meeting criteria are currently unavailable')
-
-    def _take(self, machineinfo, *args, **kw):
-        machineid = machineinfo.pop('machineid')
-        name = machineinfo.pop('name')
-        state = machineinfo.pop('state')
-        machineinfo.pop('pid')
-        update = self.connection.execute(
-            "UPDATE machines SET pid=?, state='provisioned' WHERE machineid=?"
-            "AND state=?",
-            [os.getpid(), machineid, state]).rowcount
-        if update == 1:
-            machine = CobblerMachine(*args, inventory=self,
-                                     machineinfo=machineinfo, name=name, **kw)
-            self.machines.append(machine)
-            return machine
-        elif update == 0:
-            raise UTAHProvisioningInventoryException(
-                'Machine was requested by another process '
-                'before we could request it')
-        elif update > 1:
-            raise UTAHProvisioningInventoryException(
-                'Multiple machines exist '
-                'matching those criteria; '
-                'database ' + self.db + ' may be corrupt')
-        else:
-            raise UTAHProvisioningInventoryException(
-                'Negative rowcount returned '
-                'when attempting to request machine')
-
-    def release(self, machine=None, name=None):
-        if machine is not None:
-            name = machine.name
-        if name is None:
-            raise UTAHProvisioningInventoryException(
-                'name required to release a machine')
-        query = "UPDATE machines SET state='available' WHERE name=?"
-        queryvars = [name]
-        update = self.connection.execute(query, queryvars).rowcount
-        if update == 1:
-            if machine is not None:
-                if machine in self.machines:
-                    self.machines.remove(machine)
-        elif update == 0:
-            raise UTAHProvisioningInventoryException(
-                'SERIOUS ERROR: Another process released this machine '
-                'before we could, which means two processes provisioned '
-                'the same machine simultaneously')
-        elif update > 1:
-            raise UTAHProvisioningInventoryException(
-                'Multiple machines exist matching those criteria; '
-                'database ' + self.db + ' may be corrupt')
-        else:
-            raise UTAHProvisioningInventoryException(
-                'Negative rowcount returned '
-                'when attempting to release machine')
-
-    # Here is how I currently create the database:
-    # CREATE TABLE machines (machineid INTEGER PRIMARY KEY,
-    #                        name TEXT NOT NULL UNIQUE,
-    #                        state TEXT default 'available',
-    #                        pid TEXT,
-    #                        [mac-address] TEXT NOT NULL UNIQUE,
-    #                        [power-address] TEXT DEFAULT '10.97.0.13',
-    #                        [power-id] TEXT,
-    #                        [power-user] TEXT DEFAULT 'ubuntu',
-    #                        [power-pass] TEXT DEFAULT 'ubuntu',
-    #                        [power-type] TEXT DEFAULT 'sentryswitch_cdu');
-
-    # Here is how I currently populate the database:
-    # INSERT INTO machines (name, [mac-address], [power-id])
-    # VALUES ('acer-veriton-01-Pete', 'd0:27:88:9f:73:ce', 'Veriton_1');
-    # INSERT INTO machines (name, [mac-address], [power-id])
-    # VALUES ('acer-veriton-02-Pete', 'd0:27:88:9b:84:5b', 'Veriton_2');
 
=== modified file 'utah/provisioning/provisioning.py'
--- utah/provisioning/provisioning.py	2012-12-11 09:50:08 +0000
+++ utah/provisioning/provisioning.py	2012-12-13 00:38:21 +0000
@@ -18,26 +18,24 @@
 Functions here should apply to multiple machine types (VM, bare metal, etc.)
 """
 
+import apt.cache
 import logging
 import logging.handlers
 import os
 import pipes
+import re
 import shutil
-import socket
 import subprocess
 import sys
 import time
 import urllib
 import uuid
-import paramiko
-import re
-import apt.cache
 
 from glob import glob
-from stat import S_ISDIR
 
 import utah.timeout
 
+from utah import config
 from utah.commandstr import commandstr
 from utah.iso import ISO
 from utah.orderedcollections import (
@@ -47,7 +45,6 @@
 from utah.preseed import Preseed
 from utah.provisioning.exceptions import UTAHProvisioningException
 from utah.retry import retry
-from utah import config
 
 
 class Machine(object):
@@ -679,243 +676,6 @@
                 'Try rerunning install.')
 
 
-class SSHMixin(object):
-    """
-    Provide common commands for machines accessed via ssh.
-    """
-    def __init__(self, *args, **kwargs):
-        # Note: Since this is a mixin it doesn't expect any argument
-        # However, it calls super to initialize any other mixins in the mro
-        super(SSHMixin, self).__init__(*args, **kwargs)
-        ssh_client = paramiko.SSHClient()
-        ssh_client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
-        self.ssh_client = ssh_client
-
-    def run(self, command, _quiet=None, root=False, timeout=None):
-        """
-        Run a command using ssh.
-        """
-        if isinstance(command, basestring):
-            commandstring = command
-        else:
-            commandstring = ' '.join(command)
-        if root:
-            user = 'root'
-        else:
-            user = config.user
-
-        self.activecheck()
-        # Some commands expect run to return the output status of the command
-        # We're going to try the method described here:
-        # http://stackoverflow.com/questions/3562403/
-        # With additions from here:
-        # http://od-eon.com/blogs/
-        # stefan/automating-remote-commands-over-ssh-paramiko/
-        self.logger.debug('Connecting SSH')
-        self.ssh_client.connect(self.name,
-                                username=user,
-                                key_filename=config.sshprivatekey)
-
-        self.logger.debug('Opening SSH session')
-        channel = self.ssh_client.get_transport().open_session()
-
-        self.logger.info('Running command through SSH: ' + commandstring)
-        stdout = channel.makefile('rb')
-        stderr = channel.makefile_stderr('rb')
-        if timeout is None:
-            channel.exec_command(commandstring)
-        else:
-            utah.timeout.timeout(timeout, channel.exec_command, commandstring)
-        retval = channel.recv_exit_status()
-
-        self.logger.debug('Closing SSH connection')
-        self.ssh_client.close()
-
-        log_level = logging.DEBUG if retval == 0 else logging.WARNING
-        log_message = 'Return code: {}'.format(retval)
-        self.logger.log(log_level, log_message)
-
-        self.logger.debug('Standard output follows:')
-        stdout_lines = stdout.readlines()
-        for line in stdout_lines:
-            self.logger.debug(line.strip())
-
-        self.logger.debug('Standard error follows:')
-        stderr_lines = stderr.readlines()
-        for line in stderr_lines:
-            self.logger.debug(line.strip())
-
-        return retval, ''.join(stdout_lines), ''.join(stderr_lines)
-
-    def uploadfiles(self, files, target=os.path.normpath('/tmp/')):
-        """
-        Copy a file or list of files to a target directory on the machine.
-        """
-        if isinstance(files, basestring):
-            files = [files]
-
-        self.activecheck()
-        self.ssh_client.connect(self.name,
-                                username=config.user,
-                                key_filename=config.sshprivatekey)
-        sftp_client = self.ssh_client.open_sftp()
-        failed = []
-        try:
-            for localpath in files:
-                if os.path.isfile(localpath):
-                    self.logger.info('Uploading ' + localpath
-                                     + ' from the host to ' + target
-                                     + ' on the machine')
-                    remotepath = os.path.join(target,
-                                              os.path.basename(localpath))
-                    sftp_client.put(localpath, remotepath)
-                else:
-                    failed.append(localpath)
-        finally:
-            sftp_client.close()
-            if len(failed) > 0:
-                err = UTAHProvisioningException('Files do not exist: '
-                                                + ' '.join(failed))
-                err.files = failed
-                raise err
-
-    def downloadfiles(self, files, target=os.path.normpath('/tmp/')):
-        """
-        Copy a file or list of files from the machine to a target directory on
-        the local system.
-        """
-        # TODO: check for directories and recurse into them
-        if isinstance(files, basestring):
-            files = [files]
-
-        self.activecheck()
-        self.ssh_client.connect(self.name,
-                                username=config.user,
-                                key_filename=config.sshprivatekey)
-        sftp_client = self.ssh_client.open_sftp()
-        if os.path.isdir(target):
-            get_localpath = lambda remotepath: \
-                os.path.join(target, os.path.basename(remotepath))
-        else:
-            get_localpath = lambda remotepath: target
-
-        try:
-            for remotepath in files:
-                localpath = get_localpath(remotepath)
-                self.logger.info('Downloading ' + remotepath
-                                 + ' from the machine to ' + target
-                                 + ' on the host')
-                sftp_client.get(remotepath, localpath)
-        finally:
-            sftp_client.close()
-
-    def downloadfilesrecursive(self, files, target=os.path.normpath('/tmp/')):
-        """
-        Recursively copy all files in files to the target directory target.
-        """
-        self.activecheck()
-        self.ssh_client.connect(self.name,
-                                username=config.user,
-                                key_filename=config.sshprivatekey)
-        sftp_client = self.ssh_client.open_sftp()
-        myfiles = []
-
-        if isinstance(files, basestring):
-            files = [files]
-
-        for myfile in files:
-            newtarget = os.path.join(target, os.path.basename(myfile))
-            if S_ISDIR(sftp_client.stat(myfile).st_mode):
-                self.logger.debug(myfile + ' is a directory, recursing')
-                if not os.path.isdir(newtarget):
-                    self.logger.debug('Attempting to create ' + newtarget)
-                    os.makedirs(newtarget)
-                myfiles = [os.path.join(myfile, x)
-                           for x in sftp_client.listdir(myfile)]
-#                for basename in sftp_client.listdir(dirname):
-#                    myfile = os.path.join(dirname, basename)
-#                    if S_ISDIR(sftp_client.stat(myfile).st_mode):
-#                        if not os.path.isdir(newtarget):
-#                            os.makedirs(newtarget)
-#                        self.downloadfilesrecursive(myfile, newtarget)
-#                    else:
-#                        myfiles.append(myfile)
-                self.downloadfilesrecursive(myfiles, newtarget)
-            else:
-                self.downloadfiles(myfile, newtarget)
-
-    def destroy(self, *args, **kw):
-        """
-        Clean up known hosts for machine.
-        """
-        # TODO: evaluate value of known_hosts with paramiko
-        self.logger.info('Removing machine addresses '
-                         'from ssh known_hosts file')
-        addresses = [self.name]
-        try:
-            addresses.append(socket.gethostbyname(self.name))
-        except socket.gaierror as err:
-            if err.errno in [-2, -5]:
-                self.logger.debug(self.name
-                                  + ' is not resolvable, '
-                                  + 'so not removing from known_hosts')
-            else:
-                raise err
-
-        old_host_keys = self.ssh_client.get_host_keys()
-        new_host_keys = paramiko.HostKeys()
-        addresses = set(addresses)
-        for address, key in old_host_keys.iteritems():
-            # Skip keys so that they don't get added
-            # into the new keys (i.e. they're removed)
-            if address in addresses:
-                continue
-            new_host_keys[address] = key
-        new_host_keys.save(config.sshknownhosts)
-        self.ssh_client.close()
-
-        super(SSHMixin, self).destroy(*args, **kw)
-
-    def sshcheck(self, timeout=config.checktimeout):
-        """
-        Sleep for a while and check if the machine is available via ssh.
-        Return a retryable exception if it is not.
-        Intended for use with retry.
-        """
-        self.logger.info('Sleeping {timeout} seconds'
-                         .format(timeout=timeout))
-        time.sleep(timeout)
-        self.logger.info('Checking for ssh availability')
-        try:
-            self.ssh_client.connect(self.name,
-                                    username=config.user,
-                                    key_filename=config.sshprivatekey)
-        except socket.error as err:
-            raise UTAHProvisioningException(str(err), retry=True)
-
-    def sshpoll(self, timeout=None,
-                checktimeout=config.checktimeout, logmethod=None):
-        """
-        Run sshcheck over and over until timeout expires.
-        """
-        if timeout is None:
-            timeout = self.boottimeout
-        if logmethod is None:
-            logmethod = self.logger.debug
-        utah.timeout.timeout(timeout, retry, self.sshcheck, checktimeout,
-                             logmethod=logmethod)
-
-    def activecheck(self):
-        """
-        Start the machine if needed, and check for SSH login.
-        """
-        self.logger.debug('Checking if machine is active')
-        self.provisioncheck()
-        if not self.active:
-            self._start()
-            self.sshcheck()
-
-
 class CustomInstallMixin(object):
     """
     Provide routines for unpacking necessary boot files from images,
 
=== added file 'utah/provisioning/ssh.py'
--- utah/provisioning/ssh.py	1970-01-01 00:00:00 +0000
+++ utah/provisioning/ssh.py	2012-12-13 00:38:21 +0000
@@ -0,0 +1,261 @@
+# Ubuntu Testing Automation Harness
+# Copyright 2012 Canonical Ltd.
+
+# This program is free software: you can redistribute it and/or modify it
+# under the terms of the GNU General Public License version 3, as published
+# by the Free Software Foundation.
+
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranties of
+# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
+# PURPOSE.  See the GNU General Public License for more details.
+
+# You should have received a copy of the GNU General Public License along
+# with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+"""
+Provide a mixin class for machines with SSH support.
+"""
+
+import logging
+import os
+import paramiko
+import socket
+import time
+
+from stat import S_ISDIR
+
+import utah.timeout
+
+from utah import config
+from utah.provisioning.exceptions import UTAHProvisioningException
+from utah.retry import retry
+
+
+class SSHMixin(object):
+    """
+    Provide common commands for machines accessed via ssh.
+    """
+    def __init__(self, *args, **kwargs):
+        # Note: Since this is a mixin it doesn't expect any argument
+        # However, it calls super to initialize any other mixins in the mro
+        super(SSHMixin, self).__init__(*args, **kwargs)
+        ssh_client = paramiko.SSHClient()
+        ssh_client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
+        self.ssh_client = ssh_client
+
+    def run(self, command, _quiet=None, root=False, timeout=None):
+        """
+        Run a command using ssh.
+        """
+        if isinstance(command, basestring):
+            commandstring = command
+        else:
+            commandstring = ' '.join(command)
+        if root:
+            user = 'root'
+        else:
+            user = config.user
+
+        self.activecheck()
+        # Some commands expect run to return the output status of the command
+        # We're going to try the method described here:
+        # http://stackoverflow.com/questions/3562403/
+        # With additions from here:
+        # http://od-eon.com/blogs/
+        # stefan/automating-remote-commands-over-ssh-paramiko/
+        self.logger.debug('Connecting SSH')
+        self.ssh_client.connect(self.name,
+                                username=user,
+                                key_filename=config.sshprivatekey)
+
+        self.logger.debug('Opening SSH session')
+        channel = self.ssh_client.get_transport().open_session()
+
+        self.logger.info('Running command through SSH: ' + commandstring)
+        stdout = channel.makefile('rb')
+        stderr = channel.makefile_stderr('rb')
+        if timeout is None:
+            channel.exec_command(commandstring)
+        else:
+            utah.timeout.timeout(timeout, channel.exec_command, commandstring)
+        retval = channel.recv_exit_status()
+
+        self.logger.debug('Closing SSH connection')
+        self.ssh_client.close()
+
+        log_level = logging.DEBUG if retval == 0 else logging.WARNING
+        log_message = 'Return code: {}'.format(retval)
+        self.logger.log(log_level, log_message)
+
+        self.logger.debug('Standard output follows:')
+        stdout_lines = stdout.readlines()
+        for line in stdout_lines:
+            self.logger.debug(line.strip())
+
+        self.logger.debug('Standard error follows:')
+        stderr_lines = stderr.readlines()
+        for line in stderr_lines:
+            self.logger.debug(line.strip())
+
+        return retval, ''.join(stdout_lines), ''.join(stderr_lines)
+
+    def uploadfiles(self, files, target=os.path.normpath('/tmp/')):
+        """
+        Copy a file or list of files to a target directory on the machine.
+        """
+        if isinstance(files, basestring):
+            files = [files]
+
+        self.activecheck()
+        self.ssh_client.connect(self.name,
+                                username=config.user,
+                                key_filename=config.sshprivatekey)
+        sftp_client = self.ssh_client.open_sftp()
+        failed = []
+        try:
+            for localpath in files:
+                if os.path.isfile(localpath):
+                    self.logger.info('Uploading ' + localpath
+                                     + ' from the host to ' + target
+                                     + ' on the machine')
+                    remotepath = os.path.join(target,
+                                              os.path.basename(localpath))
+                    sftp_client.put(localpath, remotepath)
+                else:
+                    failed.append(localpath)
+        finally:
+            sftp_client.close()
+            if len(failed) > 0:
+                err = UTAHProvisioningException('Files do not exist: '
+                                                + ' '.join(failed))
+                err.files = failed
+                raise err
+
+    def downloadfiles(self, files, target=os.path.normpath('/tmp/')):
+        """
+        Copy a file or list of files from the machine to a target directory on
+        the local system.
+        """
+        # TODO: check for directories and recurse into them
+        if isinstance(files, basestring):
+            files = [files]
+
+        self.activecheck()
+        self.ssh_client.connect(self.name,
+                                username=config.user,
+                                key_filename=config.sshprivatekey)
+        sftp_client = self.ssh_client.open_sftp()
+        if os.path.isdir(target):
+            get_localpath = lambda remotepath: \
+                os.path.join(target, os.path.basename(remotepath))
+        else:
+            get_localpath = lambda remotepath: target
+
+        try:
+            for remotepath in files:
+                localpath = get_localpath(remotepath)
+                self.logger.info('Downloading ' + remotepath
+                                 + ' from the machine to ' + target
+                                 + ' on the host')
+                sftp_client.get(remotepath, localpath)
+        finally:
+            sftp_client.close()
+
+    def downloadfilesrecursive(self, files, target=os.path.normpath('/tmp/')):
+        """
+        Recursively copy all files in files to the target directory target.
+        """
+        self.activecheck()
+        self.ssh_client.connect(self.name,
+                                username=config.user,
+                                key_filename=config.sshprivatekey)
+        sftp_client = self.ssh_client.open_sftp()
+        myfiles = []
+
+        if isinstance(files, basestring):
+            files = [files]
+
+        for myfile in files:
+            newtarget = os.path.join(target, os.path.basename(myfile))
+            if S_ISDIR(sftp_client.stat(myfile).st_mode):
+                self.logger.debug(myfile + ' is a directory, recursing')
+                if not os.path.isdir(newtarget):
+                    self.logger.debug('Attempting to create ' + newtarget)
+                    os.makedirs(newtarget)
+                myfiles = [os.path.join(myfile, x)
+                           for x in sftp_client.listdir(myfile)]
+                self.downloadfilesrecursive(myfiles, newtarget)
+            else:
+                self.downloadfiles(myfile, newtarget)
+
+    def destroy(self, *args, **kw):
+        """
+        Clean up known hosts for machine.
+        """
+        # TODO: evaluate value of known_hosts with paramiko
+        self.logger.info('Removing machine addresses '
+                         'from ssh known_hosts file')
+        addresses = [self.name]
+        try:
+            addresses.append(socket.gethostbyname(self.name))
+        except socket.gaierror as err:
+            if err.errno in [-2, -5]:
+                self.logger.debug(self.name
+                                  + ' is not resolvable, '
+                                  + 'so not removing from known_hosts')
+            else:
+                raise err
+
+        old_host_keys = self.ssh_client.get_host_keys()
+        new_host_keys = paramiko.HostKeys()
+        addresses = set(addresses)
+        for address, key in old_host_keys.iteritems():
+            # Skip keys so that they don't get added
+            # into the new keys (i.e. they're removed)
+            if address in addresses:
+                continue
+            new_host_keys[address] = key
+        new_host_keys.save(config.sshknownhosts)
+        self.ssh_client.close()
+
+        super(SSHMixin, self).destroy(*args, **kw)
+
+    def sshcheck(self, timeout=config.checktimeout):
+        """
+        Sleep for a while and check if the machine is available via ssh.
+        Return a retryable exception if it is not.
+        Intended for use with retry.
+        """
+        self.logger.info('Sleeping {timeout} seconds'
+                         .format(timeout=timeout))
+        time.sleep(timeout)
+        self.logger.info('Checking for ssh availability')
+        try:
+            self.ssh_client.connect(self.name,
+                                    username=config.user,
+                                    key_filename=config.sshprivatekey)
+        except socket.error as err:
+            raise UTAHProvisioningException(str(err), retry=True)
+
+    def sshpoll(self, timeout=None,
+                checktimeout=config.checktimeout, logmethod=None):
+        """
+        Run sshcheck over and over until timeout expires.
+        """
+        if timeout is None:
+            timeout = self.boottimeout
+        if logmethod is None:
+            logmethod = self.logger.debug
+        utah.timeout.timeout(timeout, retry, self.sshcheck, checktimeout,
+                             logmethod=logmethod)
+
+    def activecheck(self):
+        """
+        Start the machine if needed, and check for SSH login.
+        """
+        self.logger.debug('Checking if machine is active')
+        self.provisioncheck()
+        if not self.active:
+            self._start()
+            self.sshcheck()
 
=== removed file 'utah/provisioning/vm/libvirtvm.py'
--- utah/provisioning/vm/libvirtvm.py	2012-12-03 14:02:18 +0000
+++ utah/provisioning/vm/libvirtvm.py	1970-01-01 00:00:00 +0000
@@ -1,777 +0,0 @@
-# Ubuntu Testing Automation Harness
-# Copyright 2012 Canonical Ltd.
-
-# This program is free software: you can redistribute it and/or modify it
-# under the terms of the GNU General Public License version 3, as published
-# by the Free Software Foundation.
-
-# This program is distributed in the hope that it will be useful, but
-# WITHOUT ANY WARRANTY; without even the implied warranties of
-# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
-# PURPOSE.  See the GNU General Public License for more details.
-
-# You should have received a copy of the GNU General Public License along
-# with this program.  If not, see <http://www.gnu.org/licenses/>.
-
-"""
-Provide classes to provision libvirt-based virtual machines.
-"""
-
-import os
-import random
-import shutil
-import string
-import subprocess
-import tempfile
-
-from xml.etree import ElementTree
-
-import apt.cache
-import libvirt
-
-from utah.provisioning.provisioning import (
-    SSHMixin,
-    CustomInstallMixin,
-)
-from utah.provisioning.vm.vm import VM
-from utah.provisioning.vm.exceptions import UTAHVMProvisioningException
-from utah import config
-from utah.process import ProcessChecker
-from utah.timeout import UTAHTimeout
-
-
-class LibvirtVM(VM):
-    """
-    Provide a class to utilize VMs using libvirt.
-
-    Capable of utilizing existing VMs.
-    Creation currently handled by sublcasses.
-    """
-    def __init__(self, *args, **kw):
-        super(LibvirtVM, self).__init__(*args, **kw)
-        libvirt.registerErrorHandler(self.libvirterrorhandler, None)
-        self.lv = libvirt.open(config.qemupath)
-        if self.lv is None:
-            raise UTAHVMProvisioningException('Cannot connect to libvirt')
-        self.logger.debug('LibvirtVM init finished')
-
-    def _load(self):
-        """
-        Load an existing VM.
-        """
-        self.logger.info('Loading VM')
-        self.vm = self.lv.lookupByName(self.name)
-        self.logger.info('VM loaded')
-        return True
-
-    def _provision(self):
-        """
-        Make an existing VM available using libvirt to look up the VM by name.
-        """
-        self.logger.info('Provisioning VM')
-        if self.new:
-            self.logger.debug('New VM requested')
-            try:
-                self._load()
-                self.logger.error('VM already exists')
-                raise UTAHVMProvisioningException('Request new VM, but '
-                                                  + self.name
-                                                  + ' already exists')
-            except libvirt.libvirtError as err:
-                if err.get_error_code() == 42:
-                    self._create()
-                else:
-                    raise err
-
-        try:
-            self._load()
-        except libvirt.libvirtError as err:
-            if err.get_error_code() == 42:
-                self.logger.debug('Lookup failed')
-                try:
-                    self._create()
-                    self._load()
-                except UTAHVMProvisioningException as error:
-                    self.logger.error('VM lookup failed')
-                    raise UTAHVMProvisioningException('Cannot find VM named '
-                                                      + self.name +
-                                                      ' and ' + str(error))
-            else:
-                raise err
-        self.provisioned = True
-        self.logger.info('VM provisioned')
-
-    def activecheck(self):
-        """
-        Verify the machine is provisioned, then start it if it is not started.
-        """
-        self.logger.debug('Checking if VM is active')
-        self.provisioncheck()
-        if self.vm is not None:
-            if self.vm.isActive() == 0:
-                self._start()
-            else:
-                self.active = True
-        else:
-            raise UTAHVMProvisioningException('Failed to provision VM')
-
-    def _start(self):
-        """
-        Start the VM.
-        """
-        self.logger.info('Starting VM')
-        if self.vm is not None:
-            if self.vm.isActive() == 0:
-                self.vm.create()
-        else:
-            raise UTAHVMProvisioningException('Failed to provision VM')
-        self.active = True
-
-    def stop(self, force=False):
-        """
-        Stop the machine.
-        Setting force to true will do a hard shutdown instead of a graceful
-        one.
-        """
-        self.logger.info('Stopping VM')
-        if self.vm is not None:
-            if self.vm.isActive() == 0:
-                self.logger.info('VM is already stopped')
-            else:
-                if force:
-                    self.logger.info('Forced shutdown requested')
-                    self.vm.destroy()
-                else:
-                    self.vm.shutdown()
-        else:
-            self.logger.info('VM not yet created')
-        self.active = False
-
-    def libvirterrorhandler(self, _context, err):
-        """
-        Log libvirt errors instead of sending them directly to the console.
-        """
-        errorcode = err.get_error_code()
-        if errorcode in [9, 42]:
-            # We see these as part of normal operations,
-            # so we send them to debug
-            # 9 is trying to create a VM that already exists
-            # 42 is trying to load a VM that doesn't exist
-            logmethod = self.logger.debug
-        else:
-            logmethod = self.logger.error
-        logmethod('libvirt error: ' + err['message'])
-        logmethod('libvirt error number is: ' + str(errorcode))
-
-
-class VMToolsVM(SSHMixin, LibvirtVM):
-    """
-    Provide a class to provision a VM using the ubuntu-qa-tools vm-tools.
-    """
-    def __init__(self, machineid=None, prefix='utah', *args, **kw):
-        if not apt.cache.Cache()['vm-tools'].is_installed:
-            raise UTAHVMProvisioningException(
-                'vm-tools is not installed.  '
-                'Try: sudo apt-get install vm-tools')
-        super(VMToolsVM, self).__init__(*args, machineid=machineid, name=None,
-                                        **kw)
-        self.logger.debug('VMToolsVM init finished')
-
-    def _start(self):
-        """
-        Start the VM using vm-start, which will wait until SSH is up.
-        """
-        self.logger.info('Starting the vm using vm-start')
-        args = ['vm-start', '-v', '-w', self.name]
-        self.active = (self._runargs(args) == 0)
-        return self.active
-
-    def activecheck(self):
-        """
-        Verify the machine is provisioned, then start it if it is not started.
-        Use vm-wait to make sure it's up.
-        """
-        self.provisioncheck()
-        self.logger.debug('Checking if VMToolsVM is active')
-        self._start()
-        self.logger.info('Using vm-wait to ensure VM is active')
-        if (self._runargs(['vm-wait', self.name, '300']) != 0):
-            self.active = False
-            raise UTAHVMProvisioningException('Timed out waiting for VM '
-                                              'to be reachable')
-        else:
-            self.active = True
-
-    def _getcreationargs(self):
-        """
-        Return the vm-new syntax used to create the VM.
-        Can be used for debugging or instructional purposes.
-        """
-        args = ['vm-new', '-f', '-r', '-v', '-b', config.bridge,
-                '-t', self.installtype, self.series, self.arch, self.prefix]
-        self.logger.debug('VM Creation args: ' + str(args))
-        return args
-
-    def _create(self):
-        """
-        Run the command generated by getcreationargs
-        and either return a good status or raise an exception.
-        """
-        self.logger.info('Creating vm using vm-new')
-        self.logger.info('This may take up to two hours, '
-                         'excluding download times, and may go over an hour '
-                         'without output during virt-install')
-        args = self._getcreationargs()
-        percent = 0
-        p = subprocess.Popen(args,
-                             stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
-        while p.poll() is None:
-            line = p.stdout.readline().strip()
-            self.logger.debug(line)
-            if 'Fetching release iso' in line:
-                self.logger.info('Downloading ISO')
-            if len(line.split()) >= 3 and '%' in line.split()[-3]:
-                dlpercent = int(line.split()[-3].strip('%'))
-                if dlpercent >= percent:
-                    self.logger.info('ISO ' + str(dlpercent) + '% downloaded, '
-                                     + line.split(' ')[-1] + ' remaining')
-                    percent += self.dlpercentincrement
-            if ' saved ' in line:
-                self.logger.info('ISO download complete')
-            if 'Creating preseeded iso' in line:
-                self.logger.info('Preseeding ISO')
-            if 'virt-install' in line:
-                self.logger.info('Installing system on VM '
-                                 '(may take over an hour)')
-                self.logger.info('You can watch the progress with:')
-                self.logger.info("\tvm-view " + self.name)
-                self.logger.info('Take care not to interrupt the install')
-            if 'Verifying lsb_release' in line:
-                self.logger.info('Waiting for post-install '
-                                 'and verifying system '
-                                 '(may take over a half hour)')
-            if "No domains available for virt type 'hvm'" in line:
-                self.logger.error('No hardware virtual machine '
-                                  'support available')
-                self.logger.info('Please ensure the following:')
-                self.logger.info("\tYour processor supports hardware "
-                                 "virtualization extensions")
-                self.logger.info("\tHardware virtualization "
-                                 "is not disabled in the BIOS")
-                self.logger.info("\tkvm is installed")
-                self.logger.info("\tThe kvm and processor-specific "
-                                 "kvm kernel modules are installed and loaded")
-                self.logger.error('Software virtual machine support '
-                                  'is implemented in the CustomVM class')
-                raise UTAHVMProvisioningException(
-                    'No hardware virtual machine support available')
-            if ('Could not find' in line
-                    and ".uqt-vm-tools.conf' configuration file!" in line):
-                self.logger.error('No .uqt-vm-tools.conf configuration file '
-                                  'found in home directory')
-                self.logger.info('If you are running as the utah user, '
-                                 'you can use the packaged config file:')
-                self.logger.info("\tln -s /etc/utah/uqt-vm-tools.conf "
-                                 "~utah/.uqt-vm-tools.conf")
-                self.logger.info('As a different user, '
-                                 'run vm-new with no arguments, '
-                                 'and answer y when prompted '
-                                 'to create an initial config file')
-                self.logger.info('We recommend adding python-yaml '
-                                 'to the vm_extra_packages line '
-                                 'of this file, i.e.:')
-                self.logger.info("\tsed "
-                                 "'s/^vm_extra_packages=\""
-                                 "/vm_extra_packages=\"python-yaml /' "
-                                 "-i ~/.uqt_vm_tools.conf")
-                raise UTAHVMProvisioningException(
-                    'No vm-tools config file available; '
-                    'more info available in '
-                    + self.filehandler.baseFilename)
-        if p.returncode == 0:
-            return True
-        else:
-            raise UTAHVMProvisioningException(
-                'Failed to create VM: vm-new exit status: '
-                + str(p.returncode))
-
-    def destroy(self):
-        """
-        Use vm-remove to destroy the vm.
-        """
-        self.logger.info('Destroying vm using vm-remove')
-        self.provisioned = False
-        args = ['vm-remove', '-f', self.name]
-        return (self._runargs(args) == 0)
-
-    def _getcommandargs(self, command, quiet=None, root=False, timeout=300):
-        """
-        Get the arguments to send to vm-cmd.
-        Used by multiple other functions.
-        """
-        if quiet is None:
-            quiet = not self.debug
-        args = ['vm-cmd', '-f']
-        if timeout is not None:
-            args.extend(['-s', '-t', str(timeout)])
-        if quiet:
-            args.append('-q')
-        if root:
-            args.append('-r')
-        args.append(self.name)
-        if isinstance(command, basestring):
-            args.append(command)
-        else:
-            args.extend(command)
-        self.logger.debug('vm-cmd command args: ' + str(args))
-        return args
-
-    def run(self, command, quiet=None, root=False, timeout=300):
-        """
-        Run a command using vm-cmd.
-        Timeout defaults to 300 seconds, but can be set to None to not send a
-        timeout argument.
-        """
-        if quiet is None:
-            quiet = not self.debug
-        self.activecheck()
-        args = self._getcommandargs(command=command, quiet=quiet,
-                                    root=root, timeout=timeout)
-        self.logger.info('Running command on VM: ' + ' '.join(args))
-        p = subprocess.Popen(args,
-                             stdout=subprocess.PIPE, stderr=subprocess.PIPE)
-        while p.poll() is None:
-            pass
-        return p.returncode, p.communicate()[0], p.communicate()[1]
-
-    def uploadfiles(self, files, target=os.path.normpath('/tmp/')):
-        """
-        Copy a file or list of files to a target directory on the machine.
-        """
-        if isinstance(files, basestring):
-            files = [files]
-        if not target.endswith('/'):
-            target += '/'
-        self.activecheck()
-        success = True
-        for myfile in files:
-            self.logger.info('Uploading ' + myfile
-                             + ' from the host to ' + target + ' on the VM')
-            returncode = self._runargs(['vm-scp', '-f', '-p',
-                                        self.name, myfile, target])
-            if returncode != 0:
-                success = False
-        return success
-
-
-class CustomVM(CustomInstallMixin, SSHMixin, LibvirtVM):
-    """
-    Install a VM from an image using libvirt direct kernel booting.
-    """
-    def __init__(self, diskbus=None, disksizes=None, emulator=None,
-                 machineid=None, macs=None, name=None, prefix='utah', *args,
-                 **kw):
-        # Make sure that no other virtualization solutions are running
-        # TODO: see if this is needed for qemu or just kvm
-        process_checker = ProcessChecker()
-        for cmdline, app in [('/usr/lib/virtualbox/VirtualBox', 'VirtualBox'),
-                             ('/usr/lib/vmware/bin', 'VMware')]:
-            if process_checker.check_cmdline(cmdline):
-                message = process_checker.get_error_message(app)
-                raise UTAHVMProvisioningException(message)
-
-        if diskbus is None:
-            self.diskbus = config.diskbus
-        else:
-            self.diskbus = diskbus
-        if disksizes is None:
-            disksizes = config.disksizes
-        if disksizes is None:
-            self.disksizes = [8]
-        else:
-            self.disksizes = disksizes
-        self.disks = []
-        if name is None:
-            autoname = True
-            name = '-'.join([str(prefix), str(machineid)])
-        else:
-            autoname = False
-        super(CustomVM, self).__init__(machineid=machineid, name=name, *args,
-                                       **kw)
-        # TODO: do a better job of separating installation
-        # into _create rather than __init__
-        if self.image is None:
-            raise UTAHVMProvisioningException('Image file required '
-                                              'for custom VM installation')
-        self._custominit()
-        if autoname:
-            self._namesetup()
-            self._loggerunsetup()
-            self._loggersetup()
-            self._dirsetup()
-        if emulator is None:
-            emulator = config.emulator
-        if emulator is None:
-            if self._supportsdomaintype('kvm'):
-                self.logger.info('Setting type to kvm '
-                                 'since it is present in libvirt capabilities')
-                self.domaintype = 'kvm'
-            elif self._supportsdomaintype('qemu'):
-                self.logger.info('Setting type to qemu '
-                                 'since it is present in libvirt capabilities')
-                self.domaintype = 'qemu'
-            else:
-                raise UTAHVMProvisioningException(
-                    'kvm and qemu not supported in libvirt capabilities; '
-                    'please make sure qemu and/or kvm are installed '
-                    'and libvirt is configured correctly')
-        else:
-            self.domaintype = emulator
-        if self.domaintype == 'qemu':
-            self.logger.debug('Raising boot timeout for qemu domain')
-            self.boottimeout *= 4
-        if macs is None:
-            macs = []
-        self.macs = macs
-        self.dircheck()
-        self.logger.debug('CustomVM init finished')
-
-    def _createdisks(self, disksizes=None):
-        """
-        Create disk files if needed and build a list of them.
-        """
-        self.logger.info('Creating disks')
-        if disksizes is None:
-            disksizes = self.disksizes
-        for index, size in enumerate(disksizes):
-            disksize = '{}G'.format(size)
-            basename = 'disk{}.qcow2'.format(index)
-            diskfile = os.path.join(self.directory, basename)
-            if not os.path.isfile(diskfile):
-                cmd = ['qemu-img', 'create', '-f', 'qcow2', diskfile, disksize]
-                self.logger.debug('Creating ' + disksize + ' disk using:')
-                self.logger.debug(' '.join(cmd))
-                if self._runargs(cmd) != 0:
-                    raise UTAHVMProvisioningException(
-                        'Could not create disk image at ' + diskfile)
-            disk = {'bus': self.diskbus,
-                    'file': diskfile,
-                    'size': disksize,
-                    'type': 'qcow2'}
-            self.disks.append(disk)
-            self.logger.debug('Adding disk to list')
-
-    def _supportsdomaintype(self, domaintype):
-        """
-        Check emulator support in libvirt capabilities.
-        """
-        capabilities = ElementTree.fromstring(self.lv.getCapabilities())
-        for guest in capabilities.iterfind('guest'):
-            for arch in guest.iterfind('arch'):
-                for domain in arch.iterfind('domain'):
-                    if domaintype in domain.get('type'):
-                        return True
-        return False
-
-    def _installxml(self, cmdline=None, image=None, initrd=None,
-                    kernel=None, tmpdir=None, xml=None):
-        """
-        Return the XML tree to be passed to libvirt for VM installation.
-        """
-        self.logger.info('Creating installation XML')
-        if cmdline is None:
-            cmdline = self.cmdline
-        if image is None:
-            image = self.image.image
-        if initrd is None:
-            initrd = self.initrd
-        if kernel is None:
-            kernel = self.kernel
-        if xml is None:
-            xml = self.xml
-        if tmpdir is None:
-            tmpdir = self.tmpdir
-        xmlt = ElementTree.ElementTree(file=xml)
-        if self.rewrite in ['all', 'minimal']:
-            self.logger.debug('Setting VM to shutdown on reboot')
-            xmlt.find('on_reboot').text = 'destroy'
-            if self.rewrite == 'all':
-                self._installxml_rewrite_all(cmdline, image, initrd, kernel,
-                                             xmlt)
-        else:
-            self.logger.info('Not rewriting XML because rewrite is ' +
-                             self.rewrite)
-        if self.debug:
-            xmlt.write(os.path.join(tmpdir, 'install.xml'))
-        self.logger.info('Installation XML ready')
-        return xmlt
-
-    def _installxml_rewrite_all(self, cmdline_txt, image, initrd_txt,
-                                kernel_txt, xmlt):
-        """
-        Rewrite the whole configuration file for the VM
-        """
-        self.logger.debug('Rewriting basic info')
-        xmlt.find('name').text = self.name
-        xmlt.find('uuid').text = self.uuid
-        self.logger.debug('Setting type to qemu in case no '
-                          'hardware virtualization present')
-        xmlt.getroot().set('type', self.domaintype)
-        ose = xmlt.find('os')
-        if self.arch == ('i386'):
-            ose.find('type').set('arch', 'i686')
-        elif self.arch == ('amd64'):
-            ose.find('type').set('arch', 'x86_64')
-        else:
-            ose.find('type').set('arch', self.arch)
-        self.logger.debug('Setting up boot info')
-        for kernele in list(ose.iterfind('kernel')):
-            ose.remove(kernele)
-        kernele = ElementTree.Element('kernel')
-        kernele.text = kernel_txt
-        ose.append(kernele)
-        for initrde in list(ose.iterfind('initrd')):
-            ose.remove(initrde)
-        initrde = ElementTree.Element('initrd')
-        initrde.text = initrd_txt
-        ose.append(initrde)
-        for cmdlinee in list(ose.iterfind('cmdline')):
-            ose.remove(cmdlinee)
-        cmdlinee = ElementTree.Element('cmdline')
-        cmdlinee.text = cmdline_txt
-        ose.append(cmdlinee)
-        self.logger.debug('Setting up devices')
-        devices = xmlt.find('devices')
-        self.logger.debug('Setting up disks')
-        for disk in list(devices.iterfind('disk')):
-            if disk.get('device') == 'disk':
-                devices.remove(disk)
-                self.logger.debug('Removed existing disk')
-            #TODO: Add a cdrom if none exists
-            if disk.get('device') == 'cdrom':
-                if disk.find('source') is not None:
-                    disk.find('source').set('file', image)
-                    self.logger.debug('Rewrote existing CD-ROM')
-                else:
-                    source = ElementTree.Element('source')
-                    source.set('file', image)
-                    disk.append(source)
-                    self.logger.debug('Added source to existing '
-                                      'CD-ROM')
-        for disk in self.disks:
-            diske = ElementTree.Element('disk')
-            diske.set('type', 'file')
-            diske.set('device', 'disk')
-            driver = ElementTree.Element('driver')
-            driver.set('name', 'qemu')
-            driver.set('type', disk['type'])
-            diske.append(driver)
-            source = ElementTree.Element('source')
-            source.set('file', disk['file'])
-            diske.append(source)
-            target = ElementTree.Element('target')
-            dev = "vd%s" % (string.ascii_lowercase[self.disks.index(disk)])
-            target.set('dev', dev)
-            target.set('bus', disk['bus'])
-            diske.append(target)
-            devices.append(diske)
-            self.logger.debug('Added ' + str(disk['size']) + ' disk')
-        macs = list(self.macs)
-        for interface in devices.iterfind('interface'):
-            if interface.get('type') in ['network', 'bridge']:
-                if len(macs) > 0:
-                    mac = macs.pop(0)
-                    interface.find('mac').set('address', mac)
-                    self.logger.debug('Rewrote interface '
-                                      'to use specified mac address ' + mac)
-                else:
-                    mac = random_mac_address()
-                    interface.find('mac').set('address', mac)
-                    self.macs.append(mac)
-                    self.logger.debug('Rewrote interface '
-                                      'to use random mac address ' + mac)
-            if interface.get('type') == 'bridge':
-                interface.find('source').set('bridge', config.bridge)
-        serial = ElementTree.Element('serial')
-        serial.set('type', 'file')
-        source = ElementTree.Element('source')
-        log_filename = os.path.join(config.logpath, self.name + '.syslog.log')
-        source.set('path', log_filename)
-        serial.append(source)
-        target = ElementTree.Element('target')
-        target.set('port', '0')
-        serial.append(target)
-        devices.append(serial)
-
-    def _installvm(self, lv=None, tmpdir=None, xml=None):
-        """
-        Install a VM, then undefine it in libvirt.
-        The final installation will recreate the VM using the existing disks.
-        """
-        self.logger.info('Creating VM')
-        if lv is None:
-            lv = self.lv
-        if xml is None:
-            xml = self.xml
-        if tmpdir is None:
-            tmpdir = self.tmpdir
-        vm = lv.defineXML(ElementTree.tostring(xml.getroot()))
-        os.chmod(tmpdir, 0755)
-        vm.create()
-        self.logger.info('Installing system on VM (may take over an hour)')
-        self.logger.info('You can watch the progress with virt-viewer')
-        log_filename = os.path.join(config.logpath, self.name + '.syslog.log')
-        self.logger.info('Logs will be written to ' + log_filename)
-
-        while vm.isActive() is not 0:
-            pass
-
-        vm.undefine()
-        self.logger.info('Installation complete')
-
-    def _finalxml(self, tmpdir=None, xml=None):
-        """
-        Create the XML to be used for the post-installation VM.
-        This may be a transformation of the installation XML.
-        """
-        self.logger.info('Creating final VM XML')
-        if xml is None:
-            xml = ElementTree.ElementTree(file=self.xml)
-        if tmpdir is None:
-            tmpdir = self.tmpdir
-        if self.rewrite in ['all', 'minimal']:
-            self.logger.debug('Setting VM to reboot normally on reboot')
-            xml.find('on_reboot').text = 'restart'
-            if self.rewrite == 'all':
-                self.logger.debug('Removing VM install parameters')
-                ose = xml.find('os')
-                for kernel in ose.iterfind('kernel'):
-                    ose.remove(kernel)
-                for initrd in ose.iterfind('initrd'):
-                    ose.remove(initrd)
-                for cmdline in ose.iterfind('cmdline'):
-                    ose.remove(cmdline)
-                devices = xml.find('devices')
-                devices.remove(devices.find('serial'))
-                for disk in list(devices.iterfind('disk')):
-                    if disk.get('device') == 'cdrom':
-                        disk.remove(disk.find('source'))
-        else:
-            self.logger.info('Not rewriting XML because rewrite is ' +
-                             self.rewrite)
-        if self.debug:
-            xml.write(os.path.join(tmpdir, 'final.xml'))
-        return xml
-
-    def _tmpimage(self, image=None, tmpdir=None):
-        """
-        Create a temporary copy of the image so libvirt will lock that copy.
-        This allows other simultaneous processes to update the cached image.
-        """
-        if image is None:
-            image = self.image.image
-        if tmpdir is None:
-            tmpdir = self.tmpdir
-        self.logger.info('Making temp copy of install image')
-        tmpimage = os.path.join(tmpdir, os.path.basename(image))
-        self.logger.debug('Copying ' + image + ' to ' + tmpimage)
-        shutil.copyfile(image, tmpimage)
-        return tmpimage
-
-    def _create(self):
-        """
-        Create the VM, install the system, and prepare it to boot.
-        This primarily calls functions from CustomInstallMixin and CustomVM.
-        """
-        self.logger.info('Creating custom virtual machine')
-
-        tmpdir = tempfile.mkdtemp(prefix='/tmp/' + self.name + '_')
-        self.logger.debug('Working dir: ' + tmpdir)
-        os.chdir(tmpdir)
-
-        kernel = self._preparekernel(kernel=self.kernel, tmpdir=tmpdir)
-
-        initrd = self._prepareinitrd(initrd=self.initrd, tmpdir=tmpdir)
-
-        self._unpackinitrd(initrd=initrd, tmpdir=tmpdir)
-
-        self._setuplatecommand(tmpdir=tmpdir)
-
-        self._setuppreseed(tmpdir=tmpdir)
-
-        if self.rewrite == 'all':
-            self._setuplogging(tmpdir=tmpdir)
-        else:
-            self.logger.debug('Skipping logging setup because rewrite is' +
-                              self.rewrite)
-
-        initrd = self._repackinitrd(tmpdir=tmpdir)
-
-        self._createdisks()
-
-        image = self._tmpimage(image=self.image.image, tmpdir=tmpdir)
-
-        xml = self._installxml(cmdline=self.cmdline, image=image,
-                               initrd=initrd, kernel=kernel,
-                               tmpdir=tmpdir, xml=self.xml)
-
-        self._installvm(lv=self.lv, tmpdir=tmpdir, xml=xml)
-
-        xml = self._finalxml(tmpdir=tmpdir, xml=xml)
-
-        self.logger.info('Setting up final VM')
-        self.vm = self.lv.defineXML(ElementTree.tostring(xml.getroot()))
-
-        if self.debug:
-            self.logger.info('Leaving temp directory '
-                             'because debug is enabled: ' + tmpdir)
-        else:
-            self.logger.info('Cleaning up temp directory')
-            shutil.rmtree(tmpdir)
-        return True
-
-    def _start(self):
-        """
-        Start the VM.
-        """
-        self.logger.info('Starting CustomVM')
-        if self.vm is not None:
-            if self.vm.isActive() == 0:
-                self.vm.create()
-        else:
-            raise UTAHVMProvisioningException('Failed to provision VM')
-        self.logger.info('Waiting ' + str(self.boottimeout) +
-                         ' seconds to allow machine to boot')
-        try:
-            self.pingpoll(timeout=self.boottimeout)
-        except UTAHTimeout:
-            # Ignore timeout for ping, since depending on the network
-            # configuration ssh might still work despite of the ping failure.
-            self.logger.warning('Network connectivity (ping) failure')
-        self.sshpoll(timeout=self.boottimeout)
-        self.active = True
-
-    def destroy(self, *args, **kw):
-        """
-        Remove the machine from libvirt and remove all the disk files.
-        """
-        # TODO: make this use standard cleanup
-        super(CustomVM, self).destroy(*args, **kw)
-        self.stop(force=True)
-        if self.vm is not None:
-            self.vm.undefine()
-        else:
-            self.logger.info('VM not created')
-        for disk in self.disks:
-            os.unlink(disk['file'])
-        shutil.rmtree(self.directory)
-
-
-# See http://kennethreitz.com/blog/generate-a-random-mac-address-in-python/
-def random_mac_address():
-    """Returns a completely random Mac Address"""
-    mac = [0x52, 0x54, 0x00,
-           random.randint(0x00, 0xff),
-           random.randint(0x00, 0xff),
-           random.randint(0x00, 0xff)]
-    return ':'.join(map(lambda x: "%02x" % x, mac))
 
=== modified file 'utah/provisioning/vm/vm.py'
--- utah/provisioning/vm/vm.py	2012-12-03 14:02:18 +0000
+++ utah/provisioning/vm/vm.py	2012-12-13 00:38:21 +0000
@@ -14,11 +14,26 @@
 # with this program.  If not, see <http://www.gnu.org/licenses/>.
 
 """
-Consolidate functions specific to virtual machine provisioning,
-but not specific to any type of virtual machine.
+Consolidate functions for virtual machine provisioning.
+Non-libvirt VMs can be supported elsewhere if support for them is needed.
 """
 
-from utah.provisioning.provisioning import Machine
+import libvirt
+import os
+import random
+import shutil
+import string
+import tempfile
+
+from xml.etree import ElementTree
+
+from utah import config
+from utah.process import ProcessChecker
+from utah.provisioning.inventory.sqlite import SQLiteInventory
+from utah.provisioning.provisioning import CustomInstallMixin, Machine
+from utah.provisioning.ssh import SSHMixin
+from utah.provisioning.vm.exceptions import UTAHVMProvisioningException
+from utah.timeout import UTAHTimeout
 
 
 class VM(Machine):
@@ -29,3 +44,590 @@
         super(VM, self).__init__(*args, **kw)
         self.vm = None
         self.logger.debug('VM init finished')
+
+
+class LibvirtVM(VM):
+    """
+    Provide a class to utilize VMs using libvirt.
+
+    Capable of utilizing existing VMs.
+    Creation currently handled by sublcasses.
+    """
+    def __init__(self, *args, **kw):
+        super(LibvirtVM, self).__init__(*args, **kw)
+        libvirt.registerErrorHandler(self.libvirterrorhandler, None)
+        self.lv = libvirt.open(config.qemupath)
+        if self.lv is None:
+            raise UTAHVMProvisioningException('Cannot connect to libvirt')
+        self.logger.debug('LibvirtVM init finished')
+
+    def _load(self):
+        """
+        Load an existing VM.
+        """
+        self.logger.info('Loading VM')
+        self.vm = self.lv.lookupByName(self.name)
+        self.logger.info('VM loaded')
+        return True
+
+    def _provision(self):
+        """
+        Make an existing VM available using libvirt to look up the VM by name.
+        """
+        self.logger.info('Provisioning VM')
+        if self.new:
+            self.logger.debug('New VM requested')
+            try:
+                self._load()
+                self.logger.error('VM already exists')
+                raise UTAHVMProvisioningException('Request new VM, but '
+                                                  + self.name
+                                                  + ' already exists')
+            except libvirt.libvirtError as err:
+                if err.get_error_code() == 42:
+                    self._create()
+                else:
+                    raise err
+
+        try:
+            self._load()
+        except libvirt.libvirtError as err:
+            if err.get_error_code() == 42:
+                self.logger.debug('Lookup failed')
+                try:
+                    self._create()
+                    self._load()
+                except UTAHVMProvisioningException as error:
+                    self.logger.error('VM lookup failed')
+                    raise UTAHVMProvisioningException('Cannot find VM named '
+                                                      + self.name +
+                                                      ' and ' + str(error))
+            else:
+                raise err
+        self.provisioned = True
+        self.logger.info('VM provisioned')
+
+    def activecheck(self):
+        """
+        Verify the machine is provisioned, then start it if it is not started.
+        """
+        self.logger.debug('Checking if VM is active')
+        self.provisioncheck()
+        if self.vm is not None:
+            if self.vm.isActive() == 0:
+                self._start()
+            else:
+                self.active = True
+        else:
+            raise UTAHVMProvisioningException('Failed to provision VM')
+
+    def _start(self):
+        """
+        Start the VM.
+        """
+        self.logger.info('Starting VM')
+        if self.vm is not None:
+            if self.vm.isActive() == 0:
+                self.vm.create()
+        else:
+            raise UTAHVMProvisioningException('Failed to provision VM')
+        self.active = True
+
+    def stop(self, force=False):
+        """
+        Stop the machine.
+        Setting force to true will do a hard shutdown instead of a graceful
+        one.
+        """
+        self.logger.info('Stopping VM')
+        if self.vm is not None:
+            if self.vm.isActive() == 0:
+                self.logger.info('VM is already stopped')
+            else:
+                if force:
+                    self.logger.info('Forced shutdown requested')
+                    self.vm.destroy()
+                else:
+                    self.vm.shutdown()
+        else:
+            self.logger.info('VM not yet created')
+        self.active = False
+
+    def libvirterrorhandler(self, _context, err):
+        """
+        Log libvirt errors instead of sending them directly to the console.
+        """
+        errorcode = err.get_error_code()
+        if errorcode in [9, 42]:
+            # We see these as part of normal operations,
+            # so we send them to debug
+            # 9 is trying to create a VM that already exists
+            # 42 is trying to load a VM that doesn't exist
+            logmethod = self.logger.debug
+        else:
+            logmethod = self.logger.error
+        logmethod('libvirt error: ' + err['message'])
+        logmethod('libvirt error number is: ' + str(errorcode))
+
+
+class CustomVM(CustomInstallMixin, SSHMixin, LibvirtVM):
+    """
+    Install a VM from an image using libvirt direct kernel booting.
+    """
+    def __init__(self, diskbus=None, disksizes=None, emulator=None,
+                 machineid=None, macs=None, name=None, prefix='utah', *args,
+                 **kw):
+        # Make sure that no other virtualization solutions are running
+        # TODO: see if this is needed for qemu or just kvm
+        process_checker = ProcessChecker()
+        for cmdline, app in [('/usr/lib/virtualbox/VirtualBox', 'VirtualBox'),
+                             ('/usr/lib/vmware/bin', 'VMware')]:
+            if process_checker.check_cmdline(cmdline):
+                message = process_checker.get_error_message(app)
+                raise UTAHVMProvisioningException(message)
+
+        if diskbus is None:
+            self.diskbus = config.diskbus
+        else:
+            self.diskbus = diskbus
+        if disksizes is None:
+            disksizes = config.disksizes
+        if disksizes is None:
+            self.disksizes = [8]
+        else:
+            self.disksizes = disksizes
+        self.disks = []
+        if name is None:
+            autoname = True
+            name = '-'.join([str(prefix), str(machineid)])
+        else:
+            autoname = False
+        super(CustomVM, self).__init__(machineid=machineid, name=name, *args,
+                                       **kw)
+        # TODO: do a better job of separating installation
+        # into _create rather than __init__
+        if self.image is None:
+            raise UTAHVMProvisioningException('Image file required '
+                                              'for custom VM installation')
+        self._custominit()
+        if autoname:
+            self._namesetup()
+            self._loggerunsetup()
+            self._loggersetup()
+            self._dirsetup()
+        if emulator is None:
+            emulator = config.emulator
+        if emulator is None:
+            if self._supportsdomaintype('kvm'):
+                self.logger.info('Setting type to kvm '
+                                 'since it is present in libvirt capabilities')
+                self.domaintype = 'kvm'
+            elif self._supportsdomaintype('qemu'):
+                self.logger.info('Setting type to qemu '
+                                 'since it is present in libvirt capabilities')
+                self.domaintype = 'qemu'
+            else:
+                raise UTAHVMProvisioningException(
+                    'kvm and qemu not supported in libvirt capabilities; '
+                    'please make sure qemu and/or kvm are installed '
+                    'and libvirt is configured correctly')
+        else:
+            self.domaintype = emulator
+        if self.domaintype == 'qemu':
+            self.logger.debug('Raising boot timeout for qemu domain')
+            self.boottimeout *= 4
+        if macs is None:
+            macs = []
+        self.macs = macs
+        self.dircheck()
+        self.logger.debug('CustomVM init finished')
+
+    def _createdisks(self, disksizes=None):
+        """
+        Create disk files if needed and build a list of them.
+        """
+        self.logger.info('Creating disks')
+        if disksizes is None:
+            disksizes = self.disksizes
+        for index, size in enumerate(disksizes):
+            disksize = '{}G'.format(size)
+            basename = 'disk{}.qcow2'.format(index)
+            diskfile = os.path.join(self.directory, basename)
+            if not os.path.isfile(diskfile):
+                cmd = ['qemu-img', 'create', '-f', 'qcow2', diskfile, disksize]
+                self.logger.debug('Creating ' + disksize + ' disk using:')
+                self.logger.debug(' '.join(cmd))
+                if self._runargs(cmd) != 0:
+                    raise UTAHVMProvisioningException(
+                        'Could not create disk image at ' + diskfile)
+            disk = {'bus': self.diskbus,
+                    'file': diskfile,
+                    'size': disksize,
+                    'type': 'qcow2'}
+            self.disks.append(disk)
+            self.logger.debug('Adding disk to list')
+
+    def _supportsdomaintype(self, domaintype):
+        """
+        Check emulator support in libvirt capabilities.
+        """
+        capabilities = ElementTree.fromstring(self.lv.getCapabilities())
+        for guest in capabilities.iterfind('guest'):
+            for arch in guest.iterfind('arch'):
+                for domain in arch.iterfind('domain'):
+                    if domaintype in domain.get('type'):
+                        return True
+        return False
+
+    def _installxml(self, cmdline=None, image=None, initrd=None,
+                    kernel=None, tmpdir=None, xml=None):
+        """
+        Return the XML tree to be passed to libvirt for VM installation.
+        """
+        self.logger.info('Creating installation XML')
+        if cmdline is None:
+            cmdline = self.cmdline
+        if image is None:
+            image = self.image.image
+        if initrd is None:
+            initrd = self.initrd
+        if kernel is None:
+            kernel = self.kernel
+        if xml is None:
+            xml = self.xml
+        if tmpdir is None:
+            tmpdir = self.tmpdir
+        xmlt = ElementTree.ElementTree(file=xml)
+        if self.rewrite in ['all', 'minimal']:
+            self.logger.debug('Setting VM to shutdown on reboot')
+            xmlt.find('on_reboot').text = 'destroy'
+            if self.rewrite == 'all':
+                self._installxml_rewrite_all(cmdline, image, initrd, kernel,
+                                             xmlt)
+        else:
+            self.logger.info('Not rewriting XML because rewrite is ' +
+                             self.rewrite)
+        if self.debug:
+            xmlt.write(os.path.join(tmpdir, 'install.xml'))
+        self.logger.info('Installation XML ready')
+        return xmlt
+
+    def _installxml_rewrite_all(self, cmdline_txt, image, initrd_txt,
+                                kernel_txt, xmlt):
+        """
+        Rewrite the whole configuration file for the VM
+        """
+        self.logger.debug('Rewriting basic info')
+        xmlt.find('name').text = self.name
+        xmlt.find('uuid').text = self.uuid
+        self.logger.debug('Setting type to qemu in case no '
+                          'hardware virtualization present')
+        xmlt.getroot().set('type', self.domaintype)
+        ose = xmlt.find('os')
+        if self.arch == ('i386'):
+            ose.find('type').set('arch', 'i686')
+        elif self.arch == ('amd64'):
+            ose.find('type').set('arch', 'x86_64')
+        else:
+            ose.find('type').set('arch', self.arch)
+        self.logger.debug('Setting up boot info')
+        for kernele in list(ose.iterfind('kernel')):
+            ose.remove(kernele)
+        kernele = ElementTree.Element('kernel')
+        kernele.text = kernel_txt
+        ose.append(kernele)
+        for initrde in list(ose.iterfind('initrd')):
+            ose.remove(initrde)
+        initrde = ElementTree.Element('initrd')
+        initrde.text = initrd_txt
+        ose.append(initrde)
+        for cmdlinee in list(ose.iterfind('cmdline')):
+            ose.remove(cmdlinee)
+        cmdlinee = ElementTree.Element('cmdline')
+        cmdlinee.text = cmdline_txt
+        ose.append(cmdlinee)
+        self.logger.debug('Setting up devices')
+        devices = xmlt.find('devices')
+        self.logger.debug('Setting up disks')
+        for disk in list(devices.iterfind('disk')):
+            if disk.get('device') == 'disk':
+                devices.remove(disk)
+                self.logger.debug('Removed existing disk')
+            #TODO: Add a cdrom if none exists
+            if disk.get('device') == 'cdrom':
+                if disk.find('source') is not None:
+                    disk.find('source').set('file', image)
+                    self.logger.debug('Rewrote existing CD-ROM')
+                else:
+                    source = ElementTree.Element('source')
+                    source.set('file', image)
+                    disk.append(source)
+                    self.logger.debug('Added source to existing '
+                                      'CD-ROM')
+        for disk in self.disks:
+            diske = ElementTree.Element('disk')
+            diske.set('type', 'file')
+            diske.set('device', 'disk')
+            driver = ElementTree.Element('driver')
+            driver.set('name', 'qemu')
+            driver.set('type', disk['type'])
+            diske.append(driver)
+            source = ElementTree.Element('source')
+            source.set('file', disk['file'])
+            diske.append(source)
+            target = ElementTree.Element('target')
+            dev = "vd%s" % (string.ascii_lowercase[self.disks.index(disk)])
+            target.set('dev', dev)
+            target.set('bus', disk['bus'])
+            diske.append(target)
+            devices.append(diske)
+            self.logger.debug('Added ' + str(disk['size']) + ' disk')
+        macs = list(self.macs)
+        for interface in devices.iterfind('interface'):
+            if interface.get('type') in ['network', 'bridge']:
+                if len(macs) > 0:
+                    mac = macs.pop(0)
+                    interface.find('mac').set('address', mac)
+                    self.logger.debug('Rewrote interface '
+                                      'to use specified mac address ' + mac)
+                else:
+                    mac = random_mac_address()
+                    interface.find('mac').set('address', mac)
+                    self.macs.append(mac)
+                    self.logger.debug('Rewrote interface '
+                                      'to use random mac address ' + mac)
+            if interface.get('type') == 'bridge':
+                interface.find('source').set('bridge', config.bridge)
+        serial = ElementTree.Element('serial')
+        serial.set('type', 'file')
+        source = ElementTree.Element('source')
+        log_filename = os.path.join(config.logpath, self.name + '.syslog.log')
+        source.set('path', log_filename)
+        serial.append(source)
+        target = ElementTree.Element('target')
+        target.set('port', '0')
+        serial.append(target)
+        devices.append(serial)
+
+    def _installvm(self, lv=None, tmpdir=None, xml=None):
+        """
+        Install a VM, then undefine it in libvirt.
+        The final installation will recreate the VM using the existing disks.
+        """
+        self.logger.info('Creating VM')
+        if lv is None:
+            lv = self.lv
+        if xml is None:
+            xml = self.xml
+        if tmpdir is None:
+            tmpdir = self.tmpdir
+        vm = lv.defineXML(ElementTree.tostring(xml.getroot()))
+        os.chmod(tmpdir, 0755)
+        vm.create()
+        self.logger.info('Installing system on VM (may take over an hour)')
+        self.logger.info('You can watch the progress with virt-viewer')
+        log_filename = os.path.join(config.logpath, self.name + '.syslog.log')
+        self.logger.info('Logs will be written to ' + log_filename)
+
+        while vm.isActive() is not 0:
+            pass
+
+        vm.undefine()
+        self.logger.info('Installation complete')
+
+    def _finalxml(self, tmpdir=None, xml=None):
+        """
+        Create the XML to be used for the post-installation VM.
+        This may be a transformation of the installation XML.
+        """
+        self.logger.info('Creating final VM XML')
+        if xml is None:
+            xml = ElementTree.ElementTree(file=self.xml)
+        if tmpdir is None:
+            tmpdir = self.tmpdir
+        if self.rewrite in ['all', 'minimal']:
+            self.logger.debug('Setting VM to reboot normally on reboot')
+            xml.find('on_reboot').text = 'restart'
+            if self.rewrite == 'all':
+                self.logger.debug('Removing VM install parameters')
+                ose = xml.find('os')
+                for kernel in ose.iterfind('kernel'):
+                    ose.remove(kernel)
+                for initrd in ose.iterfind('initrd'):
+                    ose.remove(initrd)
+                for cmdline in ose.iterfind('cmdline'):
+                    ose.remove(cmdline)
+                devices = xml.find('devices')
+                devices.remove(devices.find('serial'))
+                for disk in list(devices.iterfind('disk')):
+                    if disk.get('device') == 'cdrom':
+                        disk.remove(disk.find('source'))
+        else:
+            self.logger.info('Not rewriting XML because rewrite is ' +
+                             self.rewrite)
+        if self.debug:
+            xml.write(os.path.join(tmpdir, 'final.xml'))
+        return xml
+
+    def _tmpimage(self, image=None, tmpdir=None):
+        """
+        Create a temporary copy of the image so libvirt will lock that copy.
+        This allows other simultaneous processes to update the cached image.
+        """
+        if image is None:
+            image = self.image.image
+        if tmpdir is None:
+            tmpdir = self.tmpdir
+        self.logger.info('Making temp copy of install image')
+        tmpimage = os.path.join(tmpdir, os.path.basename(image))
+        self.logger.debug('Copying ' + image + ' to ' + tmpimage)
+        shutil.copyfile(image, tmpimage)
+        return tmpimage
+
+    def _create(self):
+        """
+        Create the VM, install the system, and prepare it to boot.
+        This primarily calls functions from CustomInstallMixin and CustomVM.
+        """
+        self.logger.info('Creating custom virtual machine')
+
+        tmpdir = tempfile.mkdtemp(prefix='/tmp/' + self.name + '_')
+        self.logger.debug('Working dir: ' + tmpdir)
+        os.chdir(tmpdir)
+
+        kernel = self._preparekernel(kernel=self.kernel, tmpdir=tmpdir)
+
+        initrd = self._prepareinitrd(initrd=self.initrd, tmpdir=tmpdir)
+
+        self._unpackinitrd(initrd=initrd, tmpdir=tmpdir)
+
+        self._setuplatecommand(tmpdir=tmpdir)
+
+        self._setuppreseed(tmpdir=tmpdir)
+
+        if self.rewrite == 'all':
+            self._setuplogging(tmpdir=tmpdir)
+        else:
+            self.logger.debug('Skipping logging setup because rewrite is' +
+                              self.rewrite)
+
+        initrd = self._repackinitrd(tmpdir=tmpdir)
+
+        self._createdisks()
+
+        image = self._tmpimage(image=self.image.image, tmpdir=tmpdir)
+
+        xml = self._installxml(cmdline=self.cmdline, image=image,
+                               initrd=initrd, kernel=kernel,
+                               tmpdir=tmpdir, xml=self.xml)
+
+        self._installvm(lv=self.lv, tmpdir=tmpdir, xml=xml)
+
+        xml = self._finalxml(tmpdir=tmpdir, xml=xml)
+
+        self.logger.info('Setting up final VM')
+        self.vm = self.lv.defineXML(ElementTree.tostring(xml.getroot()))
+
+        if self.debug:
+            self.logger.info('Leaving temp directory '
+                             'because debug is enabled: ' + tmpdir)
+        else:
+            self.logger.info('Cleaning up temp directory')
+            shutil.rmtree(tmpdir)
+        return True
+
+    def _start(self):
+        """
+        Start the VM.
+        """
+        self.logger.info('Starting CustomVM')
+        if self.vm is not None:
+            if self.vm.isActive() == 0:
+                self.vm.create()
+        else:
+            raise UTAHVMProvisioningException('Failed to provision VM')
+        self.logger.info('Waiting ' + str(self.boottimeout) +
+                         ' seconds to allow machine to boot')
+        try:
+            self.pingpoll(timeout=self.boottimeout)
+        except UTAHTimeout:
+            # Ignore timeout for ping, since depending on the network
+            # configuration ssh might still work despite of the ping failure.
+            self.logger.warning('Network connectivity (ping) failure')
+        self.sshpoll(timeout=self.boottimeout)
+        self.active = True
+
+    def destroy(self, *args, **kw):
+        """
+        Remove the machine from libvirt and remove all the disk files.
+        """
+        # TODO: make this use standard cleanup
+        super(CustomVM, self).destroy(*args, **kw)
+        self.stop(force=True)
+        if self.vm is not None:
+            self.vm.undefine()
+        else:
+            self.logger.info('VM not created')
+        for disk in self.disks:
+            os.unlink(disk['file'])
+        shutil.rmtree(self.directory)
+
+
+# See http://kennethreitz.com/blog/generate-a-random-mac-address-in-python/
+def random_mac_address():
+    """Returns a completely random Mac Address"""
+    mac = [0x52, 0x54, 0x00,
+           random.randint(0x00, 0xff),
+           random.randint(0x00, 0xff),
+           random.randint(0x00, 0xff)]
+    return ':'.join(map(lambda x: "%02x" % x, mac))
+
+
+class TinySQLiteInventory(SQLiteInventory):
+    """
+    Tiny SQLite inventory that implements request, release, and destroy.
+    No authentication or conflict checking currently exists.
+    Only suitable for VMs at present.
+    """
+    def __init__(self, *args, **kw):
+        """
+        Initialize simple database.
+        """
+        super(TinySQLiteInventory, self).__init__(*args, **kw)
+        self.connection.execute(
+            'CREATE TABLE IF NOT EXISTS '
+            'machines(machineid INTEGER PRIMARY KEY, state TEXT)')
+
+    def request(self, machinetype=CustomVM, *args, **kw):
+        """
+        Takes a Machine class as machinetype, and passes the newly generated
+        machineid along with all other arguments to that class's constructor,
+        returning the resulting object.
+        """
+        cursor = self.connection.cursor()
+        cursor.execute("INSERT INTO machines (state) VALUES ('provisioned')")
+        machineid = cursor.lastrowid
+        return machinetype(machineid=machineid, *args, **kw)
+
+    def release(self, machineid):
+        """
+        Updates the database to indicate the machine is available.
+        """
+        if self.connection.execute(
+                "UPDATE machines SET state='available' WHERE machineid=?",
+                [machineid]):
+            return True
+        else:
+            return False
+
+    def destroy(self, machineid):
+        """
+        Updates the database to indicate the machine is destroyed, but does not
+        destroy the machine.
+        """
+        if self.connection.execute(
+                "UPDATE machines SET state='destroyed' ""WHERE machineid=?",
+                [machineid]):
+            return True
+        else:
+            return False