Merge lp:~anso/nova/smoketests_fixes into lp:~hudson-openstack/nova/trunk

[Vish Ishaya]

Fixes existing smoketests and splits out sysadmin tests from netadmin tests.

* sets use_ipv6 = False by default
* switches to ami-tty for tests
* generally improves error handling

Fixes existing smoketests and splits out sysadmin tests from netadmin tests.

* sets use_ipv6 = False by default
* switches to ami-tty for tests
* generally improves error handling

There is a lot more that can be done here, but I want to get this out so people actually start running them and adding more.
To run the tests:
source an rc file (make sure that it comes from a nova.zip if you want to test image uploading)
cd smoketests
python admin_smoketests.py
python sysadmin_smoketests.py
python netadmin_smoketests.py

Future cleanup:

* Run all of the available tests with nose
* Prettify the output (a la our tests in nova)
* Some of public_network_smoketests have redundant code and are broken (I'm hoping the guys who did IPV6 can add the ipv6 tests in where appropriate to the other tests and we can delete it)
* Add more tests for other api commands.
* Tests for functionality (can the instance get to the public internet, etc.)

[Jay Pipes (jaypipes)]

This is excellent refactoring, Vishy. Very impressed. Looking forward to your future cleanups. Now, let's get these running on some test machines from Hudson!

review: Approve

[Devin Carlen (devcamcar)]

cool, looks good

review: Approve

[OpenStack Infra (hudson-openstack)]

The prerequisite lp:~vishvananda/nova/fix-describe-groups has not yet been merged into lp:nova.

[Vish Ishaya (vishvananda)]

reapproving now that prereq branch is in.

[Nachi Ueno (nati-ueno)]

Hi Vish
We will work for

* Some of public_network_smoketests have redundant code and are broken (I'm hoping the guys who did IPV6 can add the ipv6 tests in where appropriate to the other tests and we can delete it)

If you have any idea for design test for some network topology ( Single Node environment in which Compute node and Network node stay in same box, Multi-Node in which Compute node and Network node stay in different box or Multiple Network node environment, would you tell me that?

=== modified file 'contrib/nova.sh'
--- contrib/nova.sh	2011-01-21 23:48:10 +0000
+++ contrib/nova.sh	2011-02-23 02:16:10 +0000
@@ -66,7 +66,7 @@
     sudo apt-get install -y user-mode-linux kvm libvirt-bin
     sudo apt-get install -y screen euca2ools vlan curl rabbitmq-server
     sudo apt-get install -y lvm2 iscsitarget open-iscsi
-    sudo apt-get install -y socat
+    sudo apt-get install -y socat unzip
     echo "ISCSITARGET_ENABLE=true" | sudo tee /etc/default/iscsitarget
     sudo /etc/init.d/iscsitarget restart
     sudo modprobe kvm
@@ -111,8 +111,7 @@
 --nodaemon
 --dhcpbridge_flagfile=$NOVA_DIR/bin/nova.conf
 --network_manager=nova.network.manager.$NET_MAN
---cc_host=$HOST_IP
---routing_source_ip=$HOST_IP
+--my_ip=$HOST_IP
 --sql_connection=$SQL_CONN
 --auth_driver=nova.auth.$AUTH
 --libvirt_type=$LIBVIRT_TYPE
@@ -151,7 +150,6 @@
     mkdir -p $NOVA_DIR/instances
     rm -rf $NOVA_DIR/networks
     mkdir -p $NOVA_DIR/networks
-    $NOVA_DIR/tools/clean-vlans
     if [ ! -d "$NOVA_DIR/images" ]; then
         ln -s $DIR/images $NOVA_DIR/images
     fi
@@ -169,10 +167,14 @@
     # create a project called 'admin' with project manager of 'admin'
     $NOVA_DIR/bin/nova-manage project create admin admin
     # export environment variables for project 'admin' and user 'admin'
-    $NOVA_DIR/bin/nova-manage project environment admin admin $NOVA_DIR/novarc
+    $NOVA_DIR/bin/nova-manage project zipfile admin admin $NOVA_DIR/nova.zip
+    unzip -o $NOVA_DIR/nova.zip -d $NOVA_DIR/
     # create a small network
     $NOVA_DIR/bin/nova-manage network create 10.0.0.0/8 1 32
 
+    # create some floating ips
+    $NOVA_DIR/bin/nova-manage floating create `hostname` 10.6.0.0/27
+
     # nova api crashes if we start it with a regular screen command,
     # so send the start command by forcing text into the window.
     screen_it api "$NOVA_DIR/bin/nova-api"

=== modified file 'nova/__init__.py'
--- nova/__init__.py	2010-07-15 15:52:11 +0000
+++ nova/__init__.py	2011-02-23 02:16:10 +0000
@@ -30,5 +30,3 @@
 .. moduleauthor:: Manish Singh <[e-mail]>
 .. moduleauthor:: Andy Smith <[e-mail]>
 """
-
-from exception import *

=== modified file 'smoketests/base.py'
--- smoketests/base.py	2010-12-24 11:38:49 +0000
+++ smoketests/base.py	2011-02-23 02:16:10 +0000
@@ -17,19 +17,21 @@
 #    under the License.
 
 import boto
-import boto_v6
 import commands
 import httplib
 import os
 import paramiko
-import random
 import sys
+import time
 import unittest
 from boto.ec2.regioninfo import RegionInfo
 
 from smoketests import flags
 
+SUITE_NAMES = '[image, instance, volume]'
 FLAGS = flags.FLAGS
+flags.DEFINE_string('suite', None, 'Specific test suite to run ' + SUITE_NAMES)
+boto_v6 = None
 
 
 class SmokeTestCase(unittest.TestCase):
@@ -39,12 +41,10 @@
         client = paramiko.SSHClient()
         client.set_missing_host_key_policy(paramiko.WarningPolicy())
         client.connect(ip, username='root', pkey=key)
-        stdin, stdout, stderr = client.exec_command('uptime')
-        print 'uptime: ', stdout.read()
         return client
 
-    def can_ping(self, ip):
-        """ Attempt to ping the specified IP, and give up after 1 second. """
+    def can_ping(self, ip, command="ping"):
+        """Attempt to ping the specified IP, and give up after 1 second."""
 
         # NOTE(devcamcar): ping timeout flag is different in OSX.
         if sys.platform == 'darwin':
@@ -52,10 +52,41 @@
         else:
             timeout_flag = 'w'
 
-        status, output = commands.getstatusoutput('ping -c1 -%s1 %s' %
-                                                  (timeout_flag, ip))
+        status, output = commands.getstatusoutput('%s -c1 -%s1 %s' %
+                                                  (command, timeout_flag, ip))
         return status == 0
 
+    def wait_for_running(self, instance, tries=60, wait=1):
+        """Wait for instance to be running"""
+        for x in xrange(tries):
+            instance.update()
+            if instance.state.startswith('running'):
+                return True
+            time.sleep(wait)
+        else:
+            return False
+
+    def wait_for_ping(self, ip, command="ping", tries=120):
+        """Wait for ip to be pingable"""
+        for x in xrange(tries):
+            if self.can_ping(ip, command):
+                return True
+        else:
+            return False
+
+    def wait_for_ssh(self, ip, key_name, tries=30, wait=5):
+        """Wait for ip to be sshable"""
+        for x in xrange(tries):
+            try:
+                conn = self.connect_ssh(ip, key_name)
+                conn.close()
+            except Exception, e:
+                time.sleep(wait)
+            else:
+                return True
+        else:
+            return False
+
     def connection_for_env(self, **kwargs):
         """
         Returns a boto ec2 connection for the current environment.
@@ -144,8 +175,21 @@
         return True
 
 
+TEST_DATA = {}
+
+
+class UserSmokeTestCase(SmokeTestCase):
+    def setUp(self):
+        global TEST_DATA
+        self.conn = self.connection_for_env()
+        self.data = TEST_DATA
+
+
 def run_tests(suites):
     argv = FLAGS(sys.argv)
+    if FLAGS.use_ipv6:
+        global boto_v6
+        boto_v6 = __import__('boto_v6')
 
     if not os.getenv('EC2_ACCESS_KEY'):
         print >> sys.stderr, 'Missing EC2 environment variables. Please ' \

=== modified file 'smoketests/flags.py'
--- smoketests/flags.py	2010-12-24 11:38:49 +0000
+++ smoketests/flags.py	2011-02-23 02:16:10 +0000
@@ -35,5 +35,5 @@
 # http://code.google.com/p/python-gflags/source/browse/trunk/gflags.py#39
 
 DEFINE_string('region', 'nova', 'Region to use')
-DEFINE_string('test_image', 'ami-tiny', 'Image to use for launch tests')
-DEFINE_string('use_ipv6', True, 'use the ipv6 or not')
+DEFINE_string('test_image', 'ami-tty', 'Image to use for launch tests')
+DEFINE_bool('use_ipv6', False, 'use the ipv6 or not')

=== added file 'smoketests/netadmin_smoketests.py'
--- smoketests/netadmin_smoketests.py	1970-01-01 00:00:00 +0000
+++ smoketests/netadmin_smoketests.py	2011-02-23 02:16:10 +0000
@@ -0,0 +1,194 @@
+# vim: tabstop=4 shiftwidth=4 softtabstop=4
+
+# Copyright 2010 United States Government as represented by the
+# Administrator of the National Aeronautics and Space Administration.
+# All Rights Reserved.
+#
+#    Licensed under the Apache License, Version 2.0 (the "License"); you may
+#    not use this file except in compliance with the License. You may obtain
+#    a copy of the License at
+#
+#         http://www.apache.org/licenses/LICENSE-2.0
+#
+#    Unless required by applicable law or agreed to in writing, software
+#    distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+#    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+#    License for the specific language governing permissions and limitations
+#    under the License.
+
+import commands
+import os
+import random
+import sys
+import time
+import unittest
+
+# If ../nova/__init__.py exists, add ../ to Python search path, so that
+# it will override what happens to be installed in /usr/(local/)lib/python...
+possible_topdir = os.path.normpath(os.path.join(os.path.abspath(sys.argv[0]),
+                                   os.pardir,
+                                   os.pardir))
+if os.path.exists(os.path.join(possible_topdir, 'nova', '__init__.py')):
+    sys.path.insert(0, possible_topdir)
+
+from smoketests import flags
+from smoketests import base
+
+
+FLAGS = flags.FLAGS
+
+TEST_PREFIX = 'test%s' % int(random.random() * 1000000)
+TEST_BUCKET = '%s_bucket' % TEST_PREFIX
+TEST_KEY = '%s_key' % TEST_PREFIX
+TEST_GROUP = '%s_group' % TEST_PREFIX
+
+
+class AddressTests(base.UserSmokeTestCase):
+    def test_000_setUp(self):
+        self.create_key_pair(self.conn, TEST_KEY)
+        reservation = self.conn.run_instances(FLAGS.test_image,
+                                              instance_type='m1.tiny',
+                                              key_name=TEST_KEY)
+        self.data['instance'] = reservation.instances[0]
+        if not self.wait_for_running(self.data['instance']):
+            self.fail('instance failed to start')
+        self.data['instance'].update()
+        if not self.wait_for_ping(self.data['instance'].private_dns_name):
+            self.fail('could not ping instance')
+        if not self.wait_for_ssh(self.data['instance'].private_dns_name,
+                                 TEST_KEY):
+            self.fail('could not ssh to instance')
+
+    def test_001_can_allocate_floating_ip(self):
+        result = self.conn.allocate_address()
+        self.assertTrue(hasattr(result, 'public_ip'))
+        self.data['public_ip'] = result.public_ip
+
+    def test_002_can_associate_ip_with_instance(self):
+        result = self.conn.associate_address(self.data['instance'].id,
+                                             self.data['public_ip'])
+        self.assertTrue(result)
+
+    def test_003_can_ssh_with_public_ip(self):
+        ssh_authorized = False
+        groups = self.conn.get_all_security_groups(['default'])
+        for rule in groups[0].rules:
+            if (rule.ip_protocol == 'tcp' and
+                rule.from_port <= 22 and rule.to_port >= 22):
+                ssh_authorized = True
+        if not ssh_authorized:
+            self.conn.authorize_security_group('default',
+                                               ip_protocol='tcp',
+                                               from_port=22,
+                                               to_port=22)
+        try:
+            if not self.wait_for_ssh(self.data['public_ip'], TEST_KEY):
+                self.fail('could not ssh to public ip')
+        finally:
+            if not ssh_authorized:
+                self.conn.revoke_security_group('default',
+                                                ip_protocol='tcp',
+                                                from_port=22,
+                                                to_port=22)
+
+    def test_004_can_disassociate_ip_from_instance(self):
+        result = self.conn.disassociate_address(self.data['public_ip'])
+        self.assertTrue(result)
+
+    def test_005_can_deallocate_floating_ip(self):
+        result = self.conn.release_address(self.data['public_ip'])
+        self.assertTrue(result)
+
+    def test_999_tearDown(self):
+        self.delete_key_pair(self.conn, TEST_KEY)
+        self.conn.terminate_instances([self.data['instance'].id])
+
+
+class SecurityGroupTests(base.UserSmokeTestCase):
+
+    def __public_instance_is_accessible(self):
+        id_url = "latest/meta-data/instance-id"
+        options = "-s --max-time 1"
+        command = "curl %s %s/%s" % (options, self.data['public_ip'], id_url)
+        instance_id = commands.getoutput(command).strip()
+        if not instance_id:
+            return False
+        if instance_id != self.data['instance'].id:
+            raise Exception("Wrong instance id")
+        return True
+
+    def test_001_can_create_security_group(self):
+        self.conn.create_security_group(TEST_GROUP, description='test')
+
+        groups = self.conn.get_all_security_groups()
+        self.assertTrue(TEST_GROUP in [group.name for group in groups])
+
+    def test_002_can_launch_instance_in_security_group(self):
+        with open("proxy.sh") as f:
+            user_data = f.read()
+        self.create_key_pair(self.conn, TEST_KEY)
+        reservation = self.conn.run_instances(FLAGS.test_image,
+                                              key_name=TEST_KEY,
+                                              security_groups=[TEST_GROUP],
+                                              user_data=user_data,
+                                              instance_type='m1.tiny')
+
+        self.data['instance'] = reservation.instances[0]
+        if not self.wait_for_running(self.data['instance']):
+            self.fail('instance failed to start')
+        self.data['instance'].update()
+        if not self.wait_for_ping(self.data['instance'].private_dns_name):
+            self.fail('could not ping instance')
+        if not self.wait_for_ssh(self.data['instance'].private_dns_name,
+                                 TEST_KEY):
+            self.fail('could not ssh to instance')
+
+    def test_003_can_authorize_security_group_ingress(self):
+        self.assertTrue(self.conn.authorize_security_group(TEST_GROUP,
+                                                           ip_protocol='tcp',
+                                                           from_port=80,
+                                                           to_port=80))
+
+    def test_004_can_access_metadata_over_public_ip(self):
+        result = self.conn.allocate_address()
+        self.assertTrue(hasattr(result, 'public_ip'))
+        self.data['public_ip'] = result.public_ip
+
+        result = self.conn.associate_address(self.data['instance'].id,
+                                             self.data['public_ip'])
+        start_time = time.time()
+        try:
+            while not self.__public_instance_is_accessible():
+                # 1 minute to launch
+                if time.time() - start_time > 60:
+                    raise Exception("Timeout")
+                time.sleep(1)
+        finally:
+            result = self.conn.disassociate_address(self.data['public_ip'])
+
+    def test_005_can_revoke_security_group_ingress(self):
+        self.assertTrue(self.conn.revoke_security_group(TEST_GROUP,
+                                                        ip_protocol='tcp',
+                                                        from_port=80,
+                                                        to_port=80))
+        start_time = time.time()
+        while self.__public_instance_is_accessible():
+            # 1 minute to teardown
+            if time.time() - start_time > 60:
+                raise Exception("Timeout")
+            time.sleep(1)
+
+    def test_999_tearDown(self):
+        self.conn.delete_key_pair(TEST_KEY)
+        self.conn.delete_security_group(TEST_GROUP)
+        groups = self.conn.get_all_security_groups()
+        self.assertFalse(TEST_GROUP in [group.name for group in groups])
+        self.conn.terminate_instances([self.data['instance'].id])
+        self.assertTrue(self.conn.release_address(self.data['public_ip']))
+
+
+if __name__ == "__main__":
+    suites = {'address': unittest.makeSuite(AddressTests),
+              'security_group': unittest.makeSuite(SecurityGroupTests)
+              }
+    sys.exit(base.run_tests(suites))

=== added file 'smoketests/proxy.sh'
--- smoketests/proxy.sh	1970-01-01 00:00:00 +0000
+++ smoketests/proxy.sh	2011-02-23 02:16:10 +0000
@@ -0,0 +1,22 @@
+#!/usr/bin/env bash
+
+# This is a simple shell script that uses netcat to set up a proxy to the
+# metadata server on port 80 and to a google ip on port 8080.  This is meant
+# to be passed in by a script to an instance via user data, so that
+# automatic testing of network connectivity can be performed.
+
+# Example usage:
+#   euca-run-instances -t m1.tiny -f proxy.sh ami-tty
+
+mkfifo backpipe1
+mkfifo backpipe2
+
+# NOTE(vish): proxy metadata on port 80
+while true; do
+    nc -l -p 80 0<backpipe1 | nc 169.254.169.254 80 1>backpipe1
+done &
+
+# NOTE(vish): proxy google on port 8080
+while true; do
+    nc -l -p 8080 0<backpipe2 | nc 74.125.19.99 80 1>backpipe2
+done &

=== modified file 'smoketests/public_network_smoketests.py'
--- smoketests/public_network_smoketests.py	2010-12-24 11:38:49 +0000
+++ smoketests/public_network_smoketests.py	2011-02-23 02:16:10 +0000
@@ -24,9 +24,16 @@
 import time
 import unittest
 
+# If ../nova/__init__.py exists, add ../ to Python search path, so that
+# it will override what happens to be installed in /usr/(local/)lib/python...
+possible_topdir = os.path.normpath(os.path.join(os.path.abspath(sys.argv[0]),
+                                   os.pardir,
+                                   os.pardir))
+if os.path.exists(os.path.join(possible_topdir, 'nova', '__init__.py')):
+    sys.path.insert(0, possible_topdir)
+
 from smoketests import flags
 from smoketests import base
-from smoketests import user_smoketests
 
 #Note that this test should run from
 #public network (outside of private network segments)
@@ -42,7 +49,7 @@
 TEST_DATA = {}
 
 
-class InstanceTestsFromPublic(user_smoketests.UserSmokeTestCase):
+class InstanceTestsFromPublic(base.UserSmokeTestCase):
     def test_001_can_create_keypair(self):
         key = self.create_key_pair(self.conn, TEST_KEY)
         self.assertEqual(key.name, TEST_KEY)

=== renamed file 'smoketests/user_smoketests.py' => 'smoketests/sysadmin_smoketests.py'
--- smoketests/user_smoketests.py	2011-01-12 08:47:54 +0000
+++ smoketests/sysadmin_smoketests.py	2011-02-23 02:16:10 +0000
@@ -19,7 +19,6 @@
 import commands
 import os
 import random
-import socket
 import sys
 import time
 import unittest
@@ -36,10 +35,8 @@
 from smoketests import base
 
 
-SUITE_NAMES = '[image, instance, volume]'
 
 FLAGS = flags.FLAGS
-flags.DEFINE_string('suite', None, 'Specific test suite to run ' + SUITE_NAMES)
 flags.DEFINE_string('bundle_kernel', 'openwrt-x86-vmlinuz',
               'Local kernel file to use for bundling tests')
 flags.DEFINE_string('bundle_image', 'openwrt-x86-ext2.image',
@@ -49,17 +46,7 @@
 TEST_BUCKET = '%s_bucket' % TEST_PREFIX
 TEST_KEY = '%s_key' % TEST_PREFIX
 TEST_GROUP = '%s_group' % TEST_PREFIX
-TEST_DATA = {}
-
-
-class UserSmokeTestCase(base.SmokeTestCase):
-    def setUp(self):
-        global TEST_DATA
-        self.conn = self.connection_for_env()
-        self.data = TEST_DATA
-
-
-class ImageTests(UserSmokeTestCase):
+class ImageTests(base.UserSmokeTestCase):
     def test_001_can_bundle_image(self):
         self.assertTrue(self.bundle_image(FLAGS.bundle_image))
 
@@ -91,7 +78,6 @@
                 break
             time.sleep(1)
         else:
-            print image.state
             self.assert_(False)  # wasn't available within 10 seconds
         self.assert_(image.type == 'machine')
 
@@ -133,7 +119,7 @@
         self.assertTrue(self.delete_bundle_bucket(TEST_BUCKET))
 
 
-class InstanceTests(UserSmokeTestCase):
+class InstanceTests(base.UserSmokeTestCase):
     def test_001_can_create_keypair(self):
         key = self.create_key_pair(self.conn, TEST_KEY)
         self.assertEqual(key.name, TEST_KEY)
@@ -143,109 +129,44 @@
                                               key_name=TEST_KEY,
                                               instance_type='m1.tiny')
         self.assertEqual(len(reservation.instances), 1)
-        self.data['instance_id'] = reservation.instances[0].id
+        self.data['instance'] = reservation.instances[0]
 
     def test_003_instance_runs_within_60_seconds(self):
-        reservations = self.conn.get_all_instances([self.data['instance_id']])
-        instance = reservations[0].instances[0]
+        instance = self.data['instance']
         # allow 60 seconds to exit pending with IP
-        for x in xrange(60):
-            instance.update()
-            if instance.state == u'running':
-                break
-            time.sleep(1)
-        else:
+        if not self.wait_for_running(self.data['instance']):
             self.fail('instance failed to start')
-        ip = reservations[0].instances[0].private_dns_name
+        self.data['instance'].update()
+        ip = self.data['instance'].private_dns_name
         self.failIf(ip == '0.0.0.0')
-        self.data['private_ip'] = ip
         if FLAGS.use_ipv6:
-            ipv6 = reservations[0].instances[0].dns_name_v6
+            ipv6 = self.data['instance'].dns_name_v6
             self.failIf(ipv6 is None)
-            self.data['ip_v6'] = ipv6
 
     def test_004_can_ping_private_ip(self):
-        for x in xrange(120):
-            # ping waits for 1 second
-            status, output = commands.getstatusoutput(
-                'ping -c1 %s' % self.data['private_ip'])
-            if status == 0:
-                break
-        else:
+        if not self.wait_for_ping(self.data['instance'].private_dns_name):
             self.fail('could not ping instance')
 
         if FLAGS.use_ipv6:
-            for x in xrange(120):
-            # ping waits for 1 second
-                status, output = commands.getstatusoutput(
-                    'ping6 -c1 %s' % self.data['ip_v6'])
-                if status == 0:
-                    break
-            else:
-                self.fail('could not ping instance')
+            if not self.wait_for_ping(self.data['instance'].ip_v6, "ping6"):
+                self.fail('could not ping instance v6')
 
     def test_005_can_ssh_to_private_ip(self):
-        for x in xrange(30):
-            try:
-                conn = self.connect_ssh(self.data['private_ip'], TEST_KEY)
-                conn.close()
-            except Exception:
-                time.sleep(1)
-            else:
-                break
-        else:
+        if not self.wait_for_ssh(self.data['instance'].private_dns_name,
+                                 TEST_KEY):
             self.fail('could not ssh to instance')
 
         if FLAGS.use_ipv6:
-            for x in xrange(30):
-                try:
-                    conn = self.connect_ssh(
-                                        self.data['ip_v6'], TEST_KEY)
-                    conn.close()
-                except Exception:
-                    time.sleep(1)
-                else:
-                    break
-            else:
+            if not self.wait_for_ssh(self.data['instance'].ip_v6,
+                                     TEST_KEY):
                 self.fail('could not ssh to instance v6')
 
-    def test_006_can_allocate_elastic_ip(self):
-        result = self.conn.allocate_address()
-        self.assertTrue(hasattr(result, 'public_ip'))
-        self.data['public_ip'] = result.public_ip
-
-    def test_007_can_associate_ip_with_instance(self):
-        result = self.conn.associate_address(self.data['instance_id'],
-                                             self.data['public_ip'])
-        self.assertTrue(result)
-
-    def test_008_can_ssh_with_public_ip(self):
-        for x in xrange(30):
-            try:
-                conn = self.connect_ssh(self.data['public_ip'], TEST_KEY)
-                conn.close()
-            except socket.error:
-                time.sleep(1)
-            else:
-                break
-        else:
-            self.fail('could not ssh to instance')
-
-    def test_009_can_disassociate_ip_from_instance(self):
-        result = self.conn.disassociate_address(self.data['public_ip'])
-        self.assertTrue(result)
-
-    def test_010_can_deallocate_elastic_ip(self):
-        result = self.conn.release_address(self.data['public_ip'])
-        self.assertTrue(result)
-
     def test_999_tearDown(self):
         self.delete_key_pair(self.conn, TEST_KEY)
-        if self.data.has_key('instance_id'):
-            self.conn.terminate_instances([self.data['instance_id']])
-
-
-class VolumeTests(UserSmokeTestCase):
+        self.conn.terminate_instances([self.data['instance'].id])
+
+
+class VolumeTests(base.UserSmokeTestCase):
     def setUp(self):
         super(VolumeTests, self).setUp()
         self.device = '/dev/vdb'
@@ -255,55 +176,65 @@
         reservation = self.conn.run_instances(FLAGS.test_image,
                                               instance_type='m1.tiny',
                                               key_name=TEST_KEY)
-        instance = reservation.instances[0]
-        self.data['instance'] = instance
-        for x in xrange(120):
-            if self.can_ping(instance.private_dns_name):
-                break
-        else:
-            self.fail('unable to start instance')
+        self.data['instance'] = reservation.instances[0]
+        if not self.wait_for_running(self.data['instance']):
+            self.fail('instance failed to start')
+        self.data['instance'].update()
+        if not self.wait_for_ping(self.data['instance'].private_dns_name):
+            self.fail('could not ping instance')
+        if not self.wait_for_ssh(self.data['instance'].private_dns_name,
+                                 TEST_KEY):
+            self.fail('could not ssh to instance')
 
     def test_001_can_create_volume(self):
         volume = self.conn.create_volume(1, 'nova')
         self.assertEqual(volume.size, 1)
         self.data['volume'] = volume
         # Give network time to find volume.
-        time.sleep(5)
+        time.sleep(10)
 
     def test_002_can_attach_volume(self):
         volume = self.data['volume']
 
         for x in xrange(10):
-            if volume.status == u'available':
+            volume.update()
+            if volume.status.startswith('available'):
                 break
-            time.sleep(5)
-            volume.update()
+            time.sleep(1)
         else:
             self.fail('cannot attach volume with state %s' % volume.status)
 
         volume.attach(self.data['instance'].id, self.device)
 
-        # Volumes seems to report "available" too soon.
+        # wait
         for x in xrange(10):
-            if volume.status == u'in-use':
+            volume.update()
+            if volume.status.startswith('in-use'):
                 break
-            time.sleep(5)
-            volume.update()
+            time.sleep(1)
+        else:
+            self.fail('volume never got to in use')
 
-        self.assertEqual(volume.status, u'in-use')
+        self.assertTrue(volume.status.startswith('in-use'))
 
         # Give instance time to recognize volume.
-        time.sleep(5)
+        time.sleep(10)
 
     def test_003_can_mount_volume(self):
         ip = self.data['instance'].private_dns_name
         conn = self.connect_ssh(ip, TEST_KEY)
-        commands = []
-        commands.append('mkdir -p /mnt/vol')
-        commands.append('mkfs.ext2 %s' % self.device)
-        commands.append('mount %s /mnt/vol' % self.device)
-        commands.append('echo success')
-        stdin, stdout, stderr = conn.exec_command(' && '.join(commands))
+        # NOTE(vish): this will create an dev for images that don't have
+        #             udev rules
+        stdin, stdout, stderr = conn.exec_command(
+                'grep %s /proc/partitions | '
+                '`awk \'{print "mknod /dev/"\\$4" b "\\$1" "\\$2}\'`'
+                % self.device.rpartition('/')[2])
+        exec_list = []
+        exec_list.append('mkdir -p /mnt/vol')
+        exec_list.append('/sbin/mke2fs %s' % self.device)
+        exec_list.append('mount %s /mnt/vol' % self.device)
+        exec_list.append('echo success')
+        stdin, stdout, stderr = conn.exec_command(' && '.join(exec_list))
         out = stdout.read()
         conn.close()
         if not out.strip().endswith('success'):
@@ -327,7 +258,7 @@
             "df -h | grep %s | awk {'print $2'}" % self.device)
         out = stdout.read()
         conn.close()
-        if not out.strip() == '1008M':
+        if not out.strip() == '1007.9M':
             self.fail('Volume is not the right size: %s %s' %
                       (out, stderr.read()))
 
@@ -354,79 +285,9 @@
         self.conn.delete_key_pair(TEST_KEY)
 
 
-class SecurityGroupTests(UserSmokeTestCase):
-
-    def __public_instance_is_accessible(self):
-        id_url = "latest/meta-data/instance-id"
-        options = "-s --max-time 1"
-        command = "curl %s %s/%s" % (options, self.data['public_ip'], id_url)
-        instance_id = commands.getoutput(command).strip()
-        if not instance_id:
-            return False
-        if instance_id != self.data['instance_id']:
-            raise Exception("Wrong instance id")
-        return True
-
-    def test_001_can_create_security_group(self):
-        self.conn.create_security_group(TEST_GROUP, description='test')
-
-        groups = self.conn.get_all_security_groups()
-        self.assertTrue(TEST_GROUP in [group.name for group in groups])
-
-    def test_002_can_launch_instance_in_security_group(self):
-        self.create_key_pair(self.conn, TEST_KEY)
-        reservation = self.conn.run_instances(FLAGS.test_image,
-                                              key_name=TEST_KEY,
-                                              security_groups=[TEST_GROUP],
-                                              instance_type='m1.tiny')
-
-        self.data['instance_id'] = reservation.instances[0].id
-
-    def test_003_can_authorize_security_group_ingress(self):
-        self.assertTrue(self.conn.authorize_security_group(TEST_GROUP,
-                                                           ip_protocol='tcp',
-                                                           from_port=80,
-                                                           to_port=80))
-
-    def test_004_can_access_instance_over_public_ip(self):
-        result = self.conn.allocate_address()
-        self.assertTrue(hasattr(result, 'public_ip'))
-        self.data['public_ip'] = result.public_ip
-
-        result = self.conn.associate_address(self.data['instance_id'],
-                                             self.data['public_ip'])
-        start_time = time.time()
-        while not self.__public_instance_is_accessible():
-            # 1 minute to launch
-            if time.time() - start_time > 60:
-                raise Exception("Timeout")
-            time.sleep(1)
-
-    def test_005_can_revoke_security_group_ingress(self):
-        self.assertTrue(self.conn.revoke_security_group(TEST_GROUP,
-                                                        ip_protocol='tcp',
-                                                        from_port=80,
-                                                        to_port=80))
-        start_time = time.time()
-        while self.__public_instance_is_accessible():
-            # 1 minute to teardown
-            if time.time() - start_time > 60:
-                raise Exception("Timeout")
-            time.sleep(1)
-
-    def test_999_tearDown(self):
-        self.conn.delete_key_pair(TEST_KEY)
-        self.conn.delete_security_group(TEST_GROUP)
-        groups = self.conn.get_all_security_groups()
-        self.assertFalse(TEST_GROUP in [group.name for group in groups])
-        self.conn.terminate_instances([self.data['instance_id']])
-        self.assertTrue(self.conn.release_address(self.data['public_ip']))
-
-
 if __name__ == "__main__":
     suites = {'image': unittest.makeSuite(ImageTests),
               'instance': unittest.makeSuite(InstanceTests),
-              'security_group': unittest.makeSuite(SecurityGroupTests),
               'volume': unittest.makeSuite(VolumeTests)
               }
     sys.exit(base.run_tests(suites))