Merge lp:~asomya/nova/dev into lp:~hudson-openstack/nova/trunk

[Arvind Somya]

Bug #796813: vmwareapi does not support distributed vswitch

The extant API assumed a standard switch network for all cases of ESX(i), was not usable when the hypervisor was added to a distributed portgroup on a virtual switch. The VM's would create a new standard switch with the same name as the dv portgroup.

Bug #796834: ESX(i) : Incorrect machine id parameters in vmx for guest_tool.py

-The machine.id string was missing required information for the guest_tool.py script to set networking in the guest VMs.
-Additionally the script crashed on Python > 2.4 due to the implicit gettext translator(_()) used.
-Added support for Ubuntu guest operating systems.

[Rick Harris (rconradharris)]

Nice work Arvind.

Some small nits:

> 32 + # NOTE: This only works on ESXi if the port binding is
> 33 + # set to ephemeral

> 81 + # NOTE: Only works on ESXi if the portgroup binding is set to

Per HACKING, this should be NOTE(asomya):

> 40 + "get_dynamic_property",network,

PEP-8 Nit: needs a space after the comma:

  "get_dynamic_property", network,

There are a few other pep-8 nits in the code, I'd suggest running the pep8 utility on it.

> 178 +def _set_ubuntu_networking(network_details=[]):

Usually not a good idea to have a kwarg default to a list like that[1].

Better would be:

  def _set_ubuntu_networking(network_details=None):
      if not network_details:
          network_details = []

      OR

      network_details = network_details or []

[1] http://www.deadlybloodyserious.com/2008/05/default-argument-blunders/

review: Needs Fixing

[Vish Ishaya (vishvananda)]

172 +_ = gettext.gettext
173 +

we generally use
gettext.install('nova', unicode=1)

not sure why it was done differently here.

The NOTE(asomya) would also be valuable.

review: Needs Information

[Rick Harris (rconradharris)]

Thanks for the fixes, looks good.

review: Approve

[Vish Ishaya (vishvananda)]

looks good now.

review: Approve

[OpenStack Infra (hudson-openstack)]

Attempt to merge into lp:nova failed due to conflicts:

text conflict in nova/virt/vmwareapi/vmops.py

[Vish Ishaya (vishvananda)]

retrying

[OpenStack Infra (hudson-openstack)]

The attempt to merge lp:~asomya/nova/dev into lp:nova failed. Below is the output from the failed tests.

FloatingIpTest
    test_floating_ip_allocate OK 0.31
    test_floating_ip_associate OK 0.11
    test_floating_ip_disassociate OK 0.10
    test_floating_ip_release OK 0.10
    test_floating_ip_show OK 0.11
    test_floating_ips_list OK 0.10
    test_translate_floating_ip_view OK 0.05
FixedIpTest
    test_add_fixed_ip OK 0.08
    test_add_fixed_ip_no_network OK 0.09
    test_remove_fixed_ip OK 0.30
    test_remove_fixed_ip_no_address OK 0.07
FlavorsExtraSpecsTest
    test_create OK 0.05
    test_create_empty_body OK 0.05
    test_delete OK 0.05
    test_index OK 0.05
    test_index_no_data OK 0.05
    test_show OK 0.05
    test_show_spec_not_found OK 0.05
    test_update_item OK 0.05
    test_update_item_body_uri_mismatch OK 0.06
    test_update_item_empty_body OK 0.05
    test_update_item_too_many_keys OK 0.06
AccountsTest
    test_account_create OK 0.39
    test_account_delete OK 0.17
    test_account_update OK 0.18
    test_get_account OK 0.17
AdminAPITest
    test_admin_disabled OK 0.14
    test_admin_enabled OK 0.41
APITest
    test_exceptions_are_converted_to_faults OK 0.01
    test_malformed_json OK 0.07
    test_malformed_xml OK 0.06
Test
    test_authorize_project OK 0.11
    test_authorize_token OK 0.11
    test_authorize_user OK 0.06
    test_bad_project OK 0.35
    test_bad_token OK 0.06
    test_bad_user_bad_key OK 0.07
    test_bad_user_good_key OK 0.06
    test_no_user OK 0.06
    test_not_existing_project OK 0.10
    test_token_expiry OK 0.06
TestFunctiona...

[Vish Ishaya (vishvananda)]

Looks like you need to do some pep8 cleanup

=== modified file 'Authors'
--- Authors	2011-07-20 19:10:01 +0000
+++ Authors	2011-07-25 15:24:59 +0000
@@ -8,6 +8,7 @@
 Anthony Young <[e-mail]>
 Antony Messerli <[e-mail]>
 Armando Migliaccio <[e-mail]>
+Arvind Somya <[e-mail]>
 Bilal Akhtar <[e-mail]>
 Brian Lamar <[e-mail]>
 Brian Schott <[e-mail]>

=== modified file 'nova/virt/vmwareapi/network_utils.py'
--- nova/virt/vmwareapi/network_utils.py	2011-03-24 16:38:31 +0000
+++ nova/virt/vmwareapi/network_utils.py	2011-07-25 15:24:59 +0000
@@ -45,10 +45,30 @@
     networks = session._call_method(vim_util,
                        "get_properties_for_a_collection_of_objects",
                        "Network", vm_networks, ["summary.name"])
-    for network in networks:
-        if network.propSet[0].val == network_name:
-            return network.obj
-    return None
+    network_obj = {}
+    for network in vm_networks:
+        # Get network properties
+        if network._type == 'DistributedVirtualPortgroup':
+            props = session._call_method(vim_util,
+                        "get_dynamic_property", network,
+                        "DistributedVirtualPortgroup", "config")
+            # NOTE(asomya): This only works on ESXi if the port binding is
+            # set to ephemeral
+            if props.name == network_name:
+                network_obj['type'] = 'DistributedVirtualPortgroup'
+                network_obj['dvpg'] = props.key
+                network_obj['dvsw'] = props.distributedVirtualSwitch.value
+        else:
+            props = session._call_method(vim_util,
+                        "get_dynamic_property", network,
+                        "Network", "summary.name")
+            if props == network_name:
+                network_obj['type'] = 'Network'
+                network_obj['name'] = network_name
+    if (len(network_obj) > 0):
+        return network_obj
+    else:
+        return None
 
 
 def get_vswitch_for_vlan_interface(session, vlan_interface):

=== modified file 'nova/virt/vmwareapi/vm_util.py'
--- nova/virt/vmwareapi/vm_util.py	2011-05-19 15:40:49 +0000
+++ nova/virt/vmwareapi/vm_util.py	2011-07-25 15:24:59 +0000
@@ -40,7 +40,7 @@
 
 def get_vm_create_spec(client_factory, instance, data_store_name,
                        network_name="vmnet0",
-                       os_type="otherGuest"):
+                       os_type="otherGuest", network_ref=None):
     """Builds the VM Create spec."""
     config_spec = client_factory.create('ns0:VirtualMachineConfigSpec')
     config_spec.name = instance.name
@@ -93,7 +93,8 @@
     return virtual_device_config
 
 
-def create_network_spec(client_factory, network_name, mac_address):
+def create_network_spec(client_factory, network_name, mac_address,
+                        network_ref=None):
     """
     Builds a config spec for the addition of a new network
     adapter to the VM.
@@ -105,9 +106,24 @@
     # Get the recommended card type for the VM based on the guest OS of the VM
     net_device = client_factory.create('ns0:VirtualPCNet32')
 
-    backing = \
-        client_factory.create('ns0:VirtualEthernetCardNetworkBackingInfo')
-    backing.deviceName = network_name
+    # NOTE(asomya): Only works on ESXi if the portgroup binding is set to
+    # ephemeral. Invalid configuration if set to static and the NIC does
+    # not come up on boot if set to dynamic.
+    backing = None
+    if (network_ref['type'] == "DistributedVirtualPortgroup"):
+        backing_name = \
+         'ns0:VirtualEthernetCardDistributedVirtualPortBackingInfo'
+        backing = \
+         client_factory.create(backing_name)
+        portgroup = \
+         client_factory.create('ns0:DistributedVirtualSwitchPortConnection')
+        portgroup.switchUuid = network_ref['dvsw']
+        portgroup.portgroupKey = network_ref['dvpg']
+        backing.port = portgroup
+    else:
+        backing = \
+         client_factory.create('ns0:VirtualEthernetCardNetworkBackingInfo')
+        backing.deviceName = network_name
 
     connectable_spec = \
         client_factory.create('ns0:VirtualDeviceConnectInfo')
@@ -278,9 +294,11 @@
     return config_spec
 
 
-def get_machine_id_change_spec(client_factory, mac, ip_addr, netmask, gateway):
+def get_machine_id_change_spec(client_factory, mac, ip_addr, netmask,
+                               gateway, broadcast, dns):
     """Builds the machine id change config spec."""
-    machine_id_str = "%s;%s;%s;%s" % (mac, ip_addr, netmask, gateway)
+    machine_id_str = "%s;%s;%s;%s;%s;%s" % (mac, ip_addr, netmask,
+                                            gateway, broadcast, dns)
     virtual_machine_config_spec = \
         client_factory.create('ns0:VirtualMachineConfigSpec')
 

=== modified file 'nova/virt/vmwareapi/vmops.py'
--- nova/virt/vmwareapi/vmops.py	2011-06-03 18:34:54 +0000
+++ nova/virt/vmwareapi/vmops.py	2011-07-25 15:24:59 +0000
@@ -116,8 +116,9 @@
                                                         net_name)
             if network_ref is None:
                 raise exception.NetworkNotFoundForBridge(bridge=net_name)
+            return network_ref
 
-        _check_if_network_bridge_exists()
+        network_obj = _check_if_network_bridge_exists()
 
         def _get_datastore_ref():
             """Get the datastore list and choose the first local storage."""
@@ -175,8 +176,10 @@
         vm_folder_mor, res_pool_mor = _get_vmfolder_and_res_pool_mors()
 
         # Get the create vm config spec
-        config_spec = vm_util.get_vm_create_spec(client_factory, instance,
-                                data_store_name, net_name, os_type)
+        config_spec = vm_util.get_vm_create_spec(
+                            client_factory, instance,
+                            data_store_name, net_name, os_type,
+                            network_obj)
 
         def _execute_create_vm():
             """Create VM on ESX host."""
@@ -718,13 +721,17 @@
 
         net_mask = network["netmask"]
         gateway = network["gateway"]
+        broadcast = network["broadcast"]
+        dns = network["dns"]
+
         addresses = db.instance_get_fixed_addresses(admin_context,
                                                     instance['id'])
         ip_addr = addresses[0] if addresses else None
 
         machine_id_chanfge_spec = \
             vm_util.get_machine_id_change_spec(client_factory, mac_address,
-                                        ip_addr, net_mask, gateway)
+                                               ip_addr, net_mask, gateway,
+                                               broadcast, dns)
         LOG.debug(_("Reconfiguring VM instance %(name)s to set the machine id "
                   "with ip - %(ip_addr)s") %
                   ({'name': instance.name,

=== modified file 'tools/esx/guest_tool.py'
--- tools/esx/guest_tool.py	2011-04-18 20:53:09 +0000
+++ tools/esx/guest_tool.py	2011-07-25 15:24:59 +0000
@@ -21,6 +21,7 @@
 """
 
 import array
+import gettext
 import logging
 import os
 import platform
@@ -30,6 +31,8 @@
 import sys
 import time
 
+gettext.install('nova', unicode=1)
+
 PLATFORM_WIN = 'win32'
 PLATFORM_LINUX = 'linux2'
 ARCH_32_BIT = '32bit'
@@ -275,7 +278,8 @@
     return final_list
 
 
-def _set_rhel_networking(network_details=[]):
+def _set_rhel_networking(network_details=None):
+    network_details = network_details or []
     all_dns_servers = []
     for network_detail in network_details:
         mac_address, ip_address, subnet_mask, gateway, broadcast,\
@@ -315,6 +319,46 @@
     _execute(['/sbin/service', 'network', 'restart'])
 
 
+def _set_ubuntu_networking(network_details=None):
+    network_details = network_details or []
+    """ Set IPv4 network settings for Ubuntu """
+    all_dns_servers = []
+    for network_detail in network_details:
+        mac_address, ip_address, subnet_mask, gateway, broadcast,\
+            dns_servers = network_detail
+        all_dns_servers.extend(dns_servers)
+        adapter_name, current_ip_address = \
+                _get_linux_adapter_name_and_ip_address(mac_address)
+
+        if adapter_name and not ip_address == current_ip_address:
+            interface_file_name = \
+                '/etc/network/interfaces'
+            # Remove file
+            os.remove(interface_file_name)
+            # Touch file
+            _execute(['touch', interface_file_name])
+            interface_file = open(interface_file_name, 'w')
+            interface_file.write('\nauto %s' % adapter_name)
+            interface_file.write('\niface %s inet static' % adapter_name)
+            interface_file.write('\nbroadcast %s' % broadcast)
+            interface_file.write('\ngateway %s' % gateway)
+            interface_file.write('\nnetmask %s' % subnet_mask)
+            interface_file.write('\naddress %s' % ip_address)
+            interface_file.close()
+    if all_dns_servers:
+        dns_file_name = "/etc/resolv.conf"
+        os.remove(dns_file_name)
+        _execute(['touch', dns_file_name])
+        dns_file = open(dns_file_name, 'w')
+        dns_file.write("; generated by OpenStack guest tools")
+        unique_entries = _filter_duplicates(all_dns_servers)
+        for dns_server in unique_entries:
+            dns_file.write("\nnameserver %s" % dns_server)
+        dns_file.close()
+    print "\nRestarting networking....\n"
+    _execute(['/etc/init.d/networking', 'restart'])
+
+
 def _linux_set_networking():
     """Set IP address for the Linux VM."""
     vmware_tools_bin = None
@@ -330,8 +374,13 @@
         cmd = [vmware_tools_bin, '--cmd', 'machine.id.get']
         network_details = _parse_network_details(_execute(cmd,
                                                 check_exit_code=False))
-        # TODO(sateesh): For other distros like ubuntu, suse, debian, BSD, etc.
-        _set_rhel_networking(network_details)
+        # TODO(sateesh): For other distros like suse, debian, BSD, etc.
+        if(platform.dist()[0] == 'Ubuntu'):
+            _set_ubuntu_networking(network_details)
+        elif (platform.dist()[0] == 'redhat'):
+            _set_rhel_networking(network_details)
+        else:
+            logging.warn(_("Distro '%s' not supported") % platform.dist()[0])
     else:
         logging.warn(_("VMware Tools is not installed"))