Merge ~pengpengs/cloud-init:fix/load-DataSourceOVF-at-the-last-on-VMware-Platform-if-no-data-found into cloud-init:master

[Pengpeng Sun]

1. Generate random instance-id when vmware customization cfg found. If use fixed
   instance-id, network configuration in customization cfg will not be applied.
2. Load DatasourceOVF when no vmware customization cfg found if machine fulfills:
   a. VMware Virtualization Platform found
   b. flag vmware_customization_supported is True
   c. flag disable_vmware_customization is False
   d. VMware tools deployPkgPlugin found
   These conditions are same with the ones in ds-identify script, if ds-identify
   found OVF, then load DatasourceOVF.
3. Read instance-id from machine and set it to metadata when no vmware customization
   cfg found, this prevents cloud-init load default network configuration, so that
   existing network configuration will not be overwritten.
4. Modify check instance-id unittest.
5. Modify default timeout for waiting vmware customization cfg, 30 seconds
   should be enough.

[Scott Moser (smoser)]

@Pengpeng,

Cloud-init has moved code-hosting and pull request to github. Please re-submit there.

https://lists.launchpad.net/cloud-init/msg00231.html

review: Needs Fixing

[Pengpeng Sun (pengpengs)]

Thanks @Scott, follows the instruction.

[Pengpeng Sun (pengpengs)]

@Scott, I have finished the instruction and "'Create pull request'", then waiting for the approval, am I doing correctly?

Unmerged commits

2120168... by Pengpeng Sun on 2019-11-15

This change is to fix LP bug #1835205, it includes:
1. Generate random instance-id when vmware customization cfg found. If use fixed
   instance-id, network configuration in customization cfg will not be applied.
2. Load DatasourceOVF when no vmware customization cfg found if machine fulfills:
   a. VMware Virtualization Platform found
   b. flag vmware_customization_supported is True
   c. flag disable_vmware_customization is False
   d. VMware tools deployPkgPlugin found
   These conditions are same with the ones in ds-identify script, if ds-identify
   found OVF, then load DatasourceOVF.
3. Read instance-id from machine and set it to metadata when no vmware customization
   cfg found, this prevents cloud-init load default network configuration, so that
   existing network configuration will not be overwritten.
4. Modify check instance-id unittest.

diff --git a/cloudinit/sources/DataSourceOVF.py b/cloudinit/sources/DataSourceOVF.py
index 896841e..2125df3 100644
--- a/cloudinit/sources/DataSourceOVF.py
+++ b/cloudinit/sources/DataSourceOVF.py
@@ -65,7 +65,7 @@ class DataSourceOVF(sources.DataSource):
         self._network_config = None
         self._vmware_nics_to_enable = None
         self._vmware_cust_conf = None
-        self._vmware_cust_found = False
+        self._vmware_cust_enabled = False
 
     def __str__(self):
         root = sources.DataSource.__str__(self)
@@ -113,6 +113,12 @@ class DataSourceOVF(sources.DataSource):
                     if deployPkgPluginPath:
                         LOG.debug("Found the customization plugin at %s",
                                   deployPkgPluginPath)
+                        # Set True when below conditions are all fulfilled
+                        # 1. VMware Virtualization Platfrom found
+                        # 2. VMware Customization supported is True
+                        # 3. disable_vmware_customization config is False
+                        # 4. libdeployPkgPlugin.so found
+                        self._vmware_cust_enabled = True
                         break
 
                 if deployPkgPluginPath:
@@ -246,8 +252,7 @@ class DataSourceOVF(sources.DataSource):
                         GuestCustEvent.GUESTCUST_EVENT_CUSTOMIZE_FAILED,
                         vmwareImcConfigFilePath)
 
-            self._vmware_cust_found = True
-            found.append('vmware-tools')
+            found.append('vmware-tools-new-config')
 
             # TODO: Need to set the status to DONE only when the
             # customization is done successfully.
@@ -272,7 +277,11 @@ class DataSourceOVF(sources.DataSource):
 
         # There was no OVF transports found
         if len(found) == 0:
-            return False
+            if self._vmware_cust_enabled:
+                found.append("vmware-tools-no-config")
+                (md, ud, cfg) = read_vmware_imc(None, self.paths)
+            else:
+                return False
 
         if 'seedfrom' in md and md['seedfrom']:
             seedfrom = md['seedfrom']
@@ -336,7 +345,7 @@ class DataSourceOVFNet(DataSourceOVF):
 
 
 def get_max_wait_from_cfg(cfg):
-    default_max_wait = 90
+    default_max_wait = 30
     max_wait_cfg_option = 'vmware_cust_file_max_wait'
     max_wait = default_max_wait
 
@@ -393,20 +402,29 @@ def get_network_config(nics=None, nameservers=None, search=None):
 
 # This will return a dict with some content
 #  meta-data, user-data, some config
-def read_vmware_imc(config):
+def read_vmware_imc(config, paths=None):
     md = {}
     cfg = {}
     ud = None
-    if config.host_name:
-        if config.domain_name:
-            md['local-hostname'] = config.host_name + "." + config.domain_name
-        else:
-            md['local-hostname'] = config.host_name
-
-    if config.timezone:
-        cfg['timezone'] = config.timezone
-
-    md['instance-id'] = "iid-vmware-imc"
+    if config is not None:
+        if config.host_name:
+            if config.domain_name:
+                md['local-hostname'] = config.host_name + "." +\
+                                       config.domain_name
+            else:
+                md['local-hostname'] = config.host_name
+        if config.timezone:
+            cfg['timezone'] = config.timezone
+        # Set a random new instance-id if there is new config
+        md['instance-id'] = "iid-vmware-imc-" + util.rand_str(strlen=8)
+    else:
+        # Set the same intance-id with previous one if no config
+        dp = paths.get_cpath('data')
+        iid_fn = os.path.join(dp, 'instance-id')
+        try:
+            md['instance-id'] = util.load_file(iid_fn).strip()
+        except Exception:
+            md['instance-id'] = "iid-vmware-imc"
     return (md, ud, cfg)
 
 
diff --git a/tests/unittests/test_vmware_config_file.py b/tests/unittests/test_vmware_config_file.py
index 16343ed..91adab9 100644
--- a/tests/unittests/test_vmware_config_file.py
+++ b/tests/unittests/test_vmware_config_file.py
@@ -56,19 +56,19 @@ class TestVmwareConfigFile(CiTestCase):
         """Tests instance id for the DatasourceOVF"""
         cf = ConfigFile("tests/data/vmware/cust-dhcp-2nic.cfg")
 
-        instance_id_prefix = 'iid-vmware-'
+        instance_id_prefix = 'iid-vmware-imc'
 
         conf = Config(cf)
 
         (md1, _, _) = read_vmware_imc(conf)
         self.assertIn(instance_id_prefix, md1["instance-id"])
-        self.assertEqual(md1["instance-id"], 'iid-vmware-imc')
+        self.assertEqual(len(md1["instance-id"]), len(instance_id_prefix) + 9)
 
         (md2, _, _) = read_vmware_imc(conf)
         self.assertIn(instance_id_prefix, md2["instance-id"])
-        self.assertEqual(md2["instance-id"], 'iid-vmware-imc')
+        self.assertEqual(len(md2["instance-id"]), len(instance_id_prefix) + 9)
 
-        self.assertEqual(md2["instance-id"], md1["instance-id"])
+        self.assertNotEqual(md2["instance-id"], md1["instance-id"])
 
     def test_configfile_static_2nics(self):
         """Tests Config class for a configuration with two static NICs."""