Merge ~chad.smith/cloud-init:ubuntu/artful into cloud-init:ubuntu/artful

[Chad Smith]

Cherry pick fixes for IBMCloud upgrades from 17.1 -> 18.2 (artful-proposed)

Addresses the following bugs:
LP: #1766401
LP: #1767166

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:5a8b71fdc0b9313bfac2c8eedde1ced0004d5d2a
https://jenkins.ubuntu.com/server/job/cloud-init-ci/1085/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    SUCCESS: MAAS Compatability Testing
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/1085/rebuild

review: Approve (continuous-integration)

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:e5596beef0618204dedd94e1cd593a8cfb6bb6c7
https://jenkins.ubuntu.com/server/job/cloud-init-ci/1087/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    SUCCESS: MAAS Compatability Testing
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/1087/rebuild

review: Approve (continuous-integration)

diff --git a/debian/changelog b/debian/changelog
index 982d0c8..c438253 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,12 @@
+cloud-init (18.2-4-g05926e48-0ubuntu1~17.10.2) artful-proposed; urgency=medium
+
+  * cherry-pick 11172924: IBMCloud: Disable config-drive and nocloud
+    only if IBMCloud (LP: #1766401)
+  * cherry-pick 6ef92c98: IBMCloud: recognize provisioning environment
+    during debug (LP: #1767166)
+
+ -- Chad Smith <[e-mail]>  Tue, 01 May 2018 10:36:14 -0600
+
 cloud-init (18.2-4-g05926e48-0ubuntu1~17.10.1) artful-proposed; urgency=medium
 
   * debian/new-upstream-snapshot: Remove script, now maintained elsewhere.
diff --git a/debian/patches/cpick-11172924-IBMCloud-Disable-config-drive-and-nocloud-only-if b/debian/patches/cpick-11172924-IBMCloud-Disable-config-drive-and-nocloud-only-if
new file mode 100644
index 0000000..534aa73
--- /dev/null
+++ b/debian/patches/cpick-11172924-IBMCloud-Disable-config-drive-and-nocloud-only-if
@@ -0,0 +1,230 @@
+From 11172924a48a47a7231d19d9cefe628dfddda8bf Mon Sep 17 00:00:00 2001
+From: Scott Moser <[e-mail]>
+Date: Mon, 30 Apr 2018 13:21:51 -0600
+Subject: [PATCH] IBMCloud: Disable config-drive and nocloud only if IBMCloud
+ is enabled.
+
+Ubuntu images on IBMCloud for 16.04 have some seed data in
+/var/lib/cloud/data/seed/nocloud-net.  In order to have systems with
+IBMCloud enabled, we modified ds-identify detection to skip that seed
+if the system was on IBMCloud.  That change did not consider the
+fact that IBMCloud might not be in the datasource list.
+
+There was similar logic in the ConfigDrive datasource in ds-identify
+and the datasource itself.
+
+Config drive is now updated to only check and avoid IBMCloud if IBMCloud
+is enabled.  The check in ds-identify for nocloud was dropped.  If a
+user provides a nocloud seed on IBMCloud, then that can be used.
+
+This means that systems running Xenial will continue to get their
+old datasources.
+
+LP: #1766401
+---
+ cloudinit/sources/DataSourceConfigDrive.py | 11 +++--
+ tests/unittests/test_ds_identify.py        | 77 +++++++++++++++++++++++++++---
+ tools/ds-identify                          | 17 +++++--
+ 3 files changed, 91 insertions(+), 14 deletions(-)
+
+--- a/cloudinit/sources/DataSourceConfigDrive.py
++++ b/cloudinit/sources/DataSourceConfigDrive.py
+@@ -69,7 +69,8 @@
+                 util.logexc(LOG, "Failed reading config drive from %s", sdir)
+ 
+         if not found:
+-            for dev in find_candidate_devs():
++            dslist = self.sys_cfg.get('datasource_list')
++            for dev in find_candidate_devs(dslist=dslist):
+                 try:
+                     # Set mtype if freebsd and turn off sync
+                     if dev.startswith("/dev/cd"):
+@@ -211,7 +212,7 @@
+                 util.logexc(LOG, "Failed writing file: %s", filename)
+ 
+ 
+-def find_candidate_devs(probe_optical=True):
++def find_candidate_devs(probe_optical=True, dslist=None):
+     """Return a list of devices that may contain the config drive.
+ 
+     The returned list is sorted by search order where the first item has
+@@ -227,6 +228,9 @@
+         * either vfat or iso9660 formated
+         * labeled with 'config-2' or 'CONFIG-2'
+     """
++    if dslist is None:
++        dslist = []
++
+     # query optical drive to get it in blkid cache for 2.6 kernels
+     if probe_optical:
+         for device in OPTICAL_DEVICES:
+@@ -257,7 +261,8 @@
+     devices = [d for d in candidates
+                if d in by_label or not util.is_partition(d)]
+ 
+-    if devices:
++    LOG.debug("devices=%s dslist=%s", devices, dslist)
++    if devices and "IBMCloud" in dslist:
+         # IBMCloud uses config-2 label, but limited to a single UUID.
+         ibm_platform, ibm_path = get_ibm_platform()
+         if ibm_path in devices:
+--- a/tests/unittests/test_ds_identify.py
++++ b/tests/unittests/test_ds_identify.py
+@@ -178,17 +178,18 @@
+             data, RC_FOUND, dslist=[data.get('ds'), DS_NONE])
+ 
+     def _check_via_dict(self, data, rc, dslist=None, **kwargs):
+-        found_rc, out, err, cfg, files = self._call_via_dict(data, **kwargs)
++        ret = self._call_via_dict(data, **kwargs)
+         good = False
+         try:
+-            self.assertEqual(rc, found_rc)
++            self.assertEqual(rc, ret.rc)
+             if dslist is not None:
+-                self.assertEqual(dslist, cfg['datasource_list'])
++                self.assertEqual(dslist, ret.cfg['datasource_list'])
+             good = True
+         finally:
+             if not good:
+-                _print_run_output(rc, out, err, cfg, files)
+-        return rc, out, err, cfg, files
++                _print_run_output(ret.rc, ret.stdout, ret.stderr, ret.cfg,
++                                  ret.files)
++        return ret
+ 
+     def test_wb_print_variables(self):
+         """_print_info reports an array of discovered variables to stderr."""
+@@ -237,13 +238,40 @@
+     def test_config_drive(self):
+         """ConfigDrive datasource has a disk with LABEL=config-2."""
+         self._test_ds_found('ConfigDrive')
+-        return
+ 
+     def test_config_drive_upper(self):
+         """ConfigDrive datasource has a disk with LABEL=CONFIG-2."""
+         self._test_ds_found('ConfigDriveUpper')
+         return
+ 
++    def test_config_drive_seed(self):
++        """Config Drive seed directory."""
++        self._test_ds_found('ConfigDrive-seed')
++
++    def test_config_drive_interacts_with_ibmcloud_config_disk(self):
++        """Verify ConfigDrive interaction with IBMCloud.
++
++        If ConfigDrive is enabled and not IBMCloud, then ConfigDrive
++        should claim the ibmcloud 'config-2' disk.
++        If IBMCloud is enabled, then ConfigDrive should skip."""
++        data = copy.deepcopy(VALID_CFG['IBMCloud-config-2'])
++        files = data.get('files', {})
++        if not files:
++            data['files'] = files
++        cfgpath = 'etc/cloud/cloud.cfg.d/99_networklayer_common.cfg'
++
++        # with list including IBMCloud, config drive should be not found.
++        files[cfgpath] = 'datasource_list: [ ConfigDrive, IBMCloud ]\n'
++        ret = self._check_via_dict(data, shell_true)
++        self.assertEqual(
++            ret.cfg.get('datasource_list'), ['IBMCloud', 'None'])
++
++        # But if IBMCloud is not enabled, config drive should claim this.
++        files[cfgpath] = 'datasource_list: [ ConfigDrive, NoCloud ]\n'
++        ret = self._check_via_dict(data, shell_true)
++        self.assertEqual(
++            ret.cfg.get('datasource_list'), ['ConfigDrive', 'None'])
++
+     def test_ibmcloud_template_userdata_in_provisioning(self):
+         """Template provisioned with user-data during provisioning stage.
+ 
+@@ -295,6 +323,37 @@
+         self._check_via_dict(
+             data, rc=RC_FOUND, dslist=['ConfigDrive', DS_NONE])
+ 
++    def test_ibmcloud_with_nocloud_seed(self):
++        """NoCloud seed should be preferred over IBMCloud.
++
++        A nocloud seed should be preferred over IBMCloud even if enabled.
++        Ubuntu 16.04 images have <vlc>/seed/nocloud-net. LP: #1766401."""
++        data = copy.deepcopy(VALID_CFG['IBMCloud-config-2'])
++        files = data.get('files', {})
++        if not files:
++            data['files'] = files
++        files.update(VALID_CFG['NoCloud-seed']['files'])
++        ret = self._check_via_dict(data, shell_true)
++        self.assertEqual(
++            ['NoCloud', 'IBMCloud', 'None'],
++            ret.cfg.get('datasource_list'))
++
++    def test_ibmcloud_with_configdrive_seed(self):
++        """ConfigDrive seed should be preferred over IBMCloud.
++
++        A ConfigDrive seed should be preferred over IBMCloud even if enabled.
++        Ubuntu 16.04 images have a fstab entry that mounts the
++        METADATA disk into <vlc>/seed/config_drive. LP: ##1766401."""
++        data = copy.deepcopy(VALID_CFG['IBMCloud-config-2'])
++        files = data.get('files', {})
++        if not files:
++            data['files'] = files
++        files.update(VALID_CFG['ConfigDrive-seed']['files'])
++        ret = self._check_via_dict(data, shell_true)
++        self.assertEqual(
++            ['ConfigDrive', 'IBMCloud', 'None'],
++            ret.cfg.get('datasource_list'))
++
+     def test_policy_disabled(self):
+         """A Builtin policy of 'disabled' should return not found.
+ 
+@@ -631,6 +690,12 @@
+              },
+         ],
+     },
++    'ConfigDrive-seed': {
++        'ds': 'ConfigDrive',
++        'files': {
++            os.path.join(P_SEED_DIR, 'config_drive', 'openstack',
++                         'latest', 'meta_data.json'): 'md\n'},
++    },
+     'Hetzner': {
+         'ds': 'Hetzner',
+         'files': {P_SYS_VENDOR: 'Hetzner\n'},
+--- a/tools/ds-identify
++++ b/tools/ds-identify
+@@ -600,7 +600,6 @@
+         *\ ds=nocloud*) return ${DS_FOUND};;
+     esac
+ 
+-    is_ibm_cloud && return ${DS_NOT_FOUND}
+     for d in nocloud nocloud-net; do
+         check_seed_dir "$d" meta-data user-data && return ${DS_FOUND}
+         check_writable_seed_dir "$d" meta-data user-data && return ${DS_FOUND}
+@@ -611,11 +610,12 @@
+     return ${DS_NOT_FOUND}
+ }
+ 
++is_ds_enabled() {
++    local name="$1" pad=" ${DI_DSLIST} "
++    [ "${pad#* $name }" != "${pad}" ]
++}
++
+ check_configdrive_v2() {
+-    is_ibm_cloud && return ${DS_NOT_FOUND}
+-    if has_fs_with_label CONFIG-2 config-2; then
+-        return ${DS_FOUND}
+-    fi
+     # look in /config-drive <vlc>/seed/config_drive for a directory
+     # openstack/YYYY-MM-DD format with a file meta_data.json
+     local d=""
+@@ -630,6 +630,13 @@
+         debug 1 "config drive seeded directory had only 'latest'"
+         return ${DS_FOUND}
+     fi
++
++    is_ds_enabled "IBMCloud"
++    debug 1 "is_ds_enabled returned $?: $DI_DSLIST"
++    is_ds_enabled "IBMCloud" && is_ibm_cloud && return ${DS_NOT_FOUND}
++    if has_fs_with_label CONFIG-2 config-2; then
++        return ${DS_FOUND}
++    fi
+     return ${DS_NOT_FOUND}
+ }
+ 
diff --git a/debian/patches/cpick-6ef92c98-IBMCloud-recognize-provisioning-environment-during b/debian/patches/cpick-6ef92c98-IBMCloud-recognize-provisioning-environment-during
new file mode 100644
index 0000000..b428120
--- /dev/null
+++ b/debian/patches/cpick-6ef92c98-IBMCloud-recognize-provisioning-environment-during
@@ -0,0 +1,389 @@
+From 6ef92c98c3d2b127b05d6708337efc8a81e00071 Mon Sep 17 00:00:00 2001
+From: Scott Moser <[e-mail]>
+Date: Thu, 26 Apr 2018 16:24:24 -0500
+Subject: [PATCH] IBMCloud: recognize provisioning environment during debug
+ boots.
+
+When images are deployed from template in a production environment
+the artifacts of the provisioning stage (provisioningConfiguration.cfg)
+that cloud-init referenced are cleaned up.  However, when provisioned
+in "debug" mode (internal to IBM) the artifacts are left.
+
+This changes the 'is_ibm_provisioning' implementations in both
+ds-identify and in the IBM datasource to identify the provisioning
+stage more correctly.  The change is to consider provisioning only
+if the provisioing file existed and there was no log file or
+the log file was older than this boot.
+
+LP: #1767166
+---
+ cloudinit/sources/DataSourceIBMCloud.py          | 42 +++++++++-----
+ cloudinit/tests/helpers.py                       | 13 ++++-
+ tests/unittests/test_datasource/test_ibmcloud.py | 50 ++++++++++++++++
+ tests/unittests/test_ds_identify.py              | 72 +++++++++++++++++++++---
+ tools/ds-identify                                | 21 ++++++-
+ 5 files changed, 175 insertions(+), 23 deletions(-)
+
+--- a/cloudinit/sources/DataSourceIBMCloud.py
++++ b/cloudinit/sources/DataSourceIBMCloud.py
+@@ -8,17 +8,11 @@
+  * template: This is the legacy method of launching instances.
+    When booting from an image template, the system boots first into
+    a "provisioning" mode.  There, host <-> guest mechanisms are utilized
+-   to execute code in the guest and provision it.
++   to execute code in the guest and configure it.  The configuration
++   includes configuring the system network and possibly installing
++   packages and other software stack.
+ 
+-   Cloud-init will disable itself when it detects that it is in the
+-   provisioning mode.  It detects this by the presence of
+-   a file '/root/provisioningConfiguration.cfg'.
+-
+-   When provided with user-data, the "first boot" will contain a
+-   ConfigDrive-like disk labeled with 'METADATA'.  If there is no user-data
+-   provided, then there is no data-source.
+-
+-   Cloud-init never does any network configuration in this mode.
++   After the provisioning is finished, the system reboots.
+ 
+  * os_code: Essentially "launch by OS Code" (Operating System Code).
+    This is a more modern approach.  There is no specific "provisioning" boot.
+@@ -138,8 +132,30 @@
+     return os.path.exists("/proc/xen")
+ 
+ 
+-def _is_ibm_provisioning():
+-    return os.path.exists("/root/provisioningConfiguration.cfg")
++def _is_ibm_provisioning(
++        prov_cfg="/root/provisioningConfiguration.cfg",
++        inst_log="/root/swinstall.log",
++        boot_ref="/proc/1/environ"):
++    """Return boolean indicating if this boot is ibm provisioning boot."""
++    if os.path.exists(prov_cfg):
++        msg = "config '%s' exists." % prov_cfg
++        result = True
++        if os.path.exists(inst_log):
++            if os.path.exists(boot_ref):
++                result = (os.stat(inst_log).st_mtime >
++                          os.stat(boot_ref).st_mtime)
++                msg += (" log '%s' from %s boot." %
++                        (inst_log, "current" if result else "previous"))
++            else:
++                msg += (" log '%s' existed, but no reference file '%s'." %
++                        (inst_log, boot_ref))
++                result = False
++        else:
++            msg += " log '%s' did not exist." % inst_log
++    else:
++        result, msg = (False, "config '%s' did not exist." % prov_cfg)
++    LOG.debug("ibm_provisioning=%s: %s", result, msg)
++    return result
+ 
+ 
+ def get_ibm_platform():
+@@ -189,7 +205,7 @@
+         else:
+             return (Platforms.TEMPLATE_LIVE_METADATA, metadata_path)
+     elif _is_ibm_provisioning():
+-            return (Platforms.TEMPLATE_PROVISIONING_NODATA, None)
++        return (Platforms.TEMPLATE_PROVISIONING_NODATA, None)
+     return not_found
+ 
+ 
+--- a/cloudinit/tests/helpers.py
++++ b/cloudinit/tests/helpers.py
+@@ -8,6 +8,7 @@
+ import shutil
+ import sys
+ import tempfile
++import time
+ import unittest
+ 
+ import mock
+@@ -285,7 +286,8 @@
+             os.path: [('isfile', 1), ('exists', 1),
+                       ('islink', 1), ('isdir', 1), ('lexists', 1)],
+             os: [('listdir', 1), ('mkdir', 1),
+-                 ('lstat', 1), ('symlink', 2)]
++                 ('lstat', 1), ('symlink', 2),
++                 ('stat', 1)]
+         }
+ 
+         if hasattr(os, 'scandir'):
+@@ -354,6 +356,15 @@
+     return ret
+ 
+ 
++def populate_dir_with_ts(path, data):
++    """data is {'file': ('contents', mtime)}.  mtime relative to now."""
++    populate_dir(path, dict((k, v[0]) for k, v in data.items()))
++    btime = time.time()
++    for fpath, (_contents, mtime) in data.items():
++        ts = btime + mtime if mtime else btime
++        os.utime(os.path.sep.join((path, fpath)), (ts, ts))
++
++
+ def dir2dict(startdir, prefix=None):
+     flist = {}
+     if prefix is None:
+--- a/tests/unittests/test_datasource/test_ibmcloud.py
++++ b/tests/unittests/test_datasource/test_ibmcloud.py
+@@ -259,4 +259,54 @@
+                          ret['metadata'])
+ 
+ 
++class TestIsIBMProvisioning(test_helpers.FilesystemMockingTestCase):
++    """Test the _is_ibm_provisioning method."""
++    inst_log = "/root/swinstall.log"
++    prov_cfg = "/root/provisioningConfiguration.cfg"
++    boot_ref = "/proc/1/environ"
++    with_logs = True
++
++    def _call_with_root(self, rootd):
++        self.reRoot(rootd)
++        return ibm._is_ibm_provisioning()
++
++    def test_no_config(self):
++        """No provisioning config means not provisioning."""
++        self.assertFalse(self._call_with_root(self.tmp_dir()))
++
++    def test_config_only(self):
++        """A provisioning config without a log means provisioning."""
++        rootd = self.tmp_dir()
++        test_helpers.populate_dir(rootd, {self.prov_cfg: "key=value"})
++        self.assertTrue(self._call_with_root(rootd))
++
++    def test_config_with_old_log(self):
++        """A config with a log from previous boot is not provisioning."""
++        rootd = self.tmp_dir()
++        data = {self.prov_cfg: ("key=value\nkey2=val2\n", -10),
++                self.inst_log: ("log data\n", -30),
++                self.boot_ref: ("PWD=/", 0)}
++        test_helpers.populate_dir_with_ts(rootd, data)
++        self.assertFalse(self._call_with_root(rootd=rootd))
++        self.assertIn("from previous boot", self.logs.getvalue())
++
++    def test_config_with_new_log(self):
++        """A config with a log from this boot is provisioning."""
++        rootd = self.tmp_dir()
++        data = {self.prov_cfg: ("key=value\nkey2=val2\n", -10),
++                self.inst_log: ("log data\n", 30),
++                self.boot_ref: ("PWD=/", 0)}
++        test_helpers.populate_dir_with_ts(rootd, data)
++        self.assertTrue(self._call_with_root(rootd=rootd))
++        self.assertIn("from current boot", self.logs.getvalue())
++
++    def test_config_and_log_no_reference(self):
++        """If the config and log existed, but no reference, assume not."""
++        rootd = self.tmp_dir()
++        test_helpers.populate_dir(
++            rootd, {self.prov_cfg: "key=value", self.inst_log: "log data\n"})
++        self.assertFalse(self._call_with_root(rootd=rootd))
++        self.assertIn("no reference file", self.logs.getvalue())
++
++
+ # vi: ts=4 expandtab
+--- a/tests/unittests/test_ds_identify.py
++++ b/tests/unittests/test_ds_identify.py
+@@ -1,5 +1,6 @@
+ # This file is part of cloud-init. See LICENSE file for license information.
+ 
++from collections import namedtuple
+ import copy
+ import os
+ from uuid import uuid4
+@@ -7,7 +8,7 @@
+ from cloudinit import safeyaml
+ from cloudinit import util
+ from cloudinit.tests.helpers import (
+-    CiTestCase, dir2dict, populate_dir)
++    CiTestCase, dir2dict, populate_dir, populate_dir_with_ts)
+ 
+ from cloudinit.sources import DataSourceIBMCloud as dsibm
+ 
+@@ -66,7 +67,6 @@
+ P_SEED_DIR = "var/lib/cloud/seed"
+ P_DSID_CFG = "etc/cloud/ds-identify.cfg"
+ 
+-IBM_PROVISIONING_CHECK_PATH = "/root/provisioningConfiguration.cfg"
+ IBM_CONFIG_UUID = "9796-932E"
+ 
+ MOCK_VIRT_IS_KVM = {'name': 'detect_virt', 'RET': 'kvm', 'ret': 0}
+@@ -74,11 +74,17 @@
+ MOCK_VIRT_IS_XEN = {'name': 'detect_virt', 'RET': 'xen', 'ret': 0}
+ MOCK_UNAME_IS_PPC64 = {'name': 'uname', 'out': UNAME_PPC64EL, 'ret': 0}
+ 
++shell_true = 0
++shell_false = 1
+ 
+-class TestDsIdentify(CiTestCase):
++CallReturn = namedtuple('CallReturn',
++                        ['rc', 'stdout', 'stderr', 'cfg', 'files'])
++
++
++class DsIdentifyBase(CiTestCase):
+     dsid_path = os.path.realpath('tools/ds-identify')
+ 
+-    def call(self, rootd=None, mocks=None, args=None, files=None,
++    def call(self, rootd=None, mocks=None, func="main", args=None, files=None,
+              policy_dmi=DI_DEFAULT_POLICY,
+              policy_no_dmi=DI_DEFAULT_POLICY_NO_DMI,
+              ec2_strict_id=DI_EC2_STRICT_ID_DEFAULT):
+@@ -135,7 +141,7 @@
+                 mocklines.append(write_mock(d))
+ 
+         endlines = [
+-            'main %s' % ' '.join(['"%s"' % s for s in args])
++            func + ' ' + ' '.join(['"%s"' % s for s in args])
+         ]
+ 
+         with open(wrap, "w") as fp:
+@@ -159,7 +165,7 @@
+                 cfg = {"_INVALID_YAML": contents,
+                        "_EXCEPTION": str(e)}
+ 
+-        return rc, out, err, cfg, dir2dict(rootd)
++        return CallReturn(rc, out, err, cfg, dir2dict(rootd))
+ 
+     def _call_via_dict(self, data, rootd=None, **kwargs):
+         # return output of self.call with a dict input like VALID_CFG[item]
+@@ -191,6 +197,8 @@
+                                   ret.files)
+         return ret
+ 
++
++class TestDsIdentify(DsIdentifyBase):
+     def test_wb_print_variables(self):
+         """_print_info reports an array of discovered variables to stderr."""
+         data = VALID_CFG['Azure-dmi-detection']
+@@ -278,7 +286,10 @@
+         Template provisioning with user-data has METADATA disk,
+         datasource should return not found."""
+         data = copy.deepcopy(VALID_CFG['IBMCloud-metadata'])
+-        data['files'] = {IBM_PROVISIONING_CHECK_PATH: 'xxx'}
++        # change the 'is_ibm_provisioning' mock to return 1 (false)
++        isprov_m = [m for m in data['mocks']
++                    if m["name"] == "is_ibm_provisioning"][0]
++        isprov_m['ret'] = shell_true
+         return self._check_via_dict(data, RC_NOT_FOUND)
+ 
+     def test_ibmcloud_template_userdata(self):
+@@ -293,7 +304,8 @@
+ 
+         no disks attached.  Datasource should return not found."""
+         data = copy.deepcopy(VALID_CFG['IBMCloud-nodisks'])
+-        data['files'] = {IBM_PROVISIONING_CHECK_PATH: 'xxx'}
++        data['mocks'].append(
++            {'name': 'is_ibm_provisioning', 'ret': shell_true})
+         return self._check_via_dict(data, RC_NOT_FOUND)
+ 
+     def test_ibmcloud_template_no_userdata(self):
+@@ -505,6 +517,47 @@
+         self._test_ds_found('Hetzner')
+ 
+ 
++class TestIsIBMProvisioning(DsIdentifyBase):
++    """Test the is_ibm_provisioning method in ds-identify."""
++
++    inst_log = "/root/swinstall.log"
++    prov_cfg = "/root/provisioningConfiguration.cfg"
++    boot_ref = "/proc/1/environ"
++    funcname = "is_ibm_provisioning"
++
++    def test_no_config(self):
++        """No provisioning config means not provisioning."""
++        ret = self.call(files={}, func=self.funcname)
++        self.assertEqual(shell_false, ret.rc)
++
++    def test_config_only(self):
++        """A provisioning config without a log means provisioning."""
++        ret = self.call(files={self.prov_cfg: "key=value"}, func=self.funcname)
++        self.assertEqual(shell_true, ret.rc)
++
++    def test_config_with_old_log(self):
++        """A config with a log from previous boot is not provisioning."""
++        rootd = self.tmp_dir()
++        data = {self.prov_cfg: ("key=value\nkey2=val2\n", -10),
++                self.inst_log: ("log data\n", -30),
++                self.boot_ref: ("PWD=/", 0)}
++        populate_dir_with_ts(rootd, data)
++        ret = self.call(rootd=rootd, func=self.funcname)
++        self.assertEqual(shell_false, ret.rc)
++        self.assertIn("from previous boot", ret.stderr)
++
++    def test_config_with_new_log(self):
++        """A config with a log from this boot is provisioning."""
++        rootd = self.tmp_dir()
++        data = {self.prov_cfg: ("key=value\nkey2=val2\n", -10),
++                self.inst_log: ("log data\n", 30),
++                self.boot_ref: ("PWD=/", 0)}
++        populate_dir_with_ts(rootd, data)
++        ret = self.call(rootd=rootd, func=self.funcname)
++        self.assertEqual(shell_true, ret.rc)
++        self.assertIn("from current boot", ret.stderr)
++
++
+ def blkid_out(disks=None):
+     """Convert a list of disk dictionaries into blkid content."""
+     if disks is None:
+@@ -704,6 +757,7 @@
+         'ds': 'IBMCloud',
+         'mocks': [
+             MOCK_VIRT_IS_XEN,
++            {'name': 'is_ibm_provisioning', 'ret': shell_false},
+             {'name': 'blkid', 'ret': 0,
+              'out': blkid_out(
+                  [{'DEVNAME': 'xvda1', 'TYPE': 'vfat', 'PARTUUID': uuid4()},
+@@ -717,6 +771,7 @@
+         'ds': 'IBMCloud',
+         'mocks': [
+             MOCK_VIRT_IS_XEN,
++            {'name': 'is_ibm_provisioning', 'ret': shell_false},
+             {'name': 'blkid', 'ret': 0,
+              'out': blkid_out(
+                  [{'DEVNAME': 'xvda1', 'TYPE': 'ext3', 'PARTUUID': uuid4(),
+@@ -734,6 +789,7 @@
+         'ds': 'IBMCloud',
+         'mocks': [
+             MOCK_VIRT_IS_XEN,
++            {'name': 'is_ibm_provisioning', 'ret': shell_false},
+             {'name': 'blkid', 'ret': 0,
+              'out': blkid_out(
+                  [{'DEVNAME': 'xvda1', 'TYPE': 'vfat', 'PARTUUID': uuid4()},
+--- a/tools/ds-identify
++++ b/tools/ds-identify
+@@ -125,6 +125,7 @@
+ DI_EC2_STRICT_ID_DEFAULT="true"
+ 
+ _IS_IBM_CLOUD=""
++_IS_IBM_PROVISIONING=""
+ 
+ error() {
+     set -- "ERROR:" "$@";
+@@ -1013,7 +1014,25 @@
+ }
+ 
+ is_ibm_provisioning() {
+-    [ -f "${PATH_ROOT}/root/provisioningConfiguration.cfg" ]
++    local pcfg="${PATH_ROOT}/root/provisioningConfiguration.cfg"
++    local logf="${PATH_ROOT}/root/swinstall.log"
++    local is_prov=false msg="config '$pcfg' did not exist."
++    if [ -f "$pcfg" ]; then
++        msg="config '$pcfg' exists."
++        is_prov=true
++        if [ -f "$logf" ]; then
++            if [ "$logf" -nt "$PATH_PROC_1_ENVIRON" ]; then
++                msg="$msg log '$logf' from current boot."
++            else
++                is_prov=false
++                msg="$msg log '$logf' from previous boot."
++            fi
++        else
++            msg="$msg log '$logf' did not exist."
++        fi
++    fi
++    debug 2 "ibm_provisioning=$is_prov: $msg"
++    [ "$is_prov" = "true" ]
+ }
+ 
+ is_ibm_cloud() {
diff --git a/debian/patches/series b/debian/patches/series
new file mode 100644
index 0000000..2f90183
--- /dev/null
+++ b/debian/patches/series
@@ -0,0 +1,2 @@
+cpick-11172924-IBMCloud-Disable-config-drive-and-nocloud-only-if
+cpick-6ef92c98-IBMCloud-recognize-provisioning-environment-during