Merge ~dojordan/cloud-init:azure-preprovisioning into cloud-init:master

[Douglas Jordan]

Azure VM Preprovisioning support.

This change will enable azure vms to report provisioning has completed twice, first to tell the fabric it has completed then a second time to enable customer settings. The datasource for the second provisioning is the Instance Metadata Service (IMDS), and the VM will poll indefinitely for the new ovf-env.xml from IMDS.

LP: #1734991

Azure VM Preprovisioning support.

This change will enable azure vms to report provisioning has completed twice, first to tell the fabric it has completed then a second time to enable customer settings. The datasource for the second provisioning is the Instance Metadata Service (IMDS), and the VM will poll indefinitely for the new ovf-env.xml from IMDS.

LP: 1734991

[Paul Meyer (paul-meyer)]

Looks good to me (after you fix the tests and flake8's)

[Scott Moser (smoser)]

Fix your commit message (press 'Set commit message' above).

 Summary
 <blank>
 More info

[Server Team CI bot (server-team-bot)]

FAILED: Continuous integration, rev:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/553/
Executed test runs:
    FAILED: Checkout

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/553/rebuild

review: Needs Fixing (continuous-integration)

[Douglas Jordan (dojordan)]

> FAILED: Continuous integration, rev:
> https://jenkins.ubuntu.com/server/job/cloud-init-ci/553/
> Executed test runs:
> FAILED: Checkout
>
> Click here to trigger a rebuild:
> https://jenkins.ubuntu.com/server/job/cloud-init-ci/553/rebuild

Looks like the git checkout failed. Thoughts?
stderr: remote: Authorisation required.
fatal: Authentication failed for 'https://git.launchpad.net/~dojordan/cloud-init:azure-preprovisioning/'

[Server Team CI bot (server-team-bot)]

FAILED: Continuous integration, rev:72e423b4a0827dd954170749812b13aba761f399
https://jenkins.ubuntu.com/server/job/cloud-init-ci/562/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    FAILED: MAAS Compatability Testing

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/562/rebuild

review: Needs Fixing (continuous-integration)

[Server Team CI bot (server-team-bot)]

FAILED: Continuous integration, rev:384b7731e99338903d91da641267ed84a7470669
https://jenkins.ubuntu.com/server/job/cloud-init-ci/567/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    FAILED: MAAS Compatability Testing

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/567/rebuild

review: Needs Fixing (continuous-integration)

[Server Team CI bot (server-team-bot)]

FAILED: Continuous integration, rev:d00ec2ec82271c3c35830a04d2f216cd7bef8ba7
https://jenkins.ubuntu.com/server/job/cloud-init-ci/575/
Executed test runs:
    SUCCESS: Checkout
    FAILED: Unit & Style Tests

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/575/rebuild

review: Needs Fixing (continuous-integration)

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:6021da2b61a55102ec7637d6fea54e02db1c1c92
https://jenkins.ubuntu.com/server/job/cloud-init-ci/576/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    SUCCESS: MAAS Compatability Testing
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/576/rebuild

review: Approve (continuous-integration)

[Paul Meyer (paul-meyer)]

review: Approve

[Douglas Jordan (dojordan)]

Thanks for the feedback. I've resolved your comments.

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:ff0b7b0b5fdd1512c7ec2ccbee0f92afde5c733d
https://jenkins.ubuntu.com/server/job/cloud-init-ci/579/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    SUCCESS: MAAS Compatability Testing
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/579/rebuild

review: Approve (continuous-integration)

[Chad Smith (chad.smith)]

Douglas, thanks for working this. I'm going to mark it work in progress as you address or respond to comments. When you'd like another review pass please just mark it back to "Needs review"

[Server Team CI bot (server-team-bot)]

FAILED: Continuous integration, rev:10c6b219ba7168c5fb3751d28a383b130a1348ae
https://jenkins.ubuntu.com/server/job/cloud-init-ci/590/
Executed test runs:
    SUCCESS: Checkout
    FAILED: Unit & Style Tests

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/590/rebuild

review: Needs Fixing (continuous-integration)

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:e06a762741953a5c55d843b0dcfd454111655cea
https://jenkins.ubuntu.com/server/job/cloud-init-ci/595/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    SUCCESS: MAAS Compatability Testing
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/595/rebuild

review: Approve (continuous-integration)

[Chad Smith (chad.smith)]

Thanks for the rework and comments. One more pass on this. I'll test against stock azure xenial today and report here.

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:c500a0184b8bc7c7ba03b1a884896702a761959f
https://jenkins.ubuntu.com/server/job/cloud-init-ci/612/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    SUCCESS: MAAS Compatability Testing
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/612/rebuild

review: Approve (continuous-integration)

[Chad Smith (chad.smith)]

Ok one last nit from me and it looks good. I need a bit of testing still on this but content looks good.

[Chad Smith (chad.smith)]

one last try on the nit

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:3c9509671f05d8dfc990b5ce35a615c9cd7b442c
https://jenkins.ubuntu.com/server/job/cloud-init-ci/618/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    SUCCESS: MAAS Compatability Testing
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/618/rebuild

review: Approve (continuous-integration)

[Chad Smith (chad.smith)]

review: Approve

[Scott Moser (smoser)]

There are some comments in line.
I'm not sure I fully understand all of this.

I could be wrong here, but I think you're using
bounce_network_with_azure_hostname in order to interact with IMDS_URL.

We have 2 ways now of brining up ephemeral networking for this purpose:
 a.) Chad has recently added code in the EC2 metadata service using cloudinit/net/dhcp.py .
 b.) the digital ocean datasource has code to negotiate a ipv4 link local address.

If 'b' is sufficient, I'd prefer that, but either one I'd prefer to the bounce_network_ which i think may actually not work for you if you rebased to trunk.

As mentioned in IRC, I'm still concerned about systemd giving up and deciding that boot has failed after some amount of time polling on a metadata service. As Douglas pointed out, cloud-init has timeouts set to 0 and is a 'oneshot', so *its* timeout is not an issue, but I think that things that it runs 'Before' or (pre-networking or other) might end up timing out.

[Scott Moser (smoser)]

some things there need fixing, definitely need rebase to trunk (I suspect you'll have conflicts), but if not, some thought is needed.

review: Needs Fixing

fd33d0a... by Douglas Jordan on 2018-01-04

Merge branch 'master' into azure-preprovisioning

8b89d2d... by Douglas Jordan on 2018-01-04

Small tweaks.

c207d44... by Douglas Jordan on 2018-01-05

Addressing CR comments

[Server Team CI bot (server-team-bot)]

FAILED: Continuous integration, rev:c207d443ca360f192509b33dacd404cd0a4d3bc5
https://jenkins.ubuntu.com/server/job/cloud-init-ci/662/
Executed test runs:
    SUCCESS: Checkout
    FAILED: Unit & Style Tests

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/662/rebuild

review: Needs Fixing (continuous-integration)

ba329ca... by Douglas Jordan on 2018-01-05

Using warning instead of warn

[Server Team CI bot (server-team-bot)]

FAILED: Continuous integration, rev:ba329ca5cde7f64b1f04ff3a600f3424e8c04515
https://jenkins.ubuntu.com/server/job/cloud-init-ci/669/
Executed test runs:
    SUCCESS: Checkout
    FAILED: Unit & Style Tests

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/669/rebuild

review: Needs Fixing (continuous-integration)

93d88e2... by Douglas Jordan on 2018-01-05

Fixing flake8

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:93d88e25a1db5220cf087bba80a1ff15dd7f7fd8
https://jenkins.ubuntu.com/server/job/cloud-init-ci/670/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    SUCCESS: MAAS Compatability Testing
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/670/rebuild

review: Approve (continuous-integration)

[Douglas Jordan (dojordan)]

Thanks for the comments Scott-

The reason for the bounce_network_with_azure_hostname is actually to get the new IP address. We can't use the digital ocean ipv4 link local approach as the way our IMDS server identifies which VM is talking to it is via the mac address and the ip address. When the control plane updates the DHCP server with the new IP, the vm has to trigger dhcp and acquire the new address. In our polling loop, we call bounce_networking_with_azure_hostname to trigger dhcp when we get an exception. This is because IMDS will not even handle our request if the mac/ip does not match what was expected. In windows, we get around this by just disconnecting and reconnecting the NIC, however on Ubuntu 16.04, we observed the link had to be disconnected for a very long time (>10s) for this behavior to occur. Hopefully, with systemd-networkd, this issue will be fixed. For now however, we are targeting 16.04 for pre=provisioning, and we will continue to test artful/bionic with the link state disconnect / reconnect approach.

> There are some comments in line.
> I'm not sure I fully understand all of this.
>
> I could be wrong here, but I think you're using
> bounce_network_with_azure_hostname in order to interact with IMDS_URL.
>
> We have 2 ways now of brining up ephemeral networking for this purpose:
> a.) Chad has recently added code in the EC2 metadata service using
> cloudinit/net/dhcp.py .
> b.) the digital ocean datasource has code to negotiate a ipv4 link local
> address.
>
> If 'b' is sufficient, I'd prefer that, but either one I'd prefer to the
> bounce_network_ which i think may actually not work for you if you rebased to
> trunk.
>
> As mentioned in IRC, I'm still concerned about systemd giving up and deciding
> that boot has failed after some amount of time polling on a metadata service.
> As Douglas pointed out, cloud-init has timeouts set to 0 and is a 'oneshot',
> so *its* timeout is not an issue, but I think that things that it runs
> 'Before' or (pre-networking or other) might end up timing out.

[sushant (sushantsharma)]

<not sure why the inline comment does not show up>

Hi Douglas, Can you please add a comment before line 83 (where you catch exception and re-DHCP) saying that this is temporary. We plan to add a networking module specific to azure in cloud-init and will address re-DHCP need in that module. The plan is to submit that module for review in a week or so. Thanks!

fc01540... by Douglas Jordan on 2018-01-08

Adding comments

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:fc0154011d4a3f0418fa71cfa5330bd9ac837dfd
https://jenkins.ubuntu.com/server/job/cloud-init-ci/679/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    SUCCESS: MAAS Compatability Testing
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/679/rebuild

review: Approve (continuous-integration)

[Scott Moser (smoser)]

As it is right now, 'bounce_network_with_azure_hostname' is a no-op
on Ubuntu 18.04 (bionic) or any system without 'ifup'.
So while you want this fixed in 16.04, the general Ubuntu development
path insists that we make things work on the development release first
and the SRU them back to 16.04.

In summary: I can't let this in as broken on bionic. We'll need to find
a way to make that work on 18.04.

Your commit message says:
 | This change will enable azure vms to report provisioning has completed
 | twice, first to tell the fabric it has completed then a second time to
 | enable customer settings.

Is that true, will it *always* report reprovisioning has completed twice?

there are some small inlinel things also.

those are the big things though. thanks for your work.

[Douglas Jordan (dojordan)]

Regarding the DHCP stuff: We are exploring an alternate solution to bounce the nic from hyper-v, but in the mean time we would like to get this checked in. So an alternate solution for bionic would be to simply change the hostname. This way, systemd-networkd will keep re triggering DHCP. Once we get the final ovf_env.xml from IMDS, we will actually apply the real, customer provided hostname.

Regarding "Is that true, will it *always* report reprovisioning has completed twice?"-
tldr; yes. Technically, it will report "provisioning" has completed twice, while we are calling the second incarnation "reprovisioning"

7f23e5c... by Douglas Jordan 1 hour ago

Changing hostname post xenial, and other PR comments

Author: Douglas Jordan <email address hidden>
Committer: Douglas Jordan <email address hidden>

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:7f23e5c4808a9c647cd4d5277625a723a58b132b
https://jenkins.ubuntu.com/server/job/cloud-init-ci/685/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    SUCCESS: MAAS Compatability Testing
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/685/rebuild

review: Approve (continuous-integration)

Unmerged commits

7f23e5c... by Douglas Jordan 1 hour ago

Changing hostname post xenial, and other PR comments

Author: Douglas Jordan <email address hidden>
Committer: Douglas Jordan <email address hidden>

fc01540... by Douglas Jordan on 2018-01-08

Adding comments

93d88e2... by Douglas Jordan on 2018-01-05

Fixing flake8

ba329ca... by Douglas Jordan on 2018-01-05

Using warning instead of warn

c207d44... by Douglas Jordan on 2018-01-05

Addressing CR comments

8b89d2d... by Douglas Jordan on 2018-01-04

Small tweaks.

fd33d0a... by Douglas Jordan on 2018-01-04

Merge branch 'master' into azure-preprovisioning

3c95096... by Douglas Jordan on 2017-12-11

nit fixes

37f9ff8... by Douglas Jordan on 2017-12-11

call extract method directly.

c500a01... by Douglas Jordan on 2017-12-11

Flake8 fixes

diff --git a/.gitignore b/.gitignore
index b0500a6..75565ed 100644
--- a/.gitignore
+++ b/.gitignore
@@ -10,3 +10,4 @@ parts
 prime
 stage
 *.snap
+*.cover
diff --git a/cloudinit/sources/DataSourceAzure.py b/cloudinit/sources/DataSourceAzure.py
index d1d0975..0bcbc99 100644
--- a/cloudinit/sources/DataSourceAzure.py
+++ b/cloudinit/sources/DataSourceAzure.py
@@ -11,6 +11,8 @@ from functools import partial
 import os
 import os.path
 import re
+from time import time
+from uuid import uuid4
 from xml.dom import minidom
 import xml.etree.ElementTree as ET
 
@@ -18,6 +20,7 @@ from cloudinit import log as logging
 from cloudinit import net
 from cloudinit import sources
 from cloudinit.sources.helpers.azure import get_metadata_from_fabric
+from cloudinit.url_helper import readurl, wait_for_url, UrlError
 from cloudinit import util
 
 LOG = logging.getLogger(__name__)
@@ -44,6 +47,8 @@ LEASE_FILE = '/var/lib/dhcp/dhclient.eth0.leases'
 DEFAULT_FS = 'ext4'
 # DMI chassis-asset-tag is set static for all azure instances
 AZURE_CHASSIS_ASSET_TAG = '7783-7084-3265-9085-8269-3286-77'
+REPROVISION_MARKER_FILE = "/var/lib/cloud/data/poll_imds"
+IMDS_URL = "http://169.254.169.254/metadata/reprovisiondata"
 
 
 def find_storvscid_from_sysctl_pnpinfo(sysctl_out, deviceid):
@@ -276,19 +281,20 @@ class DataSourceAzure(sources.DataSource):
 
         with temporary_hostname(azure_hostname, self.ds_cfg,
                                 hostname_command=hostname_command) \
-                as previous_hostname:
-            if (previous_hostname is not None and
+                as previous_hn:
+            if (previous_hn is not None and
                     util.is_true(self.ds_cfg.get('set_hostname'))):
                 cfg = self.ds_cfg['hostname_bounce']
 
                 # "Bouncing" the network
                 try:
-                    perform_hostname_bounce(hostname=azure_hostname,
-                                            cfg=cfg,
-                                            prev_hostname=previous_hostname)
+                    return perform_hostname_bounce(hostname=azure_hostname,
+                                                   cfg=cfg,
+                                                   prev_hostname=previous_hn)
                 except Exception as e:
                     LOG.warning("Failed publishing hostname: %s", e)
                     util.logexc(LOG, "handling set_hostname failed")
+        return False
 
     def get_metadata_from_agent(self):
         temp_hostname = self.metadata.get('local-hostname')
@@ -345,15 +351,20 @@ class DataSourceAzure(sources.DataSource):
         ddir = self.ds_cfg['data_dir']
 
         candidates = [self.seed_dir]
+        if os.path.isfile(REPROVISION_MARKER_FILE):
+            candidates.insert(0, "IMDS")
         candidates.extend(list_possible_azure_ds_devs())
         if ddir:
             candidates.append(ddir)
 
         found = None
-
+        reprovision = False
         for cdev in candidates:
             try:
-                if cdev.startswith("/dev/"):
+                if cdev == "IMDS":
+                    ret = None
+                    reprovision = True
+                elif cdev.startswith("/dev/"):
                     if util.is_FreeBSD():
                         ret = util.mount_cb(cdev, load_azure_ds_dir,
                                             mtype="udf", sync=False)
@@ -370,6 +381,8 @@ class DataSourceAzure(sources.DataSource):
                 LOG.warning("%s was not mountable", cdev)
                 continue
 
+            if reprovision or self._should_reprovision(ret):
+                ret = self._reprovision()
             (md, self.userdata_raw, cfg, files) = ret
             self.seed = cdev
             self.metadata = util.mergemanydict([md, DEFAULT_METADATA])
@@ -428,6 +441,83 @@ class DataSourceAzure(sources.DataSource):
             LOG.debug("negotiating already done for %s",
                       self.get_instance_id())
 
+    def _poll_imds(self):
+        """Poll IMDS for the new provisioning data until we get a valid
+        response. Then returned the returned JSON object."""
+        url = IMDS_URL + "?api-version=2017-04-02"
+        headers = {"Metadata": "true"}
+        LOG.debug("Start polling IMDS")
+
+        def exception_cb(msg, exception):
+            if isinstance(exception, UrlError) and exception.code == 404:
+                return
+            LOG.warning("Exception during polling. Will try DHCP.",
+                        exc_info=True)
+            # On certain versions of the networking stack, the link
+            # state disconnect is not long enough to retrigger DHCP.
+            # If we get an exception while trying to call IMDS, we
+            # bounce the network to force DHCP to acquire the new IP.
+            if not self.bounce_network_with_azure_hostname():
+                # On ubuntu releases > xenial, we rely on systemd-networkd to
+                # retrigger DHCP when the hostname changes as the ifup/down
+                # commands are missing. In that case, we will simply change
+                # the hostname. We can do this as the real hostname will still
+                # be applied after receiving the ovf_env.xml.
+                hn_cmd = self.ds_cfg['hostname_bounce']['hostname_command']
+                set_hostname(str(uuid4()), hn_cmd)
+
+        wait_for_url([url], max_wait=None, timeout=60, status_cb=LOG.info,
+                     headers_cb=lambda url: headers, sleep_time=1,
+                     exception_cb=exception_cb)
+        return str(readurl(url, headers=headers))
+
+    def _report_ready(self):
+        """Tells the fabric provisioning has completed
+           before we go into our polling loop."""
+        try:
+            get_metadata_from_fabric(self.dhclient_lease_file)
+        except Exception as exc:
+            LOG.warning(
+                "Error communicating with Azure fabric; You may experience."
+                "connectivity issues.", exc_info=True)
+
+    def _should_reprovision(self, ret):
+        """Whether or not we should poll IMDS for reprovisioning data.
+        Also sets a marker file to poll IMDS.
+
+        The marker file is used for the following scenario: the VM boots into
+        this polling loop, which we expect to be proceeding infinitely until
+        the VM is picked. If for whatever reason the platform moves us to a
+        new host (for instance a hardware issue), we need to keep polling.
+        However, since the VM reports ready to the Fabric, we will not attach
+        the ISO, thus cloud-init needs to have a way of knowing that it should
+        jump back into the polling loop in order to retrieve the ovf_env."""
+        if not ret:
+            return False
+        (md, self.userdata_raw, cfg, files) = ret
+        path = REPROVISION_MARKER_FILE
+        if (cfg.get('PreprovisionedVm') is True or
+                os.path.isfile(path)):
+            if not os.path.isfile(path):
+                LOG.info("Creating a marker file to poll imds")
+                util.write_file(path, "%s: %s\n" % (os.getpid(), time()))
+            return True
+        return False
+
+    def _reprovision(self):
+        """Initiate the reprovisioning workflow."""
+        LOG.info("bouncing network to enable IMDS polling")
+        if self.metadata is None:
+            self.metadata = {}
+        self.metadata['local-hostname'] = 'azurevm'
+        # This is needed in order to report our temp hostname to the platform
+        # and trigger DHCP in order to get an IP address.
+        self.bounce_network_with_azure_hostname()
+        self._report_ready()
+        contents = self._poll_imds()
+        md, ud, cfg = read_azure_ovf(contents)
+        return (md, ud, cfg, {'ovf-env.xml': contents})
+
     def _negotiate(self):
         """Negotiate with fabric and return data from it.
 
@@ -453,7 +543,7 @@ class DataSourceAzure(sources.DataSource):
                 "Error communicating with Azure fabric; You may experience."
                 "connectivity issues.", exc_info=True)
             return False
-
+        util.del_file(REPROVISION_MARKER_FILE)
         return fabric_data
 
     def activate(self, cfg, is_new_instance):
@@ -595,6 +685,7 @@ def address_ephemeral_resize(devpath=RESOURCE_DISK_PATH, maxwait=120,
 def perform_hostname_bounce(hostname, cfg, prev_hostname):
     # set the hostname to 'hostname' if it is not already set to that.
     # then, if policy is not off, bounce the interface using command
+    # Returns True if the network was bounced, False otherwise.
     command = cfg['command']
     interface = cfg['interface']
     policy = cfg['policy']
@@ -614,7 +705,8 @@ def perform_hostname_bounce(hostname, cfg, prev_hostname):
         else:
             LOG.debug(
                 "Skipping network bounce: ifupdown utils aren't present.")
-            return  # Don't bounce as networkd handles hostname DDNS updates
+            # Don't bounce as networkd handles hostname DDNS updates
+            return False
     LOG.debug("pubhname: publishing hostname [%s]", msg)
     shell = not isinstance(command, (list, tuple))
     # capture=False, see comments in bug 1202758 and bug 1206164.
@@ -622,6 +714,7 @@ def perform_hostname_bounce(hostname, cfg, prev_hostname):
                   get_uptime=True, func=util.subp,
                   kwargs={'args': command, 'shell': shell, 'capture': False,
                           'env': env})
+    return True
 
 
 def crtfile_to_pubkey(fname, data=None):
@@ -838,9 +931,35 @@ def read_azure_ovf(contents):
     if 'ssh_pwauth' not in cfg and password:
         cfg['ssh_pwauth'] = True
 
+    cfg['PreprovisionedVm'] = _extract_preprovisioned_vm_setting(dom)
+
     return (md, ud, cfg)
 
 
+def _extract_preprovisioned_vm_setting(dom):
+    """Read the preprovision flag from the ovf. It should not
+       exist unless true."""
+    platform_settings_section = find_child(
+        dom.documentElement,
+        lambda n: n.localName == "PlatformSettingsSection")
+    if not platform_settings_section or len(platform_settings_section) == 0:
+        LOG.debug("PlatformSettingsSection not found")
+        return False
+    platform_settings = find_child(
+        platform_settings_section[0],
+        lambda n: n.localName == "PlatformSettings")
+    if not platform_settings or len(platform_settings) == 0:
+        LOG.debug("PlatformSettings not found")
+        return False
+    preprovisionedVm = find_child(
+        platform_settings[0],
+        lambda n: n.localName == "PreprovisionedVm")
+    if not preprovisionedVm or len(preprovisionedVm) == 0:
+        LOG.debug("PreprovisionedVm not found")
+        return False
+    return bool(preprovisionedVm[0])
+
+
 def encrypt_pass(password, salt_id="$6$"):
     return crypt.crypt(password, salt_id + util.rand_str(strlen=16))
 
diff --git a/cloudinit/url_helper.py b/cloudinit/url_helper.py
index 0e0f5b4..281c87c 100644
--- a/cloudinit/url_helper.py
+++ b/cloudinit/url_helper.py
@@ -301,6 +301,8 @@ def wait_for_url(urls, max_wait=None, timeout=None,
     service but is not going to find one.  It is possible that the instance
     data host (169.254.169.254) may be firewalled off Entirely for a sytem,
     meaning that the connection will block forever unless a timeout is set.
+
+    A value of None for max_wait will retry indefinitely.
     """
     start_time = time.time()
 
@@ -311,8 +313,9 @@ def wait_for_url(urls, max_wait=None, timeout=None,
         status_cb = log_status_cb
 
     def timeup(max_wait, start_time):
-        return ((max_wait <= 0 or max_wait is None) or
-                (time.time() - start_time > max_wait))
+        if (max_wait is None):
+            return False
+        return ((max_wait <= 0) or (time.time() - start_time > max_wait))
 
     loop_n = 0
     while True:
@@ -322,7 +325,8 @@ def wait_for_url(urls, max_wait=None, timeout=None,
             if loop_n != 0:
                 if timeup(max_wait, start_time):
                     break
-                if timeout and (now + timeout > (start_time + max_wait)):
+                if (max_wait is not None and
+                        timeout and (now + timeout > (start_time + max_wait))):
                     # shorten timeout to not run way over max_time
                     timeout = int((start_time + max_wait) - now)
 
@@ -354,10 +358,11 @@ def wait_for_url(urls, max_wait=None, timeout=None,
                 url_exc = e
 
             time_taken = int(time.time() - start_time)
-            status_msg = "Calling '%s' failed [%s/%ss]: %s" % (url,
-                                                               time_taken,
-                                                               max_wait,
-                                                               reason)
+            max_wait_str = "%ss" % max_wait if max_wait else "unlimited"
+            status_msg = "Calling '%s' failed [%s/%s]: %s" % (url,
+                                                              time_taken,
+                                                              max_wait_str,
+                                                              reason)
             status_cb(status_msg)
             if exception_cb:
                 # This can be used to alter the headers that will be sent
diff --git a/tests/unittests/test_datasource/test_azure.py b/tests/unittests/test_datasource/test_azure.py
index 6341e1e..9680ac6 100644
--- a/tests/unittests/test_datasource/test_azure.py
+++ b/tests/unittests/test_datasource/test_azure.py
@@ -5,18 +5,20 @@ from cloudinit.util import b64e, decode_binary, load_file, write_file
 from cloudinit.sources import DataSourceAzure as dsaz
 from cloudinit.util import find_freebsd_part
 from cloudinit.util import get_path_dev_freebsd
-
+from cloudinit.version import version_string as vs
 from cloudinit.tests.helpers import (CiTestCase, TestCase, populate_dir, mock,
                                      ExitStack, PY26, SkipTest)
 
 import crypt
 import os
 import stat
+import tempfile
 import xml.etree.ElementTree as ET
 import yaml
 
 
-def construct_valid_ovf_env(data=None, pubkeys=None, userdata=None):
+def construct_valid_ovf_env(data=None, pubkeys=None,
+                            userdata=None, platform_settings=None):
     if data is None:
         data = {'HostName': 'FOOHOST'}
     if pubkeys is None:
@@ -66,10 +68,12 @@ def construct_valid_ovf_env(data=None, pubkeys=None, userdata=None):
   xmlns:i="http://www.w3.org/2001/XMLSchema-instance">
  <KmsServerHostname>kms.core.windows.net</KmsServerHostname>
  <ProvisionGuestAgent>false</ProvisionGuestAgent>
- <GuestAgentPackageName i:nil="true" />
- </PlatformSettings></wa:PlatformSettingsSection>
-</Environment>
-    """
+ <GuestAgentPackageName i:nil="true" />"""
+    if platform_settings:
+        for k, v in platform_settings.items():
+            content += "<%s>%s</%s>\n" % (k, v, k)
+    content += """</PlatformSettings></wa:PlatformSettingsSection>
+</Environment>"""
 
     return content
 
@@ -1107,4 +1111,111 @@ class TestAzureNetExists(CiTestCase):
         self.assertTrue(hasattr(dsaz, "DataSourceAzureNet"))
 
 
[e-mail]('cloudinit.sources.DataSourceAzure.util.subp')
[e-mail](dsaz, 'get_hostname')
[e-mail](dsaz, 'set_hostname')
+class TestAzureDataSourcePreprovisioning(CiTestCase):
+
+    def setUp(self):
+        super(TestAzureDataSourcePreprovisioning, self).setUp()
+        tmp = self.tmp_dir()
+        self.waagent_d = self.tmp_path('/var/lib/waagent', tmp)
+        self.paths = helpers.Paths({'cloud_dir': tmp})
+        dsaz.BUILTIN_DS_CONFIG['data_dir'] = self.waagent_d
+
+    def test_read_azure_ovf_with_flag(self, *args):
+        """The read_azure_ovf method should set the PreprovisionedVM
+           cfg flag if the propper setting is present."""
+        content = construct_valid_ovf_env(
+            platform_settings={"PreprovisionedVm": "True"})
+        ret = dsaz.read_azure_ovf(content)
+        cfg = ret[2]
+        self.assertTrue(cfg['PreprovisionedVm'])
+
+    def test_read_azure_ovf_without_flag(self, *args):
+        """The read_azure_ovf method should not set the
+           PreprovisionedVM cfg flag."""
+        content = construct_valid_ovf_env()
+        ret = dsaz.read_azure_ovf(content)
+        cfg = ret[2]
+        self.assertFalse(cfg['PreprovisionedVm'])
+
+    @mock.patch('requests.Session.request')
+    def test_poll_imds_returns_ovf_env(self, fake_resp, *args):
+        """The _poll_imds method should return the ovf_env.xml."""
+        url = 'http://{0}/metadata/reprovisiondata?api-version=2017-04-02'
+        host = "169.254.169.254"
+        full_url = url.format(host)
+        fake_resp.return_value = mock.MagicMock(status_code=200, text="ovf")
+        dsa = dsaz.DataSourceAzure({}, distro=None, paths=self.paths)
+        self.assertTrue(len(dsa._poll_imds()) > 0)
+        print(fake_resp.call_args_list)
+        self.assertEqual(fake_resp.call_args_list,
+                         [mock.call(allow_redirects=True,
+                                    headers={'Metadata': 'true',
+                                             'User-Agent':
+                                             'Cloud-Init/%s' % vs()
+                                             }, method='GET', timeout=60.0,
+                                    url=full_url),
+                          mock.call(allow_redirects=True,
+                                    headers={'Metadata': 'true',
+                                             'User-Agent':
+                                             'Cloud-Init/%s' % vs()
+                                             }, method='GET', url=full_url)])
+
+    @mock.patch('requests.Session.request')
+    def test__reprovision_calls__poll_imds(self, fake_resp, *args):
+        """The _reprovision method should call poll IMDS."""
+        url = 'http://{0}/metadata/reprovisiondata?api-version=2017-04-02'
+        host = "169.254.169.254"
+        full_url = url.format(host)
+        hostname = "myhost"
+        username = "myuser"
+        odata = {'HostName': hostname, 'UserName': username}
+        content = construct_valid_ovf_env(data=odata)
+        fake_resp.return_value = mock.MagicMock(status_code=200, text=content)
+        dsa = dsaz.DataSourceAzure({}, distro=None, paths=self.paths)
+        md, ud, cfg, d = dsa._reprovision()
+        self.assertEqual(md['local-hostname'], hostname)
+        self.assertEqual(cfg['system_info']['default_user']['name'], username)
+        self.assertEqual(fake_resp.call_args_list,
+                         [mock.call(allow_redirects=True,
+                                    headers={'Metadata': 'true',
+                                             'User-Agent':
+                                             'Cloud-Init/%s' % vs()},
+                                    method='GET', timeout=60.0, url=full_url),
+                          mock.call(allow_redirects=True,
+                                    headers={'Metadata': 'true',
+                                             'User-Agent':
+                                             'Cloud-Init/%s' % vs()},
+                                    method='GET', url=full_url)])
+
+    @mock.patch('os.path.isfile')
+    @mock.patch.object(dsaz, 'open', create=True)
+    def test__should_reprovision_with_true_cfg(self, isfile, myopen, *args):
+        """The _should_reprovision method should return true with config
+           flag present."""
+        isfile.return_value = False
+        myopen.return_value = tempfile.TemporaryFile()
+        dsa = dsaz.DataSourceAzure({}, distro=None, paths=self.paths)
+        self.assertTrue(dsa._should_reprovision(
+            (None, None, {'PreprovisionedVm': True}, None)))
+
+    @mock.patch('os.path.isfile')
+    def test__should_reprovision_with_file_existing(self, isfile, *args):
+        """The _should_reprovision method should return True if the sentinal
+           exists."""
+        isfile.return_value = True
+        dsa = dsaz.DataSourceAzure({}, distro=None, paths=self.paths)
+        self.assertTrue(dsa._should_reprovision(
+            (None, None, {'preprovisionedvm': False}, None)))
+
+    @mock.patch('os.path.isfile')
+    def test__should_reprovision_returns_false(self, isfile, *args):
+        """The _should_reprovision method should return False
+           if config and sentinal are not present."""
+        isfile.return_value = False
+        dsa = dsaz.DataSourceAzure({}, distro=None, paths=self.paths)
+        self.assertFalse(dsa._should_reprovision((None, None, {}, None)))
+
 # vi: ts=4 expandtab