Merge ~tribaal/cloud-init:feat/datasource-exoscale into cloud-init:master

[Chris Glass]

New data source for the Exoscale.com cloud platform

- dsidentify switches to the new Exoscale datasource on matching DMI name
- New Exoscale datasource added

Author: Chris Glass <email address hidden>
Signed-off-by: Mathieu Corbin <email address hidden>

New data source for the Exoscale.com cloud platform

While the Exoscale cloud platform started as a Cloudstack fork, over the years
more and more of the codebase was rewritten, replaced, or otherwise
fundamentally changed.

Our recent feature development around networking made it clear that while we
intend for our cloud platform to remain as compatible with the Cloudstack
datasource in the forseeable future (once forced - see below), it is time for
us to bite the proverbial bullet and submit our own datasource for review.

Networking glitch:

The Cloudstack datasource doesn't work for us in the scenario where a user boots
a VM attached to the default network and one or more private network(s) running
DHCP (hosted by us, or customer-deployed): in some race condition the private
interface will be the lease on which the datasource will try to query for
metadata - which can't work. The submitted datasource fixes the problem by
forcing the datasource IP address to always be the same, so that we don't need
to depend on DHCP at all to determine it, fixing the race (we ensure routes to
that address are provided).

DMI product name change:

Along with this change, we intend to switch the DMI product name from
"CloudStack KVM Hypervisor" to "Exoscale Compute Platform" and the DMI
manufacturer string from "Apache Software Foundation" to "Exoscale".

Current templates situation:

We currently do not advertise custom templates to our customers, but intend to do
so more widely in the near future.

All of our current production templates force the Cloudstack datasource usage
(via configuration in /etc/cloud*).

Once this code lands, and our DMI product name change is performed, we will
document the requirements for (future) custom templates to be either:

- A recent enough cloud-init (containing the proposed code)
- Forcing the datasource to be "Cloudstack" for older cloud-init versions. This
  will trigger the above-mentioned race, but that is a specific, relatively rare
  use-case.
- Certain (future) features might only be available to customers running a
  recent-enough cloud-init version.

Testing this code:

Inclusion in the cloud-init test suite in addition to the submitted unit tests
would be welcome.

[Server Team CI bot (server-team-bot)]

FAILED: Continuous integration, rev:7279a5570bc87909022d9f849ac609d7acdbb3ca
https://jenkins.ubuntu.com/server/job/cloud-init-ci/751/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    FAILED: Ubuntu LTS: Integration

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/751/rebuild

review: Needs Fixing (continuous-integration)

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:7279a5570bc87909022d9f849ac609d7acdbb3ca
https://jenkins.ubuntu.com/server/job/cloud-init-ci/752/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/752/rebuild

review: Approve (continuous-integration)

[Ryan Harper (raharper)]

Thanks for submitting this. Some inline comments and questions.

In your description you mention networking configuration issues. Have you looked at providing network-metadata and having your datasource provide a network-config property to allow cloud-init to render networking[1] for the instance?

1. https://cloudinit.readthedocs.io/en/latest/topics/network-config.html

[Dan Watkins (oddbloke)]

Quick drive-by review to ask if we can reuse the password fetching from the CloudStack DS.

[Server Team CI bot (server-team-bot)]

FAILED: Continuous integration, rev:64974621d238bd2bb5bbbf79fc27cc318975b1d0
https://jenkins.ubuntu.com/server/job/cloud-init-ci/758/
Executed test runs:
    SUCCESS: Checkout
    FAILED: Unit & Style Tests

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/758/rebuild

review: Needs Fixing (continuous-integration)

[Chris Glass (tribaal)]

A first pass fixing most review comments is done.

One open point here is the extra_config overwriting user-data - it seems to me that perhaps we should be using some other mechanism instead (my assumption was that extra_config would be overwritten by user data - but that doesn't seem to be the case).

Should we use "vendordata" instead for this particular case?

[Ryan Harper (raharper)]

Thanks for updating. The use of extra_config is fine for providing some default values when no-user-data is provided; You're correct that it will be overriden by user-data; I don't think you need to switch to vendor-data.

The jenkins failure was related to the switch to log_time and the time module import is no longer needed.

I've added some other questions inline. Thanks

review: Needs Information

[Chris Glass (tribaal)]

Thanks for the review(s).

I think all the points are once again addressed.

[Server Team CI bot (server-team-bot)]

FAILED: Continuous integration, rev:bb9ca041b15d36a6b368ad6af09a99dbfef6295a
https://jenkins.ubuntu.com/server/job/cloud-init-ci/760/
Executed test runs:
    SUCCESS: Checkout
    FAILED: Unit & Style Tests

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/760/rebuild

review: Needs Fixing (continuous-integration)

[Ryan Harper (raharper)]

Thanks for the updated, a couple more questions inline.

[Chris Glass (tribaal)]

Addressed all review comments.

[Server Team CI bot (server-team-bot)]

FAILED: Continuous integration, rev:874cb3a6bfd158f9523cced13bd42f2dbaed0e4b
https://jenkins.ubuntu.com/server/job/cloud-init-ci/763/
Executed test runs:
    SUCCESS: Checkout
    FAILED: Unit & Style Tests

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/763/rebuild

review: Needs Fixing (continuous-integration)

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:5c01e89c8f5abcc6150ebbcc82fc520fad87fbbb
https://jenkins.ubuntu.com/server/job/cloud-init-ci/764/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/764/rebuild

review: Approve (continuous-integration)

[Scott Moser (smoser)]

Some doc in doc/rtd/topics/datasources would be good.
https://cloudinit.readthedocs.io/en/latest/topics/datasources.html has examples.... Just at very least a link to exoscale.com and a mention on how that differs from ec2.

[Chris Glass (tribaal)]

Thanks for your review.

I think all your points have been addressed. While adding a bit of documentation I realized the datasource was not actually configurable. This is now the case.

[Scott Moser (smoser)]

This looks fine to me. one comment on the doc. i'm fine with this, but think it'd be good to have someone else also review.

[Chris Glass (tribaal)]

Dropped comments. Thanks!

Once again ready for review(s)!

[Chad Smith (chad.smith)]

thank you for the updates here. Couple of minor nits that may involve documentation changes and a request for unit test coverage.

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:2fc709c9b07cba1fc43cc0aa95c899fac4a8910d
https://jenkins.ubuntu.com/server/job/cloud-init-ci/769/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/769/rebuild

review: Approve (continuous-integration)

[Chris Glass (tribaal)]

Should be ready for another pass once again.

Thanks a lot for your reviews so far!

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:fca069fa63f8d541d6ec9ce1d25a1ffcab824bb7
https://jenkins.ubuntu.com/server/job/cloud-init-ci/770/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/770/rebuild

review: Approve (continuous-integration)

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:c5ea4ac4642feeb0deb35ec2f4117ee948491ebc
https://jenkins.ubuntu.com/server/job/cloud-init-ci/771/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/771/rebuild

review: Approve (continuous-integration)

[Ryan Harper (raharper)]

A couple of minor changes in docs requested.

review: Needs Fixing

[Chris Glass (tribaal)]

All requested changes should be addressed.

Thanks!

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:89c1eff973dcd6b8927a9835b0264fac498a94be
https://jenkins.ubuntu.com/server/job/cloud-init-ci/785/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/785/rebuild

review: Approve (continuous-integration)

[Ryan Harper (raharper)]

One typo, and question about the templates vs built-in datasource config for the set_password frequency.

fe07f84... by Chris Glass on 2019-07-30

Datasource makes password module run "always"

Doing this here ensures the config is merged the proper way (explicit
user choices via user-data override this setting).

9f73b66... by Chris Glass on 2019-07-30

Fix typo.

20f0c6a... by Chris Glass on 2019-07-30

Add extra tests.

[Chris Glass (tribaal)]

This is now ready for another round of review.

Thanks a lot!

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:20f0c6a1487572eaf5975b80665641098ba965b8
https://jenkins.ubuntu.com/server/job/cloud-init-ci/801/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/801/rebuild

review: Approve (continuous-integration)

[Ryan Harper (raharper)]

Thanks Chris,

This looks really good.

review: Approve

[Dan Watkins (oddbloke)]

Hey Chris, Mathieu!

Thanks for this change, it's looking in pretty good shape! There are quite a few inline comments here _but_ there are only two that I'm considering a hard blocker for landing this[0]:

* the missing copyright header
* my comment on line 143; I think that's the only (potential!) bug I spotted, and fixing it might drive some other changes to the data source so I'd like us to understand the scope of those before merging this

Please do review my other comments, and let me know what you think. Provided you can commit to addressing them in a follow-up merge proposal[1], once we've resolved the above we can land this and SRU it back with 19.2.

Thanks!

Dan

[0] Terms and conditions apply.
[1] These are the terms and conditions. ;)

review: Needs Fixing

afe3b3b... by Chris Glass on 2019-08-06

Address review comments

- Copyright headers
- Various phrasing fixes in the documentation
- Code fixes
- Ensure a non-responding password server still lets the datasource use
  the scraped metadata/userdata information

[Chris Glass (tribaal)]

Hey Dan, thanks for the review!

All comments should be addressed.

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:afe3b3bd3f0de7f5364115ad17b4e585dcdb75e4
https://jenkins.ubuntu.com/server/job/cloud-init-ci/1027/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/1027//rebuild

review: Approve (continuous-integration)

[Dan Watkins (oddbloke)]

Hey Chris, thanks for the updates and the thoughtful responses, this looks great!

review: Approve

diff --git a/cloudinit/apport.py b/cloudinit/apport.py
index 22cb7fd..003ff1f 100644
--- a/cloudinit/apport.py
+++ b/cloudinit/apport.py
@@ -23,6 +23,7 @@ KNOWN_CLOUD_NAMES = [
     'CloudStack',
     'DigitalOcean',
     'GCE - Google Compute Engine',
+    'Exoscale',
     'Hetzner Cloud',
     'IBM - (aka SoftLayer or BlueMix)',
     'LXD',
diff --git a/cloudinit/settings.py b/cloudinit/settings.py
index b1ebaad..2060d81 100644
--- a/cloudinit/settings.py
+++ b/cloudinit/settings.py
@@ -39,6 +39,7 @@ CFG_BUILTIN = {
         'Hetzner',
         'IBMCloud',
         'Oracle',
+        'Exoscale',
         # At the end to act as a 'catch' when none of the above work...
         'None',
     ],
diff --git a/cloudinit/sources/DataSourceExoscale.py b/cloudinit/sources/DataSourceExoscale.py
new file mode 100644
index 0000000..52e7f6f
--- /dev/null
+++ b/cloudinit/sources/DataSourceExoscale.py
@@ -0,0 +1,258 @@
+# Author: Mathieu Corbin <[e-mail]>
+# Author: Christopher Glass <[e-mail]>
+#
+# This file is part of cloud-init. See LICENSE file for license information.
+
+from cloudinit import ec2_utils as ec2
+from cloudinit import log as logging
+from cloudinit import sources
+from cloudinit import url_helper
+from cloudinit import util
+
+LOG = logging.getLogger(__name__)
+
+METADATA_URL = "http://169.254.169.254"
+API_VERSION = "1.0"
+PASSWORD_SERVER_PORT = 8080
+
+URL_TIMEOUT = 10
+URL_RETRIES = 6
+
+EXOSCALE_DMI_NAME = "Exoscale"
+
+BUILTIN_DS_CONFIG = {
+    # We run the set password config module on every boot in order to enable
+    # resetting the instance's password via the exoscale console (and a
+    # subsequent instance reboot).
+    'cloud_config_modules': [["set-passwords", "always"]]
+}
+
+
+class DataSourceExoscale(sources.DataSource):
+
+    dsname = 'Exoscale'
+
+    def __init__(self, sys_cfg, distro, paths):
+        super(DataSourceExoscale, self).__init__(sys_cfg, distro, paths)
+        LOG.debug("Initializing the Exoscale datasource")
+
+        self.metadata_url = self.ds_cfg.get('metadata_url', METADATA_URL)
+        self.api_version = self.ds_cfg.get('api_version', API_VERSION)
+        self.password_server_port = int(
+            self.ds_cfg.get('password_server_port', PASSWORD_SERVER_PORT))
+        self.url_timeout = self.ds_cfg.get('timeout', URL_TIMEOUT)
+        self.url_retries = self.ds_cfg.get('retries', URL_RETRIES)
+
+        self.extra_config = BUILTIN_DS_CONFIG
+
+    def wait_for_metadata_service(self):
+        """Wait for the metadata service to be reachable."""
+
+        metadata_url = "{}/{}/meta-data/instance-id".format(
+            self.metadata_url, self.api_version)
+
+        url = url_helper.wait_for_url(
+            urls=[metadata_url],
+            max_wait=self.url_max_wait,
+            timeout=self.url_timeout,
+            status_cb=LOG.critical)
+
+        return bool(url)
+
+    def crawl_metadata(self):
+        """
+        Crawl the metadata service when available.
+
+        @returns: Dictionary of crawled metadata content.
+        """
+        metadata_ready = util.log_time(
+            logfunc=LOG.info,
+            msg='waiting for the metadata service',
+            func=self.wait_for_metadata_service)
+
+        if not metadata_ready:
+            return {}
+
+        return read_metadata(self.metadata_url, self.api_version,
+                             self.password_server_port, self.url_timeout,
+                             self.url_retries)
+
+    def _get_data(self):
+        """Fetch the user data, the metadata and the VM password
+        from the metadata service.
+
+        Please refer to the datasource documentation for details on how the
+        metadata server and password server are crawled.
+        """
+        if not self._is_platform_viable():
+            return False
+
+        data = util.log_time(
+            logfunc=LOG.debug,
+            msg='Crawl of metadata service',
+            func=self.crawl_metadata)
+
+        if not data:
+            return False
+
+        self.userdata_raw = data['user-data']
+        self.metadata = data['meta-data']
+        password = data.get('password')
+
+        password_config = {}
+        if password:
+            # Since we have a password, let's make sure we are allowed to use
+            # it by allowing ssh_pwauth.
+            # The password module's default behavior is to leave the
+            # configuration as-is in this regard, so that means it will either
+            # leave the password always disabled if no password is ever set, or
+            # leave the password login enabled if we set it once.
+            password_config = {
+                'ssh_pwauth': True,
+                'password': password,
+                'chpasswd': {
+                    'expire': False,
+                },
+            }
+
+        # builtin extra_config overrides password_config
+        self.extra_config = util.mergemanydict(
+            [self.extra_config, password_config])
+
+        return True
+
+    def get_config_obj(self):
+        return self.extra_config
+
+    def _is_platform_viable(self):
+        return util.read_dmi_data('system-product-name').startswith(
+            EXOSCALE_DMI_NAME)
+
+
+# Used to match classes to dependencies
+datasources = [
+    (DataSourceExoscale, (sources.DEP_FILESYSTEM, sources.DEP_NETWORK)),
+]
+
+
+# Return a list of data sources that match this set of dependencies
+def get_datasource_list(depends):
+    return sources.list_from_depends(depends, datasources)
+
+
+def get_password(metadata_url=METADATA_URL,
+                 api_version=API_VERSION,
+                 password_server_port=PASSWORD_SERVER_PORT,
+                 url_timeout=URL_TIMEOUT,
+                 url_retries=URL_RETRIES):
+    """Obtain the VM's password if set.
+
+    Once fetched the password is marked saved. Future calls to this method may
+    return empty string or 'saved_password'."""
+    password_url = "{}:{}/{}/".format(metadata_url, password_server_port,
+                                      api_version)
+    response = url_helper.read_file_or_url(
+        password_url,
+        ssl_details=None,
+        headers={"DomU_Request": "send_my_password"},
+        timeout=url_timeout,
+        retries=url_retries)
+    password = response.contents.decode('utf-8')
+    # the password is empty or already saved
+    # Note: the original metadata server would answer an additional
+    # 'bad_request' status, but the Exoscale implementation does not.
+    if password in ['', 'saved_password']:
+        return None
+    # save the password
+    url_helper.read_file_or_url(
+        password_url,
+        ssl_details=None,
+        headers={"DomU_Request": "saved_password"},
+        timeout=url_timeout,
+        retries=url_retries)
+    return password
+
+
+def read_metadata(metadata_url=METADATA_URL,
+                  api_version=API_VERSION,
+                  password_server_port=PASSWORD_SERVER_PORT,
+                  url_timeout=URL_TIMEOUT,
+                  url_retries=URL_RETRIES):
+    """Query the metadata server and return the retrieved data."""
+    crawled_metadata = {}
+    crawled_metadata['_metadata_api_version'] = api_version
+    try:
+        crawled_metadata['user-data'] = ec2.get_instance_userdata(
+            api_version,
+            metadata_url,
+            timeout=url_timeout,
+            retries=url_retries)
+        crawled_metadata['meta-data'] = ec2.get_instance_metadata(
+            api_version,
+            metadata_url,
+            timeout=url_timeout,
+            retries=url_retries)
+    except Exception as e:
+        util.logexc(LOG, "failed reading from metadata url %s (%s)",
+                    metadata_url, e)
+        return {}
+
+    try:
+        crawled_metadata['password'] = get_password(
+            api_version=api_version,
+            metadata_url=metadata_url,
+            password_server_port=password_server_port,
+            url_retries=url_retries,
+            url_timeout=url_timeout)
+    except Exception as e:
+        util.logexc(LOG, "failed to read from password server url %s:%s (%s)",
+                    metadata_url, password_server_port, e)
+
+    return crawled_metadata
+
+
+if __name__ == "__main__":
+    import argparse
+
+    parser = argparse.ArgumentParser(description='Query Exoscale Metadata')
+    parser.add_argument(
+        "--endpoint",
+        metavar="URL",
+        help="The url of the metadata service.",
+        default=METADATA_URL)
+    parser.add_argument(
+        "--version",
+        metavar="VERSION",
+        help="The version of the metadata endpoint to query.",
+        default=API_VERSION)
+    parser.add_argument(
+        "--retries",
+        metavar="NUM",
+        type=int,
+        help="The number of retries querying the endpoint.",
+        default=URL_RETRIES)
+    parser.add_argument(
+        "--timeout",
+        metavar="NUM",
+        type=int,
+        help="The time in seconds to wait before timing out.",
+        default=URL_TIMEOUT)
+    parser.add_argument(
+        "--password-port",
+        metavar="PORT",
+        type=int,
+        help="The port on which the password endpoint listens",
+        default=PASSWORD_SERVER_PORT)
+
+    args = parser.parse_args()
+
+    data = read_metadata(
+        metadata_url=args.endpoint,
+        api_version=args.version,
+        password_server_port=args.password_port,
+        url_timeout=args.timeout,
+        url_retries=args.retries)
+
+    print(util.json_dumps(data))
+
+# vi: ts=4 expandtab
diff --git a/doc/rtd/topics/datasources.rst b/doc/rtd/topics/datasources.rst
index 648c606..2148cd5 100644
--- a/doc/rtd/topics/datasources.rst
+++ b/doc/rtd/topics/datasources.rst
@@ -155,6 +155,7 @@ Follow for more information.
    datasources/configdrive.rst
    datasources/digitalocean.rst
    datasources/ec2.rst
+   datasources/exoscale.rst
    datasources/maas.rst
    datasources/nocloud.rst
    datasources/opennebula.rst
diff --git a/doc/rtd/topics/datasources/exoscale.rst b/doc/rtd/topics/datasources/exoscale.rst
new file mode 100644
index 0000000..27aec9c
--- /dev/null
+++ b/doc/rtd/topics/datasources/exoscale.rst
@@ -0,0 +1,68 @@
+.. _datasource_exoscale:
+
+Exoscale
+========
+
+This datasource supports reading from the metadata server used on the
+`Exoscale platform <https://exoscale.com>`_.
+
+Use of the Exoscale datasource is recommended to benefit from new features of
+the Exoscale platform.
+
+The datasource relies on the availability of a compatible metadata server
+(``http://169.254.169.254`` is used by default) and its companion password
+server, reachable at the same address (by default on port 8080).
+
+Crawling of metadata
+--------------------
+
+The metadata service and password server are crawled slightly differently:
+
+ * The "metadata service" is crawled every boot.
+ * The password server is also crawled every boot (the Exoscale datasource
+   forces the password module to run with "frequency always").
+
+In the password server case, the following rules apply in order to enable the
+"restore instance password" functionality:
+
+ * If a password is returned by the password server, it is then marked "saved"
+   by the cloud-init datasource. Subsequent boots will skip setting the password
+   (the password server will return "saved_password").
+ * When the instance password is reset (via the Exoscale UI), the password
+   server will return the non-empty password at next boot, therefore causing
+   cloud-init to reset the instance's password.
+
+Configuration
+-------------
+
+Users of this datasource are discouraged from changing the default settings
+unless instructed to by Exoscale support.
+
+The following settings are available and can be set for the datasource in system
+configuration (in `/etc/cloud/cloud.cfg.d/`).
+
+The settings available are:
+
+ * **metadata_url**: The URL for the metadata service (defaults to
+   ``http://169.254.169.254``)
+ * **api_version**: The API version path on which to query the instance metadata
+   (defaults to ``1.0``)
+ * **password_server_port**: The port (on the metadata server) on which the
+   password server listens (defaults to ``8080``).
+ * **timeout**: the timeout value provided to urlopen for each individual http
+   request. (defaults to ``10``)
+ * **retries**: The number of retries that should be done for an http request
+   (defaults to ``6``)
+
+
+An example configuration with the default values is provided below:
+
+.. sourcecode:: yaml
+
+   datasource:
+     Exoscale:
+       metadata_url: "http://169.254.169.254"
+       api_version: "1.0"
+       password_server_port: 8080
+       timeout: 10
+       retries: 6
diff --git a/tests/unittests/test_datasource/test_common.py b/tests/unittests/test_datasource/test_common.py
index 2a9cfb2..61a7a76 100644
--- a/tests/unittests/test_datasource/test_common.py
+++ b/tests/unittests/test_datasource/test_common.py
@@ -13,6 +13,7 @@ from cloudinit.sources import (
     DataSourceConfigDrive as ConfigDrive,
     DataSourceDigitalOcean as DigitalOcean,
     DataSourceEc2 as Ec2,
+    DataSourceExoscale as Exoscale,
     DataSourceGCE as GCE,
     DataSourceHetzner as Hetzner,
     DataSourceIBMCloud as IBMCloud,
@@ -53,6 +54,7 @@ DEFAULT_NETWORK = [
     CloudStack.DataSourceCloudStack,
     DSNone.DataSourceNone,
     Ec2.DataSourceEc2,
+    Exoscale.DataSourceExoscale,
     GCE.DataSourceGCE,
     MAAS.DataSourceMAAS,
     NoCloud.DataSourceNoCloudNet,
diff --git a/tests/unittests/test_datasource/test_exoscale.py b/tests/unittests/test_datasource/test_exoscale.py
new file mode 100644
index 0000000..350c330
--- /dev/null
+++ b/tests/unittests/test_datasource/test_exoscale.py
@@ -0,0 +1,203 @@
+# Author: Mathieu Corbin <[e-mail]>
+# Author: Christopher Glass <[e-mail]>
+#
+# This file is part of cloud-init. See LICENSE file for license information.
+from cloudinit import helpers
+from cloudinit.sources.DataSourceExoscale import (
+    API_VERSION,
+    DataSourceExoscale,
+    METADATA_URL,
+    get_password,
+    PASSWORD_SERVER_PORT,
+    read_metadata)
+from cloudinit.tests.helpers import HttprettyTestCase, mock
+
+import httpretty
+import requests
+
+
+TEST_PASSWORD_URL = "{}:{}/{}/".format(METADATA_URL,
+                                       PASSWORD_SERVER_PORT,
+                                       API_VERSION)
+
+TEST_METADATA_URL = "{}/{}/meta-data/".format(METADATA_URL,
+                                              API_VERSION)
+
+TEST_USERDATA_URL = "{}/{}/user-data".format(METADATA_URL,
+                                             API_VERSION)
+
+
[e-mail]
+class TestDatasourceExoscale(HttprettyTestCase):
+
+    def setUp(self):
+        super(TestDatasourceExoscale, self).setUp()
+        self.tmp = self.tmp_dir()
+        self.password_url = TEST_PASSWORD_URL
+        self.metadata_url = TEST_METADATA_URL
+        self.userdata_url = TEST_USERDATA_URL
+
+    def test_password_saved(self):
+        """The password is not set when it is not found
+        in the metadata service."""
+        httpretty.register_uri(httpretty.GET,
+                               self.password_url,
+                               body="saved_password")
+        self.assertFalse(get_password())
+
+    def test_password_empty(self):
+        """No password is set if the metadata service returns
+        an empty string."""
+        httpretty.register_uri(httpretty.GET,
+                               self.password_url,
+                               body="")
+        self.assertFalse(get_password())
+
+    def test_password(self):
+        """The password is set to what is found in the metadata
+        service."""
+        expected_password = "p@ssw0rd"
+        httpretty.register_uri(httpretty.GET,
+                               self.password_url,
+                               body=expected_password)
+        password = get_password()
+        self.assertEqual(expected_password, password)
+
+    def test_get_data(self):
+        """The datasource conforms to expected behavior when supplied
+        full test data."""
+        path = helpers.Paths({'run_dir': self.tmp})
+        ds = DataSourceExoscale({}, None, path)
+        ds._is_platform_viable = lambda: True
+        expected_password = "p@ssw0rd"
+        expected_id = "12345"
+        expected_hostname = "myname"
+        expected_userdata = "#cloud-config"
+        httpretty.register_uri(httpretty.GET,
+                               self.userdata_url,
+                               body=expected_userdata)
+        httpretty.register_uri(httpretty.GET,
+                               self.password_url,
+                               body=expected_password)
+        httpretty.register_uri(httpretty.GET,
+                               self.metadata_url,
+                               body="instance-id\nlocal-hostname")
+        httpretty.register_uri(httpretty.GET,
+                               "{}local-hostname".format(self.metadata_url),
+                               body=expected_hostname)
+        httpretty.register_uri(httpretty.GET,
+                               "{}instance-id".format(self.metadata_url),
+                               body=expected_id)
+        self.assertTrue(ds._get_data())
+        self.assertEqual(ds.userdata_raw.decode("utf-8"), "#cloud-config")
+        self.assertEqual(ds.metadata, {"instance-id": expected_id,
+                                       "local-hostname": expected_hostname})
+        self.assertEqual(ds.get_config_obj(),
+                         {'ssh_pwauth': True,
+                          'password': expected_password,
+                          'cloud_config_modules': [
+                              ["set-passwords", "always"]],
+                          'chpasswd': {
+                              'expire': False,
+                          }})
+
+    def test_get_data_saved_password(self):
+        """The datasource conforms to expected behavior when saved_password is
+        returned by the password server."""
+        path = helpers.Paths({'run_dir': self.tmp})
+        ds = DataSourceExoscale({}, None, path)
+        ds._is_platform_viable = lambda: True
+        expected_answer = "saved_password"
+        expected_id = "12345"
+        expected_hostname = "myname"
+        expected_userdata = "#cloud-config"
+        httpretty.register_uri(httpretty.GET,
+                               self.userdata_url,
+                               body=expected_userdata)
+        httpretty.register_uri(httpretty.GET,
+                               self.password_url,
+                               body=expected_answer)
+        httpretty.register_uri(httpretty.GET,
+                               self.metadata_url,
+                               body="instance-id\nlocal-hostname")
+        httpretty.register_uri(httpretty.GET,
+                               "{}local-hostname".format(self.metadata_url),
+                               body=expected_hostname)
+        httpretty.register_uri(httpretty.GET,
+                               "{}instance-id".format(self.metadata_url),
+                               body=expected_id)
+        self.assertTrue(ds._get_data())
+        self.assertEqual(ds.userdata_raw.decode("utf-8"), "#cloud-config")
+        self.assertEqual(ds.metadata, {"instance-id": expected_id,
+                                       "local-hostname": expected_hostname})
+        self.assertEqual(ds.get_config_obj(),
+                         {'cloud_config_modules': [
+                             ["set-passwords", "always"]]})
+
+    def test_get_data_no_password(self):
+        """The datasource conforms to expected behavior when no password is
+        returned by the password server."""
+        path = helpers.Paths({'run_dir': self.tmp})
+        ds = DataSourceExoscale({}, None, path)
+        ds._is_platform_viable = lambda: True
+        expected_answer = ""
+        expected_id = "12345"
+        expected_hostname = "myname"
+        expected_userdata = "#cloud-config"
+        httpretty.register_uri(httpretty.GET,
+                               self.userdata_url,
+                               body=expected_userdata)
+        httpretty.register_uri(httpretty.GET,
+                               self.password_url,
+                               body=expected_answer)
+        httpretty.register_uri(httpretty.GET,
+                               self.metadata_url,
+                               body="instance-id\nlocal-hostname")
+        httpretty.register_uri(httpretty.GET,
+                               "{}local-hostname".format(self.metadata_url),
+                               body=expected_hostname)
+        httpretty.register_uri(httpretty.GET,
+                               "{}instance-id".format(self.metadata_url),
+                               body=expected_id)
+        self.assertTrue(ds._get_data())
+        self.assertEqual(ds.userdata_raw.decode("utf-8"), "#cloud-config")
+        self.assertEqual(ds.metadata, {"instance-id": expected_id,
+                                       "local-hostname": expected_hostname})
+        self.assertEqual(ds.get_config_obj(),
+                         {'cloud_config_modules': [
+                             ["set-passwords", "always"]]})
+
+    @mock.patch('cloudinit.sources.DataSourceExoscale.get_password')
+    def test_read_metadata_when_password_server_unreachable(self, m_password):
+        """The read_metadata function returns partial results in case the
+        password server (only) is unreachable."""
+        expected_id = "12345"
+        expected_hostname = "myname"
+        expected_userdata = "#cloud-config"
+
+        m_password.side_effect = requests.Timeout('Fake Connection Timeout')
+        httpretty.register_uri(httpretty.GET,
+                               self.userdata_url,
+                               body=expected_userdata)
+        httpretty.register_uri(httpretty.GET,
+                               self.metadata_url,
+                               body="instance-id\nlocal-hostname")
+        httpretty.register_uri(httpretty.GET,
+                               "{}local-hostname".format(self.metadata_url),
+                               body=expected_hostname)
+        httpretty.register_uri(httpretty.GET,
+                               "{}instance-id".format(self.metadata_url),
+                               body=expected_id)
+
+        result = read_metadata()
+
+        self.assertIsNone(result.get("password"))
+        self.assertEqual(result.get("user-data").decode("utf-8"),
+                         expected_userdata)
+
+    def test_non_viable_platform(self):
+        """The datasource fails fast when the platform is not viable."""
+        path = helpers.Paths({'run_dir': self.tmp})
+        ds = DataSourceExoscale({}, None, path)
+        ds._is_platform_viable = lambda: False
+        self.assertFalse(ds._get_data())
diff --git a/tools/ds-identify b/tools/ds-identify
index 0305e36..e0d4865 100755
--- a/tools/ds-identify
+++ b/tools/ds-identify
@@ -124,7 +124,7 @@ DI_DSNAME=""
 # be searched if there is no setting found in config.
 DI_DSLIST_DEFAULT="MAAS ConfigDrive NoCloud AltCloud Azure Bigstep \
 CloudSigma CloudStack DigitalOcean AliYun Ec2 GCE OpenNebula OpenStack \
-OVF SmartOS Scaleway Hetzner IBMCloud Oracle"
+OVF SmartOS Scaleway Hetzner IBMCloud Oracle Exoscale"
 DI_DSLIST=""
 DI_MODE=""
 DI_ON_FOUND=""
@@ -553,6 +553,11 @@ dscheck_CloudStack() {
     return $DS_NOT_FOUND
 }
 
+dscheck_Exoscale() {
+    dmi_product_name_matches "Exoscale*" && return $DS_FOUND
+    return $DS_NOT_FOUND
+}
+
 dscheck_CloudSigma() {
     # http://paste.ubuntu.com/23624795/
     dmi_product_name_matches "CloudSigma" && return $DS_FOUND