Merge ~chad.smith/cloud-init:ubuntu/devel into cloud-init:ubuntu/devel

[Chad Smith]

Merge upstream snapshot of 17.2 from master for release in Bionic.

[Server Team CI bot (server-team-bot)]

FAILED: Continuous integration, rev:5cd6ba3f397e278c422154da26c8f34c08079256
https://jenkins.ubuntu.com/server/job/cloud-init-ci/635/
Executed test runs:
    SUCCESS: Checkout
    FAILED: Unit & Style Tests

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/635/rebuild

review: Needs Fixing (continuous-integration)

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:5cd6ba3f397e278c422154da26c8f34c08079256
https://jenkins.ubuntu.com/server/job/cloud-init-ci/636/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    SUCCESS: MAAS Compatability Testing
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/636/rebuild

review: Approve (continuous-integration)

diff --git a/ChangeLog b/ChangeLog
index 0260c57..31c2dcb 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,88 @@
+17.2:
+ - ds-identify: failure in NoCloud due to unset variable usage.
+   (LP: #1737704)
+ - tests: fix collect_console when not implemented [Joshua Powers]
+ - ec2: Use instance-identity doc for region and instance-id
+   [Andrew Jorgensen]
+ - tests: remove leaked tmp files in config drive tests.
+ - setup.py: Do not include rendered files in SOURCES.txt
+ - SUSE: remove delta in systemd local template for SUSE [Robert Schweikert]
+ - tests: move to using tox 1.7.5
+ - OVF: improve ds-identify to support finding OVF iso transport.
+   (LP: #1731868)
+ - VMware: Support for user provided pre and post-customization scripts
+   [Maitreyee Saikia]
+ - citest: In NoCloudKVM provide keys via metadata not userdata.
+ - pylint: Update pylint to 1.7.1, run on tests/ and tools and fix
+   complaints.
+ - Datasources: Formalize DataSource get_data and related properties.
+ - cli: Add clean and status subcommands
+ - tests: consolidate platforms into specific dirs
+ - ec2: Fix sandboxed dhclient background process cleanup. (LP: #1735331)
+ - tests: NoCloudKVMImage do not modify the original local cache image.
+ - tests: Enable bionic in integration tests. [Joshua Powers]
+ - tests: Use apt-get to install a deb so that depends get resolved.
+ - sysconfig: Correctly render dns and dns search info.
+   [Ryan McCabe] (LP: #1705804)
+ - integration test: replace curtin test ppa with cloud-init test ppa.
+ - EC2: Fix bug using fallback_nic and metadata when restoring from cache.
+   (LP: #1732917)
+ - EC2: Kill dhclient process used in sandbox dhclient. (LP: #1732964)
+ - ntp: fix configuration template rendering for openSUSE and SLES
+   (LP: #1726572)
+ - centos: Provide the failed #include url in error messages
+ - Catch UrlError when #include'ing URLs [Andrew Jorgensen]
+ - hosts: Fix openSUSE and SLES setup for /etc/hosts and clarify docs.
+   [Robert Schweikert] (LP: #1731022)
+ - rh_subscription: Perform null checks for enabled and disabled repos.
+   [Dave Mulford]
+ - Improve warning message when a template is not found.
+   [Robert Schweikert] (LP: #1731035)
+ - Replace the temporary i9n.brickies.net with i9n.cloud-init.io.
+ - Azure: don't generate network configuration for SRIOV devices
+   (LP: #1721579)
+ - tests: address some minor feedback missed in last merge.
+ - tests: integration test cleanup and full pass of nocloud-kvm.
+ - Gentoo: chmod +x on all files in sysvinit/gentoo/
+   [ckonstanski] (LP: #1727126)
+ - EC2: Limit network config to fallback nic, fix local-ipv4 only
+   instances. (LP: #1728152)
+ - Gentoo: Use "rc-service" rather than "service".
+   [Carlos Konstanski] (LP: #1727121)
+ - resizefs: Fix regression when system booted with root=PARTUUID=
+   (LP: #1725067)
+ - tools: make yum package installation more reliable
+ - citest: fix remaining warnings raised by integration tests.
+ - citest: show the class actual class name in results.
+ - ntp: fix config module schema to allow empty ntp config (LP: #1724951)
+ - tools: disable fastestmirror if using proxy [Joshua Powers]
+ - schema: Log debug instead of warning when jsonschema is not available.
+   (LP: #1724354)
+ - simpletable: Fix get_string method to return table-formatted string
+   (LP: #1722566)
+ - net: Handle bridge stp values of 0 and convert to boolean type
+ - tools: Give specific --abbrev=8 to "git describe"
+ - network: bridge_stp value not always correct (LP: #1721157)
+ - tests: re-enable tox with nocloud-kvm support [Joshua Powers]
+ - systemd: remove limit on tasks created by cloud-init-final.service.
+   [Robert Schweikert] (LP: #1717969)
+ - suse: Support addition of zypper repos via cloud-config.
+   [Robert Schweikert] (LP: #1718675)
+ - tests: Combine integration configs and testcases [Joshua Powers]
+ - Azure, CloudStack: Support reading dhcp options from systemd-networkd.
+   [Dimitri John Ledkov] (LP: #1718029)
+ - packages/debian/copyright: remove mention of boto and MIT license
+ - systemd: only mention Before=apt-daily.service on debian based distros.
+   [Robert Schweikert]
+ - Add missing simpletable and simpletable tests for failed merge
+ - Remove prettytable dependency, introduce simpletable [Andrew Jorgensen]
+ - debian/copyright: dep5 updates, reorganize, add Apache 2.0 license.
+   [Joshua Powers] (LP: #1718681)
+ - tests: remove dependency on shlex [Joshua Powers]
+ - AltCloud: Trust PATH for udevadm and modprobe.
+ - DataSourceOVF: use util.find_devs_with(TYPE=iso9660) (LP: #1718287)
+ - tests: remove a temp file used in bootcmd tests.
+
 17.1:
  - doc: document GCE datasource. [Arnd Hannemann]
  - suse: updates to templates to support openSUSE and SLES.
diff --git a/cloudinit/cmd/main.py b/cloudinit/cmd/main.py
index aa56225..30b37fe 100644
--- a/cloudinit/cmd/main.py
+++ b/cloudinit/cmd/main.py
@@ -603,7 +603,11 @@ def status_wrapper(name, args, data_d=None, link_d=None):
     else:
         raise ValueError("unknown name: %s" % name)
 
-    modes = ('init', 'init-local', 'modules-config', 'modules-final')
+    modes = ('init', 'init-local', 'modules-init', 'modules-config',
+             'modules-final')
+    if mode not in modes:
+        raise ValueError(
+            "Invalid cloud init mode specified '{0}'".format(mode))
 
     status = None
     if mode == 'init-local':
@@ -615,16 +619,18 @@ def status_wrapper(name, args, data_d=None, link_d=None):
         except Exception:
             pass
 
+    nullstatus = {
+        'errors': [],
+        'start': None,
+        'finished': None,
+    }
     if status is None:
-        nullstatus = {
-            'errors': [],
-            'start': None,
-            'finished': None,
-        }
         status = {'v1': {}}
         for m in modes:
             status['v1'][m] = nullstatus.copy()
         status['v1']['datasource'] = None
+    elif mode not in status['v1']:
+        status['v1'][mode] = nullstatus.copy()
 
     v1 = status['v1']
     v1['stage'] = mode
diff --git a/cloudinit/version.py b/cloudinit/version.py
index 3255f39..be6262d 100644
--- a/cloudinit/version.py
+++ b/cloudinit/version.py
@@ -4,7 +4,7 @@
 #
 # This file is part of cloud-init. See LICENSE file for license information.
 
-__VERSION__ = "17.1"
+__VERSION__ = "17.2"
 
 FEATURES = [
     # supports network config version 1
diff --git a/debian/changelog b/debian/changelog
index c07ed7e..e555270 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,12 @@
+cloud-init (17.2-1-g4089e20c-0ubuntu1) bionic; urgency=medium
+
+  * New upstream snapshot.
+    - cli: Fix error in cloud-init modules --mode=init.
+      (LP: #1736600)
+    - release 17.2
+
+ -- Chad Smith <[e-mail]>  Fri, 15 Dec 2017 10:11:50 -0700
+
 cloud-init (17.1-60-ga30a3bb5-0ubuntu1) bionic; urgency=medium
 
   * New upstream snapshot.
diff --git a/tests/unittests/test_cli.py b/tests/unittests/test_cli.py
index a8d28ae..0c0f427 100644
--- a/tests/unittests/test_cli.py
+++ b/tests/unittests/test_cli.py
@@ -1,9 +1,12 @@
 # This file is part of cloud-init. See LICENSE file for license information.
 
+from collections import namedtuple
+import os
 import six
 
 from cloudinit.cmd import main as cli
 from cloudinit.tests import helpers as test_helpers
+from cloudinit.util import load_file, load_json
 
 
 mock = test_helpers.mock
@@ -11,6 +14,8 @@ mock = test_helpers.mock
 
 class TestCLI(test_helpers.FilesystemMockingTestCase):
 
+    with_logs = True
+
     def setUp(self):
         super(TestCLI, self).setUp()
         self.stderr = six.StringIO()
@@ -24,6 +29,76 @@ class TestCLI(test_helpers.FilesystemMockingTestCase):
         except SystemExit as e:
             return e.code
 
+    def test_status_wrapper_errors_on_invalid_name(self):
+        """status_wrapper will error when the name parameter is not valid.
+
+        Valid name values are only init and modules.
+        """
+        tmpd = self.tmp_dir()
+        data_d = self.tmp_path('data', tmpd)
+        link_d = self.tmp_path('link', tmpd)
+        FakeArgs = namedtuple('FakeArgs', ['action', 'local', 'mode'])
+
+        def myaction():
+            raise Exception('Should not call myaction')
+
+        myargs = FakeArgs(('doesnotmatter', myaction), False, 'bogusmode')
+        with self.assertRaises(ValueError) as cm:
+            cli.status_wrapper('init1', myargs, data_d, link_d)
+        self.assertEqual('unknown name: init1', str(cm.exception))
+        self.assertNotIn('Should not call myaction', self.logs.getvalue())
+
+    def test_status_wrapper_errors_on_invalid_modes(self):
+        """status_wrapper will error if a parameter combination is invalid."""
+        tmpd = self.tmp_dir()
+        data_d = self.tmp_path('data', tmpd)
+        link_d = self.tmp_path('link', tmpd)
+        FakeArgs = namedtuple('FakeArgs', ['action', 'local', 'mode'])
+
+        def myaction():
+            raise Exception('Should not call myaction')
+
+        myargs = FakeArgs(('modules_name', myaction), False, 'bogusmode')
+        with self.assertRaises(ValueError) as cm:
+            cli.status_wrapper('modules', myargs, data_d, link_d)
+        self.assertEqual(
+            "Invalid cloud init mode specified 'modules-bogusmode'",
+            str(cm.exception))
+        self.assertNotIn('Should not call myaction', self.logs.getvalue())
+
+    def test_status_wrapper_init_local_writes_fresh_status_info(self):
+        """When running in init-local mode, status_wrapper writes status.json.
+
+        Old status and results artifacts are also removed.
+        """
+        tmpd = self.tmp_dir()
+        data_d = self.tmp_path('data', tmpd)
+        link_d = self.tmp_path('link', tmpd)
+        status_link = self.tmp_path('status.json', link_d)
+        # Write old artifacts which will be removed or updated.
+        for _dir in data_d, link_d:
+            test_helpers.populate_dir(
+                _dir, {'status.json': 'old', 'result.json': 'old'})
+
+        FakeArgs = namedtuple('FakeArgs', ['action', 'local', 'mode'])
+
+        def myaction(name, args):
+            # Return an error to watch status capture them
+            return 'SomeDatasource', ['an error']
+
+        myargs = FakeArgs(('ignored_name', myaction), True, 'bogusmode')
+        cli.status_wrapper('init', myargs, data_d, link_d)
+        # No errors reported in status
+        status_v1 = load_json(load_file(status_link))['v1']
+        self.assertEqual(['an error'], status_v1['init-local']['errors'])
+        self.assertEqual('SomeDatasource', status_v1['datasource'])
+        self.assertFalse(
+            os.path.exists(self.tmp_path('result.json', data_d)),
+            'unexpected result.json found')
+        self.assertFalse(
+            os.path.exists(self.tmp_path('result.json', link_d)),
+            'unexpected result.json link found')
+
     def test_no_arguments_shows_usage(self):
         exit_code = self._call_main()
         self.assertIn('usage: cloud-init', self.stderr.getvalue())