Merge ~chad.smith/cloud-init:ubuntu/xenial into cloud-init:ubuntu/xenial

[Chad Smith]

Update xenial SRU for ubuntu/zesty cherrypicking json-schema as debug log instead of warning.

[Server Team CI bot (server-team-bot)]

FAILED: Continuous integration, rev:f1001f7e48a60d6826104f1a7f11f5ee38c6e593
https://jenkins.ubuntu.com/server/job/cloud-init-ci/410/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    FAILED: MAAS Compatability Testing

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/410/rebuild

review: Needs Fixing (continuous-integration)

diff --git a/debian/changelog b/debian/changelog
index e006629..5a66def 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,10 @@
+cloud-init (17.1-18-gd4f70470-0ubuntu1~16.04.2) xenial-proposed; urgency=medium
+
+  * cherry-pick 41152f1: schema: Log debug instead of warning when
+    jsonschema is absent (LP: #1724354)
+
+ -- Chad Smith <[e-mail]>  Wed, 18 Oct 2017 15:11:25 -0600
+
 cloud-init (17.1-18-gd4f70470-0ubuntu1~16.04.1) xenial-proposed; urgency=medium
 
   * drop the following cherry picks, now incorporated in snapshot.
diff --git a/debian/patches/cpick-41152f1-schema-Log-debug-instead-of-warning-when-jsonschema-is b/debian/patches/cpick-41152f1-schema-Log-debug-instead-of-warning-when-jsonschema-is
new file mode 100644
index 0000000..6b9e784
--- /dev/null
+++ b/debian/patches/cpick-41152f1-schema-Log-debug-instead-of-warning-when-jsonschema-is
@@ -0,0 +1,181 @@
+From 41152f10ddbd8681cdac44b408038a4f23ab02df Mon Sep 17 00:00:00 2001
+From: Scott Moser <[e-mail]>
+Date: Tue, 17 Oct 2017 16:12:59 -0400
+Subject: [PATCH] schema: Log debug instead of warning when jsonschema is not
+ available.
+
+When operating in expected path, cloud-init should avoid logging with
+warning. That causes 'WARNING' messages in /var/log/cloud-init.log.
+By default, warnings also go to the console.
+
+Since jsonschema is a optional dependency, and not present on xenial
+and zesty, cloud-init should not warn there.
+
+Also here:
+* Add a test to integration tests to assert that there are no
+  warnings in /var/log/cloud-init.log.
+* Update one integration test that did show warning and the related
+  documentation and examples.
+
+LP: #1724354
+---
+ cloudinit/config/cc_users_groups.py                             | 3 ++-
+ cloudinit/config/schema.py                                      | 2 +-
+ doc/examples/cloud-config-user-groups.txt                       | 6 +++---
+ tests/cloud_tests/testcases/base.py                             | 4 ++++
+ tests/cloud_tests/testcases/examples/including_user_groups.py   | 6 ++++++
+ tests/cloud_tests/testcases/examples/including_user_groups.yaml | 7 +++++--
+ tests/cloud_tests/testcases/modules/user_groups.py              | 6 ++++++
+ tests/cloud_tests/testcases/modules/user_groups.yaml            | 7 +++++--
+ 8 files changed, 32 insertions(+), 9 deletions(-)
+
+Index: cloud-init/cloudinit/config/cc_users_groups.py
+===================================================================
+--- cloud-init.orig/cloudinit/config/cc_users_groups.py
++++ cloud-init/cloudinit/config/cc_users_groups.py
+@@ -15,7 +15,8 @@ options, see the ``Including users and g
+ Groups to add to the system can be specified as a list under the ``groups``
+ key. Each entry in the list should either contain a the group name as a string,
+ or a dictionary with the group name as the key and a list of users who should
+-be members of the group as the value.
++be members of the group as the value. **Note**: Groups are added before users,
++so any users in a group list must already exist on the system.
+ 
+ The ``users`` config key takes a list of users to configure. The first entry in
+ this list is used as the default user for the system. To preserve the standard
+Index: cloud-init/cloudinit/config/schema.py
+===================================================================
+--- cloud-init.orig/cloudinit/config/schema.py
++++ cloud-init/cloudinit/config/schema.py
+@@ -74,7 +74,7 @@ def validate_cloudconfig_schema(config,
+     try:
+         from jsonschema import Draft4Validator, FormatChecker
+     except ImportError:
+-        logging.warning(
++        logging.debug(
+             'Ignoring schema validation. python-jsonschema is not present')
+         return
+     validator = Draft4Validator(schema, format_checker=FormatChecker())
+Index: cloud-init/doc/examples/cloud-config-user-groups.txt
+===================================================================
+--- cloud-init.orig/doc/examples/cloud-config-user-groups.txt
++++ cloud-init/doc/examples/cloud-config-user-groups.txt
+@@ -1,8 +1,8 @@
+ # Add groups to the system
+-# The following example adds the ubuntu group with members foo and bar and
+-# the group cloud-users.
++# The following example adds the ubuntu group with members 'root' and 'sys'
++# and the empty group cloud-users.
+ groups:
+-  - ubuntu: [foo,bar]
++  - ubuntu: [root,sys]
+   - cloud-users
+ 
+ # Add users to the system. Users are added after groups are added.
+Index: cloud-init/tests/cloud_tests/testcases/base.py
+===================================================================
+--- cloud-init.orig/tests/cloud_tests/testcases/base.py
++++ cloud-init/tests/cloud_tests/testcases/base.py
+@@ -72,6 +72,10 @@ class CloudTestCase(unittest.TestCase):
+         result = self.get_status_data(self.get_data_file('result.json'))
+         self.assertEqual(len(result['errors']), 0)
+ 
++    def test_no_warnings_in_log(self):
++        """Warnings should not be found in the log."""
++        self.assertNotIn("WARN", self.get_data_file('cloud-init.log'))
++
+ 
+ class PasswordListTest(CloudTestCase):
+     """Base password test case class."""
+Index: cloud-init/tests/cloud_tests/testcases/examples/including_user_groups.py
+===================================================================
+--- cloud-init.orig/tests/cloud_tests/testcases/examples/including_user_groups.py
++++ cloud-init/tests/cloud_tests/testcases/examples/including_user_groups.py
+@@ -40,4 +40,10 @@ class TestUserGroups(base.CloudTestCase)
+         out = self.get_data_file('user_cloudy')
+         self.assertRegex(out, r'cloudy:x:[0-9]{3,4}:')
+ 
++    def test_user_root_in_secret(self):
++        """Test root user is in 'secret' group."""
++        user, _, groups = self.get_data_file('root_groups').partition(":")
++        self.assertIn("secret", groups.split(),
++                      msg="User root is not in group 'secret'")
++
+ # vi: ts=4 expandtab
+Index: cloud-init/tests/cloud_tests/testcases/examples/including_user_groups.yaml
+===================================================================
+--- cloud-init.orig/tests/cloud_tests/testcases/examples/including_user_groups.yaml
++++ cloud-init/tests/cloud_tests/testcases/examples/including_user_groups.yaml
+@@ -8,7 +8,7 @@ cloud_config: |
+   #cloud-config
+   # Add groups to the system
+   groups:
+-    - secret: [foobar,barfoo]
++    - secret: [root]
+     - cloud-users
+ 
+   # Add users to the system. Users are added after groups are added.
+@@ -24,7 +24,7 @@ cloud_config: |
+     - name: barfoo
+       gecos: Bar B. Foo
+       sudo: ALL=(ALL) NOPASSWD:ALL
+-      groups: cloud-users
++      groups: [cloud-users, secret]
+       lock_passwd: true
+     - name: cloudy
+       gecos: Magic Cloud App Daemon User
+@@ -49,5 +49,8 @@ collect_scripts:
+   user_cloudy: |
+     #!/bin/bash
+     getent passwd cloudy
++  root_groups: |
++    #!/bin/bash
++    groups root
+ 
+ # vi: ts=4 expandtab
+Index: cloud-init/tests/cloud_tests/testcases/modules/user_groups.py
+===================================================================
+--- cloud-init.orig/tests/cloud_tests/testcases/modules/user_groups.py
++++ cloud-init/tests/cloud_tests/testcases/modules/user_groups.py
+@@ -40,4 +40,10 @@ class TestUserGroups(base.CloudTestCase)
+         out = self.get_data_file('user_cloudy')
+         self.assertRegex(out, r'cloudy:x:[0-9]{3,4}:')
+ 
++    def test_user_root_in_secret(self):
++        """Test root user is in 'secret' group."""
++        user, _, groups = self.get_data_file('root_groups').partition(":")
++        self.assertIn("secret", groups.split(),
++                      msg="User root is not in group 'secret'")
++
+ # vi: ts=4 expandtab
+Index: cloud-init/tests/cloud_tests/testcases/modules/user_groups.yaml
+===================================================================
+--- cloud-init.orig/tests/cloud_tests/testcases/modules/user_groups.yaml
++++ cloud-init/tests/cloud_tests/testcases/modules/user_groups.yaml
+@@ -7,7 +7,7 @@ cloud_config: |
+   #cloud-config
+   # Add groups to the system
+   groups:
+-    - secret: [foobar,barfoo]
++    - secret: [root]
+     - cloud-users
+ 
+   # Add users to the system. Users are added after groups are added.
+@@ -23,7 +23,7 @@ cloud_config: |
+     - name: barfoo
+       gecos: Bar B. Foo
+       sudo: ALL=(ALL) NOPASSWD:ALL
+-      groups: cloud-users
++      groups: [cloud-users, secret]
+       lock_passwd: true
+     - name: cloudy
+       gecos: Magic Cloud App Daemon User
+@@ -48,5 +48,8 @@ collect_scripts:
+   user_cloudy: |
+     #!/bin/bash
+     getent passwd cloudy
++  root_groups: |
++    #!/bin/bash
++    groups root
+ 
+ # vi: ts=4 expandtab
diff --git a/debian/patches/series b/debian/patches/series
index 7e909af..8bba1e4 100644
--- a/debian/patches/series
+++ b/debian/patches/series
@@ -1,3 +1,4 @@
 azure-use-walinux-agent.patch
 ds-identify-behavior-xenial.patch
 stable-release-no-jsonschema-dep.patch
+cpick-41152f1-schema-Log-debug-instead-of-warning-when-jsonschema-is