Merge ~smoser/cloud-init:fix/1768600-utf8-in-user-data into cloud-init:master

[Scott Moser]

Be more safe on string/bytes when writing multipart user-data to disk.

When creating the multipart mime message that is written as
user-data.txt.i, cloud-init losing data on conversion to some things
as a string.

LP: #1768600

see commit message

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:b6d09aa29b1c6170b4014116a44accf9b8281666
https://jenkins.ubuntu.com/server/job/cloud-init-ci/64/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    SUCCESS: MAAS Compatability Testing
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/64/rebuild

review: Approve (continuous-integration)

[Scott Moser (smoser)]

chad took this over at
 https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/347782

diff --git a/cloudinit/user_data.py b/cloudinit/user_data.py
index 8f6aba1..ed83d2d 100644
--- a/cloudinit/user_data.py
+++ b/cloudinit/user_data.py
@@ -337,8 +337,10 @@ def is_skippable(part):
 
 # Coverts a raw string into a mime message
 def convert_string(raw_data, content_type=NOT_MULTIPART_TYPE):
+    """convert a string (more likely bytes) or a message into
+    a mime message."""
     if not raw_data:
-        raw_data = ''
+        raw_data = b''
 
     def create_binmsg(data, content_type):
         maintype, subtype = content_type.split("/", 1)
@@ -346,15 +348,17 @@ def convert_string(raw_data, content_type=NOT_MULTIPART_TYPE):
         msg.set_payload(data)
         return msg
 
-    try:
-        data = util.decode_binary(util.decomp_gzip(raw_data))
-        if "mime-version:" in data[0:4096].lower():
-            msg = util.message_from_string(data)
-        else:
-            msg = create_binmsg(data, content_type)
-    except UnicodeDecodeError:
-        msg = create_binmsg(raw_data, content_type)
+    if isinstance(raw_data, six.text_type):
+        bdata = raw_data.encode('utf-8')
+    else:
+        bdata = raw_data
+    bdata = util.decomp_gzip(bdata, decode=False)
+    if b"mime-version:" in bdata[0:4096].lower():
+        msg = util.message_from_string(bdata.decode('utf-8'))
+    else:
+        msg = create_binmsg(bdata, content_type)
 
     return msg
 
+
 # vi: ts=4 expandtab