Code review comment for ~chad.smith/cloud-init:unify-datasource-get-data

userdata_raw for an instance should *always* be bytes, which is *always* unserializble in json:

>>> json.dumps({'key': b'bytes value'})
TypeError: Object of type 'bytes' is not JSON serializable

heres a test that shows more actual content of user-data:
 http://paste.ubuntu.com/25594310/

So that means that unless something is wrong in the Datasource, the 'user-data' (and maybe vendor-data) blobs are going to be redacted with the Warning.

Is meta-data one of our "normalized" values ?

I kind of expected that our exported data looks like:
 {'instance-id': <string>,
  'hostname': <hostname>,
  'user-data'
  'vendor-data':

  'base64-keys': ['user-data', 'vendor-data', 'ec2/blobs/blob1', 'ec2/more-blobs/0']
  '_datasource': {
    'ec2-specific-key1': <string>,
    'blobs': { 'blob1': <bytes> },
    'more-blobs': [b'blobby'],
  }

So we'd have generally available keys in the top (many of them taken out of 'meta-data') with well known names, and then '_datasource' or other name with non-guaranteed stuff under that key.

Maybe i'm thinknig to far ahead, but I'm afraid of putting something in /run/cloud-init/INSTANCE_JSON_FILE that then someone reads and we're stuck with the format of.

the 'base64-keys' thought above is that those would be paths in the data to keys whose value is binary data. We should also make a distinction between a value whos contents raise error on .decode('utf-8') and a value that should generally be regarded as binary. Ie, we shouldnt just "try and see" if we can decode the values because then that leads to oddness for the user where sometimes a given key path might be in base64-keys and other times not.

I don't like that the value of a key can be a string 'Warning:' ... that seems
hard for someone to identify the difference between a possible but strange
value and "didnt work". We should never come to this case I do not think.
And if we do, raising exception is prbably ok.