Ubuntu
cloud-init package

cloud-init fails with UnicodeDecodeError

Bug #1532072 reported by Bassam Tabbara
16
This bug affects 3 people
Affects Status Importance Assigned to Milestone
cloud-init
Fix Released
Medium
Scott Moser
cloud-init (Ubuntu)
Fix Released
Medium
Unassigned
Xenial
Fix Released
Medium
Scott Moser
Yakkety
Fix Released
Medium
Unassigned

Bug Description

I'm running the latest Ubuntu Wily AMI on EC2 (ami-15b7e87f). When the instance boots up cloud-init fails with the following exception:

[ 69.519513] cloud-init[901]: 2016-01-08 03:43:00,902 - util.py[WARNING]: failed of stage init
[ 69.551842] cloud-init[901]: failed run of stage init
[ 69.552029] cloud-init[901]: ------------------------------------------------------------
[ 69.552427] cloud-init[901]: Traceback (most recent call last):
[ 69.552591] cloud-init[901]: File "/usr/bin/cloud-init", line 509, in status_wrapper
[ 69.557342] cloud-init[901]: ret = functor(name, args)
[ 69.557524] cloud-init[901]: File "/usr/bin/cloud-init", line 272, in main_init
[ 69.557695] cloud-init[901]: init.update()
[ 69.557859] cloud-init[901]: File "/usr/lib/python3/dist-packages/cloudinit/stages.py", line 351, in update
[ 69.558020] cloud-init[901]: self._store_userdata()
[ 69.558185] cloud-init[901]: File "/usr/lib/python3/dist-packages/cloudinit/stages.py", line 360, in _store_userdata
[ 69.558344] cloud-init[901]: processed_ud = self.datasource.get_userdata()
[ 69.558502] cloud-init[901]: File "/usr/lib/python3/dist-packages/cloudinit/sources/__init__.py", line 83, in get_userdata
[ 69.558660] cloud-init[901]: self.userdata = self.ud_proc.process(self.get_userdata_raw())
[ 69.558832] cloud-init[901]: File "/usr/lib/python3/dist-packages/cloudinit/user_data.py", line 96, in process
[ 69.560273] cloud-init[901]: self._process_msg(convert_string(blob), accumulating_msg)
[ 69.560440] cloud-init[901]: File "/usr/lib/python3/dist-packages/cloudinit/user_data.py", line 342, in convert_string
[ 69.560603] cloud-init[901]: data = util.decode_binary(util.decomp_gzip(raw_data))
[ 69.560764] cloud-init[901]: File "/usr/lib/python3/dist-packages/cloudinit/util.py", line 86, in decode_binary
[ 69.560926] cloud-init[901]: return blob.decode(encoding)
[ 69.561094] cloud-init[901]: UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8d in position 0: invalid start byte

I've tracked it down to the following commit:

https://github.com/number5/cloud-init/commit/a7835683afd19f1ae291cc93f008320a7820b24f#diff-a543026d533dadf4f39b57d1f65dc1b7R339

The user-data that is set on our instances is beyond my control. Here's a sample of it:

b'\x8dV\xd9\x8e\xa3H\x16\xfd\x95\x92_\x9d\x99\xec\x06\xf2\x8d\xc5,\x06\x8c1;\xa3\xd1\x88}3;\x18C\xab\xff\xbd\xa32kT\xea\x87\x1e\r\x0f!\xc5\xb9q\xe3\x9es\xe3@\xf0\xc7!\n\xa7\xd4\xbe\xab\x87\xcfC1\xcf\xfd\'\x04=\xba8|\x14\xdd4\x7f\x9eh\x1a\x86\xa2\xa5|$\x13\x14\xe6i;\x9b\xe9\xf8LG\xe8\xf0v\x98\xe6p\x9c\x97\xde*\x9b\xb4[\x00\x1ewm2\x1d>O0\xfcv\xa8\xd3M\x0b[\x9002\x8f\xbc\x1b\xcb\xb9h\xaea\x93\x82\n\xe6\xd2z\x04L\x83\xfcy\\\xa6\xf9\x1fV\xdd\x14\xd9\x03K\x9at\x0e\xf9p\x0e\x0f\x9f\x7f\x00\x92M\xd4u\x1f\xa0\xf8Tv\xed\xc7\xd4\xa7q\x99\x95\xf1G\x02\xe2\x1f?\t\xcf\x00\x06\xa9\x13\x06\x04|/~\xffb\xfc>\xa6\x8f\x14(|_\xa6\xf7\x14\x81\x88\x0f\x04~\xd7\xf9w0\xc20\x05aT\x96d\x08\x1a\xa1\x08\x89#d\x16#h\x1ag\x08\x15\xa5p\x8a\x00)\xa7\x13uJ\xe2\x08\x8e\x00\x9b_\x0c\xa6\xaf\x0e|de\x0b\x88\xf7c\xd9\xce\xa0\xea\x89BI\x14\xa7\t\xb0%\r#\x08E\xa2 !\\\xa7\x8f\xaf.\xa5\x890v\r\xfb\x95\x0f\x16\x03\xe5\xe9\xef\x06\x9a\xf1X\xf6\xf3\x7f`\x10\xf8\x95\xf3E\x9b\x99\xa6\xb4\x89\x1e\x9b\xfa[\xda\xff\xaf\x8a\x8e\x93,\xca\xd2\x88"\xf18\xa2\x93\x98\xa4q\x84HQ\x92"C\x9c\x86\xd18\xc6\x12\x84\xa4\x888\xcc2\x1c\x8dOx\x94\xd11\ng\t\t\xc0\x0cO\xc2\xdfj\xcb\x16\xd0l\xe3\xf4\x1f\x1a>\xfdf\xfd\x9d\xf0\xed\x0f\xe7\xfb\x94@\xf0\xef\xc4\x0e\x7f\xbe\x1d\x96)\xb5\x96\xb6M\x1fB7J\xc0o\x87\xcf\x9f\xfd\xf8;~i\xa6\xff\xc2e\xdevc\xca\xa5\xe3\xfc\xb3z8\xa7\\\x91\xc65\x08g\xe1c\x02\xf1\xf4\x11Ns\x19\xcb\rh\x0b\xd7\xb5Y\x99/\xe3\x17599|\xa2\'\x14\xc1q\n\x98\xedk\xe7[7\xce_ \x8a\xbf}y\xfd6v\xaf\xed\x1b\xc5O\x04\x8d\xbd\x1d\xaaf\xfa5\'P\xf2\xcb\xc8\xe6\x0c\xea\x1f>\xffu\xa8@\xd5\xb7\x03\xb4`G\xd2`\xc0#\xff\x1c\xd8\x9f\x03c0R\xe0\xbe\x8a\x18\xbb\xf7\xfe\n\xe6\x8e\x9c\xc9\x18\xc9\xff\x8cW\x15\xc71~\xb7\xf2\xb9\xaf(\xab\xcf\xb2\xccy`\x8a3\xcb\x186\xc3\xca2\x9b<\x8c\xa3\x0f{\xbcBs\xa1\xe3\xd7\x1b\x8f3F\xe7c\xbd\xc9\xcbF\xe3(\xbc\xff\xcaP}h\xdc\xba\xa4\xb6a\xeco\x02\\\r\xba\x0c#\xf4\x0c\x05K\xbe\xd1\x1a^a\n){w\xcev\x90M\xb1\x10\xbb|m\xa41\x85\x9e\x1bo\x19\xdd9\xf2\x83\x92^\xafDPn\xd1\xcd\xec\xdcj\x12\xb4\xda;z\xce\xee\x19\xa5\xb0\xab\x0c\xd1\x93\xf3\xadsc\xc92\xb5\xc8\x8b\x0c\xe2dq\x86\xd8\xe3}\xe5\xa1\\\xa7\x1b\xac\xae\x1d\x17\xd2k )F\xb8\x8d\xceR\x89\xd5\xe3\x87\xeeD\xf9\x19\xc7\x11)\xa4\xf4\xa1\xd4\xe8\xca/e\xb7(\xf1-\xab\xf2=\xa9\xf2T1\xeb\x0e"\xab\x9b\x1d>\xca9V\xda\xcb\x8b\xa5k\xd1AM\x83P\xaf\xba\x13C\xfbe\xae\x1b4\x92\xb9\xcc\xee;4\xefV\xc7\xa6y\nq\xc7\xa7]\x87\xc9\xf9b\xa7(\x9b\x9cD\xc7+u+\xb8q\x8f\xe7\xa6\x0f\xc8V\x88Jx\xbaHD\xb0M\xa5 w\xb7\x1d\x7f:!tse"b\xd4\x868\xea\xb19(\xf6\x08\xb3\xe42\xc9\xd1\xd9\x83\xc9\x9e\xa6\xc74s\x85\xa5$\x8bn\x08\\\xe9:\xd1g\x86\xec\xd9\xda\x16\x0b\xa6\xc7\xeb\xe3\xc5\x1e<k\x16\xc6\xed\x86\xba\x8c\xfd\x98\xb0\xf4\x95\xc2\xa2\x073\x97\x97\xdc\x16w?\xbd\xc5\xc8\xcc;w\x895\x92\x18f\xa9\x1duZ]\xde\xd9\x1a\xb9\xa6v\x97GY\xa5\x9eY\xf9\x12\x8eCC\x95\x19\x8f\xe4C\x1bS\xe3\x14]\x9d\x0e\xc9.ao\x9dz\xa5\xe9\x80q\x82\x1d\x86\xfb;z\xfe\xe5\x19\xc1\xe5pD\xe3k0QfM\x96\xb9\xc1\x02^i\xcc\x9cYe\x96\xc9eIr\xf7p\t\xea\xb0\xd9\x99+\x9b\xd7CQ\x97"\xbd\xc2\xc0;\x93\xc0\xf0\x0c\xacY\xf2\xaa1>\xef\x18\x06\xcfs\x8f.\x91\xee\xab^R\xcfH\x04\xe7\xe1\xbe\xba\x08\xbb\xc2z\xe5\x13\xba\xc5<\xfd\xd6\xe9#\xf1\xbe\xab\xa8\xd0\x06.\x01\xdb\xa8\xb3%\x8d\xb3\xa9;\x93\n+\xfc\xbaV\xccK\xe3\xf3U\xab\x0cB\xab\x9c\xf0\x17\x06k|\xbcj\xd67\xa6\xf1\xc6K\xab\x98\x95\xcd\xaf\x0e\xcbh\x9a\xe2\x16p"1\'u\xa3\xa7\x08\xbd\x16\x91X<c\xcc8]\xab\x9a\xd08\xbaL\xdc\xc7\x14H\xda\xd3w\x93G\xd4\xde\xad\xc0\xbb\xa0\x81\'?\x81Vv\xde\x81Vs\x12\xd9m\x10M\r\xc7\xc1{\xb1\xe6\xf99e:\x91e\x94\xf5i\x0f\xcd\xb2+\xbd\xa8:X\x19\xa8\xd8\xd3?\xab\xf8\xaa\xde\x1d\xa7\xd1\xc5]s\x8b*PCd9^!w\xceL\x1d\xa5\xe5\xaep\xe2\x0c\xc2]\xff$\x04\x12j$,)7\xab\xbe\xf1[\xa9\xb3e\xcf\x93"\xe7U\x05\xc4#\xedV7Q1\xf0g/)\xacy\x80\x8b\x8bI\xe3\xc5\xd5\xbaR\xe2U\x13\xaf\x1cFxjs\x82\xaa^\x8f\xe5\x07|\xd9\x8e\xea~\\\x88\x0cCH\x94\xcd\xb1\x89\x02\x1d(\x112 h>T\xea\xe6\xc5\x146\xb3\xc8}^9\xd8\xdeG\x83o\xdf\x1ew\xd2\x0e2c\xbc;\x96\xafq\xb9\xcf\x1e/\xc1\xcbL\xe4\xf4u\xc3\x84\x82\x0e\xba\x13T\xae]3w]2\x01\xc2\xdb6\xbc\x02#a\xb0Q\xbaZ\xba\xa7$\xe6C\xf0\xa3Sv\xbfT\xa3\xa7\xaaz\xd4\xef\xb2\x80\xb9\xadM\xdf\xad\xbb\xe9F<\xe3B\xf5p\xa4\x07Gx\x16\xf1^sA\xdeg4\xc6\xcf*\xa7F\xd2\xd4-\xcf\xea\xdc\xc8\r6H\x03\xff"\xec\x86\xcf\xbd=\xb8\x08B\xba\xf2\x0f\x1d\xf5\x9b\xe1\xbe\x1fw\xd9G\xb8]\xf4\x90L\xb8\xa9uX\xb6\xc5\xb5K\xd9E\xc8u\xb60\x18\xc0\xf9\xcc\x14\xe2D(\x9b1\x9c[\x06\xca\xc2\xa9nm\x9f\xbf\x07\xf4\xdd\xbbg\xda\x98!\\\xd1:\x86eF^\xae?\x07P\xd6c\xb7\x13\x81w\x86\x1d8\xe3\x86\'\xb1\xb5)1\xc5$F\xde\x8f\xf4\x88i\xdc\x06o\xe6>\x86D\xeb.\xafg\xd8\xf7Q\xf0\x14\x840\x97\xc4\xe7\xb1q+U\xaauiT\xd7\xf5%!\xd0-\x0b\x8f\xdd\xeafi\xba\\\xabQ\xcdh\xaa}R\xd1\xc0\xc3a\xeb\x19\xf2N\xd2M\xd5\x19\xca\x13\xd1uS\x80!\xc4\xaaW\xde\xf0/J\x17\xc8\xc0}W\xc68\xab\xac\xc1\xf0\x1a\xc3\xac\xaa!\xdb|\x8d\x8d\xe9L\x8e\x99\xc3Z\xb4b\x9fo\x91\xc3c\x92\x17O\xd7\x95\x13\x0c\x8e\xe95Z\xa4)\xeb\x05\x81o\xa4\xd8\x8f&\x96;\x1c%\xf0\x9c\xd1]\xbb\x01\xf2\xd9}jl\xb2\xb8)\nl\xe3\x82\xa1\x99*\xe3\x08\x87\x7f\xff\x8f{\xe7\xd7/\xcc\xf9;\xfe\xe3\xfb\xd6\xff\xf1\xfe\x03}\x1d\xfe\xfc\x0b'

If I revert the code things seem to work.

Related branches

lp:~smoser/cloud-init/trunk.1532072
cloud-init Commiters: Pending requested
Diff: 71 lines (+29/-12)
3 files modified
ChangeLog (+1/-0)
cloudinit/user_data.py (+16/-12)
tests/unittests/test_data.py (+12/-0)
lp:~cloud-init-dev/cloud-init/trunk
Chad Smith (community): Needs Fixing
Hannes: Pending requested
Diff: 51583 lines (+11/-49380) (has conflicts)
424 files modified
.bzrignore (+0/-4)
ChangeLog (+0/-761)
HACKING.rst (+0/-48)
LICENSE (+0/-674)
MANIFEST.in (+0/-8)
Makefile (+0/-82)
README (+11/-0)
TODO.rst (+0/-43)
bin/cloud-init (+0/-673)
cloudinit/__init__.py (+0/-21)
cloudinit/cloud.py (+0/-109)
cloudinit/config/__init__.py (+0/-58)
cloudinit/config/cc_apt_configure.py (+0/-292)
cloudinit/config/cc_apt_pipelining.py (+0/-57)
cloudinit/config/cc_bootcmd.py (+0/-54)
cloudinit/config/cc_byobu.py (+0/-80)
cloudinit/config/cc_ca_certs.py (+0/-104)
cloudinit/config/cc_chef.py (+0/-342)
cloudinit/config/cc_debug.py (+0/-109)
cloudinit/config/cc_disable_ec2_metadata.py (+0/-36)
cloudinit/config/cc_emit_upstart.py (+0/-69)
cloudinit/config/cc_fan.py (+0/-101)
cloudinit/config/cc_final_message.py (+0/-73)
cloudinit/config/cc_foo.py (+0/-52)
cloudinit/config/cc_growpart.py (+0/-300)
cloudinit/config/cc_grub_dpkg.py (+0/-73)
cloudinit/config/cc_keys_to_console.py (+0/-62)
cloudinit/config/cc_landscape.py (+0/-99)
cloudinit/config/cc_locale.py (+0/-37)
cloudinit/config/cc_lxd.py (+0/-176)
cloudinit/config/cc_mcollective.py (+0/-88)
cloudinit/config/cc_migrator.py (+0/-85)
cloudinit/config/cc_mounts.py (+0/-405)
cloudinit/config/cc_package_update_upgrade_install.py (+0/-99)
cloudinit/config/cc_phone_home.py (+0/-122)
cloudinit/config/cc_power_state_change.py (+0/-223)
cloudinit/config/cc_puppet.py (+0/-118)
cloudinit/config/cc_resizefs.py (+0/-185)
cloudinit/config/cc_resolv_conf.py (+0/-116)
cloudinit/config/cc_rh_subscription.py (+0/-408)
cloudinit/config/cc_rightscale_userdata.py (+0/-102)
cloudinit/config/cc_rsyslog.py (+0/-366)
cloudinit/config/cc_runcmd.py (+0/-38)
cloudinit/config/cc_salt_minion.py (+0/-59)
cloudinit/config/cc_scripts_per_boot.py (+0/-41)
cloudinit/config/cc_scripts_per_instance.py (+0/-41)
cloudinit/config/cc_scripts_per_once.py (+0/-41)
cloudinit/config/cc_scripts_user.py (+0/-42)
cloudinit/config/cc_scripts_vendor.py (+0/-43)
cloudinit/config/cc_seed_random.py (+0/-94)
cloudinit/config/cc_set_hostname.py (+0/-37)
cloudinit/config/cc_set_passwords.py (+0/-167)
cloudinit/config/cc_snappy.py (+0/-304)
cloudinit/config/cc_ssh.py (+0/-142)
cloudinit/config/cc_ssh_authkey_fingerprints.py (+0/-105)
cloudinit/config/cc_ssh_import_id.py (+0/-99)
cloudinit/config/cc_timezone.py (+0/-39)
cloudinit/config/cc_ubuntu_init_switch.py (+0/-162)
cloudinit/config/cc_update_etc_hosts.py (+0/-60)
cloudinit/config/cc_update_hostname.py (+0/-43)
cloudinit/config/cc_users_groups.py (+0/-34)
cloudinit/config/cc_write_files.py (+0/-103)
cloudinit/config/cc_yum_add_repo.py (+0/-107)
cloudinit/cs_utils.py (+0/-105)
cloudinit/distros/__init__.py (+0/-950)
cloudinit/distros/arch.py (+0/-201)
cloudinit/distros/debian.py (+0/-227)
cloudinit/distros/fedora.py (+0/-31)
cloudinit/distros/freebsd.py (+0/-417)
cloudinit/distros/gentoo.py (+0/-160)
cloudinit/distros/net_util.py (+0/-182)
cloudinit/distros/parsers/__init__.py (+0/-28)
cloudinit/distros/parsers/hostname.py (+0/-88)
cloudinit/distros/parsers/hosts.py (+0/-92)
cloudinit/distros/parsers/resolv_conf.py (+0/-169)
cloudinit/distros/parsers/sys_conf.py (+0/-113)
cloudinit/distros/rhel.py (+0/-222)
cloudinit/distros/rhel_util.py (+0/-89)
cloudinit/distros/sles.py (+0/-179)
cloudinit/distros/ubuntu.py (+0/-31)
cloudinit/ec2_utils.py (+0/-201)
cloudinit/filters/__init__.py (+0/-21)
cloudinit/filters/launch_index.py (+0/-75)
cloudinit/handlers/__init__.py (+0/-274)
cloudinit/handlers/boot_hook.py (+0/-70)
cloudinit/handlers/cloud_config.py (+0/-163)
cloudinit/handlers/shell_script.py (+0/-55)
cloudinit/handlers/upstart_job.py (+0/-119)
cloudinit/helpers.py (+0/-458)
cloudinit/importer.py (+0/-58)
cloudinit/log.py (+0/-155)
cloudinit/mergers/__init__.py (+0/-166)
cloudinit/mergers/m_dict.py (+0/-88)
cloudinit/mergers/m_list.py (+0/-89)
cloudinit/mergers/m_str.py (+0/-46)
cloudinit/net/__init__.py (+0/-771)
cloudinit/net/network_state.py (+0/-447)
cloudinit/net/udev.py (+0/-54)
cloudinit/netinfo.py (+0/-249)
cloudinit/patcher.py (+0/-58)
cloudinit/registry.py (+0/-37)
cloudinit/reporting/__init__.py (+0/-42)
cloudinit/reporting/events.py (+0/-248)
cloudinit/reporting/handlers.py (+0/-91)
cloudinit/safeyaml.py (+0/-32)
cloudinit/settings.py (+0/-68)
cloudinit/signal_handler.py (+0/-71)
cloudinit/sources/DataSourceAltCloud.py (+0/-293)
cloudinit/sources/DataSourceAzure.py (+0/-651)
cloudinit/sources/DataSourceBigstep.py (+0/-57)
cloudinit/sources/DataSourceCloudSigma.py (+0/-141)
cloudinit/sources/DataSourceCloudStack.py (+0/-253)
cloudinit/sources/DataSourceConfigDrive.py (+0/-424)
cloudinit/sources/DataSourceDigitalOcean.py (+0/-110)
cloudinit/sources/DataSourceEc2.py (+0/-211)
cloudinit/sources/DataSourceGCE.py (+0/-167)
cloudinit/sources/DataSourceMAAS.py (+0/-353)
cloudinit/sources/DataSourceNoCloud.py (+0/-336)
cloudinit/sources/DataSourceNone.py (+0/-57)
cloudinit/sources/DataSourceOVF.py (+0/-429)
cloudinit/sources/DataSourceOpenNebula.py (+0/-455)
cloudinit/sources/DataSourceOpenStack.py (+0/-173)
cloudinit/sources/DataSourceSmartOS.py (+0/-536)
cloudinit/sources/__init__.py (+0/-337)
cloudinit/sources/helpers/__init__.py (+0/-13)
cloudinit/sources/helpers/azure.py (+0/-279)
cloudinit/sources/helpers/openstack.py (+0/-499)
cloudinit/sources/helpers/vmware/__init__.py (+0/-13)
cloudinit/sources/helpers/vmware/imc/__init__.py (+0/-13)
cloudinit/sources/helpers/vmware/imc/boot_proto.py (+0/-25)
cloudinit/sources/helpers/vmware/imc/config.py (+0/-95)
cloudinit/sources/helpers/vmware/imc/config_file.py (+0/-129)
cloudinit/sources/helpers/vmware/imc/config_namespace.py (+0/-25)
cloudinit/sources/helpers/vmware/imc/config_nic.py (+0/-247)
cloudinit/sources/helpers/vmware/imc/config_source.py (+0/-23)
cloudinit/sources/helpers/vmware/imc/guestcust_error.py (+0/-24)
cloudinit/sources/helpers/vmware/imc/guestcust_event.py (+0/-27)
cloudinit/sources/helpers/vmware/imc/guestcust_state.py (+0/-25)
cloudinit/sources/helpers/vmware/imc/guestcust_util.py (+0/-128)
cloudinit/sources/helpers/vmware/imc/ipv4_mode.py (+0/-45)
cloudinit/sources/helpers/vmware/imc/nic.py (+0/-147)
cloudinit/sources/helpers/vmware/imc/nic_base.py (+0/-154)
cloudinit/ssh_util.py (+0/-314)
cloudinit/stages.py (+0/-844)
cloudinit/templater.py (+0/-149)
cloudinit/type_utils.py (+0/-52)
cloudinit/url_helper.py (+0/-509)
cloudinit/user_data.py (+0/-352)
cloudinit/util.py (+0/-2226)
cloudinit/version.py (+0/-27)
config/cloud.cfg (+0/-115)
config/cloud.cfg-freebsd (+0/-88)
config/cloud.cfg.d/05_logging.cfg (+0/-66)
config/cloud.cfg.d/README (+0/-3)
doc/README (+0/-4)
doc/examples/cloud-config-add-apt-repos.txt (+0/-34)
doc/examples/cloud-config-archive-launch-index.txt (+0/-30)
doc/examples/cloud-config-archive.txt (+0/-16)
doc/examples/cloud-config-boot-cmds.txt (+0/-15)
doc/examples/cloud-config-ca-certs.txt (+0/-31)
doc/examples/cloud-config-chef-oneiric.txt (+0/-90)
doc/examples/cloud-config-chef.txt (+0/-95)
doc/examples/cloud-config-datasources.txt (+0/-73)
doc/examples/cloud-config-disk-setup.txt (+0/-251)
doc/examples/cloud-config-final-message.txt (+0/-7)
doc/examples/cloud-config-gluster.txt (+0/-18)
doc/examples/cloud-config-growpart.txt (+0/-31)
doc/examples/cloud-config-install-packages.txt (+0/-15)
doc/examples/cloud-config-landscape.txt (+0/-22)
doc/examples/cloud-config-launch-index.txt (+0/-23)
doc/examples/cloud-config-lxd.txt (+0/-55)
doc/examples/cloud-config-mcollective.txt (+0/-49)
doc/examples/cloud-config-mount-points.txt (+0/-46)
doc/examples/cloud-config-phone-home.txt (+0/-14)
doc/examples/cloud-config-power-state.txt (+0/-40)
doc/examples/cloud-config-puppet.txt (+0/-51)
doc/examples/cloud-config-reporting.txt (+0/-17)
doc/examples/cloud-config-resolv-conf.txt (+0/-20)
doc/examples/cloud-config-rh_subscription.txt (+0/-49)
doc/examples/cloud-config-rsyslog.txt (+0/-46)
doc/examples/cloud-config-run-cmds.txt (+0/-22)
doc/examples/cloud-config-salt-minion.txt (+0/-53)
doc/examples/cloud-config-seed-random.txt (+0/-32)
doc/examples/cloud-config-ssh-keys.txt (+0/-46)
doc/examples/cloud-config-update-apt.txt (+0/-7)
doc/examples/cloud-config-update-packages.txt (+0/-8)
doc/examples/cloud-config-user-groups.txt (+0/-109)
doc/examples/cloud-config-vendor-data.txt (+0/-16)
doc/examples/cloud-config-write-files.txt (+0/-33)
doc/examples/cloud-config-yum-repo.txt (+0/-20)
doc/examples/cloud-config.txt (+0/-645)
doc/examples/include-once.txt (+0/-7)
doc/examples/include.txt (+0/-5)
doc/examples/kernel-cmdline.txt (+0/-18)
doc/examples/part-handler-v2.txt (+0/-38)
doc/examples/part-handler.txt (+0/-23)
doc/examples/plain-ignored.txt (+0/-2)
doc/examples/seed/README (+0/-22)
doc/examples/seed/meta-data (+0/-30)
doc/examples/seed/user-data (+0/-3)
doc/examples/upstart-cloud-config.txt (+0/-12)
doc/examples/upstart-rclocal.txt (+0/-12)
doc/examples/user-script.txt (+0/-8)
doc/merging.rst (+0/-188)
doc/rtd/conf.py (+0/-77)
doc/rtd/index.rst (+0/-31)
doc/rtd/static/logo.svg (+0/-89)
doc/rtd/topics/availability.rst (+0/-20)
doc/rtd/topics/capabilities.rst (+0/-24)
doc/rtd/topics/datasources.rst (+0/-200)
doc/rtd/topics/dir_layout.rst (+0/-81)
doc/rtd/topics/examples.rst (+0/-133)
doc/rtd/topics/format.rst (+0/-159)
doc/rtd/topics/hacking.rst (+0/-1)
doc/rtd/topics/merging.rst (+0/-5)
doc/rtd/topics/modules.rst (+0/-342)
doc/rtd/topics/moreinfo.rst (+0/-12)
doc/sources/altcloud/README.rst (+0/-87)
doc/sources/azure/README.rst (+0/-134)
doc/sources/cloudsigma/README.rst (+0/-38)
doc/sources/cloudstack/README.rst (+0/-29)
doc/sources/configdrive/README.rst (+0/-123)
doc/sources/digitalocean/README.rst (+0/-21)
doc/sources/kernel-cmdline.txt (+0/-48)
doc/sources/nocloud/README.rst (+0/-71)
doc/sources/opennebula/README.rst (+0/-142)
doc/sources/openstack/README.rst (+0/-24)
doc/sources/ovf/README (+0/-83)
doc/sources/ovf/example/ovf-env.xml (+0/-46)
doc/sources/ovf/example/ubuntu-server.ovf (+0/-130)
doc/sources/ovf/make-iso (+0/-156)
doc/sources/ovf/ovf-env.xml.tmpl (+0/-28)
doc/sources/ovf/ovfdemo.pem (+0/-27)
doc/sources/ovf/user-data (+0/-7)
doc/sources/smartos/README.rst (+0/-149)
doc/status.txt (+0/-53)
doc/userdata.txt (+0/-79)
doc/var-lib-cloud.txt (+0/-63)
doc/vendordata.txt (+0/-53)
packages/bddeb (+0/-260)
packages/brpm (+0/-277)
packages/debian/changelog.in (+0/-6)
packages/debian/cloud-init.postinst (+0/-16)
packages/debian/cloud-init.preinst (+0/-20)
packages/debian/compat (+0/-1)
packages/debian/control.in (+0/-29)
packages/debian/copyright (+0/-29)
packages/debian/dirs (+0/-6)
packages/debian/rules.in (+0/-23)
packages/debian/watch (+0/-2)
packages/redhat/cloud-init.spec.in (+0/-200)
packages/suse/cloud-init.spec.in (+0/-163)
requirements.txt (+0/-36)
setup.py (+0/-215)
systemd/cloud-config.service (+0/-16)
systemd/cloud-config.target (+0/-11)
systemd/cloud-final.service (+0/-17)
systemd/cloud-init-generator (+0/-133)
systemd/cloud-init-local.service (+0/-22)
systemd/cloud-init.service (+0/-18)
systemd/cloud-init.target (+0/-6)
sysvinit/debian/cloud-config (+0/-64)
sysvinit/debian/cloud-final (+0/-66)
sysvinit/debian/cloud-init (+0/-64)
sysvinit/debian/cloud-init-local (+0/-63)
sysvinit/freebsd/cloudconfig (+0/-35)
sysvinit/freebsd/cloudfinal (+0/-35)
sysvinit/freebsd/cloudinit (+0/-35)
sysvinit/freebsd/cloudinitlocal (+0/-35)
sysvinit/gentoo/cloud-config (+0/-13)
sysvinit/gentoo/cloud-final (+0/-11)
sysvinit/gentoo/cloud-init (+0/-12)
sysvinit/gentoo/cloud-init-local (+0/-13)
sysvinit/redhat/cloud-config (+0/-121)
sysvinit/redhat/cloud-final (+0/-121)
sysvinit/redhat/cloud-init (+0/-121)
sysvinit/redhat/cloud-init-local (+0/-124)
templates/chef_client.rb.tmpl (+0/-58)
templates/hosts.debian.tmpl (+0/-26)
templates/hosts.freebsd.tmpl (+0/-24)
templates/hosts.redhat.tmpl (+0/-24)
templates/hosts.suse.tmpl (+0/-26)
templates/resolv.conf.tmpl (+0/-30)
templates/sources.list.debian.tmpl (+0/-32)
templates/sources.list.ubuntu.tmpl (+0/-58)
test-requirements.txt (+0/-17)
tests/configs/sample1.yaml (+0/-52)
tests/data/filter_cloud_multipart.yaml (+0/-30)
tests/data/filter_cloud_multipart_1.email (+0/-11)
tests/data/filter_cloud_multipart_2.email (+0/-39)
tests/data/filter_cloud_multipart_header.email (+0/-11)
tests/data/merge_sources/expected1.yaml (+0/-1)
tests/data/merge_sources/expected10.yaml (+0/-7)
tests/data/merge_sources/expected11.yaml (+0/-5)
tests/data/merge_sources/expected12.yaml (+0/-5)
tests/data/merge_sources/expected2.yaml (+0/-3)
tests/data/merge_sources/expected3.yaml (+0/-1)
tests/data/merge_sources/expected4.yaml (+0/-2)
tests/data/merge_sources/expected5.yaml (+0/-7)
tests/data/merge_sources/expected6.yaml (+0/-9)
tests/data/merge_sources/expected7.yaml (+0/-38)
tests/data/merge_sources/expected8.yaml (+0/-7)
tests/data/merge_sources/expected9.yaml (+0/-5)
tests/data/merge_sources/source1-1.yaml (+0/-3)
tests/data/merge_sources/source1-2.yaml (+0/-5)
tests/data/merge_sources/source10-1.yaml (+0/-6)
tests/data/merge_sources/source10-2.yaml (+0/-6)
tests/data/merge_sources/source11-1.yaml (+0/-5)
tests/data/merge_sources/source11-2.yaml (+0/-3)
tests/data/merge_sources/source11-3.yaml (+0/-3)
tests/data/merge_sources/source12-1.yaml (+0/-8)
tests/data/merge_sources/source12-2.yaml (+0/-5)
tests/data/merge_sources/source2-1.yaml (+0/-6)
tests/data/merge_sources/source2-2.yaml (+0/-5)
tests/data/merge_sources/source3-1.yaml (+0/-4)
tests/data/merge_sources/source3-2.yaml (+0/-4)
tests/data/merge_sources/source4-1.yaml (+0/-3)
tests/data/merge_sources/source4-2.yaml (+0/-6)
tests/data/merge_sources/source5-1.yaml (+0/-6)
tests/data/merge_sources/source5-2.yaml (+0/-8)
tests/data/merge_sources/source6-1.yaml (+0/-5)
tests/data/merge_sources/source6-2.yaml (+0/-8)
tests/data/merge_sources/source7-1.yaml (+0/-27)
tests/data/merge_sources/source7-2.yaml (+0/-17)
tests/data/merge_sources/source8-1.yaml (+0/-7)
tests/data/merge_sources/source8-2.yaml (+0/-6)
tests/data/merge_sources/source9-1.yaml (+0/-5)
tests/data/merge_sources/source9-2.yaml (+0/-6)
tests/data/mountinfo_precise_ext4.txt (+0/-24)
tests/data/mountinfo_raring_btrfs.txt (+0/-13)
tests/data/roots/simple_ubuntu/etc/networks/interfaces (+0/-3)
tests/data/user_data.1.txt (+0/-15)
tests/data/vmware/cust-dhcp-2nic.cfg (+0/-34)
tests/data/vmware/cust-static-2nic.cfg (+0/-39)
tests/unittests/helpers.py (+0/-346)
tests/unittests/test__init__.py (+0/-220)
tests/unittests/test_builtin_handlers.py (+0/-73)
tests/unittests/test_cli.py (+0/-54)
tests/unittests/test_cs_util.py (+0/-84)
tests/unittests/test_data.py (+0/-559)
tests/unittests/test_datasource/test_altcloud.py (+0/-452)
tests/unittests/test_datasource/test_azure.py (+0/-646)
tests/unittests/test_datasource/test_azure_helper.py (+0/-421)
tests/unittests/test_datasource/test_cloudsigma.py (+0/-99)
tests/unittests/test_datasource/test_cloudstack.py (+0/-87)
tests/unittests/test_datasource/test_configdrive.py (+0/-407)
tests/unittests/test_datasource/test_digitalocean.py (+0/-127)
tests/unittests/test_datasource/test_gce.py (+0/-166)
tests/unittests/test_datasource/test_maas.py (+0/-163)
tests/unittests/test_datasource/test_nocloud.py (+0/-187)
tests/unittests/test_datasource/test_opennebula.py (+0/-300)
tests/unittests/test_datasource/test_openstack.py (+0/-347)
tests/unittests/test_datasource/test_smartos.py (+0/-585)
tests/unittests/test_distros/test_generic.py (+0/-233)
tests/unittests/test_distros/test_hostname.py (+0/-38)
tests/unittests/test_distros/test_hosts.py (+0/-41)
tests/unittests/test_distros/test_netconfig.py (+0/-321)
tests/unittests/test_distros/test_resolv.py (+0/-67)
tests/unittests/test_distros/test_sysconfig.py (+0/-82)
tests/unittests/test_distros/test_user_data_normalize.py (+0/-297)
tests/unittests/test_ec2_util.py (+0/-139)
tests/unittests/test_filters/test_launch_index.py (+0/-132)
tests/unittests/test_handler/test_handler_apt_configure.py (+0/-109)
tests/unittests/test_handler/test_handler_ca_certs.py (+0/-271)
tests/unittests/test_handler/test_handler_chef.py (+0/-192)
tests/unittests/test_handler/test_handler_debug.py (+0/-81)
tests/unittests/test_handler/test_handler_disk_setup.py (+0/-30)
tests/unittests/test_handler/test_handler_growpart.py (+0/-220)
tests/unittests/test_handler/test_handler_locale.py (+0/-67)
tests/unittests/test_handler/test_handler_lxd.py (+0/-134)
tests/unittests/test_handler/test_handler_mounts.py (+0/-133)
tests/unittests/test_handler/test_handler_power_state.py (+0/-127)
tests/unittests/test_handler/test_handler_rsyslog.py (+0/-174)
tests/unittests/test_handler/test_handler_seed_random.py (+0/-227)
tests/unittests/test_handler/test_handler_set_hostname.py (+0/-72)
tests/unittests/test_handler/test_handler_snappy.py (+0/-306)
tests/unittests/test_handler/test_handler_timezone.py (+0/-76)
tests/unittests/test_handler/test_handler_write_files.py (+0/-112)
tests/unittests/test_handler/test_handler_yum_add_repo.py (+0/-68)
tests/unittests/test_helpers.py (+0/-33)
tests/unittests/test_merging.py (+0/-257)
tests/unittests/test_net.py (+0/-130)
tests/unittests/test_pathprefix2dict.py (+0/-44)
tests/unittests/test_registry.py (+0/-28)
tests/unittests/test_reporting.py (+0/-369)
tests/unittests/test_rh_subscription.py (+0/-214)
tests/unittests/test_runs/test_merge_run.py (+0/-54)
tests/unittests/test_runs/test_simple_run.py (+0/-81)
tests/unittests/test_sshutil.py (+0/-171)
tests/unittests/test_templating.py (+0/-119)
tests/unittests/test_util.py (+0/-489)
tests/unittests/test_vmware_config_file.py (+0/-103)
tools/21-cloudinit.conf (+0/-6)
tools/Z99-cloud-locale-test.sh (+0/-98)
tools/build-on-freebsd (+0/-66)
tools/ccfg-merge-debug (+0/-90)
tools/cloud-init-per (+0/-60)
tools/hacking.py (+0/-170)
tools/make-dist-tarball (+0/-21)
tools/make-mime.py (+0/-60)
tools/make-tarball (+0/-39)
tools/mock-meta.py (+0/-454)
tools/motd-hook (+0/-35)
tools/read-dependencies (+0/-29)
tools/read-version (+0/-26)
tools/run-pep8 (+0/-22)
tools/run-pyflakes (+0/-18)
tools/run-pyflakes3 (+0/-2)
tools/tox-venv (+0/-42)
tools/uncloud-init (+0/-141)
tools/validate-yaml.py (+0/-25)
tools/write-ssh-key-fingerprints (+0/-38)
tox.ini (+0/-35)
udev/66-azure-ephemeral.rules (+0/-18)
udev/79-cloud-init-net-wait.rules (+0/-10)
udev/cloud-init-wait (+0/-70)
upstart/cloud-config.conf (+0/-9)
upstart/cloud-final.conf (+0/-10)
upstart/cloud-init-blocknet.conf (+0/-83)
upstart/cloud-init-container.conf (+0/-57)
upstart/cloud-init-local.conf (+0/-16)
upstart/cloud-init-nonet.conf (+0/-66)
upstart/cloud-init.conf (+0/-9)
upstart/cloud-log-shutdown.conf (+0/-19)
JJ Asghar: Pending requested
Diff: 0 lines
Scott Moser: Pending requested
Diff: 3061 lines (+2239/-175)
39 files modified
ChangeLog (+39/-9)
cloud-init.py (+31/-1)
cloudinit/CloudConfig/__init__.py (+1/-1)
cloudinit/CloudConfig/cc_apt_pipelining.py (+53/-0)
cloudinit/CloudConfig/cc_ca_certs.py (+3/-1)
cloudinit/CloudConfig/cc_chef.py (+5/-5)
cloudinit/CloudConfig/cc_landscape.py (+5/-0)
cloudinit/CloudConfig/cc_resizefs.py (+31/-12)
cloudinit/CloudConfig/cc_salt_minion.py (+2/-1)
cloudinit/CloudConfig/cc_update_etc_hosts.py (+1/-1)
cloudinit/DataSource.py (+4/-1)
cloudinit/DataSourceCloudStack.py (+92/-0)
cloudinit/DataSourceConfigDrive.py (+231/-0)
cloudinit/DataSourceEc2.py (+2/-84)
cloudinit/DataSourceMAAS.py (+345/-0)
cloudinit/DataSourceNoCloud.py (+75/-2)
cloudinit/DataSourceOVF.py (+1/-1)
cloudinit/SshUtil.py (+2/-0)
cloudinit/UserDataHandler.py (+2/-2)
cloudinit/__init__.py (+44/-9)
cloudinit/netinfo.py (+5/-5)
cloudinit/util.py (+253/-30)
config/cloud.cfg (+3/-1)
debian.trunk/control (+1/-0)
doc/configdrive/README (+118/-0)
doc/examples/cloud-config-chef-oneiric.txt (+90/-0)
doc/examples/cloud-config-chef.txt (+48/-6)
doc/examples/cloud-config-datasources.txt (+18/-0)
doc/examples/cloud-config.txt (+11/-0)
doc/kernel-cmdline.txt (+48/-0)
doc/nocloud/README (+55/-0)
setup.py (+1/-0)
tests/unittests/test__init__.py (+242/-0)
tests/unittests/test_datasource/test_maas.py (+153/-0)
tests/unittests/test_handler/test_handler_ca_certs.py (+4/-0)
tests/unittests/test_userdata.py (+107/-0)
tests/unittests/test_util.py (+18/-2)
tools/Z99-cloud-locale-test.sh (+92/-0)
tools/run-pylint (+3/-1)
lp:~smoser/ubuntu/xenial/cloud-init/pkg
lp:~smoser/ubuntu/yakkety/cloud-init/pkg
Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in cloud-init (Ubuntu):
status: New → Confirmed
Revision history for this message
igor (icicimov-gmail) wrote :

Hi,

Just got the same on Ubuntu Vivid EC2 image.

[ 106.998590] cloud-init[878]: 2016-01-25 11:11:34,600 - DataSourceCloudSigma.py[WARNING]: failed to get hypervisor product name via dmi data
[ 107.127859] cloud-init[878]: 2016-01-25 11:11:34,747 - util.py[WARNING]: failed of stage init
[ 107.534501] cloud-init[878]: Traceback (most recent call last):
[ 107.534891] cloud-init[878]: File "/usr/bin/cloud-init", line 497, in status_wrapper
[ 107.535193] cloud-init[878]: ret = functor(name, args)
[ 107.535489] cloud-init[878]: File "/usr/bin/cloud-init", line 264, in main_init
[ 107.535787] cloud-init[878]: init.update()
[ 107.536110] cloud-init[878]: File "/usr/lib/python3/dist-packages/cloudinit/stages.py", line 335, in update
[ 107.536404] cloud-init[878]: self._store_userdata()
[ 107.536713] cloud-init[878]: File "/usr/lib/python3/dist-packages/cloudinit/stages.py", line 344, in _store_userdata
[ 107.537005] cloud-init[878]: processed_ud = self.datasource.get_userdata()
[ 107.537292] cloud-init[878]: File "/usr/lib/python3/dist-packages/cloudinit/sources/__init__.py", line 82, in get_userdata
[ 107.537585] cloud-init[878]: self.userdata = self.ud_proc.process(self.get_userdata_raw())
[ 107.537873] cloud-init[878]: File "/usr/lib/python3/dist-packages/cloudinit/user_data.py", line 96, in process
[ 107.538161] cloud-init[878]: self._process_msg(convert_string(blob), accumulating_msg)
[ 107.538453] cloud-init[878]: File "/usr/lib/python3/dist-packages/cloudinit/user_data.py", line 342, in convert_string
[ 107.538759] cloud-init[878]: data = util.decode_binary(util.decomp_gzip(raw_data))
[ 107.539047] cloud-init[878]: File "/usr/lib/python3/dist-packages/cloudinit/util.py", line 86, in decode_binary
[ 107.539338] cloud-init[878]: return blob.decode(encoding)
[ 107.539623] cloud-init[878]: UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8d in position 0: invalid start byte

Revision history for this message
igor (icicimov-gmail) wrote :

Confirming that reverting the commit worked for me too.

# systemctl status cloud-init.service
● cloud-init.service - Initial cloud-init job (metadata service crawler)
   Loaded: loaded (/lib/systemd/system/cloud-init.service; enabled; vendor preset: enabled)
   Active: active (exited) since Mon 2016-01-25 11:39:41 UTC; 5s ago
  Process: 1475 ExecStart=/usr/bin/cloud-init init (code=exited, status=0/SUCCESS)
 Main PID: 1475 (code=exited, status=0/SUCCESS)

Scott Moser (smoser)
Changed in cloud-init (Ubuntu):
importance: Undecided → Medium
Revision history for this message
Luis Arias (kaaloo) wrote :

This is still present on Xenial.

Revision history for this message
Scott Moser (smoser) wrote :

fix-committed in trunk at revno 1245.

Changed in cloud-init (Ubuntu):
status: Confirmed → Fix Committed
status: Fix Committed → Confirmed
Changed in cloud-init:
status: New → Fix Committed
importance: Undecided → Medium
Changed in cloud-init (Ubuntu Xenial):
status: New → Confirmed
importance: Undecided → Medium
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package cloud-init - 0.7.7~bzr1245-0ubuntu1

---------------
cloud-init (0.7.7~bzr1245-0ubuntu1) yakkety; urgency=medium

  * New upstream snapshot.
    - user_data: fix error when user-data is not utf-8 decodable (LP: #1532072)
    - write_files: if no permissions are provided, use the default without
      logging a warning.

 -- Scott Moser <email address hidden> Wed, 22 Jun 2016 15:11:33 -0400

Changed in cloud-init (Ubuntu Yakkety):
status: Confirmed → Fix Released
Revision history for this message
Scott Moser (smoser) wrote :

Hello,
An SRU upload of cloud-init for 16.04 that contains a fix for this bug has been made under bug 1595302. Please track that bug if you are interested.

Changed in cloud-init (Ubuntu Xenial):
status: Confirmed → In Progress
assignee: nobody → Scott Moser (smoser)
Changed in cloud-init:
assignee: nobody → Scott Moser (smoser)
Scott Moser (smoser)
Changed in cloud-init (Ubuntu Xenial):
status: In Progress → Fix Committed
Revision history for this message
Scott Moser (smoser) wrote :

fix is now released to xenial under bug 1595302. daily cloud-images with this newer version of cloud-init should appear in the next few days. Any image with a serial number *newer* than 20160707 should have cloud-init at 0.7.7~bzr1246-0ubuntu1~16.04.1 .

Changed in cloud-init (Ubuntu Xenial):
status: Fix Committed → Fix Released
Revision history for this message
Scott Moser (smoser) wrote :

This is fixed in cloud-init 0.7.7

Changed in cloud-init:
status: Fix Committed → Fix Released
Revision history for this message
James Falcon (falcojr) wrote :

Tracked in Github Issues as https://github.com/canonical/cloud-init/issues/2610

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.