Merge lp:~harlowja/cloud-init/py2-3 into lp:~cloud-init-dev/cloud-init/trunk

Proposed by Joshua Harlow
Status: Rejected
Rejected by: Scott Moser
Proposed branch: lp:~harlowja/cloud-init/py2-3
Merge into: lp:~cloud-init-dev/cloud-init/trunk
Diff against target: 1969 lines (+317/-205)
59 files modified
cloudinit/config/cc_apt_configure.py (+3/-1)
cloudinit/config/cc_debug.py (+4/-2)
cloudinit/config/cc_landscape.py (+1/-1)
cloudinit/config/cc_mcollective.py (+6/-5)
cloudinit/config/cc_phone_home.py (+4/-2)
cloudinit/config/cc_puppet.py (+4/-3)
cloudinit/config/cc_resolv_conf.py (+3/-2)
cloudinit/config/cc_seed_random.py (+2/-1)
cloudinit/config/cc_ssh.py (+4/-2)
cloudinit/config/cc_yum_add_repo.py (+4/-3)
cloudinit/distros/__init__.py (+20/-19)
cloudinit/distros/arch.py (+3/-1)
cloudinit/distros/freebsd.py (+6/-4)
cloudinit/distros/net_util.py (+4/-1)
cloudinit/distros/parsers/hostname.py (+1/-1)
cloudinit/distros/parsers/hosts.py (+1/-1)
cloudinit/distros/parsers/resolv_conf.py (+1/-1)
cloudinit/distros/parsers/sys_conf.py (+5/-5)
cloudinit/distros/rhel.py (+3/-1)
cloudinit/distros/sles.py (+3/-1)
cloudinit/ec2_utils.py (+4/-5)
cloudinit/handlers/__init__.py (+1/-1)
cloudinit/handlers/boot_hook.py (+1/-1)
cloudinit/handlers/cloud_config.py (+1/-1)
cloudinit/handlers/shell_script.py (+1/-1)
cloudinit/handlers/upstart_job.py (+1/-1)
cloudinit/helpers.py (+7/-5)
cloudinit/log.py (+4/-3)
cloudinit/mergers/__init__.py (+3/-1)
cloudinit/mergers/m_dict.py (+3/-1)
cloudinit/mergers/m_list.py (+3/-1)
cloudinit/mergers/m_str.py (+6/-4)
cloudinit/netinfo.py (+5/-3)
cloudinit/signal_handler.py (+1/-1)
cloudinit/sources/DataSourceConfigDrive.py (+3/-1)
cloudinit/sources/DataSourceEc2.py (+4/-2)
cloudinit/sources/DataSourceMAAS.py (+5/-2)
cloudinit/sources/DataSourceOVF.py (+5/-3)
cloudinit/sources/DataSourceSmartOS.py (+7/-5)
cloudinit/sources/__init__.py (+6/-4)
cloudinit/sources/helpers/openstack.py (+4/-2)
cloudinit/ssh_util.py (+3/-3)
cloudinit/stages.py (+10/-9)
cloudinit/type_utils.py (+25/-6)
cloudinit/url_helper.py (+14/-7)
cloudinit/user_data.py (+6/-4)
cloudinit/util.py (+70/-45)
packages/bddeb (+1/-0)
packages/brpm (+2/-0)
requirements.txt (+3/-0)
tests/unittests/test_data.py (+7/-6)
tests/unittests/test_datasource/test_nocloud.py (+1/-1)
tests/unittests/test_datasource/test_openstack.py (+2/-3)
tests/unittests/test_distros/test_netconfig.py (+2/-3)
tests/unittests/test_handler/test_handler_locale.py (+3/-3)
tests/unittests/test_handler/test_handler_seed_random.py (+1/-1)
tests/unittests/test_handler/test_handler_set_hostname.py (+3/-3)
tests/unittests/test_handler/test_handler_timezone.py (+3/-3)
tests/unittests/test_handler/test_handler_yum_add_repo.py (+4/-3)
To merge this branch: bzr merge lp:~harlowja/cloud-init/py2-3
Reviewer Review Type Date Requested Status
cloud-init Commiters Pending
Review via email: mp+225240@code.launchpad.net

Description of the change

Gets the basic integration of six usage going.

This change does the following:

- Moves to using the new locations of modules (using six as needed)
  - urlparse moved, configparser moved, stringio...
- Fixes the octal usage that we had previously (0o644 is the new way that works)
- The utils load_file() now decodes the files by default from binary -> utf-8 (unless decode=False, decode=False seems needed for the configobj module to correctly work)
- The utils write_file() now decodes to binary before writing (from unicode) as needed
- Adjust tests to work correctly using the new way of load/writing files

To post a comment you must log in.
lp:~harlowja/cloud-init/py2-3 updated
986. By Joshua Harlow

Fix urllib.quote moving to a new location

987. By Joshua Harlow

Consistently return the unicode/text version of responses

988. By Joshua Harlow

Explicitly use response.contents instead of str(contents)

To avoid using a function that has different meaning in
python 2 and python 3 instead prefer the explict access
of the contents attribute instead (which will now always
be unicode) to avoid the subtle issues that will happen
if we continue to use str() instead.

989. By Joshua Harlow

Fix the configparser being required to use stringio and not bytesio

990. By Joshua Harlow

Fix types that changed/moved

991. By Joshua Harlow

Remove another comparison for (str, basestring)

992. By Joshua Harlow

Fix all iteritems() usage and remove (str, basestring) usage

Revision history for this message
Bohuslav "Slavek" Kabrda (bkabrda) wrote :

I'm also interested in cloud-init being compatible with Python 3. I don't understand cloud-init codebase that much, but the patch looks fine. My question is, what minimal Python version are you targeting? I'm assuming 2.6 and higher? If so, I'd recommend using dict.{items,keys,values} instead of six.iter{items,keys,values}(dict) - it's more readable (but really just a nitpick).

Revision history for this message
Joshua Harlow (harlowja) wrote :

2.6 and higher, as for the dict stuff, meh.

Revision history for this message
Zane Bitter (zaneb) wrote :

Looks good to me.

Minor point: "import pickle" in http://bazaar.launchpad.net/~harlowja/cloud-init/py2-3/revision/983/cloudinit/stages.py could be "from six.moves import cPickle as pickle" to avoid slowing down the existing Python 2 code.

Revision history for this message
Barry Warsaw (barry) wrote :

Is this branch behind trunk? I had some merge conflicts which aren't showing up in this MP.

I'm working on a new branch, highly inspired by this one, which should be more current against trunk, and use tox to ensure py2/3 compatibility. Most of the code changes here I agree with (except the iteritems ones -- I'm with Bohuslav on that :).

We have to disable the cheetah test in py3 since that package is not compatible. The biggest problem will be with the use of mocker which isn't py3 compatible. Not sure how to deal with that yet.

Revision history for this message
Joshua Harlow (harlowja) wrote :

Likely is behind (seeing that its not updated in a while); if you are working on a newer branch (derived from this one) that's cool and probably means we don't need this one at that point. Maybe time to slowly (or fastly) move to mock then...

Revision history for this message
Barry Warsaw (barry) wrote :

Here's my WIP branch: lp:~barry/cloud-init/py2-3

This largely merges Joshua's branch here, albeit manually because of aforementioned problems. It adds tox support for bilingual test runs, and the Python 2.7 test suite passes fully for me (try `tox -e py27`).

My next steps will be to look at replacing mocker and then repairing the Python 3 test suite. I'll also need to scrounge up a Python 2.6 to make sure support for that version hasn't broken.

I'll create a MP for my branch and welcome all comments and contributions!

Revision history for this message
Scott Moser (smoser) wrote :

dont know what the right way to do this is, but this is effectively merged. so no need for this review, so i rejected it.

Unmerged revisions

992. By Joshua Harlow

Fix all iteritems() usage and remove (str, basestring) usage

991. By Joshua Harlow

Remove another comparison for (str, basestring)

990. By Joshua Harlow

Fix types that changed/moved

989. By Joshua Harlow

Fix the configparser being required to use stringio and not bytesio

988. By Joshua Harlow

Explicitly use response.contents instead of str(contents)

To avoid using a function that has different meaning in
python 2 and python 3 instead prefer the explict access
of the contents attribute instead (which will now always
be unicode) to avoid the subtle issues that will happen
if we continue to use str() instead.

987. By Joshua Harlow

Consistently return the unicode/text version of responses

986. By Joshua Harlow

Fix urllib.quote moving to a new location

985. By Joshua Harlow

Fix up the unittests due to new changes

984. By Joshua Harlow

Adjust a bunch of moved StringIO imports

983. By Joshua Harlow

Fix a bunch more octal changes and import moves

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'cloudinit/config/cc_apt_configure.py'
--- cloudinit/config/cc_apt_configure.py 2014-02-12 19:56:55 +0000
+++ cloudinit/config/cc_apt_configure.py 2014-07-08 04:39:36 +0000
@@ -22,6 +22,8 @@
22import os22import os
23import re23import re
2424
25import six
26
25from cloudinit import templater27from cloudinit import templater
26from cloudinit import util28from cloudinit import util
2729
@@ -126,7 +128,7 @@
126128
127129
128def rename_apt_lists(old_mirrors, new_mirrors, lists_d="/var/lib/apt/lists"):130def rename_apt_lists(old_mirrors, new_mirrors, lists_d="/var/lib/apt/lists"):
129 for (name, omirror) in old_mirrors.iteritems():131 for (name, omirror) in six.iteritems(old_mirrors):
130 nmirror = new_mirrors.get(name)132 nmirror = new_mirrors.get(name)
131 if not nmirror:133 if not nmirror:
132 continue134 continue
133135
=== modified file 'cloudinit/config/cc_debug.py'
--- cloudinit/config/cc_debug.py 2014-01-23 19:28:59 +0000
+++ cloudinit/config/cc_debug.py 2014-07-08 04:39:36 +0000
@@ -14,10 +14,12 @@
14# You should have received a copy of the GNU General Public License14# You should have received a copy of the GNU General Public License
15# along with this program. If not, see <http://www.gnu.org/licenses/>.15# along with this program. If not, see <http://www.gnu.org/licenses/>.
1616
17import copy
18
19from six import StringIO
20
17from cloudinit import type_utils21from cloudinit import type_utils
18from cloudinit import util22from cloudinit import util
19import copy
20from StringIO import StringIO
2123
2224
23def _make_header(text):25def _make_header(text):
2426
=== modified file 'cloudinit/config/cc_landscape.py'
--- cloudinit/config/cc_landscape.py 2014-01-27 22:34:35 +0000
+++ cloudinit/config/cc_landscape.py 2014-07-08 04:39:36 +0000
@@ -20,7 +20,7 @@
2020
21import os21import os
2222
23from StringIO import StringIO23from six import StringIO
2424
25from configobj import ConfigObj25from configobj import ConfigObj
2626
2727
=== modified file 'cloudinit/config/cc_mcollective.py'
--- cloudinit/config/cc_mcollective.py 2014-01-27 22:34:35 +0000
+++ cloudinit/config/cc_mcollective.py 2014-07-08 04:39:36 +0000
@@ -19,7 +19,8 @@
19# You should have received a copy of the GNU General Public License19# You should have received a copy of the GNU General Public License
20# along with this program. If not, see <http://www.gnu.org/licenses/>.20# along with this program. If not, see <http://www.gnu.org/licenses/>.
2121
22from StringIO import StringIO22import six
23from six import StringIO
2324
24# Used since this can maintain comments25# Used since this can maintain comments
25# and doesn't need a top level section26# and doesn't need a top level section
@@ -51,7 +52,7 @@
51 # original file in order to be able to mix the rest up52 # original file in order to be able to mix the rest up
52 mcollective_config = ConfigObj(SERVER_CFG)53 mcollective_config = ConfigObj(SERVER_CFG)
53 # See: http://tiny.cc/jh9agw54 # See: http://tiny.cc/jh9agw
54 for (cfg_name, cfg) in mcollective_cfg['conf'].iteritems():55 for (cfg_name, cfg) in six.iteritems(mcollective_cfg['conf']):
55 if cfg_name == 'public-cert':56 if cfg_name == 'public-cert':
56 util.write_file(PUBCERT_FILE, cfg, mode=0644)57 util.write_file(PUBCERT_FILE, cfg, mode=0644)
57 mcollective_config['plugin.ssl_server_public'] = PUBCERT_FILE58 mcollective_config['plugin.ssl_server_public'] = PUBCERT_FILE
@@ -61,7 +62,7 @@
61 mcollective_config['plugin.ssl_server_private'] = PRICERT_FILE62 mcollective_config['plugin.ssl_server_private'] = PRICERT_FILE
62 mcollective_config['securityprovider'] = 'ssl'63 mcollective_config['securityprovider'] = 'ssl'
63 else:64 else:
64 if isinstance(cfg, (basestring, str)):65 if isinstance(cfg, six.string_types):
65 # Just set it in the 'main' section66 # Just set it in the 'main' section
66 mcollective_config[cfg_name] = cfg67 mcollective_config[cfg_name] = cfg
67 elif isinstance(cfg, (dict)):68 elif isinstance(cfg, (dict)):
@@ -69,7 +70,7 @@
69 # if it is needed and then add/or create items as needed70 # if it is needed and then add/or create items as needed
70 if cfg_name not in mcollective_config.sections:71 if cfg_name not in mcollective_config.sections:
71 mcollective_config[cfg_name] = {}72 mcollective_config[cfg_name] = {}
72 for (o, v) in cfg.iteritems():73 for (o, v) in six.iteritems(cfg):
73 mcollective_config[cfg_name][o] = v74 mcollective_config[cfg_name][o] = v
74 else:75 else:
75 # Otherwise just try to convert it to a string76 # Otherwise just try to convert it to a string
@@ -81,7 +82,7 @@
81 contents = StringIO()82 contents = StringIO()
82 mcollective_config.write(contents)83 mcollective_config.write(contents)
83 contents = contents.getvalue()84 contents = contents.getvalue()
84 util.write_file(SERVER_CFG, contents, mode=0644)85 util.write_file(SERVER_CFG, contents, mode=0o644)
8586
86 # Start mcollective87 # Start mcollective
87 util.subp(['service', 'mcollective', 'start'], capture=False)88 util.subp(['service', 'mcollective', 'start'], capture=False)
8889
=== modified file 'cloudinit/config/cc_phone_home.py'
--- cloudinit/config/cc_phone_home.py 2014-02-05 15:36:47 +0000
+++ cloudinit/config/cc_phone_home.py 2014-07-08 04:39:36 +0000
@@ -18,6 +18,8 @@
18# You should have received a copy of the GNU General Public License18# You should have received a copy of the GNU General Public License
19# along with this program. If not, see <http://www.gnu.org/licenses/>.19# along with this program. If not, see <http://www.gnu.org/licenses/>.
2020
21import six
22
21from cloudinit import templater23from cloudinit import templater
22from cloudinit import util24from cloudinit import util
2325
@@ -81,7 +83,7 @@
81 'pub_key_ecdsa': '/etc/ssh/ssh_host_ecdsa_key.pub',83 'pub_key_ecdsa': '/etc/ssh/ssh_host_ecdsa_key.pub',
82 }84 }
8385
84 for (n, path) in pubkeys.iteritems():86 for (n, path) in six.iteritems(pubkeys):
85 try:87 try:
86 all_keys[n] = util.load_file(path)88 all_keys[n] = util.load_file(path)
87 except:89 except:
@@ -99,7 +101,7 @@
99101
100 # Get them read to be posted102 # Get them read to be posted
101 real_submit_keys = {}103 real_submit_keys = {}
102 for (k, v) in submit_keys.iteritems():104 for (k, v) in six.iteritems(submit_keys):
103 if v is None:105 if v is None:
104 real_submit_keys[k] = 'N/A'106 real_submit_keys[k] = 'N/A'
105 else:107 else:
106108
=== modified file 'cloudinit/config/cc_puppet.py'
--- cloudinit/config/cc_puppet.py 2014-02-05 15:36:47 +0000
+++ cloudinit/config/cc_puppet.py 2014-07-08 04:39:36 +0000
@@ -18,7 +18,8 @@
18# You should have received a copy of the GNU General Public License18# You should have received a copy of the GNU General Public License
19# along with this program. If not, see <http://www.gnu.org/licenses/>.19# along with this program. If not, see <http://www.gnu.org/licenses/>.
2020
21from StringIO import StringIO21import six
22from six import StringIO
2223
23import os24import os
24import socket25import socket
@@ -81,7 +82,7 @@
81 cleaned_contents = '\n'.join(cleaned_lines)82 cleaned_contents = '\n'.join(cleaned_lines)
82 puppet_config.readfp(StringIO(cleaned_contents),83 puppet_config.readfp(StringIO(cleaned_contents),
83 filename=PUPPET_CONF_PATH)84 filename=PUPPET_CONF_PATH)
84 for (cfg_name, cfg) in puppet_cfg['conf'].iteritems():85 for (cfg_name, cfg) in six.iteritems(puppet_cfg['conf']):
85 # Cert configuration is a special case86 # Cert configuration is a special case
86 # Dump the puppet master ca certificate in the correct place87 # Dump the puppet master ca certificate in the correct place
87 if cfg_name == 'ca_cert':88 if cfg_name == 'ca_cert':
@@ -96,7 +97,7 @@
96 else:97 else:
97 # Iterate throug the config items, we'll use ConfigParser.set98 # Iterate throug the config items, we'll use ConfigParser.set
98 # to overwrite or create new items as needed99 # to overwrite or create new items as needed
99 for (o, v) in cfg.iteritems():100 for (o, v) in six.iteritems(cfg):
100 if o == 'certname':101 if o == 'certname':
101 # Expand %f as the fqdn102 # Expand %f as the fqdn
102 # TODO(harlowja) should this use the cloud fqdn??103 # TODO(harlowja) should this use the cloud fqdn??
103104
=== modified file 'cloudinit/config/cc_resolv_conf.py'
--- cloudinit/config/cc_resolv_conf.py 2014-02-05 15:36:47 +0000
+++ cloudinit/config/cc_resolv_conf.py 2014-07-08 04:39:36 +0000
@@ -48,6 +48,7 @@
48# timeout: 148# timeout: 1
49#49#
5050
51import six
5152
52from cloudinit.settings import PER_INSTANCE53from cloudinit.settings import PER_INSTANCE
53from cloudinit import templater54from cloudinit import templater
@@ -67,8 +68,8 @@
67 flags = []68 flags = []
68 false_flags = []69 false_flags = []
69 if 'options' in params:70 if 'options' in params:
70 for key, val in params['options'].iteritems():71 for key, val in six.iteritems(params['options']):
71 if type(val) == bool:72 if isinstance(val, bool):
72 if val:73 if val:
73 flags.append(key)74 flags.append(key)
74 else:75 else:
7576
=== modified file 'cloudinit/config/cc_seed_random.py'
--- cloudinit/config/cc_seed_random.py 2014-03-04 19:35:09 +0000
+++ cloudinit/config/cc_seed_random.py 2014-07-08 04:39:36 +0000
@@ -21,7 +21,8 @@
2121
22import base6422import base64
23import os23import os
24from StringIO import StringIO24
25from six import StringIO
2526
26from cloudinit.settings import PER_INSTANCE27from cloudinit.settings import PER_INSTANCE
27from cloudinit import log as logging28from cloudinit import log as logging
2829
=== modified file 'cloudinit/config/cc_ssh.py'
--- cloudinit/config/cc_ssh.py 2014-01-28 14:48:47 +0000
+++ cloudinit/config/cc_ssh.py 2014-07-08 04:39:36 +0000
@@ -21,6 +21,8 @@
21import glob21import glob
22import os22import os
2323
24import six
25
24# Ensure this is aliased to a name not 'distros'26# Ensure this is aliased to a name not 'distros'
25# since the module attribute 'distros'27# since the module attribute 'distros'
26# is a list of distros that are supported, not a sub-module28# is a list of distros that are supported, not a sub-module
@@ -68,13 +70,13 @@
6870
69 if "ssh_keys" in cfg:71 if "ssh_keys" in cfg:
70 # if there are keys in cloud-config, use them72 # if there are keys in cloud-config, use them
71 for (key, val) in cfg["ssh_keys"].iteritems():73 for (key, val) in six.iteritems(cfg["ssh_keys"]):
72 if key in KEY_2_FILE:74 if key in KEY_2_FILE:
73 tgt_fn = KEY_2_FILE[key][0]75 tgt_fn = KEY_2_FILE[key][0]
74 tgt_perms = KEY_2_FILE[key][1]76 tgt_perms = KEY_2_FILE[key][1]
75 util.write_file(tgt_fn, val, tgt_perms)77 util.write_file(tgt_fn, val, tgt_perms)
7678
77 for (priv, pub) in PRIV_2_PUB.iteritems():79 for (priv, pub) in six.iteritems(PRIV_2_PUB):
78 if pub in cfg['ssh_keys'] or not priv in cfg['ssh_keys']:80 if pub in cfg['ssh_keys'] or not priv in cfg['ssh_keys']:
79 continue81 continue
80 pair = (KEY_2_FILE[priv][0], KEY_2_FILE[pub][0])82 pair = (KEY_2_FILE[priv][0], KEY_2_FILE[pub][0])
8183
=== modified file 'cloudinit/config/cc_yum_add_repo.py'
--- cloudinit/config/cc_yum_add_repo.py 2014-02-06 15:59:04 +0000
+++ cloudinit/config/cc_yum_add_repo.py 2014-07-08 04:39:36 +0000
@@ -18,10 +18,11 @@
1818
19import os19import os
2020
21import configobj
22import six
23
21from cloudinit import util24from cloudinit import util
2225
23import configobj
24
2526
26def _canonicalize_id(repo_id):27def _canonicalize_id(repo_id):
27 repo_id = repo_id.lower().replace("-", "_")28 repo_id = repo_id.lower().replace("-", "_")
@@ -37,7 +38,7 @@
37 # Can handle 'lists' in certain cases38 # Can handle 'lists' in certain cases
38 # See: http://bit.ly/Qqrf1t39 # See: http://bit.ly/Qqrf1t
39 return "\n ".join([_format_repo_value(v) for v in val])40 return "\n ".join([_format_repo_value(v) for v in val])
40 if not isinstance(val, (basestring, str)):41 if not isinstance(val, six.string_types):
41 return str(val)42 return str(val)
42 return val43 return val
4344
4445
=== modified file 'cloudinit/distros/__init__.py'
--- cloudinit/distros/__init__.py 2014-02-12 19:56:55 +0000
+++ cloudinit/distros/__init__.py 2014-07-08 04:39:36 +0000
@@ -21,7 +21,8 @@
21# You should have received a copy of the GNU General Public License21# You should have received a copy of the GNU General Public License
22# along with this program. If not, see <http://www.gnu.org/licenses/>.22# along with this program. If not, see <http://www.gnu.org/licenses/>.
2323
24from StringIO import StringIO24import six
25from six import StringIO
2526
26import abc27import abc
27import itertools28import itertools
@@ -268,7 +269,7 @@
268 if header:269 if header:
269 contents.write("%s\n" % (header))270 contents.write("%s\n" % (header))
270 contents.write("%s\n" % (eh))271 contents.write("%s\n" % (eh))
271 util.write_file(self.hosts_fn, contents.getvalue(), mode=0644)272 util.write_file(self.hosts_fn, contents.getvalue(), mode=0o644)
272273
273 def _bring_up_interface(self, device_name):274 def _bring_up_interface(self, device_name):
274 cmd = ['ifup', device_name]275 cmd = ['ifup', device_name]
@@ -330,7 +331,7 @@
330 redact_opts = ['passwd']331 redact_opts = ['passwd']
331332
332 # Check the values and create the command333 # Check the values and create the command
333 for key, val in kwargs.iteritems():334 for key, val in six.iteritems(kwargs):
334335
335 if key in adduser_opts and val and isinstance(val, str):336 if key in adduser_opts and val and isinstance(val, str):
336 adduser_cmd.extend([adduser_opts[key], val])337 adduser_cmd.extend([adduser_opts[key], val])
@@ -452,7 +453,7 @@
452 util.make_header(base="added"),453 util.make_header(base="added"),
453 "#includedir %s" % (path), '']454 "#includedir %s" % (path), '']
454 sudoers_contents = "\n".join(lines)455 sudoers_contents = "\n".join(lines)
455 util.write_file(sudo_base, sudoers_contents, 0440)456 util.write_file(sudo_base, sudoers_contents, 0o440)
456 else:457 else:
457 lines = ['', util.make_header(base="added"),458 lines = ['', util.make_header(base="added"),
458 "#includedir %s" % (path), '']459 "#includedir %s" % (path), '']
@@ -462,7 +463,7 @@
462 except IOError as e:463 except IOError as e:
463 util.logexc(LOG, "Failed to write %s", sudo_base)464 util.logexc(LOG, "Failed to write %s", sudo_base)
464 raise e465 raise e
465 util.ensure_dir(path, 0750)466 util.ensure_dir(path, 0o750)
466467
467 def write_sudo_rules(self, user, rules, sudo_file=None):468 def write_sudo_rules(self, user, rules, sudo_file=None):
468 if not sudo_file:469 if not sudo_file:
@@ -475,7 +476,7 @@
475 if isinstance(rules, (list, tuple)):476 if isinstance(rules, (list, tuple)):
476 for rule in rules:477 for rule in rules:
477 lines.append("%s %s" % (user, rule))478 lines.append("%s %s" % (user, rule))
478 elif isinstance(rules, (basestring, str)):479 elif isinstance(rules, six.string_types):
479 lines.append("%s %s" % (user, rules))480 lines.append("%s %s" % (user, rules))
480 else:481 else:
481 msg = "Can not create sudoers rule addition with type %r"482 msg = "Can not create sudoers rule addition with type %r"
@@ -490,7 +491,7 @@
490 content,491 content,
491 ]492 ]
492 try:493 try:
493 util.write_file(sudo_file, "\n".join(contents), 0440)494 util.write_file(sudo_file, "\n".join(contents), 0o440)
494 except IOError as e:495 except IOError as e:
495 util.logexc(LOG, "Failed to write sudoers file %s", sudo_file)496 util.logexc(LOG, "Failed to write sudoers file %s", sudo_file)
496 raise e497 raise e
@@ -545,10 +546,10 @@
545 subst['ec2_region'] = "%s" % availability_zone[0:-1]546 subst['ec2_region'] = "%s" % availability_zone[0:-1]
546547
547 results = {}548 results = {}
548 for (name, mirror) in mirror_info.get('failsafe', {}).iteritems():549 for (name, mirror) in six.iteritems(mirror_info.get('failsafe', {})):
549 results[name] = mirror550 results[name] = mirror
550551
551 for (name, searchlist) in mirror_info.get('search', {}).iteritems():552 for (name, searchlist) in six.iteritems(mirror_info.get('search', {})):
552 mirrors = []553 mirrors = []
553 for tmpl in searchlist:554 for tmpl in searchlist:
554 try:555 try:
@@ -588,7 +589,7 @@
588# is the standard form used in the rest589# is the standard form used in the rest
589# of cloud-init590# of cloud-init
590def _normalize_groups(grp_cfg):591def _normalize_groups(grp_cfg):
591 if isinstance(grp_cfg, (str, basestring)):592 if isinstance(grp_cfg, six.string_types):
592 grp_cfg = grp_cfg.strip().split(",")593 grp_cfg = grp_cfg.strip().split(",")
593 if isinstance(grp_cfg, (list)):594 if isinstance(grp_cfg, (list)):
594 c_grp_cfg = {}595 c_grp_cfg = {}
@@ -598,7 +599,7 @@
598 if k not in c_grp_cfg:599 if k not in c_grp_cfg:
599 if isinstance(v, (list)):600 if isinstance(v, (list)):
600 c_grp_cfg[k] = list(v)601 c_grp_cfg[k] = list(v)
601 elif isinstance(v, (basestring, str)):602 elif isinstance(v, six.string_types):
602 c_grp_cfg[k] = [v]603 c_grp_cfg[k] = [v]
603 else:604 else:
604 raise TypeError("Bad group member type %s" %605 raise TypeError("Bad group member type %s" %
@@ -606,12 +607,12 @@
606 else:607 else:
607 if isinstance(v, (list)):608 if isinstance(v, (list)):
608 c_grp_cfg[k].extend(v)609 c_grp_cfg[k].extend(v)
609 elif isinstance(v, (basestring, str)):610 elif isinstance(v, six.string_types):
610 c_grp_cfg[k].append(v)611 c_grp_cfg[k].append(v)
611 else:612 else:
612 raise TypeError("Bad group member type %s" %613 raise TypeError("Bad group member type %s" %
613 type_utils.obj_name(v))614 type_utils.obj_name(v))
614 elif isinstance(i, (str, basestring)):615 elif isinstance(i, six.string_types):
615 if i not in c_grp_cfg:616 if i not in c_grp_cfg:
616 c_grp_cfg[i] = []617 c_grp_cfg[i] = []
617 else:618 else:
@@ -648,7 +649,7 @@
648 if isinstance(u_cfg, (dict)):649 if isinstance(u_cfg, (dict)):
649 ad_ucfg = []650 ad_ucfg = []
650 for (k, v) in u_cfg.items():651 for (k, v) in u_cfg.items():
651 if isinstance(v, (bool, int, basestring, str, float)):652 if isinstance(v, (bool, int, float) + six.string_types):
652 if util.is_true(v):653 if util.is_true(v):
653 ad_ucfg.append(str(k))654 ad_ucfg.append(str(k))
654 elif isinstance(v, (dict)):655 elif isinstance(v, (dict)):
@@ -658,12 +659,12 @@
658 raise TypeError(("Unmappable user value type %s"659 raise TypeError(("Unmappable user value type %s"
659 " for key %s") % (type_utils.obj_name(v), k))660 " for key %s") % (type_utils.obj_name(v), k))
660 u_cfg = ad_ucfg661 u_cfg = ad_ucfg
661 elif isinstance(u_cfg, (str, basestring)):662 elif isinstance(u_cfg, six.string_types):
662 u_cfg = util.uniq_merge_sorted(u_cfg)663 u_cfg = util.uniq_merge_sorted(u_cfg)
663664
664 users = {}665 users = {}
665 for user_config in u_cfg:666 for user_config in u_cfg:
666 if isinstance(user_config, (str, basestring, list)):667 if isinstance(user_config, (list,) + six.string_types):
667 for u in util.uniq_merge(user_config):668 for u in util.uniq_merge(user_config):
668 if u and u not in users:669 if u and u not in users:
669 users[u] = {}670 users[u] = {}
@@ -768,7 +769,7 @@
768 old_user = cfg['user']769 old_user = cfg['user']
769 # Translate it into the format that is more useful770 # Translate it into the format that is more useful
770 # going forward771 # going forward
771 if isinstance(old_user, (basestring, str)):772 if isinstance(old_user, six.string_types):
772 old_user = {773 old_user = {
773 'name': old_user,774 'name': old_user,
774 }775 }
@@ -797,7 +798,7 @@
797 default_user_config = util.mergemanydict([old_user, distro_user_config])798 default_user_config = util.mergemanydict([old_user, distro_user_config])
798799
799 base_users = cfg.get('users', [])800 base_users = cfg.get('users', [])
800 if not isinstance(base_users, (list, dict, str, basestring)):801 if not isinstance(base_users, (list, dict) + six.string_types):
801 LOG.warn(("Format for 'users' key must be a comma separated string"802 LOG.warn(("Format for 'users' key must be a comma separated string"
802 " or a dictionary or a list and not %s"),803 " or a dictionary or a list and not %s"),
803 type_utils.obj_name(base_users))804 type_utils.obj_name(base_users))
@@ -811,7 +812,7 @@
811 base_users.append({'name': 'default'})812 base_users.append({'name': 'default'})
812 elif isinstance(base_users, (dict)):813 elif isinstance(base_users, (dict)):
813 base_users['default'] = dict(base_users).get('default', True)814 base_users['default'] = dict(base_users).get('default', True)
814 elif isinstance(base_users, (str, basestring)):815 elif isinstance(base_users, six.string_types):
815 # Just append it on to be re-parsed later816 # Just append it on to be re-parsed later
816 base_users += ",default"817 base_users += ",default"
817818
818819
=== modified file 'cloudinit/distros/arch.py'
--- cloudinit/distros/arch.py 2014-02-12 19:56:55 +0000
+++ cloudinit/distros/arch.py 2014-07-08 04:39:36 +0000
@@ -16,6 +16,8 @@
16# You should have received a copy of the GNU General Public License16# You should have received a copy of the GNU General Public License
17# along with this program. If not, see <http://www.gnu.org/licenses/>.17# along with this program. If not, see <http://www.gnu.org/licenses/>.
1818
19import six
20
19from cloudinit import distros21from cloudinit import distros
20from cloudinit import helpers22from cloudinit import helpers
21from cloudinit import log as logging23from cloudinit import log as logging
@@ -68,7 +70,7 @@
68 settings, entries)70 settings, entries)
69 dev_names = entries.keys()71 dev_names = entries.keys()
70 # Format for netctl72 # Format for netctl
71 for (dev, info) in entries.iteritems():73 for (dev, info) in six.iteritems(entries):
72 nameservers = []74 nameservers = []
73 net_fn = self.network_conf_dir + dev75 net_fn = self.network_conf_dir + dev
74 net_cfg = {76 net_cfg = {
7577
=== modified file 'cloudinit/distros/freebsd.py'
--- cloudinit/distros/freebsd.py 2014-02-28 21:40:08 +0000
+++ cloudinit/distros/freebsd.py 2014-07-08 04:39:36 +0000
@@ -16,10 +16,11 @@
16# You should have received a copy of the GNU General Public License16# You should have received a copy of the GNU General Public License
17# along with this program. If not, see <http://www.gnu.org/licenses/>.17# along with this program. If not, see <http://www.gnu.org/licenses/>.
1818
19from StringIO import StringIO
20
21import re19import re
2220
21import six
22from six import StringIO
23
23from cloudinit import distros24from cloudinit import distros
24from cloudinit import helpers25from cloudinit import helpers
25from cloudinit import log as logging26from cloudinit import log as logging
@@ -150,8 +151,9 @@
150151
151 redact_opts = ['passwd']152 redact_opts = ['passwd']
152153
153 for key, val in kwargs.iteritems():154 for key, val in six.iteritems(kwargs):
154 if key in adduser_opts and val and isinstance(val, basestring):155 if key in adduser_opts and val \
156 and isinstance(val, six.string_types):
155 adduser_cmd.extend([adduser_opts[key], val])157 adduser_cmd.extend([adduser_opts[key], val])
156158
157 # Redact certain fields from the logs159 # Redact certain fields from the logs
158160
=== modified file 'cloudinit/distros/net_util.py'
--- cloudinit/distros/net_util.py 2014-01-24 21:20:54 +0000
+++ cloudinit/distros/net_util.py 2014-07-08 04:39:36 +0000
@@ -79,6 +79,9 @@
79# }79# }
80# }80# }
8181
82import six
83
84
82def translate_network(settings):85def translate_network(settings):
83 # Get the standard cmd, args from the ubuntu format86 # Get the standard cmd, args from the ubuntu format
84 entries = []87 entries = []
@@ -103,7 +106,7 @@
103 consume[cmd] = args106 consume[cmd] = args
104 # Check if anything left over to consume107 # Check if anything left over to consume
105 absorb = False108 absorb = False
106 for (cmd, args) in consume.iteritems():109 for (cmd, args) in six.iteritems(consume):
107 if cmd == 'iface':110 if cmd == 'iface':
108 absorb = True111 absorb = True
109 if absorb:112 if absorb:
110113
=== modified file 'cloudinit/distros/parsers/hostname.py'
--- cloudinit/distros/parsers/hostname.py 2012-11-12 22:30:08 +0000
+++ cloudinit/distros/parsers/hostname.py 2014-07-08 04:39:36 +0000
@@ -16,7 +16,7 @@
16# You should have received a copy of the GNU General Public License16# You should have received a copy of the GNU General Public License
17# along with this program. If not, see <http://www.gnu.org/licenses/>.17# along with this program. If not, see <http://www.gnu.org/licenses/>.
1818
19from StringIO import StringIO19from six import StringIO
2020
21from cloudinit.distros.parsers import chop_comment21from cloudinit.distros.parsers import chop_comment
2222
2323
=== modified file 'cloudinit/distros/parsers/hosts.py'
--- cloudinit/distros/parsers/hosts.py 2012-11-13 06:14:31 +0000
+++ cloudinit/distros/parsers/hosts.py 2014-07-08 04:39:36 +0000
@@ -16,7 +16,7 @@
16# You should have received a copy of the GNU General Public License16# You should have received a copy of the GNU General Public License
17# along with this program. If not, see <http://www.gnu.org/licenses/>.17# along with this program. If not, see <http://www.gnu.org/licenses/>.
1818
19from StringIO import StringIO19from six import StringIO
2020
21from cloudinit.distros.parsers import chop_comment21from cloudinit.distros.parsers import chop_comment
2222
2323
=== modified file 'cloudinit/distros/parsers/resolv_conf.py'
--- cloudinit/distros/parsers/resolv_conf.py 2013-03-19 13:32:04 +0000
+++ cloudinit/distros/parsers/resolv_conf.py 2014-07-08 04:39:36 +0000
@@ -16,7 +16,7 @@
16# You should have received a copy of the GNU General Public License16# You should have received a copy of the GNU General Public License
17# along with this program. If not, see <http://www.gnu.org/licenses/>.17# along with this program. If not, see <http://www.gnu.org/licenses/>.
1818
19from StringIO import StringIO19from six import StringIO
2020
21from cloudinit import util21from cloudinit import util
2222
2323
=== modified file 'cloudinit/distros/parsers/sys_conf.py'
--- cloudinit/distros/parsers/sys_conf.py 2012-11-12 22:30:08 +0000
+++ cloudinit/distros/parsers/sys_conf.py 2014-07-08 04:39:36 +0000
@@ -16,11 +16,11 @@
16# You should have received a copy of the GNU General Public License16# You should have received a copy of the GNU General Public License
17# along with this program. If not, see <http://www.gnu.org/licenses/>.17# along with this program. If not, see <http://www.gnu.org/licenses/>.
1818
19from StringIO import StringIO
20
21import pipes19import pipes
22import re20import re
2321
22import six
23
24# This library is used to parse/write24# This library is used to parse/write
25# out the various sysconfig files edited (best attempt effort)25# out the various sysconfig files edited (best attempt effort)
26#26#
@@ -61,15 +61,15 @@
6161
62 def __str__(self):62 def __str__(self):
63 contents = self.write()63 contents = self.write()
64 out_contents = StringIO()64 out_contents = six.StringIO()
65 if isinstance(contents, (list, tuple)):65 if isinstance(contents, (list, tuple)):
66 out_contents.write("\n".join(contents))66 out_contents.write("\n".join(contents))
67 else:67 else:
68 out_contents.write(str(contents))68 out_contents.write(six.text_type(contents))
69 return out_contents.getvalue()69 return out_contents.getvalue()
7070
71 def _quote(self, value, multiline=False):71 def _quote(self, value, multiline=False):
72 if not isinstance(value, (str, basestring)):72 if not isinstance(value, six.string_types):
73 raise ValueError('Value "%s" is not a string' % (value))73 raise ValueError('Value "%s" is not a string' % (value))
74 if len(value) == 0:74 if len(value) == 0:
75 return ''75 return ''
7676
=== modified file 'cloudinit/distros/rhel.py'
--- cloudinit/distros/rhel.py 2014-02-03 22:03:14 +0000
+++ cloudinit/distros/rhel.py 2014-07-08 04:39:36 +0000
@@ -20,6 +20,8 @@
20# You should have received a copy of the GNU General Public License20# You should have received a copy of the GNU General Public License
21# along with this program. If not, see <http://www.gnu.org/licenses/>.21# along with this program. If not, see <http://www.gnu.org/licenses/>.
2222
23import six
24
23from cloudinit import distros25from cloudinit import distros
24from cloudinit import helpers26from cloudinit import helpers
25from cloudinit import log as logging27from cloudinit import log as logging
@@ -71,7 +73,7 @@
71 nameservers = []73 nameservers = []
72 searchservers = []74 searchservers = []
73 dev_names = entries.keys()75 dev_names = entries.keys()
74 for (dev, info) in entries.iteritems():76 for (dev, info) in six.iteritems(entries):
75 net_fn = self.network_script_tpl % (dev)77 net_fn = self.network_script_tpl % (dev)
76 net_cfg = {78 net_cfg = {
77 'DEVICE': dev,79 'DEVICE': dev,
7880
=== modified file 'cloudinit/distros/sles.py'
--- cloudinit/distros/sles.py 2014-01-23 19:06:13 +0000
+++ cloudinit/distros/sles.py 2014-07-08 04:39:36 +0000
@@ -18,6 +18,8 @@
18# You should have received a copy of the GNU General Public License18# You should have received a copy of the GNU General Public License
19# along with this program. If not, see <http://www.gnu.org/licenses/>.19# along with this program. If not, see <http://www.gnu.org/licenses/>.
2020
21import six
22
21from cloudinit import distros23from cloudinit import distros
2224
23from cloudinit.distros.parsers.hostname import HostnameConf25from cloudinit.distros.parsers.hostname import HostnameConf
@@ -62,7 +64,7 @@
62 nameservers = []64 nameservers = []
63 searchservers = []65 searchservers = []
64 dev_names = entries.keys()66 dev_names = entries.keys()
65 for (dev, info) in entries.iteritems():67 for (dev, info) in six.iteritems(entries):
66 net_fn = self.network_script_tpl % (dev)68 net_fn = self.network_script_tpl % (dev)
67 mode = info.get('auto')69 mode = info.get('auto')
68 if mode and mode.lower() == 'true':70 if mode and mode.lower() == 'true':
6971
=== modified file 'cloudinit/ec2_utils.py'
--- cloudinit/ec2_utils.py 2014-02-08 20:20:33 +0000
+++ cloudinit/ec2_utils.py 2014-07-08 04:39:36 +0000
@@ -17,7 +17,6 @@
17# along with this program. If not, see <http://www.gnu.org/licenses/>.17# along with this program. If not, see <http://www.gnu.org/licenses/>.
1818
19import functools19import functools
20import httplib
21import json20import json
2221
23from cloudinit import log as logging22from cloudinit import log as logging
@@ -25,7 +24,7 @@
25from cloudinit import util24from cloudinit import util
2625
27LOG = logging.getLogger(__name__)26LOG = logging.getLogger(__name__)
28SKIP_USERDATA_CODES = frozenset([httplib.NOT_FOUND])27SKIP_USERDATA_CODES = frozenset([url_helper.NOT_FOUND])
2928
3029
31def maybe_json_object(text):30def maybe_json_object(text):
@@ -116,7 +115,7 @@
116 leaf_contents = {}115 leaf_contents = {}
117 for (field, resource) in leaves.items():116 for (field, resource) in leaves.items():
118 leaf_url = url_helper.combine_url(base_url, resource)117 leaf_url = url_helper.combine_url(base_url, resource)
119 leaf_blob = str(self._caller(leaf_url))118 leaf_blob = self._caller(leaf_url).contents
120 leaf_contents[field] = self._decode_leaf_blob(field, leaf_blob)119 leaf_contents[field] = self._decode_leaf_blob(field, leaf_blob)
121 joined = {}120 joined = {}
122 joined.update(child_contents)121 joined.update(child_contents)
@@ -153,7 +152,7 @@
153 timeout=timeout,152 timeout=timeout,
154 retries=retries,153 retries=retries,
155 exception_cb=exception_cb)154 exception_cb=exception_cb)
156 user_data = str(response)155 user_data = response.contents
157 except url_helper.UrlError as e:156 except url_helper.UrlError as e:
158 if e.code not in SKIP_USERDATA_CODES:157 if e.code not in SKIP_USERDATA_CODES:
159 util.logexc(LOG, "Failed fetching userdata from url %s", ud_url)158 util.logexc(LOG, "Failed fetching userdata from url %s", ud_url)
@@ -173,7 +172,7 @@
173172
174 try:173 try:
175 response = caller(md_url)174 response = caller(md_url)
176 materializer = MetadataMaterializer(str(response), md_url, caller)175 materializer = MetadataMaterializer(response.contents, md_url, caller)
177 md = materializer.materialize()176 md = materializer.materialize()
178 if not isinstance(md, (dict)):177 if not isinstance(md, (dict)):
179 md = {}178 md = {}
180179
=== modified file 'cloudinit/handlers/__init__.py'
--- cloudinit/handlers/__init__.py 2014-01-16 21:57:21 +0000
+++ cloudinit/handlers/__init__.py 2014-07-08 04:39:36 +0000
@@ -147,7 +147,7 @@
147 if not modfname.endswith(".py"):147 if not modfname.endswith(".py"):
148 modfname = "%s.py" % (modfname)148 modfname = "%s.py" % (modfname)
149 # TODO(harlowja): Check if path exists??149 # TODO(harlowja): Check if path exists??
150 util.write_file(modfname, payload, 0600)150 util.write_file(modfname, payload, 0o600)
151 handlers = pdata['handlers']151 handlers = pdata['handlers']
152 try:152 try:
153 mod = fixup_handler(importer.import_module(modname))153 mod = fixup_handler(importer.import_module(modname))
154154
=== modified file 'cloudinit/handlers/boot_hook.py'
--- cloudinit/handlers/boot_hook.py 2013-07-21 16:34:26 +0000
+++ cloudinit/handlers/boot_hook.py 2014-07-08 04:39:36 +0000
@@ -50,7 +50,7 @@
50 filepath = os.path.join(self.boothook_dir, filename)50 filepath = os.path.join(self.boothook_dir, filename)
51 contents = util.strip_prefix_suffix(util.dos2unix(payload),51 contents = util.strip_prefix_suffix(util.dos2unix(payload),
52 prefix=BOOTHOOK_PREFIX)52 prefix=BOOTHOOK_PREFIX)
53 util.write_file(filepath, contents.lstrip(), 0700)53 util.write_file(filepath, contents.lstrip(), 0o700)
54 return filepath54 return filepath
5555
56 def handle_part(self, _data, ctype, filename, # pylint: disable=W022156 def handle_part(self, _data, ctype, filename, # pylint: disable=W0221
5757
=== modified file 'cloudinit/handlers/cloud_config.py'
--- cloudinit/handlers/cloud_config.py 2014-01-09 00:16:24 +0000
+++ cloudinit/handlers/cloud_config.py 2014-07-08 04:39:36 +0000
@@ -95,7 +95,7 @@
95 lines.append(util.yaml_dumps(self.cloud_buf))95 lines.append(util.yaml_dumps(self.cloud_buf))
96 else:96 else:
97 lines = []97 lines = []
98 util.write_file(self.cloud_fn, "\n".join(lines), 0600)98 util.write_file(self.cloud_fn, "\n".join(lines), 0o600)
9999
100 def _extract_mergers(self, payload, headers):100 def _extract_mergers(self, payload, headers):
101 merge_header_headers = ''101 merge_header_headers = ''
102102
=== modified file 'cloudinit/handlers/shell_script.py'
--- cloudinit/handlers/shell_script.py 2014-01-09 00:16:24 +0000
+++ cloudinit/handlers/shell_script.py 2014-07-08 04:39:36 +0000
@@ -53,4 +53,4 @@
53 filename = util.clean_filename(filename)53 filename = util.clean_filename(filename)
54 payload = util.dos2unix(payload)54 payload = util.dos2unix(payload)
55 path = os.path.join(self.script_dir, filename)55 path = os.path.join(self.script_dir, filename)
56 util.write_file(path, payload, 0700)56 util.write_file(path, payload, 0o700)
5757
=== modified file 'cloudinit/handlers/upstart_job.py'
--- cloudinit/handlers/upstart_job.py 2013-07-21 16:26:44 +0000
+++ cloudinit/handlers/upstart_job.py 2014-07-08 04:39:36 +0000
@@ -66,7 +66,7 @@
6666
67 payload = util.dos2unix(payload)67 payload = util.dos2unix(payload)
68 path = os.path.join(self.upstart_dir, filename)68 path = os.path.join(self.upstart_dir, filename)
69 util.write_file(path, payload, 0644)69 util.write_file(path, payload, 0o644)
7070
71 if SUITABLE_UPSTART:71 if SUITABLE_UPSTART:
72 util.subp(["initctl", "reload-configuration"], capture=False)72 util.subp(["initctl", "reload-configuration"], capture=False)
7373
=== modified file 'cloudinit/helpers.py'
--- cloudinit/helpers.py 2014-01-17 20:12:31 +0000
+++ cloudinit/helpers.py 2014-07-08 04:39:36 +0000
@@ -23,10 +23,12 @@
23from time import time23from time import time
2424
25import contextlib25import contextlib
26import io
27import os26import os
2827
29from ConfigParser import (NoSectionError, NoOptionError, RawConfigParser)28import six
29from six.moves.configparser import (NoSectionError,
30 NoOptionError,
31 RawConfigParser)
3032
31from cloudinit.settings import (PER_INSTANCE, PER_ALWAYS, PER_ONCE,33from cloudinit.settings import (PER_INSTANCE, PER_ALWAYS, PER_ONCE,
32 CFG_ENV_NAME)34 CFG_ENV_NAME)
@@ -318,10 +320,10 @@
318 return self.registered[content_type]320 return self.registered[content_type]
319321
320 def items(self):322 def items(self):
321 return self.registered.items()323 return list(self.iteritems())
322324
323 def iteritems(self):325 def iteritems(self):
324 return self.registered.iteritems()326 return six.iteritems(self.registered)
325327
326328
327class Paths(object):329class Paths(object):
@@ -449,7 +451,7 @@
449451
450 def stringify(self, header=None):452 def stringify(self, header=None):
451 contents = ''453 contents = ''
452 with io.BytesIO() as outputstream:454 with six.StringIO() as outputstream:
453 self.write(outputstream)455 self.write(outputstream)
454 outputstream.flush()456 outputstream.flush()
455 contents = outputstream.getvalue()457 contents = outputstream.getvalue()
456458
=== modified file 'cloudinit/log.py'
--- cloudinit/log.py 2013-04-17 16:42:55 +0000
+++ cloudinit/log.py 2014-07-08 04:39:36 +0000
@@ -28,7 +28,8 @@
28import os28import os
29import sys29import sys
3030
31from StringIO import StringIO31import six
32from six import StringIO
3233
33# Logging levels for easy access34# Logging levels for easy access
34CRITICAL = logging.CRITICAL35CRITICAL = logging.CRITICAL
@@ -72,13 +73,13 @@
7273
73 log_cfgs = []74 log_cfgs = []
74 log_cfg = cfg.get('logcfg')75 log_cfg = cfg.get('logcfg')
75 if log_cfg and isinstance(log_cfg, (str, basestring)):76 if log_cfg and isinstance(log_cfg, six.string_types):
76 # If there is a 'logcfg' entry in the config,77 # If there is a 'logcfg' entry in the config,
77 # respect it, it is the old keyname78 # respect it, it is the old keyname
78 log_cfgs.append(str(log_cfg))79 log_cfgs.append(str(log_cfg))
79 elif "log_cfgs" in cfg:80 elif "log_cfgs" in cfg:
80 for a_cfg in cfg['log_cfgs']:81 for a_cfg in cfg['log_cfgs']:
81 if isinstance(a_cfg, (basestring, str)):82 if isinstance(a_cfg, six.string_types):
82 log_cfgs.append(a_cfg)83 log_cfgs.append(a_cfg)
83 elif isinstance(a_cfg, (collections.Iterable)):84 elif isinstance(a_cfg, (collections.Iterable)):
84 cfg_str = [str(c) for c in a_cfg]85 cfg_str = [str(c) for c in a_cfg]
8586
=== modified file 'cloudinit/mergers/__init__.py'
--- cloudinit/mergers/__init__.py 2013-05-03 21:41:28 +0000
+++ cloudinit/mergers/__init__.py 2014-07-08 04:39:36 +0000
@@ -18,6 +18,8 @@
1818
19import re19import re
2020
21import six
22
21from cloudinit import importer23from cloudinit import importer
22from cloudinit import log as logging24from cloudinit import log as logging
23from cloudinit import type_utils25from cloudinit import type_utils
@@ -100,7 +102,7 @@
100 raw_mergers = config.pop('merge_type', None)102 raw_mergers = config.pop('merge_type', None)
101 if raw_mergers is None:103 if raw_mergers is None:
102 return parsed_mergers104 return parsed_mergers
103 if isinstance(raw_mergers, (str, basestring)):105 if isinstance(raw_mergers, six.string_types):
104 return string_extract_mergers(raw_mergers)106 return string_extract_mergers(raw_mergers)
105 for m in raw_mergers:107 for m in raw_mergers:
106 if isinstance(m, (dict)):108 if isinstance(m, (dict)):
107109
=== modified file 'cloudinit/mergers/m_dict.py'
--- cloudinit/mergers/m_dict.py 2013-05-03 22:05:45 +0000
+++ cloudinit/mergers/m_dict.py 2014-07-08 04:39:36 +0000
@@ -16,6 +16,8 @@
16# You should have received a copy of the GNU General Public License16# You should have received a copy of the GNU General Public License
17# along with this program. If not, see <http://www.gnu.org/licenses/>.17# along with this program. If not, see <http://www.gnu.org/licenses/>.
1818
19import six
20
19DEF_MERGE_TYPE = 'no_replace'21DEF_MERGE_TYPE = 'no_replace'
20MERGE_TYPES = ('replace', DEF_MERGE_TYPE,)22MERGE_TYPES = ('replace', DEF_MERGE_TYPE,)
2123
@@ -57,7 +59,7 @@
57 return new_v59 return new_v
58 if isinstance(new_v, (list, tuple)) and self._recurse_array:60 if isinstance(new_v, (list, tuple)) and self._recurse_array:
59 return self._merger.merge(old_v, new_v)61 return self._merger.merge(old_v, new_v)
60 if isinstance(new_v, (basestring)) and self._recurse_str:62 if isinstance(new_v, six.string_types) and self._recurse_str:
61 return self._merger.merge(old_v, new_v)63 return self._merger.merge(old_v, new_v)
62 if isinstance(new_v, (dict)) and self._recurse_dict:64 if isinstance(new_v, (dict)) and self._recurse_dict:
63 return self._merger.merge(old_v, new_v)65 return self._merger.merge(old_v, new_v)
6466
=== modified file 'cloudinit/mergers/m_list.py'
--- cloudinit/mergers/m_list.py 2013-06-19 06:46:54 +0000
+++ cloudinit/mergers/m_list.py 2014-07-08 04:39:36 +0000
@@ -16,6 +16,8 @@
16# You should have received a copy of the GNU General Public License16# You should have received a copy of the GNU General Public License
17# along with this program. If not, see <http://www.gnu.org/licenses/>.17# along with this program. If not, see <http://www.gnu.org/licenses/>.
1818
19import six
20
19DEF_MERGE_TYPE = 'replace'21DEF_MERGE_TYPE = 'replace'
20MERGE_TYPES = ('append', 'prepend', DEF_MERGE_TYPE, 'no_replace')22MERGE_TYPES = ('append', 'prepend', DEF_MERGE_TYPE, 'no_replace')
2123
@@ -73,7 +75,7 @@
73 return old_v75 return old_v
74 if isinstance(new_v, (list, tuple)) and self._recurse_array:76 if isinstance(new_v, (list, tuple)) and self._recurse_array:
75 return self._merger.merge(old_v, new_v)77 return self._merger.merge(old_v, new_v)
76 if isinstance(new_v, (str, basestring)) and self._recurse_str:78 if isinstance(new_v, six.string_types) and self._recurse_str:
77 return self._merger.merge(old_v, new_v)79 return self._merger.merge(old_v, new_v)
78 if isinstance(new_v, (dict)) and self._recurse_dict:80 if isinstance(new_v, (dict)) and self._recurse_dict:
79 return self._merger.merge(old_v, new_v)81 return self._merger.merge(old_v, new_v)
8082
=== modified file 'cloudinit/mergers/m_str.py'
--- cloudinit/mergers/m_str.py 2013-05-03 21:41:28 +0000
+++ cloudinit/mergers/m_str.py 2014-07-08 04:39:36 +0000
@@ -17,6 +17,8 @@
17# You should have received a copy of the GNU General Public License17# You should have received a copy of the GNU General Public License
18# along with this program. If not, see <http://www.gnu.org/licenses/>.18# along with this program. If not, see <http://www.gnu.org/licenses/>.
1919
20import six
21
2022
21class Merger(object):23class Merger(object):
22 def __init__(self, _merger, opts):24 def __init__(self, _merger, opts):
@@ -34,11 +36,11 @@
34 # perform the following action, if appending we will36 # perform the following action, if appending we will
35 # merge them together, otherwise we will just return value.37 # merge them together, otherwise we will just return value.
36 def _on_str(self, value, merge_with):38 def _on_str(self, value, merge_with):
37 if not isinstance(value, (basestring)):39 if not isinstance(value, six.string_types):
38 return merge_with40 return merge_with
39 if not self._append:41 if not self._append:
40 return merge_with42 return merge_with
41 if isinstance(value, unicode):43 if isinstance(value, six.text_type):
42 return value + unicode(merge_with)44 return value + six.text_type(merge_with)
43 else:45 else:
44 return value + str(merge_with)46 return value + six.binary_type(merge_with)
4547
=== modified file 'cloudinit/netinfo.py'
--- cloudinit/netinfo.py 2014-02-26 18:14:55 +0000
+++ cloudinit/netinfo.py 2014-07-08 04:39:36 +0000
@@ -20,10 +20,12 @@
20# You should have received a copy of the GNU General Public License20# You should have received a copy of the GNU General Public License
21# along with this program. If not, see <http://www.gnu.org/licenses/>.21# along with this program. If not, see <http://www.gnu.org/licenses/>.
2222
23import cloudinit.util as util
24import re23import re
2524
26from prettytable import PrettyTable25from prettytable import PrettyTable
26import six
27
28import cloudinit.util as util
2729
2830
29def netdev_info(empty=""):31def netdev_info(empty=""):
@@ -83,7 +85,7 @@
83 devs[curdev][target] = toks[i][len(field) + 1:]85 devs[curdev][target] = toks[i][len(field) + 1:]
8486
85 if empty != "":87 if empty != "":
86 for (_devname, dev) in devs.iteritems():88 for (_devname, dev) in six.iteritems(devs):
87 for field in dev:89 for field in dev:
88 if dev[field] == "":90 if dev[field] == "":
89 dev[field] = empty91 dev[field] = empty
@@ -155,7 +157,7 @@
155 if netdev is not None:157 if netdev is not None:
156 fields = ['Device', 'Up', 'Address', 'Mask', 'Hw-Address']158 fields = ['Device', 'Up', 'Address', 'Mask', 'Hw-Address']
157 tbl = PrettyTable(fields)159 tbl = PrettyTable(fields)
158 for (dev, d) in netdev.iteritems():160 for (dev, d) in six.iteritems(netdev):
159 tbl.add_row([dev, d["up"], d["addr"], d["mask"], d["hwaddr"]])161 tbl.add_row([dev, d["up"], d["addr"], d["mask"], d["hwaddr"]])
160 netdev_s = tbl.get_string()162 netdev_s = tbl.get_string()
161 max_len = len(max(netdev_s.splitlines(), key=len))163 max_len = len(max(netdev_s.splitlines(), key=len))
162164
=== modified file 'cloudinit/signal_handler.py'
--- cloudinit/signal_handler.py 2012-09-19 20:33:56 +0000
+++ cloudinit/signal_handler.py 2014-07-08 04:39:36 +0000
@@ -22,7 +22,7 @@
22import signal22import signal
23import sys23import sys
2424
25from StringIO import StringIO25from six import StringIO
2626
27from cloudinit import log as logging27from cloudinit import log as logging
28from cloudinit import util28from cloudinit import util
2929
=== modified file 'cloudinit/sources/DataSourceConfigDrive.py'
--- cloudinit/sources/DataSourceConfigDrive.py 2014-02-25 01:17:07 +0000
+++ cloudinit/sources/DataSourceConfigDrive.py 2014-07-08 04:39:36 +0000
@@ -20,6 +20,8 @@
2020
21import os21import os
2222
23import six
24
23from cloudinit import log as logging25from cloudinit import log as logging
24from cloudinit import sources26from cloudinit import sources
25from cloudinit import util27from cloudinit import util
@@ -198,7 +200,7 @@
198 files = data.get('files', {})200 files = data.get('files', {})
199 if files:201 if files:
200 LOG.debug("Writing %s injected files", len(files))202 LOG.debug("Writing %s injected files", len(files))
201 for (filename, content) in files.iteritems():203 for (filename, content) in six.iteritems(files):
202 if not filename.startswith(os.sep):204 if not filename.startswith(os.sep):
203 filename = os.sep + filename205 filename = os.sep + filename
204 try:206 try:
205207
=== modified file 'cloudinit/sources/DataSourceEc2.py'
--- cloudinit/sources/DataSourceEc2.py 2014-02-01 20:03:32 +0000
+++ cloudinit/sources/DataSourceEc2.py 2014-07-08 04:39:36 +0000
@@ -23,6 +23,8 @@
23import os23import os
24import time24import time
2525
26import six
27
26from cloudinit import ec2_utils as ec228from cloudinit import ec2_utils as ec2
27from cloudinit import log as logging29from cloudinit import log as logging
28from cloudinit import sources30from cloudinit import sources
@@ -156,8 +158,8 @@
156 # 'ephemeral0': '/dev/sdb',158 # 'ephemeral0': '/dev/sdb',
157 # 'root': '/dev/sda1'}159 # 'root': '/dev/sda1'}
158 found = None160 found = None
159 bdm_items = self.metadata['block-device-mapping'].iteritems()161 bdms = self.metadata['block-device-mapping']
160 for (entname, device) in bdm_items:162 for (entname, device) in six.iteritems(bdms):
161 if entname == name:163 if entname == name:
162 found = device164 found = device
163 break165 break
164166
=== modified file 'cloudinit/sources/DataSourceMAAS.py'
--- cloudinit/sources/DataSourceMAAS.py 2013-04-25 15:58:38 +0000
+++ cloudinit/sources/DataSourceMAAS.py 2014-07-08 04:39:36 +0000
@@ -20,11 +20,14 @@
2020
21from email.utils import parsedate21from email.utils import parsedate
22import errno22import errno
23import oauth.oauth as oauth
24import os23import os
25import time24import time
26import urllib225import urllib2
2726
27import oauth.oauth as oauth
28
29import six
30
28from cloudinit import log as logging31from cloudinit import log as logging
29from cloudinit import sources32from cloudinit import sources
30from cloudinit import url_helper33from cloudinit import url_helper
@@ -262,7 +265,7 @@
262265
263 userdata = content.get('user-data', "")266 userdata = content.get('user-data', "")
264 md = {}267 md = {}
265 for (key, val) in content.iteritems():268 for (key, val) in six.iteritems(content):
266 if key == 'user-data':269 if key == 'user-data':
267 continue270 continue
268 md[key] = val271 md[key] = val
269272
=== modified file 'cloudinit/sources/DataSourceOVF.py'
--- cloudinit/sources/DataSourceOVF.py 2013-06-07 17:30:03 +0000
+++ cloudinit/sources/DataSourceOVF.py 2014-07-08 04:39:36 +0000
@@ -26,6 +26,8 @@
26import os26import os
27import re27import re
2828
29import six
30
29from cloudinit import log as logging31from cloudinit import log as logging
30from cloudinit import sources32from cloudinit import sources
31from cloudinit import util33from cloudinit import util
@@ -66,7 +68,7 @@
66 np = {'iso': transport_iso9660,68 np = {'iso': transport_iso9660,
67 'vmware-guestd': transport_vmware_guestd, }69 'vmware-guestd': transport_vmware_guestd, }
68 name = None70 name = None
69 for (name, transfunc) in np.iteritems():71 for (name, transfunc) in six.iteritems(np):
70 (contents, _dev, _fname) = transfunc()72 (contents, _dev, _fname) = transfunc()
71 if contents:73 if contents:
72 break74 break
@@ -138,7 +140,7 @@
138 ud = ""140 ud = ""
139 cfg_props = ['password']141 cfg_props = ['password']
140 md_props = ['seedfrom', 'local-hostname', 'public-keys', 'instance-id']142 md_props = ['seedfrom', 'local-hostname', 'public-keys', 'instance-id']
141 for (prop, val) in props.iteritems():143 for (prop, val) in six.iteritems(props):
142 if prop == 'hostname':144 if prop == 'hostname':
143 prop = "local-hostname"145 prop = "local-hostname"
144 if prop in md_props:146 if prop in md_props:
@@ -183,7 +185,7 @@
183185
184 # Go through mounts to see if it was already mounted186 # Go through mounts to see if it was already mounted
185 mounts = util.mounts()187 mounts = util.mounts()
186 for (dev, info) in mounts.iteritems():188 for (dev, info) in six.iteritems(mounts):
187 fstype = info['fstype']189 fstype = info['fstype']
188 if fstype != "iso9660" and require_iso:190 if fstype != "iso9660" and require_iso:
189 continue191 continue
190192
=== modified file 'cloudinit/sources/DataSourceSmartOS.py'
--- cloudinit/sources/DataSourceSmartOS.py 2014-06-02 20:56:31 +0000
+++ cloudinit/sources/DataSourceSmartOS.py 2014-07-08 04:39:36 +0000
@@ -30,13 +30,15 @@
30# Comments with "@datadictionary" are snippets of the definition30# Comments with "@datadictionary" are snippets of the definition
3131
32import base6432import base64
33import os
34import os.path
35
36import serial
37import six
38
33from cloudinit import log as logging39from cloudinit import log as logging
34from cloudinit import sources40from cloudinit import sources
35from cloudinit import util41from cloudinit import util
36import os
37import os.path
38import serial
39
4042
41LOG = logging.getLogger(__name__)43LOG = logging.getLogger(__name__)
4244
@@ -201,7 +203,7 @@
201 if b64_all is not None:203 if b64_all is not None:
202 self.b64_all = util.is_true(b64_all)204 self.b64_all = util.is_true(b64_all)
203205
204 for ci_noun, attribute in SMARTOS_ATTRIB_MAP.iteritems():206 for ci_noun, attribute in six.iteritems(SMARTOS_ATTRIB_MAP):
205 smartos_noun, strip = attribute207 smartos_noun, strip = attribute
206 md[ci_noun] = self.query(smartos_noun, strip=strip)208 md[ci_noun] = self.query(smartos_noun, strip=strip)
207209
208210
=== modified file 'cloudinit/sources/__init__.py'
--- cloudinit/sources/__init__.py 2014-01-23 20:17:17 +0000
+++ cloudinit/sources/__init__.py 2014-07-08 04:39:36 +0000
@@ -23,6 +23,8 @@
23import abc23import abc
24import os24import os
2525
26import six
27
26from cloudinit import importer28from cloudinit import importer
27from cloudinit import log as logging29from cloudinit import log as logging
28from cloudinit import type_utils30from cloudinit import type_utils
@@ -130,7 +132,7 @@
130 # we want to return the correct value for what will actually132 # we want to return the correct value for what will actually
131 # exist in this instance133 # exist in this instance
132 mappings = {"sd": ("vd", "xvd", "vtb")}134 mappings = {"sd": ("vd", "xvd", "vtb")}
133 for (nfrom, tlist) in mappings.iteritems():135 for (nfrom, tlist) in six.iteritems(mappings):
134 if not short_name.startswith(nfrom):136 if not short_name.startswith(nfrom):
135 continue137 continue
136 for nto in tlist:138 for nto in tlist:
@@ -218,18 +220,18 @@
218 if not pubkey_data:220 if not pubkey_data:
219 return keys221 return keys
220222
221 if isinstance(pubkey_data, (basestring, str)):223 if isinstance(pubkey_data, six.string_types):
222 return str(pubkey_data).splitlines()224 return str(pubkey_data).splitlines()
223225
224 if isinstance(pubkey_data, (list, set)):226 if isinstance(pubkey_data, (list, set)):
225 return list(pubkey_data)227 return list(pubkey_data)
226228
227 if isinstance(pubkey_data, (dict)):229 if isinstance(pubkey_data, (dict)):
228 for (_keyname, klist) in pubkey_data.iteritems():230 for (_keyname, klist) in six.iteritems(pubkey_data):
229 # lp:506332 uec metadata service responds with231 # lp:506332 uec metadata service responds with
230 # data that makes boto populate a string for 'klist' rather232 # data that makes boto populate a string for 'klist' rather
231 # than a list.233 # than a list.
232 if isinstance(klist, (str, basestring)):234 if isinstance(klist, six.string_types):
233 klist = [klist]235 klist = [klist]
234 if isinstance(klist, (list, set)):236 if isinstance(klist, (list, set)):
235 for pkey in klist:237 for pkey in klist:
236238
=== modified file 'cloudinit/sources/helpers/openstack.py'
--- cloudinit/sources/helpers/openstack.py 2014-02-24 22:41:42 +0000
+++ cloudinit/sources/helpers/openstack.py 2014-07-08 04:39:36 +0000
@@ -23,6 +23,8 @@
23import copy23import copy
24import os24import os
2525
26import six
27
26from cloudinit import ec2_utils28from cloudinit import ec2_utils
27from cloudinit import log as logging29from cloudinit import log as logging
28from cloudinit import sources30from cloudinit import sources
@@ -224,7 +226,7 @@
224 'version': 2,226 'version': 2,
225 }227 }
226 data = datafiles(version)228 data = datafiles(version)
227 for (name, (path, required, translator)) in data.iteritems():229 for (name, (path, required, translator)) in six.iteritems(data):
228 path = self._path_join(self.base_path, path)230 path = self._path_join(self.base_path, path)
229 data = None231 data = None
230 found = False232 found = False
@@ -344,7 +346,7 @@
344 raise NonReadable("%s: no files found" % (self.base_path))346 raise NonReadable("%s: no files found" % (self.base_path))
345347
346 md = {}348 md = {}
347 for (name, (key, translator, default)) in FILES_V1.iteritems():349 for (name, (key, translator, default)) in six.iteritems(FILES_V1):
348 if name in found:350 if name in found:
349 path = found[name]351 path = found[name]
350 try:352 try:
351353
=== modified file 'cloudinit/ssh_util.py'
--- cloudinit/ssh_util.py 2013-06-19 06:44:00 +0000
+++ cloudinit/ssh_util.py 2014-07-08 04:39:36 +0000
@@ -239,7 +239,7 @@
239 # Make sure the users .ssh dir is setup accordingly239 # Make sure the users .ssh dir is setup accordingly
240 (ssh_dir, pwent) = users_ssh_info(username)240 (ssh_dir, pwent) = users_ssh_info(username)
241 if not os.path.isdir(ssh_dir):241 if not os.path.isdir(ssh_dir):
242 util.ensure_dir(ssh_dir, mode=0700)242 util.ensure_dir(ssh_dir, mode=0o700)
243 util.chownbyid(ssh_dir, pwent.pw_uid, pwent.pw_gid)243 util.chownbyid(ssh_dir, pwent.pw_uid, pwent.pw_gid)
244244
245 # Turn the 'update' keys given into actual entries245 # Turn the 'update' keys given into actual entries
@@ -252,8 +252,8 @@
252 (auth_key_fn, auth_key_entries) = extract_authorized_keys(username)252 (auth_key_fn, auth_key_entries) = extract_authorized_keys(username)
253 with util.SeLinuxGuard(ssh_dir, recursive=True):253 with util.SeLinuxGuard(ssh_dir, recursive=True):
254 content = update_authorized_keys(auth_key_entries, key_entries)254 content = update_authorized_keys(auth_key_entries, key_entries)
255 util.ensure_dir(os.path.dirname(auth_key_fn), mode=0700)255 util.ensure_dir(os.path.dirname(auth_key_fn), mode=0o700)
256 util.write_file(auth_key_fn, content, mode=0600)256 util.write_file(auth_key_fn, content, mode=0o600)
257 util.chownbyid(auth_key_fn, pwent.pw_uid, pwent.pw_gid)257 util.chownbyid(auth_key_fn, pwent.pw_uid, pwent.pw_gid)
258258
259259
260260
=== modified file 'cloudinit/stages.py'
--- cloudinit/stages.py 2014-02-13 18:53:08 +0000
+++ cloudinit/stages.py 2014-07-08 04:39:36 +0000
@@ -20,12 +20,13 @@
20# You should have received a copy of the GNU General Public License20# You should have received a copy of the GNU General Public License
21# along with this program. If not, see <http://www.gnu.org/licenses/>.21# along with this program. If not, see <http://www.gnu.org/licenses/>.
2222
23import cPickle as pickle
24
25import copy23import copy
26import os24import os
25import pickle
27import sys26import sys
2827
28import six
29
29from cloudinit.settings import (PER_INSTANCE, FREQUENCIES, CLOUD_CONFIG)30from cloudinit.settings import (PER_INSTANCE, FREQUENCIES, CLOUD_CONFIG)
3031
31from cloudinit import handlers32from cloudinit import handlers
@@ -202,7 +203,7 @@
202 util.logexc(LOG, "Failed pickling datasource %s", self.datasource)203 util.logexc(LOG, "Failed pickling datasource %s", self.datasource)
203 return False204 return False
204 try:205 try:
205 util.write_file(pickled_fn, pk_contents, mode=0400)206 util.write_file(pickled_fn, pk_contents, mode=0o400)
206 except Exception:207 except Exception:
207 util.logexc(LOG, "Failed pickling datasource to %s", pickled_fn)208 util.logexc(LOG, "Failed pickling datasource to %s", pickled_fn)
208 return False209 return False
@@ -324,15 +325,15 @@
324325
325 def _store_userdata(self):326 def _store_userdata(self):
326 raw_ud = "%s" % (self.datasource.get_userdata_raw())327 raw_ud = "%s" % (self.datasource.get_userdata_raw())
327 util.write_file(self._get_ipath('userdata_raw'), raw_ud, 0600)328 util.write_file(self._get_ipath('userdata_raw'), raw_ud, 0o600)
328 processed_ud = "%s" % (self.datasource.get_userdata())329 processed_ud = "%s" % (self.datasource.get_userdata())
329 util.write_file(self._get_ipath('userdata'), processed_ud, 0600)330 util.write_file(self._get_ipath('userdata'), processed_ud, 0o600)
330331
331 def _store_vendordata(self):332 def _store_vendordata(self):
332 raw_vd = "%s" % (self.datasource.get_vendordata_raw())333 raw_vd = "%s" % (self.datasource.get_vendordata_raw())
333 util.write_file(self._get_ipath('vendordata_raw'), raw_vd, 0600)334 util.write_file(self._get_ipath('vendordata_raw'), raw_vd, 0o600)
334 processed_vd = "%s" % (self.datasource.get_vendordata())335 processed_vd = "%s" % (self.datasource.get_vendordata())
335 util.write_file(self._get_ipath('vendordata'), processed_vd, 0600)336 util.write_file(self._get_ipath('vendordata'), processed_vd, 0o600)
336337
337 def _default_handlers(self, opts=None):338 def _default_handlers(self, opts=None):
338 if opts is None:339 if opts is None:
@@ -384,7 +385,7 @@
384 if not path or not os.path.isdir(path):385 if not path or not os.path.isdir(path):
385 return386 return
386 potential_handlers = util.find_modules(path)387 potential_handlers = util.find_modules(path)
387 for (fname, mod_name) in potential_handlers.iteritems():388 for (fname, mod_name) in six.iteritems(potential_handlers):
388 try:389 try:
389 mod_locs = importer.find_module(mod_name, [''],390 mod_locs = importer.find_module(mod_name, [''],
390 ['list_types',391 ['list_types',
@@ -574,7 +575,7 @@
574 for item in cfg_mods:575 for item in cfg_mods:
575 if not item:576 if not item:
576 continue577 continue
577 if isinstance(item, (str, basestring)):578 if isinstance(item, six.string_types):
578 module_list.append({579 module_list.append({
579 'mod': item.strip(),580 'mod': item.strip(),
580 })581 })
581582
=== modified file 'cloudinit/type_utils.py'
--- cloudinit/type_utils.py 2013-03-07 03:24:05 +0000
+++ cloudinit/type_utils.py 2014-07-08 04:39:36 +0000
@@ -24,11 +24,30 @@
2424
25import types25import types
2626
27import six
28
29if six.PY3:
30 _NAME_TYPES = (
31 types.ModuleType,
32 types.FunctionType,
33 types.LambdaType,
34 type,
35 )
36else:
37 _NAME_TYPES = (
38 types.TypeType,
39 types.ModuleType,
40 types.FunctionType,
41 types.LambdaType,
42 types.ClassType,
43 )
44
2745
28def obj_name(obj):46def obj_name(obj):
29 if isinstance(obj, (types.TypeType,47 if isinstance(obj, _NAME_TYPES):
30 types.ModuleType,48 return six.text_type(obj.__name__)
31 types.FunctionType,49 else:
32 types.LambdaType)):50 if not hasattr(obj, '__class__'):
33 return str(obj.__name__)51 return repr(obj)
34 return obj_name(obj.__class__)52 else:
53 return obj_name(obj.__class__)
3554
=== modified file 'cloudinit/url_helper.py'
--- cloudinit/url_helper.py 2014-02-13 17:13:42 +0000
+++ cloudinit/url_helper.py 2014-07-08 04:39:36 +0000
@@ -20,21 +20,28 @@
20# You should have received a copy of the GNU General Public License20# You should have received a copy of the GNU General Public License
21# along with this program. If not, see <http://www.gnu.org/licenses/>.21# along with this program. If not, see <http://www.gnu.org/licenses/>.
2222
23import httplib23import six
24
24import time25import time
25import urllib
2626
27import requests27import requests
28from requests import exceptions28from requests import exceptions
2929
30from urlparse import (urlparse, urlunparse)30from six.moves.urllib.parse import (urlparse, urlunparse)
31from six.moves.urllib.parse import quote as urlquote
3132
32from cloudinit import log as logging33from cloudinit import log as logging
33from cloudinit import version34from cloudinit import version
3435
35LOG = logging.getLogger(__name__)36LOG = logging.getLogger(__name__)
3637
37NOT_FOUND = httplib.NOT_FOUND38if six.PY2:
39 import httplib
40 NOT_FOUND = httplib.NOT_FOUND
41else:
42 import http.client
43 NOT_FOUND = http.client.NOT_FOUND
44
3845
39# Check if requests has ssl support (added in requests >= 0.8.8)46# Check if requests has ssl support (added in requests >= 0.8.8)
40SSL_ENABLED = False47SSL_ENABLED = False
@@ -70,7 +77,7 @@
70 path = url_parsed[2]77 path = url_parsed[2]
71 if path and not path.endswith("/"):78 if path and not path.endswith("/"):
72 path += "/"79 path += "/"
73 path += urllib.quote(str(add_on), safe="/:")80 path += urlquote(str(add_on), safe="/:")
74 url_parsed[2] = path81 url_parsed[2] = path
75 return urlunparse(url_parsed)82 return urlunparse(url_parsed)
7683
@@ -111,7 +118,7 @@
111118
112 @property119 @property
113 def contents(self):120 def contents(self):
114 return self._response.content121 return self._response.text
115122
116 @property123 @property
117 def url(self):124 def url(self):
@@ -135,7 +142,7 @@
135 return self._response.status_code142 return self._response.status_code
136143
137 def __str__(self):144 def __str__(self):
138 return self.contents145 return self._response.text
139146
140147
141class UrlError(IOError):148class UrlError(IOError):
142149
=== modified file 'cloudinit/user_data.py'
--- cloudinit/user_data.py 2014-01-24 20:29:09 +0000
+++ cloudinit/user_data.py 2014-07-08 04:39:36 +0000
@@ -29,6 +29,8 @@
29from email.mime.nonmultipart import MIMENonMultipart29from email.mime.nonmultipart import MIMENonMultipart
30from email.mime.text import MIMEText30from email.mime.text import MIMEText
3131
32import six
33
32from cloudinit import handlers34from cloudinit import handlers
33from cloudinit import log as logging35from cloudinit import log as logging
34from cloudinit import util36from cloudinit import util
@@ -235,9 +237,9 @@
235 resp = util.read_file_or_url(include_url,237 resp = util.read_file_or_url(include_url,
236 ssl_details=self.ssl_details)238 ssl_details=self.ssl_details)
237 if include_once_on and resp.ok():239 if include_once_on and resp.ok():
238 util.write_file(include_once_fn, str(resp), mode=0600)240 util.write_file(include_once_fn, resp.contents, mode=0o600)
239 if resp.ok():241 if resp.ok():
240 content = str(resp)242 content = resp.contents
241 else:243 else:
242 LOG.warn(("Fetching from %s resulted in"244 LOG.warn(("Fetching from %s resulted in"
243 " a invalid http code of %s"),245 " a invalid http code of %s"),
@@ -256,7 +258,7 @@
256 # filename and type not be present258 # filename and type not be present
257 # or259 # or
258 # scalar(payload)260 # scalar(payload)
259 if isinstance(ent, (str, basestring)):261 if isinstance(ent, six.string_types):
260 ent = {'content': ent}262 ent = {'content': ent}
261 if not isinstance(ent, (dict)):263 if not isinstance(ent, (dict)):
262 # TODO(harlowja) raise?264 # TODO(harlowja) raise?
@@ -337,7 +339,7 @@
337 data = util.decomp_gzip(raw_data)339 data = util.decomp_gzip(raw_data)
338 if "mime-version:" in data[0:4096].lower():340 if "mime-version:" in data[0:4096].lower():
339 msg = email.message_from_string(data)341 msg = email.message_from_string(data)
340 for (key, val) in headers.iteritems():342 for (key, val) in six.iteritems(headers):
341 _replace_header(msg, key, val)343 _replace_header(msg, key, val)
342 else:344 else:
343 mtype = headers.get(CONTENT_TYPE, NOT_MULTIPART_TYPE)345 mtype = headers.get(CONTENT_TYPE, NOT_MULTIPART_TYPE)
344346
=== modified file 'cloudinit/util.py'
--- cloudinit/util.py 2014-02-24 22:20:12 +0000
+++ cloudinit/util.py 2014-07-08 04:39:36 +0000
@@ -22,8 +22,6 @@
22#22#
23# pylint: disable=C030223# pylint: disable=C0302
2424
25from StringIO import StringIO
26
27import contextlib25import contextlib
28import copy as obj_copy26import copy as obj_copy
29import ctypes27import ctypes
@@ -47,8 +45,10 @@
47import sys45import sys
48import tempfile46import tempfile
49import time47import time
50import urlparse48
5149from six.moves.urllib import parse as urlparse
50
51import six
52import yaml52import yaml
5353
54from cloudinit import importer54from cloudinit import importer
@@ -71,8 +71,25 @@
71}71}
72FN_ALLOWED = ('_-.()' + string.digits + string.ascii_letters)72FN_ALLOWED = ('_-.()' + string.digits + string.ascii_letters)
7373
74TRUE_STRINGS = ('true', '1', 'on', 'yes')
75FALSE_STRINGS = ('off', '0', 'no', 'false')
76
74# Helper utils to see if running in a container77# Helper utils to see if running in a container
75CONTAINER_TESTS = ['running-in-container', 'lxc-is-container']78CONTAINER_TESTS = ('running-in-container', 'lxc-is-container')
79
80
81def decode_binary(blob, encoding='utf-8'):
82 # Converts a binary type into a text type using given encoding.
83 if isinstance(blob, six.text_type):
84 return blob
85 return blob.decode(encoding)
86
87
88def encode_text(text, encoding='utf-8'):
89 # Converts a text string into a binary type using given encoding.
90 if isinstance(text, six.binary_type):
91 return text
92 return text.encode(encoding)
7693
7794
78class ProcessExecutionError(IOError):95class ProcessExecutionError(IOError):
@@ -149,7 +166,8 @@
149 if self.selinux and self.selinux.is_selinux_enabled():166 if self.selinux and self.selinux.is_selinux_enabled():
150 path = os.path.realpath(os.path.expanduser(self.path))167 path = os.path.realpath(os.path.expanduser(self.path))
151 # path should be a string, not unicode168 # path should be a string, not unicode
152 path = str(path)169 if six.PY2:
170 path = str(path)
153 do_restore = False171 do_restore = False
154 try:172 try:
155 # See if even worth restoring??173 # See if even worth restoring??
@@ -211,10 +229,10 @@
211def is_true(val, addons=None):229def is_true(val, addons=None):
212 if isinstance(val, (bool)):230 if isinstance(val, (bool)):
213 return val is True231 return val is True
214 check_set = ['true', '1', 'on', 'yes']232 check_set = TRUE_STRINGS
215 if addons:233 if addons:
216 check_set = check_set + addons234 check_set = list(check_set) + addons
217 if str(val).lower().strip() in check_set:235 if six.text_type(val).lower().strip() in check_set:
218 return True236 return True
219 return False237 return False
220238
@@ -222,10 +240,10 @@
222def is_false(val, addons=None):240def is_false(val, addons=None):
223 if isinstance(val, (bool)):241 if isinstance(val, (bool)):
224 return val is False242 return val is False
225 check_set = ['off', '0', 'no', 'false']243 check_set = FALSE_STRINGS
226 if addons:244 if addons:
227 check_set = check_set + addons245 check_set = list(check_set) + addons
228 if str(val).lower().strip() in check_set:246 if six.text_type(val).lower().strip() in check_set:
229 return True247 return True
230 return False248 return False
231249
@@ -275,7 +293,7 @@
275def uniq_merge(*lists):293def uniq_merge(*lists):
276 combined_list = []294 combined_list = []
277 for a_list in lists:295 for a_list in lists:
278 if isinstance(a_list, (str, basestring)):296 if isinstance(a_list, six.string_types):
279 a_list = a_list.strip().split(",")297 a_list = a_list.strip().split(",")
280 # Kickout the empty ones298 # Kickout the empty ones
281 a_list = [a for a in a_list if len(a)]299 a_list = [a for a in a_list if len(a)]
@@ -284,7 +302,7 @@
284302
285303
286def clean_filename(fn):304def clean_filename(fn):
287 for (k, v) in FN_REPLACEMENTS.iteritems():305 for (k, v) in six.iteritems(FN_REPLACEMENTS):
288 fn = fn.replace(k, v)306 fn = fn.replace(k, v)
289 removals = []307 removals = []
290 for k in fn:308 for k in fn:
@@ -298,14 +316,14 @@
298316
299def decomp_gzip(data, quiet=True):317def decomp_gzip(data, quiet=True):
300 try:318 try:
301 buf = StringIO(str(data))319 buf = six.BytesIO(encode_text(data))
302 with contextlib.closing(gzip.GzipFile(None, "rb", 1, buf)) as gh:320 with contextlib.closing(gzip.GzipFile(None, "rb", 1, buf)) as gh:
303 return gh.read()321 return decode_binary(gh.read())
304 except Exception as e:322 except Exception as e:
305 if quiet:323 if quiet:
306 return data324 return data
307 else:325 else:
308 raise DecompressionError(str(e))326 raise DecompressionError(six.text_type(e))
309327
310328
311def extract_usergroup(ug_pair):329def extract_usergroup(ug_pair):
@@ -364,7 +382,7 @@
364382
365383
366def load_json(text, root_types=(dict,)):384def load_json(text, root_types=(dict,)):
367 decoded = json.loads(text)385 decoded = json.loads(decode_binary(text))
368 if not isinstance(decoded, tuple(root_types)):386 if not isinstance(decoded, tuple(root_types)):
369 expected_types = ", ".join([str(t) for t in root_types])387 expected_types = ", ".join([str(t) for t in root_types])
370 raise TypeError("(%s) root types expected, got %s instead"388 raise TypeError("(%s) root types expected, got %s instead"
@@ -396,7 +414,7 @@
396 if key not in yobj:414 if key not in yobj:
397 return default415 return default
398 val = yobj[key]416 val = yobj[key]
399 if not isinstance(val, (str, basestring)):417 if not isinstance(val, six.string_types):
400 val = str(val)418 val = str(val)
401 return val419 return val
402420
@@ -431,7 +449,7 @@
431 if isinstance(val, (list)):449 if isinstance(val, (list)):
432 cval = [v for v in val]450 cval = [v for v in val]
433 return cval451 return cval
434 if not isinstance(val, (basestring)):452 if not isinstance(val, six.string_types):
435 val = str(val)453 val = str(val)
436 return [val]454 return [val]
437455
@@ -706,11 +724,11 @@
706724
707def load_yaml(blob, default=None, allowed=(dict,)):725def load_yaml(blob, default=None, allowed=(dict,)):
708 loaded = default726 loaded = default
727 blob = decode_binary(blob)
709 try:728 try:
710 blob = str(blob)729 LOG.debug("Attempting to load yaml from string "
711 LOG.debug(("Attempting to load yaml from string "730 "of length %s with allowed root types %s",
712 "of length %s with allowed root types %s"),731 len(blob), allowed)
713 len(blob), allowed)
714 converted = safeyaml.load(blob)732 converted = safeyaml.load(blob)
715 if not isinstance(converted, allowed):733 if not isinstance(converted, allowed):
716 # Yes this will just be caught, but thats ok for now...734 # Yes this will just be caught, but thats ok for now...
@@ -744,14 +762,12 @@
744 md_resp = read_file_or_url(md_url, timeout, retries, file_retries)762 md_resp = read_file_or_url(md_url, timeout, retries, file_retries)
745 md = None763 md = None
746 if md_resp.ok():764 if md_resp.ok():
747 md_str = str(md_resp)765 md = load_yaml(md_resp.contents, default={})
748 md = load_yaml(md_str, default={})
749766
750 ud_resp = read_file_or_url(ud_url, timeout, retries, file_retries)767 ud_resp = read_file_or_url(ud_url, timeout, retries, file_retries)
751 ud = None768 ud = None
752 if ud_resp.ok():769 if ud_resp.ok():
753 ud_str = str(ud_resp)770 ud = ud_resp.contents
754 ud = ud_str
755771
756 return (md, ud)772 return (md, ud)
757773
@@ -782,7 +798,7 @@
782 if "conf_d" in cfg:798 if "conf_d" in cfg:
783 confd = cfg['conf_d']799 confd = cfg['conf_d']
784 if confd:800 if confd:
785 if not isinstance(confd, (str, basestring)):801 if not isinstance(confd, six.string_types):
786 raise TypeError(("Config file %s contains 'conf_d' "802 raise TypeError(("Config file %s contains 'conf_d' "
787 "with non-string type %s") %803 "with non-string type %s") %
788 (cfgfile, type_utils.obj_name(confd)))804 (cfgfile, type_utils.obj_name(confd)))
@@ -919,8 +935,8 @@
919 return (None, None, None)935 return (None, None, None)
920936
921 resp = read_file_or_url(url)937 resp = read_file_or_url(url)
922 if resp.contents.startswith(starts) and resp.ok():938 if resp.ok() and resp.contents.startswith(starts):
923 return (key, url, str(resp))939 return (key, url, resp.contents)
924940
925 return (key, url, None)941 return (key, url, None)
926942
@@ -1074,9 +1090,9 @@
1074 return out_list1090 return out_list
10751091
10761092
1077def load_file(fname, read_cb=None, quiet=False):1093def load_file(fname, read_cb=None, quiet=False, decode=True):
1078 LOG.debug("Reading from %s (quiet=%s)", fname, quiet)1094 LOG.debug("Reading from %s (quiet=%s)", fname, quiet)
1079 ofh = StringIO()1095 ofh = six.BytesIO()
1080 try:1096 try:
1081 with open(fname, 'rb') as ifh:1097 with open(fname, 'rb') as ifh:
1082 pipe_in_out(ifh, ofh, chunk_cb=read_cb)1098 pipe_in_out(ifh, ofh, chunk_cb=read_cb)
@@ -1087,7 +1103,10 @@
1087 raise1103 raise
1088 contents = ofh.getvalue()1104 contents = ofh.getvalue()
1089 LOG.debug("Read %s bytes from %s", len(contents), fname)1105 LOG.debug("Read %s bytes from %s", len(contents), fname)
1090 return contents1106 if decode:
1107 return decode_binary(contents)
1108 else:
1109 return contents
10911110
10921111
1093def get_cmdline():1112def get_cmdline():
@@ -1217,7 +1236,7 @@
12171236
1218def hash_blob(blob, routine, mlen=None):1237def hash_blob(blob, routine, mlen=None):
1219 hasher = hashlib.new(routine)1238 hasher = hashlib.new(routine)
1220 hasher.update(blob)1239 hasher.update(encode_text(blob))
1221 digest = hasher.hexdigest()1240 digest = hasher.hexdigest()
1222 # Don't get to long now1241 # Don't get to long now
1223 if mlen is not None:1242 if mlen is not None:
@@ -1248,7 +1267,7 @@
1248 os.rename(src, dest)1267 os.rename(src, dest)
12491268
12501269
1251def ensure_dirs(dirlist, mode=0755):1270def ensure_dirs(dirlist, mode=0o755):
1252 for d in dirlist:1271 for d in dirlist:
1253 ensure_dir(d, mode)1272 ensure_dir(d, mode)
12541273
@@ -1262,7 +1281,7 @@
1262 return1281 return
1263 try:1282 try:
1264 if key and content:1283 if key and content:
1265 write_file(target_fn, content, mode=0600)1284 write_file(target_fn, content, mode=0o600)
1266 LOG.debug(("Wrote to %s with contents of command line"1285 LOG.debug(("Wrote to %s with contents of command line"
1267 " url %s (len=%s)"), target_fn, url, len(content))1286 " url %s (len=%s)"), target_fn, url, len(content))
1268 elif key and not content:1287 elif key and not content:
@@ -1454,7 +1473,7 @@
1454 write_file(path, content, omode="ab", mode=None)1473 write_file(path, content, omode="ab", mode=None)
14551474
14561475
1457def ensure_file(path, mode=0644):1476def ensure_file(path, mode=0o644):
1458 write_file(path, content='', omode="ab", mode=mode)1477 write_file(path, content='', omode="ab", mode=mode)
14591478
14601479
@@ -1472,7 +1491,7 @@
1472 os.chmod(path, real_mode)1491 os.chmod(path, real_mode)
14731492
14741493
1475def write_file(filename, content, mode=0644, omode="wb"):1494def write_file(filename, content, mode=0o644, omode="wb"):
1476 """1495 """
1477 Writes a file with the given content and sets the file mode as specified.1496 Writes a file with the given content and sets the file mode as specified.
1478 Resotres the SELinux context if possible.1497 Resotres the SELinux context if possible.
@@ -1480,11 +1499,17 @@
1480 @param filename: The full path of the file to write.1499 @param filename: The full path of the file to write.
1481 @param content: The content to write to the file.1500 @param content: The content to write to the file.
1482 @param mode: The filesystem mode to set on the file.1501 @param mode: The filesystem mode to set on the file.
1483 @param omode: The open mode used when opening the file (r, rb, a, etc.)1502 @param omode: The open mode used when opening the file (w, wb, a, etc.)
1484 """1503 """
1485 ensure_dir(os.path.dirname(filename))1504 ensure_dir(os.path.dirname(filename))
1486 LOG.debug("Writing to %s - %s: [%s] %s bytes",1505 if 'b' in omode.lower():
1487 filename, omode, mode, len(content))1506 content = encode_text(content)
1507 write_type = 'bytes'
1508 else:
1509 content = decode_binary(content)
1510 write_type = 'characters'
1511 LOG.debug("Writing to %s - %s: [%s] %s %s",
1512 filename, omode, mode, len(content), write_type)
1488 with SeLinuxGuard(path=filename):1513 with SeLinuxGuard(path=filename):
1489 with open(filename, omode) as fh:1514 with open(filename, omode) as fh:
1490 fh.write(content)1515 fh.write(content)
@@ -1573,10 +1598,10 @@
1573 if isinstance(args, list):1598 if isinstance(args, list):
1574 fixed = []1599 fixed = []
1575 for f in args:1600 for f in args:
1576 fixed.append("'%s'" % (str(f).replace("'", escaped)))1601 fixed.append("'%s'" % (six.text_type(f).replace("'", escaped)))
1577 content = "%s%s\n" % (content, ' '.join(fixed))1602 content = "%s%s\n" % (content, ' '.join(fixed))
1578 cmds_made += 11603 cmds_made += 1
1579 elif isinstance(args, (str, basestring)):1604 elif isinstance(args, six.string_types):
1580 content = "%s%s\n" % (content, args)1605 content = "%s%s\n" % (content, args)
1581 cmds_made += 11606 cmds_made += 1
1582 else:1607 else:
@@ -1687,7 +1712,7 @@
16871712
1688 pkglist = []1713 pkglist = []
1689 for pkg in pkgs:1714 for pkg in pkgs:
1690 if isinstance(pkg, basestring):1715 if isinstance(pkg, six.string_types):
1691 pkglist.append(pkg)1716 pkglist.append(pkg)
1692 continue1717 continue
16931718
16941719
=== modified file 'packages/bddeb'
--- packages/bddeb 2014-01-17 22:08:58 +0000
+++ packages/bddeb 2014-07-08 04:39:36 +0000
@@ -37,6 +37,7 @@
37 'pyserial': 'python-serial',37 'pyserial': 'python-serial',
38 'pyyaml': 'python-yaml',38 'pyyaml': 'python-yaml',
39 'requests': 'python-requests',39 'requests': 'python-requests',
40 'six': 'python-six',
40}41}
41DEBUILD_ARGS = ["-us", "-S", "-uc", "-d"]42DEBUILD_ARGS = ["-us", "-S", "-uc", "-d"]
4243
4344
=== modified file 'packages/brpm'
--- packages/brpm 2014-01-17 22:08:58 +0000
+++ packages/brpm 2014-07-08 04:39:36 +0000
@@ -44,6 +44,7 @@
44 'pyserial': 'pyserial',44 'pyserial': 'pyserial',
45 'pyyaml': 'PyYAML',45 'pyyaml': 'PyYAML',
46 'requests': 'python-requests',46 'requests': 'python-requests',
47 'six': 'python-six',
47 },48 },
48 'suse': {49 'suse': {
49 'argparse': 'python-argparse',50 'argparse': 'python-argparse',
@@ -55,6 +56,7 @@
55 'pyserial': 'python-pyserial',56 'pyserial': 'python-pyserial',
56 'pyyaml': 'python-yaml',57 'pyyaml': 'python-yaml',
57 'requests': 'python-requests',58 'requests': 'python-requests',
59 'six': 'python-six',
58 }60 }
59}61}
6062
6163
=== modified file 'requirements.txt'
--- requirements.txt 2014-02-12 10:14:49 +0000
+++ requirements.txt 2014-07-08 04:39:36 +0000
@@ -31,3 +31,6 @@
3131
32# For patching pieces of cloud-config together32# For patching pieces of cloud-config together
33jsonpatch33jsonpatch
34
35# For py2/3 simultaneous compatibility
36six
3437
=== modified file 'tests/unittests/test_data.py'
--- tests/unittests/test_data.py 2014-01-17 15:27:09 +0000
+++ tests/unittests/test_data.py 2014-07-08 04:39:36 +0000
@@ -1,11 +1,12 @@
1"""Tests for handling of userdata within cloud init."""1"""Tests for handling of userdata within cloud init."""
22
3import StringIO
4
5import gzip3import gzip
6import logging4import logging
7import os5import os
86
7from six import BytesIO
8from six import StringIO
9
9from email.mime.application import MIMEApplication10from email.mime.application import MIMEApplication
10from email.mime.base import MIMEBase11from email.mime.base import MIMEBase
11from email.mime.multipart import MIMEMultipart12from email.mime.multipart import MIMEMultipart
@@ -53,7 +54,7 @@
53 self.patchUtils(root)54 self.patchUtils(root)
5455
55 def capture_log(self, lvl=logging.DEBUG):56 def capture_log(self, lvl=logging.DEBUG):
56 log_file = StringIO.StringIO()57 log_file = StringIO()
57 self._log_handler = logging.StreamHandler(log_file)58 self._log_handler = logging.StreamHandler(log_file)
58 self._log_handler.setLevel(lvl)59 self._log_handler.setLevel(lvl)
59 self._log = log.getLogger()60 self._log = log.getLogger()
@@ -352,9 +353,9 @@
352 """Tests that individual message gzip encoding works."""353 """Tests that individual message gzip encoding works."""
353354
354 def gzip_part(text):355 def gzip_part(text):
355 contents = StringIO.StringIO()356 contents = BytesIO()
356 f = gzip.GzipFile(fileobj=contents, mode='w')357 f = gzip.GzipFile(fileobj=contents, mode='wb')
357 f.write(str(text))358 f.write(util.encode_text(text))
358 f.flush()359 f.flush()
359 f.close()360 f.close()
360 return MIMEApplication(contents.getvalue(), 'gzip')361 return MIMEApplication(contents.getvalue(), 'gzip')
361362
=== modified file 'tests/unittests/test_datasource/test_nocloud.py'
--- tests/unittests/test_datasource/test_nocloud.py 2014-02-26 19:28:46 +0000
+++ tests/unittests/test_datasource/test_nocloud.py 2014-07-08 04:39:36 +0000
@@ -86,7 +86,7 @@
8686
87 data = {87 data = {
88 'fs_label': None,88 'fs_label': None,
89 'meta-data': {'instance-id': 'IID'},89 'meta-data': yaml.safe_dump({'instance-id': 'IID'}),
90 'user-data': "USER_DATA_RAW",90 'user-data': "USER_DATA_RAW",
91 }91 }
9292
9393
=== modified file 'tests/unittests/test_datasource/test_openstack.py'
--- tests/unittests/test_datasource/test_openstack.py 2014-02-08 20:20:33 +0000
+++ tests/unittests/test_datasource/test_openstack.py 2014-07-08 04:39:36 +0000
@@ -20,9 +20,8 @@
20import json20import json
21import re21import re
2222
23from StringIO import StringIO23from six import StringIO
2424from six.moves.urllib.parse import urlparse
25from urlparse import urlparse
2625
27from tests.unittests import helpers as test_helpers26from tests.unittests import helpers as test_helpers
2827
2928
=== modified file 'tests/unittests/test_distros/test_netconfig.py'
--- tests/unittests/test_distros/test_netconfig.py 2012-10-11 19:49:45 +0000
+++ tests/unittests/test_distros/test_netconfig.py 2014-07-08 04:39:36 +0000
@@ -4,6 +4,8 @@
44
5import os5import os
66
7from six import StringIO
8
7from cloudinit import distros9from cloudinit import distros
8from cloudinit import helpers10from cloudinit import helpers
9from cloudinit import settings11from cloudinit import settings
@@ -11,9 +13,6 @@
1113
12from cloudinit.distros.parsers.sys_conf import SysConf14from cloudinit.distros.parsers.sys_conf import SysConf
1315
14from StringIO import StringIO
15
16
17BASE_NET_CFG = '''16BASE_NET_CFG = '''
18auto lo17auto lo
19iface lo inet loopback18iface lo inet loopback
2019
=== modified file 'tests/unittests/test_handler/test_handler_locale.py'
--- tests/unittests/test_handler/test_handler_locale.py 2013-06-25 06:57:27 +0000
+++ tests/unittests/test_handler/test_handler_locale.py 2014-07-08 04:39:36 +0000
@@ -29,7 +29,7 @@
2929
30from configobj import ConfigObj30from configobj import ConfigObj
3131
32from StringIO import StringIO32from six import BytesIO
3333
34import logging34import logging
3535
@@ -59,6 +59,6 @@
59 cc = self._get_cloud('sles')59 cc = self._get_cloud('sles')
60 cc_locale.handle('cc_locale', cfg, cc, LOG, [])60 cc_locale.handle('cc_locale', cfg, cc, LOG, [])
6161
62 contents = util.load_file('/etc/sysconfig/language')62 contents = util.load_file('/etc/sysconfig/language', decode=False)
63 n_cfg = ConfigObj(StringIO(contents))63 n_cfg = ConfigObj(BytesIO(contents))
64 self.assertEquals({'RC_LANG': cfg['locale']}, dict(n_cfg))64 self.assertEquals({'RC_LANG': cfg['locale']}, dict(n_cfg))
6565
=== modified file 'tests/unittests/test_handler/test_handler_seed_random.py'
--- tests/unittests/test_handler/test_handler_seed_random.py 2014-03-04 19:35:09 +0000
+++ tests/unittests/test_handler/test_handler_seed_random.py 2014-07-08 04:39:36 +0000
@@ -22,7 +22,7 @@
22import gzip22import gzip
23import tempfile23import tempfile
2424
25from StringIO import StringIO25from six import StringIO
2626
27from cloudinit import cloud27from cloudinit import cloud
28from cloudinit import distros28from cloudinit import distros
2929
=== modified file 'tests/unittests/test_handler/test_handler_set_hostname.py'
--- tests/unittests/test_handler/test_handler_set_hostname.py 2013-06-25 06:57:27 +0000
+++ tests/unittests/test_handler/test_handler_set_hostname.py 2014-07-08 04:39:36 +0000
@@ -9,7 +9,7 @@
99
10import logging10import logging
1111
12from StringIO import StringIO12from six import BytesIO
1313
14from configobj import ConfigObj14from configobj import ConfigObj
1515
@@ -37,8 +37,8 @@
37 self.patchUtils(self.tmp)37 self.patchUtils(self.tmp)
38 cc_set_hostname.handle('cc_set_hostname',38 cc_set_hostname.handle('cc_set_hostname',
39 cfg, cc, LOG, [])39 cfg, cc, LOG, [])
40 contents = util.load_file("/etc/sysconfig/network")40 contents = util.load_file("/etc/sysconfig/network", decode=False)
41 n_cfg = ConfigObj(StringIO(contents))41 n_cfg = ConfigObj(BytesIO(contents))
42 self.assertEquals({'HOSTNAME': 'blah.blah.blah.yahoo.com'},42 self.assertEquals({'HOSTNAME': 'blah.blah.blah.yahoo.com'},
43 dict(n_cfg))43 dict(n_cfg))
4444
4545
=== modified file 'tests/unittests/test_handler/test_handler_timezone.py'
--- tests/unittests/test_handler/test_handler_timezone.py 2013-06-25 06:57:27 +0000
+++ tests/unittests/test_handler/test_handler_timezone.py 2014-07-08 04:39:36 +0000
@@ -29,7 +29,7 @@
2929
30from configobj import ConfigObj30from configobj import ConfigObj
3131
32from StringIO import StringIO32from six import BytesIO
3333
34import logging34import logging
3535
@@ -67,8 +67,8 @@
6767
68 cc_timezone.handle('cc_timezone', cfg, cc, LOG, [])68 cc_timezone.handle('cc_timezone', cfg, cc, LOG, [])
6969
70 contents = util.load_file('/etc/sysconfig/clock')70 contents = util.load_file('/etc/sysconfig/clock', decode=False)
71 n_cfg = ConfigObj(StringIO(contents))71 n_cfg = ConfigObj(BytesIO(contents))
72 self.assertEquals({'TIMEZONE': cfg['timezone']}, dict(n_cfg))72 self.assertEquals({'TIMEZONE': cfg['timezone']}, dict(n_cfg))
7373
74 contents = util.load_file('/etc/localtime')74 contents = util.load_file('/etc/localtime')
7575
=== modified file 'tests/unittests/test_handler/test_handler_yum_add_repo.py'
--- tests/unittests/test_handler/test_handler_yum_add_repo.py 2014-04-01 18:20:57 +0000
+++ tests/unittests/test_handler/test_handler_yum_add_repo.py 2014-07-08 04:39:36 +0000
@@ -6,7 +6,7 @@
66
7import logging7import logging
88
9from StringIO import StringIO9from six import BytesIO
1010
11import configobj11import configobj
1212
@@ -52,8 +52,9 @@
52 }52 }
53 self.patchUtils(self.tmp)53 self.patchUtils(self.tmp)
54 cc_yum_add_repo.handle('yum_add_repo', cfg, None, LOG, [])54 cc_yum_add_repo.handle('yum_add_repo', cfg, None, LOG, [])
55 contents = util.load_file("/etc/yum.repos.d/epel_testing.repo")55 contents = util.load_file("/etc/yum.repos.d/epel_testing.repo",
56 contents = configobj.ConfigObj(StringIO(contents))56 decode=False)
57 contents = configobj.ConfigObj(BytesIO(contents))
57 expected = {58 expected = {
58 'epel_testing': {59 'epel_testing': {
59 'name': 'Extra Packages for Enterprise Linux 5 - Testing',60 'name': 'Extra Packages for Enterprise Linux 5 - Testing',