Merge lp:~harlowja/cloud-init/py2-3 into lp:~cloud-init-dev/cloud-init/trunk

Proposed by Joshua Harlow
Status: Rejected
Rejected by: Scott Moser
Proposed branch: lp:~harlowja/cloud-init/py2-3
Merge into: lp:~cloud-init-dev/cloud-init/trunk
Diff against target: 1969 lines (+317/-205)
59 files modified
cloudinit/config/cc_apt_configure.py (+3/-1)
cloudinit/config/cc_debug.py (+4/-2)
cloudinit/config/cc_landscape.py (+1/-1)
cloudinit/config/cc_mcollective.py (+6/-5)
cloudinit/config/cc_phone_home.py (+4/-2)
cloudinit/config/cc_puppet.py (+4/-3)
cloudinit/config/cc_resolv_conf.py (+3/-2)
cloudinit/config/cc_seed_random.py (+2/-1)
cloudinit/config/cc_ssh.py (+4/-2)
cloudinit/config/cc_yum_add_repo.py (+4/-3)
cloudinit/distros/__init__.py (+20/-19)
cloudinit/distros/arch.py (+3/-1)
cloudinit/distros/freebsd.py (+6/-4)
cloudinit/distros/net_util.py (+4/-1)
cloudinit/distros/parsers/hostname.py (+1/-1)
cloudinit/distros/parsers/hosts.py (+1/-1)
cloudinit/distros/parsers/resolv_conf.py (+1/-1)
cloudinit/distros/parsers/sys_conf.py (+5/-5)
cloudinit/distros/rhel.py (+3/-1)
cloudinit/distros/sles.py (+3/-1)
cloudinit/ec2_utils.py (+4/-5)
cloudinit/handlers/__init__.py (+1/-1)
cloudinit/handlers/boot_hook.py (+1/-1)
cloudinit/handlers/cloud_config.py (+1/-1)
cloudinit/handlers/shell_script.py (+1/-1)
cloudinit/handlers/upstart_job.py (+1/-1)
cloudinit/helpers.py (+7/-5)
cloudinit/log.py (+4/-3)
cloudinit/mergers/__init__.py (+3/-1)
cloudinit/mergers/m_dict.py (+3/-1)
cloudinit/mergers/m_list.py (+3/-1)
cloudinit/mergers/m_str.py (+6/-4)
cloudinit/netinfo.py (+5/-3)
cloudinit/signal_handler.py (+1/-1)
cloudinit/sources/DataSourceConfigDrive.py (+3/-1)
cloudinit/sources/DataSourceEc2.py (+4/-2)
cloudinit/sources/DataSourceMAAS.py (+5/-2)
cloudinit/sources/DataSourceOVF.py (+5/-3)
cloudinit/sources/DataSourceSmartOS.py (+7/-5)
cloudinit/sources/__init__.py (+6/-4)
cloudinit/sources/helpers/openstack.py (+4/-2)
cloudinit/ssh_util.py (+3/-3)
cloudinit/stages.py (+10/-9)
cloudinit/type_utils.py (+25/-6)
cloudinit/url_helper.py (+14/-7)
cloudinit/user_data.py (+6/-4)
cloudinit/util.py (+70/-45)
packages/bddeb (+1/-0)
packages/brpm (+2/-0)
requirements.txt (+3/-0)
tests/unittests/test_data.py (+7/-6)
tests/unittests/test_datasource/test_nocloud.py (+1/-1)
tests/unittests/test_datasource/test_openstack.py (+2/-3)
tests/unittests/test_distros/test_netconfig.py (+2/-3)
tests/unittests/test_handler/test_handler_locale.py (+3/-3)
tests/unittests/test_handler/test_handler_seed_random.py (+1/-1)
tests/unittests/test_handler/test_handler_set_hostname.py (+3/-3)
tests/unittests/test_handler/test_handler_timezone.py (+3/-3)
tests/unittests/test_handler/test_handler_yum_add_repo.py (+4/-3)
To merge this branch: bzr merge lp:~harlowja/cloud-init/py2-3
Reviewer Review Type Date Requested Status
cloud-init Commiters Pending
Review via email: mp+225240@code.launchpad.net

Description of the change

Gets the basic integration of six usage going.

This change does the following:

- Moves to using the new locations of modules (using six as needed)
  - urlparse moved, configparser moved, stringio...
- Fixes the octal usage that we had previously (0o644 is the new way that works)
- The utils load_file() now decodes the files by default from binary -> utf-8 (unless decode=False, decode=False seems needed for the configobj module to correctly work)
- The utils write_file() now decodes to binary before writing (from unicode) as needed
- Adjust tests to work correctly using the new way of load/writing files

To post a comment you must log in.
lp:~harlowja/cloud-init/py2-3 updated
986. By Joshua Harlow

Fix urllib.quote moving to a new location

987. By Joshua Harlow

Consistently return the unicode/text version of responses

988. By Joshua Harlow

Explicitly use response.contents instead of str(contents)

To avoid using a function that has different meaning in
python 2 and python 3 instead prefer the explict access
of the contents attribute instead (which will now always
be unicode) to avoid the subtle issues that will happen
if we continue to use str() instead.

989. By Joshua Harlow

Fix the configparser being required to use stringio and not bytesio

990. By Joshua Harlow

Fix types that changed/moved

991. By Joshua Harlow

Remove another comparison for (str, basestring)

992. By Joshua Harlow

Fix all iteritems() usage and remove (str, basestring) usage

Revision history for this message
Bohuslav "Slavek" Kabrda (bkabrda) wrote :

I'm also interested in cloud-init being compatible with Python 3. I don't understand cloud-init codebase that much, but the patch looks fine. My question is, what minimal Python version are you targeting? I'm assuming 2.6 and higher? If so, I'd recommend using dict.{items,keys,values} instead of six.iter{items,keys,values}(dict) - it's more readable (but really just a nitpick).

Revision history for this message
Joshua Harlow (harlowja) wrote :

2.6 and higher, as for the dict stuff, meh.

Revision history for this message
Zane Bitter (zaneb) wrote :

Looks good to me.

Minor point: "import pickle" in http://bazaar.launchpad.net/~harlowja/cloud-init/py2-3/revision/983/cloudinit/stages.py could be "from six.moves import cPickle as pickle" to avoid slowing down the existing Python 2 code.

Revision history for this message
Barry Warsaw (barry) wrote :

Is this branch behind trunk? I had some merge conflicts which aren't showing up in this MP.

I'm working on a new branch, highly inspired by this one, which should be more current against trunk, and use tox to ensure py2/3 compatibility. Most of the code changes here I agree with (except the iteritems ones -- I'm with Bohuslav on that :).

We have to disable the cheetah test in py3 since that package is not compatible. The biggest problem will be with the use of mocker which isn't py3 compatible. Not sure how to deal with that yet.

Revision history for this message
Joshua Harlow (harlowja) wrote :

Likely is behind (seeing that its not updated in a while); if you are working on a newer branch (derived from this one) that's cool and probably means we don't need this one at that point. Maybe time to slowly (or fastly) move to mock then...

Revision history for this message
Barry Warsaw (barry) wrote :

Here's my WIP branch: lp:~barry/cloud-init/py2-3

This largely merges Joshua's branch here, albeit manually because of aforementioned problems. It adds tox support for bilingual test runs, and the Python 2.7 test suite passes fully for me (try `tox -e py27`).

My next steps will be to look at replacing mocker and then repairing the Python 3 test suite. I'll also need to scrounge up a Python 2.6 to make sure support for that version hasn't broken.

I'll create a MP for my branch and welcome all comments and contributions!

Revision history for this message
Scott Moser (smoser) wrote :

dont know what the right way to do this is, but this is effectively merged. so no need for this review, so i rejected it.

Unmerged revisions

992. By Joshua Harlow

Fix all iteritems() usage and remove (str, basestring) usage

991. By Joshua Harlow

Remove another comparison for (str, basestring)

990. By Joshua Harlow

Fix types that changed/moved

989. By Joshua Harlow

Fix the configparser being required to use stringio and not bytesio

988. By Joshua Harlow

Explicitly use response.contents instead of str(contents)

To avoid using a function that has different meaning in
python 2 and python 3 instead prefer the explict access
of the contents attribute instead (which will now always
be unicode) to avoid the subtle issues that will happen
if we continue to use str() instead.

987. By Joshua Harlow

Consistently return the unicode/text version of responses

986. By Joshua Harlow

Fix urllib.quote moving to a new location

985. By Joshua Harlow

Fix up the unittests due to new changes

984. By Joshua Harlow

Adjust a bunch of moved StringIO imports

983. By Joshua Harlow

Fix a bunch more octal changes and import moves

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'cloudinit/config/cc_apt_configure.py'
2--- cloudinit/config/cc_apt_configure.py 2014-02-12 19:56:55 +0000
3+++ cloudinit/config/cc_apt_configure.py 2014-07-08 04:39:36 +0000
4@@ -22,6 +22,8 @@
5 import os
6 import re
7
8+import six
9+
10 from cloudinit import templater
11 from cloudinit import util
12
13@@ -126,7 +128,7 @@
14
15
16 def rename_apt_lists(old_mirrors, new_mirrors, lists_d="/var/lib/apt/lists"):
17- for (name, omirror) in old_mirrors.iteritems():
18+ for (name, omirror) in six.iteritems(old_mirrors):
19 nmirror = new_mirrors.get(name)
20 if not nmirror:
21 continue
22
23=== modified file 'cloudinit/config/cc_debug.py'
24--- cloudinit/config/cc_debug.py 2014-01-23 19:28:59 +0000
25+++ cloudinit/config/cc_debug.py 2014-07-08 04:39:36 +0000
26@@ -14,10 +14,12 @@
27 # You should have received a copy of the GNU General Public License
28 # along with this program. If not, see <http://www.gnu.org/licenses/>.
29
30+import copy
31+
32+from six import StringIO
33+
34 from cloudinit import type_utils
35 from cloudinit import util
36-import copy
37-from StringIO import StringIO
38
39
40 def _make_header(text):
41
42=== modified file 'cloudinit/config/cc_landscape.py'
43--- cloudinit/config/cc_landscape.py 2014-01-27 22:34:35 +0000
44+++ cloudinit/config/cc_landscape.py 2014-07-08 04:39:36 +0000
45@@ -20,7 +20,7 @@
46
47 import os
48
49-from StringIO import StringIO
50+from six import StringIO
51
52 from configobj import ConfigObj
53
54
55=== modified file 'cloudinit/config/cc_mcollective.py'
56--- cloudinit/config/cc_mcollective.py 2014-01-27 22:34:35 +0000
57+++ cloudinit/config/cc_mcollective.py 2014-07-08 04:39:36 +0000
58@@ -19,7 +19,8 @@
59 # You should have received a copy of the GNU General Public License
60 # along with this program. If not, see <http://www.gnu.org/licenses/>.
61
62-from StringIO import StringIO
63+import six
64+from six import StringIO
65
66 # Used since this can maintain comments
67 # and doesn't need a top level section
68@@ -51,7 +52,7 @@
69 # original file in order to be able to mix the rest up
70 mcollective_config = ConfigObj(SERVER_CFG)
71 # See: http://tiny.cc/jh9agw
72- for (cfg_name, cfg) in mcollective_cfg['conf'].iteritems():
73+ for (cfg_name, cfg) in six.iteritems(mcollective_cfg['conf']):
74 if cfg_name == 'public-cert':
75 util.write_file(PUBCERT_FILE, cfg, mode=0644)
76 mcollective_config['plugin.ssl_server_public'] = PUBCERT_FILE
77@@ -61,7 +62,7 @@
78 mcollective_config['plugin.ssl_server_private'] = PRICERT_FILE
79 mcollective_config['securityprovider'] = 'ssl'
80 else:
81- if isinstance(cfg, (basestring, str)):
82+ if isinstance(cfg, six.string_types):
83 # Just set it in the 'main' section
84 mcollective_config[cfg_name] = cfg
85 elif isinstance(cfg, (dict)):
86@@ -69,7 +70,7 @@
87 # if it is needed and then add/or create items as needed
88 if cfg_name not in mcollective_config.sections:
89 mcollective_config[cfg_name] = {}
90- for (o, v) in cfg.iteritems():
91+ for (o, v) in six.iteritems(cfg):
92 mcollective_config[cfg_name][o] = v
93 else:
94 # Otherwise just try to convert it to a string
95@@ -81,7 +82,7 @@
96 contents = StringIO()
97 mcollective_config.write(contents)
98 contents = contents.getvalue()
99- util.write_file(SERVER_CFG, contents, mode=0644)
100+ util.write_file(SERVER_CFG, contents, mode=0o644)
101
102 # Start mcollective
103 util.subp(['service', 'mcollective', 'start'], capture=False)
104
105=== modified file 'cloudinit/config/cc_phone_home.py'
106--- cloudinit/config/cc_phone_home.py 2014-02-05 15:36:47 +0000
107+++ cloudinit/config/cc_phone_home.py 2014-07-08 04:39:36 +0000
108@@ -18,6 +18,8 @@
109 # You should have received a copy of the GNU General Public License
110 # along with this program. If not, see <http://www.gnu.org/licenses/>.
111
112+import six
113+
114 from cloudinit import templater
115 from cloudinit import util
116
117@@ -81,7 +83,7 @@
118 'pub_key_ecdsa': '/etc/ssh/ssh_host_ecdsa_key.pub',
119 }
120
121- for (n, path) in pubkeys.iteritems():
122+ for (n, path) in six.iteritems(pubkeys):
123 try:
124 all_keys[n] = util.load_file(path)
125 except:
126@@ -99,7 +101,7 @@
127
128 # Get them read to be posted
129 real_submit_keys = {}
130- for (k, v) in submit_keys.iteritems():
131+ for (k, v) in six.iteritems(submit_keys):
132 if v is None:
133 real_submit_keys[k] = 'N/A'
134 else:
135
136=== modified file 'cloudinit/config/cc_puppet.py'
137--- cloudinit/config/cc_puppet.py 2014-02-05 15:36:47 +0000
138+++ cloudinit/config/cc_puppet.py 2014-07-08 04:39:36 +0000
139@@ -18,7 +18,8 @@
140 # You should have received a copy of the GNU General Public License
141 # along with this program. If not, see <http://www.gnu.org/licenses/>.
142
143-from StringIO import StringIO
144+import six
145+from six import StringIO
146
147 import os
148 import socket
149@@ -81,7 +82,7 @@
150 cleaned_contents = '\n'.join(cleaned_lines)
151 puppet_config.readfp(StringIO(cleaned_contents),
152 filename=PUPPET_CONF_PATH)
153- for (cfg_name, cfg) in puppet_cfg['conf'].iteritems():
154+ for (cfg_name, cfg) in six.iteritems(puppet_cfg['conf']):
155 # Cert configuration is a special case
156 # Dump the puppet master ca certificate in the correct place
157 if cfg_name == 'ca_cert':
158@@ -96,7 +97,7 @@
159 else:
160 # Iterate throug the config items, we'll use ConfigParser.set
161 # to overwrite or create new items as needed
162- for (o, v) in cfg.iteritems():
163+ for (o, v) in six.iteritems(cfg):
164 if o == 'certname':
165 # Expand %f as the fqdn
166 # TODO(harlowja) should this use the cloud fqdn??
167
168=== modified file 'cloudinit/config/cc_resolv_conf.py'
169--- cloudinit/config/cc_resolv_conf.py 2014-02-05 15:36:47 +0000
170+++ cloudinit/config/cc_resolv_conf.py 2014-07-08 04:39:36 +0000
171@@ -48,6 +48,7 @@
172 # timeout: 1
173 #
174
175+import six
176
177 from cloudinit.settings import PER_INSTANCE
178 from cloudinit import templater
179@@ -67,8 +68,8 @@
180 flags = []
181 false_flags = []
182 if 'options' in params:
183- for key, val in params['options'].iteritems():
184- if type(val) == bool:
185+ for key, val in six.iteritems(params['options']):
186+ if isinstance(val, bool):
187 if val:
188 flags.append(key)
189 else:
190
191=== modified file 'cloudinit/config/cc_seed_random.py'
192--- cloudinit/config/cc_seed_random.py 2014-03-04 19:35:09 +0000
193+++ cloudinit/config/cc_seed_random.py 2014-07-08 04:39:36 +0000
194@@ -21,7 +21,8 @@
195
196 import base64
197 import os
198-from StringIO import StringIO
199+
200+from six import StringIO
201
202 from cloudinit.settings import PER_INSTANCE
203 from cloudinit import log as logging
204
205=== modified file 'cloudinit/config/cc_ssh.py'
206--- cloudinit/config/cc_ssh.py 2014-01-28 14:48:47 +0000
207+++ cloudinit/config/cc_ssh.py 2014-07-08 04:39:36 +0000
208@@ -21,6 +21,8 @@
209 import glob
210 import os
211
212+import six
213+
214 # Ensure this is aliased to a name not 'distros'
215 # since the module attribute 'distros'
216 # is a list of distros that are supported, not a sub-module
217@@ -68,13 +70,13 @@
218
219 if "ssh_keys" in cfg:
220 # if there are keys in cloud-config, use them
221- for (key, val) in cfg["ssh_keys"].iteritems():
222+ for (key, val) in six.iteritems(cfg["ssh_keys"]):
223 if key in KEY_2_FILE:
224 tgt_fn = KEY_2_FILE[key][0]
225 tgt_perms = KEY_2_FILE[key][1]
226 util.write_file(tgt_fn, val, tgt_perms)
227
228- for (priv, pub) in PRIV_2_PUB.iteritems():
229+ for (priv, pub) in six.iteritems(PRIV_2_PUB):
230 if pub in cfg['ssh_keys'] or not priv in cfg['ssh_keys']:
231 continue
232 pair = (KEY_2_FILE[priv][0], KEY_2_FILE[pub][0])
233
234=== modified file 'cloudinit/config/cc_yum_add_repo.py'
235--- cloudinit/config/cc_yum_add_repo.py 2014-02-06 15:59:04 +0000
236+++ cloudinit/config/cc_yum_add_repo.py 2014-07-08 04:39:36 +0000
237@@ -18,10 +18,11 @@
238
239 import os
240
241+import configobj
242+import six
243+
244 from cloudinit import util
245
246-import configobj
247-
248
249 def _canonicalize_id(repo_id):
250 repo_id = repo_id.lower().replace("-", "_")
251@@ -37,7 +38,7 @@
252 # Can handle 'lists' in certain cases
253 # See: http://bit.ly/Qqrf1t
254 return "\n ".join([_format_repo_value(v) for v in val])
255- if not isinstance(val, (basestring, str)):
256+ if not isinstance(val, six.string_types):
257 return str(val)
258 return val
259
260
261=== modified file 'cloudinit/distros/__init__.py'
262--- cloudinit/distros/__init__.py 2014-02-12 19:56:55 +0000
263+++ cloudinit/distros/__init__.py 2014-07-08 04:39:36 +0000
264@@ -21,7 +21,8 @@
265 # You should have received a copy of the GNU General Public License
266 # along with this program. If not, see <http://www.gnu.org/licenses/>.
267
268-from StringIO import StringIO
269+import six
270+from six import StringIO
271
272 import abc
273 import itertools
274@@ -268,7 +269,7 @@
275 if header:
276 contents.write("%s\n" % (header))
277 contents.write("%s\n" % (eh))
278- util.write_file(self.hosts_fn, contents.getvalue(), mode=0644)
279+ util.write_file(self.hosts_fn, contents.getvalue(), mode=0o644)
280
281 def _bring_up_interface(self, device_name):
282 cmd = ['ifup', device_name]
283@@ -330,7 +331,7 @@
284 redact_opts = ['passwd']
285
286 # Check the values and create the command
287- for key, val in kwargs.iteritems():
288+ for key, val in six.iteritems(kwargs):
289
290 if key in adduser_opts and val and isinstance(val, str):
291 adduser_cmd.extend([adduser_opts[key], val])
292@@ -452,7 +453,7 @@
293 util.make_header(base="added"),
294 "#includedir %s" % (path), '']
295 sudoers_contents = "\n".join(lines)
296- util.write_file(sudo_base, sudoers_contents, 0440)
297+ util.write_file(sudo_base, sudoers_contents, 0o440)
298 else:
299 lines = ['', util.make_header(base="added"),
300 "#includedir %s" % (path), '']
301@@ -462,7 +463,7 @@
302 except IOError as e:
303 util.logexc(LOG, "Failed to write %s", sudo_base)
304 raise e
305- util.ensure_dir(path, 0750)
306+ util.ensure_dir(path, 0o750)
307
308 def write_sudo_rules(self, user, rules, sudo_file=None):
309 if not sudo_file:
310@@ -475,7 +476,7 @@
311 if isinstance(rules, (list, tuple)):
312 for rule in rules:
313 lines.append("%s %s" % (user, rule))
314- elif isinstance(rules, (basestring, str)):
315+ elif isinstance(rules, six.string_types):
316 lines.append("%s %s" % (user, rules))
317 else:
318 msg = "Can not create sudoers rule addition with type %r"
319@@ -490,7 +491,7 @@
320 content,
321 ]
322 try:
323- util.write_file(sudo_file, "\n".join(contents), 0440)
324+ util.write_file(sudo_file, "\n".join(contents), 0o440)
325 except IOError as e:
326 util.logexc(LOG, "Failed to write sudoers file %s", sudo_file)
327 raise e
328@@ -545,10 +546,10 @@
329 subst['ec2_region'] = "%s" % availability_zone[0:-1]
330
331 results = {}
332- for (name, mirror) in mirror_info.get('failsafe', {}).iteritems():
333+ for (name, mirror) in six.iteritems(mirror_info.get('failsafe', {})):
334 results[name] = mirror
335
336- for (name, searchlist) in mirror_info.get('search', {}).iteritems():
337+ for (name, searchlist) in six.iteritems(mirror_info.get('search', {})):
338 mirrors = []
339 for tmpl in searchlist:
340 try:
341@@ -588,7 +589,7 @@
342 # is the standard form used in the rest
343 # of cloud-init
344 def _normalize_groups(grp_cfg):
345- if isinstance(grp_cfg, (str, basestring)):
346+ if isinstance(grp_cfg, six.string_types):
347 grp_cfg = grp_cfg.strip().split(",")
348 if isinstance(grp_cfg, (list)):
349 c_grp_cfg = {}
350@@ -598,7 +599,7 @@
351 if k not in c_grp_cfg:
352 if isinstance(v, (list)):
353 c_grp_cfg[k] = list(v)
354- elif isinstance(v, (basestring, str)):
355+ elif isinstance(v, six.string_types):
356 c_grp_cfg[k] = [v]
357 else:
358 raise TypeError("Bad group member type %s" %
359@@ -606,12 +607,12 @@
360 else:
361 if isinstance(v, (list)):
362 c_grp_cfg[k].extend(v)
363- elif isinstance(v, (basestring, str)):
364+ elif isinstance(v, six.string_types):
365 c_grp_cfg[k].append(v)
366 else:
367 raise TypeError("Bad group member type %s" %
368 type_utils.obj_name(v))
369- elif isinstance(i, (str, basestring)):
370+ elif isinstance(i, six.string_types):
371 if i not in c_grp_cfg:
372 c_grp_cfg[i] = []
373 else:
374@@ -648,7 +649,7 @@
375 if isinstance(u_cfg, (dict)):
376 ad_ucfg = []
377 for (k, v) in u_cfg.items():
378- if isinstance(v, (bool, int, basestring, str, float)):
379+ if isinstance(v, (bool, int, float) + six.string_types):
380 if util.is_true(v):
381 ad_ucfg.append(str(k))
382 elif isinstance(v, (dict)):
383@@ -658,12 +659,12 @@
384 raise TypeError(("Unmappable user value type %s"
385 " for key %s") % (type_utils.obj_name(v), k))
386 u_cfg = ad_ucfg
387- elif isinstance(u_cfg, (str, basestring)):
388+ elif isinstance(u_cfg, six.string_types):
389 u_cfg = util.uniq_merge_sorted(u_cfg)
390
391 users = {}
392 for user_config in u_cfg:
393- if isinstance(user_config, (str, basestring, list)):
394+ if isinstance(user_config, (list,) + six.string_types):
395 for u in util.uniq_merge(user_config):
396 if u and u not in users:
397 users[u] = {}
398@@ -768,7 +769,7 @@
399 old_user = cfg['user']
400 # Translate it into the format that is more useful
401 # going forward
402- if isinstance(old_user, (basestring, str)):
403+ if isinstance(old_user, six.string_types):
404 old_user = {
405 'name': old_user,
406 }
407@@ -797,7 +798,7 @@
408 default_user_config = util.mergemanydict([old_user, distro_user_config])
409
410 base_users = cfg.get('users', [])
411- if not isinstance(base_users, (list, dict, str, basestring)):
412+ if not isinstance(base_users, (list, dict) + six.string_types):
413 LOG.warn(("Format for 'users' key must be a comma separated string"
414 " or a dictionary or a list and not %s"),
415 type_utils.obj_name(base_users))
416@@ -811,7 +812,7 @@
417 base_users.append({'name': 'default'})
418 elif isinstance(base_users, (dict)):
419 base_users['default'] = dict(base_users).get('default', True)
420- elif isinstance(base_users, (str, basestring)):
421+ elif isinstance(base_users, six.string_types):
422 # Just append it on to be re-parsed later
423 base_users += ",default"
424
425
426=== modified file 'cloudinit/distros/arch.py'
427--- cloudinit/distros/arch.py 2014-02-12 19:56:55 +0000
428+++ cloudinit/distros/arch.py 2014-07-08 04:39:36 +0000
429@@ -16,6 +16,8 @@
430 # You should have received a copy of the GNU General Public License
431 # along with this program. If not, see <http://www.gnu.org/licenses/>.
432
433+import six
434+
435 from cloudinit import distros
436 from cloudinit import helpers
437 from cloudinit import log as logging
438@@ -68,7 +70,7 @@
439 settings, entries)
440 dev_names = entries.keys()
441 # Format for netctl
442- for (dev, info) in entries.iteritems():
443+ for (dev, info) in six.iteritems(entries):
444 nameservers = []
445 net_fn = self.network_conf_dir + dev
446 net_cfg = {
447
448=== modified file 'cloudinit/distros/freebsd.py'
449--- cloudinit/distros/freebsd.py 2014-02-28 21:40:08 +0000
450+++ cloudinit/distros/freebsd.py 2014-07-08 04:39:36 +0000
451@@ -16,10 +16,11 @@
452 # You should have received a copy of the GNU General Public License
453 # along with this program. If not, see <http://www.gnu.org/licenses/>.
454
455-from StringIO import StringIO
456-
457 import re
458
459+import six
460+from six import StringIO
461+
462 from cloudinit import distros
463 from cloudinit import helpers
464 from cloudinit import log as logging
465@@ -150,8 +151,9 @@
466
467 redact_opts = ['passwd']
468
469- for key, val in kwargs.iteritems():
470- if key in adduser_opts and val and isinstance(val, basestring):
471+ for key, val in six.iteritems(kwargs):
472+ if key in adduser_opts and val \
473+ and isinstance(val, six.string_types):
474 adduser_cmd.extend([adduser_opts[key], val])
475
476 # Redact certain fields from the logs
477
478=== modified file 'cloudinit/distros/net_util.py'
479--- cloudinit/distros/net_util.py 2014-01-24 21:20:54 +0000
480+++ cloudinit/distros/net_util.py 2014-07-08 04:39:36 +0000
481@@ -79,6 +79,9 @@
482 # }
483 # }
484
485+import six
486+
487+
488 def translate_network(settings):
489 # Get the standard cmd, args from the ubuntu format
490 entries = []
491@@ -103,7 +106,7 @@
492 consume[cmd] = args
493 # Check if anything left over to consume
494 absorb = False
495- for (cmd, args) in consume.iteritems():
496+ for (cmd, args) in six.iteritems(consume):
497 if cmd == 'iface':
498 absorb = True
499 if absorb:
500
501=== modified file 'cloudinit/distros/parsers/hostname.py'
502--- cloudinit/distros/parsers/hostname.py 2012-11-12 22:30:08 +0000
503+++ cloudinit/distros/parsers/hostname.py 2014-07-08 04:39:36 +0000
504@@ -16,7 +16,7 @@
505 # You should have received a copy of the GNU General Public License
506 # along with this program. If not, see <http://www.gnu.org/licenses/>.
507
508-from StringIO import StringIO
509+from six import StringIO
510
511 from cloudinit.distros.parsers import chop_comment
512
513
514=== modified file 'cloudinit/distros/parsers/hosts.py'
515--- cloudinit/distros/parsers/hosts.py 2012-11-13 06:14:31 +0000
516+++ cloudinit/distros/parsers/hosts.py 2014-07-08 04:39:36 +0000
517@@ -16,7 +16,7 @@
518 # You should have received a copy of the GNU General Public License
519 # along with this program. If not, see <http://www.gnu.org/licenses/>.
520
521-from StringIO import StringIO
522+from six import StringIO
523
524 from cloudinit.distros.parsers import chop_comment
525
526
527=== modified file 'cloudinit/distros/parsers/resolv_conf.py'
528--- cloudinit/distros/parsers/resolv_conf.py 2013-03-19 13:32:04 +0000
529+++ cloudinit/distros/parsers/resolv_conf.py 2014-07-08 04:39:36 +0000
530@@ -16,7 +16,7 @@
531 # You should have received a copy of the GNU General Public License
532 # along with this program. If not, see <http://www.gnu.org/licenses/>.
533
534-from StringIO import StringIO
535+from six import StringIO
536
537 from cloudinit import util
538
539
540=== modified file 'cloudinit/distros/parsers/sys_conf.py'
541--- cloudinit/distros/parsers/sys_conf.py 2012-11-12 22:30:08 +0000
542+++ cloudinit/distros/parsers/sys_conf.py 2014-07-08 04:39:36 +0000
543@@ -16,11 +16,11 @@
544 # You should have received a copy of the GNU General Public License
545 # along with this program. If not, see <http://www.gnu.org/licenses/>.
546
547-from StringIO import StringIO
548-
549 import pipes
550 import re
551
552+import six
553+
554 # This library is used to parse/write
555 # out the various sysconfig files edited (best attempt effort)
556 #
557@@ -61,15 +61,15 @@
558
559 def __str__(self):
560 contents = self.write()
561- out_contents = StringIO()
562+ out_contents = six.StringIO()
563 if isinstance(contents, (list, tuple)):
564 out_contents.write("\n".join(contents))
565 else:
566- out_contents.write(str(contents))
567+ out_contents.write(six.text_type(contents))
568 return out_contents.getvalue()
569
570 def _quote(self, value, multiline=False):
571- if not isinstance(value, (str, basestring)):
572+ if not isinstance(value, six.string_types):
573 raise ValueError('Value "%s" is not a string' % (value))
574 if len(value) == 0:
575 return ''
576
577=== modified file 'cloudinit/distros/rhel.py'
578--- cloudinit/distros/rhel.py 2014-02-03 22:03:14 +0000
579+++ cloudinit/distros/rhel.py 2014-07-08 04:39:36 +0000
580@@ -20,6 +20,8 @@
581 # You should have received a copy of the GNU General Public License
582 # along with this program. If not, see <http://www.gnu.org/licenses/>.
583
584+import six
585+
586 from cloudinit import distros
587 from cloudinit import helpers
588 from cloudinit import log as logging
589@@ -71,7 +73,7 @@
590 nameservers = []
591 searchservers = []
592 dev_names = entries.keys()
593- for (dev, info) in entries.iteritems():
594+ for (dev, info) in six.iteritems(entries):
595 net_fn = self.network_script_tpl % (dev)
596 net_cfg = {
597 'DEVICE': dev,
598
599=== modified file 'cloudinit/distros/sles.py'
600--- cloudinit/distros/sles.py 2014-01-23 19:06:13 +0000
601+++ cloudinit/distros/sles.py 2014-07-08 04:39:36 +0000
602@@ -18,6 +18,8 @@
603 # You should have received a copy of the GNU General Public License
604 # along with this program. If not, see <http://www.gnu.org/licenses/>.
605
606+import six
607+
608 from cloudinit import distros
609
610 from cloudinit.distros.parsers.hostname import HostnameConf
611@@ -62,7 +64,7 @@
612 nameservers = []
613 searchservers = []
614 dev_names = entries.keys()
615- for (dev, info) in entries.iteritems():
616+ for (dev, info) in six.iteritems(entries):
617 net_fn = self.network_script_tpl % (dev)
618 mode = info.get('auto')
619 if mode and mode.lower() == 'true':
620
621=== modified file 'cloudinit/ec2_utils.py'
622--- cloudinit/ec2_utils.py 2014-02-08 20:20:33 +0000
623+++ cloudinit/ec2_utils.py 2014-07-08 04:39:36 +0000
624@@ -17,7 +17,6 @@
625 # along with this program. If not, see <http://www.gnu.org/licenses/>.
626
627 import functools
628-import httplib
629 import json
630
631 from cloudinit import log as logging
632@@ -25,7 +24,7 @@
633 from cloudinit import util
634
635 LOG = logging.getLogger(__name__)
636-SKIP_USERDATA_CODES = frozenset([httplib.NOT_FOUND])
637+SKIP_USERDATA_CODES = frozenset([url_helper.NOT_FOUND])
638
639
640 def maybe_json_object(text):
641@@ -116,7 +115,7 @@
642 leaf_contents = {}
643 for (field, resource) in leaves.items():
644 leaf_url = url_helper.combine_url(base_url, resource)
645- leaf_blob = str(self._caller(leaf_url))
646+ leaf_blob = self._caller(leaf_url).contents
647 leaf_contents[field] = self._decode_leaf_blob(field, leaf_blob)
648 joined = {}
649 joined.update(child_contents)
650@@ -153,7 +152,7 @@
651 timeout=timeout,
652 retries=retries,
653 exception_cb=exception_cb)
654- user_data = str(response)
655+ user_data = response.contents
656 except url_helper.UrlError as e:
657 if e.code not in SKIP_USERDATA_CODES:
658 util.logexc(LOG, "Failed fetching userdata from url %s", ud_url)
659@@ -173,7 +172,7 @@
660
661 try:
662 response = caller(md_url)
663- materializer = MetadataMaterializer(str(response), md_url, caller)
664+ materializer = MetadataMaterializer(response.contents, md_url, caller)
665 md = materializer.materialize()
666 if not isinstance(md, (dict)):
667 md = {}
668
669=== modified file 'cloudinit/handlers/__init__.py'
670--- cloudinit/handlers/__init__.py 2014-01-16 21:57:21 +0000
671+++ cloudinit/handlers/__init__.py 2014-07-08 04:39:36 +0000
672@@ -147,7 +147,7 @@
673 if not modfname.endswith(".py"):
674 modfname = "%s.py" % (modfname)
675 # TODO(harlowja): Check if path exists??
676- util.write_file(modfname, payload, 0600)
677+ util.write_file(modfname, payload, 0o600)
678 handlers = pdata['handlers']
679 try:
680 mod = fixup_handler(importer.import_module(modname))
681
682=== modified file 'cloudinit/handlers/boot_hook.py'
683--- cloudinit/handlers/boot_hook.py 2013-07-21 16:34:26 +0000
684+++ cloudinit/handlers/boot_hook.py 2014-07-08 04:39:36 +0000
685@@ -50,7 +50,7 @@
686 filepath = os.path.join(self.boothook_dir, filename)
687 contents = util.strip_prefix_suffix(util.dos2unix(payload),
688 prefix=BOOTHOOK_PREFIX)
689- util.write_file(filepath, contents.lstrip(), 0700)
690+ util.write_file(filepath, contents.lstrip(), 0o700)
691 return filepath
692
693 def handle_part(self, _data, ctype, filename, # pylint: disable=W0221
694
695=== modified file 'cloudinit/handlers/cloud_config.py'
696--- cloudinit/handlers/cloud_config.py 2014-01-09 00:16:24 +0000
697+++ cloudinit/handlers/cloud_config.py 2014-07-08 04:39:36 +0000
698@@ -95,7 +95,7 @@
699 lines.append(util.yaml_dumps(self.cloud_buf))
700 else:
701 lines = []
702- util.write_file(self.cloud_fn, "\n".join(lines), 0600)
703+ util.write_file(self.cloud_fn, "\n".join(lines), 0o600)
704
705 def _extract_mergers(self, payload, headers):
706 merge_header_headers = ''
707
708=== modified file 'cloudinit/handlers/shell_script.py'
709--- cloudinit/handlers/shell_script.py 2014-01-09 00:16:24 +0000
710+++ cloudinit/handlers/shell_script.py 2014-07-08 04:39:36 +0000
711@@ -53,4 +53,4 @@
712 filename = util.clean_filename(filename)
713 payload = util.dos2unix(payload)
714 path = os.path.join(self.script_dir, filename)
715- util.write_file(path, payload, 0700)
716+ util.write_file(path, payload, 0o700)
717
718=== modified file 'cloudinit/handlers/upstart_job.py'
719--- cloudinit/handlers/upstart_job.py 2013-07-21 16:26:44 +0000
720+++ cloudinit/handlers/upstart_job.py 2014-07-08 04:39:36 +0000
721@@ -66,7 +66,7 @@
722
723 payload = util.dos2unix(payload)
724 path = os.path.join(self.upstart_dir, filename)
725- util.write_file(path, payload, 0644)
726+ util.write_file(path, payload, 0o644)
727
728 if SUITABLE_UPSTART:
729 util.subp(["initctl", "reload-configuration"], capture=False)
730
731=== modified file 'cloudinit/helpers.py'
732--- cloudinit/helpers.py 2014-01-17 20:12:31 +0000
733+++ cloudinit/helpers.py 2014-07-08 04:39:36 +0000
734@@ -23,10 +23,12 @@
735 from time import time
736
737 import contextlib
738-import io
739 import os
740
741-from ConfigParser import (NoSectionError, NoOptionError, RawConfigParser)
742+import six
743+from six.moves.configparser import (NoSectionError,
744+ NoOptionError,
745+ RawConfigParser)
746
747 from cloudinit.settings import (PER_INSTANCE, PER_ALWAYS, PER_ONCE,
748 CFG_ENV_NAME)
749@@ -318,10 +320,10 @@
750 return self.registered[content_type]
751
752 def items(self):
753- return self.registered.items()
754+ return list(self.iteritems())
755
756 def iteritems(self):
757- return self.registered.iteritems()
758+ return six.iteritems(self.registered)
759
760
761 class Paths(object):
762@@ -449,7 +451,7 @@
763
764 def stringify(self, header=None):
765 contents = ''
766- with io.BytesIO() as outputstream:
767+ with six.StringIO() as outputstream:
768 self.write(outputstream)
769 outputstream.flush()
770 contents = outputstream.getvalue()
771
772=== modified file 'cloudinit/log.py'
773--- cloudinit/log.py 2013-04-17 16:42:55 +0000
774+++ cloudinit/log.py 2014-07-08 04:39:36 +0000
775@@ -28,7 +28,8 @@
776 import os
777 import sys
778
779-from StringIO import StringIO
780+import six
781+from six import StringIO
782
783 # Logging levels for easy access
784 CRITICAL = logging.CRITICAL
785@@ -72,13 +73,13 @@
786
787 log_cfgs = []
788 log_cfg = cfg.get('logcfg')
789- if log_cfg and isinstance(log_cfg, (str, basestring)):
790+ if log_cfg and isinstance(log_cfg, six.string_types):
791 # If there is a 'logcfg' entry in the config,
792 # respect it, it is the old keyname
793 log_cfgs.append(str(log_cfg))
794 elif "log_cfgs" in cfg:
795 for a_cfg in cfg['log_cfgs']:
796- if isinstance(a_cfg, (basestring, str)):
797+ if isinstance(a_cfg, six.string_types):
798 log_cfgs.append(a_cfg)
799 elif isinstance(a_cfg, (collections.Iterable)):
800 cfg_str = [str(c) for c in a_cfg]
801
802=== modified file 'cloudinit/mergers/__init__.py'
803--- cloudinit/mergers/__init__.py 2013-05-03 21:41:28 +0000
804+++ cloudinit/mergers/__init__.py 2014-07-08 04:39:36 +0000
805@@ -18,6 +18,8 @@
806
807 import re
808
809+import six
810+
811 from cloudinit import importer
812 from cloudinit import log as logging
813 from cloudinit import type_utils
814@@ -100,7 +102,7 @@
815 raw_mergers = config.pop('merge_type', None)
816 if raw_mergers is None:
817 return parsed_mergers
818- if isinstance(raw_mergers, (str, basestring)):
819+ if isinstance(raw_mergers, six.string_types):
820 return string_extract_mergers(raw_mergers)
821 for m in raw_mergers:
822 if isinstance(m, (dict)):
823
824=== modified file 'cloudinit/mergers/m_dict.py'
825--- cloudinit/mergers/m_dict.py 2013-05-03 22:05:45 +0000
826+++ cloudinit/mergers/m_dict.py 2014-07-08 04:39:36 +0000
827@@ -16,6 +16,8 @@
828 # You should have received a copy of the GNU General Public License
829 # along with this program. If not, see <http://www.gnu.org/licenses/>.
830
831+import six
832+
833 DEF_MERGE_TYPE = 'no_replace'
834 MERGE_TYPES = ('replace', DEF_MERGE_TYPE,)
835
836@@ -57,7 +59,7 @@
837 return new_v
838 if isinstance(new_v, (list, tuple)) and self._recurse_array:
839 return self._merger.merge(old_v, new_v)
840- if isinstance(new_v, (basestring)) and self._recurse_str:
841+ if isinstance(new_v, six.string_types) and self._recurse_str:
842 return self._merger.merge(old_v, new_v)
843 if isinstance(new_v, (dict)) and self._recurse_dict:
844 return self._merger.merge(old_v, new_v)
845
846=== modified file 'cloudinit/mergers/m_list.py'
847--- cloudinit/mergers/m_list.py 2013-06-19 06:46:54 +0000
848+++ cloudinit/mergers/m_list.py 2014-07-08 04:39:36 +0000
849@@ -16,6 +16,8 @@
850 # You should have received a copy of the GNU General Public License
851 # along with this program. If not, see <http://www.gnu.org/licenses/>.
852
853+import six
854+
855 DEF_MERGE_TYPE = 'replace'
856 MERGE_TYPES = ('append', 'prepend', DEF_MERGE_TYPE, 'no_replace')
857
858@@ -73,7 +75,7 @@
859 return old_v
860 if isinstance(new_v, (list, tuple)) and self._recurse_array:
861 return self._merger.merge(old_v, new_v)
862- if isinstance(new_v, (str, basestring)) and self._recurse_str:
863+ if isinstance(new_v, six.string_types) and self._recurse_str:
864 return self._merger.merge(old_v, new_v)
865 if isinstance(new_v, (dict)) and self._recurse_dict:
866 return self._merger.merge(old_v, new_v)
867
868=== modified file 'cloudinit/mergers/m_str.py'
869--- cloudinit/mergers/m_str.py 2013-05-03 21:41:28 +0000
870+++ cloudinit/mergers/m_str.py 2014-07-08 04:39:36 +0000
871@@ -17,6 +17,8 @@
872 # You should have received a copy of the GNU General Public License
873 # along with this program. If not, see <http://www.gnu.org/licenses/>.
874
875+import six
876+
877
878 class Merger(object):
879 def __init__(self, _merger, opts):
880@@ -34,11 +36,11 @@
881 # perform the following action, if appending we will
882 # merge them together, otherwise we will just return value.
883 def _on_str(self, value, merge_with):
884- if not isinstance(value, (basestring)):
885+ if not isinstance(value, six.string_types):
886 return merge_with
887 if not self._append:
888 return merge_with
889- if isinstance(value, unicode):
890- return value + unicode(merge_with)
891+ if isinstance(value, six.text_type):
892+ return value + six.text_type(merge_with)
893 else:
894- return value + str(merge_with)
895+ return value + six.binary_type(merge_with)
896
897=== modified file 'cloudinit/netinfo.py'
898--- cloudinit/netinfo.py 2014-02-26 18:14:55 +0000
899+++ cloudinit/netinfo.py 2014-07-08 04:39:36 +0000
900@@ -20,10 +20,12 @@
901 # You should have received a copy of the GNU General Public License
902 # along with this program. If not, see <http://www.gnu.org/licenses/>.
903
904-import cloudinit.util as util
905 import re
906
907 from prettytable import PrettyTable
908+import six
909+
910+import cloudinit.util as util
911
912
913 def netdev_info(empty=""):
914@@ -83,7 +85,7 @@
915 devs[curdev][target] = toks[i][len(field) + 1:]
916
917 if empty != "":
918- for (_devname, dev) in devs.iteritems():
919+ for (_devname, dev) in six.iteritems(devs):
920 for field in dev:
921 if dev[field] == "":
922 dev[field] = empty
923@@ -155,7 +157,7 @@
924 if netdev is not None:
925 fields = ['Device', 'Up', 'Address', 'Mask', 'Hw-Address']
926 tbl = PrettyTable(fields)
927- for (dev, d) in netdev.iteritems():
928+ for (dev, d) in six.iteritems(netdev):
929 tbl.add_row([dev, d["up"], d["addr"], d["mask"], d["hwaddr"]])
930 netdev_s = tbl.get_string()
931 max_len = len(max(netdev_s.splitlines(), key=len))
932
933=== modified file 'cloudinit/signal_handler.py'
934--- cloudinit/signal_handler.py 2012-09-19 20:33:56 +0000
935+++ cloudinit/signal_handler.py 2014-07-08 04:39:36 +0000
936@@ -22,7 +22,7 @@
937 import signal
938 import sys
939
940-from StringIO import StringIO
941+from six import StringIO
942
943 from cloudinit import log as logging
944 from cloudinit import util
945
946=== modified file 'cloudinit/sources/DataSourceConfigDrive.py'
947--- cloudinit/sources/DataSourceConfigDrive.py 2014-02-25 01:17:07 +0000
948+++ cloudinit/sources/DataSourceConfigDrive.py 2014-07-08 04:39:36 +0000
949@@ -20,6 +20,8 @@
950
951 import os
952
953+import six
954+
955 from cloudinit import log as logging
956 from cloudinit import sources
957 from cloudinit import util
958@@ -198,7 +200,7 @@
959 files = data.get('files', {})
960 if files:
961 LOG.debug("Writing %s injected files", len(files))
962- for (filename, content) in files.iteritems():
963+ for (filename, content) in six.iteritems(files):
964 if not filename.startswith(os.sep):
965 filename = os.sep + filename
966 try:
967
968=== modified file 'cloudinit/sources/DataSourceEc2.py'
969--- cloudinit/sources/DataSourceEc2.py 2014-02-01 20:03:32 +0000
970+++ cloudinit/sources/DataSourceEc2.py 2014-07-08 04:39:36 +0000
971@@ -23,6 +23,8 @@
972 import os
973 import time
974
975+import six
976+
977 from cloudinit import ec2_utils as ec2
978 from cloudinit import log as logging
979 from cloudinit import sources
980@@ -156,8 +158,8 @@
981 # 'ephemeral0': '/dev/sdb',
982 # 'root': '/dev/sda1'}
983 found = None
984- bdm_items = self.metadata['block-device-mapping'].iteritems()
985- for (entname, device) in bdm_items:
986+ bdms = self.metadata['block-device-mapping']
987+ for (entname, device) in six.iteritems(bdms):
988 if entname == name:
989 found = device
990 break
991
992=== modified file 'cloudinit/sources/DataSourceMAAS.py'
993--- cloudinit/sources/DataSourceMAAS.py 2013-04-25 15:58:38 +0000
994+++ cloudinit/sources/DataSourceMAAS.py 2014-07-08 04:39:36 +0000
995@@ -20,11 +20,14 @@
996
997 from email.utils import parsedate
998 import errno
999-import oauth.oauth as oauth
1000 import os
1001 import time
1002 import urllib2
1003
1004+import oauth.oauth as oauth
1005+
1006+import six
1007+
1008 from cloudinit import log as logging
1009 from cloudinit import sources
1010 from cloudinit import url_helper
1011@@ -262,7 +265,7 @@
1012
1013 userdata = content.get('user-data', "")
1014 md = {}
1015- for (key, val) in content.iteritems():
1016+ for (key, val) in six.iteritems(content):
1017 if key == 'user-data':
1018 continue
1019 md[key] = val
1020
1021=== modified file 'cloudinit/sources/DataSourceOVF.py'
1022--- cloudinit/sources/DataSourceOVF.py 2013-06-07 17:30:03 +0000
1023+++ cloudinit/sources/DataSourceOVF.py 2014-07-08 04:39:36 +0000
1024@@ -26,6 +26,8 @@
1025 import os
1026 import re
1027
1028+import six
1029+
1030 from cloudinit import log as logging
1031 from cloudinit import sources
1032 from cloudinit import util
1033@@ -66,7 +68,7 @@
1034 np = {'iso': transport_iso9660,
1035 'vmware-guestd': transport_vmware_guestd, }
1036 name = None
1037- for (name, transfunc) in np.iteritems():
1038+ for (name, transfunc) in six.iteritems(np):
1039 (contents, _dev, _fname) = transfunc()
1040 if contents:
1041 break
1042@@ -138,7 +140,7 @@
1043 ud = ""
1044 cfg_props = ['password']
1045 md_props = ['seedfrom', 'local-hostname', 'public-keys', 'instance-id']
1046- for (prop, val) in props.iteritems():
1047+ for (prop, val) in six.iteritems(props):
1048 if prop == 'hostname':
1049 prop = "local-hostname"
1050 if prop in md_props:
1051@@ -183,7 +185,7 @@
1052
1053 # Go through mounts to see if it was already mounted
1054 mounts = util.mounts()
1055- for (dev, info) in mounts.iteritems():
1056+ for (dev, info) in six.iteritems(mounts):
1057 fstype = info['fstype']
1058 if fstype != "iso9660" and require_iso:
1059 continue
1060
1061=== modified file 'cloudinit/sources/DataSourceSmartOS.py'
1062--- cloudinit/sources/DataSourceSmartOS.py 2014-06-02 20:56:31 +0000
1063+++ cloudinit/sources/DataSourceSmartOS.py 2014-07-08 04:39:36 +0000
1064@@ -30,13 +30,15 @@
1065 # Comments with "@datadictionary" are snippets of the definition
1066
1067 import base64
1068+import os
1069+import os.path
1070+
1071+import serial
1072+import six
1073+
1074 from cloudinit import log as logging
1075 from cloudinit import sources
1076 from cloudinit import util
1077-import os
1078-import os.path
1079-import serial
1080-
1081
1082 LOG = logging.getLogger(__name__)
1083
1084@@ -201,7 +203,7 @@
1085 if b64_all is not None:
1086 self.b64_all = util.is_true(b64_all)
1087
1088- for ci_noun, attribute in SMARTOS_ATTRIB_MAP.iteritems():
1089+ for ci_noun, attribute in six.iteritems(SMARTOS_ATTRIB_MAP):
1090 smartos_noun, strip = attribute
1091 md[ci_noun] = self.query(smartos_noun, strip=strip)
1092
1093
1094=== modified file 'cloudinit/sources/__init__.py'
1095--- cloudinit/sources/__init__.py 2014-01-23 20:17:17 +0000
1096+++ cloudinit/sources/__init__.py 2014-07-08 04:39:36 +0000
1097@@ -23,6 +23,8 @@
1098 import abc
1099 import os
1100
1101+import six
1102+
1103 from cloudinit import importer
1104 from cloudinit import log as logging
1105 from cloudinit import type_utils
1106@@ -130,7 +132,7 @@
1107 # we want to return the correct value for what will actually
1108 # exist in this instance
1109 mappings = {"sd": ("vd", "xvd", "vtb")}
1110- for (nfrom, tlist) in mappings.iteritems():
1111+ for (nfrom, tlist) in six.iteritems(mappings):
1112 if not short_name.startswith(nfrom):
1113 continue
1114 for nto in tlist:
1115@@ -218,18 +220,18 @@
1116 if not pubkey_data:
1117 return keys
1118
1119- if isinstance(pubkey_data, (basestring, str)):
1120+ if isinstance(pubkey_data, six.string_types):
1121 return str(pubkey_data).splitlines()
1122
1123 if isinstance(pubkey_data, (list, set)):
1124 return list(pubkey_data)
1125
1126 if isinstance(pubkey_data, (dict)):
1127- for (_keyname, klist) in pubkey_data.iteritems():
1128+ for (_keyname, klist) in six.iteritems(pubkey_data):
1129 # lp:506332 uec metadata service responds with
1130 # data that makes boto populate a string for 'klist' rather
1131 # than a list.
1132- if isinstance(klist, (str, basestring)):
1133+ if isinstance(klist, six.string_types):
1134 klist = [klist]
1135 if isinstance(klist, (list, set)):
1136 for pkey in klist:
1137
1138=== modified file 'cloudinit/sources/helpers/openstack.py'
1139--- cloudinit/sources/helpers/openstack.py 2014-02-24 22:41:42 +0000
1140+++ cloudinit/sources/helpers/openstack.py 2014-07-08 04:39:36 +0000
1141@@ -23,6 +23,8 @@
1142 import copy
1143 import os
1144
1145+import six
1146+
1147 from cloudinit import ec2_utils
1148 from cloudinit import log as logging
1149 from cloudinit import sources
1150@@ -224,7 +226,7 @@
1151 'version': 2,
1152 }
1153 data = datafiles(version)
1154- for (name, (path, required, translator)) in data.iteritems():
1155+ for (name, (path, required, translator)) in six.iteritems(data):
1156 path = self._path_join(self.base_path, path)
1157 data = None
1158 found = False
1159@@ -344,7 +346,7 @@
1160 raise NonReadable("%s: no files found" % (self.base_path))
1161
1162 md = {}
1163- for (name, (key, translator, default)) in FILES_V1.iteritems():
1164+ for (name, (key, translator, default)) in six.iteritems(FILES_V1):
1165 if name in found:
1166 path = found[name]
1167 try:
1168
1169=== modified file 'cloudinit/ssh_util.py'
1170--- cloudinit/ssh_util.py 2013-06-19 06:44:00 +0000
1171+++ cloudinit/ssh_util.py 2014-07-08 04:39:36 +0000
1172@@ -239,7 +239,7 @@
1173 # Make sure the users .ssh dir is setup accordingly
1174 (ssh_dir, pwent) = users_ssh_info(username)
1175 if not os.path.isdir(ssh_dir):
1176- util.ensure_dir(ssh_dir, mode=0700)
1177+ util.ensure_dir(ssh_dir, mode=0o700)
1178 util.chownbyid(ssh_dir, pwent.pw_uid, pwent.pw_gid)
1179
1180 # Turn the 'update' keys given into actual entries
1181@@ -252,8 +252,8 @@
1182 (auth_key_fn, auth_key_entries) = extract_authorized_keys(username)
1183 with util.SeLinuxGuard(ssh_dir, recursive=True):
1184 content = update_authorized_keys(auth_key_entries, key_entries)
1185- util.ensure_dir(os.path.dirname(auth_key_fn), mode=0700)
1186- util.write_file(auth_key_fn, content, mode=0600)
1187+ util.ensure_dir(os.path.dirname(auth_key_fn), mode=0o700)
1188+ util.write_file(auth_key_fn, content, mode=0o600)
1189 util.chownbyid(auth_key_fn, pwent.pw_uid, pwent.pw_gid)
1190
1191
1192
1193=== modified file 'cloudinit/stages.py'
1194--- cloudinit/stages.py 2014-02-13 18:53:08 +0000
1195+++ cloudinit/stages.py 2014-07-08 04:39:36 +0000
1196@@ -20,12 +20,13 @@
1197 # You should have received a copy of the GNU General Public License
1198 # along with this program. If not, see <http://www.gnu.org/licenses/>.
1199
1200-import cPickle as pickle
1201-
1202 import copy
1203 import os
1204+import pickle
1205 import sys
1206
1207+import six
1208+
1209 from cloudinit.settings import (PER_INSTANCE, FREQUENCIES, CLOUD_CONFIG)
1210
1211 from cloudinit import handlers
1212@@ -202,7 +203,7 @@
1213 util.logexc(LOG, "Failed pickling datasource %s", self.datasource)
1214 return False
1215 try:
1216- util.write_file(pickled_fn, pk_contents, mode=0400)
1217+ util.write_file(pickled_fn, pk_contents, mode=0o400)
1218 except Exception:
1219 util.logexc(LOG, "Failed pickling datasource to %s", pickled_fn)
1220 return False
1221@@ -324,15 +325,15 @@
1222
1223 def _store_userdata(self):
1224 raw_ud = "%s" % (self.datasource.get_userdata_raw())
1225- util.write_file(self._get_ipath('userdata_raw'), raw_ud, 0600)
1226+ util.write_file(self._get_ipath('userdata_raw'), raw_ud, 0o600)
1227 processed_ud = "%s" % (self.datasource.get_userdata())
1228- util.write_file(self._get_ipath('userdata'), processed_ud, 0600)
1229+ util.write_file(self._get_ipath('userdata'), processed_ud, 0o600)
1230
1231 def _store_vendordata(self):
1232 raw_vd = "%s" % (self.datasource.get_vendordata_raw())
1233- util.write_file(self._get_ipath('vendordata_raw'), raw_vd, 0600)
1234+ util.write_file(self._get_ipath('vendordata_raw'), raw_vd, 0o600)
1235 processed_vd = "%s" % (self.datasource.get_vendordata())
1236- util.write_file(self._get_ipath('vendordata'), processed_vd, 0600)
1237+ util.write_file(self._get_ipath('vendordata'), processed_vd, 0o600)
1238
1239 def _default_handlers(self, opts=None):
1240 if opts is None:
1241@@ -384,7 +385,7 @@
1242 if not path or not os.path.isdir(path):
1243 return
1244 potential_handlers = util.find_modules(path)
1245- for (fname, mod_name) in potential_handlers.iteritems():
1246+ for (fname, mod_name) in six.iteritems(potential_handlers):
1247 try:
1248 mod_locs = importer.find_module(mod_name, [''],
1249 ['list_types',
1250@@ -574,7 +575,7 @@
1251 for item in cfg_mods:
1252 if not item:
1253 continue
1254- if isinstance(item, (str, basestring)):
1255+ if isinstance(item, six.string_types):
1256 module_list.append({
1257 'mod': item.strip(),
1258 })
1259
1260=== modified file 'cloudinit/type_utils.py'
1261--- cloudinit/type_utils.py 2013-03-07 03:24:05 +0000
1262+++ cloudinit/type_utils.py 2014-07-08 04:39:36 +0000
1263@@ -24,11 +24,30 @@
1264
1265 import types
1266
1267+import six
1268+
1269+if six.PY3:
1270+ _NAME_TYPES = (
1271+ types.ModuleType,
1272+ types.FunctionType,
1273+ types.LambdaType,
1274+ type,
1275+ )
1276+else:
1277+ _NAME_TYPES = (
1278+ types.TypeType,
1279+ types.ModuleType,
1280+ types.FunctionType,
1281+ types.LambdaType,
1282+ types.ClassType,
1283+ )
1284+
1285
1286 def obj_name(obj):
1287- if isinstance(obj, (types.TypeType,
1288- types.ModuleType,
1289- types.FunctionType,
1290- types.LambdaType)):
1291- return str(obj.__name__)
1292- return obj_name(obj.__class__)
1293+ if isinstance(obj, _NAME_TYPES):
1294+ return six.text_type(obj.__name__)
1295+ else:
1296+ if not hasattr(obj, '__class__'):
1297+ return repr(obj)
1298+ else:
1299+ return obj_name(obj.__class__)
1300
1301=== modified file 'cloudinit/url_helper.py'
1302--- cloudinit/url_helper.py 2014-02-13 17:13:42 +0000
1303+++ cloudinit/url_helper.py 2014-07-08 04:39:36 +0000
1304@@ -20,21 +20,28 @@
1305 # You should have received a copy of the GNU General Public License
1306 # along with this program. If not, see <http://www.gnu.org/licenses/>.
1307
1308-import httplib
1309+import six
1310+
1311 import time
1312-import urllib
1313
1314 import requests
1315 from requests import exceptions
1316
1317-from urlparse import (urlparse, urlunparse)
1318+from six.moves.urllib.parse import (urlparse, urlunparse)
1319+from six.moves.urllib.parse import quote as urlquote
1320
1321 from cloudinit import log as logging
1322 from cloudinit import version
1323
1324 LOG = logging.getLogger(__name__)
1325
1326-NOT_FOUND = httplib.NOT_FOUND
1327+if six.PY2:
1328+ import httplib
1329+ NOT_FOUND = httplib.NOT_FOUND
1330+else:
1331+ import http.client
1332+ NOT_FOUND = http.client.NOT_FOUND
1333+
1334
1335 # Check if requests has ssl support (added in requests >= 0.8.8)
1336 SSL_ENABLED = False
1337@@ -70,7 +77,7 @@
1338 path = url_parsed[2]
1339 if path and not path.endswith("/"):
1340 path += "/"
1341- path += urllib.quote(str(add_on), safe="/:")
1342+ path += urlquote(str(add_on), safe="/:")
1343 url_parsed[2] = path
1344 return urlunparse(url_parsed)
1345
1346@@ -111,7 +118,7 @@
1347
1348 @property
1349 def contents(self):
1350- return self._response.content
1351+ return self._response.text
1352
1353 @property
1354 def url(self):
1355@@ -135,7 +142,7 @@
1356 return self._response.status_code
1357
1358 def __str__(self):
1359- return self.contents
1360+ return self._response.text
1361
1362
1363 class UrlError(IOError):
1364
1365=== modified file 'cloudinit/user_data.py'
1366--- cloudinit/user_data.py 2014-01-24 20:29:09 +0000
1367+++ cloudinit/user_data.py 2014-07-08 04:39:36 +0000
1368@@ -29,6 +29,8 @@
1369 from email.mime.nonmultipart import MIMENonMultipart
1370 from email.mime.text import MIMEText
1371
1372+import six
1373+
1374 from cloudinit import handlers
1375 from cloudinit import log as logging
1376 from cloudinit import util
1377@@ -235,9 +237,9 @@
1378 resp = util.read_file_or_url(include_url,
1379 ssl_details=self.ssl_details)
1380 if include_once_on and resp.ok():
1381- util.write_file(include_once_fn, str(resp), mode=0600)
1382+ util.write_file(include_once_fn, resp.contents, mode=0o600)
1383 if resp.ok():
1384- content = str(resp)
1385+ content = resp.contents
1386 else:
1387 LOG.warn(("Fetching from %s resulted in"
1388 " a invalid http code of %s"),
1389@@ -256,7 +258,7 @@
1390 # filename and type not be present
1391 # or
1392 # scalar(payload)
1393- if isinstance(ent, (str, basestring)):
1394+ if isinstance(ent, six.string_types):
1395 ent = {'content': ent}
1396 if not isinstance(ent, (dict)):
1397 # TODO(harlowja) raise?
1398@@ -337,7 +339,7 @@
1399 data = util.decomp_gzip(raw_data)
1400 if "mime-version:" in data[0:4096].lower():
1401 msg = email.message_from_string(data)
1402- for (key, val) in headers.iteritems():
1403+ for (key, val) in six.iteritems(headers):
1404 _replace_header(msg, key, val)
1405 else:
1406 mtype = headers.get(CONTENT_TYPE, NOT_MULTIPART_TYPE)
1407
1408=== modified file 'cloudinit/util.py'
1409--- cloudinit/util.py 2014-02-24 22:20:12 +0000
1410+++ cloudinit/util.py 2014-07-08 04:39:36 +0000
1411@@ -22,8 +22,6 @@
1412 #
1413 # pylint: disable=C0302
1414
1415-from StringIO import StringIO
1416-
1417 import contextlib
1418 import copy as obj_copy
1419 import ctypes
1420@@ -47,8 +45,10 @@
1421 import sys
1422 import tempfile
1423 import time
1424-import urlparse
1425-
1426+
1427+from six.moves.urllib import parse as urlparse
1428+
1429+import six
1430 import yaml
1431
1432 from cloudinit import importer
1433@@ -71,8 +71,25 @@
1434 }
1435 FN_ALLOWED = ('_-.()' + string.digits + string.ascii_letters)
1436
1437+TRUE_STRINGS = ('true', '1', 'on', 'yes')
1438+FALSE_STRINGS = ('off', '0', 'no', 'false')
1439+
1440 # Helper utils to see if running in a container
1441-CONTAINER_TESTS = ['running-in-container', 'lxc-is-container']
1442+CONTAINER_TESTS = ('running-in-container', 'lxc-is-container')
1443+
1444+
1445+def decode_binary(blob, encoding='utf-8'):
1446+ # Converts a binary type into a text type using given encoding.
1447+ if isinstance(blob, six.text_type):
1448+ return blob
1449+ return blob.decode(encoding)
1450+
1451+
1452+def encode_text(text, encoding='utf-8'):
1453+ # Converts a text string into a binary type using given encoding.
1454+ if isinstance(text, six.binary_type):
1455+ return text
1456+ return text.encode(encoding)
1457
1458
1459 class ProcessExecutionError(IOError):
1460@@ -149,7 +166,8 @@
1461 if self.selinux and self.selinux.is_selinux_enabled():
1462 path = os.path.realpath(os.path.expanduser(self.path))
1463 # path should be a string, not unicode
1464- path = str(path)
1465+ if six.PY2:
1466+ path = str(path)
1467 do_restore = False
1468 try:
1469 # See if even worth restoring??
1470@@ -211,10 +229,10 @@
1471 def is_true(val, addons=None):
1472 if isinstance(val, (bool)):
1473 return val is True
1474- check_set = ['true', '1', 'on', 'yes']
1475+ check_set = TRUE_STRINGS
1476 if addons:
1477- check_set = check_set + addons
1478- if str(val).lower().strip() in check_set:
1479+ check_set = list(check_set) + addons
1480+ if six.text_type(val).lower().strip() in check_set:
1481 return True
1482 return False
1483
1484@@ -222,10 +240,10 @@
1485 def is_false(val, addons=None):
1486 if isinstance(val, (bool)):
1487 return val is False
1488- check_set = ['off', '0', 'no', 'false']
1489+ check_set = FALSE_STRINGS
1490 if addons:
1491- check_set = check_set + addons
1492- if str(val).lower().strip() in check_set:
1493+ check_set = list(check_set) + addons
1494+ if six.text_type(val).lower().strip() in check_set:
1495 return True
1496 return False
1497
1498@@ -275,7 +293,7 @@
1499 def uniq_merge(*lists):
1500 combined_list = []
1501 for a_list in lists:
1502- if isinstance(a_list, (str, basestring)):
1503+ if isinstance(a_list, six.string_types):
1504 a_list = a_list.strip().split(",")
1505 # Kickout the empty ones
1506 a_list = [a for a in a_list if len(a)]
1507@@ -284,7 +302,7 @@
1508
1509
1510 def clean_filename(fn):
1511- for (k, v) in FN_REPLACEMENTS.iteritems():
1512+ for (k, v) in six.iteritems(FN_REPLACEMENTS):
1513 fn = fn.replace(k, v)
1514 removals = []
1515 for k in fn:
1516@@ -298,14 +316,14 @@
1517
1518 def decomp_gzip(data, quiet=True):
1519 try:
1520- buf = StringIO(str(data))
1521+ buf = six.BytesIO(encode_text(data))
1522 with contextlib.closing(gzip.GzipFile(None, "rb", 1, buf)) as gh:
1523- return gh.read()
1524+ return decode_binary(gh.read())
1525 except Exception as e:
1526 if quiet:
1527 return data
1528 else:
1529- raise DecompressionError(str(e))
1530+ raise DecompressionError(six.text_type(e))
1531
1532
1533 def extract_usergroup(ug_pair):
1534@@ -364,7 +382,7 @@
1535
1536
1537 def load_json(text, root_types=(dict,)):
1538- decoded = json.loads(text)
1539+ decoded = json.loads(decode_binary(text))
1540 if not isinstance(decoded, tuple(root_types)):
1541 expected_types = ", ".join([str(t) for t in root_types])
1542 raise TypeError("(%s) root types expected, got %s instead"
1543@@ -396,7 +414,7 @@
1544 if key not in yobj:
1545 return default
1546 val = yobj[key]
1547- if not isinstance(val, (str, basestring)):
1548+ if not isinstance(val, six.string_types):
1549 val = str(val)
1550 return val
1551
1552@@ -431,7 +449,7 @@
1553 if isinstance(val, (list)):
1554 cval = [v for v in val]
1555 return cval
1556- if not isinstance(val, (basestring)):
1557+ if not isinstance(val, six.string_types):
1558 val = str(val)
1559 return [val]
1560
1561@@ -706,11 +724,11 @@
1562
1563 def load_yaml(blob, default=None, allowed=(dict,)):
1564 loaded = default
1565+ blob = decode_binary(blob)
1566 try:
1567- blob = str(blob)
1568- LOG.debug(("Attempting to load yaml from string "
1569- "of length %s with allowed root types %s"),
1570- len(blob), allowed)
1571+ LOG.debug("Attempting to load yaml from string "
1572+ "of length %s with allowed root types %s",
1573+ len(blob), allowed)
1574 converted = safeyaml.load(blob)
1575 if not isinstance(converted, allowed):
1576 # Yes this will just be caught, but thats ok for now...
1577@@ -744,14 +762,12 @@
1578 md_resp = read_file_or_url(md_url, timeout, retries, file_retries)
1579 md = None
1580 if md_resp.ok():
1581- md_str = str(md_resp)
1582- md = load_yaml(md_str, default={})
1583+ md = load_yaml(md_resp.contents, default={})
1584
1585 ud_resp = read_file_or_url(ud_url, timeout, retries, file_retries)
1586 ud = None
1587 if ud_resp.ok():
1588- ud_str = str(ud_resp)
1589- ud = ud_str
1590+ ud = ud_resp.contents
1591
1592 return (md, ud)
1593
1594@@ -782,7 +798,7 @@
1595 if "conf_d" in cfg:
1596 confd = cfg['conf_d']
1597 if confd:
1598- if not isinstance(confd, (str, basestring)):
1599+ if not isinstance(confd, six.string_types):
1600 raise TypeError(("Config file %s contains 'conf_d' "
1601 "with non-string type %s") %
1602 (cfgfile, type_utils.obj_name(confd)))
1603@@ -919,8 +935,8 @@
1604 return (None, None, None)
1605
1606 resp = read_file_or_url(url)
1607- if resp.contents.startswith(starts) and resp.ok():
1608- return (key, url, str(resp))
1609+ if resp.ok() and resp.contents.startswith(starts):
1610+ return (key, url, resp.contents)
1611
1612 return (key, url, None)
1613
1614@@ -1074,9 +1090,9 @@
1615 return out_list
1616
1617
1618-def load_file(fname, read_cb=None, quiet=False):
1619+def load_file(fname, read_cb=None, quiet=False, decode=True):
1620 LOG.debug("Reading from %s (quiet=%s)", fname, quiet)
1621- ofh = StringIO()
1622+ ofh = six.BytesIO()
1623 try:
1624 with open(fname, 'rb') as ifh:
1625 pipe_in_out(ifh, ofh, chunk_cb=read_cb)
1626@@ -1087,7 +1103,10 @@
1627 raise
1628 contents = ofh.getvalue()
1629 LOG.debug("Read %s bytes from %s", len(contents), fname)
1630- return contents
1631+ if decode:
1632+ return decode_binary(contents)
1633+ else:
1634+ return contents
1635
1636
1637 def get_cmdline():
1638@@ -1217,7 +1236,7 @@
1639
1640 def hash_blob(blob, routine, mlen=None):
1641 hasher = hashlib.new(routine)
1642- hasher.update(blob)
1643+ hasher.update(encode_text(blob))
1644 digest = hasher.hexdigest()
1645 # Don't get to long now
1646 if mlen is not None:
1647@@ -1248,7 +1267,7 @@
1648 os.rename(src, dest)
1649
1650
1651-def ensure_dirs(dirlist, mode=0755):
1652+def ensure_dirs(dirlist, mode=0o755):
1653 for d in dirlist:
1654 ensure_dir(d, mode)
1655
1656@@ -1262,7 +1281,7 @@
1657 return
1658 try:
1659 if key and content:
1660- write_file(target_fn, content, mode=0600)
1661+ write_file(target_fn, content, mode=0o600)
1662 LOG.debug(("Wrote to %s with contents of command line"
1663 " url %s (len=%s)"), target_fn, url, len(content))
1664 elif key and not content:
1665@@ -1454,7 +1473,7 @@
1666 write_file(path, content, omode="ab", mode=None)
1667
1668
1669-def ensure_file(path, mode=0644):
1670+def ensure_file(path, mode=0o644):
1671 write_file(path, content='', omode="ab", mode=mode)
1672
1673
1674@@ -1472,7 +1491,7 @@
1675 os.chmod(path, real_mode)
1676
1677
1678-def write_file(filename, content, mode=0644, omode="wb"):
1679+def write_file(filename, content, mode=0o644, omode="wb"):
1680 """
1681 Writes a file with the given content and sets the file mode as specified.
1682 Resotres the SELinux context if possible.
1683@@ -1480,11 +1499,17 @@
1684 @param filename: The full path of the file to write.
1685 @param content: The content to write to the file.
1686 @param mode: The filesystem mode to set on the file.
1687- @param omode: The open mode used when opening the file (r, rb, a, etc.)
1688+ @param omode: The open mode used when opening the file (w, wb, a, etc.)
1689 """
1690 ensure_dir(os.path.dirname(filename))
1691- LOG.debug("Writing to %s - %s: [%s] %s bytes",
1692- filename, omode, mode, len(content))
1693+ if 'b' in omode.lower():
1694+ content = encode_text(content)
1695+ write_type = 'bytes'
1696+ else:
1697+ content = decode_binary(content)
1698+ write_type = 'characters'
1699+ LOG.debug("Writing to %s - %s: [%s] %s %s",
1700+ filename, omode, mode, len(content), write_type)
1701 with SeLinuxGuard(path=filename):
1702 with open(filename, omode) as fh:
1703 fh.write(content)
1704@@ -1573,10 +1598,10 @@
1705 if isinstance(args, list):
1706 fixed = []
1707 for f in args:
1708- fixed.append("'%s'" % (str(f).replace("'", escaped)))
1709+ fixed.append("'%s'" % (six.text_type(f).replace("'", escaped)))
1710 content = "%s%s\n" % (content, ' '.join(fixed))
1711 cmds_made += 1
1712- elif isinstance(args, (str, basestring)):
1713+ elif isinstance(args, six.string_types):
1714 content = "%s%s\n" % (content, args)
1715 cmds_made += 1
1716 else:
1717@@ -1687,7 +1712,7 @@
1718
1719 pkglist = []
1720 for pkg in pkgs:
1721- if isinstance(pkg, basestring):
1722+ if isinstance(pkg, six.string_types):
1723 pkglist.append(pkg)
1724 continue
1725
1726
1727=== modified file 'packages/bddeb'
1728--- packages/bddeb 2014-01-17 22:08:58 +0000
1729+++ packages/bddeb 2014-07-08 04:39:36 +0000
1730@@ -37,6 +37,7 @@
1731 'pyserial': 'python-serial',
1732 'pyyaml': 'python-yaml',
1733 'requests': 'python-requests',
1734+ 'six': 'python-six',
1735 }
1736 DEBUILD_ARGS = ["-us", "-S", "-uc", "-d"]
1737
1738
1739=== modified file 'packages/brpm'
1740--- packages/brpm 2014-01-17 22:08:58 +0000
1741+++ packages/brpm 2014-07-08 04:39:36 +0000
1742@@ -44,6 +44,7 @@
1743 'pyserial': 'pyserial',
1744 'pyyaml': 'PyYAML',
1745 'requests': 'python-requests',
1746+ 'six': 'python-six',
1747 },
1748 'suse': {
1749 'argparse': 'python-argparse',
1750@@ -55,6 +56,7 @@
1751 'pyserial': 'python-pyserial',
1752 'pyyaml': 'python-yaml',
1753 'requests': 'python-requests',
1754+ 'six': 'python-six',
1755 }
1756 }
1757
1758
1759=== modified file 'requirements.txt'
1760--- requirements.txt 2014-02-12 10:14:49 +0000
1761+++ requirements.txt 2014-07-08 04:39:36 +0000
1762@@ -31,3 +31,6 @@
1763
1764 # For patching pieces of cloud-config together
1765 jsonpatch
1766+
1767+# For py2/3 simultaneous compatibility
1768+six
1769
1770=== modified file 'tests/unittests/test_data.py'
1771--- tests/unittests/test_data.py 2014-01-17 15:27:09 +0000
1772+++ tests/unittests/test_data.py 2014-07-08 04:39:36 +0000
1773@@ -1,11 +1,12 @@
1774 """Tests for handling of userdata within cloud init."""
1775
1776-import StringIO
1777-
1778 import gzip
1779 import logging
1780 import os
1781
1782+from six import BytesIO
1783+from six import StringIO
1784+
1785 from email.mime.application import MIMEApplication
1786 from email.mime.base import MIMEBase
1787 from email.mime.multipart import MIMEMultipart
1788@@ -53,7 +54,7 @@
1789 self.patchUtils(root)
1790
1791 def capture_log(self, lvl=logging.DEBUG):
1792- log_file = StringIO.StringIO()
1793+ log_file = StringIO()
1794 self._log_handler = logging.StreamHandler(log_file)
1795 self._log_handler.setLevel(lvl)
1796 self._log = log.getLogger()
1797@@ -352,9 +353,9 @@
1798 """Tests that individual message gzip encoding works."""
1799
1800 def gzip_part(text):
1801- contents = StringIO.StringIO()
1802- f = gzip.GzipFile(fileobj=contents, mode='w')
1803- f.write(str(text))
1804+ contents = BytesIO()
1805+ f = gzip.GzipFile(fileobj=contents, mode='wb')
1806+ f.write(util.encode_text(text))
1807 f.flush()
1808 f.close()
1809 return MIMEApplication(contents.getvalue(), 'gzip')
1810
1811=== modified file 'tests/unittests/test_datasource/test_nocloud.py'
1812--- tests/unittests/test_datasource/test_nocloud.py 2014-02-26 19:28:46 +0000
1813+++ tests/unittests/test_datasource/test_nocloud.py 2014-07-08 04:39:36 +0000
1814@@ -86,7 +86,7 @@
1815
1816 data = {
1817 'fs_label': None,
1818- 'meta-data': {'instance-id': 'IID'},
1819+ 'meta-data': yaml.safe_dump({'instance-id': 'IID'}),
1820 'user-data': "USER_DATA_RAW",
1821 }
1822
1823
1824=== modified file 'tests/unittests/test_datasource/test_openstack.py'
1825--- tests/unittests/test_datasource/test_openstack.py 2014-02-08 20:20:33 +0000
1826+++ tests/unittests/test_datasource/test_openstack.py 2014-07-08 04:39:36 +0000
1827@@ -20,9 +20,8 @@
1828 import json
1829 import re
1830
1831-from StringIO import StringIO
1832-
1833-from urlparse import urlparse
1834+from six import StringIO
1835+from six.moves.urllib.parse import urlparse
1836
1837 from tests.unittests import helpers as test_helpers
1838
1839
1840=== modified file 'tests/unittests/test_distros/test_netconfig.py'
1841--- tests/unittests/test_distros/test_netconfig.py 2012-10-11 19:49:45 +0000
1842+++ tests/unittests/test_distros/test_netconfig.py 2014-07-08 04:39:36 +0000
1843@@ -4,6 +4,8 @@
1844
1845 import os
1846
1847+from six import StringIO
1848+
1849 from cloudinit import distros
1850 from cloudinit import helpers
1851 from cloudinit import settings
1852@@ -11,9 +13,6 @@
1853
1854 from cloudinit.distros.parsers.sys_conf import SysConf
1855
1856-from StringIO import StringIO
1857-
1858-
1859 BASE_NET_CFG = '''
1860 auto lo
1861 iface lo inet loopback
1862
1863=== modified file 'tests/unittests/test_handler/test_handler_locale.py'
1864--- tests/unittests/test_handler/test_handler_locale.py 2013-06-25 06:57:27 +0000
1865+++ tests/unittests/test_handler/test_handler_locale.py 2014-07-08 04:39:36 +0000
1866@@ -29,7 +29,7 @@
1867
1868 from configobj import ConfigObj
1869
1870-from StringIO import StringIO
1871+from six import BytesIO
1872
1873 import logging
1874
1875@@ -59,6 +59,6 @@
1876 cc = self._get_cloud('sles')
1877 cc_locale.handle('cc_locale', cfg, cc, LOG, [])
1878
1879- contents = util.load_file('/etc/sysconfig/language')
1880- n_cfg = ConfigObj(StringIO(contents))
1881+ contents = util.load_file('/etc/sysconfig/language', decode=False)
1882+ n_cfg = ConfigObj(BytesIO(contents))
1883 self.assertEquals({'RC_LANG': cfg['locale']}, dict(n_cfg))
1884
1885=== modified file 'tests/unittests/test_handler/test_handler_seed_random.py'
1886--- tests/unittests/test_handler/test_handler_seed_random.py 2014-03-04 19:35:09 +0000
1887+++ tests/unittests/test_handler/test_handler_seed_random.py 2014-07-08 04:39:36 +0000
1888@@ -22,7 +22,7 @@
1889 import gzip
1890 import tempfile
1891
1892-from StringIO import StringIO
1893+from six import StringIO
1894
1895 from cloudinit import cloud
1896 from cloudinit import distros
1897
1898=== modified file 'tests/unittests/test_handler/test_handler_set_hostname.py'
1899--- tests/unittests/test_handler/test_handler_set_hostname.py 2013-06-25 06:57:27 +0000
1900+++ tests/unittests/test_handler/test_handler_set_hostname.py 2014-07-08 04:39:36 +0000
1901@@ -9,7 +9,7 @@
1902
1903 import logging
1904
1905-from StringIO import StringIO
1906+from six import BytesIO
1907
1908 from configobj import ConfigObj
1909
1910@@ -37,8 +37,8 @@
1911 self.patchUtils(self.tmp)
1912 cc_set_hostname.handle('cc_set_hostname',
1913 cfg, cc, LOG, [])
1914- contents = util.load_file("/etc/sysconfig/network")
1915- n_cfg = ConfigObj(StringIO(contents))
1916+ contents = util.load_file("/etc/sysconfig/network", decode=False)
1917+ n_cfg = ConfigObj(BytesIO(contents))
1918 self.assertEquals({'HOSTNAME': 'blah.blah.blah.yahoo.com'},
1919 dict(n_cfg))
1920
1921
1922=== modified file 'tests/unittests/test_handler/test_handler_timezone.py'
1923--- tests/unittests/test_handler/test_handler_timezone.py 2013-06-25 06:57:27 +0000
1924+++ tests/unittests/test_handler/test_handler_timezone.py 2014-07-08 04:39:36 +0000
1925@@ -29,7 +29,7 @@
1926
1927 from configobj import ConfigObj
1928
1929-from StringIO import StringIO
1930+from six import BytesIO
1931
1932 import logging
1933
1934@@ -67,8 +67,8 @@
1935
1936 cc_timezone.handle('cc_timezone', cfg, cc, LOG, [])
1937
1938- contents = util.load_file('/etc/sysconfig/clock')
1939- n_cfg = ConfigObj(StringIO(contents))
1940+ contents = util.load_file('/etc/sysconfig/clock', decode=False)
1941+ n_cfg = ConfigObj(BytesIO(contents))
1942 self.assertEquals({'TIMEZONE': cfg['timezone']}, dict(n_cfg))
1943
1944 contents = util.load_file('/etc/localtime')
1945
1946=== modified file 'tests/unittests/test_handler/test_handler_yum_add_repo.py'
1947--- tests/unittests/test_handler/test_handler_yum_add_repo.py 2014-04-01 18:20:57 +0000
1948+++ tests/unittests/test_handler/test_handler_yum_add_repo.py 2014-07-08 04:39:36 +0000
1949@@ -6,7 +6,7 @@
1950
1951 import logging
1952
1953-from StringIO import StringIO
1954+from six import BytesIO
1955
1956 import configobj
1957
1958@@ -52,8 +52,9 @@
1959 }
1960 self.patchUtils(self.tmp)
1961 cc_yum_add_repo.handle('yum_add_repo', cfg, None, LOG, [])
1962- contents = util.load_file("/etc/yum.repos.d/epel_testing.repo")
1963- contents = configobj.ConfigObj(StringIO(contents))
1964+ contents = util.load_file("/etc/yum.repos.d/epel_testing.repo",
1965+ decode=False)
1966+ contents = configobj.ConfigObj(BytesIO(contents))
1967 expected = {
1968 'epel_testing': {
1969 'name': 'Extra Packages for Enterprise Linux 5 - Testing',