Merge lp:~harlowja/cloud-init/py2-3 into lp:~cloud-init-dev/cloud-init/trunk
- py2-3
- Merge into trunk
Status: | Rejected |
---|---|
Rejected by: | Scott Moser |
Proposed branch: | lp:~harlowja/cloud-init/py2-3 |
Merge into: | lp:~cloud-init-dev/cloud-init/trunk |
Diff against target: |
1969 lines (+317/-205) 59 files modified
cloudinit/config/cc_apt_configure.py (+3/-1) cloudinit/config/cc_debug.py (+4/-2) cloudinit/config/cc_landscape.py (+1/-1) cloudinit/config/cc_mcollective.py (+6/-5) cloudinit/config/cc_phone_home.py (+4/-2) cloudinit/config/cc_puppet.py (+4/-3) cloudinit/config/cc_resolv_conf.py (+3/-2) cloudinit/config/cc_seed_random.py (+2/-1) cloudinit/config/cc_ssh.py (+4/-2) cloudinit/config/cc_yum_add_repo.py (+4/-3) cloudinit/distros/__init__.py (+20/-19) cloudinit/distros/arch.py (+3/-1) cloudinit/distros/freebsd.py (+6/-4) cloudinit/distros/net_util.py (+4/-1) cloudinit/distros/parsers/hostname.py (+1/-1) cloudinit/distros/parsers/hosts.py (+1/-1) cloudinit/distros/parsers/resolv_conf.py (+1/-1) cloudinit/distros/parsers/sys_conf.py (+5/-5) cloudinit/distros/rhel.py (+3/-1) cloudinit/distros/sles.py (+3/-1) cloudinit/ec2_utils.py (+4/-5) cloudinit/handlers/__init__.py (+1/-1) cloudinit/handlers/boot_hook.py (+1/-1) cloudinit/handlers/cloud_config.py (+1/-1) cloudinit/handlers/shell_script.py (+1/-1) cloudinit/handlers/upstart_job.py (+1/-1) cloudinit/helpers.py (+7/-5) cloudinit/log.py (+4/-3) cloudinit/mergers/__init__.py (+3/-1) cloudinit/mergers/m_dict.py (+3/-1) cloudinit/mergers/m_list.py (+3/-1) cloudinit/mergers/m_str.py (+6/-4) cloudinit/netinfo.py (+5/-3) cloudinit/signal_handler.py (+1/-1) cloudinit/sources/DataSourceConfigDrive.py (+3/-1) cloudinit/sources/DataSourceEc2.py (+4/-2) cloudinit/sources/DataSourceMAAS.py (+5/-2) cloudinit/sources/DataSourceOVF.py (+5/-3) cloudinit/sources/DataSourceSmartOS.py (+7/-5) cloudinit/sources/__init__.py (+6/-4) cloudinit/sources/helpers/openstack.py (+4/-2) cloudinit/ssh_util.py (+3/-3) cloudinit/stages.py (+10/-9) cloudinit/type_utils.py (+25/-6) cloudinit/url_helper.py (+14/-7) cloudinit/user_data.py (+6/-4) cloudinit/util.py (+70/-45) packages/bddeb (+1/-0) packages/brpm (+2/-0) requirements.txt (+3/-0) tests/unittests/test_data.py (+7/-6) tests/unittests/test_datasource/test_nocloud.py (+1/-1) tests/unittests/test_datasource/test_openstack.py (+2/-3) tests/unittests/test_distros/test_netconfig.py (+2/-3) tests/unittests/test_handler/test_handler_locale.py (+3/-3) tests/unittests/test_handler/test_handler_seed_random.py (+1/-1) tests/unittests/test_handler/test_handler_set_hostname.py (+3/-3) tests/unittests/test_handler/test_handler_timezone.py (+3/-3) tests/unittests/test_handler/test_handler_yum_add_repo.py (+4/-3) |
To merge this branch: | bzr merge lp:~harlowja/cloud-init/py2-3 |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
cloud-init Commiters | Pending | ||
Review via email: mp+225240@code.launchpad.net |
Commit message
Description of the change
Gets the basic integration of six usage going.
This change does the following:
- Moves to using the new locations of modules (using six as needed)
- urlparse moved, configparser moved, stringio...
- Fixes the octal usage that we had previously (0o644 is the new way that works)
- The utils load_file() now decodes the files by default from binary -> utf-8 (unless decode=False, decode=False seems needed for the configobj module to correctly work)
- The utils write_file() now decodes to binary before writing (from unicode) as needed
- Adjust tests to work correctly using the new way of load/writing files
- 986. By Joshua Harlow
-
Fix urllib.quote moving to a new location
- 987. By Joshua Harlow
-
Consistently return the unicode/text version of responses
- 988. By Joshua Harlow
-
Explicitly use response.contents instead of str(contents)
To avoid using a function that has different meaning in
python 2 and python 3 instead prefer the explict access
of the contents attribute instead (which will now always
be unicode) to avoid the subtle issues that will happen
if we continue to use str() instead. - 989. By Joshua Harlow
-
Fix the configparser being required to use stringio and not bytesio
- 990. By Joshua Harlow
-
Fix types that changed/moved
- 991. By Joshua Harlow
-
Remove another comparison for (str, basestring)
- 992. By Joshua Harlow
-
Fix all iteritems() usage and remove (str, basestring) usage
Bohuslav "Slavek" Kabrda (bkabrda) wrote : | # |
Joshua Harlow (harlowja) wrote : | # |
2.6 and higher, as for the dict stuff, meh.
Zane Bitter (zaneb) wrote : | # |
Looks good to me.
Minor point: "import pickle" in http://
Barry Warsaw (barry) wrote : | # |
Is this branch behind trunk? I had some merge conflicts which aren't showing up in this MP.
I'm working on a new branch, highly inspired by this one, which should be more current against trunk, and use tox to ensure py2/3 compatibility. Most of the code changes here I agree with (except the iteritems ones -- I'm with Bohuslav on that :).
We have to disable the cheetah test in py3 since that package is not compatible. The biggest problem will be with the use of mocker which isn't py3 compatible. Not sure how to deal with that yet.
Joshua Harlow (harlowja) wrote : | # |
Likely is behind (seeing that its not updated in a while); if you are working on a newer branch (derived from this one) that's cool and probably means we don't need this one at that point. Maybe time to slowly (or fastly) move to mock then...
Barry Warsaw (barry) wrote : | # |
Here's my WIP branch: lp:~barry/cloud-init/py2-3
This largely merges Joshua's branch here, albeit manually because of aforementioned problems. It adds tox support for bilingual test runs, and the Python 2.7 test suite passes fully for me (try `tox -e py27`).
My next steps will be to look at replacing mocker and then repairing the Python 3 test suite. I'll also need to scrounge up a Python 2.6 to make sure support for that version hasn't broken.
I'll create a MP for my branch and welcome all comments and contributions!
Scott Moser (smoser) wrote : | # |
dont know what the right way to do this is, but this is effectively merged. so no need for this review, so i rejected it.
Unmerged revisions
- 992. By Joshua Harlow
-
Fix all iteritems() usage and remove (str, basestring) usage
- 991. By Joshua Harlow
-
Remove another comparison for (str, basestring)
- 990. By Joshua Harlow
-
Fix types that changed/moved
- 989. By Joshua Harlow
-
Fix the configparser being required to use stringio and not bytesio
- 988. By Joshua Harlow
-
Explicitly use response.contents instead of str(contents)
To avoid using a function that has different meaning in
python 2 and python 3 instead prefer the explict access
of the contents attribute instead (which will now always
be unicode) to avoid the subtle issues that will happen
if we continue to use str() instead. - 987. By Joshua Harlow
-
Consistently return the unicode/text version of responses
- 986. By Joshua Harlow
-
Fix urllib.quote moving to a new location
- 985. By Joshua Harlow
-
Fix up the unittests due to new changes
- 984. By Joshua Harlow
-
Adjust a bunch of moved StringIO imports
- 983. By Joshua Harlow
-
Fix a bunch more octal changes and import moves
Preview Diff
1 | === modified file 'cloudinit/config/cc_apt_configure.py' |
2 | --- cloudinit/config/cc_apt_configure.py 2014-02-12 19:56:55 +0000 |
3 | +++ cloudinit/config/cc_apt_configure.py 2014-07-08 04:39:36 +0000 |
4 | @@ -22,6 +22,8 @@ |
5 | import os |
6 | import re |
7 | |
8 | +import six |
9 | + |
10 | from cloudinit import templater |
11 | from cloudinit import util |
12 | |
13 | @@ -126,7 +128,7 @@ |
14 | |
15 | |
16 | def rename_apt_lists(old_mirrors, new_mirrors, lists_d="/var/lib/apt/lists"): |
17 | - for (name, omirror) in old_mirrors.iteritems(): |
18 | + for (name, omirror) in six.iteritems(old_mirrors): |
19 | nmirror = new_mirrors.get(name) |
20 | if not nmirror: |
21 | continue |
22 | |
23 | === modified file 'cloudinit/config/cc_debug.py' |
24 | --- cloudinit/config/cc_debug.py 2014-01-23 19:28:59 +0000 |
25 | +++ cloudinit/config/cc_debug.py 2014-07-08 04:39:36 +0000 |
26 | @@ -14,10 +14,12 @@ |
27 | # You should have received a copy of the GNU General Public License |
28 | # along with this program. If not, see <http://www.gnu.org/licenses/>. |
29 | |
30 | +import copy |
31 | + |
32 | +from six import StringIO |
33 | + |
34 | from cloudinit import type_utils |
35 | from cloudinit import util |
36 | -import copy |
37 | -from StringIO import StringIO |
38 | |
39 | |
40 | def _make_header(text): |
41 | |
42 | === modified file 'cloudinit/config/cc_landscape.py' |
43 | --- cloudinit/config/cc_landscape.py 2014-01-27 22:34:35 +0000 |
44 | +++ cloudinit/config/cc_landscape.py 2014-07-08 04:39:36 +0000 |
45 | @@ -20,7 +20,7 @@ |
46 | |
47 | import os |
48 | |
49 | -from StringIO import StringIO |
50 | +from six import StringIO |
51 | |
52 | from configobj import ConfigObj |
53 | |
54 | |
55 | === modified file 'cloudinit/config/cc_mcollective.py' |
56 | --- cloudinit/config/cc_mcollective.py 2014-01-27 22:34:35 +0000 |
57 | +++ cloudinit/config/cc_mcollective.py 2014-07-08 04:39:36 +0000 |
58 | @@ -19,7 +19,8 @@ |
59 | # You should have received a copy of the GNU General Public License |
60 | # along with this program. If not, see <http://www.gnu.org/licenses/>. |
61 | |
62 | -from StringIO import StringIO |
63 | +import six |
64 | +from six import StringIO |
65 | |
66 | # Used since this can maintain comments |
67 | # and doesn't need a top level section |
68 | @@ -51,7 +52,7 @@ |
69 | # original file in order to be able to mix the rest up |
70 | mcollective_config = ConfigObj(SERVER_CFG) |
71 | # See: http://tiny.cc/jh9agw |
72 | - for (cfg_name, cfg) in mcollective_cfg['conf'].iteritems(): |
73 | + for (cfg_name, cfg) in six.iteritems(mcollective_cfg['conf']): |
74 | if cfg_name == 'public-cert': |
75 | util.write_file(PUBCERT_FILE, cfg, mode=0644) |
76 | mcollective_config['plugin.ssl_server_public'] = PUBCERT_FILE |
77 | @@ -61,7 +62,7 @@ |
78 | mcollective_config['plugin.ssl_server_private'] = PRICERT_FILE |
79 | mcollective_config['securityprovider'] = 'ssl' |
80 | else: |
81 | - if isinstance(cfg, (basestring, str)): |
82 | + if isinstance(cfg, six.string_types): |
83 | # Just set it in the 'main' section |
84 | mcollective_config[cfg_name] = cfg |
85 | elif isinstance(cfg, (dict)): |
86 | @@ -69,7 +70,7 @@ |
87 | # if it is needed and then add/or create items as needed |
88 | if cfg_name not in mcollective_config.sections: |
89 | mcollective_config[cfg_name] = {} |
90 | - for (o, v) in cfg.iteritems(): |
91 | + for (o, v) in six.iteritems(cfg): |
92 | mcollective_config[cfg_name][o] = v |
93 | else: |
94 | # Otherwise just try to convert it to a string |
95 | @@ -81,7 +82,7 @@ |
96 | contents = StringIO() |
97 | mcollective_config.write(contents) |
98 | contents = contents.getvalue() |
99 | - util.write_file(SERVER_CFG, contents, mode=0644) |
100 | + util.write_file(SERVER_CFG, contents, mode=0o644) |
101 | |
102 | # Start mcollective |
103 | util.subp(['service', 'mcollective', 'start'], capture=False) |
104 | |
105 | === modified file 'cloudinit/config/cc_phone_home.py' |
106 | --- cloudinit/config/cc_phone_home.py 2014-02-05 15:36:47 +0000 |
107 | +++ cloudinit/config/cc_phone_home.py 2014-07-08 04:39:36 +0000 |
108 | @@ -18,6 +18,8 @@ |
109 | # You should have received a copy of the GNU General Public License |
110 | # along with this program. If not, see <http://www.gnu.org/licenses/>. |
111 | |
112 | +import six |
113 | + |
114 | from cloudinit import templater |
115 | from cloudinit import util |
116 | |
117 | @@ -81,7 +83,7 @@ |
118 | 'pub_key_ecdsa': '/etc/ssh/ssh_host_ecdsa_key.pub', |
119 | } |
120 | |
121 | - for (n, path) in pubkeys.iteritems(): |
122 | + for (n, path) in six.iteritems(pubkeys): |
123 | try: |
124 | all_keys[n] = util.load_file(path) |
125 | except: |
126 | @@ -99,7 +101,7 @@ |
127 | |
128 | # Get them read to be posted |
129 | real_submit_keys = {} |
130 | - for (k, v) in submit_keys.iteritems(): |
131 | + for (k, v) in six.iteritems(submit_keys): |
132 | if v is None: |
133 | real_submit_keys[k] = 'N/A' |
134 | else: |
135 | |
136 | === modified file 'cloudinit/config/cc_puppet.py' |
137 | --- cloudinit/config/cc_puppet.py 2014-02-05 15:36:47 +0000 |
138 | +++ cloudinit/config/cc_puppet.py 2014-07-08 04:39:36 +0000 |
139 | @@ -18,7 +18,8 @@ |
140 | # You should have received a copy of the GNU General Public License |
141 | # along with this program. If not, see <http://www.gnu.org/licenses/>. |
142 | |
143 | -from StringIO import StringIO |
144 | +import six |
145 | +from six import StringIO |
146 | |
147 | import os |
148 | import socket |
149 | @@ -81,7 +82,7 @@ |
150 | cleaned_contents = '\n'.join(cleaned_lines) |
151 | puppet_config.readfp(StringIO(cleaned_contents), |
152 | filename=PUPPET_CONF_PATH) |
153 | - for (cfg_name, cfg) in puppet_cfg['conf'].iteritems(): |
154 | + for (cfg_name, cfg) in six.iteritems(puppet_cfg['conf']): |
155 | # Cert configuration is a special case |
156 | # Dump the puppet master ca certificate in the correct place |
157 | if cfg_name == 'ca_cert': |
158 | @@ -96,7 +97,7 @@ |
159 | else: |
160 | # Iterate throug the config items, we'll use ConfigParser.set |
161 | # to overwrite or create new items as needed |
162 | - for (o, v) in cfg.iteritems(): |
163 | + for (o, v) in six.iteritems(cfg): |
164 | if o == 'certname': |
165 | # Expand %f as the fqdn |
166 | # TODO(harlowja) should this use the cloud fqdn?? |
167 | |
168 | === modified file 'cloudinit/config/cc_resolv_conf.py' |
169 | --- cloudinit/config/cc_resolv_conf.py 2014-02-05 15:36:47 +0000 |
170 | +++ cloudinit/config/cc_resolv_conf.py 2014-07-08 04:39:36 +0000 |
171 | @@ -48,6 +48,7 @@ |
172 | # timeout: 1 |
173 | # |
174 | |
175 | +import six |
176 | |
177 | from cloudinit.settings import PER_INSTANCE |
178 | from cloudinit import templater |
179 | @@ -67,8 +68,8 @@ |
180 | flags = [] |
181 | false_flags = [] |
182 | if 'options' in params: |
183 | - for key, val in params['options'].iteritems(): |
184 | - if type(val) == bool: |
185 | + for key, val in six.iteritems(params['options']): |
186 | + if isinstance(val, bool): |
187 | if val: |
188 | flags.append(key) |
189 | else: |
190 | |
191 | === modified file 'cloudinit/config/cc_seed_random.py' |
192 | --- cloudinit/config/cc_seed_random.py 2014-03-04 19:35:09 +0000 |
193 | +++ cloudinit/config/cc_seed_random.py 2014-07-08 04:39:36 +0000 |
194 | @@ -21,7 +21,8 @@ |
195 | |
196 | import base64 |
197 | import os |
198 | -from StringIO import StringIO |
199 | + |
200 | +from six import StringIO |
201 | |
202 | from cloudinit.settings import PER_INSTANCE |
203 | from cloudinit import log as logging |
204 | |
205 | === modified file 'cloudinit/config/cc_ssh.py' |
206 | --- cloudinit/config/cc_ssh.py 2014-01-28 14:48:47 +0000 |
207 | +++ cloudinit/config/cc_ssh.py 2014-07-08 04:39:36 +0000 |
208 | @@ -21,6 +21,8 @@ |
209 | import glob |
210 | import os |
211 | |
212 | +import six |
213 | + |
214 | # Ensure this is aliased to a name not 'distros' |
215 | # since the module attribute 'distros' |
216 | # is a list of distros that are supported, not a sub-module |
217 | @@ -68,13 +70,13 @@ |
218 | |
219 | if "ssh_keys" in cfg: |
220 | # if there are keys in cloud-config, use them |
221 | - for (key, val) in cfg["ssh_keys"].iteritems(): |
222 | + for (key, val) in six.iteritems(cfg["ssh_keys"]): |
223 | if key in KEY_2_FILE: |
224 | tgt_fn = KEY_2_FILE[key][0] |
225 | tgt_perms = KEY_2_FILE[key][1] |
226 | util.write_file(tgt_fn, val, tgt_perms) |
227 | |
228 | - for (priv, pub) in PRIV_2_PUB.iteritems(): |
229 | + for (priv, pub) in six.iteritems(PRIV_2_PUB): |
230 | if pub in cfg['ssh_keys'] or not priv in cfg['ssh_keys']: |
231 | continue |
232 | pair = (KEY_2_FILE[priv][0], KEY_2_FILE[pub][0]) |
233 | |
234 | === modified file 'cloudinit/config/cc_yum_add_repo.py' |
235 | --- cloudinit/config/cc_yum_add_repo.py 2014-02-06 15:59:04 +0000 |
236 | +++ cloudinit/config/cc_yum_add_repo.py 2014-07-08 04:39:36 +0000 |
237 | @@ -18,10 +18,11 @@ |
238 | |
239 | import os |
240 | |
241 | +import configobj |
242 | +import six |
243 | + |
244 | from cloudinit import util |
245 | |
246 | -import configobj |
247 | - |
248 | |
249 | def _canonicalize_id(repo_id): |
250 | repo_id = repo_id.lower().replace("-", "_") |
251 | @@ -37,7 +38,7 @@ |
252 | # Can handle 'lists' in certain cases |
253 | # See: http://bit.ly/Qqrf1t |
254 | return "\n ".join([_format_repo_value(v) for v in val]) |
255 | - if not isinstance(val, (basestring, str)): |
256 | + if not isinstance(val, six.string_types): |
257 | return str(val) |
258 | return val |
259 | |
260 | |
261 | === modified file 'cloudinit/distros/__init__.py' |
262 | --- cloudinit/distros/__init__.py 2014-02-12 19:56:55 +0000 |
263 | +++ cloudinit/distros/__init__.py 2014-07-08 04:39:36 +0000 |
264 | @@ -21,7 +21,8 @@ |
265 | # You should have received a copy of the GNU General Public License |
266 | # along with this program. If not, see <http://www.gnu.org/licenses/>. |
267 | |
268 | -from StringIO import StringIO |
269 | +import six |
270 | +from six import StringIO |
271 | |
272 | import abc |
273 | import itertools |
274 | @@ -268,7 +269,7 @@ |
275 | if header: |
276 | contents.write("%s\n" % (header)) |
277 | contents.write("%s\n" % (eh)) |
278 | - util.write_file(self.hosts_fn, contents.getvalue(), mode=0644) |
279 | + util.write_file(self.hosts_fn, contents.getvalue(), mode=0o644) |
280 | |
281 | def _bring_up_interface(self, device_name): |
282 | cmd = ['ifup', device_name] |
283 | @@ -330,7 +331,7 @@ |
284 | redact_opts = ['passwd'] |
285 | |
286 | # Check the values and create the command |
287 | - for key, val in kwargs.iteritems(): |
288 | + for key, val in six.iteritems(kwargs): |
289 | |
290 | if key in adduser_opts and val and isinstance(val, str): |
291 | adduser_cmd.extend([adduser_opts[key], val]) |
292 | @@ -452,7 +453,7 @@ |
293 | util.make_header(base="added"), |
294 | "#includedir %s" % (path), ''] |
295 | sudoers_contents = "\n".join(lines) |
296 | - util.write_file(sudo_base, sudoers_contents, 0440) |
297 | + util.write_file(sudo_base, sudoers_contents, 0o440) |
298 | else: |
299 | lines = ['', util.make_header(base="added"), |
300 | "#includedir %s" % (path), ''] |
301 | @@ -462,7 +463,7 @@ |
302 | except IOError as e: |
303 | util.logexc(LOG, "Failed to write %s", sudo_base) |
304 | raise e |
305 | - util.ensure_dir(path, 0750) |
306 | + util.ensure_dir(path, 0o750) |
307 | |
308 | def write_sudo_rules(self, user, rules, sudo_file=None): |
309 | if not sudo_file: |
310 | @@ -475,7 +476,7 @@ |
311 | if isinstance(rules, (list, tuple)): |
312 | for rule in rules: |
313 | lines.append("%s %s" % (user, rule)) |
314 | - elif isinstance(rules, (basestring, str)): |
315 | + elif isinstance(rules, six.string_types): |
316 | lines.append("%s %s" % (user, rules)) |
317 | else: |
318 | msg = "Can not create sudoers rule addition with type %r" |
319 | @@ -490,7 +491,7 @@ |
320 | content, |
321 | ] |
322 | try: |
323 | - util.write_file(sudo_file, "\n".join(contents), 0440) |
324 | + util.write_file(sudo_file, "\n".join(contents), 0o440) |
325 | except IOError as e: |
326 | util.logexc(LOG, "Failed to write sudoers file %s", sudo_file) |
327 | raise e |
328 | @@ -545,10 +546,10 @@ |
329 | subst['ec2_region'] = "%s" % availability_zone[0:-1] |
330 | |
331 | results = {} |
332 | - for (name, mirror) in mirror_info.get('failsafe', {}).iteritems(): |
333 | + for (name, mirror) in six.iteritems(mirror_info.get('failsafe', {})): |
334 | results[name] = mirror |
335 | |
336 | - for (name, searchlist) in mirror_info.get('search', {}).iteritems(): |
337 | + for (name, searchlist) in six.iteritems(mirror_info.get('search', {})): |
338 | mirrors = [] |
339 | for tmpl in searchlist: |
340 | try: |
341 | @@ -588,7 +589,7 @@ |
342 | # is the standard form used in the rest |
343 | # of cloud-init |
344 | def _normalize_groups(grp_cfg): |
345 | - if isinstance(grp_cfg, (str, basestring)): |
346 | + if isinstance(grp_cfg, six.string_types): |
347 | grp_cfg = grp_cfg.strip().split(",") |
348 | if isinstance(grp_cfg, (list)): |
349 | c_grp_cfg = {} |
350 | @@ -598,7 +599,7 @@ |
351 | if k not in c_grp_cfg: |
352 | if isinstance(v, (list)): |
353 | c_grp_cfg[k] = list(v) |
354 | - elif isinstance(v, (basestring, str)): |
355 | + elif isinstance(v, six.string_types): |
356 | c_grp_cfg[k] = [v] |
357 | else: |
358 | raise TypeError("Bad group member type %s" % |
359 | @@ -606,12 +607,12 @@ |
360 | else: |
361 | if isinstance(v, (list)): |
362 | c_grp_cfg[k].extend(v) |
363 | - elif isinstance(v, (basestring, str)): |
364 | + elif isinstance(v, six.string_types): |
365 | c_grp_cfg[k].append(v) |
366 | else: |
367 | raise TypeError("Bad group member type %s" % |
368 | type_utils.obj_name(v)) |
369 | - elif isinstance(i, (str, basestring)): |
370 | + elif isinstance(i, six.string_types): |
371 | if i not in c_grp_cfg: |
372 | c_grp_cfg[i] = [] |
373 | else: |
374 | @@ -648,7 +649,7 @@ |
375 | if isinstance(u_cfg, (dict)): |
376 | ad_ucfg = [] |
377 | for (k, v) in u_cfg.items(): |
378 | - if isinstance(v, (bool, int, basestring, str, float)): |
379 | + if isinstance(v, (bool, int, float) + six.string_types): |
380 | if util.is_true(v): |
381 | ad_ucfg.append(str(k)) |
382 | elif isinstance(v, (dict)): |
383 | @@ -658,12 +659,12 @@ |
384 | raise TypeError(("Unmappable user value type %s" |
385 | " for key %s") % (type_utils.obj_name(v), k)) |
386 | u_cfg = ad_ucfg |
387 | - elif isinstance(u_cfg, (str, basestring)): |
388 | + elif isinstance(u_cfg, six.string_types): |
389 | u_cfg = util.uniq_merge_sorted(u_cfg) |
390 | |
391 | users = {} |
392 | for user_config in u_cfg: |
393 | - if isinstance(user_config, (str, basestring, list)): |
394 | + if isinstance(user_config, (list,) + six.string_types): |
395 | for u in util.uniq_merge(user_config): |
396 | if u and u not in users: |
397 | users[u] = {} |
398 | @@ -768,7 +769,7 @@ |
399 | old_user = cfg['user'] |
400 | # Translate it into the format that is more useful |
401 | # going forward |
402 | - if isinstance(old_user, (basestring, str)): |
403 | + if isinstance(old_user, six.string_types): |
404 | old_user = { |
405 | 'name': old_user, |
406 | } |
407 | @@ -797,7 +798,7 @@ |
408 | default_user_config = util.mergemanydict([old_user, distro_user_config]) |
409 | |
410 | base_users = cfg.get('users', []) |
411 | - if not isinstance(base_users, (list, dict, str, basestring)): |
412 | + if not isinstance(base_users, (list, dict) + six.string_types): |
413 | LOG.warn(("Format for 'users' key must be a comma separated string" |
414 | " or a dictionary or a list and not %s"), |
415 | type_utils.obj_name(base_users)) |
416 | @@ -811,7 +812,7 @@ |
417 | base_users.append({'name': 'default'}) |
418 | elif isinstance(base_users, (dict)): |
419 | base_users['default'] = dict(base_users).get('default', True) |
420 | - elif isinstance(base_users, (str, basestring)): |
421 | + elif isinstance(base_users, six.string_types): |
422 | # Just append it on to be re-parsed later |
423 | base_users += ",default" |
424 | |
425 | |
426 | === modified file 'cloudinit/distros/arch.py' |
427 | --- cloudinit/distros/arch.py 2014-02-12 19:56:55 +0000 |
428 | +++ cloudinit/distros/arch.py 2014-07-08 04:39:36 +0000 |
429 | @@ -16,6 +16,8 @@ |
430 | # You should have received a copy of the GNU General Public License |
431 | # along with this program. If not, see <http://www.gnu.org/licenses/>. |
432 | |
433 | +import six |
434 | + |
435 | from cloudinit import distros |
436 | from cloudinit import helpers |
437 | from cloudinit import log as logging |
438 | @@ -68,7 +70,7 @@ |
439 | settings, entries) |
440 | dev_names = entries.keys() |
441 | # Format for netctl |
442 | - for (dev, info) in entries.iteritems(): |
443 | + for (dev, info) in six.iteritems(entries): |
444 | nameservers = [] |
445 | net_fn = self.network_conf_dir + dev |
446 | net_cfg = { |
447 | |
448 | === modified file 'cloudinit/distros/freebsd.py' |
449 | --- cloudinit/distros/freebsd.py 2014-02-28 21:40:08 +0000 |
450 | +++ cloudinit/distros/freebsd.py 2014-07-08 04:39:36 +0000 |
451 | @@ -16,10 +16,11 @@ |
452 | # You should have received a copy of the GNU General Public License |
453 | # along with this program. If not, see <http://www.gnu.org/licenses/>. |
454 | |
455 | -from StringIO import StringIO |
456 | - |
457 | import re |
458 | |
459 | +import six |
460 | +from six import StringIO |
461 | + |
462 | from cloudinit import distros |
463 | from cloudinit import helpers |
464 | from cloudinit import log as logging |
465 | @@ -150,8 +151,9 @@ |
466 | |
467 | redact_opts = ['passwd'] |
468 | |
469 | - for key, val in kwargs.iteritems(): |
470 | - if key in adduser_opts and val and isinstance(val, basestring): |
471 | + for key, val in six.iteritems(kwargs): |
472 | + if key in adduser_opts and val \ |
473 | + and isinstance(val, six.string_types): |
474 | adduser_cmd.extend([adduser_opts[key], val]) |
475 | |
476 | # Redact certain fields from the logs |
477 | |
478 | === modified file 'cloudinit/distros/net_util.py' |
479 | --- cloudinit/distros/net_util.py 2014-01-24 21:20:54 +0000 |
480 | +++ cloudinit/distros/net_util.py 2014-07-08 04:39:36 +0000 |
481 | @@ -79,6 +79,9 @@ |
482 | # } |
483 | # } |
484 | |
485 | +import six |
486 | + |
487 | + |
488 | def translate_network(settings): |
489 | # Get the standard cmd, args from the ubuntu format |
490 | entries = [] |
491 | @@ -103,7 +106,7 @@ |
492 | consume[cmd] = args |
493 | # Check if anything left over to consume |
494 | absorb = False |
495 | - for (cmd, args) in consume.iteritems(): |
496 | + for (cmd, args) in six.iteritems(consume): |
497 | if cmd == 'iface': |
498 | absorb = True |
499 | if absorb: |
500 | |
501 | === modified file 'cloudinit/distros/parsers/hostname.py' |
502 | --- cloudinit/distros/parsers/hostname.py 2012-11-12 22:30:08 +0000 |
503 | +++ cloudinit/distros/parsers/hostname.py 2014-07-08 04:39:36 +0000 |
504 | @@ -16,7 +16,7 @@ |
505 | # You should have received a copy of the GNU General Public License |
506 | # along with this program. If not, see <http://www.gnu.org/licenses/>. |
507 | |
508 | -from StringIO import StringIO |
509 | +from six import StringIO |
510 | |
511 | from cloudinit.distros.parsers import chop_comment |
512 | |
513 | |
514 | === modified file 'cloudinit/distros/parsers/hosts.py' |
515 | --- cloudinit/distros/parsers/hosts.py 2012-11-13 06:14:31 +0000 |
516 | +++ cloudinit/distros/parsers/hosts.py 2014-07-08 04:39:36 +0000 |
517 | @@ -16,7 +16,7 @@ |
518 | # You should have received a copy of the GNU General Public License |
519 | # along with this program. If not, see <http://www.gnu.org/licenses/>. |
520 | |
521 | -from StringIO import StringIO |
522 | +from six import StringIO |
523 | |
524 | from cloudinit.distros.parsers import chop_comment |
525 | |
526 | |
527 | === modified file 'cloudinit/distros/parsers/resolv_conf.py' |
528 | --- cloudinit/distros/parsers/resolv_conf.py 2013-03-19 13:32:04 +0000 |
529 | +++ cloudinit/distros/parsers/resolv_conf.py 2014-07-08 04:39:36 +0000 |
530 | @@ -16,7 +16,7 @@ |
531 | # You should have received a copy of the GNU General Public License |
532 | # along with this program. If not, see <http://www.gnu.org/licenses/>. |
533 | |
534 | -from StringIO import StringIO |
535 | +from six import StringIO |
536 | |
537 | from cloudinit import util |
538 | |
539 | |
540 | === modified file 'cloudinit/distros/parsers/sys_conf.py' |
541 | --- cloudinit/distros/parsers/sys_conf.py 2012-11-12 22:30:08 +0000 |
542 | +++ cloudinit/distros/parsers/sys_conf.py 2014-07-08 04:39:36 +0000 |
543 | @@ -16,11 +16,11 @@ |
544 | # You should have received a copy of the GNU General Public License |
545 | # along with this program. If not, see <http://www.gnu.org/licenses/>. |
546 | |
547 | -from StringIO import StringIO |
548 | - |
549 | import pipes |
550 | import re |
551 | |
552 | +import six |
553 | + |
554 | # This library is used to parse/write |
555 | # out the various sysconfig files edited (best attempt effort) |
556 | # |
557 | @@ -61,15 +61,15 @@ |
558 | |
559 | def __str__(self): |
560 | contents = self.write() |
561 | - out_contents = StringIO() |
562 | + out_contents = six.StringIO() |
563 | if isinstance(contents, (list, tuple)): |
564 | out_contents.write("\n".join(contents)) |
565 | else: |
566 | - out_contents.write(str(contents)) |
567 | + out_contents.write(six.text_type(contents)) |
568 | return out_contents.getvalue() |
569 | |
570 | def _quote(self, value, multiline=False): |
571 | - if not isinstance(value, (str, basestring)): |
572 | + if not isinstance(value, six.string_types): |
573 | raise ValueError('Value "%s" is not a string' % (value)) |
574 | if len(value) == 0: |
575 | return '' |
576 | |
577 | === modified file 'cloudinit/distros/rhel.py' |
578 | --- cloudinit/distros/rhel.py 2014-02-03 22:03:14 +0000 |
579 | +++ cloudinit/distros/rhel.py 2014-07-08 04:39:36 +0000 |
580 | @@ -20,6 +20,8 @@ |
581 | # You should have received a copy of the GNU General Public License |
582 | # along with this program. If not, see <http://www.gnu.org/licenses/>. |
583 | |
584 | +import six |
585 | + |
586 | from cloudinit import distros |
587 | from cloudinit import helpers |
588 | from cloudinit import log as logging |
589 | @@ -71,7 +73,7 @@ |
590 | nameservers = [] |
591 | searchservers = [] |
592 | dev_names = entries.keys() |
593 | - for (dev, info) in entries.iteritems(): |
594 | + for (dev, info) in six.iteritems(entries): |
595 | net_fn = self.network_script_tpl % (dev) |
596 | net_cfg = { |
597 | 'DEVICE': dev, |
598 | |
599 | === modified file 'cloudinit/distros/sles.py' |
600 | --- cloudinit/distros/sles.py 2014-01-23 19:06:13 +0000 |
601 | +++ cloudinit/distros/sles.py 2014-07-08 04:39:36 +0000 |
602 | @@ -18,6 +18,8 @@ |
603 | # You should have received a copy of the GNU General Public License |
604 | # along with this program. If not, see <http://www.gnu.org/licenses/>. |
605 | |
606 | +import six |
607 | + |
608 | from cloudinit import distros |
609 | |
610 | from cloudinit.distros.parsers.hostname import HostnameConf |
611 | @@ -62,7 +64,7 @@ |
612 | nameservers = [] |
613 | searchservers = [] |
614 | dev_names = entries.keys() |
615 | - for (dev, info) in entries.iteritems(): |
616 | + for (dev, info) in six.iteritems(entries): |
617 | net_fn = self.network_script_tpl % (dev) |
618 | mode = info.get('auto') |
619 | if mode and mode.lower() == 'true': |
620 | |
621 | === modified file 'cloudinit/ec2_utils.py' |
622 | --- cloudinit/ec2_utils.py 2014-02-08 20:20:33 +0000 |
623 | +++ cloudinit/ec2_utils.py 2014-07-08 04:39:36 +0000 |
624 | @@ -17,7 +17,6 @@ |
625 | # along with this program. If not, see <http://www.gnu.org/licenses/>. |
626 | |
627 | import functools |
628 | -import httplib |
629 | import json |
630 | |
631 | from cloudinit import log as logging |
632 | @@ -25,7 +24,7 @@ |
633 | from cloudinit import util |
634 | |
635 | LOG = logging.getLogger(__name__) |
636 | -SKIP_USERDATA_CODES = frozenset([httplib.NOT_FOUND]) |
637 | +SKIP_USERDATA_CODES = frozenset([url_helper.NOT_FOUND]) |
638 | |
639 | |
640 | def maybe_json_object(text): |
641 | @@ -116,7 +115,7 @@ |
642 | leaf_contents = {} |
643 | for (field, resource) in leaves.items(): |
644 | leaf_url = url_helper.combine_url(base_url, resource) |
645 | - leaf_blob = str(self._caller(leaf_url)) |
646 | + leaf_blob = self._caller(leaf_url).contents |
647 | leaf_contents[field] = self._decode_leaf_blob(field, leaf_blob) |
648 | joined = {} |
649 | joined.update(child_contents) |
650 | @@ -153,7 +152,7 @@ |
651 | timeout=timeout, |
652 | retries=retries, |
653 | exception_cb=exception_cb) |
654 | - user_data = str(response) |
655 | + user_data = response.contents |
656 | except url_helper.UrlError as e: |
657 | if e.code not in SKIP_USERDATA_CODES: |
658 | util.logexc(LOG, "Failed fetching userdata from url %s", ud_url) |
659 | @@ -173,7 +172,7 @@ |
660 | |
661 | try: |
662 | response = caller(md_url) |
663 | - materializer = MetadataMaterializer(str(response), md_url, caller) |
664 | + materializer = MetadataMaterializer(response.contents, md_url, caller) |
665 | md = materializer.materialize() |
666 | if not isinstance(md, (dict)): |
667 | md = {} |
668 | |
669 | === modified file 'cloudinit/handlers/__init__.py' |
670 | --- cloudinit/handlers/__init__.py 2014-01-16 21:57:21 +0000 |
671 | +++ cloudinit/handlers/__init__.py 2014-07-08 04:39:36 +0000 |
672 | @@ -147,7 +147,7 @@ |
673 | if not modfname.endswith(".py"): |
674 | modfname = "%s.py" % (modfname) |
675 | # TODO(harlowja): Check if path exists?? |
676 | - util.write_file(modfname, payload, 0600) |
677 | + util.write_file(modfname, payload, 0o600) |
678 | handlers = pdata['handlers'] |
679 | try: |
680 | mod = fixup_handler(importer.import_module(modname)) |
681 | |
682 | === modified file 'cloudinit/handlers/boot_hook.py' |
683 | --- cloudinit/handlers/boot_hook.py 2013-07-21 16:34:26 +0000 |
684 | +++ cloudinit/handlers/boot_hook.py 2014-07-08 04:39:36 +0000 |
685 | @@ -50,7 +50,7 @@ |
686 | filepath = os.path.join(self.boothook_dir, filename) |
687 | contents = util.strip_prefix_suffix(util.dos2unix(payload), |
688 | prefix=BOOTHOOK_PREFIX) |
689 | - util.write_file(filepath, contents.lstrip(), 0700) |
690 | + util.write_file(filepath, contents.lstrip(), 0o700) |
691 | return filepath |
692 | |
693 | def handle_part(self, _data, ctype, filename, # pylint: disable=W0221 |
694 | |
695 | === modified file 'cloudinit/handlers/cloud_config.py' |
696 | --- cloudinit/handlers/cloud_config.py 2014-01-09 00:16:24 +0000 |
697 | +++ cloudinit/handlers/cloud_config.py 2014-07-08 04:39:36 +0000 |
698 | @@ -95,7 +95,7 @@ |
699 | lines.append(util.yaml_dumps(self.cloud_buf)) |
700 | else: |
701 | lines = [] |
702 | - util.write_file(self.cloud_fn, "\n".join(lines), 0600) |
703 | + util.write_file(self.cloud_fn, "\n".join(lines), 0o600) |
704 | |
705 | def _extract_mergers(self, payload, headers): |
706 | merge_header_headers = '' |
707 | |
708 | === modified file 'cloudinit/handlers/shell_script.py' |
709 | --- cloudinit/handlers/shell_script.py 2014-01-09 00:16:24 +0000 |
710 | +++ cloudinit/handlers/shell_script.py 2014-07-08 04:39:36 +0000 |
711 | @@ -53,4 +53,4 @@ |
712 | filename = util.clean_filename(filename) |
713 | payload = util.dos2unix(payload) |
714 | path = os.path.join(self.script_dir, filename) |
715 | - util.write_file(path, payload, 0700) |
716 | + util.write_file(path, payload, 0o700) |
717 | |
718 | === modified file 'cloudinit/handlers/upstart_job.py' |
719 | --- cloudinit/handlers/upstart_job.py 2013-07-21 16:26:44 +0000 |
720 | +++ cloudinit/handlers/upstart_job.py 2014-07-08 04:39:36 +0000 |
721 | @@ -66,7 +66,7 @@ |
722 | |
723 | payload = util.dos2unix(payload) |
724 | path = os.path.join(self.upstart_dir, filename) |
725 | - util.write_file(path, payload, 0644) |
726 | + util.write_file(path, payload, 0o644) |
727 | |
728 | if SUITABLE_UPSTART: |
729 | util.subp(["initctl", "reload-configuration"], capture=False) |
730 | |
731 | === modified file 'cloudinit/helpers.py' |
732 | --- cloudinit/helpers.py 2014-01-17 20:12:31 +0000 |
733 | +++ cloudinit/helpers.py 2014-07-08 04:39:36 +0000 |
734 | @@ -23,10 +23,12 @@ |
735 | from time import time |
736 | |
737 | import contextlib |
738 | -import io |
739 | import os |
740 | |
741 | -from ConfigParser import (NoSectionError, NoOptionError, RawConfigParser) |
742 | +import six |
743 | +from six.moves.configparser import (NoSectionError, |
744 | + NoOptionError, |
745 | + RawConfigParser) |
746 | |
747 | from cloudinit.settings import (PER_INSTANCE, PER_ALWAYS, PER_ONCE, |
748 | CFG_ENV_NAME) |
749 | @@ -318,10 +320,10 @@ |
750 | return self.registered[content_type] |
751 | |
752 | def items(self): |
753 | - return self.registered.items() |
754 | + return list(self.iteritems()) |
755 | |
756 | def iteritems(self): |
757 | - return self.registered.iteritems() |
758 | + return six.iteritems(self.registered) |
759 | |
760 | |
761 | class Paths(object): |
762 | @@ -449,7 +451,7 @@ |
763 | |
764 | def stringify(self, header=None): |
765 | contents = '' |
766 | - with io.BytesIO() as outputstream: |
767 | + with six.StringIO() as outputstream: |
768 | self.write(outputstream) |
769 | outputstream.flush() |
770 | contents = outputstream.getvalue() |
771 | |
772 | === modified file 'cloudinit/log.py' |
773 | --- cloudinit/log.py 2013-04-17 16:42:55 +0000 |
774 | +++ cloudinit/log.py 2014-07-08 04:39:36 +0000 |
775 | @@ -28,7 +28,8 @@ |
776 | import os |
777 | import sys |
778 | |
779 | -from StringIO import StringIO |
780 | +import six |
781 | +from six import StringIO |
782 | |
783 | # Logging levels for easy access |
784 | CRITICAL = logging.CRITICAL |
785 | @@ -72,13 +73,13 @@ |
786 | |
787 | log_cfgs = [] |
788 | log_cfg = cfg.get('logcfg') |
789 | - if log_cfg and isinstance(log_cfg, (str, basestring)): |
790 | + if log_cfg and isinstance(log_cfg, six.string_types): |
791 | # If there is a 'logcfg' entry in the config, |
792 | # respect it, it is the old keyname |
793 | log_cfgs.append(str(log_cfg)) |
794 | elif "log_cfgs" in cfg: |
795 | for a_cfg in cfg['log_cfgs']: |
796 | - if isinstance(a_cfg, (basestring, str)): |
797 | + if isinstance(a_cfg, six.string_types): |
798 | log_cfgs.append(a_cfg) |
799 | elif isinstance(a_cfg, (collections.Iterable)): |
800 | cfg_str = [str(c) for c in a_cfg] |
801 | |
802 | === modified file 'cloudinit/mergers/__init__.py' |
803 | --- cloudinit/mergers/__init__.py 2013-05-03 21:41:28 +0000 |
804 | +++ cloudinit/mergers/__init__.py 2014-07-08 04:39:36 +0000 |
805 | @@ -18,6 +18,8 @@ |
806 | |
807 | import re |
808 | |
809 | +import six |
810 | + |
811 | from cloudinit import importer |
812 | from cloudinit import log as logging |
813 | from cloudinit import type_utils |
814 | @@ -100,7 +102,7 @@ |
815 | raw_mergers = config.pop('merge_type', None) |
816 | if raw_mergers is None: |
817 | return parsed_mergers |
818 | - if isinstance(raw_mergers, (str, basestring)): |
819 | + if isinstance(raw_mergers, six.string_types): |
820 | return string_extract_mergers(raw_mergers) |
821 | for m in raw_mergers: |
822 | if isinstance(m, (dict)): |
823 | |
824 | === modified file 'cloudinit/mergers/m_dict.py' |
825 | --- cloudinit/mergers/m_dict.py 2013-05-03 22:05:45 +0000 |
826 | +++ cloudinit/mergers/m_dict.py 2014-07-08 04:39:36 +0000 |
827 | @@ -16,6 +16,8 @@ |
828 | # You should have received a copy of the GNU General Public License |
829 | # along with this program. If not, see <http://www.gnu.org/licenses/>. |
830 | |
831 | +import six |
832 | + |
833 | DEF_MERGE_TYPE = 'no_replace' |
834 | MERGE_TYPES = ('replace', DEF_MERGE_TYPE,) |
835 | |
836 | @@ -57,7 +59,7 @@ |
837 | return new_v |
838 | if isinstance(new_v, (list, tuple)) and self._recurse_array: |
839 | return self._merger.merge(old_v, new_v) |
840 | - if isinstance(new_v, (basestring)) and self._recurse_str: |
841 | + if isinstance(new_v, six.string_types) and self._recurse_str: |
842 | return self._merger.merge(old_v, new_v) |
843 | if isinstance(new_v, (dict)) and self._recurse_dict: |
844 | return self._merger.merge(old_v, new_v) |
845 | |
846 | === modified file 'cloudinit/mergers/m_list.py' |
847 | --- cloudinit/mergers/m_list.py 2013-06-19 06:46:54 +0000 |
848 | +++ cloudinit/mergers/m_list.py 2014-07-08 04:39:36 +0000 |
849 | @@ -16,6 +16,8 @@ |
850 | # You should have received a copy of the GNU General Public License |
851 | # along with this program. If not, see <http://www.gnu.org/licenses/>. |
852 | |
853 | +import six |
854 | + |
855 | DEF_MERGE_TYPE = 'replace' |
856 | MERGE_TYPES = ('append', 'prepend', DEF_MERGE_TYPE, 'no_replace') |
857 | |
858 | @@ -73,7 +75,7 @@ |
859 | return old_v |
860 | if isinstance(new_v, (list, tuple)) and self._recurse_array: |
861 | return self._merger.merge(old_v, new_v) |
862 | - if isinstance(new_v, (str, basestring)) and self._recurse_str: |
863 | + if isinstance(new_v, six.string_types) and self._recurse_str: |
864 | return self._merger.merge(old_v, new_v) |
865 | if isinstance(new_v, (dict)) and self._recurse_dict: |
866 | return self._merger.merge(old_v, new_v) |
867 | |
868 | === modified file 'cloudinit/mergers/m_str.py' |
869 | --- cloudinit/mergers/m_str.py 2013-05-03 21:41:28 +0000 |
870 | +++ cloudinit/mergers/m_str.py 2014-07-08 04:39:36 +0000 |
871 | @@ -17,6 +17,8 @@ |
872 | # You should have received a copy of the GNU General Public License |
873 | # along with this program. If not, see <http://www.gnu.org/licenses/>. |
874 | |
875 | +import six |
876 | + |
877 | |
878 | class Merger(object): |
879 | def __init__(self, _merger, opts): |
880 | @@ -34,11 +36,11 @@ |
881 | # perform the following action, if appending we will |
882 | # merge them together, otherwise we will just return value. |
883 | def _on_str(self, value, merge_with): |
884 | - if not isinstance(value, (basestring)): |
885 | + if not isinstance(value, six.string_types): |
886 | return merge_with |
887 | if not self._append: |
888 | return merge_with |
889 | - if isinstance(value, unicode): |
890 | - return value + unicode(merge_with) |
891 | + if isinstance(value, six.text_type): |
892 | + return value + six.text_type(merge_with) |
893 | else: |
894 | - return value + str(merge_with) |
895 | + return value + six.binary_type(merge_with) |
896 | |
897 | === modified file 'cloudinit/netinfo.py' |
898 | --- cloudinit/netinfo.py 2014-02-26 18:14:55 +0000 |
899 | +++ cloudinit/netinfo.py 2014-07-08 04:39:36 +0000 |
900 | @@ -20,10 +20,12 @@ |
901 | # You should have received a copy of the GNU General Public License |
902 | # along with this program. If not, see <http://www.gnu.org/licenses/>. |
903 | |
904 | -import cloudinit.util as util |
905 | import re |
906 | |
907 | from prettytable import PrettyTable |
908 | +import six |
909 | + |
910 | +import cloudinit.util as util |
911 | |
912 | |
913 | def netdev_info(empty=""): |
914 | @@ -83,7 +85,7 @@ |
915 | devs[curdev][target] = toks[i][len(field) + 1:] |
916 | |
917 | if empty != "": |
918 | - for (_devname, dev) in devs.iteritems(): |
919 | + for (_devname, dev) in six.iteritems(devs): |
920 | for field in dev: |
921 | if dev[field] == "": |
922 | dev[field] = empty |
923 | @@ -155,7 +157,7 @@ |
924 | if netdev is not None: |
925 | fields = ['Device', 'Up', 'Address', 'Mask', 'Hw-Address'] |
926 | tbl = PrettyTable(fields) |
927 | - for (dev, d) in netdev.iteritems(): |
928 | + for (dev, d) in six.iteritems(netdev): |
929 | tbl.add_row([dev, d["up"], d["addr"], d["mask"], d["hwaddr"]]) |
930 | netdev_s = tbl.get_string() |
931 | max_len = len(max(netdev_s.splitlines(), key=len)) |
932 | |
933 | === modified file 'cloudinit/signal_handler.py' |
934 | --- cloudinit/signal_handler.py 2012-09-19 20:33:56 +0000 |
935 | +++ cloudinit/signal_handler.py 2014-07-08 04:39:36 +0000 |
936 | @@ -22,7 +22,7 @@ |
937 | import signal |
938 | import sys |
939 | |
940 | -from StringIO import StringIO |
941 | +from six import StringIO |
942 | |
943 | from cloudinit import log as logging |
944 | from cloudinit import util |
945 | |
946 | === modified file 'cloudinit/sources/DataSourceConfigDrive.py' |
947 | --- cloudinit/sources/DataSourceConfigDrive.py 2014-02-25 01:17:07 +0000 |
948 | +++ cloudinit/sources/DataSourceConfigDrive.py 2014-07-08 04:39:36 +0000 |
949 | @@ -20,6 +20,8 @@ |
950 | |
951 | import os |
952 | |
953 | +import six |
954 | + |
955 | from cloudinit import log as logging |
956 | from cloudinit import sources |
957 | from cloudinit import util |
958 | @@ -198,7 +200,7 @@ |
959 | files = data.get('files', {}) |
960 | if files: |
961 | LOG.debug("Writing %s injected files", len(files)) |
962 | - for (filename, content) in files.iteritems(): |
963 | + for (filename, content) in six.iteritems(files): |
964 | if not filename.startswith(os.sep): |
965 | filename = os.sep + filename |
966 | try: |
967 | |
968 | === modified file 'cloudinit/sources/DataSourceEc2.py' |
969 | --- cloudinit/sources/DataSourceEc2.py 2014-02-01 20:03:32 +0000 |
970 | +++ cloudinit/sources/DataSourceEc2.py 2014-07-08 04:39:36 +0000 |
971 | @@ -23,6 +23,8 @@ |
972 | import os |
973 | import time |
974 | |
975 | +import six |
976 | + |
977 | from cloudinit import ec2_utils as ec2 |
978 | from cloudinit import log as logging |
979 | from cloudinit import sources |
980 | @@ -156,8 +158,8 @@ |
981 | # 'ephemeral0': '/dev/sdb', |
982 | # 'root': '/dev/sda1'} |
983 | found = None |
984 | - bdm_items = self.metadata['block-device-mapping'].iteritems() |
985 | - for (entname, device) in bdm_items: |
986 | + bdms = self.metadata['block-device-mapping'] |
987 | + for (entname, device) in six.iteritems(bdms): |
988 | if entname == name: |
989 | found = device |
990 | break |
991 | |
992 | === modified file 'cloudinit/sources/DataSourceMAAS.py' |
993 | --- cloudinit/sources/DataSourceMAAS.py 2013-04-25 15:58:38 +0000 |
994 | +++ cloudinit/sources/DataSourceMAAS.py 2014-07-08 04:39:36 +0000 |
995 | @@ -20,11 +20,14 @@ |
996 | |
997 | from email.utils import parsedate |
998 | import errno |
999 | -import oauth.oauth as oauth |
1000 | import os |
1001 | import time |
1002 | import urllib2 |
1003 | |
1004 | +import oauth.oauth as oauth |
1005 | + |
1006 | +import six |
1007 | + |
1008 | from cloudinit import log as logging |
1009 | from cloudinit import sources |
1010 | from cloudinit import url_helper |
1011 | @@ -262,7 +265,7 @@ |
1012 | |
1013 | userdata = content.get('user-data', "") |
1014 | md = {} |
1015 | - for (key, val) in content.iteritems(): |
1016 | + for (key, val) in six.iteritems(content): |
1017 | if key == 'user-data': |
1018 | continue |
1019 | md[key] = val |
1020 | |
1021 | === modified file 'cloudinit/sources/DataSourceOVF.py' |
1022 | --- cloudinit/sources/DataSourceOVF.py 2013-06-07 17:30:03 +0000 |
1023 | +++ cloudinit/sources/DataSourceOVF.py 2014-07-08 04:39:36 +0000 |
1024 | @@ -26,6 +26,8 @@ |
1025 | import os |
1026 | import re |
1027 | |
1028 | +import six |
1029 | + |
1030 | from cloudinit import log as logging |
1031 | from cloudinit import sources |
1032 | from cloudinit import util |
1033 | @@ -66,7 +68,7 @@ |
1034 | np = {'iso': transport_iso9660, |
1035 | 'vmware-guestd': transport_vmware_guestd, } |
1036 | name = None |
1037 | - for (name, transfunc) in np.iteritems(): |
1038 | + for (name, transfunc) in six.iteritems(np): |
1039 | (contents, _dev, _fname) = transfunc() |
1040 | if contents: |
1041 | break |
1042 | @@ -138,7 +140,7 @@ |
1043 | ud = "" |
1044 | cfg_props = ['password'] |
1045 | md_props = ['seedfrom', 'local-hostname', 'public-keys', 'instance-id'] |
1046 | - for (prop, val) in props.iteritems(): |
1047 | + for (prop, val) in six.iteritems(props): |
1048 | if prop == 'hostname': |
1049 | prop = "local-hostname" |
1050 | if prop in md_props: |
1051 | @@ -183,7 +185,7 @@ |
1052 | |
1053 | # Go through mounts to see if it was already mounted |
1054 | mounts = util.mounts() |
1055 | - for (dev, info) in mounts.iteritems(): |
1056 | + for (dev, info) in six.iteritems(mounts): |
1057 | fstype = info['fstype'] |
1058 | if fstype != "iso9660" and require_iso: |
1059 | continue |
1060 | |
1061 | === modified file 'cloudinit/sources/DataSourceSmartOS.py' |
1062 | --- cloudinit/sources/DataSourceSmartOS.py 2014-06-02 20:56:31 +0000 |
1063 | +++ cloudinit/sources/DataSourceSmartOS.py 2014-07-08 04:39:36 +0000 |
1064 | @@ -30,13 +30,15 @@ |
1065 | # Comments with "@datadictionary" are snippets of the definition |
1066 | |
1067 | import base64 |
1068 | +import os |
1069 | +import os.path |
1070 | + |
1071 | +import serial |
1072 | +import six |
1073 | + |
1074 | from cloudinit import log as logging |
1075 | from cloudinit import sources |
1076 | from cloudinit import util |
1077 | -import os |
1078 | -import os.path |
1079 | -import serial |
1080 | - |
1081 | |
1082 | LOG = logging.getLogger(__name__) |
1083 | |
1084 | @@ -201,7 +203,7 @@ |
1085 | if b64_all is not None: |
1086 | self.b64_all = util.is_true(b64_all) |
1087 | |
1088 | - for ci_noun, attribute in SMARTOS_ATTRIB_MAP.iteritems(): |
1089 | + for ci_noun, attribute in six.iteritems(SMARTOS_ATTRIB_MAP): |
1090 | smartos_noun, strip = attribute |
1091 | md[ci_noun] = self.query(smartos_noun, strip=strip) |
1092 | |
1093 | |
1094 | === modified file 'cloudinit/sources/__init__.py' |
1095 | --- cloudinit/sources/__init__.py 2014-01-23 20:17:17 +0000 |
1096 | +++ cloudinit/sources/__init__.py 2014-07-08 04:39:36 +0000 |
1097 | @@ -23,6 +23,8 @@ |
1098 | import abc |
1099 | import os |
1100 | |
1101 | +import six |
1102 | + |
1103 | from cloudinit import importer |
1104 | from cloudinit import log as logging |
1105 | from cloudinit import type_utils |
1106 | @@ -130,7 +132,7 @@ |
1107 | # we want to return the correct value for what will actually |
1108 | # exist in this instance |
1109 | mappings = {"sd": ("vd", "xvd", "vtb")} |
1110 | - for (nfrom, tlist) in mappings.iteritems(): |
1111 | + for (nfrom, tlist) in six.iteritems(mappings): |
1112 | if not short_name.startswith(nfrom): |
1113 | continue |
1114 | for nto in tlist: |
1115 | @@ -218,18 +220,18 @@ |
1116 | if not pubkey_data: |
1117 | return keys |
1118 | |
1119 | - if isinstance(pubkey_data, (basestring, str)): |
1120 | + if isinstance(pubkey_data, six.string_types): |
1121 | return str(pubkey_data).splitlines() |
1122 | |
1123 | if isinstance(pubkey_data, (list, set)): |
1124 | return list(pubkey_data) |
1125 | |
1126 | if isinstance(pubkey_data, (dict)): |
1127 | - for (_keyname, klist) in pubkey_data.iteritems(): |
1128 | + for (_keyname, klist) in six.iteritems(pubkey_data): |
1129 | # lp:506332 uec metadata service responds with |
1130 | # data that makes boto populate a string for 'klist' rather |
1131 | # than a list. |
1132 | - if isinstance(klist, (str, basestring)): |
1133 | + if isinstance(klist, six.string_types): |
1134 | klist = [klist] |
1135 | if isinstance(klist, (list, set)): |
1136 | for pkey in klist: |
1137 | |
1138 | === modified file 'cloudinit/sources/helpers/openstack.py' |
1139 | --- cloudinit/sources/helpers/openstack.py 2014-02-24 22:41:42 +0000 |
1140 | +++ cloudinit/sources/helpers/openstack.py 2014-07-08 04:39:36 +0000 |
1141 | @@ -23,6 +23,8 @@ |
1142 | import copy |
1143 | import os |
1144 | |
1145 | +import six |
1146 | + |
1147 | from cloudinit import ec2_utils |
1148 | from cloudinit import log as logging |
1149 | from cloudinit import sources |
1150 | @@ -224,7 +226,7 @@ |
1151 | 'version': 2, |
1152 | } |
1153 | data = datafiles(version) |
1154 | - for (name, (path, required, translator)) in data.iteritems(): |
1155 | + for (name, (path, required, translator)) in six.iteritems(data): |
1156 | path = self._path_join(self.base_path, path) |
1157 | data = None |
1158 | found = False |
1159 | @@ -344,7 +346,7 @@ |
1160 | raise NonReadable("%s: no files found" % (self.base_path)) |
1161 | |
1162 | md = {} |
1163 | - for (name, (key, translator, default)) in FILES_V1.iteritems(): |
1164 | + for (name, (key, translator, default)) in six.iteritems(FILES_V1): |
1165 | if name in found: |
1166 | path = found[name] |
1167 | try: |
1168 | |
1169 | === modified file 'cloudinit/ssh_util.py' |
1170 | --- cloudinit/ssh_util.py 2013-06-19 06:44:00 +0000 |
1171 | +++ cloudinit/ssh_util.py 2014-07-08 04:39:36 +0000 |
1172 | @@ -239,7 +239,7 @@ |
1173 | # Make sure the users .ssh dir is setup accordingly |
1174 | (ssh_dir, pwent) = users_ssh_info(username) |
1175 | if not os.path.isdir(ssh_dir): |
1176 | - util.ensure_dir(ssh_dir, mode=0700) |
1177 | + util.ensure_dir(ssh_dir, mode=0o700) |
1178 | util.chownbyid(ssh_dir, pwent.pw_uid, pwent.pw_gid) |
1179 | |
1180 | # Turn the 'update' keys given into actual entries |
1181 | @@ -252,8 +252,8 @@ |
1182 | (auth_key_fn, auth_key_entries) = extract_authorized_keys(username) |
1183 | with util.SeLinuxGuard(ssh_dir, recursive=True): |
1184 | content = update_authorized_keys(auth_key_entries, key_entries) |
1185 | - util.ensure_dir(os.path.dirname(auth_key_fn), mode=0700) |
1186 | - util.write_file(auth_key_fn, content, mode=0600) |
1187 | + util.ensure_dir(os.path.dirname(auth_key_fn), mode=0o700) |
1188 | + util.write_file(auth_key_fn, content, mode=0o600) |
1189 | util.chownbyid(auth_key_fn, pwent.pw_uid, pwent.pw_gid) |
1190 | |
1191 | |
1192 | |
1193 | === modified file 'cloudinit/stages.py' |
1194 | --- cloudinit/stages.py 2014-02-13 18:53:08 +0000 |
1195 | +++ cloudinit/stages.py 2014-07-08 04:39:36 +0000 |
1196 | @@ -20,12 +20,13 @@ |
1197 | # You should have received a copy of the GNU General Public License |
1198 | # along with this program. If not, see <http://www.gnu.org/licenses/>. |
1199 | |
1200 | -import cPickle as pickle |
1201 | - |
1202 | import copy |
1203 | import os |
1204 | +import pickle |
1205 | import sys |
1206 | |
1207 | +import six |
1208 | + |
1209 | from cloudinit.settings import (PER_INSTANCE, FREQUENCIES, CLOUD_CONFIG) |
1210 | |
1211 | from cloudinit import handlers |
1212 | @@ -202,7 +203,7 @@ |
1213 | util.logexc(LOG, "Failed pickling datasource %s", self.datasource) |
1214 | return False |
1215 | try: |
1216 | - util.write_file(pickled_fn, pk_contents, mode=0400) |
1217 | + util.write_file(pickled_fn, pk_contents, mode=0o400) |
1218 | except Exception: |
1219 | util.logexc(LOG, "Failed pickling datasource to %s", pickled_fn) |
1220 | return False |
1221 | @@ -324,15 +325,15 @@ |
1222 | |
1223 | def _store_userdata(self): |
1224 | raw_ud = "%s" % (self.datasource.get_userdata_raw()) |
1225 | - util.write_file(self._get_ipath('userdata_raw'), raw_ud, 0600) |
1226 | + util.write_file(self._get_ipath('userdata_raw'), raw_ud, 0o600) |
1227 | processed_ud = "%s" % (self.datasource.get_userdata()) |
1228 | - util.write_file(self._get_ipath('userdata'), processed_ud, 0600) |
1229 | + util.write_file(self._get_ipath('userdata'), processed_ud, 0o600) |
1230 | |
1231 | def _store_vendordata(self): |
1232 | raw_vd = "%s" % (self.datasource.get_vendordata_raw()) |
1233 | - util.write_file(self._get_ipath('vendordata_raw'), raw_vd, 0600) |
1234 | + util.write_file(self._get_ipath('vendordata_raw'), raw_vd, 0o600) |
1235 | processed_vd = "%s" % (self.datasource.get_vendordata()) |
1236 | - util.write_file(self._get_ipath('vendordata'), processed_vd, 0600) |
1237 | + util.write_file(self._get_ipath('vendordata'), processed_vd, 0o600) |
1238 | |
1239 | def _default_handlers(self, opts=None): |
1240 | if opts is None: |
1241 | @@ -384,7 +385,7 @@ |
1242 | if not path or not os.path.isdir(path): |
1243 | return |
1244 | potential_handlers = util.find_modules(path) |
1245 | - for (fname, mod_name) in potential_handlers.iteritems(): |
1246 | + for (fname, mod_name) in six.iteritems(potential_handlers): |
1247 | try: |
1248 | mod_locs = importer.find_module(mod_name, [''], |
1249 | ['list_types', |
1250 | @@ -574,7 +575,7 @@ |
1251 | for item in cfg_mods: |
1252 | if not item: |
1253 | continue |
1254 | - if isinstance(item, (str, basestring)): |
1255 | + if isinstance(item, six.string_types): |
1256 | module_list.append({ |
1257 | 'mod': item.strip(), |
1258 | }) |
1259 | |
1260 | === modified file 'cloudinit/type_utils.py' |
1261 | --- cloudinit/type_utils.py 2013-03-07 03:24:05 +0000 |
1262 | +++ cloudinit/type_utils.py 2014-07-08 04:39:36 +0000 |
1263 | @@ -24,11 +24,30 @@ |
1264 | |
1265 | import types |
1266 | |
1267 | +import six |
1268 | + |
1269 | +if six.PY3: |
1270 | + _NAME_TYPES = ( |
1271 | + types.ModuleType, |
1272 | + types.FunctionType, |
1273 | + types.LambdaType, |
1274 | + type, |
1275 | + ) |
1276 | +else: |
1277 | + _NAME_TYPES = ( |
1278 | + types.TypeType, |
1279 | + types.ModuleType, |
1280 | + types.FunctionType, |
1281 | + types.LambdaType, |
1282 | + types.ClassType, |
1283 | + ) |
1284 | + |
1285 | |
1286 | def obj_name(obj): |
1287 | - if isinstance(obj, (types.TypeType, |
1288 | - types.ModuleType, |
1289 | - types.FunctionType, |
1290 | - types.LambdaType)): |
1291 | - return str(obj.__name__) |
1292 | - return obj_name(obj.__class__) |
1293 | + if isinstance(obj, _NAME_TYPES): |
1294 | + return six.text_type(obj.__name__) |
1295 | + else: |
1296 | + if not hasattr(obj, '__class__'): |
1297 | + return repr(obj) |
1298 | + else: |
1299 | + return obj_name(obj.__class__) |
1300 | |
1301 | === modified file 'cloudinit/url_helper.py' |
1302 | --- cloudinit/url_helper.py 2014-02-13 17:13:42 +0000 |
1303 | +++ cloudinit/url_helper.py 2014-07-08 04:39:36 +0000 |
1304 | @@ -20,21 +20,28 @@ |
1305 | # You should have received a copy of the GNU General Public License |
1306 | # along with this program. If not, see <http://www.gnu.org/licenses/>. |
1307 | |
1308 | -import httplib |
1309 | +import six |
1310 | + |
1311 | import time |
1312 | -import urllib |
1313 | |
1314 | import requests |
1315 | from requests import exceptions |
1316 | |
1317 | -from urlparse import (urlparse, urlunparse) |
1318 | +from six.moves.urllib.parse import (urlparse, urlunparse) |
1319 | +from six.moves.urllib.parse import quote as urlquote |
1320 | |
1321 | from cloudinit import log as logging |
1322 | from cloudinit import version |
1323 | |
1324 | LOG = logging.getLogger(__name__) |
1325 | |
1326 | -NOT_FOUND = httplib.NOT_FOUND |
1327 | +if six.PY2: |
1328 | + import httplib |
1329 | + NOT_FOUND = httplib.NOT_FOUND |
1330 | +else: |
1331 | + import http.client |
1332 | + NOT_FOUND = http.client.NOT_FOUND |
1333 | + |
1334 | |
1335 | # Check if requests has ssl support (added in requests >= 0.8.8) |
1336 | SSL_ENABLED = False |
1337 | @@ -70,7 +77,7 @@ |
1338 | path = url_parsed[2] |
1339 | if path and not path.endswith("/"): |
1340 | path += "/" |
1341 | - path += urllib.quote(str(add_on), safe="/:") |
1342 | + path += urlquote(str(add_on), safe="/:") |
1343 | url_parsed[2] = path |
1344 | return urlunparse(url_parsed) |
1345 | |
1346 | @@ -111,7 +118,7 @@ |
1347 | |
1348 | @property |
1349 | def contents(self): |
1350 | - return self._response.content |
1351 | + return self._response.text |
1352 | |
1353 | @property |
1354 | def url(self): |
1355 | @@ -135,7 +142,7 @@ |
1356 | return self._response.status_code |
1357 | |
1358 | def __str__(self): |
1359 | - return self.contents |
1360 | + return self._response.text |
1361 | |
1362 | |
1363 | class UrlError(IOError): |
1364 | |
1365 | === modified file 'cloudinit/user_data.py' |
1366 | --- cloudinit/user_data.py 2014-01-24 20:29:09 +0000 |
1367 | +++ cloudinit/user_data.py 2014-07-08 04:39:36 +0000 |
1368 | @@ -29,6 +29,8 @@ |
1369 | from email.mime.nonmultipart import MIMENonMultipart |
1370 | from email.mime.text import MIMEText |
1371 | |
1372 | +import six |
1373 | + |
1374 | from cloudinit import handlers |
1375 | from cloudinit import log as logging |
1376 | from cloudinit import util |
1377 | @@ -235,9 +237,9 @@ |
1378 | resp = util.read_file_or_url(include_url, |
1379 | ssl_details=self.ssl_details) |
1380 | if include_once_on and resp.ok(): |
1381 | - util.write_file(include_once_fn, str(resp), mode=0600) |
1382 | + util.write_file(include_once_fn, resp.contents, mode=0o600) |
1383 | if resp.ok(): |
1384 | - content = str(resp) |
1385 | + content = resp.contents |
1386 | else: |
1387 | LOG.warn(("Fetching from %s resulted in" |
1388 | " a invalid http code of %s"), |
1389 | @@ -256,7 +258,7 @@ |
1390 | # filename and type not be present |
1391 | # or |
1392 | # scalar(payload) |
1393 | - if isinstance(ent, (str, basestring)): |
1394 | + if isinstance(ent, six.string_types): |
1395 | ent = {'content': ent} |
1396 | if not isinstance(ent, (dict)): |
1397 | # TODO(harlowja) raise? |
1398 | @@ -337,7 +339,7 @@ |
1399 | data = util.decomp_gzip(raw_data) |
1400 | if "mime-version:" in data[0:4096].lower(): |
1401 | msg = email.message_from_string(data) |
1402 | - for (key, val) in headers.iteritems(): |
1403 | + for (key, val) in six.iteritems(headers): |
1404 | _replace_header(msg, key, val) |
1405 | else: |
1406 | mtype = headers.get(CONTENT_TYPE, NOT_MULTIPART_TYPE) |
1407 | |
1408 | === modified file 'cloudinit/util.py' |
1409 | --- cloudinit/util.py 2014-02-24 22:20:12 +0000 |
1410 | +++ cloudinit/util.py 2014-07-08 04:39:36 +0000 |
1411 | @@ -22,8 +22,6 @@ |
1412 | # |
1413 | # pylint: disable=C0302 |
1414 | |
1415 | -from StringIO import StringIO |
1416 | - |
1417 | import contextlib |
1418 | import copy as obj_copy |
1419 | import ctypes |
1420 | @@ -47,8 +45,10 @@ |
1421 | import sys |
1422 | import tempfile |
1423 | import time |
1424 | -import urlparse |
1425 | - |
1426 | + |
1427 | +from six.moves.urllib import parse as urlparse |
1428 | + |
1429 | +import six |
1430 | import yaml |
1431 | |
1432 | from cloudinit import importer |
1433 | @@ -71,8 +71,25 @@ |
1434 | } |
1435 | FN_ALLOWED = ('_-.()' + string.digits + string.ascii_letters) |
1436 | |
1437 | +TRUE_STRINGS = ('true', '1', 'on', 'yes') |
1438 | +FALSE_STRINGS = ('off', '0', 'no', 'false') |
1439 | + |
1440 | # Helper utils to see if running in a container |
1441 | -CONTAINER_TESTS = ['running-in-container', 'lxc-is-container'] |
1442 | +CONTAINER_TESTS = ('running-in-container', 'lxc-is-container') |
1443 | + |
1444 | + |
1445 | +def decode_binary(blob, encoding='utf-8'): |
1446 | + # Converts a binary type into a text type using given encoding. |
1447 | + if isinstance(blob, six.text_type): |
1448 | + return blob |
1449 | + return blob.decode(encoding) |
1450 | + |
1451 | + |
1452 | +def encode_text(text, encoding='utf-8'): |
1453 | + # Converts a text string into a binary type using given encoding. |
1454 | + if isinstance(text, six.binary_type): |
1455 | + return text |
1456 | + return text.encode(encoding) |
1457 | |
1458 | |
1459 | class ProcessExecutionError(IOError): |
1460 | @@ -149,7 +166,8 @@ |
1461 | if self.selinux and self.selinux.is_selinux_enabled(): |
1462 | path = os.path.realpath(os.path.expanduser(self.path)) |
1463 | # path should be a string, not unicode |
1464 | - path = str(path) |
1465 | + if six.PY2: |
1466 | + path = str(path) |
1467 | do_restore = False |
1468 | try: |
1469 | # See if even worth restoring?? |
1470 | @@ -211,10 +229,10 @@ |
1471 | def is_true(val, addons=None): |
1472 | if isinstance(val, (bool)): |
1473 | return val is True |
1474 | - check_set = ['true', '1', 'on', 'yes'] |
1475 | + check_set = TRUE_STRINGS |
1476 | if addons: |
1477 | - check_set = check_set + addons |
1478 | - if str(val).lower().strip() in check_set: |
1479 | + check_set = list(check_set) + addons |
1480 | + if six.text_type(val).lower().strip() in check_set: |
1481 | return True |
1482 | return False |
1483 | |
1484 | @@ -222,10 +240,10 @@ |
1485 | def is_false(val, addons=None): |
1486 | if isinstance(val, (bool)): |
1487 | return val is False |
1488 | - check_set = ['off', '0', 'no', 'false'] |
1489 | + check_set = FALSE_STRINGS |
1490 | if addons: |
1491 | - check_set = check_set + addons |
1492 | - if str(val).lower().strip() in check_set: |
1493 | + check_set = list(check_set) + addons |
1494 | + if six.text_type(val).lower().strip() in check_set: |
1495 | return True |
1496 | return False |
1497 | |
1498 | @@ -275,7 +293,7 @@ |
1499 | def uniq_merge(*lists): |
1500 | combined_list = [] |
1501 | for a_list in lists: |
1502 | - if isinstance(a_list, (str, basestring)): |
1503 | + if isinstance(a_list, six.string_types): |
1504 | a_list = a_list.strip().split(",") |
1505 | # Kickout the empty ones |
1506 | a_list = [a for a in a_list if len(a)] |
1507 | @@ -284,7 +302,7 @@ |
1508 | |
1509 | |
1510 | def clean_filename(fn): |
1511 | - for (k, v) in FN_REPLACEMENTS.iteritems(): |
1512 | + for (k, v) in six.iteritems(FN_REPLACEMENTS): |
1513 | fn = fn.replace(k, v) |
1514 | removals = [] |
1515 | for k in fn: |
1516 | @@ -298,14 +316,14 @@ |
1517 | |
1518 | def decomp_gzip(data, quiet=True): |
1519 | try: |
1520 | - buf = StringIO(str(data)) |
1521 | + buf = six.BytesIO(encode_text(data)) |
1522 | with contextlib.closing(gzip.GzipFile(None, "rb", 1, buf)) as gh: |
1523 | - return gh.read() |
1524 | + return decode_binary(gh.read()) |
1525 | except Exception as e: |
1526 | if quiet: |
1527 | return data |
1528 | else: |
1529 | - raise DecompressionError(str(e)) |
1530 | + raise DecompressionError(six.text_type(e)) |
1531 | |
1532 | |
1533 | def extract_usergroup(ug_pair): |
1534 | @@ -364,7 +382,7 @@ |
1535 | |
1536 | |
1537 | def load_json(text, root_types=(dict,)): |
1538 | - decoded = json.loads(text) |
1539 | + decoded = json.loads(decode_binary(text)) |
1540 | if not isinstance(decoded, tuple(root_types)): |
1541 | expected_types = ", ".join([str(t) for t in root_types]) |
1542 | raise TypeError("(%s) root types expected, got %s instead" |
1543 | @@ -396,7 +414,7 @@ |
1544 | if key not in yobj: |
1545 | return default |
1546 | val = yobj[key] |
1547 | - if not isinstance(val, (str, basestring)): |
1548 | + if not isinstance(val, six.string_types): |
1549 | val = str(val) |
1550 | return val |
1551 | |
1552 | @@ -431,7 +449,7 @@ |
1553 | if isinstance(val, (list)): |
1554 | cval = [v for v in val] |
1555 | return cval |
1556 | - if not isinstance(val, (basestring)): |
1557 | + if not isinstance(val, six.string_types): |
1558 | val = str(val) |
1559 | return [val] |
1560 | |
1561 | @@ -706,11 +724,11 @@ |
1562 | |
1563 | def load_yaml(blob, default=None, allowed=(dict,)): |
1564 | loaded = default |
1565 | + blob = decode_binary(blob) |
1566 | try: |
1567 | - blob = str(blob) |
1568 | - LOG.debug(("Attempting to load yaml from string " |
1569 | - "of length %s with allowed root types %s"), |
1570 | - len(blob), allowed) |
1571 | + LOG.debug("Attempting to load yaml from string " |
1572 | + "of length %s with allowed root types %s", |
1573 | + len(blob), allowed) |
1574 | converted = safeyaml.load(blob) |
1575 | if not isinstance(converted, allowed): |
1576 | # Yes this will just be caught, but thats ok for now... |
1577 | @@ -744,14 +762,12 @@ |
1578 | md_resp = read_file_or_url(md_url, timeout, retries, file_retries) |
1579 | md = None |
1580 | if md_resp.ok(): |
1581 | - md_str = str(md_resp) |
1582 | - md = load_yaml(md_str, default={}) |
1583 | + md = load_yaml(md_resp.contents, default={}) |
1584 | |
1585 | ud_resp = read_file_or_url(ud_url, timeout, retries, file_retries) |
1586 | ud = None |
1587 | if ud_resp.ok(): |
1588 | - ud_str = str(ud_resp) |
1589 | - ud = ud_str |
1590 | + ud = ud_resp.contents |
1591 | |
1592 | return (md, ud) |
1593 | |
1594 | @@ -782,7 +798,7 @@ |
1595 | if "conf_d" in cfg: |
1596 | confd = cfg['conf_d'] |
1597 | if confd: |
1598 | - if not isinstance(confd, (str, basestring)): |
1599 | + if not isinstance(confd, six.string_types): |
1600 | raise TypeError(("Config file %s contains 'conf_d' " |
1601 | "with non-string type %s") % |
1602 | (cfgfile, type_utils.obj_name(confd))) |
1603 | @@ -919,8 +935,8 @@ |
1604 | return (None, None, None) |
1605 | |
1606 | resp = read_file_or_url(url) |
1607 | - if resp.contents.startswith(starts) and resp.ok(): |
1608 | - return (key, url, str(resp)) |
1609 | + if resp.ok() and resp.contents.startswith(starts): |
1610 | + return (key, url, resp.contents) |
1611 | |
1612 | return (key, url, None) |
1613 | |
1614 | @@ -1074,9 +1090,9 @@ |
1615 | return out_list |
1616 | |
1617 | |
1618 | -def load_file(fname, read_cb=None, quiet=False): |
1619 | +def load_file(fname, read_cb=None, quiet=False, decode=True): |
1620 | LOG.debug("Reading from %s (quiet=%s)", fname, quiet) |
1621 | - ofh = StringIO() |
1622 | + ofh = six.BytesIO() |
1623 | try: |
1624 | with open(fname, 'rb') as ifh: |
1625 | pipe_in_out(ifh, ofh, chunk_cb=read_cb) |
1626 | @@ -1087,7 +1103,10 @@ |
1627 | raise |
1628 | contents = ofh.getvalue() |
1629 | LOG.debug("Read %s bytes from %s", len(contents), fname) |
1630 | - return contents |
1631 | + if decode: |
1632 | + return decode_binary(contents) |
1633 | + else: |
1634 | + return contents |
1635 | |
1636 | |
1637 | def get_cmdline(): |
1638 | @@ -1217,7 +1236,7 @@ |
1639 | |
1640 | def hash_blob(blob, routine, mlen=None): |
1641 | hasher = hashlib.new(routine) |
1642 | - hasher.update(blob) |
1643 | + hasher.update(encode_text(blob)) |
1644 | digest = hasher.hexdigest() |
1645 | # Don't get to long now |
1646 | if mlen is not None: |
1647 | @@ -1248,7 +1267,7 @@ |
1648 | os.rename(src, dest) |
1649 | |
1650 | |
1651 | -def ensure_dirs(dirlist, mode=0755): |
1652 | +def ensure_dirs(dirlist, mode=0o755): |
1653 | for d in dirlist: |
1654 | ensure_dir(d, mode) |
1655 | |
1656 | @@ -1262,7 +1281,7 @@ |
1657 | return |
1658 | try: |
1659 | if key and content: |
1660 | - write_file(target_fn, content, mode=0600) |
1661 | + write_file(target_fn, content, mode=0o600) |
1662 | LOG.debug(("Wrote to %s with contents of command line" |
1663 | " url %s (len=%s)"), target_fn, url, len(content)) |
1664 | elif key and not content: |
1665 | @@ -1454,7 +1473,7 @@ |
1666 | write_file(path, content, omode="ab", mode=None) |
1667 | |
1668 | |
1669 | -def ensure_file(path, mode=0644): |
1670 | +def ensure_file(path, mode=0o644): |
1671 | write_file(path, content='', omode="ab", mode=mode) |
1672 | |
1673 | |
1674 | @@ -1472,7 +1491,7 @@ |
1675 | os.chmod(path, real_mode) |
1676 | |
1677 | |
1678 | -def write_file(filename, content, mode=0644, omode="wb"): |
1679 | +def write_file(filename, content, mode=0o644, omode="wb"): |
1680 | """ |
1681 | Writes a file with the given content and sets the file mode as specified. |
1682 | Resotres the SELinux context if possible. |
1683 | @@ -1480,11 +1499,17 @@ |
1684 | @param filename: The full path of the file to write. |
1685 | @param content: The content to write to the file. |
1686 | @param mode: The filesystem mode to set on the file. |
1687 | - @param omode: The open mode used when opening the file (r, rb, a, etc.) |
1688 | + @param omode: The open mode used when opening the file (w, wb, a, etc.) |
1689 | """ |
1690 | ensure_dir(os.path.dirname(filename)) |
1691 | - LOG.debug("Writing to %s - %s: [%s] %s bytes", |
1692 | - filename, omode, mode, len(content)) |
1693 | + if 'b' in omode.lower(): |
1694 | + content = encode_text(content) |
1695 | + write_type = 'bytes' |
1696 | + else: |
1697 | + content = decode_binary(content) |
1698 | + write_type = 'characters' |
1699 | + LOG.debug("Writing to %s - %s: [%s] %s %s", |
1700 | + filename, omode, mode, len(content), write_type) |
1701 | with SeLinuxGuard(path=filename): |
1702 | with open(filename, omode) as fh: |
1703 | fh.write(content) |
1704 | @@ -1573,10 +1598,10 @@ |
1705 | if isinstance(args, list): |
1706 | fixed = [] |
1707 | for f in args: |
1708 | - fixed.append("'%s'" % (str(f).replace("'", escaped))) |
1709 | + fixed.append("'%s'" % (six.text_type(f).replace("'", escaped))) |
1710 | content = "%s%s\n" % (content, ' '.join(fixed)) |
1711 | cmds_made += 1 |
1712 | - elif isinstance(args, (str, basestring)): |
1713 | + elif isinstance(args, six.string_types): |
1714 | content = "%s%s\n" % (content, args) |
1715 | cmds_made += 1 |
1716 | else: |
1717 | @@ -1687,7 +1712,7 @@ |
1718 | |
1719 | pkglist = [] |
1720 | for pkg in pkgs: |
1721 | - if isinstance(pkg, basestring): |
1722 | + if isinstance(pkg, six.string_types): |
1723 | pkglist.append(pkg) |
1724 | continue |
1725 | |
1726 | |
1727 | === modified file 'packages/bddeb' |
1728 | --- packages/bddeb 2014-01-17 22:08:58 +0000 |
1729 | +++ packages/bddeb 2014-07-08 04:39:36 +0000 |
1730 | @@ -37,6 +37,7 @@ |
1731 | 'pyserial': 'python-serial', |
1732 | 'pyyaml': 'python-yaml', |
1733 | 'requests': 'python-requests', |
1734 | + 'six': 'python-six', |
1735 | } |
1736 | DEBUILD_ARGS = ["-us", "-S", "-uc", "-d"] |
1737 | |
1738 | |
1739 | === modified file 'packages/brpm' |
1740 | --- packages/brpm 2014-01-17 22:08:58 +0000 |
1741 | +++ packages/brpm 2014-07-08 04:39:36 +0000 |
1742 | @@ -44,6 +44,7 @@ |
1743 | 'pyserial': 'pyserial', |
1744 | 'pyyaml': 'PyYAML', |
1745 | 'requests': 'python-requests', |
1746 | + 'six': 'python-six', |
1747 | }, |
1748 | 'suse': { |
1749 | 'argparse': 'python-argparse', |
1750 | @@ -55,6 +56,7 @@ |
1751 | 'pyserial': 'python-pyserial', |
1752 | 'pyyaml': 'python-yaml', |
1753 | 'requests': 'python-requests', |
1754 | + 'six': 'python-six', |
1755 | } |
1756 | } |
1757 | |
1758 | |
1759 | === modified file 'requirements.txt' |
1760 | --- requirements.txt 2014-02-12 10:14:49 +0000 |
1761 | +++ requirements.txt 2014-07-08 04:39:36 +0000 |
1762 | @@ -31,3 +31,6 @@ |
1763 | |
1764 | # For patching pieces of cloud-config together |
1765 | jsonpatch |
1766 | + |
1767 | +# For py2/3 simultaneous compatibility |
1768 | +six |
1769 | |
1770 | === modified file 'tests/unittests/test_data.py' |
1771 | --- tests/unittests/test_data.py 2014-01-17 15:27:09 +0000 |
1772 | +++ tests/unittests/test_data.py 2014-07-08 04:39:36 +0000 |
1773 | @@ -1,11 +1,12 @@ |
1774 | """Tests for handling of userdata within cloud init.""" |
1775 | |
1776 | -import StringIO |
1777 | - |
1778 | import gzip |
1779 | import logging |
1780 | import os |
1781 | |
1782 | +from six import BytesIO |
1783 | +from six import StringIO |
1784 | + |
1785 | from email.mime.application import MIMEApplication |
1786 | from email.mime.base import MIMEBase |
1787 | from email.mime.multipart import MIMEMultipart |
1788 | @@ -53,7 +54,7 @@ |
1789 | self.patchUtils(root) |
1790 | |
1791 | def capture_log(self, lvl=logging.DEBUG): |
1792 | - log_file = StringIO.StringIO() |
1793 | + log_file = StringIO() |
1794 | self._log_handler = logging.StreamHandler(log_file) |
1795 | self._log_handler.setLevel(lvl) |
1796 | self._log = log.getLogger() |
1797 | @@ -352,9 +353,9 @@ |
1798 | """Tests that individual message gzip encoding works.""" |
1799 | |
1800 | def gzip_part(text): |
1801 | - contents = StringIO.StringIO() |
1802 | - f = gzip.GzipFile(fileobj=contents, mode='w') |
1803 | - f.write(str(text)) |
1804 | + contents = BytesIO() |
1805 | + f = gzip.GzipFile(fileobj=contents, mode='wb') |
1806 | + f.write(util.encode_text(text)) |
1807 | f.flush() |
1808 | f.close() |
1809 | return MIMEApplication(contents.getvalue(), 'gzip') |
1810 | |
1811 | === modified file 'tests/unittests/test_datasource/test_nocloud.py' |
1812 | --- tests/unittests/test_datasource/test_nocloud.py 2014-02-26 19:28:46 +0000 |
1813 | +++ tests/unittests/test_datasource/test_nocloud.py 2014-07-08 04:39:36 +0000 |
1814 | @@ -86,7 +86,7 @@ |
1815 | |
1816 | data = { |
1817 | 'fs_label': None, |
1818 | - 'meta-data': {'instance-id': 'IID'}, |
1819 | + 'meta-data': yaml.safe_dump({'instance-id': 'IID'}), |
1820 | 'user-data': "USER_DATA_RAW", |
1821 | } |
1822 | |
1823 | |
1824 | === modified file 'tests/unittests/test_datasource/test_openstack.py' |
1825 | --- tests/unittests/test_datasource/test_openstack.py 2014-02-08 20:20:33 +0000 |
1826 | +++ tests/unittests/test_datasource/test_openstack.py 2014-07-08 04:39:36 +0000 |
1827 | @@ -20,9 +20,8 @@ |
1828 | import json |
1829 | import re |
1830 | |
1831 | -from StringIO import StringIO |
1832 | - |
1833 | -from urlparse import urlparse |
1834 | +from six import StringIO |
1835 | +from six.moves.urllib.parse import urlparse |
1836 | |
1837 | from tests.unittests import helpers as test_helpers |
1838 | |
1839 | |
1840 | === modified file 'tests/unittests/test_distros/test_netconfig.py' |
1841 | --- tests/unittests/test_distros/test_netconfig.py 2012-10-11 19:49:45 +0000 |
1842 | +++ tests/unittests/test_distros/test_netconfig.py 2014-07-08 04:39:36 +0000 |
1843 | @@ -4,6 +4,8 @@ |
1844 | |
1845 | import os |
1846 | |
1847 | +from six import StringIO |
1848 | + |
1849 | from cloudinit import distros |
1850 | from cloudinit import helpers |
1851 | from cloudinit import settings |
1852 | @@ -11,9 +13,6 @@ |
1853 | |
1854 | from cloudinit.distros.parsers.sys_conf import SysConf |
1855 | |
1856 | -from StringIO import StringIO |
1857 | - |
1858 | - |
1859 | BASE_NET_CFG = ''' |
1860 | auto lo |
1861 | iface lo inet loopback |
1862 | |
1863 | === modified file 'tests/unittests/test_handler/test_handler_locale.py' |
1864 | --- tests/unittests/test_handler/test_handler_locale.py 2013-06-25 06:57:27 +0000 |
1865 | +++ tests/unittests/test_handler/test_handler_locale.py 2014-07-08 04:39:36 +0000 |
1866 | @@ -29,7 +29,7 @@ |
1867 | |
1868 | from configobj import ConfigObj |
1869 | |
1870 | -from StringIO import StringIO |
1871 | +from six import BytesIO |
1872 | |
1873 | import logging |
1874 | |
1875 | @@ -59,6 +59,6 @@ |
1876 | cc = self._get_cloud('sles') |
1877 | cc_locale.handle('cc_locale', cfg, cc, LOG, []) |
1878 | |
1879 | - contents = util.load_file('/etc/sysconfig/language') |
1880 | - n_cfg = ConfigObj(StringIO(contents)) |
1881 | + contents = util.load_file('/etc/sysconfig/language', decode=False) |
1882 | + n_cfg = ConfigObj(BytesIO(contents)) |
1883 | self.assertEquals({'RC_LANG': cfg['locale']}, dict(n_cfg)) |
1884 | |
1885 | === modified file 'tests/unittests/test_handler/test_handler_seed_random.py' |
1886 | --- tests/unittests/test_handler/test_handler_seed_random.py 2014-03-04 19:35:09 +0000 |
1887 | +++ tests/unittests/test_handler/test_handler_seed_random.py 2014-07-08 04:39:36 +0000 |
1888 | @@ -22,7 +22,7 @@ |
1889 | import gzip |
1890 | import tempfile |
1891 | |
1892 | -from StringIO import StringIO |
1893 | +from six import StringIO |
1894 | |
1895 | from cloudinit import cloud |
1896 | from cloudinit import distros |
1897 | |
1898 | === modified file 'tests/unittests/test_handler/test_handler_set_hostname.py' |
1899 | --- tests/unittests/test_handler/test_handler_set_hostname.py 2013-06-25 06:57:27 +0000 |
1900 | +++ tests/unittests/test_handler/test_handler_set_hostname.py 2014-07-08 04:39:36 +0000 |
1901 | @@ -9,7 +9,7 @@ |
1902 | |
1903 | import logging |
1904 | |
1905 | -from StringIO import StringIO |
1906 | +from six import BytesIO |
1907 | |
1908 | from configobj import ConfigObj |
1909 | |
1910 | @@ -37,8 +37,8 @@ |
1911 | self.patchUtils(self.tmp) |
1912 | cc_set_hostname.handle('cc_set_hostname', |
1913 | cfg, cc, LOG, []) |
1914 | - contents = util.load_file("/etc/sysconfig/network") |
1915 | - n_cfg = ConfigObj(StringIO(contents)) |
1916 | + contents = util.load_file("/etc/sysconfig/network", decode=False) |
1917 | + n_cfg = ConfigObj(BytesIO(contents)) |
1918 | self.assertEquals({'HOSTNAME': 'blah.blah.blah.yahoo.com'}, |
1919 | dict(n_cfg)) |
1920 | |
1921 | |
1922 | === modified file 'tests/unittests/test_handler/test_handler_timezone.py' |
1923 | --- tests/unittests/test_handler/test_handler_timezone.py 2013-06-25 06:57:27 +0000 |
1924 | +++ tests/unittests/test_handler/test_handler_timezone.py 2014-07-08 04:39:36 +0000 |
1925 | @@ -29,7 +29,7 @@ |
1926 | |
1927 | from configobj import ConfigObj |
1928 | |
1929 | -from StringIO import StringIO |
1930 | +from six import BytesIO |
1931 | |
1932 | import logging |
1933 | |
1934 | @@ -67,8 +67,8 @@ |
1935 | |
1936 | cc_timezone.handle('cc_timezone', cfg, cc, LOG, []) |
1937 | |
1938 | - contents = util.load_file('/etc/sysconfig/clock') |
1939 | - n_cfg = ConfigObj(StringIO(contents)) |
1940 | + contents = util.load_file('/etc/sysconfig/clock', decode=False) |
1941 | + n_cfg = ConfigObj(BytesIO(contents)) |
1942 | self.assertEquals({'TIMEZONE': cfg['timezone']}, dict(n_cfg)) |
1943 | |
1944 | contents = util.load_file('/etc/localtime') |
1945 | |
1946 | === modified file 'tests/unittests/test_handler/test_handler_yum_add_repo.py' |
1947 | --- tests/unittests/test_handler/test_handler_yum_add_repo.py 2014-04-01 18:20:57 +0000 |
1948 | +++ tests/unittests/test_handler/test_handler_yum_add_repo.py 2014-07-08 04:39:36 +0000 |
1949 | @@ -6,7 +6,7 @@ |
1950 | |
1951 | import logging |
1952 | |
1953 | -from StringIO import StringIO |
1954 | +from six import BytesIO |
1955 | |
1956 | import configobj |
1957 | |
1958 | @@ -52,8 +52,9 @@ |
1959 | } |
1960 | self.patchUtils(self.tmp) |
1961 | cc_yum_add_repo.handle('yum_add_repo', cfg, None, LOG, []) |
1962 | - contents = util.load_file("/etc/yum.repos.d/epel_testing.repo") |
1963 | - contents = configobj.ConfigObj(StringIO(contents)) |
1964 | + contents = util.load_file("/etc/yum.repos.d/epel_testing.repo", |
1965 | + decode=False) |
1966 | + contents = configobj.ConfigObj(BytesIO(contents)) |
1967 | expected = { |
1968 | 'epel_testing': { |
1969 | 'name': 'Extra Packages for Enterprise Linux 5 - Testing', |
I'm also interested in cloud-init being compatible with Python 3. I don't understand cloud-init codebase that much, but the patch looks fine. My question is, what minimal Python version are you targeting? I'm assuming 2.6 and higher? If so, I'd recommend using dict.{items, keys,values} instead of six.iter{ items,keys, values} (dict) - it's more readable (but really just a nitpick).