Merge lp:~stub/charm-helpers/py3-2 into lp:charm-helpers

[Stuart Bishop]

Per https://code.launchpad.net/~stub/charm-helpers/py3/+merge/242639,
precise charms attempting to use the Python 3 updated charm-helperers generally will not work, as the version of six they will have installed is very, very old.

This branch backports charm-helpers to use six 1.1, rather than six 1.18.

I believe this version will be good enough until precise charms stop being maintained by maintained, or until the charm-helpers distribution story is improved (if we distribute charm-helpers as a package or similar, we can distribute modern six with it).

An alternative approach I looked at on the previous MP was embedding a copy of six. This worked fine, but would have required people to sync six.py into their charm as well as the rest of charm-helpers.

[lp:~stub/charm-helpers/py3-2]

196. By Stuart Bishop on 2014-11-25

Bootstrap six on import

197. By Stuart Bishop on 2014-11-25

Bootstrap yaml package

198. By Stuart Bishop on 2014-11-25

Fix reliance on implicit casts, such as treating subprocess.check_output result as text instead of bytes

199. By Stuart Bishop on 2014-11-25

Revert Py3 syntax hidden by mocking

200. By Stuart Bishop on 2014-11-25

antique six

201. By Stuart Bishop on 2014-11-25

Need modern distribute under precise

[Tim Van Steenburgh (tvansteenburgh)]

Running make test in a clean precise lxc:

Checking for Python syntax...
Py2 OK
charmhelpers/contrib/templating/contexts.py:99:54: E901 SyntaxError: invalid syntax
tests/contrib/templating/test_contexts.py:137:31: E901 SyntaxError: invalid syntax
tests/core/test_hookenv.py:132:40: E901 SyntaxError: invalid syntax
make: *** [lint] Error 1

Also, running apt-get as an import side-effect makes me grimace.

review: Needs Fixing

[lp:~stub/charm-helpers/py3-2]

202. By Stuart Bishop on 2014-11-25

Python3.2 backport for precise

203. By Stuart Bishop on 2014-11-25

Don't need simplejson

[Jorge Niedbalski (niedbalski)]

My preliminary review shows everything OK.

- Trusty tests OK [python2.7, python3.4]
   - No import errors

- Precise tests OK [python2.7, python3.2]
   - No import errors

---------------------------------------------------------------------------
TOTAL 3683 236 94%
----------------------------------------------------------------------
Ran 773 tests in 7.484s

OK (SKIP=7)
OK

ubuntu@precise:~/charm-helpers$ sudo python2.7
Python 2.7.3 (default, Feb 27 2014, 19:58:35)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import charmhelpers
>>>

Python 3.2.3 (default, Feb 27 2014, 21:31:18)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import charmhelpers
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
  python3-six
0 upgraded, 1 newly installed, 0 to remove and 54 not upgraded.
Need to get 5,894 B of archives.
After this operation, 50.2 kB of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu/ precise/universe python3-six all 1.1.0-2 [5,894 B]
Fetched 5,894 B in 0s (16.7 kB/s)
Selecting previously unselected package python3-six.
(Reading database ... 58662 files and directories currently installed.)
Unpacking python3-six (from .../python3-six_1.1.0-2_all.deb) ...
Setting up python3-six (1.1.0-2) ...
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
  python3-yaml
0 upgraded, 1 newly installed, 0 to remove and 54 not upgraded.
Need to get 111 kB of archives.
After this operation, 442 kB of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu/ precise/main python3-yaml amd64 3.10-2 [111 kB]
Fetched 111 kB in 0s (120 kB/s)
Selecting previously unselected package python3-yaml.
(Reading database ... 58667 files and directories currently installed.)
Unpacking python3-yaml (from .../python3-yaml_3.10-2_amd64.deb) ...
Setting up python3-yaml (3.10-2) ...
>>>

[lp:~stub/charm-helpers/py3-2]

204. By Stuart Bishop on 2014-11-25

merge trunk, revert reversion

[Jorge Niedbalski (niedbalski)]

After the latest merge, conflicts resolved. LGTM +1

review: Approve

[Tim Van Steenburgh (tvansteenburgh)]

> Running make test in a clean precise lxc:
>
> Checking for Python syntax...
> Py2 OK
> charmhelpers/contrib/templating/contexts.py:99:54: E901 SyntaxError: invalid
> syntax
> tests/contrib/templating/test_contexts.py:137:31: E901 SyntaxError: invalid
> syntax
> tests/core/test_hookenv.py:132:40: E901 SyntaxError: invalid syntax
> make: *** [lint] Error 1
>
>
> Also, running apt-get as an import side-effect makes me grimace.

Retested on precise and tests pass. Sorry for the noise, not sure what happened the first time.

[Tim Van Steenburgh (tvansteenburgh)]

+1 LGTM

review: Approve

=== modified file '.bzrignore'
--- .bzrignore	2014-11-25 14:35:22 +0000
+++ .bzrignore	2014-11-25 15:09:14 +0000
@@ -7,3 +7,5 @@
 .env/
 coverage.xml
 docs/_build
+.venv
+.venv3

=== modified file 'Makefile'
--- Makefile	2014-11-25 14:35:22 +0000
+++ Makefile	2014-11-25 15:09:14 +0000
@@ -23,11 +23,12 @@
 	python setup.py sdist
 
 clean:
-	python setup.py clean
+	-python setup.py clean
 	rm -rf build/ MANIFEST
 	find . -name '*.pyc' -delete
 	rm -rf dist/*
 	rm -rf .venv
+	rm -rf .venv3
 	(which dh_clean && dh_clean) || true
 
 userinstall:
@@ -38,19 +39,41 @@
 	sudo apt-get install -y gcc python-dev python-virtualenv python-apt
 	virtualenv .venv --system-site-packages
 	.venv/bin/pip install -U pip
+	.venv/bin/pip install -U distribute
 	.venv/bin/pip install -I -r test_requirements.txt
-
-test: .venv
-	@echo Starting tests...
+	.venv/bin/pip install bzr
+	.venv/bin/pip install GitPython
+
+.venv3:
+	sudo apt-get install -y gcc python3-dev python-virtualenv python3-apt
+	virtualenv .venv3 --python=python3 --system-site-packages
+	.venv3/bin/pip install -U pip
+	.venv3/bin/pip install -U distribute
+	.venv3/bin/pip install -I -r test_requirements.txt
+
+# Note we don't even attempt to run tests if lint isn't passing.
+test: lint test2 test3
+	@echo OK
+
+test2:
+	@echo Starting Py2 tests...
 	.venv/bin/nosetests -s --nologcapture tests/
 
-ftest: .venv
+test3:
+	@echo Starting Py3 tests...
+	.venv3/bin/nosetests -s --nologcapture tests/
+
+ftest: lint
 	@echo Starting fast tests...
 	.venv/bin/nosetests --attr '!slow' --nologcapture tests/
+	.venv3/bin/nosetests --attr '!slow' --nologcapture tests/
 
-lint:
+lint: .venv .venv3
 	@echo Checking for Python syntax...
-	@flake8 --ignore=E123,E501 $(PROJECT) $(TESTS) && echo OK
+	@.venv/bin/flake8 --ignore=E501 $(PROJECT) $(TESTS) tools/ \
+	    && echo Py2 OK
+	@.venv3/bin/flake8 --ignore=E501 $(PROJECT) $(TESTS) tools/ \
+	    && echo Py3 OK
 
 docs:
 	- [ -z "`dpkg -l | grep python-sphinx`" ] && sudo apt-get install python-sphinx -y

=== modified file 'charmhelpers/__init__.py'
--- charmhelpers/__init__.py	2013-05-11 19:55:58 +0000
+++ charmhelpers/__init__.py	2014-11-25 15:09:14 +0000
@@ -0,0 +1,22 @@
+# Bootstrap charm-helpers, installing its dependencies if necessary using
+# only standard libraries.
+import subprocess
+import sys
+
+try:
+    import six  # flake8: noqa
+except ImportError:
+    if sys.version_info.major == 2:
+        subprocess.check_call(['apt-get', 'install', '-y', 'python-six'])
+    else:
+        subprocess.check_call(['apt-get', 'install', '-y', 'python3-six'])
+    import six  # flake8: noqa
+
+try:
+    import yaml  # flake8: noqa
+except ImportError:
+    if sys.version_info.major == 2:
+        subprocess.check_call(['apt-get', 'install', '-y', 'python-yaml'])
+    else:
+        subprocess.check_call(['apt-get', 'install', '-y', 'python3-yaml'])
+    import yaml  # flake8: noqa

=== modified file 'charmhelpers/cli/__init__.py'
--- charmhelpers/cli/__init__.py	2014-11-25 14:35:22 +0000
+++ charmhelpers/cli/__init__.py	2014-11-25 15:09:14 +0000
@@ -1,8 +1,9 @@
 import inspect
-import itertools
 import argparse
 import sys
 
+from six.moves import zip
+
 
 class OutputFormatter(object):
     def __init__(self, outfile=sys.stdout):
@@ -136,7 +137,7 @@
     if argspec.defaults:
         positional_args = argspec.args[:-len(argspec.defaults)]
         keyword_names = argspec.args[-len(argspec.defaults):]
-        for arg, default in itertools.izip(keyword_names, argspec.defaults):
+        for arg, default in zip(keyword_names, argspec.defaults):
             yield ('--{}'.format(arg),), {'default': default}
     else:
         positional_args = argspec.args

=== modified file 'charmhelpers/contrib/amulet/deployment.py'
--- charmhelpers/contrib/amulet/deployment.py	2014-11-25 14:35:22 +0000
+++ charmhelpers/contrib/amulet/deployment.py	2014-11-25 15:09:14 +0000
@@ -1,6 +1,6 @@
 import amulet
-
 import os
+import six
 
 
 class AmuletDeployment(object):
@@ -52,12 +52,12 @@
 
     def _add_relations(self, relations):
         """Add all of the relations for the services."""
-        for k, v in relations.iteritems():
+        for k, v in six.iteritems(relations):
             self.d.relate(k, v)
 
     def _configure_services(self, configs):
         """Configure all of the services."""
-        for service, config in configs.iteritems():
+        for service, config in six.iteritems(configs):
             self.d.configure(service, config)
 
     def _deploy(self):

=== modified file 'charmhelpers/contrib/amulet/utils.py'
--- charmhelpers/contrib/amulet/utils.py	2014-11-25 14:35:22 +0000
+++ charmhelpers/contrib/amulet/utils.py	2014-11-25 15:09:14 +0000
@@ -5,6 +5,8 @@
 import sys
 import time
 
+import six
+
 
 class AmuletUtils(object):
     """Amulet utilities.
@@ -58,7 +60,7 @@
            Verify the specified services are running on the corresponding
            service units.
            """
-        for k, v in commands.iteritems():
+        for k, v in six.iteritems(commands):
             for cmd in v:
                 output, code = k.run(cmd)
                 if code != 0:
@@ -100,11 +102,11 @@
            longs, or can be a function that evaluate a variable and returns a
            bool.
            """
-        for k, v in expected.iteritems():
+        for k, v in six.iteritems(expected):
             if k in actual:
-                if (isinstance(v, basestring) or
+                if (isinstance(v, six.string_types) or
                         isinstance(v, bool) or
-                        isinstance(v, (int, long))):
+                        isinstance(v, six.integer_types)):
                     if v != actual[k]:
                         return "{}:{}".format(k, actual[k])
                 elif not v(actual[k]):

=== modified file 'charmhelpers/contrib/charmhelpers/__init__.py'
--- charmhelpers/contrib/charmhelpers/__init__.py	2014-11-25 14:35:22 +0000
+++ charmhelpers/contrib/charmhelpers/__init__.py	2014-11-25 15:09:14 +0000
@@ -8,19 +8,19 @@
 
 __metaclass__ = type
 __all__ = [
-    #'get_config',             # core.hookenv.config()
-    #'log',                    # core.hookenv.log()
-    #'log_entry',              # core.hookenv.log()
-    #'log_exit',               # core.hookenv.log()
-    #'relation_get',           # core.hookenv.relation_get()
-    #'relation_set',           # core.hookenv.relation_set()
-    #'relation_ids',           # core.hookenv.relation_ids()
-    #'relation_list',          # core.hookenv.relation_units()
-    #'config_get',             # core.hookenv.config()
-    #'unit_get',               # core.hookenv.unit_get()
-    #'open_port',              # core.hookenv.open_port()
-    #'close_port',             # core.hookenv.close_port()
-    #'service_control',        # core.host.service()
+    # 'get_config',             # core.hookenv.config()
+    # 'log',                    # core.hookenv.log()
+    # 'log_entry',              # core.hookenv.log()
+    # 'log_exit',               # core.hookenv.log()
+    # 'relation_get',           # core.hookenv.relation_get()
+    # 'relation_set',           # core.hookenv.relation_set()
+    # 'relation_ids',           # core.hookenv.relation_ids()
+    # 'relation_list',          # core.hookenv.relation_units()
+    # 'config_get',             # core.hookenv.config()
+    # 'unit_get',               # core.hookenv.unit_get()
+    # 'open_port',              # core.hookenv.open_port()
+    # 'close_port',             # core.hookenv.close_port()
+    # 'service_control',        # core.host.service()
     'unit_info',              # client-side, NOT IMPLEMENTED
     'wait_for_machine',       # client-side, NOT IMPLEMENTED
     'wait_for_page_contents',  # client-side, NOT IMPLEMENTED
@@ -31,17 +31,24 @@
 import operator
 import tempfile
 import time
-import urllib2
 import yaml
 import subprocess
 
+import six
+if six.PY3:
+    from urllib.request import urlopen
+    from urllib.error import (HTTPError, URLError)
+else:
+    from urllib2 import (urlopen, HTTPError, URLError)
+
+
 SLEEP_AMOUNT = 0.1
 # We create a juju_status Command here because it makes testing much,
 # much easier.
 juju_status = lambda: subprocess.check_call(['juju', 'status'])
 
 # re-implemented as charmhelpers.fetch.configure_sources()
-#def configure_source(update=False):
+# def configure_source(update=False):
 #    source = config_get('source')
 #    if ((source.startswith('ppa:') or
 #         source.startswith('cloud:') or
@@ -55,7 +62,7 @@
 
 # DEPRECATED: client-side only
 def make_charm_config_file(charm_config):
-    charm_config_file = tempfile.NamedTemporaryFile()
+    charm_config_file = tempfile.NamedTemporaryFile(mode='w+')
     charm_config_file.write(yaml.dump(charm_config))
     charm_config_file.flush()
     # The NamedTemporaryFile instance is returned instead of just the name
@@ -119,7 +126,7 @@
         # we're in LXC.
         machine_data = get_machine_data()
         non_zookeeper_machines = [
-            machine_data[key] for key in machine_data.keys()[1:]]
+            machine_data[key] for key in list(machine_data.keys())[1:]]
         if len(non_zookeeper_machines) >= num_machines:
             all_machines_running = True
             for machine in non_zookeeper_machines:
@@ -170,8 +177,8 @@
     start_time = time.time()
     while True:
         try:
-            stream = urllib2.urlopen(url)
-        except (urllib2.HTTPError, urllib2.URLError):
+            stream = urlopen(url)
+        except (HTTPError, URLError):
             pass
         else:
             page = stream.read()

=== modified file 'charmhelpers/contrib/hahelpers/cluster.py'
--- charmhelpers/contrib/hahelpers/cluster.py	2014-11-25 14:35:22 +0000
+++ charmhelpers/contrib/hahelpers/cluster.py	2014-11-25 15:09:14 +0000
@@ -13,9 +13,10 @@
 
 import subprocess
 import os
-
 from socket import gethostname as get_unit_hostname
 
+import six
+
 from charmhelpers.core.hookenv import (
     log,
     relation_ids,
@@ -77,7 +78,7 @@
         "show", resource
     ]
     try:
-        status = subprocess.check_output(cmd)
+        status = subprocess.check_output(cmd).decode('UTF-8')
     except subprocess.CalledProcessError:
         return False
     else:
@@ -197,7 +198,7 @@
     for setting in settings:
         conf[setting] = config_get(setting)
     missing = []
-    [missing.append(s) for s, v in conf.iteritems() if v is None]
+    [missing.append(s) for s, v in six.iteritems(conf) if v is None]
     if missing:
         log('Insufficient config data to configure hacluster.', level=ERROR)
         raise HAIncompleteConfig

=== modified file 'charmhelpers/contrib/network/ip.py'
--- charmhelpers/contrib/network/ip.py	2014-11-25 14:35:22 +0000
+++ charmhelpers/contrib/network/ip.py	2014-11-25 15:09:14 +0000
@@ -228,7 +228,7 @@
         raise Exception("Interface '%s' doesn't have any %s addresses." %
                         (iface, inet_type))
 
-    return addresses
+    return sorted(addresses)
 
 
 get_ipv4_addr = partial(get_iface_addr, inet_type='AF_INET')
@@ -302,7 +302,7 @@
         if global_addrs:
             # Make sure any found global addresses are not temporary
             cmd = ['ip', 'addr', 'show', iface]
-            out = subprocess.check_output(cmd)
+            out = subprocess.check_output(cmd).decode('UTF-8')
             if dynamic_only:
                 key = re.compile("inet6 (.+)/[0-9]+ scope global dynamic.*")
             else:

=== modified file 'charmhelpers/contrib/openstack/amulet/deployment.py'
--- charmhelpers/contrib/openstack/amulet/deployment.py	2014-11-25 14:35:22 +0000
+++ charmhelpers/contrib/openstack/amulet/deployment.py	2014-11-25 15:09:14 +0000
@@ -1,3 +1,4 @@
+import six
 from charmhelpers.contrib.amulet.deployment import (
     AmuletDeployment
 )
@@ -69,7 +70,7 @@
 
     def _configure_services(self, configs):
         """Configure all of the services."""
-        for service, config in configs.iteritems():
+        for service, config in six.iteritems(configs):
             self.d.configure(service, config)
 
     def _get_openstack_release(self):

=== modified file 'charmhelpers/contrib/openstack/amulet/utils.py'
--- charmhelpers/contrib/openstack/amulet/utils.py	2014-11-25 14:35:22 +0000
+++ charmhelpers/contrib/openstack/amulet/utils.py	2014-11-25 15:09:14 +0000
@@ -7,6 +7,8 @@
 import keystoneclient.v2_0 as keystone_client
 import novaclient.v1_1.client as nova_client
 
+import six
+
 from charmhelpers.contrib.amulet.utils import (
     AmuletUtils
 )
@@ -60,7 +62,7 @@
            expected service catalog endpoints.
            """
         self.log.debug('actual: {}'.format(repr(actual)))
-        for k, v in expected.iteritems():
+        for k, v in six.iteritems(expected):
             if k in actual:
                 ret = self._validate_dict_data(expected[k][0], actual[k][0])
                 if ret:

=== modified file 'charmhelpers/contrib/openstack/context.py'
--- charmhelpers/contrib/openstack/context.py	2014-11-25 14:35:22 +0000
+++ charmhelpers/contrib/openstack/context.py	2014-11-25 15:09:14 +0000
@@ -1,10 +1,11 @@
 import json
 import os
 import time
-
 from base64 import b64decode
 from subprocess import check_call
 
+import six
+
 from charmhelpers.fetch import (
     apt_install,
     filter_installed_packages,
@@ -69,7 +70,7 @@
 
 def context_complete(ctxt):
     _missing = []
-    for k, v in ctxt.iteritems():
+    for k, v in six.iteritems(ctxt):
         if v is None or v == '':
             _missing.append(k)
 
@@ -97,7 +98,7 @@
     split = config_flags.strip(' =').split('=')
     limit = len(split)
     flags = {}
-    for i in xrange(0, limit - 1):
+    for i in range(0, limit - 1):
         current = split[i]
         next = split[i + 1]
         vindex = next.rfind(',')
@@ -375,7 +376,7 @@
                     host = format_ipv6_addr(host) or host
                     rabbitmq_hosts.append(host)
 
-                ctxt['rabbitmq_hosts'] = ','.join(rabbitmq_hosts)
+                ctxt['rabbitmq_hosts'] = ','.join(sorted(rabbitmq_hosts))
 
         if not context_complete(ctxt):
             return {}
@@ -408,7 +409,7 @@
                 ceph_addr = format_ipv6_addr(ceph_addr) or ceph_addr
                 mon_hosts.append(ceph_addr)
 
-        ctxt = {'mon_hosts': ' '.join(mon_hosts),
+        ctxt = {'mon_hosts': ' '.join(sorted(mon_hosts)),
                 'auth': auth,
                 'key': key,
                 'use_syslog': use_syslog}
@@ -587,7 +588,7 @@
                     if k.startswith('ssl_key_'):
                         cns.append(k.lstrip('ssl_key_'))
 
-        return list(set(cns))
+        return sorted(list(set(cns)))
 
     def get_network_addresses(self):
         """For each network configured, return corresponding address and vip
@@ -631,10 +632,10 @@
             else:
                 addresses.append((addr, addr))
 
-        return addresses
+        return sorted(addresses)
 
     def __call__(self):
-        if isinstance(self.external_ports, basestring):
+        if isinstance(self.external_ports, six.string_types):
             self.external_ports = [self.external_ports]
 
         if not self.external_ports or not https():
@@ -651,7 +652,7 @@
             self.configure_cert(cn)
 
         addresses = self.get_network_addresses()
-        for address, endpoint in set(addresses):
+        for address, endpoint in sorted(set(addresses)):
             for api_port in self.external_ports:
                 ext_port = determine_apache_port(api_port)
                 int_port = determine_api_port(api_port)
@@ -659,7 +660,7 @@
                 ctxt['endpoints'].append(portmap)
                 ctxt['ext_ports'].append(int(ext_port))
 
-        ctxt['ext_ports'] = list(set(ctxt['ext_ports']))
+        ctxt['ext_ports'] = sorted(list(set(ctxt['ext_ports'])))
         return ctxt
 
 
@@ -921,10 +922,10 @@
                         continue
 
                     sub_config = sub_config[self.config_file]
-                    for k, v in sub_config.iteritems():
+                    for k, v in six.iteritems(sub_config):
                         if k == 'sections':
-                            for section, config_dict in v.iteritems():
-                                log("Adding section '%s'" % (section),
+                            for section, config_dict in six.iteritems(v):
+                                log("adding section '%s'" % (section),
                                     level=DEBUG)
                                 ctxt[k][section] = config_dict
                         else:

=== modified file 'charmhelpers/contrib/openstack/neutron.py'
--- charmhelpers/contrib/openstack/neutron.py	2014-11-17 15:15:52 +0000
+++ charmhelpers/contrib/openstack/neutron.py	2014-11-25 15:09:14 +0000
@@ -14,7 +14,7 @@
 def headers_package():
     """Ensures correct linux-headers for running kernel are installed,
     for building DKMS package"""
-    kver = check_output(['uname', '-r']).strip()
+    kver = check_output(['uname', '-r']).decode('UTF-8').strip()
     return 'linux-headers-%s' % kver
 
 QUANTUM_CONF_DIR = '/etc/quantum'
@@ -22,7 +22,7 @@
 
 def kernel_version():
     """ Retrieve the current major kernel version as a tuple e.g. (3, 13) """
-    kver = check_output(['uname', '-r']).strip()
+    kver = check_output(['uname', '-r']).decode('UTF-8').strip()
     kver = kver.split('.')
     return (int(kver[0]), int(kver[1]))
 

=== modified file 'charmhelpers/contrib/openstack/templates/haproxy.cfg'
--- charmhelpers/contrib/openstack/templates/haproxy.cfg	2014-11-25 14:35:22 +0000
+++ charmhelpers/contrib/openstack/templates/haproxy.cfg	2014-11-25 15:09:14 +0000
@@ -35,7 +35,7 @@
     stats auth admin:password
 
 {% if frontends -%}
-{% for service, ports in service_ports.iteritems() -%}
+{% for service, ports in service_ports.items() -%}
 frontend tcp-in_{{ service }}
     bind *:{{ ports[0] }}
     bind :::{{ ports[0] }}
@@ -46,7 +46,7 @@
 {% for frontend in frontends -%}
 backend {{ service }}_{{ frontend }}
     balance leastconn
-    {% for unit, address in frontends[frontend]['backends'].iteritems() -%}
+    {% for unit, address in frontends[frontend]['backends'].items() -%}
     server {{ unit }} {{ address }}:{{ ports[1] }} check
     {% endfor %}
 {% endfor -%}

=== modified file 'charmhelpers/contrib/openstack/templating.py'
--- charmhelpers/contrib/openstack/templating.py	2014-11-25 14:35:22 +0000
+++ charmhelpers/contrib/openstack/templating.py	2014-11-25 15:09:14 +0000
@@ -1,13 +1,13 @@
 import os
 
+import six
+
 from charmhelpers.fetch import apt_install
-
 from charmhelpers.core.hookenv import (
     log,
     ERROR,
     INFO
 )
-
 from charmhelpers.contrib.openstack.utils import OPENSTACK_CODENAMES
 
 try:
@@ -43,7 +43,7 @@
         order by OpenStack release.
     """
     tmpl_dirs = [(rel, os.path.join(templates_dir, rel))
-                 for rel in OPENSTACK_CODENAMES.itervalues()]
+                 for rel in six.itervalues(OPENSTACK_CODENAMES)]
 
     if not os.path.isdir(templates_dir):
         log('Templates directory not found @ %s.' % templates_dir,
@@ -258,7 +258,7 @@
         """
         Write out all registered config files.
         """
-        [self.write(k) for k in self.templates.iterkeys()]
+        [self.write(k) for k in six.iterkeys(self.templates)]
 
     def set_release(self, openstack_release):
         """
@@ -275,5 +275,5 @@
         '''
         interfaces = []
         [interfaces.extend(i.complete_contexts())
-         for i in self.templates.itervalues()]
+         for i in six.itervalues(self.templates)]
         return interfaces

=== modified file 'charmhelpers/contrib/openstack/utils.py'
--- charmhelpers/contrib/openstack/utils.py	2014-11-25 14:35:22 +0000
+++ charmhelpers/contrib/openstack/utils.py	2014-11-25 15:09:14 +0000
@@ -10,6 +10,8 @@
 import socket
 import sys
 
+import six
+
 from charmhelpers.core.hookenv import (
     config,
     log as juju_log,
@@ -113,7 +115,7 @@
 
     # Best guess match based on deb string provided
     if src.startswith('deb') or src.startswith('ppa'):
-        for k, v in OPENSTACK_CODENAMES.iteritems():
+        for k, v in six.iteritems(OPENSTACK_CODENAMES):
             if v in src:
                 return v
 
@@ -134,7 +136,7 @@
 
 def get_os_version_codename(codename):
     '''Determine OpenStack version number from codename.'''
-    for k, v in OPENSTACK_CODENAMES.iteritems():
+    for k, v in six.iteritems(OPENSTACK_CODENAMES):
         if v == codename:
             return k
     e = 'Could not derive OpenStack version for '\
@@ -194,7 +196,7 @@
     else:
         vers_map = OPENSTACK_CODENAMES
 
-    for version, cname in vers_map.iteritems():
+    for version, cname in six.iteritems(vers_map):
         if cname == codename:
             return version
     # e = "Could not determine OpenStack version for package: %s" % pkg
@@ -318,7 +320,7 @@
         rc_script.write(
             "#!/bin/bash\n")
         [rc_script.write('export %s=%s\n' % (u, p))
-         for u, p in env_vars.iteritems() if u != "script_path"]
+         for u, p in six.iteritems(env_vars) if u != "script_path"]
 
 
 def openstack_upgrade_available(package):
@@ -418,7 +420,7 @@
 
     if isinstance(address, dns.name.Name):
         rtype = 'PTR'
-    elif isinstance(address, basestring):
+    elif isinstance(address, six.string_types):
         rtype = 'A'
     else:
         return None
@@ -486,8 +488,7 @@
               'hostname': json.dumps(hosts)}
 
     if relation_prefix:
-        keys = kwargs.keys()
-        for key in keys:
+        for key in list(kwargs.keys()):
             kwargs["%s_%s" % (relation_prefix, key)] = kwargs[key]
             del kwargs[key]
 

=== modified file 'charmhelpers/contrib/peerstorage/__init__.py'
--- charmhelpers/contrib/peerstorage/__init__.py	2014-11-25 14:35:22 +0000
+++ charmhelpers/contrib/peerstorage/__init__.py	2014-11-25 15:09:14 +0000
@@ -1,3 +1,4 @@
+import six
 from charmhelpers.core.hookenv import relation_id as current_relation_id
 from charmhelpers.core.hookenv import (
     is_relation_made,
@@ -93,7 +94,7 @@
             if ex in echo_data:
                 echo_data.pop(ex)
     else:
-        for attribute, value in rdata.iteritems():
+        for attribute, value in six.iteritems(rdata):
             for include in includes:
                 if include in attribute:
                     echo_data[attribute] = value
@@ -119,8 +120,8 @@
                  relation_settings=relation_settings,
                  **kwargs)
     if is_relation_made(peer_relation_name):
-        for key, value in dict(kwargs.items() +
-                               relation_settings.items()).iteritems():
+        for key, value in six.iteritems(dict(list(kwargs.items()) +
+                                             list(relation_settings.items()))):
             key_prefix = relation_id or current_relation_id()
             peer_store(key_prefix + delimiter + key,
                        value,

=== modified file 'charmhelpers/contrib/python/packages.py'
--- charmhelpers/contrib/python/packages.py	2014-11-25 14:35:22 +0000
+++ charmhelpers/contrib/python/packages.py	2014-11-25 15:09:14 +0000
@@ -15,7 +15,7 @@
 
 def parse_options(given, available):
     """Given a set of options, check if available"""
-    for key, value in given.items():
+    for key, value in sorted(given.items()):
         if key in available:
             yield "--{0}={1}".format(key, value)
 

=== modified file 'charmhelpers/contrib/ssl/__init__.py'
--- charmhelpers/contrib/ssl/__init__.py	2014-11-25 14:35:22 +0000
+++ charmhelpers/contrib/ssl/__init__.py	2014-11-25 15:09:14 +0000
@@ -74,5 +74,5 @@
         subprocess.check_call(cmd)
         return True
     except Exception as e:
-        print "Execution of openssl command failed:\n{}".format(e)
+        print("Execution of openssl command failed:\n{}".format(e))
         return False

=== modified file 'charmhelpers/contrib/storage/linux/ceph.py'
--- charmhelpers/contrib/storage/linux/ceph.py	2014-11-25 14:35:22 +0000
+++ charmhelpers/contrib/storage/linux/ceph.py	2014-11-25 15:09:14 +0000
@@ -65,7 +65,8 @@
 def rbd_exists(service, pool, rbd_img):
     """Check to see if a RADOS block device exists."""
     try:
-        out = check_output(['rbd', 'list', '--id', service, '--pool', pool])
+        out = check_output(['rbd', 'list', '--id',
+                            service, '--pool', pool]).decode('UTF-8')
     except CalledProcessError:
         return False
 
@@ -82,7 +83,8 @@
 def pool_exists(service, name):
     """Check to see if a RADOS pool already exists."""
     try:
-        out = check_output(['rados', '--id', service, 'lspools'])
+        out = check_output(['rados', '--id', service,
+                            'lspools']).decode('UTF-8')
     except CalledProcessError:
         return False
 
@@ -96,7 +98,8 @@
     version = ceph_version()
     if version and version >= '0.56':
         return json.loads(check_output(['ceph', '--id', service,
-                                        'osd', 'ls', '--format=json']))
+                                        'osd', 'ls',
+                                        '--format=json']).decode('UTF-8'))
 
     return None
 
@@ -112,7 +115,7 @@
     # on upstream recommended best practices.
     osds = get_osds(service)
     if osds:
-        pgnum = (len(osds) * 100 / replicas)
+        pgnum = (len(osds) * 100 // replicas)
     else:
         # NOTE(james-page): Default to 200 for older ceph versions
         # which don't support OSD query from cli
@@ -193,7 +196,7 @@
 def image_mapped(name):
     """Determine whether a RADOS block device is mapped locally."""
     try:
-        out = check_output(['rbd', 'showmapped'])
+        out = check_output(['rbd', 'showmapped']).decode('UTF-8')
     except CalledProcessError:
         return False
 
@@ -361,7 +364,7 @@
     """Retrieve the local version of ceph."""
     if os.path.exists('/usr/bin/ceph'):
         cmd = ['ceph', '-v']
-        output = check_output(cmd)
+        output = check_output(cmd).decode('US-ASCII')
         output = output.split()
         if len(output) > 3:
             return output[2]

=== modified file 'charmhelpers/contrib/storage/linux/loopback.py'
--- charmhelpers/contrib/storage/linux/loopback.py	2014-11-25 14:35:22 +0000
+++ charmhelpers/contrib/storage/linux/loopback.py	2014-11-25 15:09:14 +0000
@@ -1,12 +1,12 @@
-
 import os
 import re
-
 from subprocess import (
     check_call,
     check_output,
 )
 
+import six
+
 
 ##################################################
 # loopback device helpers.
@@ -37,7 +37,7 @@
     '''
     file_path = os.path.abspath(file_path)
     check_call(['losetup', '--find', file_path])
-    for d, f in loopback_devices().iteritems():
+    for d, f in six.iteritems(loopback_devices()):
         if f == file_path:
             return d
 
@@ -51,7 +51,7 @@
 
     :returns: str: Full path to the ensured loopback device (eg, /dev/loop0)
     '''
-    for d, f in loopback_devices().iteritems():
+    for d, f in six.iteritems(loopback_devices()):
         if f == path:
             return d
 

=== modified file 'charmhelpers/contrib/storage/linux/lvm.py'
--- charmhelpers/contrib/storage/linux/lvm.py	2014-05-10 19:58:31 +0000
+++ charmhelpers/contrib/storage/linux/lvm.py	2014-11-25 15:09:14 +0000
@@ -61,6 +61,7 @@
     vg = None
     pvd = check_output(['pvdisplay', block_device]).splitlines()
     for l in pvd:
+        l = l.decode('UTF-8')
         if l.strip().startswith('VG Name'):
             vg = ' '.join(l.strip().split()[2:])
     return vg

=== modified file 'charmhelpers/contrib/storage/linux/utils.py'
--- charmhelpers/contrib/storage/linux/utils.py	2014-07-31 08:17:42 +0000
+++ charmhelpers/contrib/storage/linux/utils.py	2014-11-25 15:09:14 +0000
@@ -30,7 +30,8 @@
     # sometimes sgdisk exits non-zero; this is OK, dd will clean up
     call(['sgdisk', '--zap-all', '--mbrtogpt',
           '--clear', block_device])
-    dev_end = check_output(['blockdev', '--getsz', block_device])
+    dev_end = check_output(['blockdev', '--getsz',
+                            block_device]).decode('UTF-8')
     gpt_end = int(dev_end.split()[0]) - 100
     check_call(['dd', 'if=/dev/zero', 'of=%s' % (block_device),
                 'bs=1M', 'count=1'])
@@ -47,7 +48,7 @@
         it doesn't.
     '''
     is_partition = bool(re.search(r".*[0-9]+\b", device))
-    out = check_output(['mount'])
+    out = check_output(['mount']).decode('UTF-8')
     if is_partition:
         return bool(re.search(device + r"\b", out))
     return bool(re.search(device + r"[0-9]+\b", out))

=== modified file 'charmhelpers/contrib/templating/contexts.py'
--- charmhelpers/contrib/templating/contexts.py	2014-11-25 14:35:22 +0000
+++ charmhelpers/contrib/templating/contexts.py	2014-11-25 15:09:14 +0000
@@ -6,6 +6,8 @@
 import os
 import yaml
 
+import six
+
 import charmhelpers.core.hookenv
 
 
@@ -92,9 +94,9 @@
 
     # Don't use non-standard tags for unicode which will not
     # work when salt uses yaml.load_safe.
-    yaml.add_representer(unicode, lambda dumper,
-                         value: dumper.represent_scalar(
-                             u'tag:yaml.org,2002:str', value))
+    yaml.add_representer(six.text_type,
+                         lambda dumper, value: dumper.represent_scalar(
+                             six.u('tag:yaml.org,2002:str'), value))
 
     yaml_dir = os.path.dirname(yaml_path)
     if not os.path.exists(yaml_dir):

=== modified file 'charmhelpers/core/fstab.py'
--- charmhelpers/core/fstab.py	2014-11-25 14:35:22 +0000
+++ charmhelpers/core/fstab.py	2014-11-25 15:09:14 +0000
@@ -3,10 +3,11 @@
 
 __author__ = 'Jorge Niedbalski R. <[e-mail]>'
 
+import io
 import os
 
 
-class Fstab(file):
+class Fstab(io.FileIO):
     """This class extends file in order to implement a file reader/writer
     for file `/etc/fstab`
     """
@@ -24,8 +25,8 @@
                 options = "defaults"
 
             self.options = options
-            self.d = d
-            self.p = p
+            self.d = int(d)
+            self.p = int(p)
 
         def __eq__(self, o):
             return str(self) == str(o)
@@ -45,7 +46,7 @@
             self._path = path
         else:
             self._path = self.DEFAULT_PATH
-        file.__init__(self, self._path, 'r+')
+        super(Fstab, self).__init__(self._path, 'rb+')
 
     def _hydrate_entry(self, line):
         # NOTE: use split with no arguments to split on any
@@ -58,8 +59,9 @@
     def entries(self):
         self.seek(0)
         for line in self.readlines():
+            line = line.decode('us-ascii')
             try:
-                if not line.startswith("#"):
+                if line.strip() and not line.startswith("#"):
                     yield self._hydrate_entry(line)
             except ValueError:
                 pass
@@ -75,14 +77,14 @@
         if self.get_entry_by_attr('device', entry.device):
             return False
 
-        self.write(str(entry) + '\n')
+        self.write((str(entry) + '\n').encode('us-ascii'))
         self.truncate()
         return entry
 
     def remove_entry(self, entry):
         self.seek(0)
 
-        lines = self.readlines()
+        lines = [l.decode('us-ascii') for l in self.readlines()]
 
         found = False
         for index, line in enumerate(lines):
@@ -97,7 +99,7 @@
         lines.remove(line)
 
         self.seek(0)
-        self.write(''.join(lines))
+        self.write(''.join(lines).encode('us-ascii'))
         self.truncate()
         return True
 

=== modified file 'charmhelpers/core/hookenv.py'
--- charmhelpers/core/hookenv.py	2014-11-25 14:35:22 +0000
+++ charmhelpers/core/hookenv.py	2014-11-25 15:09:14 +0000
@@ -9,9 +9,14 @@
 import yaml
 import subprocess
 import sys
-import UserDict
 from subprocess import CalledProcessError
 
+import six
+if not six.PY3:
+    from UserDict import UserDict
+else:
+    from collections import UserDict
+
 CRITICAL = "CRITICAL"
 ERROR = "ERROR"
 WARNING = "WARNING"
@@ -67,12 +72,12 @@
     subprocess.call(command)
 
 
-class Serializable(UserDict.IterableUserDict):
+class Serializable(UserDict):
     """Wrapper, an object that can be serialized to yaml or json"""
 
     def __init__(self, obj):
         # wrap the object
-        UserDict.IterableUserDict.__init__(self)
+        UserDict.__init__(self)
         self.data = obj
 
     def __getattr__(self, attr):
@@ -218,7 +223,7 @@
         prev_keys = []
         if self._prev_dict is not None:
             prev_keys = self._prev_dict.keys()
-        return list(set(prev_keys + dict.keys(self)))
+        return list(set(prev_keys + list(dict.keys(self))))
 
     def load_previous(self, path=None):
         """Load previous copy of config from disk.
@@ -269,7 +274,7 @@
 
         """
         if self._prev_dict:
-            for k, v in self._prev_dict.iteritems():
+            for k, v in six.iteritems(self._prev_dict):
                 if k not in self:
                     self[k] = v
         with open(self.path, 'w') as f:
@@ -284,7 +289,8 @@
         config_cmd_line.append(scope)
     config_cmd_line.append('--format=json')
     try:
-        config_data = json.loads(subprocess.check_output(config_cmd_line))
+        config_data = json.loads(
+            subprocess.check_output(config_cmd_line).decode('UTF-8'))
         if scope is not None:
             return config_data
         return Config(config_data)
@@ -303,10 +309,10 @@
     if unit:
         _args.append(unit)
     try:
-        return json.loads(subprocess.check_output(_args))
+        return json.loads(subprocess.check_output(_args).decode('UTF-8'))
     except ValueError:
         return None
-    except CalledProcessError, e:
+    except CalledProcessError as e:
         if e.returncode == 2:
             return None
         raise
@@ -318,7 +324,7 @@
     relation_cmd_line = ['relation-set']
     if relation_id is not None:
         relation_cmd_line.extend(('-r', relation_id))
-    for k, v in (relation_settings.items() + kwargs.items()):
+    for k, v in (list(relation_settings.items()) + list(kwargs.items())):
         if v is None:
             relation_cmd_line.append('{}='.format(k))
         else:
@@ -335,7 +341,8 @@
     relid_cmd_line = ['relation-ids', '--format=json']
     if reltype is not None:
         relid_cmd_line.append(reltype)
-        return json.loads(subprocess.check_output(relid_cmd_line)) or []
+        return json.loads(
+            subprocess.check_output(relid_cmd_line).decode('UTF-8')) or []
     return []
 
 
@@ -346,7 +353,8 @@
     units_cmd_line = ['relation-list', '--format=json']
     if relid is not None:
         units_cmd_line.extend(('-r', relid))
-    return json.loads(subprocess.check_output(units_cmd_line)) or []
+    return json.loads(
+        subprocess.check_output(units_cmd_line).decode('UTF-8')) or []
 
 
 @cached
@@ -455,7 +463,7 @@
     """Get the unit ID for the remote unit"""
     _args = ['unit-get', '--format=json', attribute]
     try:
-        return json.loads(subprocess.check_output(_args))
+        return json.loads(subprocess.check_output(_args).decode('UTF-8'))
     except ValueError:
         return None
 

=== modified file 'charmhelpers/core/host.py'
--- charmhelpers/core/host.py	2014-11-25 14:35:22 +0000
+++ charmhelpers/core/host.py	2014-11-25 15:09:14 +0000
@@ -14,11 +14,12 @@
 import subprocess
 import hashlib
 from contextlib import contextmanager
-
 from collections import OrderedDict
 
-from hookenv import log
-from fstab import Fstab
+import six
+
+from .hookenv import log
+from .fstab import Fstab
 
 
 def service_start(service_name):
@@ -54,7 +55,9 @@
 def service_running(service):
     """Determine whether a system service is running"""
     try:
-        output = subprocess.check_output(['service', service, 'status'], stderr=subprocess.STDOUT)
+        output = subprocess.check_output(
+            ['service', service, 'status'],
+            stderr=subprocess.STDOUT).decode('UTF-8')
     except subprocess.CalledProcessError:
         return False
     else:
@@ -67,7 +70,9 @@
 def service_available(service_name):
     """Determine whether a system service is available"""
     try:
-        subprocess.check_output(['service', service_name, 'status'], stderr=subprocess.STDOUT)
+        subprocess.check_output(
+            ['service', service_name, 'status'],
+            stderr=subprocess.STDOUT).decode('UTF-8')
     except subprocess.CalledProcessError as e:
         return 'unrecognized service' not in e.output
     else:
@@ -115,7 +120,7 @@
     cmd.append(from_path)
     cmd.append(to_path)
     log(" ".join(cmd))
-    return subprocess.check_output(cmd).strip()
+    return subprocess.check_output(cmd).decode('UTF-8').strip()
 
 
 def symlink(source, destination):
@@ -130,7 +135,7 @@
     subprocess.check_call(cmd)
 
 
-def mkdir(path, owner='root', group='root', perms=0555, force=False):
+def mkdir(path, owner='root', group='root', perms=0o555, force=False):
     """Create a directory"""
     log("Making dir {} {}:{} {:o}".format(path, owner, group,
                                           perms))
@@ -146,7 +151,7 @@
     os.chown(realpath, uid, gid)
 
 
-def write_file(path, content, owner='root', group='root', perms=0444):
+def write_file(path, content, owner='root', group='root', perms=0o444):
     """Create or overwrite a file with the contents of a string"""
     log("Writing file {} {}:{} {:o}".format(path, owner, group, perms))
     uid = pwd.getpwnam(owner).pw_uid
@@ -177,7 +182,7 @@
     cmd_args.extend([device, mountpoint])
     try:
         subprocess.check_output(cmd_args)
-    except subprocess.CalledProcessError, e:
+    except subprocess.CalledProcessError as e:
         log('Error mounting {} at {}\n{}'.format(device, mountpoint, e.output))
         return False
 
@@ -191,7 +196,7 @@
     cmd_args = ['umount', mountpoint]
     try:
         subprocess.check_output(cmd_args)
-    except subprocess.CalledProcessError, e:
+    except subprocess.CalledProcessError as e:
         log('Error unmounting {}\n{}'.format(mountpoint, e.output))
         return False
 
@@ -218,8 +223,8 @@
     """
     if os.path.exists(path):
         h = getattr(hashlib, hash_type)()
-        with open(path, 'r') as source:
-            h.update(source.read())  # IGNORE:E1101 - it does have update
+        with open(path, 'rb') as source:
+            h.update(source.read())
         return h.hexdigest()
     else:
         return None
@@ -297,7 +302,7 @@
     if length is None:
         length = random.choice(range(35, 45))
     alphanumeric_chars = [
-        l for l in (string.letters + string.digits)
+        l for l in (string.ascii_letters + string.digits)
         if l not in 'l0QD1vAEIOUaeiou']
     random_chars = [
         random.choice(alphanumeric_chars) for _ in range(length)]
@@ -306,14 +311,14 @@
 
 def list_nics(nic_type):
     '''Return a list of nics of given type(s)'''
-    if isinstance(nic_type, basestring):
+    if isinstance(nic_type, six.string_types):
         int_types = [nic_type]
     else:
         int_types = nic_type
     interfaces = []
     for int_type in int_types:
         cmd = ['ip', 'addr', 'show', 'label', int_type + '*']
-        ip_output = subprocess.check_output(cmd).split('\n')
+        ip_output = subprocess.check_output(cmd).decode('UTF-8').split('\n')
         ip_output = (line for line in ip_output if line)
         for line in ip_output:
             if line.split()[1].startswith(int_type):
@@ -335,7 +340,7 @@
 
 def get_nic_mtu(nic):
     cmd = ['ip', 'addr', 'show', nic]
-    ip_output = subprocess.check_output(cmd).split('\n')
+    ip_output = subprocess.check_output(cmd).decode('UTF-8').split('\n')
     mtu = ""
     for line in ip_output:
         words = line.split()
@@ -346,7 +351,7 @@
 
 def get_nic_hwaddr(nic):
     cmd = ['ip', '-o', '-0', 'addr', 'show', nic]
-    ip_output = subprocess.check_output(cmd)
+    ip_output = subprocess.check_output(cmd).decode('UTF-8')
     hwaddr = ""
     words = ip_output.split()
     if 'link/ether' in words:

=== modified file 'charmhelpers/core/services/helpers.py'
--- charmhelpers/core/services/helpers.py	2014-11-25 14:35:22 +0000
+++ charmhelpers/core/services/helpers.py	2014-11-25 15:09:14 +0000
@@ -196,7 +196,7 @@
         if not os.path.isabs(file_name):
             file_name = os.path.join(hookenv.charm_dir(), file_name)
         with open(file_name, 'w') as file_stream:
-            os.fchmod(file_stream.fileno(), 0600)
+            os.fchmod(file_stream.fileno(), 0o600)
             yaml.dump(config_data, file_stream)
 
     def read_context(self, file_name):
@@ -211,15 +211,19 @@
 
 class TemplateCallback(ManagerCallback):
     """
-    Callback class that will render a Jinja2 template, for use as a ready action.
-
-    :param str source: The template source file, relative to `$CHARM_DIR/templates`
+    Callback class that will render a Jinja2 template, for use as a ready
+    action.
+
+    :param str source: The template source file, relative to
+    `$CHARM_DIR/templates`
+
     :param str target: The target to write the rendered template to
     :param str owner: The owner of the rendered file
     :param str group: The group of the rendered file
     :param int perms: The permissions of the rendered file
     """
-    def __init__(self, source, target, owner='root', group='root', perms=0444):
+    def __init__(self, source, target,
+                 owner='root', group='root', perms=0o444):
         self.source = source
         self.target = target
         self.owner = owner

=== modified file 'charmhelpers/core/templating.py'
--- charmhelpers/core/templating.py	2014-11-25 14:35:22 +0000
+++ charmhelpers/core/templating.py	2014-11-25 15:09:14 +0000
@@ -4,7 +4,8 @@
 from charmhelpers.core import hookenv
 
 
-def render(source, target, context, owner='root', group='root', perms=0444, templates_dir=None):
+def render(source, target, context, owner='root', group='root',
+           perms=0o444, templates_dir=None):
     """
     Render a template.
 

=== modified file 'charmhelpers/fetch/__init__.py'
--- charmhelpers/fetch/__init__.py	2014-11-25 14:35:22 +0000
+++ charmhelpers/fetch/__init__.py	2014-11-25 15:09:14 +0000
@@ -5,10 +5,6 @@
 from charmhelpers.core.host import (
     lsb_release
 )
-from urlparse import (
-    urlparse,
-    urlunparse,
-)
 import subprocess
 from charmhelpers.core.hookenv import (
     config,
@@ -16,6 +12,12 @@
 )
 import os
 
+import six
+if six.PY3:
+    from urllib.parse import urlparse, urlunparse
+else:
+    from urlparse import urlparse, urlunparse
+
 
 CLOUD_ARCHIVE = """# Ubuntu Cloud Archive
 deb http://ubuntu-cloud.archive.canonical.com/ubuntu {} main
@@ -149,7 +151,7 @@
     cmd = ['apt-get', '--assume-yes']
     cmd.extend(options)
     cmd.append('install')
-    if isinstance(packages, basestring):
+    if isinstance(packages, six.string_types):
         cmd.append(packages)
     else:
         cmd.extend(packages)
@@ -182,7 +184,7 @@
 def apt_purge(packages, fatal=False):
     """Purge one or more packages"""
     cmd = ['apt-get', '--assume-yes', 'purge']
-    if isinstance(packages, basestring):
+    if isinstance(packages, six.string_types):
         cmd.append(packages)
     else:
         cmd.extend(packages)
@@ -193,7 +195,7 @@
 def apt_hold(packages, fatal=False):
     """Hold one or more packages"""
     cmd = ['apt-mark', 'hold']
-    if isinstance(packages, basestring):
+    if isinstance(packages, six.string_types):
         cmd.append(packages)
     else:
         cmd.extend(packages)
@@ -260,7 +262,7 @@
 
     if key:
         if '-----BEGIN PGP PUBLIC KEY BLOCK-----' in key:
-            with NamedTemporaryFile() as key_file:
+            with NamedTemporaryFile('w+') as key_file:
                 key_file.write(key)
                 key_file.flush()
                 key_file.seek(0)
@@ -297,14 +299,14 @@
     sources = safe_load((config(sources_var) or '').strip()) or []
     keys = safe_load((config(keys_var) or '').strip()) or None
 
-    if isinstance(sources, basestring):
+    if isinstance(sources, six.string_types):
         sources = [sources]
 
     if keys is None:
         for source in sources:
             add_source(source, None)
     else:
-        if isinstance(keys, basestring):
+        if isinstance(keys, six.string_types):
             keys = [keys]
 
         if len(sources) != len(keys):
@@ -401,7 +403,7 @@
         while result is None or result == APT_NO_LOCK:
             try:
                 result = subprocess.check_call(cmd, env=env)
-            except subprocess.CalledProcessError, e:
+            except subprocess.CalledProcessError as e:
                 retry_count = retry_count + 1
                 if retry_count > APT_NO_LOCK_RETRY_COUNT:
                     raise

=== modified file 'charmhelpers/fetch/archiveurl.py'
--- charmhelpers/fetch/archiveurl.py	2014-11-25 14:35:22 +0000
+++ charmhelpers/fetch/archiveurl.py	2014-11-25 15:09:14 +0000
@@ -1,8 +1,23 @@
 import os
-import urllib2
-from urllib import urlretrieve
-import urlparse
 import hashlib
+import re
+
+import six
+if six.PY3:
+    from urllib.request import (
+        build_opener, install_opener, urlopen, urlretrieve,
+        HTTPPasswordMgrWithDefaultRealm, HTTPBasicAuthHandler,
+    )
+    from urllib.parse import urlparse, urlunparse, parse_qs
+    from urllib.error import URLError
+else:
+    from urllib import urlretrieve
+    from urllib2 import (
+        build_opener, install_opener, urlopen,
+        HTTPPasswordMgrWithDefaultRealm, HTTPBasicAuthHandler,
+        URLError
+    )
+    from urlparse import urlparse, urlunparse, parse_qs
 
 from charmhelpers.fetch import (
     BaseFetchHandler,
@@ -15,6 +30,24 @@
 from charmhelpers.core.host import mkdir, check_hash
 
 
+def splituser(host):
+    '''urllib.splituser(), but six's support of this seems broken'''
+    _userprog = re.compile('^(.*)@(.*)$')
+    match = _userprog.match(host)
+    if match:
+        return match.group(1, 2)
+    return None, host
+
+
+def splitpasswd(user):
+    '''urllib.splitpasswd(), but six's support of this is missing'''
+    _passwdprog = re.compile('^([^:]*):(.*)$', re.S)
+    match = _passwdprog.match(user)
+    if match:
+        return match.group(1, 2)
+    return user, None
+
+
 class ArchiveUrlFetchHandler(BaseFetchHandler):
     """
     Handler to download archive files from arbitrary URLs.
@@ -42,20 +75,20 @@
         """
         # propogate all exceptions
         # URLError, OSError, etc
-        proto, netloc, path, params, query, fragment = urlparse.urlparse(source)
+        proto, netloc, path, params, query, fragment = urlparse(source)
         if proto in ('http', 'https'):
-            auth, barehost = urllib2.splituser(netloc)
+            auth, barehost = splituser(netloc)
             if auth is not None:
-                source = urlparse.urlunparse((proto, barehost, path, params, query, fragment))
-                username, password = urllib2.splitpasswd(auth)
-                passman = urllib2.HTTPPasswordMgrWithDefaultRealm()
+                source = urlunparse((proto, barehost, path, params, query, fragment))
+                username, password = splitpasswd(auth)
+                passman = HTTPPasswordMgrWithDefaultRealm()
                 # Realm is set to None in add_password to force the username and password
                 # to be used whatever the realm
                 passman.add_password(None, source, username, password)
-                authhandler = urllib2.HTTPBasicAuthHandler(passman)
-                opener = urllib2.build_opener(authhandler)
-                urllib2.install_opener(opener)
-        response = urllib2.urlopen(source)
+                authhandler = HTTPBasicAuthHandler(passman)
+                opener = build_opener(authhandler)
+                install_opener(opener)
+        response = urlopen(source)
         try:
             with open(dest, 'w') as dest_file:
                 dest_file.write(response.read())
@@ -91,17 +124,21 @@
         url_parts = self.parse_url(source)
         dest_dir = os.path.join(os.environ.get('CHARM_DIR'), 'fetched')
         if not os.path.exists(dest_dir):
-            mkdir(dest_dir, perms=0755)
+            mkdir(dest_dir, perms=0o755)
         dld_file = os.path.join(dest_dir, os.path.basename(url_parts.path))
         try:
             self.download(source, dld_file)
-        except urllib2.URLError as e:
+        except URLError as e:
             raise UnhandledSource(e.reason)
         except OSError as e:
             raise UnhandledSource(e.strerror)
-        options = urlparse.parse_qs(url_parts.fragment)
+        options = parse_qs(url_parts.fragment)
         for key, value in options.items():
-            if key in hashlib.algorithms:
+            if not six.PY3:
+                algorithms = hashlib.algorithms
+            else:
+                algorithms = hashlib.algorithms_available
+            if key in algorithms:
                 check_hash(dld_file, value, key)
         if checksum:
             check_hash(dld_file, checksum, hash_type)

=== modified file 'charmhelpers/fetch/bzrurl.py'
--- charmhelpers/fetch/bzrurl.py	2014-11-25 14:35:22 +0000
+++ charmhelpers/fetch/bzrurl.py	2014-11-25 15:09:14 +0000
@@ -5,6 +5,10 @@
 )
 from charmhelpers.core.host import mkdir
 
+import six
+if six.PY3:
+    raise ImportError('bzrlib does not support Python3')
+
 try:
     from bzrlib.branch import Branch
 except ImportError:
@@ -42,7 +46,7 @@
         dest_dir = os.path.join(os.environ.get('CHARM_DIR'), "fetched",
                                 branch_name)
         if not os.path.exists(dest_dir):
-            mkdir(dest_dir, perms=0755)
+            mkdir(dest_dir, perms=0o755)
         try:
             self.branch(source, dest_dir)
         except OSError as e:

=== modified file 'charmhelpers/fetch/giturl.py'
--- charmhelpers/fetch/giturl.py	2014-11-25 14:35:22 +0000
+++ charmhelpers/fetch/giturl.py	2014-11-25 15:09:14 +0000
@@ -5,6 +5,10 @@
 )
 from charmhelpers.core.host import mkdir
 
+import six
+if six.PY3:
+    raise ImportError('GitPython does not support Python 3')
+
 try:
     from git import Repo
 except ImportError:
@@ -17,7 +21,7 @@
     """Handler for git branches via generic and github URLs"""
     def can_handle(self, source):
         url_parts = self.parse_url(source)
-        #TODO (mattyw) no support for ssh git@ yet
+        # TODO (mattyw) no support for ssh git@ yet
         if url_parts.scheme not in ('http', 'https', 'git'):
             return False
         else:
@@ -36,7 +40,7 @@
         dest_dir = os.path.join(os.environ.get('CHARM_DIR'), "fetched",
                                 branch_name)
         if not os.path.exists(dest_dir):
-            mkdir(dest_dir, perms=0755)
+            mkdir(dest_dir, perms=0o755)
         try:
             self.clone(source, dest_dir, branch)
         except OSError as e:

=== modified file 'test_requirements.txt'
--- test_requirements.txt	2014-11-25 14:35:22 +0000
+++ test_requirements.txt	2014-11-25 15:09:14 +0000
@@ -1,18 +1,17 @@
-coverage==3.6
-launchpadlib==1.10.2
---allow-external launchpadlib
---allow-unverified launchpadlib
-mock==1.0.1
-netaddr
-nose==1.3.1
-PyYAML==3.10
-simplejson==3.3.0
-testtools
-Tempita==0.5.1
-bzr+http://bazaar.launchpad.net/~yellow/python-shelltoolbox/trunk@17#egg=shelltoolbox
-http://alastairs-place.net/projects/netifaces/netifaces-0.6.tar.gz
-bzr==2.6.0
-GitPython>=0.3.2.RC1
-Jinja2==2.7.2
---allow-external lazr.authentication
---allow-unverified lazr.authentication
+# Test-only dependencies are unpinned.
+#
+pip
+distribute
+coverage>=3.6
+mock>=1.0.1
+nose>=1.3.1
+flake8
+testtools==0.9.14  # Before dependent on modern 'six'
+#
+# Specify precise versions of runtime dependencies where possible.
+netaddr==0.7.10    # trusty. precise is 0.7.5, but not in pypi.
+PyYAML==3.10       # precise
+Tempita==0.5.1     # precise
+netifaces==0.10    # trusty is 0.8, but using py3 compatible version for tests.
+Jinja2==2.6        # precise
+six==1.1           # precise

=== modified file 'tests/cli/test_cmdline.py'
--- tests/cli/test_cmdline.py	2014-11-25 14:35:22 +0000
+++ tests/cli/test_cmdline.py	2014-11-25 15:09:14 +0000
@@ -6,15 +6,13 @@
     patch,
     MagicMock,
 )
-try:
-    from cStringIO import StringIO
-except ImportError:
-    from StringIO import StringIO
 import json
 from pprint import pformat
 import yaml
 import csv
 
+from six import StringIO
+
 from charmhelpers import cli
 
 
@@ -116,7 +114,8 @@
         self.output_data = {"this": "is", "some": 1, "data": dict()}
 
     def test_supports_formats(self):
-        self.assertItemsEqual(self.expected_formats, self.of.supported_formats)
+        self.assertEqual(sorted(self.expected_formats),
+                         sorted(self.of.supported_formats))
 
     def test_adds_arguments(self):
         ap = MagicMock()
@@ -130,11 +129,12 @@
 
         for call_args in add_arg.call_args_list:
             if "--format" in call_args[0]:
-                self.assertItemsEqual(call_args[1]['choices'], self.expected_formats)
+                self.assertEqual(sorted(call_args[1]['choices']),
+                                 sorted(self.expected_formats))
                 self.assertEqual(call_args[1]['default'], 'raw')
                 break
         else:
-            print arg_group.call_args_list
+            print(arg_group.call_args_list)
             self.fail("No --format argument was created")
 
         all_args = [c[0][0] for c in add_arg.call_args_list]

=== modified file 'tests/cli/test_function_signature_analysis.py'
--- tests/cli/test_function_signature_analysis.py	2013-08-21 21:40:07 +0000
+++ tests/cli/test_function_signature_analysis.py	2014-11-25 15:09:14 +0000
@@ -1,7 +1,7 @@
 """Tests for the commandant code that analyzes a function signature to
 determine the parameters to argparse."""
 
-from testtools import TestCase, matchers
+from testtools import TestCase
 
 from charmhelpers import cli
 
@@ -42,5 +42,5 @@
         args = cli.describe_arguments(lambda x, y=3, *z, **missing: False)
         for opts, _ in args:
             # opts should be ('varname',) at this point
-            self.assertThat(opts, matchers.HasLength(1))
+            self.assertTrue(len(opts) == 1)
             self.assertNotIn('missing', opts)

=== modified file 'tests/contrib/charmhelpers/test_charmhelpers.py'
--- tests/contrib/charmhelpers/test_charmhelpers.py	2014-11-25 14:35:22 +0000
+++ tests/contrib/charmhelpers/test_charmhelpers.py	2014-11-25 15:09:14 +0000
@@ -2,10 +2,10 @@
 
 import unittest
 import yaml
-
-from StringIO import StringIO
 from testtools import TestCase
 
+from six import StringIO
+
 import sys
 # Path hack to ensure we test the local code, not a version installed in
 # /usr/local/lib.  This is necessary since /usr/local/lib is prepended before
@@ -265,11 +265,11 @@
         # wait_for_page_contents() will wait until a given string is
         # contained within the results of a given url and will return
         # once it does.
-        # We need to patch the charmhelpers instance of urllib2 so that
+        # We need to patch the charmhelpers instance of urlopen so that
         # it doesn't try to connect out.
         test_content = "Hello, world."
         new_urlopen = lambda *args: StringIO(test_content)
-        self.patch(charmhelpers.urllib2, 'urlopen', new_urlopen)
+        self.patch(charmhelpers, 'urlopen', new_urlopen)
         charmhelpers.wait_for_page_contents(
             'http://example.com', test_content, timeout=0)
 
@@ -277,10 +277,10 @@
         # If the desired contents do not appear within the page before
         # the specified timeout, wait_for_page_contents() will raise a
         # RuntimeError.
-        # We need to patch the charmhelpers instance of urllib2 so that
+        # We need to patch the charmhelpers instance of urlopen so that
         # it doesn't try to connect out.
         new_urlopen = lambda *args: StringIO("This won't work.")
-        self.patch(charmhelpers.urllib2, 'urlopen', new_urlopen)
+        self.patch(charmhelpers, 'urlopen', new_urlopen)
         self.assertRaises(
             RuntimeError, charmhelpers.wait_for_page_contents,
             'http://example.com', "This will error", timeout=0)

=== modified file 'tests/contrib/hahelpers/test_cluster_utils.py'
--- tests/contrib/hahelpers/test_cluster_utils.py	2014-09-24 09:42:52 +0000
+++ tests/contrib/hahelpers/test_cluster_utils.py	2014-11-25 15:09:14 +0000
@@ -44,7 +44,7 @@
     def test_is_crm_leader(self, check_output):
         '''It determines its unit is leader'''
         self.get_unit_hostname.return_value = 'node1'
-        crm = 'resource vip is running on: node1'
+        crm = b'resource vip is running on: node1'
         check_output.return_value = crm
         self.assertTrue(cluster_utils.is_crm_leader('vip'))
 
@@ -52,7 +52,7 @@
     def test_is_not_leader(self, check_output):
         '''It determines its unit is not leader'''
         self.get_unit_hostname.return_value = 'node1'
-        crm = 'resource vip is running on: node2'
+        crm = b'resource vip is running on: node2'
         check_output.return_value = crm
         self.assertFalse(cluster_utils.is_crm_leader('some_resource'))
 

=== modified file 'tests/contrib/network/test_ip.py'
--- tests/contrib/network/test_ip.py	2014-11-25 14:35:22 +0000
+++ tests/contrib/network/test_ip.py	2014-11-25 15:09:14 +0000
@@ -59,7 +59,7 @@
     },
 }
 
-IP_OUTPUT = """link/ether fa:16:3e:2a:cc:ce brd ff:ff:ff:ff:ff:ff
+IP_OUTPUT = b"""link/ether fa:16:3e:2a:cc:ce brd ff:ff:ff:ff:ff:ff
     inet 10.5.16.93/16 brd 10.5.255.255 scope global eth0
        valid_lft forever preferred_lft forever
     inet6 2001:db8:1:0:d0cf:528c:23eb:6000/64 scope global
@@ -72,7 +72,7 @@
        valid_lft forever preferred_lft forever
 """
 
-IP_OUTPUT_NO_VALID = """link/ether fa:16:3e:2a:cc:ce brd ff:ff:ff:ff:ff:ff
+IP_OUTPUT_NO_VALID = b"""link/ether fa:16:3e:2a:cc:ce brd ff:ff:ff:ff:ff:ff
     inet 10.5.16.93/16 brd 10.5.255.255 scope global eth0
        valid_lft forever preferred_lft forever
     inet6 2001:db8:1:0:2918:3444:852:5b8a/64 scope global temporary dynamic
@@ -100,13 +100,14 @@
             return DUMMY_ADDRESSES[iface]
 
         with mock.patch.object(netifaces, 'interfaces') as interfaces:
-            interfaces.return_value = DUMMY_ADDRESSES.keys()
+            interfaces.return_value = sorted(DUMMY_ADDRESSES.keys())
             with mock.patch.object(netifaces, 'ifaddresses') as ifaddresses:
                 ifaddresses.side_effect = side_effect
                 if not fatal:
                     self.assertEqual(expect_ip_addr,
-                                     net_ip.get_address_in_network(
-                                         network, fallback, fatal))
+                                     net_ip.get_address_in_network(network,
+                                                                   fallback,
+                                                                   fatal))
                 else:
                     net_ip.get_address_in_network(network, fallback, fatal)
 
@@ -122,7 +123,7 @@
                           None, None, fatal=True)
 
     def test_get_address_in_network_ipv4(self):
-        self._test_get_address_in_network('192.168.1.56', '192.168.1.0/24')
+        self._test_get_address_in_network('192.168.1.55', '192.168.1.0/24')
 
     def test_get_address_in_network_ipv6(self):
         self._test_get_address_in_network('2a01:348:2f4:0:685e:5748:ae62:209f',
@@ -260,7 +261,7 @@
                            mock_get_iface_from_addr):
         mock_get_iface_from_addr.return_value = 'eth0'
         mock_check_out.return_value = \
-            "inet6 2a01:348:2f4:0:685e:5748:ae62:209f/64 scope global dynamic"
+            b"inet6 2a01:348:2f4:0:685e:5748:ae62:209f/64 scope global dynamic"
         _interfaces.return_value = DUMMY_ADDRESSES.keys()
         _ifaddresses.side_effect = DUMMY_ADDRESSES.__getitem__
         result = net_ip.get_ipv6_addr(dynamic_only=False)
@@ -275,7 +276,7 @@
                                           mock_get_iface_from_addr):
         mock_get_iface_from_addr.return_value = 'eth0'
         mock_check_out.return_value = \
-            "inet6 2a01:348:2f4:0:685e:5748:ae62:209f/64 scope global dynamic"
+            b"inet6 2a01:348:2f4:0:685e:5748:ae62:209f/64 scope global dynamic"
         _interfaces.return_value = DUMMY_ADDRESSES.keys()
         _ifaddresses.side_effect = DUMMY_ADDRESSES.__getitem__
         result = net_ip.get_ipv6_addr(dynamic_only=False)
@@ -494,9 +495,9 @@
     def test_get_iface_from_addr(self, mock_log, mock_ifaddresses,
                                  mock_interfaces):
         mock_ifaddresses.side_effect = lambda iface: DUMMY_ADDRESSES[iface]
-        mock_interfaces.return_value = DUMMY_ADDRESSES.keys()
+        mock_interfaces.return_value = sorted(DUMMY_ADDRESSES.keys())
         addr = 'fe80::3e97:eff:fe8b:1cf7'
-        self.assertEqual(net_ip.get_iface_from_addr(addr), 'eth1')
+        self.assertEqual(net_ip.get_iface_from_addr(addr), 'eth0')
 
         with nose.tools.assert_raises(Exception):
             net_ip.get_iface_from_addr('1.2.3.4')

=== modified file 'tests/contrib/openstack/test_neutron_utils.py'
--- tests/contrib/openstack/test_neutron_utils.py	2014-11-07 09:54:33 +0000
+++ tests/contrib/openstack/test_neutron_utils.py	2014-11-25 15:09:14 +0000
@@ -23,12 +23,12 @@
         return mock
 
     def test_headers_package(self):
-        self.check_output.return_value = '3.13.0-19-generic'
+        self.check_output.return_value = b'3.13.0-19-generic'
         kname = neutron.headers_package()
         self.assertEquals(kname, 'linux-headers-3.13.0-19-generic')
 
     def test_kernel_version(self):
-        self.check_output.return_value = '3.13.0-19-generic'
+        self.check_output.return_value = b'3.13.0-19-generic'
         kver_maj, kver_min = neutron.kernel_version()
         self.assertEquals((kver_maj, kver_min), (3, 13))
 

=== modified file 'tests/contrib/openstack/test_openstack_utils.py'
--- tests/contrib/openstack/test_openstack_utils.py	2014-11-25 14:35:22 +0000
+++ tests/contrib/openstack/test_openstack_utils.py	2014-11-25 15:09:14 +0000
@@ -1,13 +1,22 @@
+import io
 import os
 import subprocess
 import unittest
-
 from copy import copy
 from testtools import TestCase
 from mock import MagicMock, patch, call
 
 import charmhelpers.contrib.openstack.utils as openstack
 
+import six
+
+if not six.PY3:
+    builtin_open = '__builtin__.open'
+    builtin_import = '__builtin__.__import__'
+else:
+    builtin_open = 'builtins.open'
+    builtin_import = 'builtins.__import__'
+
 # mocked return of openstack.lsb_release()
 FAKE_RELEASE = {
     'DISTRIB_CODENAME': 'precise',
@@ -222,7 +231,7 @@
         '''Test deriving OpenStack codename from an installed package'''
         with patch('apt_pkg.Cache') as cache:
             cache.return_value = self._apt_cache()
-            for pkg, vers in FAKE_REPO.iteritems():
+            for pkg, vers in six.iteritems(FAKE_REPO):
                 # test fake repo for all "installed" packages
                 if pkg.startswith('bad-'):
                     continue
@@ -291,7 +300,7 @@
         '''Test deriving OpenStack version from an installed package'''
         with patch('apt_pkg.Cache') as cache:
             cache.return_value = self._apt_cache()
-            for pkg, vers in FAKE_REPO.iteritems():
+            for pkg, vers in six.iteritems(FAKE_REPO):
                 if pkg.startswith('bad-'):
                     continue
                 if 'pkg_vers' not in vers:
@@ -354,12 +363,12 @@
             ex_cmd = ['add-apt-repository', '-y', 'ppa:gandelman-a/openstack']
             mock.assert_called_with(ex_cmd)
 
-    @patch('__builtin__.open')
+    @patch(builtin_open)
     @patch('charmhelpers.contrib.openstack.utils.juju_log')
     @patch('charmhelpers.contrib.openstack.utils.import_key')
     def test_configure_install_source_deb_url(self, _import, _log, _open):
         '''Test configuring installation source from deb repo url'''
-        _file = MagicMock(spec=file)
+        _file = MagicMock(spec=io.FileIO)
         _open.return_value = _file
         src = ('deb http://ubuntu-cloud.archive.canonical.com/ubuntu '
                'precise-havana main|KEYID')
@@ -372,12 +381,12 @@
         _file.__enter__().write.assert_called_with(src)
 
     @patch('charmhelpers.contrib.openstack.utils.lsb_release')
-    @patch('__builtin__.open')
+    @patch(builtin_open)
     @patch('charmhelpers.contrib.openstack.utils.juju_log')
     def test_configure_install_source_distro_proposed(self, _log, _open, _lsb):
         '''Test configuring installation source from deb repo url'''
         _lsb.return_value = FAKE_RELEASE
-        _file = MagicMock(spec=file)
+        _file = MagicMock(spec=io.FileIO)
         _open.return_value = _file
         openstack.configure_installation_source('distro-proposed')
         src = ('deb http://archive.ubuntu.com/ubuntu/ precise-proposed '
@@ -402,13 +411,13 @@
                    'ppa:ubuntu-cloud-archive/folsom-staging']
             _subp.assert_called_with(cmd)
 
-    @patch('__builtin__.open')
+    @patch(builtin_open)
     @patch('charmhelpers.contrib.openstack.utils.apt_install')
     @patch('charmhelpers.contrib.openstack.utils.lsb_release')
     def test_configure_install_source_uca_repos(self, _lsb, _install, _open):
         '''Test configuring installation source from UCA sources'''
         _lsb.return_value = FAKE_RELEASE
-        _file = MagicMock(spec=file)
+        _file = MagicMock(spec=io.FileIO)
         _open.return_value = _file
         for src, url in UCA_SOURCES:
             openstack.configure_installation_source(src)
@@ -455,13 +464,13 @@
     @patch('os.mkdir')
     @patch('os.path.exists')
     @patch('charmhelpers.contrib.openstack.utils.charm_dir')
-    @patch('__builtin__.open')
+    @patch(builtin_open)
     def test_save_scriptrc(self, _open, _charm_dir, _exists, _mkdir):
         '''Test generation of scriptrc from environment'''
         scriptrc = ['#!/bin/bash\n',
                     'export setting1=foo\n',
                     'export setting2=bar\n']
-        _file = MagicMock(spec=file)
+        _file = MagicMock(spec=io.FileIO)
         _open.return_value = _file
         _charm_dir.return_value = '/var/lib/juju/units/testing-foo-0/charm'
         _exists.return_value = False
@@ -583,21 +592,21 @@
     @patch.object(openstack, 'apt_install')
     def test_get_host_ip_with_hostname(self, apt_install):
         fake_dns = FakeDNS('10.0.0.1')
-        with patch('__builtin__.__import__', side_effect=[fake_dns]):
+        with patch(builtin_import, side_effect=[fake_dns]):
             ip = openstack.get_host_ip('www.ubuntu.com')
         self.assertEquals(ip, '10.0.0.1')
 
     @patch.object(openstack, 'apt_install')
     def test_get_host_ip_with_ip(self, apt_install):
         fake_dns = FakeDNS('5.5.5.5')
-        with patch('__builtin__.__import__', side_effect=[fake_dns]):
+        with patch(builtin_import, side_effect=[fake_dns]):
             ip = openstack.get_host_ip('4.2.2.1')
         self.assertEquals(ip, '4.2.2.1')
 
     @patch.object(openstack, 'apt_install')
     def test_ns_query_trigger_apt_install(self, apt_install):
         fake_dns = FakeDNS('5.5.5.5')
-        with patch('__builtin__.__import__', side_effect=[ImportError, fake_dns]):
+        with patch(builtin_import, side_effect=[ImportError, fake_dns]):
             nsq = openstack.ns_query('5.5.5.5')
             apt_install.assert_called_with('python-dnspython')
         self.assertEquals(nsq, '5.5.5.5')
@@ -605,7 +614,7 @@
     @patch.object(openstack, 'apt_install')
     def test_ns_query_ptr_record(self, apt_install):
         fake_dns = FakeDNS('127.0.0.1')
-        with patch('__builtin__.__import__', side_effect=[fake_dns]):
+        with patch(builtin_import, side_effect=[fake_dns]):
             nsq = openstack.ns_query('127.0.0.1')
         self.assertEquals(nsq, '127.0.0.1')
 
@@ -613,35 +622,35 @@
     def test_ns_query_a_record(self, apt_install):
         fake_dns = FakeDNS('127.0.0.1')
         fake_dns_name = FakeDNSName('www.somedomain.tld')
-        with patch('__builtin__.__import__', side_effect=[fake_dns]):
+        with patch(builtin_import, side_effect=[fake_dns]):
             nsq = openstack.ns_query(fake_dns_name)
         self.assertEquals(nsq, '127.0.0.1')
 
     @patch.object(openstack, 'apt_install')
     def test_ns_query_blank_record(self, apt_install):
         fake_dns = FakeDNS(None)
-        with patch('__builtin__.__import__', side_effect=[fake_dns, fake_dns]):
+        with patch(builtin_import, side_effect=[fake_dns, fake_dns]):
             nsq = openstack.ns_query(None)
         self.assertEquals(nsq, None)
 
     @patch.object(openstack, 'apt_install')
     def test_ns_query_lookup_fail(self, apt_install):
         fake_dns = FakeDNS('')
-        with patch('__builtin__.__import__', side_effect=[fake_dns, fake_dns]):
+        with patch(builtin_import, side_effect=[fake_dns, fake_dns]):
             nsq = openstack.ns_query('nonexistant')
         self.assertEquals(nsq, None)
 
     @patch.object(openstack, 'apt_install')
     def test_get_hostname_with_ip(self, apt_install):
         fake_dns = FakeDNS('www.ubuntu.com')
-        with patch('__builtin__.__import__', side_effect=[fake_dns, fake_dns]):
+        with patch(builtin_import, side_effect=[fake_dns, fake_dns]):
             hn = openstack.get_hostname('4.2.2.1')
         self.assertEquals(hn, 'www.ubuntu.com')
 
     @patch.object(openstack, 'apt_install')
     def test_get_hostname_with_ip_not_fqdn(self, apt_install):
         fake_dns = FakeDNS('packages.ubuntu.com')
-        with patch('__builtin__.__import__', side_effect=[fake_dns, fake_dns]):
+        with patch(builtin_import, side_effect=[fake_dns, fake_dns]):
             hn = openstack.get_hostname('4.2.2.1', fqdn=False)
         self.assertEquals(hn, 'packages')
 
@@ -663,7 +672,7 @@
     @patch.object(openstack, 'apt_install')
     def test_get_hostname_trigger_apt_install(self, apt_install):
         fake_dns = FakeDNS('www.ubuntu.com')
-        with patch('__builtin__.__import__', side_effect=[ImportError, fake_dns, fake_dns]):
+        with patch(builtin_import, side_effect=[ImportError, fake_dns, fake_dns]):
             hn = openstack.get_hostname('4.2.2.1')
             apt_install.assert_called_with('python-dnspython')
         self.assertEquals(hn, 'www.ubuntu.com')
@@ -673,12 +682,12 @@
     def test_get_hostname_lookup_fail(self, apt_install, ns_query):
         fake_dns = FakeDNS('www.ubuntu.com')
         ns_query.return_value = []
-        with patch('__builtin__.__import__', side_effect=[fake_dns, fake_dns]):
+        with patch(builtin_import, side_effect=[fake_dns, fake_dns]):
             hn = openstack.get_hostname('4.2.2.1')
         self.assertEquals(hn, None)
 
     @patch('os.path.isfile')
-    @patch('__builtin__.open')
+    @patch(builtin_open)
     def test_get_matchmaker_map(self, _open, _isfile):
         _isfile.return_value = True
         mm_data = """
@@ -696,7 +705,7 @@
         )
 
     @patch('os.path.isfile')
-    @patch('__builtin__.open')
+    @patch(builtin_open)
     def test_get_matchmaker_map_nofile(self, _open, _isfile):
         _isfile.return_value = False
         self.assertEqual(

=== modified file 'tests/contrib/openstack/test_os_contexts.py'
--- tests/contrib/openstack/test_os_contexts.py	2014-11-25 14:35:22 +0000
+++ tests/contrib/openstack/test_os_contexts.py	2014-11-25 15:09:14 +0000
@@ -11,6 +11,13 @@
 )
 from tests.helpers import patch_open
 
+import six
+
+if not six.PY3:
+    open_builtin = '__builtin__.open'
+else:
+    open_builtin = 'builtins.open'
+
 
 class FakeRelation(object):
 
@@ -472,7 +479,7 @@
         self.assertEquals(result, expected)
 
     @patch('os.path.exists')
-    @patch('__builtin__.open')
+    @patch(open_builtin)
     def test_db_ssl(self, _open, osexists):
         osexists.return_value = False
         ssl_dir = '/etc/dbssl'
@@ -725,7 +732,7 @@
         }
         self.assertEquals(result, expected)
 
-    @patch('__builtin__.open')
+    @patch(open_builtin)
     def test_amqp_context_with_data_ssl(self, _open):
         '''Test amqp context with all required data and ssl'''
         relation = FakeRelation(relation_data=AMQP_RELATION_WITH_SSL)
@@ -799,7 +806,7 @@
             'rabbitmq_password': 'foobar',
             'rabbitmq_user': 'adam',
             'rabbitmq_virtual_host': 'foo',
-            'rabbitmq_hosts': 'rabbithost2,rabbithost1',
+            'rabbitmq_hosts': 'rabbithost1,rabbithost2',
         }
         self.assertEquals(result, expected)
 
@@ -868,7 +875,7 @@
         ceph = context.CephContext()
         result = ceph()
         expected = {
-            'mon_hosts': 'ceph_node2 ceph_node1',
+            'mon_hosts': 'ceph_node1 ceph_node2',
             'auth': 'foo',
             'key': 'bar',
             'use_syslog': 'true'
@@ -882,8 +889,8 @@
     def test_ceph_context_with_missing_data(self, ensure_packages, mkdir):
         '''Test ceph context with missing relation data'''
         relation = copy(CEPH_RELATION)
-        for k, v in relation.iteritems():
-            for u in v.iterkeys():
+        for k, v in six.iteritems(relation):
+            for u in six.iterkeys(v):
                 del relation[k][u]['auth']
         relation = FakeRelation(relation_data=relation)
         self.relation_get.side_effect = relation.get
@@ -911,7 +918,7 @@
         ceph = context.CephContext()
         result = ceph()
         expected = {
-            'mon_hosts': '192.168.1.11 192.168.1.10',
+            'mon_hosts': '192.168.1.10 192.168.1.11',
             'auth': 'foo',
             'key': 'bar',
             'use_syslog': 'true',
@@ -1264,12 +1271,9 @@
             if len(vips) > 1:
                 ex = {
                     'namespace': 'cinder',
-                    'endpoints': [('10.5.1.100', '10.5.1.1',
-                                   8766, 8756),
-                                  ('10.5.2.100', '10.5.2.1',
-                                   8766, 8756),
-                                  ('10.5.3.100', '10.5.3.1',
-                                   8766, 8756)],
+                    'endpoints': [('10.5.1.100', '10.5.1.1', 8766, 8756),
+                                  ('10.5.2.100', '10.5.2.1', 8766, 8756),
+                                  ('10.5.3.100', '10.5.3.1', 8766, 8756)],
                     'ext_ports': [8766]
                 }
             else:
@@ -1283,12 +1287,10 @@
             if multinet:
                 ex = {
                     'namespace': 'cinder',
-                    'endpoints': [('10.5.3.100', '10.5.3.100',
-                                   8776, 8766),
-                                  ('10.5.2.100', '10.5.2.100',
-                                   8776, 8766),
-                                  ('10.5.1.100', '10.5.1.100',
-                                   8776, 8766)],
+                    'endpoints': sorted([
+                        ('10.5.3.100', '10.5.3.100', 8776, 8766),
+                        ('10.5.2.100', '10.5.2.100', 8776, 8766),
+                        ('10.5.1.100', '10.5.1.100', 8776, 8766)]),
                     'ext_ports': [8776]
                 }
             else:
@@ -1391,9 +1393,10 @@
         apache = context.ApacheSSLContext()
         self.assertEquals(apache.canonical_names(), ['cinderhost1'])
         rel.relation_data = IDENTITY_RELATION_MULTIPLE_CERT
-        self.assertEquals(apache.canonical_names(), ['cinderhost1-adm-network',
-                                                     'cinderhost1-int-network',
-                                                     'cinderhost1-pub-network'])
+        self.assertEquals(apache.canonical_names(),
+                          sorted(['cinderhost1-adm-network',
+                                  'cinderhost1-int-network',
+                                  'cinderhost1-pub-network']))
         rel.relation_data = IDENTITY_RELATION_NO_CERT
         self.assertEquals(apache.canonical_names(), [])
 
@@ -1903,17 +1906,15 @@
 
     def test_workerconfig_context_noconfig(self):
         self.config.return_value = None
-        with patch.object(context.WorkerConfigContext, 'num_cpus') as cpus:
-            cpus.__get__ = Mock(return_value=2)
+        with patch.object(context.WorkerConfigContext, 'num_cpus', 2):
             worker = context.WorkerConfigContext()
-            self.assertEqual({'workers': 2}, worker())
+            self.assertEqual({'workers': 0}, worker())
 
     def test_workerconfig_context_withconfig(self):
         self.config.side_effect = fake_config({
             'worker-multiplier': 4,
         })
-        with patch.object(context.WorkerConfigContext, 'num_cpus') as cpus:
-            cpus.__get__ = Mock(return_value=2)
+        with patch.object(context.WorkerConfigContext, 'num_cpus', 2):
             worker = context.WorkerConfigContext()
             self.assertEqual({'workers': 8}, worker())
 

=== modified file 'tests/contrib/openstack/test_os_templating.py'
--- tests/contrib/openstack/test_os_templating.py	2014-11-25 14:35:22 +0000
+++ tests/contrib/openstack/test_os_templating.py	2014-11-25 15:09:14 +0000
@@ -1,7 +1,13 @@
 
 import os
-
 import unittest
+
+import six
+if not six.PY3:
+    builtin_open = '__builtin__.open'
+else:
+    builtin_open = 'builtins.open'
+
 from mock import patch, call, MagicMock
 
 import charmhelpers.contrib.openstack.templating as templating
@@ -160,7 +166,7 @@
     def test_render_template_by_basename(self):
         '''It renders template if it finds it by config file basename'''
 
-    @patch('__builtin__.open')
+    @patch(builtin_open)
     @patch.object(templating, 'get_loader')
     def test_write_out_config(self, loader, _open):
         '''It writes a templated config when provided a complete context'''
@@ -183,7 +189,7 @@
         ]
         with patch.object(self.renderer, 'write') as _write:
             self.renderer.write_all()
-            self.assertEquals(ex_calls, _write.call_args_list)
+            self.assertEquals(sorted(ex_calls), sorted(_write.call_args_list))
             pass
 
     @patch.object(templating, 'get_loader')

=== modified file 'tests/contrib/storage/test_linux_ceph.py'
--- tests/contrib/storage/test_linux_ceph.py	2014-11-19 22:27:26 +0000
+++ tests/contrib/storage/test_linux_ceph.py	2014-11-25 15:09:14 +0000
@@ -14,19 +14,19 @@
 import time
 
 
-LS_POOLS = """
+LS_POOLS = b"""
 images
 volumes
 rbd
 """
 
-LS_RBDS = """
+LS_RBDS = b"""
 rbd1
 rbd2
 rbd3
 """
 
-IMG_MAP = """
+IMG_MAP = b"""
 bar
 baz
 """
@@ -93,7 +93,7 @@
     @patch.object(ceph_utils, 'ceph_version')
     def test_get_osds(self, version):
         version.return_value = '0.56.2'
-        self.check_output.return_value = json.dumps([1, 2, 3])
+        self.check_output.return_value = json.dumps([1, 2, 3]).encode('UTF-8')
         self.assertEquals(ceph_utils.get_osds('test'), [1, 2, 3])
 
     @patch.object(ceph_utils, 'ceph_version')
@@ -104,7 +104,7 @@
     @patch.object(ceph_utils, 'ceph_version')
     def test_get_osds_none(self, version):
         version.return_value = '0.56.2'
-        self.check_output.return_value = json.dumps(None)
+        self.check_output.return_value = json.dumps(None).encode('UTF-8')
         self.assertEquals(ceph_utils.get_osds('test'), None)
 
     @patch.object(ceph_utils, 'get_osds')
@@ -534,13 +534,13 @@
     @patch('os.path.exists')
     def test_ceph_version_error(self, path, output):
         path.return_value = True
-        output.return_value = ''
+        output.return_value = b''
         self.assertEquals(ceph_utils.ceph_version(), None)
 
     @patch.object(ceph_utils, 'check_output')
     @patch('os.path.exists')
     def test_ceph_version_ok(self, path, output):
         path.return_value = True
-        output.return_value = 'ceph version 0.67.4'\
-            ' (ad85b8bfafea6232d64cb7ba76a8b6e8252fa0c7)'
+        output.return_value = \
+            b'ceph version 0.67.4 (ad85b8bfafea6232d64cb7ba76a8b6e8252fa0c7)'
         self.assertEquals(ceph_utils.ceph_version(), '0.67.4')

=== modified file 'tests/contrib/storage/test_linux_storage_lvm.py'
--- tests/contrib/storage/test_linux_storage_lvm.py	2014-07-03 12:36:50 +0000
+++ tests/contrib/storage/test_linux_storage_lvm.py	2014-11-25 15:09:14 +0000
@@ -5,7 +5,7 @@
 
 import charmhelpers.contrib.storage.linux.lvm as lvm
 
-PVDISPLAY = """
+PVDISPLAY = b"""
   --- Physical volume ---
   PV Name               /dev/loop0
   VG Name               foo
@@ -19,7 +19,7 @@
 
 """
 
-EMPTY_VG_IN_PVDISPLAY = """
+EMPTY_VG_IN_PVDISPLAY = b"""
   --- Physical volume ---
   PV Name               /dev/loop0
   VG Name

=== modified file 'tests/contrib/storage/test_linux_storage_utils.py'
--- tests/contrib/storage/test_linux_storage_utils.py	2014-07-31 08:55:15 +0000
+++ tests/contrib/storage/test_linux_storage_utils.py	2014-11-25 15:09:14 +0000
@@ -14,7 +14,7 @@
     @patch(STORAGE_LINUX_UTILS + '.check_call')
     def test_zap_disk(self, check_call, call, check_output):
         '''It calls sgdisk correctly to zap disk'''
-        check_output.return_value = '200\n'
+        check_output.return_value = b'200\n'
         storage_utils.zap_disk('/dev/foo')
         call.assert_any_call(['sgdisk', '--zap-all', '--mbrtogpt',
                               '--clear', '/dev/foo'])
@@ -50,7 +50,7 @@
     def test_is_device_mounted(self, check_output):
         '''It detects mounted devices as mounted.'''
         check_output.return_value = (
-            "/dev/sda1 on / type ext4 (rw,errors=remount-ro)\n")
+            b"/dev/sda1 on / type ext4 (rw,errors=remount-ro)\n")
         result = storage_utils.is_device_mounted('/dev/sda')
         self.assertTrue(result)
 
@@ -58,7 +58,7 @@
     def test_is_device_mounted_partition(self, check_output):
         '''It detects mounted partitions as mounted.'''
         check_output.return_value = (
-            "/dev/sda1 on / type ext4 (rw,errors=remount-ro)\n")
+            b"/dev/sda1 on / type ext4 (rw,errors=remount-ro)\n")
         result = storage_utils.is_device_mounted('/dev/sda1')
         self.assertTrue(result)
 
@@ -67,7 +67,7 @@
         '''It detects mounted devices as mounted if "mount" shows only a
         partition as mounted.'''
         check_output.return_value = (
-            "/dev/sda1 on / type ext4 (rw,errors=remount-ro)\n")
+            b"/dev/sda1 on / type ext4 (rw,errors=remount-ro)\n")
         result = storage_utils.is_device_mounted('/dev/sda')
         self.assertTrue(result)
 
@@ -75,7 +75,7 @@
     def test_is_device_mounted_not_mounted(self, check_output):
         '''It detects unmounted devices as not mounted.'''
         check_output.return_value = (
-            "/dev/foo on / type ext4 (rw,errors=remount-ro)\n")
+            b"/dev/foo on / type ext4 (rw,errors=remount-ro)\n")
         result = storage_utils.is_device_mounted('/dev/sda')
         self.assertFalse(result)
 
@@ -83,7 +83,7 @@
     def test_is_device_mounted_not_mounted_partition(self, check_output):
         '''It detects unmounted partitions as not mounted.'''
         check_output.return_value = (
-            "/dev/foo on / type ext4 (rw,errors=remount-ro)\n")
+            b"/dev/foo on / type ext4 (rw,errors=remount-ro)\n")
         result = storage_utils.is_device_mounted('/dev/sda1')
         self.assertFalse(result)
 
@@ -91,7 +91,7 @@
     def test_is_device_mounted_cciss(self, check_output):
         '''It detects mounted cciss partitions as mounted.'''
         check_output.return_value = (
-            "/dev/cciss/c0d0 on / type ext4 (rw,errors=remount-ro)\n")
+            b"/dev/cciss/c0d0 on / type ext4 (rw,errors=remount-ro)\n")
         result = storage_utils.is_device_mounted('/dev/cciss/c0d0')
         self.assertTrue(result)
 
@@ -99,6 +99,6 @@
     def test_is_device_mounted_cciss_not_mounted(self, check_output):
         '''It detects unmounted cciss partitions as not mounted.'''
         check_output.return_value = (
-            "/dev/cciss/c0d1 on / type ext4 (rw,errors=remount-ro)\n")
+            b"/dev/cciss/c0d1 on / type ext4 (rw,errors=remount-ro)\n")
         result = storage_utils.is_device_mounted('/dev/cciss/c0d0')
         self.assertFalse(result)

=== modified file 'tests/contrib/templating/test_contexts.py'
--- tests/contrib/templating/test_contexts.py	2014-08-13 20:52:09 +0000
+++ tests/contrib/templating/test_contexts.py	2014-11-25 15:09:14 +0000
@@ -9,6 +9,8 @@
 import unittest
 import yaml
 
+import six
+
 import charmhelpers.contrib.templating.contexts
 
 
@@ -134,12 +136,12 @@
         }
         self.mock_relations.return_value = {
             'wsgi-file': {
-                u'wsgi-file:0': {
-                    u'gunicorn/1': {
-                        u'private-address': u'10.0.3.99',
+                six.u('wsgi-file:0'): {
+                    six.u('gunicorn/1'): {
+                        six.u('private-address'): six.u('10.0.3.99'),
                     },
                     'click-index/3': {
-                        u'wsgi_group': u'ubunet',
+                        six.u('wsgi_group'): six.u('ubunet'),
                     },
                 },
             },
@@ -162,8 +164,9 @@
             expected["wsgi_file:relation_key2"] = "relation_value2"
             expected["relations_full"]['wsgi-file'] = {
                 'wsgi-file:0': {
-                    'gunicorn/1': {u'private-address': u'10.0.3.99'},
-                    'click-index/3': {u'wsgi_group': u'ubunet'},
+                    'gunicorn/1': {
+                        six.u('private-address'): six.u('10.0.3.99')},
+                    'click-index/3': {six.u('wsgi_group'): six.u('ubunet')},
                 },
             }
             expected["relations"]["wsgi-file"] = [

=== modified file 'tests/contrib/unison/test_unison.py'
--- tests/contrib/unison/test_unison.py	2014-10-22 06:17:55 +0000
+++ tests/contrib/unison/test_unison.py	2014-11-25 15:09:14 +0000
@@ -110,13 +110,13 @@
 
         isfile.return_value = False
         with patch_open() as (_open, _file):
-            self.check_output.return_value = 'fookey'
+            self.check_output.return_value = b'fookey'
             unison.create_public_key(
                 user='foo', priv_key_path='/home/foo/.ssh/id_rsa',
                 pub_key_path='/home/foo/.ssh/id_rsa.pub')
             self.assertIn(call(create_cmd), self.check_output.call_args_list)
             _open.assert_called_with('/home/foo/.ssh/id_rsa.pub', 'wb')
-            _file.write.assert_called_with('fookey')
+            _file.write.assert_called_with(b'fookey')
 
     @patch('os.mkdir')
     @patch('os.path.isdir')

=== modified file 'tests/core/test_fstab.py'
--- tests/core/test_fstab.py	2014-11-25 14:35:22 +0000
+++ tests/core/test_fstab.py	2014-11-25 15:09:14 +0000
@@ -24,7 +24,7 @@
 class FstabTest(unittest.TestCase):
 
     def setUp(self):
-        self.tempfile = tempfile.NamedTemporaryFile(delete=False)
+        self.tempfile = tempfile.NamedTemporaryFile('w+', delete=False)
         self.tempfile.write(DEFAULT_FSTAB_FILE)
         self.tempfile.close()
         self.fstab = Fstab(path=self.tempfile.name)

=== modified file 'tests/core/test_hookenv.py'
--- tests/core/test_hookenv.py	2014-11-25 14:35:22 +0000
+++ tests/core/test_hookenv.py	2014-11-25 15:09:14 +0000
@@ -3,14 +3,18 @@
 from subprocess import CalledProcessError
 import shutil
 import tempfile
-
-import cPickle as pickle
 from mock import patch, call, mock_open
-from StringIO import StringIO
 from mock import MagicMock
 from testtools import TestCase
 import yaml
 
+import six
+from six.moves import StringIO
+if six.PY3:
+    import pickle
+else:
+    import cPickle as pickle
+
 from charmhelpers.core import hookenv
 
 CHARM_METADATA = """name: testmock
@@ -125,7 +129,7 @@
     def test_keys(self):
         c = hookenv.Config(dict(foo='bar'))
         c["baz"] = "bar"
-        self.assertEqual([u"foo", "baz"], c.keys())
+        self.assertEqual(sorted([six.u("foo"), "baz"]), sorted(c.keys()))
 
 
 class SerializableTest(TestCase):
@@ -165,11 +169,9 @@
         }
         wrapped = hookenv.Serializable(foo)
         for meth in ('keys', 'values', 'items'):
-            self.assertEqual(getattr(wrapped, meth)(), getattr(foo, meth)())
+            self.assertEqual(sorted(list(getattr(wrapped, meth)())),
+                             sorted(list(getattr(foo, meth)())))
 
-        for meth in ('iterkeys', 'itervalues', 'iteritems'):
-            self.assertEqual(list(getattr(wrapped, meth)()),
-                             list(getattr(foo, meth)()))
         self.assertEqual(wrapped.get('bar'), foo.get('bar'))
         self.assertEqual(wrapped.get('baz', 42), foo.get('baz', 42))
         self.assertIn('bar', wrapped)
@@ -246,7 +248,7 @@
     @patch('subprocess.check_output')
     def test_gets_charm_config_with_scope(self, check_output):
         config_data = 'bar'
-        check_output.return_value = json.dumps(config_data)
+        check_output.return_value = json.dumps(config_data).encode('UTF-8')
 
         result = hookenv.config(scope='baz')
 
@@ -257,11 +259,11 @@
         self.assertEqual(result[1], 'a')
 
         # ... because the result is actually a string
-        self.assert_(isinstance(result, basestring))
+        self.assert_(isinstance(result, six.string_types))
 
     @patch('subprocess.check_output')
     def test_gets_missing_charm_config_with_scope(self, check_output):
-        check_output.return_value = ''
+        check_output.return_value = b''
 
         result = hookenv.config(scope='baz')
 
@@ -271,7 +273,7 @@
     @patch('charmhelpers.core.hookenv.charm_dir')
     @patch('subprocess.check_output')
     def test_gets_config_without_scope(self, check_output, charm_dir):
-        check_output.return_value = json.dumps(dict(foo='bar'))
+        check_output.return_value = json.dumps(dict(foo='bar')).encode('UTF-8')
         charm_dir.side_effect = tempfile.mkdtemp
 
         result = hookenv.config()
@@ -327,7 +329,7 @@
     @patch('charmhelpers.core.hookenv.relation_type')
     def test_gets_relation_ids(self, relation_type, check_output):
         ids = [1, 2, 3]
-        check_output.return_value = json.dumps(ids)
+        check_output.return_value = json.dumps(ids).encode('UTF-8')
         reltype = 'foo'
         relation_type.return_value = reltype
 
@@ -341,7 +343,7 @@
     @patch('charmhelpers.core.hookenv.relation_type')
     def test_gets_relation_ids_empty_array(self, relation_type, check_output):
         ids = []
-        check_output.return_value = json.dumps(None)
+        check_output.return_value = json.dumps(None).encode('UTF-8')
         reltype = 'foo'
         relation_type.return_value = reltype
 
@@ -355,7 +357,7 @@
     @patch('charmhelpers.core.hookenv.relation_type')
     def test_relation_ids_no_relation_type(self, relation_type, check_output):
         ids = [1, 2, 3]
-        check_output.return_value = json.dumps(ids)
+        check_output.return_value = json.dumps(ids).encode('UTF-8')
         relation_type.return_value = None
 
         result = hookenv.relation_ids()
@@ -366,7 +368,7 @@
     @patch('charmhelpers.core.hookenv.relation_type')
     def test_gets_relation_ids_for_type(self, relation_type, check_output):
         ids = [1, 2, 3]
-        check_output.return_value = json.dumps(ids)
+        check_output.return_value = json.dumps(ids).encode('UTF-8')
         reltype = 'foo'
 
         result = hookenv.relation_ids(reltype)
@@ -382,7 +384,7 @@
         relid = 123
         units = ['foo', 'bar']
         relation_id.return_value = relid
-        check_output.return_value = json.dumps(units)
+        check_output.return_value = json.dumps(units).encode('UTF-8')
 
         result = hookenv.related_units()
 
@@ -393,10 +395,10 @@
     @patch('subprocess.check_output')
     @patch('charmhelpers.core.hookenv.relation_id')
     def test_gets_related_units_empty_array(self, relation_id, check_output):
-        relid = 123
+        relid = str(123)
         units = []
         relation_id.return_value = relid
-        check_output.return_value = json.dumps(None)
+        check_output.return_value = json.dumps(None).encode('UTF-8')
 
         result = hookenv.related_units()
 
@@ -409,7 +411,7 @@
     def test_related_units_no_relation(self, relation_id, check_output):
         units = ['foo', 'bar']
         relation_id.return_value = None
-        check_output.return_value = json.dumps(units)
+        check_output.return_value = json.dumps(units).encode('UTF-8')
 
         result = hookenv.related_units()
 
@@ -421,7 +423,7 @@
     def test_gets_related_units_for_id(self, relation_id, check_output):
         relid = 123
         units = ['foo', 'bar']
-        check_output.return_value = json.dumps(units)
+        check_output.return_value = json.dumps(units).encode('UTF-8')
 
         result = hookenv.related_units(relid)
 
@@ -765,7 +767,7 @@
     @patch('subprocess.check_output')
     def test_gets_relation(self, check_output):
         data = {"foo": "BAR"}
-        check_output.return_value = json.dumps(data)
+        check_output.return_value = json.dumps(data).encode('UTF-8')
         result = hookenv.relation_get()
 
         self.assertEqual(result['foo'], 'BAR')
@@ -773,7 +775,7 @@
 
     @patch('charmhelpers.core.hookenv.subprocess')
     def test_relation_get_none(self, mock_subprocess):
-        mock_subprocess.check_output.return_value = 'null'
+        mock_subprocess.check_output.return_value = b'null'
 
         result = hookenv.relation_get()
 
@@ -801,7 +803,7 @@
 
     @patch('subprocess.check_output')
     def test_gets_relation_with_scope(self, check_output):
-        check_output.return_value = json.dumps('bar')
+        check_output.return_value = json.dumps('bar').encode('UTF-8')
 
         result = hookenv.relation_get(attribute='baz-scope')
 
@@ -811,7 +813,7 @@
 
     @patch('subprocess.check_output')
     def test_gets_missing_relation_with_scope(self, check_output):
-        check_output.return_value = ""
+        check_output.return_value = b""
 
         result = hookenv.relation_get(attribute='baz-scope')
 
@@ -821,7 +823,7 @@
 
     @patch('subprocess.check_output')
     def test_gets_relation_with_unit_name(self, check_output):
-        check_output.return_value = json.dumps('BAR')
+        check_output.return_value = json.dumps('BAR').encode('UTF-8')
 
         result = hookenv.relation_get(attribute='baz-scope', unit='baz-unit')
 
@@ -834,7 +836,7 @@
     @patch('subprocess.check_output')
     def test_relation_set_flushes_local_unit_cache(self, check_output,
                                                    check_call, local_unit):
-        check_output.return_value = json.dumps('BAR')
+        check_output.return_value = json.dumps('BAR').encode('UTF-8')
         local_unit.return_value = 'baz_unit'
         hookenv.relation_get(attribute='baz_scope', unit='baz_unit')
         hookenv.relation_get(attribute='bar_scope')
@@ -845,7 +847,7 @@
 
     @patch('subprocess.check_output')
     def test_gets_relation_with_relation_id(self, check_output):
-        check_output.return_value = json.dumps('BAR')
+        check_output.return_value = json.dumps('BAR').encode('UTF-8')
 
         result = hookenv.relation_get(attribute='baz-scope', unit='baz-unit',
                                       rid=123)
@@ -910,13 +912,13 @@
 
     @patch('subprocess.check_output')
     def test_gets_unit_attribute(self, check_output_):
-        check_output_.return_value = json.dumps('bar')
+        check_output_.return_value = json.dumps('bar').encode('UTF-8')
         self.assertEqual(hookenv.unit_get('foo'), 'bar')
         check_output_.assert_called_with(['unit-get', '--format=json', 'foo'])
 
     @patch('subprocess.check_output')
     def test_gets_missing_unit_attribute(self, check_output_):
-        check_output_.return_value = ""
+        check_output_.return_value = b""
         self.assertEqual(hookenv.unit_get('foo'), None)
         check_output_.assert_called_with(['unit-get', '--format=json', 'foo'])
 
@@ -925,8 +927,8 @@
         values = {
             'hello': 'world',
             'foo': 'bar',
-            'baz': None
-            }
+            'baz': None,
+        }
 
         @hookenv.cached
         def cache_function(attribute):

=== modified file 'tests/core/test_host.py'
--- tests/core/test_host.py	2014-11-25 14:35:22 +0000
+++ tests/core/test_host.py	2014-11-25 15:09:14 +0000
@@ -19,30 +19,30 @@
                """rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0
 """).strip().split('\n')
 
-LSB_RELEASE = u'''DISTRIB_ID=Ubuntu
+LSB_RELEASE = '''DISTRIB_ID=Ubuntu
 DISTRIB_RELEASE=13.10
 DISTRIB_CODENAME=saucy
 DISTRIB_DESCRIPTION="Ubuntu Saucy Salamander (development branch)"
 '''
 
-IP_LINE_ETH0 = ("""
+IP_LINE_ETH0 = b"""
 2: eth0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master bond0 state UP qlen 1000
     link/ether e4:11:5b:ab:a7:3c brd ff:ff:ff:ff:ff:ff
-""")
+"""
 
-IP_LINE_ETH1 = ("""
+IP_LINE_ETH1 = b"""
 3: eth1: <BROADCAST,MULTICAST> mtu 1546 qdisc noop state DOWN qlen 1000
     link/ether e4:11:5b:ab:a7:3c brd ff:ff:ff:ff:ff:ff
-""")
+"""
 
-IP_LINE_HWADDR = ("""2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000\    link/ether e4:11:5b:ab:a7:3c brd ff:ff:ff:ff:ff:ff""")
+IP_LINE_HWADDR = b"""2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000\    link/ether e4:11:5b:ab:a7:3c brd ff:ff:ff:ff:ff:ff"""
 
 IP_LINES = IP_LINE_ETH0 + IP_LINE_ETH1
 
-IP_LINE_BONDS = ("""
+IP_LINE_BONDS = b"""
 6: bond0.10@bond0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
 link/ether 08:00:27:16:b9:5f brd ff:ff:ff:ff:ff:ff
-""")
+"""
 
 
 class HelpersTest(TestCase):
@@ -166,12 +166,12 @@
 
     @patch('subprocess.check_output')
     def test_service_running_on_stopped_service(self, check_output):
-        check_output.return_value = 'foo stop/waiting'
+        check_output.return_value = b'foo stop/waiting'
         self.assertFalse(host.service_running('foo'))
 
     @patch('subprocess.check_output')
     def test_service_running_on_running_service(self, check_output):
-        check_output.return_value = 'foo start/running, process 23871'
+        check_output.return_value = b'foo start/running, process 23871'
         self.assertTrue(host.service_running('foo'))
 
     @patch('subprocess.check_output')
@@ -284,7 +284,7 @@
     def test_rsyncs_a_path(self, log, check_output):
         from_path = '/from/this/path/foo'
         to_path = '/to/this/path/bar'
-        check_output.return_value = ' some output '
+        check_output.return_value = b' some output '  # Spaces will be stripped
 
         result = host.rsync(from_path, to_path)
 
@@ -319,7 +319,7 @@
         path = '/some/other/path/from/link'
         realpath = '/some/path'
         path_exists = False
-        perms = 0644
+        perms = 0o644
 
         getpwnam.return_value.pw_uid = uid
         getgrnam.return_value.gr_gid = gid
@@ -343,7 +343,7 @@
         path = '/some/other/path/from/link'
         realpath = '/some/path'
         path_exists = False
-        perms = 0555
+        perms = 0o555
 
         os_.path.abspath.return_value = realpath
         os_.path.exists.return_value = path_exists
@@ -370,7 +370,7 @@
         path_exists = True
         force = True
         is_dir = False
-        perms = 0644
+        perms = 0o644
 
         getpwnam.return_value.pw_uid = uid
         getgrnam.return_value.gr_gid = gid
@@ -402,7 +402,7 @@
         group = 'some-group-{bar}'
         path = '/some/path/{baz}'
         contents = 'what is {juju}'
-        perms = 0644
+        perms = 0o644
         fileno = 'some-fileno'
 
         getpwnam.return_value.pw_uid = uid
@@ -428,7 +428,7 @@
         gid = 0
         path = '/some/path/{baz}'
         fmtstr = 'what is {juju}'
-        perms = 0444
+        perms = 0o444
         fileno = 'some-fileno'
 
         with patch_open() as (mock_open, mock_file):
@@ -636,7 +636,7 @@
 
         @host.restart_on_change(restart_map)
         def make_some_changes(mock_file):
-            mock_file.read.return_value = "newstuff"
+            mock_file.read.return_value = b"newstuff"
 
         with patch_open() as (mock_open, mock_file):
             make_some_changes(mock_file)
@@ -664,7 +664,7 @@
             pass
 
         with patch_open() as (mock_open, mock_file):
-            mock_file.read.side_effect = ['exists', 'missing', 'exists2']
+            mock_file.read.side_effect = [b'exists', b'missing', b'exists2']
             make_some_changes()
 
         # Restart should only happen once per service
@@ -691,7 +691,7 @@
             pass
 
         with patch_open() as (mock_open, mock_file):
-            mock_file.read.side_effect = ['exists', 'missing', 'exists2']
+            mock_file.read.side_effect = [b'exists', b'missing', b'exists2']
             make_some_changes()
 
         # Restarts should happen in the order they are described in the
@@ -713,7 +713,6 @@
         with mocked_open('/etc/lsb-release', LSB_RELEASE):
             lsb_release = host.lsb_release()
             for key in result:
-                print lsb_release
                 self.assertEqual(result[key], lsb_release[key])
 
     def test_pwgen(self):

=== modified file 'tests/core/test_services.py'
--- tests/core/test_services.py	2014-11-25 14:35:22 +0000
+++ tests/core/test_services.py	2014-11-25 15:09:14 +0000
@@ -573,7 +573,7 @@
         with mock.patch.object(services.helpers, 'open', mopen, create=True):
             services.helpers.StoredContext('foo.yaml', {'key': 'val'})
         mopen.assert_called_once_with('charm_dir/foo.yaml', 'w')
-        fchmod.assert_called_once_with(mopen.return_value.fileno(), 0600)
+        fchmod.assert_called_once_with(mopen.return_value.fileno(), 0o600)
         yaml.dump.assert_called_once_with({'key': 'val'}, mopen.return_value)
 
     @mock.patch.object(hookenv, 'charm_dir', lambda: 'charm_dir')
@@ -637,7 +637,7 @@
         callback(manager, 'test', 'event')
         mtemplating.render.assert_called_once_with(
             'foo.yml', 'bar.yml', {'foo': 'bar'},
-            'root', 'root', 0444)
+            'root', 'root', 0o444)
 
     @mock.patch.object(services.helpers, 'templating')
     def test_template_explicit(self, mtemplating):
@@ -645,14 +645,14 @@
             'required_data': [{'foo': 'bar'}]}})
         callback = services.template(
             source='foo.yml', target='bar.yml',
-            owner='user', group='group', perms=0555
+            owner='user', group='group', perms=0o555
         )
         assert isinstance(callback, services.ManagerCallback)
         assert not mtemplating.render.called
         callback(manager, 'test', 'event')
         mtemplating.render.assert_called_once_with(
             'foo.yml', 'bar.yml', {'foo': 'bar'},
-            'user', 'group', 0555)
+            'user', 'group', 0o555)
 
 
 class TestPortsCallback(unittest.TestCase):

=== modified file 'tests/core/test_sysctl.py'
--- tests/core/test_sysctl.py	2014-11-25 14:35:22 +0000
+++ tests/core/test_sysctl.py	2014-11-25 15:09:14 +0000
@@ -4,11 +4,18 @@
 __author__ = 'Jorge Niedbalski R. <[e-mail]>'
 
 from charmhelpers.core.sysctl import create
+import io
 from mock import patch, MagicMock
-
 import unittest
 import tempfile
 
+import six
+if not six.PY3:
+    builtin_open = '__builtin__.open'
+else:
+    builtin_open = 'builtins.open'
+
+
 TO_PATCH = [
     'log',
     'check_call',
@@ -27,10 +34,10 @@
         self.addCleanup(_m.stop)
         return mock
 
-    @patch('__builtin__.open')
+    @patch(builtin_open)
     def test_create(self, mock_open):
         """Test create sysctl method"""
-        _file = MagicMock(spec=file)
+        _file = MagicMock(spec=io.FileIO)
         mock_open.return_value = _file
 
         create('{"kernel.max_pid": 1337}', "/etc/sysctl.d/test-sysctl.conf")

=== modified file 'tests/fetch/test_archiveurl.py'
--- tests/fetch/test_archiveurl.py	2014-11-25 14:35:22 +0000
+++ tests/fetch/test_archiveurl.py	2014-11-25 15:09:14 +0000
@@ -1,6 +1,14 @@
 import os
-from testtools import TestCase
-from urlparse import urlparse
+
+import six
+if six.PY3:
+    from urllib.parse import urlparse
+    from urllib.error import URLError
+else:
+    from urllib2 import URLError
+    from urlparse import urlparse
+
+from unittest import TestCase
 from mock import (
     MagicMock,
     patch,
@@ -11,7 +19,6 @@
     archiveurl,
     UnhandledSource,
 )
-import urllib2
 
 
 class ArchiveUrlFetchHandlerTest(TestCase):
@@ -50,7 +57,7 @@
             result = self.fh.can_handle(url)
             self.assertNotEqual(result, True, url)
 
-    @patch('urllib2.urlopen')
+    @patch('charmhelpers.fetch.archiveurl.urlopen')
     def test_downloads(self, _urlopen):
         for url in self.valid_urls:
             response = MagicMock()
@@ -87,7 +94,7 @@
 
         url = "http://www.example.com/archive.tar.gz"
 
-        self.fh.download.side_effect = urllib2.URLError('fail')
+        self.fh.download.side_effect = URLError('fail')
         with patch.dict('os.environ', {'CHARM_DIR': 'foo'}):
             self.assertRaises(UnhandledSource, self.fh.install, url)
 

=== modified file 'tests/fetch/test_bzrurl.py'
--- tests/fetch/test_bzrurl.py	2014-11-25 14:35:22 +0000
+++ tests/fetch/test_bzrurl.py	2014-11-25 15:09:14 +0000
@@ -1,20 +1,34 @@
 import os
 from testtools import TestCase
-from urlparse import urlparse
 from mock import (
     MagicMock,
     patch,
 )
-from charmhelpers.fetch import (
-    bzrurl,
-    UnhandledSource,
-)
-
-
+import unittest
+
+import six
+if six.PY3:
+    from urllib.parse import urlparse
+else:
+    from urlparse import urlparse
+
+try:
+    from charmhelpers.fetch import (
+        bzrurl,
+        UnhandledSource,
+    )
+except ImportError:
+    bzrurl = None
+    UnhandledSource = None
+
+
[e-mail](six.PY3, 'bzr does not support Python 3')
 class BzrUrlFetchHandlerTest(TestCase):
 
     def setUp(self):
         super(BzrUrlFetchHandlerTest, self).setUp()
+        if six.PY3:
+            return
         self.valid_urls = (
             "bzr+ssh://example.com/branch-name",
             "bzr+ssh://example.com/branch-name/",
@@ -41,6 +55,7 @@
         )
         self.fh = bzrurl.BzrUrlFetchHandler()
 
+    @unittest.skipIf(six.PY3, 'bzr does not support Python 3')
     def test_handles_bzr_urls(self):
         for url in self.valid_urls:
             result = self.fh.can_handle(url)
@@ -49,6 +64,7 @@
             result = self.fh.can_handle(url)
             self.assertNotEqual(result, True, url)
 
+    @unittest.skipIf(six.PY3, 'bzr does not support Python 3')
     @patch('bzrlib.branch.Branch.open')
     def test_branch(self, _open):
         dest_path = "/destination/path"
@@ -63,6 +79,7 @@
             with patch.dict('os.environ', {'CHARM_DIR': 'foo'}):
                 self.assertRaises(UnhandledSource, self.fh.branch, url, dest_path)
 
+    @unittest.skipIf(six.PY3, 'bzr does not support Python 3')
     @patch('charmhelpers.fetch.bzrurl.mkdir')
     def test_installs(self, _mkdir):
         self.fh.branch = MagicMock()
@@ -73,4 +90,4 @@
             with patch.dict('os.environ', {'CHARM_DIR': 'foo'}):
                 where = self.fh.install(url)
             self.assertEqual(where, dest)
-            _mkdir.assert_called_with(where, perms=0755)
+            _mkdir.assert_called_with(where, perms=0o755)

=== modified file 'tests/fetch/test_fetch.py'
--- tests/fetch/test_fetch.py	2014-11-25 14:35:22 +0000
+++ tests/fetch/test_fetch.py	2014-11-25 15:09:14 +0000
@@ -1,4 +1,3 @@
-from cStringIO import StringIO
 import subprocess
 
 from tests.helpers import patch_open
@@ -8,11 +7,18 @@
     MagicMock,
     call,
 )
-from urlparse import urlparse
 from charmhelpers import fetch
 import os
 import yaml
 
+import six
+from six.moves import StringIO
+if six.PY3:
+    from urllib.parse import urlparse
+else:
+    from urlparse import urlparse
+
+
 FAKE_APT_CACHE = {
     # an installed package
     'vim': {
@@ -404,7 +410,11 @@
     @patch('charmhelpers.fetch.log')
     def test_plugins_are_valid(self, log_):
         plugins = fetch.plugins()
-        self.assertEqual(len(fetch.FETCH_HANDLERS), len(plugins))
+        if not six.PY3:
+            self.assertEqual(len(fetch.FETCH_HANDLERS), len(plugins))
+        else:
+            # No bzr or git libraries for Python3.
+            self.assertEqual(len(fetch.FETCH_HANDLERS) - 2, len(plugins))
 
 
 class BaseFetchHandlerTest(TestCase):

=== modified file 'tests/fetch/test_giturl.py'
--- tests/fetch/test_giturl.py	2014-11-25 14:35:22 +0000
+++ tests/fetch/test_giturl.py	2014-11-25 15:09:14 +0000
@@ -1,18 +1,31 @@
 import os
 from testtools import TestCase
-from urlparse import urlparse
 from mock import (
     MagicMock,
     patch,
 )
-from charmhelpers.fetch import (
-    giturl,
-    UnhandledSource,
-)
-
-
+import unittest
+
+import six
+if six.PY3:
+    from urllib.parse import urlparse
+else:
+    from urlparse import urlparse
+
+try:
+    from charmhelpers.fetch import (
+        giturl,
+        UnhandledSource,
+    )
+except ImportError:
+    giturl = None
+    UnhandledSource = None
+
+
[e-mail](six.PY3, 'git does not support Python 3')
 class GitUrlFetchHandlerTest(TestCase):
 
+    @unittest.skipIf(six.PY3, 'git does not support Python 3')
     def setUp(self):
         super(GitUrlFetchHandlerTest, self).setUp()
         self.valid_urls = (
@@ -27,6 +40,7 @@
         )
         self.fh = giturl.GitUrlFetchHandler()
 
+    @unittest.skipIf(six.PY3, 'git does not support Python 3')
     def test_handles_git_urls(self):
         for url in self.valid_urls:
             result = self.fh.can_handle(url)
@@ -35,6 +49,7 @@
             result = self.fh.can_handle(url)
             self.assertNotEqual(result, True, url)
 
+    @unittest.skipIf(six.PY3, 'git does not support Python 3')
     @patch('git.Repo.clone_from')
     def test_branch(self, _clone_from):
         dest_path = "/destination/path"
@@ -52,6 +67,7 @@
                                   dest_path,
                                   branch)
 
+    @unittest.skipIf(six.PY3, 'git does not support Python 3')
     @patch('charmhelpers.fetch.giturl.mkdir')
     def test_installs(self, _mkdir):
         self.fh.clone = MagicMock()
@@ -63,4 +79,4 @@
             with patch.dict('os.environ', {'CHARM_DIR': 'foo'}):
                 where = self.fh.install(url)
             self.assertEqual(where, dest)
-            _mkdir.assert_called_with(where, perms=0755)
+            _mkdir.assert_called_with(where, perms=0o755)

=== modified file 'tests/helpers.py'
--- tests/helpers.py	2014-11-25 14:35:22 +0000
+++ tests/helpers.py	2014-11-25 15:09:14 +0000
@@ -3,6 +3,12 @@
 from mock import patch, MagicMock
 import io
 
+import six
+if not six.PY3:
+    builtin_open = '__builtin__.open'
+else:
+    builtin_open = 'builtins.open'
+
 
 @contextmanager
 def patch_open():
@@ -11,26 +17,29 @@
 
     Yields the mock for "open" and "file", respectively.'''
     mock_open = MagicMock(spec=open)
-    mock_file = MagicMock(spec=file)
+    mock_file = MagicMock(spec=io.FileIO)
 
     @contextmanager
     def stub_open(*args, **kwargs):
         mock_open(*args, **kwargs)
         yield mock_file
 
-    with patch('__builtin__.open', stub_open):
+    with patch(builtin_open, stub_open):
         yield mock_open, mock_file
 
 
 @contextmanager
 def mock_open(filename, contents=None):
     ''' Slightly simpler mock of open to return contents for filename '''
-    def mock_file(*args):
-        if args[0] == filename:
+    def mock_file(name, mode='r', buffering=-1):  # Python 2 signature.
+        if name == filename:
+            if (not six.PY3) or 'b' in mode:
+                return io.BytesIO(contents)
             return io.StringIO(contents)
         else:
-            return open(*args)
-    with patch('__builtin__.open', mock_file):
+            return open(name, mode, buffering)
+
+    with patch(builtin_open, mock_file):
         yield
 
 

=== modified file 'tests/payload/test_archive.py'
--- tests/payload/test_archive.py	2014-11-25 14:35:22 +0000
+++ tests/payload/test_archive.py	2014-11-25 15:09:14 +0000
@@ -84,7 +84,7 @@
         self.addCleanup(rmtree, destdir)
         try:
             zip_file, contents = self.create_archive("zip")
-        except subprocess.CalledProcessError, e:
+        except subprocess.CalledProcessError as e:
             if e.returncode == 127:
                 self.skip("Skipping - zip is not installed")
             else:

=== modified file 'tests/payload/test_execd.py'
--- tests/payload/test_execd.py	2013-07-11 08:31:49 +0000
+++ tests/payload/test_execd.py	2014-11-25 15:09:14 +0000
@@ -111,7 +111,7 @@
 
         expected = [os.path.join(self.test_charm_dir, 'exec.d', mod,
                                  'charm-pre-install') for mod in modules]
-        self.assertItemsEqual(submodules, expected)
+        self.assertEqual(sorted(submodules), sorted(expected))
 
     def test_execd_run(self):
         modules = ['basenode', 'mod2', 'c']

=== modified file 'tests/tools/test_charm_helper_sync.py'
--- tests/tools/test_charm_helper_sync.py	2014-11-25 14:35:22 +0000
+++ tests/tools/test_charm_helper_sync.py	2014-11-25 15:09:14 +0000
@@ -2,6 +2,13 @@
 from mock import call, patch
 import yaml
 
+import six
+if not six.PY3:
+    builtin_open = '__builtin__.open'
+else:
+    builtin_open = 'builtins.open'
+
+
 import tools.charm_helpers_sync.charm_helpers_sync as sync
 
 INCLUDE = """
@@ -46,7 +53,7 @@
         self.assertEquals('/tmp/mycharm/hooks/charmhelpers/contrib/openstack',
                           path)
 
-    @patch('__builtin__.open')
+    @patch(builtin_open)
     @patch('os.path.exists')
     @patch('os.walk')
     def test_ensure_init(self, walk, exists, _open):
@@ -114,7 +121,7 @@
         isfile.side_effect = _isfile
         isdir.side_effect = _isdir
         result = sync.get_filter(opts)(dir='/tmp/charm-helpers/core',
-                                       ls=files.iterkeys())
+                                       ls=six.iterkeys(files))
         return result
 
     @patch('os.path.isdir')
@@ -123,15 +130,15 @@
         '''It filters out all non-py files by default'''
         result = self._test_filter_dir(opts=None, isfile=isfile, isdir=isdir)
         ex = ['bad_file.bin', 'bad_file.img', 'some_dir']
-        self.assertEquals(ex, result)
+        self.assertEquals(sorted(ex), sorted(result))
 
     @patch('os.path.isdir')
     @patch('os.path.isfile')
     def test_filter_dir_with_include(self, isfile, isdir):
         '''It includes non-py files if specified as an include opt'''
-        result = self._test_filter_dir(opts=['inc=*.img'],
-                                       isfile=isfile, isdir=isdir)
-        ex = ['bad_file.bin', 'some_dir']
+        result = sorted(self._test_filter_dir(opts=['inc=*.img'],
+                                              isfile=isfile, isdir=isdir))
+        ex = sorted(['bad_file.bin', 'some_dir'])
         self.assertEquals(ex, result)
 
     @patch('os.path.isdir')

=== modified file 'tools/charm_helpers_sync/charm_helpers_sync.py'
--- tools/charm_helpers_sync/charm_helpers_sync.py	2014-11-25 14:35:22 +0000
+++ tools/charm_helpers_sync/charm_helpers_sync.py	2014-11-25 15:09:14 +0000
@@ -14,9 +14,10 @@
 import sys
 import tempfile
 import yaml
-
 from fnmatch import fnmatch
 
+import six
+
 CHARM_HELPERS_BRANCH = 'lp:charm-helpers'
 
 
@@ -139,7 +140,7 @@
 
 def extract_options(inc, global_options=None):
     global_options = global_options or []
-    if global_options and isinstance(global_options, basestring):
+    if global_options and isinstance(global_options, six.string_types):
         global_options = [global_options]
     if '|' not in inc:
         return (inc, global_options)
@@ -159,7 +160,7 @@
             sync(src, dest, inc, opts)
         elif isinstance(inc, dict):
             # could also do nested dicts here.
-            for k, v in inc.iteritems():
+            for k, v in six.iteritems(inc):
                 if isinstance(v, list):
                     for m in v:
                         inc, opts = extract_options(m, global_options)
@@ -217,7 +218,7 @@
         checkout = clone_helpers(tmpd, config['branch'])
         sync_helpers(config['include'], checkout, config['destination'],
                      options=sync_options)
-    except Exception, e:
+    except Exception as e:
         logging.error("Could not sync: %s" % e)
         raise e
     finally: