Merge ~racb/uvtool:python3-packaging into uvtool:ubuntu/devel

[Robie Basak]

[Christian Ehrhardt  (paelzer)]

I didn't expect you to need an explicit --buildsystem=pybuild, but it is ok.
Content changes match what I have reviewed for the non packaging branch.

Packaging changes pull in quite some old changelogs that are correct, but not part of this merge.
But it matches what is in Eoan, so that should be fine as well.

Was that the base for the test PPA that we already tested or do you want testing again?

review: Approve

[Robie Basak (racb)]

On Wed, Oct 23, 2019 at 11:20:54AM -0000, Christian Ehrhardt  wrote:
> I didn't expect you to need an explicit --buildsystem=pybuild, but it is ok.

I am under the impression that --buildsystem=pybuild is necessary
currently because it is the recommended best practice but not the
debhelper default.

> Packaging changes pull in quite some old changelogs that are correct, but not part of this merge.
> But it matches what is in Eoan, so that should be fine as well.

I think that's a problem with the preview diff. I think the actual
commits are correct and don't include any unnecessary changes.

> Was that the base for the test PPA that we already tested or do you want testing again?

I updated ppa:racb/experimental2 for Eoan (only) with essentially the
same changes. The only differences are that this branch is rebased on to
my merge into master, rather than my development Python 3 branch, I
dropped uvtool-coverage, and the changelog is different.

If you'd like to see the changes, I have been tagging previous versions
of the packaging branch and pushing those tags - see
python3-packaging/v* in my repo.

I'm confident that it isn't necessary to test further, but if you'd like
to, then please do :)

diff --git a/bin/uvt-kvm b/bin/uvt-kvm
index c96c50d..282e866 100755
--- a/bin/uvt-kvm
+++ b/bin/uvt-kvm
@@ -1,8 +1,8 @@
-#!/usr/bin/python
+#!/usr/bin/python3
 
 # Wrapper around cloud-localds and libvirt
 
-# Copyright (C) 2012-3 Canonical Ltd.
+# Copyright (C) 2012-9 Canonical Ltd.
 # Author: Robie Basak <[e-mail]>
 #
 # This program is free software: you can redistribute it and/or modify
diff --git a/bin/uvt-simplestreams-libvirt b/bin/uvt-simplestreams-libvirt
index 7693b02..b220da2 100755
--- a/bin/uvt-simplestreams-libvirt
+++ b/bin/uvt-simplestreams-libvirt
@@ -1,8 +1,8 @@
-#!/usr/bin/python
+#!/usr/bin/python3
 
 # Keep Ubuntu Cloud images synced to a local libvirt storage pool.
 
-# Copyright (C) 2013 Canonical Ltd.
+# Copyright (C) 2013-9 Canonical Ltd.
 # Author: Robie Basak <[e-mail]>
 #
 # This program is free software: you can redistribute it and/or modify
diff --git a/debian/changelog b/debian/changelog
index af18485..3f77faa 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,29 @@
+uvtool (0~git162-0ubuntu1) UNRELEASED; urgency=medium
+
+  * Port to Python 3.
+
+ -- Robie Basak <[e-mail]>  Wed, 23 Oct 2019 11:30:04 +0100
+
+uvtool (0~git148-0ubuntu1) eoan; urgency=medium
+
+  [ Christian Ehrhardt ]
+  * kvm: add --machine-type option (LP: #1775645)
+  * man: fix rendering page of option after --machine-type
+  * man: describe implications and defaults of --machine-type
+
+  [ Robie Basak ]
+  * Add --mac option (LP: #1781727). Thanks to adamretter.
+
+  [ Christian Ehrhardt ]
+  * template (x86): switch the default video to QXL
+  * template (x86): add spice graphics backend
+
+  [ Robie Basak ]
+  * Add --network-config option (LP: #1781785). Thanks to adamretter.
+  * Default to plain HTTP for cloud image downloads (LP: #1409400)
+
+ -- Robie Basak <[e-mail]>  Tue, 30 Apr 2019 16:55:01 +0100
+
 uvtool (0~git140-0ubuntu1) bionic; urgency=medium
 
   [ Christian Ehrhardt ]
diff --git a/debian/control b/debian/control
index 9f509bf..8163583 100644
--- a/debian/control
+++ b/debian/control
@@ -4,15 +4,13 @@ Priority: extra
 Standards-Version: 3.9.4
 Maintainer: Ubuntu Developers <[e-mail]>
 Build-Depends: debhelper (>= 7),
-               python-all,
-               python-setuptools,
-               python-mock,
-               python-libvirt,
-               python-lxml,
-               python-pyinotify,
-               python-simplestreams,
-               python-yaml
-X-Python-Version: >= 2.7
+               dh-python,
+               python3,
+               python3-libvirt,
+               python3-lxml,
+               python3-pyinotify,
+               python3-simplestreams,
+               python3-yaml
 
 Package: uvtool
 Architecture: all
@@ -30,18 +28,18 @@ Package: uvtool-libvirt
 Architecture: all
 Depends: libvirt-daemon-system | libvirt-bin,
          libvirt-clients | libvirt-bin,
-         python-libvirt,
-         python-simplestreams,
-         python-lxml,
-         python-pyinotify,
-         python-yaml,
+         python3-libvirt,
+         python3-simplestreams,
+         python3-lxml,
+         python3-pyinotify,
+         python3-yaml,
          distro-info,
          cloud-image-utils (>= 0.27),
          qemu-utils,
          ubuntu-cloudimage-keyring,
          socat,
          ${misc:Depends},
-         ${python:Depends}
+         ${python3:Depends}
 Recommends: qemu-kvm, cpu-checker
 Description: Library and tools for using Ubuntu Cloud Images with libvirt
  This package provides libvirt-specific tools for consuming Ubuntu Cloud
diff --git a/debian/rules b/debian/rules
index 25dccc9..82c03eb 100755
--- a/debian/rules
+++ b/debian/rules
@@ -1,13 +1,13 @@
 #!/usr/bin/make -f
 
 %:
-	dh $@ --with python2
+	dh $@ --with python3 --buildsystem=pybuild
 
 override_dh_auto_build:
 	$(MAKE) -C uvtool/tests/streams
 	dh_auto_build
-	PYTHONPATH=$(CURDIR) python -m unittest uvtool.tests.test_kvm
-	PYTHONPATH=$(CURDIR) python -m unittest uvtool.tests.test_simplestreams
+	PYTHONPATH=$(CURDIR) python3 -m unittest uvtool.tests.test_kvm
+	PYTHONPATH=$(CURDIR) python3 -m unittest uvtool.tests.test_simplestreams
 
 override_dh_auto_clean:
 	$(MAKE) -C uvtool/tests/streams clean
diff --git a/man/uvt-kvm.1 b/man/uvt-kvm.1
index 7ad22e6..e626370 100644
--- a/man/uvt-kvm.1
+++ b/man/uvt-kvm.1
@@ -50,9 +50,11 @@ wrapping libvirt and cloud-init.
 is not intended to wrap all possible use cases. Where possible, it
 provides access to some more advanced cases using options to override
 entire sections of default operation, such as the ability to directly
-override the backing volume image used, the libvirt domain definition
-and cloud-init metadata and userdata. For yet more complex cases, it is
-expected that the user will call libvirt directly (for example by using
+override the backing volume image used, the libvirt domain definition,
+cloud-init metadata and userdata, and directly provide a network-config
+file.
+For yet more complex cases, it is expected that the user will call
+libvirt directly (for example by using
 .BR virsh (1)),
 and use uvt-kvm for only the simpler operations required on affected
 VMs. See ADVANCED OVERRIDE OPTIONS and ADVANCED USAGE for details.
@@ -364,6 +366,11 @@ Replace the first defined NIC with one that connects to the given host
 bridge. Default: unaltered from the libvirt domain template.
 
 .TP
+.BI --mac\ mac
+Use this MAC address for the first defined NIC. Can be used in
+conjunction with --bridge. Default unspecified.
+
+.TP
 .B --log-console-output
 Log output to a disk file on the host instead of to a pty. With
 libvirt's default configuration on Ubuntu, this log can be found in
@@ -381,6 +388,17 @@ Instead of the default cpu model - which mostly is a compatibility focused
 lowest denominator of cpu features - use host-passthrough which will try to
 make all of the hosts cpu features available in the guest.
 
+.TP
+.BI --machine-type\  type
+Set the machine type to the specified string before instantiating the guest.
+See \fBkvm -M ?\fR for a list of types supported by your current system.
+If not set this section of the template is not altered before the guest is
+defined.
+If set this modifies the internal temporary libvirt domain template at
+the element \fBdomain->os->type\fR and sets the attribute \fBmachine\fR to the
+given value before defining the guest. This implies that libvirt will add the
+type-dependent default devices when defining the guest.
+
 .SH CLOUD-INIT CONFIGURATION OPTIONS
 
 Valid for: \fBuvt-kvm\ create\fR only.
@@ -487,6 +505,12 @@ not otherwise tunable.
 Default: minimal file with automatically generated instance-id.
 
 .TP
+.BI --network-config\  network_config_file
+Provide cloud-init network-config using the file supplied.
+
+Default: no network-config file.
+
+.TP
 .BI --backing-image-file\  image_file
 Specify the name of a local file that will be used to create the VM instead of
 relying on the volume storage pool. It must point to a qcow2 formatted file.
diff --git a/template.xml b/template.xml
index 3c795c1..b76bcb8 100644
--- a/template.xml
+++ b/template.xml
@@ -20,6 +20,11 @@
     <graphics type='vnc' autoport='yes' listen='127.0.0.1'>
       <listen type='address' address='127.0.0.1'/>
     </graphics>
-    <video/>
+    <graphics type='spice' autoport='yes' listen='127.0.0.1'>
+      <listen type='address' address='127.0.0.1'/>
+    </graphics>
+    <video>
+      <model type='qxl'/>
+    </video>
   </devices>
 </domain>
diff --git a/uvtool/libvirt/__init__.py b/uvtool/libvirt/__init__.py
index 2cd950c..3988e24 100644
--- a/uvtool/libvirt/__init__.py
+++ b/uvtool/libvirt/__init__.py
@@ -1,4 +1,4 @@
-# Copyright (C) 2013 Canonical Ltd.
+# Copyright (C) 2013-9 Canonical Ltd.
 # Author: Robie Basak <[e-mail]>
 #
 # This program is free software: you can redistribute it and/or modify
@@ -91,7 +91,7 @@ def _create_volume_from_fobj_with_size(new_volume_name, fobj, fobj_size,
         E.target(E.format(type=image_type)),
         *extra
         )
-    vol = pool.createXML(etree.tostring(new_vol), 0)
+    vol = pool.createXML(etree.tostring(new_vol, encoding=str), 0)
 
     try:
         stream = conn.newStream(0)
diff --git a/uvtool/libvirt/kvm.py b/uvtool/libvirt/kvm.py
index 5e1c15e..c5b10d7 100755
--- a/uvtool/libvirt/kvm.py
+++ b/uvtool/libvirt/kvm.py
@@ -1,8 +1,8 @@
-#!/usr/bin/python
+#!/usr/bin/python3
 
 # Wrapper around cloud-localds and libvirt
 
-# Copyright (C) 2012-3 Canonical Ltd.
+# Copyright (C) 2012-9 Canonical Ltd.
 # Author: Robie Basak <[e-mail]>
 #
 # This program is free software: you can redistribute it and/or modify
@@ -23,15 +23,16 @@ from __future__ import print_function
 from __future__ import unicode_literals
 
 import argparse
+import base64
 import errno
 import functools
+import io
 import itertools
 import os
 import platform
 import shutil
 import signal
 import string
-import StringIO
 import subprocess
 import sys
 import tempfile
@@ -92,19 +93,19 @@ def subprocess_setup():
 def run_script_once_arg_to_config(arg, unique_id):
     with open(arg, 'rb') as f:
         script = f.read()
-    encoded_script = script.encode('base64')
+    encoded_script = base64.b64encode(script).decode('ascii')
     return [
-        b'cloud-init-per',
-        b'once',
-        unique_id.encode('utf-8'),
-        b'sh', b'-c',
+        'cloud-init-per',
+        'once',
+        unique_id,
+        'sh', '-c',
         (
-            b'f=$(mktemp --tmpdir %s-XXXXXXXXXX) && ' +
-            b'echo "%s" | base64 -d > "$f" && ' +
-            b'chmod 700 "$f" && ' +
-            b'"$f" && ' +
-            b'rm "$f"'
-        ) % (unique_id.encode('utf-8'), encoded_script)
+            'f=$(mktemp --tmpdir %s-XXXXXXXXXX) && ' +
+            'echo "%s" | base64 -d > "$f" && ' +
+            'chmod 700 "$f" && ' +
+            '"$f" && ' +
+            'rm "$f"'
+        ) % (unique_id, encoded_script)
     ]
 
 
@@ -122,7 +123,7 @@ def get_ssh_agent_public_keys():
             output = subprocess.check_output(
                 ['ssh-add', '-L'],
                 stderr=fpnull
-            )
+            ).decode()
     except subprocess.CalledProcessError:
         return None
 
@@ -140,7 +141,7 @@ def read_ssh_public_key_file(filename):
         filename = os.path.join(os.environ['HOME'], '.ssh', 'id_rsa.pub')
 
     try:
-        f = open(filename, 'rb')
+        f = open(filename, 'r')
     except IOError as e:
         if e.errno != errno.ENOENT:
             raise
@@ -178,51 +179,58 @@ def create_default_user_data(fobj, args, ssh_host_keys=None):
 
     """
 
-    ssh_authorized_keys = get_ssh_authorized_keys(args.ssh_public_key_file)
+    ssh_authorized_keys = list(
+        get_ssh_authorized_keys(args.ssh_public_key_file),
+    )
 
     data = {
-        b'hostname': args.hostname.encode('ascii'),
-        b'manage_etc_hosts': b'localhost',
-        b'ssh_keys': uvtool.ssh.generate_ssh_host_keys()[0],
+        'hostname': args.hostname,
+        'manage_etc_hosts': 'localhost',
+        'ssh_keys': uvtool.ssh.generate_ssh_host_keys()[0],
     }
 
     if ssh_host_keys:
-        data[b'ssh_keys'] = ssh_host_keys
+        data['ssh_keys'] = ssh_host_keys
 
     if ssh_authorized_keys:
-        data[b'ssh_authorized_keys'] = ssh_authorized_keys
+        data['ssh_authorized_keys'] = ssh_authorized_keys
 
     if args.password:
-        data[b'password'] = args.password.encode('utf-8')
-        data[b'chpasswd'] = {b'expire': False}
-        data[b'ssh_pwauth'] = True
+        data['password'] = args.password
+        data['chpasswd'] = {'expire': False}
+        data['ssh_pwauth'] = True
 
     if args.run_script_once:
-        data[b'runcmd'] = run_script_once_args_to_config(args.run_script_once)
+        data['runcmd'] = run_script_once_args_to_config(args.run_script_once)
 
     if args.packages:
-        data[b'packages'] = [
-            s.encode('ascii')  # Debian Policy dictates a-z,0-9,+,-,.
-            for s in itertools.chain(*[p.split(',') for p in args.packages])
-        ]
+        data['packages'] = list(
+            itertools.chain(*[p.split(',') for p in args.packages])
+        )
 
-    fobj.write("#cloud-config\n")
-    fobj.write(yaml.dump(data))
+    fobj.write(b"#cloud-config\n")
+    fobj.write(yaml.dump(data).encode())
 
 
 def create_default_meta_data(fobj, args):
     data = {
-        b'instance-id': str(uuid.uuid1()).encode('ascii'),
+        'instance-id': str(uuid.uuid1()),
     }
-    fobj.write(yaml.dump(data))
+    fobj.write(yaml.dump(data).encode())
 
 
-def create_ds_image(temp_dir, hostname, user_data_fobj, meta_data_fobj):
+def create_ds_image(
+        temp_dir,
+        hostname,
+        user_data_fobj,
+        meta_data_fobj,
+        network_config_fobj=None
+    ):
     """Create a file called ds.img inside temp_dir that contains a useful
     cloud-init data source.
 
     Other temporary files created in temp_dir are currently metadata and
-    userdata and can be safely deleted.
+    userdata and network-config and can be safely deleted.
 
     """
 
@@ -230,17 +238,35 @@ def create_ds_image(temp_dir, hostname, user_data_fobj, meta_data_fobj):
         f.write(user_data_fobj.read())
     with open(os.path.join(temp_dir, 'metadata'), 'wb') as f:
         f.write(meta_data_fobj.read())
-
-    subprocess.check_call(
-        ['cloud-localds', '--disk-format=qcow2', 'ds.img', 'userdata', 'metadata'], cwd=temp_dir)
-
-
-def create_ds_volume(new_volume_name, hostname, user_data_fobj, meta_data_fobj):
+    if network_config_fobj:
+        with open(os.path.join(temp_dir, 'network-config'), 'wb') as f:
+            f.write(network_config_fobj.read())
+
+    args = ['cloud-localds', '--disk-format=qcow2']
+    if network_config_fobj:
+        args.extend(['--network-config', 'network-config'])
+    args.extend(['ds.img', 'userdata', 'metadata'])
+    subprocess.check_call(args, cwd=temp_dir)
+
+
+def create_ds_volume(
+        new_volume_name,
+        hostname,
+        user_data_fobj,
+        meta_data_fobj,
+        network_config_fobj=None
+    ):
     """Create a new libvirt cloud-init datasource volume."""
 
     temp_dir = tempfile.mkdtemp(prefix='uvt-kvm-')
     try:
-        create_ds_image(temp_dir, hostname, user_data_fobj, meta_data_fobj)
+        create_ds_image(
+            temp_dir=temp_dir,
+            hostname=hostname,
+            user_data_fobj=user_data_fobj,
+            meta_data_fobj=meta_data_fobj,
+            network_config_fobj=network_config_fobj,
+        )
         with open(os.path.join(temp_dir, 'ds.img'), 'rb') as f:
             return uvtool.libvirt.create_volume_from_fobj(
                 new_volume_name, f, image_type='qcow2', pool_name=POOL_NAME)
@@ -301,12 +327,12 @@ def create_cow_volume_by_path(backing_volume_path, new_volume_name,
             E.format(type='qcow2'),
             )
         )
-    return pool.createXML(etree.tostring(new_vol), 0)
+    return pool.createXML(etree.tostring(new_vol, encoding=str), 0)
 
 
 def compose_domain_xml(name, volumes, template_path, cpu=1, memory=512,
         unsafe_caching=False, log_console_output=False, host_passthrough=False,
-        bridge=None, ssh_known_hosts=None):
+        machine_type=None, bridge=None, mac=None, ssh_known_hosts=None):
     tree = etree.parse(template_path)
     domain = tree.getroot()
     assert domain.tag == 'domain'
@@ -351,11 +377,28 @@ def compose_domain_xml(name, volumes, template_path, cpu=1, memory=512,
 
     if bridge:
         etree.strip_elements(devices, 'interface')
-        devices.append(E.interface(
-                         E.source(bridge=bridge),
-                         E.model(type='virtio'),
-                         type='bridge'),
-                      )
+        if mac:
+            devices.append(E.interface(
+                             E.mac(address=mac),
+                             E.source(bridge=bridge),
+                             E.model(type='virtio'),
+                             type='bridge'),
+                          )
+        else:
+            devices.append(E.interface(
+                             E.source(bridge=bridge),
+                             E.model(type='virtio'),
+                             type='bridge'),
+                          )
+    else:
+        if mac:
+            etree.strip_elements(devices, 'interface')
+            devices.append(E.interface(
+                                 E.mac(address=mac),
+                                 E.source(network='default'),
+                                 E.model(type='virtio'),
+                                 type='network'),
+                          )
 
     if log_console_output:
         if ARCH == 's390x':
@@ -378,6 +421,11 @@ def compose_domain_xml(name, volumes, template_path, cpu=1, memory=512,
         else:
             etree.SubElement(domain, 'cpu', mode='host-passthrough')
 
+    if machine_type:
+        domos = domain.find('os')
+        domtype = domos.find('type')
+        domtype.set('machine', machine_type)
+
     if ssh_known_hosts:
         metadata = domain.find('metadata')
         if metadata is None:
@@ -389,7 +437,7 @@ def compose_domain_xml(name, volumes, template_path, cpu=1, memory=512,
         )
         metadata.append(EX.ssh_known_hosts(ssh_known_hosts))
 
-    return etree.tostring(tree)
+    return etree.tostring(tree, encoding=str)
 
 
 def get_base_image(filters):
@@ -403,11 +451,27 @@ def get_base_image(filters):
     return result[0]
 
 
-def create(hostname, filters, user_data_fobj, meta_data_fobj, template_path,
-           memory=512, cpu=1, disk=2, unsafe_caching=False,
-           log_console_output=False, host_passthrough=False, bridge=None,
-           backing_image_file=None, start=True, ssh_known_hosts=None,
-           ephemeral_disks=None):
+def create(
+        hostname,
+        filters,
+        user_data_fobj,
+        meta_data_fobj,
+        network_config_fobj,
+        template_path,
+        memory=512,
+        cpu=1,
+        disk=2,
+        unsafe_caching=False,
+        log_console_output=False,
+        host_passthrough=False,
+        bridge=None,
+        mac=None,
+        backing_image_file=None,
+        start=True,
+        ssh_known_hosts=None,
+        ephemeral_disks=None,
+        machine_type=None
+    ):
     if backing_image_file is None:
         base_volume_name = get_base_image(filters)
     if ephemeral_disks is None:
@@ -430,7 +494,12 @@ def create(hostname, filters, user_data_fobj, meta_data_fobj, template_path,
         undo_volume_creation.append(main_vol)
 
         ds_vol = create_ds_volume(
-            "%s-ds.qcow" % hostname, hostname, user_data_fobj, meta_data_fobj)
+            "%s-ds.qcow" % hostname,
+            hostname,
+            user_data_fobj,
+            meta_data_fobj,
+            network_config_fobj,
+        )
         undo_volume_creation.append(ds_vol)
 
         volumes = [main_vol, ds_vol]
@@ -443,6 +512,7 @@ def create(hostname, filters, user_data_fobj, meta_data_fobj, template_path,
         xml = compose_domain_xml(
             hostname, volumes=volumes,
             bridge=bridge,
+            mac=mac,
             cpu=cpu,
             log_console_output=log_console_output,
             host_passthrough=host_passthrough,
@@ -450,6 +520,7 @@ def create(hostname, filters, user_data_fobj, meta_data_fobj, template_path,
             template_path=template_path,
             unsafe_caching=unsafe_caching,
             ssh_known_hosts=ssh_known_hosts,
+            machine_type=machine_type,
         )
         conn = libvirt.open('qemu:///system')
         domain = conn.defineXML(xml)
@@ -510,7 +581,7 @@ def destroy(hostname):
 
 def get_lts_series():
     output = subprocess.check_output(['distro-info', '--lts'], close_fds=True)
-    return output.strip()
+    return output.strip().decode()
 
 
 def apply_default_fobj(args, key, create_default_data_fn):
@@ -528,7 +599,7 @@ def apply_default_fobj(args, key, create_default_data_fn):
     if specified_fobj:
         return specified_fobj
     else:
-        default_fobj = StringIO.StringIO()
+        default_fobj = io.BytesIO()
         create_default_data_fn(default_fobj, args)
         default_fobj.seek(0)
         return default_fobj
@@ -583,6 +654,7 @@ def ssh(name, login_name, arguments, stdin=None, checked=False, sysexit=True,
         )
         if ssh_known_hosts:
             ssh_known_hosts_file = tempfile.NamedTemporaryFile(
+                mode='w',
                 prefix='uvt-kvm.known_hoststmp')
             objects_to_close.append(ssh_known_hosts_file)
             ssh_known_hosts_file.write(ssh_known_hosts)
@@ -662,9 +734,14 @@ def main_create(parser, args):
     else:
         abs_image_backing_file = None
     create(
-        args.hostname, args.filters, user_data_fobj, meta_data_fobj,
+        hostname=args.hostname,
+        filters=args.filters,
+        user_data_fobj=user_data_fobj,
+        meta_data_fobj=meta_data_fobj,
+        network_config_fobj=args.network_config,
         backing_image_file=abs_image_backing_file,
         bridge=args.bridge,
+        mac=args.mac,
         cpu=args.cpu,
         disk=args.disk,
         log_console_output=args.log_console_output,
@@ -675,6 +752,7 @@ def main_create(parser, args):
         start=not args.no_start,
         ssh_known_hosts=ssh_known_hosts,
         ephemeral_disks=args.ephemeral_disks,
+        machine_type=args.machine_type,
     )
 
 
@@ -789,10 +867,16 @@ class DeveloperOptionAction(argparse.Action):
 def main(args):
     # Workaround for https://bugzilla.redhat.com/show_bug.cgi?id=1063766
     # (LP: #1228231)
-    libvirt.registerErrorHandler(lambda _: None, None)
+    def noop(*args, **kwargs): pass
+    libvirt.registerErrorHandler(noop, None)
 
     parser = argparse.ArgumentParser()
-    subparsers = parser.add_subparsers()
+    # Workarounds needed here: set dest and required
+    # https://bugs.python.org/issue9253#msg181855
+    # https://stackoverflow.com/q/22990977/478206
+    # https://stackoverflow.com/a/23354355/478206
+    subparsers = parser.add_subparsers(dest='subcommand')
+    subparsers.required = True
     create_subparser = subparsers.add_parser('create')
     create_subparser.set_defaults(func=main_create)
     create_subparser.add_argument(
@@ -805,16 +889,20 @@ def main(args):
         '--ephemeral-disk', action='append', type=int, dest='ephemeral_disks',
         help='Add an empty disk of SIZE in GB', metavar='SIZE')
     create_subparser.add_argument('--bridge')
+    create_subparser.add_argument('--mac')
     create_subparser.add_argument('--unsafe-caching', action='store_true')
     create_subparser.add_argument(
         '--user-data', type=argparse.FileType('rb'))
     create_subparser.add_argument(
         '--meta-data', type=argparse.FileType('rb'))
+    create_subparser.add_argument(
+        '--network-config', type=argparse.FileType('rb'))
     create_subparser.add_argument('--password')
     create_subparser.add_argument('--guest-arch',
         help='guest arch to select template, default is the host architecture')
     create_subparser.add_argument('--log-console-output', action='store_true')
     create_subparser.add_argument('--host-passthrough', action='store_true')
+    create_subparser.add_argument('--machine-type')
     create_subparser.add_argument('--backing-image-file')
     create_subparser.add_argument('--run-script-once', action='append')
     create_subparser.add_argument('--ssh-public-key-file')
diff --git a/uvtool/libvirt/simplestreams.py b/uvtool/libvirt/simplestreams.py
index 93fd3fb..d4cb8bc 100755
--- a/uvtool/libvirt/simplestreams.py
+++ b/uvtool/libvirt/simplestreams.py
@@ -1,8 +1,8 @@
-#!/usr/bin/python
+#!/usr/bin/python3
 
 # Keep Ubuntu Cloud images synced to a local libvirt storage pool.
 
-# Copyright (C) 2013 Canonical Ltd.
+# Copyright (C) 2013-9 Canonical Ltd.
 # Author: Robie Basak <[e-mail]>
 #
 # This program is free software: you can redistribute it and/or modify
@@ -107,7 +107,7 @@ BASE64_PREFIX = 'x-uvt-b64-'
 def _encode_libvirt_pool_name(product_name, version_name):
     return BASE64_PREFIX + base64.b64encode(
         (' '.join([product_name, version_name])).encode(), b'-_'
-    )
+    ).decode()
 
 
 def _decode_libvirt_pool_name(encoded_pool_name):
@@ -119,7 +119,7 @@ def _decode_libvirt_pool_name(encoded_pool_name):
     return base64.b64decode(
         encoded_pool_name[len(BASE64_PREFIX):],
         altchars=b'-_'
-    ).split(None, 1)
+    ).decode().split(None, 1)
 
 
 def purge_pool(conn=None):
@@ -164,16 +164,14 @@ def _load_products(path=None, content_id=None, clean=False):
     def new_product():
         return {'versions': {}}
     products = collections.defaultdict(new_product)
-    for encoded_libvirt_name_string, metadata in pool_metadata.items():
-        encoded_libvirt_name_bytes = encoded_libvirt_name_string.encode(
-            'utf-8')
+    for encoded_libvirt_name_string, metadata in list(pool_metadata.items()):
         if not uvtool.libvirt.have_volume_by_name(
-                encoded_libvirt_name_bytes, pool_name=LIBVIRT_POOL_NAME):
+                encoded_libvirt_name_string, pool_name=LIBVIRT_POOL_NAME):
             if clean:
                 del pool_metadata[encoded_libvirt_name_string]
             continue
         product, version = _decode_libvirt_pool_name(
-            encoded_libvirt_name_bytes)
+            encoded_libvirt_name_string)
         assert(product == metadata['product_name'])
         assert(version == metadata['version_name'])
         products[product]['versions'][version] = {
@@ -292,13 +290,19 @@ def main_purge(args):
 def main(argv=None):
     # Workaround for https://bugzilla.redhat.com/show_bug.cgi?id=1063766
     # (LP: #1228231)
-    libvirt.registerErrorHandler(lambda _: None, None)
+    def noop(*args, **kwargs): pass
+    libvirt.registerErrorHandler(noop, None)
 
     system_arch = subprocess.check_output(
         ['dpkg', '--print-architecture']).decode().strip()
     parser = argparse.ArgumentParser()
     parser.add_argument('--verbose', '-v', action='store_true')
-    subparsers = parser.add_subparsers()
+    # Workarounds needed here: set dest and required
+    # https://bugs.python.org/issue9253#msg181855
+    # https://stackoverflow.com/q/22990977/478206
+    # https://stackoverflow.com/a/23354355/478206
+    subparsers = parser.add_subparsers(dest='subcommand')
+    subparsers.required = True
 
     sync_subparser = subparsers.add_parser('sync')
     sync_subparser.set_defaults(func=main_sync)
@@ -312,7 +316,7 @@ def main(argv=None):
         default='/usr/share/keyrings/ubuntu-cloudimage-keyring.gpg'
     )
     sync_subparser.add_argument('--source', dest='mirror_url',
-        default='https://cloud-images.ubuntu.com/releases/')
+        default='http://cloud-images.ubuntu.com/releases/')
     sync_subparser.add_argument('--no-authentication', action='store_true')
     sync_subparser.add_argument('filters', nargs='*', metavar='filter',
         default=["arch=%s" % system_arch])
diff --git a/uvtool/ssh.py b/uvtool/ssh.py
index bb6b6d4..0516603 100644
--- a/uvtool/ssh.py
+++ b/uvtool/ssh.py
@@ -1,6 +1,6 @@
-#!/usr/bin/python
+#!/usr/bin/python3
 
-# Copyright (C) 2014 Canonical Ltd.
+# Copyright (C) 2014-9 Canonical Ltd.
 # Author: Robie Basak <[e-mail]>
 #
 # This program is free software: you can redistribute it and/or modify
@@ -36,7 +36,7 @@ def _keygen(key_type, private_path):
 
 
 def read_file(path):
-    with open(path, 'rb') as f:
+    with open(path, 'r') as f:
         return f.read()
 
 
@@ -52,9 +52,8 @@ def generate_ssh_host_keys():
             # ssh-keygen(1) defines that ".pub" is appended
             public_path = private_path + ".pub"
 
-            key_type_utf8 = key_type.encode('utf-8')
-            private_ci_key = key_type_utf8 + b'_private'
-            public_ci_key = key_type_utf8 + b'_public'
+            private_ci_key = key_type + '_private'
+            public_ci_key = key_type + '_public'
 
             private_key = read_file(private_path)
             public_key = read_file(public_path)
@@ -66,4 +65,4 @@ def generate_ssh_host_keys():
     finally:
         shutil.rmtree(tmp_dir)
 
-    return cloud_init_result, b''.join(known_hosts_result)
+    return cloud_init_result, ''.join(known_hosts_result)
diff --git a/uvtool/tests/test_kvm.py b/uvtool/tests/test_kvm.py
index 7a96ebf..13d7da6 100644
--- a/uvtool/tests/test_kvm.py
+++ b/uvtool/tests/test_kvm.py
@@ -1,4 +1,4 @@
-# Copyright (C) 2014 Canonical Ltd.
+# Copyright (C) 2014-9 Canonical Ltd.
 # Author: Robie Basak <[e-mail]>
 #
 # This program is free software: you can redistribute it and/or modify
@@ -15,8 +15,7 @@
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
 import unittest
-
-import mock
+import unittest.mock
 
 from uvtool.libvirt.kvm import main_ssh
 
@@ -25,18 +24,18 @@ class TestKVM(unittest.TestCase):
     def check_ssh(self, args_hostname, args_login_name, expected_hostname,
         expected_login_name):
 
-        parser = mock.Mock()
-        args = mock.Mock()
+        parser = unittest.mock.Mock()
+        args = unittest.mock.Mock()
         args.login_name = args_login_name
         args.name = args_hostname
-        args.ssh_arguments = mock.sentinel.ssh_arguments
+        args.ssh_arguments = unittest.mock.sentinel.ssh_arguments
         args.insecure = True
-        with mock.patch('uvtool.libvirt.kvm.ssh') as ssh_mock:
+        with unittest.mock.patch('uvtool.libvirt.kvm.ssh') as ssh_mock:
             main_ssh(parser, args)
             ssh_mock.assert_called_with(
                 expected_hostname,
                 expected_login_name,
-                mock.sentinel.ssh_arguments,
+                unittest.mock.sentinel.ssh_arguments,
                 insecure=True,
             )
 
diff --git a/uvtool/tests/test_simplestreams.py b/uvtool/tests/test_simplestreams.py
index 211454d..883391e 100644
--- a/uvtool/tests/test_simplestreams.py
+++ b/uvtool/tests/test_simplestreams.py
@@ -1,4 +1,4 @@
-# Copyright (C) 2014 Canonical Ltd.
+# Copyright (C) 2014-9 Canonical Ltd.
 # Author: Robie Basak <[e-mail]>
 #
 # This program is free software: you can redistribute it and/or modify
@@ -15,8 +15,7 @@
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
 import unittest
-
-import mock
+import unittest.mock
 
 import uvtool.libvirt.simplestreams as simplestreams
 
@@ -28,7 +27,7 @@ import uvtool.libvirt.simplestreams as simplestreams
 # NormalizedVersion as described in PEP-0386. As this is a temporary hack
 # anyway, this will do for now, since we know that the mock version on Ubuntu
 # Precise will never change.
-ON_PRECISE = mock.__version__ == '0.7.2'
+ON_PRECISE = unittest.mock.__version__ == '0.7.2'
 
 FAKE_VOLUME_PRODUCT_NAME = 'com.ubuntu.cloud:server:12.04:amd64'
 FAKE_VOLUME_VERSION_0 = '20131119'
@@ -39,9 +38,9 @@ ENCODED_FAKE_VOLUME_PRODUCT_NAME_1 = simplestreams._encode_libvirt_pool_name(
     FAKE_VOLUME_PRODUCT_NAME, FAKE_VOLUME_VERSION_1)
 
 @unittest.skipIf(ON_PRECISE, 'mock version is too old')
[e-mail]('uvtool.libvirt.simplestreams.uvtool.libvirt')
[e-mail]('uvtool.libvirt.simplestreams.pool_metadata', new={})
[e-mail]('uvtool.libvirt.simplestreams.libvirt')
[e-mail]('uvtool.libvirt.simplestreams.uvtool.libvirt')
[e-mail]('uvtool.libvirt.simplestreams.pool_metadata', new={})
[e-mail]('uvtool.libvirt.simplestreams.libvirt')
 class TestSimpleStreams(unittest.TestCase):
     def testSync(self, libvirt, uvtool_libvirt):
         uvtool_libvirt.have_volume_by_name.return_value = False
@@ -61,7 +60,7 @@ class TestSimpleStreams(unittest.TestCase):
         # least once by uvtool.libvirt.simplestreams directly. This is more of
         # an assertion about the test being correct than part of the test
         # itself.
-        libvirt.assert_has_calls([mock.call.open(u'qemu:///system')])
+        libvirt.assert_has_calls([unittest.mock.call.open(u'qemu:///system')])
 
         # create_volume_from_fobj should have been called exactly once to
         # create the volume with the name that we expect