Merge ~sylvain-pineau/plainbox-provider-snappy:cleanup_duplicates into plainbox-provider-snappy:master

[Sylvain Pineau]

Another prerequisite branch to get p-p-s and p-p-c parts in the same snapcraft.yaml.

Much simpler, only removals for this provider

[Maciej Kisielewski (kissiel)]

\o/

review: Approve

[Devices Certification Bot (ce-certification-qa)]

I tried to merge it but there are some problems. Typically you want to merge or rebase and try again.

review: Needs Fixing

[Devices Certification Bot (ce-certification-qa)]

The merge was fine but running tests failed.

[trusty] starting container
[trusty] (timing) 0.08user 0.01system 0:06.28elapsed 1%CPU (0avgtext+0avgdata 16636maxresident)k
[trusty] (timing) 0inputs+32outputs (0major+5614minor)pagefaults 0swaps
[trusty] provisioning container
[trusty] (timing) 37.27user 14.30system 1:27.76elapsed 58%CPU (0avgtext+0avgdata 69364maxresident)k
[trusty] (timing) 0inputs+1616624outputs (16major+1363858minor)pagefaults 0swaps
[trusty-testing] Starting tests...
Found a test script: ./requirements/container-tests-provider-snappy
[trusty-testing] container-tests-provider-snappy: FAIL
[trusty-testing] stdout: https://paste.ubuntu.com/25024433/
[trusty-testing] stderr: https://paste.ubuntu.com/25024434/
[trusty-testing] (timing) Command exited with non-zero status 1
[trusty-testing] (timing) 6.24user 0.51system 0:09.80elapsed 68%CPU (0avgtext+0avgdata 44144maxresident)k
[trusty-testing] (timing) 0inputs+69816outputs (0major+36174minor)pagefaults 0swaps
[trusty-testing] Fixing file permissions in source directory
[trusty-testing] Destroying container
Name: trusty-testing
State: STOPPED
[xenial] starting container
[xenial] (timing) 0.07user 0.02system 0:04.27elapsed 2%CPU (0avgtext+0avgdata 16676maxresident)k
[xenial] (timing) 0inputs+32outputs (0major+5290minor)pagefaults 0swaps
[xenial] provisioning container
[xenial] (timing) 70.78user 19.13system 2:14.92elapsed 66%CPU (0avgtext+0avgdata 107732maxresident)k
[xenial] (timing) 24inputs+2564832outputs (2major+2357284minor)pagefaults 0swaps
[xenial-testing] Starting tests...
Found a test script: ./requirements/container-tests-provider-snappy
[xenial-testing] container-tests-provider-snappy: FAIL
[xenial-testing] stdout: https://paste.ubuntu.com/25024441/
[xenial-testing] stderr: https://paste.ubuntu.com/25024442/
[xenial-testing] (timing) Command exited with non-zero status 1
[xenial-testing] (timing) 6.39user 0.55system 0:09.47elapsed 73%CPU (0avgtext+0avgdata 42364maxresident)k
[xenial-testing] (timing) 0inputs+69816outputs (0major+37446minor)pagefaults 0swaps
[xenial-testing] Fixing file permissions in source directory
[xenial-testing] Destroying container
Name: xenial-testing
State: STOPPED

review: Needs Fixing

[Sylvain Pineau (sylvain-pineau)]

rebeased

review: Approve

[Devices Certification Bot (ce-certification-qa)]

The merge was fine but running tests failed.

[trusty] starting container
[trusty] (timing) 0.08user 0.02system 0:05.26elapsed 2%CPU (0avgtext+0avgdata 16704maxresident)k
[trusty] (timing) 0inputs+32outputs (0major+5437minor)pagefaults 0swaps
[trusty] provisioning container
[trusty] (timing) 37.17user 14.38system 1:26.54elapsed 59%CPU (0avgtext+0avgdata 67792maxresident)k
[trusty] (timing) 0inputs+1616616outputs (16major+1363306minor)pagefaults 0swaps
[trusty-testing] Starting tests...
Found a test script: ./requirements/container-tests-provider-snappy
[trusty-testing] container-tests-provider-snappy: FAIL
[trusty-testing] stdout: https://paste.ubuntu.com/25024484/
[trusty-testing] stderr: https://paste.ubuntu.com/25024485/
[trusty-testing] (timing) Command exited with non-zero status 1
[trusty-testing] (timing) 6.30user 0.51system 0:09.41elapsed 72%CPU (0avgtext+0avgdata 44748maxresident)k
[trusty-testing] (timing) 0inputs+69784outputs (0major+37436minor)pagefaults 0swaps
[trusty-testing] Fixing file permissions in source directory
[trusty-testing] Destroying container
Name: trusty-testing
State: STOPPED
[xenial] starting container
[xenial] (timing) 0.09user 0.02system 0:04.28elapsed 2%CPU (0avgtext+0avgdata 16692maxresident)k
[xenial] (timing) 0inputs+32outputs (0major+5232minor)pagefaults 0swaps
[xenial] provisioning container
[xenial] (timing) 71.56user 19.38system 2:14.77elapsed 67%CPU (0avgtext+0avgdata 107732maxresident)k
[xenial] (timing) 24inputs+2564832outputs (2major+2359067minor)pagefaults 0swaps
[xenial-testing] Starting tests...
Found a test script: ./requirements/container-tests-provider-snappy
[xenial-testing] container-tests-provider-snappy: FAIL
[xenial-testing] stdout: https://paste.ubuntu.com/25024494/
[xenial-testing] stderr: https://paste.ubuntu.com/25024495/
[xenial-testing] (timing) Command exited with non-zero status 1
[xenial-testing] (timing) 6.31user 0.52system 0:09.14elapsed 74%CPU (0avgtext+0avgdata 42456maxresident)k
[xenial-testing] (timing) 0inputs+69784outputs (0major+37987minor)pagefaults 0swaps
[xenial-testing] Fixing file permissions in source directory
[xenial-testing] Destroying container
Name: xenial-testing
State: STOPPED

review: Needs Fixing

diff --git a/bin/ansi_parser b/bin/ansi_parser
deleted file mode 100755
index c7cf26c..0000000
--- a/bin/ansi_parser
+++ /dev/null
@@ -1,160 +0,0 @@
-#!/usr/bin/env python3
-import sys
-from optparse import OptionParser
-
-
-def parse_buffer(input):
-    output = [""]
-    row = -1
-    col = 0
-    escape = ""
-    saved = [0, 0]
-
-    for ch in input:
-        if ord(ch) == 27 or len(escape) > 0:
-            # On ESC
-            if chr(27) in [escape, ch]:
-                escape = ""
-                if ch == "c":
-                    output = [""]
-                    row = -1
-                    col = 0
-                    saved = [0, 0]
-                elif ch == "D":
-                    row += 1
-                    if row == 0:
-                        row = -1
-                        output.append("")
-                elif ch == "M":
-                    row -= 1
-                    if row < -len(output):
-                        output = [""] + output
-                elif ch == "7":
-                    saved = [row + len(output), col]
-                elif ch == "8":
-                    [row, col] = saved
-                    row -= len(output)
-                elif ord(ch) in [27, 91]:
-                    escape = ch
-                continue
-            # Just after hitting the extended ESC marker
-            elif escape == "[":
-                escape = ""
-
-            if ch in "0123456789;":
-                escape += ch
-                continue
-            elif ch in "Hf":
-                opts = escape.split(";") + ["", ""]
-                row = -len(output) + max(0, int("0" + opts[0]) - 1)
-                col = max(0, int("0" + opts[1]) - 1)
-            elif ch in "s":
-                saved = [row + len(output), col]
-            elif ch in "u":
-                [row, col] = saved
-                row -= len(output)
-            elif ch in "K":
-                if escape == "1":
-                    output[row] = " " * (col + 1) + output[row][col + 1:]
-                elif escape == "2":
-                    output[row] = ""
-                else:
-                    output[row] = output[row][:col]
-            elif ch in "J":
-                if len(escape) == 0:
-                    output = output[:row] + [""]
-                else:
-                    for i in range(row + len(output) + 1):
-                        output[i] = ""
-            elif ch in "A":
-                row -= max(1, int("0" + escape.split(";")[0]))
-                if row <= len(output):
-                    row = -len(output)
-            elif ch in "B":
-                row += max(1, int("0" + escape.split(";")[0]))
-                while row >= 0:
-                    output.append("")
-                    row -= 1
-            elif ch in "C":
-                col += max(1, int("0" + escape.split(";")[0]))
-            elif ch in "D":
-                col = max(0, col - max(1, int("0" + escape.split(";")[0])))
-
-            escape = ""
-            continue
-
-        # Control char
-        if ch in "\r\n\f\t\b":
-            if ch == "\r":
-                col = 0
-            if ch in "\n\f":
-                row += 1
-                if row == 0:
-                    row = -1
-                    output.append("")
-                col = 0
-            if ch == "\t":
-                col = (col + 8) & ~7
-            if ch == "\b":
-                col = max(0, col - 1)
-            continue
-
-        # Keep to ascii
-        if ord(ch) not in range(32, 127):
-            ch = "?"
-        if len(output[row]) < col:
-            output[row] += " " * (col - len(output[row]))
-        output[row] = output[row][:col] + ch + output[row][col + 1:]
-        col += 1
-
-    return "\n".join(output)
-
-
-def parse_file(file):
-    output = file.read()
-    return parse_buffer(output)
-
-
-def parse_filename(filename):
-    file = open(filename)
-    try:
-        output = parse_file(file)
-    finally:
-        file.close()
-
-    return output
-
-
-def main(args):
-    usage = "Usage: %prog [OPTIONS] [FILE...]"
-    parser = OptionParser(usage=usage)
-    parser.add_option("-o", "--output",
-        metavar="FILE",
-        help="File where to output the result.")
-    (options, args) = parser.parse_args(args)
-
-    # Write to stdout
-    if not options.output or options.output == "-":
-        output = sys.stdout
-
-    # Or from given option
-    else:
-        output = open(options.output, "w")
-
-    # Read from sdin
-    if not args or (len(args) == 1 and args[0] == "-"):
-        output.write(parse_file(sys.stdin))
-
-    # Or from filenames given as arguments
-    else:
-        for arg in args:
-            output.write(parse_filename(arg))
-
-    if options.output and options.output != "-":
-        output.close()
-
-    return 0
-
-
-if __name__ == "__main__":
-    sys.exit(main(sys.argv[1:]))
diff --git a/bin/bt_helper.py b/bin/bt_helper.py
deleted file mode 100644
index 6255cb7..0000000
--- a/bin/bt_helper.py
+++ /dev/null
@@ -1,291 +0,0 @@
-# Copyright 2016 Canonical Ltd.
-# Written by:
-#   Maciej Kisielewski <[e-mail]>
-#
-# This is free software: you can redistribute it and/or modify
-# it under the terms of the GNU General Public License version 3,
-# as published by the Free Software Foundation.
-#
-# This file is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License
-# along with this file.  If not, see <http://www.gnu.org/licenses/>.
-"""
-This module provides a set of abstractions to ease the process of automating
-typical Bluetooth task like scanning for devices and pairing with them.
-
-It talks with BlueZ stack using dbus.
-"""
-import logging
-
-import dbus
-import dbus.service
-import dbus.mainloop.glib
-from gi.repository import GObject
-
-logger = logging.getLogger(__file__)
-logger.addHandler(logging.StreamHandler())
-
-IFACE = 'org.bluez.Adapter1'
-ADAPTER_IFACE = 'org.bluez.Adapter1'
-DEVICE_IFACE = 'org.bluez.Device1'
-AGENT_IFACE = 'org.bluez.Agent1'
-
-dbus.mainloop.glib.DBusGMainLoop(set_as_default=True)
-
-# To get additional Bluetoot CoDs, check
-# https://www.bluetooth.com/specifications/assigned-numbers/baseband
-BT_ANY = 0
-BT_KEYBOARD = int('0x2540', 16)
-
-
-class BtManager:
-    """ Main point of contact with dbus factoring bt objects. """
-    def __init__(self, verbose=False):
-        if verbose:
-            logger.setLevel(logging.DEBUG)
-        self._bus = dbus.SystemBus()
-        self._bt_root = self._bus.get_object('org.bluez', '/')
-        self._manager = dbus.Interface(
-            self._bt_root, 'org.freedesktop.DBus.ObjectManager')
-        self._main_loop = GObject.MainLoop()
-        self._register_agent()
-
-    def _register_agent(self):
-        path = "/bt_helper/agent"
-        BtAgent(self._bus, path)
-        obj = self._bus.get_object('org.bluez', "/org/bluez")
-        agent_manager = dbus.Interface(obj, "org.bluez.AgentManager1")
-        agent_manager.RegisterAgent(path, 'NoInputNoOutput')
-        logger.info("Agent registered")
-
-    def _get_objects_by_iface(self, iface_name):
-        for path, ifaces in self._manager.GetManagedObjects().items():
-            if ifaces.get(iface_name):
-                yield self._bus.get_object('org.bluez', path)
-
-    def get_bt_adapters(self):
-        """Yield BtAdapter objects for each BT adapter found."""
-        for adapter in self._get_objects_by_iface(ADAPTER_IFACE):
-            yield BtAdapter(dbus.Interface(adapter, ADAPTER_IFACE), self)
-
-    def get_bt_devices(self, category=BT_ANY, filters={}):
-        """Yields BtDevice objects currently known to the system.
-
-        filters - specifies the characteristics of that a BT device must have
-        to be yielded. The keys of filters dictionary represent names of
-        parameters (as specified by the bluetooth DBus Api and represented by
-        DBus proxy object), and its values must match proxy values.
-        I.e. {'Paired': False}. For a full list of Parameters see:
-        http://git.kernel.org/cgit/bluetooth/bluez.git/tree/doc/device-api.txt
-
-        Note that this function returns objects corresponding to BT devices
-        that were seen last time scanning was done."""
-        for device in self._get_objects_by_iface(DEVICE_IFACE):
-            obj = self.get_object_by_path(device.object_path)[DEVICE_IFACE]
-            try:
-                if category != BT_ANY:
-                    if obj['Class'] != category:
-                        continue
-                rejected = False
-                for filter in filters:
-                    if obj[filter] != filters[filter]:
-                        rejected = True
-                        break
-                if rejected:
-                    continue
-                yield BtDevice(dbus.Interface(device, DEVICE_IFACE), self)
-            except KeyError as exc:
-                logger.info('Property %s not found on device %s',
-                            exc, device.object_path)
-                continue
-
-    def get_prop_iface(self, obj):
-        return dbus.Interface(self._bus.get_object(
-            'org.bluez', obj.object_path), 'org.freedesktop.DBus.Properties')
-
-    def get_object_by_path(self, path):
-        return self._manager.GetManagedObjects()[path]
-
-    def get_proxy_by_path(self, path):
-        return self._bus.get_object('org.bluez', path)
-
-    def wait(self):
-        self._main_loop.run()
-
-    def quit_loop(self):
-        self._main_loop.quit()
-
-    def ensure_adapters_powered(self):
-        for adapter in self.get_bt_adapters():
-            adapter.ensure_powered()
-
-    def scan(self, timeout=10):
-        """Scan for BT devices visible to all adapters.'"""
-        self._bus.add_signal_receiver(
-            interfaces_added,
-            dbus_interface="org.freedesktop.DBus.ObjectManager",
-            signal_name="InterfacesAdded")
-        self._bus.add_signal_receiver(
-            properties_changed,
-            dbus_interface="org.freedesktop.DBus.Properties",
-            signal_name="PropertiesChanged",
-            arg0="org.bluez.Device1",
-            path_keyword="path")
-        for adapter in self._get_objects_by_iface(ADAPTER_IFACE):
-            try:
-                dbus.Interface(adapter, ADAPTER_IFACE).StopDiscovery()
-            except dbus.exceptions.DBusException:
-                pass
-            dbus.Interface(adapter, ADAPTER_IFACE).StartDiscovery()
-        GObject.timeout_add_seconds(timeout, self._scan_timeout)
-        self._main_loop.run()
-
-    def get_devices(self, timeout=10, rescan=True):
-        """Scan for and list all devices visible to all adapters."""
-        if rescan:
-            self.scan(timeout)
-        return list(self.get_bt_devices())
-
-    def _scan_timeout(self):
-        for adapter in self._get_objects_by_iface(ADAPTER_IFACE):
-            dbus.Interface(adapter, ADAPTER_IFACE).StopDiscovery()
-        self._main_loop.quit()
-
-
-class BtAdapter:
-    def __init__(self, dbus_iface, bt_mgr):
-        self._if = dbus_iface
-        self._bt_mgr = bt_mgr
-        self._prop_if = bt_mgr.get_prop_iface(dbus_iface)
-
-    def set_bool_prop(self, prop_name, value):
-        self._prop_if.Set(IFACE, prop_name, dbus.Boolean(value))
-
-    def ensure_powered(self):
-        """Turn the adapter on, and do nothing if already on."""
-        powered = self._prop_if.Get(IFACE, 'Powered')
-        logger.info('Powering on {}'.format(
-            self._if.object_path.split('/')[-1]))
-        if powered:
-            logger.info('Device already powered')
-            return
-        try:
-            self.set_bool_prop('Powered', True)
-            logger.info('Powered on')
-        except Exception as exc:
-            logging.error('Failed to power on - {}'.format(
-                exc.get_dbus_message()))
-
-
-class BtDevice:
-    def __init__(self, dbus_iface, bt_mgr):
-        self._if = dbus_iface
-        self._obj = bt_mgr.get_object_by_path(
-            self._if.object_path)[DEVICE_IFACE]
-        self._bt_mgr = bt_mgr
-        self._prop_if = bt_mgr.get_prop_iface(dbus_iface)
-
-    def __str__(self):
-        return "{} ({})".format(self.name, self.address)
-
-    def __repr__(self):
-        return "<BtDevice name:{}, address:{}>".format(self.name, self.address)
-
-    def pair(self):
-        """Pair the device.
-
-        This function will try pairing with the device and block until device
-        is paired, error occured or default timeout elapsed (whichever comes
-        first).
-        """
-        self._prop_if.Set(DEVICE_IFACE, 'Trusted', True)
-        self._if.Pair(
-            reply_handler=self._pair_ok, error_handler=self._pair_error)
-        self._bt_mgr.wait()
-        try:
-            self._if.Connect()
-        except dbus.exceptions.DBusException as exc:
-            logging.error('Failed to connect - {}'.format(
-                exc.get_dbus_message()))
-
-    def unpair(self):
-        self._if.Disconnect()
-        adapter = self._bt_mgr.get_proxy_by_path(self._obj['Adapter'])
-        dbus.Interface(adapter, ADAPTER_IFACE).RemoveDevice(self._if)
-
-    @property
-    def name(self):
-        return self._obj.get('Name', '<Unnamed>')
-
-    @property
-    def address(self):
-        return self._obj['Address']
-
-    @property
-    def rssi(self):
-        return self._obj.get('RSSI', None)
-
-    def _pair_ok(self):
-        logger.info('%s successfully paired', self.name)
-        self._bt_mgr.quit_loop()
-
-    def _pair_error(self, error):
-        logger.warning('Pairing of %s device failed. %s', self.name, error)
-        self._bt_mgr.quit_loop()
-
-
-class Rejected(dbus.DBusException):
-    _dbus_error_name = "org.bluez.Error.Rejected"
-
-
-class BtAgent(dbus.service.Object):
-    """Agent authenticating everything that is possible."""
-    @dbus.service.method(AGENT_IFACE, in_signature="os", out_signature="")
-    def AuthorizeService(self, device, uuid):
-        logger.info("AuthorizeService (%s, %s)", device, uuid)
-
-    @dbus.service.method(AGENT_IFACE, in_signature="o", out_signature="u")
-    def RequestPasskey(self, device):
-        logger.info("RequestPasskey (%s)", device)
-        passkey = input("Enter passkey: ")
-        return dbus.UInt32(passkey)
-
-    @dbus.service.method(AGENT_IFACE, in_signature="o", out_signature="s")
-    def RequestPinCode(self, device):
-        logger.info("RequestPinCode (%s)", device)
-        return input("Enter PIN Code: ")
-
-    @dbus.service.method(AGENT_IFACE, in_signature="ouq", out_signature="")
-    def DisplayPasskey(self, device, passkey, entered):
-        print("DisplayPasskey (%s, %06u entered %u)" %
-              (device, passkey, entered), flush=True)
-
-    @dbus.service.method(AGENT_IFACE, in_signature="os", out_signature="")
-    def DisplayPinCode(self, device, pincode):
-        logger.info("DisplayPinCode (%s, %s)", device, pincode)
-        print('Type following pin on your device: {} and press ENTER'.format(
-            pincode), flush=True)
-
-    @dbus.service.method(AGENT_IFACE, in_signature="ou", out_signature="")
-    def RequestConfirmation(self, device, passkey):
-        logger.info("RequestConfirmation (%s, %06d)", device, passkey)
-
-    @dbus.service.method(AGENT_IFACE, in_signature="o", out_signature="")
-    def RequestAuthorization(self, device):
-        logger.info("RequestAuthorization (%s)", device)
-
-    @dbus.service.method(AGENT_IFACE, in_signature="", out_signature="")
-    def Cancel(self):
-        logger.info("Cancelled")
-
-
-def properties_changed(interface, changed, invalidated, path):
-    logger.info('Property changed for device @ %s. Change: %s', path, changed)
-
-
-def interfaces_added(path, interfaces):
-    logger.info('Added new bt interfaces: %s @ %s', interfaces, path)
diff --git a/bin/cpu_offlining b/bin/cpu_offlining
deleted file mode 100755
index 0e88af1..0000000
--- a/bin/cpu_offlining
+++ /dev/null
@@ -1,46 +0,0 @@
-#!/bin/bash
-
-echo "Beginning CPU Offlining Test" 1>&2
-
-result=0
-cpu_count=0
-
-# Turn CPU cores off
-for cpu_num in `ls /sys/devices/system/cpu | grep -o cpu[0-9]*`; do
-    if [ -f /sys/devices/system/cpu/$cpu_num/online ]; then
-        if [ "$cpu_num" != "cpu0" ]; then
-            ((cpu_count++))
-            echo "Offlining $cpu_num" 1>&2
-            echo 0 > /sys/devices/system/cpu/$cpu_num/online
-
-            grep -w -i -q $cpu_num /proc/interrupts
-            if [ $? -eq 0 ]; then
-                echo "ERROR: Failed to offline $cpu_num" 1>&2
-                result=1
-            fi
-        fi
-    fi
-done
-
-# Back on again
-for cpu_num in `ls /sys/devices/system/cpu | grep -o cpu[0-9]*`; do
-    if [ -f /sys/devices/system/cpu/$cpu_num/online ]; then
-        if [ "$cpu_num" != "cpu0" ]; then        
-            echo "Onlining $cpu_num" 1>&2
-            echo 1 > /sys/devices/system/cpu/$cpu_num/online
-            grep -w -i -q $cpu_num /proc/interrupts
-            if [ $? -eq 1 ]; then
-                echo "ERROR: Failed to online $cpu_num" 1>&2
-                result=1
-            fi
-        fi
-    fi
-done
-
-if [ $result -eq 0 ]; then
-    echo "Successfully turned $cpu_count cores off and back on"
-else
-    echo "Error with offlining one or more cores.  CPU offline may not work if this is an ARM system." 1>&2
-fi
-
-exit $result
diff --git a/bin/cpu_topology b/bin/cpu_topology
deleted file mode 100755
index 78317b7..0000000
--- a/bin/cpu_topology
+++ /dev/null
@@ -1,115 +0,0 @@
-#!/usr/bin/env python3
-'''
-cpu_topology
-Written by Jeffrey Lane <[e-mail]>
-'''
-import sys
-import os
-import re
-
-
-class proc_cpuinfo():
-    '''
-    Class to get and handle information from /proc/cpuinfo
-    Creates a dictionary of data gleaned from that file.
-    '''
-    def __init__(self):
-        self.cpuinfo = {}
-        cpu_fh = open('/proc/cpuinfo', 'r')
-        try:
-            temp = cpu_fh.readlines()
-        finally:
-            cpu_fh.close()
-
-        r_s390 = re.compile("processor [0-9]")
-        r_x86 = re.compile("processor\s+:")
-        for i in temp:
-            # Handle s390 first
-            if r_s390.match(i):
-                cpu_num = i.split(':')[0].split()[1].strip()
-                key = 'cpu' + cpu_num
-                self.cpuinfo[key] = {'core_id': cpu_num,
-                                     'physical_package_id': cpu_num}
-            elif r_x86.match(i):
-                key = 'cpu' + (i.split(':')[1].strip())
-                self.cpuinfo[key] = {'core_id': '', 'physical_package_id': ''}
-            elif i.startswith('core id'):
-                self.cpuinfo[key].update({'core_id': i.split(':')[1].strip()})
-            elif i.startswith('physical id'):
-                self.cpuinfo[key].update({'physical_package_id':
-                                          i.split(':')[1].strip()})
-            else:
-                continue
-
-
-class sysfs_cpu():
-    '''
-    Class to get and handle information from sysfs as relates to CPU topology
-    Creates an informational class to present information on various CPUs
-    '''
-
-    def __init__(self, proc):
-        self.syscpu = {}
-        self.path = '/sys/devices/system/cpu/' + proc + '/topology'
-        items = ['core_id', 'physical_package_id']
-        for i in items:
-            try:
-                syscpu_fh = open(os.path.join(self.path, i), 'r')
-            except OSError as e:
-                print("ERROR: %s" % e)
-                sys.exit(1)
-            else:
-                self.syscpu[i] = syscpu_fh.readline().strip()
-                syscpu_fh.close()
-
-
-def compare(proc_cpu, sys_cpu):
-    cpu_map = {}
-    '''
-    If there is only 1 CPU the test don't look for core_id
-    and physical_package_id because those information are absent in
-    /proc/cpuinfo on singlecore system
-    '''
-    for key in proc_cpu.keys():
-        if 'cpu1' not in proc_cpu:
-            cpu_map[key] = True
-        else:
-            for subkey in proc_cpu[key].keys():
-                if proc_cpu[key][subkey] == sys_cpu[key][subkey]:
-                    cpu_map[key] = True
-                else:
-                    cpu_map[key] = False
-    return cpu_map
-
-
-def main():
-    cpuinfo = proc_cpuinfo()
-    sys_cpu = {}
-    keys = cpuinfo.cpuinfo.keys()
-    for k in keys:
-        sys_cpu[k] = sysfs_cpu(k).syscpu
-    cpu_map = compare(cpuinfo.cpuinfo, sys_cpu)
-    if False in cpu_map.values() or len(cpu_map) < 1:
-        print("FAIL: CPU Topology is incorrect", file=sys.stderr)
-        print("-" * 52, file=sys.stderr)
-        print("{0}{1}".format("/proc/cpuinfo".center(30), "sysfs".center(25)),
-              file=sys.stderr)
-        print("{0}{1}{2}{3}{1}{2}".format(
-              "CPU".center(6),
-              "Physical ID".center(13),
-              "Core ID".center(9),
-              "|".center(3)), file=sys.stderr)
-        for key in sorted(sys_cpu.keys()):
-            print("{0}{1}{2}{3}{4}{5}".format(
-                  key.center(6),
-                  cpuinfo.cpuinfo[key]['physical_package_id'].center(13),
-                  cpuinfo.cpuinfo[key]['core_id'].center(9),
-                  "|".center(3),
-                  sys_cpu[key]['physical_package_id'].center(13),
-                  sys_cpu[key]['core_id'].center(9)), file=sys.stderr)
-        return 1
-    else:
-        return 0
-
-if __name__ == '__main__':
-    sys.exit(main())
diff --git a/bin/disk_info b/bin/disk_info
deleted file mode 100755
index 5efedbd..0000000
--- a/bin/disk_info
+++ /dev/null
@@ -1,81 +0,0 @@
-#!/usr/bin/env python3
-#
-# This file is part of Checkbox.
-#
-# Copyright (C) 2010-2013 by Cloud Computing Center for Mobile Applications
-# Industrial Technology Research Institute
-# Copyright 2016 Canonical Ltd.
-#
-# Authors:
-#   Nelson Chu <[e-mail]>
-#   Jeff Lane <[e-mail]>
-#   Sylvain Pineau <[e-mail]>
-#
-# Checkbox is free software: you can redistribute it and/or modify
-# it under the terms of the GNU General Public License version 3,
-# as published by the Free Software Foundation.
-#
-# Checkbox is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License
-# along with Checkbox.  If not, see <http://www.gnu.org/licenses/>.
-"""disk_info utility."""
-
-import re
-import sys
-from subprocess import check_output, CalledProcessError
-
-from checkbox_support.parsers.udevadm import find_pkname_is_root_mountpoint
-
-
-def main():
-    """
-    disk_info.
-
-    Uses lsblk to gather information about disks seen by the OS.
-    Outputs kernel name, model and size data
-    """
-    pattern = re.compile('KNAME="(?P<KNAME>.*)" '
-                         'TYPE="(?P<TYPE>.*)" '
-                         'SIZE="(?P<SIZE>.*)" '
-                         'MODEL="(?P<MODEL>.*)" '
-                         'MOUNTPOINT="(?P<MOUNTPOINT>.*)"')
-    try:
-        lsblk = check_output(["lsblk", "-i", "-n", "-P", "-o",
-                              "KNAME,TYPE,SIZE,MODEL,MOUNTPOINT"],
-                             universal_newlines=True)
-    except CalledProcessError as e:
-        sys.exit(e)
-
-    disks = 0
-    for line in lsblk.splitlines():
-        m = pattern.match(line)
-        if not m or m.group('TYPE') != 'disk':
-            continue
-        # Only consider MMC block devices if one of their mounted partitions is
-        # root (/)
-        if (
-            m.group('KNAME').startswith('mmcblk') and not
-            find_pkname_is_root_mountpoint(m.group('KNAME'), lsblk)
-        ):
-            continue
-        disks += 1
-        model = m.group('MODEL')
-        if not model:
-            model = 'Unknown'
-        print("Name: /dev/{}".format(m.group('KNAME')))
-        print("\t{:7}\t{}".format('Model:', model))
-        print("\t{:7}\t{}".format('Size:', m.group('SIZE')))
-
-    if not disks:
-        print("No disk information discovered.")
-        return 10
-
-    return 0
-
-
-if __name__ == '__main__':
-    sys.exit(main())
diff --git a/bin/disk_read_performance_test b/bin/disk_read_performance_test
deleted file mode 100755
index e2a219d..0000000
--- a/bin/disk_read_performance_test
+++ /dev/null
@@ -1,74 +0,0 @@
-#!/bin/bash
-#
-# Verify that disk storage performs at or above baseline performance
-#
-
-#Default to a lower bound of 15 MB/s
-DEFAULT_BUF_READ=15
-
-for disk in $@; do
-
-  echo "Beginning $0 test for $disk"
-  echo "---------------------------------------------------"
-
-  disk_type=`udevadm info --name /dev/$disk --query property | grep "ID_BUS" | awk '{gsub(/ID_BUS=/," ")}{printf $1}'`
-  dev_path=`udevadm info --name /dev/$disk --query property | grep "DEVPATH" | awk '{gsub(/DEVPATH=/," ")}{printf $1}'`
-  echo "INFO: $disk type is $disk_type"
-
-  case $disk_type in
-    "usb" ) 
-            #Custom metrics are guesstimates for now...
-            MIN_BUF_READ=7
-            
-            # Increase MIN_BUF_READ if a USB3 device is plugged in a USB3 hub port
-            if  [[ $dev_path =~ ((.*usb[0-9]+).*\/)[0-9]-[0-9\.:\-]+\/.* ]]; then
-                device_version=`cat '/sys/'${BASH_REMATCH[1]}'/version'`
-                hub_port_version=`cat '/sys/'${BASH_REMATCH[2]}'/version'`
-                if [ $(echo "$device_version >= 3.00"|bc -l) -eq 1 -a $(echo "$hub_port_version >= 3.00"|bc -l) -eq 1 ]; then
-                    MIN_BUF_READ=80
-                fi
-            fi
-            ;;
-    "ide" ) MIN_BUF_READ=40;;
-    *     ) MIN_BUF_READ=$DEFAULT_BUF_READ;;
-  esac
-  echo "INFO: $disk_type: Using $MIN_BUF_READ MB/sec as the minimum throughput speed"
-
-  max_speed=0
-  echo ""
-  echo "Beginning hdparm timing runs"
-  echo "---------------------------------------------------"
-
-  for iteration in `seq 1 10`; do
-    speed=`hdparm -t /dev/$disk 2>/dev/null | grep "Timing buffered disk reads" | awk -F"=" '{print $2}' | awk '{print $1}'`
-    echo "INFO: Iteration $iteration: Detected speed is $speed MB/sec"
-
-    if [ -z "$speed" ]; then
-      echo "WARNING: Device $disk is too small! Aborting test."
-      exit 0
-    fi
-
-    speed=${speed/.*}
-    if [ $speed -gt $max_speed ]; then
-      max_speed=$speed
-    fi
-  done
-  echo "INFO: Maximum detected speed is $max_speed MB/sec"
-  echo "---------------------------------------------------"
-  echo ""
-  result=0
-  if [ $max_speed -gt $MIN_BUF_READ ]; then
-    echo "PASS: $disk Max Speed of $max_speed MB/sec is faster than Minimum Buffer Read Speed of $MIN_BUF_READ MB/sec"
-  else
-    echo "FAIL: $disk Max Speed of $max_speed MB/sec is slower than Minimum Buffer Read Speed of $MIN_BUF_READ MB/sec"
-    result=1
-  fi
-done
-
-if [ $result -gt 0 ]; then
-  echo "WARNING: One or more disks failed testing!"
-  exit 1
-else
-  echo "All devices passed testing!"
-  exit 0
-fi
diff --git a/bin/disk_stats_test b/bin/disk_stats_test
deleted file mode 100755
index 851dff7..0000000
--- a/bin/disk_stats_test
+++ /dev/null
@@ -1,68 +0,0 @@
-#!/bin/bash
-
-#Simple script to gather some data about a disk to verify it's seen by the OS
-#and is properly represented.  Defaults to sda if not passed a disk at run time
-
-DISK="sda"
-
-check_return_code() {
-    if [ "${1}" -ne "0" ]; then
-        echo "ERROR: retval ${1} : ${2}" >&2
-        exit ${1}
-    fi
-}
-
-if [[ "$1" != '' ]]; then
-    DISK="$1"
-fi
-
-#Get some baseline stats for use later
-echo "Getting baseline stats"
-PROC_STAT_BEGIN=`grep -m 1 $DISK /proc/diskstats`
-SYS_STAT_BEGIN=`cat /sys/block/$DISK/stat`
-
-#Generate some disk activity using hdparm -t
-echo "Generating some disk activity"
-hdparm -t "/dev/$DISK" 2&> /dev/null
-
-#Sleep 5 to let the stats files catch up
-sleep 5
-
-#Check /proc/partitions, exit with fail if disk isn't found
-echo "Checking /proc/partitions"
-grep -q $DISK /proc/partitions
-check_return_code $? "Disk $DISK not found in /proc/partitions"
-
-#Next, check /proc/diskstats
-echo "Checking /proc/diskstats"
-grep -q -m 1 $DISK /proc/diskstats
-check_return_code $? "Disk $DISK not found in /proc/diskstats"
-
-#Verify the disk shows up in /sys/block/
-echo "Checking /sys/block"
-ls /sys/block | grep -q $DISK
-check_return_code $? "Disk $DISK not found in /sys/block"
-
-#Verify there are stats in /sys/block/$DISK/stat
-echo "Checking /sys/block/$DISK/stat"
-[[ -s "/sys/block/$DISK/stat" ]]
-check_return_code $? "stat is either empty or nonexistant in /sys/block/$DISK/"
-
-#Sleep 5 to let the stats files catch up
-sleep 5
-
-#Make sure the stats have changed:
-echo "Getting ending stats"
-PROC_STAT_END=`grep -m 1 $DISK /proc/diskstats`
-SYS_STAT_END=`cat /sys/block/$DISK/stat`
-
-echo "Checking /proc/diskstats for changes"
-[[ "$PROC_STAT_BEGIN" != "$PROC_STAT_END" ]]
-check_return_code $? "Stats in /proc/diskstats did not change"
-
-echo "Checking /sys/block/$DISK/stat for changes"
-[[ "$SYS_STAT_BEGIN" != "$SYS_STAT_END" ]]
-check_return_code $? "Stats in /sys/block/$DISK/stat did not change"
-
-echo "PASS: Finished testing stats for $DISK"
-exit 0
diff --git a/bin/disk_stress_ng b/bin/disk_stress_ng
deleted file mode 100755
index 8445488..0000000
--- a/bin/disk_stress_ng
+++ /dev/null
@@ -1,242 +0,0 @@
-#!/bin/bash
-
-# Script to disk stress tests using stress-ng
-#
-# Copyright (c) 2016 Canonical Ltd.
-#
-# Authors
-#   Rod Smith <[e-mail]>
-#
-# This program is free software: you can redistribute it and/or modify
-# it under the terms of the GNU General Public License version 3,
-# as published by the Free Software Foundation.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License
-# along with this program.  If not, see <http://www.gnu.org/licenses/>.
-#
-# The purpose of this script is to run disk stress tests using the
-# stress-ng program.
-#
-# Usage:
-#   disk_stress_ng [ <device-filename> ]
-#                  [ --base-time <time-in-seconds> ]
-#                  [ --really-run ]
-#
-# Parameters:
-#  --disk-device -- This is the WHOLE-DISK device filename WITHOUT "/dev/"
-#                   (e.g., sda). The script finds a filesystem on that
-#                   device, mounts it if necessary, and runs the tests on
-#                   that mounted filesystem.
-# Test with iostat
-
-get_params() {
-    disk_device="/dev/sda"
-    short_device="sda"
-    base_time="240"
-    really_run="N"
-    while [ $# -gt 0 ] ; do
-        case $1 in
-            --base-time) base_time="$2"
-                shift
-                ;;
-            --really-run) really_run="Y"
-                ;;
-            *) disk_device="/dev/$1"
-                disk_device=`echo $disk_device | sed "s/\/dev\/\/dev/\/dev/g"`
-                short_device=$(echo $disk_device | sed "s/\/dev//g")
-                if [ ! -b $disk_device ] ; then
-                    echo "Unknown block device \"$disk_device\""
-                    echo "Usage: $0 [ --base-time <time-in-seconds> ] [ --really-run ]"
-                    echo "           [ device-file ]"
-                    exit 1
-                fi
-                ;;
-        esac
-        shift
-    done
-    mounted_part="N"
-} # get_params()
-
-
-# Find the largest partition that holds a supported filesystem on $disk_device.
-# Output:
-#   $largest_part -- Device filename of largest qualifying partition
-#   $largest_size -- Size of largest qualifying partition
-#   $largest_fs -- Filesystem (ext4, etc.) used on largest qualifying partition
-#   $unsupported_fs -- Empty or contains name of unsupported filesystem found on disk
-find_largest_partition() {
-    largest_part=""
-    largest_size=0
-    partitions=$(lsblk -b -l -n -o NAME,SIZE,TYPE,MOUNTPOINT $disk_device | grep part | tr -s " ")
-    unsupported_fs=""
-    for partition in $(echo "$partitions" | cut -d " " -f 1) ; do
-        part_size=$(echo "$partitions" | grep "$partition " | cut -d " " -f 2)
-        local blkid_info=$(blkid -s TYPE /dev/$partition | grep -E ext2\|ext3\|ext4\|xfs\|jfs\|btrfs)
-        if [ "$part_size" -gt "$largest_size" ] && [ -n "$blkid_info" ]  ; then
-            largest_size=$part_size
-            largest_part="/dev/$partition"
-            largest_fs=$(blkid -s TYPE "/dev/$partition" | cut -d "=" -f 2)
-        fi
-        local blkid_info=$(blkid -s TYPE /dev/$partition | grep -E ntfs\|vfat\|hfs)
-        if [ -n "$blkid_info" ]  ; then
-            # If there's an NTFS, HFS+, or FAT filesystem on the disk make note of it....
-            unsupported_fs=$(blkid -s TYPE "/dev/$partition" | cut -d "=" -f 2)
-        fi
-    done
-} # find_largest_partition()
-
-# Find the largest filesystem on $disk_device. If that partition is not
-# already mounted, try to mount it.
-# Output:
-#   $test_dir -- Directory in which tests will occur
-#   $mount_point -- Location where filesystem is mounted
-#   $mounted_part -- Sets to "Y" if script mounted partition
-#   $made_mountpoint -- Sets to "Y" if script created the mount point
-mount_filesystem() {
-    test_dir="/tmp/disk_stress_ng"
-    if [ -b $disk_device ]
-    then
-        echo "$disk_device is a block device"
-
-        #Add a check for warnings
-        WARN=$(parted -s ${disk_device} print | grep "^Warning.*${disk}.*[Rr]ead-only" 2>&1)
-        if [[ $? == 0 ]]
-        then
-            echo "Warning found in parted output:"
-            echo $WARN
-            echo "Aborting Test"
-            exit 1
-        fi
-    else
-        echo "$disk_device is not a block device! Aborting!"
-        exit 1
-    fi
-
-    find_largest_partition
-
-    if [ -n "$largest_part" ] ; then
-        echo "Found largest partition: \"$largest_part\""
-        mount_point=$(df | grep "$largest_part " | tr -s " " | cut -d " " -f 6)
-        if [ "$mount_point" == "" ] && [ "$really_run" == "Y" ] ; then
-            disk_device=$(echo $disk_device | sed "s/\/dev\/\/dev/\/dev/g")
-            mount_point="/mnt$short_device"
-            echo "No partition is mounted from $disk_device; attempting to mount one...."
-            if [ ! -d $mount_point ] ; then
-                mkdir -p "$mount_point"
-                made_mountpoint="Y"
-            fi
-            mount "$largest_part" "$mount_point"
-            mounted_part="Y"
-        fi
-        if [ "$mount_point" == "/" ] ; then
-            test_dir="/tmp/disk_stress_ng"
-        else
-            test_dir="$mount_point/tmp/disk_stress_ng"
-        fi
-        echo "Test will use $largest_part, mounted at \"$mount_point\", using $largest_fs"
-    else
-        echo "There appears to be no partition with a suitable filesystem"
-        echo "on $disk_device; please create a suitable partition and re-run"
-        echo "this test."
-        if [ -n "unsupported_fs" ] ; then
-            echo "NOTE: A filesystem of type $unsupported_fs was found, but is not supported"
-            echo "by this test. A Linux-native filesystem (ext2/3/4fs, XFS, JFS, or Btrfs)"
-            echo "is required."
-        fi
-        exit 1
-    fi
-} # mount_filesystem()
-
-
-# Run an individual stressor
-# Input:
-#   $1 = stressor name (e.g., copyfile, dentry)
-#   $2 = run time
-# Output:
-#   had_error -- sets to "1" if an error occurred
-run_stressor() {
-    local runtime="$2"
-    # Multiply runtime by 5; will forcefully terminate if stress-ng
-    # fails to return in that time.
-    end_time=$((runtime*5))
-    echo "Running stress-ng $1 stressor for $2 seconds...."
-    # Use "timeout" command to launch stress-ng, to catch it should it go into
-    # la-la land
-    timeout -s 9 $end_time stress-ng --aggressive --verify --timeout $runtime \
-            --temp-path $test_dir --$1 0
-    return_code="$?"
-    echo "return_code is $return_code"
-    if [ "$return_code" != "0" ] ; then
-        had_error=1
-        echo "*****************************************************************"
-        if [ $return_code = "137" ] ; then
-            echo "** stress-ng disk test timed out and was forcefully terminated!"
-        else
-            echo "** Error $return_code reported on stressor $stressor!)"
-        fi
-        echo "*****************************************************************"
-        had_error=1
-        result=$return_code
-    fi
-} # run_stressor()
-
-
-#
-# Main program body....
-#
-
-
-get_params "$@"
-mount_filesystem
-echo "test_dir is $test_dir"
-
-had_error=0
-
-# Tests Colin said to try but that aren't present as of standard stress-ng
-# in Ubuntu 16.04:
-#
-#  "chown" "copyfile" "ioprio" "locka" "lockofd" "madvise" "msync" "seal"
-#
-# TODO: Consider adding these tests for Ubuntu 18.04, or ealier with an
-# updated stress-ng in the certification PPA....
-
-disk_stressors=("aio" "aiol" "chdir" "chmod" "dentry"  "dir" "fallocate" \
-                "fiemap" "filename" "flock" "fstat" "hdd"  "lease" "lockf" \
-                "mknod" "readahead" "seek" "sync-file" "xattr")
-
-total_runtime=$((${#disk_stressors[@]}*$base_time))
-
-echo "Estimated total run time is $total_runtime seconds"
-echo ""
-
-if [ "$really_run" == "Y" ] ; then
-    mkdir -p "$test_dir"
-    for stressor in ${disk_stressors[@]}; do
-        run_stressor $stressor $base_time
-    done
-    rm -rf "$test_dir"
-    if [ "$mounted_part" == "Y" ] ; then
-        umount "$mount_point"
-        if [ "$made_mountpoint" == "Y" ] ; then
-            rmdir "$mount_point"
-        fi
-    fi
-else
-    echo "To actually run tests, pass the --really-run option."
-    echo "Script is now terminating...."
-    exit 1
-fi
-
-echo "*******************************************************************"
-if [ $had_error = "0" ] ; then
-    echo "** stress-ng disk test passed!"
-else
-    echo "** stress-ng disk test failed; most recent error was $result"
-fi
-echo "*******************************************************************"
-exit $result
diff --git a/bin/fwts_test b/bin/fwts_test
deleted file mode 100755
index 7672a2b..0000000
--- a/bin/fwts_test
+++ /dev/null
@@ -1,492 +0,0 @@
-#! /usr/bin/python3
-
-import sys
-import re
-from time import time, sleep
-from argparse import ArgumentParser, RawTextHelpFormatter, REMAINDER
-from subprocess import Popen, PIPE
-from syslog import *
-from shutil import which
-import os
-
-# These tests require user interaction and need either special handling
-# or skipping altogether (right now, we skip them but they're kept here
-# in case we figure out a way to present the interaction to the user).
-INTERACTIVE_TESTS = ['ac_adapter',
-                     'battery',
-                     'hotkey',
-                     'power_button',
-                     'brightness',
-                     'lid']
-# These are performed on QA certification runs
-QA_TESTS = ['acpitests',
-            'acpidump',
-            'acpitables',
-            'apicinstance',
-            'aspm',
-            'bios32',
-            'dmicheck',
-            'ebda',
-            'mpcheck',
-            'msr',
-            'nx',
-            'version']
-# These are advanced tests that shouldn't affect certification status
-NON_CERT_TESTS = ['bios_info',
-                  'cmosdump',
-                  'cpufreq',
-                  'crs',
-                  'crsdump',
-                  'csm',
-                  'ebdadump',
-                  'fan',
-                  'gpedump',
-                  'hda_audio',
-                  'maxfreq',
-                  'maxreadreq',
-                  'memmapdump',
-                  'microcode',
-                  'mpdump',
-                  'os2gap',
-                  'osilinux',
-                  'pciirq',
-                  'plddump',
-                  'pnp',
-                  'prsdump',
-                  'romdump',
-                  'securebootcert',
-                  'syntaxcheck',
-                  'uefidump',
-                  'uefirtmisc',
-                  'uefirttime',
-                  'uefirtvariable',
-                  'uefivarinfo',
-                  'wakealarm'
-                  ]
-# The following tests will record logs in a separate file for the HWE team
-HWE_TESTS = ['mtrr',
-             'virt',
-             'apicedge',
-             'klog',
-             'oops',
-             'uefibootpath']
-CERT_TESTS = sorted(QA_TESTS + HWE_TESTS)
-TESTS = sorted(QA_TESTS + NON_CERT_TESTS + HWE_TESTS)
-
-
-def get_sleep_times(start_marker, end_marker, sleep_time, resume_time):
-    logfile = '/var/log/syslog'
-    log_fh = open(logfile, 'r', encoding='UTF-8')
-    line = ''
-    run = 'FAIL'
-    sleep_start_time = 0.0
-    sleep_end_time = 0.0
-    resume_start_time = 0.0
-    resume_end_time = 0.0
-
-    while start_marker not in line:
-        try:
-            line = log_fh.readline()
-        except UnicodeDecodeError:
-            continue
-        if start_marker in line:
-            loglist = log_fh.readlines()
-
-    for idx in range(0, len(loglist)):
-        if 'PM: Syncing filesystems' in loglist[idx]:
-            sleep_start_time = re.split('[\[\]]', loglist[idx])[1].strip()
-        if 'ACPI: Low-level resume complete' in loglist[idx]:
-            sleep_end_time = re.split('[\[\]]', loglist[idx - 1])[1].strip()
-            resume_start_time = re.split('[\[\]]', loglist[idx])[1].strip()
-            idx += 1
-        if 'Restarting tasks' in loglist[idx]:
-            resume_end_time = re.split('[\[\]]', loglist[idx])[1].strip()
-        if end_marker in loglist[idx]:
-            run = 'PASS'
-            break
-
-    sleep_elapsed = float(sleep_end_time) - float(sleep_start_time)
-    resume_elapsed = float(resume_end_time) - float(resume_start_time)
-    return (run, sleep_elapsed, resume_elapsed)
-
-
-def average_times(runs):
-    sleep_total = 0.0
-    resume_total = 0.0
-    run_count = 0
-    for run in runs.keys():
-        run_count += 1
-        sleep_total += runs[run][1]
-        resume_total += runs[run][2]
-    sleep_avg = sleep_total / run_count
-    resume_avg = resume_total / run_count
-    print('Average time to sleep: %0.5f' % sleep_avg)
-    print('Average time to resume: %0.5f' % resume_avg)
-
-
-def fix_sleep_args(args):
-    new_args = []
-    for arg in args:
-        if "=" in arg:
-            new_args.extend(arg.split('='))
-        else:
-            new_args.append(arg)
-    return new_args
-
-
-def detect_progress_indicator():
-    # Return a command suitable for piping progress information to its
-    # stdin (invoked via Popen), in list format.
-    # Return zenity if installed and DISPLAY (--auto-close)
-    # return dialog if installed and no DISPLAY (width height)
-    display = os.environ.get('DISPLAY')
-    if display and which('zenity'):
-        return ["zenity", "--progress", "--text", "Progress", "--auto-close"]
-    if not display and which('dialog'):
-        return ["dialog", "--gauge", "Progress", "20", "70"]
-    # Return empty list if no progress indicator is to be used
-    return []
-
-
-def main():
-    description_text = 'Tests the system BIOS using the Firmware Test Suite'
-    epilog_text = ('To perform sleep testing, you will need at least some of '
-                   'the following options: \n'
-                   's3 or s4: tells fwts which type of sleep to perform.\n'
-                   '--s3-delay-delta\n'
-                   '--s3-device-check\n'
-                   '--s3-device-check-delay\n'
-                   '--s3-hybrid-sleep\n'
-                   '--s3-max-delay\n'
-                   '--s3-min-delay\n'
-                   '--s3-multiple\n'
-                   '--s3-quirks\n'
-                   '--s3-sleep-delay\n'
-                   '--s3power-sleep-delay\n\n'
-                   'Example: fwts_test --sleep s3 --s3-min-delay 30 '
-                   '--s3-multiple 10 --s3-device-check\n\n'
-                   'For further help with sleep options:\n'
-                   'fwts_test --fwts-help')
-    parser = ArgumentParser(description=description_text,
-                            epilog=epilog_text,
-                            formatter_class=RawTextHelpFormatter)
-    parser.add_argument('-l', '--log',
-                        default='/tmp/fwts_results.log',
-                        help=('Specify the location and name '
-                              'of the log file.\n'
-                              '[Default: %(default)s]'))
-    parser.add_argument('-f', '--fail-level',
-                        default='high',
-                        choices=['critical', 'high', 'medium',
-                                 'low', 'none', 'aborted'],
-                        help=('Specify the FWTS failure level that will '
-                              'trigger this script to return a failing exit '
-                              'code. For example, if you chose "critical" as '
-                              'the fail-level, this wrapper will NOT return '
-                              'a failing exit code unless FWTS reports a '
-                              'test as FAILED_CRITICAL. You will still be '
-                              'notified of all FWTS test failures. '
-                              '[Default level: %(default)s]'))
-    sleep_args = parser.add_argument_group('Sleep Options',
-                                           ('The following arguments are to '
-                                            'only be used with the '
-                                            '--sleep test option'))
-    sleep_args.add_argument('--sleep-time',
-                            dest='sleep_time',
-                            action='store',
-                            help=('The max time in seconds that a system '
-                                  'should take\nto completely enter sleep. '
-                                  'Anything more than this\ntime will cause '
-                                  'that test iteration to fail.\n'
-                                  '[Default: 10s]'))
-    sleep_args.add_argument('--resume-time',
-                            dest='resume_time',
-                            action='store',
-                            help=('Same as --sleep-time, except this applies '
-                                  'to the\ntime it takes a system to fully '
-                                  'wake from sleep.\n[Default: 3s]'))
-
-    group = parser.add_mutually_exclusive_group()
-    group.add_argument('-t', '--test',
-                       action='append',
-                       help='Name of the test to run.')
-    group.add_argument('-a', '--all',
-                       action='store_true',
-                       help='Run ALL FWTS automated tests (assumes -w and -c)')
-    group.add_argument('-s', '--sleep',
-                       nargs=REMAINDER,
-                       action='store',
-                       help=('Perform sleep test(s) using the additional\n'
-                             'arguments provided after --sleep. Remaining\n'
-                             'items on the command line will be passed \n'
-                             'through to fwts for performing sleep tests. \n'
-                             'For info on these extra fwts options, please \n'
-                             'see the epilog below and \n'
-                             'the --fwts-help option.'))
-    group.add_argument('--hwe',
-                       action='store_true',
-                       help='Run HWE concerned tests in fwts')
-    group.add_argument('--qa',
-                       action='store_true',
-                       help='Run QA concerned tests in fwts')
-    group.add_argument('--fwts-help',
-                       dest='fwts_help',
-                       action='store_true',
-                       help='Display the help info for fwts itself (lengthy)')
-    group.add_argument('--list',
-                       action='store_true',
-                       help='List all tests in fwts.')
-    group.add_argument('--list-cert',
-                       action='store_true',
-                       help='List all certification tests in fwts.')
-    group.add_argument('--list-advanced',
-                       action='store_true',
-                       help='List all advanced tests in fwts.')
-    group.add_argument('--list-hwe',
-                       action='store_true',
-                       help='List all HWE concerned tests in fwts')
-    group.add_argument('--list-qa',
-                       action='store_true',
-                       help='List all QA concerned tests in fwts')
-    args = parser.parse_args()
-
-    tests = []
-    results = {}
-    critical_fails = []
-    high_fails = []
-    medium_fails = []
-    low_fails = []
-    passed = []
-    aborted = []
-
-    # Set correct fail level
-    if args.fail_level is not 'none':
-        args.fail_level = 'FAILED_%s' % args.fail_level.upper()
-
-        # Get our failure priority and create the priority values
-        fail_levels = {'FAILED_CRITICAL': 4,
-                       'FAILED_HIGH': 3,
-                       'FAILED_MEDIUM': 2,
-                       'FAILED_LOW': 1,
-                       'FAILED_NONE': 0,
-                       'FAILED_ABORTED': -1}
-        fail_priority = fail_levels[args.fail_level]
-
-    # Enforce only using sleep opts with --sleep
-    if args.sleep_time or args.resume_time and not args.sleep:
-        parser.error('--sleep-time and --resume-time only apply to the '
-                     '--sleep testing option.')
-    if args.fwts_help:
-        Popen('fwts -h', shell=True).communicate()[0]
-        return 0
-    elif args.list:
-        print('\n'.join(TESTS))
-        return 0
-    elif args.list_cert:
-        print('\n'.join(CERT_TESTS))
-        return 0
-    elif args.list_advanced:
-        print('\n'.join(NON_CERT_TESTS))
-        return 0
-    elif args.list_hwe:
-        print('\n'.join(HWE_TESTS))
-        return 0
-    elif args.list_qa:
-        print('\n'.join(QA_TESTS))
-        return 0
-    elif args.test:
-        tests.extend(args.test)
-    elif args.all:
-        tests.extend(TESTS)
-    elif args.hwe:
-        tests.extend(HWE_TESTS)
-    elif args.qa:
-        tests.extend(QA_TESTS)
-    elif args.sleep:
-        args.sleep = fix_sleep_args(args.sleep)
-        iterations = 1
-        # if multiple iterations are requested, we need to intercept
-        # that argument and keep it from being presented to fwts since
-        # we're handling the iterations directly.
-        s3 = '--s3-multiple'
-        s4 = '--s4-multiple'
-        if s3 in args.sleep:
-            iterations = int(args.sleep.pop(args.sleep.index(s3) + 1))
-            args.sleep.remove(s3)
-        if s4 in args.sleep:
-            iterations = int(args.sleep.pop(args.sleep.index(s4) + 1))
-            args.sleep.remove(s4)
-        # if we've passed our custom sleep arguments for resume or sleep
-        # time, we need to intercept those as well.
-        resume_time_arg = '--resume-time'
-        sleep_time_arg = '--sleep-time'
-        if resume_time_arg in args.sleep:
-            args.resume_time = int(args.sleep.pop(
-                                   args.sleep.index(resume_time_arg) + 1))
-            args.sleep.remove(resume_time_arg)
-        if sleep_time_arg in args.sleep:
-            args.sleep_time = int(args.sleep.pop(
-                                  args.sleep.index(sleep_time_arg) + 1))
-            args.sleep.remove(sleep_time_arg)
-        # if we still haven't set a sleep or resume time, use defauts.
-        if not args.sleep_time:
-            args.sleep_time = 15
-        if not args.resume_time:
-            args.resume_time = 15
-        tests.extend(args.sleep)
-    else:
-        tests.extend(CERT_TESTS)
-
-    # run the tests we want
-    if args.sleep:
-        iteration_results = {}
-        print('=' * 20 + ' Test Results ' + '=' * 20)
-        progress_indicator = None
-        if detect_progress_indicator():
-            progress_indicator = Popen(detect_progress_indicator(),
-                                       stdin=PIPE)
-        for iteration in range(1, iterations+1):
-            timestamp = int(time())
-            start_marker = 'CHECKBOX SLEEP TEST START %s' % timestamp
-            end_marker = 'CHECKBOX SLEEP TEST STOP %s' % timestamp
-            with open(args.log + '.log', 'a') as f:
-                f.write('{:=^80}\n'.format(' Iteration {} '.format(iteration)))
-            syslog(LOG_INFO, '---' + start_marker + '---' + str(time()))
-            command = ('fwts -q --stdout-summary -r %s %s'
-                       % (args.log, ' '.join(tests)))
-            results['sleep'] = (Popen(command, stdout=PIPE, shell=True)
-                                .communicate()[0].strip()).decode()
-            syslog(LOG_INFO, '---' + end_marker + '---' + str(time()))
-            sleep(2)
-            if 's4' not in args.sleep:
-                sleep_times = get_sleep_times(start_marker,
-                                              end_marker,
-                                              args.sleep_time,
-                                              args.resume_time)
-                iteration_results[iteration] = sleep_times
-                progress_tuple = (iteration,
-                                  iteration_results[iteration][0],
-                                  iteration_results[iteration][1],
-                                  iteration_results[iteration][2])
-                progress_string = (' - Cycle %s: Status: %s  '
-                                   'Sleep Elapsed: %0.5f    '
-                                   'Resume Elapsed: '
-                                   ' %0.5f' % progress_tuple)
-                progress_pct = "{}".format(int(100 * iteration / iterations))
-                if "zenity" in detect_progress_indicator():
-                    progress_indicator.stdin.write("# {}\n".format(
-                        progress_string).encode('utf-8'))
-                    progress_indicator.stdin.write("{}\n".format(
-                        progress_pct).encode('utf-8'))
-                    progress_indicator.stdin.flush()
-                elif "dialog" in detect_progress_indicator():
-                    progress_indicator.stdin.write("XXX\n".encode('utf-8'))
-                    progress_indicator.stdin.write(
-                        progress_pct.encode('utf-8'))
-                    progress_indicator.stdin.write(
-                        "\nTest progress\n".encode('utf-8'))
-                    progress_indicator.stdin.write(
-                        progress_string.encode('utf-8'))
-                    progress_indicator.stdin.write(
-                        "\nXXX\n".encode('utf-8'))
-                    progress_indicator.stdin.flush()
-                else:
-                    print(progress_string, flush=True)
-        if detect_progress_indicator():
-            progress_indicator.terminate()
-
-        if 's4' not in args.sleep:
-            average_times(iteration_results)
-            for run in iteration_results.keys():
-                if 'FAIL' in iteration_results[run]:
-                    results['sleep'] = 'FAILED_CRITICAL'
-    else:
-        for test in tests:
-            # ACPI tests can now be run with --acpitests (fwts >= 15.07.00)
-            log = args.log
-            # Split the log file for HWE (only if -t is not used)
-            if test == 'acpitests':
-                test = '--acpitests'
-            command = ('fwts -q --stdout-summary -r %s %s'
-                       % (log, test))
-            results[test] = (Popen(command, stdout=PIPE, shell=True)
-                             .communicate()[0].strip()).decode()
-
-    # parse the summaries
-    for test in results.keys():
-        if 'FAILED_CRITICAL' in results[test]:
-            critical_fails.append(test)
-        if 'FAILED_HIGH' in results[test]:
-            high_fails.append(test)
-        if 'FAILED_MEDIUM' in results[test]:
-            medium_fails.append(test)
-        if 'FAILED_LOW' in results[test]:
-            low_fails.append(test)
-        if 'PASSED' in results[test]:
-            passed.append(test)
-        if 'ABORTED' in results[test]:
-            aborted.append(test)
-        else:
-            continue
-
-    if critical_fails:
-        print("Critical Failures: %d" % len(critical_fails))
-        print("WARNING: The following test cases were reported as critical\n"
-              "level failures by fwts. Please review the log at\n"
-              "%s for more information." % args.log)
-        for test in critical_fails:
-            print(" - " + test)
-    if high_fails:
-        print("High Failures: %d" % len(high_fails))
-        print("WARNING: The following test cases were reported as high\n"
-              "level failures by fwts. Please review the log at\n"
-              "%s for more information." % args.log)
-        for test in high_fails:
-            print(" - " + test)
-    if medium_fails:
-        print("Medium Failures: %d" % len(medium_fails))
-        print("WARNING: The following test cases were reported as medium\n"
-              "level failures by fwts. Please review the log at\n"
-              "%s for more information." % args.log)
-        for test in medium_fails:
-            print(" - " + test)
-    if low_fails:
-        print("Low Failures: %d" % len(low_fails))
-        print("WARNING: The following test cases were reported as low\n"
-              "level failures by fwts. Please review the log at\n"
-              "%s for more information." % args.log)
-        for test in low_fails:
-            print(" - " + test)
-    if passed:
-        print("Passed: %d" % len(passed))
-        for test in passed:
-            print(" - " + test)
-    if aborted:
-        print("Aborted Tests: %d" % len(aborted))
-        print("WARNING: The following test cases were aborted by fwts\n"
-              "Please review the log at %s for more information."
-              % args.log)
-        for test in aborted:
-            print(" - " + test)
-
-    if args.fail_level is not 'none':
-        if fail_priority == fail_levels['FAILED_CRITICAL']:
-            if critical_fails:
-                return 1
-        if fail_priority == fail_levels['FAILED_HIGH']:
-            if critical_fails or high_fails:
-                return 1
-        if fail_priority == fail_levels['FAILED_MEDIUM']:
-            if critical_fails or high_fails or medium_fails:
-                return 1
-        if fail_priority == fail_levels['FAILED_LOW']:
-            if critical_fails or high_fails or medium_fails or low_fails:
-                return 1
-        if fail_priority == fail_levels['FAILED_ABORTED']:
-            if aborted or critical_fails or high_fails:
-                return 1
-
-    return 0
-
-if __name__ == '__main__':
-    sys.exit(main())
diff --git a/bin/gateway_ping_test b/bin/gateway_ping_test
deleted file mode 100755
index 14c5d8e..0000000
--- a/bin/gateway_ping_test
+++ /dev/null
@@ -1,286 +0,0 @@
-#!/usr/bin/python3
-# This file is part of Checkbox.
-#
-# Copyright 2007-2014 Canonical Ltd.
-# Written by:
-#   Brendan Donegan <[e-mail]>
-#   Daniel Manrique <[e-mail]>
-#   David Murphy <[e-mail]>
-#   Javier Collado <[e-mail]>
-#   Jeff Lane <[e-mail]>
-#   Marc Tardif <[e-mail]>
-#   Mathieu Trudel-Lapierre <[e-mail]>
-#   Zygmunt Krynicki <[e-mail]>
-#
-# Checkbox is free software: you can redistribute it and/or modify
-# it under the terms of the GNU General Public License version 3,
-# as published by the Free Software Foundation.
-#
-# Checkbox is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License
-# along with Checkbox.  If not, see <http://www.gnu.org/licenses/>.
-
-from gettext import gettext as _
-import argparse
-import errno
-import gettext
-import logging
-import os
-import re
-import socket
-import struct
-import subprocess
-import sys
-import time
-
-
-class Route:
-    """
-    Gets routing information from the system.
-    """
-
-    def _num_to_dotted_quad(self, number):
-        """
-        Convert long int to dotted quad string
-        """
-        return socket.inet_ntoa(struct.pack("<L", number))
-
-    def _get_default_gateway_from_proc(self):
-        """
-        Returns the current default gateway, reading that from /proc
-        """
-        logging.debug(_("Reading default gateway information from /proc"))
-        try:
-            with open("/proc/net/route", "rt") as stream:
-                route = stream.read()
-        except:
-            logging.error(_("Failed to read def gateway from /proc"))
-            return None
-        else:
-            h = re.compile("\n(?P<interface>\w+)\s+00000000\s+"
-                           "(?P<def_gateway>[\w]+)\s+")
-            w = h.search(route)
-            if w:
-                if w.group("def_gateway"):
-                    return self._num_to_dotted_quad(
-                        int(w.group("def_gateway"), 16))
-                else:
-                    logging.error(
-                        _("Could not find def gateway info in /proc"))
-                    return None
-            else:
-                logging.error(_("Could not find def gateway info in /proc"))
-                return None
-
-    def _get_default_gateway_from_bin_route(self):
-        """
-        Get default gateway from /sbin/route -n
-        Called by get_default_gateway
-        and is only used if could not get that from /proc
-        """
-        logging.debug(
-            _("Reading default gateway information from route binary"))
-        routebin = subprocess.getstatusoutput(
-            "export LANGUAGE=C; " "/usr/bin/env route -n")
-        if routebin[0] == 0:
-            h = re.compile("\n0.0.0.0\s+(?P<def_gateway>[\w.]+)\s+")
-            w = h.search(routebin[1])
-            if w:
-                def_gateway = w.group("def_gateway")
-                if def_gateway:
-                    return def_gateway
-        logging.error(_("Could not find default gateway by running route"))
-        return None
-
-    def get_hostname(self):
-        return socket.gethostname()
-
-    def get_default_gateway(self):
-        t1 = self._get_default_gateway_from_proc()
-        if not t1:
-            t1 = self._get_default_gateway_from_bin_route()
-        return t1
-
-
-def get_host_to_ping(interface=None, verbose=False, default=None):
-    # Get list of all IPs from all my interfaces,
-    interface_list = subprocess.check_output(["ip", "-o", 'addr', 'show'])
-    reg = re.compile('\d: (?P<iface>\w+) +inet (?P<address>[\d\.]+)/'
-                     '(?P<netmask>[\d]+) brd (?P<broadcast>[\d\.]+)')
-    # Will magically exclude lo because it lacks brd field
-    interfaces = reg.findall(interface_list.decode())
-    # ping -b the network on each one (one ping only)
-    # exclude the ones not specified in iface
-    for iface in interfaces:
-        if not interface or iface[0] == interface:
-            # Use check_output even if I'll discard the output
-            # looks cleaner than using .call and redirecting stdout to null
-            try:
-                subprocess.check_output(["ping", "-q", "-c", "1", "-b",
-                                         iface[3]], stderr=subprocess.STDOUT)
-            except subprocess.CalledProcessError:
-                pass
-    # If default host given, ping it as well,
-    # to try to get it into the arp table.
-    # Needed in case it's not responding to broadcasts.
-    if default:
-        try:
-            subprocess.check_output(["ping", "-q", "-c", "1", default],
-                                    stderr=subprocess.STDOUT)
-        except subprocess.CalledProcessError:
-            pass
-    ARP_POPULATE_TRIES = 10
-    num_tries = 0
-    while num_tries < ARP_POPULATE_TRIES:
-        # Get output from arp -a -n to get known IPs
-        known_ips = subprocess.check_output(["arp", "-a", "-n"])
-        reg = re.compile('\? \((?P<ip>[\d.]+)\) at (?P<mac>[a-f0-9\:]+) '
-                         '\[ether\] on (?P<iface>[\w\d]+)')
-        # Filter (if needed) IPs not on the specified interface
-        pingable_ips = [pingable[0] for pingable in reg.findall(
-                        known_ips.decode()) if not interface
-                        or pingable[2] == interface]
-        # If the default given ip is among the remaining ones,
-        # ping that.
-        if default and default in pingable_ips:
-            if verbose:
-                print(_(
-                    "Desired ip address {0} is reachable, using it"
-                ).format(default))
-            return default
-        # If not, choose another IP.
-        address_to_ping = pingable_ips[0] if len(pingable_ips) else None
-        if verbose:
-            print(_(
-                "Desired ip address {0} is not reachable from {1},"
-                " using {2} instead"
-            ).format(default, interface, address_to_ping))
-        if address_to_ping:
-            return address_to_ping
-        time.sleep(2)
-        num_tries += 1
-    # Wait time expired
-    return None
-
-
-def ping(host, interface, count, deadline, verbose=False):
-    command = ["ping", str(host),  "-c", str(count), "-w", str(deadline)]
-    if interface:
-        command.append("-I{}".format(interface))
-    reg = re.compile(
-        r"(\d+) packets transmitted, (\d+) received, (\d+)% packet loss")
-    ping_summary = {'transmitted': 0, 'received': 0, 'pct_loss': 0}
-    try:
-        output = subprocess.check_output(command, universal_newlines=True)
-    except OSError as exc:
-        if exc.errno == errno.ENOENT:
-            # No ping command present;
-            # default exception message is informative enough.
-            print(exc)
-        else:
-            raise
-    except subprocess.CalledProcessError as excp:
-        # Ping returned fail exit code
-        print(_("ERROR: ping result: {0}").format(excp))
-    else:
-        if verbose:
-            print(output)
-        received = re.findall(reg, output)
-        if received:
-            ping_summary = received[0]
-            ping_summary = {
-                'transmitted': int(ping_summary[0]),
-                'received': int(ping_summary[1]),
-                'pct_loss': int(ping_summary[2])}
-    return ping_summary
-
-
-def main(args):
-    gettext.textdomain("2015.com.canonical.qa.plano")
-    gettext.bindtextdomain("2015.com.canonical.qa.plano",
-                           os.getenv("CHECKBOX_PROVIDER_LOCALE_DIR", None))
-    default_count = 2
-    default_delay = 4
-    route = Route()
-    parser = argparse.ArgumentParser()
-    parser.add_argument(
-        "host", nargs='?', default=route.get_default_gateway(),
-        help=_("host to ping"))
-    parser.add_argument(
-        "-c", "--count", default=default_count, type=int,
-        help=_("number of packets to send"))
-    parser.add_argument(
-        "-d", "--deadline", default=default_delay, type=int,
-        help=_("timeout in seconds"))
-    parser.add_argument(
-        "-t", "--threshold", default=0, type=int,
-        help=_("allowed packet loss percentage (default: %(default)s)"))
-    parser.add_argument(
-        "-v", "--verbose", action='store_true', help=_("be verbose"))
-    parser.add_argument(
-        "-I", "--interface", help=_("use specified interface to send packets"))
-    args = parser.parse_args()
-    # Ensure count and deadline make sense. Adjust them if not.
-    if args.deadline != default_delay and args.count != default_count:
-        # Ensure they're both consistent, and exit with a warning if not,
-        # rather than modifying what the user explicitly set.
-        if args.deadline <= args.count:
-            # FIXME: this cannot ever be translated correctly
-            print(_(
-                "ERROR: not enough time for {0} pings in {1} seconds"
-            ).format(args.count, args.deadline))
-            return 1
-    elif args.deadline != default_delay:
-        # Adjust count according to delay.
-        args.count = args.deadline - 1
-        if args.count < 1:
-            args.count = 1
-        if args.verbose:
-            # FIXME: this cannot ever be translated correctly
-            print(_(
-                "Adjusting ping count to {0} to fit in {1}-second deadline"
-            ).format(args.count, args.deadline))
-    else:
-        # Adjust delay according to count
-        args.deadline = args.count + 1
-        if args.verbose:
-            # FIXME: this cannot ever be translated correctly
-            print(_(
-                "Adjusting deadline to {0} seconds to fit {1} pings"
-            ).format(args.deadline, args.count))
-    # If given host is not pingable, override with something pingable.
-    host = get_host_to_ping(
-        interface=args.interface, verbose=args.verbose, default=args.host)
-    if args.verbose:
-        print(_("Checking connectivity to {0}").format(host))
-    ping_summary = None
-    if host:
-        ping_summary = ping(host, args.interface, args.count,
-                            args.deadline, args.verbose)
-    if ping_summary is None or ping_summary['received'] == 0:
-        print(_("No Internet connection"))
-        return 1
-    elif ping_summary['transmitted'] != ping_summary['received']:
-        print(_("Connection established, but lost {0}% of packets").format(
-            ping_summary['pct_loss']))
-        if ping_summary['pct_loss'] > args.threshold:
-            print(_(
-                "FAIL: {0}% packet loss is higher than {1}% threshold"
-            ).format(ping_summary['pct_loss'], args.threshold))
-            return 1
-        else:
-            print(_(
-                "PASS: {0}% packet loss is within {1}% threshold"
-            ).format(ping_summary['pct_loss'], args.threshold))
-            return 0
-    else:
-        print(_("Connection to test host fully established"))
-        return 0
-
-
-if __name__ == "__main__":
-    sys.exit(main(sys.argv[1:]))
diff --git a/bin/lsmod_info b/bin/lsmod_info
deleted file mode 100755
index 3b81542..0000000
--- a/bin/lsmod_info
+++ /dev/null
@@ -1,40 +0,0 @@
-#!/usr/bin/env python3
-import sys
-from checkbox_support.parsers.modinfo import ModinfoParser
-from subprocess import Popen, PIPE, check_output, CalledProcessError
-
-
-def main():
-    process = Popen('lsmod', stdout=PIPE, stderr=PIPE, universal_newlines=True)
-    data = process.stdout.readlines()
-    # Delete the first item because it's headers from lsmod output
-    data.pop(0)
-    module_list = [module.split()[0].strip() for module in data]
-
-    cmd = '/sbin/modinfo'
-    for module in sorted(module_list):
-        version = ''
-        stream = b''
-        try:
-            stream = check_output([cmd, module],
-                                  stderr=PIPE,
-                                  universal_newlines=False)
-        except CalledProcessError as e:
-            if e.returncode != 1:
-                raise e
-            else:
-                version = 'Unavailable'
-
-        stream = stream.decode('utf-8')
-
-        parser = ModinfoParser(stream)
-
-        if not version:
-            version = parser.get_field('version')
-            if not version:
-                version = parser.get_field('vermagic').split()[0]
-        print('%s: %s' % (module, version))
-    return 0
-
-if __name__ == '__main__':
-    sys.exit(main())
diff --git a/bin/memory_compare b/bin/memory_compare
deleted file mode 100755
index 02e2def..0000000
--- a/bin/memory_compare
+++ /dev/null
@@ -1,138 +0,0 @@
-#!/usr/bin/env python3
-# -*- coding: utf-8 -*-
-#
-# This file is part of Checkbox.
-#
-# Copyright 2014 Canonical Ltd.
-#
-# Authors:
-#    Brendan Donegan <[e-mail]>
-#    Jeff Lane <[e-mail]>
-#    Sylvain Pineau <[e-mail]>
-#
-# Checkbox is free software: you can redistribute it and/or modify
-# it under the terms of the GNU General Public License version 3,
-# as published by the Free Software Foundation.
-#
-# Checkbox is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License
-# along with Checkbox.  If not, see <http://www.gnu.org/licenses/>.
-
-import os
-import sys
-import re
-from math import log, copysign
-from subprocess import check_output, CalledProcessError, PIPE
-
-from checkbox_support.helpers.human_readable_bytes import HumanReadableBytes
-from checkbox_support.parsers.lshwjson import LshwJsonParser
-from checkbox_support.parsers.meminfo import MeminfoParser
-
-
-class LshwJsonResult:
-
-    memory_reported = 0
-    banks_reported = 0
-
-    # jlane LP:1525009
-    # Discovered the case can change, my x86 systems used "System Memory"
-    # Failing ARM system used "System memory"
-    desc_regex = re.compile('System Memory', re.IGNORECASE)
-
-    def addHardware(self, hardware):
-        if self.desc_regex.match(str(hardware.get('description',0))):
-            self.memory_reported += int(hardware.get('size', 0))
-        elif 'bank' in hardware['id']:
-            self.banks_reported += int(hardware.get('size', 0))
-
-
-def get_installed_memory_size():
-    try:
-        output = check_output(['lshw', '-json'],
-                              universal_newlines=True,
-                              stderr=PIPE)
-    except CalledProcessError:
-        return 0
-    lshw = LshwJsonParser(output)
-    result = LshwJsonResult()
-    lshw.run(result)
-
-    if result.memory_reported:
-        return result.memory_reported
-    else:
-        return result.banks_reported
-
-
-class MeminfoResult:
-
-    memtotal = 0
-
-    def setMemory(self, memory):
-        self.memtotal = memory['total']
-
-
-def get_visible_memory_size():
-    parser = MeminfoParser(open('/proc/meminfo'))
-    result = MeminfoResult()
-    parser.run(result)
-
-    return result.memtotal
-
-
-def get_threshold(installed_memory):
-    GB = 1024**3
-    if installed_memory <= 2 * GB:
-        return 25
-    elif installed_memory <= 6 * GB:
-        return 20
-    else:
-        return 10
-
-
-def main():
-    if os.geteuid() != 0:
-        print("This script must be run as root.", file=sys.stderr)
-        return 1
-
-    installed_memory = HumanReadableBytes(get_installed_memory_size())
-    visible_memory = HumanReadableBytes(get_visible_memory_size())
-    threshold = get_threshold(installed_memory)
-
-    difference = HumanReadableBytes(installed_memory - visible_memory)
-    try:
-        percentage = difference / installed_memory * 100
-    except ZeroDivisionError:
-        print("Results:")
-        print("\t/proc/meminfo reports:\t{}".format(visible_memory),
-              file=sys.stderr)
-        print("\tlshw reports:\t{}".format(installed_memory),
-              file=sys.stderr)
-        print("\nFAIL: Either lshw or /proc/meminfo returned a memory size "
-              "of 0 kB", file=sys.stderr)
-        return 1
-
-    if percentage <= threshold:
-        print("Results:")
-        print("\t/proc/meminfo reports:\t{}".format(visible_memory))
-        print("\tlshw reports:\t{}".format(installed_memory))
-        print("\nPASS: Meminfo reports %s less than lshw, a "
-              "difference of %.2f%%. This is less than the "
-              "%d%% variance allowed." % (difference, percentage, threshold))
-        return 0
-    else:
-        print("Results:", file=sys.stderr)
-        print("\t/proc/meminfo reports:\t{}".format(visible_memory),
-              file=sys.stderr)
-        print("\tlshw reports:\t{}".format(installed_memory), file=sys.stderr)
-        print("\nFAIL: Meminfo reports %d less than lshw, "
-              "a difference of %.2f%%. Only a variance of %d%% in "
-              "reported memory is allowed." %
-              (difference, percentage, threshold), file=sys.stderr)
-        return 1
-
-if __name__ == "__main__":
-    sys.exit(main())
diff --git a/bin/memory_test b/bin/memory_test
deleted file mode 100755
index 4cd6008..0000000
--- a/bin/memory_test
+++ /dev/null
@@ -1,239 +0,0 @@
-#!/usr/bin/env python3
-
-import os
-import sys
-import re
-
-from argparse import ArgumentParser
-from subprocess import Popen, PIPE
-
-
-class MemoryTest():
-
-    def __init__(self):
-        self.free_memory = 0
-        self.system_memory = 0
-        self.swap_memory = 0
-        self.process_memory = 0
-        self.is_process_limited = False
-
-    @property
-    def threaded_memtest_script(self):
-        directory = os.path.dirname(os.path.abspath(__file__))
-        return os.path.join(directory, "threaded_memtest")
-
-    def _get_memory(self):
-        mem_info = open("/proc/meminfo", "r")
-        try:
-            while True:
-                line = mem_info.readline()
-                if line:
-                    tokens = line.split()
-                    if len(tokens) == 3:
-                        if "MemTotal:" == tokens[0].strip():
-                            self.system_memory = \
-                                int(tokens[1].strip()) // 1024
-                        elif tokens[0].strip() in ["MemFree:",
-                                                   "Cached:",
-                                                   "Buffers:"]:
-                            self.free_memory += \
-                                int(tokens[1].strip()) // 1024
-                        elif "SwapTotal:" == tokens[0].strip():
-                            self.swap_memory = \
-                                int(tokens[1].strip()) // 1024
-                else:
-                    break
-        except Exception as e:
-            print("ERROR: Unable to get data from /proc/meminfo",
-                  file=sys.stderr)
-            print(e, file=sys.stderr)
-        finally:
-            mem_info.close()
-
-    def _command(self, command, shell=True):
-        proc = Popen(command,
-                     shell=shell,
-                     stdout=PIPE,
-                     stderr=PIPE)
-        return proc
-
-    def _command_out(self, command, shell=True):
-        proc = self._command(command, shell)
-        return proc.communicate()[0].strip()
-
-    def get_limits(self):
-        self._get_memory()
-        print("System Memory: %u MB" % self.system_memory)
-        print("Free Memory: %u MB" % self.free_memory)
-        print("Swap Memory: %u MB" % self.swap_memory)
-
-        if self.system_memory == 0:
-            print("ERROR: could not determine system RAM", file=sys.stderr)
-            return False
-
-        # Process Memory
-        self.process_memory = self.free_memory
-        try:
-            arch = self._command_out("arch").decode()
-            if (re.match(r"(i[0-9]86|s390|arm.*)", arch)
-                    and self.free_memory > 1024):
-                self.is_process_limited = True
-                self.process_memory = 1024  # MB, due to 32-bit address space
-                print("%s arch, Limiting Process Memory: %u" % (
-                      arch, self.process_memory))
-        # others?  what about PAE kernel?
-        except Exception as e:
-            print("ERROR: could not determine system architecture via arch",
-                  file=sys.stderr)
-            print(e, file=sys.stderr)
-            return False
-        return True
-
-    def run(self):
-        PASSED = 0
-        FAILED = 1
-
-        limits = self.get_limits()
-        if not limits:
-            return FAILED
-
-        # if process memory is limited, run multiple processes
-        if self.is_process_limited:
-            print("Running Multiple Process Memory Test")
-            if not self.run_multiple_process_test():
-                return FAILED
-        else:
-            print("Running Single Process Memory Test")
-            if not self.run_single_process_test():
-                return FAILED
-
-        # otherwised, passed
-        return PASSED
-
-    def run_single_process_test(self):
-        if not self.run_threaded_memory_test():
-            return False
-        return True
-
-    def run_multiple_process_test(self):
-        processes = self.free_memory // self.process_memory
-        # if not swap-less, add a process to hit swap
-        if not self.swap_memory == 0:
-            processes += 1
-            # check to make sure there's enough swap
-            required_memory = self.process_memory * processes
-            if required_memory > self.system_memory + self.swap_memory:
-                print("ERROR: this test requires a minimum of %u KB of swap "
-                      "memory (%u configured)" % (
-                          required_memory - self.system_memory,
-                          self.swap_memory), file=sys.stderr)
-        print("Testing memory with %u processes" % processes)
-
-        print("Running threaded memory test:")
-        run_time = 60  # sec.
-        if not self.run_processes(processes, "%s -qv -m%um -t%u" % (
-                self.threaded_memtest_script, self.process_memory, run_time)):
-            print("Multi-process, threaded memory Test FAILED",
-                  file=sys.stderr)
-            return False
-
-        return True
-
-    def run_threaded_memory_test(self):
-        # single-process threaded test
-        print("Starting Threaded Memory Test")
-
-        # run for Free Memory plus the lessor of 5% or 1GB
-        memory = (self.free_memory * 5) / 100
-        if memory > 1024:  # MB
-            memory = 1024  # MB
-        memory = memory + self.free_memory
-        print("Running for %d MB total memory" % memory)
-
-        # run a test that will swap
-        if not self.swap_memory == 0:
-
-            # is there enough swap memory for the test?
-            if memory > self.system_memory + self.swap_memory:
-                print("ERROR: this test requires a minimum of %u KB of swap "
-                      "memory (%u configured)"
-                      % (memory - self.system_memory, self.swap_memory),
-                      file=sys.stderr)
-                return False
-
-            # otherwise
-            run_time = 60  # sec.
-            print("Running for more than free memory at %u MB for %u sec." % (
-                memory, run_time))
-
-            command = "%s -qv -m%um -t%u" % (
-                self.threaded_memtest_script, memory, run_time)
-            print("Command is: %s" % command)
-            process = self._command(command)
-            process.communicate()
-            if process.returncode != 0:
-                print("%s returned code %s" % (self.threaded_memtest_script,
-                      process.returncode), file=sys.stderr)
-                print("More Than Free Memory Test failed", file=sys.stderr)
-                return False
-            print("More than free memory test complete.")
-
-        # run again for 15 minutes
-        print("Running for free memory")
-        process = self._command("%s -qv" % self.threaded_memtest_script)
-        process.communicate()
-        if process.returncode != 0:
-            print("Free Memory Test failed", file=sys.stderr)
-        else:
-            print("Free Memory Test succeeded")
-        sys.stdout.flush()
-        return (process.returncode == 0)
-
-    def run_processes(self, number, command):
-        passed = True
-        pipe = []
-        for i in range(number):
-            pipe.append(self._command(command))
-            print("Started: process %u pid %u: %s" % (i, pipe[i].pid, command))
-        sys.stdout.flush()
-        waiting = True
-        while waiting:
-            waiting = False
-            for i in range(number):
-                if pipe[i]:
-                    line = pipe[i].communicate()[0]
-                    if line and len(line) > 1:
-                        print("process %u pid %u: %s" % (i, pipe[i].pid, line))
-                        sys.stdout.flush()
-                    if pipe[i].poll() == -1:
-                        waiting = True
-                    else:
-                        return_value = pipe[i].poll()
-                        if return_value != 0:
-                            print("ERROR: process  %u pid %u retuned %u"
-                                  % (i, pipe[i].pid, return_value),
-                                  file=sys.stderr)
-                            passed = False
-                        print("process %u pid %u returned success"
-                              % (i, pipe[i].pid))
-                        pipe[i] = None
-        sys.stdout.flush()
-        return passed
-
-
-def main(args):
-    parser = ArgumentParser()
-    parser.add_argument("-q", "--quiet", action="store_true",
-                        help="Suppress output.")
-    args = parser.parse_args(args)
-
-    if args.quiet:
-        sys.stdout = open(os.devnull, 'a')
-        sys.stderr = open(os.devnull, 'a')
-
-    test = MemoryTest()
-    return test.run()
-
-
-if __name__ == "__main__":
-    sys.exit(main(sys.argv[1:]))
diff --git a/bin/network_device_info b/bin/network_device_info
deleted file mode 100755
index 0e7479e..0000000
--- a/bin/network_device_info
+++ /dev/null
@@ -1,266 +0,0 @@
-#!/usr/bin/env python3
-#
-# This program is free software: you can redistribute it and/or modify
-# it under the terms of the GNU General Public License version 3,
-# as published by the Free Software Foundation.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License
-# along with this program; if not, write to the Free Software Foundation,
-# Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-#
-# Parts of this are based on the example python code that ships with
-# NetworkManager
-# http://cgit.freedesktop.org/NetworkManager/NetworkManager/tree/examples/python
-#
-# Copyright (C) 2012 Canonical, Ltd.
-
-from subprocess import check_output, CalledProcessError, STDOUT
-import sys
-
-import dbus
-
-from checkbox_support.parsers.modinfo import ModinfoParser
-from checkbox_support.parsers.udevadm import UdevadmParser
-
-
-# This example lists basic information about network interfaces known to NM
-devtypes = {1: "Ethernet",
-            2: "WiFi",
-            5: "Bluetooth",
-            6: "OLPC",
-            7: "WiMAX",
-            8: "Modem"}
-
-states = {0: "Unknown",
-          10: "Unmanaged",
-          20: "Unavailable",
-          30: "Disconnected",
-          40: "Prepare",
-          50: "Config",
-          60: "Need Auth",
-          70: "IP Config",
-          80: "IP Check",
-          90: "Secondaries",
-          100: "Activated",
-          110: "Deactivating",
-          120: "Failed"}
-
-attributes = ("category", "interface", "product", "vendor", "driver", "path")
-
-udev_devices = []
-nm_devices = []
-
-
-class UdevResult:
-    def addDevice(self, device):
-        if device.category == 'NETWORK' and not 'cdc' in device.driver:
-            udev_devices.append(device)
-
-
-class NetworkingDevice():
-    def __init__(self, devtype, props, dev_proxy, bus):
-        self._devtype = devtype
-        try:
-            self._interface = props['Interface']
-        except KeyError:
-            self._interface = "Unknown"
-
-        try:
-            self._ip = self._int_to_ip(props['Ip4Address'])
-        except KeyError:
-            self._ip = "Unknown"
-
-        try:
-            self._driver = props['Driver']
-        except KeyError:
-            self._driver = "Unknown"
-            self._driver_ver = "Unknown"
-
-        if self._driver != "Unknown":
-            self._modinfo = self._modinfo_parser(props['Driver'])
-            if self._modinfo:
-                self._driver_ver = self._find_driver_ver()
-            else:
-                self._driver_ver = "Unknown"
-
-        try:
-            self._firmware_missing = props['FirmwareMissing']
-        except KeyError:
-            self._firmware_missing = False
-
-        try:
-            self._state = states[props['State']]
-        except KeyError:
-            self._state = "Unknown"
-
-    def __str__(self):
-        ret = "Category: %s\n" % self._devtype
-        ret += "Interface: %s\n" % self._interface
-        ret += "IP: %s\n" % self._ip
-        ret += "Driver: %s (ver: %s)\n" % (self._driver, self._driver_ver)
-        if self._firmware_missing:
-            ret += "Warning: Required Firmware Missing for device\n"
-        ret += "State: %s\n" % self._state
-        return ret
-
-    def getstate(self):
-        return self._state
-
-    def gettype(self):
-        return self._devtype
-
-    def _bitrate_to_mbps(self, bitrate):
-        try:
-            intbr = int(bitrate)
-            return str(intbr / 1000)
-        except Exception:
-            return "NaN"
-
-    def _modinfo_parser(self, driver):
-        cmd = ['/sbin/modinfo', driver]
-        try:
-            stream = check_output(cmd, stderr=STDOUT, universal_newlines=True)
-        except CalledProcessError as err:
-            return None
-
-        if not stream:
-            return None
-        else:
-            parser = ModinfoParser(stream)
-            modinfo = parser.get_all()
-
-        return modinfo
-
-    def _find_driver_ver(self):
-        # try the version field first, then vermagic second, some audio
-        # drivers don't report version if the driver is in-tree
-        if self._modinfo['version'] and self._modinfo['version'] != 'in-tree:':
-            return self._modinfo['version']
-        else:
-            # vermagic will look like this (below) and we only care about the
-            # first part:
-            # "3.2.0-29-generic SMP mod_unload modversions"
-            return self._modinfo['vermagic'].split()[0]
-
-    def _int_to_ip(self, int_ip):
-        ip = [0, 0, 0, 0]
-        ip[0] = int_ip & 0xff
-        ip[1] = (int_ip >> 8) & 0xff
-        ip[2] = (int_ip >> 16) & 0xff
-        ip[3] = (int_ip >> 24) & 0xff
-        return "%d.%d.%d.%d" % (ip[0], ip[1], ip[2], ip[3])
-
-
-def get_nm_devices():
-    devices = []
-    bus = dbus.SystemBus()
-
-    # Get a proxy for the base NetworkManager object
-    proxy = bus.get_object("org.freedesktop.NetworkManager",
-                           "/org/freedesktop/NetworkManager")
-    manager = dbus.Interface(proxy, "org.freedesktop.NetworkManager")
-
-    # Get all devices known to NM and print their properties
-    nm_devices = manager.GetDevices()
-    for d in nm_devices:
-        dev_proxy = bus.get_object("org.freedesktop.NetworkManager", d)
-        prop_iface = dbus.Interface(dev_proxy,
-                                    "org.freedesktop.DBus.Properties")
-        props = prop_iface.GetAll("org.freedesktop.NetworkManager.Device")
-        try:
-            devtype = devtypes[props['DeviceType']]
-        except KeyError:
-            devtype = "Unknown"
-
-        # only return Ethernet devices
-        if devtype in ("Ethernet"):
-            devices.append(NetworkingDevice(devtype, props, dev_proxy, bus))
-    return devices
-
-
-def match_counts(nm_devices, udev_devices, devtype):
-    """
-    Ensures that the count of devices matching devtype is the same for the
-    two passed in lists, devices from Network Manager and devices from lspci.
-    """
-    # now check that the count (by type) matches
-    nm_type_devices = [dev for dev in nm_devices if dev.gettype() in devtype]
-    udevtype = 'NETWORK'
-    udev_type_devices = [
-        udev
-        for udev in udev_devices
-        if udev.category == udevtype]
-    if len(nm_type_devices) != len(udev_type_devices):
-        print("ERROR: devices missing - udev showed %d %s devices, but "
-              "NetworkManager saw %d devices in %s" %
-              (len(udev_type_devices), udevtype,
-               len(nm_type_devices), devtype),
-              file=sys.stderr)
-        return False
-    else:
-        return True
-
-
-def main(args):
-    try:
-        output = check_output(['udevadm', 'info', '--export-db'])
-    except CalledProcessError as err:
-        raise SystemExit(err)
-    try:
-        output = output.decode("UTF-8", errors='ignore')
-    except UnicodeDecodeError as err:
-        raise SystemExit("udevadm output is not valid UTF-8")
-    udev = UdevadmParser(output)
-    result = UdevResult()
-    udev.run(result)
-
-    if udev_devices:
-        print("[ Devices found by udev ]".center(80, '-'))
-        for device in udev_devices:
-            for attribute in attributes:
-                value = getattr(device, attribute)
-                if value is not None:
-                    if attribute == 'driver':
-                        props = {}
-                        props['Driver'] = value
-                        network_dev = NetworkingDevice(None, props, None, None)
-                        print("%s: %s (ver: %s)" % (attribute.capitalize(),
-                              value, network_dev._driver_ver))
-                    else:
-                        print("%s: %s" % (attribute.capitalize(), value))
-            vendor_id = getattr(device, 'vendor_id')
-            product_id = getattr(device, 'product_id')
-            subvendor_id = getattr(device, 'subvendor_id')
-            subproduct_id = getattr(device, 'subproduct_id')
-            if vendor_id and product_id:
-                print("ID:           [{0:04x}:{1:04x}]".format(vendor_id, product_id))
-            if subvendor_id and subproduct_id:
-                print("Subsystem ID: [{0:04x}:{1:04x}]".format(subvendor_id, subproduct_id))
-            print()
-
-    try:
-        nm_devices = get_nm_devices()
-    except dbus.exceptions.DBusException as e:
-        # server's don't have network manager installed
-        print("Warning: Exception while talking to Network Manager over dbus."
-              " Skipping the remainder of this test. If this is a server, this"
-              " is expected.", file=sys.stderr)
-        print("The Error Generated was:\n %s" % e, file=sys.stderr)
-        return 0
-
-    print("[ Devices found by Network Manager ]".center(80, '-'))
-    for nm_dev in nm_devices:
-        print(nm_dev)
-
-    if not match_counts(nm_devices, udev_devices, ("Ethernet")):
-        return 1
-    else:
-        return 0
-
-if __name__ == "__main__":
-    sys.exit(main(sys.argv[1:]))
diff --git a/bin/network_info b/bin/network_info
deleted file mode 100755
index 9c20b89..0000000
--- a/bin/network_info
+++ /dev/null
@@ -1,81 +0,0 @@
-#!/usr/bin/env python3
-
-import os
-import sys
-import subprocess
-import socket
-import fcntl
-import struct
-
-SYS_PATH = '/sys/class/net'
-
-
-def _read_file(file):
-    source = open(file, 'r')
-    content = source.read()
-    source.close()
-    return content
-
-
-def get_connected(interface):
-    STATUS = ('No', 'Yes')
-    carrier_file = os.path.join(SYS_PATH, interface, 'carrier')
-
-    carrier = 0
-    try:
-        carrier = int(_read_file(carrier_file))
-    except IOError:
-        pass
-
-    return STATUS[carrier]
-
-
-def get_ip_address(interface):
-    s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
-    return socket.inet_ntoa(fcntl.ioctl(
-        s.fileno(),
-        0x8915,  # SIOCGIFADDR
-        struct.pack('256s', interface[:15].encode())
-    )[20:24])
-
-
-def get_ipv6_address(interface):
-    cmd = ['/sbin/ip', '-6', 'addr', 'show', 'dev', interface]
-    proc = subprocess.check_output(cmd, universal_newlines=True)
-    ipaddr = proc.split()[8].strip()
-    return ipaddr
-
-
-def get_mac_address(interface):
-    address_file = os.path.join(SYS_PATH, interface, 'address')
-
-    address = ''
-    try:
-        address = _read_file(address_file)
-    except IOError:
-        pass
-
-    return address
-
-
-def main(args):
-    for interface in args:
-        connected = get_connected(interface)
-        print("Interface: %s" % interface)
-        print("Connected: %s" % connected)
-        try:
-            print("IPv4: %s" % get_ip_address(interface))
-        except IOError:
-            print("IPv4: n/a")
-        try:
-            print("IPv6: %s" % get_ipv6_address(interface))
-        except IOError:
-            print("IPv6: n/a")
-        except:
-            print("IPv6: n/a")
-        print("MAC: %s\n" % get_mac_address(interface))
-
-    return 0
-
-if __name__ == "__main__":
-    sys.exit(main(sys.argv[1:]))
diff --git a/bin/network_reconnect_resume_test b/bin/network_reconnect_resume_test
deleted file mode 100755
index 8861eb6..0000000
--- a/bin/network_reconnect_resume_test
+++ /dev/null
@@ -1,102 +0,0 @@
-#!/usr/bin/env python3
-
-# Copyright (C) 2012 Canonical, Ltd.
-
-import re
-import subprocess
-import argparse
-import sys
-
-
-def get_time_difference(device):
-    """
-    Returns the difference in seconds between the last resume from suspend (S3)
-    and the time it took to reconnect to Wifi.  If there is a problem finding
-    the information, None is returned.
-    """
-    resume_time = get_resume_time()
-    if resume_time is None:
-        print("Unable to obtain wakeup/resume time from dmesg."
-              "Please be sure the system has been suspended", file=sys.stderr)
-        return None
-    if device == "wifi":
-        reconnect_times = list(get_wifi_reconnect_times())
-    elif device == "wired":
-        reconnect_times = list(get_wired_reconnect_times())
-
-    if not reconnect_times:
-        print("Unable to obtain %s connection time after a S3. Please be sure"
-              " that the system has been suspended" % device, file=sys.stderr)
-        return None
-
-    # since some wifi & wired tests can disconnect and reconnect us multiple
-    # times after a suspend, we need to find the reconnect that occurs
-    # immediately after the resume from S3
-    for reconnect_time in reconnect_times:
-        if reconnect_time >= resume_time:
-            return round((reconnect_time - resume_time), 2)
-    return None
-
-
-def get_wifi_reconnect_times():
-    """
-    Returns a list of all the timestamps for wifi reconnects.
-    """
-    data = subprocess.check_output(['dmesg'], universal_newlines=True)
-    syntax = re.compile("\[(.*)\] wlan.* associated")
-    results = re.findall(syntax, data)
-    return map(float, results)
-
-def get_wired_reconnect_times():
-    """
-    Returns a list of all the timestamps for wired reconnects.
-    """
-    data = subprocess.check_output(['dmesg'], universal_newlines=True)
-    syntax = re.compile("\[(.*)\].*eth.* Link is [uU]p")
-    results = re.findall(syntax, data)
-    return map(float, results)
-
-
-def get_resume_time():
-    """
-    Returns the last (most recent) timestamp for an ACPI resume from sleep (S3)
-    If no resume is found, None is returned.
-    """
-    data = subprocess.check_output(['dmesg'], universal_newlines=True)
-    syntax = re.compile("\[(.*)\].ACPI: Waking up from system sleep state S3")
-    results = re.findall(syntax, data)
-    if not results:
-        return None
-    else:
-        return float(results[-1])
-
-
-def main():
-    parser = argparse.ArgumentParser()
-    parser.add_argument('-t', '--timeout',
-                        type=int,
-                        help="Specified max time allowed for Wifi/Wired to"
-                             " reconnect in seconds",
-                        required=True)
-    parser.add_argument('-d', '--device',
-                        help="Specify the device to test either,  eth or wlan",
-                        required=True,
-                        choices=['wifi', 'wired'])
-    args = parser.parse_args()
-
-    timedif = get_time_difference(args.device)
-    
-    if not timedif:
-        return 0
-
-    print("Your %s resumed in %s seconds after the last suspend" % (
-        args.device, timedif))
-    if timedif > args.timeout:
-        print("FAIL: the network failed to reconnect within the allotted time")
-        return 1
-    else:
-        print("PASS: the network connected within the allotted time")
-        return 0
-
-if __name__ == "__main__":
-    sys.exit(main())
diff --git a/bin/removable_storage_test b/bin/removable_storage_test
deleted file mode 100755
index ddd4e1b..0000000
--- a/bin/removable_storage_test
+++ /dev/null
@@ -1,870 +0,0 @@
-#!/usr/bin/env python3
-
-import argparse
-import collections
-import dbus
-import hashlib
-import logging
-import os
-import re
-import shlex
-import subprocess
-import sys
-import tempfile
-import time
-
-import gi
-gi.require_version('GUdev', '1.0')
-from gi.repository import GUdev
-
-from checkbox_support.dbus import connect_to_system_bus
-from checkbox_support.dbus.udisks2 import UDISKS2_BLOCK_INTERFACE
-from checkbox_support.dbus.udisks2 import UDISKS2_DRIVE_INTERFACE
-from checkbox_support.dbus.udisks2 import UDISKS2_FILESYSTEM_INTERFACE
-from checkbox_support.dbus.udisks2 import UDisks2Model, UDisks2Observer
-from checkbox_support.dbus.udisks2 import is_udisks2_supported
-from checkbox_support.dbus.udisks2 import lookup_udev_device
-from checkbox_support.dbus.udisks2 import map_udisks1_connection_bus
-from checkbox_support.heuristics.udisks2 import is_memory_card
-from checkbox_support.helpers.human_readable_bytes import HumanReadableBytes
-from checkbox_support.parsers.udevadm import CARD_READER_RE
-from checkbox_support.parsers.udevadm import GENERIC_RE
-from checkbox_support.parsers.udevadm import FLASH_RE
-from checkbox_support.parsers.udevadm import find_pkname_is_root_mountpoint
-from checkbox_support.udev import get_interconnect_speed
-from checkbox_support.udev import get_udev_block_devices
-from checkbox_support.udev import get_udev_xhci_devices
-
-
-class ActionTimer():
-    '''Class to implement a simple timer'''
-    def __enter__(self):
-        self.start = time.time()
-        return self
-
-    def __exit__(self, *args):
-        self.stop = time.time()
-        self.interval = self.stop - self.start
-
-
-class RandomData():
-    '''Class to create data files'''
-    def __init__(self, size):
-        self.tfile = tempfile.NamedTemporaryFile(delete=False)
-        self.path = ''
-        self.name = ''
-        self.path, self.name = os.path.split(self.tfile.name)
-        self._write_test_data_file(size)
-
-    def _generate_test_data(self):
-        seed = "104872948765827105728492766217823438120"
-        phrase = '''
-        Lorem ipsum dolor sit amet, consectetuer adipiscing elit, sed diam
-        nonummy nibh euismod tincidunt ut laoreet dolore magna aliquam erat
-        volutpat. Ut wisi enim ad minim veniam, quis nostrud exerci tation
-        ullamcorper suscipit lobortis nisl ut aliquip ex ea commodo consequat.
-        Duis autem vel eum iriure dolor in hendrerit in vulputate velit esse
-        molestie consequat, vel illum dolore eu feugiat nulla facilisis at vero
-        eros et accumsan et iusto odio dignissim qui blandit praesent luptatum
-        zzril delenit augue duis dolore te feugait nulla facilisi.
-        '''
-        words = phrase.replace('\n', '').split()
-        word_deque = collections.deque(words)
-        seed_deque = collections.deque(seed)
-        while True:
-            yield ' '.join(list(word_deque))
-            word_deque.rotate(int(seed_deque[0]))
-            seed_deque.rotate(1)
-
-    def _write_test_data_file(self, size):
-        data = self._generate_test_data()
-        while os.path.getsize(self.tfile.name) < size:
-            self.tfile.write(next(data).encode('UTF-8'))
-        return self
-
-
-def md5_hash_file(path):
-    md5 = hashlib.md5()
-    try:
-        with open(path, 'rb') as stream:
-            while True:
-                data = stream.read(8192)
-                if not data:
-                    break
-                md5.update(data)
-    except IOError as exc:
-        logging.error("unable to checksum %s: %s", path, exc)
-        return None
-    else:
-        return md5.hexdigest()
-
-
-class DiskTest():
-    ''' Class to contain various methods for testing removable disks '''
-
-    def __init__(self, device, memorycard, lsblkcommand):
-        self.rem_disks = {}     # mounted before the script running
-        self.rem_disks_nm = {}  # not mounted before the script running
-        self.rem_disks_memory_cards = {}
-        self.rem_disks_memory_cards_nm = {}
-        self.rem_disks_speed = {}
-        # LP: #1313581, TODO: extend to be rem_disks_driver
-        self.rem_disks_xhci = {}
-        self.data = ''
-        self.lsblk = ''
-        self.device = device
-        self.memorycard = memorycard
-        self._run_lsblk(lsblkcommand)
-        self._probe_disks()
-
-    def read_file(self, source):
-        with open(source, 'rb') as infile:
-            try:
-                self.data = infile.read()
-            except IOError as exc:
-                logging.error("Unable to read data from %s: %s", source, exc)
-                return False
-            else:
-                return True
-
-    def write_file(self, data, dest):
-        try:
-            outfile = open(dest, 'wb', 0)
-        except OSError as exc:
-            logging.error("Unable to open %s for writing.", dest)
-            logging.error("  %s", exc)
-            return False
-        with outfile:
-            try:
-                outfile.write(self.data)
-            except IOError as exc:
-                logging.error("Unable to write data to %s: %s", dest, exc)
-                return False
-            else:
-                outfile.flush()
-                os.fsync(outfile.fileno())
-                return True
-
-    def clean_up(self, target):
-        try:
-            os.unlink(target)
-        except OSError as exc:
-            logging.error("Unable to remove tempfile %s", target)
-            logging.error("  %s", exc)
-
-    def _find_parent(self, device):
-        if self.lsblk:
-            pattern = re.compile('KNAME="(?P<KNAME>.*)" '
-                                 'TYPE="(?P<TYPE>.*)" '
-                                 'MOUNTPOINT="(?P<MOUNTPOINT>.*)"')
-            for line in self.lsblk.splitlines():
-                m = pattern.match(line)
-                if m and device.startswith(m.group('KNAME')):
-                    return m.group('KNAME')
-        return False
-
-    def _run_lsblk(self, lsblkcommand):
-        try:
-            self.lsblk = subprocess.check_output(shlex.split(lsblkcommand),
-                                                 universal_newlines=True)
-        except subprocess.CalledProcessError as exc:
-            raise SystemExit(exc)
-
-    def _probe_disks(self):
-        """
-        Internal method used to probe for available disks
-
-        Indirectly sets:
-            self.rem_disks{,_nm,_memory_cards,_memory_cards_nm,_speed}
-        """
-        if "SNAP" in os.environ:
-            self._probe_disks_udisks2_cli()
-        else:
-            bus, loop = connect_to_system_bus()
-            if is_udisks2_supported(bus):
-                self._probe_disks_udisks2(bus)
-            else:
-                self._probe_disks_udisks1(bus)
-
-    def _probe_disks_udisks2_cli(self):
-        # First we will build up a db of udisks info by scraping the output
-        # of the dump command
-        # TODO: remove the snap prefix when the alias becomes available
-        proc = subprocess.Popen(['udisks2.udisksctl', 'dump'],
-                                stdout=subprocess.PIPE)
-        udisks_devices = {}
-        current_bd = None
-        current_interface = None
-        while True:
-            line = proc.stdout.readline().decode(sys.stdout.encoding)
-            if line == '':
-                break
-            if line == '\n':
-                current_bd = None
-                current_interface = None
-            if line.startswith('/org/freedesktop/UDisks2/'):
-                path = line.strip()
-                current_bd = os.path.basename(path).rstrip(':')
-                udisks_devices[current_bd] = {}
-                continue
-            if current_bd is None:
-                continue
-            if line.startswith('  org.freedesktop'):
-                current_interface = line.strip().rstrip(':')
-                udisks_devices[current_bd][current_interface] = {}
-                continue
-            if current_interface is None:
-                continue
-            entry = ''.join(c for c in line if c not in '\n\t\' ')
-            wanted_keys = ('Device:', 'Drive:', 'MountPoints:', 'Vendor:',
-                           'ConnectionBus:', 'Model:', 'Media:',)
-            for key in wanted_keys:
-                if entry.startswith(key):
-                    udisks_devices[current_bd][current_interface][key] = (
-                        entry[len(key):])
-
-        # Now use the populated udisks structure to fill out the API used by
-        # other _probe disks functions
-        for device, interfaces in udisks_devices.items():
-            # iterate over udisks objects that have both filesystem and
-            # block device interfaces
-            if (UDISKS2_FILESYSTEM_INTERFACE in interfaces and
-                    UDISKS2_BLOCK_INTERFACE in interfaces):
-                # To be an IO candidate there must be a drive object
-                drive = interfaces[UDISKS2_BLOCK_INTERFACE].get('Drive:')
-                if drive is None or drive is '/':
-                    continue
-                drive_object = udisks_devices[os.path.basename(drive)]
-
-                # Get the connection bus property from the drive interface of
-                # the drive object. This is required to filter out the devices
-                # we don't want to look at now.
-                connection_bus = (
-                    drive_object[UDISKS2_DRIVE_INTERFACE]['ConnectionBus:'])
-                desired_connection_buses = set([
-                    map_udisks1_connection_bus(device)
-                    for device in self.device])
-                # Skip devices that are attached to undesired connection buses
-                if connection_bus not in desired_connection_buses:
-                    continue
-
-                dev_file = (
-                    interfaces[UDISKS2_BLOCK_INTERFACE].get('Device:'))
-
-                parent = self._find_parent(dev_file.replace('/dev/', ''))
-                if (parent and
-                        find_pkname_is_root_mountpoint(parent, self.lsblk)):
-                    continue
-
-                # XXX: we actually only scrape the first one currently
-                mount_point = (
-                    interfaces[UDISKS2_FILESYSTEM_INTERFACE].get(
-                        'MountPoints:'))
-                if mount_point == '':
-                    mount_point = None
-
-                # We need to skip-non memory cards if we look for memory cards
-                # and vice-versa so let's inspect the drive and use heuristics
-                # to detect memory cards (a memory card reader actually) now.
-                if self.memorycard != is_memory_card(
-                        drive_object[UDISKS2_DRIVE_INTERFACE]['Vendor:'],
-                        drive_object[UDISKS2_DRIVE_INTERFACE]['Model:'],
-                        drive_object[UDISKS2_DRIVE_INTERFACE]['Media:']):
-                    continue
-
-                if mount_point is None:
-                    self.rem_disks_memory_cards_nm[dev_file] = None
-                    self.rem_disks_nm[dev_file] = None
-                else:
-                    self.rem_disks_memory_cards[dev_file] = mount_point
-                    self.rem_disks[dev_file] = mount_point
-
-                # Get the speed of the interconnect that is associated with the
-                # block device we're looking at. This is purely informational
-                # but it is a part of the required API
-                udev_devices = get_udev_block_devices(GUdev.Client())
-                for udev_device in udev_devices:
-                    if udev_device.get_device_file() == dev_file:
-                        interconnect_speed = get_interconnect_speed(udev_device)
-                        if interconnect_speed:
-                            self.rem_disks_speed[dev_file] = (
-                                interconnect_speed * 10 ** 6)
-                        else:
-                            self.rem_disks_speed[dev_file] = None
-
-    def _probe_disks_udisks2(self, bus):
-        """
-        Internal method used to probe / discover available disks using udisks2
-        dbus interface using the provided dbus bus (presumably the system bus)
-        """
-        # We'll need udisks2 and udev to get the data we need
-        udisks2_observer = UDisks2Observer()
-        udisks2_model = UDisks2Model(udisks2_observer)
-        udisks2_observer.connect_to_bus(bus)
-        udev_client = GUdev.Client()
-        # Get a collection of all udev devices corresponding to block devices
-        udev_devices = get_udev_block_devices(udev_client)
-        # Get a collection of all udisks2 objects
-        udisks2_objects = udisks2_model.managed_objects
-        # Let's get a helper to simplify the loop below
-
-        def iter_filesystems_on_block_devices():
-            """
-            Generate a collection of UDisks2 object paths that
-            have both the filesystem and block device interfaces
-            """
-            for udisks2_object_path, interfaces in udisks2_objects.items():
-                if (UDISKS2_FILESYSTEM_INTERFACE in interfaces and
-                        UDISKS2_BLOCK_INTERFACE in interfaces):
-                    yield udisks2_object_path
-        # We need to know about all IO candidates,
-        # let's iterate over all the block devices reported by udisks2
-        for udisks2_object_path in iter_filesystems_on_block_devices():
-            # Get interfaces implemented by this object
-            udisks2_object = udisks2_objects[udisks2_object_path]
-            # Find the path of the udisks2 object that represents the drive
-            # this object is a part of
-            drive_object_path = (
-                udisks2_object[UDISKS2_BLOCK_INTERFACE]['Drive'])
-            # Lookup the drive object, if any. This can fail when
-            try:
-                drive_object = udisks2_objects[drive_object_path]
-            except KeyError:
-                logging.error(
-                    "Unable to locate drive associated with %s",
-                    udisks2_object_path)
-                continue
-            else:
-                drive_props = drive_object[UDISKS2_DRIVE_INTERFACE]
-            # Get the connection bus property from the drive interface of the
-            # drive object. This is required to filter out the devices we don't
-            # want to look at now.
-            connection_bus = drive_props["ConnectionBus"]
-            desired_connection_buses = set([
-                map_udisks1_connection_bus(device)
-                for device in self.device])
-            # Skip devices that are attached to undesired connection buses
-            if connection_bus not in desired_connection_buses:
-                continue
-            # Lookup the udev object that corresponds to this object
-            try:
-                udev_device = lookup_udev_device(udisks2_object, udev_devices)
-            except LookupError:
-                logging.error(
-                    "Unable to locate udev object that corresponds to: %s",
-                    udisks2_object_path)
-                continue
-            # Get the block device pathname,
-            # to avoid the confusion, this is something like /dev/sdbX
-            dev_file = udev_device.get_device_file()
-            parent = self._find_parent(dev_file.replace('/dev/', ''))
-            if parent and find_pkname_is_root_mountpoint(parent, self.lsblk):
-                continue
-            # Get the list of mount points of this block device
-            mount_points = (
-                udisks2_object[UDISKS2_FILESYSTEM_INTERFACE]['MountPoints'])
-            # Get the speed of the interconnect that is associated with the
-            # block device we're looking at. This is purely informational but
-            # it is a part of the required API
-            interconnect_speed = get_interconnect_speed(udev_device)
-            if interconnect_speed:
-                self.rem_disks_speed[dev_file] = (
-                    interconnect_speed * 10 ** 6)
-            else:
-                self.rem_disks_speed[dev_file] = None
-            # We need to skip-non memory cards if we look for memory cards and
-            # vice-versa so let's inspect the drive and use heuristics to
-            # detect memory cards (a memory card reader actually) now.
-            if self.memorycard != is_memory_card(drive_props['Vendor'],
-                                                 drive_props['Model'],
-                                                 drive_props['Media']):
-                continue
-            # The if/else test below simply distributes the mount_point to the
-            # appropriate variable, to keep the API requirements. It is
-            # confusing as _memory_cards is variable is somewhat dummy.
-            if mount_points:
-                # XXX: Arbitrarily pick the first of the mount points
-                mount_point = mount_points[0]
-                self.rem_disks_memory_cards[dev_file] = mount_point
-                self.rem_disks[dev_file] = mount_point
-            else:
-                self.rem_disks_memory_cards_nm[dev_file] = None
-                self.rem_disks_nm[dev_file] = None
-
-    def _probe_disks_udisks1(self, bus):
-        """
-        Internal method used to probe / discover available disks using udisks1
-        dbus interface using the provided dbus bus (presumably the system bus)
-        """
-        ud_manager_obj = bus.get_object("org.freedesktop.UDisks",
-                                        "/org/freedesktop/UDisks")
-        ud_manager = dbus.Interface(ud_manager_obj, 'org.freedesktop.UDisks')
-        for dev in ud_manager.EnumerateDevices():
-            device_obj = bus.get_object("org.freedesktop.UDisks", dev)
-            device_props = dbus.Interface(device_obj, dbus.PROPERTIES_IFACE)
-            udisks = 'org.freedesktop.UDisks.Device'
-            if not device_props.Get(udisks, "DeviceIsDrive"):
-                dev_bus = device_props.Get(udisks, "DriveConnectionInterface")
-                if dev_bus in self.device:
-                    parent_model = parent_vendor = ''
-                    if device_props.Get(udisks, "DeviceIsPartition"):
-                        parent_obj = bus.get_object(
-                            "org.freedesktop.UDisks",
-                            device_props.Get(udisks, "PartitionSlave"))
-                        parent_props = dbus.Interface(
-                            parent_obj, dbus.PROPERTIES_IFACE)
-                        parent_model = parent_props.Get(udisks, "DriveModel")
-                        parent_vendor = parent_props.Get(udisks, "DriveVendor")
-                        parent_media = parent_props.Get(udisks, "DriveMedia")
-                    if self.memorycard:
-                        if (dev_bus != 'sdio' and not
-                                FLASH_RE.search(parent_media) and not
-                                CARD_READER_RE.search(parent_model) and not
-                                GENERIC_RE.search(parent_vendor)):
-                            continue
-                    else:
-                        if (FLASH_RE.search(parent_media) or
-                                CARD_READER_RE.search(parent_model) or
-                                GENERIC_RE.search(parent_vendor)):
-                            continue
-                    dev_file = str(device_props.Get(udisks, "DeviceFile"))
-                    dev_speed = str(device_props.Get(udisks,
-                                                     "DriveConnectionSpeed"))
-                    self.rem_disks_speed[dev_file] = dev_speed
-                    if len(device_props.Get(udisks, "DeviceMountPaths")) > 0:
-                        devPath = str(device_props.Get(udisks,
-                                                       "DeviceMountPaths")[0])
-                        self.rem_disks[dev_file] = devPath
-                        self.rem_disks_memory_cards[dev_file] = devPath
-                    else:
-                        self.rem_disks_nm[dev_file] = None
-                        self.rem_disks_memory_cards_nm[dev_file] = None
-
-    def get_disks_xhci(self):
-        """
-        Compare
-        1. the pci slot name of the devices using xhci
-        2. the pci slot name of the disks,
-           which is usb3 disks in this case so far,
-        to make sure the usb3 disk does be on the controller using xhci
-        """
-        # LP: #1378724
-        udev_client = GUdev.Client()
-        # Get a collection of all udev devices corresponding to block devices
-        udev_devices = get_udev_block_devices(udev_client)
-        # Get a collection of all udev devices corresponding to xhci devices
-        udev_devices_xhci = get_udev_xhci_devices(udev_client)
-        for udev_device_xhci in udev_devices_xhci:
-            pci_slot_name = udev_device_xhci.get_property('PCI_SLOT_NAME')
-            for udev_device in udev_devices:
-                devpath = udev_device.get_property('DEVPATH')
-                if (self._compare_pci_slot_from_devpath(devpath,
-                                                        pci_slot_name)):
-                    self.rem_disks_xhci[
-                        udev_device.get_property('DEVNAME')] = 'xhci'
-        return self.rem_disks_xhci
-
-    def mount(self):
-        passed_mount = {}
-
-        for key in self.rem_disks_nm:
-            temp_dir = tempfile.mkdtemp()
-            if self._mount(key, temp_dir) != 0:
-                logging.error("can't mount %s", key)
-            else:
-                passed_mount[key] = temp_dir
-
-        if len(self.rem_disks_nm) == len(passed_mount):
-            self.rem_disks_nm = passed_mount
-            return 0
-        else:
-            count = len(self.rem_disks_nm) - len(passed_mount)
-            self.rem_disks_nm = passed_mount
-            return count
-
-    def _mount(self, dev_file, mount_point):
-        return subprocess.call(['mount', dev_file, mount_point])
-
-    def umount(self):
-        errors = 0
-        for disk in self.rem_disks_nm:
-            if not self.rem_disks_nm[disk]:
-                continue
-            if self._umount(disk) != 0:
-                errors += 1
-                logging.error("can't umount %s on %s",
-                              disk, self.rem_disks_nm[disk])
-        return errors
-
-    def _umount(self, mount_point):
-        # '-l': lazy umount, dealing problem of unable to umount the device.
-        return subprocess.call(['umount', '-l', mount_point])
-
-    def clean_tmp_dir(self):
-        for disk in self.rem_disks_nm:
-            if not self.rem_disks_nm[disk]:
-                continue
-            if not os.path.ismount(self.rem_disks_nm[disk]):
-                os.rmdir(self.rem_disks_nm[disk])
-
-    def _compare_pci_slot_from_devpath(self, devpath, pci_slot_name):
-        # LP: #1334991
-        # a smarter parser to get and validate a pci slot name from DEVPATH
-        # then compare this pci slot name to the other
-        dl = devpath.split('/')
-        s = set([x for x in dl if dl.count(x) > 1])
-        if (
-            (pci_slot_name in dl) and
-            (dl.index(pci_slot_name) < dl.index('block')) and
-            (not(pci_slot_name in s))
-        ):
-            # 1. there is such pci_slot_name
-            # 2. sysfs topology looks like
-            #    DEVPATH = ....../pci_slot_name/....../block/......
-            # 3. pci_slot_name should be unique in DEVPATH
-            return True
-        else:
-            return False
-
-
-def main():
-    parser = argparse.ArgumentParser()
-    parser.add_argument('device',
-                        choices=['usb', 'firewire', 'sdio',
-                                 'scsi', 'ata_serial_esata'],
-                        nargs='+',
-                        help=("The type of removable media "
-                              "(usb, firewire, sdio, scsi or ata_serial_esata)"
-                              "to test."))
-    parser.add_argument('-l', '--list',
-                        action='store_true',
-                        default=False,
-                        help="List the removable devices and mounting status")
-    parser.add_argument('-m', '--min-speed',
-                        action='store',
-                        default=0,
-                        type=int,
-                        help="Minimum speed a device must support to be "
-                             "considered eligible for being tested (bits/s)")
-    parser.add_argument('-p', '--pass-speed',
-                        action='store',
-                        default=0,
-                        type=int,
-                        help="Minimum average throughput from all eligible"
-                             "devices for the test to pass (MB/s)")
-    parser.add_argument('-i', '--iterations',
-                        action='store',
-                        default='1',
-                        type=int,
-                        help=("The number of test cycles to run. One cycle is"
-                              "comprised of generating --count data files of "
-                              "--size bytes and writing them to each device."))
-    parser.add_argument('-c', '--count',
-                        action='store',
-                        default='1',
-                        type=int,
-                        help='The number of random data files to generate')
-    parser.add_argument('-s', '--size',
-                        action='store',
-                        type=HumanReadableBytes,
-                        default='1MiB',
-                        help=("The size of the test data file to use. "
-                              "You may use SI or IEC suffixes like: 'K', 'M',"
-                              "'G', 'T', 'Ki', 'Mi', 'Gi', 'Ti', etc. Default"
-                              " is %(default)s"))
-    parser.add_argument('--auto-reduce-size',
-                        action='store_true',
-                        default=False,
-                        help=("Automatically reduce size to fit in the target"
-                              "filesystem. Reducing until fits in 1MiB"))
-    parser.add_argument('-n', '--skip-not-mount',
-                        action='store_true',
-                        default=False,
-                        help=("skip the removable devices "
-                              "which haven't been mounted before the test."))
-    parser.add_argument('--memorycard', action="store_true",
-                        help=("Memory cards devices on bus other than sdio "
-                              "require this parameter to identify "
-                              "them as such"))
-    parser.add_argument('--driver',
-                        choices=['xhci_hcd'],
-                        help=("Detect the driver of the host controller."
-                              "Only xhci_hcd for usb3 is supported so far."))
-    parser.add_argument("--lsblkcommand", action='store', type=str,
-                        default="lsblk -i -n -P -o KNAME,TYPE,MOUNTPOINT",
-                        help=("Command to execute to get lsblk information. "
-                              "Only change it if you know what you're doing."))
-
-    args = parser.parse_args()
-
-    test = DiskTest(args.device, args.memorycard, args.lsblkcommand)
-
-    errors = 0
-    # If we do have removable drives attached and mounted
-    if len(test.rem_disks) > 0 or len(test.rem_disks_nm) > 0:
-        if args.list:  # Simply output a list of drives detected
-            print('-' * 20)
-            print("Removable devices currently mounted:")
-            if args.memorycard:
-                if len(test.rem_disks_memory_cards) > 0:
-                    for disk, mnt_point in test.rem_disks_memory_cards.items():
-                        print("%s : %s" % (disk, mnt_point))
-                else:
-                    print("None")
-
-                print("Removable devices currently not mounted:")
-                if len(test.rem_disks_memory_cards_nm) > 0:
-                    for disk in test.rem_disks_memory_cards_nm:
-                        print(disk)
-                else:
-                    print("None")
-            else:
-                if len(test.rem_disks) > 0:
-                    for disk, mnt_point in test.rem_disks.items():
-                        print("%s : %s" % (disk, mnt_point))
-                else:
-                    print("None")
-
-                print("Removable devices currently not mounted:")
-                if len(test.rem_disks_nm) > 0:
-                    for disk in test.rem_disks_nm:
-                        print(disk)
-                else:
-                    print("None")
-
-            print('-' * 20)
-
-            return 0
-
-        else:  # Create a file, copy to disk and compare hashes
-            if args.skip_not_mount:
-                disks_all = test.rem_disks
-            else:
-                # mount those haven't be mounted yet.
-                errors_mount = test.mount()
-
-                if errors_mount > 0:
-                    print("There're total %d device(s) failed at mounting."
-                          % errors_mount)
-                    errors += errors_mount
-
-                disks_all = dict(list(test.rem_disks.items()) +
-                                 list(test.rem_disks_nm.items()))
-
-            if len(disks_all) > 0:
-                print("Found the following mounted %s partitions:"
-                      % ', '.join(args.device))
-
-                for disk, mount_point in disks_all.items():
-                    supported_speed = test.rem_disks_speed[disk]
-                    print("    %s : %s : %s bits/s" %
-                          (disk, mount_point, supported_speed),
-                          end="")
-                    if (args.min_speed and
-                            int(args.min_speed) > int(supported_speed)):
-                        print(" (Will not test it, speed is below %s bits/s)" %
-                              args.min_speed, end="")
-
-                    print("")
-
-                print('-' * 20)
-
-                disks_eligible = {disk: disks_all[disk] for disk in disks_all
-                                  if not args.min_speed or
-                                  int(test.rem_disks_speed[disk]) >=
-                                  int(args.min_speed)}
-                if len(disks_eligible) == 0:
-                    logging.error(
-                        "No %s disks with speed higher than %s bits/s",
-                        args.device, args.min_speed)
-                    return 1
-                write_sizes = []
-                test_files = {}
-                disks_freespace = {}
-                for disk, path in disks_eligible.items():
-                    stat = os.statvfs(path)
-                    disks_freespace[disk] = stat.f_bfree * stat.f_bsize
-                smallest_freespace = min(disks_freespace.values())
-                smallest_partition = [d for d, v in disks_freespace.items() if
-                                      v == smallest_freespace][0]
-                desired_size = args.size
-                if desired_size > smallest_freespace:
-                    if args.auto_reduce_size:
-                        min_space = HumanReadableBytes("1MiB")
-                        if smallest_freespace < min_space:
-                            sys.exit("Not enough space. {} is required on {}"
-                                     .format(min_space, smallest_partition))
-                        new_size = HumanReadableBytes(
-                            int(0.8 * smallest_freespace))
-                        logging.warning("Automatically reducing test data size"
-                                        ". {} requested. Reducing to {}."
-                                        .format(desired_size, new_size))
-                        desired_size = new_size
-                    else:
-                        sys.exit("Not enough space. {} is required on {}"
-                                 .format(desired_size, smallest_partition))
-                # Generate our data file(s)
-                for count in range(args.count):
-                    test_files[count] = RandomData(desired_size)
-                    write_sizes.append(os.path.getsize(
-                        test_files[count].tfile.name))
-                    total_write_size = sum(write_sizes)
-
-                try:
-                    for disk, mount_point in disks_eligible.items():
-                        print("%s (Total Data Size / iteration: %0.4f MB):" %
-                              (disk, (total_write_size / 1024 / 1024)))
-                        iteration_write_size = (
-                            total_write_size * args.iterations) / 1024 / 1024
-                        iteration_write_times = []
-                        for iteration in range(args.iterations):
-                            target_file_list = []
-                            write_times = []
-                            for file_index in range(args.count):
-                                parent_file = test_files[file_index].tfile.name
-                                parent_hash = md5_hash_file(parent_file)
-                                target_filename = (
-                                    test_files[file_index].name +
-                                    '.%s' % iteration)
-                                target_path = mount_point
-                                target_file = os.path.join(target_path,
-                                                           target_filename)
-                                target_file_list.append(target_file)
-                                test.read_file(parent_file)
-                                with ActionTimer() as timer:
-                                    if not test.write_file(test.data,
-                                                           target_file):
-                                        logging.error(
-                                            "Failed to copy %s to %s",
-                                            parent_file, target_file)
-                                        errors += 1
-                                        continue
-                                write_times.append(timer.interval)
-                                child_hash = md5_hash_file(target_file)
-                                if parent_hash != child_hash:
-                                    logging.warning(
-                                        "[Iteration %s] Parent and Child"
-                                        " copy hashes mismatch on %s!",
-                                        iteration, target_file)
-                                    logging.warning(
-                                        "\tParent hash: %s", parent_hash)
-                                    logging.warning(
-                                        "\tChild hash: %s", child_hash)
-                                    errors += 1
-                            for file in target_file_list:
-                                test.clean_up(file)
-                            total_write_time = sum(write_times)
-                            # avg_write_time = total_write_time / args.count
-                            try:
-                                avg_write_speed = ((
-                                    total_write_size / total_write_time) /
-                                    1024 / 1024)
-                            except ZeroDivisionError:
-                                avg_write_speed = 0.00
-                            finally:
-                                iteration_write_times.append(total_write_time)
-                                print("\t[Iteration %s] Average Speed: %0.4f"
-                                      % (iteration, avg_write_speed))
-                        for iteration in range(args.iterations):
-                            iteration_write_time = sum(iteration_write_times)
-                        print("\tSummary:")
-                        print("\t\tTotal Data Attempted: %0.4f MB"
-                              % iteration_write_size)
-                        print("\t\tTotal Time to write: %0.4f secs"
-                              % iteration_write_time)
-                        print("\t\tAverage Write Time: %0.4f secs" %
-                              (iteration_write_time / args.iterations))
-                        try:
-                            avg_write_speed = (iteration_write_size /
-                                               iteration_write_time)
-                        except ZeroDivisionError:
-                            avg_write_speed = 0.00
-                        finally:
-                            print("\t\tAverage Write Speed: %0.4f MB/s" %
-                                  avg_write_speed)
-                finally:
-                    for key in range(args.count):
-                        test.clean_up(test_files[key].tfile.name)
-                    if (len(test.rem_disks_nm) > 0):
-                        if test.umount() != 0:
-                            errors += 1
-                        test.clean_tmp_dir()
-
-                if errors > 0:
-                    logging.warning(
-                        "Completed %s test iterations, but there were"
-                        " errors", args.count)
-                    return 1
-                else:
-                    # LP: 1313581
-                    # Try to figure out whether the disk
-                    # is SuperSpeed USB and using xhci_hcd driver.
-                    if (args.driver == 'xhci_hcd'):
-                        # The speed reported by udisks is sometimes
-                        # less than 5G bits/s, for example,
-                        # it may be 705032705 bits/s
-                        # So using
-                        # 500000000
-                        # = 500 M bits/s
-                        # > 480 M bits/s ( USB 2.0 spec.)
-                        # to make sure that it is higher USB version than 2.0
-                        #
-                        # int() for int(test.rem_disks_speed[disk])
-                        # is necessary
-                        # because the speed value of
-                        # the dictionary rem_disks_speed is
-                        # 1. str or int from _probe_disks_udisks2
-                        # 2. int from _probe_disks_udisks1.
-                        # This is really a mess. : (
-                        print("\t\t--------------------------------")
-                        if(500000000 < int(test.rem_disks_speed[disk])):
-                            print("\t\tDevice Detected: SuperSpeed USB")
-                            # Unlike rem_disks_speed,
-                            # which must has the connect speed
-                            # for each disk devices,
-                            # disk devices may not use xhci as
-                            # controller drivers.
-                            # This will raise KeyError for no
-                            # associated disk device was found.
-                            xhci_disks = test.get_disks_xhci()
-                            # pep8 style suggest to limit the try clause
-                            # to the absolute minimum amount of code necessary
-                            try:
-                                disk_xhci_flag = xhci_disks[disk]
-                            except KeyError:
-                                print("\t\tDisk does not use xhci_hci.")
-                                return 1
-                            else:
-                                if('xhci' == disk_xhci_flag):
-                                    print("\t\tDriver Detected: xhci_hcd")
-                                else:
-                                    print("\t\tDisk does not use xhci_hci.")
-                                    logging.debug("disk_xhci_flag is not xhci")
-                                    return 1
-                        else:
-                            # Give it a hint for the detection failure.
-                            # LP: #1362902
-                            print(("\t\tNo SuperSpeed USB using xhci_hcd "
-                                   "was detected correctly."))
-                            print(("\t\tHint: please use dmesg to check "
-                                   "the system status again."))
-                            return 1
-                    # Pass is not assured
-                    if (not args.pass_speed or
-                            avg_write_speed >= args.pass_speed):
-                        return 0
-                    else:
-                        print("FAIL: Average speed was lower than desired "
-                              "pass speed of %s MB/s" % args.pass_speed)
-                        return 1
-            else:
-                logging.error("No device being mounted successfully "
-                              "for testing, aborting")
-                return 1
-
-    else:  # If we don't have removable drives attached and mounted
-        logging.error("No removable drives were detected, aborting")
-        return 1
-
-if __name__ == '__main__':
-    sys.exit(main())
diff --git a/bin/removable_storage_watcher b/bin/removable_storage_watcher
deleted file mode 100755
index 88cae9b..0000000
--- a/bin/removable_storage_watcher
+++ /dev/null
@@ -1,901 +0,0 @@
-#!/usr/bin/env python3
-
-import argparse
-import collections
-import copy
-import dbus
-import logging
-import sys
-
-import gi
-gi.require_version('GUdev', '1.0')
-from gi.repository import GObject, GUdev
-
-from checkbox_support.dbus import connect_to_system_bus
-from checkbox_support.dbus.udisks2 import UDisks2Model, UDisks2Observer
-from checkbox_support.dbus.udisks2 import is_udisks2_supported
-from checkbox_support.dbus.udisks2 import lookup_udev_device
-from checkbox_support.dbus.udisks2 import map_udisks1_connection_bus
-from checkbox_support.heuristics.udisks2 import is_memory_card
-from checkbox_support.parsers.udevadm import CARD_READER_RE, GENERIC_RE, FLASH_RE
-from checkbox_support.udev import get_interconnect_speed, get_udev_block_devices
-
-# Record representing properties of a UDisks1 Drive object needed by the
-# UDisks1 version of the watcher implementation
-UDisks1DriveProperties = collections.namedtuple(
-    'UDisks1DriveProperties', 'file bus speed model vendor media')
-
-# Delta record that encapsulates difference:
-# delta_dir -- directon of the difference, either DELTA_DIR_PLUS or
-#              DELTA_DIR_MINUS
-# value -- the actual value being removed or added, either InterfaceDelta or
-# PropertyDelta instance, see below
-DeltaRecord = collections.namedtuple("DeltaRecord", "delta_dir value")
-
-# Delta value for representing interface changes
-InterfaceDelta = collections.namedtuple(
-    "InterfaceDelta",
-    "delta_type object_path iface_name")
-
-# Delta value for representing property changes
-PropertyDelta = collections.namedtuple(
-    "PropertyDelta",
-    "delta_type object_path iface_name prop_name prop_value")
-
-# Tokens that encode additions and removals
-DELTA_DIR_PLUS = '+'
-DELTA_DIR_MINUS = '-'
-
-# Tokens that encode interface and property deltas
-DELTA_TYPE_IFACE = 'i'
-DELTA_TYPE_PROP = 'p'
-
-
-def format_bytes(size):
-    """
-    Format size to be easily read by humans
-
-    The result is disk-size compatible (using multiples of 10
-    rather than 2) string like "4.5GB"
-    """
-    for index, prefix in enumerate(" KMGTPEZY", 0):
-        factor = 10 ** (index * 3)
-        if size // factor <= 1000:
-            break
-    return "{}{}B".format(size // factor, prefix.strip())
-
-
-class UDisks1StorageDeviceListener:
-
-    def __init__(self, system_bus, loop, action, devices, minimum_speed,
-                 memorycard):
-        self._action = action
-        self._devices = devices
-        self._minimum_speed = minimum_speed
-        self._memorycard = memorycard
-        self._bus = system_bus
-        self._loop = loop
-        self._error = False
-        self._change_cache = []
-
-    def check(self, timeout):
-        udisks = 'org.freedesktop.UDisks'
-        if self._action == 'insert':
-            signal = 'DeviceAdded'
-            logging.debug("Adding signal listener for %s.%s", udisks, signal)
-            self._bus.add_signal_receiver(self.add_detected,
-                                          signal_name=signal,
-                                          dbus_interface=udisks)
-        elif self._action == 'remove':
-            signal = 'DeviceRemoved'
-            logging.debug("Adding signal listener for %s.%s", udisks, signal)
-            self._bus.add_signal_receiver(self.remove_detected,
-                                          signal_name=signal,
-                                          dbus_interface=udisks)
-
-        self._starting_devices = self.get_existing_devices()
-        logging.debug("Starting with the following devices: %r",
-                      self._starting_devices)
-
-        def timeout_callback():
-            print("%s seconds have expired "
-                  "waiting for the device to be inserted." % timeout)
-            self._error = True
-            self._loop.quit()
-
-        logging.debug("Adding timeout listener, timeout=%r", timeout)
-        GObject.timeout_add_seconds(timeout, timeout_callback)
-        logging.debug("Starting event loop...")
-        self._loop.run()
-
-        return self._error
-
-    def verify_device_change(self, changed_devices, message=""):
-        logging.debug("Verifying device change: %s", changed_devices)
-        # Filter the applicable bus types, as provided on the command line
-        # (values of self._devices can be 'usb', 'firewire', etc)
-        desired_bus_devices = [
-            device
-            for device in changed_devices
-            if device.bus in self._devices]
-        logging.debug("Desired bus devices: %s", desired_bus_devices)
-        for dev in desired_bus_devices:
-            if self._memorycard:
-                if (dev.bus != 'sdio'
-                    and not FLASH_RE.search(dev.media)
-                    and not CARD_READER_RE.search(dev.model)
-                    and not GENERIC_RE.search(dev.vendor)):
-                    logging.debug("The device does not seem to be a memory"
-                                  " card (bus: %r, model: %r), skipping",
-                                 dev.bus, dev.model)
-                    return
-                print(message % {'bus': 'memory card', 'file': dev.file})
-            else:
-                if (FLASH_RE.search(dev.media)
-                    or CARD_READER_RE.search(dev.model)
-                    or GENERIC_RE.search(dev.vendor)):
-                    logging.debug("The device seems to be a memory"
-                                  " card (bus: %r (model: %r), skipping",
-                                  dev.bus, dev.model)
-                    return
-                print(message % {'bus': dev.bus, 'file': dev.file})
-            if self._minimum_speed:
-                if dev.speed >= self._minimum_speed:
-                    print("with speed of %(speed)s bits/s "
-                          "higher than %(min_speed)s bits/s" %
-                          {'speed': dev.speed,
-                           'min_speed': self._minimum_speed})
-                else:
-                    print("ERROR: speed of %(speed)s bits/s lower "
-                          "than %(min_speed)s bits/s" %
-                          {'speed': dev.speed,
-                           'min_speed': self._minimum_speed})
-                    self._error = True
-            logging.debug("Device matches requirements, exiting event loop")
-            self._loop.quit()
-
-    def job_change_detected(self, devices, job_in_progress, job_id,
-                            job_num_tasks, job_cur_task_id,
-                            job_cur_task_percentage):
-        logging.debug("UDisks1 reports a job change has been detected:"
-                      " devices: %s, job_in_progress: %s, job_id: %s,"
-                      " job_num_tasks: %s, job_cur_task_id: %s,"
-                      " job_cur_task_percentage: %s",
-                      devices, job_in_progress, job_id, job_num_tasks,
-                      job_cur_task_id, job_cur_task_percentage)
-        if job_id == "FilesystemMount":
-            if devices in self._change_cache:
-                logging.debug("Ignoring filesystem mount,"
-                              " the device is present in change cache")
-                return
-            logging.debug("Adding devices to change cache: %r", devices)
-            self._change_cache.append(devices)
-            logging.debug("Starting devices were: %s", self._starting_devices)
-            current_devices = self.get_existing_devices()
-            logging.debug("Current devices are: %s", current_devices)
-            inserted_devices = list(set(current_devices) -
-                                    set(self._starting_devices))
-            logging.debug("Computed inserted devices: %s", inserted_devices)
-            if self._memorycard:
-                message = "Expected memory card device %(file)s inserted"
-            else:
-                message = "Expected %(bus)s device %(file)s inserted"
-            self.verify_device_change(inserted_devices,
-                      message=message)
-
-    def add_detected(self, added_path):
-        logging.debug("UDisks1 reports device has been added: %s", added_path)
-        logging.debug("Resetting change_cache to []")
-        self._change_cache = []
-        signal_name = 'DeviceJobChanged'
-        dbus_interface = 'org.freedesktop.UDisks'
-        logging.debug("Adding signal listener for %s.%s",
-                      dbus_interface, signal_name)
-        self._bus.add_signal_receiver(self.job_change_detected,
-                                      signal_name=signal_name,
-                                      dbus_interface=dbus_interface)
-
-    def remove_detected(self, removed_path):
-        logging.debug("UDisks1 reports device has been removed: %s",
-                      removed_path)
-
-        logging.debug("Starting devices were: %s", self._starting_devices)
-        current_devices = self.get_existing_devices()
-        logging.debug("Current devices are: %s", current_devices)
-        removed_devices = list(set(self._starting_devices) -
-                               set(current_devices))
-        logging.debug("Computed removed devices: %s", removed_devices)
-        self.verify_device_change(removed_devices,
-                  message="Removable %(bus)s device %(file)s has been removed")
-
-    def get_existing_devices(self):
-        logging.debug("Getting existing devices from UDisks1")
-        ud_manager_obj = self._bus.get_object("org.freedesktop.UDisks",
-                                              "/org/freedesktop/UDisks")
-        ud_manager = dbus.Interface(ud_manager_obj, 'org.freedesktop.UDisks')
-        existing_devices = []
-        for dev in ud_manager.EnumerateDevices():
-            try:
-                device_obj = self._bus.get_object("org.freedesktop.UDisks",
-                                                  dev)
-                device_props = dbus.Interface(device_obj,
-                                              dbus.PROPERTIES_IFACE)
-                udisks = 'org.freedesktop.UDisks.Device'
-                _device_file = device_props.Get(udisks,
-                                           "DeviceFile")
-                _bus = device_props.Get(udisks,
-                                           "DriveConnectionInterface")
-                _speed = device_props.Get(udisks,
-                                                "DriveConnectionSpeed")
-                _parent_model = ''
-                _parent_media = ''
-                _parent_vendor = ''
-
-                if device_props.Get(udisks, "DeviceIsPartition"):
-                    parent_obj = self._bus.get_object(
-                        "org.freedesktop.UDisks",
-                        device_props.Get(udisks, "PartitionSlave"))
-                    parent_props = dbus.Interface(
-                        parent_obj, dbus.PROPERTIES_IFACE)
-                    _parent_model = parent_props.Get(udisks, "DriveModel")
-                    _parent_vendor = parent_props.Get(udisks, "DriveVendor")
-                    _parent_media = parent_props.Get(udisks, "DriveMedia")
-
-                if not device_props.Get(udisks, "DeviceIsDrive"):
-                    device = UDisks1DriveProperties(
-                        file=str(_device_file),
-                        bus=str(_bus),
-                        speed=int(_speed),
-                        model=str(_parent_model),
-                        vendor=str(_parent_vendor),
-                        media=str(_parent_media))
-                    existing_devices.append(device)
-
-            except dbus.DBusException:
-                pass
-
-        return existing_devices
-
-
-def udisks2_objects_delta(old, new):
-    """
-    Compute the delta between two snapshots of udisks2 objects
-
-    The objects are encoded as {s:{s:{s:v}}} where the first dictionary maps
-    from DBus object path to a dictionary that maps from interface name to a
-    dictionary that finally maps from property name to property value.
-
-    The result is a generator of DeltaRecord objects that encodes the changes:
-        * the 'delta_dir' is either DELTA_DIR_PLUS or DELTA_DIR_MINUS
-        * the 'value' is a tuple that differs for interfaces and properties.
-          Interfaces use the format (DELTA_TYPE_IFACE, object_path, iface_name)
-          while properties use the format (DELTA_TYPE_PROP, object_path,
-          iface_name, prop_name, prop_value)
-
-    Interfaces are never "changed", they are only added or removed. Properties
-    can be changed and this is encoded as removal followed by an addition where
-    both differ only by the 'delta_dir' and the last element of the 'value'
-    tuple.
-    """
-    # Traverse all objects, old or new
-    all_object_paths = set()
-    all_object_paths |= old.keys()
-    all_object_paths |= new.keys()
-    for object_path in sorted(all_object_paths):
-        old_object = old.get(object_path, {})
-        new_object = new.get(object_path, {})
-        # Traverse all interfaces of each object, old or new
-        all_iface_names = set()
-        all_iface_names |= old_object.keys()
-        all_iface_names |= new_object.keys()
-        for iface_name in sorted(all_iface_names):
-            if iface_name not in old_object and iface_name in new_object:
-                # Report each ADDED interface
-                assert iface_name in new_object
-                delta_value = InterfaceDelta(
-                    DELTA_TYPE_IFACE, object_path, iface_name)
-                yield DeltaRecord(DELTA_DIR_PLUS, delta_value)
-                # Report all properties ADDED on that interface
-                for prop_name, prop_value in new_object[iface_name].items():
-                    delta_value = PropertyDelta(DELTA_TYPE_PROP, object_path,
-                                                iface_name, prop_name,
-                                                prop_value)
-                    yield DeltaRecord(DELTA_DIR_PLUS, delta_value)
-            elif iface_name not in new_object and iface_name in old_object:
-                # Report each REMOVED interface
-                assert iface_name in old_object
-                delta_value = InterfaceDelta(
-                    DELTA_TYPE_IFACE, object_path, iface_name)
-                yield DeltaRecord(DELTA_DIR_MINUS, delta_value)
-                # Report all properties REMOVED on that interface
-                for prop_name, prop_value in old_object[iface_name].items():
-                    delta_value = PropertyDelta(DELTA_TYPE_PROP, object_path,
-                                                iface_name, prop_name,
-                                                prop_value)
-                    yield DeltaRecord(DELTA_DIR_MINUS, delta_value)
-            else:
-                # Analyze properties of each interface that existed both in old
-                # and new object trees.
-                assert iface_name in new_object
-                assert iface_name in old_object
-                old_props = old_object[iface_name]
-                new_props = new_object[iface_name]
-                all_prop_names = set()
-                all_prop_names |= old_props.keys()
-                all_prop_names |= new_props.keys()
-                # Traverse all properties, old or new
-                for prop_name in sorted(all_prop_names):
-                    if prop_name not in old_props and prop_name in new_props:
-                        # Report each ADDED property
-                        delta_value = PropertyDelta(
-                            DELTA_TYPE_PROP, object_path, iface_name,
-                            prop_name, new_props[prop_name])
-                        yield DeltaRecord(DELTA_DIR_PLUS, delta_value)
-                    elif prop_name not in new_props and prop_name in old_props:
-                        # Report each REMOVED property
-                        delta_value = PropertyDelta(
-                            DELTA_TYPE_PROP, object_path, iface_name,
-                            prop_name, old_props[prop_name])
-                        yield DeltaRecord(DELTA_DIR_MINUS, delta_value)
-                    else:
-                        old_value = old_props[prop_name]
-                        new_value = new_props[prop_name]
-                        if old_value != new_value:
-                            # Report each changed property
-                            yield DeltaRecord(DELTA_DIR_MINUS, PropertyDelta(
-                                DELTA_TYPE_PROP, object_path, iface_name,
-                                prop_name, old_value))
-                            yield DeltaRecord(DELTA_DIR_PLUS, PropertyDelta(
-                                DELTA_TYPE_PROP, object_path, iface_name,
-                                prop_name, new_value))
-
-
-class UDisks2StorageDeviceListener:
-    """
-    Implementation of the storage device listener concept for UDisks2 backend.
-    Loosely modeled on the UDisks-based implementation above.
-
-    Implementation details
-    ^^^^^^^^^^^^^^^^^^^^^^
-
-    The class, once configured reacts to asynchronous events from the event
-    loop. Those are either DBus signals or GLib timeout.
-
-    The timeout, if reached, terminates the test and fails with an appropriate
-    end-user message. The user is expected to manipulate storage devices while
-    the test is running.
-
-    DBus signals (that correspond to UDisks2 DBus signals) cause callbacks into
-    this code. Each time a signal is reported "delta" is computed and verified
-    to determine if there was a successful match. The delta contains a list or
-    DeltaRecord objects that encode difference (either addition or removal) and
-    the value of the difference (interface name or interface property value).
-    This delta is computed by udisks2_objects_delta(). The delta is then passed
-    to _validate_delta() which has a chance to end the test but also prints
-    diagnostic messages in verbose mode. This is very useful for understanding
-    what the test actually sees occurring.
-
-    Insertion/removal detection strategy
-    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-    Compared to initial state, the following changes objects need to be
-    detected
-
-    * At least one UDisks2 object with the following _all_ interfaces:
-        * UDisks2.Partition
-            (because we want a partitioned device)
-        * UDisks2.Block
-            (because we want that device to have a block device that users can
-            format)
-            - having IdUsage == 'filesystem'
-            (because it should not be a piece of raid or lvm)
-            - having Size > 0
-            (because it should not be and empty removable storage reader)
-        * UDisks2.Filesystem
-            (because we want to ensure that a filesystem gets mounted)
-            - having MountPoints != []
-            - as a special exception this rule is REMOVED from eSATA and SATA
-              devices as they are not automatically mounted anymore.
-
-    This object must be traceable to an UDisks.Drive object:
-        (because we need the medium to be inserted somewhere)
-            - having ConnectionBus in (desired_connection_buses)
-            - as a special exception this rule is weakened for eSATA because
-              for such devices the ConnectionBus property is empty.
-    """
-
-    # Name of the DBus interface exposed UDisks2 for various drives
-    UDISKS2_DRIVE_INTERFACE = "org.freedesktop.UDisks2.Drive"
-
-    # Name of the DBus property provided by the "Drive" interface above
-    UDISKS2_DRIVE_PROPERTY_CONNECTION_BUS = "ConnectionBus"
-
-    def __init__(self, system_bus, loop, action, devices, minimum_speed,
-                 memorycard, unmounted = False):
-        # Store the desired minimum speed of the device in Mbit/s. The argument
-        # is passed as the number of bits per second so let's fix that.
-        self._desired_minimum_speed = minimum_speed / 10 ** 6
-        # Compute the allowed UDisks2.Drive.ConnectionBus value based on the
-        # legacy arguments passed from the command line.
-        self._desired_connection_buses = set([
-            map_udisks1_connection_bus(device) for device in devices])
-        # Check if we are explicitly looking for memory cards
-        self._desired_memory_card = memorycard
-        # Store information whether we also want detected, but unmounted
-        # devices too
-        self._allow_unmounted = unmounted
-        # Store the desired "delta" direction depending on
-        # whether we test for insertion or removal
-        if action == "insert":
-            self._desired_delta_dir = DELTA_DIR_PLUS
-        elif action == "remove":
-            self._desired_delta_dir = DELTA_DIR_MINUS
-        else:
-            raise ValueError("Unsupported action: {}".format(action))
-        # Store DBus bus object as we need to pass it to UDisks2 observer
-        self._bus = system_bus
-        # Store event loop object
-        self._loop = loop
-        # Setup UDisks2Observer class to track changes published by UDisks2
-        self._udisks2_observer = UDisks2Observer()
-        # Set the initial value of reference_objects.
-        # The actual value is only set once in check()
-        self._reference_objects = None
-        # As above, just initializing in init for sake of consistency
-        self._is_reference = None
-        # Setup UDisks2Model to know what the current state is. This is needed
-        # when remove events are reported as they don't carry enough state for
-        # the program to work correctly. Since UDisks2Model only applies the
-        # changes _after_ processing the signals from UDisks2Observer we can
-        # reliably check all of the properties of the removed object / device.
-        self._udisks2_model = UDisks2Model(self._udisks2_observer)
-        # Whenever anything changes call our local change handler
-        # This handler always computes the full delta (versus the
-        # reference state) and decides if we have a match or not
-        self._udisks2_model.on_change.connect(self._on_change)
-        # We may need an udev context for checking the speed of USB devices
-        self._udev_client = GUdev.Client()
-        # A snapshot of udev devices, set in check()
-        self._reference_udev_devices = None
-        # Assume the test passes, this is changed when timeout expires or when
-        # an incorrect device gets inserted.
-        self._error = False
-
-    def _dump_reference_udisks_objects(self):
-        logging.debug("Reference UDisks2 objects:")
-        for udisks2_object in self._reference_objects:
-            logging.debug(" - %s", udisks2_object)
-
-    def _dump_reference_udev_devices(self):
-        logging.debug("Reference udev devices:")
-        for udev_device in self._reference_udev_devices:
-            interconnect_speed = get_interconnect_speed(udev_device)
-            if interconnect_speed:
-                logging.debug(" - %s (USB %dMBit/s)",
-                              udev_device.get_device_file(),
-                              interconnect_speed)
-            else:
-                logging.debug(" - %s", udev_device.get_device_file())
-
-    def check(self, timeout):
-        """
-        Run the configured test and return the result
-
-        The result is False if the test has failed.  The timeout, when
-        non-zero, will make the test fail after the specified seconds have
-        elapsed without conclusive result.
-        """
-        # Setup a timeout if requested
-        if timeout > 0:
-            GObject.timeout_add_seconds(timeout, self._on_timeout_expired)
-        # Connect the observer to the bus. This will start giving us events
-        # (actually when the loop starts later below)
-        self._udisks2_observer.connect_to_bus(self._bus)
-        # Get the reference snapshot of available devices
-        self._reference_objects = copy.deepcopy(self._current_objects)
-        self._dump_reference_udisks_objects()
-        # Mark the current _reference_objects as ... reference, this is sadly
-        # needed by _summarize_changes() as it sees the snapshot _after_ a
-        # change has occurred and cannot determine if the slope of the 'edge'
-        # of the change. It is purely needed for UI in verbose mode
-        self._is_reference = True
-        # A collection of objects that we gladly ignore because we already
-        # reported on them being somehow inappropriate
-        self._ignored_objects = set()
-        # Get the reference snapshot of available udev devices
-        self._reference_udev_devices = get_udev_block_devices(
-            self._udev_client)
-        self._dump_reference_udev_devices()
-        # Start the loop and wait. The loop will exit either when:
-        # 1) A proper device has been detected (either insertion or removal)
-        # 2) A timeout (optional) has expired
-        self._loop.run()
-        # Return the outcome of the test
-        return self._error
-
-    def _on_timeout_expired(self):
-        """
-        Internal function called when the timer expires.
-
-        Basically it's just here to tell the user the test failed or that the
-        user was unable to alter the device during the allowed time.
-        """
-        print("You have failed to perform the required manipulation in time")
-        # Fail the test when the timeout was reached
-        self._error = True
-        # Stop the loop now
-        self._loop.quit()
-
-    def _on_change(self):
-        """
-        Internal method called by UDisks2Model whenever a change had occurred
-        """
-        # Compute the changes that had occurred since the reference point
-        delta_records = list(self._get_delta_records())
-        # Display a summary of changes when we are done
-        self._summarize_changes(delta_records)
-        # If the changes are what we wanted stop the loop
-        matching_devices = self._get_matching_devices(delta_records)
-        if matching_devices:
-            print("Expected device manipulation complete: {}".format(
-                ', '.join(matching_devices)))
-            # And call it a day
-            self._loop.quit()
-
-    def _get_matching_devices(self, delta_records):
-        """
-        Internal method called that checks if the delta records match the type
-        of device manipulation we were expecting. Only called from _on_change()
-
-        Returns a set of paths of block devices that matched
-        """
-        # Results
-        results = set()
-        # Group changes by DBus object path
-        grouped_records = collections.defaultdict(list)
-        for record in delta_records:
-            grouped_records[record.value.object_path].append(record)
-        # Create another snapshot od udev devices so that we don't do it over
-        # and over in the loop below (besides, if we did that then results
-        # could differ each time).
-        current_udev_devices = get_udev_block_devices(self._udev_client)
-        # Iterate over all UDisks2 objects and their delta records
-        for object_path, records_for_object in grouped_records.items():
-            # Skip objects we already ignored and complained about before
-            if object_path in self._ignored_objects:
-                continue
-            needs = set(('block-fs', 'partition', 'non-empty'))
-            if not self._allow_unmounted:
-                needs.add('mounted')
-
-            # As a special exception when the ConnectionBus is allowed to be
-            # empty, as is the case with eSATA devices, do not require the
-            # filesystem to be mounted as gvfs may choose not to mount it
-            # automatically.
-            found = set()
-            drive_object_path = None
-            object_block_device = None
-            for record in records_for_object:
-                # Skip changes opposite to the ones we need
-                if record.delta_dir != self._desired_delta_dir:
-                    continue
-                # For devices with empty "ConnectionBus" property, don't
-                # require the device to be mounted
-                if (record.value.iface_name ==
-                    "org.freedesktop.UDisks2.Drive"
-                    and record.value.delta_type == DELTA_TYPE_PROP
-                    and record.value.prop_name == "ConnectionBus"
-                    and record.value.prop_value == ""):
-                    needs.remove('mounted')
-                # Detect block devices designated for filesystems
-                if (record.value.iface_name ==
-                    "org.freedesktop.UDisks2.Block"
-                    and record.value.delta_type == DELTA_TYPE_PROP
-                    and record.value.prop_name == "IdUsage"
-                    and record.value.prop_value == "filesystem"):
-                    found.add('block-fs')
-                # Memorize the block device path
-                elif (record.value.iface_name ==
-                    "org.freedesktop.UDisks2.Block"
-                    and record.value.delta_type == DELTA_TYPE_PROP
-                    and record.value.prop_name == "PreferredDevice"):
-                    object_block_device = record.value.prop_value
-                # Ensure the device is a partition
-                elif (record.value.iface_name ==
-                      "org.freedesktop.UDisks2.Partition"
-                      and record.value.delta_type == DELTA_TYPE_IFACE):
-                    found.add('partition')
-                # Ensure the device is not empty
-                elif (record.value.iface_name ==
-                      "org.freedesktop.UDisks2.Block"
-                      and record.value.delta_type == DELTA_TYPE_PROP
-                      and record.value.prop_name == "Size"
-                      and record.value.prop_value > 0):
-                    found.add('non-empty')
-                # Ensure the filesystem is mounted
-                elif (record.value.iface_name ==
-                      "org.freedesktop.UDisks2.Filesystem"
-                      and record.value.delta_type == DELTA_TYPE_PROP
-                      and record.value.prop_name == "MountPoints"
-                      and record.value.prop_value != []):
-                    found.add('mounted')
-                # Finally memorize the drive the block device belongs to
-                elif (record.value.iface_name ==
-                      "org.freedesktop.UDisks2.Block"
-                      and record.value.delta_type == DELTA_TYPE_PROP
-                      and record.value.prop_name == "Drive"):
-                    drive_object_path = record.value.prop_value
-            logging.debug("Finished analyzing %s, found: %s, needs: %s"
-                          " drive_object_path: %s", object_path, found, needs,
-                          drive_object_path)
-            if needs != found or drive_object_path is None:
-                continue
-            # We've found our candidate, let's look at the drive it belongs
-            # to. We need to do this as some properties are associated with
-            # the drive, not the filesystem/block device and the drive may
-            # not have been inserted at all.
-            try:
-                drive_object = self._current_objects[drive_object_path]
-            except KeyError:
-                # The drive may be removed along with the device, let's check
-                # if we originally saw it
-                try:
-                    drive_object = self._reference_objects[drive_object_path]
-                except KeyError:
-                    logging.error(
-                        "A block device belongs to a drive we could not find")
-                    logging.error("missing drive: %r", drive_object_path)
-                    continue
-            try:
-                drive_props = drive_object["org.freedesktop.UDisks2.Drive"]
-            except KeyError:
-                logging.error(
-                    "A block device belongs to an object that is not a Drive")
-                logging.error("strange object: %r", drive_object_path)
-                continue
-            # Ensure the drive is on the appropriate bus
-            connection_bus = drive_props["ConnectionBus"]
-            if connection_bus not in self._desired_connection_buses:
-                logging.warning("The object %r belongs to drive %r that"
-                                " is attached to the bus %r but but we are"
-                                " looking for one of %r so it cannot match",
-                                object_block_device, drive_object_path,
-                                connection_bus,
-                                ", ".join(self._desired_connection_buses))
-                # Ignore this object so that we don't spam the user twice
-                self._ignored_objects.add(object_path)
-                continue
-            # Ensure it is a media card reader if this was explicitly requested
-            drive_is_reader = is_memory_card(
-                drive_props['Vendor'], drive_props['Model'],
-                drive_props['Media'])
-            if self._desired_memory_card and not drive_is_reader:
-                logging.warning(
-                    "The object %s belongs to drive %s that does not seem to"
-                    " be a media reader", object_block_device,
-                    drive_object_path)
-                # Ignore this object so that we don't spam the user twice
-                self._ignored_objects.add(object_path)
-                continue
-            # Ensure the desired minimum speed is enforced
-            if self._desired_minimum_speed:
-                # We need to discover the speed of the UDisks2 object that is
-                # about to be matched. Sadly UDisks2 no longer supports this
-                # property so we need to poke deeper and resort to udev.
-                #
-                # The UDisks2 object that we are interested in implements a
-                # number of interfaces, most notably
-                # org.freedesktop.UDisks2.Block, that has the Device property
-                # holding the unix filesystem path (like /dev/sdb1). We already
-                # hold a reference to that as 'object_block_device'
-                #
-                # We take this as a start and attempt to locate the udev Device
-                # (don't confuse with UDisks2.Device, they are _not_ the same)
-                # that is associated with that path.
-                if self._desired_delta_dir == DELTA_DIR_PLUS:
-                    # If we are looking for additions then look at _current_
-                    # collection of udev devices
-                    udev_devices = current_udev_devices
-                    udisks2_object = self._current_objects[object_path]
-                else:
-                    # If we are looking for removals then look at referece
-                    # collection of udev devices
-                    udev_devices = self._reference_udev_devices
-                    udisks2_object = self._reference_objects[object_path]
-                try:
-                    # Try to locate the corresponding udev device among the
-                    # collection we've selected. Use the drive object as the
-                    # key -- this looks for the drive, not partition objects!
-                    udev_device = lookup_udev_device(udisks2_object,
-                                                     udev_devices)
-                except LookupError:
-                    logging.error("Unable to map UDisks2 object %s to udev",
-                                  object_block_device)
-                    # Ignore this object so that we don't spam the user twice
-                    self._ignored_objects.add(object_path)
-                    continue
-                interconnect_speed = get_interconnect_speed(udev_device)
-                # Now that we know the speed of the interconnect we can try to
-                # validate it against our desired speed.
-                if interconnect_speed is None:
-                    logging.warning("Unable to determine interconnect speed of"
-                                    " device %s", object_block_device)
-                    # Ignore this object so that we don't spam the user twice
-                    self._ignored_objects.add(object_path)
-                    continue
-                elif interconnect_speed < self._desired_minimum_speed:
-                    logging.warning(
-                        "Device %s is connected via an interconnect that has"
-                        " the speed of %dMbit/s but the required speed was"
-                        " %dMbit/s", object_block_device, interconnect_speed,
-                        self._desired_minimum_speed)
-                    # Ignore this object so that we don't spam the user twice
-                    self._ignored_objects.add(object_path)
-                    continue
-                else:
-                    logging.info("Device %s is connected via an USB"
-                                 " interconnect with the speed of %dMbit/s",
-                                 object_block_device, interconnect_speed)
-            # Yay, success
-            results.add(object_block_device)
-        return results
-
-    @property
-    def _current_objects(self):
-        return self._udisks2_model.managed_objects
-
-    def _get_delta_records(self):
-        """
-        Internal method used to compute the delta between reference devices and
-        current devices. The result is a generator of DeltaRecord objects.
-        """
-        assert self._reference_objects is not None, "Only usable after check()"
-        old = self._reference_objects
-        new = self._current_objects
-        return udisks2_objects_delta(old, new)
-
-    def _summarize_changes(self, delta_records):
-        """
-        Internal method used to summarize changes (compared to reference state)
-        called whenever _on_change() gets called. Only visible in verbose mode
-        """
-        # Filter out anything but interface changes
-        flat_records = [record
-                        for record in delta_records
-                        if record.value.delta_type == DELTA_TYPE_IFACE]
-        # Group changes by DBus object path
-        grouped_records = collections.defaultdict(list)
-        for record in flat_records:
-            grouped_records[record.value.object_path].append(record)
-        # Bail out quickly when nothing got changed
-        if not flat_records:
-            if not self._is_reference:
-                logging.info("You have returned to the reference state")
-                self._is_reference = True
-            return
-        else:
-            self._is_reference = False
-        # Iterate over grouped delta records for all objects
-        logging.info("Compared to the reference state you have:")
-        for object_path in sorted(grouped_records.keys()):
-            records_for_object = sorted(
-                grouped_records[object_path],
-                key=lambda record: record.value.iface_name)
-            # Skip any job objects as they just add noise
-            if any((record.value.iface_name == "org.freedesktop.UDisks2.Job"
-                    for record in records_for_object)):
-                continue
-            logging.info("For object %s", object_path)
-            for record in records_for_object:
-                # Ignore property changes for now
-                if record.value.delta_type != DELTA_TYPE_IFACE:
-                    continue
-                # Get the name of the interface that was affected
-                iface_name = record.value.iface_name
-                # Get the properties for that interface (for removals get the
-                # reference values, for additions get the current values)
-                if record.delta_dir == DELTA_DIR_PLUS:
-                    props = self._current_objects[object_path][iface_name]
-                    action = "inserted"
-                else:
-                    props = self._reference_objects[object_path][iface_name]
-                    action = "removed"
-                # Display some human-readable information associated with each
-                # interface change
-                if iface_name == "org.freedesktop.UDisks2.Drive":
-                    logging.info("\t * %s a drive", action)
-                    logging.info("\t   vendor and name: %r %r",
-                                 props['Vendor'], props['Model'])
-                    logging.info("\t   bus: %s", props['ConnectionBus'])
-                    logging.info("\t   size: %s", format_bytes(props['Size']))
-                    logging.info("\t   is media card: %s", is_memory_card(
-                        props['Vendor'], props['Model'], props['Media']))
-                    logging.info("\t   current media: %s",
-                                 props['Media'] or "???" if
-                                 props['MediaAvailable'] else "N/A")
-                elif iface_name == "org.freedesktop.UDisks2.Block":
-                    logging.info("\t * %s block device", action)
-                    logging.info("\t   from drive: %s", props['Drive'])
-                    logging.info("\t   having device: %s", props['Device'])
-                    logging.info("\t   having usage, type and version:"
-                                 " %s %s %s", props['IdUsage'],
-                                 props['IdType'], props['IdVersion'])
-                    logging.info("\t   having label: %s", props['IdLabel'])
-                elif iface_name == "org.freedesktop.UDisks2.PartitionTable":
-                    logging.info("\t * %s partition table", action)
-                    logging.info("\t   having type: %r", props['Type'])
-                elif iface_name == "org.freedesktop.UDisks2.Partition":
-                    logging.info("\t * %s partition", action)
-                    logging.info("\t   from partition table: %s",
-                                 props['Table'])
-                    logging.info("\t   having size: %s",
-                                 format_bytes(props['Size']))
-                    logging.info("\t   having name: %r", props['Name'])
-                elif iface_name == "org.freedesktop.UDisks2.Filesystem":
-                    logging.info("\t * %s file system", action)
-                    logging.info("\t   having mount points: %r",
-                                 props['MountPoints'])
-
-
-def main():
-    description = "Wait for the specified device to be inserted or removed."
-    parser = argparse.ArgumentParser(description=description)
-    parser.add_argument('action', choices=['insert', 'remove'])
-    parser.add_argument('device', choices=['usb', 'sdio', 'firewire', 'scsi',
-                                 'ata_serial_esata'], nargs="+")
-    memorycard_help = ("Memory cards devices on bus other than sdio require "
-                       "this parameter to identify them as such")
-    parser.add_argument('--memorycard', action="store_true",
-                        help=memorycard_help)
-    parser.add_argument('--timeout', type=int, default=20)
-    min_speed_help = ("Will only accept a device if its connection speed "
-                      "attribute is higher than this value "
-                      "(in bits/s)")
-    parser.add_argument('--minimum_speed', '-m', help=min_speed_help,
-                        type=int, default=0)
-    parser.add_argument('--verbose', action='store_const', const=logging.INFO,
-                        dest='logging_level', help="Enable verbose output")
-    parser.add_argument('--debug', action='store_const', const=logging.DEBUG,
-                        dest='logging_level', help="Enable debugging")
-    parser.add_argument('--unmounted', action='store_true',
-                        help="Don't require drive being automounted")
-    parser.set_defaults(logging_level=logging.WARNING)
-    args = parser.parse_args()
-
-    # Configure logging as requested
-    # XXX: This may be incorrect as logging.basicConfig() fails after any other
-    # call to logging.log(). The proper solution is to setup a verbose logging
-    # configuration and I didn't want to do it now.
-    logging.basicConfig(
-        level=args.logging_level,
-        format='[%(asctime)s] %(levelname)s:%(name)s:%(message)s')
-
-    # Connect to the system bus, we also get the event
-    # loop as we need it to start listening for signals.
-    system_bus, loop = connect_to_system_bus()
-
-    # Check if system bus has the UDisks2 object
-    if is_udisks2_supported(system_bus):
-        # Construct the listener with all of the arguments provided on the
-        # command line and the explicit system_bus, loop objects.
-        logging.debug("Using UDisks2 interface")
-        listener = UDisks2StorageDeviceListener(
-            system_bus, loop,
-            args.action, args.device, args.minimum_speed, args.memorycard,
-            args.unmounted)
-    else:
-        # Construct the listener with all of the arguments provided on the
-        # command line and the explicit system_bus, loop objects.
-        logging.debug("Using UDisks1 interface")
-        listener = UDisks1StorageDeviceListener(
-            system_bus, loop,
-            args.action, args.device, args.minimum_speed, args.memorycard)
-    # Run the actual listener and wait till it either times out of discovers
-    # the appropriate media changes
-    try:
-        return listener.check(args.timeout)
-    except KeyboardInterrupt:
-        return 1
-
-if __name__ == "__main__":
-    sys.exit(main())
diff --git a/bin/sleep_test_log_check b/bin/sleep_test_log_check
deleted file mode 100755
index aae7556..0000000
--- a/bin/sleep_test_log_check
+++ /dev/null
@@ -1,201 +0,0 @@
-#!/usr/bin/env python3
-# -*- coding: utf-8 -*-
-#
-# This file is part of Checkbox.
-#
-# Copyright 2014 Canonical Ltd.
-#
-# Authors
-#  Jeff Lane <[e-mail]>
-#  Daniel Manrique<[e-mail]>
-#
-# Checkbox is free software: you can redistribute it and/or modify
-# it under the terms of the GNU General Public License version 3,
-# as published by the Free Software Foundation.
-#
-# Checkbox is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License
-# along with Checkbox.  If not, see <http://www.gnu.org/licenses/>.
-
-
-'''
-This script is used to parse the log generated by fwts and check it for certain
-errors detected during testing.  It expects that this is a log file created by
-fwts at runtime using the -l <log name> option.
-
-It's written now specifically for checking ater the fwts s3 and s4 tests but
-can be adapted to look for other tests, or all tests.
-'''
-
-import sys
-import collections
-import re
-
-from argparse import ArgumentParser
-import logging
-
-# Definitions of when a level starts, how a failure looks,
-# and when a level ends.
-start_level_re = r'^(?P<level>.+) failures: (?P<numfails>NONE|\d+)$'
-start_level_re = re.compile(start_level_re)
-failure_re = re.compile(r'^ (?P<test>(s3|s4)): (?P<details>.+)$')
-end_level_re = re.compile(r"$^")
-
-
-def parse_summary(summary, results):
-    """
-    Parses an entire "Test Failure Summary" section, which contains a short
-    summary of failures observed per level. Returns nothing, but adds the
-    results to the passed results dictionary.
-
-    :param summary:
-        A list of lines comprised in this summary section
-
-    :param results:
-        The results dictionary into which to put the end result. Should be a
-        dict with keys for each level, the values are dicts with keys for each
-        test (s3, s4) which in turn contain a list of all the failures observed
-        for that level and test.
-    """
-    current_level = None
-    current_acum = []
-
-    for logline in summary:
-        level_matches = start_level_re.search(logline)
-        if level_matches:
-            logging.debug("Found a level: %s", level_matches.group('level'))
-            current_level = level_matches.group('level')
-        elif end_level_re.search(logline) and current_level:
-            if current_level:
-                logging.debug("Current level (%s) has %s",
-                              current_level, current_acum)
-                # By passing results[current_level] a key in results will be
-                # created for every level we see, regardless of whether it
-                # reports failures or not.  This is OK because we can later
-                # check results' keys to ensure we saw at least one level; if
-                # results has no keys, it could mean a malformed fwts log file.
-                parse_level(current_acum, results[current_level])
-            else:
-                logging.debug("Discarding junk")
-            current_acum = []
-            current_level = None
-        else:
-            current_acum.append(logline)
-
-
-def parse_level(level_lines, level_results):
-    """
-    Parses the level's lines, appending the failures to the level's results.
-    level_results is a dictionary with a key per test type (s3, s4, and so on).
-    Returns nothing, but adds the results to the passed results dictionary for
-    this level.
-
-    :param level_lines:
-        A list of lines comprised in this level's list of failures.
-
-    : param level_results:
-        A dictionary containing this level's results. Should be a dict with
-        keys for each test, to which the failures for the level will be
-        appended.
-    """
-    for failureline in level_lines:
-        failure_matches = failure_re.search(failureline)
-        if failure_matches:
-            test = failure_matches.group('test')
-            details = failure_matches.group('details')
-            logging.debug("fail %s was %s", test, details)
-            level_results[test].append(details)
-
-
-def main():
-    parser = ArgumentParser()
-    parser.add_argument('-d', '--debug',
-                        action='store_const',
-                        const=logging.DEBUG,
-                        default=logging.INFO,
-                        help="Show debugging information.")
-    parser.add_argument('-v', '--verbose',
-                        action='store_true',
-                        default=False,
-                        help="Display each error discovered. May provide \
-                              very long output. Also, this option will only \
-                              provide a list of UNIQUE errors encountered in \
-                              the log file. It will not display duplicates. \
-                              Default is [%(default)s]")
-    parser.add_argument('test',
-                        action='store',
-                        choices=['s3', 's4'],
-                        help='The test to check (s3 or s4)')
-    parser.add_argument('logfile',
-                        action='store',
-                        help='The log file to parse')
-
-    args = parser.parse_args()
-
-    logging.basicConfig(level=args.debug)
-
-    #Create a generator and get our lines
-    log = (line.rstrip() for line in open(args.logfile, 'rt', encoding="UTF-8"))
-
-    # End result will be a dictionary with a key per level, value is another
-    # dictionary with a key per test (s3, s4, ...) and a list of all failures
-    # for each test.  Duplicates are possible, because we should also indicate
-    # the number of instances for each failure.
-    results = collections.defaultdict(lambda: collections.defaultdict(list))
-
-    sum_acum = []
-    summaries_found = 0
-
-    # Start parsing the fwts log. Gather each "Test Failure Summary" section
-    # and when it's complete, pass it to the parse_summary function to extract
-    # levels and tests.
-    for logline in log:
-        if "Test Failure Summary" in logline:
-            parse_summary(sum_acum, results)
-            summaries_found += 1
-            sum_acum = []
-        else:
-            sum_acum.append(logline)
-    # We reached the end, so add the last accumulated summary
-    if sum_acum:
-        parse_summary(sum_acum, results)
-
-    # Report what I found
-    for level in sorted(results.keys()):
-        if results[level]:  # Yes, we can have an empty level. We may have
-                            # seen the levelheader but had it report no
-                            # failures.
-            print("{} failures:".format(level))
-            for test in results[level].keys():
-                print("  {}: {} failures".format(test,
-                                                 len(results[level][test])))
-                if args.verbose:
-                    print('='*40)
-                    counts = collections.Counter(results[level][test])
-                    for failure in counts:
-                        print("    {} (x {})".format(failure, counts[failure]))
-
-    # Decide on the outcome based on the collected information
-    if not summaries_found:
-        logging.error("No fwts test summaries found, "
-                      "possible malformed fwts log file")
-        return_code = 2
-    elif not results.keys():  # If it has no keys, means we didn't see any
-                              # FWTS levels
-        logging.error("None of the summaries contained failure levels, "
-                      "possible malformed fwts log file")
-        return_code = 2
-    elif any(results.values()):  # If any of the results' levels has errors
-        return_code = 1
-    else:
-        print("No errors detected")
-        return_code = 0
-
-    return return_code
-
-if __name__ == '__main__':
-    sys.exit(main())
diff --git a/bin/storage_test b/bin/storage_test
deleted file mode 100755
index f96f28d..0000000
--- a/bin/storage_test
+++ /dev/null
@@ -1,94 +0,0 @@
-#!/bin/bash
-
-# take the path of the storage device and test is it a block device.
-
-function run_bonnie() {
-    echo "Running bonnie++ on $1..."
-    mount_point=$(df -h | grep -m 1 $1 | awk '{print $6}')
-    echo "Putting scratch disk at $mount_point"
-    mkdir -p "$mount_point/tmp/scratchdir"
-
-    # When running on disks with small drives (SSD/flash) we need to do
-    # some tweaking. Bonnie uses 2x RAM by default to write data. If that's
-    # more than available disk space, the test will fail inappropriately.
-    free_space=$(df -m | grep -m 1 $1 | awk '{print $4}')
-    echo "    Disk $1 has ${free_space}MB available"
-    memory=$(free -m |grep Mem|awk '{print $2}')
-    echo "    System has ${memory}MB RAM"
-    disk_needed=$((memory * 2))
-    echo "    We need ${disk_needed}MB of disk space for testing"
-    if [[ "$disk_needed" -ge "$free_space" ]]; then
-        echo "    Insufficient disk space available for defaults."
-        echo "    reducing memory footprint to be 1/2 of free disk space."
-        # we need to pass an amount that's 1/2 the amount of available disk
-        # space to bonnie++ so she doesn't overwrite and fail.
-        disk_needed=$(($free_space/2))
-        bonnie++ -d $mount_point/tmp/scratchdir -u root -r $disk_needed
-    else
-        echo "   Free disk space is sufficient to continue testing."
-        bonnie++ -d $mount_point/tmp/scratchdir -u root
-    fi
-}
-
-disk=/dev/$1
-
-if [ -b "$disk" ]
-then
-    echo "$disk is a block device"
-
-    #Add a check for warnings
-    WARN=$(parted -s ${disk} print | grep "^Warning.*${disk}.*[Rr]ead-only" 2>&1)
-    if [[ $? == 0 ]]
-    then
-        echo "Warning found in parted output:"
-        echo $WARN
-        echo "Aborting Test"
-        exit 1
-    fi
-
-    # Regex changed to better handle when $disk appears more than once
-    # in parted output (such as in warning messages or not caught in the
-    # check above)
-    size=`parted -l -s 2>&1 | grep "Disk .*${disk}:" | awk '{print $3}'`
-
-    if [ -n "$size" ]
-    then
-        echo "$disk reports a size of $size."
-        # Have to account for the end of the size descriptor
-        size_range=${size:(-2)}
-
-        if mount | grep -q $disk
-        then
-            echo "$disk is mounted, proceeding."
-        else
-            echo "$disk is not mounted. It must be mounted before testing."
-            exit 1
-        fi
-
-
-        if [ "$size_range" == "KB" ]
-        then
-            echo "$disk size reported in KB, seems to be too small for testing."
-            exit 1
-        elif [ "$size_range" == "MB" ]
-        then
-            size_int=${size::${#size}-2}
-
-            if [ "$size_int" -gt 10 ]
-            then
-                run_bonnie $disk
-            else
-                echo "$disk is too small to be used for testing."
-                exit 1
-            fi
-        else
-            run_bonnie $disk
-        fi
-    else
-       echo "$disk doesn't report a size."
-       exit 1
-    fi
-else
-    echo "$disk is not listed as a block device."
-    exit 1
-fi
diff --git a/bin/test_bt_keyboard b/bin/test_bt_keyboard
index 8828b3a..1457a1d 100755
--- a/bin/test_bt_keyboard
+++ b/bin/test_bt_keyboard
@@ -5,7 +5,7 @@
 # Written by:
 #   Maciej Kisielewski <[e-mail]>
 
-import bt_helper
+import checkbox_support.bt_helper
 
 
 def main():
diff --git a/bin/xml_sanitize b/bin/xml_sanitize
deleted file mode 100755
index 0cdbc8e..0000000
--- a/bin/xml_sanitize
+++ /dev/null
@@ -1,46 +0,0 @@
-#!/usr/bin/python3
-import errno
-import io
-import sys
-
-from argparse import ArgumentParser, FileType
-
-VALID_XML_CHARS = frozenset([0x9, 0xA, 0xD] +
-                            list(range(0x20, 0xD7FF)) +
-                            list(range(0xE000, 0xFFFD)) +
-                            list(range(0x10000, 0x10FFFF)))
-
-
-def is_valid_xml_char(ch):
-    # Is this character valid in XML?
-    # http://www.w3.org/TR/xml/#charsets
-    return ord(ch) in VALID_XML_CHARS
-
-
-def main():
-    parser = ArgumentParser("Receives as input some text and outputs "
-                            "the same text without characters which are "
-                            "not valid in the XML specification.")
-    parser.add_argument('input_file',
-                        type=FileType('r'),
-                        nargs='?',
-                        help='The name of the file to sanitize.')
-    args = parser.parse_args()
-
-    if args.input_file:
-        text = ''.join([c for c in args.input_file.read() if
-                       is_valid_xml_char(c)])
-
-    else:
-        with io.TextIOWrapper(
-                sys.stdin.buffer, encoding='UTF-8', errors="ignore") as stdin:
-            text = ''.join([c for c in stdin.read() if is_valid_xml_char(c)])
-
-    print(text)
-
-if __name__ == "__main__":
-    try:
-        sys.exit(main())
-    except Exception as err:
-        if err.errno != errno.EPIPE:
-            raise(err)
diff --git a/manage.py b/manage.py
index 2b84d17..b16b593 100755
--- a/manage.py
+++ b/manage.py
@@ -247,7 +247,6 @@ class InstallSpecials(InstallCommand):
     def invoked(self, ns):
         super().invoked(ns)
         self._install_kernel_security_specials(ns)
-        self._install_bt_helper(ns)
 
     def _install_kernel_security_specials(self, ns):
         dest_map = self._get_dest_map(ns.layout, ns.prefix)
@@ -266,13 +265,6 @@ class InstallSpecials(InstallCommand):
             os.path.join(provider.bin_dir, 'test_kernel_security.py'),
             ns.root + os.path.join(dest_map['data'], 'test_kernel_security.py'))
 
-    def _install_bt_helper(self, ns):
-        dest_map = self._get_dest_map(ns.layout, ns.prefix)
-        provider = self.get_provider()
-        shutil.copy(
-            os.path.join(provider.bin_dir, 'bt_helper.py'),
-            ns.root + dest_map['bin'])
-
 
 def _copytree(src, dst):
     """Simple copytree that always overwrites."""
diff --git a/requirements/container-tests-provider-snappy b/requirements/container-tests-provider-snappy
index 97b8b5f..d0b77e7 100755