Merge lp:~charlesk/keeper/add-keeper-dbusmock-template into lp:keeper

[Charles Kerr]

add a keeper dbusmock for testing/prototyping

Add a keeper dbusmock for testing/prototyping.

Supported bits:
com.canonical.keeper.User.GetBackupChoices()
com.canonical.keeper.User.GetRestoreChoices()
com.canonical.keeper.User.StartBackup()
com.canonical.keeper.Helper.StartBackup()
com.canonical.keeper.Mock.AddBackupChoice()
com.canonical.keeper.Mock.AddRestoreChoice()
com.canonical.keeper.Mock.GetBackupData()

Example use in tests/dbusmock/keeper-keeper-template-test.cpp

Coming next:
com.canonical.keeper.User.Cancel()
com.canonical.keeper.User.StartRestore()
com.canonical.keeper.Helper.StartRestore()

[lp:~charlesk/keeper/add-keeper-dbusmock-template]

109. By Charles Kerr on 2016-07-08

in keeper-template-test, add test to check return values of com.canonical.keeper.User.RestoreChoices()

110. By Charles Kerr on 2016-07-08

in keeper-template-test, add test to check User.StartBackup() throws EINVAL if one of the uuids isn't a valid backup choice

111. By Charles Kerr on 2016-07-08

in keeper-template-test, add test to check User.StartRestore() throws EINVAL if one of the uuids isn't a valid restore choice

112. By Charles Kerr on 2016-07-08

in keeper-template-test, add test to check User.State() returns an empty set when we have no tasks queued

113. By Charles Kerr on 2016-07-08

in keeper-template-test, test User.StartBackup(), Helper.StartBackup(), and User.State() by adding a task that performs a backup

114. By Charles Kerr on 2016-07-08

in keeper mock's helper_backup_worker(), better error handling when recv() returns 0

115. By Charles Kerr on 2016-07-08

sync with lp:~charlesk/keeper/add-tar-creator

116. By Charles Kerr on 2016-07-09

sync with lp:~charlesk/keeper/add-tar-creator

117. By Charles Kerr on 2016-07-10

fix threading issues in the keeper dbusmock template

118. By Charles Kerr on 2016-07-10

tweak the error messages a little

119. By Charles Kerr on 2016-07-10

make the template's python3-gi dependencies explicit in debian/control

120. By Charles Kerr on 2016-07-10

more keeper dbusmock tepmlate tweaking

121. By Charles Kerr on 2016-07-10

make the env messages terser in the folder-helper fake

122. By Charles Kerr on 2016-07-13

rename folder-helper.cpp as fake-folder-helper.cpp

123. By Charles Kerr on 2016-07-13

sync with trunk

124. By Charles Kerr on 2016-07-15

add flake8 to debian build dep

125. By Charles Kerr on 2016-07-15

add testing methods to the mock service to let devs add choices and to see the results of a backup run

126. By Charles Kerr on 2016-07-15

add tests to run the dbusmock template through a full backup session

127. By Charles Kerr on 2016-07-15

sync with trunk

[Xavi Garcia (xavi-garcia-mena)]

Thanks Charles.
I'm not very familiar with this kind of code defining mocks in python but I haven't seen anything weird. Just a couple of questions to understand the code better.

review: Needs Information

[lp:~charlesk/keeper/add-keeper-dbusmock-template]

128. By Charles Kerr on 2016-07-19

when the last task finishes, clear the remaining_tasks list

[Charles Kerr (charlesk)]

Answers inline

[lp:~charlesk/keeper/add-keeper-dbusmock-template]

129. By Charles Kerr on 2016-07-20

sync with trunk

130. By Charles Kerr on 2016-07-20

fix FTBFS by making implicit internal library dependency chain explicit

131. By Charles Kerr on 2016-07-20

fix whitespace test

132. By Charles Kerr on 2016-07-20

sync keeper-template-test with r128 changes

133. By Charles Kerr on 2016-07-21

fix typo in parsing input arguments to mock_add_backup_choice() and mock_add_restore_choice()

134. By Charles Kerr on 2016-07-21

when launching a backup helper, set the correct working directory

135. By Charles Kerr on 2016-07-21

sync with trunk

136. By Charles Kerr on 2016-07-21

fix FileNotFound cwd bug in keeper-template-test reported by Marcus

[Xavi Garcia (xavi-garcia-mena)]

Looks good to me, thanks!

review: Approve

=== modified file 'debian/control'
--- debian/control	2016-07-05 21:09:03 +0000
+++ debian/control	2016-07-21 10:16:32 +0000
@@ -28,6 +28,10 @@
                libdbustest1-dev,
                libqtdbusmock1-dev (>= 0.4),
                libqtdbustest1-dev,
+# for the dbusmock template:
+               python3-gi,
+               gir1.2-glib-2.0 (>= 1.32),
+               flake8,
 Standards-Version: 3.9.5
 Homepage: https://launchpad.net/keeper
 # If you aren't a member of ~indicator-applet-developers but need to upload

=== modified file 'src/qdbus-stubs/CMakeLists.txt'
--- src/qdbus-stubs/CMakeLists.txt	2016-07-07 13:59:14 +0000
+++ src/qdbus-stubs/CMakeLists.txt	2016-07-21 10:16:32 +0000
@@ -115,9 +115,19 @@
 #
 #
 
+set(
+  STUBS_LIB
+  qdbus-stubs
+)
+
 add_library(
-  qdbus-stubs
+  ${STUBS_LIB}
   STATIC
   ${interface_files}
   ${adaptor_files}
 )
+
+target_link_libraries(
+  ${STUBS_LIB}
+  backup-helper
+)

=== modified file 'src/qdbus-stubs/com.canonical.keeper.User.xml'
--- src/qdbus-stubs/com.canonical.keeper.User.xml	2016-06-27 18:49:17 +0000
+++ src/qdbus-stubs/com.canonical.keeper.User.xml	2016-07-21 10:16:32 +0000
@@ -72,7 +72,7 @@
           <doc:para>Provides state information so the user interface can show
                     the progress of backup or restore tasks.</doc:para>
           <doc:para>State is a map of opaque backup keys to property maps,
-                    which will contain a 'display-name' string and 'action' int
+                    which will contain a 'display-name' string and 'action' int32
                     whose possible values are queued(0), saving(1),
                     restoring(2), complete(3), stopped(4)</doc:para>
           <doc:para>Some property maps may also have an 'item' string

=== modified file 'tests/CMakeLists.txt'
--- tests/CMakeLists.txt	2016-07-01 15:17:06 +0000
+++ tests/CMakeLists.txt	2016-07-21 10:16:32 +0000
@@ -1,11 +1,38 @@
+###
+###  Build Google Test and Google Mock
+###
+
 # We add -g so we get debug info for the gtest stack frames with gdb.
-# The warnings are suppressed so we get a noise-free build for gtest and gmock.
+# We add -w to supress warnings when compiling this external source code.
 set(old_cxx_flags ${CMAKE_CXX_FLAGS})
 set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -g -w")
 find_package(GMock REQUIRED)
 find_package(Qt5Test REQUIRED)
 set(CMAKE_CXX_FLAGS ${old_cxx_flags})
 
+###
+###  Service DBusMock
+###
+
+set(SERVICE_TEMPLATE_FILE "${CMAKE_CURRENT_SOURCE_DIR}/com_canonical_keeper.py")
+add_definitions(-DSERVICE_TEMPLATE_FILE="${SERVICE_TEMPLATE_FILE}")
+
+# get the python3 package directory
+execute_process (
+  COMMAND python3 -c "from distutils import sysconfig; print(sysconfig.get_python_lib())"
+  COMMAND sed -r -e "s|/usr/(local/)?||g"
+  OUTPUT_VARIABLE PYTHON_PACKAGE_DIR OUTPUT_STRIP_TRAILING_WHITESPACE
+)
+
+install(
+  FILES ${SERVICE_TEMPLATE_FILE}
+  DESTINATION "${PYTHON_PACKAGE_DIR}/dbusmock/templates/"
+)
+
+###
+###  The Tests
+###
+
 pkg_check_modules(TEST_DEPENDENCIES
   libqtdbustest-1 REQUIRED
   libqtdbusmock-1 REQUIRED
@@ -22,5 +49,6 @@
 )
 
 add_subdirectory(surface)
+add_subdirectory(dbusmock)
 add_subdirectory(unit)
 add_subdirectory(utils)

=== added file 'tests/com_canonical_keeper.py'
--- tests/com_canonical_keeper.py	1970-01-01 00:00:00 +0000
+++ tests/com_canonical_keeper.py	2016-07-21 10:16:32 +0000
@@ -0,0 +1,435 @@
+import copy
+import dbus
+import dbus.service
+import os
+import socket
+import subprocess
+import sys
+from dbusmock import OBJECT_MANAGER_IFACE, mockobject
+from gi.repository import GLib
+
+'''com.canonical.keeper mock template
+'''
+
+# This program is free software; you can redistribute it and/or modify it under
+# the terms of the GNU Lesser General Public License as published by the Free
+# Software Foundation; either version 3 of the License, or (at your option) any
+# later version.  See http://www.gnu.org/copyleft/lgpl.html for the full text
+# of the license.
+
+__author__ = 'Charles Kerr'
+__email__ = '[e-mail]'
+__copyright__ = '(c) 2016 Canonical Ltd.'
+__license__ = 'LGPL 3+'
+
+SERVICE_PATH = '/com/canonical/keeper'
+SERVICE_IFACE = 'com.canonical.keeper'
+USER_PATH = '/com/canonical/keeper/user'
+USER_IFACE = 'com.canonical.keeper.User'
+HELPER_PATH = '/com/canonical/keeper/helper'
+HELPER_IFACE = 'com.canonical.keeper.Helper'
+MOCK_IFACE = 'com.canonical.keeper.Mock'
+
+# magic keys used by dbusmock
+BUS_NAME = 'com.canonical.keeper'
+MAIN_IFACE = SERVICE_IFACE
+MAIN_OBJ = SERVICE_PATH
+SYSTEM_BUS = False
+
+ACTION_QUEUED = 0
+ACTION_SAVING = 1
+ACTION_RESTORING = 2
+ACTION_COMPLETE = 3
+ACTION_STOPPED = 4
+
+KEY_CTIME = 'ctime'
+KEY_BLOB = 'blob-data'
+KEY_HELPER = 'helper-exec'
+KEY_ICON = 'icon'
+KEY_NAME = 'display-name'
+KEY_SIZE = 'size'
+KEY_SUBTYPE = 'subtype'
+KEY_TYPE = 'type'
+KEY_UUID = 'uuid'
+
+TYPE_APP = 'application'
+TYPE_FOLDER = 'folder'
+TYPE_SYSTEM = 'system-data'
+
+#
+#  Utils
+#
+
+
+def badarg(msg):
+    raise dbus.exceptions.DBusException(
+        msg,
+        name='org.freedesktop.DBus.Error.InvalidArgs'
+    )
+
+
+def fail(msg):
+    raise dbus.exceptions.DBusException(
+        msg,
+        name='org.freedesktop.DBus.Error.Failed'
+    )
+
+
+def fail_if_busy():
+    user = mockobject.objects[USER_PATH]
+    if user.all_tasks:
+        fail("can't do that while service is busy")
+
+
+#
+#  User Obj
+#
+
+
+def user_get_backup_choices(user):
+    return dbus.Dictionary(
+        user.backup_choices,
+        signature='sa{sv}',
+        variant_level=1
+    )
+
+
+def user_get_restore_choices(user):
+    return dbus.Dictionary(
+        user.restore_choices,
+        signature='sa{sv}',
+        variant_level=1
+    )
+
+
+def user_start_next_task(user):
+    if not user.remaining_tasks:
+        user.log('last task finished')
+        user.all_tasks = []
+        user.current_task = None
+        user.log('setting user.current_task to None')
+        user.update_state_property(user)
+
+    else:
+        uuid = user.remaining_tasks.pop(0)
+        user.current_task = uuid
+        user.update_state_property(user)
+
+        # find the helper to run
+        choice = user.backup_choices.get(uuid)
+        if not choice:
+            choice = user.restore_choices.get(uuid)
+        helper_exec = choice.get(KEY_HELPER)
+
+        # build the env that we'll pass to the helper
+        henv = {}
+        henv['QDBUS_DEBUG'] = '1'
+        henv['G_DBUS_DEBUG'] = 'call,message,signal,return'
+        for key in ['DBUS_SESSION_BUS_ADDRESS', 'DBUS_SYSTEM_BUS_ADDRESS']:
+            val = os.environ.get(key, None)
+            if val:
+                henv[key] = val
+
+        # set the working directory for folder backups
+        helper_cwd = os.getcwd()
+        if choice.get(KEY_TYPE) == TYPE_FOLDER:
+            helper_cwd = choice.get(KEY_SUBTYPE)
+
+        # spawn the helper
+        user.log('starting %s for %s, env %s' % (helper_exec, uuid, henv))
+        subprocess.Popen(
+            [helper_exec, HELPER_PATH],
+            env=henv, stdout=sys.stdout, stderr=sys.stderr,
+            cwd=helper_cwd,
+            shell=helper_exec.endswith('.sh')
+        )
+
+
+def user_start_backup(user, uuids):
+
+    # sanity checks
+    fail_if_busy()
+    for uuid in uuids:
+        if uuid not in user.backup_choices:
+            badarg('uuid %s is not a valid backup choice' % (uuid))
+
+    user.all_tasks = uuids
+    user.remaining_tasks = copy.copy(uuids)
+    user.start_next_task(user)
+
+
+def user_start_restore(user, uuids):
+
+    # sanity checks
+    fail_if_busy()
+    for uuid in uuids:
+        if uuid not in user.restore_choices:
+            badarg('uuid %s is not a valid backup choice' % (uuid))
+
+    user.all_tasks = uuids
+    user.remaining_tasks = copy.copy(uuids)
+    user.start_next_task()
+
+
+def user_cancel(user):
+    # FIXME
+    pass
+
+
+def user_build_state(user):
+    """Returns a generated state dictionary.
+
+    State is a map of opaque backup keys to property maps,
+    which will contain a 'display-name' string and 'action' int
+    whose possible values are queued(0), saving(1),
+    restoring(2), complete(3), and stopped(4).
+
+    Some property maps may also have an 'item' string
+    and a 'percent-done' double [0..1.0].
+    For example if these are set to (1, "Photos", 0.36),
+    the user interface could show "Backing up Photos (36%)".
+    Clients should gracefully handle missing properties;
+    eg a missing percent-done could instead show
+    "Backing up Photos".
+
+    A failed task's property map may also contain an 'error'
+    string if set by the backup helpers.
+    """
+
+    tasks_states = {}
+    for uuid in user.all_tasks:
+        task_state = {}
+
+        # get the task's action
+        if uuid == user.current_task:
+            if uuid in user.backup_choices:
+                action = ACTION_SAVING
+            else:
+                action = ACTION_RESTORING
+        elif uuid in user.remaining_tasks:
+            action = ACTION_QUEUED
+        else:  # FIXME: 'ACTION_STOPPED' not handled yet
+            action = ACTION_COMPLETE
+        task_state['action'] = dbus.Int32(action)
+
+        # get the task's display-name
+        choice = user.backup_choices.get(uuid, None)
+        if not choice:
+            choice = user.restore_choices.get(uuid, None)
+        if not choice:
+            fail("couldn't find a choice for uuid %s" % (uuid))
+        display_name = choice.get(KEY_NAME, None)
+        task_state[KEY_NAME] = dbus.String(display_name)
+
+        # FIXME: use a real percentage here
+        if action == ACTION_COMPLETE:
+            percent_done = dbus.Double(1.0)
+        elif action == ACTION_SAVING or action == ACTION_RESTORING:
+            percent_done = dbus.Double(0.5)
+        else:
+            percent_done = dbus.Double(0.0)
+        task_state['percent-done'] = percent_done
+
+        tasks_states[uuid] = dbus.Dictionary(task_state)
+
+    return dbus.Dictionary(
+        tasks_states,
+        signature='sa{sv}',
+        variant_level=1
+    )
+
+
+def user_update_state_property(user):
+    old_state = user.Get(USER_IFACE, 'State')
+    new_state = user.build_state(user)
+    if old_state != new_state:
+        user.Set(USER_IFACE, 'State', new_state)
+
+
+#
+#  Helper Obj
+#
+
+class HelperWork:
+    chunks = None
+    n_bytes = None
+    n_left = None
+    sock = None
+    uuid = None
+
+
+def helper_periodic_func(helper):
+
+    if not helper.work:
+        fail("bug: helper_periodic_func called w/o helper.work")
+
+    # try to read a bit
+    chunk = helper.work.sock.recv(4096*2)
+    if len(chunk):
+        helper.work.chunks.append(chunk)
+        helper.work.n_left -= len(chunk)
+        helper.log('got %s bytes; %s left' % (len(chunk), helper.work.n_left))
+
+    # cleanup if done
+    done = helper.work.n_left <= 0
+    if done:
+        helper.work.sock.shutdown(socket.SHUT_RDWR)
+        helper.work.sock.close()
+        user = mockobject.objects[USER_PATH]
+        user.backup_data[helper.work.uuid] = b''.join(helper.work.chunks)
+        user.log(
+            'backup %s done; %s bytes' %
+            (helper.work.uuid, len(user.backup_data[helper.work.uuid]))
+        )
+        user.start_next_task(user)
+        helper.work = None
+
+    if len(chunk) or done:
+        user = mockobject.objects[USER_PATH]
+        user.update_state_property(user)
+
+    return not done
+
+
+def helper_start_backup(helper, n_bytes):
+
+    helper.log("got start_backup request for %s bytes" % (n_bytes))
+    if helper.work:
+        fail("can't start a new backup while one's already active")
+
+    parent, child = socket.socketpair()
+
+    # set up helper's workarea
+    work = HelperWork()
+    work.chunks = []
+    work.n_bytes = n_bytes
+    work.n_left = n_bytes
+    work.sock = parent
+    work.uuid = mockobject.objects[USER_PATH].current_task
+    helper.work = work
+
+    # start checking periodically
+    GLib.timeout_add(10, helper.periodic_func, helper)
+    return dbus.types.UnixFd(child.fileno())
+
+
+def helper_start_restore(helper):
+    helper.parent, child = socket.socketpair()
+    return child
+
+
+#
+#  Controlling the mock
+#
+
+
+def mock_add_backup_choice(mock, uuid, props):
+
+    keys = [KEY_NAME, KEY_TYPE, KEY_SUBTYPE, KEY_ICON, KEY_HELPER]
+    if set(keys) != set(props.keys()):
+        badarg('need props: %s got %s' % (keys, props.keys()))
+
+    user = mockobject.objects[USER_PATH]
+    user.backup_choices[uuid] = dbus.Dictionary(
+        props,
+        signature='sv',
+        variant_level=1
+    )
+
+
+def mock_add_restore_choice(mock, uuid, props):
+
+    keys = [KEY_NAME, KEY_TYPE, KEY_SUBTYPE, KEY_ICON, KEY_HELPER,
+            KEY_SIZE, KEY_CTIME, KEY_BLOB]
+    if set(keys) != set(props.keys()):
+        badarg('need props: %s got %s' % (keys, props.keys()))
+
+    user = mockobject.objects[USER_PATH]
+    user.restore_choices[uuid] = dbus.Dictionary(
+        props,
+        signature='sv',
+        variant_level=1
+    )
+
+
+def mock_get_backup_data(mock, uuid):
+    blob = mockobject.objects[USER_PATH].backup_data[uuid]
+    mock.log('returning %s byte blob for uuid %s' % (len(blob), uuid))
+    return blob
+
+
+#
+#
+#
+
+def load(main, parameters):
+
+    main.log('Keeper template paramers: "' + str(parameters) + '"')
+
+    # com.canonical.keeper.User
+    path = USER_PATH
+    main.AddObject(path, USER_IFACE, {}, [])
+    o = mockobject.objects[path]
+    o.get_backup_choices = user_get_backup_choices
+    o.start_backup = user_start_backup
+    o.get_restore_choices = user_get_restore_choices
+    o.start_restore = user_start_restore
+    o.cancel = user_cancel
+    o.build_state = user_build_state
+    o.update_state_property = user_update_state_property
+    o.start_next_task = user_start_next_task
+    o.all_tasks = []
+    o.remaining_tasks = []
+    o.backup_data = {}
+    o.backup_choices = parameters.get('backup-choices', {})
+    o.restore_choices = parameters.get('restore-choices', {})
+    o.current_task = None
+    o.defined_types = [TYPE_APP, TYPE_SYSTEM, TYPE_FOLDER]
+    o.AddMethods(USER_IFACE, [
+        ('GetBackupChoices', '', 'a{sa{sv}}',
+         'ret = self.get_backup_choices(self)'),
+        ('StartBackup', 'as', '',
+         'self.start_backup(self, args[0])'),
+        ('GetRestoreChoices', '', 'a{sa{sv}}',
+         'ret = self.get_restore_choices(self)'),
+        ('StartRestore', 'as', '',
+         'self.start_restore(self, args[0])'),
+        ('Cancel', '', '',
+         'self.cancel(self)'),
+    ])
+    o.AddProperty(USER_IFACE, "State", o.build_state(o))
+
+    # com.canonical.keeper.Helper
+    path = HELPER_PATH
+    main.AddObject(path, HELPER_IFACE, {}, [])
+    o = mockobject.objects[path]
+    o.start_backup = helper_start_backup
+    o.start_restore = helper_start_restore
+    o.periodic_func = helper_periodic_func
+    o.work = None
+    o.AddMethods(HELPER_IFACE, [
+        ('StartBackup', 't', 'h',
+         'ret = self.start_backup(self, args[0])'),
+        ('StartRestore', '', 'h',
+         'ret = self.start_restore(self)')
+    ])
+
+    # com.canonical.keeper.Mock
+    o = main
+    o.add_backup_choice = mock_add_backup_choice
+    o.add_restore_choice = mock_add_restore_choice
+    o.get_backup_data = mock_get_backup_data
+    o.AddMethods(MOCK_IFACE, [
+        ('AddBackupChoice', 'sa{sv}', '',
+         'self.add_backup_choice(self, args[0], args[1])'),
+        ('AddRestoreChoice', 'sa{sv}', '',
+         'self.add_restore_choice(self, args[0], args[1])'),
+        ('GetBackupData', 's', 'ay',
+         'ret = self.get_backup_data(self, args[0])')
+    ])
+    o.EmitSignal(
+        OBJECT_MANAGER_IFACE,
+        'InterfacesAdded',
+        'oa{sa{sv}}',
+        [SERVICE_PATH, {MOCK_IFACE: {}}]
+    )

=== added directory 'tests/dbusmock'
=== added file 'tests/dbusmock/CMakeLists.txt'
--- tests/dbusmock/CMakeLists.txt	1970-01-01 00:00:00 +0000
+++ tests/dbusmock/CMakeLists.txt	2016-07-21 10:16:32 +0000
@@ -0,0 +1,62 @@
+
+include_directories(
+    ${CMAKE_SOURCE_DIR}/src
+    ${CMAKE_BINARY_DIR}/src
+)
+
+set(
+    LINK_LIBS
+    qdbus-stubs
+    ${TEST_DEPENDENCIES_LDFLAGS}
+    test-utils
+    Qt5::Core
+    Qt5::DBus
+    Qt5::Network
+    ${GTEST_LIBRARIES}
+    ${GMOCK_LIBRARIES}
+)
+
+###
+###
+
+set(
+    BACKUP_HELPER
+    fake-backup-helper
+)
+
+add_executable(
+    ${BACKUP_HELPER}
+    fake-backup-helper.cpp
+)
+target_link_libraries(
+    ${BACKUP_HELPER}
+    ${LINK_LIBS}
+)
+
+###
+###
+
+set(
+    TEST_NAME
+    keeper-template-test
+)
+
+add_executable(
+    ${TEST_NAME}
+    keeper-template-test.cpp
+)
+
+set_property(
+    SOURCE keeper-template-test.cpp
+    PROPERTIES APPEND_STRING PROPERTY COMPILE_DEFINITIONS FAKE_BACKUP_HELPER_EXEC=\"${CMAKE_CURRENT_BINARY_DIR}/${BACKUP_HELPER}\"
+)
+
+target_link_libraries(
+    ${TEST_NAME}
+    ${LINK_LIBS}
+)
+
+add_test(
+    ${TEST_NAME}
+    ${TEST_NAME}
+)

=== added file 'tests/dbusmock/fake-backup-helper.cpp'
--- tests/dbusmock/fake-backup-helper.cpp	1970-01-01 00:00:00 +0000
+++ tests/dbusmock/fake-backup-helper.cpp	2016-07-21 10:16:32 +0000
@@ -0,0 +1,63 @@
+/*
+ * Copyright (C) 2016 Canonical, Ltd.
+ *
+ * This program is free software: you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 3, as published
+ * by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranties of
+ * MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
+ * PURPOSE.  See the GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License along
+ * with this program.  If not, see <http://www.gnu.org/licenses/>.
+ *
+ * Authors:
+ *     Charles Kerr <[e-mail]>
+ */
+
+#include "fake-backup-helper.h"
+
+#include <qdbus-stubs/dbus-types.h>
+#include <qdbus-stubs/keeper_helper_interface.h>
+
+#include <QCoreApplication>
+#include <QDebug>
+#include <QDBusConnection>
+#include <QDBusInterface>
+#include <QDBusReply>
+#include <QProcessEnvironment>
+#include <QLocalSocket>
+
+int
+main(int argc, char **argv)
+{
+    QCoreApplication app(argc, argv);
+
+    const auto blob = QByteArray(FAKE_BACKUP_HELPER_PAYLOAD);
+
+    // dump the inputs to stdout
+    qDebug() << "argc:" << argc;
+    for(int i=0; i<argc; ++i)
+        qDebug() << "argv[" << i << "] is" << argv[i];
+    const auto env = QProcessEnvironment::systemEnvironment();
+    for(const auto& key : env.keys())
+        qDebug() << "env" << qPrintable(key) << "is" << qPrintable(env.value(key));
+
+    // ask the service for a socket
+    auto conn = QDBusConnection::connectToBus(QDBusConnection::SessionBus, DBusTypes::KEEPER_SERVICE);
+    const auto object_path = QString::fromUtf8(argv[1]);
+    DBusInterfaceKeeperHelper helper_iface (DBusTypes::KEEPER_SERVICE, object_path, conn);
+    QDBusReply<QDBusUnixFileDescriptor> reply = helper_iface.call("StartBackup", blob.size());
+    const auto ufd = reply.value();
+    Q_ASSERT(reply.isValid());
+
+    // write the blob
+    QLocalSocket sock;
+    sock.setSocketDescriptor(ufd.fileDescriptor());
+    qDebug() << "wrote" << sock.write(blob) << "bytes";
+    sock.flush();
+
+    return EXIT_SUCCESS;
+}

=== added file 'tests/dbusmock/fake-backup-helper.h'
--- tests/dbusmock/fake-backup-helper.h	1970-01-01 00:00:00 +0000
+++ tests/dbusmock/fake-backup-helper.h	2016-07-21 10:16:32 +0000
@@ -0,0 +1,38 @@
+/*
+ * Copyright (C) 2016 Canonical, Ltd.
+ *
+ * This program is free software: you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 3, as published
+ * by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranties of
+ * MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
+ * PURPOSE.  See the GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License along
+ * with this program.  If not, see <http://www.gnu.org/licenses/>.
+ *
+ * Authors:
+ *     Charles Kerr <[e-mail]>
+ */
+
+#pragma once
+
+static const char* FAKE_BACKUP_HELPER_PAYLOAD = R"(
+    Oh mmm I know a place
+    Ain't nobody cryin'
+    Ain't nobody worried
+    Ain't no smilin' faces
+    Mmm, no no
+    Lyin' to the races
+    Help me, come on, come on
+    Somebody, help me now (I'll take you there)
+    Help me, ya'all (I'll take you there)
+    Help me now (I'll take you there)
+    Oh! (I'll take you there)
+    Oh! Oh! Mercy! (I'll take you there)
+    Oh, let me take you there (I'll take you there)
+    Oh-oh! Let me take you there! (I'll take you there)
+    Play your, play your piano now
+)";

=== added file 'tests/dbusmock/keeper-template-test.cpp'
--- tests/dbusmock/keeper-template-test.cpp	1970-01-01 00:00:00 +0000
+++ tests/dbusmock/keeper-template-test.cpp	2016-07-21 10:16:32 +0000
@@ -0,0 +1,285 @@
+/*
+ * Copyright 2016 Canonical Ltd.
+ *
+ * This program is free software: you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 3, as published
+ * by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranties of
+ * MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
+ * PURPOSE.  See the GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License along
+ * with this program.  If not, see <http://www.gnu.org/licenses/>.
+ *
+ * Authors:
+ *   Charles Kerr <[e-mail]>
+ */
+
+#include "fake-backup-helper.h"
+
+#include "qdbus-stubs/dbus-types.h"
+#include "qdbus-stubs/keeper_interface.h"
+#include "qdbus-stubs/keeper_user_interface.h"
+
+#include <gtest/gtest.h>
+
+#include <libqtdbusmock/DBusMock.h>
+#include <libqtdbustest/DBusTestRunner.h>
+
+#include <QDBusConnection>
+#include <QDebug>
+#include <QString>
+#include <QThread>
+#include <QUuid>
+#include <QVariant>
+
+#include <memory>
+
+#define KEY_ACTION "action"
+#define KEY_CTIME "ctime"
+#define KEY_BLOB "blob-data"
+#define KEY_HELPER "helper-exec"
+#define KEY_ICON "icon"
+#define KEY_NAME "display-name"
+#define KEY_PERCENT "percent-done"
+#define KEY_SIZE "size"
+#define KEY_SUBTYPE "subtype"
+#define KEY_TYPE "type"
+#define KEY_UUID "uuid"
+
+// FIXME: this should go in a shared header
+enum
+{
+    ACTION_QUEUED = 0,
+    ACTION_SAVING = 1,
+    ACTION_RESTORING = 2,
+    ACTION_IDLE = 3
+};
+
+using namespace QtDBusMock;
+using namespace QtDBusTest;
+
+
+class KeeperTemplateTest : public ::testing::Test
+{
+protected:
+
+    virtual void SetUp() override
+    {
+        test_runner_.reset(new DBusTestRunner{});
+
+        dbus_mock_.reset(new DBusMock(*test_runner_));
+        dbus_mock_->registerTemplate(
+            DBusTypes::KEEPER_SERVICE,
+            SERVICE_TEMPLATE_FILE,
+            QDBusConnection::SessionBus
+        );
+
+        test_runner_->startServices();
+        auto conn = connection();
+        qDebug() << "session bus is" << qPrintable(conn.name());
+
+        keeper_iface_.reset(
+            new DBusInterfaceKeeper(
+                DBusTypes::KEEPER_SERVICE,
+                DBusTypes::KEEPER_SERVICE_PATH,
+                conn
+            )
+        );
+        ASSERT_TRUE(keeper_iface_->isValid()) << qPrintable(conn.lastError().message());
+
+        user_iface_.reset(
+            new DBusInterfaceKeeperUser(
+                DBusTypes::KEEPER_SERVICE,
+                DBusTypes::KEEPER_USER_PATH,
+                conn
+            )
+        );
+        ASSERT_TRUE(user_iface_->isValid()) << qPrintable(conn.lastError().message());
+
+        mock_iface_.reset(
+            new QDBusInterface(
+                DBusTypes::KEEPER_SERVICE,
+                DBusTypes::KEEPER_SERVICE_PATH,
+                QStringLiteral("com.canonical.keeper.Mock"),
+                conn
+            )
+        );
+        ASSERT_TRUE(mock_iface_->isValid()) << qPrintable(conn.lastError().message());
+    }
+
+    virtual void TearDown() override
+    {
+        user_iface_.reset();
+        keeper_iface_.reset();
+        dbus_mock_.reset();
+        test_runner_.reset();
+    }
+
+    const QDBusConnection& connection()
+    {
+        return test_runner_->sessionConnection();
+    }
+
+    std::unique_ptr<DBusTestRunner> test_runner_;
+    std::unique_ptr<DBusMock> dbus_mock_;
+    std::unique_ptr<DBusInterfaceKeeper> keeper_iface_;
+    std::unique_ptr<DBusInterfaceKeeperUser> user_iface_;
+    std::unique_ptr<QDBusInterface> mock_iface_;
+
+    void EXPECT_EVENTUALLY(std::function<bool()>&& test, qint64 timeout_msec=5000)
+    {
+        QElapsedTimer timer;
+        timer.start();
+        bool passed;
+        do {
+            passed = test();
+            if (!passed)
+                QThread::msleep(100);
+        } while (!passed && !timer.hasExpired(timeout_msec));
+        EXPECT_TRUE(passed);
+    }
+
+    void wait_for_backup_to_finish()
+    {
+        EXPECT_EVENTUALLY([this]{return !user_iface_->state().isEmpty();}); // backup running
+        EXPECT_EVENTUALLY([this]{return user_iface_->state().isEmpty();}); // backup finished
+    }
+};
+
+
+/***
+****  Quick surface-level tests
+***/
+
+
+// test that the dbusmock scaffolding starts up and exits ok
+TEST_F(KeeperTemplateTest, HelloWorld)
+{
+}
+
+
+// test GetBackupChoices() returns what we give to AddBackupChoice()
+TEST_F(KeeperTemplateTest, BackupChoices)
+{
+    // build a backup choice
+    const auto uuid = QUuid::createUuid().toString();
+    const QMap<QString,QVariant> props {
+        { KEY_NAME, QStringLiteral("some-name") },
+        { KEY_TYPE, QStringLiteral("some-type") },
+        { KEY_SUBTYPE, QStringLiteral("some-subtype") },
+        { KEY_ICON, QStringLiteral("some-icon") },
+        { KEY_HELPER, QString::fromUtf8(FAKE_BACKUP_HELPER_EXEC) }
+    };
+
+    // add it
+    auto msg = mock_iface_->call(QStringLiteral("AddBackupChoice"), uuid, props);
+    EXPECT_NE(QDBusMessage::ErrorMessage, msg.type()) << qPrintable(msg.errorMessage());
+
+    // ask for a list of backup choices
+    QDBusReply<QVariantDictMap> choices = user_iface_->call("GetBackupChoices");
+    EXPECT_TRUE(choices.isValid()) << qPrintable(choices.error().message());
+
+    // check the results
+    const auto expected_choices = QVariantDictMap{{uuid, props}};
+    ASSERT_EQ(expected_choices, choices);
+}
+
+
+// test that StartBackup() fails if we pass an invalid arg
+TEST_F(KeeperTemplateTest, StartBackupWithInvalidArg)
+{
+    const auto invalid_uuid = QUuid::createUuid().toString();
+
+    QDBusReply<void> reply = user_iface_->call("StartBackup", QStringList{invalid_uuid});
+    EXPECT_FALSE(reply.isValid());
+    EXPECT_EQ(QDBusError::InvalidArgs, reply.error().type());
+}
+
+
+// test GetRestoreChoices() returns what we give to AddRestoreChoice()
+TEST_F(KeeperTemplateTest, RestoreChoices)
+{
+    // build a restore choice
+    const auto uuid = QUuid::createUuid().toString();
+    const auto blob = QUuid::createUuid().toByteArray();
+    const QMap<QString,QVariant> props {
+        { KEY_NAME, QStringLiteral("some-name") },
+        { KEY_TYPE, QStringLiteral("some-type") },
+        { KEY_SUBTYPE, QStringLiteral("some-subtype") },
+        { KEY_ICON, QStringLiteral("some-icon") },
+        { KEY_HELPER, QString::fromUtf8("/dev/null") },
+        { KEY_SIZE, quint64(blob.size()) },
+        { KEY_CTIME, quint64(time(nullptr)) },
+        { KEY_BLOB, blob }
+    };
+
+    // add it
+    auto msg = mock_iface_->call(QStringLiteral("AddRestoreChoice"), uuid, props);
+    EXPECT_NE(QDBusMessage::ErrorMessage, msg.type()) << qPrintable(msg.errorMessage());
+
+    // ask for a list of restore choices
+    QDBusReply<QVariantDictMap> choices = user_iface_->call("GetRestoreChoices");
+    EXPECT_TRUE(choices.isValid()) << qPrintable(choices.error().message());
+
+    // check the results
+    const auto expected_choices = QVariantDictMap{{uuid, props}};
+    ASSERT_EQ(expected_choices, choices);
+}
+
+
+// test that StartRestore() fails if we pass an invalid arg
+TEST_F(KeeperTemplateTest, StartRestoreWithInvalidArg)
+{
+    const auto invalid_uuid = QUuid::createUuid().toString();
+
+    QDBusReply<void> reply = user_iface_->call("StartRestore", QStringList{invalid_uuid});
+    EXPECT_FALSE(reply.isValid());
+    EXPECT_EQ(QDBusError::InvalidArgs, reply.error().type());
+}
+
+
+// test that Status() returns empty if we haven't done anything yet
+TEST_F(KeeperTemplateTest, TestEmptyStatus)
+{
+    EXPECT_TRUE(user_iface_->state().isEmpty());
+}
+
+
+/***
+****  Make a real backup
+***/
+
+TEST_F(KeeperTemplateTest, BackupRun)
+{
+    QTemporaryDir sandbox;
+
+    // build a backup choice
+    const auto uuid = QUuid::createUuid().toString();
+    const QMap<QString,QVariant> props {
+        { KEY_NAME, QStringLiteral("Music") },
+        { KEY_TYPE, QStringLiteral("folder") },
+        { KEY_SUBTYPE, sandbox.path() },
+        { KEY_ICON, QStringLiteral("music-icon") },
+        { KEY_HELPER, QString::fromUtf8(FAKE_BACKUP_HELPER_EXEC) }
+    };
+
+    // add it
+    auto msg = mock_iface_->call(QStringLiteral("AddBackupChoice"), uuid, props);
+    EXPECT_NE(QDBusMessage::ErrorMessage, msg.type()) << qPrintable(msg.errorMessage());
+
+    // start the backup
+    QDBusReply<void> reply = user_iface_->call("StartBackup", QStringList{uuid});
+    EXPECT_TRUE(reply.isValid()) << qPrintable(reply.error().message());
+    wait_for_backup_to_finish();
+
+    // ask keeper for the blob
+    QDBusReply<QByteArray> blob = mock_iface_->call(QStringLiteral("GetBackupData"), uuid);
+    EXPECT_TRUE(reply.isValid()) << qPrintable(reply.error().message());
+
+    // check the results
+    const auto expected_blob = QByteArray(FAKE_BACKUP_HELPER_PAYLOAD);
+    ASSERT_EQ(expected_blob, blob);
+}

=== modified file 'tests/surface/CMakeLists.txt'
--- tests/surface/CMakeLists.txt	2016-06-25 18:46:42 +0000
+++ tests/surface/CMakeLists.txt	2016-07-21 10:16:32 +0000
@@ -1,4 +1,5 @@
 add_test(cppcheck cppcheck --enable=all -USCHEMA_DIR --error-exitcode=2 --inline-suppr -I${CMAKE_SOURCE_DIR}/src ${CMAKE_SOURCE_DIR}/src ${CMAKE_SOURCE_DIR}/tests)
+add_test(flake8 flake8 ${SERVICE_TEMPLATE_FILE})
 
 add_subdirectory(copyright)
 add_subdirectory(whitespace)

=== modified file 'tests/unit/helper/CMakeLists.txt'
--- tests/unit/helper/CMakeLists.txt	2016-07-15 14:13:54 +0000
+++ tests/unit/helper/CMakeLists.txt	2016-07-21 10:16:32 +0000
@@ -1,5 +1,3 @@
-set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++11")
-
 add_definitions ( -DCMAKE_SOURCE_DIR="${CMAKE_CURRENT_SOURCE_DIR}" )
 add_definitions ( -DCMAKE_BINARY_DIR="${CMAKE_CURRENT_BINARY_DIR}" )
 

=== modified file 'tests/utils/CMakeLists.txt'
--- tests/utils/CMakeLists.txt	2016-07-11 17:04:16 +0000
+++ tests/utils/CMakeLists.txt	2016-07-21 10:16:32 +0000
@@ -1,9 +1,16 @@
 
-include_directories("${CMAKE_SOURCE_DIR}/src")
-include_directories(${CMAKE_CURRENT_BINARY_DIR})
+include_directories(
+    ${CMAKE_SOURCE_DIR}/src
+    ${CMAKE_CURRENT_BINARY_DIR}
+)
+
+set(
+    LIB_NAME
+    test-utils
+)
 
 add_library(
-    test-utils
+    ${LIB_NAME}
     STATIC
     main.cpp
     dummy-file.cpp
@@ -11,6 +18,6 @@
 )
 
 qt5_use_modules(
-    test-utils
+    ${LIB_NAME}
     Core
 )

=== modified file 'tests/utils/main.cpp'
--- tests/utils/main.cpp	2016-07-01 18:02:27 +0000
+++ tests/utils/main.cpp	2016-07-21 10:16:32 +0000
@@ -17,6 +17,8 @@
  *     Pete Woods <[e-mail]>
  */
 
+#include "qdbus-stubs/dbus-types.h"
+
 #include <libqtdbusmock/DBusMock.h>
 
 #include <gtest/gtest.h>
@@ -51,6 +53,8 @@
 
     QCoreApplication application(argc, argv);
     DBusMock::registerMetaTypes();
+    DBusTypes::registerMetaTypes();
+
     ::testing::InitGoogleTest(&argc, argv);
 
     Runner runner;