Merge lp:~michihenning/storage-framework/add-doc into lp:storage-framework/devel

[Michi Henning]

Added tutorial and provider reference documentation.

Added tutorial and provider reference documentation.

I'd appreciate any and all comments. Typos, style/grammar issues, omissions, inaccuracies, etc. In particular, what did I not say that should have been said?

Besides adding doc, this MR also fixes a few other things I stumbled into:

- A few extra tests for corner cases

- Previously protected members in UploadJob, TempfileUploadJOb, and DownloadJob are now private.

- Attempts to delete or move a root now throw PermissionException instead of LogicException.

- Attempts to list a file or download/upload a folder now throw LogicException instead of InvalidArgumentException.

[unity-api-1-bot (unity-api-1-bot)]

PASSED: Continuous integration, rev:127
https://jenkins.canonical.com/unity-api-1/job/lp-storage-framework-ci/275/
Executed test runs:
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build/1886
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-0-fetch/1893
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=xenial+overlay/1675
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=xenial+overlay/1675/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=zesty/1675
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=zesty/1675/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=xenial+overlay/1675
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=xenial+overlay/1675/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=zesty/1675
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=zesty/1675/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=xenial+overlay/1675
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=xenial+overlay/1675/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=zesty/1675
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=zesty/1675/artifact/output/*zip*/output.zip

Click here to trigger a rebuild:
https://jenkins.canonical.com/unity-api-1/job/lp-storage-framework-ci/275/rebuild

review: Approve (continuous-integration)

[unity-api-1-bot (unity-api-1-bot)]

PASSED: Continuous integration, rev:128
https://jenkins.canonical.com/unity-api-1/job/lp-storage-framework-ci/276/
Executed test runs:
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build/1887
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-0-fetch/1894
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=xenial+overlay/1676
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=xenial+overlay/1676/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=zesty/1676
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=zesty/1676/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=xenial+overlay/1676
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=xenial+overlay/1676/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=zesty/1676
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=zesty/1676/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=xenial+overlay/1676
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=xenial+overlay/1676/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=zesty/1676
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=zesty/1676/artifact/output/*zip*/output.zip

Click here to trigger a rebuild:
https://jenkins.canonical.com/unity-api-1/job/lp-storage-framework-ci/276/rebuild

review: Approve (continuous-integration)

[Marcus Tomlinson (marcustomlinson)]

First pass spell checking (inline)

[James Henstridge (jamesh)]

Some comments left inline.

review: Needs Fixing

[unity-api-1-bot (unity-api-1-bot)]

FAILED: Continuous integration, rev:130
https://jenkins.canonical.com/unity-api-1/job/lp-storage-framework-ci/277/
Executed test runs:
    FAILURE: https://jenkins.canonical.com/unity-api-1/job/build/1915/console
    FAILURE: https://jenkins.canonical.com/unity-api-1/job/build-0-fetch/1922/console

Click here to trigger a rebuild:
https://jenkins.canonical.com/unity-api-1/job/lp-storage-framework-ci/277/rebuild

review: Needs Fixing (continuous-integration)

[unity-api-1-bot (unity-api-1-bot)]

PASSED: Continuous integration, rev:131
https://jenkins.canonical.com/unity-api-1/job/lp-storage-framework-ci/278/
Executed test runs:
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build/1916
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-0-fetch/1923
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=xenial+overlay/1704
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=xenial+overlay/1704/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=zesty/1704
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=zesty/1704/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=xenial+overlay/1704
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=xenial+overlay/1704/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=zesty/1704
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=zesty/1704/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=xenial+overlay/1704
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=xenial+overlay/1704/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=zesty/1704
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=zesty/1704/artifact/output/*zip*/output.zip

Click here to trigger a rebuild:
https://jenkins.canonical.com/unity-api-1/job/lp-storage-framework-ci/278/rebuild

review: Approve (continuous-integration)

[Michi Henning (michihenning)]

Thanks heaps for the reviews, guys!

I've fixed the spelling mistakes (plus a few others I stumbled across).

I've responded to a few comments inline. Where I didn't respond, I believe I've addressed the concern. Let me know if something still doesn't look right please!

[James Henstridge (jamesh)]

Looks good.

review: Approve

[unity-api-1-bot (unity-api-1-bot)]

FAILED: Autolanding.
More details in the following jenkins job:
https://jenkins.canonical.com/unity-api-1/job/lp-storage-framework-autoland/102/
Executed test runs:
    FAILURE: https://jenkins.canonical.com/unity-api-1/job/build/1925/console
    FAILURE: https://jenkins.canonical.com/unity-api-1/job/build-0-fetch/1932/console

review: Needs Fixing (continuous-integration)

[lp:~michihenning/storage-framework/add-doc]

132. By Michi Henning on 2017-04-06

Merged devel.

[Michi Henning (michihenning)]

Re-approving for Jenkins after fixing merge conflict.

review: Approve

[unity-api-1-bot (unity-api-1-bot)]

PASSED: Continuous integration, rev:132
https://jenkins.canonical.com/unity-api-1/job/lp-storage-framework-ci/282/
Executed test runs:
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build/1926
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-0-fetch/1933
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=xenial+overlay/1713
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=xenial+overlay/1713/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=zesty/1713
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=zesty/1713/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=xenial+overlay/1713
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=xenial+overlay/1713/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=zesty/1713
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=zesty/1713/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=xenial+overlay/1713
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=xenial+overlay/1713/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=zesty/1713
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=zesty/1713/artifact/output/*zip*/output.zip

Click here to trigger a rebuild:
https://jenkins.canonical.com/unity-api-1/job/lp-storage-framework-ci/282/rebuild

review: Approve (continuous-integration)

=== modified file 'CMakeLists.txt'
--- CMakeLists.txt	2017-03-30 00:57:11 +0000
+++ CMakeLists.txt	2017-04-06 07:42:38 +0000
@@ -134,6 +134,7 @@
 add_subdirectory(tests)
 add_subdirectory(demo)
 add_subdirectory(tools)
+add_subdirectory(doc)
 
 enable_coverage_report(
     TARGETS

=== modified file 'debian/changelog'
--- debian/changelog	2017-04-06 04:57:35 +0000
+++ debian/changelog	2017-04-06 07:42:38 +0000
@@ -2,6 +2,7 @@
 
   [ Michi Henning ]
   * Removed dependency on libgio.
+  * Added tutorial and provider reference documentation.
 
   [ James Henstridge ]
   * Add systemd units for registry and local provider D-Bus services.

=== modified file 'debian/control'
--- debian/control	2017-03-20 04:51:09 +0000
+++ debian/control	2017-04-06 07:42:38 +0000
@@ -110,3 +110,11 @@
          ${misc:Depends},
 Description: Header files for the Storage Framework provider library
  Development C++ headers for the provider API.
+
+Package: storage-framework-doc
+Section: doc
+Architecture: all
+Multi-Arch: foreign
+Depends: ${misc:Depends},
+Description: Documentation for storage-framework-dev
+ Tutorial and API reference.

=== modified file 'debian/control.in'
--- debian/control.in	2017-03-17 05:04:09 +0000
+++ debian/control.in	2017-04-06 07:42:38 +0000
@@ -105,3 +105,11 @@
          ${misc:Depends},
 Description: Header files for the Storage Framework provider library
  Development C++ headers for the provider API.
+
+Package: storage-framework-doc
+Section: doc
+Architecture: all
+Multi-Arch: foreign
+Depends: ${misc:Depends},
+Description: Documentation for storage-framework-dev
+ Tutorial and API reference.

=== added file 'debian/storage-framework-doc.install'
--- debian/storage-framework-doc.install	1970-01-01 00:00:00 +0000
+++ debian/storage-framework-doc.install	2017-04-06 07:42:38 +0000
@@ -0,0 +1,1 @@
+usr/share/doc/storage-framework/*

=== added directory 'doc'
=== added file 'doc/CMakeLists.txt'
--- doc/CMakeLists.txt	1970-01-01 00:00:00 +0000
+++ doc/CMakeLists.txt	2017-04-06 07:42:38 +0000
@@ -0,0 +1,20 @@
+find_package(DoxygenBuilder)
+
+add_doxygen(
+    storage-framework-doc
+    INPUT
+        ${CMAKE_CURRENT_SOURCE_DIR}
+        ${CMAKE_SOURCE_DIR}/include/unity/storage
+    STRIP_FROM_PATH
+        ${CMAKE_SOURCE_DIR}
+    STRIP_FROM_INC_PATH
+        ${CMAKE_SOURCE_DIR}/include
+    EXCLUDE_PATTERNS
+        */internal/*
+        */qt/*
+    ALL
+)
+
+install(DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}/html
+    DESTINATION share/doc/storage-framework
+)

=== added file 'doc/provider_tut.dox'
--- doc/provider_tut.dox	1970-01-01 00:00:00 +0000
+++ doc/provider_tut.dox	2017-04-06 07:42:38 +0000
@@ -0,0 +1,588 @@
+/**
+\mainpage Tutorial
+
+\section overview Overview
+
+The storage framework provides a general-purpose file system abstraction that is independent of any physical storage
+mechanism and the APIs that are needed to manipulate different kinds of storage. This allows client applications to
+access and modify files that exist in remote storage mechanisms (such as
+<a href="https://www.google.com/drive/">Google Drive</a> or <a href="https://owncloud.org/">Owncloud</a>) as well
+as files in the local file system through a common API.
+
+The framework includes a provider for local files; to allow access to remote files, the framework
+requires provider implementations that adapt the common API to each cloud service.
+
+Each provider implementation runs as a separate DBus service on the session bus; the storage framework
+provides run-time support that takes care of DBus communications, error handling, sanitizing input parameters,
+authorization, and so on. This allows providers to be created without having to handle these details, and enables
+the addition of new providers over time without disturbing existing provider implementations.
+
+Ubuntu includes a number of provider implementations, among them an
+<a href="https://code.launchpad.net/storage-provider-webdav">Owncloud provider</a> and an
+<a href="https://launchpad.net/mcloud">mCloud provider</a>.
+
+\section semantics File System Semantics
+
+The storage framework establishes common semantics that apply to all storage backends.
+Clients must not expect anything not guaranteed by these semantics, and providers must implement the semantics
+as faithfully as possible.
+
+The operations that are available are necessarily limited to those that can be supported by the
+majority of backends:
+
+- List the root folder (or root folders; some providers support multiple roots)
+
+- Create and destroy folders or files
+
+- Move or copy a file or folder
+
+- List the contents of a folder
+
+- Look up a file or folder by name
+
+- Look up a file or folder by identity
+
+- Upload or download a file
+
+- Get additional metadata (such as modification time or mime type) for a file or folder
+
+Note that all data transfers can be performed only by uploading or downloading an entire file. The framework
+does not support random access to files.
+
+Move and copy operations can be performed only within the same account. If an application wants to copy or move files
+from one account to another, it must download the files from the source account and upload them to the target account
+(and, for a move, delete the source files after uploading them).
+
+\subsection accounts Accounts
+
+The storage framework is integrated with
+<a href="https://help.ubuntu.com/stable/ubuntu-help/accounts.html">Online Accounts</a>. When a client application
+uses the framework, it first retrieves a list of available accounts from the framework. The application only receives
+those accounts for which the user has authorized access. The application then retrieves the root (or roots) for that
+account, and uses the root to access and modify files and folders underneath that root.
+On the server side, the runtime passes any
+authorization token that is needed to access the cloud backend to the provider implementation; the runtime
+also handles re-authorization in case a token has expired.
+
+\subsection identity File and Folder Identity
+
+Each file and folder has an identity that is unique within the account that contains the file.
+Identities are opaque strings that are assigned by the provider. (Clients cannot control the identity
+of files and folders).
+
+The identity of files and folders is immutable during their life time, and identities
+can be compared for equality and less-than.
+This means that clients can store identities in an external file or a database and expect them to
+continue to refer to the same file or folder even after a process re-start.
+
+Identities are not globally unique; different files in different accounts may have the same identity.
+
+Identities may not be unique across object life times. For example, if a client creates a file and deletes that
+file again, another file that is created later may get the identity that was used by the earlier file.
+Similarly, moving a file or folder may (or may not) change its identity.
+
+\subsection etags ETags
+
+Files have an associated <a href="https://en.wikipedia.org/wiki/HTTP_ETag">ETag</a>. The ETag for a file
+changes every time a file is updated. When uploading or downloading a file, a client can specify whether to check
+the ETag of a file. If the client requests the ETag to be checked, the provider performs the
+upload or download only if the ETag sent by the client matches the ETag of the file. This allows clients to ensure
+that a file has not been modified since the client last retrieved the file's metadata.
+
+ETags have no particular format; they simply are opaque strings that change each time a file changes.
+
+\subsection names File and Folder Names
+
+Clients choose the name of files and folders they create. However, providers may modify a name that is provided
+by a client, for example, a provider could map upper-case to lower-case letters.
+The actual name of a new file or folder is returned by the provider as part of the metadata.
+
+Different providers place different restrictions on the characters that are allowed as part of a name.
+The following meta-characters are likely to cause problems and should not be used in file or folder names:
+
+<tt>/</tt>, <tt>\\</tt>, <tt>*</tt>, <tt>"</tt>, <tt>:</tt>, <tt>\<</tt>, <tt>\></tt>, <tt>|</tt>, <tt>?</tt>, <tt>#</tt>, and <tt>%</tt>.
+
+The name of a file or folder is not necessarily unique within its parent folder. (Some cloud services
+allow different files with the same name in the same folder.) Similarly, a single file or folder may
+have more than one name within the same parent folder. (The name of a file or folder does not indicate
+anything about its identity. If you want to establish whether a file is the same as another file, you must
+compare the files' identities, not their names.)
+
+The name of a root folder can be anything. (It might be <tt>/</tt>, but could be something else.)
+
+\subsection metadata Metadata
+Files and folders have metadata associated with them.
+Metadata items are key&ndash;value pairs. The key (a string) identifies the item and determines the type of
+the corresponding value (such as string or integer).
+
+Some metadata is guaranteed to always be present (such as file
+size and modification time); other metadata may or may not be available, depending on the provider. (For example,
+not all providers maintain a modification time for directories.)
+
+The framework defines a number of well-known metadata keys that are used by the majority of providers (such as
+<tt>size_in_bytes</tt>). In addition, providers can also add provider-specific metadata. Clients can specify
+which metadata items should be returned by the provider. The requested metadata items are hints only: a provider
+may return fewer or more metadata items than were requested by the client.
+
+A special metadata key (<tt>__ALL__</tt>)
+indicates that the provider should return all available metadata. If a client does not request any specific
+metadata, the provider returns a default set of items. (Clients can specify which metadata should be returned
+in order to minimize the amount of data that is copied over the network; some providers support extensive metadata
+that consumes considerable bandwidth.)
+
+If a provider supplies non-standard metadata items, the item keys have a provider-specific prefix, such as
+<tt>mcloud:</tt> or <tt>gdrive:</tt>.
+
+\subsection uploads-downloads Uploads and Downloads
+
+Uploads and downloads take place over a UNIX domain socket. When a client requests an upload, the runtime creates
+a socket pair and passes a socket that is open for writing to the client, and a socket that is open for reading
+to the provider. (For downloads, the read and write ends are reversed.) The client and provider write and read
+the data for the file to/from their respective sockets. Once the client has written all the data, it makes one
+final API call to check whether the provider has correctly received all the data. This call returns the new metadata
+for the file (or an error, if the upload failed).
+
+Downloads work the same way as uploads, but with the read and write roles reversed.
+
+\section provider Implementing a Provider
+
+This section provides an overview of the provider API and explains the semantics you are expected to adhere
+to in your provider implementation. We strongly recommend that you also read the source code
+for the <a href="https://code.launchpad.net/storage-provider-webdav">Owncloud provider</a>. It is useful
+to learn how to implement a provider that uses HTTP to access a cloud service. (While the details
+of the interactions with your provider will differ, the principles will be
+very similar.)
+
+\subsection provider-overview Overview
+
+To implement a provider, you must create an implementation of the abstract
+@ref unity::storage::provider::ProviderBase "ProviderBase" class:
+
+\code{.cpp}
+class ProviderBase : public std::enable_shared_from_this<ProviderBase>
+{
+public:
+    ProviderBase();
+    virtual ~ProviderBase();
+
+    virtual boost::future<ItemList> roots(vector<string> const& keys,
+                                          Context const& context) = 0;
+    virtual boost::future<ItemList> lookup(string const& parent_id,
+                                           string const& name,
+                                           vector<string> const& keys,
+                                           Context const& context) = 0;
+    // More methods here...
+};
+\endcode
+
+The base class defines a pure virtual method for each operation (such as
+@ref unity::storage::provider::ProviderBase::list "list()",
+@ref unity::storage::provider::ProviderBase::lookup "lookup()",
+@ref unity::storage::provider::ProviderBase::update "update()", etc.)
+
+Note that each operation returns a <code>boost::future</code> to the storage framework runtime;
+your operation implementation must make the future ready once the result of the operation is available.
+
+For this example, we assume that you want to create a provider for a (hypothetical) <code>MyCloud</code>
+storage service.
+
+To connect your implementation class to the runtime, you instantiate a <code>Server</code> instance and call its
+@ref unity::storage::provider::Server::init "init()" and
+@ref unity::storage::provider::Server::run "run()" methods:
+
+\code{.cpp}
+class MyCloudProvider : public ProviderBase
+{
+public:
+    MyCloudProvider();
+    virtual ~MyCloudProvider();
+
+    boost::future<ItemList> roots(vector<string> const& keys,
+                                  Context const& context) override;
+    boost::future<ItemList> lookup(string const& parent_id,
+                                   string const& name,
+                                   vector<string> const& keys,
+                                   Context const& context) override;
+    // More methods here...
+};
+
+int main(int argc, char* argv[])
+{
+    string const bus_name = "com.acme.StorageFramework.Provider.MyCloud";
+    string const account_service_id = "storage-provider-mycloud";
+
+    try
+    {
+        Server<MyCloudProvider> server(bus_name, account_service_id);
+        server.init(argc, argv);
+        return server.run();
+    }
+    catch (std::exception const& e)
+    {
+        cerr << argv[0] << ": " << e.what() << endl;
+        return 1;
+    }
+}
+\endcode
+
+The <code>Server</code> class is a template that instantiates your <code>MyCloudProvider</code> class.
+You must provide a DBus name for your service, as well as the service ID with which your provider is
+known to Online Accounts.
+
+The call to @ref unity::storage::provider::Server::init "init()" initializes the storage framework runtime,
+connects the service to DBus and, once
+you call @ref unity::storage::provider::Server::run "run()",
+starts an event loop that forwards incoming requests to the methods of your
+<code>MyCloudProvider</code> class.
+
+The @ref unity::storage::provider::Server::run "run()"
+method returns after some period of inactivity (30 seconds, by default), so the provider
+process shuts down when it is not needed, and is started automatically if it is not running at the time a client
+request arrives.
+
+If you need to pass additional arguments to the <code>MyCloudProvider</code> constructor, you can do so
+by creating a server class of your own that derives from @ref unity::storage::provider::ServerBase "ServerBase".
+That class has an abstract @ref unity::storage::provider::ServerBase::make_provider() "make_provider()"
+factory method that you can override. The runtime calls the factory method to instantiate your provider.
+
+\subsection threading Threading Considerations
+
+The runtime invokes all methods of your provider implementation on the main thread. Your methods
+<i>must not</i> block for any length of time. Unless you know that a method will return
+immediately (within a few milliseconds), you <i>must</i> implement the method such that it can
+complete in the background (either using asynchronous I/O or a separate thread).
+
+The runtime uses <a href="http://doc.qt.io/qt-5/qcoreapplication.html"><code>QCoreApplication</code></a>,
+so your implementation can use anything that uses the Qt or glib event loop. (Alternatively, you can also
+run a separate event loop if you prefer.)
+
+When starting an async operation, create a
+<a href="http://www.boost.org/doc/libs/1_63_0/libs/fiber/doc/html/fiber/synchronization/futures/promise.html"><code>boost::promise</code></a> and return <code>promise::get_future()</code>; on completion,
+call <code>set_value()</code> or <code>set_exception()</code> to make the future ready.
+
+For a synchronous implementation, you can use
+<a href="http://www.boost.org/doc/libs/1_63_0/doc/html/thread/synchronization.html#thread.synchronization.futures.async"><code>boost::async()</code></a> to spawn a new thread. To indicate errors, call
+<a href="http://www.boost.org/doc/libs/1_63_0/libs/exception/doc/enable_current_exception.html"><code>boost::enable_current_exception()</code></a>
+or <a href="http://www.boost.org/doc/libs/master/libs/exception/doc/throw_exception.html"><code>boost::throw_exception</code></a>.
+
+The storage framework runtime is thread-safe, that is, you can up-call into the runtime (for example, to indicate
+an error during an upload) from any thread. The runtime takes care of any necessary locking and moving data
+to the correct thread for marshaling.
+
+\subsection provider-error-handling Error Handling
+
+All error handling uses exceptions. If you call into the runtime and it detects a problem, the runtime throws
+an exception that derives from <code>std::exception</code> or, for specific well-known problems, from
+@ref unity::storage::provider::StorageException "StorageException". (The code in <code>main()</code> above
+relies on this behavior to report any errors and return non-zero exit status.)
+
+If your implementation uses a library that does not transparently handle exceptions (such Qt), you
+must catch exceptions and deal with them as appropriate when you call into the runtime.
+
+Conversely, the runtime is exception-safe. If you encounter an error during an operation that
+throws an exception and you let the exception escape, the runtime will intercept the exception, abort the
+operation, and marshal an appropriate error back to the client. (If you throw something unexpected, such as
+<tt>99</tt> or a <code>std::exception</code>, the client receives a generic error.)
+This means that, to indicate an error to the runtime, you can always throw an exception or, alternatively,
+for methods that return a future, return a future that contains the exception.
+
+When throwing an exception, be as detailed in the error message as possible and include any details you receive
+from the cloud provider, as well as the identity of the file. Without this, it may be impossible to
+diagnose a problem from log files.
+
+The storage framework defines a small number of exceptions that indicate specific error conditions to the client.
+You <i>must</i> use these exception to indicate their respective error conditions, otherwise
+the client only knows that something did not work, but does not know the real cause of the error. <i>All</i> providers
+<i>must</i> adhere to this common set of error semantics, otherwise clients cannot implement meaningful
+error messages and error recovery.
+
+All provider exceptions derive from a common abstract base exception class:
+
+\code{.cpp}
+class StorageException : public std::exception
+{
+public:
+    // ...
+    virtual char const* what() const noexcept override;
+
+    std::string type() const;
+    std::string error_message() const;
+};
+
+class RemoteCommsException : public StorageException { ... };
+class NotExistsException : public StorageException { ... };
+// etc...
+\endcode
+
+The @ref unity::storage::provider::StorageException::type "type()"
+method returns the name of the exception (such as "NotExistsException"), and
+@ref unity::storage::provider::StorageException::error_message "error_message()"
+returns a message you provide when you throw the exception.
+(@ref unity::storage::provider::StorageException::what "what()"
+returns a string that contains both the exception name and the error message.)
+
+The concrete derived exceptions indicate error semantics as follows:
+
+- @ref unity::storage::provider::RemoteCommsException "RemoteCommsException"
+
+  This exception indicates an (unexpected) error in the communication between your provider and the remote
+  cloud service, such as the remote server being unreachable or returning garbled data.
+
+- @ref unity::storage::provider::NotExistsException "NotExistsException"
+
+  This exception indicates that a file or folder the client wants to access does not exist. You must
+  use this exception only if the remote cloud service has <i>authoritatively</i> indicated non-existence
+  (such as with an HTTP 404 response). Do not use this exception for non-authoritative errors, such as
+  timeouts or similar.
+
+- @ref unity::storage::provider::ExistsException "ExistsException"
+
+  This exception indicates that a file or folder already exists, preventing an operation (such as a create or copy)
+  from completing. Use this exception only if the remote cloud service has <i>authoritatively</i>
+  indicated this error. Do not use this exception for non-authoritative errors, such as timeouts
+  or similar.
+
+- @ref unity::storage::provider::ConflictException "ConflictException"
+
+  This exception indicates an ETag mismatch: the client asked for a file to be updated or downloaded, but
+  the file's ETag has changed since.
+
+- @ref unity::storage::provider::UnauthorizedException "UnauthorizedException"
+
+  This exception indicates that the remote cloud service has indicated an authorization failure. Note that
+  you must throw this exception only if the cloud service indicates invalid credentials. Do <i>not</i> throw
+  this exception for permission errors (such as an attempt to write to a read-only file). In particular,
+  do not throw this exception for an HTTP 403 error, which indicates that access to a particular resource
+  was denied even though the credentials were valid.
+
+  Some cloud providers use authorization tokens that expire after some time. If you detect an
+  authorization failure in your code and throw this exception, the runtime re-retrieves the token from Online
+  Accounts and transparently calls your operation again if the new token differs. If the runtime receives an
+  @ref unity::storage::provider::UnauthorizedException "UnauthorizedException" on this second attempt too,
+  it returns the error to the client.
+
+- @ref unity::storage::provider::PermissionException "PermissionException"
+
+  This exception indicates that a file or folder cannot be accessed due to insufficient permissions (such as an
+  attempt to write to a read-only file). Do <i>not</i> throw this exception for authorization errors.
+
+- @ref unity::storage::provider::QuotaException "QuotaException"
+
+  This exception indicates that the provider has run out of space or that the maximum number of files and/or
+  folders was exceeded.
+
+- @ref unity::storage::provider::CancelledException "CancelledException"
+
+  This exception indicates that the client attempted to interact with an upload or download after it was cancelled.
+  Due to the way the API is structured, you will not ever need to throw this exception. It is provided for
+  completeness and to allow for provider implementations that do not use the storage framework API.
+
+- @ref unity::storage::provider::LogicException "LogicException"
+
+  This exception indicates that the client has used the API incorrectly, such as indicating that an upload has
+  completed before it has written all the data. In general, use
+  @ref unity::storage::provider::LogicException "LogicException"
+  for any errors that relate to operations being called in the wrong order, and for semantic errors
+  (such as an attempt to download a folder or to list a file).
+
+- @ref unity::storage::provider::InvalidArgumentException "InvalidArgumentException"
+
+  This exception indicates that the value of an argument supplied by the client is unacceptable, such as
+  an empty string where a non-empty string was excepted, or a negative integer where a positive integer was expected.
+
+- @ref unity::storage::provider::ResourceException "ResourceException"
+
+  This is a generic "catch-all" exception that you can throw to indicate unexpected errors that do not fit into
+  any of the other categories. For example, <code>ResourceException</code> is appropriate to indicate out of
+  file descriptors, failure to locate a configuration file, or any other unexpected system error. You should
+  provide as much contextual information about such errors as possible. In particular, unexpected errors
+  typically need to be diagnosed from log files. This means that you should provide, at least, the full error
+  message you received from the cloud service (or the operating system), together with all other relevant
+  details, such as the name and identity of the file, the URL of a failed request,
+  any HTTP error information, and so on.
+
+- @ref unity::storage::provider::UnknownException "UnknownException"
+
+  If your code throws an exception that does not derive from <code>StorageException</code>, it returns
+  <code>UnknownException</code> to the client. Do not throw this exception explicitly; is is a "catch-all"
+  exception that the runtime uses to indicate that something completely unexpected happened. (If a client
+  receives this exception from your provider, chances are that you have forgotten to handle an error condition
+  somewhere.)
+
+\subsection metadata-keys Metadata Keys
+Each method of your provider has a <code>keys</code> parameter of type <code>vector<string></code>. The parameter
+is provided by the client and indicates which metadata items you should include in the metadata you return
+from the operation.
+
+If the vector is empty, this indicates that you should return a set of default metadata items. In practice, this
+means that you should return whatever metadata you can cheaply receive from the cloud provider (in terms
+of bandwidth and number of requests).
+
+If the vector contains a single entry <code>__ALL__</code>, you should return complete metadata. In particular,
+you should return as many of the well-known metadata items that are defined in
+common.h as possible, plus
+any provider-specific metadata items you may support (prefixed with a provider name, such as <tt>Mycloud:</tt>).
+
+If the client provides a specific list of keys, you should restrict the metadata to those items requested by the
+client. This is useful to reduce traffic between your provider and the cloud service, if the cloud service
+requires additional requests for particular metadata items: you need to provide that metadata only if the client asks
+for it.
+
+The keys provided by the client are a hint only, that is, it is fine to provide more items than the client asked for
+(if you happen to have them available anyway at no extra cost), and it is legal to not provide all of the items
+the client asked for.
+
+If you receive a key that is not one of the keys defined in <code>common.h</code> and is not one of the keys
+you recognize as provider-specific, do <i>not</i> return an error from the operation. Instead, write a log message
+with the details (your provider name, the identity and name of the file and folder, and the invalid key) and
+ignore the invalid key.
+
+Note that <code>common.h</code> defines symbolic constants for metadata keys. Use these symbolic constants
+instead of hard-coding the corresponding string literals.
+
+\note For files, the <code>SIZE_IN_BYTES</code> and <code>LAST_MODIFIED_TIME</code> metadata items are mandatory;
+you must always supply these, whether the client asks for them or not.
+
+\subsection authorization Authorization
+
+Each method of your provider has a trailing parameter of type
+@ref unity::storage::provider::Context "Context":
+
+\code{.cpp}
+struct Context
+{
+    uid_t uid;
+    pid_t pid;
+    string security_label;
+
+    Credentials credentials;
+};
+\endcode
+
+The structure contains the user ID, process ID, and the
+<a href="https://help.ubuntu.com/lts/serverguide/apparmor.html">Apparmor</a> security label of the calling client,
+plus credentials that you can use for authorization with the cloud service. In turn, the credentials are a variant
+type that enables authorization with <a href="https://oauth.net/1/">OAuth1</a>,
+<a href="https://oauth.net/2/">OAuth2</a>, as well as with user name and password:
+
+\code{.cpp}
+struct NoCredentials
+{
+};
+
+struct OAuth1Credentials
+{
+    string consumer_key;
+    string consumer_secret;
+    string token;
+    string token_secret;
+};
+
+struct OAuth2Credentials
+{
+    string access_token;
+};
+
+struct PasswordCredentials
+{
+    string username;
+    string password;
+    string host;
+};
+
+typedef boost::variant<boost::blank,OAuth1Credentials,OAuth2Credentials,PasswordCredentials> Credentials;
+\endcode
+
+If you receive an authorization error from the cloud provider with the credentials that are passed to you
+by the runtime, you must throw an @ref unity::storage::provider::UnauthorizedException "UnauthorizedException".
+This prompts the runtime to refresh the token if it may have expired and call your your method
+a second time with the new token. If that second attempt also throws
+@ref unity::storage::provider::UnauthorizedException "UnauthorizedException", the runtime propagates the
+error to the client.
+
+\subsection download-upload Downloads and Uploads
+
+To implement download and upload, you must create implementations of the abstract
+@ref unity::storage::provider::DownloadJob "DownloadJob" and @ref unity::storage::provider::UploadJob "UploadJob"
+classes and return them from your @ref unity::storage::provider::ProviderBase::download "download()" and
+@ref unity::storage::provider::ProviderBase::update "update()" (or
+@ref unity::storage::provider::ProviderBase::create_file "create_file()") methods, respectively.
+The two base classes are very similar, so we focus on @ref unity::storage::provider::DownloadJob "DownloadJob" here.
+
+\code{.cpp}
+class MyDownloadJob : public unity::storage::provider::DownloadJob
+{
+public:
+    MyDownloadJob(shared_ptr<MyCloudProvider> const& provider,
+                  string const& item_id,
+                  string const& match_etag);
+    virtual ~LocalDownloadJob();
+
+    boost::future<void> cancel() override;
+    boost::future<void> finish() override;
+
+private:
+    // ...
+};
+\endcode
+
+In the implementation of your constructor, you must initiate the download. The download must be able to proceed
+without blocking. How you do this is up to you. You can run the download in a separate thread, use the
+runtime's Qt event loop, or run a separate event loop. (The storage framework runtime does not care.)
+
+Your <code>MyDownloadJob</code> receives a socket that is open for writing and connected to the client by calling
+@ref unity::storage::provider::DownloadJob::write_socket "write_socket()" on its base class. You must write the
+data for the file to the socket (from which the client can read it).
+
+At some point, the runtime either calls your @ref unity::storage::provider::DownloadJob::cancel "cancel()" method,
+in which case you simply abort the download, or it calls @ref unity::storage::provider::DownloadJob::finish "finish()".
+The purpose of @ref unity::storage::provider::DownloadJob::finish "finish()" is to confirm whether the download
+completed successfully. At this point, you must check whether you have received <a>all</a> of the data for the file
+from the cloud provider and have successfully written it to the socket. If so, you return a ready future; if not,
+you return a future that holds the appropriate @ref unity::storage::provider::StorageException "StorageException"
+to indicate the nature of the error.
+
+@ref unity::storage::provider::DownloadJob "DownloadJob" and @ref unity::storage::provider::UploadJob "UploadJob"
+classes also provide a @ref unity::storage::provider::DownloadJob::report_error "report_error()" method that
+you can use to report any errors that arise while your downloader runs in the background. (If you call 
+@ref unity::storage::provider::DownloadJob::report_error "report_error()", neither
+@ref unity::storage::provider::DownloadJob::finish "finish()" nor 
+@ref unity::storage::provider::DownloadJob::cancel "cancel()" are called by the runtime.)
+
+The @ref unity::storage::provider::DownloadJob "DownloadJob" base class also provides a
+@ref unity::storage::provider::DownloadJob::report_complete "report_complete()" method that you call
+to indicate successful completion without having to wait for the call to 
+@ref unity::storage::provider::DownloadJob::finish "finish()".
+
+\subsection buffering Download and Upload Buffering
+
+When implementing your provider, you need to be aware of when (and when not) to buffer data.
+
+For downloads, you should write the data to the client socket as soon as you receive it from the cloud provider.
+This is necessary to enable streaming. (A media player should be able to start playing a song or video as soon
+as possible.)
+
+For uploads, the situation is the opposite, and you should buffer any uploads that are larger than than a few
+kilobytes in the file system before starting the upload with the provider.
+This is particularly important for mobile devices, where applications may be suspended for extended periods.
+If you write data to the cloud provider while the client is still writing it, and the client application is suspended,
+the upload gets stuck until the client application resumes (by which time your connection with the cloud provider
+may have timed out, or the user may have walked out of reach of the network).
+
+Note that, eventually, you will either receive the @ref unity::storage::provider::DownloadJob::finish "finish()"
+call to confirm whether or not there was an error (or a @ref unity::storage::provider::DownloadJob::cancel "cancel()"
+call if the client application dies). This means that, if your cloud provider is able to resume a previously
+interrupted upload, you can re-start an upload that failed due to loss of the network. (Be judicious in how
+aggressively you try to resume though; on a battery-powered device, it is expensive to turn on the radio, so you
+should use a back-off algorithm and give up if the upload cannot be resumed within a reasonable period of time.)
+
+\section local-provider The Local Provider
+
+The storage framework includes a local provider that stores files in the local file system.
+In a classic environment, the root of the local files is <code>$XDG_DATA_DIR/storage-framework/local</code>;
+in a Snap environment, the root is <code>$SNAP_USER_COMMON/storage-framework/local</code>.
+You can set <code>$SF_LOCAL_PROVIDER_ROOT</code> to change the root directory. (This is intended mainly for testing.)
+
+The local provider uses <code>com.canonical.StorageFramework.Provider.Local</code> as the DBus name. The object path
+is <code>/provider/0</code>.
+*/

=== modified file 'include/unity/storage/common.h'
--- include/unity/storage/common.h	2016-11-03 03:41:24 +0000
+++ include/unity/storage/common.h	2017-04-06 07:42:38 +0000
@@ -23,36 +23,57 @@
 namespace storage
 {
 
+/**
+\brief Indicates the type of an item.
+*/
+
 enum class ItemType
 {
-    file,
-    folder,
-    root,
-    LAST_ENTRY__
+    file,         /*!< The item represents a file. */
+    folder,       /*!< The item represents a (non-root) folder. */
+    root,         /*!< The item represents a root folder. */
+    LAST_ENTRY__  /*!< End of enumeration marker. */
 };
 
+/**
+\brief Determines the behavior in case of an ETag mismatch.
+*/
+
 enum class ConflictPolicy
 {
-    error_if_conflict,
-    ignore_conflict,
+    error_if_conflict,  /*!< Return an error if the ETag has changed. */
+    ignore_conflict,    /*!< Ignore ETag mismatch and overwrite or replace the file. */
     overwrite = ignore_conflict, // TODO: remove this, it's here only for compatibility with v1 API
 };
 
+/**
+\brief This namespace defines well-known metadata keys.
+
+See \ref common.h for details.
+*/
+
 namespace metadata
 {
 
-static char constexpr SIZE_IN_BYTES[] = "size_in_bytes";            // int64_t, >= 0
-static char constexpr CREATION_TIME[] = "creation_time";            // String, ISO 8601 format
-static char constexpr LAST_MODIFIED_TIME[] = "last_modified_time";  // String, ISO 8601 format
-static char constexpr CHILD_COUNT[] = "child_count";                // int64_t, >= 0
-static char constexpr DESCRIPTION[] = "description";                // String
-static char constexpr DISPLAY_NAME[] = "display_name";              // String
-static char constexpr FREE_SPACE_BYTES[] = "free_space_bytes";      // int64_t, >= 0
-static char constexpr USED_SPACE_BYTES[] = "used_space_bytes";      // int64_t, >= 0
-static char constexpr CONTENT_TYPE[] = "content_type";              // String
-static char constexpr WRITABLE[] = "writable";                      // Bool
-static char constexpr MD5[] = "md5";                                // String
-static char constexpr DOWNLOAD_URL[] = "download_url";              // String
+// Symbolic constants for well-known metadata keys.
+
+// Doxygen bug makes it impossible to document each constant.
+
+static char constexpr SIZE_IN_BYTES[] = "size_in_bytes";            // int64_t >= 0
+static char constexpr CREATION_TIME[] = "creation_time";            // string, ISO 8601 format
+static char constexpr LAST_MODIFIED_TIME[] = "last_modified_time";  // string, ISO 8601 format
+static char constexpr CHILD_COUNT[] = "child_count";                // int64_t >= 0
+static char constexpr DESCRIPTION[] = "description";                // string
+static char constexpr DISPLAY_NAME[] = "display_name";              // string
+static char constexpr FREE_SPACE_BYTES[] = "free_space_bytes";      // int64_t >= 0
+static char constexpr USED_SPACE_BYTES[] = "used_space_bytes";      // int64_t >= 0
+static char constexpr CONTENT_TYPE[] = "content_type";              // string
+static char constexpr WRITABLE[] = "writable";                      // int64_t, 0 or 1
+static char constexpr MD5[] = "md5";                                // string
+static char constexpr DOWNLOAD_URL[] = "download_url";              // string
+
+// A single-element vector containing the key ALL indicates that the client would like all available
+// metadata to be returned by the provider.
 
 static char constexpr ALL[] = "__ALL__";
 

=== modified file 'include/unity/storage/provider/DownloadJob.h'
--- include/unity/storage/provider/DownloadJob.h	2016-07-12 02:22:05 +0000
+++ include/unity/storage/provider/DownloadJob.h	2017-04-06 07:42:38 +0000
@@ -37,25 +37,129 @@
 class ProviderInterface;
 }
 
+/**
+\brief Abstract base class for download implementations.
+
+When the runtime calls ProviderBase::download(), you must return
+a new downloader instance that derives from DownloadJob. Your implementation is responsible
+for retrieving the file's data from the cloud provider and writing it to
+the socket provided by the runtime.
+
+You can implement your downloader
+any way you wish, such as by running the download in a separate thread, or
+by using async I/O driven by the runtime's (or any other) event loop.
+
+The runtime invokes all methods on the downloader from the main thread.
+*/
+
 class UNITY_STORAGE_EXPORT DownloadJob
 {
 public:
+    /**
+    \brief Construct a downloader.
+    \param download_id An identifier for this particular download. You can use any non-empty string,
+    as long as it is unique among all downloads that are in progress within the corresponding account.
+    (A simple incrementing counter will work fine, or you can use the download identifier you receive
+    from the cloud service.)
+    The runtime uses the <code>download_id</code> to distinguish different downloads that may be
+    in progress concurrently, and it ensures that each ID can be used only by its corresponding client.
+    */
     DownloadJob(std::string const& download_id);
     virtual ~DownloadJob();
 
+    /**
+    \brief Returns the download ID.
+    \return The value of the <code>download_id</code> parameter that was passed to the constructor.
+    */
     std::string const& download_id() const;
+
+    /**
+    \brief Returns the socket to write the file contents to.
+    \return A socket that is open for writing. You must write the file contents to this socket (which is
+    connected to the client by the runtime).
+    */
     int write_socket() const;
 
-    // If the result of the download is reported with either of the
-    // following two functions, then neither cancel() or finish() will
-    // be called.
+    /**
+    \brief Informs the runtime that a download completed successfully.
+
+    This method makes it unnecessary to wait for the runtime to call finish() in order to confirm
+    whether a download completed successfully. You can call report_complete() as soon as you have
+    written all of a file's data to the download socket and closed it successfully. This can be convenient
+    if a download runs in a separate thread or event loop because there is no need to store success/error state
+    for use in finish().
+
+    You can call report_complete() from an arbitrary thread.
+
+    If you call report_complete(), the runtime guarantees that neither finish() nor cancel() will be called, so
+    you must reclaim any resources associated with the download before calling report_complete().
+    \see report_error(), finish()
+    */
     void report_complete();
-    void report_error(std::exception_ptr p);
-
+
+    /**
+    \brief Informs the runtime that a download encountered an error.
+
+    As for report_complete(), you can call report_error() as soon as you encounter an error during a download
+    (such as losing the connection to the cloud provider, or getting an error when writing to the download
+    socket).
+
+    You can call report_error() from an arbitrary thread.
+
+    If you call report_error(), the runtime guarantees that neither finish() nor cancel() will be called, so
+    you must reclaim any resources associated with the download before calling report_error().
+
+    \param storage_exception You <i>must</i> pass a StorageException to indicate the reason for the failure.
+    \see report_complete(), finish()
+    */
+    void report_error(std::exception_ptr storage_exception);
+
+    /**
+    \brief Cancel this download.
+
+    The runtime calls this method when a client explicitly cancels a download or crashes.
+    Your implementation should reclaim all resources that are used by the download. In particular,
+    you should stop writing any more data and close the download socket. In addition,
+    you should reclaim any resources (such as open connections) that are associated
+    with the download from the cloud provider (possibly after informing the cloud provider of
+    the cancellation).
+
+    The runtime guarantees that cancel() will be called only once, and that finish() will not be called
+    if cancel() was called.
+
+    If any errors are encountered, you <i>must</i> report them by returning a future that stores
+    a StorageException. Do <i>not</i> call report_error() from inside cancel().
+
+    \return A future that becomes ready (or contains a StorageException) once cancellation is complete.
+    */
     virtual boost::future<void> cancel() = 0;
+
+    /**
+    \brief Finalize this download.
+
+    The runtime calls this method when a client finishes a download. Your implementation <i>must</i> verify
+    that it has successfully written <i>all</i> of the file's data at this point.
+    If not, it <i>must</i> store a LogicException
+    in the returned future. (This is essential to make sure that the client can distinguish orderly socket
+    closure from disorderly closure and has not received partial data for the file.)
+
+    You should close the download socket and reclaim any resources (such as open
+    connections) that are associated with the download from the cloud provider.
+
+    The runtime guarantees that finish() will be called only once, and that cancel() will not be called
+    if finish() was called.
+
+    If any errors are encountered, you <i>must</i> report them by returning a future that stores
+    a StorageException. Do <i>not</i> call report_error() from inside finish().
+
+    Do <i>not</i> call report_complete() from inside finish().
+
+    \return A future that becomes ready (or contains a StorageException) once the download is complete.
+    \see report_complete(), report_error()
+    */
     virtual boost::future<void> finish() = 0;
 
-protected:
+private:
     DownloadJob(internal::DownloadJobImpl *p) UNITY_STORAGE_HIDDEN;
     internal::DownloadJobImpl *p_ = nullptr;
 

=== modified file 'include/unity/storage/provider/Exceptions.h'
--- include/unity/storage/provider/Exceptions.h	2017-03-30 02:52:53 +0000
+++ include/unity/storage/provider/Exceptions.h	2017-04-06 07:42:38 +0000
@@ -37,20 +37,39 @@
 //         - src/qt/internal/unmarshal_error.cpp
 
 /**
-\brief Base exception class for all server-side exceptions.
+\brief Abstract base exception class for all server-side exceptions.
 */
+
 class UNITY_STORAGE_EXPORT StorageException : public std::exception
 {
 public:
+    /**
+    \brief Constructs a StorageException.
+    \param exception_type The concrete type name of the exception, such as "ConflictException".
+    \param error_message The error message text for the exception.
+    */
     StorageException(std::string const& exception_type, std::string const& error_message);
 
 protected:
     ~StorageException();
 
 public:
+    /**
+    \brief Returns an error string.
+    \return The error message prefixed by the exception type, such as "ConflictException: ETag mismatch".
+    */
     virtual char const* what() const noexcept override;
 
+    /**
+    \brief Returns the exception type name.
+    \return The value of the <code>exception_type</code> parameter that was passed to the constructor.
+    */
     std::string type() const;
+
+    /**
+    \brief Returns the except error message.
+    \return The value of the <code>error_message</code> parameter that was passed to the constructor.
+    */
     std::string error_message() const;
 
 private:
@@ -61,23 +80,45 @@
 
 /**
 \brief Indicates errors in the communication between the storage provider and the cloud service.
+
+Use this exception for unexpected communication errors, such as the remote server being unreachable
+or returning garbled data.
 */
+
 class UNITY_STORAGE_EXPORT RemoteCommsException : public StorageException
 {
 public:
+    /**
+    \brief Construct a RemoteCommsException.
+    \param error_message The error message for the exception.
+    */
     RemoteCommsException(std::string const& error_message);
     ~RemoteCommsException();
 };
 
 /**
-\brief Indicates that an item does not exist or could not be found.
+\brief Indicates that an item does not exist.
+
+Use this exception only if the remote cloud service has <i>authoritatively</i> indicated non-existence
+(such as with an HTTP 404 response). Do not use this exception for non-authoritative errors, such as
+timeouts or similar.
 */
+
 class UNITY_STORAGE_EXPORT NotExistsException : public StorageException
 {
 public:
+    /**
+    \brief Construct a RemoteCommsException.
+    \param error_message The error message for the exception.
+    \param key The name or identity of the non-existent item.
+    */
     NotExistsException(std::string const& error_message, std::string const& key);
     ~NotExistsException();
 
+    /**
+    \brief Return the key.
+    \return The value of the <code>key</code> parameter that was passed to the constructor.
+    */
     std::string key() const;
 
 private:
@@ -85,15 +126,35 @@
 };
 
 /**
-\brief Indicates that an item cannot be created because it exists already.
+\brief Indicates that an item already exists.
+
+Use this exception only if the remote cloud service has <i>authoritatively</i> indicated that an operation
+cannot be carried out because an item exists already. Do not use this exception for non-authoritative errors,
+such as timeouts or similar.
 */
+
 class UNITY_STORAGE_EXPORT ExistsException : public StorageException
 {
 public:
+    /**
+    \brief Construct an ExistsException.
+    \param error_message The error message for the exception.
+    \param identity The identity of the item.
+    \param name The name of the item.
+    */
     ExistsException(std::string const& error_message, std::string const& identity, std::string const& name);
     ~ExistsException();
 
+    /**
+    \brief Return the identity.
+    \return The value of the <code>identity</code> parameter that was passed to the constructor.
+    */
     std::string native_identity() const;
+
+    /**
+    \brief Return the name of the item.
+    \return The value of the <code>name</code> parameter that was passed to the constructor.
+    */
     std::string name() const;
 
 private:
@@ -102,33 +163,42 @@
 };
 
 /**
-\brief Indicates that an upload or download detected a version mismatch.
+\brief Indicates that an upload or download detected a ETag mismatch.
 */
+
 class UNITY_STORAGE_EXPORT ConflictException : public StorageException
 {
 public:
+    /**
+    \brief Construct a ConflictException.
+    \param error_message The error message for the exception.
+    */
     ConflictException(std::string const& error_message);
     ~ConflictException();
 };
 
 /**
-\brief Indicates that an operation failed because the authentication credentials are invalid or expired.
+\brief Indicates that an operation failed because the credentials are invalid or have expired.
 
 A provider implementation must throw this exception if it cannot reach
-its provider because the credentials are invalid. Do not throw this
+its provider because the credentials are invalid. Do <i>not</i> throw this
 exception if the credentials are valid, but an operation failed due to
 insufficient permission for an item (such as an attempt to write to a
 read-only file).
 
-Typically, this will cause the request to be retried after refreshing
-the authentication credentials, but may be returned to the client on
-repeated failures.
+Typically, this exception will cause the runtime to retry the operation after refreshing
+the credentials; the exception may be returned to the client on repeated failures.
 
 \see PermissionException
 */
+
 class UNITY_STORAGE_EXPORT UnauthorizedException : public StorageException
 {
 public:
+    /**
+    \brief Construct an UnauthorizedException.
+    \param error_message The error message for the exception.
+    */
     UnauthorizedException(std::string const& error_message);
     ~UnauthorizedException();
 };
@@ -139,69 +209,126 @@
 A provider implementation must throw this exception if it can
 authenticate with its provider, but the provider denied the operation
 due to insufficient permission for an item (such as an attempt to
-write to a read-only file). Do not throw this exception for failure to
+write to a read-only file). Do <i>not</i> throw this exception for failure to
 authenticate with the provider.
 
 \see UnauthorizedException
 */
+
 class UNITY_STORAGE_EXPORT PermissionException : public StorageException
 {
 public:
+    /**
+    \brief Construct a PermissionException.
+    \param error_message The error message for the exception.
+    */
     PermissionException(std::string const& error_message);
     ~PermissionException();
 };
 
 /**
-\brief Indicates that an update failed because the provider ran out of space.
+\brief Indicates that an update failed because the provider ran out of space or exceeded
+the maximum number of files or folders.
 */
+
 class UNITY_STORAGE_EXPORT QuotaException : public StorageException
 {
 public:
+    /**
+    \brief Construct a QuotaException.
+    \param error_message The error message for the exception.
+    */
     QuotaException(std::string const& error_message);
     ~QuotaException();
 };
 
 /**
 \brief Indicates that an upload or download was cancelled before it could complete.
+\note Due to the way the provider API is structured, you will not ever need to throw this exception. It is provided for
+completeness and to allow for provider implementations that do not use the storage framework API.
 */
+
 class UNITY_STORAGE_EXPORT CancelledException : public StorageException
 {
 public:
+    /**
+    \brief Construct a CancelledException.
+    \param error_message The error message for the exception.
+    */
     CancelledException(std::string const& error_message);
     ~CancelledException();
 };
 
 /**
-\brief Indicates incorrect use of the API, such as calling methods in the wrong order.
+\brief Indicates incorrect use of the API.
+
+Use this exception for errors that the client could avoid, such as calling methods in the wrong order or attempting
+to list the contents of file (as opposed to a folder).
+\note Do <i>not</i> throw this exception to indicate arguments that are malformed or out of range.
+\see InvalidArgumentException
 */
 class UNITY_STORAGE_EXPORT LogicException : public StorageException
 {
 public:
+    /**
+    \brief Construct a LogicException.
+    \param error_message The error message for the exception.
+    */
     LogicException(std::string const& error_message);
     ~LogicException();
 };
 
 /**
-\brief Indicates an invalid parameter, such as a negative value when a positive one was
+\brief Indicates an invalid parameter.
+
+Use this exception for errors such as a negative parameter value when a positive one was
 expected, or a string that does not parse correctly or is empty when it should be non-empty.
+\note Do <i>not</i> throw this exception to indicate incorrect use of the API.
+\see LogicException
 */
+
 class UNITY_STORAGE_EXPORT InvalidArgumentException : public StorageException
 {
 public:
+    /**
+    \brief Construct an InvalidArgumentException.
+    \param error_message The error message for the exception.
+    */
     InvalidArgumentException(std::string const& error_message);
     ~InvalidArgumentException();
 };
 
 /**
-\brief Indicates a system error, such as failure to create a file or folder,
-or any other (usually non-recoverable) kind of error that should not arise during normal operation.
+\brief Indicates a system error.
+
+This is a generic "catch-all" exception that you can throw to indicate unexpected errors that do not fit into
+any of the other categories. For example, ResourceException is appropriate to indicate out of
+file descriptors, failure to locate a configuration file, or any other unexpected system error. You should
+provide as much contextual information about such errors as possible. In particular, unexpected errors
+typically need to be diagnosed from log files. This means that you should provide, at least, the full error
+message you received from the cloud service (or the operating system), together with all other relevant
+details, such as the name of the file, the URL of a failed request, any HTTP error information, and so on.
 */
+
 class UNITY_STORAGE_EXPORT ResourceException : public StorageException
 {
 public:
+    /**
+    \brief Construct an InvalidArgumentException.
+    \param error_message The error message for the exception.
+    \param error_code The system (or library) error code for the exception.
+    In case of system call failures, <code>error_code</code> should be the value of <code>errno</code>.
+    If the error code represents something else, such as a <code>QFileDevice::FileError</code>,
+    also indicate in the error message what that error code is; otherwise, it can be
+    impossible to diagnose a problem, especially when looking at log files after the fact.
+    */
     ResourceException(std::string const& error_message, int error_code);
     ~ResourceException();
 
+    /**
+    \brief Return the error code.
+    \return The value of the <code>error_code</code> parameter that was passed to the constructor.
+    */
     int error_code() const noexcept;
 
 private:
@@ -209,12 +336,25 @@
 };
 
 /**
-\brief Indicates that the server side caught an exception that does not derive from
-StorageException, such as a std::exception, or caught some other unknown type (such as `int`).
+\brief Indicates that a provider threw an exception not derived from StorageException.
+
+The runtime translates all exception that are thrown by a provider and that do not derive from StorageException
+to UnknownException.
+
+Do not throw this exception explicitly; is is a "catch-all" exception that the runtime uses to
+indicate that something completely unexpected happened. (If a client receives this exception from your
+provider, chances are that you have forgotten to handle an error condition somewhere.)
 */
+
 class UNITY_STORAGE_EXPORT UnknownException : public StorageException
 {
 public:
+    /**
+    \brief Construct an UnknownArgumentException.
+    \param error_message The error message for the exception. This is the string returned by <code>what()</code>
+    for <code>std::exception</code> or "Unknown exception" for exceptions that do not derive
+    from <code>std::exception</code>.
+    */
     UnknownException(std::string const& error_message);
     ~UnknownException();
 };

=== modified file 'include/unity/storage/provider/Item.h'
--- include/unity/storage/provider/Item.h	2017-03-17 02:51:05 +0000
+++ include/unity/storage/provider/Item.h	2017-04-06 07:42:38 +0000
@@ -33,17 +33,63 @@
 namespace provider
 {
 
+/**
+\brief The type of a metadata item (key&ndash;value pair).
+
+For boolean values, use <code>int64_t</code> with a zero or non-zero value to
+indicate <code>false</code> or <code>true</code>.
+
+Well-known metadata keys are defined in \ref common.h.
+*/
 // Note: When growing the set of supported variant types, add new types
 // to the *end* of the list, and update the marshaling code in dbusmarshal.cpp.
 typedef boost::variant<std::string, int64_t> MetadataValue;
 
+/**
+\brief Details about a storage item.
+*/
+
 struct UNITY_STORAGE_EXPORT Item
 {
+    /**
+    \brief The unique identity of the item.
+
+    See <a href="index.html#identity">File and Folder Identity</a> for detailed semantics.
+    */
     std::string item_id;
+
+    /**
+    \brief The list of parent folder identities.
+
+    \note Depending on the provider, it is possible for a file or folder to have more than one
+    parent folder.
+    */
     std::vector<std::string> parent_ids;
+
+    /**
+    \brief The name of a file or folder.
+
+    \note A provider may create a file or folder with a name other than the name that was requested by the
+    client (such as folding upper case to lower case). This field contains the name of the item as created
+    by the provider, not the name requested by the client.
+    */
     std::string name;
+
+    /**
+    \brief The ETag of a file.
+
+    \note For folders, the ETag is may be empty because not all providers support ETags for folders.
+    */
     std::string etag;
+
+    /**
+    \brief The item type (file, folder, or root,).
+    */
     unity::storage::ItemType type;
+
+    /**
+    \brief Additional metadata for a file or folder.
+    */
     std::map<std::string, MetadataValue> metadata;
 };
 

=== modified file 'include/unity/storage/provider/ProviderBase.h'
--- include/unity/storage/provider/ProviderBase.h	2017-03-16 02:50:03 +0000
+++ include/unity/storage/provider/ProviderBase.h	2017-04-06 07:42:38 +0000
@@ -42,15 +42,41 @@
 class DownloadJob;
 class UploadJob;
 
+/**
+\brief Security related information for an operation invocation.
+
+Each provider method has a trailing parameter of type Context that provides
+access to
+<a href="https://help.ubuntu.com/lts/serverguide/apparmor.html">Apparmor</a> details as
+well as credentials for the cloud provider.
+*/
+
 struct UNITY_STORAGE_EXPORT Context
 {
-    uid_t uid;
-    pid_t pid;
-    std::string security_label;
+    uid_t uid;                   /*!< The user ID of the client process. */
+    pid_t pid;                   /*!< The process ID of the client process. */
+    std::string security_label;  /*!< The Apparmor security label of the client process. */
 
-    Credentials credentials;
+    Credentials credentials;     /*!< Credentials to authenticate with the cloud provider. */
 };
 
+/**
+\brief Abstract base class for provider implementations.
+
+The runtime calls methods on this class in response to incoming requests from clients. Each method
+implements a specific operation on the storage backend.
+
+All methods are called from the main thread and must not block. Methods indicate error conditions to the runtime
+by throwing exceptions (which are caught and handled by the runtime). Alternatively, methods can return a future
+that stores an exception to indicate an error.
+
+\note Besides the explicitly listed exceptions for specific error conditions, all methods
+must indicate other errors by throwing an exception derived from StorageException, such
+as PermissionException or ResourceException (see <a href="index.html#provider-error-handling">Error Handling</a>).
+If an exception that does not derive from StorageException is thrown a provider method, the runtime returns a
+generic UnknownException to the client.
+*/
+
 class UNITY_STORAGE_EXPORT ProviderBase : public std::enable_shared_from_this<ProviderBase>
 {
 public:
@@ -60,41 +86,201 @@
     ProviderBase(ProviderBase const& other) = delete;
     ProviderBase& operator=(ProviderBase const& other) = delete;
 
-    virtual boost::future<ItemList> roots(std::vector<std::string> const& keys, Context const& context) = 0;
-    virtual boost::future<std::tuple<ItemList,std::string>> list(
-        std::string const& item_id, std::string const& page_token,
-        std::vector<std::string> const& keys,
-        Context const& context) = 0;
-    virtual boost::future<ItemList> lookup(
-        std::string const& parent_id, std::string const& name, std::vector<std::string> const& keys,
-        Context const& context) = 0;
-    virtual boost::future<Item> metadata(std::string const& item_id, std::vector<std::string> const& keys,
-        Context const& context) = 0;
-
-    virtual boost::future<Item> create_folder(
-        std::string const& parent_id, std::string const& name, std::vector<std::string> const& keys,
-        Context const& context) = 0;
-
-    virtual boost::future<std::unique_ptr<UploadJob>> create_file(
-        std::string const& parent_id, std::string const& name,
-        int64_t size, std::string const& content_type, bool allow_overwrite, std::vector<std::string> const& keys,
-        Context const& context) = 0;
-    virtual boost::future<std::unique_ptr<UploadJob>> update(
-        std::string const& item_id, int64_t size, std::string const& old_etag, std::vector<std::string> const& keys,
-        Context const& context) = 0;
-
-    virtual boost::future<std::unique_ptr<DownloadJob>> download(
-        std::string const& item_id, std::string const& match_etag,
-        Context const& context) = 0;
-
-    virtual boost::future<void> delete_item(
-        std::string const& item_id, Context const& context) = 0;
-    virtual boost::future<Item> move(
-        std::string const& item_id, std::string const& new_parent_id,
-        std::string const& new_name, std::vector<std::string> const& keys, Context const& context) = 0;
-    virtual boost::future<Item> copy(
-        std::string const& item_id, std::string const& new_parent_id,
-        std::string const& new_name, std::vector<std::string> const& keys, Context const& context) = 0;
+    /**
+    \brief Return the root folder (or folders) of the provider.
+    \param keys The keys of metadata items that the client wants to receive.
+    \param context The security context of the operation.
+    \return A (non-empty) list of roots.
+    */
+    virtual boost::future<ItemList> roots(std::vector<std::string> const& keys,
+                                          Context const& context) = 0;
+
+    /**
+    \brief Return a (non-recursive) list of the contents of a folder.
+    \param item_id The identity of the folder.
+    \param page_token A token identifying the next page of results (empty for the initial request).
+    \param keys The keys of metadata items that the client wants to receive.
+    \param context The security context of the operation.
+    \return A tuple containing a number of child items, plus a new page token. The page token
+    allows the method to return the results in multiple "pages". For the initial request, the runtime passes
+    an empty <code>page_token</code>. The method returns new page token to indicate whether there are more
+    results to be retrieved. If the returned token is non-empty, the runtime calls the method again,
+    passing the token that was returned by the previous call. To indicate that all results were retrieved, the method
+    must return an empty page token to the runtime.
+    \throws InvalidArgumentException <code>item_id</code> or <code>page_token</code> are invalid.
+    \throws NotExistsException <code>item_id</code> does not exist.
+    \throws LogicException If at all possible, the implementation should throw a LogicException
+    if <code>item_id</code> denotes a file. If a provider cannot distinguish between
+    attempts to list a file and other errors, it <i>must</i> return an empty list instead.
+    */
+    virtual boost::future<std::tuple<ItemList,std::string>> list(std::string const& item_id,
+                                                                 std::string const& page_token,
+                                                                 std::vector<std::string> const& keys,
+                                                                 Context const& context) = 0;
+
+    /**
+    \brief Retrieve a file or folder within a parent folder by name.
+    \param parent_id The identity of the parent folder.
+    \param name The name of the file or folder.
+    \param keys The keys of metadata items that the client wants to receive.
+    \param context The security context of the operation.
+    \return A non-empty list of items with the given name. The list may contain more than one entry
+    if the provider allows non-unique names for items within a folder.
+    The list may also contain more than one entry with the same identity if the provider allows
+    the same item to have more than one name.
+    \throws InvalidArgumentException <code>parent_id</code> or <code>name</code> are invalid.
+    \throws NotExistsException <code>parent_id</code> does not exist or does not contain a file or folder with
+    the given <code>name</code>. (Do <i>not</i> return an empty list in this case.)
+    */
+    virtual boost::future<ItemList> lookup(std::string const& parent_id,
+                                           std::string const& name,
+                                           std::vector<std::string> const& keys,
+                                           Context const& context) = 0;
+
+    /**
+    \brief Retrieve a file or folder by its identity.
+    \param item_id The identity of the item.
+    \param keys The keys of metadata items that the client wants to receive.
+    \param context The security context of the operation.
+    \return The item.
+    \throws InvalidArgumentException <code>item_id</code> is invalid.
+    \throws NotExistsException No file or folder with the given identity exists.
+    */
+    virtual boost::future<Item> metadata(std::string const& item_id,
+                                         std::vector<std::string> const& keys,
+                                         Context const& context) = 0;
+
+    /**
+    \brief Create a new folder.
+    \param parent_id The identity of the parent folder.
+    \param name The name of the new folder.
+    \param keys The keys of metadata items that the client wants to receive.
+    \param context The security context of the operation.
+    \return The item representing the folder.
+    \throws InvalidArgumentException <code>parent_id</code> or <code>name</code> are invalid.
+    \throws ExistsException An item with the given <code>name</code> exists already.
+    */
+    virtual boost::future<Item> create_folder(std::string const& parent_id,
+                                              std::string const& name,
+                                              std::vector<std::string> const& keys,
+                                              Context const& context) = 0;
+
+    /**
+    \brief Create a new file.
+    \param parent_id The identity of the parent folder.
+    \param name The name of the new file.
+    \param size The size of the file contents in bytes.
+    \param content_type The mime type of the file. If empty, the provider may (or may not) determine the mime type automatically.
+    \param allow_overwrite If true, the file will be created even if a file with the same name exists already.
+    \param keys The keys of metadata items that the client wants to receive.
+    \param context The security context of the operation.
+    \return An UploadJob that will read any data provided by the client and write it to the new file.
+    \throws InvalidArgumentException <code>parent_id</code> or <code>name</code> are invalid.
+    \throws ExistsException A file with the given <code>name</code> exists already and
+    <code>allow_overwrite</code> is <code>false</code>, or a folder with the given <code>name</code> exists already.
+    */
+    // TODO: The runtime should check that size is non-negative, so the provider implementation can rely on this.
+    virtual boost::future<std::unique_ptr<UploadJob>> create_file(std::string const& parent_id,
+                                                                  std::string const& name,
+                                                                  int64_t size,
+                                                                  std::string const& content_type,
+                                                                  bool allow_overwrite,
+                                                                  std::vector<std::string> const& keys,
+                                                                  Context const& context) = 0;
+
+    /**
+    \brief Update the contents of a file.
+    \param item_id The identity of the file.
+    \param size The size of the file contents in bytes.
+    \param old_etag The ETag of the existing file (empty if the file should be overwritten).
+    \param keys The keys of metadata items that the client wants to receive.
+    \param context The security context of the operation.
+    \return An UploadJob that will read any data provided by the client and write it to the file.
+    \throws InvalidArgumentException <code>item_id</code> or <code>size</code> are invalid.
+    \throws ConflictException The file's ETag does not match the given (non-empty) <code>old_etag</code>.
+    \throws LogicException The <code>item_id</code> denotes a folder.
+    */
+    // TODO: The runtime should check that size is non-negative, so the provider implementation can rely on this.
+    virtual boost::future<std::unique_ptr<UploadJob>> update(std::string const& item_id,
+                                                             int64_t size,
+                                                             std::string const& old_etag,
+                                                             std::vector<std::string> const& keys,
+                                                             Context const& context) = 0;
+
+    /**
+    \brief Download the contents of a file.
+    \param item_id The identity of the file.
+    \param match_etag The ETag of the existing file (empty if the file should be downloaded unconditionally).
+    \param context The security context of the operation.
+    \return A DownloadJob that will write the file data to the client socket.
+    \throws InvalidArgumentException <code>item_id</code> or <code>name</code> are invalid.
+    \throws NotExistsException <code>item_id</code> does not exist.
+    \throws LogicException The <code>item_id</code> denotes a folder.
+    \throws ConflictException The ETag for <code>item_id</code> does not match the given
+    (non-empty) <code>match_etag</code>.
+    */
+    virtual boost::future<std::unique_ptr<DownloadJob>> download(std::string const& item_id,
+                                                                 std::string const& match_etag,
+                                                                 Context const& context) = 0;
+
+    /**
+    \brief Delete an item.
+
+    Deletion is recursive. If <code>item_id</code> denotes a folder, the folder and all its contents are deleted.
+    \param item_id The identity of the file or folder.
+    \param context The security context of the operation.
+    \throws InvalidArgumentException <code>item_id</code> is invalid.
+    \throws PermissionException <code>item_id</code> denotes a root (or permission was denied for another reason).
+    */
+    virtual boost::future<void> delete_item(std::string const& item_id,
+                                            Context const& context) = 0;
+
+    /**
+    \brief Move and/or rename an item.
+
+    \param item_id The identity of the file or folder to be moved.
+    \param new_parent_id The identity of the folder to move the file or folder to. If <code>new_parent_id</code>
+    is the same as the existing parent of <code>item_id</code>, the file or folder is to be renamed within its
+    parent folder.
+    \param new_name The new name of the file (which can be the same as the old name if <code>new_parent_id</code>
+    differs).
+    \param keys The keys of metadata items that the client wants to receive.
+    \param context The security context of the operation.
+    \return The moved or renamed item.
+    \throws InvalidArgumentException <code>item_id</code>, <code>new_parent_id</code>, or <code>new_name</code>
+    are invalid.
+    \throws ExistsException The folder <code>new_parent_id</code> already contains an item with
+    name <code>new_name</code>.
+    \throws PermissionException <code>item_id</code> denotes a root (or permission was denied for another reason).
+    */
+    virtual boost::future<Item> move(std::string const& item_id,
+                                     std::string const& new_parent_id,
+                                     std::string const& new_name,
+                                     std::vector<std::string> const& keys,
+                                     Context const& context) = 0;
+
+    /**
+    \brief Copy an item.
+
+    Copy is recursive, so copying a folder copies the folder's contents.
+    \param item_id The identity of the file or folder to be copied.
+    \param new_parent_id The identity of the folder to copy the file or folder to. If <code>new_parent_id</code>
+    is the same as the existing parent of <code>item_id</code>, the file or folder is to be copied within its
+    parent folder.
+    \param new_name The new name of the file or folder.
+    \param keys The keys of metadata items that the client wants to receive.
+    \param context The security context of the operation.
+    \return The copied item.
+    \throws InvalidArgumentException <code>item_id</code>, <code>new_parent_id</code>, or <code>new_name</code>
+    are invalid.
+    \throws ExistsException The folder <code>new_parent_id</code> already contains an item with
+    name <code>new_name</code>.
+    */
+    virtual boost::future<Item> copy(std::string const& item_id,
+                                     std::string const& new_parent_id,
+                                     std::string const& new_name,
+                                     std::vector<std::string> const& keys,
+                                     Context const& context) = 0;
 };
 
 }

=== modified file 'include/unity/storage/provider/Server.h'
--- include/unity/storage/provider/Server.h	2017-03-16 03:30:18 +0000
+++ include/unity/storage/provider/Server.h	2017-04-06 07:42:38 +0000
@@ -37,29 +37,141 @@
 
 class ProviderBase;
 
+/**
+\brief Base class to register a storage provider with the runtime.
+
+\anchor extra-args This class allows you to register a provider that requires additional
+parameters with the storage framework runtime, by overriding the
+make_provider() method. (If your provider does not require constructor
+arguments, use the predefined Server template instead.)
+
+For example, here is how you can instantiate a provider that requires
+a <code>some_value</code> constructor argument:
+
+\code{.cpp}
+class MyCloudProvider : public ProviderBase
+{
+public:
+    MyCloudProvider(int some_value,
+                    string const& bus_name,
+                    string const& service_id)
+        : ProviderBase(bus_name, service_id)
+    {
+    }
+    // ...
+};
+
+class MyCloudServer : public ServerBase
+{
+public:
+    MyCloudServer(int some_value,
+                  string const& bus_name,
+                  string const& account_service_id)
+        : some_value_(some_value)
+        , bus_name_(bus_name)
+        , account_service_id_(account_service_id)
+    {
+    }
+
+protected:
+    shared_ptr<ProviderBase> make_provider() override
+    {
+        return make_shared<MyCloudProvider>(some_value, bus_name, account_service_id);
+    }
+
+private:
+    int some_value_;
+    string bus_name_;
+    string account_service_id_;
+};
+
+int main(int argc, char* argv[])
+{
+    string const bus_name = "com.acme.StorageFramework.Provider.MyCloud";
+    string const account_service_id = "storage-provider-mycloud";
+
+    try
+    {
+        MyCloudServer server(99, bus_name, account_service_id);
+        server.init(argc, argv);
+        return server.run();
+    }
+    catch (std::exception const& e)
+    {
+        cerr << argv[0] << ": " << e.what() << endl;
+        return 1;
+    }
+}
+\endcode
+*/
+
 class UNITY_STORAGE_EXPORT ServerBase
 {
 public:
+    /**
+    \brief Constructs a server instance.
+    \param bus_name The DBus name of the provider service on the session bus.
+    \param account_service_id The service ID with which the provider is known to
+    <a href="https://help.ubuntu.com/stable/ubuntu-help/accounts.html">Online Accounts</a>.
+    */
     ServerBase(std::string const& bus_name, std::string const& account_service_id);
     virtual ~ServerBase();
 
+    /**
+    \brief Initializes the storage framework runtime.
+    \param argc, argv The runtime passes these parameters through to
+    <a href="http://doc.qt.io/qt-5/qcoreapplication.html">QCoreApplication</a>.
+    \throws StorageException
+    */
     void init(int& argc, char** argv);
+
+    /**
+    \brief Starts an event loop for the service.
+
+    You <i>must</i> call init() before calling this method.
+    \return In case of an error, run() returns non-zero status.
+    */
     int run();
 
 protected:
+    /**
+    \brief Factory method to instantiate a provider.
+    The runtime calls this method to instantiate a provider as part
+    of the init() method. You can override this method if you need
+    to pass additional arguments to the constructor of your provider class.
+    (see \ref extra-args "example code").
+    \return A <code>shared_ptr</code> to the provider instance.
+    */
     virtual std::shared_ptr<ProviderBase> make_provider() = 0;
+
 private:
     std::unique_ptr<internal::ServerImpl> p_;
 
     friend class internal::ServerImpl;
 };
 
+/**
+\brief Default ServerBase implementation for providers with a default constructor.
+
+You can use this class to connect your provider class to the runtime, provided
+the class has a default constructor. If you need to pass additional arguments
+to the constructor, you must derive a class from ServerBase and override the
+ServerBase::make_provider() factory method (see \ref extra-args "example code").
+*/
+
 template <typename T>
 class Server : public ServerBase
 {
 public:
     using ServerBase::ServerBase;
+
 protected:
+    /**
+    \brief Factory method to instantiate a provider with a default constructor.
+
+    The runtime calls this method to create your provider instance of type T.
+    \return A <code>shared_ptr</code> to the provider instance.
+    */
     std::shared_ptr<ProviderBase> make_provider() override {
         return std::make_shared<T>();
     }

=== modified file 'include/unity/storage/provider/TempfileUploadJob.h'
--- include/unity/storage/provider/TempfileUploadJob.h	2016-08-24 10:52:17 +0000
+++ include/unity/storage/provider/TempfileUploadJob.h	2017-04-06 07:42:38 +0000
@@ -36,21 +36,59 @@
 class TempfileUploadJobImpl;
 }
 
+/**
+\brief Helper class to store the contents of an upload in a temporary file.
+
+For uploads, a provider implementation must decide whether to upload data to the cloud provider
+immediately (as soon as the data arrives from the client) or whether to buffer the data locally first
+and upload it to the cloud provider once the client successfully finalizes the upload. Uploading
+immediately has the down side that it is more likely to fail. This is an issue particularly
+for mobile devices, where applications may be suspended for extended periods, and where connectivity
+is often lost without warning. Uploads of large files (such as media files) are unlikely
+to ever complete in this case: if the user swipes away from the client application, no more data arrives
+at the provider until the application resumes, by which time the connection to the cloud provider has
+most likely timed out.
+
+Especially for files that are more than a few kilobytes in size, it is usually necessary to write the data
+to a local file first and to write it to the cloud provider only once all of data has been sent by the client.
+(The provider service does not get suspended and will not exit while an upload is in progress.)
+
+TempfileUploadJob is a helper class that implements an uploader that writes the data to a temporary file.
+The file is unlinked once the TempfileUploadJob is destroyed. You implementation must provide finish() and
+cancel().
+
+*/
+
 class UNITY_STORAGE_EXPORT TempfileUploadJob : public UploadJob
 {
 public:
+    /**
+    \brief Construct a TempfileUploadJob.
+    \param upload_id An identifier for this particular upload. You can use any non-empty string,
+    as long as it is unique among all uploads that are in progress within this provider.
+    The runtime uses the <code>upload_id</code> to distinguish different uploads that may be
+    in progress concurrently.
+    */
     TempfileUploadJob(std::string const& upload_id);
     virtual ~TempfileUploadJob();
 
+    /**
+    \brief Returns the name of the file.
+    \return The full path name of the temporary file.
+    */
     std::string file_name() const;
 
-    // This function should be called from your finish()
-    // implementation to read the remaining data from the socket.  If
-    // the client has not closed the socket as expected, LogicError
-    // will be thrown.
+    /**
+    \brief Reads any unread data from the upload socket and writes it to the temporary file.
+
+    You must call drain() from your UploadJob::finish() method. Its implementation reads any remaining
+    data from the upload socket and writes it to the temporary file. If an error occurs,
+    drain() throws an exception.
+    \throws LogicException The client did not close its end of the socket.
+    */
     void drain();
 
-protected:
+private:
     TempfileUploadJob(internal::TempfileUploadJobImpl *p) UNITY_STORAGE_HIDDEN;
 };
 

=== modified file 'include/unity/storage/provider/UploadJob.h'
--- include/unity/storage/provider/UploadJob.h	2016-08-24 10:52:17 +0000
+++ include/unity/storage/provider/UploadJob.h	2017-04-06 07:42:38 +0000
@@ -39,28 +39,126 @@
 class UploadJobImpl;
 }
 
+class TempfileUploadJob;
+
+/**
+\brief Abstract base class for upload implementations.
+
+When the runtime calls ProviderBase::update() or ProviderBase::create_file(), you must return
+a new uploader instance that derives from UploadJob. Your implementation is responsible
+for retrieving the file's data from the upload socket and writing it to
+the corresponding file in the cloud provider.
+
+You can implement your uploader
+any way you wish, such as by running the upload in a separate thread, or
+by using async I/O driven by the runtime's (or any other) event loop.
+
+The runtime invokes all methods on the uploader from the main thread.
+*/
+
 class UNITY_STORAGE_EXPORT UploadJob
 {
 public:
+    /**
+    \brief Construct an uploader.
+    \param upload_id An identifier for this particular upload. You can use any non-empty string,
+    as long as it is unique among all uploads that are in progress within the corresponding account.
+    (A simple incrementing counter will work fine, or you can use the upload identifier you receive
+    from the cloud service.)
+    The runtime uses the <code>upload_id</code> to distinguish different uploads that may be
+    in progress concurrently, and it ensures that each ID can be used only by its corresponding client.
+    */
     UploadJob(std::string const& upload_id);
     virtual ~UploadJob();
 
+    /**
+    \brief Returns the upload ID.
+    \return The value of the <code>upload_id</code> parameter that was passed to the constructor.
+    */
     std::string const& upload_id() const;
+
+    /**
+    \brief Returns the socket to read the file contents from.
+    \return A socket that is open for reading. You must read the file contents from this socket (which is
+    connected to the client by the runtime).
+    */
     int read_socket() const;
 
-    // If an error is reported early, cancel() or finish() will not be
-    // invoked.
-    void report_error(std::exception_ptr p);
-
+    /**
+    \brief Informs the runtime that an upload encountered an error.
+
+    This method makes it unnecessary to wait for the runtime to call finish() in order to confirm
+    whether an upload completed successfully. You can call report_error() as soon as you encounter an error
+    during an upload (such as losing the connection to the cloud provider, or getting an error when reading
+    from the upload socket). This can be convenient if an upload runs in a separate thread or event loop
+    because there is no need to store success/error state for use in finish().
+
+    You can call report_error() from an arbitrary thread.
+
+    If you call report_error(), the runtime guarantees that neither finish() nor cancel() will be called, so
+    you must reclaim any resources associated with the upload before calling report_error().
+
+    \param storage_exception You <i>must</i> pass a StorageException to indicate the reason for the failure.
+    \see finish()
+    */
+    void report_error(std::exception_ptr storage_exception);
+
+    /**
+    \brief Cancel this upload.
+
+    The runtime calls this method when a client explicitly cancels an upload or crashes.
+    Your implementation should reclaim all resources that are used by the upload. In particular,
+    you should stop reading any more data and close the upload socket. In addition,
+    you should reclaim any resources (such as open connections) that are associated
+    with the upload to the cloud provider (possibly after informing the cloud provider of
+    the cancellation).
+
+    The runtime guarantees that cancel() will be called only once, and that finish() will not be called
+    if cancel() was called.
+
+    If any errors are encountered, you <i>must</i> report them by returning a future that stores
+    a StorageException. Do <i>not</i> call report_error() from inside cancel().
+
+    \return A future that becomes ready (or contains a StorageException) once cancellation is complete.
+    */
     virtual boost::future<void> cancel() = 0;
+
+    /**
+    \brief Finalize this upload.
+
+    The runtime calls this method when a client finishes an upload. Your implementation <i>must</i> verify
+    that it has successfully read <i>and</i> written the <i>exact</i> number of bytes that were passed
+    to ProviderBase::update() or ProviderBase::create_file() before making the future ready.
+    If too few or too many bytes were sent by the client, it <i>must</i> store a LogicException in the future.
+    It must also verify that the cloud provider has received <i>all</i> of the file's data; otherwise,
+    the file may end up being partially written in the cloud provider.
+
+    Note that finish() may be called while there is still buffered data that remains to be read from the
+    upload socket, so you must take care to drain the socket of any remaining data before checking that the
+    actual number bytes uploaded by the client matches the expected number of bytes.
+
+    You should close the upload socket and reclaim any resources (such as open
+    connections) that are associated with the upload from the cloud provider.
+
+    The runtime guarantees that finish() will be called only once, and that cancel() will not be called
+    if finish() was called.
+
+    If any errors are encountered, you <i>must</i> report them by returning a future that stores
+    a StorageException. Do <i>not</i> call report_error() from inside finish().
+
+    \return A future that becomes ready and contains the metadata for the file (or contains a StorageException)
+    once the upload is complete.
+    \see report_error(), TempfileUploadJob
+    */
     virtual boost::future<Item> finish() = 0;
 
-protected:
+private:
     UploadJob(internal::UploadJobImpl *p) UNITY_STORAGE_HIDDEN;
     internal::UploadJobImpl *p_ = nullptr;
 
     friend class internal::PendingJobs;
     friend class internal::ProviderInterface;
+    friend class TempfileUploadJob;
 };
 
 }

=== modified file 'include/unity/storage/provider/testing/TestServer.h'
--- include/unity/storage/provider/testing/TestServer.h	2016-09-28 07:04:41 +0000
+++ include/unity/storage/provider/testing/TestServer.h	2017-04-06 07:42:38 +0000
@@ -23,11 +23,13 @@
 #include <memory>
 #include <string>
 
+//@cond
 namespace OnlineAccounts
 {
 class Account;
 }
 class QDBusConnection;
+//@endcond
 
 namespace unity
 {
@@ -46,16 +48,44 @@
 namespace testing
 {
 
+/**
+\brief Helper class to enable testing of provider implementations.
+
+
+TestServer is a simple helper class that allows you to test a provider implementation
+on a separate DBus connection. The class requires access to
+an <a href="https://help.ubuntu.com/stable/ubuntu-help/accounts.html">Online Accounts</a>
+service. If you do not want to test with the live Online Accounts service, you can pass
+<code>nullptr</code>, in which case your provider receives blank
+\link unity::storage::provider::Credentials Credentials\endlink.
+*/
+
 class UNITY_STORAGE_EXPORT TestServer
 {
 public:
+    /**
+    \brief Constructs a TestServer instance.
+    \param provider The provider implementation to be tested.
+    \param account The account for the provider (or <code>nullptr</code>).
+    \param connection The DBus connection to connect the provider to.
+    \param object_path The DBus object path for the provider interface.
+    */
     TestServer(std::shared_ptr<ProviderBase> const& provider,
                OnlineAccounts::Account* account,
                QDBusConnection const& connection,
                std::string const& object_path);
     ~TestServer();
 
+    /**
+    \brief Returns the DBus connection.
+    \return The value of the <code>connection</code> parameter that was passed to the constructor.
+    */
     QDBusConnection const& connection() const;
+
+    /**
+    \brief Returns the object path.
+    \return The value of the <code>object_path</code> parameter that was passed to the constructor.
+    */
     std::string const& object_path() const;
 
 private:

=== modified file 'include/unity/storage/qt/internal/StorageErrorImpl.h'
--- include/unity/storage/qt/internal/StorageErrorImpl.h	2016-10-13 06:48:11 +0000
+++ include/unity/storage/qt/internal/StorageErrorImpl.h	2017-04-06 07:42:38 +0000
@@ -63,6 +63,7 @@
     static StorageError not_exists_error(QString const& msg, QString const& key);
     static StorageError exists_error(QString const& msg, QString const& item_id, QString const& item_name);
     static StorageError cancelled_error(QString const& msg);
+    static StorageError permission_error(QString const& msg);
     static StorageError logic_error(QString const& msg);
     static StorageError invalid_argument_error(QString const& msg);
     static StorageError resource_error(QString const& msg, int error_code);

=== modified file 'src/local-provider/LocalDownloadJob.cpp'
--- src/local-provider/LocalDownloadJob.cpp	2017-03-14 07:11:40 +0000
+++ src/local-provider/LocalDownloadJob.cpp	2017-04-06 07:42:38 +0000
@@ -46,7 +46,7 @@
         auto st = status(item_id_);
         if (!is_regular_file(st))
         {
-            throw InvalidArgumentException(method + ": \"" + item_id_ + "\" is not a file");
+            throw LogicException(method + ": \"" + item_id_ + "\" is not a file");
         }
     }
     // LCOV_EXCL_START  // Too small a window to hit with a test.
@@ -60,7 +60,7 @@
         int64_t mtime = get_mtime_nsecs(method, item_id_);
         if (to_string(mtime) != match_etag)
         {
-            throw ConflictException(method + ": etag mismatch");
+            throw ConflictException(method + ": ETag mismatch");
         }
     }
 

=== modified file 'src/local-provider/LocalProvider.cpp'
--- src/local-provider/LocalProvider.cpp	2017-03-23 06:55:00 +0000
+++ src/local-provider/LocalProvider.cpp	2017-04-06 07:42:38 +0000
@@ -195,6 +195,11 @@
         using namespace boost::filesystem;
 
         This->throw_if_not_valid(method, item_id);
+        if (!is_directory(item_id))
+        {
+            string msg = method + ": \"" + item_id + "\" is not a folder";
+            throw boost::enable_current_exception(LogicException(msg));
+        }
         vector<Item> items;
         for (directory_iterator it(item_id); it != directory_iterator(); ++it)
         {
@@ -345,7 +350,7 @@
         if (canonical(item_id).native() == This->root_)
         {
             string msg = method + ": cannot delete root";
-            throw boost::enable_current_exception(LogicException(msg));
+            throw boost::enable_current_exception(PermissionException(msg));
         }
         remove_all(item_id);
     };
@@ -378,6 +383,11 @@
             string msg = method + ": \"" + target_path.native() + "\" exists already";
             throw boost::enable_current_exception(ExistsException(msg, target_path.native(), new_name));
         }
+        if (canonical(item_id).native() == This->root_)
+        {
+            string msg = method + ": cannot move root";
+            throw boost::enable_current_exception(PermissionException(msg));
+        }
 
         // Small race condition here: if exists() just said that the target does not exist, it is
         // possible for it to have been created since. If so, if the target is a file or an empty

=== modified file 'src/local-provider/LocalUploadJob.cpp'
--- src/local-provider/LocalUploadJob.cpp	2017-03-16 02:50:03 +0000
+++ src/local-provider/LocalUploadJob.cpp	2017-04-06 07:42:38 +0000
@@ -83,7 +83,7 @@
         auto st = status(item_id);
         if (!is_regular_file(st))
         {
-            throw InvalidArgumentException(method_ + ": \"" + item_id + "\" is not a file");
+            throw LogicException(method_ + ": \"" + item_id + "\" is not a file");
         }
     }
     // LCOV_EXCL_START
@@ -99,7 +99,7 @@
         int64_t mtime = get_mtime_nsecs(method_, item_id);
         if (to_string(mtime) != old_etag)
         {
-            throw ConflictException(method_ + ": etag mismatch");
+            throw ConflictException(method_ + ": ETag mismatch");
         }
     }
     old_etag_ = old_etag;
@@ -184,14 +184,13 @@
 
     try
     {
-        // We check again for an etag mismatch or overwrite, in case the file was updated after the upload started.
+        // We check again for an ETag mismatch or overwrite, in case the file was updated after the upload started.
         if (!parent_id_.empty())
         {
             // create_file()
             if (!allow_overwrite_ && boost::filesystem::exists(item_id_))
             {
                 string msg = method_ + ": \"" + item_id_ + "\" exists already";
-                boost::filesystem::path(item_id_).filename().native();
                 BOOST_THROW_EXCEPTION(
                     ExistsException(msg, item_id_, boost::filesystem::path(item_id_).filename().native()));
             }
@@ -199,10 +198,22 @@
         else if (!old_etag_.empty())
         {
             // update()
+            using namespace boost::filesystem;
+            if (exists(item_id_) && !is_regular_file(item_id_))
+            {
+                // Yes, ExistsException, not LogicException. A folder is in the way
+                // and was created after the update started, meaning that the client correctly
+                // started the operation with an existing file, but then the file was removed
+                // and replaced with a folder with the same name.
+                string msg = method_ + ": \"" + item_id_ + "\" exists already";
+                BOOST_THROW_EXCEPTION(
+                    ExistsException(msg, item_id_, boost::filesystem::path(item_id_).filename().native()));
+            }
             int64_t mtime = get_mtime_nsecs(method_, item_id_);
             if (to_string(mtime) != old_etag_)
             {
-                BOOST_THROW_EXCEPTION(ConflictException(method_ + ": etag mismatch"));
+                // File was touched after the upload started.
+                BOOST_THROW_EXCEPTION(ConflictException(method_ + ": ETag mismatch"));
             }
         }
 

=== modified file 'src/qt/internal/ItemImpl.cpp'
--- src/qt/internal/ItemImpl.cpp	2016-11-25 03:23:55 +0000
+++ src/qt/internal/ItemImpl.cpp	2017-04-06 07:42:38 +0000
@@ -235,7 +235,7 @@
     }
     if (md_.type == storage::ItemType::root)
     {
-        auto e = StorageErrorImpl::logic_error(method + ": cannot delete root");
+        auto e = StorageErrorImpl::permission_error(method + ": cannot delete root");
         return VoidJobImpl::make_job(e);
     }
 

=== modified file 'src/qt/internal/StorageErrorImpl.cpp'
--- src/qt/internal/StorageErrorImpl.cpp	2017-03-14 07:11:40 +0000
+++ src/qt/internal/StorageErrorImpl.cpp	2017-04-06 07:42:38 +0000
@@ -188,6 +188,12 @@
     return StorageError(move(p));
 }
 
+StorageError StorageErrorImpl::permission_error(QString const& msg)
+{
+    unique_ptr<StorageErrorImpl> p(new StorageErrorImpl(StorageError::Type::PermissionDenied, msg));
+    return StorageError(move(p));
+}
+
 StorageError StorageErrorImpl::logic_error(QString const& msg)
 {
     unique_ptr<StorageErrorImpl> p(new StorageErrorImpl(StorageError::Type::LogicError, msg));

=== modified file 'src/qt/internal/unmarshal_error.cpp'
--- src/qt/internal/unmarshal_error.cpp	2017-01-24 06:12:09 +0000
+++ src/qt/internal/unmarshal_error.cpp	2017-04-06 07:42:38 +0000
@@ -84,11 +84,11 @@
     { "ConflictException",        make_error<StorageError::Type::Conflict> },
     { "UnauthorizedException",    make_error<StorageError::Type::Unauthorized> },
     { "PermissionException",      make_error<StorageError::Type::PermissionDenied> },
+    { "QuotaException",           make_error<StorageError::Type::QuotaExceeded> },
     { "CancelledException",       make_error<StorageError::Type::Cancelled> },
     { "LogicException",           make_error<StorageError::Type::LogicError> },
     { "InvalidArgumentException", make_error<StorageError::Type::InvalidArgument> },
     { "ResourceException",        make_error<StorageError::Type::ResourceError> },
-    { "QuotaException",           make_error<StorageError::Type::QuotaExceeded> },
     { "UnknownException",         make_error<StorageError::Type::LocalCommsError> }  // Yes, LocalCommsError is intentional
 };
 

=== modified file 'tests/local-provider/local-provider_test.cpp'
--- tests/local-provider/local-provider_test.cpp	2017-03-23 06:55:00 +0000
+++ tests/local-provider/local-provider_test.cpp	2017-04-06 07:42:38 +0000
@@ -141,6 +141,22 @@
     "nulla pariatur. Excepteur sint occaecat cupidatat non proident, "
     "sunt in culpa qui officia deserunt mollit anim id est laborum.\n";
 
+void make_hierarchy(string const& root_dir)
+{
+    // Make a small tree so we have something to test against.
+    ASSERT_EQ(0, mkdir((root_dir + "/a").c_str(), 0755));
+    ASSERT_EQ(0, mkdir((root_dir + "/a/b").c_str(), 0755));
+    string cmd = string("echo hello >") + root_dir + "/hello";
+    ASSERT_EQ(0, system(cmd.c_str()));
+    cmd = string("echo text >") + root_dir + "/a/foo.txt";
+    ASSERT_EQ(0, system(cmd.c_str()));
+    ASSERT_EQ(0, mknod((root_dir + "/a/pipe").c_str(), S_IFIFO | 06666, 0));
+    ASSERT_EQ(0, mkdir((root_dir + "/a/.storage-framework-").c_str(), 0755));
+    ASSERT_EQ(0, mkdir((root_dir + "/a/b/.storage-framework-").c_str(), 0755));
+    ASSERT_EQ(0, mknod((root_dir + "/a/b/pipe").c_str(), S_IFIFO | 06666, 0));
+    ASSERT_EQ(0, mkdir((root_dir + "/empty_dir").c_str(), 0755));
+}
+
 }  // namespace
 
 TEST(Directories, env_vars)
@@ -307,6 +323,9 @@
         ASSERT_EQ(ItemJob::Error, job->status()) << job->error().errorString().toStdString();
         EXPECT_EQ(string("Exists: create_folder(): \"") + ROOT_DIR() + "/child\" exists already",
                   job->error().errorString().toStdString());
+        EXPECT_EQ(ROOT_DIR() + "/child", job->error().itemId().toStdString());
+        EXPECT_EQ(ROOT_DIR() + "/child", job->error().itemId().toStdString());
+        EXPECT_EQ("child", job->error().itemName().toStdString());
 
         // Again, without write permission on the root dir, to get coverage for a filesystem_error in invoke_async().
         ASSERT_EQ(0, ::rmdir((ROOT_DIR() + "/child").c_str()));
@@ -363,15 +382,17 @@
     // Client-side API does not allow us to try to delete the root, so we talk to the provider directly.
     auto p = make_shared<LocalProvider>();
 
+    make_hierarchy(ROOT_DIR());
+
     auto fut = p->delete_item(ROOT_DIR(), provider::Context());
     try
     {
         fut.get();
         FAIL();
     }
-    catch (provider::LogicException const& e)
+    catch (provider::PermissionException const& e)
     {
-        EXPECT_STREQ("LogicException: delete_item(): cannot delete root", e.what());
+        EXPECT_STREQ("PermissionException: delete_item(): cannot delete root", e.what());
     }
 }
 
@@ -435,6 +456,8 @@
     EXPECT_EQ(string("NotExists: lookup(): \"") + ROOT_DIR() + "/child\": boost::filesystem::canonical: "
               + "No such file or directory: \"" + ROOT_DIR() + "/child\"",
               job->error().errorString().toStdString());
+    EXPECT_EQ(qt::StorageError::NotExists, job->error().type());
+    EXPECT_EQ(ROOT_DIR() + "/child", job->error().itemId().toStdString());
 }
 
 TEST_F(LocalProviderTest, list)
@@ -472,19 +495,24 @@
     EXPECT_EQ(5, child.metadata().size());
 }
 
-void make_hierarchy(string const& root_dir)
+TEST_F(LocalProviderTest, list_file)
 {
-    // Make a small tree so we have something to test with for move() and copy().
-    ASSERT_EQ(0, mkdir((root_dir + "/a").c_str(), 0755));
-    ASSERT_EQ(0, mkdir((root_dir + "/a/b").c_str(), 0755));
-    string cmd = string("echo hello >") + root_dir + "/hello";
-    ASSERT_EQ(0, system(cmd.c_str()));
-    cmd = string("echo text >") + root_dir + "/a/foo.txt";
-    ASSERT_EQ(0, system(cmd.c_str()));
-    ASSERT_EQ(0, mknod((root_dir + "/a/pipe").c_str(), S_IFIFO | 06666, 0));
-    ASSERT_EQ(0, mkdir((root_dir + "/a/.storage-framework-").c_str(), 0755));
-    ASSERT_EQ(0, mkdir((root_dir + "/a/b/.storage-framework-").c_str(), 0755));
-    ASSERT_EQ(0, mknod((root_dir + "/a/b/pipe").c_str(), S_IFIFO | 06666, 0));
+    // We can't try a list on a file via the client API, so we use the provider directly.
+
+    auto p = make_shared<LocalProvider>();
+
+    make_hierarchy(ROOT_DIR());
+
+    try
+    {
+        auto fut = p->list(ROOT_DIR() + "/hello", "", {}, provider::Context());
+        fut.get();
+        FAIL();
+    }
+    catch (provider::LogicException const& e)
+    {
+        EXPECT_EQ(string("LogicException: list(): \"") + ROOT_DIR() + "/hello\" is not a folder", e.what());
+    }
 }
 
 TEST_F(LocalProviderTest, move)
@@ -571,6 +599,26 @@
               job->error().errorString().toStdString());
 }
 
+TEST_F(LocalProviderTest, move_root)
+{
+    // We can't try to move a root via the client API, so we use the LocalDownloadJob directly.
+
+    auto p = make_shared<LocalProvider>();
+
+    make_hierarchy(ROOT_DIR());
+
+    auto fut = p->move(ROOT_DIR(), ROOT_DIR() + "/empty_dir", "moved_root", {}, provider::Context());
+    try
+    {
+        fut.get();
+        FAIL();
+    }
+    catch (provider::PermissionException const& e)
+    {
+        EXPECT_STREQ("PermissionException: move(): cannot move root", e.what());
+    }
+}
+
 TEST_F(LocalProviderTest, copy_file)
 {
     using namespace unity::storage::qt;
@@ -659,6 +707,44 @@
     }
 }
 
+TEST_F(LocalProviderTest, copy_root)
+{
+    using namespace unity::storage::qt;
+    using namespace boost::filesystem;
+
+    set_provider(unique_ptr<provider::ProviderBase>(new LocalProvider));
+
+    make_hierarchy(ROOT_DIR());
+
+    auto root = get_root(acc_);
+
+    qt::Item empty_dir;
+    {
+        unique_ptr<ItemListJob> job(root.lookup("empty_dir"));
+        auto items = get_items(job.get());
+        ASSERT_EQ(ItemListJob::Finished, job->status()) << job->error().errorString().toStdString();
+        ASSERT_EQ(1, items.size());
+        empty_dir = items.at(0);
+    }
+
+    {
+        unique_ptr<ItemJob> job(root.copy(empty_dir, "copied_root"));
+        wait(job.get());
+        ASSERT_EQ(ItemJob::Finished, job->status()) << job->error().errorString().toStdString();
+    }
+
+    // Check that we only copied regular files and directories, but not a pipe or anything starting with
+    // the temp file prefix.
+    EXPECT_TRUE(exists(ROOT_DIR() + "/empty_dir/copied_root/a/b"));
+    EXPECT_TRUE(exists(ROOT_DIR() + "/empty_dir/copied_root/hello"));
+    EXPECT_TRUE(exists(ROOT_DIR() + "/empty_dir/copied_root/a/foo.txt"));
+    EXPECT_TRUE(exists(ROOT_DIR() + "/empty_dir/copied_root/empty_dir"));
+    EXPECT_FALSE(exists(ROOT_DIR() + "/empty_dir/copied_root/a/pipe"));
+    EXPECT_FALSE(exists(ROOT_DIR() + "/empty_dir/copied_root/a/storage-framework-"));
+    EXPECT_FALSE(exists(ROOT_DIR() + "/empty_dir/copied_root/a/b/pipe"));
+    EXPECT_FALSE(exists(ROOT_DIR() + "/empty_dir/copied_root/a/b/storage-framework-"));
+}
+
 TEST_F(LocalProviderTest, download)
 {
     using namespace unity::storage::qt;
@@ -785,7 +871,7 @@
 
     auto error = downloader->error();
     EXPECT_EQ(qt::StorageError::Conflict, error.type());
-    EXPECT_EQ("download(): etag mismatch", error.message().toStdString());
+    EXPECT_EQ("download(): ETag mismatch", error.message().toStdString());
 }
 
 TEST_F(LocalProviderTest, download_wrong_file_type)
@@ -802,9 +888,9 @@
         LocalDownloadJob(p, dir, "some_etag");
         FAIL();
     }
-    catch (provider::InvalidArgumentException const& e)
+    catch (provider::LogicException const& e)
     {
-        EXPECT_EQ(string("InvalidArgumentException: download(): \"" + dir + "\" is not a file"), e.what());
+        EXPECT_EQ(string("LogicException: download(): \"" + dir + "\" is not a file"), e.what());
     }
 }
 
@@ -1013,7 +1099,7 @@
     {
         ASSERT_TRUE(spy.wait(SIGNAL_WAIT_TIME));
     }
-    EXPECT_EQ("update(): etag mismatch", uploader->error().message().toStdString());
+    EXPECT_EQ("update(): ETag mismatch", uploader->error().message().toStdString());
 }
 
 TEST_F(LocalProviderTest, update_file_touched_while_uploading)
@@ -1062,7 +1148,56 @@
         ASSERT_TRUE(spy.wait(SIGNAL_WAIT_TIME));
     }
     ASSERT_EQ(Uploader::Error, uploader->status());
-    EXPECT_EQ("update(): etag mismatch", uploader->error().message().toStdString());
+    EXPECT_EQ("update(): ETag mismatch", uploader->error().message().toStdString());
+}
+
+TEST_F(LocalProviderTest, update_folder_created_while_uploading)
+{
+    using namespace unity::storage::qt;
+
+    set_provider(unique_ptr<provider::ProviderBase>(new LocalProvider));
+
+    auto full_path = ROOT_DIR() + "/foo.txt";
+    auto cmd = string("echo hello >") + full_path;
+    ASSERT_EQ(0, system(cmd.c_str()));
+
+    unique_ptr<ItemJob> job(acc_.get(QString::fromStdString(full_path)));
+    wait(job.get());
+    EXPECT_TRUE(job->isValid());
+
+    auto file = job->item();
+    auto old_etag = file.etag();
+
+    int const segments = 50;
+    unique_ptr<Uploader> uploader(file.createUploader(Item::ErrorIfConflict, file_contents.size() * segments));
+
+    int count = 0;
+    QTimer timer;
+    timer.setSingleShot(false);
+    timer.setInterval(10);
+    QObject::connect(&timer, &QTimer::timeout, [&] {
+            uploader->write(&file_contents[0], file_contents.size());
+            count++;
+            if (count == segments / 2)
+            {
+                sleep(1);
+                cmd = string("rm ") + full_path + "; mkdir " + full_path;
+                ASSERT_EQ(0, system(cmd.c_str()));
+            }
+            else if (count == segments)
+            {
+                uploader->close();
+            }
+        });
+
+    QSignalSpy spy(uploader.get(), &Uploader::statusChanged);
+    timer.start();
+    while (uploader->status() != Uploader::Error)
+    {
+        ASSERT_TRUE(spy.wait(SIGNAL_WAIT_TIME));
+    }
+    ASSERT_EQ(Uploader::Error, uploader->status());
+    EXPECT_EQ(string("update(): \"") + full_path + "\" exists already", uploader->error().message().toStdString());
 }
 
 TEST_F(LocalProviderTest, update_ignore_etag_mismatch)
@@ -1208,9 +1343,9 @@
         LocalUploadJob(p, dir, 0, "");
         FAIL();
     }
-    catch (provider::InvalidArgumentException const& e)
+    catch (provider::LogicException const& e)
     {
-        EXPECT_EQ(string("InvalidArgumentException: update(): \"" + dir + "\" is not a file"), e.what());
+        EXPECT_EQ(string("LogicException: update(): \"" + dir + "\" is not a file"), e.what());
     }
 }
 

=== modified file 'tests/remote-client/remote-client_test.cpp'
--- tests/remote-client/remote-client_test.cpp	2017-01-12 06:02:30 +0000
+++ tests/remote-client/remote-client_test.cpp	2017-04-06 07:42:38 +0000
@@ -946,7 +946,7 @@
     unique_ptr<VoidJob> j(item.deleteItem());
     EXPECT_FALSE(j->isValid());
     EXPECT_EQ(VoidJob::Status::Error, j->status());
-    EXPECT_EQ(StorageError::Type::LogicError, j->error().type());
+    EXPECT_EQ(StorageError::Type::PermissionDenied, j->error().type());
 
     // Signal must be received.
     QSignalSpy spy(j.get(), &VoidJob::statusChanged);