Merge lp:~thomas-voss/net-cpp/add-streaming into lp:net-cpp

[Thomas Voß]

Add streaming support to net-cpp in an ABI-stable way.

Add streaming support to net-cpp in an ABI-stable way.

[PS Jenkins bot (ps-jenkins)]

PASSED: Continuous integration, rev:49
http://jenkins.qa.ubuntu.com/job/net-cpp-ci/6/
Executed test runs:
    SUCCESS: http://jenkins.qa.ubuntu.com/job/net-cpp-vivid-amd64-ci/6
    SUCCESS: http://jenkins.qa.ubuntu.com/job/net-cpp-vivid-armhf-ci/6
        deb: http://jenkins.qa.ubuntu.com/job/net-cpp-vivid-armhf-ci/6/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/net-cpp-vivid-i386-ci/6

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/net-cpp-ci/6/rebuild

review: Approve (continuous-integration)

[Marcus Tomlinson (marcustomlinson)]

361 +class Client : public core::net::http::StreamingClient

Just noticing a few overriding methods in this class that don't specify "override" in their declarations. i.e. url_escape, timings, run, stop, get, head, post, and pull.

review: Needs Fixing

[PS Jenkins bot (ps-jenkins)]

PASSED: Continuous integration, rev:50
http://jenkins.qa.ubuntu.com/job/net-cpp-ci/7/
Executed test runs:
    SUCCESS: http://jenkins.qa.ubuntu.com/job/net-cpp-vivid-amd64-ci/7
    SUCCESS: http://jenkins.qa.ubuntu.com/job/net-cpp-vivid-armhf-ci/7
        deb: http://jenkins.qa.ubuntu.com/job/net-cpp-vivid-armhf-ci/7/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/net-cpp-vivid-i386-ci/7

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/net-cpp-ci/7/rebuild

review: Approve (continuous-integration)

[Marcus Tomlinson (marcustomlinson)]

Client::url_escape should also be marked override, but no biggy. Looks good.

review: Approve

=== modified file 'CMakeLists.txt'
--- CMakeLists.txt	2014-07-30 15:16:19 +0000
+++ CMakeLists.txt	2015-04-01 06:23:51 +0000
@@ -27,7 +27,7 @@
 set(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} -Wl,--no-undefined")
 
 set(NET_CPP_VERSION_MAJOR 1)
-set(NET_CPP_VERSION_MINOR 1)
+set(NET_CPP_VERSION_MINOR 2)
 set(NET_CPP_VERSION_PATCH 0)
 
 include(CTest)
@@ -36,6 +36,8 @@
   include/
 )
 
+file(GLOB_RECURSE NET_CPP_INTERFACE_HEADERS include/*.h)
+
 add_subdirectory(doc)
 add_subdirectory(data)
 add_subdirectory(include)

=== modified file 'debian/changelog'
--- debian/changelog	2015-03-05 12:08:09 +0000
+++ debian/changelog	2015-04-01 06:23:51 +0000
@@ -1,3 +1,9 @@
+net-cpp (1.2.0) UNRELEASED; urgency=medium
+
+  * Introduce a streaming http interface.
+
+ -- Thomas Voß <[e-mail]>  Wed, 01 Apr 2015 08:22:32 +0200
+
 net-cpp (1.1.0+15.04.20150305-0ubuntu1) vivid; urgency=medium
 
   [ thomas-voss ]

=== modified file 'debian/libnet-cpp1.symbols'
--- debian/libnet-cpp1.symbols	2014-12-04 20:54:26 +0000
+++ debian/libnet-cpp1.symbols	2015-04-01 06:23:51 +0000
@@ -1,5 +1,6 @@
 libnet-cpp.so.1 libnet-cpp1 #MINVER#
  (c++)"core::net::http::make_client()@Base" 0.0.1+14.10.20140611
+ (c++)"core::net::http::make_streaming_client()@Base" 1.1.0+15.04.20150305-0ubuntu1
  (c++)"core::net::make_uri(std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, std::vector<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::basic_string<char, std::char_traits<char>, std::allocator<char> > > > > const&)@Base" 1.1.0+14.10.20140804
  (c++)"core::net::http::Client::uri_to_string(core::net::Uri const&) const@Base" 1.1.0+14.10.20140804
  (c++)"core::net::http::Client::Errors::HttpMethodNotSupported::HttpMethodNotSupported(core::net::http::Method, core::Location const&)@Base" 0.0.1+14.10.20140611
@@ -41,3 +42,6 @@
  (c++)"vtable for core::net::http::Header@Base" 0.0.1+14.10.20140611
  (c++)"vtable for core::net::http::Request::Errors::AlreadyActive@Base" 0.0.1+14.10.20140611
  (c++)"vtable for core::net::http::Request@Base" 0.0.1+14.10.20140611
+ (c++)"typeinfo for core::net::http::StreamingRequest@Base" 1.1.0+15.04.20150305-0ubuntu1
+ (c++)"typeinfo name for core::net::http::StreamingRequest@Base" 1.1.0+15.04.20150305-0ubuntu1
+ (c++)"vtable for core::net::http::StreamingRequest@Base" 1.1.0+15.04.20150305-0ubuntu1
\ No newline at end of file

=== added file 'include/core/net/http/streaming_client.h'
--- include/core/net/http/streaming_client.h	1970-01-01 00:00:00 +0000
+++ include/core/net/http/streaming_client.h	2015-04-01 06:23:51 +0000
@@ -0,0 +1,88 @@
+/*
+ * Copyright © 2013 Canonical Ltd.
+ *
+ * This program is free software: you can redistribute it and/or modify it
+ * under the terms of the GNU Lesser General Public License version 3,
+ * as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ *
+ * Authored by: Thomas Voß <[e-mail]>
+ */
+#ifndef CORE_NET_HTTP_STREAMING_CLIENT_H_
+#define CORE_NET_HTTP_STREAMING_CLIENT_H_
+
+#include <core/net/http/client.h>
+
+#include <core/net/http/streaming_request.h>
+
+namespace core
+{
+namespace net
+{
+namespace http
+{
+class StreamingClient : public Client
+{
+public:
+
+    virtual ~StreamingClient() = default;
+
+    /**
+    * @brief streaming_get is a convenience method for issueing a GET request for the given URI.
+    * @throw Errors::HttpMethodNotSupported if the underlying implementation does not support the provided HTTP method.
+    * @param configuration The configuration to issue a get request for.
+    * @return An executable instance of class Request.
+    */
+    virtual std::shared_ptr<StreamingRequest> streaming_get(const Request::Configuration& configuration) = 0;
+
+    /**
+    * @brief streaming_head is a convenience method for issueing a HEAD request for the given URI.
+    * @throw Errors::HttpMethodNotSupported if the underlying implementation does not support the provided HTTP method.
+    * @param configuration The configuration to issue a get request for.
+    * @return An executable instance of class Request.
+    */
+    virtual std::shared_ptr<StreamingRequest> streaming_head(const Request::Configuration& configuration) = 0;
+
+    /**
+    * @brief streaming_put is a convenience method for issuing a PUT request for the given URI.
+    * @throw Errors::HttpMethodNotSupported if the underlying implementation does not support the provided HTTP method.
+    * @param configuration The configuration to issue a get request for.
+    * @param payload The data to be transmitted as part of the PUT request.
+    * @param size Size of the payload data in bytes.
+    * @return An executable instance of class Request.
+    */
+    virtual std::shared_ptr<StreamingRequest> streaming_put(const Request::Configuration& configuration, std::istream& payload, std::size_t size) = 0;
+
+    /**
+    * @brief streaming_post is a convenience method for issuing a POST request for the given URI.
+    * @throw Errors::HttpMethodNotSupported if the underlying implementation does not support the provided HTTP method.
+    * @param configuration The configuration to issue a get request for.
+    * @param payload The data to be transmitted as part of the POST request.
+    * @param type The content-type of the data.
+    * @return An executable instance of class Request.
+    */
+    virtual std::shared_ptr<StreamingRequest> streaming_post(const Request::Configuration& configuration, const std::string& payload, const std::string& type) = 0;
+
+    /**
+    * @brief streaming_post_form is a convenience method for issuing a POST request for the given URI, with url-encoded payload.
+    * @throw Errors::HttpMethodNotSupported if the underlying implementation does not support the provided HTTP method.
+    * @param configuration The configuration to issue a get request for.
+    * @param values Key-value pairs to be added to the payload in url-encoded format.
+    * @return An executable instance of class Request.
+    */
+    virtual std::shared_ptr<StreamingRequest> streaming_post_form(const Request::Configuration& configuration, const std::map<std::string, std::string>& values) = 0;
+};
+
+/** @brief Dispatches to the default implementation and returns a streaming client instance. */
+CORE_NET_DLL_PUBLIC std::shared_ptr<StreamingClient> make_streaming_client();
+}
+}
+}
+#endif // CORE_NET_HTTP_STREAMING_CLIENT_H_

=== added file 'include/core/net/http/streaming_request.h'
--- include/core/net/http/streaming_request.h	1970-01-01 00:00:00 +0000
+++ include/core/net/http/streaming_request.h	2015-04-01 06:23:51 +0000
@@ -0,0 +1,60 @@
+/*
+ * Copyright © 2013 Canonical Ltd.
+ *
+ * This program is free software: you can redistribute it and/or modify it
+ * under the terms of the GNU Lesser General Public License version 3,
+ * as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ *
+ * Authored by: Thomas Voß <[e-mail]>
+ */
+#ifndef CORE_NET_HTTP_STREAMING_REQUEST_H_
+#define CORE_NET_HTTP_STREAMING_REQUEST_H_
+
+#include <core/net/http/request.h>
+
+namespace core
+{
+namespace net
+{
+namespace http
+{
+/**
+ * @brief The StreamingRequest class encapsulates a request for a web resource,
+ * streaming data to the receiver as it receives in addition to accumulating all incoming data.
+ */
+class CORE_NET_DLL_PUBLIC StreamingRequest : public Request
+{
+public:
+
+    /** DataHandler is invoked when a new chunk of data arrives from the server. */
+    typedef std::function<void(const std::string&)> DataHandler;
+
+    /**
+     * @brief Synchronously executes the request.
+     * @throw core::net::http::Error in case of http-related errors.
+     * @throw core::net::Error in case of network-related errors.
+     * @return The response to the request.
+     */
+    virtual Response execute(const ProgressHandler& ph, const DataHandler& dh) = 0;
+
+    /**
+     * @brief Asynchronously executes the request, reporting errors, progress and completion to the given handlers.
+     * @param handler The handlers to called for events happening during execution of the request.
+     * @param dh The data handler receiving chunks of data while executing the request.
+     * @return The response to the request.
+     */
+    virtual void async_execute(const Handler& handler, const DataHandler& dh) = 0;
+};
+}
+}
+}
+
+#endif // CORE_NET_HTTP_STREAMING_REQUEST_H_

=== modified file 'src/CMakeLists.txt'
--- src/CMakeLists.txt	2014-06-10 14:19:14 +0000
+++ src/CMakeLists.txt	2015-04-01 06:23:51 +0000
@@ -22,6 +22,8 @@
 add_library(
   net-cpp SHARED
 
+  ${NET_CPP_INTERFACE_HEADERS}
+
   core/location.cpp
 
   core/net/error.cpp

=== modified file 'src/core/net/http/impl/curl/client.cpp'
--- src/core/net/http/impl/curl/client.cpp	2014-06-10 14:19:14 +0000
+++ src/core/net/http/impl/curl/client.cpp	2015-04-01 06:23:51 +0000
@@ -20,6 +20,7 @@
 #include "curl.h"
 #include "request.h"
 
+#include <core/net/http/content_type.h>
 #include <core/net/http/method.h>
 
 #include <boost/archive/iterators/base64_from_binary.hpp>
@@ -107,7 +108,7 @@
     multi.stop();
 }
 
-std::shared_ptr<http::Request> http::impl::curl::Client::head(const http::Request::Configuration& configuration)
+std::shared_ptr<http::impl::curl::Request> http::impl::curl::Client::head_impl(const http::Request::Configuration& configuration)
 {
     ::curl::easy::Handle handle;
     handle.method(http::Method::head)
@@ -126,10 +127,10 @@
         handle.http_credentials(credentials.username, credentials.password);
     }
 
-    return std::shared_ptr<http::Request>{new http::impl::curl::Request{multi, handle}};
+    return std::shared_ptr<http::impl::curl::Request>{new http::impl::curl::Request{multi, handle}};
 }
 
-std::shared_ptr<http::Request> http::impl::curl::Client::get(const http::Request::Configuration& configuration)
+std::shared_ptr<http::impl::curl::Request> http::impl::curl::Client::get_impl(const http::Request::Configuration& configuration)
 {
     ::curl::easy::Handle handle;
     handle.method(http::Method::get)
@@ -147,10 +148,10 @@
         handle.http_credentials(credentials.username, credentials.password);
     }
 
-    return std::shared_ptr<http::Request>{new http::impl::curl::Request{multi, handle}};
+    return std::shared_ptr<http::impl::curl::Request>{new http::impl::curl::Request{multi, handle}};
 }
 
-std::shared_ptr<http::Request> http::impl::curl::Client::post(
+std::shared_ptr<http::impl::curl::Request> http::impl::curl::Client::post_impl(
         const Request::Configuration& configuration,
         const std::string& payload,
         const std::string& ct)
@@ -172,10 +173,10 @@
         handle.http_credentials(credentials.username, credentials.password);
     }
 
-    return std::shared_ptr<http::Request>{new http::impl::curl::Request{multi, handle}};
+    return std::shared_ptr<http::impl::curl::Request>{new http::impl::curl::Request{multi, handle}};
 }
 
-std::shared_ptr<http::Request> http::impl::curl::Client::put(
+std::shared_ptr<http::impl::curl::Request> http::impl::curl::Client::put_impl(
         const Request::Configuration& configuration,
         std::istream& payload,
         std::size_t size)
@@ -201,10 +202,75 @@
         handle.http_credentials(credentials.username, credentials.password);
     }
 
-    return std::shared_ptr<http::Request>{new http::impl::curl::Request{multi, handle}};
+    return std::shared_ptr<http::impl::curl::Request>{new http::impl::curl::Request{multi, handle}};
+}
+
+std::shared_ptr<http::StreamingRequest> http::impl::curl::Client::streaming_get(const http::Request::Configuration& configuration)
+{
+    return get_impl(configuration);
+}
+
+std::shared_ptr<http::StreamingRequest> http::impl::curl::Client::streaming_head(const http::Request::Configuration& configuration)
+{
+    return head_impl(configuration);
+}
+
+std::shared_ptr<http::StreamingRequest> http::impl::curl::Client::streaming_put(const http::Request::Configuration& configuration, std::istream& payload, std::size_t size)
+{
+    return put_impl(configuration, payload, size);
+}
+
+std::shared_ptr<http::StreamingRequest> http::impl::curl::Client::streaming_post(const http::Request::Configuration& configuration, const std::string& payload, const std::string& type)
+{
+    return post_impl(configuration, payload, type);
+}
+
+std::shared_ptr<http::StreamingRequest> http::impl::curl::Client::streaming_post_form(const http::Request::Configuration& configuration, const std::map<std::string, std::string>& values)
+{
+    std::stringstream ss;
+    bool first{true};
+
+    for (const auto& pair : values)
+    {
+        ss << (first ? "" : "&") << url_escape(pair.first) << "=" << url_escape(pair.second);
+        first = false;
+    }
+
+    return post_impl(configuration, ss.str(), http::ContentType::x_www_form_urlencoded);
+}
+
+std::shared_ptr<http::Request> http::impl::curl::Client::head(const http::Request::Configuration& configuration)
+{
+    return head_impl(configuration);
+}
+
+std::shared_ptr<http::Request> http::impl::curl::Client::get(const http::Request::Configuration& configuration)
+{
+    return get_impl(configuration);
+}
+
+std::shared_ptr<http::Request> http::impl::curl::Client::post(
+        const Request::Configuration& configuration,
+        const std::string& payload,
+        const std::string& ct)
+{
+    return post_impl(configuration, payload, ct);
+}
+
+std::shared_ptr<http::Request> http::impl::curl::Client::put(
+        const Request::Configuration& configuration,
+        std::istream& payload,
+        std::size_t size)
+{
+    return put_impl(configuration, payload, size);
 }
 
 std::shared_ptr<http::Client> http::make_client()
 {
     return std::make_shared<http::impl::curl::Client>();
 }
+
+std::shared_ptr<http::StreamingClient> http::make_streaming_client()
+{
+    return std::make_shared<http::impl::curl::Client>();
+}

=== modified file 'src/core/net/http/impl/curl/client.h'
--- src/core/net/http/impl/curl/client.h	2014-06-10 14:19:14 +0000
+++ src/core/net/http/impl/curl/client.h	2015-04-01 06:23:51 +0000
@@ -18,9 +18,10 @@
 #ifndef CORE_NET_HTTP_IMPL_CURL_CLIENT_H_
 #define CORE_NET_HTTP_IMPL_CURL_CLIENT_H_
 
-#include <core/net/http/client.h>
+#include <core/net/http/streaming_client.h>
 
 #include "curl.h"
+#include "request.h"
 
 namespace core
 {
@@ -32,7 +33,7 @@
 {
 namespace curl
 {
-class Client : public core::net::http::Client
+class Client : public core::net::http::StreamingClient
 {
 public:
     Client();
@@ -45,23 +46,30 @@
 
     std::string base64_decode(const std::string& s) const override;
 
-    core::net::http::Client::Timings timings();
-
-    void run();
-
-    void stop();
-
-    std::shared_ptr<Request> get(const Request::Configuration& configuration);
-
-    std::shared_ptr<Request> head(const Request::Configuration& configuration);
-
-    std::shared_ptr<Request> post(const Request::Configuration& configuration, const std::string&, const std::string&);
-
-    std::shared_ptr<Request> put(const Request::Configuration& configuration, std::istream& payload, std::size_t size);
+    core::net::http::Client::Timings timings() override;
+
+    void run() override;
+
+    void stop() override;
+
+    std::shared_ptr<http::Request> get(const Request::Configuration& configuration) override;
+    std::shared_ptr<http::Request> head(const Request::Configuration& configuration) override;
+    std::shared_ptr<http::Request> post(const Request::Configuration& configuration, const std::string&, const std::string&) override;
+    std::shared_ptr<http::Request> put(const Request::Configuration& configuration, std::istream& payload, std::size_t size) override;
+
+    std::shared_ptr<http::StreamingRequest> streaming_get(const Request::Configuration& configuration) override;
+    std::shared_ptr<http::StreamingRequest> streaming_head(const Request::Configuration& configuration) override;
+    std::shared_ptr<http::StreamingRequest> streaming_put(const Request::Configuration& configuration, std::istream& payload, std::size_t size) override;
+    std::shared_ptr<http::StreamingRequest> streaming_post(const Request::Configuration& configuration, const std::string& payload, const std::string& type) override;
+    std::shared_ptr<http::StreamingRequest> streaming_post_form(const Request::Configuration& configuration, const std::map<std::string, std::string>& values) override;
 
 private:
-    ::curl::multi::Handle multi;
+    std::shared_ptr<curl::Request> get_impl(const Request::Configuration& configuration);
+    std::shared_ptr<curl::Request> head_impl(const Request::Configuration& configuration);
+    std::shared_ptr<curl::Request> post_impl(const Request::Configuration& configuration, const std::string&, const std::string&);
+    std::shared_ptr<curl::Request> put_impl(const Request::Configuration& configuration, std::istream& payload, std::size_t size);
 
+    ::curl::multi::Handle multi;    
 };
 }
 }

=== modified file 'src/core/net/http/impl/curl/request.h'
--- src/core/net/http/impl/curl/request.h	2015-01-23 06:23:31 +0000
+++ src/core/net/http/impl/curl/request.h	2015-04-01 06:23:51 +0000
@@ -18,7 +18,7 @@
 #ifndef CORE_NET_HTTP_IMPL_CURL_REQUEST_H_
 #define CORE_NET_HTTP_IMPL_CURL_REQUEST_H_
 
-#include <core/net/http/request.h>
+#include <core/net/http/streaming_request.h>
 
 #include <core/net/http/error.h>
 #include <core/net/http/response.h>
@@ -80,7 +80,7 @@
     std::atomic<core::net::http::Request::State>& state;
 };
 
-class Request : public core::net::http::Request,
+class Request : public core::net::http::StreamingRequest,
                 public std::enable_shared_from_this<Request>
 {
 public:
@@ -118,6 +118,11 @@
 
     Response execute(const Request::ProgressHandler& ph)
     {
+        return execute(ph, [](const std::string&){});
+    }
+
+    Response execute(const Request::ProgressHandler& ph, const StreamingRequest::DataHandler& dh)
+    {
         if (atomic_state.load() != core::net::http::Request::State::ready)
             throw core::net::http::Request::Errors::AlreadyActive{CORE_FROM_HERE()};
 
@@ -149,6 +154,8 @@
         easy.on_write_data(
                     [&](char* data, std::size_t size, std::size_t nmemb)
                     {
+                        // Report out to the data handler prior to accumulating data.
+                        dh(std::string{data, size * nmemb});
                         context.body.write(data, size * nmemb);
                         return size * nmemb;
                     });
@@ -183,6 +190,11 @@
 
     void async_execute(const Request::Handler& handler)
     {
+        async_execute(handler, [](const std::string&){});
+    }
+
+    void async_execute(const Request::Handler& handler, const StreamingRequest::DataHandler& dh)
+    {
         if (atomic_state.load() != core::net::http::Request::State::ready)
             throw core::net::http::Request::Errors::AlreadyActive{CORE_FROM_HERE()};
 
@@ -235,8 +247,10 @@
         }
 
         easy.on_write_data(
-                    [context](char* data, std::size_t size, std::size_t nmemb)
+                    [context, dh](char* data, std::size_t size, std::size_t nmemb)
                     {
+                        // Report out to the data handler prior to accumulating data.
+                        dh(std::string{data, size * nmemb});
                         context->body.write(data, size * nmemb);
                         return size * nmemb;
                     });

=== modified file 'tests/CMakeLists.txt'
--- tests/CMakeLists.txt	2014-05-22 11:50:55 +0000
+++ tests/CMakeLists.txt	2015-04-01 06:23:51 +0000
@@ -57,6 +57,11 @@
 )
 
 add_executable(
+  http_streaming_client_test
+  http_streaming_client_test.cpp
+)
+
+add_executable(
   http_client_load_test
   http_client_load_test.cpp
 )
@@ -82,6 +87,16 @@
 )
 
 target_link_libraries(
+    http_streaming_client_test
+
+    net-cpp
+
+    ${GMOCK_BOTH_LIBRARIES}
+    ${JSON_CPP_LDFLAGS}
+    ${PROCESS_CPP_LDFLAGS}
+)
+
+target_link_libraries(
     http_client_load_test
 
     net-cpp
@@ -93,4 +108,5 @@
 
 add_test(header_test ${CMAKE_CURRENT_BINARY_DIR}/header_test)
 add_test(http_client_test ${CMAKE_CURRENT_BINARY_DIR}/http_client_test)
+add_test(http_streaming_client_test ${CMAKE_CURRENT_BINARY_DIR}/http_streaming_client_test)
 add_test(http_client_load_test ${CMAKE_CURRENT_BINARY_DIR}/http_client_load_test)

=== added file 'tests/http_streaming_client_test.cpp'
--- tests/http_streaming_client_test.cpp	1970-01-01 00:00:00 +0000
+++ tests/http_streaming_client_test.cpp	2015-04-01 06:23:51 +0000
@@ -0,0 +1,508 @@
+/*
+ * Copyright © 2013 Canonical Ltd.
+ *
+ * This program is free software: you can redistribute it and/or modify it
+ * under the terms of the GNU Lesser General Public License version 3,
+ * as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ *
+ * Authored by: Thomas Voß <[e-mail]>
+ */
+
+#include <core/net/error.h>
+#include <core/net/uri.h>
+#include <core/net/http/streaming_client.h>
+#include <core/net/http/content_type.h>
+#include <core/net/http/request.h>
+#include <core/net/http/response.h>
+
+#include "httpbin.h"
+
+#include <gmock/gmock.h>
+#include <gtest/gtest.h>
+
+#include <json/json.h>
+
+#include <future>
+#include <memory>
+
+namespace http = core::net::http;
+namespace json = Json;
+namespace net = core::net;
+
+namespace
+{
+class MockDataHandler : public std::enable_shared_from_this<MockDataHandler>
+{
+public:
+    // We are enabling shared from this, thus forcing creation of
+    // managed instances here.
+    static std::shared_ptr<MockDataHandler> create()
+    {
+        return std::shared_ptr<MockDataHandler>(new MockDataHandler);
+    }
+
+    MOCK_METHOD1(on_new_data, void(const std::string&));
+
+    http::StreamingRequest::DataHandler to_data_handler()
+    {
+        auto thiz = shared_from_this();
+        return [thiz](const std::string& s)
+        {
+            thiz->on_new_data(s);
+        };
+    }
+
+private:
+    MockDataHandler() = default;
+};
+
+auto default_progress_reporter = [](const http::Request::Progress& progress)
+{
+    if (progress.download.current > 0. && progress.download.total > 0.)
+        std::cout << "Download progress: " << progress.download.current / progress.download.total << std::endl;
+    if (progress.upload.current > 0. && progress.upload.total > 0.)
+        std::cout << "Upload progress: " << progress.upload.current / progress.upload.total << std::endl;
+
+    return http::Request::Progress::Next::continue_operation;
+};
+
+bool init()
+{
+    static httpbin::Instance instance;
+    return true;
+}
+
+static const bool is_initialized = init();
+}
+
+TEST(StreamingStreamingHttpClient, head_request_for_existing_resource_succeeds)
+{
+    using namespace ::testing;
+
+    // We obtain a default client instance, dispatching to the default implementation.
+    auto client = http::make_streaming_client();
+
+    // Url pointing to the resource we would like to access via http.
+    auto url = std::string(httpbin::host) + httpbin::resources::get();
+
+    // The client mostly acts as a factory for http requests.
+    auto request = client->streaming_head(http::Request::Configuration::from_uri_as_string(url));
+
+    // Our mocked data handler.
+    auto dh = MockDataHandler::create(); EXPECT_CALL(*dh, on_new_data(_)).Times(AtLeast(1));
+
+    // We finally execute the query synchronously and story the response.
+    auto response = request->execute(default_progress_reporter, dh->to_data_handler());
+
+    // We expect the query to complete successfully
+    EXPECT_EQ(core::net::http::Status::ok, response.status);
+}
+
+TEST(StreamingHttpClient, get_request_for_existing_resource_succeeds)
+{
+    using namespace ::testing;
+
+    // We obtain a default client instance, dispatching to the default implementation.
+    auto client = http::make_streaming_client();
+
+    // Url pointing to the resource we would like to access via http.
+    auto url = std::string(httpbin::host) + httpbin::resources::get();
+
+    // The client mostly acts as a factory for http requests.
+    auto request = client->streaming_get(http::Request::Configuration::from_uri_as_string(url));
+
+    // Our mocked data handler.
+    auto dh = MockDataHandler::create(); EXPECT_CALL(*dh, on_new_data(_)).Times(AtLeast(1));
+
+    // All endpoint data on httpbin.org is JSON encoded.
+    json::Value root;
+    json::Reader reader;
+
+    // We finally execute the query synchronously and story the response.
+    auto response = request->execute(default_progress_reporter, dh->to_data_handler());
+
+    // We expect the query to complete successfully
+    EXPECT_EQ(core::net::http::Status::ok, response.status);
+    // Parsing the body of the response as JSON should succeed.
+    EXPECT_TRUE(reader.parse(response.body, root));
+    // The url field of the payload should equal the original url we requested.
+    EXPECT_EQ(url, root["url"].asString());
+}
+
+TEST(StreamingHttpClient, get_request_with_custom_headers_for_existing_resource_succeeds)
+{
+    using namespace ::testing;
+
+    // We obtain a default client instance, dispatching to the default implementation.
+    auto client = http::make_streaming_client();
+
+    // Url pointing to the resource we would like to access via http.
+    auto url = std::string(httpbin::host) + httpbin::resources::headers();
+
+    // The client mostly acts as a factory for http requests.
+    auto configuration = http::Request::Configuration::from_uri_as_string(url);
+    configuration.header.set("Test1", "42");
+    configuration.header.set("Test2", "43");
+
+    auto request = client->streaming_get(configuration);
+
+    // Our mocked data handler.
+    auto dh = MockDataHandler::create(); EXPECT_CALL(*dh, on_new_data(_)).Times(AtLeast(1));
+
+    // All endpoint data on httpbin.org is JSON encoded.
+    json::Value root;
+    json::Reader reader;
+
+    // We finally execute the query synchronously and story the response.
+    auto response = request->execute(default_progress_reporter, dh->to_data_handler());
+
+    // We expect the query to complete successfully
+    EXPECT_EQ(core::net::http::Status::ok, response.status);
+
+    // Parsing the body of the response as JSON should succeed.
+    EXPECT_TRUE(reader.parse(response.body, root));
+
+    auto headers = root["headers"];
+
+    EXPECT_EQ("42", headers["Test1"].asString());
+    EXPECT_EQ("43", headers["Test2"].asString());
+}
+
+TEST(StreamingHttpClient, empty_header_values_are_handled_correctly)
+{
+    using namespace ::testing;
+
+    // We obtain a default client instance, dispatching to the default implementation.
+    auto client = http::make_streaming_client();
+
+    // Url pointing to the resource we would like to access via http.
+    auto url = std::string(httpbin::host) + httpbin::resources::headers();
+
+    // The client mostly acts as a factory for http requests.
+    auto configuration = http::Request::Configuration::from_uri_as_string(url);
+    configuration.header.set("Empty", std::string{});
+
+    auto request = client->streaming_get(configuration);
+
+    // Our mocked data handler.
+    auto dh = MockDataHandler::create(); EXPECT_CALL(*dh, on_new_data(_)).Times(AtLeast(1));
+
+    // All endpoint data on httpbin.org is JSON encoded.
+    json::Value root;
+    json::Reader reader;
+
+    // We finally execute the query synchronously and story the response.
+    auto response = request->execute(default_progress_reporter, dh->to_data_handler());
+
+    // We expect the query to complete successfully
+    EXPECT_EQ(core::net::http::Status::ok, response.status);
+
+    // Parsing the body of the response as JSON should succeed.
+    EXPECT_TRUE(reader.parse(response.body, root));
+
+    auto headers = root["headers"];
+    EXPECT_EQ(std::string{}, headers["Empty"].asString());
+}
+
+TEST(StreamingHttpClient, get_request_for_existing_resource_guarded_by_basic_auth_succeeds)
+{
+    using namespace ::testing;
+
+    // We obtain a default client instance, dispatching to the default implementation.
+    auto client = http::make_streaming_client();
+
+    // Url pointing to the resource we would like to access via http.
+    auto url = std::string(httpbin::host) + httpbin::resources::basic_auth();
+
+    // The client mostly acts as a factory for http requests.
+    auto configuration = http::Request::Configuration::from_uri_as_string(url);
+    configuration.authentication_handler.for_http = [](const std::string&)
+    {
+        return http::Request::Credentials{"user", "passwd"};
+    };
+    auto request = client->streaming_get(configuration);
+
+    // Our mocked data handler.
+    auto dh = MockDataHandler::create(); EXPECT_CALL(*dh, on_new_data(_)).Times(AtLeast(1));
+
+    // All endpoint data on httpbin.org is JSON encoded.
+    json::Value root;
+    json::Reader reader;
+
+    // We finally execute the query synchronously and story the response.
+    auto response = request->execute(default_progress_reporter, dh->to_data_handler());
+
+    // We expect the query to complete successfully
+    EXPECT_EQ(core::net::http::Status::ok, response.status);
+    // Parsing the body of the response as JSON should succeed.
+    EXPECT_TRUE(reader.parse(response.body, root));
+    // We expect authentication to work.
+    EXPECT_TRUE(root["authenticated"].asBool());
+    // With the correct user id
+    EXPECT_EQ("user", root["user"].asString());
+}
+
+// Digest auth is broken on httpbin.org. It even fails in the browser after the first successful access.
+TEST(StreamingHttpClient, DISABLED_get_request_for_existing_resource_guarded_by_digest_auth_succeeds)
+{
+    using namespace ::testing;
+
+    // We obtain a default client instance, dispatching to the default implementation.
+    auto client = http::make_streaming_client();
+
+    // Url pointing to the resource we would like to access via http.
+    auto url = std::string(httpbin::host) + httpbin::resources::digest_auth();
+
+    // The client mostly acts as a factory for http requests.
+    auto configuration = http::Request::Configuration::from_uri_as_string(url);
+    configuration.authentication_handler.for_http = [](const std::string&)
+    {
+        return http::Request::Credentials{"user", "passwd"};
+    };
+    auto request = client->streaming_get(configuration);
+
+    // Our mocked data handler.
+    auto dh = MockDataHandler::create(); EXPECT_CALL(*dh, on_new_data(_)).Times(AtLeast(1));
+
+    // All endpoint data on httpbin.org is JSON encoded.
+    json::Value root;
+    json::Reader reader;
+
+    // We finally execute the query synchronously and story the response.
+    auto response = request->execute(default_progress_reporter, dh->to_data_handler());
+
+    // We expect the query to complete successfully
+    EXPECT_EQ(core::net::http::Status::ok, response.status);
+    // Parsing the body of the response as JSON should succeed.
+    EXPECT_TRUE(reader.parse(response.body, root));
+    // We expect authentication to work.
+    EXPECT_TRUE(root["authenticated"].asBool());
+    // With the correct user id
+    EXPECT_EQ("user", root["user"].asString());
+}
+
+TEST(StreamingHttpClient, async_get_request_for_existing_resource_succeeds)
+{
+    using namespace ::testing;
+
+    // We obtain a default client instance, dispatching to the default implementation.
+    auto client = http::make_streaming_client();
+
+    // Execute the client
+    std::thread worker{[client]() { client->run(); }};
+
+    // Url pointing to the resource we would like to access via http.
+    auto url = std::string(httpbin::host) + httpbin::resources::get();
+
+    // The client mostly acts as a factory for http requests.
+    auto request = client->streaming_get(http::Request::Configuration::from_uri_as_string(url));
+
+    // Our mocked data handler.
+    auto dh = MockDataHandler::create(); EXPECT_CALL(*dh, on_new_data(_)).Times(AtLeast(1));
+
+    std::promise<core::net::http::Response> promise;
+    auto future = promise.get_future();
+
+    // We finally execute the query asynchronously.
+    request->async_execute(
+                http::Request::Handler()
+                    .on_progress(default_progress_reporter)
+                    .on_response([&](const core::net::http::Response& response)
+                    {
+                        promise.set_value(response);
+                    })
+                    .on_error([&](const core::net::Error& e)
+                    {
+                        promise.set_exception(std::make_exception_ptr(e));
+                    }),
+                dh->to_data_handler());
+
+    auto response = future.get();
+
+    // All endpoint data on httpbin.org is JSON encoded.
+    json::Value root;
+    json::Reader reader;
+
+    // We expect the query to complete successfully
+    EXPECT_EQ(core::net::http::Status::ok, response.status);
+    // Parsing the body of the response as JSON should succeed.
+    EXPECT_TRUE(reader.parse(response.body, root));
+    // The url field of the payload should equal the original url we requested.
+    EXPECT_EQ(url, root["url"].asString());
+
+    client->stop();
+
+    // We shut down our worker thread
+    if (worker.joinable())
+        worker.join();
+}
+
+TEST(StreamingHttpClient, async_get_request_for_existing_resource_guarded_by_basic_authentication_succeeds)
+{
+    using namespace ::testing;
+
+    // We obtain a default client instance, dispatching to the default implementation.
+    auto client = http::make_streaming_client();
+
+    // Execute the client
+    std::thread worker{[client]() { client->run(); }};
+
+    // Url pointing to the resource we would like to access via http.
+    auto url = std::string(httpbin::host) + httpbin::resources::basic_auth();
+
+    // The client mostly acts as a factory for http requests.
+    auto configuration = http::Request::Configuration::from_uri_as_string(url);
+
+    configuration.authentication_handler.for_http = [](const std::string&)
+    {
+        return http::Request::Credentials{"user", "passwd"};
+    };
+
+    auto request = client->streaming_get(configuration);
+
+    // Our mocked data handler.
+    auto dh = MockDataHandler::create(); EXPECT_CALL(*dh, on_new_data(_)).Times(AtLeast(1));
+
+    // All endpoint data on httpbin.org is JSON encoded.
+    json::Value root;
+    json::Reader reader;
+
+    std::promise<core::net::http::Response> promise;
+    auto future = promise.get_future();
+
+    // We finally execute the query asynchronously.
+    request->async_execute(
+                http::Request::Handler()
+                    .on_progress(default_progress_reporter)
+                    .on_response([&](const core::net::http::Response& response)
+                    {
+                        promise.set_value(response);
+                        client->stop();
+                    })
+                    .on_error([&](const core::net::Error& e)
+                    {
+                        promise.set_exception(std::make_exception_ptr(e));
+                        client->stop();
+                    }),
+                dh->to_data_handler());
+
+    // And wait here for the response to arrive.
+    auto response = future.get();
+
+    // We shut down our worker thread
+    if (worker.joinable())
+        worker.join();
+
+    // We expect the query to complete successfully
+    EXPECT_EQ(core::net::http::Status::ok, response.status);
+    // Parsing the body of the response as JSON should succeed.
+    EXPECT_TRUE(reader.parse(response.body, root));
+    // We expect authentication to work.
+    EXPECT_TRUE(root["authenticated"].asBool());
+    // With the correct user id
+    EXPECT_EQ("user", root["user"].asString());
+}
+
+TEST(StreamingHttpClient, post_request_for_existing_resource_succeeds)
+{
+    using namespace ::testing;
+
+    // We obtain a default client instance, dispatching to the default implementation.
+    auto client = http::make_streaming_client();
+
+    // Url pointing to the resource we would like to access via http.
+    auto url = std::string(httpbin::host) + httpbin::resources::post();
+
+    std::string payload = "{ 'test': 'test' }";
+
+    // The client mostly acts as a factory for http requests.
+    auto request = client->streaming_post(http::Request::Configuration::from_uri_as_string(url),
+                                payload,
+                                core::net::http::ContentType::json);
+
+    // Our mocked data handler.
+    auto dh = MockDataHandler::create(); EXPECT_CALL(*dh, on_new_data(_)).Times(AtLeast(1));
+
+    // All endpoint data on httpbin.org is JSON encoded.
+    json::Value root;
+    json::Reader reader;
+
+    // We finally execute the query synchronously and story the response.
+    auto response = request->execute(default_progress_reporter, dh->to_data_handler());
+
+    // We expect the query to complete successfully
+    EXPECT_EQ(core::net::http::Status::ok, response.status);
+    // Parsing the body of the response as JSON should succeed.
+    EXPECT_TRUE(reader.parse(response.body, root));
+    // The url field of the payload should equal the original url we requested.
+    EXPECT_EQ(payload, root["data"].asString());
+}
+
+TEST(StreamingHttpClient, post_form_request_for_existing_resource_succeeds)
+{
+    using namespace ::testing;
+
+    // We obtain a default client instance, dispatching to the default implementation.
+    auto client = http::make_streaming_client();
+
+    // Url pointing to the resource we would like to access via http.
+    auto url = std::string(httpbin::host) + httpbin::resources::post();
+
+    std::map<std::string, std::string> values
+    {
+        {"test", "test"}
+    };
+
+    // The client mostly acts as a factory for http requests.
+    auto request = client->streaming_post_form(http::Request::Configuration::from_uri_as_string(url),
+                                               values);
+
+    // Our mocked data handler.
+    auto dh = MockDataHandler::create(); EXPECT_CALL(*dh, on_new_data(_)).Times(AtLeast(1));
+
+    // We finally execute the query synchronously and store the response.
+    auto response = request->execute(default_progress_reporter, dh->to_data_handler());
+
+    // All endpoint data on httpbin.org is JSON encoded.
+    json::Value root;
+    json::Reader reader;
+
+    EXPECT_EQ(core::net::http::Status::ok, response.status);
+    EXPECT_TRUE(reader.parse(response.body, root));
+    EXPECT_EQ("test", root["form"]["test"].asString());
+}
+
+TEST(StreamingHttpClient, put_request_for_existing_resource_succeeds)
+{
+    using namespace ::testing;
+
+    auto client = http::make_streaming_client();
+    auto url = std::string(httpbin::host) + httpbin::resources::put();
+
+    const std::string value{"{ 'test': 'test' }"};
+    std::stringstream payload(value);
+
+    auto request = client->streaming_put(http::Request::Configuration::from_uri_as_string(url),
+                               payload,
+                               value.size());
+
+    // Our mocked data handler.
+    auto dh = MockDataHandler::create(); EXPECT_CALL(*dh, on_new_data(_)).Times(AtLeast(1));
+
+    json::Value root;
+    json::Reader reader;
+
+    auto response = request->execute(default_progress_reporter, dh->to_data_handler());
+
+    EXPECT_EQ(core::net::http::Status::ok, response.status);
+    EXPECT_TRUE(reader.parse(response.body, root));
+    EXPECT_EQ(payload.str(), root["data"].asString());
+}