Merge lp:~gary-wzl77/net-cpp/reading_callback into lp:net-cpp

Proposed by Gary.Wang
Status: Merged
Merged at revision: 50
Proposed branch: lp:~gary-wzl77/net-cpp/reading_callback
Merge into: lp:net-cpp
Diff against target: 1342 lines (+717/-82)
21 files modified
CMakeLists.txt (+2/-2)
debian/changelog (+13/-0)
debian/control (+2/-6)
debian/libnet-cpp1.symbols (+0/-47)
debian/rules (+0/-8)
include/core/net/http/client.h (+19/-0)
include/core/net/http/method.h (+3/-1)
include/core/net/http/request.h (+21/-0)
include/core/net/http/streaming_client.h (+42/-0)
include/core/net/http/streaming_request.h (+1/-0)
src/core/net/http/impl/curl/client.cpp (+193/-2)
src/core/net/http/impl/curl/client.h (+11/-0)
src/core/net/http/impl/curl/easy.cpp (+23/-0)
src/core/net/http/impl/curl/easy.h (+16/-2)
src/core/net/http/impl/curl/multi.cpp (+5/-0)
src/core/net/http/impl/curl/multi.h (+3/-0)
src/core/net/http/impl/curl/request.h (+30/-1)
tests/http_client_load_test.cpp (+1/-1)
tests/http_client_test.cpp (+74/-1)
tests/http_streaming_client_test.cpp (+251/-10)
tests/httpbin.h.in (+7/-1)
To merge this branch: bzr merge lp:~gary-wzl77/net-cpp/reading_callback
Reviewer Review Type Date Requested Status
Thomas Voß Pending
Ubuntu Phablet Team Pending
Review via email: mp+300744@code.launchpad.net

Commit message

Support data reading call back function in streaming API. That makes possible to upload data into the server directly without local cache.

Description of the change

Support data reading call back function in streaming API. That makes possible to upload data into server directly without local cache.

To post a comment you must log in.
58. By Gary.Wang

Fixes typo.

59. By Gary.Wang

fixes linking error against xenial and yakkety.

60. By Gary.Wang

fixes tests failure when bulding with gcc5.

61. By Gary.Wang

update symbols file.

62. By Gary.Wang

using small package for testing(pause_and_resume).

63. By Gary.Wang

don't ship symbol file.

64. By Gary.Wang

waits enough time for server process to launch.

sometimes tests failed when running them in yakkety(armhf). As while we start to run tests, server is not ready yet sometimes.

Revision history for this message
James Henstridge (jamesh) wrote :

So they don't get lost, here's a summary of the issues I outlined in email:

1. Currently the libnet-cpp.so.2 soname essentially means "libnet-cpp.so.1 recompiled with the new C++11 ABI". If you are breaking the ABI of net-cpp, then the result should not be called libnet-cpp.so.1 _or_ libnet-cpp.so.2.

2. Following on from that, the library should still have different sonames on either side of the C++11 ABI break, so perhaps .so.3 on vivid and .so.4 on newer distros. It is possible to handle this kind of split with a single source tree going through citrain's dual landings via the bileto_pre_release_hook system. The mediascanner2 package is probably one of the simpler examples of this.

3. Replacing the net-cpp package on vivid is a no-go. It is part of the base image that Click packages depend on, and is explicitly part of the ubuntu-sdk-libs meta-package. The packages for the new and old net-cpp need to be able to co-exist in the archive, which means different source package names, and a different development package name.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'CMakeLists.txt'
2--- CMakeLists.txt 2015-04-01 06:23:30 +0000
3+++ CMakeLists.txt 2016-08-02 06:33:04 +0000
4@@ -26,8 +26,8 @@
5 set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++11 -Werror -Wall -fno-strict-aliasing -fvisibility=hidden -fvisibility-inlines-hidden -pedantic -Wextra")
6 set(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} -Wl,--no-undefined")
7
8-set(NET_CPP_VERSION_MAJOR 1)
9-set(NET_CPP_VERSION_MINOR 2)
10+set(NET_CPP_VERSION_MAJOR 2)
11+set(NET_CPP_VERSION_MINOR 0)
12 set(NET_CPP_VERSION_PATCH 0)
13
14 include(CTest)
15
16=== modified file 'debian/changelog'
17--- debian/changelog 2015-04-15 12:34:49 +0000
18+++ debian/changelog 2016-08-02 06:33:04 +0000
19@@ -1,3 +1,16 @@
20+net-cpp (2.0.0ubuntu1) UNRELEASED; urgency=medium
21+
22+ [ Thomas Voß ]
23+ [Gary Wang]
24+ * Enable pause/resume of requests.
25+ * Fix LP:#1570686 and LP:#1570687
26+
27+ [ Gary Wang ]
28+ * Add data reading callback function in streaming http interface.
29+ * Fix LP:#1605179
30+
31+ -- Thomas Voß <thomas.voss@canonical.com> Thu, 21 Jul 2016 04:05:05 -0700
32+
33 net-cpp (1.2.0+15.04.20150415.2-0ubuntu1) vivid; urgency=medium
34
35 [ Thomas Voß ]
36
37=== modified file 'debian/control'
38--- debian/control 2015-02-25 10:27:29 +0000
39+++ debian/control 2016-08-02 06:33:04 +0000
40@@ -6,10 +6,6 @@
41 doxygen,
42 google-mock,
43 graphviz,
44-# We rely on C++11 features, and to prevent from ABI breaks
45-# in libstdc++ causing us issues, we explicitly select a G++
46-# version.
47- g++-4.9,
48 libboost-dev,
49 libboost-serialization-dev,
50 libboost-system-dev,
51@@ -29,7 +25,7 @@
52 Vcs-Bzr: https://code.launchpad.net/~phablet-team/net-cpp/trunk
53 Vcs-Browser: https://bazaar.launchpad.net/~phablet-team/net-cpp/trunk/files
54
55-Package: libnet-cpp1
56+Package: libnet-cpp2
57 Architecture: any
58 Multi-Arch: same
59 Pre-Depends: ${misc:Pre-Depends},
60@@ -45,7 +41,7 @@
61 Architecture: any
62 Multi-Arch: same
63 Pre-Depends: ${misc:Pre-Depends},
64-Depends: libnet-cpp1 (= ${binary:Version}),
65+Depends: libnet-cpp2 (= ${binary:Version}),
66 ${misc:Depends},
67 Description: C++11 library for networking purposes - runtime library
68 Net-Cpp is a simple and straightforward networking library for C++11.
69
70=== removed file 'debian/libnet-cpp1.symbols'
71--- debian/libnet-cpp1.symbols 2015-03-23 16:24:27 +0000
72+++ debian/libnet-cpp1.symbols 1970-01-01 00:00:00 +0000
73@@ -1,47 +0,0 @@
74-libnet-cpp.so.1 libnet-cpp1 #MINVER#
75- (c++)"core::net::http::make_client()@Base" 0.0.1+14.10.20140611
76- (c++)"core::net::http::make_streaming_client()@Base" 1.1.0+15.04.20150305-0ubuntu1
77- (c++)"core::net::make_uri(std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, std::vector<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::basic_string<char, std::char_traits<char>, std::allocator<char> > > > > const&)@Base" 1.1.0+14.10.20140804
78- (c++)"core::net::http::Client::uri_to_string(core::net::Uri const&) const@Base" 1.1.0+14.10.20140804
79- (c++)"core::net::http::Client::Errors::HttpMethodNotSupported::HttpMethodNotSupported(core::net::http::Method, core::Location const&)@Base" 0.0.1+14.10.20140611
80- (c++)"core::net::http::Client::Errors::HttpMethodNotSupported::~HttpMethodNotSupported()@Base" 0.0.1+14.10.20140611
81- (c++)"core::net::http::Client::post_form(core::net::http::Request::Configuration const&, std::map<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::basic_string<char, std::char_traits<char>, std::allocator<char> > > > > const&)@Base" 0.0.1+14.10.20140611
82- (c++)"core::net::http::Header::canonicalize_key(std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)@Base" 0.0.1+14.10.20140611
83- (c++)"core::net::http::Header::add(std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)@Base" 0.0.1+14.10.20140611
84- (c++)"core::net::http::Header::set(std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)@Base" 0.0.1+14.10.20140611
85- (c++)"core::net::http::Header::remove(std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)@Base" 0.0.1+14.10.20140611
86- (c++)"core::net::http::Header::remove(std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)@Base" 0.0.1+14.10.20140611
87- (c++)"core::net::http::Header::enumerate(std::function<void (std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::set<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&)> const&) const@Base" 0.0.1+14.10.20140611
88- (c++)"core::net::http::Request::Credentials::~Credentials()@Base" 0.0.1+14.10.20140611
89- (c++)"core::net::http::Request::Errors::AlreadyActive::AlreadyActive(core::Location const&)@Base" 0.0.1+14.10.20140611
90- (c++)"core::net::http::Request::Errors::AlreadyActive::~AlreadyActive()@Base" 0.0.1+14.10.20140611
91- (c++)"core::net::http::Request::Handler::Handler(core::net::http::Request::Handler const&)@Base" 1.1.0+15.04.20141204
92- (c++)"core::net::http::Request::Handler::~Handler()@Base" 1.1.0+15.04.20141204
93- (c++)"core::net::http::Request::Handler::on_progress(std::function<core::net::http::Request::Progress::Next (core::net::http::Request::Progress const&)> const&)@Base" 0.0.1+14.10.20140611
94- (c++)"core::net::http::Request::Handler::on_response(std::function<void (core::net::http::Response const&)> const&)@Base" 0.0.1+14.10.20140611
95- (c++)"core::net::http::Request::Handler::on_error(std::function<void (core::net::Error const&)> const&)@Base" 0.0.1+14.10.20140611
96- (c++)"core::net::http::Response::~Response()@Base" 0.0.1+14.10.20140611
97- (c++)"core::net::http::operator<<(std::basic_ostream<char, std::char_traits<char> >&, core::net::http::Status)@Base" 0.0.1+14.10.20140611
98- (c++)"core::net::http::Header::has(std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const@Base" 0.0.1+14.10.20140611
99- (c++)"core::net::http::Header::has(std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const@Base" 0.0.1+14.10.20140611
100- (c++)"core::net::http::Request::Handler::on_progress() const@Base" 0.0.1+14.10.20140611
101- (c++)"core::net::http::Request::Handler::on_response() const@Base" 0.0.1+14.10.20140611
102- (c++)"core::net::http::Request::Handler::on_error() const@Base" 0.0.1+14.10.20140611
103- (c++)"typeinfo for core::net::http::Client::Errors::HttpMethodNotSupported@Base" 0.0.1+14.10.20140611
104- (c++)"typeinfo for core::net::http::Client@Base" 0.0.1+14.10.20140611
105- (c++)"typeinfo for core::net::http::Header@Base" 0.0.1+14.10.20140611
106- (c++)"typeinfo for core::net::http::Request::Errors::AlreadyActive@Base" 0.0.1+14.10.20140611
107- (c++)"typeinfo for core::net::http::Request@Base" 0.0.1+14.10.20140611
108- (c++)"typeinfo name for core::net::http::Client::Errors::HttpMethodNotSupported@Base" 0.0.1+14.10.20140611
109- (c++)"typeinfo name for core::net::http::Client@Base" 0.0.1+14.10.20140611
110- (c++)"typeinfo name for core::net::http::Header@Base" 0.0.1+14.10.20140611
111- (c++)"typeinfo name for core::net::http::Request::Errors::AlreadyActive@Base" 0.0.1+14.10.20140611
112- (c++)"typeinfo name for core::net::http::Request@Base" 0.0.1+14.10.20140611
113- (c++)"vtable for core::net::http::Client::Errors::HttpMethodNotSupported@Base" 0.0.1+14.10.20140611
114- (c++)"vtable for core::net::http::Client@Base" 0.0.1+14.10.20140611
115- (c++)"vtable for core::net::http::Header@Base" 0.0.1+14.10.20140611
116- (c++)"vtable for core::net::http::Request::Errors::AlreadyActive@Base" 0.0.1+14.10.20140611
117- (c++)"vtable for core::net::http::Request@Base" 0.0.1+14.10.20140611
118- (c++)"typeinfo for core::net::http::StreamingRequest@Base" 1.1.0+15.04.20150305-0ubuntu1
119- (c++)"typeinfo name for core::net::http::StreamingRequest@Base" 1.1.0+15.04.20150305-0ubuntu1
120- (c++)"vtable for core::net::http::StreamingRequest@Base" 1.1.0+15.04.20150305-0ubuntu1
121\ No newline at end of file
122
123=== renamed file 'debian/libnet-cpp1.install' => 'debian/libnet-cpp2.install'
124=== modified file 'debian/rules'
125--- debian/rules 2014-06-27 08:16:12 +0000
126+++ debian/rules 2016-08-02 06:33:04 +0000
127@@ -5,13 +5,5 @@
128
129 include /usr/share/dpkg/default.mk
130
131-# Explicitly selecting a G{CC,++}-version here to avoid accidental
132-# ABI breaks introduced by toolchain updates.
133-export CC=$(DEB_HOST_GNU_TYPE)-gcc-4.9
134-export CXX=$(DEB_HOST_GNU_TYPE)-g++-4.9
135-
136 %:
137 dh $@ --fail-missing
138-
139-override_dh_auto_configure:
140- dh_auto_configure -- -DCMAKE_C_COMPILER=$(CC) -DCMAKE_CXX_COMPILER=$(CXX)
141
142=== modified file 'include/core/net/http/client.h'
143--- include/core/net/http/client.h 2014-06-10 14:19:14 +0000
144+++ include/core/net/http/client.h 2016-08-02 06:33:04 +0000
145@@ -14,6 +14,7 @@
146 * along with this program. If not, see <http://www.gnu.org/licenses/>.
147 *
148 * Authored by: Thomas Voß <thomas.voss@canonical.com>
149+ * Gary Wang <gary.wang@canonical.com>
150 */
151 #ifndef CORE_NET_HTTP_CLIENT_H_
152 #define CORE_NET_HTTP_CLIENT_H_
153@@ -154,6 +155,16 @@
154 */
155 virtual std::shared_ptr<Request> post(const Request::Configuration& configuration, const std::string& payload, const std::string& type) = 0;
156
157+ /**
158+ * @brief post is a convenience method for issuing a POST request for the given URI.
159+ * @throw Errors::HttpMethodNotSupported if the underlying implementation does not support the pro vided HTTP method.
160+ * @param configuration The configuration to issue a get request for.
161+ * @param payload The data to be transmitted as part of the POST request.
162+ * @param size Size of the payload data in bytes.
163+ * @return An executable instance of class Request.
164+ */
165+ virtual std::shared_ptr<Request> post(const Request::Configuration& configuration, std::istream& payload, std::size_t size) = 0;
166+
167 /**
168 * @brief post_form is a convenience method for issuing a POST request for the given URI, with url-encoded payload.
169 * @throw Errors::HttpMethodNotSupported if the underlying implementation does not support the provided HTTP method.
170@@ -163,6 +174,14 @@
171 */
172 virtual std::shared_ptr<Request> post_form(const Request::Configuration& configuration, const std::map<std::string, std::string>& values);
173
174+ /**
175+ * @brief del is a convenience method for issueing a DELETE request for the given URI.
176+ * @throw Errors::HttpMethodNotSupported if the underlying implementation does not support the provided HTTP method.
177+ * @param configuration The configuration to issue a get request for.
178+ * @return An executable instance of class Request.
179+ */
180+ virtual std::shared_ptr<Request> del(const Request::Configuration& configuration) = 0;
181+
182 protected:
183 Client() = default;
184 };
185
186=== modified file 'include/core/net/http/method.h'
187--- include/core/net/http/method.h 2013-12-04 07:23:43 +0000
188+++ include/core/net/http/method.h 2016-08-02 06:33:04 +0000
189@@ -14,6 +14,7 @@
190 * along with this program. If not, see <http://www.gnu.org/licenses/>.
191 *
192 * Authored by: Thomas Voß <thomas.voss@canonical.com>
193+ * Gary Wang <gary.wang@canonical.com>
194 */
195 #ifndef CORE_NET_HTTP_METHOD_H_
196 #define CORE_NET_HTTP_METHOD_H_
197@@ -33,7 +34,8 @@
198 get,
199 head,
200 post,
201- put
202+ put,
203+ del
204 };
205 }
206 }
207
208=== modified file 'include/core/net/http/request.h'
209--- include/core/net/http/request.h 2014-05-06 11:05:04 +0000
210+++ include/core/net/http/request.h 2016-08-02 06:33:04 +0000
211@@ -14,6 +14,7 @@
212 * along with this program. If not, see <http://www.gnu.org/licenses/>.
213 *
214 * Authored by: Thomas Voß <thomas.voss@canonical.com>
215+ * Gary Wang <gary.wang@canonical.com>
216 */
217 #ifndef CORE_NET_HTTP_REQUEST_H_
218 #define CORE_NET_HTTP_REQUEST_H_
219@@ -212,6 +213,13 @@
220 /** Invoked for querying user credentials to authenticate proxy accesses. */
221 AuthenicationHandler for_proxy;
222 } authentication_handler;
223+
224+ /** Encapsulates thresholds for minimum transfer speed in [kB/s] for duration seconds. */
225+ struct
226+ {
227+ std::uint64_t limit{1};
228+ std::chrono::seconds duration{std::chrono::seconds{30}};
229+ } speed;
230 };
231
232 Request(const Request&) = delete;
233@@ -248,6 +256,19 @@
234 virtual void async_execute(const Handler& handler) = 0;
235
236 /**
237+ * @brief Pause the request with options for aborting the request.
238+ * The request will be aborted if transfer speed falls below \a limit in [bytes/second] for \a time seconds.
239+ * @throw core::net::http::Error in case of http-related errors.
240+ */
241+ virtual void pause() = 0;
242+
243+ /**
244+ * @brief Resume the request
245+ * @throw core::net::http::Error in case of http-related errors.
246+ */
247+ virtual void resume() = 0;
248+
249+ /**
250 * @brief Returns the input string in URL-escaped format.
251 * @param s The string to be URL escaped.
252 */
253
254=== modified file 'include/core/net/http/streaming_client.h'
255--- include/core/net/http/streaming_client.h 2015-03-23 16:09:05 +0000
256+++ include/core/net/http/streaming_client.h 2016-08-02 06:33:04 +0000
257@@ -14,6 +14,7 @@
258 * along with this program. If not, see <http://www.gnu.org/licenses/>.
259 *
260 * Authored by: Thomas Voß <thomas.voss@canonical.com>
261+ * Gary Wang <gary.wang@canonical.com>
262 */
263 #ifndef CORE_NET_HTTP_STREAMING_CLIENT_H_
264 #define CORE_NET_HTTP_STREAMING_CLIENT_H_
265@@ -61,6 +62,17 @@
266 virtual std::shared_ptr<StreamingRequest> streaming_put(const Request::Configuration& configuration, std::istream& payload, std::size_t size) = 0;
267
268 /**
269+ * @brief streaming_put is a convenience method for issuing a PUT request for the given URI.
270+ * @throw Errors::HttpMethodNotSupported if the underlying implementation does not support the provided HTTP method.
271+ * @param configuration The configuration to issue a get request for.
272+ * @param readdata_callback The callback function to read data in order to send it to the peer.
273+ * The data area pointed at by the pointer \a dest should be filled up with at most \a buf_size number of bytes.
274+ * @param size Size of the payload data in bytes.
275+ * @return An executable instance of class Request.
276+ */
277+ virtual std::shared_ptr<StreamingRequest> streaming_put(const Request::Configuration& configuration, std::function<size_t(void *dest, std::size_t buf_size)> readdata_callback, std::size_t size) = 0;
278+
279+ /**
280 * @brief streaming_post is a convenience method for issuing a POST request for the given URI.
281 * @throw Errors::HttpMethodNotSupported if the underlying implementation does not support the provided HTTP method.
282 * @param configuration The configuration to issue a get request for.
283@@ -69,8 +81,29 @@
284 * @return An executable instance of class Request.
285 */
286 virtual std::shared_ptr<StreamingRequest> streaming_post(const Request::Configuration& configuration, const std::string& payload, const std::string& type) = 0;
287+
288+ /**
289+ * @brief streaming_post is a convenience method for issuing a POST request for the given URI.
290+ * @throw Errors::HttpMethodNotSupported if the underlying implementation does not support the pro vided HTTP method.
291+ * @param configuration The configuration to issue a get request for.
292+ * @param payload The data to be transmitted as part of the POST request.
293+ * @param size Size of the payload data in bytes.
294+ * @return An executable instance of class Request.
295+ */
296+ virtual std::shared_ptr<StreamingRequest> streaming_post(const Request::Configuration& configuration, std::istream& payload, std::size_t size) = 0;
297
298 /**
299+ * @brief streaming_post is a convenience method for issuing a POST request for the given URI.
300+ * @throw Errors::HttpMethodNotSupported if the underlying implementation does not support the pro vided HTTP method.
301+ * @param configuration The configuration to issue a get request for.
302+ * @param readdata_callback The callback function to read data in order to send it to the peer.
303+ * The data area pointed at by the pointer \a dest should be filled up with at most \a buf_size number of bytes.
304+ * @param size Size of the payload data in bytes.
305+ * @return An executable instance of class Request.
306+ */
307+ virtual std::shared_ptr<StreamingRequest> streaming_post(const Request::Configuration& configuration, std::function<size_t(void *dest, std::size_t buf_size)> readdata_callback, std::size_t size) = 0;
308+
309+ /**
310 * @brief streaming_post_form is a convenience method for issuing a POST request for the given URI, with url-encoded payload.
311 * @throw Errors::HttpMethodNotSupported if the underlying implementation does not support the provided HTTP method.
312 * @param configuration The configuration to issue a get request for.
313@@ -78,6 +111,15 @@
314 * @return An executable instance of class Request.
315 */
316 virtual std::shared_ptr<StreamingRequest> streaming_post_form(const Request::Configuration& configuration, const std::map<std::string, std::string>& values) = 0;
317+
318+
319+ /**
320+ * @brief streaming_del is a convenience method for issuing a DELETE request for the given URI.
321+ * @throw Errors::HttpMethodNotSupported if the underlying implementation does not support the provided HTTP method.
322+ * @param configuration The configuration to issue a del request for.
323+ * @return An executable instance of class Request.
324+ */
325+ virtual std::shared_ptr<StreamingRequest> streaming_del(const Request::Configuration& configuration) = 0;
326 };
327
328 /** @brief Dispatches to the default implementation and returns a streaming client instance. */
329
330=== modified file 'include/core/net/http/streaming_request.h'
331--- include/core/net/http/streaming_request.h 2015-03-23 16:09:05 +0000
332+++ include/core/net/http/streaming_request.h 2016-08-02 06:33:04 +0000
333@@ -14,6 +14,7 @@
334 * along with this program. If not, see <http://www.gnu.org/licenses/>.
335 *
336 * Authored by: Thomas Voß <thomas.voss@canonical.com>
337+ * Gary Wang <gary.wang@canonical.com>
338 */
339 #ifndef CORE_NET_HTTP_STREAMING_REQUEST_H_
340 #define CORE_NET_HTTP_STREAMING_REQUEST_H_
341
342=== modified file 'src/core/net/http/impl/curl/client.cpp'
343--- src/core/net/http/impl/curl/client.cpp 2015-03-23 16:09:05 +0000
344+++ src/core/net/http/impl/curl/client.cpp 2016-08-02 06:33:04 +0000
345@@ -14,6 +14,7 @@
346 * along with this program. If not, see <http://www.gnu.org/licenses/>.
347 *
348 * Authored by: Thomas Voß <thomas.voss@canonical.com>
349+ * Gary Wang <gary.wang@canonical.com>
350 */
351
352 #include "client.h"
353@@ -35,6 +36,11 @@
354 namespace
355 {
356 const std::string BASE64_PADDING[] = { "", "==", "=" };
357+template<typename To, typename From>
358+To cast_without_overflow(From from)
359+{
360+ return static_cast<To>(std::min<From>(std::numeric_limits<To>::max(), from));
361+}
362 }
363
364 http::impl::curl::Client::Client()
365@@ -120,6 +126,8 @@
366 configuration.ssl.verify_host ? ::curl::easy::enable_ssl_host_verification : ::curl::easy::disable);
367 handle.set_option(::curl::Option::ssl_verify_peer,
368 configuration.ssl.verify_peer ? ::curl::easy::enable : ::curl::easy::disable);
369+ handle.set_option(::curl::Option::low_speed_limit, cast_without_overflow<long>(configuration.speed.limit));
370+ handle.set_option(::curl::Option::low_speed_time, cast_without_overflow<long>(configuration.speed.duration.count()));
371
372 if (configuration.authentication_handler.for_http)
373 {
374@@ -141,6 +149,8 @@
375 configuration.ssl.verify_host ? ::curl::easy::enable_ssl_host_verification : ::curl::easy::disable);
376 handle.set_option(::curl::Option::ssl_verify_peer,
377 configuration.ssl.verify_peer ? ::curl::easy::enable : ::curl::easy::disable);
378+ handle.set_option(::curl::Option::low_speed_limit, cast_without_overflow<long>(configuration.speed.limit));
379+ handle.set_option(::curl::Option::low_speed_time, cast_without_overflow<long>(configuration.speed.duration.count()));
380
381 if (configuration.authentication_handler.for_http)
382 {
383@@ -166,6 +176,86 @@
384 configuration.ssl.verify_host ? ::curl::easy::enable_ssl_host_verification : ::curl::easy::disable);
385 handle.set_option(::curl::Option::ssl_verify_peer,
386 configuration.ssl.verify_peer ? ::curl::easy::enable : ::curl::easy::disable);
387+ handle.set_option(::curl::Option::low_speed_limit, cast_without_overflow<long>(configuration.speed.limit));
388+ handle.set_option(::curl::Option::low_speed_time, cast_without_overflow<long>(configuration.speed.duration.count()));
389+
390+ if (configuration.authentication_handler.for_http)
391+ {
392+ auto credentials = configuration.authentication_handler.for_http(configuration.uri);
393+ handle.http_credentials(credentials.username, credentials.password);
394+ }
395+
396+ return std::shared_ptr<http::impl::curl::Request>{new http::impl::curl::Request{multi, handle}};
397+}
398+
399+std::shared_ptr<http::impl::curl::Request> http::impl::curl::Client::post_impl(
400+ const Request::Configuration& configuration,
401+ std::istream& payload,
402+ std::size_t size)
403+{
404+ ::curl::easy::Handle handle;
405+ handle.method(http::Method::post)
406+ .url(configuration.uri.c_str())
407+ .header(configuration.header)
408+ .on_read_data([&payload, size](void* dest, std::size_t in_size, std::size_t nmemb)
409+ {
410+ //use internal buffer size(in_size *nmemb) instread of size passed by parameter
411+ //to avoid client crashing when sending large chuck of data via POST method
412+ auto result = payload.readsome(static_cast<char *>(dest), in_size * nmemb);
413+ return result;
414+ }, size);
415+
416+
417+ handle.set_option(::curl::Option::post_field_size, size);
418+ handle.set_option(::curl::Option::ssl_verify_host,
419+ configuration.ssl.verify_host ? ::curl::easy::enable_ssl_host_verification : ::curl::easy::disable);
420+ handle.set_option(::curl::Option::ssl_verify_peer,
421+ configuration.ssl.verify_peer ? ::curl::easy::enable : ::curl::easy::disable);
422+ handle.set_option(::curl::Option::low_speed_limit, cast_without_overflow<long>(configuration.speed.limit));
423+ handle.set_option(::curl::Option::low_speed_time, cast_without_overflow<long>(configuration.speed.duration.count()));
424+
425+ if (configuration.authentication_handler.for_http)
426+ {
427+ auto credentials = configuration.authentication_handler.for_http(configuration.uri);
428+ handle.http_credentials(credentials.username, credentials.password);
429+ }
430+
431+ return std::shared_ptr<http::impl::curl::Request>{new http::impl::curl::Request{multi, handle}};
432+}
433+
434+std::shared_ptr<http::impl::curl::Request> http::impl::curl::Client::post_impl(
435+ const Request::Configuration& configuration,
436+ std::function<size_t(void *dest, std::size_t buf_size)> readdata_callback,
437+ std::size_t size)
438+{
439+ ::curl::easy::Handle handle;
440+ handle.method(http::Method::post)
441+ .url(configuration.uri.c_str())
442+ .header(configuration.header)
443+ .on_read_data([readdata_callback, size](void* dest, std::size_t in_size, std::size_t nmemb)
444+ {
445+ if(readdata_callback) {
446+ try
447+ {
448+ return readdata_callback(dest, in_size * nmemb);
449+ } catch (...)
450+ {
451+ //Just ignoring errors here.
452+ }
453+ }
454+
455+ //stop the current operation immediately
456+ return (size_t)::curl::Code::no_readfunc_abort;
457+ }, size);
458+
459+
460+ handle.set_option(::curl::Option::post_field_size, size);
461+ handle.set_option(::curl::Option::ssl_verify_host,
462+ configuration.ssl.verify_host ? ::curl::easy::enable_ssl_host_verification : ::curl::easy::disable);
463+ handle.set_option(::curl::Option::ssl_verify_peer,
464+ configuration.ssl.verify_peer ? ::curl::easy::enable : ::curl::easy::disable);
465+ handle.set_option(::curl::Option::low_speed_limit, cast_without_overflow<long>(configuration.speed.limit));
466+ handle.set_option(::curl::Option::low_speed_time, cast_without_overflow<long>(configuration.speed.duration.count()));
467
468 if (configuration.authentication_handler.for_http)
469 {
470@@ -185,9 +275,11 @@
471 handle.method(http::Method::put)
472 .url(configuration.uri.c_str())
473 .header(configuration.header)
474- .on_read_data([&payload, size](void* dest, std::size_t /*in_size*/, std::size_t /*nmemb*/)
475+ .on_read_data([&payload, size](void* dest, std::size_t in_size, std::size_t nmemb)
476 {
477- auto result = payload.readsome(static_cast<char*>(dest), size);
478+ //use internal buffer size(in_size *nmemb) instread of size passed by parameter
479+ //to avoid client crashing when sending large chuck of data via PUT method
480+ auto result = payload.readsome(static_cast<char*>(dest), in_size * nmemb);
481 return result;
482 }, size);
483
484@@ -195,6 +287,72 @@
485 configuration.ssl.verify_host ? ::curl::easy::enable_ssl_host_verification : ::curl::easy::disable);
486 handle.set_option(::curl::Option::ssl_verify_peer,
487 configuration.ssl.verify_peer ? ::curl::easy::enable : ::curl::easy::disable);
488+ handle.set_option(::curl::Option::low_speed_limit, cast_without_overflow<long>(configuration.speed.limit));
489+ handle.set_option(::curl::Option::low_speed_time, cast_without_overflow<long>(configuration.speed.duration.count()));
490+
491+ if (configuration.authentication_handler.for_http)
492+ {
493+ auto credentials = configuration.authentication_handler.for_http(configuration.uri);
494+ handle.http_credentials(credentials.username, credentials.password);
495+ }
496+
497+ return std::shared_ptr<http::impl::curl::Request>{new http::impl::curl::Request{multi, handle}};
498+}
499+
500+std::shared_ptr<http::impl::curl::Request> http::impl::curl::Client::put_impl(
501+ const Request::Configuration& configuration,
502+ std::function<size_t(void *dest, std::size_t buf_size)> readdata_callback,
503+ std::size_t size)
504+{
505+ ::curl::easy::Handle handle;
506+ handle.method(http::Method::put)
507+ .url(configuration.uri.c_str())
508+ .header(configuration.header)
509+ .on_read_data([readdata_callback, size](void* dest, std::size_t in_size, std::size_t nmemb)
510+ {
511+ if(readdata_callback) {
512+ try
513+ {
514+ return readdata_callback(dest, in_size * nmemb);
515+ } catch (...)
516+ {
517+ //Just ignoring errors here.
518+ }
519+ }
520+
521+ //stop the current operation immediately
522+ return (size_t)::curl::Code::no_readfunc_abort;
523+ }, size);
524+
525+ handle.set_option(::curl::Option::ssl_verify_host,
526+ configuration.ssl.verify_host ? ::curl::easy::enable_ssl_host_verification : ::curl::easy::disable);
527+ handle.set_option(::curl::Option::ssl_verify_peer,
528+ configuration.ssl.verify_peer ? ::curl::easy::enable : ::curl::easy::disable);
529+ handle.set_option(::curl::Option::low_speed_limit, cast_without_overflow<long>(configuration.speed.limit));
530+ handle.set_option(::curl::Option::low_speed_time, cast_without_overflow<long>(configuration.speed.duration.count()));
531+
532+ if (configuration.authentication_handler.for_http)
533+ {
534+ auto credentials = configuration.authentication_handler.for_http(configuration.uri);
535+ handle.http_credentials(credentials.username, credentials.password);
536+ }
537+
538+ return std::shared_ptr<http::impl::curl::Request>{new http::impl::curl::Request{multi, handle}};
539+}
540+
541+std::shared_ptr<http::impl::curl::Request> http::impl::curl::Client::del_impl(const http::Request::Configuration& configuration)
542+{
543+ ::curl::easy::Handle handle;
544+ handle.method(http::Method::del)
545+ .url(configuration.uri.c_str())
546+ .header(configuration.header);
547+
548+ handle.set_option(::curl::Option::ssl_verify_host,
549+ configuration.ssl.verify_host ? ::curl::easy::enable_ssl_host_verification : ::curl::easy::disable);
550+ handle.set_option(::curl::Option::ssl_verify_peer,
551+ configuration.ssl.verify_peer ? ::curl::easy::enable : ::curl::easy::disable);
552+ handle.set_option(::curl::Option::low_speed_limit, cast_without_overflow<long>(configuration.speed.limit));
553+ handle.set_option(::curl::Option::low_speed_time, cast_without_overflow<long>(configuration.speed.duration.count()));
554
555 if (configuration.authentication_handler.for_http)
556 {
557@@ -220,11 +378,26 @@
558 return put_impl(configuration, payload, size);
559 }
560
561+std::shared_ptr<http::StreamingRequest> http::impl::curl::Client::streaming_put(const http::Request::Configuration& configuration, std::function<size_t(void *dest, std::size_t buf_size)> readdata_callback, std::size_t size)
562+{
563+ return put_impl(configuration, readdata_callback, size);
564+}
565+
566 std::shared_ptr<http::StreamingRequest> http::impl::curl::Client::streaming_post(const http::Request::Configuration& configuration, const std::string& payload, const std::string& type)
567 {
568 return post_impl(configuration, payload, type);
569 }
570
571+std::shared_ptr<http::StreamingRequest> http::impl::curl::Client::streaming_post(const http::Request::Configuration& configuration, std::istream& payload, std::size_t size)
572+{
573+ return post_impl(configuration, payload, size);
574+}
575+
576+std::shared_ptr<http::StreamingRequest> http::impl::curl::Client::streaming_post(const http::Request::Configuration& configuration, std::function<size_t(void *dest, std::size_t buf_size)> readdata_callback, std::size_t size)
577+{
578+ return post_impl(configuration, readdata_callback, size);
579+}
580+
581 std::shared_ptr<http::StreamingRequest> http::impl::curl::Client::streaming_post_form(const http::Request::Configuration& configuration, const std::map<std::string, std::string>& values)
582 {
583 std::stringstream ss;
584@@ -239,6 +412,11 @@
585 return post_impl(configuration, ss.str(), http::ContentType::x_www_form_urlencoded);
586 }
587
588+std::shared_ptr<http::StreamingRequest> http::impl::curl::Client::streaming_del(const http::Request::Configuration& configuration)
589+{
590+ return del_impl(configuration);
591+}
592+
593 std::shared_ptr<http::Request> http::impl::curl::Client::head(const http::Request::Configuration& configuration)
594 {
595 return head_impl(configuration);
596@@ -257,6 +435,19 @@
597 return post_impl(configuration, payload, ct);
598 }
599
600+std::shared_ptr<http::Request> http::impl::curl::Client::post(
601+ const Request::Configuration& configuration,
602+ std::istream& payload,
603+ std::size_t size)
604+{
605+ return post_impl(configuration, payload, size);
606+}
607+
608+std::shared_ptr<http::Request> http::impl::curl::Client::del(const Request::Configuration& configuration)
609+{
610+ return del_impl(configuration);
611+}
612+
613 std::shared_ptr<http::Request> http::impl::curl::Client::put(
614 const Request::Configuration& configuration,
615 std::istream& payload,
616
617=== modified file 'src/core/net/http/impl/curl/client.h'
618--- src/core/net/http/impl/curl/client.h 2015-03-24 11:56:02 +0000
619+++ src/core/net/http/impl/curl/client.h 2016-08-02 06:33:04 +0000
620@@ -14,6 +14,7 @@
621 * along with this program. If not, see <http://www.gnu.org/licenses/>.
622 *
623 * Authored by: Thomas Voß <thomas.voss@canonical.com>
624+ * Gary Wang <gary.wang@canonical.com>
625 */
626 #ifndef CORE_NET_HTTP_IMPL_CURL_CLIENT_H_
627 #define CORE_NET_HTTP_IMPL_CURL_CLIENT_H_
628@@ -55,19 +56,29 @@
629 std::shared_ptr<http::Request> get(const Request::Configuration& configuration) override;
630 std::shared_ptr<http::Request> head(const Request::Configuration& configuration) override;
631 std::shared_ptr<http::Request> post(const Request::Configuration& configuration, const std::string&, const std::string&) override;
632+ std::shared_ptr<http::Request> post(const Request::Configuration& configuration, std::istream& payload, std::size_t size) override;
633 std::shared_ptr<http::Request> put(const Request::Configuration& configuration, std::istream& payload, std::size_t size) override;
634+ std::shared_ptr<http::Request> del(const Request::Configuration& configuration) override;
635
636 std::shared_ptr<http::StreamingRequest> streaming_get(const Request::Configuration& configuration) override;
637 std::shared_ptr<http::StreamingRequest> streaming_head(const Request::Configuration& configuration) override;
638 std::shared_ptr<http::StreamingRequest> streaming_put(const Request::Configuration& configuration, std::istream& payload, std::size_t size) override;
639+ std::shared_ptr<http::StreamingRequest> streaming_put(const Request::Configuration& configuration, std::function<size_t(void *dest, std::size_t buf_size)> readdata_callback, std::size_t size) override;
640 std::shared_ptr<http::StreamingRequest> streaming_post(const Request::Configuration& configuration, const std::string& payload, const std::string& type) override;
641+ std::shared_ptr<http::StreamingRequest> streaming_post(const Request::Configuration& configuration, std::istream& payload, std::size_t size) override;
642+ std::shared_ptr<http::StreamingRequest> streaming_post(const Request::Configuration& configuration, std::function<size_t(void *dest, std::size_t buf_size)> readdata_callback, std::size_t size) override;
643 std::shared_ptr<http::StreamingRequest> streaming_post_form(const Request::Configuration& configuration, const std::map<std::string, std::string>& values) override;
644+ std::shared_ptr<http::StreamingRequest> streaming_del(const Request::Configuration& configuration) override;
645
646 private:
647 std::shared_ptr<curl::Request> get_impl(const Request::Configuration& configuration);
648 std::shared_ptr<curl::Request> head_impl(const Request::Configuration& configuration);
649 std::shared_ptr<curl::Request> post_impl(const Request::Configuration& configuration, const std::string&, const std::string&);
650+ std::shared_ptr<curl::Request> post_impl(const Request::Configuration& configuration, std::istream& payload, std::size_t size);
651+ std::shared_ptr<curl::Request> post_impl(const Request::Configuration& configuration, std::function<size_t(void *dest, std::size_t buf_size)> readdata_callback, std::size_t size);
652 std::shared_ptr<curl::Request> put_impl(const Request::Configuration& configuration, std::istream& payload, std::size_t size);
653+ std::shared_ptr<curl::Request> put_impl(const Request::Configuration& configuration, std::function<size_t(void *dest, std::size_t buf_size)> readdata_callback, std::size_t size);
654+ std::shared_ptr<curl::Request> del_impl(const Request::Configuration& configuration);
655
656 ::curl::multi::Handle multi;
657 };
658
659=== modified file 'src/core/net/http/impl/curl/easy.cpp'
660--- src/core/net/http/impl/curl/easy.cpp 2015-03-03 11:12:12 +0000
661+++ src/core/net/http/impl/curl/easy.cpp 2016-08-02 06:33:04 +0000
662@@ -14,6 +14,7 @@
663 * along with this program. If not, see <http://www.gnu.org/licenses/>.
664 *
665 * Authored by: Thomas Voß <thomas.voss@canonical.com>
666+ * Gary Wang <gary.wang@canonical.com>
667 */
668
669 #include "easy.h"
670@@ -96,6 +97,11 @@
671 return static_cast<curl::Code>(curl_easy_perform(handle));
672 }
673
674+::curl::Code easy::native::pause(easy::native::Handle handle, int bitmask)
675+{
676+ return static_cast<curl::Code>(curl_easy_pause(handle, bitmask));
677+}
678+
679 std::string easy::native::escape(easy::native::Handle handle, const std::string& in)
680 {
681 auto escaped = curl_easy_escape(handle, in.c_str(), in.size());
682@@ -355,6 +361,8 @@
683 {
684 if (!d) throw easy::Handle::HandleHasBeenAbandoned{};
685
686+ static constexpr const char* http_delete = "DELETE";
687+
688 switch(method)
689 {
690 case core::net::http::Method::get:
691@@ -371,6 +379,9 @@
692 case core::net::http::Method::put:
693 set_option(Option::http_put, enable);
694 break;
695+ case core::net::http::Method::del:
696+ set_option(Option::customrequest, http_delete);
697+ break;
698 default: throw core::net::http::Client::Errors::HttpMethodNotSupported{method, CORE_FROM_HERE()};
699 }
700
701@@ -431,6 +442,18 @@
702 throw_if_not<curl::Code::ok>(easy::native::perform(native()), [this]() { return std::string{d->error};});
703 }
704
705+void easy::Handle::pause()
706+{
707+ if (!d) throw easy::Handle::HandleHasBeenAbandoned{};
708+ throw_if_not<curl::Code::ok>(easy::native::pause(native(), CURLPAUSE_ALL), [this]() { return std::string{d->error};});
709+}
710+
711+void easy::Handle::resume()
712+{
713+ if (!d) throw easy::Handle::HandleHasBeenAbandoned{};
714+ throw_if_not<curl::Code::ok>(easy::native::pause(native(), CURLPAUSE_CONT), [this]() { return std::string{d->error};});
715+}
716+
717 // URL escapes the given input string.
718 std::string easy::Handle::escape(const std::string& in)
719 {
720
721=== modified file 'src/core/net/http/impl/curl/easy.h'
722--- src/core/net/http/impl/curl/easy.h 2015-03-03 11:12:12 +0000
723+++ src/core/net/http/impl/curl/easy.h 2016-08-02 06:33:04 +0000
724@@ -14,6 +14,7 @@
725 * along with this program. If not, see <http://www.gnu.org/licenses/>.
726 *
727 * Authored by: Thomas Voß <thomas.voss@canonical.com>
728+ * Gary Wang <gary.wang@canonical.com>
729 */
730 #ifndef CORE_NET_HTTP_IMPL_CURL_EASY_H_
731 #define CORE_NET_HTTP_IMPL_CURL_EASY_H_
732@@ -88,7 +89,8 @@
733 ssl_crl_bad_file = CURLE_SSL_CRL_BADFILE,
734 ssl_issuer_error = CURLE_SSL_ISSUER_ERROR,
735 chunk_failed = CURLE_CHUNK_FAILED,
736- no_connection_available = CURLE_NO_CONNECTION_AVAILABLE
737+ no_connection_available = CURLE_NO_CONNECTION_AVAILABLE,
738+ no_readfunc_abort = CURL_READFUNC_ABORT
739 };
740
741 std::ostream& operator<<(std::ostream& out, Code code);
742@@ -136,7 +138,10 @@
743 timeout_ms = CURLOPT_TIMEOUT_MS,
744 ssl_engine_default = CURLOPT_SSLENGINE_DEFAULT,
745 ssl_verify_peer = CURLOPT_SSL_VERIFYPEER,
746- ssl_verify_host = CURLOPT_SSL_VERIFYHOST
747+ ssl_verify_host = CURLOPT_SSL_VERIFYHOST,
748+ customrequest = CURLOPT_CUSTOMREQUEST,
749+ low_speed_limit = CURLOPT_LOW_SPEED_LIMIT,
750+ low_speed_time = CURLOPT_LOW_SPEED_TIME
751 };
752
753 namespace native
754@@ -209,6 +214,9 @@
755 // Executes the operation configured on the handle.
756 ::curl::Code perform(Handle handle);
757
758+// Executes pause operation on the handle.
759+::curl::Code pause(Handle handle, int bitmask);
760+
761 // URL escapes the given input string.
762 std::string escape(Handle handle, const std::string& in);
763
764@@ -339,6 +347,12 @@
765 // Executes the operation associated with this handle.
766 void perform();
767
768+ // Executes pause operation associated with this handle.
769+ void pause();
770+
771+ // Executes resume operation associated with this handle.
772+ void resume();
773+
774 // URL escapes the given input string.
775 std::string escape(const std::string& in);
776
777
778=== modified file 'src/core/net/http/impl/curl/multi.cpp'
779--- src/core/net/http/impl/curl/multi.cpp 2015-02-25 10:26:38 +0000
780+++ src/core/net/http/impl/curl/multi.cpp 2016-08-02 06:33:04 +0000
781@@ -285,6 +285,11 @@
782 return result;
783 }
784
785+void multi::Handle::dispatch(const std::function<void ()> &task)
786+{
787+ d->dispatcher.post(task);
788+}
789+
790 void multi::Handle::run()
791 {
792 d->dispatcher.run();
793
794=== modified file 'src/core/net/http/impl/curl/multi.h'
795--- src/core/net/http/impl/curl/multi.h 2014-03-13 20:50:36 +0000
796+++ src/core/net/http/impl/curl/multi.h 2016-08-02 06:33:04 +0000
797@@ -140,6 +140,9 @@
798 // Queries statistics about the timing information of the last transfers.
799 core::net::http::Client::Timings timings();
800
801+ // Dispatch dispatches task on the underlying reactor.
802+ void dispatch(const std::function<void()>& task);
803+
804 // Executes the underlying dispatcher executing the curl multi instance.
805 // Can be called multiple times for thread-pool use-cases.
806 void run();
807
808=== modified file 'src/core/net/http/impl/curl/request.h'
809--- src/core/net/http/impl/curl/request.h 2015-03-23 16:09:05 +0000
810+++ src/core/net/http/impl/curl/request.h 2016-08-02 06:33:04 +0000
811@@ -14,6 +14,7 @@
812 * along with this program. If not, see <http://www.gnu.org/licenses/>.
813 *
814 * Authored by: Thomas Voß <thomas.voss@canonical.com>
815+ * Gary Wang <gary.wang@canonical.com>
816 */
817 #ifndef CORE_NET_HTTP_IMPL_CURL_REQUEST_H_
818 #define CORE_NET_HTTP_IMPL_CURL_REQUEST_H_
819@@ -241,7 +242,6 @@
820 case Request::Progress::Next::abort_operation: result = 1; break;
821 case Request::Progress::Next::continue_operation: result = 0; break;
822 }
823-
824 return result;
825 });
826 }
827@@ -273,6 +273,34 @@
828 multi.add(easy);
829 }
830
831+ void pause()
832+ {
833+ auto copy = easy;
834+ multi.dispatch([copy]() mutable
835+ {
836+ try
837+ {
838+ copy.pause();
839+ }
840+ catch(...) {}
841+ });
842+
843+ }
844+
845+ void resume()
846+ {
847+ auto copy = easy;
848+ multi.dispatch([copy]() mutable
849+ {
850+ try
851+ {
852+ copy.resume();
853+ }
854+ catch(...) {}
855+
856+ });
857+ }
858+
859 std::string url_escape(const std::string& s)
860 {
861 return easy.escape(s);
862@@ -282,6 +310,7 @@
863 {
864 return easy.unescape(s);
865 }
866+
867 private:
868 std::atomic<core::net::http::Request::State> atomic_state;
869 ::curl::multi::Handle multi;
870
871=== modified file 'tests/http_client_load_test.cpp'
872--- tests/http_client_load_test.cpp 2014-05-14 10:07:20 +0000
873+++ tests/http_client_load_test.cpp 2016-08-02 06:33:04 +0000
874@@ -117,7 +117,7 @@
875 return true;
876 }
877
878-static const bool is_initialized = init();
879+static const bool is_initialized __attribute__((used)) = init();
880 }
881
882 TEST_F(HttpClientLoadTest, async_head_request_for_existing_resource_succeeds)
883
884=== modified file 'tests/http_client_test.cpp'
885--- tests/http_client_test.cpp 2014-11-13 12:57:24 +0000
886+++ tests/http_client_test.cpp 2016-08-02 06:33:04 +0000
887@@ -14,6 +14,7 @@
888 * along with this program. If not, see <http://www.gnu.org/licenses/>.
889 *
890 * Authored by: Thomas Voß <thomas.voss@canonical.com>
891+ * Gary Wang <gary.wang@canonical.com>
892 */
893
894 #include <core/net/error.h>
895@@ -30,6 +31,7 @@
896 #include <json/json.h>
897
898 #include <future>
899+#include <fstream>
900
901 namespace http = core::net::http;
902 namespace json = Json;
903@@ -53,7 +55,7 @@
904 return true;
905 }
906
907-static const bool is_initialized = init();
908+static const bool is_initialized __attribute__((used)) = init();
909 }
910
911 TEST(HttpClient, uri_to_string)
912@@ -456,6 +458,33 @@
913 EXPECT_EQ("test", root["form"]["test"].asString());
914 }
915
916+TEST(HttpClient, post_request_for_file_with_large_chunk_succeeds)
917+{
918+ auto client = http::make_client();
919+ auto url = std::string(httpbin::host) + httpbin::resources::post();
920+
921+ // create temp file with large chunk
922+ const std::size_t size = 1024*1024;
923+ std::ofstream ofs("tmp.dat", std::ios::binary | std::ios::out);
924+ ofs.seekp(size);
925+ ofs.write("", 1);
926+ ofs.close();
927+
928+ std::ifstream payload("tmp.dat");
929+ auto request = client->post(http::Request::Configuration::from_uri_as_string(url),
930+ payload,
931+ size);
932+
933+ json::Value root;
934+ json::Reader reader;
935+
936+ auto response = request->execute(default_progress_reporter);
937+
938+ EXPECT_EQ(core::net::http::Status::ok, response.status);
939+ EXPECT_TRUE(reader.parse(response.body, root));
940+ EXPECT_EQ(url, root["url"].asString());
941+}
942+
943 TEST(HttpClient, put_request_for_existing_resource_succeeds)
944 {
945 auto client = http::make_client();
946@@ -478,6 +507,50 @@
947 EXPECT_EQ(payload.str(), root["data"].asString());
948 }
949
950+TEST(HttpClient, put_request_for_file_with_large_chunk_succeeds)
951+{
952+ auto client = http::make_client();
953+ auto url = std::string(httpbin::host) + httpbin::resources::put();
954+
955+ // create temp file with large chunk
956+ const std::size_t size = 1024*1024;
957+ std::ofstream ofs("tmp.dat", std::ios::binary | std::ios::out);
958+ ofs.seekp(size);
959+ ofs.write("", 1);
960+ ofs.close();
961+
962+ std::ifstream payload("tmp.dat");
963+ auto request = client->put(http::Request::Configuration::from_uri_as_string(url),
964+ payload,
965+ size);
966+
967+ json::Value root;
968+ json::Reader reader;
969+
970+ auto response = request->execute(default_progress_reporter);
971+
972+ EXPECT_EQ(core::net::http::Status::ok, response.status);
973+ EXPECT_TRUE(reader.parse(response.body, root));
974+ EXPECT_EQ(url, root["url"].asString());
975+}
976+
977+TEST(HttpClient, del_request_for_existing_resource_succeeds)
978+{
979+ auto client = http::make_client();
980+ auto url = std::string(httpbin::host) + httpbin::resources::del();
981+
982+ auto request = client->del(http::Request::Configuration::from_uri_as_string(url));
983+
984+ json::Value root;
985+ json::Reader reader;
986+
987+ auto response = request->execute(default_progress_reporter);
988+
989+ EXPECT_EQ(core::net::http::Status::ok, response.status);
990+ EXPECT_TRUE(reader.parse(response.body, root));
991+ EXPECT_EQ(url, root["url"].asString());
992+}
993+
994 namespace com
995 {
996 namespace mozilla
997
998=== modified file 'tests/http_streaming_client_test.cpp'
999--- tests/http_streaming_client_test.cpp 2015-03-23 16:24:27 +0000
1000+++ tests/http_streaming_client_test.cpp 2016-08-02 06:33:04 +0000
1001@@ -14,6 +14,7 @@
1002 * along with this program. If not, see <http://www.gnu.org/licenses/>.
1003 *
1004 * Authored by: Thomas Voß <thomas.voss@canonical.com>
1005+ * Gary Wang <gary.wang@canonical.com>
1006 */
1007
1008 #include <core/net/error.h>
1009@@ -33,6 +34,10 @@
1010 #include <future>
1011 #include <memory>
1012
1013+#include <fstream>
1014+#include <iomanip>
1015+#include <iostream>
1016+
1017 namespace http = core::net::http;
1018 namespace json = Json;
1019 namespace net = core::net;
1020@@ -64,23 +69,50 @@
1021 MockDataHandler() = default;
1022 };
1023
1024-auto default_progress_reporter = [](const http::Request::Progress& progress)
1025-{
1026- if (progress.download.current > 0. && progress.download.total > 0.)
1027- std::cout << "Download progress: " << progress.download.current / progress.download.total << std::endl;
1028- if (progress.upload.current > 0. && progress.upload.total > 0.)
1029- std::cout << "Upload progress: " << progress.upload.current / progress.upload.total << std::endl;
1030-
1031+class ProgressBar
1032+{
1033+public:
1034+ ProgressBar(std::uint32_t width)
1035+ : width{width}, out{std::cout}
1036+ {
1037+ }
1038+
1039+ ~ProgressBar()
1040+ {
1041+ out << std::endl;
1042+ }
1043+
1044+ void update(double percentage)
1045+ {
1046+ struct CursorState
1047+ {
1048+ CursorState(std::ostream& out) : out{out} { out << "\33[?25l"; }
1049+ ~CursorState() { out << "\33[?25h"; }
1050+ std::ostream& out;
1051+ } cs{out};
1052+
1053+ out << "\r" << "[" << std::setw(width) << std::left << std::setfill(' ') << std::string(percentage * width, '=') << "] " << std::setw(5) << std::fixed << std::setprecision(2) << percentage * 100 << " %" << std::flush;
1054+ }
1055+
1056+private:
1057+ std::uint32_t width;
1058+ std::ostream& out;
1059+};
1060+
1061+auto silent_progress_reporter = [](const http::Request::Progress&)
1062+{
1063 return http::Request::Progress::Next::continue_operation;
1064 };
1065
1066+auto default_progress_reporter = silent_progress_reporter;
1067+
1068 bool init()
1069 {
1070 static httpbin::Instance instance;
1071 return true;
1072 }
1073
1074-static const bool is_initialized = init();
1075+static const bool is_initialized __attribute__((used)) = init();
1076 }
1077
1078 TEST(StreamingStreamingHttpClient, head_request_for_existing_resource_succeeds)
1079@@ -480,6 +512,70 @@
1080 EXPECT_EQ("test", root["form"]["test"].asString());
1081 }
1082
1083+TEST(StreamingHttpClient, post_request_for_file_with_large_chunk_succeeds)
1084+{
1085+ using namespace ::testing;
1086+
1087+ auto client = http::make_streaming_client();
1088+ auto url = std::string(httpbin::host) + httpbin::resources::post();
1089+
1090+ // create temp file with large chunk
1091+ const std::size_t size = 1024*1024;
1092+ std::ofstream ofs("tmp.dat", std::ios::binary | std::ios::out);
1093+ ofs.seekp(size);
1094+ ofs.write("", 1);
1095+ ofs.close();
1096+
1097+ std::ifstream payload("tmp.dat");
1098+ auto request = client->streaming_post(http::Request::Configuration::from_uri_as_string(url),
1099+ payload,
1100+ size);
1101+
1102+ auto dh = MockDataHandler::create(); EXPECT_CALL(*dh, on_new_data(_)).Times(AtLeast(1));
1103+
1104+ auto response = request->execute(default_progress_reporter, dh->to_data_handler());
1105+
1106+ json::Value root;
1107+ json::Reader reader;
1108+
1109+ EXPECT_EQ(core::net::http::Status::ok, response.status);
1110+ EXPECT_TRUE(reader.parse(response.body, root));
1111+ EXPECT_EQ(url, root["url"].asString());
1112+}
1113+
1114+TEST(StreamingHttpClient, post_request_for_file_with_large_chunk_with_read_callback)
1115+{
1116+ using namespace ::testing;
1117+
1118+ auto client = http::make_streaming_client();
1119+ auto url = std::string(httpbin::host) + httpbin::resources::post();
1120+
1121+ // create temp file with large chunk
1122+ const std::size_t size = 1024*1024;
1123+ std::ofstream ofs("tmp.dat", std::ios::binary | std::ios::out);
1124+ ofs.seekp(size);
1125+ ofs.write("", 1);
1126+ ofs.close();
1127+
1128+ std::ifstream payload("tmp.dat");
1129+ auto request = client->streaming_post(http::Request::Configuration::from_uri_as_string(url),
1130+ [&payload](void *dest, size_t buf_size) -> size_t {
1131+ return payload.readsome(static_cast<char *>(dest), buf_size);
1132+ },
1133+ size);
1134+
1135+ auto dh = MockDataHandler::create(); EXPECT_CALL(*dh, on_new_data(_)).Times(AtLeast(1));
1136+
1137+ auto response = request->execute(default_progress_reporter, dh->to_data_handler());
1138+
1139+ json::Value root;
1140+ json::Reader reader;
1141+
1142+ EXPECT_EQ(core::net::http::Status::ok, response.status);
1143+ EXPECT_TRUE(reader.parse(response.body, root));
1144+ EXPECT_EQ(url, root["url"].asString());
1145+}
1146+
1147 TEST(StreamingHttpClient, put_request_for_existing_resource_succeeds)
1148 {
1149 using namespace ::testing;
1150@@ -497,12 +593,157 @@
1151 // Our mocked data handler.
1152 auto dh = MockDataHandler::create(); EXPECT_CALL(*dh, on_new_data(_)).Times(AtLeast(1));
1153
1154+ auto response = request->execute(default_progress_reporter, dh->to_data_handler());
1155+
1156 json::Value root;
1157 json::Reader reader;
1158
1159- auto response = request->execute(default_progress_reporter, dh->to_data_handler());
1160-
1161 EXPECT_EQ(core::net::http::Status::ok, response.status);
1162 EXPECT_TRUE(reader.parse(response.body, root));
1163 EXPECT_EQ(payload.str(), root["data"].asString());
1164 }
1165+
1166+TEST(StreamingHttpClient, put_request_for_file_with_large_chunk_succeeds)
1167+{
1168+ using namespace ::testing;
1169+
1170+ auto client = http::make_streaming_client();
1171+ auto url = std::string(httpbin::host) + httpbin::resources::put();
1172+
1173+ // create temp file with large chunk
1174+ const std::size_t size = 1024*1024;
1175+ std::ofstream ofs("tmp.dat", std::ios::binary | std::ios::out);
1176+ ofs.seekp(size);
1177+ ofs.write("", 1);
1178+ ofs.close();
1179+
1180+ std::ifstream payload("tmp.dat");
1181+ auto request = client->streaming_put(http::Request::Configuration::from_uri_as_string(url),
1182+ payload,
1183+ size);
1184+
1185+ auto dh = MockDataHandler::create(); EXPECT_CALL(*dh, on_new_data(_)).Times(AtLeast(1));
1186+
1187+ auto response = request->execute(default_progress_reporter, dh->to_data_handler());
1188+
1189+ json::Value root;
1190+ json::Reader reader;
1191+
1192+ EXPECT_EQ(core::net::http::Status::ok, response.status);
1193+ EXPECT_TRUE(reader.parse(response.body, root));
1194+ EXPECT_EQ(url, root["url"].asString());
1195+}
1196+
1197+TEST(StreamingHttpClient, put_request_for_file_with_large_chunk_with_read_callback)
1198+{
1199+ using namespace ::testing;
1200+
1201+ auto client = http::make_streaming_client();
1202+ auto url = std::string(httpbin::host) + httpbin::resources::put();
1203+
1204+ // create temp file with large chunk
1205+ const std::size_t size = 1024*1024;
1206+ std::ofstream ofs("tmp.dat", std::ios::binary | std::ios::out);
1207+ ofs.seekp(size);
1208+ ofs.write("", 1);
1209+ ofs.close();
1210+
1211+ std::ifstream payload("tmp.dat");
1212+ auto request = client->streaming_put(http::Request::Configuration::from_uri_as_string(url),
1213+ [&payload](void *dest, size_t buf_size) -> size_t {
1214+ return payload.readsome(static_cast<char *>(dest), buf_size);
1215+ },
1216+ size);
1217+
1218+ auto dh = MockDataHandler::create(); EXPECT_CALL(*dh, on_new_data(_)).Times(AtLeast(1));
1219+
1220+ auto response = request->execute(default_progress_reporter, dh->to_data_handler());
1221+
1222+ json::Value root;
1223+ json::Reader reader;
1224+
1225+ EXPECT_EQ(core::net::http::Status::ok, response.status);
1226+ EXPECT_TRUE(reader.parse(response.body, root));
1227+ EXPECT_EQ(url, root["url"].asString());
1228+}
1229+
1230+TEST(StreamingHttpClient, del_request_for_existing_resource_succeeds)
1231+{
1232+ using namespace ::testing;
1233+
1234+ auto client = http::make_streaming_client();
1235+ auto url = std::string(httpbin::host) + httpbin::resources::del();
1236+
1237+ auto request = client->streaming_del(http::Request::Configuration::from_uri_as_string(url));
1238+
1239+ // Our mocked data handler.
1240+ auto dh = MockDataHandler::create(); EXPECT_CALL(*dh, on_new_data(_)).Times(AtLeast(1));
1241+
1242+ auto response = request->execute(default_progress_reporter, dh->to_data_handler());
1243+
1244+ json::Value root;
1245+ json::Reader reader;
1246+
1247+ EXPECT_EQ(core::net::http::Status::ok, response.status);
1248+ EXPECT_TRUE(reader.parse(response.body, root));
1249+ EXPECT_EQ(url, root["url"].asString());
1250+}
1251+
1252+TEST(StreamingHttpClient, request_can_be_paused_and_resumed)
1253+{
1254+ using namespace ::testing;
1255+ // We obtain a default client instance, dispatching to the default implementation.
1256+ auto client = http::make_streaming_client();
1257+
1258+ // Execute the client
1259+ std::thread worker{[client]() { client->run(); }};
1260+
1261+ auto url = "http://archive.ubuntu.com/ubuntu/dists/xenial/main/installer-amd64/current/images/netboot/ubuntu-installer/amd64/linux";
1262+
1263+ // The client mostly acts as a factory for http requests.
1264+ auto request = client->streaming_get(http::Request::Configuration::from_uri_as_string(url));
1265+
1266+ // Our mocked data handler.
1267+ auto dh = MockDataHandler::create(); EXPECT_CALL(*dh, on_new_data(_)).Times(AtLeast(1));
1268+
1269+ std::promise<core::net::http::Response> promise;
1270+ auto future = promise.get_future();
1271+
1272+ ProgressBar pb{80};
1273+
1274+ // We finally execute the query asynchronously.
1275+ request->async_execute(http::Request::Handler()
1276+ .on_progress([&pb](const http::Request::Progress& progress)
1277+ {
1278+ if (progress.download.current > 0. && progress.download.total > 0.)
1279+ pb.update(progress.download.current / static_cast<double>(progress.download.total));
1280+ return http::Request::Progress::Next::continue_operation;
1281+ })
1282+ .on_response([&](const core::net::http::Response& response)
1283+ {
1284+ promise.set_value(response);
1285+ })
1286+ .on_error([&](const core::net::Error& e)
1287+ {
1288+ promise.set_exception(std::make_exception_ptr(e));
1289+ }),
1290+ dh->to_data_handler());
1291+
1292+ std::this_thread::sleep_for(std::chrono::seconds(2));
1293+
1294+ request->pause();
1295+ request->resume();
1296+
1297+ try
1298+ {
1299+ // This might very well throw.
1300+ EXPECT_EQ(core::net::http::Status::ok, future.get().status);
1301+ } catch (const std::exception& e) { FAIL() << e.what(); }
1302+
1303+ client->stop();
1304+
1305+ // We shut down our worker thread
1306+ if (worker.joinable())
1307+ worker.join();
1308+}
1309+
1310
1311=== modified file 'tests/httpbin.h.in'
1312--- tests/httpbin.h.in 2014-05-22 11:50:55 +0000
1313+++ tests/httpbin.h.in 2016-08-02 06:33:04 +0000
1314@@ -14,6 +14,7 @@
1315 * along with this program. If not, see <http://www.gnu.org/licenses/>.
1316 *
1317 * Authored by: Thomas Voß <thomas.voss@canonical.com>
1318+ * Gary Wang <gary.wang@canonical.com>
1319 */
1320
1321 #ifndef HTTPBIN_H_
1322@@ -44,7 +45,7 @@
1323 core::posix::exec("/usr/bin/python", {"@CMAKE_CURRENT_BINARY_DIR@/httpbin/run.py"}, {}, core::posix::StandardStream::stdout | core::posix::StandardStream::stderr)
1324 }
1325 {
1326- std::this_thread::sleep_for(std::chrono::milliseconds{500});
1327+ std::this_thread::sleep_for(std::chrono::milliseconds{1000});
1328 }
1329
1330 ~Instance()
1331@@ -104,6 +105,11 @@
1332 {
1333 return "/put";
1334 }
1335+/** Returns DELETE data. */
1336+const char* del()
1337+{
1338+ return "/delete";
1339+}
1340 /** Challenges basic authentication. */
1341 const char* basic_auth()
1342 {

Subscribers

People subscribed via source and target branches

to all changes: