Merge lp:~gary-wzl77/net-cpp/bug-fixing-and-features into lp:net-cpp
- bug-fixing-and-features
- Merge into trunk
Status: | Merged | ||||||||
---|---|---|---|---|---|---|---|---|---|
Merged at revision: | 50 | ||||||||
Proposed branch: | lp:~gary-wzl77/net-cpp/bug-fixing-and-features | ||||||||
Merge into: | lp:net-cpp | ||||||||
Diff against target: |
804 lines (+407/-9) 14 files modified
CMakeLists.txt (+2/-2) include/core/net/http/client.h (+19/-0) include/core/net/http/method.h (+3/-1) include/core/net/http/request.h (+21/-0) include/core/net/http/streaming_client.h (+21/-1) include/core/net/http/streaming_request.h (+13/-0) src/core/net/http/impl/curl/client.cpp (+82/-2) src/core/net/http/impl/curl/client.h (+7/-0) src/core/net/http/impl/curl/easy.cpp (+23/-0) src/core/net/http/impl/curl/easy.h (+14/-1) src/core/net/http/impl/curl/request.h (+33/-0) tests/http_client_test.cpp (+73/-0) tests/http_streaming_client_test.cpp (+90/-2) tests/httpbin.h.in (+6/-0) |
||||||||
To merge this branch: | bzr merge lp:~gary-wzl77/net-cpp/bug-fixing-and-features | ||||||||
Related bugs: |
|
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Thomas Voß (community) | Approve | ||
Review via email: mp+291975@code.launchpad.net |
Commit message
1.Fix crash issue when sending large chunk data via PUT method (lp:1570686)
3.Support DELETE method (lp:1570687)
2.Support POST method with istream
4.Support pause and resume mechanism
5.Add some test cases
Description of the change
1.Fix crash issue when sending large chunk data via PUT method (lp:1570686)
3.Support DELETE method (lp:1570687)
2.Support POST method with istream
4.Support pause and resume mechanism
5.Add some test cases
Thomas Voß (thomas-voss) wrote : | # |
Thanks for your changes, the MP looks good to me. I tried to further understand your remarks about only setting abort_request_
Hope that addresses your remark :) Or did I miss something?
Gary.Wang (gary-wzl77) wrote : | # |
Thanks, Please add the following test case in http_streaming_
http://
Then we can see the potential issue we have in this MP after it is run. (I use the async api to reproduce this issue.)
Log:
....
Download progress: 0.0154666
Download progress: 0.0154666
we pause
Download progress: 0.0154666
Download progress: 0.0154666
Download progress: 0.0154666
we resume.
I resume download after the request is paused for 5 secs. But there is no "Download progress" log printing out any more even if I resume it. It causes the curl thread to hang as well. The issue can be reproduces 100%.
However if I explicitly set async_abort_
Log:
Download progress: 0.0077333
Download progress: 0.0077333
Download progress: 0.0077333
we pause
Download progress: 0.0077333
we resume.
Download progress: 0.0077333
Download progress: 0.0077333
Download progress: 0.0077333
Download progress: 0.00883806
Download progress: 0.00883806
So it's better to mention this case in doc somewhere to reminder developer of setting abort_request_
Otherwise developer doesn't figure out why it doesn't work out after resuming from paused state.
But anyway, it works fine if developers don't use pause/resume API.
P.S:
curl utilizes progress callback(
Preview Diff
1 | === modified file 'CMakeLists.txt' |
2 | --- CMakeLists.txt 2015-04-01 06:23:30 +0000 |
3 | +++ CMakeLists.txt 2016-05-23 10:51:40 +0000 |
4 | @@ -26,8 +26,8 @@ |
5 | set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++11 -Werror -Wall -fno-strict-aliasing -fvisibility=hidden -fvisibility-inlines-hidden -pedantic -Wextra") |
6 | set(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} -Wl,--no-undefined") |
7 | |
8 | -set(NET_CPP_VERSION_MAJOR 1) |
9 | -set(NET_CPP_VERSION_MINOR 2) |
10 | +set(NET_CPP_VERSION_MAJOR 2) |
11 | +set(NET_CPP_VERSION_MINOR 0) |
12 | set(NET_CPP_VERSION_PATCH 0) |
13 | |
14 | include(CTest) |
15 | |
16 | === modified file 'include/core/net/http/client.h' |
17 | --- include/core/net/http/client.h 2014-06-10 14:19:14 +0000 |
18 | +++ include/core/net/http/client.h 2016-05-23 10:51:40 +0000 |
19 | @@ -14,6 +14,7 @@ |
20 | * along with this program. If not, see <http://www.gnu.org/licenses/>. |
21 | * |
22 | * Authored by: Thomas Voß <thomas.voss@canonical.com> |
23 | + * Gary Wang <gary.wang@canonical.com> |
24 | */ |
25 | #ifndef CORE_NET_HTTP_CLIENT_H_ |
26 | #define CORE_NET_HTTP_CLIENT_H_ |
27 | @@ -154,6 +155,16 @@ |
28 | */ |
29 | virtual std::shared_ptr<Request> post(const Request::Configuration& configuration, const std::string& payload, const std::string& type) = 0; |
30 | |
31 | + /** |
32 | + * @brief post is a convenience method for issuing a POST request for the given URI. |
33 | + * @throw Errors::HttpMethodNotSupported if the underlying implementation does not support the pro vided HTTP method. |
34 | + * @param configuration The configuration to issue a get request for. |
35 | + * @param payload The data to be transmitted as part of the POST request. |
36 | + * @param size Size of the payload data in bytes. |
37 | + * @return An executable instance of class Request. |
38 | + */ |
39 | + virtual std::shared_ptr<Request> post(const Request::Configuration& configuration, std::istream& payload, std::size_t size) = 0; |
40 | + |
41 | /** |
42 | * @brief post_form is a convenience method for issuing a POST request for the given URI, with url-encoded payload. |
43 | * @throw Errors::HttpMethodNotSupported if the underlying implementation does not support the provided HTTP method. |
44 | @@ -163,6 +174,14 @@ |
45 | */ |
46 | virtual std::shared_ptr<Request> post_form(const Request::Configuration& configuration, const std::map<std::string, std::string>& values); |
47 | |
48 | + /** |
49 | + * @brief del is a convenience method for issueing a DELETE request for the given URI. |
50 | + * @throw Errors::HttpMethodNotSupported if the underlying implementation does not support the provided HTTP method. |
51 | + * @param configuration The configuration to issue a get request for. |
52 | + * @return An executable instance of class Request. |
53 | + */ |
54 | + virtual std::shared_ptr<Request> del(const Request::Configuration& configuration) = 0; |
55 | + |
56 | protected: |
57 | Client() = default; |
58 | }; |
59 | |
60 | === modified file 'include/core/net/http/method.h' |
61 | --- include/core/net/http/method.h 2013-12-04 07:23:43 +0000 |
62 | +++ include/core/net/http/method.h 2016-05-23 10:51:40 +0000 |
63 | @@ -14,6 +14,7 @@ |
64 | * along with this program. If not, see <http://www.gnu.org/licenses/>. |
65 | * |
66 | * Authored by: Thomas Voß <thomas.voss@canonical.com> |
67 | + * Gary Wang <gary.wang@canonical.com> |
68 | */ |
69 | #ifndef CORE_NET_HTTP_METHOD_H_ |
70 | #define CORE_NET_HTTP_METHOD_H_ |
71 | @@ -33,7 +34,8 @@ |
72 | get, |
73 | head, |
74 | post, |
75 | - put |
76 | + put, |
77 | + del |
78 | }; |
79 | } |
80 | } |
81 | |
82 | === modified file 'include/core/net/http/request.h' |
83 | --- include/core/net/http/request.h 2014-05-06 11:05:04 +0000 |
84 | +++ include/core/net/http/request.h 2016-05-23 10:51:40 +0000 |
85 | @@ -14,6 +14,7 @@ |
86 | * along with this program. If not, see <http://www.gnu.org/licenses/>. |
87 | * |
88 | * Authored by: Thomas Voß <thomas.voss@canonical.com> |
89 | + * Gary Wang <gary.wang@canonical.com> |
90 | */ |
91 | #ifndef CORE_NET_HTTP_REQUEST_H_ |
92 | #define CORE_NET_HTTP_REQUEST_H_ |
93 | @@ -248,6 +249,18 @@ |
94 | virtual void async_execute(const Handler& handler) = 0; |
95 | |
96 | /** |
97 | + * @brief Pause the request |
98 | + * @throw core::net::http::Error in case of http-related errors. |
99 | + */ |
100 | + virtual void pause() = 0; |
101 | + |
102 | + /** |
103 | + * @brief Resume the request |
104 | + * @throw core::net::http::Error in case of http-related errors. |
105 | + */ |
106 | + virtual void resume() = 0; |
107 | + |
108 | + /** |
109 | * @brief Returns the input string in URL-escaped format. |
110 | * @param s The string to be URL escaped. |
111 | */ |
112 | @@ -259,6 +272,14 @@ |
113 | */ |
114 | virtual std::string url_unescape(const std::string& s) = 0; |
115 | |
116 | + /** |
117 | + * @brief Sets options for aborting the request. |
118 | + * The request will be aborted if transfer speed belows \a limit bytes per second for \a time seconds |
119 | + * @param limit The transfer speed in seconds. |
120 | + * @param time waiting period(seconds) to abort the request. |
121 | + */ |
122 | + virtual void abort_request_if(std::uint64_t limit, const std::chrono::seconds& time) = 0; |
123 | + |
124 | protected: |
125 | /** @cond */ |
126 | Request() = default; |
127 | |
128 | === modified file 'include/core/net/http/streaming_client.h' |
129 | --- include/core/net/http/streaming_client.h 2015-03-23 16:09:05 +0000 |
130 | +++ include/core/net/http/streaming_client.h 2016-05-23 10:51:40 +0000 |
131 | @@ -14,6 +14,7 @@ |
132 | * along with this program. If not, see <http://www.gnu.org/licenses/>. |
133 | * |
134 | * Authored by: Thomas Voß <thomas.voss@canonical.com> |
135 | + * Gary Wang <gary.wang@canonical.com> |
136 | */ |
137 | #ifndef CORE_NET_HTTP_STREAMING_CLIENT_H_ |
138 | #define CORE_NET_HTTP_STREAMING_CLIENT_H_ |
139 | @@ -69,7 +70,17 @@ |
140 | * @return An executable instance of class Request. |
141 | */ |
142 | virtual std::shared_ptr<StreamingRequest> streaming_post(const Request::Configuration& configuration, const std::string& payload, const std::string& type) = 0; |
143 | - |
144 | + |
145 | + /** |
146 | + * @brief streaming_post is a convenience method for issuing a POST request for the given URI. |
147 | + * @throw Errors::HttpMethodNotSupported if the underlying implementation does not support the pro vided HTTP method. |
148 | + * @param configuration The configuration to issue a get request for. |
149 | + * @param payload The data to be transmitted as part of the POST request. |
150 | + * @param size Size of the payload data in bytes. |
151 | + * @return An executable instance of class Request. |
152 | + */ |
153 | + virtual std::shared_ptr<StreamingRequest> streaming_post(const Request::Configuration& configuration, std::istream& payload, std::size_t size) = 0; |
154 | + |
155 | /** |
156 | * @brief streaming_post_form is a convenience method for issuing a POST request for the given URI, with url-encoded payload. |
157 | * @throw Errors::HttpMethodNotSupported if the underlying implementation does not support the provided HTTP method. |
158 | @@ -78,6 +89,15 @@ |
159 | * @return An executable instance of class Request. |
160 | */ |
161 | virtual std::shared_ptr<StreamingRequest> streaming_post_form(const Request::Configuration& configuration, const std::map<std::string, std::string>& values) = 0; |
162 | + |
163 | + |
164 | + /** |
165 | + * @brief streaming_del is a convenience method for issuing a DELETE request for the given URI. |
166 | + * @throw Errors::HttpMethodNotSupported if the underlying implementation does not support the provided HTTP method. |
167 | + * @param configuration The configuration to issue a del request for. |
168 | + * @return An executable instance of class Request. |
169 | + */ |
170 | + virtual std::shared_ptr<StreamingRequest> streaming_del(const Request::Configuration& configuration) = 0; |
171 | }; |
172 | |
173 | /** @brief Dispatches to the default implementation and returns a streaming client instance. */ |
174 | |
175 | === modified file 'include/core/net/http/streaming_request.h' |
176 | --- include/core/net/http/streaming_request.h 2015-03-23 16:09:05 +0000 |
177 | +++ include/core/net/http/streaming_request.h 2016-05-23 10:51:40 +0000 |
178 | @@ -14,6 +14,7 @@ |
179 | * along with this program. If not, see <http://www.gnu.org/licenses/>. |
180 | * |
181 | * Authored by: Thomas Voß <thomas.voss@canonical.com> |
182 | + * Gary Wang <gary.wang@canonical.com> |
183 | */ |
184 | #ifndef CORE_NET_HTTP_STREAMING_REQUEST_H_ |
185 | #define CORE_NET_HTTP_STREAMING_REQUEST_H_ |
186 | @@ -52,6 +53,18 @@ |
187 | * @return The response to the request. |
188 | */ |
189 | virtual void async_execute(const Handler& handler, const DataHandler& dh) = 0; |
190 | + |
191 | + /** |
192 | + * @brief Pause the request |
193 | + * @throw core::net::http::Error in case of http-related errors. |
194 | + */ |
195 | + virtual void pause() = 0; |
196 | + |
197 | + /** |
198 | + * @brief Resume the request |
199 | + * @throw core::net::http::Error in case of http-related errors. |
200 | + */ |
201 | + virtual void resume() = 0; |
202 | }; |
203 | } |
204 | } |
205 | |
206 | === modified file 'src/core/net/http/impl/curl/client.cpp' |
207 | --- src/core/net/http/impl/curl/client.cpp 2015-03-23 16:09:05 +0000 |
208 | +++ src/core/net/http/impl/curl/client.cpp 2016-05-23 10:51:40 +0000 |
209 | @@ -14,6 +14,7 @@ |
210 | * along with this program. If not, see <http://www.gnu.org/licenses/>. |
211 | * |
212 | * Authored by: Thomas Voß <thomas.voss@canonical.com> |
213 | + * Gary Wang <gary.wang@canonical.com> |
214 | */ |
215 | |
216 | #include "client.h" |
217 | @@ -176,6 +177,39 @@ |
218 | return std::shared_ptr<http::impl::curl::Request>{new http::impl::curl::Request{multi, handle}}; |
219 | } |
220 | |
221 | +std::shared_ptr<http::impl::curl::Request> http::impl::curl::Client::post_impl( |
222 | + const Request::Configuration& configuration, |
223 | + std::istream& payload, |
224 | + std::size_t size) |
225 | +{ |
226 | + ::curl::easy::Handle handle; |
227 | + handle.method(http::Method::post) |
228 | + .url(configuration.uri.c_str()) |
229 | + .header(configuration.header) |
230 | + .on_read_data([&payload, size](void* dest, std::size_t in_size, std::size_t nmemb) |
231 | + { |
232 | + //use internal buffer size(in_size *nmemb) instread of size passed by parameter |
233 | + //to avoid client crashing when sending large chuck of data via POST method |
234 | + auto result = payload.readsome(static_cast<char *>(dest), in_size * nmemb); |
235 | + return result; |
236 | + }, size); |
237 | + |
238 | + |
239 | + handle.set_option(::curl::Option::post_field_size, size); |
240 | + handle.set_option(::curl::Option::ssl_verify_host, |
241 | + configuration.ssl.verify_host ? ::curl::easy::enable_ssl_host_verification : ::curl::easy::disable); |
242 | + handle.set_option(::curl::Option::ssl_verify_peer, |
243 | + configuration.ssl.verify_peer ? ::curl::easy::enable : ::curl::easy::disable); |
244 | + |
245 | + if (configuration.authentication_handler.for_http) |
246 | + { |
247 | + auto credentials = configuration.authentication_handler.for_http(configuration.uri); |
248 | + handle.http_credentials(credentials.username, credentials.password); |
249 | + } |
250 | + |
251 | + return std::shared_ptr<http::impl::curl::Request>{new http::impl::curl::Request{multi, handle}}; |
252 | +} |
253 | + |
254 | std::shared_ptr<http::impl::curl::Request> http::impl::curl::Client::put_impl( |
255 | const Request::Configuration& configuration, |
256 | std::istream& payload, |
257 | @@ -185,9 +219,11 @@ |
258 | handle.method(http::Method::put) |
259 | .url(configuration.uri.c_str()) |
260 | .header(configuration.header) |
261 | - .on_read_data([&payload, size](void* dest, std::size_t /*in_size*/, std::size_t /*nmemb*/) |
262 | + .on_read_data([&payload, size](void* dest, std::size_t in_size, std::size_t nmemb) |
263 | { |
264 | - auto result = payload.readsome(static_cast<char*>(dest), size); |
265 | + //use internal buffer size(in_size *nmemb) instread of size passed by parameter |
266 | + //to avoid client crashing when sending large chuck of data via PUT method |
267 | + auto result = payload.readsome(static_cast<char*>(dest), in_size * nmemb); |
268 | return result; |
269 | }, size); |
270 | |
271 | @@ -205,6 +241,27 @@ |
272 | return std::shared_ptr<http::impl::curl::Request>{new http::impl::curl::Request{multi, handle}}; |
273 | } |
274 | |
275 | +std::shared_ptr<http::impl::curl::Request> http::impl::curl::Client::del_impl(const http::Request::Configuration& configuration) |
276 | +{ |
277 | + ::curl::easy::Handle handle; |
278 | + handle.method(http::Method::del) |
279 | + .url(configuration.uri.c_str()) |
280 | + .header(configuration.header); |
281 | + |
282 | + handle.set_option(::curl::Option::ssl_verify_host, |
283 | + configuration.ssl.verify_host ? ::curl::easy::enable_ssl_host_verification : ::curl::easy::disable); |
284 | + handle.set_option(::curl::Option::ssl_verify_peer, |
285 | + configuration.ssl.verify_peer ? ::curl::easy::enable : ::curl::easy::disable); |
286 | + |
287 | + if (configuration.authentication_handler.for_http) |
288 | + { |
289 | + auto credentials = configuration.authentication_handler.for_http(configuration.uri); |
290 | + handle.http_credentials(credentials.username, credentials.password); |
291 | + } |
292 | + |
293 | + return std::shared_ptr<http::impl::curl::Request>{new http::impl::curl::Request{multi, handle}}; |
294 | +} |
295 | + |
296 | std::shared_ptr<http::StreamingRequest> http::impl::curl::Client::streaming_get(const http::Request::Configuration& configuration) |
297 | { |
298 | return get_impl(configuration); |
299 | @@ -225,6 +282,11 @@ |
300 | return post_impl(configuration, payload, type); |
301 | } |
302 | |
303 | +std::shared_ptr<http::StreamingRequest> http::impl::curl::Client::streaming_post(const http::Request ::Configuration& configuration, std::istream& payload, std::size_t size) |
304 | +{ |
305 | + return post_impl(configuration, payload, size); |
306 | +} |
307 | + |
308 | std::shared_ptr<http::StreamingRequest> http::impl::curl::Client::streaming_post_form(const http::Request::Configuration& configuration, const std::map<std::string, std::string>& values) |
309 | { |
310 | std::stringstream ss; |
311 | @@ -239,6 +301,11 @@ |
312 | return post_impl(configuration, ss.str(), http::ContentType::x_www_form_urlencoded); |
313 | } |
314 | |
315 | +std::shared_ptr<http::StreamingRequest> http::impl::curl::Client::streaming_del(const http::Request::Configuration& configuration) |
316 | +{ |
317 | + return del_impl(configuration); |
318 | +} |
319 | + |
320 | std::shared_ptr<http::Request> http::impl::curl::Client::head(const http::Request::Configuration& configuration) |
321 | { |
322 | return head_impl(configuration); |
323 | @@ -257,6 +324,19 @@ |
324 | return post_impl(configuration, payload, ct); |
325 | } |
326 | |
327 | +std::shared_ptr<http::Request> http::impl::curl::Client::post( |
328 | + const Request::Configuration& configuration, |
329 | + std::istream& payload, |
330 | + std::size_t size) |
331 | +{ |
332 | + return post_impl(configuration, payload, size); |
333 | +} |
334 | + |
335 | +std::shared_ptr<http::Request> http::impl::curl::Client::del(const Request::Configuration& configuration) |
336 | +{ |
337 | + return del_impl(configuration); |
338 | +} |
339 | + |
340 | std::shared_ptr<http::Request> http::impl::curl::Client::put( |
341 | const Request::Configuration& configuration, |
342 | std::istream& payload, |
343 | |
344 | === modified file 'src/core/net/http/impl/curl/client.h' |
345 | --- src/core/net/http/impl/curl/client.h 2015-03-24 11:56:02 +0000 |
346 | +++ src/core/net/http/impl/curl/client.h 2016-05-23 10:51:40 +0000 |
347 | @@ -14,6 +14,7 @@ |
348 | * along with this program. If not, see <http://www.gnu.org/licenses/>. |
349 | * |
350 | * Authored by: Thomas Voß <thomas.voss@canonical.com> |
351 | + * Gary Wang <gary.wang@canonical.com> |
352 | */ |
353 | #ifndef CORE_NET_HTTP_IMPL_CURL_CLIENT_H_ |
354 | #define CORE_NET_HTTP_IMPL_CURL_CLIENT_H_ |
355 | @@ -55,19 +56,25 @@ |
356 | std::shared_ptr<http::Request> get(const Request::Configuration& configuration) override; |
357 | std::shared_ptr<http::Request> head(const Request::Configuration& configuration) override; |
358 | std::shared_ptr<http::Request> post(const Request::Configuration& configuration, const std::string&, const std::string&) override; |
359 | + std::shared_ptr<http::Request> post(const Request::Configuration& configuration, std::istream& payload, std::size_t size) override; |
360 | std::shared_ptr<http::Request> put(const Request::Configuration& configuration, std::istream& payload, std::size_t size) override; |
361 | + std::shared_ptr<http::Request> del(const Request::Configuration& configuration) override; |
362 | |
363 | std::shared_ptr<http::StreamingRequest> streaming_get(const Request::Configuration& configuration) override; |
364 | std::shared_ptr<http::StreamingRequest> streaming_head(const Request::Configuration& configuration) override; |
365 | std::shared_ptr<http::StreamingRequest> streaming_put(const Request::Configuration& configuration, std::istream& payload, std::size_t size) override; |
366 | std::shared_ptr<http::StreamingRequest> streaming_post(const Request::Configuration& configuration, const std::string& payload, const std::string& type) override; |
367 | + std::shared_ptr<http::StreamingRequest> streaming_post(const Request::Configuration& configuration, std::istream& payload, std::size_t size) override; |
368 | std::shared_ptr<http::StreamingRequest> streaming_post_form(const Request::Configuration& configuration, const std::map<std::string, std::string>& values) override; |
369 | + std::shared_ptr<http::StreamingRequest> streaming_del(const Request::Configuration& configuration) override; |
370 | |
371 | private: |
372 | std::shared_ptr<curl::Request> get_impl(const Request::Configuration& configuration); |
373 | std::shared_ptr<curl::Request> head_impl(const Request::Configuration& configuration); |
374 | std::shared_ptr<curl::Request> post_impl(const Request::Configuration& configuration, const std::string&, const std::string&); |
375 | + std::shared_ptr<curl::Request> post_impl(const Request::Configuration& configuration, std::istream& payload, std::size_t size); |
376 | std::shared_ptr<curl::Request> put_impl(const Request::Configuration& configuration, std::istream& payload, std::size_t size); |
377 | + std::shared_ptr<curl::Request> del_impl(const Request::Configuration& configuration); |
378 | |
379 | ::curl::multi::Handle multi; |
380 | }; |
381 | |
382 | === modified file 'src/core/net/http/impl/curl/easy.cpp' |
383 | --- src/core/net/http/impl/curl/easy.cpp 2015-03-03 11:12:12 +0000 |
384 | +++ src/core/net/http/impl/curl/easy.cpp 2016-05-23 10:51:40 +0000 |
385 | @@ -14,6 +14,7 @@ |
386 | * along with this program. If not, see <http://www.gnu.org/licenses/>. |
387 | * |
388 | * Authored by: Thomas Voß <thomas.voss@canonical.com> |
389 | + * Gary Wang <gary.wang@canonical.com> |
390 | */ |
391 | |
392 | #include "easy.h" |
393 | @@ -96,6 +97,11 @@ |
394 | return static_cast<curl::Code>(curl_easy_perform(handle)); |
395 | } |
396 | |
397 | +::curl::Code easy::native::pause(easy::native::Handle handle, int bitmask) |
398 | +{ |
399 | + return static_cast<curl::Code>(curl_easy_pause(handle, bitmask)); |
400 | +} |
401 | + |
402 | std::string easy::native::escape(easy::native::Handle handle, const std::string& in) |
403 | { |
404 | auto escaped = curl_easy_escape(handle, in.c_str(), in.size()); |
405 | @@ -355,6 +361,8 @@ |
406 | { |
407 | if (!d) throw easy::Handle::HandleHasBeenAbandoned{}; |
408 | |
409 | + static constexpr const char* http_delete = "DELETE"; |
410 | + |
411 | switch(method) |
412 | { |
413 | case core::net::http::Method::get: |
414 | @@ -371,6 +379,9 @@ |
415 | case core::net::http::Method::put: |
416 | set_option(Option::http_put, enable); |
417 | break; |
418 | + case core::net::http::Method::del: |
419 | + set_option(Option::customrequest, http_delete); |
420 | + break; |
421 | default: throw core::net::http::Client::Errors::HttpMethodNotSupported{method, CORE_FROM_HERE()}; |
422 | } |
423 | |
424 | @@ -431,6 +442,18 @@ |
425 | throw_if_not<curl::Code::ok>(easy::native::perform(native()), [this]() { return std::string{d->error};}); |
426 | } |
427 | |
428 | +void easy::Handle::pause() |
429 | +{ |
430 | + if (!d) throw easy::Handle::HandleHasBeenAbandoned{}; |
431 | + throw_if_not<curl::Code::ok>(easy::native::pause(native(), CURLPAUSE_ALL), [this]() { return std::string{d->error};}); |
432 | +} |
433 | + |
434 | +void easy::Handle::resume() |
435 | +{ |
436 | + if (!d) throw easy::Handle::HandleHasBeenAbandoned{}; |
437 | + throw_if_not<curl::Code::ok>(easy::native::pause(native(), CURLPAUSE_RECV_CONT), [this]() { return std::string{d->error};}); |
438 | +} |
439 | + |
440 | // URL escapes the given input string. |
441 | std::string easy::Handle::escape(const std::string& in) |
442 | { |
443 | |
444 | === modified file 'src/core/net/http/impl/curl/easy.h' |
445 | --- src/core/net/http/impl/curl/easy.h 2015-03-03 11:12:12 +0000 |
446 | +++ src/core/net/http/impl/curl/easy.h 2016-05-23 10:51:40 +0000 |
447 | @@ -14,6 +14,7 @@ |
448 | * along with this program. If not, see <http://www.gnu.org/licenses/>. |
449 | * |
450 | * Authored by: Thomas Voß <thomas.voss@canonical.com> |
451 | + * Gary Wang <gary.wang@canonical.com> |
452 | */ |
453 | #ifndef CORE_NET_HTTP_IMPL_CURL_EASY_H_ |
454 | #define CORE_NET_HTTP_IMPL_CURL_EASY_H_ |
455 | @@ -136,7 +137,10 @@ |
456 | timeout_ms = CURLOPT_TIMEOUT_MS, |
457 | ssl_engine_default = CURLOPT_SSLENGINE_DEFAULT, |
458 | ssl_verify_peer = CURLOPT_SSL_VERIFYPEER, |
459 | - ssl_verify_host = CURLOPT_SSL_VERIFYHOST |
460 | + ssl_verify_host = CURLOPT_SSL_VERIFYHOST, |
461 | + customrequest = CURLOPT_CUSTOMREQUEST, |
462 | + low_speed_limit = CURLOPT_LOW_SPEED_LIMIT, |
463 | + low_speed_time = CURLOPT_LOW_SPEED_TIME |
464 | }; |
465 | |
466 | namespace native |
467 | @@ -209,6 +213,9 @@ |
468 | // Executes the operation configured on the handle. |
469 | ::curl::Code perform(Handle handle); |
470 | |
471 | +// Executes pause operation on the handle. |
472 | +::curl::Code pause(Handle handle, int bitmask); |
473 | + |
474 | // URL escapes the given input string. |
475 | std::string escape(Handle handle, const std::string& in); |
476 | |
477 | @@ -339,6 +346,12 @@ |
478 | // Executes the operation associated with this handle. |
479 | void perform(); |
480 | |
481 | + // Executes pause operation associated with this handle. |
482 | + void pause(); |
483 | + |
484 | + // Executes resume operation associated with this handle. |
485 | + void resume(); |
486 | + |
487 | // URL escapes the given input string. |
488 | std::string escape(const std::string& in); |
489 | |
490 | |
491 | === modified file 'src/core/net/http/impl/curl/request.h' |
492 | --- src/core/net/http/impl/curl/request.h 2015-03-23 16:09:05 +0000 |
493 | +++ src/core/net/http/impl/curl/request.h 2016-05-23 10:51:40 +0000 |
494 | @@ -14,6 +14,7 @@ |
495 | * along with this program. If not, see <http://www.gnu.org/licenses/>. |
496 | * |
497 | * Authored by: Thomas Voß <thomas.voss@canonical.com> |
498 | + * Gary Wang <gary.wang@canonical.com> |
499 | */ |
500 | #ifndef CORE_NET_HTTP_IMPL_CURL_REQUEST_H_ |
501 | #define CORE_NET_HTTP_IMPL_CURL_REQUEST_H_ |
502 | @@ -273,6 +274,28 @@ |
503 | multi.add(easy); |
504 | } |
505 | |
506 | + void pause() |
507 | + { |
508 | + try |
509 | + { |
510 | + easy.pause(); |
511 | + } catch(const std::system_error& se) |
512 | + { |
513 | + throw core::net::http::Error(se.what(), CORE_FROM_HERE()); |
514 | + } |
515 | + } |
516 | + |
517 | + void resume() |
518 | + { |
519 | + try |
520 | + { |
521 | + easy.resume(); |
522 | + } catch(const std::system_error& se) |
523 | + { |
524 | + throw core::net::http::Error(se.what(), CORE_FROM_HERE()); |
525 | + } |
526 | + } |
527 | + |
528 | std::string url_escape(const std::string& s) |
529 | { |
530 | return easy.escape(s); |
531 | @@ -282,6 +305,16 @@ |
532 | { |
533 | return easy.unescape(s); |
534 | } |
535 | + |
536 | + void abort_request_if(std::uint64_t limit, const std::chrono::seconds& time) |
537 | + { |
538 | + if (atomic_state.load() != core::net::http::Request::State::ready) |
539 | + throw core::net::http::Request::Errors::AlreadyActive{CORE_FROM_HERE()}; |
540 | + |
541 | + easy.set_option(::curl::Option::low_speed_limit, limit); |
542 | + easy.set_option(::curl::Option::low_speed_time, time.count()); |
543 | + } |
544 | + |
545 | private: |
546 | std::atomic<core::net::http::Request::State> atomic_state; |
547 | ::curl::multi::Handle multi; |
548 | |
549 | === modified file 'tests/http_client_test.cpp' |
550 | --- tests/http_client_test.cpp 2014-11-13 12:57:24 +0000 |
551 | +++ tests/http_client_test.cpp 2016-05-23 10:51:40 +0000 |
552 | @@ -14,6 +14,7 @@ |
553 | * along with this program. If not, see <http://www.gnu.org/licenses/>. |
554 | * |
555 | * Authored by: Thomas Voß <thomas.voss@canonical.com> |
556 | + * Gary Wang <gary.wang@canonical.com> |
557 | */ |
558 | |
559 | #include <core/net/error.h> |
560 | @@ -30,6 +31,7 @@ |
561 | #include <json/json.h> |
562 | |
563 | #include <future> |
564 | +#include <fstream> |
565 | |
566 | namespace http = core::net::http; |
567 | namespace json = Json; |
568 | @@ -456,6 +458,33 @@ |
569 | EXPECT_EQ("test", root["form"]["test"].asString()); |
570 | } |
571 | |
572 | +TEST(HttpClient, post_request_for_file_with_large_chunk_succeeds) |
573 | +{ |
574 | + auto client = http::make_client(); |
575 | + auto url = std::string(httpbin::host) + httpbin::resources::post(); |
576 | + |
577 | + // create temp file with large chunk |
578 | + const std::size_t size = 1024*1024; |
579 | + std::ofstream ofs("tmp.dat", std::ios::binary | std::ios::out); |
580 | + ofs.seekp(size); |
581 | + ofs.write("", 1); |
582 | + ofs.close(); |
583 | + |
584 | + std::ifstream payload("tmp.dat"); |
585 | + auto request = client->post(http::Request::Configuration::from_uri_as_string(url), |
586 | + payload, |
587 | + size); |
588 | + |
589 | + json::Value root; |
590 | + json::Reader reader; |
591 | + |
592 | + auto response = request->execute(default_progress_reporter); |
593 | + |
594 | + EXPECT_EQ(core::net::http::Status::ok, response.status); |
595 | + EXPECT_TRUE(reader.parse(response.body, root)); |
596 | + EXPECT_EQ(url, root["url"].asString()); |
597 | +} |
598 | + |
599 | TEST(HttpClient, put_request_for_existing_resource_succeeds) |
600 | { |
601 | auto client = http::make_client(); |
602 | @@ -478,6 +507,50 @@ |
603 | EXPECT_EQ(payload.str(), root["data"].asString()); |
604 | } |
605 | |
606 | +TEST(HttpClient, put_request_for_file_with_large_chunk_succeeds) |
607 | +{ |
608 | + auto client = http::make_client(); |
609 | + auto url = std::string(httpbin::host) + httpbin::resources::put(); |
610 | + |
611 | + // create temp file with large chunk |
612 | + const std::size_t size = 1024*1024; |
613 | + std::ofstream ofs("tmp.dat", std::ios::binary | std::ios::out); |
614 | + ofs.seekp(size); |
615 | + ofs.write("", 1); |
616 | + ofs.close(); |
617 | + |
618 | + std::ifstream payload("tmp.dat"); |
619 | + auto request = client->put(http::Request::Configuration::from_uri_as_string(url), |
620 | + payload, |
621 | + size); |
622 | + |
623 | + json::Value root; |
624 | + json::Reader reader; |
625 | + |
626 | + auto response = request->execute(default_progress_reporter); |
627 | + |
628 | + EXPECT_EQ(core::net::http::Status::ok, response.status); |
629 | + EXPECT_TRUE(reader.parse(response.body, root)); |
630 | + EXPECT_EQ(url, root["url"].asString()); |
631 | +} |
632 | + |
633 | +TEST(HttpClient, del_request_for_existing_resource_succeeds) |
634 | +{ |
635 | + auto client = http::make_client(); |
636 | + auto url = std::string(httpbin::host) + httpbin::resources::del(); |
637 | + |
638 | + auto request = client->del(http::Request::Configuration::from_uri_as_string(url)); |
639 | + |
640 | + json::Value root; |
641 | + json::Reader reader; |
642 | + |
643 | + auto response = request->execute(default_progress_reporter); |
644 | + |
645 | + EXPECT_EQ(core::net::http::Status::ok, response.status); |
646 | + EXPECT_TRUE(reader.parse(response.body, root)); |
647 | + EXPECT_EQ(url, root["url"].asString()); |
648 | +} |
649 | + |
650 | namespace com |
651 | { |
652 | namespace mozilla |
653 | |
654 | === modified file 'tests/http_streaming_client_test.cpp' |
655 | --- tests/http_streaming_client_test.cpp 2015-03-23 16:24:27 +0000 |
656 | +++ tests/http_streaming_client_test.cpp 2016-05-23 10:51:40 +0000 |
657 | @@ -14,6 +14,7 @@ |
658 | * along with this program. If not, see <http://www.gnu.org/licenses/>. |
659 | * |
660 | * Authored by: Thomas Voß <thomas.voss@canonical.com> |
661 | + * Gary Wang <gary.wang@canonical.com> |
662 | */ |
663 | |
664 | #include <core/net/error.h> |
665 | @@ -33,6 +34,8 @@ |
666 | #include <future> |
667 | #include <memory> |
668 | |
669 | +#include <fstream> |
670 | + |
671 | namespace http = core::net::http; |
672 | namespace json = Json; |
673 | namespace net = core::net; |
674 | @@ -480,6 +483,37 @@ |
675 | EXPECT_EQ("test", root["form"]["test"].asString()); |
676 | } |
677 | |
678 | +TEST(StreamingHttpClient, post_request_for_file_with_large_chunk_succeeds) |
679 | +{ |
680 | + using namespace ::testing; |
681 | + |
682 | + auto client = http::make_streaming_client(); |
683 | + auto url = std::string(httpbin::host) + httpbin::resources::post(); |
684 | + |
685 | + // create temp file with large chunk |
686 | + const std::size_t size = 1024*1024; |
687 | + std::ofstream ofs("tmp.dat", std::ios::binary | std::ios::out); |
688 | + ofs.seekp(size); |
689 | + ofs.write("", 1); |
690 | + ofs.close(); |
691 | + |
692 | + std::ifstream payload("tmp.dat"); |
693 | + auto request = client->streaming_post(http::Request::Configuration::from_uri_as_string(url), |
694 | + payload, |
695 | + size); |
696 | + |
697 | + auto dh = MockDataHandler::create(); EXPECT_CALL(*dh, on_new_data(_)).Times(AtLeast(1)); |
698 | + |
699 | + auto response = request->execute(default_progress_reporter, dh->to_data_handler()); |
700 | + |
701 | + json::Value root; |
702 | + json::Reader reader; |
703 | + |
704 | + EXPECT_EQ(core::net::http::Status::ok, response.status); |
705 | + EXPECT_TRUE(reader.parse(response.body, root)); |
706 | + EXPECT_EQ(url, root["url"].asString()); |
707 | +} |
708 | + |
709 | TEST(StreamingHttpClient, put_request_for_existing_resource_succeeds) |
710 | { |
711 | using namespace ::testing; |
712 | @@ -497,12 +531,66 @@ |
713 | // Our mocked data handler. |
714 | auto dh = MockDataHandler::create(); EXPECT_CALL(*dh, on_new_data(_)).Times(AtLeast(1)); |
715 | |
716 | + auto response = request->execute(default_progress_reporter, dh->to_data_handler()); |
717 | + |
718 | json::Value root; |
719 | json::Reader reader; |
720 | |
721 | - auto response = request->execute(default_progress_reporter, dh->to_data_handler()); |
722 | - |
723 | EXPECT_EQ(core::net::http::Status::ok, response.status); |
724 | EXPECT_TRUE(reader.parse(response.body, root)); |
725 | EXPECT_EQ(payload.str(), root["data"].asString()); |
726 | } |
727 | + |
728 | +TEST(StreamingHttpClient, put_request_for_file_with_large_chunk_succeeds) |
729 | +{ |
730 | + using namespace ::testing; |
731 | + |
732 | + auto client = http::make_streaming_client(); |
733 | + auto url = std::string(httpbin::host) + httpbin::resources::put(); |
734 | + |
735 | + // create temp file with large chunk |
736 | + const std::size_t size = 1024*1024; |
737 | + std::ofstream ofs("tmp.dat", std::ios::binary | std::ios::out); |
738 | + ofs.seekp(size); |
739 | + ofs.write("", 1); |
740 | + ofs.close(); |
741 | + |
742 | + std::ifstream payload("tmp.dat"); |
743 | + auto request = client->streaming_put(http::Request::Configuration::from_uri_as_string(url), |
744 | + payload, |
745 | + size); |
746 | + |
747 | + auto dh = MockDataHandler::create(); EXPECT_CALL(*dh, on_new_data(_)).Times(AtLeast(1)); |
748 | + |
749 | + auto response = request->execute(default_progress_reporter, dh->to_data_handler()); |
750 | + |
751 | + json::Value root; |
752 | + json::Reader reader; |
753 | + |
754 | + EXPECT_EQ(core::net::http::Status::ok, response.status); |
755 | + EXPECT_TRUE(reader.parse(response.body, root)); |
756 | + EXPECT_EQ(url, root["url"].asString()); |
757 | +} |
758 | + |
759 | +TEST(StreamingHttpClient, del_request_for_existing_resource_succeeds) |
760 | +{ |
761 | + using namespace ::testing; |
762 | + |
763 | + auto client = http::make_streaming_client(); |
764 | + auto url = std::string(httpbin::host) + httpbin::resources::del(); |
765 | + |
766 | + auto request = client->streaming_del(http::Request::Configuration::from_uri_as_string(url)); |
767 | + |
768 | + // Our mocked data handler. |
769 | + auto dh = MockDataHandler::create(); EXPECT_CALL(*dh, on_new_data(_)).Times(AtLeast(1)); |
770 | + |
771 | + auto response = request->execute(default_progress_reporter, dh->to_data_handler()); |
772 | + |
773 | + json::Value root; |
774 | + json::Reader reader; |
775 | + |
776 | + EXPECT_EQ(core::net::http::Status::ok, response.status); |
777 | + EXPECT_TRUE(reader.parse(response.body, root)); |
778 | + EXPECT_EQ(url, root["url"].asString()); |
779 | +} |
780 | + |
781 | |
782 | === modified file 'tests/httpbin.h.in' |
783 | --- tests/httpbin.h.in 2014-05-22 11:50:55 +0000 |
784 | +++ tests/httpbin.h.in 2016-05-23 10:51:40 +0000 |
785 | @@ -14,6 +14,7 @@ |
786 | * along with this program. If not, see <http://www.gnu.org/licenses/>. |
787 | * |
788 | * Authored by: Thomas Voß <thomas.voss@canonical.com> |
789 | + * Gary Wang <gary.wang@canonical.com> |
790 | */ |
791 | |
792 | #ifndef HTTPBIN_H_ |
793 | @@ -104,6 +105,11 @@ |
794 | { |
795 | return "/put"; |
796 | } |
797 | +/** Returns DELETE data. */ |
798 | +const char* del() |
799 | +{ |
800 | + return "/delete"; |
801 | +} |
802 | /** Challenges basic authentication. */ |
803 | const char* basic_auth() |
804 | { |
Thanks for the changes, looking good. A few minor niggles inline,