Merge lp:~gary-wzl77/net-cpp/bug-fixing-and-features into lp:net-cpp

Proposed by Gary.Wang
Status: Merged
Merged at revision: 50
Proposed branch: lp:~gary-wzl77/net-cpp/bug-fixing-and-features
Merge into: lp:net-cpp
Diff against target: 804 lines (+407/-9)
14 files modified
CMakeLists.txt (+2/-2)
include/core/net/http/client.h (+19/-0)
include/core/net/http/method.h (+3/-1)
include/core/net/http/request.h (+21/-0)
include/core/net/http/streaming_client.h (+21/-1)
include/core/net/http/streaming_request.h (+13/-0)
src/core/net/http/impl/curl/client.cpp (+82/-2)
src/core/net/http/impl/curl/client.h (+7/-0)
src/core/net/http/impl/curl/easy.cpp (+23/-0)
src/core/net/http/impl/curl/easy.h (+14/-1)
src/core/net/http/impl/curl/request.h (+33/-0)
tests/http_client_test.cpp (+73/-0)
tests/http_streaming_client_test.cpp (+90/-2)
tests/httpbin.h.in (+6/-0)
To merge this branch: bzr merge lp:~gary-wzl77/net-cpp/bug-fixing-and-features
Reviewer Review Type Date Requested Status
Thomas Voß (community) Approve
Review via email: mp+291975@code.launchpad.net

Commit message

1.Fix crash issue when sending large chunk data via PUT method (lp:1570686)
3.Support DELETE method (lp:1570687)
2.Support POST method with istream
4.Support pause and resume mechanism
5.Add some test cases

Description of the change

1.Fix crash issue when sending large chunk data via PUT method (lp:1570686)
3.Support DELETE method (lp:1570687)
2.Support POST method with istream
4.Support pause and resume mechanism
5.Add some test cases

To post a comment you must log in.
Revision history for this message
Thomas Voß (thomas-voss) wrote :

Thanks for the changes, looking good. A few minor niggles inline,

review: Needs Fixing
51. By Gary.Wang

1.bump major revision since this change breaks ABI
2.introduce new API (abort_request_option) for request
3.fix typo

52. By Gary.Wang

1.abort_request_option -> abort_request_if to make api more self-explanatory
2.do not set abort request option by default

Revision history for this message
Thomas Voß (thomas-voss) wrote :

Thanks for your changes, the MP looks good to me. I tried to further understand your remarks about only setting abort_request_option if explicitly requested and read up on: https://curl.haxx.se/libcurl/c/curl_easy_pause.html. I do not see an issue in the case of async requests, though. For synchronous requests, the operation might block indefinitely, and we should at least document the respective error mode.

Hope that addresses your remark :) Or did I miss something?

review: Approve
Revision history for this message
Gary.Wang (gary-wzl77) wrote :

Thanks, Please add the following test case in http_streaming_client_test.cpp.
http://bazaar.launchpad.net/~gary-wzl77/+junk/net-cpp_test_pause_and_resume/revision/53/tests/http_streaming_client_test.cpp#tests/=

Then we can see the potential issue we have in this MP after it is run. (I use the async api to reproduce this issue.)

Log:
....
Download progress: 0.0154666
Download progress: 0.0154666
we pause
Download progress: 0.0154666
Download progress: 0.0154666
Download progress: 0.0154666
we resume.

I resume download after the request is paused for 5 secs. But there is no "Download progress" log printing out any more even if I resume it. It causes the curl thread to hang as well. The issue can be reproduces 100%.

However if I explicitly set async_abort_if(1k/s, 10s) before request asynchronous execution(async_execute). Everything works fine.

Log:
Download progress: 0.0077333
Download progress: 0.0077333
Download progress: 0.0077333
we pause
Download progress: 0.0077333
we resume.
Download progress: 0.0077333
Download progress: 0.0077333
Download progress: 0.0077333
Download progress: 0.00883806
Download progress: 0.00883806

So it's better to mention this case in doc somewhere to reminder developer of setting abort_request_option explicitly before request asynchronous execution if they really want to pause/resume a request in one connection.
Otherwise developer doesn't figure out why it doesn't work out after resuming from paused state.
But anyway, it works fine if developers don't use pause/resume API.

P.S:
curl utilizes progress callback(CURLOPT_PROGRESSFUNCTION) to set bitmask(CURLPAUSE_CONT/CURLPAUSE_ALL) for a connection state change(pause/resume). However the default value of speed_time(2nd argument for abort_request_if) is 0(disabled). So after request is paused, progress cb gets no chance to call in such a case. It failed to resume even if we set the bitmask(CURLPAUSE_CONT).

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'CMakeLists.txt'
--- CMakeLists.txt 2015-04-01 06:23:30 +0000
+++ CMakeLists.txt 2016-05-23 10:51:40 +0000
@@ -26,8 +26,8 @@
26set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++11 -Werror -Wall -fno-strict-aliasing -fvisibility=hidden -fvisibility-inlines-hidden -pedantic -Wextra")26set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++11 -Werror -Wall -fno-strict-aliasing -fvisibility=hidden -fvisibility-inlines-hidden -pedantic -Wextra")
27set(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} -Wl,--no-undefined")27set(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} -Wl,--no-undefined")
2828
29set(NET_CPP_VERSION_MAJOR 1)29set(NET_CPP_VERSION_MAJOR 2)
30set(NET_CPP_VERSION_MINOR 2)30set(NET_CPP_VERSION_MINOR 0)
31set(NET_CPP_VERSION_PATCH 0)31set(NET_CPP_VERSION_PATCH 0)
3232
33include(CTest)33include(CTest)
3434
=== modified file 'include/core/net/http/client.h'
--- include/core/net/http/client.h 2014-06-10 14:19:14 +0000
+++ include/core/net/http/client.h 2016-05-23 10:51:40 +0000
@@ -14,6 +14,7 @@
14 * along with this program. If not, see <http://www.gnu.org/licenses/>.14 * along with this program. If not, see <http://www.gnu.org/licenses/>.
15 *15 *
16 * Authored by: Thomas Voß <thomas.voss@canonical.com>16 * Authored by: Thomas Voß <thomas.voss@canonical.com>
17 * Gary Wang <gary.wang@canonical.com>
17 */18 */
18#ifndef CORE_NET_HTTP_CLIENT_H_19#ifndef CORE_NET_HTTP_CLIENT_H_
19#define CORE_NET_HTTP_CLIENT_H_20#define CORE_NET_HTTP_CLIENT_H_
@@ -154,6 +155,16 @@
154 */155 */
155 virtual std::shared_ptr<Request> post(const Request::Configuration& configuration, const std::string& payload, const std::string& type) = 0;156 virtual std::shared_ptr<Request> post(const Request::Configuration& configuration, const std::string& payload, const std::string& type) = 0;
156157
158 /**
159 * @brief post is a convenience method for issuing a POST request for the given URI.
160 * @throw Errors::HttpMethodNotSupported if the underlying implementation does not support the pro vided HTTP method.
161 * @param configuration The configuration to issue a get request for.
162 * @param payload The data to be transmitted as part of the POST request.
163 * @param size Size of the payload data in bytes.
164 * @return An executable instance of class Request.
165 */
166 virtual std::shared_ptr<Request> post(const Request::Configuration& configuration, std::istream& payload, std::size_t size) = 0;
167
157 /**168 /**
158 * @brief post_form is a convenience method for issuing a POST request for the given URI, with url-encoded payload.169 * @brief post_form is a convenience method for issuing a POST request for the given URI, with url-encoded payload.
159 * @throw Errors::HttpMethodNotSupported if the underlying implementation does not support the provided HTTP method.170 * @throw Errors::HttpMethodNotSupported if the underlying implementation does not support the provided HTTP method.
@@ -163,6 +174,14 @@
163 */174 */
164 virtual std::shared_ptr<Request> post_form(const Request::Configuration& configuration, const std::map<std::string, std::string>& values);175 virtual std::shared_ptr<Request> post_form(const Request::Configuration& configuration, const std::map<std::string, std::string>& values);
165176
177 /**
178 * @brief del is a convenience method for issueing a DELETE request for the given URI.
179 * @throw Errors::HttpMethodNotSupported if the underlying implementation does not support the provided HTTP method.
180 * @param configuration The configuration to issue a get request for.
181 * @return An executable instance of class Request.
182 */
183 virtual std::shared_ptr<Request> del(const Request::Configuration& configuration) = 0;
184
166protected:185protected:
167 Client() = default;186 Client() = default;
168};187};
169188
=== modified file 'include/core/net/http/method.h'
--- include/core/net/http/method.h 2013-12-04 07:23:43 +0000
+++ include/core/net/http/method.h 2016-05-23 10:51:40 +0000
@@ -14,6 +14,7 @@
14 * along with this program. If not, see <http://www.gnu.org/licenses/>.14 * along with this program. If not, see <http://www.gnu.org/licenses/>.
15 *15 *
16 * Authored by: Thomas Voß <thomas.voss@canonical.com>16 * Authored by: Thomas Voß <thomas.voss@canonical.com>
17 * Gary Wang <gary.wang@canonical.com>
17 */18 */
18#ifndef CORE_NET_HTTP_METHOD_H_19#ifndef CORE_NET_HTTP_METHOD_H_
19#define CORE_NET_HTTP_METHOD_H_20#define CORE_NET_HTTP_METHOD_H_
@@ -33,7 +34,8 @@
33 get,34 get,
34 head,35 head,
35 post,36 post,
36 put37 put,
38 del
37};39};
38}40}
39}41}
4042
=== modified file 'include/core/net/http/request.h'
--- include/core/net/http/request.h 2014-05-06 11:05:04 +0000
+++ include/core/net/http/request.h 2016-05-23 10:51:40 +0000
@@ -14,6 +14,7 @@
14 * along with this program. If not, see <http://www.gnu.org/licenses/>.14 * along with this program. If not, see <http://www.gnu.org/licenses/>.
15 *15 *
16 * Authored by: Thomas Voß <thomas.voss@canonical.com>16 * Authored by: Thomas Voß <thomas.voss@canonical.com>
17 * Gary Wang <gary.wang@canonical.com>
17 */18 */
18#ifndef CORE_NET_HTTP_REQUEST_H_19#ifndef CORE_NET_HTTP_REQUEST_H_
19#define CORE_NET_HTTP_REQUEST_H_20#define CORE_NET_HTTP_REQUEST_H_
@@ -248,6 +249,18 @@
248 virtual void async_execute(const Handler& handler) = 0;249 virtual void async_execute(const Handler& handler) = 0;
249250
250 /**251 /**
252 * @brief Pause the request
253 * @throw core::net::http::Error in case of http-related errors.
254 */
255 virtual void pause() = 0;
256
257 /**
258 * @brief Resume the request
259 * @throw core::net::http::Error in case of http-related errors.
260 */
261 virtual void resume() = 0;
262
263 /**
251 * @brief Returns the input string in URL-escaped format.264 * @brief Returns the input string in URL-escaped format.
252 * @param s The string to be URL escaped.265 * @param s The string to be URL escaped.
253 */266 */
@@ -259,6 +272,14 @@
259 */272 */
260 virtual std::string url_unescape(const std::string& s) = 0;273 virtual std::string url_unescape(const std::string& s) = 0;
261274
275 /**
276 * @brief Sets options for aborting the request.
277 * The request will be aborted if transfer speed belows \a limit bytes per second for \a time seconds
278 * @param limit The transfer speed in seconds.
279 * @param time waiting period(seconds) to abort the request.
280 */
281 virtual void abort_request_if(std::uint64_t limit, const std::chrono::seconds& time) = 0;
282
262protected:283protected:
263 /** @cond */284 /** @cond */
264 Request() = default;285 Request() = default;
265286
=== modified file 'include/core/net/http/streaming_client.h'
--- include/core/net/http/streaming_client.h 2015-03-23 16:09:05 +0000
+++ include/core/net/http/streaming_client.h 2016-05-23 10:51:40 +0000
@@ -14,6 +14,7 @@
14 * along with this program. If not, see <http://www.gnu.org/licenses/>.14 * along with this program. If not, see <http://www.gnu.org/licenses/>.
15 *15 *
16 * Authored by: Thomas Voß <thomas.voss@canonical.com>16 * Authored by: Thomas Voß <thomas.voss@canonical.com>
17 * Gary Wang <gary.wang@canonical.com>
17 */18 */
18#ifndef CORE_NET_HTTP_STREAMING_CLIENT_H_19#ifndef CORE_NET_HTTP_STREAMING_CLIENT_H_
19#define CORE_NET_HTTP_STREAMING_CLIENT_H_20#define CORE_NET_HTTP_STREAMING_CLIENT_H_
@@ -69,7 +70,17 @@
69 * @return An executable instance of class Request.70 * @return An executable instance of class Request.
70 */71 */
71 virtual std::shared_ptr<StreamingRequest> streaming_post(const Request::Configuration& configuration, const std::string& payload, const std::string& type) = 0;72 virtual std::shared_ptr<StreamingRequest> streaming_post(const Request::Configuration& configuration, const std::string& payload, const std::string& type) = 0;
7273
74 /**
75 * @brief streaming_post is a convenience method for issuing a POST request for the given URI.
76 * @throw Errors::HttpMethodNotSupported if the underlying implementation does not support the pro vided HTTP method.
77 * @param configuration The configuration to issue a get request for.
78 * @param payload The data to be transmitted as part of the POST request.
79 * @param size Size of the payload data in bytes.
80 * @return An executable instance of class Request.
81 */
82 virtual std::shared_ptr<StreamingRequest> streaming_post(const Request::Configuration& configuration, std::istream& payload, std::size_t size) = 0;
83
73 /**84 /**
74 * @brief streaming_post_form is a convenience method for issuing a POST request for the given URI, with url-encoded payload.85 * @brief streaming_post_form is a convenience method for issuing a POST request for the given URI, with url-encoded payload.
75 * @throw Errors::HttpMethodNotSupported if the underlying implementation does not support the provided HTTP method.86 * @throw Errors::HttpMethodNotSupported if the underlying implementation does not support the provided HTTP method.
@@ -78,6 +89,15 @@
78 * @return An executable instance of class Request.89 * @return An executable instance of class Request.
79 */90 */
80 virtual std::shared_ptr<StreamingRequest> streaming_post_form(const Request::Configuration& configuration, const std::map<std::string, std::string>& values) = 0;91 virtual std::shared_ptr<StreamingRequest> streaming_post_form(const Request::Configuration& configuration, const std::map<std::string, std::string>& values) = 0;
92
93
94 /**
95 * @brief streaming_del is a convenience method for issuing a DELETE request for the given URI.
96 * @throw Errors::HttpMethodNotSupported if the underlying implementation does not support the provided HTTP method.
97 * @param configuration The configuration to issue a del request for.
98 * @return An executable instance of class Request.
99 */
100 virtual std::shared_ptr<StreamingRequest> streaming_del(const Request::Configuration& configuration) = 0;
81};101};
82102
83/** @brief Dispatches to the default implementation and returns a streaming client instance. */103/** @brief Dispatches to the default implementation and returns a streaming client instance. */
84104
=== modified file 'include/core/net/http/streaming_request.h'
--- include/core/net/http/streaming_request.h 2015-03-23 16:09:05 +0000
+++ include/core/net/http/streaming_request.h 2016-05-23 10:51:40 +0000
@@ -14,6 +14,7 @@
14 * along with this program. If not, see <http://www.gnu.org/licenses/>.14 * along with this program. If not, see <http://www.gnu.org/licenses/>.
15 *15 *
16 * Authored by: Thomas Voß <thomas.voss@canonical.com>16 * Authored by: Thomas Voß <thomas.voss@canonical.com>
17 * Gary Wang <gary.wang@canonical.com>
17 */18 */
18#ifndef CORE_NET_HTTP_STREAMING_REQUEST_H_19#ifndef CORE_NET_HTTP_STREAMING_REQUEST_H_
19#define CORE_NET_HTTP_STREAMING_REQUEST_H_20#define CORE_NET_HTTP_STREAMING_REQUEST_H_
@@ -52,6 +53,18 @@
52 * @return The response to the request.53 * @return The response to the request.
53 */54 */
54 virtual void async_execute(const Handler& handler, const DataHandler& dh) = 0;55 virtual void async_execute(const Handler& handler, const DataHandler& dh) = 0;
56
57 /**
58 * @brief Pause the request
59 * @throw core::net::http::Error in case of http-related errors.
60 */
61 virtual void pause() = 0;
62
63 /**
64 * @brief Resume the request
65 * @throw core::net::http::Error in case of http-related errors.
66 */
67 virtual void resume() = 0;
55};68};
56}69}
57}70}
5871
=== modified file 'src/core/net/http/impl/curl/client.cpp'
--- src/core/net/http/impl/curl/client.cpp 2015-03-23 16:09:05 +0000
+++ src/core/net/http/impl/curl/client.cpp 2016-05-23 10:51:40 +0000
@@ -14,6 +14,7 @@
14 * along with this program. If not, see <http://www.gnu.org/licenses/>.14 * along with this program. If not, see <http://www.gnu.org/licenses/>.
15 *15 *
16 * Authored by: Thomas Voß <thomas.voss@canonical.com>16 * Authored by: Thomas Voß <thomas.voss@canonical.com>
17 * Gary Wang <gary.wang@canonical.com>
17 */18 */
1819
19#include "client.h"20#include "client.h"
@@ -176,6 +177,39 @@
176 return std::shared_ptr<http::impl::curl::Request>{new http::impl::curl::Request{multi, handle}};177 return std::shared_ptr<http::impl::curl::Request>{new http::impl::curl::Request{multi, handle}};
177}178}
178179
180std::shared_ptr<http::impl::curl::Request> http::impl::curl::Client::post_impl(
181 const Request::Configuration& configuration,
182 std::istream& payload,
183 std::size_t size)
184{
185 ::curl::easy::Handle handle;
186 handle.method(http::Method::post)
187 .url(configuration.uri.c_str())
188 .header(configuration.header)
189 .on_read_data([&payload, size](void* dest, std::size_t in_size, std::size_t nmemb)
190 {
191 //use internal buffer size(in_size *nmemb) instread of size passed by parameter
192 //to avoid client crashing when sending large chuck of data via POST method
193 auto result = payload.readsome(static_cast<char *>(dest), in_size * nmemb);
194 return result;
195 }, size);
196
197
198 handle.set_option(::curl::Option::post_field_size, size);
199 handle.set_option(::curl::Option::ssl_verify_host,
200 configuration.ssl.verify_host ? ::curl::easy::enable_ssl_host_verification : ::curl::easy::disable);
201 handle.set_option(::curl::Option::ssl_verify_peer,
202 configuration.ssl.verify_peer ? ::curl::easy::enable : ::curl::easy::disable);
203
204 if (configuration.authentication_handler.for_http)
205 {
206 auto credentials = configuration.authentication_handler.for_http(configuration.uri);
207 handle.http_credentials(credentials.username, credentials.password);
208 }
209
210 return std::shared_ptr<http::impl::curl::Request>{new http::impl::curl::Request{multi, handle}};
211}
212
179std::shared_ptr<http::impl::curl::Request> http::impl::curl::Client::put_impl(213std::shared_ptr<http::impl::curl::Request> http::impl::curl::Client::put_impl(
180 const Request::Configuration& configuration,214 const Request::Configuration& configuration,
181 std::istream& payload,215 std::istream& payload,
@@ -185,9 +219,11 @@
185 handle.method(http::Method::put)219 handle.method(http::Method::put)
186 .url(configuration.uri.c_str())220 .url(configuration.uri.c_str())
187 .header(configuration.header)221 .header(configuration.header)
188 .on_read_data([&payload, size](void* dest, std::size_t /*in_size*/, std::size_t /*nmemb*/)222 .on_read_data([&payload, size](void* dest, std::size_t in_size, std::size_t nmemb)
189 {223 {
190 auto result = payload.readsome(static_cast<char*>(dest), size);224 //use internal buffer size(in_size *nmemb) instread of size passed by parameter
225 //to avoid client crashing when sending large chuck of data via PUT method
226 auto result = payload.readsome(static_cast<char*>(dest), in_size * nmemb);
191 return result;227 return result;
192 }, size);228 }, size);
193229
@@ -205,6 +241,27 @@
205 return std::shared_ptr<http::impl::curl::Request>{new http::impl::curl::Request{multi, handle}};241 return std::shared_ptr<http::impl::curl::Request>{new http::impl::curl::Request{multi, handle}};
206}242}
207243
244std::shared_ptr<http::impl::curl::Request> http::impl::curl::Client::del_impl(const http::Request::Configuration& configuration)
245{
246 ::curl::easy::Handle handle;
247 handle.method(http::Method::del)
248 .url(configuration.uri.c_str())
249 .header(configuration.header);
250
251 handle.set_option(::curl::Option::ssl_verify_host,
252 configuration.ssl.verify_host ? ::curl::easy::enable_ssl_host_verification : ::curl::easy::disable);
253 handle.set_option(::curl::Option::ssl_verify_peer,
254 configuration.ssl.verify_peer ? ::curl::easy::enable : ::curl::easy::disable);
255
256 if (configuration.authentication_handler.for_http)
257 {
258 auto credentials = configuration.authentication_handler.for_http(configuration.uri);
259 handle.http_credentials(credentials.username, credentials.password);
260 }
261
262 return std::shared_ptr<http::impl::curl::Request>{new http::impl::curl::Request{multi, handle}};
263}
264
208std::shared_ptr<http::StreamingRequest> http::impl::curl::Client::streaming_get(const http::Request::Configuration& configuration)265std::shared_ptr<http::StreamingRequest> http::impl::curl::Client::streaming_get(const http::Request::Configuration& configuration)
209{266{
210 return get_impl(configuration);267 return get_impl(configuration);
@@ -225,6 +282,11 @@
225 return post_impl(configuration, payload, type);282 return post_impl(configuration, payload, type);
226}283}
227284
285std::shared_ptr<http::StreamingRequest> http::impl::curl::Client::streaming_post(const http::Request ::Configuration& configuration, std::istream& payload, std::size_t size)
286{
287 return post_impl(configuration, payload, size);
288}
289
228std::shared_ptr<http::StreamingRequest> http::impl::curl::Client::streaming_post_form(const http::Request::Configuration& configuration, const std::map<std::string, std::string>& values)290std::shared_ptr<http::StreamingRequest> http::impl::curl::Client::streaming_post_form(const http::Request::Configuration& configuration, const std::map<std::string, std::string>& values)
229{291{
230 std::stringstream ss;292 std::stringstream ss;
@@ -239,6 +301,11 @@
239 return post_impl(configuration, ss.str(), http::ContentType::x_www_form_urlencoded);301 return post_impl(configuration, ss.str(), http::ContentType::x_www_form_urlencoded);
240}302}
241303
304std::shared_ptr<http::StreamingRequest> http::impl::curl::Client::streaming_del(const http::Request::Configuration& configuration)
305{
306 return del_impl(configuration);
307}
308
242std::shared_ptr<http::Request> http::impl::curl::Client::head(const http::Request::Configuration& configuration)309std::shared_ptr<http::Request> http::impl::curl::Client::head(const http::Request::Configuration& configuration)
243{310{
244 return head_impl(configuration);311 return head_impl(configuration);
@@ -257,6 +324,19 @@
257 return post_impl(configuration, payload, ct);324 return post_impl(configuration, payload, ct);
258}325}
259326
327std::shared_ptr<http::Request> http::impl::curl::Client::post(
328 const Request::Configuration& configuration,
329 std::istream& payload,
330 std::size_t size)
331{
332 return post_impl(configuration, payload, size);
333}
334
335std::shared_ptr<http::Request> http::impl::curl::Client::del(const Request::Configuration& configuration)
336{
337 return del_impl(configuration);
338}
339
260std::shared_ptr<http::Request> http::impl::curl::Client::put(340std::shared_ptr<http::Request> http::impl::curl::Client::put(
261 const Request::Configuration& configuration,341 const Request::Configuration& configuration,
262 std::istream& payload,342 std::istream& payload,
263343
=== modified file 'src/core/net/http/impl/curl/client.h'
--- src/core/net/http/impl/curl/client.h 2015-03-24 11:56:02 +0000
+++ src/core/net/http/impl/curl/client.h 2016-05-23 10:51:40 +0000
@@ -14,6 +14,7 @@
14 * along with this program. If not, see <http://www.gnu.org/licenses/>.14 * along with this program. If not, see <http://www.gnu.org/licenses/>.
15 *15 *
16 * Authored by: Thomas Voß <thomas.voss@canonical.com>16 * Authored by: Thomas Voß <thomas.voss@canonical.com>
17 * Gary Wang <gary.wang@canonical.com>
17 */18 */
18#ifndef CORE_NET_HTTP_IMPL_CURL_CLIENT_H_19#ifndef CORE_NET_HTTP_IMPL_CURL_CLIENT_H_
19#define CORE_NET_HTTP_IMPL_CURL_CLIENT_H_20#define CORE_NET_HTTP_IMPL_CURL_CLIENT_H_
@@ -55,19 +56,25 @@
55 std::shared_ptr<http::Request> get(const Request::Configuration& configuration) override;56 std::shared_ptr<http::Request> get(const Request::Configuration& configuration) override;
56 std::shared_ptr<http::Request> head(const Request::Configuration& configuration) override;57 std::shared_ptr<http::Request> head(const Request::Configuration& configuration) override;
57 std::shared_ptr<http::Request> post(const Request::Configuration& configuration, const std::string&, const std::string&) override;58 std::shared_ptr<http::Request> post(const Request::Configuration& configuration, const std::string&, const std::string&) override;
59 std::shared_ptr<http::Request> post(const Request::Configuration& configuration, std::istream& payload, std::size_t size) override;
58 std::shared_ptr<http::Request> put(const Request::Configuration& configuration, std::istream& payload, std::size_t size) override;60 std::shared_ptr<http::Request> put(const Request::Configuration& configuration, std::istream& payload, std::size_t size) override;
61 std::shared_ptr<http::Request> del(const Request::Configuration& configuration) override;
5962
60 std::shared_ptr<http::StreamingRequest> streaming_get(const Request::Configuration& configuration) override;63 std::shared_ptr<http::StreamingRequest> streaming_get(const Request::Configuration& configuration) override;
61 std::shared_ptr<http::StreamingRequest> streaming_head(const Request::Configuration& configuration) override;64 std::shared_ptr<http::StreamingRequest> streaming_head(const Request::Configuration& configuration) override;
62 std::shared_ptr<http::StreamingRequest> streaming_put(const Request::Configuration& configuration, std::istream& payload, std::size_t size) override;65 std::shared_ptr<http::StreamingRequest> streaming_put(const Request::Configuration& configuration, std::istream& payload, std::size_t size) override;
63 std::shared_ptr<http::StreamingRequest> streaming_post(const Request::Configuration& configuration, const std::string& payload, const std::string& type) override;66 std::shared_ptr<http::StreamingRequest> streaming_post(const Request::Configuration& configuration, const std::string& payload, const std::string& type) override;
67 std::shared_ptr<http::StreamingRequest> streaming_post(const Request::Configuration& configuration, std::istream& payload, std::size_t size) override;
64 std::shared_ptr<http::StreamingRequest> streaming_post_form(const Request::Configuration& configuration, const std::map<std::string, std::string>& values) override;68 std::shared_ptr<http::StreamingRequest> streaming_post_form(const Request::Configuration& configuration, const std::map<std::string, std::string>& values) override;
69 std::shared_ptr<http::StreamingRequest> streaming_del(const Request::Configuration& configuration) override;
6570
66private:71private:
67 std::shared_ptr<curl::Request> get_impl(const Request::Configuration& configuration);72 std::shared_ptr<curl::Request> get_impl(const Request::Configuration& configuration);
68 std::shared_ptr<curl::Request> head_impl(const Request::Configuration& configuration);73 std::shared_ptr<curl::Request> head_impl(const Request::Configuration& configuration);
69 std::shared_ptr<curl::Request> post_impl(const Request::Configuration& configuration, const std::string&, const std::string&);74 std::shared_ptr<curl::Request> post_impl(const Request::Configuration& configuration, const std::string&, const std::string&);
75 std::shared_ptr<curl::Request> post_impl(const Request::Configuration& configuration, std::istream& payload, std::size_t size);
70 std::shared_ptr<curl::Request> put_impl(const Request::Configuration& configuration, std::istream& payload, std::size_t size);76 std::shared_ptr<curl::Request> put_impl(const Request::Configuration& configuration, std::istream& payload, std::size_t size);
77 std::shared_ptr<curl::Request> del_impl(const Request::Configuration& configuration);
7178
72 ::curl::multi::Handle multi; 79 ::curl::multi::Handle multi;
73};80};
7481
=== modified file 'src/core/net/http/impl/curl/easy.cpp'
--- src/core/net/http/impl/curl/easy.cpp 2015-03-03 11:12:12 +0000
+++ src/core/net/http/impl/curl/easy.cpp 2016-05-23 10:51:40 +0000
@@ -14,6 +14,7 @@
14 * along with this program. If not, see <http://www.gnu.org/licenses/>.14 * along with this program. If not, see <http://www.gnu.org/licenses/>.
15 *15 *
16 * Authored by: Thomas Voß <thomas.voss@canonical.com>16 * Authored by: Thomas Voß <thomas.voss@canonical.com>
17 * Gary Wang <gary.wang@canonical.com>
17 */18 */
1819
19#include "easy.h"20#include "easy.h"
@@ -96,6 +97,11 @@
96 return static_cast<curl::Code>(curl_easy_perform(handle));97 return static_cast<curl::Code>(curl_easy_perform(handle));
97}98}
9899
100::curl::Code easy::native::pause(easy::native::Handle handle, int bitmask)
101{
102 return static_cast<curl::Code>(curl_easy_pause(handle, bitmask));
103}
104
99std::string easy::native::escape(easy::native::Handle handle, const std::string& in)105std::string easy::native::escape(easy::native::Handle handle, const std::string& in)
100{106{
101 auto escaped = curl_easy_escape(handle, in.c_str(), in.size());107 auto escaped = curl_easy_escape(handle, in.c_str(), in.size());
@@ -355,6 +361,8 @@
355{361{
356 if (!d) throw easy::Handle::HandleHasBeenAbandoned{};362 if (!d) throw easy::Handle::HandleHasBeenAbandoned{};
357363
364 static constexpr const char* http_delete = "DELETE";
365
358 switch(method)366 switch(method)
359 {367 {
360 case core::net::http::Method::get:368 case core::net::http::Method::get:
@@ -371,6 +379,9 @@
371 case core::net::http::Method::put:379 case core::net::http::Method::put:
372 set_option(Option::http_put, enable);380 set_option(Option::http_put, enable);
373 break;381 break;
382 case core::net::http::Method::del:
383 set_option(Option::customrequest, http_delete);
384 break;
374 default: throw core::net::http::Client::Errors::HttpMethodNotSupported{method, CORE_FROM_HERE()};385 default: throw core::net::http::Client::Errors::HttpMethodNotSupported{method, CORE_FROM_HERE()};
375 }386 }
376387
@@ -431,6 +442,18 @@
431 throw_if_not<curl::Code::ok>(easy::native::perform(native()), [this]() { return std::string{d->error};});442 throw_if_not<curl::Code::ok>(easy::native::perform(native()), [this]() { return std::string{d->error};});
432}443}
433444
445void easy::Handle::pause()
446{
447 if (!d) throw easy::Handle::HandleHasBeenAbandoned{};
448 throw_if_not<curl::Code::ok>(easy::native::pause(native(), CURLPAUSE_ALL), [this]() { return std::string{d->error};});
449}
450
451void easy::Handle::resume()
452{
453 if (!d) throw easy::Handle::HandleHasBeenAbandoned{};
454 throw_if_not<curl::Code::ok>(easy::native::pause(native(), CURLPAUSE_RECV_CONT), [this]() { return std::string{d->error};});
455}
456
434// URL escapes the given input string.457// URL escapes the given input string.
435std::string easy::Handle::escape(const std::string& in)458std::string easy::Handle::escape(const std::string& in)
436{459{
437460
=== modified file 'src/core/net/http/impl/curl/easy.h'
--- src/core/net/http/impl/curl/easy.h 2015-03-03 11:12:12 +0000
+++ src/core/net/http/impl/curl/easy.h 2016-05-23 10:51:40 +0000
@@ -14,6 +14,7 @@
14 * along with this program. If not, see <http://www.gnu.org/licenses/>.14 * along with this program. If not, see <http://www.gnu.org/licenses/>.
15 *15 *
16 * Authored by: Thomas Voß <thomas.voss@canonical.com>16 * Authored by: Thomas Voß <thomas.voss@canonical.com>
17 * Gary Wang <gary.wang@canonical.com>
17 */18 */
18#ifndef CORE_NET_HTTP_IMPL_CURL_EASY_H_19#ifndef CORE_NET_HTTP_IMPL_CURL_EASY_H_
19#define CORE_NET_HTTP_IMPL_CURL_EASY_H_20#define CORE_NET_HTTP_IMPL_CURL_EASY_H_
@@ -136,7 +137,10 @@
136 timeout_ms = CURLOPT_TIMEOUT_MS,137 timeout_ms = CURLOPT_TIMEOUT_MS,
137 ssl_engine_default = CURLOPT_SSLENGINE_DEFAULT,138 ssl_engine_default = CURLOPT_SSLENGINE_DEFAULT,
138 ssl_verify_peer = CURLOPT_SSL_VERIFYPEER,139 ssl_verify_peer = CURLOPT_SSL_VERIFYPEER,
139 ssl_verify_host = CURLOPT_SSL_VERIFYHOST140 ssl_verify_host = CURLOPT_SSL_VERIFYHOST,
141 customrequest = CURLOPT_CUSTOMREQUEST,
142 low_speed_limit = CURLOPT_LOW_SPEED_LIMIT,
143 low_speed_time = CURLOPT_LOW_SPEED_TIME
140};144};
141145
142namespace native146namespace native
@@ -209,6 +213,9 @@
209// Executes the operation configured on the handle.213// Executes the operation configured on the handle.
210::curl::Code perform(Handle handle);214::curl::Code perform(Handle handle);
211215
216// Executes pause operation on the handle.
217::curl::Code pause(Handle handle, int bitmask);
218
212// URL escapes the given input string.219// URL escapes the given input string.
213std::string escape(Handle handle, const std::string& in);220std::string escape(Handle handle, const std::string& in);
214221
@@ -339,6 +346,12 @@
339 // Executes the operation associated with this handle.346 // Executes the operation associated with this handle.
340 void perform();347 void perform();
341348
349 // Executes pause operation associated with this handle.
350 void pause();
351
352 // Executes resume operation associated with this handle.
353 void resume();
354
342 // URL escapes the given input string.355 // URL escapes the given input string.
343 std::string escape(const std::string& in);356 std::string escape(const std::string& in);
344357
345358
=== modified file 'src/core/net/http/impl/curl/request.h'
--- src/core/net/http/impl/curl/request.h 2015-03-23 16:09:05 +0000
+++ src/core/net/http/impl/curl/request.h 2016-05-23 10:51:40 +0000
@@ -14,6 +14,7 @@
14 * along with this program. If not, see <http://www.gnu.org/licenses/>.14 * along with this program. If not, see <http://www.gnu.org/licenses/>.
15 *15 *
16 * Authored by: Thomas Voß <thomas.voss@canonical.com>16 * Authored by: Thomas Voß <thomas.voss@canonical.com>
17 * Gary Wang <gary.wang@canonical.com>
17 */18 */
18#ifndef CORE_NET_HTTP_IMPL_CURL_REQUEST_H_19#ifndef CORE_NET_HTTP_IMPL_CURL_REQUEST_H_
19#define CORE_NET_HTTP_IMPL_CURL_REQUEST_H_20#define CORE_NET_HTTP_IMPL_CURL_REQUEST_H_
@@ -273,6 +274,28 @@
273 multi.add(easy);274 multi.add(easy);
274 }275 }
275276
277 void pause()
278 {
279 try
280 {
281 easy.pause();
282 } catch(const std::system_error& se)
283 {
284 throw core::net::http::Error(se.what(), CORE_FROM_HERE());
285 }
286 }
287
288 void resume()
289 {
290 try
291 {
292 easy.resume();
293 } catch(const std::system_error& se)
294 {
295 throw core::net::http::Error(se.what(), CORE_FROM_HERE());
296 }
297 }
298
276 std::string url_escape(const std::string& s)299 std::string url_escape(const std::string& s)
277 {300 {
278 return easy.escape(s);301 return easy.escape(s);
@@ -282,6 +305,16 @@
282 {305 {
283 return easy.unescape(s);306 return easy.unescape(s);
284 }307 }
308
309 void abort_request_if(std::uint64_t limit, const std::chrono::seconds& time)
310 {
311 if (atomic_state.load() != core::net::http::Request::State::ready)
312 throw core::net::http::Request::Errors::AlreadyActive{CORE_FROM_HERE()};
313
314 easy.set_option(::curl::Option::low_speed_limit, limit);
315 easy.set_option(::curl::Option::low_speed_time, time.count());
316 }
317
285private:318private:
286 std::atomic<core::net::http::Request::State> atomic_state;319 std::atomic<core::net::http::Request::State> atomic_state;
287 ::curl::multi::Handle multi;320 ::curl::multi::Handle multi;
288321
=== modified file 'tests/http_client_test.cpp'
--- tests/http_client_test.cpp 2014-11-13 12:57:24 +0000
+++ tests/http_client_test.cpp 2016-05-23 10:51:40 +0000
@@ -14,6 +14,7 @@
14 * along with this program. If not, see <http://www.gnu.org/licenses/>.14 * along with this program. If not, see <http://www.gnu.org/licenses/>.
15 *15 *
16 * Authored by: Thomas Voß <thomas.voss@canonical.com>16 * Authored by: Thomas Voß <thomas.voss@canonical.com>
17 * Gary Wang <gary.wang@canonical.com>
17 */18 */
1819
19#include <core/net/error.h>20#include <core/net/error.h>
@@ -30,6 +31,7 @@
30#include <json/json.h>31#include <json/json.h>
3132
32#include <future>33#include <future>
34#include <fstream>
3335
34namespace http = core::net::http;36namespace http = core::net::http;
35namespace json = Json;37namespace json = Json;
@@ -456,6 +458,33 @@
456 EXPECT_EQ("test", root["form"]["test"].asString());458 EXPECT_EQ("test", root["form"]["test"].asString());
457}459}
458460
461TEST(HttpClient, post_request_for_file_with_large_chunk_succeeds)
462{
463 auto client = http::make_client();
464 auto url = std::string(httpbin::host) + httpbin::resources::post();
465
466 // create temp file with large chunk
467 const std::size_t size = 1024*1024;
468 std::ofstream ofs("tmp.dat", std::ios::binary | std::ios::out);
469 ofs.seekp(size);
470 ofs.write("", 1);
471 ofs.close();
472
473 std::ifstream payload("tmp.dat");
474 auto request = client->post(http::Request::Configuration::from_uri_as_string(url),
475 payload,
476 size);
477
478 json::Value root;
479 json::Reader reader;
480
481 auto response = request->execute(default_progress_reporter);
482
483 EXPECT_EQ(core::net::http::Status::ok, response.status);
484 EXPECT_TRUE(reader.parse(response.body, root));
485 EXPECT_EQ(url, root["url"].asString());
486}
487
459TEST(HttpClient, put_request_for_existing_resource_succeeds)488TEST(HttpClient, put_request_for_existing_resource_succeeds)
460{489{
461 auto client = http::make_client();490 auto client = http::make_client();
@@ -478,6 +507,50 @@
478 EXPECT_EQ(payload.str(), root["data"].asString());507 EXPECT_EQ(payload.str(), root["data"].asString());
479}508}
480509
510TEST(HttpClient, put_request_for_file_with_large_chunk_succeeds)
511{
512 auto client = http::make_client();
513 auto url = std::string(httpbin::host) + httpbin::resources::put();
514
515 // create temp file with large chunk
516 const std::size_t size = 1024*1024;
517 std::ofstream ofs("tmp.dat", std::ios::binary | std::ios::out);
518 ofs.seekp(size);
519 ofs.write("", 1);
520 ofs.close();
521
522 std::ifstream payload("tmp.dat");
523 auto request = client->put(http::Request::Configuration::from_uri_as_string(url),
524 payload,
525 size);
526
527 json::Value root;
528 json::Reader reader;
529
530 auto response = request->execute(default_progress_reporter);
531
532 EXPECT_EQ(core::net::http::Status::ok, response.status);
533 EXPECT_TRUE(reader.parse(response.body, root));
534 EXPECT_EQ(url, root["url"].asString());
535}
536
537TEST(HttpClient, del_request_for_existing_resource_succeeds)
538{
539 auto client = http::make_client();
540 auto url = std::string(httpbin::host) + httpbin::resources::del();
541
542 auto request = client->del(http::Request::Configuration::from_uri_as_string(url));
543
544 json::Value root;
545 json::Reader reader;
546
547 auto response = request->execute(default_progress_reporter);
548
549 EXPECT_EQ(core::net::http::Status::ok, response.status);
550 EXPECT_TRUE(reader.parse(response.body, root));
551 EXPECT_EQ(url, root["url"].asString());
552}
553
481namespace com554namespace com
482{555{
483namespace mozilla556namespace mozilla
484557
=== modified file 'tests/http_streaming_client_test.cpp'
--- tests/http_streaming_client_test.cpp 2015-03-23 16:24:27 +0000
+++ tests/http_streaming_client_test.cpp 2016-05-23 10:51:40 +0000
@@ -14,6 +14,7 @@
14 * along with this program. If not, see <http://www.gnu.org/licenses/>.14 * along with this program. If not, see <http://www.gnu.org/licenses/>.
15 *15 *
16 * Authored by: Thomas Voß <thomas.voss@canonical.com>16 * Authored by: Thomas Voß <thomas.voss@canonical.com>
17 * Gary Wang <gary.wang@canonical.com>
17 */18 */
1819
19#include <core/net/error.h>20#include <core/net/error.h>
@@ -33,6 +34,8 @@
33#include <future>34#include <future>
34#include <memory>35#include <memory>
3536
37#include <fstream>
38
36namespace http = core::net::http;39namespace http = core::net::http;
37namespace json = Json;40namespace json = Json;
38namespace net = core::net;41namespace net = core::net;
@@ -480,6 +483,37 @@
480 EXPECT_EQ("test", root["form"]["test"].asString());483 EXPECT_EQ("test", root["form"]["test"].asString());
481}484}
482485
486TEST(StreamingHttpClient, post_request_for_file_with_large_chunk_succeeds)
487{
488 using namespace ::testing;
489
490 auto client = http::make_streaming_client();
491 auto url = std::string(httpbin::host) + httpbin::resources::post();
492
493 // create temp file with large chunk
494 const std::size_t size = 1024*1024;
495 std::ofstream ofs("tmp.dat", std::ios::binary | std::ios::out);
496 ofs.seekp(size);
497 ofs.write("", 1);
498 ofs.close();
499
500 std::ifstream payload("tmp.dat");
501 auto request = client->streaming_post(http::Request::Configuration::from_uri_as_string(url),
502 payload,
503 size);
504
505 auto dh = MockDataHandler::create(); EXPECT_CALL(*dh, on_new_data(_)).Times(AtLeast(1));
506
507 auto response = request->execute(default_progress_reporter, dh->to_data_handler());
508
509 json::Value root;
510 json::Reader reader;
511
512 EXPECT_EQ(core::net::http::Status::ok, response.status);
513 EXPECT_TRUE(reader.parse(response.body, root));
514 EXPECT_EQ(url, root["url"].asString());
515}
516
483TEST(StreamingHttpClient, put_request_for_existing_resource_succeeds)517TEST(StreamingHttpClient, put_request_for_existing_resource_succeeds)
484{518{
485 using namespace ::testing;519 using namespace ::testing;
@@ -497,12 +531,66 @@
497 // Our mocked data handler.531 // Our mocked data handler.
498 auto dh = MockDataHandler::create(); EXPECT_CALL(*dh, on_new_data(_)).Times(AtLeast(1));532 auto dh = MockDataHandler::create(); EXPECT_CALL(*dh, on_new_data(_)).Times(AtLeast(1));
499533
534 auto response = request->execute(default_progress_reporter, dh->to_data_handler());
535
500 json::Value root;536 json::Value root;
501 json::Reader reader;537 json::Reader reader;
502538
503 auto response = request->execute(default_progress_reporter, dh->to_data_handler());
504
505 EXPECT_EQ(core::net::http::Status::ok, response.status);539 EXPECT_EQ(core::net::http::Status::ok, response.status);
506 EXPECT_TRUE(reader.parse(response.body, root));540 EXPECT_TRUE(reader.parse(response.body, root));
507 EXPECT_EQ(payload.str(), root["data"].asString());541 EXPECT_EQ(payload.str(), root["data"].asString());
508}542}
543
544TEST(StreamingHttpClient, put_request_for_file_with_large_chunk_succeeds)
545{
546 using namespace ::testing;
547
548 auto client = http::make_streaming_client();
549 auto url = std::string(httpbin::host) + httpbin::resources::put();
550
551 // create temp file with large chunk
552 const std::size_t size = 1024*1024;
553 std::ofstream ofs("tmp.dat", std::ios::binary | std::ios::out);
554 ofs.seekp(size);
555 ofs.write("", 1);
556 ofs.close();
557
558 std::ifstream payload("tmp.dat");
559 auto request = client->streaming_put(http::Request::Configuration::from_uri_as_string(url),
560 payload,
561 size);
562
563 auto dh = MockDataHandler::create(); EXPECT_CALL(*dh, on_new_data(_)).Times(AtLeast(1));
564
565 auto response = request->execute(default_progress_reporter, dh->to_data_handler());
566
567 json::Value root;
568 json::Reader reader;
569
570 EXPECT_EQ(core::net::http::Status::ok, response.status);
571 EXPECT_TRUE(reader.parse(response.body, root));
572 EXPECT_EQ(url, root["url"].asString());
573}
574
575TEST(StreamingHttpClient, del_request_for_existing_resource_succeeds)
576{
577 using namespace ::testing;
578
579 auto client = http::make_streaming_client();
580 auto url = std::string(httpbin::host) + httpbin::resources::del();
581
582 auto request = client->streaming_del(http::Request::Configuration::from_uri_as_string(url));
583
584 // Our mocked data handler.
585 auto dh = MockDataHandler::create(); EXPECT_CALL(*dh, on_new_data(_)).Times(AtLeast(1));
586
587 auto response = request->execute(default_progress_reporter, dh->to_data_handler());
588
589 json::Value root;
590 json::Reader reader;
591
592 EXPECT_EQ(core::net::http::Status::ok, response.status);
593 EXPECT_TRUE(reader.parse(response.body, root));
594 EXPECT_EQ(url, root["url"].asString());
595}
596
509597
=== modified file 'tests/httpbin.h.in'
--- tests/httpbin.h.in 2014-05-22 11:50:55 +0000
+++ tests/httpbin.h.in 2016-05-23 10:51:40 +0000
@@ -14,6 +14,7 @@
14 * along with this program. If not, see <http://www.gnu.org/licenses/>.14 * along with this program. If not, see <http://www.gnu.org/licenses/>.
15 *15 *
16 * Authored by: Thomas Voß <thomas.voss@canonical.com>16 * Authored by: Thomas Voß <thomas.voss@canonical.com>
17 * Gary Wang <gary.wang@canonical.com>
17 */18 */
1819
19#ifndef HTTPBIN_H_20#ifndef HTTPBIN_H_
@@ -104,6 +105,11 @@
104{105{
105 return "/put";106 return "/put";
106}107}
108/** Returns DELETE data. */
109const char* del()
110{
111 return "/delete";
112}
107/** Challenges basic authentication. */113/** Challenges basic authentication. */
108const char* basic_auth()114const char* basic_auth()
109{115{

Subscribers

People subscribed via source and target branches

to all changes: