Merge lp:~hingo/drizzle/drizzle-json_server-keyvalue into lp:drizzle

Proposed by Henrik Ingo
Status: Merged
Merged at revision: 2557
Proposed branch: lp:~hingo/drizzle/drizzle-json_server-keyvalue
Merge into: lp:drizzle
Diff against target: 1576 lines (+1147/-82)
10 files modified
drizzled/plugin/client/cached.h (+7/-4)
drizzled/sql/result_set.h (+41/-11)
drizzled/sql/result_set_meta_data.h (+24/-7)
m4/pandora_have_libevent.m4 (+13/-0)
plugin/json_server/docs/index.rst (+373/-11)
plugin/json_server/json/json_value.cpp (+41/-4)
plugin/json_server/json/value.h (+2/-1)
plugin/json_server/json_server.cc (+637/-43)
plugin/json_server/plugin.ac (+8/-0)
plugin/json_server/plugin.ini (+1/-1)
To merge this branch: bzr merge lp:~hingo/drizzle/drizzle-json_server-keyvalue
Reviewer Review Type Date Requested Status
Henrik Ingo Needs Resubmitting
Review via email: mp+105792@code.launchpad.net

Description of the change

This is 0.2 of json_server plugin. It adds a "pure json" key value api at the uri /json/. (The previous sql-over-http api is at URI /sql/.)

The new functionality still lacks tests, and all of this lacks documentation. We are already working on both, however it would be good to merge this now, because it is easier for us to work against trunk. We will shortly follow up with more tests and docs.

To post a comment you must log in.
Revision history for this message
Brian Aker (brianaker) wrote :
Download full text (97.8 KiB)

Started by upstream project "drizzle-build" build number 1994
Building remotely on ubuntu-11.04-slicehost-174.143.253.46 in workspace /home/jenkins/workspace/drizzle-build-ubuntu-debug

Deleting project workspace... done

Cleaning workspace...
$ bzr branch lp:drizzle/build /home/jenkins/workspace/drizzle-build-ubuntu-debug
You have not informed bzr of your Launchpad ID, and you must do this to
write to Launchpad or access private data. See "bzr help launchpad-login".
Branched 2556 revision(s).
Getting local revision...
$ bzr revision-info -d /home/jenkins/workspace/drizzle-build-ubuntu-debug
info result: bzr revision-info -d /home/jenkins/workspace/drizzle-build-ubuntu-debug returned 0. Command output: "2556 <email address hidden>
" stderr: ""
RevisionState revno:2556 revid:<email address hidden>
[drizzle-build-ubuntu-debug] $ /bin/sh -xe /tmp/hudson1749308941142382677.sh
+ ./config/autorun.sh
: running `python config/pandora-plugin write'
: running `/usr/bin/autoreconf --install --force --verbose -Wall'
autoreconf: Entering directory `.'
autoreconf: configure.ac: not using Gettext
autoreconf: running: aclocal --force --warnings=all -I m4 --force
autoreconf: configure.ac: tracing
configure.ac:27: warning: The macro `AC_PROG_LIBTOOL' is obsolete.
configure.ac:27: You should run autoupdate.
aclocal.m4:123: AC_PROG_LIBTOOL is expanded from...
m4/pandora_libtool.m4:6: PANDORA_LIBTOOL is expanded from...
m4/pandora_canonical.m4:41: PANDORA_CANONICAL_TARGET is expanded from...
configure.ac:27: the top level
configure.ac:27: warning: The macro `AC_LANG_SAVE' is obsolete.
configure.ac:27: You should run autoupdate.
../../lib/autoconf/lang.m4:126: AC_LANG_SAVE is expanded from...
../../lib/m4sugar/m4sh.m4:598: AS_IF is expanded from...
../../lib/autoconf/general.m4:2019: AC_CACHE_VAL is expanded from...
../../lib/autoconf/general.m4:2040: AC_CACHE_CHECK is expanded from...
m4/pandora_header_stdcxx_98.m4:21: AC_CXX_HEADER_STDCXX_98 is expanded from...
m4/pandora_check_cxx_standard.m4:6: PANDORA_CHECK_CXX_STANDARD is expanded from...
configure.ac:27: warning: The macro `AC_LANG_CPLUSPLUS' is obsolete.
configure.ac:27: You should run autoupdate.
../../lib/autoconf/c.m4:252: AC_LANG_CPLUSPLUS is expanded from...
configure.ac:27: warning: The macro `AC_TRY_COMPILE' is obsolete.
configure.ac:27: You should run autoupdate.
../../lib/autoconf/general.m4:2602: AC_TRY_COMPILE is expanded from...
configure.ac:27: warning: The macro `AC_LANG_RESTORE' is obsolete.
configure.ac:27: You should run autoupdate.
../../lib/autoconf/lang.m4:135: AC_LANG_RESTORE is expanded from...
m4/pandora_cstdint.m4:12: PANDORA_CXX_CSTDINT is expanded from...
m4/pandora_cinttypes.m4:12: PANDORA_CXX_CINTTYPES is expanded from...
configure.ac:27: warning: The macro `AC_CHECK_LIBM' is obsolete.
configure.ac:27: You should run autoupdate.
aclocal.m4:3301: AC_CHECK_LIBM is expanded from...
m4/pandora_visibility.m4:23: PANDORA_CHECK_VISIBILITY is expanded from...
m4/pandora_visibility.m4:71: PANDORA_ENABLE_VISIBILITY is expanded from...
configure.ac:27: warning: AC_RUN_IFELSE called without default to allow cross compiling
../../lib/autocon...

Revision history for this message
Henrik Ingo (hingo) wrote :

Right. json_server now requires version 2.0 of libevent library. I've now pushed an addition to pandora_have_libevent.m4 that adds a PANDORA_LIBEVENT_RECENT check. Basically, you need Ubuntu Oneiric or newer to have libevent 2.0, otherwise json_server will not be built. (I don't know about Fedora, on Centos I think they still have too old libevent.)

Also pushed the documentation that we finished on Friday.

review: Needs Resubmitting

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'drizzled/plugin/client/cached.h'
--- drizzled/plugin/client/cached.h 2011-12-23 09:24:47 +0000
+++ drizzled/plugin/client/cached.h 2012-05-20 15:31:19 +0000
@@ -47,17 +47,20 @@
47 virtual void sendFields(List<Item>& list)47 virtual void sendFields(List<Item>& list)
48 {48 {
49 List<Item>::iterator it(list.begin());49 List<Item>::iterator it(list.begin());
50
51 column= 0;50 column= 0;
52 max_column= 0;51 max_column= 0;
5352
54 while (Item* item= it++)53 while (Item* item= it++)
55 {54 {
56 SendField field;55 // max_column starts from 0, ColumnCount from 1
56 _result_set->setColumnCount(max_column+1);
57 // Get pointer to next column metadata class
58 SendField field = _result_set->getColumnInfo(max_column);
57 item->make_field(&field);59 item->make_field(&field);
60 // Is this necessary or does it just set the same pointer back?
61 _result_set->setColumnInfo(max_column, field);
58 max_column++;62 max_column++;
59 }63 }
60 _result_set->setColumnCount(max_column);
61 // Moved to checkRowBegin()64 // Moved to checkRowBegin()
62 //_result_set->createRow();65 //_result_set->createRow();
63 }66 }
@@ -75,7 +78,7 @@
75 }78 }
76 }79 }
7780
78virtual void checkRowEnd()81 virtual void checkRowEnd()
79 {82 {
80 column++;83 column++;
81 }84 }
8285
=== modified file 'drizzled/sql/result_set.h'
--- drizzled/sql/result_set.h 2011-04-21 05:05:53 +0000
+++ drizzled/sql/result_set.h 2012-05-20 15:31:19 +0000
@@ -34,6 +34,7 @@
34#include <drizzled/visibility.h>34#include <drizzled/visibility.h>
35#include <drizzled/sql/exception.h>35#include <drizzled/sql/exception.h>
36#include <drizzled/sql/result_set_meta_data.h>36#include <drizzled/sql/result_set_meta_data.h>
37#include <drizzled/field.h>
37#include <cassert>38#include <cassert>
38#include <queue>39#include <queue>
3940
@@ -94,19 +95,48 @@
94 bool error() const;95 bool error() const;
95 sql::Exception getException() const;96 sql::Exception getException() const;
9697
97 ResultSet(size_t fields) :98 ResultSet(size_t columns) :
98 _has_next_been_called(false),99 _has_next_been_called(false),
99 _current_row(_results.end()),100 _current_row(_results.end()),
100 _meta_data(fields)101 _meta_data(columns)
101 {102 {
102 }103 }
103104
104 void setColumnCount(size_t fields)105 void setColumnCount(size_t columns)
105 {106 {
106 _meta_data.setColumnCount(fields);107 _meta_data.setColumnCount(columns);
107 }108 }
108109
109 ~ResultSet();110 void setColumnInfo(size_t column_number, const SendField& field)
111 {
112 _meta_data.setColumnInfo(column_number, field);
113 }
114
115 /**
116 * @brief Get object that holds column meta data.
117 *
118 * The following info is available:
119 *
120 * metadata = rs.getColumnInfo(0);
121 * metadata.db_name;
122 * metadata.org_table_name;
123 * metadata.org_col_name;
124 * metadata.table_name;
125 * metadata.col_name;
126 * metadata.charsetnr;
127 * metadata.flags;
128 * metadata.type;
129 * metadata.length;
130 * metadata.decimals;
131 *
132 * @see drizzled/item.cc to see where these are set.
133 */
134 SendField getColumnInfo(size_t column_number)
135 {
136 return _meta_data.getColumnInfo(column_number);
137 }
138
139~ResultSet();
110140
111 void createRow();141 void createRow();
112 void setColumn(size_t column_number, const std::string &arg);142 void setColumn(size_t column_number, const std::string &arg);
113143
=== modified file 'drizzled/sql/result_set_meta_data.h'
--- drizzled/sql/result_set_meta_data.h 2011-04-21 05:05:53 +0000
+++ drizzled/sql/result_set_meta_data.h 2012-05-20 15:31:19 +0000
@@ -33,10 +33,13 @@
3333
34#include <cassert>34#include <cassert>
35#include <queue>35#include <queue>
36#include <vector>
37#include <drizzled/field.h>
3638
37namespace drizzled {39namespace drizzled {
38namespace sql {40namespace sql {
3941
42
40class ResultSetMetaData43class ResultSetMetaData
41{44{
42public:45public:
@@ -51,17 +54,31 @@
51 ResultSetMetaData(size_t columns) :54 ResultSetMetaData(size_t columns) :
52 _columns(columns)55 _columns(columns)
53 {56 {
54 }57 _columnInfo.resize(columns);
5558 }
56 void setColumnCount(size_t fields)59
57 {60 void setColumnCount(size_t columns)
58 _columns= fields;61 {
59 }62 _columns= columns;
6063 _columnInfo.resize(columns);
64 }
65
66 void setColumnInfo(size_t column_number, const SendField& field)
67 {
68 _columnInfo.at(column_number) = field;
69 }
70
71 SendField getColumnInfo(size_t column_number)
72 {
73 return _columnInfo.at(column_number);
74 }
75
61private: // Member methods76private: // Member methods
6277
63private: // Member variables78private: // Member variables
64 size_t _columns;79 size_t _columns;
80 typedef std::vector<SendField> ColumnInfoVector;
81 ColumnInfoVector _columnInfo;
65};82};
6683
67std::ostream& operator<<(std::ostream& output, const ResultSetMetaData &result_set);84std::ostream& operator<<(std::ostream& output, const ResultSetMetaData &result_set);
6885
=== modified file 'm4/pandora_have_libevent.m4'
--- m4/pandora_have_libevent.m4 2011-03-07 16:42:18 +0000
+++ m4/pandora_have_libevent.m4 2012-05-20 15:31:19 +0000
@@ -64,3 +64,16 @@
64AC_DEFUN([PANDORA_REQUIRE_LIBEVENT],[64AC_DEFUN([PANDORA_REQUIRE_LIBEVENT],[
65 AC_REQUIRE([_PANDORA_REQUIRE_LIBEVENT])65 AC_REQUIRE([_PANDORA_REQUIRE_LIBEVENT])
66])66])
67
68AC_DEFUN([PANDORA_LIBEVENT_RECENT],[
69 dnl FIXME I really wanted to check for existence of EVHTTP_REQ_DELETE,
70 dnl but autoconf gods were not favorable to me, so the below is all I got
71 AC_CACHE_CHECK([if libevent is recent enough],
72 [pandora_cv_libevent_recent],
73 [AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[
74#include <event2/event.h>
75#include <event2/http.h>
76 ]])],
77 [pandora_cv_libevent_recent=yes],
78 [pandora_cv_libevent_recent=no])])
79])
6780
=== modified file 'plugin/json_server/docs/index.rst'
--- plugin/json_server/docs/index.rst 2011-10-23 05:45:09 +0000
+++ plugin/json_server/docs/index.rst 2012-05-20 15:31:19 +0000
@@ -3,7 +3,12 @@
3JSON Server3JSON Server
4===========4===========
55
6JSON HTTP interface.6JSON Server implements a simple HTTP server that allows you to access your
7Drizzle database with JSON based protocols. Currently two API's are supported:
8a SQL-over-HTTP protocol allows you to execute any single statement SQL
9transactions and a pure JSON protocol currently supports storing of JSON
10documents as blobs in a key-value table.
11
712
8.. _json_server_loading:13.. _json_server_loading:
914
@@ -51,30 +56,380 @@
5156
52 :Scope: Global57 :Scope: Global
53 :Dynamic: No58 :Dynamic: No
54 :Option: :option:`--json-server.port`
5559
56 Port number to use for connection or 0 for default (port 8086) 60 Port number to use for connection or 0 for default (port 8086)
5761
58.. _json_server_examples:62.. _json_server_apis:
5963
60Examples64APIs
61--------65----
6266
63Sorry, there are no examples for this plugin.67JSON Server supports a few APIs that offer different functionalities. Each API
68is accessed via it's own URI, and parameters can be given in the query string
69or in the POST data.
70
71The APIs are versioned, the version number is prepended to the API name. If
72functionality is added or changed, it will not be available if an API is
73accessed via an earlier version number. Finally, the latest version of each API
74is also available from the root, without any version number.
75
76As of this writing, the following APIs exist:
77
78.. code-block:: none
79
80 /0.1/sql
81 /0.2/sql
82 /sql
83
84Because the SQL API did not change between 0.1 and 0.2, all of the above URIs
85are exactly the same.
86
87.. code-block:: none
88
89 /0.2/json
90 /json
91
92The pure JSON API did not exist in the 0.1 release, as you can see from above.
93
94.. code-block:: none
95
96 /version
97 /
98
99The ``/version`` URI will return the version of Drizzle (in a JSON document, of
100course):
101
102.. code-block:: none
103
104 $ curl http://localhost:8086/version
105 {
106 "version" : "7.1.31.2451-snapshot"
107 }
108
109The root URI returns a simple HTML GUI that can be used to test both the SQL and
110pure JSON APIs. Just point your browser to http://localhost:8086/ and try it!
111
112.. _json_server_sql_api:
113
114The SQL-over-HTTP API: /sql
115---------------------------
116
117The first API in JSON Server is the SQL-over-HTTP API. It allows you to execute
118almost any SQL and the result is returned as a 2 dimensional JSON array.
119
120On the HTTP level this is a simple API. The method is always ``POST`` and the
121functionality is determined by the SQL statement you send.
122
123.. code-block:: none
124
125 POST /sql
126
127 SELECT * from test.foo;
128
129Returns:
130
131.. code-block:: none
132
133 {
134 "query" : "SELECT * from test.foo;\n",
135 "result_set" : [
136 [ "1", "Hello Drizzle Day Audience!" ],
137 [ "2", "this text came in over http" ]
138 ],
139 "sqlstate" : "00000"
140 }
141
142The above corresponds to the following query from a drizzle command line:
143
144.. code-block:: mysql
145
146 drizzle> select * from test.foo;
147
148+----+-----------------------------+
149| id | bar |
150+====+=============================+
151| 1 | Hello Drizzle Day Audience! |
152+----+-----------------------------+
153| 2 | this text came in over http |
154+----+-----------------------------+
155
156
157.. _json_server_json_api:
158
159Pure JSON key-value API: /json
160------------------------------
161
162The pure JSON key-value API is found at the URI ``/json``. It takes a rather
163opposite approach than the ``/sql`` API. Queries are expressed as JSON query
164documents, similar to what is found in Metabase, CouchDB or MongoDB. It is not
165possible to use any SQL.
166
167The purpose of the ``/json`` API is to use Drizzle as a key-value document
168storage. This means that the table layout is determined by the JSON Server
169module. Therefore, it is not possible for the user to access arbitrary
170relational tables via the ``/json`` API, rather tables must adhere to the
171format explained further below, and it must contain valid JSON documents in the
172data columns.
173
174If you post (insert) a document to a table that doesn't exist, it will be
175automatically created. For this reason, a user mostly doesn't need to even
176know the specific format of a JSON server table.
177
178.. _json_server_json_parameters:
179
180Parameters
181^^^^^^^^^^
182
183Following parameters can be passed in the URI query string:
184
185.. _json_server_json_parameters_id:
186
187``_id``
188
189 :Type: Unsigned integer
190 :Mandatory: No
191 :Default:
192
193 Optionally, a user may also specify the _id value which is requested.
194 Typically this is given in the JSON query document instead. If both are given
195 the _id value in the query document has precedence.
196
197.. _json_server_json_parameters_query:
198
199``query``
200
201 :Type: JSON query document
202 :Mandatory: No
203 :Default:
204
205 A JSON document, the so called *query document*. This document specifies
206 which records/documents to return from the database. Currently it is only
207 possible to query for a single value by the primary key, which is
208 called ``_id``. Any other fields in the query document will be ignored.
209
210 The query parameter is used for GET, PUT and DELETE where it is passed in
211 URL encoded form in the URI query string. For POST requests the query
212 document is passed as the POST data. (In that case only the query document
213 is passed, there is no ``query=`` part, in other words the data is never
214 encoded in www-form-urlencoded format.)
215
216 Example query document:
217
218 .. code-block:: none
219
220 { "_id" : 1 }
221
222.. _json_server_json_parameters_schema:
223
224``schema``
225
226 :Type: String
227 :Mandatory: No
228 :Default: test
229
230 Name of the schema which we are querying. The schema must exist.
231
232.. _json_server_parameters_table:
233
234``table``
235
236 :Type: String
237 :Mandatory: No
238 :Default: jsonkv
239
240 Name of the table which we are querying. For POST requests, if the table
241 doesn't exist, it will be automatically created. For other requests the
242 table must exist, otherwise an error is returned.
243
244POSTing a document
245^^^^^^^^^^^^^^^^^^
246
247.. code-block:: none
248
249 POST /json?schema=test&table=people HTTP/1.1
250
251 {
252 "_id" : 2,
253 "document" : { "firstname" : "Henrik", "lastname" : "Ingo", "age" : 35}
254 }
255
256Returns:
257
258.. code-block:: none
259
260 HTTP/1.1 200 OK
261 Content-Type: text/html
262
263 {
264 "query" : {
265 "_id" : 2,
266 "document" : {
267 "age" : 35,
268 "firstname" : "Henrik",
269 "lastname" : "Ingo"
270 }
271 },
272 "sqlstate" : "00000"
273 }
274
275
276(The use of Content-type: text/html is considered a bug and will be
277fixed in a future version.)
278
279Under the hood, this has inserted the following record into a table "jsontable":
280
281.. code-block:: mysql
282
283 drizzle> select * from people where _id=2;
284
285+-----+--------------------------+
286| _id | document |
287+=====+==========================+
288| 2 |{ |
289| |"age" : 35, |
290| |"firstname" : "Henrik", |
291| |"lastname" : "Ingo" |
292| |} |
293+-----+--------------------------+
294
295The ``_id`` field is always present. If it isn't specified, an auto_increment
296value will be generated. If a record with the given ``_id`` already exists in
297the table, the record will be updated (using REPLACE INTO).
298
299In addition there are one or more columns of type TEXT.
300The column name(s) corresponds to the top level key(s) that were specified in the
301POSTed JSON document. You can use any name(s) for the top level key(s), but
302the name ``document`` is commonly used as a generic name. The contents of such a
303column is the value of the corresponding top level key and has to be valid JSON.
304
305A table of this format is automatically created when the first document is
306POSTed to the table. This means that the column names are defined from the top
307level key(s) of that first document and future JSON documents must use the same
308top level key(s). Below the top level key(s) the JSON document can be of any
309arbitrary structure. A common practice is to always use ``_id`` and ``document``
310as the top level keys, and place the actual JSON document, which can be of
311arbitrary structure, under the ``document`` key.
312
313
314GET a document
315^^^^^^^^^^^^^^
316
317The equivalent of an SQL SELECT is HTTP GET.
318
319Below we use the query document ``{"_id" : 1 }`` in URL encoded form:
320
321.. code-block:: none
322
323 GET /json?schema=test&table=people&query=%7B%22_id%22%20%3A%201%7D%0A
324
325Returns
326
327.. code-block:: none
328
329 HTTP/1.0 200 OK
330 Content-Type: text/html
331
332 {
333 "query" : {
334 "_id" : 1
335 },
336 "result_set" : [
337 {
338 "_id" : 1,
339 "document" : {
340 "age" : 21,
341 "firstname" : "Mohit",
342 "lastname" : "Srivastava"
343 }
344 }
345 ],
346 "sqlstate" : "00000"
347 }
348
349It is also allowed to specify the ``_id`` as a URI query string parameter and
350omit the query document:
351
352.. code-block:: none
353
354 GET /json?schema=test&table=people&_id=1
355
356If both are specified, the query document takes precedence.
357
358Finally, it is possible to issue a GET request to a table without specifying
359neither the ``_id`` parameter or a query document. In this case all records of
360the whole table is returned.
361
362
363Updating a record
364^^^^^^^^^^^^^^^^^
365
366To update a record, POST new version of json document with same ``_id`` as an
367already existing record.
368
369(PUT is currently not supported, instead POST is used for both inserting and
370updating.)
371
372Deleting a record
373^^^^^^^^^^^^^^^^^
374
375Below we use the query document ``{"_id" : 1 }`` in URL encoded form:
376
377.. code-block:: none
378
379 DELETE http://14.139.228.217:8086/json?schema=test&table=people&query=%7B%22_id%22%20%3A%201%7D
380
381Returns:
382
383.. code-block:: none
384
385 HTTP/1.0 200 OK
386 Content-Type: text/html
387
388 {
389 "query" : {
390 "_id" : 1
391 },
392 "sqlstate" : "00000"
393 }
394
395It is also allowed to specify the ``_id`` as a URI query string parameter and
396omit the query document:
397
398.. code-block:: none
399
400 DELETE /json?schema=test&table=people&_id=1
401
402If both are specified, the query document takes precedence.
403
404.. _json_server_limitations:
405
406Limitations
407^^^^^^^^^^^
408
409The ``/sql`` and ``/json`` APIs are both feature complete, yet JSON Server is
410still an experimental module. There are known crashes, the module is still
411single threaded and there is no authentication... and that's just a start!
412These limitations are being worked on. For a full list of the current state of
413JSON Server, please follow
414`this launchpad blueprint <https://blueprints.launchpad.net/drizzle/+spec/json-server>`_.
415
416An inherent limitation is that each HTTP request is its own transaction. While
417it would be possible to support maintaining a complex SQL transaction over the
418span of multiple HTTP requests, we currently do not plan to support that.
64419
65.. _json_server_authors:420.. _json_server_authors:
66421
67Authors422Authors
68-------423-------
69424
70Stewart Smith425Stewart Smith, Henrik Ingo, Mohit Srivastava
71426
72.. _json_server_version:427.. _json_server_version:
73428
74Version429Version
75-------430-------
76431
77This documentation applies to **json_server 0.1**.432This documentation applies to **json_server 0.2**.
78433
79To see which version of the plugin a Drizzle server is running, execute:434To see which version of the plugin a Drizzle server is running, execute:
80435
@@ -87,4 +442,11 @@
87442
88v0.1443v0.1
89^^^^444^^^^
90* First release.445* /sql API
446* Simple web based GUI at /
447* /version API
448
449v0.2
450^^^^
451* /json API supporting pure JSON key-value operations (POST, GET, DELETE)
452* Automatic creation of table on first post.
91453
=== modified file 'plugin/json_server/json/json_value.cpp'
--- plugin/json_server/json/json_value.cpp 2011-08-20 13:41:35 +0000
+++ plugin/json_server/json/json_value.cpp 2012-05-20 15:31:19 +0000
@@ -40,6 +40,7 @@
40#include <plugin/json_server/json/value.h>40#include <plugin/json_server/json/value.h>
41#include <plugin/json_server/json/writer.h>41#include <plugin/json_server/json/writer.h>
4242
43#include <cstdio>
43#include <cassert>44#include <cassert>
44#include <cstring>45#include <cstring>
45#include <iostream>46#include <iostream>
@@ -308,6 +309,7 @@
308# ifdef JSON_VALUE_USE_INTERNAL_MAP309# ifdef JSON_VALUE_USE_INTERNAL_MAP
309 , itemIsUsed_( 0 )310 , itemIsUsed_( 0 )
310#endif311#endif
312 , value_as_string_( 0 )
311{313{
312 switch ( type_arg )314 switch ( type_arg )
313 {315 {
@@ -351,6 +353,7 @@
351# ifdef JSON_VALUE_USE_INTERNAL_MAP353# ifdef JSON_VALUE_USE_INTERNAL_MAP
352 , itemIsUsed_( 0 )354 , itemIsUsed_( 0 )
353#endif355#endif
356 , value_as_string_( 0 )
354{357{
355 value_.int_ = value;358 value_.int_ = value;
356}359}
@@ -362,6 +365,7 @@
362# ifdef JSON_VALUE_USE_INTERNAL_MAP365# ifdef JSON_VALUE_USE_INTERNAL_MAP
363 , itemIsUsed_( 0 )366 , itemIsUsed_( 0 )
364#endif367#endif
368 , value_as_string_( 0 )
365{369{
366 value_.uint_ = value;370 value_.uint_ = value;
367}371}
@@ -372,6 +376,7 @@
372# ifdef JSON_VALUE_USE_INTERNAL_MAP376# ifdef JSON_VALUE_USE_INTERNAL_MAP
373 , itemIsUsed_( 0 )377 , itemIsUsed_( 0 )
374#endif378#endif
379 , value_as_string_( 0 )
375{380{
376 value_.real_ = value;381 value_.real_ = value;
377}382}
@@ -383,6 +388,7 @@
383# ifdef JSON_VALUE_USE_INTERNAL_MAP388# ifdef JSON_VALUE_USE_INTERNAL_MAP
384 , itemIsUsed_( 0 )389 , itemIsUsed_( 0 )
385#endif390#endif
391 , value_as_string_( 0 )
386{392{
387 value_.string_ = valueAllocator()->duplicateStringValue( value );393 value_.string_ = valueAllocator()->duplicateStringValue( value );
388}394}
@@ -396,6 +402,7 @@
396# ifdef JSON_VALUE_USE_INTERNAL_MAP402# ifdef JSON_VALUE_USE_INTERNAL_MAP
397 , itemIsUsed_( 0 )403 , itemIsUsed_( 0 )
398#endif404#endif
405 , value_as_string_( 0 )
399{406{
400 value_.string_ = valueAllocator()->duplicateStringValue( beginValue, 407 value_.string_ = valueAllocator()->duplicateStringValue( beginValue,
401 UInt(endValue - beginValue) );408 UInt(endValue - beginValue) );
@@ -409,6 +416,7 @@
409# ifdef JSON_VALUE_USE_INTERNAL_MAP416# ifdef JSON_VALUE_USE_INTERNAL_MAP
410 , itemIsUsed_( 0 )417 , itemIsUsed_( 0 )
411#endif418#endif
419 , value_as_string_( 0 )
412{420{
413 value_.string_ = valueAllocator()->duplicateStringValue( value.c_str(), 421 value_.string_ = valueAllocator()->duplicateStringValue( value.c_str(),
414 (unsigned int)value.length() );422 (unsigned int)value.length() );
@@ -422,6 +430,7 @@
422# ifdef JSON_VALUE_USE_INTERNAL_MAP430# ifdef JSON_VALUE_USE_INTERNAL_MAP
423 , itemIsUsed_( 0 )431 , itemIsUsed_( 0 )
424#endif432#endif
433 , value_as_string_( 0 )
425{434{
426 value_.string_ = const_cast<char *>( value.c_str() );435 value_.string_ = const_cast<char *>( value.c_str() );
427}436}
@@ -435,6 +444,7 @@
435# ifdef JSON_VALUE_USE_INTERNAL_MAP444# ifdef JSON_VALUE_USE_INTERNAL_MAP
436 , itemIsUsed_( 0 )445 , itemIsUsed_( 0 )
437#endif446#endif
447 , value_as_string_( 0 )
438{448{
439 value_.string_ = valueAllocator()->duplicateStringValue( value, value.length() );449 value_.string_ = valueAllocator()->duplicateStringValue( value, value.length() );
440}450}
@@ -446,6 +456,7 @@
446# ifdef JSON_VALUE_USE_INTERNAL_MAP456# ifdef JSON_VALUE_USE_INTERNAL_MAP
447 , itemIsUsed_( 0 )457 , itemIsUsed_( 0 )
448#endif458#endif
459 , value_as_string_( 0 )
449{460{
450 value_.bool_ = value;461 value_.bool_ = value;
451}462}
@@ -457,6 +468,7 @@
457# ifdef JSON_VALUE_USE_INTERNAL_MAP468# ifdef JSON_VALUE_USE_INTERNAL_MAP
458 , itemIsUsed_( 0 )469 , itemIsUsed_( 0 )
459#endif470#endif
471 , value_as_string_( 0 )
460{472{
461 switch ( type_ )473 switch ( type_ )
462 {474 {
@@ -504,7 +516,6 @@
504 }516 }
505}517}
506518
507
508Value::~Value()519Value::~Value()
509{520{
510 switch ( type_ )521 switch ( type_ )
@@ -514,7 +525,9 @@
514 case uintValue:525 case uintValue:
515 case realValue:526 case realValue:
516 case booleanValue:527 case booleanValue:
517 break;528 if ( value_as_string_ )
529 valueAllocator()->releaseStringValue( value_as_string_ );
530 break;
518 case stringValue:531 case stringValue:
519 if ( allocated_ )532 if ( allocated_ )
520 valueAllocator()->releaseStringValue( value_.string_ );533 valueAllocator()->releaseStringValue( value_.string_ );
@@ -715,9 +728,12 @@
715 return value_.string_;728 return value_.string_;
716}729}
717730
718731/**
732 * If type_ is not stringValue, we will here convert other values to value_as_string_ field,
733 * then return it.
734 */
719std::string 735std::string
720Value::asString() const736Value::asString()
721{737{
722 switch ( type_ )738 switch ( type_ )
723 {739 {
@@ -728,8 +744,29 @@
728 case booleanValue:744 case booleanValue:
729 return value_.bool_ ? "true" : "false";745 return value_.bool_ ? "true" : "false";
730 case intValue:746 case intValue:
747 if(!value_as_string_)
748 {
749 char buf[64];
750 sprintf(buf, "%d", value_.int_);
751 value_as_string_ = valueAllocator()->duplicateStringValue( buf );
752 }
753 return value_as_string_;
731 case uintValue:754 case uintValue:
755 if(!value_as_string_)
756 {
757 char buf[64];
758 sprintf(buf, "%d", value_.uint_);
759 value_as_string_ = valueAllocator()->duplicateStringValue( buf );
760 }
761 return value_as_string_;
732 case realValue:762 case realValue:
763 if(!value_as_string_)
764 {
765 char buf[256];
766 sprintf(buf, "%f", value_.real_);
767 value_as_string_ = valueAllocator()->duplicateStringValue( buf );
768 }
769 return value_as_string_;
733 case arrayValue:770 case arrayValue:
734 case objectValue:771 case objectValue:
735 JSON_ASSERT_MESSAGE( false, "Type is not convertible to string" );772 JSON_ASSERT_MESSAGE( false, "Type is not convertible to string" );
736773
=== modified file 'plugin/json_server/json/value.h'
--- plugin/json_server/json/value.h 2011-08-19 09:52:50 +0000
+++ plugin/json_server/json/value.h 2012-05-20 15:31:19 +0000
@@ -263,7 +263,7 @@
263 int compare( const Value &other );263 int compare( const Value &other );
264264
265 const char *asCString() const;265 const char *asCString() const;
266 std::string asString() const;266 std::string asString();
267# ifdef JSON_USE_CPPTL267# ifdef JSON_USE_CPPTL
268 CppTL::ConstString asConstString() const;268 CppTL::ConstString asConstString() const;
269# endif269# endif
@@ -481,6 +481,7 @@
481 int memberNameIsStatic_ : 1; // used by the ValueInternalMap container.481 int memberNameIsStatic_ : 1; // used by the ValueInternalMap container.
482# endif482# endif
483 CommentInfo *comments_;483 CommentInfo *comments_;
484 char *value_as_string_; // Used when asString is called on non-string types.
484 };485 };
485486
486487
487488
=== modified file 'plugin/json_server/json_server.cc'
--- plugin/json_server/json_server.cc 2012-01-16 02:37:54 +0000
+++ plugin/json_server/json_server.cc 2012-05-20 15:31:19 +0000
@@ -1,7 +1,7 @@
1/* - mode: c; c-basic-offset: 2; indent-tabs-mode: nil; -*-1/* - mode: c; c-basic-offset: 2; indent-tabs-mode: nil; -*-
2 * vim:expandtab:shiftwidth=2:tabstop=2:smarttab:2 * vim:expandtab:shiftwidth=2:tabstop=2:smarttab:
3 *3 *
4 * Copyright (C) 2011 Stewart Smith4 * Copyright (C) 2011 Stewart Smith, Henrik Ingo, Mohit Srivastava
5 *5 *
6 * This program is free software; you can redistribute it and/or modify6 * This program is free software; you can redistribute it and/or modify
7 * it under the terms of the GNU General Public License as published by7 * it under the terms of the GNU General Public License as published by
@@ -17,7 +17,21 @@
17 * along with this program; if not, write to the Free Software17 * along with this program; if not, write to the Free Software
18 * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA18 * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
19 */19 */
2020/**
21 * @file Implements an HTTP server that will parse JSON and SQL queries
22 *
23 * @todo Refactoring ideas:
24 * - Anything HTML should really be a separate file, not strings embedded
25 * in C++.
26 * - Put all json handling into try/catch blocks, the parser likes to throw
27 * exceptions which crash drizzled if not caught.
28 * - Need to implement a worker thread pool. Make workers proper OO classes.
29 *
30 * @todo Implement HTTP response codes other than just 200 as defined in
31 * http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html
32 *
33 * @todo Shouldn't we be using event2/http.h? Why does this even work without it?
34 */
2135
22#include <config.h>36#include <config.h>
2337
@@ -76,7 +90,11 @@
76extern "C" void process_root_request(struct evhttp_request *req, void* );90extern "C" void process_root_request(struct evhttp_request *req, void* );
77extern "C" void process_api01_version_req(struct evhttp_request *req, void* );91extern "C" void process_api01_version_req(struct evhttp_request *req, void* );
78extern "C" void process_api01_sql_req(struct evhttp_request *req, void* );92extern "C" void process_api01_sql_req(struct evhttp_request *req, void* );
7993extern "C" void process_api02_json_req(struct evhttp_request *req, void* );
94extern "C" void process_api02_json_get_req(struct evhttp_request *req, void* );
95extern "C" void process_api02_json_post_req(struct evhttp_request *req, void* );
96/* extern "C" void process_api02_json_put_req(struct evhttp_request *req, void* ); */
97extern "C" void process_api02_json_delete_req(struct evhttp_request *req, void* );
80extern "C" void process_request(struct evhttp_request *req, void* )98extern "C" void process_request(struct evhttp_request *req, void* )
81{99{
82 struct evbuffer *buf = evbuffer_new();100 struct evbuffer *buf = evbuffer_new();
@@ -92,40 +110,78 @@
92110
93 std::string output;111 std::string output;
94112
95 output.append("<html><head><title>JSON DATABASE interface demo</title></head>"113 output.append("<html><head><title>JSON DATABASE interface demo</title></head>\n"
96 "<body>"114 "<body>\n"
97 "<script lang=\"javascript\">"115 "<script lang=\"javascript\">\n"
98 "function to_table(obj) {"116 "function to_table(obj) {\n"
99 " var str = '<table>';"117 " var str = '<table border=\"1\">';\n"
100 "for (var r=0; r< obj.length; r++) {"118 "for (var r=0; r< obj.length; r++) {\n"
101 " str+='<tr>';"119 " str+='<tr>';\n"
102 " for (var c=0; c < obj[r].length; c++) {"120 " for (var c=0; c < obj[r].length; c++) {\n"
103 " str+= '<td>' + obj[r][c] + '</td>';"121 " str+= '<td>' + obj[r][c] + '</td>';\n"
104 " }"122 " }\n"
105 " str+='</tr>';"123 " str+='</tr>';\n"
106 "}"124 "}\n"
107 "str+='</table>';"125 "str+='</table>';\n"
108 "return str;"126 "return str;\n"
109 "}"127 "}\n"
110 "function run_query()\n"128 "function to_table_from_json(obj) {\n"
111 "{"129 " var str = '<table border=\"1\">';\n"
112 "var url = document.getElementById(\"baseurl\").innerHTML;\n"130 "for (var r=0; r< obj.length; r++) {\n"
113 "var query= document.getElementById(\"query\").value;\n"131 " str+='<tr>';\n"
132 " str+='<td>' + obj[r]['_id'] + '</td>';\n"
133 " str+='<td>' + JSON.stringify(obj[r]['document']) + '</td>';\n"
134 " str+='</tr>';\n"
135 "}\n"
136 "str+='</table>';\n"
137 "return str;\n"
138 "}\n"
139 "function run_sql_query()\n"
140 "{\n"
141 "var url = window.location;\n"
142 "var query= document.getElementById(\"sql_query\").value;\n"
114 "var xmlHttp = new XMLHttpRequest();\n"143 "var xmlHttp = new XMLHttpRequest();\n"
115 "xmlHttp.onreadystatechange = function () {\n"144 "xmlHttp.onreadystatechange = function () {\n"
145 "document.getElementById(\"responseText\").value = xmlHttp.responseText;\n"
116 "if (xmlHttp.readyState == 4 && xmlHttp.status == 200) {\n"146 "if (xmlHttp.readyState == 4 && xmlHttp.status == 200) {\n"
117 "var info = eval ( \"(\" + xmlHttp.responseText + \")\" );\n"147 "var info = eval ( \"(\" + xmlHttp.responseText + \")\" );\n"
118 "document.getElementById( \"resultset\").innerHTML= to_table(info.result_set);\n"148 "document.getElementById( \"resultset\").innerHTML= to_table(info.result_set);\n"
119 "}\n"149 "}\n"
120 "};\n"150 "};\n"
121 "xmlHttp.open(\"POST\", url + \"/0.1/sql\", true);"151 "xmlHttp.open(\"POST\", url + \"sql\", true);\n"
122 "xmlHttp.send(query);"152 "xmlHttp.send(query);\n"
123 "}"153 "}\n"
154 "\n\n"
155 "function run_json_query()\n"
156 "{\n"
157//"alert('run_json_query');"
158 "var url = window.location;\n"
159 "var method= document.getElementById(\"json_method\").value;\n"
160 "var query= document.getElementById(\"json_query\").value;\n"
161 "var schema= document.getElementById(\"schema\").value;\n"
162 "var table= document.getElementById(\"table\").value;\n"
163 "var xmlHttp = new XMLHttpRequest();\n"
164 "xmlHttp.onreadystatechange = function () {\n"
165//"alert(xmlHttp.responseText);"
166 "document.getElementById(\"responseText\").value = xmlHttp.responseText;\n"
167 "if (xmlHttp.readyState == 4 && xmlHttp.status == 200) {\n"
168 "var info = eval ( \"(\" + xmlHttp.responseText + \")\" );\n"
169 "document.getElementById( \"resultset\").innerHTML= to_table_from_json(info.result_set);\n"
170 "}\n"
171 "};\n"
172 "if( method == \"POST\" ) {\n"
173 "xmlHttp.open(method, url + \"json?schema=\" + schema + \"&table=\" + table, true);\n"
174 "xmlHttp.send(query);\n"
175 "} else {\n"
176 "xmlHttp.open(method, url + \"json?schema=\" + schema + \"&table=\" + table + \"&query=\" + encodeURIComponent(query), true);\n"
177 "xmlHttp.send();\n"
178 "}\n"
179 "}\n"
124 "\n\n"180 "\n\n"
125 "function update_version()\n"181 "function update_version()\n"
126 "{drizzle_version(document.getElementById(\"baseurl\").innerHTML);}\n\n"182 "{drizzle_version(window.location);}\n\n"
127 "function drizzle_version($url)"183 "function drizzle_version($url)\n"
128 "{"184 "{\n"
129 "var xmlHttp = new XMLHttpRequest();\n"185 "var xmlHttp = new XMLHttpRequest();\n"
130 "xmlHttp.onreadystatechange = function () {\n"186 "xmlHttp.onreadystatechange = function () {\n"
131 "if (xmlHttp.readyState == 4 && xmlHttp.status == 200) {\n"187 "if (xmlHttp.readyState == 4 && xmlHttp.status == 200) {\n"
@@ -133,19 +189,32 @@
133 "document.getElementById( \"drizzleversion\").innerHTML= info.version;\n"189 "document.getElementById( \"drizzleversion\").innerHTML= info.version;\n"
134 "}\n"190 "}\n"
135 "};\n"191 "};\n"
136 "xmlHttp.open(\"GET\", $url + \"/0.1/version\", true);"192 "xmlHttp.open(\"GET\", $url + \"version\", true);\n"
137 "xmlHttp.send(null);"193 "xmlHttp.send(null);\n"
138 "}"194 "}\n"
139 "</script>"195 "</script>\n"
140 "<p>Drizzle Server at: <a id=\"baseurl\">http://localhost:8765</a></p>"196 "<p>Drizzle server version: <a id=\"drizzleversion\"></a></p>\n"
141 "<p>Drizzle server version: <a id=\"drizzleversion\"></a></p>"197 "<p><textarea rows=\"3\" cols=\"80\" id=\"sql_query\">\n"
142 "<p><textarea rows=\"3\" cols=\"40\" id=\"query\">"198 "SELECT * from DATA_DICTIONARY.GLOBAL_STATUS;\n"
143 "SELECT * from DATA_DICTIONARY.GLOBAL_STATUS;"199 "</textarea>\n"
144 "</textarea>"200 "<button type=\"button\" onclick=\"run_sql_query();\">Execute SQL Query</button>\n"
145 "<button type=\"button\" onclick=\"run_query();\">Execute Query</button>"201 "</p><p>\n"
146 "<div id=\"resultset\"/>"202 "<textarea rows=\"8\" cols=\"80\" id=\"json_query\">\n"
147 "<script lang=\"javascript\">update_version(); run_query();</script>"203 "{\"_id\" : 1}\n"
148 "</body></html>");204 "</textarea>\n"
205 "<button type=\"button\" onclick=\"run_json_query();\">Execute JSON Query</button>\n"
206 "<br />\n"
207 "<select id=\"json_method\"><option value=\"GET\">GET</option>"
208 "<option value=\"POST\">POST</option>"
209 "<option value=\"PUT\">PUT</option>"
210 "<option value=\"DELETE\">DELETE</option></select>"
211 "<script lang=\"javascript\">document.write(window.location);</script>json?schema=\n"
212 "<input type=\"text\" id=\"schema\" value=\"test\"/>"
213 "&amp;table=<input type=\"text\" id=\"table\" value=\"jsonkv\"/>\n"
214 "</p><hr />\n<div id=\"resultset\"></div>\n"
215 "<hr /><p><textarea rows=\"12\" cols=\"80\" id=\"responseText\" ></textarea></p>"
216 "<script lang=\"javascript\">update_version(); run_sql_query();</script>\n"
217 "</body></html>\n");
149218
150 evbuffer_add(buf, output.c_str(), output.length());219 evbuffer_add(buf, output.c_str(), output.length());
151 evhttp_send_reply(req, HTTP_OK, "OK", buf);220 evhttp_send_reply(req, HTTP_OK, "OK", buf);
@@ -203,6 +272,7 @@
203 {272 {
204 root["error_message"]= exception.getErrorMessage();273 root["error_message"]= exception.getErrorMessage();
205 root["error_code"]= exception.getErrorCode();274 root["error_code"]= exception.getErrorCode();
275 root["schema"]= "test";
206 }276 }
207277
208 while (result_set.next())278 while (result_set.next())
@@ -227,6 +297,515 @@
227 evhttp_send_reply(req, HTTP_OK, "OK", buf);297 evhttp_send_reply(req, HTTP_OK, "OK", buf);
228}298}
229299
300extern "C" void process_api02_json_req(struct evhttp_request *req, void* )
301{
302 if( req->type == EVHTTP_REQ_GET )
303 {
304 process_api02_json_get_req( req, NULL);
305// } elseif ( req->type == EVHTTP_REQ_PUT ) {
306 //process_api02_json_put_req( req, NULL);
307 } else if ( req->type == EVHTTP_REQ_POST ) {
308 process_api02_json_post_req( req, NULL);
309 } else if ( req->type == EVHTTP_REQ_DELETE ) {
310 process_api02_json_delete_req( req, NULL);
311 }
312}
313
314/**
315 * Transform a HTTP GET to SELECT and return results based on input json document
316 *
317 * @todo allow DBA to set default schema (also in post,del methods)
318 * @todo allow DBA to set whether to use strict mode for parsing json (should get rid of white space), especially for POST of course.
319 *
320 * @param req should contain a "table" parameter in request uri. "query", "_id" and "schema" are optional.
321 * @return a json document is returned to client with evhttp_send_reply()
322 */
323void process_api02_json_get_req(struct evhttp_request *req, void* )
324{
325 int http_response_code = HTTP_OK;
326 const char *http_response_text;
327 http_response_text = "OK";
328
329 struct evbuffer *buf = evbuffer_new();
330 if (buf == NULL) return;
331
332 Json::Value json_out;
333
334 std::string input;
335 // Schema and table are given in request uri.
336 // TODO: If we want to be really NoSQL, we will some day allow to use synonyms like "collection" instead of "table".
337 // For GET, also the query is in the uri
338 const char *schema;
339 const char *table;
340 const char *query;
341 const char *id;
342 evhttp_parse_query(evhttp_request_uri(req), req->input_headers);
343 schema = (char *)evhttp_find_header(req->input_headers, "schema");
344 table = (char *)evhttp_find_header(req->input_headers, "table");
345 query = (char *)evhttp_find_header(req->input_headers, "query");
346 id = (char *)evhttp_find_header(req->input_headers, "_id");
347
348 // query can be null if _id was given
349 if ( query == NULL || strcmp(query, "") == 0 )
350 {
351 // Empty JSON object
352 query = "{}";
353 }
354 input.append(query, strlen(query));
355
356 // Set test as default schema
357 if ( !strcmp( schema, "") || schema == NULL)
358 {
359 schema = "test";
360 }
361
362 // Parse "input" into "json_in".
363 Json::Value json_in;
364 Json::Features json_conf;
365 Json::Reader reader(json_conf);
366 bool retval = reader.parse(input, json_in);
367 if (retval != true) {
368 json_out["error_type"]="json error";
369 json_out["error_message"]= reader.getFormatedErrorMessages();
370 }
371 else if (strcmp( table, "") == 0 || table == NULL) {
372 json_out["error_type"]="http error";
373 json_out["error_message"]= "You must specify \"table\" in the request uri query string.";
374 http_response_code = HTTP_NOTFOUND;
375 http_response_text = "You must specify \"table\" in the request uri query string.";
376 }
377 else {
378 // It is allowed to specify _id in the uri and leave it out from the json query.
379 // In that case we put the value from uri into json_in here.
380 // If both are specified, the one existing in json_in wins. (This is still valid, no error.)
381 if ( ! json_in["_id"].asBool() )
382 {
383 if( id ) {
384 json_in["_id"] = (Json::Value::UInt) atol(id);
385 }
386 }
387
388 // TODO: In a later stage we'll allow the situation where _id isn't given but some other column for where.
389 // TODO: Need to do json_in[].type() first and juggle it from there to be safe. See json/value.h
390 // TODO: Don't SELECT * but only fields given in json query document
391 char sqlformat[1024];;
392 char buffer[1024];
393 if ( json_in["_id"].asBool() )
394 {
395 // Now we build an SQL query, using _id from json_in
396 sprintf(sqlformat, "%s", "SELECT * FROM `%s`.`%s` WHERE _id=%i;");
397 sprintf(buffer, sqlformat, schema, table, json_in["_id"].asInt());
398 }
399 else {
400 // If neither _id nor query are given, we return the full table. (No error, maybe this is what you really want? Blame yourself.)
401 sprintf(sqlformat, "%s", "SELECT * FROM `%s`.`%s`;");
402 sprintf(buffer, sqlformat, schema, table);
403 }
404
405 std::string sql = "";
406 sql.append(buffer, strlen(buffer));
407
408 // We have sql string. Use Execute API to run it and convert results back to JSON.
409 drizzled::Session::shared_ptr _session= drizzled::Session::make_shared(drizzled::plugin::Listen::getNullClient(),
410 drizzled::catalog::local());
411 drizzled::identifier::user::mptr user_id= identifier::User::make_shared();
412 user_id->setUser("");
413 _session->setUser(user_id);
414 //_session->set_schema("test");
415
416 drizzled::Execute execute(*(_session.get()), true);
417
418 drizzled::sql::ResultSet result_set(1);
419
420 /* Execute wraps the SQL to run within a transaction */
421 execute.run(sql, result_set);
422 drizzled::sql::Exception exception= result_set.getException();
423
424 drizzled::error_t err= exception.getErrorCode();
425
426 json_out["sqlstate"]= exception.getSQLState();
427
428 if ((err != drizzled::EE_OK) && (err != drizzled::ER_EMPTY_QUERY))
429 {
430 json_out["error_type"]="sql error";
431 json_out["error_message"]= exception.getErrorMessage();
432 json_out["error_code"]= exception.getErrorCode();
433 json_out["internal_sql_query"]= sql;
434 json_out["schema"]= "test";
435 }
436
437 while (result_set.next())
438 {
439 Json::Value json_row;
440 bool got_error = false;
441 for (size_t x= 0; x < result_set.getMetaData().getColumnCount() && got_error == false; x++)
442 {
443 if (not result_set.isNull(x))
444 {
445 // The values are now serialized json. We must first
446 // parse them to make them part of this structure, only to immediately
447 // serialize them again in the next step. For large json documents
448 // stored into the blob this must be very, very inefficient.
449 // TODO: Implement a smarter way to push the blob value directly to the client. Probably need to hand code some string appending magic.
450 // TODO: Massimo knows of a library to create JSON in streaming mode.
451 Json::Value json_doc;
452 Json::Reader readrow(json_conf);
453 std::string col_name = result_set.getColumnInfo(x).col_name;
454 bool r = readrow.parse(result_set.getString(x), json_doc);
455 if (r != true) {
456 json_out["error_type"]="json parse error on row value";
457 json_out["error_internal_sql_column"]=col_name;
458 json_out["error_message"]= reader.getFormatedErrorMessages();
459 // Just put the string there as it is, better than nothing.
460 json_row[col_name]= result_set.getString(x);
461 got_error=true;
462 break;
463 }
464 else {
465 json_row[col_name]= json_doc;
466 }
467 }
468 }
469 // When done, append this to result set tree
470 json_out["result_set"].append(json_row);
471 }
472
473 json_out["query"]= json_in;
474 }
475 // Return either the results or an error message, in json.
476 Json::StyledWriter writer;
477 std::string output= writer.write(json_out);
478 evbuffer_add(buf, output.c_str(), output.length());
479 evhttp_send_reply(req, http_response_code, http_response_text, buf);
480}
481
482/**
483 * Input json document or update existing one from HTTP POST (and PUT?).
484 *
485 * If json document specifies _id field, then record is updated. If it doesn't
486 * exist, then a new record is created with that _id.
487 *
488 * If _id field is not specified, then a new record is created using
489 * auto_increment value. The _id of the created value is returned in the http
490 * response.
491 *
492 * @todo If there are multiple errors, last one overwrites the previous in json_out. Make them lists.
493 *
494 * @param req should contain a "table" parameter in request uri. "schema" is optional.
495 * @return a json document is returned to client with evhttp_send_reply()
496 */
497void process_api02_json_post_req(struct evhttp_request *req, void* )
498{
499 bool table_exists = true;
500 Json::Value json_out;
501
502 struct evbuffer *buf = evbuffer_new();
503 if (buf == NULL) return;
504
505 // Read from http to string "input".
506 std::string input;
507 char buffer[1024];
508 int l=0;
509 do {
510 l= evbuffer_remove(req->input_buffer, buffer, 1024);
511 input.append(buffer, l);
512 }while(l);
513
514 // Schema and table are given in request uri.
515 // TODO: If we want to be really NoSQL, we will some day allow to use synonyms like "collection" instead of "table".
516 const char *schema;
517 const char *table;
518 const char *id;
519 evhttp_parse_query(evhttp_request_uri(req), req->input_headers);
520 schema = (char *)evhttp_find_header(req->input_headers, "schema");
521 table = (char *)evhttp_find_header(req->input_headers, "table");
522 id = (char *)evhttp_find_header(req->input_headers, "_id");
523
524 // Set test as default schema
525 if ( !strcmp( schema, "") || schema == NULL)
526 {
527 schema = "test";
528 }
529
530 // Parse "input" into "json_in".
531 Json::Value json_in;
532 Json::Features json_conf;
533 Json::Reader reader(json_conf);
534 bool retval = reader.parse(input, json_in);
535 if (retval != true) {
536 json_out["error_type"]="json error";
537 json_out["error_message"]= reader.getFormatedErrorMessages();
538 }
539 else {
540 // It is allowed to specify _id in the uri and leave it out from the json query.
541 // In that case we put the value from uri into json_in here.
542 // If both are specified, the one existing in json_in wins. (This is still valid, no error.)
543 if ( ! json_in["_id"].asBool() )
544 {
545 if( id ) {
546 json_in["_id"] = (Json::Value::UInt) atol(id);
547 }
548 }
549
550 // For POST method, we check if table exists.
551 // If it doesn't, we automatically CREATE TABLE that matches the structure
552 // in the given json document. (This means, your first JSON document must
553 // contain all top-level keys you'd like to use.)
554 drizzled::Session::shared_ptr _session= drizzled::Session::make_shared(drizzled::plugin::Listen::getNullClient(),
555 drizzled::catalog::local());
556 drizzled::identifier::user::mptr user_id= identifier::User::make_shared();
557 user_id->setUser("");
558 _session->setUser(user_id);
559 drizzled::Execute execute(*(_session.get()), true);
560
561 drizzled::sql::ResultSet result_set(1);
562 std::string sql="select count(*) from information_schema.tables where table_schema = '";
563 sql.append(schema);
564 sql.append("' AND table_name = '");
565 sql.append(table); sql.append("';");
566 /* Execute wraps the SQL to run within a transaction */
567 execute.run(sql, result_set);
568
569 drizzled::sql::Exception exception= result_set.getException();
570
571 drizzled::error_t err= exception.getErrorCode();
572 while(result_set.next())
573 {
574 if(result_set.getString(0)=="0")
575 {
576 table_exists = false;
577 }
578 }
579 if(table_exists == false)
580 {
581 std::string tmp = "CREATE TABLE ";
582 tmp.append(schema);
583 tmp.append(".");
584 tmp.append(table);
585 tmp.append(" (_id BIGINT PRIMARY KEY auto_increment,");
586 // Iterate over json_in keys
587 Json::Value::Members createKeys( json_in.getMemberNames() );
588 for ( Json::Value::Members::iterator it = createKeys.begin(); it != createKeys.end(); ++it )
589 {
590 const std::string &key = *it;
591 if(key=="_id") {
592 continue;
593 }
594 tmp.append(key);
595 tmp.append(" TEXT");
596 if( it !=createKeys.end()-1 && key !="_id")
597 {
598 tmp.append(",");
599 }
600 }
601 tmp.append(")");
602 vector<string> csql;
603 csql.clear();
604 csql.push_back("COMMIT");
605 csql.push_back (tmp);
606 sql.clear();
607 BOOST_FOREACH(string& it, csql)
608 {
609 sql.append(it);
610 sql.append("; ");
611 }
612 drizzled::sql::ResultSet createtable_result_set(1);
613 execute.run(sql, createtable_result_set);
614
615 exception= createtable_result_set.getException();
616 err= exception.getErrorCode();
617 }
618 // Now we "parse" the json_in object and build an SQL query
619 sql.clear();
620 sql.append("REPLACE INTO `");
621 sql.append(schema);
622 sql.append("`.`");
623 sql.append(table);
624 sql.append("` SET ");
625 // Iterate over json_in keys
626 Json::Value::Members keys( json_in.getMemberNames() );
627 for ( Json::Value::Members::iterator it = keys.begin(); it != keys.end(); ++it )
628 {
629 if ( it != keys.begin() )
630 {
631 sql.append(", ");
632 }
633 // TODO: Need to do json_in[].type() first and juggle it from there to be safe. See json/value.h
634 const std::string &key = *it;
635 sql.append(key); sql.append("=");
636 Json::StyledWriter writeobject;
637 switch ( json_in[key].type() )
638 {
639 case Json::nullValue:
640 sql.append("NULL");
641 break;
642 case Json::intValue:
643 case Json::uintValue:
644 case Json::realValue:
645 case Json::booleanValue:
646 sql.append(json_in[key].asString());
647 break;
648 case Json::stringValue:
649 sql.append("'\"");
650 // TODO: MUST be sql quoted!
651 sql.append(json_in[key].asString());
652 sql.append("\"'");
653 break;
654 case Json::arrayValue:
655 case Json::objectValue:
656 sql.append("'");
657 sql.append(writeobject.write(json_in[key]));
658 sql.append("'");
659 break;
660 default:
661 sql.append("'Error in json_server.cc. This should never happen.'");
662 json_out["error_type"]="json error";
663 json_out["error_message"]= "json_in object had a value that wasn't of any of the types that we recognize.";
664 break;
665 }
666 sql.append(" ");
667 }
668 sql.append(";");
669 drizzled::sql::ResultSet replace_result_set(1);
670
671 // Execute wraps the SQL to run within a transaction
672 execute.run(sql, replace_result_set);
673
674 exception= replace_result_set.getException();
675
676 err= exception.getErrorCode();
677
678 json_out["sqlstate"]= exception.getSQLState();
679
680 // TODO: I should be able to return number of rows inserted/updated.
681 // TODO: Return last_insert_id();
682 if ((err != drizzled::EE_OK) && (err != drizzled::ER_EMPTY_QUERY))
683 {
684 json_out["error_type"]="sql error";
685 json_out["error_message"]= exception.getErrorMessage();
686 json_out["error_code"]= exception.getErrorCode();
687 json_out["internal_sql_query"]= sql;
688 json_out["schema"]= "test";
689 }
690 json_out["query"]= json_in;
691 }
692 // Return either the results or an error message, in json.
693 Json::StyledWriter writer;
694 std::string output= writer.write(json_out);
695 evbuffer_add(buf, output.c_str(), output.length());
696 evhttp_send_reply(req, HTTP_OK, "OK", buf);
697}
698
699/*
700void process_api02_json_put_req(struct evhttp_request *req, void* )
701{
702 struct evbuffer *buf = evbuffer_new();
703 if (buf == NULL) return;
704 evhttp_send_reply(req, HTTP_OK, "OK", buf);
705}
706*/
707
708void process_api02_json_delete_req(struct evhttp_request *req, void* )
709{
710 struct evbuffer *buf = evbuffer_new();
711 if (buf == NULL) return;
712
713 Json::Value json_out;
714
715 std::string input;
716 char buffer[1024];
717
718 // Schema and table are given in request uri.
719 // TODO: If we want to be really NoSQL, we will some day allow to use synonyms like "collection" instead of "table".
720 // For GET, also the query is in the uri
721 const char *schema;
722 const char *table;
723 const char *query;
724 const char *id;
725 evhttp_parse_query(evhttp_request_uri(req), req->input_headers);
726 schema = (char *)evhttp_find_header(req->input_headers, "schema");
727 table = (char *)evhttp_find_header(req->input_headers, "table");
728 query = (char *)evhttp_find_header(req->input_headers, "query");
729 id = (char *)evhttp_find_header(req->input_headers, "_id");
730
731 // query can be null if _id was given
732 if ( query == NULL || strcmp(query, "") == 0 )
733 {
734 // Empty JSON object
735 query = "{}";
736 }
737 input.append(query, strlen(query));
738
739 // Set test as default schema
740 if ( !strcmp( schema, "") || schema == NULL)
741 {
742 schema = "test";
743 }
744
745 // Parse "input" into "json_in".
746 Json::Value json_in;
747 Json::Features json_conf;
748 json_conf.strictMode();
749 Json::Reader reader(json_conf);
750 bool retval = reader.parse(input, json_in);
751 if (retval != true) {
752 json_out["error_type"]="json error";
753 json_out["error_message"]= reader.getFormatedErrorMessages();
754 }
755 else {
756 // It is allowed to specify _id in the uri and leave it out from the json query.
757 // In that case we put the value from uri into json_in here.
758 // If both are specified, the one existing in json_in wins. (This is still valid, no error.)
759 if ( ! json_in["_id"].asBool() )
760 {
761 if( id ) {
762 json_in["_id"] = (Json::Value::UInt) atol(id);
763 }
764 }
765 // Now we "parse" the json_in object and build an SQL query
766 char sqlformat[1024] = "DELETE FROM `%s`.`%s` WHERE _id=%i;";
767 sprintf(buffer, sqlformat, schema, table, json_in["_id"].asInt());
768 std::string sql = "";
769 sql.append(buffer, strlen(buffer));
770
771 // We have sql string. Use Execute API to run it and convert results back to JSON.
772 drizzled::Session::shared_ptr _session= drizzled::Session::make_shared(drizzled::plugin::Listen::getNullClient(),
773 drizzled::catalog::local());
774 drizzled::identifier::user::mptr user_id= identifier::User::make_shared();
775 user_id->setUser("");
776 _session->setUser(user_id);
777
778 drizzled::Execute execute(*(_session.get()), true);
779
780 drizzled::sql::ResultSet result_set(1);
781
782 /* Execute wraps the SQL to run within a transaction */
783 execute.run(sql, result_set);
784 drizzled::sql::Exception exception= result_set.getException();
785
786 drizzled::error_t err= exception.getErrorCode();
787
788 json_out["sqlstate"]= exception.getSQLState();
789
790 if ((err != drizzled::EE_OK) && (err != drizzled::ER_EMPTY_QUERY))
791 {
792 json_out["error_type"]="sql error";
793 json_out["error_message"]= exception.getErrorMessage();
794 json_out["error_code"]= exception.getErrorCode();
795 json_out["internal_sql_query"]= sql;
796 json_out["schema"]= "test";
797 }
798 json_out["query"]= json_in;
799 }
800 // Return either the results or an error message, in json.
801 Json::StyledWriter writer;
802 std::string output= writer.write(json_out);
803 evbuffer_add(buf, output.c_str(), output.length());
804 evhttp_send_reply(req, HTTP_OK, "OK", buf);
805
806 }
807
808
230static void shutdown_event(int fd, short, void *arg)809static void shutdown_event(int fd, short, void *arg)
231{810{
232 struct event_base *base= (struct event_base *)arg;811 struct event_base *base= (struct event_base *)arg;
@@ -302,10 +881,25 @@
302 return false;881 return false;
303 }882 }
304883
884 // These URLs are available. Bind worker method to each of them.
885 // Please group by api version. Also unchanged functions must be copied to next version!
305 evhttp_set_cb(httpd, "/", process_root_request, NULL);886 evhttp_set_cb(httpd, "/", process_root_request, NULL);
887 // API 0.1
306 evhttp_set_cb(httpd, "/0.1/version", process_api01_version_req, NULL);888 evhttp_set_cb(httpd, "/0.1/version", process_api01_version_req, NULL);
307 evhttp_set_cb(httpd, "/0.1/sql", process_api01_sql_req, NULL);889 evhttp_set_cb(httpd, "/0.1/sql", process_api01_sql_req, NULL);
308 evhttp_set_gencb(httpd, process_request, NULL);890 // API 0.2
891 evhttp_set_cb(httpd, "/0.2/version", process_api01_version_req, NULL);
892 evhttp_set_cb(httpd, "/0.2/sql", process_api01_sql_req, NULL);
893 evhttp_set_cb(httpd, "/0.2/json", process_api02_json_req, NULL);
894 // API "latest" and also available in top level
895 evhttp_set_cb(httpd, "/latest/version", process_api01_version_req, NULL);
896 evhttp_set_cb(httpd, "/latest/sql", process_api01_sql_req, NULL);
897 evhttp_set_cb(httpd, "/latest/json", process_api02_json_req, NULL);
898 evhttp_set_cb(httpd, "/version", process_api01_version_req, NULL);
899 evhttp_set_cb(httpd, "/sql", process_api01_sql_req, NULL);
900 evhttp_set_cb(httpd, "/json", process_api02_json_req, NULL);
901 // Catch all does nothing and returns generic message.
902 //evhttp_set_gencb(httpd, process_request, NULL);
309903
310 event_set(&wakeup_event, wakeup_fd[0], EV_READ | EV_PERSIST, shutdown_event, base);904 event_set(&wakeup_event, wakeup_fd[0], EV_READ | EV_PERSIST, shutdown_event, base);
311 event_base_set(base, &wakeup_event);905 event_base_set(base, &wakeup_event);
@@ -367,7 +961,7 @@
367 DRIZZLE_VERSION_ID,961 DRIZZLE_VERSION_ID,
368 "json_server",962 "json_server",
369 "0.1",963 "0.1",
370 "Stewart Smith",964 "Stewart Smith, Henrik Ingo, Mohit Srivastava",
371 N_("JSON HTTP interface"),965 N_("JSON HTTP interface"),
372 PLUGIN_LICENSE_GPL,966 PLUGIN_LICENSE_GPL,
373 drizzle_plugin::json_server::json_server_init,967 drizzle_plugin::json_server::json_server_init,
374968
=== modified file 'plugin/json_server/plugin.ac'
--- plugin/json_server/plugin.ac 2011-04-21 01:27:52 +0000
+++ plugin/json_server/plugin.ac 2012-05-20 15:31:19 +0000
@@ -1,1 +1,9 @@
1PANDORA_HAVE_LIBEVENT1PANDORA_HAVE_LIBEVENT
2
3AS_IF([test "x$ac_cv_libevent" = "xno"],
4 AC_MSG_WARN([libevent not found: not building json_server.]))
5
6PANDORA_LIBEVENT_RECENT
7
8AS_IF([test "$pandora_cv_libevent_recent" = "no"],
9 AC_MSG_WARN([Your version of libevent is too old. json_server requires v 2.0 or newer: not building json_server.]))
210
=== modified file 'plugin/json_server/plugin.ini'
--- plugin/json_server/plugin.ini 2012-01-14 22:02:03 +0000
+++ plugin/json_server/plugin.ini 2012-05-20 15:31:19 +0000
@@ -17,5 +17,5 @@
17 json/json_reader.cpp17 json/json_reader.cpp
18 json/json_value.cpp18 json/json_value.cpp
19 json/json_writer.cpp19 json/json_writer.cpp
20build_conditional="x${ac_cv_libevent}" = "xyes" -a "x${ac_cv_libcurl}" = "xyes"20build_conditional="x${ac_cv_libevent}" = "xyes" -a "x$pandora_cv_libevent_recent" = "xyes" -a "x${ac_cv_libcurl}" = "xyes"
21ldflags=${LTLIBEVENT}21ldflags=${LTLIBEVENT}

Subscribers

People subscribed via source and target branches