Merge lp:~tsarev/percona-server/5.1-18205_01_mysqlbinlog_fix.patch into lp:percona-server/5.5

Proposed by Oleg Tsarev
Status: Superseded
Proposed branch: lp:~tsarev/percona-server/5.1-18205_01_mysqlbinlog_fix.patch
Merge into: lp:percona-server/5.5
Diff against target: 5486343 lines (has conflicts)
Conflict adding file .bzrignore.  Moved existing file to .bzrignore.moved.
Path conflict: <deleted> / build
Path conflict: <deleted> / COPYING.show_temp_51
Path conflict: <deleted> / README.txt
Path conflict: <deleted> / lrusort.py
Conflict adding file HandlerSocket-Plugin-for-MySQL.  Moved existing file to HandlerSocket-Plugin-for-MySQL.moved.
Contents conflict in Makefile
Conflict adding file README.HandlerSocket.  Moved existing file to README.HandlerSocket.moved.
Conflict adding file UDF.  Moved existing file to UDF.moved.
Conflict adding files to build.  Created directory.
Conflict because build is not versioned, but has versioned children.  Versioned directory.
Conflict adding file build.  Moved existing file to build.moved.
Conflict adding files to build/debian.  Created directory.
Conflict because build/debian is not versioned, but has versioned children.  Versioned directory.
Contents conflict in build/debian/changelog
Contents conflict in build/debian/control
Contents conflict in build/debian/libmysqlclient-dev.docs
Contents conflict in build/debian/libmysqlclient-dev.files
Contents conflict in build/debian/libmysqlclient16.docs
Conflict adding files to build/debian/patches.  Created directory.
Conflict because build/debian/patches is not versioned, but has versioned children.  Versioned directory.
Contents conflict in build/debian/patches/00list
Contents conflict in build/debian/percona-server-client-5.1.docs
Contents conflict in build/debian/percona-server-client-5.1.files
Contents conflict in build/debian/percona-server-server-5.1.docs
Contents conflict in build/debian/percona-server-server-5.1.files
Contents conflict in build/debian/percona-server-server-5.1.links
Contents conflict in build/debian/percona-server-server-5.1.mysql.init
Contents conflict in build/debian/percona-server-server-5.1.postinst
Contents conflict in build/debian/percona-server-server-5.1.templates
Contents conflict in build/debian/rules
Contents conflict in build/percona-server.spec
Conflict adding file doc.  Moved existing file to doc.moved.
Conflict adding file handlersocket.patch.  Moved existing file to handlersocket.patch.moved.
Conflict adding file python-for-subunit2junitxml.  Moved existing file to python-for-subunit2junitxml.moved.
Conflict adding file subunit2junitxml.  Moved existing file to subunit2junitxml.moved.
To merge this branch: bzr merge lp:~tsarev/percona-server/5.1-18205_01_mysqlbinlog_fix.patch
Reviewer Review Type Date Requested Status
Percona core Pending
Review via email: mp+91876@code.launchpad.net

This proposal has been superseded by a proposal from 2012-02-07.

Description of the change

Make mysqlbinlog omit redundant `use` around BEGIN/SAVEPOINT/COMMIT/ROLLBACK in 5.0 binlogs
This is a merge of lp:percona-server/rnt-5.1 patch mysqlbinlog_fix.patch.

MySQL 5.0 does not flag BEGIN/SAVEPOINT/COMMIT/ROLLBACK statements in its binlogs with LOG_EVENT_SUPPRESS_USE_F like 5.1+ does. This causes unnecessary `use` statements around such statements when the binlog is dumped by mysqlbinlog.

Fix by always suppressing the output of `use` for these statements.

To post a comment you must log in.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== added file '.bzrignore'
2--- .bzrignore 1970-01-01 00:00:00 +0000
3+++ .bzrignore 2012-02-07 16:47:34 +0000
4@@ -0,0 +1,3 @@
5+build
6+.pc
7+Makefile.in
8
9=== renamed file '.bzrignore' => '.bzrignore.moved'
10=== added directory 'HandlerSocket-Plugin-for-MySQL'
11=== renamed directory 'HandlerSocket-Plugin-for-MySQL' => 'HandlerSocket-Plugin-for-MySQL.moved'
12=== added file 'HandlerSocket-Plugin-for-MySQL/AUTHORS'
13--- HandlerSocket-Plugin-for-MySQL/AUTHORS 1970-01-01 00:00:00 +0000
14+++ HandlerSocket-Plugin-for-MySQL/AUTHORS 2012-02-07 16:47:34 +0000
15@@ -0,0 +1,19 @@
16+Akira Higuchi (https://github.com/ahiguti)
17+ - developed HanderSocket plugin, libhsclient, and perl-Net-HandlerSocket
18+
19+Yoshinori Matsunobu (https://github.com/yoshinorim)
20+ - introduced autotools, added support for MySQL 5.5.6, added statistics
21+ variables
22+
23+Jeff Hodges (https://github.com/jmhodges)
24+ - fixed some autotools scripts
25+
26+Toru Yamaguchi (https://github.com/zigorou)
27+ - ported to MacOS X
28+
29+Moriyoshi Koizumi (https://github.com/moriyoshi)
30+ - fixed some autotools scripts
31+
32+takeda-at (https://github.com/takada-at)
33+ - added simple authorization function
34+
35
36=== added file 'HandlerSocket-Plugin-for-MySQL/COPYING'
37--- HandlerSocket-Plugin-for-MySQL/COPYING 1970-01-01 00:00:00 +0000
38+++ HandlerSocket-Plugin-for-MySQL/COPYING 2012-02-07 16:47:34 +0000
39@@ -0,0 +1,30 @@
40+-----------------------------------------------------------------------------
41+HandlerSocket plugin for MySQL
42+
43+ Copyright (c) 2010 DeNA Co.,Ltd.
44+ All rights reserved.
45+
46+ Redistribution and use in source and binary forms, with or without
47+ modification, are permitted provided that the following conditions are met:
48+
49+ * Redistributions of source code must retain the above copyright
50+ notice, this list of conditions and the following disclaimer.
51+ * Redistributions in binary form must reproduce the above copyright
52+ notice, this list of conditions and the following disclaimer in the
53+ documentation and/or other materials provided with the distribution.
54+ * Neither the name of DeNA Co.,Ltd. nor the names of its contributors
55+ may be used to endorse or promote products derived from this software
56+ without specific prior written permission.
57+
58+ THIS SOFTWARE IS PROVIDED BY DeNA Co.,Ltd. "AS IS" AND ANY EXPRESS OR
59+ IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
60+ MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO
61+ EVENT SHALL DeNA Co.,Ltd. BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
62+ SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
63+ PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS;
64+ OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
65+ WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
66+ OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
67+ ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
68+
69+
70
71=== added file 'HandlerSocket-Plugin-for-MySQL/ChangeLog'
72--- HandlerSocket-Plugin-for-MySQL/ChangeLog 1970-01-01 00:00:00 +0000
73+++ HandlerSocket-Plugin-for-MySQL/ChangeLog 2012-02-07 16:47:34 +0000
74@@ -0,0 +1,12 @@
75+1.0.6 - 2010-10-29
76+ * Changed build instruction (autoreconf/configure/make), removed auto-generated files (Contributed by jmhodges)
77+ *
78+
79+1.0.5 - 2010-10-18
80+ * Changed build procedures (using typical configure/make)
81+ * Supported 5.5.6
82+ * Added status variables
83+
84+1.0.4 - 2010-08-15
85+ * Initial public release
86+
87
88=== added file 'HandlerSocket-Plugin-for-MySQL/Makefile.am'
89--- HandlerSocket-Plugin-for-MySQL/Makefile.am 1970-01-01 00:00:00 +0000
90+++ HandlerSocket-Plugin-for-MySQL/Makefile.am 2012-02-07 16:47:34 +0000
91@@ -0,0 +1,87 @@
92+
93+ACLOCAL_AMFLAGS = -I m4
94+
95+SUBDIRS = @HANDLERSOCKET_SUBDIRS@
96+
97+perl:
98+ cd perl-Net-HandlerSocket && perl Makefile.PL && make
99+
100+install_perl:
101+ cd perl-Net-HandlerSocket && make install
102+
103+rpms: rpm_cli rpm_perl rpm_c
104+
105+rpm_dir:
106+ - mkdir dist
107+ - mkdir dist/BUILD dist/RPMS dist/SOURCES dist/SPECS dist/SRPMS
108+
109+rpm_cli: clean_cli rpm_dir
110+ sed -e "s/HANDLERSOCKET_VERSION/$(VERSION)/" \
111+ libhsclient/libhsclient.spec.template \
112+ > libhsclient/libhsclient.spec
113+ tar cvfz dist/libhsclient.tar.gz libhsclient
114+ rpmbuild --define "_topdir `pwd`/dist" -ta \
115+ dist/libhsclient.tar.gz
116+
117+rpm_perl: clean_perl rpm_dir
118+ sed -e "s/HANDLERSOCKET_VERSION/$(VERSION)/" \
119+ perl-Net-HandlerSocket/perl-Net-HandlerSocket.spec.template \
120+ > perl-Net-HandlerSocket/perl-Net-HandlerSocket.spec
121+ cd perl-Net-HandlerSocket && perl Makefile.PL && make clean && \
122+ rm -f Makefile.old
123+ tar cvfz dist/perl-Net-HandlerSocket.tar.gz perl-Net-HandlerSocket
124+ rpmbuild --define "_topdir `pwd`/dist" -ta \
125+ dist/perl-Net-HandlerSocket.tar.gz
126+
127+rpm_c: clean_c rpm_dir
128+ sed -e "s/HANDLERSOCKET_VERSION/$(VERSION)/" \
129+ handlersocket/handlersocket.spec.template \
130+ > handlersocket/handlersocket.spec
131+ sed -e "s|HANDLERSOCKET_MYSQL_INC|$(MYSQL_CFLAGS) $(MYSQL_INC)|" \
132+ -e "s|HANDLERSOCKET_MYSQL_LIB|$(MYSQL_LIB)|" \
133+ handlersocket/Makefile.plain.template \
134+ > handlersocket/Makefile.plain
135+ tar cvfz dist/handlersocket.tar.gz handlersocket
136+ rpmbuild --define "_topdir `pwd`/dist" -ta \
137+ dist/handlersocket.tar.gz
138+
139+install_rpm_pl:
140+ - sudo rpm -e perl-Net-HandlerSocket
141+ - sudo rpm -e perl-Net-HandlerSocket-debuginfo
142+ make clean
143+ make rpm_perl
144+ - sudo rpm -U dist/RPMS/*/perl*.rpm
145+
146+installrpms:
147+ - sudo rpm -e handlersocket
148+ - sudo rpm -e handlersocket-debuginfo
149+ - sudo rpm -e perl-Net-HandlerSocket
150+ - sudo rpm -e perl-Net-HandlerSocket-debuginfo
151+ - sudo rpm -e libhsclient
152+ - sudo rpm -e libhsclient-debuginfo
153+ make clean
154+ make rpm_cli
155+ - sudo rpm -U dist/RPMS/*/libhsclient*.rpm
156+ make clean
157+ make rpm_perl
158+ - sudo rpm -U dist/RPMS/*/perl*.rpm
159+ make clean
160+ make rpm_c
161+ - sudo rpm -U dist/RPMS/*/handlersocket*.rpm
162+
163+clean_cli:
164+ cd libhsclient && make clean
165+ cd client && make clean
166+
167+clean_perl:
168+ cd perl-Net-HandlerSocket && perl Makefile.PL && make clean && \
169+ rm -f Makefile.old
170+
171+clean_c:
172+ cd handlersocket && make clean
173+
174+clean_all: clean_cli clean_perl clean_c
175+ cd regtest && make clean
176+ rm -rf dist/*/*
177+ rm -f dist/*.tar.gz
178+
179
180=== added file 'HandlerSocket-Plugin-for-MySQL/README'
181--- HandlerSocket-Plugin-for-MySQL/README 1970-01-01 00:00:00 +0000
182+++ HandlerSocket-Plugin-for-MySQL/README 2012-02-07 16:47:34 +0000
183@@ -0,0 +1,76 @@
184+
185+-----------------------------------------------------------------------------
186+HandlerSocket plugin for MySQL
187+
188+Copyright (c) 2010 DeNA Co.,Ltd.
189+All rights reserved.
190+
191+Redistribution and use in source and binary forms, with or without
192+modification, are permitted provided that the following conditions are met:
193+
194+ * Redistributions of source code must retain the above copyright
195+ notice, this list of conditions and the following disclaimer.
196+ * Redistributions in binary form must reproduce the above copyright
197+ notice, this list of conditions and the following disclaimer in the
198+ documentation and/or other materials provided with the distribution.
199+ * Neither the name of DeNA Co.,Ltd. nor the names of its contributors
200+ may be used to endorse or promote products derived from this software
201+ without specific prior written permission.
202+
203+THIS SOFTWARE IS PROVIDED BY DeNA Co.,Ltd. "AS IS" AND ANY EXPRESS OR
204+IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
205+MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO
206+EVENT SHALL DeNA Co.,Ltd. BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
207+SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
208+PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS;
209+OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
210+WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
211+OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
212+ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
213+
214+
215+-----------------------------------------------------------------------------
216+About HandlerSocket
217+
218+HandlerSocket is a NoSQL plugin for MySQL. It works as a daemon inside the
219+mysqld process, accept tcp connections, and execute requests from clients.
220+HandlerSocket does not support SQL queries. Instead, it supports simple CRUD
221+operations on tables.
222+
223+Because of the following reasons, HandlerSocket is much faster than the
224+mysqld/libmysql pair in some circumstances:
225+
226+ - HandlerSocket manipulates data without parsing SQL, which causes less
227+ CPU usage.
228+ - HandlerSocket reads many requests from clients and executes their
229+ requests in bulk, which causes less CPU and disk usage.
230+ - HandlerSocket client/server protocol is more compact than the
231+ mysql/libmysql pair, which causes less network usage.
232+
233+The current version of HandlerSocket only works with GNU/Linux. The source
234+archive of HandlerSocket includes a C++ and a Perl client libraries.
235+Here is a list of client libraries for other languages:
236+
237+ - PHP
238+ http://openpear.org/package/Net_HandlerSocket
239+ http://github.com/tz-lom/HSPHP
240+ http://code.google.com/p/php-handlersocket/
241+ - Java
242+ http://code.google.com/p/hs4j/
243+ http://code.google.com/p/handlersocketforjava/
244+ - Python
245+ http://pypi.python.org/pypi/python-handler-socket
246+ https://code.launchpad.net/~songofacandy/+junk/pyhandlersocket
247+ - Ruby
248+ https://github.com/winebarrel/ruby-handlersocket
249+ https://github.com/miyucy/handlersocket
250+ - JavaScript
251+ https://github.com/koichik/node-handlersocket
252+ - Scala
253+ https://github.com/fujohnwang/hs2client
254+
255+The home of HandlerSocket is here:
256+ https://github.com/ahiguti/HandlerSocket-Plugin-for-MySQL
257+
258+More documents are available in docs-en/ and docs-ja/ directories.
259+
260
261=== added file 'HandlerSocket-Plugin-for-MySQL/autogen.sh'
262--- HandlerSocket-Plugin-for-MySQL/autogen.sh 1970-01-01 00:00:00 +0000
263+++ HandlerSocket-Plugin-for-MySQL/autogen.sh 2012-02-07 16:47:34 +0000
264@@ -0,0 +1,117 @@
265+#!/bin/sh
266+
267+warn() {
268+echo -e "\tWARNING: $@" 1>&2
269+}
270+
271+# init
272+
273+LIBTOOLIZE=libtoolize
274+ACLOCAL=aclocal
275+AUTOCONF=autoconf
276+AUTOHEADER=autoheader
277+AUTOMAKE=automake
278+
279+case `uname -s` in
280+Darwin)
281+LIBTOOLIZE=glibtoolize
282+;;
283+FreeBSD)
284+ACLOCAL_ARGS="$ACLOCAL_ARGS -I /usr/local/share/aclocal/"
285+;;
286+esac
287+
288+
289+# libtoolize
290+echo "Searching libtoolize..."
291+if [ `which $LIBTOOLIZE` ] ; then
292+echo -e "\tFOUND: libtoolize -> $LIBTOOLIZE"
293+else
294+warn "Cannot Found libtoolize... input libtool command"
295+ read LIBTOOLIZE
296+ LIBTOOLIZE=`which $LIBTOOLIZE`
297+ if [ `which $LIBTOOLIZE` ] ; then
298+echo -e "\tSET: libtoolize -> $LIBTOOLIZE"
299+ else
300+warn "$LIBTOOLIZE: Command not found."
301+ exit 1;
302+ fi
303+fi
304+
305+# aclocal
306+echo "Searching aclocal..."
307+if [ `which $ACLOCAL` ] ; then
308+echo -e "\tFOUND: aclocal -> $ACLOCAL"
309+else
310+warn "Cannot Found aclocal... input aclocal command"
311+ read ACLOCAL
312+ ACLOCAL=`which $ACLOCAL`
313+ if [ `which $ACLOCAL` ] ; then
314+echo -e "\tSET: aclocal -> $ACLOCAL"
315+ else
316+warn "$ACLOCAL: Command not found."
317+ exit 1;
318+ fi
319+fi
320+
321+# automake
322+echo "Searching automake..."
323+if [ `which $AUTOMAKE` ] ; then
324+echo -e "\tFOUND: automake -> $AUTOMAKE"
325+else
326+warn "Cannot Found automake... input automake command"
327+ read AUTOMAKE
328+ ACLOCAL=`which $AUTOMAKE`
329+ if [ `which $AUTOMAKE` ] ; then
330+echo -e "\tSET: automake -> $AUTOMAKE"
331+ else
332+warn "$AUTOMAKE: Command not found."
333+ exit 1;
334+ fi
335+fi
336+
337+# autoheader
338+echo "Searching autoheader..."
339+if [ `which $AUTOHEADER` ] ; then
340+echo -e "\tFOUND: autoheader -> $AUTOHEADER"
341+else
342+warn "Cannot Found autoheader... input autoheader command"
343+ read AUTOHEADER
344+ ACLOCAL=`which $AUTOHEADER`
345+ if [ `which $AUTOHEADER` ] ; then
346+echo -e "\tSET: autoheader -> $AUTOHEADER"
347+ else
348+warn "$AUTOHEADER: Command not found."
349+ exit 1;
350+ fi
351+fi
352+
353+# autoconf
354+echo "Searching autoconf..."
355+if [ `which $AUTOCONF` ] ; then
356+echo -e "\tFOUND: autoconf -> $AUTOCONF"
357+else
358+warn "Cannot Found autoconf... input autoconf command"
359+ read AUTOCONF
360+ ACLOCAL=`which $AUTOCONF`
361+ if [ `which $AUTOCONF` ] ; then
362+echo -e "\tSET: autoconf -> $AUTOCONF"
363+ else
364+warn "$AUTOCONF: Command not found."
365+ exit 1;
366+ fi
367+fi
368+
369+echo "Running libtoolize ..."
370+$LIBTOOLIZE --force --copy
371+echo "Running aclocal ..."
372+$ACLOCAL ${ACLOCAL_ARGS} -I .
373+echo "Running autoheader..."
374+$AUTOHEADER
375+echo "Running automake ..."
376+$AUTOMAKE --add-missing --copy
377+echo "Running autoconf ..."
378+$AUTOCONF
379+
380+mkdir -p m4
381+
382
383=== added directory 'HandlerSocket-Plugin-for-MySQL/client'
384=== added file 'HandlerSocket-Plugin-for-MySQL/client/Makefile.am'
385--- HandlerSocket-Plugin-for-MySQL/client/Makefile.am 1970-01-01 00:00:00 +0000
386+++ HandlerSocket-Plugin-for-MySQL/client/Makefile.am 2012-02-07 16:47:34 +0000
387@@ -0,0 +1,24 @@
388+AM_INCLUDES= -I../libhsclient
389+bin_PROGRAMS=hsclient
390+hsclient_SOURCES= hsclient.cpp
391+hsclient_LDFLAGS= -static -L../libhsclient -lhsclient
392+hsclient_CXXFLAGS= $(AM_INCLUDES)
393+
394+hstest: hstest.o
395+ $(CXX) $(CXXFLAGS) $(LFLAGS) hstest.o \
396+ -L../libhsclient/.libs -lhsclient $$(mysql_config --libs_r) \
397+ -o hstest
398+
399+hstest.o: hstest.cpp
400+ $(CXX) $(CXXFLAGS) $(AM_INCLUDES) $$(mysql_config --include) \
401+ -c hstest.cpp
402+
403+hslongrun: hslongrun.o
404+ $(CXX) $(CXXFLAGS) $(LFLAGS) hslongrun.o \
405+ -L../libhsclient/.libs -lhsclient $$(mysql_config --libs_r) \
406+ -o hslongrun
407+
408+hslongrun.o: hslongrun.cpp
409+ $(CXX) $(CXXFLAGS) $(AM_INCLUDES) $$(mysql_config --include) \
410+ -c hslongrun.cpp
411+
412
413=== added file 'HandlerSocket-Plugin-for-MySQL/client/hsclient.cpp'
414--- HandlerSocket-Plugin-for-MySQL/client/hsclient.cpp 1970-01-01 00:00:00 +0000
415+++ HandlerSocket-Plugin-for-MySQL/client/hsclient.cpp 2012-02-07 16:47:34 +0000
416@@ -0,0 +1,88 @@
417+
418+// vim:sw=2:ai
419+
420+#include "hstcpcli.hpp"
421+#include "string_util.hpp"
422+
423+namespace dena {
424+
425+int
426+hstcpcli_main(int argc, char **argv)
427+{
428+ config conf;
429+ parse_args(argc, argv, conf);
430+ socket_args sockargs;
431+ sockargs.set(conf);
432+ hstcpcli_ptr cli = hstcpcli_i::create(sockargs);
433+ const std::string dbname = conf.get_str("dbname", "hstest");
434+ const std::string table = conf.get_str("table", "hstest_table1");
435+ const std::string index = conf.get_str("index", "PRIMARY");
436+ const std::string fields = conf.get_str("fields", "k,v");
437+ const int limit = conf.get_int("limit", 0);
438+ const int skip = conf.get_int("skip", 0);
439+ std::vector<std::string> keys;
440+ std::vector<string_ref> keyrefs;
441+ size_t num_keys = 0;
442+ while (true) {
443+ const std::string conf_key = std::string("k") + to_stdstring(num_keys);
444+ const std::string k = conf.get_str(conf_key, "");
445+ const std::string kx = conf.get_str(conf_key, "x");
446+ if (k.empty() && kx == "x") {
447+ break;
448+ }
449+ ++num_keys;
450+ keys.push_back(k);
451+ }
452+ for (size_t i = 0; i < keys.size(); ++i) {
453+ const string_ref ref(keys[i].data(), keys[i].size());
454+ keyrefs.push_back(ref);
455+ }
456+ const std::string op = conf.get_str("op", "=");
457+ const string_ref op_ref(op.data(), op.size());
458+ cli->request_buf_open_index(0, dbname.c_str(), table.c_str(),
459+ index.c_str(), fields.c_str());
460+ cli->request_buf_exec_generic(0, op_ref, num_keys == 0 ? 0 : &keyrefs[0],
461+ num_keys, limit, skip, string_ref(), 0, 0);
462+ int code = 0;
463+ size_t numflds = 0;
464+ do {
465+ if (cli->request_send() != 0) {
466+ fprintf(stderr, "request_send: %s\n", cli->get_error().c_str());
467+ break;
468+ }
469+ if ((code = cli->response_recv(numflds)) != 0) {
470+ fprintf(stderr, "response_recv: %s\n", cli->get_error().c_str());
471+ break;
472+ }
473+ } while (false);
474+ cli->response_buf_remove();
475+ do {
476+ if ((code = cli->response_recv(numflds)) != 0) {
477+ fprintf(stderr, "response_recv: %s\n", cli->get_error().c_str());
478+ break;
479+ }
480+ while (true) {
481+ const string_ref *const row = cli->get_next_row();
482+ if (row == 0) {
483+ break;
484+ }
485+ printf("REC:");
486+ for (size_t i = 0; i < numflds; ++i) {
487+ const std::string val(row[i].begin(), row[i].size());
488+ printf(" %s", val.c_str());
489+ }
490+ printf("\n");
491+ }
492+ } while (false);
493+ cli->response_buf_remove();
494+ return 0;
495+}
496+
497+};
498+
499+int
500+main(int argc, char **argv)
501+{
502+ return dena::hstcpcli_main(argc, argv);
503+}
504+
505
506=== added file 'HandlerSocket-Plugin-for-MySQL/client/hslongrun.cpp'
507--- HandlerSocket-Plugin-for-MySQL/client/hslongrun.cpp 1970-01-01 00:00:00 +0000
508+++ HandlerSocket-Plugin-for-MySQL/client/hslongrun.cpp 2012-02-07 16:47:34 +0000
509@@ -0,0 +1,1041 @@
510+
511+// vim:sw=2:ai
512+
513+#include <signal.h>
514+#include <sys/time.h>
515+#include <stdio.h>
516+#include <string.h>
517+#include <vector>
518+#include <map>
519+#include <stdlib.h>
520+#include <memory>
521+#include <errno.h>
522+#include <mysql.h>
523+#include <time.h>
524+#include <sys/types.h>
525+#include <sys/stat.h>
526+#include <fcntl.h>
527+
528+#include "util.hpp"
529+#include "auto_ptrcontainer.hpp"
530+#include "socket.hpp"
531+#include "hstcpcli.hpp"
532+#include "string_util.hpp"
533+#include "mutex.hpp"
534+
535+namespace dena {
536+
537+struct auto_mysql : private noncopyable {
538+ auto_mysql() : db(0) {
539+ reset();
540+ }
541+ ~auto_mysql() {
542+ if (db) {
543+ mysql_close(db);
544+ }
545+ }
546+ void reset() {
547+ if (db) {
548+ mysql_close(db);
549+ }
550+ if ((db = mysql_init(0)) == 0) {
551+ fatal_exit("failed to initialize mysql client");
552+ }
553+ }
554+ operator MYSQL *() const { return db; }
555+ private:
556+ MYSQL *db;
557+};
558+
559+struct auto_mysql_res : private noncopyable {
560+ auto_mysql_res(MYSQL *db) {
561+ res = mysql_store_result(db);
562+ }
563+ ~auto_mysql_res() {
564+ if (res) {
565+ mysql_free_result(res);
566+ }
567+ }
568+ operator MYSQL_RES *() const { return res; }
569+ private:
570+ MYSQL_RES *res;
571+};
572+
573+struct auto_mysql_stmt : private noncopyable {
574+ auto_mysql_stmt(MYSQL *db) {
575+ stmt = mysql_stmt_init(db);
576+ }
577+ ~auto_mysql_stmt() {
578+ if (stmt) {
579+ mysql_stmt_close(stmt);
580+ }
581+ }
582+ operator MYSQL_STMT *() const { return stmt; }
583+ private:
584+ MYSQL_STMT *stmt;
585+};
586+
587+double
588+gettimeofday_double()
589+{
590+ struct timeval tv = { };
591+ if (gettimeofday(&tv, 0) != 0) {
592+ fatal_abort("gettimeofday");
593+ }
594+ return static_cast<double>(tv.tv_usec) / 1000000 + tv.tv_sec;
595+}
596+
597+struct record_value {
598+ mutex lock;
599+ bool deleted;
600+ bool unknown_state;
601+ std::string key;
602+ std::vector<std::string> values;
603+ record_value() : deleted(true), unknown_state(false) { }
604+};
605+
606+struct hs_longrun_shared {
607+ config conf;
608+ socket_args arg;
609+ int verbose;
610+ long num_threads;
611+ int usleep;
612+ volatile mutable int running;
613+ auto_ptrcontainer< std::vector<record_value *> > records;
614+ hs_longrun_shared() : verbose(0), num_threads(0), usleep(0), running(1) { }
615+};
616+
617+struct thread_base {
618+ thread_base() : need_join(false), stack_size(256 * 1024) { }
619+ virtual ~thread_base() {
620+ join();
621+ }
622+ virtual void run() = 0;
623+ void start() {
624+ if (!start_nothrow()) {
625+ fatal_abort("thread::start");
626+ }
627+ }
628+ bool start_nothrow() {
629+ if (need_join) {
630+ return need_join; /* true */
631+ }
632+ void *const arg = this;
633+ pthread_attr_t attr;
634+ if (pthread_attr_init(&attr) != 0) {
635+ fatal_abort("pthread_attr_init");
636+ }
637+ if (pthread_attr_setstacksize(&attr, stack_size) != 0) {
638+ fatal_abort("pthread_attr_setstacksize");
639+ }
640+ const int r = pthread_create(&thr, &attr, thread_main, arg);
641+ if (pthread_attr_destroy(&attr) != 0) {
642+ fatal_abort("pthread_attr_destroy");
643+ }
644+ if (r != 0) {
645+ return need_join; /* false */
646+ }
647+ need_join = true;
648+ return need_join; /* true */
649+ }
650+ void join() {
651+ if (!need_join) {
652+ return;
653+ }
654+ int e = 0;
655+ if ((e = pthread_join(thr, 0)) != 0) {
656+ fatal_abort("pthread_join");
657+ }
658+ need_join = false;
659+ }
660+ private:
661+ static void *thread_main(void *arg) {
662+ thread_base *p = static_cast<thread_base *>(arg);
663+ p->run();
664+ return 0;
665+ }
666+ private:
667+ pthread_t thr;
668+ bool need_join;
669+ size_t stack_size;
670+};
671+
672+struct hs_longrun_stat {
673+ unsigned long long verify_error_count;
674+ unsigned long long runtime_error_count;
675+ unsigned long long unknown_count;
676+ unsigned long long success_count;
677+ hs_longrun_stat()
678+ : verify_error_count(0), runtime_error_count(0),
679+ unknown_count(0), success_count(0) { }
680+ void add(const hs_longrun_stat& x) {
681+ verify_error_count += x.verify_error_count;
682+ runtime_error_count += x.runtime_error_count;
683+ unknown_count += x.unknown_count;
684+ success_count += x.success_count;
685+ }
686+};
687+
688+struct hs_longrun_thread_base : public thread_base {
689+ struct arg_type {
690+ int id;
691+ std::string worker_type;
692+ char op;
693+ int lock_flag;
694+ const hs_longrun_shared& sh;
695+ arg_type(int id, const std::string& worker_type, char op, int lock_flag,
696+ const hs_longrun_shared& sh)
697+ : id(id), worker_type(worker_type), op(op), lock_flag(lock_flag),
698+ sh(sh) { }
699+ };
700+ arg_type arg;
701+ hs_longrun_stat stat;
702+ drand48_data randbuf;
703+ unsigned int seed;
704+ hs_longrun_thread_base(const arg_type& arg)
705+ : arg(arg), seed(0) {
706+ seed = time(0) + arg.id + 1;
707+ srand48_r(seed, &randbuf);
708+ }
709+ virtual ~hs_longrun_thread_base() { }
710+ virtual void run() = 0;
711+ size_t rand_record() {
712+ double v = 0;
713+ drand48_r(&randbuf, &v);
714+ const size_t sz = arg.sh.records.size();
715+ size_t r = size_t(v * sz);
716+ if (r >= sz) {
717+ r = 0;
718+ }
719+ return r;
720+ }
721+ int verify_update(const std::string& k, const std::string& v1,
722+ const std::string& v2, const std::string& v3, record_value& rec,
723+ uint32_t num_rows, bool cur_unknown_state);
724+ int verify_read(const std::string& k, uint32_t num_rows, uint32_t num_flds,
725+ const std::string rrec[4], record_value& rec);
726+ int verify_readnolock(const std::string& k, uint32_t num_rows,
727+ uint32_t num_flds, const std::string rrec[4]);
728+};
729+
730+int
731+hs_longrun_thread_base::verify_update(const std::string& k,
732+ const std::string& v1, const std::string& v2, const std::string& v3,
733+ record_value& rec, uint32_t num_rows, bool cur_unknown_state)
734+{
735+ const bool op_success = num_rows == 1;
736+ int ret = 0;
737+ if (!rec.unknown_state) {
738+ if (!rec.deleted && !op_success) {
739+ ++stat.verify_error_count;
740+ if (arg.sh.verbose > 0) {
741+ fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
742+ "unexpected_update_failure\n",
743+ arg.worker_type.c_str(), arg.id, k.c_str());
744+ }
745+ ret = 1;
746+ } else if (rec.deleted && op_success) {
747+ ++stat.verify_error_count;
748+ if (arg.sh.verbose > 0) {
749+ fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
750+ "unexpected_update_success\n",
751+ arg.worker_type.c_str(), arg.id, k.c_str());
752+ }
753+ ret = 1;
754+ }
755+ }
756+ if (op_success) {
757+ rec.values.resize(4);
758+ rec.values[0] = k;
759+ rec.values[1] = v1;
760+ rec.values[2] = v2;
761+ rec.values[3] = v3;
762+ if (ret == 0 && !rec.unknown_state) {
763+ ++stat.success_count;
764+ }
765+ }
766+ rec.unknown_state = cur_unknown_state;
767+ if (arg.sh.verbose >= 100 && ret == 0) {
768+ fprintf(stderr, "%s %s %s %s %s\n", arg.worker_type.c_str(),
769+ k.c_str(), v1.c_str(), v2.c_str(), v3.c_str());
770+ }
771+ return ret;
772+}
773+
774+int
775+hs_longrun_thread_base::verify_read(const std::string& k,
776+ uint32_t num_rows, uint32_t num_flds, const std::string rrec[4],
777+ record_value& rec)
778+{
779+ const bool op_success = num_rows != 0;
780+ int ret = 0;
781+ if (!rec.unknown_state) {
782+ if (!rec.deleted && !op_success) {
783+ ++stat.verify_error_count;
784+ if (arg.sh.verbose > 0) {
785+ fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
786+ "unexpected_read_failure\n",
787+ arg.worker_type.c_str(), arg.id, k.c_str());
788+ }
789+ ret = 1;
790+ } else if (rec.deleted && op_success) {
791+ ++stat.verify_error_count;
792+ if (arg.sh.verbose > 0) {
793+ fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
794+ "unexpected_read_success\n",
795+ arg.worker_type.c_str(), arg.id, k.c_str());
796+ }
797+ ret = 1;
798+ } else if (num_flds != 4) {
799+ ++stat.verify_error_count;
800+ if (arg.sh.verbose > 0) {
801+ fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
802+ "unexpected_read_fldnum %d\n",
803+ arg.worker_type.c_str(), arg.id, k.c_str(),
804+ static_cast<int>(num_flds));
805+ }
806+ ret = 1;
807+ } else if (rec.deleted) {
808+ /* nothing to verify */
809+ } else {
810+ int diff = 0;
811+ for (size_t i = 0; i < 4; ++i) {
812+ if (rec.values[i] == rrec[i]) {
813+ /* ok */
814+ } else {
815+ diff = 1;
816+ }
817+ }
818+ if (diff) {
819+ std::string mess;
820+ for (size_t i = 0; i < 4; ++i) {
821+ const std::string& expected = rec.values[i];
822+ const std::string& val = rrec[i];
823+ mess += " " + val + "/" + expected;
824+ }
825+ if (arg.sh.verbose > 0) {
826+ fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
827+ "unexpected_read_value %s\n",
828+ arg.worker_type.c_str(), arg.id, k.c_str(), mess.c_str());
829+ }
830+ ret = 1;
831+ }
832+ }
833+ }
834+ if (arg.sh.verbose >= 100 && ret == 0) {
835+ fprintf(stderr, "%s %s\n", arg.worker_type.c_str(), k.c_str());
836+ }
837+ if (ret == 0 && !rec.unknown_state) {
838+ ++stat.success_count;
839+ }
840+ return ret;
841+}
842+
843+int
844+hs_longrun_thread_base::verify_readnolock(const std::string& k,
845+ uint32_t num_rows, uint32_t num_flds, const std::string rrec[4])
846+{
847+ int ret = 0;
848+ if (num_rows != 1 || num_flds != 4) {
849+ ++stat.verify_error_count;
850+ if (arg.sh.verbose > 0) {
851+ fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
852+ "unexpected_read_failure\n",
853+ arg.worker_type.c_str(), arg.id, k.c_str());
854+ }
855+ ret = 1;
856+ }
857+ if (arg.sh.verbose >= 100 && ret == 0) {
858+ fprintf(stderr, "%s -> %s %s %s %s %s\n", arg.worker_type.c_str(),
859+ k.c_str(), rrec[0].c_str(), rrec[1].c_str(), rrec[2].c_str(),
860+ rrec[3].c_str());
861+ }
862+ if (ret == 0) {
863+ ++stat.success_count;
864+ }
865+ return ret;
866+}
867+
868+struct hs_longrun_thread_hs : public hs_longrun_thread_base {
869+ hs_longrun_thread_hs(const arg_type& arg)
870+ : hs_longrun_thread_base(arg) { }
871+ void run();
872+ int check_hs_error(const char *mess, record_value *rec);
873+ int op_insert(record_value& rec);
874+ int op_delete(record_value& rec);
875+ int op_update(record_value& rec);
876+ int op_read(record_value& rec);
877+ int op_readnolock(int k);
878+ hstcpcli_ptr cli;
879+ socket_args sockargs;
880+};
881+
882+struct lock_guard : noncopyable {
883+ lock_guard(mutex& mtx) : mtx(mtx) {
884+ mtx.lock();
885+ }
886+ ~lock_guard() {
887+ mtx.unlock();
888+ }
889+ mutex& mtx;
890+};
891+
892+string_ref
893+to_string_ref(const std::string& s)
894+{
895+ return string_ref(s.data(), s.size());
896+}
897+
898+std::string
899+to_string(const string_ref& s)
900+{
901+ return std::string(s.begin(), s.size());
902+}
903+
904+void
905+hs_longrun_thread_hs::run()
906+{
907+ config c = arg.sh.conf;
908+ if (arg.op == 'R' || arg.op == 'N') {
909+ c["port"] = to_stdstring(arg.sh.conf.get_int("hsport", 9998));
910+ } else {
911+ c["port"] = to_stdstring(arg.sh.conf.get_int("hsport_wr", 9999));
912+ }
913+ sockargs.set(c);
914+
915+ while (arg.sh.running) {
916+ if (cli.get() == 0 || !cli->stable_point()) {
917+ cli = hstcpcli_i::create(sockargs);
918+ if (check_hs_error("connect", 0) != 0) {
919+ cli.reset();
920+ continue;
921+ }
922+ cli->request_buf_open_index(0, "hstestdb", "hstesttbl", "PRIMARY",
923+ "k,v1,v2,v3", "k,v1,v2,v3");
924+ cli->request_send();
925+ if (check_hs_error("openindex_send", 0) != 0) {
926+ cli.reset();
927+ continue;
928+ }
929+ size_t num_flds = 0;
930+ cli->response_recv(num_flds);
931+ if (check_hs_error("openindex_recv", 0) != 0) {
932+ cli.reset();
933+ continue;
934+ }
935+ cli->response_buf_remove();
936+ }
937+ const size_t rec_id = rand_record();
938+ if (arg.lock_flag) {
939+ record_value& rec = *arg.sh.records[rec_id];
940+ lock_guard g(rec.lock);
941+ int e = 0;
942+ switch (arg.op) {
943+ case 'I':
944+ e = op_insert(rec);
945+ break;
946+ case 'D':
947+ e = op_delete(rec);
948+ break;
949+ case 'U':
950+ e = op_update(rec);
951+ break;
952+ case 'R':
953+ e = op_read(rec);
954+ break;
955+ default:
956+ break;
957+ }
958+ } else {
959+ int e = 0;
960+ switch (arg.op) {
961+ case 'N':
962+ e = op_readnolock(rec_id);
963+ break;
964+ default:
965+ break;
966+ }
967+ }
968+ }
969+}
970+
971+int
972+hs_longrun_thread_hs::op_insert(record_value& rec)
973+{
974+ const std::string k = rec.key;
975+ const std::string v1 = "iv1_" + k + "_" + to_stdstring(arg.id);
976+ const std::string v2 = "iv2_" + k + "_" + to_stdstring(arg.id);
977+ const std::string v3 = "iv3_" + k + "_" + to_stdstring(arg.id);
978+ const string_ref op_ref("+", 1);
979+ const string_ref op_args[4] = {
980+ to_string_ref(k),
981+ to_string_ref(v1),
982+ to_string_ref(v2),
983+ to_string_ref(v3)
984+ };
985+ cli->request_buf_exec_generic(0, op_ref, op_args, 4, 1, 0,
986+ string_ref(), 0, 0, 0, 0);
987+ cli->request_send();
988+ if (check_hs_error("op_insert_send", &rec) != 0) { return 1; }
989+ size_t numflds = 0;
990+ cli->response_recv(numflds);
991+ if (arg.sh.verbose > 10) {
992+ const string_ref *row = cli->get_next_row();
993+ fprintf(stderr, "HS op=+ errrcode=%d errmess=[%s]\n", cli->get_error_code(),
994+ row ? to_string(row[0]).c_str() : "");
995+ }
996+ const bool op_success = cli->get_error_code() == 0;
997+ int ret = 0;
998+ if (!rec.unknown_state) {
999+ if (rec.deleted && !op_success) {
1000+ ++stat.verify_error_count;
1001+ if (arg.sh.verbose > 0) {
1002+ fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
1003+ "unexpected_insert_failure\n",
1004+ arg.worker_type.c_str(), arg.id, k.c_str());
1005+ }
1006+ ret = 1;
1007+ } else if (!rec.deleted && op_success) {
1008+ ++stat.verify_error_count;
1009+ if (arg.sh.verbose > 0) {
1010+ fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
1011+ "unexpected_insert_success\n",
1012+ arg.worker_type.c_str(), arg.id, k.c_str());
1013+ }
1014+ ret = 1;
1015+ }
1016+ } else {
1017+ ++stat.unknown_count;
1018+ }
1019+ if (op_success) {
1020+ rec.values.resize(4);
1021+ rec.values[0] = k;
1022+ rec.values[1] = v1;
1023+ rec.values[2] = v2;
1024+ rec.values[3] = v3;
1025+ rec.deleted = false;
1026+ if (arg.sh.verbose >= 100 && ret == 0) {
1027+ fprintf(stderr, "HS_INSERT %s %s %s %s\n", k.c_str(), v1.c_str(),
1028+ v2.c_str(), v3.c_str());
1029+ }
1030+ if (ret == 0 && !rec.unknown_state) {
1031+ ++stat.success_count;
1032+ }
1033+ rec.unknown_state = false;
1034+ }
1035+ cli->response_buf_remove();
1036+ return ret;
1037+}
1038+
1039+int
1040+hs_longrun_thread_hs::op_delete(record_value& rec)
1041+{
1042+ const std::string k = rec.key;
1043+ const string_ref op_ref("=", 1);
1044+ const string_ref op_args[1] = {
1045+ to_string_ref(k),
1046+ };
1047+ const string_ref modop_ref("D", 1);
1048+ cli->request_buf_exec_generic(0, op_ref, op_args, 1, 1, 0,
1049+ modop_ref, 0, 0, 0, 0);
1050+ cli->request_send();
1051+ if (check_hs_error("op_delete_send", &rec) != 0) { return 1; }
1052+ size_t numflds = 0;
1053+ cli->response_recv(numflds);
1054+ if (check_hs_error("op_delete_recv", &rec) != 0) { return 1; }
1055+ const string_ref *row = cli->get_next_row();
1056+ const bool op_success = (numflds > 0 && row != 0 &&
1057+ to_string(row[0]) == "1");
1058+ int ret = 0;
1059+ if (!rec.unknown_state) {
1060+ if (!rec.deleted && !op_success) {
1061+ ++stat.verify_error_count;
1062+ if (arg.sh.verbose > 0) {
1063+ fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
1064+ "unexpected_delete_failure\n",
1065+ arg.worker_type.c_str(), arg.id, k.c_str());
1066+ }
1067+ ret = 1;
1068+ } else if (rec.deleted && op_success) {
1069+ ++stat.verify_error_count;
1070+ if (arg.sh.verbose > 0) {
1071+ fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
1072+ "unexpected_delete_success\n",
1073+ arg.worker_type.c_str(), arg.id, k.c_str());
1074+ }
1075+ ret = 1;
1076+ }
1077+ }
1078+ cli->response_buf_remove();
1079+ if (op_success) {
1080+ rec.deleted = true;
1081+ if (ret == 0 && !rec.unknown_state) {
1082+ ++stat.success_count;
1083+ }
1084+ rec.unknown_state = false;
1085+ }
1086+ if (arg.sh.verbose >= 100 && ret == 0) {
1087+ fprintf(stderr, "HS_DELETE %s\n", k.c_str());
1088+ }
1089+ return ret;
1090+}
1091+
1092+int
1093+hs_longrun_thread_hs::op_update(record_value& rec)
1094+{
1095+ const std::string k = rec.key;
1096+ const std::string v1 = "uv1_" + k + "_" + to_stdstring(arg.id);
1097+ const std::string v2 = "uv2_" + k + "_" + to_stdstring(arg.id);
1098+ const std::string v3 = "uv3_" + k + "_" + to_stdstring(arg.id);
1099+ const string_ref op_ref("=", 1);
1100+ const string_ref op_args[1] = {
1101+ to_string_ref(k),
1102+ };
1103+ const string_ref modop_ref("U", 1);
1104+ const string_ref modop_args[4] = {
1105+ to_string_ref(k),
1106+ to_string_ref(v1),
1107+ to_string_ref(v2),
1108+ to_string_ref(v3)
1109+ };
1110+ cli->request_buf_exec_generic(0, op_ref, op_args, 1, 1, 0,
1111+ modop_ref, modop_args, 4, 0, 0);
1112+ cli->request_send();
1113+ if (check_hs_error("op_update_send", &rec) != 0) { return 1; }
1114+ size_t numflds = 0;
1115+ cli->response_recv(numflds);
1116+ if (check_hs_error("op_update_recv", &rec) != 0) { return 1; }
1117+ const string_ref *row = cli->get_next_row();
1118+ uint32_t num_rows = row
1119+ ? atoi_uint32_nocheck(row[0].begin(), row[0].end()) : 0;
1120+ cli->response_buf_remove();
1121+ const bool cur_unknown_state = (num_rows == 1);
1122+ return verify_update(k, v1, v2, v3, rec, num_rows, cur_unknown_state);
1123+}
1124+
1125+int
1126+hs_longrun_thread_hs::op_read(record_value& rec)
1127+{
1128+ const std::string k = rec.key;
1129+ const string_ref op_ref("=", 1);
1130+ const string_ref op_args[1] = {
1131+ to_string_ref(k),
1132+ };
1133+ cli->request_buf_exec_generic(0, op_ref, op_args, 1, 1, 0,
1134+ string_ref(), 0, 0, 0, 0);
1135+ cli->request_send();
1136+ if (check_hs_error("op_read_send", 0) != 0) { return 1; }
1137+ size_t num_flds = 0;
1138+ size_t num_rows = 0;
1139+ cli->response_recv(num_flds);
1140+ if (check_hs_error("op_read_recv", 0) != 0) { return 1; }
1141+ const string_ref *row = cli->get_next_row();
1142+ std::string rrec[4];
1143+ if (row != 0 && num_flds == 4) {
1144+ for (int i = 0; i < 4; ++i) {
1145+ rrec[i] = to_string(row[i]);
1146+ }
1147+ ++num_rows;
1148+ }
1149+ row = cli->get_next_row();
1150+ if (row != 0) {
1151+ ++num_rows;
1152+ }
1153+ cli->response_buf_remove();
1154+ return verify_read(k, num_rows, num_flds, rrec, rec);
1155+}
1156+
1157+int
1158+hs_longrun_thread_hs::op_readnolock(int key)
1159+{
1160+ const std::string k = to_stdstring(key);
1161+ const string_ref op_ref("=", 1);
1162+ const string_ref op_args[1] = {
1163+ to_string_ref(k),
1164+ };
1165+ cli->request_buf_exec_generic(0, op_ref, op_args, 1, 1, 0,
1166+ string_ref(), 0, 0, 0, 0);
1167+ cli->request_send();
1168+ if (check_hs_error("op_read_send", 0) != 0) { return 1; }
1169+ size_t num_flds = 0;
1170+ size_t num_rows = 0;
1171+ cli->response_recv(num_flds);
1172+ if (check_hs_error("op_read_recv", 0) != 0) { return 1; }
1173+ const string_ref *row = cli->get_next_row();
1174+ std::string rrec[4];
1175+ if (row != 0 && num_flds == 4) {
1176+ for (int i = 0; i < 4; ++i) {
1177+ rrec[i] = to_string(row[i]);
1178+ }
1179+ ++num_rows;
1180+ }
1181+ row = cli->get_next_row();
1182+ if (row != 0) {
1183+ ++num_rows;
1184+ }
1185+ cli->response_buf_remove();
1186+ return verify_readnolock(k, num_rows, num_flds, rrec);
1187+}
1188+
1189+int
1190+hs_longrun_thread_hs::check_hs_error(const char *mess, record_value *rec)
1191+{
1192+ const int err = cli->get_error_code();
1193+ if (err == 0) {
1194+ return 0;
1195+ }
1196+ ++stat.runtime_error_count;
1197+ if (arg.sh.verbose > 0) {
1198+ const std::string estr = cli->get_error();
1199+ fprintf(stderr, "RUNTIME_ERROR: op=%c wid=%d %s: %d %s\n",
1200+ arg.op, arg.id, mess, err, estr.c_str());
1201+ }
1202+ if (rec) {
1203+ rec->unknown_state = true;
1204+ }
1205+ return 1;
1206+}
1207+
1208+struct hs_longrun_thread_my : public hs_longrun_thread_base {
1209+ hs_longrun_thread_my(const arg_type& arg)
1210+ : hs_longrun_thread_base(arg), connected(false) { }
1211+ void run();
1212+ void show_mysql_error(const char *mess, record_value *rec);
1213+ int op_insert(record_value& rec);
1214+ int op_delete(record_value& rec);
1215+ int op_update(record_value& rec);
1216+ int op_delins(record_value& rec);
1217+ int op_read(record_value& rec);
1218+ auto_mysql db;
1219+ bool connected;
1220+};
1221+
1222+void
1223+hs_longrun_thread_my::run()
1224+{
1225+ const std::string mysql_host = arg.sh.conf.get_str("host", "localhost");
1226+ const std::string mysql_user = arg.sh.conf.get_str("mysqluser", "root");
1227+ const std::string mysql_passwd = arg.sh.conf.get_str("mysqlpass", "");
1228+ const std::string mysql_dbname = "hstestdb";
1229+
1230+ while (arg.sh.running) {
1231+ if (!connected) {
1232+ if (!mysql_real_connect(db, mysql_host.c_str(), mysql_user.c_str(),
1233+ mysql_passwd.c_str(), mysql_dbname.c_str(), mysql_port, 0, 0)) {
1234+ show_mysql_error("mysql_real_connect", 0);
1235+ continue;
1236+ }
1237+ }
1238+ connected = true;
1239+ const size_t rec_id = rand_record();
1240+ record_value& rec = *arg.sh.records[rec_id];
1241+ lock_guard g(rec.lock);
1242+ int e = 0;
1243+ switch (arg.op) {
1244+ #if 0
1245+ case 'I':
1246+ e = op_insert(rec);
1247+ break;
1248+ case 'D':
1249+ e = op_delete(rec);
1250+ break;
1251+ case 'U':
1252+ e = op_update(rec);
1253+ break;
1254+ #endif
1255+ case 'T':
1256+ e = op_delins(rec);
1257+ break;
1258+ case 'R':
1259+ e = op_read(rec);
1260+ break;
1261+ default:
1262+ break;
1263+ }
1264+ }
1265+}
1266+
1267+int
1268+hs_longrun_thread_my::op_delins(record_value& rec)
1269+{
1270+ const std::string k = rec.key;
1271+ const std::string v1 = "div1_" + k + "_" + to_stdstring(arg.id);
1272+ const std::string v2 = "div2_" + k + "_" + to_stdstring(arg.id);
1273+ const std::string v3 = "div3_" + k + "_" + to_stdstring(arg.id);
1274+ int success = 0;
1275+ bool cur_unknown_state = false;
1276+ do {
1277+ char query[1024];
1278+ #if 1
1279+ if (mysql_query(db, "begin") != 0) {
1280+ if (arg.sh.verbose >= 20) {
1281+ fprintf(stderr, "mysql: e=[%s] q=[%s]\n", mysql_error(db), "begin");
1282+ }
1283+ break;
1284+ }
1285+ #endif
1286+ cur_unknown_state = true;
1287+ snprintf(query, 1024,
1288+ "delete from hstesttbl where k = '%s'", k.c_str());
1289+ if (mysql_query(db, query) != 0) {
1290+ if (arg.sh.verbose >= 20) {
1291+ fprintf(stderr, "mysql: e=[%s] q=[%s]\n", mysql_error(db), query);
1292+ }
1293+ break;
1294+ }
1295+ if (mysql_affected_rows(db) != 1) {
1296+ if (arg.sh.verbose >= 20) {
1297+ fprintf(stderr, "mysql: notfound: [%s]\n", query);
1298+ }
1299+ break;
1300+ }
1301+ snprintf(query, 1024,
1302+ "insert into hstesttbl values ('%s', '%s', '%s', '%s')",
1303+ k.c_str(), v1.c_str(), v2.c_str(), v3.c_str());
1304+ if (mysql_query(db, query) != 0) {
1305+ if (arg.sh.verbose >= 20) {
1306+ fprintf(stderr, "mysql: e=[%s] q=[%s]\n", mysql_error(db), query);
1307+ }
1308+ break;
1309+ }
1310+ #if 1
1311+ if (mysql_query(db, "commit") != 0) {
1312+ if (arg.sh.verbose >= 20) {
1313+ fprintf(stderr, "mysql: e=[%s] q=[%s]\n", mysql_error(db), "commit");
1314+ }
1315+ break;
1316+ }
1317+ #endif
1318+ success = true;
1319+ cur_unknown_state = false;
1320+ } while (false);
1321+ return verify_update(k, v1, v2, v3, rec, (success != 0), cur_unknown_state);
1322+}
1323+
1324+int
1325+hs_longrun_thread_my::op_read(record_value& rec)
1326+{
1327+ const std::string k = rec.key;
1328+ char query[1024] = { 0 };
1329+ const int len = snprintf(query, 1024,
1330+ "select k,v1,v2,v3 from hstesttbl where k='%s'", k.c_str());
1331+ const int r = mysql_real_query(db, query, len > 0 ? len : 0);
1332+ if (r != 0) {
1333+ show_mysql_error(query, 0);
1334+ return 1;
1335+ }
1336+ MYSQL_ROW row = 0;
1337+ unsigned long *lengths = 0;
1338+ unsigned int num_rows = 0;
1339+ unsigned int num_flds = 0;
1340+ auto_mysql_res res(db);
1341+ std::string rrec[4];
1342+ if (res != 0) {
1343+ num_flds = mysql_num_fields(res);
1344+ row = mysql_fetch_row(res);
1345+ if (row != 0) {
1346+ lengths = mysql_fetch_lengths(res);
1347+ if (num_flds == 4) {
1348+ for (int i = 0; i < 4; ++i) {
1349+ rrec[i] = std::string(row[i], lengths[i]);
1350+ }
1351+ }
1352+ ++num_rows;
1353+ row = mysql_fetch_row(res);
1354+ if (row != 0) {
1355+ ++num_rows;
1356+ }
1357+ }
1358+ }
1359+ return verify_read(k, num_rows, num_flds, rrec, rec);
1360+}
1361+
1362+void
1363+hs_longrun_thread_my::show_mysql_error(const char *mess, record_value *rec)
1364+{
1365+ ++stat.runtime_error_count;
1366+ if (arg.sh.verbose > 0) {
1367+ fprintf(stderr, "RUNTIME_ERROR: op=%c wid=%d [%s]: %s\n",
1368+ arg.op, arg.id, mess, mysql_error(db));
1369+ }
1370+ if (rec) {
1371+ rec->unknown_state = true;
1372+ }
1373+ db.reset();
1374+ connected = false;
1375+}
1376+
1377+void
1378+mysql_do(MYSQL *db, const char *query)
1379+{
1380+ if (mysql_real_query(db, query, strlen(query)) != 0) {
1381+ fprintf(stderr, "mysql: e=[%s] q=[%s]\n", mysql_error(db), query);
1382+ fatal_exit("mysql_do");
1383+ }
1384+}
1385+
1386+void
1387+hs_longrun_init_table(const config& conf, int num_prepare,
1388+ hs_longrun_shared& shared)
1389+{
1390+ const std::string mysql_host = conf.get_str("host", "localhost");
1391+ const std::string mysql_user = conf.get_str("mysqluser", "root");
1392+ const std::string mysql_passwd = conf.get_str("mysqlpass", "");
1393+ const std::string mysql_dbname = "";
1394+ auto_mysql db;
1395+ if (!mysql_real_connect(db, mysql_host.c_str(), mysql_user.c_str(),
1396+ mysql_passwd.c_str(), mysql_dbname.c_str(), mysql_port, 0, 0)) {
1397+ fprintf(stderr, "mysql: error=[%s]\n", mysql_error(db));
1398+ fatal_exit("hs_longrun_init_table");
1399+ }
1400+ mysql_do(db, "drop database if exists hstestdb");
1401+ mysql_do(db, "create database hstestdb");
1402+ mysql_do(db, "use hstestdb");
1403+ mysql_do(db,
1404+ "create table hstesttbl ("
1405+ "k int primary key,"
1406+ "v1 varchar(32) not null,"
1407+ "v2 varchar(32) not null,"
1408+ "v3 varchar(32) not null"
1409+ ") character set utf8 collate utf8_bin engine = innodb");
1410+ for (int i = 0; i < num_prepare; ++i) {
1411+ const std::string i_str = to_stdstring(i);
1412+ const std::string v1 = "pv1_" + i_str;
1413+ const std::string v2 = "pv2_" + i_str;
1414+ const std::string v3 = "pv3_" + i_str;
1415+ char buf[1024];
1416+ snprintf(buf, 1024, "insert into hstesttbl(k, v1, v2, v3) values"
1417+ "(%d, '%s', '%s', '%s')", i, v1.c_str(), v2.c_str(), v3.c_str());
1418+ mysql_do(db, buf);
1419+ record_value *rec = shared.records[i];
1420+ rec->key = i_str;
1421+ rec->values.resize(4);
1422+ rec->values[0] = i_str;
1423+ rec->values[1] = v1;
1424+ rec->values[2] = v2;
1425+ rec->values[3] = v3;
1426+ rec->deleted = false;
1427+ }
1428+}
1429+
1430+int
1431+hs_longrun_main(int argc, char **argv)
1432+{
1433+ hs_longrun_shared shared;
1434+ parse_args(argc, argv, shared.conf);
1435+ shared.conf["host"] = shared.conf.get_str("host", "localhost");
1436+ shared.verbose = shared.conf.get_int("verbose", 1);
1437+ const int table_size = shared.conf.get_int("table_size", 10000);
1438+ for (int i = 0; i < table_size; ++i) {
1439+ std::auto_ptr<record_value> rec(new record_value());
1440+ rec->key = to_stdstring(i);
1441+ shared.records.push_back_ptr(rec);
1442+ }
1443+ mysql_library_init(0, 0, 0);
1444+ const int duration = shared.conf.get_int("duration", 10);
1445+ const int num_hsinsert = shared.conf.get_int("num_hsinsert", 10);
1446+ const int num_hsdelete = shared.conf.get_int("num_hsdelete", 10);
1447+ const int num_hsupdate = shared.conf.get_int("num_hsupdate", 10);
1448+ const int num_hsread = shared.conf.get_int("num_hsread", 10);
1449+ const int num_myread = shared.conf.get_int("num_myread", 10);
1450+ const int num_mydelins = shared.conf.get_int("num_mydelins", 10);
1451+ int num_hsreadnolock = shared.conf.get_int("num_hsreadnolock", 10);
1452+ const bool always_filled = (num_hsinsert == 0 && num_hsdelete == 0);
1453+ if (!always_filled) {
1454+ num_hsreadnolock = 0;
1455+ }
1456+ hs_longrun_init_table(shared.conf, always_filled ? table_size : 0,
1457+ shared);
1458+ /* create worker threads */
1459+ static const struct thrtmpl_type {
1460+ const char *type; char op; int num; int hs; int lock;
1461+ } thrtmpl[] = {
1462+ { "hsinsert", 'I', num_hsinsert, 1, 1 },
1463+ { "hsdelete", 'D', num_hsdelete, 1, 1 },
1464+ { "hsupdate", 'U', num_hsupdate, 1, 1 },
1465+ { "hsread", 'R', num_hsread, 1, 1 },
1466+ { "hsreadnolock", 'N', num_hsreadnolock, 1, 0 },
1467+ { "myread", 'R', num_myread, 0, 1 },
1468+ { "mydelins", 'T', num_mydelins, 0, 1 },
1469+ };
1470+ typedef auto_ptrcontainer< std::vector<hs_longrun_thread_base *> > thrs_type;
1471+ thrs_type thrs;
1472+ for (size_t i = 0; i < sizeof(thrtmpl)/sizeof(thrtmpl[0]); ++i) {
1473+ const thrtmpl_type& e = thrtmpl[i];
1474+ for (int j = 0; j < e.num; ++j) {
1475+ int id = thrs.size();
1476+ const hs_longrun_thread_hs::arg_type arg(id, e.type, e.op, e.lock,
1477+ shared);
1478+ std::auto_ptr<hs_longrun_thread_base> thr;
1479+ if (e.hs) {
1480+ thr.reset(new hs_longrun_thread_hs(arg));
1481+ } else {
1482+ thr.reset(new hs_longrun_thread_my(arg));
1483+ }
1484+ thrs.push_back_ptr(thr);
1485+ }
1486+ }
1487+ shared.num_threads = thrs.size();
1488+ /* start threads */
1489+ fprintf(stderr, "START\n");
1490+ shared.running = 1;
1491+ for (size_t i = 0; i < thrs.size(); ++i) {
1492+ thrs[i]->start();
1493+ }
1494+ /* wait */
1495+ sleep(duration);
1496+ /* stop thread */
1497+ shared.running = 0;
1498+ for (size_t i = 0; i < thrs.size(); ++i) {
1499+ thrs[i]->join();
1500+ }
1501+ fprintf(stderr, "DONE\n");
1502+ /* summary */
1503+ typedef std::map<std::string, hs_longrun_stat> stat_map;
1504+ stat_map sm;
1505+ for (size_t i = 0; i < thrs.size(); ++i) {
1506+ hs_longrun_thread_base *const thr = thrs[i];
1507+ const std::string wt = thr->arg.worker_type;
1508+ hs_longrun_stat& v = sm[wt];
1509+ v.add(thr->stat);
1510+ }
1511+ hs_longrun_stat total;
1512+ for (stat_map::const_iterator i = sm.begin(); i != sm.end(); ++i) {
1513+ if (i->second.verify_error_count != 0) {
1514+ fprintf(stderr, "%s verify_error %llu\n", i->first.c_str(),
1515+ i->second.verify_error_count);
1516+ }
1517+ if (i->second.runtime_error_count) {
1518+ fprintf(stderr, "%s runtime_error %llu\n", i->first.c_str(),
1519+ i->second.runtime_error_count);
1520+ }
1521+ if (i->second.unknown_count) {
1522+ fprintf(stderr, "%s unknown %llu\n", i->first.c_str(),
1523+ i->second.unknown_count);
1524+ }
1525+ fprintf(stderr, "%s success %llu\n", i->first.c_str(),
1526+ i->second.success_count);
1527+ total.add(i->second);
1528+ }
1529+ if (total.verify_error_count != 0) {
1530+ fprintf(stderr, "TOTAL verify_error %llu\n", total.verify_error_count);
1531+ }
1532+ if (total.runtime_error_count != 0) {
1533+ fprintf(stderr, "TOTAL runtime_error %llu\n", total.runtime_error_count);
1534+ }
1535+ if (total.unknown_count != 0) {
1536+ fprintf(stderr, "TOTAL unknown %llu\n", total.unknown_count);
1537+ }
1538+ fprintf(stderr, "TOTAL success %llu\n", total.success_count);
1539+ mysql_library_end();
1540+ return 0;
1541+}
1542+
1543+};
1544+
1545+int
1546+main(int argc, char **argv)
1547+{
1548+ return dena::hs_longrun_main(argc, argv);
1549+}
1550+
1551
1552=== added file 'HandlerSocket-Plugin-for-MySQL/client/hspool_test.pl'
1553--- HandlerSocket-Plugin-for-MySQL/client/hspool_test.pl 1970-01-01 00:00:00 +0000
1554+++ HandlerSocket-Plugin-for-MySQL/client/hspool_test.pl 2012-02-07 16:47:34 +0000
1555@@ -0,0 +1,224 @@
1556+#!/usr/bin/perl
1557+
1558+use strict;
1559+use warnings;
1560+use DB::HandlerSocket::Pool;
1561+use DBI;
1562+
1563+my %conf = ();
1564+for my $i (@ARGV) {
1565+ my ($k, $v) = split(/=/, $i);
1566+ $conf{$k} = $v;
1567+}
1568+
1569+my $verbose = get_conf("verbose", 0);
1570+my $actions_str = get_conf("actions",
1571+ "create,insert,verify,verify2,verify3,verify4,clean");
1572+my $tablesize = get_conf("tablesize", 1000);
1573+my $db = get_conf("db", "hstestdb");
1574+my $table = get_conf("table", "testtbl");
1575+my $table_schema = get_conf("table_schema", undef);
1576+my $engine = get_conf("engine", "innodb");
1577+my $host = get_conf("host", "localhost");
1578+my $mysqlport = get_conf("mysqlport", 3306);
1579+my $hsport_rd = get_conf("hsport_rd", 9998);
1580+my $hsport_wr = get_conf("hsport_wr", 9999);
1581+my $loop = get_conf("loop", 10000);
1582+my $op = get_conf("op", "=");
1583+my $ssps = get_conf("ssps", 0);
1584+my $num_moreflds = get_conf("moreflds", 0);
1585+my $moreflds_prefix = get_conf("moreflds_prefix", "f");
1586+my $mysql_user = 'root';
1587+my $mysql_password = '';
1588+
1589+my $dsn = "DBI:mysql:database=;host=$host;port=$mysqlport"
1590+ . ";mysql_server_prepare=$ssps";
1591+my $dbh = DBI->connect($dsn, $mysql_user, $mysql_password,
1592+ { RaiseError => 1 });
1593+my $hsargs = { 'host' => $host, 'port' => $hsport_rd };
1594+my $hspool = new DB::HandlerSocket::Pool({
1595+ hostmap => {
1596+ "$db.$table" => {
1597+ host => $host,
1598+ port => $hsport_rd,
1599+ },
1600+ },
1601+ resolve => undef,
1602+ error => undef,
1603+});
1604+$table_schema = "(k int primary key, fc30 varchar(30), ft text)"
1605+ if (!defined($table_schema));
1606+
1607+my @actions = split(/,/, $actions_str);
1608+for my $action (@actions) {
1609+ print "ACTION: $action\n";
1610+ eval "hstest_$action()";
1611+ if ($@) {
1612+ die $@;
1613+ }
1614+ print "ACTION: $action DONE\n";
1615+}
1616+
1617+sub get_conf {
1618+ my ($key, $def) = @_;
1619+ my $val = $conf{$key};
1620+ if ($val) {
1621+ print "$key=$val\n";
1622+ } else {
1623+ $val = $def;
1624+ my $defstr = $def || "(undef)";
1625+ print "$key=$defstr(default)\n";
1626+ }
1627+ return $val;
1628+}
1629+
1630+sub hstest_create {
1631+ $dbh->do("drop database if exists $db");
1632+ $dbh->do("create database $db");
1633+ $dbh->do("use $db");
1634+ $dbh->do("create table $table $table_schema engine=$engine");
1635+}
1636+
1637+sub hstest_dump {
1638+ $dbh->do("use $db");
1639+ my $sth = $dbh->prepare("select * from $table");
1640+ $sth->execute();
1641+ my $arr = $sth->fetchall_arrayref();
1642+ for my $rec (@$arr) {
1643+ print "REC:";
1644+ for my $row (@$rec) {
1645+ print " $row";
1646+ }
1647+ print "\n";
1648+ }
1649+}
1650+
1651+sub hstest_insert {
1652+ $dbh->do("use $db");
1653+ my $sth = $dbh->prepare("insert into $table values (?, ?, ?)");
1654+ for (my $k = 0; $k < $tablesize; ++$k) {
1655+ my $fc30 = "fc30_$k";
1656+ my $ft = "ft_$k";
1657+ $sth->execute($k, $fc30, $ft);
1658+ }
1659+}
1660+
1661+sub hstest_verify {
1662+ $dbh->do("use $db");
1663+ my $sth = $dbh->prepare("select * from $table order by k");
1664+ $sth->execute();
1665+ my $arr = $sth->fetchall_arrayref();
1666+ my $hsres = $hspool->index_find($db, $table, "PRIMARY", "k,fc30,ft",
1667+ ">=", [ 0 ], $tablesize, 0);
1668+ for (my $i = 0; $i < $tablesize; ++$i) {
1669+ my $rec = $arr->[$i];
1670+ my $differ = 0;
1671+ print "REC:" if $verbose;
1672+ for (my $j = 0; $j < 3; ++$j) {
1673+ my $fld = $rec->[$j];
1674+ my $hsidx = $i * 3 + $j;
1675+ my $hsfld = $hsres->[$hsidx];
1676+ if ($hsfld ne $fld) {
1677+ $differ = 1;
1678+ }
1679+ if ($differ) {
1680+ print " $fld:$hsfld" if $verbose;
1681+ } else {
1682+ print " $hsfld" if $verbose;
1683+ }
1684+ }
1685+ print "\n" if $verbose;
1686+ if ($differ) {
1687+ die "verification failed";
1688+ }
1689+ }
1690+}
1691+
1692+sub hstest_verify2 {
1693+ $dbh->do("use $db");
1694+ my $sth = $dbh->prepare("select * from $table order by k");
1695+ $sth->execute();
1696+ my $arr = $sth->fetchall_arrayref();
1697+ my $hsresa = $hspool->index_find_multi($db, $table, "PRIMARY",
1698+ "k,fc30,ft", [ [ -1, ">=", [ 0 ], $tablesize, 0 ] ]);
1699+ my $hsres = $hsresa->[0];
1700+ for (my $i = 0; $i < $tablesize; ++$i) {
1701+ my $rec = $arr->[$i];
1702+ my $differ = 0;
1703+ print "REC:" if $verbose;
1704+ for (my $j = 0; $j < 3; ++$j) {
1705+ my $fld = $rec->[$j];
1706+ my $hsidx = $i * 3 + $j;
1707+ my $hsfld = $hsres->[$hsidx];
1708+ if ($hsfld ne $fld) {
1709+ $differ = 1;
1710+ }
1711+ if ($differ) {
1712+ print " $fld:$hsfld" if $verbose;
1713+ } else {
1714+ print " $hsfld" if $verbose;
1715+ }
1716+ }
1717+ print "\n" if $verbose;
1718+ if ($differ) {
1719+ die "verification failed";
1720+ }
1721+ }
1722+}
1723+
1724+sub hashref_to_str {
1725+ my $href = $_[0];
1726+ my $r = '';
1727+ for my $k (sort keys %$href) {
1728+ my $v = $href->{$k};
1729+ $r .= "($k=>$v)";
1730+ }
1731+ return $r;
1732+}
1733+
1734+sub hstest_verify3 {
1735+ $dbh->do("use $db");
1736+ my $sth = $dbh->prepare("select * from $table order by k");
1737+ $sth->execute();
1738+ my $hsres_t = $hspool->index_find($db, $table, "PRIMARY", "k,fc30,ft",
1739+ ">=", [ 0 ], $tablesize, 0);
1740+ my $hsres = DB::HandlerSocket::Pool::result_single_to_hasharr(
1741+ [ 'k', 'fc30', 'ft' ], $hsres_t);
1742+ for (my $i = 0; $i < $tablesize; ++$i) {
1743+ my $mystr = hashref_to_str($sth->fetchrow_hashref());
1744+ my $hsstr = hashref_to_str($hsres->[$i]);
1745+ if ($mystr ne $hsstr) {
1746+ print "DIFF my=[$mystr] hs=[$hsstr]\n" if $verbose;
1747+ die "verification failed";
1748+ } else {
1749+ print "OK $hsstr\n" if $verbose;
1750+ }
1751+ }
1752+}
1753+
1754+sub hstest_verify4 {
1755+ $dbh->do("use $db");
1756+ my $sth = $dbh->prepare("select * from $table order by k");
1757+ $sth->execute();
1758+ my $hsres_t = $hspool->index_find($db, $table, "PRIMARY", "k,fc30,ft",
1759+ ">=", [ 0 ], $tablesize, 0);
1760+ my $hsres = DB::HandlerSocket::Pool::result_single_to_hashhash(
1761+ [ 'k', 'fc30', 'ft' ], 'k', $hsres_t);
1762+ my $rechash = $sth->fetchall_hashref('k');
1763+ while (my ($k, $href) = each (%$rechash)) {
1764+ my $mystr = hashref_to_str($href);
1765+ my $hsstr = hashref_to_str($hsres->{$k});
1766+ if ($mystr ne $hsstr) {
1767+ print "DIFF my=[$mystr] hs=[$hsstr]\n" if $verbose;
1768+ die "verification failed";
1769+ } else {
1770+ print "OK $hsstr\n" if $verbose;
1771+ }
1772+ }
1773+}
1774+
1775+sub hstest_clean {
1776+ $hspool->clear_pool();
1777+ $dbh->do("drop database if exists $db");
1778+}
1779+
1780
1781=== added file 'HandlerSocket-Plugin-for-MySQL/client/hstest.cpp'
1782--- HandlerSocket-Plugin-for-MySQL/client/hstest.cpp 1970-01-01 00:00:00 +0000
1783+++ HandlerSocket-Plugin-for-MySQL/client/hstest.cpp 2012-02-07 16:47:34 +0000
1784@@ -0,0 +1,1532 @@
1785+
1786+// vim:sw=2:ai
1787+
1788+#include <signal.h>
1789+#include <sys/time.h>
1790+#include <stdio.h>
1791+#include <string.h>
1792+#include <vector>
1793+#include <stdlib.h>
1794+#include <memory>
1795+#include <errno.h>
1796+#include <mysql.h>
1797+#include <time.h>
1798+#include <sys/types.h>
1799+#include <sys/stat.h>
1800+#include <fcntl.h>
1801+
1802+#include "util.hpp"
1803+#include "auto_ptrcontainer.hpp"
1804+#include "socket.hpp"
1805+#include "thread.hpp"
1806+#include "hstcpcli.hpp"
1807+
1808+#if __GNUC__ >= 4
1809+long atomic_exchange_and_add(volatile long *valp, long c)
1810+{
1811+ return __sync_fetch_and_add(valp, c);
1812+}
1813+#else
1814+#include <bits/atomicity.h>
1815+using namespace __gnu_cxx;
1816+long atomic_exchange_and_add(volatile long *valp, long c)
1817+{
1818+ return __exchange_and_add((volatile _Atomic_word *)valp, c);
1819+}
1820+#endif
1821+
1822+namespace dena {
1823+
1824+struct auto_mysql : private noncopyable {
1825+ auto_mysql() : db(0) {
1826+ reset();
1827+ }
1828+ ~auto_mysql() {
1829+ if (db) {
1830+ mysql_close(db);
1831+ }
1832+ }
1833+ void reset() {
1834+ if (db) {
1835+ mysql_close(db);
1836+ }
1837+ if ((db = mysql_init(0)) == 0) {
1838+ fatal_abort("failed to initialize mysql client");
1839+ }
1840+ }
1841+ operator MYSQL *() const { return db; }
1842+ private:
1843+ MYSQL *db;
1844+};
1845+
1846+struct auto_mysql_res : private noncopyable {
1847+ auto_mysql_res(MYSQL *db) {
1848+ res = mysql_store_result(db);
1849+ }
1850+ ~auto_mysql_res() {
1851+ if (res) {
1852+ mysql_free_result(res);
1853+ }
1854+ }
1855+ operator MYSQL_RES *() const { return res; }
1856+ private:
1857+ MYSQL_RES *res;
1858+};
1859+
1860+struct auto_mysql_stmt : private noncopyable {
1861+ auto_mysql_stmt(MYSQL *db) {
1862+ stmt = mysql_stmt_init(db);
1863+ }
1864+ ~auto_mysql_stmt() {
1865+ if (stmt) {
1866+ mysql_stmt_close(stmt);
1867+ }
1868+ }
1869+ operator MYSQL_STMT *() const { return stmt; }
1870+ private:
1871+ MYSQL_STMT *stmt;
1872+};
1873+
1874+namespace {
1875+
1876+double
1877+gettimeofday_double()
1878+{
1879+ struct timeval tv = { };
1880+ if (gettimeofday(&tv, 0) != 0) {
1881+ fatal_abort("gettimeofday");
1882+ }
1883+ return static_cast<double>(tv.tv_usec) / 1000000 + tv.tv_sec;
1884+}
1885+
1886+// unused
1887+void
1888+wait_close(int fd)
1889+{
1890+ char buf[1024];
1891+ while (true) {
1892+ int r = read(fd, buf, sizeof(buf));
1893+ if (r <= 0) {
1894+ break;
1895+ }
1896+ }
1897+}
1898+
1899+// unused
1900+void
1901+gentle_close(int fd)
1902+{
1903+ int r = shutdown(fd, SHUT_WR);
1904+ if (r != 0) {
1905+ return;
1906+ }
1907+ wait_close(fd);
1908+}
1909+
1910+};
1911+
1912+struct hstest_shared {
1913+ config conf;
1914+ socket_args arg;
1915+ int verbose;
1916+ size_t loop;
1917+ size_t pipe;
1918+ char op;
1919+ long num_threads;
1920+ mutable volatile long count;
1921+ mutable volatile long conn_count;
1922+ long wait_conn;
1923+ volatile char *keygen;
1924+ long keygen_size;
1925+ mutable volatile int enable_timing;
1926+ int usleep;
1927+ int dump;
1928+ hstest_shared() : verbose(0), loop(0), pipe(0), op('G'), num_threads(0),
1929+ count(0), conn_count(0), wait_conn(0), keygen(0), keygen_size(0),
1930+ enable_timing(0), usleep(0), dump(0) { }
1931+ void increment_count(unsigned int c = 1) const volatile {
1932+ atomic_exchange_and_add(&count, c);
1933+ }
1934+ void increment_conn(unsigned int c) const volatile {
1935+ atomic_exchange_and_add(&conn_count, c);
1936+ while (wait_conn != 0 && conn_count < wait_conn) {
1937+ sleep(1);
1938+ }
1939+ // fprintf(stderr, "wait_conn=%ld done\n", wait_conn);
1940+ }
1941+};
1942+
1943+struct hstest_thread {
1944+ struct arg_type {
1945+ size_t id;
1946+ const hstest_shared& sh;
1947+ bool watch_flag;
1948+ arg_type(size_t i, const hstest_shared& s, bool w)
1949+ : id(i), sh(s), watch_flag(w) { }
1950+ };
1951+ hstest_thread(const arg_type& a) : arg(a), io_success_count(0),
1952+ op_success_count(0), response_min(99999), response_max(0),
1953+ response_sum(0), response_avg(0) { }
1954+ void operator ()();
1955+ void test_1();
1956+ void test_2_3(int test_num);
1957+ void test_4_5(int test_num);
1958+ void test_6(int test_num);
1959+ void test_7(int test_num);
1960+ void test_8(int test_num);
1961+ void test_9(int test_num);
1962+ void test_10(int test_num);
1963+ void test_11(int test_num);
1964+ void test_12(int test_num);
1965+ void test_21(int test_num);
1966+ void test_22(int test_num);
1967+ void test_watch();
1968+ void sleep_if();
1969+ void set_timing(double time_spent);
1970+ arg_type arg;
1971+ auto_file fd;
1972+ size_t io_success_count;
1973+ size_t op_success_count;
1974+ double response_min, response_max, response_sum, response_avg;
1975+};
1976+
1977+void
1978+hstest_thread::test_1()
1979+{
1980+ char buf[1024];
1981+ unsigned int seed = arg.id;
1982+ seed ^= arg.sh.conf.get_int("seed_xor", 0);
1983+ std::string err;
1984+ if (socket_connect(fd, arg.sh.arg, err) != 0) {
1985+ fprintf(stderr, "connect: %d %s\n", errno, strerror(errno));
1986+ return;
1987+ }
1988+ const char op = arg.sh.op;
1989+ const int tablesize = arg.sh.conf.get_int("tablesize", 0);
1990+ for (size_t i = 0; i < arg.sh.loop; ++i) {
1991+ for (size_t j = 0; j < arg.sh.pipe; ++j) {
1992+ int k = 0, v = 0, len = 0;
1993+ if (op == 'G') {
1994+ k = rand_r(&seed);
1995+ v = rand_r(&seed); /* unused */
1996+ if (tablesize != 0) {
1997+ k &= tablesize;
1998+ }
1999+ len = snprintf(buf, sizeof(buf), "%c\tk%d\n", op, k);
2000+ } else {
2001+ k = rand_r(&seed);
2002+ v = rand_r(&seed);
2003+ if (tablesize != 0) {
2004+ k &= tablesize;
2005+ }
2006+ len = snprintf(buf, sizeof(buf), "%c\tk%d\tv%d\n", op, k, v);
2007+ }
2008+ const int wlen = write(fd.get(), buf, len);
2009+ if (wlen != len) {
2010+ return;
2011+ }
2012+ }
2013+ size_t read_cnt = 0;
2014+ size_t read_pos = 0;
2015+ while (read_cnt < arg.sh.pipe) {
2016+ const int rlen = read(fd.get(), buf + read_pos, sizeof(buf) - read_pos);
2017+ if (rlen <= 0) {
2018+ return;
2019+ }
2020+ read_pos += rlen;
2021+ while (true) {
2022+ const char *const p = static_cast<const char *>(memchr(buf, '\n',
2023+ read_pos));
2024+ if (p == 0) {
2025+ break;
2026+ }
2027+ ++read_cnt;
2028+ ++io_success_count;
2029+ arg.sh.increment_count();
2030+ if (p != buf && buf[0] == '=') {
2031+ ++op_success_count;
2032+ }
2033+ const size_t rest_size = buf + read_pos - (p + 1);
2034+ if (rest_size != 0) {
2035+ memmove(buf, p + 1, rest_size);
2036+ }
2037+ read_pos = rest_size;
2038+ }
2039+ }
2040+ }
2041+}
2042+
2043+void
2044+hstest_thread::test_2_3(int test_num)
2045+{
2046+#if 0
2047+ char buf_k[128], buf_v[128];
2048+ unsigned int seed = arg.id;
2049+ op_base_t op = static_cast<op_base_t>(arg.sh.op);
2050+ micli_ptr hnd;
2051+ if (test_num == 2) {
2052+ hnd = micli_i::create_remote(arg.sh.conf);
2053+ } else if (test_num == 3) {
2054+ // hnd = micli_i::create_inproc(arg.sh.localdb);
2055+ }
2056+ if (hnd.get() == 0) {
2057+ return;
2058+ }
2059+ for (size_t i = 0; i < arg.sh.loop; ++i) {
2060+ for (size_t j = 0; j < arg.sh.pipe; ++j) {
2061+ int k = 0, v = 0, klen = 0, vlen = 0;
2062+ k = rand_r(&seed);
2063+ klen = snprintf(buf_k, sizeof(buf_k), "k%d", k);
2064+ v = rand_r(&seed); /* unused */
2065+ vlen = snprintf(buf_v, sizeof(buf_v), "v%d", v);
2066+ string_ref arr[2];
2067+ arr[0] = string_ref(buf_k, klen);
2068+ arr[1] = string_ref(buf_v, vlen);
2069+ pstrarr_ptr rec(arr, 2);
2070+ if (hnd->execute(op, 0, 0, rec.get_const())) {
2071+ ++io_success_count;
2072+ arg.sh.increment_count();
2073+ const dataset& res = hnd->get_result_ref();
2074+ if (res.size() == 1) {
2075+ ++op_success_count;
2076+ }
2077+ }
2078+ }
2079+ }
2080+#endif
2081+}
2082+
2083+void
2084+hstest_thread::test_4_5(int test_num)
2085+{
2086+#if 0
2087+ char buf_k[128], buf_v[8192];
2088+ memset(buf_v, ' ', sizeof(buf_v));
2089+ unsigned int seed = arg.id;
2090+ op_base_t op = static_cast<op_base_t>(arg.sh.op);
2091+ micli_ptr hnd;
2092+ if (test_num == 4) {
2093+ hnd = micli_i::create_remote(arg.sh.conf);
2094+ } else if (test_num == 5) {
2095+ hnd = micli_i::create_inproc(arg.sh.localdb);
2096+ }
2097+ if (hnd.get() == 0) {
2098+ return;
2099+ }
2100+ for (size_t i = 0; i < arg.sh.loop; ++i) {
2101+ for (size_t j = 0; j < arg.sh.pipe; ++j) {
2102+ int k = 0, klen = 0, vlen = 0;
2103+ k = i & 0x0000ffffUL;
2104+ if (k == 0) {
2105+ fprintf(stderr, "k=0\n");
2106+ }
2107+ klen = snprintf(buf_k, sizeof(buf_k), "k%d", k);
2108+ vlen = rand_r(&seed) % 8192;
2109+ string_ref arr[2];
2110+ arr[0] = string_ref(buf_k, klen);
2111+ arr[1] = string_ref(buf_v, vlen);
2112+ pstrarr_ptr rec(arr, 2);
2113+ if (hnd->execute(op, 0, 0, rec.get_const())) {
2114+ ++io_success_count;
2115+ const dataset& res = hnd->get_result_ref();
2116+ if (res.size() == 1) {
2117+ ++op_success_count;
2118+ }
2119+ }
2120+ }
2121+ }
2122+#endif
2123+}
2124+
2125+void
2126+hstest_thread::test_6(int test_num)
2127+{
2128+ int count = arg.sh.conf.get_int("count", 1);
2129+ auto_file fds[count];
2130+ for (int i = 0; i < count; ++i) {
2131+ const double t1 = gettimeofday_double();
2132+ std::string err;
2133+ if (socket_connect(fds[i], arg.sh.arg, err) != 0) {
2134+ fprintf(stderr, "id=%zu i=%d err=%s\n", arg.id, i, err.c_str());
2135+ }
2136+ const double t2 = gettimeofday_double();
2137+ if (t2 - t1 > 1) {
2138+ fprintf(stderr, "id=%zu i=%d time %f\n", arg.id, i, t2 - t1);
2139+ }
2140+ }
2141+}
2142+
2143+void
2144+hstest_thread::test_7(int num)
2145+{
2146+ /*
2147+ set foo 0 0 10
2148+ 0123456789
2149+ STORED
2150+ get foo
2151+ VALUE foo 0 10
2152+ 0123456789
2153+ END
2154+ get var
2155+ END
2156+ */
2157+ char buf[1024];
2158+ const int keep_connection = arg.sh.conf.get_int("keep_connection", 1);
2159+ unsigned int seed = arg.id;
2160+ seed ^= arg.sh.conf.get_int("seed_xor", 0);
2161+ const int tablesize = arg.sh.conf.get_int("tablesize", 0);
2162+ const char op = arg.sh.op;
2163+ for (size_t i = 0; i < arg.sh.loop; ++i) {
2164+ const double tm1 = gettimeofday_double();
2165+ std::string err;
2166+ if (fd.get() < 0 && socket_connect(fd, arg.sh.arg, err) != 0) {
2167+ fprintf(stderr, "connect: %d %s\n", errno, strerror(errno));
2168+ return;
2169+ }
2170+ for (size_t j = 0; j < arg.sh.pipe; ++j) {
2171+ int k = 0, v = 0, len = 0;
2172+ if (op == 'G') {
2173+ k = rand_r(&seed);
2174+ v = rand_r(&seed); /* unused */
2175+ if (tablesize != 0) {
2176+ k &= tablesize;
2177+ }
2178+ len = snprintf(buf, sizeof(buf), "get k%d\r\n", k);
2179+ } else {
2180+ k = rand_r(&seed);
2181+ v = rand_r(&seed);
2182+ if (tablesize != 0) {
2183+ k &= tablesize;
2184+ }
2185+ char vbuf[1024];
2186+ int vlen = snprintf(vbuf, sizeof(vbuf),
2187+ "v%d"
2188+ // "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
2189+ // "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
2190+ // "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
2191+ // "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
2192+ // "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
2193+ , v);
2194+ len = snprintf(buf, sizeof(buf), "set k%d 0 0 %d\r\n%s\r\n",
2195+ k, vlen, vbuf);
2196+ }
2197+ const int wlen = write(fd.get(), buf, len);
2198+ if (wlen != len) {
2199+ return;
2200+ }
2201+ }
2202+ size_t read_cnt = 0;
2203+ size_t read_pos = 0;
2204+ bool read_response_done = false;
2205+ bool expect_value = false;
2206+ while (!read_response_done) {
2207+ const int rlen = read(fd.get(), buf + read_pos, sizeof(buf) - read_pos);
2208+ if (rlen <= 0) {
2209+ return;
2210+ }
2211+ read_pos += rlen;
2212+ while (true) {
2213+ const char *const p = static_cast<const char *>(memchr(buf, '\n',
2214+ read_pos));
2215+ if (p == 0) {
2216+ break;
2217+ }
2218+ ++read_cnt;
2219+ if (expect_value) {
2220+ expect_value = false;
2221+ } else if (p >= buf + 6 && memcmp(buf, "VALUE ", 6) == 0) {
2222+ expect_value = true;
2223+ ++op_success_count;
2224+ } else {
2225+ if (p == buf + 7 && memcmp(buf, "STORED\r", 7) == 0) {
2226+ ++op_success_count;
2227+ }
2228+ read_response_done = true;
2229+ }
2230+ const size_t rest_size = buf + read_pos - (p + 1);
2231+ if (rest_size != 0) {
2232+ memmove(buf, p + 1, rest_size);
2233+ }
2234+ read_pos = rest_size;
2235+ }
2236+ ++io_success_count;
2237+ }
2238+ arg.sh.increment_count();
2239+ if (!keep_connection) {
2240+ fd.close();
2241+ }
2242+ const double tm2 = gettimeofday_double();
2243+ set_timing(tm2 - tm1);
2244+ sleep_if();
2245+ }
2246+}
2247+
2248+struct rec {
2249+ std::string key;
2250+ std::string value;
2251+};
2252+
2253+void
2254+hstest_thread::test_8(int test_num)
2255+{
2256+#if 0
2257+ char buf_k[128], buf_v[128];
2258+ unsigned int seed = arg.id;
2259+ // op_base_t op = static_cast<op_base_t>(arg.sh.op);
2260+ using namespace boost::multi_index;
2261+ typedef member<rec, std::string, &rec::key> rec_get_key;
2262+ typedef ordered_unique<rec_get_key> oui;
2263+ typedef multi_index_container< rec, indexed_by<oui> > mic;
2264+ #if 0
2265+ typedef std::map<std::string, std::string> m_type;
2266+ m_type m;
2267+ #endif
2268+ mic m;
2269+ for (size_t i = 0; i < arg.sh.loop; ++i) {
2270+ for (size_t j = 0; j < arg.sh.pipe; ++j) {
2271+ int k = 0, v = 0, klen = 0, vlen = 0;
2272+ k = rand_r(&seed);
2273+ klen = snprintf(buf_k, sizeof(buf_k), "k%d", k);
2274+ v = rand_r(&seed); /* unused */
2275+ vlen = snprintf(buf_v, sizeof(buf_v), "v%d", v);
2276+ const std::string ks(buf_k, klen);
2277+ const std::string vs(buf_v, vlen);
2278+ rec r;
2279+ r.key = ks;
2280+ r.value = vs;
2281+ m.insert(r);
2282+ // m.insert(std::make_pair(ks, vs));
2283+ ++io_success_count;
2284+ ++op_success_count;
2285+ arg.sh.increment_count();
2286+ }
2287+ }
2288+#endif
2289+}
2290+
2291+struct mysqltest_thread_initobj : private noncopyable {
2292+ mysqltest_thread_initobj() {
2293+ mysql_thread_init();
2294+ }
2295+ ~mysqltest_thread_initobj() {
2296+ mysql_thread_end();
2297+ }
2298+};
2299+
2300+void
2301+hstest_thread::test_9(int test_num)
2302+{
2303+ /* create table hstest
2304+ * ( k varchar(255) not null, v varchar(255) not null, primary key(k))
2305+ * engine = innodb; */
2306+ auto_mysql db;
2307+ // mysqltest_thread_initobj initobj;
2308+ std::string err;
2309+ const char op = arg.sh.op;
2310+ const std::string suffix = arg.sh.conf.get_str("value_suffix", "upd");
2311+ unsigned long long err_cnt = 0;
2312+ unsigned long long query_cnt = 0;
2313+ #if 0
2314+ my_bool reconnect = 0;
2315+ if (mysql_options(db, MYSQL_OPT_RECONNECT, &reconnect) != 0) {
2316+ err = "mysql_options() failed";
2317+ ++err_cnt;
2318+ return;
2319+ }
2320+ #endif
2321+ unsigned int seed = time(0) + arg.id + 1;
2322+ seed ^= arg.sh.conf.get_int("seed_xor", 0);
2323+ drand48_data randbuf;
2324+ srand48_r(seed, &randbuf);
2325+ const std::string mysql_host = arg.sh.conf.get_str("host", "localhost");
2326+ const int mysql_port = arg.sh.conf.get_int("mysqlport", 3306);
2327+ const int num = arg.sh.loop;
2328+ const std::string mysql_user = arg.sh.conf.get_str("mysqluser", "root");
2329+ const std::string mysql_passwd = arg.sh.conf.get_str("mysqlpass", "");
2330+ const std::string mysql_dbname = arg.sh.conf.get_str("dbname", "hstest");
2331+ const int keep_connection = arg.sh.conf.get_int("keep_connection", 1);
2332+ const int verbose = arg.sh.conf.get_int("verbose", 1);
2333+ const int tablesize = arg.sh.conf.get_int("tablesize", 10000);
2334+ const int moreflds = arg.sh.conf.get_int("moreflds", 0);
2335+ const std::string moreflds_prefix = arg.sh.conf.get_str(
2336+ "moreflds_prefix", "column0123456789_");
2337+ const int use_handler = arg.sh.conf.get_int("handler", 0);
2338+ const int sched_flag = arg.sh.conf.get_int("sched", 0);
2339+ const int use_in = arg.sh.conf.get_int("in", 0);
2340+ const int ssps = use_in ? 0 : arg.sh.conf.get_int("ssps", 0);
2341+ std::string flds = "v";
2342+ for (int i = 0; i < moreflds; ++i) {
2343+ char buf[1024];
2344+ snprintf(buf, sizeof(buf), ",%s%d", moreflds_prefix.c_str(), i);
2345+ flds += std::string(buf);
2346+ }
2347+ int connected = 0;
2348+ std::auto_ptr<auto_mysql_stmt> stmt;
2349+ string_buffer wbuf;
2350+ for (int i = 0; i < num; ++i) {
2351+ const double tm1 = gettimeofday_double();
2352+ const int flags = 0;
2353+ if (connected == 0) {
2354+ if (!mysql_real_connect(db, mysql_host.c_str(),
2355+ mysql_user.c_str(), mysql_user.empty() ? 0 : mysql_passwd.c_str(),
2356+ mysql_dbname.c_str(), mysql_port, 0, flags)) {
2357+ err = "failed to connect: " + std::string(mysql_error(db));
2358+ if (verbose >= 1) {
2359+ fprintf(stderr, "e=[%s]\n", err.c_str());
2360+ }
2361+ ++err_cnt;
2362+ return;
2363+ }
2364+ arg.sh.increment_conn(1);
2365+ }
2366+ int r = 0;
2367+ if (connected == 0 && use_handler) {
2368+ const char *const q = "handler hstest_table1 open";
2369+ r = mysql_real_query(db, q, strlen(q));
2370+ if (r != 0) {
2371+ err = 1;
2372+ }
2373+ }
2374+ if (connected == 0 && ssps) {
2375+ stmt.reset(new auto_mysql_stmt(db));
2376+ const char *const q = "select v from hstest_table1 where k = ?";
2377+ r = mysql_stmt_prepare(*stmt, q, strlen(q));
2378+ if (r != 0) {
2379+ fprintf(stderr, "ssps err\n");
2380+ ++err_cnt;
2381+ return;
2382+ }
2383+ }
2384+ connected = 1;
2385+ std::string result_str;
2386+ unsigned int err = 0;
2387+ unsigned int num_flds = 0, num_affected_rows = 0;
2388+ int got_data = 0;
2389+ char buf_query[16384];
2390+ int buf_query_len = 0;
2391+ int k = 0, v = 0;
2392+ {
2393+ double kf = 0, vf = 0;
2394+ drand48_r(&randbuf, &kf);
2395+ drand48_r(&randbuf, &vf);
2396+ k = int(kf * tablesize);
2397+ v = int(vf * tablesize);
2398+ #if 0
2399+ k = rand_r(&seed);
2400+ v = rand_r(&seed);
2401+ if (tablesize != 0) {
2402+ k %= tablesize;
2403+ }
2404+ #endif
2405+ if (op == 'G') {
2406+ if (use_handler) {
2407+ buf_query_len = snprintf(buf_query, sizeof(buf_query),
2408+ "handler hstest_table1 read `primary` = ( '%d' )", k);
2409+ // TODO: moreflds
2410+ } else if (ssps) {
2411+ //
2412+ } else if (use_in) {
2413+ wbuf.clear();
2414+ char *p = wbuf.make_space(1024);
2415+ int len = snprintf(p, 1024, "select %s from hstest_table1 where k in ('%d'", flds.c_str(), k);
2416+ wbuf.space_wrote(len);
2417+ for (int j = 1; j < use_in; ++j) {
2418+ /* generate more key */
2419+ drand48_r(&randbuf, &kf);
2420+ k = int(kf * tablesize);
2421+ p = wbuf.make_space(1024);
2422+ int len = snprintf(p, 1024, ", '%d'", k);
2423+ wbuf.space_wrote(len);
2424+ }
2425+ wbuf.append_literal(")");
2426+ } else {
2427+ buf_query_len = snprintf(buf_query, sizeof(buf_query),
2428+ "select %s from hstest_table1 where k = '%d'", flds.c_str(), k);
2429+ }
2430+ } else if (op == 'U') {
2431+ buf_query_len = snprintf(buf_query, sizeof(buf_query),
2432+ "update hstest_table1 set v = '%d_%d%s' where k = '%d'",
2433+ v, k, suffix.c_str(), k);
2434+ } else if (op == 'R') {
2435+ buf_query_len = snprintf(buf_query, sizeof(buf_query),
2436+ "replace into hstest_table1 values ('%d', 'v%d')", k, v);
2437+ // TODO: moreflds
2438+ }
2439+ }
2440+ if (r == 0) {
2441+ if (ssps) {
2442+ MYSQL_BIND bind[1] = { };
2443+ bind[0].buffer_type = MYSQL_TYPE_LONG;
2444+ bind[0].buffer = (char *)&k;
2445+ bind[0].is_null = 0;
2446+ bind[0].length = 0;
2447+ if (mysql_stmt_bind_param(*stmt, bind)) {
2448+ fprintf(stderr, "err: %s\n", mysql_stmt_error(*stmt));
2449+ ++err_cnt;
2450+ return;
2451+ }
2452+ r = mysql_stmt_execute(*stmt);
2453+ // fprintf(stderr, "stmt exec\n");
2454+ } else if (use_in) {
2455+ r = mysql_real_query(db, wbuf.begin(), wbuf.size());
2456+ } else {
2457+ r = mysql_real_query(db, buf_query, buf_query_len);
2458+ // fprintf(stderr, "real query\n");
2459+ }
2460+ ++query_cnt;
2461+ }
2462+ if (r != 0) {
2463+ err = 1;
2464+ } else if (ssps) {
2465+ if (verbose >= 0) {
2466+ char resbuf[1024];
2467+ unsigned long res_len = 0;
2468+ MYSQL_BIND bind[1] = { };
2469+ bind[0].buffer_type = MYSQL_TYPE_STRING;
2470+ bind[0].buffer = resbuf;
2471+ bind[0].buffer_length = sizeof(resbuf);
2472+ bind[0].length = &res_len;
2473+ if (mysql_stmt_bind_result(*stmt, bind)) {
2474+ fprintf(stderr, "err: %s\n", mysql_stmt_error(*stmt));
2475+ ++err_cnt;
2476+ return;
2477+ }
2478+ if (mysql_stmt_fetch(*stmt)) {
2479+ fprintf(stderr, "err: %s\n", mysql_stmt_error(*stmt));
2480+ ++err_cnt;
2481+ return;
2482+ }
2483+ if (!result_str.empty()) {
2484+ result_str += " ";
2485+ }
2486+ result_str += std::string(resbuf, res_len);
2487+ // fprintf(stderr, "SSPS RES: %s\n", result_str.c_str());
2488+ got_data = 1;
2489+ } else {
2490+ got_data = 1;
2491+ }
2492+ } else {
2493+ auto_mysql_res res(db);
2494+ if (res != 0) {
2495+ if (verbose >= 0) {
2496+ num_flds = mysql_num_fields(res);
2497+ MYSQL_ROW row = 0;
2498+ while ((row = mysql_fetch_row(res)) != 0) {
2499+ got_data += 1;
2500+ unsigned long *const lengths = mysql_fetch_lengths(res);
2501+ if (verbose >= 2) {
2502+ for (unsigned int i = 0; i < num_flds; ++i) {
2503+ if (!result_str.empty()) {
2504+ result_str += " ";
2505+ }
2506+ result_str += std::string(row[i], lengths[i]);
2507+ }
2508+ }
2509+ }
2510+ } else {
2511+ MYSQL_ROW row = 0;
2512+ while ((row = mysql_fetch_row(res)) != 0) {
2513+ got_data += 1;
2514+ }
2515+ }
2516+ } else {
2517+ if (mysql_field_count(db) == 0) {
2518+ num_affected_rows = mysql_affected_rows(db);
2519+ } else {
2520+ err = 1;
2521+ }
2522+ }
2523+ }
2524+ if (verbose >= 2 || (verbose >= 1 && err != 0)) {
2525+ if (err) {
2526+ ++err_cnt;
2527+ const char *const errstr = mysql_error(db);
2528+ fprintf(stderr, "e=[%s] a=%u q=[%s]\n", errstr,
2529+ num_affected_rows, buf_query);
2530+ } else {
2531+ fprintf(stderr, "a=%u q=[%s] r=[%s]\n", num_affected_rows, buf_query,
2532+ result_str.c_str());
2533+ }
2534+ }
2535+ if (err == 0) {
2536+ ++io_success_count;
2537+ if (num_affected_rows > 0 || got_data > 0) {
2538+ op_success_count += got_data;
2539+ } else {
2540+ if (verbose >= 1) {
2541+ fprintf(stderr, "k=%d numaff=%u gotdata=%d\n",
2542+ k, num_affected_rows, got_data);
2543+ }
2544+ }
2545+ arg.sh.increment_count();
2546+ }
2547+ if (!keep_connection) {
2548+ if (stmt.get() != 0) {
2549+ stmt.reset();
2550+ }
2551+ db.reset();
2552+ connected = 0;
2553+ }
2554+ const double tm2 = gettimeofday_double();
2555+ set_timing(tm2 - tm1);
2556+ sleep_if();
2557+ if (sched_flag) {
2558+ sched_yield();
2559+ }
2560+ }
2561+ if (verbose >= 1) {
2562+ fprintf(stderr, "thread finished (error_count=%llu)\n", err_cnt);
2563+ }
2564+}
2565+
2566+void
2567+hstest_thread::test_10(int test_num)
2568+{
2569+ const int keep_connection = arg.sh.conf.get_int("keep_connection", 1);
2570+ unsigned int seed = time(0) + arg.id + 1;
2571+ seed ^= arg.sh.conf.get_int("seed_xor", 0);
2572+ drand48_data randbuf;
2573+ srand48_r(seed, &randbuf);
2574+ std::string err;
2575+ int keepconn_count = 0;
2576+ const char op = arg.sh.op;
2577+ const int verbose = arg.sh.conf.get_int("verbose", 1);
2578+ const std::string suffix = arg.sh.conf.get_str("value_suffix", "upd");
2579+ const int tablesize = arg.sh.conf.get_int("tablesize", 10000);
2580+ const int firstkey = arg.sh.conf.get_int("firstkey", 0);
2581+ const int sched_flag = arg.sh.conf.get_int("sched", 0);
2582+ const int moreflds = arg.sh.conf.get_int("moreflds", 0);
2583+ const std::string dbname = arg.sh.conf.get_str("dbname", "hstest");
2584+ const std::string table = arg.sh.conf.get_str("table", "hstest_table1");
2585+ const std::string index = arg.sh.conf.get_str("index", "PRIMARY");
2586+ const std::string field = arg.sh.conf.get_str("field", "v");
2587+ const int use_in = arg.sh.conf.get_int("in", 0);
2588+ const std::string moreflds_prefix = arg.sh.conf.get_str(
2589+ "moreflds_prefix", "column0123456789_");
2590+ const int dump = arg.sh.dump;
2591+ const int nodup = arg.sh.conf.get_int("nodup", 0);
2592+ std::string moreflds_str;
2593+ for (int i = 0; i < moreflds; ++i) {
2594+ char sbuf[1024];
2595+ snprintf(sbuf, sizeof(sbuf), ",%s%d", moreflds_prefix.c_str(), i);
2596+ moreflds_str += std::string(sbuf);
2597+ }
2598+ string_buffer wbuf;
2599+ char rbuf[16384];
2600+ for (size_t i = 0; i < arg.sh.loop; ++i) {
2601+ int len = 0, rlen = 0, wlen = 0;
2602+ #if 0
2603+ const double tm1 = gettimeofday_double();
2604+ #endif
2605+ if (fd.get() < 0) {
2606+ if (socket_connect(fd, arg.sh.arg, err) != 0) {
2607+ fprintf(stderr, "connect: %d %s\n", errno, strerror(errno));
2608+ return;
2609+ }
2610+ char *wp = wbuf.make_space(1024);
2611+ len = snprintf(wp, 1024,
2612+ "P\t1\t%s\t%s\tPRIMARY\t%s%s\n", dbname.c_str(), table.c_str(),
2613+ field.c_str(), moreflds_str.c_str());
2614+ /* pst_num, db, table, index, retflds */
2615+ wbuf.space_wrote(len);
2616+ wlen = write(fd.get(), wbuf.begin(), len);
2617+ if (len != wlen) {
2618+ fprintf(stderr, "write: %d %d\n", len, wlen);
2619+ return;
2620+ }
2621+ wbuf.clear();
2622+ rlen = read(fd.get(), rbuf, sizeof(rbuf));
2623+ if (rlen <= 0 || rbuf[rlen - 1] != '\n') {
2624+ fprintf(stderr, "read: rlen=%d errno=%d\n", rlen, errno);
2625+ return;
2626+ }
2627+ if (rbuf[0] != '0') {
2628+ fprintf(stderr, "failed to open table\n");
2629+ return;
2630+ }
2631+ arg.sh.increment_conn(1);
2632+ }
2633+ const double tm1 = gettimeofday_double();
2634+ for (size_t j = 0; j < arg.sh.pipe; ++j) {
2635+ int k = 0, v = 0;
2636+ {
2637+ while (true) {
2638+ double kf = 0, vf = 0;
2639+ drand48_r(&randbuf, &kf);
2640+ drand48_r(&randbuf, &vf);
2641+ k = int(kf * tablesize) + firstkey;
2642+ v = int(vf * tablesize) + firstkey;
2643+ if (k - firstkey < arg.sh.keygen_size) {
2644+ volatile char *const ptr = arg.sh.keygen + (k - firstkey);
2645+ // int oldv = __sync_fetch_and_or(ptr, 1);
2646+ int oldv = *ptr;
2647+ *ptr += 1;
2648+ if (nodup && oldv != 0) {
2649+ if (dump) {
2650+ fprintf(stderr, "retry\n");
2651+ }
2652+ continue;
2653+ }
2654+ } else {
2655+ if (nodup) {
2656+ if (dump) {
2657+ fprintf(stderr, "retry2\n");
2658+ }
2659+ continue;
2660+ }
2661+ }
2662+ size_t len = 0;
2663+ if (op == 'G') {
2664+ if (use_in) {
2665+ char *wp = wbuf.make_space(1024);
2666+ len = snprintf(wp, 1024, "1\t=\t1\t\t%d\t0\t@\t0\t%d\t%d",
2667+ use_in, use_in, k);
2668+ wbuf.space_wrote(len);
2669+ for (int j = 1; j < use_in; ++j) {
2670+ drand48_r(&randbuf, &kf);
2671+ k = int(kf * tablesize) + firstkey;
2672+ char *wp = wbuf.make_space(1024);
2673+ len = snprintf(wp, 1024, "\t%d", k);
2674+ wbuf.space_wrote(len);
2675+ }
2676+ wbuf.append_literal("\n");
2677+ } else {
2678+ char *wp = wbuf.make_space(1024);
2679+ len = snprintf(wp, 1024, "1\t=\t1\t%d\n", k);
2680+ wbuf.space_wrote(len);
2681+ }
2682+ } else if (op == 'U') {
2683+ char *wp = wbuf.make_space(1024);
2684+ len = snprintf(wp, 1024,
2685+ "1\t=\t1\t%d\t1\t0\tU\t%d_%d%s\n", k, v, k, suffix.c_str());
2686+ wbuf.space_wrote(len);
2687+ }
2688+ break;
2689+ }
2690+ }
2691+ }
2692+ wlen = write(fd.get(), wbuf.begin(), wbuf.size());
2693+ if (wlen != wbuf.size()) {
2694+ fprintf(stderr, "write: %d %d\n", (int)wbuf.size(), wlen);
2695+ return;
2696+ }
2697+ wbuf.clear();
2698+ size_t read_cnt = 0;
2699+ size_t read_pos = 0;
2700+ while (read_cnt < arg.sh.pipe) {
2701+ rlen = read(fd.get(), rbuf + read_pos, sizeof(rbuf) - read_pos);
2702+ if (rlen <= 0) {
2703+ fprintf(stderr, "read: %d\n", rlen);
2704+ return;
2705+ }
2706+ read_pos += rlen;
2707+ while (true) {
2708+ const char *const nl = static_cast<const char *>(memchr(rbuf, '\n',
2709+ read_pos));
2710+ if (nl == 0) {
2711+ break;
2712+ }
2713+ ++read_cnt;
2714+ ++io_success_count;
2715+ const char *t1 = static_cast<const char *>(memchr(rbuf, '\t',
2716+ nl - rbuf));
2717+ if (t1 == 0) {
2718+ fprintf(stderr, "error \n");
2719+ break;
2720+ }
2721+ ++t1;
2722+ const char *t2 = static_cast<const char *>(memchr(t1, '\t',
2723+ nl - t1));
2724+ if (t2 == 0) {
2725+ if (verbose > 1) {
2726+ fprintf(stderr, "key: notfound \n");
2727+ }
2728+ break;
2729+ }
2730+ ++t2;
2731+ if (t1 == rbuf + 2 && rbuf[0] == '0') {
2732+ if (op == 'G') {
2733+ ++op_success_count;
2734+ arg.sh.increment_count();
2735+ } else if (op == 'U') {
2736+ const char *t3 = t2;
2737+ while (t3 != nl && t3[0] >= 0x10) {
2738+ ++t3;
2739+ }
2740+ if (t3 != t2 + 1 || t2[0] != '1') {
2741+ const std::string mess(t2, t3);
2742+ fprintf(stderr, "mod: %s\n", mess.c_str());
2743+ } else {
2744+ ++op_success_count;
2745+ arg.sh.increment_count();
2746+ if (arg.sh.dump && arg.sh.pipe == 1) {
2747+ fwrite(wbuf, wbuflen, 1, stderr);
2748+ }
2749+ }
2750+ }
2751+ } else {
2752+ const char *t3 = t2;
2753+ while (t3 != nl && t3[0] >= 0x10) {
2754+ ++t3;
2755+ }
2756+ const std::string mess(t2, t3);
2757+ fprintf(stderr, "err: %s\n", mess.c_str());
2758+ }
2759+ const size_t rest_size = rbuf + read_pos - (nl + 1);
2760+ if (rest_size != 0) {
2761+ memmove(rbuf, nl + 1, rest_size);
2762+ }
2763+ read_pos = rest_size;
2764+ }
2765+ }
2766+ if (!keep_connection) {
2767+ fd.reset();
2768+ arg.sh.increment_conn(-1);
2769+ } else if (keep_connection > 1 && ++keepconn_count > keep_connection) {
2770+ keepconn_count = 0;
2771+ fd.reset();
2772+ arg.sh.increment_conn(-1);
2773+ }
2774+ const double tm2 = gettimeofday_double();
2775+ set_timing(tm2 - tm1);
2776+ sleep_if();
2777+ if (sched_flag) {
2778+ sched_yield();
2779+ }
2780+ }
2781+ if (dump) {
2782+ fprintf(stderr, "done\n");
2783+ }
2784+}
2785+
2786+void
2787+hstest_thread::sleep_if()
2788+{
2789+ if (arg.sh.usleep) {
2790+ struct timespec ts = {
2791+ arg.sh.usleep / 1000000,
2792+ (arg.sh.usleep % 1000000) * 1000
2793+ };
2794+ nanosleep(&ts, 0);
2795+ }
2796+}
2797+
2798+void
2799+hstest_thread::set_timing(double time_spent)
2800+{
2801+ response_min = std::min(response_min, time_spent);
2802+ response_max = std::max(response_max, time_spent);
2803+ response_sum += time_spent;
2804+ if (op_success_count != 0) {
2805+ response_avg = response_sum / op_success_count;
2806+ }
2807+}
2808+
2809+void
2810+hstest_thread::test_11(int test_num)
2811+{
2812+ const int keep_connection = arg.sh.conf.get_int("keep_connection", 1);
2813+ const int tablesize = arg.sh.conf.get_int("tablesize", 0);
2814+ unsigned int seed = arg.id;
2815+ seed ^= arg.sh.conf.get_int("seed_xor", 0);
2816+ std::string err;
2817+ hstcpcli_ptr cli;
2818+ for (size_t i = 0; i < arg.sh.loop; ++i) {
2819+ if (cli.get() == 0) {
2820+ cli = hstcpcli_i::create(arg.sh.arg);
2821+ cli->request_buf_open_index(0, "hstest", "hstest_table1", "", "v");
2822+ /* pst_num, db, table, index, retflds */
2823+ if (cli->request_send() != 0) {
2824+ fprintf(stderr, "reuqest_send: %s\n", cli->get_error().c_str());
2825+ return;
2826+ }
2827+ size_t num_flds = 0;
2828+ if (cli->response_recv(num_flds) != 0) {
2829+ fprintf(stderr, "reuqest_recv: %s\n", cli->get_error().c_str());
2830+ return;
2831+ }
2832+ cli->response_buf_remove();
2833+ }
2834+ for (size_t j = 0; j < arg.sh.pipe; ++j) {
2835+ char buf[256];
2836+ int k = 0, v = 0, len = 0;
2837+ {
2838+ k = rand_r(&seed);
2839+ v = rand_r(&seed); /* unused */
2840+ if (tablesize != 0) {
2841+ k &= tablesize;
2842+ }
2843+ len = snprintf(buf, sizeof(buf), "%d", k);
2844+ }
2845+ const string_ref key(buf, len);
2846+ const string_ref op("=", 1);
2847+ cli->request_buf_exec_generic(0, op, &key, 1, 1, 0, string_ref(), 0, 0);
2848+ }
2849+ if (cli->request_send() != 0) {
2850+ fprintf(stderr, "reuqest_send: %s\n", cli->get_error().c_str());
2851+ return;
2852+ }
2853+ size_t read_cnt = 0;
2854+ for (size_t j = 0; j < arg.sh.pipe; ++j) {
2855+ size_t num_flds = 0;
2856+ if (cli->response_recv(num_flds) != 0) {
2857+ fprintf(stderr, "reuqest_recv: %s\n", cli->get_error().c_str());
2858+ return;
2859+ }
2860+ {
2861+ ++read_cnt;
2862+ ++io_success_count;
2863+ arg.sh.increment_count();
2864+ {
2865+ ++op_success_count;
2866+ }
2867+ }
2868+ cli->response_buf_remove();
2869+ }
2870+ if (!keep_connection) {
2871+ cli.reset();
2872+ }
2873+ }
2874+}
2875+
2876+void
2877+hstest_thread::test_watch()
2878+{
2879+ const int timelimit = arg.sh.conf.get_int("timelimit", 0);
2880+ const int timelimit_offset = timelimit / 2;
2881+ int loop = 0;
2882+ double t1 = 0, t2 = 0;
2883+ size_t cnt_t1 = 0, cnt_t2 = 0;
2884+ size_t prev_cnt = 0;
2885+ double now_f = 0;
2886+ while (true) {
2887+ sleep(1);
2888+ const size_t cnt = arg.sh.count;
2889+ const size_t df = cnt - prev_cnt;
2890+ prev_cnt = cnt;
2891+ const double now_prev = now_f;
2892+ now_f = gettimeofday_double();
2893+ if (now_prev != 0) {
2894+ const double rps = static_cast<double>(df) / (now_f - now_prev);
2895+ fprintf(stderr, "now: %zu cntdiff: %zu tdiff: %f rps: %f\n",
2896+ static_cast<size_t>(now_f), df, now_f - now_prev, rps);
2897+ }
2898+ if (timelimit != 0) {
2899+ if (arg.sh.wait_conn == 0 || arg.sh.conn_count >= arg.sh.wait_conn) {
2900+ ++loop;
2901+ }
2902+ if (loop == timelimit_offset) {
2903+ t1 = gettimeofday_double();
2904+ cnt_t1 = cnt;
2905+ arg.sh.enable_timing = 1;
2906+ fprintf(stderr, "start timing\n");
2907+ } else if (loop == timelimit_offset + timelimit) {
2908+ t2 = gettimeofday_double();
2909+ cnt_t2 = cnt;
2910+ const size_t cnt_diff = cnt_t2 - cnt_t1;
2911+ const double tdiff = t2 - t1;
2912+ const double qps = cnt_diff / (tdiff != 0 ? tdiff : 1);
2913+ fprintf(stderr, "(%f: %zu, %f: %zu), %10.5f qps\n",
2914+ t1, cnt_t1, t2, cnt_t2, qps);
2915+ size_t keycnt = 0;
2916+ for (int i = 0; i < arg.sh.keygen_size; ++i) {
2917+ if (arg.sh.keygen[i]) {
2918+ ++keycnt;
2919+ }
2920+ }
2921+ fprintf(stderr, "keygen=%zu\n", keycnt);
2922+ break;
2923+ }
2924+ }
2925+ }
2926+#if 0
2927+ int loop = 0;
2928+ double t1 = 0, t2 = 0;
2929+ size_t cnt_t1 = 0, cnt_t2 = 0;
2930+ size_t prev_cnt = 0;
2931+ while (true) {
2932+ sleep(1);
2933+ const size_t cnt = arg.sh.count;
2934+ const size_t df = cnt - prev_cnt;
2935+ prev_cnt = cnt;
2936+ const size_t now = time(0);
2937+ fprintf(stderr, "%zu %zu\n", now, df);
2938+ if (timelimit != 0) {
2939+ ++loop;
2940+ if (loop == timelimit_offset) {
2941+ t1 = gettimeofday_double();
2942+ cnt_t1 = cnt;
2943+ } else if (loop == timelimit_offset + timelimit) {
2944+ t2 = gettimeofday_double();
2945+ cnt_t2 = cnt;
2946+ const size_t cnt_diff = cnt_t2 - cnt_t1;
2947+ const double tdiff = t2 - t1;
2948+ const double qps = cnt_diff / (tdiff != 0 ? tdiff : 1);
2949+ fprintf(stderr, "(%f: %zu, %f: %zu), %10.5f qps\n",
2950+ t1, cnt_t1, t2, cnt_t2, qps);
2951+ size_t keycnt = 0;
2952+ for (int i = 0; i < arg.sh.keygen_size; ++i) {
2953+ if (arg.sh.keygen[i]) {
2954+ ++keycnt;
2955+ }
2956+ }
2957+ fprintf(stderr, "keygen=%zu\n", keycnt);
2958+ _exit(0);
2959+ }
2960+ }
2961+ }
2962+#endif
2963+}
2964+
2965+void
2966+hstest_thread::test_12(int test_num)
2967+{
2968+ /* NOTE: num_threads should be 1 */
2969+ /* create table hstest
2970+ * ( k varchar(255) not null, v varchar(255) not null, primary key(k))
2971+ * engine = innodb; */
2972+ mysqltest_thread_initobj initobj;
2973+ auto_mysql db;
2974+ std::string err;
2975+ unsigned long long err_cnt = 0;
2976+ unsigned long long query_cnt = 0;
2977+ #if 0
2978+ my_bool reconnect = 0;
2979+ if (mysql_options(db, MYSQL_OPT_RECONNECT, &reconnect) != 0) {
2980+ err = "mysql_options() failed";
2981+ ++err_cnt;
2982+ return;
2983+ }
2984+ #endif
2985+ const std::string mysql_host = arg.sh.conf.get_str("host", "localhost");
2986+ const int mysql_port = arg.sh.conf.get_int("mysqlport", 3306);
2987+ const unsigned int num = arg.sh.loop;
2988+ const size_t pipe = arg.sh.pipe;
2989+ const std::string mysql_user = arg.sh.conf.get_str("mysqluser", "root");
2990+ const std::string mysql_passwd = arg.sh.conf.get_str("mysqlpass", "");
2991+ const std::string mysql_dbname = arg.sh.conf.get_str("db", "hstest");
2992+ const int keep_connection = arg.sh.conf.get_int("keep_connection", 1);
2993+ const int verbose = arg.sh.conf.get_int("verbose", 1);
2994+ const int use_handler = arg.sh.conf.get_int("handler", 0);
2995+ int connected = 0;
2996+ unsigned int k = 0;
2997+ string_buffer buf;
2998+ for (unsigned int i = 0; i < num; ++i) {
2999+ const int flags = 0;
3000+ if (connected == 0 && !mysql_real_connect(db, mysql_host.c_str(),
3001+ mysql_user.c_str(), mysql_user.empty() ? 0 : mysql_passwd.c_str(),
3002+ mysql_dbname.c_str(), mysql_port, 0, flags)) {
3003+ err = "failed to connect: " + std::string(mysql_error(db));
3004+ if (verbose >= 1) {
3005+ fprintf(stderr, "e=[%s]\n", err.c_str());
3006+ }
3007+ ++err_cnt;
3008+ return;
3009+ }
3010+ int r = 0;
3011+ if (connected == 0 && use_handler) {
3012+ const char *const q = "handler hstest open";
3013+ r = mysql_real_query(db, q, strlen(q));
3014+ if (r != 0) {
3015+ err = 1;
3016+ }
3017+ }
3018+ connected = 1;
3019+ std::string result_str;
3020+ unsigned int err = 0;
3021+ unsigned int num_flds = 0, num_affected_rows = 0;
3022+ int got_data = 0;
3023+ buf.clear();
3024+ buf.append_literal("insert into hstest values ");
3025+ for (size_t j = 0; j < pipe; ++j) {
3026+ const unsigned int v = ~k;
3027+ if (j != 0) {
3028+ buf.append_literal(",");
3029+ }
3030+ char *wp = buf.make_space(64);
3031+ int buf_query_len = snprintf(wp, 64, "('k%u', 'v%u')", k, v);
3032+ buf.space_wrote(buf_query_len);
3033+ ++k;
3034+ }
3035+ if (r == 0) {
3036+ r = mysql_real_query(db, buf.begin(), buf.size());
3037+ ++query_cnt;
3038+ }
3039+ if (r != 0) {
3040+ err = 1;
3041+ } else {
3042+ auto_mysql_res res(db);
3043+ if (res != 0) {
3044+ if (verbose >= 0) {
3045+ num_flds = mysql_num_fields(res);
3046+ MYSQL_ROW row = 0;
3047+ while ((row = mysql_fetch_row(res)) != 0) {
3048+ got_data = 1;
3049+ unsigned long *const lengths = mysql_fetch_lengths(res);
3050+ if (verbose >= 2) {
3051+ for (unsigned int i = 0; i < num_flds; ++i) {
3052+ if (!result_str.empty()) {
3053+ result_str += " ";
3054+ }
3055+ result_str += std::string(row[i], lengths[i]);
3056+ }
3057+ }
3058+ }
3059+ }
3060+ } else {
3061+ if (mysql_field_count(db) == 0) {
3062+ num_affected_rows = mysql_affected_rows(db);
3063+ } else {
3064+ err = 1;
3065+ }
3066+ }
3067+ }
3068+ if (verbose >= 2 || (verbose >= 1 && err != 0)) {
3069+ if (err) {
3070+ ++err_cnt;
3071+ const char *const errstr = mysql_error(db);
3072+ fprintf(stderr, "e=[%s] a=%u q=[%s]\n", errstr,
3073+ num_affected_rows, std::string(buf.begin(), buf.size()).c_str());
3074+ } else {
3075+ fprintf(stderr, "a=%u q=[%s] r=[%s]\n", num_affected_rows,
3076+ std::string(buf.begin(), buf.size()).c_str(),
3077+ result_str.c_str());
3078+ }
3079+ }
3080+ if (err == 0) {
3081+ ++io_success_count;
3082+ if (num_affected_rows > 0 || got_data > 0) {
3083+ ++op_success_count;
3084+ }
3085+ arg.sh.increment_count(pipe);
3086+ }
3087+ if (!keep_connection) {
3088+ db.reset();
3089+ connected = 0;
3090+ }
3091+ }
3092+ if (verbose >= 1) {
3093+ fprintf(stderr, "thread finished (error_count=%llu)\n", err_cnt);
3094+ }
3095+}
3096+
3097+void
3098+hstest_thread::test_21(int num)
3099+{
3100+ /* fsync test */
3101+ unsigned int id = arg.id;
3102+ std::string err;
3103+ #if 0
3104+ if (socket_connect(fd, arg.sh.arg, err) != 0) {
3105+ fprintf(stderr, "connect: %d %s\n", errno, strerror(errno));
3106+ return;
3107+ }
3108+ #endif
3109+ auto_file logfd;
3110+ char fname[1024];
3111+ snprintf(fname, sizeof(fname), "synctest_%u", id);
3112+ int open_flags = O_WRONLY | O_CREAT | O_TRUNC | O_APPEND;
3113+ logfd.reset(open(fname, open_flags, 0644));
3114+ if (logfd.get() < 0) {
3115+ fprintf(stderr, "open: %s: %d %s\n", fname, errno, strerror(errno));
3116+ return;
3117+ }
3118+ char buf[1024];
3119+ unsigned long long count = 0;
3120+ while (true) {
3121+ snprintf(buf, sizeof(buf), "%u %llu\n", id, count);
3122+ const size_t len = strlen(buf);
3123+ if (write(logfd.get(), buf, len) != (ssize_t)len) {
3124+ fprintf(stderr, "write: %s: %d %s\n", fname, errno, strerror(errno));
3125+ return;
3126+ }
3127+ #if 0
3128+ if (write(fd.get(), buf, len) != (ssize_t)len) {
3129+ fprintf(stderr, "write(sock): %d %s\n", errno, strerror(errno));
3130+ return;
3131+ }
3132+ #endif
3133+ if (fdatasync(logfd.get()) != 0) {
3134+ fprintf(stderr, "fsync: %s: %d %s\n", fname, errno, strerror(errno));
3135+ return;
3136+ }
3137+ ++count;
3138+ ++op_success_count;
3139+ arg.sh.increment_count();
3140+ }
3141+}
3142+
3143+void
3144+hstest_thread::test_22(int num)
3145+{
3146+ /* dd if=/dev/zero of=dummy.dat bs=1024M count=100 */
3147+ unsigned int id = arg.id;
3148+ std::string err;
3149+ auto_file filefd;
3150+ char fname[1024];
3151+ snprintf(fname, sizeof(fname), "dummy.dat");
3152+ int open_flags = O_RDONLY | O_DIRECT;
3153+ filefd.reset(open(fname, open_flags, 0644));
3154+ if (filefd.get() < 0) {
3155+ fprintf(stderr, "open: %s: %d %s\n", fname, errno, strerror(errno));
3156+ return;
3157+ }
3158+ char buf_x[4096 * 2];
3159+ char *const buf = (char *)(size_t(buf_x + 4096) / 4096 * 4096);
3160+ unsigned long long count = 0;
3161+ drand48_data randbuf;
3162+ unsigned long long seed = time(0);
3163+ seed *= 10;
3164+ seed += id;
3165+ srand48_r(seed, &randbuf);
3166+ for (unsigned int i = 0; i < arg.sh.loop; ++i) {
3167+ double kf = 0;
3168+ drand48_r(&randbuf, &kf);
3169+ kf *= (209715200 / 1);
3170+ // fprintf(stderr, "v=%f\n", kf);
3171+ off_t v = static_cast<off_t>(kf);
3172+ v %= (209715200 / 1);
3173+ v *= (512 * 1);
3174+ const double tm1 = gettimeofday_double();
3175+ const ssize_t r = pread(filefd.get(), buf, (512 * 1), v);
3176+ const double tm2 = gettimeofday_double();
3177+ if (r < 0) {
3178+ fprintf(stderr, "pread: %s: %d %s\n", fname, errno, strerror(errno));
3179+ return;
3180+ }
3181+ ++count;
3182+ ++op_success_count;
3183+ arg.sh.increment_count();
3184+ set_timing(tm2 - tm1);
3185+ }
3186+}
3187+
3188+void
3189+hstest_thread::operator ()()
3190+{
3191+ if (arg.watch_flag) {
3192+ return test_watch();
3193+ }
3194+ int test_num = arg.sh.conf.get_int("test", 1);
3195+ if (test_num == 1) {
3196+ test_1();
3197+ } else if (test_num == 2 || test_num == 3) {
3198+ test_2_3(test_num);
3199+ } else if (test_num == 4 || test_num == 5) {
3200+ test_4_5(test_num);
3201+ } else if (test_num == 6) {
3202+ test_6(test_num);
3203+ } else if (test_num == 7) {
3204+ test_7(test_num);
3205+ } else if (test_num == 8) {
3206+ test_8(test_num);
3207+ } else if (test_num == 9) {
3208+ test_9(test_num);
3209+ } else if (test_num == 10) {
3210+ test_10(test_num);
3211+ } else if (test_num == 11) {
3212+ test_11(test_num);
3213+ } else if (test_num == 12) {
3214+ test_12(test_num);
3215+ } else if (test_num == 21) {
3216+ test_21(test_num);
3217+ } else if (test_num == 22) {
3218+ test_22(test_num);
3219+ }
3220+ const int halt = arg.sh.conf.get_int("halt", 0);
3221+ if (halt) {
3222+ fprintf(stderr, "thread halted\n");
3223+ while (true) {
3224+ sleep(100000);
3225+ }
3226+ }
3227+ fprintf(stderr, "thread finished\n");
3228+}
3229+
3230+int
3231+hstest_main(int argc, char **argv)
3232+{
3233+ ignore_sigpipe();
3234+ hstest_shared shared;
3235+ parse_args(argc, argv, shared.conf);
3236+ shared.conf["port"] = shared.conf["hsport"];
3237+ shared.arg.set(shared.conf);
3238+ shared.loop = shared.conf.get_int("num", 1000);
3239+ shared.pipe = shared.conf.get_int("pipe", 1);
3240+ shared.verbose = shared.conf.get_int("verbose", 1);
3241+ const int tablesize = shared.conf.get_int("tablesize", 0);
3242+ std::vector<char> keygen(tablesize);
3243+ shared.keygen = &keygen[0];
3244+ shared.keygen_size = tablesize;
3245+ shared.usleep = shared.conf.get_int("usleep", 0);
3246+ shared.dump = shared.conf.get_int("dump", 0);
3247+ shared.num_threads = shared.conf.get_int("num_threads", 10);
3248+ shared.wait_conn = shared.conf.get_int("wait_conn", 0);
3249+ const std::string op = shared.conf.get_str("op", "G");
3250+ if (op.size() > 0) {
3251+ shared.op = op[0];
3252+ }
3253+ #if 0
3254+ const int localdb_flag = shared.conf.get_int("local", 0);
3255+ if (localdb_flag) {
3256+ shared.localdb = database_i::create(shared.conf);
3257+ }
3258+ #endif
3259+ const int num_thrs = shared.num_threads;
3260+ typedef thread<hstest_thread> thread_type;
3261+ typedef std::auto_ptr<thread_type> thread_ptr;
3262+ typedef auto_ptrcontainer< std::vector<thread_type *> > thrs_type;
3263+ thrs_type thrs;
3264+ for (int i = 0; i < num_thrs; ++i) {
3265+ const hstest_thread::arg_type arg(i, shared, false);
3266+ thread_ptr thr(new thread<hstest_thread>(arg));
3267+ thrs.push_back_ptr(thr);
3268+ }
3269+ for (size_t i = 0; i < thrs.size(); ++i) {
3270+ thrs[i]->start();
3271+ }
3272+ thread_ptr watch_thread;
3273+ const int timelimit = shared.conf.get_int("timelimit", 0);
3274+ {
3275+ const hstest_thread::arg_type arg(0, shared, true);
3276+ watch_thread = thread_ptr(new thread<hstest_thread>(arg));
3277+ watch_thread->start();
3278+ }
3279+ size_t iocnt = 0, opcnt = 0;
3280+ double respmin = 999999, respmax = 0;
3281+ double respsum = 0;
3282+ if (timelimit != 0) {
3283+ watch_thread->join();
3284+ }
3285+ for (size_t i = 0; i < thrs.size(); ++i) {
3286+ if (timelimit == 0) {
3287+ thrs[i]->join();
3288+ }
3289+ iocnt += (*thrs[i])->io_success_count;
3290+ opcnt += (*thrs[i])->op_success_count;
3291+ respmin = std::min(respmin, (*thrs[i])->response_min);
3292+ respmax = std::max(respmax, (*thrs[i])->response_max);
3293+ respsum += (*thrs[i])->response_sum;
3294+ }
3295+ fprintf(stderr, "io_success_count=%zu op_success_count=%zu\n", iocnt, opcnt);
3296+ fprintf(stderr, "respmin=%f respmax=%f respsum=%f respavg=%f\n",
3297+ respmin, respmax, respsum, respsum / opcnt);
3298+ size_t keycnt = 0;
3299+ for (size_t i = 0; i < keygen.size(); ++i) {
3300+ if (keygen[i]) {
3301+ ++keycnt;
3302+ }
3303+ }
3304+ fprintf(stderr, "keycnt=%zu\n", keycnt);
3305+ _exit(0);
3306+ return 0;
3307+}
3308+
3309+};
3310+
3311+int
3312+main(int argc, char **argv)
3313+{
3314+ return dena::hstest_main(argc, argv);
3315+}
3316+
3317
3318=== added file 'HandlerSocket-Plugin-for-MySQL/client/hstest.pl'
3319--- HandlerSocket-Plugin-for-MySQL/client/hstest.pl 1970-01-01 00:00:00 +0000
3320+++ HandlerSocket-Plugin-for-MySQL/client/hstest.pl 2012-02-07 16:47:34 +0000
3321@@ -0,0 +1,228 @@
3322+#!/usr/bin/perl
3323+
3324+# vim:sw=8:ai:ts=8
3325+
3326+use strict;
3327+use warnings;
3328+
3329+use DBI;
3330+use Net::HandlerSocket;
3331+
3332+my %conf = ();
3333+for my $i (@ARGV) {
3334+ my ($k, $v) = split(/=/, $i);
3335+ $conf{$k} = $v;
3336+}
3337+
3338+my $verbose = get_conf("verbose", 0);
3339+my $actions_str = get_conf("actions", "hsread");
3340+my $tablesize = get_conf("tablesize", 10000);
3341+my $db = get_conf("db", "hstest");
3342+my $table = get_conf("table", "hstest_table1");
3343+my $engine = get_conf("engine", "innodb");
3344+my $host = get_conf("host", "localhost");
3345+my $mysqlport = get_conf("mysqlport", 3306);
3346+my $mysqluser = get_conf("mysqluser", "root");
3347+my $mysqlpass = get_conf("mysqlpass", "");
3348+my $hsport = get_conf("hsport", 9999);
3349+my $loop = get_conf("loop", 10000);
3350+my $op = get_conf("op", "=");
3351+my $ssps = get_conf("ssps", 0);
3352+my $num_moreflds = get_conf("moreflds", 0);
3353+my $moreflds_prefix = get_conf("moreflds_prefix", "column0123456789_");
3354+my $keytype = get_conf("keytype", "varchar(32)");
3355+my $file = get_conf("file", undef);
3356+
3357+my $dsn = "DBI:mysql:database=;host=$host;port=$mysqlport"
3358+ . ";mysql_server_prepare=$ssps";
3359+my $dbh = DBI->connect($dsn, $mysqluser, $mysqlpass, { RaiseError => 1 });
3360+my $hsargs = { 'host' => $host, 'port' => $hsport };
3361+my $cli = new Net::HandlerSocket($hsargs);
3362+
3363+my @actions = split(/,/, $actions_str);
3364+for my $action (@actions) {
3365+ if ($action eq "table") {
3366+ print("TABLE $db.$table\n");
3367+ $dbh->do("drop database if exists $db");
3368+ $dbh->do("create database $db");
3369+ $dbh->do("use $db");
3370+ my $moreflds = get_createtbl_moreflds_str();
3371+ $dbh->do(
3372+ "create table $table (" .
3373+ "k $keytype primary key" .
3374+ ",v varchar(32) not null" .
3375+ $moreflds .
3376+ ") character set utf8 collate utf8_bin " .
3377+ "engine = $engine");
3378+ } elsif ($action eq "insert") {
3379+ print("INSERT $db.$table tablesize=$tablesize\n");
3380+ $dbh->do("use $db");
3381+ my $moreflds = get_insert_moreflds_str();
3382+ for (my $i = 0; $i < $tablesize; $i += 100) {
3383+ my $qstr = "insert into $db.$table values";
3384+ for (my $j = 0; $j < 100; ++$j) {
3385+ if ($j != 0) {
3386+ $qstr .= ",";
3387+ }
3388+ my $k = "" . ($i + $j);
3389+ my $v = "v" . int(rand(1000)) . ($i + $j);
3390+ $qstr .= "('$k', '$v'";
3391+ for (my $j = 0; $j < $num_moreflds; ++$j) {
3392+ $qstr .= ",'$j'";
3393+ }
3394+ $qstr .= ")";
3395+ }
3396+ $dbh->do($qstr);
3397+ print "$i/$tablesize\n" if $i % 1000 == 0;
3398+ }
3399+ } elsif ($action eq "read") {
3400+ print("READ $db.$table op=$op loop=$loop\n");
3401+ $dbh->do("use $db");
3402+ my $moreflds = get_select_moreflds_str();
3403+ my $sth = $dbh->prepare(
3404+ "select k,v$moreflds from $db.$table where k = ?");
3405+ for (my $i = 0; $i < $loop; ++$i) {
3406+ my $k = "" . int(rand($tablesize));
3407+ # print "k=$k\n";
3408+ $sth->execute($k);
3409+ if ($verbose >= 10) {
3410+ print "RET:";
3411+ while (my $ref = $sth->fetchrow_arrayref()) {
3412+ my $rk = $ref->[0];
3413+ my $rv = $ref->[1];
3414+ print " $rk $rv";
3415+ }
3416+ print "\n";
3417+ }
3418+ print "$i/$loop\n" if $i % 1000 == 0;
3419+ }
3420+ } elsif ($action eq "hsinsert") {
3421+ print("HSINSERT $db.$table tablesize=$tablesize\n");
3422+ $cli->open_index(1, $db, $table, '', 'k,v');
3423+ for (my $i = 0; $i < $tablesize; ++$i) {
3424+ my $k = "" . $i;
3425+ my $v = "v" . int(rand(1000)) . $i;
3426+ my $r = $cli->execute_insert(1, [ $k, $v ]);
3427+ if ($r->[0] != 0) {
3428+ die;
3429+ }
3430+ print "$i/$tablesize\n" if $i % 1000 == 0;
3431+ }
3432+ } elsif ($action eq "hsread") {
3433+ print("HSREAD $db.$table op=$op loop=$loop\n");
3434+ my $moreflds = get_select_moreflds_str();
3435+ $cli->open_index(1, $db, $table, '', "k,v$moreflds");
3436+ for (my $i = 0; $i < $loop; ++$i) {
3437+ my $k = "" . int(rand($tablesize));
3438+ # print "k=$k\n";
3439+ my $r = $cli->execute_find(1, $op, [ $k ], 1, 0);
3440+ if ($verbose >= 10) {
3441+ my $len = scalar(@{$r});
3442+ print "LEN=$len";
3443+ for my $e (@{$r}) {
3444+ print " [$e]";
3445+ }
3446+ print "\n";
3447+ }
3448+ print "$i/$loop\n" if $i % 1000 == 0;
3449+ }
3450+ } elsif ($action eq "hsupdate") {
3451+ my $vbase = "v" . int(rand(1000));
3452+ print("HSUPDATE $db.$table op=$op loop=$loop vbase=$vbase\n");
3453+ $cli->open_index(1, $db, $table, '', 'v');
3454+ for (my $i = 0; $i < $loop; ++$i) {
3455+ my $k = "" . int(rand($tablesize));
3456+ my $v = $vbase . $i;
3457+ print "k=$k v=$v\n";
3458+ my $r = $cli->execute_update(1, $op, [ $k ], 1, 0,
3459+ [ $v ]);
3460+ if ($verbose >= 10) {
3461+ print "UP k=$k v=$v\n";
3462+ }
3463+ print "$i/$loop\n" if $i % 1000 == 0;
3464+ }
3465+ } elsif ($action eq "hsdelete") {
3466+ print("HSDELETE $db.$table op=$op loop=$loop\n");
3467+ $cli->open_index(1, $db, $table, '', '');
3468+ for (my $i = 0; $i < $loop; ++$i) {
3469+ my $k = "" . int(rand($tablesize));
3470+ print "k=$k\n";
3471+ my $r = $cli->execute_delete(1, $op, [ $k ], 1, 0);
3472+ if ($verbose >= 10) {
3473+ print "DEL k=$k\n";
3474+ }
3475+ print "$i/$loop\n" if $i % 1000 == 0;
3476+ }
3477+ } elsif ($action eq "verify") {
3478+ verify_do();
3479+ }
3480+}
3481+
3482+sub verify_do {
3483+ my ($fail_cnt, $ok_cnt) = (0, 0);
3484+ my $sth = $dbh->prepare("select v from $db.$table where k = ?");
3485+ use FileHandle;
3486+ my $fh = new FileHandle($file, "r");
3487+ while (my $line = <$fh>) {
3488+ chomp($line);
3489+ my @vec = split(/\t/, $line);
3490+ my $k = $vec[3];
3491+ my $v = $vec[7];
3492+ next if (!defined($k) || !defined($v));
3493+ # print "$k $v\n";
3494+ $sth->execute($k);
3495+ my $aref = $sth->fetchrow_arrayref();
3496+ if (!defined($aref)) {
3497+ print "FAILED: $k notfound\n";
3498+ ++$fail_cnt;
3499+ } else {
3500+ my $gv = $aref->[0];
3501+ if ($gv ne $v) {
3502+ print "FAILED: $k got=$gv expected=$v\n";
3503+ ++$fail_cnt;
3504+ } else {
3505+ print "OK: $k $v $gv\n" if $verbose >= 10;
3506+ ++$ok_cnt;
3507+ }
3508+ }
3509+ }
3510+ print "OK=$ok_cnt FAIL=$fail_cnt\n";
3511+}
3512+
3513+sub get_conf {
3514+ my ($key, $def) = @_;
3515+ my $val = $conf{$key};
3516+ if ($val) {
3517+ print "$key=$val\n";
3518+ } else {
3519+ $val = $def;
3520+ $def ||= '';
3521+ print "$key=$def(default)\n";
3522+ }
3523+ return $val;
3524+}
3525+
3526+sub get_createtbl_moreflds_str {
3527+ my $s = "";
3528+ for (my $j = 0; $j < $num_moreflds; ++$j) {
3529+ $s .= ",$moreflds_prefix$j varchar(30)";
3530+ }
3531+ return $s;
3532+}
3533+
3534+sub get_select_moreflds_str {
3535+ my $s = "";
3536+ for (my $i = 0; $i < $num_moreflds; ++$i) {
3537+ $s .= ",$moreflds_prefix$i";
3538+ }
3539+ return $s;
3540+}
3541+
3542+sub get_insert_moreflds_str {
3543+ my $s = "";
3544+ for (my $i = 0; $i < $num_moreflds; ++$i) {
3545+ $s .= ",?";
3546+ }
3547+ return $s;
3548+}
3549+
3550
3551=== added file 'HandlerSocket-Plugin-for-MySQL/client/hstest_hs.sh'
3552--- HandlerSocket-Plugin-for-MySQL/client/hstest_hs.sh 1970-01-01 00:00:00 +0000
3553+++ HandlerSocket-Plugin-for-MySQL/client/hstest_hs.sh 2012-02-07 16:47:34 +0000
3554@@ -0,0 +1,4 @@
3555+#!/bin/bash
3556+
3557+exec ./hstest test=10 tablesize=10000 host=localhost hsport=9998 num=10000000 \
3558+ num_threads=100 timelimit=10 $@
3559
3560=== added file 'HandlerSocket-Plugin-for-MySQL/client/hstest_hs_more50.sh'
3561--- HandlerSocket-Plugin-for-MySQL/client/hstest_hs_more50.sh 1970-01-01 00:00:00 +0000
3562+++ HandlerSocket-Plugin-for-MySQL/client/hstest_hs_more50.sh 2012-02-07 16:47:34 +0000
3563@@ -0,0 +1,4 @@
3564+#!/bin/bash
3565+
3566+exec ./hstest test=10 key_mask=9999 host=localhost port=9998 num=10000000 \
3567+ num_threads=100 timelimit=10 moreflds=50 $@
3568
3569=== added file 'HandlerSocket-Plugin-for-MySQL/client/hstest_md.sh'
3570--- HandlerSocket-Plugin-for-MySQL/client/hstest_md.sh 1970-01-01 00:00:00 +0000
3571+++ HandlerSocket-Plugin-for-MySQL/client/hstest_md.sh 2012-02-07 16:47:34 +0000
3572@@ -0,0 +1,7 @@
3573+#!/bin/bash
3574+
3575+./hstest test=7 key_mask=9999 host=localhost port=11211 num=10000 \
3576+ num_threads=10 timelimit=10 op=R $@
3577+./hstest test=7 key_mask=9999 host=localhost port=11211 num=1000000 \
3578+ num_threads=100 timelimit=10 op=G $@
3579+
3580
3581=== added file 'HandlerSocket-Plugin-for-MySQL/client/hstest_my.sh'
3582--- HandlerSocket-Plugin-for-MySQL/client/hstest_my.sh 1970-01-01 00:00:00 +0000
3583+++ HandlerSocket-Plugin-for-MySQL/client/hstest_my.sh 2012-02-07 16:47:34 +0000
3584@@ -0,0 +1,3 @@
3585+#!/bin/bash
3586+exec ./hstest test=9 tablesize=9999 host=localhost mysqlport=3306 num=1000000 \
3587+ num_threads=100 verbose=1 timelimit=10 $@
3588
3589=== added file 'HandlerSocket-Plugin-for-MySQL/client/hstest_my_more50.sh'
3590--- HandlerSocket-Plugin-for-MySQL/client/hstest_my_more50.sh 1970-01-01 00:00:00 +0000
3591+++ HandlerSocket-Plugin-for-MySQL/client/hstest_my_more50.sh 2012-02-07 16:47:34 +0000
3592@@ -0,0 +1,3 @@
3593+#!/bin/bash
3594+exec ./hstest test=9 key_mask=9999 host=localhost port=3306 num=1000000 \
3595+ num_threads=100 verbose=1 timelimit=10 moreflds=50 $@
3596
3597=== added file 'HandlerSocket-Plugin-for-MySQL/configure.ac'
3598--- HandlerSocket-Plugin-for-MySQL/configure.ac 1970-01-01 00:00:00 +0000
3599+++ HandlerSocket-Plugin-for-MySQL/configure.ac 2012-02-07 16:47:34 +0000
3600@@ -0,0 +1,144 @@
3601+# -*- Autoconf -*-
3602+# Process this file with autoconf to produce a configure script.
3603+
3604+#AC_PREREQ([2.63b])
3605+AC_INIT([handlersocket-plugin], [1.0.6], [https://github.com/ahiguti/HandlerSocket-Plugin-for-MySQL/issues])
3606+AC_CONFIG_HEADERS([config.h])
3607+AM_INIT_AUTOMAKE([-Wall -Werror foreign])
3608+AC_CONFIG_SRCDIR([libhsclient/fatal.cpp])
3609+AC_CONFIG_MACRO_DIR([m4])
3610+
3611+AC_PROG_CC
3612+AC_PROG_CXX
3613+AC_PROG_CPP
3614+AC_PROG_LIBTOOL
3615+
3616+ac_mysql_debug=
3617+AC_ARG_ENABLE(mysql-debug,
3618+ [AS_HELP_STRING([--enable-mysql-debug], [specify whether MySQL is build with DBUG_ON])],[ac_mysql_debug="$enableval"],[ac_mysql_debug=no])
3619+AC_MSG_CHECKING([if --enable-mysql-debug is specified])
3620+AC_MSG_RESULT($ac_mysql_debug)
3621+
3622+AC_DEFUN([CONFIG_OPTION_MYSQL],[
3623+ AC_MSG_CHECKING([mysql source])
3624+
3625+ MYSQL_SOURCE_VERSION=
3626+ MYSQL_INC=
3627+ ac_mysql_source_dir=
3628+ AC_ARG_WITH([mysql-source],
3629+ [AS_HELP_STRING([--with-mysql-source=PATH], [MySQL source directory PATH])],
3630+ [
3631+ ac_mysql_source_dir=`cd $withval && pwd`
3632+ if test -f "$ac_mysql_source_dir/sql/handler.h" ; then
3633+ MYSQL_INC="-I$ac_mysql_source_dir/sql"
3634+ MYSQL_INC="$MYSQL_INC -I$ac_mysql_source_dir/include"
3635+ MYSQL_INC="$MYSQL_INC -I$ac_mysql_source_dir/regex"
3636+ MYSQL_INC="$MYSQL_INC -I$ac_mysql_source_dir"
3637+ AC_SUBST(MYSQL_INC)
3638+ if test -f "$ac_mysql_source_dir/VERSION"; then
3639+ source "$ac_mysql_source_dir/VERSION"
3640+ MYSQL_SOURCE_VERSION="$MYSQL_VERSION_MAJOR.$MYSQL_VERSION_MINOR.$MYSQL_VERSION_PATCH"
3641+ else
3642+ if test -f "$ac_mysql_source_dir/configure.in"; then
3643+ MYSQL_SOURCE_VERSION=`cat $ac_mysql_source_dir/configure.in | grep "\[[MySQL Server\]]" | sed -e "s|.*\([[0-9]]\+\.[[0-9]]\+\.[[0-9]]\+[[0-9a-zA-Z\_\-]]*\).*|\1|"`
3644+ else
3645+ AC_MSG_ERROR([invalid MySQL source directory: $ac_mysql_source_dir])
3646+ fi
3647+ fi
3648+ AC_MSG_RESULT([yes: Using $ac_mysql_source_dir, version $MYSQL_SOURCE_VERSION])
3649+ else
3650+ AC_MSG_ERROR([invalid MySQL source directory: $ac_mysql_source_dir])
3651+ fi
3652+ ],
3653+ [AC_MSG_ERROR([--with-mysql-source=PATH is required for standalone build])]
3654+ )
3655+
3656+ MYSQL_BIN_VERSION=
3657+ ac_mysql_config=
3658+ AC_ARG_WITH([mysql-bindir],
3659+ [AS_HELP_STRING([--with-mysql-bindir=PATH], [MySQL binary directory PATH. This should be the directory where mysql_config is located.])],
3660+ [
3661+ mysql_bin_dir=`cd $withval 2> /dev/null && pwd || echo ""`
3662+ ac_mysql_config="$mysql_bin_dir/mysql_config"
3663+ ],
3664+ [
3665+ AC_PATH_PROG([ac_mysql_config], [mysql_config])
3666+ ]
3667+ )
3668+
3669+ AC_MSG_CHECKING([mysql binary])
3670+ if test ! -x "$ac_mysql_config" ; then
3671+ AC_MSG_ERROR([mysql_config not found! You have to specify the directory where mysql_config resides to --with-mysql-bindir=PATH.])
3672+ fi
3673+
3674+ MYSQL_CFLAGS_ADD=`"$ac_mysql_config" --cflags`
3675+ MYSQL_CFLAGS="$MYSQL_CFLAGS $MYSQL_CFLAGS_ADD"
3676+ if test "$ac_mysql_debug" = "yes"; then
3677+ MYSQL_CFLAGS="$MYSQL_CFLAGS -DDBUG_ON -DENABLED_DEBUG_SYNC"
3678+ else
3679+ MYSQL_CFLAGS="$MYSQL_CFLAGS -DDBUG_OFF"
3680+ fi
3681+ AC_SUBST(MYSQL_CFLAGS)
3682+
3683+ MYSQL_BIN_VERSION=`"$ac_mysql_config" --version`
3684+ AC_MSG_RESULT([yes: Using $ac_mysql_config, version $MYSQL_BIN_VERSION])
3685+
3686+ MYSQL_LIB=`"$ac_mysql_config" --libs_r`
3687+ LIB_DIR=`echo $MYSQL_LIB | sed -e "s|.*-L/|/|" | sed -e "s| .*||"`
3688+ # FIXME
3689+ if test a`basename "$LIB_DIR"` = amysql ; then
3690+ MYSQL_LIB="-L`dirname $LIB_DIR` $MYSQL_LIB"
3691+ # FIXME
3692+ fi
3693+ AC_SUBST(MYSQL_LIB)
3694+
3695+ if test a$MYSQL_SOURCE_VERSION != a$MYSQL_BIN_VERSION ; then
3696+ AC_MSG_ERROR([MySQL source version does not match MySQL binary version])
3697+ fi
3698+
3699+ AC_MSG_CHECKING([mysql plugin dir])
3700+ ac_mysql_plugin_dir=
3701+ AC_ARG_WITH([mysql-plugindir],
3702+ [AS_HELP_STRING([--with-mysql-plugindir=PATH], [MySQL plugin directory where handlersocket.so to be copied])],
3703+ [
3704+ ac_mysql_plugin_dir=`cd $withval && pwd`
3705+ if test -d "$ac_mysql_plugin_dir/" ; then
3706+ PLUGIN_DIR="$ac_mysql_plugin_dir"
3707+ AC_SUBST(PLUGIN_DIR)
3708+ AC_MSG_RESULT([yes: Using $ac_mysql_plugin_dir])
3709+ else
3710+ AC_MSG_ERROR([invalid MySQL plugin directory : $ac_mysql_plugin_dir])
3711+ fi
3712+ ],
3713+ [
3714+ LIB_DIR_TMP=`"$ac_mysql_config" --plugindir`
3715+ if test ! -d "$LIB_DIR_TMP"; then
3716+ LIB_DIR_TMP=`"$ac_mysql_config" --libs_r | sed -e "s|.*-L/|/|" | sed -e "s| .*||"`/plugin
3717+ # FIXME
3718+ fi
3719+ ac_mysql_plugin_dir=$LIB_DIR_TMP
3720+ PLUGIN_DIR="$ac_mysql_plugin_dir"
3721+ AC_SUBST(PLUGIN_DIR)
3722+ AC_MSG_RESULT([--with-mysql-plugindir was not set. Using $ac_mysql_plugin_dir])
3723+ ]
3724+ )
3725+])
3726+
3727+HANDLERSOCKET_SUBDIRS="libhsclient"
3728+AC_ARG_ENABLE(handlersocket_server,
3729+ [ --enable-handlersocket-server build HandlerSocket plugin (defalut=yes)])
3730+if test "$enable_handlersocket_server" != "no"; then
3731+ CONFIG_OPTION_MYSQL
3732+ HANDLERSOCKET_SUBDIRS="libhsclient handlersocket client"
3733+fi
3734+AC_SUBST(HANDLERSOCKET_SUBDIRS)
3735+
3736+CFLAGS="$CFLAGS -Werror"
3737+CXXFLAGS="$CXXFLAGS -Wall -g -fno-rtti -fno-exceptions -fPIC -DPIC"
3738+
3739+AC_CONFIG_FILES([Makefile
3740+ handlersocket/Makefile
3741+ libhsclient/Makefile
3742+ client/Makefile])
3743+
3744+AC_OUTPUT
3745
3746=== added directory 'HandlerSocket-Plugin-for-MySQL/docs-en'
3747=== added file 'HandlerSocket-Plugin-for-MySQL/docs-en/about-handlersocket.en.txt'
3748--- HandlerSocket-Plugin-for-MySQL/docs-en/about-handlersocket.en.txt 1970-01-01 00:00:00 +0000
3749+++ HandlerSocket-Plugin-for-MySQL/docs-en/about-handlersocket.en.txt 2012-02-07 16:47:34 +0000
3750@@ -0,0 +1,72 @@
3751+
3752+-----------------------------------------------------------------------------
3753+HandlerSocket plugin for MySQL
3754+
3755+Copyright (c) 2010 DeNA Co.,Ltd.
3756+All rights reserved.
3757+
3758+Redistribution and use in source and binary forms, with or without
3759+modification, are permitted provided that the following conditions are met:
3760+
3761+ * Redistributions of source code must retain the above copyright
3762+ notice, this list of conditions and the following disclaimer.
3763+ * Redistributions in binary form must reproduce the above copyright
3764+ notice, this list of conditions and the following disclaimer in the
3765+ documentation and/or other materials provided with the distribution.
3766+ * Neither the name of DeNA Co.,Ltd. nor the names of its contributors
3767+ may be used to endorse or promote products derived from this software
3768+ without specific prior written permission.
3769+
3770+THIS SOFTWARE IS PROVIDED BY DeNA Co.,Ltd. "AS IS" AND ANY EXPRESS OR
3771+IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
3772+MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO
3773+EVENT SHALL DeNA Co.,Ltd. BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
3774+SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
3775+PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS;
3776+OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
3777+WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
3778+OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
3779+ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
3780+
3781+
3782+-----------------------------------------------------------------------------
3783+About HandlerSocket
3784+
3785+HandlerSocket is a NoSQL plugin for MySQL. It works as a daemon inside the
3786+mysqld process, accept tcp connections, and execute requests from clients.
3787+HandlerSocket does not support SQL queries. Instead, it supports simple CRUD
3788+operations on tables.
3789+
3790+Because of the following reasons, HandlerSocket is much faster than the
3791+mysqld/libmysql pair in some circumstances:
3792+
3793+ - HandlerSocket manipulates data without parsing SQL, which causes less
3794+ CPU usage.
3795+ - HandlerSocket reads many requests from clients and executes their
3796+ requests in bulk, which causes less CPU and disk usage.
3797+ - HandlerSocket client/server protocol is more compact than the
3798+ mysql/libmysql pair, which causes less network usage.
3799+
3800+The current version of HandlerSocket only works with GNU/Linux. The source
3801+archive of HandlerSocket includes a C++ and a Perl client libraries.
3802+Here is a list of other language bindings:
3803+
3804+ - PHP
3805+ http://openpear.org/package/Net_HandlerSocket
3806+ http://github.com/tz-lom/HSPHP
3807+ http://code.google.com/p/php-handlersocket/
3808+ - Java
3809+ http://code.google.com/p/handlersocketforjava/
3810+ - Python
3811+ https://code.launchpad.net/~songofacandy/+junk/pyhandlersocket
3812+ - Ruby
3813+ https://github.com/winebarrel/ruby-handlersocket
3814+ https://github.com/miyucy/handlersocket
3815+ - JavaScript(Node.js)
3816+ https://github.com/koichik/node-handlersocket
3817+
3818+The home of HandlerSocket is here:
3819+ https://github.com/ahiguti/HandlerSocket-Plugin-for-MySQL
3820+
3821+More documents are available in docs-en/ and docs-ja/ directories.
3822+
3823
3824=== added file 'HandlerSocket-Plugin-for-MySQL/docs-en/configuration-options.en.txt'
3825--- HandlerSocket-Plugin-for-MySQL/docs-en/configuration-options.en.txt 1970-01-01 00:00:00 +0000
3826+++ HandlerSocket-Plugin-for-MySQL/docs-en/configuration-options.en.txt 2012-02-07 16:47:34 +0000
3827@@ -0,0 +1,87 @@
3828+
3829+-----------------------------------------------------------------
3830+handlersocket_verbose (default = 10, min = 0, max = 10000)
3831+
3832+ Specify the logging verboseness.
3833+
3834+-----------------------------------------------------------------
3835+handlersocket_address (default = '')
3836+
3837+ Specify the address to bind. If empty, it binds to 0.0.0.0.
3838+
3839+-----------------------------------------------------------------
3840+handlersocket_port (default = '9998')
3841+
3842+ Specify the port to bind. This option is for the listener for
3843+ read requests. If empty, the listener is disabled.
3844+
3845+-----------------------------------------------------------------
3846+handlersocket_port_wr (default = '9999')
3847+
3848+ Specify the port to bind. This option is for the listener for
3849+ write requests. If empty, the listener is disabled.
3850+
3851+-----------------------------------------------------------------
3852+handlersocket_epoll (default = 1, min = 0, max = 1)
3853+
3854+ Specify whether handlersocket uses epoll for I/O multiplexing.
3855+
3856+-----------------------------------------------------------------
3857+handlersocket_threads (default = 16, min = 1, max = 3000)
3858+
3859+ Specify the number of handlersocket worker threads. This option
3860+ is for the listener for read requests. Recommended value is
3861+ (the number of CPU cores * 2).
3862+
3863+-----------------------------------------------------------------
3864+handlersocket_threads_wr (default = 1, min = 1, max = 3000)
3865+
3866+ Specify the number of handlersocket worker threads. This option
3867+ is for the listener for write requests. Recommended value is 1.
3868+
3869+-----------------------------------------------------------------
3870+handlersocket_timeout (default = 300, min = 30, max = 3600)
3871+
3872+ Specify the socket timeout in seconds.
3873+
3874+-----------------------------------------------------------------
3875+handlersocket_backlog (default = 32768, min = 5, max = 1000000)
3876+
3877+ Specify the length of the listen backlog.
3878+
3879+-----------------------------------------------------------------
3880+handlersocket_sndbuf (default = 0, min = 0, max = 1677216)
3881+
3882+ Specify the maximum socket send buffer in bytes. If 0, the
3883+ system-wide default value is set.
3884+
3885+-----------------------------------------------------------------
3886+handlersocket_rcvbuf (default = 0, min = 0, max = 1677216)
3887+
3888+ Specify the maximum socket receive buffer in bytes. If 0, the
3889+ system-wide default value is set.
3890+
3891+-----------------------------------------------------------------
3892+handlersocket_readsize (default = 0, min = 0, max = 1677216)
3893+
3894+ Specify the minimum length of the handlersocket request buffer.
3895+ Larger value can make handlersocket faster for large requests,
3896+ but can consume memory. The default value is possibly 4096.
3897+
3898+-----------------------------------------------------------------
3899+handlersocket_accept_balance (default = 0, min = 0, max = 10000)
3900+
3901+ When this option is set to non-zero, handlersocket tries to
3902+ balance accepted connections among threads. Non-zero is
3903+ recommended if you use persistent connections (i.e., connection
3904+ pooling on the client side).
3905+
3906+-----------------------------------------------------------------
3907+handlersocket_wrlock_timeout (default = 12, min = 0, max = 3600)
3908+
3909+ Specify the lock timeout in seconds. When a write request is
3910+ performed, handlersocket locks an advisory lock named
3911+ 'handlersocket_wr'. This option sets the timeout for the
3912+ locking.
3913+
3914+
3915
3916=== added file 'HandlerSocket-Plugin-for-MySQL/docs-en/installation.en.txt'
3917--- HandlerSocket-Plugin-for-MySQL/docs-en/installation.en.txt 1970-01-01 00:00:00 +0000
3918+++ HandlerSocket-Plugin-for-MySQL/docs-en/installation.en.txt 2012-02-07 16:47:34 +0000
3919@@ -0,0 +1,91 @@
3920+1. Building Handlersocket
3921+
3922+ Handlersocket mainly consists of libhsclient, handlersocket, and C++/Perl clients. libhsclient is a common library shared from both client and server(plugin). handlersocket is a MySQL daemon plugin.
3923+ To build Handlersocket, you need both MySQL source code and MySQL binary. It is not required to pre-build MySQL source code, but source itself is needed because Handlersocket depends on MySQL header files that only MySQL source distribution contains. MySQL binary is just a normal MySQL binary distribution. You can use official MySQL binaries provided by Oracle.
3924+ Since Handlersocket uses daemon plugin interface supported from MySQL 5.1,
3925+MySQL 5.1 or higher version is required.
3926+ Please make sure that you use identical MySQL version between MySQL source
3927+and MySQL binary. Otherwise you might encounter serious problems (i.e. server
3928+crash, etc).
3929+ Here are steps to build Handlersocket.
3930+
3931+* Get MySQL source code
3932+
3933+* Get MySQL binary
3934+
3935+* Build Handlersocket
3936+ $ ./autogen.sh
3937+ $ ./configure --with-mysql-source=/work/mysql-5.1.50 --with-mysql-bindir=/work/mysql-5.1.50-linux-x86_64-glibc23/bin --with-mysql-plugindir=/work/mysql-5.1.50-linux-x86_64-glibc23/lib/plugin
3938+
3939+ --with-mysql-source refers to the top of MySQL source directory,
3940+--with-mysql-bindir refers to where MySQL binary executables (i.e.
3941+mysql_config) are located, and --with-mysql-plugindir refers to a plugin
3942+directory where plugin libraries (*.so) are installed.
3943+
3944+ $ make
3945+ $ sudo make install
3946+
3947+ Both libhsclient and the handlersocket plugin will be installed.
3948+
3949+
3950+2. Using Handlersocket
3951+
3952+Append configuration options for handlersocket to my.cnf.
3953+
3954+ [mysqld]
3955+ loose_handlersocket_port = 9998
3956+ # the port number to bind to (for read requests)
3957+ loose_handlersocket_port_wr = 9999
3958+ # the port number to bind to (for write requests)
3959+ loose_handlersocket_threads = 16
3960+ # the number of worker threads (for read requests)
3961+ loose_handlersocket_threads_wr = 1
3962+ # the number of worker threads (for write requests)
3963+ open_files_limit = 65535
3964+ # to allow handlersocket accept many concurrent
3965+ # connections, make open_files_limit as large as
3966+ # possible.
3967+
3968+Log in to mysql as root, and execute the following query.
3969+
3970+ mysql> install plugin handlersocket soname 'handlersocket.so';
3971+
3972+If handlersocket.so is successfully installed, it starts
3973+accepting connections on port 9998 and 9999. Running
3974+'show processlist' should show handlersocket worker threads.
3975+
3976+-----------------------------------------------------------------
3977+On the client side, you need to install libhsclient for c++ apps
3978+and perl-Net-HandlerSocket for perl apps. They do not require
3979+MySQL to compile.
3980+
3981+ $ ./autogen.sh
3982+ $ ./configure --disable-handlersocket-server
3983+ $ make
3984+ $ sudo make install
3985+ $ cd perl-Net-HandlerSocket
3986+ $ perl Makefile.PL
3987+ $ make
3988+ $ sudo make install
3989+
3990+-----------------------------------------------------------------
3991+Alternatively, you can use the rpm installation. If your OS
3992+supports rpms, you can use the following commands to build and
3993+install handlersocket rpm packages.
3994+
3995+(Server side, installs HandlerSocket plugin)
3996+ $ ./autogen.sh
3997+ $ ./configure --with-mysql-source=/work/mysql-5.1.50 --with-mysql-bindir=/work/mysql-5.1.50-linux-x86_64-glibc23/bin --with-mysql-plugindir=/work/mysql-5.1.50-linux-x86_64-glibc23/lib/plugin
3998+ $ make rpm_cli
3999+ $ sudo rpm -U dist/RPMS/*/libhsclient*.rpm
4000+ $ make rpm_c
4001+ $ sudo rpm -U dist/RPMS/*/handlersocket*.rpm
4002+
4003+(Client side, installs client libraries)
4004+ $ ./autogen.sh
4005+ $ ./configure --disable-handlersocket-server
4006+ $ make rpm_cli
4007+ $ sudo rpm -U dist/RPMS/*/libhsclient*.rpm
4008+ $ make rpm_perl
4009+ $ sudo rpm -U dist/RPMS/*/perl-Net-HandlerSocket*.rpm
4010+
4011
4012=== added file 'HandlerSocket-Plugin-for-MySQL/docs-en/perl-client.en.txt'
4013--- HandlerSocket-Plugin-for-MySQL/docs-en/perl-client.en.txt 1970-01-01 00:00:00 +0000
4014+++ HandlerSocket-Plugin-for-MySQL/docs-en/perl-client.en.txt 2012-02-07 16:47:34 +0000
4015@@ -0,0 +1,126 @@
4016+
4017+-----------------------------------------------------------------
4018+To open a connection to the handlersocket plugin, you need to
4019+create a Net::HandlerSocket object.
4020+
4021+ use Net::HandlerSocket;
4022+ my $args = { host => 'localhost', port => 9998 };
4023+ my $hs = new Net::HandlerSocket($args);
4024+
4025+-----------------------------------------------------------------
4026+Before executing table operations, you need to open an index to
4027+work with.
4028+
4029+ my $err = $hs->open_index(3, 'database1', 'table1', 'PRIMARY',
4030+ 'f1,f2');
4031+ die $hs->get_error() if $res->[0] != 0;
4032+
4033+The first argument for open_index is an integer value which is
4034+used to identify an open table, which is only valid within the
4035+same Net::HandlerSocket object. The 4th argument is the name of
4036+index to open. If 'PRIMARY' is specified, the primary index is
4037+open. The 5th argument is a comma-separated list of column names.
4038+
4039+-----------------------------------------------------------------
4040+To read a record from a table using an index, call the
4041+execute_single method.
4042+
4043+ my $res = $hs->execute_single(3, '=', [ 'foo' ], 1, 0);
4044+ die $hs->get_error() if $res->[0] != 0;
4045+ shift(@$res);
4046+
4047+The first argument must be an integer which has specified as the
4048+first argument for open_index on the same Net::HandlerSocket
4049+object. The second argument specifies the search operation. The
4050+current version of handlersocket supports '=', '>=', '<=', '>',
4051+and '<'. The 3rd argument specifies the key to find, which must
4052+an arrayref whose length is equal to or smaller than the number
4053+of key columns of the index. The 4th and the 5th arguments
4054+specify the maximum number of records to be retrieved, and the
4055+number of records skipped before retrieving records. The columns
4056+to be retrieved are specified by the 5th argument for the
4057+corresponding open_index call.
4058+
4059+The execute_single method always returns an arrayref. The first
4060+element is the error code, which is 0 when no error is occured.
4061+The remaining are the field values. If more than one record is
4062+returned, it is flatten to an 1-dimensional array. For example,
4063+when 5 records that have 3 columns are returned, you can retrieve
4064+values using the following code.
4065+
4066+ die $hs->get_error() if $res->[0] != 0;
4067+ shift(@$res);
4068+ for (my $row = 0; $row < 5; ++$row) {
4069+ for (my $col = 0; $col < 3; ++$col) {
4070+ my $value = $res->[$row * 5 + $col];
4071+ # ...
4072+ }
4073+ }
4074+
4075+-----------------------------------------------------------------
4076+To update or delete records, you need to specify more arguments
4077+for the execute_single method. Note that the Net::HandlerSocket
4078+object must be connected to a handlersocket worker for write
4079+operations, which is port 9999 by default.
4080+(For safety, the port 9998 only allows read operations, and the
4081+port 9999 allows write operations also. The port 9999 allows
4082+read operations too, but slower than 9998 because of record
4083+locking etc.. Port numbers can be changed using the
4084+'handlersocket_port' and the 'handlersocket_port_wr'
4085+configuration options of mysqld.)
4086+
4087+ my $args = { host => 'localhost', port => 9999 };
4088+ my $hs = new Net::HandlerSocket($args);
4089+
4090+ my $res = $hs->execute_single(3, '=', [ 'bar' ], 1, 0, 'U',
4091+ [ 'fubar', 'hoge' ]);
4092+ die $hs->get_error() if $res->[0] != 0;
4093+ my $num_updated_rows = $res->[1];
4094+
4095+ my $res = $hs->execute_single(3, '=', [ 'baz' ], 1, 0, 'D');
4096+ die $hs->get_error() if $res->[0] != 0;
4097+ my $num_deleted_rows = $res->[1];
4098+
4099+The 6th argument for execute_single specifies the modification
4100+operation. The current version supports 'U' and 'D'. For the 'U'
4101+operation, the 7th argument specifies the new value for the row.
4102+The columns to be modified are specified by the 5th argument for
4103+the corresponding open_index call. For the 'D' operation, the
4104+7th argument can be omitted.
4105+
4106+-----------------------------------------------------------------
4107+The execute_single method can be used for inserting records also.
4108+
4109+ my $res = $hs->execute_single(3, '+', [ 'foo', 'bar', 'baz' ]);
4110+ die $hs->get_error() if $res->[0] != 0;
4111+ my $num_inserted_rows = $res->[1];
4112+
4113+The 3rd argument must be an arrayref whose elements correspond to
4114+the 5th argument for the corresponding open_index call. If there
4115+is a column which is not appeared in the 5th argument for the
4116+open_index, the default value for the column is set.
4117+
4118+-----------------------------------------------------------------
4119+Multiple operations can be executed in a single call. Executing
4120+multiple operations in a single call is much faster than
4121+executing them separatedly.
4122+
4123+ my $rarr = $hs->execute_multi([
4124+ [ 0, '>=', [ 'foo' ], 5, 0 ],
4125+ [ 2, '=', [ 'bar' ], 1, 0 ],
4126+ [ 4, '<', [ 'baz' ], 10, 5 ],
4127+ ]);
4128+ for my $res (@$rarr) {
4129+ die $hs->get_error() if $res->[0] != 0;
4130+ shift(@$res);
4131+ # ...
4132+ }
4133+
4134+-----------------------------------------------------------------
4135+When an error is occured, the first element of the returned
4136+arrayref becomes a non-zero value. A negative value indicates
4137+that an I/O error is occured and the Net::HandlerSocket object
4138+should be disposed. A positive value means that the connection is
4139+still active and the Net::HandlerSocket object can be reused
4140+later.
4141+
4142
4143=== added file 'HandlerSocket-Plugin-for-MySQL/docs-en/protocol.en.txt'
4144--- HandlerSocket-Plugin-for-MySQL/docs-en/protocol.en.txt 1970-01-01 00:00:00 +0000
4145+++ HandlerSocket-Plugin-for-MySQL/docs-en/protocol.en.txt 2012-02-07 16:47:34 +0000
4146@@ -0,0 +1,148 @@
4147+
4148+----------------------------------------------------------------------------
4149+The HandlerSocket protocol
4150+
4151+----------------------------------------------------------------------------
4152+Basic syntax
4153+
4154+- The HandlerSocket protocol is line-based. Each line ends with LF(0x0a).
4155+- Each line consists a concatenation of tokens separated by HT(0x09).
4156+- A token is either NULL or an encoded string. Note that you need to
4157+ distinguish NULL from an empty string, as most DBMs does so.
4158+- NULL is expressed as a single NUL(0x00).
4159+- An encoded string is a string with the following encoding rules.
4160+ - Characters in the range [0x10 - 0xff] are not encoded.
4161+ - A character in the range [0x00 - 0x0f] is prefixed by 0x01 and
4162+ shifted by 0x40. For example, 0x03 is encoded as 0x01 0x43.
4163+- Note that a string can be empty. A continuation of 0x09 0x09 means that
4164+ there is an empty string between them. A continuation of 0x09 0x0a means
4165+ that there is an empty string at the end of the line.
4166+
4167+----------------------------------------------------------------------------
4168+Request and Response
4169+
4170+- The HandlerSocket protocol is a simple request/response protocol. After a
4171+ connection is established, the client side sends a request, and then the
4172+ server side sends a response.
4173+- A request/response consists of a single line.
4174+- Requests can be pipelined; That is, you can send multiple requests (ie.
4175+ lines) at one time, and receive responses for them at one time.
4176+
4177+----------------------------------------------------------------------------
4178+'open_index' request
4179+
4180+The 'open_index' request has the following syntax.
4181+
4182+ P <indexid> <dbname> <tablename> <indexname> <columns>
4183+
4184+- <indexid> is a number in decimal.
4185+- <dbname>, <tablename>, and <indexname> are strings. To open the primary
4186+ key, use PRIMARY as <indexname>.
4187+- <columns> is a comma-separated list of column names.
4188+
4189+Once an 'open_index' request is issued, the HandlerSocket plugin opens the
4190+specified index and keep it open until the client connection is closed. Each
4191+open index is identified by <indexid>. If <indexid> is already open, the old
4192+open index is closed. You can open the same combination of <dbname>
4193+<tablename> <indexname> multple times, possibly with different <columns>.
4194+For efficiency, keep <indexid> small as far as possible.
4195+
4196+----------------------------------------------------------------------------
4197+Getting data
4198+
4199+The 'find' request has the following syntax.
4200+
4201+ <indexid> <op> <vlen> <v1> ... <vn> <limit> <offset>
4202+
4203+- <indexid> is a number. This number must be an <indexid> specified by a
4204+ 'open_index' request executed previously on the same connection.
4205+- <op> specifies the comparison operation to use. The current version of
4206+ HandlerSocket supports '=', '>', '>=', '<', and '<='.
4207+- <vlen> indicates the length of the trailing parameters <v1> ... <vn>. This
4208+ must be smaller than or equal to the number of index columns specified by
4209+ specified by the corresponding 'open_index' request.
4210+- <v1> ... <vn> specify the index column values to fetch.
4211+- <limit> and <offset> are numbers. These parameters can be omitted. When
4212+ omitted, it works as if 1 and 0 are specified.
4213+
4214+----------------------------------------------------------------------------
4215+Updating/Deleting data
4216+
4217+The 'find_modify' request has the following syntax.
4218+
4219+ <indexid> <op> <vlen> <v1> ... <vn> <limit> <offset> <mop> <m1> ... <mk>
4220+
4221+- <mop> is either 'U' (update) or 'D' (delete).
4222+- <m1> ... <mk> specifies the column values to set. The length of <m1> ...
4223+ <mk> must be smaller than or equal to the length of <columns> specified by
4224+ the corresponding 'open_index' request. If <mop> is 'D', these parameters
4225+ are ignored.
4226+
4227+----------------------------------------------------------------------------
4228+Inserting data
4229+
4230+The 'insert' request has the following syntax.
4231+
4232+ <indexid> '+' <vlen> <v1> ... <vn>
4233+
4234+- <vlen> indicates the length of the trailing parameters <v1> ... <vn>. This
4235+ must be smaller than or equal to the length of <columns> specified by the
4236+ corresponding 'open_index' request.
4237+- <v1> ... <vn> specify the column values to set. For columns not in
4238+ <columns>, the default values for each column are set.
4239+
4240+----------------------------------------------------------------------------
4241+Response syntax
4242+
4243+HandlerSocket returns a response of the following syntax for each request.
4244+
4245+ <errorcode> <numcolumns> <r1> ... <rn>
4246+
4247+- <errorcode> indicates whether the request has successfully executed or not.
4248+ '0' means success. Non-zero means an error.
4249+- <numcolumns> indicates the number of columns of the result set.
4250+- <r1> ... <rn> is the result set. The length of <r1> ... <rn> is always a
4251+ multiple of <numcolumns>. It is possible that <r1> ... <rn> is empty.
4252+
4253+If <errorcode> is non-zero, <numcolumns> is always 1 and <r1> indicates a
4254+human-readable error message, though sometimes <r1> is not provided.
4255+
4256+----------------------------------------------------------------------------
4257+Response for 'open_index'
4258+
4259+If 'open_index' is succeeded, HandlerSocket returns a line of the following
4260+syntax.
4261+
4262+ 0 1
4263+
4264+----------------------------------------------------------------------------
4265+Response for 'find'
4266+
4267+If 'find' is succeeded, HandlerSocket returns a line of the following
4268+syntax.
4269+
4270+ 0 <numcolumns> <r1> ... <rn>
4271+
4272+- <numcolumns> always equals to the length of <columns> of the corresponding
4273+ 'open_index' request.
4274+- <r1> ... <rn> is the result set. If N rows are found, the length of <r1>
4275+ ... <rn> becomes ( <numcolumns> * N ).
4276+
4277+----------------------------------------------------------------------------
4278+Response for 'find_modify'
4279+
4280+If 'find_modify' is succeeded, HandlerSocket returns a line of the following
4281+syntax.
4282+
4283+ 0 1 <nummod>
4284+
4285+- <nummod> is the number of modified rows.
4286+
4287+----------------------------------------------------------------------------
4288+Response for 'insert'
4289+
4290+If 'insert' is succeeded, HanderSocket returns a line of the following
4291+syntax.
4292+
4293+ 0 1
4294+
4295
4296=== added directory 'HandlerSocket-Plugin-for-MySQL/docs-ja'
4297=== added file 'HandlerSocket-Plugin-for-MySQL/docs-ja/about-handlersocket.ja.txt'
4298--- HandlerSocket-Plugin-for-MySQL/docs-ja/about-handlersocket.ja.txt 1970-01-01 00:00:00 +0000
4299+++ HandlerSocket-Plugin-for-MySQL/docs-ja/about-handlersocket.ja.txt 2012-02-07 16:47:34 +0000
4300@@ -0,0 +1,51 @@
4301+
4302+
4303+-----------------------------------------------------------------
4304+ソースコードの利用にあたっての免責事項
4305+
4306+本ソフトウェアの開発者および株式会社ディー・エヌ・エーは、本フト
4307+ウェアの不稼動、稼動不良を含む法律上の瑕疵担保責任、その他保証責
4308+任を負わないものとします。また、本ソフトウエアの開発者および株式
4309+会社ディー・エヌ・エーは、本ソフトウェアの商品性、またはお客様の
4310+特定の目的に対する適合性について、いかなる保証も負わないものとし
4311+ます。
4312+
4313+-----------------------------------------------------------------
4314+handlersocket pluginについて
4315+
4316+mysqlサーバに常駐し、innodb等のストレージエンジンへの直接のアクセ
4317+スを提供するプラグインです。handlersocketプラグインは自前のリス
4318+ナーを持ち、専用のクライアントライブラリ(libhsclient)を使ってそれ
4319+にアクセスします。
4320+
4321+mysqlの標準クライアントライブラリ(libmysql)を使ったアクセスと比べ
4322+て、以下のような利点があります。
4323+・接続あたりに消費するリソースが少ないため、同時接続数が事実上無
4324+ 制限。したがって接続数を気にせず持続接続を使えます。
4325+・高速(単純な参照クエリで3倍〜10倍程度)。
4326+・通信プロトコルがコンパクト。libmysqlを使うとデータ転送時にレ
4327+ コード名などが付随するために通信内容が冗長ですが、libhsclientで
4328+ はデータのみが転送されるため、帯域消費が少なくなります。場合に
4329+ よっては10倍以上libmysqlのほうが冗長になります。
4330+
4331+現在のバージョンでは以下のような処理をサポートしています。
4332+・指定された索引について、指定された値と完全一致するようなレコー
4333+ ドを取得。(SELECT ??? FROM tbl WHERE k1 = v1 AND k2 = v2...)。
4334+ 索引を使わない検索はサポートしていません。
4335+・指定された索引について、指定された値の位置の前後のレコードを取
4336+ 得。(SELECT ??? FROM tbl WHERE k1 >= v1 LIMIT 100)
4337+・前述のような手段で取得したレコードに対するUPDATEとDELETE
4338+・レコードのINSERT
4339+
4340+以下のような言語をサポートします。
4341+・C++。libhsclientをリンクします。
4342+・Perl。Net::HandlerSocketをuseします。
4343+
4344+現在のバージョンではGNU/Linuxでのみ動作します。
4345+
4346+-----------------------------------------------------------------
4347+既知の問題
4348+
4349+・killでhandlersocketスレッドを殺すと、スレッド数が減ったまま回復
4350+ しません。
4351+
4352
4353=== added file 'HandlerSocket-Plugin-for-MySQL/docs-ja/installation.ja.txt'
4354--- HandlerSocket-Plugin-for-MySQL/docs-ja/installation.ja.txt 1970-01-01 00:00:00 +0000
4355+++ HandlerSocket-Plugin-for-MySQL/docs-ja/installation.ja.txt 2012-02-07 16:47:34 +0000
4356@@ -0,0 +1,87 @@
4357+
4358+-----------------------------------------------------------------
4359+HandlerSocketプラグインのビルド方法(RPMを使わない方法)
4360+
4361+以下のようにしてconfigureを実行します。
4362+
4363+ $ ./autogen.sh
4364+ $ ./configure --with-mysql-source=/work/mysql-5.1.50 --with-mysql-bindir=/work/mysql-5.1.50-linux-x86_64-glibc23/bin --with-mysql-plugindir=/work/mysql-5.1.50-linux-x86_64-glibc23/lib/plugin
4365+
4366+ここで--with-mysql-sourceにはMySQLのソースコードのトップディレク
4367+トリを指定します。--with-mysql-bindirにはインストール済みのMySQL
4368+のmysql_configコマンドが有るディレクトリを指定します。
4369+その後以下のようにビルド・インストールします。
4370+
4371+ $ make
4372+ $ sudo make install
4373+
4374+-----------------------------------------------------------------
4375+クライアントライブラリのビルド方法(RPMを使わない方法)
4376+
4377+クライアントライブラリをビルドする際には、MySQLのソースコードは
4378+必要ありません。またMySQLがインストールされている必要もありません。
4379+
4380+ $ ./autogen.sh
4381+ $ ./configure --disable-handlersocket-server
4382+ $ make
4383+ $ sudo make install
4384+ $ cd perl-Net-HandlerSocket
4385+ $ perl Makefile.PL
4386+ $ make
4387+ $ sudo make install
4388+
4389+-----------------------------------------------------------------
4390+ビルド方法(RPM)
4391+
4392+以下のように実行すれば、rpmパッケージがビルド&インストールされま
4393+す。
4394+
4395+(MySQLサーバ側、HandlerSocketプラグインをインストールする)
4396+ $ ./autogen.sh
4397+ $ ./configure --with-mysql-source=/work/mysql-5.1.50 --with-mysql-bindir=/work/mysql-5.1.50-linux-x86_64-glibc23/bin --with-mysql-plugindir=/work/mysql-5.1.50-linux-x86_64-glibc23/lib/plugin
4398+ $ make rpm_cli
4399+ $ sudo rpm -U dist/RPMS/*/libhsclient*.rpm
4400+ $ make rpm_c
4401+ $ sudo rpm -U dist/RPMS/*/handlersocket*.rpm
4402+
4403+(クライアント側、クライアントライブラリをインストールする)
4404+ $ ./autogen.sh
4405+ $ ./configure --disable-handlersocket-server
4406+ $ make rpm_cli
4407+ $ sudo rpm -U dist/RPMS/*/libhsclient*.rpm
4408+ $ make rpm_perl
4409+ $ sudo rpm -U dist/RPMS/*/perl-Net-HandlerSocket*.rpm
4410+
4411+-----------------------------------------------------------------
4412+起動
4413+
4414+mysqlを起動した状態で、mysqlの設定ファイル(my.cnf等)に以下の内容を
4415+追加します。
4416+
4417+ [mysqld]
4418+ handlersocket_port = 9998
4419+ # handlersocketが接続を受け付けるポート(参照系リクエスト用)
4420+ handlersocket_port_wr = 9999
4421+ # handlersocketが接続を受け付けるポート(更新系リクエスト用)
4422+ handlersocket_address =
4423+ # handlersocketがバインドするアドレス(空のままでOK)
4424+ handlersocket_verbose = 0
4425+ # デバッグ用
4426+ handlersocket_timeout = 300
4427+ # 通信タイムアウト(秒)
4428+ handlersocket_threads = 16
4429+ # handlersocketのワーカースレッド数
4430+ thread_concurrency = 128
4431+ # handlersocketが幾つかのスレッドを占有するため、大きめの
4432+ # 値を指定してください
4433+ open_files_limit = 65535
4434+ # ソケットを大量に開けるようにするため、大きめの値を指定し
4435+ # てください
4436+
4437+以下のクエリを実行します。
4438+
4439+ mysql> install plugin handlersocket soname 'handlersocket.so';
4440+ Query OK, 0 rows affected (0.06 sec)
4441+
4442+以上でhandlersocketへクライアントからアクセスできるようになります。
4443+
4444
4445=== added file 'HandlerSocket-Plugin-for-MySQL/docs-ja/perl-client.ja.txt'
4446--- HandlerSocket-Plugin-for-MySQL/docs-ja/perl-client.ja.txt 1970-01-01 00:00:00 +0000
4447+++ HandlerSocket-Plugin-for-MySQL/docs-ja/perl-client.ja.txt 2012-02-07 16:47:34 +0000
4448@@ -0,0 +1,118 @@
4449+
4450+-----------------------------------------------------------------
4451+handlersocketプラグインへの接続を開くには、Net::HandlerSocketオブ
4452+ジェクトを作成します。
4453+
4454+ use Net::HandlerSocket;
4455+ my $args = { host => 'localhost', port => 9998 };
4456+ my $hs = new Net::HandlerSocket($args);
4457+
4458+-----------------------------------------------------------------
4459+検索などの命令を実行する前に、処理対象となる索引を開く必要があり
4460+ます。
4461+
4462+ my $err = $hs->open_index(3, 'database1', 'table1', 'PRIMARY',
4463+ 'f1,f2');
4464+ die $hs->get_error() if $res->[0] != 0;
4465+
4466+最初の引数は開く索引に付ける番号です。付けた番号は同一の
4467+Net::HandlerSocketオブジェクトについてのみ有効です。第4引数は開く
4468+索引の名前で、「PRIMARY」が指定され場合はプライマリキーが開かれま
4469+す。第5引数は「,」で区切られた列名のリストです。
4470+
4471+-----------------------------------------------------------------
4472+テーブルから索引を使って行を取得するには、execute_singleメソッド
4473+を呼びます。
4474+
4475+ my $res = $hs->execute_single(3, '=', [ 'foo' ], 1, 0);
4476+ die $hs->get_error() if $res->[0] != 0;
4477+ shift(@$res);
4478+
4479+最初の引数は索引の番号で、同じNet::HandlerSocketオブジェクトへ
4480+open_indexで付けたものでなければなりません。第2引数には操作を指定
4481+します。現在のバージョンでは、「=」、「>=」、「<=」、「>」、「<」
4482+の操作が利用可能です。第3引数は配列への参照で、これは探すべき行の
4483+キー値を指定します。配列の長さは索引のキーの個数と同じか少ない数
4484+でなければなりません。第4引数と第5引数はそれぞれ、取得する最大行
4485+数、取得前に読み飛ばす行数を指定します。取得される列は対応する
4486+open_index呼び出しの第5引数で指定されたものになります。
4487+
4488+execute_singleメソッドは常に配列への参照を返します。最初の要素は
4489+エラーコードで、これが0ならば成功を表します。残りの要素は列の値で
4490+す。もし取得されたデータが複数行の場合は、それが一つの配列へ連結
4491+された形で格納されています。例えば、5行3列のデータの場合、次のよ
4492+うなコードによってその内容を取得できます。
4493+
4494+ die $hs->get_error() if $res->[0] != 0;
4495+ shift(@$res);
4496+ for (my $row = 0; $row < 5; ++$row) {
4497+ for (my $col = 0; $col < 3; ++$col) {
4498+ my $value = $res->[$row * 5 + $col];
4499+ # ...
4500+ }
4501+ }
4502+
4503+-----------------------------------------------------------------
4504+行を更新または削除するには、更に多くの引数を指定して
4505+execute_singleメソッドを呼び出します。書き込み処理をするには、
4506+対象のNet::HandlerSocketオブジェクトは更新用handlersocketワーカ(既
4507+定ではポート9999)へ接続されたものでなくてはなりません。
4508+(安全のため、ポート9998は参照処理だけを受け付け、ポート9999は更新
4509+処理も受け付けるようになっています。ポート9999は参照処理も受け付
4510+けますが、レコードロック等の影響で遅くなります。ポート番号は
4511+mysqldの「handlersocket_port」と「handlersocket_port_wr」の設定項
4512+目で変更できます。)
4513+
4514+ my $args = { host => 'localhost', port => 9999 };
4515+ my $hs = new Net::HandlerSocket($args);
4516+
4517+ my $res = $hs->execute_single(3, '=', [ 'bar' ], 1, 0, 'U',
4518+ [ 'fubar', 'hoge' ]);
4519+ die $hs->get_error() if $res->[0] != 0;
4520+ my $num_updated_rows = $res->[1];
4521+
4522+ my $res = $hs->execute_single(3, '=', [ 'baz' ], 1, 0, 'D');
4523+ die $hs->get_error() if $res->[0] != 0;
4524+ my $num_deleted_rows = $res->[1];
4525+
4526+execute_singleの第6引数は変更処理の種類を指定します。現在のバー
4527+ジョンでは「U」と「D」が利用可能です。「U」については、第7引数で
4528+新しい値を指定します。更新される列は、対応するopen_index呼び出し
4529+の第5引数で指定されたものになります。「D」については第7引数は省
4530+略できます。
4531+
4532+-----------------------------------------------------------------
4533+execute_singleメソッドは列の挿入にも使用できます。
4534+
4535+ my $res = $hs->execute_single(3, '+', [ 'foo', 'bar', 'baz' ]);
4536+ die $hs->get_error() if $res->[0] != 0;
4537+ my $num_inserted_rows = $res->[1];
4538+
4539+第3引数は、対応するopen_index呼び出しの第5引数の列リストと同じだ
4540+けの長さの配列への参照でなければなりません。open_index呼び出しの
4541+第5引数に指定されていない列については、その列の既定値がセットされ
4542+ます。
4543+
4544+-----------------------------------------------------------------
4545+execute_multiメソッドを使えば、複数のリクエストを一つの呼び出しで
4546+実行することができます。これはリクエストを個別に実行するより高速
4547+です。
4548+
4549+ my $rarr = $hs->execute_multi([
4550+ [ 0, '>=', [ 'foo' ], 5, 0 ],
4551+ [ 2, '=', [ 'bar' ], 1, 0 ],
4552+ [ 4, '<', [ 'baz' ], 10, 5 ],
4553+ ]);
4554+ for my $res (@$rarr) {
4555+ die $hs->get_error() if $res->[0] != 0;
4556+ shift(@$res);
4557+ # ...
4558+ }
4559+
4560+-----------------------------------------------------------------
4561+エラーが起こると返値の配列参照の最初の要素が0以外になります。負の
4562+数の場合はI/Oエラーが起こったことを示し、その場合はその
4563+Net::HandlerSocketオブジェクトは破棄するべきです。正の値の場合は
4564+接続は維持されているため、そのオブジェクトはそれ以後も再利用でき
4565+ます。
4566+
4567
4568=== added file 'HandlerSocket-Plugin-for-MySQL/docs-ja/protocol.ja.txt'
4569--- HandlerSocket-Plugin-for-MySQL/docs-ja/protocol.ja.txt 1970-01-01 00:00:00 +0000
4570+++ HandlerSocket-Plugin-for-MySQL/docs-ja/protocol.ja.txt 2012-02-07 16:47:34 +0000
4571@@ -0,0 +1,94 @@
4572+
4573+-----------------------------------------------------------------
4574+handlersocketの通信プロトコル
4575+
4576+-----------------------------------------------------------------
4577+構文
4578+
4579+・コマンド行は改行(LF)で終わる。
4580+・コマンド行は複数のトークンからなり、トークン間はTABで区切られる。
4581+・トークンはNULLトークンか、文字列トークンのいずれか。
4582+・NULLトークンは単一のNUL文字であらわされる。
4583+・文字列トークンは、0バイト以上の文字列であらわされる。ただし0x10
4584+ 未満の文字については0x01を前置し、0x40を加えたコードであらわさ
4585+ れる。それ以外の文字はその文字自身のコードであらわされる。
4586+
4587+-----------------------------------------------------------------
4588+リクエストとレスポンス
4589+
4590+・接続が確立した直後の状態では、まずクライアントがコマンド行を送
4591+ る。(リクエスト)
4592+・サーバはクライアントが送ったリクエストと丁度同じ数のコマンド行
4593+ を返す。(レスポンス)
4594+・リクエストはパイプライン化してよい。つまりクライアントは前に
4595+ 送ったリクエストに対する返事を待たずに次のリクエストを送っても
4596+ よい。
4597+
4598+-----------------------------------------------------------------
4599+リクエスト
4600+
4601+・open_index命令は次のような構文を持つ。
4602+ 'P' indexid dbname tablename indexname fieldlist
4603+ indexidは開いている索引に付けられる番号で、同一接続上で後に実行
4604+ する命令の、対象索引を指定するために使われる。dbname、tablename、
4605+ indexnameはそれぞれ開きたいDB、テーブル、索引の名前。索引の名前
4606+ として"PRIMARY"を指定するとプライマリキーが開かれる。fieldlist
4607+ はカンマ区切りの列名のリスト。
4608+・find命令は次のような構文を持つ。
4609+ indexid op nflds v1 ... vn limit offset
4610+ indexidは実行対象の索引を指定する。opは索引検索の演算子(後述)。
4611+ v1からvnは可変長で、その個数はnflds。nfldsはindexidで指定された
4612+ open_index命令のindexnameの索引のfieldlistのフィールド数に等し
4613+ いか小さくなくてはならない。m2からmkは可変長で、その個数は
4614+ indexidで指定されたopen_index命令が発行された際のfieldlistに一
4615+ 致しなければならない。コマンド行のlimit以降は省略できる。limit
4616+ とoffsetは、検索条件に合致する列のうちレスポンスに返す列数の上
4617+ 限と、スキップする列数。limitとoffsetを省略した場合はそれぞれ1
4618+ と0が指定されたときと同じ動作をする。find命令はレスポンスとして、
4619+ 条件に合致した列のリストを返す。opとして指定できる演算子は次の
4620+ とおり。
4621+ '=' - v1 ... vnと一致するものを取得
4622+ '>' - v1 ... vnより大きいものを昇順に取得
4623+ '>=' - v1 ... vnに一致するか大きいものを昇順に取得
4624+ '<' - v1 ... vnより小さいものを降順に取得
4625+ '<=' - v1 ... vnに一致するか等しいものを降順に取得
4626+ nfldsが1より大きい(v1 ... vnが2個以上)ときは辞書式順序で比較さ
4627+ れる。
4628+・find_modify命令は次のような構文を持つ。
4629+ indexid op nflds v1 ... vn limit offset modop m1 ... mk
4630+ modopより前の部分はfind命令と同等で、これによって操作対象の行を
4631+ 指定する。その操作対象の行に対しmodopで指定された変更処理を実行
4632+ する。m1 ... mkは可変長で、省略できる。modopは次いずれか。
4633+ 'U' - indexidで指定されたopen_index命令のfieldlist列
4634+ の内容を、m1 ... mkの値で更新する。
4635+ 'D' - 対象の行を削除する。m1 ... mkの値は無視される。
4636+・insert命令はのような構文を持つ。
4637+ indexid '+' nflds v1 ... vn
4638+ indexidで指定されたテーブルに、列を挿入する。v1 ... vnは可変長
4639+ で、その個数はnflds。nfldsはindexidで指定されたopen_index命令の
4640+ indexnameの索引のfieldlistのフィールド数に等しいか小さくなくて
4641+ はならない。
4642+
4643+-----------------------------------------------------------------
4644+レスポンス
4645+
4646+・open_index命令が成功したとき、レスポンスは次の構文を持つ。
4647+ '0' '1'
4648+・find命令が成功したとき、レスポンスは次の構文を持つ。
4649+ '0' nflds v1 ... vn
4650+ nfldsは結果セットの列の数をあらわす。v1 ... vnは可変長で、その
4651+ 長さはnfldsの整数倍。v1 ... vnは空のこともあり、それは条件に合
4652+ 致するレコードが存在しなかったことをあらわす。結果セットが複数
4653+ 行になったときはv1 ... vnの長さがnfldsの2倍以上となり、最初の
4654+ 行から順にv1 ... vnにセットされる。
4655+・modify命令が成功したとき、レスポンスは次の構文を持つ。
4656+ '0' '1' nummod
4657+ nummodは変更が施された行数。nummodが0のときは変更された行が無
4658+ かったことをあらわす。
4659+・insert命令が成功したとき、レスポンスは次の構文を持つ。
4660+ '0' '1'
4661+・命令が失敗したとき、レスポンスは命令に関わらず次の構文を持つ。
4662+ err '1' message
4663+ errは0以外の数値で、エラーコードをあらわす。messageは人間可読な
4664+ エラーメッセージ。ただしmessageが無いこともある。
4665+
4666
4667=== added directory 'HandlerSocket-Plugin-for-MySQL/handlersocket'
4668=== added file 'HandlerSocket-Plugin-for-MySQL/handlersocket/COPYRIGHT.txt'
4669--- HandlerSocket-Plugin-for-MySQL/handlersocket/COPYRIGHT.txt 1970-01-01 00:00:00 +0000
4670+++ HandlerSocket-Plugin-for-MySQL/handlersocket/COPYRIGHT.txt 2012-02-07 16:47:34 +0000
4671@@ -0,0 +1,27 @@
4672+
4673+ Copyright (c) 2010 DeNA Co.,Ltd.
4674+ All rights reserved.
4675+
4676+ Redistribution and use in source and binary forms, with or without
4677+ modification, are permitted provided that the following conditions are met:
4678+
4679+ * Redistributions of source code must retain the above copyright
4680+ notice, this list of conditions and the following disclaimer.
4681+ * Redistributions in binary form must reproduce the above copyright
4682+ notice, this list of conditions and the following disclaimer in the
4683+ documentation and/or other materials provided with the distribution.
4684+ * Neither the name of DeNA Co.,Ltd. nor the names of its contributors
4685+ may be used to endorse or promote products derived from this software
4686+ without specific prior written permission.
4687+
4688+ THIS SOFTWARE IS PROVIDED BY DeNA Co.,Ltd. "AS IS" AND ANY EXPRESS OR
4689+ IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
4690+ MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO
4691+ EVENT SHALL DeNA Co.,Ltd. BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
4692+ SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
4693+ PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS;
4694+ OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
4695+ WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
4696+ OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
4697+ ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
4698+
4699
4700=== added file 'HandlerSocket-Plugin-for-MySQL/handlersocket/Makefile.am'
4701--- HandlerSocket-Plugin-for-MySQL/handlersocket/Makefile.am 1970-01-01 00:00:00 +0000
4702+++ HandlerSocket-Plugin-for-MySQL/handlersocket/Makefile.am 2012-02-07 16:47:34 +0000
4703@@ -0,0 +1,10 @@
4704+pkgplugindir = $(PLUGIN_DIR)
4705+noinst_HEADERS = database.hpp hstcpsvr.hpp hstcpsvr_worker.hpp mysql_incl.hpp
4706+pkgplugin_LTLIBRARIES = handlersocket.la
4707+handlersocket_la_LDFLAGS = -module ../libhsclient/libhsclient.la
4708+handlersocket_la_CFLAGS = $(MYSQL_INC) $(MYSQL_CFLAGS) $(AM_CFLAGS) \
4709+ -I../libhsclient
4710+handlersocket_la_CXXFLAGS = $(MYSQL_INC) $(MYSQL_CFLAGS) $(AM_CFLAGS) \
4711+ -I../libhsclient
4712+handlersocket_la_SOURCES = database.cpp handlersocket.cpp \
4713+ hstcpsvr_worker.cpp hstcpsvr.cpp
4714
4715=== added file 'HandlerSocket-Plugin-for-MySQL/handlersocket/Makefile.plain.template'
4716--- HandlerSocket-Plugin-for-MySQL/handlersocket/Makefile.plain.template 1970-01-01 00:00:00 +0000
4717+++ HandlerSocket-Plugin-for-MySQL/handlersocket/Makefile.plain.template 2012-02-07 16:47:34 +0000
4718@@ -0,0 +1,31 @@
4719+
4720+MYSQL_INC = HANDLERSOCKET_MYSQL_INC
4721+MYSQL_LIB = HANDLERSOCKET_MYSQL_LIB
4722+
4723+CXX = g++ -Wall -g -fno-rtti -fno-exceptions -fPIC -DPIC
4724+LIBS = $(MYSQL_LIB) -lhsclient -lpthread -lz
4725+CXXFLAGS = -I/usr/include/handlersocket $(MYSQL_INC)
4726+LDFLAGS =
4727+
4728+CXXFLAGS += -O3 -DNDEBUG
4729+
4730+HANDLERSOCKET_OBJS = database.o hstcpsvr.o hstcpsvr_worker.o
4731+
4732+all: handlersocket.so
4733+
4734+handlersocket.so: $(HANDLERSOCKET_OBJS) handlersocket.cpp
4735+ $(CXX) $(CXXFLAGS) -fno-strict-aliasing -shared $^ -o $@ $(LDFLAGS) \
4736+ -Wl,-soname -Wl,$@ $(LIBS)
4737+clean:
4738+ rm -f *.a *.so *.o
4739+
4740+LIBDIR = $(shell \
4741+ if [ -e /usr/lib64/mysql ]; then echo /usr/lib64; else echo /usr/lib; fi)
4742+
4743+install: handlersocket.so
4744+ sudo sh -c 'ulimit -c unlimited ; /etc/init.d/mysql stop ; \
4745+ cp handlersocket.so handlersocket.so.cpy && \
4746+ mv handlersocket.so.cpy \
4747+ $(LIBDIR)/mysql/plugin/handlersocket.so && \
4748+ /etc/init.d/mysql start'
4749+
4750
4751=== added file 'HandlerSocket-Plugin-for-MySQL/handlersocket/database.cpp'
4752--- HandlerSocket-Plugin-for-MySQL/handlersocket/database.cpp 1970-01-01 00:00:00 +0000
4753+++ HandlerSocket-Plugin-for-MySQL/handlersocket/database.cpp 2012-02-07 16:47:34 +0000
4754@@ -0,0 +1,1183 @@
4755+
4756+// vim:sw=2:ai
4757+
4758+/*
4759+ * Copyright (C) 2010 DeNA Co.,Ltd.. All rights reserved.
4760+ * See COPYRIGHT.txt for details.
4761+ */
4762+
4763+#include <stdlib.h>
4764+#include <stdio.h>
4765+#include <string.h>
4766+
4767+#include "database.hpp"
4768+#include "string_util.hpp"
4769+#include "escape.hpp"
4770+#include "mysql_incl.hpp"
4771+
4772+#define DBG_KEY(x)
4773+#define DBG_SHUT(x)
4774+#define DBG_LOCK(x)
4775+#define DBG_THR(x)
4776+#define DBG_CMP(x)
4777+#define DBG_FLD(x)
4778+#define DBG_FILTER(x)
4779+#define DBG_REFCNT(x)
4780+#define DBG_KEYLEN(x)
4781+#define DBG_DELETED
4782+
4783+/* status variables */
4784+unsigned long long int open_tables_count;
4785+unsigned long long int close_tables_count;
4786+unsigned long long int lock_tables_count;
4787+unsigned long long int unlock_tables_count;
4788+unsigned long long int index_exec_count;
4789+
4790+namespace dena {
4791+
4792+prep_stmt::prep_stmt()
4793+ : dbctx(0), table_id(static_cast<size_t>(-1)),
4794+ idxnum(static_cast<size_t>(-1))
4795+{
4796+}
4797+prep_stmt::prep_stmt(dbcontext_i *c, size_t tbl, size_t idx,
4798+ const fields_type& rf, const fields_type& ff)
4799+ : dbctx(c), table_id(tbl), idxnum(idx), ret_fields(rf), filter_fields(ff)
4800+{
4801+ if (dbctx) {
4802+ dbctx->table_addref(table_id);
4803+ }
4804+}
4805+prep_stmt::~prep_stmt()
4806+{
4807+ if (dbctx) {
4808+ dbctx->table_release(table_id);
4809+ }
4810+}
4811+
4812+prep_stmt::prep_stmt(const prep_stmt& x)
4813+ : dbctx(x.dbctx), table_id(x.table_id), idxnum(x.idxnum),
4814+ ret_fields(x.ret_fields), filter_fields(x.filter_fields)
4815+{
4816+ if (dbctx) {
4817+ dbctx->table_addref(table_id);
4818+ }
4819+}
4820+
4821+prep_stmt&
4822+prep_stmt::operator =(const prep_stmt& x)
4823+{
4824+ if (this != &x) {
4825+ if (dbctx) {
4826+ dbctx->table_release(table_id);
4827+ }
4828+ dbctx = x.dbctx;
4829+ table_id = x.table_id;
4830+ idxnum = x.idxnum;
4831+ ret_fields = x.ret_fields;
4832+ filter_fields = x.filter_fields;
4833+ if (dbctx) {
4834+ dbctx->table_addref(table_id);
4835+ }
4836+ }
4837+ return *this;
4838+}
4839+
4840+struct database : public database_i, private noncopyable {
4841+ database(const config& c);
4842+ virtual ~database();
4843+ virtual dbcontext_ptr create_context(bool for_write) volatile;
4844+ virtual void stop() volatile;
4845+ virtual const config& get_conf() const volatile;
4846+ public:
4847+ int child_running;
4848+ private:
4849+ config conf;
4850+};
4851+
4852+struct tablevec_entry {
4853+ TABLE *table;
4854+ size_t refcount;
4855+ bool modified;
4856+ tablevec_entry() : table(0), refcount(0), modified(false) { }
4857+};
4858+
4859+struct expr_user_lock : private noncopyable {
4860+ expr_user_lock(THD *thd, int timeout)
4861+ : lck_key("handlersocket_wr", 16, &my_charset_latin1),
4862+ lck_timeout(timeout),
4863+ lck_func_get_lock(&lck_key, &lck_timeout),
4864+ lck_func_release_lock(&lck_key)
4865+ {
4866+ lck_key.fix_fields(thd, 0);
4867+ lck_timeout.fix_fields(thd, 0);
4868+ lck_func_get_lock.fix_fields(thd, 0);
4869+ lck_func_release_lock.fix_fields(thd, 0);
4870+ }
4871+ long long get_lock() {
4872+ return lck_func_get_lock.val_int();
4873+ }
4874+ long long release_lock() {
4875+ return lck_func_release_lock.val_int();
4876+ }
4877+ private:
4878+ Item_string lck_key;
4879+ Item_int lck_timeout;
4880+ Item_func_get_lock lck_func_get_lock;
4881+ Item_func_release_lock lck_func_release_lock;
4882+};
4883+
4884+struct dbcontext : public dbcontext_i, private noncopyable {
4885+ dbcontext(volatile database *d, bool for_write);
4886+ virtual ~dbcontext();
4887+ virtual void init_thread(const void *stack_botton,
4888+ volatile int& shutdown_flag);
4889+ virtual void term_thread();
4890+ virtual bool check_alive();
4891+ virtual void lock_tables_if();
4892+ virtual void unlock_tables_if();
4893+ virtual bool get_commit_error();
4894+ virtual void clear_error();
4895+ virtual void close_tables_if();
4896+ virtual void table_addref(size_t tbl_id);
4897+ virtual void table_release(size_t tbl_id);
4898+ virtual void cmd_open(dbcallback_i& cb, const cmd_open_args& args);
4899+ virtual void cmd_exec(dbcallback_i& cb, const cmd_exec_args& args);
4900+ virtual void set_statistics(size_t num_conns, size_t num_active);
4901+ private:
4902+ int set_thread_message(const char *fmt, ...)
4903+ __attribute__((format (printf, 2, 3)));
4904+ bool parse_fields(TABLE *const table, const char *str,
4905+ prep_stmt::fields_type& flds);
4906+ void cmd_insert_internal(dbcallback_i& cb, const prep_stmt& pst,
4907+ const string_ref *fvals, size_t fvalslen);
4908+ void cmd_sql_internal(dbcallback_i& cb, const prep_stmt& pst,
4909+ const string_ref *fvals, size_t fvalslen);
4910+ void cmd_find_internal(dbcallback_i& cb, const prep_stmt& pst,
4911+ ha_rkey_function find_flag, const cmd_exec_args& args);
4912+ size_t calc_filter_buf_size(TABLE *table, const prep_stmt& pst,
4913+ const record_filter *filters);
4914+ bool fill_filter_buf(TABLE *table, const prep_stmt& pst,
4915+ const record_filter *filters, uchar *filter_buf, size_t len);
4916+ int check_filter(dbcallback_i& cb, TABLE *table, const prep_stmt& pst,
4917+ const record_filter *filters, const uchar *filter_buf);
4918+ void resp_record(dbcallback_i& cb, TABLE *const table, const prep_stmt& pst);
4919+ void dump_record(dbcallback_i& cb, TABLE *const table, const prep_stmt& pst);
4920+ int modify_record(dbcallback_i& cb, TABLE *const table,
4921+ const prep_stmt& pst, const cmd_exec_args& args, char mod_op,
4922+ size_t& modified_count);
4923+ private:
4924+ typedef std::vector<tablevec_entry> table_vec_type;
4925+ typedef std::pair<std::string, std::string> table_name_type;
4926+ typedef std::map<table_name_type, size_t> table_map_type;
4927+ private:
4928+ volatile database *const dbref;
4929+ bool for_write_flag;
4930+ THD *thd;
4931+ MYSQL_LOCK *lock;
4932+ bool lock_failed;
4933+ std::auto_ptr<expr_user_lock> user_lock;
4934+ int user_level_lock_timeout;
4935+ bool user_level_lock_locked;
4936+ bool commit_error;
4937+ std::vector<char> info_message_buf;
4938+ table_vec_type table_vec;
4939+ table_map_type table_map;
4940+};
4941+
4942+database::database(const config& c)
4943+ : child_running(1), conf(c)
4944+{
4945+}
4946+
4947+database::~database()
4948+{
4949+}
4950+
4951+dbcontext_ptr
4952+database::create_context(bool for_write) volatile
4953+{
4954+ return dbcontext_ptr(new dbcontext(this, for_write));
4955+}
4956+
4957+void
4958+database::stop() volatile
4959+{
4960+ child_running = false;
4961+}
4962+
4963+const config&
4964+database::get_conf() const volatile
4965+{
4966+ return const_cast<const config&>(conf);
4967+}
4968+
4969+database_ptr
4970+database_i::create(const config& conf)
4971+{
4972+ return database_ptr(new database(conf));
4973+}
4974+
4975+dbcontext::dbcontext(volatile database *d, bool for_write)
4976+ : dbref(d), for_write_flag(for_write), thd(0), lock(0), lock_failed(false),
4977+ user_level_lock_timeout(0), user_level_lock_locked(false),
4978+ commit_error(false)
4979+{
4980+ info_message_buf.resize(8192);
4981+ user_level_lock_timeout = d->get_conf().get_int("wrlock_timeout", 12);
4982+}
4983+
4984+dbcontext::~dbcontext()
4985+{
4986+}
4987+
4988+namespace {
4989+
4990+int
4991+wait_server_to_start(THD *thd, volatile int& shutdown_flag)
4992+{
4993+ int r = 0;
4994+ DBG_SHUT(fprintf(stderr, "HNDSOCK wsts\n"));
4995+ pthread_mutex_lock(&LOCK_server_started);
4996+ while (!mysqld_server_started) {
4997+ timespec abstime = { };
4998+ set_timespec(abstime, 1);
4999+ pthread_cond_timedwait(&COND_server_started, &LOCK_server_started,
5000+ &abstime);
The diff has been truncated for viewing.

Subscribers

People subscribed via source and target branches