Merge lp:~tsarev/percona-server/5.5.12-794345 into lp:~percona-dev/percona-server/5.5.11

Proposed by Oleg Tsarev
Status: Superseded
Proposed branch: lp:~tsarev/percona-server/5.5.12-794345
Merge into: lp:~percona-dev/percona-server/5.5.11
Diff against target: 65549 lines
To merge this branch: bzr merge lp:~tsarev/percona-server/5.5.12-794345
Reviewer Review Type Date Requested Status
Alexey Kopytov (community) Needs Fixing
Valentine Gostev qa Pending
Review via email: mp+63792@code.launchpad.net

This proposal has been superseded by a proposal from 2011-06-08.

Description of the change

Reason of bugs #786645 and #794345: when we run test under valgrind test work long time, and slow query log has extra query "call mtr.check_testcase();".
I disable slow query log on startup, and enable only when I need.
This is actual for tests:
 * percona_slow_extended-log_slow_filter
 * percona_slow_extended-long_query_time
 * percona_slow_extended-microseconds_in_slow_extended
 * percona_slow_extended-min_examined_row_limit
 * percona_slow_extended-use_global_long_query_time

To post a comment you must log in.
Revision history for this message
Alexey Kopytov (akopytov) wrote :

You should be merging to release-5.5.12-20.3 rather than lp:percona-server which is a very outdated branch currently. Makes reviewing the diff impossible.

review: Needs Fixing

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== added directory 'HandlerSocket-Plugin-for-MySQL'
2=== added file 'HandlerSocket-Plugin-for-MySQL/AUTHORS'
3--- HandlerSocket-Plugin-for-MySQL/AUTHORS 1970-01-01 00:00:00 +0000
4+++ HandlerSocket-Plugin-for-MySQL/AUTHORS 2011-06-08 01:57:28 +0000
5@@ -0,0 +1,19 @@
6+Akira Higuchi (https://github.com/ahiguti)
7+ - developed HanderSocket plugin, libhsclient, and perl-Net-HandlerSocket
8+
9+Yoshinori Matsunobu (https://github.com/yoshinorim)
10+ - introduced autotools, added support for MySQL 5.5.6, added statistics
11+ variables
12+
13+Jeff Hodges (https://github.com/jmhodges)
14+ - fixed some autotools scripts
15+
16+Toru Yamaguchi (https://github.com/zigorou)
17+ - ported to MacOS X
18+
19+Moriyoshi Koizumi (https://github.com/moriyoshi)
20+ - fixed some autotools scripts
21+
22+takeda-at (https://github.com/takada-at)
23+ - added simple authorization function
24+
25
26=== added file 'HandlerSocket-Plugin-for-MySQL/COPYING'
27--- HandlerSocket-Plugin-for-MySQL/COPYING 1970-01-01 00:00:00 +0000
28+++ HandlerSocket-Plugin-for-MySQL/COPYING 2011-06-08 01:57:28 +0000
29@@ -0,0 +1,30 @@
30+-----------------------------------------------------------------------------
31+HandlerSocket plugin for MySQL
32+
33+ Copyright (c) 2010 DeNA Co.,Ltd.
34+ All rights reserved.
35+
36+ Redistribution and use in source and binary forms, with or without
37+ modification, are permitted provided that the following conditions are met:
38+
39+ * Redistributions of source code must retain the above copyright
40+ notice, this list of conditions and the following disclaimer.
41+ * Redistributions in binary form must reproduce the above copyright
42+ notice, this list of conditions and the following disclaimer in the
43+ documentation and/or other materials provided with the distribution.
44+ * Neither the name of DeNA Co.,Ltd. nor the names of its contributors
45+ may be used to endorse or promote products derived from this software
46+ without specific prior written permission.
47+
48+ THIS SOFTWARE IS PROVIDED BY DeNA Co.,Ltd. "AS IS" AND ANY EXPRESS OR
49+ IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
50+ MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO
51+ EVENT SHALL DeNA Co.,Ltd. BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
52+ SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
53+ PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS;
54+ OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
55+ WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
56+ OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
57+ ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
58+
59+
60
61=== added file 'HandlerSocket-Plugin-for-MySQL/ChangeLog'
62--- HandlerSocket-Plugin-for-MySQL/ChangeLog 1970-01-01 00:00:00 +0000
63+++ HandlerSocket-Plugin-for-MySQL/ChangeLog 2011-06-08 01:57:28 +0000
64@@ -0,0 +1,12 @@
65+1.0.6 - 2010-10-29
66+ * Changed build instruction (autoreconf/configure/make), removed auto-generated files (Contributed by jmhodges)
67+ *
68+
69+1.0.5 - 2010-10-18
70+ * Changed build procedures (using typical configure/make)
71+ * Supported 5.5.6
72+ * Added status variables
73+
74+1.0.4 - 2010-08-15
75+ * Initial public release
76+
77
78=== added file 'HandlerSocket-Plugin-for-MySQL/Makefile.am'
79--- HandlerSocket-Plugin-for-MySQL/Makefile.am 1970-01-01 00:00:00 +0000
80+++ HandlerSocket-Plugin-for-MySQL/Makefile.am 2011-06-08 01:57:28 +0000
81@@ -0,0 +1,87 @@
82+
83+ACLOCAL_AMFLAGS = -I m4
84+
85+SUBDIRS = @HANDLERSOCKET_SUBDIRS@
86+
87+perl:
88+ cd perl-Net-HandlerSocket && perl Makefile.PL && make
89+
90+install_perl:
91+ cd perl-Net-HandlerSocket && make install
92+
93+rpms: rpm_cli rpm_perl rpm_c
94+
95+rpm_dir:
96+ - mkdir dist
97+ - mkdir dist/BUILD dist/RPMS dist/SOURCES dist/SPECS dist/SRPMS
98+
99+rpm_cli: clean_cli rpm_dir
100+ sed -e "s/HANDLERSOCKET_VERSION/$(VERSION)/" \
101+ libhsclient/libhsclient.spec.template \
102+ > libhsclient/libhsclient.spec
103+ tar cvfz dist/libhsclient.tar.gz libhsclient
104+ rpmbuild --define "_topdir `pwd`/dist" -ta \
105+ dist/libhsclient.tar.gz
106+
107+rpm_perl: clean_perl rpm_dir
108+ sed -e "s/HANDLERSOCKET_VERSION/$(VERSION)/" \
109+ perl-Net-HandlerSocket/perl-Net-HandlerSocket.spec.template \
110+ > perl-Net-HandlerSocket/perl-Net-HandlerSocket.spec
111+ cd perl-Net-HandlerSocket && perl Makefile.PL && make clean && \
112+ rm -f Makefile.old
113+ tar cvfz dist/perl-Net-HandlerSocket.tar.gz perl-Net-HandlerSocket
114+ rpmbuild --define "_topdir `pwd`/dist" -ta \
115+ dist/perl-Net-HandlerSocket.tar.gz
116+
117+rpm_c: clean_c rpm_dir
118+ sed -e "s/HANDLERSOCKET_VERSION/$(VERSION)/" \
119+ handlersocket/handlersocket.spec.template \
120+ > handlersocket/handlersocket.spec
121+ sed -e "s|HANDLERSOCKET_MYSQL_INC|$(MYSQL_CFLAGS) $(MYSQL_INC)|" \
122+ -e "s|HANDLERSOCKET_MYSQL_LIB|$(MYSQL_LIB)|" \
123+ handlersocket/Makefile.plain.template \
124+ > handlersocket/Makefile.plain
125+ tar cvfz dist/handlersocket.tar.gz handlersocket
126+ rpmbuild --define "_topdir `pwd`/dist" -ta \
127+ dist/handlersocket.tar.gz
128+
129+install_rpm_pl:
130+ - sudo rpm -e perl-Net-HandlerSocket
131+ - sudo rpm -e perl-Net-HandlerSocket-debuginfo
132+ make clean
133+ make rpm_perl
134+ - sudo rpm -U dist/RPMS/*/perl*.rpm
135+
136+installrpms:
137+ - sudo rpm -e handlersocket
138+ - sudo rpm -e handlersocket-debuginfo
139+ - sudo rpm -e perl-Net-HandlerSocket
140+ - sudo rpm -e perl-Net-HandlerSocket-debuginfo
141+ - sudo rpm -e libhsclient
142+ - sudo rpm -e libhsclient-debuginfo
143+ make clean
144+ make rpm_cli
145+ - sudo rpm -U dist/RPMS/*/libhsclient*.rpm
146+ make clean
147+ make rpm_perl
148+ - sudo rpm -U dist/RPMS/*/perl*.rpm
149+ make clean
150+ make rpm_c
151+ - sudo rpm -U dist/RPMS/*/handlersocket*.rpm
152+
153+clean_cli:
154+ cd libhsclient && make clean
155+ cd client && make clean
156+
157+clean_perl:
158+ cd perl-Net-HandlerSocket && perl Makefile.PL && make clean && \
159+ rm -f Makefile.old
160+
161+clean_c:
162+ cd handlersocket && make clean
163+
164+clean_all: clean_cli clean_perl clean_c
165+ cd regtest && make clean
166+ rm -rf dist/*/*
167+ rm -f dist/*.tar.gz
168+
169
170=== added file 'HandlerSocket-Plugin-for-MySQL/README'
171--- HandlerSocket-Plugin-for-MySQL/README 1970-01-01 00:00:00 +0000
172+++ HandlerSocket-Plugin-for-MySQL/README 2011-06-08 01:57:28 +0000
173@@ -0,0 +1,76 @@
174+
175+-----------------------------------------------------------------------------
176+HandlerSocket plugin for MySQL
177+
178+Copyright (c) 2010 DeNA Co.,Ltd.
179+All rights reserved.
180+
181+Redistribution and use in source and binary forms, with or without
182+modification, are permitted provided that the following conditions are met:
183+
184+ * Redistributions of source code must retain the above copyright
185+ notice, this list of conditions and the following disclaimer.
186+ * Redistributions in binary form must reproduce the above copyright
187+ notice, this list of conditions and the following disclaimer in the
188+ documentation and/or other materials provided with the distribution.
189+ * Neither the name of DeNA Co.,Ltd. nor the names of its contributors
190+ may be used to endorse or promote products derived from this software
191+ without specific prior written permission.
192+
193+THIS SOFTWARE IS PROVIDED BY DeNA Co.,Ltd. "AS IS" AND ANY EXPRESS OR
194+IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
195+MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO
196+EVENT SHALL DeNA Co.,Ltd. BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
197+SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
198+PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS;
199+OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
200+WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
201+OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
202+ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
203+
204+
205+-----------------------------------------------------------------------------
206+About HandlerSocket
207+
208+HandlerSocket is a NoSQL plugin for MySQL. It works as a daemon inside the
209+mysqld process, accept tcp connections, and execute requests from clients.
210+HandlerSocket does not support SQL queries. Instead, it supports simple CRUD
211+operations on tables.
212+
213+Because of the following reasons, HandlerSocket is much faster than the
214+mysqld/libmysql pair in some circumstances:
215+
216+ - HandlerSocket manipulates data without parsing SQL, which causes less
217+ CPU usage.
218+ - HandlerSocket reads many requests from clients and executes their
219+ requests in bulk, which causes less CPU and disk usage.
220+ - HandlerSocket client/server protocol is more compact than the
221+ mysql/libmysql pair, which causes less network usage.
222+
223+The current version of HandlerSocket only works with GNU/Linux. The source
224+archive of HandlerSocket includes a C++ and a Perl client libraries.
225+Here is a list of client libraries for other languages:
226+
227+ - PHP
228+ http://openpear.org/package/Net_HandlerSocket
229+ http://github.com/tz-lom/HSPHP
230+ http://code.google.com/p/php-handlersocket/
231+ - Java
232+ http://code.google.com/p/hs4j/
233+ http://code.google.com/p/handlersocketforjava/
234+ - Python
235+ http://pypi.python.org/pypi/python-handler-socket
236+ https://code.launchpad.net/~songofacandy/+junk/pyhandlersocket
237+ - Ruby
238+ https://github.com/winebarrel/ruby-handlersocket
239+ https://github.com/miyucy/handlersocket
240+ - JavaScript
241+ https://github.com/koichik/node-handlersocket
242+ - Scala
243+ https://github.com/fujohnwang/hs2client
244+
245+The home of HandlerSocket is here:
246+ https://github.com/ahiguti/HandlerSocket-Plugin-for-MySQL
247+
248+More documents are available in docs-en/ and docs-ja/ directories.
249+
250
251=== added file 'HandlerSocket-Plugin-for-MySQL/autogen.sh'
252--- HandlerSocket-Plugin-for-MySQL/autogen.sh 1970-01-01 00:00:00 +0000
253+++ HandlerSocket-Plugin-for-MySQL/autogen.sh 2011-06-08 01:57:28 +0000
254@@ -0,0 +1,117 @@
255+#!/bin/sh
256+
257+warn() {
258+echo -e "\tWARNING: $@" 1>&2
259+}
260+
261+# init
262+
263+LIBTOOLIZE=libtoolize
264+ACLOCAL=aclocal
265+AUTOCONF=autoconf
266+AUTOHEADER=autoheader
267+AUTOMAKE=automake
268+
269+case `uname -s` in
270+Darwin)
271+LIBTOOLIZE=glibtoolize
272+;;
273+FreeBSD)
274+ACLOCAL_ARGS="$ACLOCAL_ARGS -I /usr/local/share/aclocal/"
275+;;
276+esac
277+
278+
279+# libtoolize
280+echo "Searching libtoolize..."
281+if [ `which $LIBTOOLIZE` ] ; then
282+echo -e "\tFOUND: libtoolize -> $LIBTOOLIZE"
283+else
284+warn "Cannot Found libtoolize... input libtool command"
285+ read LIBTOOLIZE
286+ LIBTOOLIZE=`which $LIBTOOLIZE`
287+ if [ `which $LIBTOOLIZE` ] ; then
288+echo -e "\tSET: libtoolize -> $LIBTOOLIZE"
289+ else
290+warn "$LIBTOOLIZE: Command not found."
291+ exit 1;
292+ fi
293+fi
294+
295+# aclocal
296+echo "Searching aclocal..."
297+if [ `which $ACLOCAL` ] ; then
298+echo -e "\tFOUND: aclocal -> $ACLOCAL"
299+else
300+warn "Cannot Found aclocal... input aclocal command"
301+ read ACLOCAL
302+ ACLOCAL=`which $ACLOCAL`
303+ if [ `which $ACLOCAL` ] ; then
304+echo -e "\tSET: aclocal -> $ACLOCAL"
305+ else
306+warn "$ACLOCAL: Command not found."
307+ exit 1;
308+ fi
309+fi
310+
311+# automake
312+echo "Searching automake..."
313+if [ `which $AUTOMAKE` ] ; then
314+echo -e "\tFOUND: automake -> $AUTOMAKE"
315+else
316+warn "Cannot Found automake... input automake command"
317+ read AUTOMAKE
318+ ACLOCAL=`which $AUTOMAKE`
319+ if [ `which $AUTOMAKE` ] ; then
320+echo -e "\tSET: automake -> $AUTOMAKE"
321+ else
322+warn "$AUTOMAKE: Command not found."
323+ exit 1;
324+ fi
325+fi
326+
327+# autoheader
328+echo "Searching autoheader..."
329+if [ `which $AUTOHEADER` ] ; then
330+echo -e "\tFOUND: autoheader -> $AUTOHEADER"
331+else
332+warn "Cannot Found autoheader... input autoheader command"
333+ read AUTOHEADER
334+ ACLOCAL=`which $AUTOHEADER`
335+ if [ `which $AUTOHEADER` ] ; then
336+echo -e "\tSET: autoheader -> $AUTOHEADER"
337+ else
338+warn "$AUTOHEADER: Command not found."
339+ exit 1;
340+ fi
341+fi
342+
343+# autoconf
344+echo "Searching autoconf..."
345+if [ `which $AUTOCONF` ] ; then
346+echo -e "\tFOUND: autoconf -> $AUTOCONF"
347+else
348+warn "Cannot Found autoconf... input autoconf command"
349+ read AUTOCONF
350+ ACLOCAL=`which $AUTOCONF`
351+ if [ `which $AUTOCONF` ] ; then
352+echo -e "\tSET: autoconf -> $AUTOCONF"
353+ else
354+warn "$AUTOCONF: Command not found."
355+ exit 1;
356+ fi
357+fi
358+
359+echo "Running libtoolize ..."
360+$LIBTOOLIZE --force --copy
361+echo "Running aclocal ..."
362+$ACLOCAL ${ACLOCAL_ARGS} -I .
363+echo "Running autoheader..."
364+$AUTOHEADER
365+echo "Running automake ..."
366+$AUTOMAKE --add-missing --copy
367+echo "Running autoconf ..."
368+$AUTOCONF
369+
370+mkdir -p m4
371+
372
373=== added directory 'HandlerSocket-Plugin-for-MySQL/client'
374=== added file 'HandlerSocket-Plugin-for-MySQL/client/Makefile.am'
375--- HandlerSocket-Plugin-for-MySQL/client/Makefile.am 1970-01-01 00:00:00 +0000
376+++ HandlerSocket-Plugin-for-MySQL/client/Makefile.am 2011-06-08 01:57:28 +0000
377@@ -0,0 +1,24 @@
378+AM_INCLUDES= -I../libhsclient
379+bin_PROGRAMS=hsclient
380+hsclient_SOURCES= hsclient.cpp
381+hsclient_LDFLAGS= -static -L../libhsclient -lhsclient
382+hsclient_CXXFLAGS= $(AM_INCLUDES)
383+
384+hstest: hstest.o
385+ $(CXX) $(CXXFLAGS) $(LFLAGS) hstest.o \
386+ -L../libhsclient/.libs -lhsclient $$(mysql_config --libs_r) \
387+ -o hstest
388+
389+hstest.o: hstest.cpp
390+ $(CXX) $(CXXFLAGS) $(AM_INCLUDES) $$(mysql_config --include) \
391+ -c hstest.cpp
392+
393+hslongrun: hslongrun.o
394+ $(CXX) $(CXXFLAGS) $(LFLAGS) hslongrun.o \
395+ -L../libhsclient/.libs -lhsclient $$(mysql_config --libs_r) \
396+ -o hslongrun
397+
398+hslongrun.o: hslongrun.cpp
399+ $(CXX) $(CXXFLAGS) $(AM_INCLUDES) $$(mysql_config --include) \
400+ -c hslongrun.cpp
401+
402
403=== added file 'HandlerSocket-Plugin-for-MySQL/client/hsclient.cpp'
404--- HandlerSocket-Plugin-for-MySQL/client/hsclient.cpp 1970-01-01 00:00:00 +0000
405+++ HandlerSocket-Plugin-for-MySQL/client/hsclient.cpp 2011-06-08 01:57:28 +0000
406@@ -0,0 +1,88 @@
407+
408+// vim:sw=2:ai
409+
410+#include "hstcpcli.hpp"
411+#include "string_util.hpp"
412+
413+namespace dena {
414+
415+int
416+hstcpcli_main(int argc, char **argv)
417+{
418+ config conf;
419+ parse_args(argc, argv, conf);
420+ socket_args sockargs;
421+ sockargs.set(conf);
422+ hstcpcli_ptr cli = hstcpcli_i::create(sockargs);
423+ const std::string dbname = conf.get_str("dbname", "hstest");
424+ const std::string table = conf.get_str("table", "hstest_table1");
425+ const std::string index = conf.get_str("index", "PRIMARY");
426+ const std::string fields = conf.get_str("fields", "k,v");
427+ const int limit = conf.get_int("limit", 0);
428+ const int skip = conf.get_int("skip", 0);
429+ std::vector<std::string> keys;
430+ std::vector<string_ref> keyrefs;
431+ size_t num_keys = 0;
432+ while (true) {
433+ const std::string conf_key = std::string("k") + to_stdstring(num_keys);
434+ const std::string k = conf.get_str(conf_key, "");
435+ const std::string kx = conf.get_str(conf_key, "x");
436+ if (k.empty() && kx == "x") {
437+ break;
438+ }
439+ ++num_keys;
440+ keys.push_back(k);
441+ }
442+ for (size_t i = 0; i < keys.size(); ++i) {
443+ const string_ref ref(keys[i].data(), keys[i].size());
444+ keyrefs.push_back(ref);
445+ }
446+ const std::string op = conf.get_str("op", "=");
447+ const string_ref op_ref(op.data(), op.size());
448+ cli->request_buf_open_index(0, dbname.c_str(), table.c_str(),
449+ index.c_str(), fields.c_str());
450+ cli->request_buf_exec_generic(0, op_ref, num_keys == 0 ? 0 : &keyrefs[0],
451+ num_keys, limit, skip, string_ref(), 0, 0);
452+ int code = 0;
453+ size_t numflds = 0;
454+ do {
455+ if (cli->request_send() != 0) {
456+ fprintf(stderr, "request_send: %s\n", cli->get_error().c_str());
457+ break;
458+ }
459+ if ((code = cli->response_recv(numflds)) != 0) {
460+ fprintf(stderr, "response_recv: %s\n", cli->get_error().c_str());
461+ break;
462+ }
463+ } while (false);
464+ cli->response_buf_remove();
465+ do {
466+ if ((code = cli->response_recv(numflds)) != 0) {
467+ fprintf(stderr, "response_recv: %s\n", cli->get_error().c_str());
468+ break;
469+ }
470+ while (true) {
471+ const string_ref *const row = cli->get_next_row();
472+ if (row == 0) {
473+ break;
474+ }
475+ printf("REC:");
476+ for (size_t i = 0; i < numflds; ++i) {
477+ const std::string val(row[i].begin(), row[i].size());
478+ printf(" %s", val.c_str());
479+ }
480+ printf("\n");
481+ }
482+ } while (false);
483+ cli->response_buf_remove();
484+ return 0;
485+}
486+
487+};
488+
489+int
490+main(int argc, char **argv)
491+{
492+ return dena::hstcpcli_main(argc, argv);
493+}
494+
495
496=== added file 'HandlerSocket-Plugin-for-MySQL/client/hslongrun.cpp'
497--- HandlerSocket-Plugin-for-MySQL/client/hslongrun.cpp 1970-01-01 00:00:00 +0000
498+++ HandlerSocket-Plugin-for-MySQL/client/hslongrun.cpp 2011-06-08 01:57:28 +0000
499@@ -0,0 +1,1041 @@
500+
501+// vim:sw=2:ai
502+
503+#include <signal.h>
504+#include <sys/time.h>
505+#include <stdio.h>
506+#include <string.h>
507+#include <vector>
508+#include <map>
509+#include <stdlib.h>
510+#include <memory>
511+#include <errno.h>
512+#include <mysql.h>
513+#include <time.h>
514+#include <sys/types.h>
515+#include <sys/stat.h>
516+#include <fcntl.h>
517+
518+#include "util.hpp"
519+#include "auto_ptrcontainer.hpp"
520+#include "socket.hpp"
521+#include "hstcpcli.hpp"
522+#include "string_util.hpp"
523+#include "mutex.hpp"
524+
525+namespace dena {
526+
527+struct auto_mysql : private noncopyable {
528+ auto_mysql() : db(0) {
529+ reset();
530+ }
531+ ~auto_mysql() {
532+ if (db) {
533+ mysql_close(db);
534+ }
535+ }
536+ void reset() {
537+ if (db) {
538+ mysql_close(db);
539+ }
540+ if ((db = mysql_init(0)) == 0) {
541+ fatal_exit("failed to initialize mysql client");
542+ }
543+ }
544+ operator MYSQL *() const { return db; }
545+ private:
546+ MYSQL *db;
547+};
548+
549+struct auto_mysql_res : private noncopyable {
550+ auto_mysql_res(MYSQL *db) {
551+ res = mysql_store_result(db);
552+ }
553+ ~auto_mysql_res() {
554+ if (res) {
555+ mysql_free_result(res);
556+ }
557+ }
558+ operator MYSQL_RES *() const { return res; }
559+ private:
560+ MYSQL_RES *res;
561+};
562+
563+struct auto_mysql_stmt : private noncopyable {
564+ auto_mysql_stmt(MYSQL *db) {
565+ stmt = mysql_stmt_init(db);
566+ }
567+ ~auto_mysql_stmt() {
568+ if (stmt) {
569+ mysql_stmt_close(stmt);
570+ }
571+ }
572+ operator MYSQL_STMT *() const { return stmt; }
573+ private:
574+ MYSQL_STMT *stmt;
575+};
576+
577+double
578+gettimeofday_double()
579+{
580+ struct timeval tv = { };
581+ if (gettimeofday(&tv, 0) != 0) {
582+ fatal_abort("gettimeofday");
583+ }
584+ return static_cast<double>(tv.tv_usec) / 1000000 + tv.tv_sec;
585+}
586+
587+struct record_value {
588+ mutex lock;
589+ bool deleted;
590+ bool unknown_state;
591+ std::string key;
592+ std::vector<std::string> values;
593+ record_value() : deleted(true), unknown_state(false) { }
594+};
595+
596+struct hs_longrun_shared {
597+ config conf;
598+ socket_args arg;
599+ int verbose;
600+ long num_threads;
601+ int usleep;
602+ volatile mutable int running;
603+ auto_ptrcontainer< std::vector<record_value *> > records;
604+ hs_longrun_shared() : verbose(0), num_threads(0), usleep(0), running(1) { }
605+};
606+
607+struct thread_base {
608+ thread_base() : need_join(false), stack_size(256 * 1024) { }
609+ virtual ~thread_base() {
610+ join();
611+ }
612+ virtual void run() = 0;
613+ void start() {
614+ if (!start_nothrow()) {
615+ fatal_abort("thread::start");
616+ }
617+ }
618+ bool start_nothrow() {
619+ if (need_join) {
620+ return need_join; /* true */
621+ }
622+ void *const arg = this;
623+ pthread_attr_t attr;
624+ if (pthread_attr_init(&attr) != 0) {
625+ fatal_abort("pthread_attr_init");
626+ }
627+ if (pthread_attr_setstacksize(&attr, stack_size) != 0) {
628+ fatal_abort("pthread_attr_setstacksize");
629+ }
630+ const int r = pthread_create(&thr, &attr, thread_main, arg);
631+ if (pthread_attr_destroy(&attr) != 0) {
632+ fatal_abort("pthread_attr_destroy");
633+ }
634+ if (r != 0) {
635+ return need_join; /* false */
636+ }
637+ need_join = true;
638+ return need_join; /* true */
639+ }
640+ void join() {
641+ if (!need_join) {
642+ return;
643+ }
644+ int e = 0;
645+ if ((e = pthread_join(thr, 0)) != 0) {
646+ fatal_abort("pthread_join");
647+ }
648+ need_join = false;
649+ }
650+ private:
651+ static void *thread_main(void *arg) {
652+ thread_base *p = static_cast<thread_base *>(arg);
653+ p->run();
654+ return 0;
655+ }
656+ private:
657+ pthread_t thr;
658+ bool need_join;
659+ size_t stack_size;
660+};
661+
662+struct hs_longrun_stat {
663+ unsigned long long verify_error_count;
664+ unsigned long long runtime_error_count;
665+ unsigned long long unknown_count;
666+ unsigned long long success_count;
667+ hs_longrun_stat()
668+ : verify_error_count(0), runtime_error_count(0),
669+ unknown_count(0), success_count(0) { }
670+ void add(const hs_longrun_stat& x) {
671+ verify_error_count += x.verify_error_count;
672+ runtime_error_count += x.runtime_error_count;
673+ unknown_count += x.unknown_count;
674+ success_count += x.success_count;
675+ }
676+};
677+
678+struct hs_longrun_thread_base : public thread_base {
679+ struct arg_type {
680+ int id;
681+ std::string worker_type;
682+ char op;
683+ int lock_flag;
684+ const hs_longrun_shared& sh;
685+ arg_type(int id, const std::string& worker_type, char op, int lock_flag,
686+ const hs_longrun_shared& sh)
687+ : id(id), worker_type(worker_type), op(op), lock_flag(lock_flag),
688+ sh(sh) { }
689+ };
690+ arg_type arg;
691+ hs_longrun_stat stat;
692+ drand48_data randbuf;
693+ unsigned int seed;
694+ hs_longrun_thread_base(const arg_type& arg)
695+ : arg(arg), seed(0) {
696+ seed = time(0) + arg.id + 1;
697+ srand48_r(seed, &randbuf);
698+ }
699+ virtual ~hs_longrun_thread_base() { }
700+ virtual void run() = 0;
701+ size_t rand_record() {
702+ double v = 0;
703+ drand48_r(&randbuf, &v);
704+ const size_t sz = arg.sh.records.size();
705+ size_t r = size_t(v * sz);
706+ if (r >= sz) {
707+ r = 0;
708+ }
709+ return r;
710+ }
711+ int verify_update(const std::string& k, const std::string& v1,
712+ const std::string& v2, const std::string& v3, record_value& rec,
713+ uint32_t num_rows, bool cur_unknown_state);
714+ int verify_read(const std::string& k, uint32_t num_rows, uint32_t num_flds,
715+ const std::string rrec[4], record_value& rec);
716+ int verify_readnolock(const std::string& k, uint32_t num_rows,
717+ uint32_t num_flds, const std::string rrec[4]);
718+};
719+
720+int
721+hs_longrun_thread_base::verify_update(const std::string& k,
722+ const std::string& v1, const std::string& v2, const std::string& v3,
723+ record_value& rec, uint32_t num_rows, bool cur_unknown_state)
724+{
725+ const bool op_success = num_rows == 1;
726+ int ret = 0;
727+ if (!rec.unknown_state) {
728+ if (!rec.deleted && !op_success) {
729+ ++stat.verify_error_count;
730+ if (arg.sh.verbose > 0) {
731+ fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
732+ "unexpected_update_failure\n",
733+ arg.worker_type.c_str(), arg.id, k.c_str());
734+ }
735+ ret = 1;
736+ } else if (rec.deleted && op_success) {
737+ ++stat.verify_error_count;
738+ if (arg.sh.verbose > 0) {
739+ fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
740+ "unexpected_update_success\n",
741+ arg.worker_type.c_str(), arg.id, k.c_str());
742+ }
743+ ret = 1;
744+ }
745+ }
746+ if (op_success) {
747+ rec.values.resize(4);
748+ rec.values[0] = k;
749+ rec.values[1] = v1;
750+ rec.values[2] = v2;
751+ rec.values[3] = v3;
752+ if (ret == 0 && !rec.unknown_state) {
753+ ++stat.success_count;
754+ }
755+ }
756+ rec.unknown_state = cur_unknown_state;
757+ if (arg.sh.verbose >= 100 && ret == 0) {
758+ fprintf(stderr, "%s %s %s %s %s\n", arg.worker_type.c_str(),
759+ k.c_str(), v1.c_str(), v2.c_str(), v3.c_str());
760+ }
761+ return ret;
762+}
763+
764+int
765+hs_longrun_thread_base::verify_read(const std::string& k,
766+ uint32_t num_rows, uint32_t num_flds, const std::string rrec[4],
767+ record_value& rec)
768+{
769+ const bool op_success = num_rows != 0;
770+ int ret = 0;
771+ if (!rec.unknown_state) {
772+ if (!rec.deleted && !op_success) {
773+ ++stat.verify_error_count;
774+ if (arg.sh.verbose > 0) {
775+ fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
776+ "unexpected_read_failure\n",
777+ arg.worker_type.c_str(), arg.id, k.c_str());
778+ }
779+ ret = 1;
780+ } else if (rec.deleted && op_success) {
781+ ++stat.verify_error_count;
782+ if (arg.sh.verbose > 0) {
783+ fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
784+ "unexpected_read_success\n",
785+ arg.worker_type.c_str(), arg.id, k.c_str());
786+ }
787+ ret = 1;
788+ } else if (num_flds != 4) {
789+ ++stat.verify_error_count;
790+ if (arg.sh.verbose > 0) {
791+ fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
792+ "unexpected_read_fldnum %d\n",
793+ arg.worker_type.c_str(), arg.id, k.c_str(),
794+ static_cast<int>(num_flds));
795+ }
796+ ret = 1;
797+ } else if (rec.deleted) {
798+ /* nothing to verify */
799+ } else {
800+ int diff = 0;
801+ for (size_t i = 0; i < 4; ++i) {
802+ if (rec.values[i] == rrec[i]) {
803+ /* ok */
804+ } else {
805+ diff = 1;
806+ }
807+ }
808+ if (diff) {
809+ std::string mess;
810+ for (size_t i = 0; i < 4; ++i) {
811+ const std::string& expected = rec.values[i];
812+ const std::string& val = rrec[i];
813+ mess += " " + val + "/" + expected;
814+ }
815+ if (arg.sh.verbose > 0) {
816+ fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
817+ "unexpected_read_value %s\n",
818+ arg.worker_type.c_str(), arg.id, k.c_str(), mess.c_str());
819+ }
820+ ret = 1;
821+ }
822+ }
823+ }
824+ if (arg.sh.verbose >= 100 && ret == 0) {
825+ fprintf(stderr, "%s %s\n", arg.worker_type.c_str(), k.c_str());
826+ }
827+ if (ret == 0 && !rec.unknown_state) {
828+ ++stat.success_count;
829+ }
830+ return ret;
831+}
832+
833+int
834+hs_longrun_thread_base::verify_readnolock(const std::string& k,
835+ uint32_t num_rows, uint32_t num_flds, const std::string rrec[4])
836+{
837+ int ret = 0;
838+ if (num_rows != 1 || num_flds != 4) {
839+ ++stat.verify_error_count;
840+ if (arg.sh.verbose > 0) {
841+ fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
842+ "unexpected_read_failure\n",
843+ arg.worker_type.c_str(), arg.id, k.c_str());
844+ }
845+ ret = 1;
846+ }
847+ if (arg.sh.verbose >= 100 && ret == 0) {
848+ fprintf(stderr, "%s -> %s %s %s %s %s\n", arg.worker_type.c_str(),
849+ k.c_str(), rrec[0].c_str(), rrec[1].c_str(), rrec[2].c_str(),
850+ rrec[3].c_str());
851+ }
852+ if (ret == 0) {
853+ ++stat.success_count;
854+ }
855+ return ret;
856+}
857+
858+struct hs_longrun_thread_hs : public hs_longrun_thread_base {
859+ hs_longrun_thread_hs(const arg_type& arg)
860+ : hs_longrun_thread_base(arg) { }
861+ void run();
862+ int check_hs_error(const char *mess, record_value *rec);
863+ int op_insert(record_value& rec);
864+ int op_delete(record_value& rec);
865+ int op_update(record_value& rec);
866+ int op_read(record_value& rec);
867+ int op_readnolock(int k);
868+ hstcpcli_ptr cli;
869+ socket_args sockargs;
870+};
871+
872+struct lock_guard : noncopyable {
873+ lock_guard(mutex& mtx) : mtx(mtx) {
874+ mtx.lock();
875+ }
876+ ~lock_guard() {
877+ mtx.unlock();
878+ }
879+ mutex& mtx;
880+};
881+
882+string_ref
883+to_string_ref(const std::string& s)
884+{
885+ return string_ref(s.data(), s.size());
886+}
887+
888+std::string
889+to_string(const string_ref& s)
890+{
891+ return std::string(s.begin(), s.size());
892+}
893+
894+void
895+hs_longrun_thread_hs::run()
896+{
897+ config c = arg.sh.conf;
898+ if (arg.op == 'R' || arg.op == 'N') {
899+ c["port"] = to_stdstring(arg.sh.conf.get_int("hsport", 9998));
900+ } else {
901+ c["port"] = to_stdstring(arg.sh.conf.get_int("hsport_wr", 9999));
902+ }
903+ sockargs.set(c);
904+
905+ while (arg.sh.running) {
906+ if (cli.get() == 0 || !cli->stable_point()) {
907+ cli = hstcpcli_i::create(sockargs);
908+ if (check_hs_error("connect", 0) != 0) {
909+ cli.reset();
910+ continue;
911+ }
912+ cli->request_buf_open_index(0, "hstestdb", "hstesttbl", "PRIMARY",
913+ "k,v1,v2,v3", "k,v1,v2,v3");
914+ cli->request_send();
915+ if (check_hs_error("openindex_send", 0) != 0) {
916+ cli.reset();
917+ continue;
918+ }
919+ size_t num_flds = 0;
920+ cli->response_recv(num_flds);
921+ if (check_hs_error("openindex_recv", 0) != 0) {
922+ cli.reset();
923+ continue;
924+ }
925+ cli->response_buf_remove();
926+ }
927+ const size_t rec_id = rand_record();
928+ if (arg.lock_flag) {
929+ record_value& rec = *arg.sh.records[rec_id];
930+ lock_guard g(rec.lock);
931+ int e = 0;
932+ switch (arg.op) {
933+ case 'I':
934+ e = op_insert(rec);
935+ break;
936+ case 'D':
937+ e = op_delete(rec);
938+ break;
939+ case 'U':
940+ e = op_update(rec);
941+ break;
942+ case 'R':
943+ e = op_read(rec);
944+ break;
945+ default:
946+ break;
947+ }
948+ } else {
949+ int e = 0;
950+ switch (arg.op) {
951+ case 'N':
952+ e = op_readnolock(rec_id);
953+ break;
954+ default:
955+ break;
956+ }
957+ }
958+ }
959+}
960+
961+int
962+hs_longrun_thread_hs::op_insert(record_value& rec)
963+{
964+ const std::string k = rec.key;
965+ const std::string v1 = "iv1_" + k + "_" + to_stdstring(arg.id);
966+ const std::string v2 = "iv2_" + k + "_" + to_stdstring(arg.id);
967+ const std::string v3 = "iv3_" + k + "_" + to_stdstring(arg.id);
968+ const string_ref op_ref("+", 1);
969+ const string_ref op_args[4] = {
970+ to_string_ref(k),
971+ to_string_ref(v1),
972+ to_string_ref(v2),
973+ to_string_ref(v3)
974+ };
975+ cli->request_buf_exec_generic(0, op_ref, op_args, 4, 1, 0,
976+ string_ref(), 0, 0, 0, 0);
977+ cli->request_send();
978+ if (check_hs_error("op_insert_send", &rec) != 0) { return 1; }
979+ size_t numflds = 0;
980+ cli->response_recv(numflds);
981+ if (arg.sh.verbose > 10) {
982+ const string_ref *row = cli->get_next_row();
983+ fprintf(stderr, "HS op=+ errrcode=%d errmess=[%s]\n", cli->get_error_code(),
984+ row ? to_string(row[0]).c_str() : "");
985+ }
986+ const bool op_success = cli->get_error_code() == 0;
987+ int ret = 0;
988+ if (!rec.unknown_state) {
989+ if (rec.deleted && !op_success) {
990+ ++stat.verify_error_count;
991+ if (arg.sh.verbose > 0) {
992+ fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
993+ "unexpected_insert_failure\n",
994+ arg.worker_type.c_str(), arg.id, k.c_str());
995+ }
996+ ret = 1;
997+ } else if (!rec.deleted && op_success) {
998+ ++stat.verify_error_count;
999+ if (arg.sh.verbose > 0) {
1000+ fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
1001+ "unexpected_insert_success\n",
1002+ arg.worker_type.c_str(), arg.id, k.c_str());
1003+ }
1004+ ret = 1;
1005+ }
1006+ } else {
1007+ ++stat.unknown_count;
1008+ }
1009+ if (op_success) {
1010+ rec.values.resize(4);
1011+ rec.values[0] = k;
1012+ rec.values[1] = v1;
1013+ rec.values[2] = v2;
1014+ rec.values[3] = v3;
1015+ rec.deleted = false;
1016+ if (arg.sh.verbose >= 100 && ret == 0) {
1017+ fprintf(stderr, "HS_INSERT %s %s %s %s\n", k.c_str(), v1.c_str(),
1018+ v2.c_str(), v3.c_str());
1019+ }
1020+ if (ret == 0 && !rec.unknown_state) {
1021+ ++stat.success_count;
1022+ }
1023+ rec.unknown_state = false;
1024+ }
1025+ cli->response_buf_remove();
1026+ return ret;
1027+}
1028+
1029+int
1030+hs_longrun_thread_hs::op_delete(record_value& rec)
1031+{
1032+ const std::string k = rec.key;
1033+ const string_ref op_ref("=", 1);
1034+ const string_ref op_args[1] = {
1035+ to_string_ref(k),
1036+ };
1037+ const string_ref modop_ref("D", 1);
1038+ cli->request_buf_exec_generic(0, op_ref, op_args, 1, 1, 0,
1039+ modop_ref, 0, 0, 0, 0);
1040+ cli->request_send();
1041+ if (check_hs_error("op_delete_send", &rec) != 0) { return 1; }
1042+ size_t numflds = 0;
1043+ cli->response_recv(numflds);
1044+ if (check_hs_error("op_delete_recv", &rec) != 0) { return 1; }
1045+ const string_ref *row = cli->get_next_row();
1046+ const bool op_success = (numflds > 0 && row != 0 &&
1047+ to_string(row[0]) == "1");
1048+ int ret = 0;
1049+ if (!rec.unknown_state) {
1050+ if (!rec.deleted && !op_success) {
1051+ ++stat.verify_error_count;
1052+ if (arg.sh.verbose > 0) {
1053+ fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
1054+ "unexpected_delete_failure\n",
1055+ arg.worker_type.c_str(), arg.id, k.c_str());
1056+ }
1057+ ret = 1;
1058+ } else if (rec.deleted && op_success) {
1059+ ++stat.verify_error_count;
1060+ if (arg.sh.verbose > 0) {
1061+ fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
1062+ "unexpected_delete_success\n",
1063+ arg.worker_type.c_str(), arg.id, k.c_str());
1064+ }
1065+ ret = 1;
1066+ }
1067+ }
1068+ cli->response_buf_remove();
1069+ if (op_success) {
1070+ rec.deleted = true;
1071+ if (ret == 0 && !rec.unknown_state) {
1072+ ++stat.success_count;
1073+ }
1074+ rec.unknown_state = false;
1075+ }
1076+ if (arg.sh.verbose >= 100 && ret == 0) {
1077+ fprintf(stderr, "HS_DELETE %s\n", k.c_str());
1078+ }
1079+ return ret;
1080+}
1081+
1082+int
1083+hs_longrun_thread_hs::op_update(record_value& rec)
1084+{
1085+ const std::string k = rec.key;
1086+ const std::string v1 = "uv1_" + k + "_" + to_stdstring(arg.id);
1087+ const std::string v2 = "uv2_" + k + "_" + to_stdstring(arg.id);
1088+ const std::string v3 = "uv3_" + k + "_" + to_stdstring(arg.id);
1089+ const string_ref op_ref("=", 1);
1090+ const string_ref op_args[1] = {
1091+ to_string_ref(k),
1092+ };
1093+ const string_ref modop_ref("U", 1);
1094+ const string_ref modop_args[4] = {
1095+ to_string_ref(k),
1096+ to_string_ref(v1),
1097+ to_string_ref(v2),
1098+ to_string_ref(v3)
1099+ };
1100+ cli->request_buf_exec_generic(0, op_ref, op_args, 1, 1, 0,
1101+ modop_ref, modop_args, 4, 0, 0);
1102+ cli->request_send();
1103+ if (check_hs_error("op_update_send", &rec) != 0) { return 1; }
1104+ size_t numflds = 0;
1105+ cli->response_recv(numflds);
1106+ if (check_hs_error("op_update_recv", &rec) != 0) { return 1; }
1107+ const string_ref *row = cli->get_next_row();
1108+ uint32_t num_rows = row
1109+ ? atoi_uint32_nocheck(row[0].begin(), row[0].end()) : 0;
1110+ cli->response_buf_remove();
1111+ const bool cur_unknown_state = (num_rows == 1);
1112+ return verify_update(k, v1, v2, v3, rec, num_rows, cur_unknown_state);
1113+}
1114+
1115+int
1116+hs_longrun_thread_hs::op_read(record_value& rec)
1117+{
1118+ const std::string k = rec.key;
1119+ const string_ref op_ref("=", 1);
1120+ const string_ref op_args[1] = {
1121+ to_string_ref(k),
1122+ };
1123+ cli->request_buf_exec_generic(0, op_ref, op_args, 1, 1, 0,
1124+ string_ref(), 0, 0, 0, 0);
1125+ cli->request_send();
1126+ if (check_hs_error("op_read_send", 0) != 0) { return 1; }
1127+ size_t num_flds = 0;
1128+ size_t num_rows = 0;
1129+ cli->response_recv(num_flds);
1130+ if (check_hs_error("op_read_recv", 0) != 0) { return 1; }
1131+ const string_ref *row = cli->get_next_row();
1132+ std::string rrec[4];
1133+ if (row != 0 && num_flds == 4) {
1134+ for (int i = 0; i < 4; ++i) {
1135+ rrec[i] = to_string(row[i]);
1136+ }
1137+ ++num_rows;
1138+ }
1139+ row = cli->get_next_row();
1140+ if (row != 0) {
1141+ ++num_rows;
1142+ }
1143+ cli->response_buf_remove();
1144+ return verify_read(k, num_rows, num_flds, rrec, rec);
1145+}
1146+
1147+int
1148+hs_longrun_thread_hs::op_readnolock(int key)
1149+{
1150+ const std::string k = to_stdstring(key);
1151+ const string_ref op_ref("=", 1);
1152+ const string_ref op_args[1] = {
1153+ to_string_ref(k),
1154+ };
1155+ cli->request_buf_exec_generic(0, op_ref, op_args, 1, 1, 0,
1156+ string_ref(), 0, 0, 0, 0);
1157+ cli->request_send();
1158+ if (check_hs_error("op_read_send", 0) != 0) { return 1; }
1159+ size_t num_flds = 0;
1160+ size_t num_rows = 0;
1161+ cli->response_recv(num_flds);
1162+ if (check_hs_error("op_read_recv", 0) != 0) { return 1; }
1163+ const string_ref *row = cli->get_next_row();
1164+ std::string rrec[4];
1165+ if (row != 0 && num_flds == 4) {
1166+ for (int i = 0; i < 4; ++i) {
1167+ rrec[i] = to_string(row[i]);
1168+ }
1169+ ++num_rows;
1170+ }
1171+ row = cli->get_next_row();
1172+ if (row != 0) {
1173+ ++num_rows;
1174+ }
1175+ cli->response_buf_remove();
1176+ return verify_readnolock(k, num_rows, num_flds, rrec);
1177+}
1178+
1179+int
1180+hs_longrun_thread_hs::check_hs_error(const char *mess, record_value *rec)
1181+{
1182+ const int err = cli->get_error_code();
1183+ if (err == 0) {
1184+ return 0;
1185+ }
1186+ ++stat.runtime_error_count;
1187+ if (arg.sh.verbose > 0) {
1188+ const std::string estr = cli->get_error();
1189+ fprintf(stderr, "RUNTIME_ERROR: op=%c wid=%d %s: %d %s\n",
1190+ arg.op, arg.id, mess, err, estr.c_str());
1191+ }
1192+ if (rec) {
1193+ rec->unknown_state = true;
1194+ }
1195+ return 1;
1196+}
1197+
1198+struct hs_longrun_thread_my : public hs_longrun_thread_base {
1199+ hs_longrun_thread_my(const arg_type& arg)
1200+ : hs_longrun_thread_base(arg), connected(false) { }
1201+ void run();
1202+ void show_mysql_error(const char *mess, record_value *rec);
1203+ int op_insert(record_value& rec);
1204+ int op_delete(record_value& rec);
1205+ int op_update(record_value& rec);
1206+ int op_delins(record_value& rec);
1207+ int op_read(record_value& rec);
1208+ auto_mysql db;
1209+ bool connected;
1210+};
1211+
1212+void
1213+hs_longrun_thread_my::run()
1214+{
1215+ const std::string mysql_host = arg.sh.conf.get_str("host", "localhost");
1216+ const std::string mysql_user = arg.sh.conf.get_str("mysqluser", "root");
1217+ const std::string mysql_passwd = arg.sh.conf.get_str("mysqlpass", "");
1218+ const std::string mysql_dbname = "hstestdb";
1219+
1220+ while (arg.sh.running) {
1221+ if (!connected) {
1222+ if (!mysql_real_connect(db, mysql_host.c_str(), mysql_user.c_str(),
1223+ mysql_passwd.c_str(), mysql_dbname.c_str(), mysql_port, 0, 0)) {
1224+ show_mysql_error("mysql_real_connect", 0);
1225+ continue;
1226+ }
1227+ }
1228+ connected = true;
1229+ const size_t rec_id = rand_record();
1230+ record_value& rec = *arg.sh.records[rec_id];
1231+ lock_guard g(rec.lock);
1232+ int e = 0;
1233+ switch (arg.op) {
1234+ #if 0
1235+ case 'I':
1236+ e = op_insert(rec);
1237+ break;
1238+ case 'D':
1239+ e = op_delete(rec);
1240+ break;
1241+ case 'U':
1242+ e = op_update(rec);
1243+ break;
1244+ #endif
1245+ case 'T':
1246+ e = op_delins(rec);
1247+ break;
1248+ case 'R':
1249+ e = op_read(rec);
1250+ break;
1251+ default:
1252+ break;
1253+ }
1254+ }
1255+}
1256+
1257+int
1258+hs_longrun_thread_my::op_delins(record_value& rec)
1259+{
1260+ const std::string k = rec.key;
1261+ const std::string v1 = "div1_" + k + "_" + to_stdstring(arg.id);
1262+ const std::string v2 = "div2_" + k + "_" + to_stdstring(arg.id);
1263+ const std::string v3 = "div3_" + k + "_" + to_stdstring(arg.id);
1264+ int success = 0;
1265+ bool cur_unknown_state = false;
1266+ do {
1267+ char query[1024];
1268+ #if 1
1269+ if (mysql_query(db, "begin") != 0) {
1270+ if (arg.sh.verbose >= 20) {
1271+ fprintf(stderr, "mysql: e=[%s] q=[%s]\n", mysql_error(db), "begin");
1272+ }
1273+ break;
1274+ }
1275+ #endif
1276+ cur_unknown_state = true;
1277+ snprintf(query, 1024,
1278+ "delete from hstesttbl where k = '%s'", k.c_str());
1279+ if (mysql_query(db, query) != 0) {
1280+ if (arg.sh.verbose >= 20) {
1281+ fprintf(stderr, "mysql: e=[%s] q=[%s]\n", mysql_error(db), query);
1282+ }
1283+ break;
1284+ }
1285+ if (mysql_affected_rows(db) != 1) {
1286+ if (arg.sh.verbose >= 20) {
1287+ fprintf(stderr, "mysql: notfound: [%s]\n", query);
1288+ }
1289+ break;
1290+ }
1291+ snprintf(query, 1024,
1292+ "insert into hstesttbl values ('%s', '%s', '%s', '%s')",
1293+ k.c_str(), v1.c_str(), v2.c_str(), v3.c_str());
1294+ if (mysql_query(db, query) != 0) {
1295+ if (arg.sh.verbose >= 20) {
1296+ fprintf(stderr, "mysql: e=[%s] q=[%s]\n", mysql_error(db), query);
1297+ }
1298+ break;
1299+ }
1300+ #if 1
1301+ if (mysql_query(db, "commit") != 0) {
1302+ if (arg.sh.verbose >= 20) {
1303+ fprintf(stderr, "mysql: e=[%s] q=[%s]\n", mysql_error(db), "commit");
1304+ }
1305+ break;
1306+ }
1307+ #endif
1308+ success = true;
1309+ cur_unknown_state = false;
1310+ } while (false);
1311+ return verify_update(k, v1, v2, v3, rec, (success != 0), cur_unknown_state);
1312+}
1313+
1314+int
1315+hs_longrun_thread_my::op_read(record_value& rec)
1316+{
1317+ const std::string k = rec.key;
1318+ char query[1024] = { 0 };
1319+ const int len = snprintf(query, 1024,
1320+ "select k,v1,v2,v3 from hstesttbl where k='%s'", k.c_str());
1321+ const int r = mysql_real_query(db, query, len > 0 ? len : 0);
1322+ if (r != 0) {
1323+ show_mysql_error(query, 0);
1324+ return 1;
1325+ }
1326+ MYSQL_ROW row = 0;
1327+ unsigned long *lengths = 0;
1328+ unsigned int num_rows = 0;
1329+ unsigned int num_flds = 0;
1330+ auto_mysql_res res(db);
1331+ std::string rrec[4];
1332+ if (res != 0) {
1333+ num_flds = mysql_num_fields(res);
1334+ row = mysql_fetch_row(res);
1335+ if (row != 0) {
1336+ lengths = mysql_fetch_lengths(res);
1337+ if (num_flds == 4) {
1338+ for (int i = 0; i < 4; ++i) {
1339+ rrec[i] = std::string(row[i], lengths[i]);
1340+ }
1341+ }
1342+ ++num_rows;
1343+ row = mysql_fetch_row(res);
1344+ if (row != 0) {
1345+ ++num_rows;
1346+ }
1347+ }
1348+ }
1349+ return verify_read(k, num_rows, num_flds, rrec, rec);
1350+}
1351+
1352+void
1353+hs_longrun_thread_my::show_mysql_error(const char *mess, record_value *rec)
1354+{
1355+ ++stat.runtime_error_count;
1356+ if (arg.sh.verbose > 0) {
1357+ fprintf(stderr, "RUNTIME_ERROR: op=%c wid=%d [%s]: %s\n",
1358+ arg.op, arg.id, mess, mysql_error(db));
1359+ }
1360+ if (rec) {
1361+ rec->unknown_state = true;
1362+ }
1363+ db.reset();
1364+ connected = false;
1365+}
1366+
1367+void
1368+mysql_do(MYSQL *db, const char *query)
1369+{
1370+ if (mysql_real_query(db, query, strlen(query)) != 0) {
1371+ fprintf(stderr, "mysql: e=[%s] q=[%s]\n", mysql_error(db), query);
1372+ fatal_exit("mysql_do");
1373+ }
1374+}
1375+
1376+void
1377+hs_longrun_init_table(const config& conf, int num_prepare,
1378+ hs_longrun_shared& shared)
1379+{
1380+ const std::string mysql_host = conf.get_str("host", "localhost");
1381+ const std::string mysql_user = conf.get_str("mysqluser", "root");
1382+ const std::string mysql_passwd = conf.get_str("mysqlpass", "");
1383+ const std::string mysql_dbname = "";
1384+ auto_mysql db;
1385+ if (!mysql_real_connect(db, mysql_host.c_str(), mysql_user.c_str(),
1386+ mysql_passwd.c_str(), mysql_dbname.c_str(), mysql_port, 0, 0)) {
1387+ fprintf(stderr, "mysql: error=[%s]\n", mysql_error(db));
1388+ fatal_exit("hs_longrun_init_table");
1389+ }
1390+ mysql_do(db, "drop database if exists hstestdb");
1391+ mysql_do(db, "create database hstestdb");
1392+ mysql_do(db, "use hstestdb");
1393+ mysql_do(db,
1394+ "create table hstesttbl ("
1395+ "k int primary key,"
1396+ "v1 varchar(32) not null,"
1397+ "v2 varchar(32) not null,"
1398+ "v3 varchar(32) not null"
1399+ ") character set utf8 collate utf8_bin engine = innodb");
1400+ for (int i = 0; i < num_prepare; ++i) {
1401+ const std::string i_str = to_stdstring(i);
1402+ const std::string v1 = "pv1_" + i_str;
1403+ const std::string v2 = "pv2_" + i_str;
1404+ const std::string v3 = "pv3_" + i_str;
1405+ char buf[1024];
1406+ snprintf(buf, 1024, "insert into hstesttbl(k, v1, v2, v3) values"
1407+ "(%d, '%s', '%s', '%s')", i, v1.c_str(), v2.c_str(), v3.c_str());
1408+ mysql_do(db, buf);
1409+ record_value *rec = shared.records[i];
1410+ rec->key = i_str;
1411+ rec->values.resize(4);
1412+ rec->values[0] = i_str;
1413+ rec->values[1] = v1;
1414+ rec->values[2] = v2;
1415+ rec->values[3] = v3;
1416+ rec->deleted = false;
1417+ }
1418+}
1419+
1420+int
1421+hs_longrun_main(int argc, char **argv)
1422+{
1423+ hs_longrun_shared shared;
1424+ parse_args(argc, argv, shared.conf);
1425+ shared.conf["host"] = shared.conf.get_str("host", "localhost");
1426+ shared.verbose = shared.conf.get_int("verbose", 1);
1427+ const int table_size = shared.conf.get_int("table_size", 10000);
1428+ for (int i = 0; i < table_size; ++i) {
1429+ std::auto_ptr<record_value> rec(new record_value());
1430+ rec->key = to_stdstring(i);
1431+ shared.records.push_back_ptr(rec);
1432+ }
1433+ mysql_library_init(0, 0, 0);
1434+ const int duration = shared.conf.get_int("duration", 10);
1435+ const int num_hsinsert = shared.conf.get_int("num_hsinsert", 10);
1436+ const int num_hsdelete = shared.conf.get_int("num_hsdelete", 10);
1437+ const int num_hsupdate = shared.conf.get_int("num_hsupdate", 10);
1438+ const int num_hsread = shared.conf.get_int("num_hsread", 10);
1439+ const int num_myread = shared.conf.get_int("num_myread", 10);
1440+ const int num_mydelins = shared.conf.get_int("num_mydelins", 10);
1441+ int num_hsreadnolock = shared.conf.get_int("num_hsreadnolock", 10);
1442+ const bool always_filled = (num_hsinsert == 0 && num_hsdelete == 0);
1443+ if (!always_filled) {
1444+ num_hsreadnolock = 0;
1445+ }
1446+ hs_longrun_init_table(shared.conf, always_filled ? table_size : 0,
1447+ shared);
1448+ /* create worker threads */
1449+ static const struct thrtmpl_type {
1450+ const char *type; char op; int num; int hs; int lock;
1451+ } thrtmpl[] = {
1452+ { "hsinsert", 'I', num_hsinsert, 1, 1 },
1453+ { "hsdelete", 'D', num_hsdelete, 1, 1 },
1454+ { "hsupdate", 'U', num_hsupdate, 1, 1 },
1455+ { "hsread", 'R', num_hsread, 1, 1 },
1456+ { "hsreadnolock", 'N', num_hsreadnolock, 1, 0 },
1457+ { "myread", 'R', num_myread, 0, 1 },
1458+ { "mydelins", 'T', num_mydelins, 0, 1 },
1459+ };
1460+ typedef auto_ptrcontainer< std::vector<hs_longrun_thread_base *> > thrs_type;
1461+ thrs_type thrs;
1462+ for (size_t i = 0; i < sizeof(thrtmpl)/sizeof(thrtmpl[0]); ++i) {
1463+ const thrtmpl_type& e = thrtmpl[i];
1464+ for (int j = 0; j < e.num; ++j) {
1465+ int id = thrs.size();
1466+ const hs_longrun_thread_hs::arg_type arg(id, e.type, e.op, e.lock,
1467+ shared);
1468+ std::auto_ptr<hs_longrun_thread_base> thr;
1469+ if (e.hs) {
1470+ thr.reset(new hs_longrun_thread_hs(arg));
1471+ } else {
1472+ thr.reset(new hs_longrun_thread_my(arg));
1473+ }
1474+ thrs.push_back_ptr(thr);
1475+ }
1476+ }
1477+ shared.num_threads = thrs.size();
1478+ /* start threads */
1479+ fprintf(stderr, "START\n");
1480+ shared.running = 1;
1481+ for (size_t i = 0; i < thrs.size(); ++i) {
1482+ thrs[i]->start();
1483+ }
1484+ /* wait */
1485+ sleep(duration);
1486+ /* stop thread */
1487+ shared.running = 0;
1488+ for (size_t i = 0; i < thrs.size(); ++i) {
1489+ thrs[i]->join();
1490+ }
1491+ fprintf(stderr, "DONE\n");
1492+ /* summary */
1493+ typedef std::map<std::string, hs_longrun_stat> stat_map;
1494+ stat_map sm;
1495+ for (size_t i = 0; i < thrs.size(); ++i) {
1496+ hs_longrun_thread_base *const thr = thrs[i];
1497+ const std::string wt = thr->arg.worker_type;
1498+ hs_longrun_stat& v = sm[wt];
1499+ v.add(thr->stat);
1500+ }
1501+ hs_longrun_stat total;
1502+ for (stat_map::const_iterator i = sm.begin(); i != sm.end(); ++i) {
1503+ if (i->second.verify_error_count != 0) {
1504+ fprintf(stderr, "%s verify_error %llu\n", i->first.c_str(),
1505+ i->second.verify_error_count);
1506+ }
1507+ if (i->second.runtime_error_count) {
1508+ fprintf(stderr, "%s runtime_error %llu\n", i->first.c_str(),
1509+ i->second.runtime_error_count);
1510+ }
1511+ if (i->second.unknown_count) {
1512+ fprintf(stderr, "%s unknown %llu\n", i->first.c_str(),
1513+ i->second.unknown_count);
1514+ }
1515+ fprintf(stderr, "%s success %llu\n", i->first.c_str(),
1516+ i->second.success_count);
1517+ total.add(i->second);
1518+ }
1519+ if (total.verify_error_count != 0) {
1520+ fprintf(stderr, "TOTAL verify_error %llu\n", total.verify_error_count);
1521+ }
1522+ if (total.runtime_error_count != 0) {
1523+ fprintf(stderr, "TOTAL runtime_error %llu\n", total.runtime_error_count);
1524+ }
1525+ if (total.unknown_count != 0) {
1526+ fprintf(stderr, "TOTAL unknown %llu\n", total.unknown_count);
1527+ }
1528+ fprintf(stderr, "TOTAL success %llu\n", total.success_count);
1529+ mysql_library_end();
1530+ return 0;
1531+}
1532+
1533+};
1534+
1535+int
1536+main(int argc, char **argv)
1537+{
1538+ return dena::hs_longrun_main(argc, argv);
1539+}
1540+
1541
1542=== added file 'HandlerSocket-Plugin-for-MySQL/client/hspool_test.pl'
1543--- HandlerSocket-Plugin-for-MySQL/client/hspool_test.pl 1970-01-01 00:00:00 +0000
1544+++ HandlerSocket-Plugin-for-MySQL/client/hspool_test.pl 2011-06-08 01:57:28 +0000
1545@@ -0,0 +1,224 @@
1546+#!/usr/bin/perl
1547+
1548+use strict;
1549+use warnings;
1550+use DB::HandlerSocket::Pool;
1551+use DBI;
1552+
1553+my %conf = ();
1554+for my $i (@ARGV) {
1555+ my ($k, $v) = split(/=/, $i);
1556+ $conf{$k} = $v;
1557+}
1558+
1559+my $verbose = get_conf("verbose", 0);
1560+my $actions_str = get_conf("actions",
1561+ "create,insert,verify,verify2,verify3,verify4,clean");
1562+my $tablesize = get_conf("tablesize", 1000);
1563+my $db = get_conf("db", "hstestdb");
1564+my $table = get_conf("table", "testtbl");
1565+my $table_schema = get_conf("table_schema", undef);
1566+my $engine = get_conf("engine", "innodb");
1567+my $host = get_conf("host", "localhost");
1568+my $mysqlport = get_conf("mysqlport", 3306);
1569+my $hsport_rd = get_conf("hsport_rd", 9998);
1570+my $hsport_wr = get_conf("hsport_wr", 9999);
1571+my $loop = get_conf("loop", 10000);
1572+my $op = get_conf("op", "=");
1573+my $ssps = get_conf("ssps", 0);
1574+my $num_moreflds = get_conf("moreflds", 0);
1575+my $moreflds_prefix = get_conf("moreflds_prefix", "f");
1576+my $mysql_user = 'root';
1577+my $mysql_password = '';
1578+
1579+my $dsn = "DBI:mysql:database=;host=$host;port=$mysqlport"
1580+ . ";mysql_server_prepare=$ssps";
1581+my $dbh = DBI->connect($dsn, $mysql_user, $mysql_password,
1582+ { RaiseError => 1 });
1583+my $hsargs = { 'host' => $host, 'port' => $hsport_rd };
1584+my $hspool = new DB::HandlerSocket::Pool({
1585+ hostmap => {
1586+ "$db.$table" => {
1587+ host => $host,
1588+ port => $hsport_rd,
1589+ },
1590+ },
1591+ resolve => undef,
1592+ error => undef,
1593+});
1594+$table_schema = "(k int primary key, fc30 varchar(30), ft text)"
1595+ if (!defined($table_schema));
1596+
1597+my @actions = split(/,/, $actions_str);
1598+for my $action (@actions) {
1599+ print "ACTION: $action\n";
1600+ eval "hstest_$action()";
1601+ if ($@) {
1602+ die $@;
1603+ }
1604+ print "ACTION: $action DONE\n";
1605+}
1606+
1607+sub get_conf {
1608+ my ($key, $def) = @_;
1609+ my $val = $conf{$key};
1610+ if ($val) {
1611+ print "$key=$val\n";
1612+ } else {
1613+ $val = $def;
1614+ my $defstr = $def || "(undef)";
1615+ print "$key=$defstr(default)\n";
1616+ }
1617+ return $val;
1618+}
1619+
1620+sub hstest_create {
1621+ $dbh->do("drop database if exists $db");
1622+ $dbh->do("create database $db");
1623+ $dbh->do("use $db");
1624+ $dbh->do("create table $table $table_schema engine=$engine");
1625+}
1626+
1627+sub hstest_dump {
1628+ $dbh->do("use $db");
1629+ my $sth = $dbh->prepare("select * from $table");
1630+ $sth->execute();
1631+ my $arr = $sth->fetchall_arrayref();
1632+ for my $rec (@$arr) {
1633+ print "REC:";
1634+ for my $row (@$rec) {
1635+ print " $row";
1636+ }
1637+ print "\n";
1638+ }
1639+}
1640+
1641+sub hstest_insert {
1642+ $dbh->do("use $db");
1643+ my $sth = $dbh->prepare("insert into $table values (?, ?, ?)");
1644+ for (my $k = 0; $k < $tablesize; ++$k) {
1645+ my $fc30 = "fc30_$k";
1646+ my $ft = "ft_$k";
1647+ $sth->execute($k, $fc30, $ft);
1648+ }
1649+}
1650+
1651+sub hstest_verify {
1652+ $dbh->do("use $db");
1653+ my $sth = $dbh->prepare("select * from $table order by k");
1654+ $sth->execute();
1655+ my $arr = $sth->fetchall_arrayref();
1656+ my $hsres = $hspool->index_find($db, $table, "PRIMARY", "k,fc30,ft",
1657+ ">=", [ 0 ], $tablesize, 0);
1658+ for (my $i = 0; $i < $tablesize; ++$i) {
1659+ my $rec = $arr->[$i];
1660+ my $differ = 0;
1661+ print "REC:" if $verbose;
1662+ for (my $j = 0; $j < 3; ++$j) {
1663+ my $fld = $rec->[$j];
1664+ my $hsidx = $i * 3 + $j;
1665+ my $hsfld = $hsres->[$hsidx];
1666+ if ($hsfld ne $fld) {
1667+ $differ = 1;
1668+ }
1669+ if ($differ) {
1670+ print " $fld:$hsfld" if $verbose;
1671+ } else {
1672+ print " $hsfld" if $verbose;
1673+ }
1674+ }
1675+ print "\n" if $verbose;
1676+ if ($differ) {
1677+ die "verification failed";
1678+ }
1679+ }
1680+}
1681+
1682+sub hstest_verify2 {
1683+ $dbh->do("use $db");
1684+ my $sth = $dbh->prepare("select * from $table order by k");
1685+ $sth->execute();
1686+ my $arr = $sth->fetchall_arrayref();
1687+ my $hsresa = $hspool->index_find_multi($db, $table, "PRIMARY",
1688+ "k,fc30,ft", [ [ -1, ">=", [ 0 ], $tablesize, 0 ] ]);
1689+ my $hsres = $hsresa->[0];
1690+ for (my $i = 0; $i < $tablesize; ++$i) {
1691+ my $rec = $arr->[$i];
1692+ my $differ = 0;
1693+ print "REC:" if $verbose;
1694+ for (my $j = 0; $j < 3; ++$j) {
1695+ my $fld = $rec->[$j];
1696+ my $hsidx = $i * 3 + $j;
1697+ my $hsfld = $hsres->[$hsidx];
1698+ if ($hsfld ne $fld) {
1699+ $differ = 1;
1700+ }
1701+ if ($differ) {
1702+ print " $fld:$hsfld" if $verbose;
1703+ } else {
1704+ print " $hsfld" if $verbose;
1705+ }
1706+ }
1707+ print "\n" if $verbose;
1708+ if ($differ) {
1709+ die "verification failed";
1710+ }
1711+ }
1712+}
1713+
1714+sub hashref_to_str {
1715+ my $href = $_[0];
1716+ my $r = '';
1717+ for my $k (sort keys %$href) {
1718+ my $v = $href->{$k};
1719+ $r .= "($k=>$v)";
1720+ }
1721+ return $r;
1722+}
1723+
1724+sub hstest_verify3 {
1725+ $dbh->do("use $db");
1726+ my $sth = $dbh->prepare("select * from $table order by k");
1727+ $sth->execute();
1728+ my $hsres_t = $hspool->index_find($db, $table, "PRIMARY", "k,fc30,ft",
1729+ ">=", [ 0 ], $tablesize, 0);
1730+ my $hsres = DB::HandlerSocket::Pool::result_single_to_hasharr(
1731+ [ 'k', 'fc30', 'ft' ], $hsres_t);
1732+ for (my $i = 0; $i < $tablesize; ++$i) {
1733+ my $mystr = hashref_to_str($sth->fetchrow_hashref());
1734+ my $hsstr = hashref_to_str($hsres->[$i]);
1735+ if ($mystr ne $hsstr) {
1736+ print "DIFF my=[$mystr] hs=[$hsstr]\n" if $verbose;
1737+ die "verification failed";
1738+ } else {
1739+ print "OK $hsstr\n" if $verbose;
1740+ }
1741+ }
1742+}
1743+
1744+sub hstest_verify4 {
1745+ $dbh->do("use $db");
1746+ my $sth = $dbh->prepare("select * from $table order by k");
1747+ $sth->execute();
1748+ my $hsres_t = $hspool->index_find($db, $table, "PRIMARY", "k,fc30,ft",
1749+ ">=", [ 0 ], $tablesize, 0);
1750+ my $hsres = DB::HandlerSocket::Pool::result_single_to_hashhash(
1751+ [ 'k', 'fc30', 'ft' ], 'k', $hsres_t);
1752+ my $rechash = $sth->fetchall_hashref('k');
1753+ while (my ($k, $href) = each (%$rechash)) {
1754+ my $mystr = hashref_to_str($href);
1755+ my $hsstr = hashref_to_str($hsres->{$k});
1756+ if ($mystr ne $hsstr) {
1757+ print "DIFF my=[$mystr] hs=[$hsstr]\n" if $verbose;
1758+ die "verification failed";
1759+ } else {
1760+ print "OK $hsstr\n" if $verbose;
1761+ }
1762+ }
1763+}
1764+
1765+sub hstest_clean {
1766+ $hspool->clear_pool();
1767+ $dbh->do("drop database if exists $db");
1768+}
1769+
1770
1771=== added file 'HandlerSocket-Plugin-for-MySQL/client/hstest.cpp'
1772--- HandlerSocket-Plugin-for-MySQL/client/hstest.cpp 1970-01-01 00:00:00 +0000
1773+++ HandlerSocket-Plugin-for-MySQL/client/hstest.cpp 2011-06-08 01:57:28 +0000
1774@@ -0,0 +1,1532 @@
1775+
1776+// vim:sw=2:ai
1777+
1778+#include <signal.h>
1779+#include <sys/time.h>
1780+#include <stdio.h>
1781+#include <string.h>
1782+#include <vector>
1783+#include <stdlib.h>
1784+#include <memory>
1785+#include <errno.h>
1786+#include <mysql.h>
1787+#include <time.h>
1788+#include <sys/types.h>
1789+#include <sys/stat.h>
1790+#include <fcntl.h>
1791+
1792+#include "util.hpp"
1793+#include "auto_ptrcontainer.hpp"
1794+#include "socket.hpp"
1795+#include "thread.hpp"
1796+#include "hstcpcli.hpp"
1797+
1798+#if __GNUC__ >= 4
1799+long atomic_exchange_and_add(volatile long *valp, long c)
1800+{
1801+ return __sync_fetch_and_add(valp, c);
1802+}
1803+#else
1804+#include <bits/atomicity.h>
1805+using namespace __gnu_cxx;
1806+long atomic_exchange_and_add(volatile long *valp, long c)
1807+{
1808+ return __exchange_and_add((volatile _Atomic_word *)valp, c);
1809+}
1810+#endif
1811+
1812+namespace dena {
1813+
1814+struct auto_mysql : private noncopyable {
1815+ auto_mysql() : db(0) {
1816+ reset();
1817+ }
1818+ ~auto_mysql() {
1819+ if (db) {
1820+ mysql_close(db);
1821+ }
1822+ }
1823+ void reset() {
1824+ if (db) {
1825+ mysql_close(db);
1826+ }
1827+ if ((db = mysql_init(0)) == 0) {
1828+ fatal_abort("failed to initialize mysql client");
1829+ }
1830+ }
1831+ operator MYSQL *() const { return db; }
1832+ private:
1833+ MYSQL *db;
1834+};
1835+
1836+struct auto_mysql_res : private noncopyable {
1837+ auto_mysql_res(MYSQL *db) {
1838+ res = mysql_store_result(db);
1839+ }
1840+ ~auto_mysql_res() {
1841+ if (res) {
1842+ mysql_free_result(res);
1843+ }
1844+ }
1845+ operator MYSQL_RES *() const { return res; }
1846+ private:
1847+ MYSQL_RES *res;
1848+};
1849+
1850+struct auto_mysql_stmt : private noncopyable {
1851+ auto_mysql_stmt(MYSQL *db) {
1852+ stmt = mysql_stmt_init(db);
1853+ }
1854+ ~auto_mysql_stmt() {
1855+ if (stmt) {
1856+ mysql_stmt_close(stmt);
1857+ }
1858+ }
1859+ operator MYSQL_STMT *() const { return stmt; }
1860+ private:
1861+ MYSQL_STMT *stmt;
1862+};
1863+
1864+namespace {
1865+
1866+double
1867+gettimeofday_double()
1868+{
1869+ struct timeval tv = { };
1870+ if (gettimeofday(&tv, 0) != 0) {
1871+ fatal_abort("gettimeofday");
1872+ }
1873+ return static_cast<double>(tv.tv_usec) / 1000000 + tv.tv_sec;
1874+}
1875+
1876+// unused
1877+void
1878+wait_close(int fd)
1879+{
1880+ char buf[1024];
1881+ while (true) {
1882+ int r = read(fd, buf, sizeof(buf));
1883+ if (r <= 0) {
1884+ break;
1885+ }
1886+ }
1887+}
1888+
1889+// unused
1890+void
1891+gentle_close(int fd)
1892+{
1893+ int r = shutdown(fd, SHUT_WR);
1894+ if (r != 0) {
1895+ return;
1896+ }
1897+ wait_close(fd);
1898+}
1899+
1900+};
1901+
1902+struct hstest_shared {
1903+ config conf;
1904+ socket_args arg;
1905+ int verbose;
1906+ size_t loop;
1907+ size_t pipe;
1908+ char op;
1909+ long num_threads;
1910+ mutable volatile long count;
1911+ mutable volatile long conn_count;
1912+ long wait_conn;
1913+ volatile char *keygen;
1914+ long keygen_size;
1915+ mutable volatile int enable_timing;
1916+ int usleep;
1917+ int dump;
1918+ hstest_shared() : verbose(0), loop(0), pipe(0), op('G'), num_threads(0),
1919+ count(0), conn_count(0), wait_conn(0), keygen(0), keygen_size(0),
1920+ enable_timing(0), usleep(0), dump(0) { }
1921+ void increment_count(unsigned int c = 1) const volatile {
1922+ atomic_exchange_and_add(&count, c);
1923+ }
1924+ void increment_conn(unsigned int c) const volatile {
1925+ atomic_exchange_and_add(&conn_count, c);
1926+ while (wait_conn != 0 && conn_count < wait_conn) {
1927+ sleep(1);
1928+ }
1929+ // fprintf(stderr, "wait_conn=%ld done\n", wait_conn);
1930+ }
1931+};
1932+
1933+struct hstest_thread {
1934+ struct arg_type {
1935+ size_t id;
1936+ const hstest_shared& sh;
1937+ bool watch_flag;
1938+ arg_type(size_t i, const hstest_shared& s, bool w)
1939+ : id(i), sh(s), watch_flag(w) { }
1940+ };
1941+ hstest_thread(const arg_type& a) : arg(a), io_success_count(0),
1942+ op_success_count(0), response_min(99999), response_max(0),
1943+ response_sum(0), response_avg(0) { }
1944+ void operator ()();
1945+ void test_1();
1946+ void test_2_3(int test_num);
1947+ void test_4_5(int test_num);
1948+ void test_6(int test_num);
1949+ void test_7(int test_num);
1950+ void test_8(int test_num);
1951+ void test_9(int test_num);
1952+ void test_10(int test_num);
1953+ void test_11(int test_num);
1954+ void test_12(int test_num);
1955+ void test_21(int test_num);
1956+ void test_22(int test_num);
1957+ void test_watch();
1958+ void sleep_if();
1959+ void set_timing(double time_spent);
1960+ arg_type arg;
1961+ auto_file fd;
1962+ size_t io_success_count;
1963+ size_t op_success_count;
1964+ double response_min, response_max, response_sum, response_avg;
1965+};
1966+
1967+void
1968+hstest_thread::test_1()
1969+{
1970+ char buf[1024];
1971+ unsigned int seed = arg.id;
1972+ seed ^= arg.sh.conf.get_int("seed_xor", 0);
1973+ std::string err;
1974+ if (socket_connect(fd, arg.sh.arg, err) != 0) {
1975+ fprintf(stderr, "connect: %d %s\n", errno, strerror(errno));
1976+ return;
1977+ }
1978+ const char op = arg.sh.op;
1979+ const int tablesize = arg.sh.conf.get_int("tablesize", 0);
1980+ for (size_t i = 0; i < arg.sh.loop; ++i) {
1981+ for (size_t j = 0; j < arg.sh.pipe; ++j) {
1982+ int k = 0, v = 0, len = 0;
1983+ if (op == 'G') {
1984+ k = rand_r(&seed);
1985+ v = rand_r(&seed); /* unused */
1986+ if (tablesize != 0) {
1987+ k &= tablesize;
1988+ }
1989+ len = snprintf(buf, sizeof(buf), "%c\tk%d\n", op, k);
1990+ } else {
1991+ k = rand_r(&seed);
1992+ v = rand_r(&seed);
1993+ if (tablesize != 0) {
1994+ k &= tablesize;
1995+ }
1996+ len = snprintf(buf, sizeof(buf), "%c\tk%d\tv%d\n", op, k, v);
1997+ }
1998+ const int wlen = write(fd.get(), buf, len);
1999+ if (wlen != len) {
2000+ return;
2001+ }
2002+ }
2003+ size_t read_cnt = 0;
2004+ size_t read_pos = 0;
2005+ while (read_cnt < arg.sh.pipe) {
2006+ const int rlen = read(fd.get(), buf + read_pos, sizeof(buf) - read_pos);
2007+ if (rlen <= 0) {
2008+ return;
2009+ }
2010+ read_pos += rlen;
2011+ while (true) {
2012+ const char *const p = static_cast<const char *>(memchr(buf, '\n',
2013+ read_pos));
2014+ if (p == 0) {
2015+ break;
2016+ }
2017+ ++read_cnt;
2018+ ++io_success_count;
2019+ arg.sh.increment_count();
2020+ if (p != buf && buf[0] == '=') {
2021+ ++op_success_count;
2022+ }
2023+ const size_t rest_size = buf + read_pos - (p + 1);
2024+ if (rest_size != 0) {
2025+ memmove(buf, p + 1, rest_size);
2026+ }
2027+ read_pos = rest_size;
2028+ }
2029+ }
2030+ }
2031+}
2032+
2033+void
2034+hstest_thread::test_2_3(int test_num)
2035+{
2036+#if 0
2037+ char buf_k[128], buf_v[128];
2038+ unsigned int seed = arg.id;
2039+ op_base_t op = static_cast<op_base_t>(arg.sh.op);
2040+ micli_ptr hnd;
2041+ if (test_num == 2) {
2042+ hnd = micli_i::create_remote(arg.sh.conf);
2043+ } else if (test_num == 3) {
2044+ // hnd = micli_i::create_inproc(arg.sh.localdb);
2045+ }
2046+ if (hnd.get() == 0) {
2047+ return;
2048+ }
2049+ for (size_t i = 0; i < arg.sh.loop; ++i) {
2050+ for (size_t j = 0; j < arg.sh.pipe; ++j) {
2051+ int k = 0, v = 0, klen = 0, vlen = 0;
2052+ k = rand_r(&seed);
2053+ klen = snprintf(buf_k, sizeof(buf_k), "k%d", k);
2054+ v = rand_r(&seed); /* unused */
2055+ vlen = snprintf(buf_v, sizeof(buf_v), "v%d", v);
2056+ string_ref arr[2];
2057+ arr[0] = string_ref(buf_k, klen);
2058+ arr[1] = string_ref(buf_v, vlen);
2059+ pstrarr_ptr rec(arr, 2);
2060+ if (hnd->execute(op, 0, 0, rec.get_const())) {
2061+ ++io_success_count;
2062+ arg.sh.increment_count();
2063+ const dataset& res = hnd->get_result_ref();
2064+ if (res.size() == 1) {
2065+ ++op_success_count;
2066+ }
2067+ }
2068+ }
2069+ }
2070+#endif
2071+}
2072+
2073+void
2074+hstest_thread::test_4_5(int test_num)
2075+{
2076+#if 0
2077+ char buf_k[128], buf_v[8192];
2078+ memset(buf_v, ' ', sizeof(buf_v));
2079+ unsigned int seed = arg.id;
2080+ op_base_t op = static_cast<op_base_t>(arg.sh.op);
2081+ micli_ptr hnd;
2082+ if (test_num == 4) {
2083+ hnd = micli_i::create_remote(arg.sh.conf);
2084+ } else if (test_num == 5) {
2085+ hnd = micli_i::create_inproc(arg.sh.localdb);
2086+ }
2087+ if (hnd.get() == 0) {
2088+ return;
2089+ }
2090+ for (size_t i = 0; i < arg.sh.loop; ++i) {
2091+ for (size_t j = 0; j < arg.sh.pipe; ++j) {
2092+ int k = 0, klen = 0, vlen = 0;
2093+ k = i & 0x0000ffffUL;
2094+ if (k == 0) {
2095+ fprintf(stderr, "k=0\n");
2096+ }
2097+ klen = snprintf(buf_k, sizeof(buf_k), "k%d", k);
2098+ vlen = rand_r(&seed) % 8192;
2099+ string_ref arr[2];
2100+ arr[0] = string_ref(buf_k, klen);
2101+ arr[1] = string_ref(buf_v, vlen);
2102+ pstrarr_ptr rec(arr, 2);
2103+ if (hnd->execute(op, 0, 0, rec.get_const())) {
2104+ ++io_success_count;
2105+ const dataset& res = hnd->get_result_ref();
2106+ if (res.size() == 1) {
2107+ ++op_success_count;
2108+ }
2109+ }
2110+ }
2111+ }
2112+#endif
2113+}
2114+
2115+void
2116+hstest_thread::test_6(int test_num)
2117+{
2118+ int count = arg.sh.conf.get_int("count", 1);
2119+ auto_file fds[count];
2120+ for (int i = 0; i < count; ++i) {
2121+ const double t1 = gettimeofday_double();
2122+ std::string err;
2123+ if (socket_connect(fds[i], arg.sh.arg, err) != 0) {
2124+ fprintf(stderr, "id=%zu i=%d err=%s\n", arg.id, i, err.c_str());
2125+ }
2126+ const double t2 = gettimeofday_double();
2127+ if (t2 - t1 > 1) {
2128+ fprintf(stderr, "id=%zu i=%d time %f\n", arg.id, i, t2 - t1);
2129+ }
2130+ }
2131+}
2132+
2133+void
2134+hstest_thread::test_7(int num)
2135+{
2136+ /*
2137+ set foo 0 0 10
2138+ 0123456789
2139+ STORED
2140+ get foo
2141+ VALUE foo 0 10
2142+ 0123456789
2143+ END
2144+ get var
2145+ END
2146+ */
2147+ char buf[1024];
2148+ const int keep_connection = arg.sh.conf.get_int("keep_connection", 1);
2149+ unsigned int seed = arg.id;
2150+ seed ^= arg.sh.conf.get_int("seed_xor", 0);
2151+ const int tablesize = arg.sh.conf.get_int("tablesize", 0);
2152+ const char op = arg.sh.op;
2153+ for (size_t i = 0; i < arg.sh.loop; ++i) {
2154+ const double tm1 = gettimeofday_double();
2155+ std::string err;
2156+ if (fd.get() < 0 && socket_connect(fd, arg.sh.arg, err) != 0) {
2157+ fprintf(stderr, "connect: %d %s\n", errno, strerror(errno));
2158+ return;
2159+ }
2160+ for (size_t j = 0; j < arg.sh.pipe; ++j) {
2161+ int k = 0, v = 0, len = 0;
2162+ if (op == 'G') {
2163+ k = rand_r(&seed);
2164+ v = rand_r(&seed); /* unused */
2165+ if (tablesize != 0) {
2166+ k &= tablesize;
2167+ }
2168+ len = snprintf(buf, sizeof(buf), "get k%d\r\n", k);
2169+ } else {
2170+ k = rand_r(&seed);
2171+ v = rand_r(&seed);
2172+ if (tablesize != 0) {
2173+ k &= tablesize;
2174+ }
2175+ char vbuf[1024];
2176+ int vlen = snprintf(vbuf, sizeof(vbuf),
2177+ "v%d"
2178+ // "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
2179+ // "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
2180+ // "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
2181+ // "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
2182+ // "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
2183+ , v);
2184+ len = snprintf(buf, sizeof(buf), "set k%d 0 0 %d\r\n%s\r\n",
2185+ k, vlen, vbuf);
2186+ }
2187+ const int wlen = write(fd.get(), buf, len);
2188+ if (wlen != len) {
2189+ return;
2190+ }
2191+ }
2192+ size_t read_cnt = 0;
2193+ size_t read_pos = 0;
2194+ bool read_response_done = false;
2195+ bool expect_value = false;
2196+ while (!read_response_done) {
2197+ const int rlen = read(fd.get(), buf + read_pos, sizeof(buf) - read_pos);
2198+ if (rlen <= 0) {
2199+ return;
2200+ }
2201+ read_pos += rlen;
2202+ while (true) {
2203+ const char *const p = static_cast<const char *>(memchr(buf, '\n',
2204+ read_pos));
2205+ if (p == 0) {
2206+ break;
2207+ }
2208+ ++read_cnt;
2209+ if (expect_value) {
2210+ expect_value = false;
2211+ } else if (p >= buf + 6 && memcmp(buf, "VALUE ", 6) == 0) {
2212+ expect_value = true;
2213+ ++op_success_count;
2214+ } else {
2215+ if (p == buf + 7 && memcmp(buf, "STORED\r", 7) == 0) {
2216+ ++op_success_count;
2217+ }
2218+ read_response_done = true;
2219+ }
2220+ const size_t rest_size = buf + read_pos - (p + 1);
2221+ if (rest_size != 0) {
2222+ memmove(buf, p + 1, rest_size);
2223+ }
2224+ read_pos = rest_size;
2225+ }
2226+ ++io_success_count;
2227+ }
2228+ arg.sh.increment_count();
2229+ if (!keep_connection) {
2230+ fd.close();
2231+ }
2232+ const double tm2 = gettimeofday_double();
2233+ set_timing(tm2 - tm1);
2234+ sleep_if();
2235+ }
2236+}
2237+
2238+struct rec {
2239+ std::string key;
2240+ std::string value;
2241+};
2242+
2243+void
2244+hstest_thread::test_8(int test_num)
2245+{
2246+#if 0
2247+ char buf_k[128], buf_v[128];
2248+ unsigned int seed = arg.id;
2249+ // op_base_t op = static_cast<op_base_t>(arg.sh.op);
2250+ using namespace boost::multi_index;
2251+ typedef member<rec, std::string, &rec::key> rec_get_key;
2252+ typedef ordered_unique<rec_get_key> oui;
2253+ typedef multi_index_container< rec, indexed_by<oui> > mic;
2254+ #if 0
2255+ typedef std::map<std::string, std::string> m_type;
2256+ m_type m;
2257+ #endif
2258+ mic m;
2259+ for (size_t i = 0; i < arg.sh.loop; ++i) {
2260+ for (size_t j = 0; j < arg.sh.pipe; ++j) {
2261+ int k = 0, v = 0, klen = 0, vlen = 0;
2262+ k = rand_r(&seed);
2263+ klen = snprintf(buf_k, sizeof(buf_k), "k%d", k);
2264+ v = rand_r(&seed); /* unused */
2265+ vlen = snprintf(buf_v, sizeof(buf_v), "v%d", v);
2266+ const std::string ks(buf_k, klen);
2267+ const std::string vs(buf_v, vlen);
2268+ rec r;
2269+ r.key = ks;
2270+ r.value = vs;
2271+ m.insert(r);
2272+ // m.insert(std::make_pair(ks, vs));
2273+ ++io_success_count;
2274+ ++op_success_count;
2275+ arg.sh.increment_count();
2276+ }
2277+ }
2278+#endif
2279+}
2280+
2281+struct mysqltest_thread_initobj : private noncopyable {
2282+ mysqltest_thread_initobj() {
2283+ mysql_thread_init();
2284+ }
2285+ ~mysqltest_thread_initobj() {
2286+ mysql_thread_end();
2287+ }
2288+};
2289+
2290+void
2291+hstest_thread::test_9(int test_num)
2292+{
2293+ /* create table hstest
2294+ * ( k varchar(255) not null, v varchar(255) not null, primary key(k))
2295+ * engine = innodb; */
2296+ auto_mysql db;
2297+ // mysqltest_thread_initobj initobj;
2298+ std::string err;
2299+ const char op = arg.sh.op;
2300+ const std::string suffix = arg.sh.conf.get_str("value_suffix", "upd");
2301+ unsigned long long err_cnt = 0;
2302+ unsigned long long query_cnt = 0;
2303+ #if 0
2304+ my_bool reconnect = 0;
2305+ if (mysql_options(db, MYSQL_OPT_RECONNECT, &reconnect) != 0) {
2306+ err = "mysql_options() failed";
2307+ ++err_cnt;
2308+ return;
2309+ }
2310+ #endif
2311+ unsigned int seed = time(0) + arg.id + 1;
2312+ seed ^= arg.sh.conf.get_int("seed_xor", 0);
2313+ drand48_data randbuf;
2314+ srand48_r(seed, &randbuf);
2315+ const std::string mysql_host = arg.sh.conf.get_str("host", "localhost");
2316+ const int mysql_port = arg.sh.conf.get_int("mysqlport", 3306);
2317+ const int num = arg.sh.loop;
2318+ const std::string mysql_user = arg.sh.conf.get_str("mysqluser", "root");
2319+ const std::string mysql_passwd = arg.sh.conf.get_str("mysqlpass", "");
2320+ const std::string mysql_dbname = arg.sh.conf.get_str("dbname", "hstest");
2321+ const int keep_connection = arg.sh.conf.get_int("keep_connection", 1);
2322+ const int verbose = arg.sh.conf.get_int("verbose", 1);
2323+ const int tablesize = arg.sh.conf.get_int("tablesize", 10000);
2324+ const int moreflds = arg.sh.conf.get_int("moreflds", 0);
2325+ const std::string moreflds_prefix = arg.sh.conf.get_str(
2326+ "moreflds_prefix", "column0123456789_");
2327+ const int use_handler = arg.sh.conf.get_int("handler", 0);
2328+ const int sched_flag = arg.sh.conf.get_int("sched", 0);
2329+ const int use_in = arg.sh.conf.get_int("in", 0);
2330+ const int ssps = use_in ? 0 : arg.sh.conf.get_int("ssps", 0);
2331+ std::string flds = "v";
2332+ for (int i = 0; i < moreflds; ++i) {
2333+ char buf[1024];
2334+ snprintf(buf, sizeof(buf), ",%s%d", moreflds_prefix.c_str(), i);
2335+ flds += std::string(buf);
2336+ }
2337+ int connected = 0;
2338+ std::auto_ptr<auto_mysql_stmt> stmt;
2339+ string_buffer wbuf;
2340+ for (int i = 0; i < num; ++i) {
2341+ const double tm1 = gettimeofday_double();
2342+ const int flags = 0;
2343+ if (connected == 0) {
2344+ if (!mysql_real_connect(db, mysql_host.c_str(),
2345+ mysql_user.c_str(), mysql_user.empty() ? 0 : mysql_passwd.c_str(),
2346+ mysql_dbname.c_str(), mysql_port, 0, flags)) {
2347+ err = "failed to connect: " + std::string(mysql_error(db));
2348+ if (verbose >= 1) {
2349+ fprintf(stderr, "e=[%s]\n", err.c_str());
2350+ }
2351+ ++err_cnt;
2352+ return;
2353+ }
2354+ arg.sh.increment_conn(1);
2355+ }
2356+ int r = 0;
2357+ if (connected == 0 && use_handler) {
2358+ const char *const q = "handler hstest_table1 open";
2359+ r = mysql_real_query(db, q, strlen(q));
2360+ if (r != 0) {
2361+ err = 1;
2362+ }
2363+ }
2364+ if (connected == 0 && ssps) {
2365+ stmt.reset(new auto_mysql_stmt(db));
2366+ const char *const q = "select v from hstest_table1 where k = ?";
2367+ r = mysql_stmt_prepare(*stmt, q, strlen(q));
2368+ if (r != 0) {
2369+ fprintf(stderr, "ssps err\n");
2370+ ++err_cnt;
2371+ return;
2372+ }
2373+ }
2374+ connected = 1;
2375+ std::string result_str;
2376+ unsigned int err = 0;
2377+ unsigned int num_flds = 0, num_affected_rows = 0;
2378+ int got_data = 0;
2379+ char buf_query[16384];
2380+ int buf_query_len = 0;
2381+ int k = 0, v = 0;
2382+ {
2383+ double kf = 0, vf = 0;
2384+ drand48_r(&randbuf, &kf);
2385+ drand48_r(&randbuf, &vf);
2386+ k = int(kf * tablesize);
2387+ v = int(vf * tablesize);
2388+ #if 0
2389+ k = rand_r(&seed);
2390+ v = rand_r(&seed);
2391+ if (tablesize != 0) {
2392+ k %= tablesize;
2393+ }
2394+ #endif
2395+ if (op == 'G') {
2396+ if (use_handler) {
2397+ buf_query_len = snprintf(buf_query, sizeof(buf_query),
2398+ "handler hstest_table1 read `primary` = ( '%d' )", k);
2399+ // TODO: moreflds
2400+ } else if (ssps) {
2401+ //
2402+ } else if (use_in) {
2403+ wbuf.clear();
2404+ char *p = wbuf.make_space(1024);
2405+ int len = snprintf(p, 1024, "select %s from hstest_table1 where k in ('%d'", flds.c_str(), k);
2406+ wbuf.space_wrote(len);
2407+ for (int j = 1; j < use_in; ++j) {
2408+ /* generate more key */
2409+ drand48_r(&randbuf, &kf);
2410+ k = int(kf * tablesize);
2411+ p = wbuf.make_space(1024);
2412+ int len = snprintf(p, 1024, ", '%d'", k);
2413+ wbuf.space_wrote(len);
2414+ }
2415+ wbuf.append_literal(")");
2416+ } else {
2417+ buf_query_len = snprintf(buf_query, sizeof(buf_query),
2418+ "select %s from hstest_table1 where k = '%d'", flds.c_str(), k);
2419+ }
2420+ } else if (op == 'U') {
2421+ buf_query_len = snprintf(buf_query, sizeof(buf_query),
2422+ "update hstest_table1 set v = '%d_%d%s' where k = '%d'",
2423+ v, k, suffix.c_str(), k);
2424+ } else if (op == 'R') {
2425+ buf_query_len = snprintf(buf_query, sizeof(buf_query),
2426+ "replace into hstest_table1 values ('%d', 'v%d')", k, v);
2427+ // TODO: moreflds
2428+ }
2429+ }
2430+ if (r == 0) {
2431+ if (ssps) {
2432+ MYSQL_BIND bind[1] = { };
2433+ bind[0].buffer_type = MYSQL_TYPE_LONG;
2434+ bind[0].buffer = (char *)&k;
2435+ bind[0].is_null = 0;
2436+ bind[0].length = 0;
2437+ if (mysql_stmt_bind_param(*stmt, bind)) {
2438+ fprintf(stderr, "err: %s\n", mysql_stmt_error(*stmt));
2439+ ++err_cnt;
2440+ return;
2441+ }
2442+ r = mysql_stmt_execute(*stmt);
2443+ // fprintf(stderr, "stmt exec\n");
2444+ } else if (use_in) {
2445+ r = mysql_real_query(db, wbuf.begin(), wbuf.size());
2446+ } else {
2447+ r = mysql_real_query(db, buf_query, buf_query_len);
2448+ // fprintf(stderr, "real query\n");
2449+ }
2450+ ++query_cnt;
2451+ }
2452+ if (r != 0) {
2453+ err = 1;
2454+ } else if (ssps) {
2455+ if (verbose >= 0) {
2456+ char resbuf[1024];
2457+ unsigned long res_len = 0;
2458+ MYSQL_BIND bind[1] = { };
2459+ bind[0].buffer_type = MYSQL_TYPE_STRING;
2460+ bind[0].buffer = resbuf;
2461+ bind[0].buffer_length = sizeof(resbuf);
2462+ bind[0].length = &res_len;
2463+ if (mysql_stmt_bind_result(*stmt, bind)) {
2464+ fprintf(stderr, "err: %s\n", mysql_stmt_error(*stmt));
2465+ ++err_cnt;
2466+ return;
2467+ }
2468+ if (mysql_stmt_fetch(*stmt)) {
2469+ fprintf(stderr, "err: %s\n", mysql_stmt_error(*stmt));
2470+ ++err_cnt;
2471+ return;
2472+ }
2473+ if (!result_str.empty()) {
2474+ result_str += " ";
2475+ }
2476+ result_str += std::string(resbuf, res_len);
2477+ // fprintf(stderr, "SSPS RES: %s\n", result_str.c_str());
2478+ got_data = 1;
2479+ } else {
2480+ got_data = 1;
2481+ }
2482+ } else {
2483+ auto_mysql_res res(db);
2484+ if (res != 0) {
2485+ if (verbose >= 0) {
2486+ num_flds = mysql_num_fields(res);
2487+ MYSQL_ROW row = 0;
2488+ while ((row = mysql_fetch_row(res)) != 0) {
2489+ got_data += 1;
2490+ unsigned long *const lengths = mysql_fetch_lengths(res);
2491+ if (verbose >= 2) {
2492+ for (unsigned int i = 0; i < num_flds; ++i) {
2493+ if (!result_str.empty()) {
2494+ result_str += " ";
2495+ }
2496+ result_str += std::string(row[i], lengths[i]);
2497+ }
2498+ }
2499+ }
2500+ } else {
2501+ MYSQL_ROW row = 0;
2502+ while ((row = mysql_fetch_row(res)) != 0) {
2503+ got_data += 1;
2504+ }
2505+ }
2506+ } else {
2507+ if (mysql_field_count(db) == 0) {
2508+ num_affected_rows = mysql_affected_rows(db);
2509+ } else {
2510+ err = 1;
2511+ }
2512+ }
2513+ }
2514+ if (verbose >= 2 || (verbose >= 1 && err != 0)) {
2515+ if (err) {
2516+ ++err_cnt;
2517+ const char *const errstr = mysql_error(db);
2518+ fprintf(stderr, "e=[%s] a=%u q=[%s]\n", errstr,
2519+ num_affected_rows, buf_query);
2520+ } else {
2521+ fprintf(stderr, "a=%u q=[%s] r=[%s]\n", num_affected_rows, buf_query,
2522+ result_str.c_str());
2523+ }
2524+ }
2525+ if (err == 0) {
2526+ ++io_success_count;
2527+ if (num_affected_rows > 0 || got_data > 0) {
2528+ op_success_count += got_data;
2529+ } else {
2530+ if (verbose >= 1) {
2531+ fprintf(stderr, "k=%d numaff=%u gotdata=%d\n",
2532+ k, num_affected_rows, got_data);
2533+ }
2534+ }
2535+ arg.sh.increment_count();
2536+ }
2537+ if (!keep_connection) {
2538+ if (stmt.get() != 0) {
2539+ stmt.reset();
2540+ }
2541+ db.reset();
2542+ connected = 0;
2543+ }
2544+ const double tm2 = gettimeofday_double();
2545+ set_timing(tm2 - tm1);
2546+ sleep_if();
2547+ if (sched_flag) {
2548+ sched_yield();
2549+ }
2550+ }
2551+ if (verbose >= 1) {
2552+ fprintf(stderr, "thread finished (error_count=%llu)\n", err_cnt);
2553+ }
2554+}
2555+
2556+void
2557+hstest_thread::test_10(int test_num)
2558+{
2559+ const int keep_connection = arg.sh.conf.get_int("keep_connection", 1);
2560+ unsigned int seed = time(0) + arg.id + 1;
2561+ seed ^= arg.sh.conf.get_int("seed_xor", 0);
2562+ drand48_data randbuf;
2563+ srand48_r(seed, &randbuf);
2564+ std::string err;
2565+ int keepconn_count = 0;
2566+ const char op = arg.sh.op;
2567+ const int verbose = arg.sh.conf.get_int("verbose", 1);
2568+ const std::string suffix = arg.sh.conf.get_str("value_suffix", "upd");
2569+ const int tablesize = arg.sh.conf.get_int("tablesize", 10000);
2570+ const int firstkey = arg.sh.conf.get_int("firstkey", 0);
2571+ const int sched_flag = arg.sh.conf.get_int("sched", 0);
2572+ const int moreflds = arg.sh.conf.get_int("moreflds", 0);
2573+ const std::string dbname = arg.sh.conf.get_str("dbname", "hstest");
2574+ const std::string table = arg.sh.conf.get_str("table", "hstest_table1");
2575+ const std::string index = arg.sh.conf.get_str("index", "PRIMARY");
2576+ const std::string field = arg.sh.conf.get_str("field", "v");
2577+ const int use_in = arg.sh.conf.get_int("in", 0);
2578+ const std::string moreflds_prefix = arg.sh.conf.get_str(
2579+ "moreflds_prefix", "column0123456789_");
2580+ const int dump = arg.sh.dump;
2581+ const int nodup = arg.sh.conf.get_int("nodup", 0);
2582+ std::string moreflds_str;
2583+ for (int i = 0; i < moreflds; ++i) {
2584+ char sbuf[1024];
2585+ snprintf(sbuf, sizeof(sbuf), ",%s%d", moreflds_prefix.c_str(), i);
2586+ moreflds_str += std::string(sbuf);
2587+ }
2588+ string_buffer wbuf;
2589+ char rbuf[16384];
2590+ for (size_t i = 0; i < arg.sh.loop; ++i) {
2591+ int len = 0, rlen = 0, wlen = 0;
2592+ #if 0
2593+ const double tm1 = gettimeofday_double();
2594+ #endif
2595+ if (fd.get() < 0) {
2596+ if (socket_connect(fd, arg.sh.arg, err) != 0) {
2597+ fprintf(stderr, "connect: %d %s\n", errno, strerror(errno));
2598+ return;
2599+ }
2600+ char *wp = wbuf.make_space(1024);
2601+ len = snprintf(wp, 1024,
2602+ "P\t1\t%s\t%s\tPRIMARY\t%s%s\n", dbname.c_str(), table.c_str(),
2603+ field.c_str(), moreflds_str.c_str());
2604+ /* pst_num, db, table, index, retflds */
2605+ wbuf.space_wrote(len);
2606+ wlen = write(fd.get(), wbuf.begin(), len);
2607+ if (len != wlen) {
2608+ fprintf(stderr, "write: %d %d\n", len, wlen);
2609+ return;
2610+ }
2611+ wbuf.clear();
2612+ rlen = read(fd.get(), rbuf, sizeof(rbuf));
2613+ if (rlen <= 0 || rbuf[rlen - 1] != '\n') {
2614+ fprintf(stderr, "read: rlen=%d errno=%d\n", rlen, errno);
2615+ return;
2616+ }
2617+ if (rbuf[0] != '0') {
2618+ fprintf(stderr, "failed to open table\n");
2619+ return;
2620+ }
2621+ arg.sh.increment_conn(1);
2622+ }
2623+ const double tm1 = gettimeofday_double();
2624+ for (size_t j = 0; j < arg.sh.pipe; ++j) {
2625+ int k = 0, v = 0;
2626+ {
2627+ while (true) {
2628+ double kf = 0, vf = 0;
2629+ drand48_r(&randbuf, &kf);
2630+ drand48_r(&randbuf, &vf);
2631+ k = int(kf * tablesize) + firstkey;
2632+ v = int(vf * tablesize) + firstkey;
2633+ if (k - firstkey < arg.sh.keygen_size) {
2634+ volatile char *const ptr = arg.sh.keygen + (k - firstkey);
2635+ // int oldv = __sync_fetch_and_or(ptr, 1);
2636+ int oldv = *ptr;
2637+ *ptr += 1;
2638+ if (nodup && oldv != 0) {
2639+ if (dump) {
2640+ fprintf(stderr, "retry\n");
2641+ }
2642+ continue;
2643+ }
2644+ } else {
2645+ if (nodup) {
2646+ if (dump) {
2647+ fprintf(stderr, "retry2\n");
2648+ }
2649+ continue;
2650+ }
2651+ }
2652+ size_t len = 0;
2653+ if (op == 'G') {
2654+ if (use_in) {
2655+ char *wp = wbuf.make_space(1024);
2656+ len = snprintf(wp, 1024, "1\t=\t1\t\t%d\t0\t@\t0\t%d\t%d",
2657+ use_in, use_in, k);
2658+ wbuf.space_wrote(len);
2659+ for (int j = 1; j < use_in; ++j) {
2660+ drand48_r(&randbuf, &kf);
2661+ k = int(kf * tablesize) + firstkey;
2662+ char *wp = wbuf.make_space(1024);
2663+ len = snprintf(wp, 1024, "\t%d", k);
2664+ wbuf.space_wrote(len);
2665+ }
2666+ wbuf.append_literal("\n");
2667+ } else {
2668+ char *wp = wbuf.make_space(1024);
2669+ len = snprintf(wp, 1024, "1\t=\t1\t%d\n", k);
2670+ wbuf.space_wrote(len);
2671+ }
2672+ } else if (op == 'U') {
2673+ char *wp = wbuf.make_space(1024);
2674+ len = snprintf(wp, 1024,
2675+ "1\t=\t1\t%d\t1\t0\tU\t%d_%d%s\n", k, v, k, suffix.c_str());
2676+ wbuf.space_wrote(len);
2677+ }
2678+ break;
2679+ }
2680+ }
2681+ }
2682+ wlen = write(fd.get(), wbuf.begin(), wbuf.size());
2683+ if (wlen != wbuf.size()) {
2684+ fprintf(stderr, "write: %d %d\n", (int)wbuf.size(), wlen);
2685+ return;
2686+ }
2687+ wbuf.clear();
2688+ size_t read_cnt = 0;
2689+ size_t read_pos = 0;
2690+ while (read_cnt < arg.sh.pipe) {
2691+ rlen = read(fd.get(), rbuf + read_pos, sizeof(rbuf) - read_pos);
2692+ if (rlen <= 0) {
2693+ fprintf(stderr, "read: %d\n", rlen);
2694+ return;
2695+ }
2696+ read_pos += rlen;
2697+ while (true) {
2698+ const char *const nl = static_cast<const char *>(memchr(rbuf, '\n',
2699+ read_pos));
2700+ if (nl == 0) {
2701+ break;
2702+ }
2703+ ++read_cnt;
2704+ ++io_success_count;
2705+ const char *t1 = static_cast<const char *>(memchr(rbuf, '\t',
2706+ nl - rbuf));
2707+ if (t1 == 0) {
2708+ fprintf(stderr, "error \n");
2709+ break;
2710+ }
2711+ ++t1;
2712+ const char *t2 = static_cast<const char *>(memchr(t1, '\t',
2713+ nl - t1));
2714+ if (t2 == 0) {
2715+ if (verbose > 1) {
2716+ fprintf(stderr, "key: notfound \n");
2717+ }
2718+ break;
2719+ }
2720+ ++t2;
2721+ if (t1 == rbuf + 2 && rbuf[0] == '0') {
2722+ if (op == 'G') {
2723+ ++op_success_count;
2724+ arg.sh.increment_count();
2725+ } else if (op == 'U') {
2726+ const char *t3 = t2;
2727+ while (t3 != nl && t3[0] >= 0x10) {
2728+ ++t3;
2729+ }
2730+ if (t3 != t2 + 1 || t2[0] != '1') {
2731+ const std::string mess(t2, t3);
2732+ fprintf(stderr, "mod: %s\n", mess.c_str());
2733+ } else {
2734+ ++op_success_count;
2735+ arg.sh.increment_count();
2736+ if (arg.sh.dump && arg.sh.pipe == 1) {
2737+ fwrite(wbuf, wbuflen, 1, stderr);
2738+ }
2739+ }
2740+ }
2741+ } else {
2742+ const char *t3 = t2;
2743+ while (t3 != nl && t3[0] >= 0x10) {
2744+ ++t3;
2745+ }
2746+ const std::string mess(t2, t3);
2747+ fprintf(stderr, "err: %s\n", mess.c_str());
2748+ }
2749+ const size_t rest_size = rbuf + read_pos - (nl + 1);
2750+ if (rest_size != 0) {
2751+ memmove(rbuf, nl + 1, rest_size);
2752+ }
2753+ read_pos = rest_size;
2754+ }
2755+ }
2756+ if (!keep_connection) {
2757+ fd.reset();
2758+ arg.sh.increment_conn(-1);
2759+ } else if (keep_connection > 1 && ++keepconn_count > keep_connection) {
2760+ keepconn_count = 0;
2761+ fd.reset();
2762+ arg.sh.increment_conn(-1);
2763+ }
2764+ const double tm2 = gettimeofday_double();
2765+ set_timing(tm2 - tm1);
2766+ sleep_if();
2767+ if (sched_flag) {
2768+ sched_yield();
2769+ }
2770+ }
2771+ if (dump) {
2772+ fprintf(stderr, "done\n");
2773+ }
2774+}
2775+
2776+void
2777+hstest_thread::sleep_if()
2778+{
2779+ if (arg.sh.usleep) {
2780+ struct timespec ts = {
2781+ arg.sh.usleep / 1000000,
2782+ (arg.sh.usleep % 1000000) * 1000
2783+ };
2784+ nanosleep(&ts, 0);
2785+ }
2786+}
2787+
2788+void
2789+hstest_thread::set_timing(double time_spent)
2790+{
2791+ response_min = std::min(response_min, time_spent);
2792+ response_max = std::max(response_max, time_spent);
2793+ response_sum += time_spent;
2794+ if (op_success_count != 0) {
2795+ response_avg = response_sum / op_success_count;
2796+ }
2797+}
2798+
2799+void
2800+hstest_thread::test_11(int test_num)
2801+{
2802+ const int keep_connection = arg.sh.conf.get_int("keep_connection", 1);
2803+ const int tablesize = arg.sh.conf.get_int("tablesize", 0);
2804+ unsigned int seed = arg.id;
2805+ seed ^= arg.sh.conf.get_int("seed_xor", 0);
2806+ std::string err;
2807+ hstcpcli_ptr cli;
2808+ for (size_t i = 0; i < arg.sh.loop; ++i) {
2809+ if (cli.get() == 0) {
2810+ cli = hstcpcli_i::create(arg.sh.arg);
2811+ cli->request_buf_open_index(0, "hstest", "hstest_table1", "", "v");
2812+ /* pst_num, db, table, index, retflds */
2813+ if (cli->request_send() != 0) {
2814+ fprintf(stderr, "reuqest_send: %s\n", cli->get_error().c_str());
2815+ return;
2816+ }
2817+ size_t num_flds = 0;
2818+ if (cli->response_recv(num_flds) != 0) {
2819+ fprintf(stderr, "reuqest_recv: %s\n", cli->get_error().c_str());
2820+ return;
2821+ }
2822+ cli->response_buf_remove();
2823+ }
2824+ for (size_t j = 0; j < arg.sh.pipe; ++j) {
2825+ char buf[256];
2826+ int k = 0, v = 0, len = 0;
2827+ {
2828+ k = rand_r(&seed);
2829+ v = rand_r(&seed); /* unused */
2830+ if (tablesize != 0) {
2831+ k &= tablesize;
2832+ }
2833+ len = snprintf(buf, sizeof(buf), "%d", k);
2834+ }
2835+ const string_ref key(buf, len);
2836+ const string_ref op("=", 1);
2837+ cli->request_buf_exec_generic(0, op, &key, 1, 1, 0, string_ref(), 0, 0);
2838+ }
2839+ if (cli->request_send() != 0) {
2840+ fprintf(stderr, "reuqest_send: %s\n", cli->get_error().c_str());
2841+ return;
2842+ }
2843+ size_t read_cnt = 0;
2844+ for (size_t j = 0; j < arg.sh.pipe; ++j) {
2845+ size_t num_flds = 0;
2846+ if (cli->response_recv(num_flds) != 0) {
2847+ fprintf(stderr, "reuqest_recv: %s\n", cli->get_error().c_str());
2848+ return;
2849+ }
2850+ {
2851+ ++read_cnt;
2852+ ++io_success_count;
2853+ arg.sh.increment_count();
2854+ {
2855+ ++op_success_count;
2856+ }
2857+ }
2858+ cli->response_buf_remove();
2859+ }
2860+ if (!keep_connection) {
2861+ cli.reset();
2862+ }
2863+ }
2864+}
2865+
2866+void
2867+hstest_thread::test_watch()
2868+{
2869+ const int timelimit = arg.sh.conf.get_int("timelimit", 0);
2870+ const int timelimit_offset = timelimit / 2;
2871+ int loop = 0;
2872+ double t1 = 0, t2 = 0;
2873+ size_t cnt_t1 = 0, cnt_t2 = 0;
2874+ size_t prev_cnt = 0;
2875+ double now_f = 0;
2876+ while (true) {
2877+ sleep(1);
2878+ const size_t cnt = arg.sh.count;
2879+ const size_t df = cnt - prev_cnt;
2880+ prev_cnt = cnt;
2881+ const double now_prev = now_f;
2882+ now_f = gettimeofday_double();
2883+ if (now_prev != 0) {
2884+ const double rps = static_cast<double>(df) / (now_f - now_prev);
2885+ fprintf(stderr, "now: %zu cntdiff: %zu tdiff: %f rps: %f\n",
2886+ static_cast<size_t>(now_f), df, now_f - now_prev, rps);
2887+ }
2888+ if (timelimit != 0) {
2889+ if (arg.sh.wait_conn == 0 || arg.sh.conn_count >= arg.sh.wait_conn) {
2890+ ++loop;
2891+ }
2892+ if (loop == timelimit_offset) {
2893+ t1 = gettimeofday_double();
2894+ cnt_t1 = cnt;
2895+ arg.sh.enable_timing = 1;
2896+ fprintf(stderr, "start timing\n");
2897+ } else if (loop == timelimit_offset + timelimit) {
2898+ t2 = gettimeofday_double();
2899+ cnt_t2 = cnt;
2900+ const size_t cnt_diff = cnt_t2 - cnt_t1;
2901+ const double tdiff = t2 - t1;
2902+ const double qps = cnt_diff / (tdiff != 0 ? tdiff : 1);
2903+ fprintf(stderr, "(%f: %zu, %f: %zu), %10.5f qps\n",
2904+ t1, cnt_t1, t2, cnt_t2, qps);
2905+ size_t keycnt = 0;
2906+ for (int i = 0; i < arg.sh.keygen_size; ++i) {
2907+ if (arg.sh.keygen[i]) {
2908+ ++keycnt;
2909+ }
2910+ }
2911+ fprintf(stderr, "keygen=%zu\n", keycnt);
2912+ break;
2913+ }
2914+ }
2915+ }
2916+#if 0
2917+ int loop = 0;
2918+ double t1 = 0, t2 = 0;
2919+ size_t cnt_t1 = 0, cnt_t2 = 0;
2920+ size_t prev_cnt = 0;
2921+ while (true) {
2922+ sleep(1);
2923+ const size_t cnt = arg.sh.count;
2924+ const size_t df = cnt - prev_cnt;
2925+ prev_cnt = cnt;
2926+ const size_t now = time(0);
2927+ fprintf(stderr, "%zu %zu\n", now, df);
2928+ if (timelimit != 0) {
2929+ ++loop;
2930+ if (loop == timelimit_offset) {
2931+ t1 = gettimeofday_double();
2932+ cnt_t1 = cnt;
2933+ } else if (loop == timelimit_offset + timelimit) {
2934+ t2 = gettimeofday_double();
2935+ cnt_t2 = cnt;
2936+ const size_t cnt_diff = cnt_t2 - cnt_t1;
2937+ const double tdiff = t2 - t1;
2938+ const double qps = cnt_diff / (tdiff != 0 ? tdiff : 1);
2939+ fprintf(stderr, "(%f: %zu, %f: %zu), %10.5f qps\n",
2940+ t1, cnt_t1, t2, cnt_t2, qps);
2941+ size_t keycnt = 0;
2942+ for (int i = 0; i < arg.sh.keygen_size; ++i) {
2943+ if (arg.sh.keygen[i]) {
2944+ ++keycnt;
2945+ }
2946+ }
2947+ fprintf(stderr, "keygen=%zu\n", keycnt);
2948+ _exit(0);
2949+ }
2950+ }
2951+ }
2952+#endif
2953+}
2954+
2955+void
2956+hstest_thread::test_12(int test_num)
2957+{
2958+ /* NOTE: num_threads should be 1 */
2959+ /* create table hstest
2960+ * ( k varchar(255) not null, v varchar(255) not null, primary key(k))
2961+ * engine = innodb; */
2962+ mysqltest_thread_initobj initobj;
2963+ auto_mysql db;
2964+ std::string err;
2965+ unsigned long long err_cnt = 0;
2966+ unsigned long long query_cnt = 0;
2967+ #if 0
2968+ my_bool reconnect = 0;
2969+ if (mysql_options(db, MYSQL_OPT_RECONNECT, &reconnect) != 0) {
2970+ err = "mysql_options() failed";
2971+ ++err_cnt;
2972+ return;
2973+ }
2974+ #endif
2975+ const std::string mysql_host = arg.sh.conf.get_str("host", "localhost");
2976+ const int mysql_port = arg.sh.conf.get_int("mysqlport", 3306);
2977+ const unsigned int num = arg.sh.loop;
2978+ const size_t pipe = arg.sh.pipe;
2979+ const std::string mysql_user = arg.sh.conf.get_str("mysqluser", "root");
2980+ const std::string mysql_passwd = arg.sh.conf.get_str("mysqlpass", "");
2981+ const std::string mysql_dbname = arg.sh.conf.get_str("db", "hstest");
2982+ const int keep_connection = arg.sh.conf.get_int("keep_connection", 1);
2983+ const int verbose = arg.sh.conf.get_int("verbose", 1);
2984+ const int use_handler = arg.sh.conf.get_int("handler", 0);
2985+ int connected = 0;
2986+ unsigned int k = 0;
2987+ string_buffer buf;
2988+ for (unsigned int i = 0; i < num; ++i) {
2989+ const int flags = 0;
2990+ if (connected == 0 && !mysql_real_connect(db, mysql_host.c_str(),
2991+ mysql_user.c_str(), mysql_user.empty() ? 0 : mysql_passwd.c_str(),
2992+ mysql_dbname.c_str(), mysql_port, 0, flags)) {
2993+ err = "failed to connect: " + std::string(mysql_error(db));
2994+ if (verbose >= 1) {
2995+ fprintf(stderr, "e=[%s]\n", err.c_str());
2996+ }
2997+ ++err_cnt;
2998+ return;
2999+ }
3000+ int r = 0;
3001+ if (connected == 0 && use_handler) {
3002+ const char *const q = "handler hstest open";
3003+ r = mysql_real_query(db, q, strlen(q));
3004+ if (r != 0) {
3005+ err = 1;
3006+ }
3007+ }
3008+ connected = 1;
3009+ std::string result_str;
3010+ unsigned int err = 0;
3011+ unsigned int num_flds = 0, num_affected_rows = 0;
3012+ int got_data = 0;
3013+ buf.clear();
3014+ buf.append_literal("insert into hstest values ");
3015+ for (size_t j = 0; j < pipe; ++j) {
3016+ const unsigned int v = ~k;
3017+ if (j != 0) {
3018+ buf.append_literal(",");
3019+ }
3020+ char *wp = buf.make_space(64);
3021+ int buf_query_len = snprintf(wp, 64, "('k%u', 'v%u')", k, v);
3022+ buf.space_wrote(buf_query_len);
3023+ ++k;
3024+ }
3025+ if (r == 0) {
3026+ r = mysql_real_query(db, buf.begin(), buf.size());
3027+ ++query_cnt;
3028+ }
3029+ if (r != 0) {
3030+ err = 1;
3031+ } else {
3032+ auto_mysql_res res(db);
3033+ if (res != 0) {
3034+ if (verbose >= 0) {
3035+ num_flds = mysql_num_fields(res);
3036+ MYSQL_ROW row = 0;
3037+ while ((row = mysql_fetch_row(res)) != 0) {
3038+ got_data = 1;
3039+ unsigned long *const lengths = mysql_fetch_lengths(res);
3040+ if (verbose >= 2) {
3041+ for (unsigned int i = 0; i < num_flds; ++i) {
3042+ if (!result_str.empty()) {
3043+ result_str += " ";
3044+ }
3045+ result_str += std::string(row[i], lengths[i]);
3046+ }
3047+ }
3048+ }
3049+ }
3050+ } else {
3051+ if (mysql_field_count(db) == 0) {
3052+ num_affected_rows = mysql_affected_rows(db);
3053+ } else {
3054+ err = 1;
3055+ }
3056+ }
3057+ }
3058+ if (verbose >= 2 || (verbose >= 1 && err != 0)) {
3059+ if (err) {
3060+ ++err_cnt;
3061+ const char *const errstr = mysql_error(db);
3062+ fprintf(stderr, "e=[%s] a=%u q=[%s]\n", errstr,
3063+ num_affected_rows, std::string(buf.begin(), buf.size()).c_str());
3064+ } else {
3065+ fprintf(stderr, "a=%u q=[%s] r=[%s]\n", num_affected_rows,
3066+ std::string(buf.begin(), buf.size()).c_str(),
3067+ result_str.c_str());
3068+ }
3069+ }
3070+ if (err == 0) {
3071+ ++io_success_count;
3072+ if (num_affected_rows > 0 || got_data > 0) {
3073+ ++op_success_count;
3074+ }
3075+ arg.sh.increment_count(pipe);
3076+ }
3077+ if (!keep_connection) {
3078+ db.reset();
3079+ connected = 0;
3080+ }
3081+ }
3082+ if (verbose >= 1) {
3083+ fprintf(stderr, "thread finished (error_count=%llu)\n", err_cnt);
3084+ }
3085+}
3086+
3087+void
3088+hstest_thread::test_21(int num)
3089+{
3090+ /* fsync test */
3091+ unsigned int id = arg.id;
3092+ std::string err;
3093+ #if 0
3094+ if (socket_connect(fd, arg.sh.arg, err) != 0) {
3095+ fprintf(stderr, "connect: %d %s\n", errno, strerror(errno));
3096+ return;
3097+ }
3098+ #endif
3099+ auto_file logfd;
3100+ char fname[1024];
3101+ snprintf(fname, sizeof(fname), "synctest_%u", id);
3102+ int open_flags = O_WRONLY | O_CREAT | O_TRUNC | O_APPEND;
3103+ logfd.reset(open(fname, open_flags, 0644));
3104+ if (logfd.get() < 0) {
3105+ fprintf(stderr, "open: %s: %d %s\n", fname, errno, strerror(errno));
3106+ return;
3107+ }
3108+ char buf[1024];
3109+ unsigned long long count = 0;
3110+ while (true) {
3111+ snprintf(buf, sizeof(buf), "%u %llu\n", id, count);
3112+ const size_t len = strlen(buf);
3113+ if (write(logfd.get(), buf, len) != (ssize_t)len) {
3114+ fprintf(stderr, "write: %s: %d %s\n", fname, errno, strerror(errno));
3115+ return;
3116+ }
3117+ #if 0
3118+ if (write(fd.get(), buf, len) != (ssize_t)len) {
3119+ fprintf(stderr, "write(sock): %d %s\n", errno, strerror(errno));
3120+ return;
3121+ }
3122+ #endif
3123+ if (fdatasync(logfd.get()) != 0) {
3124+ fprintf(stderr, "fsync: %s: %d %s\n", fname, errno, strerror(errno));
3125+ return;
3126+ }
3127+ ++count;
3128+ ++op_success_count;
3129+ arg.sh.increment_count();
3130+ }
3131+}
3132+
3133+void
3134+hstest_thread::test_22(int num)
3135+{
3136+ /* dd if=/dev/zero of=dummy.dat bs=1024M count=100 */
3137+ unsigned int id = arg.id;
3138+ std::string err;
3139+ auto_file filefd;
3140+ char fname[1024];
3141+ snprintf(fname, sizeof(fname), "dummy.dat");
3142+ int open_flags = O_RDONLY | O_DIRECT;
3143+ filefd.reset(open(fname, open_flags, 0644));
3144+ if (filefd.get() < 0) {
3145+ fprintf(stderr, "open: %s: %d %s\n", fname, errno, strerror(errno));
3146+ return;
3147+ }
3148+ char buf_x[4096 * 2];
3149+ char *const buf = (char *)(size_t(buf_x + 4096) / 4096 * 4096);
3150+ unsigned long long count = 0;
3151+ drand48_data randbuf;
3152+ unsigned long long seed = time(0);
3153+ seed *= 10;
3154+ seed += id;
3155+ srand48_r(seed, &randbuf);
3156+ for (unsigned int i = 0; i < arg.sh.loop; ++i) {
3157+ double kf = 0;
3158+ drand48_r(&randbuf, &kf);
3159+ kf *= (209715200 / 1);
3160+ // fprintf(stderr, "v=%f\n", kf);
3161+ off_t v = static_cast<off_t>(kf);
3162+ v %= (209715200 / 1);
3163+ v *= (512 * 1);
3164+ const double tm1 = gettimeofday_double();
3165+ const ssize_t r = pread(filefd.get(), buf, (512 * 1), v);
3166+ const double tm2 = gettimeofday_double();
3167+ if (r < 0) {
3168+ fprintf(stderr, "pread: %s: %d %s\n", fname, errno, strerror(errno));
3169+ return;
3170+ }
3171+ ++count;
3172+ ++op_success_count;
3173+ arg.sh.increment_count();
3174+ set_timing(tm2 - tm1);
3175+ }
3176+}
3177+
3178+void
3179+hstest_thread::operator ()()
3180+{
3181+ if (arg.watch_flag) {
3182+ return test_watch();
3183+ }
3184+ int test_num = arg.sh.conf.get_int("test", 1);
3185+ if (test_num == 1) {
3186+ test_1();
3187+ } else if (test_num == 2 || test_num == 3) {
3188+ test_2_3(test_num);
3189+ } else if (test_num == 4 || test_num == 5) {
3190+ test_4_5(test_num);
3191+ } else if (test_num == 6) {
3192+ test_6(test_num);
3193+ } else if (test_num == 7) {
3194+ test_7(test_num);
3195+ } else if (test_num == 8) {
3196+ test_8(test_num);
3197+ } else if (test_num == 9) {
3198+ test_9(test_num);
3199+ } else if (test_num == 10) {
3200+ test_10(test_num);
3201+ } else if (test_num == 11) {
3202+ test_11(test_num);
3203+ } else if (test_num == 12) {
3204+ test_12(test_num);
3205+ } else if (test_num == 21) {
3206+ test_21(test_num);
3207+ } else if (test_num == 22) {
3208+ test_22(test_num);
3209+ }
3210+ const int halt = arg.sh.conf.get_int("halt", 0);
3211+ if (halt) {
3212+ fprintf(stderr, "thread halted\n");
3213+ while (true) {
3214+ sleep(100000);
3215+ }
3216+ }
3217+ fprintf(stderr, "thread finished\n");
3218+}
3219+
3220+int
3221+hstest_main(int argc, char **argv)
3222+{
3223+ ignore_sigpipe();
3224+ hstest_shared shared;
3225+ parse_args(argc, argv, shared.conf);
3226+ shared.conf["port"] = shared.conf["hsport"];
3227+ shared.arg.set(shared.conf);
3228+ shared.loop = shared.conf.get_int("num", 1000);
3229+ shared.pipe = shared.conf.get_int("pipe", 1);
3230+ shared.verbose = shared.conf.get_int("verbose", 1);
3231+ const int tablesize = shared.conf.get_int("tablesize", 0);
3232+ std::vector<char> keygen(tablesize);
3233+ shared.keygen = &keygen[0];
3234+ shared.keygen_size = tablesize;
3235+ shared.usleep = shared.conf.get_int("usleep", 0);
3236+ shared.dump = shared.conf.get_int("dump", 0);
3237+ shared.num_threads = shared.conf.get_int("num_threads", 10);
3238+ shared.wait_conn = shared.conf.get_int("wait_conn", 0);
3239+ const std::string op = shared.conf.get_str("op", "G");
3240+ if (op.size() > 0) {
3241+ shared.op = op[0];
3242+ }
3243+ #if 0
3244+ const int localdb_flag = shared.conf.get_int("local", 0);
3245+ if (localdb_flag) {
3246+ shared.localdb = database_i::create(shared.conf);
3247+ }
3248+ #endif
3249+ const int num_thrs = shared.num_threads;
3250+ typedef thread<hstest_thread> thread_type;
3251+ typedef std::auto_ptr<thread_type> thread_ptr;
3252+ typedef auto_ptrcontainer< std::vector<thread_type *> > thrs_type;
3253+ thrs_type thrs;
3254+ for (int i = 0; i < num_thrs; ++i) {
3255+ const hstest_thread::arg_type arg(i, shared, false);
3256+ thread_ptr thr(new thread<hstest_thread>(arg));
3257+ thrs.push_back_ptr(thr);
3258+ }
3259+ for (size_t i = 0; i < thrs.size(); ++i) {
3260+ thrs[i]->start();
3261+ }
3262+ thread_ptr watch_thread;
3263+ const int timelimit = shared.conf.get_int("timelimit", 0);
3264+ {
3265+ const hstest_thread::arg_type arg(0, shared, true);
3266+ watch_thread = thread_ptr(new thread<hstest_thread>(arg));
3267+ watch_thread->start();
3268+ }
3269+ size_t iocnt = 0, opcnt = 0;
3270+ double respmin = 999999, respmax = 0;
3271+ double respsum = 0;
3272+ if (timelimit != 0) {
3273+ watch_thread->join();
3274+ }
3275+ for (size_t i = 0; i < thrs.size(); ++i) {
3276+ if (timelimit == 0) {
3277+ thrs[i]->join();
3278+ }
3279+ iocnt += (*thrs[i])->io_success_count;
3280+ opcnt += (*thrs[i])->op_success_count;
3281+ respmin = std::min(respmin, (*thrs[i])->response_min);
3282+ respmax = std::max(respmax, (*thrs[i])->response_max);
3283+ respsum += (*thrs[i])->response_sum;
3284+ }
3285+ fprintf(stderr, "io_success_count=%zu op_success_count=%zu\n", iocnt, opcnt);
3286+ fprintf(stderr, "respmin=%f respmax=%f respsum=%f respavg=%f\n",
3287+ respmin, respmax, respsum, respsum / opcnt);
3288+ size_t keycnt = 0;
3289+ for (size_t i = 0; i < keygen.size(); ++i) {
3290+ if (keygen[i]) {
3291+ ++keycnt;
3292+ }
3293+ }
3294+ fprintf(stderr, "keycnt=%zu\n", keycnt);
3295+ _exit(0);
3296+ return 0;
3297+}
3298+
3299+};
3300+
3301+int
3302+main(int argc, char **argv)
3303+{
3304+ return dena::hstest_main(argc, argv);
3305+}
3306+
3307
3308=== added file 'HandlerSocket-Plugin-for-MySQL/client/hstest.pl'
3309--- HandlerSocket-Plugin-for-MySQL/client/hstest.pl 1970-01-01 00:00:00 +0000
3310+++ HandlerSocket-Plugin-for-MySQL/client/hstest.pl 2011-06-08 01:57:28 +0000
3311@@ -0,0 +1,228 @@
3312+#!/usr/bin/perl
3313+
3314+# vim:sw=8:ai:ts=8
3315+
3316+use strict;
3317+use warnings;
3318+
3319+use DBI;
3320+use Net::HandlerSocket;
3321+
3322+my %conf = ();
3323+for my $i (@ARGV) {
3324+ my ($k, $v) = split(/=/, $i);
3325+ $conf{$k} = $v;
3326+}
3327+
3328+my $verbose = get_conf("verbose", 0);
3329+my $actions_str = get_conf("actions", "hsread");
3330+my $tablesize = get_conf("tablesize", 10000);
3331+my $db = get_conf("db", "hstest");
3332+my $table = get_conf("table", "hstest_table1");
3333+my $engine = get_conf("engine", "innodb");
3334+my $host = get_conf("host", "localhost");
3335+my $mysqlport = get_conf("mysqlport", 3306);
3336+my $mysqluser = get_conf("mysqluser", "root");
3337+my $mysqlpass = get_conf("mysqlpass", "");
3338+my $hsport = get_conf("hsport", 9999);
3339+my $loop = get_conf("loop", 10000);
3340+my $op = get_conf("op", "=");
3341+my $ssps = get_conf("ssps", 0);
3342+my $num_moreflds = get_conf("moreflds", 0);
3343+my $moreflds_prefix = get_conf("moreflds_prefix", "column0123456789_");
3344+my $keytype = get_conf("keytype", "varchar(32)");
3345+my $file = get_conf("file", undef);
3346+
3347+my $dsn = "DBI:mysql:database=;host=$host;port=$mysqlport"
3348+ . ";mysql_server_prepare=$ssps";
3349+my $dbh = DBI->connect($dsn, $mysqluser, $mysqlpass, { RaiseError => 1 });
3350+my $hsargs = { 'host' => $host, 'port' => $hsport };
3351+my $cli = new Net::HandlerSocket($hsargs);
3352+
3353+my @actions = split(/,/, $actions_str);
3354+for my $action (@actions) {
3355+ if ($action eq "table") {
3356+ print("TABLE $db.$table\n");
3357+ $dbh->do("drop database if exists $db");
3358+ $dbh->do("create database $db");
3359+ $dbh->do("use $db");
3360+ my $moreflds = get_createtbl_moreflds_str();
3361+ $dbh->do(
3362+ "create table $table (" .
3363+ "k $keytype primary key" .
3364+ ",v varchar(32) not null" .
3365+ $moreflds .
3366+ ") character set utf8 collate utf8_bin " .
3367+ "engine = $engine");
3368+ } elsif ($action eq "insert") {
3369+ print("INSERT $db.$table tablesize=$tablesize\n");
3370+ $dbh->do("use $db");
3371+ my $moreflds = get_insert_moreflds_str();
3372+ for (my $i = 0; $i < $tablesize; $i += 100) {
3373+ my $qstr = "insert into $db.$table values";
3374+ for (my $j = 0; $j < 100; ++$j) {
3375+ if ($j != 0) {
3376+ $qstr .= ",";
3377+ }
3378+ my $k = "" . ($i + $j);
3379+ my $v = "v" . int(rand(1000)) . ($i + $j);
3380+ $qstr .= "('$k', '$v'";
3381+ for (my $j = 0; $j < $num_moreflds; ++$j) {
3382+ $qstr .= ",'$j'";
3383+ }
3384+ $qstr .= ")";
3385+ }
3386+ $dbh->do($qstr);
3387+ print "$i/$tablesize\n" if $i % 1000 == 0;
3388+ }
3389+ } elsif ($action eq "read") {
3390+ print("READ $db.$table op=$op loop=$loop\n");
3391+ $dbh->do("use $db");
3392+ my $moreflds = get_select_moreflds_str();
3393+ my $sth = $dbh->prepare(
3394+ "select k,v$moreflds from $db.$table where k = ?");
3395+ for (my $i = 0; $i < $loop; ++$i) {
3396+ my $k = "" . int(rand($tablesize));
3397+ # print "k=$k\n";
3398+ $sth->execute($k);
3399+ if ($verbose >= 10) {
3400+ print "RET:";
3401+ while (my $ref = $sth->fetchrow_arrayref()) {
3402+ my $rk = $ref->[0];
3403+ my $rv = $ref->[1];
3404+ print " $rk $rv";
3405+ }
3406+ print "\n";
3407+ }
3408+ print "$i/$loop\n" if $i % 1000 == 0;
3409+ }
3410+ } elsif ($action eq "hsinsert") {
3411+ print("HSINSERT $db.$table tablesize=$tablesize\n");
3412+ $cli->open_index(1, $db, $table, '', 'k,v');
3413+ for (my $i = 0; $i < $tablesize; ++$i) {
3414+ my $k = "" . $i;
3415+ my $v = "v" . int(rand(1000)) . $i;
3416+ my $r = $cli->execute_insert(1, [ $k, $v ]);
3417+ if ($r->[0] != 0) {
3418+ die;
3419+ }
3420+ print "$i/$tablesize\n" if $i % 1000 == 0;
3421+ }
3422+ } elsif ($action eq "hsread") {
3423+ print("HSREAD $db.$table op=$op loop=$loop\n");
3424+ my $moreflds = get_select_moreflds_str();
3425+ $cli->open_index(1, $db, $table, '', "k,v$moreflds");
3426+ for (my $i = 0; $i < $loop; ++$i) {
3427+ my $k = "" . int(rand($tablesize));
3428+ # print "k=$k\n";
3429+ my $r = $cli->execute_find(1, $op, [ $k ], 1, 0);
3430+ if ($verbose >= 10) {
3431+ my $len = scalar(@{$r});
3432+ print "LEN=$len";
3433+ for my $e (@{$r}) {
3434+ print " [$e]";
3435+ }
3436+ print "\n";
3437+ }
3438+ print "$i/$loop\n" if $i % 1000 == 0;
3439+ }
3440+ } elsif ($action eq "hsupdate") {
3441+ my $vbase = "v" . int(rand(1000));
3442+ print("HSUPDATE $db.$table op=$op loop=$loop vbase=$vbase\n");
3443+ $cli->open_index(1, $db, $table, '', 'v');
3444+ for (my $i = 0; $i < $loop; ++$i) {
3445+ my $k = "" . int(rand($tablesize));
3446+ my $v = $vbase . $i;
3447+ print "k=$k v=$v\n";
3448+ my $r = $cli->execute_update(1, $op, [ $k ], 1, 0,
3449+ [ $v ]);
3450+ if ($verbose >= 10) {
3451+ print "UP k=$k v=$v\n";
3452+ }
3453+ print "$i/$loop\n" if $i % 1000 == 0;
3454+ }
3455+ } elsif ($action eq "hsdelete") {
3456+ print("HSDELETE $db.$table op=$op loop=$loop\n");
3457+ $cli->open_index(1, $db, $table, '', '');
3458+ for (my $i = 0; $i < $loop; ++$i) {
3459+ my $k = "" . int(rand($tablesize));
3460+ print "k=$k\n";
3461+ my $r = $cli->execute_delete(1, $op, [ $k ], 1, 0);
3462+ if ($verbose >= 10) {
3463+ print "DEL k=$k\n";
3464+ }
3465+ print "$i/$loop\n" if $i % 1000 == 0;
3466+ }
3467+ } elsif ($action eq "verify") {
3468+ verify_do();
3469+ }
3470+}
3471+
3472+sub verify_do {
3473+ my ($fail_cnt, $ok_cnt) = (0, 0);
3474+ my $sth = $dbh->prepare("select v from $db.$table where k = ?");
3475+ use FileHandle;
3476+ my $fh = new FileHandle($file, "r");
3477+ while (my $line = <$fh>) {
3478+ chomp($line);
3479+ my @vec = split(/\t/, $line);
3480+ my $k = $vec[3];
3481+ my $v = $vec[7];
3482+ next if (!defined($k) || !defined($v));
3483+ # print "$k $v\n";
3484+ $sth->execute($k);
3485+ my $aref = $sth->fetchrow_arrayref();
3486+ if (!defined($aref)) {
3487+ print "FAILED: $k notfound\n";
3488+ ++$fail_cnt;
3489+ } else {
3490+ my $gv = $aref->[0];
3491+ if ($gv ne $v) {
3492+ print "FAILED: $k got=$gv expected=$v\n";
3493+ ++$fail_cnt;
3494+ } else {
3495+ print "OK: $k $v $gv\n" if $verbose >= 10;
3496+ ++$ok_cnt;
3497+ }
3498+ }
3499+ }
3500+ print "OK=$ok_cnt FAIL=$fail_cnt\n";
3501+}
3502+
3503+sub get_conf {
3504+ my ($key, $def) = @_;
3505+ my $val = $conf{$key};
3506+ if ($val) {
3507+ print "$key=$val\n";
3508+ } else {
3509+ $val = $def;
3510+ $def ||= '';
3511+ print "$key=$def(default)\n";
3512+ }
3513+ return $val;
3514+}
3515+
3516+sub get_createtbl_moreflds_str {
3517+ my $s = "";
3518+ for (my $j = 0; $j < $num_moreflds; ++$j) {
3519+ $s .= ",$moreflds_prefix$j varchar(30)";
3520+ }
3521+ return $s;
3522+}
3523+
3524+sub get_select_moreflds_str {
3525+ my $s = "";
3526+ for (my $i = 0; $i < $num_moreflds; ++$i) {
3527+ $s .= ",$moreflds_prefix$i";
3528+ }
3529+ return $s;
3530+}
3531+
3532+sub get_insert_moreflds_str {
3533+ my $s = "";
3534+ for (my $i = 0; $i < $num_moreflds; ++$i) {
3535+ $s .= ",?";
3536+ }
3537+ return $s;
3538+}
3539+
3540
3541=== added file 'HandlerSocket-Plugin-for-MySQL/client/hstest_hs.sh'
3542--- HandlerSocket-Plugin-for-MySQL/client/hstest_hs.sh 1970-01-01 00:00:00 +0000
3543+++ HandlerSocket-Plugin-for-MySQL/client/hstest_hs.sh 2011-06-08 01:57:28 +0000
3544@@ -0,0 +1,4 @@
3545+#!/bin/bash
3546+
3547+exec ./hstest test=10 tablesize=10000 host=localhost hsport=9998 num=10000000 \
3548+ num_threads=100 timelimit=10 $@
3549
3550=== added file 'HandlerSocket-Plugin-for-MySQL/client/hstest_hs_more50.sh'
3551--- HandlerSocket-Plugin-for-MySQL/client/hstest_hs_more50.sh 1970-01-01 00:00:00 +0000
3552+++ HandlerSocket-Plugin-for-MySQL/client/hstest_hs_more50.sh 2011-06-08 01:57:28 +0000
3553@@ -0,0 +1,4 @@
3554+#!/bin/bash
3555+
3556+exec ./hstest test=10 key_mask=9999 host=localhost port=9998 num=10000000 \
3557+ num_threads=100 timelimit=10 moreflds=50 $@
3558
3559=== added file 'HandlerSocket-Plugin-for-MySQL/client/hstest_md.sh'
3560--- HandlerSocket-Plugin-for-MySQL/client/hstest_md.sh 1970-01-01 00:00:00 +0000
3561+++ HandlerSocket-Plugin-for-MySQL/client/hstest_md.sh 2011-06-08 01:57:28 +0000
3562@@ -0,0 +1,7 @@
3563+#!/bin/bash
3564+
3565+./hstest test=7 key_mask=9999 host=localhost port=11211 num=10000 \
3566+ num_threads=10 timelimit=10 op=R $@
3567+./hstest test=7 key_mask=9999 host=localhost port=11211 num=1000000 \
3568+ num_threads=100 timelimit=10 op=G $@
3569+
3570
3571=== added file 'HandlerSocket-Plugin-for-MySQL/client/hstest_my.sh'
3572--- HandlerSocket-Plugin-for-MySQL/client/hstest_my.sh 1970-01-01 00:00:00 +0000
3573+++ HandlerSocket-Plugin-for-MySQL/client/hstest_my.sh 2011-06-08 01:57:28 +0000
3574@@ -0,0 +1,3 @@
3575+#!/bin/bash
3576+exec ./hstest test=9 tablesize=9999 host=localhost mysqlport=3306 num=1000000 \
3577+ num_threads=100 verbose=1 timelimit=10 $@
3578
3579=== added file 'HandlerSocket-Plugin-for-MySQL/client/hstest_my_more50.sh'
3580--- HandlerSocket-Plugin-for-MySQL/client/hstest_my_more50.sh 1970-01-01 00:00:00 +0000
3581+++ HandlerSocket-Plugin-for-MySQL/client/hstest_my_more50.sh 2011-06-08 01:57:28 +0000
3582@@ -0,0 +1,3 @@
3583+#!/bin/bash
3584+exec ./hstest test=9 key_mask=9999 host=localhost port=3306 num=1000000 \
3585+ num_threads=100 verbose=1 timelimit=10 moreflds=50 $@
3586
3587=== added file 'HandlerSocket-Plugin-for-MySQL/configure.ac'
3588--- HandlerSocket-Plugin-for-MySQL/configure.ac 1970-01-01 00:00:00 +0000
3589+++ HandlerSocket-Plugin-for-MySQL/configure.ac 2011-06-08 01:57:28 +0000
3590@@ -0,0 +1,144 @@
3591+# -*- Autoconf -*-
3592+# Process this file with autoconf to produce a configure script.
3593+
3594+#AC_PREREQ([2.63b])
3595+AC_INIT([handlersocket-plugin], [1.0.6], [https://github.com/ahiguti/HandlerSocket-Plugin-for-MySQL/issues])
3596+AC_CONFIG_HEADERS([config.h])
3597+AM_INIT_AUTOMAKE([-Wall -Werror foreign])
3598+AC_CONFIG_SRCDIR([libhsclient/fatal.cpp])
3599+AC_CONFIG_MACRO_DIR([m4])
3600+
3601+AC_PROG_CC
3602+AC_PROG_CXX
3603+AC_PROG_CPP
3604+AC_PROG_LIBTOOL
3605+
3606+ac_mysql_debug=
3607+AC_ARG_ENABLE(mysql-debug,
3608+ [AS_HELP_STRING([--enable-mysql-debug], [specify whether MySQL is build with DBUG_ON])],[ac_mysql_debug="$enableval"],[ac_mysql_debug=no])
3609+AC_MSG_CHECKING([if --enable-mysql-debug is specified])
3610+AC_MSG_RESULT($ac_mysql_debug)
3611+
3612+AC_DEFUN([CONFIG_OPTION_MYSQL],[
3613+ AC_MSG_CHECKING([mysql source])
3614+
3615+ MYSQL_SOURCE_VERSION=
3616+ MYSQL_INC=
3617+ ac_mysql_source_dir=
3618+ AC_ARG_WITH([mysql-source],
3619+ [AS_HELP_STRING([--with-mysql-source=PATH], [MySQL source directory PATH])],
3620+ [
3621+ ac_mysql_source_dir=`cd $withval && pwd`
3622+ if test -f "$ac_mysql_source_dir/sql/handler.h" ; then
3623+ MYSQL_INC="-I$ac_mysql_source_dir/sql"
3624+ MYSQL_INC="$MYSQL_INC -I$ac_mysql_source_dir/include"
3625+ MYSQL_INC="$MYSQL_INC -I$ac_mysql_source_dir/regex"
3626+ MYSQL_INC="$MYSQL_INC -I$ac_mysql_source_dir"
3627+ AC_SUBST(MYSQL_INC)
3628+ if test -f "$ac_mysql_source_dir/VERSION"; then
3629+ source "$ac_mysql_source_dir/VERSION"
3630+ MYSQL_SOURCE_VERSION="$MYSQL_VERSION_MAJOR.$MYSQL_VERSION_MINOR.$MYSQL_VERSION_PATCH"
3631+ else
3632+ if test -f "$ac_mysql_source_dir/configure.in"; then
3633+ MYSQL_SOURCE_VERSION=`cat $ac_mysql_source_dir/configure.in | grep "\[[MySQL Server\]]" | sed -e "s|.*\([[0-9]]\+\.[[0-9]]\+\.[[0-9]]\+[[0-9a-zA-Z\_\-]]*\).*|\1|"`
3634+ else
3635+ AC_MSG_ERROR([invalid MySQL source directory: $ac_mysql_source_dir])
3636+ fi
3637+ fi
3638+ AC_MSG_RESULT([yes: Using $ac_mysql_source_dir, version $MYSQL_SOURCE_VERSION])
3639+ else
3640+ AC_MSG_ERROR([invalid MySQL source directory: $ac_mysql_source_dir])
3641+ fi
3642+ ],
3643+ [AC_MSG_ERROR([--with-mysql-source=PATH is required for standalone build])]
3644+ )
3645+
3646+ MYSQL_BIN_VERSION=
3647+ ac_mysql_config=
3648+ AC_ARG_WITH([mysql-bindir],
3649+ [AS_HELP_STRING([--with-mysql-bindir=PATH], [MySQL binary directory PATH. This should be the directory where mysql_config is located.])],
3650+ [
3651+ mysql_bin_dir=`cd $withval 2> /dev/null && pwd || echo ""`
3652+ ac_mysql_config="$mysql_bin_dir/mysql_config"
3653+ ],
3654+ [
3655+ AC_PATH_PROG([ac_mysql_config], [mysql_config])
3656+ ]
3657+ )
3658+
3659+ AC_MSG_CHECKING([mysql binary])
3660+ if test ! -x "$ac_mysql_config" ; then
3661+ AC_MSG_ERROR([mysql_config not found! You have to specify the directory where mysql_config resides to --with-mysql-bindir=PATH.])
3662+ fi
3663+
3664+ MYSQL_CFLAGS_ADD=`"$ac_mysql_config" --cflags`
3665+ MYSQL_CFLAGS="$MYSQL_CFLAGS $MYSQL_CFLAGS_ADD"
3666+ if test "$ac_mysql_debug" = "yes"; then
3667+ MYSQL_CFLAGS="$MYSQL_CFLAGS -DDBUG_ON -DENABLED_DEBUG_SYNC"
3668+ else
3669+ MYSQL_CFLAGS="$MYSQL_CFLAGS -DDBUG_OFF"
3670+ fi
3671+ AC_SUBST(MYSQL_CFLAGS)
3672+
3673+ MYSQL_BIN_VERSION=`"$ac_mysql_config" --version`
3674+ AC_MSG_RESULT([yes: Using $ac_mysql_config, version $MYSQL_BIN_VERSION])
3675+
3676+ MYSQL_LIB=`"$ac_mysql_config" --libs_r`
3677+ LIB_DIR=`echo $MYSQL_LIB | sed -e "s|.*-L/|/|" | sed -e "s| .*||"`
3678+ # FIXME
3679+ if test a`basename "$LIB_DIR"` = amysql ; then
3680+ MYSQL_LIB="-L`dirname $LIB_DIR` $MYSQL_LIB"
3681+ # FIXME
3682+ fi
3683+ AC_SUBST(MYSQL_LIB)
3684+
3685+ if test a$MYSQL_SOURCE_VERSION != a$MYSQL_BIN_VERSION ; then
3686+ AC_MSG_ERROR([MySQL source version does not match MySQL binary version])
3687+ fi
3688+
3689+ AC_MSG_CHECKING([mysql plugin dir])
3690+ ac_mysql_plugin_dir=
3691+ AC_ARG_WITH([mysql-plugindir],
3692+ [AS_HELP_STRING([--with-mysql-plugindir=PATH], [MySQL plugin directory where handlersocket.so to be copied])],
3693+ [
3694+ ac_mysql_plugin_dir=`cd $withval && pwd`
3695+ if test -d "$ac_mysql_plugin_dir/" ; then
3696+ PLUGIN_DIR="$ac_mysql_plugin_dir"
3697+ AC_SUBST(PLUGIN_DIR)
3698+ AC_MSG_RESULT([yes: Using $ac_mysql_plugin_dir])
3699+ else
3700+ AC_MSG_ERROR([invalid MySQL plugin directory : $ac_mysql_plugin_dir])
3701+ fi
3702+ ],
3703+ [
3704+ LIB_DIR_TMP=`"$ac_mysql_config" --plugindir`
3705+ if test ! -d "$LIB_DIR_TMP"; then
3706+ LIB_DIR_TMP=`"$ac_mysql_config" --libs_r | sed -e "s|.*-L/|/|" | sed -e "s| .*||"`/plugin
3707+ # FIXME
3708+ fi
3709+ ac_mysql_plugin_dir=$LIB_DIR_TMP
3710+ PLUGIN_DIR="$ac_mysql_plugin_dir"
3711+ AC_SUBST(PLUGIN_DIR)
3712+ AC_MSG_RESULT([--with-mysql-plugindir was not set. Using $ac_mysql_plugin_dir])
3713+ ]
3714+ )
3715+])
3716+
3717+HANDLERSOCKET_SUBDIRS="libhsclient"
3718+AC_ARG_ENABLE(handlersocket_server,
3719+ [ --enable-handlersocket-server build HandlerSocket plugin (defalut=yes)])
3720+if test "$enable_handlersocket_server" != "no"; then
3721+ CONFIG_OPTION_MYSQL
3722+ HANDLERSOCKET_SUBDIRS="libhsclient handlersocket client"
3723+fi
3724+AC_SUBST(HANDLERSOCKET_SUBDIRS)
3725+
3726+CFLAGS="$CFLAGS -Werror"
3727+CXXFLAGS="$CXXFLAGS -Wall -g -fno-rtti -fno-exceptions -fPIC -DPIC"
3728+
3729+AC_CONFIG_FILES([Makefile
3730+ handlersocket/Makefile
3731+ libhsclient/Makefile
3732+ client/Makefile])
3733+
3734+AC_OUTPUT
3735
3736=== added directory 'HandlerSocket-Plugin-for-MySQL/docs-en'
3737=== added file 'HandlerSocket-Plugin-for-MySQL/docs-en/about-handlersocket.en.txt'
3738--- HandlerSocket-Plugin-for-MySQL/docs-en/about-handlersocket.en.txt 1970-01-01 00:00:00 +0000
3739+++ HandlerSocket-Plugin-for-MySQL/docs-en/about-handlersocket.en.txt 2011-06-08 01:57:28 +0000
3740@@ -0,0 +1,72 @@
3741+
3742+-----------------------------------------------------------------------------
3743+HandlerSocket plugin for MySQL
3744+
3745+Copyright (c) 2010 DeNA Co.,Ltd.
3746+All rights reserved.
3747+
3748+Redistribution and use in source and binary forms, with or without
3749+modification, are permitted provided that the following conditions are met:
3750+
3751+ * Redistributions of source code must retain the above copyright
3752+ notice, this list of conditions and the following disclaimer.
3753+ * Redistributions in binary form must reproduce the above copyright
3754+ notice, this list of conditions and the following disclaimer in the
3755+ documentation and/or other materials provided with the distribution.
3756+ * Neither the name of DeNA Co.,Ltd. nor the names of its contributors
3757+ may be used to endorse or promote products derived from this software
3758+ without specific prior written permission.
3759+
3760+THIS SOFTWARE IS PROVIDED BY DeNA Co.,Ltd. "AS IS" AND ANY EXPRESS OR
3761+IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
3762+MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO
3763+EVENT SHALL DeNA Co.,Ltd. BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
3764+SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
3765+PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS;
3766+OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
3767+WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
3768+OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
3769+ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
3770+
3771+
3772+-----------------------------------------------------------------------------
3773+About HandlerSocket
3774+
3775+HandlerSocket is a NoSQL plugin for MySQL. It works as a daemon inside the
3776+mysqld process, accept tcp connections, and execute requests from clients.
3777+HandlerSocket does not support SQL queries. Instead, it supports simple CRUD
3778+operations on tables.
3779+
3780+Because of the following reasons, HandlerSocket is much faster than the
3781+mysqld/libmysql pair in some circumstances:
3782+
3783+ - HandlerSocket manipulates data without parsing SQL, which causes less
3784+ CPU usage.
3785+ - HandlerSocket reads many requests from clients and executes their
3786+ requests in bulk, which causes less CPU and disk usage.
3787+ - HandlerSocket client/server protocol is more compact than the
3788+ mysql/libmysql pair, which causes less network usage.
3789+
3790+The current version of HandlerSocket only works with GNU/Linux. The source
3791+archive of HandlerSocket includes a C++ and a Perl client libraries.
3792+Here is a list of other language bindings:
3793+
3794+ - PHP
3795+ http://openpear.org/package/Net_HandlerSocket
3796+ http://github.com/tz-lom/HSPHP
3797+ http://code.google.com/p/php-handlersocket/
3798+ - Java
3799+ http://code.google.com/p/handlersocketforjava/
3800+ - Python
3801+ https://code.launchpad.net/~songofacandy/+junk/pyhandlersocket
3802+ - Ruby
3803+ https://github.com/winebarrel/ruby-handlersocket
3804+ https://github.com/miyucy/handlersocket
3805+ - JavaScript(Node.js)
3806+ https://github.com/koichik/node-handlersocket
3807+
3808+The home of HandlerSocket is here:
3809+ https://github.com/ahiguti/HandlerSocket-Plugin-for-MySQL
3810+
3811+More documents are available in docs-en/ and docs-ja/ directories.
3812+
3813
3814=== added file 'HandlerSocket-Plugin-for-MySQL/docs-en/configuration-options.en.txt'
3815--- HandlerSocket-Plugin-for-MySQL/docs-en/configuration-options.en.txt 1970-01-01 00:00:00 +0000
3816+++ HandlerSocket-Plugin-for-MySQL/docs-en/configuration-options.en.txt 2011-06-08 01:57:28 +0000
3817@@ -0,0 +1,87 @@
3818+
3819+-----------------------------------------------------------------
3820+handlersocket_verbose (default = 10, min = 0, max = 10000)
3821+
3822+ Specify the logging verboseness.
3823+
3824+-----------------------------------------------------------------
3825+handlersocket_address (default = '')
3826+
3827+ Specify the address to bind. If empty, it binds to 0.0.0.0.
3828+
3829+-----------------------------------------------------------------
3830+handlersocket_port (default = '9998')
3831+
3832+ Specify the port to bind. This option is for the listener for
3833+ read requests. If empty, the listener is disabled.
3834+
3835+-----------------------------------------------------------------
3836+handlersocket_port_wr (default = '9999')
3837+
3838+ Specify the port to bind. This option is for the listener for
3839+ write requests. If empty, the listener is disabled.
3840+
3841+-----------------------------------------------------------------
3842+handlersocket_epoll (default = 1, min = 0, max = 1)
3843+
3844+ Specify whether handlersocket uses epoll for I/O multiplexing.
3845+
3846+-----------------------------------------------------------------
3847+handlersocket_threads (default = 16, min = 1, max = 3000)
3848+
3849+ Specify the number of handlersocket worker threads. This option
3850+ is for the listener for read requests. Recommended value is
3851+ (the number of CPU cores * 2).
3852+
3853+-----------------------------------------------------------------
3854+handlersocket_threads_wr (default = 1, min = 1, max = 3000)
3855+
3856+ Specify the number of handlersocket worker threads. This option
3857+ is for the listener for write requests. Recommended value is 1.
3858+
3859+-----------------------------------------------------------------
3860+handlersocket_timeout (default = 300, min = 30, max = 3600)
3861+
3862+ Specify the socket timeout in seconds.
3863+
3864+-----------------------------------------------------------------
3865+handlersocket_backlog (default = 32768, min = 5, max = 1000000)
3866+
3867+ Specify the length of the listen backlog.
3868+
3869+-----------------------------------------------------------------
3870+handlersocket_sndbuf (default = 0, min = 0, max = 1677216)
3871+
3872+ Specify the maximum socket send buffer in bytes. If 0, the
3873+ system-wide default value is set.
3874+
3875+-----------------------------------------------------------------
3876+handlersocket_rcvbuf (default = 0, min = 0, max = 1677216)
3877+
3878+ Specify the maximum socket receive buffer in bytes. If 0, the
3879+ system-wide default value is set.
3880+
3881+-----------------------------------------------------------------
3882+handlersocket_readsize (default = 0, min = 0, max = 1677216)
3883+
3884+ Specify the minimum length of the handlersocket request buffer.
3885+ Larger value can make handlersocket faster for large requests,
3886+ but can consume memory. The default value is possibly 4096.
3887+
3888+-----------------------------------------------------------------
3889+handlersocket_accept_balance (default = 0, min = 0, max = 10000)
3890+
3891+ When this option is set to non-zero, handlersocket tries to
3892+ balance accepted connections among threads. Non-zero is
3893+ recommended if you use persistent connections (i.e., connection
3894+ pooling on the client side).
3895+
3896+-----------------------------------------------------------------
3897+handlersocket_wrlock_timeout (default = 12, min = 0, max = 3600)
3898+
3899+ Specify the lock timeout in seconds. When a write request is
3900+ performed, handlersocket locks an advisory lock named
3901+ 'handlersocket_wr'. This option sets the timeout for the
3902+ locking.
3903+
3904+
3905
3906=== added file 'HandlerSocket-Plugin-for-MySQL/docs-en/installation.en.txt'
3907--- HandlerSocket-Plugin-for-MySQL/docs-en/installation.en.txt 1970-01-01 00:00:00 +0000
3908+++ HandlerSocket-Plugin-for-MySQL/docs-en/installation.en.txt 2011-06-08 01:57:28 +0000
3909@@ -0,0 +1,91 @@
3910+1. Building Handlersocket
3911+
3912+ Handlersocket mainly consists of libhsclient, handlersocket, and C++/Perl clients. libhsclient is a common library shared from both client and server(plugin). handlersocket is a MySQL daemon plugin.
3913+ To build Handlersocket, you need both MySQL source code and MySQL binary. It is not required to pre-build MySQL source code, but source itself is needed because Handlersocket depends on MySQL header files that only MySQL source distribution contains. MySQL binary is just a normal MySQL binary distribution. You can use official MySQL binaries provided by Oracle.
3914+ Since Handlersocket uses daemon plugin interface supported from MySQL 5.1,
3915+MySQL 5.1 or higher version is required.
3916+ Please make sure that you use identical MySQL version between MySQL source
3917+and MySQL binary. Otherwise you might encounter serious problems (i.e. server
3918+crash, etc).
3919+ Here are steps to build Handlersocket.
3920+
3921+* Get MySQL source code
3922+
3923+* Get MySQL binary
3924+
3925+* Build Handlersocket
3926+ $ ./autogen.sh
3927+ $ ./configure --with-mysql-source=/work/mysql-5.1.50 --with-mysql-bindir=/work/mysql-5.1.50-linux-x86_64-glibc23/bin --with-mysql-plugindir=/work/mysql-5.1.50-linux-x86_64-glibc23/lib/plugin
3928+
3929+ --with-mysql-source refers to the top of MySQL source directory,
3930+--with-mysql-bindir refers to where MySQL binary executables (i.e.
3931+mysql_config) are located, and --with-mysql-plugindir refers to a plugin
3932+directory where plugin libraries (*.so) are installed.
3933+
3934+ $ make
3935+ $ sudo make install
3936+
3937+ Both libhsclient and the handlersocket plugin will be installed.
3938+
3939+
3940+2. Using Handlersocket
3941+
3942+Append configuration options for handlersocket to my.cnf.
3943+
3944+ [mysqld]
3945+ loose_handlersocket_port = 9998
3946+ # the port number to bind to (for read requests)
3947+ loose_handlersocket_port_wr = 9999
3948+ # the port number to bind to (for write requests)
3949+ loose_handlersocket_threads = 16
3950+ # the number of worker threads (for read requests)
3951+ loose_handlersocket_threads_wr = 1
3952+ # the number of worker threads (for write requests)
3953+ open_files_limit = 65535
3954+ # to allow handlersocket accept many concurrent
3955+ # connections, make open_files_limit as large as
3956+ # possible.
3957+
3958+Log in to mysql as root, and execute the following query.
3959+
3960+ mysql> install plugin handlersocket soname 'handlersocket.so';
3961+
3962+If handlersocket.so is successfully installed, it starts
3963+accepting connections on port 9998 and 9999. Running
3964+'show processlist' should show handlersocket worker threads.
3965+
3966+-----------------------------------------------------------------
3967+On the client side, you need to install libhsclient for c++ apps
3968+and perl-Net-HandlerSocket for perl apps. They do not require
3969+MySQL to compile.
3970+
3971+ $ ./autogen.sh
3972+ $ ./configure --disable-handlersocket-server
3973+ $ make
3974+ $ sudo make install
3975+ $ cd perl-Net-HandlerSocket
3976+ $ perl Makefile.PL
3977+ $ make
3978+ $ sudo make install
3979+
3980+-----------------------------------------------------------------
3981+Alternatively, you can use the rpm installation. If your OS
3982+supports rpms, you can use the following commands to build and
3983+install handlersocket rpm packages.
3984+
3985+(Server side, installs HandlerSocket plugin)
3986+ $ ./autogen.sh
3987+ $ ./configure --with-mysql-source=/work/mysql-5.1.50 --with-mysql-bindir=/work/mysql-5.1.50-linux-x86_64-glibc23/bin --with-mysql-plugindir=/work/mysql-5.1.50-linux-x86_64-glibc23/lib/plugin
3988+ $ make rpm_cli
3989+ $ sudo rpm -U dist/RPMS/*/libhsclient*.rpm
3990+ $ make rpm_c
3991+ $ sudo rpm -U dist/RPMS/*/handlersocket*.rpm
3992+
3993+(Client side, installs client libraries)
3994+ $ ./autogen.sh
3995+ $ ./configure --disable-handlersocket-server
3996+ $ make rpm_cli
3997+ $ sudo rpm -U dist/RPMS/*/libhsclient*.rpm
3998+ $ make rpm_perl
3999+ $ sudo rpm -U dist/RPMS/*/perl-Net-HandlerSocket*.rpm
4000+
4001
4002=== added file 'HandlerSocket-Plugin-for-MySQL/docs-en/perl-client.en.txt'
4003--- HandlerSocket-Plugin-for-MySQL/docs-en/perl-client.en.txt 1970-01-01 00:00:00 +0000
4004+++ HandlerSocket-Plugin-for-MySQL/docs-en/perl-client.en.txt 2011-06-08 01:57:28 +0000
4005@@ -0,0 +1,126 @@
4006+
4007+-----------------------------------------------------------------
4008+To open a connection to the handlersocket plugin, you need to
4009+create a Net::HandlerSocket object.
4010+
4011+ use Net::HandlerSocket;
4012+ my $args = { host => 'localhost', port => 9998 };
4013+ my $hs = new Net::HandlerSocket($args);
4014+
4015+-----------------------------------------------------------------
4016+Before executing table operations, you need to open an index to
4017+work with.
4018+
4019+ my $err = $hs->open_index(3, 'database1', 'table1', 'PRIMARY',
4020+ 'f1,f2');
4021+ die $hs->get_error() if $res->[0] != 0;
4022+
4023+The first argument for open_index is an integer value which is
4024+used to identify an open table, which is only valid within the
4025+same Net::HandlerSocket object. The 4th argument is the name of
4026+index to open. If 'PRIMARY' is specified, the primary index is
4027+open. The 5th argument is a comma-separated list of column names.
4028+
4029+-----------------------------------------------------------------
4030+To read a record from a table using an index, call the
4031+execute_single method.
4032+
4033+ my $res = $hs->execute_single(3, '=', [ 'foo' ], 1, 0);
4034+ die $hs->get_error() if $res->[0] != 0;
4035+ shift(@$res);
4036+
4037+The first argument must be an integer which has specified as the
4038+first argument for open_index on the same Net::HandlerSocket
4039+object. The second argument specifies the search operation. The
4040+current version of handlersocket supports '=', '>=', '<=', '>',
4041+and '<'. The 3rd argument specifies the key to find, which must
4042+an arrayref whose length is equal to or smaller than the number
4043+of key columns of the index. The 4th and the 5th arguments
4044+specify the maximum number of records to be retrieved, and the
4045+number of records skipped before retrieving records. The columns
4046+to be retrieved are specified by the 5th argument for the
4047+corresponding open_index call.
4048+
4049+The execute_single method always returns an arrayref. The first
4050+element is the error code, which is 0 when no error is occured.
4051+The remaining are the field values. If more than one record is
4052+returned, it is flatten to an 1-dimensional array. For example,
4053+when 5 records that have 3 columns are returned, you can retrieve
4054+values using the following code.
4055+
4056+ die $hs->get_error() if $res->[0] != 0;
4057+ shift(@$res);
4058+ for (my $row = 0; $row < 5; ++$row) {
4059+ for (my $col = 0; $col < 3; ++$col) {
4060+ my $value = $res->[$row * 5 + $col];
4061+ # ...
4062+ }
4063+ }
4064+
4065+-----------------------------------------------------------------
4066+To update or delete records, you need to specify more arguments
4067+for the execute_single method. Note that the Net::HandlerSocket
4068+object must be connected to a handlersocket worker for write
4069+operations, which is port 9999 by default.
4070+(For safety, the port 9998 only allows read operations, and the
4071+port 9999 allows write operations also. The port 9999 allows
4072+read operations too, but slower than 9998 because of record
4073+locking etc.. Port numbers can be changed using the
4074+'handlersocket_port' and the 'handlersocket_port_wr'
4075+configuration options of mysqld.)
4076+
4077+ my $args = { host => 'localhost', port => 9999 };
4078+ my $hs = new Net::HandlerSocket($args);
4079+
4080+ my $res = $hs->execute_single(3, '=', [ 'bar' ], 1, 0, 'U',
4081+ [ 'fubar', 'hoge' ]);
4082+ die $hs->get_error() if $res->[0] != 0;
4083+ my $num_updated_rows = $res->[1];
4084+
4085+ my $res = $hs->execute_single(3, '=', [ 'baz' ], 1, 0, 'D');
4086+ die $hs->get_error() if $res->[0] != 0;
4087+ my $num_deleted_rows = $res->[1];
4088+
4089+The 6th argument for execute_single specifies the modification
4090+operation. The current version supports 'U' and 'D'. For the 'U'
4091+operation, the 7th argument specifies the new value for the row.
4092+The columns to be modified are specified by the 5th argument for
4093+the corresponding open_index call. For the 'D' operation, the
4094+7th argument can be omitted.
4095+
4096+-----------------------------------------------------------------
4097+The execute_single method can be used for inserting records also.
4098+
4099+ my $res = $hs->execute_single(3, '+', [ 'foo', 'bar', 'baz' ]);
4100+ die $hs->get_error() if $res->[0] != 0;
4101+ my $num_inserted_rows = $res->[1];
4102+
4103+The 3rd argument must be an arrayref whose elements correspond to
4104+the 5th argument for the corresponding open_index call. If there
4105+is a column which is not appeared in the 5th argument for the
4106+open_index, the default value for the column is set.
4107+
4108+-----------------------------------------------------------------
4109+Multiple operations can be executed in a single call. Executing
4110+multiple operations in a single call is much faster than
4111+executing them separatedly.
4112+
4113+ my $rarr = $hs->execute_multi([
4114+ [ 0, '>=', [ 'foo' ], 5, 0 ],
4115+ [ 2, '=', [ 'bar' ], 1, 0 ],
4116+ [ 4, '<', [ 'baz' ], 10, 5 ],
4117+ ]);
4118+ for my $res (@$rarr) {
4119+ die $hs->get_error() if $res->[0] != 0;
4120+ shift(@$res);
4121+ # ...
4122+ }
4123+
4124+-----------------------------------------------------------------
4125+When an error is occured, the first element of the returned
4126+arrayref becomes a non-zero value. A negative value indicates
4127+that an I/O error is occured and the Net::HandlerSocket object
4128+should be disposed. A positive value means that the connection is
4129+still active and the Net::HandlerSocket object can be reused
4130+later.
4131+
4132
4133=== added file 'HandlerSocket-Plugin-for-MySQL/docs-en/protocol.en.txt'
4134--- HandlerSocket-Plugin-for-MySQL/docs-en/protocol.en.txt 1970-01-01 00:00:00 +0000
4135+++ HandlerSocket-Plugin-for-MySQL/docs-en/protocol.en.txt 2011-06-08 01:57:28 +0000
4136@@ -0,0 +1,148 @@
4137+
4138+----------------------------------------------------------------------------
4139+The HandlerSocket protocol
4140+
4141+----------------------------------------------------------------------------
4142+Basic syntax
4143+
4144+- The HandlerSocket protocol is line-based. Each line ends with LF(0x0a).
4145+- Each line consists a concatenation of tokens separated by HT(0x09).
4146+- A token is either NULL or an encoded string. Note that you need to
4147+ distinguish NULL from an empty string, as most DBMs does so.
4148+- NULL is expressed as a single NUL(0x00).
4149+- An encoded string is a string with the following encoding rules.
4150+ - Characters in the range [0x10 - 0xff] are not encoded.
4151+ - A character in the range [0x00 - 0x0f] is prefixed by 0x01 and
4152+ shifted by 0x40. For example, 0x03 is encoded as 0x01 0x43.
4153+- Note that a string can be empty. A continuation of 0x09 0x09 means that
4154+ there is an empty string between them. A continuation of 0x09 0x0a means
4155+ that there is an empty string at the end of the line.
4156+
4157+----------------------------------------------------------------------------
4158+Request and Response
4159+
4160+- The HandlerSocket protocol is a simple request/response protocol. After a
4161+ connection is established, the client side sends a request, and then the
4162+ server side sends a response.
4163+- A request/response consists of a single line.
4164+- Requests can be pipelined; That is, you can send multiple requests (ie.
4165+ lines) at one time, and receive responses for them at one time.
4166+
4167+----------------------------------------------------------------------------
4168+'open_index' request
4169+
4170+The 'open_index' request has the following syntax.
4171+
4172+ P <indexid> <dbname> <tablename> <indexname> <columns>
4173+
4174+- <indexid> is a number in decimal.
4175+- <dbname>, <tablename>, and <indexname> are strings. To open the primary
4176+ key, use PRIMARY as <indexname>.
4177+- <columns> is a comma-separated list of column names.
4178+
4179+Once an 'open_index' request is issued, the HandlerSocket plugin opens the
4180+specified index and keep it open until the client connection is closed. Each
4181+open index is identified by <indexid>. If <indexid> is already open, the old
4182+open index is closed. You can open the same combination of <dbname>
4183+<tablename> <indexname> multple times, possibly with different <columns>.
4184+For efficiency, keep <indexid> small as far as possible.
4185+
4186+----------------------------------------------------------------------------
4187+Getting data
4188+
4189+The 'find' request has the following syntax.
4190+
4191+ <indexid> <op> <vlen> <v1> ... <vn> <limit> <offset>
4192+
4193+- <indexid> is a number. This number must be an <indexid> specified by a
4194+ 'open_index' request executed previously on the same connection.
4195+- <op> specifies the comparison operation to use. The current version of
4196+ HandlerSocket supports '=', '>', '>=', '<', and '<='.
4197+- <vlen> indicates the length of the trailing parameters <v1> ... <vn>. This
4198+ must be smaller than or equal to the number of index columns specified by
4199+ specified by the corresponding 'open_index' request.
4200+- <v1> ... <vn> specify the index column values to fetch.
4201+- <limit> and <offset> are numbers. These parameters can be omitted. When
4202+ omitted, it works as if 1 and 0 are specified.
4203+
4204+----------------------------------------------------------------------------
4205+Updating/Deleting data
4206+
4207+The 'find_modify' request has the following syntax.
4208+
4209+ <indexid> <op> <vlen> <v1> ... <vn> <limit> <offset> <mop> <m1> ... <mk>
4210+
4211+- <mop> is either 'U' (update) or 'D' (delete).
4212+- <m1> ... <mk> specifies the column values to set. The length of <m1> ...
4213+ <mk> must be smaller than or equal to the length of <columns> specified by
4214+ the corresponding 'open_index' request. If <mop> is 'D', these parameters
4215+ are ignored.
4216+
4217+----------------------------------------------------------------------------
4218+Inserting data
4219+
4220+The 'insert' request has the following syntax.
4221+
4222+ <indexid> '+' <vlen> <v1> ... <vn>
4223+
4224+- <vlen> indicates the length of the trailing parameters <v1> ... <vn>. This
4225+ must be smaller than or equal to the length of <columns> specified by the
4226+ corresponding 'open_index' request.
4227+- <v1> ... <vn> specify the column values to set. For columns not in
4228+ <columns>, the default values for each column are set.
4229+
4230+----------------------------------------------------------------------------
4231+Response syntax
4232+
4233+HandlerSocket returns a response of the following syntax for each request.
4234+
4235+ <errorcode> <numcolumns> <r1> ... <rn>
4236+
4237+- <errorcode> indicates whether the request has successfully executed or not.
4238+ '0' means success. Non-zero means an error.
4239+- <numcolumns> indicates the number of columns of the result set.
4240+- <r1> ... <rn> is the result set. The length of <r1> ... <rn> is always a
4241+ multiple of <numcolumns>. It is possible that <r1> ... <rn> is empty.
4242+
4243+If <errorcode> is non-zero, <numcolumns> is always 1 and <r1> indicates a
4244+human-readable error message, though sometimes <r1> is not provided.
4245+
4246+----------------------------------------------------------------------------
4247+Response for 'open_index'
4248+
4249+If 'open_index' is succeeded, HandlerSocket returns a line of the following
4250+syntax.
4251+
4252+ 0 1
4253+
4254+----------------------------------------------------------------------------
4255+Response for 'find'
4256+
4257+If 'find' is succeeded, HandlerSocket returns a line of the following
4258+syntax.
4259+
4260+ 0 <numcolumns> <r1> ... <rn>
4261+
4262+- <numcolumns> always equals to the length of <columns> of the corresponding
4263+ 'open_index' request.
4264+- <r1> ... <rn> is the result set. If N rows are found, the length of <r1>
4265+ ... <rn> becomes ( <numcolumns> * N ).
4266+
4267+----------------------------------------------------------------------------
4268+Response for 'find_modify'
4269+
4270+If 'find_modify' is succeeded, HandlerSocket returns a line of the following
4271+syntax.
4272+
4273+ 0 1 <nummod>
4274+
4275+- <nummod> is the number of modified rows.
4276+
4277+----------------------------------------------------------------------------
4278+Response for 'insert'
4279+
4280+If 'insert' is succeeded, HanderSocket returns a line of the following
4281+syntax.
4282+
4283+ 0 1
4284+
4285
4286=== added directory 'HandlerSocket-Plugin-for-MySQL/docs-ja'
4287=== added file 'HandlerSocket-Plugin-for-MySQL/docs-ja/about-handlersocket.ja.txt'
4288--- HandlerSocket-Plugin-for-MySQL/docs-ja/about-handlersocket.ja.txt 1970-01-01 00:00:00 +0000
4289+++ HandlerSocket-Plugin-for-MySQL/docs-ja/about-handlersocket.ja.txt 2011-06-08 01:57:28 +0000
4290@@ -0,0 +1,51 @@
4291+
4292+
4293+-----------------------------------------------------------------
4294+ソースコードの利用にあたっての免責事項
4295+
4296+本ソフトウェアの開発者および株式会社ディー・エヌ・エーは、本フト
4297+ウェアの不稼動、稼動不良を含む法律上の瑕疵担保責任、その他保証責
4298+任を負わないものとします。また、本ソフトウエアの開発者および株式
4299+会社ディー・エヌ・エーは、本ソフトウェアの商品性、またはお客様の
4300+特定の目的に対する適合性について、いかなる保証も負わないものとし
4301+ます。
4302+
4303+-----------------------------------------------------------------
4304+handlersocket pluginについて
4305+
4306+mysqlサーバに常駐し、innodb等のストレージエンジンへの直接のアクセ
4307+スを提供するプラグインです。handlersocketプラグインは自前のリス
4308+ナーを持ち、専用のクライアントライブラリ(libhsclient)を使ってそれ
4309+にアクセスします。
4310+
4311+mysqlの標準クライアントライブラリ(libmysql)を使ったアクセスと比べ
4312+て、以下のような利点があります。
4313+・接続あたりに消費するリソースが少ないため、同時接続数が事実上無
4314+ 制限。したがって接続数を気にせず持続接続を使えます。
4315+・高速(単純な参照クエリで3倍〜10倍程度)。
4316+・通信プロトコルがコンパクト。libmysqlを使うとデータ転送時にレ
4317+ コード名などが付随するために通信内容が冗長ですが、libhsclientで
4318+ はデータのみが転送されるため、帯域消費が少なくなります。場合に
4319+ よっては10倍以上libmysqlのほうが冗長になります。
4320+
4321+現在のバージョンでは以下のような処理をサポートしています。
4322+・指定された索引について、指定された値と完全一致するようなレコー
4323+ ドを取得。(SELECT ??? FROM tbl WHERE k1 = v1 AND k2 = v2...)。
4324+ 索引を使わない検索はサポートしていません。
4325+・指定された索引について、指定された値の位置の前後のレコードを取
4326+ 得。(SELECT ??? FROM tbl WHERE k1 >= v1 LIMIT 100)
4327+・前述のような手段で取得したレコードに対するUPDATEとDELETE
4328+・レコードのINSERT
4329+
4330+以下のような言語をサポートします。
4331+・C++。libhsclientをリンクします。
4332+・Perl。Net::HandlerSocketをuseします。
4333+
4334+現在のバージョンではGNU/Linuxでのみ動作します。
4335+
4336+-----------------------------------------------------------------
4337+既知の問題
4338+
4339+・killでhandlersocketスレッドを殺すと、スレッド数が減ったまま回復
4340+ しません。
4341+
4342
4343=== added file 'HandlerSocket-Plugin-for-MySQL/docs-ja/installation.ja.txt'
4344--- HandlerSocket-Plugin-for-MySQL/docs-ja/installation.ja.txt 1970-01-01 00:00:00 +0000
4345+++ HandlerSocket-Plugin-for-MySQL/docs-ja/installation.ja.txt 2011-06-08 01:57:28 +0000
4346@@ -0,0 +1,87 @@
4347+
4348+-----------------------------------------------------------------
4349+HandlerSocketプラグインのビルド方法(RPMを使わない方法)
4350+
4351+以下のようにしてconfigureを実行します。
4352+
4353+ $ ./autogen.sh
4354+ $ ./configure --with-mysql-source=/work/mysql-5.1.50 --with-mysql-bindir=/work/mysql-5.1.50-linux-x86_64-glibc23/bin --with-mysql-plugindir=/work/mysql-5.1.50-linux-x86_64-glibc23/lib/plugin
4355+
4356+ここで--with-mysql-sourceにはMySQLのソースコードのトップディレク
4357+トリを指定します。--with-mysql-bindirにはインストール済みのMySQL
4358+のmysql_configコマンドが有るディレクトリを指定します。
4359+その後以下のようにビルド・インストールします。
4360+
4361+ $ make
4362+ $ sudo make install
4363+
4364+-----------------------------------------------------------------
4365+クライアントライブラリのビルド方法(RPMを使わない方法)
4366+
4367+クライアントライブラリをビルドする際には、MySQLのソースコードは
4368+必要ありません。またMySQLがインストールされている必要もありません。
4369+
4370+ $ ./autogen.sh
4371+ $ ./configure --disable-handlersocket-server
4372+ $ make
4373+ $ sudo make install
4374+ $ cd perl-Net-HandlerSocket
4375+ $ perl Makefile.PL
4376+ $ make
4377+ $ sudo make install
4378+
4379+-----------------------------------------------------------------
4380+ビルド方法(RPM)
4381+
4382+以下のように実行すれば、rpmパッケージがビルド&インストールされま
4383+す。
4384+
4385+(MySQLサーバ側、HandlerSocketプラグインをインストールする)
4386+ $ ./autogen.sh
4387+ $ ./configure --with-mysql-source=/work/mysql-5.1.50 --with-mysql-bindir=/work/mysql-5.1.50-linux-x86_64-glibc23/bin --with-mysql-plugindir=/work/mysql-5.1.50-linux-x86_64-glibc23/lib/plugin
4388+ $ make rpm_cli
4389+ $ sudo rpm -U dist/RPMS/*/libhsclient*.rpm
4390+ $ make rpm_c
4391+ $ sudo rpm -U dist/RPMS/*/handlersocket*.rpm
4392+
4393+(クライアント側、クライアントライブラリをインストールする)
4394+ $ ./autogen.sh
4395+ $ ./configure --disable-handlersocket-server
4396+ $ make rpm_cli
4397+ $ sudo rpm -U dist/RPMS/*/libhsclient*.rpm
4398+ $ make rpm_perl
4399+ $ sudo rpm -U dist/RPMS/*/perl-Net-HandlerSocket*.rpm
4400+
4401+-----------------------------------------------------------------
4402+起動
4403+
4404+mysqlを起動した状態で、mysqlの設定ファイル(my.cnf等)に以下の内容を
4405+追加します。
4406+
4407+ [mysqld]
4408+ handlersocket_port = 9998
4409+ # handlersocketが接続を受け付けるポート(参照系リクエスト用)
4410+ handlersocket_port_wr = 9999
4411+ # handlersocketが接続を受け付けるポート(更新系リクエスト用)
4412+ handlersocket_address =
4413+ # handlersocketがバインドするアドレス(空のままでOK)
4414+ handlersocket_verbose = 0
4415+ # デバッグ用
4416+ handlersocket_timeout = 300
4417+ # 通信タイムアウト(秒)
4418+ handlersocket_threads = 16
4419+ # handlersocketのワーカースレッド数
4420+ thread_concurrency = 128
4421+ # handlersocketが幾つかのスレッドを占有するため、大きめの
4422+ # 値を指定してください
4423+ open_files_limit = 65535
4424+ # ソケットを大量に開けるようにするため、大きめの値を指定し
4425+ # てください
4426+
4427+以下のクエリを実行します。
4428+
4429+ mysql> install plugin handlersocket soname 'handlersocket.so';
4430+ Query OK, 0 rows affected (0.06 sec)
4431+
4432+以上でhandlersocketへクライアントからアクセスできるようになります。
4433+
4434
4435=== added file 'HandlerSocket-Plugin-for-MySQL/docs-ja/perl-client.ja.txt'
4436--- HandlerSocket-Plugin-for-MySQL/docs-ja/perl-client.ja.txt 1970-01-01 00:00:00 +0000
4437+++ HandlerSocket-Plugin-for-MySQL/docs-ja/perl-client.ja.txt 2011-06-08 01:57:28 +0000
4438@@ -0,0 +1,118 @@
4439+
4440+-----------------------------------------------------------------
4441+handlersocketプラグインへの接続を開くには、Net::HandlerSocketオブ
4442+ジェクトを作成します。
4443+
4444+ use Net::HandlerSocket;
4445+ my $args = { host => 'localhost', port => 9998 };
4446+ my $hs = new Net::HandlerSocket($args);
4447+
4448+-----------------------------------------------------------------
4449+検索などの命令を実行する前に、処理対象となる索引を開く必要があり
4450+ます。
4451+
4452+ my $err = $hs->open_index(3, 'database1', 'table1', 'PRIMARY',
4453+ 'f1,f2');
4454+ die $hs->get_error() if $res->[0] != 0;
4455+
4456+最初の引数は開く索引に付ける番号です。付けた番号は同一の
4457+Net::HandlerSocketオブジェクトについてのみ有効です。第4引数は開く
4458+索引の名前で、「PRIMARY」が指定され場合はプライマリキーが開かれま
4459+す。第5引数は「,」で区切られた列名のリストです。
4460+
4461+-----------------------------------------------------------------
4462+テーブルから索引を使って行を取得するには、execute_singleメソッド
4463+を呼びます。
4464+
4465+ my $res = $hs->execute_single(3, '=', [ 'foo' ], 1, 0);
4466+ die $hs->get_error() if $res->[0] != 0;
4467+ shift(@$res);
4468+
4469+最初の引数は索引の番号で、同じNet::HandlerSocketオブジェクトへ
4470+open_indexで付けたものでなければなりません。第2引数には操作を指定
4471+します。現在のバージョンでは、「=」、「>=」、「<=」、「>」、「<」
4472+の操作が利用可能です。第3引数は配列への参照で、これは探すべき行の
4473+キー値を指定します。配列の長さは索引のキーの個数と同じか少ない数
4474+でなければなりません。第4引数と第5引数はそれぞれ、取得する最大行
4475+数、取得前に読み飛ばす行数を指定します。取得される列は対応する
4476+open_index呼び出しの第5引数で指定されたものになります。
4477+
4478+execute_singleメソッドは常に配列への参照を返します。最初の要素は
4479+エラーコードで、これが0ならば成功を表します。残りの要素は列の値で
4480+す。もし取得されたデータが複数行の場合は、それが一つの配列へ連結
4481+された形で格納されています。例えば、5行3列のデータの場合、次のよ
4482+うなコードによってその内容を取得できます。
4483+
4484+ die $hs->get_error() if $res->[0] != 0;
4485+ shift(@$res);
4486+ for (my $row = 0; $row < 5; ++$row) {
4487+ for (my $col = 0; $col < 3; ++$col) {
4488+ my $value = $res->[$row * 5 + $col];
4489+ # ...
4490+ }
4491+ }
4492+
4493+-----------------------------------------------------------------
4494+行を更新または削除するには、更に多くの引数を指定して
4495+execute_singleメソッドを呼び出します。書き込み処理をするには、
4496+対象のNet::HandlerSocketオブジェクトは更新用handlersocketワーカ(既
4497+定ではポート9999)へ接続されたものでなくてはなりません。
4498+(安全のため、ポート9998は参照処理だけを受け付け、ポート9999は更新
4499+処理も受け付けるようになっています。ポート9999は参照処理も受け付
4500+けますが、レコードロック等の影響で遅くなります。ポート番号は
4501+mysqldの「handlersocket_port」と「handlersocket_port_wr」の設定項
4502+目で変更できます。)
4503+
4504+ my $args = { host => 'localhost', port => 9999 };
4505+ my $hs = new Net::HandlerSocket($args);
4506+
4507+ my $res = $hs->execute_single(3, '=', [ 'bar' ], 1, 0, 'U',
4508+ [ 'fubar', 'hoge' ]);
4509+ die $hs->get_error() if $res->[0] != 0;
4510+ my $num_updated_rows = $res->[1];
4511+
4512+ my $res = $hs->execute_single(3, '=', [ 'baz' ], 1, 0, 'D');
4513+ die $hs->get_error() if $res->[0] != 0;
4514+ my $num_deleted_rows = $res->[1];
4515+
4516+execute_singleの第6引数は変更処理の種類を指定します。現在のバー
4517+ジョンでは「U」と「D」が利用可能です。「U」については、第7引数で
4518+新しい値を指定します。更新される列は、対応するopen_index呼び出し
4519+の第5引数で指定されたものになります。「D」については第7引数は省
4520+略できます。
4521+
4522+-----------------------------------------------------------------
4523+execute_singleメソッドは列の挿入にも使用できます。
4524+
4525+ my $res = $hs->execute_single(3, '+', [ 'foo', 'bar', 'baz' ]);
4526+ die $hs->get_error() if $res->[0] != 0;
4527+ my $num_inserted_rows = $res->[1];
4528+
4529+第3引数は、対応するopen_index呼び出しの第5引数の列リストと同じだ
4530+けの長さの配列への参照でなければなりません。open_index呼び出しの
4531+第5引数に指定されていない列については、その列の既定値がセットされ
4532+ます。
4533+
4534+-----------------------------------------------------------------
4535+execute_multiメソッドを使えば、複数のリクエストを一つの呼び出しで
4536+実行することができます。これはリクエストを個別に実行するより高速
4537+です。
4538+
4539+ my $rarr = $hs->execute_multi([
4540+ [ 0, '>=', [ 'foo' ], 5, 0 ],
4541+ [ 2, '=', [ 'bar' ], 1, 0 ],
4542+ [ 4, '<', [ 'baz' ], 10, 5 ],
4543+ ]);
4544+ for my $res (@$rarr) {
4545+ die $hs->get_error() if $res->[0] != 0;
4546+ shift(@$res);
4547+ # ...
4548+ }
4549+
4550+-----------------------------------------------------------------
4551+エラーが起こると返値の配列参照の最初の要素が0以外になります。負の
4552+数の場合はI/Oエラーが起こったことを示し、その場合はその
4553+Net::HandlerSocketオブジェクトは破棄するべきです。正の値の場合は
4554+接続は維持されているため、そのオブジェクトはそれ以後も再利用でき
4555+ます。
4556+
4557
4558=== added file 'HandlerSocket-Plugin-for-MySQL/docs-ja/protocol.ja.txt'
4559--- HandlerSocket-Plugin-for-MySQL/docs-ja/protocol.ja.txt 1970-01-01 00:00:00 +0000
4560+++ HandlerSocket-Plugin-for-MySQL/docs-ja/protocol.ja.txt 2011-06-08 01:57:28 +0000
4561@@ -0,0 +1,94 @@
4562+
4563+-----------------------------------------------------------------
4564+handlersocketの通信プロトコル
4565+
4566+-----------------------------------------------------------------
4567+構文
4568+
4569+・コマンド行は改行(LF)で終わる。
4570+・コマンド行は複数のトークンからなり、トークン間はTABで区切られる。
4571+・トークンはNULLトークンか、文字列トークンのいずれか。
4572+・NULLトークンは単一のNUL文字であらわされる。
4573+・文字列トークンは、0バイト以上の文字列であらわされる。ただし0x10
4574+ 未満の文字については0x01を前置し、0x40を加えたコードであらわさ
4575+ れる。それ以外の文字はその文字自身のコードであらわされる。
4576+
4577+-----------------------------------------------------------------
4578+リクエストとレスポンス
4579+
4580+・接続が確立した直後の状態では、まずクライアントがコマンド行を送
4581+ る。(リクエスト)
4582+・サーバはクライアントが送ったリクエストと丁度同じ数のコマンド行
4583+ を返す。(レスポンス)
4584+・リクエストはパイプライン化してよい。つまりクライアントは前に
4585+ 送ったリクエストに対する返事を待たずに次のリクエストを送っても
4586+ よい。
4587+
4588+-----------------------------------------------------------------
4589+リクエスト
4590+
4591+・open_index命令は次のような構文を持つ。
4592+ 'P' indexid dbname tablename indexname fieldlist
4593+ indexidは開いている索引に付けられる番号で、同一接続上で後に実行
4594+ する命令の、対象索引を指定するために使われる。dbname、tablename、
4595+ indexnameはそれぞれ開きたいDB、テーブル、索引の名前。索引の名前
4596+ として"PRIMARY"を指定するとプライマリキーが開かれる。fieldlist
4597+ はカンマ区切りの列名のリスト。
4598+・find命令は次のような構文を持つ。
4599+ indexid op nflds v1 ... vn limit offset
4600+ indexidは実行対象の索引を指定する。opは索引検索の演算子(後述)。
4601+ v1からvnは可変長で、その個数はnflds。nfldsはindexidで指定された
4602+ open_index命令のindexnameの索引のfieldlistのフィールド数に等し
4603+ いか小さくなくてはならない。m2からmkは可変長で、その個数は
4604+ indexidで指定されたopen_index命令が発行された際のfieldlistに一
4605+ 致しなければならない。コマンド行のlimit以降は省略できる。limit
4606+ とoffsetは、検索条件に合致する列のうちレスポンスに返す列数の上
4607+ 限と、スキップする列数。limitとoffsetを省略した場合はそれぞれ1
4608+ と0が指定されたときと同じ動作をする。find命令はレスポンスとして、
4609+ 条件に合致した列のリストを返す。opとして指定できる演算子は次の
4610+ とおり。
4611+ '=' - v1 ... vnと一致するものを取得
4612+ '>' - v1 ... vnより大きいものを昇順に取得
4613+ '>=' - v1 ... vnに一致するか大きいものを昇順に取得
4614+ '<' - v1 ... vnより小さいものを降順に取得
4615+ '<=' - v1 ... vnに一致するか等しいものを降順に取得
4616+ nfldsが1より大きい(v1 ... vnが2個以上)ときは辞書式順序で比較さ
4617+ れる。
4618+・find_modify命令は次のような構文を持つ。
4619+ indexid op nflds v1 ... vn limit offset modop m1 ... mk
4620+ modopより前の部分はfind命令と同等で、これによって操作対象の行を
4621+ 指定する。その操作対象の行に対しmodopで指定された変更処理を実行
4622+ する。m1 ... mkは可変長で、省略できる。modopは次いずれか。
4623+ 'U' - indexidで指定されたopen_index命令のfieldlist列
4624+ の内容を、m1 ... mkの値で更新する。
4625+ 'D' - 対象の行を削除する。m1 ... mkの値は無視される。
4626+・insert命令はのような構文を持つ。
4627+ indexid '+' nflds v1 ... vn
4628+ indexidで指定されたテーブルに、列を挿入する。v1 ... vnは可変長
4629+ で、その個数はnflds。nfldsはindexidで指定されたopen_index命令の
4630+ indexnameの索引のfieldlistのフィールド数に等しいか小さくなくて
4631+ はならない。
4632+
4633+-----------------------------------------------------------------
4634+レスポンス
4635+
4636+・open_index命令が成功したとき、レスポンスは次の構文を持つ。
4637+ '0' '1'
4638+・find命令が成功したとき、レスポンスは次の構文を持つ。
4639+ '0' nflds v1 ... vn
4640+ nfldsは結果セットの列の数をあらわす。v1 ... vnは可変長で、その
4641+ 長さはnfldsの整数倍。v1 ... vnは空のこともあり、それは条件に合
4642+ 致するレコードが存在しなかったことをあらわす。結果セットが複数
4643+ 行になったときはv1 ... vnの長さがnfldsの2倍以上となり、最初の
4644+ 行から順にv1 ... vnにセットされる。
4645+・modify命令が成功したとき、レスポンスは次の構文を持つ。
4646+ '0' '1' nummod
4647+ nummodは変更が施された行数。nummodが0のときは変更された行が無
4648+ かったことをあらわす。
4649+・insert命令が成功したとき、レスポンスは次の構文を持つ。
4650+ '0' '1'
4651+・命令が失敗したとき、レスポンスは命令に関わらず次の構文を持つ。
4652+ err '1' message
4653+ errは0以外の数値で、エラーコードをあらわす。messageは人間可読な
4654+ エラーメッセージ。ただしmessageが無いこともある。
4655+
4656
4657=== added directory 'HandlerSocket-Plugin-for-MySQL/handlersocket'
4658=== added file 'HandlerSocket-Plugin-for-MySQL/handlersocket/COPYRIGHT.txt'
4659--- HandlerSocket-Plugin-for-MySQL/handlersocket/COPYRIGHT.txt 1970-01-01 00:00:00 +0000
4660+++ HandlerSocket-Plugin-for-MySQL/handlersocket/COPYRIGHT.txt 2011-06-08 01:57:28 +0000
4661@@ -0,0 +1,27 @@
4662+
4663+ Copyright (c) 2010 DeNA Co.,Ltd.
4664+ All rights reserved.
4665+
4666+ Redistribution and use in source and binary forms, with or without
4667+ modification, are permitted provided that the following conditions are met:
4668+
4669+ * Redistributions of source code must retain the above copyright
4670+ notice, this list of conditions and the following disclaimer.
4671+ * Redistributions in binary form must reproduce the above copyright
4672+ notice, this list of conditions and the following disclaimer in the
4673+ documentation and/or other materials provided with the distribution.
4674+ * Neither the name of DeNA Co.,Ltd. nor the names of its contributors
4675+ may be used to endorse or promote products derived from this software
4676+ without specific prior written permission.
4677+
4678+ THIS SOFTWARE IS PROVIDED BY DeNA Co.,Ltd. "AS IS" AND ANY EXPRESS OR
4679+ IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
4680+ MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO
4681+ EVENT SHALL DeNA Co.,Ltd. BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
4682+ SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
4683+ PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS;
4684+ OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
4685+ WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
4686+ OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
4687+ ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
4688+
4689
4690=== added file 'HandlerSocket-Plugin-for-MySQL/handlersocket/Makefile.am'
4691--- HandlerSocket-Plugin-for-MySQL/handlersocket/Makefile.am 1970-01-01 00:00:00 +0000
4692+++ HandlerSocket-Plugin-for-MySQL/handlersocket/Makefile.am 2011-06-08 01:57:28 +0000
4693@@ -0,0 +1,10 @@
4694+pkgplugindir = $(PLUGIN_DIR)
4695+noinst_HEADERS = database.hpp hstcpsvr.hpp hstcpsvr_worker.hpp mysql_incl.hpp
4696+pkgplugin_LTLIBRARIES = handlersocket.la
4697+handlersocket_la_LDFLAGS = -module ../libhsclient/libhsclient.la
4698+handlersocket_la_CFLAGS = $(MYSQL_INC) $(MYSQL_CFLAGS) $(AM_CFLAGS) \
4699+ -I../libhsclient
4700+handlersocket_la_CXXFLAGS = $(MYSQL_INC) $(MYSQL_CFLAGS) $(AM_CFLAGS) \
4701+ -I../libhsclient
4702+handlersocket_la_SOURCES = database.cpp handlersocket.cpp \
4703+ hstcpsvr_worker.cpp hstcpsvr.cpp
4704
4705=== added file 'HandlerSocket-Plugin-for-MySQL/handlersocket/Makefile.plain.template'
4706--- HandlerSocket-Plugin-for-MySQL/handlersocket/Makefile.plain.template 1970-01-01 00:00:00 +0000
4707+++ HandlerSocket-Plugin-for-MySQL/handlersocket/Makefile.plain.template 2011-06-08 01:57:28 +0000
4708@@ -0,0 +1,31 @@
4709+
4710+MYSQL_INC = HANDLERSOCKET_MYSQL_INC
4711+MYSQL_LIB = HANDLERSOCKET_MYSQL_LIB
4712+
4713+CXX = g++ -Wall -g -fno-rtti -fno-exceptions -fPIC -DPIC
4714+LIBS = $(MYSQL_LIB) -lhsclient -lpthread -lz
4715+CXXFLAGS = -I/usr/include/handlersocket $(MYSQL_INC)
4716+LDFLAGS =
4717+
4718+CXXFLAGS += -O3 -DNDEBUG
4719+
4720+HANDLERSOCKET_OBJS = database.o hstcpsvr.o hstcpsvr_worker.o
4721+
4722+all: handlersocket.so
4723+
4724+handlersocket.so: $(HANDLERSOCKET_OBJS) handlersocket.cpp
4725+ $(CXX) $(CXXFLAGS) -fno-strict-aliasing -shared $^ -o $@ $(LDFLAGS) \
4726+ -Wl,-soname -Wl,$@ $(LIBS)
4727+clean:
4728+ rm -f *.a *.so *.o
4729+
4730+LIBDIR = $(shell \
4731+ if [ -e /usr/lib64/mysql ]; then echo /usr/lib64; else echo /usr/lib; fi)
4732+
4733+install: handlersocket.so
4734+ sudo sh -c 'ulimit -c unlimited ; /etc/init.d/mysql stop ; \
4735+ cp handlersocket.so handlersocket.so.cpy && \
4736+ mv handlersocket.so.cpy \
4737+ $(LIBDIR)/mysql/plugin/handlersocket.so && \
4738+ /etc/init.d/mysql start'
4739+
4740
4741=== added file 'HandlerSocket-Plugin-for-MySQL/handlersocket/database.cpp'
4742--- HandlerSocket-Plugin-for-MySQL/handlersocket/database.cpp 1970-01-01 00:00:00 +0000
4743+++ HandlerSocket-Plugin-for-MySQL/handlersocket/database.cpp 2011-06-08 01:57:28 +0000
4744@@ -0,0 +1,1183 @@
4745+
4746+// vim:sw=2:ai
4747+
4748+/*
4749+ * Copyright (C) 2010 DeNA Co.,Ltd.. All rights reserved.
4750+ * See COPYRIGHT.txt for details.
4751+ */
4752+
4753+#include <stdlib.h>
4754+#include <stdio.h>
4755+#include <string.h>
4756+
4757+#include "database.hpp"
4758+#include "string_util.hpp"
4759+#include "escape.hpp"
4760+#include "mysql_incl.hpp"
4761+
4762+#define DBG_KEY(x)
4763+#define DBG_SHUT(x)
4764+#define DBG_LOCK(x)
4765+#define DBG_THR(x)
4766+#define DBG_CMP(x)
4767+#define DBG_FLD(x)
4768+#define DBG_FILTER(x)
4769+#define DBG_REFCNT(x)
4770+#define DBG_KEYLEN(x)
4771+#define DBG_DELETED
4772+
4773+/* status variables */
4774+unsigned long long int open_tables_count;
4775+unsigned long long int close_tables_count;
4776+unsigned long long int lock_tables_count;
4777+unsigned long long int unlock_tables_count;
4778+unsigned long long int index_exec_count;
4779+
4780+namespace dena {
4781+
4782+prep_stmt::prep_stmt()
4783+ : dbctx(0), table_id(static_cast<size_t>(-1)),
4784+ idxnum(static_cast<size_t>(-1))
4785+{
4786+}
4787+prep_stmt::prep_stmt(dbcontext_i *c, size_t tbl, size_t idx,
4788+ const fields_type& rf, const fields_type& ff)
4789+ : dbctx(c), table_id(tbl), idxnum(idx), ret_fields(rf), filter_fields(ff)
4790+{
4791+ if (dbctx) {
4792+ dbctx->table_addref(table_id);
4793+ }
4794+}
4795+prep_stmt::~prep_stmt()
4796+{
4797+ if (dbctx) {
4798+ dbctx->table_release(table_id);
4799+ }
4800+}
4801+
4802+prep_stmt::prep_stmt(const prep_stmt& x)
4803+ : dbctx(x.dbctx), table_id(x.table_id), idxnum(x.idxnum),
4804+ ret_fields(x.ret_fields), filter_fields(x.filter_fields)
4805+{
4806+ if (dbctx) {
4807+ dbctx->table_addref(table_id);
4808+ }
4809+}
4810+
4811+prep_stmt&
4812+prep_stmt::operator =(const prep_stmt& x)
4813+{
4814+ if (this != &x) {
4815+ if (dbctx) {
4816+ dbctx->table_release(table_id);
4817+ }
4818+ dbctx = x.dbctx;
4819+ table_id = x.table_id;
4820+ idxnum = x.idxnum;
4821+ ret_fields = x.ret_fields;
4822+ filter_fields = x.filter_fields;
4823+ if (dbctx) {
4824+ dbctx->table_addref(table_id);
4825+ }
4826+ }
4827+ return *this;
4828+}
4829+
4830+struct database : public database_i, private noncopyable {
4831+ database(const config& c);
4832+ virtual ~database();
4833+ virtual dbcontext_ptr create_context(bool for_write) volatile;
4834+ virtual void stop() volatile;
4835+ virtual const config& get_conf() const volatile;
4836+ public:
4837+ int child_running;
4838+ private:
4839+ config conf;
4840+};
4841+
4842+struct tablevec_entry {
4843+ TABLE *table;
4844+ size_t refcount;
4845+ bool modified;
4846+ tablevec_entry() : table(0), refcount(0), modified(false) { }
4847+};
4848+
4849+struct expr_user_lock : private noncopyable {
4850+ expr_user_lock(THD *thd, int timeout)
4851+ : lck_key("handlersocket_wr", 16, &my_charset_latin1),
4852+ lck_timeout(timeout),
4853+ lck_func_get_lock(&lck_key, &lck_timeout),
4854+ lck_func_release_lock(&lck_key)
4855+ {
4856+ lck_key.fix_fields(thd, 0);
4857+ lck_timeout.fix_fields(thd, 0);
4858+ lck_func_get_lock.fix_fields(thd, 0);
4859+ lck_func_release_lock.fix_fields(thd, 0);
4860+ }
4861+ long long get_lock() {
4862+ return lck_func_get_lock.val_int();
4863+ }
4864+ long long release_lock() {
4865+ return lck_func_release_lock.val_int();
4866+ }
4867+ private:
4868+ Item_string lck_key;
4869+ Item_int lck_timeout;
4870+ Item_func_get_lock lck_func_get_lock;
4871+ Item_func_release_lock lck_func_release_lock;
4872+};
4873+
4874+struct dbcontext : public dbcontext_i, private noncopyable {
4875+ dbcontext(volatile database *d, bool for_write);
4876+ virtual ~dbcontext();
4877+ virtual void init_thread(const void *stack_botton,
4878+ volatile int& shutdown_flag);
4879+ virtual void term_thread();
4880+ virtual bool check_alive();
4881+ virtual void lock_tables_if();
4882+ virtual void unlock_tables_if();
4883+ virtual bool get_commit_error();
4884+ virtual void clear_error();
4885+ virtual void close_tables_if();
4886+ virtual void table_addref(size_t tbl_id);
4887+ virtual void table_release(size_t tbl_id);
4888+ virtual void cmd_open(dbcallback_i& cb, const cmd_open_args& args);
4889+ virtual void cmd_exec(dbcallback_i& cb, const cmd_exec_args& args);
4890+ virtual void set_statistics(size_t num_conns, size_t num_active);
4891+ private:
4892+ int set_thread_message(const char *fmt, ...)
4893+ __attribute__((format (printf, 2, 3)));
4894+ bool parse_fields(TABLE *const table, const char *str,
4895+ prep_stmt::fields_type& flds);
4896+ void cmd_insert_internal(dbcallback_i& cb, const prep_stmt& pst,
4897+ const string_ref *fvals, size_t fvalslen);
4898+ void cmd_sql_internal(dbcallback_i& cb, const prep_stmt& pst,
4899+ const string_ref *fvals, size_t fvalslen);
4900+ void cmd_find_internal(dbcallback_i& cb, const prep_stmt& pst,
4901+ ha_rkey_function find_flag, const cmd_exec_args& args);
4902+ size_t calc_filter_buf_size(TABLE *table, const prep_stmt& pst,
4903+ const record_filter *filters);
4904+ bool fill_filter_buf(TABLE *table, const prep_stmt& pst,
4905+ const record_filter *filters, uchar *filter_buf, size_t len);
4906+ int check_filter(dbcallback_i& cb, TABLE *table, const prep_stmt& pst,
4907+ const record_filter *filters, const uchar *filter_buf);
4908+ void resp_record(dbcallback_i& cb, TABLE *const table, const prep_stmt& pst);
4909+ void dump_record(dbcallback_i& cb, TABLE *const table, const prep_stmt& pst);
4910+ int modify_record(dbcallback_i& cb, TABLE *const table,
4911+ const prep_stmt& pst, const cmd_exec_args& args, char mod_op,
4912+ size_t& modified_count);
4913+ private:
4914+ typedef std::vector<tablevec_entry> table_vec_type;
4915+ typedef std::pair<std::string, std::string> table_name_type;
4916+ typedef std::map<table_name_type, size_t> table_map_type;
4917+ private:
4918+ volatile database *const dbref;
4919+ bool for_write_flag;
4920+ THD *thd;
4921+ MYSQL_LOCK *lock;
4922+ bool lock_failed;
4923+ std::auto_ptr<expr_user_lock> user_lock;
4924+ int user_level_lock_timeout;
4925+ bool user_level_lock_locked;
4926+ bool commit_error;
4927+ std::vector<char> info_message_buf;
4928+ table_vec_type table_vec;
4929+ table_map_type table_map;
4930+};
4931+
4932+database::database(const config& c)
4933+ : child_running(1), conf(c)
4934+{
4935+}
4936+
4937+database::~database()
4938+{
4939+}
4940+
4941+dbcontext_ptr
4942+database::create_context(bool for_write) volatile
4943+{
4944+ return dbcontext_ptr(new dbcontext(this, for_write));
4945+}
4946+
4947+void
4948+database::stop() volatile
4949+{
4950+ child_running = false;
4951+}
4952+
4953+const config&
4954+database::get_conf() const volatile
4955+{
4956+ return const_cast<const config&>(conf);
4957+}
4958+
4959+database_ptr
4960+database_i::create(const config& conf)
4961+{
4962+ return database_ptr(new database(conf));
4963+}
4964+
4965+dbcontext::dbcontext(volatile database *d, bool for_write)
4966+ : dbref(d), for_write_flag(for_write), thd(0), lock(0), lock_failed(false),
4967+ user_level_lock_timeout(0), user_level_lock_locked(false),
4968+ commit_error(false)
4969+{
4970+ info_message_buf.resize(8192);
4971+ user_level_lock_timeout = d->get_conf().get_int("wrlock_timeout", 12);
4972+}
4973+
4974+dbcontext::~dbcontext()
4975+{
4976+}
4977+
4978+namespace {
4979+
4980+int
4981+wait_server_to_start(THD *thd, volatile int& shutdown_flag)
4982+{
4983+ int r = 0;
4984+ DBG_SHUT(fprintf(stderr, "HNDSOCK wsts\n"));
4985+ pthread_mutex_lock(&LOCK_server_started);
4986+ while (!mysqld_server_started) {
4987+ timespec abstime = { };
4988+ set_timespec(abstime, 1);
4989+ pthread_cond_timedwait(&COND_server_started, &LOCK_server_started,
4990+ &abstime);
4991+ pthread_mutex_unlock(&LOCK_server_started);
4992+ pthread_mutex_lock(&thd->mysys_var->mutex);
4993+ THD::killed_state st = thd->killed;
4994+ pthread_mutex_unlock(&thd->mysys_var->mutex);
4995+ DBG_SHUT(fprintf(stderr, "HNDSOCK wsts kst %d\n", (int)st));
4996+ pthread_mutex_lock(&LOCK_server_started);
4997+ if (st != THD::NOT_KILLED) {
4998+ DBG_SHUT(fprintf(stderr, "HNDSOCK wsts kst %d break\n", (int)st));
4999+ r = -1;
5000+ break;
The diff has been truncated for viewing.

Subscribers

People subscribed via source and target branches