Merge lp:~stewart/percona-server/5.5-update-handlersocket into lp:percona-server/5.5

Proposed by Stewart Smith
Status: Work in progress
Proposed branch: lp:~stewart/percona-server/5.5-update-handlersocket
Merge into: lp:percona-server/5.5
Diff against target: 81747 lines
To merge this branch: bzr merge lp:~stewart/percona-server/5.5-update-handlersocket
Reviewer Review Type Date Requested Status
Laurynas Biveinis (community) Needs Fixing
Review via email: mp+111767@code.launchpad.net

Description of the change

Update HandlerSocket to 1.1.0 from a git branch (imported with bzr-git). This should make it easier to keep up to date in the future.

http://jenkins.percona.com/job/percona-server-5.5-param/442/

To post a comment you must log in.
Revision history for this message
Laurynas Biveinis (laurynas-biveinis) wrote :

Same comment as for the 5.1 MP.

Also, for 5.5, is it a merge from 5.1? I'm asking because the commit message does not say "merge from 5.1" or similar.

review: Needs Information
Revision history for this message
Laurynas Biveinis (laurynas-biveinis) wrote :

Setting to WiP as will need to be redone the same way 5.1 was after the bzr bug breaking trunk.

review: Needs Fixing

Unmerged revisions

266. By Stewart Smith

update HandlerSocket to 1.1.0 (by removing old HandlerSocket tree and replacing with a 'bzr join'ed one)

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== added directory 'HandlerSocket-Plugin-for-MySQL'
=== removed directory 'HandlerSocket-Plugin-for-MySQL'
=== added file 'HandlerSocket-Plugin-for-MySQL/.gitignore'
--- HandlerSocket-Plugin-for-MySQL/.gitignore 1970-01-01 00:00:00 +0000
+++ HandlerSocket-Plugin-for-MySQL/.gitignore 2012-06-25 04:42:22 +0000
@@ -0,0 +1,35 @@
1configure
2Makefile.in
3Makefile
4.deps
5*.lo
6*.o
7INSTALL
8m4
9stamp-h1
10libtool
11ltmain.sh
12missing
13install-sh
14autom4te.cache
15config.h*
16config.status
17config.guess
18config.sub
19config.log
20depcomp
21TAGS
22aclocal.m4
23.libs
24ltmain.sh
25*.la
26handlersocket/Makefile.plain
27handlersocket/*.spec
28libhsclient/*.spec
29client/hstest
30client/hsclient
31perl-Net-HandlerSocket/*.spec
32regtest/*/*.log
33regtest/*/*.log2
34regtest/*/DONE
35dist/*
036
=== added file 'HandlerSocket-Plugin-for-MySQL/AUTHORS'
--- HandlerSocket-Plugin-for-MySQL/AUTHORS 1970-01-01 00:00:00 +0000
+++ HandlerSocket-Plugin-for-MySQL/AUTHORS 2012-06-25 04:42:22 +0000
@@ -0,0 +1,22 @@
1Akira Higuchi (https://github.com/ahiguti)
2 - developed HanderSocket plugin, libhsclient, and perl-Net-HandlerSocket
3
4Yoshinori Matsunobu (https://github.com/yoshinorim)
5 - introduced autotools, added support for MySQL 5.5.6, added statistics
6 variables
7
8Jeff Hodges (https://github.com/jmhodges)
9 - fixed some autotools scripts
10
11Toru Yamaguchi (https://github.com/zigorou)
12 - ported to MacOS X
13
14Moriyoshi Koizumi (https://github.com/moriyoshi)
15 - fixed some autotools scripts
16
17takeda-at (https://github.com/takada-at)
18 - added simple authorization function
19
20WheresWardy (https://github.com/WheresWardy)
21 - added authentication functions to libhsclient
22
023
=== removed file 'HandlerSocket-Plugin-for-MySQL/AUTHORS'
--- HandlerSocket-Plugin-for-MySQL/AUTHORS 2011-01-10 13:39:35 +0000
+++ HandlerSocket-Plugin-for-MySQL/AUTHORS 1970-01-01 00:00:00 +0000
@@ -1,19 +0,0 @@
1Akira Higuchi (https://github.com/ahiguti)
2 - developed HanderSocket plugin, libhsclient, and perl-Net-HandlerSocket
3
4Yoshinori Matsunobu (https://github.com/yoshinorim)
5 - introduced autotools, added support for MySQL 5.5.6, added statistics
6 variables
7
8Jeff Hodges (https://github.com/jmhodges)
9 - fixed some autotools scripts
10
11Toru Yamaguchi (https://github.com/zigorou)
12 - ported to MacOS X
13
14Moriyoshi Koizumi (https://github.com/moriyoshi)
15 - fixed some autotools scripts
16
17takeda-at (https://github.com/takada-at)
18 - added simple authorization function
19
200
=== added file 'HandlerSocket-Plugin-for-MySQL/COPYING'
--- HandlerSocket-Plugin-for-MySQL/COPYING 1970-01-01 00:00:00 +0000
+++ HandlerSocket-Plugin-for-MySQL/COPYING 2012-06-25 04:42:22 +0000
@@ -0,0 +1,30 @@
1-----------------------------------------------------------------------------
2HandlerSocket plugin for MySQL
3
4 Copyright (c) 2010 DeNA Co.,Ltd.
5 All rights reserved.
6
7 Redistribution and use in source and binary forms, with or without
8 modification, are permitted provided that the following conditions are met:
9
10 * Redistributions of source code must retain the above copyright
11 notice, this list of conditions and the following disclaimer.
12 * Redistributions in binary form must reproduce the above copyright
13 notice, this list of conditions and the following disclaimer in the
14 documentation and/or other materials provided with the distribution.
15 * Neither the name of DeNA Co.,Ltd. nor the names of its contributors
16 may be used to endorse or promote products derived from this software
17 without specific prior written permission.
18
19 THIS SOFTWARE IS PROVIDED BY DeNA Co.,Ltd. "AS IS" AND ANY EXPRESS OR
20 IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
21 MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO
22 EVENT SHALL DeNA Co.,Ltd. BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
23 SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
24 PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS;
25 OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
26 WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
27 OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
28 ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
29
30
031
=== removed file 'HandlerSocket-Plugin-for-MySQL/COPYING'
--- HandlerSocket-Plugin-for-MySQL/COPYING 2011-01-10 13:39:35 +0000
+++ HandlerSocket-Plugin-for-MySQL/COPYING 1970-01-01 00:00:00 +0000
@@ -1,30 +0,0 @@
1-----------------------------------------------------------------------------
2HandlerSocket plugin for MySQL
3
4 Copyright (c) 2010 DeNA Co.,Ltd.
5 All rights reserved.
6
7 Redistribution and use in source and binary forms, with or without
8 modification, are permitted provided that the following conditions are met:
9
10 * Redistributions of source code must retain the above copyright
11 notice, this list of conditions and the following disclaimer.
12 * Redistributions in binary form must reproduce the above copyright
13 notice, this list of conditions and the following disclaimer in the
14 documentation and/or other materials provided with the distribution.
15 * Neither the name of DeNA Co.,Ltd. nor the names of its contributors
16 may be used to endorse or promote products derived from this software
17 without specific prior written permission.
18
19 THIS SOFTWARE IS PROVIDED BY DeNA Co.,Ltd. "AS IS" AND ANY EXPRESS OR
20 IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
21 MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO
22 EVENT SHALL DeNA Co.,Ltd. BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
23 SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
24 PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS;
25 OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
26 WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
27 OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
28 ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
29
30
310
=== added file 'HandlerSocket-Plugin-for-MySQL/ChangeLog'
--- HandlerSocket-Plugin-for-MySQL/ChangeLog 1970-01-01 00:00:00 +0000
+++ HandlerSocket-Plugin-for-MySQL/ChangeLog 2012-06-25 04:42:22 +0000
@@ -0,0 +1,15 @@
11.1.0 - 2011-12-29
2 * 1.1.0
3
41.0.6 - 2010-10-29
5 * Changed build instruction (autoreconf/configure/make), removed auto-generated files (Contributed by jmhodges)
6 *
7
81.0.5 - 2010-10-18
9 * Changed build procedures (using typical configure/make)
10 * Supported 5.5.6
11 * Added status variables
12
131.0.4 - 2010-08-15
14 * Initial public release
15
016
=== removed file 'HandlerSocket-Plugin-for-MySQL/ChangeLog'
--- HandlerSocket-Plugin-for-MySQL/ChangeLog 2011-01-10 13:39:35 +0000
+++ HandlerSocket-Plugin-for-MySQL/ChangeLog 1970-01-01 00:00:00 +0000
@@ -1,12 +0,0 @@
11.0.6 - 2010-10-29
2 * Changed build instruction (autoreconf/configure/make), removed auto-generated files (Contributed by jmhodges)
3 *
4
51.0.5 - 2010-10-18
6 * Changed build procedures (using typical configure/make)
7 * Supported 5.5.6
8 * Added status variables
9
101.0.4 - 2010-08-15
11 * Initial public release
12
130
=== added file 'HandlerSocket-Plugin-for-MySQL/Makefile.am'
--- HandlerSocket-Plugin-for-MySQL/Makefile.am 1970-01-01 00:00:00 +0000
+++ HandlerSocket-Plugin-for-MySQL/Makefile.am 2012-06-25 04:42:22 +0000
@@ -0,0 +1,87 @@
1
2ACLOCAL_AMFLAGS = -I m4
3
4SUBDIRS = @HANDLERSOCKET_SUBDIRS@
5
6perl:
7 cd perl-Net-HandlerSocket && perl Makefile.PL && make
8
9install_perl:
10 cd perl-Net-HandlerSocket && make install
11
12rpms: rpm_cli rpm_perl rpm_c
13
14rpm_dir:
15 - mkdir dist
16 - mkdir dist/BUILD dist/RPMS dist/SOURCES dist/SPECS dist/SRPMS
17
18rpm_cli: clean_cli rpm_dir
19 sed -e "s/HANDLERSOCKET_VERSION/$(VERSION)/" \
20 libhsclient/libhsclient.spec.template \
21 > libhsclient/libhsclient.spec
22 tar cvfz dist/libhsclient.tar.gz libhsclient
23 rpmbuild --define "_topdir `pwd`/dist" -ta \
24 dist/libhsclient.tar.gz
25
26rpm_perl: clean_perl rpm_dir
27 sed -e "s/HANDLERSOCKET_VERSION/$(VERSION)/" \
28 perl-Net-HandlerSocket/perl-Net-HandlerSocket.spec.template \
29 > perl-Net-HandlerSocket/perl-Net-HandlerSocket.spec
30 cd perl-Net-HandlerSocket && perl Makefile.PL && make clean && \
31 rm -f Makefile.old
32 tar cvfz dist/perl-Net-HandlerSocket.tar.gz perl-Net-HandlerSocket
33 rpmbuild --define "_topdir `pwd`/dist" -ta \
34 dist/perl-Net-HandlerSocket.tar.gz
35
36rpm_c: clean_c rpm_dir
37 sed -e "s/HANDLERSOCKET_VERSION/$(VERSION)/" \
38 handlersocket/handlersocket.spec.template \
39 > handlersocket/handlersocket.spec
40 sed -e "s|HANDLERSOCKET_MYSQL_INC|$(MYSQL_CFLAGS) $(MYSQL_INC)|" \
41 -e "s|HANDLERSOCKET_MYSQL_LIB|$(MYSQL_LIB)|" \
42 handlersocket/Makefile.plain.template \
43 > handlersocket/Makefile.plain
44 tar cvfz dist/handlersocket.tar.gz handlersocket
45 rpmbuild --define "_topdir `pwd`/dist" -ta \
46 dist/handlersocket.tar.gz
47
48install_rpm_pl:
49 - sudo rpm -e perl-Net-HandlerSocket
50 - sudo rpm -e perl-Net-HandlerSocket-debuginfo
51 make clean
52 make rpm_perl
53 - sudo rpm -U dist/RPMS/*/perl*.rpm
54
55installrpms:
56 - sudo rpm -e handlersocket
57 - sudo rpm -e handlersocket-debuginfo
58 - sudo rpm -e perl-Net-HandlerSocket
59 - sudo rpm -e perl-Net-HandlerSocket-debuginfo
60 - sudo rpm -e libhsclient
61 - sudo rpm -e libhsclient-debuginfo
62 make clean
63 make rpm_cli
64 - sudo rpm -U dist/RPMS/*/libhsclient*.rpm
65 make clean
66 make rpm_perl
67 - sudo rpm -U dist/RPMS/*/perl*.rpm
68 make clean
69 make rpm_c
70 - sudo rpm -U dist/RPMS/*/handlersocket*.rpm
71
72clean_cli:
73 cd libhsclient && make clean
74 cd client && make clean
75
76clean_perl:
77 cd perl-Net-HandlerSocket && perl Makefile.PL && make clean && \
78 rm -f Makefile.old
79
80clean_c:
81 cd handlersocket && make clean
82
83clean_all: clean_cli clean_perl clean_c
84 cd regtest && make clean
85 rm -rf dist/*/*
86 rm -f dist/*.tar.gz
87
088
=== removed file 'HandlerSocket-Plugin-for-MySQL/Makefile.am'
--- HandlerSocket-Plugin-for-MySQL/Makefile.am 2011-01-10 13:39:35 +0000
+++ HandlerSocket-Plugin-for-MySQL/Makefile.am 1970-01-01 00:00:00 +0000
@@ -1,87 +0,0 @@
1
2ACLOCAL_AMFLAGS = -I m4
3
4SUBDIRS = @HANDLERSOCKET_SUBDIRS@
5
6perl:
7 cd perl-Net-HandlerSocket && perl Makefile.PL && make
8
9install_perl:
10 cd perl-Net-HandlerSocket && make install
11
12rpms: rpm_cli rpm_perl rpm_c
13
14rpm_dir:
15 - mkdir dist
16 - mkdir dist/BUILD dist/RPMS dist/SOURCES dist/SPECS dist/SRPMS
17
18rpm_cli: clean_cli rpm_dir
19 sed -e "s/HANDLERSOCKET_VERSION/$(VERSION)/" \
20 libhsclient/libhsclient.spec.template \
21 > libhsclient/libhsclient.spec
22 tar cvfz dist/libhsclient.tar.gz libhsclient
23 rpmbuild --define "_topdir `pwd`/dist" -ta \
24 dist/libhsclient.tar.gz
25
26rpm_perl: clean_perl rpm_dir
27 sed -e "s/HANDLERSOCKET_VERSION/$(VERSION)/" \
28 perl-Net-HandlerSocket/perl-Net-HandlerSocket.spec.template \
29 > perl-Net-HandlerSocket/perl-Net-HandlerSocket.spec
30 cd perl-Net-HandlerSocket && perl Makefile.PL && make clean && \
31 rm -f Makefile.old
32 tar cvfz dist/perl-Net-HandlerSocket.tar.gz perl-Net-HandlerSocket
33 rpmbuild --define "_topdir `pwd`/dist" -ta \
34 dist/perl-Net-HandlerSocket.tar.gz
35
36rpm_c: clean_c rpm_dir
37 sed -e "s/HANDLERSOCKET_VERSION/$(VERSION)/" \
38 handlersocket/handlersocket.spec.template \
39 > handlersocket/handlersocket.spec
40 sed -e "s|HANDLERSOCKET_MYSQL_INC|$(MYSQL_CFLAGS) $(MYSQL_INC)|" \
41 -e "s|HANDLERSOCKET_MYSQL_LIB|$(MYSQL_LIB)|" \
42 handlersocket/Makefile.plain.template \
43 > handlersocket/Makefile.plain
44 tar cvfz dist/handlersocket.tar.gz handlersocket
45 rpmbuild --define "_topdir `pwd`/dist" -ta \
46 dist/handlersocket.tar.gz
47
48install_rpm_pl:
49 - sudo rpm -e perl-Net-HandlerSocket
50 - sudo rpm -e perl-Net-HandlerSocket-debuginfo
51 make clean
52 make rpm_perl
53 - sudo rpm -U dist/RPMS/*/perl*.rpm
54
55installrpms:
56 - sudo rpm -e handlersocket
57 - sudo rpm -e handlersocket-debuginfo
58 - sudo rpm -e perl-Net-HandlerSocket
59 - sudo rpm -e perl-Net-HandlerSocket-debuginfo
60 - sudo rpm -e libhsclient
61 - sudo rpm -e libhsclient-debuginfo
62 make clean
63 make rpm_cli
64 - sudo rpm -U dist/RPMS/*/libhsclient*.rpm
65 make clean
66 make rpm_perl
67 - sudo rpm -U dist/RPMS/*/perl*.rpm
68 make clean
69 make rpm_c
70 - sudo rpm -U dist/RPMS/*/handlersocket*.rpm
71
72clean_cli:
73 cd libhsclient && make clean
74 cd client && make clean
75
76clean_perl:
77 cd perl-Net-HandlerSocket && perl Makefile.PL && make clean && \
78 rm -f Makefile.old
79
80clean_c:
81 cd handlersocket && make clean
82
83clean_all: clean_cli clean_perl clean_c
84 cd regtest && make clean
85 rm -rf dist/*/*
86 rm -f dist/*.tar.gz
87
880
=== added file 'HandlerSocket-Plugin-for-MySQL/README'
--- HandlerSocket-Plugin-for-MySQL/README 1970-01-01 00:00:00 +0000
+++ HandlerSocket-Plugin-for-MySQL/README 2012-06-25 04:42:22 +0000
@@ -0,0 +1,78 @@
1
2-----------------------------------------------------------------------------
3HandlerSocket plugin for MySQL
4
5Copyright (c) 2010 DeNA Co.,Ltd.
6All rights reserved.
7
8Redistribution and use in source and binary forms, with or without
9modification, are permitted provided that the following conditions are met:
10
11 * Redistributions of source code must retain the above copyright
12 notice, this list of conditions and the following disclaimer.
13 * Redistributions in binary form must reproduce the above copyright
14 notice, this list of conditions and the following disclaimer in the
15 documentation and/or other materials provided with the distribution.
16 * Neither the name of DeNA Co.,Ltd. nor the names of its contributors
17 may be used to endorse or promote products derived from this software
18 without specific prior written permission.
19
20THIS SOFTWARE IS PROVIDED BY DeNA Co.,Ltd. "AS IS" AND ANY EXPRESS OR
21IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
22MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO
23EVENT SHALL DeNA Co.,Ltd. BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
24SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
25PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS;
26OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
27WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
28OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
29ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
30
31
32-----------------------------------------------------------------------------
33About HandlerSocket
34
35HandlerSocket is a NoSQL plugin for MySQL. It works as a daemon inside the
36mysqld process, accept tcp connections, and execute requests from clients.
37HandlerSocket does not support SQL queries. Instead, it supports simple CRUD
38operations on tables.
39
40Because of the following reasons, HandlerSocket is much faster than the
41mysqld/libmysql pair in some circumstances:
42
43 - HandlerSocket manipulates data without parsing SQL, which causes less
44 CPU usage.
45 - HandlerSocket reads many requests from clients and executes their
46 requests in bulk, which causes less CPU and disk usage.
47 - HandlerSocket client/server protocol is more compact than the
48 mysql/libmysql pair, which causes less network usage.
49
50The current version of HandlerSocket only works with GNU/Linux. The source
51archive of HandlerSocket includes a C++ and a Perl client libraries.
52Here is a list of client libraries for other languages:
53
54 - PHP
55 http://openpear.org/package/Net_HandlerSocket
56 http://github.com/tz-lom/HSPHP
57 http://code.google.com/p/php-handlersocket/
58 - Java
59 http://code.google.com/p/hs4j/
60 http://code.google.com/p/handlersocketforjava/
61 - Python
62 http://pypi.python.org/pypi/python-handler-socket
63 https://code.launchpad.net/~songofacandy/+junk/pyhandlersocket
64 - Ruby
65 https://github.com/winebarrel/ruby-handlersocket
66 https://github.com/miyucy/handlersocket
67 - JavaScript
68 https://github.com/koichik/node-handlersocket
69 - Scala
70 https://github.com/fujohnwang/hs2client
71 - Haskell
72 https://github.com/wuxb45/HandlerSocket-Haskell-Client
73
74The home of HandlerSocket is here:
75 https://github.com/ahiguti/HandlerSocket-Plugin-for-MySQL
76
77More documents are available in docs-en/ and docs-ja/ directories.
78
079
=== removed file 'HandlerSocket-Plugin-for-MySQL/README'
--- HandlerSocket-Plugin-for-MySQL/README 2011-01-10 13:39:35 +0000
+++ HandlerSocket-Plugin-for-MySQL/README 1970-01-01 00:00:00 +0000
@@ -1,76 +0,0 @@
1
2-----------------------------------------------------------------------------
3HandlerSocket plugin for MySQL
4
5Copyright (c) 2010 DeNA Co.,Ltd.
6All rights reserved.
7
8Redistribution and use in source and binary forms, with or without
9modification, are permitted provided that the following conditions are met:
10
11 * Redistributions of source code must retain the above copyright
12 notice, this list of conditions and the following disclaimer.
13 * Redistributions in binary form must reproduce the above copyright
14 notice, this list of conditions and the following disclaimer in the
15 documentation and/or other materials provided with the distribution.
16 * Neither the name of DeNA Co.,Ltd. nor the names of its contributors
17 may be used to endorse or promote products derived from this software
18 without specific prior written permission.
19
20THIS SOFTWARE IS PROVIDED BY DeNA Co.,Ltd. "AS IS" AND ANY EXPRESS OR
21IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
22MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO
23EVENT SHALL DeNA Co.,Ltd. BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
24SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
25PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS;
26OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
27WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
28OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
29ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
30
31
32-----------------------------------------------------------------------------
33About HandlerSocket
34
35HandlerSocket is a NoSQL plugin for MySQL. It works as a daemon inside the
36mysqld process, accept tcp connections, and execute requests from clients.
37HandlerSocket does not support SQL queries. Instead, it supports simple CRUD
38operations on tables.
39
40Because of the following reasons, HandlerSocket is much faster than the
41mysqld/libmysql pair in some circumstances:
42
43 - HandlerSocket manipulates data without parsing SQL, which causes less
44 CPU usage.
45 - HandlerSocket reads many requests from clients and executes their
46 requests in bulk, which causes less CPU and disk usage.
47 - HandlerSocket client/server protocol is more compact than the
48 mysql/libmysql pair, which causes less network usage.
49
50The current version of HandlerSocket only works with GNU/Linux. The source
51archive of HandlerSocket includes a C++ and a Perl client libraries.
52Here is a list of client libraries for other languages:
53
54 - PHP
55 http://openpear.org/package/Net_HandlerSocket
56 http://github.com/tz-lom/HSPHP
57 http://code.google.com/p/php-handlersocket/
58 - Java
59 http://code.google.com/p/hs4j/
60 http://code.google.com/p/handlersocketforjava/
61 - Python
62 http://pypi.python.org/pypi/python-handler-socket
63 https://code.launchpad.net/~songofacandy/+junk/pyhandlersocket
64 - Ruby
65 https://github.com/winebarrel/ruby-handlersocket
66 https://github.com/miyucy/handlersocket
67 - JavaScript
68 https://github.com/koichik/node-handlersocket
69 - Scala
70 https://github.com/fujohnwang/hs2client
71
72The home of HandlerSocket is here:
73 https://github.com/ahiguti/HandlerSocket-Plugin-for-MySQL
74
75More documents are available in docs-en/ and docs-ja/ directories.
76
770
=== added file 'HandlerSocket-Plugin-for-MySQL/autogen.sh'
--- HandlerSocket-Plugin-for-MySQL/autogen.sh 1970-01-01 00:00:00 +0000
+++ HandlerSocket-Plugin-for-MySQL/autogen.sh 2012-06-25 04:42:22 +0000
@@ -0,0 +1,117 @@
1#!/bin/sh
2
3warn() {
4echo -e "\tWARNING: $@" 1>&2
5}
6
7# init
8
9LIBTOOLIZE=libtoolize
10ACLOCAL=aclocal
11AUTOCONF=autoconf
12AUTOHEADER=autoheader
13AUTOMAKE=automake
14
15case `uname -s` in
16Darwin)
17LIBTOOLIZE=glibtoolize
18;;
19FreeBSD)
20ACLOCAL_ARGS="$ACLOCAL_ARGS -I /usr/local/share/aclocal/"
21;;
22esac
23
24
25# libtoolize
26echo "Searching libtoolize..."
27if [ `which $LIBTOOLIZE` ] ; then
28echo -e "\tFOUND: libtoolize -> $LIBTOOLIZE"
29else
30warn "Cannot Found libtoolize... input libtool command"
31 read LIBTOOLIZE
32 LIBTOOLIZE=`which $LIBTOOLIZE`
33 if [ `which $LIBTOOLIZE` ] ; then
34echo -e "\tSET: libtoolize -> $LIBTOOLIZE"
35 else
36warn "$LIBTOOLIZE: Command not found."
37 exit 1;
38 fi
39fi
40
41# aclocal
42echo "Searching aclocal..."
43if [ `which $ACLOCAL` ] ; then
44echo -e "\tFOUND: aclocal -> $ACLOCAL"
45else
46warn "Cannot Found aclocal... input aclocal command"
47 read ACLOCAL
48 ACLOCAL=`which $ACLOCAL`
49 if [ `which $ACLOCAL` ] ; then
50echo -e "\tSET: aclocal -> $ACLOCAL"
51 else
52warn "$ACLOCAL: Command not found."
53 exit 1;
54 fi
55fi
56
57# automake
58echo "Searching automake..."
59if [ `which $AUTOMAKE` ] ; then
60echo -e "\tFOUND: automake -> $AUTOMAKE"
61else
62warn "Cannot Found automake... input automake command"
63 read AUTOMAKE
64 ACLOCAL=`which $AUTOMAKE`
65 if [ `which $AUTOMAKE` ] ; then
66echo -e "\tSET: automake -> $AUTOMAKE"
67 else
68warn "$AUTOMAKE: Command not found."
69 exit 1;
70 fi
71fi
72
73# autoheader
74echo "Searching autoheader..."
75if [ `which $AUTOHEADER` ] ; then
76echo -e "\tFOUND: autoheader -> $AUTOHEADER"
77else
78warn "Cannot Found autoheader... input autoheader command"
79 read AUTOHEADER
80 ACLOCAL=`which $AUTOHEADER`
81 if [ `which $AUTOHEADER` ] ; then
82echo -e "\tSET: autoheader -> $AUTOHEADER"
83 else
84warn "$AUTOHEADER: Command not found."
85 exit 1;
86 fi
87fi
88
89# autoconf
90echo "Searching autoconf..."
91if [ `which $AUTOCONF` ] ; then
92echo -e "\tFOUND: autoconf -> $AUTOCONF"
93else
94warn "Cannot Found autoconf... input autoconf command"
95 read AUTOCONF
96 ACLOCAL=`which $AUTOCONF`
97 if [ `which $AUTOCONF` ] ; then
98echo -e "\tSET: autoconf -> $AUTOCONF"
99 else
100warn "$AUTOCONF: Command not found."
101 exit 1;
102 fi
103fi
104
105echo "Running libtoolize ..."
106$LIBTOOLIZE --force --copy
107echo "Running aclocal ..."
108$ACLOCAL ${ACLOCAL_ARGS} -I .
109echo "Running autoheader..."
110$AUTOHEADER
111echo "Running automake ..."
112$AUTOMAKE --add-missing --copy
113echo "Running autoconf ..."
114$AUTOCONF
115
116mkdir -p m4
117
0118
=== removed file 'HandlerSocket-Plugin-for-MySQL/autogen.sh'
--- HandlerSocket-Plugin-for-MySQL/autogen.sh 2011-04-19 12:57:00 +0000
+++ HandlerSocket-Plugin-for-MySQL/autogen.sh 1970-01-01 00:00:00 +0000
@@ -1,117 +0,0 @@
1#!/bin/sh
2
3warn() {
4echo -e "\tWARNING: $@" 1>&2
5}
6
7# init
8
9LIBTOOLIZE=libtoolize
10ACLOCAL=aclocal
11AUTOCONF=autoconf
12AUTOHEADER=autoheader
13AUTOMAKE=automake
14
15case `uname -s` in
16Darwin)
17LIBTOOLIZE=glibtoolize
18;;
19FreeBSD)
20ACLOCAL_ARGS="$ACLOCAL_ARGS -I /usr/local/share/aclocal/"
21;;
22esac
23
24
25# libtoolize
26echo "Searching libtoolize..."
27if [ `which $LIBTOOLIZE` ] ; then
28echo -e "\tFOUND: libtoolize -> $LIBTOOLIZE"
29else
30warn "Cannot Found libtoolize... input libtool command"
31 read LIBTOOLIZE
32 LIBTOOLIZE=`which $LIBTOOLIZE`
33 if [ `which $LIBTOOLIZE` ] ; then
34echo -e "\tSET: libtoolize -> $LIBTOOLIZE"
35 else
36warn "$LIBTOOLIZE: Command not found."
37 exit 1;
38 fi
39fi
40
41# aclocal
42echo "Searching aclocal..."
43if [ `which $ACLOCAL` ] ; then
44echo -e "\tFOUND: aclocal -> $ACLOCAL"
45else
46warn "Cannot Found aclocal... input aclocal command"
47 read ACLOCAL
48 ACLOCAL=`which $ACLOCAL`
49 if [ `which $ACLOCAL` ] ; then
50echo -e "\tSET: aclocal -> $ACLOCAL"
51 else
52warn "$ACLOCAL: Command not found."
53 exit 1;
54 fi
55fi
56
57# automake
58echo "Searching automake..."
59if [ `which $AUTOMAKE` ] ; then
60echo -e "\tFOUND: automake -> $AUTOMAKE"
61else
62warn "Cannot Found automake... input automake command"
63 read AUTOMAKE
64 ACLOCAL=`which $AUTOMAKE`
65 if [ `which $AUTOMAKE` ] ; then
66echo -e "\tSET: automake -> $AUTOMAKE"
67 else
68warn "$AUTOMAKE: Command not found."
69 exit 1;
70 fi
71fi
72
73# autoheader
74echo "Searching autoheader..."
75if [ `which $AUTOHEADER` ] ; then
76echo -e "\tFOUND: autoheader -> $AUTOHEADER"
77else
78warn "Cannot Found autoheader... input autoheader command"
79 read AUTOHEADER
80 ACLOCAL=`which $AUTOHEADER`
81 if [ `which $AUTOHEADER` ] ; then
82echo -e "\tSET: autoheader -> $AUTOHEADER"
83 else
84warn "$AUTOHEADER: Command not found."
85 exit 1;
86 fi
87fi
88
89# autoconf
90echo "Searching autoconf..."
91if [ `which $AUTOCONF` ] ; then
92echo -e "\tFOUND: autoconf -> $AUTOCONF"
93else
94warn "Cannot Found autoconf... input autoconf command"
95 read AUTOCONF
96 ACLOCAL=`which $AUTOCONF`
97 if [ `which $AUTOCONF` ] ; then
98echo -e "\tSET: autoconf -> $AUTOCONF"
99 else
100warn "$AUTOCONF: Command not found."
101 exit 1;
102 fi
103fi
104
105echo "Running libtoolize ..."
106$LIBTOOLIZE --force --copy
107echo "Running aclocal ..."
108$ACLOCAL ${ACLOCAL_ARGS} -I .
109echo "Running autoheader..."
110$AUTOHEADER
111echo "Running automake ..."
112$AUTOMAKE --add-missing --copy
113echo "Running autoconf ..."
114$AUTOCONF
115
116mkdir -p m4
117
1180
=== added directory 'HandlerSocket-Plugin-for-MySQL/client'
=== removed directory 'HandlerSocket-Plugin-for-MySQL/client'
=== added file 'HandlerSocket-Plugin-for-MySQL/client/Makefile.am'
--- HandlerSocket-Plugin-for-MySQL/client/Makefile.am 1970-01-01 00:00:00 +0000
+++ HandlerSocket-Plugin-for-MySQL/client/Makefile.am 2012-06-25 04:42:22 +0000
@@ -0,0 +1,24 @@
1AM_INCLUDES= -I../libhsclient
2bin_PROGRAMS=hsclient
3hsclient_SOURCES= hsclient.cpp
4hsclient_LDFLAGS= -static -L../libhsclient -lhsclient
5hsclient_CXXFLAGS= $(AM_INCLUDES)
6
7hstest: hstest.o
8 $(CXX) $(CXXFLAGS) $(LFLAGS) hstest.o \
9 -L../libhsclient/.libs -lhsclient $$(mysql_config --libs_r) \
10 -o hstest
11
12hstest.o: hstest.cpp
13 $(CXX) $(CXXFLAGS) $(AM_INCLUDES) $$(mysql_config --include) \
14 -c hstest.cpp
15
16hslongrun: hslongrun.o
17 $(CXX) $(CXXFLAGS) $(LFLAGS) hslongrun.o \
18 -L../libhsclient/.libs -lhsclient $$(mysql_config --libs_r) \
19 -o hslongrun
20
21hslongrun.o: hslongrun.cpp
22 $(CXX) $(CXXFLAGS) $(AM_INCLUDES) $$(mysql_config --include) \
23 -c hslongrun.cpp
24
025
=== removed file 'HandlerSocket-Plugin-for-MySQL/client/Makefile.am'
--- HandlerSocket-Plugin-for-MySQL/client/Makefile.am 2011-04-19 12:57:00 +0000
+++ HandlerSocket-Plugin-for-MySQL/client/Makefile.am 1970-01-01 00:00:00 +0000
@@ -1,24 +0,0 @@
1AM_INCLUDES= -I../libhsclient
2bin_PROGRAMS=hsclient
3hsclient_SOURCES= hsclient.cpp
4hsclient_LDFLAGS= -static -L../libhsclient -lhsclient
5hsclient_CXXFLAGS= $(AM_INCLUDES)
6
7hstest: hstest.o
8 $(CXX) $(CXXFLAGS) $(LFLAGS) hstest.o \
9 -L../libhsclient/.libs -lhsclient $$(mysql_config --libs_r) \
10 -o hstest
11
12hstest.o: hstest.cpp
13 $(CXX) $(CXXFLAGS) $(AM_INCLUDES) $$(mysql_config --include) \
14 -c hstest.cpp
15
16hslongrun: hslongrun.o
17 $(CXX) $(CXXFLAGS) $(LFLAGS) hslongrun.o \
18 -L../libhsclient/.libs -lhsclient $$(mysql_config --libs_r) \
19 -o hslongrun
20
21hslongrun.o: hslongrun.cpp
22 $(CXX) $(CXXFLAGS) $(AM_INCLUDES) $$(mysql_config --include) \
23 -c hslongrun.cpp
24
250
=== added file 'HandlerSocket-Plugin-for-MySQL/client/hsclient.cpp'
--- HandlerSocket-Plugin-for-MySQL/client/hsclient.cpp 1970-01-01 00:00:00 +0000
+++ HandlerSocket-Plugin-for-MySQL/client/hsclient.cpp 2012-06-25 04:42:22 +0000
@@ -0,0 +1,88 @@
1
2// vim:sw=2:ai
3
4#include "hstcpcli.hpp"
5#include "string_util.hpp"
6
7namespace dena {
8
9int
10hstcpcli_main(int argc, char **argv)
11{
12 config conf;
13 parse_args(argc, argv, conf);
14 socket_args sockargs;
15 sockargs.set(conf);
16 hstcpcli_ptr cli = hstcpcli_i::create(sockargs);
17 const std::string dbname = conf.get_str("dbname", "hstest");
18 const std::string table = conf.get_str("table", "hstest_table1");
19 const std::string index = conf.get_str("index", "PRIMARY");
20 const std::string fields = conf.get_str("fields", "k,v");
21 const int limit = conf.get_int("limit", 0);
22 const int skip = conf.get_int("skip", 0);
23 std::vector<std::string> keys;
24 std::vector<string_ref> keyrefs;
25 size_t num_keys = 0;
26 while (true) {
27 const std::string conf_key = std::string("k") + to_stdstring(num_keys);
28 const std::string k = conf.get_str(conf_key, "");
29 const std::string kx = conf.get_str(conf_key, "x");
30 if (k.empty() && kx == "x") {
31 break;
32 }
33 ++num_keys;
34 keys.push_back(k);
35 }
36 for (size_t i = 0; i < keys.size(); ++i) {
37 const string_ref ref(keys[i].data(), keys[i].size());
38 keyrefs.push_back(ref);
39 }
40 const std::string op = conf.get_str("op", "=");
41 const string_ref op_ref(op.data(), op.size());
42 cli->request_buf_open_index(0, dbname.c_str(), table.c_str(),
43 index.c_str(), fields.c_str());
44 cli->request_buf_exec_generic(0, op_ref, num_keys == 0 ? 0 : &keyrefs[0],
45 num_keys, limit, skip, string_ref(), 0, 0);
46 int code = 0;
47 size_t numflds = 0;
48 do {
49 if (cli->request_send() != 0) {
50 fprintf(stderr, "request_send: %s\n", cli->get_error().c_str());
51 break;
52 }
53 if ((code = cli->response_recv(numflds)) != 0) {
54 fprintf(stderr, "response_recv: %s\n", cli->get_error().c_str());
55 break;
56 }
57 } while (false);
58 cli->response_buf_remove();
59 do {
60 if ((code = cli->response_recv(numflds)) != 0) {
61 fprintf(stderr, "response_recv: %s\n", cli->get_error().c_str());
62 break;
63 }
64 while (true) {
65 const string_ref *const row = cli->get_next_row();
66 if (row == 0) {
67 break;
68 }
69 printf("REC:");
70 for (size_t i = 0; i < numflds; ++i) {
71 const std::string val(row[i].begin(), row[i].size());
72 printf(" %s", val.c_str());
73 }
74 printf("\n");
75 }
76 } while (false);
77 cli->response_buf_remove();
78 return 0;
79}
80
81};
82
83int
84main(int argc, char **argv)
85{
86 return dena::hstcpcli_main(argc, argv);
87}
88
089
=== removed file 'HandlerSocket-Plugin-for-MySQL/client/hsclient.cpp'
--- HandlerSocket-Plugin-for-MySQL/client/hsclient.cpp 2011-01-10 13:39:35 +0000
+++ HandlerSocket-Plugin-for-MySQL/client/hsclient.cpp 1970-01-01 00:00:00 +0000
@@ -1,88 +0,0 @@
1
2// vim:sw=2:ai
3
4#include "hstcpcli.hpp"
5#include "string_util.hpp"
6
7namespace dena {
8
9int
10hstcpcli_main(int argc, char **argv)
11{
12 config conf;
13 parse_args(argc, argv, conf);
14 socket_args sockargs;
15 sockargs.set(conf);
16 hstcpcli_ptr cli = hstcpcli_i::create(sockargs);
17 const std::string dbname = conf.get_str("dbname", "hstest");
18 const std::string table = conf.get_str("table", "hstest_table1");
19 const std::string index = conf.get_str("index", "PRIMARY");
20 const std::string fields = conf.get_str("fields", "k,v");
21 const int limit = conf.get_int("limit", 0);
22 const int skip = conf.get_int("skip", 0);
23 std::vector<std::string> keys;
24 std::vector<string_ref> keyrefs;
25 size_t num_keys = 0;
26 while (true) {
27 const std::string conf_key = std::string("k") + to_stdstring(num_keys);
28 const std::string k = conf.get_str(conf_key, "");
29 const std::string kx = conf.get_str(conf_key, "x");
30 if (k.empty() && kx == "x") {
31 break;
32 }
33 ++num_keys;
34 keys.push_back(k);
35 }
36 for (size_t i = 0; i < keys.size(); ++i) {
37 const string_ref ref(keys[i].data(), keys[i].size());
38 keyrefs.push_back(ref);
39 }
40 const std::string op = conf.get_str("op", "=");
41 const string_ref op_ref(op.data(), op.size());
42 cli->request_buf_open_index(0, dbname.c_str(), table.c_str(),
43 index.c_str(), fields.c_str());
44 cli->request_buf_exec_generic(0, op_ref, num_keys == 0 ? 0 : &keyrefs[0],
45 num_keys, limit, skip, string_ref(), 0, 0);
46 int code = 0;
47 size_t numflds = 0;
48 do {
49 if (cli->request_send() != 0) {
50 fprintf(stderr, "request_send: %s\n", cli->get_error().c_str());
51 break;
52 }
53 if ((code = cli->response_recv(numflds)) != 0) {
54 fprintf(stderr, "response_recv: %s\n", cli->get_error().c_str());
55 break;
56 }
57 } while (false);
58 cli->response_buf_remove();
59 do {
60 if ((code = cli->response_recv(numflds)) != 0) {
61 fprintf(stderr, "response_recv: %s\n", cli->get_error().c_str());
62 break;
63 }
64 while (true) {
65 const string_ref *const row = cli->get_next_row();
66 if (row == 0) {
67 break;
68 }
69 printf("REC:");
70 for (size_t i = 0; i < numflds; ++i) {
71 const std::string val(row[i].begin(), row[i].size());
72 printf(" %s", val.c_str());
73 }
74 printf("\n");
75 }
76 } while (false);
77 cli->response_buf_remove();
78 return 0;
79}
80
81};
82
83int
84main(int argc, char **argv)
85{
86 return dena::hstcpcli_main(argc, argv);
87}
88
890
=== added file 'HandlerSocket-Plugin-for-MySQL/client/hslongrun.cpp'
--- HandlerSocket-Plugin-for-MySQL/client/hslongrun.cpp 1970-01-01 00:00:00 +0000
+++ HandlerSocket-Plugin-for-MySQL/client/hslongrun.cpp 2012-06-25 04:42:22 +0000
@@ -0,0 +1,1041 @@
1
2// vim:sw=2:ai
3
4#include <signal.h>
5#include <sys/time.h>
6#include <stdio.h>
7#include <string.h>
8#include <vector>
9#include <map>
10#include <stdlib.h>
11#include <memory>
12#include <errno.h>
13#include <mysql.h>
14#include <time.h>
15#include <sys/types.h>
16#include <sys/stat.h>
17#include <fcntl.h>
18
19#include "util.hpp"
20#include "auto_ptrcontainer.hpp"
21#include "socket.hpp"
22#include "hstcpcli.hpp"
23#include "string_util.hpp"
24#include "mutex.hpp"
25
26namespace dena {
27
28struct auto_mysql : private noncopyable {
29 auto_mysql() : db(0) {
30 reset();
31 }
32 ~auto_mysql() {
33 if (db) {
34 mysql_close(db);
35 }
36 }
37 void reset() {
38 if (db) {
39 mysql_close(db);
40 }
41 if ((db = mysql_init(0)) == 0) {
42 fatal_exit("failed to initialize mysql client");
43 }
44 }
45 operator MYSQL *() const { return db; }
46 private:
47 MYSQL *db;
48};
49
50struct auto_mysql_res : private noncopyable {
51 auto_mysql_res(MYSQL *db) {
52 res = mysql_store_result(db);
53 }
54 ~auto_mysql_res() {
55 if (res) {
56 mysql_free_result(res);
57 }
58 }
59 operator MYSQL_RES *() const { return res; }
60 private:
61 MYSQL_RES *res;
62};
63
64struct auto_mysql_stmt : private noncopyable {
65 auto_mysql_stmt(MYSQL *db) {
66 stmt = mysql_stmt_init(db);
67 }
68 ~auto_mysql_stmt() {
69 if (stmt) {
70 mysql_stmt_close(stmt);
71 }
72 }
73 operator MYSQL_STMT *() const { return stmt; }
74 private:
75 MYSQL_STMT *stmt;
76};
77
78double
79gettimeofday_double()
80{
81 struct timeval tv = { };
82 if (gettimeofday(&tv, 0) != 0) {
83 fatal_abort("gettimeofday");
84 }
85 return static_cast<double>(tv.tv_usec) / 1000000 + tv.tv_sec;
86}
87
88struct record_value {
89 mutex lock;
90 bool deleted;
91 bool unknown_state;
92 std::string key;
93 std::vector<std::string> values;
94 record_value() : deleted(true), unknown_state(false) { }
95};
96
97struct hs_longrun_shared {
98 config conf;
99 socket_args arg;
100 int verbose;
101 long num_threads;
102 int usleep;
103 volatile mutable int running;
104 auto_ptrcontainer< std::vector<record_value *> > records;
105 hs_longrun_shared() : verbose(0), num_threads(0), usleep(0), running(1) { }
106};
107
108struct thread_base {
109 thread_base() : need_join(false), stack_size(256 * 1024) { }
110 virtual ~thread_base() {
111 join();
112 }
113 virtual void run() = 0;
114 void start() {
115 if (!start_nothrow()) {
116 fatal_abort("thread::start");
117 }
118 }
119 bool start_nothrow() {
120 if (need_join) {
121 return need_join; /* true */
122 }
123 void *const arg = this;
124 pthread_attr_t attr;
125 if (pthread_attr_init(&attr) != 0) {
126 fatal_abort("pthread_attr_init");
127 }
128 if (pthread_attr_setstacksize(&attr, stack_size) != 0) {
129 fatal_abort("pthread_attr_setstacksize");
130 }
131 const int r = pthread_create(&thr, &attr, thread_main, arg);
132 if (pthread_attr_destroy(&attr) != 0) {
133 fatal_abort("pthread_attr_destroy");
134 }
135 if (r != 0) {
136 return need_join; /* false */
137 }
138 need_join = true;
139 return need_join; /* true */
140 }
141 void join() {
142 if (!need_join) {
143 return;
144 }
145 int e = 0;
146 if ((e = pthread_join(thr, 0)) != 0) {
147 fatal_abort("pthread_join");
148 }
149 need_join = false;
150 }
151 private:
152 static void *thread_main(void *arg) {
153 thread_base *p = static_cast<thread_base *>(arg);
154 p->run();
155 return 0;
156 }
157 private:
158 pthread_t thr;
159 bool need_join;
160 size_t stack_size;
161};
162
163struct hs_longrun_stat {
164 unsigned long long verify_error_count;
165 unsigned long long runtime_error_count;
166 unsigned long long unknown_count;
167 unsigned long long success_count;
168 hs_longrun_stat()
169 : verify_error_count(0), runtime_error_count(0),
170 unknown_count(0), success_count(0) { }
171 void add(const hs_longrun_stat& x) {
172 verify_error_count += x.verify_error_count;
173 runtime_error_count += x.runtime_error_count;
174 unknown_count += x.unknown_count;
175 success_count += x.success_count;
176 }
177};
178
179struct hs_longrun_thread_base : public thread_base {
180 struct arg_type {
181 int id;
182 std::string worker_type;
183 char op;
184 int lock_flag;
185 const hs_longrun_shared& sh;
186 arg_type(int id, const std::string& worker_type, char op, int lock_flag,
187 const hs_longrun_shared& sh)
188 : id(id), worker_type(worker_type), op(op), lock_flag(lock_flag),
189 sh(sh) { }
190 };
191 arg_type arg;
192 hs_longrun_stat stat;
193 drand48_data randbuf;
194 unsigned int seed;
195 hs_longrun_thread_base(const arg_type& arg)
196 : arg(arg), seed(0) {
197 seed = time(0) + arg.id + 1;
198 srand48_r(seed, &randbuf);
199 }
200 virtual ~hs_longrun_thread_base() { }
201 virtual void run() = 0;
202 size_t rand_record() {
203 double v = 0;
204 drand48_r(&randbuf, &v);
205 const size_t sz = arg.sh.records.size();
206 size_t r = size_t(v * sz);
207 if (r >= sz) {
208 r = 0;
209 }
210 return r;
211 }
212 int verify_update(const std::string& k, const std::string& v1,
213 const std::string& v2, const std::string& v3, record_value& rec,
214 uint32_t num_rows, bool cur_unknown_state);
215 int verify_read(const std::string& k, uint32_t num_rows, uint32_t num_flds,
216 const std::string rrec[4], record_value& rec);
217 int verify_readnolock(const std::string& k, uint32_t num_rows,
218 uint32_t num_flds, const std::string rrec[4]);
219};
220
221int
222hs_longrun_thread_base::verify_update(const std::string& k,
223 const std::string& v1, const std::string& v2, const std::string& v3,
224 record_value& rec, uint32_t num_rows, bool cur_unknown_state)
225{
226 const bool op_success = num_rows == 1;
227 int ret = 0;
228 if (!rec.unknown_state) {
229 if (!rec.deleted && !op_success) {
230 ++stat.verify_error_count;
231 if (arg.sh.verbose > 0) {
232 fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
233 "unexpected_update_failure\n",
234 arg.worker_type.c_str(), arg.id, k.c_str());
235 }
236 ret = 1;
237 } else if (rec.deleted && op_success) {
238 ++stat.verify_error_count;
239 if (arg.sh.verbose > 0) {
240 fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
241 "unexpected_update_success\n",
242 arg.worker_type.c_str(), arg.id, k.c_str());
243 }
244 ret = 1;
245 }
246 }
247 if (op_success) {
248 rec.values.resize(4);
249 rec.values[0] = k;
250 rec.values[1] = v1;
251 rec.values[2] = v2;
252 rec.values[3] = v3;
253 if (ret == 0 && !rec.unknown_state) {
254 ++stat.success_count;
255 }
256 }
257 rec.unknown_state = cur_unknown_state;
258 if (arg.sh.verbose >= 100 && ret == 0) {
259 fprintf(stderr, "%s %s %s %s %s\n", arg.worker_type.c_str(),
260 k.c_str(), v1.c_str(), v2.c_str(), v3.c_str());
261 }
262 return ret;
263}
264
265int
266hs_longrun_thread_base::verify_read(const std::string& k,
267 uint32_t num_rows, uint32_t num_flds, const std::string rrec[4],
268 record_value& rec)
269{
270 const bool op_success = num_rows != 0;
271 int ret = 0;
272 if (!rec.unknown_state) {
273 if (!rec.deleted && !op_success) {
274 ++stat.verify_error_count;
275 if (arg.sh.verbose > 0) {
276 fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
277 "unexpected_read_failure\n",
278 arg.worker_type.c_str(), arg.id, k.c_str());
279 }
280 ret = 1;
281 } else if (rec.deleted && op_success) {
282 ++stat.verify_error_count;
283 if (arg.sh.verbose > 0) {
284 fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
285 "unexpected_read_success\n",
286 arg.worker_type.c_str(), arg.id, k.c_str());
287 }
288 ret = 1;
289 } else if (num_flds != 4) {
290 ++stat.verify_error_count;
291 if (arg.sh.verbose > 0) {
292 fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
293 "unexpected_read_fldnum %d\n",
294 arg.worker_type.c_str(), arg.id, k.c_str(),
295 static_cast<int>(num_flds));
296 }
297 ret = 1;
298 } else if (rec.deleted) {
299 /* nothing to verify */
300 } else {
301 int diff = 0;
302 for (size_t i = 0; i < 4; ++i) {
303 if (rec.values[i] == rrec[i]) {
304 /* ok */
305 } else {
306 diff = 1;
307 }
308 }
309 if (diff) {
310 std::string mess;
311 for (size_t i = 0; i < 4; ++i) {
312 const std::string& expected = rec.values[i];
313 const std::string& val = rrec[i];
314 mess += " " + val + "/" + expected;
315 }
316 if (arg.sh.verbose > 0) {
317 fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
318 "unexpected_read_value %s\n",
319 arg.worker_type.c_str(), arg.id, k.c_str(), mess.c_str());
320 }
321 ret = 1;
322 }
323 }
324 }
325 if (arg.sh.verbose >= 100 && ret == 0) {
326 fprintf(stderr, "%s %s\n", arg.worker_type.c_str(), k.c_str());
327 }
328 if (ret == 0 && !rec.unknown_state) {
329 ++stat.success_count;
330 }
331 return ret;
332}
333
334int
335hs_longrun_thread_base::verify_readnolock(const std::string& k,
336 uint32_t num_rows, uint32_t num_flds, const std::string rrec[4])
337{
338 int ret = 0;
339 if (num_rows != 1 || num_flds != 4) {
340 ++stat.verify_error_count;
341 if (arg.sh.verbose > 0) {
342 fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
343 "unexpected_read_failure\n",
344 arg.worker_type.c_str(), arg.id, k.c_str());
345 }
346 ret = 1;
347 }
348 if (arg.sh.verbose >= 100 && ret == 0) {
349 fprintf(stderr, "%s -> %s %s %s %s %s\n", arg.worker_type.c_str(),
350 k.c_str(), rrec[0].c_str(), rrec[1].c_str(), rrec[2].c_str(),
351 rrec[3].c_str());
352 }
353 if (ret == 0) {
354 ++stat.success_count;
355 }
356 return ret;
357}
358
359struct hs_longrun_thread_hs : public hs_longrun_thread_base {
360 hs_longrun_thread_hs(const arg_type& arg)
361 : hs_longrun_thread_base(arg) { }
362 void run();
363 int check_hs_error(const char *mess, record_value *rec);
364 int op_insert(record_value& rec);
365 int op_delete(record_value& rec);
366 int op_update(record_value& rec);
367 int op_read(record_value& rec);
368 int op_readnolock(int k);
369 hstcpcli_ptr cli;
370 socket_args sockargs;
371};
372
373struct lock_guard : noncopyable {
374 lock_guard(mutex& mtx) : mtx(mtx) {
375 mtx.lock();
376 }
377 ~lock_guard() {
378 mtx.unlock();
379 }
380 mutex& mtx;
381};
382
383string_ref
384to_string_ref(const std::string& s)
385{
386 return string_ref(s.data(), s.size());
387}
388
389std::string
390to_string(const string_ref& s)
391{
392 return std::string(s.begin(), s.size());
393}
394
395void
396hs_longrun_thread_hs::run()
397{
398 config c = arg.sh.conf;
399 if (arg.op == 'R' || arg.op == 'N') {
400 c["port"] = to_stdstring(arg.sh.conf.get_int("hsport", 9998));
401 } else {
402 c["port"] = to_stdstring(arg.sh.conf.get_int("hsport_wr", 9999));
403 }
404 sockargs.set(c);
405
406 while (arg.sh.running) {
407 if (cli.get() == 0 || !cli->stable_point()) {
408 cli = hstcpcli_i::create(sockargs);
409 if (check_hs_error("connect", 0) != 0) {
410 cli.reset();
411 continue;
412 }
413 cli->request_buf_open_index(0, "hstestdb", "hstesttbl", "PRIMARY",
414 "k,v1,v2,v3", "k,v1,v2,v3");
415 cli->request_send();
416 if (check_hs_error("openindex_send", 0) != 0) {
417 cli.reset();
418 continue;
419 }
420 size_t num_flds = 0;
421 cli->response_recv(num_flds);
422 if (check_hs_error("openindex_recv", 0) != 0) {
423 cli.reset();
424 continue;
425 }
426 cli->response_buf_remove();
427 }
428 const size_t rec_id = rand_record();
429 if (arg.lock_flag) {
430 record_value& rec = *arg.sh.records[rec_id];
431 lock_guard g(rec.lock);
432 int e = 0;
433 switch (arg.op) {
434 case 'I':
435 e = op_insert(rec);
436 break;
437 case 'D':
438 e = op_delete(rec);
439 break;
440 case 'U':
441 e = op_update(rec);
442 break;
443 case 'R':
444 e = op_read(rec);
445 break;
446 default:
447 break;
448 }
449 } else {
450 int e = 0;
451 switch (arg.op) {
452 case 'N':
453 e = op_readnolock(rec_id);
454 break;
455 default:
456 break;
457 }
458 }
459 }
460}
461
462int
463hs_longrun_thread_hs::op_insert(record_value& rec)
464{
465 const std::string k = rec.key;
466 const std::string v1 = "iv1_" + k + "_" + to_stdstring(arg.id);
467 const std::string v2 = "iv2_" + k + "_" + to_stdstring(arg.id);
468 const std::string v3 = "iv3_" + k + "_" + to_stdstring(arg.id);
469 const string_ref op_ref("+", 1);
470 const string_ref op_args[4] = {
471 to_string_ref(k),
472 to_string_ref(v1),
473 to_string_ref(v2),
474 to_string_ref(v3)
475 };
476 cli->request_buf_exec_generic(0, op_ref, op_args, 4, 1, 0,
477 string_ref(), 0, 0, 0, 0);
478 cli->request_send();
479 if (check_hs_error("op_insert_send", &rec) != 0) { return 1; }
480 size_t numflds = 0;
481 cli->response_recv(numflds);
482 if (arg.sh.verbose > 10) {
483 const string_ref *row = cli->get_next_row();
484 fprintf(stderr, "HS op=+ errrcode=%d errmess=[%s]\n", cli->get_error_code(),
485 row ? to_string(row[0]).c_str() : "");
486 }
487 const bool op_success = cli->get_error_code() == 0;
488 int ret = 0;
489 if (!rec.unknown_state) {
490 if (rec.deleted && !op_success) {
491 ++stat.verify_error_count;
492 if (arg.sh.verbose > 0) {
493 fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
494 "unexpected_insert_failure\n",
495 arg.worker_type.c_str(), arg.id, k.c_str());
496 }
497 ret = 1;
498 } else if (!rec.deleted && op_success) {
499 ++stat.verify_error_count;
500 if (arg.sh.verbose > 0) {
501 fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
502 "unexpected_insert_success\n",
503 arg.worker_type.c_str(), arg.id, k.c_str());
504 }
505 ret = 1;
506 }
507 } else {
508 ++stat.unknown_count;
509 }
510 if (op_success) {
511 rec.values.resize(4);
512 rec.values[0] = k;
513 rec.values[1] = v1;
514 rec.values[2] = v2;
515 rec.values[3] = v3;
516 rec.deleted = false;
517 if (arg.sh.verbose >= 100 && ret == 0) {
518 fprintf(stderr, "HS_INSERT %s %s %s %s\n", k.c_str(), v1.c_str(),
519 v2.c_str(), v3.c_str());
520 }
521 if (ret == 0 && !rec.unknown_state) {
522 ++stat.success_count;
523 }
524 rec.unknown_state = false;
525 }
526 cli->response_buf_remove();
527 return ret;
528}
529
530int
531hs_longrun_thread_hs::op_delete(record_value& rec)
532{
533 const std::string k = rec.key;
534 const string_ref op_ref("=", 1);
535 const string_ref op_args[1] = {
536 to_string_ref(k),
537 };
538 const string_ref modop_ref("D", 1);
539 cli->request_buf_exec_generic(0, op_ref, op_args, 1, 1, 0,
540 modop_ref, 0, 0, 0, 0);
541 cli->request_send();
542 if (check_hs_error("op_delete_send", &rec) != 0) { return 1; }
543 size_t numflds = 0;
544 cli->response_recv(numflds);
545 if (check_hs_error("op_delete_recv", &rec) != 0) { return 1; }
546 const string_ref *row = cli->get_next_row();
547 const bool op_success = (numflds > 0 && row != 0 &&
548 to_string(row[0]) == "1");
549 int ret = 0;
550 if (!rec.unknown_state) {
551 if (!rec.deleted && !op_success) {
552 ++stat.verify_error_count;
553 if (arg.sh.verbose > 0) {
554 fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
555 "unexpected_delete_failure\n",
556 arg.worker_type.c_str(), arg.id, k.c_str());
557 }
558 ret = 1;
559 } else if (rec.deleted && op_success) {
560 ++stat.verify_error_count;
561 if (arg.sh.verbose > 0) {
562 fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
563 "unexpected_delete_success\n",
564 arg.worker_type.c_str(), arg.id, k.c_str());
565 }
566 ret = 1;
567 }
568 }
569 cli->response_buf_remove();
570 if (op_success) {
571 rec.deleted = true;
572 if (ret == 0 && !rec.unknown_state) {
573 ++stat.success_count;
574 }
575 rec.unknown_state = false;
576 }
577 if (arg.sh.verbose >= 100 && ret == 0) {
578 fprintf(stderr, "HS_DELETE %s\n", k.c_str());
579 }
580 return ret;
581}
582
583int
584hs_longrun_thread_hs::op_update(record_value& rec)
585{
586 const std::string k = rec.key;
587 const std::string v1 = "uv1_" + k + "_" + to_stdstring(arg.id);
588 const std::string v2 = "uv2_" + k + "_" + to_stdstring(arg.id);
589 const std::string v3 = "uv3_" + k + "_" + to_stdstring(arg.id);
590 const string_ref op_ref("=", 1);
591 const string_ref op_args[1] = {
592 to_string_ref(k),
593 };
594 const string_ref modop_ref("U", 1);
595 const string_ref modop_args[4] = {
596 to_string_ref(k),
597 to_string_ref(v1),
598 to_string_ref(v2),
599 to_string_ref(v3)
600 };
601 cli->request_buf_exec_generic(0, op_ref, op_args, 1, 1, 0,
602 modop_ref, modop_args, 4, 0, 0);
603 cli->request_send();
604 if (check_hs_error("op_update_send", &rec) != 0) { return 1; }
605 size_t numflds = 0;
606 cli->response_recv(numflds);
607 if (check_hs_error("op_update_recv", &rec) != 0) { return 1; }
608 const string_ref *row = cli->get_next_row();
609 uint32_t num_rows = row
610 ? atoi_uint32_nocheck(row[0].begin(), row[0].end()) : 0;
611 cli->response_buf_remove();
612 const bool cur_unknown_state = (num_rows == 1);
613 return verify_update(k, v1, v2, v3, rec, num_rows, cur_unknown_state);
614}
615
616int
617hs_longrun_thread_hs::op_read(record_value& rec)
618{
619 const std::string k = rec.key;
620 const string_ref op_ref("=", 1);
621 const string_ref op_args[1] = {
622 to_string_ref(k),
623 };
624 cli->request_buf_exec_generic(0, op_ref, op_args, 1, 1, 0,
625 string_ref(), 0, 0, 0, 0);
626 cli->request_send();
627 if (check_hs_error("op_read_send", 0) != 0) { return 1; }
628 size_t num_flds = 0;
629 size_t num_rows = 0;
630 cli->response_recv(num_flds);
631 if (check_hs_error("op_read_recv", 0) != 0) { return 1; }
632 const string_ref *row = cli->get_next_row();
633 std::string rrec[4];
634 if (row != 0 && num_flds == 4) {
635 for (int i = 0; i < 4; ++i) {
636 rrec[i] = to_string(row[i]);
637 }
638 ++num_rows;
639 }
640 row = cli->get_next_row();
641 if (row != 0) {
642 ++num_rows;
643 }
644 cli->response_buf_remove();
645 return verify_read(k, num_rows, num_flds, rrec, rec);
646}
647
648int
649hs_longrun_thread_hs::op_readnolock(int key)
650{
651 const std::string k = to_stdstring(key);
652 const string_ref op_ref("=", 1);
653 const string_ref op_args[1] = {
654 to_string_ref(k),
655 };
656 cli->request_buf_exec_generic(0, op_ref, op_args, 1, 1, 0,
657 string_ref(), 0, 0, 0, 0);
658 cli->request_send();
659 if (check_hs_error("op_read_send", 0) != 0) { return 1; }
660 size_t num_flds = 0;
661 size_t num_rows = 0;
662 cli->response_recv(num_flds);
663 if (check_hs_error("op_read_recv", 0) != 0) { return 1; }
664 const string_ref *row = cli->get_next_row();
665 std::string rrec[4];
666 if (row != 0 && num_flds == 4) {
667 for (int i = 0; i < 4; ++i) {
668 rrec[i] = to_string(row[i]);
669 }
670 ++num_rows;
671 }
672 row = cli->get_next_row();
673 if (row != 0) {
674 ++num_rows;
675 }
676 cli->response_buf_remove();
677 return verify_readnolock(k, num_rows, num_flds, rrec);
678}
679
680int
681hs_longrun_thread_hs::check_hs_error(const char *mess, record_value *rec)
682{
683 const int err = cli->get_error_code();
684 if (err == 0) {
685 return 0;
686 }
687 ++stat.runtime_error_count;
688 if (arg.sh.verbose > 0) {
689 const std::string estr = cli->get_error();
690 fprintf(stderr, "RUNTIME_ERROR: op=%c wid=%d %s: %d %s\n",
691 arg.op, arg.id, mess, err, estr.c_str());
692 }
693 if (rec) {
694 rec->unknown_state = true;
695 }
696 return 1;
697}
698
699struct hs_longrun_thread_my : public hs_longrun_thread_base {
700 hs_longrun_thread_my(const arg_type& arg)
701 : hs_longrun_thread_base(arg), connected(false) { }
702 void run();
703 void show_mysql_error(const char *mess, record_value *rec);
704 int op_insert(record_value& rec);
705 int op_delete(record_value& rec);
706 int op_update(record_value& rec);
707 int op_delins(record_value& rec);
708 int op_read(record_value& rec);
709 auto_mysql db;
710 bool connected;
711};
712
713void
714hs_longrun_thread_my::run()
715{
716 const std::string mysql_host = arg.sh.conf.get_str("host", "localhost");
717 const std::string mysql_user = arg.sh.conf.get_str("mysqluser", "root");
718 const std::string mysql_passwd = arg.sh.conf.get_str("mysqlpass", "");
719 const std::string mysql_dbname = "hstestdb";
720
721 while (arg.sh.running) {
722 if (!connected) {
723 if (!mysql_real_connect(db, mysql_host.c_str(), mysql_user.c_str(),
724 mysql_passwd.c_str(), mysql_dbname.c_str(), mysql_port, 0, 0)) {
725 show_mysql_error("mysql_real_connect", 0);
726 continue;
727 }
728 }
729 connected = true;
730 const size_t rec_id = rand_record();
731 record_value& rec = *arg.sh.records[rec_id];
732 lock_guard g(rec.lock);
733 int e = 0;
734 switch (arg.op) {
735 #if 0
736 case 'I':
737 e = op_insert(rec);
738 break;
739 case 'D':
740 e = op_delete(rec);
741 break;
742 case 'U':
743 e = op_update(rec);
744 break;
745 #endif
746 case 'T':
747 e = op_delins(rec);
748 break;
749 case 'R':
750 e = op_read(rec);
751 break;
752 default:
753 break;
754 }
755 }
756}
757
758int
759hs_longrun_thread_my::op_delins(record_value& rec)
760{
761 const std::string k = rec.key;
762 const std::string v1 = "div1_" + k + "_" + to_stdstring(arg.id);
763 const std::string v2 = "div2_" + k + "_" + to_stdstring(arg.id);
764 const std::string v3 = "div3_" + k + "_" + to_stdstring(arg.id);
765 int success = 0;
766 bool cur_unknown_state = false;
767 do {
768 char query[1024];
769 #if 1
770 if (mysql_query(db, "begin") != 0) {
771 if (arg.sh.verbose >= 20) {
772 fprintf(stderr, "mysql: e=[%s] q=[%s]\n", mysql_error(db), "begin");
773 }
774 break;
775 }
776 #endif
777 cur_unknown_state = true;
778 snprintf(query, 1024,
779 "delete from hstesttbl where k = '%s'", k.c_str());
780 if (mysql_query(db, query) != 0) {
781 if (arg.sh.verbose >= 20) {
782 fprintf(stderr, "mysql: e=[%s] q=[%s]\n", mysql_error(db), query);
783 }
784 break;
785 }
786 if (mysql_affected_rows(db) != 1) {
787 if (arg.sh.verbose >= 20) {
788 fprintf(stderr, "mysql: notfound: [%s]\n", query);
789 }
790 break;
791 }
792 snprintf(query, 1024,
793 "insert into hstesttbl values ('%s', '%s', '%s', '%s')",
794 k.c_str(), v1.c_str(), v2.c_str(), v3.c_str());
795 if (mysql_query(db, query) != 0) {
796 if (arg.sh.verbose >= 20) {
797 fprintf(stderr, "mysql: e=[%s] q=[%s]\n", mysql_error(db), query);
798 }
799 break;
800 }
801 #if 1
802 if (mysql_query(db, "commit") != 0) {
803 if (arg.sh.verbose >= 20) {
804 fprintf(stderr, "mysql: e=[%s] q=[%s]\n", mysql_error(db), "commit");
805 }
806 break;
807 }
808 #endif
809 success = true;
810 cur_unknown_state = false;
811 } while (false);
812 return verify_update(k, v1, v2, v3, rec, (success != 0), cur_unknown_state);
813}
814
815int
816hs_longrun_thread_my::op_read(record_value& rec)
817{
818 const std::string k = rec.key;
819 char query[1024] = { 0 };
820 const int len = snprintf(query, 1024,
821 "select k,v1,v2,v3 from hstesttbl where k='%s'", k.c_str());
822 const int r = mysql_real_query(db, query, len > 0 ? len : 0);
823 if (r != 0) {
824 show_mysql_error(query, 0);
825 return 1;
826 }
827 MYSQL_ROW row = 0;
828 unsigned long *lengths = 0;
829 unsigned int num_rows = 0;
830 unsigned int num_flds = 0;
831 auto_mysql_res res(db);
832 std::string rrec[4];
833 if (res != 0) {
834 num_flds = mysql_num_fields(res);
835 row = mysql_fetch_row(res);
836 if (row != 0) {
837 lengths = mysql_fetch_lengths(res);
838 if (num_flds == 4) {
839 for (int i = 0; i < 4; ++i) {
840 rrec[i] = std::string(row[i], lengths[i]);
841 }
842 }
843 ++num_rows;
844 row = mysql_fetch_row(res);
845 if (row != 0) {
846 ++num_rows;
847 }
848 }
849 }
850 return verify_read(k, num_rows, num_flds, rrec, rec);
851}
852
853void
854hs_longrun_thread_my::show_mysql_error(const char *mess, record_value *rec)
855{
856 ++stat.runtime_error_count;
857 if (arg.sh.verbose > 0) {
858 fprintf(stderr, "RUNTIME_ERROR: op=%c wid=%d [%s]: %s\n",
859 arg.op, arg.id, mess, mysql_error(db));
860 }
861 if (rec) {
862 rec->unknown_state = true;
863 }
864 db.reset();
865 connected = false;
866}
867
868void
869mysql_do(MYSQL *db, const char *query)
870{
871 if (mysql_real_query(db, query, strlen(query)) != 0) {
872 fprintf(stderr, "mysql: e=[%s] q=[%s]\n", mysql_error(db), query);
873 fatal_exit("mysql_do");
874 }
875}
876
877void
878hs_longrun_init_table(const config& conf, int num_prepare,
879 hs_longrun_shared& shared)
880{
881 const std::string mysql_host = conf.get_str("host", "localhost");
882 const std::string mysql_user = conf.get_str("mysqluser", "root");
883 const std::string mysql_passwd = conf.get_str("mysqlpass", "");
884 const std::string mysql_dbname = "";
885 auto_mysql db;
886 if (!mysql_real_connect(db, mysql_host.c_str(), mysql_user.c_str(),
887 mysql_passwd.c_str(), mysql_dbname.c_str(), mysql_port, 0, 0)) {
888 fprintf(stderr, "mysql: error=[%s]\n", mysql_error(db));
889 fatal_exit("hs_longrun_init_table");
890 }
891 mysql_do(db, "drop database if exists hstestdb");
892 mysql_do(db, "create database hstestdb");
893 mysql_do(db, "use hstestdb");
894 mysql_do(db,
895 "create table hstesttbl ("
896 "k int primary key,"
897 "v1 varchar(32) not null,"
898 "v2 varchar(32) not null,"
899 "v3 varchar(32) not null"
900 ") character set utf8 collate utf8_bin engine = innodb");
901 for (int i = 0; i < num_prepare; ++i) {
902 const std::string i_str = to_stdstring(i);
903 const std::string v1 = "pv1_" + i_str;
904 const std::string v2 = "pv2_" + i_str;
905 const std::string v3 = "pv3_" + i_str;
906 char buf[1024];
907 snprintf(buf, 1024, "insert into hstesttbl(k, v1, v2, v3) values"
908 "(%d, '%s', '%s', '%s')", i, v1.c_str(), v2.c_str(), v3.c_str());
909 mysql_do(db, buf);
910 record_value *rec = shared.records[i];
911 rec->key = i_str;
912 rec->values.resize(4);
913 rec->values[0] = i_str;
914 rec->values[1] = v1;
915 rec->values[2] = v2;
916 rec->values[3] = v3;
917 rec->deleted = false;
918 }
919}
920
921int
922hs_longrun_main(int argc, char **argv)
923{
924 hs_longrun_shared shared;
925 parse_args(argc, argv, shared.conf);
926 shared.conf["host"] = shared.conf.get_str("host", "localhost");
927 shared.verbose = shared.conf.get_int("verbose", 1);
928 const int table_size = shared.conf.get_int("table_size", 10000);
929 for (int i = 0; i < table_size; ++i) {
930 std::auto_ptr<record_value> rec(new record_value());
931 rec->key = to_stdstring(i);
932 shared.records.push_back_ptr(rec);
933 }
934 mysql_library_init(0, 0, 0);
935 const int duration = shared.conf.get_int("duration", 10);
936 const int num_hsinsert = shared.conf.get_int("num_hsinsert", 10);
937 const int num_hsdelete = shared.conf.get_int("num_hsdelete", 10);
938 const int num_hsupdate = shared.conf.get_int("num_hsupdate", 10);
939 const int num_hsread = shared.conf.get_int("num_hsread", 10);
940 const int num_myread = shared.conf.get_int("num_myread", 10);
941 const int num_mydelins = shared.conf.get_int("num_mydelins", 10);
942 int num_hsreadnolock = shared.conf.get_int("num_hsreadnolock", 10);
943 const bool always_filled = (num_hsinsert == 0 && num_hsdelete == 0);
944 if (!always_filled) {
945 num_hsreadnolock = 0;
946 }
947 hs_longrun_init_table(shared.conf, always_filled ? table_size : 0,
948 shared);
949 /* create worker threads */
950 static const struct thrtmpl_type {
951 const char *type; char op; int num; int hs; int lock;
952 } thrtmpl[] = {
953 { "hsinsert", 'I', num_hsinsert, 1, 1 },
954 { "hsdelete", 'D', num_hsdelete, 1, 1 },
955 { "hsupdate", 'U', num_hsupdate, 1, 1 },
956 { "hsread", 'R', num_hsread, 1, 1 },
957 { "hsreadnolock", 'N', num_hsreadnolock, 1, 0 },
958 { "myread", 'R', num_myread, 0, 1 },
959 { "mydelins", 'T', num_mydelins, 0, 1 },
960 };
961 typedef auto_ptrcontainer< std::vector<hs_longrun_thread_base *> > thrs_type;
962 thrs_type thrs;
963 for (size_t i = 0; i < sizeof(thrtmpl)/sizeof(thrtmpl[0]); ++i) {
964 const thrtmpl_type& e = thrtmpl[i];
965 for (int j = 0; j < e.num; ++j) {
966 int id = thrs.size();
967 const hs_longrun_thread_hs::arg_type arg(id, e.type, e.op, e.lock,
968 shared);
969 std::auto_ptr<hs_longrun_thread_base> thr;
970 if (e.hs) {
971 thr.reset(new hs_longrun_thread_hs(arg));
972 } else {
973 thr.reset(new hs_longrun_thread_my(arg));
974 }
975 thrs.push_back_ptr(thr);
976 }
977 }
978 shared.num_threads = thrs.size();
979 /* start threads */
980 fprintf(stderr, "START\n");
981 shared.running = 1;
982 for (size_t i = 0; i < thrs.size(); ++i) {
983 thrs[i]->start();
984 }
985 /* wait */
986 sleep(duration);
987 /* stop thread */
988 shared.running = 0;
989 for (size_t i = 0; i < thrs.size(); ++i) {
990 thrs[i]->join();
991 }
992 fprintf(stderr, "DONE\n");
993 /* summary */
994 typedef std::map<std::string, hs_longrun_stat> stat_map;
995 stat_map sm;
996 for (size_t i = 0; i < thrs.size(); ++i) {
997 hs_longrun_thread_base *const thr = thrs[i];
998 const std::string wt = thr->arg.worker_type;
999 hs_longrun_stat& v = sm[wt];
1000 v.add(thr->stat);
1001 }
1002 hs_longrun_stat total;
1003 for (stat_map::const_iterator i = sm.begin(); i != sm.end(); ++i) {
1004 if (i->second.verify_error_count != 0) {
1005 fprintf(stderr, "%s verify_error %llu\n", i->first.c_str(),
1006 i->second.verify_error_count);
1007 }
1008 if (i->second.runtime_error_count) {
1009 fprintf(stderr, "%s runtime_error %llu\n", i->first.c_str(),
1010 i->second.runtime_error_count);
1011 }
1012 if (i->second.unknown_count) {
1013 fprintf(stderr, "%s unknown %llu\n", i->first.c_str(),
1014 i->second.unknown_count);
1015 }
1016 fprintf(stderr, "%s success %llu\n", i->first.c_str(),
1017 i->second.success_count);
1018 total.add(i->second);
1019 }
1020 if (total.verify_error_count != 0) {
1021 fprintf(stderr, "TOTAL verify_error %llu\n", total.verify_error_count);
1022 }
1023 if (total.runtime_error_count != 0) {
1024 fprintf(stderr, "TOTAL runtime_error %llu\n", total.runtime_error_count);
1025 }
1026 if (total.unknown_count != 0) {
1027 fprintf(stderr, "TOTAL unknown %llu\n", total.unknown_count);
1028 }
1029 fprintf(stderr, "TOTAL success %llu\n", total.success_count);
1030 mysql_library_end();
1031 return 0;
1032}
1033
1034};
1035
1036int
1037main(int argc, char **argv)
1038{
1039 return dena::hs_longrun_main(argc, argv);
1040}
1041
01042
=== removed file 'HandlerSocket-Plugin-for-MySQL/client/hslongrun.cpp'
--- HandlerSocket-Plugin-for-MySQL/client/hslongrun.cpp 2011-04-19 12:57:00 +0000
+++ HandlerSocket-Plugin-for-MySQL/client/hslongrun.cpp 1970-01-01 00:00:00 +0000
@@ -1,1041 +0,0 @@
1
2// vim:sw=2:ai
3
4#include <signal.h>
5#include <sys/time.h>
6#include <stdio.h>
7#include <string.h>
8#include <vector>
9#include <map>
10#include <stdlib.h>
11#include <memory>
12#include <errno.h>
13#include <mysql.h>
14#include <time.h>
15#include <sys/types.h>
16#include <sys/stat.h>
17#include <fcntl.h>
18
19#include "util.hpp"
20#include "auto_ptrcontainer.hpp"
21#include "socket.hpp"
22#include "hstcpcli.hpp"
23#include "string_util.hpp"
24#include "mutex.hpp"
25
26namespace dena {
27
28struct auto_mysql : private noncopyable {
29 auto_mysql() : db(0) {
30 reset();
31 }
32 ~auto_mysql() {
33 if (db) {
34 mysql_close(db);
35 }
36 }
37 void reset() {
38 if (db) {
39 mysql_close(db);
40 }
41 if ((db = mysql_init(0)) == 0) {
42 fatal_exit("failed to initialize mysql client");
43 }
44 }
45 operator MYSQL *() const { return db; }
46 private:
47 MYSQL *db;
48};
49
50struct auto_mysql_res : private noncopyable {
51 auto_mysql_res(MYSQL *db) {
52 res = mysql_store_result(db);
53 }
54 ~auto_mysql_res() {
55 if (res) {
56 mysql_free_result(res);
57 }
58 }
59 operator MYSQL_RES *() const { return res; }
60 private:
61 MYSQL_RES *res;
62};
63
64struct auto_mysql_stmt : private noncopyable {
65 auto_mysql_stmt(MYSQL *db) {
66 stmt = mysql_stmt_init(db);
67 }
68 ~auto_mysql_stmt() {
69 if (stmt) {
70 mysql_stmt_close(stmt);
71 }
72 }
73 operator MYSQL_STMT *() const { return stmt; }
74 private:
75 MYSQL_STMT *stmt;
76};
77
78double
79gettimeofday_double()
80{
81 struct timeval tv = { };
82 if (gettimeofday(&tv, 0) != 0) {
83 fatal_abort("gettimeofday");
84 }
85 return static_cast<double>(tv.tv_usec) / 1000000 + tv.tv_sec;
86}
87
88struct record_value {
89 mutex lock;
90 bool deleted;
91 bool unknown_state;
92 std::string key;
93 std::vector<std::string> values;
94 record_value() : deleted(true), unknown_state(false) { }
95};
96
97struct hs_longrun_shared {
98 config conf;
99 socket_args arg;
100 int verbose;
101 long num_threads;
102 int usleep;
103 volatile mutable int running;
104 auto_ptrcontainer< std::vector<record_value *> > records;
105 hs_longrun_shared() : verbose(0), num_threads(0), usleep(0), running(1) { }
106};
107
108struct thread_base {
109 thread_base() : need_join(false), stack_size(256 * 1024) { }
110 virtual ~thread_base() {
111 join();
112 }
113 virtual void run() = 0;
114 void start() {
115 if (!start_nothrow()) {
116 fatal_abort("thread::start");
117 }
118 }
119 bool start_nothrow() {
120 if (need_join) {
121 return need_join; /* true */
122 }
123 void *const arg = this;
124 pthread_attr_t attr;
125 if (pthread_attr_init(&attr) != 0) {
126 fatal_abort("pthread_attr_init");
127 }
128 if (pthread_attr_setstacksize(&attr, stack_size) != 0) {
129 fatal_abort("pthread_attr_setstacksize");
130 }
131 const int r = pthread_create(&thr, &attr, thread_main, arg);
132 if (pthread_attr_destroy(&attr) != 0) {
133 fatal_abort("pthread_attr_destroy");
134 }
135 if (r != 0) {
136 return need_join; /* false */
137 }
138 need_join = true;
139 return need_join; /* true */
140 }
141 void join() {
142 if (!need_join) {
143 return;
144 }
145 int e = 0;
146 if ((e = pthread_join(thr, 0)) != 0) {
147 fatal_abort("pthread_join");
148 }
149 need_join = false;
150 }
151 private:
152 static void *thread_main(void *arg) {
153 thread_base *p = static_cast<thread_base *>(arg);
154 p->run();
155 return 0;
156 }
157 private:
158 pthread_t thr;
159 bool need_join;
160 size_t stack_size;
161};
162
163struct hs_longrun_stat {
164 unsigned long long verify_error_count;
165 unsigned long long runtime_error_count;
166 unsigned long long unknown_count;
167 unsigned long long success_count;
168 hs_longrun_stat()
169 : verify_error_count(0), runtime_error_count(0),
170 unknown_count(0), success_count(0) { }
171 void add(const hs_longrun_stat& x) {
172 verify_error_count += x.verify_error_count;
173 runtime_error_count += x.runtime_error_count;
174 unknown_count += x.unknown_count;
175 success_count += x.success_count;
176 }
177};
178
179struct hs_longrun_thread_base : public thread_base {
180 struct arg_type {
181 int id;
182 std::string worker_type;
183 char op;
184 int lock_flag;
185 const hs_longrun_shared& sh;
186 arg_type(int id, const std::string& worker_type, char op, int lock_flag,
187 const hs_longrun_shared& sh)
188 : id(id), worker_type(worker_type), op(op), lock_flag(lock_flag),
189 sh(sh) { }
190 };
191 arg_type arg;
192 hs_longrun_stat stat;
193 drand48_data randbuf;
194 unsigned int seed;
195 hs_longrun_thread_base(const arg_type& arg)
196 : arg(arg), seed(0) {
197 seed = time(0) + arg.id + 1;
198 srand48_r(seed, &randbuf);
199 }
200 virtual ~hs_longrun_thread_base() { }
201 virtual void run() = 0;
202 size_t rand_record() {
203 double v = 0;
204 drand48_r(&randbuf, &v);
205 const size_t sz = arg.sh.records.size();
206 size_t r = size_t(v * sz);
207 if (r >= sz) {
208 r = 0;
209 }
210 return r;
211 }
212 int verify_update(const std::string& k, const std::string& v1,
213 const std::string& v2, const std::string& v3, record_value& rec,
214 uint32_t num_rows, bool cur_unknown_state);
215 int verify_read(const std::string& k, uint32_t num_rows, uint32_t num_flds,
216 const std::string rrec[4], record_value& rec);
217 int verify_readnolock(const std::string& k, uint32_t num_rows,
218 uint32_t num_flds, const std::string rrec[4]);
219};
220
221int
222hs_longrun_thread_base::verify_update(const std::string& k,
223 const std::string& v1, const std::string& v2, const std::string& v3,
224 record_value& rec, uint32_t num_rows, bool cur_unknown_state)
225{
226 const bool op_success = num_rows == 1;
227 int ret = 0;
228 if (!rec.unknown_state) {
229 if (!rec.deleted && !op_success) {
230 ++stat.verify_error_count;
231 if (arg.sh.verbose > 0) {
232 fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
233 "unexpected_update_failure\n",
234 arg.worker_type.c_str(), arg.id, k.c_str());
235 }
236 ret = 1;
237 } else if (rec.deleted && op_success) {
238 ++stat.verify_error_count;
239 if (arg.sh.verbose > 0) {
240 fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
241 "unexpected_update_success\n",
242 arg.worker_type.c_str(), arg.id, k.c_str());
243 }
244 ret = 1;
245 }
246 }
247 if (op_success) {
248 rec.values.resize(4);
249 rec.values[0] = k;
250 rec.values[1] = v1;
251 rec.values[2] = v2;
252 rec.values[3] = v3;
253 if (ret == 0 && !rec.unknown_state) {
254 ++stat.success_count;
255 }
256 }
257 rec.unknown_state = cur_unknown_state;
258 if (arg.sh.verbose >= 100 && ret == 0) {
259 fprintf(stderr, "%s %s %s %s %s\n", arg.worker_type.c_str(),
260 k.c_str(), v1.c_str(), v2.c_str(), v3.c_str());
261 }
262 return ret;
263}
264
265int
266hs_longrun_thread_base::verify_read(const std::string& k,
267 uint32_t num_rows, uint32_t num_flds, const std::string rrec[4],
268 record_value& rec)
269{
270 const bool op_success = num_rows != 0;
271 int ret = 0;
272 if (!rec.unknown_state) {
273 if (!rec.deleted && !op_success) {
274 ++stat.verify_error_count;
275 if (arg.sh.verbose > 0) {
276 fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
277 "unexpected_read_failure\n",
278 arg.worker_type.c_str(), arg.id, k.c_str());
279 }
280 ret = 1;
281 } else if (rec.deleted && op_success) {
282 ++stat.verify_error_count;
283 if (arg.sh.verbose > 0) {
284 fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
285 "unexpected_read_success\n",
286 arg.worker_type.c_str(), arg.id, k.c_str());
287 }
288 ret = 1;
289 } else if (num_flds != 4) {
290 ++stat.verify_error_count;
291 if (arg.sh.verbose > 0) {
292 fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
293 "unexpected_read_fldnum %d\n",
294 arg.worker_type.c_str(), arg.id, k.c_str(),
295 static_cast<int>(num_flds));
296 }
297 ret = 1;
298 } else if (rec.deleted) {
299 /* nothing to verify */
300 } else {
301 int diff = 0;
302 for (size_t i = 0; i < 4; ++i) {
303 if (rec.values[i] == rrec[i]) {
304 /* ok */
305 } else {
306 diff = 1;
307 }
308 }
309 if (diff) {
310 std::string mess;
311 for (size_t i = 0; i < 4; ++i) {
312 const std::string& expected = rec.values[i];
313 const std::string& val = rrec[i];
314 mess += " " + val + "/" + expected;
315 }
316 if (arg.sh.verbose > 0) {
317 fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
318 "unexpected_read_value %s\n",
319 arg.worker_type.c_str(), arg.id, k.c_str(), mess.c_str());
320 }
321 ret = 1;
322 }
323 }
324 }
325 if (arg.sh.verbose >= 100 && ret == 0) {
326 fprintf(stderr, "%s %s\n", arg.worker_type.c_str(), k.c_str());
327 }
328 if (ret == 0 && !rec.unknown_state) {
329 ++stat.success_count;
330 }
331 return ret;
332}
333
334int
335hs_longrun_thread_base::verify_readnolock(const std::string& k,
336 uint32_t num_rows, uint32_t num_flds, const std::string rrec[4])
337{
338 int ret = 0;
339 if (num_rows != 1 || num_flds != 4) {
340 ++stat.verify_error_count;
341 if (arg.sh.verbose > 0) {
342 fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
343 "unexpected_read_failure\n",
344 arg.worker_type.c_str(), arg.id, k.c_str());
345 }
346 ret = 1;
347 }
348 if (arg.sh.verbose >= 100 && ret == 0) {
349 fprintf(stderr, "%s -> %s %s %s %s %s\n", arg.worker_type.c_str(),
350 k.c_str(), rrec[0].c_str(), rrec[1].c_str(), rrec[2].c_str(),
351 rrec[3].c_str());
352 }
353 if (ret == 0) {
354 ++stat.success_count;
355 }
356 return ret;
357}
358
359struct hs_longrun_thread_hs : public hs_longrun_thread_base {
360 hs_longrun_thread_hs(const arg_type& arg)
361 : hs_longrun_thread_base(arg) { }
362 void run();
363 int check_hs_error(const char *mess, record_value *rec);
364 int op_insert(record_value& rec);
365 int op_delete(record_value& rec);
366 int op_update(record_value& rec);
367 int op_read(record_value& rec);
368 int op_readnolock(int k);
369 hstcpcli_ptr cli;
370 socket_args sockargs;
371};
372
373struct lock_guard : noncopyable {
374 lock_guard(mutex& mtx) : mtx(mtx) {
375 mtx.lock();
376 }
377 ~lock_guard() {
378 mtx.unlock();
379 }
380 mutex& mtx;
381};
382
383string_ref
384to_string_ref(const std::string& s)
385{
386 return string_ref(s.data(), s.size());
387}
388
389std::string
390to_string(const string_ref& s)
391{
392 return std::string(s.begin(), s.size());
393}
394
395void
396hs_longrun_thread_hs::run()
397{
398 config c = arg.sh.conf;
399 if (arg.op == 'R' || arg.op == 'N') {
400 c["port"] = to_stdstring(arg.sh.conf.get_int("hsport", 9998));
401 } else {
402 c["port"] = to_stdstring(arg.sh.conf.get_int("hsport_wr", 9999));
403 }
404 sockargs.set(c);
405
406 while (arg.sh.running) {
407 if (cli.get() == 0 || !cli->stable_point()) {
408 cli = hstcpcli_i::create(sockargs);
409 if (check_hs_error("connect", 0) != 0) {
410 cli.reset();
411 continue;
412 }
413 cli->request_buf_open_index(0, "hstestdb", "hstesttbl", "PRIMARY",
414 "k,v1,v2,v3", "k,v1,v2,v3");
415 cli->request_send();
416 if (check_hs_error("openindex_send", 0) != 0) {
417 cli.reset();
418 continue;
419 }
420 size_t num_flds = 0;
421 cli->response_recv(num_flds);
422 if (check_hs_error("openindex_recv", 0) != 0) {
423 cli.reset();
424 continue;
425 }
426 cli->response_buf_remove();
427 }
428 const size_t rec_id = rand_record();
429 if (arg.lock_flag) {
430 record_value& rec = *arg.sh.records[rec_id];
431 lock_guard g(rec.lock);
432 int e = 0;
433 switch (arg.op) {
434 case 'I':
435 e = op_insert(rec);
436 break;
437 case 'D':
438 e = op_delete(rec);
439 break;
440 case 'U':
441 e = op_update(rec);
442 break;
443 case 'R':
444 e = op_read(rec);
445 break;
446 default:
447 break;
448 }
449 } else {
450 int e = 0;
451 switch (arg.op) {
452 case 'N':
453 e = op_readnolock(rec_id);
454 break;
455 default:
456 break;
457 }
458 }
459 }
460}
461
462int
463hs_longrun_thread_hs::op_insert(record_value& rec)
464{
465 const std::string k = rec.key;
466 const std::string v1 = "iv1_" + k + "_" + to_stdstring(arg.id);
467 const std::string v2 = "iv2_" + k + "_" + to_stdstring(arg.id);
468 const std::string v3 = "iv3_" + k + "_" + to_stdstring(arg.id);
469 const string_ref op_ref("+", 1);
470 const string_ref op_args[4] = {
471 to_string_ref(k),
472 to_string_ref(v1),
473 to_string_ref(v2),
474 to_string_ref(v3)
475 };
476 cli->request_buf_exec_generic(0, op_ref, op_args, 4, 1, 0,
477 string_ref(), 0, 0, 0, 0);
478 cli->request_send();
479 if (check_hs_error("op_insert_send", &rec) != 0) { return 1; }
480 size_t numflds = 0;
481 cli->response_recv(numflds);
482 if (arg.sh.verbose > 10) {
483 const string_ref *row = cli->get_next_row();
484 fprintf(stderr, "HS op=+ errrcode=%d errmess=[%s]\n", cli->get_error_code(),
485 row ? to_string(row[0]).c_str() : "");
486 }
487 const bool op_success = cli->get_error_code() == 0;
488 int ret = 0;
489 if (!rec.unknown_state) {
490 if (rec.deleted && !op_success) {
491 ++stat.verify_error_count;
492 if (arg.sh.verbose > 0) {
493 fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
494 "unexpected_insert_failure\n",
495 arg.worker_type.c_str(), arg.id, k.c_str());
496 }
497 ret = 1;
498 } else if (!rec.deleted && op_success) {
499 ++stat.verify_error_count;
500 if (arg.sh.verbose > 0) {
501 fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
502 "unexpected_insert_success\n",
503 arg.worker_type.c_str(), arg.id, k.c_str());
504 }
505 ret = 1;
506 }
507 } else {
508 ++stat.unknown_count;
509 }
510 if (op_success) {
511 rec.values.resize(4);
512 rec.values[0] = k;
513 rec.values[1] = v1;
514 rec.values[2] = v2;
515 rec.values[3] = v3;
516 rec.deleted = false;
517 if (arg.sh.verbose >= 100 && ret == 0) {
518 fprintf(stderr, "HS_INSERT %s %s %s %s\n", k.c_str(), v1.c_str(),
519 v2.c_str(), v3.c_str());
520 }
521 if (ret == 0 && !rec.unknown_state) {
522 ++stat.success_count;
523 }
524 rec.unknown_state = false;
525 }
526 cli->response_buf_remove();
527 return ret;
528}
529
530int
531hs_longrun_thread_hs::op_delete(record_value& rec)
532{
533 const std::string k = rec.key;
534 const string_ref op_ref("=", 1);
535 const string_ref op_args[1] = {
536 to_string_ref(k),
537 };
538 const string_ref modop_ref("D", 1);
539 cli->request_buf_exec_generic(0, op_ref, op_args, 1, 1, 0,
540 modop_ref, 0, 0, 0, 0);
541 cli->request_send();
542 if (check_hs_error("op_delete_send", &rec) != 0) { return 1; }
543 size_t numflds = 0;
544 cli->response_recv(numflds);
545 if (check_hs_error("op_delete_recv", &rec) != 0) { return 1; }
546 const string_ref *row = cli->get_next_row();
547 const bool op_success = (numflds > 0 && row != 0 &&
548 to_string(row[0]) == "1");
549 int ret = 0;
550 if (!rec.unknown_state) {
551 if (!rec.deleted && !op_success) {
552 ++stat.verify_error_count;
553 if (arg.sh.verbose > 0) {
554 fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
555 "unexpected_delete_failure\n",
556 arg.worker_type.c_str(), arg.id, k.c_str());
557 }
558 ret = 1;
559 } else if (rec.deleted && op_success) {
560 ++stat.verify_error_count;
561 if (arg.sh.verbose > 0) {
562 fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
563 "unexpected_delete_success\n",
564 arg.worker_type.c_str(), arg.id, k.c_str());
565 }
566 ret = 1;
567 }
568 }
569 cli->response_buf_remove();
570 if (op_success) {
571 rec.deleted = true;
572 if (ret == 0 && !rec.unknown_state) {
573 ++stat.success_count;
574 }
575 rec.unknown_state = false;
576 }
577 if (arg.sh.verbose >= 100 && ret == 0) {
578 fprintf(stderr, "HS_DELETE %s\n", k.c_str());
579 }
580 return ret;
581}
582
583int
584hs_longrun_thread_hs::op_update(record_value& rec)
585{
586 const std::string k = rec.key;
587 const std::string v1 = "uv1_" + k + "_" + to_stdstring(arg.id);
588 const std::string v2 = "uv2_" + k + "_" + to_stdstring(arg.id);
589 const std::string v3 = "uv3_" + k + "_" + to_stdstring(arg.id);
590 const string_ref op_ref("=", 1);
591 const string_ref op_args[1] = {
592 to_string_ref(k),
593 };
594 const string_ref modop_ref("U", 1);
595 const string_ref modop_args[4] = {
596 to_string_ref(k),
597 to_string_ref(v1),
598 to_string_ref(v2),
599 to_string_ref(v3)
600 };
601 cli->request_buf_exec_generic(0, op_ref, op_args, 1, 1, 0,
602 modop_ref, modop_args, 4, 0, 0);
603 cli->request_send();
604 if (check_hs_error("op_update_send", &rec) != 0) { return 1; }
605 size_t numflds = 0;
606 cli->response_recv(numflds);
607 if (check_hs_error("op_update_recv", &rec) != 0) { return 1; }
608 const string_ref *row = cli->get_next_row();
609 uint32_t num_rows = row
610 ? atoi_uint32_nocheck(row[0].begin(), row[0].end()) : 0;
611 cli->response_buf_remove();
612 const bool cur_unknown_state = (num_rows == 1);
613 return verify_update(k, v1, v2, v3, rec, num_rows, cur_unknown_state);
614}
615
616int
617hs_longrun_thread_hs::op_read(record_value& rec)
618{
619 const std::string k = rec.key;
620 const string_ref op_ref("=", 1);
621 const string_ref op_args[1] = {
622 to_string_ref(k),
623 };
624 cli->request_buf_exec_generic(0, op_ref, op_args, 1, 1, 0,
625 string_ref(), 0, 0, 0, 0);
626 cli->request_send();
627 if (check_hs_error("op_read_send", 0) != 0) { return 1; }
628 size_t num_flds = 0;
629 size_t num_rows = 0;
630 cli->response_recv(num_flds);
631 if (check_hs_error("op_read_recv", 0) != 0) { return 1; }
632 const string_ref *row = cli->get_next_row();
633 std::string rrec[4];
634 if (row != 0 && num_flds == 4) {
635 for (int i = 0; i < 4; ++i) {
636 rrec[i] = to_string(row[i]);
637 }
638 ++num_rows;
639 }
640 row = cli->get_next_row();
641 if (row != 0) {
642 ++num_rows;
643 }
644 cli->response_buf_remove();
645 return verify_read(k, num_rows, num_flds, rrec, rec);
646}
647
648int
649hs_longrun_thread_hs::op_readnolock(int key)
650{
651 const std::string k = to_stdstring(key);
652 const string_ref op_ref("=", 1);
653 const string_ref op_args[1] = {
654 to_string_ref(k),
655 };
656 cli->request_buf_exec_generic(0, op_ref, op_args, 1, 1, 0,
657 string_ref(), 0, 0, 0, 0);
658 cli->request_send();
659 if (check_hs_error("op_read_send", 0) != 0) { return 1; }
660 size_t num_flds = 0;
661 size_t num_rows = 0;
662 cli->response_recv(num_flds);
663 if (check_hs_error("op_read_recv", 0) != 0) { return 1; }
664 const string_ref *row = cli->get_next_row();
665 std::string rrec[4];
666 if (row != 0 && num_flds == 4) {
667 for (int i = 0; i < 4; ++i) {
668 rrec[i] = to_string(row[i]);
669 }
670 ++num_rows;
671 }
672 row = cli->get_next_row();
673 if (row != 0) {
674 ++num_rows;
675 }
676 cli->response_buf_remove();
677 return verify_readnolock(k, num_rows, num_flds, rrec);
678}
679
680int
681hs_longrun_thread_hs::check_hs_error(const char *mess, record_value *rec)
682{
683 const int err = cli->get_error_code();
684 if (err == 0) {
685 return 0;
686 }
687 ++stat.runtime_error_count;
688 if (arg.sh.verbose > 0) {
689 const std::string estr = cli->get_error();
690 fprintf(stderr, "RUNTIME_ERROR: op=%c wid=%d %s: %d %s\n",
691 arg.op, arg.id, mess, err, estr.c_str());
692 }
693 if (rec) {
694 rec->unknown_state = true;
695 }
696 return 1;
697}
698
699struct hs_longrun_thread_my : public hs_longrun_thread_base {
700 hs_longrun_thread_my(const arg_type& arg)
701 : hs_longrun_thread_base(arg), connected(false) { }
702 void run();
703 void show_mysql_error(const char *mess, record_value *rec);
704 int op_insert(record_value& rec);
705 int op_delete(record_value& rec);
706 int op_update(record_value& rec);
707 int op_delins(record_value& rec);
708 int op_read(record_value& rec);
709 auto_mysql db;
710 bool connected;
711};
712
713void
714hs_longrun_thread_my::run()
715{
716 const std::string mysql_host = arg.sh.conf.get_str("host", "localhost");
717 const std::string mysql_user = arg.sh.conf.get_str("mysqluser", "root");
718 const std::string mysql_passwd = arg.sh.conf.get_str("mysqlpass", "");
719 const std::string mysql_dbname = "hstestdb";
720
721 while (arg.sh.running) {
722 if (!connected) {
723 if (!mysql_real_connect(db, mysql_host.c_str(), mysql_user.c_str(),
724 mysql_passwd.c_str(), mysql_dbname.c_str(), mysql_port, 0, 0)) {
725 show_mysql_error("mysql_real_connect", 0);
726 continue;
727 }
728 }
729 connected = true;
730 const size_t rec_id = rand_record();
731 record_value& rec = *arg.sh.records[rec_id];
732 lock_guard g(rec.lock);
733 int e = 0;
734 switch (arg.op) {
735 #if 0
736 case 'I':
737 e = op_insert(rec);
738 break;
739 case 'D':
740 e = op_delete(rec);
741 break;
742 case 'U':
743 e = op_update(rec);
744 break;
745 #endif
746 case 'T':
747 e = op_delins(rec);
748 break;
749 case 'R':
750 e = op_read(rec);
751 break;
752 default:
753 break;
754 }
755 }
756}
757
758int
759hs_longrun_thread_my::op_delins(record_value& rec)
760{
761 const std::string k = rec.key;
762 const std::string v1 = "div1_" + k + "_" + to_stdstring(arg.id);
763 const std::string v2 = "div2_" + k + "_" + to_stdstring(arg.id);
764 const std::string v3 = "div3_" + k + "_" + to_stdstring(arg.id);
765 int success = 0;
766 bool cur_unknown_state = false;
767 do {
768 char query[1024];
769 #if 1
770 if (mysql_query(db, "begin") != 0) {
771 if (arg.sh.verbose >= 20) {
772 fprintf(stderr, "mysql: e=[%s] q=[%s]\n", mysql_error(db), "begin");
773 }
774 break;
775 }
776 #endif
777 cur_unknown_state = true;
778 snprintf(query, 1024,
779 "delete from hstesttbl where k = '%s'", k.c_str());
780 if (mysql_query(db, query) != 0) {
781 if (arg.sh.verbose >= 20) {
782 fprintf(stderr, "mysql: e=[%s] q=[%s]\n", mysql_error(db), query);
783 }
784 break;
785 }
786 if (mysql_affected_rows(db) != 1) {
787 if (arg.sh.verbose >= 20) {
788 fprintf(stderr, "mysql: notfound: [%s]\n", query);
789 }
790 break;
791 }
792 snprintf(query, 1024,
793 "insert into hstesttbl values ('%s', '%s', '%s', '%s')",
794 k.c_str(), v1.c_str(), v2.c_str(), v3.c_str());
795 if (mysql_query(db, query) != 0) {
796 if (arg.sh.verbose >= 20) {
797 fprintf(stderr, "mysql: e=[%s] q=[%s]\n", mysql_error(db), query);
798 }
799 break;
800 }
801 #if 1
802 if (mysql_query(db, "commit") != 0) {
803 if (arg.sh.verbose >= 20) {
804 fprintf(stderr, "mysql: e=[%s] q=[%s]\n", mysql_error(db), "commit");
805 }
806 break;
807 }
808 #endif
809 success = true;
810 cur_unknown_state = false;
811 } while (false);
812 return verify_update(k, v1, v2, v3, rec, (success != 0), cur_unknown_state);
813}
814
815int
816hs_longrun_thread_my::op_read(record_value& rec)
817{
818 const std::string k = rec.key;
819 char query[1024] = { 0 };
820 const int len = snprintf(query, 1024,
821 "select k,v1,v2,v3 from hstesttbl where k='%s'", k.c_str());
822 const int r = mysql_real_query(db, query, len > 0 ? len : 0);
823 if (r != 0) {
824 show_mysql_error(query, 0);
825 return 1;
826 }
827 MYSQL_ROW row = 0;
828 unsigned long *lengths = 0;
829 unsigned int num_rows = 0;
830 unsigned int num_flds = 0;
831 auto_mysql_res res(db);
832 std::string rrec[4];
833 if (res != 0) {
834 num_flds = mysql_num_fields(res);
835 row = mysql_fetch_row(res);
836 if (row != 0) {
837 lengths = mysql_fetch_lengths(res);
838 if (num_flds == 4) {
839 for (int i = 0; i < 4; ++i) {
840 rrec[i] = std::string(row[i], lengths[i]);
841 }
842 }
843 ++num_rows;
844 row = mysql_fetch_row(res);
845 if (row != 0) {
846 ++num_rows;
847 }
848 }
849 }
850 return verify_read(k, num_rows, num_flds, rrec, rec);
851}
852
853void
854hs_longrun_thread_my::show_mysql_error(const char *mess, record_value *rec)
855{
856 ++stat.runtime_error_count;
857 if (arg.sh.verbose > 0) {
858 fprintf(stderr, "RUNTIME_ERROR: op=%c wid=%d [%s]: %s\n",
859 arg.op, arg.id, mess, mysql_error(db));
860 }
861 if (rec) {
862 rec->unknown_state = true;
863 }
864 db.reset();
865 connected = false;
866}
867
868void
869mysql_do(MYSQL *db, const char *query)
870{
871 if (mysql_real_query(db, query, strlen(query)) != 0) {
872 fprintf(stderr, "mysql: e=[%s] q=[%s]\n", mysql_error(db), query);
873 fatal_exit("mysql_do");
874 }
875}
876
877void
878hs_longrun_init_table(const config& conf, int num_prepare,
879 hs_longrun_shared& shared)
880{
881 const std::string mysql_host = conf.get_str("host", "localhost");
882 const std::string mysql_user = conf.get_str("mysqluser", "root");
883 const std::string mysql_passwd = conf.get_str("mysqlpass", "");
884 const std::string mysql_dbname = "";
885 auto_mysql db;
886 if (!mysql_real_connect(db, mysql_host.c_str(), mysql_user.c_str(),
887 mysql_passwd.c_str(), mysql_dbname.c_str(), mysql_port, 0, 0)) {
888 fprintf(stderr, "mysql: error=[%s]\n", mysql_error(db));
889 fatal_exit("hs_longrun_init_table");
890 }
891 mysql_do(db, "drop database if exists hstestdb");
892 mysql_do(db, "create database hstestdb");
893 mysql_do(db, "use hstestdb");
894 mysql_do(db,
895 "create table hstesttbl ("
896 "k int primary key,"
897 "v1 varchar(32) not null,"
898 "v2 varchar(32) not null,"
899 "v3 varchar(32) not null"
900 ") character set utf8 collate utf8_bin engine = innodb");
901 for (int i = 0; i < num_prepare; ++i) {
902 const std::string i_str = to_stdstring(i);
903 const std::string v1 = "pv1_" + i_str;
904 const std::string v2 = "pv2_" + i_str;
905 const std::string v3 = "pv3_" + i_str;
906 char buf[1024];
907 snprintf(buf, 1024, "insert into hstesttbl(k, v1, v2, v3) values"
908 "(%d, '%s', '%s', '%s')", i, v1.c_str(), v2.c_str(), v3.c_str());
909 mysql_do(db, buf);
910 record_value *rec = shared.records[i];
911 rec->key = i_str;
912 rec->values.resize(4);
913 rec->values[0] = i_str;
914 rec->values[1] = v1;
915 rec->values[2] = v2;
916 rec->values[3] = v3;
917 rec->deleted = false;
918 }
919}
920
921int
922hs_longrun_main(int argc, char **argv)
923{
924 hs_longrun_shared shared;
925 parse_args(argc, argv, shared.conf);
926 shared.conf["host"] = shared.conf.get_str("host", "localhost");
927 shared.verbose = shared.conf.get_int("verbose", 1);
928 const int table_size = shared.conf.get_int("table_size", 10000);
929 for (int i = 0; i < table_size; ++i) {
930 std::auto_ptr<record_value> rec(new record_value());
931 rec->key = to_stdstring(i);
932 shared.records.push_back_ptr(rec);
933 }
934 mysql_library_init(0, 0, 0);
935 const int duration = shared.conf.get_int("duration", 10);
936 const int num_hsinsert = shared.conf.get_int("num_hsinsert", 10);
937 const int num_hsdelete = shared.conf.get_int("num_hsdelete", 10);
938 const int num_hsupdate = shared.conf.get_int("num_hsupdate", 10);
939 const int num_hsread = shared.conf.get_int("num_hsread", 10);
940 const int num_myread = shared.conf.get_int("num_myread", 10);
941 const int num_mydelins = shared.conf.get_int("num_mydelins", 10);
942 int num_hsreadnolock = shared.conf.get_int("num_hsreadnolock", 10);
943 const bool always_filled = (num_hsinsert == 0 && num_hsdelete == 0);
944 if (!always_filled) {
945 num_hsreadnolock = 0;
946 }
947 hs_longrun_init_table(shared.conf, always_filled ? table_size : 0,
948 shared);
949 /* create worker threads */
950 static const struct thrtmpl_type {
951 const char *type; char op; int num; int hs; int lock;
952 } thrtmpl[] = {
953 { "hsinsert", 'I', num_hsinsert, 1, 1 },
954 { "hsdelete", 'D', num_hsdelete, 1, 1 },
955 { "hsupdate", 'U', num_hsupdate, 1, 1 },
956 { "hsread", 'R', num_hsread, 1, 1 },
957 { "hsreadnolock", 'N', num_hsreadnolock, 1, 0 },
958 { "myread", 'R', num_myread, 0, 1 },
959 { "mydelins", 'T', num_mydelins, 0, 1 },
960 };
961 typedef auto_ptrcontainer< std::vector<hs_longrun_thread_base *> > thrs_type;
962 thrs_type thrs;
963 for (size_t i = 0; i < sizeof(thrtmpl)/sizeof(thrtmpl[0]); ++i) {
964 const thrtmpl_type& e = thrtmpl[i];
965 for (int j = 0; j < e.num; ++j) {
966 int id = thrs.size();
967 const hs_longrun_thread_hs::arg_type arg(id, e.type, e.op, e.lock,
968 shared);
969 std::auto_ptr<hs_longrun_thread_base> thr;
970 if (e.hs) {
971 thr.reset(new hs_longrun_thread_hs(arg));
972 } else {
973 thr.reset(new hs_longrun_thread_my(arg));
974 }
975 thrs.push_back_ptr(thr);
976 }
977 }
978 shared.num_threads = thrs.size();
979 /* start threads */
980 fprintf(stderr, "START\n");
981 shared.running = 1;
982 for (size_t i = 0; i < thrs.size(); ++i) {
983 thrs[i]->start();
984 }
985 /* wait */
986 sleep(duration);
987 /* stop thread */
988 shared.running = 0;
989 for (size_t i = 0; i < thrs.size(); ++i) {
990 thrs[i]->join();
991 }
992 fprintf(stderr, "DONE\n");
993 /* summary */
994 typedef std::map<std::string, hs_longrun_stat> stat_map;
995 stat_map sm;
996 for (size_t i = 0; i < thrs.size(); ++i) {
997 hs_longrun_thread_base *const thr = thrs[i];
998 const std::string wt = thr->arg.worker_type;
999 hs_longrun_stat& v = sm[wt];
1000 v.add(thr->stat);
1001 }
1002 hs_longrun_stat total;
1003 for (stat_map::const_iterator i = sm.begin(); i != sm.end(); ++i) {
1004 if (i->second.verify_error_count != 0) {
1005 fprintf(stderr, "%s verify_error %llu\n", i->first.c_str(),
1006 i->second.verify_error_count);
1007 }
1008 if (i->second.runtime_error_count) {
1009 fprintf(stderr, "%s runtime_error %llu\n", i->first.c_str(),
1010 i->second.runtime_error_count);
1011 }
1012 if (i->second.unknown_count) {
1013 fprintf(stderr, "%s unknown %llu\n", i->first.c_str(),
1014 i->second.unknown_count);
1015 }
1016 fprintf(stderr, "%s success %llu\n", i->first.c_str(),
1017 i->second.success_count);
1018 total.add(i->second);
1019 }
1020 if (total.verify_error_count != 0) {
1021 fprintf(stderr, "TOTAL verify_error %llu\n", total.verify_error_count);
1022 }
1023 if (total.runtime_error_count != 0) {
1024 fprintf(stderr, "TOTAL runtime_error %llu\n", total.runtime_error_count);
1025 }
1026 if (total.unknown_count != 0) {
1027 fprintf(stderr, "TOTAL unknown %llu\n", total.unknown_count);
1028 }
1029 fprintf(stderr, "TOTAL success %llu\n", total.success_count);
1030 mysql_library_end();
1031 return 0;
1032}
1033
1034};
1035
1036int
1037main(int argc, char **argv)
1038{
1039 return dena::hs_longrun_main(argc, argv);
1040}
1041
10420
=== added file 'HandlerSocket-Plugin-for-MySQL/client/hspool_test.pl'
--- HandlerSocket-Plugin-for-MySQL/client/hspool_test.pl 1970-01-01 00:00:00 +0000
+++ HandlerSocket-Plugin-for-MySQL/client/hspool_test.pl 2012-06-25 04:42:22 +0000
@@ -0,0 +1,224 @@
1#!/usr/bin/perl
2
3use strict;
4use warnings;
5use DB::HandlerSocket::Pool;
6use DBI;
7
8my %conf = ();
9for my $i (@ARGV) {
10 my ($k, $v) = split(/=/, $i);
11 $conf{$k} = $v;
12}
13
14my $verbose = get_conf("verbose", 0);
15my $actions_str = get_conf("actions",
16 "create,insert,verify,verify2,verify3,verify4,clean");
17my $tablesize = get_conf("tablesize", 1000);
18my $db = get_conf("db", "hstestdb");
19my $table = get_conf("table", "testtbl");
20my $table_schema = get_conf("table_schema", undef);
21my $engine = get_conf("engine", "innodb");
22my $host = get_conf("host", "localhost");
23my $mysqlport = get_conf("mysqlport", 3306);
24my $hsport_rd = get_conf("hsport_rd", 9998);
25my $hsport_wr = get_conf("hsport_wr", 9999);
26my $loop = get_conf("loop", 10000);
27my $op = get_conf("op", "=");
28my $ssps = get_conf("ssps", 0);
29my $num_moreflds = get_conf("moreflds", 0);
30my $moreflds_prefix = get_conf("moreflds_prefix", "f");
31my $mysql_user = 'root';
32my $mysql_password = '';
33
34my $dsn = "DBI:mysql:database=;host=$host;port=$mysqlport"
35 . ";mysql_server_prepare=$ssps";
36my $dbh = DBI->connect($dsn, $mysql_user, $mysql_password,
37 { RaiseError => 1 });
38my $hsargs = { 'host' => $host, 'port' => $hsport_rd };
39my $hspool = new DB::HandlerSocket::Pool({
40 hostmap => {
41 "$db.$table" => {
42 host => $host,
43 port => $hsport_rd,
44 },
45 },
46 resolve => undef,
47 error => undef,
48});
49$table_schema = "(k int primary key, fc30 varchar(30), ft text)"
50 if (!defined($table_schema));
51
52my @actions = split(/,/, $actions_str);
53for my $action (@actions) {
54 print "ACTION: $action\n";
55 eval "hstest_$action()";
56 if ($@) {
57 die $@;
58 }
59 print "ACTION: $action DONE\n";
60}
61
62sub get_conf {
63 my ($key, $def) = @_;
64 my $val = $conf{$key};
65 if ($val) {
66 print "$key=$val\n";
67 } else {
68 $val = $def;
69 my $defstr = $def || "(undef)";
70 print "$key=$defstr(default)\n";
71 }
72 return $val;
73}
74
75sub hstest_create {
76 $dbh->do("drop database if exists $db");
77 $dbh->do("create database $db");
78 $dbh->do("use $db");
79 $dbh->do("create table $table $table_schema engine=$engine");
80}
81
82sub hstest_dump {
83 $dbh->do("use $db");
84 my $sth = $dbh->prepare("select * from $table");
85 $sth->execute();
86 my $arr = $sth->fetchall_arrayref();
87 for my $rec (@$arr) {
88 print "REC:";
89 for my $row (@$rec) {
90 print " $row";
91 }
92 print "\n";
93 }
94}
95
96sub hstest_insert {
97 $dbh->do("use $db");
98 my $sth = $dbh->prepare("insert into $table values (?, ?, ?)");
99 for (my $k = 0; $k < $tablesize; ++$k) {
100 my $fc30 = "fc30_$k";
101 my $ft = "ft_$k";
102 $sth->execute($k, $fc30, $ft);
103 }
104}
105
106sub hstest_verify {
107 $dbh->do("use $db");
108 my $sth = $dbh->prepare("select * from $table order by k");
109 $sth->execute();
110 my $arr = $sth->fetchall_arrayref();
111 my $hsres = $hspool->index_find($db, $table, "PRIMARY", "k,fc30,ft",
112 ">=", [ 0 ], $tablesize, 0);
113 for (my $i = 0; $i < $tablesize; ++$i) {
114 my $rec = $arr->[$i];
115 my $differ = 0;
116 print "REC:" if $verbose;
117 for (my $j = 0; $j < 3; ++$j) {
118 my $fld = $rec->[$j];
119 my $hsidx = $i * 3 + $j;
120 my $hsfld = $hsres->[$hsidx];
121 if ($hsfld ne $fld) {
122 $differ = 1;
123 }
124 if ($differ) {
125 print " $fld:$hsfld" if $verbose;
126 } else {
127 print " $hsfld" if $verbose;
128 }
129 }
130 print "\n" if $verbose;
131 if ($differ) {
132 die "verification failed";
133 }
134 }
135}
136
137sub hstest_verify2 {
138 $dbh->do("use $db");
139 my $sth = $dbh->prepare("select * from $table order by k");
140 $sth->execute();
141 my $arr = $sth->fetchall_arrayref();
142 my $hsresa = $hspool->index_find_multi($db, $table, "PRIMARY",
143 "k,fc30,ft", [ [ -1, ">=", [ 0 ], $tablesize, 0 ] ]);
144 my $hsres = $hsresa->[0];
145 for (my $i = 0; $i < $tablesize; ++$i) {
146 my $rec = $arr->[$i];
147 my $differ = 0;
148 print "REC:" if $verbose;
149 for (my $j = 0; $j < 3; ++$j) {
150 my $fld = $rec->[$j];
151 my $hsidx = $i * 3 + $j;
152 my $hsfld = $hsres->[$hsidx];
153 if ($hsfld ne $fld) {
154 $differ = 1;
155 }
156 if ($differ) {
157 print " $fld:$hsfld" if $verbose;
158 } else {
159 print " $hsfld" if $verbose;
160 }
161 }
162 print "\n" if $verbose;
163 if ($differ) {
164 die "verification failed";
165 }
166 }
167}
168
169sub hashref_to_str {
170 my $href = $_[0];
171 my $r = '';
172 for my $k (sort keys %$href) {
173 my $v = $href->{$k};
174 $r .= "($k=>$v)";
175 }
176 return $r;
177}
178
179sub hstest_verify3 {
180 $dbh->do("use $db");
181 my $sth = $dbh->prepare("select * from $table order by k");
182 $sth->execute();
183 my $hsres_t = $hspool->index_find($db, $table, "PRIMARY", "k,fc30,ft",
184 ">=", [ 0 ], $tablesize, 0);
185 my $hsres = DB::HandlerSocket::Pool::result_single_to_hasharr(
186 [ 'k', 'fc30', 'ft' ], $hsres_t);
187 for (my $i = 0; $i < $tablesize; ++$i) {
188 my $mystr = hashref_to_str($sth->fetchrow_hashref());
189 my $hsstr = hashref_to_str($hsres->[$i]);
190 if ($mystr ne $hsstr) {
191 print "DIFF my=[$mystr] hs=[$hsstr]\n" if $verbose;
192 die "verification failed";
193 } else {
194 print "OK $hsstr\n" if $verbose;
195 }
196 }
197}
198
199sub hstest_verify4 {
200 $dbh->do("use $db");
201 my $sth = $dbh->prepare("select * from $table order by k");
202 $sth->execute();
203 my $hsres_t = $hspool->index_find($db, $table, "PRIMARY", "k,fc30,ft",
204 ">=", [ 0 ], $tablesize, 0);
205 my $hsres = DB::HandlerSocket::Pool::result_single_to_hashhash(
206 [ 'k', 'fc30', 'ft' ], 'k', $hsres_t);
207 my $rechash = $sth->fetchall_hashref('k');
208 while (my ($k, $href) = each (%$rechash)) {
209 my $mystr = hashref_to_str($href);
210 my $hsstr = hashref_to_str($hsres->{$k});
211 if ($mystr ne $hsstr) {
212 print "DIFF my=[$mystr] hs=[$hsstr]\n" if $verbose;
213 die "verification failed";
214 } else {
215 print "OK $hsstr\n" if $verbose;
216 }
217 }
218}
219
220sub hstest_clean {
221 $hspool->clear_pool();
222 $dbh->do("drop database if exists $db");
223}
224
0225
=== removed file 'HandlerSocket-Plugin-for-MySQL/client/hspool_test.pl'
--- HandlerSocket-Plugin-for-MySQL/client/hspool_test.pl 2011-01-10 13:39:35 +0000
+++ HandlerSocket-Plugin-for-MySQL/client/hspool_test.pl 1970-01-01 00:00:00 +0000
@@ -1,224 +0,0 @@
1#!/usr/bin/perl
2
3use strict;
4use warnings;
5use DB::HandlerSocket::Pool;
6use DBI;
7
8my %conf = ();
9for my $i (@ARGV) {
10 my ($k, $v) = split(/=/, $i);
11 $conf{$k} = $v;
12}
13
14my $verbose = get_conf("verbose", 0);
15my $actions_str = get_conf("actions",
16 "create,insert,verify,verify2,verify3,verify4,clean");
17my $tablesize = get_conf("tablesize", 1000);
18my $db = get_conf("db", "hstestdb");
19my $table = get_conf("table", "testtbl");
20my $table_schema = get_conf("table_schema", undef);
21my $engine = get_conf("engine", "innodb");
22my $host = get_conf("host", "localhost");
23my $mysqlport = get_conf("mysqlport", 3306);
24my $hsport_rd = get_conf("hsport_rd", 9998);
25my $hsport_wr = get_conf("hsport_wr", 9999);
26my $loop = get_conf("loop", 10000);
27my $op = get_conf("op", "=");
28my $ssps = get_conf("ssps", 0);
29my $num_moreflds = get_conf("moreflds", 0);
30my $moreflds_prefix = get_conf("moreflds_prefix", "f");
31my $mysql_user = 'root';
32my $mysql_password = '';
33
34my $dsn = "DBI:mysql:database=;host=$host;port=$mysqlport"
35 . ";mysql_server_prepare=$ssps";
36my $dbh = DBI->connect($dsn, $mysql_user, $mysql_password,
37 { RaiseError => 1 });
38my $hsargs = { 'host' => $host, 'port' => $hsport_rd };
39my $hspool = new DB::HandlerSocket::Pool({
40 hostmap => {
41 "$db.$table" => {
42 host => $host,
43 port => $hsport_rd,
44 },
45 },
46 resolve => undef,
47 error => undef,
48});
49$table_schema = "(k int primary key, fc30 varchar(30), ft text)"
50 if (!defined($table_schema));
51
52my @actions = split(/,/, $actions_str);
53for my $action (@actions) {
54 print "ACTION: $action\n";
55 eval "hstest_$action()";
56 if ($@) {
57 die $@;
58 }
59 print "ACTION: $action DONE\n";
60}
61
62sub get_conf {
63 my ($key, $def) = @_;
64 my $val = $conf{$key};
65 if ($val) {
66 print "$key=$val\n";
67 } else {
68 $val = $def;
69 my $defstr = $def || "(undef)";
70 print "$key=$defstr(default)\n";
71 }
72 return $val;
73}
74
75sub hstest_create {
76 $dbh->do("drop database if exists $db");
77 $dbh->do("create database $db");
78 $dbh->do("use $db");
79 $dbh->do("create table $table $table_schema engine=$engine");
80}
81
82sub hstest_dump {
83 $dbh->do("use $db");
84 my $sth = $dbh->prepare("select * from $table");
85 $sth->execute();
86 my $arr = $sth->fetchall_arrayref();
87 for my $rec (@$arr) {
88 print "REC:";
89 for my $row (@$rec) {
90 print " $row";
91 }
92 print "\n";
93 }
94}
95
96sub hstest_insert {
97 $dbh->do("use $db");
98 my $sth = $dbh->prepare("insert into $table values (?, ?, ?)");
99 for (my $k = 0; $k < $tablesize; ++$k) {
100 my $fc30 = "fc30_$k";
101 my $ft = "ft_$k";
102 $sth->execute($k, $fc30, $ft);
103 }
104}
105
106sub hstest_verify {
107 $dbh->do("use $db");
108 my $sth = $dbh->prepare("select * from $table order by k");
109 $sth->execute();
110 my $arr = $sth->fetchall_arrayref();
111 my $hsres = $hspool->index_find($db, $table, "PRIMARY", "k,fc30,ft",
112 ">=", [ 0 ], $tablesize, 0);
113 for (my $i = 0; $i < $tablesize; ++$i) {
114 my $rec = $arr->[$i];
115 my $differ = 0;
116 print "REC:" if $verbose;
117 for (my $j = 0; $j < 3; ++$j) {
118 my $fld = $rec->[$j];
119 my $hsidx = $i * 3 + $j;
120 my $hsfld = $hsres->[$hsidx];
121 if ($hsfld ne $fld) {
122 $differ = 1;
123 }
124 if ($differ) {
125 print " $fld:$hsfld" if $verbose;
126 } else {
127 print " $hsfld" if $verbose;
128 }
129 }
130 print "\n" if $verbose;
131 if ($differ) {
132 die "verification failed";
133 }
134 }
135}
136
137sub hstest_verify2 {
138 $dbh->do("use $db");
139 my $sth = $dbh->prepare("select * from $table order by k");
140 $sth->execute();
141 my $arr = $sth->fetchall_arrayref();
142 my $hsresa = $hspool->index_find_multi($db, $table, "PRIMARY",
143 "k,fc30,ft", [ [ -1, ">=", [ 0 ], $tablesize, 0 ] ]);
144 my $hsres = $hsresa->[0];
145 for (my $i = 0; $i < $tablesize; ++$i) {
146 my $rec = $arr->[$i];
147 my $differ = 0;
148 print "REC:" if $verbose;
149 for (my $j = 0; $j < 3; ++$j) {
150 my $fld = $rec->[$j];
151 my $hsidx = $i * 3 + $j;
152 my $hsfld = $hsres->[$hsidx];
153 if ($hsfld ne $fld) {
154 $differ = 1;
155 }
156 if ($differ) {
157 print " $fld:$hsfld" if $verbose;
158 } else {
159 print " $hsfld" if $verbose;
160 }
161 }
162 print "\n" if $verbose;
163 if ($differ) {
164 die "verification failed";
165 }
166 }
167}
168
169sub hashref_to_str {
170 my $href = $_[0];
171 my $r = '';
172 for my $k (sort keys %$href) {
173 my $v = $href->{$k};
174 $r .= "($k=>$v)";
175 }
176 return $r;
177}
178
179sub hstest_verify3 {
180 $dbh->do("use $db");
181 my $sth = $dbh->prepare("select * from $table order by k");
182 $sth->execute();
183 my $hsres_t = $hspool->index_find($db, $table, "PRIMARY", "k,fc30,ft",
184 ">=", [ 0 ], $tablesize, 0);
185 my $hsres = DB::HandlerSocket::Pool::result_single_to_hasharr(
186 [ 'k', 'fc30', 'ft' ], $hsres_t);
187 for (my $i = 0; $i < $tablesize; ++$i) {
188 my $mystr = hashref_to_str($sth->fetchrow_hashref());
189 my $hsstr = hashref_to_str($hsres->[$i]);
190 if ($mystr ne $hsstr) {
191 print "DIFF my=[$mystr] hs=[$hsstr]\n" if $verbose;
192 die "verification failed";
193 } else {
194 print "OK $hsstr\n" if $verbose;
195 }
196 }
197}
198
199sub hstest_verify4 {
200 $dbh->do("use $db");
201 my $sth = $dbh->prepare("select * from $table order by k");
202 $sth->execute();
203 my $hsres_t = $hspool->index_find($db, $table, "PRIMARY", "k,fc30,ft",
204 ">=", [ 0 ], $tablesize, 0);
205 my $hsres = DB::HandlerSocket::Pool::result_single_to_hashhash(
206 [ 'k', 'fc30', 'ft' ], 'k', $hsres_t);
207 my $rechash = $sth->fetchall_hashref('k');
208 while (my ($k, $href) = each (%$rechash)) {
209 my $mystr = hashref_to_str($href);
210 my $hsstr = hashref_to_str($hsres->{$k});
211 if ($mystr ne $hsstr) {
212 print "DIFF my=[$mystr] hs=[$hsstr]\n" if $verbose;
213 die "verification failed";
214 } else {
215 print "OK $hsstr\n" if $verbose;
216 }
217 }
218}
219
220sub hstest_clean {
221 $hspool->clear_pool();
222 $dbh->do("drop database if exists $db");
223}
224
2250
=== added file 'HandlerSocket-Plugin-for-MySQL/client/hstest.cpp'
--- HandlerSocket-Plugin-for-MySQL/client/hstest.cpp 1970-01-01 00:00:00 +0000
+++ HandlerSocket-Plugin-for-MySQL/client/hstest.cpp 2012-06-25 04:42:22 +0000
@@ -0,0 +1,1532 @@
1
2// vim:sw=2:ai
3
4#include <signal.h>
5#include <sys/time.h>
6#include <stdio.h>
7#include <string.h>
8#include <vector>
9#include <stdlib.h>
10#include <memory>
11#include <errno.h>
12#include <mysql.h>
13#include <time.h>
14#include <sys/types.h>
15#include <sys/stat.h>
16#include <fcntl.h>
17
18#include "util.hpp"
19#include "auto_ptrcontainer.hpp"
20#include "socket.hpp"
21#include "thread.hpp"
22#include "hstcpcli.hpp"
23
24#if __GNUC__ >= 4
25long atomic_exchange_and_add(volatile long *valp, long c)
26{
27 return __sync_fetch_and_add(valp, c);
28}
29#else
30#include <bits/atomicity.h>
31using namespace __gnu_cxx;
32long atomic_exchange_and_add(volatile long *valp, long c)
33{
34 return __exchange_and_add((volatile _Atomic_word *)valp, c);
35}
36#endif
37
38namespace dena {
39
40struct auto_mysql : private noncopyable {
41 auto_mysql() : db(0) {
42 reset();
43 }
44 ~auto_mysql() {
45 if (db) {
46 mysql_close(db);
47 }
48 }
49 void reset() {
50 if (db) {
51 mysql_close(db);
52 }
53 if ((db = mysql_init(0)) == 0) {
54 fatal_abort("failed to initialize mysql client");
55 }
56 }
57 operator MYSQL *() const { return db; }
58 private:
59 MYSQL *db;
60};
61
62struct auto_mysql_res : private noncopyable {
63 auto_mysql_res(MYSQL *db) {
64 res = mysql_store_result(db);
65 }
66 ~auto_mysql_res() {
67 if (res) {
68 mysql_free_result(res);
69 }
70 }
71 operator MYSQL_RES *() const { return res; }
72 private:
73 MYSQL_RES *res;
74};
75
76struct auto_mysql_stmt : private noncopyable {
77 auto_mysql_stmt(MYSQL *db) {
78 stmt = mysql_stmt_init(db);
79 }
80 ~auto_mysql_stmt() {
81 if (stmt) {
82 mysql_stmt_close(stmt);
83 }
84 }
85 operator MYSQL_STMT *() const { return stmt; }
86 private:
87 MYSQL_STMT *stmt;
88};
89
90namespace {
91
92double
93gettimeofday_double()
94{
95 struct timeval tv = { };
96 if (gettimeofday(&tv, 0) != 0) {
97 fatal_abort("gettimeofday");
98 }
99 return static_cast<double>(tv.tv_usec) / 1000000 + tv.tv_sec;
100}
101
102// unused
103void
104wait_close(int fd)
105{
106 char buf[1024];
107 while (true) {
108 int r = read(fd, buf, sizeof(buf));
109 if (r <= 0) {
110 break;
111 }
112 }
113}
114
115// unused
116void
117gentle_close(int fd)
118{
119 int r = shutdown(fd, SHUT_WR);
120 if (r != 0) {
121 return;
122 }
123 wait_close(fd);
124}
125
126};
127
128struct hstest_shared {
129 config conf;
130 socket_args arg;
131 int verbose;
132 size_t loop;
133 size_t pipe;
134 char op;
135 long num_threads;
136 mutable volatile long count;
137 mutable volatile long conn_count;
138 long wait_conn;
139 volatile char *keygen;
140 long keygen_size;
141 mutable volatile int enable_timing;
142 int usleep;
143 int dump;
144 hstest_shared() : verbose(0), loop(0), pipe(0), op('G'), num_threads(0),
145 count(0), conn_count(0), wait_conn(0), keygen(0), keygen_size(0),
146 enable_timing(0), usleep(0), dump(0) { }
147 void increment_count(unsigned int c = 1) const volatile {
148 atomic_exchange_and_add(&count, c);
149 }
150 void increment_conn(unsigned int c) const volatile {
151 atomic_exchange_and_add(&conn_count, c);
152 while (wait_conn != 0 && conn_count < wait_conn) {
153 sleep(1);
154 }
155 // fprintf(stderr, "wait_conn=%ld done\n", wait_conn);
156 }
157};
158
159struct hstest_thread {
160 struct arg_type {
161 size_t id;
162 const hstest_shared& sh;
163 bool watch_flag;
164 arg_type(size_t i, const hstest_shared& s, bool w)
165 : id(i), sh(s), watch_flag(w) { }
166 };
167 hstest_thread(const arg_type& a) : arg(a), io_success_count(0),
168 op_success_count(0), response_min(99999), response_max(0),
169 response_sum(0), response_avg(0) { }
170 void operator ()();
171 void test_1();
172 void test_2_3(int test_num);
173 void test_4_5(int test_num);
174 void test_6(int test_num);
175 void test_7(int test_num);
176 void test_8(int test_num);
177 void test_9(int test_num);
178 void test_10(int test_num);
179 void test_11(int test_num);
180 void test_12(int test_num);
181 void test_21(int test_num);
182 void test_22(int test_num);
183 void test_watch();
184 void sleep_if();
185 void set_timing(double time_spent);
186 arg_type arg;
187 auto_file fd;
188 size_t io_success_count;
189 size_t op_success_count;
190 double response_min, response_max, response_sum, response_avg;
191};
192
193void
194hstest_thread::test_1()
195{
196 char buf[1024];
197 unsigned int seed = arg.id;
198 seed ^= arg.sh.conf.get_int("seed_xor", 0);
199 std::string err;
200 if (socket_connect(fd, arg.sh.arg, err) != 0) {
201 fprintf(stderr, "connect: %d %s\n", errno, strerror(errno));
202 return;
203 }
204 const char op = arg.sh.op;
205 const int tablesize = arg.sh.conf.get_int("tablesize", 0);
206 for (size_t i = 0; i < arg.sh.loop; ++i) {
207 for (size_t j = 0; j < arg.sh.pipe; ++j) {
208 int k = 0, v = 0, len = 0;
209 if (op == 'G') {
210 k = rand_r(&seed);
211 v = rand_r(&seed); /* unused */
212 if (tablesize != 0) {
213 k &= tablesize;
214 }
215 len = snprintf(buf, sizeof(buf), "%c\tk%d\n", op, k);
216 } else {
217 k = rand_r(&seed);
218 v = rand_r(&seed);
219 if (tablesize != 0) {
220 k &= tablesize;
221 }
222 len = snprintf(buf, sizeof(buf), "%c\tk%d\tv%d\n", op, k, v);
223 }
224 const int wlen = write(fd.get(), buf, len);
225 if (wlen != len) {
226 return;
227 }
228 }
229 size_t read_cnt = 0;
230 size_t read_pos = 0;
231 while (read_cnt < arg.sh.pipe) {
232 const int rlen = read(fd.get(), buf + read_pos, sizeof(buf) - read_pos);
233 if (rlen <= 0) {
234 return;
235 }
236 read_pos += rlen;
237 while (true) {
238 const char *const p = static_cast<const char *>(memchr(buf, '\n',
239 read_pos));
240 if (p == 0) {
241 break;
242 }
243 ++read_cnt;
244 ++io_success_count;
245 arg.sh.increment_count();
246 if (p != buf && buf[0] == '=') {
247 ++op_success_count;
248 }
249 const size_t rest_size = buf + read_pos - (p + 1);
250 if (rest_size != 0) {
251 memmove(buf, p + 1, rest_size);
252 }
253 read_pos = rest_size;
254 }
255 }
256 }
257}
258
259void
260hstest_thread::test_2_3(int test_num)
261{
262#if 0
263 char buf_k[128], buf_v[128];
264 unsigned int seed = arg.id;
265 op_base_t op = static_cast<op_base_t>(arg.sh.op);
266 micli_ptr hnd;
267 if (test_num == 2) {
268 hnd = micli_i::create_remote(arg.sh.conf);
269 } else if (test_num == 3) {
270 // hnd = micli_i::create_inproc(arg.sh.localdb);
271 }
272 if (hnd.get() == 0) {
273 return;
274 }
275 for (size_t i = 0; i < arg.sh.loop; ++i) {
276 for (size_t j = 0; j < arg.sh.pipe; ++j) {
277 int k = 0, v = 0, klen = 0, vlen = 0;
278 k = rand_r(&seed);
279 klen = snprintf(buf_k, sizeof(buf_k), "k%d", k);
280 v = rand_r(&seed); /* unused */
281 vlen = snprintf(buf_v, sizeof(buf_v), "v%d", v);
282 string_ref arr[2];
283 arr[0] = string_ref(buf_k, klen);
284 arr[1] = string_ref(buf_v, vlen);
285 pstrarr_ptr rec(arr, 2);
286 if (hnd->execute(op, 0, 0, rec.get_const())) {
287 ++io_success_count;
288 arg.sh.increment_count();
289 const dataset& res = hnd->get_result_ref();
290 if (res.size() == 1) {
291 ++op_success_count;
292 }
293 }
294 }
295 }
296#endif
297}
298
299void
300hstest_thread::test_4_5(int test_num)
301{
302#if 0
303 char buf_k[128], buf_v[8192];
304 memset(buf_v, ' ', sizeof(buf_v));
305 unsigned int seed = arg.id;
306 op_base_t op = static_cast<op_base_t>(arg.sh.op);
307 micli_ptr hnd;
308 if (test_num == 4) {
309 hnd = micli_i::create_remote(arg.sh.conf);
310 } else if (test_num == 5) {
311 hnd = micli_i::create_inproc(arg.sh.localdb);
312 }
313 if (hnd.get() == 0) {
314 return;
315 }
316 for (size_t i = 0; i < arg.sh.loop; ++i) {
317 for (size_t j = 0; j < arg.sh.pipe; ++j) {
318 int k = 0, klen = 0, vlen = 0;
319 k = i & 0x0000ffffUL;
320 if (k == 0) {
321 fprintf(stderr, "k=0\n");
322 }
323 klen = snprintf(buf_k, sizeof(buf_k), "k%d", k);
324 vlen = rand_r(&seed) % 8192;
325 string_ref arr[2];
326 arr[0] = string_ref(buf_k, klen);
327 arr[1] = string_ref(buf_v, vlen);
328 pstrarr_ptr rec(arr, 2);
329 if (hnd->execute(op, 0, 0, rec.get_const())) {
330 ++io_success_count;
331 const dataset& res = hnd->get_result_ref();
332 if (res.size() == 1) {
333 ++op_success_count;
334 }
335 }
336 }
337 }
338#endif
339}
340
341void
342hstest_thread::test_6(int test_num)
343{
344 int count = arg.sh.conf.get_int("count", 1);
345 auto_file fds[count];
346 for (int i = 0; i < count; ++i) {
347 const double t1 = gettimeofday_double();
348 std::string err;
349 if (socket_connect(fds[i], arg.sh.arg, err) != 0) {
350 fprintf(stderr, "id=%zu i=%d err=%s\n", arg.id, i, err.c_str());
351 }
352 const double t2 = gettimeofday_double();
353 if (t2 - t1 > 1) {
354 fprintf(stderr, "id=%zu i=%d time %f\n", arg.id, i, t2 - t1);
355 }
356 }
357}
358
359void
360hstest_thread::test_7(int num)
361{
362 /*
363 set foo 0 0 10
364 0123456789
365 STORED
366 get foo
367 VALUE foo 0 10
368 0123456789
369 END
370 get var
371 END
372 */
373 char buf[1024];
374 const int keep_connection = arg.sh.conf.get_int("keep_connection", 1);
375 unsigned int seed = arg.id;
376 seed ^= arg.sh.conf.get_int("seed_xor", 0);
377 const int tablesize = arg.sh.conf.get_int("tablesize", 0);
378 const char op = arg.sh.op;
379 for (size_t i = 0; i < arg.sh.loop; ++i) {
380 const double tm1 = gettimeofday_double();
381 std::string err;
382 if (fd.get() < 0 && socket_connect(fd, arg.sh.arg, err) != 0) {
383 fprintf(stderr, "connect: %d %s\n", errno, strerror(errno));
384 return;
385 }
386 for (size_t j = 0; j < arg.sh.pipe; ++j) {
387 int k = 0, v = 0, len = 0;
388 if (op == 'G') {
389 k = rand_r(&seed);
390 v = rand_r(&seed); /* unused */
391 if (tablesize != 0) {
392 k &= tablesize;
393 }
394 len = snprintf(buf, sizeof(buf), "get k%d\r\n", k);
395 } else {
396 k = rand_r(&seed);
397 v = rand_r(&seed);
398 if (tablesize != 0) {
399 k &= tablesize;
400 }
401 char vbuf[1024];
402 int vlen = snprintf(vbuf, sizeof(vbuf),
403 "v%d"
404 // "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
405 // "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
406 // "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
407 // "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
408 // "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
409 , v);
410 len = snprintf(buf, sizeof(buf), "set k%d 0 0 %d\r\n%s\r\n",
411 k, vlen, vbuf);
412 }
413 const int wlen = write(fd.get(), buf, len);
414 if (wlen != len) {
415 return;
416 }
417 }
418 size_t read_cnt = 0;
419 size_t read_pos = 0;
420 bool read_response_done = false;
421 bool expect_value = false;
422 while (!read_response_done) {
423 const int rlen = read(fd.get(), buf + read_pos, sizeof(buf) - read_pos);
424 if (rlen <= 0) {
425 return;
426 }
427 read_pos += rlen;
428 while (true) {
429 const char *const p = static_cast<const char *>(memchr(buf, '\n',
430 read_pos));
431 if (p == 0) {
432 break;
433 }
434 ++read_cnt;
435 if (expect_value) {
436 expect_value = false;
437 } else if (p >= buf + 6 && memcmp(buf, "VALUE ", 6) == 0) {
438 expect_value = true;
439 ++op_success_count;
440 } else {
441 if (p == buf + 7 && memcmp(buf, "STORED\r", 7) == 0) {
442 ++op_success_count;
443 }
444 read_response_done = true;
445 }
446 const size_t rest_size = buf + read_pos - (p + 1);
447 if (rest_size != 0) {
448 memmove(buf, p + 1, rest_size);
449 }
450 read_pos = rest_size;
451 }
452 ++io_success_count;
453 }
454 arg.sh.increment_count();
455 if (!keep_connection) {
456 fd.close();
457 }
458 const double tm2 = gettimeofday_double();
459 set_timing(tm2 - tm1);
460 sleep_if();
461 }
462}
463
464struct rec {
465 std::string key;
466 std::string value;
467};
468
469void
470hstest_thread::test_8(int test_num)
471{
472#if 0
473 char buf_k[128], buf_v[128];
474 unsigned int seed = arg.id;
475 // op_base_t op = static_cast<op_base_t>(arg.sh.op);
476 using namespace boost::multi_index;
477 typedef member<rec, std::string, &rec::key> rec_get_key;
478 typedef ordered_unique<rec_get_key> oui;
479 typedef multi_index_container< rec, indexed_by<oui> > mic;
480 #if 0
481 typedef std::map<std::string, std::string> m_type;
482 m_type m;
483 #endif
484 mic m;
485 for (size_t i = 0; i < arg.sh.loop; ++i) {
486 for (size_t j = 0; j < arg.sh.pipe; ++j) {
487 int k = 0, v = 0, klen = 0, vlen = 0;
488 k = rand_r(&seed);
489 klen = snprintf(buf_k, sizeof(buf_k), "k%d", k);
490 v = rand_r(&seed); /* unused */
491 vlen = snprintf(buf_v, sizeof(buf_v), "v%d", v);
492 const std::string ks(buf_k, klen);
493 const std::string vs(buf_v, vlen);
494 rec r;
495 r.key = ks;
496 r.value = vs;
497 m.insert(r);
498 // m.insert(std::make_pair(ks, vs));
499 ++io_success_count;
500 ++op_success_count;
501 arg.sh.increment_count();
502 }
503 }
504#endif
505}
506
507struct mysqltest_thread_initobj : private noncopyable {
508 mysqltest_thread_initobj() {
509 mysql_thread_init();
510 }
511 ~mysqltest_thread_initobj() {
512 mysql_thread_end();
513 }
514};
515
516void
517hstest_thread::test_9(int test_num)
518{
519 /* create table hstest
520 * ( k varchar(255) not null, v varchar(255) not null, primary key(k))
521 * engine = innodb; */
522 auto_mysql db;
523 // mysqltest_thread_initobj initobj;
524 std::string err;
525 const char op = arg.sh.op;
526 const std::string suffix = arg.sh.conf.get_str("value_suffix", "upd");
527 unsigned long long err_cnt = 0;
528 unsigned long long query_cnt = 0;
529 #if 0
530 my_bool reconnect = 0;
531 if (mysql_options(db, MYSQL_OPT_RECONNECT, &reconnect) != 0) {
532 err = "mysql_options() failed";
533 ++err_cnt;
534 return;
535 }
536 #endif
537 unsigned int seed = time(0) + arg.id + 1;
538 seed ^= arg.sh.conf.get_int("seed_xor", 0);
539 drand48_data randbuf;
540 srand48_r(seed, &randbuf);
541 const std::string mysql_host = arg.sh.conf.get_str("host", "localhost");
542 const int mysql_port = arg.sh.conf.get_int("mysqlport", 3306);
543 const int num = arg.sh.loop;
544 const std::string mysql_user = arg.sh.conf.get_str("mysqluser", "root");
545 const std::string mysql_passwd = arg.sh.conf.get_str("mysqlpass", "");
546 const std::string mysql_dbname = arg.sh.conf.get_str("dbname", "hstest");
547 const int keep_connection = arg.sh.conf.get_int("keep_connection", 1);
548 const int verbose = arg.sh.conf.get_int("verbose", 1);
549 const int tablesize = arg.sh.conf.get_int("tablesize", 10000);
550 const int moreflds = arg.sh.conf.get_int("moreflds", 0);
551 const std::string moreflds_prefix = arg.sh.conf.get_str(
552 "moreflds_prefix", "column0123456789_");
553 const int use_handler = arg.sh.conf.get_int("handler", 0);
554 const int sched_flag = arg.sh.conf.get_int("sched", 0);
555 const int use_in = arg.sh.conf.get_int("in", 0);
556 const int ssps = use_in ? 0 : arg.sh.conf.get_int("ssps", 0);
557 std::string flds = "v";
558 for (int i = 0; i < moreflds; ++i) {
559 char buf[1024];
560 snprintf(buf, sizeof(buf), ",%s%d", moreflds_prefix.c_str(), i);
561 flds += std::string(buf);
562 }
563 int connected = 0;
564 std::auto_ptr<auto_mysql_stmt> stmt;
565 string_buffer wbuf;
566 for (int i = 0; i < num; ++i) {
567 const double tm1 = gettimeofday_double();
568 const int flags = 0;
569 if (connected == 0) {
570 if (!mysql_real_connect(db, mysql_host.c_str(),
571 mysql_user.c_str(), mysql_user.empty() ? 0 : mysql_passwd.c_str(),
572 mysql_dbname.c_str(), mysql_port, 0, flags)) {
573 err = "failed to connect: " + std::string(mysql_error(db));
574 if (verbose >= 1) {
575 fprintf(stderr, "e=[%s]\n", err.c_str());
576 }
577 ++err_cnt;
578 return;
579 }
580 arg.sh.increment_conn(1);
581 }
582 int r = 0;
583 if (connected == 0 && use_handler) {
584 const char *const q = "handler hstest_table1 open";
585 r = mysql_real_query(db, q, strlen(q));
586 if (r != 0) {
587 err = 1;
588 }
589 }
590 if (connected == 0 && ssps) {
591 stmt.reset(new auto_mysql_stmt(db));
592 const char *const q = "select v from hstest_table1 where k = ?";
593 r = mysql_stmt_prepare(*stmt, q, strlen(q));
594 if (r != 0) {
595 fprintf(stderr, "ssps err\n");
596 ++err_cnt;
597 return;
598 }
599 }
600 connected = 1;
601 std::string result_str;
602 unsigned int err = 0;
603 unsigned int num_flds = 0, num_affected_rows = 0;
604 int got_data = 0;
605 char buf_query[16384];
606 int buf_query_len = 0;
607 int k = 0, v = 0;
608 {
609 double kf = 0, vf = 0;
610 drand48_r(&randbuf, &kf);
611 drand48_r(&randbuf, &vf);
612 k = int(kf * tablesize);
613 v = int(vf * tablesize);
614 #if 0
615 k = rand_r(&seed);
616 v = rand_r(&seed);
617 if (tablesize != 0) {
618 k %= tablesize;
619 }
620 #endif
621 if (op == 'G') {
622 if (use_handler) {
623 buf_query_len = snprintf(buf_query, sizeof(buf_query),
624 "handler hstest_table1 read `primary` = ( '%d' )", k);
625 // TODO: moreflds
626 } else if (ssps) {
627 //
628 } else if (use_in) {
629 wbuf.clear();
630 char *p = wbuf.make_space(1024);
631 int len = snprintf(p, 1024, "select %s from hstest_table1 where k in ('%d'", flds.c_str(), k);
632 wbuf.space_wrote(len);
633 for (int j = 1; j < use_in; ++j) {
634 /* generate more key */
635 drand48_r(&randbuf, &kf);
636 k = int(kf * tablesize);
637 p = wbuf.make_space(1024);
638 int len = snprintf(p, 1024, ", '%d'", k);
639 wbuf.space_wrote(len);
640 }
641 wbuf.append_literal(")");
642 } else {
643 buf_query_len = snprintf(buf_query, sizeof(buf_query),
644 "select %s from hstest_table1 where k = '%d'", flds.c_str(), k);
645 }
646 } else if (op == 'U') {
647 buf_query_len = snprintf(buf_query, sizeof(buf_query),
648 "update hstest_table1 set v = '%d_%d%s' where k = '%d'",
649 v, k, suffix.c_str(), k);
650 } else if (op == 'R') {
651 buf_query_len = snprintf(buf_query, sizeof(buf_query),
652 "replace into hstest_table1 values ('%d', 'v%d')", k, v);
653 // TODO: moreflds
654 }
655 }
656 if (r == 0) {
657 if (ssps) {
658 MYSQL_BIND bind[1] = { };
659 bind[0].buffer_type = MYSQL_TYPE_LONG;
660 bind[0].buffer = (char *)&k;
661 bind[0].is_null = 0;
662 bind[0].length = 0;
663 if (mysql_stmt_bind_param(*stmt, bind)) {
664 fprintf(stderr, "err: %s\n", mysql_stmt_error(*stmt));
665 ++err_cnt;
666 return;
667 }
668 r = mysql_stmt_execute(*stmt);
669 // fprintf(stderr, "stmt exec\n");
670 } else if (use_in) {
671 r = mysql_real_query(db, wbuf.begin(), wbuf.size());
672 } else {
673 r = mysql_real_query(db, buf_query, buf_query_len);
674 // fprintf(stderr, "real query\n");
675 }
676 ++query_cnt;
677 }
678 if (r != 0) {
679 err = 1;
680 } else if (ssps) {
681 if (verbose >= 0) {
682 char resbuf[1024];
683 unsigned long res_len = 0;
684 MYSQL_BIND bind[1] = { };
685 bind[0].buffer_type = MYSQL_TYPE_STRING;
686 bind[0].buffer = resbuf;
687 bind[0].buffer_length = sizeof(resbuf);
688 bind[0].length = &res_len;
689 if (mysql_stmt_bind_result(*stmt, bind)) {
690 fprintf(stderr, "err: %s\n", mysql_stmt_error(*stmt));
691 ++err_cnt;
692 return;
693 }
694 if (mysql_stmt_fetch(*stmt)) {
695 fprintf(stderr, "err: %s\n", mysql_stmt_error(*stmt));
696 ++err_cnt;
697 return;
698 }
699 if (!result_str.empty()) {
700 result_str += " ";
701 }
702 result_str += std::string(resbuf, res_len);
703 // fprintf(stderr, "SSPS RES: %s\n", result_str.c_str());
704 got_data = 1;
705 } else {
706 got_data = 1;
707 }
708 } else {
709 auto_mysql_res res(db);
710 if (res != 0) {
711 if (verbose >= 0) {
712 num_flds = mysql_num_fields(res);
713 MYSQL_ROW row = 0;
714 while ((row = mysql_fetch_row(res)) != 0) {
715 got_data += 1;
716 unsigned long *const lengths = mysql_fetch_lengths(res);
717 if (verbose >= 2) {
718 for (unsigned int i = 0; i < num_flds; ++i) {
719 if (!result_str.empty()) {
720 result_str += " ";
721 }
722 result_str += std::string(row[i], lengths[i]);
723 }
724 }
725 }
726 } else {
727 MYSQL_ROW row = 0;
728 while ((row = mysql_fetch_row(res)) != 0) {
729 got_data += 1;
730 }
731 }
732 } else {
733 if (mysql_field_count(db) == 0) {
734 num_affected_rows = mysql_affected_rows(db);
735 } else {
736 err = 1;
737 }
738 }
739 }
740 if (verbose >= 2 || (verbose >= 1 && err != 0)) {
741 if (err) {
742 ++err_cnt;
743 const char *const errstr = mysql_error(db);
744 fprintf(stderr, "e=[%s] a=%u q=[%s]\n", errstr,
745 num_affected_rows, buf_query);
746 } else {
747 fprintf(stderr, "a=%u q=[%s] r=[%s]\n", num_affected_rows, buf_query,
748 result_str.c_str());
749 }
750 }
751 if (err == 0) {
752 ++io_success_count;
753 if (num_affected_rows > 0 || got_data > 0) {
754 op_success_count += got_data;
755 } else {
756 if (verbose >= 1) {
757 fprintf(stderr, "k=%d numaff=%u gotdata=%d\n",
758 k, num_affected_rows, got_data);
759 }
760 }
761 arg.sh.increment_count();
762 }
763 if (!keep_connection) {
764 if (stmt.get() != 0) {
765 stmt.reset();
766 }
767 db.reset();
768 connected = 0;
769 }
770 const double tm2 = gettimeofday_double();
771 set_timing(tm2 - tm1);
772 sleep_if();
773 if (sched_flag) {
774 sched_yield();
775 }
776 }
777 if (verbose >= 1) {
778 fprintf(stderr, "thread finished (error_count=%llu)\n", err_cnt);
779 }
780}
781
782void
783hstest_thread::test_10(int test_num)
784{
785 const int keep_connection = arg.sh.conf.get_int("keep_connection", 1);
786 unsigned int seed = time(0) + arg.id + 1;
787 seed ^= arg.sh.conf.get_int("seed_xor", 0);
788 drand48_data randbuf;
789 srand48_r(seed, &randbuf);
790 std::string err;
791 int keepconn_count = 0;
792 const char op = arg.sh.op;
793 const int verbose = arg.sh.conf.get_int("verbose", 1);
794 const std::string suffix = arg.sh.conf.get_str("value_suffix", "upd");
795 const int tablesize = arg.sh.conf.get_int("tablesize", 10000);
796 const int firstkey = arg.sh.conf.get_int("firstkey", 0);
797 const int sched_flag = arg.sh.conf.get_int("sched", 0);
798 const int moreflds = arg.sh.conf.get_int("moreflds", 0);
799 const std::string dbname = arg.sh.conf.get_str("dbname", "hstest");
800 const std::string table = arg.sh.conf.get_str("table", "hstest_table1");
801 const std::string index = arg.sh.conf.get_str("index", "PRIMARY");
802 const std::string field = arg.sh.conf.get_str("field", "v");
803 const int use_in = arg.sh.conf.get_int("in", 0);
804 const std::string moreflds_prefix = arg.sh.conf.get_str(
805 "moreflds_prefix", "column0123456789_");
806 const int dump = arg.sh.dump;
807 const int nodup = arg.sh.conf.get_int("nodup", 0);
808 std::string moreflds_str;
809 for (int i = 0; i < moreflds; ++i) {
810 char sbuf[1024];
811 snprintf(sbuf, sizeof(sbuf), ",%s%d", moreflds_prefix.c_str(), i);
812 moreflds_str += std::string(sbuf);
813 }
814 string_buffer wbuf;
815 char rbuf[16384];
816 for (size_t i = 0; i < arg.sh.loop; ++i) {
817 int len = 0, rlen = 0, wlen = 0;
818 #if 0
819 const double tm1 = gettimeofday_double();
820 #endif
821 if (fd.get() < 0) {
822 if (socket_connect(fd, arg.sh.arg, err) != 0) {
823 fprintf(stderr, "connect: %d %s\n", errno, strerror(errno));
824 return;
825 }
826 char *wp = wbuf.make_space(1024);
827 len = snprintf(wp, 1024,
828 "P\t1\t%s\t%s\tPRIMARY\t%s%s\n", dbname.c_str(), table.c_str(),
829 field.c_str(), moreflds_str.c_str());
830 /* pst_num, db, table, index, retflds */
831 wbuf.space_wrote(len);
832 wlen = write(fd.get(), wbuf.begin(), len);
833 if (len != wlen) {
834 fprintf(stderr, "write: %d %d\n", len, wlen);
835 return;
836 }
837 wbuf.clear();
838 rlen = read(fd.get(), rbuf, sizeof(rbuf));
839 if (rlen <= 0 || rbuf[rlen - 1] != '\n') {
840 fprintf(stderr, "read: rlen=%d errno=%d\n", rlen, errno);
841 return;
842 }
843 if (rbuf[0] != '0') {
844 fprintf(stderr, "failed to open table\n");
845 return;
846 }
847 arg.sh.increment_conn(1);
848 }
849 const double tm1 = gettimeofday_double();
850 for (size_t j = 0; j < arg.sh.pipe; ++j) {
851 int k = 0, v = 0;
852 {
853 while (true) {
854 double kf = 0, vf = 0;
855 drand48_r(&randbuf, &kf);
856 drand48_r(&randbuf, &vf);
857 k = int(kf * tablesize) + firstkey;
858 v = int(vf * tablesize) + firstkey;
859 if (k - firstkey < arg.sh.keygen_size) {
860 volatile char *const ptr = arg.sh.keygen + (k - firstkey);
861 // int oldv = __sync_fetch_and_or(ptr, 1);
862 int oldv = *ptr;
863 *ptr += 1;
864 if (nodup && oldv != 0) {
865 if (dump) {
866 fprintf(stderr, "retry\n");
867 }
868 continue;
869 }
870 } else {
871 if (nodup) {
872 if (dump) {
873 fprintf(stderr, "retry2\n");
874 }
875 continue;
876 }
877 }
878 size_t len = 0;
879 if (op == 'G') {
880 if (use_in) {
881 char *wp = wbuf.make_space(1024);
882 len = snprintf(wp, 1024, "1\t=\t1\t\t%d\t0\t@\t0\t%d\t%d",
883 use_in, use_in, k);
884 wbuf.space_wrote(len);
885 for (int j = 1; j < use_in; ++j) {
886 drand48_r(&randbuf, &kf);
887 k = int(kf * tablesize) + firstkey;
888 char *wp = wbuf.make_space(1024);
889 len = snprintf(wp, 1024, "\t%d", k);
890 wbuf.space_wrote(len);
891 }
892 wbuf.append_literal("\n");
893 } else {
894 char *wp = wbuf.make_space(1024);
895 len = snprintf(wp, 1024, "1\t=\t1\t%d\n", k);
896 wbuf.space_wrote(len);
897 }
898 } else if (op == 'U') {
899 char *wp = wbuf.make_space(1024);
900 len = snprintf(wp, 1024,
901 "1\t=\t1\t%d\t1\t0\tU\t%d_%d%s\n", k, v, k, suffix.c_str());
902 wbuf.space_wrote(len);
903 }
904 break;
905 }
906 }
907 }
908 wlen = write(fd.get(), wbuf.begin(), wbuf.size());
909 if (wlen != wbuf.size()) {
910 fprintf(stderr, "write: %d %d\n", (int)wbuf.size(), wlen);
911 return;
912 }
913 wbuf.clear();
914 size_t read_cnt = 0;
915 size_t read_pos = 0;
916 while (read_cnt < arg.sh.pipe) {
917 rlen = read(fd.get(), rbuf + read_pos, sizeof(rbuf) - read_pos);
918 if (rlen <= 0) {
919 fprintf(stderr, "read: %d\n", rlen);
920 return;
921 }
922 read_pos += rlen;
923 while (true) {
924 const char *const nl = static_cast<const char *>(memchr(rbuf, '\n',
925 read_pos));
926 if (nl == 0) {
927 break;
928 }
929 ++read_cnt;
930 ++io_success_count;
931 const char *t1 = static_cast<const char *>(memchr(rbuf, '\t',
932 nl - rbuf));
933 if (t1 == 0) {
934 fprintf(stderr, "error \n");
935 break;
936 }
937 ++t1;
938 const char *t2 = static_cast<const char *>(memchr(t1, '\t',
939 nl - t1));
940 if (t2 == 0) {
941 if (verbose > 1) {
942 fprintf(stderr, "key: notfound \n");
943 }
944 break;
945 }
946 ++t2;
947 if (t1 == rbuf + 2 && rbuf[0] == '0') {
948 if (op == 'G') {
949 ++op_success_count;
950 arg.sh.increment_count();
951 } else if (op == 'U') {
952 const char *t3 = t2;
953 while (t3 != nl && t3[0] >= 0x10) {
954 ++t3;
955 }
956 if (t3 != t2 + 1 || t2[0] != '1') {
957 const std::string mess(t2, t3);
958 fprintf(stderr, "mod: %s\n", mess.c_str());
959 } else {
960 ++op_success_count;
961 arg.sh.increment_count();
962 if (arg.sh.dump && arg.sh.pipe == 1) {
963 fwrite(wbuf.begin(), wbuf.size(), 1, stderr);
964 }
965 }
966 }
967 } else {
968 const char *t3 = t2;
969 while (t3 != nl && t3[0] >= 0x10) {
970 ++t3;
971 }
972 const std::string mess(t2, t3);
973 fprintf(stderr, "err: %s\n", mess.c_str());
974 }
975 const size_t rest_size = rbuf + read_pos - (nl + 1);
976 if (rest_size != 0) {
977 memmove(rbuf, nl + 1, rest_size);
978 }
979 read_pos = rest_size;
980 }
981 }
982 if (!keep_connection) {
983 fd.reset();
984 arg.sh.increment_conn(-1);
985 } else if (keep_connection > 1 && ++keepconn_count > keep_connection) {
986 keepconn_count = 0;
987 fd.reset();
988 arg.sh.increment_conn(-1);
989 }
990 const double tm2 = gettimeofday_double();
991 set_timing(tm2 - tm1);
992 sleep_if();
993 if (sched_flag) {
994 sched_yield();
995 }
996 }
997 if (dump) {
998 fprintf(stderr, "done\n");
999 }
1000}
1001
1002void
1003hstest_thread::sleep_if()
1004{
1005 if (arg.sh.usleep) {
1006 struct timespec ts = {
1007 arg.sh.usleep / 1000000,
1008 (arg.sh.usleep % 1000000) * 1000
1009 };
1010 nanosleep(&ts, 0);
1011 }
1012}
1013
1014void
1015hstest_thread::set_timing(double time_spent)
1016{
1017 response_min = std::min(response_min, time_spent);
1018 response_max = std::max(response_max, time_spent);
1019 response_sum += time_spent;
1020 if (op_success_count != 0) {
1021 response_avg = response_sum / op_success_count;
1022 }
1023}
1024
1025void
1026hstest_thread::test_11(int test_num)
1027{
1028 const int keep_connection = arg.sh.conf.get_int("keep_connection", 1);
1029 const int tablesize = arg.sh.conf.get_int("tablesize", 0);
1030 unsigned int seed = arg.id;
1031 seed ^= arg.sh.conf.get_int("seed_xor", 0);
1032 std::string err;
1033 hstcpcli_ptr cli;
1034 for (size_t i = 0; i < arg.sh.loop; ++i) {
1035 if (cli.get() == 0) {
1036 cli = hstcpcli_i::create(arg.sh.arg);
1037 cli->request_buf_open_index(0, "hstest", "hstest_table1", "", "v");
1038 /* pst_num, db, table, index, retflds */
1039 if (cli->request_send() != 0) {
1040 fprintf(stderr, "reuqest_send: %s\n", cli->get_error().c_str());
1041 return;
1042 }
1043 size_t num_flds = 0;
1044 if (cli->response_recv(num_flds) != 0) {
1045 fprintf(stderr, "reuqest_recv: %s\n", cli->get_error().c_str());
1046 return;
1047 }
1048 cli->response_buf_remove();
1049 }
1050 for (size_t j = 0; j < arg.sh.pipe; ++j) {
1051 char buf[256];
1052 int k = 0, v = 0, len = 0;
1053 {
1054 k = rand_r(&seed);
1055 v = rand_r(&seed); /* unused */
1056 if (tablesize != 0) {
1057 k &= tablesize;
1058 }
1059 len = snprintf(buf, sizeof(buf), "%d", k);
1060 }
1061 const string_ref key(buf, len);
1062 const string_ref op("=", 1);
1063 cli->request_buf_exec_generic(0, op, &key, 1, 1, 0, string_ref(), 0, 0);
1064 }
1065 if (cli->request_send() != 0) {
1066 fprintf(stderr, "reuqest_send: %s\n", cli->get_error().c_str());
1067 return;
1068 }
1069 size_t read_cnt = 0;
1070 for (size_t j = 0; j < arg.sh.pipe; ++j) {
1071 size_t num_flds = 0;
1072 if (cli->response_recv(num_flds) != 0) {
1073 fprintf(stderr, "reuqest_recv: %s\n", cli->get_error().c_str());
1074 return;
1075 }
1076 {
1077 ++read_cnt;
1078 ++io_success_count;
1079 arg.sh.increment_count();
1080 {
1081 ++op_success_count;
1082 }
1083 }
1084 cli->response_buf_remove();
1085 }
1086 if (!keep_connection) {
1087 cli.reset();
1088 }
1089 }
1090}
1091
1092void
1093hstest_thread::test_watch()
1094{
1095 const int timelimit = arg.sh.conf.get_int("timelimit", 0);
1096 const int timelimit_offset = timelimit / 2;
1097 int loop = 0;
1098 double t1 = 0, t2 = 0;
1099 size_t cnt_t1 = 0, cnt_t2 = 0;
1100 size_t prev_cnt = 0;
1101 double now_f = 0;
1102 while (true) {
1103 sleep(1);
1104 const size_t cnt = arg.sh.count;
1105 const size_t df = cnt - prev_cnt;
1106 prev_cnt = cnt;
1107 const double now_prev = now_f;
1108 now_f = gettimeofday_double();
1109 if (now_prev != 0) {
1110 const double rps = static_cast<double>(df) / (now_f - now_prev);
1111 fprintf(stderr, "now: %zu cntdiff: %zu tdiff: %f rps: %f\n",
1112 static_cast<size_t>(now_f), df, now_f - now_prev, rps);
1113 }
1114 if (timelimit != 0) {
1115 if (arg.sh.wait_conn == 0 || arg.sh.conn_count >= arg.sh.wait_conn) {
1116 ++loop;
1117 }
1118 if (loop == timelimit_offset) {
1119 t1 = gettimeofday_double();
1120 cnt_t1 = cnt;
1121 arg.sh.enable_timing = 1;
1122 fprintf(stderr, "start timing\n");
1123 } else if (loop == timelimit_offset + timelimit) {
1124 t2 = gettimeofday_double();
1125 cnt_t2 = cnt;
1126 const size_t cnt_diff = cnt_t2 - cnt_t1;
1127 const double tdiff = t2 - t1;
1128 const double qps = cnt_diff / (tdiff != 0 ? tdiff : 1);
1129 fprintf(stderr, "(%f: %zu, %f: %zu), %10.5f qps\n",
1130 t1, cnt_t1, t2, cnt_t2, qps);
1131 size_t keycnt = 0;
1132 for (int i = 0; i < arg.sh.keygen_size; ++i) {
1133 if (arg.sh.keygen[i]) {
1134 ++keycnt;
1135 }
1136 }
1137 fprintf(stderr, "keygen=%zu\n", keycnt);
1138 break;
1139 }
1140 }
1141 }
1142#if 0
1143 int loop = 0;
1144 double t1 = 0, t2 = 0;
1145 size_t cnt_t1 = 0, cnt_t2 = 0;
1146 size_t prev_cnt = 0;
1147 while (true) {
1148 sleep(1);
1149 const size_t cnt = arg.sh.count;
1150 const size_t df = cnt - prev_cnt;
1151 prev_cnt = cnt;
1152 const size_t now = time(0);
1153 fprintf(stderr, "%zu %zu\n", now, df);
1154 if (timelimit != 0) {
1155 ++loop;
1156 if (loop == timelimit_offset) {
1157 t1 = gettimeofday_double();
1158 cnt_t1 = cnt;
1159 } else if (loop == timelimit_offset + timelimit) {
1160 t2 = gettimeofday_double();
1161 cnt_t2 = cnt;
1162 const size_t cnt_diff = cnt_t2 - cnt_t1;
1163 const double tdiff = t2 - t1;
1164 const double qps = cnt_diff / (tdiff != 0 ? tdiff : 1);
1165 fprintf(stderr, "(%f: %zu, %f: %zu), %10.5f qps\n",
1166 t1, cnt_t1, t2, cnt_t2, qps);
1167 size_t keycnt = 0;
1168 for (int i = 0; i < arg.sh.keygen_size; ++i) {
1169 if (arg.sh.keygen[i]) {
1170 ++keycnt;
1171 }
1172 }
1173 fprintf(stderr, "keygen=%zu\n", keycnt);
1174 _exit(0);
1175 }
1176 }
1177 }
1178#endif
1179}
1180
1181void
1182hstest_thread::test_12(int test_num)
1183{
1184 /* NOTE: num_threads should be 1 */
1185 /* create table hstest
1186 * ( k varchar(255) not null, v varchar(255) not null, primary key(k))
1187 * engine = innodb; */
1188 mysqltest_thread_initobj initobj;
1189 auto_mysql db;
1190 std::string err;
1191 unsigned long long err_cnt = 0;
1192 unsigned long long query_cnt = 0;
1193 #if 0
1194 my_bool reconnect = 0;
1195 if (mysql_options(db, MYSQL_OPT_RECONNECT, &reconnect) != 0) {
1196 err = "mysql_options() failed";
1197 ++err_cnt;
1198 return;
1199 }
1200 #endif
1201 const std::string mysql_host = arg.sh.conf.get_str("host", "localhost");
1202 const int mysql_port = arg.sh.conf.get_int("mysqlport", 3306);
1203 const unsigned int num = arg.sh.loop;
1204 const size_t pipe = arg.sh.pipe;
1205 const std::string mysql_user = arg.sh.conf.get_str("mysqluser", "root");
1206 const std::string mysql_passwd = arg.sh.conf.get_str("mysqlpass", "");
1207 const std::string mysql_dbname = arg.sh.conf.get_str("db", "hstest");
1208 const int keep_connection = arg.sh.conf.get_int("keep_connection", 1);
1209 const int verbose = arg.sh.conf.get_int("verbose", 1);
1210 const int use_handler = arg.sh.conf.get_int("handler", 0);
1211 int connected = 0;
1212 unsigned int k = 0;
1213 string_buffer buf;
1214 for (unsigned int i = 0; i < num; ++i) {
1215 const int flags = 0;
1216 if (connected == 0 && !mysql_real_connect(db, mysql_host.c_str(),
1217 mysql_user.c_str(), mysql_user.empty() ? 0 : mysql_passwd.c_str(),
1218 mysql_dbname.c_str(), mysql_port, 0, flags)) {
1219 err = "failed to connect: " + std::string(mysql_error(db));
1220 if (verbose >= 1) {
1221 fprintf(stderr, "e=[%s]\n", err.c_str());
1222 }
1223 ++err_cnt;
1224 return;
1225 }
1226 int r = 0;
1227 if (connected == 0 && use_handler) {
1228 const char *const q = "handler hstest open";
1229 r = mysql_real_query(db, q, strlen(q));
1230 if (r != 0) {
1231 err = 1;
1232 }
1233 }
1234 connected = 1;
1235 std::string result_str;
1236 unsigned int err = 0;
1237 unsigned int num_flds = 0, num_affected_rows = 0;
1238 int got_data = 0;
1239 buf.clear();
1240 buf.append_literal("insert into hstest values ");
1241 for (size_t j = 0; j < pipe; ++j) {
1242 const unsigned int v = ~k;
1243 if (j != 0) {
1244 buf.append_literal(",");
1245 }
1246 char *wp = buf.make_space(64);
1247 int buf_query_len = snprintf(wp, 64, "('k%u', 'v%u')", k, v);
1248 buf.space_wrote(buf_query_len);
1249 ++k;
1250 }
1251 if (r == 0) {
1252 r = mysql_real_query(db, buf.begin(), buf.size());
1253 ++query_cnt;
1254 }
1255 if (r != 0) {
1256 err = 1;
1257 } else {
1258 auto_mysql_res res(db);
1259 if (res != 0) {
1260 if (verbose >= 0) {
1261 num_flds = mysql_num_fields(res);
1262 MYSQL_ROW row = 0;
1263 while ((row = mysql_fetch_row(res)) != 0) {
1264 got_data = 1;
1265 unsigned long *const lengths = mysql_fetch_lengths(res);
1266 if (verbose >= 2) {
1267 for (unsigned int i = 0; i < num_flds; ++i) {
1268 if (!result_str.empty()) {
1269 result_str += " ";
1270 }
1271 result_str += std::string(row[i], lengths[i]);
1272 }
1273 }
1274 }
1275 }
1276 } else {
1277 if (mysql_field_count(db) == 0) {
1278 num_affected_rows = mysql_affected_rows(db);
1279 } else {
1280 err = 1;
1281 }
1282 }
1283 }
1284 if (verbose >= 2 || (verbose >= 1 && err != 0)) {
1285 if (err) {
1286 ++err_cnt;
1287 const char *const errstr = mysql_error(db);
1288 fprintf(stderr, "e=[%s] a=%u q=[%s]\n", errstr,
1289 num_affected_rows, std::string(buf.begin(), buf.size()).c_str());
1290 } else {
1291 fprintf(stderr, "a=%u q=[%s] r=[%s]\n", num_affected_rows,
1292 std::string(buf.begin(), buf.size()).c_str(),
1293 result_str.c_str());
1294 }
1295 }
1296 if (err == 0) {
1297 ++io_success_count;
1298 if (num_affected_rows > 0 || got_data > 0) {
1299 ++op_success_count;
1300 }
1301 arg.sh.increment_count(pipe);
1302 }
1303 if (!keep_connection) {
1304 db.reset();
1305 connected = 0;
1306 }
1307 }
1308 if (verbose >= 1) {
1309 fprintf(stderr, "thread finished (error_count=%llu)\n", err_cnt);
1310 }
1311}
1312
1313void
1314hstest_thread::test_21(int num)
1315{
1316 /* fsync test */
1317 unsigned int id = arg.id;
1318 std::string err;
1319 #if 0
1320 if (socket_connect(fd, arg.sh.arg, err) != 0) {
1321 fprintf(stderr, "connect: %d %s\n", errno, strerror(errno));
1322 return;
1323 }
1324 #endif
1325 auto_file logfd;
1326 char fname[1024];
1327 snprintf(fname, sizeof(fname), "synctest_%u", id);
1328 int open_flags = O_WRONLY | O_CREAT | O_TRUNC | O_APPEND;
1329 logfd.reset(open(fname, open_flags, 0644));
1330 if (logfd.get() < 0) {
1331 fprintf(stderr, "open: %s: %d %s\n", fname, errno, strerror(errno));
1332 return;
1333 }
1334 char buf[1024];
1335 unsigned long long count = 0;
1336 while (true) {
1337 snprintf(buf, sizeof(buf), "%u %llu\n", id, count);
1338 const size_t len = strlen(buf);
1339 if (write(logfd.get(), buf, len) != (ssize_t)len) {
1340 fprintf(stderr, "write: %s: %d %s\n", fname, errno, strerror(errno));
1341 return;
1342 }
1343 #if 0
1344 if (write(fd.get(), buf, len) != (ssize_t)len) {
1345 fprintf(stderr, "write(sock): %d %s\n", errno, strerror(errno));
1346 return;
1347 }
1348 #endif
1349 if (fdatasync(logfd.get()) != 0) {
1350 fprintf(stderr, "fsync: %s: %d %s\n", fname, errno, strerror(errno));
1351 return;
1352 }
1353 ++count;
1354 ++op_success_count;
1355 arg.sh.increment_count();
1356 }
1357}
1358
1359void
1360hstest_thread::test_22(int num)
1361{
1362 /* dd if=/dev/zero of=dummy.dat bs=1024M count=100 */
1363 unsigned int id = arg.id;
1364 std::string err;
1365 auto_file filefd;
1366 char fname[1024];
1367 snprintf(fname, sizeof(fname), "dummy.dat");
1368 int open_flags = O_RDONLY | O_DIRECT;
1369 filefd.reset(open(fname, open_flags, 0644));
1370 if (filefd.get() < 0) {
1371 fprintf(stderr, "open: %s: %d %s\n", fname, errno, strerror(errno));
1372 return;
1373 }
1374 char buf_x[4096 * 2];
1375 char *const buf = (char *)(size_t(buf_x + 4096) / 4096 * 4096);
1376 unsigned long long count = 0;
1377 drand48_data randbuf;
1378 unsigned long long seed = time(0);
1379 seed *= 10;
1380 seed += id;
1381 srand48_r(seed, &randbuf);
1382 for (unsigned int i = 0; i < arg.sh.loop; ++i) {
1383 double kf = 0;
1384 drand48_r(&randbuf, &kf);
1385 kf *= (209715200 / 1);
1386 // fprintf(stderr, "v=%f\n", kf);
1387 off_t v = static_cast<off_t>(kf);
1388 v %= (209715200 / 1);
1389 v *= (512 * 1);
1390 const double tm1 = gettimeofday_double();
1391 const ssize_t r = pread(filefd.get(), buf, (512 * 1), v);
1392 const double tm2 = gettimeofday_double();
1393 if (r < 0) {
1394 fprintf(stderr, "pread: %s: %d %s\n", fname, errno, strerror(errno));
1395 return;
1396 }
1397 ++count;
1398 ++op_success_count;
1399 arg.sh.increment_count();
1400 set_timing(tm2 - tm1);
1401 }
1402}
1403
1404void
1405hstest_thread::operator ()()
1406{
1407 if (arg.watch_flag) {
1408 return test_watch();
The diff has been truncated for viewing.

Subscribers

People subscribed via source and target branches