Merge lp:~percona-dev/percona-xtradb-cluster/5.5.28 into lp:~percona-core/percona-xtradb-cluster/5.5.24

Proposed by Ignacio Nin
Status: Work in progress
Proposed branch: lp:~percona-dev/percona-xtradb-cluster/5.5.28
Merge into: lp:~percona-core/percona-xtradb-cluster/5.5.24
Diff against target: 127920 lines (has conflicts)
Conflict adding file doc-pxc/source/release-notes/Percona-XtraDB-Cluster-5.5.28.rst.  Moved existing file to doc-pxc/source/release-notes/Percona-XtraDB-Cluster-5.5.28.rst.moved.
To merge this branch: bzr merge lp:~percona-dev/percona-xtradb-cluster/5.5.28
Reviewer Review Type Date Requested Status
Vadim Tkachenko Needs Fixing
Review via email: mp+137396@code.launchpad.net

Description of the change

Changes for release 5.5.28

To post a comment you must log in.
Revision history for this message
Vadim Tkachenko (vadim-tk) wrote :

merge says
"Diff against target: 114271 lines (has conflicts)
Conflict adding file doc-pxc/source/release-notes/Percona-XtraDB-Cluster-5.5.28.rst. Moved existing file to doc-pxc/source/release-notes/Percona-XtraDB-Cluster-5.5.28.rst.moved"

review: Needs Fixing
374. By Vadim Tkachenko

fixed release date in release notes

375. By Vadim Tkachenko

merged lp:percona-server rev 384

376. By Vadim Tkachenko

merged lp:~codership/codership-mysql/5.5-23 rev 3838

Unmerged revisions

376. By Vadim Tkachenko

merged lp:~codership/codership-mysql/5.5-23 rev 3838

375. By Vadim Tkachenko

merged lp:percona-server rev 384

374. By Vadim Tkachenko

fixed release date in release notes

373. By Ignacio Nin

In build-dpkg.sh, updated the dch(1) change so it sets the correct
distribution to use in the repository. Not having it explicitely
forced resulted in UNRELEASED being chosen in quantal, due to a
change in the default in dch(1) (see the --release-heuristic
explanation in the man page of dch(1)).

372. By Ignacio Nin

Merge fix from 5.5

371. By Ignacio Nin

Use the wsrep revision in the server wsrep version for the binary builder

370. By Hrvoje Matijakovic

* merged lp:~hrvojem/percona-xtradb-cluster/rn-5.5.28-update

369. By Ignacio Nin

Use the wsrep revision in the server wsrep version.

Instead of using our branch's revno for the wsrep revision, use upstream's.
To this end add a WSREP-REVISION file to be updated when wsrep is merged.

368. By Vadim Tkachenko

merged lp:codership-mysql rev3821

367. By Vadim Tkachenko

merged lp:codership-mysql rev3820

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
<
=== modified file '.bzrignore'
--- .bzrignore 2012-08-21 05:55:28 +0000
+++ .bzrignore 2012-12-22 02:45:40 +0000
@@ -1,4 +1,3 @@
1build1build
2.pc2.pc
3Makefile.in3Makefile.in
4percona-theme
54
=== added directory 'HandlerSocket-Plugin-for-MySQL'
=== removed directory 'HandlerSocket-Plugin-for-MySQL'
=== added file 'HandlerSocket-Plugin-for-MySQL/AUTHORS'
--- HandlerSocket-Plugin-for-MySQL/AUTHORS 1970-01-01 00:00:00 +0000
+++ HandlerSocket-Plugin-for-MySQL/AUTHORS 2012-12-22 02:45:40 +0000
@@ -0,0 +1,22 @@
1Akira Higuchi (https://github.com/ahiguti)
2 - developed HanderSocket plugin, libhsclient, and perl-Net-HandlerSocket
3
4Yoshinori Matsunobu (https://github.com/yoshinorim)
5 - introduced autotools, added support for MySQL 5.5.6, added statistics
6 variables
7
8Jeff Hodges (https://github.com/jmhodges)
9 - fixed some autotools scripts
10
11Toru Yamaguchi (https://github.com/zigorou)
12 - ported to MacOS X
13
14Moriyoshi Koizumi (https://github.com/moriyoshi)
15 - fixed some autotools scripts
16
17takeda-at (https://github.com/takada-at)
18 - added simple authorization function
19
20WheresWardy (https://github.com/WheresWardy)
21 - added authentication functions to libhsclient
22
023
=== removed file 'HandlerSocket-Plugin-for-MySQL/AUTHORS'
--- HandlerSocket-Plugin-for-MySQL/AUTHORS 2011-01-10 13:39:35 +0000
+++ HandlerSocket-Plugin-for-MySQL/AUTHORS 1970-01-01 00:00:00 +0000
@@ -1,19 +0,0 @@
1Akira Higuchi (https://github.com/ahiguti)
2 - developed HanderSocket plugin, libhsclient, and perl-Net-HandlerSocket
3
4Yoshinori Matsunobu (https://github.com/yoshinorim)
5 - introduced autotools, added support for MySQL 5.5.6, added statistics
6 variables
7
8Jeff Hodges (https://github.com/jmhodges)
9 - fixed some autotools scripts
10
11Toru Yamaguchi (https://github.com/zigorou)
12 - ported to MacOS X
13
14Moriyoshi Koizumi (https://github.com/moriyoshi)
15 - fixed some autotools scripts
16
17takeda-at (https://github.com/takada-at)
18 - added simple authorization function
19
200
=== added file 'HandlerSocket-Plugin-for-MySQL/COPYING'
--- HandlerSocket-Plugin-for-MySQL/COPYING 1970-01-01 00:00:00 +0000
+++ HandlerSocket-Plugin-for-MySQL/COPYING 2012-12-22 02:45:40 +0000
@@ -0,0 +1,30 @@
1-----------------------------------------------------------------------------
2HandlerSocket plugin for MySQL
3
4 Copyright (c) 2010 DeNA Co.,Ltd.
5 All rights reserved.
6
7 Redistribution and use in source and binary forms, with or without
8 modification, are permitted provided that the following conditions are met:
9
10 * Redistributions of source code must retain the above copyright
11 notice, this list of conditions and the following disclaimer.
12 * Redistributions in binary form must reproduce the above copyright
13 notice, this list of conditions and the following disclaimer in the
14 documentation and/or other materials provided with the distribution.
15 * Neither the name of DeNA Co.,Ltd. nor the names of its contributors
16 may be used to endorse or promote products derived from this software
17 without specific prior written permission.
18
19 THIS SOFTWARE IS PROVIDED BY DeNA Co.,Ltd. "AS IS" AND ANY EXPRESS OR
20 IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
21 MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO
22 EVENT SHALL DeNA Co.,Ltd. BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
23 SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
24 PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS;
25 OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
26 WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
27 OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
28 ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
29
30
031
=== removed file 'HandlerSocket-Plugin-for-MySQL/COPYING'
--- HandlerSocket-Plugin-for-MySQL/COPYING 2011-01-10 13:39:35 +0000
+++ HandlerSocket-Plugin-for-MySQL/COPYING 1970-01-01 00:00:00 +0000
@@ -1,30 +0,0 @@
1-----------------------------------------------------------------------------
2HandlerSocket plugin for MySQL
3
4 Copyright (c) 2010 DeNA Co.,Ltd.
5 All rights reserved.
6
7 Redistribution and use in source and binary forms, with or without
8 modification, are permitted provided that the following conditions are met:
9
10 * Redistributions of source code must retain the above copyright
11 notice, this list of conditions and the following disclaimer.
12 * Redistributions in binary form must reproduce the above copyright
13 notice, this list of conditions and the following disclaimer in the
14 documentation and/or other materials provided with the distribution.
15 * Neither the name of DeNA Co.,Ltd. nor the names of its contributors
16 may be used to endorse or promote products derived from this software
17 without specific prior written permission.
18
19 THIS SOFTWARE IS PROVIDED BY DeNA Co.,Ltd. "AS IS" AND ANY EXPRESS OR
20 IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
21 MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO
22 EVENT SHALL DeNA Co.,Ltd. BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
23 SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
24 PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS;
25 OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
26 WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
27 OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
28 ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
29
30
310
=== added file 'HandlerSocket-Plugin-for-MySQL/ChangeLog'
--- HandlerSocket-Plugin-for-MySQL/ChangeLog 1970-01-01 00:00:00 +0000
+++ HandlerSocket-Plugin-for-MySQL/ChangeLog 2012-12-22 02:45:40 +0000
@@ -0,0 +1,15 @@
11.1.0 - 2011-12-29
2 * 1.1.0
3
41.0.6 - 2010-10-29
5 * Changed build instruction (autoreconf/configure/make), removed auto-generated files (Contributed by jmhodges)
6 *
7
81.0.5 - 2010-10-18
9 * Changed build procedures (using typical configure/make)
10 * Supported 5.5.6
11 * Added status variables
12
131.0.4 - 2010-08-15
14 * Initial public release
15
016
=== removed file 'HandlerSocket-Plugin-for-MySQL/ChangeLog'
--- HandlerSocket-Plugin-for-MySQL/ChangeLog 2011-01-10 13:39:35 +0000
+++ HandlerSocket-Plugin-for-MySQL/ChangeLog 1970-01-01 00:00:00 +0000
@@ -1,12 +0,0 @@
11.0.6 - 2010-10-29
2 * Changed build instruction (autoreconf/configure/make), removed auto-generated files (Contributed by jmhodges)
3 *
4
51.0.5 - 2010-10-18
6 * Changed build procedures (using typical configure/make)
7 * Supported 5.5.6
8 * Added status variables
9
101.0.4 - 2010-08-15
11 * Initial public release
12
130
=== added file 'HandlerSocket-Plugin-for-MySQL/Makefile.am'
--- HandlerSocket-Plugin-for-MySQL/Makefile.am 1970-01-01 00:00:00 +0000
+++ HandlerSocket-Plugin-for-MySQL/Makefile.am 2012-12-22 02:45:40 +0000
@@ -0,0 +1,87 @@
1
2ACLOCAL_AMFLAGS = -I m4
3
4SUBDIRS = @HANDLERSOCKET_SUBDIRS@
5
6perl:
7 cd perl-Net-HandlerSocket && perl Makefile.PL && make
8
9install_perl:
10 cd perl-Net-HandlerSocket && make install
11
12rpms: rpm_cli rpm_perl rpm_c
13
14rpm_dir:
15 - mkdir dist
16 - mkdir dist/BUILD dist/RPMS dist/SOURCES dist/SPECS dist/SRPMS
17
18rpm_cli: clean_cli rpm_dir
19 sed -e "s/HANDLERSOCKET_VERSION/$(VERSION)/" \
20 libhsclient/libhsclient.spec.template \
21 > libhsclient/libhsclient.spec
22 tar cvfz dist/libhsclient.tar.gz libhsclient
23 rpmbuild --define "_topdir `pwd`/dist" -ta \
24 dist/libhsclient.tar.gz
25
26rpm_perl: clean_perl rpm_dir
27 sed -e "s/HANDLERSOCKET_VERSION/$(VERSION)/" \
28 perl-Net-HandlerSocket/perl-Net-HandlerSocket.spec.template \
29 > perl-Net-HandlerSocket/perl-Net-HandlerSocket.spec
30 cd perl-Net-HandlerSocket && perl Makefile.PL && make clean && \
31 rm -f Makefile.old
32 tar cvfz dist/perl-Net-HandlerSocket.tar.gz perl-Net-HandlerSocket
33 rpmbuild --define "_topdir `pwd`/dist" -ta \
34 dist/perl-Net-HandlerSocket.tar.gz
35
36rpm_c: clean_c rpm_dir
37 sed -e "s/HANDLERSOCKET_VERSION/$(VERSION)/" \
38 handlersocket/handlersocket.spec.template \
39 > handlersocket/handlersocket.spec
40 sed -e "s|HANDLERSOCKET_MYSQL_INC|$(MYSQL_CFLAGS) $(MYSQL_INC)|" \
41 -e "s|HANDLERSOCKET_MYSQL_LIB|$(MYSQL_LIB)|" \
42 handlersocket/Makefile.plain.template \
43 > handlersocket/Makefile.plain
44 tar cvfz dist/handlersocket.tar.gz handlersocket
45 rpmbuild --define "_topdir `pwd`/dist" -ta \
46 dist/handlersocket.tar.gz
47
48install_rpm_pl:
49 - sudo rpm -e perl-Net-HandlerSocket
50 - sudo rpm -e perl-Net-HandlerSocket-debuginfo
51 make clean
52 make rpm_perl
53 - sudo rpm -U dist/RPMS/*/perl*.rpm
54
55installrpms:
56 - sudo rpm -e handlersocket
57 - sudo rpm -e handlersocket-debuginfo
58 - sudo rpm -e perl-Net-HandlerSocket
59 - sudo rpm -e perl-Net-HandlerSocket-debuginfo
60 - sudo rpm -e libhsclient
61 - sudo rpm -e libhsclient-debuginfo
62 make clean
63 make rpm_cli
64 - sudo rpm -U dist/RPMS/*/libhsclient*.rpm
65 make clean
66 make rpm_perl
67 - sudo rpm -U dist/RPMS/*/perl*.rpm
68 make clean
69 make rpm_c
70 - sudo rpm -U dist/RPMS/*/handlersocket*.rpm
71
72clean_cli:
73 cd libhsclient && make clean
74 cd client && make clean
75
76clean_perl:
77 cd perl-Net-HandlerSocket && perl Makefile.PL && make clean && \
78 rm -f Makefile.old
79
80clean_c:
81 cd handlersocket && make clean
82
83clean_all: clean_cli clean_perl clean_c
84 cd regtest && make clean
85 rm -rf dist/*/*
86 rm -f dist/*.tar.gz
87
088
=== removed file 'HandlerSocket-Plugin-for-MySQL/Makefile.am'
--- HandlerSocket-Plugin-for-MySQL/Makefile.am 2011-01-10 13:39:35 +0000
+++ HandlerSocket-Plugin-for-MySQL/Makefile.am 1970-01-01 00:00:00 +0000
@@ -1,87 +0,0 @@
1
2ACLOCAL_AMFLAGS = -I m4
3
4SUBDIRS = @HANDLERSOCKET_SUBDIRS@
5
6perl:
7 cd perl-Net-HandlerSocket && perl Makefile.PL && make
8
9install_perl:
10 cd perl-Net-HandlerSocket && make install
11
12rpms: rpm_cli rpm_perl rpm_c
13
14rpm_dir:
15 - mkdir dist
16 - mkdir dist/BUILD dist/RPMS dist/SOURCES dist/SPECS dist/SRPMS
17
18rpm_cli: clean_cli rpm_dir
19 sed -e "s/HANDLERSOCKET_VERSION/$(VERSION)/" \
20 libhsclient/libhsclient.spec.template \
21 > libhsclient/libhsclient.spec
22 tar cvfz dist/libhsclient.tar.gz libhsclient
23 rpmbuild --define "_topdir `pwd`/dist" -ta \
24 dist/libhsclient.tar.gz
25
26rpm_perl: clean_perl rpm_dir
27 sed -e "s/HANDLERSOCKET_VERSION/$(VERSION)/" \
28 perl-Net-HandlerSocket/perl-Net-HandlerSocket.spec.template \
29 > perl-Net-HandlerSocket/perl-Net-HandlerSocket.spec
30 cd perl-Net-HandlerSocket && perl Makefile.PL && make clean && \
31 rm -f Makefile.old
32 tar cvfz dist/perl-Net-HandlerSocket.tar.gz perl-Net-HandlerSocket
33 rpmbuild --define "_topdir `pwd`/dist" -ta \
34 dist/perl-Net-HandlerSocket.tar.gz
35
36rpm_c: clean_c rpm_dir
37 sed -e "s/HANDLERSOCKET_VERSION/$(VERSION)/" \
38 handlersocket/handlersocket.spec.template \
39 > handlersocket/handlersocket.spec
40 sed -e "s|HANDLERSOCKET_MYSQL_INC|$(MYSQL_CFLAGS) $(MYSQL_INC)|" \
41 -e "s|HANDLERSOCKET_MYSQL_LIB|$(MYSQL_LIB)|" \
42 handlersocket/Makefile.plain.template \
43 > handlersocket/Makefile.plain
44 tar cvfz dist/handlersocket.tar.gz handlersocket
45 rpmbuild --define "_topdir `pwd`/dist" -ta \
46 dist/handlersocket.tar.gz
47
48install_rpm_pl:
49 - sudo rpm -e perl-Net-HandlerSocket
50 - sudo rpm -e perl-Net-HandlerSocket-debuginfo
51 make clean
52 make rpm_perl
53 - sudo rpm -U dist/RPMS/*/perl*.rpm
54
55installrpms:
56 - sudo rpm -e handlersocket
57 - sudo rpm -e handlersocket-debuginfo
58 - sudo rpm -e perl-Net-HandlerSocket
59 - sudo rpm -e perl-Net-HandlerSocket-debuginfo
60 - sudo rpm -e libhsclient
61 - sudo rpm -e libhsclient-debuginfo
62 make clean
63 make rpm_cli
64 - sudo rpm -U dist/RPMS/*/libhsclient*.rpm
65 make clean
66 make rpm_perl
67 - sudo rpm -U dist/RPMS/*/perl*.rpm
68 make clean
69 make rpm_c
70 - sudo rpm -U dist/RPMS/*/handlersocket*.rpm
71
72clean_cli:
73 cd libhsclient && make clean
74 cd client && make clean
75
76clean_perl:
77 cd perl-Net-HandlerSocket && perl Makefile.PL && make clean && \
78 rm -f Makefile.old
79
80clean_c:
81 cd handlersocket && make clean
82
83clean_all: clean_cli clean_perl clean_c
84 cd regtest && make clean
85 rm -rf dist/*/*
86 rm -f dist/*.tar.gz
87
880
=== added file 'HandlerSocket-Plugin-for-MySQL/README'
--- HandlerSocket-Plugin-for-MySQL/README 1970-01-01 00:00:00 +0000
+++ HandlerSocket-Plugin-for-MySQL/README 2012-12-22 02:45:40 +0000
@@ -0,0 +1,78 @@
1
2-----------------------------------------------------------------------------
3HandlerSocket plugin for MySQL
4
5Copyright (c) 2010 DeNA Co.,Ltd.
6All rights reserved.
7
8Redistribution and use in source and binary forms, with or without
9modification, are permitted provided that the following conditions are met:
10
11 * Redistributions of source code must retain the above copyright
12 notice, this list of conditions and the following disclaimer.
13 * Redistributions in binary form must reproduce the above copyright
14 notice, this list of conditions and the following disclaimer in the
15 documentation and/or other materials provided with the distribution.
16 * Neither the name of DeNA Co.,Ltd. nor the names of its contributors
17 may be used to endorse or promote products derived from this software
18 without specific prior written permission.
19
20THIS SOFTWARE IS PROVIDED BY DeNA Co.,Ltd. "AS IS" AND ANY EXPRESS OR
21IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
22MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO
23EVENT SHALL DeNA Co.,Ltd. BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
24SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
25PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS;
26OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
27WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
28OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
29ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
30
31
32-----------------------------------------------------------------------------
33About HandlerSocket
34
35HandlerSocket is a NoSQL plugin for MySQL. It works as a daemon inside the
36mysqld process, accept tcp connections, and execute requests from clients.
37HandlerSocket does not support SQL queries. Instead, it supports simple CRUD
38operations on tables.
39
40Because of the following reasons, HandlerSocket is much faster than the
41mysqld/libmysql pair in some circumstances:
42
43 - HandlerSocket manipulates data without parsing SQL, which causes less
44 CPU usage.
45 - HandlerSocket reads many requests from clients and executes their
46 requests in bulk, which causes less CPU and disk usage.
47 - HandlerSocket client/server protocol is more compact than the
48 mysql/libmysql pair, which causes less network usage.
49
50The current version of HandlerSocket only works with GNU/Linux. The source
51archive of HandlerSocket includes a C++ and a Perl client libraries.
52Here is a list of client libraries for other languages:
53
54 - PHP
55 http://openpear.org/package/Net_HandlerSocket
56 http://github.com/tz-lom/HSPHP
57 http://code.google.com/p/php-handlersocket/
58 - Java
59 http://code.google.com/p/hs4j/
60 http://code.google.com/p/handlersocketforjava/
61 - Python
62 http://pypi.python.org/pypi/python-handler-socket
63 https://code.launchpad.net/~songofacandy/+junk/pyhandlersocket
64 - Ruby
65 https://github.com/winebarrel/ruby-handlersocket
66 https://github.com/miyucy/handlersocket
67 - JavaScript
68 https://github.com/koichik/node-handlersocket
69 - Scala
70 https://github.com/fujohnwang/hs2client
71 - Haskell
72 https://github.com/wuxb45/HandlerSocket-Haskell-Client
73
74The home of HandlerSocket is here:
75 https://github.com/DeNADev/HandlerSocket-Plugin-for-MySQL
76
77More documents are available in docs-en/ and docs-ja/ directories.
78
079
=== removed file 'HandlerSocket-Plugin-for-MySQL/README'
--- HandlerSocket-Plugin-for-MySQL/README 2011-01-10 13:39:35 +0000
+++ HandlerSocket-Plugin-for-MySQL/README 1970-01-01 00:00:00 +0000
@@ -1,76 +0,0 @@
1
2-----------------------------------------------------------------------------
3HandlerSocket plugin for MySQL
4
5Copyright (c) 2010 DeNA Co.,Ltd.
6All rights reserved.
7
8Redistribution and use in source and binary forms, with or without
9modification, are permitted provided that the following conditions are met:
10
11 * Redistributions of source code must retain the above copyright
12 notice, this list of conditions and the following disclaimer.
13 * Redistributions in binary form must reproduce the above copyright
14 notice, this list of conditions and the following disclaimer in the
15 documentation and/or other materials provided with the distribution.
16 * Neither the name of DeNA Co.,Ltd. nor the names of its contributors
17 may be used to endorse or promote products derived from this software
18 without specific prior written permission.
19
20THIS SOFTWARE IS PROVIDED BY DeNA Co.,Ltd. "AS IS" AND ANY EXPRESS OR
21IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
22MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO
23EVENT SHALL DeNA Co.,Ltd. BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
24SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
25PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS;
26OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
27WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
28OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
29ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
30
31
32-----------------------------------------------------------------------------
33About HandlerSocket
34
35HandlerSocket is a NoSQL plugin for MySQL. It works as a daemon inside the
36mysqld process, accept tcp connections, and execute requests from clients.
37HandlerSocket does not support SQL queries. Instead, it supports simple CRUD
38operations on tables.
39
40Because of the following reasons, HandlerSocket is much faster than the
41mysqld/libmysql pair in some circumstances:
42
43 - HandlerSocket manipulates data without parsing SQL, which causes less
44 CPU usage.
45 - HandlerSocket reads many requests from clients and executes their
46 requests in bulk, which causes less CPU and disk usage.
47 - HandlerSocket client/server protocol is more compact than the
48 mysql/libmysql pair, which causes less network usage.
49
50The current version of HandlerSocket only works with GNU/Linux. The source
51archive of HandlerSocket includes a C++ and a Perl client libraries.
52Here is a list of client libraries for other languages:
53
54 - PHP
55 http://openpear.org/package/Net_HandlerSocket
56 http://github.com/tz-lom/HSPHP
57 http://code.google.com/p/php-handlersocket/
58 - Java
59 http://code.google.com/p/hs4j/
60 http://code.google.com/p/handlersocketforjava/
61 - Python
62 http://pypi.python.org/pypi/python-handler-socket
63 https://code.launchpad.net/~songofacandy/+junk/pyhandlersocket
64 - Ruby
65 https://github.com/winebarrel/ruby-handlersocket
66 https://github.com/miyucy/handlersocket
67 - JavaScript
68 https://github.com/koichik/node-handlersocket
69 - Scala
70 https://github.com/fujohnwang/hs2client
71
72The home of HandlerSocket is here:
73 https://github.com/ahiguti/HandlerSocket-Plugin-for-MySQL
74
75More documents are available in docs-en/ and docs-ja/ directories.
76
770
=== added file 'HandlerSocket-Plugin-for-MySQL/autogen.sh'
--- HandlerSocket-Plugin-for-MySQL/autogen.sh 1970-01-01 00:00:00 +0000
+++ HandlerSocket-Plugin-for-MySQL/autogen.sh 2012-12-22 02:45:40 +0000
@@ -0,0 +1,117 @@
1#!/bin/sh
2
3warn() {
4echo -e "\tWARNING: $@" 1>&2
5}
6
7# init
8
9LIBTOOLIZE=libtoolize
10ACLOCAL=aclocal
11AUTOCONF=autoconf
12AUTOHEADER=autoheader
13AUTOMAKE=automake
14
15case `uname -s` in
16Darwin)
17LIBTOOLIZE=glibtoolize
18;;
19FreeBSD)
20ACLOCAL_ARGS="$ACLOCAL_ARGS -I /usr/local/share/aclocal/"
21;;
22esac
23
24
25# libtoolize
26echo "Searching libtoolize..."
27if [ `which $LIBTOOLIZE` ] ; then
28echo -e "\tFOUND: libtoolize -> $LIBTOOLIZE"
29else
30warn "Cannot Found libtoolize... input libtool command"
31 read LIBTOOLIZE
32 LIBTOOLIZE=`which $LIBTOOLIZE`
33 if [ `which $LIBTOOLIZE` ] ; then
34echo -e "\tSET: libtoolize -> $LIBTOOLIZE"
35 else
36warn "$LIBTOOLIZE: Command not found."
37 exit 1;
38 fi
39fi
40
41# aclocal
42echo "Searching aclocal..."
43if [ `which $ACLOCAL` ] ; then
44echo -e "\tFOUND: aclocal -> $ACLOCAL"
45else
46warn "Cannot Found aclocal... input aclocal command"
47 read ACLOCAL
48 ACLOCAL=`which $ACLOCAL`
49 if [ `which $ACLOCAL` ] ; then
50echo -e "\tSET: aclocal -> $ACLOCAL"
51 else
52warn "$ACLOCAL: Command not found."
53 exit 1;
54 fi
55fi
56
57# automake
58echo "Searching automake..."
59if [ `which $AUTOMAKE` ] ; then
60echo -e "\tFOUND: automake -> $AUTOMAKE"
61else
62warn "Cannot Found automake... input automake command"
63 read AUTOMAKE
64 ACLOCAL=`which $AUTOMAKE`
65 if [ `which $AUTOMAKE` ] ; then
66echo -e "\tSET: automake -> $AUTOMAKE"
67 else
68warn "$AUTOMAKE: Command not found."
69 exit 1;
70 fi
71fi
72
73# autoheader
74echo "Searching autoheader..."
75if [ `which $AUTOHEADER` ] ; then
76echo -e "\tFOUND: autoheader -> $AUTOHEADER"
77else
78warn "Cannot Found autoheader... input autoheader command"
79 read AUTOHEADER
80 ACLOCAL=`which $AUTOHEADER`
81 if [ `which $AUTOHEADER` ] ; then
82echo -e "\tSET: autoheader -> $AUTOHEADER"
83 else
84warn "$AUTOHEADER: Command not found."
85 exit 1;
86 fi
87fi
88
89# autoconf
90echo "Searching autoconf..."
91if [ `which $AUTOCONF` ] ; then
92echo -e "\tFOUND: autoconf -> $AUTOCONF"
93else
94warn "Cannot Found autoconf... input autoconf command"
95 read AUTOCONF
96 ACLOCAL=`which $AUTOCONF`
97 if [ `which $AUTOCONF` ] ; then
98echo -e "\tSET: autoconf -> $AUTOCONF"
99 else
100warn "$AUTOCONF: Command not found."
101 exit 1;
102 fi
103fi
104
105echo "Running libtoolize ..."
106$LIBTOOLIZE --force --copy
107echo "Running aclocal ..."
108$ACLOCAL ${ACLOCAL_ARGS} -I .
109echo "Running autoheader..."
110$AUTOHEADER
111echo "Running automake ..."
112$AUTOMAKE --add-missing --copy
113echo "Running autoconf ..."
114$AUTOCONF
115
116mkdir -p m4
117
0118
=== removed file 'HandlerSocket-Plugin-for-MySQL/autogen.sh'
--- HandlerSocket-Plugin-for-MySQL/autogen.sh 2011-04-19 12:57:00 +0000
+++ HandlerSocket-Plugin-for-MySQL/autogen.sh 1970-01-01 00:00:00 +0000
@@ -1,117 +0,0 @@
1#!/bin/sh
2
3warn() {
4echo -e "\tWARNING: $@" 1>&2
5}
6
7# init
8
9LIBTOOLIZE=libtoolize
10ACLOCAL=aclocal
11AUTOCONF=autoconf
12AUTOHEADER=autoheader
13AUTOMAKE=automake
14
15case `uname -s` in
16Darwin)
17LIBTOOLIZE=glibtoolize
18;;
19FreeBSD)
20ACLOCAL_ARGS="$ACLOCAL_ARGS -I /usr/local/share/aclocal/"
21;;
22esac
23
24
25# libtoolize
26echo "Searching libtoolize..."
27if [ `which $LIBTOOLIZE` ] ; then
28echo -e "\tFOUND: libtoolize -> $LIBTOOLIZE"
29else
30warn "Cannot Found libtoolize... input libtool command"
31 read LIBTOOLIZE
32 LIBTOOLIZE=`which $LIBTOOLIZE`
33 if [ `which $LIBTOOLIZE` ] ; then
34echo -e "\tSET: libtoolize -> $LIBTOOLIZE"
35 else
36warn "$LIBTOOLIZE: Command not found."
37 exit 1;
38 fi
39fi
40
41# aclocal
42echo "Searching aclocal..."
43if [ `which $ACLOCAL` ] ; then
44echo -e "\tFOUND: aclocal -> $ACLOCAL"
45else
46warn "Cannot Found aclocal... input aclocal command"
47 read ACLOCAL
48 ACLOCAL=`which $ACLOCAL`
49 if [ `which $ACLOCAL` ] ; then
50echo -e "\tSET: aclocal -> $ACLOCAL"
51 else
52warn "$ACLOCAL: Command not found."
53 exit 1;
54 fi
55fi
56
57# automake
58echo "Searching automake..."
59if [ `which $AUTOMAKE` ] ; then
60echo -e "\tFOUND: automake -> $AUTOMAKE"
61else
62warn "Cannot Found automake... input automake command"
63 read AUTOMAKE
64 ACLOCAL=`which $AUTOMAKE`
65 if [ `which $AUTOMAKE` ] ; then
66echo -e "\tSET: automake -> $AUTOMAKE"
67 else
68warn "$AUTOMAKE: Command not found."
69 exit 1;
70 fi
71fi
72
73# autoheader
74echo "Searching autoheader..."
75if [ `which $AUTOHEADER` ] ; then
76echo -e "\tFOUND: autoheader -> $AUTOHEADER"
77else
78warn "Cannot Found autoheader... input autoheader command"
79 read AUTOHEADER
80 ACLOCAL=`which $AUTOHEADER`
81 if [ `which $AUTOHEADER` ] ; then
82echo -e "\tSET: autoheader -> $AUTOHEADER"
83 else
84warn "$AUTOHEADER: Command not found."
85 exit 1;
86 fi
87fi
88
89# autoconf
90echo "Searching autoconf..."
91if [ `which $AUTOCONF` ] ; then
92echo -e "\tFOUND: autoconf -> $AUTOCONF"
93else
94warn "Cannot Found autoconf... input autoconf command"
95 read AUTOCONF
96 ACLOCAL=`which $AUTOCONF`
97 if [ `which $AUTOCONF` ] ; then
98echo -e "\tSET: autoconf -> $AUTOCONF"
99 else
100warn "$AUTOCONF: Command not found."
101 exit 1;
102 fi
103fi
104
105echo "Running libtoolize ..."
106$LIBTOOLIZE --force --copy
107echo "Running aclocal ..."
108$ACLOCAL ${ACLOCAL_ARGS} -I .
109echo "Running autoheader..."
110$AUTOHEADER
111echo "Running automake ..."
112$AUTOMAKE --add-missing --copy
113echo "Running autoconf ..."
114$AUTOCONF
115
116mkdir -p m4
117
1180
=== added directory 'HandlerSocket-Plugin-for-MySQL/client'
=== removed directory 'HandlerSocket-Plugin-for-MySQL/client'
=== added file 'HandlerSocket-Plugin-for-MySQL/client/Makefile.am'
--- HandlerSocket-Plugin-for-MySQL/client/Makefile.am 1970-01-01 00:00:00 +0000
+++ HandlerSocket-Plugin-for-MySQL/client/Makefile.am 2012-12-22 02:45:40 +0000
@@ -0,0 +1,24 @@
1AM_INCLUDES= -I../libhsclient
2bin_PROGRAMS=hsclient
3hsclient_SOURCES= hsclient.cpp
4hsclient_LDFLAGS= -static -L../libhsclient -lhsclient
5hsclient_CXXFLAGS= $(AM_INCLUDES)
6
7hstest: hstest.o
8 $(CXX) $(CXXFLAGS) $(LFLAGS) hstest.o \
9 -L../libhsclient/.libs -lhsclient $$(mysql_config --libs_r) \
10 -o hstest
11
12hstest.o: hstest.cpp
13 $(CXX) $(CXXFLAGS) $(AM_INCLUDES) $$(mysql_config --include) \
14 -c hstest.cpp
15
16hslongrun: hslongrun.o
17 $(CXX) $(CXXFLAGS) $(LFLAGS) hslongrun.o \
18 -L../libhsclient/.libs -lhsclient $$(mysql_config --libs_r) \
19 -o hslongrun
20
21hslongrun.o: hslongrun.cpp
22 $(CXX) $(CXXFLAGS) $(AM_INCLUDES) $$(mysql_config --include) \
23 -c hslongrun.cpp
24
025
=== removed file 'HandlerSocket-Plugin-for-MySQL/client/Makefile.am'
--- HandlerSocket-Plugin-for-MySQL/client/Makefile.am 2011-04-19 12:57:00 +0000
+++ HandlerSocket-Plugin-for-MySQL/client/Makefile.am 1970-01-01 00:00:00 +0000
@@ -1,24 +0,0 @@
1AM_INCLUDES= -I../libhsclient
2bin_PROGRAMS=hsclient
3hsclient_SOURCES= hsclient.cpp
4hsclient_LDFLAGS= -static -L../libhsclient -lhsclient
5hsclient_CXXFLAGS= $(AM_INCLUDES)
6
7hstest: hstest.o
8 $(CXX) $(CXXFLAGS) $(LFLAGS) hstest.o \
9 -L../libhsclient/.libs -lhsclient $$(mysql_config --libs_r) \
10 -o hstest
11
12hstest.o: hstest.cpp
13 $(CXX) $(CXXFLAGS) $(AM_INCLUDES) $$(mysql_config --include) \
14 -c hstest.cpp
15
16hslongrun: hslongrun.o
17 $(CXX) $(CXXFLAGS) $(LFLAGS) hslongrun.o \
18 -L../libhsclient/.libs -lhsclient $$(mysql_config --libs_r) \
19 -o hslongrun
20
21hslongrun.o: hslongrun.cpp
22 $(CXX) $(CXXFLAGS) $(AM_INCLUDES) $$(mysql_config --include) \
23 -c hslongrun.cpp
24
250
=== added file 'HandlerSocket-Plugin-for-MySQL/client/hsclient.cpp'
--- HandlerSocket-Plugin-for-MySQL/client/hsclient.cpp 1970-01-01 00:00:00 +0000
+++ HandlerSocket-Plugin-for-MySQL/client/hsclient.cpp 2012-12-22 02:45:40 +0000
@@ -0,0 +1,88 @@
1
2// vim:sw=2:ai
3
4#include "hstcpcli.hpp"
5#include "string_util.hpp"
6
7namespace dena {
8
9int
10hstcpcli_main(int argc, char **argv)
11{
12 config conf;
13 parse_args(argc, argv, conf);
14 socket_args sockargs;
15 sockargs.set(conf);
16 hstcpcli_ptr cli = hstcpcli_i::create(sockargs);
17 const std::string dbname = conf.get_str("dbname", "hstest");
18 const std::string table = conf.get_str("table", "hstest_table1");
19 const std::string index = conf.get_str("index", "PRIMARY");
20 const std::string fields = conf.get_str("fields", "k,v");
21 const int limit = conf.get_int("limit", 0);
22 const int skip = conf.get_int("skip", 0);
23 std::vector<std::string> keys;
24 std::vector<string_ref> keyrefs;
25 size_t num_keys = 0;
26 while (true) {
27 const std::string conf_key = std::string("k") + to_stdstring(num_keys);
28 const std::string k = conf.get_str(conf_key, "");
29 const std::string kx = conf.get_str(conf_key, "x");
30 if (k.empty() && kx == "x") {
31 break;
32 }
33 ++num_keys;
34 keys.push_back(k);
35 }
36 for (size_t i = 0; i < keys.size(); ++i) {
37 const string_ref ref(keys[i].data(), keys[i].size());
38 keyrefs.push_back(ref);
39 }
40 const std::string op = conf.get_str("op", "=");
41 const string_ref op_ref(op.data(), op.size());
42 cli->request_buf_open_index(0, dbname.c_str(), table.c_str(),
43 index.c_str(), fields.c_str());
44 cli->request_buf_exec_generic(0, op_ref, num_keys == 0 ? 0 : &keyrefs[0],
45 num_keys, limit, skip, string_ref(), 0, 0);
46 int code = 0;
47 size_t numflds = 0;
48 do {
49 if (cli->request_send() != 0) {
50 fprintf(stderr, "request_send: %s\n", cli->get_error().c_str());
51 break;
52 }
53 if ((code = cli->response_recv(numflds)) != 0) {
54 fprintf(stderr, "response_recv: %s\n", cli->get_error().c_str());
55 break;
56 }
57 } while (false);
58 cli->response_buf_remove();
59 do {
60 if ((code = cli->response_recv(numflds)) != 0) {
61 fprintf(stderr, "response_recv: %s\n", cli->get_error().c_str());
62 break;
63 }
64 while (true) {
65 const string_ref *const row = cli->get_next_row();
66 if (row == 0) {
67 break;
68 }
69 printf("REC:");
70 for (size_t i = 0; i < numflds; ++i) {
71 const std::string val(row[i].begin(), row[i].size());
72 printf(" %s", val.c_str());
73 }
74 printf("\n");
75 }
76 } while (false);
77 cli->response_buf_remove();
78 return 0;
79}
80
81};
82
83int
84main(int argc, char **argv)
85{
86 return dena::hstcpcli_main(argc, argv);
87}
88
089
=== removed file 'HandlerSocket-Plugin-for-MySQL/client/hsclient.cpp'
--- HandlerSocket-Plugin-for-MySQL/client/hsclient.cpp 2011-01-10 13:39:35 +0000
+++ HandlerSocket-Plugin-for-MySQL/client/hsclient.cpp 1970-01-01 00:00:00 +0000
@@ -1,88 +0,0 @@
1
2// vim:sw=2:ai
3
4#include "hstcpcli.hpp"
5#include "string_util.hpp"
6
7namespace dena {
8
9int
10hstcpcli_main(int argc, char **argv)
11{
12 config conf;
13 parse_args(argc, argv, conf);
14 socket_args sockargs;
15 sockargs.set(conf);
16 hstcpcli_ptr cli = hstcpcli_i::create(sockargs);
17 const std::string dbname = conf.get_str("dbname", "hstest");
18 const std::string table = conf.get_str("table", "hstest_table1");
19 const std::string index = conf.get_str("index", "PRIMARY");
20 const std::string fields = conf.get_str("fields", "k,v");
21 const int limit = conf.get_int("limit", 0);
22 const int skip = conf.get_int("skip", 0);
23 std::vector<std::string> keys;
24 std::vector<string_ref> keyrefs;
25 size_t num_keys = 0;
26 while (true) {
27 const std::string conf_key = std::string("k") + to_stdstring(num_keys);
28 const std::string k = conf.get_str(conf_key, "");
29 const std::string kx = conf.get_str(conf_key, "x");
30 if (k.empty() && kx == "x") {
31 break;
32 }
33 ++num_keys;
34 keys.push_back(k);
35 }
36 for (size_t i = 0; i < keys.size(); ++i) {
37 const string_ref ref(keys[i].data(), keys[i].size());
38 keyrefs.push_back(ref);
39 }
40 const std::string op = conf.get_str("op", "=");
41 const string_ref op_ref(op.data(), op.size());
42 cli->request_buf_open_index(0, dbname.c_str(), table.c_str(),
43 index.c_str(), fields.c_str());
44 cli->request_buf_exec_generic(0, op_ref, num_keys == 0 ? 0 : &keyrefs[0],
45 num_keys, limit, skip, string_ref(), 0, 0);
46 int code = 0;
47 size_t numflds = 0;
48 do {
49 if (cli->request_send() != 0) {
50 fprintf(stderr, "request_send: %s\n", cli->get_error().c_str());
51 break;
52 }
53 if ((code = cli->response_recv(numflds)) != 0) {
54 fprintf(stderr, "response_recv: %s\n", cli->get_error().c_str());
55 break;
56 }
57 } while (false);
58 cli->response_buf_remove();
59 do {
60 if ((code = cli->response_recv(numflds)) != 0) {
61 fprintf(stderr, "response_recv: %s\n", cli->get_error().c_str());
62 break;
63 }
64 while (true) {
65 const string_ref *const row = cli->get_next_row();
66 if (row == 0) {
67 break;
68 }
69 printf("REC:");
70 for (size_t i = 0; i < numflds; ++i) {
71 const std::string val(row[i].begin(), row[i].size());
72 printf(" %s", val.c_str());
73 }
74 printf("\n");
75 }
76 } while (false);
77 cli->response_buf_remove();
78 return 0;
79}
80
81};
82
83int
84main(int argc, char **argv)
85{
86 return dena::hstcpcli_main(argc, argv);
87}
88
890
=== added file 'HandlerSocket-Plugin-for-MySQL/client/hslongrun.cpp'
--- HandlerSocket-Plugin-for-MySQL/client/hslongrun.cpp 1970-01-01 00:00:00 +0000
+++ HandlerSocket-Plugin-for-MySQL/client/hslongrun.cpp 2012-12-22 02:45:40 +0000
@@ -0,0 +1,1041 @@
1
2// vim:sw=2:ai
3
4#include <signal.h>
5#include <sys/time.h>
6#include <stdio.h>
7#include <string.h>
8#include <vector>
9#include <map>
10#include <stdlib.h>
11#include <memory>
12#include <errno.h>
13#include <mysql.h>
14#include <time.h>
15#include <sys/types.h>
16#include <sys/stat.h>
17#include <fcntl.h>
18
19#include "util.hpp"
20#include "auto_ptrcontainer.hpp"
21#include "socket.hpp"
22#include "hstcpcli.hpp"
23#include "string_util.hpp"
24#include "mutex.hpp"
25
26namespace dena {
27
28struct auto_mysql : private noncopyable {
29 auto_mysql() : db(0) {
30 reset();
31 }
32 ~auto_mysql() {
33 if (db) {
34 mysql_close(db);
35 }
36 }
37 void reset() {
38 if (db) {
39 mysql_close(db);
40 }
41 if ((db = mysql_init(0)) == 0) {
42 fatal_exit("failed to initialize mysql client");
43 }
44 }
45 operator MYSQL *() const { return db; }
46 private:
47 MYSQL *db;
48};
49
50struct auto_mysql_res : private noncopyable {
51 auto_mysql_res(MYSQL *db) {
52 res = mysql_store_result(db);
53 }
54 ~auto_mysql_res() {
55 if (res) {
56 mysql_free_result(res);
57 }
58 }
59 operator MYSQL_RES *() const { return res; }
60 private:
61 MYSQL_RES *res;
62};
63
64struct auto_mysql_stmt : private noncopyable {
65 auto_mysql_stmt(MYSQL *db) {
66 stmt = mysql_stmt_init(db);
67 }
68 ~auto_mysql_stmt() {
69 if (stmt) {
70 mysql_stmt_close(stmt);
71 }
72 }
73 operator MYSQL_STMT *() const { return stmt; }
74 private:
75 MYSQL_STMT *stmt;
76};
77
78double
79gettimeofday_double()
80{
81 struct timeval tv = { };
82 if (gettimeofday(&tv, 0) != 0) {
83 fatal_abort("gettimeofday");
84 }
85 return static_cast<double>(tv.tv_usec) / 1000000 + tv.tv_sec;
86}
87
88struct record_value {
89 mutex lock;
90 bool deleted;
91 bool unknown_state;
92 std::string key;
93 std::vector<std::string> values;
94 record_value() : deleted(true), unknown_state(false) { }
95};
96
97struct hs_longrun_shared {
98 config conf;
99 socket_args arg;
100 int verbose;
101 long num_threads;
102 int usleep;
103 volatile mutable int running;
104 auto_ptrcontainer< std::vector<record_value *> > records;
105 hs_longrun_shared() : verbose(0), num_threads(0), usleep(0), running(1) { }
106};
107
108struct thread_base {
109 thread_base() : need_join(false), stack_size(256 * 1024) { }
110 virtual ~thread_base() {
111 join();
112 }
113 virtual void run() = 0;
114 void start() {
115 if (!start_nothrow()) {
116 fatal_abort("thread::start");
117 }
118 }
119 bool start_nothrow() {
120 if (need_join) {
121 return need_join; /* true */
122 }
123 void *const arg = this;
124 pthread_attr_t attr;
125 if (pthread_attr_init(&attr) != 0) {
126 fatal_abort("pthread_attr_init");
127 }
128 if (pthread_attr_setstacksize(&attr, stack_size) != 0) {
129 fatal_abort("pthread_attr_setstacksize");
130 }
131 const int r = pthread_create(&thr, &attr, thread_main, arg);
132 if (pthread_attr_destroy(&attr) != 0) {
133 fatal_abort("pthread_attr_destroy");
134 }
135 if (r != 0) {
136 return need_join; /* false */
137 }
138 need_join = true;
139 return need_join; /* true */
140 }
141 void join() {
142 if (!need_join) {
143 return;
144 }
145 int e = 0;
146 if ((e = pthread_join(thr, 0)) != 0) {
147 fatal_abort("pthread_join");
148 }
149 need_join = false;
150 }
151 private:
152 static void *thread_main(void *arg) {
153 thread_base *p = static_cast<thread_base *>(arg);
154 p->run();
155 return 0;
156 }
157 private:
158 pthread_t thr;
159 bool need_join;
160 size_t stack_size;
161};
162
163struct hs_longrun_stat {
164 unsigned long long verify_error_count;
165 unsigned long long runtime_error_count;
166 unsigned long long unknown_count;
167 unsigned long long success_count;
168 hs_longrun_stat()
169 : verify_error_count(0), runtime_error_count(0),
170 unknown_count(0), success_count(0) { }
171 void add(const hs_longrun_stat& x) {
172 verify_error_count += x.verify_error_count;
173 runtime_error_count += x.runtime_error_count;
174 unknown_count += x.unknown_count;
175 success_count += x.success_count;
176 }
177};
178
179struct hs_longrun_thread_base : public thread_base {
180 struct arg_type {
181 int id;
182 std::string worker_type;
183 char op;
184 int lock_flag;
185 const hs_longrun_shared& sh;
186 arg_type(int id, const std::string& worker_type, char op, int lock_flag,
187 const hs_longrun_shared& sh)
188 : id(id), worker_type(worker_type), op(op), lock_flag(lock_flag),
189 sh(sh) { }
190 };
191 arg_type arg;
192 hs_longrun_stat stat;
193 drand48_data randbuf;
194 unsigned int seed;
195 hs_longrun_thread_base(const arg_type& arg)
196 : arg(arg), seed(0) {
197 seed = time(0) + arg.id + 1;
198 srand48_r(seed, &randbuf);
199 }
200 virtual ~hs_longrun_thread_base() { }
201 virtual void run() = 0;
202 size_t rand_record() {
203 double v = 0;
204 drand48_r(&randbuf, &v);
205 const size_t sz = arg.sh.records.size();
206 size_t r = size_t(v * sz);
207 if (r >= sz) {
208 r = 0;
209 }
210 return r;
211 }
212 int verify_update(const std::string& k, const std::string& v1,
213 const std::string& v2, const std::string& v3, record_value& rec,
214 uint32_t num_rows, bool cur_unknown_state);
215 int verify_read(const std::string& k, uint32_t num_rows, uint32_t num_flds,
216 const std::string rrec[4], record_value& rec);
217 int verify_readnolock(const std::string& k, uint32_t num_rows,
218 uint32_t num_flds, const std::string rrec[4]);
219};
220
221int
222hs_longrun_thread_base::verify_update(const std::string& k,
223 const std::string& v1, const std::string& v2, const std::string& v3,
224 record_value& rec, uint32_t num_rows, bool cur_unknown_state)
225{
226 const bool op_success = num_rows == 1;
227 int ret = 0;
228 if (!rec.unknown_state) {
229 if (!rec.deleted && !op_success) {
230 ++stat.verify_error_count;
231 if (arg.sh.verbose > 0) {
232 fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
233 "unexpected_update_failure\n",
234 arg.worker_type.c_str(), arg.id, k.c_str());
235 }
236 ret = 1;
237 } else if (rec.deleted && op_success) {
238 ++stat.verify_error_count;
239 if (arg.sh.verbose > 0) {
240 fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
241 "unexpected_update_success\n",
242 arg.worker_type.c_str(), arg.id, k.c_str());
243 }
244 ret = 1;
245 }
246 }
247 if (op_success) {
248 rec.values.resize(4);
249 rec.values[0] = k;
250 rec.values[1] = v1;
251 rec.values[2] = v2;
252 rec.values[3] = v3;
253 if (ret == 0 && !rec.unknown_state) {
254 ++stat.success_count;
255 }
256 }
257 rec.unknown_state = cur_unknown_state;
258 if (arg.sh.verbose >= 100 && ret == 0) {
259 fprintf(stderr, "%s %s %s %s %s\n", arg.worker_type.c_str(),
260 k.c_str(), v1.c_str(), v2.c_str(), v3.c_str());
261 }
262 return ret;
263}
264
265int
266hs_longrun_thread_base::verify_read(const std::string& k,
267 uint32_t num_rows, uint32_t num_flds, const std::string rrec[4],
268 record_value& rec)
269{
270 const bool op_success = num_rows != 0;
271 int ret = 0;
272 if (!rec.unknown_state) {
273 if (!rec.deleted && !op_success) {
274 ++stat.verify_error_count;
275 if (arg.sh.verbose > 0) {
276 fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
277 "unexpected_read_failure\n",
278 arg.worker_type.c_str(), arg.id, k.c_str());
279 }
280 ret = 1;
281 } else if (rec.deleted && op_success) {
282 ++stat.verify_error_count;
283 if (arg.sh.verbose > 0) {
284 fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
285 "unexpected_read_success\n",
286 arg.worker_type.c_str(), arg.id, k.c_str());
287 }
288 ret = 1;
289 } else if (num_flds != 4) {
290 ++stat.verify_error_count;
291 if (arg.sh.verbose > 0) {
292 fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
293 "unexpected_read_fldnum %d\n",
294 arg.worker_type.c_str(), arg.id, k.c_str(),
295 static_cast<int>(num_flds));
296 }
297 ret = 1;
298 } else if (rec.deleted) {
299 /* nothing to verify */
300 } else {
301 int diff = 0;
302 for (size_t i = 0; i < 4; ++i) {
303 if (rec.values[i] == rrec[i]) {
304 /* ok */
305 } else {
306 diff = 1;
307 }
308 }
309 if (diff) {
310 std::string mess;
311 for (size_t i = 0; i < 4; ++i) {
312 const std::string& expected = rec.values[i];
313 const std::string& val = rrec[i];
314 mess += " " + val + "/" + expected;
315 }
316 if (arg.sh.verbose > 0) {
317 fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
318 "unexpected_read_value %s\n",
319 arg.worker_type.c_str(), arg.id, k.c_str(), mess.c_str());
320 }
321 ret = 1;
322 }
323 }
324 }
325 if (arg.sh.verbose >= 100 && ret == 0) {
326 fprintf(stderr, "%s %s\n", arg.worker_type.c_str(), k.c_str());
327 }
328 if (ret == 0 && !rec.unknown_state) {
329 ++stat.success_count;
330 }
331 return ret;
332}
333
334int
335hs_longrun_thread_base::verify_readnolock(const std::string& k,
336 uint32_t num_rows, uint32_t num_flds, const std::string rrec[4])
337{
338 int ret = 0;
339 if (num_rows != 1 || num_flds != 4) {
340 ++stat.verify_error_count;
341 if (arg.sh.verbose > 0) {
342 fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
343 "unexpected_read_failure\n",
344 arg.worker_type.c_str(), arg.id, k.c_str());
345 }
346 ret = 1;
347 }
348 if (arg.sh.verbose >= 100 && ret == 0) {
349 fprintf(stderr, "%s -> %s %s %s %s %s\n", arg.worker_type.c_str(),
350 k.c_str(), rrec[0].c_str(), rrec[1].c_str(), rrec[2].c_str(),
351 rrec[3].c_str());
352 }
353 if (ret == 0) {
354 ++stat.success_count;
355 }
356 return ret;
357}
358
359struct hs_longrun_thread_hs : public hs_longrun_thread_base {
360 hs_longrun_thread_hs(const arg_type& arg)
361 : hs_longrun_thread_base(arg) { }
362 void run();
363 int check_hs_error(const char *mess, record_value *rec);
364 int op_insert(record_value& rec);
365 int op_delete(record_value& rec);
366 int op_update(record_value& rec);
367 int op_read(record_value& rec);
368 int op_readnolock(int k);
369 hstcpcli_ptr cli;
370 socket_args sockargs;
371};
372
373struct lock_guard : noncopyable {
374 lock_guard(mutex& mtx) : mtx(mtx) {
375 mtx.lock();
376 }
377 ~lock_guard() {
378 mtx.unlock();
379 }
380 mutex& mtx;
381};
382
383string_ref
384to_string_ref(const std::string& s)
385{
386 return string_ref(s.data(), s.size());
387}
388
389std::string
390to_string(const string_ref& s)
391{
392 return std::string(s.begin(), s.size());
393}
394
395void
396hs_longrun_thread_hs::run()
397{
398 config c = arg.sh.conf;
399 if (arg.op == 'R' || arg.op == 'N') {
400 c["port"] = to_stdstring(arg.sh.conf.get_int("hsport", 9998));
401 } else {
402 c["port"] = to_stdstring(arg.sh.conf.get_int("hsport_wr", 9999));
403 }
404 sockargs.set(c);
405
406 while (arg.sh.running) {
407 if (cli.get() == 0 || !cli->stable_point()) {
408 cli = hstcpcli_i::create(sockargs);
409 if (check_hs_error("connect", 0) != 0) {
410 cli.reset();
411 continue;
412 }
413 cli->request_buf_open_index(0, "hstestdb", "hstesttbl", "PRIMARY",
414 "k,v1,v2,v3", "k,v1,v2,v3");
415 cli->request_send();
416 if (check_hs_error("openindex_send", 0) != 0) {
417 cli.reset();
418 continue;
419 }
420 size_t num_flds = 0;
421 cli->response_recv(num_flds);
422 if (check_hs_error("openindex_recv", 0) != 0) {
423 cli.reset();
424 continue;
425 }
426 cli->response_buf_remove();
427 }
428 const size_t rec_id = rand_record();
429 if (arg.lock_flag) {
430 record_value& rec = *arg.sh.records[rec_id];
431 lock_guard g(rec.lock);
432 int e = 0;
433 switch (arg.op) {
434 case 'I':
435 e = op_insert(rec);
436 break;
437 case 'D':
438 e = op_delete(rec);
439 break;
440 case 'U':
441 e = op_update(rec);
442 break;
443 case 'R':
444 e = op_read(rec);
445 break;
446 default:
447 break;
448 }
449 } else {
450 int e = 0;
451 switch (arg.op) {
452 case 'N':
453 e = op_readnolock(rec_id);
454 break;
455 default:
456 break;
457 }
458 }
459 }
460}
461
462int
463hs_longrun_thread_hs::op_insert(record_value& rec)
464{
465 const std::string k = rec.key;
466 const std::string v1 = "iv1_" + k + "_" + to_stdstring(arg.id);
467 const std::string v2 = "iv2_" + k + "_" + to_stdstring(arg.id);
468 const std::string v3 = "iv3_" + k + "_" + to_stdstring(arg.id);
469 const string_ref op_ref("+", 1);
470 const string_ref op_args[4] = {
471 to_string_ref(k),
472 to_string_ref(v1),
473 to_string_ref(v2),
474 to_string_ref(v3)
475 };
476 cli->request_buf_exec_generic(0, op_ref, op_args, 4, 1, 0,
477 string_ref(), 0, 0, 0, 0);
478 cli->request_send();
479 if (check_hs_error("op_insert_send", &rec) != 0) { return 1; }
480 size_t numflds = 0;
481 cli->response_recv(numflds);
482 if (arg.sh.verbose > 10) {
483 const string_ref *row = cli->get_next_row();
484 fprintf(stderr, "HS op=+ errrcode=%d errmess=[%s]\n", cli->get_error_code(),
485 row ? to_string(row[0]).c_str() : "");
486 }
487 const bool op_success = cli->get_error_code() == 0;
488 int ret = 0;
489 if (!rec.unknown_state) {
490 if (rec.deleted && !op_success) {
491 ++stat.verify_error_count;
492 if (arg.sh.verbose > 0) {
493 fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
494 "unexpected_insert_failure\n",
495 arg.worker_type.c_str(), arg.id, k.c_str());
496 }
497 ret = 1;
498 } else if (!rec.deleted && op_success) {
499 ++stat.verify_error_count;
500 if (arg.sh.verbose > 0) {
501 fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
502 "unexpected_insert_success\n",
503 arg.worker_type.c_str(), arg.id, k.c_str());
504 }
505 ret = 1;
506 }
507 } else {
508 ++stat.unknown_count;
509 }
510 if (op_success) {
511 rec.values.resize(4);
512 rec.values[0] = k;
513 rec.values[1] = v1;
514 rec.values[2] = v2;
515 rec.values[3] = v3;
516 rec.deleted = false;
517 if (arg.sh.verbose >= 100 && ret == 0) {
518 fprintf(stderr, "HS_INSERT %s %s %s %s\n", k.c_str(), v1.c_str(),
519 v2.c_str(), v3.c_str());
520 }
521 if (ret == 0 && !rec.unknown_state) {
522 ++stat.success_count;
523 }
524 rec.unknown_state = false;
525 }
526 cli->response_buf_remove();
527 return ret;
528}
529
530int
531hs_longrun_thread_hs::op_delete(record_value& rec)
532{
533 const std::string k = rec.key;
534 const string_ref op_ref("=", 1);
535 const string_ref op_args[1] = {
536 to_string_ref(k),
537 };
538 const string_ref modop_ref("D", 1);
539 cli->request_buf_exec_generic(0, op_ref, op_args, 1, 1, 0,
540 modop_ref, 0, 0, 0, 0);
541 cli->request_send();
542 if (check_hs_error("op_delete_send", &rec) != 0) { return 1; }
543 size_t numflds = 0;
544 cli->response_recv(numflds);
545 if (check_hs_error("op_delete_recv", &rec) != 0) { return 1; }
546 const string_ref *row = cli->get_next_row();
547 const bool op_success = (numflds > 0 && row != 0 &&
548 to_string(row[0]) == "1");
549 int ret = 0;
550 if (!rec.unknown_state) {
551 if (!rec.deleted && !op_success) {
552 ++stat.verify_error_count;
553 if (arg.sh.verbose > 0) {
554 fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
555 "unexpected_delete_failure\n",
556 arg.worker_type.c_str(), arg.id, k.c_str());
557 }
558 ret = 1;
559 } else if (rec.deleted && op_success) {
560 ++stat.verify_error_count;
561 if (arg.sh.verbose > 0) {
562 fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
563 "unexpected_delete_success\n",
564 arg.worker_type.c_str(), arg.id, k.c_str());
565 }
566 ret = 1;
567 }
568 }
569 cli->response_buf_remove();
570 if (op_success) {
571 rec.deleted = true;
572 if (ret == 0 && !rec.unknown_state) {
573 ++stat.success_count;
574 }
575 rec.unknown_state = false;
576 }
577 if (arg.sh.verbose >= 100 && ret == 0) {
578 fprintf(stderr, "HS_DELETE %s\n", k.c_str());
579 }
580 return ret;
581}
582
583int
584hs_longrun_thread_hs::op_update(record_value& rec)
585{
586 const std::string k = rec.key;
587 const std::string v1 = "uv1_" + k + "_" + to_stdstring(arg.id);
588 const std::string v2 = "uv2_" + k + "_" + to_stdstring(arg.id);
589 const std::string v3 = "uv3_" + k + "_" + to_stdstring(arg.id);
590 const string_ref op_ref("=", 1);
591 const string_ref op_args[1] = {
592 to_string_ref(k),
593 };
594 const string_ref modop_ref("U", 1);
595 const string_ref modop_args[4] = {
596 to_string_ref(k),
597 to_string_ref(v1),
598 to_string_ref(v2),
599 to_string_ref(v3)
600 };
601 cli->request_buf_exec_generic(0, op_ref, op_args, 1, 1, 0,
602 modop_ref, modop_args, 4, 0, 0);
603 cli->request_send();
604 if (check_hs_error("op_update_send", &rec) != 0) { return 1; }
605 size_t numflds = 0;
606 cli->response_recv(numflds);
607 if (check_hs_error("op_update_recv", &rec) != 0) { return 1; }
608 const string_ref *row = cli->get_next_row();
609 uint32_t num_rows = row
610 ? atoi_uint32_nocheck(row[0].begin(), row[0].end()) : 0;
611 cli->response_buf_remove();
612 const bool cur_unknown_state = (num_rows == 1);
613 return verify_update(k, v1, v2, v3, rec, num_rows, cur_unknown_state);
614}
615
616int
617hs_longrun_thread_hs::op_read(record_value& rec)
618{
619 const std::string k = rec.key;
620 const string_ref op_ref("=", 1);
621 const string_ref op_args[1] = {
622 to_string_ref(k),
623 };
624 cli->request_buf_exec_generic(0, op_ref, op_args, 1, 1, 0,
625 string_ref(), 0, 0, 0, 0);
626 cli->request_send();
627 if (check_hs_error("op_read_send", 0) != 0) { return 1; }
628 size_t num_flds = 0;
629 size_t num_rows = 0;
630 cli->response_recv(num_flds);
631 if (check_hs_error("op_read_recv", 0) != 0) { return 1; }
632 const string_ref *row = cli->get_next_row();
633 std::string rrec[4];
634 if (row != 0 && num_flds == 4) {
635 for (int i = 0; i < 4; ++i) {
636 rrec[i] = to_string(row[i]);
637 }
638 ++num_rows;
639 }
640 row = cli->get_next_row();
641 if (row != 0) {
642 ++num_rows;
643 }
644 cli->response_buf_remove();
645 return verify_read(k, num_rows, num_flds, rrec, rec);
646}
647
648int
649hs_longrun_thread_hs::op_readnolock(int key)
650{
651 const std::string k = to_stdstring(key);
652 const string_ref op_ref("=", 1);
653 const string_ref op_args[1] = {
654 to_string_ref(k),
655 };
656 cli->request_buf_exec_generic(0, op_ref, op_args, 1, 1, 0,
657 string_ref(), 0, 0, 0, 0);
658 cli->request_send();
659 if (check_hs_error("op_read_send", 0) != 0) { return 1; }
660 size_t num_flds = 0;
661 size_t num_rows = 0;
662 cli->response_recv(num_flds);
663 if (check_hs_error("op_read_recv", 0) != 0) { return 1; }
664 const string_ref *row = cli->get_next_row();
665 std::string rrec[4];
666 if (row != 0 && num_flds == 4) {
667 for (int i = 0; i < 4; ++i) {
668 rrec[i] = to_string(row[i]);
669 }
670 ++num_rows;
671 }
672 row = cli->get_next_row();
673 if (row != 0) {
674 ++num_rows;
675 }
676 cli->response_buf_remove();
677 return verify_readnolock(k, num_rows, num_flds, rrec);
678}
679
680int
681hs_longrun_thread_hs::check_hs_error(const char *mess, record_value *rec)
682{
683 const int err = cli->get_error_code();
684 if (err == 0) {
685 return 0;
686 }
687 ++stat.runtime_error_count;
688 if (arg.sh.verbose > 0) {
689 const std::string estr = cli->get_error();
690 fprintf(stderr, "RUNTIME_ERROR: op=%c wid=%d %s: %d %s\n",
691 arg.op, arg.id, mess, err, estr.c_str());
692 }
693 if (rec) {
694 rec->unknown_state = true;
695 }
696 return 1;
697}
698
699struct hs_longrun_thread_my : public hs_longrun_thread_base {
700 hs_longrun_thread_my(const arg_type& arg)
701 : hs_longrun_thread_base(arg), connected(false) { }
702 void run();
703 void show_mysql_error(const char *mess, record_value *rec);
704 int op_insert(record_value& rec);
705 int op_delete(record_value& rec);
706 int op_update(record_value& rec);
707 int op_delins(record_value& rec);
708 int op_read(record_value& rec);
709 auto_mysql db;
710 bool connected;
711};
712
713void
714hs_longrun_thread_my::run()
715{
716 const std::string mysql_host = arg.sh.conf.get_str("host", "localhost");
717 const std::string mysql_user = arg.sh.conf.get_str("mysqluser", "root");
718 const std::string mysql_passwd = arg.sh.conf.get_str("mysqlpass", "");
719 const std::string mysql_dbname = "hstestdb";
720
721 while (arg.sh.running) {
722 if (!connected) {
723 if (!mysql_real_connect(db, mysql_host.c_str(), mysql_user.c_str(),
724 mysql_passwd.c_str(), mysql_dbname.c_str(), mysql_port, 0, 0)) {
725 show_mysql_error("mysql_real_connect", 0);
726 continue;
727 }
728 }
729 connected = true;
730 const size_t rec_id = rand_record();
731 record_value& rec = *arg.sh.records[rec_id];
732 lock_guard g(rec.lock);
733 int e = 0;
734 switch (arg.op) {
735 #if 0
736 case 'I':
737 e = op_insert(rec);
738 break;
739 case 'D':
740 e = op_delete(rec);
741 break;
742 case 'U':
743 e = op_update(rec);
744 break;
745 #endif
746 case 'T':
747 e = op_delins(rec);
748 break;
749 case 'R':
750 e = op_read(rec);
751 break;
752 default:
753 break;
754 }
755 }
756}
757
758int
759hs_longrun_thread_my::op_delins(record_value& rec)
760{
761 const std::string k = rec.key;
762 const std::string v1 = "div1_" + k + "_" + to_stdstring(arg.id);
763 const std::string v2 = "div2_" + k + "_" + to_stdstring(arg.id);
764 const std::string v3 = "div3_" + k + "_" + to_stdstring(arg.id);
765 int success = 0;
766 bool cur_unknown_state = false;
767 do {
768 char query[1024];
769 #if 1
770 if (mysql_query(db, "begin") != 0) {
771 if (arg.sh.verbose >= 20) {
772 fprintf(stderr, "mysql: e=[%s] q=[%s]\n", mysql_error(db), "begin");
773 }
774 break;
775 }
776 #endif
777 cur_unknown_state = true;
778 snprintf(query, 1024,
779 "delete from hstesttbl where k = '%s'", k.c_str());
780 if (mysql_query(db, query) != 0) {
781 if (arg.sh.verbose >= 20) {
782 fprintf(stderr, "mysql: e=[%s] q=[%s]\n", mysql_error(db), query);
783 }
784 break;
785 }
786 if (mysql_affected_rows(db) != 1) {
787 if (arg.sh.verbose >= 20) {
788 fprintf(stderr, "mysql: notfound: [%s]\n", query);
789 }
790 break;
791 }
792 snprintf(query, 1024,
793 "insert into hstesttbl values ('%s', '%s', '%s', '%s')",
794 k.c_str(), v1.c_str(), v2.c_str(), v3.c_str());
795 if (mysql_query(db, query) != 0) {
796 if (arg.sh.verbose >= 20) {
797 fprintf(stderr, "mysql: e=[%s] q=[%s]\n", mysql_error(db), query);
798 }
799 break;
800 }
801 #if 1
802 if (mysql_query(db, "commit") != 0) {
803 if (arg.sh.verbose >= 20) {
804 fprintf(stderr, "mysql: e=[%s] q=[%s]\n", mysql_error(db), "commit");
805 }
806 break;
807 }
808 #endif
809 success = true;
810 cur_unknown_state = false;
811 } while (false);
812 return verify_update(k, v1, v2, v3, rec, (success != 0), cur_unknown_state);
813}
814
815int
816hs_longrun_thread_my::op_read(record_value& rec)
817{
818 const std::string k = rec.key;
819 char query[1024] = { 0 };
820 const int len = snprintf(query, 1024,
821 "select k,v1,v2,v3 from hstesttbl where k='%s'", k.c_str());
822 const int r = mysql_real_query(db, query, len > 0 ? len : 0);
823 if (r != 0) {
824 show_mysql_error(query, 0);
825 return 1;
826 }
827 MYSQL_ROW row = 0;
828 unsigned long *lengths = 0;
829 unsigned int num_rows = 0;
830 unsigned int num_flds = 0;
831 auto_mysql_res res(db);
832 std::string rrec[4];
833 if (res != 0) {
834 num_flds = mysql_num_fields(res);
835 row = mysql_fetch_row(res);
836 if (row != 0) {
837 lengths = mysql_fetch_lengths(res);
838 if (num_flds == 4) {
839 for (int i = 0; i < 4; ++i) {
840 rrec[i] = std::string(row[i], lengths[i]);
841 }
842 }
843 ++num_rows;
844 row = mysql_fetch_row(res);
845 if (row != 0) {
846 ++num_rows;
847 }
848 }
849 }
850 return verify_read(k, num_rows, num_flds, rrec, rec);
851}
852
853void
854hs_longrun_thread_my::show_mysql_error(const char *mess, record_value *rec)
855{
856 ++stat.runtime_error_count;
857 if (arg.sh.verbose > 0) {
858 fprintf(stderr, "RUNTIME_ERROR: op=%c wid=%d [%s]: %s\n",
859 arg.op, arg.id, mess, mysql_error(db));
860 }
861 if (rec) {
862 rec->unknown_state = true;
863 }
864 db.reset();
865 connected = false;
866}
867
868void
869mysql_do(MYSQL *db, const char *query)
870{
871 if (mysql_real_query(db, query, strlen(query)) != 0) {
872 fprintf(stderr, "mysql: e=[%s] q=[%s]\n", mysql_error(db), query);
873 fatal_exit("mysql_do");
874 }
875}
876
877void
878hs_longrun_init_table(const config& conf, int num_prepare,
879 hs_longrun_shared& shared)
880{
881 const std::string mysql_host = conf.get_str("host", "localhost");
882 const std::string mysql_user = conf.get_str("mysqluser", "root");
883 const std::string mysql_passwd = conf.get_str("mysqlpass", "");
884 const std::string mysql_dbname = "";
885 auto_mysql db;
886 if (!mysql_real_connect(db, mysql_host.c_str(), mysql_user.c_str(),
887 mysql_passwd.c_str(), mysql_dbname.c_str(), mysql_port, 0, 0)) {
888 fprintf(stderr, "mysql: error=[%s]\n", mysql_error(db));
889 fatal_exit("hs_longrun_init_table");
890 }
891 mysql_do(db, "drop database if exists hstestdb");
892 mysql_do(db, "create database hstestdb");
893 mysql_do(db, "use hstestdb");
894 mysql_do(db,
895 "create table hstesttbl ("
896 "k int primary key,"
897 "v1 varchar(32) not null,"
898 "v2 varchar(32) not null,"
899 "v3 varchar(32) not null"
900 ") character set utf8 collate utf8_bin engine = innodb");
901 for (int i = 0; i < num_prepare; ++i) {
902 const std::string i_str = to_stdstring(i);
903 const std::string v1 = "pv1_" + i_str;
904 const std::string v2 = "pv2_" + i_str;
905 const std::string v3 = "pv3_" + i_str;
906 char buf[1024];
907 snprintf(buf, 1024, "insert into hstesttbl(k, v1, v2, v3) values"
908 "(%d, '%s', '%s', '%s')", i, v1.c_str(), v2.c_str(), v3.c_str());
909 mysql_do(db, buf);
910 record_value *rec = shared.records[i];
911 rec->key = i_str;
912 rec->values.resize(4);
913 rec->values[0] = i_str;
914 rec->values[1] = v1;
915 rec->values[2] = v2;
916 rec->values[3] = v3;
917 rec->deleted = false;
918 }
919}
920
921int
922hs_longrun_main(int argc, char **argv)
923{
924 hs_longrun_shared shared;
925 parse_args(argc, argv, shared.conf);
926 shared.conf["host"] = shared.conf.get_str("host", "localhost");
927 shared.verbose = shared.conf.get_int("verbose", 1);
928 const int table_size = shared.conf.get_int("table_size", 10000);
929 for (int i = 0; i < table_size; ++i) {
930 std::auto_ptr<record_value> rec(new record_value());
931 rec->key = to_stdstring(i);
932 shared.records.push_back_ptr(rec);
933 }
934 mysql_library_init(0, 0, 0);
935 const int duration = shared.conf.get_int("duration", 10);
936 const int num_hsinsert = shared.conf.get_int("num_hsinsert", 10);
937 const int num_hsdelete = shared.conf.get_int("num_hsdelete", 10);
938 const int num_hsupdate = shared.conf.get_int("num_hsupdate", 10);
939 const int num_hsread = shared.conf.get_int("num_hsread", 10);
940 const int num_myread = shared.conf.get_int("num_myread", 10);
941 const int num_mydelins = shared.conf.get_int("num_mydelins", 10);
942 int num_hsreadnolock = shared.conf.get_int("num_hsreadnolock", 10);
943 const bool always_filled = (num_hsinsert == 0 && num_hsdelete == 0);
944 if (!always_filled) {
945 num_hsreadnolock = 0;
946 }
947 hs_longrun_init_table(shared.conf, always_filled ? table_size : 0,
948 shared);
949 /* create worker threads */
950 static const struct thrtmpl_type {
951 const char *type; char op; int num; int hs; int lock;
952 } thrtmpl[] = {
953 { "hsinsert", 'I', num_hsinsert, 1, 1 },
954 { "hsdelete", 'D', num_hsdelete, 1, 1 },
955 { "hsupdate", 'U', num_hsupdate, 1, 1 },
956 { "hsread", 'R', num_hsread, 1, 1 },
957 { "hsreadnolock", 'N', num_hsreadnolock, 1, 0 },
958 { "myread", 'R', num_myread, 0, 1 },
959 { "mydelins", 'T', num_mydelins, 0, 1 },
960 };
961 typedef auto_ptrcontainer< std::vector<hs_longrun_thread_base *> > thrs_type;
962 thrs_type thrs;
963 for (size_t i = 0; i < sizeof(thrtmpl)/sizeof(thrtmpl[0]); ++i) {
964 const thrtmpl_type& e = thrtmpl[i];
965 for (int j = 0; j < e.num; ++j) {
966 int id = thrs.size();
967 const hs_longrun_thread_hs::arg_type arg(id, e.type, e.op, e.lock,
968 shared);
969 std::auto_ptr<hs_longrun_thread_base> thr;
970 if (e.hs) {
971 thr.reset(new hs_longrun_thread_hs(arg));
972 } else {
973 thr.reset(new hs_longrun_thread_my(arg));
974 }
975 thrs.push_back_ptr(thr);
976 }
977 }
978 shared.num_threads = thrs.size();
979 /* start threads */
980 fprintf(stderr, "START\n");
981 shared.running = 1;
982 for (size_t i = 0; i < thrs.size(); ++i) {
983 thrs[i]->start();
984 }
985 /* wait */
986 sleep(duration);
987 /* stop thread */
988 shared.running = 0;
989 for (size_t i = 0; i < thrs.size(); ++i) {
990 thrs[i]->join();
991 }
992 fprintf(stderr, "DONE\n");
993 /* summary */
994 typedef std::map<std::string, hs_longrun_stat> stat_map;
995 stat_map sm;
996 for (size_t i = 0; i < thrs.size(); ++i) {
997 hs_longrun_thread_base *const thr = thrs[i];
998 const std::string wt = thr->arg.worker_type;
999 hs_longrun_stat& v = sm[wt];
1000 v.add(thr->stat);
1001 }
1002 hs_longrun_stat total;
1003 for (stat_map::const_iterator i = sm.begin(); i != sm.end(); ++i) {
1004 if (i->second.verify_error_count != 0) {
1005 fprintf(stderr, "%s verify_error %llu\n", i->first.c_str(),
1006 i->second.verify_error_count);
1007 }
1008 if (i->second.runtime_error_count) {
1009 fprintf(stderr, "%s runtime_error %llu\n", i->first.c_str(),
1010 i->second.runtime_error_count);
1011 }
1012 if (i->second.unknown_count) {
1013 fprintf(stderr, "%s unknown %llu\n", i->first.c_str(),
1014 i->second.unknown_count);
1015 }
1016 fprintf(stderr, "%s success %llu\n", i->first.c_str(),
1017 i->second.success_count);
1018 total.add(i->second);
1019 }
1020 if (total.verify_error_count != 0) {
1021 fprintf(stderr, "TOTAL verify_error %llu\n", total.verify_error_count);
1022 }
1023 if (total.runtime_error_count != 0) {
1024 fprintf(stderr, "TOTAL runtime_error %llu\n", total.runtime_error_count);
1025 }
1026 if (total.unknown_count != 0) {
1027 fprintf(stderr, "TOTAL unknown %llu\n", total.unknown_count);
1028 }
1029 fprintf(stderr, "TOTAL success %llu\n", total.success_count);
1030 mysql_library_end();
1031 return 0;
1032}
1033
1034};
1035
1036int
1037main(int argc, char **argv)
1038{
1039 return dena::hs_longrun_main(argc, argv);
1040}
1041
01042
=== removed file 'HandlerSocket-Plugin-for-MySQL/client/hslongrun.cpp'
--- HandlerSocket-Plugin-for-MySQL/client/hslongrun.cpp 2011-04-19 12:57:00 +0000
+++ HandlerSocket-Plugin-for-MySQL/client/hslongrun.cpp 1970-01-01 00:00:00 +0000
@@ -1,1041 +0,0 @@
1
2// vim:sw=2:ai
3
4#include <signal.h>
5#include <sys/time.h>
6#include <stdio.h>
7#include <string.h>
8#include <vector>
9#include <map>
10#include <stdlib.h>
11#include <memory>
12#include <errno.h>
13#include <mysql.h>
14#include <time.h>
15#include <sys/types.h>
16#include <sys/stat.h>
17#include <fcntl.h>
18
19#include "util.hpp"
20#include "auto_ptrcontainer.hpp"
21#include "socket.hpp"
22#include "hstcpcli.hpp"
23#include "string_util.hpp"
24#include "mutex.hpp"
25
26namespace dena {
27
28struct auto_mysql : private noncopyable {
29 auto_mysql() : db(0) {
30 reset();
31 }
32 ~auto_mysql() {
33 if (db) {
34 mysql_close(db);
35 }
36 }
37 void reset() {
38 if (db) {
39 mysql_close(db);
40 }
41 if ((db = mysql_init(0)) == 0) {
42 fatal_exit("failed to initialize mysql client");
43 }
44 }
45 operator MYSQL *() const { return db; }
46 private:
47 MYSQL *db;
48};
49
50struct auto_mysql_res : private noncopyable {
51 auto_mysql_res(MYSQL *db) {
52 res = mysql_store_result(db);
53 }
54 ~auto_mysql_res() {
55 if (res) {
56 mysql_free_result(res);
57 }
58 }
59 operator MYSQL_RES *() const { return res; }
60 private:
61 MYSQL_RES *res;
62};
63
64struct auto_mysql_stmt : private noncopyable {
65 auto_mysql_stmt(MYSQL *db) {
66 stmt = mysql_stmt_init(db);
67 }
68 ~auto_mysql_stmt() {
69 if (stmt) {
70 mysql_stmt_close(stmt);
71 }
72 }
73 operator MYSQL_STMT *() const { return stmt; }
74 private:
75 MYSQL_STMT *stmt;
76};
77
78double
79gettimeofday_double()
80{
81 struct timeval tv = { };
82 if (gettimeofday(&tv, 0) != 0) {
83 fatal_abort("gettimeofday");
84 }
85 return static_cast<double>(tv.tv_usec) / 1000000 + tv.tv_sec;
86}
87
88struct record_value {
89 mutex lock;
90 bool deleted;
91 bool unknown_state;
92 std::string key;
93 std::vector<std::string> values;
94 record_value() : deleted(true), unknown_state(false) { }
95};
96
97struct hs_longrun_shared {
98 config conf;
99 socket_args arg;
100 int verbose;
101 long num_threads;
102 int usleep;
103 volatile mutable int running;
104 auto_ptrcontainer< std::vector<record_value *> > records;
105 hs_longrun_shared() : verbose(0), num_threads(0), usleep(0), running(1) { }
106};
107
108struct thread_base {
109 thread_base() : need_join(false), stack_size(256 * 1024) { }
110 virtual ~thread_base() {
111 join();
112 }
113 virtual void run() = 0;
114 void start() {
115 if (!start_nothrow()) {
116 fatal_abort("thread::start");
117 }
118 }
119 bool start_nothrow() {
120 if (need_join) {
121 return need_join; /* true */
122 }
123 void *const arg = this;
124 pthread_attr_t attr;
125 if (pthread_attr_init(&attr) != 0) {
126 fatal_abort("pthread_attr_init");
127 }
128 if (pthread_attr_setstacksize(&attr, stack_size) != 0) {
129 fatal_abort("pthread_attr_setstacksize");
130 }
131 const int r = pthread_create(&thr, &attr, thread_main, arg);
132 if (pthread_attr_destroy(&attr) != 0) {
133 fatal_abort("pthread_attr_destroy");
134 }
135 if (r != 0) {
136 return need_join; /* false */
137 }
138 need_join = true;
139 return need_join; /* true */
140 }
141 void join() {
142 if (!need_join) {
143 return;
144 }
145 int e = 0;
146 if ((e = pthread_join(thr, 0)) != 0) {
147 fatal_abort("pthread_join");
148 }
149 need_join = false;
150 }
151 private:
152 static void *thread_main(void *arg) {
153 thread_base *p = static_cast<thread_base *>(arg);
154 p->run();
155 return 0;
156 }
157 private:
158 pthread_t thr;
159 bool need_join;
160 size_t stack_size;
161};
162
163struct hs_longrun_stat {
164 unsigned long long verify_error_count;
165 unsigned long long runtime_error_count;
166 unsigned long long unknown_count;
167 unsigned long long success_count;
168 hs_longrun_stat()
169 : verify_error_count(0), runtime_error_count(0),
170 unknown_count(0), success_count(0) { }
171 void add(const hs_longrun_stat& x) {
172 verify_error_count += x.verify_error_count;
173 runtime_error_count += x.runtime_error_count;
174 unknown_count += x.unknown_count;
175 success_count += x.success_count;
176 }
177};
178
179struct hs_longrun_thread_base : public thread_base {
180 struct arg_type {
181 int id;
182 std::string worker_type;
183 char op;
184 int lock_flag;
185 const hs_longrun_shared& sh;
186 arg_type(int id, const std::string& worker_type, char op, int lock_flag,
187 const hs_longrun_shared& sh)
188 : id(id), worker_type(worker_type), op(op), lock_flag(lock_flag),
189 sh(sh) { }
190 };
191 arg_type arg;
192 hs_longrun_stat stat;
193 drand48_data randbuf;
194 unsigned int seed;
195 hs_longrun_thread_base(const arg_type& arg)
196 : arg(arg), seed(0) {
197 seed = time(0) + arg.id + 1;
198 srand48_r(seed, &randbuf);
199 }
200 virtual ~hs_longrun_thread_base() { }
201 virtual void run() = 0;
202 size_t rand_record() {
203 double v = 0;
204 drand48_r(&randbuf, &v);
205 const size_t sz = arg.sh.records.size();
206 size_t r = size_t(v * sz);
207 if (r >= sz) {
208 r = 0;
209 }
210 return r;
211 }
212 int verify_update(const std::string& k, const std::string& v1,
213 const std::string& v2, const std::string& v3, record_value& rec,
214 uint32_t num_rows, bool cur_unknown_state);
215 int verify_read(const std::string& k, uint32_t num_rows, uint32_t num_flds,
216 const std::string rrec[4], record_value& rec);
217 int verify_readnolock(const std::string& k, uint32_t num_rows,
218 uint32_t num_flds, const std::string rrec[4]);
219};
220
221int
222hs_longrun_thread_base::verify_update(const std::string& k,
223 const std::string& v1, const std::string& v2, const std::string& v3,
224 record_value& rec, uint32_t num_rows, bool cur_unknown_state)
225{
226 const bool op_success = num_rows == 1;
227 int ret = 0;
228 if (!rec.unknown_state) {
229 if (!rec.deleted && !op_success) {
230 ++stat.verify_error_count;
231 if (arg.sh.verbose > 0) {
232 fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
233 "unexpected_update_failure\n",
234 arg.worker_type.c_str(), arg.id, k.c_str());
235 }
236 ret = 1;
237 } else if (rec.deleted && op_success) {
238 ++stat.verify_error_count;
239 if (arg.sh.verbose > 0) {
240 fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
241 "unexpected_update_success\n",
242 arg.worker_type.c_str(), arg.id, k.c_str());
243 }
244 ret = 1;
245 }
246 }
247 if (op_success) {
248 rec.values.resize(4);
249 rec.values[0] = k;
250 rec.values[1] = v1;
251 rec.values[2] = v2;
252 rec.values[3] = v3;
253 if (ret == 0 && !rec.unknown_state) {
254 ++stat.success_count;
255 }
256 }
257 rec.unknown_state = cur_unknown_state;
258 if (arg.sh.verbose >= 100 && ret == 0) {
259 fprintf(stderr, "%s %s %s %s %s\n", arg.worker_type.c_str(),
260 k.c_str(), v1.c_str(), v2.c_str(), v3.c_str());
261 }
262 return ret;
263}
264
265int
266hs_longrun_thread_base::verify_read(const std::string& k,
267 uint32_t num_rows, uint32_t num_flds, const std::string rrec[4],
268 record_value& rec)
269{
270 const bool op_success = num_rows != 0;
271 int ret = 0;
272 if (!rec.unknown_state) {
273 if (!rec.deleted && !op_success) {
274 ++stat.verify_error_count;
275 if (arg.sh.verbose > 0) {
276 fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
277 "unexpected_read_failure\n",
278 arg.worker_type.c_str(), arg.id, k.c_str());
279 }
280 ret = 1;
281 } else if (rec.deleted && op_success) {
282 ++stat.verify_error_count;
283 if (arg.sh.verbose > 0) {
284 fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
285 "unexpected_read_success\n",
286 arg.worker_type.c_str(), arg.id, k.c_str());
287 }
288 ret = 1;
289 } else if (num_flds != 4) {
290 ++stat.verify_error_count;
291 if (arg.sh.verbose > 0) {
292 fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
293 "unexpected_read_fldnum %d\n",
294 arg.worker_type.c_str(), arg.id, k.c_str(),
295 static_cast<int>(num_flds));
296 }
297 ret = 1;
298 } else if (rec.deleted) {
299 /* nothing to verify */
300 } else {
301 int diff = 0;
302 for (size_t i = 0; i < 4; ++i) {
303 if (rec.values[i] == rrec[i]) {
304 /* ok */
305 } else {
306 diff = 1;
307 }
308 }
309 if (diff) {
310 std::string mess;
311 for (size_t i = 0; i < 4; ++i) {
312 const std::string& expected = rec.values[i];
313 const std::string& val = rrec[i];
314 mess += " " + val + "/" + expected;
315 }
316 if (arg.sh.verbose > 0) {
317 fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
318 "unexpected_read_value %s\n",
319 arg.worker_type.c_str(), arg.id, k.c_str(), mess.c_str());
320 }
321 ret = 1;
322 }
323 }
324 }
325 if (arg.sh.verbose >= 100 && ret == 0) {
326 fprintf(stderr, "%s %s\n", arg.worker_type.c_str(), k.c_str());
327 }
328 if (ret == 0 && !rec.unknown_state) {
329 ++stat.success_count;
330 }
331 return ret;
332}
333
334int
335hs_longrun_thread_base::verify_readnolock(const std::string& k,
336 uint32_t num_rows, uint32_t num_flds, const std::string rrec[4])
337{
338 int ret = 0;
339 if (num_rows != 1 || num_flds != 4) {
340 ++stat.verify_error_count;
341 if (arg.sh.verbose > 0) {
342 fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
343 "unexpected_read_failure\n",
344 arg.worker_type.c_str(), arg.id, k.c_str());
345 }
346 ret = 1;
347 }
348 if (arg.sh.verbose >= 100 && ret == 0) {
349 fprintf(stderr, "%s -> %s %s %s %s %s\n", arg.worker_type.c_str(),
350 k.c_str(), rrec[0].c_str(), rrec[1].c_str(), rrec[2].c_str(),
351 rrec[3].c_str());
352 }
353 if (ret == 0) {
354 ++stat.success_count;
355 }
356 return ret;
357}
358
359struct hs_longrun_thread_hs : public hs_longrun_thread_base {
360 hs_longrun_thread_hs(const arg_type& arg)
361 : hs_longrun_thread_base(arg) { }
362 void run();
363 int check_hs_error(const char *mess, record_value *rec);
364 int op_insert(record_value& rec);
365 int op_delete(record_value& rec);
366 int op_update(record_value& rec);
367 int op_read(record_value& rec);
368 int op_readnolock(int k);
369 hstcpcli_ptr cli;
370 socket_args sockargs;
371};
372
373struct lock_guard : noncopyable {
374 lock_guard(mutex& mtx) : mtx(mtx) {
375 mtx.lock();
376 }
377 ~lock_guard() {
378 mtx.unlock();
379 }
380 mutex& mtx;
381};
382
383string_ref
384to_string_ref(const std::string& s)
385{
386 return string_ref(s.data(), s.size());
387}
388
389std::string
390to_string(const string_ref& s)
391{
392 return std::string(s.begin(), s.size());
393}
394
395void
396hs_longrun_thread_hs::run()
397{
398 config c = arg.sh.conf;
399 if (arg.op == 'R' || arg.op == 'N') {
400 c["port"] = to_stdstring(arg.sh.conf.get_int("hsport", 9998));
401 } else {
402 c["port"] = to_stdstring(arg.sh.conf.get_int("hsport_wr", 9999));
403 }
404 sockargs.set(c);
405
406 while (arg.sh.running) {
407 if (cli.get() == 0 || !cli->stable_point()) {
408 cli = hstcpcli_i::create(sockargs);
409 if (check_hs_error("connect", 0) != 0) {
410 cli.reset();
411 continue;
412 }
413 cli->request_buf_open_index(0, "hstestdb", "hstesttbl", "PRIMARY",
414 "k,v1,v2,v3", "k,v1,v2,v3");
415 cli->request_send();
416 if (check_hs_error("openindex_send", 0) != 0) {
417 cli.reset();
418 continue;
419 }
420 size_t num_flds = 0;
421 cli->response_recv(num_flds);
422 if (check_hs_error("openindex_recv", 0) != 0) {
423 cli.reset();
424 continue;
425 }
426 cli->response_buf_remove();
427 }
428 const size_t rec_id = rand_record();
429 if (arg.lock_flag) {
430 record_value& rec = *arg.sh.records[rec_id];
431 lock_guard g(rec.lock);
432 int e = 0;
433 switch (arg.op) {
434 case 'I':
435 e = op_insert(rec);
436 break;
437 case 'D':
438 e = op_delete(rec);
439 break;
440 case 'U':
441 e = op_update(rec);
442 break;
443 case 'R':
444 e = op_read(rec);
445 break;
446 default:
447 break;
448 }
449 } else {
450 int e = 0;
451 switch (arg.op) {
452 case 'N':
453 e = op_readnolock(rec_id);
454 break;
455 default:
456 break;
457 }
458 }
459 }
460}
461
462int
463hs_longrun_thread_hs::op_insert(record_value& rec)
464{
465 const std::string k = rec.key;
466 const std::string v1 = "iv1_" + k + "_" + to_stdstring(arg.id);
467 const std::string v2 = "iv2_" + k + "_" + to_stdstring(arg.id);
468 const std::string v3 = "iv3_" + k + "_" + to_stdstring(arg.id);
469 const string_ref op_ref("+", 1);
470 const string_ref op_args[4] = {
471 to_string_ref(k),
472 to_string_ref(v1),
473 to_string_ref(v2),
474 to_string_ref(v3)
475 };
476 cli->request_buf_exec_generic(0, op_ref, op_args, 4, 1, 0,
477 string_ref(), 0, 0, 0, 0);
478 cli->request_send();
479 if (check_hs_error("op_insert_send", &rec) != 0) { return 1; }
480 size_t numflds = 0;
481 cli->response_recv(numflds);
482 if (arg.sh.verbose > 10) {
483 const string_ref *row = cli->get_next_row();
484 fprintf(stderr, "HS op=+ errrcode=%d errmess=[%s]\n", cli->get_error_code(),
485 row ? to_string(row[0]).c_str() : "");
486 }
487 const bool op_success = cli->get_error_code() == 0;
488 int ret = 0;
489 if (!rec.unknown_state) {
490 if (rec.deleted && !op_success) {
491 ++stat.verify_error_count;
492 if (arg.sh.verbose > 0) {
493 fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
494 "unexpected_insert_failure\n",
495 arg.worker_type.c_str(), arg.id, k.c_str());
496 }
497 ret = 1;
498 } else if (!rec.deleted && op_success) {
499 ++stat.verify_error_count;
500 if (arg.sh.verbose > 0) {
501 fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
502 "unexpected_insert_success\n",
503 arg.worker_type.c_str(), arg.id, k.c_str());
504 }
505 ret = 1;
506 }
507 } else {
508 ++stat.unknown_count;
509 }
510 if (op_success) {
511 rec.values.resize(4);
512 rec.values[0] = k;
513 rec.values[1] = v1;
514 rec.values[2] = v2;
515 rec.values[3] = v3;
516 rec.deleted = false;
517 if (arg.sh.verbose >= 100 && ret == 0) {
518 fprintf(stderr, "HS_INSERT %s %s %s %s\n", k.c_str(), v1.c_str(),
519 v2.c_str(), v3.c_str());
520 }
521 if (ret == 0 && !rec.unknown_state) {
522 ++stat.success_count;
523 }
524 rec.unknown_state = false;
525 }
526 cli->response_buf_remove();
527 return ret;
528}
529
530int
531hs_longrun_thread_hs::op_delete(record_value& rec)
532{
533 const std::string k = rec.key;
534 const string_ref op_ref("=", 1);
535 const string_ref op_args[1] = {
536 to_string_ref(k),
537 };
538 const string_ref modop_ref("D", 1);
539 cli->request_buf_exec_generic(0, op_ref, op_args, 1, 1, 0,
540 modop_ref, 0, 0, 0, 0);
541 cli->request_send();
542 if (check_hs_error("op_delete_send", &rec) != 0) { return 1; }
543 size_t numflds = 0;
544 cli->response_recv(numflds);
545 if (check_hs_error("op_delete_recv", &rec) != 0) { return 1; }
546 const string_ref *row = cli->get_next_row();
547 const bool op_success = (numflds > 0 && row != 0 &&
548 to_string(row[0]) == "1");
549 int ret = 0;
550 if (!rec.unknown_state) {
551 if (!rec.deleted && !op_success) {
552 ++stat.verify_error_count;
553 if (arg.sh.verbose > 0) {
554 fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
555 "unexpected_delete_failure\n",
556 arg.worker_type.c_str(), arg.id, k.c_str());
557 }
558 ret = 1;
559 } else if (rec.deleted && op_success) {
560 ++stat.verify_error_count;
561 if (arg.sh.verbose > 0) {
562 fprintf(stderr, "VERIFY_ERROR: %s wid=%d k=%s "
563 "unexpected_delete_success\n",
564 arg.worker_type.c_str(), arg.id, k.c_str());
565 }
566 ret = 1;
567 }
568 }
569 cli->response_buf_remove();
570 if (op_success) {
571 rec.deleted = true;
572 if (ret == 0 && !rec.unknown_state) {
573 ++stat.success_count;
574 }
575 rec.unknown_state = false;
576 }
577 if (arg.sh.verbose >= 100 && ret == 0) {
578 fprintf(stderr, "HS_DELETE %s\n", k.c_str());
579 }
580 return ret;
581}
582
583int
584hs_longrun_thread_hs::op_update(record_value& rec)
585{
586 const std::string k = rec.key;
587 const std::string v1 = "uv1_" + k + "_" + to_stdstring(arg.id);
588 const std::string v2 = "uv2_" + k + "_" + to_stdstring(arg.id);
589 const std::string v3 = "uv3_" + k + "_" + to_stdstring(arg.id);
590 const string_ref op_ref("=", 1);
591 const string_ref op_args[1] = {
592 to_string_ref(k),
593 };
594 const string_ref modop_ref("U", 1);
595 const string_ref modop_args[4] = {
596 to_string_ref(k),
597 to_string_ref(v1),
598 to_string_ref(v2),
599 to_string_ref(v3)
600 };
601 cli->request_buf_exec_generic(0, op_ref, op_args, 1, 1, 0,
602 modop_ref, modop_args, 4, 0, 0);
603 cli->request_send();
604 if (check_hs_error("op_update_send", &rec) != 0) { return 1; }
605 size_t numflds = 0;
606 cli->response_recv(numflds);
607 if (check_hs_error("op_update_recv", &rec) != 0) { return 1; }
608 const string_ref *row = cli->get_next_row();
609 uint32_t num_rows = row
610 ? atoi_uint32_nocheck(row[0].begin(), row[0].end()) : 0;
611 cli->response_buf_remove();
612 const bool cur_unknown_state = (num_rows == 1);
613 return verify_update(k, v1, v2, v3, rec, num_rows, cur_unknown_state);
614}
615
616int
617hs_longrun_thread_hs::op_read(record_value& rec)
618{
619 const std::string k = rec.key;
620 const string_ref op_ref("=", 1);
621 const string_ref op_args[1] = {
622 to_string_ref(k),
623 };
624 cli->request_buf_exec_generic(0, op_ref, op_args, 1, 1, 0,
625 string_ref(), 0, 0, 0, 0);
626 cli->request_send();
627 if (check_hs_error("op_read_send", 0) != 0) { return 1; }
628 size_t num_flds = 0;
629 size_t num_rows = 0;
630 cli->response_recv(num_flds);
631 if (check_hs_error("op_read_recv", 0) != 0) { return 1; }
632 const string_ref *row = cli->get_next_row();
633 std::string rrec[4];
634 if (row != 0 && num_flds == 4) {
635 for (int i = 0; i < 4; ++i) {
636 rrec[i] = to_string(row[i]);
637 }
638 ++num_rows;
639 }
640 row = cli->get_next_row();
641 if (row != 0) {
642 ++num_rows;
643 }
644 cli->response_buf_remove();
645 return verify_read(k, num_rows, num_flds, rrec, rec);
646}
647
648int
649hs_longrun_thread_hs::op_readnolock(int key)
650{
651 const std::string k = to_stdstring(key);
652 const string_ref op_ref("=", 1);
653 const string_ref op_args[1] = {
654 to_string_ref(k),
655 };
656 cli->request_buf_exec_generic(0, op_ref, op_args, 1, 1, 0,
657 string_ref(), 0, 0, 0, 0);
658 cli->request_send();
659 if (check_hs_error("op_read_send", 0) != 0) { return 1; }
660 size_t num_flds = 0;
661 size_t num_rows = 0;
662 cli->response_recv(num_flds);
663 if (check_hs_error("op_read_recv", 0) != 0) { return 1; }
664 const string_ref *row = cli->get_next_row();
665 std::string rrec[4];
666 if (row != 0 && num_flds == 4) {
667 for (int i = 0; i < 4; ++i) {
668 rrec[i] = to_string(row[i]);
669 }
670 ++num_rows;
671 }
672 row = cli->get_next_row();
673 if (row != 0) {
674 ++num_rows;
675 }
676 cli->response_buf_remove();
677 return verify_readnolock(k, num_rows, num_flds, rrec);
678}
679
680int
681hs_longrun_thread_hs::check_hs_error(const char *mess, record_value *rec)
682{
683 const int err = cli->get_error_code();
684 if (err == 0) {
685 return 0;
686 }
687 ++stat.runtime_error_count;
688 if (arg.sh.verbose > 0) {
689 const std::string estr = cli->get_error();
690 fprintf(stderr, "RUNTIME_ERROR: op=%c wid=%d %s: %d %s\n",
691 arg.op, arg.id, mess, err, estr.c_str());
692 }
693 if (rec) {
694 rec->unknown_state = true;
695 }
696 return 1;
697}
698
699struct hs_longrun_thread_my : public hs_longrun_thread_base {
700 hs_longrun_thread_my(const arg_type& arg)
701 : hs_longrun_thread_base(arg), connected(false) { }
702 void run();
703 void show_mysql_error(const char *mess, record_value *rec);
704 int op_insert(record_value& rec);
705 int op_delete(record_value& rec);
706 int op_update(record_value& rec);
707 int op_delins(record_value& rec);
708 int op_read(record_value& rec);
709 auto_mysql db;
710 bool connected;
711};
712
713void
714hs_longrun_thread_my::run()
715{
716 const std::string mysql_host = arg.sh.conf.get_str("host", "localhost");
717 const std::string mysql_user = arg.sh.conf.get_str("mysqluser", "root");
718 const std::string mysql_passwd = arg.sh.conf.get_str("mysqlpass", "");
719 const std::string mysql_dbname = "hstestdb";
720
721 while (arg.sh.running) {
722 if (!connected) {
723 if (!mysql_real_connect(db, mysql_host.c_str(), mysql_user.c_str(),
724 mysql_passwd.c_str(), mysql_dbname.c_str(), mysql_port, 0, 0)) {
725 show_mysql_error("mysql_real_connect", 0);
726 continue;
727 }
728 }
729 connected = true;
730 const size_t rec_id = rand_record();
731 record_value& rec = *arg.sh.records[rec_id];
732 lock_guard g(rec.lock);
733 int e = 0;
734 switch (arg.op) {
735 #if 0
736 case 'I':
737 e = op_insert(rec);
738 break;
739 case 'D':
740 e = op_delete(rec);
741 break;
742 case 'U':
743 e = op_update(rec);
744 break;
745 #endif
746 case 'T':
747 e = op_delins(rec);
748 break;
749 case 'R':
750 e = op_read(rec);
751 break;
752 default:
753 break;
754 }
755 }
756}
757
758int
759hs_longrun_thread_my::op_delins(record_value& rec)
760{
761 const std::string k = rec.key;
762 const std::string v1 = "div1_" + k + "_" + to_stdstring(arg.id);
763 const std::string v2 = "div2_" + k + "_" + to_stdstring(arg.id);
764 const std::string v3 = "div3_" + k + "_" + to_stdstring(arg.id);
765 int success = 0;
766 bool cur_unknown_state = false;
767 do {
768 char query[1024];
769 #if 1
770 if (mysql_query(db, "begin") != 0) {
771 if (arg.sh.verbose >= 20) {
772 fprintf(stderr, "mysql: e=[%s] q=[%s]\n", mysql_error(db), "begin");
773 }
774 break;
775 }
776 #endif
777 cur_unknown_state = true;
778 snprintf(query, 1024,
779 "delete from hstesttbl where k = '%s'", k.c_str());
780 if (mysql_query(db, query) != 0) {
781 if (arg.sh.verbose >= 20) {
782 fprintf(stderr, "mysql: e=[%s] q=[%s]\n", mysql_error(db), query);
783 }
784 break;
785 }
786 if (mysql_affected_rows(db) != 1) {
787 if (arg.sh.verbose >= 20) {
788 fprintf(stderr, "mysql: notfound: [%s]\n", query);
789 }
790 break;
791 }
792 snprintf(query, 1024,
793 "insert into hstesttbl values ('%s', '%s', '%s', '%s')",
794 k.c_str(), v1.c_str(), v2.c_str(), v3.c_str());
795 if (mysql_query(db, query) != 0) {
796 if (arg.sh.verbose >= 20) {
797 fprintf(stderr, "mysql: e=[%s] q=[%s]\n", mysql_error(db), query);
798 }
799 break;
800 }
801 #if 1
802 if (mysql_query(db, "commit") != 0) {
803 if (arg.sh.verbose >= 20) {
804 fprintf(stderr, "mysql: e=[%s] q=[%s]\n", mysql_error(db), "commit");
805 }
806 break;
807 }
808 #endif
809 success = true;
810 cur_unknown_state = false;
811 } while (false);
812 return verify_update(k, v1, v2, v3, rec, (success != 0), cur_unknown_state);
813}
814
815int
816hs_longrun_thread_my::op_read(record_value& rec)
817{
818 const std::string k = rec.key;
819 char query[1024] = { 0 };
820 const int len = snprintf(query, 1024,
821 "select k,v1,v2,v3 from hstesttbl where k='%s'", k.c_str());
822 const int r = mysql_real_query(db, query, len > 0 ? len : 0);
823 if (r != 0) {
824 show_mysql_error(query, 0);
825 return 1;
826 }
827 MYSQL_ROW row = 0;
828 unsigned long *lengths = 0;
829 unsigned int num_rows = 0;
830 unsigned int num_flds = 0;
831 auto_mysql_res res(db);
832 std::string rrec[4];
833 if (res != 0) {
834 num_flds = mysql_num_fields(res);
835 row = mysql_fetch_row(res);
836 if (row != 0) {
837 lengths = mysql_fetch_lengths(res);
838 if (num_flds == 4) {
839 for (int i = 0; i < 4; ++i) {
840 rrec[i] = std::string(row[i], lengths[i]);
841 }
842 }
843 ++num_rows;
844 row = mysql_fetch_row(res);
845 if (row != 0) {
846 ++num_rows;
847 }
848 }
849 }
850 return verify_read(k, num_rows, num_flds, rrec, rec);
851}
852
853void
854hs_longrun_thread_my::show_mysql_error(const char *mess, record_value *rec)
855{
856 ++stat.runtime_error_count;
857 if (arg.sh.verbose > 0) {
858 fprintf(stderr, "RUNTIME_ERROR: op=%c wid=%d [%s]: %s\n",
859 arg.op, arg.id, mess, mysql_error(db));
860 }
861 if (rec) {
862 rec->unknown_state = true;
863 }
864 db.reset();
865 connected = false;
866}
867
868void
869mysql_do(MYSQL *db, const char *query)
870{
871 if (mysql_real_query(db, query, strlen(query)) != 0) {
872 fprintf(stderr, "mysql: e=[%s] q=[%s]\n", mysql_error(db), query);
873 fatal_exit("mysql_do");
874 }
875}
876
877void
878hs_longrun_init_table(const config& conf, int num_prepare,
879 hs_longrun_shared& shared)
880{
881 const std::string mysql_host = conf.get_str("host", "localhost");
882 const std::string mysql_user = conf.get_str("mysqluser", "root");
883 const std::string mysql_passwd = conf.get_str("mysqlpass", "");
884 const std::string mysql_dbname = "";
885 auto_mysql db;
886 if (!mysql_real_connect(db, mysql_host.c_str(), mysql_user.c_str(),
887 mysql_passwd.c_str(), mysql_dbname.c_str(), mysql_port, 0, 0)) {
888 fprintf(stderr, "mysql: error=[%s]\n", mysql_error(db));
889 fatal_exit("hs_longrun_init_table");
890 }
891 mysql_do(db, "drop database if exists hstestdb");
892 mysql_do(db, "create database hstestdb");
893 mysql_do(db, "use hstestdb");
894 mysql_do(db,
895 "create table hstesttbl ("
896 "k int primary key,"
897 "v1 varchar(32) not null,"
898 "v2 varchar(32) not null,"
899 "v3 varchar(32) not null"
900 ") character set utf8 collate utf8_bin engine = innodb");
901 for (int i = 0; i < num_prepare; ++i) {
902 const std::string i_str = to_stdstring(i);
903 const std::string v1 = "pv1_" + i_str;
904 const std::string v2 = "pv2_" + i_str;
905 const std::string v3 = "pv3_" + i_str;
906 char buf[1024];
907 snprintf(buf, 1024, "insert into hstesttbl(k, v1, v2, v3) values"
908 "(%d, '%s', '%s', '%s')", i, v1.c_str(), v2.c_str(), v3.c_str());
909 mysql_do(db, buf);
910 record_value *rec = shared.records[i];
911 rec->key = i_str;
912 rec->values.resize(4);
913 rec->values[0] = i_str;
914 rec->values[1] = v1;
915 rec->values[2] = v2;
916 rec->values[3] = v3;
917 rec->deleted = false;
918 }
919}
920
921int
922hs_longrun_main(int argc, char **argv)
923{
924 hs_longrun_shared shared;
925 parse_args(argc, argv, shared.conf);
926 shared.conf["host"] = shared.conf.get_str("host", "localhost");
927 shared.verbose = shared.conf.get_int("verbose", 1);
928 const int table_size = shared.conf.get_int("table_size", 10000);
929 for (int i = 0; i < table_size; ++i) {
930 std::auto_ptr<record_value> rec(new record_value());
931 rec->key = to_stdstring(i);
932 shared.records.push_back_ptr(rec);
933 }
934 mysql_library_init(0, 0, 0);
935 const int duration = shared.conf.get_int("duration", 10);
936 const int num_hsinsert = shared.conf.get_int("num_hsinsert", 10);
937 const int num_hsdelete = shared.conf.get_int("num_hsdelete", 10);
938 const int num_hsupdate = shared.conf.get_int("num_hsupdate", 10);
939 const int num_hsread = shared.conf.get_int("num_hsread", 10);
940 const int num_myread = shared.conf.get_int("num_myread", 10);
941 const int num_mydelins = shared.conf.get_int("num_mydelins", 10);
942 int num_hsreadnolock = shared.conf.get_int("num_hsreadnolock", 10);
943 const bool always_filled = (num_hsinsert == 0 && num_hsdelete == 0);
944 if (!always_filled) {
945 num_hsreadnolock = 0;
946 }
947 hs_longrun_init_table(shared.conf, always_filled ? table_size : 0,
948 shared);
949 /* create worker threads */
950 static const struct thrtmpl_type {
951 const char *type; char op; int num; int hs; int lock;
952 } thrtmpl[] = {
953 { "hsinsert", 'I', num_hsinsert, 1, 1 },
954 { "hsdelete", 'D', num_hsdelete, 1, 1 },
955 { "hsupdate", 'U', num_hsupdate, 1, 1 },
956 { "hsread", 'R', num_hsread, 1, 1 },
957 { "hsreadnolock", 'N', num_hsreadnolock, 1, 0 },
958 { "myread", 'R', num_myread, 0, 1 },
959 { "mydelins", 'T', num_mydelins, 0, 1 },
960 };
961 typedef auto_ptrcontainer< std::vector<hs_longrun_thread_base *> > thrs_type;
962 thrs_type thrs;
963 for (size_t i = 0; i < sizeof(thrtmpl)/sizeof(thrtmpl[0]); ++i) {
964 const thrtmpl_type& e = thrtmpl[i];
965 for (int j = 0; j < e.num; ++j) {
966 int id = thrs.size();
967 const hs_longrun_thread_hs::arg_type arg(id, e.type, e.op, e.lock,
968 shared);
969 std::auto_ptr<hs_longrun_thread_base> thr;
970 if (e.hs) {
971 thr.reset(new hs_longrun_thread_hs(arg));
972 } else {
973 thr.reset(new hs_longrun_thread_my(arg));
974 }
975 thrs.push_back_ptr(thr);
976 }
977 }
978 shared.num_threads = thrs.size();
979 /* start threads */
980 fprintf(stderr, "START\n");
981 shared.running = 1;
982 for (size_t i = 0; i < thrs.size(); ++i) {
983 thrs[i]->start();
984 }
985 /* wait */
986 sleep(duration);
987 /* stop thread */
988 shared.running = 0;
989 for (size_t i = 0; i < thrs.size(); ++i) {
990 thrs[i]->join();
991 }
992 fprintf(stderr, "DONE\n");
993 /* summary */
994 typedef std::map<std::string, hs_longrun_stat> stat_map;
995 stat_map sm;
996 for (size_t i = 0; i < thrs.size(); ++i) {
997 hs_longrun_thread_base *const thr = thrs[i];
998 const std::string wt = thr->arg.worker_type;
999 hs_longrun_stat& v = sm[wt];
1000 v.add(thr->stat);
1001 }
1002 hs_longrun_stat total;
1003 for (stat_map::const_iterator i = sm.begin(); i != sm.end(); ++i) {
1004 if (i->second.verify_error_count != 0) {
1005 fprintf(stderr, "%s verify_error %llu\n", i->first.c_str(),
1006 i->second.verify_error_count);
1007 }
1008 if (i->second.runtime_error_count) {
1009 fprintf(stderr, "%s runtime_error %llu\n", i->first.c_str(),
1010 i->second.runtime_error_count);
1011 }
1012 if (i->second.unknown_count) {
1013 fprintf(stderr, "%s unknown %llu\n", i->first.c_str(),
1014 i->second.unknown_count);
1015 }
1016 fprintf(stderr, "%s success %llu\n", i->first.c_str(),
1017 i->second.success_count);
1018 total.add(i->second);
1019 }
1020 if (total.verify_error_count != 0) {
1021 fprintf(stderr, "TOTAL verify_error %llu\n", total.verify_error_count);
1022 }
1023 if (total.runtime_error_count != 0) {
1024 fprintf(stderr, "TOTAL runtime_error %llu\n", total.runtime_error_count);
1025 }
1026 if (total.unknown_count != 0) {
1027 fprintf(stderr, "TOTAL unknown %llu\n", total.unknown_count);
1028 }
1029 fprintf(stderr, "TOTAL success %llu\n", total.success_count);
1030 mysql_library_end();
1031 return 0;
1032}
1033
1034};
1035
1036int
1037main(int argc, char **argv)
1038{
1039 return dena::hs_longrun_main(argc, argv);
1040}
1041
10420
=== added file 'HandlerSocket-Plugin-for-MySQL/client/hspool_test.pl'
--- HandlerSocket-Plugin-for-MySQL/client/hspool_test.pl 1970-01-01 00:00:00 +0000
+++ HandlerSocket-Plugin-for-MySQL/client/hspool_test.pl 2012-12-22 02:45:40 +0000
@@ -0,0 +1,224 @@
1#!/usr/bin/perl
2
3use strict;
4use warnings;
5use DB::HandlerSocket::Pool;
6use DBI;
7
8my %conf = ();
9for my $i (@ARGV) {
10 my ($k, $v) = split(/=/, $i);
11 $conf{$k} = $v;
12}
13
14my $verbose = get_conf("verbose", 0);
15my $actions_str = get_conf("actions",
16 "create,insert,verify,verify2,verify3,verify4,clean");
17my $tablesize = get_conf("tablesize", 1000);
18my $db = get_conf("db", "hstestdb");
19my $table = get_conf("table", "testtbl");
20my $table_schema = get_conf("table_schema", undef);
21my $engine = get_conf("engine", "innodb");
22my $host = get_conf("host", "localhost");
23my $mysqlport = get_conf("mysqlport", 3306);
24my $hsport_rd = get_conf("hsport_rd", 9998);
25my $hsport_wr = get_conf("hsport_wr", 9999);
26my $loop = get_conf("loop", 10000);
27my $op = get_conf("op", "=");
28my $ssps = get_conf("ssps", 0);
29my $num_moreflds = get_conf("moreflds", 0);
30my $moreflds_prefix = get_conf("moreflds_prefix", "f");
31my $mysql_user = 'root';
32my $mysql_password = '';
33
34my $dsn = "DBI:mysql:database=;host=$host;port=$mysqlport"
35 . ";mysql_server_prepare=$ssps";
36my $dbh = DBI->connect($dsn, $mysql_user, $mysql_password,
37 { RaiseError => 1 });
38my $hsargs = { 'host' => $host, 'port' => $hsport_rd };
39my $hspool = new DB::HandlerSocket::Pool({
40 hostmap => {
41 "$db.$table" => {
42 host => $host,
43 port => $hsport_rd,
44 },
45 },
46 resolve => undef,
47 error => undef,
48});
49$table_schema = "(k int primary key, fc30 varchar(30), ft text)"
50 if (!defined($table_schema));
51
52my @actions = split(/,/, $actions_str);
53for my $action (@actions) {
54 print "ACTION: $action\n";
55 eval "hstest_$action()";
56 if ($@) {
57 die $@;
58 }
59 print "ACTION: $action DONE\n";
60}
61
62sub get_conf {
63 my ($key, $def) = @_;
64 my $val = $conf{$key};
65 if ($val) {
66 print "$key=$val\n";
67 } else {
68 $val = $def;
69 my $defstr = $def || "(undef)";
70 print "$key=$defstr(default)\n";
71 }
72 return $val;
73}
74
75sub hstest_create {
76 $dbh->do("drop database if exists $db");
77 $dbh->do("create database $db");
78 $dbh->do("use $db");
79 $dbh->do("create table $table $table_schema engine=$engine");
80}
81
82sub hstest_dump {
83 $dbh->do("use $db");
84 my $sth = $dbh->prepare("select * from $table");
85 $sth->execute();
86 my $arr = $sth->fetchall_arrayref();
87 for my $rec (@$arr) {
88 print "REC:";
89 for my $row (@$rec) {
90 print " $row";
91 }
92 print "\n";
93 }
94}
95
96sub hstest_insert {
97 $dbh->do("use $db");
98 my $sth = $dbh->prepare("insert into $table values (?, ?, ?)");
99 for (my $k = 0; $k < $tablesize; ++$k) {
100 my $fc30 = "fc30_$k";
101 my $ft = "ft_$k";
102 $sth->execute($k, $fc30, $ft);
103 }
104}
105
106sub hstest_verify {
107 $dbh->do("use $db");
108 my $sth = $dbh->prepare("select * from $table order by k");
109 $sth->execute();
110 my $arr = $sth->fetchall_arrayref();
111 my $hsres = $hspool->index_find($db, $table, "PRIMARY", "k,fc30,ft",
112 ">=", [ 0 ], $tablesize, 0);
113 for (my $i = 0; $i < $tablesize; ++$i) {
114 my $rec = $arr->[$i];
115 my $differ = 0;
116 print "REC:" if $verbose;
117 for (my $j = 0; $j < 3; ++$j) {
118 my $fld = $rec->[$j];
119 my $hsidx = $i * 3 + $j;
120 my $hsfld = $hsres->[$hsidx];
121 if ($hsfld ne $fld) {
122 $differ = 1;
123 }
124 if ($differ) {
125 print " $fld:$hsfld" if $verbose;
126 } else {
127 print " $hsfld" if $verbose;
128 }
129 }
130 print "\n" if $verbose;
131 if ($differ) {
132 die "verification failed";
133 }
134 }
135}
136
137sub hstest_verify2 {
138 $dbh->do("use $db");
139 my $sth = $dbh->prepare("select * from $table order by k");
140 $sth->execute();
141 my $arr = $sth->fetchall_arrayref();
142 my $hsresa = $hspool->index_find_multi($db, $table, "PRIMARY",
143 "k,fc30,ft", [ [ -1, ">=", [ 0 ], $tablesize, 0 ] ]);
144 my $hsres = $hsresa->[0];
145 for (my $i = 0; $i < $tablesize; ++$i) {
146 my $rec = $arr->[$i];
147 my $differ = 0;
148 print "REC:" if $verbose;
149 for (my $j = 0; $j < 3; ++$j) {
150 my $fld = $rec->[$j];
151 my $hsidx = $i * 3 + $j;
152 my $hsfld = $hsres->[$hsidx];
153 if ($hsfld ne $fld) {
154 $differ = 1;
155 }
156 if ($differ) {
157 print " $fld:$hsfld" if $verbose;
158 } else {
159 print " $hsfld" if $verbose;
160 }
161 }
162 print "\n" if $verbose;
163 if ($differ) {
164 die "verification failed";
165 }
166 }
167}
168
169sub hashref_to_str {
170 my $href = $_[0];
171 my $r = '';
172 for my $k (sort keys %$href) {
173 my $v = $href->{$k};
174 $r .= "($k=>$v)";
175 }
176 return $r;
177}
178
179sub hstest_verify3 {
180 $dbh->do("use $db");
181 my $sth = $dbh->prepare("select * from $table order by k");
182 $sth->execute();
183 my $hsres_t = $hspool->index_find($db, $table, "PRIMARY", "k,fc30,ft",
184 ">=", [ 0 ], $tablesize, 0);
185 my $hsres = DB::HandlerSocket::Pool::result_single_to_hasharr(
186 [ 'k', 'fc30', 'ft' ], $hsres_t);
187 for (my $i = 0; $i < $tablesize; ++$i) {
188 my $mystr = hashref_to_str($sth->fetchrow_hashref());
189 my $hsstr = hashref_to_str($hsres->[$i]);
190 if ($mystr ne $hsstr) {
191 print "DIFF my=[$mystr] hs=[$hsstr]\n" if $verbose;
192 die "verification failed";
193 } else {
194 print "OK $hsstr\n" if $verbose;
195 }
196 }
197}
198
199sub hstest_verify4 {
200 $dbh->do("use $db");
201 my $sth = $dbh->prepare("select * from $table order by k");
202 $sth->execute();
203 my $hsres_t = $hspool->index_find($db, $table, "PRIMARY", "k,fc30,ft",
204 ">=", [ 0 ], $tablesize, 0);
205 my $hsres = DB::HandlerSocket::Pool::result_single_to_hashhash(
206 [ 'k', 'fc30', 'ft' ], 'k', $hsres_t);
207 my $rechash = $sth->fetchall_hashref('k');
208 while (my ($k, $href) = each (%$rechash)) {
209 my $mystr = hashref_to_str($href);
210 my $hsstr = hashref_to_str($hsres->{$k});
211 if ($mystr ne $hsstr) {
212 print "DIFF my=[$mystr] hs=[$hsstr]\n" if $verbose;
213 die "verification failed";
214 } else {
215 print "OK $hsstr\n" if $verbose;
216 }
217 }
218}
219
220sub hstest_clean {
221 $hspool->clear_pool();
222 $dbh->do("drop database if exists $db");
223}
224
0225
=== removed file 'HandlerSocket-Plugin-for-MySQL/client/hspool_test.pl'
--- HandlerSocket-Plugin-for-MySQL/client/hspool_test.pl 2011-01-10 13:39:35 +0000
+++ HandlerSocket-Plugin-for-MySQL/client/hspool_test.pl 1970-01-01 00:00:00 +0000
@@ -1,224 +0,0 @@
1#!/usr/bin/perl
2
3use strict;
4use warnings;
5use DB::HandlerSocket::Pool;
6use DBI;
7
8my %conf = ();
9for my $i (@ARGV) {
10 my ($k, $v) = split(/=/, $i);
11 $conf{$k} = $v;
12}
13
14my $verbose = get_conf("verbose", 0);
15my $actions_str = get_conf("actions",
16 "create,insert,verify,verify2,verify3,verify4,clean");
17my $tablesize = get_conf("tablesize", 1000);
18my $db = get_conf("db", "hstestdb");
19my $table = get_conf("table", "testtbl");
20my $table_schema = get_conf("table_schema", undef);
21my $engine = get_conf("engine", "innodb");
22my $host = get_conf("host", "localhost");
23my $mysqlport = get_conf("mysqlport", 3306);
24my $hsport_rd = get_conf("hsport_rd", 9998);
25my $hsport_wr = get_conf("hsport_wr", 9999);
26my $loop = get_conf("loop", 10000);
27my $op = get_conf("op", "=");
28my $ssps = get_conf("ssps", 0);
29my $num_moreflds = get_conf("moreflds", 0);
30my $moreflds_prefix = get_conf("moreflds_prefix", "f");
31my $mysql_user = 'root';
32my $mysql_password = '';
33
34my $dsn = "DBI:mysql:database=;host=$host;port=$mysqlport"
35 . ";mysql_server_prepare=$ssps";
36my $dbh = DBI->connect($dsn, $mysql_user, $mysql_password,
37 { RaiseError => 1 });
38my $hsargs = { 'host' => $host, 'port' => $hsport_rd };
39my $hspool = new DB::HandlerSocket::Pool({
40 hostmap => {
41 "$db.$table" => {
42 host => $host,
43 port => $hsport_rd,
44 },
45 },
46 resolve => undef,
47 error => undef,
48});
49$table_schema = "(k int primary key, fc30 varchar(30), ft text)"
50 if (!defined($table_schema));
51
52my @actions = split(/,/, $actions_str);
53for my $action (@actions) {
54 print "ACTION: $action\n";
55 eval "hstest_$action()";
56 if ($@) {
57 die $@;
58 }
59 print "ACTION: $action DONE\n";
60}
61
62sub get_conf {
63 my ($key, $def) = @_;
64 my $val = $conf{$key};
65 if ($val) {
66 print "$key=$val\n";
67 } else {
68 $val = $def;
69 my $defstr = $def || "(undef)";
70 print "$key=$defstr(default)\n";
71 }
72 return $val;
73}
74
75sub hstest_create {
76 $dbh->do("drop database if exists $db");
77 $dbh->do("create database $db");
78 $dbh->do("use $db");
79 $dbh->do("create table $table $table_schema engine=$engine");
80}
81
82sub hstest_dump {
83 $dbh->do("use $db");
84 my $sth = $dbh->prepare("select * from $table");
85 $sth->execute();
86 my $arr = $sth->fetchall_arrayref();
87 for my $rec (@$arr) {
88 print "REC:";
89 for my $row (@$rec) {
90 print " $row";
91 }
92 print "\n";
93 }
94}
95
96sub hstest_insert {
97 $dbh->do("use $db");
98 my $sth = $dbh->prepare("insert into $table values (?, ?, ?)");
99 for (my $k = 0; $k < $tablesize; ++$k) {
100 my $fc30 = "fc30_$k";
101 my $ft = "ft_$k";
102 $sth->execute($k, $fc30, $ft);
103 }
104}
105
106sub hstest_verify {
107 $dbh->do("use $db");
108 my $sth = $dbh->prepare("select * from $table order by k");
109 $sth->execute();
110 my $arr = $sth->fetchall_arrayref();
111 my $hsres = $hspool->index_find($db, $table, "PRIMARY", "k,fc30,ft",
112 ">=", [ 0 ], $tablesize, 0);
113 for (my $i = 0; $i < $tablesize; ++$i) {
114 my $rec = $arr->[$i];
115 my $differ = 0;
116 print "REC:" if $verbose;
117 for (my $j = 0; $j < 3; ++$j) {
118 my $fld = $rec->[$j];
119 my $hsidx = $i * 3 + $j;
120 my $hsfld = $hsres->[$hsidx];
121 if ($hsfld ne $fld) {
122 $differ = 1;
123 }
124 if ($differ) {
125 print " $fld:$hsfld" if $verbose;
126 } else {
127 print " $hsfld" if $verbose;
128 }
129 }
130 print "\n" if $verbose;
131 if ($differ) {
132 die "verification failed";
133 }
134 }
135}
136
137sub hstest_verify2 {
138 $dbh->do("use $db");
139 my $sth = $dbh->prepare("select * from $table order by k");
140 $sth->execute();
141 my $arr = $sth->fetchall_arrayref();
142 my $hsresa = $hspool->index_find_multi($db, $table, "PRIMARY",
143 "k,fc30,ft", [ [ -1, ">=", [ 0 ], $tablesize, 0 ] ]);
144 my $hsres = $hsresa->[0];
145 for (my $i = 0; $i < $tablesize; ++$i) {
146 my $rec = $arr->[$i];
147 my $differ = 0;
148 print "REC:" if $verbose;
149 for (my $j = 0; $j < 3; ++$j) {
150 my $fld = $rec->[$j];
151 my $hsidx = $i * 3 + $j;
152 my $hsfld = $hsres->[$hsidx];
153 if ($hsfld ne $fld) {
154 $differ = 1;
155 }
156 if ($differ) {
157 print " $fld:$hsfld" if $verbose;
158 } else {
159 print " $hsfld" if $verbose;
160 }
161 }
162 print "\n" if $verbose;
163 if ($differ) {
164 die "verification failed";
165 }
166 }
167}
168
169sub hashref_to_str {
170 my $href = $_[0];
171 my $r = '';
172 for my $k (sort keys %$href) {
173 my $v = $href->{$k};
174 $r .= "($k=>$v)";
175 }
176 return $r;
177}
178
179sub hstest_verify3 {
180 $dbh->do("use $db");
181 my $sth = $dbh->prepare("select * from $table order by k");
182 $sth->execute();
183 my $hsres_t = $hspool->index_find($db, $table, "PRIMARY", "k,fc30,ft",
184 ">=", [ 0 ], $tablesize, 0);
185 my $hsres = DB::HandlerSocket::Pool::result_single_to_hasharr(
186 [ 'k', 'fc30', 'ft' ], $hsres_t);
187 for (my $i = 0; $i < $tablesize; ++$i) {
188 my $mystr = hashref_to_str($sth->fetchrow_hashref());
189 my $hsstr = hashref_to_str($hsres->[$i]);
190 if ($mystr ne $hsstr) {
191 print "DIFF my=[$mystr] hs=[$hsstr]\n" if $verbose;
192 die "verification failed";
193 } else {
194 print "OK $hsstr\n" if $verbose;
195 }
196 }
197}
198
199sub hstest_verify4 {
200 $dbh->do("use $db");
201 my $sth = $dbh->prepare("select * from $table order by k");
202 $sth->execute();
203 my $hsres_t = $hspool->index_find($db, $table, "PRIMARY", "k,fc30,ft",
204 ">=", [ 0 ], $tablesize, 0);
205 my $hsres = DB::HandlerSocket::Pool::result_single_to_hashhash(
206 [ 'k', 'fc30', 'ft' ], 'k', $hsres_t);
207 my $rechash = $sth->fetchall_hashref('k');
208 while (my ($k, $href) = each (%$rechash)) {
209 my $mystr = hashref_to_str($href);
210 my $hsstr = hashref_to_str($hsres->{$k});
211 if ($mystr ne $hsstr) {
212 print "DIFF my=[$mystr] hs=[$hsstr]\n" if $verbose;
213 die "verification failed";
214 } else {
215 print "OK $hsstr\n" if $verbose;
216 }
217 }
218}
219
220sub hstest_clean {
221 $hspool->clear_pool();
222 $dbh->do("drop database if exists $db");
223}
224
2250
=== added file 'HandlerSocket-Plugin-for-MySQL/client/hstest.cpp'
--- HandlerSocket-Plugin-for-MySQL/client/hstest.cpp 1970-01-01 00:00:00 +0000
+++ HandlerSocket-Plugin-for-MySQL/client/hstest.cpp 2012-12-22 02:45:40 +0000
@@ -0,0 +1,1532 @@
1
2// vim:sw=2:ai
3
4#include <signal.h>
5#include <sys/time.h>
6#include <stdio.h>
7#include <string.h>
8#include <vector>
9#include <stdlib.h>
10#include <memory>
11#include <errno.h>
12#include <mysql.h>
13#include <time.h>
14#include <sys/types.h>
15#include <sys/stat.h>
16#include <fcntl.h>
17
18#include "util.hpp"
19#include "auto_ptrcontainer.hpp"
20#include "socket.hpp"
21#include "thread.hpp"
22#include "hstcpcli.hpp"
23
24#if __GNUC__ >= 4
25long atomic_exchange_and_add(volatile long *valp, long c)
26{
27 return __sync_fetch_and_add(valp, c);
28}
29#else
30#include <bits/atomicity.h>
31using namespace __gnu_cxx;
32long atomic_exchange_and_add(volatile long *valp, long c)
33{
34 return __exchange_and_add((volatile _Atomic_word *)valp, c);
35}
36#endif
37
38namespace dena {
39
40struct auto_mysql : private noncopyable {
41 auto_mysql() : db(0) {
42 reset();
43 }
44 ~auto_mysql() {
45 if (db) {
46 mysql_close(db);
47 }
48 }
49 void reset() {
50 if (db) {
51 mysql_close(db);
52 }
53 if ((db = mysql_init(0)) == 0) {
54 fatal_abort("failed to initialize mysql client");
55 }
56 }
57 operator MYSQL *() const { return db; }
58 private:
59 MYSQL *db;
60};
61
62struct auto_mysql_res : private noncopyable {
63 auto_mysql_res(MYSQL *db) {
64 res = mysql_store_result(db);
65 }
66 ~auto_mysql_res() {
67 if (res) {
68 mysql_free_result(res);
69 }
70 }
71 operator MYSQL_RES *() const { return res; }
72 private:
73 MYSQL_RES *res;
74};
75
76struct auto_mysql_stmt : private noncopyable {
77 auto_mysql_stmt(MYSQL *db) {
78 stmt = mysql_stmt_init(db);
79 }
80 ~auto_mysql_stmt() {
81 if (stmt) {
82 mysql_stmt_close(stmt);
83 }
84 }
85 operator MYSQL_STMT *() const { return stmt; }
86 private:
87 MYSQL_STMT *stmt;
88};
89
90namespace {
91
92double
93gettimeofday_double()
94{
95 struct timeval tv = { };
96 if (gettimeofday(&tv, 0) != 0) {
97 fatal_abort("gettimeofday");
98 }
99 return static_cast<double>(tv.tv_usec) / 1000000 + tv.tv_sec;
100}
101
102// unused
103void
104wait_close(int fd)
105{
106 char buf[1024];
107 while (true) {
108 int r = read(fd, buf, sizeof(buf));
109 if (r <= 0) {
110 break;
111 }
112 }
113}
114
115// unused
116void
117gentle_close(int fd)
118{
119 int r = shutdown(fd, SHUT_WR);
120 if (r != 0) {
121 return;
122 }
123 wait_close(fd);
124}
125
126};
127
128struct hstest_shared {
129 config conf;
130 socket_args arg;
131 int verbose;
132 size_t loop;
133 size_t pipe;
134 char op;
135 long num_threads;
136 mutable volatile long count;
137 mutable volatile long conn_count;
138 long wait_conn;
139 volatile char *keygen;
140 long keygen_size;
141 mutable volatile int enable_timing;
142 int usleep;
143 int dump;
144 hstest_shared() : verbose(0), loop(0), pipe(0), op('G'), num_threads(0),
145 count(0), conn_count(0), wait_conn(0), keygen(0), keygen_size(0),
146 enable_timing(0), usleep(0), dump(0) { }
147 void increment_count(unsigned int c = 1) const volatile {
148 atomic_exchange_and_add(&count, c);
149 }
150 void increment_conn(unsigned int c) const volatile {
151 atomic_exchange_and_add(&conn_count, c);
152 while (wait_conn != 0 && conn_count < wait_conn) {
153 sleep(1);
154 }
155 // fprintf(stderr, "wait_conn=%ld done\n", wait_conn);
156 }
157};
158
159struct hstest_thread {
160 struct arg_type {
161 size_t id;
162 const hstest_shared& sh;
163 bool watch_flag;
164 arg_type(size_t i, const hstest_shared& s, bool w)
165 : id(i), sh(s), watch_flag(w) { }
166 };
167 hstest_thread(const arg_type& a) : arg(a), io_success_count(0),
168 op_success_count(0), response_min(99999), response_max(0),
169 response_sum(0), response_avg(0) { }
170 void operator ()();
171 void test_1();
172 void test_2_3(int test_num);
173 void test_4_5(int test_num);
174 void test_6(int test_num);
175 void test_7(int test_num);
176 void test_8(int test_num);
177 void test_9(int test_num);
178 void test_10(int test_num);
179 void test_11(int test_num);
180 void test_12(int test_num);
181 void test_21(int test_num);
182 void test_22(int test_num);
183 void test_watch();
184 void sleep_if();
185 void set_timing(double time_spent);
186 arg_type arg;
187 auto_file fd;
188 size_t io_success_count;
189 size_t op_success_count;
190 double response_min, response_max, response_sum, response_avg;
191};
192
193void
194hstest_thread::test_1()
195{
196 char buf[1024];
197