Merge lp:~percona-dev/percona-xtrabackup/lp713799_test into lp:percona-xtrabackup/2.0

[Valentine Gostev]

Added test case for bug lp:713799

[Alexey Kopytov (akopytov)]

Hi Valentine,

Thanks for the quick turnaround with the test case. Some comments below:

On 07.02.11 8:36, Valentine Gostev wrote:
> Valentine Gostev has proposed merging lp:~percona-dev/percona-xtrabackup/lp713799_test into lp:percona-xtrabackup.
>
> Requested reviews:
> Alexey Kopytov (akopytov)
> Related bugs:
> #712441 Test backup during intensive create/drop/alter table statements
> https://bugs.launchpad.net/bugs/712441
> #713799 xtrabackup: race condition when trying to open an already removed tablespace
> https://bugs.launchpad.net/bugs/713799
>
> For more details, see:
> https://code.launchpad.net/~percona-dev/percona-xtrabackup/lp713799_test/+merge/48750
>
> Added test case for bug lp:713799

> === added file 'test/inc/race_table_create.sh'
> --- test/inc/race_table_create.sh 1970-01-01 00:00:00 +0000
> +++ test/inc/race_table_create.sh 2011-02-07 05:36:05 +0000
> @@ -0,0 +1,20 @@
> +. inc/common.sh
> +
> +COMMD="mysql --no-defaults --socket=/tmp/xtrabackup.mysql.sock --user=root"
> +run_cmd ${COMMD} -e "create database race;"
> +

All other tests use " ${MYSQL} ${MYSQL_ARGS} -e ..." to run the mysql
client. It looks like the same should work here. Or am I missing something?

> +rows=1000

I can imaging that on a fast machine (and with a bit of luck) the
create/drop loop can finish before the parent process actually starts to
backup. Which makes the test unreliable.

> +rowc=0
> +while [ "$rows" -gt "$rowc" ]
> +do
> + t1=tr$RANDOM
> + t2=tr$RANDOM
> + t3=tr$RANDOM
> + ${COMMD} -e "create table $t1 (a int) ENGINE=InnoDB;" race
> + ${COMMD} -e "create table $t2 (a int) ENGINE=InnoDB;" race
> + ${COMMD} -e "create table $t3 (a int) ENGINE=InnoDB;" race
> + ${COMMD} -e "drop table $t1;" race
> + ${COMMD} -e "drop table $t2;" race
> + ${COMMD} -e "drop table $t3;" race
> + let "rowc=rowc+1"
> +done
>
> === added file 'test/t/xb_table_race.sh'
> --- test/t/xb_table_race.sh 1970-01-01 00:00:00 +0000
> +++ test/t/xb_table_race.sh 2011-02-07 05:36:05 +0000
> @@ -0,0 +1,20 @@
> +. inc/common.sh
> +
> +init
> +run_mysqld --innodb_file_per_table
> +
> +run_cmd bash inc/race_table_create.sh &
> +

I would make race_table_create a function rather than a separate script.

> +sleep 3
> +

Same comments as before. Using sleeps to synchronize asynchronous
process is generally a bad idea.

> +# Full backup
> +# Full backup folder
> +mkdir -p $topdir/data/full
> +vlog "Starting backup"
> +
> +xtrabackup --datadir=$mysql_datadir --backup --target-dir=$topdir/data/full
> +
> +vlog "Full backup done"
> +

It would also make sense to actually test restoring from the backup to
verify its integrity. That would require creating some permanent
table(s) in addition to the transient ones being created in
race_table_create.

> +stop_mysqld
> +clean
>

On a general level, I think this test is good for an ad-hoc testing of
the fix for bug #713799 (when you can manually verify the results), but
it's too flaky to include it into the test suite. We either need to add
some synchronization facilities to the xtrabackup test framework, or
create a separate stress test suite for such tes...

[Vadim Tkachenko (vadim-tk)]

I think we need special tests which run manually, not in automatic mode.
This test is good to include into manual tests

[Valentine Gostev (longbow)]

Changes:

Added suites support, usage:
./run.sh -s <suite name>, where suite name is a name of directory with tests
Added function for cycling create/drop table statements (function accepts number of cycles as argument, 1000 cycles - default number)
Test case for bug 713799 was improved.

=== added directory 'test/experimental'
=== added file 'test/experimental/xb_race_drop.sh'
--- test/experimental/xb_race_drop.sh	1970-01-01 00:00:00 +0000
+++ test/experimental/xb_race_drop.sh	2011-03-01 17:05:48 +0000
@@ -0,0 +1,15 @@
+. inc/common.sh
+
+init
+run_mysqld --innodb_file_per_table
+
+run_cmd race_create_drop &
+
+sleep 3
+# Full backup
+mkdir -p $topdir/data/full
+vlog "Starting backup"
+xtrabackup --datadir=$mysql_datadir --backup --target-dir=$topdir/data/full
+vlog "Full backup done"
+stop_mysqld
+clean 
 
=== modified file 'test/inc/common.sh'
--- test/inc/common.sh	2011-01-03 07:55:23 +0000
+++ test/inc/common.sh	2011-03-01 17:05:48 +0000
@@ -237,3 +237,28 @@
 init_mysql_dir
 set_mysl_port
 }
+
+function race_create_drop()
+{
+  vlog "Started create/drop table cycle"
+  race_cycle_num=$1
+  if [ -z $race_cycle_num ]
+  then
+    race_cycle_num=1000
+  fi
+  run_cmd ${MYSQL} ${MYSQL_ARGS} -e "create database race;"
+  race_cycle_cnt=0;
+  while [ "$race_cycle_num" -gt "$race_cycle_cnt"]
+  do
+	t1=tr$RANDOM
+	t2=tr$RANDOM
+	t3=tr$RANDOM
+	${MYSQL} ${MYSQL_ARGS} -e "create table $t1 (a int) ENGINE=InnoDB;" race
+	${MYSQL} ${MYSQL_ARGS} -e "create table $t2 (a int) ENGINE=InnoDB;" race
+	${MYSQL} ${MYSQL_ARGS} -e "create table $t3 (a int) ENGINE=InnoDB;" race
+	${MYSQL} ${MYSQL_ARGS} -e "drop table $t1;" race
+	${MYSQL} ${MYSQL_ARGS} -e "drop table $t2;" race
+	${MYSQL} ${MYSQL_ARGS} -e "drop table $t3;" race
+	let "race_cycle_cnt=race_cycle_cnt+1"
+  done
+}
 
=== modified file 'test/run.sh'
--- test/run.sh	2010-12-30 16:52:32 +0000
+++ test/run.sh	2011-03-01 17:05:48 +0000
@@ -10,16 +10,18 @@
 	echo "-g          Output debug information to results/*.out"
 	echo "-t          Run only a single named test"
 	echo "-h          Print this help megssage"
+	echo "-s	  Select a test suite to run. Possible values: experimental, t. Default is t"
 }
 XTRACE_OPTION=""
 export SKIPPED_EXIT_CODE=200
 export MYSQL_VERSION="system"
-while getopts "gh?m:t:" options; do
+while getopts "gh?m:t:s:" options; do
 	case $options in
 		m ) export MYSQL_VERSION="$OPTARG";;
 		t ) tname="$OPTARG";;
 		g ) XTRACE_OPTION="-x";;
 		h ) usage; exit;;
+		s ) tname="$OPTARG/*.sh";;
 		\? ) usage; exit -1;;
 		* ) usage; exit -1;;
 	esac