Merge lp:~akopytov/percona-xtrabackup/test-suite-cleanups-2.0 into lp:percona-xtrabackup/2.0
- test-suite-cleanups-2.0
- Merge into 2.0
Status: | Merged | ||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Approved by: | Alexey Kopytov | ||||||||||||||||||||||||
Approved revision: | no longer in the source branch. | ||||||||||||||||||||||||
Merged at revision: | 565 | ||||||||||||||||||||||||
Proposed branch: | lp:~akopytov/percona-xtrabackup/test-suite-cleanups-2.0 | ||||||||||||||||||||||||
Merge into: | lp:percona-xtrabackup/2.0 | ||||||||||||||||||||||||
Diff against target: |
1804 lines (+792/-621) 16 files modified
patches/innodb56.patch (+17/-1) src/xtrabackup.cc (+6/-14) test/inc/common.sh (+102/-65) test/run.sh (+653/-96) test/subunit.sh (+4/-1) test/t/bug1085099.sh (+1/-3) test/t/bug664986.sh (+0/-8) test/t/bug996493.sh (+0/-4) test/t/ib_stream_incremental.sh (+1/-1) test/t/remote_tablespaces.sh (+1/-1) test/t/undo_tablespaces.sh (+1/-1) test/t/xb_part_range.sh (+2/-12) test/t/xb_perm_basic.sh (+0/-2) test/t/xb_perm_stream.sh (+0/-2) test/t/xb_stats_datadir.sh (+4/-5) test/testrun.c (+0/-405) |
||||||||||||||||||||||||
To merge this branch: | bzr merge lp:~akopytov/percona-xtrabackup/test-suite-cleanups-2.0 | ||||||||||||||||||||||||
Related bugs: |
|
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
George Ormond Lorch III (community) | g2 | Approve | |
Stewart Smith | Pending | ||
Review via email: mp+172962@code.launchpad.net |
Commit message
Description of the change
Various test suite cleanups:
* Re-implemented testrun.c in testrun.sh. This allows cleaner and
more efficient implementations of parallel tests execution and
status aggregation. Unlike previously, a server instance to detect
version is now started just once before tests execution, which
should save a lot of time. And we don't need to hash the
binary. Progress is now reported and stats are also collected in
the main script. And we also don't need multuple subunit files.
* As we don't need the run.sh wrapper now either, renamed testrun.sh
back to its original name.
* Fixed bug #1097434 "CTRL-C or kill run.sh only terminates current
test, not entire run" by defining appropriate signal handler in
the script.
* Fixed bug #1097444 "xtrabackup test suite leaves
xtrabacku
/tmp" by making sure everything is killed and cleaned up in all
circumstances
* Fixed bug #1194813 "The port lock mechanism in XB test suite does
not work"
* Time taken by each test is now reported
* The summary is now a lot more similar to MTR and includes test
execution time stats and skipped tests summary.
* When a test fails in debug mode, information on where the
artifacts can be found is now displayed in the summary
* record_db_state / verify_db_state now pass --skip-
to mysqldump, which makes identifying difference in data rows much
easier.
* reduced the time we are waiting for a server to start by tweaking
mysql_ping()
* fixed bug #1196475 "xtrabackup does not recognize checkpoint #0 as
a valid checkpoint on --prepare" as the previous change result in
sporatic test failures without the fix.
* fixed bug #1196894 "Wrong check for partitioning support in
xb_
* fixed bug #1197249 "Debug xtrabackup_56 builds are affected by
upstream bug #69653"
Alexey Kopytov (akopytov) wrote : | # |
George Ormond Lorch III (gl-az) wrote : | # |
One minor thing, line 1172 of the diff. run.sh sources ./inc/common.sh, then a few lines later sources the test which may (likely) also source ./inc/common.sh. Not a huge deal as they should both be sourcing the same file and common.sh doesn't do much of anything in the global space. If something changes in the future it could be possible for a test to potentially source a different common.sh or if any new scripting is added to the common.sh global space it would/could be executed twice possibly unknowingly causing some kind of issue.
Alexey Kopytov (akopytov) wrote : | # |
Hi George,
On Tue, 09 Jul 2013 18:48:26 -0000, George Ormond Lorch III wrote:
> One minor thing, line 1172 of the diff. run.sh sources ./inc/common.sh, then a few lines later sources the test which may (likely) also source ./inc/common.sh. Not a huge deal as they should both be sourcing the same file and common.sh doesn't do much of anything in the global space. If something changes in the future it could be possible for a test to potentially source a different common.sh or if any new scripting is added to the common.sh global space it would/could be executed twice possibly unknowingly causing some kind of issue.
>
Yes, basically inc/common.sh is now redundant and not required to be
included by tests. So it can be merged to run.sh.
Currently it can be safely included any number of times (i.e. does not
change anything in the environment), so I decided to leave it as is for
now. Should it cause any problems, we can implement the above in the
next cleanups round :)
George Ormond Lorch III (gl-az) wrote : | # |
Also I see that common is included at the head of run.sh as well as later included in the sub statement () that sources the tests. Another possible redundancy but again, very, very minor issue that can/will be eliminated as you say if/when common.sh is absorbed into run.sh.
George Ormond Lorch III (gl-az) wrote : | # |
Overall, some very nice work here. You do write quite pretty code. Everything appears to work well and I do not see anything obvious that needs fixing but I am pretty sure I missed some little juicy nugget that will cause heartache later on :) As always, I learned another cool scripting trick or two from this that I will likely use in PTB.
Stewart Smith (stewart) wrote : | # |
Alexey Kopytov <email address hidden> writes:
> Yes, basically inc/common.sh is now redundant and not required to be
> included by tests. So it can be merged to run.sh.
If you're going to clean this up at some point it may be good to rename
the files to something that isn't .sh too, as otherwise it trips up some
automated check-for-
--
Stewart Smith
Preview Diff
1 | === modified file 'patches/innodb56.patch' |
2 | --- patches/innodb56.patch 2013-07-01 12:20:49 +0000 |
3 | +++ patches/innodb56.patch 2013-07-04 07:45:32 +0000 |
4 | @@ -1303,4 +1303,20 @@ |
5 | int main() |
6 | { |
7 | extern void __attribute__((weak)) foo(void); |
8 | - |
9 | +--- a/mysys/my_thr_init.c |
10 | ++++ b/mysys/my_thr_init.c |
11 | +@@ -462,8 +462,12 @@ |
12 | + |
13 | + extern void **my_thread_var_dbug() |
14 | + { |
15 | +- struct st_my_thread_var *tmp= |
16 | +- my_pthread_getspecific(struct st_my_thread_var*,THR_KEY_mysys); |
17 | ++ struct st_my_thread_var *tmp; |
18 | ++ |
19 | ++ if (!my_thread_global_init_done) |
20 | ++ return NULL; |
21 | ++ |
22 | ++ tmp= my_pthread_getspecific(struct st_my_thread_var*,THR_KEY_mysys); |
23 | + return tmp && tmp->init ? &tmp->dbug : 0; |
24 | + } |
25 | + #endif /* DBUG_OFF */ |
26 | |
27 | === modified file 'src/xtrabackup.cc' |
28 | --- src/xtrabackup.cc 2013-06-18 14:46:51 +0000 |
29 | +++ src/xtrabackup.cc 2013-07-04 07:45:32 +0000 |
30 | @@ -6471,6 +6471,8 @@ |
31 | |
32 | ulint fold; |
33 | |
34 | + bool checkpoint_found; |
35 | + |
36 | max_no = ut_dulint_zero; |
37 | |
38 | if (!xb_init_log_block_size()) { |
39 | @@ -6587,6 +6589,8 @@ |
40 | // ' ', 4); |
41 | } |
42 | |
43 | + checkpoint_found = false; |
44 | + |
45 | /* read last checkpoint lsn */ |
46 | for (field = LOG_CHECKPOINT_1; field <= LOG_CHECKPOINT_2; |
47 | field += LOG_CHECKPOINT_2 - LOG_CHECKPOINT_1) { |
48 | @@ -6599,25 +6603,13 @@ |
49 | if (ut_dulint_cmp(checkpoint_no, max_no) >= 0) { |
50 | max_no = checkpoint_no; |
51 | max_lsn = MACH_READ_64(log_buf + field + LOG_CHECKPOINT_LSN); |
52 | -/* |
53 | - mach_write_to_4(log_buf + field + LOG_CHECKPOINT_OFFSET, |
54 | - LOG_FILE_HDR_SIZE + ut_dulint_minus(max_lsn, |
55 | - ut_dulint_align_down(max_lsn,OS_FILE_LOG_BLOCK_SIZE))); |
56 | - |
57 | - ulint fold; |
58 | - fold = ut_fold_binary(log_buf + field, LOG_CHECKPOINT_CHECKSUM_1); |
59 | - mach_write_to_4(log_buf + field + LOG_CHECKPOINT_CHECKSUM_1, fold); |
60 | - |
61 | - fold = ut_fold_binary(log_buf + field + LOG_CHECKPOINT_LSN, |
62 | - LOG_CHECKPOINT_CHECKSUM_2 - LOG_CHECKPOINT_LSN); |
63 | - mach_write_to_4(log_buf + field + LOG_CHECKPOINT_CHECKSUM_2, fold); |
64 | -*/ |
65 | + checkpoint_found = true; |
66 | } |
67 | not_consistent: |
68 | ; |
69 | } |
70 | |
71 | - if (ut_dulint_cmp(max_no, ut_dulint_zero) == 0) { |
72 | + if (!checkpoint_found) { |
73 | msg("xtrabackup: No valid checkpoint found.\n"); |
74 | goto error; |
75 | } |
76 | |
77 | === modified file 'test/inc/common.sh' |
78 | --- test/inc/common.sh 2013-06-28 15:21:38 +0000 |
79 | +++ test/inc/common.sh 2013-07-04 07:45:32 +0000 |
80 | @@ -16,9 +16,12 @@ |
81 | } |
82 | |
83 | |
84 | -function clean() |
85 | +######################################################################## |
86 | +# Remove server var* directories from the worker's var root |
87 | +######################################################################## |
88 | +function remove_var_dirs() |
89 | { |
90 | - rm -rf ${TEST_BASEDIR}/var[0-9] |
91 | + rm -rf ${TEST_VAR_ROOT}/var[0-9] |
92 | } |
93 | |
94 | function die() |
95 | @@ -34,29 +37,33 @@ |
96 | $MYSQL_INSTALL_DB --defaults-file=${MYSQLD_VARDIR}/my.cnf \ |
97 | --basedir=${MYSQL_BASEDIR} \ |
98 | ${MYSQLD_EXTRA_ARGS} |
99 | - cd - |
100 | + cd - >/dev/null 2>&1 |
101 | } |
102 | |
103 | ######################################################################## |
104 | -# Checks whether MySQL is alive |
105 | +# Checks whether MySQL with the PID specified as an argument is alive |
106 | ######################################################################## |
107 | function mysql_ping() |
108 | { |
109 | - $MYSQLADMIN $MYSQL_ARGS --wait=100 ping >/dev/null 2>&1 || return 1 |
110 | -} |
111 | + local pid=$1 |
112 | + local attempts=60 |
113 | + local i |
114 | |
115 | -function kill_leftovers() |
116 | -{ |
117 | - local file |
118 | - for file in ${TEST_BASEDIR}/mysqld*.pid |
119 | + for ((i=1; i<=attempts; i++)) |
120 | do |
121 | - if [ -f $file ] |
122 | - then |
123 | - vlog "Found a leftover mysqld processes with PID `cat $file`, stopping it" |
124 | - kill -9 `cat $file` 2>/dev/null || true |
125 | - rm -f $file |
126 | - fi |
127 | + $MYSQLADMIN $MYSQL_ARGS ping >/dev/null 2>&1 && return 0 |
128 | + sleep 1 |
129 | + # Is the server process still alive? |
130 | + if ! kill -0 $pid >/dev/null 2>&1 |
131 | + then |
132 | + vlog "Server process PID=$pid died." |
133 | + wait $pid |
134 | + return 1 |
135 | + fi |
136 | + vlog "Made $i attempts to connect to server" |
137 | done |
138 | + |
139 | + return 1 |
140 | } |
141 | |
142 | function run_cmd() |
143 | @@ -123,21 +130,14 @@ |
144 | local port |
145 | local lockfile |
146 | |
147 | - for (( port=PORT_BASE+id; port < 65535; port++)) |
148 | + for (( port=3307 + RANDOM; port < 65535; port++)) |
149 | do |
150 | lockfile="/tmp/xtrabackup_port_lock.$port" |
151 | # Try to atomically lock the current port number |
152 | - if ! set -C > $lockfile |
153 | + if ! (set -C; > $lockfile) |
154 | then |
155 | - set +C |
156 | - # check if the process still exists |
157 | - if kill -0 "`cat $lockfile`" |
158 | - then |
159 | - continue; |
160 | - fi |
161 | - # stale file, overwrite it with the current PID |
162 | + continue; |
163 | fi |
164 | - set +C |
165 | echo $$ > $lockfile |
166 | if ! nc -z -w1 localhost $port >/dev/null 2>&1 |
167 | then |
168 | @@ -173,13 +173,14 @@ |
169 | fi |
170 | |
171 | SRV_MYSQLD_IDS[$id]="$id" |
172 | - local vardir="${TEST_BASEDIR}/var${id}" |
173 | + local vardir="${TEST_VAR_ROOT}/var${id}" |
174 | SRV_MYSQLD_VARDIR[$id]="$vardir" |
175 | SRV_MYSQLD_DATADIR[$id]="$vardir/data" |
176 | SRV_MYSQLD_TMPDIR[$id]="$vardir/tmp" |
177 | - SRV_MYSQLD_PIDFILE[$id]="${TEST_BASEDIR}/mysqld${id}.pid" |
178 | + SRV_MYSQLD_PIDFILE[$id]="${TEST_VAR_ROOT}/mysqld${id}.pid" |
179 | + SRV_MYSQLD_ERRFILE[$id]="$vardir/data/mysqld${id}.err" |
180 | SRV_MYSQLD_PORT[$id]=`get_free_port $id` |
181 | - SRV_MYSQLD_SOCKET[$id]=`mktemp -t xtrabackup.mysql.sock.XXXXXX` |
182 | + SRV_MYSQLD_SOCKET[$id]=`mktemp -t mysql.sock.XXXXXX` |
183 | } |
184 | |
185 | ######################################################################## |
186 | @@ -200,6 +201,7 @@ |
187 | SRV_MYSQLD_DATADIR[$id]= |
188 | SRV_MYSQLD_TMPDIR[$id]= |
189 | SRV_MYSQLD_PIDFILE[$id]= |
190 | + SRV_MYSQLD_ERRFILE[$id]= |
191 | SRV_MYSQLD_PORT[$id]= |
192 | SRV_MYSQLD_SOCKET[$id]= |
193 | } |
194 | @@ -216,6 +218,7 @@ |
195 | MYSQLD_DATADIR="${SRV_MYSQLD_DATADIR[$id]}" |
196 | MYSQLD_TMPDIR="${SRV_MYSQLD_TMPDIR[$id]}" |
197 | MYSQLD_PIDFILE="${SRV_MYSQLD_PIDFILE[$id]}" |
198 | + MYSQLD_ERRFILE="${SRV_MYSQLD_ERRFILE[$id]}" |
199 | MYSQLD_PORT="${SRV_MYSQLD_PORT[$id]}" |
200 | MYSQLD_SOCKET="${SRV_MYSQLD_SOCKET[$id]}" |
201 | |
202 | @@ -242,27 +245,36 @@ |
203 | function start_server_with_id() |
204 | { |
205 | local id=$1 |
206 | + local attempts=0 |
207 | + local max_attempts=5 |
208 | shift |
209 | |
210 | vlog "Starting server with id=$id..." |
211 | |
212 | - init_server_variables $id |
213 | - switch_server $id |
214 | - |
215 | - if [ ! -d "$MYSQLD_VARDIR" ] |
216 | - then |
217 | - vlog "Creating server root directory: $MYSQLD_VARDIR" |
218 | - mkdir "$MYSQLD_VARDIR" |
219 | - fi |
220 | - if [ ! -d "$MYSQLD_TMPDIR" ] |
221 | - then |
222 | - vlog "Creating server temporary directory: $MYSQLD_TMPDIR" |
223 | - mkdir "$MYSQLD_TMPDIR" |
224 | - fi |
225 | - |
226 | - # Create the configuration file used by mysql_install_db, the server |
227 | - # and the xtrabackup binary |
228 | - cat > ${MYSQLD_VARDIR}/my.cnf <<EOF |
229 | + while true |
230 | + do |
231 | + init_server_variables $id |
232 | + switch_server $id |
233 | + |
234 | + if [ ! -d "$MYSQLD_VARDIR" ] |
235 | + then |
236 | + vlog "Creating server root directory: $MYSQLD_VARDIR" |
237 | + mkdir "$MYSQLD_VARDIR" |
238 | + fi |
239 | + if [ ! -d "$MYSQLD_TMPDIR" ] |
240 | + then |
241 | + vlog "Creating server temporary directory: $MYSQLD_TMPDIR" |
242 | + mkdir "$MYSQLD_TMPDIR" |
243 | + fi |
244 | + |
245 | + if [ -f "$MYSQLD_ERRFILE" ] |
246 | + then |
247 | + rm "$MYSQLD_ERRFILE" |
248 | + fi |
249 | + |
250 | + # Create the configuration file used by mysql_install_db, the server |
251 | + # and the xtrabackup binary |
252 | + cat > ${MYSQLD_VARDIR}/my.cnf <<EOF |
253 | [mysqld] |
254 | socket=${MYSQLD_SOCKET} |
255 | port=${MYSQLD_PORT} |
256 | @@ -270,6 +282,7 @@ |
257 | basedir=${MYSQL_BASEDIR} |
258 | datadir=${MYSQLD_DATADIR} |
259 | tmpdir=${MYSQLD_TMPDIR} |
260 | +log-error=${MYSQLD_ERRFILE} |
261 | log-bin=mysql-bin |
262 | relay-log=mysql-relay-bin |
263 | pid-file=${MYSQLD_PIDFILE} |
264 | @@ -281,22 +294,42 @@ |
265 | user=root |
266 | EOF |
267 | |
268 | - # Create datadir and call mysql_install_db if it doesn't exist |
269 | - if [ ! -d "$MYSQLD_DATADIR" ] |
270 | - then |
271 | - vlog "Creating server data directory: $MYSQLD_DATADIR" |
272 | - mkdir -p "$MYSQLD_DATADIR" |
273 | - call_mysql_install_db |
274 | - fi |
275 | - |
276 | - # Start the server |
277 | - echo "Starting ${MYSQLD} ${MYSQLD_ARGS} $* " |
278 | - ${MYSQLD} ${MYSQLD_ARGS} $* & |
279 | - if ! mysql_ping |
280 | - then |
281 | - die "Can't start the server!" |
282 | - fi |
283 | - vlog "Server with id=$id has been started on port $MYSQLD_PORT, \ |
284 | + # Create datadir and call mysql_install_db if it doesn't exist |
285 | + if [ ! -d "$MYSQLD_DATADIR" ] |
286 | + then |
287 | + vlog "Creating server data directory: $MYSQLD_DATADIR" |
288 | + mkdir -p "$MYSQLD_DATADIR" |
289 | + call_mysql_install_db |
290 | + fi |
291 | + |
292 | + # Start the server |
293 | + echo "Starting ${MYSQLD} ${MYSQLD_ARGS} $* " |
294 | + ${MYSQLD} ${MYSQLD_ARGS} $* & |
295 | + if ! mysql_ping $! |
296 | + then |
297 | + if grep "another mysqld server running on port" $MYSQLD_ERRFILE |
298 | + then |
299 | + if ((++attempts < max_attempts)) |
300 | + then |
301 | + vlog "Made $attempts attempts to find a free port" |
302 | + reset_server_variables $id |
303 | + free_reserved_port $MYSQLD_PORT |
304 | + continue |
305 | + else |
306 | + vlog "Failed to find a free port after $attempts attempts" |
307 | + fi |
308 | + fi |
309 | + vlog "Can't start the server. Server log (if exists):" |
310 | + vlog "----------------" |
311 | + cat ${MYSQLD_ERRFILE} >&2 || true |
312 | + vlog "----------------" |
313 | + exit -1 |
314 | + else |
315 | + break |
316 | + fi |
317 | + done |
318 | + |
319 | + vlog "Server with id=$id has been started on port $MYSQLD_PORT, \ |
320 | socket $MYSQLD_SOCKET" |
321 | } |
322 | |
323 | @@ -313,7 +346,9 @@ |
324 | |
325 | if [ -f "${MYSQLD_PIDFILE}" ] |
326 | then |
327 | - kill -9 `cat ${MYSQLD_PIDFILE}` |
328 | + local pid=`cat $MYSQLD_PIDFILE` |
329 | + kill -9 $pid |
330 | + wait $pid || true |
331 | rm -f ${MYSQLD_PIDFILE} |
332 | else |
333 | vlog "Server PID file '${MYSQLD_PIDFILE}' doesn't exist!" |
334 | @@ -480,7 +515,8 @@ |
335 | ########################################################################## |
336 | function record_db_state() |
337 | { |
338 | - $MYSQLDUMP $MYSQL_ARGS -t --compact $1 >"$topdir/tmp/$1_old.sql" |
339 | + $MYSQLDUMP $MYSQL_ARGS -t --compact --skip-extended-insert \ |
340 | + $1 >"$topdir/tmp/$1_old.sql" |
341 | } |
342 | |
343 | |
344 | @@ -490,7 +526,8 @@ |
345 | ########################################################################## |
346 | function verify_db_state() |
347 | { |
348 | - $MYSQLDUMP $MYSQL_ARGS -t --compact $1 >"$topdir/tmp/$1_new.sql" |
349 | + $MYSQLDUMP $MYSQL_ARGS -t --compact --skip-extended-insert \ |
350 | + $1 >"$topdir/tmp/$1_new.sql" |
351 | diff -u "$topdir/tmp/$1_old.sql" "$topdir/tmp/$1_new.sql" |
352 | } |
353 | |
354 | |
355 | === removed file 'test/run.sh' |
356 | --- test/run.sh 2012-11-30 04:28:06 +0000 |
357 | +++ test/run.sh 1970-01-01 00:00:00 +0000 |
358 | @@ -1,14 +0,0 @@ |
359 | -#!/bin/bash |
360 | - |
361 | -XB_BUILD="autodetect" |
362 | -while getopts "fc:" options; do |
363 | - case $options in |
364 | - c ) XB_BUILD="$OPTARG";; |
365 | - f ) ;; # ignored |
366 | - esac |
367 | -done |
368 | - |
369 | -rm -rf results/ var/ test_results.subunit |
370 | - |
371 | -CFLAGS=-g make testrun |
372 | -./testrun -c $XB_BUILD |
373 | |
374 | === renamed file 'test/testrun.sh' => 'test/run.sh' |
375 | --- test/testrun.sh 2013-05-07 07:53:25 +0000 |
376 | +++ test/run.sh 2013-07-04 07:45:32 +0000 |
377 | @@ -2,14 +2,39 @@ |
378 | |
379 | export DEBUG= |
380 | |
381 | +shopt -s extglob |
382 | + |
383 | +set +e |
384 | + |
385 | . inc/common.sh |
386 | - |
387 | -trap kill_leftovers 1 2 3 15 |
388 | - |
389 | -set +e |
390 | +. subunit.sh |
391 | + |
392 | +trap cleanup_on_exit EXIT |
393 | +trap terminate SIGHUP SIGINT SIGQUIT SIGTERM |
394 | |
395 | result=0 |
396 | |
397 | +# Default test timeout in seconds |
398 | +TEST_TIMEOUT=600 |
399 | + |
400 | +# Magic exit code to indicate a skipped test |
401 | +export SKIPPED_EXIT_CODE=200 |
402 | + |
403 | +# Default server installation directory (-d option) |
404 | +MYSQL_BASEDIR=${MYSQL_BASEDIR:-"$PWD/server"} |
405 | + |
406 | +TEST_BASEDIR="$PWD" |
407 | +TEST_VAR_ROOT="$TEST_BASEDIR/var" |
408 | + |
409 | +# Global statistics |
410 | +FAILED_COUNT=0 |
411 | +FAILED_TESTS= |
412 | +SKIPPED_COUNT=0 |
413 | +SKIPPED_TESTS= |
414 | +SUCCESSFUL_COUNT=0 |
415 | +TOTAL_COUNT=0 |
416 | +TOTAL_TIME=0 |
417 | + |
418 | function usage() |
419 | { |
420 | cat <<EOF |
421 | @@ -24,7 +49,183 @@ |
422 | -c conf XtraBackup build configuration as specified to build.sh. |
423 | Default is 'autodetect', i.e. xtrabackup binary is determined based |
424 | on the MySQL version. |
425 | -EOF |
426 | +-j N Run tests in N parallel processes. |
427 | +-T seconds Test timeout (default is $TEST_TIMEOUT seconds). |
428 | +EOF |
429 | +} |
430 | + |
431 | +############################################################################### |
432 | +# Calculate the number of parallel workers automatically based on the number of |
433 | +# available cores |
434 | +############################################################################### |
435 | +function autocalc_nworkers() |
436 | +{ |
437 | + if [ -r /proc/cpuinfo ] |
438 | + then |
439 | + NWORKERS=`grep processor /proc/cpuinfo | wc -l` |
440 | + elif which sysctl >/dev/null 2>&1 |
441 | + then |
442 | + NWORKERS=`sysctl -n hw.ncpu` |
443 | + fi |
444 | + |
445 | + if [[ ! $NWORKERS =~ ^[0-9]+$ || $NWORKERS < 1 ]] |
446 | + then |
447 | + echo "Cannot determine the number of available CPU cures!" |
448 | + exit -1 |
449 | + fi |
450 | + |
451 | + if [ "$NWORKERS" -gt 16 ] |
452 | + then |
453 | + echo "Autodetected number of cores: $NWORKERS" |
454 | + echo "Limiting to 16 to avoid excessive resource consumption" |
455 | + |
456 | + NWORKERS=16 |
457 | + fi |
458 | +} |
459 | + |
460 | +############################################################################### |
461 | +# Kill a specified worker |
462 | +############################################################################### |
463 | +function kill_worker() |
464 | +{ |
465 | + local worker=$1 |
466 | + local pid=${worker_pids[$worker]} |
467 | + |
468 | + worker_pids[$worker]="" |
469 | + |
470 | + if ! kill -0 $pid >/dev/null 2>&1 |
471 | + then |
472 | + wait $pid >/dev/null 2>&1 || true |
473 | + return 0 |
474 | + fi |
475 | + |
476 | + # First send SIGTERM to let worker exit gracefully |
477 | + kill -SIGTERM $pid >/dev/null 2>&1 || true |
478 | + |
479 | + sleep 1 |
480 | + |
481 | + if ! kill -0 $pid >/dev/null 2>&1 |
482 | + then |
483 | + wait $pid >/dev/null 2>&1 || true |
484 | + return 0 |
485 | + fi |
486 | + |
487 | + # Now kill with SIGKILL |
488 | + kill -SIGKILL $pid >/dev/null 2>&1 || true |
489 | + wait $pid >/dev/null 2>&1 || true |
490 | + release_port_locks $pid |
491 | +} |
492 | + |
493 | +######################################################################## |
494 | +# Kill all running workers |
495 | +######################################################################## |
496 | +function kill_all_workers() |
497 | +{ |
498 | + while true |
499 | + do |
500 | + found="" |
501 | + |
502 | + # First send SIGTERM to let workers exit gracefully |
503 | + for ((i = 1; i <= NWORKERS; i++)) |
504 | + do |
505 | + [ -z ${worker_pids[$i]:-""} ] && continue |
506 | + |
507 | + found="yes" |
508 | + |
509 | + if ! kill -0 ${worker_pids[$i]} >/dev/null 2>&1 |
510 | + then |
511 | + worker_pids[$i]="" |
512 | + continue |
513 | + fi |
514 | + |
515 | + kill -SIGTERM ${worker_pids[$i]} >/dev/null 2>&1 || true |
516 | + done |
517 | + |
518 | + [ -z "$found" ] && break |
519 | + |
520 | + sleep 1 |
521 | + |
522 | + # Now kill with SIGKILL |
523 | + for ((i = 1; i <= NWORKERS; i++)) |
524 | + do |
525 | + [ -z ${worker_pids[$i]:-""} ] && continue |
526 | + |
527 | + if ! kill -0 ${worker_pids[$i]} >/dev/null 2>&1 |
528 | + then |
529 | + wait ${worker_pids[$i]} >/dev/null 2>&1 || true |
530 | + worker_pids[$i]="" |
531 | + continue |
532 | + fi |
533 | + |
534 | + kill -SIGKILL ${worker_pids[$i]} >/dev/null 2>&1 || true |
535 | + wait ${worker_pids[$i]} >/dev/null 2>&1 || true |
536 | + release_port_locks ${worker_pids[$i]} |
537 | + worker_pids[$i]="" |
538 | + done |
539 | + done |
540 | +} |
541 | + |
542 | +######################################################################## |
543 | +# Handler called from a fatal signal handler trap |
544 | +######################################################################## |
545 | +function terminate() |
546 | +{ |
547 | + echo "Terminated, cleaning up..." |
548 | + |
549 | + # The following will call cleanup_on_exit() |
550 | + exit 2 |
551 | +} |
552 | + |
553 | +######################################################################## |
554 | +# Display the test run summary and exit |
555 | +######################################################################## |
556 | +function print_status_and_exit() |
557 | +{ |
558 | + local test_time=$((`now` - TEST_START_TIME)) |
559 | + cat <<EOF |
560 | +============================================================================== |
561 | +Spent $TOTAL_TIME of $test_time seconds executing testcases |
562 | + |
563 | +SUMMARY: $TOTAL_COUNT run, $SUCCESSFUL_COUNT successful, $SKIPPED_COUNT skipped, $FAILED_COUNT failed |
564 | + |
565 | +EOF |
566 | + |
567 | + if [ -n "$SKIPPED_TESTS" ] |
568 | + then |
569 | + echo "Skipped tests: $SKIPPED_TESTS" |
570 | + echo |
571 | + fi |
572 | + |
573 | + if [ -n "$FAILED_TESTS" ] |
574 | + then |
575 | + echo "Failed tests: $FAILED_TESTS" |
576 | + echo |
577 | + fi |
578 | + |
579 | + echo "See results/ for detailed output" |
580 | + |
581 | + if [ "$FAILED_COUNT" = 0 ] |
582 | + then |
583 | + exit 0 |
584 | + fi |
585 | + |
586 | + exit 1 |
587 | +} |
588 | + |
589 | +######################################################################## |
590 | +# Cleanup procedure invoked on process exit |
591 | +######################################################################## |
592 | +function cleanup_on_exit() |
593 | +{ |
594 | + kill_servers $TEST_VAR_ROOT |
595 | + |
596 | + remove_var_dirs |
597 | + |
598 | + release_port_locks $$ |
599 | + |
600 | + kill_all_workers |
601 | + |
602 | + cleanup_all_workers |
603 | } |
604 | |
605 | function find_program() |
606 | @@ -52,12 +253,11 @@ |
607 | fi |
608 | } |
609 | |
610 | +######################################################################## |
611 | +# Explore environment and setup global variables |
612 | +######################################################################## |
613 | function set_vars() |
614 | { |
615 | - TEST_BASEDIR=${TEST_BASEDIR:-"$PWD"} |
616 | - MYSQL_BASEDIR=${MYSQL_BASEDIR:-"$PWD/server"} |
617 | - PORT_BASE=$((3306 + $RANDOM)) |
618 | - |
619 | if gnutar --version > /dev/null 2>&1 |
620 | then |
621 | TAR=gnutar |
622 | @@ -88,7 +288,7 @@ |
623 | fi |
624 | DYLD_LIBRARY_PATH="$LD_LIBRARY_PATH" |
625 | |
626 | - export TEST_BASEDIR PORT_BASE TAR MYSQL_BASEDIR MYSQL MYSQLD MYSQLADMIN \ |
627 | + export TAR MYSQL_BASEDIR MYSQL MYSQLD MYSQLADMIN \ |
628 | MYSQL_INSTALL_DB PATH LD_LIBRARY_PATH DYLD_LIBRARY_PATH MYSQLDUMP |
629 | } |
630 | |
631 | @@ -232,7 +432,7 @@ |
632 | return 1 |
633 | fi |
634 | |
635 | - stop_server |
636 | + stop_server >>$OUTFILE 2>&1 |
637 | |
638 | export MYSQL_VERSION MYSQL_VERSION_COMMENT MYSQL_FLAVOR \ |
639 | INNODB_VERSION XTRADB_VERSION INNODB_FLAVOR \ |
640 | @@ -240,39 +440,362 @@ |
641 | DEFAULT_IBDATA_SIZE |
642 | } |
643 | |
644 | -export SKIPPED_EXIT_CODE=200 |
645 | +########################################################################### |
646 | +# Kill all server processes started by a worker specified with its var root |
647 | +# directory |
648 | +########################################################################### |
649 | +function kill_servers() |
650 | +{ |
651 | + local var_root=$1 |
652 | + local file |
653 | + |
654 | + [ -d $var_root ] || return 0 |
655 | + |
656 | + cd $var_root |
657 | + |
658 | + for file in mysqld*.pid |
659 | + do |
660 | + if [ -f $file ] |
661 | + then |
662 | + vlog "Found a leftover mysqld processes with PID `cat $file`, \ |
663 | +stopping it" |
664 | + kill -9 `cat $file` 2>/dev/null || true |
665 | + rm -f $file |
666 | + fi |
667 | + done |
668 | + |
669 | + cd - >/dev/null 2>&1 |
670 | +} |
671 | + |
672 | +########################################################################### |
673 | +# Kill all server processes started by a worker specified with its number |
674 | +########################################################################### |
675 | +function kill_servers_for_worker() |
676 | +{ |
677 | + local worker=$1 |
678 | + |
679 | + kill_servers $TEST_BASEDIR/var/w$worker |
680 | +} |
681 | + |
682 | +################################################################################ |
683 | +# Clean up all workers (except DEBUG_WORKER if set) We can't use worker* arrays |
684 | +# here and examine the directory structure, because the same functions is called |
685 | +# on startup when we want to cleanup all workers started on previous invokations |
686 | +################################################################################ |
687 | +function cleanup_all_workers() { local worker |
688 | + |
689 | + for worker_dir in $TEST_BASEDIR/var/w+([0-9]) |
690 | + do |
691 | + [ -d $worker_dir ] || continue |
692 | + [[ $worker_dir =~ w([0-9]+)$ ]] || continue |
693 | + |
694 | + worker=${BASH_REMATCH[1]} |
695 | + |
696 | + if [ "$worker" = "$DEBUG_WORKER" ] |
697 | + then |
698 | + echo |
699 | + echo "Skipping cleanup for worker #$worker due to debug mode." |
700 | + echo "You can do post-mortem analysis by examining test data in \ |
701 | +$worker_dir" |
702 | + echo "and the server process if it was running at the failure time." |
703 | + continue |
704 | + fi |
705 | + |
706 | + cleanup_worker $worker |
707 | + done |
708 | +} |
709 | +######################################################################## |
710 | +# Clean up a specified worker |
711 | +######################################################################## |
712 | +function cleanup_worker() |
713 | +{ |
714 | + local worker=$1 |
715 | + local tmpdir |
716 | + |
717 | + kill_servers_for_worker $worker >>$OUTFILE 2>&1 |
718 | + |
719 | + # It is possible that a file is created while rm is in progress |
720 | + # which results in "rm: cannot remove ...: Directory not empty |
721 | + # hence the loop below |
722 | + while true |
723 | + do |
724 | + # Fix permissions as some tests modify them so the following 'rm' fails |
725 | + chmod -R 0700 $TEST_BASEDIR/var/w$worker >/dev/null 2>&1 |
726 | + rm -rf $TEST_BASEDIR/var/w$worker && break |
727 | + done |
728 | + |
729 | + tmpdir=${worker_tmpdirs[$worker]:-""} |
730 | + if [ -n "$tmpdir" ] |
731 | + then |
732 | + rm -rf $tmpdir |
733 | + fi |
734 | +} |
735 | + |
736 | +######################################################################## |
737 | +# Return the number of seconds since the Epoch, UTC |
738 | +######################################################################## |
739 | +function now() |
740 | +{ |
741 | + date '+%s' |
742 | +} |
743 | + |
744 | +######################################################################## |
745 | +# Process the exit code of a specified worker |
746 | +######################################################################## |
747 | +function reap_worker() |
748 | +{ |
749 | + local worker=$1 |
750 | + local pid=${worker_pids[$worker]} |
751 | + local skip_file=${worker_skip_files[$worker]} |
752 | + local tpath=${worker_names[$worker]} |
753 | + local tname=`basename $tpath .sh` |
754 | + local test_time=$((`now` - worker_stime[$worker])) |
755 | + local status_file=${worker_status_files[$worker]} |
756 | + local rc |
757 | + |
758 | + if [ -f "$status_file" ] |
759 | + then |
760 | + rc=`cat $status_file` |
761 | + # Assume exit code 1 if status file is empty |
762 | + rc=${rc:-"1"} |
763 | + else |
764 | + # Assume exit code 1 if status file does not exist |
765 | + rc="1" |
766 | + fi |
767 | + |
768 | + printf "%-40s w%d\t" $tname $worker |
769 | + |
770 | + ((TOTAL_TIME+=test_time)) |
771 | + |
772 | + # Have to call subunit_start_test here, as currently tests cannot be |
773 | + # interleaved in the subunit output |
774 | + subunit_start_test $tpath "${worker_stime_txt[$worker]}" >> $SUBUNIT_OUT |
775 | + |
776 | + if [ $rc -eq 0 ] |
777 | + then |
778 | + echo "[passed] $test_time" |
779 | + |
780 | + SUCCESSFUL_COUNT=$((SUCCESSFUL_COUNT + 1)) |
781 | + subunit_pass_test $tpath >> $SUBUNIT_OUT |
782 | + |
783 | + cleanup_worker $worker |
784 | + elif [ $rc -eq $SKIPPED_EXIT_CODE ] |
785 | + then |
786 | + sreason="" |
787 | + test -r $skip_file && sreason=`cat $skip_file` |
788 | + |
789 | + SKIPPED_COUNT=$((SKIPPED_COUNT + 1)) |
790 | + SKIPPED_TESTS="$SKIPPED_TESTS $tname" |
791 | + |
792 | + echo "[skipped] $sreason" |
793 | + |
794 | + subunit_skip_test $tpath >> $SUBUNIT_OUT |
795 | + |
796 | + cleanup_worker $worker |
797 | + else |
798 | + echo "[failed] $test_time" |
799 | + |
800 | + ( |
801 | + (echo "Something went wrong running $tpath. Exited with $rc"; |
802 | + echo; echo; cat ${worker_outfiles[$worker]} |
803 | + ) | subunit_fail_test $tpath |
804 | + ) >> $SUBUNIT_OUT |
805 | + |
806 | + FAILED_COUNT=$((FAILED_COUNT + 1)) |
807 | + FAILED_TESTS="$FAILED_TESTS $tname" |
808 | + |
809 | + # Return 0 on failed tests in the -f mode |
810 | + if [ -z "$force" ] |
811 | + then |
812 | + |
813 | + if [ -z "$DEBUG" ] |
814 | + then |
815 | + cleanup_worker $worker |
816 | + else |
817 | + DEBUG_WORKER=$worker |
818 | + fi |
819 | + |
820 | + return 1 |
821 | + else |
822 | + cleanup_worker $worker |
823 | + |
824 | + return 0 |
825 | + fi |
826 | + fi |
827 | + |
828 | +} |
829 | + |
830 | +############################################################################# |
831 | +# Check if a specified worker has exceed TEST_TIMEOUT and if so, terminate it |
832 | +############################################################################# |
833 | +function check_timeout_for_worker() |
834 | +{ |
835 | + local worker=$1 |
836 | + local tpath=${worker_names[$worker]} |
837 | + local tname=`basename $tpath .sh` |
838 | + |
839 | + if (( `now` - worker_stime[$worker] > TEST_TIMEOUT )) |
840 | + then |
841 | + kill_worker $worker |
842 | + printf "%-40s w%d\t" $tname $worker |
843 | + echo "[failed] Timed out after $TEST_TIMEOUT seconds." |
844 | + |
845 | + ( |
846 | + # Have to call subunit_start_test here, as currently tests cannot be |
847 | + # interleaved in the subunit output |
848 | + subunit_start_test $tpath "${worker_stime_txt[$worker]}" |
849 | + |
850 | + (echo "Timeout exceeded running $tpath."; |
851 | + echo; echo; cat ${worker_outfiles[$worker]} |
852 | + ) | subunit_fail_test $tpath |
853 | + ) >> $SUBUNIT_OUT |
854 | + |
855 | + FAILED_COUNT=$((FAILED_COUNT + 1)) |
856 | + FAILED_TESTS="$FAILED_TESTS $tname" |
857 | + |
858 | + # Return 0 on failed tests in the -f mode |
859 | + if [ -z "$force" ] |
860 | + then |
861 | + |
862 | + if [ -z "$DEBUG" ] |
863 | + then |
864 | + cleanup_worker $worker |
865 | + else |
866 | + DEBUG_WORKER=$worker |
867 | + fi |
868 | + |
869 | + return 1 |
870 | + else |
871 | + cleanup_worker $worker |
872 | + |
873 | + return 0 |
874 | + fi |
875 | + fi |
876 | +} |
877 | +######################################################################### |
878 | +# Wait for all currently running worker to finish |
879 | +######################################################################### |
880 | +function reap_all_workers() |
881 | +{ |
882 | + while true |
883 | + do |
884 | + found="" |
885 | + |
886 | + for ((i = 1; i <= NWORKERS; i++)) |
887 | + do |
888 | + [ -z ${worker_pids[$i]:-""} ] && continue |
889 | + |
890 | + found="yes" |
891 | + |
892 | + # Check if it's alive |
893 | + if kill -0 ${worker_pids[$i]} >/dev/null 2>&1 |
894 | + then |
895 | + check_timeout_for_worker $i || print_status_and_exit |
896 | + continue |
897 | + fi |
898 | + |
899 | + reap_worker $i || print_status_and_exit |
900 | + |
901 | + worker_pids[$i]="" |
902 | + done |
903 | + |
904 | + [ -z "$found" ] && break |
905 | + |
906 | + sleep 1 |
907 | + done |
908 | +} |
909 | + |
910 | +######################################################################## |
911 | +# Release all port locks reserved by the current process |
912 | +# Used by the EXIT trap (in both normal and abnormal shell termination) |
913 | +######################################################################## |
914 | +function release_port_locks() |
915 | +{ |
916 | + local process=$1 |
917 | + local lockfile |
918 | + |
919 | + # Suppress errors when no port lock files are found |
920 | + shopt -s nullglob |
921 | + |
922 | + for lockfile in /tmp/xtrabackup_port_lock.* |
923 | + do |
924 | + if [ "`cat $lockfile 2>/dev/null`" = $process ] |
925 | + then |
926 | + rm -rf $lockfile |
927 | + fi |
928 | + done |
929 | + |
930 | + shopt -u nullglob |
931 | +} |
932 | + |
933 | +######################################################################## |
934 | +# Report status and release port locks on exit |
935 | +######################################################################## |
936 | +function cleanup_on_test_exit() |
937 | +{ |
938 | + local rc=$? |
939 | + |
940 | + echo $rc > $STATUS_FILE |
941 | + |
942 | + release_port_locks $$ |
943 | +} |
944 | + |
945 | +######################################################################## |
946 | +# Script body |
947 | +######################################################################## |
948 | + |
949 | +TEST_START_TIME=`now` |
950 | |
951 | tname="" |
952 | XTRACE_OPTION="" |
953 | XB_BUILD="autodetect" |
954 | force="" |
955 | -KEEP_RESULTS=0 |
956 | SUBUNIT_OUT=test_results.subunit |
957 | - |
958 | -while getopts "fgh?:t:s:d:c:b:nr:" options; do |
959 | - case $options in |
960 | - f ) force="yes";; |
961 | - t ) tname="$OPTARG";; |
962 | - g ) XTRACE_OPTION="-x"; DEBUG=on;; |
963 | - h ) usage; exit;; |
964 | - s ) tname="$OPTARG/*.sh";; |
965 | - d ) export MYSQL_BASEDIR="$OPTARG";; |
966 | - b ) TEST_BASEDIR="$OPTARG";; |
967 | - c ) XB_BUILD="$OPTARG";; |
968 | - n ) KEEP_RESULTS=1;; |
969 | - r ) SUBUNIT_OUT="$OPTARG";; |
970 | - ? ) echo "Use \`$0 -h' for the list of available options." |
971 | +NWORKERS= |
972 | +DEBUG_WORKER="" |
973 | + |
974 | +while getopts "fgh?:t:s:d:c:j:T:" options; do |
975 | + case $options in |
976 | + f ) force="yes";; |
977 | + t ) |
978 | + |
979 | + tname="$OPTARG"; |
980 | + if ! [ -r "$tname" ] |
981 | + then |
982 | + echo "Cannot find test $tname." |
983 | + exit -1 |
984 | + fi |
985 | + ;; |
986 | + |
987 | + g ) DEBUG=on;; |
988 | + h ) usage; exit;; |
989 | + s ) tname="$OPTARG/*.sh";; |
990 | + d ) export MYSQL_BASEDIR="$OPTARG";; |
991 | + c ) XB_BUILD="$OPTARG";; |
992 | + j ) |
993 | + |
994 | + if [[ ! $OPTARG =~ ^[0-9]+$ || $NWORKERS < 1 ]] |
995 | + then |
996 | + echo "Wrong -j argument: $OPTARG" |
997 | + exit -1 |
998 | + fi |
999 | + NWORKERS="$OPTARG" |
1000 | + ;; |
1001 | + |
1002 | + T ) |
1003 | + if [[ ! $OPTARG =~ ^[0-9]+$ ]] |
1004 | + then |
1005 | + echo "Wrong -T argument: $OPTARG" |
1006 | + exit -1 |
1007 | + fi |
1008 | + TEST_TIMEOUT="$OPTARG" |
1009 | + ;; |
1010 | + |
1011 | + ? ) echo "Use \`$0 -h' for the list of available options." |
1012 | exit -1;; |
1013 | - esac |
1014 | + esac |
1015 | done |
1016 | |
1017 | -if [ $KEEP_RESULTS -eq 0 ]; |
1018 | -then |
1019 | - rm -rf results |
1020 | - rm -f $SUBUNIT_OUT |
1021 | -fi |
1022 | -mkdir results |
1023 | - |
1024 | set_vars |
1025 | |
1026 | if [ -n "$tname" ] |
1027 | @@ -282,13 +805,17 @@ |
1028 | tests="t/*.sh" |
1029 | fi |
1030 | |
1031 | -failed_count=0 |
1032 | -failed_tests= |
1033 | -total_count=0 |
1034 | - |
1035 | export OUTFILE="$PWD/results/setup" |
1036 | |
1037 | -clean >>$OUTFILE 2>&1 |
1038 | +rm -rf results |
1039 | +mkdir results |
1040 | + |
1041 | +cleanup_all_workers >>$OUTFILE 2>&1 |
1042 | + |
1043 | +rm -rf var test_results.subunit |
1044 | +mkdir var |
1045 | + |
1046 | +echo "Detecting server version..." | tee -a $OUTFILE |
1047 | |
1048 | if ! get_version_info |
1049 | then |
1050 | @@ -300,69 +827,99 @@ |
1051 | tee -a $OUTFILE |
1052 | |
1053 | echo "Using '`basename $XB_BIN`' as xtrabackup binary" | tee -a $OUTFILE |
1054 | + |
1055 | +[ -z "$NWORKERS" ] && autocalc_nworkers |
1056 | + |
1057 | +if [ "$NWORKERS" -gt 1 ] |
1058 | +then |
1059 | + echo "Using $NWORKERS parallel workers" | tee -a $OUTFILE |
1060 | +fi |
1061 | echo | tee -a $OUTFILE |
1062 | |
1063 | -kill_leftovers >>$OUTFILE 2>&1 |
1064 | -clean >>$OUTFILE 2>&1 |
1065 | - |
1066 | -source subunit.sh |
1067 | - |
1068 | -echo "========================================================================" |
1069 | +cat <<EOF |
1070 | +============================================================================== |
1071 | +TEST WORKER RESULT TIME(s) or COMMENT |
1072 | +------------------------------------------------------------------------------ |
1073 | +EOF |
1074 | |
1075 | for t in $tests |
1076 | do |
1077 | - total_count=$((total_count+1)) |
1078 | - |
1079 | - printf "%-40s" $t |
1080 | - subunit_start_test $t >> $SUBUNIT_OUT |
1081 | + # Check if we have available workers |
1082 | + found="" |
1083 | + |
1084 | + while [ -z "$found" ] |
1085 | + do |
1086 | + for ((i = 1; i <= NWORKERS; i++)) |
1087 | + do |
1088 | + if [ -z ${worker_pids[$i]:-""} ] |
1089 | + then |
1090 | + found="yes" |
1091 | + break |
1092 | + else |
1093 | + # Check if it's alive |
1094 | + if kill -0 ${worker_pids[$i]} >/dev/null 2>&1 |
1095 | + then |
1096 | + check_timeout_for_worker $i || print_status_and_exit |
1097 | + continue |
1098 | + fi |
1099 | + reap_worker $i || print_status_and_exit |
1100 | + |
1101 | + worker_pids[$i]="" |
1102 | + found="yes" |
1103 | + |
1104 | + break |
1105 | + fi |
1106 | + done |
1107 | + |
1108 | + if [ -z "$found" ] |
1109 | + then |
1110 | + sleep 1 |
1111 | + fi |
1112 | + done |
1113 | + |
1114 | + worker=$i |
1115 | + |
1116 | + TOTAL_COUNT=$((TOTAL_COUNT+1)) |
1117 | + |
1118 | name=`basename $t .sh` |
1119 | - export OUTFILE="$PWD/results/$name" |
1120 | - export SKIPPED_REASON="$PWD/results/$name.skipped" |
1121 | - bash $XTRACE_OPTION $t > $OUTFILE 2>&1 |
1122 | - rc=$? |
1123 | - |
1124 | - if [[ -z "$DEBUG" || -n "$force" ]] |
1125 | - then |
1126 | - kill_leftovers >>$OUTFILE 2>&1 |
1127 | - clean >>$OUTFILE 2>&1 |
1128 | - fi |
1129 | - |
1130 | - if [ $rc -eq 0 ] |
1131 | - then |
1132 | - echo "[passed]" |
1133 | - subunit_pass_test $t >> $SUBUNIT_OUT |
1134 | - elif [ $rc -eq $SKIPPED_EXIT_CODE ] |
1135 | - then |
1136 | - sreason="" |
1137 | - test -r $SKIPPED_REASON && sreason=`cat $SKIPPED_REASON` |
1138 | - echo "[skipped] $sreason" |
1139 | - subunit_skip_test $t >> $SUBUNIT_OUT |
1140 | - else |
1141 | - echo "[failed]" |
1142 | - |
1143 | - ( |
1144 | - (echo "Something went wrong running $t. Exited with $rc"; |
1145 | - echo; echo; cat $OUTFILE |
1146 | - ) | subunit_fail_test $t |
1147 | - ) >> $SUBUNIT_OUT |
1148 | - |
1149 | - failed_count=$((failed_count+1)) |
1150 | - failed_tests="$failed_tests $t" |
1151 | - result=1 |
1152 | - if [ -z "$force" ] |
1153 | + worker_names[$worker]=$t |
1154 | + worker_outfiles[$worker]="$PWD/results/$name" |
1155 | + worker_skip_files[$worker]="$PWD/results/$name.skipped" |
1156 | + worker_status_files[$worker]="$PWD/results/$name.status" |
1157 | + # Create a unique TMPDIR for each worker so that it can be removed as a part |
1158 | + # of the cleanup procedure. Server socket files will also be created there. |
1159 | + worker_tmpdirs[$worker]="`mktemp -d -t xbtemp.XXXXXX`" |
1160 | + |
1161 | + ( |
1162 | + set -eu |
1163 | + if [ -n "$DEBUG" ] |
1164 | then |
1165 | - break; |
1166 | + set -x |
1167 | fi |
1168 | - fi |
1169 | + |
1170 | + trap "cleanup_on_test_exit" EXIT |
1171 | + |
1172 | + . inc/common.sh |
1173 | + |
1174 | + export OUTFILE=${worker_outfiles[$worker]} |
1175 | + export SKIPPED_REASON=${worker_skip_files[$worker]} |
1176 | + export TEST_VAR_ROOT=$TEST_BASEDIR/var/w$worker |
1177 | + export TMPDIR=${worker_tmpdirs[$worker]} |
1178 | + export STATUS_FILE=${worker_status_files[$worker]} |
1179 | + |
1180 | + mkdir $TEST_VAR_ROOT |
1181 | + |
1182 | + . $t |
1183 | + ) > ${worker_outfiles[$worker]} 2>&1 & |
1184 | + |
1185 | + worker_pids[$worker]=$! |
1186 | + worker_stime[$worker]="`now`" |
1187 | + # Used in subunit reports |
1188 | + worker_stime_txt[$worker]="`date -u '+%Y-%m-%d %H:%M:%S'`" |
1189 | done |
1190 | |
1191 | -echo "========================================================================" |
1192 | -echo |
1193 | - |
1194 | -if [ $result -eq 1 ] |
1195 | -then |
1196 | - echo |
1197 | - echo "$failed_count/$total_count tests have failed: $failed_tests" |
1198 | - echo "See results/ for detailed output" |
1199 | - exit -1 |
1200 | -fi |
1201 | +# Wait for in-progress workers to finish |
1202 | +reap_all_workers |
1203 | + |
1204 | +print_status_and_exit |
1205 | + |
1206 | |
1207 | === modified file 'test/subunit.sh' |
1208 | --- test/subunit.sh 2011-06-12 09:57:00 +0000 |
1209 | +++ test/subunit.sh 2013-07-04 07:45:32 +0000 |
1210 | @@ -15,8 +15,11 @@ |
1211 | # |
1212 | |
1213 | subunit_start_test () { |
1214 | + local stime=$2 |
1215 | # emit the current protocol start-marker for test $1 |
1216 | - echo "time: `date -u '+%Y-%m-%d %H:%M:%S'`" |
1217 | + # adjust time by the number of seconds to execute the test to get the actual |
1218 | + # start time |
1219 | + echo "time: $stime" |
1220 | echo "test: $1" |
1221 | } |
1222 | |
1223 | |
1224 | === modified file 'test/t/bug1085099.sh' |
1225 | --- test/t/bug1085099.sh 2013-03-06 10:04:13 +0000 |
1226 | +++ test/t/bug1085099.sh 2013-07-04 07:45:32 +0000 |
1227 | @@ -13,9 +13,7 @@ |
1228 | # Create a new tmpdir |
1229 | mkdir $topdir/new_tmpdir |
1230 | |
1231 | -# Make the default tmpdir inaccessible and make sure permissions are restored on |
1232 | -# failure |
1233 | -trap "chmod 755 $MYSQLD_TMPDIR" INT TERM EXIT |
1234 | +# Make the default tmpdir inaccessible |
1235 | chmod 000 $MYSQLD_TMPDIR |
1236 | |
1237 | innobackupex --tmpdir=$topdir/new_tmpdir --no-timestamp $topdir/backup |
1238 | |
1239 | === modified file 'test/t/bug664986.sh' |
1240 | --- test/t/bug664986.sh 2013-03-03 11:01:19 +0000 |
1241 | +++ test/t/bug664986.sh 2013-07-04 07:45:32 +0000 |
1242 | @@ -15,14 +15,6 @@ |
1243 | |
1244 | EOF |
1245 | |
1246 | -# Make sure we restore permissions to not leave stale files in Jenkins |
1247 | -# on a test failure |
1248 | -trap "vlog restoring directories and files permissions ; \ |
1249 | - chmod 777 $MYSQLD_DATADIR/test_bug664986_innodb; \ |
1250 | - chmod 777 $MYSQLD_DATADIR/test_bug664986_myisam; \ |
1251 | - chmod 644 $MYSQLD_DATADIR/test_bug664986_innodb/*; \ |
1252 | - chmod 644 $MYSQLD_DATADIR/test_bug664986_myisam/*" INT TERM EXIT |
1253 | - |
1254 | # Test that wrong directory permissions result in a backup failure |
1255 | # for both InnoDB and non-InnoDB files |
1256 | chmod 000 $MYSQLD_DATADIR/test_bug664986_innodb |
1257 | |
1258 | === modified file 'test/t/bug996493.sh' |
1259 | --- test/t/bug996493.sh 2012-11-29 04:37:56 +0000 |
1260 | +++ test/t/bug996493.sh 2013-07-04 07:45:32 +0000 |
1261 | @@ -20,9 +20,6 @@ |
1262 | vlog "Remove my.cnf" |
1263 | mv $topdir/my.cnf $topdir/my.cnf.bak |
1264 | |
1265 | -trap "vlog restoring $topdir/my.cnf ; \ |
1266 | -mv $topdir/my.cnf.bak $topdir/my.cnf" INT TERM EXIT |
1267 | - |
1268 | vlog "Apply log" |
1269 | # Do not run innobackupex, because it pass option --defaults-file |
1270 | # which we should avoid. Our goal is to test that backup-my.cnf |
1271 | @@ -31,7 +28,6 @@ |
1272 | |
1273 | vlog "Get my.cnf back" |
1274 | mv $topdir/my.cnf.bak $topdir/my.cnf |
1275 | -trap "vlog test exit" INT TERM EXIT |
1276 | |
1277 | vlog "Copy back" |
1278 | innobackupex --copy-back $backup_dir |
1279 | |
1280 | === modified file 'test/t/ib_stream_incremental.sh' |
1281 | --- test/t/ib_stream_incremental.sh 2012-06-05 12:35:33 +0000 |
1282 | +++ test/t/ib_stream_incremental.sh 2013-07-04 07:45:32 +0000 |
1283 | @@ -98,7 +98,7 @@ |
1284 | exit -1 |
1285 | fi |
1286 | |
1287 | - clean |
1288 | + remove_var_dirs |
1289 | } |
1290 | |
1291 | stream_format="xbstream" |
1292 | |
1293 | === modified file 'test/t/remote_tablespaces.sh' |
1294 | --- test/t/remote_tablespaces.sh 2013-04-23 06:38:06 +0000 |
1295 | +++ test/t/remote_tablespaces.sh 2013-07-04 07:45:32 +0000 |
1296 | @@ -12,7 +12,7 @@ |
1297 | |
1298 | start_server --innodb_file_per_table |
1299 | |
1300 | -remote_dir=$TEST_BASEDIR/var1/remote_dir |
1301 | +remote_dir=$TEST_VAR_ROOT/var1/remote_dir |
1302 | |
1303 | $MYSQL $MYSQL_ARGS test <<EOF |
1304 | CREATE TABLE t(id INT AUTO_INCREMENT PRIMARY KEY, c INT) |
1305 | |
1306 | === modified file 'test/t/undo_tablespaces.sh' |
1307 | --- test/t/undo_tablespaces.sh 2013-04-27 18:46:54 +0000 |
1308 | +++ test/t/undo_tablespaces.sh 2013-07-04 07:45:32 +0000 |
1309 | @@ -23,7 +23,7 @@ |
1310 | EOF |
1311 | } |
1312 | |
1313 | -undo_directory=$TEST_BASEDIR/var1/undo_dir |
1314 | +undo_directory=$TEST_VAR_ROOT/var1/undo_dir |
1315 | |
1316 | MYSQLD_EXTRA_MY_CNF_OPTS=" |
1317 | innodb_file_per_table=1 |
1318 | |
1319 | === modified file 'test/t/xb_part_range.sh' |
1320 | --- test/t/xb_part_range.sh 2012-07-06 05:13:15 +0000 |
1321 | +++ test/t/xb_part_range.sh 2013-07-04 07:45:32 +0000 |
1322 | @@ -1,19 +1,9 @@ |
1323 | . inc/common.sh |
1324 | +. inc/ib_part.sh |
1325 | |
1326 | start_server |
1327 | |
1328 | -function check_partitioning() |
1329 | -{ |
1330 | - $MYSQL $MYSQL_ARGS -Ns -e "show variables like 'have_partitioning'" |
1331 | -} |
1332 | - |
1333 | -PARTITION_CHECK=`check_partitioning` |
1334 | - |
1335 | -if [ -z "$PARTITION_CHECK" ]; then |
1336 | - echo "Requires Partitioning." > $SKIPPED_REASON |
1337 | - stop_server |
1338 | - exit $SKIPPED_EXIT_CODE |
1339 | -fi |
1340 | +require_partitioning |
1341 | |
1342 | run_cmd $MYSQL $MYSQL_ARGS test <<EOF |
1343 | CREATE TABLE test ( |
1344 | |
1345 | === modified file 'test/t/xb_perm_basic.sh' |
1346 | --- test/t/xb_perm_basic.sh 2012-10-15 16:14:59 +0000 |
1347 | +++ test/t/xb_perm_basic.sh 2013-07-04 07:45:32 +0000 |
1348 | @@ -4,6 +4,4 @@ |
1349 | start_server |
1350 | |
1351 | chmod -R 500 $mysql_datadir |
1352 | -trap "vlog restoring permissions on datadir $mysql_datadir ; \ |
1353 | -chmod -R 700 $mysql_datadir" INT TERM EXIT |
1354 | innobackupex --no-timestamp $topdir/backup |
1355 | |
1356 | === modified file 'test/t/xb_perm_stream.sh' |
1357 | --- test/t/xb_perm_stream.sh 2012-10-15 16:14:59 +0000 |
1358 | +++ test/t/xb_perm_stream.sh 2013-07-04 07:45:32 +0000 |
1359 | @@ -5,7 +5,5 @@ |
1360 | |
1361 | # Take backup |
1362 | chmod -R 500 $mysql_datadir |
1363 | -trap "vlog restoring permissions on datadir $mysql_datadir ; \ |
1364 | -chmod -R 700 $mysql_datadir" INT TERM EXIT |
1365 | mkdir -p $topdir/backup |
1366 | innobackupex --stream=tar $topdir/backup > $topdir/backup/out.tar |
1367 | |
1368 | === modified file 'test/t/xb_stats_datadir.sh' |
1369 | --- test/t/xb_stats_datadir.sh 2013-05-02 13:05:59 +0000 |
1370 | +++ test/t/xb_stats_datadir.sh 2013-07-04 07:45:32 +0000 |
1371 | @@ -5,12 +5,11 @@ |
1372 | |
1373 | . inc/common.sh |
1374 | |
1375 | -mkdir ${TEST_BASEDIR}/logs |
1376 | - |
1377 | -trap "rm -rf ${TEST_BASEDIR}/logs" INT TERM EXIT |
1378 | +logdir=${TEST_VAR_ROOT}/logs |
1379 | +mkdir $logdir |
1380 | |
1381 | MYSQLD_EXTRA_MY_CNF_OPTS=" |
1382 | -innodb_log_group_home_dir=${TEST_BASEDIR}/logs |
1383 | +innodb_log_group_home_dir=$logdir |
1384 | " |
1385 | |
1386 | start_server |
1387 | @@ -28,6 +27,6 @@ |
1388 | # we pass all necessary options as an arguments, so if someday this |
1389 | # will be changed, test still will work |
1390 | xtrabackup --stats --datadir=${MYSQLD_DATADIR} \ |
1391 | - --innodb_log_group_home_dir=${TEST_BASEDIR}/logs |
1392 | + --innodb_log_group_home_dir=$logdir |
1393 | |
1394 | vlog "stats did not fail" |
1395 | |
1396 | === removed file 'test/testrun.c' |
1397 | --- test/testrun.c 2012-11-30 04:28:06 +0000 |
1398 | +++ test/testrun.c 1970-01-01 00:00:00 +0000 |
1399 | @@ -1,405 +0,0 @@ |
1400 | -/* |
1401 | - * testrun.c - a parallel test runner for the XtraBackup test suite |
1402 | - */ |
1403 | -/* BEGIN LICENSE |
1404 | - * Copyright (C) 2012 Percona Inc. |
1405 | - * |
1406 | - * Written by Stewart Smith |
1407 | - * |
1408 | - * This program is free software: you can redistribute it and/or modify it |
1409 | - * under the terms of the GNU General Public License version 2, as published |
1410 | - * by the Free Software Foundation. |
1411 | - * |
1412 | - * This program is distributed in the hope that it will be useful, but |
1413 | - * WITHOUT ANY WARRANTY; without even the implied warranties of |
1414 | - * MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR |
1415 | - * PURPOSE. See the GNU General Public License for more details. |
1416 | - * |
1417 | - * You should have received a copy of the GNU General Public License along |
1418 | - * with this program. If not, see <http://www.gnu.org/licenses/>. |
1419 | - * END LICENSE */ |
1420 | - |
1421 | -#include <stdio.h> |
1422 | -#include <unistd.h> |
1423 | -#include <dirent.h> |
1424 | -#include <string.h> |
1425 | -#include <limits.h> |
1426 | -#include <stdlib.h> |
1427 | -#include <sys/select.h> |
1428 | - |
1429 | -#include <sys/stat.h> |
1430 | -#include <sys/types.h> |
1431 | -#include <sys/wait.h> |
1432 | -#include <fcntl.h> |
1433 | -#include <assert.h> |
1434 | - |
1435 | -#include <signal.h> |
1436 | - |
1437 | -struct testcase |
1438 | -{ |
1439 | - char *name; |
1440 | - char *buf; |
1441 | - size_t end_of_buf; |
1442 | - size_t bufsz; |
1443 | -}; |
1444 | - |
1445 | -pid_t *childpid; |
1446 | -int nchildpid=0; |
1447 | - |
1448 | -static void kill_children(int sig) |
1449 | -{ |
1450 | - int i; |
1451 | - int status; |
1452 | - |
1453 | - (void)sig; |
1454 | - |
1455 | - fprintf(stderr, "Killing child processes...\n"); |
1456 | - |
1457 | - for(i=0; i<nchildpid; i++) |
1458 | - if(childpid[i] > 0) |
1459 | - { |
1460 | - kill(childpid[i], SIGKILL); |
1461 | - wait(&status); |
1462 | - } |
1463 | - |
1464 | - exit(EXIT_SUCCESS); |
1465 | -} |
1466 | - |
1467 | -int collect_testcases_filter(const struct dirent *a) |
1468 | -{ |
1469 | - int l; |
1470 | - |
1471 | - if (a->d_name[0] == '.') |
1472 | - return 0; |
1473 | - |
1474 | - l= strlen(a->d_name); |
1475 | - |
1476 | - if (l > 2 && strncmp(a->d_name + l - 3, ".sh", 3)==0) |
1477 | - return 1; |
1478 | - |
1479 | - return 0; |
1480 | -} |
1481 | - |
1482 | -static int collect_testcases(const char* suitedir, struct testcase **cases) |
1483 | -{ |
1484 | - struct dirent **namelist; |
1485 | - int n; |
1486 | - int i; |
1487 | - |
1488 | - n= scandir(suitedir, &namelist, collect_testcases_filter, alphasort); |
1489 | - |
1490 | - *cases= (struct testcase*) malloc(sizeof(struct testcase)*n); |
1491 | - |
1492 | - for(i=0; i<n; i++) |
1493 | - { |
1494 | - (*cases)[i].name= strdup(namelist[i]->d_name); |
1495 | - (*cases)[i].buf= NULL; |
1496 | - (*cases)[i].end_of_buf= 0; |
1497 | - (*cases)[i].bufsz= 0; |
1498 | - free(namelist[i]); |
1499 | - } |
1500 | - |
1501 | - free(namelist); |
1502 | - |
1503 | - return n; |
1504 | -} |
1505 | - |
1506 | -static void free_testcases(struct testcase *cases, int n) |
1507 | -{ |
1508 | - while (n>0) |
1509 | - { |
1510 | - free(cases[--n].name); |
1511 | - } |
1512 | - free(cases); |
1513 | -} |
1514 | - |
1515 | -static int run_testcase_in_child(int nr, struct testcase *t, pid_t *cpid, const char* xbtarget) |
1516 | -{ |
1517 | - int fd[2]; |
1518 | - |
1519 | - printf("[%d] LAUNCHING - %s\n", nr, t->name); |
1520 | - |
1521 | - if (pipe(fd) == -1) |
1522 | - { |
1523 | - perror("pipe"); |
1524 | - exit(EXIT_FAILURE); |
1525 | - } |
1526 | - |
1527 | - *cpid= fork(); |
1528 | - if (*cpid == 0) |
1529 | - { |
1530 | - /* child */ |
1531 | - close(fd[0]); |
1532 | - |
1533 | - char tname[500]; |
1534 | - snprintf(tname, sizeof(tname), "t/%s",t->name); |
1535 | - |
1536 | - char basedir[PATH_MAX]; |
1537 | - char cwd[PATH_MAX]; |
1538 | - snprintf(basedir, sizeof(basedir), "%s/var/%d", getcwd(cwd,sizeof(cwd)), nr); |
1539 | - |
1540 | - mkdir("var",0700); |
1541 | - mkdir(basedir,0700); |
1542 | - |
1543 | - char logname[PATH_MAX]; |
1544 | - snprintf(logname, sizeof(logname), "%s/var/%d.log", getcwd(cwd,sizeof(cwd)), nr); |
1545 | - |
1546 | - int logfd= open(logname, O_WRONLY|O_APPEND|O_CREAT, 0600); |
1547 | - dup2(logfd, STDOUT_FILENO); |
1548 | - dup2(logfd, STDERR_FILENO); |
1549 | - close(logfd); |
1550 | - |
1551 | - char subunitfd[50]; |
1552 | - snprintf(subunitfd, sizeof(subunitfd), "/dev/fd/%d", fd[1]); |
1553 | - |
1554 | - char* xbtarget_param; |
1555 | - if (xbtarget) |
1556 | - xbtarget_param= strdup(xbtarget); |
1557 | - else |
1558 | - xbtarget_param= NULL; |
1559 | - |
1560 | - char *const newargv[] = {"testrun.sh", "-n", |
1561 | - "-t", tname, |
1562 | - "-b", basedir, |
1563 | - "-r", subunitfd, |
1564 | - (xbtarget)? "-c" : NULL, xbtarget_param, |
1565 | - NULL }; |
1566 | - char *newenviron[] = { NULL }; |
1567 | - execve(newargv[0], newargv, newenviron); |
1568 | - perror("execve"); |
1569 | - exit(EXIT_FAILURE); |
1570 | - } |
1571 | - else |
1572 | - { |
1573 | - /* parent */ |
1574 | - close(fd[1]); |
1575 | - fcntl(fd[0], F_SETFL, O_NONBLOCK); |
1576 | - return fd[0]; |
1577 | - } |
1578 | -} |
1579 | - |
1580 | -static inline void subunit_progress_sign(int fd, int n, char *sign) |
1581 | -{ |
1582 | - char *buf; |
1583 | - const char* fmt= "progress: %s%d\n"; |
1584 | - size_t sz= 1+strlen(fmt)+100; |
1585 | - size_t l; |
1586 | - |
1587 | - buf= (char*)malloc(sz); |
1588 | - |
1589 | - l= snprintf(buf, sz, fmt, sign, n); |
1590 | - assert(l < sz); |
1591 | - |
1592 | - write(fd, buf, l); |
1593 | - |
1594 | - free(buf); |
1595 | -} |
1596 | - |
1597 | -static void run_testcases(struct testcase *testcases, int nrcases, |
1598 | - int njobs, int timeout, const char* xbtarget) |
1599 | -{ |
1600 | - int childfd[njobs]; |
1601 | - int nfds= 0; |
1602 | - int retval; |
1603 | - pid_t chpid[njobs]; |
1604 | - int status; |
1605 | - int next_testcase= 0; |
1606 | - int i; |
1607 | - fd_set rfds; |
1608 | - fd_set efds; |
1609 | - struct timeval tv; |
1610 | - int nchildren; |
1611 | - int childtest[njobs]; |
1612 | - |
1613 | - int subunitfd= open("test_results.subunit", O_TRUNC|O_WRONLY|O_APPEND|O_CREAT, 0600); |
1614 | - subunit_progress_sign(subunitfd, nrcases, ""); |
1615 | - |
1616 | - if (nrcases < njobs) |
1617 | - njobs= nrcases; |
1618 | - |
1619 | - childpid= chpid; |
1620 | - nchildpid= njobs; |
1621 | - |
1622 | - for(i=0; i<njobs; i++) |
1623 | - { |
1624 | - childtest[i]=next_testcase++; |
1625 | - childfd[i]= run_testcase_in_child(i, &testcases[childtest[i]], &childpid[i], xbtarget); |
1626 | - } |
1627 | - |
1628 | - fflush(stdout); |
1629 | - |
1630 | -loop: |
1631 | - FD_ZERO(&efds); |
1632 | - FD_ZERO(&rfds); |
1633 | - |
1634 | - nchildren=0; |
1635 | - |
1636 | - for (i=0; i<njobs; i++) |
1637 | - { |
1638 | - if (childfd[i] != -1) |
1639 | - { |
1640 | - FD_SET(childfd[i], &efds); |
1641 | - FD_SET(childfd[i], &rfds); |
1642 | - nfds= (childfd[i] > nfds)? childfd[i] : nfds; |
1643 | - nchildren++; |
1644 | - } |
1645 | - } |
1646 | - |
1647 | - tv.tv_sec= timeout; |
1648 | - tv.tv_usec= 0; |
1649 | - |
1650 | - retval = select(nfds+1, &rfds, NULL, &efds, &tv); |
1651 | - |
1652 | - if (retval == -1) |
1653 | - perror("select()"); |
1654 | - else if (retval) |
1655 | - { |
1656 | - int childexited=0; |
1657 | - |
1658 | - for (i=0; i<njobs; i++) |
1659 | - { |
1660 | - if(childfd[i] != -1 |
1661 | - && (FD_ISSET(childfd[i], &efds) || FD_ISSET(childfd[i], &rfds))) |
1662 | - { |
1663 | - ssize_t r=0; |
1664 | - |
1665 | - do { |
1666 | - struct testcase *t= &testcases[childtest[i]]; |
1667 | - |
1668 | - if(t->bufsz == t->end_of_buf) |
1669 | - { |
1670 | - t->bufsz+=4000; |
1671 | - t->buf= (char*)realloc(t->buf, t->bufsz); |
1672 | - } |
1673 | - |
1674 | - r= read(childfd[i], t->buf+t->end_of_buf, t->bufsz - t->end_of_buf); |
1675 | - if (r>0) |
1676 | - t->end_of_buf+= r; |
1677 | - } while(r>0); |
1678 | - |
1679 | - pid_t waited= waitpid(childpid[i], &status, WNOHANG); |
1680 | - if (!(WIFEXITED(status) || WIFSIGNALED(status))) |
1681 | - continue; |
1682 | - |
1683 | - if (waited != childpid[i]) |
1684 | - continue; |
1685 | - |
1686 | - write(subunitfd, testcases[childtest[i]].buf, |
1687 | - testcases[childtest[i]].end_of_buf); |
1688 | - |
1689 | - close(childfd[i]); |
1690 | - printf("[%d] completed %s status %d\n", |
1691 | - i, testcases[childtest[i]].name, WEXITSTATUS(status)); |
1692 | - childfd[i]=-1; |
1693 | - nchildren--; |
1694 | - |
1695 | - if (next_testcase < nrcases) |
1696 | - { |
1697 | - childtest[i]=next_testcase++; |
1698 | - childfd[i]= run_testcase_in_child(i, &testcases[childtest[i]], &childpid[i], xbtarget); |
1699 | - nfds= (childfd[i] > nfds)? childfd[i] : nfds; |
1700 | - nchildren++; |
1701 | - } |
1702 | - printf("\nnrchildren= %d, %d tests remaining\n", |
1703 | - nchildren, nrcases-next_testcase); |
1704 | - childexited=1; |
1705 | - fflush(stdout); |
1706 | - } |
1707 | - } |
1708 | - if (childexited) |
1709 | - { |
1710 | - printf ("Running: "); |
1711 | - for(i=0; i<njobs; i++) |
1712 | - if (childfd[i] != -1) |
1713 | - printf("%s ",testcases[childtest[i]].name); |
1714 | - printf("\n"); |
1715 | - } |
1716 | - } |
1717 | - else |
1718 | - { |
1719 | - printf("Timeout\n"); |
1720 | - kill_children(SIGKILL); |
1721 | - exit(EXIT_FAILURE); |
1722 | - } |
1723 | - |
1724 | - if (nchildren==0) |
1725 | - goto end; |
1726 | - |
1727 | - goto loop; |
1728 | - |
1729 | -end: |
1730 | - |
1731 | - close(subunitfd); |
1732 | - return; |
1733 | -} |
1734 | - |
1735 | -int main(int argc, char* argv[]) |
1736 | -{ |
1737 | - const char* suitedir= "t/"; |
1738 | - int njobs= 4; |
1739 | - int opt; |
1740 | - int nrcases; |
1741 | - struct testcase *testcases; |
1742 | - int timeout= 600; |
1743 | - const char* xbtarget= NULL; |
1744 | - |
1745 | - struct sigaction sa; |
1746 | - |
1747 | - sa.sa_flags= 0; |
1748 | - sigemptyset(&sa.sa_mask); |
1749 | - sa.sa_handler = kill_children; |
1750 | - sigaction(SIGHUP, &sa, NULL); |
1751 | - sigaction(SIGINT, &sa, NULL); |
1752 | - sigaction(SIGQUIT, &sa, NULL); |
1753 | - |
1754 | -#ifdef _SC_NPROCESSORS_ONLN |
1755 | - njobs= sysconf(_SC_NPROCESSORS_ONLN) /2; |
1756 | -#endif |
1757 | - |
1758 | -#ifdef _SC_PHYS_PAGES |
1759 | - long nrpages= sysconf(_SC_PHYS_PAGES); |
1760 | - long pagesize= sysconf(_SC_PAGESIZE); |
1761 | - long pages_per_job= (128*(1 << 20)) / pagesize; |
1762 | - nrpages= nrpages/2; |
1763 | - if ((pages_per_job * njobs) > nrpages) |
1764 | - njobs= nrpages / pages_per_job; |
1765 | -#endif |
1766 | - |
1767 | - if (njobs == 0) |
1768 | - njobs= 1; |
1769 | - |
1770 | - while ((opt = getopt(argc, argv, "j:s:t:c:")) != -1) |
1771 | - { |
1772 | - switch (opt) { |
1773 | - case 'c': |
1774 | - xbtarget= optarg; |
1775 | - break; |
1776 | - case 's': |
1777 | - suitedir= optarg; |
1778 | - break; |
1779 | - case 'j': |
1780 | - njobs= atoi(optarg); |
1781 | - break; |
1782 | - case 't': |
1783 | - timeout= atoi(optarg); |
1784 | - break; |
1785 | - default: |
1786 | - fprintf(stderr, "Usage: %s [-s suite] [-j parallel] [-t timeout]\n", |
1787 | - argv[0]); |
1788 | - exit(EXIT_FAILURE); |
1789 | - } |
1790 | - } |
1791 | - |
1792 | - printf("%s running with: -s %s -j %d -t %d\n\n", |
1793 | - argv[0], suitedir,njobs, timeout); |
1794 | - |
1795 | - nrcases= collect_testcases(suitedir, &testcases); |
1796 | - |
1797 | - printf("Found %d testcases\n", nrcases); |
1798 | - |
1799 | - run_testcases(testcases, nrcases, njobs, timeout, xbtarget); |
1800 | - |
1801 | - free_testcases(testcases, nrcases); |
1802 | - |
1803 | - return 0; |
1804 | -} |
http:// jenkins. percona. com/view/ XtraBackup/ job/percona- xtrabackup- 2.0-param/ 476/
xtradb56 failures on Lucid/Debian is bug #1193950 (will be fixed after updating links in bootstrap.sh)
xtradb56 failure on Centos5 is another (unreported) issue with PS 5.6.10 (should be fixed in later releases).
ddl.sh failure on Precise/Raring is bug #1098501.
galera55 failure on Quantal is JIRA JEN-60.
Debug build failures on Quantal/Raring is bug #1192454.