Merge lp:~laurynas-biveinis/percona-xtrabackup/bug1098498-1132763-2.0 into lp:percona-xtrabackup/2.0

Proposed by Laurynas Biveinis
Status: Merged
Approved by: Alexey Kopytov
Approved revision: no longer in the source branch.
Merged at revision: 513
Proposed branch: lp:~laurynas-biveinis/percona-xtrabackup/bug1098498-1132763-2.0
Merge into: lp:percona-xtrabackup/2.0
Diff against target: 450 lines (+58/-201)
14 files modified
test/inc/common.sh (+23/-19)
test/inc/ib_part.sh (+16/-2)
test/t/bug766033.sh (+1/-1)
test/t/bug766607.sh (+9/-12)
test/t/ib_part_databases.sh (+2/-0)
test/t/ib_part_include.sh (+1/-4)
test/t/ib_part_include_stream.sh (+1/-1)
test/t/ib_part_tf_innodb.sh (+1/-0)
test/t/ib_part_tf_innodb_stream.sh (+1/-0)
test/t/ib_part_tf_myisam.sh (+1/-0)
test/t/xb_export.sh (+1/-1)
test/t/xb_incremental.sh (+0/-106)
test/t/xb_partial.sh (+0/-53)
test/testrun.sh (+1/-2)
To merge this branch: bzr merge lp:~laurynas-biveinis/percona-xtrabackup/bug1098498-1132763-2.0
Reviewer Review Type Date Requested Status
Alexey Kopytov (community) Approve
George Ormond Lorch III g2 Pending
Review via email: mp+152262@code.launchpad.net

This proposal supersedes a proposal from 2013-03-04.

Description of the change

Take 3:
Addressed the review comments.
http://jenkins.percona.com/job/percona-xtrabackup-2.0-param/363/
BT 28340.

Take 2:
http://jenkins.percona.com/job/percona-xtrabackup-2.0-param/347/

Alexey's suggestion of killing always is implemented. But xb_export.sh still needs a shutdown due to what seems to be a table import crash-non-safety. Also, not all the testcases have the server datadirs removed before the restore. Bug reports pending for both of these issues.

Take 1:
No BT or ST but is one of the prerequisites for BT 28340.
http://jenkins.percona.com/job/percona-xtrabackup-2.0-param/342/

Fix bug 1098498 (Sporadic xb_incremental_compressed_* failures in
Jenkins) and bug 1132763 (No need to shutdown servers cleanly before
wiping their datadirs in the testsuite).

Most tests shutdown the server cleanly before copying back the
restored datadir. These shutdowns may:
1) timeout sporadically (bug 1098498);
2) are not required in the first place (bug 1132763), because the
server data is about to be wiped, thus kill -9 would work just as
fine.

Fixed by
1) introducing new function kill_server() in test/inc/common.sh that
kill -9s, removes the PID file, releases the mysqld port and testsuite
server variables.
2) removing the shutdown timeout and kill-if-not-shutdown logic from
stop_server_with_id(). For those remaining cases where the server is
required to stop cleanly, kill is a wrong substitute, and the shutdown
timeouts can be taken care of by the overall testcase timeouts.
3) reviewing the testcases and replacing stop_server calls with
kill_server where safe.
4) fixing a typo in testrun.sh and using kill_server() in
get_version_info() too.
5) removing an unused function stop_all_servers() in common.sh.

To post a comment you must log in.
Revision history for this message
George Ormond Lorch III (gl-az) wrote : Posted in a previous version of this proposal

Nice work. I had started on something similar but like your solution better.

review: Approve (g2)
Revision history for this message
Alexey Kopytov (akopytov) wrote : Posted in a previous version of this proposal

I went through the remaining stop_server occurrences and I see no reason to keep them. But then, if we don't really need stop_server, why not just make it do "kill -9" and avoid mass changes in test files?

*If* we ever need to stop the server and keep its data in a test case, then we can add either a new function, or a flag to stop_server.

review: Needs Fixing
Revision history for this message
Laurynas Biveinis (laurynas-biveinis) wrote : Posted in a previous version of this proposal

Alexey -

We do need to keep the current stop_server in its remaining uses. These are the partial backups, where the old datadir is not wiped, and a corrupted ibd backup test - where a server kill precludes a checkpoint and apparently crash recovery fixes the corruption attempt.

If we renamed kill_server to stop_server and stop_server to shutdown_server, then the diff would be a bit smaller. What do you think?

Revision history for this message
Alexey Kopytov (akopytov) wrote : Posted in a previous version of this proposal

The old datadir is wiped in partial backup tests, as you cannot restore a part of the dataset, you have to overwrite it.

In the corrupted ibd backup test, xtrabackup is started right after the server shutdown, so no crash recovery occurs.

Revision history for this message
Laurynas Biveinis (laurynas-biveinis) wrote : Posted in a previous version of this proposal

Alexey -

In the partial backup testcases I reviewed I found the datadir wipe to be partial too. E.g. ib_part_tf_innodb.sh, after stopping the server, calls ib_part_restore, which takes care to wipe the datadir selectively only:

 rm -rf $mysql_datadir/test/*
 rm -rf $topdir/ext/*

It is possible that there are bugs too. Here is an example of xb_partial.sh failure when the server is killed instead of stopped: http://jenkins.percona.com/job/percona-xtrabackup-2.0-param/341/BUILD_TYPE=release,Host=debian6-32,xtrabackuptarget=innodb51_builtin/testReport/junit/%28root%29/t_xb_partial/sh/

Likewise for bug722638.sh: http://jenkins.percona.com/job/percona-xtrabackup-2.0-param/333/BUILD_TYPE=release,Host=ubuntu-precise-32bit,xtrabackuptarget=galera55/testReport/junit/%28root%29/t_bug722638/sh/

I cannot find the bug766033.sh failure I have mentioned, but I believe that something is wrong there, I did see that xtrabackup passed with a corrupted ibd. Not the crash recovery though.

If what I'm saying is correct, then we have to keep stop_server() for the partial-restore testcases, and we have to examine in more detail the rest. But the latter may result in bug reports, and, until they are fixed, they should keep on using stop_server().

Revision history for this message
Alexey Kopytov (akopytov) wrote : Posted in a previous version of this proposal

The partial datadir wipe in in_part is wrong, it should remove it fully and restore from the backup directory. You can fix it in the same MP.

The second link for bug722638.sh failure is Galera-specific, and has nothing to do with the server shutdown. Bug #1098501.

The point is, until XtraBackup supports offline backups, it should never rely on the datadir state after stopping the server. If that's not true, those are test suite bugs and should be fixed together with this change.

Revision history for this message
Laurynas Biveinis (laurynas-biveinis) wrote : Posted in a previous version of this proposal

Ack. Sorry for the Galera, I failed to noticed that somehow.

For the record, there are server restarts to force a checkpoint: bug766607, tar4ibd_symlink, xb_export. These can be replaced with kill + start for the same effect.

I will replace all shutdowns with kills then.

Revision history for this message
Laurynas Biveinis (laurynas-biveinis) wrote : Posted in a previous version of this proposal

Actual BT 28340.

Revision history for this message
George Ormond Lorch III (gl-az) : Posted in a previous version of this proposal
review: Approve (g2)
Revision history for this message
Alexey Kopytov (akopytov) wrote : Posted in a previous version of this proposal

- I don't understand the reason to call free_reserved_port and reset_server_variables only if the PID file exists in stop_server / shutdown_server. Don't we still want to cleanup in case the PID file disappeared for whatever reasons?

- Was the following change really necessary?

128 -dd if=/dev/zero of=$mysql_datadir/sakila/rental.ibd seek=1000 count=1
129 +printf '\xAA\xAA\xAA\xAA' | dd of=$mysql_datadir/sakila/rental.ibd seek=16384 count=4

- Was the following change really necessary?

215 # also test xtrabackup --stats work with --tables-file
216 -COUNT=`xtrabackup --stats --tables='test.test$' --datadir=$topdir/backup \
217 +COUNT=`xtrabackup --stats --tables='^(mysql.*|performance_schema.*|test.test)$' --datadir=$topdir/backup \

review: Needs Fixing
Revision history for this message
Laurynas Biveinis (laurynas-biveinis) wrote : Posted in a previous version of this proposal

> - I don't understand the reason to call free_reserved_port and
> reset_server_variables only if the PID file exists in stop_server /
> shutdown_server. Don't we still want to cleanup in case the PID file
> disappeared for whatever reasons?

I'll revert it as it would make for a smaller diff. But I am not sure what would be the whatever reasons for PID to disappear. If this is a "can't happen" situation, then it's probably the case that the testsuite run wouldn't be able to recover to continue running for trusted results anyway?

> - Was the following change really necessary?
>
> 128 -dd if=/dev/zero of=$mysql_datadir/sakila/rental.ibd seek=1000 count=1
> 129 +printf '\xAA\xAA\xAA\xAA' | dd of=$mysql_datadir/sakila/rental.ibd
> seek=16384 count=4

I described it in the commit message:

"2) bug766033: replace the database corruption of poking one zero byte
to the first page with overwriting the start of the second page with
0xAAAAAAAA to reduce the risk of the previous corruption attempt not
corrupting anything due to the zero byte already existing at the
location."

Replacing the shutdown with the crash and using the original corruption attempt makes that testcase pass reliably. Poking a single zero byte to a first tablespace page doesn't look like a very reliable corruption attempt to me due to the chances that that byte will be zero already, which is presumably what happens with the crashed server.

> - Was the following change really necessary?
>
> 215 # also test xtrabackup --stats work with --tables-file
> 216 -COUNT=`xtrabackup --stats --tables='test.test$'
> --datadir=$topdir/backup \
> 217 +COUNT=`xtrabackup --stats
> --tables='^(mysql.*|performance_schema.*|test.test)$' --datadir=$topdir/backup
> \

No, thank you.

Revision history for this message
Laurynas Biveinis (laurynas-biveinis) wrote : Posted in a previous version of this proposal

> Replacing the shutdown with the crash and using the original corruption
> attempt makes that testcase pass reliably.

Err, "makes xtrabackup pass and the testcase fail reliably".

Revision history for this message
Alexey Kopytov (akopytov) wrote : Posted in a previous version of this proposal

Hi Laurynas,

On Wed, 06 Mar 2013 12:45:33 -0000, Laurynas Biveinis wrote:
>> - I don't understand the reason to call free_reserved_port and
>> reset_server_variables only if the PID file exists in stop_server /
>> shutdown_server. Don't we still want to cleanup in case the PID file
>> disappeared for whatever reasons?
>
> I'll revert it as it would make for a smaller diff. But I am not sure what would be the whatever reasons for PID to disappear. If this is a "can't happen" situation, then it's probably the case that the testsuite run wouldn't be able to recover to continue running for trusted results anyway?
>

I don't know. A test case removing the PID file or its directory for
some evil server/backup torturing? Or shutting down the server on its
own with mysqladmin shutdown for the same reason?

Just don't see any sense in avoiding those calls in case of a missing
PID file. It looks less reliable this way.

>> - Was the following change really necessary?
>>
>> 128 -dd if=/dev/zero of=$mysql_datadir/sakila/rental.ibd seek=1000 count=1
>> 129 +printf '\xAA\xAA\xAA\xAA' | dd of=$mysql_datadir/sakila/rental.ibd
>> seek=16384 count=4
>
> I described it in the commit message:
>
> "2) bug766033: replace the database corruption of poking one zero byte
> to the first page with overwriting the start of the second page with
> 0xAAAAAAAA to reduce the risk of the previous corruption attempt not
> corrupting anything due to the zero byte already existing at the
> location."
>
> Replacing the shutdown with the crash and using the original corruption attempt makes that testcase pass reliably. Poking a single zero byte to a first tablespace page doesn't look like a very reliable corruption attempt to me due to the chances that that byte will be zero already, which is presumably what happens with the crashed server.
>

OK. The reason I asked was that the code is not portable (dd semantics
is different on OSX/BSD). So the test case was corrupting another page
and worked mostly by accident on anything but Linux.

But I have checked it on my laptop, and the new code still works by
accident on OSX/BSD.

>> - Was the following change really necessary?
>>
>> 215 # also test xtrabackup --stats work with --tables-file
>> 216 -COUNT=`xtrabackup --stats --tables='test.test$'
>> --datadir=$topdir/backup \
>> 217 +COUNT=`xtrabackup --stats
>> --tables='^(mysql.*|performance_schema.*|test.test)$' --datadir=$topdir/backup
>> \
>
> No, thank you.
>

Revision history for this message
Alexey Kopytov (akopytov) :
review: Approve
Revision history for this message
Laurynas Biveinis (laurynas-biveinis) wrote :

All the issues described in the MP and its reviews have their bug reports now:
1) testcases that don't wipe the server dirs before restore: https://bugs.launchpad.net/percona-xtrabackup/+bug/1155076;
2) tablespace import not crash safe: https://bugs.launchpad.net/percona-server/+bug/1000224;
3) dd nonportable: https://bugs.launchpad.net/percona-xtrabackup/+bug/1155079

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'test/inc/common.sh'
2--- test/inc/common.sh 2013-01-15 01:39:48 +0000
3+++ test/inc/common.sh 2013-03-07 19:24:26 +0000
4@@ -283,28 +283,22 @@
5 }
6
7 ########################################################################
8-# Stop server with the id specified as the first argument and additional
9-# command line arguments (if specified)
10+# Stop server with the id specified as the first argument. The server
11+# is stopped in the fastest possible way.
12 ########################################################################
13 function stop_server_with_id()
14 {
15 local id=$1
16 switch_server $id
17
18- vlog "Stopping server with id=$id..."
19+ vlog "Killing server with id=$id..."
20
21 if [ -f "${MYSQLD_PIDFILE}" ]
22 then
23- ${MYSQLADMIN} ${MYSQL_ARGS} --shutdown-timeout=60 shutdown
24- if [ -f "${MYSQLD_PIDFILE}" ]
25- then
26- vlog "Could not stop the server with id=$id, using kill -9"
27- kill -9 `cat ${MYSQLD_PIDFILE}`
28- rm -f ${MYSQLD_PIDFILE}
29- fi
30- vlog "Server with id=$id has been stopped"
31+ kill -9 `cat ${MYSQLD_PIDFILE}`
32+ rm -f ${MYSQLD_PIDFILE}
33 else
34- vlog "Server pid file '${MYSQLD_PIDFILE}' doesn't exist!"
35+ vlog "Server PID file '${MYSQLD_PIDFILE}' doesn't exist!"
36 fi
37
38 # unlock the port number
39@@ -331,15 +325,25 @@
40 }
41
42 ########################################################################
43-# Stop all running servers
44+# Shutdown server with id=1 cleanly
45 ########################################################################
46-function stop_all_servers()
47+function shutdown_server()
48 {
49- local id
50- for id in ${SRV_MYSQLD_IDS[*]}
51- do
52- stop_server_with_id ${SRV_MYSQLD_IDS[$id]}
53- done
54+ switch_server 1
55+
56+ vlog "Shutting down server with id=1..."
57+
58+ if [ -f "${MYSQLD_PIDFILE}" ]
59+ then
60+ ${MYSQLADMIN} ${MYSQL_ARGS} shutdown
61+ else
62+ vlog "Server PID file '${MYSQLD_PIDFILE}' doesn't exist!"
63+ fi
64+
65+ # unlock the port number
66+ free_reserved_port $MYSQLD_PORT
67+
68+ reset_server_variables 1
69 }
70
71 ########################################################################
72
73=== modified file 'test/inc/ib_part.sh'
74--- test/inc/ib_part.sh 2013-01-02 15:52:13 +0000
75+++ test/inc/ib_part.sh 2013-03-07 19:24:26 +0000
76@@ -56,18 +56,32 @@
77 ib_part_data $topdir $engine | run_cmd $MYSQL $MYSQL_ARGS test
78 }
79
80+function ib_part_add_mandatory_tables()
81+{
82+ local mysql_datadir=$1
83+ local tables_file=$2
84+ for table in $mysql_datadir/mysql/*.frm
85+ do
86+ echo mysql.`basename $table` >> $tables_file
87+ done
88+ for table in $mysql_datadir/performance_schema/*.frm
89+ do
90+ echo performance_schema.`basename $table` >> $tables_file
91+ done
92+}
93+
94 function ib_part_restore()
95 {
96 topdir=$1
97 mysql_datadir=$2
98
99 # Remove database
100- rm -rf $mysql_datadir/test/*
101+ rm -rf $mysql_datadir/*
102 rm -rf $topdir/ext/*
103 vlog "Original database removed"
104
105 # Restore database from backup
106- cp -rv $topdir/backup/test/* $mysql_datadir/test
107+ cp -rv $topdir/backup/* $mysql_datadir
108 vlog "database restored from backup"
109
110 }
111
112=== modified file 'test/t/bug766033.sh'
113--- test/t/bug766033.sh 2012-06-05 12:35:33 +0000
114+++ test/t/bug766033.sh 2013-03-07 19:24:26 +0000
115@@ -13,7 +13,7 @@
116 vlog "Starting backup"
117
118 # corrupt database
119-dd if=/dev/zero of=$mysql_datadir/sakila/rental.ibd seek=1000 count=1
120+printf '\xAA\xAA\xAA\xAA' | dd of=$mysql_datadir/sakila/rental.ibd seek=16384 count=4
121
122 # we want xtrabackup to be failed on rental.ibd
123 run_cmd_expect_failure $XB_BIN $XB_ARGS --backup --datadir=$mysql_datadir \
124
125=== modified file 'test/t/bug766607.sh'
126--- test/t/bug766607.sh 2012-06-05 12:35:33 +0000
127+++ test/t/bug766607.sh 2013-03-07 19:24:26 +0000
128@@ -3,12 +3,10 @@
129 start_server --innodb_file_per_table
130 load_dbase_schema incremental_sample
131
132-# Full backup dir
133-mkdir -p $topdir/data/full
134-# Incremental backup dir
135-mkdir -p $topdir/data/delta
136+# Backup dir
137+mkdir -p $topdir/backup
138
139-xtrabackup --datadir=$mysql_datadir --backup --target-dir=$topdir/data/full
140+innobackupex --no-timestamp $topdir/backup/full
141
142 vlog "Creating and filling a new table"
143 run_cmd $MYSQL $MYSQL_ARGS test <<EOF
144@@ -21,26 +19,25 @@
145 start_server --innodb_file_per_table
146
147 vlog "Making incremental backup"
148-xtrabackup --datadir=$mysql_datadir --backup --target-dir=$topdir/data/delta --incremental-basedir=$topdir/data/full
149+innobackupex --incremental --no-timestamp --incremental-basedir=$topdir/backup/full $topdir/backup/delta
150
151 vlog "Preparing full backup"
152-xtrabackup --datadir=$mysql_datadir --prepare --apply-log-only \
153- --target-dir=$topdir/data/full
154+innobackupex --apply-log --redo-only $topdir/backup/full
155
156 # The following would fail before the bugfix
157 vlog "Applying incremental delta"
158-xtrabackup --datadir=$mysql_datadir --prepare --apply-log-only \
159- --target-dir=$topdir/data/full --incremental-dir=$topdir/data/delta
160+innobackupex --apply-log --redo-only --incremental-dir=$topdir/backup/delta $topdir/backup/full
161
162 vlog "Preparing full backup"
163-xtrabackup --datadir=$mysql_datadir --prepare --target-dir=$topdir/data/full
164+innobackupex --apply-log $topdir/backup/full
165 vlog "Data prepared for restore"
166
167 stop_server
168+rm -r $mysql_datadir/*
169
170 vlog "Copying files"
171
172-cd $topdir/data/full/
173+cd $topdir/backup/full/
174 cp -r * $mysql_datadir
175 cd $topdir
176
177
178=== modified file 'test/t/ib_part_databases.sh'
179--- test/t/ib_part_databases.sh 2013-01-02 15:52:13 +0000
180+++ test/t/ib_part_databases.sh 2013-03-07 19:24:26 +0000
181@@ -22,6 +22,8 @@
182
183 # Take a backup
184 cat >$topdir/databases_file <<EOF
185+mysql
186+performance_schema
187 test.test
188 EOF
189 innobackupex --no-timestamp --databases=$topdir/databases_file $topdir/backup
190
191=== modified file 'test/t/ib_part_include.sh'
192--- test/t/ib_part_include.sh 2013-01-02 15:52:13 +0000
193+++ test/t/ib_part_include.sh 2013-03-07 19:24:26 +0000
194@@ -21,10 +21,7 @@
195
196 # Take a backup
197 # Only backup of test.test table will be taken
198-cat >$topdir/tables <<EOF
199-test.test
200-EOF
201-innobackupex --no-timestamp --include='test.test$' $topdir/backup
202+innobackupex --no-timestamp --include='^(mysql.*|performance_schema.*|test.test)$' $topdir/backup
203 innobackupex --apply-log $topdir/backup
204 vlog "Backup taken"
205
206
207=== modified file 'test/t/ib_part_include_stream.sh'
208--- test/t/ib_part_include_stream.sh 2013-01-02 15:52:13 +0000
209+++ test/t/ib_part_include_stream.sh 2013-03-07 19:24:26 +0000
210@@ -21,7 +21,7 @@
211
212 # Take a backup
213 mkdir -p $topdir/backup
214-innobackupex --stream=tar --include='test.test$' $topdir/backup > $topdir/backup/backup.tar
215+innobackupex --stream=tar --include='^(mysql.*|performance_schema.*|test.test)$' $topdir/backup > $topdir/backup/backup.tar
216 $TAR ixvf $topdir/backup/backup.tar -C $topdir/backup
217 $TAR cvhf $topdir/backup/backup11.tar $mysql_datadir/test/*
218
219
220=== modified file 'test/t/ib_part_tf_innodb.sh'
221--- test/t/ib_part_tf_innodb.sh 2013-01-02 15:52:13 +0000
222+++ test/t/ib_part_tf_innodb.sh 2013-03-07 19:24:26 +0000
223@@ -24,6 +24,7 @@
224 cat >$topdir/tables <<EOF
225 test.test
226 EOF
227+ib_part_add_mandatory_tables $mysql_datadir $topdir/tables
228 innobackupex --no-timestamp --tables-file=$topdir/tables $topdir/backup
229 innobackupex --apply-log $topdir/backup
230 vlog "Backup taken"
231
232=== modified file 'test/t/ib_part_tf_innodb_stream.sh'
233--- test/t/ib_part_tf_innodb_stream.sh 2013-01-02 15:52:13 +0000
234+++ test/t/ib_part_tf_innodb_stream.sh 2013-03-07 19:24:26 +0000
235@@ -24,6 +24,7 @@
236 cat >$topdir/tables <<EOF
237 test.test
238 EOF
239+ib_part_add_mandatory_tables $mysql_datadir $topdir/tables
240 mkdir -p $topdir/backup
241 innobackupex --stream=tar --no-timestamp --tables-file=$topdir/tables $topdir/backup > $topdir/backup/backup.tar
242 $TAR ixvf $topdir/backup/backup.tar -C $topdir/backup
243
244=== modified file 'test/t/ib_part_tf_myisam.sh'
245--- test/t/ib_part_tf_myisam.sh 2013-01-02 15:52:13 +0000
246+++ test/t/ib_part_tf_myisam.sh 2013-03-07 19:24:26 +0000
247@@ -25,6 +25,7 @@
248 cat >$topdir/tables <<EOF
249 test.test
250 EOF
251+ib_part_add_mandatory_tables $mysql_datadir $topdir/tables
252 innobackupex --no-timestamp --tables-file=$topdir/tables $topdir/backup
253 innobackupex --apply-log $topdir/backup
254 vlog "Backup taken"
255
256=== modified file 'test/t/xb_export.sh'
257--- test/t/xb_export.sh 2012-10-15 16:14:59 +0000
258+++ test/t/xb_export.sh 2013-03-07 19:24:26 +0000
259@@ -88,7 +88,7 @@
260 # consistent backup results. Otherwise we risk ending up with no test.ibd
261 # in the backup in case importing has not finished before taking backup
262
263-stop_server
264+shutdown_server
265 start_server $mysql_extra_args
266
267 # Some testing queries
268
269=== removed file 'test/t/xb_incremental.sh'
270--- test/t/xb_incremental.sh 2012-10-15 16:14:59 +0000
271+++ test/t/xb_incremental.sh 1970-01-01 00:00:00 +0000
272@@ -1,106 +0,0 @@
273-. inc/common.sh
274-
275-start_server --innodb_file_per_table
276-
277-load_dbase_schema incremental_sample
278-
279-# Adding 10k rows
280-
281-vlog "Adding initial rows to database..."
282-
283-numrow=100
284-count=0
285-while [ "$numrow" -gt "$count" ]
286-do
287- ${MYSQL} ${MYSQL_ARGS} -e "insert into test values ($count, $numrow);" incremental_sample
288- let "count=count+1"
289-done
290-
291-
292-vlog "Initial rows added"
293-
294-# Full backup
295-
296-# Full backup folder
297-mkdir -p $topdir/data/full
298-# Incremental data
299-mkdir -p $topdir/data/delta
300-
301-vlog "Starting backup"
302-
303-xtrabackup --datadir=$mysql_datadir --backup --target-dir=$topdir/data/full
304-
305-vlog "Full backup done"
306-
307-# Changing data in sakila
308-
309-vlog "Making changes to database"
310-
311-${MYSQL} ${MYSQL_ARGS} -e "create table t2 (a int(11) default null, number int(11) default null) engine=innodb" incremental_sample
312-let "count=numrow+1"
313-let "numrow=1000"
314-while [ "$numrow" -gt "$count" ]
315-do
316- ${MYSQL} ${MYSQL_ARGS} -e "insert into test values ($count, $numrow);" incremental_sample
317- ${MYSQL} ${MYSQL_ARGS} -e "insert into t2 values ($count, $numrow);" incremental_sample
318- let "count=count+1"
319-done
320-
321-vlog "Changes done"
322-
323-# Saving the checksum of original table
324-checksum_test_a=`checksum_table incremental_sample test`
325-checksum_t2_a=`checksum_table incremental_sample t2`
326-
327-vlog "Table 'test' checksum is $checksum_test_a"
328-vlog "Table 't2' checksum is $checksum_t2_a"
329-vlog "Making incremental backup"
330-
331-# Incremental backup
332-xtrabackup --datadir=$mysql_datadir --backup --target-dir=$topdir/data/delta --incremental-basedir=$topdir/data/full
333-
334-vlog "Incremental backup done"
335-vlog "Preparing backup"
336-
337-# Prepare backup
338-xtrabackup --datadir=$mysql_datadir --prepare --apply-log-only \
339- --target-dir=$topdir/data/full
340-vlog "Log applied to backup"
341-xtrabackup --datadir=$mysql_datadir --prepare --apply-log-only \
342- --target-dir=$topdir/data/full --incremental-dir=$topdir/data/delta
343-vlog "Delta applied to backup"
344-xtrabackup --datadir=$mysql_datadir --prepare --target-dir=$topdir/data/full
345-vlog "Data prepared for restore"
346-
347-# removing rows
348-run_cmd ${MYSQL} ${MYSQL_ARGS} -e "delete from test;" incremental_sample
349-run_cmd ${MYSQL} ${MYSQL_ARGS} -e "delete from t2;" incremental_sample
350-vlog "Tables cleared"
351-
352-# Restore backup
353-stop_server
354-vlog "Copying files"
355-cd $topdir/data/full/
356-cp -r * $mysql_datadir
357-cd $topdir
358-
359-vlog "Data restored"
360-start_server --innodb_file_per_table
361-
362-vlog "Checking checksums"
363-checksum_test_b=`checksum_table incremental_sample test`
364-checksum_t2_b=`checksum_table incremental_sample t2`
365-
366-if [ "$checksum_test_a" != "$checksum_test_b" ]
367-then
368- vlog "Checksums of table 'test' are not equal"
369- exit -1
370-fi
371-
372-if [ "$checksum_t2_a" != "$checksum_t2_b" ]
373-then
374- vlog "Checksums of table 't2' are not equal"
375- exit -1
376-fi
377-
378-vlog "Checksums are OK"
379
380=== removed file 'test/t/xb_partial.sh'
381--- test/t/xb_partial.sh 2012-10-15 16:14:59 +0000
382+++ test/t/xb_partial.sh 1970-01-01 00:00:00 +0000
383@@ -1,53 +0,0 @@
384-. inc/common.sh
385-
386-start_server --innodb_file_per_table
387-
388-load_dbase_schema incremental_sample
389-
390-# Adding 10k rows
391-vlog "Adding initial rows to database..."
392-numrow=1000
393-count=0
394-while [ "$numrow" -gt "$count" ]
395-do
396- ${MYSQL} ${MYSQL_ARGS} -e "insert into test values ($count, $numrow);" incremental_sample
397- let "count=count+1"
398-done
399-vlog "Initial rows added"
400-checksum_a=`checksum_table incremental_sample test`
401-vlog "Table checksum is $checksum_a"
402-
403-# Backup directory
404-mkdir -p $topdir/data/parted
405-
406-vlog "Starting backup"
407-xtrabackup --datadir=$mysql_datadir --backup --target-dir=$topdir/data/parted \
408- --tables="^incremental_sample[.]test"
409-vlog "Partial backup done"
410-
411-# Prepare backup
412-xtrabackup --datadir=$mysql_datadir --prepare --target-dir=$topdir/data/parted
413-vlog "Data prepared for restore"
414-
415-# removing rows
416-run_cmd ${MYSQL} ${MYSQL_ARGS} -e "delete from test;" incremental_sample
417-vlog "Table cleared"
418-
419-# Restore backup
420-stop_server
421-vlog "Copying files"
422-cd $topdir/data/parted/
423-cp -r * $mysql_datadir
424-cd $topdir
425-vlog "Data restored"
426-start_server --innodb_file_per_table
427-vlog "Checking checksums"
428-checksum_b=`checksum_table incremental_sample test`
429-
430-if [ "$checksum_a" != "$checksum_b" ]
431-then
432- vlog "Checksums are not equal"
433- exit -1
434-fi
435-
436-vlog "Checksums are OK"
437
438=== modified file 'test/testrun.sh'
439--- test/testrun.sh 2013-01-08 06:23:22 +0000
440+++ test/testrun.sh 2013-03-07 19:24:26 +0000
441@@ -171,8 +171,7 @@
442 else
443 XB_BIN="xtrabackup" # InnoDB 5.1 plugin or Percona Server 5.1
444 fi
445- elif [ "${MYSQL_VERSION:0:3}" = "5.2" -o
446- "${MYSQL_VERSION:0:3}" = "5.3"]
447+ elif [ "${MYSQL_VERSION:0:3}" = "5.2" -o "${MYSQL_VERSION:0:3}" = "5.3" ]
448 then
449 XB_BIN="xtrabackup"
450 elif [ "${MYSQL_VERSION:0:3}" = "5.5" ]

Subscribers

People subscribed via source and target branches